Step 1: Added a config file in ~/.ssh/ folder which looks like

   User git
   Hostname gitlab.com
   IdentityFile ~/.ssh/id_rsa_gitlab
   TCPKeepAlive yes
   IdentitiesOnly yes

Step 2: Just clone the git repo WITHOUT sudo.