Edit the "config" file located in the .git dir of your repo and find the line that begins with "url =" under "[remote "origin"]".

Change the line to this, "url = ssh://[email protected]:port#/remote/dir"

example: url = ssh://[email protected]:2222/srv/dir/files