What ultimately helped me, (after unsuccessfully trying other answers), was to input C:\Program Files\Git\usr\bin\ssh-pageant.exe (not ssh.exe) in the TortoiseGit SSH client.

You can find the setting at TortoiseGit -> Settings -> Network -> SSH client