I don't have enough reputation to write comment, so I apologize for asking questions in an "answer".

What git hosting services are you using behind those remoteHost1 and remoteHost2? Are they the same or they are different? For example GitHub, BitBucket, Gogs, etc? If they are the same, are they both configured the same? Or if they are not the same, does the other one support large files? Does it have enough space on disk? Are they in the same or in different networks? Maybe some firewall in front of remoteHost2 is blocking large requests? Does git push --verbose show any meaningful message?

Regarding that last part of your question, it can't be done. If it's one repo locally, you can't add one file for one remote and ignore it for another.