This article mentions: There seems to be a roughly 2.9gb limit on the size of an upcoming pack, and this must be the problem, which I've never encountered before.
Referring to phd's answer, I found I can do push separately. I was pushing a fork of llvm/llvm-project, which consists of about 500K commits. I tried to push via:

git push origin <local-branch-name>~400000:refs/heads/<remote-branch-name>
git push origin <local-branch-name>~200000:<remote-branch-name>
git push origin <local-branch-name>:<remote-branch-name>

This resolved the issue.