I have the same problem, but my case is worst.

The org has more than thousands repositories and include all repos into the credential is not an option.
I would love to know if there is other way to solve this.