I am a bit late (nearly a year), i had issues with RDS proxy for postgres and I was able to successfully connect to the DB using the proxy by modifying some settings in the proxy:

• Client authentication type: I set it to be PostgreSQL MD5 instead of SCRAMSHA
• Enable require TLS

Also make sure the security group rules allow the traffic to pass between the proxy and the db