friend.
Apparently, even when using RDS in the free tier, if you're using a non-private VPC, you'll be charged for the use of the IPv4, since the public IPv4 addresses are not free when used with RDS, only with EC2 instances.
I was facing the same problem, found this answer here: [https://repost.aws/questions/QUGjraaPYUQLmVOvUph0NWgQ/charged-on-free-tier-for-vpc][1]
Hope this helps you.