According to the documentation, the IP is determined from headers passed into the requests : https://docs.konghq.com/hub/kong-inc/rate-limiting/#limit-by-ip-address
Is there any proxy, firewall or load balancer in front of your API gateway that could possibly always set the same IP into those headers, which would lead the plugin to think that all the requests come from the same IP address ?