One of our production magento 2.4.5 website was getting frequent down issues and we have checked the case in detail and we could see that the we are getting excessive crawler request from meta-externalagent. See sample log entry below.

57.141.0.9 - - [xx/Dec/2024:12:xx:09 +0530] "GET /our-xxxxxxxxxs/ms-xxx?karat=23&size=6%2C7%2C13%2C16%2C17%2C18%2C24 HTTP/1.1" 200 780515 "-" "meta-externalagent/1.1 (+https://developers.facebook.com/docs/sharing/webmasters/crawler

While checking further further i could see from logs that we have received "64941" requests from "meta-externalagent/1.1" in 12 hours.

I can see lots of people are facing similar issues, But no clear solution is mentioned for magento version 2.4.5.

Is there any possible option we can do some rate limiting for the meta crawler ? As we are doing Facebook ads we cannot completely block requests from meta-externalagent.

Currently i i have blocked meta-externalagent using 7G Firewall in nginx.