79284647

Date: 2024-12-16 12:24:43
Score: 0.5
Natty:
Report link

We hit a "feature" where cloud run would, for GRPC services, declare new revisions as healthy once a single instance of a revision is healthy. This bug surfaces when you have really slow startup times (15-20s).

Google didn't accept this as a bug, but the evidence provided by them is that they would enqueue requests in the nodes waiting for the new instances of the revisions to start instead of routing them to old instances, causing brief downtimes between deploys.

Another bug that you may encounter is that if you have some processing to be done before your instance gets healthy, if you enable cpu-boost, Cloud Run will understand that everything has gone south and limit traffic to 1 request per instance, causing your services to overscale for no reason.

Relevant bug https://issuetracker.google.com/issues/377764060?pli=1

Reasons:
  • Long answer (-0.5):
  • No code block (0.5):
  • Low reputation (0.5):
Posted by: txomon