This is related to the visibility_timeout
configuration of SQS queues.
as per documentation if a task isn’t acknowledged within the visibility_timeout, the task will be redelivered to another worker and executed.
This causes problems with retry tasks where the time to execute exceeds the visibility timeout; if that happens it will be executed again, and again in a loop.
So we have to increase the visibility timeout to match the time of the longest ETA(retry exhaustion) we’re planning to use.