I contacted AWS support and they mentioned there's indeed a warm up period. This is what the response said:

Although SQS FIFO queue high throughput supports up to 70,000 TPS with batching, internally SQS must scale up its backend hosts for the queue to handle the increased load. The scaling process typically takes low hundreds of milliseconds to complete, during which temporary throttling may occur.

It seems the scaling up is not instantaneous, and there could be throttling even with high throughput, and even if you are not reaching 70,000 TPS limit (depends on region to region).

The only option here is to build a retry mechanism, you can also try reaching out to AWS support so they can keep the queue scaled up.