Ok this was in fact pretty simple but I miss it: in the subnet selection I have set some private subnet, so when the task lauch it select (randomly) one of the available subnet, and some time it take one of the private subnate that dosent have the right to reach the secret manager.

Removing these wrong subnet was the solution.