79124382

Date: 2024-10-25 05:43:52
Score: 0.5
Natty:
Report link

May or may not fit your usecase but it did in my case: Generate a simple CSV file of all the keys you want to delete. The format of the csv file columns is

bucket-name|key

Create a batch job with operation type 'Replace all object tags'. Place some object tag such as delete=true on these objects. Monitor your batch job using AWS. Should not be too complicated hopefully. After that, create a LifeCycle policy to Expire Current/Non-Current versions (Again based on your usecase), of objects from your S3 Bucket and let it sit for 1-2 days. And poof, all hands off, all serverless, all objects deleted! My organization used this technique to delete 36 Million and 30 Million objects from 2 different buckets. It's not an immediate solution, but at this scale, we wanted a cheap, error-free and hands off system instead of something manual.

Reasons:
  • Long answer (-0.5):
  • No code block (0.5):
  • Low reputation (0.5):
Posted by: Sankalp Mukim