79240708

Date: 2024-12-01 00:33:53
Score: 0.5
Natty:
Report link

This is for: How to copy lot of files from one s3 bucket in account A, to another in account B. I know this could be different use case, but for anyone who wants some help on this, here It is.

For small amount of files to copy yeah It could work using aws cli, aws s3 sync, but for buckets with more than 3TB of size and 12 Millions of files It just does not work well. you will deal with other issues like token expiration after 1hour, 12h after configuring the expiration time as the max value allowed.

So, for you use case, I saw by using aws DataSync you can schedule when to run the sync task, I do not think It can be right after adding a new file but It can be after, just configure the task schedule

Schedule options

Here is the documentation, just follow the steps: https://docs.aws.amazon.com/datasync/latest/userguide/tutorial_s3-s3-cross-account-transfer.html

Reasons:
  • Long answer (-0.5):
  • No code block (0.5):
  • Low reputation (0.5):
Posted by: Oswaldo Zapata