If you're just experimenting, you can simply use lifecycle rules to periodically clean up (for example, retaining files for 90 days).

For a production environment, we recommend:

Write a script to identify "active" S3 objects.

Only clean up historical ZIP files and templates that are no longer referenced.

Alternatively, use a custom S3 bucket and overwrite the same file name each time you build to avoid overruns.