How to Delete a Folder from an S3 Bucket

avatar

Borislav Hadzhiev

Tue Sep 21 20213 min read

banner

Photo by Luke Pamer

Table of Contents #

  1. Delete an entire Folder from an S3 Bucket
  2. Filter which Files to Delete from an S3 Bucket

Delete an entire Folder from an S3 Bucket #

To delete an entire folder in an AWS S3 bucket, use the s3 rm command, passing in the path of the objects to be deleted along with the --recursive parameter, which applies the action to all files under the specified path.

Let's first run the s3 rm command in test mode to make sure the output matches the expectations:

shell
aws s3 rm s3://YOUR_BUCKET/ --recursive --dryrun --exclude "*" --include "my-folder/*"

delete files in folder test mode

The example shows that all of the files in the specified folder would get deleted. The folder also gets deleted, because S3 doesn't keep empty folders around.

Note that the order of the --exclude and --include parameters matters. Filters passed later in the command have higher precedence.

We've passed the following parameters to the s3 rm command:

NameDescription
recursiveapplies the s3 rm command to all the nested objects under the specified path
dryrunshows the command's output without actually running it
excludewe only want to delete the contents of a specific folder, so we exclude all other paths in the bucket
includewe include the path that matches all of the files we want to delete

Now that we've made sure the output from the s3 rm command is what we expected, let's run it without the --dryrun parameter:

shell
aws s3 rm s3://YOUR_BUCKET/ --recursive --exclude "*" --include "my-folder/*"

delete files in folder

To verify all files in the folder have been successfully deleted, we can run the s3 ls command. If the command receives a path that does not exist, is has no return value.

shell
aws s3 ls s3://YOUR__BUCKET/YOUR_FOLDER --recursive

all files in folder deleted

It's always a best practice to run destructive commands like s3 rm with the --dryrun parameter first. Make sure that the command does what you intend, without actually running it.

Filter which Files to Delete from an S3 Bucket #

The --include and --exclude parameters can be passed to the s3 rm command for more fine grained filtering.

The following command, deletes all objects in the folder, except for objects with the .png extension:

shell
aws s3 rm s3://YOUR_BUCKET/ --recursive --exclude "*" --include "my-folder/*" --exclude "*.png"

delete files in folder with extension filter

The output of the s3 ls command shows that the image at path my-folder-2/hook-flow.png has not been deleted.

If you wanted to preserve all .png files and all .txt files you would just add another --exclude "*.txt" flag at the end of the command.

The order of the --exclude and --include parameters is very important. For instance, if we reverse the order and pass--include "my-folder-2/*" before the exclude "*" parameter, that would delete all of the files in the S3 bucket, because exclude comes afterinclude and overrides it.

Finally, let's look at an example where we have the following folder structure:

shell
bucket my-folder-3/ image.webp file.json nested-folder/ file.txt file.json

We have a nested folder that we want to preserve, but we want to delete all of the files in the my-folder-3 directory.

shell
aws s3 rm s3://YOUR_BUCKET/ --recursive --exclude "*" --include "my-folder-3/*" --exclude "my-folder-3/nested-folder/*"

exclude nested folder from deletion

To verify the nested folder did not get deleted, we can run the s3 ls command:

shell
aws s3 ls s3://YOUR_BUCKET/my-folder-3/nested-folder --recursive --human-readable

verify nested folder not deleted

The output shows that the nested folder was excluded successfully and has not been deleted.

Further Reading #

Join my newsletter

I'll send you 1 email a week with links to all of the articles I've written that week

Buy Me A Coffee