Delete Multiple Files from S3 Bucket with AWS CLI

avatar

Borislav Hadzhiev

Tue Sep 21 20212 min read

banner

Photo by Erik Mclean

Deleting Multiple Files from S3 Bucket with the AWS CLI #

To delete multiple files from an S3 Bucket with the AWS CLI, run the s3 rm command, passing in the exclude and include parameters to filter the files the command is applied to.

Let's run the command in test mode first. By setting the --dryrun parameter, we instruct the AWS CLI to only print the outputs of the s3 rm command, without actually running it.

shell
aws s3 rm s3://YOUR_BUCKET/YOUR_FOLDER/ --dryrun --recursive --exclude "*" --include "file1.txt" --include "file2.txt"

delete multiple files test mode

The output shows the names of the files that would get deleted, had we run the command without the --dryrun parameter.

We've passed the following parameters to the s3 rm command:

NameDescription
recursiveapplies the s3 rm command to all the nested objects under the specified path
dryrunshows the command's output without actually running it
excludea filter specifying which paths should be excluded from the command
includea filter specifying which paths should be included in the command
Note that the order of the --exclude and --include parameters matters. Filters passed later in the command have higher precedence and override those that come before them.

This means that passing the --exclude "*" parameter after --include "file1.txt" would delete all files in the S3 bucket.

It's always a best practice to run destructive commands like s3 rm with the --dryrun parameter first. Make sure that the command does what you intend, without actually running it.

In the example above, both of the files are located in the same folder, otherwise we would include the path to the files in the --include parameters:

shell
aws s3 rm s3://YOUR_BUCKET/ --dryrun --recursive --exclude "*" --include "folder1/file1.txt" --include "folder2/file2.txt"

delete files in different folders

To exit test mode and perform the s3 rm operation, simply remove the --dryrun parameter.

For the commands that follow, I'll remove the --dryrun parameter.

The include parameter can also be set as a matcher, for example to delete all files with a .png extension, under a specific prefix, use the following command.

shell
aws s3 rm s3://YOUR_BUCKET/ --recursive --exclude "*" --include "folder2/*.png"

delete files that match pattern

The output shows, that the --include parameter matches files with the .pngextension in nested directories.

The command deleted my-folder/image.png as well as my-folder/nested-folder/another-image.png.

Running the s3 rm command with an --include parameter that does not match any files, produces no output.

no output when no match

Further Reading #

Join my newsletter

I'll send you 1 email a week with links to all of the articles I've written that week

Buy Me A Coffee