Exclude multiple Folders with AWS S3 Sync

avatar

Borislav Hadzhiev

Mon Sep 27 20212 min read

banner

Photo by Joshua Earle

How to Exclude multiple Folders with AWS S3 Sync #

In order to exclude multiple folders when using S3 sync, pass multiple --exclude parameters to the command, specifying the path to the folders you want to exclude.

shell
aws s3 sync s3://YOUR_BUCKET . --exclude "my-folder-1/*" --exclude "my-folder-2/*" --exclude "my-folder-3/*"

multiple exclude parameters

In the example above we've used the exclude parameter to filter out 3 folders from the sync command.

Because we've passed the root of the bucket after the sync keyword in the command (e.g. s3://my-bucket ), we have to specify the whole path for the values of the --exclude parameter.

The values of the --exclude parameter must be in double quotes when issuing the command on Windows.

Let's look at an example where we have the following folder structure in the S3 bucket:

shell
my-folder-1/ file1.txt file2.txt nested-folder-1/ nested-folder-2/

We want to exclude nested-folder-1 and nested-folder-2 from the sync command and both of them are in the my-folder-1 directory. Therefore we can add the suffix to the bucket name, instead of repeating it in the value of all --exclude parameters.

shell
aws s3 sync s3://YOUR_BUCKET/my-folder-1 . --exclude "nested-folder-1/*" --exclude "nested-folder-2/*"

multiple exclude with bucket prefix

In the example above we've specified the my-folder-1 suffix to the bucket name, which means that all of our --exclude parameters start from that path.

We can also use the --exclude parameter to filter out for specific files, including use wildcards.

The following example excludes all files with the .png and .pdf extensions that are in the my-folder-1 directory.

shell
aws s3 sync s3://YOUR_BUCKET . --exclude "my-folder-1/*.png" --exclude "my-folder-1/*.pdf"

multiple exclude with wildcards

In the example above we've excluded all of the .png and .pdf files in the my-folder-1 directory, however files with other extensions in the folder have not been excluded, nor .png or .pdf files in other directories in the bucket.

Further Reading #

Join my newsletter

I'll send you 1 email a week with links to all of the articles I've written that week

Buy Me A Coffee