Exclude multiple Folders with AWS S3 Sync

avatar

Borislav Hadzhiev

Last updated: Sep 27, 2021

banner

Photo from Unsplash

How to Exclude multiple Folders with AWS S3 Sync #

In order to exclude multiple folders when using S3 sync, pass multiple --exclude parameters to the command, specifying the path to the folders you want to exclude.

shell
aws s3 sync s3://YOUR_BUCKET . --exclude "my-folder-1/*" --exclude "my-folder-2/*" --exclude "my-folder-3/*"

multiple exclude parameters

In the example above we've used the exclude parameter to filter out 3 folders from the sync command.

Because we've passed the root of the bucket after the sync keyword in the command (e.g. s3://my-bucket ), we have to specify the whole path for the values of the --exclude parameter.

The values of the --exclude parameter must be in double quotes when issuing the command on Windows.

Let's look at an example where we have the following folder structure in the S3 bucket:

shell
my-folder-1/ file1.txt file2.txt nested-folder-1/ nested-folder-2/

We want to exclude nested-folder-1 and nested-folder-2 from the sync command and both of them are in the my-folder-1 directory. Therefore we can add the suffix to the bucket name, instead of repeating it in the value of all --exclude parameters.

shell
aws s3 sync s3://YOUR_BUCKET/my-folder-1 . --exclude "nested-folder-1/*" --exclude "nested-folder-2/*"

multiple exclude with bucket prefix

In the example above we've specified the my-folder-1 suffix to the bucket name, which means that all of our --exclude parameters start from that path.

We can also use the --exclude parameter to filter out for specific files, including use wildcards.

The following example excludes all files with the .png and .pdf extensions that are in the my-folder-1 directory.

shell
aws s3 sync s3://YOUR_BUCKET . --exclude "my-folder-1/*.png" --exclude "my-folder-1/*.pdf"

multiple exclude with wildcards

In the example above we've excluded all of the .png and .pdf files in the my-folder-1 directory, however files with other extensions in the folder have not been excluded, nor .png or .pdf files in other directories in the bucket.

Further Reading #

I wrote a book in which I share everything I know about how to become a better, more efficient programmer.
book cover
You can use the search field on my Home Page to filter through all of my articles.