Exclude multiple Folders with AWS S3 Sync

How to Exclude multiple Folders with AWS S3 Sync #

In order to exclude multiple folders when using S3 sync, pass multiple --exclude parameters to the command, specifying the path to the folders you want to exclude.

shell

Copied!
aws s3 sync s3://YOUR_BUCKET . --exclude "my-folder-1/*" --exclude "my-folder-2/*" --exclude "my-folder-3/*"

multiple exclude parameters

In the example above we've used the exclude parameter to filter out 3 folders from the sync command.

Because we've passed the root of the bucket after the sync keyword in the command (e.g. s3://my-bucket ), we have to specify the whole path for the values of the --exclude parameter.

The values of the --exclude parameter must be in double quotes when issuing the command on Windows.

Let's look at an example where we have the following folder structure in the S3 bucket:

shell

Copied!
my-folder-1/
    file1.txt
    file2.txt
    nested-folder-1/
    nested-folder-2/

We want to exclude nested-folder-1 and nested-folder-2 from the sync command and both of them are in the my-folder-1 directory. Therefore we can add the suffix to the bucket name, instead of repeating it in the value of all --exclude parameters.

shell

Copied!
aws s3 sync s3://YOUR_BUCKET/my-folder-1 . --exclude "nested-folder-1/*" --exclude "nested-folder-2/*"

multiple exclude with bucket prefix

In the example above we've specified the my-folder-1 suffix to the bucket name, which means that all of our --exclude parameters start from that path.

We can also use the --exclude parameter to filter out for specific files, including use wildcards.

The following example excludes all files with the .png and .pdf extensions that are in the my-folder-1 directory.

shell

Copied!
aws s3 sync s3://YOUR_BUCKET . --exclude "my-folder-1/*.png" --exclude "my-folder-1/*.pdf"

multiple exclude with wildcards

In the example above we've excluded all of the .png and .pdf files in the my-folder-1 directory, however files with other extensions in the folder have not been excluded, nor .png or .pdf files in other directories in the bucket.

Exclude multiple Folders with AWS S3 Sync

How to Exclude multiple Folders with AWS S3 Sync #

Further Reading #

Borislav Hadzhiev