List all Files in an S3 Bucket with AWS CLI

avatar

Borislav Hadzhiev

Tue Sep 21 20212 min read

Table of Contents #

  1. List all Files in an S3 Bucket with AWS CLI
  2. List all Files in a Folder of an S3 Bucket
  3. List only the Filenames of an S3 Bucket

List all Files in an S3 Bucket with AWS CLI #

To list all of the files of an S3 bucket with the AWS CLI, use the s3 ls command, passing in the --recursive parameter.

shell
aws s3 ls s3://YOUR_BUCKET --recursive --human-readable --summarize

list all files in bucket

The output of the command shows the date the objects were created, their file size and their path.

List requests are associated with a cost. At the time of writing the cost of 1,000 list requests is $0.005 in the us-east-1 region.

The parameters we've passed to the s3 ls command are:

NameDescription
recursiveperforms the command on all files under the set prefix
human-readabledisplays the file sizes in human readable format
summarizedisplays the total number of objects and their total size
If you only want to list all top-level objects remove the --recursive parameter from the command.

List all Files in a Folder of an S3 Bucket #

To list all files, located in a folder of an S3 bucket, use the s3 ls command, passing in the entire path to the folder and setting the --recursive parameter.

shell
aws s3 ls s3://YOUR_BUCKET/YOUR_FOLDER/ --recursive --human-readable --summarize

list all files in bucket folder

The output of the command only shows the files in the /my-folder-1 directory.

List only the Filenames of an S3 Bucket #

To only list the filenames of an S3 bucket, we have to:

  1. Use the s3api list-objects command
  2. Set the --output parameter to text
  3. Use the --query parameter to filter the output from the command
shell
aws s3api list-objects --bucket YOUR_BUCKET --output text --query "Contents[].{Key: Key}"

only list filenames of bucket

The output of the command shows all of the filenames, or in other words - path names in the S3 bucket.

To list only the filenames in a specific folder, add the --prefix parameter to the command:

shell
aws s3api list-objects --bucket YOUR_BUCKET --prefix "my-folder/" --output text --query "Contents[].{Key: Key}"

only list filenames in folder

We can also redirect the output to a file on the local file system:

shell
aws s3api list-objects --bucket YOUR_BUCKET --prefix "my-folder-1/" --output text --query "Contents[].{Key: Key}" > file-names.txt

list filenames redirect output

Further Reading #

Join my newsletter

I'll send you 1 email a week with links to all of the articles I've written that week

Buy Me A Coffee