Configure S3 Lifecycle Rules in AWS CDK

avatar

Borislav Hadzhiev

Mon May 10 20214 min read

Configuring S3 Lifecycle Rules in AWS CDK #

S3 lifecycle rules enable us to transition the objects we store to different categories, in an attempt to lower cost.

In order to add lifecycle rules to an S3 bucket in AWS CDK, we have to provide the lifecycleRules prop when instantiating the Bucket class.

The code for this article is available on GitHub

Let's look at an example where we add a lifecycle rule with multiple transitions to an S3 bucket:

lib/cdk-starter-stack.ts
import * as s3 from '@aws-cdk/aws-s3'; import * as cdk from '@aws-cdk/core'; export class CdkStarterStack extends cdk.Stack { constructor(scope: cdk.App, id: string, props?: cdk.StackProps) { super(scope, id, props); const s3Bucket = new s3.Bucket(this, 's3-bucket', { removalPolicy: cdk.RemovalPolicy.DESTROY, autoDeleteObjects: true, // ๐Ÿ‘‡ set up lifecycle rules lifecycleRules: [ { // ๐Ÿ‘‡ optionally apply object name filtering // prefix: 'data/', abortIncompleteMultipartUploadAfter: cdk.Duration.days(90), expiration: cdk.Duration.days(365), transitions: [ { storageClass: s3.StorageClass.INFREQUENT_ACCESS, transitionAfter: cdk.Duration.days(30), }, { storageClass: s3.StorageClass.INTELLIGENT_TIERING, transitionAfter: cdk.Duration.days(60), }, { storageClass: s3.StorageClass.GLACIER, transitionAfter: cdk.Duration.days(90), }, { storageClass: s3.StorageClass.DEEP_ARCHIVE, transitionAfter: cdk.Duration.days(180), }, ], }, ], }); } }

Let's go over what we did in the code snippet.

  1. we created an S3 bucket, to which we provided some clean up props like removalPolicy and autoDeleteObjects, which will take care of emptying and deleting the bucket when we delete the CDK stack

  2. we set the lifecycleRules prop when instantiating the Bucket class. Some of the configuration properties a lifecycle rule consists of, are:

    • prefix - apply the rule only to objects that match the prefix, i.e. only objects with a prefix of logs/.
    • abortIncompleteMultipartUploadAfter - delete incomplete multipart upload after a number of days, in our case - after 90 days
    • expiration - delete files from S3 and Glacier after a specified number of days, in our case - after 365 days
    • transitions - An array of transition rules that specify after how many days the object should be transitioned to a different storage tier

The transition rules we've set end up looking like this:

DayAction
Day 0Objects were uploaded to the bucket
Day 30Objects transition to standard Infrequent Access
Day 60Objects transition to Intelligent Tiering
Day 90Objects transition to Glacier
Day 180Objects transition to Glacier Deep Archive
Day 365Objects expire and get deleted from S3 and Glacier

There are a couple of things we should keep in mind when configuring S3 lifecycle rules in AWS CDK:

The transition to INTELLIGENT_TIERING must come at least 30 days after the transition to INFREQUENT_ACCESS, if both are set.

The transition to GLACIER must come at least 30 days after the transition to INTELLIGENT_TIERING.

The transition to DEEP_ARCHIVE must come at least 90 days after the transition to GLACIER. That's because, if we store data in Glacier for less than 90 days, we end up paying for the whole 90 days.

Let's execute a deployment:

shell
npx cdk deploy

If we look at the S3 management console, we can see that the lifecycle rule has been created. It applies to the entire bucket, because we didn't specify an object prefix:

s3 bucket lifecycle rule

Now that the lifecycle rule has been applied to the S3 bucket, all objects we upload would follow the transitions from the table.

Adding an S3 Lifecycle rule after Bucket creation in AWS CDK #

To add a lifecycle rule to an S3 bucket after it has been created, we can use the addLifecycleRule method on the bucket instance.

The code for this article is available on GitHub

Let's add a lifecycle rule to our bucket, that only applies to objects with a prefix of logs/:

lib/cdk-starter-stack.ts
import * as s3 from '@aws-cdk/aws-s3'; import * as cdk from '@aws-cdk/core'; export class CdkStarterStack extends cdk.Stack { constructor(scope: cdk.App, id: string, props?: cdk.StackProps) { super(scope, id, props); // ... rest // ๐Ÿ‘‡ add a life cycle rule after bucket creation s3Bucket.addLifecycleRule({ prefix: 'logs/', expiration: cdk.Duration.days(90), transitions: [ { storageClass: s3.StorageClass.ONE_ZONE_INFREQUENT_ACCESS, transitionAfter: cdk.Duration.days(60), }, ], }); } }

In the code snippet, we've specified a prefix of logs/. This means that the lifecycle rule we've provided will only be applied if the object has that prefix.

Let's execute a deployment:

shell
npx cdk deploy

After a successful deployment, we can see that a second lifecycle rule has been created, which only applies to objects with a prefix of logs/:

s3 lifecycle rule prefix

The transition rules for our S3 Bucket, for objects with a prefix of logs/, end up looking like this:

DayAction
Day 0Objects uploaded
Day 60Objects transition to One Zone Infrequent Access
Day 90Objects expire and get deleted from s3 and glacier

Clean up #

To delete the resources we have provisioned, execute the destroy command:

shell
npx cdk destroy

Further Reading #

Add me on LinkedIn

I'm a Web Developer with TypeScript, React.js, Node.js and AWS experience.

Let's connect on LinkedIn

Join my newsletter

I'll send you 1 email a week with links to all of the articles I've written that week

Buy Me A Coffee