In this evolving field of the IT industry, the world is now migrating from direct application deployment to a containerized application deployment approach. We might have all heard of the term “Docker,” where we package our entire application along with all its dependencies, enabling it to run smoothly anywhere, regardless of the underlying platform or operating system.

While working with Docker, we often end up creating multiple iterations of our application images during the development phase. These images are stored in the Container registry and might become outdated after a certain period of time. In such situations, it is crucial to manage these images efficiently to reduce our AWS costs.

In this blog, we will explore how to efficiently manage our container registry using lifecycle policies to avoid any unnecessary charges in our AWS billing.

What is AWS ECR?

Amazon ECR is an AWS managed Container image registry that is secure, scalable and reliable. It is kinda similar to docker hub if you have worked with it. We can create public and private repositories and using our preferred CLI can push, pull and manage our application images over there.

What is lifecycle policy?

Amazon ECR provides a lifecycle policy to offer us more control over the lifecycle of images existing in our ECR repositories. A lifecycle policy consists of one or more sets of rules where each rule defines the action that needs to be taken on our ECR repository. With the help of this lifecycle policy, we can automate the cleanup of expired application images in our ECR repository based on age or count.

How lifecycle policies work?

A lifecycle policy consists of one or more rules based on which images in our repository should be expired. Before applying lifecycle rules to our repository, we should use the lifecycle policy preview in order to see the images which will be affected by our lifecycle policy. Once we apply the lifecycle policy, our images will be expired within 24 hours after they meet our expiration criteria. Please note that the lifecycle policy is not applied to the ECR repository if you have enabled replication into a different region or account.

The following diagram shows the lifecycle policy workflow.

1. Create one or more test rules.
2. Save the test rules and run the preview.
3. The lifecycle policy evaluator goes through all of the rules and marks the images that each rule affects.
4. The lifecycle policy evaluator then applies the rules, based on rule priority, and displays which images in the repository are set to be expired.
5. Review the results of the test, ensuring that the images that are marked to be expired are what you intended.
6. Apply the test rules as the lifecycle policy for the repository.
7. Once the lifecycle policy is created, you should expect that images become expired within 24 hours after they meet the expiration criteria.

For more detailed information, check out the official AWS documentation. Enough theory; let’s dive into the actual implementation.

Prerequisites:

• Active AWS account
• IAM user with appropriate permissions on ECR
• ECR repository with some Docker images present

Implementation

I’ve created an ECR private repository named “node” with Docker image versions ranging from 11 to 17.

Open your desired repository and you will see lifecycle policy option in the left drop down menu

Here, you can view all the available lifecycle policies for this repository, and the lifecycle policy evaluator will be responsible for evaluating all the rules and applying them to the ECR images based on priority.

You can click on “Edit test rule” to create a lifecycle policy and run a test to check the impact of our policy before applying it to our repository.

As we don’t have any existing lifecycle policy rules, we can create a new one by clicking on “Actions” > “Edit JSON” to write the policy in JSON syntax.

Here is an example of a JSON policy:

{
    "rules": [
        {
            "rulePriority": integer,
            "description": "string",
            "selection": {
                "tagStatus": "tagged"|"untagged"|"any",
                "tagPatternList": list<string>,
                "tagPrefixList": list<string>,
                "countType": "imageCountMoreThan"|"sinceImagePushed",
                "countUnit": "string",
                "countNumber": integer
            },
            "action": {
                "type": "expire"
            }
        }
    ]
}

Or alternatively we can also create new policy by clicking on Create rule option.

You can view the history of your lifecycle rule in the Lifecycle event history tab shown in the console.

In the create rule page we need to mention the parameters for our lifecycle policy based on which our ECR images will gets expired.

Let’s understand all these terms before proceeding with the policy creation task:

1. Priority

This field sets the order in which rules are applied to the ECR repository, and it is required. It is an integer value, and priority is assigned from the lowest to the highest number order. For each rule in the lifecycle policy, a unique number must be assigned to set its priority. When creating the policy, it’s crucial to set the priority carefully; otherwise, essential images might expire if the rules are not tested properly. A rule with a tag status of “any” must be assigned a high priority value so that it can be evaluated last, as it impacts all images in the repository irrespective of their tags.

2. Description

This field is used to assign a description to our lifecycle policy rule, and it is optional. It’s good practice to provide a proper descriptive purpose for future references.

3. Image Status

This is an essential part of a lifecycle policy where we specify the actual logic based on which the policy gets evaluated, and images get expired.

Here is the explaination for each of these image status

• Tagged (wildcard matching)

Here we specify a comma-separated list of image tag patterns that may contain wildcards(*) on which to take action with your lifecycle policy.

For example if our images are tagged as prod, prod1, prod2 and so on then you can use tag pettern as prod* to specify all the prod images.

Note: If you specify multiple tags then image satisying all the expressions are selected.

For example, if we specify tag pattern list prod*, prod*web then images with prod1web, prod2web will be selected and the images with prod1, prod2 and so on will not be selected.

• Tagged (prefix matching)

Here we need to specify the comma separated list of image tag prefixes on which action will be taken by lifecycle policy.

For example, if we have images tagged with prod, prod1, prod2, and so on then specify the tag prefix prod to target all these images.

• Untagged

This is used when we have untagged images in our ECR and we want to apply lifecycle policy rule on them. We don’t have to specify any matching rule for this and this rule will not have any impact on tagged images

• Any

This image status is specified when we want to target all the images residing in our repository irrespective of whether they are tagged or not.

This rule must be assigned higher priority number so that it can be evaluated at the end by the lifecycle policy rule evaluator.

For example, Consider you have 100s of untagged, tagged or improperly tagged images then in such case selecting the any image status will impact all your 100s of images.

4. Match Criteria

This match criteria consists of the options/conditions based on which your selected images gets evaluated for the expiration action.

• Since Image Pushed

The sinceimagepushed match criteria in an ECR lifecycle policy specifies how long an image should be kept after it’s pushed to the repository.

Here we need to specify the count number and count unit. Count unit is by default set to days.

For example, if we select sinceimagepushed match criteria and set count number as 30, then it will identify all the images pushed longer than 30 days.

• Image Count More Than

The imagecountmorethan match criteria is used in ECR lifecycle policy when we want to set the limit on the total number of images that we want to retain in our ECR repository.

We can specify the images count number and it should be positive integer and must be greater than 0.

For example, if we select imagecountmorethan match criteria and set count number as 5, then it will identify all the images existing in the repository more than 5

5. Rule Action

This specifies the action that needs to be taken on the images that match the specified criteria in our rule.

There are 2 available actions:

• Expire: This action marks the selected images for deletion but keeps them accessible for a temporary period of time. This can be configured in ECR settings. Selecting this action allows us to recover the image in case it is accidentally marked for deletion.
• Delete: This action permanently removes the images from the repository. This is the most space-saving option but offers no chance of retrival after deletion.

As we have understood all the details, let’s continue with our implementation.

Here I have created a lifecycle policy rule with any image status to retain 5 images in my ECR repository.

JSON Syntax for the above created policy

{
  "rules": [
    {
      "rulePriority": 1,
      "description": "Retain 5 images",
      "selection": {
        "tagStatus": "any",
        "countType": "imageCountMoreThan",
        "countNumber": 5
      },
      "action": {
        "type": "expire"
      }
    }
  ]
}

Once our lifecycle policy rule gets created, it will usually take around 24 hours until the images in our repository starts getting deleted.

But wait! We need to test the lifecycle rules before it gets applied to our ECR.

Testing

In order to test your created lifecycle policy rule click on actions > Copy policy to test rules.

Click on “Run test” to evaluate your ECR images against the test lifecycle rules. Running these test lifecycle rules will take few minutes and and you will see the results in the console.

Wohoo🎉, Our lifecycle policy dry run is executed successfully and shows the impact as expected.

Conclusion

In this blog, we have understood how we can efficiently manage the images in our container registry using lifecycle policy and hence reduce the cost in our AWS billings.

I hope you found this content informative and enjoyable. For more blogs and updates, please consider following and clicking the 👏 button below to show your support. Happy coding! 🚀

Thank you for reading! 💚