CFN AWS SOCI Index Builder on AWS

Partner Solution Deployment Guide

March 2023
Amazon Elastic Container Registry team
Dylan Owen and Tony Bulding, AWS Integration & Automation team

Refer to the GitHub repository to view source files, report bugs, submit feature ideas, and post feedback about this Partner Solution. To comment on the documentation, refer to Feedback.

This Partner Solution was created by Amazon in collaboration with Amazon Web Services (AWS). Partner Solutions are automated reference deployments that help people deploy popular technologies on AWS according to AWS best practices. If you’re unfamiliar with AWS Partner Solutions, refer to the AWS Partner Solution General Information Guide.

Overview

This guide covers information you need to deploy the Seekable OCI (SOCI) Index Builder to the AWS Cloud.

Costs and licenses

There is no cost to use this solution, but you will be billed for any AWS services or resources that this solution deploys. For more information, refer to the AWS Quick Starts General Information Guide.

Architecture

Deploying this solution with default parameters builds the following SOCI Index Builder environment in the AWS Cloud.

Figure 1. Solution architecture for SOCI Index Builder on AWS

As shown in Figure 1, this solution sets up the following:

An Amazon Elastic Container Registry (Amazon ECR) to store container images and SOCI artifacts.
An AWS CloudFormation stack that contains the following resources:
- An Amazon EventBridge rule to detect Amazon ECR image action events and invoke the filtering AWS Lambda function.
- Two AWS Lambda functions to do the following:
  - Filter Amazon ECR image action events that match the filters provided in the CloudFormation stack parameters.
  - Generate SOCI indices for matching images and push indices back to image repositories in the Amazon ECR registry.
- AWS Identity and Access Management (IAM) to provide the following permissions:
  - EventBridge to invoke the filtering Lambda function.
  - Filtering Lambda function to create Amazon CloudWatch logs and to invoke the SOCI index generator Lambda function.
  - SOCI index generator Lambda function to create CloudWatch logs and access the Amazon ECR registry.
CloudWatch to collect logs from both Lambda functions for troubleshooting.

Deployment options

This solution provides the following deployment options:

Deploy SOCI Index Builder. This builds a new workflow consisting of an Amazon EventBridge rule, filtering Lambda function, SOCI index generator Lambda function, and IAM resources.
- Deploy SOCI Index Builder to GovCloud (US)
- Deploy SOCI Index Builder to China (Beijing)

Predeployment steps

Create an Amazon ECR repository to store container images and generated SOCI artifacts.

Sign in to the AWS CloudFormation console or call the CloudFormation CreateStack API using IAM credentials that have permissions to create, update, rollback, and delete a stack. Minimally, the IAM role must have permissions to create, modify, and delete stack resources. For more information, refer to Controlling access with Amazon Identity and Access Management.

Amazon EventBridge permissions

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": [
                "events:DescribeRule",
                "events:PutRule",
                "events:PutTargets",
                "events:DeleteRule",
                "events:RemoveTargets"
            ],
            "Resource": "arn:aws:events:<Region>:<AccountId>:rule/ECRImageActionEventBridgeRule"
        }
    ]
}

Amazon S3 permissions (to fetch Lambda code)

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": [
                "s3:GetObject"
            ],
            "Resource": [
                "arn:aws:s3:::aws-quickstart-<Region>/*soci_index_generator_lambda.zip",
                "arn:aws:s3:::aws-quickstart-<Region>/*ecr-image-action-event-filtering/lambda.zip"
            ]
        }
    ]
}

Lambda permissions

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": [
                "lambda:GetFunction",
                "lambda:CreateFunction",
                "lambda:UpdateFunctionConfiguration",
                "lambda:AddPermission",
                "lambda:DeleteFunction",
                "lambda:RemovePermission",
                "lambda:InvokeFunction"
            ],
            "Resource": [
                "arn:aws:lambda:<Region>:<AccountId>:function:*-ECRImageActionEventFilteringLambda-*",
                "arn:aws:lambda:<Region>:<AccountId>:function:*-SociIndexGeneratorLambda-*",
                "arn:aws:lambda:<Region>:<AccountId>:function:RepositoryNameParsingLambda"
            ]
        }
    ]
}

IAM permissions

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": [
                "iam:GetRole",
                "iam:GetRolePolicy",
                "iam:CreateRole",
                "iam:PutRolePolicy",
                "iam:PassRole",
                "iam:DeleteRole",
                "iam:DeleteRolePolicy"
            ],
            "Resource": [
                "arn:aws:iam::<AccountId>:role/*-ECRImageActionEventFilteringLambdaRole-*",
                "arn:aws:iam::<AccountId>:role/*-SociIndexGeneratorLambdaRole-*",
                "arn:aws:iam::<AccountId>:role/*-RepositoryNameParsingLambdaRole-*"
            ]
        }
    ]
}

Notes:

Replace <AccountId> with the AWS account ID from the account you deploy to, and replace <Region> with the Region you deploy to (for example, us-east-1).
The Lambda function name ECRImageActionEventFilteringLambda, specified in the Resources section, is truncated if your stack name exceeds 15 characters.
The role name ECRImageActionEventFilteringLambdaRole, specified in the Resources section, is truncated if your stack name exceeds 11 characters.
The role name SociIndexGeneratorLambdaRole, specified in the Resources section, is truncated if your stack name exceeds 21 characters.
The role name RepositoryNameParsingLambdaRole, specified in the Resources section, is truncated if your stack name exceeds 18 characters.

Deployment steps

Sign in to your AWS account, and launch this Partner Solution, as described under Deployment options. The AWS CloudFormation console opens with a prepopulated template.
Choose the correct AWS Region, and then choose Next.
On the Create stack page, keep the default setting for the template URL, and then choose Next.

On the Specify stack details page, change the stack name if needed. Review the parameters for the template. Provide values for the parameters that require input. For all other parameters, review the default settings and customize them as necessary. When you finish reviewing and customizing the parameters, choose Next.

Unless you’re customizing the Partner Solution templates or are instructed otherwise in this guide’s Predeployment section, don’t change the default settings for the following parameters: QSS3BucketName, QSS3BucketRegion, and QSS3KeyPrefix. Changing the values of these parameters will modify code references that point to the Amazon Simple Storage Service (Amazon S3) bucket name and key prefix. For more information, refer to the AWS Partner Solutions Contributor’s Guide.

On the Configure stack options page, you can specify tags (key-value pairs) for resources in your stack and set advanced options. When you finish, choose Next.
On the Review page, review and confirm the template settings. Under Capabilities, select all of the check boxes to acknowledge that the template creates AWS Identity and Access Management (IAM) resources that might require the ability to automatically expand macros.
Choose Create stack. The stack takes about 5 minutes to deploy.
Monitor the stack’s status, and when the status is CREATE_COMPLETE, the CFN AWS SOCI Index Builder deployment is ready.
To view the created resources, choose the Outputs tab.

Cleanup

To remove this solution, delete the Amazon CloudFormation stack created in the Deployment steps section.
In the Amazon CloudWatch Logs console, delete the log groups /aws/lambda/<StackName>-SociIndexGeneratorLambda and /aws/lambda/<StackName>-ECRImageActionEventFilteringLambda where <StackName> is the name of the stack that you entered in the Deployment steps section. You can also use the DeleteLogGroup API to delete the log groups.
In the Amazon ECR console, delete the SOCI index artifacts. You can also use the BatchDeleteImage API to delete the artifacts.

Troubleshooting

For troubleshooting common issues, refer to Troubleshooting CloudFormation and the AWS Quick Start General Information Guide.

Lambda logs for generating SOCI index artifacts can be found in the CloudWatch log group /aws/lambda/<StackName>-SociIndexGeneratorLambda. Lambda logs for repository image tag filtering and matches can be found in the CloudWatch log group /aws/lambda/<StackName>-ECRImageActionEventFilteringLambda, where <StackName> is the name of the stack that you entered in the Deployment steps section.

If Lambda logs for generating SOCI index artifacts show that the Lambda function is timing out (that is, Task timed out is in the log), this is likely caused by the image size. SOCI Index Builder supports a maximum compressed image size of 6 GB because larger sizes may force the SOCI index generator Lambda to time out after 15 minutes.

A SOCI index will not be generated for an image if every layer in the image is under 10 MiB. For container images greater than 250 MiB (compressed), we recommend lazy loading. You are less likely to see a time reduction for smaller images.

Resources

Customer responsibility

After you deploy a Partner Solution, confirm that your resources and services are updated and configured—including any required patches—to meet your security and other needs. For more information, refer to the Shared Responsibility Model.

Feedback

To submit feature ideas and report bugs, use the Issues section of the GitHub repository for this Partner Solution. To submit code, refer to the Partner Solution Contributor’s Guide. To submit feedback on this deployment guide, use the following GitHub links:

Notices

This document is provided for informational purposes only. It represents current AWS product offerings and practices as of the date of issue of this document, which are subject to change without notice. Customers are responsible for making their own independent assessment of the information in this document and any use of AWS products or services, each of which is provided "as is" without warranty of any kind, whether expressed or implied. This document does not create any warranties, representations, contractual commitments, conditions, or assurances from AWS, its affiliates, suppliers, or licensors. The responsibilities and liabilities of AWS to its customers are controlled by AWS agreements, and this document is not part of, nor does it modify, any agreement between AWS and its customers.

The software included with this paper is licensed under the Apache License, version 2.0 (the "License"). You may not use this file except in compliance with the License. A copy of the License is located at https://aws.amazon.com/apache2.0/ or in the accompanying "license" file. This code is distributed on an "as is" basis, without warranties or conditions of any kind, either expressed or implied. Refer to the License for specific language governing permissions and limitations.