Git webhooks on AWS

Partner Solution Deployment Guide

QS

January 2021
Kirankumar Chandrashekar and Jay McConnell, AWS Integration & Automation team

Refer to the GitHub repository to view source files, report bugs, submit feature ideas, and post feedback about this Partner Solution. To comment on the documentation, refer to Feedback.

This Partner Solution was created by Amazon Web Services (AWS). Partner Solutions are automated reference deployments that help people deploy popular technologies on AWS according to AWS best practices. If you’re unfamiliar with AWS Partner Solutions, refer to the AWS Partner Solution General Information Guide.

Overview

This guide covers the information you need to deploy the Git webhooks solution in the AWS Cloud.

Costs and licenses

There is no cost to use this AWS Solution, but you will be billed for any AWS services or resources that this AWS Solution deploys. For more information, refer to the AWS Partner Solution General Information Guide.

The solution provides an Amazon API Gateway endpoint, Lambda functions, and an AWS CodeBuild project to access, compress, and upload code to Amazon S3. AWS CodePipeline carries a cost for each active pipeline. For more information, refer to AWS CodePipeline pricing.

Depending on your configuration, the solution may deploy an AWS Key Management Service (AWS KMS) key which incurs a monthly cost for key storage and usage. For more information, refer to AWS Key Management Service pricing.

API Gateway, Amazon S3, Lambda, and AWS CodeBuild costs vary depending on how often you commit code to the connected Git repository. For more information, refer to Amazon API Gateway pricing, Amazon S3 pricing, AWS Lambda pricing and AWS CodeBuild pricing.

Architecture

Deploying this AWS Solution with default parameters builds the following Git webhooks environment in the AWS Cloud.

architecture_diagram
Figure 1. Solution architecture for Git webhooks on AWS

As shown in Figure 1, this solution sets up the following:

  • Amazon API Gateway to receive Git webhook requests and forward them to AWS Lambda.

  • An AWS Lambda function to process Git webhook requests from API Gateway and invoke an AWS CodeBuild project.

  • An AWS CodeBuild project to connect to your Git service, then retrieve, zip, and upload the latest version of your Git repository to Amazon S3.

  • An AWS Key Management Service (AWS KMS) key to encrypt/decrypt the SSH (Secure Shell) keys used by AWS CodeBuild to connect to your Git repository using SSH. The SSH key pair is generated by a Lambda-backed AWS CloudFormation custom resource when the stack is deployed.

  • Two Amazon S3 buckets: one for Git repository contents, and another for encrypted SSH keys. A Lambda-backed AWS CloudFormation custom resource deletes the contents of the S3 buckets when you delete the CloudFormation stack. If you need backups, copy the S3 buckets before deleting the stack.

* The template that deploys this Partner Solution into an existing VPC skips the components marked by asterisks and prompts you for your existing VPC configuration.

  • The solution deploys AWS Identity and Access Management (IAM) roles required by Lambda and API Gateway. The inline permissions attached to the roles are scoped using the least-privilege model. For more information, refer to Apply least-privilege permissions.

  • The AWS CodeBuild project must be able to communicate with your Git repository. For example, you can employ a SaaS-based Git service like GitHub to which CodeBuild can connect over the internet.

  • The Git repository S3 bucket this solution deploys has versioning enabled, and all previous versions are retained indefinitely. To modify the retention period, see How do I create a lifecycle rule for an S3 bucket?

Deployment options

This solution provides a single deployment option:

Deployment steps

  1. Sign in to your AWS account, and launch this Partner Solution, as described under Deployment options. The AWS CloudFormation console opens with a prepopulated template.

  2. Choose the correct AWS Region, and then choose Next.

  3. On the Create stack page, keep the default setting for the template URL, and then choose Next.

  4. On the Specify stack details page, change the stack name if needed. Review the parameters for the template. Provide values for the parameters that require input. For all other parameters, review the default settings and customize them as necessary. When you finish reviewing and customizing the parameters, choose Next.

    Unless you’re customizing the Partner Solution templates or are instructed otherwise in this guide’s Predeployment section, don’t change the default settings for the following parameters: QSS3BucketName, QSS3BucketRegion, and QSS3KeyPrefix. Changing the values of these parameters will modify code references that point to the Amazon Simple Storage Service (Amazon S3) bucket name and key prefix. For more information, refer to the AWS Partner Solutions Contributor’s Guide.

  5. On the Configure stack options page, you can specify tags (key-value pairs) for resources in your stack and set advanced options. When you finish, choose Next.

  6. On the Review page, review and confirm the template settings. Under Capabilities, select all of the check boxes to acknowledge that the template creates AWS Identity and Access Management (IAM) resources that might require the ability to automatically expand macros.

  7. Choose Create stack. The stack takes about 15 minutes to deploy.

  8. Monitor the stack’s status, and when the status is CREATE_COMPLETE, the Git webhooks deployment is ready.

  9. To view the created resources, choose the Outputs tab.

Troubleshooting

For troubleshooting common Partner Solution issues, refer to the AWS Partner Solution General Information Guide and Troubleshooting CloudFormation.

If commits to your repository do not show up in Amazon S3, do the following:

  • Check the security parameters and endpoint in your Git webhook configuration. See Configuring Git services earlier in this guide and consult your Git service documentation for help with configuring webhooks.

  • Check the AWS Lambda logs for errors. These are stored in Amazon CloudWatch Logs. For help with accessing them, see Accessing Amazon CloudWatch logs for AWS Lambda.

  • Check the AWS CodeBuild project logs for errors. To access them, do the following:

    1. Open the AWS CodeBuild console.

    2. On the Build history page, choose the Build run link for the project.

    3. On the Build status page, see the Build logs tab.

Additional resources

AWS services

Webhooks

Customer responsibility

After you deploy a Partner Solution, confirm that your resources and services are updated and configured—including any required patches—to meet your security and other needs. For more information, refer to the Shared Responsibility Model.

Feedback

To submit feature ideas and report bugs, use the Issues section of the GitHub repository for this Partner Solution. To submit code, refer to the Partner Solution Contributor’s Guide. To submit feedback on this deployment guide, use the following GitHub links:

Notices

This document is provided for informational purposes only. It represents current AWS product offerings and practices as of the date of issue of this document, which are subject to change without notice. Customers are responsible for making their own independent assessment of the information in this document and any use of AWS products or services, each of which is provided "as is" without warranty of any kind, whether expressed or implied. This document does not create any warranties, representations, contractual commitments, conditions, or assurances from AWS, its affiliates, suppliers, or licensors. The responsibilities and liabilities of AWS to its customers are controlled by AWS agreements, and this document is not part of, nor does it modify, any agreement between AWS and its customers.

The software included with this paper is licensed under the Apache License, version 2.0 (the "License"). You may not use this file except in compliance with the License. A copy of the License is located at https://aws.amazon.com/apache2.0/ or in the accompanying "license" file. This code is distributed on an "as is" basis, without warranties or conditions of any kind, either expressed or implied. Refer to the License for specific language governing permissions and limitations.