New Relic Infrastructure Amazon EC2 Integration on AWS

Partner Solution Deployment Guide

February 2021
Rohit Kaul, New Relic
Dylan Owen, AWS Integration & Automation team

Refer to the GitHub repository to view source files, report bugs, submit feature ideas, and post feedback about this Partner Solution. To comment on the documentation, refer to Feedback.

This Partner Solution was created by New Relic in collaboration with Amazon Web Services (AWS). Partner Solutions are automated reference deployments that help people deploy popular technologies on AWS according to AWS best practices. If you’re unfamiliar with AWS Partner Solutions, refer to the AWS Partner Solution General Information Guide.

Overview

This guide covers the information you need to deploy the New Relic Infrastructure Amazon EC2 Integration Partner Solution in the AWS Cloud.

This Amazon Web Services (AWS) Partner Solution deploys New Relic Infrastructure Amazon EC2 Integration in the AWS Cloud.

This Partner Solution is for New Relic customers who use AWS to launch applications in Amazon Elastic Compute Cloud (Amazon EC2) instances with built-in monitoring using the New Relic infrastructure agent. After you install the New Relic infrastructure agent on each EC2 instance, you can view metrics from that instance and get a comprehensive, real-time view of the performance and status of your EC2 instances.

Costs and licenses

This Quick Start requires a New Relic license key, which is available with your New Relic account. If you are using a third-party software vendor (such as Microsoft, SAP, Oracle, or IBM) with your EC2 instances, you are responsible for complying with the license agreements that are required by that software vendor. To learn more about managing software licenses with third-party vendors, see AWS License Manager.

Architecture

Deploying this Partner Solution with default parameters builds the following New Relic environment in the AWS Cloud.

Figure 1. Partner Solution architecture for New Relic on AWS

As shown in Figure 1, this Partner Solution sets up the following:

(Optional) Amazon EC2 instance profile with a linked AWS Identity and Access Management (IAM) role used for granting access to AWS Systems Manager service functionality within the Amazon EC2 instance.*
An AWS Systems Manager Automation execution IAM role to allow the Amazon EC2 instances to run the AWS Systems Manager Automation runbook.
In the Region:
- An Amazon EventBridge rule to filter the Amazon EC2 instances for the matching instance ID and then pass the ID to AWS Systems Manager.
- An AWS Systems Manager Automation runbook with instructions for retrieving the New Relic license key and installing the New Relic infrastructure agent on Amazon EC2 instances with matching tags.
- An AWS Systems Manager Command document to run the installation commands listed in the Automation runbook.
IAM configurations that include groups, roles, and instance profiles as well as customizable IAM policies.
Amazon Eventbridge for providing the filtering rule.
AWS Systems Manager for providing the Automation runbook and Command document.
AWS Secrets Manager for storing the New Relic license key.

* The template that deploys this Partner Solution into an existing VPC skips the components marked by asterisks and prompts you for your existing VPC configuration.

Deployment options

This Partner Solution provides the following deployment options:

Deploy New Relic into a stack set. This recommended option is useful when your environment includes EC2 instances that span across multiple Regions and accounts.
Deploy New Relic into a single AWS Account. This option is useful for testing or troubleshooting, for example when you want to quickly deploy the stack in a given Region, and then test the automation to confirm that it works as expected.

This Partner Solution provides a single template for both options. If using a stack set, you can also launch the Quick Start using the command line.

Predeployment steps

Prepare your AWS account

To prepare your AWS account, do the following:

Obtain administrator access to your AWS account so you can deploy, manage, and test the deployment. If using AWS Control Tower, deploy the solution to the Control Tower management account in your home Region where the Control Tower landing zone was configured.
Set up AWS CloudFormation stack set permissions in your AWS account. Use self-managed permissions for better security and control. If using AWS Control Tower, use the preconfigured IAM roles AWSControlTowerStackSetRole and AWSControlTowerExecution.
Install AWS Systems Manager Automation and confirm that the SSM Agent is installed on the EC2 instances. By default, the SSM Agent is already installed on many Windows and Linux Amazon Machine Images (AMI). If you are using an AMI that doesn’t have the agent installed, you must manually install the SSM Agent. For installation instructions, see Manually install SSM Agent on EC2 instances for Linux and Manually install SSM Agent on EC2 instances for Windows Server.

Prepare your New Relic account

Confirm that you have an active New Relic account. You can create an account using the New Relic One pricing plan. Be sure to choose a Standard, Pro, or Enterprise plan for access to administrator features and support. If you don’t already have a New Relic account, you can sign up for a free one in the AWS Marketplace. After setting up your New Relic account, log in to get started.

Deployment steps

Sign in to your AWS account, and launch this Partner Solution, as described under Deployment options. The AWS CloudFormation console opens with a prepopulated template.
Choose the correct AWS Region, and then choose Next.
On the Create stack page, keep the default setting for the template URL, and then choose Next.

On the Specify stack details page, change the stack name if needed. Review the parameters for the template. Provide values for the parameters that require input. For all other parameters, review the default settings and customize them as necessary. When you finish reviewing and customizing the parameters, choose Next.

Unless you’re customizing the Partner Solution templates or are instructed otherwise in this guide’s Predeployment section, don’t change the default settings for the following parameters: QSS3BucketName, QSS3BucketRegion, and QSS3KeyPrefix. Changing the values of these parameters will modify code references that point to the Amazon Simple Storage Service (Amazon S3) bucket name and key prefix. For more information, refer to the AWS Partner Solutions Contributor’s Guide.

On the Configure stack options page, you can specify tags (key-value pairs) for resources in your stack and set advanced options. When you finish, choose Next.
On the Review page, review and confirm the template settings. Under Capabilities, select all of the check boxes to acknowledge that the template creates AWS Identity and Access Management (IAM) resources that might require the ability to automatically expand macros.
Choose Create stack. The stack takes about 5 minutes to deploy.
Monitor the stack’s status, and when the status is CREATE_COMPLETE, the New Relic Infrastructure Amazon EC2 Integration deployment is ready.
To view the created resources, choose the Outputs tab.

Deploy using CloudFormation StackSets

Deploy the stack set instances in your AWS accounts across all Regions where you plan to install the New Relic infrastructure agent on your EC2 instances. Each deployment takes about 5 minutes to complete.

The following instructions provide steps for deploying New Relic using the command line. However, you can also deploy the stack set directly from the AWS Console by creating a stack set, uploading the template file, and providing parameter field values (including your AWS accounts and Regions).

Create a stack set named NR-EC2InfraSSMAutomation in your AWS account in your StackSets administration account or your Control Tower management account. You can create the stack in another account, if necessary. Replace the following placeholders with your own values:

AWS_ACCOUNT_ID: Your AWS account ID
NEW_RELIC_LICENSE_KEY: Your New Relic license key
STACK_SET_ADMINISTRATION_ROLE_NAME: Your IAM role name for the StackSets administration role

STACK_SET_EXECUTION_ROLE_NAME: Your IAM role names for the StackSets execution role

aws cloudformation create-stack-set \
    --stack-set-name NR-EC2InfraSSMAutomation \
    --template-body file:./templates/NR-EC2InfraAgentSSMAutomation.yml \
    --description "SSM Automation workflow for installing New Relic Infrastructure Agent on EC2 instances" \
    --parameters ParameterKey=NewRelicLicenseKey,ParameterValue=<NEW_RELIC_LICENSE_KEY> \
    --capabilities CAPABILITY_NAMED_IAM \
    --administration-role-arn arn:aws:iam::<AWS_ACCOUNT_ID>:role/<STACK_SET_ADMINISTRATION_ROLE_NAME> \
    --execution-role-name <STACK_SET_EXECUTION_ROLE_NAME> \
    --permission-model SELF_MANAGED

After a few minutes, verify that the stack was created by running the following command. Make sure the stack name NR-EC2InfraSSMAutomation is returned.

aws cloudformation describe-stack-set --stack-set-name NR-EC2InfraSSMAutomation --query "StackSet.StackSetName"

Next, create instances of the NR-EC2InfraSSMAutomation StackSet across various AWS accounts and Regions where you plan to launch EC2 instances. The following example creates instances in two accounts and in four Regions in the United States.
```
aws cloudformation create-stack-instances \
    --stack-set-name NR-EC2InfraSSMAutomation \
    --accounts 123456789012 223456789012 \
    --regions us-east-1 us-east-2 us-west-1 us-west-2
```
After the command returns successfully, the Stack instances creation operation starts and an OperationId is returned. You can check the status of the operation after a few seconds. To verify the operation was successfully completed, run the below command, and make sure the status, viz. SUCCEEDED is returned. Use the returned OperationId value in place of OPERATION_ID placeholder.
```
aws cloudformation describe-stack-set-operation \
    --stack-set-name NR-EC2InfraSSMAutomation \
    --operation-id <OPERATION_ID> \
    --query "StackSetOperation.Status"
```

Postdeployment steps

Test the deployment

To test the deployment, log in to your AWS account, choose the Region where the StackSet instance is deployed, and launch an EC2 instance. Confirm the following:

Review the requirements for the infrastructure agent to confirm that it supports the AMI.
Confirm that the SSM Agent is installed. The agent can be installed either by default or manually, preferably at launch or afterwards.
Confirm that the instance profile has permissions to run SSM commands on the EC2 instance. If you can’t create a new instance profile or use an existing one, use the profile that was created by the deployment (NRSSMInstanceProfile). The deployment creates the instance profile when you set the CloudFormation template parameter CreateEC2InstanceProfile to Yes. For more information, see Create an IAM instance profile for Systems Manager.
Add the tag named NR-Infrastructure and set the value to Install. This tag key-value pair allows the automation workflow to automatically install the New Relic infrastructure agent on the EC2 instance as soon as it is launched and in running state.

Connect to your EC2 instance, open a terminal client, and run the newrelic-infra --version command. Running this command displays the New Relic infrastructure agent version information and confirms that the agent was successfully installed on the EC2 instance. Now you can monitor the instance from your New Relic account.

Log in to your New Relic account, navigate to the entity explorer, and look for the EC2 instance on the Hosts page. You can launch more instances using a variety of different AMIs that are supported by the New Relic infrastructure agent. These instances automatically show up in your New Relic account.

Troubleshooting

For troubleshooting common Partner Solution issues, refer to the AWS Partner Solution General Information Guide and Troubleshooting CloudFormation.

FAQ

Q. I can’t connect to the instance I launched for testing using Session Manager.

A. Confirm that your instance is associated with an instance profile that is configured to use AWS Systems Manager and that the SSM Agent is installed on the instance. For more information, see Create an IAM instance profile for Systems Manager.

Also be sure that the security group associated with your instance allows incoming SSH traffic from your IP address.

After you launch an instance, sometimes it takes a few minutes before you can connect to it. Check that your instance has passed the status checks. You can view this information in the Status check column on the Instances page.

If you have tried these steps and the issue is still not resolved, see Troubleshooting Session Manager. For general connection issues, see Troubleshooting connecting to your instance.

Q. The New Relic infrastructure agent is installed, but no data shows up in New Relic.

A. Be sure that the agent configuration file (newrelic-infra.yml) contains a valid license key (license_key) and that it matches your New Relic license key.

Also verify that no firewalls or proxies are blocking outbound connections from the agent process to the infrastructure domains and ports. For more information about domains and ports, see Infrastructure agents.

For accounts installing the infrastructure agent for the first time, several minutes may pass before you see data in the infrastructure monitoring UI. If you’ve verified the installation and no obvious error conditions appear in the verbose logs, monitor the infrastructure UI for a while longer. If the data still does not appear, contact New Relic Support for assistance. For more information, see the following resources:

Q. The New Relic infrastructure agent is not running and there are no logs.

A. Make sure you tagged the instance appropriately, with NR-Infrastructure set to Install. Verify that the AMI’s operating system and architecture is supported by the New Relic infrastructure agent. If supported, investigate the SSM Automation that runs in response to the EC2 instance starting up, and see if it produced any errors.

SSM Automation reports provide information about various statuses that an automation action or step goes through when you run an automation, and for the overall automation. To check status, do these steps:

Navigate to the Systems Manager console.
Choose the same Region where you launched your instance.
In the navigation pane, choose Automation.
In the Executions tab, locate the status message beside the document named NR-EC2InfraAgentSSMAutomationDoc. If an error has occurred, check the message for the possible root cause and address or report the error. If no error has occurred, try running the automation manually. For more information, see Running an automation manually.

Resources

Customer responsibility

After you deploy a Partner Solution, confirm that your resources and services are updated and configured—including any required patches—to meet your security and other needs. For more information, refer to the Shared Responsibility Model.

Feedback

To submit feature ideas and report bugs, use the Issues section of the GitHub repository for this Partner Solution. To submit code, refer to the Partner Solution Contributor’s Guide. To submit feedback on this deployment guide, use the following GitHub links:

Notices

This document is provided for informational purposes only. It represents current AWS product offerings and practices as of the date of issue of this document, which are subject to change without notice. Customers are responsible for making their own independent assessment of the information in this document and any use of AWS products or services, each of which is provided "as is" without warranty of any kind, whether expressed or implied. This document does not create any warranties, representations, contractual commitments, conditions, or assurances from AWS, its affiliates, suppliers, or licensors. The responsibilities and liabilities of AWS to its customers are controlled by AWS agreements, and this document is not part of, nor does it modify, any agreement between AWS and its customers.

The software included with this paper is licensed under the Apache License, version 2.0 (the "License"). You may not use this file except in compliance with the License. A copy of the License is located at https://aws.amazon.com/apache2.0/ or in the accompanying "license" file. This code is distributed on an "as is" basis, without warranties or conditions of any kind, either expressed or implied. Refer to the License for specific language governing permissions and limitations.