Set Up Core Datadog AWS Integration

Integrating with Datadog

Datadog is a cloud monitoring platform you can use to provide observability for your Flightcontrol services.

AWS integration is part of Datadog, and you will need to follow these directions to get Datadog and AWS working together.

  1. Sign up for Datadog (opens in a new tab)
    • Note the “region” you select (Example: us1, us2, etc.)
  2. Open the Account settings (opens in a new tab)
    • NOTE - You’ll need to select the region you are using from the dropdown at top right Datadog region dropdown
    • Create a new API key for AWS integration
    • Copy the API key
  3. Go to /account/settings#integrations/amazon-web-services in Datadog
    • Under Configuration, click Automatically Using CloudFormation, Datadog AWS Configuration
  4. Paste the API key in the DdApiKey field API Key Field on AWS CloudFormation Create Stack screen
  5. Click Create Stack Create Stack button
  6. Back in Datadog, paste in the following:
    • AWS account ID (accessible in AWS from the top right menu)
    • AWS role name (DatadogIntegrationRole unless you changed it) AWS account id and role name
  7. Wait until the Datadog CloudFormation stack has completed
    • If the Forwarder stack fails with error like this, you’ll need to request an increase of your Lambda Concurrency Limit (see instructions at the bottom of this page) Specified ReservedConcurrentExecutions for function decreases account's UnreservedConcurrentExecution below its minimum value of [50]
  8. Click Install Integration, then you should see a success message Install Integration success message
  9. You should now see some infrastructure under the Infrastructure menu item.

To Set Up Log Forwarding

Log forwarding to Datadog requires the datadog-forwarder stack to be created in every region you want to forward logs from. The default aws datadog integration automatically installs it in us-east-1. See instructions below for installing it in a new region.

How to Install datadog-forwarder in a new region

  1. Go to (opens in a new tab)

  2. Click on the CloudFormation stack link, then click Launch Stack

    CloudFormation stack
  3. Change the AWS region to your desired region, using the AWS menu

    AWS region selector
  4. In the CloudFormation create stack screen, make sure you:

  5. Click the orange Create Stack button.

How to set up automatic forwarding of some services

  1. Go to the Datadog Account Settings (opens in a new tab) (remember you’ll need to select your datadog region from the little dropdown)

  2. Click on Collect Logs tab

    Datadog Collect Logs tab
  3. Inside AWS, find the ARN value of the datadog-forwarder lambda function and paste that into this Datadog form.

    Datadog form for ARN
  4. Select all the services you’d like to receive logs from, then click Save

For more details on setting up log triggers, see Send AWS services logs with the datadog lambda function (opens in a new tab).

How to set up log forwarding for ECS logs

  1. Find the datadog-forwarder lambda in the region you want
  2. Click Add trigger Adding trigger to the lambda function
  3. Click Select Trigger and select CloudWatch Logs
  4. Click log group and find the log group for your ECS service in format /ecs/<service-id>-randomeTaskDefinition
  5. Enter a name to remember this by in “filter name”
  6. Click Add Adding trigger for CloudWatch Logs

To Increase Lambda Concurrency Limit

  1. Go to Quotas (opens in a new tab)

  2. Select Concurrent Executions, then click Request quota increase

    AWS Lambda Concurrency Quota Increase
  3. Enter a higher quota value, then click Request

    AWS Lambda Concurrency Quota Increase Value