Skip to main content

AWS Cloudwatch Metric Stream

AWS Cloudwatch Metric Stream enables customers to send their Cloudwatch metrics to Last9.

Pre-requisites​

Obtain the following and copy it to your clipboard from the Home > Integrations > Cloudwatch section.

  1. HTTP Endpoint URL
  2. Username
  3. Password

Cloudwatch Integration

Setting up required IAM policy​

info

Ensuring your AWS Identity and Access Management (IAM) user account has access permissions is crucial. The following access policy is specifically crafted to enable actions related to creating Kinesis Data Streams and CloudWatch Metric Streams.

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": [
        "cloudwatch:StartMetricStreams",
        "cloudwatch:PutMetricStream",
        "cloudwatch:GetMetricStream",
        "cloudwatch:GetMetricData",
        "cloudwatch:ListMetrics",
        "cloudwatch:ListMetricStreams"
      ],
      "Resource": ["*"]
    },
    {
      "Effect": "Allow",
      "Action": [
        "firehose:PutRecord",
        "firehose:CreateDeliveryStream",
        "firehose:DescribeDeliveryStream",
        "firehose:PutRecordBatch",
        "firehose:UpdateDestination",
        "firehose:ListDeliveryStreams"
      ],
      "Resource": ["*"]
    },
    {
      "Effect": "Allow",
      "Action": [
        "s3:ListAllMyBuckets",
        "s3:CreateBucket",
        "s3:GetBucketLocation",
        "s3:GetObject",
        "s3:ListBucket",
        "s3:ListBucketMultipartUploads",
        "s3:PutObject"
      ],
      "Resource": ["arn:aws:s3:::*"]
    },
    {
      "Effect": "Allow",
      "Action": [
        "iam:CreateRole",
        "iam:CreatePolicy",
        "iam:AttachRolePolicy",
        "iam:CreatePolicyVersion",
        "iam:DeletePolicyVersion",
        "iam:PassRole"
      ],
      "Resource": [
        "arn:aws:iam::<account_id>:policy/*",
        "arn:aws:iam::<account_id>:role/*"
      ]
    },
    {
      "Effect": "Allow",
      "Action": ["logs:CreateLogGroup", "logs:CreateLogStream"],
      "Resource": [
        "arn:aws:logs:<region>:<account_id>:log-group:*:log-stream:*"
      ]
    }
  ]
}

Creating an AWS Kinesis Delivery Stream​

  1. Open the AWS Kinesis homepage (console.aws.amazon.com/kinesis/home)

  2. Open the left sidebar (click on the ☰ icon, if it is not expanded already)

  3. Click on Delivery Streams

    Delivery streams

  4. Click on Create delivery stream

    Create delivery stream

  5. Choose Direct PUT

    Direct PUT

  6. Delivery stream name = last9-$your_organization_name

    Delivery stream name

  7. Set the copied write HTTP Endpoint URL from the Last9 cluster as an HTTP endpoint.

    metrics endpoint

  8. Add username and password. Add Username and Password

  9. Choose or create an S3 bucket to save data the stream failed to deliver

    S3 Bucket for failed data

  10. Click on Create delivery stream

    Create delivery stream

Sending data from Cloudwatch to the delivery stream​

  1. Open the Cloudwatch console and click on Metrics -> Streams

    Cloudwatch console

  2. Click on Create metric stream

    Create metric stream

  3. Choose All metrics to send all Cloudwatch metrics. Optionally, you can also select the metrics you want to stream. You can include or exclude specific namespaces and metrics you want to send by using Select metrics option.

    Select metrics

  4. Ensure that you use the delivery stream created in the earlier step and that the output format is Opentelemetry 0.7.

    Delivery stream settings

  5. Enter the Custom Metric stream name as last9-$your_organization_name and then click on Create metric stream

    Create metric stream

Verification​

Once the Cloudwatch metric stream is enabled, it sends metrics with the prefix amazonaws_com_AWS prefix. They can be observed in the Hosted Grafana in the Grafana tab in Last9.

Troubleshooting​

Please get in touch with us on Discord or Email if you have any questions.