AWS CloudWatch Metrics

Overview

The Coralogix AWS Metrics integration offers a simple and easy way to ingest AWS metrics into Coralogix. It’s a cost-efficient alternative to Amazon Firehose. In addition, it accepts S3 metrics with 24-hour frequency that can’t be collected via Firehose.

The metrics data is collected by Amazon CloudWatch across all your AWS resources. Integrate AWS metrics to retrieve and ingest the data into Coralogix using CloudWatch API.

Poll intervals

By default, metrics are pulled every 10 minutes. To increase savings, you can configure a higher number.
Metric definitions are polled every 5 minutes.
Tag poll interval is 5 minutes for both resource group tagging and specialized APIs.

Prerequisites

STEP 1. Create IAM policy and with CloudFormation template.

STEP 2. Pick AWSAccount based on target Coralogix environment.
Pay attention that the ['ExternalId'] value passed to --parameter-overrides must be in the Key=Value format.

aws cloudformation deploy --capabilities CAPABILITY_IAM CAPABILITY_NAMED_IAM \
   --template-file template.yaml \
   --stack-name coralogix-cloudwatch-metrics-integration \
   --parameter-overrides AWSAccount=EU2 RoleName=coralogix-cloudwatch-metrics-integration ExternalId=my_super-secret-Id-1234

STEP 3. Run it in:

A terminal with installed AWS CLI and configured credentials, or
AWS CloudShell.

STEP 4. Get ARN of the role for the integration.

aws cloudformation describe-stacks --stack-name coralogix-cloudwatch-metrics-integration --query 'Stacks[0].Outputs[?OutputKey==`CoralogixAwsMetricsRoleArn`].OutputValue' --output text

AWS metrics integration deployment

STEP 1. From your Coralogix toolbar, go to Data Flow > Integrations.

STEP 2. From the Integrations section, select AWS Metrics.

STEP 3. Enter configuration parameters according to your application requirements (some of the fields are optional):

Integration name - A meaningful name assigned to the integration for easier identification of this integration instance.
AWS Role ARN - The role ARN created as described in [Prerequisites].
AWS Region - Metrics collection region. To collect data from multiple regions, create multiple integrations.
ExternalId - External ID that the role depends on, so it can’t be assumed only with ARN.
CloudWatch Metric Namespaces - Predefined list of all known AWS metric namespaces for metrics collection.
If With Aggregations is selected, the _min, _max and _avg metrics will be collected too, if available. Note that you can get the average from the metric with a PromQL query.
If Enrich With Tags is selected, the resource tags of the metrics related to those resources will be added to the metrics prefixed with cloud_tag. Note that some metrics are related to specific resources, while others are not. For example, the metric CPUUtilization in AWS/EC2 has label InstanceId, making the metric values relatable to a specific EC2 instance. On the other hand, the metric TotalBackupStorageBilled in AWS/RDS isn’t linked to a specific database instance that can have tags.

Application Name and Subsystem Name fields define cx_application_name and cx_subsystem_name labels to be added to all imported metrics.
Resource Tags Filter allows to filter the resources on which to collect metrics. Note that some metrics are not related to specific resources (see Enrich With Tags above). This filter has no effect on these metrics. The filter is defined by a comma-separated list of key-value pairs. For example, team,eks:cluster-name=staging will select all resources that either have the tag team set with any value or have tag eks:cluster-name=staging.

Important: The AWS API does not return tags for AWS/AutoScaling, AWS/DMS, AWS/EC2Spot, AWS/Prometheus and AWS/StorageGateway. As a result, the resources from these namespaces are excluded from filtering.

Tags Key Filter allows selection of the tags to be added as labels to the metrics. If empty, all tags will be added, prefixed with cloud_tag.
Static Labels allows entering a list of key-value pairs to be added as labels to all metrics in the integration.
Poll Interval In Minutes defines how often the metrics are collected. Shorter intervals mean lower latency and higher costs, and vice versa.
Additional Statistics Configuration allows defining additional metric statistics in JSON format. For example, [{"metric_selector":{"namespace":"AWS/RDS","metric_names":["CPUUtilization"]},"extended_statistics":["p99"]}] .

STEP 4. Click Save to create the integration.

If the provided account ARN cannot be assumed by our service, an error message will be displayed. If the integration is created, metrics will start appearing in several minutes. The integration checks the list of available resources with a period of 10 minutes. As a result, it may take up to 10 minutes and above between the creation of a resource and the appearance of the metric. Resources that are created and removed between two scans may never show in the metrics.

Additional resources

Documentation	Get Started with Coralogix AWS CloudWatch Metric Streams with Amazon Data Firehose
External links	What is CloudWatch View available metrics

Support

Need help?

Our world-class customer success team is available 24/7 to walk you through your setup and answer any questions that may come up.

Feel free to reach out to us via our in-app chat or by sending us an email at [email protected].

APM using AWS EC2

Next Amazon Data Firehose Terraform Module