An instance runs out of memory at the busiest hour of the night. You open CloudWatch the next morning to find out why, and the AWS/EC2 namespace offers CPU, network, disk operations and status checks. There is no memory graph to look at. Nothing is broken: EC2 does not publish memory, and no setting on the instance changes that. The number lives inside the operating system, and something inside the operating system has to send it.
What AWS/EC2 actually contains
The EC2 User Guide lists the instance metrics in AWS/EC2:
CPUUtilization, the percentage of physical CPU time EC2 uses to run the instance.DiskReadOps,DiskWriteOps,DiskReadBytesandDiskWriteBytes, for instance store volumes only. With no instance store, the value is 0 or the metric is not reported.NetworkIn,NetworkOutand the packet counts.- On Nitro-based instances that are not bare metal, EBS metrics such as
EBSReadOpsandEBSWriteBytes. - The status check metrics, such as
StatusCheckFailed.
Memory is not on the list. Neither is free disk space. Those are figures the guest keeps for itself: when the CloudWatch agent collects memory, its source is “the host’s memory management subsystem”, such as /proc/meminfo on Linux. Even CPU is counted differently: CloudWatch’s figure includes time EC2 spends running the instance, and the guide warns that tools in your operating system can show a different percentage.
Detailed monitoring changes the period, not the list
By default, each EC2 data point covers 5 minutes. Status check metrics are the exception: they arrive every minute at no charge. Detailed monitoring, which is charged, makes each data point cover one minute. Mainly it turns a five-minute average into a one-minute one, which matters when a spike at 23:00 vanishes into the average of 22:55 to 23:00. It does not add memory, or any other metric, to the list.
It also sets the shortest sensible alarm. The CloudWatch guide says to pick an alarm period of at least 300 seconds on basic monitoring and at least 60 seconds on detailed monitoring.
So if a scenario asks for memory and offers “enable detailed monitoring”, that option is a trap. And if it asks for one-minute CPU, detailed monitoring is the simple answer: the number is already in AWS/EC2.
The CloudWatch agent
The CloudWatch agent is software you install on the server. It collects metrics, logs and traces from EC2 instances and from on-premises servers, on Linux, Windows Server and macOS. On EC2, its metrics are in addition to the ones EC2 publishes by default.
- On Linux, it offers metrics such as
mem_used_percentanddisk_used_percent. - On Windows Server, it collects Windows Performance Monitor counters, named by joining the object and the counter, so
% Interrupt TimeonProcessorbecomesProcessor % Interrupt Time. - Its namespace is
CWAgentby default, and you can change it. - Its metrics are billed as custom metrics.
Setting it up follows four steps: an IAM role or user that lets the agent publish, the agent package, a configuration file naming the metrics to collect, and starting the agent. You can install it from the command line or through AWS Systems Manager. It can also take custom metrics from applications over StatsD and collectd.
Resolution and retention
Every CloudWatch metric is either standard resolution, at one minute, or high resolution, at one second. Metrics from AWS services are standard resolution by default. A custom metric can be either. Only custom metrics stored at a resolution of 1 second can be read at sub-minute periods of 1, 5, 10 or 30 seconds. Read a standard metric at 10 seconds and you get nothing finer than a minute.
High resolution has two costs. Every PutMetricData call is charged, so publishing every second means more calls. And short data is kept briefly. CloudWatch keeps data points of less than 60 seconds for 3 hours, one-minute points for 15 days, five-minute points for 63 days and one-hour points for 455 days, aggregating as it goes. An alarm on a high-resolution metric can use a 10 or 30 second period, at a higher charge.
Metrics from logs: embedded metric format
Code that already writes logs can create metrics without calling PutMetricData at all. The embedded metric format is a structured log line that holds metric values and dimensions. When the line reaches CloudWatch Logs, CloudWatch extracts the custom metrics automatically. The guide presents it as a way to generate custom metrics from ephemeral resources such as Lambda functions and containers. It needs logs:PutLogEvents permission but not cloudwatch:PutMetricData.
It is not free. You pay for log ingestion and archival, and for the custom metrics it creates. Choose dimensions with care: each unique combination of dimension values is a separate metric, so a dimension such as requestId creates one metric per request.
What the exam guides ask
The SOA-C03 guide asks you to “use AWS services (for example, Amazon CloudWatch, AWS CloudTrail, Amazon Managed Service for Prometheus) to configure monitoring and logging for workloads” (Skill 1.1.1), to “configure and manage the CloudWatch agent to collect metrics and logs from EC2 instances, Amazon ECS clusters, or Amazon EKS clusters” (Skill 1.1.2), and to “optimize compute resources and remediate performance problems by using performance metrics, resource tags, and AWS tools” (Skill 1.3.1). The DOP-C02 guide asks you to “configure the collection, aggregation, and storage of logs and metrics” (Task 4.1) and “automate monitoring and event management of complex environments” (Task 4.3). The DVA-C02 guide asks you to “implement custom metrics (for example, Amazon CloudWatch embedded metric format [EMF])” (Skill 4.1.4) and “implement code that emits custom metrics” (Skill 4.2.3).
Scenario questions on this topic name a number and ask where it comes from. The distractors are predictable:
- Detailed monitoring for memory. It changes the period of metrics that exist.
- A memory metric in AWS/EC2. There is none to alarm on.
- A standard-resolution metric read at 10 seconds. The data is stored per minute.
- One-second data kept for a week. It is kept for 3 hours.
PutMetricDataon every request in code that already logs each request, where the embedded metric format avoids the calls.
Play it
The free stage below is a monitoring bill to fix. Kestrel Parcels’ Windows sorting hosts stall at the 23:00 cutoff and no metric shows why. Someone turned on detailed monitoring, and last month’s bill doubled on one-second metrics nobody reads. You give each of the ops lead’s needs a source: memory on 40 sorting hosts, one-minute CPU, three on-premises depot servers, a Lambda function’s render time per depot, scan rates read every 10 seconds at the Christmas peak, Fargate tasks and an EKS cluster’s Prometheus metrics. Every row must get its number, and the whole design must come in under the month’s budget.