efs-mt-1.0.1
© 2018 Amazon Web Services, Inc. and its affiliates. All rights reserved. This sample code is made available under the MIT-0 license. See the LICENSE file.
Errors or corrections? Email us at [email protected].
This tutorial is designed to help you better understand how to use Amazon CloudWatch to monitor the performance of your Amazon EFS file systems.
- An AWS account
- An Amazon EFS file system
The best way to understand the workload your file system serves is by monitoring the metrics Amazon CloudWatch collects and processes. By monitoring the EFS CloudWatch metrics TotalIOBytes, DataWriteIOBytes, DataReadIOBytes, and MetaDataIOBytes in the CloudWatch console, you can see in near real-time your file system performance. These metrics are sent to CloudWatch at 1-minute intervals and are available for the next 15 months, so you can access historical information about the workload that has run on your file system over time.
Metric Math, a feature within Amazon CloudWatch, makes it easy to perform math analytics on your metrics to derive additional insights into the health and performance of your AWS resources and applications. In this case your Amazon EFS file system.
Amazon CloudWatch dashboards are customizable home pages in the CloudWatch console that you can use to monitor your resources in a single view. You can use CloudWatch dashboards to create customized views of the metrics and alarms for your AWS resources. You can have up to 500 dashboards in your AWS account. All dashboards are global, not region-specific.
Below are some useful metric math expressions for EFS file systems.
To calculate the average throughput (MiB/s) for a period, divide the respective IOBytes sum statistic (Total, DataRead, DataWrite, Metadata) by the number of seconds in the period.
Logic: ( sum of …IOBytes ÷ 1048576(to convert to MiB) ) ÷ seconds in the period
| CloudWatch Attribute | Id | Details | Statistic | Period |
|---|---|---|---|---|
| Metric math expression | e1 | (m1/1048576)/PERIOD(m1) | ||
| Metric | m1 | …IOBytes | Sum | 1 Minute |
To calculate the average operations per second (iops) for a period, divide the respective IOBytes sample count statistic (Total, DataRead, DataWrite, Metadata) by the number of seconds in the period.
Logic: ( sample count of …IOBytes ) ÷ seconds in the period
| CloudWatch Attribute | Id | Details | Statistic | Period |
|---|---|---|---|---|
| Metric math expression | e1 | m1/PERIOD(m1) | ||
| Metric | m1 | …IOBytes | Sample count | 1 Minute |
To calculate the percent throughput (%) of the different IO types (DataRead, DataWrite, Metadata) for a period, multiple the respective IOBytes sum statistic by 100 and divide by the sum statistic of TotalIOBytes for the same period.
Logic: ( sum of …IOBytes x 100(to convert to %) ) ÷ sum of TotalIOBytes
| CloudWatch Attribute | Id | Details | Statistic | Period |
|---|---|---|---|---|
| Metric math expression | e1 | (m2*100)/m1 | ||
| Metric | m1 | TotalIOBytes | Sum | 1 Minute |
| Metric | m2 | …IOBytes | Sum | 1 Minute |
To calculate the percent operations per second (%) of the different IO types (DataRead, DataWrite, Metadata) for a period, multiple the respective IOBytes sample count statistic by 100 and divide by the sample count statistic of TotalIOBytes for the same period.
Logic: ( sample count of …IOBytes x 100(to convert to %) ) ÷ sample count of TotalIOBytes
| CloudWatch Attribute | Id | Details | Statistic | Period |
|---|---|---|---|---|
| Metric math expression | e1 | (m2*100)/m1 | ||
| Metric | m1 | TotalIOBytes | Sample count | 1 Minute |
| Metric | m2 | …IOBytes | Sample count | 1 Minute |
To calculate the percent throughput utilization for a time period, first calculate the average throughput (in MiB/second) for a time period by dividing the sum statistic of TotalIOBytes by 1048576 (to convert it to MiB) and divide by the number of seconds in the period. Then take that result and multiply by 100, divide by the sum statistic of PermittedThroughput multiplied by 1048576 (to convert to it MiB).
Logic: ((( sum of TotalIOBytes ÷ 1048576 (to convert to MiB) ) ÷ seconds in the period ) x 100 ) ÷ ( sum of PermittedThroughput ÷ seconds in the period )
| CloudWatch Attribute | Id | Details | Statistic | Period |
|---|---|---|---|---|
| Metric math expression | e1 | (m1/1048576)/PERIOD(m1) | ||
| Metric math expression | e2 | m2/1048576 | ||
| Metric math expression | e3 | e2-e1 | ||
| Metric math expression | e4 | ((e1)*100)/(e2) | ||
| Metric | m1 | TotalIOBytes | Sum | 1 Minute |
| Metric | m2 | PermittedThroughput | Average | 1 Minute |
No metric math is needed to calculate the percent of IOPS utilization for General Purpose performance mode file systems. This metric is available as PercentIOLimit. This metric is not available for Max I/O performance mode file systems.
| CloudWatch Attribute | Id | Details | Statistic | Period |
|---|---|---|---|---|
| Metric | m1 | PercentIOLimit | Average | 1 Minute |
To calculate the average IO size (KiB) for a period, divide the respective IOBytes sum statistic (DataRead, DataWrite, Metadata) by the same IOBytes sample count statistic.
Logic: ( sum of …IOBytes ÷ 1024(to convert to KiB) ) ÷ ( sample count of …IOBytes )
| CloudWatch Attribute | Id | Details | Statistic | Period |
|---|---|---|---|---|
| Metric math expression | e1 | (m1/1024)/m2 | ||
| Metric | m1 | …IOBytes | Sum | 1 Minute |
| Metric | m2 | …IOBytes | Sample count | 1 Minute |
For more information about Amazon CloudWatch, please refer to https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/WhatIsCloudWatch.html
Click the 
link below to create the AWS CloudFormation stack in your account and desired AWS region. You must also have an Amazon EFS file system in this region to use with this tutorial. It creates an Amazon CloudWatch dashboard with widgets displaying file system performance metrics (throughput, IOPS, etc.) using metric math expressions. The metric math expressions described above are included in this template. An AWS Lambda function is also created and scheduled to run every minute which gets the metered size of the file system, from the describe-file-system command, and adds the value as a custom metric to CloudWatch. This custom metric is added to the File system metrics widget.
| AWS Region Code | Name | Launch |
|---|---|---|
| us-east-1 | US East (N. Virginia) |  |
| us-east-2 | US East (Ohio) | |
| us-west-1 | US West (N. California) | |
| us-west-2 | US West (Oregon) | |
| eu-west-1 | EU (Ireland) | |
| eu-central-1 | EU (Frankfurt) | |
| ap-southeast-2 | AP (Sydney) | |
| ap-northeast-1 | AP (Tokyo) | |
| ap-northeast-2 | AP (Seoul) | |
After launching the AWS CloudFormation Stack above, you should see a new dashboard in your Amazon CloudWatch console using the naming convention aws-region_efs-filesystem-id_mm.
For feedback, suggestions, or corrections, please email me at [email protected].
This sample code is made available under the MIT-0 license. See the LICENSE file.