Add ADOT resource monitoring example

Signed-off-by: Wesley Pettit <[email protected]>

Author: PettitWesley

README.md

@@ -25,6 +25,7 @@ We want examples of as many use cases in this repository as possible! Submit a P
 ### Monitoring Fluent Bit
 * [Send Fluent Bit internal metrics to CloudWatch](examples/fluent-bit/send-fb-internal-metrics-to-cw)
 * [Fluent Bit Container Health Check Options](examples/fluent-bit/health-check)
+* [CPU, Disk, and Memory Usage Monitoring with ADOT](examples/fluent-bit/adot-resource-monitoring/)
 
 ### Fluent Bit Examples
 

examples/fluent-bit/adot-resource-monitoring/README.md

@@ -0,0 +1,48 @@
+# FireLens Example: Quickly Enable Resource Monitoring with ADOT
+
+The [AWS Distro for OpenTelemetry](https://aws-otel.github.io/) has many features for collecting telemetry from your applications. In this tutorial, you will learn how to quickly enable resource utilization monitoring for Fluent Bit using ADOT. OpenTelemetry can do a lot more than is shown in this tutorial. 
+
+This guide is broken down into two tutorials:
+- [Tutorial 1: Quickly Enable Task Resource Monitoring](#tutorial-1-quickly-enable-task-resource-monitoring)
+- [Tutorial 2: Enable Container Level Metric Collection](#tutorial-2-enable-container-level-metric-collection)
+
+### Tutorial 1: Quickly Enable Task Resource Monitoring
+
+To quickly enable CPU, memory, network and disk usage metrics for your FireLens task, simply add the following container definition to your current task definition:
+
+```
+        {
+            "essential": true,
+            "image": "public.ecr.aws/aws-observability/aws-otel-collector:latest",
+            "name": "aws-otel-collector",
+            "logConfiguration": {
+                "logDriver": "awslogs",
+                "options": {
+                    "awslogs-group": "aws-otel-collector",
+                    "awslogs-region": "us-west-1",
+                    "awslogs-create-group": "true",
+                    "awslogs-stream-prefix": "ecs-"
+                }
+            },
+            "memoryReservation": 100,
+            "command": [
+                "--config=/etc/ecs/container-insights/otel-task-metrics-config.yaml"
+            ]
+        }
+```
+
+This uses the [built-in ADOT configuration](https://github.com/aws-observability/aws-otel-collector/blob/main/config/ecs/container-insights/otel-task-metrics-config.yaml) for ECS Task Level Metrics. Please read the full [documentation](https://aws-otel.github.io/docs/components/ecs-metrics-receiver).
+
+This configuration will send metrics to CloudWatch with the following sets of dimensions:
+- `ClusterName`
+- `ClusterName` and `TaskDefinitionFamily`
+
+This means that you will get CPU, Memory, Disk, and Network metrics for the entire task aggregated across your cluster, and the union of cluster and TaskDefinition Family.
+
+### Tutorial 2: Enable Container Level Metric Collection
+
+For this tutorial, you must do the following:
+
+1. Copy the "Full configuration for task- and container-level metrics" example from the [ADOT ECS Documentation](https://aws-otel.github.io/docs/components/ecs-metrics-receiver). 
+2. Optionally modify the dimensions and metrics sent to CloudWatch with the `awsemf` exporter. For example, you could add `TaskId` and `ContainerName` as dimensions to monitoring specific Fluent Bit containers. However, please understand that this will lead to a very large number of individual metrics in CloudWatch which will induce cost. 
+3. Follow the [ADOT ECS Custom Config tutorial](https://aws-otel.github.io/docs/setup/ecs/config-through-ssm) to run ADOT with your custom configuration.

examples/fluent-bit/adot-resource-monitoring/permissions.json

@@ -0,0 +1,12 @@
+{
+	"Version": "2012-10-17",
+	"Statement": [{
+		"Effect": "Allow",
+		"Action": [
+			"logs:CreateLogStream",
+			"logs:CreateLogGroup",
+			"logs:PutLogEvents"
+		],
+		"Resource": "*"
+	}]
+}

examples/fluent-bit/adot-resource-monitoring/task-definition.json

@@ -0,0 +1,61 @@
+{
+	"family": "firelens-example-cloudwatch",
+	"taskRoleArn": "arn:aws:iam::XXXXXXXXXXXX:role/ecs_task_iam_role",
+	"executionRoleArn": "arn:aws:iam::XXXXXXXXXXXX:role/ecs_task_execution_role",
+	"containerDefinitions": [
+		{
+			"essential": true,
+			"image": "906394416424.dkr.ecr.us-east-1.amazonaws.com/aws-for-fluent-bit:stable",
+			"name": "log_router",
+			"firelensConfiguration": {
+				"type": "fluentbit"
+			},
+			"logConfiguration": {
+				"logDriver": "awslogs",
+				"options": {
+					"awslogs-group": "firelens-container",
+					"awslogs-region": "us-west-2",
+					"awslogs-create-group": "true",
+					"awslogs-stream-prefix": "firelens"
+				}
+			},
+			"memoryReservation": 50
+		},
+		{
+            "essential": true,
+            "image": "public.ecr.aws/aws-observability/aws-otel-collector:latest",
+            "name": "aws-otel-collector",
+            "logConfiguration": {
+                "logDriver": "awslogs",
+                "options": {
+                    "awslogs-group": "aws-otel-collector",
+                    "awslogs-region": "us-west-1",
+                    "awslogs-create-group": "true",
+                    "awslogs-stream-prefix": "ecs-"
+                }
+            },
+            "memoryReservation": 100,
+            "command": [
+                "--config=/etc/ecs/container-insights/otel-task-metrics-config.yaml"
+            ]
+        },
+		{
+			 "essential": true,
+			 "image": "nginx",
+			 "name": "app",
+			 "logConfiguration": {
+				"logDriver":"awsfirelens",
+				"options": {
+					"Name": "cloudwatch_logs",
+					"region": "us-west-2",
+					"log_key": "log",
+					"log_group_name": "ecs/application",
+					"auto_create_group": "true",
+					"log_stream_prefix": "app-",
+					"retry_limit": "2"
+				}
+			},
+			"memoryReservation": 100
+		}
+	]
+}