Amazon Elastic Container Service (Amazon ECS) is a fully managed container orchestration service that helps you easily deploy, manage, and scale containerized applications.

With the Infrastructure Agent (Infra Agent), you can effectively monitor your ECS containers and tasks running on ECS EC2 instances and ECS Fargate in your cloud, on-prem, and hybrid architectures.

Database Integrations and the Amazon ECS Anywhere approach are not currently supported.

Fargate Setup

Use the Middleware and AWS Fargate integration to monitor your applications without having to manage servers.

Create a Sidecar

For Each ECS Task that you want to monitor, add the following sidecar container in your task definition.

JSON

{
    "name": "mw-agent",
    "image": "ghcr.io/middleware-labs/mw-host-agent:master",
    "cpu": 256,
    "portMappings": [
        {
            "name": "8006-tcp",
            "containerPort": 8006,
            "hostPort": 8006,
            "protocol": "tcp",
            "appProtocol": "http"
        }
    ],
    "essential": true,
    "environment": [
        {
            "name": "MW_API_KEY",
            "value": "<Your Middleware API Key>"
        },
        {
            "name": "MW_TARGET",
            "value": "<Your Middleware Target Endpoint>"
        },
    ],
    "mountPoints": [],
    "volumesFrom": [],
    "logConfiguration": {
        <Setup Log Configurations as per your requirement>
    }
}

Collect Metrics

Adding the sidecar container mentioned above will automatically collect metrics data from your ECS tasks and containers. The Infra Agent uses the AWS ECS Task Metadata Endpoint to fetch metrics data.

Collect Logs

Forward your ECS Task logs to Middleware. The mw-agent sidecar, mentioned above, must be running before proceeeding.

Step 1: Add a Fluent sidecar in your ECS task

JSON

{
    "name": "log_router",
    "image": "amazon/aws-for-fluent-bit:stable",
    "cpu": 0,
    "portMappings": [],
    "essential": true,
    "environment": [],
    "mountPoints": [],
    "volumesFrom": [],
    "user": "0",
    "firelensConfiguration": {
        "type": "fluentbit",
        "options": {
            "enable-ecs-log-metadata": "true"
        }
    }
}

Step 2: Add a log Configuration in your main container

The Infra Agent sidecar will fetch these logs and send them to Middleware.

JSON

"logConfiguration": {
    "logDriver": "awsfirelens",
    "options": {
        "Host": "127.0.0.1",
        "Name": "forward",
        "Port": "8006"
    }
}

Step 3 [Optional]: Parsing JSON Structured Logs

Firelens by default sends a JSON blob which Middleware displays as the log body. Parsing the JSON logs allows the Message of the log being displayed while the attributes will be viewable on expansion.

To change the behavior you will need to add the below options to the firelensConfiguration created in Step 1.

the JSON structured logs generated need to have a Message field. Otherwise Middleware will not know what should be shown as a body of the log.

JSON

"firelensConfiguration": {
    "type": "fluentbit",
    "options": {
        "config-file-type": "file",
        "config-file-value": "/fluent-bit/configs/parse-json.conf",
        "enable-ecs-log-metadata": "true"
    }
}

EC2 Setup

Step 1: Create & Configure a Infra Agent ECS Task Definition

Begin by creating a Task Definition for the Infra Agent container.

JSON

{
    "family": "mw-agent-task",
    "containerDefinitions": [
        {
            "name": "mw-agent",
            "image": "ghcr.io/middleware-labs/mw-host-agent:master",
            "cpu": 100,
            "memory": 512,
            "portMappings": [
                {
                    "name": "mw-agent-9319-tcp",
                    "containerPort": 9319,
                    "hostPort": 9319,
                    "protocol": "tcp",
                    "appProtocol": "grpc"
                },
                {
                    "name": "mw-agent-8006-tcp",
                    "containerPort": 8006,
                    "hostPort": 8006,
                    "protocol": "tcp"
                },
                {
                    "name": "mw-agent-9320-tcp",
                    "containerPort": 9320,
                    "hostPort": 9320,
                    "protocol": "tcp"
                }
            ],
            "essential": true,
            "environment": [
                {
                    "name": "MW_API_KEY",
                    "value": "<Your MW API Key here>"
                },
                {
                    "name": "MW_TARGET",
                    "value": "<Your MW Target here>"
                }
            ],
            "mountPoints": [
                {
                    "sourceVolume": "docker-sock",
                    "containerPath": "/var/run/docker.sock",
                    "readOnly": false
                },
                {
                    "sourceVolume": "proc",
                    "containerPath": "/rootfs/proc",
                    "readOnly": true
                },
                {
                    "sourceVolume": "dev",
                    "containerPath": "/rootfs/dev",
                    "readOnly": true
                },
                {
                    "sourceVolume": "al2_cgroup",
                    "containerPath": "/sys/fs/cgroup",
                    "readOnly": true
                },
                {
                    "sourceVolume": "al1_cgroup",
                    "containerPath": "/cgroup",
                    "readOnly": true
                },
                {
                    "sourceVolume": "al2_cgroup",
                    "containerPath": "/rootfs/sys/fs/cgroup",
                    "readOnly": true
                },
                {
                    "sourceVolume": "al1_cgroup",
                    "containerPath": "/rootfs/cgroup",
                    "readOnly": true
                },
                {
                    "sourceVolume": "docker-containers-root",
                    "containerPath": "/var/lib/docker/containers",
                    "readOnly": true
                },
                {
                    "sourceVolume": "var-log-host",
                    "containerPath": "/var/log",
                    "readOnly": true
                }
            ],
            "volumesFrom": [],
            "privileged": true,
            "systemControls": []
        }
    ],
    "networkMode": "host",
    "volumes": [
        {
            "name": "docker-sock",
            "host": {
                "sourcePath": "/var/run/docker.sock"
            }
        },
        {
            "name": "proc",
            "host": {
                "sourcePath": "/proc"
            }
        },
        {
            "name": "dev",
            "host": {
                "sourcePath": "/dev"
            }
        },
        {
            "name": "al1_cgroup",
            "host": {
                "sourcePath": "/cgroup"
            }
        },
        {
            "name": "al2_cgroup",
            "host": {
                "sourcePath": "/sys/fs/cgroup"
            }
        },
        {
            "name": "docker-containers-root",
            "host": {
                "sourcePath": "/var/lib/docker/containers"
            }
        },
        {
            "name": "var-log-host",
            "host": {
                "sourcePath": "/var/log"
            }
        }
    ],
    "requiresCompatibilities": [
        "EC2"
    ],
    "pidMode": "host"
}

Step 2: Register your Task Definition File

You can register your Task Definition File using the the AWS CLI or the Amazon Web Console.

AWS CLI

Execute the following command to register your Task Definition File in AWS. Learn more about the Amazon ECS CLI here.

CLI

aws ecs register-task-definition --cli-input-json file://<mw-ecs-agent.json>

AWS Web UI

Step 3: Schedule the Infra Agent as a Daemon Service

Container Trace and Log Collection

Begin collecting container trace and log data in your EC2 instance. The Infra Agent will collect ECS container logs emitted to the stdout and stderr log stream and receive traces from your application and send to your Middleware account.

The Infra Agent must be running as an app before setting up collection. If not, attempted trace collection will throw an error while connecting to the agent and logs will not be sent at all.

Modify your Application Task Definition

You can setup container trace and log monitoring by modifying your Task Definition JSON file or following the below steps in the AWS Management Console.

JSON

To collect traces, configure the environment variable in the Task Definition
To collect logs, configure the logConfiguration variable to enable the fluentd logging driver

You do not have to collect container trace and log data at the same time

JSON

"containerDefinitions": [
        {
        (...)
        "environment": [
                {
        "name": "MW_AGENT_SERVICE",
        "value": "172.17.0.1"
            }
],
            "logConfiguration": {
                "logDriver": "fluentd",
                "options": {
                    "fluentd-address": "172.17.0.1:8006"
                }
            }
        (...)
        }
    ]

AWS Management Console

Visualize Your Data

Quickly access your Amazon ECS data with Middleware’s default ECS dashboard. Navigate to the Dashboard Builder and select the AWS ECS Dashboard.

Your AWS ECS data comes from the awsecscontainermetricsreceiver whereas your container data comes from the dockerstatsreceiver.

Troubleshooting

Missing Integrations Menu

Need assistance or want to learn more about Middleware? Contact us at support[at]middleware.io.

Middleware Docs

​Fargate Setup

​Create a Sidecar

​Collect Metrics

​Collect Logs

​EC2 Setup

​Container Trace and Log Collection

​Modify your Application Task Definition

​Visualize Your Data

​Troubleshooting

Fargate Setup

Create a Sidecar

Collect Metrics

Collect Logs

EC2 Setup

Container Trace and Log Collection

Modify your Application Task Definition

Visualize Your Data

Troubleshooting