Alerts
Creating Alerts
The Alerts page is a tool for monitoring and responding to critical events across your entire infrastructure. Create high-level alerts to detect system-level anomalies or supervise a single Kubernetes node. Set up alerts using Telemetry data your team collects to effectively manage system-wide issues.
How Alerts Work
Alerts can be created with metric, log, or trace data. Log and trace alerts are triggered when the MW Backend Alerts Server (which has already fetched the pre-defined rules specified by the user) identifies that the total number (count) of logs or traces have fired within the pre-defined time window. Metric Alerts are triggered when the pre-defined rules that the user specifies are satisfied at the exact time that the MW Backend Alerts Server checks the metrics endpoint (e.g. If your window is set to five minutes, the MW Backend Alerts Server checks the Metrics endpoint if the conditions are met once every five minutes).Create New Alert
Step 1: Create Rule
Navigate to the Alerts section and select Create Rule:
Step 2: Configure Rule
Step 2a: Select Your Data Source
The Data Source is the foundation upon which your Alert is configured. Create an alert with any OpenTelemetry metric that your team is currently ingesting.Step 2b: Add Metrics
Metrics are parts of the Data Source that you are interested in monitoring. Perform mathematical operations on selected metrics and group them by their corresponding attributes. For more information on metrics collected in your environment, see the Metrics Control Panel.
Step 2c: Set Filters
The Filters section allows you to consolidate the data collected in the Metrics section from your specified attribute. Use inequalities and SQL operators to consolidate your search. Corresponding letters in the Metrics and Filters section indicate a relationship (e.g. Metric A is filtered by Filter A).
Step 2d: Create Triggers
A trigger is a Warning or Critical message that is sent to users when the data source a user has selected exceeds a pre-defined threshold. The window indicates the interval by which these trigger conditions are checked. Alerts will auto-resolve themselves to
Okay or No Data status from a Warning or Critical status based on the triggers you set. Step 2e: Average By
The Average By filter computes the mean of your ingested data across the designated time frame within the window section.Step 3: Set Rule Name
Create a unique name for your Alert and select additional tags that your team can use to sort and filter alerts with:
Step 4: Select Notification Medium
Send your Alerts to any Slack channel or email them directly to members of your organization. You can send alerts to multiple Slack channels or emails at a time.Step 5: Save Your Rule
Once you have finished configuring your rule, select Create. You have successfully created an Alert when it is visible on the home page in the Alerts table.Edit Existing Alerts
Edit your existing and Default Alerts from the Alert table. Hover over your desired Alert to edit, pause, or delete the Alert. Need assistance or want to learn more about Middleware? Contact us at support[at]middleware.io.