Reference: Alertmanager

Version 1.6.9-option-3 | Updated: August 07, 2026

Reference: Alertmanager

10. Alertmanager Configuration

Alertmanager handles alert notifications from Prometheus. It supports multiple notification channels.

API Documentation: See Alertmanager API Guide for application integration (send alerts, manage silences, query status).

SNMP Integration: See SNMP Notifier Guide for NMS integration (custom OIDs, MIB file, SNMPv3 configuration).

Supported Channels

Channel Required Settings Description

Email

SMTP + ALERT_EMAIL

Uses SMTP settings from Grafana configuration

Telegram

Bot Token + Chat ID

Auto-detection supported

Slack

Webhook URL

Incoming Webhook from Slack App

MS Teams

Webhook URL

Power Automate Workflows

Webhook

URL

Custom HTTP endpoint

SMS

Gateway URL + Token

HTTP gateway (Twilio, Vonage)

SNMP

Trap Address + Version

SNMP v2c/v3 traps to NMS

Interactive Setup

Alertmanager is configured during stack deployment:

curl -fsSL https://hub.friendly-tech.com/bootstrap/install.sh | sudo bash -s -- stack

# Configure alert notifications? [y/N]: y
# Select channels: 1,2  (Email + Telegram)

The bootstrap script will:

  • Guide you through each channel configuration

  • Offer testing options (MailPit for email, SNMP Trap Catcher for SNMP)

  • Auto-detect Telegram chat_id when possible

  • Validate input format

  • Save settings to .env

Manual Configuration

Edit .env and restart:

cd /opt/grafana/prometheus-grafana-stack
vi .env

Email

ALERT_EMAIL=alerts@company.com

During bootstrap, you have 3 options for Email configuration:

  1. Use Grafana SMTP settings - Uses SMTP already configured for Grafana

  2. Configure new SMTP settings - Enter custom SMTP host, user, password

  3. Use MailPit for testing - Auto-configures fake SMTP (no real emails sent)

Testing with MailPit (fake SMTP):

# Start MailPit (auto-started when option 3 is selected)
docker compose --profile testing up -d mailpit

# Web UI to view emails (opens automatically in bootstrap output)
open http://<server-ip>:8025

  • MailPit captures all emails without sending them

  • Modern HTML email templates with severity colors

  • View email history in web interface

Telegram

TELEGRAM_BOT_TOKEN=123456789:ABCdefGHI...
TELEGRAM_CHAT_ID=-1001234567890

Slack

SLACK_WEBHOOK_URL=https://hooks.slack.com/services/T00/B00/XXXX
SLACK_CHANNEL=#alerts

MS Teams

MSTEAMS_WEBHOOK_URL=https://outlook.office.com/webhook/...

Custom Webhook

APP_WEBHOOK_URL=https://your-app.com/api/alerts

SMS Gateway

SMS_GATEWAY_URL=https://api.twilio.com/...
SMS_API_TOKEN=your-token

SNMP Traps

During bootstrap, you have 2 options for SNMP configuration:

  1. Configure external NMS - Enter NMS address, SNMP version, community/credentials

  2. Use SNMP Trap Catcher for testing - Auto-configures local trap receiver with web UI

# SNMPv2c
SNMP_TRAP_ADDRESS=nms.company.com:162
SNMP_VERSION=V2c
SNMP_COMMUNITY=public

# SNMPv3 (authentication + encryption)
SNMP_VERSION=V3
SNMP_AUTH_USERNAME=snmpuser
SNMP_AUTH_PASSWORD=authpassword
SNMP_PRIV_PASSWORD=privpassword

Testing with SNMP Trap Catcher:

# Start SNMP services (auto-started when option 2 is selected)
docker compose --profile snmp up -d

# Web UI to view traps (opens automatically in bootstrap output)
open http://<server-ip>:8089

  • Captures all SNMP traps locally

  • Real-time web interface showing trap history

  • Supports SNMPv2c (for testing)

SNMP Documentation:

  • SNMP Notifier Guide — full setup guide, OID tables per protocol (TR-069, USP, LWM2M), SNMPv3 configuration, device limit alarms, infrastructure resource alarms

  • Architecture: SNMP Trap Flow — diagram showing Alert → Alertmanager → snmp-notifier → SNMP trap → NMS, component descriptions, OID structure

  • OID Mapping Reference — complete OID-to-alert mapping table with numeric OIDs, alert names, and descriptions

  • MIB File — ASN.1 MIB definition (FRIENDLY-TECH-ACS-MIB v2.0.0) for import into NMS systems; on an installed server: /opt/grafana/prometheus-grafana-stack/alertmanager/mibs/FRIENDLY-TECH-ACS-MIB.txt

  • PROD-468 Implementation Report — detailed report of granular OID mapping implementation, testing results, acceptance criteria coverage

  • Thresholds UI — https://<domain>/alertmanager-config/thresholds — web interface for editing alert thresholds without restart (auto-discovers all configurable alerts)

  • Alert Rules Editor — https://<domain>/alertmanager-config/alerts — web interface for full CRUD on Prometheus alert rules (add/edit/delete alerts with OID labels)

Apply Changes
docker compose restart alertmanager

Web Interface

Testing Services

If you selected testing options during bootstrap, these UIs are available:

Service URL Purpose

MailPit

http://<server>:8025

View captured emails

SNMP Trap Catcher

http://<server>:8089

View captured SNMP traps

Testing Notification Channels

When you send a test alert to Alertmanager, it routes to ALL configured channels:

Test Alert → Alertmanager (9093) → Email (MailPit if testing)
                                 → Telegram
                                 → Slack
                                 → MS Teams
                                 → Webhook
                                 → SNMP Notifier (9464) → NMS / Trap Catcher (162)
Step 1: Send Test Alert
curl -X POST http://localhost:9093/api/v2/alerts \
  -H "Content-Type: application/json" \
  -d '[{
    "labels": {"alertname": "TestAlert", "severity": "warning", "instance": "test"},
    "annotations": {"summary": "Test alert - please ignore"}
  }]'
Step 2: Verify Each Channel
Channel How to Verify Expected Result

Email

Check inbox of ALERT_EMAIL

Email with alert details

Telegram

Check configured chat/group

Message with 🟠 WARNING

Slack

Check configured channel

Message in #alerts

MS Teams

Check configured channel

Adaptive Card message

Webhook

Check your app logs

POST request with JSON

SNMP

Check NMS or snmp-catcher

Trap with OID 1.3.6.1.4.1.65666.2.0.99
SNMP Testing
# 1. Check SNMP services are running
docker ps | grep -E "snmp-notifier|snmp-catcher"

# 2. Send test alert
curl -X POST http://localhost:9093/api/v2/alerts \
  -H "Content-Type: application/json" \
  -d '[{"labels":{"alertname":"DbUnreachable","severity":"critical"}}]'

# 3. Check snmp-notifier logs
docker logs snmp-notifier --tail 10

# 4. View traps in web UI (if using SNMP Trap Catcher)
open http://<server>:8089

SNMP Trap Catcher features:

  • Web UI at port 8089 showing all received traps

  • Real-time updates

  • Trap details: timestamp, OID, variables, source IP