This guide explains how alerting works in Ntrospect: how alerts are created, how their severity is determined, how notifications reach people, and — most importantly — how to prove the whole chain works before the night it matters. It covers the three ways an alert is born (one click from a card, the rule editor, and the Ntuition Engine's own analysis), the routing that decides who is told, and the delivery log that shows what actually happened. Keep it with your operations documentation; every screen it references states these same rules in place.
The one-glance health check
Open Settings → Alert Rules. The header strip answers the three questions that matter before any detail does:
- Is anything evaluating? — a green dot with "engine evaluating every 30s · last cycle <time>". Red means rules exist but nothing is checking them.
- How many rules are there?
- Do any fire into the void? — "N enabled with no destination — they fire and notify nobody" in red. A rule without a destination evaluates, records history, and tells no one. This number should be zero.
How alerts are organized
Rules are grouped by what they watch — Server (CPU, memory, disk, host health), Network (flows, SNMP, synthetics, paths), Logs, AI (GPU fleet and AI workloads), and Platform (gateways, clusters, the fleet itself). Each card shows the rule's state, severity, condition, its destination, and the outcome of its last delivery. Each category header counts its own notify-nobody rules.
Two controls on every card save immediately — no Save button:
- the enable switch;
- the Notify picker — the routing policy that decides who is told. A red "— nobody —" picker (and a red card border) means exactly that.
Three ways an alert is born
1 — One click from the thing itself (recommended)
Anywhere the platform shows a metric-backed card, it can become an alert pre-filled from its own context and data:
- Dashboards — hover any metric card and click the bell. The rule inherits the card's metric, aggregation and filters, and the threshold is suggested from the card's own last hour.
- Synthetics (User Experience → Endpoints) — + Alert on any test. The suggestion knows the test type: ping/DNS/HTTP latency, route RTT, or low bandwidth throughput — scoped to the target, with the evaluation window automatically sized to the test's cadence (a six-hour test must not be judged in a five-minute window).
- AI → GPU Fleet — + Alert on any GPU card, scoped to that host and GPU.
Every suggested number is stated in the rule's description along with the reasoning, and everything is editable before saving. The suggestion aims to be the correct answer, nearly — never a mystery.
2 — The rule editor
New Alert Rule opens an editor that reads as four questions: Identity (name, category, signal type), What to watch (a categorized metric picker — CPU, Memory, Disk, SNMP, Flows, Synthetics, Databases…), When to fire (thresholds, below), and Who to tell (routing, re-notify, escalation — and the test button).
3 — Ntuition decides for itself
Anomalies, correlated incidents and forecast breaches are not rules you write — the Ntuition Engine detects them from its own analysis. Its alerting is configured on Settings → Ntuition Engine → Alerting: one card per insight type with an enable switch, a sensitivity slider, a destination picker, and a test button. You configure which insights page a human and who hears about it; the engine decides when.
How severity is determined
A rule carries either a single threshold with a fixed severity, or — the recommended model — tiered severities: up to three configurable lines on one rule.
Tiered severities (editor → When to fire → "Tiered severities"): set any of Info at / Warning at / Critical at. The alert's severity is the deepest line the value has crossed, in the rule's direction — rising rules put critical above warning (CPU ≥ 80 / 90 / 95%); falling rules put it below (throughput < 80 / 60 / 40% of normal). Leave any line blank to skip that band. While bands are set, the fixed severity is ignored.
- Crossing bands notifies immediately. An incident that escalates from warning into critical pages again at the moment of crossing — not at the next re-notify interval. De-escalation notifies too, labeled as such.
- Band ordering is validated when you save — a configuration that cannot mean what it says is refused with a plain-English reason.
- One-click alerts from cards arrive with all three bands pre-filled (capacity 80/90/95%, latency 1.2×/1.5×/2× the recent worst, throughput 80/60/40% of the recent best).
Who gets told — routing
Settings → Alert Rules → Notifications tab holds routing policies: named destinations ("Network Ops — Teams", "On-call — email + push") listing the channels that receive alerts. Rules point at a policy via the Notify picker; editing the policy retargets every rule using it. Direct per-rule channels can be added on top in the editor.
The rule that notifies nobody is the failure mode to hunt. It happens when no policy is assigned, or an assigned policy was deleted, or a policy's integration is disabled. The platform surfaces it everywhere — the header count, red card borders, "— nobody —" pickers, and no_destination rows in the delivery log — but only a human can decide who should be told.
Proving it works
- Test from inside the rule — every rule's editor (and every Ntuition alerting card) has Send test notification. It pushes a test through the rule's real route: policy resolution, integrations, channel fan-out. If the test reaches Slack, the real alert will too. Outcomes are reported inline.
- The delivery log — the Alerts page lists every delivery attempt with its outcome:
sent,failed(with the error),silenced,no_destination. "Did anyone get notified last night?" is answered by looking, not guessing. - Last-delivery chips — every rule card wears what happened the last time it tried to reach a human.
Recommended after any deployment or routing change: walk the rule cards and click Test on each. Green chips across the board is the finished state.
Silences
Maintenance windows live on the Silences tab. A silenced alert still fires and records history — only the notification is muted, and the delivery log records silenced so the suppression is visible evidence, not a mystery. Test notifications deliberately bypass silences (the point is proving the route).
Troubleshooting
| Symptom | Meaning / action |
|---|---|
| Header shows "N enabled with no destination" | Those rules fire and tell no one — assign a policy in each red card's Notify picker (saves immediately) |
Delivery log shows no_destination |
The rule fired into the void at that moment — same fix as above; the row names the rule |
Delivery log shows failed |
The channel was tried and errored — the row carries the error (bad webhook URL, expired credential, unreachable integration) |
| Test says sent but nothing arrived | The integration accepted it — look downstream (channel routing, muted chats, spam filters) |
| Rule never fires though the condition looks met | Check the engine line is green; check the rule is enabled; for sparse/synthetic series confirm the evaluation window covers at least two test runs (card-created synthetic alerts size this automatically) |
| Alert fires constantly / flaps | Raise the threshold or add a hold duration ("for 5 min"); for tiered rules, widen the info band so early noise stays info |
| Too many pages for one incident | Use a correlated-incident alert (Ntuition Engine → Alerting) — one page for the cascade with the likely root cause, instead of one per symptom |
| Silenced window ended but no alerts | Silences mute notifications only — check the delivery log for silenced rows during the window; firing resumed notifications the moment the silence lapsed |