Anomaly Detection

Overview

Anomaly Detection continuously monitors your project's key metrics for statistically significant deviations. When a metric moves far enough from its historical average — measured using z-scores — the system flags it as an anomaly and can trigger alerts.

The detector is completeness-aware: it knows the difference between "your visibility dropped" and "the worker only captured 12 out of 105 expected snapshots that day." Partial-capture days are excluded entirely, so sparse data can never trigger false alerts.

Monitored Metrics

The system analyzes three categories of metrics in a single pass:

Visibility Score Anomalies

Detects unusual spikes or drops in your daily visibility scores. A sudden drop could indicate that an AI model has changed how it references your brand, while a spike might signal a successful optimization or a temporary ranking artifact.

Sentiment Shift Anomalies

Aggregates sentiment scores by day (not per-snapshot, which would double-count) and watches for sudden moves — especially toward negative — that signal AI models are presenting your brand in a less favorable light.

Brand Mention Rate Anomalies

Tracks the daily rate at which your brand is mentioned across snapshots. A declining mention rate means AI models are responding to industry prompts without referencing your brand, which is a direct visibility concern.

Data Quality Safeguards

This is what separates a production-grade detector from a noisy one. Every run applies four safeguards before any z-score is computed.

Completeness Gate

Each day's snapshot count is compared to the expected count for your project:

expected = active_prompts × active_models

A day is analyzed only if its capture count meets both thresholds:

• ≥ 80% of expected (the completeness ratio)
• ≥ 10 snapshots absolute (to prevent tiny projects from running detection on statistically meaningless samples)

Example: A project with 21 prompts × 5 models expects 105 snapshots/day. Days with fewer than 84 snapshots are skipped. A day with 15 snapshots is clearly a partial capture — it gets excluded, not flagged.

Today Is Excluded

Today's date is always skipped. Today's captures are still running — treating a half-complete day as a full day would guarantee false drops at 9 AM.

Bonferroni Correction

Three metrics are tested simultaneously (visibility, sentiment, mention rate). Without correction, the chance of at least one false positive per run is ~14%, not 5%. The detector applies a Bonferroni correction, dividing the alpha by the number of tests so the family-wise false-positive rate stays near 5%.

Persistence Check

Every flagged anomaly is marked with a persistent flag. An anomaly is persistent when the previous day's point is also anomalous — a single spike is noise; two in a row is signal. Use the persistent flag to filter out one-off events in your alert rules.

How Detection Works

Z-Score Analysis

For each metric, the system computes the mean and standard deviation over the lookback period (default: 30 days, excluding partial-capture days). The z-score for each recent data point is:

z = (actual_value - mean) / standard_deviation

A z-score of 2.0 means the value is two standard deviations from the mean. The default threshold is 2.0, automatically raised by Bonferroni correction for multiple simultaneous tests.

Requirements

Before any metric is tested:

• ≥ 7 clean data points in the lookback window (after completeness filtering)
• Standard deviation > 0 (there must be actual variation, not a flat series)
• Only the 3 most recent data points are checked on each run

Severity Levels

Anomaly Types

Segment Filtering

Anomaly detection can run for any prompt segment:

• All prompts (default)
• Branded — Spot shifts in how your brand is referenced when explicitly asked.
• Non-Branded — Spot shifts in organic discoverability (the most sensitive segment).
• Competitor — Spot shifts in head-to-head positioning.

Pass the ?segment= parameter to the API or use the segment toggle on the analytics page.

Alert Integration

Detected anomalies are stored in the anomaly_events table with full context:

1. The event is recorded with expected value, actual value, z-score, persistent flag, and detection date.
2. If alert rules are configured for anomaly events, notifications fire via email or webhooks.
3. Anomalies remain in the active queue until acknowledged.
4. New runs deduplicate against existing unacknowledged events — you don't get alerted twice for the same anomaly.

Configuration

How to Use

1. Navigate to Analytics → Anomalies, or use the MCP tool detect_anomalies.
2. Focus on persistent anomalies first — these have the strongest signal.
3. Check severity: Critical ≥ 3.0 σ is truly unusual; investigate immediately.
4. Correlate with recent changes — deploys, content launches, competitor activity, LLM model updates.
5. Acknowledge anomalies once investigated to clear them.
6. Configure alert rules for automatic notifications.

Plan Requirements

Anomaly Detection is available on Pro-SME and above.