Skip to content

feat: Add "correlate" rule type to deduplication rules#6568

Open
ausias-armesto wants to merge 9 commits into
keephq:mainfrom
ausias-armesto:feat/deduplication-type-split
Open

feat: Add "correlate" rule type to deduplication rules#6568
ausias-armesto wants to merge 9 commits into
keephq:mainfrom
ausias-armesto:feat/deduplication-type-split

Conversation

@ausias-armesto

@ausias-armesto ausias-armesto commented Jun 9, 2026

Copy link
Copy Markdown
Contributor

Summary

Closes #6567

Extends the deduplication rule model with a rule_type field (split | correlate) to support correlating related alerts across different fingerprints — without merging them or losing per-alert detail.

What changed

  • AlertDeduplicationRule gains a rule_type field:
Value Behavior
split Existing behavior. Computes the alert's primary fingerprint from fingerprint_fields. All existing rules default to this.
correlate New. Computes a secondary correlation_fingerprint from fingerprint_fields without touching the primary fingerprint.

Each provider can have one rule of each type simultaneously.

  • New fields on AlertDto
Field Type Description
correlation_fingerprint str | None Hash computed by the correlate rule. Set on every alert that matches the rule.
is_correlated bool True when another active (non-resolved, non-suppressed) alert with the same correlation_fingerprint already exists. The first alert in a group is False; subsequent ones are True.
correlated_to str | None Fingerprint of the representative (first) alert in the correlation group. None for the representative itself.

Pipeline

incoming alert
      │
      ▼
format_alert (base_provider.py)
  ├─ split rule?  → recompute alert.fingerprint
  └─ correlate rule? → compute alert.correlation_fingerprint
      │
      ▼
apply_deduplication (alert_deduplicator.py)
  ├─ split check: is fingerprint already in LastAlert? → isFullDuplicate / isPartialDuplicate
  └─ correlate check: is correlation_fingerprint already in LastAlert?
       → alert.is_correlated = True
       → alert.correlated_to = representative fingerprint
      │
      ▼
save to DB
  └─ set_last_alert stores correlation_fingerprint on LastAlert row

Files changed

  • DB models: rule_type on deduplication rules; correlation_fingerprint (indexed) on last alerts
  • Alert DTO: new correlation_fingerprint, is_correlated, and correlated_to fields
  • Deduplication pipeline: correlate rule applied in format step; correlation check sets is_correlated/correlated_to before save
  • DB helpers: lookup and persistence of correlation_fingerprint
  • Migrations: rule_type column (existing rules default to split); correlation_fingerprint column on last alerts
  • Frontend: rule_type support with a split/correlate selector and conflict detection

Migration notes

All existing custom deduplication rules are migrated to rule_type="split" — this is correct because the original purpose of the feature was exclusively to control fingerprint computation (alert identity).

Action required after deploying: Review your existing custom rules. If any rule was created with the intent of correlating related alerts (e.g., grouping pod instances), change its rule_type to "correlate" via the UI.

Usage example

Goal: The same alert fires on multiple instances simultaneously. Each instance alert should be stored individually, but only one notification should fire per group.

Split rule (controls identity — keep existing):

  • fingerprint_fields: [fingerprint, startsAt]
  • Type: Split

Correlate rule (new):

  • fingerprint_fields: [alertname, service]
  • Type: Correlate

Workflow condition (only notify for the representative alert):

filters:
  - key: is_correlated
    value: "false"

@dosubot dosubot Bot added size:L This PR changes 100-499 lines, ignoring generated files. Enhancement New feature or request labels Jun 9, 2026
@codecov

codecov Bot commented Jul 8, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 42.30769% with 30 lines in your changes missing coverage. Please review.
✅ Project coverage is 46.33%. Comparing base (968a762) to head (9a06a26).
⚠️ Report is 14 commits behind head on main.

Files with missing lines Patch % Lines
keep/api/tasks/process_event_task.py 21.73% 18 Missing ⚠️
keep/api/core/db.py 56.25% 7 Missing ⚠️
keep/api/alert_deduplicator/alert_deduplicator.py 16.66% 5 Missing ⚠️
Additional details and impacted files
@@             Coverage Diff             @@
##             main    #6568       +/-   ##
===========================================
+ Coverage   30.51%   46.33%   +15.81%     
===========================================
  Files         101      178       +77     
  Lines       11729    18741     +7012     
===========================================
+ Hits         3579     8683     +5104     
- Misses       8150    10058     +1908     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Enhancement New feature or request size:L This PR changes 100-499 lines, ignoring generated files.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[➕ Feature]: Add "correlate" rule type to deduplication rules

1 participant