Skip to content

Handle empty anomaly lists in the point metrics - #672

Open
DMZ22 wants to merge 1 commit into
sintel-dev:masterfrom
DMZ22:handle-empty-point-metrics
Open

Handle empty anomaly lists in the point metrics#672
DMZ22 wants to merge 1 commit into
sintel-dev:masterfrom
DMZ22:handle-empty-point-metrics

Conversation

@DMZ22

@DMZ22 DMZ22 commented Jul 24, 2026

Copy link
Copy Markdown

Closes #527.

Problem

_point_partition derives the partition range with min()/max() over the union of the expected and observed anomalies, before the start/end overrides are applied:

edge_start = min(expected.union(observed))
if start is not None:
    edge_start = start

When neither side has any anomalies that union is empty, so every point metric raises ValueError: min() arg is an empty sequence — and because the min() runs first, supplying a range through data/start/end doesn't help either:

>>> point_confusion_matrix([], [])
ValueError: min() arg is an empty sequence
>>> point_confusion_matrix([], [], start=0, end=9)   # a range is known, still raises
ValueError: min() arg is an empty sequence

Fix

Take start/end into account first and only fall back to min()/max() when they aren't given, so a known range is enough to build the partition. If there are no anomalies and no range to fall back on, return an empty confusion matrix as you specified on the issue — there are no true or false positives and no false negatives, but the number of true negatives is unknown. That is the same (None, fp, fn, tp) shape contextual.py already returns when the true negatives can't be determined, so it keeps the 4-tuple contract the metrics unpack.

Resulting behaviour

call before after
point_confusion_matrix([], []) ValueError (None, 0, 0, 0)
point_confusion_matrix([], [], start=0, end=9) ValueError [10, 0, 0, 0] (all true negatives)
point_precision/recall/f1_score([], []) ValueError nan, via the existing zero-division handling
point_accuracy([], []) ValueError: min() arg… the existing "cannot obtain accuracy" error, since the true negatives are unknown
point_accuracy([], [], start=0, end=9) ValueError 1.0

Everything with at least one anomaly on either side is untouched — point_confusion_matrix([1,2,3], [2,3,4]) still returns [0, 1, 1, 2], and [2, 1, 1, 2] with start=0, end=5.

Tests

Added test_point_confusion_matrix_empty, test_point_confusion_matrix_empty_with_range and test_point_scores_empty; all three fail on master and pass here. tests/unit/evaluation passes (36 tests), and flake8/isort are clean on the changed files.

_point_partition derived the partition range with min()/max() over the union
of the expected and observed anomalies, before the start/end overrides were
applied. With no anomalies on either side that union is empty, so every point
metric raised "ValueError: min() arg is an empty sequence" - including when a
range was supplied through data/start/end.

Take start/end into account first and only fall back to min()/max() when they
are not given, so a known range is enough to build the partition. When there
are no anomalies and no range to fall back on, return an empty confusion
matrix instead: there are no true or false positives and no false negatives,
but the number of true negatives is unknown, which is the same
(None, fp, fn, tp) shape the contextual metrics already use.

Precision, recall and f1 then return nan through the existing zero-division
handling, and accuracy raises the existing error for a missing true-negative
count unless a range makes it computable.

Closes sintel-dev#527.
@CLAassistant

CLAassistant commented Jul 24, 2026

Copy link
Copy Markdown

CLA assistant check
All committers have signed the CLA.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

point_confusion_matrix when both ground truth and predicted anomalies lists are empty.

2 participants