Skip to content

Two separate VarianceDetector instances track edge measurements; only one feeds offloading #73

Description

@ff225

Priority: Medium

Edge inference measurements are recorded into two different VarianceDetector instances, and only one of them influences offloading decisions.

The two detectors

  1. Edge.variance_detector (edge_initialization.py:31) — a module-level instance handed to ModelManager (edge_initialization.py:142,309) and fed per layer from model_manager.py:226.
  2. The per-device detector owned by DeviceStateManager (device_state.py, since Review and reduce class-level mutable state in RequestHandler #11) — fed from RequestHandler after inference and read via variance_stats(device_id) to build OffloadingContext.variance_stats.

Only #2 reaches the offloading algorithm. #1 accumulates measurements, logs "variance detected" warnings, and is otherwise never consulted for a decision — should_retest_offloading() is not called on it.

Why it matters

  • Duplicated bookkeeping on the hot path for measurements that are then discarded.
  • Misleading warnings: Edge.variance_detector logs re-test warnings that trigger no re-test.
  • Edge.variance_detector is process-global, so it mixes devices — the exact problem the per-device split in Review and reduce class-level mutable state in RequestHandler #11 fixed for the other detector.

Questions to settle

  • Is Edge.variance_detector meant to drive something that was never wired up, or is it leftover?
  • If the edge-side detector is redundant, ModelManager should stop taking a variance_detector and RequestHandler should remain the only writer.

Files

  • src/server/edge/edge_initialization.py:31,142,309
  • src/server/models/model_manager.py:225-226,280,291
  • src/server/communication/device_state.py

Found while refactoring RequestHandler for #11 / #45. Related to #72 (the same model_manager code path double-applies the EMA).

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions