You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Edge inference measurements are recorded into two differentVarianceDetector instances, and only one of them influences offloading decisions.
The two detectors
Edge.variance_detector (edge_initialization.py:31) — a module-level instance handed to ModelManager (edge_initialization.py:142,309) and fed per layer from model_manager.py:226.
The per-device detector owned by DeviceStateManager (device_state.py, since Review and reduce class-level mutable state in RequestHandler #11) — fed from RequestHandler after inference and read via variance_stats(device_id) to build OffloadingContext.variance_stats.
Only #2 reaches the offloading algorithm. #1 accumulates measurements, logs "variance detected" warnings, and is otherwise never consulted for a decision — should_retest_offloading() is not called on it.
Why it matters
Duplicated bookkeeping on the hot path for measurements that are then discarded.
Misleading warnings: Edge.variance_detector logs re-test warnings that trigger no re-test.
Priority: Medium
Edge inference measurements are recorded into two different
VarianceDetectorinstances, and only one of them influences offloading decisions.The two detectors
Edge.variance_detector(edge_initialization.py:31) — a module-level instance handed toModelManager(edge_initialization.py:142,309) and fed per layer frommodel_manager.py:226.DeviceStateManager(device_state.py, since Review and reduce class-level mutable state in RequestHandler #11) — fed fromRequestHandlerafter inference and read viavariance_stats(device_id)to buildOffloadingContext.variance_stats.Only #2 reaches the offloading algorithm. #1 accumulates measurements, logs "variance detected" warnings, and is otherwise never consulted for a decision —
should_retest_offloading()is not called on it.Why it matters
Edge.variance_detectorlogs re-test warnings that trigger no re-test.Edge.variance_detectoris process-global, so it mixes devices — the exact problem the per-device split in Review and reduce class-level mutable state in RequestHandler #11 fixed for the other detector.Questions to settle
Edge.variance_detectormeant to drive something that was never wired up, or is it leftover?ModelManagershould stop taking avariance_detectorandRequestHandlershould remain the only writer.Files
src/server/edge/edge_initialization.py:31,142,309src/server/models/model_manager.py:225-226,280,291src/server/communication/device_state.pyFound while refactoring
RequestHandlerfor #11 / #45. Related to #72 (the samemodel_managercode path double-applies the EMA).