Skip to content

#268: download refuses instead of returning None — C-79's untreated sibling - #270

Merged
Polichinel merged 2 commits into
developmentfrom
fix/268-download-refuses-instead-of-failing-open
Aug 14, 2026
Merged

#268: download refuses instead of returning None — C-79's untreated sibling#270
Polichinel merged 2 commits into
developmentfrom
fix/268-download-refuses-instead-of-failing-open

Conversation

@Polichinel

Copy link
Copy Markdown
Collaborator

views-crafdapi found this in our code on the first un_crafd delivery attempt, and it is C-79 with the method name changed.

The defect

self._dsm.download_prediction(file_id).to_dict().get("data", {}).get("file_bytes", None)

When data is present and null, the {} default never applies — the key exists — so the next .get raises AttributeError: 'NoneType' object has no attribute 'get'. It fires from inside a dict comprehension over pinned ids in TargetLease.load, three frames from the port, naming neither the file_id nor the fact that a download had failed. views-crafdapi spent an evening ruling out an OOM kill that turned out to be a different pid three minutes later.

The method was byte-identical in both partners, so the FAO leg carries it too. It has simply not fired there yet.

Why nothing caught it

tests/test_store_port.py was written for C-79, with five parametrised tests across both partners — one of them named test_an_unrecognised_result_is_refused_rather_than_assumed_good. It mentioned download zero times.

And contract/store_metadata.py already writes .get("data", {}) or {} — the guard this lacked, one file away, never applied. C-79's own resolution note had been the specification all along: "an unrecognised result should be refused and named, not adapted to silently." It just never got applied a second time.

The fix

Refuse anything that is not non-empty bytes, naming the file_id, that a download failed, and the types actually received. Empty bytes are refused with the rest — no shard, sidecar or manifest is ever zero-length, so b"" is a failed download wearing a valid type, and returning it only moves the same crash to the parser. This protects all three call sites, not just the one that crashed; two of them are manifest reads.

Mutation-proven on three: restoring the original one-liner fails 18 of the module's tests; accepting empty bytes fails exactly 2 (one per partner); dropping the file_id from the message fails exactly 2.

It also moved, because a guard said so

The refusal pushed managers/ to 469 lines against epic #148's 450 bound. That budget's own instruction is to move something out rather than raise the number, so _ContractStorePort — which is not the manager — is now {partner}/store_port.py. 388 lines, 62 of headroom.

The port stopped naming DatastoreModule in its constructor on the way out. A DIP seam whose stated purpose is that nothing downstream sees the client's types should not name one — and a new module mentioning views_pipeline_core would have widened C-40's blast radius past the two files test_views_pipeline_core_is_confined_to_the_partner_managers pins. The contract is the four methods, and it now says so.

Registered

Verification

ruff clean; 446 passed, 1 skipped, 39 xfailed.

Closes #268.

Polichinel and others added 2 commits August 14, 2026 22:58
…ves out of the manager

views-crafdapi found this in our code, on the first un_crafd delivery attempt, and it
is C-79 with the method name changed.

THE DEFECT. `download` chained .get() onto an unvalidated store result:

    self._dsm.download_prediction(file_id).to_dict().get("data", {}).get("file_bytes", None)

When `data` is PRESENT and null the {} default never applies, so the next .get raised
AttributeError from inside a dict comprehension over pinned ids in TargetLease.load —
three frames from the port, naming neither the file_id nor the fact that a download had
failed. They spent an evening ruling out an OOM kill that turned out to be a different
pid three minutes later.

The method was byte-identical in both partners, so THE FAO LEG CARRIES IT TOO. It has
simply not fired there yet.

WHY NOTHING CAUGHT IT. tests/test_store_port.py was written for C-79 with five
parametrised tests across both partners, one of them literally named
test_an_unrecognised_result_is_refused_rather_than_assumed_good. It mentioned `download`
zero times. And contract/store_metadata.py already wrote `.get("data", {}) or {}` — the
guard this lacked, one file away, never applied. C-79's own resolution note was the
specification the whole time: "an unrecognised result should be refused and named, not
adapted to silently."

THE FIX. Refuse anything that is not non-empty bytes, naming the file_id, that a
DOWNLOAD failed, and the types actually received. Empty bytes are refused with the rest:
no shard, sidecar or manifest is ever zero-length, so b"" is a failed download wearing a
valid type, and returning it only moves the same crash to the parser. Kept byte-identical
across partners, as C-79 chose for upload (C-33). This protects all three call sites, not
only the one that crashed — two of them are manifest reads.

Mutation-proven on three: restoring the original one-liner fails 18 of the module's
tests, accepting empty bytes fails exactly 2 (one per partner), dropping the file_id from
the message fails exactly 2.

IT ALSO MOVED, because a guard said so. The refusal pushed managers/ to 469 lines against
epic #148's 450 bound, and that budget's instruction is to move something out rather than
raise the number. _ContractStorePort is not the manager, so it is now
{partner}/store_port.py — 388 lines, 62 of headroom. Two identical files, per the standing
per-partner-track decision.

The port stopped naming DatastoreModule in its constructor on the way out. A DIP seam
whose stated purpose is that nothing downstream sees the client's types should not name
one; and a new module mentioning views_pipeline_core would have widened C-40's blast
radius past the two files test_views_pipeline_core_is_confined_to_the_partner_managers
pins. The contract is the four methods, and it is now stated as such.

REGISTERED. C-99 for the defect, resolved here. C-100 for what reading the whole port
turned up: `file_metadata` has no caller in the package — latest_file_id runs three
times, download three times, upload from the sink, and the fourth method is reachable
only from a test, with contract/store_metadata.py dead behind it. Not deleted here; that
is a decision belonging with the second store (#97), not with a download bug.

Suite 446 passed / 1 skipped / 39 xfailed, ruff clean.

Closes #268.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…o satisfy

Five reviewers on the download fix. The download refusal itself survived — one agent
traced the pipeline-core SDK and confirmed the realistic failures (yanked file, expired
key, rate limit, network blip) all arrive as success=False rather than as exceptions,
that nothing on the path returns memoryview, that bytes(b) is a no-op passthrough on the
common path, and that no artifact on the contract path is legitimately zero-length. What
did not survive was how I described the part around it.

THE MOVE DODGED THE BUDGET RATHER THAN SATISFYING IT. I reported "388 lines, 62 of
headroom" as compliance. Re-measured: managers/ fell 441 -> 388 while each partner
package grew 441 -> 488. The guard counts managers/, and store_port.py is a sibling OF
managers/, so 47 lines left the budget's view rather than the codebase. The budget's own
docstring had already named this exact failure — "an 800-line helper module beside a
406-line manager was previously unbudgeted, which is the same regrowth wearing a
different filename" — and closed it one level in. I evaded it one level out.

This is C-98 again: a guard that watches a proxy reports on the proxy, and the number it
prints is true and irrelevant. test_the_partner_package_stays_within_its_line_budget now
bounds the whole partner package at 700 (measured today: unfao 626, crafd 635), and is
mutation-proven by dropping a 200-line module beside the manager and watching it fire.
The extraction still stands — a store adapter is not the manager.

C-100 SAID "Measured" AND THEN DID NOT MEASURE ONE OF THE FOUR. It gave exact counts for
latest_file_id and download and wrote "upload is called by the sink". upload has THREE
call sites: contract/wire/sink.py:164 and each partner's historical artifact at
managers/<partner>.py:325. Corrected, with the sites named.

C-33's EXTRACTION TRIGGER HAS FIRED, TWICE, AND NOBODY SAID SO. Its trigger is "a third
in-repo partner package, or the first bug that must be hand-patched identically in both
manager files". C-79 was that bug; C-99 is the same fault in the sibling method, patched
by hand in both again. The decision is still to duplicate, but the reason is now
different and is recorded: the shape the two incidents showed is a result-check, not the
store-identity DeliveryProfile this entry proposes extracting, which would have prevented
neither.

What did change is that the duplication is now held mechanically —
test_the_two_partners_ports_have_not_drifted. And the docstring is precise about what
that does NOT buy: it would not have caught C-79 or C-99, because both files stayed
byte-identical while carrying the defect in the untreated method. It closes the
partner-vs-partner axis, which was never the axis that bit.

FOUR STALE LOCATIONS, from moving a class four documents point at. C-40 cited the port at
unfao.py:37-78 in two places, C-15/C-24 at unfao.py:37-64, and the register header still
said Last Updated 2026-08-12. The one doc that WAS updated in the first commit —
contract/store_metadata.py — showed the fix pattern was known and applied one file over,
which is ADR-014 §1's failure mode exactly.

Suite 449 passed / 1 skipped / 39 xfailed, ruff clean.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@Polichinel
Polichinel merged commit ff0278b into development Aug 14, 2026
4 checks passed
@Polichinel
Polichinel deleted the fix/268-download-refuses-instead-of-failing-open branch August 14, 2026 21:13
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant