Skip to content

test(serwalker): unit tests for stream, tags, header and string clusters - #86

Open
caverav wants to merge 2 commits into
flutterdec-dart-serwalkerfrom
test/serwalker-unit-tests
Open

test(serwalker): unit tests for stream, tags, header and string clusters#86
caverav wants to merge 2 commits into
flutterdec-dart-serwalkerfrom
test/serwalker-unit-tests

Conversation

@caverav

@caverav caverav commented Jul 28, 2026

Copy link
Copy Markdown
Owner

What this is

Unit tests for the four layers every cluster sits on: the stream readers, tag
decoding, the snapshot header, and the string cluster. 31 tests, no new
dependencies.

Two commits, kept separate on purpose so the first can be dropped once you
finish the scaffolding.

1. fix(serwalker): make the crate compile again

Heads up: the branch has not built since f50d581, so nine commits back. I
could not run tests against a crate that does not compile, so this commit does
the minimum to get it building. Four things were in the way:

  • Stream lost its 'a lifetime parameter when read_bytes_zero_copy landed.
    The struct, the impl block and two signatures stopped resolving. Restored as
    it was.
  • walk_snapshot_and_enrich_model used an uninitialised binding, applied ? in
    a function returning ProgramModel rather than Result, and called
    enrich_model_object_pool, which is not exported yet.
  • DataSnapshot referenced ProgramRoots, which is not defined anywhere.
  • resolve_root_library returned an uninitialised binding.

The unfinished bodies are todo!() now so they typecheck without pretending to
work. Nothing about the intent changed. Drop this commit whenever those are
filled in for real.

2. test(serwalker): cover the stream primitives, tag decoding, header and strings

Each suite encodes its own input the way Dart's WriteStream does, so the tests
assert against the format rather than against our own decoder. A test that just
re-runs the parser would pass with the parser wrong.

suite tests what they pin
stream 14 LEB128 round trips for both markers, the Read vs ReadUnsigned distinction, big endian ref ids and the five stage cap, read_byte staying raw, Latin-1 over the full 0..=255 range, bounds checks on every reader
utils 4 ClassIdTag at bit 12, CanonicalBit at 1, ImmutableBit at 7, proven independent of each other, plus unknown cids erroring
snapshot 7 header round trip, the 32 char version and features split, magic and kind rejection, kind numbering with no kModule, counts decoding with the unsigned marker
cluster 6 string cluster round trips for one byte, two byte and mixed clusters, Latin-1 payloads, unpaired surrogates decoding lossily, truncated payload erroring

Three of them pin bugs this PR already hit, so they should stay even if the
surrounding code moves: the_two_markers_are_not_interchangeable,
read_byte_is_raw_not_leb128, ref_ids_are_big_endian_not_leb128.

Proof they would catch a regression

Passing tests on their own do not show much, so I mutation tested them. Fifteen
deliberate regressions, introduced one at a time with the source restored
between each:

mutation caught by
the two end byte markers swapped signed_reads_sign_extend
ref ids decoded little endian ref_ids_are_big_endian_not_leb128
the +128 ref id correction dropped read_ref_id_round_trips
five stage ref id cap removed ref_id_is_bounded_to_five_stages
Latin-1 reverted to from_utf8 latin1_accepts_the_high_half
bounds check removed from take truncated_input_errors_instead_of_panicking
read_byte turned into a LEB read compile break
each of the three tag shifts moved by one the_flag_bits_do_not_overlap
version split shortened to 31 splits_version_from_features_at_thirty_two_chars
num_objects read with the signed marker header_counts_use_the_unsigned_marker
magic check disabled rejects_a_bad_magic
a seventh snapshot kind accepted snapshot_kind_numbering_matches_dart
string length shift dropped one_byte_strings_carry_latin1_payloads
from_utf16_lossy made strict unpaired_surrogates_do_not_panic

Fifteen out of fifteen. One test was weak on the first pass:
ref_id_is_bounded_to_five_stages originally used six truncated bytes, so it
was really testing truncation and survived the cap being removed. It now uses
eight continuation bytes followed by a valid terminator, so only the cap can
reject it.

One source change outside the tests

DECLARE_FIXED_LENGTH_CLUSTER and DECLARE_VARIABLE_LENGTH_CLUSTER now derive
Default on the generated struct. Without it a cluster can only be built
through decide_cluster, which still returns Err for nearly everything, so
_StringCluster was untestable. Two lines, no behaviour change.

Verifying locally

cargo test -p flutterdec-serwalker
cargo fmt --all --check

Current state on this branch: fmt clean, builds, 31 passed 0 failed.

Not included

Clippy still reports 18 findings, none of them in the test code. They are the
unfinished scaffolding: unused imports in info_producer, std::env::var,
std::default, anyhow::Error, and five u8 as u8 casts left over from the
read_byte conversion. Left alone so this PR stays about tests.

caverav added 2 commits July 28, 2026 09:57
The branch has not built since f50d581. Four in-flight stubs were blocking:

- Stream lost its 'a lifetime parameter when read_bytes_zero_copy landed,
  so the struct, the impl block and two signatures no longer resolved
- walk_snapshot_and_enrich_model used an uninitialised binding, applied ?
  in a function returning ProgramModel, and called a symbol that is not
  exported yet
- DataSnapshot referenced ProgramRoots, which is not defined
- resolve_root_library returned an uninitialised binding

Unfinished bodies are now todo!() so they typecheck without pretending to
work. The lifetime is restored as it was.
…d strings

31 unit tests over the four pieces every cluster sits on. Each encodes its
own input the way Dart's WriteStream does, so the tests assert against the
format rather than against our decoder.

stream (14): LEB128 round trips for both markers, the Read vs ReadUnsigned
distinction that produced the cluster-tag and kind_tag bugs, big-endian
ref ids and their five-stage cap, read_byte staying raw so byte-sized
fields cannot be LEB decoded, Latin-1 over the full 0..=255 range, and
bounds checks on every reader.

utils (4): ClassIdTag at bit 12, CanonicalBit at 1, ImmutableBit at 7,
proven independent so an off-by-one in any shift fails, plus unknown cids
erroring instead of defaulting to IllegalCid.

snapshot (7): header round trip, the 32-char version/features split, magic
and kind rejection, kind numbering with no kModule, and counts decoding
with the unsigned marker.

cluster (6): string cluster round trips for one-byte, two-byte and mixed
clusters, Latin-1 payloads, unpaired surrogates decoding lossily, and a
truncated payload erroring.

Verified by mutation: 15 deliberate regressions were introduced one at a
time (markers swapped, ref ids made little endian, Latin-1 reverted to
from_utf8, bounds checks removed, every tag shift moved by one, the
version split shortened, from_utf16_lossy made strict) and all 15 were
caught.

DECLARE_*_LENGTH_CLUSTER now derives Default on the generated struct so a
cluster can be built in a test without going through decide_cluster.
@caverav caverav mentioned this pull request Jul 28, 2026
7 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant