test(serwalker): unit tests for stream, tags, header and string clusters - #86
Open
caverav wants to merge 2 commits into
Open
test(serwalker): unit tests for stream, tags, header and string clusters#86caverav wants to merge 2 commits into
caverav wants to merge 2 commits into
Conversation
The branch has not built since f50d581. Four in-flight stubs were blocking: - Stream lost its 'a lifetime parameter when read_bytes_zero_copy landed, so the struct, the impl block and two signatures no longer resolved - walk_snapshot_and_enrich_model used an uninitialised binding, applied ? in a function returning ProgramModel, and called a symbol that is not exported yet - DataSnapshot referenced ProgramRoots, which is not defined - resolve_root_library returned an uninitialised binding Unfinished bodies are now todo!() so they typecheck without pretending to work. The lifetime is restored as it was.
…d strings 31 unit tests over the four pieces every cluster sits on. Each encodes its own input the way Dart's WriteStream does, so the tests assert against the format rather than against our decoder. stream (14): LEB128 round trips for both markers, the Read vs ReadUnsigned distinction that produced the cluster-tag and kind_tag bugs, big-endian ref ids and their five-stage cap, read_byte staying raw so byte-sized fields cannot be LEB decoded, Latin-1 over the full 0..=255 range, and bounds checks on every reader. utils (4): ClassIdTag at bit 12, CanonicalBit at 1, ImmutableBit at 7, proven independent so an off-by-one in any shift fails, plus unknown cids erroring instead of defaulting to IllegalCid. snapshot (7): header round trip, the 32-char version/features split, magic and kind rejection, kind numbering with no kModule, and counts decoding with the unsigned marker. cluster (6): string cluster round trips for one-byte, two-byte and mixed clusters, Latin-1 payloads, unpaired surrogates decoding lossily, and a truncated payload erroring. Verified by mutation: 15 deliberate regressions were introduced one at a time (markers swapped, ref ids made little endian, Latin-1 reverted to from_utf8, bounds checks removed, every tag shift moved by one, the version split shortened, from_utf16_lossy made strict) and all 15 were caught. DECLARE_*_LENGTH_CLUSTER now derives Default on the generated struct so a cluster can be built in a test without going through decide_cluster.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What this is
Unit tests for the four layers every cluster sits on: the stream readers, tag
decoding, the snapshot header, and the string cluster. 31 tests, no new
dependencies.
Two commits, kept separate on purpose so the first can be dropped once you
finish the scaffolding.
1.
fix(serwalker): make the crate compile againHeads up: the branch has not built since
f50d581, so nine commits back. Icould not run tests against a crate that does not compile, so this commit does
the minimum to get it building. Four things were in the way:
Streamlost its'alifetime parameter whenread_bytes_zero_copylanded.The struct, the impl block and two signatures stopped resolving. Restored as
it was.
walk_snapshot_and_enrich_modelused an uninitialised binding, applied?ina function returning
ProgramModelrather thanResult, and calledenrich_model_object_pool, which is not exported yet.DataSnapshotreferencedProgramRoots, which is not defined anywhere.resolve_root_libraryreturned an uninitialised binding.The unfinished bodies are
todo!()now so they typecheck without pretending towork. Nothing about the intent changed. Drop this commit whenever those are
filled in for real.
2.
test(serwalker): cover the stream primitives, tag decoding, header and stringsEach suite encodes its own input the way Dart's
WriteStreamdoes, so the testsassert against the format rather than against our own decoder. A test that just
re-runs the parser would pass with the parser wrong.
streamReadvsReadUnsigneddistinction, big endian ref ids and the five stage cap,read_bytestaying raw, Latin-1 over the full 0..=255 range, bounds checks on every readerutilsClassIdTagat bit 12,CanonicalBitat 1,ImmutableBitat 7, proven independent of each other, plus unknown cids erroringsnapshotkModule, counts decoding with the unsigned markerclusterThree of them pin bugs this PR already hit, so they should stay even if the
surrounding code moves:
the_two_markers_are_not_interchangeable,read_byte_is_raw_not_leb128,ref_ids_are_big_endian_not_leb128.Proof they would catch a regression
Passing tests on their own do not show much, so I mutation tested them. Fifteen
deliberate regressions, introduced one at a time with the source restored
between each:
signed_reads_sign_extendref_ids_are_big_endian_not_leb128+128ref id correction droppedread_ref_id_round_tripsref_id_is_bounded_to_five_stagesfrom_utf8latin1_accepts_the_high_halftaketruncated_input_errors_instead_of_panickingread_byteturned into a LEB readthe_flag_bits_do_not_overlapsplits_version_from_features_at_thirty_two_charsnum_objectsread with the signed markerheader_counts_use_the_unsigned_markerrejects_a_bad_magicsnapshot_kind_numbering_matches_dartone_byte_strings_carry_latin1_payloadsfrom_utf16_lossymade strictunpaired_surrogates_do_not_panicFifteen out of fifteen. One test was weak on the first pass:
ref_id_is_bounded_to_five_stagesoriginally used six truncated bytes, so itwas really testing truncation and survived the cap being removed. It now uses
eight continuation bytes followed by a valid terminator, so only the cap can
reject it.
One source change outside the tests
DECLARE_FIXED_LENGTH_CLUSTERandDECLARE_VARIABLE_LENGTH_CLUSTERnow deriveDefaulton the generated struct. Without it a cluster can only be builtthrough
decide_cluster, which still returnsErrfor nearly everything, so_StringClusterwas untestable. Two lines, no behaviour change.Verifying locally
Current state on this branch: fmt clean, builds, 31 passed 0 failed.
Not included
Clippy still reports 18 findings, none of them in the test code. They are the
unfinished scaffolding: unused imports in
info_producer,std::env::var,std::default,anyhow::Error, and fiveu8 as u8casts left over from theread_byteconversion. Left alone so this PR stays about tests.