feat: a probe fixture that reaches the dynamic branch - #53
Merged
Conversation
Phase 7.3. One fixture added to blueprints/thousandeyes/probe.plan.json, and
it is the step that makes every conditional finding observable rather than
hand-established.
The pilot's two fixtures use objectType `test` and `dashboard`. Both are static
object types, so no fixture ever disagreed with another on the axis that
decides anything -- which is why the conditional structure was invisible to
every probe and had to be found by hand.
Every field in the new fixture is load-bearing, and each is one of the
findings:
objectType: endpoint-agent requires type=dynamic; static is refused
type: dynamic omitting it defaults to static and then fails
matchType: and a dynamic tag refuses without it
filters: [scope=custom] a dynamic tag refuses without it, and custom is
the scope whose key contract the document states
correctly
Cost from `probe -list`, before and after: 147 -> 174 requests, 47 -> 56
creates. Inside the plan's existing budget of 200 and 60, so no cap change --
the earlier 62-create overflow came from the nested enumerations, not from
this. The increase lands almost entirely in write.required, 16 -> 30, which is
exactly the probe whose disagreements 7.1 turns into conditional facts.
Verified rather than assumed: the fixture's own body was sent to the sandbox
and returned 201, with matchType coming back as "and" where a static tag
returns null. That is the conditional returnedOnRead finding confirmed from the
other side, and it is what the re-record will observe. The object was swept and
the tenant left with nothing.
Worth verifying because a refused fixture wastes a whole recording: the
requiredness probe reports "fixture X was refused with 400, so no omission from
it can be attributed" and moves on, so the run would complete and produce
nothing from the branch it exists to reach.
One coverage consequence, stated rather than discovered. Scope.fixtureKeys is
deliberately cross-fixture, so setting matchType and filters here removes them
from the omitted set and the server-default protocol stops probing them. That
is the design working as documented -- a valid dynamic body cannot omit either,
so claiming to know a valid value for them is simply true -- and their omission
behaviour is covered by write.required, which is where it belongs.
Cassette untouched: probe -mode verify still reproduces all 38 facts, because
the plan change only takes effect on a recording run.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Phase 7.3. One fixture, and it is the step that makes every conditional finding observable rather than hand-established.
Why one fixture is the whole change
The pilot's two fixtures use
objectType: testandobjectType: dashboard. Both are static object types, so no fixture ever disagreed with another on the axis that decides anything — which is precisely why the conditional structure was invisible to every probe and had to be found by hand with curl.Every field in the new fixture is load-bearing, and each one is one of the findings:
objectType: endpoint-agenttype: dynamic; static is refusedtype: dynamicmatchType: andfilters: [scope: custom]customis the scope whose key contract the document states correctlyCost, from
probe -listrather than from a failureInside the existing budget, so no cap change. The earlier 62-create overflow came from the nested enumerations, not from this — worth separating, because the plan predicted a cap raise here and it turns out not to be needed.
The increase lands almost entirely in
write.required(16 → 30 requests), which is exactly the probe whose disagreements 7.1 turns into conditional facts.Verified, not assumed
A refused fixture would waste the whole recording: the requiredness probe reports "fixture X was refused with 400, so no omission from it can be attributed" and moves on, so the run completes and produces nothing from the branch it exists to reach.
So the fixture's own body was sent to the sandbox: 201, and the response returns
matchTypecomes back as"and"where a static tag returnsnull. That is the conditionalreturnedOnReadfinding confirmed from the other side — and it is what the re-record will now be able to observe. The object was swept; the tenant has nothing left.One coverage consequence, stated rather than discovered
Scope.fixtureKeysis deliberately cross-fixture — "a field one fixture sets is a field the operator has told us a valid value for, so it is not a candidate for what does the server do when this is omitted". SettingmatchTypeandfiltershere therefore removes them from the omitted set, and the server-default protocol stops probing them.That is the design working as documented: a valid dynamic body cannot omit either, so claiming to know a valid value for them is simply true. Their omission behaviour is covered by
write.required, which is where it belongs.Cassette untouched
probe -mode verifystill reproduces all 38 facts, because a plan change only takes effect on a recording run. Which is the next step, and the one thing I cannot do alone — see below.Needs you: the sandbox profile
The re-record needs a profile with
sandbox: trueand writtensandboxEvidence. I have deliberately not written it. The gate's own comment is that "Profile.Sandbox is what an operator wrote in a file" — it is the single guard between this toolkit and a production tenant, and self-certifying it would empty it of meaning.docs/examples/probe-profile.example.jsonhas the shape. It also needsassertions.accountGroupId, which is a tenant identifier and is why the file is gitignored.🤖 Generated with Claude Code