fix(PER-8519): initialize percy_screenshot_url before try to stop exception masking - #226
Draft
pranavz28 wants to merge 1 commit into
Draft
fix(PER-8519): initialize percy_screenshot_url before try to stop exception masking#226pranavz28 wants to merge 1 commit into
pranavz28 wants to merge 1 commit into
Conversation
…eption masking
When super().screenshot() raised, the except block referenced
percy_screenshot_url before it was ever assigned, raising UnboundLocalError
from inside the handler. That replaced the original exception and skipped the
execute_percy_screenshot_end('failure') notification entirely, leaving the
BrowserStack session stuck in the 'begin' state and reporting the wrong error
to /percy/events.
Initializing the variable before the try makes the original exception
propagate and the failure notification fire with an empty screenshot URL.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What this fixes
PER-8519 / F-006 —
UnboundLocalErrormasks the real exception inAppAutomate.screenshotpercy/providers/app_automate.pyassignedpercy_screenshot_urlinside thetry, on the lineafter
super().screenshot(), but referenced it in theexceptblock:So whenever
super().screenshot()itself raised, the handler touched an unbound local and blew upinside the handler. Reproduced on
masterbefore the change:Two consequences, not one:
UnboundLocalErrorinstead of the realcapture failure.
percy_screenshot()forwardsstr(e)to/percy/eventsviapost_failed_event, so our own telemetry recorded "cannot access local variable'percy_screenshot_url'" rather than the actual fault — the error text is useless for triage.
'failure'notification is never sent.execute_percy_screenshot_endwas never called(
screenshot_end calls: []), so the BrowserStack session was left dangling in thebeginstatewith no terminal status. This is the more damaging half, and the reason the fix is not merely
"silence the name error".
The fix initializes
percy_screenshot_url = ''before thetry. After the change:The original exception propagates, and the session is properly closed out as
failurewith thereal message. An empty string is the right default: it is the same value the success path uses when
the response carries no
link(response.get('link', '')), so the executor payload shape isunchanged and the
percyScreenshotUrlfield stays a string.Why this is safe. It is a strictly-widening change on a path that previously always crashed.
The success path is untouched (the assignment inside the
trystill overwrites the default). Theonly behaviour that changes is a path that could only ever produce
UnboundLocalErrorplus adangling session. It improves error visibility rather than changing failure policy — which serves
part of PER-8522's intent without any breaking change.
Regression test added:
test_screenshot_propagates_capture_error_and_reports_failureasserts theoriginal
ValueErrorpropagates (notUnboundLocalError) and that the failure notification firesexactly once with
('name', '', 'failure', None, 'capture failed'). The existingtest_screenshot_reraises_after_failure_notificationonly covered the case where the notificationfails after a successful capture, which is why this bug survived the 100% coverage gate — the line
was covered, the unbound path was not.
What was deliberately NOT done
The ticket's nominated chain-breaker — flipping
PercyOptions.ignore_errorstoFalse— is not in this PRPER-8535 nominates flipping the
ignore_errorsdefault fromTruetoFalseas the chain-breaker.That should not be done as a unilateral change in this SDK, for four reasons.
1. Fail-open is a cross-SDK convention, implemented five independent times.
percy/lib/percy_options.py:15,22Truepercy/driver/driverWrapper.js:83Truepercy/lib/percy_options.rb:24,32TruePercyOptions.java:37TruePercy/AppPercy.cs:11TrueFive independent implementations agreeing is a deliberate product decision, not five copies of one
oversight. Flipping Python alone makes it the sole outlier and breaks the same-capability
same-behaviour contract that polyglot suites depend on: the identical
percy:optionscapabilitywould mean "swallow" in four SDKs and "fail the test" in the fifth.
2. The option is completely undocumented.
ignoreErrorsdoes not appear in any of the six SDKREADMEs. Customers cannot have knowingly opted into a behaviour they were never told exists, so a
flip would change behaviour under users who have no documented way to know the knob is there — and
no documentation to find when their suite starts failing.
3. The blast radius is every screenshot call.
percy_screenshot()is the sole public entrypoint of this SDK. Flipping the default converts every currently-silent Percy failure into a
test-suite failure. A Percy outage, a CLI version mismatch, or a transient healthcheck blip would
start failing customers' unrelated functional tests — visual-testing infrastructure taking down
functional CI is a much worse failure mode than a missed snapshot, and it is exactly what fail-open
was chosen to prevent.
4. There is a pre-existing opt-out bug that must be fixed first.
percy/lib/percy_options.py:13short-circuits before the legacy-capability fallback:
When
percy:optionsis present and non-empty, the legacypercy.ignoreErrorscapability issilently ignored. Verified against this branch:
Ruby has no such early return, so this is a Python-only divergence. Flipping the default while this
bug exists means the customers most likely to want the old behaviour — those on the legacy
capability — would find their opt-out does not take effect. They would have no working escape
hatch from the new failure mode.
Recommendation
Take the default flip as a cross-SDK product decision with a deprecation path, not an SDK-local
bugfix:
ignoreErrorsin all six SDK READMEs first, so the opt-out is discoverable.percy_options.py:13opt-out bug sopercy.ignoreErrorsis honoured alongsidepercy:options, and confirm the other four SDKs honour their legacy capability too.being swallowed, telling users the default will change and how to pin the current behaviour.
contract survives.
PER-8535 therefore cannot be closed by this PR. This PR lands the safe, independently-correct
subset only. The chain-breaker it nominates remains open and needs the product decision above.
PER-8527 / F-014 — the TOCTOU race on class-level globals: investigated, not changed
I investigated the shared mutable state (
percy/environment.py:7-9, written incli_wrapper.py:26-28inside an
@lru_cached static method, read atscreenshot.py:11) and could not reproduce thedescribed failure mode, because it is not reachable as stated. A 40-thread concurrent probe:
The
lru_cachefirst-call race is real — the healthcheck body ran twice for 40 threads, confirminglru_cacheis not atomic. Butsession_type=Nonecannot be observed afteris_percy_enabled()returns
True, for three structural reasons:lru_cacheonly publishes after the return. Every reader thatreceives
Trueis reading values that were already written.PERCY_CLI_APIis a module-level constant, soevery execution in a process talks to the same CLI and writes the same build id / url / type. A
"clobber" is a no-op. (Separate processes — e.g. pytest-xdist — have separate memory entirely.)
Noneover a good value.if not data['success']: raiseprecedes theassignments, and the
exceptreturnsFalsewithout touching the globals, so a failing ortransient healthcheck can never null out previously-good state.
The one genuine residue is the duplicated healthcheck HTTP request under a first-call race — benign
redundancy, not state corruption or wrong-provider selection.
I chose not to apply the ticket's preferred fix (returning the
(enabled, session_type, build_id, build_url)tuple) because the cost/benefit is inverted here:is_percy_enabled's return type fromboolto a tuple, which breaks theif not CLIWrapper.is_percy_enabled():truthiness check at the call site and the three existingtest patches that stub it as
MagicMock(return_value=True).screenshot.py:11). The other two(
app_automate.py:81-82and:132, which needpercy_build_id/percy_build_url) would stillread the globals unless the values are threaded through
AppPercy.__init__→ProviderResolver.resolve→AppAutomate.__init__→provider.screenshot()— four signature changes across the provider class surface.
That is a broad change to the SDK's constructor surface, plus a partial fix that leaves the shared
mutable state in place, in exchange for a failure mode I can show is unreachable. A clean small PR
is worth more here. If we want the
lru_cacheduplicate-healthcheck tidied up, that is aself-contained follow-up in
cli_wrapper.pyalone and should be its own ticket.Testing
make coverage(venv on Python 3.12): 165 tests, OK, 100% coverage /fail_under = 100satisfied —
percy/providers/app_automate.pyat 100%, TOTAL 778 stmts / 0 missed. The newbranch is covered, so the repo's coverage gate stays green.
pylint percy/providers/app_automate.py→ 10.00/10.make lintlocally crashes with anastroid.AstroidErroron anyappium-importing file. This is pre-existing and environmental — it reproduces identically on
an unmodified
mastertree (verified viagit stash), and is a local Python 3.12 artifact(
.python-versionpins 3.10; CI lints on 3.11). Not introduced by this change.🤖 Generated with Claude Code