Skip to content

OR-280 Nudge new installs to run the demo experiment - #338

Merged
myles332 merged 4 commits into
mainfrom
myles/demo-first-experiment-nudge
Sep 15, 2026
Merged

myles332 merged 4 commits into
mainfrom
myles/demo-first-experiment-nudge

Conversation

@myles332

@myles332 myles332 commented Sep 15, 2026

Copy link
Copy Markdown
Contributor

Summary

Sep 9–12 install analytics showed the largest funnel leak between "onboarding complete" and "first message sent" (43.8%). The demo welcome pushed users to read the recorded sessions rather than send anything. This PR makes the first message the obvious next step.

  • Welcome modal primary button is now Try the demo experiment; it hands focus to the composer, which already holds the run-experiment prompt. The telemetry choice value stays explore_demo.
  • The prefilled prompt stays on offer until the user has sent anything in the demo. A send inside a recorded session appends to it rather than creating a new one, so the check compares each session's activeLeafId against its seeded leaf (DEMO_SEEDED_LEAF_IDS) instead of only looking at session ids.
  • A dismissible hint above the composer ("This prompt runs a small 200-step training probe on your machine. Hit enter to get started.") with a ring on the send button while the prefill is untouched. Textarea is linked via aria-describedby; dismiss returns focus to the composer.
  • Once a demo run is running, a second dismissible hint appears: "The experiment is now running. Tap [Logs] to monitor it." The Logs button opens that run's logs. It seeds from live run events and from the runs baseline, so it survives a reload mid-run, clears when the run leaves the running state, and is announced via a polite live region.
  • First demo visit opens only the experiments tab in the right pane (the overview artifact tab is no longer pre-opened).
  • Default right-pane width for installs with no saved layout is now a quarter of the window, capped at 480px (was 40%/760px). The projects home used to write the old 760 into the saved layout before the demo opened, so the default now lives in ui/src/panelLayout.ts and both callers use it. Existing installs keep their saved width.
  • Locale catalogs (en, zh-CN, fa, ar) updated, including the Arabic locale that landed on main mid-review; tour_explore_demo renamed to tour_run_demo_experiment everywhere. ui/dist rebuilt on the merged tree.

Not changed: the demoOverviewLeading logic in App.tsx that pins the overview artifact first when present. Nothing opens that tab automatically anymore, so it only applies to layouts saved by older builds or a manually opened overview. Left for a separate cleanup.

Test plan

  • node ui/scripts/check-i18n.mjs, node ui/scripts/check-styles.mjs, UI typecheck, ui/tests (162 pass)
  • cargo fmt --check, cargo clippy --all-targets -D warnings, cargo build --locked, cargo test --locked (830 pass) on the merged tree
  • Fresh dev slot: onboarding → demo lands with hint, prefill, and experiments-only pane; sending the prompt withdraws the prefill; the running hint appears once the probe starts and its Logs button opens the run
  • Fresh install: confirm the right pane opens at ~25% width on the first demo landing (not 760px)
  • Dismiss either hint with keyboard only; focus should land in the composer
  • Reload the page while the probe is running: the running hint should return
  • Leave the demo and return after sending: the prompt hint must not reappear
  • Dark theme: both hint bubbles and the send-button ring legible
  • Arabic or Persian locale: the "Tap [Logs] to monitor it" sentence reads correctly right-to-left

🤖 Generated with Claude Code

Analytics for Sep 9-12 installs showed the largest funnel leak between
"onboarding complete" and "first message sent" (43.8%). The demo welcome
steered users to read the recorded sessions rather than send anything.

- Rename the welcome's primary button to "Run the demo experiment" and hand
  focus to the composer, which already holds the run-experiment prompt.
- Keep that prefill on offer until the user has sent anything in the demo,
  detected by a recorded session's active leaf moving off its seeded message
  or a new session appearing, instead of only until the welcome closes.
- Show a dismissible hint above the composer explaining the 200-step probe,
  with a ring on the send button while the prefilled prompt is untouched.
- Open only the experiments tab on the first demo visit.
- Default the right pane to a quarter of the window (max 480px) on installs
  with no saved layout, and share that default with the projects home so it
  is not overwritten before the demo opens.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
@myles332 myles332 changed the title Nudge new installs to run the demo experiment OR-280 Nudge new installs to run the demo experiment Sep 15, 2026
myles332 and others added 3 commits September 15, 2026 11:31
- Relabel the welcome button "Try the demo experiment": the click only
  focuses the prefilled prompt, so "Run" over-promised.
- Style the composer hint as a neutral surface bubble instead of a tint.
- When a demo run starts, show a second dismissible hint above the composer
  with a Logs button that opens that run's logs; it clears when the run
  leaves the running state.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…riment-nudge

# Conflicts:
#	ui/dist/assets/index-BJLMzvdf.css
#	ui/dist/assets/index-BJUUwp49.js
#	ui/dist/assets/index-CQbtnwiZ.js
#	ui/dist/assets/index-mNCWAddR.js
#	ui/dist/index.html
- Seed the running-run hint from the runs baseline so it survives a reload
  or a return to the demo project mid-run.
- Announce the hint to assistive tech via a polite live region and return
  focus to the composer on dismiss, matching the prompt hint.
- Share the hint bubble classes, drop the redundant Logs tooltip key, and
  use the idiomatic Arabic phrasing for the tap instruction.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
@myles332
myles332 merged commit b38c813 into main Sep 15, 2026
11 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant