Skip to content

Run the AI image editor's Bloom-side tests on the nightly (Notion test cases 816, 817) - #8355

Draft
hatton wants to merge 1 commit into
masterfrom
ai-image-editor-e2e
Draft

hatton wants to merge 1 commit into
masterfrom
ai-image-editor-e2e

Conversation

@hatton

@hatton hatton commented Sep 14, 2026

Copy link
Copy Markdown
Member

[Claude Opus 5 from Hatton's machine during add-e2e-test]

Bloom production code changes

None. The diff is src/BloomE2E/**, .github/**, Markdown, and the deletion of two
Playwright *.uitest.ts files. No Bloom production code changed; Devin not run.

Purpose

The "Edit with AI…" integration had two Playwright files that nothing in CI ran. They
attached over CDP to whatever Bloom the developer happened to have open, needed a book
already selected, and wrote into that book, so the coverage they describe was protecting
nothing. This moves them into src/BloomE2E, whose fixture launches its own Bloom on a
temp copy of a collection, and which the nightly already runs as a selectable suite.

What they cover is the part the Vitest suite cannot reach. aiImageEditorOverlay.test.ts
exercises Bloom's half of the postMessage protocol against a stub, so it passes just as
happily when browser/aiImageEditor is missing entirely, which is how the editor once
stopped opening without any test noticing.

Neither card existed before. Test case 805, "AI Image Editing (basic tests)", stays
entirely manual: it covers the UI and real AI generation, and none of its steps is
automated here, so it was not split.

Steps

  • Select A5 Portrait in the basic collection, the one precondition the launch has.
  • Ask Bloom to open an AI image editor session, and load the editor app in an iframe the
    way the overlay does.
  • Write, read back, and delete a history sidecar through Bloom's file endpoint.
  • Write a PNG and read it back.
  • Present no session token, then a wrong one.
  • Try four file names that are outside the endpoint's allow-list.

How it verifies the result

The boot test waits for the editor app's own ready handshake on the
bloom-ai-image-tools channel, which the app posts only after its HTML and JS have
loaded and its host component has mounted. The file tests read the HTTP status Bloom
answers with, plus the body for the round-trip and the content-type for the image, and
expect 401 for a missing or wrong session and 400 for a refused name.

I checked the boot test is not falsely passing: with output/browser/aiImageEditor
renamed away it fails with Bloom's own message, launch returned 503. Body: The AI Image Editor is not included in this build of Bloom.

Five tests, green three runs in a row, about six seconds after Bloom starts. No
Bloom.exe from the worktree survived any run.

Problems

The UI path, right-clicking an image and choosing "Edit with AI…", is still not covered
and has no journey test; Notion 805 runs it by hand. Loading the iframe directly, as the
boot test does, deliberately stops short of that: it does not register the overlay's own
postMessage handler, so the commit and current-page save path is not exercised.

Also in this PR: a papercut recording that .github/skills/add-e2e-test/SKILL.md still
says the BloomE2E suite is not wired into the nightly, when nightly.yml has run it as a
selectable suite for a while. That claim sent me planning workflow changes that already
existed.

🤖 Generated with Claude Code

Devin review


This change is Reviewable

The "Edit with AI..." integration had two Playwright files that could only be run by
hand. They attached over CDP to whatever Bloom the developer happened to have open,
needed a book already selected, and wrote into that book. Nothing in CI ran them, so
the coverage they describe was protecting nothing.

They move into src/BloomE2E, whose fixture launches its own Bloom on a temp copy of a
collection, and which the nightly already runs as a selectable suite. The tests now
select their own book, so they depend on no developer state and touch no real book.

What they cover is the part the Vitest suite cannot reach. aiImageEditorOverlay.test.ts
exercises Bloom's half of the postMessage protocol against a stub, so it passes just as
happily when browser/aiImageEditor is missing entirely, which is how the editor once
stopped opening without any test noticing. With that folder renamed away, the boot test
now fails with Bloom's own "not included in this build" message.

Per the add-e2e-test skill, the launch and file-endpoint knowledge lives in a new
helpers/aiImageEditor.ts, so the tests themselves carry no API paths.

Notion test cases 816 and 817.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant