Sync docs and Claude skills with v0.4.0

[aboucaud]

v0.4.0 shipped the JupyterHub-on-GCP backend (#163) and the MyST report scaffold (#157) without touching claude/lightcone/ or most of docs/. This brings both back on par with the code.

Everything here was verified behaviorally — scaffolding throwaway projects, running generated Snakefiles, and checking real --help output — rather than by reading code alone.

Skills: instructions that actively broke agent runs

These are pre-existing bugs, not v0.4.0 fallout, and several would cause silent wrong behavior rather than an error:

• figure-comparison and lc-from-paper/references/compare.md looked for results in the wrong place. Both searched for flat files at results/<universe>/<output>.<ext>. Outputs actually materialize as directories containing the artifact plus .lightcone-manifest.json (confirmed by running a generated Snakefile). Every comparison would have failed to locate its file and reported a mismatch.
• figure-comparison read the wrong index. It keyed off work/reference/metadata.json, a Docling intermediate that only exists on the PDF fallback path — on the primary arXiv path it is never created. The canonical index is index.json.
• references/run.md invented a status vocabulary and a log file. It told loop iterations to reason about pending/no_recipe (real states: ok/stale/missing/alias, and it omitted stale — the one that matters after a spec edit) and to monitor a per-output .log that is never written.
• references/architect.md's stub astra.yaml failed the astra validate gate the same file imposes. Replaced with a version iterated through astra validate until clean.
• references/implement.md documented a --universe auto-injection convention that does not exist — a recipe using it raises Unknown recipe placeholder.
• lc-feedback always reported the ASTRA version as n/a, because astra.__version__ doesn't exist on a namespace package. Now uses astra --version.
• session-start.sh still advertised the removed ASTRA narrative field in every session's primer.

Skills also gained v0.4.0 coverage: the kubernetes runtime and Cloud Build path, Dask Gateway detection and the one-image-per-run constraint, and the MyST report scaffold.

Docs

• Documented what v0.4.0 added: report scaffold, Gateway branch, pre-flight image builds, COPY/ADD source hashing (a rerun-triggering behavior change), worker_image, and the new failure modes in troubleshooting.
• Corrected pages that contradicted the code: site_registry.md described a now-central module as orphaned; runner.md was a "(removed)" tombstone for the module that forms the body of every generated rule; architecture.md claimed .lightcone/lightcone.yaml is never read (it is).
• Fixed lc eval install instructions, wrong in three places — pip install lightcone-cli[eval] doesn't work, since eval is a dependency-group, not an extra. Added a docs/cli/eval.md page.
• Fixed docs/cli/init.md, which was wrong about its own flags and about the permission tiers (/scratch and /pscratch are ask, not deny).
• New docs/api/scratch.md and docs/api/wrroc.md; contributing/{backends,hpc-sites}.md corrected and linked into nav.
• Deleted the orphaned docs/hpc/ section: unreachable from nav, redundant with user/cluster.md, and it told readers to run lc setup, a removed command.
• Adds CHANGELOG.md.

Verification

389 tests pass, ruff clean, just docs reports no issues, every nav entry resolves, and the scaffold tree documented in init.md matches a fresh lc init exactly.

Known issues found but deliberately not fixed

Code-level, out of scope for a docs/skills sync — worth separate issues:

1. claude/lightcone/templates/CLAUDE.md is dead. lc init writes an inline _PROJECT_CLAUDE_MD; the template is still copied to .claude/templates/, so projects carry two divergent versions.
2. The scaffolded index.md is coupled to the boilerplate ids example_method/main_result. Every authoring skill reshapes the spec and breaks those references, and lc-new is forbidden from fixing it by its own allowed-tools. Skills now warn the user instead.
3. justfile eval recipes are broken (lc eval run takes a positional config path, has no --skill flag, and --extra eval targets a dependency-group).
4. extract_path_b passes arxiv_id=None, doi=None when writing the stub astra.yaml, so --doi is silently dropped on the PDF path. Looks like a real bug.
5. lc run fails on ARM Macs — snakemake's ILP scheduler shells out to pulp's bundled x86-64 CBC binary (Bad CPU type in executable). --scheduler greedy works.
6. Five docs/skills/*.md pages document an "Allowed tools" section for skills that declare no allowed-tools frontmatter — fabricated, never drifted. Needs a decision either way.
7. manifest.SCHEMA_VERSION is still 1 despite worker_image and git_remote being added.
8. No test coverage for the plugin bundle — nothing would have caught any of the above.

One judgment call worth review

review.md now asks the user to run /check-sentence-by-sentence rather than invoking it, because that skill's description states other skills must not invoke it. Relaxing the skill's description instead would be an equally valid resolution.

🤖 Generated with Claude Code

[github-actions]

❌ Eval Results

Metric	Value
Score	0.00
Build complete	❌
Cost	$0.00
Turns	0
Duration	0s
lightcone-cli	0.4.1.dev3+geb5dfeb36 (eb5dfeb3)
Results	Download

Graders

No grader results

Full output

      ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/runner/work/lightcone-cli/lightcone-cli/.venv/lib/python3.12/site-packages/daytona_api_client/exceptions.py", line 148, in from_response
    raise ForbiddenException(http_resp=http_resp, body=body, data=data)
daytona_api_client.exceptions.ForbiddenException: (403)
Reason: Forbidden
HTTP response headers: HTTPHeaderDict({'Content-Type': 'application/json; charset=utf-8', 'Content-Length': '155', 'Connection': 'keep-alive', 'Date': 'Wed, 29 Jul 2026 16:47:13 GMT', 'ETag': 'W/"9b-V/Xx4rzMDKb4+HQZfsZxuOJaFW8"', 'X-Powered-By': 'Express', 'Vary': 'Origin', 'X-Daytona-Api-Version': 'v0.202.0', 'X-RateLimit-Limit-sandbox-create': '600', 'X-RateLimit-Remaining-sandbox-create': '599', 'X-RateLimit-Reset-sandbox-create': '60', 'X-Cache': 'Error from cloudfront', 'Via': '1.1 e7c1fac5920ba451129493421444f012.cloudfront.net (CloudFront)', 'X-Amz-Cf-Pop': 'IAD61-P5', 'X-Amz-Cf-Id': 'O8LfzK6FZHg5ILXf0HFxK6_bjZKW123JlpToBb6gwpStrKJHADrt0w==', 'X-Frame-Options': 'DENY', 'Referrer-Policy': 'strict-origin-when-cross-origin', 'Content-Security-Policy': "frame-ancestors 'none'", 'X-Content-Type-Options': 'nosniff', 'Strict-Transport-Security': 'max-age=31536000; includeSubDomains; preload'})
HTTP response body: {"path":"/api/sandbox","timestamp":"2026-07-29T16:47:13.182Z","statusCode":403,"error":"Forbidden","message":"Organization is suspended: Depleted credits"}


The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/runner/work/lightcone-cli/lightcone-cli/src/lightcone/eval/harness.py", line 112, in run_trial
    sandbox.create()
  File "/home/runner/work/lightcone-cli/lightcone-cli/src/lightcone/eval/sandbox.py", line 149, in create
    self._sandbox = self._daytona.create(
                    ^^^^^^^^^^^^^^^^^^^^^
  File "/home/runner/work/lightcone-cli/lightcone-cli/.venv/lib/python3.12/site-packages/daytona_sdk/_utils/errors.py", line 224, in sync_wrapper
    process_n_raise_exception(e)
  File "/home/runner/work/lightcone-cli/lightcone-cli/.venv/lib/python3.12/site-packages/daytona_sdk/_utils/errors.py", line 156, in process_n_raise_exception
    raise create_daytona_error(
daytona_sdk.common.errors.DaytonaForbiddenError: Failed to create sandbox: Organization is suspended: Depleted credits
  snae trial 0: score=0.00 error: Failed to create sandbox: Organization is 
suspended: Deplete

lightcone-cli: 0.4.1.dev3+geb5dfeb36 (HEAD eb5dfeb3)
ASTRA: 0.2.11

  Eval Results: Scores  
┏━━━━━━┳━━━━━━━━━━━━━━━┓
┃ Task ┃     Score     ┃
┡━━━━━━╇━━━━━━━━━━━━━━━┩
│ snae │ 0.00 +/- 0.00 │
│      │  pass@k: 0%   │
│      │   1 errors    │
└──────┴───────────────┘

   Eval Results: Cost &   
         Duration         
┏━━━━━━┳━━━━━━━━━━━━━━━━━┓
┃ Task ┃ Cost / Duration ┃
┡━━━━━━╇━━━━━━━━━━━━━━━━━┩
│ snae │      $0.00      │
│      │       0s        │
└──────┴─────────────────┘

Total: 1 trials, $0.00, 0s

Results saved to: eval-results/build-eb5dfeb3/results.json

[cloudflare-workers-and-pages]

Deploying with    Cloudflare Workers

The latest updates on your project. Learn more about integrating Git with Workers.

Status	Name	Latest Commit	Preview URL	Updated (UTC)
✅ Deployment successful! View logs	lightcone-cli	3d64746	Commit Preview URL Branch Preview URL	Jul 29 2026, 04:48 PM

[aboucaud]

@cailmdaley what's your view on this ?

[cailmdaley]

this is also said just above on L43, i think it's over-repetitive but this whole skill needs to be reworked so i think it's fine for now

[cailmdaley]

i think this is an improvement on the existing state, although i'm still concerned that it's not quite up to our standard of quality. some nits, the final one is the most major

[cailmdaley]

not sure if this is critical info

[cailmdaley]

this is also said just above on L43, i think it's over-repetitive but this whole skill needs to be reworked so i think it's fine for now

[cailmdaley]

this is a band-aid, but probably we don't have time to rework the extractor agent. so probably fine for now.

[cailmdaley]

this is not going to be a pleasant experience for users, getting a bug report when they start a new project

[aboucaud]

well noted, will remove this