Myst yml metadata fallback - #13
Open
agahkarakuzu wants to merge 12 commits into
Open
Conversation
added 12 commits
August 2, 2026 22:13
…iations
An unquoted `date: 2024-01-15` in myst.yml -- the MyST-canonical form -- is
parsed by yaml.safe_load into a datetime.date, copied into the paper metadata,
and handed to a Celery task whose payload is serialized as JSON. That raised
inside a Flask route with no handler: HTTP 500, no GitHub comment, no buckets.
Map a non-string date to its string form.
A front-matter key that is present but empty (None, "", [], {}) now counts as
absent, for both the scalar fills and the authors/affiliations pair check, so a
paper.md that merely lists the keys no longer defeats the fallback.
A bare string is accepted where MyST accepts one: an author entry becomes a
named author with no affiliations, an affiliation entry becomes an affiliation
named after the string with no id, and either still holds its index position --
matching inara's Lua filter, so a PDF and a Zenodo record cannot credit
different institutions. A scalar `affiliations:` value is one entry rather than
one per character.
In the deposit task, drop the dead affiliation_mapping line that indexed
data['affiliations'] unguarded, and pass data.get('affiliations') or [] so
authors-without-affiliations metadata resolves every author to None instead of
raising KeyError. An affiliation index the list does not define now logs a
warning naming the author rather than silently recording no institution.
Claude-Session: https://claude.ai/code/session_01YFCLUc1iAr5jjjuUTMMEB8
Uncalled, and its 3001-3099 range never covered the content server's 3100-3200. Claude-Session: https://claude.ai/code/session_016qxMAGM5tpB5okf7hJJgJW
Hooks celeryd_after_setup so it runs once per node, before tasks are consumed. Claude-Session: https://claude.ai/code/session_016qxMAGM5tpB5okf7hJJgJW
Re-adds the DOCKER-USER rule after docker.service on every boot; idempotent so restarts cannot stack duplicates. Claude-Session: https://claude.ai/code/session_016qxMAGM5tpB5okf7hJJgJW
An exception in builder.cleanup() aborted the whole finally block, leaving the container running and the repo locked until the 6000s timeout. Claude-Session: https://claude.ai/code/session_016qxMAGM5tpB5okf7hJJgJW
The myst.yml fallback made author metadata richer than the deposit path
can use. A myst.yml author routinely carries email, github, twitter, url
and corresponding; Zenodo's legacy schema allows a creator only name,
affiliation, orcid and gnd. The sanitizer in zenodo_create_buckets_task
popped corresponding and equal-contrib by name but had no case for
email, and its substring repair could not invent one -- no valid field
name is a substring of "email" -- so it reached metadata.creators. That
risks a validation error on the deposit, and publishes nine author email
addresses on a public record if it does not.
So the deposit boundary now decides what a creator is, in zenodo_metadata,
applied inside zenodo_create_bucket rather than in its caller so every
deposit path is covered. It keeps the typo repair the old loop did, and
scans an ordered tuple to do it: the loop iterated a set, so a key
matching more than one field mapped differently between runs.
Also closes the gaps around it, all of which turn a bad submission into
an error nobody sees:
- api_zenodo_post checked only that metadata named authors, while the
task also requires a title. Missing it raised KeyError inside Celery.
- first_affiliations subscripted affiliation["index"]/["name"], so one
malformed entry in a hand-written paper.md failed the whole deposit.
A bare string author raised AttributeError for the same reason.
- Replacing the front matter's author list with myst.yml's is correct
-- an affiliation index only means something relative to its list --
but silent, so a stale myst.yml outranking a current paper.md was
indistinguishable from a working fallback. It logs now.
- The pending comment was passed the author list where a task id goes.
Verified against the live haplante/oct-t1-paper, whose paper.md declares
only a title and puts its authors in myst.yml: 9 authors in, 9 creators
out, carrying name, orcid and affiliation alone.
Claude-Session: https://claude.ai/code/session_016Tcpjta8yB3my2RLeH5tty
zenodo_upload_docker_task could not find an image that was sitting in the
registry. It asked for binder-roboneurolibre-2doct-2dt1-2dpaper-f65dae;
the repository is registry.evidencepub.io/binder-roboneurolibre-2doct-2dt1
-2dpaper-f65dae. The registry host doubles as the repository namespace --
the "registry url entered twice" the docker_save call below already notes
-- and bh_project_name is the REES config that prepends it.
This task was the only REES call site that omitted it;
preview_build_myst_task passes it and resolves its images fine. The name
was otherwise correct: f65dae is sha256("roboneurolibre-oct-t1-paper")[:6]
and the -2d encoding matches, so the lookup 404'd on a path that was one
segment short rather than on an absent image. The catalog fallback missed
it for the same reason, its pattern being built from the same config.
found_image_name now carries the namespace, which is what the image_name
below expects: prepending the host to it yields host/namespace/repo:tag,
the same reference myst-libre's pull_image builds.
Also report the failure to GitHub. Since myst-libre 0.4.1 a missing image
raises ImageNotFoundError from REES.__init__ rather than returning False
from the search, so the one branch that called task.fail became
unreachable and the error escaped as a bare traceback -- Celery recorded
the failure and the issue comment sat orange indefinitely, saying nothing.
That is why this looked like a hang rather than a 404.
Claude-Session: https://claude.ai/code/session_016Tcpjta8yB3my2RLeH5tty
With the namespace fixed the image resolved, and the next line raised AttributeError: 'REES' object has no attribute 'search_img_by_repo_name'. That method is on DockerRegistryClient. It was reachable through REES in an earlier myst-libre; in 0.4.1 discovery moved into REES.__init__, so the call is both wrong and redundant -- reaching it means the constructor already found the image, because it raises when it does not. Also catch anything else the block can raise. An AttributeError is not a MystLibreError, so the guard added with the namespace fix did not cover it, and the issue comment went orange and silent again -- the same failure mode one exception class over. This task's only channel to the submitter is that comment, so report the exception class there rather than leaving it in a worker log nobody is watching. Claude-Session: https://claude.ai/code/session_016Tcpjta8yB3my2RLeH5tty
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Make zenodo endpoints use bibliographic metadata from myst.yml when paper.md frontmatter (JOSS style) is missing.
Relates to neurolibre/inara#2.