Skip to content

fix(ci): prevent base image manifest races - #662

Merged
Randy808 merged 2 commits into
Blockstream:masterfrom
DeviaVir:fix/ci-base-manifest-race
Aug 25, 2026
Merged

Randy808 merged 2 commits into
Blockstream:masterfrom
DeviaVir:fix/ci-base-manifest-race

Conversation

@DeviaVir

@DeviaVir DeviaVir commented Aug 25, 2026

Copy link
Copy Markdown
Contributor

Summary

  • keep verification-only base images local to each CI job and switch Compose to the default dockerd builder
  • publish official per-architecture base images only from default-branch push pipelines
  • make master image builds wait for a rebuilt multi-arch base manifest when one is required
  • allow a manual (web-run) default-branch pipeline to republish the bases and the multi-arch manifest

Root cause

Both verification paths built a single platform but pushed it to the same blockstream/esplora-base:latest tag. The parallel jobs raced, and the last writer left a single-platform index. A later production build for the other architecture then failed with no match for platform in manifest.

docker buildx build --load now imports each verification result into the isolated Docker-in-Docker engine. Selecting the default builder afterward lets the subsequent Compose build resolve that local image without mutating the registry. Base publication moves to default-branch pushes; when base inputs or their CI flow change, the architecture builds are recomposed before the master images start.

Including gitlab/build.yml in the publication change set makes this merge rebuild both current bases and repair the shared manifest.

Manual repair path

Branch pipelines evaluate the CI configuration of their own ref. A push to a stale pre-fix branch can therefore still replace the shared manifest with a single-platform image. A web-run pipeline on the default branch now rebuilds both bases and re-creates the manifest, so recovery is one click instead of a dummy commit.

Validation

  • git diff --check
  • parsed all GitLab CI YAML with yq
  • GitLab CI lint on the merged local configuration (re-run after the web-rule addition: valid, no errors)
  • isolated Docker reproduction: the container builder could not resolve a dockerd-local base, then the default-builder switch let Compose consume it and build successfully
  • confirmed against the failed master pipeline that the amd64 production build fails with no match for platform in manifest, and that mirror updates create push-source pipelines, which the new rules match

The current GitHub-to-GitLab trigger evaluates CI YAML from mirrored master and only clones the PR as build input, so its reported architecture checks do not execute this CI configuration diff.

@DeviaVir
DeviaVir force-pushed the fix/ci-base-manifest-race branch from 99cc8a3 to 9a426d4 Compare August 25, 2026 08:06
@DeviaVir
DeviaVir force-pushed the fix/ci-base-manifest-race branch from 9a426d4 to d9270e0 Compare August 25, 2026 08:44
@DeviaVir DeviaVir changed the title fix(ci): keep verification base images local fix(ci): prevent base image manifest races Aug 25, 2026
A web-run pipeline on the default branch now republishes both
architecture bases and the multi-arch manifest. This restores a
one-click repair path when a stale branch pipeline running the old
CI configuration replaces the shared manifest again.
@DeviaVir
DeviaVir requested a review from Randy808 August 25, 2026 08:57
@Randy808
Randy808 merged commit e04dd89 into Blockstream:master Aug 25, 2026
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants