Skip to content

hack: let the kind node pull images in parallel [CI flake fix] - #1318

Open
Benjamin Elder (BenTheElder) wants to merge 1 commit into
agent-substrate:mainfrom
BenTheElder:kind-parallel-image-pulls
Open

hack: let the kind node pull images in parallel [CI flake fix]#1318
Benjamin Elder (BenTheElder) wants to merge 1 commit into
agent-substrate:mainfrom
BenTheElder:kind-parallel-image-pulls

Conversation

@BenTheElder

Copy link
Copy Markdown
Collaborator

Installing the control plane pulls roughly 570MB of third-party images (postgres, prometheus, the otel collector, rustfs, envoy, jaeger) onto a single node.

kubelet serializes image pulls by default, so they queue behind one another and whichever workload lands at the back of the queue can miss its readiness deadline.

e2e has failed repeatedly with postgres still in PodInitializing after 60s, its image not yet pulled, while everything else was already running.

Sample failure: https://github.com/agent-substrate/substrate/actions/runs/33258860897/job/99117220741

Note: This is after PR merge, and it's clearly unrelated to the PR.

AI-assisted.

Installing the control plane pulls roughly 570MB of third-party images
(postgres, prometheus, the otel collector, rustfs, envoy, jaeger) onto a
single node. kubelet serializes image pulls by default, so they queue
behind one another and whichever workload lands at the back of the queue
can miss its readiness deadline: e2e has failed with postgres still in
PodInitializing after 60s, its image not yet pulled, while everything
else was already running.
@BenTheElder Benjamin Elder (BenTheElder) added kind/cleanup Small fixes that are not bugs, for example a typo in a code comment area/tests Enhancing / fixing test coverage. area/dev-infra labels Aug 29, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area/dev-infra area/tests Enhancing / fixing test coverage. kind/cleanup Small fixes that are not bugs, for example a typo in a code comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants