fix(ci): unbreak Linux agents (no Docker Hub, emulated tests on 24.04-min)

[Ahmed Muhsin (ahmedmuhsin)]

Fixes for the Linux CI agents, plus a related build-script cleanup. Supersedes #893.

1. Test apps can no longer be packaged via Docker Hub

The Docker integration tests failed on dev every night from 2026-08-15 (builds 298300, 298418, 298540, 298743). All four legs failed at Build and package test apps:

Unable to find image 'ubuntu:22.04' locally
docker: Error response from daemon: Get "https://registry-1.docker.io/v2/":
        context deadline exceeded (Client.Timeout exceeded while awaiting headers)

No code change caused this: dev had not moved since 08-12, and 08-11 through 08-14 were green. The agents can no longer pull from Docker Hub.

build-apps.ps1 pulled ubuntu:22.04 only to get mksquashfs, then ran apt-get install squashfs-tools inside that container — two dependencies on public network access.

Fix: run mksquashfs on the agent and install squashfs-tools in the job, which removes the image pull and the in-container apt-get together. The Docker path stays as a fallback for machines without squashfs-tools (Windows dev boxes) and now points at the MCR mirror. Everything else under dockertests/ already used MCR (azure-storage/azurite, azure-functions/mesh); ubuntu:22.04 was the last Docker Hub reference in the suite.

2. Both Linux jobs move to 1es-ubuntu-24.04-min

1es-ubuntu-22.04 boots ~92% full on / (67G used of 73G), almost all of it toolsets these jobs never touch: Android SDK and Haskell under /usr/local (~24G), /var/lib including cached docker images (~12G), Swift and miniconda under /usr/share (~9.5G), CodeQL and PyPy in the tool cache (~6.7G). With that little headroom dotnet test filled the disk, the agent aborted with exit 134, and ADO reported the job as cancelled rather than failed.

#883 worked around it by rm -rf-ing those toolsets at job start. That held, but it deletes paths out from under a hosted image, so it breaks silently whenever they move — and it cost real time: on the docker legs alone the cleanup step ran 75–119s each, roughly 12% of a 10–16 minute leg.

1es-ubuntu-24.04-min starts at 15% used (62G free) — more headroom than the cleanup produced (34G), with nothing to delete. Both Linux jobs now use it and the cleanup steps are gone from the repo entirely.

The tradeoff is that -min ships very little, so each job installs what it needs:

Gap on -min	Emulated Linux	Docker integration
no Maven	apt-get install -y maven	same
no Node/npm (for npm install -g azurite)	NodeTool@0	not needed
no squashfs-tools (for build-apps.ps1)	not needed	apt-get install -y squashfs-tools
/usr/lib/jvm empty, so JavaToolInstaller PreInstalled resolves nothing	already downloaded its matrix JDK	download the matrix JDK, install with LocalDirectory
no Python in the tool cache, so UsePythonVersion cannot resolve	not needed	system python3 (test kit needs >=3.8), with a venv because 24.04's system interpreter is externally managed (PEP 668)
no mono, so the nuget CLI cannot run	-skipNuget, see below	already used -skipNuget

Anything added to these jobs from now on has to install what it needs.

Note on JDK vendors. The docker legs previously used jdkSourceOption: 'PreInstalled', which resolves JAVA_HOME_<n>_X64 from the agent image — that supplied Temurin for every version, so 11+ had quietly diverged from what we ship. They now download pinned JDKs: Temurin for 8, Microsoft OpenJDK for 11/17/21, matching production and the emulated jobs. Versions come from java-versions.yml, so they no longer drift when the agent image is refreshed.

Plain 1es-ubuntu-24.04 is not an option — it does not exist in the pool (Image 1es-ubuntu-24.04 doesn't exist in pool 1es-pool-azfunc-public).

3. Emulated tests stop building a NuGet package

Both emulated jobs ran package-pipeline.ps1 without -skipNuget, building a .nupkg neither of them reads: setup-tests-pipeline.ps1 copies the worker out of target/, and only build-artifacts.yml publishes packages.

On Linux this became a hard failure, because the nuget CLI needs mono and -min does not ship it. On Windows it was only wasted work. Both now pass -skipNuget and drop the NuGet tool installer that existed to serve the pack step; Authenticate NuGet to CFS and the dotnet test restore do not depend on it.

4. Remove installMavenPluginLocally.ps1

The script cloned https://github.com/ahmedmuhsin/azure-maven-plugins (a personal fork) at branch sdk-types and ran mvn clean install, publishing the result into the local ~/.m2 repository that later worker builds resolve from. That puts untrusted build code into the build environment (CWE-829).

Nothing references it — no pipeline, script, or doc mentions installMavenPluginLocally — so it is not reachable from CI and removing it changes no build behaviour. The sibling installAdditionsLocally.ps1 shows the intended convention, cloning the official Azure/azure-functions-java-additions.

Validation

• All 5 emulated Linux legs pass on -min (9.8–13.7 min), disk steady at 62G free with no cleanup step.
• All 4 Docker integration legs pass on -min with downloaded JDKs and the venv-based Python, and got faster: 14.6/13.1/10.7/16.0 min → 13.5/9.8/10.2/10.9 min. Single sample, so treat the magnitude as directional, but every leg improved despite adding a JDK download.
• Both official-build.yml and public-build.yml consume these job templates, so the fixes apply to both pipelines.

CI and local build scripts only, no product change.

Follow-up (not in this PR)

integration-tests.yml still runs its E2E Linux job on 1es-ubuntu-22.04 and still packs a NuGet package. It is unaffected by either problem here — it never invokes docker, and mono is present on 22.04 — so it is left alone. Moving it to -min would need the same Maven, JDK, and -skipNuget treatment.

[Copilot]

Pull request overview

This PR updates CI templates and build scripts to restore reliability of Linux CI agents by removing a Docker Hub dependency during test-app packaging, reducing disk pressure on hosted agents by moving jobs to a leaner Ubuntu image, and skipping unnecessary NuGet packaging in emulated test legs.

Changes:

• Update Linux job templates to use 1es-ubuntu-24.04-min and explicitly install missing toolchain components (e.g., Maven/Node/Python venv) while removing disk-cleanup steps.
• Skip NuGet packaging in emulated test jobs by passing -skipNuget to package-pipeline.ps1 and removing the NuGet tool installer steps.
• Prefer host mksquashfs for Docker test app packaging, with a Docker fallback pointing to the MCR Ubuntu mirror; remove an unused/unsafe Maven plugin install script.

Reviewed changes

Copilot reviewed 5 out of 5 changed files in this pull request and generated 1 comment.

Show a summary per file

File	Description
installMavenPluginLocally.ps1	Removes an unused script that cloned and built a personal forked Maven plugin repo.
eng/ci/templates/jobs/run-emulated-tests-windows.yml	Drops NuGet tool install and skips NuGet packaging in the emulated Windows job.
eng/ci/templates/jobs/run-emulated-tests-linux.yml	Moves emulated Linux tests to 1es-ubuntu-24.04-min, installs Maven/Node, removes disk cleanup, and skips NuGet packaging.
eng/ci/templates/jobs/run-docker-tests-linux.yml	Updates Docker integration test job to 1es-ubuntu-24.04-min, installs required build tools, downloads JDKs, and uses a Python venv for pip-based tooling.
dockertests/build-apps.ps1	Uses host mksquashfs when available to avoid Docker Hub pulls; Docker fallback uses MCR mirror.

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

[Copilot]

Pull request overview

Copilot reviewed 5 out of 5 changed files in this pull request and generated 1 comment.