Skip to content

guest: add Ubuntu userland support - #116

Open
Pedro Henrique Penna (ppenna) wants to merge 9 commits into
devfrom
feat-ubuntu-guest
Open

Pedro Henrique Penna (ppenna) wants to merge 9 commits into
devfrom
feat-ubuntu-guest

Conversation

@ppenna

Copy link
Copy Markdown
Contributor

Summary

  • add typed Alpine and Ubuntu guest descriptors and explicit --guest selection for build, run, and correctness workflows while preserving Alpine defaults
  • build Ubuntu Base 26.04.1 from verified immutable inputs, install a locked supplemental .deb closure without maintainer-script execution, and emit deterministic initramfs and EROFS artifacts with package/helper provenance
  • add exact Ubuntu corresponding-source collection, including Launchpad publishing-history fallback for superseded source versions
  • move shared guest helpers into guest/common, retain Alpine-only sandbox entry helpers under guest/alpine, and add the Ubuntu shell-ready and sandbox smoke profiles
  • wire Ubuntu artifacts through Docker, caching, CI, release/download packaging, setup scripts, notices, and user/design documentation

Validation

  • python3 -m unittest scripts/test_performance.py scripts/test_nvx_tools.py scripts/test_microvm_tests.py -v — 217 tests passed, 1 skipped
  • Ruff lint/format and strict Pyright checks for Linux and Windows
  • ShellCheck and shfmt for common, Alpine, and Ubuntu guest scripts
  • python3 scripts/nvx.py verify, CLI help smoke tests, host inventory tests, and PowerShell syntax parsing
  • native Alpine and Ubuntu initramfs builds plus Ubuntu EROFS construction
  • two independent Ubuntu initramfs/EROFS builds with matching SHA-256 values
  • local WHP Ubuntu boot, identity, lifecycle, workload identity, SMP, console exit, virtio-net, HostFs, managed lifecycle, blockless snapshot/restore, and EROFS sandbox execution
  • all 65 exact Ubuntu source identities resolved; a superseded source package was downloaded and checksum-validated end to end

Scope

This change does not modify the OpenVMM submodule or kernel/config-microvm. Ubuntu uses the existing NVX kernel and fixed microVM ABI. Systemd remains unsupported for the initial Ubuntu profiles.

Copilot AI balanced review requested due to automatic review settings September 18, 2026 18:13

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🟡 Changes recommended

Verification failures and unresolved artifact determinism, cache, benchmark, and source-provenance gaps remain.

Get a fresh assessment by requesting another Copilot review.

Review effort: Balanced
Findings: 2 High severity · 6 Medium severity

Open (8)
What changed in this PR

Adds Ubuntu Base 26.04.1 guest support alongside Alpine while retaining the shared NVX kernel and Alpine defaults.

Changes:

  • Adds typed guest selection, Ubuntu artifact builds, and runtime tests.
  • Adds deterministic EROFS packaging and corresponding-source collection.
  • Integrates Ubuntu artifacts into CI, releases, setup, and documentation.
File Description
ubuntu/​packages.lock.json Pins supplemental Ubuntu packages.
THIRD_PARTY_NOTICES.md Documents Ubuntu licensing obligations.
SOURCE-MANIFEST.json Records Ubuntu source inputs.
scripts/​test_nvx_tools.py Tests guest tooling and source handling.
scripts/​test_microvm_tests.py Tests Ubuntu microVM selection.
scripts/​setup/​setup-windows-whp.ps1 Installs Ubuntu artifacts on Windows.
scripts/​setup/​setup-linux-mshv.sh Builds and bundles all guests.
scripts/​nvx.py Adds guest-aware CLI commands.
scripts/​nvx_tools/​release.py Packages Ubuntu binaries and sources.
scripts/​nvx_tools/​microvm_tests.py Adds Ubuntu correctness profiles.
scripts/​nvx_tools/​guests.py Defines typed guest descriptors.
scripts/​nvx_tools/​common.py Shares cache and verified-download helpers.
scripts/​nvx_tools/​collect_ubuntu_sources.py Collects exact Ubuntu sources.
scripts/​nvx_tools/​build.py Builds Ubuntu initramfs and EROFS artifacts.
scripts/​nvx_tools/​benchmark.py Supports guest-specific boot markers.
README.md Introduces Ubuntu guest selection.
guest/​ubuntu/​nvx-bashrc Configures the Ubuntu shell.
guest/​common/​nvx-virtio-restore-probe Shares virtio restore validation.
guest/​common/​nvx-snapshot Shares snapshot and restore handling.
guest/​common/​nvx-sandbox-smoke Validates Ubuntu sandbox execution.
guest/​common/​nvx-reseed.c Implements restore reseeding.
guest/​common/​nvx-port-io.c Implements restore port operations.
guest/​common/​nvx-mmio-write.c Adds MMIO notification support.
guest/​common/​nvx-init-agent Bootstraps sandbox workloads.
guest/​common/​nvx-identity-probe Validates workload isolation.
guest/​common/​nvx-hostmount Mounts virtio-fs shares.
guest/​common/​nvx-exit Implements guest shutdown.
guest/​common/​nvx-device-io.c Provides static I/O benchmarking.
guest/​common/​nvx-console-pending.c Reports pending console input.
guest/​common/​init Boots Alpine or Ubuntu userland.
guest/​alpine/​nvx-container-launch Launches Alpine-controlled workloads.
guest/​alpine/​nvx-container-enter Constructs workload namespaces.
docker/​Dockerfile Builds all guest artifact variants.
doc/​usage.md Documents new CLI options.
doc/​setup.md Updates validation paths.
doc/​run.md Documents Ubuntu runtime usage.
doc/​project-structure.md Describes the guest layout.
doc/​distribution.md Documents Ubuntu source distribution.
doc/​design/​validation.md Extends validation architecture.
doc/​design/​sandbox-filesystem-and-agent-architecture.md Updates sandbox helper locations.
doc/​design/​goals.md Expands guest portability goals.
doc/​design/​code-ownership-map.md Updates guest ownership paths.
doc/​design.md Links the Ubuntu design documentation.
doc/​ci.md Documents Ubuntu CI coverage.
doc/​build.md Documents Ubuntu build workflows.
.github/​workflows/​ci.yml Runs Ubuntu guest and sandbox tests.
.github/​actions/​check-quality/​action.yml Checks relocated guest scripts.
.github/​actions/​build-guest-artifacts/​action.yml Builds and caches Ubuntu artifacts.
.gitattributes Enforces guest-source line endings.

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread SOURCE-MANIFEST.json Outdated
Comment thread scripts/nvx_tools/build.py
Comment thread .github/actions/build-guest-artifacts/action.yml Outdated
Comment thread docker/Dockerfile Outdated
Comment thread scripts/nvx_tools/collect_ubuntu_sources.py Outdated
Comment thread scripts/nvx_tools/collect_ubuntu_sources.py Outdated
Comment thread scripts/nvx_tools/microvm_tests.py Outdated
Comment thread scripts/nvx_tools/ubuntu.py

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Comment thread scripts/nvx_tools/collect_ubuntu_sources.py
Comment thread scripts/nvx_tools/collect_ubuntu_sources.py
Comment thread .github/actions/check-quality/action.yml Outdated
Comment thread scripts/nvx_tools/microvm_tests.py

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🟡 Changes recommended

Unresolved extraction safety, release-integrity, workflow migration, and runtime compatibility issues can break builds or affect host files.

Get a fresh assessment by requesting another Copilot review.

Review effort: Balanced
Findings: 2 High severity · 4 Medium severity

Open (6)
Resolved since last review (4)

Comment thread .github/actions/check-quality/action.yml
Comment thread scripts/nvx_tools/ubuntu.py Outdated
Comment thread scripts/nvx.py Outdated
Comment thread scripts/nvx_tools/benchmark.py
Comment thread scripts/nvx_tools/microvm_tests.py
Comment thread scripts/nvx_tools/release.py
Introduce typed guest descriptors and explicit --guest selection for native and Docker builds, run commands, and correctness workflows while preserving Alpine defaults and artifact names.

Build Ubuntu Base 26.04.1 from a repository-owned SHA-256 pin, install a complete locked supplemental .deb closure without executing maintainer scripts, enforce safe archive extraction and sandbox metadata policy, and emit deterministic initramfs and EROFS artifacts with binary-package and helper provenance.

Add exact Ubuntu source collection with Launchpad publishing-history fallback for superseded versions, extend release and download packaging, add Docker and CI artifact coverage, move common guest helpers into a distribution-neutral layout, and document both the interactive initramfs and non-root sandbox layer profiles. The OpenVMM submodule and kernel configuration remain unchanged.

Validated with the 217-test Python suite, Ruff and Pyright on Linux and Windows targets, ShellCheck and shfmt, deterministic double builds, native Alpine and Ubuntu artifact builds, and local WHP initramfs, lifecycle, SMP, console, networking, HostFs, snapshot, and sandbox execution.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 6f9e68de-6b8a-4d23-b59e-ce1026001e15
Fix deterministic inputs, source provenance, guest scenario selection, and related regression coverage.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Keep the merged Ubuntu release imports in repository-standard order.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Use the canonical package-lock digest in the provenance-aware release fixture.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Authenticate Ubuntu source collection through signed live and historical indexes, harden output handling and guest identity checks, and fix the Windows, Ubuntu network, and low-memory benchmark regressions.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Keep the wget customization regression test privilege-independent on Windows and align performance collection with the supported 128 MiB minimum shell-snapshot profile.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🟡 Changes recommended

Ubuntu release artifact provenance is not validated, and the new systemd guard introduces an avoidable sandbox API regression.

Get a fresh assessment by requesting another Copilot review.

Review effort: Balanced
Findings: 2 High severity · 5 Medium severity · 3 Low severity

Open (10)

Comment thread scripts/nvx.py Outdated
Comment thread doc/benchmarks.md
Comment thread doc/setup.md Outdated
Comment thread scripts/nvx_tools/benchmark.py
Drop the unsupported 64 MiB shell-snapshot metrics from collection and matrix verification, and keep the Ubuntu wget regression test portable to unprivileged Windows runners.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot AI review requested due to automatic review settings September 22, 2026 02:26

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🔵 Needs a closer look

Release provenance can accept stale Ubuntu artifacts, and the build environment is not fully immutable.

Review effort: Balanced
Findings: 2 High severity · 5 Medium severity · 3 Low severity

Open (10)
Previously missed (1)

In code that hasn't changed since last review

Medium severity Pin Debian toolchain packages for reproducible builds

docker/​Dockerfile:4

The base-image digest does not make this build environment immutable because the later apt-get update resolves unversioned compiler, libc, cpio, gzip, and zstd packages from moving Debian repositories. Those tools contribute bytes to the Ubuntu helpers/initramfs, while the determinism target only compares two builds against the same current repository state. Use a dated Debian snapshot with pinned package versions (or a fully provisioned immutable tool image) so clean rebuilds remain reproducible over time.

Finish the guest-path workflow migration, root archive symlinks safely, bind Ubuntu artifacts to manifests, enforce systemd policy from canonical paths and validated distro metadata, and align scenario and benchmark documentation.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot AI review requested due to automatic review settings September 22, 2026 02:56

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Comment thread scripts/nvx_tools/release.py
Comment thread doc/usage.md Outdated
Comment thread scripts/nvx_tools/performance.py
Record the shared Ubuntu input digest in both artifact manifests, require it during source collection and packaging, keep absolute archive links rooted safely, and document the complete Ubuntu scenario contract.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot AI review requested due to automatic review settings September 22, 2026 03:15

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🔵 Needs a closer look

Guest cache keys omit an executed build input, and two CLI commands accept guest values they cannot process.

Review effort: Balanced
Findings: None

Resolved since last review (3)
Previously missed (3)

In code that hasn't changed since last review

Medium severity Include scripts/nvx.py in Alpine cache key

.github/​actions/​build-guest-artifacts/​action.yml:48

The Alpine cache has the same missing build input: Docker runs scripts/nvx.py build-initramfs --guest alpine, but this key does not hash scripts/nvx.py. CLI wiring changes can therefore reuse stale Alpine artifacts; include the entry point in this hash as well.

Medium severity Include scripts/nvx.py in Ubuntu cache key

.github/​actions/​build-guest-artifacts/​action.yml:49

This cache key omits scripts/nvx.py, even though the Docker stages invoke that file to construct the Ubuntu build configuration. A change to the CLI's Ubuntu build wiring can therefore restore and publish stale cached artifacts instead of rebuilding them. Include the entry point among the hashed inputs.

Medium severity Restrict guest argument to supported Ubuntu value

scripts/​nvx.py:587

The parser advertises and accepts --guest alpine, but command_build_distro_layer immediately rejects it because this command only supports Ubuntu. Restrict the argparse choice so invalid input fails at parsing and the generated help matches the actual API.

This issue also appears on line 600 of the same file.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants