From dec7fe912f61a19550587730c47f7a5be3e6bf68 Mon Sep 17 00:00:00 2001 From: Zexin Yuan Date: Wed, 24 Jun 2026 16:48:27 +0800 Subject: [PATCH] Support lock files for reproducible builds PixiBuilder and UvBuilder now accept a lock file (uv.lock / pixi.lock) via new lockContent/lockFile/lockUrl methods that mirror the existing content/file/url declaration API and are defined once on the Builder interface. When a lock is supplied it is copied into the environment directory and the install runs in strict lock mode so the environment matches the committed lock (uv: --frozen; pixi: --frozen). appose.json records a lockHash (SHA-256) of the lock content, so a change to the lock forces a rebuild through the normal isUpToDate() check. When no lock is supplied the lockHash key is omitted entirely, keeping appose.json byte-identical to before (no spurious rebuilds of existing environments) and never passing the strict flag. DynamicBuilder forwards the lock to the detected pixi/uv delegate; MambaBuilder and SimpleBuilder override lockContent to throw UnsupportedOperationException. Note that pixi --frozen uses the lock as-is (the lock is authoritative) while uv --frozen fails fast on a stale lock; both make builds reproducible, and the tests assert the actual semantics of each tool. --- CLAUDE.md | 37 ++++ src/main/java/org/apposed/appose/Builder.java | 95 ++++++++++ .../apposed/appose/builder/BaseBuilder.java | 43 +++++ .../appose/builder/DynamicBuilder.java | 3 + .../apposed/appose/builder/MambaBuilder.java | 7 + .../apposed/appose/builder/PixiBuilder.java | 48 ++++- .../apposed/appose/builder/SimpleBuilder.java | 7 + .../org/apposed/appose/builder/UvBuilder.java | 32 +++- src/main/java/org/apposed/appose/tool/Uv.java | 25 +++ .../java/org/apposed/appose/TestBase.java | 24 +++ .../appose/builder/BaseBuilderTest.java | 104 +++++++++++ .../appose/builder/MambaBuilderTest.java | 21 +++ .../appose/builder/PixiBuilderTest.java | 173 ++++++++++++++++++ .../apposed/appose/builder/UvBuilderTest.java | 162 ++++++++++++++++ 14 files changed, 776 insertions(+), 5 deletions(-) create mode 100644 src/test/java/org/apposed/appose/builder/BaseBuilderTest.java diff --git a/CLAUDE.md b/CLAUDE.md index 39c81d9..e6935fe 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -170,12 +170,14 @@ The project provides type-safe builder classes for different environment types: - Supports `pixi.toml` and `environment.yml` files - Uses `pixi run --manifest-path /pixi.toml` for activation - Environment structure: `/.pixi/envs/default` +- Lock files: `.lockFile()/.lockContent()/.lockUrl()` for reproducible builds (copies `pixi.lock` into envDir, installs with `pixi install --frozen`) - Location: `org.apposed.appose.builder.PixiBuilder` **MambaBuilder** - Traditional conda environments via micromamba - Created via `Appose.mamba()` or `Appose.mamba(source)` - Supports `environment.yml` files - Uses `mamba run -p ` for activation +- Does not support lock files (conda/micromamba has no lockfile mechanism) - Location: `org.apposed.appose.builder.MambaBuilder` **UvBuilder** - Fast Python virtual environments via uv @@ -184,6 +186,7 @@ The project provides type-safe builder classes for different environment types: - Supports `requirements.txt` files - Standard Python venv structure (no special activation needed) - Environment structure: `/bin` (or `Scripts` on Windows) +- Lock files: `.lockFile()/.lockContent()/.lockUrl()` for reproducible builds (copies `uv.lock` into envDir, installs with `uv sync --frozen`; requires a `pyproject.toml` declaration) - Location: `org.apposed.appose.builder.UvBuilder` **DynamicBuilder** - Auto-detects appropriate builder based on configuration content @@ -197,6 +200,7 @@ The project provides type-safe builder classes for different environment types: - Created via `Appose.custom()` or implicitly via `Appose.system()` - No package installation; uses whatever executables are on the system - Methods: `binPaths(paths...)`, `appendSystemPath()`, `inheritRunningJava()` +- Does not support lock files (no package management) - Location: `org.apposed.appose.builder.SimpleBuilder` ### API Examples @@ -235,6 +239,16 @@ Environment env = Appose.file("path/to/environment.yml") .logDebug() .build(); +// Reproducible build pinned by a lock file (uv: uv.lock -> uv sync --frozen) +Environment env = Appose.uv("path/to/pyproject.toml") + .lockFile("path/to/uv.lock") + .build(); + +// Reproducible build pinned by a lock file (pixi: pixi.lock -> pixi install --frozen) +Environment env = Appose.pixi("path/to/pixi.toml") + .lockFile("path/to/pixi.lock") + .build(); + // Wrap existing environment Environment env = Appose.wrap("/path/to/existing/env"); @@ -264,6 +278,29 @@ All builders support subscription methods for monitoring: - `subscribeError(consumer)` - Error output from build process - `logDebug()` - Convenience method that logs output and errors to stderr +### Lock Files & Reproducible Builds + +`PixiBuilder` and `UvBuilder` support reproducible, lock-file-pinned builds via the +`lockContent(String)`, `lockFile(String|File)`, and `lockUrl(String|URL)` methods (mirroring the +`content`/`file`/`url` declaration API). When a lock is supplied: + +- The lock is copied into the environment directory (`uv.lock` / `pixi.lock`) before install. +- The install runs in strict mode so the environment matches the lock exactly, or the build fails: + - **uv**: `uv sync --frozen` (fails if `uv.lock` is missing or out of date with `pyproject.toml`). + Lock files require a `pyproject.toml` declaration (the `requirements.txt` path uses `pip install` + and has no lockfile). + - **pixi**: `pixi install --frozen` (installs the environment exactly as defined in `pixi.lock`, + without re-resolving or updating it — the lock is authoritative). Unlike uv's `--frozen`, pixi's + `--frozen` uses a stale lock as-is rather than failing, but both make builds reproducible. +- The `appose.json` state snapshot records a `lockHash` (SHA-256 of the lock content), so a change to + the lock forces a rebuild via the normal `isUpToDate()` check. +- `DynamicBuilder` (`Appose.file/url/content`) forwards the lock to the detected pixi/uv builder. +- `MambaBuilder` and `SimpleBuilder` do **not** support lock files and throw + `UnsupportedOperationException`. + +When no lock is supplied, behavior is unchanged: no strict flag is passed and `appose.json` omits +`lockHash`, so the snapshot is byte-identical to pre-lock-file builds (no spurious rebuilds). + ## Related Projects - appose-python: Python implementation of Appose (https://github.com/apposed/appose-python) diff --git a/src/main/java/org/apposed/appose/Builder.java b/src/main/java/org/apposed/appose/Builder.java index a2b6203..c511688 100644 --- a/src/main/java/org/apposed/appose/Builder.java +++ b/src/main/java/org/apposed/appose/Builder.java @@ -294,6 +294,101 @@ default T url(URL url) throws BuildException { */ T scheme(String scheme); + /** + * Specifies lock file content for reproducible builds. When provided, the + * lock file is copied into the environment directory and the install runs in + * strict lock mode so the environment matches the lock (uv: {@code --frozen}; + * pixi: {@code --frozen}); the committed lock anchors the build for + * reproducibility. Exact stale-lock handling is tool-specific — see the + * implementing builders. + *

+ * Not all builders support lock files; builders that do not will throw + * {@link UnsupportedOperationException}. The default implementation throws, + * so only builders that explicitly support locks override this method. + *

+ * + * @param lockContent Lock file content (e.g., uv.lock, pixi.lock) + * @return This builder instance, for fluent-style programming. + */ + default T lockContent(String lockContent) { + throw new UnsupportedOperationException( + getClass().getSimpleName() + " does not support lock files"); + } + + /** + * Specifies a lock file path for reproducible builds. + * Reads the file content immediately and delegates to {@link #lockContent(String)}. + * + * @param path Path to the lock file (e.g., "uv.lock", "pixi.lock") + * @return This builder instance, for fluent-style programming. + * @throws BuildException If the file cannot be read + */ + default T lockFile(String path) throws BuildException { + return lockFile(new File(path)); + } + + /** + * Specifies a lock file for reproducible builds. + * Reads the file content immediately and delegates to {@link #lockContent(String)}. + * + * @param file Lock file (e.g., uv.lock, pixi.lock) + * @return This builder instance, for fluent-style programming. + * @throws BuildException If the file cannot be read + */ + default T lockFile(File file) throws BuildException { + try { + Path filePath = file.toPath(); + String fileContent = new String( + Files.readAllBytes(filePath), + StandardCharsets.UTF_8 + ); + return lockContent(fileContent); + } + catch (IOException e) { + throw new BuildException(this, e); + } + } + + /** + * Specifies a URL to fetch lock file content from for reproducible builds. + * Reads the URL content immediately and delegates to {@link #lockContent(String)}. + * + * @param path URL path of the lock file + * @return This builder instance, for fluent-style programming. + * @throws BuildException If the URL cannot be read or is invalid + */ + default T lockUrl(String path) throws BuildException { + try { + return lockUrl(new URL(path)); + } + catch (MalformedURLException e) { + throw new BuildException(this, e); + } + } + + /** + * Specifies a URL to fetch lock file content from for reproducible builds. + * Reads the URL content immediately and delegates to {@link #lockContent(String)}. + * + * @param url URL to the lock file + * @return This builder instance, for fluent-style programming. + * @throws BuildException If the URL cannot be read + */ + default T lockUrl(URL url) throws BuildException { + try (InputStream stream = url.openStream()) { + ByteArrayOutputStream result = new ByteArrayOutputStream(); + byte[] buffer = new byte[8192]; + int length; + while ((length = stream.read(buffer)) != -1) { + result.write(buffer, 0, length); + } + return lockContent(result.toString(StandardCharsets.UTF_8.name())); + } + catch (IOException e) { + throw new BuildException(this, e); + } + } + /** * Registers a callback method to be invoked when progress happens during environment building. * diff --git a/src/main/java/org/apposed/appose/builder/BaseBuilder.java b/src/main/java/org/apposed/appose/builder/BaseBuilder.java index 3e13fbb..d60a658 100644 --- a/src/main/java/org/apposed/appose/builder/BaseBuilder.java +++ b/src/main/java/org/apposed/appose/builder/BaseBuilder.java @@ -43,6 +43,8 @@ import java.nio.charset.StandardCharsets; import java.nio.file.Files; import java.nio.file.Paths; +import java.security.MessageDigest; +import java.security.NoSuchAlgorithmException; import java.util.ArrayList; import java.util.HashMap; import java.util.LinkedHashMap; @@ -72,6 +74,9 @@ public abstract class BaseBuilder> implements Builder state) { state.put("channels", channels); state.put("flags", flags); state.put("envVars", new TreeMap<>(envVars)); + // Record a hash of the lock file content so that lock changes trigger a + // rebuild via the exact-match isUpToDate() comparison. Only added when a + // lock is supplied, so lock-less builds produce a byte-identical + // appose.json (backward compatibility). + if (lockContent != null) state.put("lockHash", computeLockHash(lockContent)); + } + + /** + * Computes a SHA-256 hash (lowercase hex) of the given content, or null if + * the content is null. Used to snapshot lock files into {@code appose.json} + * without storing the (potentially large) lock content verbatim. + * + * @param content The content to hash (e.g., lock file content). + * @return The SHA-256 hex hash, or null if content is null. + */ + protected static String computeLockHash(String content) { + if (content == null) return null; + try { + MessageDigest md = MessageDigest.getInstance("SHA-256"); + byte[] digest = md.digest(content.getBytes(StandardCharsets.UTF_8)); + char[] hex = new char[digest.length * 2]; + for (int i = 0; i < digest.length; i++) { + int v = digest[i] & 0xff; + hex[i * 2] = Character.forDigit(v >>> 4, 16); + hex[i * 2 + 1] = Character.forDigit(v & 0x0f, 16); + } + return new String(hex); + } + catch (NoSuchAlgorithmException e) { + // SHA-256 is mandated by the JVM specification; this never happens. + throw new RuntimeException("SHA-256 algorithm not available", e); + } } /** diff --git a/src/main/java/org/apposed/appose/builder/DynamicBuilder.java b/src/main/java/org/apposed/appose/builder/DynamicBuilder.java index 2b73a51..c2b8a9e 100644 --- a/src/main/java/org/apposed/appose/builder/DynamicBuilder.java +++ b/src/main/java/org/apposed/appose/builder/DynamicBuilder.java @@ -88,6 +88,9 @@ private void copyConfigToDelegate(Builder delegate) { if (envDir != null) delegate.base(envDir); if (content != null) delegate.content(content); if (scheme != null) delegate.scheme(scheme.name()); + // Forward the lock file if one was provided. Builders that do not + // support locks (mamba/custom) override lockContent() to throw. + if (lockContent != null) delegate.lockContent(lockContent); delegate.channels(channels); progressSubscribers.forEach(delegate::subscribeProgress); outputSubscribers.forEach(delegate::subscribeOutput); diff --git a/src/main/java/org/apposed/appose/builder/MambaBuilder.java b/src/main/java/org/apposed/appose/builder/MambaBuilder.java index 59b2097..ebf48af 100644 --- a/src/main/java/org/apposed/appose/builder/MambaBuilder.java +++ b/src/main/java/org/apposed/appose/builder/MambaBuilder.java @@ -57,6 +57,13 @@ public String envType() { return "mamba"; } + @Override + public MambaBuilder lockContent(String lockContent) { + throw new UnsupportedOperationException( + "MambaBuilder does not support lock files; " + + "conda/micromamba environments have no lockfile mechanism."); + } + @Override public Environment build() throws BuildException { File envDir = resolveEnvDir(); diff --git a/src/main/java/org/apposed/appose/builder/PixiBuilder.java b/src/main/java/org/apposed/appose/builder/PixiBuilder.java index 572901c..4e97cb6 100644 --- a/src/main/java/org/apposed/appose/builder/PixiBuilder.java +++ b/src/main/java/org/apposed/appose/builder/PixiBuilder.java @@ -114,6 +114,22 @@ public Environment build() throws BuildException { } } + // Validate lock-file compatibility. pixi lockfiles apply to manifest- + // based builds (pixi.toml / pyproject.toml); programmatic builds and + // imported environment.yml have no user manifest to lock against. + if (lockContent != null) { + if (content == null) { + throw new IllegalArgumentException( + "PixiBuilder lock files require a declaration file via .file()/.content(); " + + "programmatic builds cannot be locked."); + } + if (!"pixi.toml".equals(scheme.name()) && !"pyproject.toml".equals(scheme.name())) { + throw new IllegalArgumentException( + "PixiBuilder lock files require a pixi.toml or pyproject.toml declaration; " + + "environment.yml imports have no lockfile mechanism."); + } + } + Pixi pixi = new Pixi(); // Set up progress/output consumers. @@ -160,6 +176,13 @@ else if ("environment.yml".equals(scheme.name())) { pixi.exec("init", "--import", environmentYamlFile.getAbsolutePath(), envDir.getAbsolutePath()); } + // If a lock file was provided, copy it into the env dir so the + // subsequent install runs strictly from it (--frozen). + if (lockContent != null) { + File pixiLockFile = new File(envDir, "pixi.lock"); + Files.write(pixiLockFile.toPath(), lockContent.getBytes(StandardCharsets.UTF_8)); + } + // Add any programmatic channels to augment source file. if (!channels.isEmpty()) { pixi.addChannels(envDir, channels.toArray(new String[0])); @@ -213,7 +236,7 @@ else if ("environment.yml".equals(scheme.name())) { } } - runPixiInstall(pixi, envDir); + runPixiInstall(pixi, envDir, lockContent != null); writeApposeStateFile(envDir); return buildPixiEnvironment(pixi, envDir); } @@ -242,6 +265,12 @@ public Environment wrap(File envDir) throws BuildException { scheme = Schemes.fromName("pyproject.toml"); } } + // If a pixi.lock is present, capture it too so rebuild() reproduces + // the exact locked environment even after the directory is deleted. + File pixiLock = new File(envDir, "pixi.lock"); + if (pixiLock.exists() && pixiLock.isFile()) { + lockContent = new String(Files.readAllBytes(pixiLock.toPath()), StandardCharsets.UTF_8); + } } catch (IOException e) { throw new BuildException(this, e); @@ -261,7 +290,7 @@ private static List withFlag(List flags, String flag) { return result; } - private void runPixiInstall(Pixi pixi, File envDir) throws IOException, InterruptedException { + private void runPixiInstall(Pixi pixi, File envDir, boolean frozen) throws IOException, InterruptedException { File manifestFile = new File(envDir, "pyproject.toml"); if (!manifestFile.exists()) manifestFile = new File(envDir, "pixi.toml"); @@ -278,9 +307,20 @@ private void runPixiInstall(Pixi pixi, File envDir) throws IOException, Interrup pixi.setErrorConsumer(monitor::intercept); } - // Ensure the pixi environment is fully installed. + // Ensure the pixi environment is fully installed. When a lock was + // provided, pass --frozen so pixi installs the environment exactly as + // defined in pixi.lock, without re-resolving or updating it. This makes + // the build reproducible: the installed environment always matches the + // committed lock, byte for byte. (Note: pixi's --frozen, unlike uv's, + // uses the lock as-is even when the manifest has drifted -- the lock is + // authoritative, which is the reproducibility contract.) try { - pixi.exec("install", "--manifest-path", manifestFile.getAbsolutePath()); + if (frozen) { + pixi.exec("install", "--manifest-path", manifestFile.getAbsolutePath(), "--frozen"); + } + else { + pixi.exec("install", "--manifest-path", manifestFile.getAbsolutePath()); + } } finally { if (monitor != null) { diff --git a/src/main/java/org/apposed/appose/builder/SimpleBuilder.java b/src/main/java/org/apposed/appose/builder/SimpleBuilder.java index a8f7ec6..f566554 100644 --- a/src/main/java/org/apposed/appose/builder/SimpleBuilder.java +++ b/src/main/java/org/apposed/appose/builder/SimpleBuilder.java @@ -177,6 +177,13 @@ public SimpleBuilder channels(List channels) { "It uses existing executables without package management."); } + @Override + public SimpleBuilder lockContent(String lockContent) { + throw new UnsupportedOperationException( + "SimpleBuilder does not support lock files. " + + "Custom environments use existing executables without package management."); + } + // -- Internal methods -- @Override diff --git a/src/main/java/org/apposed/appose/builder/UvBuilder.java b/src/main/java/org/apposed/appose/builder/UvBuilder.java index 95306aa..568c80b 100644 --- a/src/main/java/org/apposed/appose/builder/UvBuilder.java +++ b/src/main/java/org/apposed/appose/builder/UvBuilder.java @@ -136,6 +136,22 @@ public Environment build() throws BuildException { } } + // Validate lock-file compatibility. uv lockfiles only apply to the + // pyproject.toml / uv sync path: requirements.txt uses pip install (no + // lockfile), and programmatic builds have no manifest to lock against. + if (lockContent != null) { + if (content == null) { + throw new IllegalArgumentException( + "UvBuilder lock files require a declaration file via .file()/.content(); " + + "programmatic builds cannot be locked."); + } + if (!"pyproject.toml".equals(scheme.name())) { + throw new IllegalArgumentException( + "UvBuilder lock files require a pyproject.toml declaration; " + + "requirements.txt has no lockfile mechanism."); + } + } + try { // If the env state matches our current configuration, // skip all package management and return immediately. @@ -163,8 +179,16 @@ public Environment build() throws BuildException { File pyprojectFile = new File(envDir, "pyproject.toml"); Files.write(pyprojectFile.toPath(), content.getBytes(StandardCharsets.UTF_8)); + // If a lock file was provided, copy it into the env dir and + // install strictly from it (--frozen) for reproducibility. + boolean frozen = lockContent != null; + if (frozen) { + File uvLockFile = new File(envDir, "uv.lock"); + Files.write(uvLockFile.toPath(), lockContent.getBytes(StandardCharsets.UTF_8)); + } + // Run uv sync to create .venv and install dependencies. - uv.sync(envDir, pythonVersion); + uv.sync(envDir, pythonVersion, frozen); } else { // Handle requirements.txt - traditional venv + pip install. // Create virtual environment if it doesn't exist. @@ -226,6 +250,12 @@ public Environment wrap(File envDir) throws BuildException { scheme = Schemes.fromName("requirements.txt"); } } + // If a uv.lock is present, capture it too so rebuild() reproduces + // the exact locked environment even after the directory is deleted. + File uvLock = new File(envDir, "uv.lock"); + if (uvLock.exists() && uvLock.isFile()) { + lockContent = new String(Files.readAllBytes(uvLock.toPath()), StandardCharsets.UTF_8); + } } catch (IOException e) { throw new BuildException(this, e); diff --git a/src/main/java/org/apposed/appose/tool/Uv.java b/src/main/java/org/apposed/appose/tool/Uv.java index 9a325c6..afcc4a4 100644 --- a/src/main/java/org/apposed/appose/tool/Uv.java +++ b/src/main/java/org/apposed/appose/tool/Uv.java @@ -263,12 +263,37 @@ public void pipInstallFromRequirements(final File envDir, String requirementsFil * @throws IllegalStateException if uv has not been installed */ public void sync(final File projectDir, String pythonVersion) throws IOException, InterruptedException { + sync(projectDir, pythonVersion, false); + } + + /** + * Synchronize a project's dependencies from pyproject.toml. + * Creates a virtual environment at projectDir/.venv and installs dependencies. + *

+ * When {@code frozen} is true, runs with {@code --frozen} so uv installs + * exactly from the lockfile ({@code uv.lock}) without re-resolving or + * updating it. uv will fail if the lockfile is missing or out of date + * relative to {@code pyproject.toml}, which is what makes the build + * reproducible. + *

+ * + * @param projectDir The project directory containing pyproject.toml. + * @param pythonVersion Optional Python version (e.g., "3.11"). Can be null for default. + * @param frozen If true, pass {@code --frozen} to enforce strict lockfile adherence. + * @throws IOException If an I/O error occurs. + * @throws InterruptedException If the current thread is interrupted. + * @throws IllegalStateException if uv has not been installed + */ + public void sync(final File projectDir, String pythonVersion, boolean frozen) throws IOException, InterruptedException { List args = new ArrayList<>(); args.add("sync"); if (pythonVersion != null && !pythonVersion.isEmpty()) { args.add("--python"); args.add(pythonVersion); } + if (frozen) { + args.add("--frozen"); + } // Run uv sync with working directory set to projectDir. exec(projectDir, args.toArray(new String[0])); diff --git a/src/test/java/org/apposed/appose/TestBase.java b/src/test/java/org/apposed/appose/TestBase.java index f225595..864044d 100644 --- a/src/test/java/org/apposed/appose/TestBase.java +++ b/src/test/java/org/apposed/appose/TestBase.java @@ -32,9 +32,15 @@ import org.apposed.appose.Service.ResponseType; import org.apposed.appose.Service.Task; import org.apposed.appose.Service.TaskStatus; +import org.apposed.appose.util.Json; +import java.io.File; +import java.io.IOException; +import java.nio.charset.StandardCharsets; +import java.nio.file.Files; import java.util.ArrayList; import java.util.List; +import java.util.Map; import static org.junit.jupiter.api.Assertions.assertEquals; import static org.junit.jupiter.api.Assertions.assertNotNull; @@ -185,4 +191,22 @@ public void assertComplete(Task task) { } assertEquals(TaskStatus.COMPLETE, task.status, errorMessage); } + + /** + * Reads and parses the {@code appose.json} state file from the given + * environment directory. Used by tests to assert the presence/absence of + * state keys such as {@code lockHash}. + * + * @param envDir The environment directory containing {@code appose.json}. + * @return The parsed state as a {@link Map}. + * @throws IOException If the file cannot be read. + */ + public Map apposeJsonMap(File envDir) throws IOException { + File apposeJson = new File(envDir, "appose.json"); + String json = new String(Files.readAllBytes(apposeJson.toPath()), StandardCharsets.UTF_8); + Object parsed = Json.parseJson(json); + @SuppressWarnings("unchecked") + Map map = (Map) parsed; + return map; + } } diff --git a/src/test/java/org/apposed/appose/builder/BaseBuilderTest.java b/src/test/java/org/apposed/appose/builder/BaseBuilderTest.java new file mode 100644 index 0000000..538437d --- /dev/null +++ b/src/test/java/org/apposed/appose/builder/BaseBuilderTest.java @@ -0,0 +1,104 @@ +/*- + * #%L + * Appose: multi-language interprocess cooperation with shared memory. + * %% + * Copyright (C) 2023 - 2026 Appose developers. + * %% + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions are met: + * + * 1. Redistributions of source code must retain the above copyright notice, + * this list of conditions and the following disclaimer. + * 2. Redistributions in binary form must reproduce the above copyright notice, + * this list of conditions and the following disclaimer in the documentation + * and/or other materials provided with the distribution. + * + * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" + * AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE + * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE + * ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDERS OR CONTRIBUTORS BE + * LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR + * CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF + * SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS + * INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN + * CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) + * ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE + * POSSIBILITY OF SUCH DAMAGE. + * #L% + */ + +package org.apposed.appose.builder; + +import java.io.File; + +import org.junit.jupiter.api.Test; + +import static org.junit.jupiter.api.Assertions.assertEquals; +import static org.junit.jupiter.api.Assertions.assertNotEquals; +import static org.junit.jupiter.api.Assertions.assertNull; +import static org.junit.jupiter.api.Assertions.assertTrue; + +/** + * Unit tests for {@link BaseBuilder} helpers. These are fast and require no + * external tools or network access. + */ +public class BaseBuilderTest { + + /** Same lock content must produce an identical hash. */ + @Test + public void testComputeLockHashDeterministic() { + String content = "version = 1\npackages = []\n"; + assertEquals(BaseBuilder.computeLockHash(content), BaseBuilder.computeLockHash(content), + "identical content must hash identically"); + } + + /** Different lock content must produce different hashes. */ + @Test + public void testComputeLockHashContentSensitive() { + String a = BaseBuilder.computeLockHash("packages = [\"a\"]\n"); + String b = BaseBuilder.computeLockHash("packages = [\"b\"]\n"); + assertNotEquals(a, b, "different content must hash differently"); + } + + /** Null content must hash to null (so no lockHash key is emitted). */ + @Test + public void testComputeLockHashNull() { + assertNull(BaseBuilder.computeLockHash(null)); + } + + /** The hash must be a 64-character lowercase hex string (SHA-256). */ + @Test + public void testComputeLockHashFormat() { + String hash = BaseBuilder.computeLockHash("appose"); + assertEquals(64, hash.length(), "SHA-256 hex must be 64 chars"); + assertTrue(hash.matches("[0-9a-f]{64}"), "hash must be lowercase hex: " + hash); + } + + /** Lock files are opt-in: conda/custom builders that cannot honor them reject them early. */ + @Test + public void testMambaRejectsLock() { + org.junit.jupiter.api.Assertions.assertThrows(UnsupportedOperationException.class, + () -> new MambaBuilder().lockContent("anything")); + } + + /** Lock files are opt-in: conda/custom builders that cannot honor them reject them early. */ + @Test + public void testSimpleRejectsLock() { + org.junit.jupiter.api.Assertions.assertThrows(UnsupportedOperationException.class, + () -> new SimpleBuilder().lockContent("anything")); + } + + /** A missing lock file surfaces as a BuildException (not a raw IOException). */ + @Test + public void testLockFileMissingThrowsBuildException() { + org.junit.jupiter.api.Assertions.assertThrows(org.apposed.appose.BuildException.class, + () -> new UvBuilder().lockFile(new File("this-lock-does-not-exist.lock"))); + } + + /** A malformed lock URL surfaces as a BuildException (not a raw MalformedURLException). */ + @Test + public void testLockUrlMalformedThrowsBuildException() { + org.junit.jupiter.api.Assertions.assertThrows(org.apposed.appose.BuildException.class, + () -> new UvBuilder().lockUrl("ht!tp://not a valid url")); + } +} diff --git a/src/test/java/org/apposed/appose/builder/MambaBuilderTest.java b/src/test/java/org/apposed/appose/builder/MambaBuilderTest.java index 962f07a..ce5cb9f 100644 --- a/src/test/java/org/apposed/appose/builder/MambaBuilderTest.java +++ b/src/test/java/org/apposed/appose/builder/MambaBuilderTest.java @@ -37,6 +37,7 @@ import java.io.File; import static org.junit.jupiter.api.Assertions.assertInstanceOf; +import static org.junit.jupiter.api.Assertions.assertThrows; import static org.junit.jupiter.api.Assertions.assertTrue; /** End-to-end tests for {@link MambaBuilder}. */ @@ -62,4 +63,24 @@ public void testExplicitMambaBuilder() throws Exception { cowsayAndAssert(env, "yay"); } + + /** + * Forwarding a lock to a mamba delegate via a dynamic builder must fail + * clearly, since conda/micromamba has no lockfile mechanism. + */ + @Test + public void testDynamicLockMambaFails() { + String envYml = + "name: lock-mamba-fail\n" + + "channels:\n" + + " - conda-forge\n" + + "dependencies:\n" + + " - python>=3.8\n"; + assertThrows(UnsupportedOperationException.class, () -> + Appose.content(envYml) + .builder("mamba") + .lockContent("bogus") + .base("target/envs/mamba-lock-fail") + .build()); + } } diff --git a/src/test/java/org/apposed/appose/builder/PixiBuilderTest.java b/src/test/java/org/apposed/appose/builder/PixiBuilderTest.java index f1964c8..23971ca 100644 --- a/src/test/java/org/apposed/appose/builder/PixiBuilderTest.java +++ b/src/test/java/org/apposed/appose/builder/PixiBuilderTest.java @@ -38,10 +38,15 @@ import java.io.File; import java.io.IOException; import java.net.URL; +import java.nio.charset.StandardCharsets; +import java.nio.file.Files; import java.util.List; import static org.junit.jupiter.api.Assertions.assertEquals; +import static org.junit.jupiter.api.Assertions.assertFalse; import static org.junit.jupiter.api.Assertions.assertInstanceOf; +import static org.junit.jupiter.api.Assertions.assertNotEquals; +import static org.junit.jupiter.api.Assertions.assertNotNull; import static org.junit.jupiter.api.Assertions.assertThrows; import static org.junit.jupiter.api.Assertions.assertTrue; @@ -254,4 +259,172 @@ public void testURLSupport() throws Exception { assertInstanceOf(PixiBuilder.class, env.builder()); cowsayAndAssert(env, "url!"); } + + // -- Lock-file reproducible builds -- + + /** + * A user-supplied lock is copied into the env dir and the install runs with + * --frozen, yielding a reproducible, working environment. Exercises both the + * {@code .lockFile(File)} and {@code .lockUrl(URL)} entry points. + */ + @Test + public void testPixiLockFrozen() throws Exception { + // Build without a lock first to generate a valid pixi.lock. + String baseA = "target/envs/pixi-lock-src"; + FilePaths.deleteRecursively(new File(baseA)); + Appose.pixi("src/test/resources/envs/cowsay-pixi.toml") + .base(baseA).logDebug().build(); + File lockFileA = new File(baseA, "pixi.lock"); + assertTrue(lockFileA.isFile(), "first build should generate a pixi.lock"); + String lockContent = new String(Files.readAllBytes(lockFileA.toPath()), StandardCharsets.UTF_8); + + // .lockFile(File): lock copied in, install runs --frozen. + String baseB = "target/envs/pixi-lock-file"; + FilePaths.deleteRecursively(new File(baseB)); + Environment envB = Appose.pixi("src/test/resources/envs/cowsay-pixi.toml") + .base(baseB).lockFile(lockFileA).logDebug().build(); + assertTrue(new File(baseB, "pixi.lock").isFile(), "lock should be copied into the env dir"); + assertTrue(apposeJsonMap(new File(baseB)).containsKey("lockHash"), + "appose.json should record lockHash when a lock is supplied"); + cowsayAndAssert(envB, "frozen"); + + // .lockUrl(URL): same outcome via a file:// URL. + String baseC = "target/envs/pixi-lock-url"; + FilePaths.deleteRecursively(new File(baseC)); + Environment envC = Appose.pixi("src/test/resources/envs/cowsay-pixi.toml") + .base(baseC).lockUrl(lockFileA.toURI().toURL()).logDebug().build(); + assertTrue(apposeJsonMap(new File(baseC)).containsKey("lockHash")); + cowsayAndAssert(envC, "url-lock"); + } + + /** + * With --frozen, pixi installs the environment exactly as defined in + * pixi.lock and does NOT re-resolve or update it. Building a manifest that + * drifted from the lock (it additionally requires `requests`) must therefore + * leave the lock unchanged (no `requests` resolved in). Without --frozen, + * pixi would update the lock to include `requests` -- so an unchanged lock + * proves the --frozen flag is wired in and the build is reproducible. + */ + @Test + public void testPixiLockFrozenUsesLockAsIs() throws Exception { + // A valid lock for the cowsay manifest (contains no `requests`). + String baseA = "target/envs/pixi-frozen-src"; + FilePaths.deleteRecursively(new File(baseA)); + Appose.pixi("src/test/resources/envs/cowsay-pixi.toml") + .base(baseA).logDebug().build(); + String cowsayLock = new String(Files.readAllBytes( + new File(baseA, "pixi.lock").toPath()), StandardCharsets.UTF_8); + assertFalse(cowsayLock.contains("requests"), "baseline lock should not contain requests"); + + // A manifest that additionally requires `requests`, built with the cowsay lock. + String pixiExtra = + "[workspace]\n" + + "name = \"cowsay-extra\"\n" + + "channels = [\"conda-forge\"]\n" + + "platforms = [\"linux-64\", \"osx-64\", \"osx-arm64\", \"win-64\"]\n" + + "\n" + + "[dependencies]\n" + + "python = \">=3.8\"\n" + + "pip = \"*\"\n" + + "\n" + + "[pypi-dependencies]\n" + + "cowsay = \"==6.1\"\n" + + "appose = \"*\"\n" + + "requests = \"*\"\n"; + String base = "target/envs/pixi-frozen"; + FilePaths.deleteRecursively(new File(base)); + Appose.pixi().content(pixiExtra).base(base).lockContent(cowsayLock).logDebug().build(); + + // --frozen must use the lock AS-IS: `requests` must NOT have been resolved in. + String lockAfter = new String(Files.readAllBytes( + new File(base, "pixi.lock").toPath()), StandardCharsets.UTF_8); + assertFalse(lockAfter.contains("requests"), + "--frozen must use the lock as-is, not re-resolve `requests` into it"); + } + + /** + * Programmatic builds (no manifest) cannot be locked. + */ + @Test + public void testPixiLockProgrammaticUnsupported() { + assertThrows(IllegalArgumentException.class, () -> + Appose.pixi() + .conda("python>=3.8") + .pypi("cowsay==6.1") + .base("target/envs/pixi-lock-prog") + .lockContent("bogus") + .build()); + } + + /** + * When no lock is supplied, appose.json must NOT contain a lockHash key, so + * the snapshot stays byte-identical to pre-lock-file builds (backward + * compatibility) and existing environments are never spuriously rebuilt. + */ + @Test + public void testPixiNoLockBackwardCompat() throws Exception { + String base = "target/envs/pixi-no-lock"; + FilePaths.deleteRecursively(new File(base)); + Environment env = Appose.pixi("src/test/resources/envs/cowsay-pixi.toml") + .base(base).logDebug().build(); + assertFalse(apposeJsonMap(new File(base)).containsKey("lockHash"), + "appose.json must NOT contain lockHash when no lock is supplied"); + cowsayAndAssert(env, "nolock"); + } + + /** + * Changing the lock content must change the lockHash in appose.json and thus + * force a rebuild. The lock is edited with a trailing comment, which doesn't + * change the resolved package set, so --frozen still succeeds. + */ + @Test + public void testPixiLockChangeTriggersRebuild() throws Exception { + String baseA = "target/envs/pixi-lock-change-src"; + FilePaths.deleteRecursively(new File(baseA)); + Appose.pixi("src/test/resources/envs/cowsay-pixi.toml").base(baseA).logDebug().build(); + String lock = new String(Files.readAllBytes(new File(baseA, "pixi.lock").toPath()), StandardCharsets.UTF_8); + + String base = "target/envs/pixi-lock-change"; + FilePaths.deleteRecursively(new File(base)); + Appose.pixi("src/test/resources/envs/cowsay-pixi.toml") + .base(base).lockContent(lock).logDebug().build(); + String hashBefore = (String) apposeJsonMap(new File(base)).get("lockHash"); + + Appose.pixi("src/test/resources/envs/cowsay-pixi.toml") + .base(base).lockContent(lock + "# trailing comment\n").logDebug().build(); + String hashAfter = (String) apposeJsonMap(new File(base)).get("lockHash"); + + assertNotNull(hashBefore, "lockHash should be present after a locked build"); + assertNotNull(hashAfter, "lockHash should be present after rebuild"); + assertNotEquals(hashBefore, hashAfter, + "a lock change must produce a different lockHash and force a rebuild"); + } + + /** + * wrap() captures the lock file into builder state, so rebuild() reproduces + * the locked environment even after its directory has been deleted. + */ + @Test + public void testPixiWrapLockSurvivesRebuild() throws Exception { + // Build a locked environment (generate a valid lock first). + String srcBase = "target/envs/pixi-wrap-src"; + FilePaths.deleteRecursively(new File(srcBase)); + Appose.pixi("src/test/resources/envs/cowsay-pixi.toml").base(srcBase).logDebug().build(); + String lock = new String(Files.readAllBytes(new File(srcBase, "pixi.lock").toPath()), StandardCharsets.UTF_8); + + String baseA = "target/envs/pixi-wrap-locked"; + FilePaths.deleteRecursively(new File(baseA)); + Appose.pixi("src/test/resources/envs/cowsay-pixi.toml") + .base(baseA).lockContent(lock).logDebug().build(); + assertTrue(apposeJsonMap(new File(baseA)).containsKey("lockHash")); + + // Wrap the locked env via the typed builder (captures pixi.toml + pixi.lock), + // then wipe + rebuild. + Environment env = Appose.pixi().wrap(new File(baseA)); + Environment rebuilt = env.rebuild(); + + assertTrue(apposeJsonMap(new File(baseA)).containsKey("lockHash"), + "rebuild after wrap must reproduce lockHash from the captured lock"); + cowsayAndAssert(rebuilt, "rewrapped"); + } } diff --git a/src/test/java/org/apposed/appose/builder/UvBuilderTest.java b/src/test/java/org/apposed/appose/builder/UvBuilderTest.java index 2d55c95..e38b793 100644 --- a/src/test/java/org/apposed/appose/builder/UvBuilderTest.java +++ b/src/test/java/org/apposed/appose/builder/UvBuilderTest.java @@ -30,11 +30,22 @@ package org.apposed.appose.builder; import org.apposed.appose.Appose; +import org.apposed.appose.BuildException; import org.apposed.appose.Environment; import org.apposed.appose.TestBase; +import org.apposed.appose.util.FilePaths; import org.junit.jupiter.api.Test; +import java.io.File; +import java.nio.charset.StandardCharsets; +import java.nio.file.Files; + import static org.junit.jupiter.api.Assertions.assertInstanceOf; +import static org.junit.jupiter.api.Assertions.assertFalse; +import static org.junit.jupiter.api.Assertions.assertNotEquals; +import static org.junit.jupiter.api.Assertions.assertNotNull; +import static org.junit.jupiter.api.Assertions.assertThrows; +import static org.junit.jupiter.api.Assertions.assertTrue; /** End-to-end tests for {@link UvBuilder}. */ public class UvBuilderTest extends TestBase { @@ -71,4 +82,155 @@ public void testUvPyproject() throws Exception { .build(); cowsayAndAssert(env, "pyproject"); } + + // -- Lock-file reproducible builds -- + + /** + * A user-supplied lock is copied into the env dir and the install runs with + * --frozen, yielding a reproducible, working environment. Exercises both the + * {@code .lockFile(File)} and {@code .lockUrl(URL)} entry points. + */ + @Test + public void testUvLockFrozen() throws Exception { + // First, build without a lock to generate a valid uv.lock for the manifest. + String baseA = "target/envs/uv-lock-src"; + FilePaths.deleteRecursively(new File(baseA)); + Appose.uv("src/test/resources/envs/cowsay-pyproject.toml") + .base(baseA).logDebug().build(); + File lockFileA = new File(baseA, "uv.lock"); + assertTrue(lockFileA.isFile(), "first build should generate a uv.lock"); + String lockContent = new String(Files.readAllBytes(lockFileA.toPath()), StandardCharsets.UTF_8); + + // .lockFile(File): lock copied in, install runs --frozen. + String baseB = "target/envs/uv-lock-file"; + FilePaths.deleteRecursively(new File(baseB)); + Environment envB = Appose.uv("src/test/resources/envs/cowsay-pyproject.toml") + .base(baseB).lockFile(lockFileA).logDebug().build(); + assertTrue(new File(baseB, "uv.lock").isFile(), "lock should be copied into the env dir"); + assertTrue(apposeJsonMap(new File(baseB)).containsKey("lockHash"), + "appose.json should record lockHash when a lock is supplied"); + cowsayAndAssert(envB, "frozen"); + + // .lockUrl(URL): same outcome via a file:// URL. + String baseC = "target/envs/uv-lock-url"; + FilePaths.deleteRecursively(new File(baseC)); + Environment envC = Appose.uv("src/test/resources/envs/cowsay-pyproject.toml") + .base(baseC).lockUrl(lockFileA.toURI().toURL()).logDebug().build(); + assertTrue(apposeJsonMap(new File(baseC)).containsKey("lockHash")); + cowsayAndAssert(envC, "url-lock"); + } + + /** + * An empty/stale lock cannot satisfy the manifest, so --frozen must reject + * it. (Without --frozen, uv would regenerate the lock and succeed — so this + * failure proves the --frozen flag is actually wired in.) + */ + @Test + public void testUvLockStaleFails() throws Exception { + // A valid lock for the cowsay manifest... + String baseA = "target/envs/uv-stale-src"; + FilePaths.deleteRecursively(new File(baseA)); + Appose.uv("src/test/resources/envs/cowsay-pyproject.toml") + .base(baseA).logDebug().build(); + String cowsayLock = new String(Files.readAllBytes( + new File(baseA, "uv.lock").toPath()), StandardCharsets.UTF_8); + + // ...does not satisfy a manifest that additionally requires `requests`. + // With --frozen, uv must reject the stale lock. (Without --frozen, uv + // would simply add requests and succeed — so this failure proves the + // --frozen flag is actually wired in, not just that a corrupt lock fails.) + String pyprojectExtra = + "[project]\n" + + "name = \"cowsay-extra\"\n" + + "version = \"0.1.0\"\n" + + "requires-python = \">=3.10\"\n" + + "dependencies = [\"cowsay>=6.0\", \"appose>=0.1.0\", \"requests\"]\n"; + assertThrows(BuildException.class, () -> + Appose.uv().content(pyprojectExtra) + .base("target/envs/uv-lock-stale").lockContent(cowsayLock).logDebug().build()); + } + + /** + * Lock files only apply to the pyproject.toml / uv sync path; the + * requirements.txt path uses pip install and has no lockfile. + */ + @Test + public void testUvLockUnsupportedScheme() { + assertThrows(IllegalArgumentException.class, () -> + Appose.uv("src/test/resources/envs/cowsay-requirements.txt") + .base("target/envs/uv-lock-reqs").lockContent("bogus").build()); + } + + /** + * When no lock is supplied, appose.json must NOT contain a lockHash key, so + * the snapshot stays byte-identical to pre-lock-file builds (backward + * compatibility) and existing environments are never spuriously rebuilt. + */ + @Test + public void testUvNoLockBackwardCompat() throws Exception { + String base = "target/envs/uv-no-lock"; + FilePaths.deleteRecursively(new File(base)); + Environment env = Appose.uv("src/test/resources/envs/cowsay-pyproject.toml") + .base(base).logDebug().build(); + assertFalse(apposeJsonMap(new File(base)).containsKey("lockHash"), + "appose.json must NOT contain lockHash when no lock is supplied"); + cowsayAndAssert(env, "nolock"); + } + + /** + * Changing the lock content must change the lockHash in appose.json and thus + * force a rebuild. The lock is edited with a trailing TOML comment, which + * doesn't change the resolved package set, so --frozen still succeeds. + */ + @Test + public void testUvLockChangeTriggersRebuild() throws Exception { + String baseA = "target/envs/uv-lock-change-src"; + FilePaths.deleteRecursively(new File(baseA)); + Appose.uv("src/test/resources/envs/cowsay-pyproject.toml").base(baseA).logDebug().build(); + String lock = new String(Files.readAllBytes(new File(baseA, "uv.lock").toPath()), StandardCharsets.UTF_8); + + String base = "target/envs/uv-lock-change"; + FilePaths.deleteRecursively(new File(base)); + Appose.uv("src/test/resources/envs/cowsay-pyproject.toml") + .base(base).lockContent(lock).logDebug().build(); + String hashBefore = (String) apposeJsonMap(new File(base)).get("lockHash"); + + Appose.uv("src/test/resources/envs/cowsay-pyproject.toml") + .base(base).lockContent(lock + "# trailing comment\n").logDebug().build(); + String hashAfter = (String) apposeJsonMap(new File(base)).get("lockHash"); + + assertNotNull(hashBefore, "lockHash should be present after a locked build"); + assertNotNull(hashAfter, "lockHash should be present after rebuild"); + assertNotEquals(hashBefore, hashAfter, + "a lock change must produce a different lockHash and force a rebuild"); + } + + /** + * wrap() captures the lock file into builder state, so rebuild() reproduces + * the locked environment even after its directory has been deleted. + */ + @Test + public void testUvWrapLockSurvivesRebuild() throws Exception { + // Build a locked environment (generate a valid lock first). + String srcBase = "target/envs/uv-wrap-src"; + FilePaths.deleteRecursively(new File(srcBase)); + Appose.uv("src/test/resources/envs/cowsay-pyproject.toml").base(srcBase).logDebug().build(); + String lock = new String(Files.readAllBytes(new File(srcBase, "uv.lock").toPath()), StandardCharsets.UTF_8); + + String baseA = "target/envs/uv-wrap-locked"; + FilePaths.deleteRecursively(new File(baseA)); + Appose.uv("src/test/resources/envs/cowsay-pyproject.toml") + .base(baseA).lockContent(lock).logDebug().build(); + assertTrue(apposeJsonMap(new File(baseA)).containsKey("lockHash")); + + // Wrap the locked env via the typed builder (captures pyproject.toml + uv.lock), + // then wipe + rebuild. (Appose.wrap() auto-detection does not recognize uv envs, + // so the typed UvBuilder.wrap() is the path that captures the lock.) + Environment env = Appose.uv().wrap(new File(baseA)); + Environment rebuilt = env.rebuild(); + + assertTrue(apposeJsonMap(new File(baseA)).containsKey("lockHash"), + "rebuild after wrap must reproduce lockHash from the captured lock"); + cowsayAndAssert(rebuilt, "rewrapped"); + } }