From 894fe541cc28d82e38758d5c6d66aa579760d2d9 Mon Sep 17 00:00:00 2001 From: Claude Date: Fri, 11 Sep 2026 08:41:02 +0000 Subject: [PATCH] fix(s390x): guard the VXE-only helpers upstream left at file scope The Linux s390x job has been red since b10902 landed, with nothing in this project involved. Upstream #28667 added a new source file, ggml/src/ggml-cpu/arch/s390/repack.cpp (absent at b10883), which ggml-cpu/CMakeLists.txt compiles unconditionally for s390x. Every function body in it is guarded #if defined(__VXE__) || defined(__VXE2__), but three static inline helpers -- vxe_dot_acc, vxe_splat_granule, vxe_fold -- sit at file scope between two guarded blocks with no guard of their own, and their signatures name int16x8_t / int8x16_t / int32x4_t, which ggml-cpu-impl.h only typedefs inside that same guard. A non-VXE s390x build therefore dies with three "does not name a type" errors before reaching any of the code it is meant to skip. patches/0013 wraps the three definitions in the identical guard; all 21 call sites are already inside __VXE__ blocks, so nothing moves. Why this build is non-VXE is the half that is ours, and it is not obvious: ggml declares option(GGML_VXE "ggml: enable vxe" ${GGML_NATIVE}), so the VXE default follows GGML_NATIVE. The job passes -DGGML_NATIVE=OFF, which is correct for a cross-build -- an x86 host must not bake -march=native into an s390x artifact -- and silently switches VXE off with it. The job has always produced a scalar s390x binary, which is right for what it is (a big-endian correctness gate for our own layer, not a performance target); it was simply invisible until a file arrived that does not compile that way. A comment on the build step now says so. The flag-side alternatives were measured with the real cross toolchain, not reasoned about. -DGGML_VXE=ON alone is strictly worse: the self-define sets __VXE__ and __VXE2__ together as soon as __VEC__ exists, while -march stays at the toolchain default arch11, so three errors become dozens of "'__builtin_s390_vec_*' matching variant requires z14 or higher". Adding -march=z15 on top does compile, but raises the shipped artifact's hardware floor to z15 and makes the qemu ctest gate depend on VXE2 emulation -- a real trade for vector kernels this job never uses. The patch keeps exactly the configuration that worked through b10883. Verified: unpatched + the job's own flags reproduces the three CI errors verbatim; patched + those flags compiles clean; patched + -mvx -mzvector -march=z15 also compiles clean, so a future vector build is not foreclosed. Fresh configure applies all ten patches (stamp head 481c65f0), full Release build clean, ctest 537/537, NativeLibraryLoadSmokeTest 4/4 after a clean. The docs/history row for b10883-b10902 carried a claim that this disproves -- "the range does not touch our layer" was right about behaviour and wrong about the build -- and is corrected in place rather than left to mislead. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_01AnNYn8W1xuVxVJtyL34GyH --- .github/workflows/publish.yml | 9 +++++++++ CLAUDE.md | 1 + docs/history/llama-cpp-breaking-changes.md | 2 +- ...-s390x-repack-guard-vxe-only-helpers.patch | 20 +++++++++++++++++++ 4 files changed, 31 insertions(+), 1 deletion(-) create mode 100644 llama/patches/0013-s390x-repack-guard-vxe-only-helpers.patch diff --git a/.github/workflows/publish.yml b/.github/workflows/publish.yml index 084f4c9a..b6bc86d4 100644 --- a/.github/workflows/publish.yml +++ b/.github/workflows/publish.yml @@ -795,6 +795,15 @@ jobs: run: | sudo apt-get update sudo apt-get install -y gcc-s390x-linux-gnu g++-s390x-linux-gnu qemu-user-static + # NOTE: GGML_NATIVE=OFF is required here (an x86 build host must not bake -march=native into an + # s390x artifact), and it has a non-obvious second effect: ggml declares + # `option(GGML_VXE "ggml: enable vxe" ${GGML_NATIVE})`, so VXE is off too. No `-mvx -mzvector` is + # passed, `__VEC__` stays undefined, and ggml-cpu-impl.h's `#if defined(__s390x__) && defined(__VEC__)` + # never self-defines `__VXE__`/`__VXE2__`. This job therefore builds a SCALAR s390x binary -- which is + # exactly right for what it is (a big-endian correctness gate for our own layer, not a perf target). + # Do NOT "fix" a VXE-related compile error by adding -DGGML_VXE=ON: that define sets __VXE__ and + # __VXE2__ together while -march stays at the toolchain default arch11, so every z14+ builtin is + # rejected. See the patches/0013 row in CLAUDE.md for the measured comparison. - name: Build libraries (cross-compile s390x) shell: bash run: | diff --git a/CLAUDE.md b/CLAUDE.md index 780b2445..2e559254 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -749,6 +749,7 @@ Current patches: | `0011-peg-parser-lenient-invalid-utf8.patch` | **A model that emits one malformed UTF-8 byte turns a finished generation into an HTTP 500.** The server parses *every* completion through `common_chat_parse()`; with no chat parser configured (plain `/completion`) that is the content-only fallback `content(rest()) + end()`, whose scan is `common_peg_until_parser` (`common/peg-parser.cpp`). `common_chat_peg_parse()` always parses in **lenient** mode, and that scan tolerates an `INCOMPLETE` trailing UTF-8 sequence by keeping the text before it — but the `INVALID` branch right below it returns `FAIL` unconditionally, ignoring leniency. One stray byte anywhere in the generated text therefore throws `"The model produced output that does not match the expected Content-only format"` and the request 500s even though generation completed normally (`stop processing: n_tokens = 4, truncated = 0`). The patch makes the `INVALID` branch respect `ctx.is_lenient()` exactly like the `INCOMPLETE` branch — keep the text up to the malformed byte — and adds an upstream `tests/peg-parser/test-unicode.cpp` case pinning both the lenient and the still-failing strict behavior. **Strict mode is unchanged**, which is what keeps upstream's own tests green: `tests/peg-parser/test-unicode.cpp` *does* assert `FAIL` on invalid UTF-8 through the *until* parser (a `malformed UTF-8` block with three `p.until("")` cases), but each builds a bare `common_peg_parse_context` with no `COMMON_PEG_PARSE_FLAG_LENIENT`, so the lenient-only change cannot reach them. This patch adds its case inside that same block. Found by `NativeServerAttachIntegrationTest.completion_overHttp_served`, which 500s on all six Java CI platforms. Upstream-submittable; **not yet filed upstream**. Touches only `common/peg-parser.cpp` + that test, which no other patch touches, so it is independent of `0001`/`0006`/`0007`. Runnable guard: the `ContentOnlyParseUtf8` tests in `src/test/cpp/test_utils.cpp` — unlike the upstream test they are compiled and run in CI on every platform, so a bump that drops this patch reds `C++ Tests` instead of one Java job. | | `0006-server-embed-native-server-jni.patch` | **Makes `server.cpp`'s `llama_server` embeddable in the JVM** so the `NativeServer` JNI bridge can run the full upstream HTTP server (WebUI included) inside `libjllama` — see "Two server modes" below. b9870 already exposes `int llama_server(int, char**)` (non-static; no `main` in the file), so the patch only adds embedded-mode support: (1) a `g_llama_server_embedded` flag + `llama_server_set_embedded()` / `llama_server_request_shutdown()` (declared in the committed `src/main/cpp/native_server_bridge.h`); (2) skips installing the process-wide SIGINT/SIGTERM handlers when embedded (they would hijack the JVM's); (3) in embedded mode parses the **forwarded** argv via `common_params_parse` instead of `common_params_parse_main` (whose `GetCommandLineW` recovery would pick up `java.exe`'s command line — the same Windows class of bug `0001` fixes). `llama_server_request_shutdown()` mirrors the SIGTERM path (invokes the installed `shutdown_handler` → `ctx_server.terminate()` unblocks `start_loop()`), giving JNI an out-of-band stop since `ctx_server` is loop-local. Applies **after `0001`** (which flips this call site to `common_params_parse_main`), so its context is the post-`0001` tree; regenerate against `0001`+source on a bump. Only touches `tools/server/server.cpp`. | | `0012-model-guard-zero-split-sum-and-name-the-device-index.patch` | **A GPU that reports zero free memory makes every model load fail with the unactionable `error loading model: vector`.** `llama_model_base::load_tensors` (`src/llama-model.cpp`) weights the per-device layer split by `ggml_backend_dev_memory()`'s `free`, then normalises: `splits[i] /= split_sum`. With a single device reporting `free == 0` that is `0/0` → **NaN** in every split point; NaN compares false against everything, so the `std::upper_bound` below returns the end iterator, `layer_gpu == n_devices()`, and `devices.at(layer_gpu)` throws `std::out_of_range` — whose libc++ `what()` is the bare string `"vector"`, which `llama.cpp`'s `catch (const std::exception &)` prints verbatim. Upstream's `free == 0 && total == 0` host-memory fallback does **not** fire, because `total` is `recommendedMaxWorkingSetSize` and is non-zero. **Reachable since b10618..b10797**: upstream `8c0b9cd04` ("metal : fix memory query under low-memory conditions", [#27701](https://github.com/ggml-org/llama.cpp/pull/27701)) changed `ggml-metal-device.m` to `*free = *total > cur ? *total - cur : 0`; before that clamp an over-committed device (`currentAllocatedSize > recommendedMaxWorkingSetSize`) *underflowed* to a huge `size_t`, which normalised fine, so the same precondition was harmless. That is why the `Java Tests macOS …` jobs went red at the b10792→b10797 step while every Linux/Windows job stayed green — **and why only a GPU build can fail this way at all**: `act_gpu_layers` is `devices.empty() ? 0 : …`, so with no GPU backend `devices` is empty, every layer returns early on `cpu_dev`, and the `.at()` line is unreachable. **Shape:** the two blocks are lifted out of `load_tensors` into free functions declared in `src/llama-model.h`, purely so they can be driven by a test — the failing state needs a real over-committed GPU and cannot be arranged through any public API. `llama_model_splits_normalize()` carries **the fix**: on `split_sum == 0` it `LLAMA_LOG_WARN`s and falls back to an even split (`splits[i] = float(i+1)/splits.size()`), the only neutral choice when no device can be preferred and exactly right for a single device. `llama_model_splits_select_device()` carries **the diagnostic**: it bounds-checks the index and throws a `std::runtime_error` naming the function, the offloaded layer, the device index, the split-point count **and the split points themselves** — with NaN splits that message prints `nan` and names the cause outright, which is precisely what was missing when this had to be diagnosed by reading source. **A second, backend-independent trigger reaches the same line**, found while writing this up and verified against the unfixed library: `--tensor-split` values are parsed with `std::stof` and never range-checked (`common/arg.cpp`), so `-ts 1,-1` cancels out, `split_sum` is 0 again, the split points become `[inf, -nan]`, and every layer maps one past the last device — on CUDA, Vulkan or ROCm just as much as on Metal, with no memory pressure involved. That is what makes this an ordinary upstream defect rather than a Metal edge case, and the warning names both causes rather than only the memory one. Also adds upstream `tests/test-model-split.cpp` (5 cases in upstream's `testing.h` style) + its `llama_build_and_test` registration. Touches `src/llama-model.{cpp,h}`, `tests/test-model-split.cpp` and `tests/CMakeLists.txt` — **none** of which any other patch touches, so it is independent of all of them. Upstream-submittable ("model: fall back to an even split when no device reports free memory"); **not yet filed upstream**. **Runnable guard: `src/test/cpp/test_model_split.cpp`** — a FetchContent subproject builds with `LLAMA_BUILD_TESTS=OFF`, so the upstream test above is applied-but-never-compiled here (same as `0001`'s test). That file drives the same two functions from `jllama_test`, which runs on **every** platform in `C++ Tests`, so a bump that drops this patch fails the build at link time everywhere instead of surfacing as one red macOS Java job. **Verification limit — read before assuming this can be dropped:** the *failing path* still cannot be reached without a GPU backend, so the guard pins the arithmetic (what actually broke), not the end-to-end load; the end-to-end proof is the macOS CI job. On a bump, re-check whether upstream added its own `split_sum == 0` guard (grep `split_sum` in `src/llama-model.cpp`) and **drop this patch rather than refreshing it** if they did — the fail-loud applier detects "does not apply", never "upstream already fixed this". | +| `0013-s390x-repack-guard-vxe-only-helpers.patch` | **A new upstream file makes the s390x build fail to compile, with nothing in this project involved.** b10902 added `ggml/src/ggml-cpu/arch/s390/repack.cpp` (upstream #28667, s390x q4_0 repack; the file does not exist at b10883). Every *function body* in it is guarded `#if defined(__VXE__) || defined(__VXE2__)`, but three `static inline` helpers — `vxe_dot_acc`, `vxe_splat_granule`, `vxe_fold` — sit at file scope **between** two guarded blocks with no guard of their own, and their signatures name `int16x8_t` / `int8x16_t` / `int32x4_t`, which `ggml-cpu-impl.h` only typedefs *inside* that same guard. So a non-VXE s390x build dies with three `does not name a type` errors before it reaches any of the code it is supposed to skip. The patch wraps the three definitions in the identical guard — all 21 call sites are already inside `__VXE__` blocks, so nothing else moves. **Why this project builds s390x without VXE, which is the half that is ours:** `ggml/CMakeLists.txt` declares `option(GGML_VXE "ggml: enable vxe" ${GGML_NATIVE})` — the VXE default *follows* `GGML_NATIVE`. The `build-linux-s390x` job passes `-DGGML_NATIVE=OFF`, which is correct for a cross-build (it must not bake the x86 build host's `-march=native` into an s390x artifact) but also silently switches VXE off. No `-mvx -mzvector` is then passed, `__VEC__` is undefined, and `ggml-cpu-impl.h`'s `#if defined(__s390x__) && defined(__VEC__)` self-define of `__VXE__`/`__VXE2__` never fires. The job has therefore always produced a **scalar** s390x binary; that was invisible until a file arrived that does not compile that way. **Do not "fix" this with `-DGGML_VXE=ON`** — measured, not assumed: that makes it strictly worse, because the self-define above sets `__VXE__` *and* `__VXE2__` together as soon as `__VEC__` exists, while `-march` stays at the toolchain default `arch11`, so the three errors become dozens of `'__builtin_s390_vec_*' matching variant requires z14 or higher`. The only flag-side alternative is `-DGGML_VXE=ON` **plus** `-march=z15`, which does compile but raises the shipped artifact's hardware floor to z15 and makes the qemu `ctest` gate depend on VXE2 emulation — a real trade for vector kernels this job does not use (it is a big-endian *correctness* gate for our own layer, not a performance target). The patch keeps the configuration that worked through b10883 and changes nothing about the artifact. **Verified with the real cross toolchain** (`s390x-linux-gnu-g++`), not by inspection: unpatched + CI's flags reproduces the three CI errors exactly; patched + CI's flags compiles clean; patched + `-mvx -mzvector -march=z15` also compiles clean, so the patch does not foreclose a future vector build. Touches only that one file, which no other patch touches. Upstream-submittable ("ggml-cpu: guard the VXE-only helpers in the s390x repack path"); **not yet filed upstream**. **On a bump, check whether upstream guarded them itself and DROP this patch rather than refreshing it** — the fail-loud applier detects "does not apply", never "upstream already fixed this". There is no runnable guard for it beyond CI: the file is compiled only for s390x, so `build-linux-s390x` *is* the test, and it fails loudly at compile time. | **`0009` was dropped at the b10280 bump.** Upstream merged [sheredom/subprocess.h#104](https://github.com/sheredom/subprocess.h/pull/104) — the exact fix this diff --git a/docs/history/llama-cpp-breaking-changes.md b/docs/history/llama-cpp-breaking-changes.md index 02b1ca3c..7b3e0a99 100644 --- a/docs/history/llama-cpp-breaking-changes.md +++ b/docs/history/llama-cpp-breaking-changes.md @@ -704,7 +704,7 @@ Used during `llama.cpp` version bumps: when upgrading, scan this file from the r | b10870–b10878 | patches + upstream verification | **All nine patches still apply, and `0012` is still required.** The range touches two patch targets — `common/arg.cpp` (`0001`) and `src/llama-model.{cpp,h}` (`0012`) — so both were checked against the pristine tag rather than assumed. `0001`: `b10878:common/arg.cpp` still carries the `#ifdef _WIN32` count-guarded `argv = utf8.ptrs.data()` override, and `common_params_parse_main` appears **0 times** in `b10878:common/arg.h`, so upstream has still not adopted the fix. `0012`: `b10878:src/llama-model.cpp` still normalises with a bare `splits[i] /= split_sum;` and has **no `split_sum == 0` guard** of its own — the CLAUDE.md instruction to *drop rather than refresh* this patch does not fire, and its `llama-model` diff is only the new enumerator. Verified for real: fresh `cmake -S llama -B /tmp/b10878-build -DBUILD_TESTING=ON` through the real `FetchContent` path, configure clean, stamp written at head `4850c7727fa73bbe3098e10ee369fbc3467c445f` (= `b10878`) with **all nine hashes recorded**; full `cmake --build --config Release` clean; `ctest` **527/527**, including the four `LlamaModelSplits.*` cases that are the only place `0012`'s two extracted functions are linked in CI. `nm -D` on the fresh `libjllama.so` reports **40** `Java_*` exports. `mvn -pl llama clean test -Dtest=NativeLibraryLoadSmokeTest` **4/4, 0 skipped**, including `nativeBuildInfoMatchesPinnedVersionConstant` — the end-to-end proof that the four pin sites and the linked binary agree. Run with `clean`: `LLAMA_CPP_VERSION` is a compile-time constant javac inlines into the test class, and Maven's incremental compilation cannot see that dependency. | | b10878–b10883 | **Nothing on the review surface.** The raw diff is large — 29 files, 2360 insertions, 2276 deletions, **426 KiB, well over the runbook's 100 KiB chunking threshold** — and was bumped straight through anyway, for a reason that is recorded here rather than asserted: the byte count is entirely GPU backend internals and Python pins. Restricting `git diff --stat` to the paths this project actually compiles, links or includes (`common/`, `include/`, `tools/server/`, `tools/mtmd/`, `ggml/include/`, `src/`, the top-level `CMakeLists.txt`) leaves **one file, one line**: `tools/server/tests/requirements.txt`, a Python test-requirement pin that is neither compiled nor linked. The remainder is `ggml/src/ggml-vulkan/**` (1804 lines in `ggml-vulkan.cpp` plus ~15 shader files), `ggml/src/ggml-hexagon/**`, `tests/test-backend-ops.cpp`, and six `requirements*.txt` / `pyproject.toml` version pins. **Zero** priority-8 headers moved; `common/arg.h`, `common/chat.h`, `include/llama.h`, `tools/mtmd/mtmd-helper.h` are all byte-identical. This is the same shape as the b10819–b10850 row: a headline number dominated by backends the project builds but whose internals it never calls. The one thing the size *does* imply is CI cost — the Vulkan rewrite is upstream-compiled code the `vulkan-linux-*` and `vulkan-windows-*` classifier jobs must still build, so a compile break there would surface in those jobs rather than in any project source. | | b10878–b10883 | patches + upstream verification | **All nine patches apply, and not one needed refreshing — every patch-target file is byte-unchanged in the range.** Checked file by file rather than inferred from the aggregate: `common/arg.cpp`, `common/arg.h`, `common/peg-parser.cpp`, `tools/server/server.cpp`, `tools/server/server-context.{cpp,h}`, `tools/server/server-models.cpp`, `src/llama-model.{cpp,h}` and `tests/CMakeLists.txt` all report no diff between the two tags. The two standing drop-checks were still run against the pristine tag, because the fail-loud applier detects "does not apply" but never "upstream already fixed this": `0001` — `common_params_parse_main` appears **0 times** in `b10883:common/arg.h` and the `#ifdef _WIN32` `argv = utf8.ptrs.data()` override is still at `common/arg.cpp:1282`, so it stays; `0012` — `b10883:src/llama-model.cpp:1489` still normalises with a bare `splits[i] /= split_sum;` and has **no `split_sum == 0` guard**, so the CLAUDE.md instruction to *drop rather than refresh* does not fire. Verified for real: `rm -rf build && cmake -B build -DBUILD_TESTING=ON` through the real `FetchContent` path, configure clean, stamp written at head `91f6a6cf361385700bbe15981f0f39909df77498` (= `b10883`) with **all nine hashes recorded**; full `cmake --build --config Release` clean; `ctest` **531/531**. That total is 4 up from b10878 because this is the first bump after `test_model_flags.cpp` landed — and it is the first bump whose **flag contract** was machine-checked rather than reasoned about: `JavaCliFlagContract` re-derives the 138 flags the Java layer emits and re-runs them through `common_params_parser_init(params, LLAMA_EXAMPLE_SERVER)` at the new tag, so "`arg.cpp` is unchanged, therefore the registered option set is unchanged" is now an assertion the build makes, not an inference a reviewer makes. `nm -D` on the fresh `libjllama.so` reports **40** `Java_*` exports. `mvn -pl llama clean test -Dtest=NativeLibraryLoadSmokeTest` **4/4, 0 skipped**, including `nativeBuildInfoMatchesPinnedVersionConstant` — the end-to-end proof that the four pin sites and the linked binary agree (`clean` is required: javac inlines `LLAMA_CPP_VERSION` into the test class and Maven's incremental compile cannot see that dependency). Full `mvn test`: **1759 tests, 0 failures, 0 errors**. | -| b10883–b10902 | `src/llama-model.{cpp,h}` (**additive only**: two new `llm_type` enumerators, `LLM_TYPE_3B_A800M` for Granite3 MoE and `LLM_TYPE_32B_A9B` for Granite4 Hybrid, plus their `llm_type_name` cases — four lines total, far from `patches/0012`'s hunks), `common/speculative.cpp` (**#28587**: the DFlash draft implementation now skips an M-RoPE image whose rows are all pinned to one position, because a windowed draft cache cannot free cells for it — internal to `common_speculative_impl_draft_dflash`, **no signature moved**, so `common_speculative_init` / `_draft` / `_accept` are untouched), `src/llama-memory-hybrid-idx.cpp` (**#28330**: sets `n_embd_head_{k,v}_mla_impl` so `llama_kv_cache` skips allocating the unused V cache for the indexer — three lines, internal), `src/models/*` (**#28643/#28669**: Granite-family parameter-count fix and dead-switch-branch cleanup across bert / jina-bert-v3 / nomic-bert{,-moe} / granite{,-hybrid,-moe}), `tests/test-backend-ops.cpp`, `ggml/src/ggml-{opencl,vulkan,cpu}/**`, `.github/**`, `scripts/make-release-checks.sh`, `conversion/nemotron.py`. 30 files, 1106 insertions, 423 deletions, **110 KiB** raw. | **No project-source change, and not one file on the priority review list moved.** Zero files under `common/*.h`, `include/`, `tools/server/` or `tools/mtmd/`, so every row of the API-compatibility table is vacuously satisfied and the three mechanical server-contract greps have **no input to compare** — the request-field set, its `set_hard_limits` bounds and the emitted response keys cannot have moved. The one `common/` file in the range is `speculative.cpp`, an implementation TU compiled by upstream, not a header this project includes. **Sizing, and why it was bumped straight through at 110 KiB.** The runbook's threshold is 100 KiB, so `llama-next-version.sh` proposed an intermediate stop at b10901 (87 KiB). The 23 KiB that pushes b10902 over is **entirely** `ggml/src/ggml-opencl/**` — one `CMakeLists.txt` line, `ggml-opencl.cpp`, and a new `gemv_noshuffle_q4_0_f32_32b_trans.cl` kernel (#28268). Bucketed by top-level directory the whole range is 998 lines of `ggml/src` backend internals against **93 lines across 13 files** of everything this project actually compiles, links or includes — so chunking would have split a backend-kernel addition in half and reviewed nothing extra. Same call, and same reasoning, as the b10870–b10878 row. **Worth knowing for the s390x job:** two of the 19 commits are upstream's own `ggml-cpu` s390x work (#28667 q4_0 repack, #28606 Q1_0 vector intrinsics, 291 new lines under `ggml/src/ggml-cpu/arch/s390/`). Those are upstream's big-endian kernels; the `build-linux-s390x` job's qemu `ctest` gate covers *this project's* endian-sensitive layer (the little-endian WAV writer, the JSON/token/embedding transforms), which the range does not touch. | +| b10883–b10902 | `src/llama-model.{cpp,h}` (**additive only**: two new `llm_type` enumerators, `LLM_TYPE_3B_A800M` for Granite3 MoE and `LLM_TYPE_32B_A9B` for Granite4 Hybrid, plus their `llm_type_name` cases — four lines total, far from `patches/0012`'s hunks), `common/speculative.cpp` (**#28587**: the DFlash draft implementation now skips an M-RoPE image whose rows are all pinned to one position, because a windowed draft cache cannot free cells for it — internal to `common_speculative_impl_draft_dflash`, **no signature moved**, so `common_speculative_init` / `_draft` / `_accept` are untouched), `src/llama-memory-hybrid-idx.cpp` (**#28330**: sets `n_embd_head_{k,v}_mla_impl` so `llama_kv_cache` skips allocating the unused V cache for the indexer — three lines, internal), `src/models/*` (**#28643/#28669**: Granite-family parameter-count fix and dead-switch-branch cleanup across bert / jina-bert-v3 / nomic-bert{,-moe} / granite{,-hybrid,-moe}), `tests/test-backend-ops.cpp`, `ggml/src/ggml-{opencl,vulkan,cpu}/**`, `.github/**`, `scripts/make-release-checks.sh`, `conversion/nemotron.py`. 30 files, 1106 insertions, 423 deletions, **110 KiB** raw. | **No project-source change, and not one file on the priority review list moved.** Zero files under `common/*.h`, `include/`, `tools/server/` or `tools/mtmd/`, so every row of the API-compatibility table is vacuously satisfied and the three mechanical server-contract greps have **no input to compare** — the request-field set, its `set_hard_limits` bounds and the emitted response keys cannot have moved. The one `common/` file in the range is `speculative.cpp`, an implementation TU compiled by upstream, not a header this project includes. **Sizing, and why it was bumped straight through at 110 KiB.** The runbook's threshold is 100 KiB, so `llama-next-version.sh` proposed an intermediate stop at b10901 (87 KiB). The 23 KiB that pushes b10902 over is **entirely** `ggml/src/ggml-opencl/**` — one `CMakeLists.txt` line, `ggml-opencl.cpp`, and a new `gemv_noshuffle_q4_0_f32_32b_trans.cl` kernel (#28268). Bucketed by top-level directory the whole range is 998 lines of `ggml/src` backend internals against **93 lines across 13 files** of everything this project actually compiles, links or includes — so chunking would have split a backend-kernel addition in half and reviewed nothing extra. Same call, and same reasoning, as the b10870–b10878 row. **Worth knowing for the s390x job:** two of the 19 commits are upstream's own `ggml-cpu` s390x work (#28667 q4_0 repack, #28606 Q1_0 vector intrinsics, 291 new lines under `ggml/src/ggml-cpu/arch/s390/`). Those are upstream's big-endian kernels; the `build-linux-s390x` job's qemu `ctest` gate covers *this project's* endian-sensitive layer (the little-endian WAV writer, the JSON/token/embedding transforms), which the range does not touch. **Correction, added after run #940: that last clause was right about the gate and wrong about the job.** #28667 does not merely add kernels, it adds a *new source file* — `ggml/src/ggml-cpu/arch/s390/repack.cpp`, absent at b10883 — which `ggml-cpu/CMakeLists.txt` compiles unconditionally for s390x and which **does not compile in a non-VXE configuration**, so `build-linux-s390x` went red without a single line of this project's code being involved. Fixed by `patches/0013`; the mechanism and why our build is non-VXE are in that patch's row in `CLAUDE.md`. The general lesson for this table: "the range does not touch our layer" answers whether *behaviour* can change, never whether the *build* still succeeds — a new upstream file compiled for a platform is a build-surface change even when it is unreachable from our code. | | b10883–b10902 | patches + upstream verification | **All nine patches still apply, and `0001`, `0010` and `0012` are all still required.** Only one patch target moved in the range — `src/llama-model.{cpp,h}` (`0012`) — and its diff is the two additive enumerators above, nowhere near `load_tensors`. The other eight targets (`common/arg.{cpp,h}`, `common/peg-parser.cpp`, every `tools/server/*`, `tests/CMakeLists.txt`, `vendor/*`) are **byte-unchanged**, verified by diffing those paths explicitly rather than inferred from the aggregate. All three standing drop-checks were run against the pristine tag, because the fail-loud applier detects "does not apply" but never "upstream already fixed this": **`0001`** — `common_params_parse_main` appears **0 times** in `b10902:common/arg.h` and the `#ifdef _WIN32` count-guarded `argv = utf8.ptrs.data()` override is still at `common/arg.cpp:1282`; **`0010`** — `b10902:tools/server/server-context.cpp:4554` still emits `{"vocab_type", meta.model_vocab_type}` uncast, so the `common_json` enum-to-bool trap is still live; **`0012`** — `b10902:src/llama-model.cpp:1491` still normalises with a bare `splits[i] /= split_sum;` and carries no `split_sum == 0` guard. Verified for real: `rm -rf build` then `cmake -B build -DBUILD_TESTING=ON` through the real `FetchContent` path, configure clean, stamp written at head `df03399b885831b2a1603b3abb0d8c156808e363` (= `b10902`) with **all nine SHA-256 lines**; full `cmake --build --config Release` clean; `ctest` **537/537**, including the seven `LlamaModelSplits.*` cases that are the only place `0012`'s two extracted functions are linked in CI, and the six `test_wire_contracts.cpp` cases whose configure-time `OAI_LAYER` reader sweep re-ran against b10902's sources (138 CLI / 57 request / 15 trainer names extracted, unchanged). `nm -D` on the fresh `libjllama.so`: **40** `Java_*` exports, **0** C++-mangled ones. `mvn -pl llama clean test -Dtest=NativeLibraryLoadSmokeTest` **4/4, 0 skipped**, including `nativeBuildInfoMatchesPinnedVersionConstant` — the end-to-end proof that the four pin sites and the linked binary agree. Run with `clean`: `LLAMA_CPP_VERSION` is a compile-time constant javac inlines into the test class, and Maven's incremental compilation cannot see that dependency. | | b10902–b10903 | `ggml/src/ggml-vulkan/vulkan-shaders/argsort.comp` + `argsort_large.comp` — and nothing else. One commit (**#28705**, "vulkan: fix data race and OOB access in argsort(large)"), 2 files, 21 insertions, 12 deletions, **3 KiB**. | **The smallest bump in this file's history, and a genuinely empty review surface.** Upstream fixes two defects in the Vulkan bitonic argsort. The *OOB read*: the initialising write now leaves `value.y` at 0 for a padded column instead of reading `data_a[row_offset + col]` past `p.ncols` (both shaders). The *data race*: the compare-exchange body is hoisted **inside** the `ixj > col` guard, so only one thread of each pair touches the two shared-memory slots — previously both threads ran the read-modify-write and only the final store was guarded, i.e. the partner thread read `dst_row[idx_0]`/`dst_row[idx_1]` concurrently with its own unsynchronised write. No C++, no header, no build-system file, no Python pin. Every row of the API-compatibility table is **vacuously satisfied** and the three mechanical `tools/server/` contract greps have **no input** — for the second bump running. Well under the 100 KiB chunking threshold, so no chunking question arises. **Where it does land:** GLSL under `ggml/src/ggml-vulkan/vulkan-shaders/` is compiled by `glslc` at build time and embedded, so the change reaches exactly the three Vulkan classifier artifacts (`vulkan-linux-x86-64`, `vulkan-linux-aarch64`, `vulkan-windows-x86-64`). All three are **build-only** jobs on GPU-less runners, so CI proves the shaders still compile, not that the race is fixed — that needs real Vulkan hardware, which no job here has. The default JAR and every non-Vulkan classifier are bit-for-bit unaffected by this range. | | b10902–b10903 | patches + upstream verification | **All nine patches apply untouched, and the intersection with `patches/` is empty by inspection rather than by aggregate:** the two changed files are Vulkan shaders, which no patch in this repo touches. The three standing drop-checks were nevertheless run against the pristine tag rather than waved through on that basis — the fail-loud applier detects "does not apply" but never "upstream already fixed this", and a drop-check firing is a reason to **delete** a patch, which no amount of "the diff is small" substitutes for. **`0001`** — `common_params_parse_main` appears **0 times** in `b10903:common/arg.h` and the `#ifdef _WIN32` count-guarded `argv = utf8.ptrs.data()` override is still at `common/arg.cpp:1282`; **`0010`** — `b10903:tools/server/server-context.cpp:4554` still emits `{"vocab_type", meta.model_vocab_type}` uncast, so the `common_json` enum-to-bool trap is still live; **`0012`** — `b10903:src/llama-model.cpp:1491` still normalises with a bare `splits[i] /= split_sum;` and carries no `split_sum == 0` guard. All three line numbers are **identical to b10902**, which is what a byte-unchanged file looks like. Verified for real: `rm -rf build` then `cmake -B build -DBUILD_TESTING=ON` through the real `FetchContent` path, configure clean, stamp written at head `481c65f091f74c5e7089dd0a3a1cc6b50cced31e` (= `b10903`) with **all nine SHA-256 lines**; full `cmake --build --config Release` clean, zero errors; `ctest` **537/537**; wire-name extraction unchanged at **138 CLI / 57 request / 15 trainer** names (the configure-time `OAI_LAYER` reader sweep re-ran against b10903's sources). `nm -D` on the fresh `libjllama.so`: **40** `Java_*` exports, **0** C++-mangled ones. `mvn -pl llama clean test -Dtest=NativeLibraryLoadSmokeTest` **4/4, 0 skipped**, including `nativeBuildInfoMatchesPinnedVersionConstant` — run with `clean` because javac inlines `LLAMA_CPP_VERSION` into the test class and Maven's incremental compile cannot see that dependency. Full `mvn test`: **1755 run, 0 failures, 0 errors** (269 skipped — the model-gated classes, no GGUF in this sandbox). SpotBugs **0** findings; `spotless:check` clean; `javadoc:jar` BUILD SUCCESS. | diff --git a/llama/patches/0013-s390x-repack-guard-vxe-only-helpers.patch b/llama/patches/0013-s390x-repack-guard-vxe-only-helpers.patch new file mode 100644 index 00000000..c4586592 --- /dev/null +++ b/llama/patches/0013-s390x-repack-guard-vxe-only-helpers.patch @@ -0,0 +1,20 @@ +diff --git a/ggml/src/ggml-cpu/arch/s390/repack.cpp b/ggml/src/ggml-cpu/arch/s390/repack.cpp +index 3990a6b04..ca6f38201 100644 +--- a/ggml/src/ggml-cpu/arch/s390/repack.cpp ++++ b/ggml/src/ggml-cpu/arch/s390/repack.cpp +@@ -70,6 +70,7 @@ void ggml_quantize_mat_q8_0_4x4(const float * GGML_RESTRICT x, void * GGML_RESTR + #endif + } + ++#if defined(__VXE__) || defined(__VXE2__) + static inline int16x8_t vxe_dot_acc(const int8x16_t v_x, const int8x16_t v_y, const int16x8_t v_acc) { + return vec_meadd(v_x, v_y, vec_moadd(v_x, v_y, v_acc)); + } +@@ -84,6 +85,7 @@ static inline int32x4_t vxe_fold(const int16x8_t v_sumi) { + const int16x8_t v_ones = vec_splats((int16_t)1); + return vec_add(vec_mule(v_sumi, v_ones), vec_mulo(v_sumi, v_ones)); + } ++#endif // __VXE__ || __VXE2__ + + void ggml_gemv_q4_0_4x4_q8_0(int n, float * GGML_RESTRICT s, size_t bs, const void * GGML_RESTRICT vx, const void * GGML_RESTRICT vy, int nr, int nc) { + const int qk = QK8_0;