fix(s390x): guard the VXE-only helpers upstream left at file scope

.github/workflows/publish.yml

@@ -795,6 +795,15 @@ jobs:
         run: |
           sudo apt-get update
           sudo apt-get install -y gcc-s390x-linux-gnu g++-s390x-linux-gnu qemu-user-static
+      # NOTE: GGML_NATIVE=OFF is required here (an x86 build host must not bake -march=native into an
+      # s390x artifact), and it has a non-obvious second effect: ggml declares
+      # `option(GGML_VXE "ggml: enable vxe" ${GGML_NATIVE})`, so VXE is off too. No `-mvx -mzvector` is
+      # passed, `__VEC__` stays undefined, and ggml-cpu-impl.h's `#if defined(__s390x__) && defined(__VEC__)`
+      # never self-defines `__VXE__`/`__VXE2__`. This job therefore builds a SCALAR s390x binary -- which is
+      # exactly right for what it is (a big-endian correctness gate for our own layer, not a perf target).
+      # Do NOT "fix" a VXE-related compile error by adding -DGGML_VXE=ON: that define sets __VXE__ and
+      # __VXE2__ together while -march stays at the toolchain default arch11, so every z14+ builtin is
+      # rejected. See the patches/0013 row in CLAUDE.md for the measured comparison.
       - name: Build libraries (cross-compile s390x)
         shell: bash
         run: |

CLAUDE.md

@@ -749,6 +749,7 @@ Current patches:
 | `0011-peg-parser-lenient-invalid-utf8.patch` | **A model that emits one malformed UTF-8 byte turns a finished generation into an HTTP 500.** The server parses *every* completion through `common_chat_parse()`; with no chat parser configured (plain `/completion`) that is the content-only fallback `content(rest()) + end()`, whose scan is `common_peg_until_parser` (`common/peg-parser.cpp`). `common_chat_peg_parse()` always parses in **lenient** mode, and that scan tolerates an `INCOMPLETE` trailing UTF-8 sequence by keeping the text before it — but the `INVALID` branch right below it returns `FAIL` unconditionally, ignoring leniency. One stray byte anywhere in the generated text therefore throws `"The model produced output that does not match the expected Content-only format"` and the request 500s even though generation completed normally (`stop processing: n_tokens = 4, truncated = 0`). The patch makes the `INVALID` branch respect `ctx.is_lenient()` exactly like the `INCOMPLETE` branch — keep the text up to the malformed byte — and adds an upstream `tests/peg-parser/test-unicode.cpp` case pinning both the lenient and the still-failing strict behavior. **Strict mode is unchanged**, which is what keeps upstream's own tests green: `tests/peg-parser/test-unicode.cpp` *does* assert `FAIL` on invalid UTF-8 through the *until* parser (a `malformed UTF-8` block with three `p.until("</tag>")` cases), but each builds a bare `common_peg_parse_context` with no `COMMON_PEG_PARSE_FLAG_LENIENT`, so the lenient-only change cannot reach them. This patch adds its case inside that same block. Found by `NativeServerAttachIntegrationTest.completion_overHttp_served`, which 500s on all six Java CI platforms. Upstream-submittable; **not yet filed upstream**. Touches only `common/peg-parser.cpp` + that test, which no other patch touches, so it is independent of `0001`/`0006`/`0007`. Runnable guard: the `ContentOnlyParseUtf8` tests in `src/test/cpp/test_utils.cpp` — unlike the upstream test they are compiled and run in CI on every platform, so a bump that drops this patch reds `C++ Tests` instead of one Java job. |
 | `0006-server-embed-native-server-jni.patch` | **Makes `server.cpp`'s `llama_server` embeddable in the JVM** so the `NativeServer` JNI bridge can run the full upstream HTTP server (WebUI included) inside `libjllama` — see "Two server modes" below. b9870 already exposes `int llama_server(int, char**)` (non-static; no `main` in the file), so the patch only adds embedded-mode support: (1) a `g_llama_server_embedded` flag + `llama_server_set_embedded()` / `llama_server_request_shutdown()` (declared in the committed `src/main/cpp/native_server_bridge.h`); (2) skips installing the process-wide SIGINT/SIGTERM handlers when embedded (they would hijack the JVM's); (3) in embedded mode parses the **forwarded** argv via `common_params_parse` instead of `common_params_parse_main` (whose `GetCommandLineW` recovery would pick up `java.exe`'s command line — the same Windows class of bug `0001` fixes). `llama_server_request_shutdown()` mirrors the SIGTERM path (invokes the installed `shutdown_handler` → `ctx_server.terminate()` unblocks `start_loop()`), giving JNI an out-of-band stop since `ctx_server` is loop-local. Applies **after `0001`** (which flips this call site to `common_params_parse_main`), so its context is the post-`0001` tree; regenerate against `0001`+source on a bump. Only touches `tools/server/server.cpp`. |
 | `0012-model-guard-zero-split-sum-and-name-the-device-index.patch` | **A GPU that reports zero free memory makes every model load fail with the unactionable `error loading model: vector`.** `llama_model_base::load_tensors` (`src/llama-model.cpp`) weights the per-device layer split by `ggml_backend_dev_memory()`'s `free`, then normalises: `splits[i] /= split_sum`. With a single device reporting `free == 0` that is `0/0` → **NaN** in every split point; NaN compares false against everything, so the `std::upper_bound` below returns the end iterator, `layer_gpu == n_devices()`, and `devices.at(layer_gpu)` throws `std::out_of_range` — whose libc++ `what()` is the bare string `"vector"`, which `llama.cpp`'s `catch (const std::exception &)` prints verbatim. Upstream's `free == 0 && total == 0` host-memory fallback does **not** fire, because `total` is `recommendedMaxWorkingSetSize` and is non-zero. **Reachable since b10618..b10797**: upstream `8c0b9cd04` ("metal : fix memory query under low-memory conditions", [#27701](https://github.com/ggml-org/llama.cpp/pull/27701)) changed `ggml-metal-device.m` to `*free = *total > cur ? *total - cur : 0`; before that clamp an over-committed device (`currentAllocatedSize > recommendedMaxWorkingSetSize`) *underflowed* to a huge `size_t`, which normalised fine, so the same precondition was harmless. That is why the `Java Tests macOS …` jobs went red at the b10792→b10797 step while every Linux/Windows job stayed green — **and why only a GPU build can fail this way at all**: `act_gpu_layers` is `devices.empty() ? 0 : …`, so with no GPU backend `devices` is empty, every layer returns early on `cpu_dev`, and the `.at()` line is unreachable. **Shape:** the two blocks are lifted out of `load_tensors` into free functions declared in `src/llama-model.h`, purely so they can be driven by a test — the failing state needs a real over-committed GPU and cannot be arranged through any public API. `llama_model_splits_normalize()` carries **the fix**: on `split_sum == 0` it `LLAMA_LOG_WARN`s and falls back to an even split (`splits[i] = float(i+1)/splits.size()`), the only neutral choice when no device can be preferred and exactly right for a single device. `llama_model_splits_select_device()` carries **the diagnostic**: it bounds-checks the index and throws a `std::runtime_error` naming the function, the offloaded layer, the device index, the split-point count **and the split points themselves** — with NaN splits that message prints `nan` and names the cause outright, which is precisely what was missing when this had to be diagnosed by reading source. **A second, backend-independent trigger reaches the same line**, found while writing this up and verified against the unfixed library: `--tensor-split` values are parsed with `std::stof` and never range-checked (`common/arg.cpp`), so `-ts 1,-1` cancels out, `split_sum` is 0 again, the split points become `[inf, -nan]`, and every layer maps one past the last device — on CUDA, Vulkan or ROCm just as much as on Metal, with no memory pressure involved. That is what makes this an ordinary upstream defect rather than a Metal edge case, and the warning names both causes rather than only the memory one. Also adds upstream `tests/test-model-split.cpp` (5 cases in upstream's `testing.h` style) + its `llama_build_and_test` registration. Touches `src/llama-model.{cpp,h}`, `tests/test-model-split.cpp` and `tests/CMakeLists.txt` — **none** of which any other patch touches, so it is independent of all of them. Upstream-submittable ("model: fall back to an even split when no device reports free memory"); **not yet filed upstream**. **Runnable guard: `src/test/cpp/test_model_split.cpp`** — a FetchContent subproject builds with `LLAMA_BUILD_TESTS=OFF`, so the upstream test above is applied-but-never-compiled here (same as `0001`'s test). That file drives the same two functions from `jllama_test`, which runs on **every** platform in `C++ Tests`, so a bump that drops this patch fails the build at link time everywhere instead of surfacing as one red macOS Java job. **Verification limit — read before assuming this can be dropped:** the *failing path* still cannot be reached without a GPU backend, so the guard pins the arithmetic (what actually broke), not the end-to-end load; the end-to-end proof is the macOS CI job. On a bump, re-check whether upstream added its own `split_sum == 0` guard (grep `split_sum` in `src/llama-model.cpp`) and **drop this patch rather than refreshing it** if they did — the fail-loud applier detects "does not apply", never "upstream already fixed this". |
+| `0013-s390x-repack-guard-vxe-only-helpers.patch` | **A new upstream file makes the s390x build fail to compile, with nothing in this project involved.** b10902 added `ggml/src/ggml-cpu/arch/s390/repack.cpp` (upstream #28667, s390x q4_0 repack; the file does not exist at b10883). Every *function body* in it is guarded `#if defined(__VXE__) || defined(__VXE2__)`, but three `static inline` helpers — `vxe_dot_acc`, `vxe_splat_granule`, `vxe_fold` — sit at file scope **between** two guarded blocks with no guard of their own, and their signatures name `int16x8_t` / `int8x16_t` / `int32x4_t`, which `ggml-cpu-impl.h` only typedefs *inside* that same guard. So a non-VXE s390x build dies with three `does not name a type` errors before it reaches any of the code it is supposed to skip. The patch wraps the three definitions in the identical guard — all 21 call sites are already inside `__VXE__` blocks, so nothing else moves. **Why this project builds s390x without VXE, which is the half that is ours:** `ggml/CMakeLists.txt` declares `option(GGML_VXE "ggml: enable vxe" ${GGML_NATIVE})` — the VXE default *follows* `GGML_NATIVE`. The `build-linux-s390x` job passes `-DGGML_NATIVE=OFF`, which is correct for a cross-build (it must not bake the x86 build host's `-march=native` into an s390x artifact) but also silently switches VXE off. No `-mvx -mzvector` is then passed, `__VEC__` is undefined, and `ggml-cpu-impl.h`'s `#if defined(__s390x__) && defined(__VEC__)` self-define of `__VXE__`/`__VXE2__` never fires. The job has therefore always produced a **scalar** s390x binary; that was invisible until a file arrived that does not compile that way. **Do not "fix" this with `-DGGML_VXE=ON`** — measured, not assumed: that makes it strictly worse, because the self-define above sets `__VXE__` *and* `__VXE2__` together as soon as `__VEC__` exists, while `-march` stays at the toolchain default `arch11`, so the three errors become dozens of `'__builtin_s390_vec_*' matching variant requires z14 or higher`. The only flag-side alternative is `-DGGML_VXE=ON` **plus** `-march=z15`, which does compile but raises the shipped artifact's hardware floor to z15 and makes the qemu `ctest` gate depend on VXE2 emulation — a real trade for vector kernels this job does not use (it is a big-endian *correctness* gate for our own layer, not a performance target). The patch keeps the configuration that worked through b10883 and changes nothing about the artifact. **Verified with the real cross toolchain** (`s390x-linux-gnu-g++`), not by inspection: unpatched + CI's flags reproduces the three CI errors exactly; patched + CI's flags compiles clean; patched + `-mvx -mzvector -march=z15` also compiles clean, so the patch does not foreclose a future vector build. Touches only that one file, which no other patch touches. Upstream-submittable ("ggml-cpu: guard the VXE-only helpers in the s390x repack path"); **not yet filed upstream**. **On a bump, check whether upstream guarded them itself and DROP this patch rather than refreshing it** — the fail-loud applier detects "does not apply", never "upstream already fixed this". There is no runnable guard for it beyond CI: the file is compiled only for s390x, so `build-linux-s390x` *is* the test, and it fails loudly at compile time. |
 
 **`0009` was dropped at the b10280 bump.** Upstream merged
 [sheredom/subprocess.h#104](https://github.com/sheredom/subprocess.h/pull/104) — the exact fix this