From bec1f14236e57b178aa1dfe8523617c47c02b506 Mon Sep 17 00:00:00 2001 From: Claude Date: Fri, 11 Sep 2026 06:39:12 +0000 Subject: [PATCH] feat: upgrade llama.cpp from b10883 to b10902 No project-source change, and not one file on the priority review list moved. 110 KiB / 19 commits / 30 files, which is over the runbook's 100 KiB chunking threshold -- so the decision to bump straight through is recorded rather than taken quietly. llama-next-version.sh proposed stopping at b10901 (87 KiB); the 23 KiB that pushes b10902 over is entirely ggml/src/ggml-opencl/** -- one CMakeLists line, ggml-opencl.cpp, and a new gemv_noshuffle_q4_0_f32_32b_trans.cl kernel (#28268). Bucketed by top-level directory the range is 998 lines of ggml/src backend internals against 93 lines across 13 files of everything this project compiles, links or includes. Chunking would have split a backend-kernel addition in half and reviewed nothing extra. Zero files under common/*.h, include/, tools/server/ or tools/mtmd/, so every row of the API-compatibility table is vacuously satisfied and the three mechanical server-contract greps have no input -- the request-field set, its set_hard_limits bounds and the emitted response keys cannot have moved. The one common/ file in the range is speculative.cpp, an implementation TU compiled by upstream, not a header this project includes; its change is internal to the DFlash draft path (#28587) and moves no signature. Every patch-target file is byte-unchanged except src/llama-model.{cpp,h}, verified by diffing those paths explicitly rather than inferred from the aggregate -- and that diff is two additive llm_type enumerators (LLM_TYPE_3B_A800M, LLM_TYPE_32B_A9B) nowhere near patches/0012's hunks. All three standing drop-checks were still run against the pristine tag, because the fail-loud applier detects "does not apply" but never "upstream already fixed this": * 0001 -- common_params_parse_main appears 0 times in b10902:common/arg.h and the #ifdef _WIN32 argv = utf8.ptrs.data() override is still at common/arg.cpp:1282. Still required. * 0010 -- b10902:tools/server/server-context.cpp:4554 still emits {"vocab_type", meta.model_vocab_type} uncast, so the common_json enum-to-bool trap is still live. Still required. * 0012 -- b10902:src/llama-model.cpp:1491 still normalises with a bare splits[i] /= split_sum; and carries no split_sum == 0 guard. Still required. Worth knowing for the s390x job: two of the 19 commits are upstream's own ggml-cpu s390x work (#28667 q4_0 repack, #28606 Q1_0 vector intrinsics, 291 new lines under ggml/src/ggml-cpu/arch/s390/). Those are upstream's big-endian kernels; the build-linux-s390x qemu ctest gate covers this project's own endian-sensitive layer, which the range does not touch. Verified: fresh rm -rf build && cmake -B build -DBUILD_TESTING=ON through the real FetchContent path, configure clean, stamp written at head df03399b8 (= b10902) with all nine hashes; Release build clean; ctest 537/537, including the seven LlamaModelSplits.* cases that are the only place 0012's extracted functions are linked and the six wire-contract cases whose configure-time OAI_LAYER reader sweep re-ran against b10902 (138 CLI / 57 request / 15 trainer names, unchanged); nm -D reports 40 Java_* exports and 0 C++-mangled ones; mvn -pl llama clean test -Dtest=NativeLibraryLoadSmokeTest 4/4, 0 skipped, including nativeBuildInfoMatchesPinnedVersionConstant; full mvn test 1755 tests, 0 failures; SpotBugs 0, spotless and javadoc:jar clean. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_01AnNYn8W1xuVxVJtyL34GyH --- CLAUDE.md | 8 ++++---- README.md | 2 +- docs/history/llama-cpp-breaking-changes.md | 2 ++ llama/CMakeLists.txt | 2 +- .../java/net/ladenthin/llama/value/LlamaCppVersion.java | 8 ++++---- 5 files changed, 12 insertions(+), 10 deletions(-) diff --git a/CLAUDE.md b/CLAUDE.md index 3cd8299a..244c6691 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -6,7 +6,7 @@ This file provides guidance to Claude Code (claude.ai/code) when working with co Java bindings for [llama.cpp](https://github.com/ggerganov/llama.cpp) via JNI, providing a high-level API for LLM inference in Java. The Java layer communicates with a native C++ library through JNI. -Current llama.cpp pinned version: **b10883** +Current llama.cpp pinned version: **b10902** ## Upgrading CUDA Version @@ -502,7 +502,7 @@ needs no extra step here, `build-webui` re-reads the tag and rebuilds the matchi ships no UI): ```bash # needs node/npm + network for the asset build; the embed step is plain cmake -P -git clone --depth 1 --branch b10883 https://github.com/ggml-org/llama.cpp /tmp/lc +git clone --depth 1 --branch b10902 https://github.com/ggml-org/llama.cpp /tmp/lc ( cd /tmp/lc/tools/ui && npm ci && npm run build ) mkdir -p webui-generated /tmp/ui-gen cmake -DUI_SOURCE_DIR=/tmp/lc/tools/ui -DUI_BINARY_DIR=/tmp/ui-gen \ @@ -542,7 +542,7 @@ cache lives in **Depot Cache** over sccache's **WebDAV** backend: - `SCCACHE_WEBDAV_TOKEN: ${{ secrets.DEPOT_TOKEN }}` — a Depot **organization** token, stored as the repo secret **`DEPOT_TOKEN`**. -Because `sccache` is **content-addressed** and llama.cpp is pinned (`GIT_TAG b10883`), the +Because `sccache` is **content-addressed** and llama.cpp is pinned (`GIT_TAG b10902`), the ~280 upstream object files are byte-identical every run, so a warm cache recompiles only the *changed* files. Depot's cache is **shared across all branches** (unlike GitHub's per-branch `actions/cache`), so every branch builds incrementally; a `b` version bump @@ -1535,7 +1535,7 @@ ctest --test-dir build --output-on-failure -R "ResultsToJson" #### Upstream source location (in CMake build tree) -llama.cpp is fetched via CMake FetchContent, pinned to `GIT_TAG b10883`. +llama.cpp is fetched via CMake FetchContent, pinned to `GIT_TAG b10902`. **GoogleTest** is a separate `BUILD_TESTING`-only FetchContent (`GIT_TAG v1.17.0`), used solely by the `jllama_test` C++ unit-test binary — not by the shipped library, and not coupled to the diff --git a/README.md b/README.md index 0667f269..82f439bd 100644 --- a/README.md +++ b/README.md @@ -11,7 +11,7 @@ **Build:** ![Java 8+](https://img.shields.io/badge/Java-8%2B-informational) ![Platform](https://img.shields.io/badge/Platform-Linux%20%7C%20macOS%20%7C%20Windows%20%7C%20Android-lightgrey) -[![llama.cpp b10883](https://img.shields.io/badge/llama.cpp-%23b10883-informational)](https://github.com/ggml-org/llama.cpp/releases/tag/b10883) +[![llama.cpp b10902](https://img.shields.io/badge/llama.cpp-%23b10902-informational)](https://github.com/ggml-org/llama.cpp/releases/tag/b10902) [![JPMS](https://img.shields.io/badge/JPMS-modular%20JAR-25A162)](https://openjdk.org/projects/jigsaw/) ![JUnit](https://img.shields.io/badge/tested%20with-JUnit6-25A162) [![JSpecify](https://img.shields.io/badge/JSpecify-1.0.0%20%40NullMarked-25A162)](https://jspecify.dev) diff --git a/docs/history/llama-cpp-breaking-changes.md b/docs/history/llama-cpp-breaking-changes.md index 6b10c432..d7752571 100644 --- a/docs/history/llama-cpp-breaking-changes.md +++ b/docs/history/llama-cpp-breaking-changes.md @@ -704,3 +704,5 @@ Used during `llama.cpp` version bumps: when upgrading, scan this file from the r | b10870–b10878 | patches + upstream verification | **All nine patches still apply, and `0012` is still required.** The range touches two patch targets — `common/arg.cpp` (`0001`) and `src/llama-model.{cpp,h}` (`0012`) — so both were checked against the pristine tag rather than assumed. `0001`: `b10878:common/arg.cpp` still carries the `#ifdef _WIN32` count-guarded `argv = utf8.ptrs.data()` override, and `common_params_parse_main` appears **0 times** in `b10878:common/arg.h`, so upstream has still not adopted the fix. `0012`: `b10878:src/llama-model.cpp` still normalises with a bare `splits[i] /= split_sum;` and has **no `split_sum == 0` guard** of its own — the CLAUDE.md instruction to *drop rather than refresh* this patch does not fire, and its `llama-model` diff is only the new enumerator. Verified for real: fresh `cmake -S llama -B /tmp/b10878-build -DBUILD_TESTING=ON` through the real `FetchContent` path, configure clean, stamp written at head `4850c7727fa73bbe3098e10ee369fbc3467c445f` (= `b10878`) with **all nine hashes recorded**; full `cmake --build --config Release` clean; `ctest` **527/527**, including the four `LlamaModelSplits.*` cases that are the only place `0012`'s two extracted functions are linked in CI. `nm -D` on the fresh `libjllama.so` reports **40** `Java_*` exports. `mvn -pl llama clean test -Dtest=NativeLibraryLoadSmokeTest` **4/4, 0 skipped**, including `nativeBuildInfoMatchesPinnedVersionConstant` — the end-to-end proof that the four pin sites and the linked binary agree. Run with `clean`: `LLAMA_CPP_VERSION` is a compile-time constant javac inlines into the test class, and Maven's incremental compilation cannot see that dependency. | | b10878–b10883 | **Nothing on the review surface.** The raw diff is large — 29 files, 2360 insertions, 2276 deletions, **426 KiB, well over the runbook's 100 KiB chunking threshold** — and was bumped straight through anyway, for a reason that is recorded here rather than asserted: the byte count is entirely GPU backend internals and Python pins. Restricting `git diff --stat` to the paths this project actually compiles, links or includes (`common/`, `include/`, `tools/server/`, `tools/mtmd/`, `ggml/include/`, `src/`, the top-level `CMakeLists.txt`) leaves **one file, one line**: `tools/server/tests/requirements.txt`, a Python test-requirement pin that is neither compiled nor linked. The remainder is `ggml/src/ggml-vulkan/**` (1804 lines in `ggml-vulkan.cpp` plus ~15 shader files), `ggml/src/ggml-hexagon/**`, `tests/test-backend-ops.cpp`, and six `requirements*.txt` / `pyproject.toml` version pins. **Zero** priority-8 headers moved; `common/arg.h`, `common/chat.h`, `include/llama.h`, `tools/mtmd/mtmd-helper.h` are all byte-identical. This is the same shape as the b10819–b10850 row: a headline number dominated by backends the project builds but whose internals it never calls. The one thing the size *does* imply is CI cost — the Vulkan rewrite is upstream-compiled code the `vulkan-linux-*` and `vulkan-windows-*` classifier jobs must still build, so a compile break there would surface in those jobs rather than in any project source. | | b10878–b10883 | patches + upstream verification | **All nine patches apply, and not one needed refreshing — every patch-target file is byte-unchanged in the range.** Checked file by file rather than inferred from the aggregate: `common/arg.cpp`, `common/arg.h`, `common/peg-parser.cpp`, `tools/server/server.cpp`, `tools/server/server-context.{cpp,h}`, `tools/server/server-models.cpp`, `src/llama-model.{cpp,h}` and `tests/CMakeLists.txt` all report no diff between the two tags. The two standing drop-checks were still run against the pristine tag, because the fail-loud applier detects "does not apply" but never "upstream already fixed this": `0001` — `common_params_parse_main` appears **0 times** in `b10883:common/arg.h` and the `#ifdef _WIN32` `argv = utf8.ptrs.data()` override is still at `common/arg.cpp:1282`, so it stays; `0012` — `b10883:src/llama-model.cpp:1489` still normalises with a bare `splits[i] /= split_sum;` and has **no `split_sum == 0` guard**, so the CLAUDE.md instruction to *drop rather than refresh* does not fire. Verified for real: `rm -rf build && cmake -B build -DBUILD_TESTING=ON` through the real `FetchContent` path, configure clean, stamp written at head `91f6a6cf361385700bbe15981f0f39909df77498` (= `b10883`) with **all nine hashes recorded**; full `cmake --build --config Release` clean; `ctest` **531/531**. That total is 4 up from b10878 because this is the first bump after `test_model_flags.cpp` landed — and it is the first bump whose **flag contract** was machine-checked rather than reasoned about: `JavaCliFlagContract` re-derives the 138 flags the Java layer emits and re-runs them through `common_params_parser_init(params, LLAMA_EXAMPLE_SERVER)` at the new tag, so "`arg.cpp` is unchanged, therefore the registered option set is unchanged" is now an assertion the build makes, not an inference a reviewer makes. `nm -D` on the fresh `libjllama.so` reports **40** `Java_*` exports. `mvn -pl llama clean test -Dtest=NativeLibraryLoadSmokeTest` **4/4, 0 skipped**, including `nativeBuildInfoMatchesPinnedVersionConstant` — the end-to-end proof that the four pin sites and the linked binary agree (`clean` is required: javac inlines `LLAMA_CPP_VERSION` into the test class and Maven's incremental compile cannot see that dependency). Full `mvn test`: **1759 tests, 0 failures, 0 errors**. | +| b10883–b10902 | `src/llama-model.{cpp,h}` (**additive only**: two new `llm_type` enumerators, `LLM_TYPE_3B_A800M` for Granite3 MoE and `LLM_TYPE_32B_A9B` for Granite4 Hybrid, plus their `llm_type_name` cases — four lines total, far from `patches/0012`'s hunks), `common/speculative.cpp` (**#28587**: the DFlash draft implementation now skips an M-RoPE image whose rows are all pinned to one position, because a windowed draft cache cannot free cells for it — internal to `common_speculative_impl_draft_dflash`, **no signature moved**, so `common_speculative_init` / `_draft` / `_accept` are untouched), `src/llama-memory-hybrid-idx.cpp` (**#28330**: sets `n_embd_head_{k,v}_mla_impl` so `llama_kv_cache` skips allocating the unused V cache for the indexer — three lines, internal), `src/models/*` (**#28643/#28669**: Granite-family parameter-count fix and dead-switch-branch cleanup across bert / jina-bert-v3 / nomic-bert{,-moe} / granite{,-hybrid,-moe}), `tests/test-backend-ops.cpp`, `ggml/src/ggml-{opencl,vulkan,cpu}/**`, `.github/**`, `scripts/make-release-checks.sh`, `conversion/nemotron.py`. 30 files, 1106 insertions, 423 deletions, **110 KiB** raw. | **No project-source change, and not one file on the priority review list moved.** Zero files under `common/*.h`, `include/`, `tools/server/` or `tools/mtmd/`, so every row of the API-compatibility table is vacuously satisfied and the three mechanical server-contract greps have **no input to compare** — the request-field set, its `set_hard_limits` bounds and the emitted response keys cannot have moved. The one `common/` file in the range is `speculative.cpp`, an implementation TU compiled by upstream, not a header this project includes. **Sizing, and why it was bumped straight through at 110 KiB.** The runbook's threshold is 100 KiB, so `llama-next-version.sh` proposed an intermediate stop at b10901 (87 KiB). The 23 KiB that pushes b10902 over is **entirely** `ggml/src/ggml-opencl/**` — one `CMakeLists.txt` line, `ggml-opencl.cpp`, and a new `gemv_noshuffle_q4_0_f32_32b_trans.cl` kernel (#28268). Bucketed by top-level directory the whole range is 998 lines of `ggml/src` backend internals against **93 lines across 13 files** of everything this project actually compiles, links or includes — so chunking would have split a backend-kernel addition in half and reviewed nothing extra. Same call, and same reasoning, as the b10870–b10878 row. **Worth knowing for the s390x job:** two of the 19 commits are upstream's own `ggml-cpu` s390x work (#28667 q4_0 repack, #28606 Q1_0 vector intrinsics, 291 new lines under `ggml/src/ggml-cpu/arch/s390/`). Those are upstream's big-endian kernels; the `build-linux-s390x` job's qemu `ctest` gate covers *this project's* endian-sensitive layer (the little-endian WAV writer, the JSON/token/embedding transforms), which the range does not touch. | +| b10883–b10902 | patches + upstream verification | **All nine patches still apply, and `0001`, `0010` and `0012` are all still required.** Only one patch target moved in the range — `src/llama-model.{cpp,h}` (`0012`) — and its diff is the two additive enumerators above, nowhere near `load_tensors`. The other eight targets (`common/arg.{cpp,h}`, `common/peg-parser.cpp`, every `tools/server/*`, `tests/CMakeLists.txt`, `vendor/*`) are **byte-unchanged**, verified by diffing those paths explicitly rather than inferred from the aggregate. All three standing drop-checks were run against the pristine tag, because the fail-loud applier detects "does not apply" but never "upstream already fixed this": **`0001`** — `common_params_parse_main` appears **0 times** in `b10902:common/arg.h` and the `#ifdef _WIN32` count-guarded `argv = utf8.ptrs.data()` override is still at `common/arg.cpp:1282`; **`0010`** — `b10902:tools/server/server-context.cpp:4554` still emits `{"vocab_type", meta.model_vocab_type}` uncast, so the `common_json` enum-to-bool trap is still live; **`0012`** — `b10902:src/llama-model.cpp:1491` still normalises with a bare `splits[i] /= split_sum;` and carries no `split_sum == 0` guard. Verified for real: `rm -rf build` then `cmake -B build -DBUILD_TESTING=ON` through the real `FetchContent` path, configure clean, stamp written at head `df03399b885831b2a1603b3abb0d8c156808e363` (= `b10902`) with **all nine SHA-256 lines**; full `cmake --build --config Release` clean; `ctest` **537/537**, including the seven `LlamaModelSplits.*` cases that are the only place `0012`'s two extracted functions are linked in CI, and the six `test_wire_contracts.cpp` cases whose configure-time `OAI_LAYER` reader sweep re-ran against b10902's sources (138 CLI / 57 request / 15 trainer names extracted, unchanged). `nm -D` on the fresh `libjllama.so`: **40** `Java_*` exports, **0** C++-mangled ones. `mvn -pl llama clean test -Dtest=NativeLibraryLoadSmokeTest` **4/4, 0 skipped**, including `nativeBuildInfoMatchesPinnedVersionConstant` — the end-to-end proof that the four pin sites and the linked binary agree. Run with `clean`: `LLAMA_CPP_VERSION` is a compile-time constant javac inlines into the test class, and Maven's incremental compilation cannot see that dependency. | diff --git a/llama/CMakeLists.txt b/llama/CMakeLists.txt index 9b370e2f..a7799ef4 100644 --- a/llama/CMakeLists.txt +++ b/llama/CMakeLists.txt @@ -173,7 +173,7 @@ set(LLAMA_BUILD_APP OFF CACHE BOOL "" FORCE) FetchContent_Declare( llama.cpp GIT_REPOSITORY https://github.com/ggerganov/llama.cpp.git - GIT_TAG b10883 + GIT_TAG b10902 PATCH_COMMAND ${CMAKE_COMMAND} -DPATCH_DIR=${CMAKE_CURRENT_SOURCE_DIR}/patches -DLLAMA_SRC= diff --git a/llama/src/main/java/net/ladenthin/llama/value/LlamaCppVersion.java b/llama/src/main/java/net/ladenthin/llama/value/LlamaCppVersion.java index 7dea5d85..e926a825 100644 --- a/llama/src/main/java/net/ladenthin/llama/value/LlamaCppVersion.java +++ b/llama/src/main/java/net/ladenthin/llama/value/LlamaCppVersion.java @@ -10,13 +10,13 @@ * library was compiled against, exposed as a compile-time constant so callers can render a badge or * emit a startup log line without loading the native library. * - *

{@link #LLAMA_CPP_VERSION} is a pure-Java string ({@code "b10883"}) that mirrors the + *

{@link #LLAMA_CPP_VERSION} is a pure-Java string ({@code "b10902"}) that mirrors the * {@code GIT_TAG} in {@code llama/CMakeLists.txt}. It is available even when {@code libjllama} is * absent (pure-Java checkout, before {@code System.load}), which is what makes it suitable for a * lightweight version badge in Android or other UIs.

* *

For the authoritative value that is baked into the native binary — the build number - * plus the resolved upstream commit, e.g. {@code "b10883-"} — call + * plus the resolved upstream commit, e.g. {@code "b10902-"} — call * {@link net.ladenthin.llama.LlamaModel#getLlamaCppBuildInfo()} instead; that reads llama.cpp's own * {@code build-info} through JNI and therefore cannot drift from the compiled library (but requires * the native library to be loaded).

@@ -24,14 +24,14 @@ public final class LlamaCppVersion { /** - * The pinned llama.cpp release tag this library was built against, e.g. {@code "b10883"}. + * The pinned llama.cpp release tag this library was built against, e.g. {@code "b10902"}. * *

Kept in lockstep with {@code GIT_TAG} in {@code llama/CMakeLists.txt} — see the * "Upgrading/Downgrading llama.cpp Version" checklist in {@code CLAUDE.md}. This is the * compile-time pin; use {@link net.ladenthin.llama.LlamaModel#getLlamaCppBuildInfo()} for the * value actually linked into the native binary.

*/ - public static final String LLAMA_CPP_VERSION = "b10883"; + public static final String LLAMA_CPP_VERSION = "b10902"; // Constants holder — not instantiable. private LlamaCppVersion() {}