Fetch Chicory/ASM jars from Maven instead of bundling in gem#3019
Fetch Chicory/ASM jars from Maven instead of bundling in gem#3019soutaro wants to merge 4 commits into
Conversation
The WebAssembly-backed JRuby parser added in #2998 bundled all of the Chicory and ASM jars (~1MB) inside the -java gem under lib/rbs/wasm/jars. Shipping third-party jars bloats the gem and risks loading conflicting copies when another gem ships the same jars. Follow the established JRuby convention (e.g. Psych): declare the jars as jar-dependencies requirements so they are fetched from Maven Central, the canonical source, at gem install time. Only RBS's own build artifact (rbs_parser.wasm) is shipped. - lib/rbs/wasm/jars.rb: single source of truth for the Maven coordinates, shared by the gemspec, the runtime, and the vendoring rake task. - rbs.gemspec: depend on jar-dependencies and add a `jar` requirement per coordinate; stop adding the jars to spec.files. - runtime.rb: require_jar the coordinates for an installed gem; keep loading vendored jars by path when running from source. - Rakefile: derive the vendor download URLs from the shared coordinates; the vendored dir is gitignored and only used to run the suite from source. Refs #3018 Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01RtMsDQEmuayTdn3rYsteYT
Follow Psych's model more closely and drop the development-only branch: the runtime now always loads the Chicory/ASM jars from the local Maven repository (~/.m2) with require_jar, both for the installed gem and when running from source. - runtime.rb: load_jars just require_jars each coordinate; the local vendored-directory path and the shared coordinates helper are gone. - Rakefile: `wasm:install_jars` runs Jars::Installer against rbs.gemspec to download the jars into ~/.m2 (the same path `gem install` uses), without copying them into the gem. Replaces the curl-based vendoring. - jruby.yml: download the jars on JRuby (jar-dependencies resolves through the JVM), after building the wasm on CRuby. - Remove lib/rbs/wasm/jars.rb; inline the coordinates in the gemspec and the runtime instead of abstracting them. Refs #3018 Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01RtMsDQEmuayTdn3rYsteYT
jar-dependencies can generate the require_jar calls from the gem's `jar` requirements, so the runtime no longer repeats the coordinate list that the gemspec already declares. - `rake wasm:install_jars` now runs Jars::Installer with its default require-file generation: it downloads the jars into ~/.m2 and writes lib/rbs_jars.rb (require_jar calls for the whole resolved set). - runtime.rb: load_jars is just `require "rbs_jars"`. - rbs.gemspec: ship the generated lib/rbs_jars.rb alongside rbs_parser.wasm; the `jar` requirements stay the single source. - lib/rbs_jars.rb is a build artifact (gitignored, Steep-ignored, rebuilt in the Docker image and CI). Refs #3018 Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01RtMsDQEmuayTdn3rYsteYT
Runtime loads the Chicory/ASM jars with a single `require "rbs_jars"`. lib/rbs_jars.rb is committed and hand-maintained rather than generated: jar-dependencies' require-file generator mangles the `com.dylibso.chicory:runtime` artifact id into `jar` (the artifact name collides with a Maven scope keyword), which would make the installed gem fail to load Chicory's runtime jar. Omitting the "this is a generated file" marker also stops jar-dependencies from overwriting the file at gem install. Verified on JRuby 10.0.6: `rake wasm:install_jars` resolves the `jar` requirements to exactly these ten jars and downloads them into ~/.m2, `require "rbs_jars"` loads them, and the Chicory classes the runtime uses link successfully. - runtime.rb: inline `require "rbs_jars"` (drop the load_jars method). - Rakefile: install_jars forces the java platform (the installer skips non-java gems) and downloads only (the require file is hand-maintained). - ship lib/rbs_jars.rb via git ls-files; drop its ignore entries. Refs #3018 Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01RtMsDQEmuayTdn3rYsteYT
headius
left a comment
headius
left a comment
There was a problem hiding this comment.
This is basically correct but a mess of conflicting comments from the LLM.
- Only the direct dependency jars need to be specified. I believe that's just
runtimeandcompiler. - Jar specifications are duplicated several places even though the comments claim they should live in the jars source file.
- rbs_jars.rb says it is hand-maintained but other places say it is generated. LLM is getting confused and contradicting itself.
This matches the work I did locally as an experiment, but quite a bit messier with nonsense comments.
| require_jar "com.dylibso.chicory", "log", "1.7.5" | ||
| require_jar "com.dylibso.chicory", "wasi", "1.7.5" | ||
| require_jar "com.dylibso.chicory", "compiler", "1.7.5" | ||
| require_jar "org.ow2.asm", "asm", "9.9.1" |
There was a problem hiding this comment.
Only top-level dependencies are required here; jar-dependencies will load any transitive dependencies necessary. I believe only runtime and compiler are needed from experimenting locally.
| spec.requirements << "jar com.dylibso.chicory:log, 1.7.5" | ||
| spec.requirements << "jar com.dylibso.chicory:wasi, 1.7.5" | ||
| spec.requirements << "jar com.dylibso.chicory:compiler, 1.7.5" | ||
| spec.requirements << "jar org.ow2.asm:asm, 9.9.1" |
There was a problem hiding this comment.
Only need runtime and compiler here as well I believe.
These dependencies are also being duplicated multiple places despite Claude saying that the jars file is the master list.
| "lib/rbs/wasm/location.rb", | ||
| "lib/rbs/wasm/runtime.rb", | ||
| "lib/rbs/wasm/parser.rb", | ||
| # Generated by `rake wasm:install_jars` (jar-dependencies require_jar calls). |
There was a problem hiding this comment.
This documentation says it is generated, but the comments in that file say "Hand-maintained, NOT auto-generated". Claude is getting confused here.
|
LLM is making up nonsense, or getting confused...
I think in all scenarios jars are downloaded to local .m2 for use, so I'm not sure what this means.
This file does not exist. The actual file, rbs_jars.rb, does seem to be the single source of truth... except its contents get duplicated several other places.
Probably no need for this. The WASM interpreter will probably be unusably slow without the compiler, and there's no reason it won't load now. -- The actual changes to support jar-dependencies look fine but the documentation has a lot of nonsense in it. |
|
Has this bundled gem degenerated from methodical development to prompt engineering and botsitting? If the bot cannot do it right, throw its slop out and do it ourselves, and consult human experts rather than a black box of probabilities. Relying on bot output is not “hand-maintained”; it’s vibed. |
4 participants
Summary
This change decouples jar dependency management from the gem distribution. Instead of bundling Chicory and ASM jars in the
-javagem, they are now declared asjar-dependenciesrequirements and fetched from Maven Central at install time. This keeps the gem smaller and ensures all users resolve the same versions from a canonical source.Key Changes
New
lib/rbs/wasm/jars.rb: Centralized module defining Maven coordinates for all required and optional jars (Chicory 1.7.5 and ASM 9.9.1). Provides:CHICORY_VERSION,ASM_VERSIONconstantsREQUIRED_JARSandOPTIONAL_JARSlists with[group, artifact, version]tuplesjar_requirements()method to generatejar-dependenciesrequirement stringsmaven_url()helper to construct Maven Central download URLsUpdated
lib/rbs/wasm/runtime.rb:jars.rb)jars_dirtolocal_jars_dirand made it returnnilfor installed gems (only returns a path when jars are vendored locally for development)load_jars()to support two modes:lib/rbs/wasm/jars/directory (set byrake wasm:vendor_jars)require_jarfromjar-dependenciesto load jars from MavenUpdated
rbs.gemspec:jar-dependenciesas a dependency for JRuby/Java platformspec.requirementsfromRBS::WASM.jar_requirements()Updated
Rakefile:rake wasm:vendor_jarsto use centralized coordinates fromjars.rbDocumentation updates: Updated
docs/release.mdand.gitignorecomments to reflect that jars are fetched from Maven, not bundledImplementation Details
The
jars.rbmodule serves as the single source of truth for jar coordinates, consumed by:jar-dependenciesdeclarations)Graceful degradation: optional AOT compiler jars are wrapped in error handling; missing jars just leave Chicory on the interpreter instead of failing the load
The local
lib/rbs/wasm/jars/directory remains gitignored and is only used when running tests from sourcehttps://claude.ai/code/session_01RtMsDQEmuayTdn3rYsteYT