Conversation
Replace the LcmGenerate MSBuild task (which ran the NVelocity templates in a child MSBuild process to avoid VS DLL-locking, writing 9 Generated*.cs files into the source tree) with a Roslyn incremental source generator that produces the domain model at compile time. The new SIL.LCModel.SourceGenerators project reuses the existing NVelocity templates and model-wrapper classes unchanged; only the host is rewritten: - LcmModelGenerator (IIncrementalGenerator) reads MasterLCModel.xml, HandGenerated.xml and IntPropTypeOverrides.xml as AdditionalFiles and emits via AddSource. - LcmGenerateImpl accumulates outputs in memory instead of writing files. - EmbeddedTemplateLoader serves the 49 templates from embedded resources, replacing the filesystem #parse + Directory.SetCurrentDirectory mechanism. NVelocity is a single self-contained DLL that runs inside the Roslyn analyzer host; it is flowed alongside the generator via GetDependencyTargetPaths. SIL.LCModel.Build.Tasks keeps only the IdlImp task (kernel-interface codegen, left as-is). Generated output is byte-for-byte identical to the previous task; SIL.LCModel.Tests pass (1709 passed, 0 failed, 18 skipped). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Property looked up the current generator via NVelocity's process-global RuntimeSingleton to read the override lists, which forced generation to be serialized. Store the LcmGenerateImpl on the root Model instead and have Property reach it through the existing parent chain (Property -> Class -> CellarModule -> Model). With that global gone (and the never-read "LcmGenerate.Engine" attribute and the write-only static Generator field removed), a generation run is entirely instance-local, so LcmModelRunner no longer needs a lock; the immutable template set is cached once via Lazy. Verified with a concurrency stress test (32 simultaneous generations, repeated) all producing byte-identical output, and the output remains byte-for-byte identical to the previous commit. SIL.LCModel.Tests pass (1709 passed, 0 failed, 18 skipped). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
|
There is one consequence of this that we have to solve first. FieldWorks was using Roughly what it takes:
|
…nerate task FieldWorks uses the LcmGenerate MSBuild task to render a C++ header (CellarConstants.vm.h) from the LCM model, so removing it would break the FieldWorks native build on the next SilLcmVersion bump (reported by Jason on the PR). Restore the task, but without going back to the old design. Move the NVelocity engine (LcmGenerateImpl, EmbeddedTemplateLoader, and the model wrappers) out of the source generator into a new netstandard2.0 project, SIL.LCModel.ModelGeneration, whose only dependency is NVelocity. Both the source generator and Build.Tasks reference it by project reference: - SIL.LCModel.SourceGenerators keeps the Roslyn driver, the runner and the embedded templates; it flows the engine + NVelocity into the analyzer load context as analyzer assets. - SIL.LCModel.Build.Tasks gains a thin LcmGenerate task over the same engine (original property surface, so FieldWorks needs no changes) and repacks the engine + NVelocity into tools/, with the UsingTask restored. The engine stays internal, shared via InternalsVisibleTo, rather than source-linking the files into two assemblies. Domain-model output is byte-for- byte identical to the original task (verified against a fresh master build); the LcmGenerate task renders a template end-to-end through MSBuild; the full solution builds and SIL.LCModel.Tests pass (1709 passed, 0 failed, 18 skipped). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
SIL.LCModel.ModelGeneration is IsPackable=false (its assembly is bundled directly into the analyzer and into SIL.LCModel.Build.Tasks tools/), but the plain ProjectReference from Build.Tasks emitted a <dependency> on it in the Build.Tasks nuspec. Consumers pull it transitively (SIL.LCModel -> Core -> Build.Tasks -> ModelGeneration), so restore fails with NU1101: package SIL.LCModel.ModelGeneration not found. A full FieldWorks build against a local package feed surfaced this. Mark the ModelGeneration project references PrivateAssets="all" in both Build.Tasks and SourceGenerators so the engine is not advertised as a package dependency; the DLL continues to flow via the explicit tools/ pack and the analyzer's GetDependencyTargetPaths. SIL.LCModel still builds with the generator (ModelGeneration.dll + NVelocity.dll reach the analyzer load context). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
|
@jasonleenaylor after looking at the issue I decided to fix it by making a new project, both the build tasks and the source generator reference it. Claude tested the new package building against FLEx and it worked fine without any changes, so you shouldn't be affected once this is merged in. |
jasonleenaylor
left a comment
There was a problem hiding this comment.
Verified with a FieldWorks build LGTM
Does what it says, this is the blessed way to generate source code at build time, and avoids any race conditions or parallel build issues.
Mostly vibe coded, though I have read all the code and more or less understand the new code.
AI summary
Ports LCM's domain-model code generation from the
LcmGenerateMSBuild task to a modern Roslyn incremental source generator, reusing the existing NVelocity templates and model wrappers unchanged.Before: an MSBuild
Taskshelled out to a child MSBuild process (GenerateModel.proj) to run NVelocity — a workaround for the task DLL being locked by Visual Studio — writing 9Generated*.csfiles into the source tree (gitignored, regenerated every build). This depended on locating an msbuild/dotnet executable, usedDirectory.SetCurrentDirectoryand process-global state, and cluttered the tree.After: a new
SIL.LCModel.SourceGeneratorsproject (IIncrementalGenerator) produces the domain model at compile time, directly into the compilation. No child process, no DLL-locking workaround, no on-disk generated files.Approach — reuse, don't rewrite:
LcmModelGeneratorreadsMasterLCModel.xml,HandGenerated.xmlandIntPropTypeOverrides.xmlasAdditionalFilesand emits viaAddSource;LcmGenerateImplaccumulates outputs in memory instead of writing files;EmbeddedTemplateLoader(a custom NVelocityResourceLoader) serves the templates from embedded resources, replacing the filesystem#parse+ current-directory mechanism.GetDependencyTargetPaths.Property → Class → CellarModule → Model, which holds the generator) rather than any process-global state, so generation is fully instance-local and lock-free.Scope: only the domain-model generator (Generator A) is ported. The kernel IDL importer (
IdlImp, which shells out tocl.exe/gccto preprocess native COM IDL) is a poor fit for source generators and stays as-is inSIL.LCModel.Build.Tasks.Compatibility: the generated code is compiled into
SIL.LCModel.dllas before, so downstream NuGet consumers are unaffected and no public API changes. Downstream packaging/content files are left untouched.Notable findings while porting:
ExtendedPropertiestreats commas as list separators, so the loader class name must be passed asType; Assembly(semicolon), matching NVelocity's own escaped-comma convention.EmitCompilerGeneratedFileson-disk emission differs from the old output only by a 3-byte UTF-8 BOM; the compiler-consumed content is byte-for-byte identical.Test plan
SIL.LCModelbuilds with the generator active (NVelocity loads and executes in-process).SIL.LCModel.Tests: 1709 passed, 0 failed, 18 skipped.This change is