Add AMD ROCm support (Linux + Windows) - #267
Open
lionrayonnant wants to merge 1 commit into
Open
Conversation
Adds AMD GPU detection and a PyTorch/ROCm redirect for extension setup, on top of the existing NVIDIA/MPS/CPU paths. - electron/main/gpu-detect.ts: detects AMD GPUs (KFD topology on Linux, Win32_VideoController on Windows), resolves the ROCm pip index and requirements. NVIDIA keeps detection priority; explicit overrides (MODLY_TORCH_FLAVOR, MODLY_ROCM_GFX, MODLY_ROCM_INDEX, MODLY_ROCM_TORCH_SPEC) are available for machines the auto-detection gets wrong. - electron/main/setup-launcher.ts: extracted from ipc-handlers.ts, adds a compatibility shim that redirects an extension's pip torch install to ROCm wheels — needed because most third-party extension setup.py scripts predate AMD support and hardcode a CUDA index. - api/routers/extensions.py: the FastAPI-side GPU detection no longer mistakes a ROCm build's device capability for CUDA compute capability (both answer torch.cuda.get_device_capability the same way), and reads the AMD compute target from the same KFD topology as the Electron side. - electron/main/copy-runtime.ts: fixes an unrelated but blocking AppImage bug found while verifying this end-to-end — fs.cp rewrote the bundled Python runtime's relative symlinks into absolute paths pointing at the ephemeral AppImage mount, so every extension venv died on the next launch. verbatimSymlinks keeps them relative. - docs/running-on-amd-rocm.md, arch/decisions/AMD-ROCM-SUPPORT.md: usage, verified configuration, and known limitations. Verified end-to-end on a Radeon RX 9060 XT (gfx1200): detection, ROCm wheel install (torch 2.13.0+rocm7.2), and a full image-to-3D generation through hunyuan3d-mini all complete successfully on the GPU.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds AMD GPU support alongside the existing NVIDIA/MPS/CPU paths. Modly never
installs PyTorch itself — each extension's
setup.pydoes, from an index ithardcodes — so this is really two pieces: detecting AMD hardware, and
redirecting extension installs to the right ROCm wheels.
electron/main/gpu-detect.ts(new) — detects AMD GPUs via KFD topologyon Linux and
Win32_VideoControlleron Windows, resolves the ROCm pip indexand requirements per platform. NVIDIA keeps detection priority, so nothing
changes for existing CUDA/MPS users. Escape hatches (
MODLY_TORCH_FLAVOR,MODLY_ROCM_GFX,MODLY_ROCM_INDEX,MODLY_ROCM_TORCH_SPEC) are availablefor machines the auto-detection gets wrong, or to roll back to an older
ROCm/torch pairing.
electron/main/setup-launcher.ts(extracted fromipc-handlers.ts) —adds a compatibility shim that rewrites an extension's pip torch install to
ROCm wheels. This is needed because most extension
setup.pyscriptspredate AMD support and hardcode a CUDA index; the official
hunyuan3d-miniextension already accepts atorch_flavorargument, butModly never sent it, and other extensions don't know the argument exists
at all.
api/routers/extensions.py— the FastAPI-side GPU detection no longermistakes a ROCm build's device capability for a CUDA one (both answer
torch.cuda.get_device_capability()the same way under PyTorch's HIPbuild), and reads the AMD compute target from KFD topology directly (no
torchdependency in this process).electron/main/copy-runtime.ts(new) — fixes an unrelated but blockingbug found while verifying this end-to-end on Linux:
fs.cprewrites thebundled Python runtime's relative symlinks into absolute paths pointing at
the source tree. On AppImage, the source is the ephemeral
/tmp/.mount_Modly-XXXXXX/mount, so the "stable" runtime copy silentlywasn't stable — every extension venv recorded that path and died on the
next launch.
verbatimSymlinks: truekeeps the links relative.docs/running-on-amd-rocm.md,arch/decisions/AMD-ROCM-SUPPORT.md—usage, the verified configuration, escape hatches, and known limitations.
Verified on real hardware
Radeon RX 9060 XT (Navi 44, gfx1200), Linux:
accelerator=rocm gfx=gfx1200correctly.hunyuan3d-miniextension setup installstorch 2.13.0+rocm7.2and loadson the GPU (
torch.cuda.is_available()isTrue,gcnArchNameisgfx1200).ExtensionProcesssubprocess path completes successfully (18s load, 221s generation, valid
GLB output).
ROCm/ROCm#6295, which reports
this exact card capped near 8GB, did not reproduce on this stack.
Windows is implemented (wheel URLs and
cp311availability were checkedagainst the AMD and PyTorch package indexes) but not run end-to-end — no
Windows/AMD machine was available for this PR.
Extensions tried beyond hunyuan3d-mini
Two other official extensions were tried while verifying this, both without success:
modly-trellis2-extension— not attempted. Itsbuild_vendor.pyshipsprecompiled CUDA binaries (
nvdiffrast,cumesh,spconv) invendor/,built against
nvcc. Independent of anything in this PR, those.sofilescannot run under ROCm/HIP — there's no ROCm build shipped, and nothing here
can redirect a vendored binary the way it redirects a pip install.
modly-triposplat-extension— attempted and failed. It's pure Python +PyTorch (no compiled extensions), so the ROCm redirect in this PR applies
cleanly and the diffusion stage runs correctly on the GPU. Generation
consistently fails afterwards, during mesh reconstruction
(
splat_mesh.py's Gaussian-to-mesh step): comparing CPU vs GPU on the samecaptured real output showed the accumulated density values diverge
significantly (~13x) between devices, though I could not isolate this to
a single faulty op —
_inverse_covarianceandindex_add_each checkedout correctly in isolation with the same data. This looks like a
ROCm-specific numerical issue somewhere in that extension's reconstruction
pipeline (torch 2.13.0+rocm7.2), not something this PR's detection/redirect
layer can fix. Flagging it here rather than silently leaving it unverified.
Extensions with vendored CUDA-compiled native code (e.g.
nvdiffrast,spconvbinaries built againstnvcc) cannot work under ROCm regardless ofthis change — there's no ROCm equivalent shipped. Pure-PyTorch extensions
are the ones this PR actually helps;
hunyuan3d-miniis confirmed workingend-to-end,
triposplatis not (see above).Test plan
npm run lint./node_modules/.bin/tsc --noEmit -p tsconfig.node.json(no new errors)npm test(23 Python + 77 Node tests, including 25 new tests for thedetection/redirect logic added here)