amdxdna NPU (Strix Halo 1022:17f0): IO_PAGE_FAULT storm on every exec with IOMMU on + PMF telemetry ENODEV — linux-unstable-ogc 7.2.0-next-20260821-unstable-ogc-g2a559b27-1

[bong-water-water-bong]

Filed here because OpenGamingCollective/linux-unstable has issues disabled (it is the automated build repo and only accepts PRs). This bug concerns the linux-unstable-ogc kernel built from that repo — release v7.2.0-next-20260821-g2a559b27 (source commit 2a559b27, kernel 7.2.0-next-20260821-unstable-ogc-g2a559b27-1). Community follow-up also possible on the OGC Discord (discord.gg/4K2pZ6abQm).

---

Summary

Two NPU (amdxdna) problems on AMD Strix Halo hardware running this exact tag (v7.2.0-next-20260821-g2a559b27 → kernel 7.2.0-next-20260821-unstable-ogc-g2a559b27-1, source commit 2a559b27):

1. IO_PAGE_FAULT storm on every NPU exec when the IOMMU is enabled (the default) — turns ~7 ms NPU execs into ~10 s each (~1000× slowdown). A/B verified: amd_iommu=off restores 7.2 ms/layer with zero faults.
2. PMF telemetry broken at boot — npu4_update_counters: PMF get npu data failed, ret -19 and aie2_query_sensors: PMF get npu data failed, ret -19 (ENODEV), reproduced on this exact build.

Hardware / environment

• Host strixhalo — AMD Strix Halo (Radeon 8060S + XDNA2 NPU)
• NPU at 0000:c6:00.1: [1022:17f0] rev 11 "Strix/Krackan/Strix Halo Neural Processing Unit", kernel driver amdxdna
• Kernel: 7.2.0-next-20260821-unstable-ogc-g2a559b27-1 (clang build, Aug 31 23:36 UTC), Ubuntu 26.04 base
• Firmware: /lib/firmware/amdnpu/{1502_00,17f0_10,17f0_11}.sbin
• /dev/accel/accel0 present, amdxdna_accel_driver 1.0.0 initialized, PASID address mode enabled

Finding 1 — IO_PAGE_FAULT on every exec with IOMMU enabled

Root-caused on 2026-07-31 on this hardware (distro kernel, amdxdna 7.0.0-28 + XRT 2.21.75):

• dmesg: amdxdna 0000:c6:00.1: AMD-Vi: IO_PAGE_FAULT domain=0xNN address=0x7xxx flags=0x0027 — sustained bursts (~17k callbacks suppressed per 5 s during fast churn)
• Fault pattern: 24 KB stride (0x6000) with 512 B sub-offset (0x200) — a descriptor / Buffer-Descriptor chain walk; fault region ~6 MB, base varies per run (0x74da…, 0x70af…, 0x781d2a8…, 0x7f0b8f6…)
• Every AMDXDNA_EXEC_CMD faults. Wall time per NPU layer: 9762 ms with IOMMU on vs 7.2 ms with IOMMU off (GPU-side FFN unaffected at ~66 ms)
• strace of the hung engine: syncobj timeline waits return 0 — the NPU actually completes; it is just catastrophically slow (looks like a "hang" only because of short timeouts)
• Regression timeline: 07-24/25 (5 tok/s benchmark era, amd_iommu=off on cmdline) → clean. 07-30 18:32 (grub edit removed amd_iommu=off, 4 reboots later) → every exec faults, ~350 ms–10 s per layer. Grub backups diff-verified. All other causes eliminated: driver pristine (md5), firmware stock (md5), 3 xclbin generations, model files, module reload/sysfs rebind.

Finding 2 — NPU is the only identity-type IOMMU group

On this tag (verified live):

• iommu: Default domain type: Translated; no amd_iommu=off in cmdline
• /sys/kernel/iommu_groups/26/type (the NPU's group) = identity — the only non-DMA-FQ group on the box (both GPU groups and everything else are DMA-FQ)
• Hypothesis (matches the fault signature): identity domain + amdxdna's BO/IOVA mapping path → BDs reference unmapped IOVAs → fault per exec → fault-recovery path is the slowdown

Finding 3 — PMF telemetry returns ENODEV at boot (reproduces on this tag)

Verified live on this exact build (journal, ~8 s after boot):

amdxdna 0000:c6:00.1: npu4_update_counters: PMF get npu data failed, ret -19
amdxdna 0000:c6:00.1: aie2_query_sensors: PMF get npu data failed, ret -19

• amdxdna.ko declares depends: amd-pmf and amd_pmf is loaded (referenced by amdxdna) — yet the PMF channel returns -ENODEV
• Impact: NPU counter/sensor telemetry is dead (no perf monitoring of the NPU)

Impact on our workload

We run 1-bit LLM inference (W4A16/Q4NX) on the NPU via XRT. With the IOMMU on, prefill went from ~2 s to ~90 s and decode collapsed to ~0.1 tok/s (documented 5–40 tok/s with IOMMU off). The engine still completes and produces correct tokens, so the driver is functional when the IOMMU is disabled — which is why we initially misread this as a "hang".

Current workaround

amd_iommu=off in GRUB_CMDLINE_LINUX_DEFAULT (verified: zero faults, 7.2 ms/layer). Disabling the IOMMU for the whole SoC to keep one accelerator working is not a great security posture, so we'd prefer a real fix.

Requests

1. Investigate why the NPU's IOMMU group is identity-typed while everything else is DMA-FQ, and whether amdxdna's BO/IOVA mapping is compatible with a translated domain. Ideal outcome: NPU works with the IOMMU enabled.
2. Fix the PMF telemetry path so npu4_update_counters / aie2_query_sensors stop returning -ENODEV with amd_pmf loaded.

Happy to run A/B tests on this build (exec with/without IOMMU, strace of the exec path, more dmesg) or provide the full root-cause write-up on request.