Skip to content

issue/1393 - nsa paged attention and nsa compress paged cache for metax#1394

Open
Lfan-ke wants to merge 1 commit into
InfiniTensor:mainfrom
Lfan-ke:issue/1393
Open

issue/1393 - nsa paged attention and nsa compress paged cache for metax#1394
Lfan-ke wants to merge 1 commit into
InfiniTensor:mainfrom
Lfan-ke:issue/1393

Conversation

@Lfan-ke

@Lfan-ke Lfan-ke commented Jul 13, 2026

Copy link
Copy Markdown

关联 issue:#1393

做了什么

nsa_paged_attentionnsa_compress_paged_cache 补上 metax/ 后端,使 NSA 解码路径在 Metax GPU 上可用。这两个算子此前只有 cuda/nvidia/ 实现。

每个算子新增 metax/*_metax.h + metax/*_metax.maca,并在 operator.cc 的 create / get-workspace / calculate / destroy 四处接入 INFINI_DEVICE_METAX。设备端 kernel 直接复用共享的 cuda/kernel.cuh,不复制一行计算逻辑。xmake/metax.lua 是 glob,无需改构建。

.macanvidia/*.cu 的差异是同族 paged_attention 已经确立的那几类机械替换:运行时头(mc_runtime.h / hc_runtime.h)、devices/metax/*、命名空间、cudaStream_thcStream_tdevice::nvidia::Handledevice::metax::HandleCHECK_CUDACHECK_METAX

共享头需要两处平台守卫

cuda/kernel.cuhnvidia/*.cu 与新增的 metax/*.maca 共同包含,两处平台差异必须在头里区分。

其一,<cuda_bf16.h><cuda_fp16.h> 在 MACA 上不存在,用 mxcc 编译时报 fatal error: cuda_bf16.h: file not found。仓库既有做法(见 ldexp/cuda/kernel.cuh)是共享头不包含这两个文件,交给 .maca 侧先包含的 devices/metax/metax_kernel_common.h 提供,它同时给出 using __nv_bfloat16 = __hpcc_bfloat16; 这类别名。

其二,#pragma unroll 作用于运行时边界的循环时会被 MACA 的 clang 拒绝(-Wpass-failed=transform-warning,而 xmake/metax.lua-Werror 编译 .maca)。kernel.cuh 里这些 pragma 本来就被 #if !defined(ENABLE_ILUVATAR_API) && !defined(ENABLE_HYGON_API) 守着,同为 clang 系的这两家早已踩过同一个坑,把 MACA 并进同一个守卫即可。

计算逻辑一行未动,blockReduceSum128 保持原样。

守卫为什么用 __MACA__ 而不是 ENABLE_METAX_API

ENABLE_METAX_APIxmake.lua:159 在项目作用域添加(不在任何 target() 内),而 nv-gpu:49)与 metax-gpu:146)是彼此独立的开关。因此在 --nv-gpu=y --metax-gpu=y 的联合构建里,nvcc 编译 nsa_paged_attention_nvidia.cu 时同样会拿到 -DENABLE_METAX_API,而这个翻译单元正是通过 #include "../cuda/kernel.cuh" 引入本头的。

若拿它做厂商分支,NVIDIA 侧会被跳过 <cuda_bf16.h> 的包含,构建直接失败。改用编译器自带的宏则不受项目级 define 影响:

#if !defined(__MACA__) && !defined(__MACACC__)
#include <cuda_bf16.h>
#include <cuda_fp16.h>
#endif

nvcc 定义 __CUDACC__ 而不定义 __MACA__ / __MACACC__;mxcc 反之。仓库里 scatter_metax.maca:257index_copy_metax.maca:145 用的也是这一对宏。

测试

MetaX C500(MACA 3.5.3.20 / 驱动 3.8.30):

XMAKE_ROOT=y xmake f --metax-gpu=y --use-mc=y -y && xmake && xmake install
python test/infiniop/nsa_paged_attention.py --metax
python test/infiniop/nsa_compress_paged_cache.py --metax

.maca-Werror 编译,两个算子在 F16 与 BF16 下均 Test passed

NVIDIA 侧(nvcc 12.0):把两种守卫写法放进同一个 .cu,加 -DENABLE_METAX_API 编译(模拟联合构建)。ENABLE_METAX_API 那一支被 nvcc 选中,__MACA__ 那一支不会——nvcc 定义 __CUDACC__ 而不定义 __MACA__ / __MACACC__

metax/ 后端由 130 增加到 132。

@Lfan-ke Lfan-ke requested a review from a team July 13, 2026 11:35
@chatgpt-codex-connector

Copy link
Copy Markdown

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.

@Lfan-ke

Lfan-ke commented Jul 13, 2026

Copy link
Copy Markdown
Author

@codex review please. read code at pr and repositories.

@chatgpt-codex-connector

Copy link
Copy Markdown

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.

@Lfan-ke Lfan-ke changed the title issue/1393 nsa_paged_attention / nsa_compress_paged_cache 的 Metax 后端支持 issue/1393 - feat(metax): add Metax backends for nsa_paged_attention and nsa_compress_paged_cache Jul 14, 2026
@Lfan-ke Lfan-ke changed the title issue/1393 - feat(metax): add Metax backends for nsa_paged_attention and nsa_compress_paged_cache issue/1393 - nsa paged attention and nsa compress paged cache for metax Jul 14, 2026
…ess_paged_cache

The two operators only had cuda/ and nvidia/ implementations. Each gains metax/*_metax.h + metax/*_metax.maca and an INFINI_DEVICE_METAX branch in operator.cc; the device kernels reuse the shared cuda/kernel.cuh unchanged.

That shared header needs two platform guards, keyed on the compiler's own __MACA__ / __MACACC__ rather than on ENABLE_METAX_API, which xmake.lua adds at project scope and would therefore also reach nvcc in a --nv-gpu=y --metax-gpu=y build:

- <cuda_bf16.h> and <cuda_fp16.h> do not exist on MACA; metax_kernel_common.h provides the equivalents.
- #pragma unroll on a runtime-bounded loop is rejected by MACA's clang under -Werror, so MACA joins the existing ILUVATAR/HYGON guard.

Signed-off-by: 林晨 (Leo Cheng) <chengkelfan@qq.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant