issue/1393 - nsa paged attention and nsa compress paged cache for metax#1394
Open
Lfan-ke wants to merge 1 commit into
Open
issue/1393 - nsa paged attention and nsa compress paged cache for metax#1394Lfan-ke wants to merge 1 commit into
Lfan-ke wants to merge 1 commit into
Conversation
|
You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard. |
Author
|
@codex review please. read code at pr and repositories. |
|
You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard. |
…ess_paged_cache The two operators only had cuda/ and nvidia/ implementations. Each gains metax/*_metax.h + metax/*_metax.maca and an INFINI_DEVICE_METAX branch in operator.cc; the device kernels reuse the shared cuda/kernel.cuh unchanged. That shared header needs two platform guards, keyed on the compiler's own __MACA__ / __MACACC__ rather than on ENABLE_METAX_API, which xmake.lua adds at project scope and would therefore also reach nvcc in a --nv-gpu=y --metax-gpu=y build: - <cuda_bf16.h> and <cuda_fp16.h> do not exist on MACA; metax_kernel_common.h provides the equivalents. - #pragma unroll on a runtime-bounded loop is rejected by MACA's clang under -Werror, so MACA joins the existing ILUVATAR/HYGON guard. Signed-off-by: 林晨 (Leo Cheng) <chengkelfan@qq.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
关联 issue:#1393
做了什么
为
nsa_paged_attention与nsa_compress_paged_cache补上metax/后端,使 NSA 解码路径在 Metax GPU 上可用。这两个算子此前只有cuda/与nvidia/实现。每个算子新增
metax/*_metax.h+metax/*_metax.maca,并在operator.cc的 create / get-workspace / calculate / destroy 四处接入INFINI_DEVICE_METAX。设备端 kernel 直接复用共享的cuda/kernel.cuh,不复制一行计算逻辑。xmake/metax.lua是 glob,无需改构建。.maca与nvidia/*.cu的差异是同族paged_attention已经确立的那几类机械替换:运行时头(mc_runtime.h/hc_runtime.h)、devices/metax/*、命名空间、cudaStream_t→hcStream_t、device::nvidia::Handle→device::metax::Handle、CHECK_CUDA→CHECK_METAX。共享头需要两处平台守卫
cuda/kernel.cuh由nvidia/*.cu与新增的metax/*.maca共同包含,两处平台差异必须在头里区分。其一,
<cuda_bf16.h>与<cuda_fp16.h>在 MACA 上不存在,用 mxcc 编译时报fatal error: cuda_bf16.h: file not found。仓库既有做法(见ldexp/cuda/kernel.cuh)是共享头不包含这两个文件,交给.maca侧先包含的devices/metax/metax_kernel_common.h提供,它同时给出using __nv_bfloat16 = __hpcc_bfloat16;这类别名。其二,
#pragma unroll作用于运行时边界的循环时会被 MACA 的 clang 拒绝(-Wpass-failed=transform-warning,而xmake/metax.lua用-Werror编译.maca)。kernel.cuh里这些 pragma 本来就被#if !defined(ENABLE_ILUVATAR_API) && !defined(ENABLE_HYGON_API)守着,同为 clang 系的这两家早已踩过同一个坑,把 MACA 并进同一个守卫即可。计算逻辑一行未动,
blockReduceSum128保持原样。守卫为什么用
__MACA__而不是ENABLE_METAX_APIENABLE_METAX_API由xmake.lua:159在项目作用域添加(不在任何target()内),而nv-gpu(:49)与metax-gpu(:146)是彼此独立的开关。因此在--nv-gpu=y --metax-gpu=y的联合构建里,nvcc 编译nsa_paged_attention_nvidia.cu时同样会拿到-DENABLE_METAX_API,而这个翻译单元正是通过#include "../cuda/kernel.cuh"引入本头的。若拿它做厂商分支,NVIDIA 侧会被跳过
<cuda_bf16.h>的包含,构建直接失败。改用编译器自带的宏则不受项目级 define 影响:nvcc 定义
__CUDACC__而不定义__MACA__/__MACACC__;mxcc 反之。仓库里scatter_metax.maca:257、index_copy_metax.maca:145用的也是这一对宏。测试
MetaX C500(MACA 3.5.3.20 / 驱动 3.8.30):
.maca以-Werror编译,两个算子在 F16 与 BF16 下均Test passed。NVIDIA 侧(nvcc 12.0):把两种守卫写法放进同一个
.cu,加-DENABLE_METAX_API编译(模拟联合构建)。ENABLE_METAX_API那一支被 nvcc 选中,__MACA__那一支不会——nvcc 定义__CUDACC__而不定义__MACA__/__MACACC__。metax/后端由 130 增加到 132。