Skip to content

Add gated GLM-5.2 bit-exact compatibility paths - #4

Draft
zrr1999 wants to merge 7 commits into
PFCCLab:mainfrom
zrr1999:glm52-bit-exact-alignment
Draft

Add gated GLM-5.2 bit-exact compatibility paths#4
zrr1999 wants to merge 7 commits into
PFCCLab:mainfrom
zrr1999:glm52-bit-exact-alignment

Conversation

@zrr1999

@zrr1999 zrr1999 commented Jul 27, 2026

Copy link
Copy Markdown

Summary

  • add opt-in accuracy-compatible DSA execution and backward behavior
  • add opt-in RMSNorm formulation/backward compatibility and wire it through MTP specs
  • add opt-in explicit fp32 router gating for deterministic GLM-5.2 alignment

All compatibility paths are gated; default Megatron behavior remains unchanged.

Validation

  • focused unit tests and Ruff checks passed for the individual commits
  • official-weight GLM-5.2 reduced-profile probes ran on NVIDIA H800 with CUDA/bfloat16
  • forward step-0 loss matches the independent Paddle path at 12.416634559631348
  • multiple named backward boundaries are bit-exact; selected embedding gradient, optimizer, and multi-step parity are not yet established

Status

Draft while strict cross-framework backward localization continues. This PR does not claim full-model or 100-step bit-exact completion.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant