[WS2][gemm][2th merge] ffn collectives on test

[frank-2077]

Summary

• Add a deterministic Qwen3 dense FFN forward/backward: column-parallel gate/up, row-parallel down, with TP / CP / SP.
• Use fixed-tree collectives instead of NCCL.
• For CP weight grads, AllGather tokens then run full-token det_gemm_db so they match CP=1 bitwise.

Test plan

pytest tests/test_qwen_ffn.py

• Single GPU: forward/backward vs FP32 reference; infer vs train forward bitwise; token-slice batch invariance (T=8/31/32/33/64/96/128); Qwen3-8B shape smoke (H=4096, I=12288).
• Parallel correctness: TP / TP+SP / TP+CP / TP+CP+SP output and grads vs FP32, plus local-token batch invariance.
• Parallel bitwise vs TP=1/CP=1:
  • TP=2 and TP=8
  • CP=2/4/8 at T=8/32/64/96/128/256
  • TP+CP and TP+CP+SP meshes (including world=8 tp2_cp4 / tp4_cp2)
  • Qwen3-8B shapes at TP=2
• Collective cache reuse/grow/close; reject uneven SP tokens.

[coderabbitai]

Important

Review skipped

Auto reviews are disabled on base/target branches other than the default branch.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: 21115a2c-b42a-433c-8fd0-18d851f1f983

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

• 🔍 Trigger review

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}