Skip to content

fix: detect vision patch size for unsplit (HF-format) Qwen3-VL#1811

Merged
leejet merged 1 commit into
leejet:masterfrom
fszontagh:fix/qwen3vl-hf-patch-embed
Jul 23, 2026
Merged

fix: detect vision patch size for unsplit (HF-format) Qwen3-VL#1811
leejet merged 1 commit into
leejet:masterfrom
fszontagh:fix/qwen3vl-hf-patch-embed

Conversation

@fszontagh

Copy link
Copy Markdown
Contributor

Summary

HF/ComfyUI-format Qwen3-VL text encoders keep the vision patch embed unsplit as visual.patch_embed.proj.weight. The existing detection only matches the split visual.patch_embed.proj.0.weight, so vision.patch_size silently stays at the Qwen2.5-VL default of 14 instead of 16. Any model that builds the Qwen3-VL vision tower (e.g. Mage-Flow) then fails to load such an encoder with patch_embed.proj.weight got [16, 16, ...], expected [14, 14, ...].

This reads the patch size from the unsplit weight's kernel extent (ne[0]), and additionally derives hidden_size from visual.patch_embed.proj.bias (the split name visual.patch_embed.bias does not match the unsplit .proj.bias). GGUF-sourced and split-format encoders are unaffected.

The model.language_model.* naming difference is already handled by scripts/convert_qwen3_vl.py; this only fixes the remaining patch-size detection, which that script does not touch.

Related Issue / Discussion

Related to #1708 - users hit this loading the Comfy-Org bf16 Qwen3-VL; the convert script fixes naming but the unsplit patch size still mis-detects.

Additional Information

Verified with the bf16 Qwen3-VL-4B from Comfy-Org/Krea-2, run through scripts/convert_qwen3_vl.py, then loaded as --llm for Mage-Flow: before this change it fails with patch_embed.proj.weight ... expected [14, 14, ...]; after, it loads and generates correctly. GGUF encoders continue to work unchanged.

Checklist

@fszontagh
fszontagh force-pushed the fix/qwen3vl-hf-patch-embed branch from ad52e9d to de9646f Compare July 23, 2026 05:57
@fszontagh
fszontagh force-pushed the fix/qwen3vl-hf-patch-embed branch from de9646f to 9334ec8 Compare July 23, 2026 06:41
@leejet
leejet merged commit 5114672 into leejet:master Jul 23, 2026
12 checks passed
@fszontagh
fszontagh deleted the fix/qwen3vl-hf-patch-embed branch July 23, 2026 15:02
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants