fix: normalize WebP images for multimodal providers

[lorenzozanee]

Fixes #9743

Normalize provider-bound WebP image bytes to JPEG before strict multimodal requests while preserving existing supported PNG behavior. Add regression coverage for media normalization and OpenAI image payloads.

Modifications / 改动点

• Normalize unsupported provider-bound image formats before constructing OpenAI data URLs.
• Preserve PNG bytes and MIME metadata when the format is already supported.
• Add media utility and OpenAI provider regression tests.

Screenshots or Test Results / 运行截图和测试结果

• env -u ALL_PROXY -u all_proxy -u HTTP_PROXY -u http_proxy -u HTTPS_PROXY -u https_proxy uv run pytest -q tests/test_media_utils.py tests/test_openai_source.py (129 passed)
• uv run ruff format --check astrbot/core/utils/media_utils.py astrbot/core/provider/sources/openai_source.py tests/test_media_utils.py tests/test_openai_source.py
• uv run ruff check astrbot/core/utils/media_utils.py astrbot/core/provider/sources/openai_source.py tests/test_media_utils.py tests/test_openai_source.py

---

Checklist / 检查清单

• 😊 If there are new features added in the PR, I have discussed it with the authors through issues/emails, etc. / 如果 PR 中有新加入的功能，已经通过 Issue / 邮件等方式和作者讨论过。

• 👀 My changes have been well-tested, and "Verification Steps" and "Screenshots" have been provided above. / 我的更改经过了良好的测试，并已在上方提供了“验证步骤”和“运行截图”。

• 📚 I checked the affected WebUI instructions and screenshots in docs/zh and docs/en against the changed navigation, page structure, and labels, and updated them in this PR (or explained why no documentation update is needed). For renamed, moved, or merged entry points, I included an old entry → new entry mapping in the documentation and changelog. / 本次修改不涉及 WebUI 导航、页面结构或标签，无需文档更新。

• 🤓 I have ensured that no new dependencies are introduced, OR if new dependencies are introduced, they have been added to the appropriate locations in requirements.txt and pyproject.toml. / 本次修改未引入新依赖。

• 😮 My changes do not introduce malicious code. / 我的更改没有引入恶意代码。

Summary by Sourcery

Normalize provider-bound images to ensure strict multimodal requests receive supported formats without altering existing PNG behavior.

Bug Fixes:

• Normalize unsupported provider-bound image formats such as WebP to accepted formats before constructing OpenAI multimodal payloads.

Enhancements:

• Preserve supported image bytes and correct MIME metadata while providing fallback conversion for provider-compatible formats.

Tests:

• Add regression coverage for WebP normalization, PNG preservation, and OpenAI image payload MIME and format handling.

[sourcery-ai]

Hey - I've found 2 issues

Prompt for AI Agents

Please address the comments from this code review:

## Individual Comments

### Comment 1
<location path="astrbot/core/provider/sources/openai_source.py" line_range="191" />
<code_context>
             strict=mode == "strict",
         )
-        return image_data.to_data_url() if image_data else None
+        normalized = normalize_image_for_provider(image_data)
+        return normalized.to_data_url() if normalized else None

</code_context>
<issue_to_address>
**issue (bug_risk):** An unsupported or incorrectly labelled image whose resolved bytes are not a valid Pillow image causes `PILImage.open` to raise, and `_image_ref_to_data_url` does not catch that exception. This breaks the safe image path by propagating the decoding error instead of returning `None` and letting `_resolve_image_part` ignore the invalid attachment.

**Triggers:** When a safe-mode image reference resolves successfully but contains malformed bytes or a format unsupported by Pillow.

**Suggested fix:** Catch Pillow decoding/conversion errors in the safe path, or make `normalize_image_for_provider` return `None` for invalid image bytes.
</issue_to_address>

### Comment 2
<location path="astrbot/core/utils/media_utils.py" line_range="176" />
<code_context>
+    if image_data is None:
+        return None
+
+    supported = supported_mimes or IMAGE_PROVIDER_SUPPORTED_MIME_TYPES
+    if image_data.mime_type in supported:
+        return image_data
</code_context>
<issue_to_address>
**nitpick (bug_risk):** An explicitly supplied empty `supported_mimes` set is treated as if no provider-specific set was supplied, so the function silently falls back to JPEG, PNG, and GIF instead of honoring the empty accepted-format set. The function can therefore return or create a format that the caller explicitly declared unsupported.

**Triggers:** When a caller passes `supported_mimes=set()` to indicate that no formats are accepted.

**Suggested fix:** Use `supported_mimes if supported_mimes is not None else IMAGE_PROVIDER_SUPPORTED_MIME_TYPES`.

```suggestion
    supported = supported_mimes if supported_mimes is not None else IMAGE_PROVIDER_SUPPORTED_MIME_TYPES
```
</issue_to_address>

Sourcery assessment

Approval pending. 1 finding to address first.

Blocking findings: astrbot/core/provider/sources/openai_source.py:191

---

Sourcery is free for open source - if you like our reviews please consider sharing them ✨

• X
• Mastodon
• LinkedIn
• Facebook

[sourcery-ai]

issue (bug_risk): An unsupported or incorrectly labelled image whose resolved bytes are not a valid Pillow image causes PILImage.open to raise, and _image_ref_to_data_url does not catch that exception. This breaks the safe image path by propagating the decoding error instead of returning None and letting _resolve_image_part ignore the invalid attachment.

Triggers: When a safe-mode image reference resolves successfully but contains malformed bytes or a format unsupported by Pillow.

Suggested fix: Catch Pillow decoding/conversion errors in the safe path, or make normalize_image_for_provider return None for invalid image bytes.

[sourcery-ai]

nitpick (bug_risk): An explicitly supplied empty supported_mimes set is treated as if no provider-specific set was supplied, so the function silently falls back to JPEG, PNG, and GIF instead of honoring the empty accepted-format set. The function can therefore return or create a format that the caller explicitly declared unsupported.

Triggers: When a caller passes supported_mimes=set() to indicate that no formats are accepted.

Suggested fix: Use supported_mimes if supported_mimes is not None else IMAGE_PROVIDER_SUPPORTED_MIME_TYPES.

Suggested change

	supported = supported_mimes or IMAGE_PROVIDER_SUPPORTED_MIME_TYPES
	supported = supported_mimes if supported_mimes is not None else IMAGE_PROVIDER_SUPPORTED_MIME_TYPES