Skip to content

Add Tempr's DeepInfra, Together AI, Fireworks AI and Hugging Face models - #7587

Closed
chrissalomon wants to merge 1 commit into
anomalyco:devfrom
chrissalomon:add-tempr-inference-hosts
Closed

chrissalomon wants to merge 1 commit into
anomalyco:devfrom
chrissalomon:add-tempr-inference-hosts

Conversation

@chrissalomon

@chrissalomon chrissalomon commented Sep 20, 2026

Copy link
Copy Markdown
Contributor

Fourth batch for Tempr's AI Gateway (api.temprhq.io/v1), after #7448, #7452 and #7586: the inference hosts it serves — DeepInfra (49), Together AI (10), Fireworks AI (24) and Hugging Face (54). Tempr relays each on the caller's own key for that host, with no markup on tokens.

Each entry mirrors the host's own base_model attribution, real per-model deltas and pricing, since it's the same model on the same account. Reasoning is the one thing not mirrored: it comes from Tempr's GET /v1/models, so it can differ from the host's own entry where Tempr deliberately exposes less (below).

Reasoning controls per host, not per lab. These were established by live testing against each host, because the same model family genuinely behaves differently across them: GLM-5.3's effort levels are a proper ladder on Fireworks, can't stop reasoning on DeepInfra, and are inverted on Together AI (where "low" produces no thinking and "max" does), so Together's GLM models are listed with no levels at all, and each of those files says why rather than leaving the empty set unexplained. Fireworks rejects any unknown field, so it takes reasoning_effort only. Bedrock's entries are in #7586. Each file's leading comment records the fields a caller sets for that model.

Hugging Face is listed with no reasoning options at all. It's a router rather than a host: of its models, most have more than one live upstream provider and none of the reasoning models has only one, so there's no deterministic behavior to advertise — zai-org/GLM-5.3-Flash is served by Together AI and DeepInfra at once, and those two disagree. Tempr therefore sends no reasoning settings there, and each file says so.

Not included: 78 models these hosts define themselves, whose own entries carry no base_model (Together AI's Qwen/…-Turbo variants, Fireworks' accounts/fireworks/models/…, and similar). Each needs a lab entry created under its creating lab first, which is per-model research rather than a mirror, so they're better as their own change.

Docs: https://temprhq.io/docs/gateway-chat-completions#reasoning

🤖 Generated with Claude Code

@chrissalomon
chrissalomon force-pushed the add-tempr-inference-hosts branch from a8e25a0 to f9e63d1 Compare September 20, 2026 18:51
@github-actions

Copy link
Copy Markdown
Contributor

Action items

  • [high] [possible mistake] providers/tempr/models/together-ai/zai-org/GLM-5.3.toml:7 - Check: Relay reasoning_options must match this host’s real controls and same-surface peers (AGENTS.md → Reasoning options; audit skill evidence bar). Why: Tempr sets reasoning_options = [] for Together GLM-5.3 / GLM-5.3-Flash / GLM-5.2 / GLM-5, but the established providers/togetherai/ peers expose caller controls (effort low|high|max on 5.3/5.3-Flash; toggle + high|max on 5.2; toggle on GLM-5). The PR body says entries mirror each host and that Together GLM has “no levels,” which contradicts those peer files. Empty means no control, not uncertainty—if Tempr’s Together path truly differs, that needs affirmative proof; otherwise callers lose documented controls. Action: Reconcile with providers/togetherai/models/zai-org/GLM-5*.toml: either copy the peer option sets (and wire comments), or keep [] only with concrete evidence that Tempr’s Together route does not forward those controls (and update the PR body so it does not claim a host mirror for these).
  • [high] [violation] providers/tempr/models/together-ai/nvidia/nemotron-3-ultra-550b-a55b.toml:5 - Check: After base_model, provider files must be override-only—no restated identical lab fields (AGENTS.md → Override-only provider files). Why: This file (and the same pattern on together-ai/zai-org/GLM-5.2.toml and GLM-5.toml) copies lab-identical name, family, attachment, reasoning, temperature, tool_call, open_weights, and full [modalities] that already resolve from base_model. That is a documented merge blocker and bloats the catalog with false “deltas.” Action: Strip fields that match the lab entry; keep only real Together deltas (cost, reasoning_options, limit, status, and any truly different dates/structured_output/name).
  • [medium] [violation] providers/tempr/models/huggingface/deepseek-ai/DeepSeek-V4-Flash-Vision-Exp.toml:6 - Check: Do not restate unchanged inherited primitives after base_model. Why: open_weights = true is already true on models/deepseek/deepseek-v4-flash-vision-exp.toml (same issue on HF DeepSeek-V4-Pro-0813, GLM-5.3, and GLM-5.3-Flash). Restating identical lab values violates override-only rules. Action: Remove redundant open_weights = true (and any other identical lab copies) from those HF Tempr files; keep only real host deltas (cost, reasoning_options, limits, differing description, etc.).
  • [medium] [possible mistake] providers/tempr/models/huggingface/openai/gpt-oss-120b.toml:6 - Check: On a multi-provider router path, [] is valid only with affirmative no-control evidence; do not drop controls the upstream HF catalog documents without that bar. Why: Tempr HF reasoners use blanket reasoning_options = [], while several mirrored HF entries document real controls (e.g. gpt-oss-120b/20b low|medium|high; GLM-5.3 / 5.3-Flash low|high|max; Qwen3.8-2.4T / 3.8-27B effort; DeepSeek V4 Pro 0813 / Vision-Exp effort; MiMo-V2.5 toggle+effort). The PR’s “multi-upstream / Tempr doesn’t pass settings” rationale may be correct for Tempr’s gateway, but it is not evidenced per model against those HF peers, so the catalog may under-advertise controls if any HF route is single-provider or does forward settings. Action: For each HF reasoner, either keep [] with a short leading note tied to Tempr’s no-forward policy and confirm no single-upstream exception, or align reasoning_options with the HF peer when Tempr does forward that model’s controls.

- The mirror copied each host entry's root scalars and tables
  verbatim, including ones restating the lab's own values (Together's
  Nemotron and GLM entries, Hugging Face's open_weights). Now compared
  against the lab entry and kept only where they genuinely differ.
- Where a host lists controls but Tempr exposes none, the leading
  comment now says so and why, instead of reading as a gap: Together
  AI's GLM effort levels are inverted, and Hugging Face is a router
  Tempr forwards no reasoning settings to at all.
@chrissalomon
chrissalomon force-pushed the add-tempr-inference-hosts branch from f9e63d1 to bf0519d Compare September 20, 2026 19:01
@github-actions

Copy link
Copy Markdown
Contributor

No actionable findings.

@github-actions github-actions Bot added the reviewer: ready Automated review found no actionable items label Sep 20, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

reviewer: ready Automated review found no actionable items

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant