Add Atlas Cloud as an LLM adapter (SDK v1) - #2292
binyangzhu000-sudo wants to merge 2 commits into
Conversation
Atlas Cloud (https://www.atlascloud.ai/) is an OpenAI-compatible gateway to 100+ LLMs from a single endpoint, in the same category as the existing OpenRouter/NVIDIA Build/MiniMax adapters. Atlas Cloud's own model ids are already `vendor/model` (e.g. openai/gpt-4.1-mini), unlike LiteLLM's own provider prefixes, so it's routed through the generic `custom_openai/` path (AtlasCloudLLMParameters extends OpenAICompatibleLLMParameters, mirroring NvidiaBuildLLMParameters) rather than a native LiteLLM provider prefix the way OpenRouter is. Signed-off-by: binyangzhu000-sudo <224954946+binyangzhu000-sudo@users.noreply.github.com>
|
|
for more information, see https://pre-commit.ci
|
|
| api_base: str = _ATLASCLOUD_API_BASE | ||
|
|
||
| @staticmethod | ||
| def validate(adapter_metadata: dict[str, "Any"]) -> dict[str, "Any"]: |
There was a problem hiding this comment.
Atlas validation leaves context_window unset while retaining the custom_openai/ model prefix, which LiteLLM cannot resolve for model metadata. LLM.get_context_window_size() therefore falls back to 4096 tokens. Callers such as the classifier derive their input budget from that value and truncate requests to roughly 4 KB, even when the selected Atlas model supports a much larger context window. Supply resolvable context metadata instead of relying on this fallback.
Prompt To Fix With AI
This is a comment left during a code review.
Path: unstract/sdk1/src/unstract/sdk1/adapters/base1.py
Line: 599
Comment:
**Context Window Falls Back**
Atlas validation leaves `context_window` unset while retaining the `custom_openai/` model prefix, which LiteLLM cannot resolve for model metadata. `LLM.get_context_window_size()` therefore falls back to 4096 tokens. Callers such as the classifier derive their input budget from that value and truncate requests to roughly 4 KB, even when the selected Atlas model supports a much larger context window. Supply resolvable context metadata instead of relying on this fallback.
---
For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.


What & Why
Adds Atlas Cloud as a new LLM adapter in
unstract-sdk1, in the same category as the existing OpenRouter/NVIDIA Build/MiniMax adapters. Atlas Cloud is an OpenAI-compatible gateway to 100+ LLMs from a single endpoint (https://api.atlascloud.ai/v1), identified byvendor/modelids (e.g.openai/gpt-4.1-mini,anthropic/claude-sonnet-4.6,deepseek-ai/deepseek-v3.2).Disclosure: I work at Atlas Cloud.
Implementation notes
AtlasCloudLLMParametersextendsOpenAICompatibleLLMParametersand calls the existing_validate_branded_openai_compatible()helper — the same shape asNvidiaBuildLLMParameters, notOpenRouterLLMParameters.vendor/modelstrings, unlike LiteLLM's native provider prefixes.OpenRouterLLMParametersroutes through LiteLLM'sopenrouter/prefix (a provider LiteLLM recognizes natively, per the class's own docstring: "Routed through LiteLLM's nativeopenrouter/provider so per-token costs resolve...")."atlascloud"isn't one of those, so the correct route is the genericcustom_openai/path NVIDIA Build already uses — verified this is required, not just a style choice, with a real (non-mocked)litellm.completion()call.llm1/static/atlascloud.json(required byget_json_schema(), which loads{provider}.jsonby filename) and a newAtlasCloud.pngicon (512×512, same format as the existing adapter icons) so the adapter picker doesn't break on a missing schema/icon.llm1/__init__.py(this package doesn't auto-discover adapters — each one is explicitly imported and added to__all__, same as every other adapter here).Testing
Extended
tests/test_branded_openai_adapters.py— addedAtlasCloudLLMAdapter/AtlasCloudLLMParametersto the existing parametrized branded-adapter tests (registration, blank/overriddenapi_base, schema exposure) plus one dedicated test for thecustom_openai/prefixing behavior:ruff checkpasses on all changed files.Manual test — a real, unmocked
litellm.completion()call against the production Atlas Cloud API using the adapter's ownvalidate()output: