* fix(cost): split a provider-reported total between input and output by the cost map rate ratio
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(cost): expect the kyma reported-cost headers to follow the cost map rate split
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(cost): keep the kyma test file at its original formatting
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(cost): price the response-cost header split from the served model, not the request alias
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: devin-ai-integration[bot] <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* feat: add Hostinger Router as an OpenAI-compatible provider
Adds hostinger_router (https://router.hostinger.com/v1) to the
JSON-configured openai_like providers with chat completions, responses,
messages, and embeddings endpoints, cost-map entries for claude-sonnet-5,
gpt-5.2 and the two text-embedding-3 models (credits converted at
1 credit = $0.01, sourced from the public
https://router.hostinger.com/v1/models catalog), the endpoint-support
matrix, Admin UI add-model form fields and logo, and respx-mocked unit
tests covering all declared endpoints.
Also routes /v1/embeddings for JSON-configured providers that declare it
in providers.json; previously embedding calls for these providers fell
through to LiteLLMUnknownProvider.
* fix(hostinger_router): match prices to the live catalog, drop retired max_tokens, regenerate the endpoint matrix
* fix(hostinger_router): keep api_base detection off, send embeddings headers as HTTP headers, drop generic key fallback
Unprefixed models called with Hostinger's api_base keep their own provider, JSON provider
embeddings no longer borrow OPENAI_LIKE_API_KEY or litellm.api_key, and the openai_like
embedding handler sends extra_headers as request headers instead of in the JSON body
* fix(openai_like): route Hostinger messages and responses per model, keep the configured key header
Native /v1/messages and /v1/responses calls for a JSON provider now also follow the
model's cost-map supported_endpoints, so Hostinger models without the route go
through chat completions instead of a 403. Embedding key headers replace
extra_headers case-insensitively, and pinstripes stops declaring /v1/embeddings.
* fix(openai_like): let embeddings extra_headers override the key's headers as chat does
Embeddings now match chat through the OpenAI SDK, so an extra_headers entry replaces the key's header of the same name in any letter case. The Hostinger capability test checks that the supports_* lookups agree with each cost-map row instead of pinning vendor flags
* fix(openai_like): send numeric embeddings extra_headers as text and keep Hostinger's 1h cache TTL
A config.yaml header like X-Org-Id: 12345 loads as an int, which the embeddings header check rejected with a 500. Int and float values now go out as their decimal text. Hostinger Router accepts and bills a 1h cache_control TTL on /v1/messages, so its provider entry opts into keeping it.
* fix(openai_like): send Hostinger's 1h cache TTL as the 5-minute default so cache writes bill at its charge
* fix(openai_like): route Hostinger natives per model from providers.json, refuse non-text embeddings headers unless drop_params
* fix(openai_like): refuse Host, Content-Length and Transfer-Encoding in embeddings extra_headers unless drop_params
* fix(openai_like): refuse malformed embeddings extra_headers and ignore transport headers
Header names must be HTTP tokens and values printable ASCII on one line, so a
colon in a name or a line break in a value answers 400 unless drop_params drops
it. Host, Content-Length and Transfer-Encoding are ignored, since a 400 for them
sends the call to the deployment's fallback with the same headers.
* fix(openai_like): trim embeddings extra_headers values and drop values HTTP cannot carry
HTTP ignores spaces and tabs around a header value, so values are trimmed
before they are sent. A value that is not printable ASCII on one line after
trimming is dropped with a warning instead of failing the call, since the
proxy adds headers itself from key aliases and forwarded client headers that
the caller cannot fix. A malformed header name or a non-text value still
answers 400 unless drop_params drops it.
* fix(openai_like): ignore hop-by-hop extra_headers and trim line breaks around values
TE, Upgrade, Connection, Keep-Alive, Proxy-Connection and Trailer in
extra_headers are now ignored like Host, Content-Length and
Transfer-Encoding, since litellm builds the request and TE: gzip fails
HTTP/2 while Upgrade makes some upstreams hang. Line breaks at the ends of a
value are trimmed along with spaces and tabs, so a secret read from a file
with a trailing newline is sent instead of dropped. The drop warning now
names the model.
* fix(embeddings): keep one request's extra_headers out of the deployment's headers and ignore Expect
embedding() merged a request's extra_headers into the headers dict it got
from the deployment's litellm_params. The router only shallow-copies those
params, so the merge wrote the caller's headers into the deployment itself,
and later embeddings calls through that deployment sent them too. Build a
new dict instead, and drop the None checks that can no longer be true.
Also treat Expect as a transport header that hostinger_router embeddings
never forwards, and name the model with %r in the dropped-value warning.
* fix(embeddings): keep name and value pairs working in headers and answer 400 for other non-object values
The previous commit merged headers and extra_headers with dict unpacking,
which only takes a mapping, so a list of name and value pairs that
dict.update used to accept now raised a TypeError and the proxy answered
500. Merge through dict() so pairs keep working, and answer 400 for any
other value, or drop it under drop_params, as the litellm convention does
for an unsupported param.
---------
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
* feat(ui): show ai&'s name and logo in the dashboard
The Add Model form already lists ai& from its provider metadata, but the dashboard had no entry for it, so the provider showed no logo and model tables printed the raw aiand slug. Adds ai& to the dashboard provider list with its official mark, plated in dark mode so the red and black mark keep their colours, and the aiand/openai/gpt-oss-120b model placeholder
* fix(aiand): retire deepseek-v4-flash, glm-5.2 and kimi-k2.7-code
ai& retired these three models on 2026-10-27 and https://api.aiand.com/v1/models now lists ten. The rows keep their prices with a deprecation_date, so earlier spend still prices, and aiand/* stops listing them, the same way baseten handles retired rows. The other ten rows already match the live prices, context windows and output limits
* feat(aiand): add deepseek-v4.1-flash-1017
ai& launches deepseek-ai/deepseek-v4.1-flash-1017 on 2026-11-02. It is served on https://staging-api.aiand.com/v1/models with the same price, context window, output limit and capabilities as deepseek-v4.1-flash, so its row copies that one
* fix(aiand): retire deepseek-v4.1-flash
ai& retires deepseek-ai/deepseek-v4.1-flash on 2026-10-30, now that deepseek-v4.1-flash-1017 is live in production on https://api.aiand.com/v1/models with the same price, limits and capabilities. Its row keeps its prices with a deprecation_date, like the other three retired ai& rows
* fix(aiand): retire deepseek-v4.1-flash on 2026-11-06
https://api.aiand.com/v1/models now gives deepseek-v4.1-flash a deprecation_date of 2026-11-05T15:00Z, which is 2026-11-06 00:00 in Japan, so its row uses 2026-11-06 instead of 2026-10-30 and aiand/* keeps listing it until then. The wildcard test now leaves a model out only once its retirement date has passed
* test(aiand): derive retired models from the rows instead of pinning dates
The wildcard test now reads each row's deprecation_date, so it holds whichever dates ai& sets, and the test that only restated the JSON dates is gone
* fix(aiand): retire models on their UTC date while the proxy runs, and add muse-spark-open
The wildcard listing now checks each row's deprecation_date when it expands, against the UTC date the field
documents, so a model drops on its day without a restart. deepseek-v4.1-flash now carries 2026-11-05, the
UTC day ai& retires it, and meta/muse-spark-open gets its price row from ai&'s live /v1/models
* fix(aiand): hide retired models only from bundled listings, including the wildcard health check
* fix(aiand): share the UTC retirement check with Together AI and the health check sort
---------
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
* fix(cost): bill a provider reported usage.cost on every path unless the deployment is priced
* test(integration): a reported cost is attributed to output in the gateway parity cases
* fix(cost): skip a chat usage.cost that already holds litellm's stamped bill and Responses API objects
* fix(cost): record litellm's bill on the Responses terminal event so a later pricing pass does not read it as the upstream's charge
* fix(cost): mark usage.cost litellm wrote as its own bill and keep a breakdown for a reported cost
* fix(cost): keep the token breakdown when a header reported cost repeats an already priced bill
* fix(cost): read the stored breakdown total through the logging object's typed field
* refactor(cost): name the providers billed from their own usage fields for what they are
* fix(cost): keep the upstream's usage.cost beside litellm's bill and stamp a zero operator price on the stream frame
---------
Co-authored-by: devin-ai-integration[bot] <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Adds Kyma API (kyma/<model>) as a JSON-configured OpenAI-compatible provider. Spend reads the usage.cost Kyma reports, with 83 catalog rows as the fallback price, and image and audio routes refuse before calling Kyma. Carries over #39799 with its author's commits kept
* feat: add StreamLake provider metadata
* fix(streamlake): complete provider discovery and test request flows
* test(streamlake): assert cost and tool-param invariants instead of pinning vendor prices
* fix(streamlake): accept tools and reasoning on endpoint ids and match the current price page
Endpoint ids have no cost-map row, so tools and reasoning_effort were rejected
before reaching StreamLake. The provider now declares both capabilities, and
named non-reasoning rows opt out explicitly.
The rows now follow StreamLake's 2026-09-17 price page: drop the KAT-Coder and
Qwen3 rows it no longer serves, add the served models that had no price, drop
account-level rpm and tpm, and fill missing context windows. StreamLake is also
registered for streamlake/* wildcards and its messages and responses support.
* fix(streamlake): regenerate the endpoint matrix and type the new tests
The backup endpoint matrix advertised chat only while the root map listed
Messages and Responses, so /public/endpoints hid both. Regenerating with
scripts/generate_provider_endpoints_support.py makes the two files match.
Types the StreamLake tests and sets the Add Model test's API key with
fireEvent.change, per the dashboard test guide.
* fix(streamlake): drop the retired max_tokens field from the StreamLake cost-map rows
* fix(streamlake): price the four served models StreamLake publishes rates for and add the Qwen3.8 cache prices
* fix(streamlake): send token caps as max_completion_tokens and mark image models
StreamLake ignores max_tokens on kimi-k2.6, kimi-k2.7-code, qwen3.6-27b and
qwen3.6-35b-a3b, while every served model honors max_completion_tokens, so
the JSON provider now maps max_tokens to max_completion_tokens instead of the
reverse. Rows whose models read an image in a live probe set supports_vision,
and the Qwen3.8 rows drop the explicit cache write price, which this route
never uses since cache_control is dropped
* fix(streamlake): keep cache_control marks, price explicit cache reads, and use API input limits
StreamLake reads cache_control, so the provider keeps the marks through a preserve_cache_control flag. Qwen3.8 explicit cache hits now use their own read price, rows with a cache read price declare prompt caching, three models StreamLake serves without reasoning drop reasoning_effort, and max_input_tokens follows the limits the API reports.
* fix(utils): warn when a custom_provider_map name matches a built-in provider
---------
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
* fix(vertex_ai): split Gemini SSE lines on CR/LF only so U+2028 in text keeps the stream
The Gemini and Vertex AI streaming path read the SSE body with httpx
iter_lines()/aiter_lines(). Those follow str.splitlines(), which also breaks
on U+0085, U+2028, U+2029, \x0b, \x0c and \x1c-\x1e. JSON leaves U+0085,
U+2028 and U+2029 unescaped inside strings, so a data: line whose text holds
one of them was cut in two. The tail is not a data: line and the head never
parses, and the accumulate-and-retry fallback then carries the unparsed
prefix forward, so every later event is appended to it and the stream ends
with no more content and no usage.
SSE lines end only at CRLF, LF or CR. The new iter_sse_lines/aiter_sse_lines
split decoded text on exactly those, treating a CR at the end of one chunk and
an LF at the start of the next as one break. The first request and every
long-decode hop now read through them.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* fix(streaming): keep merged think tags balanced and split shared SSE lines only on CR/LF
Close an open <think> block on the first content or tool-call delta and at stream end, reopen it when reasoning returns, and route the shared chat completions stream through the SSE line splitter so U+2028 and U+2029 inside JSON no longer drop chunks.
* fix(streaming): exact-match the SSE [DONE] marker in BaseModelResponseIterator
* fix(streaming): split every SSE reader on CR/LF only and keep the logged think block closed
Switch the remaining httpx stream readers (Anthropic chat, Google GenAI,
Interactions, Codestral, openai_like, Predibase, LangGraph, Vertex Agent
Engine, A2A providers, Azure AI agents, sandboxes, the A2A endpoint, and
others) from iter_lines/aiter_lines to iter_sse_lines/aiter_sse_lines, so a
raw U+2028 or U+2029 inside a JSON string no longer splits an event.
Keep the chunk that closes a think block in the logged and cached response
when a stream ends without a finish_reason, close the block before a legacy
function_call, and make sync Codestral FIM streaming send stream=true and use
the text-completion-codestral stream parser.
Test fakes that only mocked iter_lines now stream real httpx bytes and
assert the content they feed in.
* fix(streaming): split buffered SSE parsers only on CR and LF, shard the codestral tests
Buffered SSE and JSONL parsers used str.splitlines(), which also breaks on
U+2028, U+2029 and U+0085 that JSON allows raw inside strings. They now use
split_sse_lines from sse_lines.py, which also gains a str.split fast path.
Adds tests/unit/llms/codestral to the unit workflow shard.
* fix(streaming): keep U+2028 in restamped Anthropic frames and fold CRLF before splitting SSE lines
* test: type the new stream tests, drop a comment, and benchmark the SSE line splitter
* test: type the iterator class parameters in the line-separator stream tests
* fix(responses): stream tool call arguments opened in a text chunk
* fix(responses): hold a tool call that shares a chunk with text until the text ends
* test: type the a2a SSE handler helper and the sandbox fake responses
---------
Co-authored-by: Tim Vink <vinktim@gmail.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
Co-authored-by: michelligabriele <gabriele.michelli@icloud.com>
* feat(providers): add ZeroGPU as an OpenAI-compatible provider
Registers zerogpu through the JSON provider registry so zerogpu/<model> routes to
https://api.zerogpu.ai/v1/chat/completions with ZEROGPU_API_KEY and an optional ZEROGPU_API_BASE.
Adds LlmProviders.ZEROGPU, the constants.py provider and endpoint lists, cost map entries for the
10 ZeroGPU text generation models, the endpoint matrix, the Add Model form fields, the README row
and the dashboard provider entry
* feat(zerogpu): add capability flags verified against the live API
Live calls to https://api.zerogpu.ai/v1/chat/completions confirmed tools, tool_choice and json_schema
response_format on 8 of the 10 catalog text models, cached token reporting on deepseek-v4.1-flash,
gpt-4.1-mini, gpt-5.4-nano and gpt-5.6-luna, and reasoning_effort on gpt-5.6-luna. LFM2.5-1.2B-Instruct
answers stream requests with plain JSON instead of SSE and LFM2.5-1.2B-Thinking returns 500 on every
request, so both stay out of the cost map until the API serves them correctly
* feat(zerogpu): price LFM2.5-1.2B-Instruct
Adds zerogpu/LFM2.5-1.2B-Instruct to the cost map with the catalog prices and context window. It
gets no capability flags because the live API ignores tools, response_format and
max_completion_tokens for this model, so LiteLLM drops tool params for it instead of sending them.
LFM2.5-1.2B-Thinking stays out of the cost map
* fix(zerogpu): reject transcription requests before calling the upstream API
ZeroGPU only serves chat completions, so drop it from the OpenAI transcription transport set. A zerogpu transcription request now fails locally instead of hitting an /audio/transcriptions endpoint that does not exist
* test(zerogpu): cover the responses and messages bridges
Both endpoints reach ZeroGPU through chat completions, so assert the request each one sends upstream and the reply it returns
* refactor(zerogpu): move zerogpu out of the provider list tails
New upstream providers append to the end of openai_compatible_endpoints, openai_compatible_providers and providers.json, so an entry at the tail conflicts with every one of them. Placing zerogpu next to long-stable entries keeps those merges clean. No behavior change
* fix(zerogpu): price cached input for qwen3-30b-a3b-fp8
The ZeroGPU model catalog (https://docs.zerogpu.ai/docs/model-catalog, checked 2026-10-09) lists cached input for qwen3-30b-a3b-fp8 at $0.05 per 1M tokens. Without cache_read_input_token_cost, cached tokens were billed at the full $0.10 input rate
* revert(zerogpu): leave the audio transcription provider set unchanged
Per review, keep OPENAI_AUDIO_TRANSCRIPTION_PROVIDERS as upstream defines it. This PR only covers chat completions, and ZeroGPU now serves /audio/transcriptions anyway, so the local refusal and its test are removed
* fix(zerogpu): add max_tokens so trim_messages can size ZeroGPU history
trim_messages reads max_tokens as the fallback for max_input_tokens and evaluates it eagerly, so without it the lookup raised, got swallowed, and the full history came back untrimmed. max_tokens matches each model's maxTokens in ZeroGPU's models API (https://api-dashboard.zerogpu.ai/api/models, checked 2026-10-10)
* test(zerogpu): record mocked requests on a recorder object
The mock handler appended to a list the caller passed in, which AGENTS.md
forbids. A recorder now owns the captured requests as a tuple
* fix(zerogpu): cap max_tokens at each model's output limit
Each cap is the max_output_tokens ZeroGPU stores for the model in its production models table (read 2026-10-20), the same column its /v1/models serves as max_length. That gives 65536 for deepseek-v4.1-flash, 32768 for gpt-4.1-mini and gpt-oss-120b, 128000 for gpt-5.4-nano and gpt-5.6-luna, 8192 for LFM2.5-1.2B-Instruct and llama-3.1-8b-instruct-fast, and 16384 for qwen3-30b-a3b-fp8. glm-5.3-flash is stored as 32786, which reads as a typo for 32768, so that row uses 32768
trim_messages sizes history from max_input_tokens, so lowering max_tokens leaves it unaffected
* fix(zerogpu): flag prompt caching on glm-5.3-flash
Repeating the same ~4.3K-token prompt against ZeroGPU on 2026-10-20 returned cached_tokens of 4320 on some glm-5.3-flash calls. gpt-oss-120b and llama-3.1-8b-instruct-fast returned 0 on all nine calls, and qwen3-30b-a3b-fp8 never returned cached_tokens, so they stay unflagged. Their cached prices stay as ZeroGPU publishes them, so any cache hit is still billed at the right rate
* fix(zerogpu): set gpt-5.4-nano context window to 272000
ZeroGPU accepts a 271001-token gpt-5.4-nano prompt and rejects 273000 tokens as over the context window (checked 2026-10-21), so max_input_tokens is 272000 rather than the 400000 its catalog lists
* fix(zerogpu): list zerogpu models for zerogpu/* wildcards
zerogpu had no models_by_provider entry, so get_provider_models("zerogpu") returned None and a zerogpu/* wildcard listed no models. zerogpu_models is now filled from the cost map rows like openinfer_models and registered in models_by_provider
* fix(zerogpu): flag gpt-5.4-nano as a reasoning model
ZeroGPU honors reasoning_effort on gpt-5.4-nano (checked 2026-10-21). Without it the model spends 0 reasoning tokens, and with low or high it spends 32 or more. LiteLLM only treated it as a reasoning model by falling back to OpenAI's bare gpt-5.4-nano row, so get_model_info and /model/info reported supports_reasoning as null. The row now carries the flag itself
* fix(zerogpu): flag deepseek, glm, gpt-oss and qwen3 as reasoning models
ZeroGPU's /v1/models marks these four as reasoning models and accepts reasoning_effort on all of them (checked 2026-10-21), but LiteLLM rejected the param with UnsupportedParamsError because their rows had no supports_reasoning flag. glm reports reasoning_tokens under reasoning_effort, and gpt-oss and qwen3 spend hidden reasoning tokens on every reply. The gpt-oss test case now expects reasoning_effort to reach ZeroGPU
* fix(zerogpu): set gpt-oss-120b context window to 128000
ZeroGPU routes gpt-oss-120b across several upstream backends, and at least one of them returns 413 past a 128000-token window, so max_input_tokens uses that as the safe floor. The backends reached on 2026-10-21 accepted up to about 131K prompt tokens, so this only gives up a little headroom on the larger ones
* feat(zerogpu): call ZeroGPU's Responses API natively
ZeroGPU serves /v1/responses for all nine catalog models, so LiteLLM no longer bridges Responses requests through chat completions. Native calls return ZeroGPU's reasoning output items for glm, gpt-oss and qwen3, which the bridge dropped. Checked live on 2026-10-22 through litellm.responses for every model, non-streaming and streaming, plus a gpt-oss tool call. LFM2.5-1.2B-Instruct streams correctly now that ZeroGPU's text_generation lane emits Responses SSE
* fix(zerogpu): stop inferring the zerogpu provider from api_base alone
Listing https://api.zerogpu.ai/v1 in openai_compatible_endpoints rerouted bare model names on that api_base from openai to zerogpu. A bare gpt-5.6-luna call with verbosity used to return 200 and started raising UnsupportedParamsError. Drop the entry and its api_base-alone test, so zerogpu is only selected through the zerogpu/ prefix
* fix(zerogpu): price gpt-6-luna, llama-3.2-3b, LFM2.5-1.2B-Thinking and qwen3.6
A zerogpu/* wildcard served these four chat models with no cost-map row, so their calls logged $0 spend. Add rows at ZeroGPU's catalog prices from https://api-dashboard.zerogpu.ai/api/models, with context windows and output caps from ZeroGPU's models table. LFM2.5-1.2B-Thinking has no cached price, like LFM2.5-1.2B-Instruct. gpt-6-luna is flagged supports_reasoning because LiteLLM already routes gpt-6 names as reasoning models. Checked live on 2026-10-28, all four now report nonzero response_cost
* fix(zerogpu): drop the retired max_tokens field from the zerogpu rows
Upstream #48471 retired max_tokens from the cost map in favor of max_input_tokens and max_output_tokens, and test_no_row_carries_the_retired_max_tokens_field failed on all 13 zerogpu rows. Every row already carries the same value in max_output_tokens, so nothing is lost. The zerogpu limits test now checks that each row's output limit fits its context window
* chore(zerogpu): regenerate provider endpoint support for zerogpu
Upstream #48619 generates provider_endpoints_support.json from the cost map and checks it in CI, and the hand-written zerogpu entry no longer matched. Regenerated with scripts/generate_provider_endpoints_support.py. Only the zerogpu entry changed, and it now matches what the generator emits for other JSON-registry providers such as openinfer
* fix(zerogpu): flag tools, json_schema and prompt caching on gpt-6-luna
ZeroGPU serves function calling, tool_choice, json_schema response_format and prompt caching on gpt-6-luna, so set supports_function_calling, supports_tool_choice, supports_response_schema and supports_prompt_caching on its row in both cost maps. Checked live on 2026-10-28, a weather tool call came back with the right arguments, json_schema output validated, and repeated prompts reported 3260 of 3263 tokens cached
* fix(zerogpu): set qwen3.6-35b-a3b context to 262144 and flag its tools and reasoning
ZeroGPU accepts up to 262,144 input tokens on qwen3.6-35b-a3b and serves tools, tool_choice, json_schema response_format and reasoning_effort on it. Raise max_input_tokens from 16384 and set supports_function_calling, supports_tool_choice, supports_response_schema and supports_reasoning in both cost maps. Before this, LiteLLM rejected tools and reasoning_effort for the model. Checked live on 2026-10-28, a 209,905-token prompt succeeded, an oversized one was refused with a 262144-token limit, and tool calls, json_schema output and reasoning_effort all worked
* fix(zerogpu): flag vision on gpt-6-luna and qwen3.6-35b-a3b
ZeroGPU reads image inputs on gpt-6-luna and qwen3.6-35b-a3b, so set supports_vision on both rows in both cost maps. Checked live on 2026-10-28, both models named the animal in an image_url photo, and the image showed up in their prompt token counts
* feat: add aiand provider
* feat: expose aiand provider in admin UI
* feat(aiand): add ai& model pricing to the cost map
* feat(aiand): list ai& models for aiand/* wildcards and mark /v1/messages supported
* feat(aiand): send Anthropic /v1/messages to ai& natively
* fix(aiand): drop the retired max_tokens field and regenerate endpoint support
---------
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
* refactor(anthropic): rename llms/anthropic/pass_through to llms/anthropic/messages
Move the unified /v1/messages code out of a directory called pass_through, since it is not the
passthrough feature. The inner messages sub-package becomes native, and the Vertex AI partner
model experimental_pass_through package becomes messages. The old packages stay as import shims
that warn once and alias every submodule in sys.modules, so old deep imports still resolve
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* refactor(anthropic): keep Final variables out of loops in the rename shim
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(anthropic): match the warning text in the rename shim test
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* refactor(anthropic): resolve old pass_through imports lazily through a meta path finder
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* refactor(anthropic): mark the alias loader spec restore for the type discipline gate
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* style(anthropic): format the renamed package shim
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* style(anthropic): keep the rebind-ok marker on the spec restore line
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* refactor(anthropic): carry the new package search path on alias specs
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(anthropic): declare the shim test locals Final
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* feat(refuseless): add provider and model pricing
* feat(refuseless): expose provider in Add Model catalog
* test(refuseless): verify model input trimming after upstream fix
* fix(refuseless): exclude unavailable free model from catalogs
* feat(abliteration): add model pricing and capabilities
* feat(abliteration): expose provider in Add Model catalog
* chore(abliteration): keep provider entries independent of other additions
* fix(abliteration): correct reasoning capabilities and verify input trimming
* refactor(cost-map): drop the legacy max_tokens field in favor of max_input_tokens and max_output_tokens
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test: drop assertions that pinned the retired cost-map max_tokens field
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* feat(openinfer): add OpenInfer as OpenAI-compatible provider
Register OpenInfer in the JSON provider registry with catalog models,
proxy create fields, and unit coverage so requests route via openinfer/
Co-authored-by: Cursor <cursoragent@cursor.com>
* fix(openinfer): assert catalog invariants instead of vendor rates
Pinning per-token prices made CI fail when the vendor list changed. Keep
the backup agreement and positivity checks our code owns.
Co-authored-by: Cursor <cursoragent@cursor.com>
* fix(openinfer): add openinfer to the Rust OpenAI-compatible provider mirror
* fix(openinfer): align catalog with OpenInfer's live /v1/models
Replace the four catalog rows with the two models OpenInfer serves and
publishes a price for (@oi/GLM-5.3-Flash, @oi/Qwen3.5-27B), using its
listed per-token rates and context windows, and price-check them in tests
* fix(openinfer): use OpenInfer's native Responses and Messages APIs, allow tools on unpriced models, price Qwen3.5-9B
Declare /v1/responses and /v1/messages for openinfer, since OpenInfer serves both natively. Let JSON providers declare provider-wide capabilities in providers.json, read through get_provider_info, so tool calls reach OpenInfer for models without a cost map row. Send max_completion_tokens as is, set output caps to each model's context window, mark the three rows as reasoning models, add Qwen3.5-9B at its listed rate, and wire openinfer into models_by_provider so the openinfer/* wildcard lists its models.
* fix(openinfer): let a model's own capability row win over provider defaults in get_model_info
Provider defaults now only fill capability fields the cost-map row leaves unset, so a row or deployment that turns function calling off stays off. Also marks openinfer_models as Final.
---------
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
* fix(anthropic): add the dangerous-tool-use beta when a /v1/messages request carries safeguards
* test(azure_ai): pin the outgoing Foundry request keeps the dangerous-tool-use beta next to safeguards
* test(azure_ai): inject the Foundry client and scope the beta cache reset to the test
* test(azure_ai): own and close the mock transport client, drop the docstring entry
* test(integration): audit the safeguards beta derivation across anthropic-compatible providers
* fix(github_copilot): keep safeguards from deriving the dangerous-tool-use beta
* test(integration): kill the chaos victim worker through the harness helper
* fix(anthropic): derive the dangerous-tool-use beta only where the beta map supports it and honor additional_drop_params
The safeguards beta is now added only for providers whose beta map column accepts it, so github_copilot, Eden AI and the OpenAI-compatible JSON providers keep their base wire. additional_drop_params now runs before the provider headers are built, so a dropped param no longer drives its derived beta.
* fix(anthropic): load the beta map off the event loop and pin a dropped context_management
The first /v1/messages request that needs the remote beta map now loads it in a worker thread,
so a slow fetch no longer stalls every other request on the worker. An integration cell pins that a
deployment dropping context_management sends neither the field nor its derived beta.
* fix(anthropic): load the beta map only for safeguards requests, once per worker
* fix(anthropic): gate safeguards on the bundled beta map and load the remote map on its own thread
---------
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
* fix(openai_like): surface provider response headers on non-streaming chat completions
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(openai_like): mark the header test locals Final
* fix(proxy): leave out response header values HTTP cannot carry
Non-streaming OpenAI-like chat now forwards every provider header, so on the
httpx transport a provider header value outside latin-1 made the proxy answer
500 after the provider had served the call. get_custom_headers now leaves such
a value out and keeps the other headers, which also ends the same 500 on
streaming
Adds integration cells for the provider request id on non-streaming chat and
the Responses bridge, LiteLLM headers an upstream cannot overwrite, spend
parity under an upstream cost header, the latin-1 case, and the logging
callback payload
* fix(cost): ignore an upstream response cost header that is negative or not a number
* test(integration): cover invalid upstream cost headers and non-latin-1 headers on every endpoint
* fix(prometheus): keep deployment metrics when a provider rate-limit header is not a number
* test(prometheus): type the rate-limit header test and drop its clock reads
* test(integration): cover deployment metrics under an unparseable provider rate-limit header
---------
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
* fix(bedrock): keep reasoning, thinking blocks and the json_mode finish_reason on fake streams
A Bedrock Converse model without native structured output fakes the stream
when response_format carries a json_schema. The fake stream transformed the
response without json_mode and rebuilt a chunk from text, tool_use and token
counts only, so it dropped reasoning_content, the signed thinking_blocks and
the reasoning token count, and ended on finish_reason tool_calls.
The three fake-stream sites now pass json_mode to _transform_response and
stream the resulting ModelResponse through the shared
convert_model_response_to_streaming converter, which now numbers each
streamed thinking block with its index the way real streams do
* fix(bedrock): parse text tool calls on Converse fake streams like the non-stream answer
* fix(responses): stream the answer text when one chat chunk carries reasoning and the answer
* fix(streaming): keep every field on the OpenAI and openai_like fake streams and split multi-block thinking
Azure o-series and openai_like fake streams now use the shared MockResponseIterator, so extra tool calls,
reasoning and cached and reasoning tokens survive; the Bedrock-local iterator is deleted.
The /v1/responses bridge also splits a chunk whose thinking block is signed or redacted with no reasoning
text, and the sync iterator closes the reasoning item before the answer opens, like the async one.
The /v1/messages bridge gives each thinking block in a chunk its own content block, starts a new block when
the thinking block index changes, and streams redacted thinking as redacted_thinking.
Snowflake replays drop the stream index from thinking blocks.
* fix(anthropic): stream each thinking block's text once on /v1/messages
An unsigned thinking chunk now opens its content block with empty text, since its thinking_delta carries the text. A multi-block chunk keeps each piece's thinking block, so an unsigned block followed by a signed one keeps its own text and index.
* docs(anthropic): describe which thinking block start carries its text
* fix(streaming): close merged reasoning before the answer when one chunk carries both
* test(streaming): cover fake-stream reasoning across the proxy bridges and type the Snowflake replay test
* feat(providers): add Inference APIs as OpenAI-compatible provider
Registers https://api.inferenceapis.com/v1 as a JSON-configured provider (chat completions and Responses API).
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* fix(providers): list Inference APIs in Add Model and the provider docs index, add request-level tests
- provider_create_fields.json entry so the Add Model dropdown test passes
- provider_endpoints_support.json (+ backup) entry so the provider docs check passes
- tests now capture the outgoing HTTP request (URL, auth header, body) instead of only inspecting registrations
- drop a redundant comment
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* test(providers): inject a recording HTTP session instead of patching httpx.Client
Request-level tests now set litellm.client_session to an httpx.Client with a MockTransport, so nothing is
monkeypatched on a class.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011qUBY2P8qTTuJCUeMzTCD4
* chore(providers): drop redundant comment on the inferenceapis entry
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011qUBY2P8qTTuJCUeMzTCD4
* feat(inferenceapis): add cost map rows for the 12 chat models and test the responses route
Prices and context windows come from https://inferenceapis.com/pricing and GET /v1/models.
Moves the provider test to tests/unit and adds the interactions flag the endpoint matrix now carries.
* feat(inferenceapis): price the vendor's chat model aliases and mark /v1/messages supported
* fix(inferenceapis): serve /v1/messages natively and keep audio routes off the provider
* fix(json providers): send provider-specific params in extra_body and report context overflow as ContextWindowExceededError
* fix(exception mapping): match the Inference APIs overflow phrase instead of any context_length_exceeded substring
* fix(exception mapping): map context overflow to ContextWindowExceededError for Inference APIs only
Providers without their own mapper keep reporting an overflow as BadRequestError, since the router raises a ContextWindowExceededError instead of trying generic fallbacks when context_window_fallbacks has no entry for the group. Adds the integration cells for the provider, its fallbacks, and the extra_body and overflow families.
* test(inferenceapis): hold upstream replies across the worker kill in the chaos cell
A reply the killed worker sent before SIGKILL can lose its queued spend row, which the
product never promised to keep. Holding the upstream until after the kill means only the
surviving worker answers, so every 200 must log exactly once.
* test(inferenceapis): make the chaos cell kill the quieter worker and always release held replies
The survivor now must hold at least one in-flight request, so the burst check cannot pass as 0 == 0, and the held vendor is released on exit so a failed assertion cannot stall the wire server teardown
---------
Co-authored-by: TaylorHawkes <857858+TaylorHawkes@users.noreply.github.com>
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
* feat(providers): add CoralBricks as a native provider
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(coralbricks): harden cost map consistency tests
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(coralbricks): type cost helpers, read cost maps from disk, cover streaming spend
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(coralbricks): drop Final on loop-rebound names
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(coralbricks): add the provider audit cells, the Add Model browser cell and the README row
* test(coralbricks): derive UI spend from model info, type browser rows, always release held upstream
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(coralbricks): release held upstream before the wire server joins its threads
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(coralbricks): number forwarded chats with itertools.count instead of a mutable list
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(coralbricks): drop retired glm-5.3-flash-fp4 and test against deepseek-v4.1-flash-fast-fp4
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(coralbricks): pin retries, await the respawned worker, read the UI rate from the catalog
The retry cells now set num_retries on the deployment, since the CI rig posts
router_settings.num_retries 0 after readiness and the router default of 2 no
longer applies there. The SIGKILL chaos cell waits for the respawned worker's
startup before the owned proxy is stopped and derives its timeout from the
harness budget. The Add Model spec waits for the catalog the form loads, reads
the expected rate from it, and finds the deployment through /v1/model/info,
which does not paginate
* fix(coralbricks): price the current model names and the new DeepSeek input rate
CoralBricks renamed glm-5.3-fp4 to glm-5.3-fast and deepseek-v4.1-flash-fast-fp4
to deepseek-v4.1-flash-fast (the fp4 names stay as deprecated aliases), and cut
DeepSeek V4.1 Flash input from $0.30 to $0.01 per 1M on 2026-10-08. Rates read
from https://www.coralbricks.ai/api/public/models on 2026-10-10; a live cache
write billed 6014 tokens at 9e-08 each, so the cache-write rate stays input
plus cacheWritePerM.
The test config builders now build new dicts instead of editing the parsed
YAML, and the cost-map test locals carry Final.
* fix(coralbricks): keep pricing for the deprecated fp4 slugs CoralBricks still serves
* fix(coralbricks): list the catalog rows for wildcard deployments and derive the test's input limit from the row
* fix(coralbricks): price the deprecated glm-5.3 and deepseek-v4.1-flash slugs
---------
Co-authored-by: Mubashir Osmani <mubashir@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
* test: migrate local_testing SDK and provider tests out of build_and_test
* test: cover sagemaker region, hf template and nova paths end to end
Region rows now go through litellm.completion on the sagemaker/ handler that
the legacy tests exercised, the prompt template test mocks the Hugging Face
tokenizer config instead of switching to custom_prompt_dict, streaming tests
assert role, finish reason and body, and Nova single turn and streaming run
through litellm.completion with a recorded request body
* test: tighten migrated integration tests for local OpenAI routes, timeouts and air-gapped boot
Split the catch-all providers migration file by behavior. The fake endpoint
route tests now hit the owned upstream directly, as the legacy harness tests
did. LM Studio asserts the outbound key, path and stripped model. The embedding
timeout test asserts the proxy maps a deployment timeout to a 408 and forwards
the read timeout to the provider client, instead of timing out the test client
The Azure streaming tool-call test also asserts the second identical stream is
served from the Redis cache with the same reassembled tool call
The Docker network test always skipped in CI because the security job has no
litellm/litellm image and no egress to pull one, and it tested a published image
rather than the checkout. It now boots the checkout proxy with every outbound
HTTP connection refused and asserts readiness, the bundled cost map fallback and
that the cost map host is the only egress attempted
* test: tighten the migrated e2e rows for local_testing SDK and provider tests
Drop the duplicated Anthropic covers marker and the Anthropic cell wrongly claimed by
the Bedrock and Vertex cache tests. Keep the file-wide Anthropic model as it is on main.
Send the 2023-06-01 version header to Anthropic through the deployment's extra_headers
so the header actually reaches the provider, and mark the creation-token case 1h then
5m like the legacy test did
Reuse the Bedrock file's own registration for the pydantic response test, swap the
hand-built PNGs for the shared cat.jpg fixture, assert the SageMaker Nova image answer
names the cat and that top_logprobs=3 is honored, and register the SageMaker Nova opt-in
markers the same way as the other opt-in markers
* test: restore lost assertions in migrated test_main tests
Restore the role alternation check in test_completion_anthropic_hanging, the
embedding and streaming response-header checks, the logit_bias drop check and
the gemini system and safety settings checks. Add an azure 429 retry-after
test for test_exception_with_headers, trigger the router fallback with
mock_timeout like the legacy test, and stop the mock timeout tests sleeping 3s
* test: restore lost legacy assertions in migrated vertex ai unit tests
* test: tighten migrated cost, streaming and embedding tests and scope the hosted mock guard
* test: strengthen migrated anthropic, azure, bedrock and cohere provider tests
Assert full request bodies and usage instead of presence checks, restore the
dummy-tool and custom_llm_provider assertions the legacy tests made, add the
async and per-model variants the legacy parametrization covered, split
streamed tool arguments across deltas, and move the Azure chat streaming
tests to the azure.py mirror file
* test: tighten migrated llms provider tests and cover legacy intents lost in the move
Strengthen the migrated fireworks, gemini, mistral, together, openai text completion, openrouter, perplexity and openai error tests so they assert what the legacy tests proved (cost, provider tag, usage-only-on-last-chunk, function_call rebuild, reasoning param, sync/async/stream error fields). Replace the huggingface and jina tests that doubled litellm code with respx end-to-end tests. Add the tests the pre-existing mapped tests never covered: cohere 400 mapping, token-array text completion, hosted_vllm/llamafile/lm_studio env api base embeddings, watsonx embedding auth, ollama ssl_verify client and ollama vision prompt
* test: cover residual legacy paths for anthropic tool streams, watsonx async stream, ollama http images and caller async embedding clients
* test: restore single request assertions in the deep_infra and azure_ai embedding tests
* test: drop a no-op streaming cost flag from the openai streaming cost test
* test: cover residual legacy-test lines in caching, main, utils, replicate, vertex and bedrock invoke
* test: cover streamed audio assembly, choice membership, untranslated responses events and the stream log guard
* test: pin cost rates in unit tests, use sdk for cohere e2e, drop vertex llama e2e
* test: send the vertex maps e2e request through the openai sdk
* test: drop the local_testing jobs that select no tests after the migrations
* test: wait for the streamed cache entry and keep custom Mistral rates test-local
* test: keep untouched Mistral test lines as they were
---------
Co-authored-by: yuneng <yuneng@berri.ai>
* test: delete unconditionally skipped legacy tests
* test: move whole-unit legacy test files into tests/unit
* test: keep moved legacy tests free of import-time global state
* test: keep the module-level invocation scan pointed at tests/local_testing
* ci: drop the agent_testing CircleCI job emptied by the move
* test: fix moved-test isolation and router coverage
* test: add Tinyfish search package marker
* test: isolate moved tests from logger state leaks
* test: isolate Helicone logging fixture state
* test: isolate Vertex pass-through credentials between moved tests
* test: cancel S3 periodic flush tasks started by moved tests
* ci: restore CircleCI assistant test selection after move
* test: prevent Datadog datetime import shadowing
* ci: drop the litellm_assistants_api_testing CircleCI job emptied by the move
* test: deduplicate imports in rebased unit tests
* test: remove duplicate passthrough router patch import
* test: remove moved legacy source files after rebase
* test: align moved tests with rebased main
* test: carry main's legacy-file edits into moved destinations
* test: make the moved cost map fallback tests assert the fetch and the backup
The four fallback cases only checked the result was non-empty, so they still
passed with integrity validation disabled. They now inject a mock client, assert
one fetch happened, and assert the result is exactly the local backup with the
fallback reason recorded.
---------
Co-authored-by: yuneng <yuneng@berri.ai>
* Add Reka as an OpenAI-compatible provider
* Add Reka supported endpoints
* fix: remove stray fragment after reka entry in provider_endpoints_support.json
Co-authored-by: Cursor <cursoragent@cursor.com>
* test(reka): register provider identity and cover routing, credentials, and bridged endpoints
Co-authored-by: Cursor <cursoragent@cursor.com>
---------
Co-authored-by: Cursor <cursoragent@cursor.com>
* feat(providers): add Prism provider
* fix(providers): complete Prism registration
* feat(providers): expose Prism responses and messages
* feat(providers): add DeepSeek V4.1 Flash to Prism
* test(providers): exercise Prism endpoint requests
* fix(providers): align Prism pricing and limits with the live catalog
deepseek-v4.1-flash bills 0.17/0.63 USD per 1M input/output tokens and takes image input;
deepseek-v4-flash bills 0.17/0.21 and caps output at 384000 tokens, per GET /v1/models
* test(prism): assert cost-map invariants instead of pinning catalog facts
* test(prism): derive the asserted model list from the cost map instead of pinning it
* test(prism): capture requests through respx instead of appending to a list and swapping the client transport
---------
Co-authored-by: rajitkhanna <rajitskhanna@gmail.com>
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
Co-authored-by: ryan <ryan@berri.ai>
* ci: run the unit_selection.sh shard files on every event instead of only fork pull requests
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* ci: rename fork-flag to unit-flag now that it applies on every event
* test: move tests/test_litellm root and small trees into tests/unit
Pure renames, no content changes. Follow-up commits in this PR fix
references, merge the three files that already existed in tests/unit,
keep live-provider tests in tests/test_litellm and wire CI.
* test: carry tests/test_litellm conftest isolation into tests/unit
Callback lists, routing fallbacks, cached HTTP clients, logger state, AWS,
proxy-URL and keychain env, and session-end client cleanup now reset for
unit tests too. The environment isolation owns its MonkeyPatch so a test's
own monkeypatch is undone before the model-cost teardown runs.
* test: merge, split and prune the moved root and small-tree tests
Merge batches/test_batch_utils.py and the chat_completions and messages
dispatch tests into the files that already existed in tests/unit. Keep
the live Gemini interactions tests, the async image-fetch format test and
the OpenAI embedding scorer test in tests/test_litellm since they need
real network or keys. Put test_router.py under tests/unit/test_router so
the existing package no longer shadows it. Delete eight tests the audit
found superseded by stronger ones kept in this move.
* ci: run the moved root and small-tree tests under their legacy flags
Add the misc and responses-caching-types flags to unit_selection.sh and
CircleCI, extend enterprise-routing and mcp-integration, and point the
legacy GHA shards, Makefile, redis-compat workflow, merge smoke manifest
and change classifier at the new paths.
* test: make the new tests/unit directories packages
tests/unit/test_package_layout.py requires every directory to carry an
__init__.py, and without one the moved and retained
test_litellm_responses_bridge.py modules collide on import.
* test: scope the unit socket block to tests/unit in shared sessions
The GHA shards collect the legacy test-path and the unit selection in one
pytest session. The unit conftest's loopback-only block leaked into legacy
modules that reach the network at import. The legacy conftest now lifts the
restriction at collect and setup time, and the unit conftest re-applies it
when collecting its own modules.
* test: move tests/test_litellm/llms into tests/unit/llms
Rename-only. Moves the provider tests and the fine-tuning fixtures they
load, mirroring the old paths. Follow-up commits merge, split and wire them.
* test: merge, split and prune the moved llms tests
Merges the Databricks chat transformation tests into the existing unit
file, keeps the tests that need real keys or the network in
tests/test_litellm, deletes the audited tests a stronger unit test
already covers, and points imports at tests.unit.llms.
* ci: run the moved llms tests under their legacy flags
The Vertex AI and All Other Providers shards keep their legacy test-path
for the retained files and add the llm-vertex-ai and llm-other-providers
unit selections. CircleCI gets matching unit jobs.
* test: make the tests/unit/llms directories packages
Adds __init__.py to the moved dirs and drops the legacy ones whose
directories no longer hold tests.
* test: drop script runners and path hacks the llms split left dangling
The __main__ runners in the split openai_like files and the Databricks e2e
runner called tests that now live in the other half of the split or were
deleted. The retained legacy halves also no longer need sys.path edits.
* test: give the shard-script tests their own GITHUB_OUTPUT
They only passed where the runner set it. The CircleCI unit job's env
allowlist drops it, so the script's redirect failed there.
* test: point the router and module-deletion checks at tests/unit
router_code_coverage and code_qa_check_tests only searched tests/test_litellm,
so the moved router tests no longer counted. The two silent-experiment tests
the audit deleted were the only direct callers of those methods; they are
replaced with tests that assert the forwarded shadow request and the
recursion guard.
* test: keep the Databricks manual e2e runner and fix the SageMaker Nova run path
The Databricks e2e file is a manual script whose main() calls the tests
that were pruned, so pruning them broke the documented run. It is back to
its main version. The SageMaker Nova docstring now points at the file's
real location in tests/local_testing.
* test: keep the job's UNIT_FLAG out of the shard-script tests
---------
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
