mirror of
https://github.com/BerriAI/litellm.git
synced 2026-11-03 07:23:33 +00:00
14 commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
Vineeth Sai Varikuntla
|
a7b66dd8b9
|
fix(openai): stop stripping cache_control from the caller's own messages and tools (#35602)
* fix(openai): stop stripping cache_control from the caller's messages Copy before removing the field, so the caller's own message objects are left alone. * fix(fireworks_ai): strip cache_control on a copy so a fallback keeps the caller's hints * fix: keep the caller's request intact on more providers and trim extra Claude cache hints under drop_params * fix: drop only the oldest extra cache hints inside one message when trimming to the Claude limit * fix: keep replayed session turns yielding their hints as whole turns while a request's own message keeps its newest hints * fix(cache_control): count only hints that reach Claude and cap hints on OpenAI-compatible Claude routes under drop_params * fix(cache_control): keep the capped Claude messages a list in OpenAI-compatible request bodies * fix(caching): cap Claude hints for string drop_params, a Claude base_model, and Converse tool messages The 4-hint cap now reads drop_params through drops_unsupported_params on chat, Converse, and Anthropic routes, names Claude by base_model on OpenAI-compatible routes, and counts a Converse tool message's hints as the one cache point it sends. * fix(caching): trim Claude cache hints in one wire-order pass and reserve a top-level cache_control on OpenAI-compatible routes Dropping the oldest hint one at a time re-walked the whole request per hint, which took seconds on long tool histories. The cap now drops each message's oldest hints in the order the provider counts them, so a string assistant turn loses its text hint before its tool call hints. OpenAI-compatible Claude routes also keep room for a request-level or extra_body cache_control under drop_params. * test: drop narrating comments and stacked generator clauses from cache_control tests * fix(vertex_ai): put every system message in the Gemini context cache so generateContent never sends it beside cachedContent --------- Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com> |
||
|
tessellark
|
5d5e3685c1
|
fix(logging): warn when drop_params drops a requested reasoning level on the remaining paths (#48933)
* fix(logging): warn when drop_params drops a requested reasoning level on the remaining paths
* fix(logging): keep guardrail and token-count reads of a request from logging drop warnings
Only the real Anthropic-messages adapter request reports a dropped thinking budget. Guardrail,
shadow-eval and token-count translations of the same request now drop it silently, so one request
logs one warning. The 422 retry also no longer hashes each loc entry, so a list inside loc cannot
raise TypeError.
* fix(bedrock): keep the DeepSeek R1 Converse reasoning_effort drop at debug level
Only allowed_openai_params reaches that branch, and it puts reasoning_effort back after the provider mapping, so a warning there claimed a drop that never happened
* fix(logging): warn on Gemini non-integer budget drops and keep the auto-router baseline silent
Gemini 2.x now logs the drop of a non-integer thinking budget. The auto-router
savings baseline builds a body it never sends, so it no longer warns. Logged
values are capped with reprlib, Azure AI marks only its drop_params retry as
gated, and Bedrock Chat Completions names the model without its route prefix.
* fix(logging): warn on the Gemini 3, Magistral, Ollama, Fireworks and DeepSeek ARN reasoning drops
Gemini 3 and later now log a dropped thinking budget, Magistral logs the
level it swaps for a reasoning prompt, Ollama logs a level it maps to
think off, Fireworks logs chat_template_kwargs effort keys dropped for a
model the map does not list as reasoning, and DeepSeek on Bedrock logs a
drop when the model is addressed by an ARN that advertises the param.
* fix(logging): warn on the remaining Ollama, DeepSeek and Tencent reasoning drops
The ollama/ completion route now shares the chat route's think mapping, so
a level it turns off logs the same warning. DeepSeek warns when it drops
thinking.budget_tokens, and Tencent warns when an adaptive-only model gets a
non-integer budget. Magistral no longer warns on low, medium, high or
enabled thinking, the values its reasoning prompt stands in for. The Gemini 3
budget warning no longer claims the model uses its default level.
* fix(reasoning): warn when Claude drops a non-string reasoning_effort
Claude chat and native /v1/messages dropped a reasoning_effort like 5 or
{"effort": 5} with no log line. Both now log one warn-only WARNING. A
Magistral thinking.type sent as a list no longer raises TypeError.
---------
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
|
||
|
Mateo Wang
|
73aaf9e3da
|
fix(mistral): forward prompt_cache_key to the Mistral API (#47432)
* fix(mistral): forward prompt_cache_key to the Mistral API * fix(mistral): send prompt_cache_key only to Mistral's API and accept reasoning_effort on devstral and code-agent models * fix(mistral): keep the upstream api_base out of the prompt_cache_key rejection message * fix(mistral): honor allowed_openai_params for prompt_cache_key and gate codestral async and env hosts * test(mistral): annotate the new adapter prompt_cache_key tests * fix(mistral): drop or reject a prompt_cache_key the Mistral API cannot take A non-ASCII prompt_cache_key gets a 500 from api.mistral.ai, and a non-string one a 422, so both now drop under drop_params and answer 400 without it, like any value the provider cannot take. The /v1/messages adapter derives no key from a non-ASCII Claude Code session for Mistral, and the 400 names the caller's provider, so a codestral deployment's error says codestral. * fix(mistral): send a reasoning_effort object as its level and keep UnsupportedParamsError on sync calls * fix(mistral): keep async codestral file uploads working by not rewriting the caller's file blocks |
||
|
rj-cairn
|
7abff3dd63
|
fix(mistral): replay assistant reasoning as a Mistral thinking chunk (#48778)
* fix(mistral): replay assistant reasoning as a Mistral thinking chunk When a client sends back an earlier Mistral assistant turn, its reasoning (reasoning_content, thinking_blocks, or Mistral's own thinking chunk in content) now reaches Mistral as a ThinkChunk inside content. The top-level LiteLLM fields stay stripped because Mistral rejects them with 422. * fix(mistral): replay reasoning only to Mistral models that accept it Mistral answers 400 to a thinking chunk on models without reasoning support, so a mid-chat model switch, a fallback, or Vertex AI Mistral broke once the replay was added. Gate the replay on the same check reasoning_effort uses and keep dropping the reasoning for the other models, as before. * fix(mistral): drop native thinking chunks for models without reasoning and mark the devstral, code agent, small and leanstral aliases as reasoning * fix(mistral): keep the leanstral rows as on main, since their reasoning support cannot be checked live * fix(mistral): send only the text items of a native thinking chunk, so a reference or image item no longer gets Mistral's 422 * fix(mistral): keep closed false only on the last chunk of a final prefix turn, so an earlier turn no longer gets Mistral's 400 * fix(mistral): drop closed false when merged text follows the thinking chunk, so a prefix turn with text first no longer gets Mistral's 400 --------- Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com> |
||
|
Mateo Wang
|
b51b9b7d43
|
fix(anthropic)!: answer 400 for a malformed tool_result text on /v1/messages and count_tokens (#47347)
* fix(anthropic): answer 400 for a malformed tool_result text on the Responses bridge and count_tokens A tool_result text block whose text is null, a number, or missing made /v1/messages and /v1/messages/count_tokens answer 500 on OpenAI and Azure models. Both routes now answer 400 naming the block, and drop the block when drop_params is on. * fix(anthropic): keep a missing tool_result text as empty text, bound the error, honor body drop_params on count_tokens * fix(anthropic): apply the tool_result text check on the chat/completions bridge and every count route The chat/completions bridge used for Azure, Gemini and other providers now answers the same 400 for a null or numeric tool_result text and drops the block under drop_params, so a model group that mixes OpenAI with another provider answers the same way on every deployment. Gemini and Vertex count_tokens apply the same check, count_tokens turns on drop_params for Claude Code and Codex the way /v1/messages does, and a tool_result without a tool_use_id is checked too * fix(proxy): answer 400 for a malformed tool_result text on count_tokens models without a count API Providers with no token counting API fall back to the local tokenizer, which counted a null or numeric tool_result text as fine while /v1/messages on the same model answers 400. The fallback now runs the same check, unless drop_params applies to the deployment or the request * fix(proxy): honor request drop_params on the local count fallback when no deployment resolves * test(integration): cover non-string tool_result text on both /v1/messages bridges and count routes * fix(anthropic): keep a dropped tool_result on Mistral and reject it before a guardrail rewrite Under drop_params a tool_result whose only text block was null or a number became an empty tool message, and Mistral's empty-message filter removed it, so Mistral answered 400 for a missing tool reply. The filter now only removes empty assistant messages, which is what Mistral rejects. A guardrail that rewrites structured_messages wrote the dropped text back as "", so a null or numeric text was silently blanked even without drop_params. The rewrite path now answers the same 400 as the bridges unless the request or global drop_params is set. * test(integration): cover the Mistral empty tool reply and the guardrail rewrite tool_result text check * fix(anthropic): keep a malformed tool_result text through a guardrail rewrite and keep Mistral's null-message drop A guardrail that rewrites structured_messages now writes the caller's null or numeric tool_result text back in place, so the route's bridge and the deployment's drop_params decide after routing, the same as with no guardrail. The rewrite path no longer answers its own 400 before routing. Mistral drops a null-content system or user message again, as main does, and only a tool reply is exempt from that filter so a dropped tool_result keeps its tool call. * test(mistral): expect empty system and user turns dropped and the empty tool reply kept * perf(anthropic): restore kept tool_result text parts in one linear pass * test(integration): cover the tool_result text restore across tool_results, the chat bridge, and a batch upload behind a rewriting guardrail * fix(anthropic): let guardrails scan malformed tool_result text and honor drop_params on native Anthropic A rewriting guardrail now sees a non-string tool_result text as its JSON text and the write-back restores the caller's value by the sanitized tool_use_id when the guardrail left it unchanged. Native Anthropic drops the bad block under drop_params on /v1/messages and count_tokens, and Mistral keeps an empty tool reply only when it answers a tool call, sent as empty text. * fix(anthropic): compare a kept tool_result text to its scanned form once and align the wire tests with drop_params * test(integration): cover native Anthropic drop_params resolution, guardrail scanning of numeric tool_result text, and the Mistral orphan tool message * fix(count_tokens): honor the router's default drop_params when counting a malformed tool_result text * test(count_tokens): send the tool_result count tests through a real request body instead of a mocked parser * test(count_tokens): send the native Claude count test through a real request body * fix(anthropic): keep each non-string tool_result text when tool_use_ids sanitize to the same id * fix(anthropic): keep non-ASCII characters in the guardrail's view of a non-string tool_result text * test(gemini): mock the count_tokens upstream with respx instead of patching the client factory * fix(mistral): keep an empty tool reply only for an open call, and forward non-object message items on the native drop An empty tool reply re-answering a call the conversation already moved past made Mistral reject the request, and a tool call id that is not a string crashed the filter. Native Anthropic under drop_params assumed every message item was an object. * fix(mistral): find answering tool replies in one pass instead of copying the open call set per reply * fix(mistral): ignore dropped empty placeholders when tracking open tool calls * fix(mistral): drop an empty tool reply when another reply to the same call carries the result * refactor(anthropic): share tool_result text helpers to shrink the diff * fix(anthropic): answer 400 for a non-string tool_result text on the OpenAI Responses bridge |
||
|
Mateo Wang
|
52fc37888b
|
fix(responses bridge): round-trip tool_search_call and tool_search_output items between turns (#46444)
* fix(responses bridge): keep defer_loading on function tools sent to the Responses API Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix(responses bridge): give the defer_loading casts a cast-ok reason Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * refactor(responses bridge): build defer_loading outside the tool loop so Final stays valid Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix(responses bridge): round-trip tool_search_call and tool_search_output items between turns Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * refactor(responses bridge): read replayed tool-search and namespace fields through typed helpers Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix(responses bridge): keep tool_search items on a turn with no message or tool call Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix(responses bridge): keep every reasoning item on a search-only turn Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix(responses bridge): leave content unset on a search-only turn so replay adds no empty message Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix(responses bridge): replay tool search items only with tool_search, count replayed tools in the allowlist, keep namespace off guardrails * fix(responses bridge): honor drop_params for defer_loading and route tool_search to Responses Under drop_params, defer_loading is dropped when no tool_search tool is sent, so the 400 it would cause upstream goes away as before. A non-bool value answers 400 without drop_params and is dropped with it. A null tool-level value falls back to the nested function value. A request carrying a tool_search tool now bridges to Responses on OpenAI and Azure endpoints even with reasoning_effort none, since chat completions rejects the tool_search type. * fix(responses bridge): replay tool search items in output order and count replayed tools in tool policy Store the ordered output prefix through the last tool search item, reasoning included, so a second reasoning item no longer makes OpenAI reject the replay. The shared chat-to-Responses converter now replays tool search items and namespace only when the bridge opts in, so Noma, the complexity router and guardrail payloads keep their shape. The tool policy guardrail and the allowlist count replayed tools only when the request can send them. * fix(responses bridge): forward defer_loading only beside a tool_search tool defer_loading only means something with tool_search, and backends without it reject the key (xAI answers 403). A true value with no tool_search tool now answers 400 naming drop_params and is dropped under drop_params; a false value there is a no-op and is dropped. * fix(tool_loop): keep tool search items and namespace between rounds * fix(responses bridge): check replayed tool search loads when a deployment adds tool_search * fix(chat): drop litellm response fields before strict OpenAI-compatible providers A chat reply from the Responses bridge carries provider_specific_fields on the message and on each tool call, plus reasoning_items. Groq, Fireworks, Mistral and Together reject unknown fields, so sending that reply back to them as history failed with a 400 or 422. Strip them in the shared helper and use it in those providers. * fix(caching): keep reasoning_items and tool search items on response cache hits A cached bridge response lost reasoning_items on a non-streaming hit and lost both reasoning_items and provider_specific_fields (tool_search_items) on a streamed hit, so the next turn could not replay the search or the reasoning around it * test(integration): cover tool search item replay on the chat to Responses bridge Proxy, SDK, authorization and chaos cells for returning and replaying tool_search_call and tool_search_output items and the tool call namespace * test(integration): add response cache twins for the tool search guardrail cells Each guardrail cell now has a twin where the answer is already in the Redis response cache: a key opted out of global guardrails caches a replay that loads a policy-blocked tool, the same replay without guardrails is cached before block mode sees it, rewrite mode is not served the unrewritten conversation's cached answer, and an allowed replay is served from the cache. * fix(responses bridge): forward defer_loading beside a client-executed tool_search The bridge now replays a client tool_search_call and the client's tool_search_output through provider_specific_fields.tool_search_items, so a deferred function beside a client tool_search keeps defer_loading like it does beside a hosted one * test(responses bridge): pin the client tool_search turn and its replay, streamed and not * fix(responses bridge): strip nested namespace for strict providers and keep streamed reasoning_items on text replies Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix(responses bridge): keep dropping defer_loading beside a client-executed tool_search Forwarding defer_loading beside a client tool_search made GPT stop with a search call the chat client has to answer by hand, so a plain question that used to get tool calls got none. Keep main's behavior of dropping it there. --------- Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> |
||
|
Mateo Wang
|
0ebec123fd
|
chore(typing): clear 670 basedpyright Any errors by validating provider payloads at the boundary (#45388)
* chore(typing): validate provider payloads at the boundary to clear 464 basedpyright Any errors Replaces untyped reads of decoded JSON in 48 provider, guardrail and proxy modules with one pydantic validation at the point the payload enters the module, so every value read afterwards carries a real type. reportAny 6111 -> 5663 and reportExplicitAny 1416 -> 1400. No basedpyright, ruff-strict or type-discipline rule is higher repo-wide or in any single file, and no cast, ignore or suppression marker was added. * chore(typing): replace 47 Any annotations that every caller already satisfies with object Each annotation here is a parameter or return type whose value is only stored, compared, logged or forwarded, so object describes it exactly and no function body changes. A site was kept only if no basedpyright, ruff-strict or type-discipline rule rose in any file that references it. reportExplicitAny 1400 -> 1353 and reportAny 5663 -> 5655. * chore(typing): type upstream responses and loose annotations across providers, proxy and integrations Second pass of the Any reduction. Decoded upstream JSON is validated once at the provider boundary and read through typed views afterwards, annotations that every caller already satisfies move from Any to a concrete type, and the nine search transformations validate each result item on its own so error text no longer carries the item index. No cast, no type: ignore, no suppression marker and no budget file edit * ci(unit): run the brave, google_pse, nlp_cloud and opensandbox tests in the providers shard |
||
|
devin-ai-integration[bot]
|
e0398455d0
|
test: migrate tests/llm_translation into unit, integration and e2e tiers (#45759)
* test: migrate selected llm translation contracts * test: migrate anthropic completion translation tests * test: migrate bedrock and cohere translation tests * test: tighten bedrock translation migration * test: migrate azure and openai model translation tests * test: migrate openai and small-provider translation tests * test: migrate gemini and vertex translation tests * test: migrate optional params and rerank translation tests * test: migrate huggingface, json and openrouter translation tests * test: fix openrouter integration expectations * test: migrate databricks, xai, cometapi and fireworks translation tests * test: migrate ovhcloud, prompt and small provider translation tests * test: split reasoning effort grid into unit and e2e * test: migrate router and realtime guardrail translation tests * test: migrate google interactions translation tests * test: migrate a2a, evals, skills and hosted vllm translation tests * test: remove remaining llm translation suite * test: tighten migrated llm translation unit tests and land missing rows Restores rows the first pass dropped or weakened (bedrock agentcore and openwebui thinking stream, anthropic prefix, citations and tool_choice, azure agents handler, huggingface bare api_base, optional params num_retries, alternating roles, rerank response type), replaces truthiness and call-count checks with whole-value assertions, and moves tests to the mirror file of the code they cover * test: move cohere, databricks, fireworks and gemini rows to mirror files over mocked HTTP Restores three TestCohereV2Transform methods the first pass nested out of collection, replaces direct transform calls with respx-backed completion calls, and drops the databricks and fireworks files that sat outside their mirror paths * test: close lost-intent gaps found in review of the llm translation migration Points the hosted-mock guard at the triton test's new integration path, restores exact rerank result assertions for infinity, drops the reasoning e2e fallback that passed for any multi-line answer, and adds the mixtral invoke system prompt variants, eval deletion, and async pdf url download that the legacy suite exercised * test: assert provider acceptance in the reasoning effort e2e The live half of the split grid proves each route accepts reasoning_effort. claude-fable-5-1 on the direct route answers without surfacing reasoning content, so a reasoning-presence check either fails on head or needs a fallback that passes for any answer. Wire shape stays covered by the unit grid * test: cover async and stream-end paths the legacy translation suite exercised Adds async variants for bedrock converse file and thinking content, bedrock embedding timeout and region resolution, interaction stream termination and success logging, xai reasoning token folding, and bytes chunks with a prefix before data:. Also scopes the vertex safety settings global in test_vertex_safety_settings with monkeypatch so it stops leaking into gemini request bodies on the same worker * test: type the new llm translation e2e calls and mark moved fixture dirs as packages * test: name the reasoning effort e2e models as module constants * test: assert reasoning content, register the new gemini coverage ids and give the guardrail config e2e a unique name * test: block the prompt the shared guardrail denies and drop the reasoning presence check * test: drop the duplicate litellm import left by the main sync --------- Co-authored-by: yuneng <yuneng@berri.ai> |
||
|
devin-ai-integration[bot]
|
a2e334febe
|
test: migrate local_testing SDK and provider tests out of build_and_test (#45760)
* test: migrate local_testing SDK and provider tests out of build_and_test * test: cover sagemaker region, hf template and nova paths end to end Region rows now go through litellm.completion on the sagemaker/ handler that the legacy tests exercised, the prompt template test mocks the Hugging Face tokenizer config instead of switching to custom_prompt_dict, streaming tests assert role, finish reason and body, and Nova single turn and streaming run through litellm.completion with a recorded request body * test: tighten migrated integration tests for local OpenAI routes, timeouts and air-gapped boot Split the catch-all providers migration file by behavior. The fake endpoint route tests now hit the owned upstream directly, as the legacy harness tests did. LM Studio asserts the outbound key, path and stripped model. The embedding timeout test asserts the proxy maps a deployment timeout to a 408 and forwards the read timeout to the provider client, instead of timing out the test client The Azure streaming tool-call test also asserts the second identical stream is served from the Redis cache with the same reassembled tool call The Docker network test always skipped in CI because the security job has no litellm/litellm image and no egress to pull one, and it tested a published image rather than the checkout. It now boots the checkout proxy with every outbound HTTP connection refused and asserts readiness, the bundled cost map fallback and that the cost map host is the only egress attempted * test: tighten the migrated e2e rows for local_testing SDK and provider tests Drop the duplicated Anthropic covers marker and the Anthropic cell wrongly claimed by the Bedrock and Vertex cache tests. Keep the file-wide Anthropic model as it is on main. Send the 2023-06-01 version header to Anthropic through the deployment's extra_headers so the header actually reaches the provider, and mark the creation-token case 1h then 5m like the legacy test did Reuse the Bedrock file's own registration for the pydantic response test, swap the hand-built PNGs for the shared cat.jpg fixture, assert the SageMaker Nova image answer names the cat and that top_logprobs=3 is honored, and register the SageMaker Nova opt-in markers the same way as the other opt-in markers * test: restore lost assertions in migrated test_main tests Restore the role alternation check in test_completion_anthropic_hanging, the embedding and streaming response-header checks, the logit_bias drop check and the gemini system and safety settings checks. Add an azure 429 retry-after test for test_exception_with_headers, trigger the router fallback with mock_timeout like the legacy test, and stop the mock timeout tests sleeping 3s * test: restore lost legacy assertions in migrated vertex ai unit tests * test: tighten migrated cost, streaming and embedding tests and scope the hosted mock guard * test: strengthen migrated anthropic, azure, bedrock and cohere provider tests Assert full request bodies and usage instead of presence checks, restore the dummy-tool and custom_llm_provider assertions the legacy tests made, add the async and per-model variants the legacy parametrization covered, split streamed tool arguments across deltas, and move the Azure chat streaming tests to the azure.py mirror file * test: tighten migrated llms provider tests and cover legacy intents lost in the move Strengthen the migrated fireworks, gemini, mistral, together, openai text completion, openrouter, perplexity and openai error tests so they assert what the legacy tests proved (cost, provider tag, usage-only-on-last-chunk, function_call rebuild, reasoning param, sync/async/stream error fields). Replace the huggingface and jina tests that doubled litellm code with respx end-to-end tests. Add the tests the pre-existing mapped tests never covered: cohere 400 mapping, token-array text completion, hosted_vllm/llamafile/lm_studio env api base embeddings, watsonx embedding auth, ollama ssl_verify client and ollama vision prompt * test: cover residual legacy paths for anthropic tool streams, watsonx async stream, ollama http images and caller async embedding clients * test: restore single request assertions in the deep_infra and azure_ai embedding tests * test: drop a no-op streaming cost flag from the openai streaming cost test * test: cover residual legacy-test lines in caching, main, utils, replicate, vertex and bedrock invoke * test: cover streamed audio assembly, choice membership, untranslated responses events and the stream log guard * test: pin cost rates in unit tests, use sdk for cohere e2e, drop vertex llama e2e * test: send the vertex maps e2e request through the openai sdk * test: drop the local_testing jobs that select no tests after the migrations * test: wait for the streamed cache entry and keep custom Mistral rates test-local * test: keep untouched Mistral test lines as they were --------- Co-authored-by: yuneng <yuneng@berri.ai> |
||
|
devin-ai-integration[bot]
|
e2971e0af4
|
refactor(llms): expose public names for private provider helpers (#45037)
* refactor(litellm): migrate private usage in llms Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix(litellm): preserve Bedrock batch signature marker Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * refactor(litellm): migrate llms private usage symbols Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix(litellm): preserve UUID Watsonx project IDs Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(litellm): retarget llms mocks to public names Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * refactor(litellm): limit llms changes to renames and forwarders Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix(litellm): keep GCS mock client patching private Vertex auth methods Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(litellm): assert forwarder arguments and type forwarder signatures Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> --------- Co-authored-by: mateo <mateo@berri.ai> Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> |
||
|
yuneng-jiang
|
5e6dc89ba1
|
test: move tests/test_litellm/llms into tests/unit/llms (#43191)
* ci: run the unit_selection.sh shard files on every event instead of only fork pull requests Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * ci: rename fork-flag to unit-flag now that it applies on every event * test: move tests/test_litellm root and small trees into tests/unit Pure renames, no content changes. Follow-up commits in this PR fix references, merge the three files that already existed in tests/unit, keep live-provider tests in tests/test_litellm and wire CI. * test: carry tests/test_litellm conftest isolation into tests/unit Callback lists, routing fallbacks, cached HTTP clients, logger state, AWS, proxy-URL and keychain env, and session-end client cleanup now reset for unit tests too. The environment isolation owns its MonkeyPatch so a test's own monkeypatch is undone before the model-cost teardown runs. * test: merge, split and prune the moved root and small-tree tests Merge batches/test_batch_utils.py and the chat_completions and messages dispatch tests into the files that already existed in tests/unit. Keep the live Gemini interactions tests, the async image-fetch format test and the OpenAI embedding scorer test in tests/test_litellm since they need real network or keys. Put test_router.py under tests/unit/test_router so the existing package no longer shadows it. Delete eight tests the audit found superseded by stronger ones kept in this move. * ci: run the moved root and small-tree tests under their legacy flags Add the misc and responses-caching-types flags to unit_selection.sh and CircleCI, extend enterprise-routing and mcp-integration, and point the legacy GHA shards, Makefile, redis-compat workflow, merge smoke manifest and change classifier at the new paths. * test: make the new tests/unit directories packages tests/unit/test_package_layout.py requires every directory to carry an __init__.py, and without one the moved and retained test_litellm_responses_bridge.py modules collide on import. * test: scope the unit socket block to tests/unit in shared sessions The GHA shards collect the legacy test-path and the unit selection in one pytest session. The unit conftest's loopback-only block leaked into legacy modules that reach the network at import. The legacy conftest now lifts the restriction at collect and setup time, and the unit conftest re-applies it when collecting its own modules. * test: move tests/test_litellm/llms into tests/unit/llms Rename-only. Moves the provider tests and the fine-tuning fixtures they load, mirroring the old paths. Follow-up commits merge, split and wire them. * test: merge, split and prune the moved llms tests Merges the Databricks chat transformation tests into the existing unit file, keeps the tests that need real keys or the network in tests/test_litellm, deletes the audited tests a stronger unit test already covers, and points imports at tests.unit.llms. * ci: run the moved llms tests under their legacy flags The Vertex AI and All Other Providers shards keep their legacy test-path for the retained files and add the llm-vertex-ai and llm-other-providers unit selections. CircleCI gets matching unit jobs. * test: make the tests/unit/llms directories packages Adds __init__.py to the moved dirs and drops the legacy ones whose directories no longer hold tests. * test: drop script runners and path hacks the llms split left dangling The __main__ runners in the split openai_like files and the Databricks e2e runner called tests that now live in the other half of the split or were deleted. The retained legacy halves also no longer need sys.path edits. * test: give the shard-script tests their own GITHUB_OUTPUT They only passed where the runner set it. The CircleCI unit job's env allowlist drops it, so the script's redirect failed there. * test: point the router and module-deletion checks at tests/unit router_code_coverage and code_qa_check_tests only searched tests/test_litellm, so the moved router tests no longer counted. The two silent-experiment tests the audit deleted were the only direct callers of those methods; they are replaced with tests that assert the forwarded shadow request and the recursion guard. * test: keep the Databricks manual e2e runner and fix the SageMaker Nova run path The Databricks e2e file is a manual script whose main() calls the tests that were pruned, so pruning them broke the documented run. It is back to its main version. The SageMaker Nova docstring now points at the file's real location in tests/local_testing. * test: keep the job's UNIT_FLAG out of the shard-script tests --------- Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> |
||
|
devin-ai-integration[bot]
|
e2302be068
|
refactor(ocr): remove the Python OCR execution path and require the Rust route (#43081)
* refactor(ocr): remove the Python OCR execution path and require the Rust route Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fmt * refactor(ocr): tidy the native OCR passthrough binding Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * style(ocr): ruff format the azure passthrough transformation Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * refactor(ocr): resolve passthrough OCR costing in one Rust call Replace passthrough_url/passthrough_transform with passthrough_response, which matches the relayed endpoint against each Azure config's path segments instead of building a fake request to call get_complete_url. The binding drops the unused headers, status and api_base arguments. Catch the ValueError/RuntimeError the binding raises so a relayed body that is not OCR-shaped falls back to the passthrough object instead of failing logging, and cover the relay against the real binding. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(ocr): drop the unused LlmProviders import from health check helpers Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * ci: drop the ocr_testing job now that tests/ocr_tests is gone Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * test(ocr): restore the live OCR matrix and the ocr_testing job The public litellm.ocr / aocr / Router interface is unchanged by the Rust migration, so the live provider matrix still applies. Drops the stale VCR skip list for the deleted test_rust_bridge.py. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * test(ocr): import Final in the health check helper tests Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Yujong Lee <yujong@berri.ai> Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
yuneng
|
e4a58ef91a |
test(unit): make every tests/unit directory a package so pytest collection is unique
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> |
||
|
yuneng
|
ad523edb25 | test: migrate phase 9 legacy llm provider tests to tests/unit |