mirror of
https://github.com/BerriAI/litellm.git
synced 2026-09-27 01:22:18 +00:00
6 commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
devin-ai-integration[bot]
|
e450f2d54c
|
fix(router): serve Responses turns from a sibling when the encrypted content origin has no boundary peer (#43015)
* test(integration): reproduce encrypted_content_affinity 503 when origin has no boundary peer Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(integration): keep encrypted content affinity turn one out of the response cache Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix(router): degrade encrypted_content_affinity when the origin has no encryption-boundary peer A routed-group candidate that is currently unavailable and shares its (api_base, api_key) with no healthy deployment used to raise a proxy-level 503/429 from _unavailable_origin_error, even though healthy siblings in the same model group could still serve the turn. Strip the encrypted reasoning and dispatch to the healthy pool instead, matching the existing cross-group behavior, and log a warning naming the origin model_id and routed group Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix(router): bound the degraded-affinity log marker and assert the strip on the wire Address review findings: restore num_retries alongside optional_pre_call_checks in the integration test teardown, record scenario request bodies on the scripted upstream so the tests can assert no encrypted reasoning reaches the sibling, and truncate the client-supplied model_id in the degraded-dispatch warning Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix(tests): only record JSON bodies on the scripted upstream Multipart uploads to scripted POST routes have no JSON body, so gate the observation recording on the request content-type Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix(router): make encrypted_content_affinity runtime-toggleable so /config/update can turn it off --------- Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com> |
||
|
devin-ai-integration[bot]
|
b72a030501
|
test: take keys out of the legacy proxy, enterprise and mcp unit tests before moving them (#42901)
* ci: fix the litellm-tests unit job with sysmon coverage, an env allowlist and coverage upload on failure * test: replace key-dependent proxy, enterprise and mcp unit tests with synthetic values and integration and e2e coverage * test: drop key reads at the legacy proxy, enterprise and mcp paths and wire the gemini pass-through split * ci: fail the unit shard when circleci tests split errors * test: drop restating comments from the gemini pass-through split * ci: exit the unit shard cleanly when circleci tests split assigns it no files --------- Co-authored-by: yuneng <yuneng@berri.ai> |
||
|
devin-ai-integration[bot]
|
3fb6f8740b
|
test(integration): add read-replica routing harness to the CircleCI integration suite (#42692)
* test(integration): add read-replica routing harness * refactor(integration): hoist the maintenance url imports * fix(integration): keep per-test databases and the witness sequence readable under replica roles * fix(integration): opt bespoke database and pool tests out of the injected read replica * test(integration): commit recorded replica routing expectations * fix(integration): judge routing by role containment so shrinking role sets do not fail * fix(integration): run the pool-limit shutdown choreography on the superuser database url * ci(integration): add the mcp group to the replica matrix * fix(integration): judge routing by exact role sets with a named either-role allowlist * test(integration): drop containment-era routing expectations for re-recording * chore(integration): drop docstrings from the replica harness scripts * docs(integration): describe exact routing matching and the either-role list * test(integration): record exact replica routing expectations * test(integration): allow the SELECT 1 health probe on either role * test(integration): replace committed routing expectations with an on-demand base-vs-head parity run * test(integration): fix parity env scope, readme wording, and seed-deterministic serialization test * test(integration): make the sorted-role serialization test deterministic in-process * test(integration): swap all product code in parity runs and pin role gains * ci(integration): force tracked-file removal before parity checkout --------- Co-authored-by: yuneng <yuneng@berri.ai> |
||
|
devin-ai-integration[bot]
|
a286ebf42e
|
test(integration): regression tests for July provider translation, routing and streaming bugs (#42693)
* test(integration): optional Anthropic tool properties stay optional on the OpenAI Responses wire (Pylon #6619) Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(integration): Bedrock InvokeModel count-suffixed cache usage fields are reported and charged (Pylon #6708) Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(integration): anthropic messages honors the deployment request timeout (Pylon #6505) Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(integration): drop client_metadata before the Bedrock Converse body reaches the provider (Pylon #6645) Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(integration): repeat Bedrock requests under one session name assume the role once (Pylon #6681) Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(integration): messages stream keeps include_usage off the Responses wire with always_include_stream_usage (Pylon #6466) Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(integration): clamp sub-16 max_tokens to the Responses API floor instead of 400 (Pylon #6539) Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(integration): forwarded client x- headers reach the provider on /v1/responses (Pylon #6565) Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(integration): Anthropic messages stop_sequences reach OpenAI-compatible providers as stop (Pylon #6536) Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(integration): Codex namespace tools reach a chat upstream flattened and round-trip through /v1/responses (Pylon #6409) Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(integration): keep Claude 4.6 legacy thinking budget_tokens on /v1/messages (Pylon #6727) Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(integration): nvidia nim ranking keeps image passages and applies top_n without sending top_k (Pylon #6401) Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(integration): tpm-only model rejects priority traffic once recorded tokens reach the model tpm (Pylon #6344) Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(integration): reasoning-only chunks open an Anthropic thinking block at index zero on /v1/messages streams (Pylon #6337) Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(integration): file content streams to the client before the upstream finishes sending (Pylon #6315) Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(integration): agent whose card lives only at agentCard/v1.0 is reached with bearer auth (Pylon #6249) Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(integration): vertex batch create returns a batch when outputInfo is null (Pylon #6374) Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(integration): fireworks session id is sent as x-session-affinity and cached tokens land in spend log metadata (Pylon #6220) Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(integration): advisor sub-call failure does not cool down the executor deployment (Pylon #6212) Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(integration): Gemini /v1/messages cache_control creates cachedContent with Anthropic ttl (Pylon #6221) Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(integration): bedrock_mantle max_output_tokens below 16 is clamped before reaching Mantle (Pylon #6262) Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(integration): missing thinking signature 400 on /v1/messages retries without thinking blocks (Pylon #6222) Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(integration): format Gemini messages cache_control wire test (Pylon #6221) Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(integration): rebuilt shared aiohttp session keeps the configured keepalive timeout (Pylon #6387) Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(integration): sagemaker_chat signs the inference component header and sends hf_model_name as the body model (Pylon #6187) Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(integration): Bedrock Converse DeepSeek drops Anthropic thinking and sends V3 reasoning_effort raw (Pylon #6149) Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(integration): concurrent team model TPM requests are reserved before the provider call (Pylon #6075) Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(integration): /v1/messages honors the configured timeout against a stalled upstream (Pylon #6025) Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(integration): Codex additional_tools input items reach Bedrock Mantle as top-level tools (Pylon #6012) Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(integration): advisor api_base without api_key never sends the proxy Anthropic key to the caller host (Pylon #6226) Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(integration): rerank responses carry call id, latency and cost headers (Pylon #5981) Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(integration): parse the outbound Anthropic body with the typed JSON adapter (Pylon #6025) Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(integration): Bedrock Knowledge Base search forwards userContext to the Retrieve body (Pylon #5991) Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(integration): Marengo 3.0 text embeddings reach Bedrock nested under inputType (Pylon #5949) Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(integration): sub-16 max_tokens over a responses deployment reaches OpenAI as 16 (Pylon #6008) Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(integration): midturn system correction reaches the OpenAI Responses wire via /v1/messages (Pylon #6449) Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(integration): concurrent requests over a key tpm limit are rejected before reaching the provider (Pylon #5737) Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(integration): chat to responses bridge keeps deployment AWS credentials for Bedrock Mantle SigV4 (Pylon #5870) Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(integration): vertex gemini stream split across many fragments completes without stalling the proxy (Pylon #5838) Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(integration): bedrock mantle /v1/messages stream keeps stream true and relays SSE events (Pylon #5596) Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(integration): large chat payloads are released from worker memory after the request ends (Pylon #5920) Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(integration): streaming success logs v3 rate limit remaining values for callbacks (Pylon #5767) Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(integration): a database created search tool backs Anthropic web search interception (Pylon #5669) Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(integration): register july provider regression contracts Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(integration): make july provider regression tests deterministic under cache and worker sharing Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(integration): drop order-fragile worker memory probe pending a real retention regression check Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(integration): apply ruff import sorting and formatting Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(integration): drop stale contract entry and pass question to advisor executor Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(integration): use tiktoken-backed executor model in advisor tests Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> --------- Co-authored-by: kerry <kerry@berri.ai> Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> |
||
|
devin-ai-integration[bot]
|
5c0b374f0a
|
test(integration): regression tests for August provider translation and streaming bugs (#42621)
* test(integration): Bedrock batch files upload completions and responses records as user messages (Pylon #6882)
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): client Anthropic OAuth token never replaces Bedrock SigV4 authorization (Pylon #6888)
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): bridge /v1/messages and /v1/responses streams through empty-choices chunks (Pylon #6992)
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): prepend azure content-filter metadata chunk to the messages stream (Pylon #6992)
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): fireworks routers/ slug reaches the provider as accounts/fireworks/routers/<id> (Pylon #7030)
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): bedrock hidden thinking tokens are not reported as text tokens (Pylon #7067)
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): azure_ai FLUX.2-flex image generation targets the flex provider path with the BFL body (Pylon #7092)
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): keep Databricks streaming usage and cache reads in the client stream and spend log (Pylon #7094)
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): openai-compatible image edits forward provider-specific form fields to the backend (Pylon #7122)
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): replayed intercepted web search turn reaches Bedrock as text through /v1/messages (Pylon #7181)
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): streamed web search turn capped by max_agentic_loops ends the turn with snippets and ordered blocks (Pylon #7230)
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): bedrock rerank keeps forwarded client headers out of the SigV4 signature (Pylon #7284)
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): azure_ai rerank authenticates with an Entra token when no api key is set (Pylon #7303)
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): perplexity stream with cost breakdown object completes and bills total_cost (Pylon #7331)
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): azure_ai strips Anthropic message fields before the Foundry request (Pylon #7336)
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): capped intercepted web search ends the turn without an internal tool_use block (Pylon #7378)
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): Bedrock passthrough converse-stream keeps event-stream content-type (Pylon #7482)
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): v1/messages success exposes v3 priority rate limit headers (Pylon #7532)
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): config deployment dropped by a stale boot cost map is restored after reload (Pylon #7564)
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): Bedrock Mantle project id reaches the provider as anthropic-workspace-id (Pylon #7583)
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): prefixed Opus 4.8 reasoning_effort reaches Bedrock as adaptive thinking (Pylon #7586)
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): dashscope chat forwards reasoning_effort to the provider (Pylon #7606)
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): failing stream logging callback still releases the max_parallel_requests slot (Pylon #7608)
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): gen 5 Claude Bedrock Invoke tool search sends the Bedrock beta field (Pylon #7642)
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): databricks ai gateway api_base requests OAuth token from workspace origin (Pylon #7724)
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): deepseek vision image content list reaches the provider unchanged (Pylon #7729)
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): Bedrock Mantle context overflow surfaces as 400 prompt is too long (Pylon #7732)
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): Codex history items reach Bedrock Mantle as supported Responses input types (Pylon #7783)
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): chat over responses deployment returns finish_reason length when output tokens run out (Pylon #7784)
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): bedrock_mantle rewrites Codex history items before the Responses wire (Pylon #7812)
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): advisor sub-call on /v1/messages uses the configured advisor deployment (Pylon #7828)
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): tencent thinking reaches the provider body instead of failing the request (Pylon #7834)
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): xAI chat web search reaches /v1/responses with instructions and nested filters (Pylon #7835)
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): send Bedrock Converse config blocks once at top level (Pylon #7839)
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): bedrock converse sends gpt-5 reasoning_effort as reasoning.effort (Pylon #7850)
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): bedrock cohere.embed-english-v3 embeddings accept encoding_format and dimensions (Pylon #7963)
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): streamed chat completions emit SSE keepalive pings while the upstream is silent before its first token (Pylon #7987)
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* style(integration): format the TTFT keepalive regression test (Pylon #7987)
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): openai chat drops tool_choice when the request has no tools (Pylon #8022)
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): stream whose first chunk has no choices falls back and bills the fallback (Pylon #8006)
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* Revert "test(integration): Bedrock Mantle project id reaches the provider as anthropic-workspace-id (Pylon #7583)"
This reverts commit
|
||
|
Yuneng Jiang
|
7679e42736
|
test: cover provider wire contracts, streaming and recovery
Adds owned local TCP peers, a Redis process helper and SigV4 helpers to tests/integration, and integration contracts for Anthropic and Bedrock wire shapes, Bedrock role configuration, S3 wire, observed routing, Redis recovery and stream contracts. Consolidates the four commits previously stacked on litellm_integration_accounting onto its main-merged tip |