rust-analyzer cannot resolve the bare alias names emitted by
macro_rules_attribute::apply, so every aliased type was invisible to it
(no go-to-definition or find-references). Referencing the aliases through
crate:: resolves them via the re-export that attribute_alias! generates.
Co-authored-by: Nate Armstrong <narmstrong@Nates-MBP.localdomain>
* fix(proxy): mark background responses stale_expired when provider returns 404
Responses deleted upstream (store=false / ZDR rows dropped after provider retention) now move to stale_expired on the poll instead of being retried every cycle until the 7 day stale cleanup.
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* refactor(proxy): drop inline comment in check_responses_cost
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(proxy): drive _get_response seam in 404 stale_expired tests
Assigning AsyncMock on the instance keeps the new tests inside the TQ002/TQ008 test-quality ceilings
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(proxy): only expire 404 responses polled through a resolved deployment
A NotFoundError from the bare SDK fallback can mean a missing or misconfigured deployment, which a config fix inside the staleness window can still recover. Rows whose poll went through their router deployment are the ones the provider actually dropped, so only those move to stale_expired.
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(proxy): resolve the deployment before fetching so via_router binds once
Drops the reinitialised loop flag flagged by review and modernizes the moved annotation.
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(proxy): drive the real response fetch in the poll 404 regression tests
* fix(enterprise): expire background response rows on any provider 404 status
The GET path maps the provider's 404 body to litellm.BadRequestError that still carries status_code 404, so an except on NotFoundError never fired and the poll job kept retrying the row every cycle. Key the terminal decision on the status code instead of the class
* fix(enterprise): expire a background response only on a 404 naming it and skip a bad deployment entry per job
* chore(enterprise): record the poll-cycle bound on the managed-object IN lists
* test(integration): cover background response poll retirement on the real proxy
---------
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
* test: move 76 live top-level tests to offline unit and integration coverage
* test: keep the mock-param opt-in guard valid once the top-level senders are gone
* test: address review on offline replacements
* docs(test): drop the deleted harness test file from the harness check command
* test: make the key rebind, team member delete and routes integration tests exercise the legacy paths
* test: make fallback, rpm and spend integration contracts deterministic and clean up their rows, assert the rpm limit in usage-based routing
* test: expect the no-deployments error at the rpm limit and cover the strategy check without pre-call checks
* test: hand member cleanups to the scenario instead of growing a budget list, flatten callback kinds
* test: scope the admin health check to the test's own deployment
---------
Co-authored-by: yuneng <yuneng@berri.ai>
* docs(rust): add ADR folder with ADR 000 and ADR 001 scaffold
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* docs(rust): add ADR 000 and ADR 001 on typing requests inside Rust
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* docs(rust): list four ADR sections in ADR 000
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* docs(rust): move ADR process doc from adr_000 to AGENTS.md
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: Nate Armstrong <narmstrong@Nates-MacBook-Pro.local>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(responses): keep tool_result next to tool_use on Anthropic previous_response_id continuations
Replayed history no longer re-adds each stored turn's instructions, and the
current request's instructions go before the replayed history instead of
between the last tool_use and its tool_result
* test(responses): fake the Anthropic transport instead of acompletion in the continuation regression test
* fix(responses): keep the previous turn's instructions when a continuation sends none
* fix(responses): carry only the latest stored instructions when a continuation sends none
Replaying each spend log's own instructions put a system message between a
replayed tool_use and its tool_result, which Anthropic rejects with a 400
* refactor(rust): add typed LLM wire contracts to llms-types
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* style(rust): drop blank lines left in new wire types
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* feat(rust): complete additive wire type contracts
* feat(rust): complete additive nested wire contracts
* feat(rust): give Messages content blocks concrete per-block contracts
Replace the shared all-optional ContentBlockPayload with one struct per
documented block and nested tagged unions for server tool results, so
required fields from the official Messages reference are enforced.
Make CustomTool a plain struct with required name and input_schema, add
MessagesToolParam plus the toolset and undated tool-search tags, model
MiniMax media sources as a tagged union, require documented fields on
chat audio, image URLs, and logprobs, and rename the new block enum to
MessagesContentPart so it no longer shadows the streaming struct.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* fix(rust): tighten new wire contracts against the official references
Give each Messages builtin tool a concrete contract with its documented
required fields, fixed names, and typed allowed callers in place of the
shared all-optional ToolDefinition bag. Make MCP tool result content and
fallback triggers optional as the request params document, and narrow
tool search references, web fetch content, and bash results to their
documented block types.
Keep unmodeled Responses output items and new usage iteration and stop
detail types as Recognized values instead of rejecting the whole payload,
accept find_in_page and optional open_page URLs, preserve JSON Schema
property order, and move prompt_cache_breakpoint to Chat Completions
content parts where OpenAI documents it.
Drop OCR table and key/value types that only matched Azure Document
Intelligence and would reject other providers' normalized passthrough,
along with unsourced bounding box and applied edit fields.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* feat(rust): type every documented Responses output item and close wire gaps
Add the 18 remaining documented Responses output item variants with
concrete nested actions, outputs, and safety checks, and add the
documented namespace, async, caller, and image generation fields to the
existing items.
Restrict MCP tool result content to a string or text blocks, model the
Messages diagnostics cache-miss union and its request param with null
distinct from absent, and accept draft-07 tuple items and prefixItems in
JSON Schema.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* test(rust): keep Messages contract assertions in test bodies
Compare built-in tool decodes against fully built expected values and
split the required-field rejections per type, so no case carries its
assertions in a callback.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* docs(rust): cite the Mistral OCR reference beside the OCR wire types
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
---------
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
The Add Model selector read the merged runtime cost map, so deployment
ids and provider-prefixed backend keys registered for proxy deployments
showed up as duplicate model choices. The public cost map endpoint now
accepts catalog_only=true to return the catalog as loaded, and the Add
Model panel requests that view. The default response is unchanged.
Resolves LIT-9263
Co-authored-by: yassin <yassin@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(lens): reject feedback writes from callers with no team or key
* test(lens): teamless callers cannot overwrite feedback
* test(lens): expect lens_feedback retention during schema setup
* refactor(lens): flatten feedback summaries without a stacked comprehension
* chore(lens-ui): drop routine comment on the feedback panel
* chore(lens-ui): drop routine comment on the feedback view
* chore(lens-ui): drop routine comment on the low feedback check
* chore(lens-ui): drop routine comment on the post stub
* test(integration): run the response cache tool-call tests on a proxy without the message cap
Since #43878 the response cache skips any request past 4 messages by default, so the three providers-lane tests that post a seven-message tool conversation twice never saw a cache hit. They now boot one owned proxy from the lane config with max_messages set to null and keep their assertions; the shared proxy and the caching-lane cap test are unchanged
* test(integration): give the uncapped-proxy cache tests the owned-proxy timeout budget
The providers lane runs pytest with a 90 second per-test timeout that includes fixture setup, and the first of the three tests to run pays the owned proxy boot before the spend-log test can poll for up to 70 seconds. The sibling owned-proxy tests already carry pytest.mark.timeout(240), so these three take the same budget
---------
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
* fix(proxy-extras): name the database error when the Lens rename check cannot run
The Lens rename check runs before prisma db push and raised a generic
"Cannot verify Lens data safety" RuntimeError on any psycopg error, with
the cause only in the chained exception. proxy_cli prints the message and
exits, so a Postgres at its connection limit read as a Lens problem.
The raised message now ends with the redacted database error text, and
the connection opener is an injected parameter so the unit test drives
the check without a database.
* test(proxy-extras): drop the Lens check test class docstring
---------
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
* feat(mcp): support upstream OAuth client metadata identities
* fix(mcp): load client metadata on cold OAuth requests
* fix(mcp): discover client metadata with configured OAuth endpoints
* fix(mcp): preserve configured endpoints when discovery fails
* fix(mcp): retain CIMD identities across refresh and catalog reloads
* fix(mcp): reuse saved CIMD identity for token endpoint refresh
* fix(mcp): keep dynamic client registration ahead of CIMD unless the deployment opts in
A provider that advertises both a registration endpoint and client ID
metadata documents now gets the registration flow the gateway used
before, so a gateway on a private network keeps working against it;
the metadata document identity applies when the provider offers no
registration, or when general_settings.mcp_prefer_client_id_metadata_document
is true. The saved CIMD refresh identity compares tokens as bytes so a
non-ASCII refresh token cannot crash the token route.
* chore(ui): regenerate dashboard API types for the new MCP general setting
---------
Co-authored-by: Joshua Valluru <326636767+joshua-berri@users.noreply.github.com>
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
* feat(ui): add TypeSafe and Strands Decider to the Add Model provider list
* test(public_endpoints): type the provider fields helper against the route's model
---------
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
* test: make legacy live tests in spend, batches, openai endpoints and audio dirs offline (partial)
* test: migrate wave-1b live tests offline (guardrails, images, ocr, search, openai endpoints)
* test: fix wave-1b review items, add responses/ocr integration tests and firecrawl unit test
* test: anthropic messages router/bedrock/openai-bridge unit tests for wave-1b nodes
* test: anthropic messages logging, prompt-caching and tool-search unit tests; drop migrated base nodes
* test: finish pass_through_unit_tests nodes, logging drain fix and mutations
* test: migrate anthropic passthrough tests to integration wire tests
* test: fix passthrough migration wire spend row lookup and wildcard config
* test: migrate hosted vllm and openai file passthrough tests offline
* test: move assemblyai and vertex passthrough nodes to in-process unit tests
* test: restore unlisted router node and fix logging worker drain in passthrough unit tests
* test: drop spend-row BUG skip and sharpen non-streaming skip reason for anthropic messages
* test: use public presidio alias after merge
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test: restore the batch and file unit tests the migration rewrote away
* test: restore full legacy intent in anthropic messages router unit tests
Drop the false BUG skip on non-streaming aanthropic_messages logging (success
callbacks do fire; the skipped body filtered on the wrong model), assert the
logged model_group, messages, cost and usage, cover streaming logging for both
Anthropic and Bedrock invoke, assert dict content blocks for Anthropic, Bedrock
invoke and the OpenAI bridge, add the Bedrock invoke leg of the router test,
fall back from a real 401, test system-prompt caching and streaming
message_start cache fields on converse and invoke, send the legacy tool-search
tools and beta header, remove the type: ignore and bare dict helper, and add
in-process native /anthropic passthrough spend logging tests
* test: fix passthrough migration wire tests and drop false native spend BUG skip
The native spend row test read rows by the shared master-key digest, so it
matched other tests' rows; read each row by its own message id instead and
assert tokens, total, spend, tags, provider, api_base and end_user on both the
non-streaming and streaming native routes. Merge the streaming test that never
checked spend, assert exact tags from litellm_metadata, stop rebinding a Final
in a loop, require cost > 0, and add the chat-completions bridge cost case the
legacy test covered
* test: harden the spend, OCR, image, search and guardrail migration replacements
The OCR spend tests now build fresh kwargs per case instead of mutating a
shared fixture, and every payload case asserts the exact logged spend. The
OCR wire test reads its spend row by request id and checks exact page
pricing. Image edit, Nova Canvas, DuckDuckGo, Firecrawl, Bedrock guardrail
and Presidio replacements now fake only the provider HTTP boundary (respx or
an in-process aiohttp server) and assert the outbound request, so the
DuckDuckGo limit, Azure base_model pricing and guardrail masking are proven
rather than assumed
* test: cover Exa and Perplexity search structure and max_results offline and retire the two base search methods
* test: drive batch and file replacements through the provider HTTP boundary and real logging callback
Replace monkeypatched litellm.afile_content and AsyncHTTPHandler doubles with respx routes,
read batch logging metadata from a registered success callback instead of get_logging_payload,
use the real managed-files hook for the GEN-2166 regression, assert outbound request bodies,
pin poller ownership explicitly in the migrated DB-sync tests, and require the scripted
upstream to be hit in the responses error-status wire tests
* test: assert file content download headers pass through the proxy
* test: point spend coverage references at the tests that replaced the retired spend job
* test: assert passthrough identity, spend and route dispatch from the code under test
The AssemblyAI non-admin test asserted metadata it wrote itself and leaked a
background poll to the real AssemblyAI host. It now drives assemblyai_proxy_route
with a real Request and waits for the success callback for its own transcript id.
The Vertex spend test matches its log by call id instead of taking the first event.
The OpenAI files wire test hit the native /{provider}/v1/files route; it now calls
/openai/files so the passthrough is what forwards the upload and delete.
* test: mock only the HTTP boundary in the migrated audio tests
Vertex TTS tests no longer replace _ensure_access_token or AsyncHTTPHandler.post;
the token comes from a mocked Google OAuth endpoint and the synthesize call from
respx. Speech tests assert the outbound body, the transcription cache test polls
for the cache write instead of relying on test ordering, and the model pass-through
test checks the multipart model field per model.
* test: wait on a logger event instead of polling the clock in anthropic messages unit tests
Recorders keep payloads in a rebound tuple and set an asyncio.Event; tests
await it with asyncio.wait_for instead of a sleep-and-deadline poll loop
* test: freeze module-level batch and file response fixtures as Final MappingProxyType
* test: fake the presidio analyzer with an in-memory aiohttp connector
The blocked-entity tests started an aiohttp TestServer, which binds a local
socket. They now hand the guardrail a ClientSession whose connector answers
/analyze and /anonymize in process, so no socket is opened and the outbound
analyze text and entities are still asserted
* test: type anthropic messages router test helpers with LiteLLM's Anthropic TypedDicts
Messages, cached system blocks and tool-search tools now use
AnthropicMessagesUserMessageParam, AnthropicMessagesTextParam,
AnthropicToolSearchToolRegex and AnthropicMessagesTool instead of bare
dict shapes; tools are converted to plain dicts only at the acreate call,
whose tools parameter is list[dict]
* test: type batch limiter helpers with TypedDicts and wait on the logging callback event instead of polling
* test: give the migrated OCR, image and presidio helpers precise types
OCR spend helpers take ReadOnly TypedDicts for kwargs and responses and use
LiteLLM's OCRResponse/OCRUsageInfo instead of local pydantic stand-ins; spend
metadata is validated with a TypeAdapter. The presidio fake uses LiteLLM's
PresidioAnalyzeRequest/ResponseItem types, and the image-edit logger validates
the logged payload instead of storing an untyped dict
* test: signal callback and cache events instead of polling
Recorders keep tuples and set an asyncio.Event, thread-safely, when the payload
for this test's transcript id or upstream URL arrives. The transcription cache
test waits on a Cache subclass that signals after async_add_cache. No clock
polling or sleeps remain in these tests.
* test: assert the batch limiter hook updates the caller's request in place
* test: tolerate model-list probes and read native passthrough rows by owned key
The router's OpenAI-compatible model-info refresh (litellm/router.py:10710)
sends GET /v1/models to configured openai api_bases, so the wire answers it
with an empty list and excludes it from the provider-call assertions. Native
/anthropic spend rows are now read by a per-request virtual key digest and
call_type, then the row's request_id is checked against the message id
* test: expect the OCR alias in the proxy response model
The proxy restamps every OpenAI-compatible response model to the name the
client requested (_override_openai_response_model), so /v1/ocr returns the
scenario alias. The upstream model is now checked on the drained request body
instead of inside the peer, where a failed assert never reached the test
---------
Co-authored-by: yuneng <yuneng@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test: make 77 legacy live tests offline in litellm_utils, router_unit and responses dirs
* test: restore lost coverage in litellm_utils offline replacements
Bound live health-check tasks to the concurrency limit so eager task
creation behind a semaphore fails, route the Langfuse trace-id check
through completion -> Logging.get_trace_id with all four metadata
combinations, run the callback dedup checks through acompletion, pass
the dynamic key via litellm_params, cover the env-inferred default
model list, and add the per-provider audio transcription config lookup
* test: restore router coverage lost in the offline move
Add the missing non-stream LIT-3058 header/count-once test, pin the UTC minute on every
usage-counter test, assert router-level client reuse for transcription, and replace the
private selector-attribute checks with routing outcomes per strategy. Assert outbound bodies
for speech, rerank, image, assistants and moderation, add timeouts to event waits, and drop
the router_unit_tests husk files that no longer hold tests
* test: restore sync stream, router sync stream, error event and field-type coverage for migrated responses tests
Add the sync streaming logging and sync Router.responses streaming cases the
offline replacements dropped, port the legacy per-event and response field-type
validation, assert raw headers, the float created_at conversion, MCP tool headers,
the search_context_size input, and add an offline replacement for the in-stream
context-window error event. Remove helpers left dead in the legacy file.
* test: run numpydoc-backed unit tests in GHA and drop a test that pinned a crash
* test: restore sequence number, item id and content part checks in the responses stream validator
* test: match router logging events to the test's own deployment so late events from other tests are ignored
* test: restore the s3 cold storage history test to its original assertions
---------
Co-authored-by: yuneng <yuneng@berri.ai>
* fix(cache_control): let a hosted deployment opt into the OpenAI cache dialect
_targets_openai_prompt_cache_breakpoint gated on custom_llm_provider == "openai"
unconditionally, so an OpenAI-shaped model served by another provider could never
qualify, even with supports_prompt_cache_breakpoint set explicitly on its own
cost-map entry. bedrock_mantle honours prompt_cache_breakpoint end to end, and
had no way to say so.
model_cost is keyed per exact deployment string, so a flag on the deployment's own
entry states the dialect more precisely than a provider name can. Consult it
before the provider check, and require the entry's litellm_provider to match the
serving provider so an openai entry cannot license another provider. Entries for
the openai provider keep their api_base check, so an OpenAI-compatible third-party
host is still not assumed to speak the dialect.
Fixes#38666
* test: drop a laziness assertion the existing suite already makes
test_provider_lookup_skipped_for_models_below_gpt_5_6 already pins that
_resolve_provider is not called for an unflagged model, and it covers the new
flag lookup unchanged. The duplicate patched a litellm internal for no added
coverage, which the test-quality gate counts against TQ008.
* fix(bedrock_mantle): flag GPT-5.6 and newer rows for OpenAI explicit prompt cache breakpoints
* fix(responses): predict the chat-completions bridge with the model name the dispatch resolves
* fix(cache_control): read a region-prefixed Mantle GPT deployment through its region-free price-map row
* fix(cache_control): let a bare hosted deployment name read its own provider's price-map row
* fix(responses): hand the hook the provider the router resolved for a Foundry GPT deployment
* fix(cache_control): read the deployment breakpoint flag strictly and default a null prompt_cache_options
A cost-map or deployment `supports_prompt_cache_breakpoint` that is not the boolean `true` (the string "true",
the integer 1, "false", 0, a long string) no longer opts a deployment into the OpenAI prompt cache dialect; only
`true` does, the same reading Bedrock Converse applies to its own flag.
A client that sends `"prompt_cache_options": null` on `/v1/chat/completions` or `/v1/messages` now gets the
implicit default the hook already applied on `/v1/responses` when it placed a breakpoint; before, the null
suppressed the default and the request left with a breakpoint and no options.
* test(integration): audit cells for the Mantle GPT prompt cache breakpoint dialect
Deterministic cells for the configured-breakpoint path on Bedrock Mantle GPT and Azure AI Foundry GPT-6
deployments: every endpoint, streaming and not, sync and async SDKs and raw httpx, the hostile option shapes,
flag precedence, the response cache twin, a two-worker burst, a worker kill and a graceful restart.
* fix(cache_control): ignore a malformed cache_control_injection_points value instead of failing the request
A deployment or client `cache_control_injection_points` that is not a list of points (a string, an
integer, a bare point dict, a list of strings) raised inside the prompt hook (`'str' object has no
attribute 'get'`, `'int' object is not iterable`) and turned every request to that deployment into a
500 on `/v1/chat/completions`, `/v1/messages` and `/v1/responses`. Every entry point now reads such a
value as no configured points, the way a `null` already read, and the request leaves without a
breakpoint; entries of a list that are not points are dropped and the point entries kept.
* test(integration): cover int, dict and string-list injection point shapes in the Mantle audit cells
The D9 cell now also sends an integer, a bare point dict and a list of strings as the deployment's
`cache_control_injection_points`, which the merge base answered with a 500 on every request.
* fix(cache_control): stamp the OpenAI dialect for a bare deployment name served by a flagged provider
A deployment written as `model: openai.gpt-5.6-sol` with `custom_llm_provider: bedrock_mantle` has no cost-map row
of its own and no openai row of the same name, so the dialect stamp's cheap gate (`supports_openai_prompt_cache_breakpoint`)
returned early and the configured point was never stamped. On `/v1/responses` and on the chat seeding path the hook
sees no provider, so it fell back to the Anthropic `cache_control` marker, which Bedrock Mantle strips.
The gate now also passes a model whose serving provider is already known and whose provider-keyed row carries the
flag, which is the same row the dialect resolution reads and costs no provider lookup. A bare name without a provider
is still left alone, so models below GPT-5.6 keep their points untouched and resolve nothing.
* test(integration): cover a bare Mantle deployment name with its provider in the audit cells
A deployment configured as `model: openai.gpt-5.6-sol` plus `custom_llm_provider: bedrock_mantle` sends the
breakpoint and the implicit options on all three endpoints; the merge base leaves it on the Anthropic dialect.
* fix(cache_control): keep a configured point that sits beside junk entries on every chat seed path
`configured_injection_points` kept the point dicts of a mixed `cache_control_injection_points` list as a tuple,
but only read a `list` back. The chat seed and the `/v1/responses` dialect stamp write the normalized value back
onto the request, and on a deployment the OpenAI dialect does not stamp (an Anthropic model, Claude on Bedrock
Mantle) that value is the tuple itself, so the hook then read it as no configured points and the valid point was
silently dropped. The stamped OpenAI dialect and the `/v1/messages` path kept it only because they build a new list.
The normalizer now reads back the tuple it wrote, so the point reaches the wire on every path; an all-dict list
still passes through as the same object.
* test(integration): cover a configured point beside junk entries on a Claude Mantle deployment
A deployment configured with `cache_control_injection_points: ["system", {system point}, 3]` marks the system
block on the Anthropic dialect on all three endpoints; the merge base answers 500 on the junk entry.
---------
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
* test(integration): boot owned proxies with a readiness-sized worker healthcheck budget
* test(integration): keep owned_proxy_process's extra_arguments for cells that pass their own flags
---------
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
* ci(lint): derive gate ceilings from merge-base counts and drop the budget files
The four lint gates (ruff strict, type discipline, basedpyright, test quality)
now fail a branch only when a rule's codebase count grows past its count at the
merge-base with litellm_internal_staging plus a fixed per-rule headroom, which
is zero everywhere except the LIT010/LIT011 and reportAny/reportExplicitAny
seeds. Base counts come from a disk cache, then the CI artifact the renamed
publish-lint-base-counts workflow uploads for every staging push (all four
checkers, one artifact per checker and sha), then a scan of the base worktree.
The four *-budget.json files, make lint-budget-update, budget_ratchet_check.py,
and the unratcheted check are gone, so no PR carries a budget edit again.
* test(lint-gates): assert checker identities by behavior and type the new gate tests
* test(lint-gates): resolve base points over an injected git so unit tests stay in-process
---------
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
* fix(proxy): answer a rejected websocket handshake without crashing the HTTP exception handler
* fix(proxy): read the OTLP route check's scope keys with .get and drive the websocket auth tests through the real auth path
* test(proxy): build the websocket auth test keys without reading the clock
* fix(responses): map input_audio blocks in the chat-to-Responses bridge
* fix(responses): honor base_model when deciding to drop input_audio under drop_params
* fix(responses): carry a dropped audio part's cache breakpoint to the preceding part
Dropping an input_audio part under drop_params lost the prompt_cache_breakpoint
riding on it, so an injected system marker landing on a trailing audio block left
prompt_cache_options explicit with no breakpoint on the wire. The marker now moves
to the nearest preceding kept part when that part has none.
Update the CircleCI-only Responses bridge wire tests that pinned input_audio to
its stringified input_text shape: video_url becomes the stringified kind so the
marker-on-stringified-block coverage survives, and dedicated audio tests assert
the forwarded part and the drop_params carry-over.
* refactor(responses): carry dropped audio markers in one pass
Replace the per-block slice that re-read the rest of the content list for
every retained part with a single reverse scan, and skip the copy for a
message that carries no audio part. Type the unit test helper's messages
as a sequence of mappings instead of bare lists.
* test(responses): add integration cells for input_audio parts through the bridge
Cover the chat-to-Responses bridge's input_audio handling on the real proxy: the part is forwarded
as input_audio without drop_params and dropped under drop_params unless the model or its base_model
supports audio input, across httpx, the OpenAI SDK (sync stream and async), the Anthropic SDK, native
/v1/responses, tool and assistant messages, hostile input_audio values, duplicate parts, an audio-only
message, marker carry-over, a null drop_params, a global litellm_settings.drop_params, a mixed burst,
and a burst with dropped upstream connections
* test(responses): check every upstream attempt of a dropped connection
The dropped-connection cell required exactly one upstream POST per call,
which tied it to the retry policy instead of the fix. It now groups the
recorded POSTs by marker, requires one attempt per answered call, and
checks that every attempt of a doomed call carries the dropped audio shape.
* fix(router): honor a per-request fallbacks list on a mid-stream fallback
The completion entrypoints record the per-request fallbacks, context_window_fallbacks, and content_policy_fallbacks in the mid-stream controls carrier the Responses and Messages paths already use, and each completion attempt restores them into the kwargs its stream re-enters the fallback chain with, so a streaming request that dies before its first chunk fails over to the list the request named instead of the router-level list only. The sync vs async parity cell now asserts the backup's text on both twins.
* test(router): annotate the new fallback test locals as Final and flatten the leak check
* fix(proxy): skip null key and team router settings when merging per-request overrides
* test(proxy): annotate the null router settings test locals and drop a redundant comment
* fix(router): keep include_fallback_errors off the provider call on the sync Router path
Router.completion spread include_fallback_errors into litellm.completion, so OpenAI
answered 400 Unknown parameter and a fallback walk lost the backup's reply. Only
Router._acompletion popped the router-only flags. Both paths now build the provider
call through one shared helper that drops them, and a wire-level respx test covers
each path.
* test(router): pin include_fallback_errors off the wire across the sync, async, and proxy paths
* feat(lint): add LIT013 requiring pydantic models to be frozen
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(lint): let an explicit frozen=False override earlier or inherited frozen=True in LIT013
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* refactor(lint): annotate LIT013 helper locals with Final
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* refactor(lint): document the frozen pydantic model rule as LIT015
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* feat(lint): honor the frozen class keyword in LIT015
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* style(lint): annotate main and scan_paths locals as Final
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(lint): cover LiteLLMBaseModel, BaseSettings, and SettingsConfigDict in LIT015
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(lint): resolve LIT015 model bases per class node
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(lint): treat the OpenAIObject alias as a LIT015 model base
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(lint): resolve shared LIT015 config constants and list frozen-ok in the gate hint
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(lint): resolve LIT015 shared config at class definition
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: mateo <mateo@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* feat(decisions): serve System One format at /v1/systemone and OpenAI format at /v1/decisions
The System One request format moves to /v1/systemone and /systemone. /v1/decisions and
/decisions now accept the OpenAI Decisions API format, translate it into a System One
request, route it through the same pipeline, and translate the answers back.
* fix(decisions): import assert_never from typing_extensions for Python 3.10
* fix(decisions): cap OpenAI-format questions at the System One limit
/v1/decisions accepted up to 200 questions, but the System One request it translates to takes at most 128, so 129 to 200 questions failed with an internal validation error. Both now share MAX_DECISION_QUESTIONS.
* fix(decisions): return 400 for bodies that are not JSON
* test(decisions): send System One bodies to /v1/systemone in integration tests
* feat(decisions): add OpenAI as a Decisions provider
System One requests to openai/ models are translated to OpenAI's Decisions shape on the way out and OpenAI's answers are translated back, so both /v1/systemone and /v1/decisions can route to gpt-6-luna.
* refactor(decisions): move the OpenAI wire translation into llms/openai
* feat(decisions): route both request formats through a shared decisions IR
System One and OpenAI-format bodies now convert to one internal representation, and each provider translates it to its own wire format. OpenAI deployments receive the caller's messages, images, names, typed choice values, level descriptions and safety_identifier unchanged. System One providers return a 400 for image input. Cached and cache-write tokens are priced on both response shapes, and provider response extras survive the round trip.
* fix(decisions): keep provider fields inside System One answers
* fix(decisions): load on Python 3.10 and 3.11 and bill cached tokens once under custom pricing
Decisions IR dataclasses used a mappingproxy default, which is only hashable from Python 3.12, so import litellm failed on 3.10 and 3.11. Decisions usage now reaches cost_per_token as a Usage with prompt_tokens_details, so custom pricing no longer adds cached and cache-write tokens on top of input_tokens that already include them
* fix(decisions): honor litellm OpenAI key and base settings for the openai provider
OpenAI decisions now resolve the key from litellm.api_key, litellm.openai_key, then OPENAI_API_KEY, and the base from litellm.api_base, OPENAI_BASE_URL, then OPENAI_API_BASE, matching other OpenAI calls. Providers own their configured key and base lookup through the endpoint config.
* test(decisions): clear every OpenAI base setting in the proxy OpenAI deployment test
* fix(decisions): send OpenAI instructions for System One questions without them and reject single-option questions
* fix(decisions): return guardrail_information on OpenAI-format decisions when requested
* refactor(decisions): validate proxy request data before reading guardrail settings
* feat(decisions): serve System One format at /v1/systemone and OpenAI format at /v1/decisions
The System One request format moves to /v1/systemone and /systemone. /v1/decisions and
/decisions now accept the OpenAI Decisions API format, translate it into a System One
request, route it through the same pipeline, and translate the answers back.
* fix(decisions): import assert_never from typing_extensions for Python 3.10
* fix(decisions): cap OpenAI-format questions at the System One limit
/v1/decisions accepted up to 200 questions, but the System One request it translates to takes at most 128, so 129 to 200 questions failed with an internal validation error. Both now share MAX_DECISION_QUESTIONS.
* fix(decisions): return 400 for bodies that are not JSON
* test(decisions): send System One bodies to /v1/systemone in integration tests
* fix(decisions): return guardrail_information on OpenAI-format decisions when requested
* refactor(decisions): validate proxy request data before reading guardrail settings
* fix(proxy): admit litellm_proxy/hosted_vllm in provider-endpoint discovery
get_provider_models gated all endpoint discovery on membership in the
static litellm.models_by_provider dict, which litellm_proxy and
hosted_vllm are intentionally absent from since their model list only
exists behind the provider's own endpoint. GET /v1/models returned the
literal wildcard string instead of the expanded model list for these
providers. Now falls through to ProviderConfigManager, which already
knows about them, before giving up.
* fix(tests): stop patching an SDK internal in the discovery regression test
TQ008 flagged patching litellm.proxy.auth.model_checks.get_valid_models.
Assert the gate's own return value instead of mocking past it.
* chore: drop redundant comment per repo convention
* test(proxy): cover litellm_proxy wildcard model discovery end to end
---------
Signed-off-by: mayuriphad <163738104+mayuriphad@users.noreply.github.com>
* feat(lens): add lens_feedback ClickHouse table for human trace scores
ReplacingMergeTree(UpdatedAt, IsDeleted) keyed like agent_traces_by_key so
feedback joins traces in sort order, with a bloom filter on TraceId, a score
CHECK, and the same retention as traces
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* feat(lens): add scoped ClickHouse reads for trace feedback
feedback_target resolves a visible trace's team, key and trace_ref before a
write; feedback lists the latest live entry per author; feedback_summary
returns count, average and lowest score for a batch of traces
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* test(lens): pin the Claude session to trace id hash shared with feedback
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* feat(lens): expose feedback queries to the Python trace bridge
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* feat(lens): add typed feedback models and ClickHouse feedback store
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* feat(lens): add /lens/feedback API for 0-10 human scores on traces
PUT saves or replaces the caller's score and comment, GET lists every
entry, DELETE removes the caller's, POST /summary feeds the trace list.
Callers can target a trace_id or a Claude Code session_id
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* feat(lens-ui): add typed feedback calls to the traces API
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* feat(lens-ui): flag rated traces and filter the list by feedback
A Feedback column shows each run's average score and rating count, marked
red when any score is 4 or below, and a filter narrows to rated or
low-score runs through the URL
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* feat(lens-ui): show and edit human feedback inside a trace
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* feat(lens): let app keys post end-user feedback on their own traces
Any key can now PUT/DELETE feedback on traces its team or key sent, naming
the end user in an optional user field. Reading feedback stays with Lens
admins
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* feat(lens-ui): show end-user feedback first in a run and flag low-scored rows
Replace the edit popover and feedback filter with a read-only view: the
list shows each run's score in a fixed-width badge and tints the row red
when a user scored it 4 or below, and opening a run shows what users said
with their score right under the header
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
* fix(proxy): declare the config_params service target for the Moyai UI settings write
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(proxy): exercise the Moyai settings write against a real DualCache
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(proxy): allowlist the Moyai quick-connect routes on the backend component
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(proxy): observe the Moyai settings write target through a composed DualCache
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* feat(lens): add who started the run to RunSource
* feat(lens): carry agent.source.user on trace span rows
* feat(lens): resolve the run source user from the span row
* feat(lens): read agent.source.user in trace_spans query
* feat(lens): read agent.source.user in trace_page_spans query
* feat(lens): read agent.source.user in trace_span_batch query
* feat(lens): read agent.source.user in trace_list_span_batch query
* feat(lens): decode the source user from clickhouse span rows
* test(lens): add source user to trace cache test rows
* test(lens): add source user to trace cache read fixtures
* test(lens): add source user to trace cache snapshot fixtures
* test(lens): add source user to capture fixture rows
* test(lens): round-trip source user on span row contract
* test(lens): cover run source user resolution
* chore(lens): regenerate python trace types with source user
* chore(lens): regenerate trace json schema with source user
* chore(lens): regenerate trace page json schema with source user
* chore(ui): add source user to api types
* feat(lens): add run user chip and slack thread chip
* feat(lens): put agent, user and thread on the first row of the run header
* test(lens): cover the run header identity row and compact totals
* feat(lens): start demo runs from a slack thread
* refactor(decisions): dispatch /v1/decisions through provider configs and the shared HTTP handler
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(decisions): map OpenRouter connection failures to APIConnectionError and drop explanatory docstrings
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(decisions): keep APIConnectionError for unreachable deployments on every provider
`_map_upstream_exception` now maps a status `_handle_error` synthesized for a
non-HTTP failure to `APIConnectionError` before `exception_type` runs, so a
Perplexity (openai-compatible) deployment that cannot be reached no longer
answers `InternalServerError` where main answers `APIConnectionError`.
Audit cells: the unreachable-deployment integration cell is parametrized over
the five providers, OpenRouter connection failures are pinned on chat, stream,
embeddings, decisions and the SDK, a deployment whose provider has no decisions
config gets the 400 naming every supported provider, and a unit cell pins the
mapper alone. The first docstring sentence of the five translation bases is
dropped.
---------
Co-authored-by: kerry <kerry@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
* fix(lens-ui): open a trace on its top-level step instead of the first failed one
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* test(lens-ui): pin that failed runs open on the root step
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* refactor(lens-ui): drop the unused firstErrorSpan helper
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* test(lens-ui): stop testing the removed firstErrorSpan helper
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* fix(lens-ui): stop showing the model name as a run's input when none was recorded
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* test(lens-ui): cover the trace list row with no recorded input
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
* add config-driven bound to /responses websocket connection
* test(proxy): cover responses websocket session limit and idle first frame
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* chore(ci): sync API schema and Ruff formatting
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(proxy): close responses websocket client before upstream cleanup at session limit
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(proxy): assert recorded websocket outcomes and simplify session reaping
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(proxy): move responses websocket session limit timing coverage to integration
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(proxy): run provider-close checks off the event loop
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): cover handshake auth, subprotocol, DB override and worker kill for the responses websocket session limit
Adds the audit cells for the configurable session limit: a rejected bearer fails the handshake with 403, a requested Sec-WebSocket-Protocol is echoed back, a DB override through /config/field/update caps new sessions on every worker without a restart while sessions accepted before it keep their limit, deleting the override restores the default, out-of-range and wrong-type updates answer 400 and leave the stored row alone, and SIGKILLing the worker holding idle sockets drops only its sockets while the proxy keeps serving
* test(integration): open override sockets until both proxy workers hold one
uvicorn workers share one listening socket and a worker that wakes first accepts a whole simultaneous burst, so eight sockets opened at once can all land on one worker. The DB override cell now opens its sockets one at a time, alternating the two paths, until at least eight are open and every worker holds one, bounded at forty
---------
Co-authored-by: Mrinal Chanshetty <mrinal@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
* fix(proxy): accept router-wide default_litellm_params in required-param validation
The required-body-param check added in #43787 only consulted request data
and deployment-level litellm_params, so a param supplied solely by
router_settings.default_litellm_params (e.g. max_tokens for
/v1/messages, documents for /rerank) was rejected with a 400 at route
entry even though the router would have injected it during dispatch.
Consult the router-wide defaults in the same place, treating None-valued
defaults as absent to mirror the setdefault merge.
* fix(proxy): consult only the defaults the dispatching router will merge
Review follow-up: router-wide defaults are now consulted only where
dispatch actually applies them. user_config requests dispatch on their
own throwaway Router, so they consult that config's defaults instead of
the global router's; search, managed-agent and eval routes, and
model-less direct dispatch never pass through the router's defaults
merge, so they keep the route-entry 400. Also adds rerank router-default
coverage (unit + e2e) and drops the redundant source comment.
* fix(proxy): traceparent/baggage fallback must not override caller metadata
The W3C traceparent/baggage fallback in
add_litellm_metadata_from_request_headers documents itself as last-resort:
Lower priority than everything above - only fires when neither the
explicit litellm headers nor the Anthropic-metadata path found anything
But it guards on the top-level `litellm_trace_id` / `litellm_session_id` body
keys and never checks `metadata`, which is the documented way callers set
trace_id / session_id (`metadata: {"trace_id": ...}` on /chat/completions).
A caller that explicitly sets metadata.trace_id has it silently replaced by the
header value, so the implementation contradicts its own stated precedence.
This is not a corner case on managed platforms: GCP's front end injects a
traceparent into every inbound request, so the fallback fires on traffic whose
caller never sent the header. The request still returns 200 and the trace still
reaches the logging backend, just under an id the caller never chose, so any
caller correlating by its own id silently fails to find its trace.
Guard both fallbacks on the caller's request-body metadata as well.
Deliberately narrow:
- x-litellm-trace-id still outranks the body (documented priority #1)
- a request that steers neither field still adopts traceparent/baggage exactly
as before
- steering is per-field: setting only trace_id still lets session_id come from
baggage
- litellm_metadata is checked too, for LITELLM_METADATA_ROUTES (/v1/responses,
/v1/messages, batches, files)
Also corrects the comment, which understated the guard.
4 new tests; each fails without the source change. The existing traceparent and
baggage tests are unchanged and still pass.
* fix(proxy): check only the active metadata container and ignore empty values
Addresses review. The first version treated any value in either `metadata` or
`litellm_metadata` as caller steering. On LITELLM_METADATA_ROUTES the body
`metadata` is provider-facing and is only promoted later, so a session_id there
suppressed baggage before apply_missing_session_id_policy ran, and a request
with a usable baggage session id got a 400 under `missing_session_id: reject`.
An empty session_id did the same, since the policy treats "" as absent
Now only the active metadata container counts, and only a truthy value, which
matches how apply_missing_session_id_policy decides a session id is present.
The helper takes a typed `object` rather than a bare dict
Adds regression tests for both cases, including the end to end reject path,
and drops the test docstrings
* fix(proxy): count promoted caller trace ids on litellm_metadata routes
Addresses review. On LITELLM_METADATA_ROUTES the proxy promotes the caller's
trace control fields (trace_id, session_id, ...) from `metadata` into
`litellm_metadata`, but only after the header fallback runs. Checking only
`litellm_metadata` let traceparent and baggage fill those fields first, and the
promotion then skipped them because they were already set, so the trace was
recorded under the header ids
The check now also reads `metadata` for fields in
LITELLM_TRACE_CONTROL_METADATA_FIELDS on those routes, and
apply_missing_session_id_policy uses the same check, so `reject` no longer
refuses a request whose session id is about to be promoted
The earlier test asserting that a session_id in `metadata` must not block
baggage on /v1/responses had the premise wrong, since that value is promoted.
It is replaced by tests that assert the promoted caller ids win on
/v1/responses and /v1/messages, with no policy, reject, and generate
* fix(proxy): an empty trace field in litellm_metadata shadows the promoted one
Addresses review. Promotion copies a trace control field from `metadata` into
`litellm_metadata` only when the key is absent, so a key that is present but
empty in `litellm_metadata` wins and the `metadata` value is never used. The
check still looked at `metadata` in that case, which let
`missing_session_id: reject` accept a request that ends up with an empty
session id, and let an empty trace_id or session_id block the headers
The check now follows the same rule as promotion. If the active container has
the key, its value decides. Only when the key is absent does the promoted
`metadata` value count. This makes the conflicting-bucket cases behave exactly
as on main
* fix(proxy): stay within the LIT006 cast budget
The type-discipline gate failed: this branch added 4 unsuppressed `cast()`
calls and LIT006 was already at its limit. Each cast now carries a
`# cast-ok` reason. The two in the helper follow an `isinstance` check that
proves the Mapping. The two at the call sites stay because the method's
`data` parameter is a bare `dict`, and dropping those casts adds
basedpyright unknown-type errors instead. No behavior change
* test(proxy): cover caller metadata precedence over W3C trace headers
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(proxy): generated session uses promoted caller trace id
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(proxy): move test context into assertion messages
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): audit cells for w3c fallback precedence
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): tighten audit chaos cells
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* style(test): drop stray blank lines
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): ignore body-less model-info probes in langfuse precedence upstreams
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(proxy): keep root trace/session fields on equal ids and usable caller values
The _caller_trace_field gating added for caller-metadata precedence was
presence-aware, which regressed three pre-call behaviors versus main:
- Equal ids in W3C headers and body metadata suppressed the
traceparent/baggage branch entirely, leaving the root
litellm_trace_id/litellm_session_id unset so router fallbacks minted a
fresh uuid4 per request. The header branch now also fires when the
caller value equals the header-derived id, so equal ids stamp the root
fields exactly like main.
- A truthy but non-string metadata value (e.g. session_id 4815162342)
counted as caller-supplied and suppressed the baggage fallback, so
downstream str-only consumers (code interpreter sandbox reuse) got a
new sandbox per turn. _caller_trace_field now counts only non-empty
string values, and the generate policy falls through to generation
when the caller value is not usable.
- On litellm_metadata routes a usable caller session id suppressed the
missing_session_id policies while never landing on the root field, so
the root session stayed unset. The policy now promotes the caller's
usable session id to litellm_session_id instead of leaving it stranded.
* test(integration): non-string caller ids fall back to W3C like empty ids
The audit cell compared a headers leg against a headerless leg for
equality, which only held while a truthy non-string id suppressed the
W3C fallback on both legs. With unusable values ignored again, the
headers leg resolves to the header ids while the headerless leg cannot,
so assert the headers leg's concrete outcome instead, mirroring the
empty/null cell.
* style(proxy): trim the session promotion comment to the non-obvious why
---------
Co-authored-by: Filipe Andujar <filipeandujar@gmail.com>
Co-authored-by: yucheng <yucheng@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
