The LLM pass-through route builds its response headers before the response
pipeline runs, so the pre_call sweep claims those steps as on main.
Adds Final to the new test locals.
The applied-guardrails header check now reads the A2A streaming methods from the
same map the A2A endpoint uses to translate method names, instead of its own copy
An A2A response step opens litellm_metadata on the request, so the next step's
name landed there while the spend log and the trace read metadata and lost it
Covers /v1/responses and /v1/messages, the OpenAI and Anthropic SDKs sync and async, the spend log and OTel span, a step whose service errored, forged names and a concurrent burst across two workers
The pre_call header sweep claimed every policy pipeline step up front, so steps a pipeline never reached showed up in x-litellm-applied-guardrails and in spend logs. The sweep now leaves pipeline steps to the executor, which names each step when its hook runs, whatever it decides. Steps a stream runs after its headers go out (a pipeline the stream cannot hold, or one whose steps all stream live) are still named up front, as before.
Resolves LIT-10096
* fix(streaming): stop re-raising and re-logging a failed stream on every next()
A stream that raised keeps its provider iterator alive, so a consumer that iterates past the error re-detected, re-raised and re-logged the same failure on every call, flooding failure callbacks for one request. Latch the failure and re-raise it without logging again.
* fix(streaming): drop the retained traceback before re-raising a failed stream
* test(streaming): count failures through a real logging callback and require a clean stop on later reads
* test(streaming): require the stuck stream to close early and release its socket
The provider stream now records a GeneratorExit and caps its chunks, so draining it or detecting the loop late fails. Two wire cases check a sync and an async stuck stream release their TCP session.
* test(streaming): avoid failure recorder name collision
---------
Co-authored-by: Devin AI <devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
Co-authored-by: moe-berri <moe@berri.ai>
* build(docker): move to newer wolfi-base and nginx base images
Bump the pinned wolfi-base digest in every Wolfi Dockerfile to 360896ef, which
ships brotli 1.2.1-r1 and ca-certificates-bundle 20260909-r3. The previous
digest already carried libbrotli 1.2.0-r6 as an OpenSSL dependency, and
`apk add --no-cache` never upgrades a package the base already has
Move the UI image to nginx 1.31.8-alpine3.24, whose CA bundle is the 20261020
Mozilla set, and upgrade brotli-libs in its runtime stage since no nginx
digest ships 1.2.1 yet
* build(docker): copy only runtime files into the gateway, backend and migrations images
The component runtime stages copied the builder's whole /app, which is the
entire repository plus build output: six lockfiles, helm, terraform, docs,
.env.example and the Rust build tree. Measured at runtime with an audit hook
on a booted container, the gateway reads only gateway/, litellm/ and
enterprise/ from /app, the backend only backend/, litellm/ and enterprise/,
and the migrations job only litellm/ and run.py. The pyproject.toml read is
prisma-python's optional [tool.prisma] lookup, which ours does not define
Copy just those, plus .venv, the entrypoint script and schema.prisma for the
gateway and backend. PYTHONPATH=/app and the in-Dockerfile asserts are
unchanged
Preserve oversized JSON integers in proxy requests and model rewrites
Parse large bodies in a worker thread to avoid blocking request handling
Fixes#47947
Co-authored-by: Jeevan Babu Gotru <114808955+jeevan1098@users.noreply.github.com>
Co-authored-by: moe-berri <moe@berri.ai>
* fix(deepseek): pass reasoning_effort through to the API
DeepSeekChatConfig.map_openai_params mapped reasoning_effort to a binary
thinking {type: enabled/disabled} toggle and discarded the effort value, so
DeepSeek always ran default high effort. High-effort reasoning can run away
and exhaust max_tokens during reasoning, returning empty content. Preserve
the reasoning_effort value into the request body when thinking is enabled so
callers can bound effort (e.g. reasoning_effort="low"). Backward-compatible.
See https://api-docs.deepseek.com/guides/thinking_mode
* test(deepseek): reasoning_effort passthrough coverage
* fix(deepseek): exclude reasoning_effort=none sentinel from passthrough
"none" is the thinking-OFF sentinel, not a real effort value. Passing it
alongside explicit thinking:enabled would send contradictory instructions to
DeepSeek. Only forward real effort values (low/medium/high/xhigh/max).
Addresses review feedback on the reasoning_effort passthrough change.
* refactor(deepseek): name supported effort values and type mapped params
* fix(deepseek): keep mapping types local to the adapter
* fix(deepseek): remove cast and type mapped return
---------
Co-authored-by: moe-berri <moe@berri.ai>
* fix(proxy): reconcile team member total_spend with current cycle spend
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): read /team/list as a list and drain the scripted provider
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* refactor(repositories): compose the membership reset writer and drop SQL text asserts
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test: type the cascade repair test and flatten memberships without a stacked comprehension
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test: keep the reset job test file's existing typing untouched
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: joshua <joshua@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* feat(cost): parse 12-hour cache write rates for batch and ultrafast tiers
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(cost): bill 12-hour batch cache writes once when only the 12hr batch key exists
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(ui): show "Never resets" instead of "n/a" for an unset budget reset period
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(ui): label unset reset period as Keep current/No change where blank keeps the stored value
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(ui): show a single Never resets option in create team when no default is set
Without a default team budget_duration, the create team Reset Budget dropdown
offered both "Not set" and "Never resets", and both created a team that never
resets. The explicit Never resets item is now only shown when there is a
default to opt out of, otherwise the empty option itself reads "Never resets"
Also makes the Reset Budget create tests use chooseSelectOption so they no
longer flake on the select popup's pointer-events transition
---------
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* feat(teams): show caller member spend and budget on Your Teams
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(teams): use member-linked reset dates
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(teams): keep type-discipline suppressions on cast lines
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(teams): preserve empty page list response
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(teams): keep the member reset date inside the spend column
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(teams): match email-only roster members for caller membership
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(ui): show team member budget reset date in UTC
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* refactor(teams): address review on caller membership enrichment
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* feat(ui): label the caller's team spend line My Spend
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(opencode): bill OpenCode Go requests from the reported list_cost
OpenCode returns usage.opencode.list_cost on every response. Read it into the provider-reported cost the cost calculator checks before the price map, for chat and Responses, streamed and not. Keep the static rows as the fallback, and stop the generic usage.cost path from billing Go requests at 0 (usage.cost is what hit the prepaid balance, 0 inside the plan).
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(core): resolve reported cost breakdowns without recursion
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(opencode): cover reported list_cost spend through the proxy
Replaces the recursive _reported_cost_value with a flat _cost_number helper
so the code-quality gate passes, and adds an integration test that runs the
proxy against a scripted OpenCode upstream for a non-stream and a stream
chat completion, asserting the response cost header and the spend log row
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(opencode): bill native /v1/messages from the reported list_cost
The native Anthropic Messages route priced OpenCode Go bodies from the price
rows twice over: the non-stream body is a TypedDict the response-cost header
path priced as-is, and the stream assembled its ModelResponse with the rows
cost already stamped. Both now read usage.opencode.list_cost, the non-stream
body through billable_result and the stream from the final message_delta.
The integration test now covers chat, responses and messages, stream and
non-stream, through the proxy against a scripted OpenCode upstream
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(opencode): keep the stream cost stamp inside the lint gate
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(opencode): read list_cost off dict chunks and trust it only from OpenCode on /v1/messages
Stream builders receive chunks both as mappings and as ModelResponseStream
objects, so the list_cost scan now reads usage through chunk_usage. The
native Messages route only trusts a reported cost from opencode_go, every
other upstream keeps the price map, which also keeps a zero usage.cost from
being billed as free. Served bodies without stop_sequence still price, and
the new code stays inside the basedpyright gate
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* feat(proxy): add disable_pre_routing_error_logs to skip spend rows for pre-routing rejections
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(proxy): decide pre-routing rejections by pipeline stage, not exception type
Guardrails such as AIM raise ProxyException(400), the same shape as an unparsable
body, so the status code list skipped their spend rows. The pipeline stamps
litellm_call_id only after auth, body parsing and model resolution, so a failure
reaching the hook without one died before any provider was considered.
_handle_logging_proxy_only_error flags those requests when it mints the call id,
and is_pre_routing_rejection keys off that flag plus ProxyModelNotFoundError.
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(proxy): mark pre-routing rejections with a server-owned sentinel
request_data is the client body, so a boolean marker or a litellm_call_id in it
can be sent by the client to hide a guardrail block or force a spend row. The
failure handler now stores an object() sentinel whenever it finds no logging
object, and is_pre_routing_rejection checks identity, which JSON cannot forge.
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: joshua <joshua@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(router): skip declared text-only deployments for image requests without enable_pre_call_checks
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(router): run the vision filter after the cooldown filter so a cooled vision leg falls back to the text leg
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(router): apply the vision filter after pre-call checks and tidy the new tests
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(router): mark the remaining locals in the vision filter tests Final
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: joshua <joshua@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* feat(guardrails): add native Reco guardrail provider (pre_call)
Registers guardrail: reco as a first-class vendor integration, built on
the existing generic guardrail API wire contract rather than reimplementing
the HTTP client and payload mapping. Maps reco_tenant_id to the
X-Reco-Tenant-Id header and hardcodes fail_open/fail_on_error since those
should not be customer-configurable. v1 only supports mode: pre_call;
post_call and streaming redaction already work against the existing
generic pilot and are deferred.
Also overrides apply_guardrail to re-raise blocked-request errors with the
guardrail's own configured name: the base class hardcodes a module-level
name in that exception, which would otherwise mislabel every blocked Reco
request in logs.
* feat(guardrails): configurable Reco fail modes
Expose unreachable_fallback and fail_on_error on the Reco guardrail
and keep fail-open as the default. Values under optional_params win.
Top-level litellm_params values apply only when set explicitly, since
their defaults are fail-closed and would otherwise flip the default
Also type RecoGuardrail's forwarded kwargs with an Unpack'd TypedDict
instead of silencing ANN003
Signed-off-by: MickeyShnaiderman-RecoLabs <mickeys@reco.ai>
Co-authored-by: Cursor <cursoragent@cursor.com>
* chore(guardrails): align Reco with main's CI gates
Upstream moved unit tests from tests/test_litellm to tests/unit, so
test_reco.py was invoked by no CI job. Move it next to the other
guardrail tests
Replace RecoGuardrail's forwarded kwargs with explicit keyword
params, read optional_params through a pydantic adapter instead of
untyped dict/getattr access, freeze RecoOptionalParams, and drop
mutable-ok markers that suppress nothing, to stay within main's
type-discipline and basedpyright budgets
Signed-off-by: MickeyShnaiderman-RecoLabs <mickeys@reco.ai>
Co-authored-by: Cursor <cursoragent@cursor.com>
* feat(guardrails): fail closed by default in the Reco guardrail
Matches GenericGuardrailAPI and every other guardrail here: a Reco error or outage now blocks the request unless the admin sets fail_on_error: false. Tests cover the default, explicit fail-open, and top-level vs optional_params precedence
Co-authored-by: RecoLabs-YaronBratspiess <208601538+RecoLabs-YaronBratspiess@users.noreply.github.com>
Co-authored-by: MickeyShnaiderman-RecoLabs <77006830+MickeyShnaiderman-RecoLabs@users.noreply.github.com>
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(guardrails): honor explicit fail-open on Reco when optional_params is a model
model_dump(exclude_unset=True) so pydantic defaults in optional_params no longer override an explicit top-level fail_on_error. Also drop the duplicate log_guardrail_information decorator, call the parent by keyword, freeze RecoConfigModel, and type the tests
Co-authored-by: RecoLabs-YaronBratspiess <208601538+RecoLabs-YaronBratspiess@users.noreply.github.com>
Co-authored-by: MickeyShnaiderman-RecoLabs <77006830+MickeyShnaiderman-RecoLabs@users.noreply.github.com>
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(guardrails): keep Reco config model assignable and the hook decorator the gate requires
frozen=True on RecoConfigModel froze the shared LitellmParams mixin, which the guardrail registry assigns resolved secrets on, so every guardrail failed to load. Dropped it and added a test that LitellmParams stays assignable. Restored log_guardrail_information on the Reco override since tests/code_coverage_tests/check_guardrail_apply_decorator.py requires it on every hook. Test params typed as Mapping for LIT001
Co-authored-by: RecoLabs-YaronBratspiess <208601538+RecoLabs-YaronBratspiess@users.noreply.github.com>
Co-authored-by: MickeyShnaiderman-RecoLabs <77006830+MickeyShnaiderman-RecoLabs@users.noreply.github.com>
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* style(guardrails): format the Reco test file
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Signed-off-by: MickeyShnaiderman-RecoLabs <mickeys@reco.ai>
Co-authored-by: Yaron Bratspiess <yaronb@reco.ai>
Co-authored-by: MickeyShnaiderman-RecoLabs <mickeys@reco.ai>
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: joshua <joshua@berri.ai>
Co-authored-by: MickeyShnaiderman-RecoLabs <77006830+MickeyShnaiderman-RecoLabs@users.noreply.github.com>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(bedrock): price claude_platform calls from the anthropic cost map rows
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix: avoid recursive Claude Platform lookups
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(bedrock): price claude_platform calls from their own cost map rows
Replace the claude_platform -> anthropic lookup redirect with
bedrock/claude_platform/* rows, and apply the inference_geo multiplier
in the bedrock cost path so US-only inference is priced at 1.1x
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* test(bedrock): exempt claude_platform rows from the Converse routing guard
The claude_platform/ route is explicit in the row key, so these rows are
not accidental non-Converse Bedrock chat rows
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
---------
Co-authored-by: devin-ai-integration[bot] <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
* fix(anthropic): stop injecting empty-content placeholder into tool-call-only assistant turns
_sanitize_empty_text_content replaced empty or whitespace-only text with
"[System: Empty message content sanitised to satisfy protocol]" for every user and
assistant turn. anthropic_messages_pt calls it unconditionally, independent of
modify_params, because an empty text block always makes Anthropic 400.
An assistant turn carrying tool_calls and no text is the normal shape an OpenAI-format
agent emits, and Anthropic accepts an assistant turn whose content is just tool_use, so
there was nothing to substitute there. Substituting anyway put a protocol diagnostic in
the content field, which is the assistant's utterance as far as the model and the client
are concerned, so it got persisted into conversation history and echoed back on later
turns.
Empty text is now dropped rather than replaced when the assistant turn already carries
substantive content (tool_calls, or non-text blocks such as tool_use). Turns that would
otherwise be left with no content at all keep the placeholder, since Anthropic rejects
those too.
Fixes#24498
* fix(anthropic): only drop empty text when a sibling block survives conversion
_carries_non_text_content treated any non-text block as substantive, but the assistant
list-content converter only keeps thinking (when signable), server_tool_use and
*_tool_result blocks. For an assistant turn holding empty text next to a block it
discards, such as an image_url or a signature-less thinking block, dropping the empty
text left nothing behind and the whole turn disappeared from the request.
The predicate now mirrors what the conversion actually keeps, so those turns retain the
placeholder and survive. This also improves on the previous behaviour, where such a turn
was dropped outright.
* test(anthropic): cover malformed content blocks in the survivor predicate
_survives_assistant_conversion had a separate non-dict guard that no test reached,
since _is_empty_text_block already rejects non-dict blocks before the predicate runs
on them in practice. Folding the check into the type lookup keeps the same behaviour
for bare strings and blocks without a type key, with no unreachable branch.
Extends the sibling-block test with a block missing its type key.
* test(anthropic): set modify_params through monkeypatch in sanitization tests
The test-quality gate flags direct writes to litellm module globals. Swap the
class setup/teardown for an autouse monkeypatch fixture, drop writes that just
repeat its default, and route the False cases through monkeypatch
---------
Co-authored-by: tbartik <22518863+tbartik@users.noreply.github.com>
Co-authored-by: ryan-crabbe-berri <ryan@berri.ai>
* fix(vector_stores): put azure vector store id before api-version query
Azure OpenAI's get_complete_url returns a url ending in ?api-version=...,
and the vector store handlers appended the id with an f-string, producing
.../openai/vector_stores?api-version=X/vs_abc and a 404 from Azure.
Add BaseVectorStoreConfig.get_vector_store_url, defaulting to the previous
behaviour so other providers are unchanged, and override it in
AzureOpenAIVectorStoreConfig to keep the query string last. The six
retrieve/update/delete handlers now call it.
Co-Authored-By: Claude <noreply@anthropic.com>
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* refactor(vector_stores): trim comments on get_vector_store_url
Drops the inline comment in the Azure override and shortens the base docstring, per AGENTS.md
Co-authored-by: Keith-Hardaway <276775292+hardawaykwork@users.noreply.github.com>
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(vector_stores): capture the Azure retrieve request without a mutable url list
Greptile on #48224: the handler test now follows the file's received list pattern with Final bindings
Co-authored-by: Keith-Hardaway <276775292+hardawaykwork@users.noreply.github.com>
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: Keith-Hardaway <keith.hardaway@docusign.com>
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: joshua <joshua@berri.ai>
Co-authored-by: Keith-Hardaway <276775292+hardawaykwork@users.noreply.github.com>
* fix(router): keep latency routing alive when latency samples are unusable
latency-based-routing ranks deployments by the average of the cached samples
and keeps everything within lowest_latency_buffer times the lowest. A negative
or NaN sample breaks that invariant, because the minimum no longer satisfies
its own bound. The candidate list then comes back empty and random.choice
raises IndexError, which turns every request to the group into a 500 until the
cache entry expires.
Samples come from completion_start_time - start_time, and that difference is
negative whenever the two timestamps invert. Production recorded samples
around -600s from successful calls.
Drop non-finite and negative samples before ranking, and fall back to every
candidate if the window is empty anyway, for example with a negative buffer.
Co-Authored-By: Claude Code <noreply@anthropic.com>
Co-Authored-By: AaronHowell <111272993+AaronHowell@users.noreply.github.com>
* test(router): seed latency_normalized in the unusable TTFT fallback test
Main reads latency_normalized since #47727, so seeding latency left both deployments ranked at zero and the test picked at random.
Co-authored-by: Rick <26716961+Bytechoreographer@users.noreply.github.com>
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(router): assert ranked latency after dropping unusable TTFT samples
Greptile on #48216: the new tests passed with the sample filter removed because the empty-window fallback alone saved them. They now assert _latency_per_deployment so unfiltered negative or NaN samples fail deterministically. Also adds Final to ttft_samples and drops a comment that restated the code
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(router): drop Final on a loop-local in latency ranking
basedpyright rejects Final on a variable assigned inside a loop, and the type_check_gate counts it as a new reportGeneralTypeIssues error
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(router): type the request kwargs in the unusable TTFT tests
Greptile on #48216: bare dict annotations. The tests now hold the metadata dict in a typed local and assert on it directly
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: Rick <26716961+Bytechoreographer@users.noreply.github.com>
Co-authored-by: Claude Code <noreply@anthropic.com>
Co-authored-by: AaronHowell <111272993+AaronHowell@users.noreply.github.com>
Co-authored-by: joshua <joshua@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* refactor(typing): replace pass-through Any with object in 13 annotations
* docs(client): say dict[str, object] in the models get docstring
* refactor(typing): drop the unused Span aliases in base_cache and levo
* refactor(typing): take a Mapping in delete_nested_value and cover the fresh copy
* fix(typing): deep-copy a read-only Mapping in delete_nested_value through a dict
* fix(proxy): buffer tool discoveries in redis and let the leader write LiteLLM_ToolTable in one statement
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(proxy): declare the redis family on the tool discovery buffer and drop the blind except and cast
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(proxy): satisfy LIT010 and LIT014 in the tool discovery buffer
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(proxy): mark the aggregation fold Final
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(proxy): requeue drained tool rows to redis when the write provably did not apply
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(proxy): drain tool discoveries on their own lock and scheduler job, not the spend commit
* chore(proxy): suppress the untyped apscheduler add_job on the tool discovery job
* refactor(proxy): move the LiteLLM_ToolTable bulk upsert into its own module next to the other bulk upserts
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(proxy): name the LiteLLM_ToolTable bulk write span so the prisma span check resolves it
* refactor(proxy): move the tool discovery drain out of DBSpendUpdateWriter into tool_registry_writer
* refactor(proxy): share db_write_failure_is_requeue_safe from exception_handler so the spend writer stays untouched
---------
Co-authored-by: devin-ai-integration[bot] <158243242+devin-ai-integration[bot]@users.noreply.github.com>
