* test(token_counter): price PDF pages for OpenAI and Azure models the way they bill
* fix(token_counter): price PDF pages for OpenAI and Azure models the way they bill
* fix(token_counter): count azure, data-zone and dated GPT names with their family's encoding
* fix(token_counter): price only drawn PDF images and keep other gateways' GPT pricing
Detect raster images from the page content stream, including inline
images on pages with empty resources, and walk each form once. Strip
only openai/, azure/ and Azure data-zone prefixes, bound the model
family scan, read Azure-only chat models from the cost map, and keep
cl100k_base for listed non-chat OpenAI models.
* fix(token_counter): price GPT names behind any prefix like the bare name and walk shared forms per resources
Prices a PDF by the model name's last path segment, the same name the proxy counts with,
keeps listed non-chat models on Claude pricing, walks each form once per inherited resources
dictionary so a shared child form drawing an image is not missed, and resolves PDF pricing
only when a message carries a PDF
* fix(token_counter): keep the rendered-page count for image pages, parse each form once, and match chat models only
A PDF page that draws a raster image keeps the rendered-page estimate, since newer GPT models bill such a page near
it. The image walk parses each form once and stops at a fixed number of drawn XObjects, so a form shared under many
resources dictionaries no longer slows the count. The family prefix match skips non-chat entries, and ft: fine-tunes
count like their base model
* fix(token_counter): count Azure-only chat models with o200k like other OpenAI chat models
* fix(token_counter): count reservation and project ITPM with the serving deployment's model
Budget reservation and project ITPM counted with the requested group name, so a
GPT-named group served by Claude reserved a PDF at OpenAI's price. They now count
with each deployment's model minus its provider prefix, the name count_tokens uses,
and take the largest. A custom OpenAI tokenizer with no model counts with cl100k
again instead of raising, and listed Azure gpt- chat entries price as OpenAI.
* perf(token_counter): find drawn PDF images with a byte scan instead of parsing every operator
pypdf parsed every operator of a page's content stream to find the images it draws, so a
compressed page of a million operators took about 30 seconds to price for an OpenAI model.
A linear scan of the decoded stream for name Do and inline images finds the same draws.
* fix(token-counter): count each team deployment by its PDF pricing and honor base_model
Budget reservation and the TPM limiter now look up the deployments that serve the
caller's team, by base_model when set. A deployment that reads PDF pages and prices
them unlike the group name counts with its own model, any other with the group name.
A comment between an XObject name and Do now reads as whitespace, and the timing
tests moved to CodSpeed benchmarks.
* fix(benchmarks): install pypdf in the benchmark environment
The PDF token-count benchmarks parse pages with pypdf, which only the proxy-runtime extra installed, so CodSpeed took the no-parser path.
* fix(proxy): count PDF pages for deployments served under multi-segment model names
* fix(proxy): count a team's wildcard route the way routing serves it and reserve OpenAI chat's per-PDF charge
* fix(proxy): count routing-group and alias members with the caller's team, as routing serves them
* fix(anthropic): retry /v1/messages without strict when Claude rejects strict tools under drop_params
The native /v1/messages path sent tools with `strict` straight through, so
when Claude's strict-tool limits rejected them with a 400 the caller got that
400 even with drop_params set. Under drop_params, a 400 that matches Claude's
strict-limit phrases on a request with a strict tool is now sent once more
with `strict` removed from every tool. Without drop_params nothing changes.
The Claude strict-limit matcher moves from bedrock/common_utils.py to
anthropic/common_utils.py so Bedrock converse and the base messages config
share it, and the Rust bridge routes strict tools under drop_params to Python.
* fix(anthropic): retry without strict only when a tool sends strict true
A tool with only strict false no longer triggers a resend of an identical body, the warning no longer
blames tool schemas when the rejected schema may be the output format, and requests with only
additional_drop_params stay on the Rust messages route since the Python retry ignores them
* fix(anthropic): keep sync messages calls with strict tools on Rust
Python has no sync /v1/messages handler, so routing a sync call with a
strict tool to Python raised ValueError where Rust served it before.
The strict-tools detector now applies to async calls only. The retry
warning also names the strict tool count without a plural.
---------
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
* fix(anthropic): drop the non-spec custom object from /v1/messages tool definitions
Anthropic and Vertex AI reject a tool carrying a custom object with tools.N.custom.custom: Extra inputs are not permitted. Drop it on the shared Anthropic request path, keep the hoist of a boolean custom.defer_loading to top-level defer_loading, reuse the helper on Bedrock, and stop the chat adapter from merging custom into the tool's JSON schema
* fix(messages): hoist custom.defer_loading only on Claude routes and strip custom on count_tokens
Non-Claude Messages providers such as DeepSeek now get custom dropped without a defer_loading hoist, since they have no tool search. /v1/messages/count_tokens on Anthropic, Azure AI, and Vertex AI drops custom too, and the local token counter no longer counts it as schema.
* fix(anthropic): drop tool custom object only under drop_params and hoist defer_loading for Claude models
Without drop_params the tool is forwarded as sent, so the provider's 400 stands. A boolean custom.defer_loading still becomes the top-level flag for Claude models and on Bedrock Invoke and Mantle. Bedrock count_tokens now normalizes invokeModel tools the way Bedrock Messages sends them.
* fix(bedrock): drop tool custom object on the Mantle count_tokens fallback
Mantle Messages drops custom unconditionally, so its count_tokens call now counts the tools the same way.
* fix(messages): keep custom-tool Messages calls on Python and match Mantle count_tokens to Mantle Messages
* test(vertex): annotate count_tokens tool test locals with Final
* fix(messages): pass tools without custom through untouched so Bedrock fallbacks keep filled tool names
* fix(messages): keep Bedrock's in-place tool cleanup so fallbacks still work, and stop hoisting defer_loading on the OpenAI-like passthrough
---------
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
* fix(token_counter): count OpenAI models newer than tiktoken's table with o200k_base
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(token_counter): keep tiktoken's encoding for suffixed variants of models it knows
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(token_counter): move the o200k provenance note into the assert message
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(token_counter): strip openai/ and azure/ prefixes before picking the encoding
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(rust_bridge): azure/gpt-4o now resolves to o200k_base
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(token_counter): check the provider prefix through _fix_model_name to keep the type gate flat
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(token_counter): check o200k on the gpt-6 models tiktoken 0.14 still does not know
* fix(token_counter): count Azure deployment names and OpenAI alias groups with their OpenAI tokenizer
* fix(token_counter): count mixed, alias, and base-model deployments with every served model's tokenizer
A group that serves OpenAI and non-OpenAI models now reserves at least each served model's count, model_group_alias names count as their target group, custom Azure deployment names count with their base model, and azure_ai/, chatgpt/ and similar prefixes resolve to the listed OpenAI model.
* fix(token_counter): keep mixed groups on OpenAI tokenizers, cache served models, and count context checks, failed rows and stream fallback with the served model
* fix(token-counter): keep mixed groups at their old reservation and count new OpenAI release names with o200k
A group that serves an OpenAI model next to another vendor now counts its own name the way tiktoken reads it as written, plus its OpenAI models, so it reserves what main did instead of the lower o200k count
base_model picks the tokenizer for estimated usage only when it names an OpenAI model, now also in the Cursor stream counter and the output-limit response
Unlisted gpt-5+ and o-series names count with o200k_base, and wildcard routes run the context-window check with the matched OpenAI model
* fix(token-counter): count an alias to a group no OpenAI model serves by the alias as written
Keeps the count main gives such an alias, so its budget reservation and rate limit match main and no other vendor's tokenizer loads for it. Also sorts the import the main merge left unsorted.
* test: type the new token-counting tests' parameters
* fix(token-counter): count a release-named group no OpenAI model serves as main does, and snapshot the OpenAI model set before iterating it
* fix(token-counter): read served models through the router's allowlisted deployments_named, and coordinate the registration race test
* perf(token-counter): classify each distinct served model once when walking a model group
Budgeted requests walk every deployment in their model group, since the
router's cached copy is outside the router surface the proxy may use.
Classifying each distinct served model once instead of once per
deployment cuts a 200-deployment group from 2.3-4.5 ms to about 1.4 ms
with the same result.
---------
Co-authored-by: mateo <mateo@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
* fix(bedrock): omit system prompt from audio transcription
* fix(bedrock): report Converse token usage on audio transcription
Bedrock transcriptions returned usage null and were billed at $0. Map
Converse inputTokens, outputTokens and totalTokens into the transcription
usage so spend uses the model's per-token price. input_token_details is
optional because Converse does not split audio and text tokens.
Also switch the updated request-shape test to rstest.
* fix(bedrock): default transcription temperature to 0 and price ap-southeast-2 Voxtral regionally
Voxtral wrapped some transcripts in quotes at its default sampling
temperature. In direct Converse calls in ap-southeast-2, 0 of 48 calls at
temperature 0 were framed against 7 of 72 at the default, and instruction
wording made no clear difference, so the request now sends temperature 0
when the caller sets none, matching OpenAI's transcription default.
Calls in ap-southeast-2 billed at the us-east-1 price. The cost map gains
bedrock/ap-southeast-2 Voxtral rows with all four service tiers, the Bedrock
transcription response carries the deployment region in its hidden params,
and the transcription token cost uses the region-resolved model key.
The bridge now builds the response with model_copy instead of assigning
usage after construction, and the native wheel test checks the usage the
compiled route returns.
* fix(bedrock): forward transcription language, reject unsupported formats, and bill env and Tokyo/London regions
get_optional_params_transcription had no Bedrock branch, so language,
response_format and timestamp_granularities were dropped before the Rust
transform ever saw them. Bedrock now forwards language, prompt and
temperature, accepts response_format json or text (the only shapes it can
return), and answers 400 for srt, vtt, verbose_json or
timestamp_granularities unless drop_params is set.
The billing region now falls back to AWS_REGION_NAME and then AWS_REGION
when aws_region_name is unset, the same order the Rust resolver uses to
pick the endpoint, so an env-configured region gets its regional price.
Adds ap-northeast-1 and eu-west-2 Voxtral rows (standard, batch, flex and
priority) from the AWS Bedrock offer files, the other two regions AWS
serves Voxtral audio in.
* fix(bedrock): send transcription temperature as a number and name the rejected param
The proxy keeps multipart form values as strings, so -F temperature=0.2
reached Converse as "0.2" and Bedrock answered 400. The Bedrock mapper now
converts temperature to a float and answers 400 (or drops it under
drop_params) when it is not a finite number.
get_optional_params_transcription copied prompt and temperature back from
the raw call after the Bedrock branch ran, which undid that conversion and
any drop. The Bedrock branch now owns every OpenAI transcription param.
An unsupported param such as timestamp_granularities now gets a 400 that
names it instead of the generic "user/encoding format" message.
* refactor(bedrock): return unsupported transcription params as a value and raise at the param step
* fix(bedrock): drop the retired max_tokens field from the regional voxtral price rows
---------
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
* refactor(rust-bridge): require the native extension
_native ships in every wheel and the sdist will not install without it, yet the bridge loaded it optionally and carried None paths that could fail open for fork safety, exception mapping and OTLP errors. Delete loader.py and import _native directly. Tests inject a fake module through NativeBinding(source=...) or with_source() instead of patching the loader
The install smoke checks now verify the compiled module, and tests fail instead of skipping when _native is missing. The interactions spec canary installs the project so it has the extension
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* fix(ci): build Rust bridge in Windows smoke tests
---------
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* refactor(anthropic): rename llms/anthropic/pass_through to llms/anthropic/messages
Move the unified /v1/messages code out of a directory called pass_through, since it is not the
passthrough feature. The inner messages sub-package becomes native, and the Vertex AI partner
model experimental_pass_through package becomes messages. The old packages stay as import shims
that warn once and alias every submodule in sys.modules, so old deep imports still resolve
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* refactor(anthropic): keep Final variables out of loops in the rename shim
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(anthropic): match the warning text in the rename shim test
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* refactor(anthropic): resolve old pass_through imports lazily through a meta path finder
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* refactor(anthropic): mark the alias loader spec restore for the type discipline gate
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* style(anthropic): format the renamed package shim
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* style(anthropic): keep the rebind-ok marker on the spec restore line
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* refactor(anthropic): carry the new package search path on alias specs
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(anthropic): declare the shim test locals Final
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(logging): fire one CustomLogger hook per async call outside the async flag list
aspeech, atranscription, aimage_generation, aimage_edit, aocr and asearch are
missing from is_sync_litellm_request's flag list, so the sync success and
failure handlers called a CustomLogger's sync hook on top of the async hook the
async handler had already called for the same request.
The sync handlers now skip a CustomLogger's log_success_event and
log_failure_event when that run is paired with an async handler run
(for_async_request) and the async handler gives the logger its own async hook.
The async client wrapper, dispatch_failure_handlers, the proxy-only failure
dispatch and the native bridge's sync failure step for an async call mark their
sync failure run as paired.
Resolves LIT-9486
* fix(logging): keep the sync CustomLogger hook when no async success runs
The async client wrapper skips async success logging for calls carrying a
fallbacks kwarg, and the native bridge skips it for internal and fallbacks
calls, yet the paired sync success run still dropped a both-hooks logger's
sync hook, so that logger got no success event at all.
The sync success run now suppresses the sync hook only when the async success
handler was actually scheduled for the call: the client wrapper passes
async_handler_runs=not is_completion_with_fallbacks, and the native host
records per call whether it enqueued or deferred the async success.
Tests that pinned the doubled failure hook on native aocr, messages and
responses calls and on proxy-rejected requests now expect the async hook
alone. The integration test adds /v1/messages and /v1/responses cells and
runs the key-auth rejection on every route.
* fix(logging): type dispatch_success_handlers kwargs
The new async_handler_runs keyword on success_handler made the untyped
**kwargs that dispatch_success_handlers forwards an unknown argument for it,
which grew the reportUnknownArgumentType count past its merge-base ceiling.
Typing the forwarded kwargs as Any, as _success_handler_body already does,
removes those diagnostics without changing what is forwarded.
* test(integration): cover CustomLogger hook counts across modes, chaos and router strategies
Moves the hook count rig into tests/integration/_support/hook_counts.py and
extends the cells it drives:
- streamed chat, Messages and Responses requests, success and failure
- loggers the async handler never reaches (sync-only overrides, sync event
lists only) keep one sync hook per request on every surface outside the
async flag list, key-auth rejections included
- a concurrent burst while the logger's destination stalls and then fails,
and a burst while a worker is killed, never give a request two hooks
- least-busy keeps counting a request in flight while async speech requests
finish around it, and usage-based routing v2 counts an async image
request's tokens once
* test(integration): Datadog and LangSmith deliver one record per async request
Points the built-in Datadog, LangSmith and OpenTelemetry loggers at a local
sink and checks every async surface outside the flag list: one Datadog log
entry and one LangSmith run per request (two on the merge base), and never two
OpenTelemetry spans, which its own emit-once guard already ensured.
* fix(logging): give a both-hooks logger its sync hook when the async run never reaches it
The sync hook was skipped whenever the logger sat in the async callback list, so a
logger the async run never reached got no hook at all: a sibling stalled past the
logging worker timeout, a batch poll that skips default logging, or an async
failure run cut off. The sync run now holds the hook and the async run hands it
back unless it reached the logger
* fix(logging): run the held sync success hooks when the proxy drops a blocked response's success
A post-call guardrail block drops the parked async success, which left the sync
success hook held for it unrun. Base gave a both-hooks logger that hook on these
call types, so the drop now ends the pairing and the held hook runs as before
* fix(logging): run held sync hooks only after every sync run of the request has closed
A held hook released by the async run could run on another thread while the sync
run was still walking its other callbacks. The pairing now waits until the async
run has ended and every sync run has closed, and the last sync run to close runs
the held hooks itself
* fix(logging): end the async run when its coroutine is dropped before it starts
A logging queue that drops the async success coroutine, or a deferred success
that fails to schedule, left the sync hooks held for it unrun. The enqueued async
run now ends when its coroutine is discarded, and the proxy ends it when firing
the deferred success raises
* fix(logging): leave a still-queued async run to the exit flush
The discard finalizer ran at interpreter exit before the logging worker's exit
flush, so a queued request could get its held sync hook and then its async hook
* test: keep the router and file-stream tests in step with main
Main gives each owned proxy a private Redis by default, so the router cells
share the worker's Redis to read its counters, and the file-stream stub takes
the hook pairing the stream now passes
* fix(logging): run held sync hooks when a during_call guardrail raises or the exit flush gives up
A during_call guardrail that raised after the LLM call parked its deferred
success skipped the proxy's flush, and async runs the exit flush never started
were never ended, so a both-hooks logger lost its sync hook in both cases
* fix(logging): skip the sync failure hook when the async failure run logs to the logger
An async call outside is_sync_litellm_request's flag list ran both failure
handlers, so a CustomLogger with both failure hooks got both. The sync run
of an async request now skips log_failure_event for a logger that overrides
async_log_failure_event and is registered for async failures. The four sync
runs of async failures (dispatch_failure_handlers, the @client async wrapper,
the proxy-only failure thread and the Rust host) pass for_async_request=True.
Drops the HookPairing fallback. Successes are left to #47192, which already
gives such a logger only async_log_success_event.
---------
Co-authored-by: yucheng-berri <293926326+yucheng-berri@users.noreply.github.com>
* refactor(rust-bridge): detect Python-only features in the catalog instead of Rust declines
* refactor(rust-bridge): split and document route-specific detectors
---------
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* refactor(rust-bridge): decide native routing in the catalog, drop the bypass predicates
* refactor(rust-bridge): simplify catalog compatibility checks
---------
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* refactor(rust-bridge): let each route host pick the body params
The native routes projected request body params from a name list, hard-coded in
Rust for Messages and read from `PARAMETERS` for Chat Completions and Responses.
Each route host now answers `body_params(request)` over the keywords the call would
send, so the Python side owns which params reach the provider. No behavior change
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* fix(rust-host): use read-only body parameter types
Co-Authored-By: yujonglee.dev@gmail.com <yujong@berri.ai>
---------
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* refactor(rust-bridge): move the route host modules under host/routes
Each native route's Python half lived in its own `<route>/route_host.py`, apart
from the shared host modules Rust also imports by name. They now sit together in
`litellm/rust_bridge/host/routes/<route>.py`, with their unit tests moved to the
mirrored path. No behavior change
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* test(rust-host): reference messages route alias
Co-Authored-By: yujonglee.dev@gmail.com <yujong@berri.ai>
---------
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
The route host now reports get_configured_request_timeout() in MessagesSettings, and prepare uses the call timeout first, then that setting, then the 600s default. Before this, litellm_settings.request_timeout and REQUEST_TIMEOUT were ignored on the native path, so a long turn that works on Python timed out after 10 minutes
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
The Responses route now resolves openrouter alongside openai. The
OpenRouter config reuses the OpenAI request and response transforms and
owns only its bearer credential (OPENROUTER_API_KEY, then OR_API_KEY) and
base URL (OPENROUTER_API_BASE). Wire model ids follow Python's
get_llm_provider, including the openrouter/auto native id with an
explicit provider. The Python route host no longer hands litellm.openai_key
to OpenRouter.
The OpenRouter live test lives in tests/live/openrouter.rs and returns
early when the provider key is not exported, so cargo test stays offline
by default instead of hiding it behind --ignored.
Co-authored-by: nate-berri <nate@berri.ai>
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
* feat(rust-bridge): scope catalog rules with a contextvar
Co-Authored-By: Yujong Lee <yujong@berri.ai>
* test(rust-bridge): select native integration routes with scoped rules
Co-Authored-By: Yujong Lee <yujong@berri.ai>
* test(rust-bridge): run the Rust extension suite without LITELLM_RUST
Co-Authored-By: Yujong Lee <yujong@berri.ai>
* test(rust-bridge): use scoped catalog rules in dispatch tests
Co-Authored-By: Yujong Lee <yujong@berri.ai>
* refactor(rust-bridge): keep catalog rules a static global and override them once per test
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
---------
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
* refactor(rust-bridge): return an unstarted Execution from native entrypoints and drive it in Python
Rust's run_call built Python streams itself and imported
litellm.rust_bridge.streams by name, so sync calls were driven in Rust
while async calls came back as a Rust-made coroutine. Every native
inference entrypoint now returns one unstarted Execution and
public_execution.py drives it with lifecycle.run or lifecycle.drive,
wrapping streams and _hidden_params on the Python side. Rust only
imports the lifecycle step classes through StepTypes, and
python-bridge/src/lifecycle.rs, PythonLifecycle and streams.py are gone.
The adapter trusts the _native.pyi contract with a single cast instead
of a per-request runtime_checkable isinstance. Binding tests assert the
adapter starts the execution and returns its completed value rather
than checking callable identity.
* refactor(rust-bridge): fold lifecycle and public_execution into one execution module
The public stream wrappers only forwarded to the lifecycle streams and added the
header dict, so the streams now carry _hidden_params themselves. lifecycle.py no
longer held any call lifecycle, just the Execution driver, so it is renamed to
execution.py and absorbs the synchronous and asynchronous adapters
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* fix(rust-bridge): keep execution.py free of third-party imports
The host-python tests load execution.py into a bare interpreter, so the
pydantic header validation broke them. The stream head comes from the
binding's encode_stream_head, which always builds a str-keyed dict
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* test(host-python): preserve pre-async hook coverage after execution handoff
* style(rust-bridge): sort litellm_host_python imports
Co-Authored-By: Yujong Lee <yujong@berri.ai>
* test(rust-bridge): drive unstarted native calls through execution in the split lifecycle tests
Co-Authored-By: Yujong Lee <yujong@berri.ai>
* fix(rust-bridge): mark execution TypeVar as Final
Co-Authored-By: Yujong Lee <yujong@berri.ai>
* test(rust-bridge): cover stream close, read failure, and sync suspension in execution
Co-Authored-By: Yujong Lee <yujong@berri.ai>
---------
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* refactor(rust-bridge): drive executions with generator-style send, throw and close
Co-Authored-By: Yujong Lee <yujong@berri.ai>
* refactor(rust-bridge): name stream steps StreamOpen and StreamChunk and reject out-of-phase steps
Co-Authored-By: Yujong Lee <yujong@berri.ai>
* refactor(rust-bridge): drive with ExitStack and require the stream factory
Co-Authored-By: Yujong Lee <yujong@berri.ai>
* test(host-python): give the hook_chain lifecycle binding the stream factory
Co-Authored-By: Yujong Lee <yujong@berri.ai>
---------
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(rust-bridge): pin Messages and Responses callback contracts on the native path
Each test runs the native route and asserts what the Python path does today, so a
known gap is a strict xfail that names its reason and the exception it fails with
Messages and Responses share one Surface and hook recorder in support/callbacks.py,
and each surface folder lists its own expected counts and gaps.
Body-level hooks from the function body (pre-request hook, websearch short-circuit,
Messages interceptors, planned and legacy agentic loops, streaming deployment hook)
are xfail until native runs them
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* test(rust-bridge): give every route folder the same one-concern-per-file layout
Route data, the route Surface and the server fixture move into each route folder, and support/ keeps only
route-neutral helpers. test_inference.py splits into chat_completions/ and responses/, ocr/test_requests.py
splits into requests, failures and settings, and Messages and Responses get streaming and body hook files.
Messages request tests that duplicate inference-messages request.rs are removed, and drop_params read
from litellm.drop_params is now covered
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* test(rust-bridge): expect the required-route decline error for unsupported Responses requests
Co-Authored-By: Yujong Lee <yujong@berri.ai>
* test(rust-bridge): import route data from the route folders in the secrets unit tests
Co-Authored-By: Yujong Lee <yujong@berri.ai>
* test(rust-bridge): import NoReturn in the OCR dispatch tests
Co-Authored-By: Yujong Lee <yujong@berri.ai>
* test(rust-bridge): move route-neutral helpers out of support/callbacks.py
Co-Authored-By: Yujong Lee <yujong@berri.ai>
* test(rust-bridge): cover deployment rejection and concurrent callbacks
* test(rust-bridge): assert native serving via the x-litellm-rust marker in the route folder tests
Co-Authored-By: Yujong Lee <yujong@berri.ai>
---------
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* refactor(rust_bridge): group the Python modules Rust imports by name under rust_bridge/host
Rust reads Python-owned state (legacy callback registries, model capabilities) through
modules it imports by path. Put them in one folder with an AGENTS.md that states the
contract, and update the module path constants on the Rust side.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* refactor(rust_bridge): route Python state reads through host accessors
* docs(rust_bridge): point test_support at the moved host contract test
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* refactor(rust_bridge): move secret manager, transport and timeout accessors under host
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* chore(greptile): register rust_bridge/host AGENTS.md in files.json
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* feat(rust): add the DeepSeek Anthropic Messages config
* refactor(rust): drop the tool discriminator without mutation and use rstest in DeepSeek tests
* feat(rust): add the Vertex AI Anthropic Messages config
* fix(rust): normalize Vertex system turns the way Python does
Vertex uses the shared normalize_system_role_messages: a leading run is
hoisted, a later turn stays or is converted in place by the model's
supports_mid_conversation_system, and billing blocks are filtered after the
hoist so one inside a system-role message no longer reaches Vertex.
* fix(rust_bridge): keep non-Claude Vertex Messages calls on Python
The catalog admits every vertex_ai Messages call, but Rust serves only Claude
there and reports anything else as a terminal InvalidProvider instead of a
decline. The dispatcher now bypasses Rust for those models, as
get_provider_anthropic_messages_config does.
* refactor(rust): template the Vertex location and project errors
The location failure is the shared InvalidType detail and the missing project
is MissingSetting with the provider, setting, param and variable name, so no
sentence is spelled at the return site.
* test(rust): use rstest in the Vertex AI Messages tests
* refactor(rust): derive the missing Vertex project error from its spec
* refactor(rust): give Messages configs typed litellm params
* feat(rust): port _normalize_system_role_messages for Messages hosts
mid_conversation_system mirrors Python's module function for function: a
leading run of system turns is hoisted into the top-level system, a later
turn stays in place when the model map flags supports_mid_conversation_system
and otherwise becomes a user turn where it was, never between a tool_use and
its tool_result, and billing blocks are stripped from the hoisted system. The
capability joins MessagesModelCapabilities and the bridge projection. Azure AI
uses it in place of fold_system_role_messages, which hoisted every turn.
* fix(rust): keep the Bedrock runtime endpoint behind a blank api_base
* refactor(rust): import LitellmParams from its crate instead of a re-export
* refactor(rust): template the Messages request decode errors
Both decode failures go through ErrorDetail::invalid with the subject and the
serde error as source instead of a preformatted sentence.
* refactor(rust_bridge): read the module globals the param specs name
The Messages host no longer keeps its own list of litellm globals; a spec
whose spellings the call leaves out names the global to read, so the next
family that has one needs no host change.
* refactor(rust): give each route lookup Python's implicit None
ProviderConfigManager's per-route methods list only the providers the route
serves and fall through to None; the Rust lookups now do the same, so a new
LlmProviders variant touches only the route that serves it.
* refactor(python-bridge): ship a signature base and read the resolved call
NativeCall carries base (positionals by name plus signature defaults) instead
of the fully bound dict. resolved lays kwargs over base, which is what bound
held, so every pre-hook read and every route host keeps seeing the same values.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* fix(bedrock): build the transcription NativeCall with an empty base
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* test(dispatch): read the resolved call instead of bound
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* refactor(host-python): rename effective to effective_py_args and note the shallow copy
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* refactor(python-bridge): take NativeCall directly and fold routes into per-route folders
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* test(python-bridge): use rstest for updated tests and keep embedding's stub parameter name
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
* refactor(rust): separate Messages settings from capability inputs
* refactor(rust-bridge): unify Messages OCR and Responses call inputs
* refactor(rust-bridge): share NativeCall across inference entrypoints
* fix(rust-bridge): preserve Responses URL aliases and public test inputs
* perf(lens): bound single trace reads by the sampled start time
Co-Authored-By: Ishaan Jaffer <155045088+ishaan-berri@users.noreply.github.com>
* test(lens): allow unused query fixture field in load tests
Co-Authored-By: Ishaan Jaffer <155045088+ishaan-berri@users.noreply.github.com>
* test(lens): pass start_time in every lens content and evidence test
Co-Authored-By: Ishaan Jaffer <155045088+ishaan-berri@users.noreply.github.com>
---------
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: Ishaan Jaffer <155045088+ishaan-berri@users.noreply.github.com>
* fix(lens): price agent traces by joining gen_ai.response.id to spend logs
Trace spend now joins each model call to spend_logs on one key: the
span's response id (gen_ai.response.id or the id the normalizers read
from OpenInference/LangChain output) against spend_logs.response_id or
the upstream id embedded in a managed resp_ id. The litellm.call_id and
traceparent transport join paths and the per-row ownership gate are
removed; the spend SQL still restricts rows to what the reader can see.
A run with some unpriced calls now reports the sum of its priced calls
plus priced_calls, instead of an unknown total.
* test(lens): cover response id spend join and partial trace totals
* chore(lens): regenerate trace types for priced_calls
* feat(lens): show partial run cost as a lower bound with priced call count
* fix(lens): treat litellm.call_id as the same assigned call id for spend joins
The id LiteLLM assigned to a call is either the response id it returned
(gen_ai.response.id -> spend_logs.response_id) or its gateway call id
(litellm.call_id -> spend_logs.litellm_call_id). Both are exact ids the
gateway mints and logs, so the join stays one rule. Transport span
matching and the ownership gate stay removed.
* test(lens): cover litellm.call_id spend joins and restore captured totals
* feat(lens): link each priced model call to its spend log
Spans gain spend_log_request_id, the spend_logs.request_id the call was
priced from, and spend_match, which says whether a model call matched or
why not (no assigned id on the span, no spend log with that id, or an
ambiguous match). A model call span is priced from the same ids as the
run total, so its cost and the total agree.
* test(lens): cover spend log links on model call spans
* chore(lens): regenerate trace types for spend log links
* feat(lens): open the matched spend log from an LLM step
An LLM step's header now shows a Spend log chip with the matched
request id and cost; clicking it opens the request log drawer over the
run, fetched by the exact spend_logs.request_id instead of the span's
own response id. Unpriced steps say why (no assigned id on the span, or
no spend log with it). Tree rows show each model call's cost, and a
partial run cost shows its priced call count inline.
* test(lens): cover the spend log link and unmatched cost reasons
* feat(lens): show the spend log link as a bordered LiteLLM Spend Log button
* feat(lens): add a back link from the spend log drawer to the agent trace
* feat(lens): label the spend log back link Back to Lens trace with the Lens icon
* fix(lens): ignore assigned ids that name no spend log when pricing a call
An id that names no row no longer vetoes the call, so a span carrying
both a response id and a call id still prices from a spend row logged
before litellm_call_id existed. An id naming two or more rows makes the
call ambiguous, and the match reason comes from the same per-id result,
so a single matched row with no cost is reported as matched.
* fix(lens): price a trace only from spend logs in its own team
A reader with several teams could see the same assigned id in another
team's spend log; only rows from the trace's team now price it. The
user and key ownership gate stays removed.
* perf(lens): resolve each model call's spend once per trace
Model call matches are computed once when the trace is resolved and
looked up by span index, instead of scanning the model call list for
every span and walking the graph again for spans, agents and the run
total.
* fix(lens): hide a step's Cost fact only when its spend log link shows the cost
* chore(lens): drop narrative doc comments from the spend join
* fix(lens): price a model call only when its ids agree on one spend log per span
* fix(lens): keep pricing spend logs written before litellm_call_id by their request id
* fix(lens): price every attempt a model call's ids name when they agree
* refactor(rust_bridge): remove rule-gated native secret-manager selection
Mirror the cache treatment: SecretManagerRule/SecretManagerContext and the resolve_native_* binding plumbing are gone. The Rust bridge now selects the native backend from the explicitly configured client (capture_secret_manager / _SecretManagerRuntime.from_client), and get_secret_from_manager is the plain Python handler path.
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* chore(deps): bump sharp to 0.35.5 for GHSA-wq5f-xc86-pv6w
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: Yujong Lee <yujong@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* feat(dev): seed linked tracing and spend fixtures
* chore(dev): use OpenAI model in tracing config
* chore(dev): align tracing credentials with UI E2E
* fix(dev): update fixture seeder query scope
* feat(dev): seed linked tracing and spend fixtures
* chore(dev): use OpenAI model in tracing config
* chore(dev): align tracing credentials with UI E2E
* fix(dev): update fixture seeder query scope
* wip
* wip
* wip
* chore(trace): checkpoint ongoing Rust migration
* refactor(trace): group Python bridge under trace package
* refactor(traces): read span conventions through a Convention trait
Each span format (Claude Code, LangSmith, OpenInference, gen_ai) now lives under
normalize/convention/ as a unit struct implementing Convention, owning both its
detection and its extraction. Precedence is one ordered registry instead of an
if-chain in mod.rs that reached into each module differently.
The modules now share one way to read attributes: present() for the first
non-empty key and Payload for a text that also reports the key it consumed,
replacing three different idioms and the &mut Vec threaded through payload
readers. Instrumentation::adjust returns a new Extraction instead of mutating
one, with each SDK rule as its own function, and the LangChain middleware
suffix list exists once.
* feat(trace): export Rust-owned wire schemas and enforce contract bounds
* fix(trace): bound quoted counts in ClickHouse wire schemas
* feat(trace): generate Python wire contracts with datamodel-code-generator
* test(trace): validate migrated callers and generated contracts at the native boundary
* refactor(traces): rename normalization convention to format
* fix(traces): reconcile spend evidence and preserve unknown costs
* feat(traces): normalize additional telemetry formats
* test(traces): cover captured normalization fixtures
* refactor(traces): isolate SDK normalization rules
* feat(tracing): seed all trace exports for local dashboard
* fix(clickhouse): preserve custom LiteLLM request metadata
* docs(traces): define normalization module boundaries
* docs(traces): define resolution and OTLP boundaries
* fix(ui): normalize nullable trace message names
* refactor(traces): split resolver modules and cover resolution behavior
* test(traces): replace normalization snapshots with behavior assertions
* fix(ui): align dashboard API contracts with generated types
* refactor(traces): type normalization and storage boundaries
* fix(traces): seed captured SDK spend and preserve provider identities
* wip
* test(traces): verify guide discovery and content ordering
---------
Co-authored-by: Yujong Lee <yujong@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* refactor(proxy): extract shared spend log read policy
* test(proxy): use named bindings for spend scope regression
* test(proxy): reuse existing spend log query harness
* test(proxy): cover spend log permission lookup adoption
* chore(proxy): relocate existing spend query baseline
* refactor(proxy): make scope query returns explicit
* refactor(proxy): inject deferred log permission lookup
* test(proxy): cover teamless management compatibility lookup
* refactor(proxy): compose user and team log grants
* refactor(proxy): share generic authorization composition
* refactor(proxy): compose trace read permissions
* refactor(proxy): centralize spend and trace authorization
* refactor(proxy): strengthen spend and trace scope types
* refactor(proxy): flatten log read scope into owned logs
Replace the AnyOf grant tree with a flat OwnedLogs(user_id, team_ids) scope,
and OwnedTraces(logs, api_key_hash) for traces, since every consumer flattened
the tree back into that shape.
A caller with no user id now gets an empty scope instead of matching ownerless
rows through Prisma's IS NULL. The dead request_id guard in ui_view_spend_logs
is removed, and the management facets inject the log team lookup and reuse
read_scope_sql instead of the list shim.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* test(proxy): run spend scope tests through one SQLite emulator
Replace the string-matching payload emulator and the hand-rolled Prisma where
interpreter with one SQLite helper that runs the real scope SQL. Session scope
tests now go through the endpoint, including the no-user caller that must not
match ownerless rows. Drop duplicated lookup-failure and trace mapping cases.
load_permitted_log_team_ids returns no teams without a database instead of
relying on the resolver's broad except.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* fix(proxy): unify log and trace ownership permissions
* test(tracing): align fixtures with ownership read scopes
* refactor(tracing): align query scopes with row ownership
* refactor(spend): make ownership SQL predicates explicit
* test(spend): validate ownership SQL against PostgreSQL
* docs(traces): drop key-row visibility from query help guide
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(spend): reach the empty-memberships branch in team lookup test
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* chore(ui): regenerate dashboard API types
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* wip
* wip
* test(traces): separate root status from diagnostic error counts
* test(traces): cover normalization precedence and fallbacks
* chore(cache): remove stray comments from trace PR
* test(traces): name lens test for shared query path
* fix(traces): place query implementation before test module
* test(traces): use unified read scope in migration tests
* ci(rust): allow feature checks to finish
* ci(mcp): allow dependency resolution to finish
* fix(traces): preserve key visibility and safe spend attribution
* refactor(rust): extract litellm-host-native as the shared Rust host driver
Move service and hook dispatch out of host-http into a Driver that owns the
machine and Rust handlers, returning at completion or a stream boundary and
holding the demand reply until the consumer advances. Move the in-process
runner onto the same driver. host-http now layers encoding, SSE, body polling
and lifecycle observation over it. host-python keeps driving litellm-host
directly
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(rust): interrupt the machine when the in-process stream consumer fails
Restores the pre-refactor interruption path for StreamConsumer errors via
Driver::fail and ports the generic run lifecycle tests into host-native.
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* refactor(rust): separate the machine contract from coroutine execution
* auth update
* refactor(rust): use standard flow control for host requests
* style(rust): keep host driver imports formatted
* chores
* mostly relocation
* refactor(rust): separate interceptors from queued observers
* refactor(rust): centralize legacy callback mappings and lifecycle
* docs: define Python host boundaries and migration plan
* refactor: enforce Python host and bridge boundaries
* refactor(rust): separate operations from callback composition
* refactor(rust): compose SDK policy through call hooks
---------
Co-authored-by: Yujong Lee <yujong@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* refactor(rust): prepare inference and auth foundations
* fix(rust): keep textract operations parsing from kebab-case model names
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* done
* refactor(types): derive Anthropic beta string conversions with Strum
* fix(anthropic): report missing max_tokens as a missing field
* refactor(rust): type Anthropic messages headers and auth after the Python layout
Delete anthropic/messages/headers.rs. Its OAuth handling, credential ladder
and beta merging move to anthropic/common_utils.rs where Python keeps them
(optionally_handle_anthropic_oauth, get_auth_header, _merge_beta_headers),
and the feature beta injection becomes update_headers_with_anthropic_beta on
the messages config, as in Python. The BaseAnthropicMessagesConfig impl is
unchanged apart from the bodies of validate_environment and request_headers
Beta values are now the AnthropicBeta enum and BetaSet, which sort, dedupe
and comma-join by construction. Request params gain typed speed, tools and
context_management through Recognized, so the beta logic matches on enums
instead of string-comparing JSON. OauthToken parses the sk-ant-oat token once
and the chat config shares that detection instead of its own copy
Case-insensitive header helpers move next to has_header in litellm-http.
One deliberate divergence: a Bearer-prefixed OAuth key configured through
api_key or ANTHROPIC_API_KEY is sent with a single Bearer scheme, where
Python would emit "Bearer Bearer"
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* done
* fix(rust): repair test compilation and clippy failures
resolve auth before building the outbound request in prepare tests, give the host hook tests their own error type, and drop the disallowed reqwest client and err().expect() from core tests
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: Yujong Lee <yujong@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
The legacy callback crate carried two rewrites that have nothing to do with the
Logging contract: litellm_credential_name inheritance and the max_budget and
num_retries_per_request checks. Any later callback host would need them
unchanged, which is the smell the crate's AGENTS.md now names. They are now a
Preflight the driver in litellm-host-python runs on the keyword view begin
returned, before the host projects from it, supplied by python-bridge and passed
through run_legacy_call. The call order is unchanged (setup, deployment hook,
credentials, limits) and a rejection still fails the call as a host failure, so
the failure callbacks run as before. The preflight rewrites the adapter's own
copy in place, so no extra dict copy and no new lifecycle method
Co-authored-by: Yujong Lee <yujong@berri.ai>
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
* ci: run the unit_selection.sh shard files on every event instead of only fork pull requests
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* ci: rename fork-flag to unit-flag now that it applies on every event
* test: move tests/test_litellm root and small trees into tests/unit
Pure renames, no content changes. Follow-up commits in this PR fix
references, merge the three files that already existed in tests/unit,
keep live-provider tests in tests/test_litellm and wire CI.
* test: carry tests/test_litellm conftest isolation into tests/unit
Callback lists, routing fallbacks, cached HTTP clients, logger state, AWS,
proxy-URL and keychain env, and session-end client cleanup now reset for
unit tests too. The environment isolation owns its MonkeyPatch so a test's
own monkeypatch is undone before the model-cost teardown runs.
* test: merge, split and prune the moved root and small-tree tests
Merge batches/test_batch_utils.py and the chat_completions and messages
dispatch tests into the files that already existed in tests/unit. Keep
the live Gemini interactions tests, the async image-fetch format test and
the OpenAI embedding scorer test in tests/test_litellm since they need
real network or keys. Put test_router.py under tests/unit/test_router so
the existing package no longer shadows it. Delete eight tests the audit
found superseded by stronger ones kept in this move.
* ci: run the moved root and small-tree tests under their legacy flags
Add the misc and responses-caching-types flags to unit_selection.sh and
CircleCI, extend enterprise-routing and mcp-integration, and point the
legacy GHA shards, Makefile, redis-compat workflow, merge smoke manifest
and change classifier at the new paths.
* test: make the new tests/unit directories packages
tests/unit/test_package_layout.py requires every directory to carry an
__init__.py, and without one the moved and retained
test_litellm_responses_bridge.py modules collide on import.
* test: scope the unit socket block to tests/unit in shared sessions
The GHA shards collect the legacy test-path and the unit selection in one
pytest session. The unit conftest's loopback-only block leaked into legacy
modules that reach the network at import. The legacy conftest now lifts the
restriction at collect and setup time, and the unit conftest re-applies it
when collecting its own modules.
* test: move tests/test_litellm/llms into tests/unit/llms
Rename-only. Moves the provider tests and the fine-tuning fixtures they
load, mirroring the old paths. Follow-up commits merge, split and wire them.
* test: merge, split and prune the moved llms tests
Merges the Databricks chat transformation tests into the existing unit
file, keeps the tests that need real keys or the network in
tests/test_litellm, deletes the audited tests a stronger unit test
already covers, and points imports at tests.unit.llms.
* ci: run the moved llms tests under their legacy flags
The Vertex AI and All Other Providers shards keep their legacy test-path
for the retained files and add the llm-vertex-ai and llm-other-providers
unit selections. CircleCI gets matching unit jobs.
* test: make the tests/unit/llms directories packages
Adds __init__.py to the moved dirs and drops the legacy ones whose
directories no longer hold tests.
* test: drop script runners and path hacks the llms split left dangling
The __main__ runners in the split openai_like files and the Databricks e2e
runner called tests that now live in the other half of the split or were
deleted. The retained legacy halves also no longer need sys.path edits.
* test: give the shard-script tests their own GITHUB_OUTPUT
They only passed where the runner set it. The CircleCI unit job's env
allowlist drops it, so the script's redirect failed there.
* test: point the router and module-deletion checks at tests/unit
router_code_coverage and code_qa_check_tests only searched tests/test_litellm,
so the moved router tests no longer counted. The two silent-experiment tests
the audit deleted were the only direct callers of those methods; they are
replaced with tests that assert the forwarded shadow request and the
recursion guard.
* test: move tests/test_litellm integrations and secret_managers into tests/unit
Rename-only. Mirrors the old paths, including the directory conftests
and the prompt and JSON fixtures. Follow-up commits prune and wire them.
* test: prune and repoint the moved integrations tests
Deletes the 7 audited tests a stronger test in the same tree already
covers, imports the TLS sink helpers from their new conftest path, and
restores os.environ after each integrations test. Some presets write
OTEL_EXPORTER_OTLP_HEADERS straight into os.environ, and without the
legacy tree's test ordering that header leaked into the AgentOps tests.
* ci: run the moved integrations tests under their legacy flag
The integrations GHA shard and a new CircleCI job run the integrations
unit selection. secret_managers joins the misc selection.
* docs: point integrations and secret_managers references at tests/unit
* test: make the moved integrations directories packages
* test: keep the Databricks manual e2e runner and fix the SageMaker Nova run path
The Databricks e2e file is a manual script whose main() calls the tests
that were pruned, so pruning them broke the documented run. It is back to
its main version. The SageMaker Nova docstring now points at the file's
real location in tests/local_testing.
* test: move tests/test_litellm core utils, routing, responses, caching and rust_bridge into tests/unit
Rename-only. Mirrors the old paths, including fixtures, the stubtest config
and the native-route wheel script. Two files that collide with existing unit
files are merged in a follow-up commit.
* test: merge, prune and repoint the moved core, routing, responses, caching and rust_bridge tests
Merges the two files that collided with existing unit files, folding the
legacy extra case into test_is_chat_completion_cached_dict, and deletes the
9 audited tests a stronger test in the same file already covers.
Keeps what needs the network in tests/test_litellm: test_tokenizers pulls a
tokenizer from the Hugging Face hub, and the gpt2 and r50k_base tokenizer
cases download their BPE files. The unit core_utils conftest points
TIKTOKEN_CACHE_DIR at litellm's bundled encodings so the rest never depend on
import order to stay offline, and FakeSecretVault moves to a shared module
so both trees can build it.
* ci: run the moved core, routing, responses, caching and rust_bridge tests under their flags
core_utils gets a core-utils flag and CircleCI job, and its GHA shard keeps
the legacy path for the retained network tests. router_utils and
router_strategy join enterprise-routing, responses joins
responses-caching-types (minus responses/mcp, which mcp-integration owns),
caching joins caching-local and rust_bridge joins misc. The redis-compat,
test-rust, stubtest and merge-smoke paths follow the move.
* docs: point the Rust crate references at tests/unit
* test: make the moved core, routing and rust_bridge directories packages
* test: keep the no-loop DualCache batch_get_cache regression test
It runs the sync path outside any event loop, which the inside-loop test
cannot, so a change that picks the Redis client by loop state would only
show up there.
* test: keep the job's UNIT_FLAG out of the shard-script tests
* fix(url_utils): block 192.0.0.0/24 on every Python patch release
* test: move the new budget limiter tests into tests/unit/router_strategy
* test: move the new sentry scrubbing tests into tests/unit/litellm_core_utils
* test: move the new zerobus tests into tests/unit/integrations
* test: make tests/unit/integrations/zerobus a package
* test: load litellm's own tiktoken cache setup once instead of resetting it per test
---------
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
