* ci: run the unit_selection.sh shard files on every event instead of only fork pull requests
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* ci: rename fork-flag to unit-flag now that it applies on every event
* test: move tests/test_litellm root and small trees into tests/unit
Pure renames, no content changes. Follow-up commits in this PR fix
references, merge the three files that already existed in tests/unit,
keep live-provider tests in tests/test_litellm and wire CI.
* test: carry tests/test_litellm conftest isolation into tests/unit
Callback lists, routing fallbacks, cached HTTP clients, logger state, AWS,
proxy-URL and keychain env, and session-end client cleanup now reset for
unit tests too. The environment isolation owns its MonkeyPatch so a test's
own monkeypatch is undone before the model-cost teardown runs.
* test: merge, split and prune the moved root and small-tree tests
Merge batches/test_batch_utils.py and the chat_completions and messages
dispatch tests into the files that already existed in tests/unit. Keep
the live Gemini interactions tests, the async image-fetch format test and
the OpenAI embedding scorer test in tests/test_litellm since they need
real network or keys. Put test_router.py under tests/unit/test_router so
the existing package no longer shadows it. Delete eight tests the audit
found superseded by stronger ones kept in this move.
* ci: run the moved root and small-tree tests under their legacy flags
Add the misc and responses-caching-types flags to unit_selection.sh and
CircleCI, extend enterprise-routing and mcp-integration, and point the
legacy GHA shards, Makefile, redis-compat workflow, merge smoke manifest
and change classifier at the new paths.
* test: make the new tests/unit directories packages
tests/unit/test_package_layout.py requires every directory to carry an
__init__.py, and without one the moved and retained
test_litellm_responses_bridge.py modules collide on import.
* test: scope the unit socket block to tests/unit in shared sessions
The GHA shards collect the legacy test-path and the unit selection in one
pytest session. The unit conftest's loopback-only block leaked into legacy
modules that reach the network at import. The legacy conftest now lifts the
restriction at collect and setup time, and the unit conftest re-applies it
when collecting its own modules.
* test: move tests/test_litellm/llms into tests/unit/llms
Rename-only. Moves the provider tests and the fine-tuning fixtures they
load, mirroring the old paths. Follow-up commits merge, split and wire them.
* test: merge, split and prune the moved llms tests
Merges the Databricks chat transformation tests into the existing unit
file, keeps the tests that need real keys or the network in
tests/test_litellm, deletes the audited tests a stronger unit test
already covers, and points imports at tests.unit.llms.
* ci: run the moved llms tests under their legacy flags
The Vertex AI and All Other Providers shards keep their legacy test-path
for the retained files and add the llm-vertex-ai and llm-other-providers
unit selections. CircleCI gets matching unit jobs.
* test: make the tests/unit/llms directories packages
Adds __init__.py to the moved dirs and drops the legacy ones whose
directories no longer hold tests.
* test: drop script runners and path hacks the llms split left dangling
The __main__ runners in the split openai_like files and the Databricks e2e
runner called tests that now live in the other half of the split or were
deleted. The retained legacy halves also no longer need sys.path edits.
* test: give the shard-script tests their own GITHUB_OUTPUT
They only passed where the runner set it. The CircleCI unit job's env
allowlist drops it, so the script's redirect failed there.
* test: point the router and module-deletion checks at tests/unit
router_code_coverage and code_qa_check_tests only searched tests/test_litellm,
so the moved router tests no longer counted. The two silent-experiment tests
the audit deleted were the only direct callers of those methods; they are
replaced with tests that assert the forwarded shadow request and the
recursion guard.
* test: move tests/test_litellm integrations and secret_managers into tests/unit
Rename-only. Mirrors the old paths, including the directory conftests
and the prompt and JSON fixtures. Follow-up commits prune and wire them.
* test: prune and repoint the moved integrations tests
Deletes the 7 audited tests a stronger test in the same tree already
covers, imports the TLS sink helpers from their new conftest path, and
restores os.environ after each integrations test. Some presets write
OTEL_EXPORTER_OTLP_HEADERS straight into os.environ, and without the
legacy tree's test ordering that header leaked into the AgentOps tests.
* ci: run the moved integrations tests under their legacy flag
The integrations GHA shard and a new CircleCI job run the integrations
unit selection. secret_managers joins the misc selection.
* docs: point integrations and secret_managers references at tests/unit
* test: make the moved integrations directories packages
* test: keep the Databricks manual e2e runner and fix the SageMaker Nova run path
The Databricks e2e file is a manual script whose main() calls the tests
that were pruned, so pruning them broke the documented run. It is back to
its main version. The SageMaker Nova docstring now points at the file's
real location in tests/local_testing.
* test: move tests/test_litellm core utils, routing, responses, caching and rust_bridge into tests/unit
Rename-only. Mirrors the old paths, including fixtures, the stubtest config
and the native-route wheel script. Two files that collide with existing unit
files are merged in a follow-up commit.
* test: merge, prune and repoint the moved core, routing, responses, caching and rust_bridge tests
Merges the two files that collided with existing unit files, folding the
legacy extra case into test_is_chat_completion_cached_dict, and deletes the
9 audited tests a stronger test in the same file already covers.
Keeps what needs the network in tests/test_litellm: test_tokenizers pulls a
tokenizer from the Hugging Face hub, and the gpt2 and r50k_base tokenizer
cases download their BPE files. The unit core_utils conftest points
TIKTOKEN_CACHE_DIR at litellm's bundled encodings so the rest never depend on
import order to stay offline, and FakeSecretVault moves to a shared module
so both trees can build it.
* ci: run the moved core, routing, responses, caching and rust_bridge tests under their flags
core_utils gets a core-utils flag and CircleCI job, and its GHA shard keeps
the legacy path for the retained network tests. router_utils and
router_strategy join enterprise-routing, responses joins
responses-caching-types (minus responses/mcp, which mcp-integration owns),
caching joins caching-local and rust_bridge joins misc. The redis-compat,
test-rust, stubtest and merge-smoke paths follow the move.
* docs: point the Rust crate references at tests/unit
* test: make the moved core, routing and rust_bridge directories packages
* test: keep the no-loop DualCache batch_get_cache regression test
It runs the sync path outside any event loop, which the inside-loop test
cannot, so a change that picks the Redis client by loop state would only
show up there.
* test: keep the job's UNIT_FLAG out of the shard-script tests
* fix(url_utils): block 192.0.0.0/24 on every Python patch release
* test: move the new budget limiter tests into tests/unit/router_strategy
* test: move the new sentry scrubbing tests into tests/unit/litellm_core_utils
* test: move the new zerobus tests into tests/unit/integrations
* test: make tests/unit/integrations/zerobus a package
* test: load litellm's own tiktoken cache setup once instead of resetting it per test
---------
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(router): parse the classifier verdict out of prose and fences on every parse path
The complexity router's classifier parsers only tolerated a bare JSON object
(labeled tier) or a leading Markdown fence (capability, LLM V2), so a
json_object classifier that writes its verdict fenced and then explains it in
markdown, which Bedrock Haiku 4.5 does on nearly every Claude Code request,
failed validation and every request fell to the fallback tier.
All three parse paths now extract the first complete JSON object from the
reply with json.JSONDecoder.raw_decode, whatever prose or fence surrounds it,
and a reply that still fails validation is logged with the pydantic field
problems and the raw reply text, withheld when the request turns off message
logging. The capability failure reason names the exception type like the
labeled path does instead of interpolating str(e), which for a ValidationError
carried the whole reply as input_value and for TimeoutError was empty.
* fix(router): stop rejecting an LLM V2 forecast over a long explanation field
LLMV2Verdict capped crux and each forecast's likely_failure at 512 characters
through the ShortText alias, so a verdict whose explanation ran long failed
validation and the request fell to the capable tier, even though nothing
downstream reads either field. Five of nineteen real Claude Code replies from
Bedrock Haiku 4.5 tripped the cap. Both fields keep the strip and non-empty
constraints and lose the length cap; the operator-set calibration version keeps
ShortText.
* fix(complexity_router): withhold the rejected classifier reply under every message-logging opt-out and survive undecodable replies
* fix(complexity_router): withhold the rejected classifier reply when the redaction decision cannot be made
---------
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
* fix(logging): redact raw_request when turn_off_message_logging is set in the proxy config
The raw request branch bound turn_off_message_logging by name at import, before the proxy config set it, so loggers kept receiving the prompt in metadata.raw_request and raw_request_typed_dict. It now runs the same per-request redaction check messages use, and json_logs is read at call time for the same reason
* fix(logging): keep raw_request_typed_dict for the explicit readers and tolerate missing headers in the json debug log
* refactor(logging): drop the stale comment above the raw request typed dict
---------
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
* fix(proxy): send a real error event when a /v1/messages stream fails
When a stream failed halfway through, the proxy wrote the error as a plain
`data: {"error": ...}` line with no `event:` in front of it. Anthropic clients
pick stream events by that name, so they skip the line and the request looks
like it simply stopped with nothing in it.
Write the failure as an `event: error` frame with Anthropic's own payload, and
take the error type from the status code
* fix(proxy): use the shared Anthropic error mapping for the stream error frame
The first pass added a third copy of the status to error-type table, and it
disagreed with the documented one: 529 came out as `api_error` rather than
`overloaded_error`, and 413 as `invalid_request_error` rather than
`request_too_large`, which hides the two failures a client can actually act on.
Drop that copy and put the frame builder next to the table litellm already
keeps in anthropic_interface/exceptions. The bridged adapter path was building
the same frame inline, so it uses the shared one now too
* fix(proxy): seal a torn SSE frame before the /v1/messages error event
An upstream that drops mid-frame leaves the client inside an open event,
so the error frame that follows is glued onto the torn data line and the
Anthropic SDK raises a JSON decode error instead of an APIStatusError.
Close the open frame with a ping event the SDK skips before writing the
error event, and add the e2e stream-cut edge with Bedrock, Anthropic
boundary, and Anthropic mid-frame legs.
* fix(proxy): keep the SSE tail unchanged on a chunk that is not text
A serializer that hands a dict or model object through as-is has no bytes
the frame tail can learn from, so advance_sse_tail leaves it alone instead
of slicing it.
* fix(proxy): answer a /v1/messages stream that fails before its first byte as a JSON error carrying its status
* test: move the Anthropic error frame tests into tests/unit
* test(e2e): cut the upstream stream only after content has been relayed
* test(e2e): carry split SSE lines across chunks and always tear a data line mid-frame
* test(e2e): find the next data line across a chunk boundary before tearing it
---------
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
* feat(xai): add native xAI batches and files support
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(xai): tighten batch handler typing and avoid Final redeclaration on star import
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* refactor(xai): walk batch result pages iteratively
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(xai): stop paging on empty pagination token and honor litellm.xai_key
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(xai): import NotRequired and TypedDict from typing_extensions for Python 3.10
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* feat(batches): accept image and video endpoints on batch create
* test(xai): lock batch endpoint, auth, and result contracts
The batches test package collided with litellm/batches under pytest prepend, so the All Other Providers shard could not collect the new tests.
* fix(xai): price grok batch usage at xAI's 20 percent batch discount
* fix(xai): map not-found file reads to 404, bill batch reasoning tokens, and add 200k batch tier rates
* refactor(xai): drop routine prose and move tests under tests/unit
* fix(health): hand the resolved provider to list_batches in batch-mode health checks
* test(xai): make tests/unit/llms/xai/batches a package
* fix(xai): walk every page of the files list by pagination_token
---------
Co-authored-by: Mubashir Osmani <mubashir@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: mubashir1osmani <mubashir.osmani777@gmail.com>
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
* fix(vertex_ai): map legacy bucket_name to gcs_bucket_name and add GCS_BATCH_BUCKET_NAME env var
* refactor(router): keep legacy bucket_name as a credential field instead of a validator
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(vertex_ai): pass the RAG corpus bucket to the file upload instead of hopping through GCS_BUCKET_NAME
* fix(vertex_ai): accept existing_file_id in the RAG Engine store step so ingest() runs end to end
---------
Co-authored-by: Mubashir Osmani <mubashir@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
* feat(integrations): add Databricks Zerobus trace logging callback
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(zerobus): escape regex in pytest.raises match
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(zerobus): use unique test module basenames
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(zerobus): hold the queue cap while an insert is in flight
Rows arriving during a slow insert are dropped once the queue is at
max_queue_size, since trimming the head would corrupt the in-flight batch.
Test fakes are typed and record calls as frozen dataclasses; the
litellm_logging init and reuse branches are covered.
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(zerobus): type the row payload and dashboard config helpers
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(zerobus): assert the trace row survives a JSON round trip
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(zerobus): keep the client secret and access token out of dataclass reprs
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: yassin <yassin@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(jwt): let x-litellm-team-id select DB membership teams when the token also carries a team claim
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* docs(jwt): describe header team selection under fallback_to_db_teams
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: yassin <yassin@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(bedrock): surface a converse-stream 200 that decodes to no events as a 502 instead of an empty turn
* fix(bedrock): quote the body head only when a stream decoded no events
The leftover-bytes error keeps the byte and event counts, the content type and the request id but no longer quotes the first bytes of a stream that already decoded events, since that head is the start of a healthy stream and can hold model output. The anthropic_messages empty-stream warning no longer prints the request's model name.
---------
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
* fix(sentry): scrub PII and secrets inside object reprs and nested locals, add SENTRY_SEND_DEFAULT_PII opt-in
* fix(sentry): keep the SDK denylist and filter the request headers a virtual key arrives in
* fix(sentry): leave source context lines unscrubbed
* fix(sentry): filter bracketed secret values and cap the JSON walk depth
* ci(deps): install sentry-sdk in the proxy-dev group so the unit shards import it
* fix(sentry): scrub source-context names outside real stack frames
* fix(sentry): tie the key pattern floor to the custom key minimum
* fix(sentry): keep the key pattern floor at or below a generated key's length
---------
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
* fix(responses): fall back on pre-output stream drops, fail truncated streams, honor request_timeout
A native /v1/responses stream that drops before any output item now raises
the router's fallback-eligible MidStreamFallbackError, so configured
fallbacks retry the original input. A stream that ends with a clean EOF or
a [DONE] marker but no response.completed, response.incomplete or
response.failed event now raises litellm.APIConnectionError instead of
ending as if it had completed: fallback-eligible before any output, an
explicit error after partial output. The sync iterator mirrors every branch.
resolve_llm_passthrough_timeout now consults an explicitly set
litellm_settings.request_timeout right after the router timeout and before
general_settings.pass_through_request_timeout, so the router's native
responses path honors it.
* test(responses): give the normal-completion stream tests a terminal event
---------
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
* fix(router): count provider budget spend on every API surface
RouterBudgetLimiting read custom_llm_provider from litellm_params, which only
chat completions populates. Responses, anthropic_messages, embedding and rerank
calls raised inside the success callback before any spend was recorded, so those
budgets never moved and a ceiling made up mostly of that traffic was never hit.
Read the provider from the standard logging payload, which every surface fills
in. Dropping the raise also stops one missing field from taking the deployment
and tag budgets down with it.
* chore(router): drop the inline comment and type the budget limiter test helper
---------
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
* test(e2e): cover Azure code_interpreter container files by native id with a service-account key
* test(e2e): require the code_interpreter tool, skip at collection, and scope the container call timeout
* fix(e2e): fail the containers suite when the Azure credentials are missing instead of skipping
* fix(proxy): record streamed /v1/responses container ownership before the response.completed frame
---------
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
* ci: run the unit_selection.sh shard files on every event instead of only fork pull requests
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* ci: rename fork-flag to unit-flag now that it applies on every event
* test: move tests/test_litellm root and small trees into tests/unit
Pure renames, no content changes. Follow-up commits in this PR fix
references, merge the three files that already existed in tests/unit,
keep live-provider tests in tests/test_litellm and wire CI.
* test: carry tests/test_litellm conftest isolation into tests/unit
Callback lists, routing fallbacks, cached HTTP clients, logger state, AWS,
proxy-URL and keychain env, and session-end client cleanup now reset for
unit tests too. The environment isolation owns its MonkeyPatch so a test's
own monkeypatch is undone before the model-cost teardown runs.
* test: merge, split and prune the moved root and small-tree tests
Merge batches/test_batch_utils.py and the chat_completions and messages
dispatch tests into the files that already existed in tests/unit. Keep
the live Gemini interactions tests, the async image-fetch format test and
the OpenAI embedding scorer test in tests/test_litellm since they need
real network or keys. Put test_router.py under tests/unit/test_router so
the existing package no longer shadows it. Delete eight tests the audit
found superseded by stronger ones kept in this move.
* ci: run the moved root and small-tree tests under their legacy flags
Add the misc and responses-caching-types flags to unit_selection.sh and
CircleCI, extend enterprise-routing and mcp-integration, and point the
legacy GHA shards, Makefile, redis-compat workflow, merge smoke manifest
and change classifier at the new paths.
* test: make the new tests/unit directories packages
tests/unit/test_package_layout.py requires every directory to carry an
__init__.py, and without one the moved and retained
test_litellm_responses_bridge.py modules collide on import.
* test: scope the unit socket block to tests/unit in shared sessions
The GHA shards collect the legacy test-path and the unit selection in one
pytest session. The unit conftest's loopback-only block leaked into legacy
modules that reach the network at import. The legacy conftest now lifts the
restriction at collect and setup time, and the unit conftest re-applies it
when collecting its own modules.
* test: move tests/test_litellm/llms into tests/unit/llms
Rename-only. Moves the provider tests and the fine-tuning fixtures they
load, mirroring the old paths. Follow-up commits merge, split and wire them.
* test: merge, split and prune the moved llms tests
Merges the Databricks chat transformation tests into the existing unit
file, keeps the tests that need real keys or the network in
tests/test_litellm, deletes the audited tests a stronger unit test
already covers, and points imports at tests.unit.llms.
* ci: run the moved llms tests under their legacy flags
The Vertex AI and All Other Providers shards keep their legacy test-path
for the retained files and add the llm-vertex-ai and llm-other-providers
unit selections. CircleCI gets matching unit jobs.
* test: make the tests/unit/llms directories packages
Adds __init__.py to the moved dirs and drops the legacy ones whose
directories no longer hold tests.
* test: drop script runners and path hacks the llms split left dangling
The __main__ runners in the split openai_like files and the Databricks e2e
runner called tests that now live in the other half of the split or were
deleted. The retained legacy halves also no longer need sys.path edits.
* test: give the shard-script tests their own GITHUB_OUTPUT
They only passed where the runner set it. The CircleCI unit job's env
allowlist drops it, so the script's redirect failed there.
* test: point the router and module-deletion checks at tests/unit
router_code_coverage and code_qa_check_tests only searched tests/test_litellm,
so the moved router tests no longer counted. The two silent-experiment tests
the audit deleted were the only direct callers of those methods; they are
replaced with tests that assert the forwarded shadow request and the
recursion guard.
* test: move tests/test_litellm integrations and secret_managers into tests/unit
Rename-only. Mirrors the old paths, including the directory conftests
and the prompt and JSON fixtures. Follow-up commits prune and wire them.
* test: prune and repoint the moved integrations tests
Deletes the 7 audited tests a stronger test in the same tree already
covers, imports the TLS sink helpers from their new conftest path, and
restores os.environ after each integrations test. Some presets write
OTEL_EXPORTER_OTLP_HEADERS straight into os.environ, and without the
legacy tree's test ordering that header leaked into the AgentOps tests.
* ci: run the moved integrations tests under their legacy flag
The integrations GHA shard and a new CircleCI job run the integrations
unit selection. secret_managers joins the misc selection.
* docs: point integrations and secret_managers references at tests/unit
* test: make the moved integrations directories packages
* test: keep the Databricks manual e2e runner and fix the SageMaker Nova run path
The Databricks e2e file is a manual script whose main() calls the tests
that were pruned, so pruning them broke the documented run. It is back to
its main version. The SageMaker Nova docstring now points at the file's
real location in tests/local_testing.
* test: keep the job's UNIT_FLAG out of the shard-script tests
---------
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* ci: run the unit_selection.sh shard files on every event instead of only fork pull requests
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* ci: rename fork-flag to unit-flag now that it applies on every event
* test: move tests/test_litellm root and small trees into tests/unit
Pure renames, no content changes. Follow-up commits in this PR fix
references, merge the three files that already existed in tests/unit,
keep live-provider tests in tests/test_litellm and wire CI.
* test: carry tests/test_litellm conftest isolation into tests/unit
Callback lists, routing fallbacks, cached HTTP clients, logger state, AWS,
proxy-URL and keychain env, and session-end client cleanup now reset for
unit tests too. The environment isolation owns its MonkeyPatch so a test's
own monkeypatch is undone before the model-cost teardown runs.
* test: merge, split and prune the moved root and small-tree tests
Merge batches/test_batch_utils.py and the chat_completions and messages
dispatch tests into the files that already existed in tests/unit. Keep
the live Gemini interactions tests, the async image-fetch format test and
the OpenAI embedding scorer test in tests/test_litellm since they need
real network or keys. Put test_router.py under tests/unit/test_router so
the existing package no longer shadows it. Delete eight tests the audit
found superseded by stronger ones kept in this move.
* ci: run the moved root and small-tree tests under their legacy flags
Add the misc and responses-caching-types flags to unit_selection.sh and
CircleCI, extend enterprise-routing and mcp-integration, and point the
legacy GHA shards, Makefile, redis-compat workflow, merge smoke manifest
and change classifier at the new paths.
* test: make the new tests/unit directories packages
tests/unit/test_package_layout.py requires every directory to carry an
__init__.py, and without one the moved and retained
test_litellm_responses_bridge.py modules collide on import.
* test: scope the unit socket block to tests/unit in shared sessions
The GHA shards collect the legacy test-path and the unit selection in one
pytest session. The unit conftest's loopback-only block leaked into legacy
modules that reach the network at import. The legacy conftest now lifts the
restriction at collect and setup time, and the unit conftest re-applies it
when collecting its own modules.
* test: move tests/test_litellm/llms into tests/unit/llms
Rename-only. Moves the provider tests and the fine-tuning fixtures they
load, mirroring the old paths. Follow-up commits merge, split and wire them.
* test: merge, split and prune the moved llms tests
Merges the Databricks chat transformation tests into the existing unit
file, keeps the tests that need real keys or the network in
tests/test_litellm, deletes the audited tests a stronger unit test
already covers, and points imports at tests.unit.llms.
* ci: run the moved llms tests under their legacy flags
The Vertex AI and All Other Providers shards keep their legacy test-path
for the retained files and add the llm-vertex-ai and llm-other-providers
unit selections. CircleCI gets matching unit jobs.
* test: make the tests/unit/llms directories packages
Adds __init__.py to the moved dirs and drops the legacy ones whose
directories no longer hold tests.
* test: drop script runners and path hacks the llms split left dangling
The __main__ runners in the split openai_like files and the Databricks e2e
runner called tests that now live in the other half of the split or were
deleted. The retained legacy halves also no longer need sys.path edits.
* test: give the shard-script tests their own GITHUB_OUTPUT
They only passed where the runner set it. The CircleCI unit job's env
allowlist drops it, so the script's redirect failed there.
* test: point the router and module-deletion checks at tests/unit
router_code_coverage and code_qa_check_tests only searched tests/test_litellm,
so the moved router tests no longer counted. The two silent-experiment tests
the audit deleted were the only direct callers of those methods; they are
replaced with tests that assert the forwarded shadow request and the
recursion guard.
* test: keep the Databricks manual e2e runner and fix the SageMaker Nova run path
The Databricks e2e file is a manual script whose main() calls the tests
that were pruned, so pruning them broke the documented run. It is back to
its main version. The SageMaker Nova docstring now points at the file's
real location in tests/local_testing.
* test: keep the job's UNIT_FLAG out of the shard-script tests
---------
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(anthropic): surface Responses bridge stream failures as Anthropic error events
The /v1/messages Responses bridge logged every upstream failure and ended the
SSE stream as if it had completed, so a rate limit, a provider 500, a dropped
connection, or a read timeout reached the client as HTTP 200 with a lone
message_start and no error event. Map response.failed and any raised upstream
exception to a redacted Anthropic error frame, stop pulling upstream after it,
and never fabricate end_turn or message_stop after a failure.
* fix(anthropic): close a Responses bridge stream that ends without a terminal event with an error event
Normalize the failure status behind the error type to an int or digit string
within 400..599, narrow the response.failed event through pydantic, reuse the
native Messages path's incomplete-stream message for a clean upstream EOF, and
cover the pydantic event, the unwrapped fallback error, and the EOF cases
---------
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
* refactor(framer): replace Framer trait with tokio-util codecs
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* refactor(framer): port SSE and AWS event stream framing to codecs
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(rust): drop clone on Copy capabilities in messages request test
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(rust): use field init shorthand in messages request test
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: Yujong Lee <yujong@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
/sso/debug/login and /sso/debug/callback are diagnostic pages that had
no off switch. They cannot carry a bearer credential because the IdP
redirects a bare browser to the callback, so the gate is an explicit
opt-in flag rather than key auth: both routes return 404 unless
ENABLE_SSO_DEBUG is set to a truthy value.
* ci: run the unit_selection.sh shard files on every event instead of only fork pull requests
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* ci: rename fork-flag to unit-flag now that it applies on every event
* test: move tests/test_litellm root and small trees into tests/unit
Pure renames, no content changes. Follow-up commits in this PR fix
references, merge the three files that already existed in tests/unit,
keep live-provider tests in tests/test_litellm and wire CI.
* test: carry tests/test_litellm conftest isolation into tests/unit
Callback lists, routing fallbacks, cached HTTP clients, logger state, AWS,
proxy-URL and keychain env, and session-end client cleanup now reset for
unit tests too. The environment isolation owns its MonkeyPatch so a test's
own monkeypatch is undone before the model-cost teardown runs.
* test: merge, split and prune the moved root and small-tree tests
Merge batches/test_batch_utils.py and the chat_completions and messages
dispatch tests into the files that already existed in tests/unit. Keep
the live Gemini interactions tests, the async image-fetch format test and
the OpenAI embedding scorer test in tests/test_litellm since they need
real network or keys. Put test_router.py under tests/unit/test_router so
the existing package no longer shadows it. Delete eight tests the audit
found superseded by stronger ones kept in this move.
* ci: run the moved root and small-tree tests under their legacy flags
Add the misc and responses-caching-types flags to unit_selection.sh and
CircleCI, extend enterprise-routing and mcp-integration, and point the
legacy GHA shards, Makefile, redis-compat workflow, merge smoke manifest
and change classifier at the new paths.
* test: make the new tests/unit directories packages
tests/unit/test_package_layout.py requires every directory to carry an
__init__.py, and without one the moved and retained
test_litellm_responses_bridge.py modules collide on import.
* test: scope the unit socket block to tests/unit in shared sessions
The GHA shards collect the legacy test-path and the unit selection in one
pytest session. The unit conftest's loopback-only block leaked into legacy
modules that reach the network at import. The legacy conftest now lifts the
restriction at collect and setup time, and the unit conftest re-applies it
when collecting its own modules.
* test: give the shard-script tests their own GITHUB_OUTPUT
They only passed where the runner set it. The CircleCI unit job's env
allowlist drops it, so the script's redirect failed there.
* test: point the router and module-deletion checks at tests/unit
router_code_coverage and code_qa_check_tests only searched tests/test_litellm,
so the moved router tests no longer counted. The two silent-experiment tests
the audit deleted were the only direct callers of those methods; they are
replaced with tests that assert the forwarded shadow request and the
recursion guard.
---------
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* ci: run the unit_selection.sh shard files on every event instead of only fork pull requests
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* ci: rename fork-flag to unit-flag now that it applies on every event
---------
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Once the preview budget is crossed and the report is dispatched, an upstream
httpx.HTTPError is re-raised so the client still sees the truncated framing
instead of a clean terminator. Report tasks now live in a module registry
behind a semaphore and are drained with a timeout during proxy shutdown, and
a client disconnect before upstream completion logs the preview byte count
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* feat(testkit): install and configure Claude Code, Codex and opencode against a gateway
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
* refactor(testkit): derive targets from target-lexicon and split agents behind a trait
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
* refactor(testkit): group sources into agent and install folders
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
* feat(testkit): split agents into install, configure and drive with semver-aware launch
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
* style(testkit): drop a needless borrow
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
---------
Co-authored-by: Yujong Lee <yujong@berri.ai>
Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
