mirror of
https://github.com/BerriAI/litellm.git
synced 2026-10-25 05:54:53 +00:00
16 commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
yuneng-jiang
|
77c2ac18bd
|
test(caching): assert sync cache-hit logging without depending on thread order (#47093)
Sync success callbacks run on a thread pool, so the miss and the two hits can reach the logger in any order. Compare counts instead of the list order. |
||
|
devin-ai-integration[bot]
|
8226b2be9a
|
fix(otel): report cache hits at zero cost with a cache-hit flag in V1 and V2 spans (#43604)
* fix(litellm): report zero cost and cache-hit attrs on cache-served spans
* fix(litellm): stamp cache-hit marker on HiddenParams carriers and restore probe side effects
gauntlet-mini on
|
||
|
yuneng-jiang
|
a4e0570bea
|
fix(logging): keep the async request's logging payload when the sync success thread runs late (#47019)
* fix(logging): keep the async request's logging payload when sync callbacks run late Treat the "cache" success callback as internal when deciding whether to start the sync success thread for async requests. It has no branch in the sync handler, so with `cache: true` that thread did nothing except rebuild and overwrite model_call_details["standard_logging_object"]. When the sync handler does run for an async request (a real sync-only logger is configured), it now builds its own payload and hands it only to its own callbacks. The async handler keeps sole ownership of the shared payload, so edits made by async logging hooks (for example the v3 rate limiter's mirrored x-ratelimit headers that Prometheus reads) are no longer discarded. Sync requests keep their existing behavior. * fix(logging): keep sync cache hits logged and isolate the async-request payload build The sync completion cache-hit path reused handle_sync_success_callbacks_for_async_calls, so treating "cache" as internal silently dropped sync callbacks on sync cache hits. It now submits the sync handler through its own method, as it effectively always did. For async requests the sync handler builds its payload from a copy of the call details, so message redaction cannot rewrite the shared standard_logging_object, and when it builds no payload its callbacks keep the shared one instead of getting None. * test(caching): lock sync cache-hit logging for completion, embedding and text_completion * test(logging): keep sync hook edits off a caller-supplied payload for async requests * fix(logging): give sync hooks a copy of the async payload when the thread builds none * test(logging): check every legacy sync logger gets the thread's own payload * test(logging): cover traceloop and supabase sync logging for async requests |
||
|
devin-ai-integration[bot]
|
86a91f35de
|
fix(responses): log and bill every round of non-stream MCP auto-execute (#43667)
* fix(responses): log and bill every round of non-stream MCP auto-execute Both model rounds run as internal sub-calls on the request's logging object, and the outer call logs the response the client received with usage, cost, and cost breakdown summed across the rounds. The internal-call gate now also covers the sync success callbacks, so an inner round no longer takes their first-wins slot * fix(caching): let the parent call log and bill an internal sub-call's cache hit A round of non-stream MCP auto-execute that the response cache served fired the cache-hit callbacks on the shared logging object, so the spend log kept round 1's function_call row and dropped the billed row for the answer. Internal sub-call cache hits now fire no callbacks and carry a zero cost, and the summed cost breakdown leaves out rounds served from the cache * fix(logging): keep a parent row unique when its response came from the response cache A RAG query or an MCP turn whose final round is served from the cache hands back the cached response's id, which already belongs to the row that cached it, so SpendLogs dropped the new row as a duplicate. Such a row now gets the cache-hit id suffix, and it is logged as a cache hit when it cost nothing. * fix(logging): skip the zero-cost warning for a response served from the cache at no charge A non-stream MCP auto-execute request whose rounds the response cache served is priced at $0 before the parent marks it a cache hit, so it logged a false 'priced at $0' warning and counted in litellm_zero_cost_requests_total. The diagnostic now treats a response the cache served at no charge the same as a marked cache hit * fix(responses): bill the paid first MCP round when a later step of the request fails * fix(responses): log each MCP tool call under its own id so a failed follow-up keeps its spend row The non-stream auto-execute path passed the request's call id to the tool call, so the tool's spend row and the request's failure row shared one request_id and the failure row was dropped on insert, leaving the paid first round unbilled. * fix(responses): bill round 1 on a client disconnect, keep MCP tool rows in the request's session, and charge a fixed margin once per request * fix(responses): keep bridged MCP auto-execute in one session and trace and continue its conversation The bridged follow-up no longer chains on round 1, so a thinking model keeps its reasoning and the request stays in one session. The tool call log continues the request trace without renaming it, the tools are fetched once, and a later turn finds the MCP turn under the id the client got, skipping the tool rows. * fix(responses): log the non-stream MCP tool listing in the request's session and trace * fix(responses): find a bridged MCP turn by the id the client holds and log its listing in the bridged trace * fix(responses): derive the MCP tool row id from the request's call id The tool call's spend row now uses <call id>_<tool call id>, so it stays linked to the request without colliding with the request's failure row. Also tighten the internal cache-hit test to a fixed clock and typed params. * fix(responses): bill a retried MCP attempt for its own rounds A router retry or fallback reuses the request's logging object, so a failed attempt's stashed round 1 usage made the next attempt's success row log stale tokens. Each non-stream MCP attempt now starts without that stash. * fix(responses): bill every failed MCP attempt's paid first round on the request failure A retry or fallback now sets a failed attempt's stashed round 1 aside instead of dropping it, and the proxy's failure row adds it to the last attempt's own, so a request whose later attempt fails on its first call still bills the round the provider charged. A cached final round also keeps the paid first round's cost breakdown. * fix(responses): keep MCP tool catalog and results out of the bridged next-turn replay * fix(responses): keep MCP tool spend, client body keys, and Langfuse traces working on auto-execute Count the MCP tools an auto-executed Responses API call ran in the tool spend rollup, accept a client body key named previous_response_client_ids without a 500, and point MCP listing and tool spans at the trace Langfuse gives the parent request, including x-litellm-session-id requests * fix(responses): join MCP spans to the Langfuse trace when the request sends body metadata * test(integration): cover non-stream MCP auto-execute billing, failure rows, and bridged follow-ups --------- Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com> |
||
|
devin-ai-integration[bot]
|
da2f5a498f
|
feat(agents): support Databricks Agent Platform agents in the agent gateway (#45681)
* feat(agents): support Databricks Agent Platform agents in the agent gateway Add a databricks_agent provider that speaks the MLflow ResponsesAgent wire (Responses-shaped input, output_text items, output_text.delta SSE) to Model Serving endpoints (/invocations or the unified /serving-endpoints/responses) and Databricks Apps (/responses), with a PAT or OAuth M2M client credentials Register it through the agents config block or the Admin UI agent catalog and call it through /a2a/<agent> (message/send, message/stream) or model: a2a/<agent> chat completions. The chat route now runs every completion-bridge agent through its own provider, and attaches the minted Databricks OAuth token for both bridge and URL agents. Consumed OAuth settings never reach the provider call * fix(agents): harden databricks_agent input, output, and OAuth handling Non-text content parts answer 400 unless drop_params is set, a reply without an output list answers 502, flat OAuth fields resolve an os.environ api_base, the bridge debug log no longer echoes the request model, routing tests stub only the HTTP boundary, and the catalog description fits the dropdown * fix(agents): merge agent and client extra_headers on the a2a chat route, and reject an empty databricks_oauth block * fix(agents): bridge only databricks_agent agents on the a2a chat route, keep the card url over a client api_base, and require workspace_url for a Databricks App * refactor(agents): read the a2a chat bridge providers from a constants set instead of branching on a provider name * fix(databricks_agent): let the configured credential win, reject malformed client extra_headers, and keep a missing endpoint name empty * test(agents): drive the flat OAuth a2a send through the real bridge * fix(agents): keep an empty databricks_oauth block ignored on url agents, as before this change * fix(agents): accept bare App hosts, env service principals, and os.environ refs on Databricks agents * fix(databricks_agent): address review caveats on chat bridging, params, and auth Map max_tokens and max_completion_tokens to max_output_tokens and pass temperature, top_p, and user, so /v1/messages works without drop_params. Send A2A message metadata only with string values, answering 400 or dropping the rest under drop_params. Join message items with a blank line in both stream and non-stream replies. Cache the env service principal token until shortly before it expires. Keep databricks_agent replies out of the response cache. On the chat bridge, let request fields win over the agent's stored params while pinning the agent's api_base and api_key, and apply the agent's static_headers between client and agent extra_headers, as /a2a does. Resolve os.environ/ only for databricks_agent api_base and api_key instead of every bridge provider. Ignore an unusable databricks_oauth block on URL agents on chat, as at the merge base, instead of answering 400. * test(internal_context): list the databricks agent token cache as process-local * fix(databricks_agent): keep the agent's endpoint, input, and target out of client control extra_body model and input answer 400, or are dropped under drop_params. A client api_base or base_url never replaces the agent's card URL on the bridge. Chat to a URL agent keeps its merge-base call, and an incomplete databricks_oauth block on a URL agent keeps its merge-base ValueError. * test(databricks): expect the merge-base 400 for a client api_base on agent chat * fix(databricks): keep agent-owned inputs and identity headers out of a chat caller's reach * fix(databricks): bound the agent-over-caller merge to two levels so it needs no recursion --------- Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com> |
||
|
Apeksha_2022
|
9423966f5d
|
fix(caching): report zero response_cost on a response-cache hit (#46471)
* fix(caching): report zero response_cost on a response-cache hit * test(caching): cover response_cost on a cache hit * fix(caching): pass cache_hit when stamping a cached response's metadata |
||
|
Mateo Wang
|
d81a5719ad
|
fix(cache): keep cache hits free of unset fields, and follow intentional behavior changes in CircleCI suites (#46359)
* test(integration): follow count_tokens error surfacing and decisions pricing in CircleCI suites Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(integration): restore Responses input_file PDF pricing coverage Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(integration): keep integration AGENTS guide unchanged Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix(cache): preserve provider field presence on cache hits Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(integration): expect credential display name in response Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(integration): align OpenInference refusal field expectations Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(integration): expect Akto request ingestion Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(mcp): expect missing tool error wording Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(integration): expect no refusal field in cursor chat wire Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> --------- Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> |
||
|
Yassin Kortam
|
fbf4af301c
|
feat(microsoft_365_copilot): add Microsoft 365 Copilot chat provider with OAuth token exchange (#45158)
* feat(microsoft_365_copilot): add Microsoft 365 Copilot chat provider with OAuth token exchange * fix(ui): create a credential for credential-only auth types in Add Model * fix(microsoft_365_copilot): accept max_tokens and list the chat model * fix(microsoft_365_copilot): register provider model set * fix(ui): hide other auth types' fields when credential-only types are filtered * fix(health): do not inherit stored auth settings when a connection test brings its own api_key * docs(health): restore connection-test inheritance docstrings * fix(lint): suppress justified provider boundary casts * fix(lint): satisfy M365 type-discipline gate * fix(types): eliminate type-check gate regressions * style: shorten suppression reasons so ruff format leaves them on one line * refactor(types): drop shared-file type widening unrelated to the Copilot provider * fix(health): pass caller headers as secret_fields to connection-test probes * fix(types): type connection-test probe and token counter locally * fix(lint): drop cast import from connection-test header getter * chore: remove explanatory comments flagged by review * fix(proxy): gate OAuth client credentials behind proxy admin * fix(router): skip cooldown on caller-scoped OAuth auth failures * fix(router): skip fallback cooldown on caller-scoped OAuth auth failures * fix(router): type caller-scoped cooldown lookups * fix(ci): include Microsoft 365 Copilot tests in unit shard * fix(proxy): keep saved endpoint when test_connection retests a deployment by id * fix(m365): collapse doubled Graph replies * fix(m365): preserve trailing system messages --------- Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> |
||
|
devin-ai-integration[bot]
|
56bafc29cf
|
feat(gemini): accept file content blocks with video_metadata on multimodal embeddings (#45305)
* feat(gemini): accept file content blocks with video_metadata on multimodal embeddings * test(gemini): annotate the embedding mock helper and keep the type comment short * fix(gemini-embeddings): keep file blocks through a partial embedding cache hit and validate video_metadata strictly * test(caching): type the partial-hit cache test and pin its clock * fix(gemini-embeddings): accept the Files API URI /v1/files returns as a file block's file_id * fix(gemini-embeddings): 400 on bad input shapes, drop unknown block keys under drop_params, cache multi-block inputs A bare object `input` answered 500 from both the caching handler and the transformation; both now answer 400. An unresolved `files/` reference on vertex_ai/ answered a ValueError 500; it now answers 400 naming the gemini/ provider. An empty `format` passed through to the provider; it now answers 400 naming file.format. Unknown block keys (`detail`, an unknown video_metadata key, a top-level block key) are dropped under drop_params, global or per-request, and still answer 400 without it. The embedding cache counted file blocks against `max_messages`, so a request with 5 or more blocks was never cached; file blocks no longer count. * fix(caching): answer 400 for an object embedding input on the cache lookup * test(gemini-embeddings): annotate the new embedding tests with return types and Final * test(gemini-embeddings): annotate the transformation and files batch tests with return types and Final * test(gemini-embeddings): audit file content blocks on the wire, in the cache, in batch uploads and under chaos Integration cells for gemini/ and vertex_ai/ embedding file blocks with video_metadata: the batchEmbedContents and embedContent wire shapes, format overrides, nested and repeated inputs, every malformed block answering 400 before any provider call, drop_params at the request, deployment and YAML levels, the OpenAI SDK sync and async clients, Redis cache fills, hits, partial hits and the metadata in the key with zero-spend hit rows, chat, responses and messages controls, the vertex_ai/ batch JSONL upload, a mixed fast/slow/malformed/dropped burst and a worker kill. The wire peer now reads chunked request bodies, which the streamed GCS media upload sends * test(integration): treat an upload the client abandons before its terminating chunk as a disconnect, never a stored request --------- Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com> |
||
|
devin-ai-integration[bot]
|
d364d5e7ca
|
refactor: expose public hidden_params accessors (#44668)
* refactor: expose public hidden_params accessors
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix: keep hidden params writes on duck-typed responses
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* refactor: move hidden params helpers to core utils
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix: preserve hidden params in duck-typed cost responses
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test: cover hidden params accessors across response types
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix: preserve hidden params for dynamic responses
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix: expose logs through backend component
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* Revert "fix: expose logs through backend component"
This reverts commit
|
||
|
devin-ai-integration[bot]
|
c3a23fe499
|
refactor: expose core private helpers under public names (#44871)
* refactor: expose core private symbols with compatibility aliases Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * types: narrow core migration diagnostics Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix: preserve runtime behavior in core symbol migration Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix: preserve core private usage migration behavior Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix: preserve private value rebinding compatibility Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix: restore optional imports and cover public helpers Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test: exempt router property from call coverage Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test: align recursive detector ignore names Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> --------- Co-authored-by: mateo <mateo@berri.ai> Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> |
||
|
devin-ai-integration[bot]
|
abd3af422d
|
fix(caching): never store or serve a chat completion with no choices (#44709)
* fix(caching): never store or serve a chat completion with no choices A provider response with empty choices was written to the response cache and served on every identical request until the TTL ended, with no provider call in between. The cache now skips storing such a response and treats an already stored one as a miss, so the next request goes back to the provider and its answer replaces the entry. * fix(caching): skip responses with no output on the Responses API and Anthropic Messages too * fix(caching): skip streams and stored entries that carry no output A chat or text completion stream whose chunks carried no choice is closed by the stream wrapper with one empty choice of its own, so the assembled response passed the choices check and was cached. The assembled stream is now judged on its content: a stream with no text, tool call, or other output in any choice is never stored, on the async and sync writers alike. The Responses API stream writer and the Anthropic Messages stream writer apply the same no-output check before storing. A stored entry with no output read through the worker memory tier is now evicted from that tier on the miss, so the next read reaches Redis where the refill lands; the text completion and messages writers only write to Redis, and the memory copy otherwise kept missing until its own TTL. * test(integration): response cache cells for answers without output Deterministic cells for the response cache on every unified endpoint, streamed and not, through the OpenAI and Anthropic SDKs and raw httpx, plus the sync SDK paths, stale entries, malformed answers, per-request TTLs, cache delete, and chaos (Redis stopped or paused mid burst, a worker killed, in-memory cache mode). The scripted upstream counts only POSTs as deployment calls, since the proxy's boot-time GET /v1/models discovery of a config deployment is not one. * test(caching): pin the stored entry timestamp in the worker-copy test * test(integration): drop the restating comments from the chaos cells * test(integration): close the breaker on the first call after the Redis restart --------- Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com> |
||
|
devin-ai-integration[bot]
|
564d236985
|
fix(otel): nest cache spans under their operation and name service spans by purpose (#44150)
Response cache reads and writes open cache.get llm_response and cache.set llm_response phase spans with their Redis spans nested underneath, on the Python path and on the native Rust path, and deployment selection runs inside a route {model_group} phase so the cooldown, usage and model-id reads the router issues nest under it before chat {model}. The autorouter classifier call nests under that route phase as well and carries its typed internal origin on litellm.request.purpose, so it is told apart from the provider attempt. Service spans are named {service}.{verb} {target} from a low-cardinality key family the producer declares (llm_response, auth_objects, spend_counters, router_cooldowns, claude_code_session_router_binding, rate_limits, pod_lock, budget_reset, ...) instead of the raw method or a per-request pipeline length; a pipeline flush is targeted by the one family its ops share or by mixed with the sorted families on litellm.redis.families, a batch op keeps the family it was declared under whichever pipeline or standalone read settles it, and the ambient family labels Redis spans only, never the DB write-back a task spawned inside that context performs later. The raw method stays on litellm.service.call_type and on the Prometheus and Datadog labels. Caller attribution is carried across asyncio task boundaries on a ContextVar so forwarder-only chains no longer surface, the raw cache key is dropped from Redis span metadata, pipeline op counts land as an integer attribute, every call_type the Redis cache layer emits maps to a verb, and a scan over litellm/ and enterprise/ fails when a Redis producer, batch reservation included, declares no key family.
A V2 logger built for a key or team logging entry while the operator's V2 logger is already registered keeps only the exporters its own preset contributed, whether or not the operator holds credentials for that backend, so every chat span no longer reaches the operator's collector twice. A span the success callback has to open itself, with no pre-call carrier, starts at the provider handoff (api_call_start_time) instead of the logging object's creation.
Co-authored-by: yassin <yassin@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
|
||
|
devin-ai-integration[bot]
|
d18fcb09d6
|
fix(otel): detach post-response service spans by request phase, name redis spans by operation (#43237)
* fix(otel): detach post-response service spans by request phase, name redis spans by operation Service spans logged from the post-response phase (success callbacks, the response-cache write) now root their own trace linked to the request span even while the server span is still recording, instead of only when they happen to end after it. Redis service spans are named `redis <operation>`; the litellm call chain that issued them moves to the `litellm.service.caller` attribute via a typed `ServiceLoggerPayload.caller` field. Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix(otel): keep the service caller on failure and legacy spans, test the production phase dispatch sites Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix(otel): mark anthropic messages stream cache write as post-response phase The /v1/messages streaming cache writer awaits async_add_cache inline instead of going through create_cache_write_task, so its redis span stayed parented under the request trace. Wrap the write in post_response_phase so it detaches like the chat completions write. Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix(anthropic): write the Messages stream cache in a background task after handoff Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> --------- Co-authored-by: yassin <yassin@berri.ai> Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> |
||
|
yuneng-jiang
|
a11a93f44a
|
test: move tests/test_litellm core utils, routing, responses, caching and rust_bridge into tests/unit (#43199)
* ci: run the unit_selection.sh shard files on every event instead of only fork pull requests Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * ci: rename fork-flag to unit-flag now that it applies on every event * test: move tests/test_litellm root and small trees into tests/unit Pure renames, no content changes. Follow-up commits in this PR fix references, merge the three files that already existed in tests/unit, keep live-provider tests in tests/test_litellm and wire CI. * test: carry tests/test_litellm conftest isolation into tests/unit Callback lists, routing fallbacks, cached HTTP clients, logger state, AWS, proxy-URL and keychain env, and session-end client cleanup now reset for unit tests too. The environment isolation owns its MonkeyPatch so a test's own monkeypatch is undone before the model-cost teardown runs. * test: merge, split and prune the moved root and small-tree tests Merge batches/test_batch_utils.py and the chat_completions and messages dispatch tests into the files that already existed in tests/unit. Keep the live Gemini interactions tests, the async image-fetch format test and the OpenAI embedding scorer test in tests/test_litellm since they need real network or keys. Put test_router.py under tests/unit/test_router so the existing package no longer shadows it. Delete eight tests the audit found superseded by stronger ones kept in this move. * ci: run the moved root and small-tree tests under their legacy flags Add the misc and responses-caching-types flags to unit_selection.sh and CircleCI, extend enterprise-routing and mcp-integration, and point the legacy GHA shards, Makefile, redis-compat workflow, merge smoke manifest and change classifier at the new paths. * test: make the new tests/unit directories packages tests/unit/test_package_layout.py requires every directory to carry an __init__.py, and without one the moved and retained test_litellm_responses_bridge.py modules collide on import. * test: scope the unit socket block to tests/unit in shared sessions The GHA shards collect the legacy test-path and the unit selection in one pytest session. The unit conftest's loopback-only block leaked into legacy modules that reach the network at import. The legacy conftest now lifts the restriction at collect and setup time, and the unit conftest re-applies it when collecting its own modules. * test: move tests/test_litellm/llms into tests/unit/llms Rename-only. Moves the provider tests and the fine-tuning fixtures they load, mirroring the old paths. Follow-up commits merge, split and wire them. * test: merge, split and prune the moved llms tests Merges the Databricks chat transformation tests into the existing unit file, keeps the tests that need real keys or the network in tests/test_litellm, deletes the audited tests a stronger unit test already covers, and points imports at tests.unit.llms. * ci: run the moved llms tests under their legacy flags The Vertex AI and All Other Providers shards keep their legacy test-path for the retained files and add the llm-vertex-ai and llm-other-providers unit selections. CircleCI gets matching unit jobs. * test: make the tests/unit/llms directories packages Adds __init__.py to the moved dirs and drops the legacy ones whose directories no longer hold tests. * test: drop script runners and path hacks the llms split left dangling The __main__ runners in the split openai_like files and the Databricks e2e runner called tests that now live in the other half of the split or were deleted. The retained legacy halves also no longer need sys.path edits. * test: give the shard-script tests their own GITHUB_OUTPUT They only passed where the runner set it. The CircleCI unit job's env allowlist drops it, so the script's redirect failed there. * test: point the router and module-deletion checks at tests/unit router_code_coverage and code_qa_check_tests only searched tests/test_litellm, so the moved router tests no longer counted. The two silent-experiment tests the audit deleted were the only direct callers of those methods; they are replaced with tests that assert the forwarded shadow request and the recursion guard. * test: move tests/test_litellm integrations and secret_managers into tests/unit Rename-only. Mirrors the old paths, including the directory conftests and the prompt and JSON fixtures. Follow-up commits prune and wire them. * test: prune and repoint the moved integrations tests Deletes the 7 audited tests a stronger test in the same tree already covers, imports the TLS sink helpers from their new conftest path, and restores os.environ after each integrations test. Some presets write OTEL_EXPORTER_OTLP_HEADERS straight into os.environ, and without the legacy tree's test ordering that header leaked into the AgentOps tests. * ci: run the moved integrations tests under their legacy flag The integrations GHA shard and a new CircleCI job run the integrations unit selection. secret_managers joins the misc selection. * docs: point integrations and secret_managers references at tests/unit * test: make the moved integrations directories packages * test: keep the Databricks manual e2e runner and fix the SageMaker Nova run path The Databricks e2e file is a manual script whose main() calls the tests that were pruned, so pruning them broke the documented run. It is back to its main version. The SageMaker Nova docstring now points at the file's real location in tests/local_testing. * test: move tests/test_litellm core utils, routing, responses, caching and rust_bridge into tests/unit Rename-only. Mirrors the old paths, including fixtures, the stubtest config and the native-route wheel script. Two files that collide with existing unit files are merged in a follow-up commit. * test: merge, prune and repoint the moved core, routing, responses, caching and rust_bridge tests Merges the two files that collided with existing unit files, folding the legacy extra case into test_is_chat_completion_cached_dict, and deletes the 9 audited tests a stronger test in the same file already covers. Keeps what needs the network in tests/test_litellm: test_tokenizers pulls a tokenizer from the Hugging Face hub, and the gpt2 and r50k_base tokenizer cases download their BPE files. The unit core_utils conftest points TIKTOKEN_CACHE_DIR at litellm's bundled encodings so the rest never depend on import order to stay offline, and FakeSecretVault moves to a shared module so both trees can build it. * ci: run the moved core, routing, responses, caching and rust_bridge tests under their flags core_utils gets a core-utils flag and CircleCI job, and its GHA shard keeps the legacy path for the retained network tests. router_utils and router_strategy join enterprise-routing, responses joins responses-caching-types (minus responses/mcp, which mcp-integration owns), caching joins caching-local and rust_bridge joins misc. The redis-compat, test-rust, stubtest and merge-smoke paths follow the move. * docs: point the Rust crate references at tests/unit * test: make the moved core, routing and rust_bridge directories packages * test: keep the no-loop DualCache batch_get_cache regression test It runs the sync path outside any event loop, which the inside-loop test cannot, so a change that picks the Redis client by loop state would only show up there. * test: keep the job's UNIT_FLAG out of the shard-script tests * fix(url_utils): block 192.0.0.0/24 on every Python patch release * test: move the new budget limiter tests into tests/unit/router_strategy * test: move the new sentry scrubbing tests into tests/unit/litellm_core_utils * test: move the new zerobus tests into tests/unit/integrations * test: make tests/unit/integrations/zerobus a package * test: load litellm's own tiktoken cache setup once instead of resetting it per test --------- Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> |
||
|
devin-ai-integration[bot]
|
bf0187072b
|
ci: move caching, proxy-extras, gateway and enterprise tests into tests/unit and run them from litellm-tests (#42902)
* ci: fix the litellm-tests unit job with sysmon coverage, an env allowlist and coverage upload on failure * test: replace key-dependent proxy, enterprise and mcp unit tests with synthetic values and integration and e2e coverage * test: drop key reads at the legacy proxy, enterprise and mcp paths and wire the gemini pass-through split * ci: move caching, proxy-extras, gateway and enterprise tests into tests/unit and run them from litellm-tests under their legacy flags * ci: move caching, proxy-extras, gateway and enterprise tests into tests/unit and run them from litellm-tests under their legacy flags * ci: fail the unit shard when circleci tests split errors * test: drop restating comments from the gemini pass-through split * ci: exit the unit shard cleanly when circleci tests split assigns it no files --------- Co-authored-by: yuneng <yuneng@berri.ai> |
Renamed from tests/local_testing/test_caching_handler.py (Browse further)