* test: drop the cwd-relative sys.path.insert calls from the test suite
TQ003 stands at 1,077 across 1,058 files, and 1,015 of them are the same shape:
sys.path.insert(0, os.path.abspath("../..")) and its deeper siblings. The
argument resolves against the working directory rather than the file, so from
the repo root, where every job runs pytest, it inserts the directory two levels
above the checkout. It has never pointed at litellm. The package is installed
into the environment anyway, which is what actually makes the import work, and
what the rule's message has said all along.
Removing them leaves 1,634 imports of sys and os with no remaining reference,
and those go too, except where another test module imports the name back out of
the file. The rest of TQ003 is 62 call sites that resolve against __file__ or a
variable, which are a different question and are left alone.
Collection is identical either way: 45,871 tests and the same 51 pre-existing
collection errors before and after, and ruff reports no new undefined name.
* test: drop the duplicate imports the sys.path sweep exposed to F811
* test(pre-call-utils): restore the os import the new bedrock tests need
* test: replace blind sleeps with deadline waits in callback and caching tests
tests/local_testing/test_custom_callback_input.py slept a fixed 1-3s after
every call and then asserted the callback handler recorded no errors. Because
the handler only appends to `states` when a callback actually fires, an assert
of `len(errors) == 0` passes just as happily when nothing fired at all, so the
sleep was buying flakiness in exchange for a vacuous check. The async tests
were worse: `time.sleep` blocks the event loop, so the success/failure tasks
scheduled on it could not run before the assertion.
Adds tests/_wait_helpers.py with `wait_until` / `await_until`, which poll a
predicate against a deadline, and converts all 17 sites to wait on the thing
the test actually cares about (the terminal state landing in `states`, or the
patched log hook being called). The waits assert the callback fired, so these
tests now fail on a dropped callback instead of passing silently.
The three sleeps in test_caching_handler.py sat between `sync_set_cache` and
`_sync_get_cache`, both fully synchronous against a local in-memory cache, so
they are just deleted.
* fix(test): wait on the priming call's own logging in the cache-hit test
The 3s sleep in test_logging_async_cache_hit_sync_call was not waiting for the
cache write, which lands before the stream iterator is exhausted. It was
waiting for the priming call's success callback to drain, so the handler
installed right after it only ever sees the second, cache-hit call. Waiting on
a populated cache_dict let the priming call's still-pending log_success_event
reach the new mock, and the test then read cache_hit off the wrong payload.
Waits on the priming handler's own sync_success state instead.
* fix(caching): replay openai/responses bridge cache hits as chat streams
When chat completions route through openai/responses, cached ModelResponse
payloads under aresponses keys were deserialized as ResponsesAPIResponse
(500) or re-translated as responses events (empty streaming deltas). Deserialize
chat-shaped cache entries as acompletion and bypass the responses stream iterator
for cached CustomStreamWrapper replay.
Co-authored-by: Cursor <cursoragent@cursor.com>
* fix(caching): map responses bridge call_type for sync vs async stream replay
Co-authored-by: Yassin Kortam <yassin@berri.ai>
* fix: handle ModelResponse cache return in responses bridge and drop dead acompletion check
Co-authored-by: Yassin Kortam <yassin@berri.ai>
* fix(caching): detect chat cache hits via object field before choices fallback
Prefer chat.completion object type over the broad choices-key heuristic so
Responses API cached payloads are not misclassified if their schema changes.
Co-authored-by: Cursor <cursoragent@cursor.com>
* test(caching): cover responses bridge cache-hit paths in CI-tracked test suite
The new bridge cache replay logic in caching_handler.py and the
preformatted-stream guard in litellm_responses_transformation/handler.py
were exercised only by tests under tests/local_testing/, which the
responses-caching-types and misc shards do not run. Codecov flagged the
patch as 29.72% covered.
Add equivalent unit tests under tests/test_litellm/ so the responses,
caching, types, and misc shards execute them and ship their coverage
data to Codecov:
- _is_chat_completion_cached_dict happy/sad paths
- aresponses streaming bridge cache hit -> CustomStreamWrapper
- responses non-streaming bridge cache hit -> ModelResponse
- legacy ResponsesAPIResponse stream + non-stream replay
- _is_preformatted_cached_chat_stream true/false
- completion/acompletion early return on cached ModelResponse
- completion/acompletion skip rewrap on preformatted cached stream
* fix: add negative guard on object field in _is_chat_completion_cached_dict
Co-authored-by: Yassin Kortam <yassin@berri.ai>
* fix(vcr): treat corrupt cassette payloads as cache miss
* test: bump EOL'd NVIDIA rerank and OpenAI realtime models in CI
The NVIDIA hosted rerank endpoint for nvidia/llama-3_2-nv-rerankqa-1b-v2
reached end-of-life on 2026-05-18 and now returns HTTP 410 Gone, breaking
TestNvidiaNim::test_basic_rerank. Switch to nvidia/nv-rerankqa-mistral-4b-v3,
which is still hosted on the NVIDIA API catalog and is already listed in
model_prices_and_context_window.json.
OpenAI also retired the gpt-4o-realtime-preview-2024-12-17 model used by
test_realtime_guardrails_openai (now returns model_not_found). Switch the
realtime test URL to the GA gpt-realtime alias.
Unrelated to the responses-bridge cache fix in this PR, but committing
here to unblock CI per maintainer guidance.
Co-authored-by: Mateo Wang <mateo-berri@users.noreply.github.com>
* test(realtime): switch retired gpt-4o-realtime-preview to gpt-realtime
OpenAI removed gpt-4o-realtime-preview and all its date snapshots on
2026-05-18 (every variant now returns model_not_found), breaking the
live-WebSocket OpenAI realtime tests in CI:
- test_openai_realtime_direct_call_no_intent
- test_openai_realtime_direct_call_with_intent
- TestOpenAIRealtime.test_realtime_connection
- TestOpenAIRealtime.test_realtime_with_query_params
Point each of those to the current GA alias gpt-realtime (verified live).
Pure unit/mock tests that just assert the string value (e.g. in
test_realtime_query_params_construction and the
test_realtime_query_params_use_normalized_model_name mock) are left
alone since they do not depend on model availability.
Also relax the AI-response assertion in
test_text_message_blocked_by_guardrail_no_ai_response: gpt-realtime
occasionally produces a polite refusal ("I'm sorry, but I can't say
that") when the cancel arrives after the model has already started
generating, which is the expected outcome (no real AI content) but does
not contain the words 'blocked' or 'guardrail'. The primary guardrail
behaviour (guardrail_violation error event + transcript_delta block
message) is still asserted unchanged.
Co-authored-by: Mateo Wang <mateo-berri@users.noreply.github.com>
* test(nvidia_nim): mock rerank live API instead of hitting EOL'd endpoint
NVIDIA reached end-of-life for the hosted nvidia/llama-3.2-nv-rerankqa-1b-v2
rerank API on 2026-05-18 (returns HTTP 410 Gone), and the proposed
replacement nv-rerankqa-mistral-4b-v3 returns HTTP 404 for the CI account,
breaking TestNvidiaNim::test_basic_rerank.
Override test_basic_rerank to mock the HTTP transport (same pattern as
test_nvidia_nim_rerank_ranking_endpoint above) so the request/response
transformation and cost calculation stay covered without depending on
NVIDIA's hosted catalog rotation. The model identifier reverts to the
original llama-3.2-nv-rerankqa-1b-v2 since the request never leaves
the test process.
---------
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Yassin Kortam <yassin@berri.ai>
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
Co-authored-by: Mateo Wang <mateo-berri@users.noreply.github.com>
Success handlers already run when CustomStreamWrapper or
CachedResponsesAPIStreamingIterator finishes replay. Logging at
cache-hit time for acompletion/completion streaming duplicated spend
and callbacks. Align tests with deferred behavior.
Made-with: Cursor
* fix: use fastuuid helper across the codebase
First batch of changes, simple drop in replacement.
* second batch of changes
* fixed: script mistake on helper file
* fix(together_ai/chat): only return response_format + tools for supported models
Fixes https://github.com/BerriAI/litellm/issues/6972
* feat(bedrock/rerank): initial working commit for bedrock rerank api support
Closes https://github.com/BerriAI/litellm/issues/7021
* feat(bedrock/rerank): async bedrock rerank api support
Addresses https://github.com/BerriAI/litellm/issues/7021
* build(model_prices_and_context_window.json): add 'supports_prompt_caching' for bedrock models + cleanup cross-region from model list (duplicate information - lead to inconsistencies )
* docs(json_mode.md): clarify model support for json schema
Closes https://github.com/BerriAI/litellm/issues/6998
* fix(_service_logger.py): handle dd callback in list
ensure failed spend tracking is logged to datadog
* feat(converse_transformation.py): translate from anthropic format to bedrock format
Closes https://github.com/BerriAI/litellm/issues/7030
* fix: fix linting errors
* test: fix test
* add unit testing for test_async_set_cache
* test test_async_log_cache_hit_on_callbacks
* assert the correct response type is returned
* test_convert_cached_result_to_model_response
* unit testing for caching handler
