mirror of
https://github.com/BerriAI/litellm.git
synced 2026-11-02 07:13:02 +00:00
a44e44b980
* fix(cache_control): resolve the chat-path dialect with the deployment api_base * test(cache_control): type the chat-path resolver helpers * fix(cache_control): pick the chat-path dialect from the deployment's provider prefix Resolving with the deployment api_base followed the dispatch's azure_ai to azure flip, so chat on a non-Foundry Azure host sent Anthropic cache_control while /v1/messages, told azure_ai, sent the OpenAI breakpoint. Use the model's provider prefix when the caller does not name a provider, so chat matches the told path whatever AZURE_AI_API_BASE holds, and keep name-only resolution everywhere else. * fix(cache_control): keep litellm_proxy routing when the chat path reads the provider prefix With USE_LITELLM_PROXY or litellm.use_litellm_proxy on, every call goes to the upstream proxy, so the provider prefix no longer decides the cache format there and resolution matches main. The plain OpenAI check reads the same provider value as the hosted check, so one function no longer mixes two lookups * fix(cache_control): resolve the chat-path dialect with the deployment api_base, as the call is routed The hosted cache-format check now resolves the provider with the deployment's own api_base when the caller names none, the same provider the call dispatches to and the Responses path passes, so AZURE_AI_API_BASE no longer decides the chat format. The plain OpenAI check keeps main's name-only lookup. * fix(cache_control): send acompletion fallbacks the caller's request, not the primary's hooked kwargs acompletion handed async_completion_with_fallbacks the kwargs the cache seed and prompt hook had already rewritten for the first model, so a Claude fallback behind a Foundry GPT model received prompt_cache_options and got a 400. Fallbacks now get a copy of the caller's model, messages, tools, and kwargs taken before the hooks, as completion already does with its args. Both cache seeds also read base_url before api_base, since completion sends the call to base_url when both are set. * fix(cache_control): keep the caller's tools list intact for acompletion fallbacks The vector-store hook pops recognized file_search tools from the request's tools list in place, so the fallback request built from the pre-hook snapshot lost the retrieved context. Snapshot a copy of the list instead. * fix(cache_control): skip acompletion's prompt hooks when fallbacks will run them Each fallback attempt runs the prompt hooks for its own model, so running them first in the outer call only cost a vector-store search whose result the hand-off threw away. * fix(cache_control): run acompletion's prompt hooks for fallbacks and leave only the cache hook to each attempt --------- Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com> |
||
|---|---|---|
| .. | ||
| _support | ||
| audio_tests | ||
| base_sdk_tests | ||
| benchmarks | ||
| code_coverage_tests | ||
| documentation_tests | ||
| e2e | ||
| e2e_harness | ||
| fixtures | ||
| harness_e2e | ||
| integration | ||
| rust-python-harness | ||
| test_litellm_rust | ||
| typing | ||
| unit | ||
| windows_tests | ||
| __init__.py | ||
| _fake_openai_endpoint_server.py | ||
| _flush_vcr_cache.py | ||
| _live_test_helpers.py | ||
| _master_key.py | ||
| _openai_record_replay_proxy.py | ||
| _process_helpers.py | ||
| _vcr_conftest_common.py | ||
| _vcr_redis_persister.py | ||
| _wait_helpers.py | ||
| _ws_vcr.py | ||
| AGENTS.md | ||
| capturing_transport.py | ||
| eval_swe_bench.py | ||
| fake_openai_endpoint.py | ||
| gettysburg.wav | ||
| large_text.py | ||
| openai_batch_completions.jsonl | ||
| pyrightconfig.json | ||
| README.MD | ||
| test_logging.conf | ||
| test_rust_python_harness.py | ||
| VCR_TESTING.md | ||
| white_100x100.png | ||
In total litellm runs 1000+ tests
[02/20/2025] Update:
To make it easier to contribute and map what behavior is tested,
we've started mapping the litellm directory in tests/unit
This folder can only run mock tests.