litellm/tests
devin-ai-integration[bot] f3e1307ed9
fix(vertex_ai): log Gemini 3 sampling deprecation once per model per worker (#40868)
* fix(vertex_ai): log Gemini 3 sampling deprecation once per process

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(vertex_ai): bound the Gemini 3 sampling warn-once cache and test it by behavior

* fix(vertex_ai): log the Gemini 3 sampling deprecation once even when first calls race

Pin the warned-model cap to its own constant so the shared LRU size knob cannot turn the dedupe off, and serialize the cached call so concurrent first requests for one model log it once

* fix(vertex_ai): stop logging the Gemini 3 sampling deprecation for new models once 64 are remembered

* fix(vertex_ai): give each mapped Gemini 3 model its own sampling notice and share one for unmapped names

* fix(vertex_ai): key the Gemini 3 sampling notice on the loaded catalog so runtime-priced names share one notice

---------

Co-authored-by: shivam <shivam@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
2026-10-20 23:26:52 +00:00
..
_support fix(proxy)!: end a chat completions stream with an error event when a guardrail blocks mid-stream (#46953) 2026-10-17 18:00:12 -07:00
base_sdk_tests fix(sdk): decide MCP gateway routing without importing the proxy handler (#45961) 2026-10-20 01:44:54 +00:00
benchmarks feat(extraction): add /v1/extraction with Cloudflare Clairic support (#47473) 2026-10-20 15:51:45 -07:00
code_coverage_tests fix(proxy): price earlier timed-out retry attempts at their own deployment's model rate (#47648) 2026-10-20 22:26:09 +00:00
documentation_tests
e2e test(perf): tag a UI journey sample with its API response, not a Next.js prefetch under /ui/ (#47650) 2026-10-20 14:59:34 -07:00
e2e_harness test(perf): emit a top-contributors breakdown property per perf metric (#47632) 2026-10-20 13:48:10 -07:00
fixtures
harness_e2e fix(harness): set PWD to the working directory of local sandbox commands (#47530) 2026-10-20 20:06:11 +00:00
integration fix(responses): send the billed background response cost to the creating key's and team's loggers (#47506) 2026-10-20 23:22:21 +00:00
rust-python-harness feat(rust): encode LiteLLM response ids on native responses (#46572) 2026-10-18 20:52:34 -07:00
test_litellm_rust fix(logging)!: fire one CustomLogger failure hook per async call outside the async flag list (#47124) 2026-10-20 15:19:14 -07:00
typing
unit fix(vertex_ai): log Gemini 3 sampling deprecation once per model per worker (#40868) 2026-10-20 23:26:52 +00:00
windows_tests
__init__.py
_fake_openai_endpoint_server.py
_flush_vcr_cache.py
_live_test_helpers.py
_master_key.py
_openai_record_replay_proxy.py
_process_helpers.py
_vcr_conftest_common.py
_vcr_redis_persister.py
_wait_helpers.py
_ws_vcr.py
AGENTS.md
capturing_transport.py
eval_swe_bench.py
fake_openai_endpoint.py
gettysburg.wav
large_text.py
openai_batch_completions.jsonl
pyrightconfig.json
README.MD
test_logging.conf
test_rust_python_harness.py
VCR_TESTING.md
white_100x100.png

In total litellm runs 1000+ tests

[02/20/2025] Update:

To make it easier to contribute and map what behavior is tested,

we've started mapping the litellm directory in tests/unit

This folder can only run mock tests.