litellm/tests
mika-lt c0ded7b282
fix(responses)!: send a dict reasoning_effort the same way on every Responses bridge (#49082)
* fix(responses): send a dict reasoning_effort the same way on every Responses bridge

Azure and ChatGPT bridges dropped reasoning mode and context unless the dict
also carried summary. Every bridge now runs the reasoning step the OpenAI bridge
uses, and each Responses config declares the reasoning values its API accepts,
so drop_params drops what a backend refuses. Perplexity ignores mode and context,
so it answers 400 for them without drop_params.

* fix(responses): refuse ignored reasoning fields on xAI and Fireworks, resolve openai/ names for OpenRouter

* fix(responses): refuse a reasoning mode Copilot ignores, keep catalog modes, return refusals as values

Apply additional_drop_params before the refusal check, keep a null reasoning field instead of refusing it, let a cost-map row's reasoning_modes and reasoning_contexts override a backend's declared values, and declare GitHub Copilot's accepted values from a live probe

* fix(responses): read Azure base_model and deployment rows, accept standard mode, refuse only mode and context

* fix(responses): send only effort and summary to an Azure deployment litellm cannot identify under drop_params

Azure and Azure AI deployment names are user-chosen, so when neither the name, model_info.base_model, nor a cost map row resolves the model, drop_params now drops mode and context instead of forwarding a pro the model may refuse. Names resolve by their last path segment, so a regional base_model like azure/eu/gpt-5.4 follows gpt-5.4's rows

* fix(responses): resolve deployment names by model family and keep reasoning_modes per deployment

A custom Azure deployment name now resolves to the longest OpenAI catalog
prefix of its base_model or name, an unresolved name forwards mode and
context, and a deployment's model_info reasoning_modes and
reasoning_contexts apply only to that deployment instead of every sibling
sharing its model name. litellm_proxy forwards every reasoning field.

* fix(responses): find the model family anywhere in an Azure deployment name and keep only effort and summary for unknown gpt-5 or gpt-6 names under drop_params

* fix(responses): look up OpenAI model names inside deployment names only on Azure

Databricks endpoints like databricks-gpt-5-5-pro spell versions with hyphens and other backends name local models freely, so only Azure and Azure AI search inside the name, and a run that cuts a hyphen-spelled version never matches

* fix(responses): judge Azure AI reasoning fields by the request api_base

---------

Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
2026-10-30 03:47:59 -07:00
..
_support fix(proxy)!: end a chat completions stream with an error event when a guardrail blocks mid-stream (#46953) 2026-10-17 18:00:12 -07:00
audio_tests feat(vertex-ai)!: support gemini 3.1 flash tts (#31915) 2026-10-21 04:18:42 +00:00
base_sdk_tests build(model-catalog): ship the cost map inside the native extension (#48907) 2026-10-29 07:55:02 -07:00
benchmarks fix(realtime)!: send the Gemini Live setup at the first content so session.update settings apply (#39864) 2026-10-30 01:31:39 -07:00
code_coverage_tests fix(realtime)!: send the Gemini Live setup at the first content so session.update settings apply (#39864) 2026-10-30 01:31:39 -07:00
documentation_tests fix(proxy): let configured pass-through endpoints outrank built-in provider prefix routes (#44133) 2026-10-10 12:26:17 -07:00
e2e fix(realtime)!: send the Gemini Live setup at the first content so session.update settings apply (#39864) 2026-10-30 01:31:39 -07:00
e2e_harness refactor(helm)!: connect both charts to a separately deployed Lens (#49169) 2026-10-30 02:29:44 +00:00
fixtures test: migrate tests/llm_translation into unit, integration and e2e tiers (#45759) 2026-10-11 07:58:53 +00:00
harness_e2e fix(harness): set PWD to the working directory of local sandbox commands (#47530) 2026-10-20 20:06:11 +00:00
integration fix(proxy): apply new model rows on every worker as soon as they are written (#49034) 2026-10-30 03:19:25 -07:00
rust-python-harness feat(rust): encode LiteLLM response ids on native responses (#46572) 2026-10-18 20:52:34 -07:00
test_litellm_rust fix(anthropic): retry /v1/messages without strict when Claude rejects strict tools under drop_params (#49102) 2026-10-29 17:18:59 -07:00
typing fix: support OpenAI SDK 3 while retaining SDK 2 compatibility (#44927) 2026-10-09 19:45:50 -07:00
unit fix(responses)!: send a dict reasoning_effort the same way on every Responses bridge (#49082) 2026-10-30 03:47:59 -07:00
windows_tests refactor(rust-bridge): require the native extension (#48852) 2026-10-28 13:16:56 -07:00
__init__.py refactor: expose core private helpers under public names (#44871) 2026-10-07 00:05:27 +00:00
_fake_openai_endpoint_server.py
_flush_vcr_cache.py
_live_test_helpers.py
_master_key.py fix(security): remove the publicly known master key from the repo (#44718) 2026-10-06 10:55:24 -07:00
_openai_record_replay_proxy.py
_process_helpers.py test: count a zombie grandchild as gone in the migrate deploy timeout test (#42570) 2026-09-22 14:59:26 -07:00
_vcr_conftest_common.py test: migrate tests/llm_translation into unit, integration and e2e tiers (#45759) 2026-10-11 07:58:53 +00:00
_vcr_redis_persister.py
_wait_helpers.py
_ws_vcr.py
AGENTS.md docs(agents): ban semicolons and colons in human-facing text (#45684) 2026-10-09 16:36:32 -07:00
capturing_transport.py test(vcr): guard leaked cassette patches and make injected-transport embedding tests immune (#42542) 2026-09-22 14:20:18 -07:00
eval_swe_bench.py
fake_openai_endpoint.py
gettysburg.wav
large_text.py
openai_batch_completions.jsonl
pyrightconfig.json
README.MD test: finish the non-proxy half of tests/test_litellm (#43281) 2026-09-25 22:43:41 -07:00
test_logging.conf
test_rust_python_harness.py test: move 76 live top-level tests to offline unit and integration coverage (#45352) 2026-10-08 10:33:35 -07:00
VCR_TESTING.md test: migrate tests/llm_translation into unit, integration and e2e tiers (#45759) 2026-10-11 07:58:53 +00:00
white_100x100.png test: stop CI tests from downloading tokenizer files and images (#43257) 2026-09-25 19:27:48 -07:00

In total litellm runs 1000+ tests

[02/20/2025] Update:

To make it easier to contribute and map what behavior is tested,

we've started mapping the litellm directory in tests/unit

This folder can only run mock tests.