scriptoriumadmin/litellm
1
0
Fork 0
mirror of https://github.com/BerriAI/litellm.git synced 2026-10-17 04:34:25 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 172
litellm/tests/unit
History
Mateo Wang d76ec3a47e
fix(proxy)!: answer 400 for malformed tools on count_tokens instead of crashing (#46583) 
* fix(proxy): answer 400 for malformed tools on count_tokens instead of crashing

A non-list tools value crashed the vector store lookup during auth and the
built-in tool cost lookup in the failure logging path, so the proxy answered
an unhandled 500 and closed the keep-alive connection. count_tokens now
answers a 400 invalid_request_error for a malformed body, drops malformed
tools when drop_params is on, and both lookups skip values of the wrong shape.

* fix(proxy): skip non-list tools in the tool allowlist check

Also validate the count_tokens body without recursion, which the recursion detector rejects.

* test(integration): restore the int tools case on count_tokens

The proxy now answers a non-list tools value with a 400, so the case is deterministic again.

* fix(proxy): reject malformed tools at the LLM route entry and keep vector store checks on malformed ids

* fix(proxy): apply drop_params like core when dropping malformed tools, and reject malformed tool vector store ids

Check tools after request setup so the Claude Code default and the logged call type apply, read the request,
global, and model group drop_params the way core does, drop tool_choice and friends with tools, cover Gemini
generateContent, and answer 400 for a malformed tools[].vector_store_ids with the strict flag off

* fix(proxy): keep the request's call type when logging a gate rejection after request setup

The body-shape fallback relabeled a /v1/messages rejection as acompletion and a list-input /v1/responses
rejection as aembedding, so it now only applies when the logging object carries no known call type

* fix(proxy): share the malformed tools check across tool routes and honor drop_params per entry

* fix(proxy): accept null vector_store_ids, read every named store id in auth, check the responses websocket

* refactor(proxy): leave the responses websocket out of the request-level tool check

* fix(proxy): let a request's additional_drop_params replace the deployment list and keep compact's call type on rejections

* fix(proxy): read router default drop lists and deployment ids in the malformed tools check

* fix(proxy): leave malformed tools to the router when it can pick another deployment

Fallbacks, auto-routers, model strings, comma batches and team model names
let the router pick a deployment after the proxy checks the request, so the
proxy now forwards a malformed tools field there as main does, and only
rejects it when the deployment it checks is the one that serves. Token
counting never reroutes, so count_tokens and input_tokens keep reading the
group they count with. additional_drop_params names are read the way core
reads them, so a list with a non-string entry still names its strings.

* test(integration): cover malformed tools on reroutable models, router defaults and deployment ids

* fix(proxy): resolve malformed tools at the deployment the router picks

* fix(proxy): keep provider-repaired tool schemas, honor fallback_on_unsupported_params, skip cooldown on malformed tools

Empty schemas and JSON-string schemas pass the tools check the way litellm's provider paths already read them. The rejection is a BadRequestError subclass carrying the deployment's provider, so fallback_on_unsupported_params: false keeps the 400, and the router never cools a deployment down for a request litellm rejected before calling it. Adds the audit's pre_call guardrail and cross-provider bridge cells.

* test(integration): expect a 400 for an empty-string tools field on /v1/messages

Real Anthropic rejects tools "" with a 400 on main too, so the case joins the malformed tools test

* fix(router): keep web search routing for tools passed as a tuple

* fix(router): keep a deployment's own tools when drop_params drops a malformed request tools

* test(router): list _merged_tools as covered through the deployment tools merge tests

* fix(router): drop the request's tool settings with its malformed tools when deployment tools replace them

* fix(router): skip a tool whose type is not a string in the web search filter instead of raising TypeError

* fix(prometheus): keep deployment health metrics clean on a malformed tools rejection

* test: type the new malformed tools tests

* test: mark new test locals Final and type kwargs dicts
2026-10-16 14:57:53 -07:00
..
a2a_protocol feat(agents): support Databricks Agent Platform agents in the agent gateway (#45681) 2026-10-15 12:42:06 -07:00
anthropic_interface fix(proxy): send a real error event when a /v1/messages stream fails (#41826) 2026-09-25 15:36:35 -07:00
batch_completion test: move routing, caching and callback tests out of local_testing (#45761) 2026-10-10 09:33:09 -07:00
batches feat(batches): store each batch line's request and response in S3 and GCS (#41691) 2026-10-15 11:32:56 -07:00
caching fix(responses): log and bill every round of non-stream MCP auto-execute (#43667) 2026-10-16 17:37:33 +00:00
completion_extras fix(responses): map Anthropic tool search to OpenAI server tool search (#46441) 2026-10-15 05:13:56 +00:00
compression test: migrate tests/llm_translation into unit, integration and e2e tiers (#45759) 2026-10-11 07:58:53 +00:00
containers refactor(proxy): expose public names for private proxy helpers (#45170) 2026-10-07 18:59:27 -07:00
decisions fix(decisions): read a deployment's own supports_box_questions flag before the cost map (#46696) 2026-10-15 12:34:36 -07:00
endpoints test: add __init__.py to migrated tests/unit packages 2026-09-20 11:53:49 +00:00
enterprise fix(responses): bill a background response once, on the first read or poll that sees it finished (#45703) 2026-10-15 11:00:14 -07:00
expected_fine_tuning_api test: move tests/test_litellm/llms into tests/unit/llms (#43191) 2026-09-25 12:43:23 -07:00
expected_responses_api_request test: move tests/test_litellm core utils, routing, responses, caching and rust_bridge into tests/unit (#43199) 2026-09-25 17:10:13 -07:00
experimental_mcp_client feat(mcp): relay authorized modern upstream subscriptions (#46153) 2026-10-15 16:16:49 -04:00
files test: move 81 legacy live tests in pass-through, spend, batches, audio, search, guardrails, image and ocr dirs offline (#45288) 2026-10-08 02:38:12 -07:00
fixtures test: move tests/test_litellm root and small trees into tests/unit (#43186) 2026-09-25 11:30:43 -07:00
gateway test(ci): fix six CircleCI test regressions on main (#44429) 2026-10-03 20:55:01 -07:00
google_genai feat(gemini)!: auto-continue gemini-4-argon long decodes with continuationToken (#46240) 2026-10-15 22:54:23 +00:00
harness feat(harness): add Harness.TOOL_LOOP, a minimal in-process tool-calling loop (#44391) 2026-10-03 17:33:01 +00:00
images fix: preserve explicit provider for async image, audio, and realtime routes (#36346) 2026-10-15 05:21:43 -07:00
integration_support fix(realtime)!: keep turn_detection.type when a transcript guardrail disables auto-response (#44939) 2026-10-16 15:24:40 +00:00
integrations fix(proxy)!: answer 400 for malformed tools on count_tokens instead of crashing (#46583) 2026-10-16 14:57:53 -07:00
interactions fix(interactions): keep background interaction bills through crashes, late restarts, and deployment keys (#45743) 2026-10-16 12:00:08 +00:00
litellm_core_utils fix(proxy)!: answer 400 for malformed tools on count_tokens instead of crashing (#46583) 2026-10-16 14:57:53 -07:00
litellm_proxy_extras fix(proxy-extras): accept unencoded DB passwords in the v2 migration lock and redact its errors (#46592) 2026-10-15 14:47:23 -07:00
llms fix(bedrock): drop dangerous-tool-use beta without safeguards (#44471) 2026-10-16 21:00:37 +00:00
models fix(proxy): allow model instances in LiteLLM_EndUserTable validation (#41677) 2026-10-15 10:57:26 -07:00
ocr test(rust-bridge): relocate dispatch tests before routing changes (#46926) 2026-10-16 12:29:33 -07:00
passthrough test: move 81 legacy live tests in pass-through, spend, batches, audio, search, guardrails, image and ocr dirs offline (#45288) 2026-10-08 02:38:12 -07:00
proxy fix(proxy)!: answer 400 for malformed tools on count_tokens instead of crashing (#46583) 2026-10-16 14:57:53 -07:00
rag fix(logging): skip sync success callbacks for internal sub-calls, deflake RAG and Langfuse tests (#45345) 2026-10-08 04:45:21 -07:00
realtime_api fix: preserve explicit provider for async image, audio, and realtime routes (#36346) 2026-10-15 05:21:43 -07:00
repositories fix(proxy): reset end-user spend in committed pages so the budget cascade survives a large LiteLLM_EndUserTable (#46502) 2026-10-15 19:47:12 -07:00
rerank_api test: migrate tests/llm_translation into unit, integration and e2e tiers (#45759) 2026-10-11 07:58:53 +00:00
responses test(rust-bridge): relocate dispatch tests before routing changes (#46926) 2026-10-16 12:29:33 -07:00
router_strategy fix(router): stop queueing budget increments for Redis when no Redis is configured (#46831) 2026-10-16 15:52:42 +00:00
router_utils fix(proxy)!: answer 400 for malformed tools on count_tokens instead of crashing (#46583) 2026-10-16 14:57:53 -07:00
rust_bridge test(rust-bridge): relocate dispatch tests before routing changes (#46926) 2026-10-16 12:29:33 -07:00
sandbox test: add __init__.py to every tests/unit directory phase 16 touches 2026-09-20 11:52:38 +00:00
search fix(search): bill an unpriced fallback search tool at its own price, not the failed primary's (#46734) 2026-10-15 12:20:33 -07:00
secret_managers chore(typing): clear 670 basedpyright Any errors by validating provider payloads at the boundary (#45388) 2026-10-13 15:35:23 -07:00
skills test: migrate tests/llm_translation into unit, integration and e2e tiers (#45759) 2026-10-11 07:58:53 +00:00
systemone fix(decisions): answer 400 on a malformed provider_specific_header instead of dropping it (#46619) 2026-10-15 14:04:03 +00:00
telemetry feat(telemetry): add AggregatingSink, fixed-bucket histograms, Report and HttpExporter (#45487) 2026-10-13 18:33:01 -07:00
test_router fix(proxy)!: answer 400 for malformed tools on count_tokens instead of crashing (#46583) 2026-10-16 14:57:53 -07:00
tracing fix(proxy): cooldown errors carry 429, Sentry starts at boot without request bodies, tracing failures log at ERROR, exporter counters exported (#46344) 2026-10-16 00:53:52 +00:00
types feat(ui): add the 1 hour cache write cost to the add model and model edit forms (#46830) 2026-10-16 13:44:24 -07:00
vector_stores fix(proxy)!: answer 400 for malformed tools on count_tokens instead of crashing (#46583) 2026-10-16 14:57:53 -07:00
videos test: migrate phase 16 legacy tests to tests/unit 2026-09-20 10:59:02 +00:00
__init__.py test(unit): add package markers to migrated unit test directories 2026-09-20 11:53:14 +00:00
AGENTS.md docs(agents): ban semicolons and colons in human-facing text (#45684) 2026-10-09 16:36:32 -07:00
conftest.py perf(ci): isolate and shard integration suites (#44146) 2026-10-10 11:15:19 -07:00
create_mock_standard_logging_payload.py test: migrate legacy unit dirs and unrun folders into tests/unit, tests/integration and tests/e2e (#45940) 2026-10-11 06:24:57 -07:00
test_a2a_registry_lookup.py test: move tests/test_litellm root and small trees into tests/unit (#43186) 2026-09-25 11:30:43 -07:00
test_acompletion_session_reuse_e2e.py test: move tests/test_litellm root and small trees into tests/unit (#43186) 2026-09-25 11:30:43 -07:00
test_add_deployment_no_master_key.py test: move tests/test_litellm root and small trees into tests/unit (#43186) 2026-09-25 11:30:43 -07:00
test_aembedding_session_reuse_e2e.py refactor(llms): expose public names for private provider helpers (#45037) 2026-10-07 08:49:25 -07:00
test_anthropic_beta_headers_filtering.py fix(vertex_ai): forward the inline-tools-2026-09-15 beta to Vertex and Anthropic (#45297) 2026-10-08 12:27:52 -07:00
test_anthropic_skills_transformation.py feat(anthropic): workload identity federation and pluggable identity sources (#44448) 2026-10-03 17:08:30 -07:00
test_assert_ci_coverage.py docs(agents): keep directory rules in nested AGENTS.md and pin integration tests to CircleCI (#46796) 2026-10-16 14:57:49 -07:00
test_assert_workflow_dir_hygiene.py test: move tests/test_litellm root and small trees into tests/unit (#43186) 2026-09-25 11:30:43 -07:00
test_audio_transcription_rust_bridge.py refactor(python-bridge): ship a signature base and read the resolved call (#45450) 2026-10-08 17:20:29 -07:00
test_auto_update_price_and_context_window_file.py test: move tests/test_litellm root and small trees into tests/unit (#43186) 2026-09-25 11:30:43 -07:00
test_azure_ad_token_credential_resolution.py test: move tests/test_litellm root and small trees into tests/unit (#43186) 2026-09-25 11:30:43 -07:00
test_azure_ai_grok_4_3_model_metadata.py test: move tests/test_litellm root and small trees into tests/unit (#43186) 2026-09-25 11:30:43 -07:00
test_azure_ai_grok_4_6_model_metadata.py test: move tests/test_litellm root and small trees into tests/unit (#43186) 2026-09-25 11:30:43 -07:00
test_baseten_glm_5_3_model_metadata.py test: move tests/test_litellm root and small trees into tests/unit (#43186) 2026-09-25 11:30:43 -07:00
test_batch_completion_models_all_responses.py test: move routing, caching and callback tests out of local_testing (#45761) 2026-10-10 09:33:09 -07:00
test_bedrock_marengo_embed_3_model_metadata.py test: move tests/test_litellm root and small trees into tests/unit (#43186) 2026-09-25 11:30:43 -07:00
test_build_core_distribution.py test: move packaging and install checks off CircleCI build_and_test (#45757) 2026-10-10 09:33:51 -07:00
test_chat_ui_responses_session.py test: move tests/test_litellm root and small trees into tests/unit (#43186) 2026-09-25 11:30:43 -07:00
test_check_licenses.py feat(proxy): embed enterprise LiteAdmin MCP in LiteLLM images (#44610) 2026-10-05 15:24:34 -07:00
test_check_mcp_operation_boundary.py test: move tests/test_litellm root and small trees into tests/unit (#43186) 2026-09-25 11:30:43 -07:00
test_check_migrations_lock_timeout.py fix(proxy-extras): bound migration lock waits with SET LOCAL lock_timeout in the migration SQL (#46329) 2026-10-13 20:51:03 -07:00
test_check_migrations_no_data_rewrites.py fix(proxy-extras): build the SpendLogs indexes in the migration job instead of in migrations (#43948) 2026-10-01 14:12:22 -07:00
test_check_openapi_docs.py ci: gate undocumented POST routes in the proxy OpenAPI document (#45843) 2026-10-10 21:51:00 +00:00
test_check_py310_typing_imports.py test: move tests/test_litellm root and small trees into tests/unit (#43186) 2026-09-25 11:30:43 -07:00
test_check_test_quality.py test(integration): hand every harness server a port from one per-worker block (#46761) 2026-10-15 16:56:40 -07:00
test_check_type_discipline.py ci(lint): gate every rule on its merge-base count and cap Anys at a fixed total (#40193) 2026-10-08 05:39:29 +00:00
test_check_unbounded_in_lists.py test: migrate legacy unit dirs and unrun folders into tests/unit, tests/integration and tests/e2e (#45940) 2026-10-11 06:24:57 -07:00
test_check_windows_wheel_install.py test: move packaging and install checks off CircleCI build_and_test (#45757) 2026-10-10 09:33:51 -07:00
test_circleci_path_filter.py ci: move redis-compat and MCP unit suites from CircleCI to GitHub Actions (#46026) 2026-10-12 02:24:31 -07:00
test_circleci_run_migration_tests.py test: migrate proxy security, schema migration and MCP legacy tests (#45937) 2026-10-11 16:17:24 -07:00
test_circleci_rust_toolchain.py test: migrate tests/llm_translation into unit, integration and e2e tiers (#45759) 2026-10-11 07:58:53 +00:00
test_claude_fable_5_config.py refactor(llms): expose public names for private provider helpers (#45037) 2026-10-07 08:49:25 -07:00
test_claude_haiku_5_5_config.py fix(model_prices): consolidate claude-haiku-5-5 over-100k pricing and capability flags (#45151) 2026-10-07 14:14:47 -07:00
test_claude_opus_4_6_config.py test: move tests/test_litellm root and small trees into tests/unit (#43186) 2026-09-25 11:30:43 -07:00
test_claude_opus_4_8_config.py test: move tests/test_litellm root and small trees into tests/unit (#43186) 2026-09-25 11:30:43 -07:00
test_claude_opus_5_config.py refactor(llms): expose public names for private provider helpers (#45037) 2026-10-07 08:49:25 -07:00
test_claude_sonnet_5_config.py refactor(cost-map): remove openrouter rows, part 5 of 5 (#45669) 2026-10-09 22:48:55 +00:00
test_cloudflare_workers_ai_model_metadata.py test: move tests/test_litellm root and small trees into tests/unit (#43186) 2026-09-25 11:30:43 -07:00
test_completion_timeout_resolution.py fix: support OpenAI SDK 3 while retaining SDK 2 compatibility (#44927) 2026-10-09 19:45:50 -07:00
test_component_entrypoint.py feat(prometheus): cap series per metric for every labeled metric (#44420) 2026-10-06 01:28:44 +00:00
test_compression.py test: move tests/test_litellm root and small trees into tests/unit (#43186) 2026-09-25 11:30:43 -07:00
test_conftest_isolation.py perf(ci): isolate and shard integration suites (#44146) 2026-10-10 11:15:19 -07:00
test_constants.py fix(config): treat blank env values as unset so the proxy neither crash-loops nor silently drops a guardrail (#46341) 2026-10-14 12:49:31 -07:00
test_container_router.py test: move tests/test_litellm root and small trees into tests/unit (#43186) 2026-09-25 11:30:43 -07:00
test_cost_calculation_log_level.py test: move tests/test_litellm root and small trees into tests/unit (#43186) 2026-09-25 11:30:43 -07:00
test_cost_calculator.py fix(openai)!: route gpt-transcribe to GPT audio transcription config (#44021) 2026-10-16 20:27:47 +00:00
test_cost_map_guard.py feat(decisions): bill decisions and systemone from _decisions cost map fields (#45904) 2026-10-12 14:31:12 -07:00
test_count_tokens_public_api.py fix(count_tokens): count Anthropic-format requests for OpenAI the way /v1/messages inference converts them (#46759) 2026-10-16 01:58:46 +00:00
test_dashscope_image_generation.py refactor(types): replace Any with proven types in 137 files (#44478) 2026-10-05 11:01:45 -07:00
test_daybreak_model_metadata.py test: move tests/test_litellm root and small trees into tests/unit (#43186) 2026-09-25 11:30:43 -07:00
test_deepseek_model_metadata.py refactor: expose core private helpers under public names (#44871) 2026-10-07 00:05:27 +00:00
test_default_branch.py ci(lint): gate every rule on its merge-base count and cap Anys at a fixed total (#40193) 2026-10-08 05:39:29 +00:00
test_detect_changes.py ci: render lint, unit and smoke checks as <tier> / <job> with one collector per tier (#45480) 2026-10-09 04:28:41 +00:00
test_docker_no_network_on_deploy.py test: migrate local_testing SDK and provider tests out of build_and_test (#45760) 2026-10-10 23:14:25 +00:00
test_dockerfile_apk_repository.py test: move tests/test_litellm root and small trees into tests/unit (#43186) 2026-09-25 11:30:43 -07:00
test_dockerfile_bedrock_realtime_extra.py test: move tests/test_litellm root and small trees into tests/unit (#43186) 2026-09-25 11:30:43 -07:00
test_dockerfile_non_root.py test: move tests/test_litellm root and small trees into tests/unit (#43186) 2026-09-25 11:30:43 -07:00
test_drop_params_env_var.py test: move tests/test_litellm root and small trees into tests/unit (#43186) 2026-09-25 11:30:43 -07:00
test_e2e_egress_sentinel.py test: move tests/test_litellm root and small trees into tests/unit (#43186) 2026-09-25 11:30:43 -07:00
test_eager_tiktoken_load.py test: move tests/test_litellm root and small trees into tests/unit (#43186) 2026-09-25 11:30:43 -07:00
test_env_key_doc_gate.py test: move tests/test_litellm root and small trees into tests/unit (#43186) 2026-09-25 11:30:43 -07:00
test_exception_exports.py test: move tests/test_litellm root and small trees into tests/unit (#43186) 2026-09-25 11:30:43 -07:00
test_exception_header_preservation.py fix: support OpenAI SDK 3 while retaining SDK 2 compatibility (#44927) 2026-10-09 19:45:50 -07:00
test_exception_mapping_request_attribute.py test: move tests/test_litellm root and small trees into tests/unit (#43186) 2026-09-25 11:30:43 -07:00
test_fake_openai_endpoint.py test: migrate local_testing SDK and provider tests out of build_and_test (#45760) 2026-10-10 23:14:25 +00:00
test_filter_out_litellm_params.py fix(params): validate stream_chunk_size once, before any provider call (#43222) 2026-09-26 23:01:20 +00:00
test_fireworks_serverless_model_costs.py test: move tests/test_litellm root and small trees into tests/unit (#43186) 2026-09-25 11:30:43 -07:00
test_gate_slot_lock.py test: move tests/test_litellm root and small trees into tests/unit (#43186) 2026-09-25 11:30:43 -07:00
test_gemini_3_1_flash_lite_image_pricing.py test: move tests/test_litellm root and small trees into tests/unit (#43186) 2026-09-25 11:30:43 -07:00
test_gemini_4_argon_model_metadata.py feat(gemini)!: auto-continue gemini-4-argon long decodes with continuationToken (#46240) 2026-10-15 22:54:23 +00:00
test_gemini_tts_native_audio_pricing.py test: move tests/test_litellm root and small trees into tests/unit (#43186) 2026-09-25 11:30:43 -07:00
test_get_blog_posts.py feat(ui): source Home What's new from day 0 support blog posts (#46260) 2026-10-13 13:11:59 -07:00
test_git_hooks.py test: move tests/test_litellm root and small trees into tests/unit (#43186) 2026-09-25 11:30:43 -07:00
test_gpt_5_4_model_metadata.py test: move tests/test_litellm root and small trees into tests/unit (#43186) 2026-09-25 11:30:43 -07:00
test_gpt_5_5_model_metadata.py test: move tests/test_litellm root and small trees into tests/unit (#43186) 2026-09-25 11:30:43 -07:00
test_gpt_image_cost_calculator.py test: move tests/test_litellm root and small trees into tests/unit (#43186) 2026-09-25 11:30:43 -07:00
test_gpt_realtime_mode.py test: move tests/test_litellm root and small trees into tests/unit (#43186) 2026-09-25 11:30:43 -07:00
test_groq_streaming_encoding.py test: move tests/test_litellm root and small trees into tests/unit (#43186) 2026-09-25 11:30:43 -07:00
test_guardrail_exception_status_codes.py test: move tests/test_litellm root and small trees into tests/unit (#43186) 2026-09-25 11:30:43 -07:00
test_integration_run.py test(integration): isolate every multi-worker integration suite by default (#46027) 2026-10-12 03:46:06 -07:00
test_internal_context.py feat(agents): support Databricks Agent Platform agents in the agent gateway (#45681) 2026-10-15 12:42:06 -07:00
test_lazy_imports.py feat(sdk): build core from an independent packaging manifest (#44340) 2026-10-08 18:40:14 -07:00
test_lint_base_counts.py revert(lint-gates): accept an empty base scan again, since zero violations is a legitimate count (#45319) 2026-10-08 08:40:08 +00:00
test_lint_workflow_diff_gates.py fix(ci): align misc unit tests with NativeCall bridge and widened e2e diff gates (#45180) 2026-10-07 16:33:49 -07:00
test_litellm_overhead.py test(unit): wait on the work itself instead of fixed sleeps (#46261) 2026-10-13 12:41:06 -07:00
test_litellm_params_reserved_keys.py test: move tests/test_litellm root and small trees into tests/unit (#43186) 2026-09-25 11:30:43 -07:00
test_logging.py perf(proxy): keep stream end output and Redis cache debug strings off the event loop (#45252) 2026-10-09 14:56:16 -07:00
test_lowest_latency_zero_tokens.py test: move tests/test_litellm root and small trees into tests/unit (#43186) 2026-09-25 11:30:43 -07:00
test_main.py fix: preserve explicit provider for async image, audio, and realtime routes (#36346) 2026-10-15 05:21:43 -07:00
test_main_module_header.py test: move tests/test_litellm root and small trees into tests/unit (#43186) 2026-09-25 11:30:43 -07:00
test_mistral_large_4_model_metadata.py fix(model_prices): add Bedrock flex prices, Anthropic web search flags, Gemini shutdown dates, OpenRouter alias drift, Mistral Large 4 context (#44910) 2026-10-08 13:12:42 -07:00
test_mistral_medium_3_5_model_metadata.py test: move tests/test_litellm root and small trees into tests/unit (#43186) 2026-09-25 11:30:43 -07:00
test_mistral_small_4_0_model_metadata.py test: move tests/test_litellm root and small trees into tests/unit (#43186) 2026-09-25 11:30:43 -07:00
test_mistral_zai_glm_5_2_model_metadata.py test: move tests/test_litellm root and small trees into tests/unit (#43186) 2026-09-25 11:30:43 -07:00
test_model_block_unblock.py fix(proxy): return 4xx instead of 500 for missing required params, invalid pagination and unknown ids (#43787) 2026-10-02 22:48:53 -07:00
test_model_cost_aliases.py test: move tests/test_litellm root and small trees into tests/unit (#43186) 2026-09-25 11:30:43 -07:00
test_model_param_helper.py refactor: expose core private helpers under public names (#44871) 2026-10-07 00:05:27 +00:00
test_model_prices_schema.py ci(cost map): classify token_count_framing and token_count_fallback_model in the schema generator (#46784) 2026-10-16 00:30:12 +00:00
test_model_response_normalization.py test: move tests/test_litellm root and small trees into tests/unit (#43186) 2026-09-25 11:30:43 -07:00
test_muse_spark_1_1_model_metadata.py test: move tests/test_litellm root and small trees into tests/unit (#43186) 2026-09-25 11:30:43 -07:00
test_muse_spark_1_2_model_metadata.py test: move tests/test_litellm root and small trees into tests/unit (#43186) 2026-09-25 11:30:43 -07:00
test_muse_spark_1_3_model_metadata.py feat(model_prices): add meta/muse-wonder-1.0 (#46072) 2026-10-12 19:36:52 +00:00
test_mutation_report.py test: move tests/test_litellm root and small trees into tests/unit (#43186) 2026-09-25 11:30:43 -07:00
test_nested_drop_params.py test: move tests/test_litellm root and small trees into tests/unit (#43186) 2026-09-25 11:30:43 -07:00
test_no_top_level_test_invocations.py test: move whole-unit legacy test files into tests/unit and delete dead skips (#44807) 2026-10-07 05:02:28 +00:00
test_non_chat_routes_open_llm_spans.py test: move tests/test_litellm root and small trees into tests/unit (#43186) 2026-09-25 11:30:43 -07:00
test_openai_embedding_encoding_format_default.py test: migrate local_testing SDK and provider tests out of build_and_test (#45760) 2026-10-10 23:14:25 +00:00
test_openai_record_replay_proxy.py test: move whole-unit legacy test files into tests/unit and delete dead skips (#44807) 2026-10-07 05:02:28 +00:00
test_openai_service_tier_long_context_pricing.py fix(cost_calculator): bill ultrafast prompts above 272k at the ultrafast long-context rates (#43764) 2026-09-30 07:32:16 -07:00
test_openapi_docs_gate.py ci: gate undocumented POST routes in the proxy OpenAPI document (#45843) 2026-10-10 21:51:00 +00:00
test_package_layout.py test(unit): make every tests/unit directory a package so pytest collection is unique 2026-09-20 11:50:59 +00:00
test_package_metadata.py test: move packaging and install checks off CircleCI build_and_test (#45757) 2026-10-10 09:33:51 -07:00
test_pre_commit_lint.py ci(lint): gate every rule on its merge-base count and cap Anys at a fixed total (#40193) 2026-10-08 05:39:29 +00:00
test_prisma_generate_if_needed.py test: move tests/test_litellm root and small trees into tests/unit (#43186) 2026-09-25 11:30:43 -07:00
test_private_usage_aliases.py refactor(proxy): expose public names for private proxy helpers (#45170) 2026-10-07 18:59:27 -07:00
test_process_helpers.py test: move tests/test_litellm root and small trees into tests/unit (#43186) 2026-09-25 11:30:43 -07:00
test_project_alias_tracking.py test: move tests/test_litellm root and small trees into tests/unit (#43186) 2026-09-25 11:30:43 -07:00
test_project_tags_pydantic.py test: move tests/test_litellm root and small trees into tests/unit (#43186) 2026-09-25 11:30:43 -07:00
test_proxy_auth.py test: move tests/test_litellm root and small trees into tests/unit (#43186) 2026-09-25 11:30:43 -07:00
test_pydantic_namespaces.py test: move whole-unit legacy test files into tests/unit and delete dead skips (#44807) 2026-10-07 05:02:28 +00:00
test_rag_openai_ingestion.py test: move tests/test_litellm root and small trees into tests/unit (#43186) 2026-09-25 11:30:43 -07:00
test_rate_limit_error_unification.py refactor(proxy): expose public names for private proxy helpers (#45170) 2026-10-07 18:59:27 -07:00
test_read_rc_version.py test: move tests/test_litellm root and small trees into tests/unit (#43186) 2026-09-25 11:30:43 -07:00
test_redact_string_in_error_paths.py refactor: expose core private helpers under public names (#44871) 2026-10-07 00:05:27 +00:00
test_redis.py test: move routing, caching and callback tests out of local_testing (#45761) 2026-10-10 09:33:09 -07:00
test_redis_credential_provider.py test: move tests/test_litellm root and small trees into tests/unit (#43186) 2026-09-25 11:30:43 -07:00
test_register_model_custom_pricing.py fix(router): match deployment pricing ids against the cost map only within the deployment's provider (#45472) 2026-10-08 19:20:21 -07:00
test_register_model_zero_cost_persistence.py test: move tests/test_litellm root and small trees into tests/unit (#43186) 2026-09-25 11:30:43 -07:00
test_replicate_model_key_format.py test: move tests/test_litellm root and small trees into tests/unit (#43186) 2026-09-25 11:30:43 -07:00
test_responses_api_bridge_non_stream.py refactor: expose core private helpers under public names (#44871) 2026-10-07 00:05:27 +00:00
test_responses_id_security.py fix(proxy): check previous_response_id ownership on /v1/responses/compact (#46408) 2026-10-14 17:57:34 +00:00
test_responses_streaming_container_ownership.py fix(responses): bill a background response once, on the first read or poll that sees it finished (#45703) 2026-10-15 11:00:14 -07:00
test_retrieve_batch_bedrock_dispatch.py refactor(llms): expose public names for private provider helpers (#45037) 2026-10-07 08:49:25 -07:00
test_router_block_helpers.py refactor: expose core private helpers under public names (#44871) 2026-10-07 00:05:27 +00:00
test_router_exception_redaction.py test: move tests/test_litellm root and small trees into tests/unit (#43186) 2026-09-25 11:30:43 -07:00
test_router_get_settings.py ci(circleci): test Redis behavior against local Redis and print short tracebacks (#44062) 2026-10-01 13:00:54 -07:00
test_router_google_genai.py test: move tests/test_litellm root and small trees into tests/unit (#43186) 2026-09-25 11:30:43 -07:00
test_router_model_cost_isolation.py feat(router): mid-stream fallback continuation for chat completions (#41127) 2026-10-16 13:51:32 +00:00
test_router_order_fallback.py fix(router): keep the prompt caching affinity pin alive as long as the prefix's cache_control ttl (#46084) 2026-10-14 21:10:09 -07:00
test_router_per_deployment_num_retries.py test: move tests/test_litellm root and small trees into tests/unit (#43186) 2026-09-25 11:30:43 -07:00
test_router_redis_init.py test: move tests/test_litellm root and small trees into tests/unit (#43186) 2026-09-25 11:30:43 -07:00
test_router_retry_backoff_headers.py test: move tests/test_litellm root and small trees into tests/unit (#43186) 2026-09-25 11:30:43 -07:00
test_router_retry_non_retryable_errors.py test: move tests/test_litellm root and small trees into tests/unit (#43186) 2026-09-25 11:30:43 -07:00
test_router_retry_policy_update.py fix(router): add fallback_on_unsupported_params so param rejections can raise instead of falling back (#46350) 2026-10-14 14:43:01 -07:00
test_router_silent_experiment.py test: fix stale and flaky tests across CircleCI, GHA and Buildkite (#45530) 2026-10-09 11:40:14 -07:00
test_router_streaming_fallback_metadata.py refactor: expose public hidden_params accessors (#44668) 2026-10-07 00:01:33 -07:00
test_router_weighted_failover.py refactor: expose core private helpers under public names (#44871) 2026-10-07 00:05:27 +00:00
test_ruff_strict_gate.py ci(lint): gate every rule on its merge-base count and cap Anys at a fixed total (#40193) 2026-10-08 05:39:29 +00:00
test_sambanova_model_metadata.py test: move tests/test_litellm root and small trees into tests/unit (#43186) 2026-09-25 11:30:43 -07:00
test_scaledown_model_prices.py feat(scaledown): add native text provider support (#44168) 2026-10-10 14:01:17 -07:00
test_scheduler.py fix(scheduler): remove a request's queue entry once it stops waiting (#43061) 2026-10-07 17:34:38 -07:00
test_secret_redaction.py fix(vertex): report an unreadable credentials file as a read failure (#40173) 2026-10-15 02:03:56 -07:00
test_seed_tracing_fixtures.py refactor(lens): connect LiteLLM to the independent Lens service (#45529) 2026-10-09 16:17:05 -07:00
test_select_ui_test_scope.py ci: render lint, unit and smoke checks as <tier> / <job> with one collector per tier (#45480) 2026-10-09 04:28:41 +00:00
test_service_logger.py test: move 76 live top-level tests to offline unit and integration coverage (#45352) 2026-10-08 10:33:35 -07:00
test_setup_wizard.py test: move tests/test_litellm root and small trees into tests/unit (#43186) 2026-09-25 11:30:43 -07:00
test_shared_session_integration.py test: move tests/test_litellm root and small trees into tests/unit (#43186) 2026-09-25 11:30:43 -07:00
test_socket_policy.py test(unit): block external sockets at import time and add a socket policy regression test 2026-09-20 08:07:50 +00:00
test_ssl_verify_unit.py test(unit): stop test modules from putting their own directory on sys.path (#43421) 2026-09-28 12:20:20 -07:00
test_stream_chunk_builder_annotations.py test: move tests/test_litellm root and small trees into tests/unit (#43186) 2026-09-25 11:30:43 -07:00
test_stream_chunk_builder_citations.py test: move tests/test_litellm root and small trees into tests/unit (#43186) 2026-09-25 11:30:43 -07:00
test_stream_chunk_builder_images.py test: move tests/test_litellm root and small trees into tests/unit (#43186) 2026-09-25 11:30:43 -07:00
test_streaming_connection_cleanup.py test: move tests/test_litellm root and small trees into tests/unit (#43186) 2026-09-25 11:30:43 -07:00
test_sync_together_ai_models.py test: move tests/test_litellm root and small trees into tests/unit (#43186) 2026-09-25 11:30:43 -07:00
test_system_message_format_bug.py fix(prompt_templates): do not mutate messages in-place in function_call_prompt (#41676) 2026-10-15 14:37:11 -07:00
test_test_quality_gate.py ci(lint): gate every rule on its merge-base count and cap Anys at a fixed total (#40193) 2026-10-08 05:39:29 +00:00
test_thinking_enabled.py test: move tests/test_litellm root and small trees into tests/unit (#43186) 2026-09-25 11:30:43 -07:00
test_together_ai_model_metadata.py test: move tests/test_litellm root and small trees into tests/unit (#43186) 2026-09-25 11:30:43 -07:00
test_tool_loop.py fix(responses bridge): round-trip tool_search_call and tool_search_output items between turns (#46444) 2026-10-14 21:40:14 -07:00
test_type_check_gate.py ci(lint): gate every rule on its merge-base count and cap Anys at a fixed total (#40193) 2026-10-08 05:39:29 +00:00
test_type_discipline_gate.py ci(lint): gate every rule on its merge-base count and cap Anys at a fixed total (#40193) 2026-10-08 05:39:29 +00:00
test_typesafe_model_metadata.py fix(cost): price typesafe/jev-1.14.0 (#46594) 2026-10-14 20:27:35 -07:00
test_unit_shard_missing_paths.py ci: render lint, unit and smoke checks as <tier> / <job> with one collector per tier (#45480) 2026-10-09 04:28:41 +00:00
test_unit_shard_per_test_timeout.py ci: render lint, unit and smoke checks as <tier> / <job> with one collector per tier (#45480) 2026-10-09 04:28:41 +00:00
test_utils.py fix(openai)!: route gpt-transcribe to GPT audio transcription config (#44021) 2026-10-16 20:27:47 +00:00
test_utils_get_model_info.py test: move offline logging, secret manager and provider tests from legacy dirs into tests/unit (#45554) 2026-10-09 09:48:10 -07:00
test_utils_get_optional_params.py test: migrate tests/llm_translation into unit, integration and e2e tiers (#45759) 2026-10-11 07:58:53 +00:00
test_utils_module_docstring.py test: move tests/test_litellm root and small trees into tests/unit (#43186) 2026-09-25 11:30:43 -07:00
test_uuid_helper.py test: move tests/test_litellm root and small trees into tests/unit (#43186) 2026-09-25 11:30:43 -07:00
test_vcr_classification.py test: move offline logging, secret manager and provider tests from legacy dirs into tests/unit (#45554) 2026-10-09 09:48:10 -07:00
test_vcr_conftest_common_banner.py test: move whole-unit legacy test files into tests/unit and delete dead skips (#44807) 2026-10-07 05:02:28 +00:00
test_vcr_filters.py test: move whole-unit legacy test files into tests/unit and delete dead skips (#44807) 2026-10-07 05:02:28 +00:00
test_vcr_leak_guard.py test: move whole-unit legacy test files into tests/unit and delete dead skips (#44807) 2026-10-07 05:02:28 +00:00
test_vcr_redis_persister.py test: migrate tests/llm_translation into unit, integration and e2e tiers (#45759) 2026-10-11 07:58:53 +00:00
test_vcr_safe_body_matcher.py fix(tests): stop VCR recording and replaying a test's own localhost upstream (#43346) 2026-09-26 15:49:39 -07:00
test_version.py feat(sdk): build core from an independent packaging manifest (#44340) 2026-10-08 18:40:14 -07:00
test_vertex_ai_xai_grok_prompt_caching_metadata.py test: move tests/test_litellm root and small trees into tests/unit (#43186) 2026-09-25 11:30:43 -07:00
test_video_generation.py refactor(proxy): expose public names for private proxy helpers (#45170) 2026-10-07 18:59:27 -07:00
test_with_dashboard_node.py test: move tests/test_litellm root and small trees into tests/unit (#43186) 2026-09-25 11:30:43 -07:00
test_ws_vcr.py test: move whole-unit legacy test files into tests/unit and delete dead skips (#44807) 2026-10-07 05:02:28 +00:00
test_xai_grok_4_3_model_metadata.py test: move tests/test_litellm root and small trees into tests/unit (#43186) 2026-09-25 11:30:43 -07:00
test_xai_responses_auto_routing.py test: move tests/test_litellm root and small trees into tests/unit (#43186) 2026-09-25 11:30:43 -07:00