scriptoriumadmin/litellm
1
0
Fork 0
mirror of https://github.com/BerriAI/litellm.git synced 2026-11-01 07:03:02 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 129
litellm/tests/unit
History
tavren27 b25ea48c4b
fix(anthropic)!: drop beta-gated body fields under drop_params (#48793) 
* fix(anthropic): strip beta-gated body fields when a leg drops the anthropic-beta

When a provider leg drops an anthropic-beta value (the chat handler's allowlist
filter, /v1/messages' filtered headers, Bedrock Converse's additionalModelRequestFields),
also drop the body fields that beta gates (thinking.block_binding, thinking.display=updates,
mid-conversation system clear_at, output_config, tool_addition/tool_removal blocks) so the
request degrades on that leg instead of failing with a provider 400. Copy-on-write so a
fallback leg still sends the full request, and each strip is logged at WARNING.

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(integration): prove a leg that drops a beta sends neither the beta nor clear_at

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(bedrock): strip beta-gated fields against the final body anthropic_beta list

Converse counts the body anthropic_beta list as active, Invoke strips after
its auto-injected betas are known and injects thinking-binding-controls for
thinking.block_binding, test helpers are fully typed

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(anthropic): strip beta-gated system fields before an older model's system turn becomes a user turn

Models without supports_mid_conversation_system send no mid-conversation beta,
so tool_addition and tool_removal blocks are dropped (and logged) before the
turn is converted instead of riding along as user content. The Invoke
output_config regression test reads the real cost map instead of patching
_bedrock_model_supports

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(anthropic): pin the integration proxy to the bundled beta allowlist and type the helper locals

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(anthropic): annotate every new test local with Final and type the fixtures and returns

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(anthropic): strip beta-gated body fields only under drop_params

Without drop_params the request now goes out as sent and the provider
answers its own 400, like any other unsupported param. inline-tools-2026-09-15
also unlocks tool_addition and tool_removal blocks, so a leg that carries it
keeps them.

* fix(anthropic): strip beta-gated fields on Foundry chat and Bedrock chat Invoke under drop_params

Foundry chat and Bedrock chat Invoke now drop beta-gated body fields the same way the other legs do when drop_params is on and the request does not send the gating beta. The mid-conversation system normalizer validates only on the conversion path, and the strip WARNING names the beta the caller can send to keep the field.

* fix(anthropic): send the thinking betas for block_binding and display updates, strip Databricks chat thinking

Every Claude leg now adds the thinking-binding-controls beta when thinking carries block_binding and the display-updates beta for display updates, so those fields go through instead of a 400. Databricks chat rejects both fields with or without the beta, so it drops them under drop_params. A system turn the strip empties stays when it still carries output_config, and the drop warning names both causes.

* fix(anthropic): send the thinking betas on every leg and keep fields where the beta is unknown

Vertex messages and Claude Platform chat now add the thinking betas when thinking carries
block_binding or display updates. Converse sends only the caller's header betas when it has them.
A provider with no row in the beta config forwards beta-gated fields instead of dropping them,
an emptied trailing system turn that would leave an assistant prefill is sent as is, and the
prompt cache baseline body matches the body that goes out. The drop WARNING now names the beta
that unlocks each field

* fix(anthropic): filter Converse body betas, keep Foundry auto betas, read the bundled beta config for row checks

* fix(anthropic): validate messages only when a beta-gated drop runs

A non-object messages entry on a Claude model without mid-conversation
system support raised a raw pydantic ValidationError before the
beta-config and drop_params checks. The strip now passes the messages
through untouched unless it drops something, so the provider's 400
reaches SDK callers as BadRequestError again

* fix(anthropic): keep system blocks with a non-string type instead of raising

---------

Co-authored-by: mateo <mateo@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
2026-10-29 06:25:31 -07:00
..
a2a_protocol fix(streaming): keep merged think tags balanced and split SSE lines only on CR/LF (#48755) 2026-10-28 17:00:29 -07:00
accounting feat(accounting): add accounting contracts, PricingSnapshot and ledger schema (LIT-10193) (#48862) 2026-10-28 14:39:25 -07:00
anthropic_interface fix(proxy): send a real error event when a /v1/messages stream fails (#41826) 2026-09-25 15:36:35 -07:00
batch_completion test: move routing, caching and callback tests out of local_testing (#45761) 2026-10-10 09:33:09 -07:00
batches fix(batches): apply the Vertex regional endpoint uplift to batch cost (#47466) 2026-10-20 01:51:51 +00:00
caching fix(caching): read the cache key's model group from the router's metadata for the call (#48917) 2026-10-29 05:35:23 -07:00
completion_extras fix(responses): replay the original Responses item order, phase, and boundaries on the next chat turn (#48912) 2026-10-29 00:56:02 -07:00
compression test: migrate tests/llm_translation into unit, integration and e2e tiers (#45759) 2026-10-11 07:58:53 +00:00
containers refactor(typing): clear 1181 basedpyright Any errors across 431 files (#47789) 2026-10-24 11:31:09 -07:00
db_scripts fix(vector-stores): keep proxy context out of provider vector store metadata (#47021) 2026-10-17 12:51:47 -07:00
decisions feat(decisions): support Ollama decision models (#48871) 2026-10-28 12:35:26 -07:00
endpoints feat(vertex-ai)!: support gemini 3.1 flash tts (#31915) 2026-10-21 04:18:42 +00:00
enterprise feat(proxy)!: add suppress_startup_output to turn off startup prints (#47708) 2026-10-29 03:47:04 -07:00
expected_fine_tuning_api test: move tests/test_litellm/llms into tests/unit/llms (#43191) 2026-09-25 12:43:23 -07:00
expected_responses_api_request test: move tests/test_litellm core utils, routing, responses, caching and rust_bridge into tests/unit (#43199) 2026-09-25 17:10:13 -07:00
experimental_mcp_client fix(mcp): load deprecated package aliases on demand (#48615) 2026-10-27 19:46:19 +00:00
extraction feat(extraction): add /v1/extraction with Cloudflare Clairic support (#47473) 2026-10-20 15:51:45 -07:00
files test: move 81 legacy live tests in pass-through, spend, batches, audio, search, guardrails, image and ocr dirs offline (#45288) 2026-10-08 02:38:12 -07:00
fine_tuning fix(fine_tuning): stop forwarding api_version to the OpenAI client (#44995) 2026-10-21 01:58:25 +00:00
fixtures test: move tests/test_litellm root and small trees into tests/unit (#43186) 2026-09-25 11:30:43 -07:00
gateway test(unit): keep unix socket paths in unit tests under the macOS AF_UNIX limit (#45040) 2026-10-28 20:23:22 -07:00
google_genai fix(google_genai): read one-object and null spellings of repeated fields in countTokens and generateContent (#48771) 2026-10-28 21:46:06 -07:00
harness fix(streaming)!: send the include_usage chunk with an empty choices array (#47109) 2026-10-21 02:26:05 +00:00
images refactor(images): remove dall-e-2 and dall-e-3 code paths (#48703) 2026-10-28 17:48:09 -07:00
integration_support test(integration): wait for every worker in Scenario.model (#47277) 2026-10-18 14:49:25 -07:00
integrations fix(logging): mark mirrored silent_model calls with is_silent_experiment in spend logs (#48888) 2026-10-29 05:12:50 -07:00
interactions fix(streaming): keep merged think tags balanced and split SSE lines only on CR/LF (#48755) 2026-10-28 17:00:29 -07:00
litellm_core_utils fix(cost): bill a deployment's per-second rates on native Anthropic /v1/messages (#42417) 2026-10-29 05:29:03 -07:00
litellm_proxy_extras fix(migration_data_checks): size btree entries by the bytes Postgres stores (LIT-10244) (#48911) 2026-10-28 15:34:15 -07:00
llms fix(anthropic)!: drop beta-gated body fields under drop_params (#48793) 2026-10-29 06:25:31 -07:00
mcp_client refactor(mcp): drop the experimental label from the MCP server and client paths (#48513) 2026-10-27 17:47:50 +00:00
models fix(proxy): allow model instances in LiteLLM_EndUserTable validation (#41677) 2026-10-15 10:57:26 -07:00
ocr test(rust-bridge): relocate dispatch tests before routing changes (#46926) 2026-10-16 12:29:33 -07:00
passthrough fix(bedrock)!: stream Converse passthrough frames through post_call guardrails instead of buffering (#46307) 2026-10-17 00:53:53 -07:00
proxy fix(passthrough): forward OpenAI-Beta and default agents=v1 on the OpenAI passthrough (#49009) 2026-10-29 05:52:34 -07:00
rag fix(rag)!: use team provider credentials and allow viewer and JWT team access (#45497) 2026-10-17 23:51:48 +00:00
realtime_api fix(health)!: openai realtime health check reports healthy with a missing or invalid api key (#41233) 2026-10-29 00:40:02 -07:00
repositories fix(proxy): reconcile team member total_spend with current cycle spend (#48260) 2026-10-24 17:57:09 -07:00
rerank_api test: migrate tests/llm_translation into unit, integration and e2e tiers (#45759) 2026-10-11 07:58:53 +00:00
responses fix(responses): resume background response streams with stream and starting_after (#48819) 2026-10-29 02:31:22 -07:00
router_strategy test(router_strategy): keep foreign asyncio task noise out of the budget limiter caplog asserts (#43375) 2026-10-29 06:22:06 -07:00
router_utils fix(openai): warn when the GPT-5 reasoning effort gate drops a level under drop_params (#47761) 2026-10-29 04:46:29 -07:00
rust_bridge fix(anthropic): drop the non-spec custom object from /v1/messages tool definitions (#48893) 2026-10-29 03:36:20 -07:00
sandbox fix(streaming): keep merged think tags balanced and split SSE lines only on CR/LF (#48755) 2026-10-28 17:00:29 -07:00
search fix(search): price Tavily advanced and Linkup deep searches at their own cost map entries (#47411) 2026-10-19 23:32:31 -07:00
secret_managers fix(azure): share one Entra token request per worker and audit #47348 (#48769) 2026-10-28 12:24:20 -07:00
skills feat(skills): add a litellm-setup agent skill (#47731) 2026-10-28 15:44:43 -07:00
systemone fix(databricks): reuse base OAuth for decisions (#48395) (#48527) 2026-10-26 20:18:16 -07:00
telemetry feat(telemetry): add AggregatingSink, fixed-bucket histograms, Report and HttpExporter (#45487) 2026-10-13 18:33:01 -07:00
test_router feat(responses): support Codex client tool_search on Anthropic and other bridged providers (#48972) 2026-10-29 00:29:40 -07:00
text_completions fix(completions): omit stream usage unless requested (#47953) (#48052) 2026-10-23 19:36:20 +00:00
tracing fix(proxy): cooldown errors carry 429, Sentry starts at boot without request bodies, tracing failures log at ERROR, exporter counters exported (#46344) 2026-10-16 00:53:52 +00:00
types fix(responses): drop the null created_by OpenAI rejects when output items are replayed (#48783) 2026-10-28 21:11:49 -07:00
vector_stores fix(proxy): enforce team and key Allowed Vector Stores when no vector store registry is loaded (#48915) 2026-10-28 23:25:14 -07:00
videos test: migrate phase 16 legacy tests to tests/unit 2026-09-20 10:59:02 +00:00
__init__.py test(unit): add package markers to migrated unit test directories 2026-09-20 11:53:14 +00:00
AGENTS.md test(unit): keep unix socket paths in unit tests under the macOS AF_UNIX limit (#45040) 2026-10-28 20:23:22 -07:00
conftest.py fix(llms): move later developer messages for models declared system-first (#48897) 2026-10-29 04:12:13 -07:00
create_mock_standard_logging_payload.py test: migrate legacy unit dirs and unrun folders into tests/unit, tests/integration and tests/e2e (#45940) 2026-10-11 06:24:57 -07:00
test_a2a_registry_lookup.py test: move tests/test_litellm root and small trees into tests/unit (#43186) 2026-09-25 11:30:43 -07:00
test_acompletion_session_reuse_e2e.py test: move tests/test_litellm root and small trees into tests/unit (#43186) 2026-09-25 11:30:43 -07:00
test_add_deployment_no_master_key.py test: move tests/test_litellm root and small trees into tests/unit (#43186) 2026-09-25 11:30:43 -07:00
test_aembedding_session_reuse_e2e.py refactor(llms): expose public names for private provider helpers (#45037) 2026-10-07 08:49:25 -07:00
test_anthropic_beta_gated_fields.py fix(anthropic)!: drop beta-gated body fields under drop_params (#48793) 2026-10-29 06:25:31 -07:00
test_anthropic_beta_headers_filtering.py feat(proxy): add LITELLM_OFFLINE_MODE, one switch for the fetches LiteLLM makes on its own (#48096) 2026-10-23 16:54:19 -07:00
test_anthropic_skills_transformation.py feat(anthropic): workload identity federation and pluggable identity sources (#44448) 2026-10-03 17:08:30 -07:00
test_assert_ci_coverage.py docs(agents): keep directory rules in nested AGENTS.md and pin integration tests to CircleCI (#46796) 2026-10-16 14:57:49 -07:00
test_assert_workflow_dir_hygiene.py test: move tests/test_litellm root and small trees into tests/unit (#43186) 2026-09-25 11:30:43 -07:00
test_audio_transcription_rust_bridge.py fix(bedrock): omit system prompt from audio transcription and report its token usage (#41565) 2026-10-28 19:23:15 -07:00
test_azure_ad_token_credential_resolution.py test: move tests/test_litellm root and small trees into tests/unit (#43186) 2026-09-25 11:30:43 -07:00
test_azure_ai_grok_4_3_model_metadata.py test: move tests/test_litellm root and small trees into tests/unit (#43186) 2026-09-25 11:30:43 -07:00
test_azure_ai_grok_4_6_model_metadata.py test: move tests/test_litellm root and small trees into tests/unit (#43186) 2026-09-25 11:30:43 -07:00
test_baseten_glm_5_3_model_metadata.py test: move tests/test_litellm root and small trees into tests/unit (#43186) 2026-09-25 11:30:43 -07:00
test_batch_completion_models_all_responses.py test: move routing, caching and callback tests out of local_testing (#45761) 2026-10-10 09:33:09 -07:00
test_bedrock_marengo_embed_3_model_metadata.py test: move tests/test_litellm root and small trees into tests/unit (#43186) 2026-09-25 11:30:43 -07:00
test_build_core_distribution.py test: move packaging and install checks off CircleCI build_and_test (#45757) 2026-10-10 09:33:51 -07:00
test_cerebras_llama_model_metadata.py feat(models): add Cerebras Llama 3.1 hyphenated aliases (cerebras/llama-3.1-8b, cerebras/llama-3.1-70b) (#48157) (#48158) 2026-10-24 17:16:33 -07:00
test_chat_ui_responses_session.py test: move tests/test_litellm root and small trees into tests/unit (#43186) 2026-09-25 11:30:43 -07:00
test_check_licenses.py feat(proxy): embed enterprise LiteAdmin MCP in LiteLLM images (#44610) 2026-10-05 15:24:34 -07:00
test_check_mcp_operation_boundary.py refactor(mcp): drop the experimental label from the MCP server and client paths (#48513) 2026-10-27 17:47:50 +00:00
test_check_migrations_lock_timeout.py fix(proxy-extras): bound migration lock waits with SET LOCAL lock_timeout in the migration SQL (#46329) 2026-10-13 20:51:03 -07:00
test_check_migrations_no_data_rewrites.py fix(proxy-extras): build the SpendLogs indexes in the migration job instead of in migrations (#43948) 2026-10-01 14:12:22 -07:00
test_check_openapi_docs.py ci: gate undocumented POST routes in the proxy OpenAPI document (#45843) 2026-10-10 21:51:00 +00:00
test_check_py310_typing_imports.py test: move tests/test_litellm root and small trees into tests/unit (#43186) 2026-09-25 11:30:43 -07:00
test_check_test_quality.py test(integration): hand every harness server a port from one per-worker block (#46761) 2026-10-15 16:56:40 -07:00
test_check_type_discipline.py ci(lint): gate every rule on its merge-base count and cap Anys at a fixed total (#40193) 2026-10-08 05:39:29 +00:00
test_check_unbounded_in_lists.py test: migrate legacy unit dirs and unrun folders into tests/unit, tests/integration and tests/e2e (#45940) 2026-10-11 06:24:57 -07:00
test_check_windows_wheel_install.py test: move packaging and install checks off CircleCI build_and_test (#45757) 2026-10-10 09:33:51 -07:00
test_circleci_path_filter.py refactor(mcp): drop the experimental label from the MCP server and client paths (#48513) 2026-10-27 17:47:50 +00:00
test_circleci_run_migration_tests.py test: migrate proxy security, schema migration and MCP legacy tests (#45937) 2026-10-11 16:17:24 -07:00
test_circleci_rust_toolchain.py test: migrate tests/llm_translation into unit, integration and e2e tiers (#45759) 2026-10-11 07:58:53 +00:00
test_claude_fable_5_config.py refactor(llms): expose public names for private provider helpers (#45037) 2026-10-07 08:49:25 -07:00
test_claude_haiku_5_5_config.py fix(model_prices): consolidate claude-haiku-5-5 over-100k pricing and capability flags (#45151) 2026-10-07 14:14:47 -07:00
test_claude_opus_4_6_config.py test: move tests/test_litellm root and small trees into tests/unit (#43186) 2026-09-25 11:30:43 -07:00
test_claude_opus_4_8_config.py test: move tests/test_litellm root and small trees into tests/unit (#43186) 2026-09-25 11:30:43 -07:00
test_claude_opus_5_config.py refactor(llms): expose public names for private provider helpers (#45037) 2026-10-07 08:49:25 -07:00
test_claude_sonnet_5_config.py refactor(cost-map): remove openrouter rows, part 5 of 5 (#45669) 2026-10-09 22:48:55 +00:00
test_cloudflare_workers_ai_model_metadata.py test: move tests/test_litellm root and small trees into tests/unit (#43186) 2026-09-25 11:30:43 -07:00
test_completion_timeout_resolution.py fix(router)!: bound fake-streamed calls by the request timeout, not stream_timeout (#47162) 2026-10-17 22:08:02 +00:00
test_component_entrypoint.py feat(prometheus): cap series per metric for every labeled metric (#44420) 2026-10-06 01:28:44 +00:00
test_compression.py test: move tests/test_litellm root and small trees into tests/unit (#43186) 2026-09-25 11:30:43 -07:00
test_conftest_isolation.py test: isolate model cost maps in runtime registration tests (#47559) 2026-10-20 16:18:13 +00:00
test_constants.py fix(mcp): upgrade to MCP SDK 2.3.0 and raise the per-event SSE size cap (#47034) 2026-10-20 16:00:00 -07:00
test_container_router.py test: move tests/test_litellm root and small trees into tests/unit (#43186) 2026-09-25 11:30:43 -07:00
test_cost_calculation_log_level.py test: move tests/test_litellm root and small trees into tests/unit (#43186) 2026-09-25 11:30:43 -07:00
test_cost_calculator.py test(cost): cover unpriced provider slugs and explicit zero rates (#42467) 2026-10-27 18:00:40 -07:00
test_cost_map_guard.py fix(cost_map): mark fireworks_ai/minimax-m3 as vision and pin capabilities verified against live calls (#43390) 2026-10-29 00:23:11 -07:00
test_count_tokens_public_api.py fix(responses)!: count input tokens the way inference sends the request (#47363) 2026-10-20 15:13:59 +00:00
test_dashscope_image_generation.py refactor(types): replace Any with proven types in 137 files (#44478) 2026-10-05 11:01:45 -07:00
test_daybreak_model_metadata.py test: move tests/test_litellm root and small trees into tests/unit (#43186) 2026-09-25 11:30:43 -07:00
test_deepseek_model_metadata.py refactor: expose core private helpers under public names (#44871) 2026-10-07 00:05:27 +00:00
test_default_branch.py ci(lint): gate every rule on its merge-base count and cap Anys at a fixed total (#40193) 2026-10-08 05:39:29 +00:00
test_detect_changes.py ci: render lint, unit and smoke checks as <tier> / <job> with one collector per tier (#45480) 2026-10-09 04:28:41 +00:00
test_docker_no_network_on_deploy.py test: migrate local_testing SDK and provider tests out of build_and_test (#45760) 2026-10-10 23:14:25 +00:00
test_dockerfile_apk_repository.py test: move tests/test_litellm root and small trees into tests/unit (#43186) 2026-09-25 11:30:43 -07:00
test_dockerfile_bedrock_realtime_extra.py test: move tests/test_litellm root and small trees into tests/unit (#43186) 2026-09-25 11:30:43 -07:00
test_dockerfile_non_root.py test: move tests/test_litellm root and small trees into tests/unit (#43186) 2026-09-25 11:30:43 -07:00
test_drop_params_env_var.py test: move tests/test_litellm root and small trees into tests/unit (#43186) 2026-09-25 11:30:43 -07:00
test_e2e_egress_sentinel.py test: move tests/test_litellm root and small trees into tests/unit (#43186) 2026-09-25 11:30:43 -07:00
test_eager_tiktoken_load.py test: move tests/test_litellm root and small trees into tests/unit (#43186) 2026-09-25 11:30:43 -07:00
test_elevenlabs_tts_pricing.py fix(cost-map): rolling registry audit, add elevenlabs api rates, together minimax-m3.1-flash, mistral voxtral 3 and azure_ai phi-4 dates (#47623) 2026-10-23 14:57:35 -07:00
test_empty_mock_response.py fix: honor empty mock completion responses (#48162) (#48266) 2026-10-28 00:56:45 +00:00
test_env_key_doc_gate.py test: move tests/test_litellm root and small trees into tests/unit (#43186) 2026-09-25 11:30:43 -07:00
test_exception_exports.py test: move tests/test_litellm root and small trees into tests/unit (#43186) 2026-09-25 11:30:43 -07:00
test_exception_header_preservation.py fix: support OpenAI SDK 3 while retaining SDK 2 compatibility (#44927) 2026-10-09 19:45:50 -07:00
test_exception_mapping_request_attribute.py test: move tests/test_litellm root and small trees into tests/unit (#43186) 2026-09-25 11:30:43 -07:00
test_fake_openai_endpoint.py test: migrate local_testing SDK and provider tests out of build_and_test (#45760) 2026-10-10 23:14:25 +00:00
test_filter_out_litellm_params.py fix(params): validate stream_chunk_size once, before any provider call (#43222) 2026-09-26 23:01:20 +00:00
test_fireworks_serverless_model_costs.py test: move tests/test_litellm root and small trees into tests/unit (#43186) 2026-09-25 11:30:43 -07:00
test_gate_slot_lock.py test: move tests/test_litellm root and small trees into tests/unit (#43186) 2026-09-25 11:30:43 -07:00
test_gemini_3_1_flash_lite_image_pricing.py test: move tests/test_litellm root and small trees into tests/unit (#43186) 2026-09-25 11:30:43 -07:00
test_gemini_4_argon_model_metadata.py feat(gemini)!: auto-continue gemini-4-argon long decodes with continuationToken (#46240) 2026-10-15 22:54:23 +00:00
test_gemini_tts_native_audio_pricing.py test: move tests/test_litellm root and small trees into tests/unit (#43186) 2026-09-25 11:30:43 -07:00
test_get_blog_posts.py feat(proxy): add LITELLM_OFFLINE_MODE, one switch for the fetches LiteLLM makes on its own (#48096) 2026-10-23 16:54:19 -07:00
test_git_hooks.py chore(hooks): drop Conventional Branches pre-push hook in favor of litellm_ prefix (#44951) 2026-10-20 00:35:39 +00:00
test_gpt_5_4_model_metadata.py test: move tests/test_litellm root and small trees into tests/unit (#43186) 2026-09-25 11:30:43 -07:00
test_gpt_5_5_model_metadata.py test: move tests/test_litellm root and small trees into tests/unit (#43186) 2026-09-25 11:30:43 -07:00
test_gpt_image_cost_calculator.py test: move tests/test_litellm root and small trees into tests/unit (#43186) 2026-09-25 11:30:43 -07:00
test_gpt_realtime_mode.py test: move tests/test_litellm root and small trees into tests/unit (#43186) 2026-09-25 11:30:43 -07:00
test_groq_streaming_encoding.py fix(streaming): keep merged think tags balanced and split SSE lines only on CR/LF (#48755) 2026-10-28 17:00:29 -07:00
test_guardrail_exception_status_codes.py test: move tests/test_litellm root and small trees into tests/unit (#43186) 2026-09-25 11:30:43 -07:00
test_integration_run.py test(integration): isolate every multi-worker integration suite by default (#46027) 2026-10-12 03:46:06 -07:00
test_internal_context.py feat(proxy)!: model family and access group entries in model_max_budget (#47722) 2026-10-28 15:40:57 -07:00
test_lazy_imports.py fix(health)!: openai realtime health check reports healthy with a missing or invalid api key (#41233) 2026-10-29 00:40:02 -07:00
test_lint_base_counts.py revert(lint-gates): accept an empty base scan again, since zero violations is a legitimate count (#45319) 2026-10-08 08:40:08 +00:00
test_lint_workflow_diff_gates.py fix(ci): align misc unit tests with NativeCall bridge and widened e2e diff gates (#45180) 2026-10-07 16:33:49 -07:00
test_litellm_cli.py perf(cli): start lite without initializing the litellm SDK (#46803) 2026-10-17 04:53:32 +00:00
test_litellm_overhead.py test(unit): wait on the work itself instead of fixed sleeps (#46261) 2026-10-13 12:41:06 -07:00
test_litellm_params_reserved_keys.py test: move tests/test_litellm root and small trees into tests/unit (#43186) 2026-09-25 11:30:43 -07:00
test_logging.py feat(proxy)!: add suppress_startup_output to turn off startup prints (#47708) 2026-10-29 03:47:04 -07:00
test_lowest_latency_zero_tokens.py test: move tests/test_litellm root and small trees into tests/unit (#43186) 2026-09-25 11:30:43 -07:00
test_main.py fix(anthropic): drop the non-spec custom object from /v1/messages tool definitions (#48893) 2026-10-29 03:36:20 -07:00
test_main_module_header.py test: move tests/test_litellm root and small trees into tests/unit (#43186) 2026-09-25 11:30:43 -07:00
test_mistral_large_4_model_metadata.py fix(cost-map): price Mistral Large 4 at its list price now that the launch sale ended (#48825) 2026-10-28 18:12:55 +00:00
test_mistral_medium_3_5_model_metadata.py test: move tests/test_litellm root and small trees into tests/unit (#43186) 2026-09-25 11:30:43 -07:00
test_mistral_small_4_0_model_metadata.py test: move tests/test_litellm root and small trees into tests/unit (#43186) 2026-09-25 11:30:43 -07:00
test_mistral_zai_glm_5_2_model_metadata.py test: move tests/test_litellm root and small trees into tests/unit (#43186) 2026-09-25 11:30:43 -07:00
test_model_block_unblock.py fix(proxy): return 4xx instead of 500 for missing required params, invalid pagination and unknown ids (#43787) 2026-10-02 22:48:53 -07:00
test_model_cost_aliases.py test: move tests/test_litellm root and small trees into tests/unit (#43186) 2026-09-25 11:30:43 -07:00
test_model_param_helper.py refactor: expose core private helpers under public names (#44871) 2026-10-07 00:05:27 +00:00
test_model_prices_and_context_window.py fix(proxy)!: list every priced Vertex family under a vertex_ai/* grant (#41959) 2026-10-29 00:31:04 -07:00
test_model_response_normalization.py test: move tests/test_litellm root and small trees into tests/unit (#43186) 2026-09-25 11:30:43 -07:00
test_muse_spark_1_1_model_metadata.py test: move tests/test_litellm root and small trees into tests/unit (#43186) 2026-09-25 11:30:43 -07:00
test_muse_spark_1_2_model_metadata.py test: move tests/test_litellm root and small trees into tests/unit (#43186) 2026-09-25 11:30:43 -07:00
test_muse_spark_1_3_model_metadata.py feat(model_prices): add meta/muse-wonder-1.0 (#46072) 2026-10-12 19:36:52 +00:00
test_mutation_report.py test: move tests/test_litellm root and small trees into tests/unit (#43186) 2026-09-25 11:30:43 -07:00
test_nested_drop_params.py test: move tests/test_litellm root and small trees into tests/unit (#43186) 2026-09-25 11:30:43 -07:00
test_no_top_level_test_invocations.py test: move whole-unit legacy test files into tests/unit and delete dead skips (#44807) 2026-10-07 05:02:28 +00:00
test_non_chat_routes_open_llm_spans.py test: move tests/test_litellm root and small trees into tests/unit (#43186) 2026-09-25 11:30:43 -07:00
test_openai_embedding_encoding_format_default.py test: migrate local_testing SDK and provider tests out of build_and_test (#45760) 2026-10-10 23:14:25 +00:00
test_openai_record_replay_proxy.py test: move whole-unit legacy test files into tests/unit and delete dead skips (#44807) 2026-10-07 05:02:28 +00:00
test_openai_service_tier_long_context_pricing.py fix(cost_calculator): bill ultrafast prompts above 272k at the ultrafast long-context rates (#43764) 2026-09-30 07:32:16 -07:00
test_openapi_docs_gate.py ci: gate undocumented POST routes in the proxy OpenAPI document (#45843) 2026-10-10 21:51:00 +00:00
test_package_layout.py test(unit): make every tests/unit directory a package so pytest collection is unique 2026-09-20 11:50:59 +00:00
test_package_metadata.py chore(deps): require anyio>=4.14.0 to rule out the lock-waiter deadlock (#47378) 2026-10-19 12:37:47 -07:00
test_pre_commit_lint.py ci(lint): gate every rule on its merge-base count and cap Anys at a fixed total (#40193) 2026-10-08 05:39:29 +00:00
test_prisma_generate_if_needed.py test: move tests/test_litellm root and small trees into tests/unit (#43186) 2026-09-25 11:30:43 -07:00
test_private_usage_aliases.py refactor(router): fence the router surface and document the router-core boundary (#48839) 2026-10-29 00:01:52 +00:00
test_process_helpers.py test: move tests/test_litellm root and small trees into tests/unit (#43186) 2026-09-25 11:30:43 -07:00
test_project_alias_tracking.py test: move tests/test_litellm root and small trees into tests/unit (#43186) 2026-09-25 11:30:43 -07:00
test_project_tags_pydantic.py test: move tests/test_litellm root and small trees into tests/unit (#43186) 2026-09-25 11:30:43 -07:00
test_proxy_auth.py test: move tests/test_litellm root and small trees into tests/unit (#43186) 2026-09-25 11:30:43 -07:00
test_pydantic_namespaces.py test: move whole-unit legacy test files into tests/unit and delete dead skips (#44807) 2026-10-07 05:02:28 +00:00
test_rag_openai_ingestion.py test: move tests/test_litellm root and small trees into tests/unit (#43186) 2026-09-25 11:30:43 -07:00
test_rate_limit_error_unification.py refactor(proxy): expose public names for private proxy helpers (#45170) 2026-10-07 18:59:27 -07:00
test_read_rc_version.py test: move tests/test_litellm root and small trees into tests/unit (#43186) 2026-09-25 11:30:43 -07:00
test_redact_string_in_error_paths.py refactor: expose core private helpers under public names (#44871) 2026-10-07 00:05:27 +00:00
test_redis.py test: move routing, caching and callback tests out of local_testing (#45761) 2026-10-10 09:33:09 -07:00
test_redis_credential_provider.py test: move tests/test_litellm root and small trees into tests/unit (#43186) 2026-09-25 11:30:43 -07:00
test_register_model_custom_pricing.py fix(register_model)!: keep bedrock geo deployments priced across price data reloads (#41523) 2026-10-17 19:10:04 +00:00
test_register_model_zero_cost_persistence.py test: move tests/test_litellm root and small trees into tests/unit (#43186) 2026-09-25 11:30:43 -07:00
test_replicate_model_key_format.py test: move tests/test_litellm root and small trees into tests/unit (#43186) 2026-09-25 11:30:43 -07:00
test_repo_agent_skills.py feat(skills): add a litellm-setup agent skill (#47731) 2026-10-28 15:44:43 -07:00
test_responses_api_bridge_non_stream.py refactor: expose core private helpers under public names (#44871) 2026-10-07 00:05:27 +00:00
test_responses_id_security.py fix(responses): resume background response streams with stream and starting_after (#48819) 2026-10-29 02:31:22 -07:00
test_responses_streaming_container_ownership.py fix(responses): bill a background response once, on the first read or poll that sees it finished (#45703) 2026-10-15 11:00:14 -07:00
test_retrieve_batch_bedrock_dispatch.py refactor(llms): expose public names for private provider helpers (#45037) 2026-10-07 08:49:25 -07:00
test_router_block_helpers.py refactor: expose core private helpers under public names (#44871) 2026-10-07 00:05:27 +00:00
test_router_exception_redaction.py fix(streaming)!: map stream read timeouts to litellm.Timeout on chat, messages, and responses (#47198) 2026-10-18 13:58:34 +00:00
test_router_get_settings.py ci(circleci): test Redis behavior against local Redis and print short tracebacks (#44062) 2026-10-01 13:00:54 -07:00
test_router_google_genai.py test: move tests/test_litellm root and small trees into tests/unit (#43186) 2026-09-25 11:30:43 -07:00
test_router_model_cost_isolation.py fix(router): keep DB-only deployments priced across price data reloads (#48486) 2026-10-26 17:17:52 -07:00
test_router_order_fallback.py fix(router): keep the prompt caching affinity pin alive as long as the prefix's cache_control ttl (#46084) 2026-10-14 21:10:09 -07:00
test_router_per_deployment_num_retries.py fix(images): honor max_retries and prevent Router retry amplification (#48705) 2026-10-28 10:46:47 -07:00
test_router_redis_init.py test: move tests/test_litellm root and small trees into tests/unit (#43186) 2026-09-25 11:30:43 -07:00
test_router_retry_backoff_headers.py test: move tests/test_litellm root and small trees into tests/unit (#43186) 2026-09-25 11:30:43 -07:00
test_router_retry_non_retryable_errors.py test: move tests/test_litellm root and small trees into tests/unit (#43186) 2026-09-25 11:30:43 -07:00
test_router_retry_policy_update.py fix(proxy): unset DB environment variables removed from the row on every worker by row revision (#46709) 2026-10-17 03:51:05 +00:00
test_router_silent_experiment.py fix(logging): mark mirrored silent_model calls with is_silent_experiment in spend logs (#48888) 2026-10-29 05:12:50 -07:00
test_router_streaming_fallback_metadata.py refactor: expose public hidden_params accessors (#44668) 2026-10-07 00:01:33 -07:00
test_router_weighted_failover.py fix(router): let affinity pins yield to deployments that already failed this request (#32309) 2026-10-28 03:24:26 -07:00
test_ruff_strict_gate.py ci(lint): gate every rule on its merge-base count and cap Anys at a fixed total (#40193) 2026-10-08 05:39:29 +00:00
test_sambanova_model_metadata.py test: move tests/test_litellm root and small trees into tests/unit (#43186) 2026-09-25 11:30:43 -07:00
test_scaledown_model_prices.py feat(scaledown): add native text provider support (#44168) 2026-10-10 14:01:17 -07:00
test_scheduler.py fix(scheduler): remove a request's queue entry once it stops waiting (#43061) 2026-10-07 17:34:38 -07:00
test_secret_redaction.py refactor(rust-bridge): require the native extension (#48852) 2026-10-28 13:16:56 -07:00
test_seed_tracing_fixtures.py refactor(lens): connect LiteLLM to the independent Lens service (#45529) 2026-10-09 16:17:05 -07:00
test_select_ui_test_scope.py ci: render lint, unit and smoke checks as <tier> / <job> with one collector per tier (#45480) 2026-10-09 04:28:41 +00:00
test_service_logger.py test: move 76 live top-level tests to offline unit and integration coverage (#45352) 2026-10-08 10:33:35 -07:00
test_setup_wizard.py test: move tests/test_litellm root and small trees into tests/unit (#43186) 2026-09-25 11:30:43 -07:00
test_shared_session_integration.py test: move tests/test_litellm root and small trees into tests/unit (#43186) 2026-09-25 11:30:43 -07:00
test_socket_policy.py test(unit): block external sockets at import time and add a socket policy regression test 2026-09-20 08:07:50 +00:00
test_spend_logs_guard.py fix(ci): flag any schema.prisma change inside model LiteLLM_SpendLogs in the spend logs guard (LIT-10240) (#48905) 2026-10-28 15:52:15 -07:00
test_ssl_verify_unit.py test(unit): stop test modules from putting their own directory on sys.path (#43421) 2026-09-28 12:20:20 -07:00
test_stream_chunk_builder_annotations.py test: move tests/test_litellm root and small trees into tests/unit (#43186) 2026-09-25 11:30:43 -07:00
test_stream_chunk_builder_citations.py test: move tests/test_litellm root and small trees into tests/unit (#43186) 2026-09-25 11:30:43 -07:00
test_stream_chunk_builder_images.py test: move tests/test_litellm root and small trees into tests/unit (#43186) 2026-09-25 11:30:43 -07:00
test_streaming_connection_cleanup.py test: move tests/test_litellm root and small trees into tests/unit (#43186) 2026-09-25 11:30:43 -07:00
test_system_message_format_bug.py fix(prompt_templates): do not mutate messages in-place in function_call_prompt (#41676) 2026-10-15 14:37:11 -07:00
test_test_quality_gate.py ci(lint): gate every rule on its merge-base count and cap Anys at a fixed total (#40193) 2026-10-08 05:39:29 +00:00
test_thinking_enabled.py test: move tests/test_litellm root and small trees into tests/unit (#43186) 2026-09-25 11:30:43 -07:00
test_together_ai_model_metadata.py refactor(cost-map): drop the legacy max_tokens field in favor of max_input_tokens and max_output_tokens (#48471) 2026-10-26 17:03:05 -07:00
test_tool_loop.py fix(responses): replay the original Responses item order, phase, and boundaries on the next chat turn (#48912) 2026-10-29 00:56:02 -07:00
test_type_check_gate.py ci(lint): gate every rule on its merge-base count and cap Anys at a fixed total (#40193) 2026-10-08 05:39:29 +00:00
test_type_discipline_gate.py ci(lint): gate every rule on its merge-base count and cap Anys at a fixed total (#40193) 2026-10-08 05:39:29 +00:00
test_typesafe_model_metadata.py fix(cost): price typesafe/jev-1.14.0 (#46594) 2026-10-14 20:27:35 -07:00
test_unit_shard_missing_paths.py ci: render lint, unit and smoke checks as <tier> / <job> with one collector per tier (#45480) 2026-10-09 04:28:41 +00:00
test_unit_shard_per_test_timeout.py ci: render lint, unit and smoke checks as <tier> / <job> with one collector per tier (#45480) 2026-10-09 04:28:41 +00:00
test_utils.py fix(llms): move later developer messages for models declared system-first (#48897) 2026-10-29 04:12:13 -07:00
test_utils_get_model_info.py fix(bedrock): price claude_platform calls from their own cost map rows (#48150) 2026-10-24 13:54:18 -07:00
test_utils_get_optional_params.py fix(openai): warn when the GPT-5 reasoning effort gate drops a level under drop_params (#47761) 2026-10-29 04:46:29 -07:00
test_utils_module_docstring.py test: move tests/test_litellm root and small trees into tests/unit (#43186) 2026-09-25 11:30:43 -07:00
test_uuid_helper.py fix: support Python 3.15 (#48359) 2026-10-26 16:00:11 +00:00
test_vcr_classification.py test: move offline logging, secret manager and provider tests from legacy dirs into tests/unit (#45554) 2026-10-09 09:48:10 -07:00
test_vcr_conftest_common_banner.py test: move whole-unit legacy test files into tests/unit and delete dead skips (#44807) 2026-10-07 05:02:28 +00:00
test_vcr_filters.py test: move whole-unit legacy test files into tests/unit and delete dead skips (#44807) 2026-10-07 05:02:28 +00:00
test_vcr_leak_guard.py test: move whole-unit legacy test files into tests/unit and delete dead skips (#44807) 2026-10-07 05:02:28 +00:00
test_vcr_redis_persister.py test: migrate tests/llm_translation into unit, integration and e2e tiers (#45759) 2026-10-11 07:58:53 +00:00
test_vcr_safe_body_matcher.py fix(tests): stop VCR recording and replaying a test's own localhost upstream (#43346) 2026-09-26 15:49:39 -07:00
test_version.py feat(sdk): build core from an independent packaging manifest (#44340) 2026-10-08 18:40:14 -07:00
test_vertex_ai_xai_grok_prompt_caching_metadata.py test: move tests/test_litellm root and small trees into tests/unit (#43186) 2026-09-25 11:30:43 -07:00
test_video_generation.py fix(videos)!: keep model_id in proxy video ids so per-model keys resolve on status/content (#44614) 2026-10-20 05:42:59 +00:00
test_with_dashboard_node.py test: move tests/test_litellm root and small trees into tests/unit (#43186) 2026-09-25 11:30:43 -07:00
test_ws_vcr.py test: move whole-unit legacy test files into tests/unit and delete dead skips (#44807) 2026-10-07 05:02:28 +00:00
test_xai_grok_4_3_model_metadata.py test: move tests/test_litellm root and small trees into tests/unit (#43186) 2026-09-25 11:30:43 -07:00
test_xai_responses_auto_routing.py fix(router)!: stop a chat-mode Mantle deployment from rewriting the shared Responses routing row (#46263) 2026-10-17 21:39:14 +00:00