scriptoriumadmin/litellm
1
0
Fork 0
mirror of https://github.com/BerriAI/litellm.git synced 2026-09-24 00:52:24 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 91
litellm/tests/test_litellm
History
Mateo Wang deba473821
fix(cost): bill batch prompts above 272K at OpenAI's long-context batch tier (#39861) 
* fix(cost): bill batch prompts above 272K at OpenAI's long-context batch tier

* fix(cost): mirror batch long-context keys on custom pricing params

Register the two *_above_272k_tokens_batches keys on CustomPricingLiteLLMParams so a per-deployment override stays out of the shared backend key, add them to the inline model-info schema and alias-count tests, and build LiteLLM_Params and GenericLiteLLMParams through model_validate at the two dict-splat call sites so basedpyright's reportArgumentType budget ratchets down instead of blocking the new fields.

* fix(cost): add the gpt-5.5-pro batch long-context tier and ignore malformed batch tier keys

* fix(cost): bill cached batch tokens at OpenAI's cached batch rate

Adds cache_read_input_token_cost_batches and
cache_read_input_token_cost_above_272k_tokens_batches for the tiered
OpenAI entries at half the standard cached rate, bills cached batch
tokens at that rate per output line, and parses string-valued batch
rates in deployment-level model_info.

* fix(cost): bill batch cache writes at the batch cache-write rate and carry published batch rates for one-sided deployments

OpenAI's Batch table prices cache writes for gpt-6-astra, gpt-5.6, gpt-5.6-sol, gpt-5.6-terra and gpt-5.6-luna at half the standard cache-write rate, so the cost map gains cache_creation_input_token_cost_batches and its above_272k tier for those entries and batch cost pulls written tokens out of the input bucket at that rate; models without the key keep billing writes at the batch input rate.

A deployment declaring only one side of its batch pricing now carries every published batch rate of the other side (tier, cached, cache write), its own keys win, and a lone tier, cached or cache-write batch key counts as declared pricing instead of being ignored.

* fix(cost): select the batch long-context tier from any batch tier key

A deployment that declares its own flat standard input rate keeps every
published batch rate of the output direction, including the 272K output
tier, but the tier was only ever selected when an input tier key was also
present. Detect the crossed tier from any of the four batch tier keys so
the carried output, cache-read, and cache-write tiers bill at their tier
rate above 272K tokens.

* chore(proxy): keep the OpenAPI snapshot as CI generates it

* fix(cost): pick each batch price component's tier from its own keys

The batch rate picker crossed one threshold for every component, so a
deployment declaring only an output tier also moved its input, cached, and
cache-write rates to that cutoff. Each component now crosses its own
*_above_<N>k_tokens_batches keys and falls back to its flat key.

The JSON schema is regenerated with the generator as it is on main:
cost-map-guard renders the PR's cost map with the base branch's generator,
so the descriptions for the new batch cache keys move to a follow-up.

* chore(proxy): restore the lazy OpenAPI snapshot to what CI's Python 3.12 generates

The merge commit carried a snapshot regenerated on a Python 3.14 venv, which dedents
docstrings at compile time, so one description line differed from the file CI regenerates
on 3.12 and the schema.d.ts sync check went red. The snapshot is byte-identical to main again
2026-09-22 10:22:41 -07:00
..
a2a_protocol fix(a2a): send message/stream for Bedrock AgentCore streaming requests 2026-09-21 13:39:32 +00:00
batches test: migrate wave 1 phase 1 legacy tests to tests/unit 2026-09-20 09:17:57 +00:00
caching fix(cache-qdrant-semantic): wait for Qdrant upserts to be indexed 2026-09-21 21:47:56 +00:00
chat_completions test: migrate wave 1 phase 1 legacy tests to tests/unit 2026-09-20 09:17:57 +00:00
completion_extras fix(edenai): restore unrelated files the squashed PR commit had reverted 2026-09-21 19:42:37 +00:00
containers test: delete assertions that pin vendor cost map facts 2026-09-18 00:28:49 +00:00
endpoints test: migrate wave 1 phase 1 legacy tests to tests/unit 2026-09-20 09:17:57 +00:00
enterprise test: migrate wave 1 phase 1 legacy tests to tests/unit 2026-09-20 09:17:57 +00:00
expected_fine_tuning_api
expected_responses_api_request
experimental_mcp_client fix(mcp): paginate optional discovery lists 2026-09-21 11:37:55 -07:00
fixtures/together_ai_sync feat(models): add daily Together AI model registry sync script and workflow 2026-08-25 13:12:06 -07:00
google_genai fix(google_genai): drop non-object tool parameters instead of forwarding them 2026-09-19 19:19:33 -07:00
images test(images): pin scalar-array edit params survive as repeated multipart fields 2026-08-24 12:49:54 -07:00
integrations feat(arize): per-team success and error sampling rates for the Arize AX callback (#42383) 2026-09-22 11:21:40 -05:00
interactions refactor(interactions): remove expired use_legacy_interactions_schema shim 2026-09-17 20:14:07 +00:00
litellm_core_utils fix(cost): bill batch prompts above 272K at OpenAI's long-context batch tier (#39861) 2026-09-22 10:22:41 -07:00
llms fix(bedrock/claude_platform): strip body params the AWS endpoint rejects (#31203) 2026-09-22 09:57:05 -07:00
messages test: migrate phase 15 legacy tests to tests/unit 2026-09-20 11:55:43 +00:00
ocr test: migrate phase 15 legacy tests to tests/unit 2026-09-20 11:55:43 +00:00
passthrough test: migrate phase 15 legacy tests to tests/unit 2026-09-20 11:55:43 +00:00
proxy fix(proxy): never render credential-bearing config keys in the bug report (#42493) 2026-09-22 10:17:19 -07:00
rag test: migrate phase 15 legacy tests to tests/unit 2026-09-20 11:55:43 +00:00
rerank_api ci(unit): fail a hung test in 120s with a traceback instead of idling the shard to its step timeout 2026-09-19 02:53:36 -07:00
responses feat(rust-bridge): add cache and secret migration foundations (#42328) 2026-09-22 03:41:04 +00:00
router_strategy feat(router): native compact-to-fit across conversation APIs (#42074) 2026-09-21 22:52:29 -07:00
router_utils Merge pull request #42283 from BerriAI/litellm_mid_stream_fallback_walks_full_list 2026-09-21 15:14:04 -07:00
rust_bridge feat(tokenizer): preserve Python defaults with opt-in Rust dispatch (#42174) 2026-09-22 04:41:11 +00:00
secret_managers merge(main): integrate upstream Azure secret changes 2026-09-21 22:10:22 +00:00
types fix(fal_ai): price non-canonical image sizes from the nearest row and honour dump options (#42336) 2026-09-21 19:19:18 -07:00
vector_stores fix(bedrock): forward userContext in Knowledge Base Retrieve requests 2026-09-16 11:50:36 -07:00
videos test: remove phase 16 legacy test files from tests/test_litellm 2026-09-20 10:59:22 +00:00
__init__.py
conftest.py fix(bedrock): gate Invoke tool search on the model map's supports_tool_search flag 2026-09-16 18:23:14 -07:00
log.txt
readme.md
test_a2a_registry_lookup.py fix(a2a): Entra credentials own the chat route bearer over a stored api_key or authorization header 2026-09-17 16:09:36 -07:00
test_acompletion_session_reuse_e2e.py test: drop the cwd-relative sys.path.insert calls from the test suite (#37802) 2026-08-22 09:25:58 -07:00
test_add_deployment_no_master_key.py test: drop the cwd-relative sys.path.insert calls from the test suite (#37802) 2026-08-22 09:25:58 -07:00
test_aembedding_session_reuse_e2e.py test: drop the cwd-relative sys.path.insert calls from the test suite (#37802) 2026-08-22 09:25:58 -07:00
test_anthropic_beta_headers_filtering.py feat(router): native compact-to-fit across conversation APIs (#42074) 2026-09-21 22:52:29 -07:00
test_anthropic_skills_transformation.py
test_assert_ci_coverage.py test: add CircleCI integration contract foundation 2026-09-14 03:30:52 -07:00
test_assert_workflow_dir_hygiene.py feat(ci): assert .github/workflows holds only workflows, correctly named (#37616) 2026-08-20 21:36:26 +00:00
test_audio_transcription_rust_bridge.py refactor(rust_bridge): group route modules into packages and split ocr into main and rust 2026-09-16 20:34:51 +00:00
test_auto_update_price_and_context_window_file.py feat(cost_map): derive source_revision from the loaded bytes instead of a _metadata stamp 2026-09-07 17:47:51 -07:00
test_azure_ad_token_credential_resolution.py test(router): cover s3_output_bucket_name surviving the trusted credential snapshot 2026-08-17 14:51:10 -07:00
test_azure_ai_grok_4_3_model_metadata.py test: keep tests that survive correct cost-map updates 2026-09-15 22:16:08 +00:00
test_azure_ai_grok_4_6_model_metadata.py test: keep behavior tests that read the cost map for a later fixture rewrite 2026-09-18 04:52:06 +00:00
test_baseten_glm_5_3_model_metadata.py test: drop tests that pin provider-owned cost map values 2026-09-18 03:55:51 +00:00
test_batch_completion_models_all_responses.py
test_bedrock_marengo_embed_3_model_metadata.py test: drop tests that pin provider-owned cost map values 2026-09-18 03:55:51 +00:00
test_budget_ratchet_check.py test(ci): annotate new test locals as Final 2026-09-20 09:07:51 +00:00
test_chat_ui_responses_session.py
test_check_licenses.py fix(ci): retry transient PyPI license lookups 2026-08-23 09:23:35 +00:00
test_check_mcp_operation_boundary.py refactor(mcp): extract explicit operation context and dispatch 2026-09-21 12:24:15 -07:00
test_check_migrations_no_data_rewrites.py fix(migrations-check): read the table name past comments, ignore referential SET DEFAULT 2026-09-16 18:14:36 +00:00
test_check_py310_typing_imports.py fix: keep litellm importable on Python 3.10 and guard 3.11-only typing imports in CI (#39448) 2026-09-02 18:27:19 -07:00
test_check_test_quality.py test(ci): annotate new test locals as Final 2026-09-20 09:07:51 +00:00
test_check_type_discipline.py perf(ci): fan the budget checkers out across cores (#37784) 2026-08-22 22:44:58 -07:00
test_circleci_path_filter.py fix(mcp): preserve session expiry signals and scope dependency CI 2026-09-18 22:52:10 -07:00
test_circleci_rust_toolchain.py fix(ci): pin workflow toolchain dependencies 2026-09-02 12:16:25 -07:00
test_claude_fable_5_config.py test: keep the pinning-test removal free of unrelated reformatting 2026-09-18 04:27:28 +00:00
test_claude_opus_4_6_config.py test: keep the pinning-test removal free of unrelated reformatting 2026-09-18 04:27:28 +00:00
test_claude_opus_4_8_config.py test: keep the pinning-test removal free of unrelated reformatting 2026-09-18 04:27:28 +00:00
test_claude_opus_5_config.py feat(anthropic): add Claude Opus 5.5 (#42489) 2026-09-22 09:52:35 -07:00
test_claude_sonnet_5_config.py test: keep the pinning-test removal free of unrelated reformatting 2026-09-18 04:27:28 +00:00
test_cloudflare_workers_ai_model_metadata.py test: keep tests that survive correct cost-map updates 2026-09-15 22:16:08 +00:00
test_completion_timeout_resolution.py
test_component_entrypoint.py feat(proxy): share database connections across workers with an in-container pgbouncer (#39683) 2026-09-10 22:14:37 +00:00
test_compression.py
test_conftest.py test: trim the PROXY_BASE_URL fixture and regression docstrings 2026-08-19 00:56:37 -07:00
test_conftest_isolation.py test: roll back live router replay membership between tests (#36278) 2026-08-08 10:45:43 -07:00
test_constants.py test: drop the cwd-relative sys.path.insert calls from the test suite (#37802) 2026-08-22 09:25:58 -07:00
test_container_router.py
test_cost_calculation_log_level.py test: drop the cwd-relative sys.path.insert calls from the test suite (#37802) 2026-08-22 09:25:58 -07:00
test_cost_calculator.py fix(cost): bill batch prompts above 272K at OpenAI's long-context batch tier (#39861) 2026-09-22 10:22:41 -07:00
test_cost_map_guard.py ci: skip cost map file checks on PRs that leave the cost map untouched (#42406) 2026-09-21 21:40:48 -07:00
test_count_tokens_public_api.py fix(proxy): run the remaining inline token counts off the event loop 2026-09-08 18:42:38 +00:00
test_dashscope_image_generation.py test: keep the pinning-test removal free of unrelated reformatting 2026-09-18 04:27:28 +00:00
test_daybreak_model_metadata.py test: keep tests that survive correct cost-map updates 2026-09-15 22:16:08 +00:00
test_deepseek_model_metadata.py test: drop tests that pin provider-owned cost map values 2026-09-18 03:55:51 +00:00
test_default_branch.py ci: avoid duplicate default branch fetches 2026-09-07 15:28:01 -07:00
test_detect_changes.py perf(ci): gate the lint, MCP and dashboard jobs on the pull request's file list (#37559) 2026-08-19 18:32:21 -07:00
test_dockerfile_apk_repository.py fix(docker): add public Wolfi apk repo to runtime image (#39033) 2026-09-01 15:11:15 -07:00
test_dockerfile_bedrock_realtime_extra.py fix(bedrock): keep realtime SDK error range inside websocket close reason 2026-09-17 01:57:33 +00:00
test_dockerfile_non_root.py
test_drop_params_env_var.py fix(init): keep non-flag LITELLM_DROP_PARAMS values on with a warning 2026-09-07 22:13:23 -07:00
test_e2e_egress_sentinel.py ci(e2e): record the e2e suite weekly and replay it on weekdays with zero egress (#38163) 2026-08-24 23:49:03 -04:00
test_eager_tiktoken_load.py
test_env_key_doc_gate.py fix(ci): make the env-key doc gate see bare get_secret and get_secret_str reads (#35996) 2026-08-05 14:49:55 -07:00
test_exception_exports.py
test_exception_header_preservation.py fix(bedrock): keep x-amzn-RequestId on chat error responses (#40089) 2026-09-07 17:16:47 -07:00
test_exception_mapping_request_attribute.py
test_filter_out_litellm_params.py
test_fireworks_serverless_model_costs.py test: keep tests that survive correct cost-map updates 2026-09-15 22:16:08 +00:00
test_gate_slot_lock.py ci: avoid duplicate default branch fetches 2026-09-07 15:28:01 -07:00
test_gemini_3_1_flash_lite_image_pricing.py test: drop gemini-3.1-flash-lite-image capability pins 2026-09-16 19:21:50 +00:00
test_gemini_tts_native_audio_pricing.py test: keep tests that survive correct cost-map updates 2026-09-15 22:16:08 +00:00
test_get_blog_posts.py test(lint): ban blind pytest.raises(Exception) with ruff B017 (#37731) 2026-08-20 18:09:42 -07:00
test_git_hooks.py
test_gpt_5_4_model_metadata.py test: keep tests that survive correct cost-map updates 2026-09-15 22:16:08 +00:00
test_gpt_5_5_model_metadata.py test: keep tests that survive correct cost-map updates 2026-09-15 22:16:08 +00:00
test_gpt_image_cost_calculator.py test: drop remaining tests that pin cost-map vendor facts 2026-09-15 21:34:00 +00:00
test_gpt_realtime_mode.py test: keep tests that survive correct cost-map updates 2026-09-15 22:16:08 +00:00
test_groq_streaming_encoding.py
test_guardrail_exception_status_codes.py
test_lazy_imports.py perf: defer fastapi and tiktoken BPE imports out of import litellm 2026-09-17 08:59:48 +00:00
test_lint_workflow_diff_gates.py ci(lint): gate top-level tests/e2e and litellm files in the diff-scoped lint steps 2026-09-01 22:30:46 -07:00
test_litellm_params_reserved_keys.py
test_logging.py fix(logging): stamp scrubbed records with a private sentinel a caller cannot supply 2026-09-16 17:30:49 -07:00
test_lowest_latency_zero_tokens.py test: drop the cwd-relative sys.path.insert calls from the test suite (#37802) 2026-08-22 09:25:58 -07:00
test_main.py fix(edenai): restore unrelated files the squashed PR commit had reverted 2026-09-21 19:42:37 +00:00
test_main_module_header.py
test_mistral_medium_3_5_model_metadata.py test: keep tests that survive correct cost-map updates 2026-09-15 22:16:08 +00:00
test_mistral_small_4_0_model_metadata.py test: keep tests that survive correct cost-map updates 2026-09-15 22:16:08 +00:00
test_mistral_zai_glm_5_2_model_metadata.py test: keep behavior tests that read the cost map for a later fixture rewrite 2026-09-18 04:52:06 +00:00
test_model_block_unblock.py fix(proxy): serialize model reconciles so concurrent model writes stop evicting each other (#36687) 2026-08-12 13:42:26 -07:00
test_model_cost_aliases.py
test_model_param_helper.py
test_model_prices_schema.py fix(cost): bill DeepSeek V4.1 Flash and V4 Pro at their off-peak rates outside peak hours 2026-09-19 04:31:55 -07:00
test_model_response_normalization.py
test_muse_spark_1_1_model_metadata.py test: keep tests that survive correct cost-map updates 2026-09-15 22:16:08 +00:00
test_muse_spark_1_2_model_metadata.py test: keep tests that survive correct cost-map updates 2026-09-15 22:16:08 +00:00
test_muse_spark_1_3_model_metadata.py test: delete assertions that pin vendor cost map facts 2026-09-18 00:28:49 +00:00
test_mutation_report.py fix(ci): stop the mutation report publishing a score it never measured (#37825) 2026-08-21 20:15:52 -07:00
test_nested_drop_params.py
test_non_chat_routes_open_llm_spans.py fix(otel): emit LLM Call spans for speech, image, moderation, ocr and transcription (#37752) 2026-08-22 11:11:21 -07:00
test_openai_embedding_encoding_format_default.py test(embeddings): move legacy intercepts to the wire for the omitted-format path 2026-08-29 12:04:44 -07:00
test_openai_service_tier_long_context_pricing.py test: delete assertions that pin vendor cost map facts 2026-09-18 00:28:49 +00:00
test_pre_commit_lint.py ci: follow the default branch in development tooling 2026-09-07 14:34:45 -07:00
test_prisma_generate_if_needed.py fix(lint): generate the prisma client into the gate-owned venv 2026-08-06 01:54:26 -07:00
test_project_alias_tracking.py test: drop the cwd-relative sys.path.insert calls from the test suite (#37802) 2026-08-22 09:25:58 -07:00
test_project_tags_pydantic.py test(lint): ban blind pytest.raises(Exception) with ruff B017 (#37731) 2026-08-20 18:09:42 -07:00
test_proxy_auth.py test: run the 30 test files stranded in the second mirror (#37595) 2026-08-20 10:59:43 -07:00
test_rag_openai_ingestion.py
test_rate_limit_error_unification.py feat(proxy): add budget_exceeded_status_code setting to restore 429 for budget refusals 2026-09-20 07:00:10 +00:00
test_redact_string_in_error_paths.py fix(proxy): stop leaking internal exception details to clients (#39380) 2026-09-02 17:32:00 -07:00
test_redis.py test(redis): pin ElastiCache IAM signing and TLS coercion invariants 2026-09-10 10:13:03 -04:00
test_redis_credential_provider.py test(redis): pin ElastiCache IAM signing and TLS coercion invariants 2026-09-10 10:13:03 -04:00
test_register_model_custom_pricing.py fix(cost): bill off-peak rates for deployments that set only off_peak_pricing 2026-09-01 12:14:46 -07:00
test_register_model_zero_cost_persistence.py
test_replicate_model_key_format.py test: keep tests that survive correct cost-map updates 2026-09-15 22:16:08 +00:00
test_responses_api_bridge_non_stream.py test: drop the cwd-relative sys.path.insert calls from the test suite (#37802) 2026-08-22 09:25:58 -07:00
test_responses_id_security.py fix(proxy): authorize every Responses API id, not only the ones the proxy issued (#39548) 2026-09-11 11:47:05 -07:00
test_responses_streaming_container_ownership.py
test_retrieve_batch_bedrock_dispatch.py test: drop the cwd-relative sys.path.insert calls from the test suite (#37802) 2026-08-22 09:25:58 -07:00
test_router.py Merge pull request #42283 from BerriAI/litellm_mid_stream_fallback_walks_full_list 2026-09-21 15:14:04 -07:00
test_router_block_helpers.py
test_router_exception_redaction.py test(router): let monkeypatch own expose_router_debug_in_errors (#37848) 2026-08-22 09:02:40 -07:00
test_router_google_genai.py test: drop the cwd-relative sys.path.insert calls from the test suite (#37802) 2026-08-22 09:25:58 -07:00
test_router_model_cost_isolation.py fix(router): preserve discovered limits and model info fallbacks 2026-09-17 00:23:50 +00:00
test_router_order_fallback.py fix(router): skip the refusing deployment when retrying a non-transient error 2026-09-05 22:25:13 -07:00
test_router_per_deployment_num_retries.py refactor(router): resolve retry policy by exception MRO and add DefaultRetries 2026-09-04 16:09:01 -07:00
test_router_redis_init.py
test_router_retry_backoff_headers.py test: run the 30 test files stranded in the second mirror (#37595) 2026-08-20 10:59:43 -07:00
test_router_retry_non_retryable_errors.py fix(timing): union provider timing windows and anchor detailed pre-processing at receive time 2026-09-19 00:48:17 +00:00
test_router_retry_policy_update.py fix(proxy): persist only the router settings keys the request set 2026-09-18 14:01:05 -07:00
test_router_silent_experiment.py fix(router): snapshot shadow kwargs per target so concurrent shadows never share metadata 2026-09-16 04:31:43 +00:00
test_router_streaming_fallback_metadata.py
test_router_weighted_failover.py fix(router): skip the refusing deployment when retrying a non-transient error 2026-09-05 22:25:13 -07:00
test_ruff_strict_gate.py fix: address cross-version CI failures 2026-09-02 14:17:19 -07:00
test_sambanova_model_metadata.py test: keep behavior tests that read the cost map for a later fixture rewrite 2026-09-18 04:52:06 +00:00
test_secret_redaction.py fix(bedrock): keep batch S3 credentials out of chat requests and debug logs 2026-09-21 20:42:04 +00:00
test_select_ui_test_scope.py test(ui): align workflow expectations with Vitest 4 2026-09-08 16:13:58 -07:00
test_service_logger.py
test_setup_wizard.py
test_shared_session_integration.py test: drop the cwd-relative sys.path.insert calls from the test suite (#37802) 2026-08-22 09:25:58 -07:00
test_ssl_verify_unit.py
test_stream_chunk_builder_annotations.py
test_stream_chunk_builder_citations.py fix(streaming): join block-list citation deltas without extra nesting 2026-08-28 13:56:07 -07:00
test_stream_chunk_builder_images.py test: run the 30 test files stranded in the second mirror (#37595) 2026-08-20 10:59:43 -07:00
test_streaming_connection_cleanup.py test: drop the cwd-relative sys.path.insert calls from the test suite (#37802) 2026-08-22 09:25:58 -07:00
test_sync_together_ai_models.py feat(cost_map): derive source_revision from the loaded bytes instead of a _metadata stamp 2026-09-07 17:47:51 -07:00
test_system_message_format_bug.py
test_test_quality_gate.py ci(tests): wire tests/unit into CircleCI and drain legacy unit shards green 2026-09-20 07:05:42 +00:00
test_thinking_enabled.py test: drop restating comment and wrap long call in thinking tests 2026-08-18 19:55:22 -07:00
test_together_ai_model_metadata.py test: delete assertions that pin vendor cost map facts 2026-09-18 00:28:49 +00:00
test_type_check_gate.py fix(lint): retire the single-slot base-counts cache 2026-08-06 02:23:52 -07:00
test_type_discipline_gate.py fix(lint): pick the merge-aware base so in-progress merges are not blamed for base drift 2026-08-04 17:58:41 -07:00
test_typesafe_model_metadata.py feat(proxy): add TypeSafe Jev passthrough spend tracking 2026-09-17 15:53:19 +00:00
test_unit_shard_per_test_timeout.py test(ci): drop the structure-only assertion on the shard script; the parametrized hang test covers both invocations 2026-09-19 03:20:48 -07:00
test_utils.py fix(cost): bill batch prompts above 272K at OpenAI's long-context batch tier (#39861) 2026-09-22 10:22:41 -07:00
test_utils_module_docstring.py
test_uuid_helper.py
test_vcr_safe_body_matcher.py
test_vertex_ai_xai_grok_prompt_caching_metadata.py test(pricing): assert cache-priced vertex grok rows advertise supports_prompt_caching (#41526) 2026-09-21 20:58:11 -07:00
test_video_generation.py test: delete assertions that pin vendor cost map facts 2026-09-18 00:28:49 +00:00
test_with_dashboard_node.py fix(bootstrap): fail fast when nvm cannot activate the pinned node 2026-08-04 21:18:58 -07:00
test_xai_grok_4_3_model_metadata.py test: keep tests that survive correct cost-map updates 2026-09-15 22:16:08 +00:00
test_xai_responses_auto_routing.py fix(xai): keep 'instructions' on the xAI Responses API so system messages survive web_search bridging 2026-09-16 01:35:13 +00:00

readme.md

Testing for litellm/

This directory 1:1 maps the the litellm/ directory, and can only contain mocked tests.

The point of this is to:

  1. Increase test coverage of litellm/
  2. Make it easy for contributors to add tests for the litellm/ package and easily run tests without needing LLM API keys.

File name conventions

  • litellm/proxy/test_caching_routes.py maps to litellm/proxy/caching_routes.py
  • test_<filename>.py maps to litellm/<filename>.py