zoveki 33457d3ac9 fix(proxy): check the model in every OpenAI websocket passthrough frame (#48979) ... * test(proxy): cover per-frame model checks and session pricing on the OpenAI websocket passthrough * fix(proxy): check the model in every OpenAI websocket passthrough frame and price the session * fix(proxy): bill each OpenAI websocket turn on its own row and run full auth per frame Each finished turn (completed, incomplete, or failed) now logs its own spend row under the model and access groups that allowed it, so group budgets see spend as turns end instead of at close. Every model a frame names goes through the key's full auth, and a stored prompt reference with no model is refused for a key held to some models. A frame still in flight when the client leaves is recorded before the relay gives up on the send * fix(proxy): keep the stored prompt refusal's first sentence inside the websocket close reason The close reason carries the refusal's first sentence, and the stored prompt sentence ran past the 123 bytes a close frame allows, so it was cut mid-word. The sentence is shorter now and the rest of the explanation stays in the error event. * fix(proxy): bill pipelined websocket turns to their own frame and close the review gaps * fix(proxy): log no extra websocket spend row for frames after the last turn * fix(proxy): price websocket Responses turns by served tier and hosted tools, and bill turns a client walked out of A websocket Responses turn is now priced through the same Responses cost path the HTTP passthrough uses, so a priority-tier answer and a hosted web search cost what they cost over HTTP. When the client leaves mid-answer, the gateway keeps reading the provider's socket until that turn ends, up to 10 minutes, so the turn the provider still bills reaches spend * fix(proxy): bill websocket turns by lane, skip turns OpenAI refuses, and stop frame checks counting the session's own reservation An error event refuses a queued response.create, so the next turn is no longer charged to the refused frame's model. Turns on separate stream_id lanes are charged to the frame that started each one. The frame model gate settles the connect reservation before it re-runs auth, so a key with less budget left than one turn's max cost is no longer refused on its first frame. * fix(proxy): bill each websocket lane cut off at session end on its own row When a client leaves while turns on several stream_id lanes are still running, each unfinished turn now gets its own spend row under the groups and model its own frame asked for, instead of one merged row. Also pins the PR's new single-assigned test variables as Final and swaps datetime.now() in the logging-handler and success-handler fixtures for fixed timestamps. * test(proxy): mark the remaining new websocket test variables Final Builds the logging mocks with model_call_details in the constructor instead of setting it afterwards. * fix(proxy): check every model the realtime URL names before the OpenAI websocket relay connects * fix(proxy): keep reading running answers after a refused websocket frame so they are billed * fix(proxy): bill realtime transcription sessions on the OpenAI websocket passthrough A transcription session names its model only in session.updated under audio.input.transcription, and reports usage on each completed transcription instead of response.done, so its row logged unknown at $0 and a budgeted key never spent. * test(proxy): mark the turn-taking client's delivered list Final * fix(proxy): hold a websocket session's parallel slot until the socket closes Each finished turn logged its spend row under the session's own call id, so the first turn's success callback freed the key's max_parallel_requests slot while the socket was still open, and a key limited to one session could open several at once. The session now claims its slot under its own call id, every turn row logs under a call id of its own, and the slot is released when the session ends, as before Priced turn rows also take their row's call id as their id, so two turns that start in the same instant no longer share a spend log id and drop one row * fix(proxy): bill realtime transcription clips that finish outside an answer A conversation.item.input_audio_transcription.completed frame that arrived after a session's last response.done was never billed, because the session end only owes a row for a turn still running. A clip that completes while no answer is running now logs its own row, and a clip inside a running answer stays in that answer's row Each row also carries the session's latest session.updated frame, since later rows lost the transcription model it names and priced clips against the realtime model. A newer session.updated replaces the older one, so the stale model is never read first It also drops the docstrings this PR had added, since the names already say what each helper does * fix(cost): price a realtime transcription clip at the transcription model the session set A realtime conversation session opens with session.created naming the realtime model and no transcription model, and the client sets the transcription model later in session.update. The cost code took the first session frame and fell back to its session model, so a clip was priced at the realtime model. That model has no per-second rate, so a gpt-transcribe clip cost $0, and a token-billed clip used the realtime model's audio rate instead of the transcription model's. Now the newest session frame that names a transcription model sets the price, and the first session model is only the fallback when no frame names one, as before * fix(cost): price each realtime transcription clip at the model set when it ran A session that changes its transcription model mid-way kept every logged clip on the newest model. Each clip now takes the transcription model from the last session event before it, and falls back to the session's own model * fix(passthrough): keep the session update a buffered clip ran under until its answer is billed A transcription that completes while an answer streams waits in the buffer for that answer's response.done. A session.updated in between used to drop the earlier update, so the clip was priced at the opening configuration. An earlier update now stays while an unbilled clip follows it, and drops at the next update once that clip is billed. * fix(passthrough): keep reading a websocket turn the client left before OpenAI started it A client that sends response.create and hangs up before OpenAI answers with response.created made the walkout checks see no turn in flight, so the gateway closed the provider socket and the turn OpenAI still ran and billed never reached spend. The relay now counts every forwarded create until the provider starts or refuses it, and keeps draining while one is owed * fix(passthrough): settle a websocket turn that ends without its start frame A turn the provider ended without sending response.created first left its create counted as not started, so a client leaving later held the provider socket for the whole walkout drain with nothing left to bill. A turn's end now settles one waiting create when its lane saw no start frame * fix(proxy): refuse unparseable and unroutable websocket frames and hold legacy slots per socket A client frame too deep or too long for the JSON parser closed the provider socket mid-turn, so the turn in flight was never billed. The gate now refuses that frame with an error event and keeps draining the open turn. A stream_id OpenAI rejects comes back as an error that names no lane, which dropped another turn's authorization. The gate now refuses those frames before they reach OpenAI. With the legacy parallel request limiter, each per-turn spend row freed the key's slot while the socket stayed open. Websocket passthrough rows now free nothing, and the socket frees its slot once when it closes. --------- Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>		2026-11-02 09:48:59 -08:00
..
a2a_protocol	fix(a2a)!: bill completion-bridge agents at their own prices on /a2a send and stream (#49150)	2026-10-30 19:43:45 -07:00
accounting	feat(accounting): add accounting contracts, PricingSnapshot and ledger schema (LIT-10193) (#48862)	2026-10-28 14:39:25 -07:00
anthropic_interface	fix(proxy): send a real error event when a /v1/messages stream fails (#41826)	2026-09-25 15:36:35 -07:00
batch_completion	test: move routing, caching and callback tests out of local_testing (#45761)	2026-10-10 09:33:09 -07:00
batches	feat(accounting): capture request usage through the durable meter (#49133)	2026-10-31 07:32:25 -07:00
caching	test: deflake the json accumulator drain test and the armed routing prefetch test (#49505)	2026-10-31 15:14:33 -07:00
completion_extras	fix(streaming): rebuild each choice of an n>1 chat stream on its own (#49002)	2026-10-29 18:29:14 -07:00
compression	test: migrate tests/llm_translation into unit, integration and e2e tiers (#45759)	2026-10-11 07:58:53 +00:00
containers	refactor(typing): clear 1181 basedpyright Any errors across 431 files (#47789)	2026-10-24 11:31:09 -07:00
db_scripts	fix(vector-stores): keep proxy context out of provider vector store metadata (#47021)	2026-10-17 12:51:47 -07:00
decisions	fix(decisions)!: honor a deployment's own supports_audio_input and supports_video_input flags (#49207)	2026-10-31 01:09:46 -07:00
endpoints	feat(vertex-ai)!: support gemini 3.1 flash tts (#31915)	2026-10-21 04:18:42 +00:00
enterprise	Return configured guardrail native findings (#49217) (#49460)	2026-10-31 23:37:45 +00:00
expected_fine_tuning_api	test: move tests/test_litellm/llms into tests/unit/llms (#43191)	2026-09-25 12:43:23 -07:00
expected_responses_api_request	test: move tests/test_litellm core utils, routing, responses, caching and rust_bridge into tests/unit (#43199)	2026-09-25 17:10:13 -07:00
experimental_mcp_client	fix(mcp): load deprecated package aliases on demand (#48615)	2026-10-27 19:46:19 +00:00
extraction	feat(extraction): add /v1/extraction with Cloudflare Clairic support (#47473)	2026-10-20 15:51:45 -07:00
files	fix(hosted_vllm): accept the vLLM production-stack router's file and batch objects (#49068)	2026-10-30 23:11:34 -07:00
fine_tuning	fix(fine_tuning): stop forwarding api_version to the OpenAI client (#44995)	2026-10-21 01:58:25 +00:00
fixtures	test: move tests/test_litellm root and small trees into tests/unit (#43186)	2026-09-25 11:30:43 -07:00
gateway	fix(proxy): keep the litellm command light so spawned workers answer uvicorn's first health ping (#45039)	2026-10-29 10:25:14 -07:00
google_genai	fix(google_genai): read one-object and null spellings of repeated fields in countTokens and generateContent (#48771)	2026-10-28 21:46:06 -07:00
harness	fix(streaming)!: send the include_usage chunk with an empty choices array (#47109)	2026-10-21 02:26:05 +00:00
images	feat(dashscope): add image edits and text-to-speech (#48998)	2026-10-29 17:48:38 -07:00
integration_support	test(mcp): cancel disconnected HTTP tool handlers (#49436)	2026-10-31 16:41:49 -07:00
integrations	fix(anthropic): answer 400 naming a non-string role or tool call id instead of 500 (#49237)	2026-10-31 21:13:24 +00:00
interactions	fix(streaming): keep merged think tags balanced and split SSE lines only on CR/LF (#48755)	2026-10-28 17:00:29 -07:00
litellm_core_utils	fix(streaming): preserve served models in OpenRouter streams (#49498)	2026-10-31 16:12:51 -07:00
litellm_proxy_extras	feat(proxy): replace the pre-migration row check with a rolled-back dry run of the pending migrations (#48961)	2026-10-29 09:39:26 -07:00
llms	fix(anthropic): stream azure_ai natively on an Azure OpenAI host on /v1/messages (#49216)	2026-11-02 09:31:05 -08:00
mcp_client	refactor(mcp): drop the experimental label from the MCP server and client paths (#48513)	2026-10-27 17:47:50 +00:00
models	fix(proxy): allow model instances in LiteLLM_EndUserTable validation (#41677)	2026-10-15 10:57:26 -07:00
ocr	test(rust-bridge): relocate dispatch tests before routing changes (#46926)	2026-10-16 12:29:33 -07:00
passthrough	fix(vllm): send the pass-through key as a bearer on both /vllm routes (#46861)	2026-10-30 08:57:48 -07:00
proxy	fix(proxy): check the model in every OpenAI websocket passthrough frame (#48979)	2026-11-02 09:48:59 -08:00
rag	fix(rag)!: use team provider credentials and allow viewer and JWT team access (#45497)	2026-10-17 23:51:48 +00:00
realtime_api	feat(soniox): add realtime speech-to-text on /v1/realtime (#44822)	2026-10-29 07:58:38 -07:00
repositories	refactor(proxy): inject gateway activity through a typed repository (#49116)	2026-10-30 16:25:11 -07:00
rerank_api	test: migrate tests/llm_translation into unit, integration and e2e tiers (#45759)	2026-10-11 07:58:53 +00:00
responses	fix(responses): serialize sequence_number on every bridged streaming event (#44944)	2026-11-02 09:01:58 -08:00
router_strategy	fix(router): restore four base behaviors the RoutingContext refactor changed (#49395)	2026-10-31 22:58:34 +00:00
router_utils	fix(router): keep encrypted reasoning when a fallback hops between same-key deployments with no api_base (#49396)	2026-11-02 05:47:04 -08:00
rust_bridge	refactor(router): route the router family through RoutingContext and close the outside surface (#49118)	2026-10-30 19:18:54 -07:00
sandbox	fix(streaming): keep merged think tags balanced and split SSE lines only on CR/LF (#48755)	2026-10-28 17:00:29 -07:00
search	fix(search): price Tavily advanced and Linkup deep searches at their own cost map entries (#47411)	2026-10-19 23:32:31 -07:00
secret_managers	feat(secret_managers): resolve oidc/google api_key references per request (#47703)	2026-10-29 08:44:46 -07:00
skills	feat(skills): add a litellm-setup agent skill (#47731)	2026-10-28 15:44:43 -07:00
systemone	fix(databricks): reuse base OAuth for decisions (#48395) (#48527)	2026-10-26 20:18:16 -07:00
telemetry	feat(telemetry): add AggregatingSink, fixed-bucket histograms, Report and HttpExporter (#45487)	2026-10-13 18:33:01 -07:00
test_router	fix(responses): serialize sequence_number on every bridged streaming event (#44944)	2026-11-02 09:01:58 -08:00
text_completions	fix(proxy)!: bill streamed /v1/completions without stream_options like chat (#45366)	2026-10-29 17:18:51 -07:00
tracing	fix(proxy): cooldown errors carry 429, Sentry starts at boot without request bodies, tracing failures log at ERROR, exporter counters exported (#46344)	2026-10-16 00:53:52 +00:00
types	fix(responses): serialize sequence_number on every bridged streaming event (#44944)	2026-11-02 09:01:58 -08:00
vector_stores	fix(exceptions)!: honor status code when mapping OpenAI invalid_request_error exceptions (#32537)	2026-10-29 08:05:47 -07:00
videos	test: migrate phase 16 legacy tests to tests/unit	2026-09-20 10:59:02 +00:00
__init__.py	test(unit): add package markers to migrated unit test directories	2026-09-20 11:53:14 +00:00
AGENTS.md	feat(vector_stores): add OpenSearch k-NN search as a vector store provider (#47759)	2026-10-29 08:15:05 -07:00
conftest.py	fix(llms): move later developer messages for models declared system-first (#48897)	2026-10-29 04:12:13 -07:00
create_mock_standard_logging_payload.py	test: migrate legacy unit dirs and unrun folders into tests/unit, tests/integration and tests/e2e (#45940)	2026-10-11 06:24:57 -07:00
test_a2a_registry_lookup.py	test: move tests/test_litellm root and small trees into tests/unit (#43186)	2026-09-25 11:30:43 -07:00
test_acompletion_session_reuse_e2e.py	test: move tests/test_litellm root and small trees into tests/unit (#43186)	2026-09-25 11:30:43 -07:00
test_add_deployment_no_master_key.py	test: move tests/test_litellm root and small trees into tests/unit (#43186)	2026-09-25 11:30:43 -07:00
test_aembedding_session_reuse_e2e.py	refactor(llms): expose public names for private provider helpers (#45037)	2026-10-07 08:49:25 -07:00
test_anthropic_beta_gated_fields.py	fix(anthropic)!: drop beta-gated body fields under drop_params (#48793)	2026-10-29 06:25:31 -07:00
test_anthropic_beta_headers_filtering.py	fix(anthropic)!: sync the beta header allowlist with what anthropic, vertex_ai, and azure_ai accept (#49409)	2026-11-02 05:57:51 -08:00
test_anthropic_skills_transformation.py	feat(anthropic): workload identity federation and pluggable identity sources (#44448)	2026-10-03 17:08:30 -07:00
test_assert_ci_coverage.py	docs(agents): keep directory rules in nested AGENTS.md and pin integration tests to CircleCI (#46796)	2026-10-16 14:57:49 -07:00
test_assert_secret_absent.py	ci: run the integration groups on GitHub Actions (#48985)	2026-10-29 12:50:08 -07:00
test_assert_workflow_dir_hygiene.py	test: move tests/test_litellm root and small trees into tests/unit (#43186)	2026-09-25 11:30:43 -07:00
test_audio_transcription_rust_bridge.py	fix(bedrock): omit system prompt from audio transcription and report its token usage (#41565)	2026-10-28 19:23:15 -07:00
test_azure_ad_token_credential_resolution.py	test: move tests/test_litellm root and small trees into tests/unit (#43186)	2026-09-25 11:30:43 -07:00
test_azure_ai_grok_4_3_model_metadata.py	test: move tests/test_litellm root and small trees into tests/unit (#43186)	2026-09-25 11:30:43 -07:00
test_azure_ai_grok_4_6_model_metadata.py	test: move tests/test_litellm root and small trees into tests/unit (#43186)	2026-09-25 11:30:43 -07:00
test_baseten_glm_5_3_model_metadata.py	test: move tests/test_litellm root and small trees into tests/unit (#43186)	2026-09-25 11:30:43 -07:00
test_batch_completion_models_all_responses.py	test: move routing, caching and callback tests out of local_testing (#45761)	2026-10-10 09:33:09 -07:00
test_bedrock_marengo_embed_3_model_metadata.py	test: move tests/test_litellm root and small trees into tests/unit (#43186)	2026-09-25 11:30:43 -07:00
test_build_core_distribution.py	test: move packaging and install checks off CircleCI build_and_test (#45757)	2026-10-10 09:33:51 -07:00
test_cerebras_llama_model_metadata.py	feat(models): add Cerebras Llama 3.1 hyphenated aliases (cerebras/llama-3.1-8b, cerebras/llama-3.1-70b) (#48157) (#48158)	2026-10-24 17:16:33 -07:00
test_chat_ui_responses_session.py	test: move tests/test_litellm root and small trees into tests/unit (#43186)	2026-09-25 11:30:43 -07:00
test_check_licenses.py	feat(proxy): embed enterprise LiteAdmin MCP in LiteLLM images (#44610)	2026-10-05 15:24:34 -07:00
test_check_mcp_operation_boundary.py	refactor(mcp): drop the experimental label from the MCP server and client paths (#48513)	2026-10-27 17:47:50 +00:00
test_check_migrations_lock_timeout.py	fix(proxy-extras): bound migration lock waits with SET LOCAL lock_timeout in the migration SQL (#46329)	2026-10-13 20:51:03 -07:00
test_check_migrations_no_data_rewrites.py	fix(proxy-extras): build the SpendLogs indexes in the migration job instead of in migrations (#43948)	2026-10-01 14:12:22 -07:00
test_check_openapi_docs.py	ci: gate undocumented POST routes in the proxy OpenAPI document (#45843)	2026-10-10 21:51:00 +00:00
test_check_py310_typing_imports.py	test: move tests/test_litellm root and small trees into tests/unit (#43186)	2026-09-25 11:30:43 -07:00
test_check_test_quality.py	test(integration): hand every harness server a port from one per-worker block (#46761)	2026-10-15 16:56:40 -07:00
test_check_type_discipline.py	ci(lint): gate every rule on its merge-base count and cap Anys at a fixed total (#40193)	2026-10-08 05:39:29 +00:00
test_check_unbounded_in_lists.py	test: migrate legacy unit dirs and unrun folders into tests/unit, tests/integration and tests/e2e (#45940)	2026-10-11 06:24:57 -07:00
test_check_windows_wheel_install.py	test: move packaging and install checks off CircleCI build_and_test (#45757)	2026-10-10 09:33:51 -07:00
test_circleci_checkout.py	ci(circleci): clone the public repo over HTTPS so main jobs stop failing at checkout (#49394)	2026-10-31 02:53:54 -07:00
test_circleci_path_filter.py	refactor(mcp): drop the experimental label from the MCP server and client paths (#48513)	2026-10-27 17:47:50 +00:00
test_circleci_run_migration_tests.py	test: migrate proxy security, schema migration and MCP legacy tests (#45937)	2026-10-11 16:17:24 -07:00
test_circleci_rust_toolchain.py	test: migrate tests/llm_translation into unit, integration and e2e tiers (#45759)	2026-10-11 07:58:53 +00:00
test_claude_fable_5_config.py	refactor(llms): expose public names for private provider helpers (#45037)	2026-10-07 08:49:25 -07:00
test_claude_haiku_5_5_config.py	fix(model_prices): consolidate claude-haiku-5-5 over-100k pricing and capability flags (#45151)	2026-10-07 14:14:47 -07:00
test_claude_opus_4_6_config.py	test: move tests/test_litellm root and small trees into tests/unit (#43186)	2026-09-25 11:30:43 -07:00
test_claude_opus_4_8_config.py	test: move tests/test_litellm root and small trees into tests/unit (#43186)	2026-09-25 11:30:43 -07:00
test_claude_opus_5_config.py	refactor(llms): expose public names for private provider helpers (#45037)	2026-10-07 08:49:25 -07:00
test_claude_sonnet_5_config.py	refactor(cost-map): remove openrouter rows, part 5 of 5 (#45669)	2026-10-09 22:48:55 +00:00
test_cloudflare_workers_ai_model_metadata.py	test: move tests/test_litellm root and small trees into tests/unit (#43186)	2026-09-25 11:30:43 -07:00
test_completion_timeout_resolution.py	fix(router)!: bound fake-streamed calls by the request timeout, not stream_timeout (#47162)	2026-10-17 22:08:02 +00:00
test_component_entrypoint.py	feat(prometheus): cap series per metric for every labeled metric (#44420)	2026-10-06 01:28:44 +00:00
test_compression.py	test: move tests/test_litellm root and small trees into tests/unit (#43186)	2026-09-25 11:30:43 -07:00
test_conftest_isolation.py	refactor(router): extract PythonCore behind RouterCore and pass the router its config (#49119)	2026-10-31 12:54:23 -07:00
test_constants.py	fix(router): cap the default OTPM reservation and reconcile token counters after each call (#49054)	2026-10-29 20:32:04 -07:00
test_container_router.py	test: move tests/test_litellm root and small trees into tests/unit (#43186)	2026-09-25 11:30:43 -07:00
test_cost_calculation_log_level.py	test: move tests/test_litellm root and small trees into tests/unit (#43186)	2026-09-25 11:30:43 -07:00
test_cost_calculator.py	fix(proxy): check the model in every OpenAI websocket passthrough frame (#48979)	2026-11-02 09:48:59 -08:00
test_cost_map_guard.py	fix(cost_map): mark fireworks_ai/minimax-m3 as vision and pin capabilities verified against live calls (#43390)	2026-10-29 00:23:11 -07:00
test_count_tokens_public_api.py	fix(responses)!: count input tokens the way inference sends the request (#47363)	2026-10-20 15:13:59 +00:00
test_dashscope_image_generation.py	feat(dashscope): add image edits and text-to-speech (#48998)	2026-10-29 17:48:38 -07:00
test_daybreak_model_metadata.py	test: move tests/test_litellm root and small trees into tests/unit (#43186)	2026-09-25 11:30:43 -07:00
test_deepseek_model_metadata.py	refactor: expose core private helpers under public names (#44871)	2026-10-07 00:05:27 +00:00
test_default_branch.py	feat(lite): offer each model group's reasoning efforts in pi, OpenCode and Codex (#48949)	2026-10-29 18:15:44 -07:00
test_detect_changes.py	ci: render lint, unit and smoke checks as <tier> / <job> with one collector per tier (#45480)	2026-10-09 04:28:41 +00:00
test_docker_no_network_on_deploy.py	test: migrate local_testing SDK and provider tests out of build_and_test (#45760)	2026-10-10 23:14:25 +00:00
test_dockerfile_apk_repository.py	test: move tests/test_litellm root and small trees into tests/unit (#43186)	2026-09-25 11:30:43 -07:00
test_dockerfile_bedrock_realtime_extra.py	test: move tests/test_litellm root and small trees into tests/unit (#43186)	2026-09-25 11:30:43 -07:00
test_dockerfile_non_root.py	test: move tests/test_litellm root and small trees into tests/unit (#43186)	2026-09-25 11:30:43 -07:00
test_drop_params_env_var.py	test: move tests/test_litellm root and small trees into tests/unit (#43186)	2026-09-25 11:30:43 -07:00
test_e2e_egress_sentinel.py	test: move tests/test_litellm root and small trees into tests/unit (#43186)	2026-09-25 11:30:43 -07:00
test_eager_tiktoken_load.py	test: move tests/test_litellm root and small trees into tests/unit (#43186)	2026-09-25 11:30:43 -07:00
test_elevenlabs_tts_pricing.py	fix(cost-map): rolling registry audit, add elevenlabs api rates, together minimax-m3.1-flash, mistral voxtral 3 and azure_ai phi-4 dates (#47623)	2026-10-23 14:57:35 -07:00
test_empty_mock_response.py	fix: honor empty mock completion responses (#48162) (#48266)	2026-10-28 00:56:45 +00:00
test_env_key_doc_gate.py	test: move tests/test_litellm root and small trees into tests/unit (#43186)	2026-09-25 11:30:43 -07:00
test_exception_exports.py	test: move tests/test_litellm root and small trees into tests/unit (#43186)	2026-09-25 11:30:43 -07:00
test_exception_header_preservation.py	fix(openai): keep the upstream error body on e.response for OpenAI and Azure errors (#49079)	2026-10-29 23:30:02 -07:00
test_exception_mapping_request_attribute.py	test: move tests/test_litellm root and small trees into tests/unit (#43186)	2026-09-25 11:30:43 -07:00
test_fake_openai_endpoint.py	test: migrate local_testing SDK and provider tests out of build_and_test (#45760)	2026-10-10 23:14:25 +00:00
test_filter_out_litellm_params.py	fix(params): validate stream_chunk_size once, before any provider call (#43222)	2026-09-26 23:01:20 +00:00
test_fireworks_serverless_model_costs.py	test: move tests/test_litellm root and small trees into tests/unit (#43186)	2026-09-25 11:30:43 -07:00
test_gate_slot_lock.py	test: move tests/test_litellm root and small trees into tests/unit (#43186)	2026-09-25 11:30:43 -07:00
test_gemini_3_1_flash_lite_image_pricing.py	test: move tests/test_litellm root and small trees into tests/unit (#43186)	2026-09-25 11:30:43 -07:00
test_gemini_4_argon_model_metadata.py	feat(gemini)!: auto-continue gemini-4-argon long decodes with continuationToken (#46240)	2026-10-15 22:54:23 +00:00
test_gemini_tts_native_audio_pricing.py	test: move tests/test_litellm root and small trees into tests/unit (#43186)	2026-09-25 11:30:43 -07:00
test_get_blog_posts.py	feat(proxy): add LITELLM_OFFLINE_MODE, one switch for the fetches LiteLLM makes on its own (#48096)	2026-10-23 16:54:19 -07:00
test_git_hooks.py	chore(hooks): drop Conventional Branches pre-push hook in favor of litellm_ prefix (#44951)	2026-10-20 00:35:39 +00:00
test_gpt_5_4_model_metadata.py	test: move tests/test_litellm root and small trees into tests/unit (#43186)	2026-09-25 11:30:43 -07:00
test_gpt_5_5_model_metadata.py	test: move tests/test_litellm root and small trees into tests/unit (#43186)	2026-09-25 11:30:43 -07:00
test_gpt_image_cost_calculator.py	test: move tests/test_litellm root and small trees into tests/unit (#43186)	2026-09-25 11:30:43 -07:00
test_gpt_realtime_mode.py	test: move tests/test_litellm root and small trees into tests/unit (#43186)	2026-09-25 11:30:43 -07:00
test_groq_streaming_encoding.py	fix(streaming): keep merged think tags balanced and split SSE lines only on CR/LF (#48755)	2026-10-28 17:00:29 -07:00
test_guardrail_exception_status_codes.py	test: move tests/test_litellm root and small trees into tests/unit (#43186)	2026-09-25 11:30:43 -07:00
test_guardrails.py	Add guardrails.apply SDK helper (#49217) (#49469)	2026-10-31 23:37:45 +00:00
test_integration_run.py	test(integration): isolate every multi-worker integration suite by default (#46027)	2026-10-12 03:46:06 -07:00
test_internal_context.py	feat(proxy)!: model family and access group entries in model_max_budget (#47722)	2026-10-28 15:40:57 -07:00
test_lazy_imports.py	fix(health)!: openai realtime health check reports healthy with a missing or invalid api key (#41233)	2026-10-29 00:40:02 -07:00
test_lint_base_counts.py	revert(lint-gates): accept an empty base scan again, since zero violations is a legitimate count (#45319)	2026-10-08 08:40:08 +00:00
test_lint_workflow_diff_gates.py	fix(ci): measure PR gates from the target branch's current tip (#49078)	2026-10-29 18:28:16 -07:00
test_litellm_cli.py	fix(proxy): keep the litellm command light so spawned workers answer uvicorn's first health ping (#45039)	2026-10-29 10:25:14 -07:00
test_litellm_overhead.py	test(unit): wait on the work itself instead of fixed sleeps (#46261)	2026-10-13 12:41:06 -07:00
test_litellm_params_reserved_keys.py	test: move tests/test_litellm root and small trees into tests/unit (#43186)	2026-09-25 11:30:43 -07:00
test_logging.py	feat(proxy)!: add suppress_startup_output to turn off startup prints (#47708)	2026-10-29 03:47:04 -07:00
test_lowest_latency_zero_tokens.py	test: move tests/test_litellm root and small trees into tests/unit (#43186)	2026-09-25 11:30:43 -07:00
test_main.py	fix(gemini): add thought_signature_in_tool_call_id to keep tool call ids short for clients that cap them (#47692)	2026-10-31 03:20:42 -07:00
test_main_module_header.py	test: move tests/test_litellm root and small trees into tests/unit (#43186)	2026-09-25 11:30:43 -07:00
test_mistral_large_4_model_metadata.py	fix(cost-map): price Mistral Large 4 at its list price now that the launch sale ended (#48825)	2026-10-28 18:12:55 +00:00
test_mistral_medium_3_5_model_metadata.py	test: move tests/test_litellm root and small trees into tests/unit (#43186)	2026-09-25 11:30:43 -07:00
test_mistral_small_4_0_model_metadata.py	test: move tests/test_litellm root and small trees into tests/unit (#43186)	2026-09-25 11:30:43 -07:00
test_mistral_zai_glm_5_2_model_metadata.py	test: move tests/test_litellm root and small trees into tests/unit (#43186)	2026-09-25 11:30:43 -07:00
test_model_block_unblock.py	refactor(router): move router callers onto the entry points and invert the proxy imports (#48847)	2026-10-29 17:26:33 +00:00
test_model_cost_aliases.py	test: move tests/test_litellm root and small trees into tests/unit (#43186)	2026-09-25 11:30:43 -07:00
test_model_param_helper.py	refactor: expose core private helpers under public names (#44871)	2026-10-07 00:05:27 +00:00
test_model_prices_and_context_window.py	refactor(router): route the router family through RoutingContext and close the outside surface (#49118)	2026-10-30 19:18:54 -07:00
test_model_response_normalization.py	test: move tests/test_litellm root and small trees into tests/unit (#43186)	2026-09-25 11:30:43 -07:00
test_muse_spark_1_1_model_metadata.py	test: move tests/test_litellm root and small trees into tests/unit (#43186)	2026-09-25 11:30:43 -07:00
test_muse_spark_1_2_model_metadata.py	test: move tests/test_litellm root and small trees into tests/unit (#43186)	2026-09-25 11:30:43 -07:00
test_muse_spark_1_3_model_metadata.py	refactor(router): route the router family through RoutingContext and close the outside surface (#49118)	2026-10-30 19:18:54 -07:00
test_mutation_report.py	test: move tests/test_litellm root and small trees into tests/unit (#43186)	2026-09-25 11:30:43 -07:00
test_nested_drop_params.py	test: move tests/test_litellm root and small trees into tests/unit (#43186)	2026-09-25 11:30:43 -07:00
test_no_top_level_test_invocations.py	test: move whole-unit legacy test files into tests/unit and delete dead skips (#44807)	2026-10-07 05:02:28 +00:00
test_non_chat_routes_open_llm_spans.py	test: move tests/test_litellm root and small trees into tests/unit (#43186)	2026-09-25 11:30:43 -07:00
test_openai_embedding_encoding_format_default.py	test: migrate local_testing SDK and provider tests out of build_and_test (#45760)	2026-10-10 23:14:25 +00:00
test_openai_record_replay_proxy.py	test: move whole-unit legacy test files into tests/unit and delete dead skips (#44807)	2026-10-07 05:02:28 +00:00
test_openai_service_tier_long_context_pricing.py	fix(cost_calculator): bill ultrafast prompts above 272k at the ultrafast long-context rates (#43764)	2026-09-30 07:32:16 -07:00
test_openapi_docs_gate.py	fix(ci): build base native extension for OpenAPI lint (#49136)	2026-10-29 16:56:44 -07:00
test_package_layout.py	test(unit): make every tests/unit directory a package so pytest collection is unique	2026-09-20 11:50:59 +00:00
test_package_metadata.py	chore(deps): require anyio>=4.14.0 to rule out the lock-waiter deadlock (#47378)	2026-10-19 12:37:47 -07:00
test_pre_commit_lint.py	ci: run the dashboard API types and Terraform endpoint checks on every change that can alter the spec (#49022)	2026-10-29 07:19:06 -07:00
test_prisma_generate_if_needed.py	test: move tests/test_litellm root and small trees into tests/unit (#43186)	2026-09-25 11:30:43 -07:00
test_private_usage_aliases.py	refactor(router): route the router family through RoutingContext and close the outside surface (#49118)	2026-10-30 19:18:54 -07:00
test_process_helpers.py	test: move tests/test_litellm root and small trees into tests/unit (#43186)	2026-09-25 11:30:43 -07:00
test_project_alias_tracking.py	test: move tests/test_litellm root and small trees into tests/unit (#43186)	2026-09-25 11:30:43 -07:00
test_project_tags_pydantic.py	test: move tests/test_litellm root and small trees into tests/unit (#43186)	2026-09-25 11:30:43 -07:00
test_proxy_auth.py	test: move tests/test_litellm root and small trees into tests/unit (#43186)	2026-09-25 11:30:43 -07:00
test_pydantic_namespaces.py	test: move whole-unit legacy test files into tests/unit and delete dead skips (#44807)	2026-10-07 05:02:28 +00:00
test_rag_openai_ingestion.py	test: move tests/test_litellm root and small trees into tests/unit (#43186)	2026-09-25 11:30:43 -07:00
test_rate_limit_error_unification.py	refactor(proxy): expose public names for private proxy helpers (#45170)	2026-10-07 18:59:27 -07:00
test_read_rc_version.py	test: move tests/test_litellm root and small trees into tests/unit (#43186)	2026-09-25 11:30:43 -07:00
test_redact_string_in_error_paths.py	refactor: expose core private helpers under public names (#44871)	2026-10-07 00:05:27 +00:00
test_redis.py	test: move routing, caching and callback tests out of local_testing (#45761)	2026-10-10 09:33:09 -07:00
test_redis_credential_provider.py	test: move tests/test_litellm root and small trees into tests/unit (#43186)	2026-09-25 11:30:43 -07:00
test_register_model_custom_pricing.py	fix(register_model)!: keep bedrock geo deployments priced across price data reloads (#41523)	2026-10-17 19:10:04 +00:00
test_register_model_zero_cost_persistence.py	test: move tests/test_litellm root and small trees into tests/unit (#43186)	2026-09-25 11:30:43 -07:00
test_replicate_model_key_format.py	test: move tests/test_litellm root and small trees into tests/unit (#43186)	2026-09-25 11:30:43 -07:00
test_repo_agent_skills.py	feat(skills): add a litellm-setup agent skill (#47731)	2026-10-28 15:44:43 -07:00
test_responses_api_bridge_non_stream.py	refactor: expose core private helpers under public names (#44871)	2026-10-07 00:05:27 +00:00
test_responses_id_security.py	fix(responses): resume background response streams with stream and starting_after (#48819)	2026-10-29 02:31:22 -07:00
test_responses_streaming_container_ownership.py	fix(responses): bill a background response once, on the first read or poll that sees it finished (#45703)	2026-10-15 11:00:14 -07:00
test_retrieve_batch_bedrock_dispatch.py	refactor(llms): expose public names for private provider helpers (#45037)	2026-10-07 08:49:25 -07:00
test_router_block_helpers.py	refactor: expose core private helpers under public names (#44871)	2026-10-07 00:05:27 +00:00
test_router_exception_redaction.py	fix(streaming)!: map stream read timeouts to litellm.Timeout on chat, messages, and responses (#47198)	2026-10-18 13:58:34 +00:00
test_router_get_settings.py	ci(circleci): test Redis behavior against local Redis and print short tracebacks (#44062)	2026-10-01 13:00:54 -07:00
test_router_google_genai.py	test: move tests/test_litellm root and small trees into tests/unit (#43186)	2026-09-25 11:30:43 -07:00
test_router_model_cost_isolation.py	fix(anthropic)!: send /v1/messages PDF documents to OpenAI-shaped providers as file parts (#47166)	2026-10-29 08:06:12 -07:00
test_router_order_fallback.py	fix(router): keep the prompt caching affinity pin alive as long as the prefix's cache_control ttl (#46084)	2026-10-14 21:10:09 -07:00
test_router_per_deployment_num_retries.py	fix(images): honor max_retries and prevent Router retry amplification (#48705)	2026-10-28 10:46:47 -07:00
test_router_redis_init.py	test: move tests/test_litellm root and small trees into tests/unit (#43186)	2026-09-25 11:30:43 -07:00
test_router_retry_backoff_headers.py	test: move tests/test_litellm root and small trees into tests/unit (#43186)	2026-09-25 11:30:43 -07:00
test_router_retry_non_retryable_errors.py	test: move tests/test_litellm root and small trees into tests/unit (#43186)	2026-09-25 11:30:43 -07:00
test_router_retry_policy_update.py	fix(proxy): unset DB environment variables removed from the row on every worker by row revision (#46709)	2026-10-17 03:51:05 +00:00
test_router_silent_experiment.py	refactor(router): extract PythonCore behind RouterCore and pass the router its config (#49119)	2026-10-31 12:54:23 -07:00
test_router_streaming_fallback_metadata.py	refactor(router): route the router family through RoutingContext and close the outside surface (#49118)	2026-10-30 19:18:54 -07:00
test_router_weighted_failover.py	refactor(router): extract PythonCore behind RouterCore and pass the router its config (#49119)	2026-10-31 12:54:23 -07:00
test_ruff_strict_gate.py	ci(lint): gate every rule on its merge-base count and cap Anys at a fixed total (#40193)	2026-10-08 05:39:29 +00:00
test_sambanova_model_metadata.py	test: move tests/test_litellm root and small trees into tests/unit (#43186)	2026-09-25 11:30:43 -07:00
test_scaledown_model_prices.py	feat(scaledown): add native text provider support (#44168)	2026-10-10 14:01:17 -07:00
test_scheduler.py	fix(scheduler): read shared queue from Redis so expired dead-replica entries stop blocking waiters (#49387)	2026-10-31 11:43:46 -07:00
test_secret_redaction.py	refactor(rust-bridge): require the native extension (#48852)	2026-10-28 13:16:56 -07:00
test_seed_tracing_fixtures.py	refactor(lens): connect LiteLLM to the independent Lens service (#45529)	2026-10-09 16:17:05 -07:00
test_select_ui_test_scope.py	fix(ci): measure PR gates from the target branch's current tip (#49078)	2026-10-29 18:28:16 -07:00
test_service_logger.py	test: move 76 live top-level tests to offline unit and integration coverage (#45352)	2026-10-08 10:33:35 -07:00
test_setup_wizard.py	test: move tests/test_litellm root and small trees into tests/unit (#43186)	2026-09-25 11:30:43 -07:00
test_shared_session_integration.py	test: move tests/test_litellm root and small trees into tests/unit (#43186)	2026-09-25 11:30:43 -07:00
test_socket_policy.py	test(unit): block external sockets at import time and add a socket policy regression test	2026-09-20 08:07:50 +00:00
test_spend_logs_guard.py	fix(ci): flag any schema.prisma change inside model LiteLLM_SpendLogs in the spend logs guard (LIT-10240) (#48905)	2026-10-28 15:52:15 -07:00
test_ssl_verify_unit.py	test(unit): stop test modules from putting their own directory on sys.path (#43421)	2026-09-28 12:20:20 -07:00
test_stream_chunk_builder_annotations.py	fix(anthropic)!: send /v1/messages PDF documents to OpenAI-shaped providers as file parts (#47166)	2026-10-29 08:06:12 -07:00
test_stream_chunk_builder_citations.py	test: move tests/test_litellm root and small trees into tests/unit (#43186)	2026-09-25 11:30:43 -07:00
test_stream_chunk_builder_images.py	test: move tests/test_litellm root and small trees into tests/unit (#43186)	2026-09-25 11:30:43 -07:00
test_streaming_connection_cleanup.py	test: move tests/test_litellm root and small trees into tests/unit (#43186)	2026-09-25 11:30:43 -07:00
test_system_message_format_bug.py	fix(prompt_templates): do not mutate messages in-place in function_call_prompt (#41676)	2026-10-15 14:37:11 -07:00
test_test_quality_gate.py	ci(lint): gate every rule on its merge-base count and cap Anys at a fixed total (#40193)	2026-10-08 05:39:29 +00:00
test_thinking_enabled.py	test: move tests/test_litellm root and small trees into tests/unit (#43186)	2026-09-25 11:30:43 -07:00
test_together_ai_model_metadata.py	refactor(cost-map): drop the legacy max_tokens field in favor of max_input_tokens and max_output_tokens (#48471)	2026-10-26 17:03:05 -07:00
test_tool_loop.py	fix(responses): replay the original Responses item order, phase, and boundaries on the next chat turn (#48912)	2026-10-29 00:56:02 -07:00
test_type_check_gate.py	ci(lint): gate every rule on its merge-base count and cap Anys at a fixed total (#40193)	2026-10-08 05:39:29 +00:00
test_type_discipline_gate.py	ci(lint): gate every rule on its merge-base count and cap Anys at a fixed total (#40193)	2026-10-08 05:39:29 +00:00
test_typesafe_model_metadata.py	fix(cost): price typesafe/jev-1.14.0 (#46594)	2026-10-14 20:27:35 -07:00
test_unit_shard_missing_paths.py	ci: render lint, unit and smoke checks as <tier> / <job> with one collector per tier (#45480)	2026-10-09 04:28:41 +00:00
test_unit_shard_per_test_timeout.py	ci: render lint, unit and smoke checks as <tier> / <job> with one collector per tier (#45480)	2026-10-09 04:28:41 +00:00
test_utils.py	fix(openrouter)!: list OpenRouter's live catalog for openrouter/* when check_provider_endpoint is on (#48820)	2026-10-30 12:19:51 -07:00
test_utils_get_model_info.py	fix(bedrock): price claude_platform calls from their own cost map rows (#48150)	2026-10-24 13:54:18 -07:00
test_utils_get_optional_params.py	fix(openai): warn when the GPT-5 reasoning effort gate drops a level under drop_params (#47761)	2026-10-29 04:46:29 -07:00
test_utils_module_docstring.py	test: move tests/test_litellm root and small trees into tests/unit (#43186)	2026-09-25 11:30:43 -07:00
test_uuid_helper.py	fix: support Python 3.15 (#48359)	2026-10-26 16:00:11 +00:00
test_vcr_classification.py	test: move offline logging, secret manager and provider tests from legacy dirs into tests/unit (#45554)	2026-10-09 09:48:10 -07:00
test_vcr_conftest_common_banner.py	test: move whole-unit legacy test files into tests/unit and delete dead skips (#44807)	2026-10-07 05:02:28 +00:00
test_vcr_filters.py	test: move whole-unit legacy test files into tests/unit and delete dead skips (#44807)	2026-10-07 05:02:28 +00:00
test_vcr_leak_guard.py	test: move whole-unit legacy test files into tests/unit and delete dead skips (#44807)	2026-10-07 05:02:28 +00:00
test_vcr_redis_persister.py	test: migrate tests/llm_translation into unit, integration and e2e tiers (#45759)	2026-10-11 07:58:53 +00:00
test_vcr_safe_body_matcher.py	fix(tests): stop VCR recording and replaying a test's own localhost upstream (#43346)	2026-09-26 15:49:39 -07:00
test_version.py	feat(sdk): build core from an independent packaging manifest (#44340)	2026-10-08 18:40:14 -07:00
test_vertex_ai_xai_grok_prompt_caching_metadata.py	test: move tests/test_litellm root and small trees into tests/unit (#43186)	2026-09-25 11:30:43 -07:00
test_video_generation.py	refactor(router): move router callers onto the entry points and invert the proxy imports (#48847)	2026-10-29 17:26:33 +00:00
test_with_dashboard_node.py	test: move tests/test_litellm root and small trees into tests/unit (#43186)	2026-09-25 11:30:43 -07:00
test_ws_vcr.py	test: move whole-unit legacy test files into tests/unit and delete dead skips (#44807)	2026-10-07 05:02:28 +00:00
test_xai_grok_4_3_model_metadata.py	test: move tests/test_litellm root and small trees into tests/unit (#43186)	2026-09-25 11:30:43 -07:00
test_xai_responses_auto_routing.py	fix(router)!: stop a chat-mode Mantle deployment from rewriting the shared Responses routing row (#46263)	2026-10-17 21:39:14 +00:00