litellm/tests/unit/proxy/common_utils
Mateo Wang 326f17594e
fix(batches): return real batch token counts next to OpenAI's batch usage fields (#47740)
* fix(batches): read OpenAI batch usage into the batch token counts

* fix(batches): serve OpenAI's batch usage shape from retrieve and the cost poller

* fix(batches): keep the chat usage keys next to OpenAI's batch usage fields

Batch usage now carries every chat key it had plus OpenAI's five batch fields with the same counts, so poller-billed batches keep prompt_tokens and cache-write tokens, and older proxies still read the new rows. Usage with missing or null detail counts reads its real input and output counts.

* test(batches): check the batch usage counts in the e2e terminal-state test

* fix(types): apply nested include and exclude to both batch usage shapes

* fix(batches): log no cost for a batch retrieve that is not final yet

A retrieve now reads OpenAI's batch usage into the chat counts, so a retrieve
that saw the batch completed before its output file arrived priced those counts
for the logging callbacks, and the final retrieve priced the whole batch again.
A batch that is not final now logs no cost, as it did before.

Batch usage dumps also pass exclude_unset and exclude_defaults through to the
usage keys.

* fix(batches): pass through every usage detail field the provider sends

Batch dumps rebuilt input_tokens_details and output_tokens_details from the chat keys, which dropped any detail field past cached_tokens and reasoning_tokens that the merge base passed through. The dump now starts from the provider's stored details and only sets the two counts derived from the chat keys

* fix(batches): skip sync loggers on the cost poller's own retrieve and honor exclude_none on provider details

* fix(batches): keep sync logger events at $0 on poller-owned retrieves, pass unparsed provider counts through, log batch usage in chat shape

* fix(batches): store only chat usage extras on batch spend rows

Spend rows for batch retrieves copied OpenAI's batch fields (input_tokens,
output_tokens and their details) into metadata.additional_usage_values next
to the chat counts they duplicate. Build the row from the batch's chat usage,
shared with the logging payload through batch_chat_usage.
2026-10-29 18:29:30 -07:00
..
html_forms test(proxy): move middleware, spend_tracking, pass_through, common_utils and root proxy tests into tests/unit/proxy (#44015) 2026-10-01 18:23:31 +00:00
__init__.py ci: move tests/proxy_unit_tests to tests/unit/proxy and run the proxy-db shards from litellm-tests (#42903) 2026-09-24 22:59:11 +00:00
test_admin_ui_utils.py test(proxy): move middleware, spend_tracking, pass_through, common_utils and root proxy tests into tests/unit/proxy (#44015) 2026-10-01 18:23:31 +00:00
test_auth_cache_invalidation_pubsub.py fix(scim): write a group's roster in one locked transaction instead of one member at a time (#45197) 2026-10-15 10:57:00 -07:00
test_cache_aware_routing.py feat(router): respect caller output budgets in complexity routing (#48632) 2026-10-27 14:04:30 -07:00
test_cache_codec.py test(proxy): move middleware, spend_tracking, pass_through, common_utils and root proxy tests into tests/unit/proxy (#44015) 2026-10-01 18:23:31 +00:00
test_callback_config_validation.py feat(otel): let a team or key excluded_services override the global setting for its OTel v2 destination (#45880) 2026-10-12 19:25:52 -04:00
test_callback_utils.py feat(router): opt in to non-chat deployment filtering (#46257) 2026-10-29 08:33:36 -07:00
test_check_batch_cost.py fix(batches): return real batch token counts next to OpenAI's batch usage fields (#47740) 2026-10-29 18:29:30 -07:00
test_check_responses_cost.py refactor(router): move router callers onto the entry points and invert the proxy imports (#48847) 2026-10-29 17:26:33 +00:00
test_codex_model_catalog.py feat(lite): offer each model group's reasoning efforts in pi, OpenCode and Codex (#48949) 2026-10-29 18:15:44 -07:00
test_config_sync_pubsub.py fix(proxy): resync configuration after Redis reconnect (#43313) 2026-10-19 21:21:53 -07:00
test_credential_hydration.py fix(secret_managers)!: resolve oidc/file and oidc/env refs in admin-configured api_key per request (#47341) 2026-10-20 01:43:59 +00:00
test_custom_openapi_spec.py docs(proxy): describe /v1/decisions and /v1/systemone in Swagger UI (#45838) 2026-10-10 21:04:59 +00:00
test_debug_utils.py fix(router): keep the prompt caching affinity pin alive as long as the prefix's cache_control ttl (#46084) 2026-10-14 21:10:09 -07:00
test_discoverable_model_filter.py test(proxy): move middleware, spend_tracking, pass_through, common_utils and root proxy tests into tests/unit/proxy (#44015) 2026-10-01 18:23:31 +00:00
test_encrypt_decrypt_utils.py refactor(proxy): expose public names for private proxy helpers (#45170) 2026-10-07 18:59:27 -07:00
test_error_body_call_id.py test(proxy): move middleware, spend_tracking, pass_through, common_utils and root proxy tests into tests/unit/proxy (#44015) 2026-10-01 18:23:31 +00:00
test_expired_ui_session_key_cleanup_manager.py test(proxy): move middleware, spend_tracking, pass_through, common_utils and root proxy tests into tests/unit/proxy (#44015) 2026-10-01 18:23:31 +00:00
test_get_routes.py test(proxy): move middleware, spend_tracking, pass_through, common_utils and root proxy tests into tests/unit/proxy (#44015) 2026-10-01 18:23:31 +00:00
test_http_parsing_utils.py fix(proxy): abort the request with a 499 when the client disconnects during the body upload (#48640) 2026-10-27 16:49:56 -07:00
test_json_merge_patch.py test(proxy): move middleware, spend_tracking, pass_through, common_utils and root proxy tests into tests/unit/proxy (#44015) 2026-10-01 18:23:31 +00:00
test_key_rotation_e2e.py test(proxy): move middleware, spend_tracking, pass_through, common_utils and root proxy tests into tests/unit/proxy (#44015) 2026-10-01 18:23:31 +00:00
test_key_rotation_integration.py fix(security): remove the publicly known master key from the repo (#44718) 2026-10-06 10:55:24 -07:00
test_key_rotation_lock.py test(proxy): move middleware, spend_tracking, pass_through, common_utils and root proxy tests into tests/unit/proxy (#44015) 2026-10-01 18:23:31 +00:00
test_key_rotation_manager.py test(proxy): move middleware, spend_tracking, pass_through, common_utils and root proxy tests into tests/unit/proxy (#44015) 2026-10-01 18:23:31 +00:00
test_load_config_utils.py fix(security): remove the publicly known master key from the repo (#44718) 2026-10-06 10:55:24 -07:00
test_model_deprecation.py refactor(router): move router callers onto the entry points and invert the proxy imports (#48847) 2026-10-29 17:26:33 +00:00
test_model_listing_utils.py fix(proxy): resolve model aliases to their target deployment for listing metadata (#41041) 2026-10-15 00:38:50 -07:00
test_openai_endpoint_utils.py test(proxy): move middleware, spend_tracking, pass_through, common_utils and root proxy tests into tests/unit/proxy (#44015) 2026-10-01 18:23:31 +00:00
test_openai_error_payload.py fix(proxy)!: stop labelling management and health rejections as auth failures (#48739) 2026-10-28 21:49:03 -07:00
test_openapi_route.py fix(proxy): build the OpenAPI schema off the event loop so a cold /openapi.json no longer stalls the worker (#47352) 2026-10-20 04:31:37 +00:00
test_openapi_schema_compat.py fix(proxy): build the OpenAPI schema off the event loop so a cold /openapi.json no longer stalls the worker (#47352) 2026-10-20 04:31:37 +00:00
test_periodic_reload_schedule.py test(proxy): move middleware, spend_tracking, pass_through, common_utils and root proxy tests into tests/unit/proxy (#44015) 2026-10-01 18:23:31 +00:00
test_proxy_encrypt_decrypt.py fix(security): remove the publicly known master key from the repo (#44718) 2026-10-06 10:55:24 -07:00
test_rbac_utils.py refactor(proxy): expose public names for private proxy helpers (#45170) 2026-10-07 18:59:27 -07:00
test_realtime_cache.py refactor(proxy): expose public names for private proxy helpers (#45170) 2026-10-07 18:59:27 -07:00
test_registry_read_through.py refactor(router): move router callers onto the entry points and invert the proxy imports (#48847) 2026-10-29 17:26:33 +00:00
test_request_tag_mappings.py feat(proxy): map baggage values to budget tags (#47358) 2026-10-20 09:09:42 +00:00
test_reset_budget_job.py fix(proxy): reconcile team member total_spend with current cycle spend (#48260) 2026-10-24 17:57:09 -07:00
test_resource_ownership.py refactor(core_utils): move resource_ownership out of the proxy (#45805) 2026-10-10 21:02:53 +00:00
test_scheduled_job_stagger.py test(proxy): move middleware, spend_tracking, pass_through, common_utils and root proxy tests into tests/unit/proxy (#44015) 2026-10-01 18:23:31 +00:00
test_semantic_text_index.py fix(mcp): leave out tool texts too long to embed instead of failing tool search (#43554) 2026-10-14 14:53:00 -07:00
test_slow_gc_logger.py test(e2e): event_loop_stalls perf case fails on a gateway event-loop hold over 100 ms that is not on its allowlist (#46974) 2026-10-17 06:55:53 +00:00
test_sse_keepalive.py test(proxy): move middleware, spend_tracking, pass_through, common_utils and root proxy tests into tests/unit/proxy (#44015) 2026-10-01 18:23:31 +00:00
test_static_asset_utils.py test(proxy): move middleware, spend_tracking, pass_through, common_utils and root proxy tests into tests/unit/proxy (#44015) 2026-10-01 18:23:31 +00:00
test_swagger_utils.py test(proxy): move middleware, spend_tracking, pass_through, common_utils and root proxy tests into tests/unit/proxy (#44015) 2026-10-01 18:23:31 +00:00
test_timezone_utils.py test(proxy): move middleware, spend_tracking, pass_through, common_utils and root proxy tests into tests/unit/proxy (#44015) 2026-10-01 18:23:31 +00:00
test_token_count_log.py feat(proxy): log one line per token count call (#47657) 2026-10-28 11:23:52 -07:00
test_token_count_settings.py fix(responses)!: answer 400 before counting a lone Responses built-in tool on Claude (#48798) 2026-10-28 12:12:45 -07:00
test_upsert_budget_membership.py fix(team): keep the team reset schedule when a temp budget increase is set on a member inheriting the team default (#48495) 2026-10-26 17:13:56 -07:00
test_user_api_key_cache.py test(proxy): move middleware, spend_tracking, pass_through, common_utils and root proxy tests into tests/unit/proxy (#44015) 2026-10-01 18:23:31 +00:00
test_validation_error_body.py fix(proxy): keep the submitted body out of 422 validation errors (#43231) 2026-09-25 17:05:53 -07:00