Mateo Wang ea6dbd2109 feat(proxy): log one line per token count call (#47657) ... * feat(proxy): log one line per token count call Each count call on /v1/messages/count_tokens, /utils/token_counter, /v1/responses/input_tokens and the Gemini countTokens routes now logs one INFO line with the litellm call id, the requested model, the model the count ran against, the count, the count source and the status code, so it joins the inference call that sends the same x-litellm-call-id * test(google_endpoints): give the count test's fake request a headers list * fix(proxy): keep token count log lines one per call and trusted in JSON logs, and log Bedrock native counts Quote log values that carry whitespace, quotes, equals, or control characters so a model name can't split or forge a line. Let JsonFormatter record extras win over a dict parsed from the message. Log the Bedrock native count-tokens route through the same line. * fix(proxy): escape braces in token count log values, log the Bedrock model AWS counts, and skip the line when there is nothing to count * fix(proxy): log the deployment id the count routed to on every token count line * fix(proxy): log count requests rejected before counting with the status they answer A count route that rejects a request before the shared count function runs now logs the same token count line with its 400. An unsupported content block on the Messages count route becomes its 400 inside the logged wrapper, so its line says 400 instead of 500. A count that ends in a BaseException other than cancellation logs a 500. * fix(proxy): compute the Bedrock counted model id only after a successful count * revert(logging): keep the JSON log formatter's field order as on main The count line already escapes braces in its values, so the formatter never parses a dict from it. Changing the order for every logger is a separate decision.		2026-10-28 11:23:52 -07:00
..
html_forms	test(proxy): move middleware, spend_tracking, pass_through, common_utils and root proxy tests into tests/unit/proxy (#44015)	2026-10-01 18:23:31 +00:00
__init__.py	ci: move tests/proxy_unit_tests to tests/unit/proxy and run the proxy-db shards from litellm-tests (#42903)	2026-09-24 22:59:11 +00:00
test_admin_ui_utils.py	test(proxy): move middleware, spend_tracking, pass_through, common_utils and root proxy tests into tests/unit/proxy (#44015)	2026-10-01 18:23:31 +00:00
test_auth_cache_invalidation_pubsub.py	fix(scim): write a group's roster in one locked transaction instead of one member at a time (#45197)	2026-10-15 10:57:00 -07:00
test_cache_aware_routing.py	feat(router): respect caller output budgets in complexity routing (#48632)	2026-10-27 14:04:30 -07:00
test_cache_codec.py	test(proxy): move middleware, spend_tracking, pass_through, common_utils and root proxy tests into tests/unit/proxy (#44015)	2026-10-01 18:23:31 +00:00
test_callback_config_validation.py	feat(otel): let a team or key excluded_services override the global setting for its OTel v2 destination (#45880)	2026-10-12 19:25:52 -04:00
test_callback_utils.py	test(proxy): move middleware, spend_tracking, pass_through, common_utils and root proxy tests into tests/unit/proxy (#44015)	2026-10-01 18:23:31 +00:00
test_check_batch_cost.py	refactor(typing): clear 1181 basedpyright Any errors across 431 files (#47789)	2026-10-24 11:31:09 -07:00
test_check_responses_cost.py	fix(responses): send the billed background response cost to the creating key's and team's loggers (#47506)	2026-10-20 23:22:21 +00:00
test_codex_model_catalog.py	feat(proxy): serve a Codex-native model catalog with per-model service_tiers from /v1/models (#44136)	2026-10-03 20:42:55 +00:00
test_config_sync_pubsub.py	fix(proxy): resync configuration after Redis reconnect (#43313)	2026-10-19 21:21:53 -07:00
test_credential_hydration.py	fix(secret_managers)!: resolve oidc/file and oidc/env refs in admin-configured api_key per request (#47341)	2026-10-20 01:43:59 +00:00
test_custom_openapi_spec.py	docs(proxy): describe /v1/decisions and /v1/systemone in Swagger UI (#45838)	2026-10-10 21:04:59 +00:00
test_debug_utils.py	fix(router): keep the prompt caching affinity pin alive as long as the prefix's cache_control ttl (#46084)	2026-10-14 21:10:09 -07:00
test_discoverable_model_filter.py	test(proxy): move middleware, spend_tracking, pass_through, common_utils and root proxy tests into tests/unit/proxy (#44015)	2026-10-01 18:23:31 +00:00
test_encrypt_decrypt_utils.py	refactor(proxy): expose public names for private proxy helpers (#45170)	2026-10-07 18:59:27 -07:00
test_error_body_call_id.py	test(proxy): move middleware, spend_tracking, pass_through, common_utils and root proxy tests into tests/unit/proxy (#44015)	2026-10-01 18:23:31 +00:00
test_expired_ui_session_key_cleanup_manager.py	test(proxy): move middleware, spend_tracking, pass_through, common_utils and root proxy tests into tests/unit/proxy (#44015)	2026-10-01 18:23:31 +00:00
test_get_routes.py	test(proxy): move middleware, spend_tracking, pass_through, common_utils and root proxy tests into tests/unit/proxy (#44015)	2026-10-01 18:23:31 +00:00
test_http_parsing_utils.py	fix(proxy): abort the request with a 499 when the client disconnects during the body upload (#48640)	2026-10-27 16:49:56 -07:00
test_json_merge_patch.py	test(proxy): move middleware, spend_tracking, pass_through, common_utils and root proxy tests into tests/unit/proxy (#44015)	2026-10-01 18:23:31 +00:00
test_key_rotation_e2e.py	test(proxy): move middleware, spend_tracking, pass_through, common_utils and root proxy tests into tests/unit/proxy (#44015)	2026-10-01 18:23:31 +00:00
test_key_rotation_integration.py	fix(security): remove the publicly known master key from the repo (#44718)	2026-10-06 10:55:24 -07:00
test_key_rotation_lock.py	test(proxy): move middleware, spend_tracking, pass_through, common_utils and root proxy tests into tests/unit/proxy (#44015)	2026-10-01 18:23:31 +00:00
test_key_rotation_manager.py	test(proxy): move middleware, spend_tracking, pass_through, common_utils and root proxy tests into tests/unit/proxy (#44015)	2026-10-01 18:23:31 +00:00
test_load_config_utils.py	fix(security): remove the publicly known master key from the repo (#44718)	2026-10-06 10:55:24 -07:00
test_model_deprecation.py	feat(proxy): name the successor model for deprecating deployments (#41670)	2026-10-15 14:50:14 -07:00
test_model_listing_utils.py	fix(proxy): resolve model aliases to their target deployment for listing metadata (#41041)	2026-10-15 00:38:50 -07:00
test_openai_endpoint_utils.py	test(proxy): move middleware, spend_tracking, pass_through, common_utils and root proxy tests into tests/unit/proxy (#44015)	2026-10-01 18:23:31 +00:00
test_openai_error_payload.py	test(proxy): move middleware, spend_tracking, pass_through, common_utils and root proxy tests into tests/unit/proxy (#44015)	2026-10-01 18:23:31 +00:00
test_openapi_route.py	fix(proxy): build the OpenAPI schema off the event loop so a cold /openapi.json no longer stalls the worker (#47352)	2026-10-20 04:31:37 +00:00
test_openapi_schema_compat.py	fix(proxy): build the OpenAPI schema off the event loop so a cold /openapi.json no longer stalls the worker (#47352)	2026-10-20 04:31:37 +00:00
test_periodic_reload_schedule.py	test(proxy): move middleware, spend_tracking, pass_through, common_utils and root proxy tests into tests/unit/proxy (#44015)	2026-10-01 18:23:31 +00:00
test_proxy_encrypt_decrypt.py	fix(security): remove the publicly known master key from the repo (#44718)	2026-10-06 10:55:24 -07:00
test_rbac_utils.py	refactor(proxy): expose public names for private proxy helpers (#45170)	2026-10-07 18:59:27 -07:00
test_realtime_cache.py	refactor(proxy): expose public names for private proxy helpers (#45170)	2026-10-07 18:59:27 -07:00
test_registry_read_through.py	feat(decisions): add Databricks ai_decide as a /v1/decisions provider and auto-router decider (#45200)	2026-10-09 16:19:23 -07:00
test_request_tag_mappings.py	feat(proxy): map baggage values to budget tags (#47358)	2026-10-20 09:09:42 +00:00
test_reset_budget_job.py	fix(proxy): reconcile team member total_spend with current cycle spend (#48260)	2026-10-24 17:57:09 -07:00
test_resource_ownership.py	refactor(core_utils): move resource_ownership out of the proxy (#45805)	2026-10-10 21:02:53 +00:00
test_scheduled_job_stagger.py	test(proxy): move middleware, spend_tracking, pass_through, common_utils and root proxy tests into tests/unit/proxy (#44015)	2026-10-01 18:23:31 +00:00
test_semantic_text_index.py	fix(mcp): leave out tool texts too long to embed instead of failing tool search (#43554)	2026-10-14 14:53:00 -07:00
test_slow_gc_logger.py	test(e2e): event_loop_stalls perf case fails on a gateway event-loop hold over 100 ms that is not on its allowlist (#46974)	2026-10-17 06:55:53 +00:00
test_sse_keepalive.py	test(proxy): move middleware, spend_tracking, pass_through, common_utils and root proxy tests into tests/unit/proxy (#44015)	2026-10-01 18:23:31 +00:00
test_static_asset_utils.py	test(proxy): move middleware, spend_tracking, pass_through, common_utils and root proxy tests into tests/unit/proxy (#44015)	2026-10-01 18:23:31 +00:00
test_swagger_utils.py	test(proxy): move middleware, spend_tracking, pass_through, common_utils and root proxy tests into tests/unit/proxy (#44015)	2026-10-01 18:23:31 +00:00
test_timezone_utils.py	test(proxy): move middleware, spend_tracking, pass_through, common_utils and root proxy tests into tests/unit/proxy (#44015)	2026-10-01 18:23:31 +00:00
test_token_count_log.py	feat(proxy): log one line per token count call (#47657)	2026-10-28 11:23:52 -07:00
test_token_count_settings.py	fix(bedrock): correct gpt max_input_tokens and add Mantle token counting for claude-opus-5-5 (#48479)	2026-10-27 19:38:14 +00:00
test_upsert_budget_membership.py	fix(team): keep the team reset schedule when a temp budget increase is set on a member inheriting the team default (#48495)	2026-10-26 17:13:56 -07:00
test_user_api_key_cache.py	test(proxy): move middleware, spend_tracking, pass_through, common_utils and root proxy tests into tests/unit/proxy (#44015)	2026-10-01 18:23:31 +00:00
test_validation_error_body.py	fix(proxy): keep the submitted body out of 422 validation errors (#43231)	2026-09-25 17:05:53 -07:00