litellm/tests/unit/proxy/db
joshua-berri 076ded32be
feat(auto_router): report a session's whole spend and name why savings stay unestimated (#51066)
* feat(auto_router): report a session's whole spend on /auto_router/session

Claude Code sends its background calls straight to Haiku under the same session id, so those requests never
reached the auto-router session rollup and the status line under-reported the session. Every billed spend
row in a caller-supplied session now increments a key-scoped Redis counter whose one-hour expiry each write
resets, and the endpoint reports it as total_spend, with spend unchanged. The increment runs where the spend
row is accepted, so it counts each row exactly as often as the key's spend counters do. It reads as no total
without a shared Redis

* fix(auto_router): name the unsupported provider when a turn routes off Anthropic

A turn routed to a non-Anthropic tier with no native wire response, such as a Cloudflare DeepSeek model,
failed strict native validation and was logged as observation_unavailable, which reads like a broken
capture. It now reports unsupported_selected_provider. Turns with a native wire response keep their
existing reasons, and no estimate is added, so savings for these turns stay unknown

* docs(auto_router): explain when session savings stay unestimated

saved_spend on /auto_router/session now says a turn inside the longest prompt-cache TTL of the session's
first observed turn, or of its last incomplete observation, is not estimated, so a null there is expected
for that window

* fix(auto_router): say why a system-only cache prefix has no savings estimate

Claude Code puts its cache breakpoint on the system prompt, so the baseline prefix count sends Anthropic a
body with no messages, which it rejects, and every turn read as token_count_unavailable. A count that fails
only on system or tools prefixes now reports uncountable_system_or_tools_prefix. Gateways that accept the
empty body still count it. A live check against claude-opus-5-5 showed a differential count with a
placeholder message is off by -2 tokens for system-only and +68 for tools, so these turns stay unestimated
and the savings gap for Claude Code is not fixed here

* fix(auto_router): declare the session total's Redis key family and test its expiry

The session total read and wrote Redis without a service target, so its calls showed up as bare redis
spans. Both now run under session_spend_totals. The key no longer hashes the api key, which is already a
hashed token, and only digests the session id. A new test drives Redis expiry on a test clock to show an
active session keeps its total and an idle one restarts

* docs(auto_router): note which turns are still estimated inside the history window

Turns that took the identical baseline path, and turns whose whole cached prefix the session had already
written, are estimated even inside the first cache TTL window, so saved_spend now says so

---------

Co-authored-by: Joshua Valluru <326636767+joshua-berri@users.noreply.github.com>
2026-11-10 17:07:11 -08:00
..
db_transaction_queue feat(spend): report guardrail spend separately in daily activity (#51010) 2026-11-10 13:02:26 -08:00
mcp_server refactor(mcp): drop the experimental label from the MCP server and client paths (#48513) 2026-10-27 17:47:50 +00:00
__init__.py
conftest.py test(proxy): move proxy_server, _experimental and db tests into tests/unit/proxy (#44012) 2026-10-01 18:14:24 +00:00
fake_prisma_engine.py fix(proxy): cancel the Admin UI's raw spend-log reads in Postgres after 30s (#49965) 2026-11-03 16:07:00 -08:00
test_autorouter_session_rollup.py test: migrate tests/proxy_behavior into integration and unit tiers (#45939) 2026-10-11 08:28:42 -07:00
test_budget_window_spend_writer.py feat(accounting): add opt-in SpendLogs ledger (#49856) 2026-11-04 17:49:15 -08:00
test_check_migration.py feat(proxy)!: add suppress_startup_output to turn off startup prints (#47708) 2026-10-29 03:47:04 -07:00
test_create_views.py test(proxy): move proxy_server, _experimental and db tests into tests/unit/proxy (#44012) 2026-10-01 18:14:24 +00:00
test_daily_spend_bulk_upsert.py feat(spend): report guardrail spend separately in daily activity (#51010) 2026-11-10 13:02:26 -08:00
test_db_lookup_gate.py test: make three flaky unit tests deterministic (#50114) 2026-11-04 14:38:44 -08:00
test_db_span.py fix(otel): name postgres service spans by operation and table (#44240) 2026-10-03 09:20:28 -07:00
test_db_spend_update_writer.py feat(auto_router): report a session's whole spend and name why savings stay unestimated (#51066) 2026-11-10 17:07:11 -08:00
test_db_url_settings.py fix(proxy): sign RDS IAM tokens for the database's own region (#44782) 2026-10-09 15:41:07 -07:00
test_exception_handler.py fix(proxy): treat Postgres restart and shutdown errors as DB unavailable, not poison spend-log rows (#50132) 2026-11-04 17:39:30 -08:00
test_exception_handler_reconnect_retry.py test(proxy): move proxy_server, _experimental and db tests into tests/unit/proxy (#44012) 2026-10-01 18:14:24 +00:00
test_gateway_request_tracking.py feat(proxy): track failed requests by HTTP status and caller (#45244) 2026-10-09 14:27:56 -07:00
test_health_check_latest.py fix(proxy): show the newest health check per model group on the model hub (#47552) 2026-10-20 16:42:53 -07:00
test_log_db_metrics.py fix(proxy): keep the database error when the log_db_metrics failure hook raises (#44383) 2026-10-03 16:38:15 +00:00
test_log_db_metrics_service_spans.py test: replace 61 live logging and otel tests with offline unit and integration coverage (#45362) 2026-10-08 15:55:57 -07:00
test_master_key_migration.py fix(security): remove the publicly known master key from the repo (#44718) 2026-10-06 10:55:24 -07:00
test_model_access_group_spend.py refactor(router): move router callers onto the entry points and invert the proxy imports (#48847) 2026-10-29 17:26:33 +00:00
test_model_insights_tasks.py test(proxy): move proxy_server, _experimental and db tests into tests/unit/proxy (#44012) 2026-10-01 18:14:24 +00:00
test_model_usage_rollup.py fix(responses): bill a background response once, on the first read or poll that sees it finished (#45703) 2026-10-15 11:00:14 -07:00
test_object_permission_repository.py refactor(types): replace Any with proven types in 137 files (#44478) 2026-10-05 11:01:45 -07:00
test_pgbouncer.py test(unit): keep unix socket paths in unit tests under the macOS AF_UNIX limit (#45040) 2026-10-28 20:23:22 -07:00
test_prisma_client.py feat(proxy): replace the pre-migration row check with a rolled-back dry run of the pending migrations (#48961) 2026-10-29 09:39:26 -07:00
test_prisma_engine.py fix(proxy): let the OS assign Prisma engine ports (#50850) 2026-11-10 07:28:55 +00:00
test_prisma_planned_engine_restart.py test(proxy): move proxy_server, _experimental and db tests into tests/unit/proxy (#44012) 2026-10-01 18:14:24 +00:00
test_prisma_query_span.py fix(proxy): cancel the Admin UI's raw spend-log reads in Postgres after 30s (#49965) 2026-11-03 16:07:00 -08:00
test_prisma_self_heal.py test(proxy): move proxy_server, _experimental and db tests into tests/unit/proxy (#44012) 2026-10-01 18:14:24 +00:00
test_proxy_worker_heartbeat.py fix(otel): name postgres service spans by operation and table (#44240) 2026-10-03 09:20:28 -07:00
test_query_engine_reaper.py test(proxy): move proxy_server, _experimental and db tests into tests/unit/proxy (#44012) 2026-10-01 18:14:24 +00:00
test_rds_iam_token_expiry.py test(proxy): move proxy_server, _experimental and db tests into tests/unit/proxy (#44012) 2026-10-01 18:14:24 +00:00
test_replica_identity.py test: migrate proxy security, schema migration and MCP legacy tests (#45937) 2026-10-11 16:17:24 -07:00
test_request_error_tracking.py feat(proxy): track failed requests by HTTP status and caller (#45244) 2026-10-09 14:27:56 -07:00
test_routing_prisma_wrapper.py fix(proxy): let the OS assign Prisma engine ports (#50850) 2026-11-10 07:28:55 +00:00
test_shadow_eval_funnel.py fix(otel): name postgres service spans by operation and table (#44240) 2026-10-03 09:20:28 -07:00
test_spend_counter_reseed.py feat(redis): serve the proxy's common Redis call with Rust stores (#50170) 2026-11-06 06:01:11 +00:00
test_spend_log_batching.py test(proxy): move proxy_server, _experimental and db tests into tests/unit/proxy (#44012) 2026-10-01 18:14:24 +00:00
test_spend_log_tool_index.py fix(anthropic)!: do not surface server_tool_use as client tool_calls (#48385) 2026-11-06 03:54:34 -08:00
test_token_auth.py fix(proxy): sign RDS IAM tokens for the database's own region (#44782) 2026-10-09 15:41:07 -07:00
test_tool_registry_writer.py fix(proxy): treat Postgres restart and shutdown errors as DB unavailable, not poison spend-log rows (#50132) 2026-11-04 17:39:30 -08:00
test_tool_table_bulk_upsert.py fix(proxy): buffer tool discoveries in Redis and write LiteLLM_ToolTable in one statement per drain, on its own lock (#48089) 2026-10-24 12:34:13 -07:00
test_update_daily_tag_spend.py refactor(proxy): expose public names for private proxy helpers (#45170) 2026-10-07 18:59:27 -07:00