* fix(proxy): re-assert the authenticated identity on passthrough requests
The passthrough merges the client's litellm_metadata into the request metadata
and then re-asserts only user_api_key and the parent span. Every other identity
field the spend and budget pipeline reads stays whatever the request body set,
so a body carrying user_api_key_user_id, user_api_key_team_id,
user_api_key_org_id or user_api_key_end_user_id charges that user, team, org or
end user instead of the caller.
Re-assert the whole sanitized identity after the merge, so the client's copy of
any of those fields is overwritten by the authenticated key's own values.
* test(passthrough): assert no authenticated identity field is client settable
The existing regression names seven fields; the re-assertion covers every field
get_sanitized_user_information_from_key returns, which is twenty today. Derive
the set from the helper so a field added to StandardLoggingUserAPIKeyMetadata is
covered without touching the test.
Two of the twenty were not covered before, including user_api_key_hash, which is
distinct from user_api_key and was client settable.
get_litellm_params returned metadata=None whenever only litellm_metadata was
supplied, which overwrote the fallback function_setup had already applied and
left litellm_params["metadata"] empty. On the /v1/responses
completion-transformation bridge, used by every provider without a native
Responses API config, and on /v1/messages, that discarded the caller's trace
fields a second time after the proxy had promoted them.
Resolve metadata to a copy of litellm_metadata when metadata is empty, guarding
on isinstance because the proxy leaves an unparseable litellm_metadata string in
place and a null metadata would otherwise suppress the backfill and break the
merge. update_from_kwargs copies rather than aliases for the same reason: on
these routes it is handed the caller's provider-bound dict and would otherwise
write user_api_key_auth into it.
The shared `timeout` guardrail param already parsed into LitellmParams, but
the Zscaler initializer never forwarded it and _send_request hardcoded a 5
second constant, so a configured value was silently ignored and slow scans
failed with `Timeout passed=5` regardless of config.
Forward litellm_params.timeout through to the HTTP call, keep 5 seconds as
the default, fall back to it for non-positive values, and declare the field
on the config model so the dashboard renders it.
* fix(proxy): promote caller metadata trace fields into litellm_metadata
Routes in LITELLM_METADATA_ROUTES keep the caller's metadata as a provider
passthrough field and track proxy state in litellm_metadata, which is the dict
the logging integrations read. The caller's trace_id, session_id, trace_user_id
and trace_metadata therefore never reached any callback on /v1/responses,
/v1/messages, /v1/batches or /v1/files, and mask_input / mask_output were
dropped with them so a caller asking for redaction had their prompt logged in
full.
Promote an explicit allow-list of those fields from the requester_metadata
snapshot into litellm_metadata, never overwriting a value already set so
header-derived ids keep precedence. Trace-mutation controls (existing_trace_id,
update_trace_keys) and trace_public are deliberately excluded: langfuse applies
them to an arbitrary caller-chosen trace with no ownership check. tags is
excluded because per-tag budget enforcement runs earlier, at auth time.
This covers providers with a native Responses API config. Providers reaching
/v1/responses through the chat-completions bridge need the companion change to
get_litellm_params.
* ci: retrigger workflows
* warn at startup when a proxy-wide budget is set but no DB is connected
litellm.max_budget is only enforced via DB-loaded global spend, so a DB-less proxy silently ignores it. Log a one-time startup warning.
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* refactor(proxy): inject max_budget into DB-less budget warning
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(proxy): cover DB-less budget warning startup call
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(proxy): pin DB-less budget warning call site
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(proxy): stabilize budget warning call-site pin
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: tin <tin@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Bare fireworks_ai/<slug> only resolved to accounts/fireworks/models/<slug>,
so Fireworks routers (served at accounts/fireworks/routers/<id>, e.g.
glm-latest and firerouter) could not be reached without passing the full
resource id. Add a shared resolve_fireworks_resource_name helper that maps an
explicit routers/<id> or models/<id> segment to the right resource path, keeps
the existing -fast router heuristic, and defaults bare slugs to models/ for
backward compatibility. Wire it into both the chat and text-completion
transforms, which had drifted (completion lacked router handling entirely)
PR #35956 added the max_input_chars passthrough to the AutoRouter
constructor but left this mock assertion in tests/router_unit_tests
unchanged, so test_init_auto_router_deployment_success has been failing
on litellm_internal_staging ever since.
The passthrough itself is intentional and its behaviour is already
covered by TestAutoRouterMaxInputCharsWiring in tests/test_litellm, so
only the stale expected kwargs need updating. Assert the shared constant
rather than the literal 2000 so tuning the default does not break this
test again.
When store_model_in_db is true, general_settings are persisted to the
LiteLLM_Config DB table. On subsequent startups and periodic reloads,
_add_general_settings_from_db_config() unconditionally overwrites the
in-memory general_settings with DB-cached values, including
store_prompts_in_spend_logs.
This means a YAML config change (e.g. store_prompts_in_spend_logs: false)
deployed via CI/CD has no effect because the stale DB value (true) always
wins. The admin must manually update via /config/update API after every
deploy, defeating config-as-code.
Fix: track which general_settings keys were explicitly set in YAML at
startup (_yaml_general_settings_keys). During DB config merge, prefer the
YAML value for tracked keys. The DB value is only used as fallback when
YAML does not set the key, preserving the admin UI's ability to change
settings at runtime.
Steps to reproduce:
1. Start proxy with store_model_in_db: true, store_prompts_in_spend_logs: true
2. Change YAML to store_prompts_in_spend_logs: false, restart
3. Send a request, query LiteLLM_SpendLogs - prompts still stored
4. Check LiteLLM_Config table - DB still has true, overriding YAML
Slack thread: https://dataset-jsonhackathon.slack.com/archives/C0ACUS7LM29/p1785835131860139
Map the remaining gateway-documented effort keys: thinking as an alias
for enable_thinking (enable_thinking wins when both are present),
reasoning_budget to an integer reasoning_effort (skipped when thinking
is explicitly off), and low_effort=true to reasoning_effort=low (budget
wins when both are set). guided_json and guided_choice response_format
wrappers now include the name field (response and choice) to match the
gateway wire shape.
GET /v1/files filters data down to the caller's own managed files but left first_id and last_id as the upstream page's, so a non-owner got back file ids belonging to other users even with an empty data array
The native files and batches routes declare /{provider}/v1/... and their routers are mounted before the passthrough router, so /openai_passthrough/v1/files and /openai_passthrough/v1/batches matched them with provider="openai_passthrough" and 500'd on the LlmProviders lookup instead of reaching openai_proxy_route.
Move the dedicated /openai_passthrough prefix onto its own router mounted ahead of the batches and files routers. /openai/... and every other provider prefix keep their current behavior.
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Revert the LIT001 build-clause inserts, phrase the LIT002 menu as
'or (if it really must be dynamic) a MappingProxyType wrapping a dict
literal or comprehension', and fold the two freezing-wrapper exemption
sentences into one that names MappingProxyType beside tuple/frozenset.
LIT001/LIT002 and the typing.Dict ban all steered dict-shaped values to
frozen dataclasses or suppression even though the checker already accepts
MappingProxyType as a freezing wrapper; the messages now name it so the
dict-shaped freeze path is actually discoverable at fix time.
Storing a baseline used to prune every other cache entry, so gate runs in
concurrent worktrees kept evicting each other's baselines and forcing full
recomputes: this bit six times across two nights of benchmarking. The store
now writes alongside existing entries and evicts only the oldest beyond
eight, keyed as before by merge-base and environment fingerprint, so
parallel worktrees' baselines simply coexist
Prompt Security drops tool and function rows unless check_tool_results
is on, so it now reports scan-only support from that setting and the
registry refuses the pairing at boot. Pairing scan_only_tool_results
with skip_tool_message_in_guardrail excludes every message, so guardrail
initialization now rejects that combination too.
prisma resolves its prisma-client-py generator through a plain /bin/sh PATH
lookup, never through the interpreter that ran prisma generate, so the gate's
generate step landed the client in whatever venv the caller had on PATH: the
owned env never received one, every gate run regenerated, the caller's venv
was mutated instead, and any invocation without a venv on PATH (the rewritten
publisher workflow) failed outright
The generate now runs with the target interpreter's bin directory pinned to
the front of the child PATH. The prisma schema joins the environment
fingerprint so clientless counts recorded before this commit can never be
compared against clientful ones, a cold provision announces itself on stderr
instead of sitting silent for two minutes, and the CI gate step reuses the
job's prisma binary cache
Under scan_only_tool_results, legacy OpenAI function-role messages now count as tool results, and duplicate names among guardrail-returned tools keep only the first occurrence. CustomGuardrail.structured_messages_cover_full_request lets CrowdStrike AIDR declare that its writeback already rebuilds the whole conversation, so handlers install it as-is instead of merging it into the full message list a second time and duplicating out-of-scope rows. Lint budget ceilings ratchet down to match the tree
Replaces the hand-rolled recording double with AsyncMock so the awaited team ids
come from await_args_list instead of a mutated list, and annotates the response
factory now that SearchToolInfoResponse is imported at module level.
The aggregated route was missing from LiteLLMRoutes.self_managed_routes
while its paginated sibling /user/daily/activity was listed, so auth
rejected every internal user with a 401 before the handler ran. That
route backs the default "Your Usage" view in the dashboard, which left
the main Usage page broken for non-admin users.
The handler already self-scopes: it checks admin view first, then falls
back to require_caller_user_id_for_non_admin, defaults a missing user_id
to the caller's own, and returns 403 when a non-admin asks for someone
else's data. Listing the route restores reachability without widening
what a caller can read.
check_route_access matches exactly (plus explicit wildcards), so the
parent entry never covered the /aggregated sub-path.
Every Admin UI session key is stamped with the reserved team id
`litellm-dashboard`, which never has a row in LiteLLM_TeamTable, so the
team lookup in _filter_visible_search_tools raised 404 and the endpoint
returned 500 for every non-admin dashboard session.
Skip the lookup for that sentinel and scope the caller by its key-level
allowlist alone, matching how MCP and agent permission checks already
treat it. A real team id is still resolved, and a genuine lookup failure
now surfaces with its own status instead of being masked as a 500.
