mirror of
https://github.com/BerriAI/litellm.git
synced 2026-11-15 09:12:18 +00:00
dc378de207
* fix(cli): let Claude Code renew a --pkce login's key through lite's apiKeyHelper * fix(cli): keep an explicit --api-key over lite's apiKeyHelper and persist its absolute path An explicit --api-key or LITELLM_PROXY_API_KEY is now exported to Claude Code even when lite's renewing helper is configured, since the helper prints the stored login's key instead. The helper command also uses the absolute path of the lite found on PATH, so a relative PATH entry cannot make Claude Code run another repository's bin/lite * fix(cli): keep an explicit key and --pkce when lite's renewing helper is set A --settings that blanks apiKeyHelper now keeps the exported key, and a key passed with --api-key turns lite's helper off for that launch, so Claude Code no longer warns about two credentials or sends the stored login's key beside it The helper names the lite installed beside the Python that ran the login before one found on PATH, so a pyenv shim no longer fails in projects pinned to another Python lite claude signs in again with --pkce when the stored login used it, and print-token's hints name the proxy and --pkce * fix(cli): keep a shell token from beating the renewing helper and reuse --pkce only for the same proxy The --pkce --config-claude login now writes an empty env.ANTHROPIC_AUTH_TOKEN beside the helper, since a token exported in the shell wins over apiKeyHelper lite claude sends any other key through a private settings file that blanks the helper and merges the user's own --settings A re-login keeps --pkce only when the stored login is for the same proxy, else the saved default decides * fix(cli): let any lite install's helper renew for its proxy and name the renewing login in print-token hints lite claude now treats a lite auth print-token helper for this proxy as renewing whichever lite path it names, since every lite reads the same token store Every print-token hint to sign in again, including an expired key and an unreadable keychain, names lite login --pkce with the proxy Shorter docstrings * fix(cli): send a launch key to the proxy it was issued for The private settings file lite claude writes for a key that lite's helper would not renew now also sets ANTHROPIC_BASE_URL. Without it, lite --base-url B --api-key <B key> claude after a --pkce --config-claude login to A sent the B key to A, since the login's settings point Claude Code at A * fix(proxy): preserve caller metadata on auth failures (#51745) * fix(proxy): keep x-litellm-spend-logs-metadata on auth-rejected failure spend logs Budget, invalid-key and other auth-stage rejections are raised before the request headers are merged into metadata, so the persisted failure row lost the caller's spend logs metadata header. The auth failure handler now parses it with the same pre-call helper and stamps it into the metadata bucket it already uses for the requester IP and user agent. * fix(proxy): type the spend logs metadata header parser honestly The parser takes the request headers as a str mapping and returns the parsed JSON as object, which is what successful requests already store. The auth failure path keeps only JSON objects. The precedence regression now also asserts the caller's request body is left untouched. * fix(proxy): stamp the auth failure spend logs metadata into the route's bucket A chat request whose body also carried litellm_metadata got the header in litellm_metadata, which the failure spend row never reads on chat routes. The header now goes into the bucket picked by metadata_variable_name_for_request for the route, and the failure row lifts spend_logs_metadata from that bucket the same way it already lifts the requester IP and user agent. The regression tests now assert on the built failure spend row. * fix(proxy): keep the auth failure spend logs metadata fix inside the auth handler Restores the failure spend row's lifted fields to main, so failures after auth on litellm_metadata routes keep their current logging. The auth handler now writes the parsed header into metadata, which the failure spend row reads on every route, and into the route's own bucket for callbacks on litellm_metadata routes. * fix(proxy): make the spend logs metadata header field read-only and test the real spend writer The header collector now sets spend_logs_metadata when it is built, so the field can be ReadOnly with the same stored value. The auth failure tests run the real DBSpendUpdateWriter and read the queued spend log row from a fake database client instead of an autospec writer. * test(proxy): read auth failure spend rows from a real unconnected PrismaClient Replaces the mocked database client and the class-level queue byte reset. The test now enqueues into a real PrismaClient's own queue and dequeues its row in cleanup, which keeps the shared byte counter balanced. --------- Co-authored-by: Joshua Valluru <326636767+joshua-berri@users.noreply.github.com> * fix(logging): keep a content-block system prompt in logged messages (#51699) * fix(logging): keep a content-block system prompt in logged messages A /v1/messages request whose top-level system prompt is a list of content blocks never reached the logged messages, because only a plain-text system prompt was prepended. Accept a non-empty list too, so every logger that reads messages sees it. Claude on Vertex also normalizes a plain-text system prompt into blocks, so it was dropped there as well. Fixes #36402 * fix(logging): type the logged system message * fix(logging): type the system message without a mutable annotation, close the test client * refactor(mcp): let the semantic filter take its MCP tool loader as a parameter (#50483) * test(mcp): check the tool dict conversion without patching the MCP handler * refactor(mcp): let the semantic filter take its MCP tool loader as a parameter * refactor(mcp): type the semantic filter's default tool loader Annotate the chosen loader as _OpenAIToolLoader and narrow its return to FunctionToolParam, so a default that returns raw MCP tools fails the type check. Drop the unused patch import. * fix(mcp): give the MCP client exactly the servers left on in the connect page (#50973) * fix(cli): send the launch key under a project apiKeyHelper and give each key its own settings file A `lite login --pkce --config-claude` leaves an empty ANTHROPIC_AUTH_TOKEN in Claude Code's user settings. When a repo's project settings set their own apiKeyHelper, `lite claude` skipped its private settings file, so the empty token hid the exported key and Claude Code sent the project helper's key. `lite claude` now also writes that file whenever the settings files leave ANTHROPIC_AUTH_TOKEN empty Every key launch also wrote the one file `--subscription` uses, so two launches started together could read each other's key and proxy. A key launch's file is now named after a hash of its contents, and every overlay is written to a temp file and renamed so a starting Claude Code never reads half of one * fix(proxy): keep carried end-user spend when a budget reset is retried (#51143) * fix(proxy): keep carried end-user spend when a budget reset is retried End-user pages commit before the budget window advance. When the advance failed, the tier stayed due and the next tick replayed every page, so a rollover-capped row already cut to its carryover (15 -> 5) matched the under-cap zero and lost it, and spend accrued since the first pass was zeroed again. Each end-user reset now stamps the tier window it closes in a new spend_reset_window column and skips rows already stamped with it. Fixes #50898 * fix(proxy): leave never-reset tiers unguarded and test each tier's own window A tier with no budget_reset_at falls back to the old unguarded reset instead of stamping created_at, which an admin clearing the reset time would reuse and silently skip end users. Tests now give tiers distinct windows and cover a second window closing. * test(proxy): give the end-user ledger its own fake prisma client * fix(proxy): pin a reset time on tiers without one before end-user pages commit * fix(proxy): read the reset-time pin from the writer and skip tiers another sweep advanced * fix(proxy): keep linked rows of a tier another sweep advanced out of the rollover reset * fix(proxy): order the end-user reset window migration after upstream's same-timestamp migration * fix(proxy): reset end users again when an admin moves a tier's reset time back to a closed window * fix(proxy): clear end-user reset stamps in the same transaction as the tier's reset time A failed /budget/update used to leave the new reset time written with the stamps still set, so repeating the request found the time unchanged and skipped the clear. The update and the clear now commit together, and a reset time in the future no longer scans the tier's end users * fix(proxy): name each end-user reset window by the tier's last write Stamp end-user resets with the tier's updated_at read on the writer instead of its reset time. A failed window advance leaves updated_at as it was, so the retry still skips the end users already reset, while the advance or any /budget/update moves it and opens a new window. That covers a forced reset on the default end-user tier and drops the stamp clear, so /budget/update is the one-row update it is on main. The writer read also leaves out a tier another sweep already advanced, which a lagging read replica still showed as due * fix(proxy): name end-user reset windows by the tier and its advance count Tiers advanced in one batch share updated_at, so a customer moved between them could skip the new tier's reset, and any unrelated tier edit during a failed advance reopened the window. A budget_reset_count that only the window advance bumps, prefixed with the tier id, names the window instead. * fix(proxy): keep carried spend for customer teams and members on a retried reset Customer team and member pages commit before the window advance like end-user pages, so they now stamp the window they close and a retry skips them too. A member runs on its unexpired override when that has its own duration, else on its template. The window name now joins the tier id, its creation time, its count of closed windows and its reset time, so a duration change or a tier recreated under the same id opens a new window even when the advance failed. * test(integration): list every budget column and relation in the budget update field check --------- Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com> * test(otel): keep proxy startup out of the per-test timeout in the excluded services matrix (#51759) Co-authored-by: mateo <277851410+mateo-berri@users.noreply.github.com> Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix(cli): skip the working directory when the login looks up lite for its helper On Windows, shutil.which searches the current directory before the directories it is given, so a lite.exe planted in the repo where someone ran `lite login --pkce --config-claude` could become their user-level apiKeyHelper The login now checks each directory by full path, this Python's scripts directory first and then each PATH entry, skipping empty entries * fix(bedrock): answer 400 for service tiers Bedrock cannot serve, forward reserved (#50516) * fix(bedrock): answer 400 for service tiers Bedrock cannot serve, forward reserved Converse used to drop any service_tier outside default, flex, and priority with no error, rewrite auto to default, and drop reserved. Now reserved and standard_only map to Bedrock tiers, auto sends no tier, and any other value answers 400 unless drop_params is on. Nova batch rows honor the row's drop_params. * fix(bedrock): keep a Responses batch row's drop_params and keep it out of passthrough rows * test(bedrock): annotate service tier test locals as Final and drop the row helper's kwargs * fix(bedrock): refuse a service tier on Nova batch rows unless drop_params, and honor the deployment's drop_params * fix(bedrock): map standard and any-case service tiers, and drop tiers from Claude batch rows under the deployment's drop_params * fix(bedrock): forward the tiers OpenAI-format batch rows take, accept auto on Claude rows, and label tiers in lowercase * fix(bedrock): mark the service tier block field read-only * fix(bedrock): read drop_params given as a string on batch rows, and cover the Nova row mapper's drop_params * test(prometheus): sort the service tier label test imports * fix(bedrock): send the Invoke route's service tier as a header and name the accepted tiers in the 400 * fix(bedrock): report and bill the tier Bedrock served on Invoke calls and streams, and send the tier header only to Nova * fix(bedrock): return copies when stamping the served tier, read its header without re-encoding the others, and pin it per stream chunk The Nova Invoke mapper and the event-stream decoder now return model_copy results instead of writing service_tier in place. One shared reader finds the X-Amzn-Bedrock-Service-Tier response header case-insensitively, so a non-ASCII request id no longer breaks the stream decoder, and the passthrough uses the same reader. Tests now vary the served tier in both directions, check every chunk, and cover a stream with no tier header or an empty one * test(bedrock): check the served tier on a stream whose headers come as a plain dict with an upper-case name The non-ASCII request id stream test now also carries X-AMZN-BEDROCK-SERVICE-TIER in a plain dict and asserts every chunk reports that tier, so a case-sensitive header reader or a lookup by the canonical name fails it * refactor(bedrock): drop the passthrough's own service tier header reader that main re-added The passthrough already reads the served tier through the shared case-insensitive reader in common_utils, so the httpx.Headers copy main brought back with the embeddings change has no callers and references a constant this branch removed --------- Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com> * fix(cli): delete the key files lite claude leaves on logout and unconfigure lite claude hands its process to Claude Code, so the --settings files holding its key stayed in the Claude config dir. lite logout and lite unconfigure claude now delete them, and Claude Code reads one only at startup, so open sessions keep working --------- Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com> Co-authored-by: joshua-berri <joshua@berri.ai> Co-authored-by: Joshua Valluru <326636767+joshua-berri@users.noreply.github.com> Co-authored-by: ryan-crabbe-berri <ryan@berri.ai> Co-authored-by: shrey <shrey@berri.ai> Co-authored-by: Ankit Jha <45779665+PyExtreme@users.noreply.github.com> Co-authored-by: devin-ai-integration[bot] <158243242+devin-ai-integration[bot]@users.noreply.github.com> Co-authored-by: quietjules <349048734+quietjules@users.noreply.github.com> |
||
|---|---|---|
| .. | ||
| cli | ||
| __init__.py | ||
| conftest.py | ||
| test_chat.py | ||
| test_client.py | ||
| test_credentials.py | ||
| test_http_client.py | ||
| test_http_commands.py | ||
| test_keys.py | ||
| test_model_groups.py | ||
| test_models.py | ||
| test_teams.py | ||
| test_users.py | ||