mirror of
https://github.com/BerriAI/litellm.git
synced 2026-11-15 09:12:18 +00:00
22 commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
norkelbyte
|
dc378de207
|
fix(cli): let Claude Code renew a --pkce login's key through lite's apiKeyHelper (#51319)
* fix(cli): let Claude Code renew a --pkce login's key through lite's apiKeyHelper * fix(cli): keep an explicit --api-key over lite's apiKeyHelper and persist its absolute path An explicit --api-key or LITELLM_PROXY_API_KEY is now exported to Claude Code even when lite's renewing helper is configured, since the helper prints the stored login's key instead. The helper command also uses the absolute path of the lite found on PATH, so a relative PATH entry cannot make Claude Code run another repository's bin/lite * fix(cli): keep an explicit key and --pkce when lite's renewing helper is set A --settings that blanks apiKeyHelper now keeps the exported key, and a key passed with --api-key turns lite's helper off for that launch, so Claude Code no longer warns about two credentials or sends the stored login's key beside it The helper names the lite installed beside the Python that ran the login before one found on PATH, so a pyenv shim no longer fails in projects pinned to another Python lite claude signs in again with --pkce when the stored login used it, and print-token's hints name the proxy and --pkce * fix(cli): keep a shell token from beating the renewing helper and reuse --pkce only for the same proxy The --pkce --config-claude login now writes an empty env.ANTHROPIC_AUTH_TOKEN beside the helper, since a token exported in the shell wins over apiKeyHelper lite claude sends any other key through a private settings file that blanks the helper and merges the user's own --settings A re-login keeps --pkce only when the stored login is for the same proxy, else the saved default decides * fix(cli): let any lite install's helper renew for its proxy and name the renewing login in print-token hints lite claude now treats a lite auth print-token helper for this proxy as renewing whichever lite path it names, since every lite reads the same token store Every print-token hint to sign in again, including an expired key and an unreadable keychain, names lite login --pkce with the proxy Shorter docstrings * fix(cli): send a launch key to the proxy it was issued for The private settings file lite claude writes for a key that lite's helper would not renew now also sets ANTHROPIC_BASE_URL. Without it, lite --base-url B --api-key <B key> claude after a --pkce --config-claude login to A sent the B key to A, since the login's settings point Claude Code at A * fix(proxy): preserve caller metadata on auth failures (#51745) * fix(proxy): keep x-litellm-spend-logs-metadata on auth-rejected failure spend logs Budget, invalid-key and other auth-stage rejections are raised before the request headers are merged into metadata, so the persisted failure row lost the caller's spend logs metadata header. The auth failure handler now parses it with the same pre-call helper and stamps it into the metadata bucket it already uses for the requester IP and user agent. * fix(proxy): type the spend logs metadata header parser honestly The parser takes the request headers as a str mapping and returns the parsed JSON as object, which is what successful requests already store. The auth failure path keeps only JSON objects. The precedence regression now also asserts the caller's request body is left untouched. * fix(proxy): stamp the auth failure spend logs metadata into the route's bucket A chat request whose body also carried litellm_metadata got the header in litellm_metadata, which the failure spend row never reads on chat routes. The header now goes into the bucket picked by metadata_variable_name_for_request for the route, and the failure row lifts spend_logs_metadata from that bucket the same way it already lifts the requester IP and user agent. The regression tests now assert on the built failure spend row. * fix(proxy): keep the auth failure spend logs metadata fix inside the auth handler Restores the failure spend row's lifted fields to main, so failures after auth on litellm_metadata routes keep their current logging. The auth handler now writes the parsed header into metadata, which the failure spend row reads on every route, and into the route's own bucket for callbacks on litellm_metadata routes. * fix(proxy): make the spend logs metadata header field read-only and test the real spend writer The header collector now sets spend_logs_metadata when it is built, so the field can be ReadOnly with the same stored value. The auth failure tests run the real DBSpendUpdateWriter and read the queued spend log row from a fake database client instead of an autospec writer. * test(proxy): read auth failure spend rows from a real unconnected PrismaClient Replaces the mocked database client and the class-level queue byte reset. The test now enqueues into a real PrismaClient's own queue and dequeues its row in cleanup, which keeps the shared byte counter balanced. --------- Co-authored-by: Joshua Valluru <326636767+joshua-berri@users.noreply.github.com> * fix(logging): keep a content-block system prompt in logged messages (#51699) * fix(logging): keep a content-block system prompt in logged messages A /v1/messages request whose top-level system prompt is a list of content blocks never reached the logged messages, because only a plain-text system prompt was prepended. Accept a non-empty list too, so every logger that reads messages sees it. Claude on Vertex also normalizes a plain-text system prompt into blocks, so it was dropped there as well. Fixes #36402 * fix(logging): type the logged system message * fix(logging): type the system message without a mutable annotation, close the test client * refactor(mcp): let the semantic filter take its MCP tool loader as a parameter (#50483) * test(mcp): check the tool dict conversion without patching the MCP handler * refactor(mcp): let the semantic filter take its MCP tool loader as a parameter * refactor(mcp): type the semantic filter's default tool loader Annotate the chosen loader as _OpenAIToolLoader and narrow its return to FunctionToolParam, so a default that returns raw MCP tools fails the type check. Drop the unused patch import. * fix(mcp): give the MCP client exactly the servers left on in the connect page (#50973) * fix(cli): send the launch key under a project apiKeyHelper and give each key its own settings file A `lite login --pkce --config-claude` leaves an empty ANTHROPIC_AUTH_TOKEN in Claude Code's user settings. When a repo's project settings set their own apiKeyHelper, `lite claude` skipped its private settings file, so the empty token hid the exported key and Claude Code sent the project helper's key. `lite claude` now also writes that file whenever the settings files leave ANTHROPIC_AUTH_TOKEN empty Every key launch also wrote the one file `--subscription` uses, so two launches started together could read each other's key and proxy. A key launch's file is now named after a hash of its contents, and every overlay is written to a temp file and renamed so a starting Claude Code never reads half of one * fix(proxy): keep carried end-user spend when a budget reset is retried (#51143) * fix(proxy): keep carried end-user spend when a budget reset is retried End-user pages commit before the budget window advance. When the advance failed, the tier stayed due and the next tick replayed every page, so a rollover-capped row already cut to its carryover (15 -> 5) matched the under-cap zero and lost it, and spend accrued since the first pass was zeroed again. Each end-user reset now stamps the tier window it closes in a new spend_reset_window column and skips rows already stamped with it. Fixes #50898 * fix(proxy): leave never-reset tiers unguarded and test each tier's own window A tier with no budget_reset_at falls back to the old unguarded reset instead of stamping created_at, which an admin clearing the reset time would reuse and silently skip end users. Tests now give tiers distinct windows and cover a second window closing. * test(proxy): give the end-user ledger its own fake prisma client * fix(proxy): pin a reset time on tiers without one before end-user pages commit * fix(proxy): read the reset-time pin from the writer and skip tiers another sweep advanced * fix(proxy): keep linked rows of a tier another sweep advanced out of the rollover reset * fix(proxy): order the end-user reset window migration after upstream's same-timestamp migration * fix(proxy): reset end users again when an admin moves a tier's reset time back to a closed window * fix(proxy): clear end-user reset stamps in the same transaction as the tier's reset time A failed /budget/update used to leave the new reset time written with the stamps still set, so repeating the request found the time unchanged and skipped the clear. The update and the clear now commit together, and a reset time in the future no longer scans the tier's end users * fix(proxy): name each end-user reset window by the tier's last write Stamp end-user resets with the tier's updated_at read on the writer instead of its reset time. A failed window advance leaves updated_at as it was, so the retry still skips the end users already reset, while the advance or any /budget/update moves it and opens a new window. That covers a forced reset on the default end-user tier and drops the stamp clear, so /budget/update is the one-row update it is on main. The writer read also leaves out a tier another sweep already advanced, which a lagging read replica still showed as due * fix(proxy): name end-user reset windows by the tier and its advance count Tiers advanced in one batch share updated_at, so a customer moved between them could skip the new tier's reset, and any unrelated tier edit during a failed advance reopened the window. A budget_reset_count that only the window advance bumps, prefixed with the tier id, names the window instead. * fix(proxy): keep carried spend for customer teams and members on a retried reset Customer team and member pages commit before the window advance like end-user pages, so they now stamp the window they close and a retry skips them too. A member runs on its unexpired override when that has its own duration, else on its template. The window name now joins the tier id, its creation time, its count of closed windows and its reset time, so a duration change or a tier recreated under the same id opens a new window even when the advance failed. * test(integration): list every budget column and relation in the budget update field check --------- Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com> * test(otel): keep proxy startup out of the per-test timeout in the excluded services matrix (#51759) Co-authored-by: mateo <277851410+mateo-berri@users.noreply.github.com> Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix(cli): skip the working directory when the login looks up lite for its helper On Windows, shutil.which searches the current directory before the directories it is given, so a lite.exe planted in the repo where someone ran `lite login --pkce --config-claude` could become their user-level apiKeyHelper The login now checks each directory by full path, this Python's scripts directory first and then each PATH entry, skipping empty entries * fix(bedrock): answer 400 for service tiers Bedrock cannot serve, forward reserved (#50516) * fix(bedrock): answer 400 for service tiers Bedrock cannot serve, forward reserved Converse used to drop any service_tier outside default, flex, and priority with no error, rewrite auto to default, and drop reserved. Now reserved and standard_only map to Bedrock tiers, auto sends no tier, and any other value answers 400 unless drop_params is on. Nova batch rows honor the row's drop_params. * fix(bedrock): keep a Responses batch row's drop_params and keep it out of passthrough rows * test(bedrock): annotate service tier test locals as Final and drop the row helper's kwargs * fix(bedrock): refuse a service tier on Nova batch rows unless drop_params, and honor the deployment's drop_params * fix(bedrock): map standard and any-case service tiers, and drop tiers from Claude batch rows under the deployment's drop_params * fix(bedrock): forward the tiers OpenAI-format batch rows take, accept auto on Claude rows, and label tiers in lowercase * fix(bedrock): mark the service tier block field read-only * fix(bedrock): read drop_params given as a string on batch rows, and cover the Nova row mapper's drop_params * test(prometheus): sort the service tier label test imports * fix(bedrock): send the Invoke route's service tier as a header and name the accepted tiers in the 400 * fix(bedrock): report and bill the tier Bedrock served on Invoke calls and streams, and send the tier header only to Nova * fix(bedrock): return copies when stamping the served tier, read its header without re-encoding the others, and pin it per stream chunk The Nova Invoke mapper and the event-stream decoder now return model_copy results instead of writing service_tier in place. One shared reader finds the X-Amzn-Bedrock-Service-Tier response header case-insensitively, so a non-ASCII request id no longer breaks the stream decoder, and the passthrough uses the same reader. Tests now vary the served tier in both directions, check every chunk, and cover a stream with no tier header or an empty one * test(bedrock): check the served tier on a stream whose headers come as a plain dict with an upper-case name The non-ASCII request id stream test now also carries X-AMZN-BEDROCK-SERVICE-TIER in a plain dict and asserts every chunk reports that tier, so a case-sensitive header reader or a lookup by the canonical name fails it * refactor(bedrock): drop the passthrough's own service tier header reader that main re-added The passthrough already reads the served tier through the shared case-insensitive reader in common_utils, so the httpx.Headers copy main brought back with the embeddings change has no callers and references a constant this branch removed --------- Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com> * fix(cli): delete the key files lite claude leaves on logout and unconfigure lite claude hands its process to Claude Code, so the --settings files holding its key stayed in the Claude config dir. lite logout and lite unconfigure claude now delete them, and Claude Code reads one only at startup, so open sessions keep working --------- Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com> Co-authored-by: joshua-berri <joshua@berri.ai> Co-authored-by: Joshua Valluru <326636767+joshua-berri@users.noreply.github.com> Co-authored-by: ryan-crabbe-berri <ryan@berri.ai> Co-authored-by: shrey <shrey@berri.ai> Co-authored-by: Ankit Jha <45779665+PyExtreme@users.noreply.github.com> Co-authored-by: devin-ai-integration[bot] <158243242+devin-ai-integration[bot]@users.noreply.github.com> Co-authored-by: quietjules <349048734+quietjules@users.noreply.github.com> |
||
|
ryan-crabbe-berri
|
e95bf288f3
|
feat(cli): connect a per-user GitHub Copilot credential from the command line (#50863)
* feat(cli): connect a per-user GitHub Copilot credential from the command line lite credentials connect runs the GitHub device login with the caller's own key, so users can connect Copilot without opening the LiteLLM UI. * fix(cli): stop connect before the GitHub code expires and explain lost connections The connect command kept polling until the proxy rejected the expired code with a raw HTTP 400, so it now stops one interval before expiry and prints its own expired message. A dropped connection or timeout while waiting now prints a short rerun message instead of a traceback. * fix(cli): keep checking for approval until the GitHub code actually expires Connect stopped one interval early, so an approval in the last seconds was missed. The countdown now starts when the code is requested, which is never later than the proxy's own cutoff, and polling continues while another poll fits. |
||
|
zoveki
|
cfddaafe18
|
fix(cli): end the local login when lite logout cannot check the OS keychain (#50098)
* fix(cli): end the local login when logout cannot check the OS keychain With no usable keyring, lite logout kept a secret-free token.json as a note that the keychain went unchecked. Every reader took that note for a live keychain-backed login, so lite whoami, lite auth print-token, and the bare lite shell all reported a login the user had just ended, and a keychain that answered again could pair the note with an earlier login's entry and sign the machine back in. Logout now removes token.json whatever the keychain answers. When the keychain could not be checked and there was a login to end, it records that in a separate owner-only keychain_unchecked file beside it, written after the removal so a full disk still has room for it. A repeat logout still warns while that file is there, and a logout that confirms the keychain is clean removes it. * fix(cli): keep warning after a logout the keychain refused to clear A keychain that will not give up its entry gets the same note as one that could not be checked, so a later logout made while the keychain is out of reach does not report a clean logout while the credential is still there. * fix(cli): revoke the login a logout left in an unreachable keychain once it answers A logout made while the keychain was locked ended the local login and dropped the metadata that pairs with the refresh token still in the keychain, so the next logout had nothing to revoke. The not-cleared note now keeps that record without its secret, and a later logout pairs it with the keychain entry and revokes it before erasing. * fix(cli): revoke a locked-keychain login on the next login, and only once after a refused erase A login made after a logout that could not reach the keychain now revokes the login it replaces, read from the keychain_not_cleared note, as the merge base did through the kept token.json. A keychain that refused the erase had already handed that logout the refresh token, so its note no longer pairs with the entry and the next logout does not revoke the same token again. * fix(cli): keep the first login revocable after a second locked-keychain logout A logout whose login kept its secret in token.json never reached the keychain, so it now leaves the note an earlier logout wrote in place, or writes an empty one, instead of overwriting it. The first login's refresh token stays revocable once the keychain answers, and a keychain-less machine no longer keeps user and team ids in the note. Also documents the limits that remain in the CLI README and drops the docstrings from the tests this PR added. * fix(cli): revoke the keychain's login after a login made while the keychain was locked * fix(cli): stop lite from reading the stored login before login and logout The cli group resolved the stored API key before every command, and that read moved a locked-keychain login's secret over the keychain entry the logout note names, so logout and login never saw the first login to revoke it. * fix(cli): keep the login a locked logout left in the keychain revocable While the keychain still holds the login an earlier logout could not end, reading the stored login now serves the secret token.json kept and leaves that keychain entry alone, whatever the login times say. A clock that stepped back no longer serves the ended login, and commands other than logout and login no longer move the newer secret over it. A silent renewal that writes over the entry revokes that login on the proxy. * fix(cli): keep a login revocable when its note cannot be written, and never block logout on an earlier login's proxy A locked logout whose note cannot take the login's record now puts token.json back, so the next logout that reaches the keychain still revokes that login, as the base did. A logout that ends both a current login and an earlier one left in the keychain warns when the earlier login's proxy cannot record the revocation, as lite login does, instead of refusing to log out of the current proxy. * fix(cli): move a locked keychain login's token file into its note instead of putting it back * fix(cli): never tell a logout to delete the record of a keychain login it could not move aside A read-only ~/.litellm made the locked-keychain logout say to delete token.json, which would leave the keychain's login unrevoked. It now says the logout did not happen and asks to make the directory writable. The README also says an older lite cannot finish a logout this release started. --------- Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com> |
||
|
ryan-crabbe-berri
|
9a473fbbb0
|
feat(proxy): give Codex accurate model entries for every model in /v1/models (#49825)
* feat(proxy): give Codex accurate model entries for every model in /v1/models * fix(router): read a deployment's upstream model without a not-required key access * fix(proxy): keep Codex tiers for unpriced deployments and match stock entries by declared provider A deployment with no token price no longer votes a Codex service tier off, an undescribed model keeps Codex's default reasoning summary parameter, a Bedrock deployment naming its provider in custom_llm_provider gets Codex's stock entry, and Bedrock provider namespaces follow the AWS model id format. * fix(proxy): import cost tracking and provider helpers lazily in the Codex catalog lite codex imports the catalog before the SDK is initialized, so importing cost tracking, the Bedrock helpers or provider resolution at module level hit a circular import and broke every lite command. * fix(proxy): drop Codex's fast speed tier wherever its priority tier is dropped Codex shows /fast for a leftover fast speed tier even when service_tiers no longer offers priority, so an alias, an empty configured tier list or a configured list without priority showed a /fast toggle that sends nothing. One helper now sets service tiers, speed tiers and the default together. * fix(proxy): give an older Codex a translated entry for models past its version Codex's own entries name the oldest release that can read them in minimal_client_version. A release older than that can fail to decode the entry, such as a reasoning effort it has no name for, which fails the whole catalog, so it now gets the translated entry, whose levels are trimmed to that release |
||
|
quietjules
|
d302096b5a
|
feat(lite): offer each model group's reasoning efforts in pi, OpenCode and Codex (#48949)
* feat(lite): offer each model group's reasoning efforts in pi, OpenCode and Codex lite pi, lite opencode and lite codex now read supported_reasoning_efforts from /model_group/info and write them into each client's model config. pi gets reasoning and a thinkingLevelMap, OpenCode gets reasoning_options plus one variant per effort, and Codex fallback entries get supported_reasoning_levels with a default level. The proxy-served Codex catalog on /v1/models?client_version= offers the same levels Codex before 0.138.0 fails the whole catalog on an effort it cannot name, so those releases, and an unreadable version, get only the six efforts every release parses. A group with unknown efforts keeps today's output, and a group with none is written as non-reasoning The strength order and nearest_declared_reasoning_effort move to an SDK-free module so the lite CLI, which loads without initializing the SDK, can import the Codex catalog * fix(lite): describe team models by their public name and keep Codex's none default /model_group/info now looks up a team's own deployments when the caller's team serves the requested name, so a team model reports its reasoning levels. Codex fallback entries default to none when the group lists it, and the proxy builds the Codex catalog in a worker thread * fix(lite): round Codex's default level down and describe a team's shared-name model from its own deployments Move the reasoning strength helper to litellm_core_utils so lite and the Codex catalog import it without the SDK or the router, and let make lint-format-check-changed diff the working tree so a staged rename passes. * fix(lite): keep pi's system role on reasoning model groups pi switches its system prompt to the developer role for any model marked reasoning, so an operator's cache_control_injection_points entry for role system stopped matching on Claude groups. lite pi now sets compat.supportsDeveloperRole false on those entries, so pi sends system as it did before. * fix(codex): treat a client_version too long to parse as an old Codex and keep undescribed stock levels A client_version past Python's integer digit limit made the Codex catalog answer 500. It now gets the levels every Codex parses. A stock Codex level with no description validates again, as on base, and lite codex treats undecodable codex --version output as an unknown version * fix(lite): keep REASONING_EFFORT_STRENGTH_ORDER importable from router_utils and skip the Codex sync on undecodable debug models output --------- Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com> |
||
|
Mateo Wang
|
65d912fc2e
|
feat(responses): support Codex client tool_search on Anthropic and other bridged providers (#48972)
* feat(responses): support Codex client tool_search on Anthropic and other bridged providers * test(router): cover get_search_tool_support provider defaults and model_info overrides * fix(responses): keep tools loaded by earlier Codex tool searches across previous_response_id * perf(responses): read stored turns once on client tool search continuations * fix(router): avoid OAuth refreshes when listing search capabilities * test(cli): expect disabled search on fallback Codex models |
||
|
tessellark
|
5bd47442fb
|
fix(cli): give lite codex aliases of a Codex-known model that model's catalog entry (#48894)
* fix(cli): give lite codex aliases of a Codex-known model that model's catalog entry lite codex built its Codex model catalog from /v1/models ids only, so a proxy alias of a model Codex knows (api-gpt-6-luna for openai/gpt-6-luna) got the fallback entry. That entry has no supports_search_tool or tool_mode, so Codex sent every MCP and app tool inline and each turn cost far more input tokens. lite now reads each listed model's upstream from the proxy's /v1/model/info (team_public_model_name when set, else model_name, first deployment wins) and passes it to the shared catalog builder, the same way the proxy's own Codex route does. If that request fails, the catalog is built as before. * fix(cli): give a lite codex alias an upstream only when all its deployments route to one OpenAI model * fix(cli): cap a lite codex alias's context window at its listed max_input_tokens and count only OpenAI-hosted deployments * fix(cli): keep Codex's tier, upgrade and reasoning defaults off lite codex aliases * fix(cli): keep Codex's own effort, verbosity and promo off lite codex aliases and skip credential-hosted deployments --------- Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com> |
||
|
Mateo Wang
|
26e53cb738
|
fix(cli): let the proxy decide when a classic lite login token has expired (#47712)
* test(cli): print-token and whoami let the proxy judge a classic token past the client window * fix(cli): let the proxy decide when a classic lite login token has expired * fix(cli): treat 403 and 404 as a rejected token and judge lite up logins the same way * fix(cli): keep lite up on the client window and bound the proxy check to 3 seconds * test(cli): prove the proxy check gives up on a silent proxy and pin stale fixtures Point the timeout test at a local socket that never answers, use fixed timestamps for stale CLI token fixtures, and correct the print-token docs for tokens younger than the client window. * fix(cli): expire a CLI token with no recorded proxy and stop treating 404 as rejected A token file without base_url cannot be checked, so it keeps the client window's verdict instead of printing forever. A 404 also comes from a proxy that does not serve /v1/models, so only 401 and 403 count as a rejected key, matching the CLI's existing key check. The timeout test now runs in process. * test(cli): judge the whoami proxy check on a fixed timestamp and name the client window for lite up |
||
|
rj-cairn
|
295d79fc7d
|
feat(cli): keep the ChatGPT login with lite codex --subscription (#48741)
* feat(cli): keep the ChatGPT login with lite codex --subscription `lite codex --subscription` points Codex at a `litellm_subscription` provider with requires_openai_auth on and no env_key, so Codex keeps its own ChatGPT login and the LiteLLM key is sent only in the x-litellm-api-key header from LITELLM_PROXY_API_KEY. OPENAI_API_KEY is left exactly as the user had it, and the pre-launch key check sends the key the same way `codex_subscription` is a new `lite config` key that turns the flag on by default, like `claude_subscription` does for `lite claude`. The Claude `--settings` overlay now applies to Claude Code only * fix(cli): leave the Codex Stop hook out of subscription launches and load tomli on Python 3.10 * fix(cli): leave OPENAI_BASE_URL as the user had it with lite codex --subscription * fix(cli): give lite codex --subscription its own Codex provider id * test(cli): type the Codex config override helpers * fix(cli): leave the Codex model list to Codex in subscription mode --------- Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com> |
||
|
moe-berri
|
34c10829b1
|
fix(ui): percent-encode request id in spend log detail URL (#48630)
* test: cover request ids with URL special characters in log detail fetches Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix(ui): percent-encode request id in spend log detail URL Co-authored-by: Charlie Freeman <243496084+cf3901646@users.noreply.github.com> Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> --------- Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> Co-authored-by: Charlie Freeman <243496084+cf3901646@users.noreply.github.com> |
||
|
Mateo Wang
|
464932b1d1
|
refactor(typing): clear 1181 basedpyright Any errors across 431 files (#47789)
* refactor(typing): narrow Any sources across providers, proxy and guardrails (wave 1) * refactor(typing): narrow Any sources across hubs, providers and proxy (wave 2) * refactor(typing): validate RunwayML task JSON and let dotprompt loaders accept exported prompts * fix: restore pass-through behavior on paths the typing pass tightened MCP client IP extraction trusts X-Forwarded-For only when use_x_forwarded_for is True, dotprompt skips a malformed JSON prompt entry instead of raising, RunwayML task payloads pass through unvalidated, and a malformed provider body raises the same AttributeError as before at the CompactifAI, Gemini, and mock transport parse sites * fix(runwayml): keep a malformed 2xx task body on the 500 path * fix(edenai,cli): keep a non-object body in the error and fail on a malformed migration report EdenAI's embedding and image generation parse dropped a non-object body from the error they raised, so the message no longer named what the provider sent. A helper now raises with the body in the message, and both paths have a regression test. `lite encryption migrate` exited 0 with no message when the proxy's report was not an object or its residual_legacy was not a number, where base crashed. It now fails with a message naming the value. The CompactifAI and Gemini image-edit non-object body tests accept TypeError or AttributeError, which both base and this branch raise, so they guard against a ValidationError instead of pinning one class. RunwayML's async parse gets the same non-object body test as the sync one. * fix: keep base's error classes, duck-typed checks and fail-closed guardrails the typing pass changed * test: annotate the new regression tests' locals with Final * test: annotate the remaining new test locals with Final |
||
|
Mateo Wang
|
8d6b8dab3d
|
fix(cli): send pi's session id from lite pi so sessions group and stick (#47700)
* fix(cli): send pi's session id from lite pi so sessions group and stick * docs(cli): drop the provider_block docstring lines the README already covers * docs(cli): note the session id format the proxy reads for lite pi |
||
|
Yassin Kortam
|
1525e4bce5
|
feat(cli): keep the claude.ai login with lite claude --subscription and save pkce in config (#47385)
`lite claude --subscription` sends the LiteLLM key in the x-litellm-api-key header through ANTHROPIC_CUSTOM_HEADERS and leaves ANTHROPIC_AUTH_TOKEN and ANTHROPIC_API_KEY unset, so Claude Code stays on its claude.ai login and its connectors keep working. The pre-launch key check sends the key the same way. `claude_subscription` and `pkce` are now `lite config` keys that default the new `--subscription/--no-subscription` flag on `lite claude` and the `--pkce/--no-pkce` flag on `lite login`. The flags still win for a single run. Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> |
||
|
Mateo Wang
|
68a8b12ab3
|
fix(proxy)!: end a chat completions stream with an error event when a guardrail blocks mid-stream (#46953)
* fix(proxy): end a chat completions stream with an error event when a guardrail blocks mid-stream A streaming guardrail that raised an HTTPException after the first chunk re-raised it inside the SSE generator, so the 200 body was cut off with no finish_reason, no error event and no [DONE]. Build the error frame through sse_error_payload instead, keeping the status, message and structured detail, and encode values JSON cannot represent as strings so the frame always goes out. * fix(proxy): keep early guardrail blocks on their HTTP status and end mid-stream errors with [DONE] A guardrail block before the first chunk is raised again, so its response stays the 400 event-stream body it is on main. A mid-stream error event now carries the failure hook's replacement error when one is returned, is followed by data: [DONE] on OpenAI-format streams, and leaves the Gemini route's raise unchanged. Error details JSON cannot encode are written as strings instead of breaking the event. * fix(proxy): keep a per-chunk guardrail block on the first chunk on its event-stream 400 * fix(proxy): treat a keepalive ping as a started stream so a later guardrail block ends in-band * fix(proxy): encode non-JSON guardrail details in early stream error frames An early guardrail block whose detail holds a value JSON cannot encode sent 400 headers and then cut the body. The error frame now writes such a value as its string form, the same as the mid-stream event. The keepalive test now runs on a virtual clock instead of real sleeps. * fix(proxy): show an in-band stream error event as an error in the CLI, Python client, and dashboard playgrounds ChatClient.completions_stream raises StreamError on a data frame carrying an error, so litellm-proxy chat prints the block reason and keeps the partial reply out of history. The prompt playground and the Interactions playground read the same frame through a shared streamErrorMessage helper. Adds the audit's integration cells for chat stream error events, and makes the endpoint tests' failure-hook mocks return None so they check the real error payload. * fix(proxy): print stream error text verbatim in the CLI and read split error events in the prompt playground The CLI printed the server's error message as Rich markup, so a closing tag like [/pii] in a guardrail message crashed lite chat. The prompt playground now buffers partial lines across reads, ChatClient raises only on a non-empty error, StreamError redacts keys like the other client exceptions, and StreamError is exported from litellm.proxy.client * fix(proxy): print CLI stream errors without Rich highlighting or emoji codes * test(proxy): cover the CLI's plain-text 400 error print * test(proxy): annotate the new client stream test locals as Final * fix(proxy): send the failure hook replacement on Responses streams and before the first chunk * fix(proxy): end a Responses stream with response.failed when it fails on a built-in exception * fix(proxy): keep a failure hook replacement before the first chunk in-band so bare stream routes still answer it A non-HTTP failure whose hook returns an HTTP error was raised before the first chunk, which create_response turned into the hook's status but which a bare StreamingResponse route such as /queue/chat/completions answered as a 200 with zero bytes. Only the original HTTP errors raise now. The replacement goes out as the stream's error event, which create_response still answers with the hook's status as a JSON error. Adds audit round 2's integration cells for failure hook replacements on chat and Responses streams, built-in exceptions on Responses streams, and the Python client. * test(proxy): cover a queue stream whose failure hook replaces a failure before the first chunk The audit cell posts to /queue/chat/completions with a callback that raises before the first chunk and a failure hook that answers 403. It checks for a 200 event stream whose only event is the replacement, then one [DONE], with the callback's own message absent |
||
|
Mateo Wang
|
355dea26e7
|
perf(cli): start lite without initializing the litellm SDK (#46803)
* perf(cli): start lite without initializing the litellm SDK lite and litellm-proxy now enter through a small litellm_cli package that imports the CLI against an uninitialized litellm package and runs the SDK __init__ in place only when something first uses it, so commands start in a fraction of a second instead of loading about 3,000 modules first. * perf(cli): import only the lite subcommand being run Each lite subcommand now loads its module when Click looks it up, so lite logout no longer imports the autoroute wizard, agents, or the other command groups. importlib.import_module and attribute reads on the SDK also initialize it on first use, and an already imported litellm is kept as is. * fix(cli): initialize the SDK on a star import and test command bodies in thin mode A from litellm import * after lite loaded the CLI used to see only the names the thin package had, so it now runs the full SDK init first. The thin-vs-full parity test now also runs logout and whoami, not just --help * fix(cli): keep the import hooks until the lazy SDK init finishes A thread that imported SDK code after the init restored the interpreter imports could hold that module's import lock while waiting for the init, and the init then waited on the same lock, so both threads hung. Other threads now wait for the init before they load any new SDK code * fix(cli): retry a failed lazy SDK init on the next use instead of leaving it half-run * test(cli): bind the patched credentials client and autoroute dir in an order-independent way |
||
|
Mateo Wang
|
95edbd3773
|
fix(cli): run group subcommands in the lite shell and default to the logged-in gateway (#46511)
* fix(cli): run group subcommands in the lite shell and default to the logged-in gateway Typing a command group alone in the lite shell printed 'Usage: lite lite', the next subcommand was unknown, quoted arguments were split on spaces, and a bare lite connected to localhost:4000 even after lite login to a gateway. The shell now names the group in usage text, keeps the group as the prompt's scope so its subcommands run next, accepts a leading 'lite', parses lines with shlex, and a bare lite falls back to the server the stored lite login credential was issued for before localhost:4000. * refactor(cli): walk the shell's group path without recursion * fix(cli): keep backslashes and compact JSON in the lite shell and pair the login server with its key * fix(cli): keep the login server to the bare lite shell and accept lite options and = quoting at its prompt * fix(cli): keep the shell's server for a lite options line and name the right login in the keychain notice * fix(cli): keep compact JSON with a trailing = intact in the shell and pin options lines to the shell's server over LITELLM_PROXY_URL |
||
|
Mateo Wang
|
0ebec123fd
|
chore(typing): clear 670 basedpyright Any errors by validating provider payloads at the boundary (#45388)
* chore(typing): validate provider payloads at the boundary to clear 464 basedpyright Any errors Replaces untyped reads of decoded JSON in 48 provider, guardrail and proxy modules with one pydantic validation at the point the payload enters the module, so every value read afterwards carries a real type. reportAny 6111 -> 5663 and reportExplicitAny 1416 -> 1400. No basedpyright, ruff-strict or type-discipline rule is higher repo-wide or in any single file, and no cast, ignore or suppression marker was added. * chore(typing): replace 47 Any annotations that every caller already satisfies with object Each annotation here is a parameter or return type whose value is only stored, compared, logged or forwarded, so object describes it exactly and no function body changes. A site was kept only if no basedpyright, ruff-strict or type-discipline rule rose in any file that references it. reportExplicitAny 1400 -> 1353 and reportAny 5663 -> 5655. * chore(typing): type upstream responses and loose annotations across providers, proxy and integrations Second pass of the Any reduction. Decoded upstream JSON is validated once at the provider boundary and read through typed views afterwards, annotations that every caller already satisfies move from Any to a concrete type, and the nine search transformations validate each result item on its own so error text no longer carries the item index. No cast, no type: ignore, no suppression marker and no budget file edit * ci(unit): run the brave, google_pse, nlp_cloud and opensandbox tests in the providers shard |
||
|
devin-ai-integration[bot]
|
0d17f954c0
|
feat(credentials): add display_name and make credential_name immutable on PATCH (#43148)
* feat(credentials): add credential_alias and make credential_name immutable on PATCH Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(ui): query credential rows by role to stay under the no-node-access budget Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(proxy): expect credential_alias in load_credential_list dump Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(ui): drive the credential SearchSelect by placeholder and option roles Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix(proxy): pass the decrypted CredentialItem to update_db_credential during master key rotation Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix(ui): trim credential_alias in CredentialModal so whitespace-only input clears the alias Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * feat(credentials): replace credential_alias with display_name and reject edits to config credentials credential_name stays the immutable reference key. display_name is a nullable, trimmed, max 255 character label set on POST or PATCH (omit keeps it, null clears it, blank is a 400). Reads return display_name plus an in-memory source tag (db or config), and PATCH or DELETE on a config-defined credential answers 400 instead of a misleading 404 * feat(ui): show credential display names everywhere and lock config credentials The credentials table shows the display name with the credential name beneath it, badges config credentials and disables their edit and delete actions. The edit modal adds an editable display name next to the read-only credential name and only sends it when it changed. Every credential picker and reference (add model, model info, models table, vector store form and info) shows the label while still submitting credential_name * feat(cli): set display names on credentials and show them in the list * refactor(credentials): keep the new credential types inside the lint gate budgets * refactor(cli): print credential command JSON through one helper * fix(credentials): treat an empty credential_name on PATCH as omitted and pin the 404 for vanished rows A blank credential_name never renamed anything, so PATCH accepts it again instead of answering 400. New tests pin a trimmed display_name on PATCH, the repository carrying display_name, and a 404 (not the config-owned 400) for a DB credential another worker already deleted. The hydration helper no longer copies display_name, since none of its callers read it, and the CLI update passes display_name straight through because the exactly-one check already makes it None when clearing. * fix(ui): keep a model's credential when its picker text is emptied and skip the admin-only list for other roles Clearing the search text in the model edit form's credential picker used to submit null, silently detaching the model's credential on save. None is now the only way to clear it. The models table fetches /credentials only for proxy admins, since everyone else got a 403 on each page load and falls back to the raw name anyway. Also fixes a type error in the re-use credential dialog and adds tests for the display name surviving a provider switch, the 255 character limit, the request payload, and the vector store None choice. * fix(client): percent-encode the credential name when updating its display name A name with ? or # was cut short in the URL, so the PATCH landed on a different credential or 404'd. * fix(credentials): answer 405 with Allow: GET for PATCH and DELETE on config credentials A config-defined credential exists (GET returns it) but is read-only through the API, which is what 405 Method Not Allowed means. 400 described a malformed request, and 404 would claim the credential does not exist. * fix(proxy): ignore a display_name set on config credential_list entries so non-string values cannot fail boot --------- Co-authored-by: yuneng <yuneng@berri.ai> Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> |
||
|
devin-ai-integration[bot]
|
c5c5154c4b
|
fix(cli): pin pi compat flags so lite pi stops sending store to Anthropic models (#40739)
* fix(cli): pin pi compat flags for gateway models Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix(cli): annotate pi compatibility field Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix(cli): clarify pi compatibility suppression Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix(cli): drop the stale mutable-ok suppression on the pi compat block --------- Co-authored-by: jesus <jesus@berri.ai> Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com> |
||
|
tin-berri
|
9a3f000c9b
|
fix(cli): show full-session auto-router cost comparison (#44959) | ||
|
Mateo Wang
|
d80f8c28ca
|
refactor(types): replace Any with proven types in 137 files (#44478)
* refactor(types): prove runtime types at harness, search, rag and client boundaries Replace Any with adapter-validated types in the litellm.agent() harness, the search provider transformations, RAG ingestion and query, the vector store pre-call hook and registry, the galileo and opik logging integrations and the proxy client CLI. Each boundary gets unit tests for well-formed and malformed payloads. * chore(typing): prove types at more provider boundaries and restore search transformations Second pass over a2a, embedding, rerank, image, audio and small provider modules. Search transformations go back to their previous form because validating their response bodies would change the proxy status for malformed upstream bodies from 400 to 500. * refactor(types): prove types at logging, files, rerank, image, audio and management boundaries Replace Any with validated or annotation-only types in 37 more files: logging integrations, token counters, provider files/rerank/image generation/audio transcription transformations, pass-through logging handlers and management endpoints. No proxy HTTP status or error type changes. * refactor(types): prove types at repository, spend, files and router boundaries Replace Any with repository table accessors, validated mappings and annotation-only types in 33 more files: Prisma repositories, the enterprise batch and responses cost checkers, budget reservation, files endpoints, management endpoints, the policy registry, the adaptive and complexity routers and the secret managers. No proxy HTTP status or error type changes. * test(types): run the aiohttp transformation test in-process and cover repository row conversion The aiohttp chat transformation test no longer starts a server. It feeds the transformation a response whose json() returns the body under test. The proxy unit shards now exercise stored model rows whose params are JSON strings and the object permission create and update paths. |
||
|
devin-ai-integration[bot]
|
39e31958f8
|
test(proxy): move auth, hooks, policy_engine and client tests into tests/unit/proxy (#43998)
* test(proxy): move auth, hooks, policy_engine and client tests into tests/unit/proxy Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(proxy): stub HIBP through respx by disabling the aiohttp transport Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(proxy): share the httpx transport fixture across proxy unit tests Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(proxy): restore proxy globals without a missing-value sentinel Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(proxy): package moved dirs and stub the login breach check at the HTTP boundary Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(proxy): isolate the mcp server manager per test Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> --------- Co-authored-by: yuneng <yuneng@berri.ai> Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> |