mirror of
https://github.com/BerriAI/litellm.git
synced 2026-08-28 05:25:59 +00:00
47 commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
ryan-crabbe-berri
|
78addb230b
|
fix(proxy): deny agent access when key and team grants resolve to nothing (#36221)
* fix(proxy): deny when agent grants resolve to nothing `get_allowed_agents` returned a plain list where the empty value meant both "this caller was never restricted" and "this caller's grants resolved to nothing". Downstream read either as allow-all, so a key restricted to one agent inside a team restricted to another reached every agent on the proxy, and an access group that resolved to no agents did the same. Replace it with `resolve_agent_access`, returning a tagged UnrestrictedAgentAccess | RestrictedAgentAccess. Only a caller with no grant anywhere is unrestricted; an empty restricted set denies. Access group lookup failures now propagate to the key/team resolvers so a DB error still fails open exactly as before, while a group that genuinely resolves to nothing denies. * style(proxy): drop redundant comments from the agent access match |
||
|
mateo-berri
|
ca7453bc69
|
chore(lint): recompute budget ceilings after merging base
The base branch ratcheted the same limits in
|
||
|
mateo-berri
|
28ff7f3f0b |
fix(guardrails): scan function-role results and dedupe returned tools
Under scan_only_tool_results, legacy OpenAI function-role messages now count as tool results, and duplicate names among guardrail-returned tools keep only the first occurrence. CustomGuardrail.structured_messages_cover_full_request lets CrowdStrike AIDR declare that its writeback already rebuilds the whole conversation, so handlers install it as-is instead of merging it into the full message list a second time and duplicating out-of-scope rows. Lint budget ceilings ratchet down to match the tree |
||
|
mateo-berri
|
28a277e99e | refactor(guardrails): drop dead tool extraction and an Any annotation, ratchet lint budgets | ||
|
mateo-berri
|
3728f3ea62 | Merge remote-tracking branch 'origin/litellm_internal_staging' into litellm_daily_any_cleanup_08_04_2026 | ||
|
mateo-berri
|
2bd6af1b64 | Merge remote-tracking branch 'origin/litellm_internal_staging' into litellm_daily_any_cleanup_08_04_2026 | ||
|
mateo-berri
|
18572fe86f |
Merge remote-tracking branch 'origin/litellm_internal_staging' into litellm_passthrough_live_credentials
# Conflicts: # basedpyright-code-budget.json # litellm/types/router.py # ruff-strict-budget.json # type-discipline-budget.json |
||
|
mateo-berri
|
f7bdc10b21 |
Merge remote-tracking branch 'origin/litellm_internal_staging' into litellm_passthrough_live_credentials
# Conflicts: # ruff-strict-budget.json # type-discipline-budget.json |
||
|
mateo-berri
|
6c76f5f9c6 |
chore(lint): clear grandfathered over-limit lint drift and ratchet budgets down
Every ruff-strict rule that sat above its budget limit (FURB188, RUF022, SIM118, UP007, UP032, UP037) is now at zero, LIT001 and LIT006 are back under their ceilings, and the freed headroom is ratcheted out of ruff-strict-budget.json, type-discipline-budget.json, and basedpyright-code-budget.json so the gates take the fast path again |
||
|
mateo-berri
|
d259070fdf | Merge remote-tracking branch 'origin/litellm_internal_staging' into litellm_daily_any_cleanup_08_04_2026 | ||
|
mateo-berri
|
0c50661307 | chore(lint): re-tighten reportPrivateUsage and reportDeprecated to post-merge counts | ||
|
mateo-berri
|
d4611a1253 | Merge origin/litellm_internal_staging | ||
|
mateo-berri
|
021b52527b | chore(lint): zero out seven more purely local basedpyright rules | ||
|
mateo-berri
|
1fefd80925 | fix(proxy): resolve pass-through credentials live from router deployments | ||
|
yucheng-berri
|
bcce83a17e
|
fix(guardrails): scan model output on the /openai/v1/responses alias (#35818)
The proxy serves POST /openai/v1/responses alongside /responses and /v1/responses, but only the latter two were in API_ROUTE_TO_CALL_TYPES. UnifiedLLMGuardrails.async_post_call_success_hook resolves the call type from request_route, so on the alias it resolved to None and returned the response unscanned; model output reached the client with post-call guardrails never running. The key and team tool allowlist was unenforced on the same alias for the same reason. Register the alias family in API_ROUTE_TO_CALL_TYPES and in LiteLLMRoutes.openai_routes, mirroring how the /openai/v1/realtime aliases are registered, and log a warning at the two points where the unified guardrail skips post-call scanning so a future unmapped route is visible instead of silent. The Responses block of API_ROUTE_TO_CALL_TYPES moves from list to tuple literals because the LIT002 budget rejects net-new mutable-collection construction; the map is read-only, so it is now typed as a Mapping of Sequence and the budgets ratchet down accordingly. |
||
|
Mateo Wang
|
05204795cf
|
Merge pull request #35828 from BerriAI/litellm_zero_local_basedpyright_headroom
chore(lint): zero out basedpyright headroom for purely local rules |
||
|
mateo-berri
|
98d4f9151c | chore(lint): zero out basedpyright headroom for purely local rules | ||
|
mateo-berri
|
258d154f18 | chore(lint): zero out reportGeneralTypeIssues headroom | ||
|
mateo-berri
|
42564e896f
|
chore(typing): replace Any seams with real types across responses, proxy, and provider adapters
Replace Any-typed payload dicts, record shapes, and provider request/response seams with TypedDicts, Protocols, and precise annotations in the ten litellm/ files carrying the highest combined basedpyright reportAny + reportExplicitAny counts. No behavior changes. Adds a regression test covering the managed-id list path so a prisma client missing the managed tables keeps returning a fail-closed empty page. |
||
|
mateo-berri
|
1ae5297ef7 | Merge remote-tracking branch 'origin/litellm_internal_staging' into litellm_daily_any_cleanup_08_03_2026 | ||
|
yucheng-berri
|
2d1f650e9a
|
fix(guardrails/rubrik): attribute blocked requests to the caller that made them (#35734)
The block event Rubrik receives sourced caller identity from model_call_details[metadata], where the enriched litellm metadata never lives; it sits under litellm_params. Every block therefore reported user_api_key_hash as an empty string, so a security block could not be traced to a key, user, or team. Read identity off the authenticated UserAPIKeyAuth the failure hook is already handed, via the same mapper the success path and the proxy spend logger use, so a block log and a success log describe their caller with an identical key set. |
||
|
mateo-berri
|
bd8b377dd7
|
chore(typing): clear basedpyright Any errors in budget reset, access groups, and cache settings
Replace Any seams in three proxy modules with real types so the values keep their shape through the call graph: - reset_budget_job: Protocols for the Prisma spend-linked tables, the reset batcher, and each cascade row shape, with the per-table counter/cache key lambdas promoted to typed module functions so the row type is inferred - access_group_endpoints: Protocols for the access group record, the team and key tables, and the transaction handle; record to response conversion now goes through model_validate on the record dict - cache_settings_endpoints: the opaque cache settings blobs are Mapping[str, object] / dict[str, object] instead of Any, keeping Any only on the two returns that feed the dynamic litellm.Cache kwargs bag Whole-tree basedpyright: reportAny 19435 -> 19306, reportExplicitAny 6518 -> 6487, total errors 148372 -> 148117, with no rule above its baseline and no untouched file changed. No behavior changes. |
||
|
mateo-berri
|
23d26d5e64 |
Merge remote-tracking branch 'origin/litellm_internal_staging' into litellm_lit4395_cursor_agent
# Conflicts: # litellm/completion_extras/litellm_responses_transformation/transformation.py # litellm/litellm_core_utils/llm_response_utils/convert_dict_to_response.py # litellm/litellm_core_utils/prompt_templates/common_utils.py # litellm/litellm_core_utils/streaming_chunk_builder_utils.py # litellm/llms/openai/chat/gpt_transformation.py # litellm/main.py # litellm/proxy/response_api_endpoints/endpoints.py # ruff-strict-budget.json # type-discipline-budget.json |
||
|
mateo-berri
|
075babd00f |
chore(lint): clear the new LIT001/LIT002 violations and ratchet the lint budgets
The type-discipline gate flagged 17 new mutable-collection annotations and 31 new mutable-collection constructions added by this branch. Replace raw dict literals with the OpenAI SDK's TypedDict call forms, annotate read-only params as Mapping/Sequence, precompute the custom tool call id set as a frozenset, and accumulate streamed arguments as tuples. The few places where a plain list/dict is a hard contract (pydantic response fields, fastapi route tags, parsed request bodies, in-place tool call patching) carry reasoned mutable-ok suppressions instead. Ratchet the ruff, type-discipline, and basedpyright budgets down by the violations this branch now fixes on net |
||
|
mateo-berri
|
add2a1ce3f |
chore(typing): clear 2.4k basedpyright errors across 15 Any hotspot files
Replace Any-typed seams with real types in the files carrying the highest reportAny/reportExplicitAny density. The dominant source was the repository layer: BaseRepository.table is declared Any, so every repository read poisoned its rows and every downstream call. Typed pass-through accessors under a _PrismaTableActions Protocol pay that crossing once per table, and TypedDicts and Protocols replace the remaining Any-typed request, row, and tool payloads across the team, key, SCIM, spend, MCP, guardrail, video, and websearch surfaces No casts, no type: ignore, no noqa, no new Any annotations, no behavior changes. Whole-tree basedpyright: reportAny 20,840 -> 19,397, reportExplicitAny 7,253 -> 6,518, all rules 151,424 -> 149,066, with no rule increased in any file. Budgets ratcheted: basedpyright -2,358, ruff-strict -300, type-discipline -68 |
||
|
mateo-berri
|
c0cab45350
|
chore(typing): replace Any kwargs unpacking with validated model parsing
Swap `Model(**payload)` for `Model.model_validate(payload)` at the seams where the payload comes back untyped, so basedpyright stops widening every target field to Any. None of the models involved override `__init__`, so validation goes through the same core validator either way. Also route UserRepository through its own typed helpers (find_many, update, find_by_id) instead of the raw Prisma table, drop the redundant `_to_model` override signature, and call generate_key_helper_fn with explicit arguments in the SSO callback rather than splatting an untyped dict. Whole-tree basedpyright: reportAny 21481 -> 20834, reportExplicitAny 7258 -> 7252, with every other rule unchanged or lower. |
||
|
mateo-berri
|
a97233067d |
Merge remote-tracking branch 'origin/litellm_internal_staging' into litellm_batch_output_single_pass
# Conflicts: # basedpyright-code-budget.json # ruff-strict-budget.json # type-discipline-budget.json |
||
|
mateo-berri
|
76cf3bf6ac
|
chore(typing): clear basedpyright Any errors in proxy auth, repositories, and openai transforms
Replace `Model(**untyped_dict)` construction with `Model.model_validate(...)` at the hot Any seams, and give the repository layer a real record type instead of `Any`. reportAny 22710 -> 21448, reportExplicitAny 7283 -> 7269, with every other rule at or below its baseline repo-wide. |
||
|
mateo-berri
|
0b09588685 |
refactor(batches): aggregate batch output cost, usage, and models in a single pass
Completed-batch cost tracking parsed the whole output file into a list of dicts, pretty-printed it into debug strings even with debug logging off, and walked the list three times (cost, usage, models), so a large batch output could pin a worker's memory. The output is now folded line by line into small per-line stats records via _aggregate_batch_cost_usage_models, the eager json.dumps debug calls are gone, and the raw-vertex path computes cost and usage in one call instead of two. _get_batch_output_file_content_as_dictionary becomes _fetch_batch_output_file_content (returns bytes); the superseded three-pass helpers are deleted and their tests migrated |
||
|
mateo-berri
|
a895249923 | refactor(bedrock): remove the dead BedrockLLM invoke code path | ||
|
mateo-berri
|
44e091aedb
|
chore(typing): clear basedpyright Any errors in proxy management endpoints
Convert pydantic table-model construction from Cls(**row.model_dump()) kwargs-unpacking to Cls.model_validate(...) across the management endpoint hotspot files (team, key, internal user, scim, model management, spend tracking, auth checks, proxy_server). Unpacking an untyped dict reports one Any-typed argument per matched model field, so each converted site clears 10-35 diagnostics while running the exact same pydantic validation. Conversions were limited to models verified to use pydantic's default __init__; UserAPIKeyAuth and LiteLLM_VerificationTokenView keep their custom kwargs-rewriting __init__ and are untouched. Two locally-verified helper params move from Any to object. Whole-tree basedpyright, measured against the branch point in the same environment: reportAny 24,431 -> 22,741 (-1,690), reportArgumentType 2,189 -> 2,136 (-53), reportUnknownArgumentType 34,370 -> 34,067 (-303), reportExplicitAny 7,285 -> 7,283 (-2); total 154,882 -> 152,834 (-2,048) with no rule increasing anywhere and no per-file increases. No casts, no suppressions, no behavior changes. Budgets ratcheted: basedpyright -2,048 across 4 rules, ruff ANN401 -2. |
||
|
mateo-berri
|
fbfb63c948 |
chore(typing): clear 2.7k basedpyright Any errors across 15 hotspot files
Replace Any-typed seams with real types in the files carrying the highest reportAny/reportExplicitAny density: typed Prisma read helpers in the MCP db layer and verification token repository, TypedDicts for OAuth credential payloads and aggregated spend rows, a DailySpendRecord protocol for the daily activity endpoints, and concrete request/response types in the volcengine, openai evals, azure batches, azure_ai count_tokens, and ocr transformation modules. Modernize touched annotations to PEP 604/585 forms. No casts, no type: ignore, no noqa, no new Any annotations, no behavior changes. Whole-tree basedpyright: reportAny 27,005 -> 24,427, reportExplicitAny 7,439 -> 7,280, no rule increased anywhere. Budgets ratcheted: basedpyright -2,869, ruff-strict -1,505, type-discipline -167. |
||
|
devin-ai-integration[bot]
|
c4f19c3e4c
|
feat(messages): route Azure Anthropic /messages through Rust behind rust:true (#33616)
* feat(messages): route Azure Anthropic /messages through Rust behind rust:true
Adds an opt-in Rust path for non-streaming Azure Anthropic Messages. A
deployment sets rust: true in litellm_params to route litellm.messages()
and the proxy /v1/messages endpoint through the native Rust bridge; a
missing flag or rust: false keeps the existing Python path, and non-Azure
providers, streaming, an unavailable bridge, or a None result all fall
back to Python. Rust-backed responses carry an x-litellm-rust: true
response header so callers can see which path served the request.
Co-Authored-By: Ishaan Jaffer <155045088+ishaan-berri@users.noreply.github.com>
* test(docs): exclude LITELLM_USE_RUST_MESSAGES rollout flag from env-doc check
Mirrors the existing LITELLM_USE_RUST_OCR entry; the flag is an internal
rollout toggle that is intentionally not in the public environment settings
docs yet.
Co-Authored-By: Ishaan Jaffer <155045088+ishaan-berri@users.noreply.github.com>
* fix(rust_bridge): isolate OCR enable flag and drop dead messages global toggle
use_litellm_rust only mutates the OCR enabled flag when configuring OCR (or
called with no bridge kwargs, preserving the legacy contract), so configuring
only the messages bridge no longer flips OCR state.
Remove the vestigial global enabled/env state from the messages bridge. Routing
is controlled per deployment by rust:true in the shared handler gate, so the
messages module never consulted the global toggle; drop it rather than leave a
no-op switch.
Co-Authored-By: Ishaan Jaffer <155045088+ishaan-berri@users.noreply.github.com>
* refactor(rust/messages): split Anthropic config into its own provider file and type the request/response contract
Co-Authored-By: Ishaan Jaffer <155045088+ishaan-berri@users.noreply.github.com>
* feat(messages): route eligible Azure Anthropic streaming through Rust via buffered fake-stream
Co-Authored-By: Ishaan Jaffer <155045088+ishaan-berri@users.noreply.github.com>
* fix(messages): fold system-role messages for Azure Anthropic and fall back to Python on Rust bridge errors
Co-Authored-By: Ishaan Jaffer <155045088+ishaan-berri@users.noreply.github.com>
* fix(rust_bridge): use Python::attach for amessages after pyo3 bump
Co-Authored-By: Ishaan Jaffer <155045088+ishaan-berri@users.noreply.github.com>
* test(proxy): mock get_configured_token_limits in model_info tests
Co-Authored-By: Ishaan Jaffer <155045088+ishaan-berri@users.noreply.github.com>
* ci: run rust_bridge unit tests in misc shard
Co-Authored-By: Ishaan Jaffer <155045088+ishaan-berri@users.noreply.github.com>
* Revert "ci: run rust_bridge unit tests in misc shard"
This reverts commit
|
||
|
mateo-berri
|
81e639398a | fix(fallback-generalizations): let exact cost-map entries beat capability rules across lookup-candidate ladders | ||
|
mateo-berri
|
1329036ca1 | fix(fallback-generalizations): tolerate legacy remote rule schema and keep register_model cache-pricing inheritance | ||
|
mateo-berri
|
77885779ca | refactor(fallback-generalizations): split rules into routing and provider-neutral capability kinds | ||
|
Mateo Wang
|
9076c33347
|
fix(batches): price anthropic passthrough message batches correctly in batch cost job (#32307)
* fix(batches): price anthropic passthrough message batches correctly in batch cost job
Anthropic message batches created via the /anthropic passthrough were never
cost tracked. The CheckBatchCost job fetched batch results from the Files API
(POST /v1/files/msgbatch_.../content), which Anthropic rejects with "File id
must have file_ prefix"; the error response was silently wrapped as file
content, parsed as zero successful rows, logged as a $0 aretrieve_batch spend
row, and the job was marked batch_processed=true so the $0 was permanent.
Route msgbatch_ file ids to GET /v1/messages/batches/{id}/results in the
anthropic files transformation, raise on HTTP error status in
retrieve_file_content instead of returning the error body as content, parse
Anthropic's results JSONL shape (result.type == "succeeded",
result.message.usage with cache creation/read tokens) in batch_utils, price
cache creation tokens at cache_creation_input_token_cost in the batch cost
fallback (50% batch discount preserved for base input, cache reads, cache
writes, and output), and leave the managed object row unprocessed when cost
tracking fails so a later poll retries instead of permanently recording $0.
* fix(batches): carry cache token details into aggregated anthropic batch usage
|
||
|
Mateo Wang
|
5f864c83ce
|
chore(lint): zero out crash-class pyright rules and ban new type: ignore comments (#32152)
* fix: zero out crash-class basedpyright rules across litellm/ * feat(lint): add LIT009 banning inert type: ignore comments * docs: require bracketed rule and reason on every suppression * chore(lint): ratchet budgets down and zero crash-class pyright limits * fix: narrow auto router routelayer through a local before calling * test: add regression tests for crash-class fixes * fix: drop dead AZURE_AD_TOKEN lookups and word-bound the type-ignore regex |
||
|
ryan-crabbe-berri
|
27069bd74f
|
feat(ui): shadcn migration foundation: Tailwind v4, shadcn init, antd cascade fix (#31995)
* feat(ui): shadcn migration foundation: Tailwind v4, shadcn init, antd cascade fix
Upgrade the dashboard from Tailwind v3 to v4 with CSS-first config: the
official upgrade codemod renamed utilities across 151 files, and
tailwind.config.js (plus the dead tailwind.config.ts) is replaced by
@theme tokens, @source globs, and @plugin directives in globals.css. The
Tremor safelist becomes @source inline patterns and the legacy tremor
theme tokens carry over verbatim. ui_colors.json was build-time only and
fed the dying Tremor palette, so its brand values are inlined and the
file removed; runtime theming replaces that path next.
shadcn is initialized with a hand-authored components.json (rsc,
cssVariables, baseColor gray) pointing utils at the existing
lib/cva.config.ts, which now exports cn (cva beta cx + twMerge) instead
of adding class-variance-authority as a second variant library. The two
ad-hoc cn helpers fold into it. Button lands as the canary primitive,
adapted to cva beta and React 18 forwardRef, with tests covering the
variant, twMerge, asChild, and ref seams. --radius is 0.5rem so the
shadcn radius scale reproduces Tailwind defaults and legacy rounded-*
classes render unchanged.
antd v5 emits unlayered CSS-in-JS that would beat every layered v4
utility, so AntdGlobalProvider now wraps the app in StyleProvider layer
and ConfigProvider cssVar, and globals.css declares
@layer theme, base, antd, components, utilities. antd wins over
preflight but yields to utilities, which is what lets migrated shadcn
pages coexist with legacy antd pages. Preflight stays global with the
three v3 behaviors pinned (default border color, button cursor,
placeholder color).
* fix(ui): restore tremor opacity tints removed by tailwind v4
Tailwind v4 removed the *-opacity-* utilities, but the precompiled
@tremor/react dist still composes them with shade-500 palette classes
(bg-opacity-10 over bg-<color>-500 etc.), so Badge, BadgeDelta, Callout,
light Icon and Button, BarList, and ProgressBar lost their tints and
rendered solid 500-shade fills. Adversarial review caught it; the
original smoke pages only exercised antd Tags.
tremor-v3-compat.css restores exactly the pairs tremor emits: for each
of the 22 safelisted colors, bg-opacity-{10,20,40}, hover/group-hover
bg-opacity-{20,30}, and ring-opacity-{20,40} against the -500 shade,
via color-mix into the utilities layer. Tremor's colorPalette maps both
background and iconRing to 500, so the -500 pairing covers every
composition in the dist; dark: variants are inert until dark mode ships.
The shim dies with @tremor/react at the end of the migration.
The upgrade codemod also missed two hand-rolled modal scrims using
bg-black bg-opacity-{30,50} (solid black under v4); now bg-black/30 and
bg-black/50. Removed the docker/build_admin_ui.sh copy of
enterprise_colors.json into the deleted ui_colors.json; that build-time
rebrand path is retired and its runtime replacement lands with the
theming phase.
* fix(ui): pair ring-opacity-40 with shade 300 in tremor compat shim
Tremor's colorPalette maps ring to shade 300, and the only consumer of
ring-opacity-40 (Icon variant outlined) composes it with that shade,
so the shade-500 rows were dead and outlined icon rings would render
at full opacity. Latent today (no dashboard usage of the outlined
variant); caught by adversarial review. ring-opacity-20 stays at 500
(iconRing), matching Badge and BadgeDelta.
|
||
|
Mateo Wang
|
e141596204
|
refactor(lint): collapse type/lint budgets to a single per-rule limit (#31883)
* chore(lint): raise basedpyright per-rule slack to 50% of baseline The per-rule ceilings in basedpyright-code-budget.json sat at roughly 10% slack over baseline, which several in-flight PRs are already bumping into. Raise the slack on every rule to at least 50% of its baseline so there is ample headroom for a long while, while never lowering any rule that already had more generous slack (e.g. reportReturnType stays at 100). Co-authored-by: Mateo Wang <mateo-berri@users.noreply.github.com> * refactor(lint): collapse type/lint budgets to a single per-rule limit The three non-frontend budget files (ruff-strict, type-discipline, basedpyright-code) tracked a per-rule baseline and slack whose sum was the ceiling. Nothing consumed the split beyond that sum, so this replaces both keys with a single limit equal to the old baseline + slack; the original baselines live in git history if anyone needs them. The gate scripts and the ratchet guard now read limit directly. lint-budget-update no longer re-captures raw counts; it ratchets each rule's limit down by the number of violations this branch cleared since its branch point (the merge-base), so the granted headroom shrinks by exactly what was fixed and a limit never rises. The ratchet guard reads either schema so it still compares correctly across the migration boundary. Co-authored-by: Mateo Wang <mateo-berri@users.noreply.github.com> * chore(lint): surface staged-vs-working parity for pre-commit and budget-update make pre-commit selects which checks to run from the staged index but runs the linters over the working tree, so unstaged edits to tracked files and untracked files skew a green/red away from what a commit of only the staged changes would produce. There is no safe in-place way to lint the index, so the script now warns when unstaged or untracked changes are present, and CLAUDE.md documents that you must stage everything first for both make pre-commit and make lint-budget-update to predict CI correctly. Co-authored-by: Mateo Wang <mateo-berri@users.noreply.github.com> * docs(lint): list type-discipline budget in lint-budget-update instruction --------- Co-authored-by: Cursor Agent <cursoragent@cursor.com> Co-authored-by: Mateo Wang <mateo-berri@users.noreply.github.com> |
||
|
Sameer Kankute
|
687a62e561
|
fix(cli): mint per-session agent credential on lite login (#31072)
* fix(cli): mint per-session agent credential on lite login
The `lite login` command was producing a shared UI session token that broke agent use in three ways: a $0.25 budget cap (from max_ui_session_budget) that killed agent sessions in minutes, a fixed identity "cli-jwt-token" shared across every user preventing per-session spend attribution, and auth gated behind EXPERIMENTAL_UI_LOGIN so the token was rejected on default deployments.
This fixes all three. Each login now generates a unique cli-session-{uuid} token with no per-key budget cap (enforced via shared team/user counters instead), and the decrypt path activates for any non-sk- token without requiring EXPERIMENTAL_UI_LOGIN.
* fix(cli): address review feedback on EXPERIMENTAL_UI_LOGIN gate and e2e test
Restore EXPERIMENTAL_UI_LOGIN=false as an explicit opt-out: operators who set it to false keep the old boundary; unset (new default) and true both attempt NaCl decryption, which fails closed for non-blob tokens.
In the e2e test: replace the silent Redis fallback with pytest.skip so a missing Redis instance is explicit rather than silently degrading to a directly-minted token. Write the seeded flow back as JSON (proxy reads it via json.loads on cache fetch) instead of Python repr, and build the updated flow immutably.
* fix(key-management): cap CLI session token delegation budget to team ceiling
A CLI session token intentionally carries max_budget=None to avoid a per-session LLM spend cap. The key-generation delegation check (GHSA-q775-qw9r-2r4g) previously skipped non-admin callers with max_budget=None, treating them as having unlimited delegation authority. This allowed any internal user with a lite login session to mint virtual keys with arbitrary budgets.
Adds is_session_token=True to UserAPIKeyAuth for CLI session tokens and uses the caller's team budget as the delegation ceiling in that case, so the effective limit is min(requested_budget, team.max_budget) rather than unbounded.
* chore: regenerate dashboard OpenAPI types
The is_session_token field added to UserAPIKeyAuth cascades to the
dashboard schema. Regenerate types from the updated OpenAPI spec.
* fix(key-management): block personal key budget delegation from CLI session tokens
When team_table is None (personal key, no team_id in request), the personal key
has no team-budget enforcement at request time. A session token therefore cannot
delegate any explicit max_budget for a personal key -- that would open a budget
bypass path. Block the request with a clear 400 directing the caller to use a
team_id instead.
* test(auth): add unit coverage for non-admin CLI session token production path
* fix(type-check): use model_validate in _return_user_api_key_auth_obj to fix reportArgumentType gate
UserAPIKeyAuth(**user_api_key_kwargs) spread triggers a basedpyright
reportArgumentType error for each named field in UserAPIKeyAuth because
the dict's inferred value type (str | Span | LitellmUserRoles | Unknown)
is not assignable to each field's specific type. Adding is_session_token:
bool introduced +2 more such errors, breaching the gate cap.
model_validate accepts an untyped dict without per-field argument checking,
which eliminates the +2 new errors and also ratchets down the pre-existing
333 errors at those call sites. basedpyright-code-budget.json is updated
to reflect the new lower baseline (1814, down from 1934).
* fix(type-check): ratchet down reportArgumentType baseline only
The previous lint-budget-update captured all baselines from the local
environment, raising many ceilings vs the merge-base and failing the
non-gating budget_ratchet_check. Restore staging's values for every
rule and only lower reportArgumentType (1934 -> 1814) to reflect the
reduction from switching to model_validate in _return_user_api_key_auth_obj.
* fix(auth): set max_budget on CLI session token to enforce max_ui_session_budget
CLI session tokens were missing max_budget, so _virtual_key_max_budget_check
had no per-session ceiling to enforce. Operators relying on max_ui_session_budget
could be bypassed for the full token lifetime. Mirrors the existing UI token path.
* revert(auth): remove max_ui_session_budget from CLI session token
max_ui_session_budget defaults to $0.25 and is sized for the UI chat
pane (10-min sessions). CLI sessions are 24-hour tokens for real work;
capping them at that ceiling would throttle users under their actual
user/team budget. Budget enforcement for CLI sessions is via the shared
user and team counters as originally intended.
* fix(auth): cap CLI session at max_ui_session_budget only when user and team have no budget
When neither the user nor their team has a budget configured, CLI sessions
were fully uncapped. The poll endpoint now looks up the real user and team
objects from DB; if both have no max_budget, it passes litellm.max_ui_session_budget
as the token's per-key ceiling. Users or teams that already have a budget
configured are unaffected and continue to rely on the shared counters.
* fix(auth): fix black formatting and update test mock for cli_poll_key budget lookup
The get_user_object and get_team_object async calls in cli_poll_key were
not mocked in the existing test, causing MagicMock await errors. Patch
both functions at the auth_checks module level. Also apply black formatting
to ui_sso.py which CI rejected.
* fix(auth): skip fallback budget cap when team lookup fails for cli session token
* test(auth): pin cli session budget cap to user/team budget presence
The session_max_budget fallback in cli_poll_key only applied
max_ui_session_budget when neither the user nor the resolved team had a
budget. The existing coverage exercised only the team-lookup-failure
branch. Add two regression tests: a user with a configured budget must
not receive the fallback cap, and a session with no user and no team
budget must fall back to max_ui_session_budget. Mutating either guard
out of the branch now fails these tests.
* fix: remove CLI poll session budget cap
* revert(auth): restore CLI session fallback budget cap
Bugbot autofix (
|
||
|
tin-berri
|
bbef1b84ab
|
feat(mcp): graft v2 resolver onto _create_mcp_client (none + api_key static family) (#31058)
* feat(mcp): add v1 bridge + none/api_key resolver arms (unwired) PR4a of the MCP v2 outbound-credential migration, stacked on the resolver skeleton. Builds the bridge for the first live modes without wiring it onto the request path: - resolver.py: the none arm (NoOpAuth) and the api_key shared-key arm (StaticHeaderAuth from the config); the BYOK source and the other five arms stay not_implemented. - adapter.py: the v1 <-> v2 edge (to_subject, to_server_spec, raise_public, should_defer). to_server_spec maps only none + the static-header family and returns None to defer every other mode to v1. Imports v1, kept out of the package __init__ so the resolver core stays v1-free. - MCPClient gains an optional resolved_auth that feeds the factory's auth= slot, taking precedence over the SigV4 aws_auth; default None keeps current behavior. Nothing calls these from _create_mcp_client yet, so production behavior is unchanged; the graft lands in PR4b. Unit tests cover the two arms, the full mapping table, and the auth plumbing. * feat(mcp): graft v2 resolver onto _create_mcp_client for migrated modes Wire the none + api_key static-family resolver arms from PR4a onto v1's live request path. In _create_mcp_client's HTTP/SSE branch, to_server_spec decides per mode: a migrated mode resolves through the injected UpstreamCredentialProvider and feeds the resulting httpx.Auth into the new resolved_auth slot; every other mode returns None and falls through to the unchanged v1 construction. resolve_mcp_auth now runs only when the mode defers, so a migrated server skips the v1 token-exchange / M2M I/O. stdio is untouched: auth_type/auth_value never reach the upstream on the stdio path (_get_auth_headers is HTTP/SSE only), so there is nothing to graft there. No v1 code is deleted yet; resolve_mcp_auth's static return still backs stdio and the not-yet-migrated modes until later PRs retire it. * test(mcp): cover the v2-resolver graft in _create_mcp_client Regression tests for the PR4 graft. Migrated HTTP modes resolve through the provider into resolved_auth: none -> NoOpAuth, and the static api_key family emits the right header per scheme (X-API-Key, Bearer, token, raw authorization, base64 basic). Deferred modes (oauth2) and a missing static token fall back to v1's auth_value. A stdio server with a migrated auth_type still defers to v1, since httpx.Auth never reaches the subprocess. A resolver Error is mapped to the public HTTP contract (401) via an injected provider, exercising the DI seam. * fix(mcp): defer to v1 when an inbound credential would be overridden The graft attaches the resolved static credential as an httpx.Auth, whose auth flow writes its header after extra_headers. That silently overrode an inbound Authorization: a per-request mcp_auth_header override, or a header supplied via a guardrail hook / static_headers / forwarded caller header. v1 lets those win, so the graft had inverted the credential precedence for the migrated static modes. Mirror the v2 egress credential-isolation invariant: defer the request to v1 when mcp_auth_header is set, or when the header the resolved credential would write is already present in extra_headers. none writes no header, so it never defers. * test(mcp): cover the credential-isolation defer guard Regression tests for the precedence fix. A per-request mcp_auth_header override and an Authorization already present in extra_headers (guardrail hook like the JWT signer, static_headers, or a forwarded caller header) both defer a migrated static server to v1 so the inbound credential wins; none stays on v2 and does not clobber an inbound Authorization since NoOpAuth writes nothing. The deferred cases assert resolved_auth is None, which fails if the guard is removed. * refactor(mcp): resolve inbound-header conflict on v2 instead of deferring For an Authorization already supplied via extra_headers (a guardrail hook such as the JWT signer, static_headers, or a forwarded caller header), keep the request on the v2 path and skip resolved_auth rather than deferring to v1. The inbound header still wins since nothing overwrites it, but hooks no longer pin a v1 fallback, which is what lets resolve_mcp_auth be retired once the remaining modes migrate. The mcp_auth_header per-request override still defers to v1, since that value becomes the upstream credential rather than sitting in extra_headers; that defer falls away once the per-user modes stop writing mcp_auth_header. * fix(mcp): clear UP037 lint gate and fix allowed-servers test under the graft adapter.py uses `from __future__ import annotations`, so the quoted "UserAPIKeyAuth" / "MCPServer" annotations in to_subject/to_server_spec/_shared_key_spec were unnecessary and pushed UP037 over the strict-rule budget; drop the quotes. test_list_tools_only_returns_allowed_servers passed a MagicMock as user_api_key_auth. The graft now builds a Subject from the principal, and the MagicMock's non-string org_id/user_id fail Subject validation, so the listing came back empty. Use a real UserAPIKeyAuth instead (MagicMock for an injected dependency was the anti-pattern here). * test(mcp): assert config token via resolved_auth, not the headers dict test_mcp_server_config_auth_value_header_used inspected _get_auth_headers(), but the graft now carries the static credential on the client's httpx.Auth (resolved_auth) and writes the header at send time, so that dict is empty. Assert the header the StaticHeaderAuth emits onto the request instead. Both config keys (authentication_token, auth_value) stay covered. * chore(typecheck): set reportMatchNotExhaustive slack to 0 The previous slack of 3 put the ceiling at baseline + slack = 4, so a newly non-exhaustive match (for instance dropping an Error arm off a Result match) could land without tripping the gate. Setting slack to 0 pins the ceiling at the current baseline of 1, so any added non-exhaustive match now fails CI while the one pre-existing violation in router.py stays within budget |
||
|
Mateo Wang
|
f26dbb60be
|
ci: make the basedpyright budget gate delta-vs-base (#31106)
Some checks are pending
GitHub Actions Security Analysis / zizmor (push) Waiting to run
* ci: re-run absolute basedpyright budget gate on push to long-lived branches The basedpyright budget gate counts codebase-wide errors per rule against a committed ceiling, but it only ran on pull_request against each PR's own head. Two PRs that each pass in isolation can together push a per-rule count over its ceiling once both merge, and nothing re-evaluated the budget on the merge commit, so the breach only surfaced on the next PR that happened to be checked out after the count crossed the line. Add a push trigger on the long-lived branches and a post-merge-budget job that re-runs the absolute gate on the merged tree, catching the accumulation on the merge commit itself. The existing pull_request jobs are guarded so their delta-vs-base gates don't misfire on push, where no PR base SHA exists. * ci: shallow-fetch the post-merge-budget checkout The post-merge-budget job only runs basedpyright over the working tree and the committed budget file; it never inspects git history, unlike the lint job whose delta-vs-base gates need full history. Drop its checkout from fetch-depth: 0 to fetch-depth: 1 to avoid cloning the whole repo history. * ci: scope post-merge-budget push trigger to long-lived branches On a push event the branches filter matches the branch being pushed to, not the PR target. The litellm_** glob, correct for the pull_request filter where it matches the target branch, therefore fired the post-merge-budget basedpyright job on every short-lived feature branch carrying the litellm_ prefix (litellm_dev_*, litellm_add_*, and so on), duplicating the PR lint job and burning ~10 minutes of CI per push. Restrict the push trigger to the long-lived branches PRs actually merge into (main, litellm_internal_staging, litellm_oss_branch), where budget accumulation happens. The pull_request filter keeps litellm_** so PRs targeting any long-lived branch are still linted. * ci: make the basedpyright budget gate delta-vs-base The basedpyright gate counted absolute codebase-wide errors per rule against a committed ceiling and ran only on each PR's own head. Two PRs that each pass in isolation could together push a rule past its ceiling once both merged, and because the gate had no comparison against the base, the next unrelated PR branched off the now-over-ceiling tree inherited a red it did nothing to cause. Give it the same shape as the ruff strict gate: a rule fails only when its total is both over the ceiling and higher than the count on the merge-base it merges into. Drift already in the base is never blamed on a bystander, while any change that actually grows a rule past the cap still fails. Head counts come from the existing stdin pipe; the base count is a second basedpyright pass over a detached worktree at the merge-base, reusing the head environment so import resolution matches and no second uv sync is needed. This obsoletes the push-triggered post-merge-budget job (and its event guards), which only detected accumulation after the fact; the delta check blocks it on the PR instead. Slack for reportReturnType and reportUnnecessaryComparison is raised to give real headroom under the cap. * refactor(ci): give the base ref its own name in type_check_gate cmd_check cmd_check took a parameter named base that held a git ref string, then rebound the same name to the dict of base-tree error counts returned by base_counts. Rename the parameter to base_ref so the ref and the counts each keep a single name and type, matching the no-reassignment style used elsewhere; behavior is unchanged. --------- Co-authored-by: Claude <noreply@anthropic.com> |
||
|
Mateo Wang
|
1bd603d1ac
|
chore(typing): add boto3/botocore stubs so basedpyright resolves the AWS SDK (#30815)
Some checks are pending
GitHub Actions Security Analysis / zizmor (push) Waiting to run
|
||
|
Sameer Kankute
|
1ccc1e5b23
|
chore: litellm oss staging160626 (#30527)
* feat(ui): gate "Default Credentials" hint on /ui/login behind env flag (#30234) Adds LITELLM_HIDE_DEFAULT_CREDENTIALS_HINT (and an equivalent general_settings.hide_default_credentials_hint) that suppresses the "By default, Username is admin and Password is your set LiteLLM Proxy MASTER_KEY" info card rendered on /ui/login and /fallback/login. Motivation: in production deployments operators set UI_USERNAME / UI_PASSWORD (or SSO), and the hardcoded hint becomes factually incorrect and is flagged by security scanners (Tenable WAS plugin 114625) as information disclosure. There is currently no way to suppress it without forking the dashboard. Behaviour: - Default is unchanged (hint shown), so existing deployments are unaffected. - New field hide_default_credentials_hint on the well-known UI config endpoint, populated from the env var or general_settings. - LoginPage.tsx conditionally renders the Alert based on the flag. Refs: BerriAI/litellm#30232 * fix(router): clean pattern_router state on upsert/delete (#29601) * fix(router): clean pattern_router state on upsert/delete PatternMatchRouter.add_pattern was append-only, and neither Router.upsert_deployment nor Router.delete_deployment removed the existing entry. Rotated-out api_keys stayed in the routing rotation for wildcard deployments (model_name with `*`) until proxy restart, silently defeating key rotation as an admin operation. The same leak applied to provider_default_deployment_ids and per-team pattern routers, and the patterns list grew unboundedly on every edit * test(router): direct unit tests for _remove_deployment_from_wildcard_state router_code_coverage.py greps test files for AST Call nodes and flagged the helper as untested because the existing coverage only exercised it transitively through upsert/delete. Adds two direct tests that pin the helper's contract (cleans across global pattern router, per-team routers with empty-router pop, and provider_default_deployment_ids; noop on falsy model_id) * fix(router): address Greptile review on pattern_router cleanup Widen PatternMatchRouter.remove_deployment annotation to Optional[str]; the implementation already handles None via the falsy guard and the unit test exercises it directly. Move _remove_deployment_from_wildcard_state up one level in upsert_deployment so it runs whenever the prior deployment is on the router, not only when the model_id is present in the fast-mapping index. The scenario is currently unreachable (get_deployment shares the same index), but the cleanup is idempotent so this is defensive against any future divergence between those code paths. * fix(router): widen _remove_deployment_from_wildcard_state to Optional[str] Moving the call out of the inner `deployment_id in deployment_fast_mapping` block in the previous commit lost mypy's narrowing of `deployment_id` from Optional[str] to str, tripping the lint CI. The helper already handles None via its falsy guard, so widening the annotation matches the actual contract. * fix(router): make delete_deployment wildcard cleanup symmetric with upsert After the previous commit moved _remove_deployment_from_wildcard_state out of the inner index-map guard in upsert_deployment, delete_deployment was still calling it only inside `if deployment_idx is not None`. Greptile flagged the asymmetry: under a desynced index_map, delete would silently leave the stale wildcard credential in pattern_router. Moves the cleanup call to the top of the try block, mirroring the upsert path. Cleanup is idempotent so the change is a no-op on the happy path. Adds a regression test that simulates the desync by removing the entry from model_id_to_deployment_index_map and asserts delete still clears pattern_router. * fix(pricing): add 1h cache-write cost for Anthropic Sonnet 4.5/4.6 (#30474) The native anthropic claude-sonnet-4-5/4-6 price-map entries were missing cache_creation_input_token_cost_above_1hr (and the >200K long-context sub-tier for 4.5), so 1-hour-TTL cache writes were costed at the 5-minute rate. Adds 6e-06 regular (and 1.2e-05 long-context) = 2x base input, matching the vertex_ai/azure_ai/bedrock siblings and the older claude-sonnet-4-20250514 entry. Adds a regression test. * fix(proxy): cancel upstream gemini request and release httpx connection on client disconnect (#30075) * fix(proxy): cancel upstream gemini request and release httpx connection on client disconnect - add _check_request_disconnection to common_request_processing; wrap llm_call as asyncio.Task so it can be cancelled; catch CancelledError and raise HTTPException(499) when client disconnects before LLM responds (non-streaming path) - pass raw httpx.Response into ModelResponseIterator in make_call/make_sync_call so the iterator holds a reference to the underlying connection - implement ModelResponseIterator.aclose() and .close(): close the line iterator then explicitly call response.aclose()/response.close() to release the httpx connection when the client drops mid-stream; errors are debug-logged, not raised - add tests for _check_request_disconnection (cancels task, graceful on exception, does not cancel when client stays connected) and base_process_llm_request 499 behavior; add TestModelResponseIteratorCleanup verifying aclose/close propagation through CustomStreamWrapper * fix(proxy): record 499 on streaming disconnect and cancel orphaned gather tasks Wire streaming generator cleanup to log client_disconnected with error_code 499 in spend logs, cancel pending during_call_hook tasks when the LLM call is cancelled on disconnect, and align the 600s poll limit comment with proxy_server. * fix: extract client disconnect logging helper to satisfy PLR0915 * fix: resolve mypy and code-quality CI failures for client disconnect logging Cast client disconnect error_information for mypy, only await pending gather tasks to avoid masking LLM errors, and add tests for the new logging helper and gather cleanup. * fix(proxy): harden gather cleanup so finally cannot mask LLM errors * fix(proxy): shield streaming disconnect logging and strip spoofable metadata Move streaming disconnect recording into a shielded cancel scope, add gather cleanup regression coverage for guardrail-converted cancels, and strip client_disconnected/error_information from user metadata at the proxy boundary. * fix(proxy): only map CancelledError to 499 for client disconnect Track when the disconnect poller cancels the LLM task and re-raise other CancelledError paths so graceful shutdown is not reported as HTTP 499. * fix(proxy): remove dead _check_request_disconnection helper Non-streaming client disconnect is handled by staging's cancel_on_disconnect path via _await_llm_call_cancelling_on_disconnect. Drop the unused is_disconnected poller and its unit tests; rename the remaining integration tests to TestDisconnectGatherCleanup. * feat(mistral): add mistral-medium-3-5 to model_prices_and_context_wind.. (#29303) * feat(mistral): add mistral-medium-3-5 to model_prices_and_context_window.json Mistral's docs page lists mistral-medium-3-5 as a new model offering. Pricing/specs sourced from Mistral's published model metadata: - input: $1.50 / 1M tokens - output: $7.50 / 1M tokens - context: 262,144 tokens - capabilities: vision, function calling, structured outputs, assistant prefill Adds entry: `mistral/mistral-medium-3-5`, mirroring the pattern used for the rest of the Mistral family. test(mistral): add model_info test for mistral-medium-3-5 + sync backup cost map - Mirror mistral/mistral-medium-3-5 entries into litellm/model_prices_and_context_window_backup.json so the bundled model cost map matches the canonical model_prices_and_context_window.json. - Add tests/test_litellm/test_mistral_medium_3_5_model_metadata.py covering pricing tiers, capability flags, context window, provider routing, and parity between the main and backup cost maps. - Point 'source' at the live Mistral models documentation page. * fix(ui): three small UI fixes — Gemini api_base + credential form reset + Mode badge (#30419) * fix(ui): three small UI fixes — Gemini api_base field + credential form reset + Mode badge Three independent fixes; bundled because they all touch the credential-form / logging-callbacks area. 1. expose api_base field on Google AI Studio credential form The runtime gemini provider supports custom api_base via `vertex_llm_base._check_custom_proxy`; the UI just needs to expose the field. Adds api_base to the Google_AI_Studio credential form ordered before api_key (matching OpenAI/Anthropic conventions). Default value matches the canonical Google AI Studio endpoint that LiteLLM's gemini provider talks to when api_base is unset, so leaving the default in the form behaves identically to leaving it blank. 2. reset credential form state when switching providers Switching the Provider select in AddCredentialModal / EditCredentialModal left the previous provider's field values populated. The form then submitted a mixed payload (e.g. Azure deployment fields under an OpenAI credential), producing confusing failures. Extract `getProviderFieldDefaults` helper and reset the form to it on provider change. Unit-tested via the extracted helper because Antd Select's portal/dropdown behaviour is unreliable in jsdom. 3. logging callbacks table reads backend `type` for Mode badge (#35) The `/get_callbacks` proxy endpoint returns each callback as `{name, type, variables}` where `type` is `"success"` or `"failure"`. The same callback name can appear twice (one per event class) and the two entries fire on disjoint events. `LoggingCallbacksTable` ignored `type` and read `record.mode` (always undefined), so every row fell back to the "Success" badge. A `generic_api` callback registered for both classes showed up as two identical "Success" rows + React duplicate-key warning. Read `record.type` first (fall back to `record.mode` for newly- added not-yet-server-acknowledged rows). Composite rowKey `${name}-${type ?? mode ?? 'success'}`. Removed leftover debug `console.log`. * fix(ui): drop api_base default_value to preserve Gemini v1alpha auto-routing Greptile P2 (PR #30419, threads on lines 1255-1256 of provider_create_fields.json): the api_base field's `default_value` was hard-coded to "https://generativelanguage.googleapis.com/v1beta". This: 1. Bakes v1beta into every credential record saved through the form, even when the user never touched the field. If LiteLLM's internal gemini default URL ever changes, those persisted credentials keep hitting the stale path. 2. Bypasses `_get_gemini_url`'s automatic version routing for Gemini 3+ models. That helper picks v1alpha for Gemini 3+ and v1beta for older models when api_base is unset. With the default pre-filled (and `_check_custom_proxy` then taking over because api_base is non-empty), Gemini 3+ requests get pinned to v1beta and may fail or behave unexpectedly — purely because the user accepted the visible default. Fix: set `default_value` to `null` and move the canonical URL guidance into the `placeholder` (visible to the user, never persisted) and an expanded tooltip. UX is unchanged — the URL is still shown in the greyed-out input — but the auto-version-routing path stays default. Updated test_google_ai_studio_provider_fields_expose_api_base to assert the new contract (`default_value is None`, `placeholder` carries the canonical URL), with a comment pointing at the Greptile threads as the rationale so future contributors don't accidentally re-introduce the default. 26/26 tests in the file pass. JSON validates (`json.load` clean). * feat(azure_ai): add gpt-5.5 to model cost map (#30428) * feat(azure_ai): add gpt-5.5 to model cost map Adds azure_ai/gpt-5.5 and its dated snapshot azure_ai/gpt-5.5-2026-04-23 to both the canonical and bundled cost maps. gpt-5.5 is generally available on Azure AI Foundry; pricing mirrors the openai gpt-5.5 entry, matching the established azure_ai convention (verified identical for gpt-5.4), in the azure tier structure (base / above-272k / priority). supports_minimal_ reasoning_effort is false, the capability that changed from gpt-5.4. Fixes #30306 * Update tests/test_litellm/test_gpt_5_5_model_metadata.py Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com> --------- Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com> * fix: guard check_and_fix_namespace against None key (#30435) * fix: guard check_and_fix_namespace against None key When user_id is None, the cache key can be None, causing AttributeError: 'NoneType' object has no attribute 'startswith' in check_and_fix_namespace. Add an early return for None key to prevent the error and the ERROR-level log noise it produces on every unauthenticated request. Fixes #30424 * fix: update type annotations for check_and_fix_namespace - key: str -> Optional[str] (now handles None input) - return: str -> Optional[str] (returns None when input is None) Addresses Greptile review concern about type signature mismatch. * fix: revert check_and_fix_namespace type signature to str to fix MyPy downstream errors * fix: update type annotations for check_and_fix_namespace - Change signature from str -> str to Optional[str] -> Optional[str] - Remove type: ignore comment on None return - Add None guard in async_set_cache_sadd before passing to helper Addresses review feedback from Sameerlite on type mismatch. * Revert "fix: update type annotations for check_and_fix_namespace" This reverts commit |
||
|
Mateo Wang
|
38a9a60484
|
fix: greatly increase slack (#30563) | ||
|
Mateo Wang
|
d0c2e87810
|
ci: ratchet lint and type-check gates (ruff preview, ANN, mypy, basedpyright) (#30379)
* ci: enable ruff preview rules under the budgeted strict gate
Turn on ruff preview in the strict-budget lane (ruff-strict.toml) only,
leaving the clean gate (ruff.toml) untouched so make lint-ruff stays at
zero. Enumerate the 118 firing codes explicitly with
explicit-preview-rules so the gate is deterministic and stable across
ruff upgrades rather than depending on preview auto-selecting the broad
catalog.
Grandfather the existing 58438 violations into ruff-strict-budget.json
as per-rule baselines with headroom, so only net-new violations fail CI.
The existing ten rules keep their hand-tuned slack; the new rules get
slack 10 when the baseline is 50 or more and 3 otherwise.
* ci: add ANN return-type rules to the budgeted strict gate
Add ANN201/202/204/205/206 (missing return annotations) to the strict
lane and grandfather the existing counts into ruff-strict-budget.json so
the codebase ratchets toward explicit return types without breaking CI.
* ci: add mypy (disallow_untyped_defs) and basedpyright strict gates with baselines
Add two type-check gates, each grandfathering the current tree so only
net-new violations fail CI, matching the ruff strict-budget ratchet.
mypy gains disallow_untyped_defs in litellm/mypy.ini (the config the CI
invocation actually reads; the root [tool.mypy] is not picked up from the
litellm/ working dir). The 4885 existing missing-annotation errors are
captured in litellm/.mypy-baseline.txt and the run is piped through
mypy-baseline filter so new untyped defs are rejected.
basedpyright runs in strict mode over litellm/, with
enableTypeIgnoreComments disabled so it only honors '# pyright: ignore'
and never polices mypy's '# type: ignore'. The existing strict diagnostics
are grandfathered into .basedpyright/baseline.json.
Both tools are pinned in the dev group and uv.lock; the lint workflow and
Makefile run them filtered through their baselines, with
lint-mypy-baseline-update and lint-basedpyright-baseline-update to ratchet.
* ci: raise lint job timeout to 15m for the basedpyright strict pass
* ci: pin pythonVersion 3.12 and regenerate baselines against merged base
Merge litellm_internal_staging so the baselines cover code the CI merge
includes (e.g. the cisco_ai_defense guardrail), which otherwise tripped
the mypy gate with 3 ungrandfathered no-untyped-def errors. Pin
pythonVersion 3.12 in pyrightconfig so basedpyright's strict analysis is
reproducible across interpreter versions (CI runs 3.12).
* ci: regenerate basedpyright baseline against the frozen lint env
The previous baseline was generated with optional provider deps (azure,
google, anthropic, mcp, numpydoc, google-genai) installed locally, so CI's
dev-only env surfaced ~3500 reportUnknown*/reportMissingTypeStubs errors
not in the baseline. Regenerate after uv sync --frozen so the baseline
reflects the same dependency set the lint job sees.
* ci: regenerate basedpyright baseline on python 3.12 frozen env
The prior baseline still carried proxy-dev packages (e.g. prisma) that the
lint job's dev-only, python 3.12 env lacks, leaving 2 unresolved-import
errors ungrandfathered. Regenerate in a python 3.12 venv synced to the
frozen lock with default groups only, so the baseline matches exactly what
CI sees.
* ci: replace type-check baselines with per-file count budgets
The mypy and basedpyright baselines were position-sensitive (and the
basedpyright one was a 27MB file), so ordinary line shifts churned them.
Replace both with a per-file count gate: scripts/type_check_gate.py reduces
each tool's output to errors-per-file and checks it against a committed
{file: max} budget, ignoring line and column numbers. A file fails only
when it gains more errors than its ceiling; debt can't be shuffled between
files because each file has its own cap and new files default to zero.
Budgets (mypy-file-budget.json 48K, basedpyright-file-budget.json 96K) are
generated in the python 3.12 frozen lint env so they match CI. Drops the
mypy-baseline dependency; basedpyright runs without its native baseline.
ratchet via make lint-mypy-budget-update / lint-basedpyright-budget-update.
* ci: add a small per-file slack to the type-check gate
Allow each file to drift PER_FILE_SLACK (5) errors past its recorded count
before failing, so a basedpyright inference ripple in an unrelated file
doesn't break the build over a couple of errors. Budgets still record exact
counts; the tolerance is applied at check time.
* ci: move type-check slack into the budget json and trim lint timeout
Make slack declarative: the budget is now {"slack": N, "files": {path: count}}
so the tolerance is tuned in JSON without editing the script, mirroring how
ruff-strict-budget.json carries its slack. --update preserves the existing
slack. Also drop the lint job timeout from 15m to 10m; the mypy and
basedpyright passes add ~2m, leaving the job around 4-5m, so 10m is a
comfortable margin.
* ci: collapse fully-adopted ruff categories and drop inert preview flag
ANN (all nine non-removed rules) and BLE (its only rule) were spelled out
code-by-code; replace each with its category selector, which is exactly
equivalent in 0.15.3 (the removed ANN101/ANN102 are skipped by a category
selector and error when named explicitly). explicit-preview-rules was inert:
every selected rule is stable and nothing is selected by category, so the flag
had nothing to gate. Verified the strict-rule counts are identical before and
after (62379 each, zero per-rule drift), so no budget change.
* ci: drop redundant pyright dev dependency
Nothing invokes bare pyright in the Makefile, the linting workflow, or
scripts; the basedpyright gate added on this branch is the only type
checker that runs. basedpyright is a superset fork that reads the same
pyrightconfig.json and honors the same "# pyright: ignore" comments, so
pyright==1.1.408 in the ci group was dead weight. Regenerated uv.lock
under the same exclude-newer cutoff so the only change is removing
pyright and its package stanza
* ci: un-weaken mypy and error on Any in basedpyright
mypy: enable warn_return_any, drop the valid-type silencer, and stop globally ignoring missing first-party imports via [mypy-litellm.*] ignore_missing_imports = False, which surfaced eight real broken litellm.* imports the blanket ignore was hiding; third-party imports stay ignored. The per-file budget moves 4888 -> 5799 (902 no-any-return, 1 valid-type, 8 import-not-found), all grandfathered so only net-new errors fail and the ceilings ratchet down
basedpyright: error on reportExplicitAny and reportAny. The per-file budget moves 117033 -> 148946 (6931 explicit-Any, 24954 Any-typed expressions), grandfathered the same way
* ci: add Any-discipline gate on changed lines under litellm/
Add scripts/check_any_discipline.py, a type-aware gate that fails when a
changed line holds a value typed Any -- including the X | Any unions that
mypy --strict / basedpyright accept (e.g. re.Match.group() -> str | Any,
json.loads() -> Any, bare dict -> dict[Any, Any]).
It reuses the repo's mypyc-compiled mypy 1.19 via a custom generic AST
walker (mypyc precludes subclassing TraverserVisitor), loads litellm/mypy.ini
for parity with lint-mypy, and uses a dedicated incremental cache
(.mypy_cache_any) with mtime+hash invalidation to force re-checks. Scope is
changed-lines-only so editing a legacy file never forces cleaning its
existing Any debt; suppress a genuine typed/untyped boundary with
# any-ok: <reason> (ANY002 requires the reason).
Wire it into the Makefile (lint-any, lint, lint-dev), a parallel
any-discipline CI job with its own actions/cache, .gitignore, and the
CLAUDE.md / CONTRIBUTING.md docs.
* ci: move Any-gate codes into the shared LIT namespace
Renumber the Any-discipline checker into the LIT*** scheme owned by
scripts/check_type_discipline.py (PR #30500) so the two checkers share one
rule namespace and suppression convention:
ANY001 -> LIT002 (Any-typed value; LIT002 was the retired/free slot)
ANY002 -> LIT005 (any-ok without a reason; the shared suppression-reason code)
ANY000 -> LIT000 (setup/build/read error; the shared error code)
Messages and behavior are unchanged; LIT005's text already matches the
"<token> requires a reason" shape used for cast-ok/guard-ok.
* ci: gate mypy and basedpyright per error rule, not per file
Switch the mypy/basedpyright budget gate from per-file error counts to
per-rule-code totals, mirroring the {rule: {baseline, slack}} shape of
ruff-strict-budget.json. A rule fails when its codebase-wide error count
exceeds baseline + slack, so violations are tracked by category rather
than by file location.
scripts/type_check_gate.py now parses mypy from its text output (trailing
[code]) and basedpyright from --outputjson (the JSON `rule` field), since
basedpyright's wrapped text diagnostics mis-attribute the rule on
continuation lines. Replace the *-file-budget.json files with freshly
captured *-code-budget.json baselines and update the Makefile, CI, and
CLAUDE.md accordingly.
* docs: prefer Pydantic validation over any-ok suppression
Point the Any-discipline guidance at validating Any with Pydantic (a model
or TypeAdapter that returns a typed value or raises) and frame
# any-ok as a last resort that should ideally never be used.
* chore: remove extraneous comment
* chore: make the CLAUDE.md more concise
* chore: clean up bloated CONTRIBUTING.md additions
* chore: make Makefile more concise
* ci: add the lint-budget-update target CLAUDE.md references
CLAUDE.md tells contributors to run make lint-budget-update, but the
target was never defined. Add it as an aggregate that re-captures the
ruff, mypy, and basedpyright budgets in one shot.
* ci: recapture mypy and basedpyright budgets in the lint env
The per-rule baselines were captured in a richer dependency env than the
CI lint job's uv sync --frozen, so CI resolved fewer types and reported
more errors than the budgets allowed (no-any-return 902 over cap 900, plus
several basedpyright reportUnknown* rules). Regenerate both in the frozen
env so they grandfather the true CI debt: mypy 5786 -> 5799 (no-any-return
890 -> 902, valid-type 1 restored), basedpyright 146213 -> 148942.
* ci: check out PR head sha in lint and any-discipline jobs
The default pull_request checkout uses refs/pull/N/merge, which folds the
latest base commits into HEAD. The diff-based gates (ruff delta, Any
discipline) then diff against the event's older base.sha and blame base's
own new commits on this branch; staging's otel-v2 and streaming changes
(#30326, #30485) tripped the Any gate on files this branch never touched.
Checking out the PR head sha makes the gates diff the real branch tip
against base, and pins the tree the mypy/basedpyright budgets were captured
against so their counts stay deterministic as the base advances.
* ci(lint): renumber Any-typed-value rule LIT002 -> LIT009
Free up LIT002 for the sibling type-discipline gate (check_type_discipline.py,
#30500), which groups its mutable-collection family at LIT001 (annotation) and
LIT002 (construction). This gate's Any-typed-value rule moves to LIT009 so the
shared LIT namespace stays contiguous with no holes; LIT000 and LIT005 are
unchanged.
* style: rename lint-strict-budget -> lint-ruff-budget
* ci: harden type-check gates against silent passes (greptile review)
type_check_gate.py: refuse to certify a vacuous run. The CI pipe swallows
the tool's exit code ('tool || true'), so a crashed mypy/basedpyright that
emits nothing would parse to zero errors, breach no ceiling, and pass.
is_vacuous_run() now fails when nothing was parsed but the budget expects
errors. Also wrap basedpyright's json.loads in a JSONDecodeError handler
that prints the offending output instead of dumping a raw traceback.
check_any_discipline.py: ALL_LINES was None, which dict.get() also returns
for a path absent from the line map, so a path-normalisation mismatch could
let a violation on an unchanged file pass the scope filter. Make ALL_LINES a
distinct sentinel object so 'whole file' and 'path missing' are unambiguous.
Adds tests for all three.
|