Commit graph

35352 commits

Author SHA1 Message Date
Yuneng Jiang
9e460aa63a
chore: fixes
Some checks failed
Unit Tests: Caching (Redis) / caching-redis (push) Has been cancelled
Unit Tests: Proxy DB Operations / proxy-db (auth-checks, tests/proxy_unit_tests/test_auth_checks.py tests/proxy_unit_tests/test_user_api_key_auth.py, 20, 8) (push) Has been cancelled
Unit Tests: Proxy DB Operations / proxy-db (key-generation, tests/proxy_unit_tests/test_key_generate_prisma.py, 30, 0) (push) Has been cancelled
Unit Tests: Proxy DB Operations / proxy-db (remaining, tests/proxy_unit_tests --ignore=tests/proxy_unit_tests/test_key_generate_prisma.py --ignore=tests/proxy_unit_tests/test_auth_checks.py --ignore=tests/proxy_unit_tests/test_user_api_key_auth.py, 20, 8) (push) Has been cancelled
Unit Tests: Security / security (push) Has been cancelled
2026-04-05 00:15:37 -07:00
Ryan Crabbe
f40c3f45de Add unit tests for view_logs columns.tsx
Test cell rendering logic for Duration (ms→s conversion), TTFT
(timestamp math with edge cases), Tags (overflow count), and Model
(logo fallback chain) columns, plus createColumns sort behavior.
2026-03-13 22:40:54 -07:00
yuneng-jiang
82cdb5b0fb
Merge pull request #23595 from BerriAI/litellm_ui_keys_org_13
[Feature] UI - Keys: Add Organization Dropdown to Create/Edit Key
2026-03-13 20:44:03 -07:00
yuneng-jiang
f6a8087375 [Test] Add test for all-team-models sentinel skip in team change validation
Verifies that validate_key_team_change does not call can_team_access_model
for the "all-team-models" sentinel, allowing keys with that value to be
moved between teams without model validation failures.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-13 18:17:18 -07:00
yuneng-jiang
bce37e28c3 [Test] Add tests for organization dropdown in key create/edit
- OrganizationDropdown: renders options, calls onChange on selection,
  applies disabled state, handles empty list
- CreateKey: org dropdown renders, disabled for non-admin users,
  enabled for admins, form state updates on org selection
- KeyEditView: org dropdown renders, disabled for non-admin,
  enabled for admin, initializes from keyData.organization_id

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-13 18:15:42 -07:00
yuneng-jiang
c637c93a6a [Fix] Skip all-team-models sentinel in team change validation
When moving a key to a different team, `validate_key_team_change` was
treating "all-team-models" as a literal model name and checking if the
target team could access it. This always failed because "all-team-models"
is a UI/backend sentinel meaning "use whatever the team allows."

Also reorder checks so the membership check runs after data validation
(models, rate limits) but before permission checks, keeping the admin
early-return after all validation.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-13 18:08:02 -07:00
yuneng-jiang
8069829c59 Merge branch 'litellm_internal_dev_03_13_2026' of github.com:BerriAI/litellm into litellm_ui_keys_org_13 2026-03-13 17:13:43 -07:00
yuneng-jiang
c6da45795b temp commit 2026-03-13 17:12:30 -07:00
yuneng-jiang
6549b4583f
Merge pull request #23591 from BerriAI/litellm_internal_dev_03_12_2026
[Infra] Merge internal dev branch to main
2026-03-13 16:43:35 -07:00
yuneng-jiang
811ce13ad4
Merge pull request #23590 from BerriAI/litellm_refactor_update_key_fn
[Refactor] Extract validation from update_key_fn to fix PLR0915 lint
2026-03-13 16:33:34 -07:00
yuneng-jiang
1d403e9bd8 refactor(key_management): extract validation logic from update_key_fn to fix PLR0915
Extract permission checks and constraint validation from update_key_fn
into _validate_update_key_data helper to reduce statement count below
the 50-statement limit.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-13 16:29:28 -07:00
yuneng-jiang
85e5ef45d3
Merge pull request #23589 from BerriAI/litellm_fix_responses_background_cost_tests
[Fix] Update Outdated test_responses_background_cost Assertions
2026-03-13 16:23:44 -07:00
yuneng-jiang
3aeca22031 fix(test): update test_responses_background_cost assertions for pagination and stale cleanup
Tests were outdated after #23472 added pagination (take/order) to find_many
and stale-row cleanup via update_many. Updated assertions to match new call
signatures.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-13 16:18:15 -07:00
yuneng-jiang
dca231e3f6
Merge pull request #23557 from BerriAI/litellm_ui_key_org-2
[Feature] Allow Setting organization_id on Key Update
2026-03-13 16:10:35 -07:00
yuneng-jiang
818c097ca9 Fix self-exclusion hash mismatch and missing throughput field checks
The self-exclusion filter compared raw key strings against SHA-256
hashed tokens from the DB, so keys were never excluded and
double-counting persisted. Now hash data.key before comparison.

Also add tpm_limit_type/rpm_limit_type to _throughput_fields_changed
guard, fall back to existing_key_row.team_id for team limit checks
(matching the org pattern), and add team self-exclusion test.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-13 15:59:23 -07:00
yuneng-jiang
1038a119ce Skip org limit check when non-throughput fields are updated
Only run org validation (get_org_object + _check_org_key_limits) when
the update actually touches throughput-related fields (tpm_limit,
rpm_limit, or organization_id). Previously, any update to a key
belonging to an org would trigger the check, which would fail with a
400 if the org had been deleted — blocking unrelated field changes.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-13 15:38:42 -07:00
yuneng-jiang
133471f882 Fix double-counting bug in org/team key limit checks on update
When updating a key, _check_org_key_limits and _check_team_key_limits
would include the key being updated in the find_many results, causing
its current limits to be counted twice (once from the DB query, once
from the new requested limits). This caused false 400 errors on valid
limit adjustments.

Fix: exclude the key being updated (by matching token) from the
allocated totals before checking limits.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-13 15:27:56 -07:00
yuneng-jiang
0b3dc00440 Merge remote-tracking branch 'origin' into litellm_internal_dev_03_12_2026 2026-03-13 15:11:49 -07:00
yuneng-jiang
0e44c460b5
Merge pull request #23584 from BerriAI/litellm_release_day_03_12_2026
[Infra] Merge Release Day Branch with Main
2026-03-13 14:31:27 -07:00
yuneng-jiang
f351bbdb36 [Fix] Derive SPEND_PER_REQUEST dynamically in spend accuracy tests
Instead of hardcoding SPEND_PER_REQUEST (which broke when the model
changed from gpt-3.5-turbo-0301 to gpt-3.5-turbo), make a single
calibration request first, poll for its spend, and use that as the
per-request cost. Fails fast with pytest.fail() after 5 retries if
calibration cannot determine the cost.

Also fixes a bug in test_basic_spend_accuracy where the user spend
assertion error message referenced user_info['info'] instead of
user_info['user_info'].

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-13 13:27:06 -07:00
yuneng-jiang
f5662eeb8a [Fix] Update otel and spend tracking test configs to use gpt-3.5-turbo
Same deprecated model fix as proxy_server_config.yaml — these two CI
configs also referenced gpt-3.5-turbo-0301 which has no pricing data.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-13 13:08:36 -07:00
yuneng-jiang
3ec7c2a81e [Fix] Fail fast when team member spend not flushed in time
Increase wait timeout to 90s and pytest.fail() instead of silently
continuing, so the failure message points at the real cause.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-13 12:40:17 -07:00
yuneng-jiang
e2779639c0 [Fix] Fix test_users_in_team_budget using model with no pricing data
gpt-3.5-turbo-0301 was removed from the model cost map, so every call
had response_cost=0 and team member spend never increased. The wait
helper also returned True after 3s regardless of whether spend updated.

- Switch fake-openai-endpoint to gpt-3.5-turbo (has pricing in cost map)
- Remove premature early-return in wait_for_team_member_spend_update

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-13 12:35:20 -07:00
yuneng-jiang
4e70254d56 [Fix] Increase _delete_file retry budget to reduce flakiness
Increase max_retries from 6 to 9 and retry_delay from 10s to 20s
(180s total wait, up from 60s) to give batch cost tracking more time
to finish before cleanup attempts file deletion.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-13 12:32:55 -07:00
yuneng-jiang
57865ec96e Revert "[Fix] Populate _hidden_params.model_id in batch terminal-state shortcut path"
This reverts commit ca2033036f.
2026-03-13 12:21:39 -07:00
yuneng-jiang
ca2033036f [Fix] Populate _hidden_params.model_id in batch terminal-state shortcut path
The terminal-state DB shortcut in retrieve_batch returned a LiteLLMBatch
with empty _hidden_params, causing the managed_files hook to skip encoding
output_file_id into a unified ID. This adds the same model_id extraction
from unified_batch_id that the non-terminal path already has.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-13 12:14:58 -07:00
yuneng-jiang
5b755db329 [Fix] Correct mypy fixes for realtime_api and presidio
- realtime_api/main.py: Revert param types to Dict, explicitly construct
  RealtimeSessionConfig/RealtimeExpiresAfter before passing to
  RealtimeClientSecretRequest
- presidio.py: Move type:ignore[override] to def line where mypy reports it

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-13 12:10:27 -07:00
yuneng-jiang
1f71578de2 [Fix] Add flaky reruns to test_e2e_managed_batch
The test_e2e_managed_batch test intermittently fails during cleanup
when deleting the input file — the batch cost tracking hasn't finished
processing yet (batch_processed=true not set), causing a 400 error.
This is a timing race condition unrelated to batch retrieval logic.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-13 11:57:22 -07:00
yuneng-jiang
e40ebf262a Revert "[Fix] Populate _hidden_params.model_id in batch terminal-state shortcut path"
This reverts commit 1642a2f7ac.
2026-03-13 11:57:22 -07:00
yuneng-jiang
25e161a0fa fix(tests): remove test_completion_bedrock_claude_sts_oidc_auth and test_completion_bedrock_httpx_command_r_sts_oidc_auth that depend on external infra
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-13 11:53:53 -07:00
yuneng-jiang
efe6ddfc68
Merge pull request #23577 from Sameerlite/litellm_gpt54-tools-reasoning-routing
feat(openai): route gpt-5.4+ tools+reasoning to Responses API
2026-03-13 11:49:46 -07:00
Sameer Kankute
7dce61ce48 fix(tests): update TestGPT5ReasoningEffortPreservation for dict normalization
Made-with: Cursor
2026-03-14 00:13:16 +05:30
yuneng-jiang
a6ab172db0 [Fix] Use type:ignore instead of Union return type for realtime endpoint
The Union[RealtimeClientSecretResponse, Response] annotation breaks
FastAPI's response model generation. Revert to the original annotation
and suppress mypy on the error-path return instead.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-13 11:42:26 -07:00
yuneng-jiang
57397e0d26
Merge pull request #23576 from Sameerlite/litellm_gpt54-tools-reasoning-routing
Litellm gpt54 tools reasoning routing
2026-03-13 11:38:50 -07:00
yuneng-jiang
1642a2f7ac [Fix] Populate _hidden_params.model_id in batch terminal-state shortcut path
The terminal-state DB shortcut in retrieve_batch returned a LiteLLMBatch
with empty _hidden_params, causing the managed_files hook to skip encoding
output_file_id into a unified ID. This adds the same model_id extraction
from unified_batch_id that the non-terminal path already has.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-13 11:38:09 -07:00
Sameer Kankute
408b717cb9 Fix gpt 5 transformation tests 2026-03-14 00:00:02 +05:30
Sameer Kankute
30645d683f Reserve reasoning for responses via chat completion 2026-03-13 23:59:57 +05:30
Sameer Kankute
7abbe2dc06 Fix routing of tool call + reasoning effor for gpt-5.4 2026-03-13 23:59:53 +05:30
Sameer Kankute
3596464d11 Revert "feat(openai): drop reasoning_effort for gpt-5.4 when tools present"
This reverts commit 14b52b1318.
2026-03-13 23:59:48 +05:30
yuneng-jiang
e88ee338bd [Fix] Resolve 7 mypy linting errors across 5 files
- realtime_api/main.py: Widen param types to accept both Dict and Pydantic models
- proxy/realtime_endpoints/endpoints.py: Widen return type to Union[..., Response]
- proxy/guardrails/guardrail_hooks/presidio.py: Add type:ignore[override] for bytes in streaming return
- proxy/_experimental/mcp_server/rest_endpoints.py: Annotate _oauth2_flow with Literal type
- proxy/management_endpoints/ui_sso.py: Add httpx import under TYPE_CHECKING, remove invalid timeout kwarg from AsyncHTTPHandler.get()

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-13 11:21:49 -07:00
yuneng-jiang
6a90596377 updating Dockerfile to tar 7.5.11 2026-03-13 11:16:17 -07:00
Ishaan Jaff
1b96064600
fix(proxy): prevent OOM/Prisma connection loss from unbounded managed-object poll (#23472)
* fix(proxy): cap managed-object poll size + expire stale rows + kill-switch flag to prevent OOM/Prisma connection loss

* fix(constants): simplify PROXY_BATCH_POLLING_ENABLED readability

* docs+test: document new polling env vars, add pagination+stale-cleanup tests

* fix: exclude stale_expired from batch poll queries; fix update_many assertions in tests

* fix: scope stale cleanup to file_purpose, fix file_object mocks, add CheckBatchCost tests

* fix: avoid duplicate cost logging in fallback path; guard integer constants against zero/negative values

* fix: cache _has_batch_processed_column; guard cleanup from aborting poll; narrow fallback except

* fix: add complete/completed to primary query not_in; fix vacuous test assertion

- Primary find_many was missing "complete" and "completed" in its not_in
  filter, creating asymmetry with the fallback query. A job whose status
  was set to "complete" but whose batch_processed flag update failed would
  be silently re-fetched and re-processed every cycle, emitting duplicate
  cost logs.

- test_fallback_completion_update_omits_batch_processed patched
  _is_base64_encoded_unified_file_id to return None, causing an immediate
  continue — so update() was never called and the assertion looped over an
  empty list (vacuously true). Rewrote the test to mock the full
  completion pipeline, verify update() is called exactly once, and assert
  batch_processed is absent from the update data.

- Added symmetric test (primary path) proving batch_processed IS included
  when the column exists.

Made-with: Cursor
2026-03-13 11:01:40 -07:00
yuneng-jiang
488a4d2b63 bumping tar for security 2026-03-13 10:43:16 -07:00
yuneng-jiang
0fbfb8e772 fix(tests): remove test_oidc_circle_v1_with_amazon_fips that depends on external infra
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-13 10:33:52 -07:00
yuneng-jiang
9ad2223531 [Fix] Convert _has_attribute_error_in_chain from recursive to iterative
The recursive implementation was flagged by the recursive function detector
lint check. Converted to an iterative approach using an explicit stack and
seen set, with depth capped at DEFAULT_MAX_RECURSE_DEPTH.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-13 10:24:01 -07:00
yuneng-jiang
4c3a4e6c38
Merge pull request #23566 from BerriAI/revert-23535-litellm_improve_qa-5.4
Revert "QA:  improve gpt-5.4 code/bugs"
2026-03-13 10:16:34 -07:00
yuneng-jiang
2b71b0fb25
Revert "QA: improve gpt-5.4 code/bugs" 2026-03-13 10:15:47 -07:00
yuneng-jiang
8dc198eccf
Merge pull request #23535 from Sameerlite/litellm_improve_qa-5.4
QA:  improve gpt-5.4 code/bugs
2026-03-13 09:37:30 -07:00
yuneng-jiang
6ebd457146 fix(tests): update remaining PKCE SSO tests to mock get_async_httpx_client
The previous fix (124b44ec) only updated 3 tests but missed 10 more
that still patched the old `ui_sso.httpx.AsyncClient` path. Also
updated credential assertions to check Authorization header instead
of httpx.BasicAuth kwargs, matching the production code change.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-13 09:13:05 -07:00
yuneng-jiang
11cf288f98 fix(tests): fix broken test_router_fallbacks_with_cooldowns_and_model_id
The test used fallbacks=[{"gpt-3.5-turbo": ["123"]}] where "123" is a
model_id, but the fallback mechanism treats values as model group names.
This caused a ValueError since no model group "123" exists. Additionally,
mock_response propagates to fallback calls, making mock-based fallback
tests unreliable.

Simplified the test to verify that a RateLimitError doesn't permanently
cool down a deployment for subsequent requests.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-13 09:10:41 -07:00