Commit graph

36183 commits

Author SHA1 Message Date
Sameer Kankute
a174dc6db2
Fix greptile reviews and mock test 2026-04-03 14:25:03 -07:00
Sameer Kankute
2169ac43b3
Fix greptile reviews and mock test 2026-04-03 14:25:03 -07:00
Sameer Kankute
7af677af97
Fix greptile reviews and mock test 2026-04-03 14:25:03 -07:00
Sameer Kankute
55e8fa777d
Fix code qa issues 2026-04-03 14:25:03 -07:00
Sameer Kankute
b904e78899
Fix greptile comments 2026-04-03 14:25:03 -07:00
Sameer Kankute
acbe06b8f2
Fix greptile comments 2026-04-03 14:24:45 -07:00
Sameer Kankute
d00d7939cf
Fix greptile comments 2026-04-03 14:24:45 -07:00
Sameer Kankute
5f15ab864e
Fix greptile comments 2026-04-03 14:24:45 -07:00
Sameer Kankute
dc60e8e6f7
Fix greptile comments 2026-04-03 14:24:44 -07:00
Sameer Kankute
b1c97dcad9
fix(routing): address state consistency and type safety issues
- Check alias target pattern to detect stale team aliases
- Fix PrismaClient type annotation to Optional
- Eliminate in-place mutation in index update logic

Made-with: Cursor
2026-04-03 14:24:21 -07:00
Sameer Kankute
ef5591b561
perf(routing): optimize team model checks and improve test coverage
- Use O(1) team index lookup instead of map_team_model in alias guard
- Fix MockPrismaClient to validate where clause filters
- Add comment explaining DB query trade-off for team deployments

Made-with: Cursor
2026-04-03 14:24:21 -07:00
Sameer Kankute
b1800a6d38
fix(routing): prevent stale model_aliases from interfering with team routing
- Skip model_aliases rewrite if model resolves to team deployments
- Add test coverage for sibling-preservation branch
- Update MockPrismaClient to support sibling deployment scenarios

Made-with: Cursor
2026-04-03 14:24:21 -07:00
Sameer Kankute
023b5a52f0
fix(router): guard None model_info and deduplicate team index logic
- Guard against None model_info in sibling deployment check
- Extract _update_team_model_index helper to eliminate duplication

Made-with: Cursor
2026-04-03 14:24:05 -07:00
Sameer Kankute
d4969cfb1b
fix(management): query DB directly for sibling deployments on rename
- Add clarifying comments to test assertions
- Query prisma DB instead of in-memory router to avoid stale state
- Prevents incorrect deletion of old public name when siblings exist

Made-with: Cursor
2026-04-03 14:24:05 -07:00
Sameer Kankute
ea00646d5c
fix(router): prevent cross-team deployment leakage in fallback path
Guard should_include_deployment fallback to only return deployments
matching the requested team_id, preventing public-name collisions
from leaking deployments across teams

Made-with: Cursor
2026-04-03 14:24:05 -07:00
Sameer Kankute
b932aac33a
fix(router): address Greptile P1/P2 performance issues
- Guard against llm_router=None to prevent silent deletion
- Add O(1) team_model index to avoid O(n) scan on every team request

Made-with: Cursor
2026-04-03 14:24:05 -07:00
Sameer Kankute
da29da3bc3
fix(router): address remaining Greptile P0/P1 issues
- Update map_team_model test to expect public name return
- Only remove old public name if no sibling deployments use it

Made-with: Cursor
2026-04-03 14:24:04 -07:00
Sameer Kankute
114afc3a2d
fix(router): address Greptile review comments
- Add None guard for original_model_name in _add_team_model_to_db
- Remove stale old public name when renaming team model
- Add comment clarifying team deployment early-return priority

Made-with: Cursor
2026-04-03 14:23:49 -07:00
Sameer Kankute
2ddb56e00d
chore(team-routing): remove temporary candidate pool logs
Remove temporary fire-emoji router logs used for local verification while keeping team sibling deployment routing behavior unchanged.

Made-with: Cursor
2026-04-03 14:23:49 -07:00
Sameer Kankute
1245ab49bb
fix(team-routing): keep team model routing on public names
Remove team model_alias rewrites and resolve team deployments by team_public_model_name with team_id so sibling deployments stay in the routing candidate pool, with explicit logs showing candidate selection before load balancing.

Made-with: Cursor
2026-04-03 14:23:49 -07:00
Sameer Kankute
fbc4baebc4
docs: remove enable_pre_call_checks requirement from order docs
Order-based routing and fallback work without enable_pre_call_checks
in the current code. Remove the stale requirement from both doc files.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-04-03 14:22:28 -07:00
Sameer Kankute
6fdc29a292
fix(router): handle non-standard fallback formats with order-based fallback
When fallbacks use non-standard formats (e.g. ["claude-3-haiku"] or
[{"model": "...", "messages": [...]}]), detect them with
_check_non_standard_fallback_format and pass them through directly
instead of trying to parse with get_fallback_model_group which only
handles the standard dict-keyed format.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-04-03 14:22:28 -07:00
Sameer Kankute
08d1303c3c
feat(router): add order-based fallback so higher order deployments are tried on failure
When a request to an order=1 deployment fails, the router now
automatically tries order=2, order=3, etc. before falling through to
external fallbacks. Works for all error types (429, 404, connection
errors). Requires enable_pre_call_checks=True.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-04-03 14:22:28 -07:00
Sameer Kankute
040c7b53d3
feat(router): add order-based fallback so higher order deployments are tried on failure
When order=1 deployments fail, the router now automatically tries order=2,
then order=3, etc. before falling through to external fallbacks. This removes
the need for enable_pre_call_checks and makes order work as a true priority-based
fallback chain within a model group.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-04-03 14:22:04 -07:00
Harshit28j
6300a92aed
fix: address Greptile review feedback on key rotation lock 2026-04-03 14:21:35 -07:00
Krrish Dholakia
40879e7066 fix: remove dead migration sql 2026-03-23 20:14:37 -07:00
Ishaan Jaffer
659d4013c5 fix: address Greptile review issues
- GCPIAMCredentialProvider now inherits from redis.credentials.CredentialProvider
  so redis-py's async path calls get_credentials_async() properly
- move _redis_credential_provider import to top of _redis.py (PEP 8)
- remove dead else-branch that silently no-oped (gcp_service_account from
  redis_kwargs.get() was always None since it's popped by _get_redis_client_logic)
- remove mid-function 'from litellm import get_secret_str' inline import
- remove unused 'call' import from test_redis.py
2026-03-23 10:17:43 -07:00
Ishaan Jaffer
a1494b59a3 refactor(redis): move GCPIAMCredentialProvider to its own file
Extract GCPIAMCredentialProvider and _generate_gcp_iam_access_token
into litellm/_redis_credential_provider.py. _redis.py imports them
from there, keeping the public API unchanged.
2026-03-23 10:17:43 -07:00
Ishaan Jaffer
67b752ca8f fix(redis): regenerate GCP IAM token per connection for async cluster clients
Async RedisCluster was generating the IAM token once at startup and
storing it as a static password. After the 1-hour GCP token TTL, any
new connection (including to newly-discovered cluster nodes) would fail
to authenticate.

Fix: introduce GCPIAMCredentialProvider that implements redis-py's
CredentialProvider protocol. It calls _generate_gcp_iam_access_token()
on every new connection, matching what the sync redis_connect_func
already does. async_redis.RedisCluster accepts a credential_provider
kwarg which is invoked per-connection.
2026-03-23 10:17:43 -07:00
yuneng-jiang
9e98a7d7b3 Merge branch 'main' of github.com:BerriAI/litellm into litellm_rc_branch 2026-03-21 23:50:45 -07:00
yuneng-jiang
c89496f378
Merge pull request #24342 from BerriAI/litellm_yj_march_21_2026
[Fix] UI - Teams: Table refresh, infinite dropdown, leftnav migration
2026-03-21 23:36:21 -07:00
yuneng-jiang
38d477507d remove outdated e2e test 2026-03-21 23:14:53 -07:00
yuneng-jiang
6bb08883af adding poetry lock 2026-03-21 23:01:28 -07:00
yuneng-jiang
fa65433c8c bump: version 1.82.5 → 1.82.6 2026-03-21 22:56:09 -07:00
yuneng-jiang
d91980dc45 adding build 2026-03-21 22:55:04 -07:00
yuneng-jiang
071c8641de bump: version 0.4.59 → 0.4.60 2026-03-21 22:54:41 -07:00
yuneng-jiang
88a4c7aeaf bump: version 0.4.58 → 0.4.59 2026-03-21 22:54:38 -07:00
yuneng-jiang
34d079910b chore: update Next.js build artifacts (2026-03-22 05:53 UTC, node v22.16.0) 2026-03-21 22:53:50 -07:00
yuneng-jiang
9073daeebc [Fix] UI - TeamDropdown: Match org dropdown styling and fix test mock
- Use Select.Option with font-medium alias + Text secondary ID to match OrganizationDropdown
- Default page size to 20
- Add useInfiniteTeams mock to AddModelForm tests

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-21 22:52:16 -07:00
yuneng-jiang
aea8e32048 [Fix] UI: Team table refresh, infinite team dropdown, leftnav for dashboard routes
- OldTeams: refresh table via fetchTeamsV2 after team create instead of appending
- TeamDropdown: rewrite with useInfiniteTeams for paginated fetch, scroll-to-load, and debounced search
- Update all TeamDropdown consumers to use the new self-fetching API
- Dashboard layout: switch from Sidebar2 to SidebarProvider (leftnav)
- Leftnav: add MIGRATED_PAGES routing for path-based navigation (api-reference)
- Navbar: remove chat button

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-21 22:05:26 -07:00
yuneng-jiang
9963b31e07 Revert "fix(proxy): restore per-entity breakdown in aggregated daily activity endpoint"
This reverts commit 9c3fab24ad.
2026-03-21 21:37:29 -07:00
yuneng-jiang
f5194b5ce3
Merge pull request #24334 from BerriAI/litellm_yj_march_21_2026
[Infra] Build UI for release
2026-03-21 18:03:38 -07:00
yuneng-jiang
d217e49193 chore: update Next.js build artifacts (2026-03-22 01:02 UTC, node v22.16.0) 2026-03-21 18:02:32 -07:00
yuneng-jiang
e3d4c29d37
Merge pull request #24323 from BerriAI/litellm_ryan_march_20
litellm ryan march 20
2026-03-21 15:57:28 -07:00
Ryan Crabbe
3e27ff1b78 fix: resolve mypy type errors in audit_logs.py 2026-03-21 15:42:01 -07:00
Ryan Crabbe
f494ab513f docs: add High Availability Control Plane documentation
New docs page covering the HA control plane architecture where each
worker instance has its own DB, Redis, and master key. Includes a
React component diagram, setup configs, SSO notes, and local testing
instructions.
2026-03-21 15:31:49 -07:00
yuneng-jiang
1986f1034e
Merge pull request #24211 from BerriAI/litellm_dev_sameer_16_march_week
Litellm dev sameer 16 march week
2026-03-21 15:19:15 -07:00
yuneng-jiang
72fba093c8 Merge remote-tracking branch 'origin/main' into litellm_dev_sameer_16_march_week 2026-03-21 15:11:29 -07:00
yuneng-jiang
7b31ea40a9
Merge pull request #22844 from BerriAI/litellm_oss_staging_03_05_2026
Litellm oss staging 03 05 2026
2026-03-21 15:09:48 -07:00
Ryan Crabbe
9b90e80f71 fix: resolve mypy type errors in audit_logs.py
Extract multiline `or` chain from LiteLLM_AuditLogs constructor to fix
pydantic mypy plugin field-type misattribution, and add explicit
Optional[bool] annotation to avoid variable name shadowing conflict.
2026-03-21 15:08:47 -07:00