scriptoriumadmin/litellm
1
0
Fork 0
mirror of https://github.com/BerriAI/litellm.git synced 2026-09-17 23:51:30 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 67
48983 commits 2515 branches 1410 tags 3 GiB
Commit graph

16591 commits

Author SHA1 Message Date
yassin
df5e193645 Merge remote-tracking branch 'origin/main' into pr-40997
Some checks failed
ai-gateway image / ai-gateway release image (push) Has been cancelled
Details
LiteLLM Rust / rust-lint (push) Has been cancelled
Details
LiteLLM Rust / rust-test (push) Has been cancelled
Details 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

# Conflicts:
#	ui/litellm-dashboard/src/components/templates/key_edit_view.tsx
 2026-09-14 22:07:48 +00:00
Yassin Kortam
e766277846
Merge pull request #40991 from BerriAI/litellm_team_model_cooldown_siblings 
fix(router): cool down team deployments on 429 when a sibling serves the same public model
 2026-09-14 14:57:07 -07:00
Yassin Kortam
3e417fa6e0
Merge pull request #41113 from BerriAI/litellm_realtime_release_max_parallel_slot 
fix(proxy): release max_parallel_requests slot when a realtime session ends without LLM callbacks
 2026-09-14 14:56:11 -07:00
yujonglee
9bbb869b13
Merge pull request #41120 from BerriAI/litellm_trace_profiler_threads 
refactor(harness): expand independent trace coverage
 2026-09-14 14:50:18 -07:00
Yassin Kortam
24bfd5fba1
Merge pull request #41102 from BerriAI/litellm_team_membership_once_main 
fix(auth): load team membership once per request and skip prisma on an L1 hit
 2026-09-14 14:46:06 -07:00
yassin
fbc1011d27 fix(bedrock): end the realtime session when the client disconnects instead of waiting for Nova Sonic 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-14 21:42:42 +00:00
yassin
b4d0f4ad26 refactor(realtime): move session ownership marker keys into constants 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-14 21:33:48 +00:00
kerry-berri
c6e4c5582d
Merge pull request #41093 from BerriAI/litellm_fallback_backfill_opt_in_main 
feat(model_info): provider-scoped fill_missing_for_providers backfill from fallback generalization rules
 2026-09-14 14:20:36 -07:00
yassin
d2342f06ce fix(bedrock): stamp the realtime success ownership marker when Nova Sonic spend is logged 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-14 21:13:29 +00:00
yassin
db8dfe93a5 fix(auth): wait for the in-flight membership load before evicting its cache key on invalidation 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-14 21:10:43 +00:00
yassin
2441e8a2a9 Merge remote-tracking branch 'origin/main' into litellm_realtime_release_max_parallel_slot 2026-09-14 21:03:28 +00:00
Yujong Lee
da839d4a11 fix(harness): skip unavailable Rust traces 2026-09-14 14:01:28 -07:00
Yassin Kortam
4123b4bc2b
Merge pull request #41086 from BerriAI/litellm_team_list_org_admin_membership_lit3723 
fix(proxy): keep org admins' own team memberships in other orgs visible on team list
 2026-09-14 13:57:08 -07:00
Yassin Kortam
ee03bad8c6
Merge pull request #41064 from BerriAI/litellm_bedrock_realtime_propagate_provider_failures 
fix(bedrock/realtime): propagate deferred Nova Sonic stream failures to the router
 2026-09-14 13:56:26 -07:00
yassin
ffeea30f23 test(auth): cover a stale membership write landing after a fresh reload 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-14 20:55:22 +00:00
tin-berri
2cad7a49af
Merge pull request #41116 from BerriAI/litellm_auto_router_session_api_access 
fix(proxy): allow LLM API keys to read auto-router sessions
 2026-09-14 13:49:03 -07:00
yassin
abc85ba607 fix(proxy): leave the realtime max_parallel_requests slot to the limiter failure callback when a refusal was logged 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-14 20:47:37 +00:00
yassin
91c964a338 fix(auth): evict the membership cache entry when invalidation lands during the cache write 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-14 20:46:09 +00:00
Yassin Kortam
161ae2f3b4
Merge pull request #41107 from BerriAI/litellm_hide_credentials_hint_when_ui_password_set 
fix(proxy): hide default credentials login hint when UI_PASSWORD is set
 2026-09-14 13:44:09 -07:00
Yassin Kortam
97ddb9494e
Merge pull request #40930 from BerriAI/litellm_flat_retry_records 
fix(router): record flat retry attempts and cap retries from attempted_retries
 2026-09-14 13:37:10 -07:00
Tin Chi Lo
3177c37e22 fix(cli): show routed models and session stats for LLM API keys 2026-09-14 13:36:00 -07:00
Yassin Kortam
856aedd14a
Merge pull request #40817 from BerriAI/litellm_redis_timeout_log_throttle 
fix(redis): log a timeout streak once per interval instead of one line per cache call
 2026-09-14 13:34:45 -07:00
Yassin Kortam
ab09abc3c8
Merge pull request #41115 from BerriAI/litellm_fix_wrapper_async_double_sync_success_handler_main 
fix(utils): stop wrapper_async submitting the sync success handler twice
 2026-09-14 13:28:16 -07:00
Yassin Kortam
963522207d
Merge pull request #41114 from BerriAI/litellm_model_info_skip_jsonable_encoder_main 
perf(proxy): serialize /model/info listing once with orjson
 2026-09-14 13:28:06 -07:00
yassin
1b31be1a9c fix(proxy): leave the realtime max_parallel slot to the success callback when one is enqueued 
Releasing the slot unconditionally from the route raced the limiter's own
success handler on the logging worker: both could read the same stashed
acquisition before either cleared it, and under the integer in-memory
fallback that double-decrements the counter. The route now releases only on
exits without a success callback (pre-call rejection, pre-call cancellation,
and Phase 2 exits without the success stamp), matching the HTTP disconnect
path's ownership rule.

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-14 20:23:23 +00:00
yassin
4ac168b3fc fix(auth): drop in-flight membership load on invalidation so it cannot repopulate the cache 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-14 20:13:39 +00:00
yassin
a4e34d6e1b test(caching): wrap the DualCache fixture line to the 120 character limit 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-14 20:05:14 +00:00
yassin
bea22df6c2 fix(caching): keep generic add_cache failures at ERROR unless the backend is Redis 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-14 19:59:10 +00:00
yassin
845596063d fix(prometheus): label pre-call rate limit failures with the resolved api_provider 
Pre-call limiters reject before a deployment is attached to request_data, so
the failure hook could not resolve api_provider for router aliases and emitted
api_provider="None" on litellm_proxy_failed_requests_metric_total and
litellm_proxy_total_requests_metric_total. Fall back to the provider the
limiter already resolved onto RateLimitError.llm_provider, keeping request
data as the first source and ignoring the proxy placeholder.

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-14 19:50:21 +00:00
yassin
81ffc3125f fix(proxy): release realtime max_parallel_requests slot when the task is cancelled during pre-call 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-14 19:50:01 +00:00
yassin
39f6ac4788 perf(proxy): serialize /model/info listing once with orjson 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-14 19:47:51 +00:00
yassin
d13e8dcae2 fix(utils): stop wrapper_async submitting the sync success handler twice 
_client_async_logging_helper re-submitted logging_obj.success_handler to the
executor after _dispatch_success_logging had already done so, running the same
success pipeline twice per async request and racing on shared logging state.

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-14 19:47:43 +00:00
Yassin Kortam
b94b8bca21
Merge pull request #41096 from BerriAI/litellm_disable_access_log_paths 
feat(proxy): honor LITELLM_DISABLE_ACCESS_LOG_PATHS to drop noisy uvicorn access log lines
 2026-09-14 12:41:34 -07:00
Yassin Kortam
f67cc62032
Merge pull request #40995 from BerriAI/litellm_all_deployments_cooldown_error 
fix(router): name the all-deployments-in-cooldown error on 429 responses
 2026-09-14 12:39:42 -07:00
yassin
c4354c248a fix(redis): route the per-TTL pipeline write timeout log through the shared throttle 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-14 19:38:14 +00:00
Yassin Kortam
7c55c4ba7d
Merge pull request #40925 from BerriAI/litellm_proxy_error_headers_from_litellm_response_headers 
fix(proxy): forward provider request id headers on mapped error responses
 2026-09-14 12:36:59 -07:00
Yassin Kortam
d541792b30
Merge pull request #41106 from BerriAI/litellm_allowed_file_extensions 
feat(proxy): add general_settings.allowed_file_extensions for /v1/files uploads
 2026-09-14 12:34:19 -07:00
yassin
f9d0cd6409 chore(proxy): drop explanatory docstrings from credentials hint helper and tests 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-14 19:29:30 +00:00
yassin
ebcd9bcb18 fix(proxy): release max_parallel_requests slot when a realtime session ends without LLM callbacks 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-14 19:29:11 +00:00
yassin
ff02390685 test(auth): drop docstrings from team membership tests 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-14 19:26:14 +00:00
Yujong Lee
27981c7d20 fix: preserve rust setting in trace parity 2026-09-14 12:17:15 -07:00
yassin
53c8a82a31 Merge remote-tracking branch 'origin/main' into pr-40817 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

# Conflicts:
#	litellm/caching/redis_cache.py
#	litellm/proxy/proxy_server.py
#	tests/test_litellm/caching/test_redis_cache.py
 2026-09-14 19:11:03 +00:00
Yujong Lee
db378d8963 fix(harness): preserve unmapped calls in execution traces 2026-09-14 12:08:31 -07:00
yassin
771430df7b fix(bedrock/realtime): keep partial spend on cancelled output task and make the committed-session refusal non-retryable 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-14 19:07:21 +00:00
yassin
5c04ec2b93 fix(bedrock/realtime): keep the pending session.update until a provider stream is committed 
Peek at the pending session.update instead of popping it, so an eager fallback failure before the bridge starts does not lose the replay for the next attempt. Move the websocket scope keys to constants.py

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-14 19:07:20 +00:00
yassin
cb5d901774 fix(bedrock/realtime): propagate deferred Nova Sonic stream failures to the router 
Bedrock realtime caught every exception inside both forwarding tasks and
gathered them with return_exceptions=True, so a provider failure surfacing
after the websocket handshake (lazy duplex stream: 503/429/validation only
show up on await_output or the input publisher) made async_realtime return
normally and the router recorded a success instead of running fallbacks and
cooldown accounting. session.updated is now acked only after Bedrock is
ready, provider failures escape as BedrockError with the AWS status code,
a failure after the client disconnected is not reported as a provider
failure, and a fallback attempt on the same websocket replays the pending
session.update instead of emitting a second session.created.

Resolves LIT-6484

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-14 19:07:20 +00:00
yassin
b00bb35563 fix(auth): drop the membership write-epoch and background Redis replicate, load membership lazily 
Move the cache-miss marker out of litellm.constants into auth_checks (CodeQL cyclic import) and stop
logging user_id/team_id in the lookup failure (CodeQL log injection).

Write the membership row through DualCache synchronously again instead of a background Redis task
guarded by a bounded write-epoch map: the epoch was sampled after the Prisma read, so an invalidate
that raced the read could be cached as current, and eviction of the epoch entry could let an old
Redis write land. The synchronous write keeps invalidate_team_member_spend_state authoritative.

Lookup failures return None again (fail-open like main) instead of 503, and the load is skipped on
routes that neither resolve a model nor run budget checks.

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-14 19:06:29 +00:00
yassin
ba171d9eb4 fix(proxy): hide default credentials login hint when UI_PASSWORD is set 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-14 19:06:04 +00:00
Yujong Lee
209fc7afb0 fix(harness): trace OCR Python and native dispatch paths 2026-09-14 12:03:54 -07:00
yassin
ce0301c23f refactor(proxy): drop redundant docstrings from upload allowlist helpers and tests 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-14 19:00:49 +00:00