Commit graph

50471 commits

Author SHA1 Message Date
yassin
5bcc23014e refactor(auth): drop the membership fetch docstring
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-14 19:12:35 +00:00
yassin
53c8a82a31 Merge remote-tracking branch 'origin/main' into pr-40817
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

# Conflicts:
#	litellm/caching/redis_cache.py
#	litellm/proxy/proxy_server.py
#	tests/test_litellm/caching/test_redis_cache.py
2026-09-14 19:11:03 +00:00
Yujong Lee
18b6049808 chore: reconcile trace branch history 2026-09-14 12:08:45 -07:00
Yujong Lee
db378d8963 fix(harness): preserve unmapped calls in execution traces 2026-09-14 12:08:31 -07:00
yassin
771430df7b fix(bedrock/realtime): keep partial spend on cancelled output task and make the committed-session refusal non-retryable
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-14 19:07:21 +00:00
yassin
5c04ec2b93 fix(bedrock/realtime): keep the pending session.update until a provider stream is committed
Peek at the pending session.update instead of popping it, so an eager fallback failure before the bridge starts does not lose the replay for the next attempt. Move the websocket scope keys to constants.py

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-14 19:07:20 +00:00
yassin
cb5d901774 fix(bedrock/realtime): propagate deferred Nova Sonic stream failures to the router
Bedrock realtime caught every exception inside both forwarding tasks and
gathered them with return_exceptions=True, so a provider failure surfacing
after the websocket handshake (lazy duplex stream: 503/429/validation only
show up on await_output or the input publisher) made async_realtime return
normally and the router recorded a success instead of running fallbacks and
cooldown accounting. session.updated is now acked only after Bedrock is
ready, provider failures escape as BedrockError with the AWS status code,
a failure after the client disconnected is not reported as a provider
failure, and a fallback attempt on the same websocket replays the pending
session.update instead of emitting a second session.created.

Resolves LIT-6484

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-14 19:07:20 +00:00
yassin
b00bb35563 fix(auth): drop the membership write-epoch and background Redis replicate, load membership lazily
Move the cache-miss marker out of litellm.constants into auth_checks (CodeQL cyclic import) and stop
logging user_id/team_id in the lookup failure (CodeQL log injection).

Write the membership row through DualCache synchronously again instead of a background Redis task
guarded by a bounded write-epoch map: the epoch was sampled after the Prisma read, so an invalidate
that raced the read could be cached as current, and eviction of the epoch entry could let an old
Redis write land. The synchronous write keeps invalidate_team_member_spend_state authoritative.

Lookup failures return None again (fail-open like main) instead of 503, and the load is skipped on
routes that neither resolve a model nor run budget checks.

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-14 19:06:29 +00:00
yassin
ba171d9eb4 fix(proxy): hide default credentials login hint when UI_PASSWORD is set
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-14 19:06:04 +00:00
Yujong Lee
209fc7afb0 fix(harness): trace OCR Python and native dispatch paths 2026-09-14 12:03:54 -07:00
yassin
ce0301c23f refactor(proxy): drop redundant docstrings from upload allowlist helpers and tests
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-14 19:00:49 +00:00
kerry
250ff03a04 feat(model_info): scope fill_missing rules to azure, bedrock and vertex hosts
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-14 18:58:53 +00:00
kerry
4cf20f31ed refactor(model_info): drop types import from fallback_generalizations
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-14 18:58:53 +00:00
kerry
0f846cecb8 fix(model_info): single-assign fill_missing_for_providers in rule compiler
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-14 18:58:53 +00:00
kerry
e366d3bd30 refactor(model_info): return Mapping from fill_missing matchers
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-14 18:58:53 +00:00
kerry
5781678477 refactor(model_info): drop litellm._logging import from fallback_generalizations
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-14 18:58:53 +00:00
kerry
e4e6b5daeb fix(lint): allow provider backfill dict annotations
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-14 18:58:53 +00:00
kerry
6f086f74b3 style(model_info): parameterize fill_missing annotations
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-14 18:58:53 +00:00
kerry
71744fb9d1 style(model_info): format provider backfill call
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-14 18:58:53 +00:00
kerry
812bbee0b3 fix(model_info): scope fill_missing backfill to the rule's providers
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-14 18:58:53 +00:00
Devin AI
fb2057fde7 refactor(model_info): rename backfill_exact_entries to fill_missing_fields
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-14 18:58:53 +00:00
Devin AI
42c708670d fix(model_info): guard backfill by mode, drop provider key, tighten claude major regex
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-14 18:58:53 +00:00
Devin AI
cee7215b24 feat(model_info): opt-in field-level backfill from fallback generalization rules
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-14 18:58:53 +00:00
yucheng-berri
cae4a65545
Merge pull request #40562 from BerriAI/litellm_otel_openinference_message_cap
fix(otel): cap per-index OpenInference message attributes span-wide
2026-09-14 11:55:37 -07:00
yucheng
80725addb4 Merge remote-tracking branch 'origin/main' into litellm_lit6314_guardrail_metadata_transfer 2026-09-14 18:54:24 +00:00
yucheng
b1a006ea66 test(ui): hoist not_run guardrail fixtures out of inline call args
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-14 18:54:24 +00:00
Mateo Wang
329ab62c05
Merge pull request #41104 from BerriAI/litellm_osv_mlflow_resuppress_main
build(deps): re-suppress GHSA-h7x2-h6g9-p789 in osv-scan on main, mlflow still has no fixed release
2026-09-14 11:53:45 -07:00
yassin
82ef6ea6ab feat(proxy): add general_settings.allowed_file_extensions for /v1/files
Opt-in allowlist for upload filename extensions, checked before the existing blocked_file_extensions blocklist and mapped through the same upload validation failure path. None keeps today's behaviour, [] rejects every upload, matching is case-insensitive on both sides, and a filename with no extension is rejected when the allowlist is set.

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-14 18:53:29 +00:00
Yujong Lee
d4869ba710 refactor: print rust and python traces independently 2026-09-14 11:48:25 -07:00
Yujong Lee
eb2ffaae79 fix(harness): rebuild bridge for trace parity 2026-09-14 11:48:25 -07:00
Yujong Lee
561533c596 wip 2026-09-14 11:48:24 -07:00
Yujong Lee
ad5b87eb91 fix: capture reused worker threads in Python traces 2026-09-14 11:47:30 -07:00
devin-ai-integration[bot]
32699a0be9 build(deps): re-suppress GHSA-h7x2-h6g9-p789 in osv-scan, mlflow still has no fixed release (#41036)
Co-authored-by: mateo <mateo@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-14 18:42:36 +00:00
yucheng
e41faf54a8 fix(ui): anchor guardrail lifecycle on timed entries and show not_run skip reason
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-14 18:39:10 +00:00
Yujong Lee
209913c1c7 refactor: print rust and python traces independently 2026-09-14 11:38:13 -07:00
Yujong Lee
f486579a27 fix(harness): rebuild bridge for trace parity 2026-09-14 11:37:40 -07:00
Yujong Lee
ed653ce9bf wip 2026-09-14 11:33:10 -07:00
Mateo Wang
cab1e113f7
Merge pull request #40976 from BerriAI/litellm_azure_gpt_chat_latest_pricing
feat(pricing): add azure gpt-chat-latest rates and drop retired friendliai llama-3.1 entries
2026-09-14 11:21:20 -07:00
Yujong Lee
cc1e8eb20f fix: capture reused worker threads in Python traces 2026-09-14 11:19:28 -07:00
Yassin Kortam
08a78a3982
Merge pull request #41054 from BerriAI/litellm_gate_correlation_contextvar_stamp
perf(logging): skip correlation contextvar stamping when request_correlation_in_logs is off
2026-09-14 11:14:44 -07:00
Shivi Jain
f4e21430c0 test(auth): freeze prefetch cache clock so the org getter cannot miss on a slow runner
Prefetch still writes org entries with the 5s getter TTL. This test only asserts the SQL join, and wall-clock expiry on CI turned that into a MagicMock await TypeError.

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-09-14 18:13:49 +00:00
Shivi Jain
75b16df6fa fix(auth): keep prefetched org entries on the 5s getter TTL
Organization mutations do not evict those cache keys, so stretching prefetch to the management TTL would leave stale org grants in L1.

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-09-14 18:13:49 +00:00
Shivi Jain
abaa2f8b81 fix(auth): put TQ008 suppressions on the patch call lines
The test-quality gate attributes the comment to the `patch(` line, so reasons on the closing paren did not count and lint failed after format started passing.

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-09-14 18:13:49 +00:00
Shivi Jain
53ba8b9866 fix(auth): load team membership once per request and skip prisma on an L1 hit
common_checks was querying get_team_membership twice, and DualCache awaited Redis SET on the auth path, so LRU eviction plus a hung Redis write showed up as two postgres spans
2026-09-14 18:13:49 +00:00
Shivi Jain
70ddc7e492 fix(auth): load team membership once per request and skip prisma on an L1 hit
common_checks was querying get_team_membership twice, and DualCache awaited Redis SET on the auth path, so LRU eviction plus a hung Redis write showed up as two postgres spans
2026-09-14 18:13:49 +00:00
Shivi Jain
5457f48290 fix(auth): load team membership once per request and skip prisma on an L1 hit
common_checks was querying get_team_membership twice, and DualCache awaited Redis SET on the auth path, so LRU eviction plus a hung Redis write showed up as two postgres spans
2026-09-14 18:13:49 +00:00
joshua-berri
fb67d8335c style(model_armor): format guardrail handler
Some checks failed
ai-gateway image / ai-gateway release image (push) Has been cancelled
LiteLLM Rust / rust-lint (push) Has been cancelled
LiteLLM Rust / rust-test (push) Has been cancelled
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-14 18:06:41 +00:00
joshua-berri
eceb67b69a test(model_armor): inject HTTP handler and token provider
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-14 18:03:22 +00:00
yucheng
4be84a859d Merge remote-tracking branch 'origin/main' into litellm_lit6314_guardrail_metadata_transfer 2026-09-14 18:01:01 +00:00
Yuneng Jiang
9b4717269f
test: preserve immutable integration observations and cleanup outcomes 2026-09-14 10:58:36 -07:00