scriptoriumadmin/litellm
1
0
Fork 0
mirror of https://github.com/BerriAI/litellm.git synced 2026-10-05 02:41:56 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 49
49274 commits 3118 branches 1436 tags 4.4 GiB
Commit graph

49274 commits

This branch
This branch
All branches
Author SHA1 Message Date
yucheng
3a3075b8a2 fix(prompt_security): poll only on queued statuses, keep 500 for terminal or missing status 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-14 22:01:39 +00:00
berriai-litellm-provider-info-sync[bot]
c3f8c07c43
chore(prices): sync Vertex AI prices: 14 models 
vertex_ai/deep-research-pro-preview-12-2025: cache_read_input_token_cost
vertex_ai/gemini-2.5-flash-image: input_cost_per_token_flex, output_cost_per_token_flex, input_cost_per_token_batches, input_cost_per_token_priority, output_cost_per_token_batches
vertex_ai/gemini-3-flash-preview: input_cost_per_token_flex, output_cost_per_token_flex, input_cost_per_token_batches, output_cost_per_token_batches, cache_read_input_token_cost_flex, cache_read_input_audio_token_cost
vertex_ai/gemini-3-pro-image: input_cost_per_token_flex, output_cost_per_token_flex, cache_read_input_token_cost, input_cost_per_token_priority, output_cost_per_token_priority, cache_read_input_token_cost_flex, cache_read_input_token_cost_priority, input_cost_per_token_above_200k_tokens, output_cost_per_token_above_200k_tokens, cache_read_input_token_cost_above_200k_tokens, input_cost_per_token_above_200k_tokens_priority, output_cost_per_token_above_200k_tokens_priority, cache_read_input_token_cost_above_200k_tokens_priority
vertex_ai/gemini-3.1-flash-image: input_cost_per_token_flex, output_cost_per_token_flex, cache_read_input_token_cost, input_cost_per_token_batches, output_cost_per_token_batches, cache_read_input_token_cost_flex
vertex_ai/gemini-3.1-flash-lite: cache_read_input_audio_token_cost
vertex_ai/gemini-3.1-flash-lite-image: input_cost_per_token_flex, output_cost_per_token_flex, cache_read_input_token_cost_flex
vertex_ai/gemini-3.1-pro-preview: input_cost_per_token_flex, output_cost_per_token_flex, cache_read_input_token_cost_flex
vertex_ai/gemini-3.5-flash: 
vertex_ai/gemini-3.5-flash-lite: cache_read_input_token_cost_priority
vertex_ai/gemini-3.6-flash: 
vertex_ai/gemini-3.7-flash: 
vertex_ai/gemini-3.8-flash: 
vertex_ai/gemini-embedding-2:
 2026-09-14 22:00:57 +00:00
Yassin Kortam
e766277846
Merge pull request #40991 from BerriAI/litellm_team_model_cooldown_siblings 
fix(router): cool down team deployments on 429 when a sibling serves the same public model
 2026-09-14 14:57:07 -07:00
Yassin Kortam
3e417fa6e0
Merge pull request #41113 from BerriAI/litellm_realtime_release_max_parallel_slot 
fix(proxy): release max_parallel_requests slot when a realtime session ends without LLM callbacks
 2026-09-14 14:56:11 -07:00
ryan-crabbe-berri
f9ba42860c fix(azure_ai): keep api_version as a property alongside the new getter 
The PLR0206 fix renamed the property to get_api_version and dropped it.
AzureFoundryModelInfo().api_version does resolve today, so removing it
breaks external callers. Restore it as a zero-arg property delegating to
the staticmethod, which satisfies PLR0206 either way.
 2026-09-14 14:56:07 -07:00
ryan
52a3b0ede7 fix(ui): trim and drop blank tags moved out of key metadata JSON 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-14 21:50:25 +00:00
yujonglee
9bbb869b13
Merge pull request #41120 from BerriAI/litellm_trace_profiler_threads 
refactor(harness): expand independent trace coverage
 2026-09-14 14:50:18 -07:00
Devin AI
e4ebeae800 fix(prices): add text output rate to Vertex TTS entries, sync gemini-embedding-2 alias 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-14 21:49:15 +00:00
Yassin Kortam
24bfd5fba1
Merge pull request #41102 from BerriAI/litellm_team_membership_once_main 
fix(auth): load team membership once per request and skip prisma on an L1 hit
 2026-09-14 14:46:06 -07:00
yassin
fbc1011d27 fix(bedrock): end the realtime session when the client disconnects instead of waiting for Nova Sonic 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-14 21:42:42 +00:00
Devin AI
f45e20e6c2 fix(prices): mark Vertex Gemini TTS entries as audio_speech 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-14 21:41:43 +00:00
kerry
abbf5aae20 chore(prices): flag gpt-5.5-cyber as a reasoning model 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-14 21:41:10 +00:00
ryan
df87a49f8b fix(ui): offer no models when an org team's ceiling never arrives 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-14 21:35:55 +00:00
yassin
b4d0f4ad26 refactor(realtime): move session ownership marker keys into constants 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-14 21:33:48 +00:00
berriai-litellm-provider-info-sync[bot]
93e65995d6
chore(prices): sync Vertex AI prices: 4 models 
gemini-2.5-flash: cache_read_input_audio_token_cost
gemini-2.5-flash-lite: cache_read_input_audio_token_cost
gemini-3-flash-preview: cache_read_input_audio_token_cost
gemini-3.1-flash-lite: cache_read_input_audio_token_cost
 2026-09-14 21:26:02 +00:00
kerry
ddf13e9505 Merge remote-tracking branch 'origin/main' into litellm-providers/price-sync 2026-09-14 21:21:55 +00:00
kerry-berri
c6e4c5582d
Merge pull request #41093 from BerriAI/litellm_fallback_backfill_opt_in_main 
feat(model_info): provider-scoped fill_missing_for_providers backfill from fallback generalization rules
 2026-09-14 14:20:36 -07:00
yucheng
26e2e208ad fix(bedrock guardrails): derive contextual grounding source and query from plain messages 
Bedrock only runs a contextualGroundingPolicy when the ApplyGuardrail payload
carries grounding_source and query qualifiers. Callers sending ordinary system
and user messages never got those, so a configured grounding threshold was
silently skipped on /v1/chat/completions and /guardrails/apply_guardrail.

When no explicit grounding_source or query tags are present, system and
developer text is sent as grounding_source and the latest user message as
query. The apply_guardrail response branch now forwards the request messages,
which it previously dropped.

Resolves LIT-4224

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-14 21:18:16 +00:00
yucheng
33fb6625ad test(guardrails): cover nameless evaluated entries in the malformed usage batch 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-14 21:18:00 +00:00
yassin
d2342f06ce fix(bedrock): stamp the realtime success ownership marker when Nova Sonic spend is logged 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-14 21:13:29 +00:00
yassin
db8dfe93a5 fix(auth): wait for the in-flight membership load before evicting its cache key on invalidation 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-14 21:10:43 +00:00
yassin
2441e8a2a9 Merge remote-tracking branch 'origin/main' into litellm_realtime_release_max_parallel_slot 2026-09-14 21:03:28 +00:00
yucheng
fb60f80c80 fix(prompt_security): keep polling file sanitization through non-terminal statuses 
Prompt Security reports a queued sanitization job as status "created" before
it moves to "in progress" and "done". The poller treated anything other than
those two known strings as an error and returned HTTP 500 on the first poll, so
every image or file request through the guardrail failed while the vendor job was
still queued.

Only "done" is terminal now. Every other status is logged and polled again until
max_poll_attempts or the outer file_sanitization_timeout, after which the existing
fail-open or fail-closed (408) policy applies.

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-14 21:03:02 +00:00
Yujong Lee
da839d4a11 fix(harness): skip unavailable Rust traces 2026-09-14 14:01:28 -07:00
Yassin Kortam
4123b4bc2b
Merge pull request #41086 from BerriAI/litellm_team_list_org_admin_membership_lit3723 
fix(proxy): keep org admins' own team memberships in other orgs visible on team list
 2026-09-14 13:57:08 -07:00
Yassin Kortam
ee03bad8c6
Merge pull request #41064 from BerriAI/litellm_bedrock_realtime_propagate_provider_failures 
fix(bedrock/realtime): propagate deferred Nova Sonic stream failures to the router
 2026-09-14 13:56:26 -07:00
yassin
ffeea30f23 test(auth): cover a stale membership write landing after a fresh reload 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-14 20:55:22 +00:00
yucheng
937179bde1 fix(guardrails): never map empty guardrail names onto a sibling id 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-14 20:53:11 +00:00
yucheng
586d51f15e fix(guardrails): skip malformed guardrail entries instead of failing the usage batch 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-14 20:50:41 +00:00
tin-berri
2cad7a49af
Merge pull request #41116 from BerriAI/litellm_auto_router_session_api_access 
fix(proxy): allow LLM API keys to read auto-router sessions
 2026-09-14 13:49:03 -07:00
yassin
abc85ba607 fix(proxy): leave the realtime max_parallel_requests slot to the limiter failure callback when a refusal was logged 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-14 20:47:37 +00:00
yassin
91c964a338 fix(auth): evict the membership cache entry when invalidation lands during the cache write 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-14 20:46:09 +00:00
yucheng
7ebb169a4d fix(guardrails): coalesce usage index rows per request and guardrail, keeping policy linkage 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-14 20:45:47 +00:00
Yassin Kortam
161ae2f3b4
Merge pull request #41107 from BerriAI/litellm_hide_credentials_hint_when_ui_password_set 
fix(proxy): hide default credentials login hint when UI_PASSWORD is set
 2026-09-14 13:44:09 -07:00
Yassin Kortam
43c3dbb312
Merge pull request #41118 from BerriAI/litellm_prometheus_proxy_provider_fallback_constant 
refactor(prometheus): source PROXY_LLM_PROVIDER_FALLBACK from litellm.constants
 2026-09-14 13:40:54 -07:00
Yassin Kortam
97ddb9494e
Merge pull request #40930 from BerriAI/litellm_flat_retry_records 
fix(router): record flat retry attempts and cap retries from attempted_retries
 2026-09-14 13:37:10 -07:00
yucheng
c3f52fe0d5 fix(guardrails): key not_run index rows by the sibling evaluation's guardrail_id 
A not_run entry from the base guardrail only carries guardrail_name, while the
content filter's evaluated entry carries guardrail_id. Keyed apart, one request
listed twice in the monitor for a logging_only guardrail (Not run and Passed).
Resolve the id from a same-name sibling in the payload so the severity pick applies

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-14 20:37:09 +00:00
Tin Chi Lo
3177c37e22 fix(cli): show routed models and session stats for LLM API keys 2026-09-14 13:36:00 -07:00
yassin
c8440b5638 fix(proxy): move credentials hint helper into a leaf html_forms module to clear CodeQL cyclic import 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-14 20:34:58 +00:00
Yassin Kortam
856aedd14a
Merge pull request #40817 from BerriAI/litellm_redis_timeout_log_throttle 
fix(redis): log a timeout streak once per interval instead of one line per cache call
 2026-09-14 13:34:45 -07:00
yassin
9a62a5ebee refactor(prometheus): source PROXY_LLM_PROVIDER_FALLBACK from litellm.constants 
Importing the fallback from litellm.proxy.hooks.rate_limiter_utils at the top
of litellm/integrations/prometheus.py closed an import cycle CodeQL flagged on
#41059 (rate_limiter_utils -> litellm -> ... -> prometheus). Hoist the constant
into litellm/constants.py so both modules read it from a leaf module.

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-14 20:29:35 +00:00
Yassin Kortam
ab09abc3c8
Merge pull request #41115 from BerriAI/litellm_fix_wrapper_async_double_sync_success_handler_main 
fix(utils): stop wrapper_async submitting the sync success handler twice
 2026-09-14 13:28:16 -07:00
Yassin Kortam
963522207d
Merge pull request #41114 from BerriAI/litellm_model_info_skip_jsonable_encoder_main 
perf(proxy): serialize /model/info listing once with orjson
 2026-09-14 13:28:06 -07:00
yassin
336ead5106 fix(proxy): import UserNotFoundError in grants resolver from the types module 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-14 20:23:54 +00:00
yassin
1b31be1a9c fix(proxy): leave the realtime max_parallel slot to the success callback when one is enqueued 
Releasing the slot unconditionally from the route raced the limiter's own
success handler on the logging worker: both could read the same stashed
acquisition before either cleared it, and under the integer in-memory
fallback that double-decrements the counter. The route now releases only on
exits without a success callback (pre-call rejection, pre-call cancellation,
and Phase 2 exits without the success stamp), matching the HTTP disconnect
path's ownership rule.

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-14 20:23:23 +00:00
yassin
4ac168b3fc fix(auth): drop in-flight membership load on invalidation so it cannot repopulate the cache 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-14 20:13:39 +00:00
yassin
d7900df73f chore(constants): drop the restating comment above the Bedrock realtime scope keys 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-14 20:13:02 +00:00
yassin
4fee71b7ad fix(proxy): define UserNotFoundError in an import-free types module so no importer sits in the CodeQL cycle 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-14 20:10:36 +00:00
yassin
a4e34d6e1b test(caching): wrap the DualCache fixture line to the 120 character limit 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-14 20:05:14 +00:00
Yassin Kortam
34fe9f71d5
Merge pull request #41059 from BerriAI/litellm_prometheus_pre_call_rate_limit_api_provider 
fix(prometheus): label pre-call rate limit failures with the resolved api_provider
 2026-09-14 13:02:07 -07:00