scriptoriumadmin/litellm
1
0
Fork 0
mirror of https://github.com/BerriAI/litellm.git synced 2026-09-17 23:51:30 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 67
litellm/litellm
History
Abhimanyu Kapur 43265a5292 fix: three review findings — cap bypass, start race, request-path DB read 
Zero-estimate cap bypass (cursor): a key quiet during the estimate
lookback gets cost_estimate 0.0, which the spend cap treated the same
as 'no estimate' and left uncapped — a later traffic spike on exactly
that job would bill until ends_at. Only a NULL estimate (rows predating
estimates) is uncapped now; $0 still gets the $1 floor.

Concurrent-start race (cursor): the find_first-then-create check passes
on both sides of a race, giving one key two active jobs and double
judge spend. A partial unique index (api_key_id WHERE status IN
(pending, running)) — raw SQL in the unshipped migration, since
schema.prisma cannot express partial indexes — makes the DB the
arbiter; the losing create surfaces as the same 409 as the advisory
check. The old find_many('desc') + reversed() insertion in the logger
cache already prefers the newest job for any legacy duplicates.

Request-path DB read (greptile): an expired job snapshot awaited
find_many inside the success callback, so every N seconds one request
per pod paid a synchronous Prisma read. The lookup is now sync-only:
it serves the current snapshot and kicks a detached refresh task when
stale. Cost: a cold pod's first ~1 refresh-window of samples are
missed (acceptable for a sampled eval); stale-if-error semantics keep
a DB blip from disabling the feature.

Also from review: a collapsed previous-job row said 'no verdicts' for
jobs with thousands of verdicts, because the list endpoint omits
results by design — it now says 'view results' when completed_count>0.

Co-Authored-By: Claude <noreply@anthropic.com>
2026-08-08 17:02:49 -07:00
..
a2a_protocol chore(typing): clear 1.4k basedpyright Any errors across 21 hotspot files 2026-08-08 08:14:29 -07:00
anthropic_interface chore(lint): strip inert type: ignore comments and zero LIT009, LIT010, LIT011 headroom 2026-08-05 02:37:24 -07:00
assistants chore(lint): clear grandfathered over-limit lint drift and ratchet budgets down 2026-08-05 12:18:13 -07:00
batch_completion feat(lint): enforce Final on locals and freeze function parameters (LIT010, LIT011) 2026-08-04 12:54:39 -07:00
batches Merge pull request #35928 from BerriAI/litellm_zero_ruff_lit_headroom 2026-08-05 09:46:05 -07:00
caching refactor(types): replace implicit and explicit Any across 11 modules 2026-08-06 03:34:06 +00:00
completion_extras chore(lint): clear grandfathered over-limit lint drift and ratchet budgets down 2026-08-05 12:18:13 -07:00
compression chore(lint): strip inert type: ignore comments and zero LIT009, LIT010, LIT011 headroom 2026-08-05 02:37:24 -07:00
containers chore(lint): clear grandfathered over-limit lint drift and ratchet budgets down 2026-08-05 12:18:13 -07:00
endpoints/speech/speech_to_completion_bridge feat(lint): enforce Final on locals and freeze function parameters (LIT010, LIT011) 2026-08-04 12:54:39 -07:00
evals chore(lint): strip inert type: ignore comments and zero LIT009, LIT010, LIT011 headroom 2026-08-05 02:37:24 -07:00
experimental_mcp_client chore(lint): strip inert type: ignore comments and zero LIT009, LIT010, LIT011 headroom 2026-08-05 02:37:24 -07:00
files chore(lint): strip inert type: ignore comments and zero LIT009, LIT010, LIT011 headroom 2026-08-05 02:37:24 -07:00
fine_tuning Merge branch 'litellm_internal_staging' into litellm_remove_types_ruff_exclusion 2026-08-05 11:05:33 -07:00
google_genai Merge remote-tracking branch 'origin/litellm_internal_staging' into litellm_daily_any_cleanup_08_04_2026 2026-08-05 12:13:11 -07:00
images chore(lint): strip inert type: ignore comments and zero LIT009, LIT010, LIT011 headroom 2026-08-05 02:37:24 -07:00
integrations fix: three review findings — cap bypass, start race, request-path DB read 2026-08-08 17:02:49 -07:00
interactions chore(lint): clear grandfathered over-limit lint drift and ratchet budgets down 2026-08-05 12:18:13 -07:00
litellm_core_utils Merge remote-tracking branch 'origin/litellm_internal_staging' into shadow-eval-pre-adoption 2026-08-08 11:54:47 -07:00
llms chore(typing): clear 1.4k basedpyright Any errors across 21 hotspot files 2026-08-08 08:14:29 -07:00
models chore(lint): strip inert type: ignore comments and zero LIT009, LIT010, LIT011 headroom 2026-08-05 02:37:24 -07:00
ocr feat(lint): enforce Final on locals and freeze function parameters (LIT010, LIT011) 2026-08-04 12:54:39 -07:00
passthrough chore(lint): strip inert type: ignore comments and zero LIT009, LIT010, LIT011 headroom 2026-08-05 02:37:24 -07:00
proxy fix: three review findings — cap bypass, start race, request-path DB read 2026-08-08 17:02:49 -07:00
proxy_auth feat(lint): enforce Final on locals and freeze function parameters (LIT010, LIT011) 2026-08-04 12:54:39 -07:00
rag chore(lint): strip inert type: ignore comments and zero LIT009, LIT010, LIT011 headroom 2026-08-05 02:37:24 -07:00
realtime_api chore(lint): strip inert type: ignore comments and zero LIT009, LIT010, LIT011 headroom 2026-08-05 02:37:24 -07:00
repositories fix(proxy): derive config agent ids from agent_name so grants survive secret rotation (#36020) 2026-08-07 19:19:49 +00:00
rerank_api chore(lint): strip inert type: ignore comments and zero LIT009, LIT010, LIT011 headroom 2026-08-05 02:37:24 -07:00
responses chore(typing): clear 1.4k basedpyright Any errors across 21 hotspot files 2026-08-08 08:14:29 -07:00
router_strategy Merge remote-tracking branch 'origin/litellm_internal_staging' into shadow-eval-pre-adoption 2026-08-08 13:17:18 -07:00
router_utils feat(router): independent, default-on deployment affinity for the auto-router (#36146) 2026-08-08 13:02:29 -07:00
rust_bridge feat(lint): enforce Final on locals and freeze function parameters (LIT010, LIT011) 2026-08-04 12:54:39 -07:00
sandbox feat(lint): enforce Final on locals and freeze function parameters (LIT010, LIT011) 2026-08-04 12:54:39 -07:00
search chore(lint): strip inert type: ignore comments and zero LIT009, LIT010, LIT011 headroom 2026-08-05 02:37:24 -07:00
secret_managers chore(lint): strip inert type: ignore comments and zero LIT009, LIT010, LIT011 headroom 2026-08-05 02:37:24 -07:00
skills chore(lint): strip inert type: ignore comments and zero LIT009, LIT010, LIT011 headroom 2026-08-05 02:37:24 -07:00
types Merge remote-tracking branch 'origin/litellm_internal_staging' into shadow-eval-pre-adoption 2026-08-08 13:17:18 -07:00
vector_store_files chore(lint): clear grandfathered over-limit lint drift and ratchet budgets down 2026-08-05 12:18:13 -07:00
vector_stores chore(lint): strip inert type: ignore comments and zero LIT009, LIT010, LIT011 headroom 2026-08-05 02:37:24 -07:00
videos chore(lint): strip inert type: ignore comments and zero LIT009, LIT010, LIT011 headroom 2026-08-05 02:37:24 -07:00
__init__.py Merge pull request #36024 from BerriAI/litellm_anthropic_sse_keepalive 2026-08-05 19:41:35 -07:00
_internal_context.py feat(lint): enforce Final on locals and freeze function parameters (LIT010, LIT011) 2026-08-04 12:54:39 -07:00
_lazy_imports.py fix(lint): bring basedpyright rule counts back under their budget limits 2026-08-05 10:23:02 -07:00
_lazy_imports_registry.py chore(lint): clear grandfathered over-limit lint drift and ratchet budgets down 2026-08-05 12:18:13 -07:00
_logging.py fix(logging): extend secret redaction to records litellm does not emit directly (#35977) 2026-08-05 14:50:42 -07:00
_redis.py Merge remote-tracking branch 'origin/litellm_internal_staging' into litellm_daily_any_cleanup_08_04_2026 2026-08-05 12:13:11 -07:00
_redis_credential_provider.py chore(lint): strip inert type: ignore comments and zero LIT009, LIT010, LIT011 headroom 2026-08-05 02:37:24 -07:00
_service_logger.py chore(lint): clear grandfathered over-limit lint drift and ratchet budgets down 2026-08-05 12:18:13 -07:00
_uuid.py chore(lint): strip inert type: ignore comments and zero LIT009, LIT010, LIT011 headroom 2026-08-05 02:37:24 -07:00
_version.py
anthropic_beta_headers_config.json fix(bedrock-invoke): retain clear_tool_uses_20250919 context_management edits and emit context-management-2025-06-27 beta (LIT-3393) (#32658) 2026-07-09 14:31:27 -07:00
anthropic_beta_headers_manager.py feat(lint): enforce Final on locals and freeze function parameters (LIT010, LIT011) 2026-08-04 12:54:39 -07:00
blog_posts.json docs(blog): add WebRTC blog post link 2026-03-13 17:54:33 +05:30
budget_manager.py feat(lint): enforce Final on locals and freeze function parameters (LIT010, LIT011) 2026-08-04 12:54:39 -07:00
constants.py feat(router): independent, default-on deployment affinity for the auto-router (#36146) 2026-08-08 13:02:29 -07:00
cost.json
cost_calculator.py Merge remote-tracking branch 'origin/litellm_internal_staging' into litellm_zero_ruff_lit_headroom 2026-08-05 02:53:06 -07:00
exceptions.py chore(lint): strip inert type: ignore comments and zero LIT009, LIT010, LIT011 headroom 2026-08-05 02:37:24 -07:00
main.py fix(router): keep custom model_info across a price data reload (#35491) 2026-08-05 19:56:50 +00:00
model_prices_and_context_window_backup.json Merge pull request #35923 from BerriAI/litellm_dated_variant_tier_pricing_sync 2026-08-05 21:09:37 -07:00
policy_templates_backup.json feat(add-new-block_code_execution-guardrail): prevent agent from executing code (#22154) 2026-02-25 22:02:14 -08:00
provider_endpoints_support_backup.json chore: litellm oss staging (#30968) 2026-06-23 07:31:44 -07:00
py.typed
router.py feat(router): independent, default-on deployment affinity for the auto-router (#36146) 2026-08-08 13:02:29 -07:00
scheduler.py feat(lint): enforce Final on locals and freeze function parameters (LIT010, LIT011) 2026-08-04 12:54:39 -07:00
setup_wizard.py chore(lint): strip inert type: ignore comments and zero LIT009, LIT010, LIT011 headroom 2026-08-05 02:37:24 -07:00
timeout.py feat(lint): enforce Final on locals and freeze function parameters (LIT010, LIT011) 2026-08-04 12:54:39 -07:00
utils.py fix(lint): make strict-gate noqas survive base ruff and flag stale ones 2026-08-07 21:27:00 -07:00