scriptoriumadmin/litellm
1
0
Fork 0
mirror of https://github.com/BerriAI/litellm.git synced 2026-09-28 01:32:17 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 43
53121 commits 2896 branches 1423 tags 3.7 GiB
Commit graph

53121 commits

This branch
This branch
All branches
Author SHA1 Message Date
kerry
7be0bf8be6 Merge remote-tracking branch 'origin/litellm_cost_shard_pricing_dimensions' into litellm_cost_shard_provider_wires 2026-09-21 20:55:49 +00:00
kerry
36a21f7215 Merge remote-tracking branch 'origin/litellm_cost_shard_passthrough' into litellm_cost_shard_pricing_dimensions 2026-09-21 20:55:41 +00:00
kerry
31a58836dd Merge remote-tracking branch 'origin/main' into litellm_cost_shard_passthrough 2026-09-21 20:55:20 +00:00
kerry-berri
24af8bb756
Merge pull request #42024 from BerriAI/litellm_cost_shard_audio_images 
test(integration): audio, image and per-unit cost cases
 2026-09-21 13:54:57 -07:00
Yujong Lee
4a0151f77f test(rust): add redis-semantic native parity fixtures 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-21 20:52:28 +00:00
Yujong Lee
25af094e27 fix(python-bridge): allow instance attributes to shadow class defaults 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-21 20:52:28 +00:00
Yuneng Jiang
f998ab53d5
fix(proxy): treat a config-owned alerting_args as shadowing the stored row 
When the config file declares alerting_args at all, the resolver hands the
file's dict to every reader and the stored row never reaches one. Reporting
a nested field as "db" because the row happens to carry it told the admin a
value was in effect that the proxy does not serve: a live proxy answered
source "db" for outage_alert_ttl while serving the default.

A config-owned parent now reports the field's own default, and the DB is
consulted only when the file leaves the parent alone.
 2026-09-21 13:52:16 -07:00
Yujong Lee
cf8a211048 test(rust): pin disk counter restart after a fractional value 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-21 20:52:03 +00:00
mateo-berri
d8ce49de06 fix(proxy): evict the cached user row when SCIM or /user/delete removes a user 2026-09-21 13:51:26 -07:00
mateo-berri
e5a9b5c113 fix(alerting): deliver every distinct alert queued in one flush window 2026-09-21 13:51:03 -07:00
kerry
6f6b525d7a chore: merge litellm_cost_shard_proxy_behaviour into litellm_cost_shard_batches_realtime 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-21 20:50:46 +00:00
kerry
94b8842928 chore: merge litellm_cost_shard_provider_wires into litellm_cost_shard_proxy_behaviour 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-21 20:50:28 +00:00
kerry
edc34a2921 chore: merge litellm_cost_shard_pricing_dimensions into litellm_cost_shard_provider_wires 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-21 20:49:55 +00:00
Yujong Lee
83d4ce05c9 feat(rust-cache): serve RedisClusterCache natively as a Redis topology 
Extend cache-redis so RedisTopology::Cluster routes single-key commands by
hash slot, groups pipelines by slot while keeping reply order, scans and
scoped-flushes every primary, and fans admin commands out to all nodes.
The bridge projects RedisClusterCache startup_nodes into the typed topology,
accepts the exact RedisClusterCache identity, guards nodes_manager state,
and falls back to Python for anything it cannot project.

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-21 20:49:21 +00:00
kerry
19e6ad5b1b test(integration): give perplexity sonar-next a unique input rate after merge 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-21 20:48:40 +00:00
Yuneng Jiang
a45be4f276
fix(proxy): let the config file win when reporting nested alerting sources 
_nested_setting_source returned "db" whenever the stored row held a value,
without first asking whether the config file declares the same key. For a
config-owned alerting_args field that disagrees with the database, the
endpoint reported source "db" while the proxy actually serves the file's
value and rejects any write to it.

Config ownership is now checked first, matching SettingsStore.source and
the precedence the rest of the resolver applies. The source test set grows
a field that only the database sets, a field only the file sets, and a
stored empty list, so each reported source is discriminating.
 2026-09-21 13:48:37 -07:00
mateo-berri
5f84e83316 chore(responses): drop narrative comments from the MCP streaming iterator 2026-09-21 13:48:11 -07:00
kerry
7ca6fda118 chore: merge litellm_cost_shard_passthrough into litellm_cost_shard_pricing_dimensions 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-21 20:47:34 +00:00
kerry
61ea3e0226 chore: merge litellm_cost_shard_audio_images into litellm_cost_shard_passthrough 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-21 20:46:44 +00:00
kerry
0b9035b48f fix(fal_ai): handle transient result errors and inject clients 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-21 20:45:58 +00:00
mateo-berri
b8c793b962 fix(types): drop restating docstrings, import search tool types at runtime, close the spend-table match 2026-09-21 13:44:58 -07:00
yassin
930c7e2638 fix(utils): drop model from extra_body for openai-compatible providers 
A caller-supplied extra_body model overrode the authorized model in the request the shared HTTP handler sends upstream. Strip it before dispatch

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-21 20:43:08 +00:00
Yujong Lee
c7458bf5fe refactor(rust): build vault requests without local reassignment 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-21 20:42:49 +00:00
Yujong Lee
56237af7a9 test(cache): add Valkey semantic contract coverage 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-21 20:42:41 +00:00
yucheng
b41edb57c1 fix(bedrock): keep batch S3 credentials out of chat requests and debug logs 
Register s3_access_key_id, s3_secret_access_key and s3_encryption_key_id as
LiteLLM-owned batch params so they are no longer forwarded to Bedrock as
additionalModelRequestFields (which 400s ordinary chat on a batch-configured
deployment), keep them on CredentialLiteLLMParams so the batch/file paths
still receive them, and redact the S3 credential key names in debug logs.

Resolves LIT-8290

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-21 20:42:04 +00:00
kerry-berri
3597d61e6f
Merge pull request #42305 from BerriAI/litellm-providers/price-sync-openrouter 
chore(prices): sync OpenRouter prices: 4 models, 3 new
 2026-09-21 13:40:29 -07:00
Yujong Lee
605729a194 fix(rust): pin aws-sdk-s3 without relaxing eventstream pins 
Keep the exact eventstream pins but move them to =0.61.4 so aws-runtime 1.9.4's eventstream requirement resolves, downgrade aws-sdk-s3 to 1.146.1, and echo the context ttl from S3Cache::get_ttl to match Python

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-21 20:39:17 +00:00
Yujong Lee
10b977fe29 feat(python-bridge): serve redis-semantic caches natively 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-21 20:38:33 +00:00
Yujong Lee
a6d1497932 test(rust): add GCS native cache parity fixtures 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-21 20:38:20 +00:00
Yujong Lee
df89665919 test: narrow Azure parity exception assertion 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-21 20:37:57 +00:00
Yujong Lee
94b8bf7fd6 feat(rust): add native disk cache backend 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-21 20:37:26 +00:00
Yujong Lee
7d597b2dd4 fix(rust): coalesce CyberArk authentication 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-21 20:36:59 +00:00
kerry
0afb9bb587 chore: merge main into litellm_cost_shard_audio_images 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-21 20:36:48 +00:00
Yujong Lee
bfb4a8a2b3 refactor(rust): build Azure Key Vault auth inputs directly and keep credential tests offline 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-21 20:36:10 +00:00
Yujong Lee
788e24655a feat(rust): add native S3 cache backend 
Mirror the Redis vertical slice for S3Cache: a litellm-cache-s3 crate built on aws-sdk-s3 with path-style custom endpoints, python-identical put_object metadata (cache-control, expires, content headers), expires-aware get_object, and no-op flush/unsupported test_connection. Wire it through python-bridge config projection, facade guards, test handle, and binding dispatch, plus an in-process S3 stub and parity tests.

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-21 20:35:08 +00:00
Yujong Lee
ac1c4a399f refactor(cache-redis-semantic): drop the unused default index name 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-21 20:34:57 +00:00
Yujong Lee
c311073a17 feat(cache-redis-semantic): expose backend accessors for bridge binding 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-21 20:34:51 +00:00
Yujong Lee
a69ebb7ac4 feat(cache): add the unsupported operation error 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-21 20:34:39 +00:00
Yujong Lee
f6db876a3d test(cache): disambiguate Valkey semantic test module 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-21 20:34:21 +00:00
kerry-berri
dcb195e805
Merge pull request #42020 from BerriAI/litellm_cost_shard_embeddings_rerank 
test(integration): embeddings, rerank, completions and moderations cost cases
 2026-09-21 13:33:42 -07:00
Yujong Lee
4db3481244 feat(python-bridge): serve ValkeySemanticCache natively 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-21 20:33:32 +00:00
kerry-berri
00c41e8cfb
Merge pull request #41999 from BerriAI/litellm_cost_shard_harness_extensions 
test(integration): endpoint, breakdown component and failure support in the cost harness
 2026-09-21 13:32:48 -07:00
berriai-litellm-provider-info-sync[bot]
07aadfd192
chore(prices): sync OpenRouter prices: 4 models, 3 new 
openrouter/deepseek/deepseek-v4-pro: input_cost_per_token, output_cost_per_token, cache_read_input_token_cost
openrouter/xiaomi/mimo-v2.6-flash: max_tokens, supports_vision, max_input_tokens, max_output_tokens, supports_pdf_input, supports_reasoning, supports_web_search, supports_audio_input, supports_tool_choice, supports_prompt_caching, supports_response_schema, supports_function_calling, input_cost_per_token, output_cost_per_token, cache_read_input_token_cost
openrouter/xiaomi/mimo-v2.6-pro: max_tokens, supports_vision, max_input_tokens, max_output_tokens, supports_pdf_input, supports_reasoning, supports_web_search, supports_audio_input, supports_tool_choice, supports_prompt_caching, supports_response_schema, supports_function_calling, input_cost_per_token, output_cost_per_token, cache_read_input_token_cost
openrouter/xiaomi/mimo-v2.6-pro-ultraspeed: max_tokens, supports_vision, max_input_tokens, max_output_tokens, supports_pdf_input, supports_reasoning, supports_web_search, supports_audio_input, supports_tool_choice, supports_prompt_caching, supports_response_schema, supports_function_calling, input_cost_per_token, output_cost_per_token, cache_read_input_token_cost
 2026-09-21 20:31:35 +00:00
Yujong Lee
ae69a8c79a feat(rust): add Azure Key Vault secret manager backend 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-21 20:30:48 +00:00
mateo-berri
6b8e988ff0 test(e2e): settle for the replica propagation window and trim the disconnect cell's prose 2026-09-21 13:30:25 -07:00
Yujong Lee
6ab121a3e7 feat(cache-redis-semantic): add native Redis Semantic cache backend 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-21 20:29:49 +00:00
Yujong Lee
beba2576be fix(rust): align vault namespace handling with python and drop unused settings hook 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-21 20:29:06 +00:00
Yujong Lee
ca31149040 feat(rust): add native GCS object-store cache backend 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-21 20:28:27 +00:00
kerry
eaa6936f13 fix(fal_ai): carry fal response into content errors 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-21 20:27:08 +00:00
Yujong Lee
3ba4a60d5e feat(rust): add HashiCorp Vault secret manager crate 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-21 20:26:28 +00:00