Commit graph

52190 commits

Author SHA1 Message Date
Yuneng Jiang
d1b160310f
fix(docker): require a generated master key in the quickstart stack
The committed sk-1234 placeholder is in PUBLICLY_KNOWN_MASTER_KEYS, so once an
image ships fe480533e8 the proxy refuses to boot and the quickstart stops
working. It also meant the documented stack came up on port 4000 with a
credential anyone could guess.

Both keys now come from .env and compose refuses to render without them. The
salt key is generated alongside so it stays stable across restarts, which
keeps stored credentials readable.

Verified: no .env -> compose fails closed naming the missing variable; with a
generated .env the stack is healthy, /v1/models returns 200 for the generated
key, 401 for sk-1234 and 401 unauthenticated, /ui/ serves, and the key still
works after a restart.
2026-09-21 14:27:29 -07:00
Yuneng Jiang
7590012a03
feat(docker): add a quickstart compose file served from the product repo
The docs quickstart pipes a compose file hosted on the docs site straight
into `docker compose -f -`. That puts the content users execute in the docs
repo rather than here, and nothing lands on disk for them to read first.

Move the two-service stack (gateway + Postgres) into docker/ so it ships and
is reviewed alongside the code it starts, and pin the image to main-stable
instead of latest. The docs change to download-then-run follows separately.

Verified: `docker compose up -d` brings the stack healthy, /health/liveliness
returns 200, /v1/models returns 200 with the placeholder key and 401 without,
and /ui/ serves.
2026-09-21 14:12:55 -07:00
Yassin Kortam
da1ccaec67
Merge pull request #40322 from BerriAI/litellm_lit7351_reservation_lease_renewal
fix(proxy): renew budget reservation counter TTL while the request is in flight
2026-09-21 16:07:14 -05:00
yuneng-jiang
5e0512b611
Merge pull request #42291 from BerriAI/litellm_lit7597_detach_credential
fix(proxy): detach stored credential when model editor selects None
2026-09-21 14:05:28 -07:00
kerry-berri
b5f02f72fe
Merge pull request #42028 from BerriAI/litellm_cost_shard_passthrough
test(integration): passthrough route cost cases
2026-09-21 14:04:03 -07:00
kerry-berri
89de508d6e
Merge pull request #42306 from BerriAI/litellm_fal_ai_surface_video_errors
fix(fal_ai): surface fal errors in video status and content instead of completed and generic 500
2026-09-21 13:59:04 -07:00
kerry-berri
f5f53a4cf4
Merge pull request #40429 from BerriAI/litellm_upgrade_banner_changelog_stats
feat(ui): add upgrade banner with latest release changelog stats
2026-09-21 13:56:05 -07:00
kerry
31a58836dd Merge remote-tracking branch 'origin/main' into litellm_cost_shard_passthrough 2026-09-21 20:55:20 +00:00
kerry-berri
24af8bb756
Merge pull request #42024 from BerriAI/litellm_cost_shard_audio_images
test(integration): audio, image and per-unit cost cases
2026-09-21 13:54:57 -07:00
kerry
61ea3e0226 chore: merge litellm_cost_shard_audio_images into litellm_cost_shard_passthrough
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-21 20:46:44 +00:00
kerry
0b9035b48f fix(fal_ai): handle transient result errors and inject clients
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-21 20:45:58 +00:00
kerry-berri
3597d61e6f
Merge pull request #42305 from BerriAI/litellm-providers/price-sync-openrouter
chore(prices): sync OpenRouter prices: 4 models, 3 new
2026-09-21 13:40:29 -07:00
kerry
0afb9bb587 chore: merge main into litellm_cost_shard_audio_images
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-21 20:36:48 +00:00
kerry-berri
dcb195e805
Merge pull request #42020 from BerriAI/litellm_cost_shard_embeddings_rerank
test(integration): embeddings, rerank, completions and moderations cost cases
2026-09-21 13:33:42 -07:00
kerry-berri
00c41e8cfb
Merge pull request #41999 from BerriAI/litellm_cost_shard_harness_extensions
test(integration): endpoint, breakdown component and failure support in the cost harness
2026-09-21 13:32:48 -07:00
berriai-litellm-provider-info-sync[bot]
07aadfd192
chore(prices): sync OpenRouter prices: 4 models, 3 new
openrouter/deepseek/deepseek-v4-pro: input_cost_per_token, output_cost_per_token, cache_read_input_token_cost
openrouter/xiaomi/mimo-v2.6-flash: max_tokens, supports_vision, max_input_tokens, max_output_tokens, supports_pdf_input, supports_reasoning, supports_web_search, supports_audio_input, supports_tool_choice, supports_prompt_caching, supports_response_schema, supports_function_calling, input_cost_per_token, output_cost_per_token, cache_read_input_token_cost
openrouter/xiaomi/mimo-v2.6-pro: max_tokens, supports_vision, max_input_tokens, max_output_tokens, supports_pdf_input, supports_reasoning, supports_web_search, supports_audio_input, supports_tool_choice, supports_prompt_caching, supports_response_schema, supports_function_calling, input_cost_per_token, output_cost_per_token, cache_read_input_token_cost
openrouter/xiaomi/mimo-v2.6-pro-ultraspeed: max_tokens, supports_vision, max_input_tokens, max_output_tokens, supports_pdf_input, supports_reasoning, supports_web_search, supports_audio_input, supports_tool_choice, supports_prompt_caching, supports_response_schema, supports_function_calling, input_cost_per_token, output_cost_per_token, cache_read_input_token_cost
2026-09-21 20:31:35 +00:00
kerry
eaa6936f13 fix(fal_ai): carry fal response into content errors
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-21 20:27:08 +00:00
kerry-berri
f63a85e9cc chore: merge litellm_cost_shard_audio_images into litellm_cost_shard_passthrough 2026-09-21 20:25:10 +00:00
kerry-berri
134c7111a5 chore: merge litellm_cost_shard_harness_extensions into litellm_cost_shard_audio_images 2026-09-21 20:24:37 +00:00
kerry-berri
7b1283ba18 chore: merge litellm_cost_shard_harness_extensions into litellm_cost_shard_embeddings_rerank 2026-09-21 20:24:32 +00:00
kerry
1ae2c0938a chore: merge main into litellm_cost_shard_harness_extensions
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-21 20:23:58 +00:00
Mateo Wang
662e5b6e32
Merge pull request #42284 from BerriAI/litellm_qianwen_ai_platform_rename
fix: rename the mainland China brand to Qianwen AI Platform
2026-09-21 13:18:20 -07:00
kerry
ffa1cceb11 refactor(fal_ai): share result request derivation between status fetchers
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-21 20:17:40 +00:00
kerry
a909a7908e fix(fal_ai): surface fal errors in video status and content
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-21 20:17:29 +00:00
Mateo Wang
13c604c124
Merge pull request #42296 from BerriAI/litellm_ci_smoke_test_master_key
fix(ci): let the install smoke test boot its key-less proxy config
2026-09-21 13:16:44 -07:00
kerry-berri
5216844c40
Merge pull request #42286 from BerriAI/litellm_fal_ai_minimax_h3
feat(fal_ai): add MiniMax H3 text-to-video and reference-to-video
2026-09-21 13:16:08 -07:00
kerry-berri
d79eadfe8c
Merge pull request #42297 from BerriAI/litellm-providers/price-sync-openrouter
chore(prices): sync OpenRouter prices: 1 model
2026-09-21 13:11:01 -07:00
kerry-berri
dd73c9fe3d
Merge pull request #42298 from BerriAI/litellm-providers/price-sync-aws-bedrock
chore(prices): sync AWS Bedrock prices: 1 model [enrichment failed: AWS Bedrock, 16 held]
2026-09-21 13:10:57 -07:00
kerry-berri
246a6ea54a
Merge pull request #42282 from BerriAI/litellm_fal_price_from_response_dims
fix(fal_ai): price images from the dimensions fal returns
2026-09-21 13:08:49 -07:00
kerry
9611af7817 Merge remote-tracking branch 'origin/main' into litellm_upgrade_banner_changelog_stats 2026-09-21 20:06:41 +00:00
yujonglee
18f77e96b5
Merge pull request #42196 from BerriAI/litellm_cache_static_dispatch
feat(rust): scaffold cache foundation for Python parity
2026-09-21 13:05:41 -07:00
berriai-litellm-provider-info-sync[bot]
4a825a7259
chore(prices): sync AWS Bedrock prices: 1 model [enrichment failed: AWS Bedrock, 16 held]
zai.glm-5: supports_vision, supports_audio_input, supports_response_schema
2026-09-21 20:01:35 +00:00
berriai-litellm-provider-info-sync[bot]
a2bb1f7e57
chore(prices): sync OpenRouter prices: 1 model
openrouter/deepseek/deepseek-v4-pro: input_cost_per_token, output_cost_per_token, cache_read_input_token_cost
2026-09-21 20:01:21 +00:00
yucheng-berri
42519a7680
Merge pull request #42262 from BerriAI/litellm_bedrock_batch_s3_bucket_owner
* fix(bedrock): send s3BucketOwner on batch input and output data config

Resolve s3_bucket_owner from litellm_params, then optional_params, then
AWS_S3_BUCKET_OWNER and emit it on both S3 data configs so cross-account
batch buckets pass Bedrock ownership validation. Omitted when unset

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* refactor(bedrock): build batch output config with explicit returns

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

---------

Co-authored-by: yucheng <yucheng@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-21 12:59:08 -07:00
kerry
b53f9ad658 fix(fal_ai): validate returned image dimensions
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-21 19:58:59 +00:00
joshua-berri
3549143bcd
Merge pull request #34919 from BerriAI/litellm_fix_mcp_peek_utf8_boundary
fix(mcp): handle split UTF-8 routing previews
2026-09-21 19:56:50 +00:00
mateo-berri
a8745f24a2 fix(ci): let the install smoke test boot its key-less proxy config
The install smoke test starts the proxy on test_config_no_auth.yaml, which has no master key on purpose, and #42019's boot check now refuses that, so the three installing_litellm_on_python jobs have been red on main since 2026-09-20. Pass the documented local-dev override to the proxy child so the test keeps its no-auth config and the boot check stays as it is
2026-09-21 12:56:21 -07:00
yucheng-berri
884407dad4
Merge pull request #42119 from BerriAI/litellm_default_policy_attachments
* feat(policy_engine): add default fallback policy attachments

A policy attachment with default: true applies only when no non-default
attachment matches the request, so an opt-in guardrail policy replaces the
fallback one instead of running alongside it. Supported in config.yaml,
/policies/attachments, the Admin UI Attachments tab and the resolver
(matched_via is prefixed with default:).

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(policy_engine): ignore inapplicable non-default attachments when selecting defaults

A non-default attachment whose policy is missing or whose condition does not match the request
no longer suppresses default attachments. The impact preview marks default counts as an upper bound

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* refactor(policy_engine): accept any sequence of policy names in condition matching

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(policy_engine): resolve policies once and apply fallback semantics in get_matching_policies

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

---------

Co-authored-by: yucheng <yucheng@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-21 12:55:42 -07:00
mateo-berri
bd8df0ed8f test: keep the Qwen brand missing-key tests off the network and split the ChatGPT fields test back out 2026-09-21 12:54:23 -07:00
Mateo Wang
2e35ae1065
Merge pull request #42049 from BerriAI/litellm_mantle_native_anthropic_messages
feat(bedrock_mantle): serve /v1/messages for Claude models on Mantle's native Anthropic Messages API
2026-09-21 12:52:16 -07:00
Yujong Lee
991108cc48 refactor(rust): align cache foundation with Python backends 2026-09-21 12:50:21 -07:00
yuneng
d69eb7f095 fix(proxy): fall back to the credentials table when validating litellm_credential_name
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-21 19:47:27 +00:00
tin-berri
457b01e96d
Merge pull request #42055 from BerriAI/litellm_prompt_caching_request_table
feat(ui): show prompt caching requests and net savings
2026-09-21 12:45:11 -07:00
Mateo Wang
e8d97d381a
Merge pull request #42080 from BerriAI/litellm_prompt_caching_affinity_lookback
fix(router): keep prompt caching affinity when the breakpoint moves
2026-09-21 12:44:34 -07:00
kerry-berri
c3663aaac8
Merge pull request #42289 from BerriAI/litellm-providers/price-sync-openrouter
chore(prices): sync OpenRouter prices: 1 model
2026-09-21 12:41:02 -07:00
kerry-berri
2692a275fa
Merge pull request #42290 from BerriAI/litellm-providers/price-sync-aws-bedrock
chore(prices): sync AWS Bedrock prices: 3 models [enrichment failed: AWS Bedrock, 18 held]
2026-09-21 12:40:58 -07:00
joshua-berri
211ff96943
Merge pull request #39189 from BerriAI/litellm_mcp_list_pagination_lit5594
fix(mcp): paginate prompt and resource discovery
2026-09-21 19:35:17 +00:00
Mateo Wang
0e7cf5113e
Merge pull request #42069 from BerriAI/litellm_redacted_thinking_prompt_caching_pin
fix(token_counter): count replayed redacted_thinking blocks so prompt_caching keeps pinning
2026-09-21 12:35:14 -07:00
berriai-litellm-provider-info-sync[bot]
f851e6ddb7
chore(prices): sync AWS Bedrock prices: 3 models [enrichment failed: AWS Bedrock, 18 held]
us.moonshotai.kimi-k3: 
zai.glm-4.7: max_tokens, supports_vision, max_input_tokens, max_output_tokens, supports_audio_input, supports_response_schema
zai.glm-4.7-flash: max_tokens, supports_vision, max_input_tokens, max_output_tokens, supports_audio_input, supports_response_schema
2026-09-21 19:31:34 +00:00
berriai-litellm-provider-info-sync[bot]
cbf5bb5e71
chore(prices): sync OpenRouter prices: 1 model
openrouter/deepseek/deepseek-v4-pro: input_cost_per_token, output_cost_per_token, cache_read_input_token_cost
2026-09-21 19:31:29 +00:00