48834 commits

This branch

This branch

All branches

Author	SHA1	Message	Date
mateo-berri	19da217167	fix(openai): mint workload identity tokens for PrivateLink and regional api.openai.com hosts	2026-09-03 14:38:47 -07:00
yuneng-jiang	918ada8d57	Merge pull request #39611 from BerriAI/litellm_/dazzling-ellis-c9e9ff ... test(team-race): wait on pg_locks instead of a fixed sleep	2026-09-03 14:38:41 -07:00
Mateo Wang	10aaea5fae	Merge pull request #39243 from BerriAI/litellm_fix_guardrail_event_hook_resync ... fix(guardrails): rebuild the serving worker guardrail on PUT instead of patching it in place	2026-09-03 14:37:52 -07:00
yucheng-berri	e6e5be0989	fix(cache): use sync Redis batch reads (#39358) ... * fix(cache): use sync Redis batch reads * fix(cache): type sync circuit breaker decorator * test(cache): isolate sync Redis breaker coverage * fix(cache): keep batch result merge budget compliant * style(cache): format batch read * style(cache): satisfy type-discipline budget * test(cache): mock Redis before sync breaker setup * style(cache): avoid mutable batch placeholder * test(cache): document sync breaker patch target * fix(types): widen batch result params to Sequence * fix(cache): report real callers through breaker guards The sync guard's lambda and runner frames replaced the actual caller in _get_call_stack_info, so Redis service logs attributed every guarded call to the guard machinery. Skip guard-internal frames when walking the stack and ratchet the lint budgets this branch lowered * style(imports): import Sequence from collections.abc * test(cache): cover concurrent sync and async Redis batch reads * refactor: build sync batch_get_cache results as tuples to satisfy the LIT002 gate * chore: ratchet budgets after staging merge * fix: preserve DualCache batch list contract * style: format DualCache batch result * fix: satisfy mutable collection lint gate * fix(caching): keep breaker guard-frame skipping in bytecode-only deploys * chore: preserve staging budget ratchets * test(cache): isolate sync Redis batch reads * fix(cache): isolate service hook failures * fix(cache): preserve sync batch fallback on open breaker	2026-09-03 14:37:48 -07:00
Mateo Wang	f39ffbd760	Merge pull request #39465 from BerriAI/litellm_helm_ingress_nginx_dotted_paths ... fix(helm): render ingress-nginx compatible path types via ingress.controller	2026-09-03 14:36:54 -07:00
Mateo Wang	117ef33a41	Merge pull request #39541 from BerriAI/litellm_spend_logs_bridged_streaming_message_id ... fix(anthropic_messages): key bridged streaming spend rows on the streamed msg_ id	2026-09-03 14:36:51 -07:00
Mateo Wang	3a00e8020a	Merge pull request #39464 from BerriAI/litellm_containers_error_passthrough_pagination ... fix(containers): pass upstream error status through and forward list pagination params	2026-09-03 14:36:44 -07:00
Mateo Wang	2e5a54f28c	Merge pull request #39538 from BerriAI/litellm_redact_optional_params_verbose_line ... fix(utils): redact credentials nested in extra_body on the verbose optional-params line	2026-09-03 14:36:39 -07:00
Mateo Wang	beaf2d4043	Merge pull request #39452 from BerriAI/litellm_fix_rag_query_store_credentials ... fix(rag): forward the managed vector store's params to the search call	2026-09-03 14:36:34 -07:00
Mateo Wang	c053fac7b7	Merge pull request #39466 from BerriAI/litellm_prisma_timeout_killpg ... fix(proxy-extras): kill the whole Prisma process group when a command times out	2026-09-03 14:36:31 -07:00
Mateo Wang	f7691a3d85	Merge pull request #39411 from BerriAI/litellm_bedrock_bearer_skip_sigv4_chain ... fix(bedrock): skip the SigV4 credential chain when a bearer token is configured	2026-09-03 14:36:27 -07:00
Mateo Wang	aa9f3d927b	Merge pull request #39526 from BerriAI/litellm_mask_verbose_request_kwargs ... fix(utils): redact credential kwargs from the set_verbose request line	2026-09-03 14:36:25 -07:00
Mateo Wang	00faaa17f4	Merge pull request #39495 from BerriAI/litellm_vector_store_hook_router_injection ... fix(vector-stores): survive a failing vector store search in the chat completions hook	2026-09-03 14:36:19 -07:00
Mateo Wang	a4fc936b9d	Merge pull request #39491 from BerriAI/litellm_fix_failing_request_slowdown ... fix(router): keep retry breadcrumbs per request and out of the request snapshot	2026-09-03 14:36:16 -07:00
Mateo Wang	c67fe2d3bf	Merge pull request #39521 from BerriAI/litellm_fix_guardrail_error_stringified_none ... fix(proxy): stop putting the literal string "None" in error payloads	2026-09-03 14:36:14 -07:00
Mateo Wang	8cc131ad39	Merge pull request #38984 from BerriAI/litellm_fix_search_results_with_guardrails ... fix: attach vector store search_results when a guardrail is registered	2026-09-03 14:36:09 -07:00
Mateo Wang	29ea2bd2cd	Merge pull request #39426 from BerriAI/litellm_azure_ai_grok_4_6_cost_map ... feat(azure_ai): add grok-4.6 to the model cost map	2026-09-03 14:36:07 -07:00
Mateo Wang	f5f2781c6f	Merge pull request #39543 from BerriAI/litellm_fix_guardrail_mapping_test_order_dependence ... fix(guardrails): remove the module-global translation mapping that leaked between tests	2026-09-03 14:36:03 -07:00
Mateo Wang	9eaa6e345e	Merge pull request #39510 from BerriAI/litellm_coerce_multipart_numeric_fields ... fix(proxy): parse numeric multipart fields on /v1/images/edits back into numbers	2026-09-03 14:36:00 -07:00
Mateo Wang	5e5cc54b28	Merge pull request #39496 from BerriAI/litellm_ui_build_check_image_boundary ... ci(ui): run the UI build check through the image's ui-builder stage	2026-09-03 14:35:56 -07:00
Mateo Wang	c99149ac0f	Merge pull request #39493 from BerriAI/litellm_ci_rust_server_feature ... ci(rust): build and test the ai-gateway server feature	2026-09-03 14:35:53 -07:00
Mateo Wang	0c8510d47c	Merge pull request #39511 from BerriAI/litellm_spend_logs_provider_response_id ... fix(spend_tracking): key /v1/messages spend rows on the msg_ id the client received	2026-09-03 14:35:50 -07:00
mateo-berri	2f5d9ae194	Merge remote-tracking branch 'origin/litellm_internal_staging' into litellm_fix_responses_queued_id_encryption ... # Conflicts: # type-discipline-budget.json	2026-09-03 14:29:02 -07:00
mubashir1osmani	c276813cb4	feat(batches): enrich batch cost rows with breakdown, identity, session, and org spend	2026-09-03 17:23:53 -04:00
yuneng-jiang	7d5b6456ba	Merge pull request #39648 from BerriAI/litellm_internal_staging ... chore(ci): promote internal staging to main	2026-09-03 14:21:37 -07:00
Yuneng Jiang	4a3190b68d	Merge remote-tracking branch 'origin/litellm_internal_staging' into litellm_/dazzling-ellis-c9e9ff ... # Conflicts: # tests/openai_endpoints_tests/test_e2e_openai_responses_api.py	2026-09-03 14:18:37 -07:00
mateo	00bdfe797a	fix(registry): mark gemini-3.5-live-translate-preview as realtime with official token limits ... Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>	2026-09-03 21:16:55 +00:00
Mateo Wang	8699998c9e	Merge pull request #39625 from BerriAI/litellm_lit6873_vertex_passthrough_api_version ... fix(vertex): add the API version to versionless project routes on the Vertex passthrough	2026-09-03 14:16:37 -07:00
Mateo Wang	1ea95fa18f	Merge pull request #39617 from BerriAI/litellm_fix_responses_background_stream_flake ... test(responses): bound the background stream cancel e2e so an upstream stall skips fast	2026-09-03 14:16:08 -07:00
ryan-crabbe-berri	ed9865c945	Merge pull request #39604 from BerriAI/litellm_lit_4738_users_search_by_id ... fix(ui): let the Internal Users search box match user_id as well as email	2026-09-03 14:14:22 -07:00
mateo	8d82f28c85	test(ocr): register the azure ocr4 mixed-rate cost test in the parity ledger ... Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>	2026-09-03 21:14:18 +00:00
mateo	5a3a2f3d0a	Merge remote-tracking branch 'origin/litellm_internal_staging' into litellm_registry_audit_2026_09_02	2026-09-03 21:09:30 +00:00
Mateo Wang	5a821b593c	Merge pull request #38440 from BerriAI/litellm_prompt_registry_env ... fix(prompts): key the in-memory prompt registry by environment	2026-09-03 14:09:26 -07:00
ryan	792852b4f3	test(ui): name the mocked user list body to stay within the inline-object lint budget ... Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>	2026-09-03 21:04:55 +00:00
Mateo Wang	025a3ca42f	Merge pull request #39631 from BerriAI/litellm_gpt_6_astra_detection ... fix: treat gpt-6 names as the gpt-5 request family in OpenAI and Azure configs	2026-09-03 14:00:59 -07:00
yuneng-jiang	de87b198ea	Merge pull request #39630 from BerriAI/litellm_router_get_configured_mode_test ... test(router): cover get_configured_mode so router_code_coverage passes	2026-09-03 13:57:57 -07:00
devin-ai-integration[bot]	eb6c24a2a0	fix(auto_router): bill the routing embedding to the caller's key and team (#39532) ... * fix(auto_router): bill the routing embedding to the caller's key and team Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * refactor(auto_router): validate the forwarded caller metadata with a pydantic model Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> --------- Co-authored-by: yassin <yassin@berri.ai> Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>	2026-09-03 13:53:30 -07:00
devin-ai-integration[bot]	aff6b7e212	fix(ui): clear agents when updating team permissions (#39600) ... Always serialize object_permission.agents and agent_access_groups in the team update payload so removing the last agent in the dashboard sends an explicit empty array instead of omitting the key, which the backend merge treats as no change Resolves LIT-6861 Co-authored-by: yassin <yassin@berri.ai>	2026-09-03 13:51:04 -07:00
mateo	eae7b806e3	fix(registry): point Bedrock Qwen3 Coder 480B source at the us-west-2 on-demand price list ... Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>	2026-09-03 20:49:44 +00:00
Yuneng Jiang	52e24aebba	refactor(tests): assign the streamed id and lock poll once instead of rebinding ... The cancel test accumulated chunk_count and reassigned response_id on every iteration, and the lock watcher rebound its query result on every poll. Both are the mutable-local pattern the repo avoids. The stream now drains through a generator that stops at the first chunk carrying a response id, so the caller binds streamed_ids once and reads the id off the tail. Empty stream, no-id stream and first-chunk-id all behave exactly as the loop did. The watcher inlines its poll result.	2026-09-03 13:49:43 -07:00
mateo-berri	108f558946	test: drop the internal patch from the gpt-6-astra bridge test	2026-09-03 13:46:33 -07:00
mateo-berri	e65e3d0e2b	fix(cost): bill fireworks cached tokens at the off-peak input rate when no cache-read rate exists	2026-09-03 13:45:17 -07:00
mateo-berri	e297968826	feat(cost): honor off_peak_pricing reasoning and cache-creation rates ... The block accepts output_cost_per_reasoning_token and cache_creation_input_token_cost. The generic cost path and the DashScope calculator swap them in while a window is open, and unset keys keep the standard rate. One shared TokenRates value replaces the DashScope-local copy, and apply_off_peak_pricing takes and returns it.	2026-09-03 13:45:06 -07:00
moe-berri	a264c62b04	test(router): cover configured mode lookup	2026-09-03 13:43:30 -07:00
mubashir1osmani	4e0907fb2d	test(router): use an unmapped model so get_configured_mode tests do not write into the global cost map	2026-09-03 16:40:58 -04:00
tin-berri	df73c623b2	feat(router): limit heuristic_v2 auto-routers to one without the auto_router license feature (#39468) ... Without the auto_router feature in the signed enterprise license a proxy may hold one complexity router with classifier_type heuristic_v2 across config.yaml and the DB; with it the limit is lifted. The ceiling is derived once from LicenseCheck and handed to the Router, which refuses the extra router at registration. config.yaml over the limit refuses to start, and /model/new, /model/update and PATCH /model/{id}/update refuse the write with a 403 before touching the DB. Expiry follows the existing max_users/max_teams pattern: judged when the license is verified, not on every call, and a verify that rejects the license (expired or unreadable) leaves no signed payload behind. The rollback after a failed upsert re-admits state that was already serving, so it is exempt from the ceiling: an edit that fails, including one refused by a ceiling that has since tightened, leaves the router serving its previous configuration. A write that leaves a row on heuristic_v2 under a limited license runs in one transaction that takes a Postgres advisory lock before counting the DB rows plus this proxy's config.yaml routers, so concurrent writes on any pod cannot both claim the sole slot and no surplus row is ever persisted. Only the row insert runs under that lock: the team model bookkeeping, which needs a second pool connection, runs after the transaction has committed. PATCH /model/{id}/update follows the same order as create: the row is written through the slot first and the team's model list is updated only afterwards, so a refused write leaves the team as it was. The slot transaction bypasses the repository's publish-on-write, so it publishes the config change once after commit, as delete_team_models does.	2026-09-03 13:39:58 -07:00
mateo-berri	bba75c7ce9	Merge remote-tracking branch 'origin/litellm_internal_staging' into litellm_mistral_voxtral_tts_speech ... # Conflicts: # tests/test_litellm/test_cost_calculator.py # tests/test_litellm/test_main.py	2026-09-03 13:35:30 -07:00
mateo-berri	d0ac494144	fix(cost): honor off_peak_pricing in the fireworks_ai and perplexity cost calculators	2026-09-03 13:31:56 -07:00
mateo	080e364d5e	fix(registry): carry Anthropic thinking/sampling flags on new Perplexity and OpenRouter Claude entries ... Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>	2026-09-03 20:28:48 +00:00
mateo-berri	35d3478818	fix(responses/mcp): keep reasoning order and caller previous_response_id on stateless follow-ups	2026-09-03 13:27:48 -07:00