Commit graph

45929 commits

Author SHA1 Message Date
yucheng-berri
30f3228510
test(newrelic): cover static default_team_settings per-team routing (#38857)
* test(newrelic): cover static default_team_settings per-team routing

The dynamic POST /team/{team_id}/callback path for New Relic is tested, but
the static default_team_settings twin had no regression coverage. Add a test
that drives default_team_settings -> add_team_based_callbacks_from_config and
asserts the resolved trusted vars dispatch to BOTH the per-team metrics logger
(cost/usage) and the trace logger (LLM/agent spans), so a config-file customer
gets the same per-team routing as the API customer.

Also correct the /team/callback docstring: callback_name is a str validated
against the credential-capable callbacks, not a fixed langfuse/langsmith/gcs
Literal, and document the newrelic_api_key / newrelic_region vars.

* chore(ui): sync schema.d.ts with the /team/callback docstring

Regenerate the dashboard OpenAPI types for the add_team_callbacks description
change: callback_name is a validated str (not a langfuse/langsmith/gcs
Literal) and the newrelic_api_key / newrelic_region vars are documented.

* docs(newrelic): note LITELLM_OTEL_V2 prerequisite, trim test comments

Address review: team-scoped New Relic config is rejected with a 400 unless the
proxy runs with LITELLM_OTEL_V2=true, so document that in the /team/callback
endpoint and sync schema.d.ts. Drop the narrative setup comments in the new
test per the repo comment convention; the test name and docstring already say why.
2026-08-31 16:58:35 +00:00
mateo-berri
39473745dd fix(docker): bump wolfi-base for glibc 2.44 and pin apk python to 3.13 in migrations image 2026-08-31 09:53:01 -07:00
mateo-berri
6e59ce1773 Merge remote-tracking branch 'origin/litellm_internal_staging' into litellm_pr36008_head
# Conflicts:
#	basedpyright-code-budget.json
#	type-discipline-budget.json
2026-08-31 09:41:15 -07:00
Mateo Wang
d6cce13308
Merge pull request #38772 from BerriAI/litellm_claude_md_test_behavior_not_structure
docs(claude.md): require tests to check behavior, not code structure
2026-08-31 09:32:48 -07:00
nuernber
2f50988fed fix(anthropic): resolve TRY300 lint violation and ratchet budgets after litellm_internal_staging merge 2026-08-31 09:30:29 -07:00
devin-ai-integration[bot]
f079e4061b
fix(proxy): deliver budget alerts on webhook-only alerting and accept ALERTING_WEBHOOK_URL (#38441)
* fix(proxy): deliver budget alerts on webhook-only alerting and accept ALERTING_WEBHOOK_URL

ProxyLogging.budget_alerts forwarded to the alerting pipeline only when
'slack' was in general_settings.alerting, so alerting: ['webhook'] plus
WEBHOOK_URL silently never delivered a budget alert (the config
/health/services?service=webhook exists to test). Forward when 'webhook'
is present too; SlackAlerting.send_alert already fans out per channel.

Also accept a provider-neutral ALERTING_WEBHOOK_URL env fallback for the
Slack-format channel (any Slack-compatible receiver works), mark it as a
sensitive var, and de-brand the admin UI alerting copy.

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(ui): format settings.tsx with prettier

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* chore(ui): regenerate schema.d.ts for updated alerting description

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* ci: retrigger checks after ALERTING_WEBHOOK_URL docs merged

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

---------

Co-authored-by: yassin <yassin@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-08-31 09:22:36 -07:00
nuernber
b5affa63aa Merge remote-tracking branch 'origin/litellm_internal_staging' into litellm_bedrock_messages_disconnect_billing
# Conflicts:
#	basedpyright-code-budget.json
#	type-discipline-budget.json
2026-08-31 09:22:00 -07:00
nuernber
8e92f98905 docs(constants): clarify which env var the cap=0 fallback note applies to 2026-08-31 09:18:12 -07:00
nuernber
79fd2f4872 test(anthropic): cover ANTHROPIC_MESSAGES_MAX_DETACHED_STREAM_DRAINS=0 fallback to partial billing 2026-08-31 09:15:36 -07:00
nuernber
e1fece511a test(anthropic): fix PT012 lint violation in upstream-error regression test 2026-08-31 09:11:51 -07:00
nuernber
b2df72f980 chore: ratchet down lint/type budgets after disconnect-billing merge fix 2026-08-31 09:05:37 -07:00
Mateo Wang
32291c9ad2
Merge pull request #38942 from BerriAI/litellm_lite_claude_enable_tool_search-8eee
feat(cli): set ENABLE_TOOL_SEARCH=true for lite claude
2026-08-31 09:02:48 -07:00
nuernber
95a2586228 Merge remote-tracking branch 'origin/main' into litellm_bedrock_messages_disconnect_billing
# Conflicts:
#	basedpyright-code-budget.json
#	litellm/llms/anthropic/experimental_pass_through/messages/streaming_iterator.py
#	tests/test_litellm/llms/anthropic/experimental_pass_through/messages/test_streaming_iterator.py
#	type-discipline-budget.json
2026-08-31 08:58:37 -07:00
Devin AI
4291afbfa5 fix(registry): correct OpenAI preview shutdown dates, add whisper/transcribe and Bedrock/Vertex deprecation dates
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-08-31 13:08:06 +00:00
Devin AI
f111262e54 Merge remote-tracking branch 'origin/litellm_internal_staging' into devin_ai_1788036971-stale-cost-map-sources 2026-08-31 13:03:35 +00:00
Devin AI
69fc449666 chore(techdebt): drop restating comments from the 2026-08-30 window
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-08-31 07:54:31 +00:00
Devin AI
f3d0da10d1 chore: merge litellm_internal_staging into litellm_techdebt_20260830
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-08-31 07:46:09 +00:00
Cursor Agent
43c64e2471
fix(cli): keep an existing ENABLE_TOOL_SEARCH value
Default remains true so lite claude turns tool search back on through
a proxy. An explicit false or auto in the env or settings is left alone

Co-authored-by: Mateo Wang <mateo-berri@users.noreply.github.com>
2026-08-31 06:00:45 +00:00
Cursor Agent
096e016baf
fix(cli): sort claude_settings __all__ for ruff RUF022
Co-authored-by: Mateo Wang <mateo-berri@users.noreply.github.com>
2026-08-31 05:58:00 +00:00
Cursor Agent
cafdfda8ba
feat(cli): set ENABLE_TOOL_SEARCH=true for lite claude
Claude Code turns tool search off when ANTHROPIC_BASE_URL is a proxy.
lite claude, lite up, login --config-claude, and autoroute now force
ENABLE_TOOL_SEARCH=true so MCP tools stay deferred through the proxy

Co-authored-by: Mateo Wang <mateo-berri@users.noreply.github.com>
2026-08-31 05:52:17 +00:00
feng.tsai
dc034a086a docs: trim the _image_sources docstring
It restated the source union that the type definition already carries.
2026-08-31 13:26:02 +08:00
feng.tsai
bb51c121cf docs: reference the source union by type instead of a line number
The line number went stale when the base moved.
2026-08-31 12:21:24 +08:00
samtsai15
0e7562dbc6 test(guardrails): cover every Anthropic image source shape in the extractor's own suite
_image_sources had no test asserting what it extracts. The existing image tests
live on the Bedrock side and all use base64 without a media_type, which is the one
path the fix left unchanged, so both behaviors it does change went unverified: the
url shape reaching the guardrail at all, and base64 arriving as a data URI.

Against the pre-fix extractor the url case sees [] and the media_type case sees
['AAAA'] instead of ['data:image/png;base64,AAAA'].

The remaining three assert behavior the fix deliberately preserves -- bare base64
passed through, a file source yielding nothing, a malformed source dropped rather
than handed on for a consumer to choke on.

Each message carries a text block because a message with no text never reaches the
guardrail, which would make every source shape look equally dropped.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-31 10:37:51 +08:00
samtsai15
1f80f93750 fix(guardrails): carry Anthropic url and file image sources through to guardrails
_image_sources returned source["data"] only. An Anthropic image block has three
shapes (types/llms/anthropic.py:259) and only the base64 one carries "data", so
{"type": "url", "url": ...} yielded nothing and the image never reached any
guardrail at all.

This is not Bedrock-specific. Five guardrails consume
GenericGuardrailAPIInputs["images"] (vigil_guard, custom_code, deepkeep, straiker,
generic_guardrail_api) and every one of them was blind to url sources on
/v1/messages.

base64 now returns a data URI rather than the bare payload. A consumer otherwise
has no way to recover media_type, and an API like Bedrock's ApplyGuardrail needs
the format to build its request.

The file shape stays unresolvable here: the bytes live behind the Files API and
this extractor has no client to fetch them. Documented rather than silently
dropped, so a consumer treating a missing entry as "no image to scan" is a known
gap and not a surprise.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-31 10:37:51 +08:00
mateo-berri
de1f38820a fix(passthrough): flush interrupted streams on client disconnect and reuse cached gigachat http clients 2026-08-30 13:36:51 -07:00
mateo-berri
14f392bb9b fix(docker): bump wolfi-base for glibc 2.44 and pin apk python to 3.13 2026-08-30 13:11:32 -07:00
mateo-berri
a5fa8ebfa7 fix(passthrough): keep upstream error body readable for streaming error status mapping 2026-08-30 12:59:18 -07:00
mateo-berri
611750cd11 Merge remote-tracking branch 'origin/litellm_internal_staging' into litellm_fix_master_key_rotation_blocked
# Conflicts:
#	litellm/proxy/management_endpoints/key_management_endpoints.py
2026-08-30 12:35:21 -07:00
mateo-berri
ce52e39052 fix(gigachat): honor ssl_verify config on router passthrough and type the request body 2026-08-30 12:28:41 -07:00
mateo-berri
739f61df7d test(model_management): drop docstring that restates the serialization path 2026-08-30 12:26:06 -07:00
Mateo Wang
4ba8517134
Merge pull request #38722 from BerriAI/litellm_bedrock_guardrail_stream_audit
feat(bedrock): honor streaming buffer/sampling config for unbuffered post_call scans
2026-08-30 10:17:11 -07:00
Mateo Wang
8a156ed42d
Merge pull request #36722 from BerriAI/litellm_decrease_anys_fable6
chore(typing): clear 1.2k basedpyright Any errors across 16 hotspot files
2026-08-30 10:09:02 -07:00
mateo-berri
fbf7644676 Merge remote-tracking branch 'origin/litellm_internal_staging' into litellm_decrease_anys_fable6
# Conflicts:
#	basedpyright-code-budget.json
#	litellm/integrations/websearch_interception/handler.py
#	litellm/proxy/response_polling/background_streaming.py
#	ruff-strict-budget.json
#	type-discipline-budget.json
2026-08-30 10:02:06 -07:00
Devin AI
693279afb4 chore(techdebt): clear fresh debt from the 2026-08-29 window
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-08-30 07:57:16 +00:00
Devin AI
36c53e1288 chore(ui): regenerate schema.d.ts for updated endpoint descriptions
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-08-30 07:46:45 +00:00
Devin AI
4fb1440747 docs(proxy): clarify spend semantics on /v2/user/info and /user/daily/activity
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-08-30 07:40:56 +00:00
siyoon
3822ecc8ff chore(tests): drop redundant capability comment
Per greptile review + CLAUDE.md comment policy: the comment restated
the immediately following assertion without adding value.
2026-08-30 14:29:42 +09:00
siyoon
e9f1af8473 chore(tests): drop redundant capability comment
Per greptile review + CLAUDE.md comment policy: the comment restated
the immediately following assertions without adding value.
2026-08-30 14:29:28 +09:00
siyoon
e7bfe99cd3 feat(friendli): add zai-org/GLM-5.3 model pricing
Per https://api.friendli.ai/serverless/v1/models:
- $1.40 input / $4.40 output / $0.26 cached input per MTok
- 1M context, 1M max output, reasoning with effort low/high/max
  (per HF chat_template.jinja: low/high honored, anything else -> max)
- tool calling, parallel tool calls, structured output, prompt caching
- text-only (no vision), flagship GLM model
2026-08-30 14:23:36 +09:00
siyoon
3ea4b715ba feat(friendli): add zai-org/GLM-5.3-Flash model pricing
Per https://api.friendli.ai/serverless/v1/models:
- $0.15 input / $0.50 output / $0.03 cached input per MTok
- 1M context, 1M max output, reasoning with effort low/high/max
  (per HF chat_template.jinja: low/high honored, anything else -> max)
- tool calling, parallel tool calls, structured output, prompt caching
- image + video input (native multimodal)
2026-08-30 14:22:41 +09:00
mateo-berri
a928c1429e fix(proxy): preserve model table columns on master key rotation 2026-08-29 22:12:35 -07:00
mateo-berri
b0ce17c755 fix(gigachat): generic env-credential passthrough fallback plus type hardening
- forward unrouted /gigachat/* requests with env credentials like other passthrough providers (the old fallback returned 400 on any request without a routed model, /gigachat/models included)
- fix basedpyright budget breaches across the gigachat provider, common_request_processing, and llm_passthrough_endpoints with real narrowing, no new suppressions
- add regression tests for the fallback target, auth header, and model-less endpoints
2026-08-29 22:08:54 -07:00
mateo-berri
70e2f4e68f Merge remote-tracking branch 'origin/litellm_internal_staging' into litellm_gigachat_passthrough_25886
# Conflicts:
#	litellm/llms/gigachat/chat/transformation.py
2026-08-29 22:08:54 -07:00
mateo-berri
60296cb540 Merge remote-tracking branch 'origin/litellm_internal_staging' into litellm_bedrock_guardrail_stream_audit 2026-08-29 21:49:13 -07:00
Mateo Wang
5e4b3838aa
Merge pull request #37778 from BerriAI/litellm_decrease_anys_opus5
chore(typing): clear Any seams across 47 files, ratchet basedpyright ceilings -3,302
2026-08-29 21:48:11 -07:00
mateo-berri
667f761f1d Merge remote-tracking branch 'origin/litellm_internal_staging' into litellm_decrease_anys_fable6 2026-08-29 21:43:56 -07:00
mateo-berri
43c838f4b9 Merge branch 'litellm_internal_staging' into litellm_decrease_anys_opus5 2026-08-29 21:41:04 -07:00
Mateo Wang
b63bced31b
Merge pull request #38863 from BerriAI/litellm_fix_savings_test_cache_read_premise
fix(tests): derive the no-cache-read-rate savings baseline from the model map
2026-08-29 21:40:18 -07:00
mateo-berri
fd72ae830c test(model_management): drive /model/block and /model/unblock through response serialization
The route-level regression test returns a real prisma row from a mocked
update and asserts both routes serialize it to a 200 with the toggled
blocked flag, which is exactly the path that raised AttributeError before
the validator guard. Also binds the loop variable in the e2e poll lambda
(ruff B023).
2026-08-29 21:39:07 -07:00
mateo-berri
22ff5a8a69 Merge remote-tracking branch 'origin/litellm_internal_staging' into litellm_decrease_anys_fable6
# Conflicts:
#	basedpyright-code-budget.json
#	ruff-strict-budget.json
#	type-discipline-budget.json
2026-08-29 21:32:59 -07:00