Commit graph

39980 commits

Author SHA1 Message Date
Yuneng Jiang
dbe70086c6
Remove Chat UI link from Swagger docs message 2026-04-15 09:52:24 -07:00
Yuneng Jiang
81e38491ff
[Docs] Regenerate v1.83.3-stable release notes from v1.82.3-stable baseline
The previous v1.83.3 changelog was generated against v1.83.0-nightly and
missed ~3 weeks of work. This regenerates it against the previous stable
release and restructures the LLM API Endpoints section to group by API
type (Responses, Batch, Count Tokens, Video Generation, Pass-Through,
etc.) matching the convention used in v1.82.3, v1.82.0, and v1.81.14.
Adds ~25 previously uncited PRs, cross-section duplications for
cross-cutting changes, and a verified first-time-contributors list.
2026-04-15 09:52:24 -07:00
Ryan Crabbe
3b8f651eaf
fix: default invite user modal global role to least-privilege
Pre-select "Internal User Viewer" in the Global Proxy Role dropdown
on both the standalone and embedded Invite User forms so admins don't
have to remember to pick a role, and the default lands on the least
privileged option rather than silently posting an undefined role.
2026-04-15 09:52:24 -07:00
Ryan Crabbe
8837138c8f
[Docs] Use GitHub avatar for Ryan Crabbe in release notes
Replace the expiring LinkedIn CDN image URL with a stable GitHub
avatar URL for v1.83.3 and v1.83.7.rc.1 release notes.
2026-04-15 09:52:24 -07:00
Yuneng Jiang
c94f5d56a8
[Docs] Add missed content PRs to v1.83.7.rc.1 and update runbook
- Add 8 content PRs that merged directly to the release branch outside the listed staging PRs: #23769 (Ramp callback), #25252 (JWT OAuth2 override), #25254 (AWS GovCloud mode), #25258 (batch-limit cleanup), #25334 (router custom_llm_provider), #25345 (Triton embeddings), #25347 (tag-based routing), #25358 (Baseten pricing attribution)
- Add @kedarthakkar to new contributors (first-ever PR via #23769)
- Update RELEASE_NOTES_GENERATION_INSTRUCTIONS: require walking git log range between release tags in addition to staging PRs, and verify new-contributor status per author rather than trusting the GH release body floor
2026-04-15 09:52:24 -07:00
Yuneng Jiang
d92e65cc6f
[Fix] Correct pip install versions for v1.83.3-stable and v1.83.7.rc.1 docs
PyPI publishes 1.83.3 and 1.83.7 (no .post1 / rc1 suffixes) — align the pip install commands with the actual published versions.
2026-04-15 09:52:24 -07:00
Yuneng Jiang
719f4cafca
[Docs] Add release notes for v1.83.3-stable and v1.83.7.rc.1
- Retitle existing v1.83.3 preview file to v1.83.3-stable (same commit)
- Add new v1.83.7.rc.1 preview release notes
- Update RELEASE_NOTES_GENERATION_INSTRUCTIONS runbook with guidance on resolving staging PRs to their underlying commits
2026-04-15 09:52:23 -07:00
Ryan Crabbe
a23408d937
test(ui): add getCookie to cookieUtils mock in user_dashboard test
user_dashboard.tsx imports getCookie from @/utils/cookieUtils, but the
vi.mock factory in user_dashboard.test.tsx only exports clearTokenCookies.
Vitest throws `No "getCookie" export is defined on the "@/utils/cookieUtils"
mock`, breaking all three beforeunload-listener tests.

Add getCookie to the mock factory so it matches the current imports.
2026-04-15 09:52:23 -07:00
Ryan Crabbe
27484c4a41
fix: isolate logs team filter dropdown from root teams state bleed
The Logs view's Team ID filter dropdown was reading `allTeams` from the
root `teams` state in page.tsx, which the Teams page search overwrites
with its filtered subset. Applying a team search on the Teams page made
filtered-out teams disappear from the Logs filter dropdown.

Swap the Team ID filter to use the existing `TeamDropdown` component via
a small `FilterTeamDropdown` wrapper that adapts it to the filter slot's
`FilterOptionCustomComponentProps` contract. The dropdown now drives its
own `useInfiniteTeams` query against `/v2/team/list` with server-side
search and an isolated react-query cache, unreachable from root state.

Rename the now-unused `hookAllTeams` destructure to `allTeams` so the
`KeyInfoView` passthrough receives the hook's unpolluted fetch instead
of the polluted prop, and drop the dead `allTeams` prop from
`SpendLogsTable` and both of its call sites.
2026-04-15 09:52:23 -07:00
Ryan Crabbe
f27bf8e711
fix(ui): pre-select backend default for boolean guardrail provider fields
Boolean fields in the auto-generated guardrail provider form (e.g. Noma
`use_v2`) rendered as empty Selects because the Form.Item only populated
`initialValue` for percentage fields, and the `defaultValue` passed to the
Select child was silently dropped by antd's controlled-component wrapper.
Users could not tell what the backend default was, and the visual ambiguity
made flags like `use_v2` look inoperative even though the save path worked.

Unify `initialValue` to fall back through `fieldValue → field.default_value →
(percentage ? 0.5 : undefined)`, and switch Select.Option values from
"true"/"false" strings to real booleans so the backend default flows through
without stringification.
2026-04-15 09:52:23 -07:00
Sameer Kankute
3fdd67ff23
Delete docs/my-website/blog/debug_cost_discrepancy/index.md 2026-04-15 21:35:05 +05:30
Ryan Crabbe
17568e81f2
chore(ui): use antd Typography in GuardrailTestPlayground
- Replace plain text/heading tags with Typography.Text, Title, Paragraph
- Document Typography preference in CLAUDE.md UI guidelines
2026-04-15 08:45:11 -07:00
Sameer Kankute
277be4c50e
Add input + output tokens for anthropic message type 2026-04-15 21:08:34 +05:30
Sameer Kankute
9ed90d53cd
fix(router): enable order fallback for wildcard model groups
Use wildcard-aware deployment lookup when building order-based fallback levels so requests like openai/gpt-4.1-mini can advance from order=1 to order=2, and add a regression test for wildcard routing.

Made-with: Cursor
2026-04-15 17:48:25 +05:30
Vinh Pham Huu
61aee29b41 feat: Update video metadata handling and media resolution checks for Gemini models 2026-04-15 17:56:15 +07:00
Vinh Pham Huu
9a3d9b2632 feat: Enhance support for video metadata across all Gemini models in transformation logic and tests 2026-04-15 17:11:12 +07:00
Yuneng Jiang
6426bc41f5
Merge remote-tracking branch 'origin/litellm_internal_staging' into litellm_yj_apr14 2026-04-14 22:40:04 -07:00
Ryan Crabbe
b28b82c105
chore(ui): migrate GuardrailTestPlayground off @tremor/react to antd
Part of the ongoing migration away from @tremor/react per CLAUDE.md.
Swaps tremor Card/Title/Text/TextInput for antd Card/Input and plain
h2/h3/span/p tags with Tailwind. Drops the redundant Checkbox avatar in
the sidebar list since row clicks already toggle selection and the
blue-border + bg-blue-50 styling already signals selected state.
2026-04-14 21:46:13 -07:00
shivam
9d089c88f0
Merge remote-tracking branch 'upstream/litellm_internal_staging' into litellm_vector-store-team-byok-model-none 2026-04-14 21:14:41 -07:00
shin-berri
4a73e94618
Merge pull request #25747 from BerriAI/main
[Infra] Merge main into litellm_internal_staging
2026-04-14 21:13:12 -07:00
yuneng-jiang
72a461ba4a
Merge pull request #25733 from BerriAI/litellm_guardMainBranch
Some checks are pending
CodeQL / Analyze (actions) (push) Waiting to run
CodeQL / Analyze (javascript-typescript) (push) Waiting to run
CodeQL / Analyze (python) (push) Waiting to run
CodSpeed Benchmarks / benchmarks (push) Waiting to run
Helm unit test / unit-test (push) Waiting to run
Read Version from pyproject.toml / read-version (push) Waiting to run
Scorecard supply-chain security / Scorecard analysis (push) Waiting to run
Unit Tests: Proxy DB Operations / proxy-db (auth-checks, tests/proxy_unit_tests/test_auth_checks.py tests/proxy_unit_tests/test_user_api_key_auth.py, 20, 8) (push) Waiting to run
Unit Tests: Proxy DB Operations / proxy-db (key-generation, tests/proxy_unit_tests/test_key_generate_prisma.py, 30, 0) (push) Waiting to run
Unit Tests: Proxy DB Operations / proxy-db (remaining, tests/proxy_unit_tests --ignore=tests/proxy_unit_tests/test_key_generate_prisma.py --ignore=tests/proxy_unit_tests/test_auth_checks.py --ignore=tests/proxy_unit_tests/test_user_api_key_auth.py, 30, 8) (push) Waiting to run
Unit Tests: Security / security (push) Waiting to run
GitHub Actions Security Analysis / zizmor (push) Waiting to run
[Infra] Guard main to only accept PRs from staging and hotfix branches
2026-04-14 20:54:29 -07:00
shivam
f122aa1b73
fix(router): restore BYOK key injection for vector store endpoints with team-scoped deployments
When vector store endpoints (POST/GET /v1/vector_stores) are called, model=None
is passed to the router. map_team_model(None, team_id) was returning None
unchanged after the team model routing fix in #25148, so the router never found
the team's BYOK deployment and forwarded requests without the API key.

Fix: when team_model_name is None, return the matched deployment's
team_public_model_name (or model_name fallback) so the router can route to it
and inject the BYOK credentials. Does not affect the sibling-deployment
load-balancing fix since that only applies when a non-None model is passed.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-14 20:47:57 -07:00
harish876
b3c413aefe add a composite index on the model_name, model_id and checked_at key for lookup. 2026-04-15 03:41:52 +00:00
yuneng-jiang
9790a46f69
Merge pull request #25730 from BerriAI/yj_bump_apr14_2
bump: version 1.83.7 → 1.83.8
2026-04-14 20:11:58 -07:00
yuneng-jiang
bdb4f396bb
Merge pull request #25741 from joereyna/fix/test-server-root-path-timeout
fix(ci): increase test-server-root-path timeout to 30m
2026-04-14 19:54:23 -07:00
yuneng-jiang
3284dee734
Merge pull request #25737 from joereyna/fix/remove-missing-mcps-coverage-path
fix: remove non-existent litellm_mcps_tests_coverage from coverage combine
2026-04-14 19:54:13 -07:00
yuneng-jiang
50786007fc
Merge pull request #25736 from BerriAI/docs_visual_guide_for_guardrail_fallbacks
docs update
2026-04-14 19:51:54 -07:00
yuneng-jiang
ffc3a9736d
Merge pull request #25739 from BerriAI/litellm_flakyBedrockGptOssToolCall
[Test] Replace flaky bedrock gpt-oss tool-call live test with request-body mock
2026-04-14 19:49:04 -07:00
joereyna
ccbdaa9187
fix(ci): increase test-server-root-path timeout to 30m 2026-04-14 19:42:10 -07:00
Yuneng Jiang
e2043e11f1
[Test] add request-body mock test for bedrock gpt-oss tool schema
Complements the stubbed-out live integration test by verifying the
outgoing Bedrock Converse request body for GPT-OSS is well-formed when
the caller supplies a tool schema with OpenAI-style metadata
($id, $schema, additionalProperties, strict):
- correct converse URL for bedrock/converse/openai.gpt-oss-20b-1:0
- toolConfig.tools[0].toolSpec has the expected name/description
- inputSchema.json keeps type/properties/required and strips fields
  Bedrock does not accept
2026-04-14 19:36:57 -07:00
Yuneng Jiang
8e44a02a22
[Test] stub flaky bedrock gpt-oss function-calling stream test
GPT-OSS on Bedrock intermittently emits truncated toolUse.input deltas
(e.g. accumulated args of '{"":"'), causing
test_function_calling_with_tool_response to hard-fail on json.loads.
The model flakiness is not a litellm regression: the same base test
passes for Anthropic in the same CI run, and the streaming delta path
at invoke_handler.py has not changed recently.

Follow the existing override pattern in TestBedrockGPTOSS
(test_prompt_caching, test_completion_cost, test_tool_call_no_arguments)
and stub the test to pass. The underlying bedrock converse streaming
tool-call path is already covered by Claude/Nova/Llama Converse suites
in test_bedrock_completion.py and test_bedrock_llama.py, so removing
the live GPT-OSS check loses no unique litellm-side signal.
2026-04-14 19:13:42 -07:00
Yuneng Jiang
d6a69b9c81
[Test] mark bedrock gpt-oss function-calling stream test flaky
Bedrock GPT-OSS occasionally emits truncated toolUse.input deltas
(e.g. accumulated args of '{"":"'), which causes
test_function_calling_with_tool_response to hard-fail on json.loads.
Other overrides in TestBedrockGPTOSS already handle similar
model-side flakiness; apply retries=6 delay=5 scoped to this subclass
so other providers keep strict behavior.
2026-04-14 19:10:55 -07:00
Ishaan Jaffer
84c507bc14
fix(mypy): use explicit None check for cache rate values to satisfy type checker 2026-04-14 19:03:56 -07:00
joereyna
a01cf44c35
fix: remove non-existent litellm_mcps_tests_coverage from coverage combine 2026-04-14 18:59:25 -07:00
Yuneng Jiang
38f8d7a008
Point contributors toward litellm_oss_branch in guard error messages 2026-04-14 18:41:59 -07:00
Yuneng Jiang
ab71d3d700
Also reject PRs from forks, not just non-allowlisted branches 2026-04-14 18:39:54 -07:00
shivam
fd110cd5cf
docs update 2026-04-14 18:33:42 -07:00
Ishaan Jaffer
e20d9df1b6
remove test file 2026-04-14 18:29:15 -07:00
Ishaan Jaffer
e0a988e39a
feat(ui/log-details): pass rawInputTokens, cacheReadTokens, cacheCreationTokens to CostBreakdownViewer from SpendLogs 2026-04-14 18:29:12 -07:00
Ishaan Jaffer
0148effd6e
feat(ui/cost-breakdown): show separate Input / Cache Read / Cache Write line items in cost breakdown drawer 2026-04-14 18:29:12 -07:00
Ishaan Jaffer
6b1dc1156e
fix(ui/usage): subtract cache tokens from Input Tokens summary card to avoid double-counting 2026-04-14 18:29:12 -07:00
Ishaan Jaffer
781fc6311b
feat(cost-calculator): compute and store per-type cache costs in CostBreakdown (cache_read_cost, cache_creation_cost) 2026-04-14 18:29:12 -07:00
Ishaan Jaffer
b5a4c26248
feat(logging): pass cache_read_cost and cache_creation_cost through set_cost_breakdown 2026-04-14 18:29:12 -07:00
Ishaan Jaffer
c84597ecd0
fix(bedrock/converse): capture raw input_tokens as text_tokens before cache inflation in PromptTokensDetailsWrapper 2026-04-14 18:29:12 -07:00
Ishaan Jaffer
5c056cae9f
fix(anthropic): store raw text_tokens in PromptTokensDetailsWrapper before cache inflation 2026-04-14 18:29:12 -07:00
Ishaan Jaffer
c1dcfa70c9
feat(types): add cache_read_cost and cache_creation_cost fields to CostBreakdown TypedDict 2026-04-14 18:29:12 -07:00
Ishaan Jaffer
d805fe7103
test(bedrock): add unit tests for cache token billing with prompt caching
Adds TestBedrockInvokeCacheTokenBilling covering the Bedrock InvokeModel path:
- baseline: no cache tokens, prompt_tokens equals input_tokens
- cache_read: prompt_tokens inflated by design, prompt_tokens_details carries breakdown
- cache_creation: same pattern for write tokens
- cost_calculation_correct_with_cache_read: core billing regression test
- cost_calculation_correct_with_cache_creation: write-rate billing regression test
- back_to_back_requests_cost: full end-to-end scenario (cache write then read)

These lock in the fix from PR #25517 - cache tokens were being double-counted
in AnthropicConfig.calculate_usage causing 10-50x inflated cost on cache reads.
2026-04-14 18:29:12 -07:00
Yuneng Jiang
45d1e1b341
[Infra] Guard main branch with PR source-branch check
Adds a GHA that fails PRs to main unless the head branch is
'litellm_internal_staging' or 'litellm_hotfix_*'. Also fails merge_group
events since merge queue is not in use.
2026-04-14 18:19:14 -07:00
yuneng-jiang
5c1f7d99bf
Merge pull request #25731 from BerriAI/docs_guardrail
fallbacks image
2026-04-14 18:13:12 -07:00
shivam
65ce89dc67
update 2026-04-14 18:02:41 -07:00