Commit graph

37680 commits

Author SHA1 Message Date
Ashton Sidhu
6343148c95
Hiddenlayer Integration: Add V2 Integration (#22708)
* Serialize error message to a string; only scan last message

* Update litellm/proxy/guardrails/guardrail_hooks/hiddenlayer/hiddenlayer.py

Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>

* Add v2 of hiddenlayer guardrail implementation

* Update litellm/proxy/guardrails/guardrail_hooks/hiddenlayer/hiddenlayer.py

Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>

* Fix potential header issue

* linting

* Add image support

---------

Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
2026-04-14 23:37:49 +05:30
Daan
1d45cfd1fc
fix(proxy) - #25506 Team members added before team_member_budget is configured have no budget enforcement (#25557)
* fix #25506

* address greptile review feedback

* [Test] UI - Models: Add E2E tests for Add Model flow

Add E2E tests covering:
- Test connection with bad credentials shows failure modal
- Adding a specific model and verifying it appears in All Models table
- Adding a wildcard route and verifying it appears in All Models table
- Verifying model dropdown shows provider-specific models (existing test updated)

Added data-testid attributes to UI components to support stable test selectors.

Tests verified passing 3/3 consecutive runs with zero flakiness.

* address greptile review feedback (greploop iteration 1)

Add cleanup helper to delete models created during tests, preventing
stale data accumulation across repeated test runs.

* fix CI: replace data-testid selectors with text/role-based selectors

The data-testid attributes added to React components are not present
in the CI-built UI output. Switch to using getByRole and getByText
selectors which work with the rendered DOM regardless of build cache.

* remove unnecessary cleanup helper

The database is freshly seeded on every test run via seed.sql,
so per-test cleanup is not needed.

---------

Co-authored-by: Yuneng Jiang <yuneng@berri.ai>
Co-authored-by: Krrish Dholakia <krrish+github@berri.ai>
2026-04-14 23:37:49 +05:30
Kris Yang
15245a5eb7
fix: emit input_json_delta for tool args bundled in first streaming chunk (#25533)
* fix: emit input_json_delta for tool args bundled in first streaming chunk

Some providers (xAI, Gemini) include tool_call function arguments in the
same streaming chunk as the function name/id. The AnthropicStreamWrapper
was discarding the trigger chunk entirely when starting a new content
block, which silently dropped the input_json_delta carrying tool
arguments. This caused tool_use blocks to arrive with empty input {}.

Now queue the processed_chunk after content_block_start when it carries
non-empty input_json_delta data. Backward compatible: providers that send
empty arguments in the first chunk (OpenAI-style) are unaffected since
the condition checks for truthy partial_json.

* test: add tests for input_json_delta emission on bundled tool args

Covers the fix for providers (xAI, Gemini) that bundle tool_call
arguments in the same streaming chunk as the function name/id.
Verifies the AnthropicStreamWrapper emits input_json_delta after
content_block_start, and that empty-arg chunks (OpenAI-style) are
unaffected.

* style: apply Black formatting to streaming_iterator.py

* fix: mirror input_json_delta fix to sync __next__ and add sync tests

* test: make no_extra_delta tests assert explicitly instead of passing silently
2026-04-14 23:37:49 +05:30
lucassz
dd93d2698b
fix(gemini): assign correct indices in batch embedding response (#25656)
### Background

The Gemini batchEmbedContents response handler hardcoded `index=0` for
every embedding in the response. Any consumer relying on the OpenAI-format
`index` field to match embeddings back to inputs would silently get wrong
associations.

### Changes

Use `enumerate` in `process_response` so each embedding gets its
positional index instead of 0.

### Test Plan

Added unit test asserting sequential indices and correct vector ordering
for a 3-element batch response.
2026-04-14 23:37:49 +05:30
yuneng-jiang
3f3760bdf4
Merge pull request #25324 from Lucas-Song-Dev/fix-ui-policy-attachment-delete
fix(ui): delete policy attachments via controlled modal
2026-04-14 11:05:29 -07:00
Ishaan Jaffer
8c505634bd
chore: sync uv.lock with pyproject.toml (v1.83.6 -> v1.83.7) 2026-04-14 11:05:25 -07:00
Ishaan Jaffer
e20c114811
fix(mcp): set instructions=None in SigV4BuildFromTable test mocks
New MCPServer.instructions field requires a str; MagicMock attributes
not explicitly set return a MagicMock object, which fails Pydantic
validation.
2026-04-14 11:05:22 -07:00
Sameer Kankute
b8f7d61400
Merge pull request #25589 from BerriAI/litellm_oss_staging_04_11_2026
Litellm oss staging 04 11 2026
2026-04-14 23:34:25 +05:30
Sameer Kankute
b1c77d22f0
Merge pull request #25192 from BerriAI/litellm_oss_staging_04_04_2026
litellm_staging_04_04_2026
2026-04-14 23:33:16 +05:30
Krrish Dholakia
6d2b94261a fix(mypy): resolve type errors in compression/compress.py and __init__.py
Cast message lists to the expected `List[Union[AllMessageValues, Message]]`
type at `token_counter` call sites, and suppress the `no-redef` warning for
the `compress` import in `__init__.py` caused by the wildcard `main` import.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-04-14 10:54:31 -07:00
ishaan-berri
4a71583951
Merge pull request #25348 from BerriAI/litellm_gemini-veo-video-resolution-pricing2
feat(gemini): Veo Lite pricing, video resolution usage and tiered cost
2026-04-14 10:23:22 -07:00
ishaan-berri
e2fc7d64e8
Merge pull request #25396 from BerriAI/litellm_bedrock-normalize-custom-tool-schema
feat(bedrock): normalize custom tool JSON schema for Invoke and Converse
2026-04-14 10:21:15 -07:00
Ryan Crabbe
6da4e0b901
fix(ui): pre-select backend default for boolean guardrail provider fields
Boolean fields in the auto-generated guardrail provider form (e.g. Noma
`use_v2`) rendered as empty Selects because the Form.Item only populated
`initialValue` for percentage fields, and the `defaultValue` passed to the
Select child was silently dropped by antd's controlled-component wrapper.
Users could not tell what the backend default was, and the visual ambiguity
made flags like `use_v2` look inoperative even though the save path worked.

Unify `initialValue` to fall back through `fieldValue → field.default_value →
(percentage ? 0.5 : undefined)`, and switch Select.Option values from
"true"/"false" strings to real booleans so the backend default flows through
without stringification.
2026-04-14 10:13:58 -07:00
ishaan-berri
693c846617
Merge pull request #25674 from BerriAI/litellm_anthropic-messages-thinking-signature-retry
feat(anthropic): retry /v1/messages after invalid thinking signature
2026-04-14 10:10:43 -07:00
Lucas Song
9c869732a5 fix(ui): remove unused useMutation import and add React import to test 2026-04-14 10:08:18 -07:00
Lucas Song
977245c30e fix(ui): rename test file to tsx and remove unused useMutation import 2026-04-14 10:08:07 -07:00
ishaan-berri
9810a1b3b7
Merge pull request #25344 from BerriAI/litellm_Sameerlite/healthcheck-max-tokens
feat(health-check): add BACKGROUND_HEALTH_CHECK_MAX_TOKENS env var
2026-04-14 10:04:50 -07:00
Lucas Song
ef774a1991 Merge remote-tracking branch 'origin/main' into fix-ui-policy-attachment-delete 2026-04-14 10:02:23 -07:00
Lucas Song
53828dda76 refactor: migrate policy attachment deletion to useMutation hook with tests 2026-04-14 09:54:41 -07:00
ishaan-berri
f6058bd0ca
Merge pull request #25673 from michelligabriele/fix/responses-api-cache-key
fix(caching): add Responses API params to cache key allow-list
2026-04-14 09:53:05 -07:00
ishaan-berri
d479234f0e
Merge pull request #25694 from milan-berri/feat/mcp-initialize-instructions
feat(mcp): expose per-server InitializeResult.instructions from gateway
2026-04-14 09:51:07 -07:00
ishaan-berri
15ef3fa8ca
Merge pull request #25679 from michelligabriele/fix/azure-passthrough-standard-logging-object
fix(azure/passthrough): populate standard_logging_object via logging hook
2026-04-14 09:38:33 -07:00
Sameer Kankute
69bf2bfb9a
Fix tests 2026-04-14 21:18:23 +05:30
Sameer Kankute
a0e61a9d49
Fix code qa 2026-04-14 21:01:39 +05:30
Sameer Kankute
ee40da58a2
Merge branch 'main' into litellm_oss_staging_04_11_2026 2026-04-14 20:54:12 +05:30
Sameer Kankute
ffb87dcac9
Fix failing test and code qa + lint 2026-04-14 20:53:17 +05:30
Sameer Kankute
ef94f5fc4d
Fix budget reset test 2026-04-14 20:50:42 +05:30
Sameer Kankute
f6e526c5be
Fix bulk update tests 2026-04-14 20:46:21 +05:30
Sameer Kankute
3d567c34dd
Merge pull request #25698 from BerriAI/revert-25395-fix/25388-embedding-encoding-format
Revert "fix(embedding): omit null encoding_format for openai requests"
2026-04-14 20:37:35 +05:30
Sameer Kankute
e6771feace
Revert "fix(embedding): omit null encoding_format for openai requests (#25395)"
This reverts commit e3d160f158.
2026-04-14 20:36:28 +05:30
Sameer Kankute
972e42c7fd
Merge branch 'main' into litellm_oss_staging_04_04_2026 2026-04-14 20:23:06 +05:30
Milan
e7c630ed19
refactor: inline get_upstream_initialize_instructions
Remove the trivial one-line wrapper and access the dict directly.

Made-with: Cursor
2026-04-14 15:59:00 +03:00
Milan
7e656f4329
test: add unit tests for MCP initialize instructions feature
Extend existing test modules with coverage for the instructions merge
logic, upstream cache, ContextVar-based injection, and client-side
capture — following each file's established patterns.

Made-with: Cursor
2026-04-14 15:54:27 +03:00
Milan
8c2ebee4de
refactor(mcp): reuse existing sessions for initialize instructions
Remove the gateway-specific initialize fetch path and reuse instructions captured during existing MCP calls (list_tools/health_check/call_tool), while keeping YAML/DB instructions as immediate overrides.

Made-with: Cursor
2026-04-14 14:46:06 +03:00
Milan
96ed00e184
feat(mcp): gateway InitializeResult.instructions from upstream or YAML
- Add optional instructions on MCPServer (config/DB/types) and Prisma migration.
- MCPClient: fetch_upstream_initialize_instructions() for one-shot initialize.
- Gateway merges per-request instructions: YAML/API overrides; otherwise fetch
  upstream initialize instructions (skip spec_path/OpenAPI-only servers).
- Pass auth headers into instruction merge; ContextVar for gateway Server.
- REST: wire instructions on connection-test MCPServer payloads.

Made-with: Cursor
2026-04-14 14:19:31 +03:00
Milan
b9cd32b6d4
fix(proxy): enforce team membership in team-scoped key management checks
Block cross-team key update/regenerate operations by raising when the caller is not a member of the target key's team, and add unit coverage for deny/allow team membership paths.

Made-with: Cursor
2026-04-14 12:11:11 +03:00
michelligabriele
63281e8330
fix(azure/passthrough): populate standard_logging_object via logging hook 2026-04-14 08:10:39 +02:00
Sameer Kankute
5670f6c7d4
fix(anthropic): tighten thinking-signature retry (Greptile)
- Omit messages whose list content is empty after stripping thinking blocks
- Retry only on HTTP 400 plus invalid-signature body match
- Return response inline from retry loop; drop unreachable None guard
- Tests: thinking-only turn dropped, non-400 no retry

Made-with: Cursor
2026-04-14 10:03:10 +05:30
Sameer Kankute
0f453cc59d
Fic code qa 2026-04-14 10:00:27 +05:30
yuneng-jiang
b0a40fde6d
Merge pull request #25559 from shreyescodes/fix/cors-and-db-safety-bugs
fix: harden CORS credentials, create_views exception handling, and spend log cleanup loop
2026-04-13 21:17:12 -07:00
Sameer Kankute
c7f7708d27
feat(anthropic): retry /v1/messages after invalid thinking signature
Strip thinking blocks from the request body and retry once when Anthropic returns an invalid thinking signature error (e.g. after credential or deployment change). Applies to all BaseAnthropicMessagesConfig providers (direct Anthropic, Bedrock, Vertex, Azure AI).

Made-with: Cursor
2026-04-14 09:39:39 +05:30
michelligabriele
85497740ff
fix(caching): add Responses API params to cache key allow-list 2026-04-14 05:40:42 +02:00
yuneng-jiang
e64d98f725
Merge pull request #25590 from BerriAI/litellm_add_model_e2e_tests
[Test] UI - Models: Add E2E tests for Add Model flow
2026-04-13 19:03:39 -07:00
Yuneng Jiang
9b74ff3ef7
remove unnecessary cleanup helper
The database is freshly seeded on every test run via seed.sql,
so per-test cleanup is not needed.
2026-04-13 17:29:49 -07:00
Yuneng Jiang
cce7163348
fix CI: replace data-testid selectors with text/role-based selectors
The data-testid attributes added to React components are not present
in the CI-built UI output. Switch to using getByRole and getByText
selectors which work with the rendered DOM regardless of build cache.
2026-04-13 17:29:49 -07:00
Yuneng Jiang
5e07c1cbc9
address greptile review feedback (greploop iteration 1)
Add cleanup helper to delete models created during tests, preventing
stale data accumulation across repeated test runs.
2026-04-13 17:29:49 -07:00
Yuneng Jiang
4f364a8138
[Test] UI - Models: Add E2E tests for Add Model flow
Add E2E tests covering:
- Test connection with bad credentials shows failure modal
- Adding a specific model and verifying it appears in All Models table
- Adding a wildcard route and verifying it appears in All Models table
- Verifying model dropdown shows provider-specific models (existing test updated)

Added data-testid attributes to UI components to support stable test selectors.

Tests verified passing 3/3 consecutive runs with zero flakiness.
2026-04-13 17:29:49 -07:00
yuneng-jiang
8427534f13
Merge pull request #25647 from BerriAI/litellm_yj_apr_11
Some checks are pending
CodeQL / Analyze (actions) (push) Waiting to run
CodeQL / Analyze (javascript-typescript) (push) Waiting to run
CodeQL / Analyze (python) (push) Waiting to run
CodSpeed Benchmarks / benchmarks (push) Waiting to run
Helm unit test / unit-test (push) Waiting to run
Read Version from pyproject.toml / read-version (push) Waiting to run
Scorecard supply-chain security / Scorecard analysis (push) Waiting to run
Unit Tests: Proxy DB Operations / proxy-db (auth-checks, tests/proxy_unit_tests/test_auth_checks.py tests/proxy_unit_tests/test_user_api_key_auth.py, 20, 8) (push) Waiting to run
Unit Tests: Proxy DB Operations / proxy-db (key-generation, tests/proxy_unit_tests/test_key_generate_prisma.py, 30, 0) (push) Waiting to run
Unit Tests: Proxy DB Operations / proxy-db (remaining, tests/proxy_unit_tests --ignore=tests/proxy_unit_tests/test_key_generate_prisma.py --ignore=tests/proxy_unit_tests/test_auth_checks.py --ignore=tests/proxy_unit_tests/test_user_api_key_auth.py, 30, 8) (push) Waiting to run
Unit Tests: Security / security (push) Waiting to run
GitHub Actions Security Analysis / zizmor (push) Waiting to run
[Infra] Merge dev branch with main
2026-04-13 17:28:38 -07:00
yuneng-jiang
c25b4b2ce8
Merge pull request #25398 from BerriAI/litellm_team_settings_router
[Feature] UI - Teams: Allow Editing Router Settings After Team Creation
2026-04-13 17:27:15 -07:00
yuneng-jiang
a306092d47
Merge pull request #25463 from BerriAI/litellm_oss_staging_04_09_2026
Litellm oss staging 04 09 2026
2026-04-13 17:25:53 -07:00