Commit graph

33248 commits

Author SHA1 Message Date
Sameer Kankute
7a35116148 Fix : test_video_content_handler_uses_get_for_openai 2026-02-17 20:06:08 +05:30
Sameer Kankute
cd7fc38d21 Update poetry.lock 2026-02-17 19:51:59 +05:30
Sameer Kankute
211d6e9d30 Add vllm e2e test for embedding 2026-02-17 19:42:46 +05:30
Sameer Kankute
90be817b32 Add support for evals via sdk 2026-02-17 19:32:15 +05:30
Sameer Kankute
59408387ef Add eval run endpoints 2026-02-17 19:32:15 +05:30
Sameer Kankute
b246c3c56c Add eval run endpoints and methods 2026-02-17 19:32:15 +05:30
Sameer Kankute
8b75979fdc Add eval run endpoints methods 2026-02-17 19:32:15 +05:30
Sameer Kankute
782b048372 Update tests/test_litellm/llms/openai/evals/test_openai_evals_transformation.py
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
2026-02-17 19:30:58 +05:30
Sameer Kankute
ea72ffb03b Update litellm/llms/custom_httpx/llm_http_handler.py
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
2026-02-17 19:30:58 +05:30
Sameer Kankute
590b98630d Update litellm/llms/custom_httpx/llm_http_handler.py
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
2026-02-17 19:30:58 +05:30
Sameer Kankute
32263deb02 Add tests for openai evals 2026-02-17 19:30:58 +05:30
Sameer Kankute
71385a4e12 Add openai evals in base http handler 2026-02-17 19:30:58 +05:30
Sameer Kankute
5cc0036c87 Add openai evals endpoints and routing 2026-02-17 19:30:58 +05:30
Sameer Kankute
6b95bdeb12 Add OpenAIEvalsConfig 2026-02-17 19:30:58 +05:30
Sameer Kankute
a4eae1bc53 Add Evals API operations 2026-02-17 19:30:58 +05:30
Sameer Kankute
357a2b640e Add Evals API operations 2026-02-17 19:30:58 +05:30
Sameer Kankute
525acaf755 Add BaseEvalsAPIConfig for openai evals 2026-02-17 19:30:58 +05:30
Sameer Kankute
1c2e1148f3
Merge branch 'main' into litellm_oss_staging_02_16_2026 2026-02-17 18:24:56 +05:30
Sameer Kankute
f4b79fa635
Merge pull request #21368 from TomeHirata/databricks/response-schema
Add Databricks to supported LLM providers for response schema
2026-02-17 14:04:18 +05:30
Tomu Hirata
43ba7d0a07 Add test case for Databricks Meta LLaMA 3.1 70B instruct model in content parsing tests
Signed-off-by: Tomu Hirata <tomu.hirata@gmail.com>
2026-02-17 15:36:00 +09:00
Tomu Hirata
3053fedad2 Add Databricks to supported LLM providers for response schema
Signed-off-by: Tomu Hirata <tomu.hirata@gmail.com>
2026-02-17 15:21:55 +09:00
yuneng-jiang
69ee2c21ea
Merge pull request #21362 from BerriAI/litellm_default_key_fix
[Fix] Key Expiry Default Duration
2026-02-16 21:09:23 -08:00
yuneng-jiang
8576683f39
Update tests/test_litellm/proxy/management_endpoints/test_key_management_endpoints.py
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
2026-02-16 21:09:13 -08:00
yuneng-jiang
ec19005942 refactor - change key type label 2026-02-16 21:08:13 -08:00
yuneng-jiang
ee5120bfc2 fix default duration 2026-02-16 20:54:10 -08:00
Emerson Gomes
d859f0687d
fix(router): avoid alias scan for non-alias get_model_list lookups (#21136)
Co-authored-by: Codex <codex@example.com>
2026-02-16 20:40:24 -08:00
Emerson Gomes
93b848494d
perf: increase default LRU cache size to reduce multi-model thrash (#21139)
* perf: increase default LRU cache size to 64

* chore: remove default LRU constant test

* docs: update DEFAULT_MAX_LRU_CACHE_SIZE default to 64
2026-02-16 20:39:43 -08:00
Ryan H
5749ca6c47 feat(bedrock): broaden Nova 2 model detection to support nova-2-pro reasoning
- Rename _is_nova_lite_2_model → _is_nova_2_model to match all nova-2-* variants
- Add bedrock/converse/ routing prefix stripping in model detection
- Fix pre-existing test_get_supported_openai_params_bedrock_converse failure
- Remove thinking_blocks tests from Nova 2 test file (not Nova 2 behavior)
- Add end-to-end request, response, multi-turn, and model detection tests
- Parametrize key tests across both nova-2-lite and nova-2-pro model IDs
2026-02-16 20:35:00 -08:00
shin-bot-litellm
b609f5841b
fix: add missing OpenAI chat completion params to OPENAI_CHAT_COMPLETION_PARAMS (#21360)
* allow filtering by user in global usage

* add server root path test to github actions

* Update .github/workflows/test_server_root_path.yml

Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>

* address greptile review feedback (greploop iteration 1)

- Fix HTTPException swallowed by broad except block in get_user_daily_activity
  and get_user_daily_activity_aggregated: re-raise HTTPException before the
  generic handler so 403 status codes propagate correctly
- Add status_code assertions in non-admin access tests

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* address greptile review feedback (greploop iteration 2)

- Default user_id to caller's own ID for non-admins instead of 403 when
  omitted, preserving backward compatibility for API consumers
- Apply same fix to aggregated endpoint
- Update test to verify defaulting behavior instead of expecting 403
- Add useEffect to sync selectedUserId when auth state settles in
  UsagePageView to handle async auth initialization

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fixing syntax

* remove artifacts

* feat: guardrail tracing UI - policy, detection method, match details (#21349)

* feat: add GuardrailTracingDetail TypedDict and tracing fields to StandardLoggingGuardrailInformation

* feat: add policy_template field to Guardrail config TypedDict

* feat: accept GuardrailTracingDetail in base guardrail logging method

* feat: populate tracing fields in content filter guardrail

* test: add tracing fields tests for custom guardrail base class

* test: add tracing fields e2e tests for content filter guardrail

* feat: add guardrail tracing UI - policy badges, match details, timeline

* feat: redesign GuardrailViewer to Guardrails & Policy Compliance layout

Two-column layout with request lifecycle timeline on the left
and compact evaluation detail cards on the right. Header shows
guardrail count, pass/fail status, total overhead, policy info,
and an export button.

* feat: add clickable guardrail link in metrics + show policy names

* feat: add risk_score field to StandardLoggingGuardrailInformation

* feat: compute risk_score in content filter guardrail

* feat: display backend risk_score badge on evaluation cards

* fix: fallback to frontend risk score when backend doesn't provide one

* passing in masster key for api calls

* Fix: Add blog as incident report

* Fix: Add blog as incident report

* remove timeline

* feat(models): add github_copilot/gpt-5.3-codex and github_copilot/claude-opus-4.6-fast (#21316)

Add missing GitHub Copilot model entries for gpt-5.3-codex (GA) and
claude-opus-4.6-fast (Public Preview) to both the root and backup
model pricing JSON files.

* only tests for /ui

* bump: version 1.81.12 → 1.81.13

* Fixing mapped tests

* fixing no_config test

* fixing container tests

* fixing test_basic_openai_responses_api

* Adding bedrock thinking budget tokens to docs

* fixing regen key tests

* fix: add missing OpenAI chat completion params to OPENAI_CHAT_COMPLETION_PARAMS

Add store, prompt_cache_key, prompt_cache_retention, safety_identifier, and verbosity
to OPENAI_CHAT_COMPLETION_PARAMS list.

These params were already in DEFAULT_CHAT_COMPLETION_PARAM_VALUES but missing from
the OPENAI_CHAT_COMPLETION_PARAMS list, causing them to be dropped when passed to
OpenAI-compatible providers.

---------

Co-authored-by: yuneng-jiang <yuneng.jiang@gmail.com>
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-authored-by: Ishaan Jaff <ishaanjaffer0324@gmail.com>
Co-authored-by: Sameer Kankute <sameer@berri.ai>
Co-authored-by: Cesar Garcia <128240629+Chesars@users.noreply.github.com>
Co-authored-by: Krish Dholakia <krrishdholakia@gmail.com>
2026-02-16 20:31:21 -08:00
Emerson Gomes
b67c140938
Fix Bedrock service_tier cost propagation (#21172) 2026-02-16 20:30:10 -08:00
Nick Amabile
4978df8ebd
fix: add store to OPENAI_CHAT_COMPLETION_PARAMS (#21195)
The OpenAI `store` parameter (used for storing completions for
distillation/evals) was missing from `OPENAI_CHAT_COMPLETION_PARAMS`.

This caused it to be unrecognized by `get_standard_openai_params()` and
the `litellm_proxy` provider config. It also meant that code paths using
this list (rather than `DEFAULT_CHAT_COMPLETION_PARAM_VALUES`) would
treat `store` as a provider-specific parameter and forward it to
non-OpenAI providers like Anthropic, resulting in:

    "store: Extra inputs are not permitted"

Fixes #19700
2026-02-16 20:28:34 -08:00
Emerson Gomes
ddb48fa116
perf(router): use set membership in team deployment filter (#21210) 2026-02-16 20:22:23 -08:00
Emerson Gomes
fb2c22ecce
perf(router): optimize v2 deployment selection lookup (#21211) 2026-02-16 20:20:48 -08:00
sahukanishka
d184b3cae7
fix: preserve provider_specific_fields from proxy responses (#21153) (#21220)
Co-authored-by: kanishka sahu <kanishkasahu@mercor.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
2026-02-16 20:17:58 -08:00
yuneng-jiang
96d7585ea4
Merge pull request #21353 from BerriAI/litellm_gh_server_root_test
[Infra] Add Server Root Test to GitHub Actions
2026-02-16 20:16:25 -08:00
yuneng-jiang
3b5096d142
Merge pull request #21357 from BerriAI/release_feb16_copy
[Infra] v1.81.13-nightly Change Copy to main
2026-02-16 20:15:57 -08:00
Emerson Gomes
f162371b93
fix(pod-lock): make release lock compare-and-delete atomic (#21226) 2026-02-16 20:15:46 -08:00
yuneng-jiang
3669d547f6 fixing regen key tests 2026-02-16 20:14:40 -08:00
yuneng-jiang
aa7bc6ae3b Adding bedrock thinking budget tokens to docs 2026-02-16 20:14:26 -08:00
yuneng-jiang
45e6440b0a fixing test_basic_openai_responses_api 2026-02-16 20:14:06 -08:00
yuneng-jiang
23219d9217 fixing container tests 2026-02-16 20:13:55 -08:00
yuneng-jiang
efe84777e5 fixing no_config test 2026-02-16 20:13:45 -08:00
yuneng-jiang
349e3dad55 Fixing mapped tests 2026-02-16 20:13:37 -08:00
yuneng-jiang
c4f0fc9819 bump: version 1.81.12 → 1.81.13 2026-02-16 20:13:26 -08:00
Mateusz Szewczyk
72af441159
feat: Add IBM watsonx.ai rerank support (#21303)
* feat: Add IBM watsonx.ai rerank support

* feat: added unit tests

* fix docstring

* added documentataion

* Update litellm/llms/watsonx/rerank/transformation.py

Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>

* Update litellm/rerank_api/main.py

Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>

* Update litellm/llms/watsonx/rerank/transformation.py

Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>

* update validate_environment signature

* fix ruff check and mypy

* fix CR

* CR fix

* CR fix

---------

Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
2026-02-16 20:12:16 -08:00
Adam Reed
8d50956051
fix(proxy): preserve and forward OAuth Authorization headers through proxy layer (#19912)
PR #21039 fixed OAuth token handling at the LLM layer (Authorization: Bearer
instead of x-api-key), but the proxy layer still strips the Authorization
header in clean_headers() before it reaches the Anthropic code. This breaks
OAuth for proxy users (e.g., Claude Code Max through LiteLLM proxy).

Changes:
- Add is_anthropic_oauth_key() helper to detect OAuth tokens (sk-ant-oat*)
- Preserve OAuth Authorization headers in clean_headers() instead of stripping
- Forward OAuth Authorization via ProviderSpecificHeader in
  add_provider_specific_headers_to_request() so tokens only reach
  Anthropic-compatible providers (anthropic, bedrock, vertex_ai)

Fixes #19618

Co-authored-by: Adam Reed <iamadamreed@users.noreply.github.com>
2026-02-16 20:09:07 -08:00
yuneng-jiang
5ac3430866 only tests for /ui 2026-02-16 20:08:38 -08:00
Cesar Garcia
757acb4634
feat(models): add github_copilot/gpt-5.3-codex and github_copilot/claude-opus-4.6-fast (#21316)
Add missing GitHub Copilot model entries for gpt-5.3-codex (GA) and
claude-opus-4.6-fast (Public Preview) to both the root and backup
model pricing JSON files.
2026-02-16 20:04:48 -08:00
Sameer Kankute
4936aabdba
Merge pull request #21356 from BerriAI/litellm_incident_claude_code
Fix: Add blog as incident report
2026-02-17 09:33:31 +05:30
Sameer Kankute
6acf63f8b3 remove timeline 2026-02-17 09:31:17 +05:30