scriptoriumadmin/litellm
1
0
Fork 0
mirror of https://github.com/BerriAI/litellm.git synced 2026-08-28 05:25:59 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 8
43876 commits 3745 branches 1395 tags 3.9 GiB
Commit graph

8121 commits

Author SHA1 Message Date
Alan Ponnachan
5f12e4be1e
fix(langfuse): Handle null usage values to prevent validation errors (#16396) 
* langfuse null validation fix

* formatting
 2025-11-10 19:34:56 -08:00
Alan Ponnachan
7b292ccdf5
router fallback for unknown models (#16419) 2025-11-10 19:33:01 -08:00
Alan Ponnachan
df33f36c06
Correctly handle date filters in /spend/logs endpoint (#16443) 2025-11-10 19:26:55 -08:00
Val Miscenko
19e6e60d30
fix: remove strict master_key check in add_deployment (#16453) 
Allows proxy to save spend logs without requiring master_key.
  Decryption now gracefully handles both encrypted and unencrypted values.
 2025-11-10 19:23:13 -08:00
Sameer Kankute
6cab77f53f
Added thinking streaming support for mistral (#16434) 2025-11-10 18:41:45 -08:00
Sameer Kankute
be3c09e6d5
Add GET list of providers endpoint (#16432) 2025-11-10 18:38:09 -08:00
Ishaan Jaff
8140d85d28
[Bug Fix] - LiteLLM Usage shows key_hash- (#16471) 
* test_get_logging_payload_api_key_preserved_when_standard_logging_payload_is_none

* fix get_logging_payload

* test_api_key_preserved_through_failure_hook_to_database
 2025-11-10 18:15:52 -08:00
Jehandad Kamal
912be308b2
fix: allow internal users to access video generation routes (#16472) 
Fixes #16470

Video generation endpoints (/v1/videos, /videos/{video_id}, etc.) were
incorrectly restricted to proxy_admin role only. These routes are now
added to openai_routes list, making them accessible to internal_user
role as they should be - video generation is a legitimate user feature,
not a management/admin operation.

Changes:
- Added 8 video route patterns to LiteLLMRoutes.openai_routes in _types.py
- Added comprehensive tests verifying internal_user and virtual key access
- All existing route permission tests continue to pass

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-authored-by: Claude <noreply@anthropic.com>
 2025-11-10 17:44:16 -08:00
yuneng-jiang
7833b3fdb4 Addressing comments 2025-11-10 17:28:13 -08:00
Ishaan Jaff
dc76b6c76e
[Fix] Management Endpoints - Fixes inconsistent error responses in customer management endpoints. Non-existent user errors now return proper 404 status codes with consistent error schema format across all endpoints. (#16450) 
* fix: ensure end user endpoints use "handle_exception_on_proxy" correctly

* test 404 on info and update for non-existent user

* test 404 for no customer found

* fix 404 handling for customer endpoints

* test_error_schema_consistency

* test_customer_endpoints_error_schema_consistency
 2025-11-10 17:16:53 -08:00
Ishaan Jaffer
8c1dd78c13 fix imports 2025-11-08 18:01:32 -08:00
Krish Dholakia
06906534b3
feat(audio_transcriptions/): calculate duration of audio file for cost calculation + feat (image_generations): cost tracking accuracy improved with output_format, quality, size values fixed per openai model 
* feat(audio_transcriptions/): calculate duration of audio file for cost calculation

Fixes https://github.com/BerriAI/litellm/issues/11846

Closes https://github.com/BerriAI/litellm/issues/14605

* fix(cost_calculator.py): correctly use base model, when set

Fixes issue where azure base model was being ignored

* feat(cost_calculator.py): fix default cost tracking quality param for image generation

* feat(image_generations/): return output_format, quality, size

aligns response to openai spec and improves cost tracking accuracy

* fix(cost_calculator.py): refactor cost calculation for image generation to use image response instead of hidden params

* build: update build

* fix: fix cost calculation

* build: update poetry lock

* fix: fix ruff checks

* fix: fix aembedding

* fix: fix ruff errors

* fix: modify to catch errors

* fix: test

* fix: loosen test to handle openai lib out of sync

* fix: fix base models

* fix: fix usage object
 2025-11-08 16:24:31 -08:00
Sameer Kankute
86d73c918c
Adds support for returning Azure Content Policy error information when exceptions from Azure OpenAI occur (#16231) 
* add provider_specific_fields to ContentPolicyViolationError

* use provider_specific_fields in ProxyException

* update openai_exception_handler

* fix use exception checker for content policy violation azure

* add AzureOpenAIExceptionMapping

* test_azure_with_content_safety_error

* Accessing Provider-Specific Error Details

* TestExceptionCheckers

* unit test got provider_specific_fields=

* add clear types for error dict

* fix test_azure_with_content_safety_error

---------

Co-authored-by: Ishaan Jaff <ishaanjaffer0324@gmail.com>
 2025-11-08 16:04:36 -08:00
Sameer Kankute
e037d9315d
Add Vertex and Gemini Videos API with Cost Tracking + UI support (#16323) 
* Use video id for videos api

* remove mock code

* Potential fix for code scanning alert no. 3630: Clear-text logging of sensitive information

Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>

* remove print statements

* Update video prefix for 'video_'

* Add veo with openai videos unified specs

* Add videos testing to UI

* remove mock code

* Remove not need ui changes:

* Fix mypy errors related to gemini

* fix test_transform_video_create_request

* Add vertex ai veo config

* Add vertex ai veo config

* Add cost tracking for gemini and add optional param passing

* fix bugs related to vertex ai veo

* Add Gemini Veo Video Generation in Openai Videos Unified Spec (#16229)

* Add veo with openai videos unified specs

* Add videos testing to UI

* remove mock code

* Remove not need ui changes:

* Fix mypy errors related to gemini

* fix test_transform_video_create_request

* Add contant video duration for gemini and vertex

* Fix litellm_mapped_tests tests

* fix azure videos issue

* Added doc for videos vertex ai

* fix seconds param error

* fix lint errors

* test_transform_video_create_response_cost_tracking_no_duration

---------

Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>
Co-authored-by: Ishaan Jaffer <ishaanjaffer0324@gmail.com>
 2025-11-08 16:03:51 -08:00
Krish Dholakia
202eaeb1a2
Revert "(feat) Audio transcription - cost tracking + (feat) image generation …" (#16409) 
This reverts commit c96da44265.
 2025-11-08 15:38:16 -08:00
Krish Dholakia
c96da44265
(feat) Audio transcription - cost tracking + (feat) image generation - accurate cost tracking based on output_format/quality/size 
* feat(audio_transcriptions/): calculate duration of audio file for cost calculation

Fixes https://github.com/BerriAI/litellm/issues/11846

Closes https://github.com/BerriAI/litellm/issues/14605

* fix(cost_calculator.py): correctly use base model, when set

Fixes issue where azure base model was being ignored

* feat(cost_calculator.py): fix default cost tracking quality param for image generation

* feat(image_generations/): return output_format, quality, size

aligns response to openai spec and improves cost tracking accuracy

* fix(cost_calculator.py): refactor cost calculation for image generation to use image response instead of hidden params

* build: update build

* fix: fix cost calculation

* build: update poetry lock

* fix: fix ruff checks

* fix: fix aembedding

* fix: fix ruff errors

* fix: modify to catch errors

* fix: test

* fix: loosen test to handle openai lib out of sync
 2025-11-08 15:30:46 -08:00
Krish Dholakia
bce8a5d6b7
Revert "(feat) audio transcriptions cost tracking (for azure/other non-openai…" (#16402) 
This reverts commit 9a88fe0861.
 2025-11-08 14:24:51 -08:00
Ishaan Jaffer
e4c31add58 test_dd_llms_obs_redaction 2025-11-08 14:23:42 -08:00
Ishaan Jaffer
1bf284abca test_delete_callbacks_in_db 2025-11-08 14:05:44 -08:00
Krish Dholakia
9a88fe0861
(feat) audio transcriptions cost tracking (for azure/other non-openai models) + (fix) image generations - accurate cost tracking for dalle3/gpt-image-1 - uses the correct max image quality (#16076) 
* feat(audio_transcriptions/): calculate duration of audio file for cost calculation

Fixes https://github.com/BerriAI/litellm/issues/11846

Closes https://github.com/BerriAI/litellm/issues/14605

* fix(cost_calculator.py): correctly use base model, when set

Fixes issue where azure base model was being ignored

* feat(cost_calculator.py): fix default cost tracking quality param for image generation

* feat(image_generations/): return output_format, quality, size

aligns response to openai spec and improves cost tracking accuracy

* fix(cost_calculator.py): refactor cost calculation for image generation to use image response instead of hidden params

* build: update build

* fix: fix cost calculation

* build: update poetry lock

* fix: fix ruff checks
 2025-11-08 13:54:37 -08:00
Sameer Kankute
6fb0a8fc58
Added xai responses support (#16391) 
* Added xai responses support

* add the xai provider config above

* remove init file

* remove init file

* Fix f string lint error

---------

Co-authored-by: Ishaan Jaffer <ishaanjaffer0324@gmail.com>
 2025-11-08 12:30:09 -08:00
Ishaan Jaffer
c60a13c919 fixes 2025-11-08 11:11:22 -08:00
Ishaan Jaffer
68d24b40c6 test_gemini_25_implicit_caching_cost 2025-11-08 10:13:04 -08:00
Ishaan Jaffer
45323dd537 test_patch_guardrail_endpoint 2025-11-07 22:09:52 -08:00
Alan Ponnachan
2408e09f6a
feat: Add support for Anthropic Memory Tool (#16115) 
* add memory tool in anthropic.py

* add memory tool test

* make format

* update transformation

* adding memory to hosted tools

* add test

* make format
 2025-11-07 19:27:28 -08:00
YutaSaito
6eb74bd62a
Feat/persist mcp credentials in db (#16308) 
* feat: persist mcp credentials in db

* feat: remove Auth Value field from MCP Tool Testing Playground

* fix: test
 2025-11-07 19:22:49 -08:00
Jack Cherng
2ab34f9a52
Fix HostedVLLMRerankConfig will not be used (#16352) 
* Fix HostedVLLMRerankConfig will not be used

Signed-off-by: Jun-Fei Cherng <jfcherng@realtek.com>

* Fix no usage statistics in rerank with hosted_vllm

Signed-off-by: Jun-Fei Cherng <jfcherng@realtek.com>

* Revise typo in comment

Signed-off-by: Jun-Fei Cherng <jfcherng@realtek.com>

---------

Signed-off-by: Jun-Fei Cherng <jfcherng@realtek.com>
 2025-11-07 19:11:59 -08:00
Ishaan Jaffer
a8533dc5c4 Revert "Added xai responses support (#16310)" 
This reverts commit ee50f09e73.
 2025-11-07 18:39:38 -08:00
Ishaan Jaffer
736b6b3e1e TestVertexAIRerankTransform 2025-11-07 18:18:48 -08:00
Ishaan Jaff
674d4b4cab
[Feat] Guardrails - LiteLLM Content Filter, Allow Viewing/Editing Content Filter Settings (#16383) 
* fix safe dumps

* add patterns.json

* add PrebuiltPattern

* add test patterns

* fix edit and view

* fix backend handling

* fix CF ui edit

* fix init
 2025-11-07 18:15:09 -08:00
Ishaan Jaff
ca229fe030
[Feat] LiteLLM Guardrail - UI Fix, ensure you can see UI Friendly name for PII Patterns (#16382) 
* fix safe dumps

* add patterns.json

* add PrebuiltPattern

* add test patterns
 2025-11-07 18:14:58 -08:00
Alan Ponnachan
5b01fe0a81
fix(vertex_ai): Correctly map 429 Resource Exhausted to RateLimitError (#16363) 2025-11-07 14:36:20 -08:00
Emerson Gomes
940a72ceb0
Add Vertex MiniMAX m2 (#16373) 2025-11-07 14:27:42 -08:00
Sameer Kankute
94e1a1ecac
Use vertex creds passed via arguments (#16266) 2025-11-06 19:35:22 -08:00
Jason Roberts
5733f711fd
feat(guardrails): panw prisma airs guardrail deduplication and enhanced session tracking (#16273) 
* feat(guardrails): Add deduplication and session tracking

- Implement deduplication logic to prevent duplicate scans (via call_id; add _check_and_mark_scanned) caused by LiteLLM callback system
- Add session tracking using litellm_trace_id as AI Session ID for Prisma AIRS SCM logging
- Extract helper methods _extract_prompt_from_request maintainability
- Use httpxSpecialProvider import (LoggingCallback -> GuardrailCallback)
- Add comprehensive tests for deduplication and session tracking (7 new tests)
- Update documentation with multi-turn conversation tracking examples

* docs: update PANW Prisma AIRS multi-turn conversation example to use industry-standard terminology

- Clearer example for conversation tracking
- Updated terminology from 'AI Session ID' to 'Prisma AIRS AI Session ID' for clarity

* fix: remove unused asyncio import

* fix: correct mypy type ignore comment
 2025-11-06 19:34:37 -08:00
Andrii Kislitsyn
c497b6f239
Add retry-after header support for errors 502, 503, 504 (#16288) 
* retry-after-header-support-for-502-503-504-initial

* retry-after-header-support-for-502-503-504-tests-and-linters
 2025-11-06 19:33:29 -08:00
Ishaan Jaff
bd7d653bae
Revert "Update perplexity cost tracking (#15743)" (#16345) 
This reverts commit ad6a0f4d44.
 2025-11-06 19:00:45 -08:00
Ishaan Jaffer
5157078e0a test_delete_callbacks_in_db 2025-11-06 18:37:36 -08:00
Ishaan Jaffer
dc71eb12e3 fix _get_spend_logs_metadata 2025-11-06 17:08:53 -08:00
Ishaan Jaffer
8e73e11fc8 SEARCH_PROVIDERS 2025-11-06 17:07:53 -08:00
Ishaan Jaffer
8a9fe7b056 fix delete callbacks 2025-11-06 17:06:34 -08:00
Ishaan Jaffer
c727a2ad63 test noma, presidio 2025-11-06 17:06:03 -08:00
Ishaan Jaffer
0a4e2a88e3 TestIsAllowedToCallVectorStoreEndpoint 2025-11-06 17:02:59 -08:00
Ishaan Jaff
18a5c4f75a
[Feat] Add LiteLLM Gateway built in guardrail (#16338) 
* add ContentFilterAction

* store pre-built regex patterns

* add v0 of content filter guard

* add _filter_messages

* test content filter guard

* init ContentFilterGuardrail

* fix ContentFilterGuardrail enums

* rename folder

* fix litellm_content_filter

* refactor content filter guard

* test content filter

* add streaming for ContentFilterGuardrail

* test_streaming_hook_mask

* add litellm_content_filter

* docs show litellm content filter

* docs litellm content filter

* fix lnting

* Potential fix for code scanning alert no. 3675: Clear-text logging of sensitive information

Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>

---------

Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>
 2025-11-06 16:02:28 -08:00
Ishaan Jaff
c98b125851
[Feat] OTEL - Log Cost Breakdown on OTEL Logger (#16334) 
* add gen_ai cost metrics

* TestOpenTelemetryCostBreakdown

* fix QA check

* validate_redacted_message_span_attributes
 2025-11-06 13:53:53 -08:00
Sameer Kankute
b762493ec5
Remove aws params from claude4_5 request (#16315) 2025-11-06 10:10:30 -08:00
Sameer Kankute
ee50f09e73
Added xai responses support (#16310) 2025-11-06 08:39:28 -08:00
Krish Dholakia
80440e11a7
Revert "Update MCP version from 1.10.1 to 1.20.0 (#16281)" (#16300) 
This reverts commit 34c12d0131.
 2025-11-05 18:42:34 -08:00
Jamie Goodyear
2c9970742f
[LiteLLM-16250] Proxy to Bedrock will add name to file content, breaks when cache_control in use (#16275) 2025-11-05 18:39:38 -08:00
Guan Zheng Huang
34c12d0131
Update MCP version from 1.10.1 to 1.20.0 (#16281) 
* Update MCP version from 1.10.1 to 1.20.0

- Update mcp dependency: 1.10.1 -> 1.20.0 in requirements.txt, pyproject.toml, and CI config
- Update uvicorn dependency: 0.29.0 -> 0.31.1 (required by MCP 1.20.0)
- Update PyJWT constraint to support newer versions required by MCP
- Update all CI pipeline references to MCP 1.20.0
- Add test to verify MCP version and import compatibility

MCP 1.20.0 requires uvicorn >=0.31.1 and PyJWT >=2.10.1.
MCP package remains Python >=3.10 only (no change to version constraint).

* Update poetry.lock for MCP 1.20.0
 2025-11-05 18:34:28 -08:00