scriptoriumadmin/litellm
1
0
Fork 0
mirror of https://github.com/BerriAI/litellm.git synced 2026-09-07 08:26:10 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 89
26525 commits 4135 branches 1402 tags 2.1 GiB
Commit graph

3723 commits

Author SHA1 Message Date
Ishaan Jaffer
bcf53c6ceb working - errors from bedrock through pass throughs 2025-10-16 15:40:42 -07:00
Ishaan Jaffer
51f1907e98 test_bedrock_error_handling_returns_actual_error 2025-10-16 15:36:51 -07:00
Ishaan Jaffer
cc5eac4965 fix _add_deployment_model_to_endpoint_for_llm_passthrough_route 2025-10-16 15:10:54 -07:00
Ishaan Jaffer
da72a812c9 test_bedrock_passthrough_router 2025-10-16 14:23:34 -07:00
Ishaan Jaff
6f69b009c8
[Fix] Dall-e-2 for Image Edits API (#15604) 
* feat: add DallE2ImageEditConfig

* fix: get_openai_image_edit_config

* fix get_provider_image_edit_config

* fix _add_image_to_files

* fix DallE2ImageEditConfig

* fix TestOpenAIImageEditDallE2
 2025-10-16 13:24:24 -07:00
Krish Dholakia
3bf32e8e5c
feature: update pillar security integration to support no persistence mode in litellm proxy 
feature: update pillar security integration to support no persistence mode in litellm proxy
 2025-10-16 12:13:37 -07:00
Ariel Fogel
59c3aa02c3 respond to review comments 2025-10-16 20:38:49 +03:00
Ariel Fogel
b075cf4a6c PLR-2400: support no persistence in litellm proxy 2025-10-16 17:22:58 +03:00
berri-teddy
1b55803c76 test: reduce EntraID app roles tests to essential scenarios 
- Keep only 2 focused tests: single role extraction and no roles claim
- Remove complex fixtures and edge case tests
- Maintain clean, maintainable test coverage
 2025-10-15 18:15:04 -07:00
berri-teddy
6b166e046d fix: correct EntraID app roles JWT claim extraction 
- Fix get_app_roles_from_id_token to use 'app_roles' claim instead of 'roles'
- Add comprehensive unit tests for EntraID app roles functionality
- Prevent regressions in Microsoft EntraID SSO authentication

Resolves issue where EntraID app roles were not being extracted correctly
from JWT tokens, causing authentication failures for users with assigned
app roles in Microsoft EntraID.
 2025-10-15 18:06:03 -07:00
Ishaan Jaffer
fd83b292a6 test_bedrock_anthropic_prompt_caching 2025-10-15 17:46:44 -07:00
Ishaan Jaff
bc26845ec4
[Feat] Native /ocr endpoint support (#15573) 
* [Feat] Add native litellm.ocr() functions (#15567)

* fix get_supported_ocr_params

* add get_provider_ocr_config

* init OCR

* init ocr functions

* add OCRResponse Base Model

* add ocr to llm http handlers

* add main.py for OCR

* fix linting for OCR

* TestMistralOCR

* update to use DocumentType for Mistral

* fix _prepare_ocr_request

* fix transform

* add main.py for OCR

* add spec to init

* fix OCR

* TestMistralOCR

* ruff fix

* Potential fix for code scanning alert no. 3521: Clear-text logging of sensitive information

Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>

---------

Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>

* [Feat] Add /ocr route on LiteLLM AI Gateway - Adds support for native mistral ocr calling (#15571)

* fix get_supported_ocr_params

* add get_provider_ocr_config

* init OCR

* init ocr functions

* add OCRResponse Base Model

* add ocr to llm http handlers

* add main.py for OCR

* fix linting for OCR

* TestMistralOCR

* update to use DocumentType for Mistral

* fix _prepare_ocr_request

* fix transform

* add main.py for OCR

* add spec to init

* fix OCR

* TestMistralOCR

* ruff fix

* add router.ocr() methods

* add OCR routes

* feat add ocr routes

* add OCR routes

* feat: add OCR routes in proxy server

* working /ocr routes

* test_router_aocr_with_mistral

* docs Mistral OCR

* docs OCR

* [Feat] Add Azure AI Mistral OCR Integration  (#15572)

* fix get_supported_ocr_params

* add get_provider_ocr_config

* init OCR

* init ocr functions

* add OCRResponse Base Model

* add ocr to llm http handlers

* add main.py for OCR

* fix linting for OCR

* TestMistralOCR

* update to use DocumentType for Mistral

* fix _prepare_ocr_request

* fix transform

* add main.py for OCR

* add spec to init

* fix OCR

* TestMistralOCR

* ruff fix

* add router.ocr() methods

* add OCR routes

* feat add ocr routes

* add OCR routes

* feat: add OCR routes in proxy server

* working /ocr routes

* test_router_aocr_with_mistral

* docs Mistral OCR

* docs OCR

* add azure ai to get_provider_ocr_config

* add AzureAIOCRConfig

* TestAzureAIOCR

* TestAzureAIOCR

* test fixes for azure ai ocr

* fix async OCR transform for Azure

* fix transform_ocr_request

---------

Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>
 2025-10-15 17:20:01 -07:00
Ishaan Jaff
f13eb283e1
[Fix] GEMINI - CLI - add google_routes to llm_api_routes (#15500) 
* fix: add google_routes to llm_api_routes

* test: test_virtual_key_llm_api_routes_allows_google_routes
 2025-10-13 10:58:47 -07:00
Krish Dholakia
2ea7005c40
Merge pull request #15448 from dhruvyad/main 
Get completion cost directly from OpenRouter
 2025-10-12 22:12:52 -07:00
Krrish Dholakia
1949436047 test: update test 2025-10-12 21:58:06 -07:00
Krish Dholakia
04dc9d091c
Merge branch 'main' into litellm_staging_branch_10_11_2025_p1 2025-10-12 21:55:44 -07:00
Dhruv Yadav
84a65440c5 add tests for openrouter cost tracking 2025-10-12 12:36:24 +05:30
Copilot
f5359ba007
Fix apply_guardrail endpoint returning raw string instead of ApplyGuardrailResponse (#15436) 
* Initial plan

* Fix apply_guardrail endpoint to return ApplyGuardrailResponse

Co-authored-by: ishaan-jaff <29436595+ishaan-jaff@users.noreply.github.com>

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: ishaan-jaff <29436595+ishaan-jaff@users.noreply.github.com>
 2025-10-11 17:50:37 -07:00
Krrish Dholakia
6bd722bba4 fix: always retain config models 2025-10-11 16:09:33 -07:00
Krrish Dholakia
090a2c0cfe fix: remove unused uuid import 2025-10-11 15:38:28 -07:00
Krish Dholakia
323389c244
(feat) Passthrough - allow admin to give access to specific passthrough endpoints 
(feat) Passthrough - allow admin to give access to specific passthrough endpoints
 2025-10-11 13:51:10 -07:00
Krish Dholakia
5507d50acf
Merge branch 'main' into litellm_dev_10_09_2025_p1 2025-10-11 13:06:34 -07:00
Krish Dholakia
ff930dc6df
Merge branch 'main' into litellm_dev_10_10_2025_p2 2025-10-11 13:03:16 -07:00
Krish Dholakia
07ba5c1ea9
Merge branch 'main' into litellm_dev_10_10_2025_p3 2025-10-11 12:58:53 -07:00
Ishaan Jaffer
97626f2d02 fix test 2025-10-11 11:30:19 -07:00
Ishaan Jaffer
ce6102d54f test_azure_responses_api_status_error 2025-10-11 10:41:15 -07:00
Ishaan Jaffer
86881a8fc1 test_azure_responses_api_status_error 2025-10-11 10:21:07 -07:00
Ishaan Jaffer
33a4676184 test fix 2025-10-11 10:05:22 -07:00
Ishaan Jaffer
732618f55f test_together_ai_embedding 2025-10-11 09:33:19 -07:00
Ishaan Jaffer
c467986b8b test_gemini_context_caching_with_ttl 2025-10-11 09:20:20 -07:00
Ishaan Jaffer
a163bf9f51 test fix: MicrosoftSSOHandler 2025-10-11 08:55:23 -07:00
Ishaan Jaffer
3aa0f6ed72 test_generate_key_with_object_permission 2025-10-11 08:54:15 -07:00
Ishaan Jaffer
4d79280175 Gemma test fixes 2025-10-11 08:51:08 -07:00
Ishaan Jaffer
87719640eb test tag mgmt 2025-10-11 08:48:37 -07:00
Sameer Kankute
138bdbb6d8
fix mapped tests 1 (#15445) 
* fix mapped tests

* fix mapped tests
 2025-10-11 08:33:08 -07:00
Ishaan Jaffer
9bde1c1a4c TestEnkryptAIGuardrailHooks 2025-10-10 19:59:41 -07:00
Ishaan Jaffer
8cb66168bc test fix 2025-10-10 19:57:17 -07:00
Krrish Dholakia
4ad1b9c9e1 fix(litellm_responses_transformation/transformation.py): several fixes for chat completion to responses api bridge 
Fix tool calling, prevent passing invalid stream options, prevent passing invalid user string (>64 char)

Allows gpt-5-codex to work on claude code
 2025-10-10 19:52:01 -07:00
Alexsander Hamir
5f5a7c917a
fix: handle closed aiohttp sessions with detection and retry (#15442) 
Fixes RuntimeError "Session is closed" by:
- Checking session.closed before use and recreating if needed
- Catching RuntimeError during requests and retrying with new session
- Validating newly created sessions aren't already closed

Adds tests for both proactive detection and reactive retry scenarios.
 2025-10-10 19:42:58 -07:00
Krrish Dholakia
fed1ba2891 feat(openai/): convert chat completion tool calls to responses api 
enables gpt-5-codex to work on claude code

 Closes LIT-1088
 2025-10-10 19:41:11 -07:00
Alexsander Hamir
2c9356c437
[Fix] - shared session parsing and usage issue (#15440) 
* Fix: Add shared_session to all_litellm_params to prevent JSON serialization error

The shared_session parameter (aiohttp.ClientSession) was being passed through
to provider API calls, causing "Object of type ClientSession is not JSON
serializable" errors during embedding requests.

Added shared_session to the all_litellm_params list so it's properly filtered
out as a LiteLLM-internal parameter and not passed to the provider's API.

* Fix: Add shared_session support for embedding calls with connection pooling

The shared_session parameter was not being properly handled in embedding calls,
causing it to be passed through to provider API requests where it's not needed.

Changes:
- Added shared_session to all_litellm_params to filter it from provider API request body
- Extract shared_session in main embedding() function and pass it explicitly
- Updated OpenAI embedding handlers (embedding() and aembedding()) to accept shared_session
- Pass shared_session to _get_openai_client for HTTP client creation

This enables proper connection pooling for embedding requests when shared_session
is provided, improving performance for high-throughput scenarios.

* test: add regression test for shared_session in embedding calls

Add comprehensive test to prevent JSON serialization errors when using
shared_session.

The test verifies two critical aspects:
1. shared_session is in all_litellm_params to prevent "Object of type
   ClientSession is not JSON serializable" errors
2. shared_session flows through the complete call chain across 6 layers:
   - litellm.embedding()
   - OpenAI.embedding/aembedding()
   - _get_openai_client()
   - AsyncHTTPHandler.create_client()
   - _create_async_transport()
   - _create_aiohttp_transport()

Similar to test_acompletion_session_reuse_e2e.py but focused on
embedding endpoints. Uses inspect.getsource() to verify the parameter
is not only accepted but actually passed through each layer.
 2025-10-10 19:26:53 -07:00
Ishaan Jaff
527c8f59fa
[Feat] Tag Management - Add support for setting tag based budgets (#15433) 
* feat: add LiteLLM_TagTable

* fix: use new table for tag management

* fix - allow setting budgets for tags

* working tag creation

* fix schema.prisma

* add tag info

* ui fixes

* ui fix tag info

* TAG_CACHE_IN_MEMORY_TTL_SECONDS

* add Litellm_EntityType

* fix get_aggregated_db_spend_update_transactions

* fix: _update_entity_spend_in_db

* fix _tag_max_budget_check

* add tag budget check

* add tag_list_transactions

* test_get_tag_objects_batch

* test_update_tag_db_without_prisma_client

* fix get_tags_from_request_body

* get_tags_from_request_body

* fix get_tags_from_request_body

* fix spend tracking utils

* get_tags_from_request_body

* test_get_tags_from_request_body_with_metadata_tags

* feat: add _update_tag_cache spend tracking

* fix _PROXY_track_cost_callback

* test_tag_cache_update_multiple_tags

* fix tag info

* docs fix

* docs tag budgets

* doc fix

* docs fix

* fix tag budget

* docs tag budgets

* docs fix

* ruff fix
 2025-10-10 19:24:50 -07:00
Krrish Dholakia
144ea09549 docs(openai.md): document how to get reasoning content for gpt-5 models 2025-10-10 18:42:19 -07:00
Krrish Dholakia
bff46da060 test(test_openai.py): add gpt-5 streaming with reasoning content test 2025-10-10 18:36:41 -07:00
Krrish Dholakia
f49f540b6d feat(litellm_responses_transformation/transformation.py): parse thinking content in response<-> chat completion bridge 
allows gpt-5 to return thinking content when called via responses api
 2025-10-10 18:29:37 -07:00
Krrish Dholakia
0cb9ef5fb3 fix(openai/responses): fallback to model construct, don't use it by default (causes downstream errors for nested values) 2025-10-10 18:07:18 -07:00
Alexsander Hamir
9d7dea42d0
[Fix] - SensitiveDataMasker converts lists to string (#15420) 
* fix: preserve lists in SensitiveDataMasker to prevent string conversion

Added 'list' to allowed primitive types in mask_dict() to prevent lists like
tags from being converted to string representations in API responses.

Before: {"tags": "['East US 2', 'production', 'test']"}
After:  {"tags": ["East US 2", "production", "test"]}

* add: unit test
 2025-10-10 17:50:51 -07:00
Copilot
b9eb05ea63
[Feature]: Include server_name in /v1/mcp/server/health endpoint response (#15431) 
* Initial plan

* Add server_name to MCP health check response

Co-authored-by: ishaan-jaff <29436595+ishaan-jaff@users.noreply.github.com>

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: ishaan-jaff <29436595+ishaan-jaff@users.noreply.github.com>
 2025-10-10 17:37:53 -07:00
Krrish Dholakia
15b5e6f5d9 feat(responses_api/): fix missing streaming events on responses api <-> chat completion bridge 
ensure we are passing the required events when streaming non-openai models via responses api
 2025-10-10 14:53:15 -07:00
Ishaan Jaff
bf209415da
[Feat] VertexAI Gemma model family streaming support + Added MedGemma (#15427) 
* test_acompletion_filters_stream_and_stream_options

* fix: stream_options

* docs medgemma

* lint fix

* docs
 2025-10-10 14:22:27 -07:00