scriptoriumadmin/litellm
1
0
Fork 0
mirror of https://github.com/BerriAI/litellm.git synced 2026-10-06 02:48:13 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 103
28119 commits 3198 branches 1436 tags 4.5 GiB
Commit graph

28119 commits

This branch
This branch
All branches
Author SHA1 Message Date
Ishaan Jaffer
9bde1c1a4c TestEnkryptAIGuardrailHooks 2025-10-10 19:59:41 -07:00
Ishaan Jaffer
8cb66168bc test fix 2025-10-10 19:57:17 -07:00
Ishaan Jaffer
5377b17bdd doc fix 2025-10-10 19:55:02 -07:00
Krrish Dholakia
4ad1b9c9e1 fix(litellm_responses_transformation/transformation.py): several fixes for chat completion to responses api bridge 
Fix tool calling, prevent passing invalid stream options, prevent passing invalid user string (>64 char)

Allows gpt-5-codex to work on claude code
 2025-10-10 19:52:01 -07:00
Alexsander Hamir
0374fe02f7
fix: prevent session leaks when recreating aiohttp sessions (#15443) 
Closes old sessions before creating new ones to prevent accumulation. Replaces 'pass' placeholder with asyncio.create_task(old_session.close()) when sessions are recreated due to event loop changes.
 2025-10-10 19:43:52 -07:00
Alexsander Hamir
5f5a7c917a
fix: handle closed aiohttp sessions with detection and retry (#15442) 
Fixes RuntimeError "Session is closed" by:
- Checking session.closed before use and recreating if needed
- Catching RuntimeError during requests and retrying with new session
- Validating newly created sessions aren't already closed

Adds tests for both proactive detection and reactive retry scenarios.
 2025-10-10 19:42:58 -07:00
Krrish Dholakia
fed1ba2891 feat(openai/): convert chat completion tool calls to responses api 
enables gpt-5-codex to work on claude code

 Closes LIT-1088
 2025-10-10 19:41:11 -07:00
Alexsander Hamir
2c9356c437
[Fix] - shared session parsing and usage issue (#15440) 
* Fix: Add shared_session to all_litellm_params to prevent JSON serialization error

The shared_session parameter (aiohttp.ClientSession) was being passed through
to provider API calls, causing "Object of type ClientSession is not JSON
serializable" errors during embedding requests.

Added shared_session to the all_litellm_params list so it's properly filtered
out as a LiteLLM-internal parameter and not passed to the provider's API.

* Fix: Add shared_session support for embedding calls with connection pooling

The shared_session parameter was not being properly handled in embedding calls,
causing it to be passed through to provider API requests where it's not needed.

Changes:
- Added shared_session to all_litellm_params to filter it from provider API request body
- Extract shared_session in main embedding() function and pass it explicitly
- Updated OpenAI embedding handlers (embedding() and aembedding()) to accept shared_session
- Pass shared_session to _get_openai_client for HTTP client creation

This enables proper connection pooling for embedding requests when shared_session
is provided, improving performance for high-throughput scenarios.

* test: add regression test for shared_session in embedding calls

Add comprehensive test to prevent JSON serialization errors when using
shared_session.

The test verifies two critical aspects:
1. shared_session is in all_litellm_params to prevent "Object of type
   ClientSession is not JSON serializable" errors
2. shared_session flows through the complete call chain across 6 layers:
   - litellm.embedding()
   - OpenAI.embedding/aembedding()
   - _get_openai_client()
   - AsyncHTTPHandler.create_client()
   - _create_async_transport()
   - _create_aiohttp_transport()

Similar to test_acompletion_session_reuse_e2e.py but focused on
embedding endpoints. Uses inspect.getsource() to verify the parameter
is not only accepted but actually passed through each layer.
 2025-10-10 19:26:53 -07:00
Ishaan Jaff
527c8f59fa
[Feat] Tag Management - Add support for setting tag based budgets (#15433) 
* feat: add LiteLLM_TagTable

* fix: use new table for tag management

* fix - allow setting budgets for tags

* working tag creation

* fix schema.prisma

* add tag info

* ui fixes

* ui fix tag info

* TAG_CACHE_IN_MEMORY_TTL_SECONDS

* add Litellm_EntityType

* fix get_aggregated_db_spend_update_transactions

* fix: _update_entity_spend_in_db

* fix _tag_max_budget_check

* add tag budget check

* add tag_list_transactions

* test_get_tag_objects_batch

* test_update_tag_db_without_prisma_client

* fix get_tags_from_request_body

* get_tags_from_request_body

* fix get_tags_from_request_body

* fix spend tracking utils

* get_tags_from_request_body

* test_get_tags_from_request_body_with_metadata_tags

* feat: add _update_tag_cache spend tracking

* fix _PROXY_track_cost_callback

* test_tag_cache_update_multiple_tags

* fix tag info

* docs fix

* docs tag budgets

* doc fix

* docs fix

* fix tag budget

* docs tag budgets

* docs fix

* ruff fix
 2025-10-10 19:24:50 -07:00
Krrish Dholakia
144ea09549 docs(openai.md): document how to get reasoning content for gpt-5 models 2025-10-10 18:42:19 -07:00
Krrish Dholakia
bff46da060 test(test_openai.py): add gpt-5 streaming with reasoning content test 2025-10-10 18:36:41 -07:00
Krrish Dholakia
f49f540b6d feat(litellm_responses_transformation/transformation.py): parse thinking content in response<-> chat completion bridge 
allows gpt-5 to return thinking content when called via responses api
 2025-10-10 18:29:37 -07:00
Achintya Rajan
6b66e12dea
Litellm UI API Reference page updates (#15438) 
* added new CodeBlock for better visuals

* added doc link
 2025-10-10 18:18:31 -07:00
Krrish Dholakia
0cb9ef5fb3 fix(openai/responses): fallback to model construct, don't use it by default (causes downstream errors for nested values) 2025-10-10 18:07:18 -07:00
Pablo Gomez
f27a823256
Deletion of unnecessary and error causing volume section comment (#15425) 2025-10-10 17:51:27 -07:00
Alexsander Hamir
9d7dea42d0
[Fix] - SensitiveDataMasker converts lists to string (#15420) 
* fix: preserve lists in SensitiveDataMasker to prevent string conversion

Added 'list' to allowed primitive types in mask_dict() to prevent lists like
tags from being converted to string representations in API responses.

Before: {"tags": "['East US 2', 'production', 'test']"}
After:  {"tags": ["East US 2", "production", "test"]}

* add: unit test
 2025-10-10 17:50:51 -07:00
Copilot
b9eb05ea63
[Feature]: Include server_name in /v1/mcp/server/health endpoint response (#15431) 
* Initial plan

* Add server_name to MCP health check response

Co-authored-by: ishaan-jaff <29436595+ishaan-jaff@users.noreply.github.com>

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: ishaan-jaff <29436595+ishaan-jaff@users.noreply.github.com>
 2025-10-10 17:37:53 -07:00
Krish Dholakia
bb8dfe46c0
Merge pull request #15435 from BerriAI/litellm_ui_models_and_endpoints_refactor 
Models & Endpoints Initial Refactor
 2025-10-10 16:48:45 -07:00
Achintya Rajan
44ded6c4b5 import fixes & cleanup 2025-10-10 16:39:42 -07:00
Achintya Rajan
5a32a336e6 renamed component 2025-10-10 16:32:35 -07:00
Achintya Rajan
6cf9cce38f Update model_dashboard.tsx 2025-10-10 16:31:28 -07:00
Achintya Rajan
f8f94b71f0 extracted FilterByContent 2025-10-10 16:29:49 -07:00
Achintya Rajan
a7e897a671 Update model_dashboard.tsx 2025-10-10 16:20:24 -07:00
Achintya Rajan
f456dbecbd moved updateModelMetrics down 2025-10-10 16:15:35 -07:00
Achintya Rajan
25252d06d1 moved FilterByContent down 2025-10-10 16:03:07 -07:00
Krrish Dholakia
43826e3cdd fix(litellm-proxy-extras/utils.py): run resolve_all_migrations after running prisma migrate deploy 
catch any drift between prisma schema and db

Fixes reoccuring issues for users where db columns are missing
 2025-10-10 15:55:21 -07:00
Achintya Rajan
29002b4f3b extracted ModelAnalyticsTab.tsx 2025-10-10 15:48:02 -07:00
Achintya Rajan
020cecfe33 ModelRetrySettingsTab and PriceDataManagementTab 2025-10-10 15:32:55 -07:00
Krrish Dholakia
c74eb6403d fix(streaming_iterator.py): return done chunks in the order expected by responses api 
ensures responses api sdk's (e.g. openai ruby) work when calling non-openai models
 2025-10-10 15:30:10 -07:00
Achintya Rajan
ae39196315 removed props, transferred management down 2025-10-10 15:11:22 -07:00
Achintya Rajan
ff06e6d4ea extracted AllModelsTab, eliminating props 2025-10-10 15:02:38 -07:00
Krrish Dholakia
15b5e6f5d9 feat(responses_api/): fix missing streaming events on responses api <-> chat completion bridge 
ensure we are passing the required events when streaming non-openai models via responses api
 2025-10-10 14:53:15 -07:00
Ishaan Jaff
bf209415da
[Feat] VertexAI Gemma model family streaming support + Added MedGemma (#15427) 
* test_acompletion_filters_stream_and_stream_options

* fix: stream_options

* docs medgemma

* lint fix

* docs
 2025-10-10 14:22:27 -07:00
Brandon Cook
a0e81a7f1c
feat: add Bedrock AU Cross-Region Inference (#15402) 2025-10-10 14:18:51 -07:00
Sameer Kankute
a2123f7596
fix lint errors (#15406) 
* fix lint errors

* fix mypy error

* Update docs error
 2025-10-10 14:16:23 -07:00
Ishaan Jaff
ed62d6c943
[Fix] VertexAI - gemma model family support (custom endpoints) (#15419) 
* TestVertexGemmaiCompletion

* test vertex Gemma

* fix file name

* fix file naming

* add VertexAIGemmaModels

* add cost_router for vertexai

* fix main.py

* fix VertexGemmaConfig

* fix Vertex AI Gemma-AI Models Handler

* docs gemma

* fix ids

* test fix

* ruff check fixes

* docs fix

* docs fix

* test_acompletion_basic_request

* Revert "test_acompletion_basic_request"

This reverts commit fdaa5bc49e.

* test_acompletion_basic_request

* fix: async transform

* fix gemma: stream param

* test_acompletion_fake_streaming
 2025-10-10 13:30:43 -07:00
Krish Dholakia
c8b93c3940
Merge pull request #15389 from BerriAI/litellm_usage_spend_per_day_chart_fix 
fix: usage page >> Model Activity >> spend per day graph: y-axis clipping on large spend values
 2025-10-10 13:08:26 -07:00
Krish Dholakia
a83cf6d042
Merge pull request #15416 from BerriAI/litellm_ui_remove_unused_imports 
Enforces removal of unused imports from UI
 2025-10-10 13:07:34 -07:00
Krish Dholakia
192097813a
Merge pull request #15418 from BerriAI/litellm_ui_teams_refactor_2 
LiteLLM Dashboard Teams UI refactor
 2025-10-10 13:06:54 -07:00
Krish Dholakia
419a5e8a2d
Merge pull request #15421 from BerriAI/litellm_ui_guardrails_provider_logos_updates 
Updates guardrail provider logos
 2025-10-10 13:04:34 -07:00
Achintya Rajan
824517ee37 updates guardrail provider logos 2025-10-10 11:39:14 -07:00
Sameer Kankute
513925fd12
Add OCI Cohere support with tool calling and streaming capabilities (#15365) 
* Add cohere models for OCI integration

* fix mypy error
 2025-10-10 11:20:18 -07:00
Achintya Rajan
6d5c76d81b Empty commit for file merge diff 2025-10-10 10:30:28 -07:00
Achintya Rajan
c4ed4916e9 Empty commit for CI/CD 2025-10-10 10:29:37 -07:00
Achintya Rajan
b4444ead0d rebased, removed unused imports 2025-10-10 10:25:48 -07:00
Achintya Rajan
73eafed51c removed unused imports 2025-10-10 10:25:00 -07:00
Achintya Rajan
5fca481ebb removing unused imports 2025-10-10 10:24:57 -07:00
Krish Dholakia
d944717d4b
Merge pull request #15278 from xprilion/wandb-inference-docs 
Add W&B Inference documentation
 2025-10-09 22:35:16 -07:00
Krish Dholakia
9c3e29b9f0
Merge pull request #15330 from jlan-nl/litellm-fix-erroneous-gpt5-cooldown-trigger 
Minimal fix: gpt5 models should not go on cooldown when called with temperature!=1
 2025-10-09 22:34:30 -07:00
Krish Dholakia
ab2f866510
Merge pull request #15363 from vkolehmainen/fix-20250918083359_drop_spec_version_column_from_mcp_table 
Remove hardcoded "public" schema in 20250918083359_drop_spec_version_column_from_mcp_table/migration.sql
 2025-10-09 22:27:02 -07:00