scriptoriumadmin/litellm
1
0
Fork 0
mirror of https://github.com/BerriAI/litellm.git synced 2026-08-28 05:25:59 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 7
43876 commits 3745 branches 1395 tags 3.9 GiB
Commit graph

8121 commits

Author SHA1 Message Date
Ivan Krakhmaliuk
76dd9d8110 GPT-5: Test for unsupported params 2025-09-01 20:30:43 +03:00
retanoj
002c2f16dc fix token count error when proxy gemini cli to openai like model 2025-09-01 20:02:53 +08:00
Krish Dholakia
fd0f47d48d Merge pull request #14005 from moshemorad/bedrock_fix_structure_output 
Bedrock fix structure output
 2025-09-01 14:50:08 +03:00
TomeHirata
1ac7f875c9 fix test 2025-09-01 17:21:47 +09:00
TomeHirata
ca6d77b479 fix citation field name 2025-09-01 16:41:35 +09:00
onlylhf
bfed4e0a6a Add a complete URL generation method for embedding Volcengine API and optimize request and response processing logic; Delete redundant test files and refactor integration testing to improve readability and maintainability. 2025-09-01 13:18:19 +08:00
Krrish Dholakia
2ad77d9bf6 feat(ollama/completion): output parse thinking content on streaming + non-streaming for ollama completion calls 
Completes 'thinking' param support for ollama
 2025-08-31 21:17:10 -07:00
Dharamendra Kumar
4278c3b596 feat: Add dependency injection support to BaseLLMAIOHTTPHandler 
- Add optional client_session, transport, and connector parameters to constructor
- Implement session ownership tracking to prevent closing user-provided sessions
- Add comprehensive session resolution hierarchy (dynamic > instance > create new)
- Include transport control for advanced HTTP stack management
- Add 29 comprehensive tests covering all injection scenarios
- Maintain backward compatibility with existing code

This enhancement allows users to inject their own configured aiohttp sessions,
transports, and connectors for fine-grained control over connection pooling,
SSL settings, proxy configurations, and other HTTP stack parameters.
 2025-08-31 11:14:19 -07:00
Yuta Saito
d83c420d48 feat: Add guardrail for the Anthropic API endpoint 2025-08-31 16:44:01 +09:00
Krrish Dholakia
b6f6dc5c1c feat(vertex_ai.py): support parsing thinking content into gemini format 
allows function calls with thought signatures to be sent back to gemini

  Closes https://github.com/BerriAI/litellm/pull/13842
 2025-08-30 18:49:34 -07:00
John Lathouwers
cf5a695571
Merge branch 'BerriAI:main' into main 2025-08-30 22:03:39 +01:00
gotsysdba
07d9d14b49
Update OCIResposeClass 2025-08-30 22:02:00 +01:00
Ishaan Jaff
34275ab273 Revert "Merge pull request #14005 from moshemorad/bedrock_fix_structure_output" 
This reverts commit 75c327cb9e, reversing
changes made to afa687182a.
 2025-08-30 13:37:10 -07:00
John Lathouwers
d137371026
Merge branch 'BerriAI:main' into main 2025-08-30 21:17:28 +01:00
gotsysdba
1ae5466daf
Test Case 2025-08-30 21:15:54 +01:00
Ishaan Jaff
7ff93cd1b1 Revert "Merge pull request #12865 from aholmberg/holmberg/default-reasoning" 
This reverts commit 68f4847659, reversing
changes made to df39320889.
 2025-08-30 13:07:33 -07:00
gotsysdba
a27604b67a
Fixes #14090 2025-08-30 20:57:57 +01:00
Ishaan Jaff
de6b08e76e test_supports_tool_choice 2025-08-30 10:35:16 -07:00
Krish Dholakia
8b11452d9e
Merge branch 'main' into add-vercel-ai-gateway-provider 2025-08-29 22:27:08 -07:00
Krish Dholakia
ead7e7259b
Merge pull request #13152 from betterthanbreakfast/bug-openmeter-user-id-from-token 
[Bug]: Set user from token user_id for OpenMeter integration
 2025-08-29 22:21:26 -07:00
Krish Dholakia
75c327cb9e
Merge pull request #14005 from moshemorad/bedrock_fix_structure_output 
Bedrock fix structure output
 2025-08-29 22:04:51 -07:00
Ishaan Jaff
562790cb31 fix: /v1/realtime check 2025-08-29 18:38:14 -07:00
frankzye1
75a7b39700 pass function tool description for databricks provider 
Signed-off-by: frankzye1 <frankzye@qq.com>
 2025-08-30 09:27:39 +08:00
Ishaan Jaff
607f425cc6
[Bug]: Fix Can't set reasoning_effort for DeepSeek-V3.1 on DeepInfra by default (#14053) 
* fix DeepInfraConfig

* track supports reasoning

* fix: get_supported_openai_params

* test_deepseek_supported_openai_params
 2025-08-29 13:15:43 -07:00
Krish Dholakia
bcd38dde2e
Merge pull request #14036 from gotsysdba/main 
OCI Provider: add oci_key_file as an optional_parameter
 2025-08-29 06:27:09 -07:00
Krish Dholakia
fffccc0675
Merge pull request #14042 from WilsonSunBritten/10988-set-truncation-threshold 
Allow configuration to set threshold before request entry in spend log gets truncated
 2025-08-29 06:24:28 -07:00
Krish Dholakia
dd33e3f928
Merge pull request #14050 from lmwang9527/fix/logging-worker-graceful-shutdown 
Fix LoggingWorker graceful shutdown to prevent CancelledError warnings
 2025-08-29 06:23:53 -07:00
WilsonSunBritten
b3edff02e8 fix mock tests 2025-08-28 17:40:16 -06:00
Ishaan Jaff
37a599932d
[Bug Fix] Virtual keys with llm_api type cause Internal Server Error when using /anthropic/* and other llm passthrough routes (#14046) 
* fix route checks passthrough_routes_wildcard

* refactor check_route_access

* test_virtual_key_llm_api_route_includes_passthrough_prefix
 2025-08-28 15:14:09 -07:00
lmwang
6760a6aaa1 Fix LoggingWorker graceful shutdown to prevent CancelledError warnings 
- Change exception logging to debug level during shutdown cancellation
- Add clear_queue() call to process remaining coroutines during shutdown
- Prevents 'RuntimeWarning: coroutine was never awaited' messages
- Add comprehensive tests for graceful shutdown behavior

Fixes #13970
 2025-08-28 14:15:17 -07:00
gotsysdba
15f1d82536
Fix tests 2025-08-28 14:54:38 +01:00
gotsysdba
87abfc3c11
Adding oci_key_file support for oci provider 2025-08-28 14:37:53 +01:00
TomuHirata
7333060fb0 feat(databricks): add anthropic citation support 2025-08-28 18:43:43 +09:00
Mohse Morad
8cce5e6307 Added tests 2025-08-28 12:21:35 +03:00
李海峰
2d0a57a719 Add Volcengine embedding module with handler and transformation logic 
- Implemented VolcEngineEmbeddingHandler for synchronous and asynchronous embedding requests.
- Created VolcEngineEmbeddingConfig for transforming requests and responses to/from Volcengine format.
- Added integration tests for embedding functionality, covering various scenarios including error handling and parameter validation.
- Established test structure for Volcengine embedding, ensuring compliance with LiteLLM testing patterns.
- Included comprehensive tests for parameter mapping, request transformation, and response handling.
 2025-08-28 15:05:11 +08:00
Krish Dholakia
e638c5490d
Merge pull request #13945 from NoWall57/feature/force-anthropic-custom-api-base 
feat: Add support for custom Anthropic-compatible API endpoints
 2025-08-27 22:28:55 -07:00
Krish Dholakia
74c9663189
Merge pull request #12797 from ericgtkb/fix-vertex-ai-anyof 
Fix additional anyOf corner cases for Vertex AI Gemini tool calls - issue #11164
 2025-08-27 22:26:13 -07:00
Krish Dholakia
fc738c5822
Merge branch 'main' into holmberg/default-reasoning 2025-08-27 22:25:22 -07:00
Krish Dholakia
c43706061f
Merge branch 'main' into heroku-llms 2025-08-27 22:21:36 -07:00
Krish Dholakia
67d432a6d7
Merge pull request #13243 from nagyv-team/main 
Ensure that `function_call_prompt` extends system messages following its current schema
 2025-08-27 22:16:53 -07:00
Krish Dholakia
10816a44d4
Merge pull request #13160 from pangeacyber/aidr-update 
Update Pangea Guardrail to support new AIDR endpoint
 2025-08-27 22:15:32 -07:00
Krish Dholakia
a913ecf60b
Merge branch 'main' into feat/github-copilot-thinking-reasoning-support 2025-08-27 22:09:15 -07:00
Ishaan Jaff
04dc1a5351
[Feat] Add support for returning images with gemini/gemini-2.5-flash-image-preview with /chat/completions (#13983) 
* add gemini-2.5-flash-image-preview

* add gemini-2.5-flash-image-preview

* add image in ChatCompletionResponseMessage

* test_gemini_image_generation_async

* Revert "Merge pull request #13394 from Deviad/feature/enhance_logging_for_containers"

This reverts commit 539b94ad4e, reversing
changes made to 71af7bcf9c.

* include `image` in Delta

* fix _process_candidates should show the image response

* fix: _handle_special_delta_attributes

* test_gemini_image_generation_async_stream

* image_generation_chat

* UI - allow looking at generated images from /chat/completions

* _create_streaming_choice

* fix import StreamingChoices

* fix ChatCompletionResponseMessage

* test_gemini_image_generation

* add gemini img migration

* fix _extract_candidate_metadata

* ui fix

* fix batch endpoint test
 2025-08-27 16:16:19 -07:00
Xingyao Wang
9acf80b8ad
Fix issue #13995: Handle None metadata in batch requests (#13996) 
* Fix issue #13995: Handle None metadata in batch requests

- Added null check in add_key_level_controls method to prevent NoneType error
- Updated type hint to Optional[dict] for better type safety
- Added comprehensive test suite to verify the fix works correctly
- All existing tests pass, confirming no regression

Fixes #13995

Co-authored-by: openhands <openhands@all-hands.dev>

* Move test file to tests/test_litellm/proxy/ directory

- Moved test_batch_metadata_none_fix.py from tests/ to tests/test_litellm/proxy/
- Updated import structure to match existing test patterns
- This ensures the test runs in GitHub Actions as requested by @krrishdholakia

Co-authored-by: openhands <openhands@all-hands.dev>

---------

Co-authored-by: openhands <openhands@all-hands.dev>
 2025-08-27 14:51:09 -07:00
Ishaan Jaff
165242e31f
[Feat] langfuse_otel logger - allow using LANGFUSE_OTEL_HOST for configuring host (#14013) 
* feat - add _get_langfuse_otel_host

* test_get_langfuse_otel_config_with_otel_host_priority

* docs: LANGFUSE_OTEL_HOST
 2025-08-27 14:43:47 -07:00
Sam Xie
ee13c65701
[Bug]: /responses endpoint proxy ignores extra_headers in GitHub Copilot (#13775) 
* fix: pass extra_headers parameter through responses API transformation chain

Ensure extra_headers parameter is properly forwarded from the responses() function
through the transformation handler and config to maintain header propagation in
litellm_completion_request dict.

* Add tests
 2025-08-27 12:35:09 -07:00
Ishaan Jaff
4fff05f1cc
[Feature]: Support Gemini requests with only system prompt (#14010) 
* _default_user_message_when_system_message_passed

* test_system_prompt_only_adds_blank_user_message

* test_system_message_with_no_user_message
 2025-08-27 12:12:47 -07:00
Ishaan Jaff
76f78f7b2a fix: LITELLM_LOG_FILE test 2025-08-27 08:25:24 -07:00
Krish Dholakia
900bd10905
Merge branch 'main' into feature/enhance_logging_for_containers 2025-08-26 23:21:15 -07:00
Krish Dholakia
71af7bcf9c
Merge pull request #13374 from blahgeek/fix-count-special-token 
Fix token_counter with special token input
 2025-08-26 23:19:26 -07:00