scriptoriumadmin/litellm
1
0
Fork 0
mirror of https://github.com/BerriAI/litellm.git synced 2026-10-08 03:08:45 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 115
46359 commits 3285 branches 1443 tags 4.8 GiB
Commit graph

1000 commits

Author SHA1 Message Date
Krrish Dholakia
d4cc748b40 feat(openrouter.py): add image generation via google on openrouter support 
Closes https://github.com/BerriAI/litellm/pull/14151
 2025-09-01 18:39:41 -07:00
Krrish Dholakia
4b8f2dc051 test: update tests 2025-09-01 18:20:46 -07:00
Krrish Dholakia
0fbcc01173 feat(types/utils.py): support 'images' field in output 
Compatible with Openrouter response format

 Closes https://github.com/BerriAI/litellm/pull/14151
 2025-09-01 18:05:16 -07:00
Ishaan Jaff
4c7b3e10ed
[Bug Fix] Gemini Tool Calling - fix gemini empty enum property (#14155) 
* fix: _convert_schema_types

* fix recursive detector

* test_convert_schema_types_type_array_conversion

* fix: DEFAULT_NUM_WORKERS_LITELLM_PROXY

* add _fix_enum_empty_strings

* test_tool_call_with_empty_enum_property

* test_fix_enum_empty_strings

* fix _fix_enum_empty_strings
 2025-09-01 17:05:01 -07:00
Ishaan Jaff
2331fb45d5
[Bug]: Gemini 2.5 Pro – schema validation fails with OpenAI-style type arrays in tools (#14154) 
* fix: _convert_schema_types

* fix recursive detector

* test_convert_schema_types_type_array_conversion

* fix: DEFAULT_NUM_WORKERS_LITELLM_PROXY
 2025-09-01 16:53:20 -07:00
TomeHirata
e334f48699 Add supported text field to anthropic citation response 2025-09-01 16:46:54 +09:00
Krrish Dholakia
b6f6dc5c1c feat(vertex_ai.py): support parsing thinking content into gemini format 
allows function calls with thought signatures to be sent back to gemini

  Closes https://github.com/BerriAI/litellm/pull/13842
 2025-08-30 18:49:34 -07:00
Krrish Dholakia
51c73dc60b fix(vertex_and_google_ai_studio_gemini.py): bubble up thoughtsignature back to client 2025-08-30 17:26:18 -07:00
Ishaan Jaff
c9b86775a7 test_system_message_with_no_user_message 2025-08-30 13:09:25 -07:00
Ishaan Jaff
3cb8e23dfe Revert "Merge pull request #14026 from TomeHirata/citation-supported-text" 
This reverts commit afa687182a, reversing
changes made to 55186dad4c.
 2025-08-30 11:23:18 -07:00
Ishaan Jaff
dc6161535a fix for supports_reasoning check added in 68f4847659 2025-08-30 11:06:02 -07:00
Ishaan Jaff
3fc2198d27 fix: test_gemini_image_generation_async_stream 2025-08-30 10:33:23 -07:00
Ishaan Jaff
efebf7ec7b test_gemini_image_generation_async 2025-08-30 10:29:24 -07:00
Ishaan Jaff
fd39f22e3e test_completion_openrouter_reasoning_content 2025-08-30 09:27:37 -07:00
Krish Dholakia
afa687182a
Merge pull request #14026 from TomeHirata/citation-supported-text 
Add supported text field to anthropic citation response
 2025-08-29 22:02:18 -07:00
Ishaan Jaff
5ee7a675f2
[Bug]: grok-4 does not support frequency_penalty, litellm should drop this param for grok-4 (#14078) 
* fix: _supports_frequency_penalty

* _supports_frequency_penalty

* test_xai_grok_4_frequency_penalty_not_supported
 2025-08-29 17:31:48 -07:00
Oscar Craviotto
323cabb98d
fix: ensure Gemini calls include a user message (fixes #9733) 
When a Gemini model supports system messages, messages with the
system role get filtered out from the list of messages, so never
appended to the content (and passed as system_instructions instead).

This causes a 400 error, as both generateContent and streamGenerateContent
require contents (the systemInstruction object is optional).

This is a quick fix that does not check whether there was a system msg
or if the model supports it. It simply makes sure that if the list
of messages passed to  _gemini_convert_messages_with_history()
yields no contents, that there is at least a single (empty space)
user message set to prevent the call from failing.

The potential side effect of this is that we will not fail
a call lacking any message at all, but if that needs to be checked
(and AFAIU the OpenAI spec messages are required) it should be done
elsewhere (e.g. validate_and_fix_openai_messages() @ litellm/utils.py).

PS: Applied `make format` (`poetry run black [file changed]`) only to
files changed by me. The `make lint` command identified a bunch of
files requiring formatting that I am not modifying as I did not touch
them.
 2025-08-29 20:41:58 +03:00
Tomu Hirata
11228e946b add supported text field to anthropic citation response 2025-08-28 14:39:00 +09:00
Krish Dholakia
f01c7d13e7
Merge pull request #13991 from ikaadil/fix/gemini-imagen-model-name-validation 
[Bug Fix] Resolve invalid model name error for Gemini Imagen models
 2025-08-27 21:56:19 -07:00
Ishaan Jaff
04dc1a5351
[Feat] Add support for returning images with gemini/gemini-2.5-flash-image-preview with /chat/completions (#13983) 
* add gemini-2.5-flash-image-preview

* add gemini-2.5-flash-image-preview

* add image in ChatCompletionResponseMessage

* test_gemini_image_generation_async

* Revert "Merge pull request #13394 from Deviad/feature/enhance_logging_for_containers"

This reverts commit 539b94ad4e, reversing
changes made to 71af7bcf9c.

* include `image` in Delta

* fix _process_candidates should show the image response

* fix: _handle_special_delta_attributes

* test_gemini_image_generation_async_stream

* image_generation_chat

* UI - allow looking at generated images from /chat/completions

* _create_streaming_choice

* fix import StreamingChoices

* fix ChatCompletionResponseMessage

* test_gemini_image_generation

* add gemini img migration

* fix _extract_candidate_metadata

* ui fix

* fix batch endpoint test
 2025-08-27 16:16:19 -07:00
Ifta Khairul Alam Adil
75aeca14b2 Merge branch 'main' into fix/gemini-imagen-model-name-validation 2025-08-27 21:52:47 +02:00
Ishaan Jaff
4fff05f1cc
[Feature]: Support Gemini requests with only system prompt (#14010) 
* _default_user_message_when_system_message_passed

* test_system_prompt_only_adds_blank_user_message

* test_system_message_with_no_user_message
 2025-08-27 12:12:47 -07:00
Ifta Khairul Alam Adil
0f5b31fd78 fix: resolve invalid model name error for Gemini Imagen models (#13851) 
- Fix URL construction in Gemini image generation to strip 'gemini/' prefix
- Google AI API expects base model name without the prefix
- Update model references and pricing information for consistency
- Remove outdated image generation pricing entries

Fixes issue where models like 'gemini/imagen-4.0-fast-generate-preview-06-06'
were being rejected by the Google AI API due to incorrect URL formatting.
 2025-08-27 04:17:35 +02:00
Steven Anton
d95578464e Add support for AWS assume_role with a session token 2025-08-23 22:37:21 -07:00
Ishaan Jaff
b9132968b2
[Perf] Improvements for Async Success Handler (Logging Callbacks) - Approx +130 RPS (#13905) 
* [Performance] Reduce Significant CPU overhead from litellm_logging.py (#13895)

* fix: litellm.configured_cold_storage_logger

* fix Session Management - Non-OpenAI Models docs

* ruff fix

* test fix

* create LoggingWorker

* add GLOBAL_LOGGING_WORKER for async task handling

* fix logging tests

* add conftest

* fix conftest

* test fix location of encode bedrock runtime modelid arn

* fix conftest.py

* tuning LoggingWorker

* conftest.py

* fix conftest batches/

* test_async_chat_azure

* event_loop

* test_bedrock_streaming_passthrough_test2

* fix GLOBAL_LOGGING_WORKER

* logging worker

* add flush for global logging worker

* Revert "fix GLOBAL_LOGGING_WORKER"

This reverts commit d254f508f4.

* fix conftest clear_queue

* fix conftest clear_queue

* setup_and_teardown for llm translation

* docs AWS_REGION

* test_async_chat_azure

* change test DIR

* run ci/cd again

* use 1 job for litellm_router_unit_testing

* fix space

* fix litellm_router_unit_testing

* test_aaarouter_dynamic_cooldown_message_retry_time

* litellm_router_unit_testing

* conftest.py clearing qu

* fixes litellm_router_unit_testing

* fixes clear_queue

* fix router_unit_tests

* remove conftest

* add back conftest for router

* fix event loop test

* test fix

* fixes for LoggingWorker

* ruff fix
 2025-08-23 13:13:23 -07:00
Krrish Dholakia
4d7433b5ad test: reduce mistral direct tests b/c of rate limit errors 2025-08-23 11:15:03 -07:00
Krish Dholakia
f0b0ee51e0
Merge branch 'main' into litellm_dev_08_15_2025_p1 2025-08-23 10:51:56 -07:00
Krish Dholakia
9c4a86e98e
Merge pull request #13868 from kankute-sameer/litellm_feat_voyage_context_3_embedding_model 
[Feat] Add support for voyage-context-3 embedding model
 2025-08-21 23:18:20 -07:00
Ishaan Jaff
e93e266f84
[Performance] Use O(1) Set lookups for model routing (#13879) 
* o(1) lookups

* Revert "o(1) lookups"

This reverts commit 620d142469.

* o(1) lookups

* Revert "o(1) lookups"

This reverts commit 676a9f5bcc.

* o(1) lookups

* register_model fix

* test_aget_valid_models

* lambda ai models fix

* test_utils.py

* test fix vertex ai
 2025-08-21 22:56:46 -07:00
Sameer Kankute
5ac4fb512c Add support for voyage-context-3 embedding model 2025-08-22 00:15:12 +05:30
Krish Dholakia
422447b7f1
Responses API - add default api version for openai responses api calls + Openrouter - fix claude-sonnet-4 on openrouter + Azure - Handle openai/v1/responses 
Responses API - add default api version for openai responses api calls + Openrouter - fix claude-sonnet-4 on openrouter + Azure - Handle `openai/v1/responses`
 2025-08-18 18:59:28 -07:00
Ishaan Jaff
ba0881d728
[Bug Fix] image_edit() function returns APIConnectionError with litellm_proxy - Support for both image edits and image generations (#13735) 
* add image edits litellm proxy on SDK

* add image gen provider

* add IMG Gen support for litellm_proxy provider
 2025-08-18 18:26:32 -07:00
Krrish Dholakia
ff7bdb6290 fix(mistral/chat/transformation.py): handle empty message content for mistral calls 
Fixes https://github.com/BerriAI/litellm/issues/13355
 2025-08-16 00:41:11 -07:00
Krrish Dholakia
b83b1686c2 feat(support-allowed_openai_params-for-responses-api): Fixes https://github.com/BerriAI/litellm/issues/13559 2025-08-16 00:23:37 -07:00
Jugal D. Bhatt
aea0605eed
[LLM Translation] Fix Realtime API endpoint for no intent (#13476) 
* fix intent params

* Add responses

* fix unrelated test

* test fix - fireworks API endpoint is down

* test fix fireworks ai is having an active outage

* test_completion_cost_databricks

* dbrx fix test API currently not responding

* Update OpenAI Realtime handler to use the correct endpoint and include all query parameters. Adjusted error messages for missing API base and key. Updated health check URL construction to pass model as a query parameter.

* Enhance OpenAI Realtime handler tests to ensure model parameter inclusion in WebSocket URL. Added new tests to verify correct URL construction with model and additional parameters, preventing 'missing_model' errors. Updated existing tests for consistency.

* Remove debug print statements for API base and key in OpenAIRealtime handler to clean up the code.

---------

Co-authored-by: Ishaan Jaff <ishaanjaffer0324@gmail.com>
 2025-08-14 16:24:14 -07:00
Krish Dholakia
f00e891004
LiteLLM SDK <-> Proxy: support user param + Prisma - remove use_prisma_migrate flag - redundant as this is now default (#13555) 
* fix(litellm_proxy/chat/transformation.py): support 'user' and all other openai chat completion params

Fixes issue where 'user' was not being sent in request to litellm proxy via sdk

* fix(prisma_migration.py): remove 'use_prisma_migrate' flag, is now default

* docs: cleanup docs

* fix(proxy_cli.py): remove --use_prisma_migrate flag

* refactor: remove references to use_prisma_migrate env var

This is now the default flow for db migrations
 2025-08-12 22:03:39 -07:00
Ishaan Jaff
828e3e3deb
[Feat] Add Streaming support + Docs for bedrock gpt-oss model family (#13346) 
* add openai.gpt-oss-20b-1:0

* update BEDROCK_CONVERSE_MODELS

* openai.gpt-oss-20b-1:0 fixes

* fix PDF input

* fix for should_fake_stream

* TestBedrockGPTOSS

* should_fake_stream

* update supports vision field for openai.gpt-oss models

* fixes for bedrock gpt oss

* fixes for should_fake_stream

* docs bedrock gpt oss models
 2025-08-12 08:39:36 -07:00
Krrish Dholakia
79e262d12b feat(common_utils.py): make default azure openai responses api use /openai/v1/responses logic 
Fixes https://github.com/BerriAI/litellm/issues/13527#issuecomment-3177882103
 2025-08-11 23:40:05 -07:00
Ishaan Jaff
9f78287000
[Bug Fix]: Azure OpenAI GPT-5 max_tokens + reasoning param support (#13510) 
* add AzureOpenAIGPT5Config

* add AzureOpenAIGPT5Config

* add AzureOpenAIGPT5Config

* add AzureOpenAIGPT5Config

* test_azure_gpt5_supports_reasoning_effort

* test_azure_gpt5_reasoning

* test_azure_gpt5_reasoning

* ruff check fixes

* docs azure gpt5
 2025-08-11 15:40:53 -07:00
Ishaan Jaff
1cd827874f
[Bug Fix] - Allow using reasoning_effort for gpt-5 model family and reasoning for Responses API (#13475) 
* test_openai_gpt5_reasoning

* test_openai_gpt5_reasoning_effort_parameter

* add OpenAIGPT5ResponsesAPIConfig

* test_openai_gpt5_reasoning_effort_parameter

* fixes
 2025-08-10 09:55:36 -07:00
Ishaan Jaff
621b3dca7b
[Bug Fix] Mistral Tool Calling - Grammar error: at 3(11): failed to compile JSON schema (#13389) 
* test_claude_tool_use_with_gemini

* add _remove_json_schema_refs

* add _clean_tool_schema_for_mistral

* fixes mistral tool calls

* _remove_json_schema_refs

* fix - vertex, remove hardcoded test
 2025-08-07 13:50:22 -07:00
Ishaan Jaff
5a02eb473b test_function_calling_with_tool_response 2025-08-05 09:55:47 -07:00
Krish Dholakia
416da066eb
fix(main.py): handle tool being a pydantic object (#13274) 
* fix(main.py): handle tool being a pydantic object

Fixes https://github.com/BerriAI/litellm/issues/13064

* fix(prompt_templates/common_utils.py): fix unpack defs deepcopy issue

Fixes https://github.com/BerriAI/litellm/issues/13151

* fix(utils.py): handle tools is none
 2025-08-04 23:44:02 -07:00
Ishaan Jaff
cf4c639dad test fix xai - it goes through base llm tests already 2025-07-30 18:18:49 -07:00
Krrish Dholakia
09cc748871 test: handle api instability 2025-07-30 16:32:23 -07:00
Krrish Dholakia
378db1b62d test: remove o1-preview 2025-07-28 17:47:57 -07:00
Krrish Dholakia
16af2d9a50 test: skip dbrx claude 3-7 sonnet test - rate limit errors 2025-07-28 17:34:42 -07:00
Krish Dholakia
1737cf4257
VertexAI - camelcase optional params for image generation + Anthropic - streaming, always ensure assistant role set on only first chunk (#12889) 
* fix(vertex_ai/image_generation): transform `_` param to camelcase

Fixes https://github.com/BerriAI/litellm/issues/12690

* test(test_vertex_image_generation.py): add unit tests

* fix(streaming_handler.py): assert only 1 assistant chunk in stream

Fixes https://github.com/BerriAI/litellm/issues/12616

* fix(streaming_handler.py): fix check
 2025-07-27 10:09:43 -07:00
Ishaan Jaff
461cd0c30a test_completion_cost_deepseek 2025-07-23 13:16:12 -07:00
Ishaan Jaff
c21dc46a33 fix morph api tests 2025-07-22 18:44:44 -07:00