scriptoriumadmin/litellm
1
0
Fork 0
mirror of https://github.com/BerriAI/litellm.git synced 2026-09-09 22:31:41 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 87
46529 commits 4181 branches 1403 tags 2.3 GiB
Commit graph

14665 commits

Author SHA1 Message Date
Utkash Dubey
7dbc434318 remove incorrect moderations mode 2025-03-10 19:33:40 -07:00
Utkash Dubey
a9d50a615b move test file to correct location 2025-03-10 19:21:51 -07:00
Utkash Dubey
45f0ca68c4
Merge branch 'BerriAI:main' into main 2025-03-10 19:21:37 -07:00
Ishaan Jaff
9dcc25d63b Merge branch 'main' into litellm_fix_team_model_access_checks 2025-03-10 19:05:11 -07:00
Ishaan Jaff
8811d4cd11 generate_key 2025-03-10 19:03:50 -07:00
Krish Dholakia
c58941d49c
Merge branch 'main' into litellm_dev_03_06_2025_p4 2025-03-10 18:41:10 -07:00
Krrish Dholakia
68bd05ac24 fix(base_invoke_transformation.py): support extra_headers on bedrock invoke route 
Fixes https://github.com/BerriAI/litellm/issues/9106
 2025-03-10 16:13:11 -07:00
Ishaan Jaff
94667e1cf0
Merge pull request #8386 from minwhoo/triton-completions-streaming-fix 
Fix triton streaming completions bug
 2025-03-10 16:07:19 -07:00
Krrish Dholakia
bfbe26b91d feat(azure.py): add azure bad request error support 2025-03-10 15:59:06 -07:00
Krrish Dholakia
5f87dc229a feat(openai.py): bubble all error information back to client 2025-03-10 15:27:43 -07:00
Krrish Dholakia
c1ec82fbd5 refactor: instrument body param to bubble up on exception 2025-03-10 15:21:04 -07:00
Utkash Dubey
495b3dc026 feedback 2025-03-10 14:52:30 -07:00
Utkash Dubey
c7a04140e9 Revert "changes" 
This reverts commit fa88bc9632.
 2025-03-10 14:46:21 -07:00
Utkash Dubey
1f17daf52c
Merge branch 'BerriAI:main' into main 2025-03-10 14:42:30 -07:00
omrishiv
0674491386
add support for Amazon Nova Canvas model (#7838) 
* add initial support for Amazon Nova Canvas model

Signed-off-by: omrishiv <327609+omrishiv@users.noreply.github.com>

* adjust name to AmazonNovaCanvas and map function variables to config

Signed-off-by: omrishiv <327609+omrishiv@users.noreply.github.com>

* tighten model name check

Signed-off-by: omrishiv <327609+omrishiv@users.noreply.github.com>

* fix quality mapping

Signed-off-by: omrishiv <327609+omrishiv@users.noreply.github.com>

* add premium quality in config

Signed-off-by: omrishiv <327609+omrishiv@users.noreply.github.com>

* support all Amazon Nova Canvas tasks

* remove unused import

Signed-off-by: omrishiv <327609+omrishiv@users.noreply.github.com>

* add tests for image generation tasks and fix payload

Signed-off-by: omrishiv <327609+omrishiv@users.noreply.github.com>

* add missing util file

Signed-off-by: omrishiv <327609+omrishiv@users.noreply.github.com>

* update model prices backup file

Signed-off-by: omrishiv <327609+omrishiv@users.noreply.github.com>

* remove image tasks other than text->image

Signed-off-by: omrishiv <327609+omrishiv@users.noreply.github.com>

---------

Signed-off-by: omrishiv <327609+omrishiv@users.noreply.github.com>
Co-authored-by: Krish Dholakia <krrishdholakia@gmail.com>
 2025-03-10 08:02:00 -07:00
vivek-athina
cd4a53d6f2
Merge pull request #4 from BerriAI/main 
Update main
 2025-03-10 11:13:21 +05:30
Krish Dholakia
f899b828cf
Support openrouter reasoning_content on streaming (#9094) 
* feat(convert_dict_to_response.py): support openrouter format of reasoning content

* fix(transformation.py): fix openrouter streaming with reasoning content

Fixes https://github.com/BerriAI/litellm/issues/8193#issuecomment-270892962

* fix: fix type error
 2025-03-09 20:03:59 -07:00
Krrish Dholakia
c08705517b test: fix test 2025-03-09 19:40:03 -07:00
Krish Dholakia
65ef65d360
feat: prioritize api_key over tenant_id for more Azure AD token provi… (#8701) 
* feat: prioritize api_key over tenant_id for more Azure AD token provider (#8318)

* fix: prioritize api_key over tenant_id for Azure AD token provider

* test: Add test for Azure AD token provider in router

* fix: fix linting error

---------

Co-authored-by: you-n-g <you-n-g@users.noreply.github.com>
 2025-03-09 18:59:37 -07:00
Krish Dholakia
e00d4fb18c
Litellm dev 03 08 2025 p3 (#9089) 
* feat(ollama_chat.py): pass down http client to ollama_chat

enables easier testing

* fix(factory.py): fix passing images to ollama's `/api/generate` endpoint

Fixes https://github.com/BerriAI/litellm/issues/6683

* fix(factory.py): fix ollama pt to handle templating correctly
 2025-03-09 18:20:56 -07:00
Ishaan Jaff
83062b892b fix test_list_key_helper_team_filtering 2025-03-08 17:21:32 -08:00
Ishaan Jaff
6e3b21775f test_cost_azure_openai_prompt_caching 2025-03-08 16:19:28 -08:00
Ishaan Jaff
b41311bb21
(UI) - Fix show correct count of internal user keys on Users Page (#9082) 
* get_user_key_counts

* fix get_user_key_counts

* fix get_user_key_counts

* test_get_users_filters_dashboard_keys

* remove unused func
 2025-03-08 16:13:18 -08:00
Ishaan Jaff
73df319f4e
(Clean up) - Allow switching off storing Error Logs in DB (#9084) 
* fix - cleanup, dont store ErrorLogs in 2 tables

* async_post_call_failure_hook

* docs disable error logs

* disable_error_logs
 2025-03-08 16:12:03 -08:00
Krish Dholakia
4330ef8e81
Fix batches api cost tracking + Log batch models in spend logs / standard logging payload (#9077) 
* feat(batches/): fix batch cost calculation - ensure it's accurate

use the correct cost value - prev. defaulting to non-batch cost

* feat(batch_utils.py): log batch models to spend logs + standard logging payload

makes it easy to understand how cost was calculated

* fix: fix stored payload for test

* test: fix test
 2025-03-08 11:47:25 -08:00
Teja Vishwanadha
8c049dfffc
support bytes.IO for audio transcription (#9071) 2025-03-08 08:47:15 -08:00
Ishaan Jaff
e2d612efd9
Bug fix - String data: stripped from entire content in streamed Gemini responses (#9070) 
* _strip_sse_data_from_chunk

* use _strip_sse_data_from_chunk

* use _strip_sse_data_from_chunk

* use _strip_sse_data_from_chunk

* _strip_sse_data_from_chunk

* test_strip_sse_data_from_chunk

* _strip_sse_data_from_chunk

* testing

* _strip_sse_data_from_chunk
 2025-03-07 21:06:39 -08:00
Krish Dholakia
5591354309
Support master key rotations (#9041) 
* feat(key_management_endpoints.py): adding support for rotating master key

* feat(key_management_endpoints.py): support decryption-re-encryption of models in db, when master key rotated

* fix(user_api_key_auth.py): raise valid token is None error earlier

enables easier debugging with api key hash in error message

* feat(key_management_endpoints.py): rotate any env vars

* fix(key_management_endpoints.py): uncomment check

* fix: fix linting error
 2025-03-06 23:13:30 -08:00
Krrish Dholakia
805679becc feat(handle_jwt.py): support multiple jwt url's 2025-03-06 23:05:54 -08:00
Krish Dholakia
274147bc5e
fix(team_endpoints.py): ensure 404 raised when team not found (#9038) 
* fix(team_endpoints.py): ensure 404 raised when team not found

* fix(key_management_endpoints.py): fix adding tags to key when metadata is empty

* fix(key_management_endpoints.py): refactor set metadata field to use common function across keys + teams

reduces scope for errors + easier testing

* fix: fix linting error
 2025-03-06 22:04:36 -08:00
Ishaan Jaff
04e839d846
(AWS Secret Manager) - Using K/V pairs in 1 AWS Secret (#9039) 
* fixes for primary_secret_kv_pairs

* _parse_primary_secret

* Using K/V pairs in 1 AWS Secret

* test_primary_secret_functionality
 2025-03-06 19:30:18 -08:00
Ishaan Jaff
b02af305de
[Feat] - Display thinking tokens on OpenWebUI (Bedrock, Anthropic, Deepseek) (#9029) 
* if merge_reasoning_content_in_choices

* _optional_combine_thinking_block_in_choices

* stash changes

* working merge_reasoning_content_in_choices with bedrock

* fix litellm_params accessor

* fix streaming handler

* merge_reasoning_content_in_choices

* _optional_combine_thinking_block_in_choices

* test_bedrock_stream_thinking_content_openwebui

* merge_reasoning_content_in_choices

* fix for _optional_combine_thinking_block_in_choices

* linting error fix
 2025-03-06 18:32:58 -08:00
Ogun Oz
85d1427710
Fix: Create RedisClusterCache when startup nodes provided in cache args of router (#9010) 
Co-authored-by: Ogün Öz <ogun.oz@cobrainer.com>
 2025-03-06 17:14:32 -08:00
Ishaan Jaff
cca71b41b3 test_e2e_batches_files 2025-03-06 08:42:17 -08:00
Ishaan Jaff
3d87bf9fe6 test fix await_batch_completion 2025-03-06 08:15:45 -08:00
Ishaan Jaff
1d2a9e423c test - remove anthropic_adapter tests. no longer used 2025-03-06 06:47:35 -08:00
Ishaan Jaff
29dc67a2aa test fix anthopic completion 2025-03-06 06:42:26 -08:00
Ishaan Jaff
f47987e673
(Refactor) /v1/messages to follow simpler logic for Anthropic API spec (#9013) 
* anthropic_messages_handler v0

* fix /messages

* working messages with router methods

* test_anthropic_messages_handler_litellm_router_non_streaming

* test_anthropic_messages_litellm_router_non_streaming_with_logging

* AnthropicMessagesConfig

* _handle_anthropic_messages_response_logging

* working with /v1/messages endpoint

* working /v1/messages endpoint

* refactor to use router factory function

* use aanthropic_messages

* use BaseConfig for Anthropic /v1/messages

* track api key, team on /v1/messages endpoint

* fix get_logging_payload

* BaseAnthropicMessagesTest

* align test config

* test_anthropic_messages_with_thinking

* test_anthropic_streaming_with_thinking

* fix - display anthropic url for debugging

* test_bad_request_error_handling

* test_anthropic_messages_router_streaming_with_bad_request

* fix ProxyException

* test_bad_request_error_handling_streaming

* use provider_specific_header

* test_anthropic_messages_with_extra_headers

* test_anthropic_messages_to_wildcard_model

* fix gcs pub sub test

* standard_logging_payload

* fix unit testing for anthopic /v1/messages support

* fix pass through anthropic messages api

* delete dead code

* fix anthropic pass through response

* revert change to spend tracking utils

* fix get_litellm_metadata_from_kwargs

* fix spend logs payload json

* proxy_pass_through_endpoint_tests

* TestAnthropicPassthroughBasic

* fix pass through tests

* test_async_vertex_proxy_route_api_key_auth

* _handle_anthropic_messages_response_logging

* vertex_credentials

* test_set_default_vertex_config

* test_anthropic_messages_litellm_router_non_streaming_with_logging

* test_ageneric_api_call_with_fallbacks_basic

* test__aadapter_completion
 2025-03-06 00:43:08 -08:00
Krrish Dholakia
320cb1d51a docs: cleanup 'signature_delta' from docs 2025-03-05 23:53:38 -08:00
Krish Dholakia
744e10b0f0
Litellm dev 03 05 2025 p3 (#9023) 
* fix(invoke_handler.py): fix converse streaming - return signature + ensure consistency with anthropic api response

* build(model_prices_and_context_window.json): fix anthropic api claude-3-7 max output tokens

with beta header this is 128k

Resolves https://github.com/BerriAI/litellm/issues/8964

* feat(handler.py): handle new anthropic 'thinking_delta' block on streaming

Fixes https://github.com/BerriAI/litellm/issues/8825
 2025-03-05 22:31:39 -08:00
Krish Dholakia
f6535ae6ad
Support format param for specifying image type (#9019) 
* fix(transformation.py): support a 'format' parameter for image's

allow user to specify mime type

* fix: pass mimetype via 'format' param

* feat(gemini/chat/transformation.py): support 'format' param for gemini

* fix(factory.py): support 'format' param on sync bedrock converse calls

* feat(bedrock/converse_transformation.py): support 'format' param for bedrock async calls

* refactor(factory.py): move to supporting 'format' param in base helper

ensures consistency in param support

* feat(gpt_transformation.py): filter out 'format' param

don't send invalid param to openai

* fix(gpt_transformation.py): fix translation

* fix: fix translation error
 2025-03-05 19:52:53 -08:00
Krish Dholakia
ec4f665e29
Return signature on anthropic streaming + migrate to signature field instead of signature_delta [MINOR bump] (#9021) 
* Fix missing signature_delta in thinking blocks when streaming from Claude 3.7 (#8797)

Co-authored-by: Krish Dholakia <krrishdholakia@gmail.com>

* test: update test to enforce signature found

* feat(refactor-signature-param-to-be-'signature'-instead-of-'signature_delta'): keeps it in sync with anthropic

* fix: fix linting error

---------

Co-authored-by: Martin Krasser <krasserm@googlemail.com>
 2025-03-05 19:33:54 -08:00
Ishaan Jaff
8d6815ce98 Revert "(UI) - Security Improvement, move to JWT Auth for Admin UI Sessions (#8995)" 
This reverts commit 01a44a4e47.
 2025-03-05 08:49:20 -08:00
Krish Dholakia
c69ec66dc5
fix(base_aws_llm.py): remove region name before sending in args (#8998) 
* fix(base_aws_llm.py): remove region name before sending in args

* fix(base_aws_llm.py): fix optional param pop position

* fix: fix linting error
 2025-03-04 23:05:28 -08:00
Krish Dholakia
5e386c28b2
Litellm dev 03 04 2025 p3 (#8997) 
* fix(core_helpers.py): handle litellm_metadata instead of 'metadata'

* feat(batches/): ensure batches logs are written to db

makes batches response dict compatible

* fix(cost_calculator.py): handle batch response being a dictionary

* fix(batches/main.py): modify retrieve endpoints to use @client decorator

enables logging to work on retrieve call

* fix(batches/main.py): fix retrieve batch response type to be 'dict' compatible

* fix(spend_tracking_utils.py): send unique uuid for retrieve batch call type

create batch and retrieve batch share the same id

* fix(spend_tracking_utils.py): prevent duplicate retrieve batch calls from being double counted

* refactor(batches/): refactor cost tracking for batches - do it on retrieve, and within the established litellm_logging pipeline

ensures cost is always logged to db

* fix: fix linting errors

* fix: fix linting error
 2025-03-04 21:58:03 -08:00
Ishaan Jaff
01a44a4e47
(UI) - Security Improvement, move to JWT Auth for Admin UI Sessions (#8995) 
* (UI) - Improvements to session handling logic  (#8970)

* add cookieUtils

* use utils for clearing cookies

* on logout use clearTokenCookies

* ui use correct clearTokenCookies

* navbar show userEmail on UserID page

* add timestamp on token cookie

* update generate_authenticated_redirect_response

* use common getAuthToken

* fix clearTokenCookies

* fixes for get auth token

* fix invitation link sign in logic

* Revert "fix invitation link sign in logic"

This reverts commit 30e5308cb3.

* fix getAuthToken

* update setAuthToken

* fix ui session handling

* fix ui session handler

* bug fix stop generating LiteLLM Virtual keys for access

* working JWT insert into cookies

* use central place to build UI JWT token

* add _validate_ui_token

* fix ui session handler

* fix fetchWithCredentials

* check allowed routes for ui session tokens

* expose validate_session endpoint

* validate session endpoint

* call sso/session/validate

* getUISessionDetails

* ui move to getUISessionDetails

* /sso/session/validate

* fix cookie utils

* use getUISessionDetails

* use ui_session_id

* "/spend/logs/ui" in spend_tracking_routes

* working sign in JWT flow for proxy admin

* allow proxy admin to access ui routes

* use check_route_access

* update types

* update login method

* fixes to ui session handler

* working flow for admin and internal users

* fixes for invite links

* use JWTs for SSO sign in

* fix /invitation/new flow

* fix code quality checks

* fix _get_ui_session_token_from_cookies

* /organization/list

* ui sso sign in

* TestUISessionHandler

* TestUISessionHandler
 2025-03-04 21:48:23 -08:00
Ishaan Jaff
42931638df
(bug fix) - Fix Cache Health Check for Redis when redis_version is float (#8979) 
* fix allow flexible types for redis version

* test_cache_ping_with_redis_version_float

* test_cache_ping_with_redis_version_float
 2025-03-04 21:26:18 -08:00
Krish Dholakia
f1a44d1fdc
fix(common_utils.py): handle $id in response schema when calling vert… (#8991) 
* fix(common_utils.py): handle $id in response schema when calling vertex ai

Fixes issue where `$id` present in response_schema was not accepted by vertex ai

* test(test_vertex.py): add unit test to ensure $id stripped out of vertex schema
 2025-03-04 21:19:50 -08:00
Krish Dholakia
662c59adcf
Support caching on reasoning content + other fixes (#8973) 
* fix(factory.py): pass on anthropic thinking content from assistant call

* fix(factory.py): fix anthropic messages to handle thinking blocks

Fixes https://github.com/BerriAI/litellm/issues/8961

* fix(factory.py): fix bedrock handling for assistant content in messages

Fixes https://github.com/BerriAI/litellm/issues/8961

* feat(convert_dict_to_response.py): handle reasoning content + thinking blocks in chat completion block

ensures caching works for anthropic thinking block

* fix(convert_dict_to_response.py): pass all message params to delta block

ensures streaming delta also contains the reasoning content / thinking block

* test(test_prompt_factory.py): remove redundant test

anthropic now supports assistant as the first message

* fix(factory.py): fix linting errors

* fix: fix code qa

* test: remove falsy test

* fix(litellm_logging.py): fix str conversion
 2025-03-04 21:12:16 -08:00
Sunny Wan
ce55a2d06e
Merge branch 'BerriAI:main' into main 2025-03-04 18:00:58 -05:00