Commit graph

21862 commits

Author SHA1 Message Date
Ishaan Jaff
3ddaf313c2
fix: fix update_db_model (#10726) 2025-05-10 14:28:37 -07:00
Ishaan Jaff
3d8c3d7b96
[DB] Add index for session_id on LiteLLM_SpendLogs (#10727)
* build: add index session_id

* build: add db migration
2025-05-10 14:28:20 -07:00
Ishaan Jaff
214a427038
fix: allow using credentials with amoderation (#10723) 2025-05-10 14:27:57 -07:00
Krish Dholakia
7210b713dc
Add target model name validation (#10722)
* fix(auth_checks.py): enforce auth checks on target model names

ensures user has access to models they are trying to call

* test(test_auth_utils.py): add unit tests for auth check

* fix(exception_mapping_utils.py): handle mistral 429 exception

* fix: fix linting error

* fix(auth_checks.py): add max fallback depth
2025-05-10 14:27:06 -07:00
Ishaan Jaff
02f0e13642
[UI] QA Logs page - Fix bug where log did not remain in focus + text overflow on error logs (#10725)
* logs page, ensure that the selected log remains in focus

* fix error view
2025-05-10 14:20:59 -07:00
Ishaan Jaff
e88ac57176
[UI] Bug Fix - Allow Copying Request / Response on Logs Page (#10720)
* dont show country with API req

* ui: allow copying request / response on logs
2025-05-10 12:54:16 -07:00
Krish Dholakia
b8fe0e057f
complete unified batch id support - replace model in jsonl to be deployment model name (#10719)
* feat(router.py): translate the model in jsonl for create file deployment to use the deployment model name

* test: add unit test for replace model in jsonl

* test(test_router.py): add unit tests

* test: add unit tests
2025-05-10 12:04:01 -07:00
Krrish Dholakia
b13ed6b860 docs(litellm_managed_files.md): cleanup 2025-05-10 11:55:12 -07:00
Krrish Dholakia
5c0b972135 docs(managed_batches.md): add line numbers to code examples 2025-05-10 11:54:17 -07:00
Krrish Dholakia
3ec8a394a6 docs(managed_batches.md): file cleanup 2025-05-10 11:52:39 -07:00
Krrish Dholakia
990d6f5c93 docs(managed_batches.md): add docs on unified file id with batches 2025-05-10 11:28:24 -07:00
Krish Dholakia
8a8dc7ceda
Litellm Unified File ID output file id support (#10713)
* fix(router.py): write file to all deployments

allows unified file id to work across multiple deployments

* fix(view_logs/index.tsx): show call type in request logs

* fix(router.py): pass a deep copy of kwargs to avoid conflict across multiple runs

* fix(batch_utils.py): broaden check

* fix(router_utils.py): handle null type for function name

* fix(proxy_track_cost_callback.py): fix ruff check error

* fix(router.py): handle healthy_deployments as a dict

* feat(managed_files.py): support encoding / decoding unified batch id … (#10711)

* feat(managed_files.py): support encoding / decoding unified batch id when using managed files

allows routing retrieve batch to the right model id

* fix: fix linting error

* feat(managed_files.py): support unified output file id

enables batch output file id to be used to retrieve the actual file

* fix(managed_files.py): attempt to fix ci/cd linting error

* fix: fix ruff check
2025-05-10 11:02:09 -07:00
Ishaan Jaff
2c8f4efd33 (UI) fix regression on Logs page 2025-05-10 09:25:21 -07:00
Ishaan Jaff
8ab4ad1355 fix merge conflicts 2025-05-10 09:08:43 -07:00
Ishaan Jaff
56b5626554 Revert "Filter logs on status and model (#10670)"
This reverts commit d4f7403cb9.
2025-05-10 09:07:49 -07:00
Ishaan Jaff
9bc8130ad5 Revert "Fix log table bugs (after filtering logic was added) (#10712)"
This reverts commit 3fafe37eb9.
2025-05-10 09:07:32 -07:00
Krrish Dholakia
3a782856ea build: update build 2025-05-10 00:26:16 -07:00
Krish Dholakia
9bfd3e4819
fix(router.py): write file to all deployments (#10708)
* fix(router.py): write file to all deployments

allows unified file id to work across multiple deployments

* fix(view_logs/index.tsx): show call type in request logs

* fix(router.py): pass a deep copy of kwargs to avoid conflict across multiple runs

* fix(batch_utils.py): broaden check

* fix(router_utils.py): handle null type for function name

* fix(proxy_track_cost_callback.py): fix ruff check error

* fix(router.py): handle healthy_deployments as a dict

* feat(managed_files.py): support encoding / decoding unified batch id … (#10711)

* feat(managed_files.py): support encoding / decoding unified batch id when using managed files

allows routing retrieve batch to the right model id

* fix: fix linting error

* test: add unit tests

* fix: fix ruff check
2025-05-10 00:08:30 -07:00
tanjiro
3fafe37eb9
Fix log table bugs (after filtering logic was added) (#10712)
* fix status filter

* fix refresh button

* minor

* fix refresh and auto refresh
2025-05-09 23:46:05 -07:00
Mattias Michaux
a7550c0b3d
Update model_prices_and_context_window.json (#10690)
set correct context window length for all gemini 2.5 variants
2025-05-09 23:38:43 -07:00
Clara Pohland
6084d6c437
Azure LLM: fix passing through of azure_ad_token_provider parameter (#10694)
* Azure LLM: fix passing through of azure_ad_token_provider parameter

* add test

---------

Co-authored-by: Clara Luise Pohland <clara-luise.pohland@telekom.de>
2025-05-09 23:38:13 -07:00
Krish Dholakia
6f32189093
fix(caching_handler.py): fix embedding str caching result (#10700)
* fix(caching_handler.py): fix embedding str caching result

Fixes issue where str caching results were not being correctly assembled on str input

* feat(azure/image_generation): Support dropping response_format for azure gpt-image-1

Fixes LIT-118

* test(test_utils.py): add unit testing

* test: rename file to avoid testing conflict
2025-05-09 23:37:02 -07:00
Marc Abramowitz
3787aa27b0
Add management client docs (#10703)
* Add management client docs

* Docs enhancements

* Add "proxy/management_cli" to sidebar

* Small formatting changes

* Add overview table

* Wrap long line

* Add "Quick Start" section

* Document uv tool upgrade and uninstall

* Add blank lines to eliminate yellow squigglies in VS Code

* For each section link to section on swagger used

* Wrap some long command examples

* Break long http request command on to multiple lines
2025-05-09 18:59:24 -07:00
Marc Abramowitz
3d87da555a
Add --version flag to litellm-proxy CLI (#10704)
* Add --version flag to litellm-proxy CLI

```shell
$ litellm-proxy --version
litellm-proxy version: 1.68.1
```

* Return both client and server version

* Update docs

* Add a test for the version command

* Add litellm/proxy/client/health.py
2025-05-09 18:58:37 -07:00
Ishaan Jaff
e5a08a5ae1
[Feat] Add streaming support for using bedrock invoke models with /v1/messages (#10710)
* add basic bedrock transform

* test_anthropic_messages_streaming_bedrock_invoke

* fix: typing ant

* fix: get async response iterator

* fix: code quality check
2025-05-09 18:56:23 -07:00
Ishaan Jaff
3731ee436a
[Refactor] Use pip package for enterprise/ folder (#10709)
* init enterprise pip

* init enterprise pip

* init enterprise pip

* test: enterprise pip

* add litellm-enterprise to pip

* litellm ent check

* litellm ent check

* fix import email router

* fix setup_litellm_enterprise_pip

* fix local testing with enterprise pip
2025-05-09 17:18:48 -07:00
Ishaan Jaff
c50abd3254 bump litellm proxy extras 2025-05-09 14:18:21 -07:00
Ishaan Jaff
a8e7b01608 build: use big int for daily spend tables 2025-05-09 14:16:18 -07:00
Muhammad Husnain
0675dc109e
[FIX] Update token fields in schema.prisma to use BigInt for improved… (#10697)
* [FIX] Update token fields in schema.prisma to use BigInt for improved handling of large values across LiteLLM models.

* [FIX] revert SpendLogs back to Int
2025-05-09 14:14:39 -07:00
Ishaan Jaff
479c6e4017
fix: support for python 3.11- (re datetime UTC) (#10471) (#10701)
* fix support for python 3.11-

3.11 introduced datetime.UTC, this provides a fallback for 3.11-

* use litellm.utils.get_utc_datetime

* remove unused timezone import

Co-authored-by: Matthew Farrellee <matt@cs.wisc.edu>
2025-05-09 14:13:57 -07:00
Krish Dholakia
9c32525c17
build: update model in test (#10706) 2025-05-09 13:33:11 -07:00
Ishaan Jaff
02efffcbcb fix: test_team_update_sc_2 2025-05-09 11:59:13 -07:00
Tom Matthews
e2a9cd0c75
Add nscale support for streaming (#10698) 2025-05-09 11:39:23 -07:00
Ishaan Jaff
1a42245da6 test: fix test_team_update_sc_2 2025-05-09 09:38:03 -07:00
Krrish Dholakia
584facebf1 test: temporarily skip test due to change testing model change - need to update test for new model 2025-05-09 09:02:08 -07:00
GPTLocalhost (Word Add-in)
86f19d2fae
Add GPTLocalhost to "docs/my-website/docs/projects" (#10687)
* Create GPTLocalhost.md

* Update sidebars.js
2025-05-09 07:44:48 -07:00
Emerson Gomes
7372978c1a
New azure models (#9956)
* Adding Azure Deepseek-v3-0324

* Adding Azure Llama4 models

* add function call support to azure deepseek-v3-0324
2025-05-09 07:20:59 -07:00
Krish Dholakia
d6bdd0473a
Extract 'thinking' from nova response + Add 'drop_params' support for gpt-image-1 (#10680)
* test(base_llm_unit_tests.py): return '<thinking>' tag in response content

* fix(converse_transformation.py): extract `<thinking>` block from nova tool use response

Fixes https://github.com/BerriAI/litellm/issues/9063

* fix(factory.py): handle non-signature reasoning blocks to bedrock

pass as text input - bedrock raises ""User messages cannot contain reasoning content. Please remove the r
easoning content and try again." otherwise

* fix(main.py): Add drop params support for gpt

Fixes https://github.com/BerriAI/litellm/issues/10501

* fix(converse_transformation.py): fix linting error

* fix(utils.py): fix linting error

* test: cleanup test

* test: skip test until we have bedrock prompt caching permission
2025-05-08 23:01:38 -07:00
Krish Dholakia
a1964eab18
Realtime API - Set 'headers' in scope for websocket auth requests + reliability fix infinite loop when model_name not found for realtime models (#10679)
* fix(user_api_key_auth.py): add 'headers' to constructed request for websocket

Fix issue on some datastructure versions which require a headers field in scope

* test(test_user_api_key_auth.py): add unit testing for headers in scope change

* fix(router.py): migrate `_arealtime` to generic router endpoint

Fix infinite loop on model name missing for realtime api calls

* test(test_router_helper_utils.py): cleanup test post refactor
2025-05-08 22:50:09 -07:00
Krish Dholakia
5325ee4382
fix(factory.py): Handle system only message to anthropic (#10678)
Fixes error where passing system only message to anthropic causes requests to fail
2025-05-08 22:44:39 -07:00
Krrish Dholakia
96e31edad3 build(proxy_server_config.yaml): move to model with higher quota 2025-05-08 22:18:27 -07:00
Ishaan Jaff
a0ee31edf8
[Feat] Add support for using Bedrock Invoke models in /v1/messages format (#10681)
* fix: add transform_anthropic_messages_request

* fix: add get_requested_response_api_optional_param

* fix: use base llm http handler for anthropic messages

* fix: add anthropic transform response

* fix: transform_anthropic_messages_response

* fix: fixes for anthropic messages

* fix: code qa fixes

* fix: pass thinking to anthropic

* fix: linting

* fixes

* feat: add folder for bedrock invoke messages

* feat: init bedrock invoke messages for anthropic claude family

* test: add bedrock invoke test for us anthropic

* test: test_anthropic_messages_non_streaming_bedrock_invokec

* feat: update anthropic messages transforms

* feat: update anthropic messages transforms

* Update litellm/utils.py

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* fix: test_anthropic_messages_non_streaming

* fix: linting override

* fix: linting error

---------

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
2025-05-08 21:16:47 -07:00
Krrish Dholakia
49deea0df9 test: update test 2025-05-08 21:12:14 -07:00
Ishaan Jaff
42b8be526d ui new build 2025-05-08 20:21:14 -07:00
Ishaan Jaff
9d8f570f14
[Refactor] Anthropic /v1/messages endpoint - Refactor to use base llm http handler and transformations (#10677)
* fix: add transform_anthropic_messages_request

* fix: add get_requested_response_api_optional_param

* fix: use base llm http handler for anthropic messages

* fix: add anthropic transform response

* fix: transform_anthropic_messages_response

* fix: fixes for anthropic messages

* fix: code qa fixes

* fix: pass thinking to anthropic

* fix: linting

* fixes
2025-05-08 17:56:50 -07:00
Ishaan Jaff
5886b96135 bump litellm proxy extras 2025-05-08 16:23:45 -07:00
tanjiro
d4f7403cb9
Filter logs on status and model (#10670)
* added status filtering on logs

* added model filter

* fix linter

* fix model filtering

* modified status filter to use status column from LiteLLM_SpendLogs

* remove json import
2025-05-08 13:50:17 -07:00
Ishaan Jaff
fcd51a8f4b fix: nscale models 2025-05-08 13:49:10 -07:00
Ishaan Jaff
c2ce9c537b fix: model cost map 2025-05-08 12:46:13 -07:00
Ishaan Jaff
e85323e8dd fix: linting 2025-05-08 12:42:42 -07:00