Commit graph

11430 commits

Author SHA1 Message Date
Marc Abramowitz
bdb995f436 Revert = None changes
because @krrishdholakia said [here](https://github.com/BerriAI/litellm/pull/3600#discussion_r1599108453)
that they were problematic in the past.
2024-05-13 16:49:36 -07:00
Marc Abramowitz
1efa2e65ac Merge branch 'main' into msabramo/fix-pydantic-warnings 2024-05-13 11:34:06 -07:00
Marc Abramowitz
73541b1f17 Merge branch 'msabramo/pydantic_replace_root_validator_with_model_validator' into msabramo/fix-pydantic-warnings 2024-05-13 11:25:55 -07:00
Marc Abramowitz
e261a9b2c2 Use Config class w/ arbitrary_types_allowed for pydantic v1
pydantic v2 warns about using a Config class.
But without this, pydantic v1 will raise an error:
    RuntimeError: no validator found for <class 'openai.Timeout'>,
    see `arbitrary_types_allowed` in Config
Putting arbitrary_types_allowed = True in the ConfigDict doesn't work in pydantic v1.
So we mostly use model_config = ConfigDict(...) and then only use the Config
class with arbitrary_types_allowed = True for pydantic v1.
2024-05-13 11:22:50 -07:00
Marc Abramowitz
f233cde36c Replace some Config classes with model_config
in `litellm/types/router.py`

I didn't replace the ones that have `allow_arbitrary_types` because changing
those seems to break pydantic v1 compatibility.
2024-05-13 11:04:13 -07:00
Krrish Dholakia
04ae285001 fix(vertex_ai.py): support tool call list response async completion 2024-05-13 10:42:31 -07:00
Marc Abramowitz
4ccdbfd67d model_config = ConfigDict instead of model_config = dict 2024-05-13 10:38:01 -07:00
Marc Abramowitz
eacafa7cd7 Revert litellm/types/router.py changes
These don't work in pydantic v1.
2024-05-13 10:34:05 -07:00
Marc Abramowitz
abd90e6296 Revert root_validator => model_validator change
I have it in a separate PR:

https://github.com/BerriAI/litellm/pull/3611
2024-05-13 10:33:04 -07:00
Marc Abramowitz
ad8c3ac2c3 Change pydantic root_validator to model_validator
pydantic v1 uses `root_validator` and pydantic v2 uses `model_validator`.

pydantic v2 emits a warning when `root_validator` is used. E.g.:

```
litellm/proxy/_types.py:225
  /Users/abramowi/Code/OpenSource/litellm/litellm/proxy/_types.py:225:
  PydanticDeprecatedSince20: Pydantic V1 style `@root_validator`
  validators are deprecated.
  You should migrate to Pydantic V2 style `@model_validator` validators,
  see the migration guide for more details.
  Deprecated in Pydantic V2.0 to be removed in V3.0.
  See Pydantic V2 Migration Guide at
  https://errors.pydantic.dev/2.7/migration/
    @root_validator(pre=True)
```

This change eliminates those warnings with pydantic v2, while retaining
compatibility with pydantic v1.

pydantic 2.7.1 before

```
$ env -i PATH=$PATH poetry run pytest litellm/tests/test_proxy_server.py
...
litellm/proxy/_types.py:225
  /Users/abramowi/Code/OpenSource/litellm/litellm/proxy/_types.py:225: PydanticDeprecatedSince20: Pydantic V1 style `@root_validator` validators are deprecated. You should migrate to Pydantic V2 style `@model_validator` validators, see the migration guide for more details. Deprecated in Pydantic V2.0 to be removed in V3.0. See Pydantic V2 Migration Guide at https://errors.pydantic.dev/2.7/migration/
    @root_validator(pre=True)
...
========================== 10 passed, 2 skipped, 39 warnings in 8.67s ===========================
```

pydantic 2.7.1 after

```
$ env -i PATH=$PATH poetry run pytest litellm/tests/test_proxy_server.py
...
========================== 10 passed, 2 skipped, 27 warnings in 9.85s ===========================
```

pydantic 1.10.5 after

```
$ poetry run pip install 'pydantic<2'
...
Successfully installed pydantic-1.10.15
$ env -i PATH=$PATH poetry run pytest litellm/tests/test_proxy_server.py
...
=========================== 10 passed, 2 skipped, 1 warning in 8.13s ============================
```
2024-05-13 10:23:44 -07:00
Krrish Dholakia
7f6e933372 fix(router.py): give an 'info' log when fallbacks work successfully 2024-05-13 10:17:32 -07:00
Krrish Dholakia
13e1577753 fix(slack_alerting.py): don't fire spam alerts when backend api call fails 2024-05-13 10:04:43 -07:00
Krrish Dholakia
b063ef7a47 bump: version 1.37.5 → 1.37.6 2024-05-13 09:08:04 -07:00
Krrish Dholakia
5342b3dc05 fix(router.py): fix error message to return if pre-call-checks + allowed model region 2024-05-13 09:04:38 -07:00
Krrish Dholakia
c3293474dd fix(proxy_server.py): return 'allowed-model-region' in headers 2024-05-13 08:48:16 -07:00
Ishaan Jaff
514c5737f8
Merge pull request #3587 from BerriAI/litellm_proxy_use_batch_completions_model_csv
[Feat] Use csv values for proxy batch completions (OpenAI Python compatible)
2024-05-13 07:55:12 -07:00
Marc Abramowitz
abe2514ba1 Add my commit to .git-blame-ignore-revs
because I made a lot of fairly mindless changes to pydantic code to fix warnings
and I don't want git blame to give people the impression that I know more about
this code than I do.
2024-05-12 10:21:10 -07:00
Marc Abramowitz
876840e995 Update pydantic code to fix warnings
Before:

```shell
$ env -i PATH=$PATH poetry run pytest litellm/tests/test_proxy_server.py
====================================== test session starts ======================================
platform darwin -- Python 3.12.3, pytest-7.4.4, pluggy-1.5.0
rootdir: /Users/abramowi/Code/OpenSource/litellm
plugins: anyio-4.3.0, asyncio-0.23.6, mock-3.14.0
asyncio: mode=Mode.STRICT
collected 12 items

litellm/tests/test_proxy_server.py s..........s                                           [100%]

======================================= warnings summary ========================================
../../../Library/Caches/pypoetry/virtualenvs/litellm-Fe7WjZrx-py3.12/lib/python3.12/site-packages/pydantic/_internal/_config.py:284: 25 warnings
  /Users/abramowi/Library/Caches/pypoetry/virtualenvs/litellm-Fe7WjZrx-py3.12/lib/python3.12/site-packages/pydantic/_internal/_config.py:284: PydanticDeprecatedSince20: Support for class-based `config` is deprecated, use ConfigDict instead. Deprecated in Pydantic V2.0 to be removed in V3.0. See Pydantic V2 Migration Guide at https://errors.pydantic.dev/2.7/migration/
    warnings.warn(DEPRECATION_MESSAGE, DeprecationWarning)

litellm/proxy/_types.py:225
  /Users/abramowi/Code/OpenSource/litellm/litellm/proxy/_types.py:225: PydanticDeprecatedSince20: Pydantic V1 style `@root_validator` validators are deprecated. You should migrate to Pydantic V2 style `@model_validator` validators, see the migration guide for more details. Deprecated in Pydantic V2.0 to be removed in V3.0. See Pydantic V2 Migration Guide at https://errors.pydantic.dev/2.7/migration/
    @root_validator(pre=True)

litellm/proxy/_types.py:312
  /Users/abramowi/Code/OpenSource/litellm/litellm/proxy/_types.py:312: PydanticDeprecatedSince20: `pydantic.config.Extra` is deprecated, use literal values instead (e.g. `extra='allow'`). Deprecated in Pydantic V2.0 to be removed in V3.0. See Pydantic V2 Migration Guide at https://errors.pydantic.dev/2.7/migration/
    extra = Extra.allow  # Allow extra fields

litellm/proxy/_types.py:315
  /Users/abramowi/Code/OpenSource/litellm/litellm/proxy/_types.py:315: PydanticDeprecatedSince20: Pydantic V1 style `@root_validator` validators are deprecated. You should migrate to Pydantic V2 style `@model_validator` validators, see the migration guide for more details. Deprecated in Pydantic V2.0 to be removed in V3.0. See Pydantic V2 Migration Guide at https://errors.pydantic.dev/2.7/migration/
    @root_validator(pre=True)

litellm/proxy/_types.py:344
  /Users/abramowi/Code/OpenSource/litellm/litellm/proxy/_types.py:344: PydanticDeprecatedSince20: Pydantic V1 style `@root_validator` validators are deprecated. You should migrate to Pydantic V2 style `@model_validator` validators, see the migration guide for more details. Deprecated in Pydantic V2.0 to be removed in V3.0. See Pydantic V2 Migration Guide at https://errors.pydantic.dev/2.7/migration/
    @root_validator(pre=True)

litellm/proxy/_types.py:391
  /Users/abramowi/Code/OpenSource/litellm/litellm/proxy/_types.py:391: PydanticDeprecatedSince20: Pydantic V1 style `@root_validator` validators are deprecated. You should migrate to Pydantic V2 style `@model_validator` validators, see the migration guide for more details. Deprecated in Pydantic V2.0 to be removed in V3.0. See Pydantic V2 Migration Guide at https://errors.pydantic.dev/2.7/migration/
    @root_validator(pre=True)

litellm/proxy/_types.py:460
  /Users/abramowi/Code/OpenSource/litellm/litellm/proxy/_types.py:460: PydanticDeprecatedSince20: Pydantic V1 style `@root_validator` validators are deprecated. You should migrate to Pydantic V2 style `@model_validator` validators, see the migration guide for more details. Deprecated in Pydantic V2.0 to be removed in V3.0. See Pydantic V2 Migration Guide at https://errors.pydantic.dev/2.7/migration/
    @root_validator(pre=True)

litellm/proxy/_types.py:480
  /Users/abramowi/Code/OpenSource/litellm/litellm/proxy/_types.py:480: PydanticDeprecatedSince20: Pydantic V1 style `@root_validator` validators are deprecated. You should migrate to Pydantic V2 style `@model_validator` validators, see the migration guide for more details. Deprecated in Pydantic V2.0 to be removed in V3.0. See Pydantic V2 Migration Guide at https://errors.pydantic.dev/2.7/migration/
    @root_validator(pre=True)

litellm/proxy/_types.py:493
  /Users/abramowi/Code/OpenSource/litellm/litellm/proxy/_types.py:493: PydanticDeprecatedSince20: Pydantic V1 style `@root_validator` validators are deprecated. You should migrate to Pydantic V2 style `@model_validator` validators, see the migration guide for more details. Deprecated in Pydantic V2.0 to be removed in V3.0. See Pydantic V2 Migration Guide at https://errors.pydantic.dev/2.7/migration/
    @root_validator(pre=True)

litellm/proxy/_types.py:538
  /Users/abramowi/Code/OpenSource/litellm/litellm/proxy/_types.py:538: PydanticDeprecatedSince20: Pydantic V1 style `@root_validator` validators are deprecated. You should migrate to Pydantic V2 style `@model_validator` validators, see the migration guide for more details. Deprecated in Pydantic V2.0 to be removed in V3.0. See Pydantic V2 Migration Guide at https://errors.pydantic.dev/2.7/migration/
    @root_validator(pre=True)

litellm/proxy/_types.py:575
  /Users/abramowi/Code/OpenSource/litellm/litellm/proxy/_types.py:575: PydanticDeprecatedSince20: Pydantic V1 style `@root_validator` validators are deprecated. You should migrate to Pydantic V2 style `@model_validator` validators, see the migration guide for more details. Deprecated in Pydantic V2.0 to be removed in V3.0. See Pydantic V2 Migration Guide at https://errors.pydantic.dev/2.7/migration/
    @root_validator(pre=True)

litellm/proxy/_types.py:870
  /Users/abramowi/Code/OpenSource/litellm/litellm/proxy/_types.py:870: PydanticDeprecatedSince20: Pydantic V1 style `@root_validator` validators are deprecated. You should migrate to Pydantic V2 style `@model_validator` validators, see the migration guide for more details. Deprecated in Pydantic V2.0 to be removed in V3.0. See Pydantic V2 Migration Guide at https://errors.pydantic.dev/2.7/migration/
    @root_validator(pre=True)

litellm/proxy/_types.py:897
  /Users/abramowi/Code/OpenSource/litellm/litellm/proxy/_types.py:897: PydanticDeprecatedSince20: Pydantic V1 style `@root_validator` validators are deprecated. You should migrate to Pydantic V2 style `@model_validator` validators, see the migration guide for more details. Deprecated in Pydantic V2.0 to be removed in V3.0. See Pydantic V2 Migration Guide at https://errors.pydantic.dev/2.7/migration/
    @root_validator(pre=True)

litellm/proxy/_types.py:918
  /Users/abramowi/Code/OpenSource/litellm/litellm/proxy/_types.py:918: PydanticDeprecatedSince20: Pydantic V1 style `@root_validator` validators are deprecated. You should migrate to Pydantic V2 style `@model_validator` validators, see the migration guide for more details. Deprecated in Pydantic V2.0 to be removed in V3.0. See Pydantic V2 Migration Guide at https://errors.pydantic.dev/2.7/migration/
    @root_validator(pre=True)

litellm/utils.py:43
  /Users/abramowi/Code/OpenSource/litellm/litellm/utils.py:43: DeprecationWarning: pkg_resources is deprecated as an API. See https://setuptools.pypa.io/en/latest/pkg_resources.html
    import pkg_resources  # type: ignore

litellm/tests/test_proxy_server.py::test_bedrock_embedding
litellm/tests/test_proxy_server.py::test_chat_completion
litellm/tests/test_proxy_server.py::test_chat_completion_azure
litellm/tests/test_proxy_server.py::test_chat_completion_optional_params
litellm/tests/test_proxy_server.py::test_embedding
litellm/tests/test_proxy_server.py::test_engines_model_chat_completions
litellm/tests/test_proxy_server.py::test_health
litellm/tests/test_proxy_server.py::test_img_gen
litellm/tests/test_proxy_server.py::test_openai_deployments_model_chat_completions_azure
  /Users/abramowi/Library/Caches/pypoetry/virtualenvs/litellm-Fe7WjZrx-py3.12/lib/python3.12/site-packages/httpx/_client.py:680: DeprecationWarning: The 'app' shortcut is now deprecated. Use the explicit style 'transport=WSGITransport(app=...)' instead.
    warnings.warn(message, DeprecationWarning)

-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
========================== 10 passed, 2 skipped, 48 warnings in 9.83s ===========================
```

After:

```shell
$ env -i PATH=$PATH poetry run pytest litellm/tests/test_proxy_server.py
====================================== test session starts ======================================
platform darwin -- Python 3.12.3, pytest-7.4.4, pluggy-1.5.0
rootdir: /Users/abramowi/Code/OpenSource/litellm
plugins: anyio-4.3.0, asyncio-0.23.6, mock-3.14.0
asyncio: mode=Mode.STRICT
collected 12 items

litellm/tests/test_proxy_server.py s..........s                                           [100%]

======================================= warnings summary ========================================
litellm/utils.py:43
  /Users/abramowi/Code/OpenSource/litellm/litellm/utils.py:43: DeprecationWarning: pkg_resources is deprecated as an API. See https://setuptools.pypa.io/en/latest/pkg_resources.html
    import pkg_resources  # type: ignore

litellm/tests/test_proxy_server.py::test_bedrock_embedding
litellm/tests/test_proxy_server.py::test_chat_completion
litellm/tests/test_proxy_server.py::test_chat_completion_azure
litellm/tests/test_proxy_server.py::test_chat_completion_optional_params
litellm/tests/test_proxy_server.py::test_embedding
litellm/tests/test_proxy_server.py::test_engines_model_chat_completions
litellm/tests/test_proxy_server.py::test_health
litellm/tests/test_proxy_server.py::test_img_gen
litellm/tests/test_proxy_server.py::test_openai_deployments_model_chat_completions_azure
  /Users/abramowi/Library/Caches/pypoetry/virtualenvs/litellm-Fe7WjZrx-py3.12/lib/python3.12/site-packages/httpx/_client.py:680: DeprecationWarning: The 'app' shortcut is now deprecated. Use the explicit style 'transport=WSGITransport(app=...)' instead.
    warnings.warn(message, DeprecationWarning)

-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
========================== 10 passed, 2 skipped, 10 warnings in 9.62s ===========================
```
2024-05-12 10:11:48 -07:00
Krrish Dholakia
61143c8b45 refactor(main.py): trigger new build 2024-05-11 22:53:09 -07:00
Krrish Dholakia
99e8f0715e test(test_end_users.py): fix end user region routing test 2024-05-11 22:42:43 -07:00
Krrish Dholakia
b4684d5132 fix(proxy_server.py): linting fix 2024-05-11 22:05:01 -07:00
Krrish Dholakia
094f20121a build(model_prices_and_context_window.json): add bedrock cohere command r pricing 2024-05-11 21:38:53 -07:00
Krrish Dholakia
15a6e59431 fix(proxy/_types.py): allow jwt admin to access spend routes 2024-05-11 21:31:34 -07:00
Krrish Dholakia
7276c6eb1e docs(token_auth.md): add end user cost tracking to jwt auth docs 2024-05-11 21:28:31 -07:00
Krish Dholakia
94c9df969e
Merge pull request #3586 from BerriAI/litellm_bedrock_command_r_support
feat(bedrock_httpx.py): Make Bedrock-Cohere calls Async
2024-05-11 21:24:51 -07:00
Krish Dholakia
1d651c6049
Merge branch 'main' into litellm_bedrock_command_r_support 2024-05-11 21:24:42 -07:00
Krrish Dholakia
e8437e52fa test(test_rules.py): fix test 2024-05-11 21:22:37 -07:00
Krish Dholakia
7566a2fc78
Merge pull request #3589 from msabramo/msabramo/make_test_load_router_config_pass
Make `test_load_router_config` pass
2024-05-11 21:15:07 -07:00
Krrish Dholakia
d142478b75 fix(langfuse.py): fix handling of dict object for langfuse prompt management 2024-05-11 20:42:55 -07:00
Ishaan Jaff
d3371fc81d fix langfuse logging metadata 2024-05-11 20:39:44 -07:00
Ishaan Jaff
2b3414c667 ci/cd run again 2024-05-11 20:34:55 -07:00
Krrish Dholakia
15ba244e46 fix(utils.py): correctly exception map 'request too large' as rate limit error 2024-05-11 20:20:34 -07:00
Krrish Dholakia
a456f6bf2b fix(anthropic.py): fix tool calling + streaming issue 2024-05-11 20:15:36 -07:00
Krrish Dholakia
83beb41096 fix(anthropic_text.py): fix linting error 2024-05-11 20:01:50 -07:00
Ishaan Jaff
beac60ed12 test - router retry policy 2024-05-11 19:58:17 -07:00
Krrish Dholakia
65d0be85fc fix(bedrock_httpx.py): compatibility fix 2024-05-11 19:55:38 -07:00
Krrish Dholakia
f6c84f1aa6 fix(anthropic.py): compatibility fix 2024-05-11 19:51:29 -07:00
Krrish Dholakia
6d67d6d5ad fix(types/bedrock.py): linting fix 2024-05-11 19:49:46 -07:00
Ishaan Jaff
61a3e5d5a9 fix get healthy deployments 2024-05-11 19:46:35 -07:00
Krrish Dholakia
ae0c061b46 fix(anthropic.py): fix version compatibility 2024-05-11 19:46:26 -07:00
Krrish Dholakia
b1448cd244 test(test_streaming.py): fix test 2024-05-11 19:44:47 -07:00
Krrish Dholakia
2f3fd3e2f0 fix(anthropic.py): fix linting error 2024-05-11 19:42:14 -07:00
Krrish Dholakia
64650c0279 feat(bedrock_httpx.py): working bedrock command-r sync+async streaming 2024-05-11 19:39:51 -07:00
Ishaan Jaff
04ac352407 test fix - test_async_fallbacks_embeddings 2024-05-11 19:20:24 -07:00
Ishaan Jaff
7930653872 fix - test router fallbacks 2024-05-11 19:13:22 -07:00
Ishaan Jaff
32e445c59d fix - unit tests for router retries 2024-05-11 19:10:33 -07:00
Ishaan Jaff
e0d1f96544 test router - fallbacks 2024-05-11 19:08:31 -07:00
Ishaan Jaff
4d648a6d89 fix - _time_to_sleep_before_retry 2024-05-11 19:08:10 -07:00
Ishaan Jaff
bfcb640d21
Merge pull request #3590 from BerriAI/litellm_router_retry_logic
[Feat] Proxy + Router - Retry on RateLimitErrors when fallbacks, other deployments exists
2024-05-11 18:21:12 -07:00
Ishaan Jaff
c56b44f779 fix failing azure content safety errors 2024-05-11 18:19:00 -07:00