scriptoriumadmin/litellm
1
0
Fork 0
mirror of https://github.com/BerriAI/litellm.git synced 2026-10-08 03:08:45 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 114
18628 commits 3285 branches 1443 tags 4.8 GiB
Commit graph

18628 commits

This branch
This branch
All branches
Author SHA1 Message Date
Krrish Dholakia
30e147a315 bump: version 1.55.1 → 1.55.2 2024-12-13 12:55:56 -08:00
Krish Dholakia
7643b3087c
Litellm dev 12 11 2024 v2 (#7215) 
* feat(bedrock/): add bedrock converse top k param

Closes https://github.com/BerriAI/litellm/issues/7087

* Fix bedrock empty content error (#7177)

* add resolver

* handle empty content on bedrock with default content

* use existing default message, tests

* Update tests/llm_translation/test_bedrock_completion.py

* fix tests

* Revert "add resolver"

This reverts commit c717e376ee.

* fallback to empty

---------

Co-authored-by: Krish Dholakia <krrishdholakia@gmail.com>

* fix(factory.py): handle empty content blocks in messages

Fixes https://github.com/BerriAI/litellm/issues/7169

* feat(router.py): add stripped model check to model fallback search

if model_name="openai/gpt-3.5-turbo" and fallback=[{"gpt-3.5-turbo"..}] the fallback should just work as expected

* fix: fix linting error

* fix(factory.py): fix linting error

* fix(factory.py): in base case still support skip empty text blocks

---------

Co-authored-by: Engel Nyst <enyst@users.noreply.github.com>
 2024-12-13 12:49:57 -08:00
Krish Dholakia
e68bb4e051
Litellm dev 12 12 2024 (#7203) 
* fix(azure/): support passing headers to azure openai endpoints

Fixes https://github.com/BerriAI/litellm/issues/6217

* fix(utils.py): move default tokenizer to just openai

hf tokenizer makes network calls when trying to get the tokenizer - this slows down execution time calls

* fix(router.py): fix pattern matching router - add generic "*" to it as well

Fixes issue where generic "*" model access group wouldn't show up

* fix(pattern_match_deployments.py): match to more specific pattern

match to more specific pattern

allows setting generic wildcard model access group and excluding specific models more easily

* fix(proxy_server.py): fix _delete_deployment to handle base case where db_model list is empty

don't delete all router models  b/c of empty list

Fixes https://github.com/BerriAI/litellm/issues/7196

* fix(anthropic/): fix handling response_format for anthropic messages with anthropic api

* fix(fireworks_ai/): support passing response_format + tool call in same message

Addresses https://github.com/BerriAI/litellm/issues/7135

* Revert "fix(fireworks_ai/): support passing response_format + tool call in same message"

This reverts commit 6a30dc6929.

* test: fix test

* fix(replicate/): fix replicate default retry/polling logic

* test: add unit testing for router pattern matching

* test: update test to use default oai tokenizer

* test: mark flaky test

* test: skip flaky test
 2024-12-13 08:54:03 -08:00
Ishaan Jaff
15a0572a06 bump: version 1.55.0 → 1.55.1 2024-12-12 20:50:45 -08:00
Ishaan Jaff
7ff9a905d2
(fix) latency fix - revert prompt caching check on litellm router (#7211) 
* attempt to fix latency issue

* fix latency issues for router prompt caching
 2024-12-12 20:50:16 -08:00
Ishaan Jaff
3de32f4106
(minor fix proxy) Clarify Proxy Rate limit errors are showing hash of litellm virtual key (#7210) 
* fix clarify rate limit errors are showing litellm virtual key

* fix constants.py

* update test

* fix test parallel limiter
 2024-12-12 20:13:14 -08:00
Ishaan Jaff
36862d0a98 fix testing retry audio test 3 times 2024-12-12 20:09:14 -08:00
Ishaan Jaff
b889d7c72f
(feat) UI - Disable Usage Tab once SpendLogs is 1M+ Rows (#7208) 
* use utils to set proxy spend logs row count

* store proxy state variables

* fix check for _has_user_setup_sso

* fix proxyStateVariables

* fix dup code

* rename getProxyUISettings

* add fixes

* ui emit num spend logs rows

* test_proxy_server_prisma_setup

* use MAX_SPENDLOG_ROWS_TO_QUERY to constants

* test_get_ui_settings_spend_logs_threshold
 2024-12-12 18:43:17 -08:00
Ishaan Jaff
ce69357e9d
fix: Support WebP image format and avoid token calculation error (#7182) 
* fix get_image_dimensions

* attempt without pillow

* add clear type hints

* fix run_async_function_within_sync_function

* fix calculage_img_tokens

* fix is_prompt_caching_valid_prompt

* fix naming

* fix calculate_img_tokens

* fix unused imports

* fix calculate_img_tokens

* test test_is_prompt_caching_enabled_error_handling

* test_is_prompt_caching_enabled_return_default_image_dimensions

* fix openai_token_counter

* fix get_image_dimensions

* test_token_counter_with_image_url_with_detail_high

* test_img_url_token_counter

* fix test utils

* fix testing

* test_is_prompt_caching_enabled
 2024-12-12 14:32:39 -08:00
Ishaan Jaff
621c713400
(docs) Document StandardLoggingPayload Spec (#7201) 
* add slp spec to docs

* docs slp

* test slp enforcement
 2024-12-12 14:00:42 -08:00
Ishaan Jaff
431c86cbf5
(feat) add error_code, error_class, llm_provider to StandardLoggingPayload (#7200) 
* add StandardLoggingPayloadErrorInformation to error

* test_get_error_information
 2024-12-12 12:18:10 -08:00
Ishaan Jaff
b45777c268
(Feat) DataDog Logger - Add HOSTNAME and POD_NAME to DataDog logs (#7189) 
* add unit test for test_datadog_static_methods

* docs dd vars

* test_datadog_payload_environment_variables

* test_datadog_static_methods

* docs env vars

* fix table
 2024-12-12 12:06:26 -08:00
dependabot[bot]
b237854d54
build(deps): bump nanoid from 3.3.7 to 3.3.8 in /ui (#7198) 
Bumps [nanoid](https://github.com/ai/nanoid) from 3.3.7 to 3.3.8.
- [Release notes](https://github.com/ai/nanoid/releases)
- [Changelog](https://github.com/ai/nanoid/blob/main/CHANGELOG.md)
- [Commits](https://github.com/ai/nanoid/compare/3.3.7...3.3.8)

---
updated-dependencies:
- dependency-name: nanoid
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2024-12-12 12:04:54 -08:00
Ishaan Jaff
153ab055d6
(feat) add response_time to StandardLoggingPayload - logged on datadog, gcs_bucket, s3_bucket etc (#7199) 
* feat - add response_time to slp

* test_get_response_time

* docs slp

* fix test_datadog_logging_http_request
 2024-12-12 12:04:43 -08:00
Krrish Dholakia
aa7f416b7f test: update hf test to check if client closed 2024-12-12 11:34:50 -08:00
Ishaan Jaff
90f9aded9f ci/cd run release pipeline 2024-12-12 10:48:47 -08:00
Ishaan Jaff
ecd219fa4f fix hf failing streaming test 2024-12-12 10:48:00 -08:00
Ishaan Jaff
893f6aa971 bump: version 1.54.1 → 1.55.0 2024-12-12 10:39:04 -08:00
Krish Dholakia
481645e49c
fix(acompletion): support fallbacks on acompletion (#7184) 
* fix(acompletion): support fallbacks on acompletion

allows health checks for wildcard routes to use fallback models

* test: update cohere generate api testing

* add max tokens to health check (#7000)

* fix: fix health check test

* test: update testing

---------

Co-authored-by: Cameron <561860+wallies@users.noreply.github.com>
 2024-12-11 19:20:54 -08:00
Krrish Dholakia
5ec649b512 build(model_prices_and_context_window.json): add new dbrx llama 3.3 model 
fixes llama cost calc on databricks
 2024-12-11 13:01:22 -08:00
Ishaan Jaff
5885ee5e14 fix test_vertexai_model_garden_model_completion 2024-12-11 12:07:32 -08:00
Krish Dholakia
9f32631592
fix(get_supported_openai_params.py): cleanup (#7176) 2024-12-11 01:15:53 -08:00
Ishaan Jaff
59daac5a3f fix merge conflicts 2024-12-11 01:11:53 -08:00
Krrish Dholakia
02dd0c6e7e build: Squashed commit of https://github.com/BerriAI/litellm/pull/7171 
Closes https://github.com/BerriAI/litellm/pull/7171
 2024-12-11 01:10:12 -08:00
Ishaan Jaff
efbec4230b fix merge conflicts 2024-12-11 01:08:43 -08:00
Ishaan Jaff
fe768a9ab7 fix - handle merge conflicts 2024-12-11 01:06:40 -08:00
Krrish Dholakia
06074bb13b build: Squashed commit of https://github.com/BerriAI/litellm/pull/7170 
Closes https://github.com/BerriAI/litellm/pull/7170
 2024-12-11 01:03:57 -08:00
Ishaan Jaff
5d1274cb6e
add enforce_llms_folder_style (#7175) 2024-12-11 01:01:49 -08:00
Krrish Dholakia
b9b34a7b99 build: Squashed commit of https://github.com/BerriAI/litellm/pull/7165 
Closes https://github.com/BerriAI/litellm/pull/7165
 2024-12-11 01:00:33 -08:00
Ishaan Jaff
78d132c1fb
(Refactor) Code Quality improvement - rename text_completion_codestral.py -> codestral/completion/ (#7172) 
* rename files

* fix codestral fim organization

* fix CodestralTextCompletionConfig

* fix import CodestralTextCompletion

* fix BaseLLM

* fix imports

* fix CodestralTextCompletionConfig

* fix imports CodestralTextCompletion
 2024-12-11 00:55:47 -08:00
Ishaan Jaff
400eb28a91
Code Quality Improvement - move aleph_alpha to deprecated_providers (#7168) 
* move aleph alpha to deprecated providers

* fix import location

* fix aleph_alpha

* pytest skip

* undo change to test file
 2024-12-11 00:50:40 -08:00
Ishaan Jaff
21003c4337
Code Quality Improvement - use vertex_ai/ as folder name for vertexAI (#7166) 
* fix rename vertex ai

* run ci/cd again
 2024-12-11 00:32:41 -08:00
Ishaan Jaff
b5d55688e5
(Refactor) Code Quality improvement - remove /prompt_templates/ , base_aws_llm.py from /llms folder (#7164) 
* fix move base_aws_llm

* fix import

* update enforce llms folder style

* move prompt_templates

* update prompt_templates location

* fix imports

* fix imports

* fix imports

* fix imports

* fix checks
 2024-12-11 00:02:46 -08:00
dependabot[bot]
b328d42ebc
build(deps): bump nanoid from 3.3.7 to 3.3.8 in /docs/my-website (#7159) 
Bumps [nanoid](https://github.com/ai/nanoid) from 3.3.7 to 3.3.8.
- [Release notes](https://github.com/ai/nanoid/releases)
- [Changelog](https://github.com/ai/nanoid/blob/main/CHANGELOG.md)
- [Commits](https://github.com/ai/nanoid/compare/3.3.7...3.3.8)

---
updated-dependencies:
- dependency-name: nanoid
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2024-12-10 23:51:05 -08:00
Ishaan Jaff
3055d9b81c
Code Quality Improvement - remove tokenizers/ from /llms (#7163) 
* move tokenizers out of /llms

* use updated tokenizers location

* fix test_google_secret_manager_read_in_memory
 2024-12-10 23:50:15 -08:00
Krish Dholakia
350cfc36f7
Litellm merge pr (#7161) 
* build: merge branch

* test: fix openai naming

* fix(main.py): fix openai renaming

* style: ignore function length for config factory

* fix(sagemaker/): fix routing logic

* fix: fix imports

* fix: fix override
 2024-12-10 22:49:26 -08:00
Krish Dholakia
d5aae81c6d
Litellm vllm refactor (#7158) 
* refactor(vllm/): move vllm to use base llm config

* test: mark flaky test
 2024-12-10 21:48:35 -08:00
Krish Dholakia
405080396d
Litellm ollama refactor (#7162) 
* refactor(ollama/): refactor ollama `/api/generate` to use base llm config

Addresses https://github.com/andrewyng/aisuite/issues/113#issuecomment-2512369132

* test: skip unresponsive test

* test(test_secret_manager.py): mark flaky test

* test: fix google sm test

* fix: fix init.py
 2024-12-10 21:45:35 -08:00
Krish Dholakia
488913c69f
Revert "LiteLLM Common Base LLM Config (pt.4): Move Ollama to Base LLM Config…" (#7160) 
This reverts commit 40a22eb4c6.
 2024-12-10 21:44:54 -08:00
Ishaan Jaff
1d8956a3d4
Code Quality Improvement - remove file_apis, fine_tuning_apis from /llms (#7156) 
* remove files_apis from /llms

* fix imports

* move fine tuning api from /llms

* fix importing fine tuning handlers

* fix imports
 2024-12-10 21:44:25 -08:00
Krish Dholakia
40a22eb4c6
LiteLLM Common Base LLM Config (pt.4): Move Ollama to Base LLM Config (#7157) 
* refactor(ollama/): refactor ollama `/api/generate` to use base llm config

Addresses https://github.com/andrewyng/aisuite/issues/113#issuecomment-2512369132

* test: skip unresponsive test

* test(test_secret_manager.py): mark flaky test

* test: fix google sm test
 2024-12-10 21:39:28 -08:00
Ishaan Jaff
3b5485a14e
remove symlink (#7155) 2024-12-10 21:04:21 -08:00
Ishaan Jaff
1df5d73e3c fix import 2024-12-10 20:26:16 -08:00
Ishaan Jaff
bfb6891eb7
rename llms/OpenAI/ -> llms/openai/ (#7154) 
* rename OpenAI -> openai

* fix file rename

* fix rename changes

* fix organization of openai/transcription

* fix import OA fine tuning API

* fix openai ft handler

* fix handler import
 2024-12-10 20:14:07 -08:00
Krish Dholakia
e903fe6038
refactor(sagemaker/): separate chat + completion routes + make them b… (#7151) 
* refactor(sagemaker/): separate chat + completion routes + make them both use base llm config

Addresses https://github.com/andrewyng/aisuite/issues/113#issuecomment-2512369132

* fix(main.py): pass hf model name + custom prompt dict to litellm params
 2024-12-10 19:40:05 -08:00
Krish Dholakia
1e87782215
LiteLLM Common Base LLM Config (pt.3): Move all OAI compatible providers to base llm config (#7148) 
* refactor(fireworks_ai/): inherit from openai like base config

refactors fireworks ai to use a common config

* test: fix import in test

* refactor(watsonx/): refactor watsonx to use llm base config

refactors chat + completion routes to base config path

* fix: fix linting error

* refactor: inherit base llm config for oai compatible routes

* test: fix test

* test: fix test
 2024-12-10 17:12:42 -08:00
Krish Dholakia
311432ca17
refactor(fireworks_ai/): inherit from openai like base config (#7146) 
* refactor(fireworks_ai/): inherit from openai like base config

refactors fireworks ai to use a common config

* test: fix import in test

* refactor(watsonx/): refactor watsonx to use llm base config

refactors chat + completion routes to base config path

* fix: fix linting error

* test: fix test

* fix: fix test
 2024-12-10 16:15:19 -08:00
Ishaan Jaff
2fb2801eb4
(Refactor) Code Quality improvement - stop redefining LiteLLMBase (#7147) 
* fix stop redefining  LiteLLMBase

* use better name for base pydantic obj
 2024-12-10 15:49:01 -08:00
Krish Dholakia
f4b5a491b6
docs: document code quality (#7149) 
* docs: document code quality

* build(readme.md): cleanup
 2024-12-10 15:44:59 -08:00
Ishaan Jaff
bdb20821ea
(Refactor) Code Quality improvement - Use Common base handler for anthropic_text/ (#7143) 
* add anthropic text provider

* add ANTHROPIC_TEXT to LlmProviders

* fix anthropic text implementation

* working anthropic text claude-2

* test_acompletion_claude2_stream

* add param mapping for anthropic text

* fix unused imports

* fix anthropic completion handler.py
 2024-12-10 12:23:58 -08:00