Krish Dholakia
1e3370f3cb
LiteLLM Minor Fixes & Improvements (01/08/2025) - p2 ( #7643 )
...
* fix(streaming_chunk_builder_utils.py): add test for groq tool calling + streaming + combine chunks
Addresses https://github.com/BerriAI/litellm/issues/7621
* fix(streaming_utils.py): fix modelresponseiterator for openai like chunk parser
ensures chunk parser uses the correct tool call id when translating the chunk
Fixes https://github.com/BerriAI/litellm/issues/7621
* build(model_hub.tsx): display cost pricing on model hub
* build(model_hub.tsx): show cost per token pricing + complete model information
* fix(types/utils.py): fix usage object handling
2025-01-08 19:45:19 -08:00
Krrish Dholakia
39ee4c6bb4
docs(intro.md): add a section on 'why pass through endpoints'
...
helps proxy admin understand when these would be useful
2025-01-08 19:15:41 -08:00
Krrish Dholakia
3193cb3912
build(model_prices_and_context_window.json): omni-moderation-latest-intents
2025-01-08 19:06:04 -08:00
Ishaan Jaff
782b59788f
bump: version 1.57.3 → 1.57.4
2025-01-08 18:50:11 -08:00
Ishaan Jaff
7259452285
(helm) - bug fix - allow using migrationJob.enabled variable within job ( #7639 )
...
* wrap migration job in conditional
* use enabled flag in values.yaml
* run migration job
* argo cd fixes
* fix values in image repo
2025-01-08 18:47:33 -08:00
Ishaan Jaff
48d4f79206
fix is llm api route check ( #7631 )
2025-01-08 18:45:59 -08:00
Ishaan Jaff
9eb24df113
ci/cd run again
2025-01-08 18:36:39 -08:00
Krish Dholakia
4af23353d6
Allow assigning teams to org on UI + OpenAI omni-moderation cost model tracking ( #7566 )
...
* feat(cost_calculator.py): add cost tracking ($0) for openai moderations endpoint
removes sentry cost tracking errors caused by this
* build(teams.tsx): allow assigning teams to orgs
2025-01-08 16:58:21 -08:00
Krish Dholakia
0ffc5379ea
Litellm dev 01 07 2025 p2 ( #7622 )
...
* build(ui/): update ui
* fix: drop unsupported non-whitespace characters for real when calling… (#7484 )
* fix: drop unsupported non-whitespace characters for real when calling anthropic with stop sequences
* test: add parameterized test for _map_stop_sequences method in AnthropicConfig
---------
Co-authored-by: Wolfram Ravenwolf <52386626+WolframRavenwolf@users.noreply.github.com>
2025-01-08 16:56:39 -08:00
Ishaan Jaff
fd0a03f719
(feat) - allow building litellm proxy from pip package ( #7633 )
...
* fix working build from pip
* add tests for proxy_build_from_pip_tests
* doc clean up for deployment
* docs cleanup
* docs build from pip
* fix cd docker/build_from_pip
2025-01-08 16:36:57 -08:00
Ishaan Jaff
43566e9842
fix docs
2025-01-08 12:51:59 -08:00
Ishaan Jaff
e5717d2cb0
update load test docs
2025-01-08 12:48:21 -08:00
Ishaan Jaff
74b41d29d3
sort rn
2025-01-08 12:16:01 -08:00
Ishaan Jaff
f95439af26
docs v1.57.3
2025-01-08 12:08:19 -08:00
Krish Dholakia
a187cee538
Litellm dev 01 07 2025 p3 ( #7635 )
...
* fix(__init__.py): fix mistral large tool calling
map bedrock mistral large to converse endpoint
Fixes https://github.com/BerriAI/litellm/issues/7521
* braintrust logging: respect project_id, add more metrics + more (#7613 )
* braintrust logging: respect project_id, add more metrics
* braintrust logger: improve json formatting
* braintrust logger: add test for passing specific project_id
* rm unneeded import
* braintrust logging: rm unneeded var in tets
* add project_name
* update docs
---------
Co-authored-by: H <no@email.com>
---------
Co-authored-by: hi019 <65871571+hi019@users.noreply.github.com>
Co-authored-by: H <no@email.com>
2025-01-08 11:46:24 -08:00
Krish Dholakia
07c5f136f1
fix(utils.py): fix select tokenizer for custom tokenizer ( #7599 )
...
* fix(utils.py): fix select tokenizer for custom tokenizer
* fix(router.py): fix 'utils/token_counter' endpoint
2025-01-07 22:37:09 -08:00
Ishaan Jaff
04eb718f7a
update docs
2025-01-07 22:35:07 -08:00
Krrish Dholakia
7bf6769aa7
bump: version 1.57.2 → 1.57.3
2025-01-07 22:23:19 -08:00
Krish Dholakia
e8ed40a27b
Litellm dev 01 01 2025 p2 ( #7615 )
...
* fix(utils.py): prevent double logging when passing 'fallbacks=' to .completion()
Fixes https://github.com/BerriAI/litellm/issues/7477
* fix(utils.py): fix vertex anthropic check
* fix(utils.py): ensure supported params is always set
Fixes https://github.com/BerriAI/litellm/issues/7470
* test(test_optional_params.py): add unit testing to prevent mistranslation
Fixes https://github.com/BerriAI/litellm/issues/7470
* fix: fix linting error
* test: cleanup
2025-01-07 21:40:33 -08:00
Ishaan Jaff
081826a5d6
(Feat) soft budget alerts on keys ( #7623 )
...
* class WebhookEvent(CallInfo):
Add
* handle soft budget alerts
* handle soft budget
* fix budget alerts
* fix CallInfo
* fix _get_user_info_str
* test_soft_budget_alerts
* test_soft_budget_alert
2025-01-07 21:36:34 -08:00
Krish Dholakia
4e69711411
Litellm dev 01 07 2025 p1 ( #7618 )
...
* fix(main.py): pass custom llm provider on litellm logging provider update
* fix(cost_calculator.py): don't append provider name to return model if existing llm provider
Fixes https://github.com/BerriAI/litellm/issues/7607
* fix(prometheus_services.py): fix prometheus system health error logging
Fixes https://github.com/BerriAI/litellm/issues/7611
2025-01-07 21:22:31 -08:00
Ishaan Jaff
60c89a3e8a
(Fix) security of base image ( #7620 )
...
* fix security of base images
* fix dockerfile
2025-01-07 20:35:57 -08:00
Ishaan Jaff
59133e8fe6
ci/cd run again
2025-01-07 10:01:29 -08:00
Ishaan Jaff
55139b8fd6
update tests
2025-01-06 22:36:00 -08:00
Krrish Dholakia
d5a288e29e
docs: cleanup keys
2025-01-06 21:57:18 -08:00
Ishaan Jaff
ae3d8feb50
bump: version 1.57.1 → 1.57.2
2025-01-06 21:48:25 -08:00
Ishaan Jaff
2ca0977921
aiohttp_openai/ fixes - allow using aiohttp_openai/gpt-4o (#7598 )
...
* fixes for get_complete_url
* update aiohttp tests
* fix event loop for aiohtto
* ci/cd run again
* test_aiohttp_openai
2025-01-06 21:39:11 -08:00
Ishaan Jaff
744beac754
ci/cd run again
2025-01-06 21:35:34 -08:00
Krish Dholakia
fef7839e8a
Litellm dev 01 06 2025 p1 ( #7594 )
...
* fix(custom_logger.py): expose new 'async_get_chat_completion_prompt' event hook
* fix(custom_logger.py): langfuse_prompt_management.py
remove 'headers' from custom logger 'async_get_chat_completion_prompt' and 'get_chat_completion_prompt' event hooks
* feat(router.py): expose new function for prompt management based routing
* feat(router.py): partial working router prompt factory logic
allows load balanced model to be used for model name w/ langfuse prompt management call
* feat(router.py): fix prompt management with load balanced model group
* feat(langfuse_prompt_management.py): support reading in openai params from langfuse
enables user to define optional params on langfuse vs. client code
* test(test_Router.py): add unit test for router based langfuse prompt management
* fix: fix linting errors
2025-01-06 21:26:21 -08:00
Krrish Dholakia
7133cf5b74
bump: version 1.57.0 → 1.57.1
2025-01-06 21:13:56 -08:00
Krrish Dholakia
16f13dd55c
docs(prompt_management.md): update docs to show how to point to load balanced model name
2025-01-06 21:09:09 -08:00
Krish Dholakia
0c3fef24cd
Litellm dev 01 06 2025 p2 ( #7597 )
...
* test(test_amazing_vertex_completion.py): fix test
* test: initial working code gecko test
* fix(vertex_ai_non_gemini.py): support vertex ai code gecko fake streaming
Fixes https://github.com/BerriAI/litellm/issues/7360
* test(test_get_model_info.py): add test for getting custom provider model info
Covers https://github.com/BerriAI/litellm/issues/7575
* fix(utils.py): fix get_provider_model_info check
Handle custom llm provider scenario
Fixes https://github.com/
BerriAI/litellm/issues/7575
2025-01-06 21:04:49 -08:00
Krish Dholakia
b397dc1497
Litellm dev 01 06 2025 p3 ( #7596 )
...
* build(model_prices_and_context_window.json): add gemini-1.5-pro 'supports_vision' = true
Fixes https://github.com/BerriAI/litellm/issues/7592
* build(model_prices_and_context_window.json): add new mistral models pricing + model info
2025-01-06 20:44:04 -08:00
fzowl
c22d9208da
Refresh VoyageAI models, prices and context ( #7472 )
...
* Refresh VoyageAI models and prices and context
* Refresh VoyageAI models and prices and context
* Refresh VoyageAI models and prices and context
2025-01-06 20:38:01 -08:00
Ishaan Jaff
819079f23b
(proxy perf improvement) - remove redundant .copy() operation ( #7564 )
...
* latency fix proxy
* remove useless copy in add_key_level_controls
2025-01-06 20:36:47 -08:00
Ishaan Jaff
6125ba1e2b
(Feat) - allow including dd-trace in litellm base image ( #7587 )
...
* introduce USE_DDTRACE=true
* update dd tracer
* update
* bump dd trace
* use og slim image
* DD tracing
* fix _init_dd_tracer
2025-01-06 17:27:09 -08:00
Ishaan Jaff
0b5c1392f7
fix _return_user_api_key_auth_obj ( #7591 )
2025-01-06 16:43:14 -08:00
Ishaan Jaff
61d67cfa43
(perf) - fixes for aiohttp handler to hit 1K RPS ( #7590 )
...
* fix getting aiohttp sesson
* fix _get_async_client_session
2025-01-06 15:41:39 -08:00
Krrish Dholakia
23685e93f3
test: skip tests pending vertex credentials
2025-01-05 15:29:51 -08:00
Ishaan Jaff
a40baec5ed
use latest bucket for testing
2025-01-05 14:55:48 -08:00
Krrish Dholakia
8ae2ca4ed9
test: fix test
2025-01-05 14:52:38 -08:00
Krrish Dholakia
8bda3006fa
fix: test
2025-01-05 14:37:17 -08:00
Krrish Dholakia
32538f09fc
test: cleanup test
2025-01-05 14:18:29 -08:00
Ishaan Jaff
3110bb0723
use pathrise-convert-1606954137718
2025-01-05 14:14:43 -08:00
Ishaan Jaff
616211daee
ci/cd run again
2025-01-05 14:11:27 -08:00
Ishaan Jaff
137879ffea
vertex testing use pathrise-convert-1606954137718
2025-01-05 14:00:17 -08:00
Krrish Dholakia
c0e4485fe0
test: update test amazing vertex
2025-01-05 13:56:31 -08:00
Ishaan Jaff
ef8812d150
ci/cd update vertex acct
2025-01-05 13:43:32 -08:00
Low Jian Sheng
9cfd7c790a
Prevent istio injection for db migrations cron job ( #7513 )
...
* allow option to prevent istio injection
* make annotations more generic
* make annotations empty
* revert comment deletions
2025-01-04 22:45:19 -08:00
minpeter
f7931b659b
FriendliAI: Documentation Updates ( #7517 )
...
* docs(friendliai.md): update FriendliAI documentation and model details
* docs(friendliai.md): remove unused imports for cleaner documentation
* feat: add support for parallel function calling, system messages, and response schema in model configuration
2025-01-04 22:44:24 -08:00
