Commit graph

18903 commits

Author SHA1 Message Date
Krish Dholakia
34a9833b85
Support deleting keys by key_alias (#7552)
* feat(key_management_endpoints.py): allow deleting keys based on key alias

easier for proxy admin to delete known bad key

* fix(key_management_event_hooks.py): fix linting error

* docs(key_management_endpoints.py): document new key_aliases param

* fix(key_management_endpoints.py): return deleted keys to user

fixes return when passing key aliases
2025-01-04 19:41:48 -08:00
Krish Dholakia
ce97e7e054
fix(groq/chat/transformation.py): fix groq response_format transformation (#7565)
Fixes https://github.com/BerriAI/litellm/issues/4804
2025-01-04 19:39:04 -08:00
Ishaan Jaff
d74fa39454
fix [PROXY] returned data from litellm_pre_call_util (#7558) 2025-01-04 18:47:36 -08:00
Krish Dholakia
f1540ceeab
Create and view organizations + assign org admins on the Proxy UI (#7557)
* feat: initial commit for new 'organizations' tab on ui

* build(ui/): create generic card for rendering complete org data table

can be reused in teams as well

simplifies things

* build(ui/): display created orgs on ui

* build(ui/): support adding orgs via UI

* build(ui/): add org in selection dropdown

* build(organizations.tsx): allow assigning org admins

* build(ui/): show org members on ui

* build(ui/): cleanup + show actual models on org dropdown

* build(ui/): explain user roles within organization
2025-01-04 17:31:24 -08:00
Ishaan Jaff
46d9d29bff
(Feat) Hashicorp Secret Manager - Allow storing virtual keys in secret manager (#7549)
* use a base abstract class

* async_write_secret for hcorp

* fix hcorp

* async_write_secret for hashicopr secret manager

* store virtual keys in hcorp

* add delete secret

* test_hashicorp_secret_manager_write_secret

* test_hashicorp_secret_manager_delete_secret

* docs Supported Secret Managers

* docs storing keys in hcorp

* docs hcorp

* docs secret managers

* test_key_generate_with_secret_manager_call

* fix unused imports
2025-01-04 11:35:59 -08:00
Ishaan Jaff
7f7222ce30
fix get_custom_logger_compatible_class (#7554) 2025-01-04 11:22:09 -08:00
Ishaan Jaff
e804a1850e
add cohere/command-r7b-12-2024 (#7553) 2025-01-04 11:21:04 -08:00
Ishaan Jaff
d1b101b9d7
(Fix) - Slack Alerting , don't send duplicate spend report when used on multi instance settings (#7546)
* fix send_weekly_spend_report

* test_spend_report_cache
2025-01-04 10:54:35 -08:00
Ishaan Jaff
06eea38c69
(Fix) - Docker build error with pyproject.toml (#7550)
* fix pyproject

* fix homepage
2025-01-04 10:50:24 -08:00
Krrish Dholakia
3230c902c9 bump: version 1.56.9 → 1.56.10 2025-01-03 19:36:14 -08:00
Krish Dholakia
d43d83f9ef
feat(router.py): support request prioritization for text completion c… (#7540)
* feat(router.py): support request prioritization for text completion calls

* fix(internal_user_endpoints.py): fix sql query to return all keys, including null team id keys on `/user/info`

Fixes https://github.com/BerriAI/litellm/issues/7485

* fix: fix linting errors

* fix: fix linting error

* test(test_router_helper_utils.py): add direct test for '_schedule_factory'

Fixes code qa test
2025-01-03 19:35:44 -08:00
Krish Dholakia
f770dd0c95
Support checking provider-specific /models endpoints for available models based on key (#7538)
* test(test_utils.py): initial test for valid models

Addresses https://github.com/BerriAI/litellm/issues/7525

* fix: test

* feat(fireworks_ai/transformation.py): support retrieving valid models from fireworks ai endpoint

* refactor(fireworks_ai/): support checking model info on `/v1/models` route

* docs(set_keys.md): update docs to clarify check llm provider api usage

* fix(watsonx/common_utils.py): support 'WATSONX_ZENAPIKEY' for iam auth

* fix(watsonx): read in watsonx token from env var

* fix: fix linting errors

* fix(utils.py): fix provider config check

* style: cleanup unused imports
2025-01-03 19:29:59 -08:00
Krish Dholakia
cac06a32b8
fix(aws_secret_manager_V2.py): Error reading secret from AWS Secrets Manager: (#7541) 2025-01-03 18:22:12 -08:00
Ishaan Jaff
8ad9526625 bump: version 1.56.8 → 1.56.9 2025-01-03 17:49:23 -08:00
Ishaan Jaff
716efd5fad
(fix proxy perf) use _read_request_body instead of ast.literal_eval to get better performance (#7545)
* fix ast literal eval

* run ci/cd again
2025-01-03 17:48:32 -08:00
Ishaan Jaff
1bb4941036
[Feature]: - allow print alert log to console (#7534)
* update send_to_webhook

* test_print_alerting_payload_warning

* add alerting_args spec

* test_alerting.py
2025-01-03 17:48:13 -08:00
Krish Dholakia
6843f3a2bb
Revert "fix: add missing parameters order, limit, before, and after in get_as…" (#7542)
This reverts commit 4b0505dffd.
2025-01-03 16:32:12 -08:00
Ishaan Jaff
c8f9e2afab fix _make_common_async_call 2025-01-03 15:15:38 -08:00
Ishaan Jaff
23104d9a14 test_aiohttp_openai 2025-01-03 15:12:56 -08:00
Ishaan Jaff
02875d4ae8
(fix) aiohttp_openai/ route - get to 1K RPS on single instance (#7539)
* ClientSession

* re use client_session

* _init_client_session

* fix aiohttp
2025-01-03 15:12:17 -08:00
மனோஜ்குமார் பழனிச்சாமி
4c6a1e6c9c
Add missing prefix for deepseek (#7508)
Solves https://github.com/All-Hands-AI/OpenHands/issues/5978
2025-01-03 14:42:18 -08:00
Jean Carlo de Souza
4b0505dffd
fix: add missing parameters order, limit, before, and after in get_assistants method for openai (#7537)
- Ensured that `before` and `after` parameters are only passed when provided to avoid AttributeError.
- Implemented safe access using default values for `before` and `after` to prevent missing attribute issues.
- Added consistent handling of `order` and `limit` to improve flexibility and robustness in API calls.
2025-01-03 14:41:54 -08:00
Krish Dholakia
33f301ec86
Litellm dev 01 02 2025 p1 (#7516)
* fix(redact_messages.py): fix redact messages for non-model response input to be dictionary

fixes issue with otel logging when message redaction is enabled

* fix(proxy_server.py): fix langfuse key leak in exception string

* test: fix test

* test: fix test

* test: fix tests
2025-01-03 14:40:57 -08:00
Ishaan Jaff
fb59f20979
(Feat) - Hashicorp secret manager, use TLS cert authentication (#7532)
* fix - don't print hcorp secrets in debug logs

* hcorp - tls auth fixes

* fix tls_ca_cert_path

* test_hashicorp_secret_manager_tls_cert_auth

* hcp secret docs
2025-01-03 14:23:53 -08:00
Ishaan Jaff
d3a3e45e5b docs pass through routes 2025-01-03 12:55:23 -08:00
Krish Dholakia
f6698e871f
Fix langfuse prompt management on proxy (#7535)
* fix(types/utils.py): support langfuse + humanloop routes on llm router

* fix(main.py): remove acompletion elif block

just await if coroutine returned
2025-01-03 12:42:37 -08:00
David Manouchehri
24589c49af
Bump anthropic.claude-3-5-haiku-20241022-v1:0 to new limits. (#7118) 2025-01-03 11:37:38 -08:00
Ishaan Jaff
6e31bcf5a7
fix - access metadata (#7523) 2025-01-03 10:02:10 -08:00
Ishaan Jaff
4899ed1b42
fix - don't print hcorp secrets in debug logs (#7529) 2025-01-03 10:01:54 -08:00
Ishaan Jaff
9fef0a6d16
(fix) GCS bucket logger - apply truncate_standard_logging_payload_content to standard_logging_payload and ensure GCS flushes queue on fails (#7519)
* fix async_send_batch for gcs

* fix truncate GCS logger

* test_truncate_standard_logging_payload
2025-01-03 08:09:03 -08:00
Ishaan Jaff
b8173c2e9f fix unused imports 2025-01-02 22:28:22 -08:00
Ishaan Jaff
a9a2c7afd2 bump: version 1.56.7 → 1.56.8 2025-01-02 22:27:06 -08:00
Ishaan Jaff
087e8c0969 bump: version 1.56.6 → 1.56.7 2025-01-02 22:27:01 -08:00
Ishaan Jaff
8bd6c977e2
fix - use tp executor (#7509) 2025-01-02 22:25:15 -08:00
Ishaan Jaff
d861aa8ff3
(perf) use aiohttp for custom_openai (#7514)
* use aiohttp handler

* BaseLLMAIOHTTPHandler

* use CustomOpenAIChatConfig

* CustomOpenAIChatConfig

* CustomOpenAIChatConfig

* fix linting

* AiohttpOpenAIChatConfig

* fix order

* aiohttp_openai
2025-01-02 22:15:17 -08:00
Ishaan Jaff
4d93fe787b
Revert "(fix) GCS bucket logger - apply `truncate_standard_logging_payload_co…" (#7515)
This reverts commit 26a37c50c9.
2025-01-02 22:01:02 -08:00
Krish Dholakia
25e6f46910
Litellm dev 01 02 2025 p2 (#7512)
* feat(deepgram/transformation.py): support reading in deepgram api base from env var

* fix(litellm_logging.py): make skipping log message a .info

easier to see

* docs(logging.md): add doc on turn off all tracking/logging for a request
2025-01-02 21:57:51 -08:00
Ishaan Jaff
b9280528d3 docs enable_pre_call_checks 2025-01-02 08:27:03 -08:00
Krrish Dholakia
c292f5805a docs(humanloop.md): add humanloop docs 2025-01-01 22:18:01 -08:00
Krish Dholakia
45b93f2721
Litellm dev 01 01 2025 p3 (#7503)
* fix(utils.py): add new validate tool choice helper function

Prevents https://github.com/BerriAI/litellm/issues/7483

* fix(main.py): add tool choice validation on .completion()

prevents user error like - https://github.com/BerriAI/litellm/issues/7483

* fix(utils.py): fix return val of tool choice validation logic
2025-01-01 22:12:15 -08:00
Ishaan Jaff
f96f54a0f5 test_aview_spend_per_user 2025-01-01 21:48:35 -08:00
Ishaan Jaff
8c29489c40 test_aadmin_only_routes 2025-01-01 21:48:20 -08:00
Ishaan Jaff
953f8c1ca8 ci/cd run again 2025-01-01 21:45:48 -08:00
Ishaan Jaff
0f1f39cc84 ci/cd run again 2025-01-01 20:49:53 -08:00
Ishaan Jaff
26a37c50c9
(fix) GCS bucket logger - apply truncate_standard_logging_payload_content to standard_logging_payload and ensure GCS flushes queue on fails (#7500)
* use truncate_standard_logging_payload_content

* update truncate_standard_logging_payload_content

* update dd logger

* update gcs async_send_batch

* fix code check

* test_datadog_payload_content_truncation

* fix code quality
2025-01-01 20:21:01 -08:00
Ishaan Jaff
ec33fab922 update litellm_proxy_unit_testing 2025-01-01 20:19:27 -08:00
Krish Dholakia
07fc394072
Litellm dev 01 01 2025 p1 (#7498)
* refactor(prometheus.py): refactor to remove `_tag` metrics and incorporate in regular metrics

* fix(prometheus.py): handle label values not set in enum values

* feat(prometheus.py): working e2e custom metadata labels

* docs(prometheus.md): update docs to clarify how custom metrics would work

* test(test_prometheus_unit_tests.py): fix test

* test: add unit testing
2025-01-01 18:59:28 -08:00
Krish Dholakia
0120176541
Litellm dev 12 30 2024 p2 (#7495)
* test(azure_openai_o1.py): initial commit with testing for azure openai o1 preview model

* fix(base_llm_unit_tests.py): handle azure o1 preview response format tests

skip as o1 on azure doesn't support tool calling yet

* fix: initial commit of azure o1 handler using openai caller

simplifies calling + allows fake streaming logic alr. implemented for openai to just work

* feat(azure/o1_handler.py): fake o1 streaming for azure o1 models

azure does not currently support streaming for o1

* feat(o1_transformation.py): support overriding 'should_fake_stream' on azure/o1 via 'supports_native_streaming' param on model info

enables user to toggle on when azure allows o1 streaming without needing to bump versions

* style(router.py): remove 'give feedback/get help' messaging when router is used

Prevents noisy messaging

Closes https://github.com/BerriAI/litellm/issues/5942

* fix(types/utils.py): handle none logprobs

Fixes https://github.com/BerriAI/litellm/issues/328

* fix(exception_mapping_utils.py): fix error str unbound error

* refactor(azure_ai/): move to openai_like chat completion handler

allows for easy swapping of api base url's (e.g. ai.services.com)

Fixes https://github.com/BerriAI/litellm/issues/7275

* refactor(azure_ai/): move to base llm http handler

* fix(azure_ai/): handle differing api endpoints

* fix(azure_ai/): make sure all unit tests are passing

* fix: fix linting errors

* fix: fix linting errors

* fix: fix linting error

* fix: fix linting errors

* fix(azure_ai/transformation.py): handle extra body param

* fix(azure_ai/transformation.py): fix max retries param handling

* fix: fix test

* test(test_azure_o1.py): fix test

* fix(llm_http_handler.py): support handling azure ai unprocessable entity error

* fix(llm_http_handler.py): handle sync invalid param error for azure ai

* fix(azure_ai/): streaming support with base_llm_http_handler

* fix(llm_http_handler.py): working sync stream calls with unprocessable entity handling for azure ai

* fix: fix linting errors

* fix(llm_http_handler.py): fix linting error

* fix(azure_ai/): handle cohere tool call invalid index param error
2025-01-01 18:57:29 -08:00
Ishaan Jaff
0f1b298fe0 ci/cd run again 2025-01-01 18:50:22 -08:00
Ishaan Jaff
665fb59f48 doc update 2025-01-01 18:40:59 -08:00