Commit graph

855 commits

Author SHA1 Message Date
Ishaan Jaff
f1a8b1984a fix mcp test deps 2025-03-24 21:34:18 -07:00
Ishaan Jaff
f2b9a7f2ea pip install "langchain_mcp_adapters==0.0.5" 2025-03-24 21:20:11 -07:00
Ishaan Jaff
dcc2edbd4d fix mcp import 2025-03-24 21:02:54 -07:00
Ishaan Jaff
4a675da0f9 fix setup 2025-03-24 19:10:59 -07:00
Ishaan Jaff
08a4ba1b7e Merge branch 'main' into litellm_exp_mcp_server 2025-03-24 19:03:56 -07:00
Ishaan Jaff
395e4e73d0 langchain_mcp_adapters 2025-03-24 18:59:32 -07:00
Ishaan Jaff
ab1f87bb59 add "mcp==1.5.0" to optional pyproject for litellm proxy 2025-03-24 18:56:32 -07:00
Ishaan Jaff
6e3f86f12f fix using mcp router 2025-03-24 18:36:42 -07:00
Krrish Dholakia
34166eb65c ci: update with clean db
db had expired key
2025-03-24 17:12:18 -07:00
Ishaan Jaff
5411aed2a8 fix mcp version 2025-03-24 16:57:47 -07:00
Ishaan Jaff
fe2e2cb357 Show current pydantic version 2025-03-24 16:46:53 -07:00
Ishaan Jaff
60242f9e48 fix py version 2025-03-24 16:45:43 -07:00
Ishaan Jaff
ff8a6d7cea litellm_utils_testing 2025-03-24 16:41:50 -07:00
Ishaan Jaff
0faf23c751 update to cimg/python:3.13.1 on ci/cd 2025-03-24 16:01:32 -07:00
Ishaan Jaff
ade2ea8f20 fix order of load tests on config.yml 2025-03-24 15:55:51 -07:00
Ishaan Jaff
3395004f0a bump pydantic 2025-03-22 19:00:25 -07:00
Ishaan Jaff
e85e3bc52a bump to pip install "openai==1.68.2" 2025-03-21 21:49:54 -07:00
Ishaan Jaff
0eb3ceff60 pip install "mcp==1.4.1" 2025-03-21 19:04:53 -07:00
Ishaan Jaff
ff2aed0e2f pip install "pydantic==2.7.2" 2025-03-21 19:03:31 -07:00
Ishaan Jaff
1faf82f768 test mcp on ci/cd 2025-03-21 17:51:37 -07:00
Ishaan Jaff
c58aff01e0 bump to openai==1.67.0 2025-03-20 09:26:05 -07:00
Krish Dholakia
d4caaae1be
Merge pull request #9274 from BerriAI/litellm_contributor_rebase_branch
Litellm contributor rebase branch
2025-03-14 21:57:49 -07:00
Krrish Dholakia
4bc5f27d7d fix(config.yml): fix ci/cd config.yml 2025-03-14 20:30:09 -07:00
Krrish Dholakia
0f6e0ac06d ci(config.yml): bump websocket version on ci/cd 2025-03-13 20:01:57 -07:00
Ishaan Jaff
08f4a6844b rename folder to test openai endpoints 2025-03-13 15:13:48 -07:00
Ishaan Jaff
036ac70b72 rename test config 2025-03-13 14:55:33 -07:00
Krrish Dholakia
ee6c9576d4 fix: bump websockets version on ci/cd 2025-03-13 14:38:22 -07:00
Ishaan Jaff
aa72f95a06 "openai==1.66.1" in testing 2025-03-12 13:00:59 -07:00
Ishaan Jaff
6702e492f5 pip install openai==1.66.1 2025-03-12 11:56:34 -07:00
Ishaan Jaff
595af6898f llm_responses_api_testing 2025-03-12 11:51:22 -07:00
Ishaan Jaff
1b50e6d65f openai 1.66.1 2025-03-12 11:44:16 -07:00
Ishaan Jaff
8e30f1fb2f new test for llm_responses_api_coverage 2025-03-11 13:49:43 -07:00
Ishaan Jaff
f47987e673
(Refactor) /v1/messages to follow simpler logic for Anthropic API spec (#9013)
* anthropic_messages_handler v0

* fix /messages

* working messages with router methods

* test_anthropic_messages_handler_litellm_router_non_streaming

* test_anthropic_messages_litellm_router_non_streaming_with_logging

* AnthropicMessagesConfig

* _handle_anthropic_messages_response_logging

* working with /v1/messages endpoint

* working /v1/messages endpoint

* refactor to use router factory function

* use aanthropic_messages

* use BaseConfig for Anthropic /v1/messages

* track api key, team on /v1/messages endpoint

* fix get_logging_payload

* BaseAnthropicMessagesTest

* align test config

* test_anthropic_messages_with_thinking

* test_anthropic_streaming_with_thinking

* fix - display anthropic url for debugging

* test_bad_request_error_handling

* test_anthropic_messages_router_streaming_with_bad_request

* fix ProxyException

* test_bad_request_error_handling_streaming

* use provider_specific_header

* test_anthropic_messages_with_extra_headers

* test_anthropic_messages_to_wildcard_model

* fix gcs pub sub test

* standard_logging_payload

* fix unit testing for anthopic /v1/messages support

* fix pass through anthropic messages api

* delete dead code

* fix anthropic pass through response

* revert change to spend tracking utils

* fix get_litellm_metadata_from_kwargs

* fix spend logs payload json

* proxy_pass_through_endpoint_tests

* TestAnthropicPassthroughBasic

* fix pass through tests

* test_async_vertex_proxy_route_api_key_auth

* _handle_anthropic_messages_response_logging

* vertex_credentials

* test_set_default_vertex_config

* test_anthropic_messages_litellm_router_non_streaming_with_logging

* test_ageneric_api_call_with_fallbacks_basic

* test__aadapter_completion
2025-03-06 00:43:08 -08:00
Ishaan Jaff
f7bafbf2ab Install Node.js 2025-02-27 21:09:04 -08:00
Ishaan Jaff
51a6a219cd
(Improvements) use /openai/ pass through with OpenAI Ruby for Assistants API (#8884)
* add ruby assistants testing

* _join_url_paths

* run ruby tests on ci/cd

* TestBaseOpenAIPassThroughHandler

* _join_url_paths

* fix _join_url_paths

* Install Ruby and Bundler

* Install Ruby and Bundler
2025-02-27 20:01:16 -08:00
Ishaan Jaff
70923f7d01 add hypercorn dep to circle ci 2025-02-25 15:21:14 -08:00
Krish Dholakia
142b195784
Add anthropic thinking + reasoning content support (#8778)
* feat(anthropic/chat/transformation.py): add anthropic thinking param support

* feat(anthropic/chat/transformation.py): support returning thinking content for anthropic on streaming responses

* feat(anthropic/chat/transformation.py): return list of thinking blocks (include block signature)

allows usage in tool call responses

* fix(types/utils.py): extract and map reasoning_content from anthropic as content str

* test: add testing to ensure thinking_blocks are returned at the root

* fix(anthropic/chat/handler.py): return thinking blocks on streaming - include signature

* feat(factory.py): handle anthropic thinking blocks translation if in assistant response

* test: handle openai internal instability

* test: handle openai audio instability

* ci: pin anthropic dep

* test: handle openai audio instability

* fix: fix linting error

* refactor(anthropic/chat/transformation.py): refactor function to remain <50 LOC

* fix: fix linting error

* fix: fix linting error

* fix: fix linting error

* fix: fix linting error
2025-02-24 21:54:30 -08:00
Ishaan Jaff
55b938dd6e
(Infra/DB) - Allow running older litellm version when out of sync with current state of DB (#8695)
* fix check migration

* clean up should_update_prisma_schema

* update test

* db_migration_disable_update_check

* Check container logs for expected message

* db_migration_disable_update_check

* test_check_migration_out_of_sync

* test_should_update_prisma_schema

* db_migration_disable_update_check

* pip install aiohttp
2025-02-20 18:30:23 -08:00
Ishaan Jaff
fff15543d9
(UI + Proxy) Cache Health Check Page - Cleanup/Improvements (#8665)
* fixes for redis cache ping serialization

* fix cache ping check

* fix cache health check ui

* working error details on ui

* ui expand / collapse error

* move cache health check to diff file

* fix displaying error from cache health check

* ui allow copying errors

* ui cache health fixes

* show redis details

* clean up cache health page

* ui polish fixes

* fix error handling on cache health page

* fix redis_cache_params on cache ping response

* error handling

* cache health ping response

* fx error response from cache ping

* parsedLitellmParams

* fix cache health check

* fix cache health page

* cache safely handle json dumps issues

* test caching routes

* test_primitive_types

* fix caching routes

* litellm_mapped_tests

* fix pytest-mock

* fix _serialize

* fix linting on safe dumps

* test_default_max_depth

* pip install "pytest-mock==3.12.0"

* litellm_mapped_tests_coverage

* add readme on new litellm test dir
2025-02-19 19:08:50 -08:00
Krish Dholakia
4e38940694
Add OSS license check to ci/cd (#8626)
* feat: add oss license check for related packages

ensure all packages used are permissable

* ci(config.yml): add check_licenses.py

script to ci/cd

* ci(check_licenses.py): fix license check
2025-02-18 18:11:08 -08:00
Krish Dholakia
a9276f27f9
fix(main.py): fix key leak error when unknown provider given (#8556)
* fix(main.py): fix key leak error when unknown provider given

don't return passed in args if unknown route on embedding

* fix(main.py): remove instances of {args} being passed in exception

prevent potential key leaks

* test(code_coverage/prevent_key_leaks_in_codebase.py): ban usage of {args} in codebase

* fix: fix linting errors

* fix: remove unused variable
2025-02-15 14:02:55 -08:00
Krish Dholakia
b710407763
ci(config.yml): mark daily docker builds with -nightly (#8499)
Resolves https://github.com/BerriAI/litellm/discussions/8495
2025-02-12 18:01:08 -08:00
Ishaan Jaff
a04204db40 update load testing script 2025-02-12 16:07:06 -08:00
Ishaan Jaff
64a4229606
(e2e testing) - add tests for using litellm /team/ updates in multi-instance deployments with Redis (#8440)
* add team block/unblock test

* test_team_blocking_behavior_multi_instance

* proxy_multi_instance_tests

* test - Run Docker container 2
2025-02-10 19:33:27 -08:00
Krish Dholakia
13a3e8630e
Org UI Improvements (#8436)
* feat(team_endpoints.py): support returning teams filtered by organization_id

allows user to just get teams they belong to, within the org

Enables org admin to see filtered list of teams on UI

* fix(teams.tsx): simple filter for team on ui - just filter team based on selected org id

* feat(ui/organizations): show 'default org' in switcher, filter teams based on selected org

* feat(user_dashboard.tsx): update team in switcher when org changes

* feat(schema.prisma): add new 'organization_id' value to key table

allow org admin to directly issue keys to a user within their org

* fix(view_key_table.tsx): fix regression where admin couldn't see keys

caused by bad console log statement

* fix(team_endpoints.py): handle default org value in /team/list

* fix(key_management_endpoints.py): allow proxy admin to create keys for team they're not in

* fix(team_endpoints.py): fix team endpoint to handle org id not being passed in

* build(config.yml): investigate what pkg is installing posthog in ci/cd

* ci(config.yml): uninstall posthog

prevent it from being added in ci/cd

* ci: auto-install ci
2025-02-10 19:13:32 -08:00
Ishaan Jaff
65c91cbbbc
(QA+UI) - e2e flow for adding assembly ai passthrough endpoints (#8337)
* add initial test for assembly ai

* start using PassthroughEndpointRouter

* migrate to lllm passthrough endpoints

* add assembly ai as a known provider

* fix PassthroughEndpointRouter

* fix set_pass_through_credentials

* working EU request to assembly ai pass through endpoint

* add e2e test assembly

* test_assemblyai_routes_with_bad_api_key

* clean up pass through endpoint router

* e2e testing for assembly ai pass through

* test assembly ai e2e testing

* delete assembly ai models

* fix code quality

* ui working assembly ai api base flow

* fix install assembly ai

* update model call details with kwargs for pass through logging

* fix tracking assembly ai model in response

* _handle_assemblyai_passthrough_logging

* fix test_initialize_deployment_for_pass_through_unsupported_provider

* TestPassthroughEndpointRouter

* _get_assembly_transcript

* fix assembly ai pt logging tests

* fix assemblyai_proxy_route

* fix _get_assembly_region_from_url
2025-02-06 18:27:54 -08:00
Krish Dholakia
7e873538f6
Fix edit team on ui (#8295)
* fix(columns.tsx): fix request logs team column to indicate the value is the alias not the id

* fix(team_info.tsx): add edit team logic to team info page

* fix(team_info.tsx): re-enable updating team settings on UI

Fixes https://github.com/BerriAI/litellm/issues/8281

* fix(team_info.tsx): fix save changes on team update

* fix(teams.tsx): allow edit button to still act as a quick action button -> drop user into settings page for team

* test(config.yml): run dev ui during testing

make sure no ui regressions are pushed on main

* build: update ci/cd

* ci(config.yml): fix test

* ci: fix ci

* ci: update

* ci: fix

* ci: another attempt to get nvm working in ci/cd

* ci: fix ci

* ci: test update

* ci: test update 2

* ci: test 3

* fix(team_info.tsx): fix linting error
2025-02-05 20:13:17 -08:00
Ishaan Jaff
6cef115bb0
(Security fix) - remove code block that inserts master key hash into DB (#8268)
* remove code block upserting master key hash to db

* run test to check if key upserted into db

* run ci/cd again

* litellm_proxy_security_tests

* litellm_proxy_security_tests

* run prisma entrypoint

* ci/cd run again

* fix test master key not in db
2025-02-05 17:25:42 -08:00
Ishaan Jaff
7e1b79d446
(Bug fix) - Langfuse / Callback settings stored in DB (#8251)
* fix _decrypt_and_set_db_env_variables

* fix proxy config

* test callbacks in DB

* test langfuse callbacks in db

* test_e2e_langfuse_callbacks_in_db

* proxy_store_model_in_db_tests

* fix proxy_store_model_in_db_tests

* proxy_store_model_in_db_tests

* fix store_model_db_config.yaml

* fix check_langfuse_request

* fix test langfuse base url

* ci/cd run again
2025-02-04 21:09:37 -08:00
Ishaan Jaff
8fd60a420d
(Feat) - New pass through add assembly ai passthrough endpoints (#8220)
* add assembly ai pass through request

* fix assembly pass through

* fix test_assemblyai_basic_transcribe

* fix assemblyai auth check

* test_assemblyai_transcribe_with_non_admin_key

* working assembly ai test

* working assembly ai proxy route

* use helper func to pass through logging

* clean up logging assembly ai

* test: update test to handle gemini token counter change

* fix(factory.py): fix bedrock http:// handling

* add unit testing for assembly pt handler

* docs assembly ai pass through endpoint

* fix proxy_pass_through_endpoint_tests

* fix standard_passthrough_logging_object

* fix ASSEMBLYAI_API_KEY

* test test_assemblyai_proxy_route_basic_post

* test_assemblyai_proxy_route_get_transcript

* fix is is_assemblyai_route

* test_is_assemblyai_route

---------

Co-authored-by: Krrish Dholakia <krrishdholakia@gmail.com>
2025-02-03 21:54:32 -08:00