Commit graph

2651 commits

Author SHA1 Message Date
Ishaan Jaff
81eb2fdd30
[Feat] UI Vector Stores - Allow adding Vertex RAG Engine, OpenAI, Azure (#12752)
* fix _pass_through_endpoint_without_required_model

* add get_litellm_managed_vector_store_from_registry

* undo router change

* fix for using router + vector search methods

* add simple helper for _update_request_data_with_litellm_managed_vector_store_registry

* add vector_stores routes

* test_router_avector_store_search_passes_correct_args

* [Feat] UI - Allow clicking into Vector Stores (#12741)

* Add View Vector Store

* add /info for vector store

* fix updated_at

* allow easily testing the KB on litellm

* fix

* rename test

* test_init_vector_store_api_endpoints

* add get_vertex_ai_project

* fixes to vertex transformation for RAG Engine

* fix vectorStoreProviderFields

* Add Vertex Rag engine

* add oai, azure

* fix validate_environment

* fix provider name

* fix tester

* working vertex vector store
2025-07-18 18:25:26 -07:00
Ishaan Jaff
5802a5bbe3
[Feat] LLM API Endpoint - Expose OpenAI Compatible /vector_stores/{vector_store_id}/search endpoint (#12749)
* fix _pass_through_endpoint_without_required_model

* add get_litellm_managed_vector_store_from_registry

* undo router change

* fix for using router + vector search methods

* add simple helper for _update_request_data_with_litellm_managed_vector_store_registry

* add vector_stores routes

* test_router_avector_store_search_passes_correct_args

* [Feat] UI - Allow clicking into Vector Stores (#12741)

* Add View Vector Store

* add /info for vector store

* fix updated_at

* allow easily testing the KB on litellm

* fix

* rename test

* test_init_vector_store_api_endpoints

* test_update_request_data_with_litellm_managed_vector_store_registry
2025-07-18 18:18:53 -07:00
Ishaan Jaff
99e2ea081d
[Feat] UI - Allow clicking into Vector Stores (#12741)
* Add View Vector Store

* add /info for vector store

* fix updated_at
2025-07-18 14:28:57 -07:00
Ishaan Jaff
3051a9c68a
[Bug Fix] QA - Use PG Vector Vector Store with LiteLLM (#12716)
* store in generic litellm params

* handle storing litellm_params_json

* add litellm_params

* update file loc

* fix startup issue

* ui fix litellm_params

* fix typing

* fix adding PG Vector

* use litellm params

* fix transform_search_vector_store_request

* fix transform

* fix URL

* test fix

* test_pg_vector_search_request_construction
2025-07-18 08:41:18 -07:00
tanjiro
663abbe275
Regenerate Key State Management and Authentication Issues (#12729)
* fix bug

When Max Budget, TPM, RPM, Expire Key are set on Regenerate Key, these values are not reflected on the settings page. Refreshing the page is required.

* fix editing settings after key generation
2025-07-18 07:25:01 -07:00
Krish Dholakia
bc2ab84342
Teams - allow setting custom key duration + show many user + service account keys have been created (#12722)
* fix(team_info.tsx): allow setting custom key duration

more flexible than previous pre-set options

* feat(team_info.tsx): show how many user + service account keys have been created within a team
2025-07-17 22:33:59 -07:00
Jugal D. Bhatt
f4a94ec911
Added name change (#12702) 2025-07-17 18:35:01 -07:00
Jugal D. Bhatt
842af61a05
[MCP Gateway] add fix to update object permission on update/delete key/team (#12701)
* add fix to update object permission on update/delete

* fix edit team page dropdown
2025-07-17 18:34:35 -07:00
Ishaan Jaff
e5f0a8477b
[Feat] New Vector Store - PG Vector (#12667)
* add PGVectorStoreConfig

* add PGVectorStoreConfig

* test_environment_variable_support

* fix code QA check

* rename test

* add PG vector img

* allow adding vector stores

* add pg vector

* add vector store

* TestPGVectorStoreConfig

* TestPGVectorStoreConfig
2025-07-16 18:17:05 -07:00
Ishaan Jaff
538339e1a8
[Feat] UI - Add end_user filter on UI (#12663)
* add end_user filter on Logs

* add end user filter
2025-07-16 15:20:31 -07:00
Krrish Dholakia
0fb7c01382 fix(make_model_public_form.tsx): make distinction clearer 2025-07-16 10:58:23 -07:00
tanjiro
720b94fd2b
Add Copy-on-Click for IDs (#12615)
* added the mcp_connect copy button to key_info_view

* user id copy button

* copy button on session view

* model id copy

* more copy buttons

* icon size
2025-07-15 11:25:57 -07:00
tanjiro
c0f1b8119e
Wildcard model filter (#12597)
* filter by wildcard models

* merge conflict
2025-07-14 20:01:10 -07:00
Ishaan Jaff
8bcdb07fcd ui new build 2025-07-12 16:44:13 -07:00
Ishaan Jaff
50b106be11
[MCP Gateway] Ensure we use the same param for specifying groups (#12561)
* just use 1 param for mcp groups

* fix just use 1 param for access groups

* test_get_tools_from_mcp_servers

* docs access groups

* group MCPs

* test fix

* fix screenshots on docs

* TestMCPAccessGroupsE2E

* update img

* fix MCP connect
2025-07-12 16:41:58 -07:00
Ishaan Jaff
00ab400500
UI - v1.74.3-stable QA fixes (#12559)
* fix chat ui

* fixes for types

* ui - fix back button on MCP

* mcp - show existing groups when creating new mcps

* fix design

* fix color scheme

* fix linting

* fixes for mapped tests
2025-07-12 15:31:39 -07:00
Krish Dholakia
d8ae044ec5
UI - Model Hub - refactor 'Make Public' to have a select + confirm form (#12555)
* fix(model_hub_table.tsx): fix null link

* fix(model_hub_table.tsx): add tooltip on disabled make public

tell user how to enable button

* feat(model_hub_table.tsx): make Model Hub OSS

allow more teams to share available models via litellm

* fix(model_hub_table_columns.tsx): make table view only model hub view, rename 'model' column to 'public model name', make model hub url copyable

* fix(page.tsx): fix logo re-rendering

* fix(public_model_hub.tsx): fix theme

* style(public_model_hub.tsx): move filters into same card as model table

filters are for table

* fix(page.tsx): fix ui linting error

* refactor(model_hub_table.tsx): refactor public model selection to inside the 'make public' button - modal

similar flow to adding guardrails

consistent flow for doing stuff on the proxy

* style(ui/): make 'make public' modal wider, to have all filters on same horizontal axis

* style(model_hub_table.tsx): have already public models be pre-selected

* fix(model_hub_table.tsx): ensure only admin can make models public via UI

* fix: fix linting errors

* fix(model_hub_table.tsx): make model hub purpose clearer

* fix(model_hub_table_columns.tsx): fix linting error
2025-07-12 14:12:52 -07:00
Krish Dholakia
1f5ab2ee74
UI - Model Hub Page - minor fixes + improvements (+ Make Model Hub OSS 🚀) (#12553)
* fix(model_hub_table.tsx): fix null link

* fix(model_hub_table.tsx): add tooltip on disabled make public

tell user how to enable button

* feat(model_hub_table.tsx): make Model Hub OSS

allow more teams to share available models via litellm

* fix(model_hub_table_columns.tsx): make table view only model hub view, rename 'model' column to 'public model name', make model hub url copyable

* fix(page.tsx): fix logo re-rendering

* fix(public_model_hub.tsx): fix theme

* style(public_model_hub.tsx): move filters into same card as model table

filters are for table

* fix(page.tsx): fix ui linting error
2025-07-12 14:10:03 -07:00
Jugal D. Bhatt
5bd63c5a72
[MCP Gateway] access group fixes on UI for keys and teams (#12556)
* add changes to mcp servers in team and keys

* added access group in bracket

* added access group capitalise
2025-07-12 12:55:36 -07:00
Ishaan Jaff
8fe0007f3c ui new build 2025-07-12 11:01:20 -07:00
Jugal D. Bhatt
3a122bc48e
[MCP Gateway] UI headers groups example on connect tab (#12550)
* added mcp tools on internal user and divide it by teams

* add support for server api call

* Added frontend for test key

* added tools used output

* fix ui for servers

* All servers to personal

* change columns format

* revert ui logic

* Added vertical align

* fix mapped tests

* fix lint

* fix lint

* remove extra file

* fix ui test

* comments fixes

* change query type

* change query type

* mcp acces group init

* add ability to change server display on ui through access groups

* Mcp access group names UI (#12486)

* Added ui changes to reflect mcp_access_groups

* fix edit mcp page

* change to string array (#12491)

* change to string array

* Remove print

* add ability to change server display on ui through access groups

* Litellm mcp access groups accesses (#12498)

* added mcp access groups for keys and teams

* added access groups above servers

* fixed ruff

* fixed mypy

* revert couple changes

* fix test

* fixed double asterisks

* Litellm mcp groups UI (#12522)

* add ui for teams

* fix object permissions

* fix mcp servers test object permission

* remove print

* add helper method

* add tests + remove logs

* add mcp access group servers to test key

* add mcp access group support for headers

* lint fix

* add tests and helper  function

* fixed test

* change list -> List

* tests

* added ui for selecting servers
2025-07-12 10:42:43 -07:00
Krish Dholakia
d202ce229b
Prevent writing default user setting updates to yaml (error in non-root env) + Use central team member budget when max_budget_in_team set on UI (#12533)
* fix(proxy_setting_endpoints.py): require store model in db is enabled for setting user default settings

* test(test_proxy_server.py): update test

* fix(reset_budget_job.py): initial commit adding reset budget logic for team members

* test: update unit testing

* test(test_proxy_budget_reset.py): validate team member budget was reset

* test(test_reset_budget_job.py): update unit tests

* test: update tests
2025-07-12 10:13:07 -07:00
tanjiro
2faba5f215
Consistent layout for Create and Back buttons on all the pages (#12542)
* create new model button

* cleanup + prettier

* prettier

* create new model button added for model and model info+ back button added

* prettier

* team and team info has create team button + back to teams button added in team_info

* prettier

* better prettier

* always show create new organization button and add back to organizations button

* minor, prettier

* prettier

* remove user header and prettier

* move top buttons to the left

* prettier

* back to all logs button style fixed

* prettier

* modify button text

* create new guardrail button style

* prettier

* button modifications

* guardrails page done

* prettier

* add new guardrail text button + prettier on mcp

* button related changes

* prettier

* mcp servers page done
2025-07-12 08:18:02 -07:00
Krrish Dholakia
8b4673fe84 build(ui/): UI - Public Model Hub v2 (#12532)
Closes https://github.com/BerriAI/litellm/pull/12532
2025-07-11 22:24:31 -07:00
Ishaan Jaff
a1a979fae1 add back MCPConnectionStatus 2025-07-11 22:20:43 -07:00
Ishaan Jaff
8e06a42c63 new ui build 2025-07-11 20:23:58 -07:00
Ishaan Jaff
d931446a79
[MCP Gateway] Allow using stdio MCPs with LiteLLM (#12530)
* update types

* add STDIO to client

* allow using STDIO with MCP manager

* add Stdio-specific fields to schema.prisma

* fixes for MCP mgmt

* fix for adding stdio MCP to DB

* ui - allow adding stdio MCPs

* fix MCP server manager

* docs stdio MCP

* add_stdio_mcp.png

* new stdio tests

* allow adding MCPs through config.yaml

* fix tool test panel

* use TestMCPClient

* ui fixes for testing circle ci mcp
2025-07-11 20:21:02 -07:00
Jugal D. Bhatt
db57e765a5
[MCP Gateway] UI Quality check fixes (#12521)
* Add tooltip to mcp server access groups

* fix ui issues
2025-07-11 18:26:43 -07:00
Jugal D. Bhatt
38c9b69e8f
[MCP Gateway] access group UI object permission fix (#12523)
* added mcp tools on internal user and divide it by teams

* add support for server api call

* Added frontend for test key

* added tools used output

* fix ui for servers

* All servers to personal

* change columns format

* revert ui logic

* Added vertical align

* fix mapped tests

* fix lint

* fix lint

* remove extra file

* fix ui test

* comments fixes

* change query type

* change query type

* mcp acces group init

* add ability to change server display on ui through access groups

* Mcp access group names UI (#12486)

* Added ui changes to reflect mcp_access_groups

* fix edit mcp page

* change to string array (#12491)

* change to string array

* Remove print

* add ability to change server display on ui through access groups

* Litellm mcp access groups accesses (#12498)

* added mcp access groups for keys and teams

* added access groups above servers

* fixed ruff

* fixed mypy

* revert couple changes

* fix test

* fixed double asterisks

* Litellm mcp groups UI (#12522)

* add ui for teams

* fix object permissions

* fix mcp servers test object permission

* remove print

* add helper method

* add tests + remove logs
2025-07-11 18:25:40 -07:00
Ishaan Jaff
4f9793951d
[Bug Fix] - QA for MCP Gateway - show the cost config on the root of MCP Settings (#12526)
* show cost display

* fixes MCP cost display
2025-07-11 17:34:34 -07:00
Ishaan Jaff
23759c9a64
[MCP Gateway] QA - MCP Tool Testing Playground (#12520)
* fix test tool modal

* show formatted / JSON

* fix layout

* show duration

* add new layout

* v2

* fix MCP tool

* add handleCopyToolName

* fix ui qa checklist
2025-07-11 15:11:39 -07:00
Ishaan Jaff
cb96f1d740
[Feat] - New guardrail - OpenAI Moderations API (#12519)
* add OpenAIModerationGuardrailConfigModel

* add OpenAIGuardrailBase

* _extract_response_text

* fixes for openai moderation guard

* docs OpenAI Moderation

* add OpenAIModerationGuardrailConfigModel

* use literal for model

* fix init openai moderation

* test_openai_moderation_guardrail_adds_to_litellm_callbacks

* show default on on UI

* fix code qa
2025-07-11 11:25:45 -07:00
Jugal D. Bhatt
7b44576216
Validation to mcp server name (#12515)
* added validation

* add helper util
2025-07-11 11:24:42 -07:00
Jugal D. Bhatt
1b10dda2f4
Litellm mcp access group (#12514)
* added mcp tools on internal user and divide it by teams

* add support for server api call

* Added frontend for test key

* added tools used output

* fix ui for servers

* All servers to personal

* change columns format

* revert ui logic

* Added vertical align

* fix mapped tests

* fix lint

* fix lint

* remove extra file

* fix ui test

* comments fixes

* change query type

* change query type

* mcp acces group init

* add ability to change server display on ui through access groups

* Mcp access group names UI (#12486)

* Added ui changes to reflect mcp_access_groups

* fix edit mcp page

* change to string array (#12491)

* change to string array

* Remove print

* add ability to change server display on ui through access groups

* Litellm mcp access groups accesses (#12498)

* added mcp access groups for keys and teams

* added access groups above servers

* fixed ruff

* fixed mypy

* revert couple changes

* fix test

* fixed double asterisks
2025-07-11 07:50:18 -07:00
Jugal D. Bhatt
e3437795bb
Litellm mcp access group on UI (#12470)
* added mcp tools on internal user and divide it by teams

* add support for server api call

* Added frontend for test key

* added tools used output

* fix ui for servers

* All servers to personal

* change columns format

* revert ui logic

* Added vertical align

* fix mapped tests

* fix lint

* fix lint

* remove extra file

* fix ui test

* comments fixes

* change query type

* change query type

* mcp acces group init

* add ability to change server display on ui through access groups

* Mcp access group names UI (#12486)

* Added ui changes to reflect mcp_access_groups

* fix edit mcp page

* change to string array (#12491)

* change to string array

* Remove print

* add ability to change server display on ui through access groups

* Litellm mcp access groups accesses (#12498)

* added mcp access groups for keys and teams

* added access groups above servers

* fixed ruff

* fixed mypy

* revert couple changes
2025-07-11 10:29:25 +05:30
Ishaan Jaff
feb6cd8732 ui new build 2025-07-10 18:17:03 -07:00
Cole McIntosh
28a6cec77c
feat: improve user dropdown UI with premium badge and cleaner layout (#12502)
- Add premium/standard badge with crown icon next to user ID
- Remove colons and improve visual hierarchy with icons
- Add tooltips for user status (Premium User / Upgrade message)
- Enhance spacing and typography for better readability
- Update dropdown styling with proper hover states
2025-07-10 18:14:00 -07:00
Ishaan Jaff
52e4cc9a04
[Feat] Add support for editing MCP cost per tool (#12501)
* fix - allow editing cost per tool

* add MCPServerCostDisplay

* fix MCP tool call

* update_mcp_server

* ui fix MCP

* fix for viewing MCPs
2025-07-10 17:58:51 -07:00
Ishaan Jaff
5fdbb218a8
[MCP Gateway] - Add custom cost configuration for each MCP tool (#12499)
* add endpoints to test MCP connection

* fix route names

* add testMCPToolsListRequest

* fix MCP connection test

* add tool_name_to_cost_per_query

* fixes tool_name_to_cost_per_query

* fix networking

* fix test MCP connection

* use POST for test tools endpoints

* ui fixes

* fix config

* fixes for cost config

* fixes

* decent connection status

* fix Created At

* fix MCP table

* Potential fix for code scanning alert no. 2928: Information exposure through an exception

Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>

* MCPServerCostConfig

---------

Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>
2025-07-10 17:00:39 -07:00
Krish Dholakia
0730f61127
OTEL - OTEL_RESOURCE_ATTRIBUTES support + Model Hub - new model hub table view, new /public/model_hub endpoint, fix duplicates in /model_group/info (#12468)
* fix(opentelemetry.py): initial commit adding support for OTEL_RESOURCE_ATTRIBUTES

* test: update unit tests

* feat(ui/): initial commit with revamped model hub

* fix(model_hub_table.tsx): generic 'supports_' rendering as capabilities

* feat(model_hub_table.tsx): indicate if a model is publicly available

* refactor(model_hub_table.tsx): refactor to use common model data table component

* fix(model_hub_table.tsx): fix box sizes

* fix(model_hub_table.tsx): enable selecting / deselecting models in columns

* feat(public_endpoints.py): initial commit adding `/public/model_hub` endpoint

enables sharing public models

* feat(public_endpoints/): instrumentation for public model hub route

* feat(proxy_server.py): support request access form for model groups

allows user to request access to a model

* refactor: use a dictionary of text + link instead of 1 hardcoded request access form

* fix(proxy_server.py): prevent duplicates in model_group info

* fix: fix linting error

* fix(__init__.py): fix linting error
2025-07-09 22:00:57 -07:00
Jugal D. Bhatt
a12bbe7da4
Litellm mcp internal users (#12458)
* added mcp tools on internal user and divide it by teams

* add support for server api call

* Added frontend for test key

* added tools used output

* fix ui for servers

* All servers to personal

* change columns format

* revert ui logic

* Added vertical align

* fix mapped tests

* fix lint

* fix lint

* remove extra file

* fix ui test

* comments fixes

* change query type

* change query type

* change query type

* Update model_checks.py
2025-07-09 16:30:51 -07:00
Krish Dholakia
c4af2eb5e2
MCP - usage tracking (#12397)
* fix(common_daily_activity.py): initial commit with working mock BE endpoint for mcp usage

* feat(ui/): show mcp server activity on UI

allows admin to know which mcp's are being used

* feat(common_daily_activity.py): return activity by key

* feat(ui/): show top api keys for a given model / mcp server

allow user to know which key is driving spend

* fix(common_daily_activity.py): use known mcp server names

* feat(server.py): log the namespaced tool name (includes server prefix)

allow accurate cost tracking

* feat(db_spend_update_writer.py): log by mcp_namespaced_tool_name

store aggregate daily activity by mcp_namespaced_tool_name

Enables cost / usage tracking by mcp tool name

* fix(server.py): add key/user metadata to mcp calls

* refactor(common_daily_activity.py): update to return mcp activity in API

* fix(common_daily_activity.py): handle empty key

* fix(common_daily_activity.py): track when api key is empty

* test(test_spend_management_endpoints.py): update tests

* fix: fix ui linting error

* fix: fix linting errors

* test: add missing key

* build(schema.prisma): add mcp tool tracking

* fix(migration.sql): add schema migration file

* feat(server.py): add request logging for mcp calls

enables storing the mcp calls

* fix(new_usage.tsx): fix linting errors

* fix: fix code qa errors

* fix(activity_metrics.tsx): fix ui linting errors post-merge

* fix(types/utils.py): fix linting error

* fix(server.py): always have name
2025-07-08 22:08:16 -07:00
tanjiro
df2a2fac43
Prevent navigation reset after team member operations (#12424)
* stay on member page when you delete or add a member

* update member fix
2025-07-08 15:07:13 -07:00
Jorge Piedrahita Ortiz
7fdecffc9f
style: update sambanova logos (#12431) 2025-07-08 13:56:50 -07:00
tanjiro
3cb45ccb6a
Improve Chart Readability for Tags (#12378)
* Enhance chart readability with short-form notation for large numbers (#11571)

* format y-axis value for total tokens

* format y-axis for the rest of the charts on model-activity

* revert changes for requests per day

* labels modified to plain text

* added plain text label for api_requests and spend

* minor

* improve keyname display (#11573)

* fix tags overlap

- no need to fix decimal points for valueFormatterSpend() bcz we're only going to use it for values on the x and y axis of a chart
- show only top 5 values of tags
- format the values on the x-axis

* format spend values to short form

* format spend to short form

* showing top 5 tags data more perceptively

* remove mock data

* remove console.log

* prettier

* prettier + remove unused functions
2025-07-07 23:02:44 -07:00
Ishaan Jaff
779791f7b3 ui new build 2025-07-07 21:13:53 -07:00
Ishaan Jaff
5cad0dd94b
[Feat] Add MCP Cost Tracking (#12385)
* fix MCP_TOOL_NAME_PREFIX

* test_mcp_cost_tracking

* init MCPCostCalculator

* add call_mcp_tool

* calculate_mcp_tool_call_cost

* add MCPServerCostInfo

* add mcp_server_cost_info

* add mcp_server_cost_info

* pass through litellm_logging_obj

* logged_standard_logging_payload

* fix logging MCP tool call

* test_mcp_cost_tracking_per_tool

* fix NewMCPServerRequest

* fix add_update_server

* add MCP info to schema.prisma

* fix create_mcp_server

* working custom cost per call

* fix mcp server cost

* fix MCPCostCalculator

* TestMCPCostCalculator
2025-07-07 21:08:10 -07:00
tanjiro
32f3887a01
improve keyname display (#12371) 2025-07-07 12:58:06 -07:00
tanjiro
8bb9ea8568
sticky session for test key (#12365) 2025-07-07 10:12:39 -07:00
Krrish Dholakia
0cd4d4f76d build(ui/): new ui build 2025-07-05 20:04:42 -07:00