litellm/tests/test_litellm
Ishaan Jaff 373e5e316b
feat(mcp): BYOM — non-admin MCP server submission + admin review workflow (#23205)
* feat(mcp): add BYOM (Bring Your Own MCPs) submission + admin review workflow

Non-admins can now submit MCP servers for review via POST /v1/mcp/server/register.
Admins get a Submissions tab in the UI to approve or reject pending servers.
Approved servers enter the active runtime; rejected ones stay out with notes.

- DB: add approval_status, submitted_by, submitted_at, reviewed_at, review_notes
  to LiteLLM_MCPServerTable with migration
- Backend: new endpoints register, submissions, approve, reject
- reload_servers_from_database now only loads approval_status=active servers
- UI: Submissions tab with stat cards, card list, confirm dialogs; non-admin
  "Submit MCP Server" button wired to /register endpoint
- Fix get_mcp_submissions to filter by submitted_at IS NOT NULL (not submitted_by,
  which can be null for team-scoped keys without an associated user)

* feat(mcp): rename nav item to Team MCPs + add New badge

* fix(mcp): revert nav label, rename Submissions tab to Team MCPs + New badge

* feat(mcp): add MCP Standards — required fields config + CI-style checks on submissions

Adds a "Standards" tab (admin-only) to MCP Servers where admins define which
server fields are required for a submission to pass. Each submission card in
Team MCPs then shows a green ✓ or red ✗ for each required field, with a
summary "N/M checks" badge in the header — like GitHub CI status rows.

Also adds a `source_url` field (GitHub / Source URL) to the MCP server schema
so non-admins can link to the source repo when submitting a server.

- schema.prisma: add `source_url String?` to LiteLLM_MCPServerTable
- migration: 20260309000001_add_mcp_source_url
- _types.py: source_url on NewMCPServerRequest, UpdateMCPServerRequest, LiteLLM_MCPServerTable
- types.tsx: source_url on MCPServer interface
- create_mcp_server.tsx: GitHub/Source URL form field
- MCPStandardsSettings.tsx: new — toggle which fields are required (stored in general settings as mcp_required_fields)
- mcp_servers.tsx: Standards tab (admin-only)
- MCPSubmissionsTab.tsx: load required fields + CI-style check pills on each card

* refactor(mcp): move submission rules into Team MCPs tab, grouped free-form UI

Folds the Standards tab into Team MCPs. Submission Rules panel now lives at the
top of the Team MCPs tab — collapsible, shows active rules as chips when closed,
expands to a grouped checkbox editor (Documentation / Source / Connection /
Security). Removes the separate Standards tab from the nav.

MCPStandardsSettings.tsx is now constants-only (FIELD_GROUPS, MCP_REQUIRED_FIELD_DEFS,
SETTINGS_KEY) — the UI lives in MCPSubmissionsTab.

* feat(mcp): add mcp_required_fields to ConfigGeneralSettings + config/list endpoint

Registers mcp_required_fields as a proper general_settings field so the UI
can read/write it via /config/list and /config/field/update without the
"Invalid field" error. Also fixes a pre-existing pyright None-check issue
in _sync_ui_settings_to_general_settings.

* ui(mcp): GitHub-style PR checks panel on submission cards

* ui: rename Team MCPs -> Submitted Tools, Team Guardrails -> Submitted Guardrails

* address greptile review feedback (greploop iteration 1)

* fix: inline import, add approval workflow tests, rename Submitted MCPs

* fix(mcp): allow re-approval of rejected MCP server submissions

* fix(mcp): evict rejected servers from runtime; enforce mcp_required_fields on /register

* fix(mcp): sort submissions newest-first; force active status on admin-created servers

* fix(mcp): add missing mock in test, show Approve for rejected, clear submission metadata, drop spurious Content-Type

* fix(mcp/ui): show Reject for active servers; show submit form to non-admins with team-key note

* fix(mcp): conditional reload on reject; view-only admin for submissions; block admin from /register

* fix(mcp): match auth_type required-field validation to UI compliance check (reject 'none')

* fix(mcp): block view-only admin from /register; log settings failure; warn on active server reject

* fix(mcp): allow view-only admin to use /register; add _validate_mcp_required_fields tests

* fix(mcp): validate field names in mcp_required_fields; surface backend error in submit UI

* fix(mcp): fix falsy field check; add field-name validation; add take limit; document server-managed fields; close dialog on error
2026-03-10 13:58:59 -07:00
..
a2a_protocol [Fix] A2a Agent Gateway Fixes - A2A agents deployed with localhost/internal URLs in their agent cards (e.g., http://0.0.0.0:8001/) (#20604) 2026-02-06 15:02:34 -08:00
anthropic_interface/exceptions [bug fix] do not fallback to token counter if disable_token_counter is enabled (#19041) 2026-01-13 16:53:38 -08:00
caching Merge branch 'main' into litellm_oss_staging_03_02_2026 2026-03-10 17:19:37 +05:30
completion_extras Merge pull request #23151 from BerriAI/litellm_preserve-reasoning-summary-for-responses-api 2026-03-10 18:18:46 +05:30
containers fix: add missing OpenAI chat completion params to OPENAI_CHAT_COMPLETION_PARAMS (#21360) 2026-02-16 20:31:21 -08:00
enterprise Address review feedback: remove dead code, add error handling, strengthen test assertions 2026-03-06 09:25:50 -08:00
expected_responses_api_request [Feat] Adds support for server-side compaction on the OpenAI Responses API context_management (#21058) 2026-02-12 10:00:30 -08:00
experimental_mcp_client feat(mcp): add token authentication support for MCP servers 2026-03-10 18:33:08 +02:00
google_genai litellm_fix_mapped_tests_core: fix test isolation and mock injection issues (#20209) 2026-01-31 17:53:54 -08:00
images Merge pull request #22307 from Chesars/fix/22244-image-edit-custom-pricing 2026-02-27 16:38:34 -03:00
integrations fix: anthropic prompt caching returns a string instead of an int 2026-03-10 13:51:17 -04:00
interactions fix(test): Update status enum values to match Google Interactions OpenAPI spec (#22061) 2026-02-24 20:26:11 -08:00
litellm_core_utils fix: req changes by greptile 2026-03-10 18:33:17 +05:30
llms Merge pull request #23243 from BerriAI/litellm_bedrock-completion-tokens-details 2026-03-10 18:19:28 +05:30
ocr Enable local file support for OCR (#22133) 2026-02-27 10:50:02 -08:00
passthrough fix(passthrough): propagate Azure 429/5xx errors in async streaming instead of silent HTTP 200 (#22913) 2026-03-05 10:12:43 -08:00
proxy feat(mcp): BYOM — non-admin MCP server submission + admin review workflow (#23205) 2026-03-10 13:58:59 -07:00
responses fix(responses): merge parallel function_call items into single assistant message (#23116) 2026-03-09 09:01:31 -07:00
router_strategy fix: complexity_router crashes on list-format message content (OpenAI multi-part messages) (#22761) 2026-03-04 16:18:49 -08:00
router_utils Fix encrypted content streaming affinity issue 2026-03-03 18:37:22 +05:30
secret_managers fix(tests): isolate flaky files endpoint tests from global proxy state (#21788) 2026-02-21 11:20:32 -08:00
test_router fix: use atomic increment-first pattern for model RPM rate limiting 2026-02-24 09:55:07 -03:00
types fix: add video_tokens to expected completion_tokens_details in test 2026-03-03 19:46:20 -03:00
vector_stores litellm_fix_mapped_tests_core: fix test isolation and mock injection issues (#20209) 2026-01-31 17:53:54 -08:00
__init__.py
conftest.py fix(tests): restore disable_aiohttp_transport and force_ipv4 in isolate_litellm_state 2026-02-17 21:18:49 -03:00
log.txt
readme.md
test_a2a_registry_lookup.py [Feat] Use A2A registered agents with /chat/completions (#20362) 2026-02-03 15:25:38 -08:00
test_acompletion_session_reuse_e2e.py
test_add_deployment_no_master_key.py
test_aembedding_session_reuse_e2e.py
test_anthropic_beta_headers_filtering.py Make tests run with local beta header mapping json 2026-02-13 22:31:42 +05:30
test_azure_video_router.py
test_claude_haiku_4_5_config.py
test_claude_opus_4_6_config.py Fix apac.anthropic.claude-sonnet-4-6 -> au.anthropic.claude-sonnet-4-6 (#21466) 2026-03-05 15:54:26 -08:00
test_constants.py added configurable env for mcp timeouts (#22287) 2026-03-02 13:13:41 -08:00
test_container_router.py
test_cost_calculation_log_level.py fix(tests): use record.getMessage() instead of record.message for LogRecord 2026-02-18 11:46:32 -03:00
test_cost_calculator.py perf: optimize completion_cost() — eliminate enum overhead, reduce function call indirection 2026-02-21 12:14:55 -08:00
test_deepseek_model_metadata.py fix(model-info): sync DeepSeek model metadata and add bare-name fallback (#20885) 2026-02-11 12:48:10 +05:30
test_eager_tiktoken_load.py fix(main): use local tiktoken cache in lazy loading (#19774) 2026-01-27 18:16:58 -08:00
test_exception_exports.py fix: export PermissionDeniedError from litellm.__init__ 2026-02-11 13:39:19 +01:00
test_exception_header_preservation.py Update test to righ place 2026-02-26 13:26:51 -08:00
test_exception_mapping_request_attribute.py
test_filter_out_litellm_params.py
test_get_blog_posts.py fix(ollama): thread api_base to get_model_info + graceful fallback (#21970) 2026-02-23 21:00:37 -08:00
test_gpt_image_cost_calculator.py Fix gpt-image-1.5 cost calculation not including output image tokens (#19515) 2026-01-22 19:42:15 -08:00
test_groq_streaming_encoding.py
test_lazy_imports.py Fix: test_token_counter_lazy_imports 2026-01-08 16:44:35 +05:30
test_litellm_params_reserved_keys.py Fix TypeError: LiteLLM_Params.__init__() got multiple values for argument 'self' (#23220) 2026-03-09 19:33:52 -07:00
test_logging.py fix:Parse embedded JSON in the message field of logs (#20366) 2026-02-10 16:13:33 +05:30
test_lowest_latency_zero_tokens.py
test_main.py fix(models): set gpt-5.4-pro mode to responses instead of chat 2026-03-09 12:10:08 +01:00
test_model_param_helper.py perf: cache _get_relevant_args_to_use_for_logging() at module level (#20077) 2026-02-02 10:54:49 -08:00
test_model_response_normalization.py fix(types): remove StreamingChoices from ModelResponse, use ModelResponseStream 2026-02-20 17:47:42 -03:00
test_nested_drop_params.py
test_project_tags_pydantic.py fix: req changes 2026-02-27 13:33:34 +05:30
test_redis.py
test_register_model_custom_pricing.py test: fix misleading precedence test per review feedback 2026-03-02 08:31:33 +00:00
test_responses_api_bridge_non_stream.py fix: Pydantic will fail to parse it because cached_tokens is required but not provided 2026-01-28 11:51:26 +05:30
test_responses_id_security.py Fix responses ID security test for new request_cache parameter 2026-03-04 11:29:51 -03:00
test_router.py CircleCI test stability (#23055) 2026-03-07 15:19:39 -08:00
test_router_google_genai.py
test_router_model_cost_isolation.py [Fix] prevent shared backend model key from being polluted by per-deployment custom pricing (#20679) 2026-02-09 19:38:44 -08:00
test_router_per_deployment_num_retries.py Bugfix/19481 num retries env var type (#19507) 2026-01-22 19:39:58 -08:00
test_router_redis_init.py fix: handle deprecated 'redis_db' arg to prevent crash (#19808) 2026-02-02 18:18:05 +05:30
test_router_retry_non_retryable_errors.py fix(router): break retry loop on non-retryable errors (#21370) 2026-02-17 22:58:08 -08:00
test_router_silent_experiment.py fix: req changes from greptile 2026-03-10 13:03:29 +05:30
test_service_logger.py fix(proxy): fix master key rotation Prisma validation errors (#21330) 2026-02-16 15:13:05 -08:00
test_shared_session_integration.py
test_ssl_verify_unit.py BUMP Enterprise PIP 2026-02-14 13:40:48 -08:00
test_streaming_connection_cleanup.py fix: add debug logging to stream cleanup, improve tests 2026-02-14 17:31:39 -08:00
test_system_message_format_bug.py
test_utils.py Merge pull request #23182 from BerriAI/litellm_/exciting-swanson 2026-03-09 14:26:21 -07:00
test_uuid_helper.py
test_video_generation.py fix(ollama): thread api_base to get_model_info + graceful fallback (#21970) 2026-02-23 21:00:37 -08:00
test_xai_responses_auto_routing.py Add routing of xai chat completions to responses when web search options is present 2026-01-30 14:15:35 +05:30

Testing for litellm/

This directory 1:1 maps the the litellm/ directory, and can only contain mocked tests.

The point of this is to:

  1. Increase test coverage of litellm/
  2. Make it easy for contributors to add tests for the litellm/ package and easily run tests without needing LLM API keys.

File name conventions

  • litellm/proxy/test_caching_routes.py maps to litellm/proxy/caching_routes.py
  • test_<filename>.py maps to litellm/<filename>.py