litellm/tests
tin-berri 2e38da6b3e
feat(mcp): add entra_obo profile to the token_exchange (OBO) arm (#31983)
* feat(mcp): add entra_obo profile to the token_exchange (OBO) arm

Microsoft Entra On-Behalf-Of uses the RFC 7523 jwt-bearer grant rather than RFC 8693, so the existing token_exchange arm cannot mint tokens against Entra. This adds an entra_obo profile on TokenExchangeConfig that switches the request form to Entra's jwt-bearer OBO dialect (the inbound token as assertion, the target resource carried in scope, and requested_token_use=on_behalf_of), while reusing the shared caching, single-flight, TTL, and fail-closed machinery. The exchanger is renamed from Rfc8693TokenExchanger to OboTokenExchanger since it now serves both dialects

The profile is threaded through the config-load path, the DB credentials blob, and the MCPCredentials request schema, so an operator can select it from YAML or the management API. It rides in the existing credentials JSON blob, so there is no new auth_type and no DB migration

Resolves LIT-4163

* feat(mcp): propagate the Entra Conditional Access step-up challenge on the OBO 401

An entra_obo exchange that the IdP rejects for Conditional Access returns a 4xx with
error=interaction_required and a claims blob the client must satisfy to step up. The arm
dropped both and emitted a static RFC 9728 challenge, so a CA-protected Entra upstream was
unreachable through the gateway. The provider now reads the RFC 6749 error code and the claims
string off the rejection body (error_description is still never carried; it can leak IdP
internals), threads them through SubjectTokenRejected -> CredError.unauthorized, and the
challenge builder folds them into WWW-Authenticate: the machine error only when it is a plain
OAuth token (guards against header injection from a hostile body) and the claims base64-encoded
in a claims parameter, the convention MSAL-family clients decode. With neither field the header
is byte-identical to the static challenge. The multi-server aggregate still absorbs a
step-up 401 to an empty listing; only single-server routes surface it

* fix(mcp): use error=insufficient_claims for the Entra step-up challenge

Per Microsoft's claims-challenge format, a WWW-Authenticate carrying a claims challenge must set
error=insufficient_claims (the value MSAL-family clients key on to recognize the challenge and
replay the claims), not the raw token-endpoint code. The challenge now sets insufficient_claims
whenever a claims blob is present and keeps invalid_token otherwise, with a step-up-accurate
error_description in the claims case. The presence of claims now drives the error value, so the
raw oauth_error no longer needs threading from the provider through CredError to the edge; that
plumbing is removed (the provider still reads the error code for its gateway-fault classification).
Both the error value and the base64 claims are fixed-alphabet, so nothing from the IdP body reaches
the header unescaped. Cross-checked field-by-field against Microsoft Learn; a bogus jwt-bearer OBO
POST to the real login.microsoftonline.com/common endpoint confirmed Entra recognizes the grant and
returns the error shape the parser reads. Follow-up: also emit authorization_uri alongside
resource_metadata for strict non-MCP MSAL clients (RFC 9728 resource_metadata already serves MCP)

* fix(mcp): filter blank scopes on the config-load path so entra_obo fails closed

The DB-build path already runs YAML/DB scopes through _extract_scopes (which drops blanks), but the
config-load path read server_config["scopes"] raw. A YAML `scopes: [""]` therefore reached the
exchanger as a ("",) tuple: non-empty, so the entra_obo `not config.scopes` precondition skipped its
fail-closed misconfigured path and the form builder POSTed an empty scope to the IdP instead of
failing before any network call. Config-load now filters blanks the same way, so an all-blank list
normalizes to None and the precondition fails closed. Regression test loads a blank-scope entra_obo
server and asserts the exchange returns misconfigured without POSTing
2026-07-04 16:48:36 -07:00
..
agent_tests feat(a2a): support a2a-sdk 1.x proxy routing for 0.3 and 1.0 agents (#30950) 2026-06-29 09:32:39 +05:30
audio_tests fix(tests): stabilize image-edit VCR cassettes to stop live gpt-image-1 spend (#28110) 2026-05-18 09:15:39 -07:00
basic_proxy_startup_tests build: migrate packaging, CI, and Docker from Poetry to uv (#25007) 2026-04-09 11:46:23 -07:00
batches_tests test(e2e): add live batches suite across providers and routing scenarios (#30958) 2026-07-02 08:05:23 -07:00
benchmarks test(benchmarks): add CodSpeed benchmarks for inference, MCP and A2A hot paths (#31716) 2026-06-30 10:27:12 -07:00
code_coverage_tests fix(otel): hashable scope for _emit_once when guardrail_mode is list (#31262) 2026-06-25 11:59:35 -07:00
documentation_tests feat: add Rust OCR providers (#31272) 2026-06-25 15:12:30 -07:00
e2e feat(proxy): add key-level budget_fallbacks to reroute requests when a per-model budget is exceeded (#31783) 2026-07-03 12:20:12 -07:00
enterprise feat(prometheus): add api_provider label to token, latency, request and cache metrics (#32126) 2026-07-04 15:16:08 -07:00
guardrails_tests feat: litellm oss 110626 (#30202) 2026-06-11 22:30:26 -07:00
image_gen_tests test(bedrock): switch image gen live test off EOL Titan to Nova Canvas (#31937) 2026-07-01 22:57:29 -07:00
integration feat(sandbox): e2b code execution primitive (#30898) 2026-06-20 16:30:01 -07:00
litellm chore: litellm oss staging (#31185) 2026-06-26 09:17:44 -07:00
litellm-proxy-extras style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
litellm_core_utils Merge branch 'litellm_internal_staging' into litellm_staging_03_22_2026 2026-04-20 19:56:00 +05:30
litellm_utils_tests feat(fireworks_ai): sync chat completions endpoint with full API surface (#30885) 2026-06-20 19:49:07 -07:00
llm_responses_api_testing test(responses): bump deprecated gemini-3-pro-preview to gemini-3.1-pro-preview (#29433) 2026-06-01 09:54:30 -07:00
llm_translation feat(guardrails/headroom): add CCR (compress-cache-retrieve) via agentic loop (#31681) 2026-06-30 19:19:34 -07:00
load_tests style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
local_testing fix(bedrock): drop strict/additionalProperties from toolSpec for Claude Sonnet 4 (#31943) 2026-07-01 23:56:25 -07:00
logging_callback_tests fix(proxy): restore admin key/team callback_vars.turn_off_message_logging override (LIT-3587) (#31905) 2026-07-02 15:01:58 -07:00
mcp_tests feat(mcp): make token_exchange (OBO) production-ready - discovery threading + audit hardening + RFC 9728 challenge (#31622) 2026-07-03 17:12:25 -07:00
multi_instance_e2e_tests
ocr_tests fix(azure_ai): preserve content, tables, and keyValuePairs in doc-intelligence /v1/ocr (#32018) 2026-07-02 22:38:36 -07:00
old_proxy_tests/tests fix: cleanup tests 2026-03-30 16:24:35 -07:00
openai_endpoints_tests chore(ci): modernize model references in tests and configs (#27856) 2026-05-15 15:44:28 -07:00
otel_tests fix(cli): mint per-session agent credential on lite login (#31072) 2026-06-26 09:05:15 -07:00
pass_through_tests test(pass-through): de-flake vertex spend-log test by routing through the proxy (#31689) 2026-06-30 15:27:48 -07:00
pass_through_unit_tests test(proxy): set status_code on chunk_processor response mocks 2026-07-04 12:49:54 -07:00
proxy_admin_ui_tests Revert "chore: remove _experimental/out (#31546)" 2026-07-01 13:25:47 -07:00
proxy_behavior feat(proxy): add AES-256-GCM at-rest credential encryption with versioned format and re-encryption migration (#31215) 2026-06-29 20:14:22 +02:00
proxy_e2e_anthropic_messages_tests Extend the record/replay proxy to chat, embeddings, moderations, rerank, and Anthropic (#29847) 2026-06-06 14:33:42 -07:00
proxy_migration_tests test(proxy): stop running real-DB tests in GitHub Actions unit jobs (#29700) 2026-06-04 14:56:02 -07:00
proxy_security_tests test(proxy): stop running real-DB tests in GitHub Actions unit jobs (#29700) 2026-06-04 14:56:02 -07:00
proxy_unit_tests feat(proxy): add key-level budget_fallbacks to reroute requests when a per-model budget is exceeded (#31783) 2026-07-03 12:20:12 -07:00
router_unit_tests fix(vertex/files): stream OpenAI->Vertex batch JSONL uploads (#31036) 2026-06-24 13:19:57 -07:00
scim_tests
search_tests feat(tinyfish): make search provider permissive, attribute errors (#31997) 2026-07-03 10:17:11 -07:00
spend_tracking_tests chore(ci): modernize model references in tests and configs (#27856) 2026-05-15 15:44:28 -07:00
store_model_in_db_tests style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
test_litellm feat(mcp): add entra_obo profile to the token_exchange (OBO) arm (#31983) 2026-07-04 16:48:36 -07:00
unified_google_tests fix(google_genai): preserve complete SSE events in Vertex/Gemini image streaming (#30270) 2026-06-12 07:49:30 -07:00
vector_store_tests Revert "chore(tests): migrate Bedrock CI to AWS account 941277531214 (#28728)" (#29326) 2026-05-30 11:26:24 -07:00
windows_tests ci: reproduce default-Windows wheel install to guard MAX_PATH (#29597) 2026-06-03 11:28:08 -07:00
__init__.py
_fake_openai_endpoint_server.py test: point router/completion/triton tests at the local fake OpenAI endpoint (#30900) 2026-06-20 16:20:35 -07:00
_flush_vcr_cache.py tests(vcr): isolate cassette redis to CASSETTE_REDIS_URL 2026-05-01 12:32:59 -07:00
_live_test_helpers.py test(vcr): close out the remaining VCR live-call leaks (#29603) 2026-06-03 13:46:43 -07:00
_openai_record_replay_proxy.py Extend the record/replay proxy to chat, embeddings, moderations, rerank, and Anthropic (#29847) 2026-06-06 14:33:42 -07:00
_vcr_conftest_common.py test(vcr): close out the remaining VCR live-call leaks (#29603) 2026-06-03 13:46:43 -07:00
_vcr_redis_persister.py test(vcr): stop refreshing cassette TTL on read so cassettes lapse after 24h (#29784) 2026-06-05 10:22:41 -07:00
eval_swe_bench.py Prompt Compression - add it to the proxy (#25729) 2026-04-20 15:08:00 -07:00
fake_openai_endpoint.py test: point router/completion/triton tests at the local fake OpenAI endpoint (#30900) 2026-06-20 16:20:35 -07:00
gettysburg.wav
large_text.py
openai_batch_completions.jsonl
pyrightconfig.json test: add e2e tests for spend, budgets and llms (#30869) 2026-06-24 15:01:57 -07:00
README.MD
test_anthropic_compaction_usage.py chore(oss): litellm oss staging 150626 (#30463) 2026-06-16 12:06:41 -07:00
test_budget_management.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
test_callbacks_on_proxy.py test(callbacks): harden flaky proxy callback-leak detector (#28195) 2026-05-18 16:39:02 -07:00
test_config.py
test_debug_warning.py
test_default_encoding_non_root.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
test_end_users.py chore(ci): modernize model references in tests and configs (#27856) 2026-05-15 15:44:28 -07:00
test_entrypoint.py
test_fallbacks.py
test_gpt5_azure_temperature_support.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
test_health.py fix(tests): swap dall-e to gpt-image-1 after openai deprecation 2026-05-12 16:55:18 -07:00
test_keys.py test: harden remaining pass-through CI flakes (image-gen spend poll, ruby assistants timeout) (#30685) 2026-06-17 14:35:47 -07:00
test_litellm_proxy_responses_config.py chore(ci): modernize model references in tests and configs (#27856) 2026-05-15 15:44:28 -07:00
test_logging.conf
test_models.py test: replace test_add_and_delete_models integration test with mock 2026-03-30 21:30:57 -07:00
test_new_vector_store_endpoints.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
test_openai_endpoints.py Extend the record/replay proxy to chat, embeddings, moderations, rerank, and Anthropic (#29847) 2026-06-06 14:33:42 -07:00
test_organizations.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
test_otel_thread_leak.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
test_passthrough_endpoints.py
test_presidio_latency.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
test_proxy_server_non_root.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
test_ratelimit.py chore(ci): modernize model references in tests and configs (#27856) 2026-05-15 15:44:28 -07:00
test_resource_cleanup.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
test_service_logger_otel.py
test_spend_logs.py Litellm oss staging 04 21 2026 2 (#26569) 2026-05-20 21:25:19 -07:00
test_team.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
test_team_logging.py test: cleanup dead tests 2026-03-28 20:49:02 -07:00
test_team_members.py Litellm oss staging 04 21 2026 2 (#26569) 2026-05-20 21:25:19 -07:00
test_users.py Fix: tag budget reset must drop stale management-cache entry (#27568) 2026-05-10 00:18:55 +00:00

In total litellm runs 1000+ tests

[02/20/2025] Update:

To make it easier to contribute and map what behavior is tested,

we've started mapping the litellm directory in tests/test_litellm

This folder can only run mock tests.