scriptoriumadmin/litellm
1
0
Fork 0
mirror of https://github.com/BerriAI/litellm.git synced 2026-09-06 08:16:43 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 78
litellm/tests
History
Tin Chi Lo 2ccdb0896d feat(mcp): send RFC 8707 resource indicators on upstream OAuth legs 
The gateway acts as an MCP client toward upstream MCP servers, and the MCP
authorization spec requires an MCP client to send the RFC 8707 resource
parameter on both the authorization request and every token request. The
gateway sent it on none of its upstream OAuth legs, so an authorization server
that requires resource indicators rejected the exchange with invalid_target
with no way to configure around it.

Authorization servers disagree irreconcilably and nothing advertises which
camp they are in, so this is a per-server opt-in rather than a default: most
providers ignore the parameter, some hard-reject it and carry audience in
scopes instead, and strict or MCP-native ones refuse to mint a correctly
scoped token without it. The new upstream_resource setting is unset by
default, which keeps today's requests byte-identical.

Both outbound OAuth stacks resolve the value from the server exactly once and
carry it structurally rather than attaching it per call site. In v1 every
plain-OAuth2 token leg builds its body through one helper that resolves the
resource in the same call as the mandatory client authentication; in v2 the
adapter, the single place an MCPServer becomes an outbound config, resolves it
onto the client_credentials config that the HTTP/SSE M2M path uses, and it
joins the config's mint identity so retargeting a live server refreshes the
token rather than serving the previous audience's. A leg cannot authenticate
without also naming the resource its sibling legs named, which is what an
attach-per-call-site approach kept getting wrong.

The setting is non-secret admin config sharing a blob with real secrets, and
the backend classifies which key is which rather than nulling the blob
wholesale or gating on its truthiness: redaction returns admin config to an
admin, session inheritance ignores it when deciding whether a real credential
was supplied and carries it onto the derived server, and the edit form renders
the same shared OAuth component as create so the field exists on both, an
emptied field submitting an explicit null that the credential merge drops.
2026-07-24 15:01:38 -07:00
..
agent_tests feat(a2a): support a2a-sdk 1.x proxy routing for 0.3 and 1.0 agents (#30950) 2026-06-29 09:32:39 +05:30
audio_tests fix(tests): stabilize image-edit VCR cassettes to stop live gpt-image-1 spend (#28110) 2026-05-18 09:15:39 -07:00
basic_proxy_startup_tests build: migrate packaging, CI, and Docker from Poetry to uv (#25007) 2026-04-09 11:46:23 -07:00
batches_tests Revert "chore(ci): sync litellm_internal_staging into daily OSS branch (#33337)" (#33339) 2026-07-14 19:32:25 -07:00
benchmarks test(benchmarks): run shared logging executor inline to make CodSpeed measurements deterministic (#32435) 2026-07-09 11:14:22 +03:00
code_coverage_tests test(e2e): drive a real Linear OAuth MCP through chat completions under both ingress headers 2026-07-22 23:29:10 -07:00
documentation_tests feat(messages): route native Anthropic /messages through Rust behind LITELLM_RUST env var (#33848) 2026-07-18 18:27:20 -07:00
e2e refactor(e2e): drop require_env, read os.environ where a cred is used (#34413) 2026-07-23 19:14:22 +00:00
enterprise fix(guardrails/bedrock): honor disable_exception_on_block by raising ModifyResponseException (#32289) 2026-07-09 13:18:51 -07:00
guardrails_tests feat: add deepkeep as custom guardrail (#33844) 2026-07-20 19:27:40 -07:00
image_gen_tests test(bedrock): switch image gen live test off EOL Titan to Nova Canvas (#31937) 2026-07-01 22:57:29 -07:00
integration feat(sandbox): e2b code execution primitive (#30898) 2026-06-20 16:30:01 -07:00
litellm fix(anthropic): strip all remaining output_format schema keywords rejected by Anthropic 2026-07-22 16:16:04 -07:00
litellm-proxy-extras style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
litellm_core_utils Merge branch 'litellm_internal_staging' into litellm_staging_03_22_2026 2026-04-20 19:56:00 +05:30
litellm_utils_tests feat(budgets): add configurable budget_reset_time of day (#31007) 2026-07-21 13:35:01 -07:00
llm_responses_api_testing Revert "chore(ci): sync litellm_internal_staging into daily OSS branch (#33337)" (#33339) 2026-07-14 19:32:25 -07:00
llm_translation test(reasoning_effort_grid): enable azure fable-5 and opus-4-8 grid cells 2026-07-15 17:33:46 -07:00
load_tests style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
local_testing fix(anthropic): self-heal on missing thinking-signature errors from Bedrock/Vertex (#33719) 2026-07-17 18:18:38 +00:00
logging_callback_tests test(logging): pin compression_savings in the gcs pubsub spend log fixture (#34204) 2026-07-22 00:56:20 +00:00
mcp_tests Merge pull request #33631 from BerriAI/litellm_lit4517_messages_mcp_gateway 2026-07-20 16:22:03 -07:00
multi_instance_e2e_tests
ocr_tests test(ocr): use mistral-document-ai-2512 in azure_ai OCR tests 2026-07-15 18:13:22 -07:00
old_proxy_tests/tests fix: cleanup tests 2026-03-30 16:24:35 -07:00
openai_endpoints_tests chore(ci): modernize model references in tests and configs (#27856) 2026-05-15 15:44:28 -07:00
otel_tests fix(cli): mint per-session agent credential on lite login (#31072) 2026-06-26 09:05:15 -07:00
pass_through_tests test(pass-through): de-flake vertex spend-log test by routing through the proxy (#31689) 2026-06-30 15:27:48 -07:00
pass_through_unit_tests fix(streaming): stamp completion_start_time on first chunk for /v1/messages and /v1/responses (#32284) 2026-07-06 19:30:31 -07:00
proxy_admin_ui_tests Revert "chore: remove _experimental/out (#31546)" 2026-07-01 13:25:47 -07:00
proxy_behavior fix(proxy): resolve team org from team_id so org admins can update team budgets 2026-07-08 19:10:35 -07:00
proxy_e2e_anthropic_messages_tests Extend the record/replay proxy to chat, embeddings, moderations, rerank, and Anthropic (#29847) 2026-06-06 14:33:42 -07:00
proxy_migration_tests fix(docker): bake non_root prisma engines at /opt/prisma so migrations run offline for any uid (#34325) 2026-07-22 23:03:39 -07:00
proxy_security_tests test(proxy): stop running real-DB tests in GitHub Actions unit jobs (#29700) 2026-06-04 14:56:02 -07:00
proxy_unit_tests fix(team): make team member add atomic to prevent concurrent-add member loss (#34185) 2026-07-22 21:47:22 +00:00
router_unit_tests fix(router): tag-aware pre-routing strategy selection for shared model_name (#33691) 2026-07-17 09:26:07 -07:00
scim_tests
search_tests feat(tinyfish): make search provider permissive, attribute errors (#31997) 2026-07-03 10:17:11 -07:00
spend_tracking_tests chore(ci): modernize model references in tests and configs (#27856) 2026-05-15 15:44:28 -07:00
store_model_in_db_tests test: de-flake langfuse callbacks-in-db e2e test 2026-07-04 19:14:53 -07:00
test_litellm feat(mcp): send RFC 8707 resource indicators on upstream OAuth legs 2026-07-24 15:01:38 -07:00
unified_google_tests fix(google_genai): preserve complete SSE events in Vertex/Gemini image streaming (#30270) 2026-06-12 07:49:30 -07:00
vector_store_tests Revert "chore(tests): migrate Bedrock CI to AWS account 941277531214 (#28728)" (#29326) 2026-05-30 11:26:24 -07:00
windows_tests ci: reproduce default-Windows wheel install to guard MAX_PATH (#29597) 2026-06-03 11:28:08 -07:00
__init__.py
_fake_openai_endpoint_server.py test: point router/completion/triton tests at the local fake OpenAI endpoint (#30900) 2026-06-20 16:20:35 -07:00
_flush_vcr_cache.py tests(vcr): isolate cassette redis to CASSETTE_REDIS_URL 2026-05-01 12:32:59 -07:00
_live_test_helpers.py test(vcr): close out the remaining VCR live-call leaks (#29603) 2026-06-03 13:46:43 -07:00
_openai_record_replay_proxy.py Extend the record/replay proxy to chat, embeddings, moderations, rerank, and Anthropic (#29847) 2026-06-06 14:33:42 -07:00
_vcr_conftest_common.py test(vcr): close out the remaining VCR live-call leaks (#29603) 2026-06-03 13:46:43 -07:00
_vcr_redis_persister.py test(vcr): stop refreshing cassette TTL on read so cassettes lapse after 24h (#29784) 2026-06-05 10:22:41 -07:00
_ws_vcr.py test(realtime): record and replay websocket traffic in redis vcr cassettes (#32390) 2026-07-08 00:19:06 -07:00
eval_swe_bench.py Prompt Compression - add it to the proxy (#25729) 2026-04-20 15:08:00 -07:00
fake_openai_endpoint.py test: point router/completion/triton tests at the local fake OpenAI endpoint (#30900) 2026-06-20 16:20:35 -07:00
gettysburg.wav
large_text.py
openai_batch_completions.jsonl
pyrightconfig.json test: add e2e tests for spend, budgets and llms (#30869) 2026-06-24 15:01:57 -07:00
README.MD
test_anthropic_compaction_usage.py chore(oss): litellm oss staging 150626 (#30463) 2026-06-16 12:06:41 -07:00
test_budget_management.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
test_callbacks_on_proxy.py test(callbacks): harden flaky proxy callback-leak detector (#28195) 2026-05-18 16:39:02 -07:00
test_config.py
test_debug_warning.py
test_default_encoding_non_root.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
test_end_users.py chore(ci): modernize model references in tests and configs (#27856) 2026-05-15 15:44:28 -07:00
test_entrypoint.py
test_fallbacks.py
test_gpt5_azure_temperature_support.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
test_health.py fix(tests): swap dall-e to gpt-image-1 after openai deprecation 2026-05-12 16:55:18 -07:00
test_keys.py test: harden remaining pass-through CI flakes (image-gen spend poll, ruby assistants timeout) (#30685) 2026-06-17 14:35:47 -07:00
test_litellm_proxy_responses_config.py chore(ci): modernize model references in tests and configs (#27856) 2026-05-15 15:44:28 -07:00
test_logging.conf
test_models.py test: replace test_add_and_delete_models integration test with mock 2026-03-30 21:30:57 -07:00
test_new_vector_store_endpoints.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
test_openai_endpoints.py Extend the record/replay proxy to chat, embeddings, moderations, rerank, and Anthropic (#29847) 2026-06-06 14:33:42 -07:00
test_organizations.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
test_otel_thread_leak.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
test_passthrough_endpoints.py
test_presidio_latency.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
test_proxy_server_non_root.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
test_ratelimit.py chore(ci): modernize model references in tests and configs (#27856) 2026-05-15 15:44:28 -07:00
test_resource_cleanup.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
test_service_logger_otel.py fix(langfuse): send v4 ingestion header for otel callback (#33907) 2026-07-18 20:36:51 -07:00
test_spend_logs.py Litellm oss staging 04 21 2026 2 (#26569) 2026-05-20 21:25:19 -07:00
test_team.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
test_team_logging.py test: cleanup dead tests 2026-03-28 20:49:02 -07:00
test_team_members.py Litellm oss staging 04 21 2026 2 (#26569) 2026-05-20 21:25:19 -07:00
test_users.py Fix: tag budget reset must drop stale management-cache entry (#27568) 2026-05-10 00:18:55 +00:00

README.MD

In total litellm runs 1000+ tests

[02/20/2025] Update:

To make it easier to contribute and map what behavior is tested,

we've started mapping the litellm directory in tests/test_litellm

This folder can only run mock tests.