scriptoriumadmin/litellm
1
0
Fork 0
mirror of https://github.com/BerriAI/litellm.git synced 2026-08-28 05:25:59 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 9
litellm/tests
History
Yassin Kortam 7263aa0028
fix(otel): keep an MCP tool call in one trace, anchored to its own request (#34537) 
Under otel_v2 a single MCP tool call surfaced in APM as two disconnected
traces joined only by a span link: the HTTP transport transaction
POST /{mcp_server_name}/mcp and the tools/call span carrying
error.type=MCPToolResultError. resolve_mcp_span_context parented the MCP
span to the W3C trace context the client propagates in params._meta
(SEP-414) and recorded the transport as a link, so with no traceparent
propagated (the common case today, including MCP Inspector) the span
started its own root trace.

Nest the MCP span under the transport span when nothing is propagated, so
the call stays in one trace; the propagated-context path is unchanged and
still parents to the remote context and links the transport per the OTel
GenAI MCP semconv.

The transport has to be resolved per message rather than read from the
request-root ContextVar. A stateful streamable-HTTP session runs every
message on the single task the session's initialize POST spawned, so that
ContextVar is frozen at initialize inside the handler: live capture on
staging showed the tools/call span linking the initialize POST rather than
the POST that carried it, and nesting on that anchor would hang every tool
call of a session off the first request's already-ended span. The gateway
now resolves the current request's span on the ASGI task and carries it to
the handler on the authenticated-user object, the same way per-request auth
already crosses that boundary.
2026-07-24 15:11:00 -07:00
..
agent_tests feat(a2a): support a2a-sdk 1.x proxy routing for 0.3 and 1.0 agents (#30950) 2026-06-29 09:32:39 +05:30
audio_tests fix(tests): stabilize image-edit VCR cassettes to stop live gpt-image-1 spend (#28110) 2026-05-18 09:15:39 -07:00
basic_proxy_startup_tests build: migrate packaging, CI, and Docker from Poetry to uv (#25007) 2026-04-09 11:46:23 -07:00
batches_tests Revert "chore(ci): sync litellm_internal_staging into daily OSS branch (#33337)" (#33339) 2026-07-14 19:32:25 -07:00
benchmarks test(benchmarks): run shared logging executor inline to make CodSpeed measurements deterministic (#32435) 2026-07-09 11:14:22 +03:00
code_coverage_tests test(e2e): drive a real Linear OAuth MCP through chat completions under both ingress headers 2026-07-22 23:29:10 -07:00
documentation_tests feat(messages): route native Anthropic /messages through Rust behind LITELLM_RUST env var (#33848) 2026-07-18 18:27:20 -07:00
e2e test(a2a): assert the property agent returns listings, not just non-empty text (#34515) 2026-07-24 11:03:28 -07:00
enterprise fix(guardrails/bedrock): honor disable_exception_on_block by raising ModifyResponseException (#32289) 2026-07-09 13:18:51 -07:00
guardrails_tests feat: add deepkeep as custom guardrail (#33844) 2026-07-20 19:27:40 -07:00
image_gen_tests test(bedrock): switch image gen live test off EOL Titan to Nova Canvas (#31937) 2026-07-01 22:57:29 -07:00
integration feat(sandbox): e2b code execution primitive (#30898) 2026-06-20 16:30:01 -07:00
litellm fix(anthropic): strip all remaining output_format schema keywords rejected by Anthropic 2026-07-22 16:16:04 -07:00
litellm-proxy-extras style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
litellm_core_utils Merge branch 'litellm_internal_staging' into litellm_staging_03_22_2026 2026-04-20 19:56:00 +05:30
litellm_utils_tests feat(budgets): add configurable budget_reset_time of day (#31007) 2026-07-21 13:35:01 -07:00
llm_responses_api_testing Revert "chore(ci): sync litellm_internal_staging into daily OSS branch (#33337)" (#33339) 2026-07-14 19:32:25 -07:00
llm_translation Merge pull request #34520 from BerriAI/litellm_/replace-gpt5-codex-test-27520b 2026-07-24 12:03:24 -07:00
load_tests style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
local_testing fix(anthropic): self-heal on missing thinking-signature errors from Bedrock/Vertex (#33719) 2026-07-17 18:18:38 +00:00
logging_callback_tests test(logging): pin compression_savings in the gcs pubsub spend log fixture (#34204) 2026-07-22 00:56:20 +00:00
mcp_tests Merge pull request #33631 from BerriAI/litellm_lit4517_messages_mcp_gateway 2026-07-20 16:22:03 -07:00
multi_instance_e2e_tests
ocr_tests test(ocr): use mistral-document-ai-2512 in azure_ai OCR tests 2026-07-15 18:13:22 -07:00
old_proxy_tests/tests fix: cleanup tests 2026-03-30 16:24:35 -07:00
openai_endpoints_tests chore(ci): modernize model references in tests and configs (#27856) 2026-05-15 15:44:28 -07:00
otel_tests fix(cli): mint per-session agent credential on lite login (#31072) 2026-06-26 09:05:15 -07:00
pass_through_tests test(pass-through): de-flake vertex spend-log test by routing through the proxy (#31689) 2026-06-30 15:27:48 -07:00
pass_through_unit_tests fix(streaming): stamp completion_start_time on first chunk for /v1/messages and /v1/responses (#32284) 2026-07-06 19:30:31 -07:00
proxy_admin_ui_tests Revert "chore: remove _experimental/out (#31546)" 2026-07-01 13:25:47 -07:00
proxy_behavior fix(proxy): resolve team org from team_id so org admins can update team budgets 2026-07-08 19:10:35 -07:00
proxy_e2e_anthropic_messages_tests Extend the record/replay proxy to chat, embeddings, moderations, rerank, and Anthropic (#29847) 2026-06-06 14:33:42 -07:00
proxy_migration_tests fix(docker): bake non_root prisma engines at /opt/prisma so migrations run offline for any uid (#34325) 2026-07-22 23:03:39 -07:00
proxy_security_tests test(proxy): stop running real-DB tests in GitHub Actions unit jobs (#29700) 2026-06-04 14:56:02 -07:00
proxy_unit_tests feat(guardrails): add run_in_parallel opt-in for concurrent pre_call and post_call guardrails (#33770) 2026-07-24 13:25:58 -07:00
router_unit_tests fix(router): tag-aware pre-routing strategy selection for shared model_name (#33691) 2026-07-17 09:26:07 -07:00
scim_tests
search_tests feat(tinyfish): make search provider permissive, attribute errors (#31997) 2026-07-03 10:17:11 -07:00
spend_tracking_tests chore(ci): modernize model references in tests and configs (#27856) 2026-05-15 15:44:28 -07:00
store_model_in_db_tests test: de-flake langfuse callbacks-in-db e2e test 2026-07-04 19:14:53 -07:00
test_litellm fix(otel): keep an MCP tool call in one trace, anchored to its own request (#34537) 2026-07-24 15:11:00 -07:00
unified_google_tests fix(google_genai): preserve complete SSE events in Vertex/Gemini image streaming (#30270) 2026-06-12 07:49:30 -07:00
vector_store_tests Revert "chore(tests): migrate Bedrock CI to AWS account 941277531214 (#28728)" (#29326) 2026-05-30 11:26:24 -07:00
windows_tests ci: reproduce default-Windows wheel install to guard MAX_PATH (#29597) 2026-06-03 11:28:08 -07:00
__init__.py
_fake_openai_endpoint_server.py test: point router/completion/triton tests at the local fake OpenAI endpoint (#30900) 2026-06-20 16:20:35 -07:00
_flush_vcr_cache.py tests(vcr): isolate cassette redis to CASSETTE_REDIS_URL 2026-05-01 12:32:59 -07:00
_live_test_helpers.py test(vcr): close out the remaining VCR live-call leaks (#29603) 2026-06-03 13:46:43 -07:00
_openai_record_replay_proxy.py Extend the record/replay proxy to chat, embeddings, moderations, rerank, and Anthropic (#29847) 2026-06-06 14:33:42 -07:00
_vcr_conftest_common.py test(vcr): close out the remaining VCR live-call leaks (#29603) 2026-06-03 13:46:43 -07:00
_vcr_redis_persister.py test(vcr): stop refreshing cassette TTL on read so cassettes lapse after 24h (#29784) 2026-06-05 10:22:41 -07:00
_ws_vcr.py test(realtime): record and replay websocket traffic in redis vcr cassettes (#32390) 2026-07-08 00:19:06 -07:00
eval_swe_bench.py Prompt Compression - add it to the proxy (#25729) 2026-04-20 15:08:00 -07:00
fake_openai_endpoint.py test: point router/completion/triton tests at the local fake OpenAI endpoint (#30900) 2026-06-20 16:20:35 -07:00
gettysburg.wav
large_text.py
openai_batch_completions.jsonl
pyrightconfig.json test: add e2e tests for spend, budgets and llms (#30869) 2026-06-24 15:01:57 -07:00
README.MD
test_anthropic_compaction_usage.py chore(oss): litellm oss staging 150626 (#30463) 2026-06-16 12:06:41 -07:00
test_budget_management.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
test_callbacks_on_proxy.py test(callbacks): harden flaky proxy callback-leak detector (#28195) 2026-05-18 16:39:02 -07:00
test_config.py
test_debug_warning.py
test_default_encoding_non_root.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
test_end_users.py chore(ci): modernize model references in tests and configs (#27856) 2026-05-15 15:44:28 -07:00
test_entrypoint.py
test_fallbacks.py
test_gpt5_azure_temperature_support.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
test_health.py fix(tests): swap dall-e to gpt-image-1 after openai deprecation 2026-05-12 16:55:18 -07:00
test_keys.py test: harden remaining pass-through CI flakes (image-gen spend poll, ruby assistants timeout) (#30685) 2026-06-17 14:35:47 -07:00
test_litellm_proxy_responses_config.py chore(ci): modernize model references in tests and configs (#27856) 2026-05-15 15:44:28 -07:00
test_logging.conf
test_models.py test: replace test_add_and_delete_models integration test with mock 2026-03-30 21:30:57 -07:00
test_new_vector_store_endpoints.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
test_openai_endpoints.py Extend the record/replay proxy to chat, embeddings, moderations, rerank, and Anthropic (#29847) 2026-06-06 14:33:42 -07:00
test_organizations.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
test_otel_thread_leak.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
test_passthrough_endpoints.py
test_presidio_latency.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
test_proxy_server_non_root.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
test_ratelimit.py chore(ci): modernize model references in tests and configs (#27856) 2026-05-15 15:44:28 -07:00
test_resource_cleanup.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
test_service_logger_otel.py fix(langfuse): send v4 ingestion header for otel callback (#33907) 2026-07-18 20:36:51 -07:00
test_spend_logs.py Litellm oss staging 04 21 2026 2 (#26569) 2026-05-20 21:25:19 -07:00
test_team.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
test_team_logging.py test: cleanup dead tests 2026-03-28 20:49:02 -07:00
test_team_members.py Litellm oss staging 04 21 2026 2 (#26569) 2026-05-20 21:25:19 -07:00
test_users.py Fix: tag budget reset must drop stale management-cache entry (#27568) 2026-05-10 00:18:55 +00:00

README.MD

In total litellm runs 1000+ tests

[02/20/2025] Update:

To make it easier to contribute and map what behavior is tested,

we've started mapping the litellm directory in tests/test_litellm

This folder can only run mock tests.