scriptoriumadmin/litellm
1
0
Fork 0
mirror of https://github.com/BerriAI/litellm.git synced 2026-09-24 00:52:24 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 91
litellm/tests/e2e
History
devin-ai-integration[bot] 30d8b12512
test(e2e): add conversational matrix across chat, messages and responses (#42359) 
* test(e2e): add conversational matrix across chat, messages and responses

Parameterizes one behavioral contract (reply, stream, cost log, tool call,
tool round trip) across /v1/chat/completions, /v1/messages and /v1/responses,
OpenAI and Anthropic models, and env-ref vs stored-credential auth, with
record/replay fixtures.

Adds general_settings.disable_model_info_refresh so the proxy fronting a
replay fixture does not poll every OpenAI-compatible deployment's /v1/models
in the background, which otherwise leaves unconsumed interactions in the
recorded bundle.

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(e2e): force the weather tool on the first turn and rename Provider to Deployment

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

---------

Co-authored-by: mateo <mateo@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-21 22:32:56 -07:00
..
a2a test(e2e): settle control-plane writes across every replica, not just one 2026-08-07 19:36:30 -07:00
access_control Merge remote-tracking branch 'github/main' into litellm_rust_bridge_declarative_route_catalog 2026-09-17 11:08:08 -07:00
batches fix(bedrock): sign batch S3 requests with s3_access_key_id and s3_secret_access_key (#42342) 2026-09-21 19:01:18 -07:00
claude_code chore: consolidate CLAUDE.md into AGENTS.md 2026-09-19 02:30:35 +00:00
coverage_registry test(e2e): add conversational matrix across chat, messages and responses (#42359) 2026-09-21 22:32:56 -07:00
gateway test(e2e): add conversational matrix across chat, messages and responses (#42359) 2026-09-21 22:32:56 -07:00
guardrails fix(presidio): mask PII in streaming /v1/messages output (#42351) 2026-09-21 21:42:52 -07:00
llm_translation test(e2e): add conversational matrix across chat, messages and responses (#42359) 2026-09-21 22:32:56 -07:00
load chore: merge litellm_internal_staging into litellm_e2e_reliability_module_cells 2026-09-12 13:05:02 -07:00
logging fix(otel v2): map completions, images, speech, transcription and moderation output onto the Langfuse generation output (#42394) 2026-09-21 21:57:44 -07:00
management test: expect 422 for budget refusals in unification, e2e and integration suites 2026-09-20 06:33:57 +00:00
mcp test(e2e): restore LIT-3467 implementation for rework 2026-09-19 16:21:53 -07:00
migrations test(migrations): close the gaps the upgrade assertions left open 2026-09-21 13:15:46 -07:00
other test(e2e): harden JWT fixtures and cover management lifecycles 2026-09-11 16:36:12 -07:00
quota_management test: expect 422 for budget refusals in unification, e2e and integration suites 2026-09-20 06:33:57 +00:00
router test(e2e): retry the hang-up when the model answers inside the window 2026-09-21 14:48:04 -07:00
ui test(e2e-ui): check the MCP Tools tab against the upstream's own tools/list (#42397) 2026-09-21 20:35:30 -07:00
AGENTS.md Merge branch 'main' into claude/e2e-tests-custom-endpoints-qxoi1o 2026-09-21 11:34:21 -07:00
conftest.py fix(otel v2): map completions, images, speech, transcription and moderation output onto the Langfuse generation output (#42394) 2026-09-21 21:57:44 -07:00
CONTRIBUTING.md fix(otel v2): map completions, images, speech, transcription and moderation output onto the Langfuse generation output (#42394) 2026-09-21 21:57:44 -07:00
e2e_config.py fix(otel v2): map completions, images, speech, transcription and moderation output onto the Langfuse generation output (#42394) 2026-09-21 21:57:44 -07:00
e2e_db.py test(e2e): guard destructive spend-log truncate behind an explicit opt-in (#33751) 2026-07-20 08:47:39 -07:00
e2e_http.py test(e2e): settle for the replica propagation window and trim the disconnect cell's prose 2026-09-21 13:30:25 -07:00
fixture_bundle.py test: add strict stateless provider replay identity 2026-09-14 16:55:43 -07:00
fixture_canonical.py feat(e2e): key the provider cache per test and mount Bedrock behind it 2026-09-16 02:15:46 -07:00
fixture_mode.py fix(e2e): own a shared fixture's deployment by the fixture's node, not the first test 2026-09-16 17:35:05 -07:00
fixture_profile.py test: preserve strict replay numeric spelling 2026-09-14 17:11:56 -07:00
idp.py test(e2e): restore LIT-3467 implementation for rework 2026-09-19 16:21:53 -07:00
idp_realm.json test(e2e): harden JWT fixtures and cover management lifecycles 2026-09-11 16:36:12 -07:00
junit_properties.py test: bind management E2E callers and isolate JWT actors 2026-09-12 13:29:04 -07:00
lifecycle.py fix(e2e): clean up batch files reliably and expire Azure inputs 2026-09-07 12:15:06 -07:00
models.py fix(otel v2): map completions, images, speech, transcription and moderation output onto the Langfuse generation output (#42394) 2026-09-21 21:57:44 -07:00
otel_client.py test(e2e): harden the suite against response-cache cross-talk, slow providers and single upstream blips (#37957) 2026-08-22 14:47:03 -07:00
PROVIDER_CACHE.md fix(e2e): own a shared fixture's deployment by the fixture's node, not the first test 2026-09-16 17:35:05 -07:00
provider_cache.py fix(e2e): bind provider-cache recordings to the deployment's test, not the serving process 2026-09-16 17:08:17 -07:00
provider_cache_redis.py chore(e2e): report the key components behind a mount that never converges 2026-09-16 15:01:08 -07:00
provider_cache_routing.py revert(e2e): unmount Gemini, its api_base means two things 2026-09-16 08:37:25 -07:00
provider_edge.py fix(responses): forward safety_identifier through the chat completion bridge 2026-09-21 04:09:59 +00:00
provider_edge_bedrock.py feat(e2e): cache the responses and embeddings endpoints behind the edge 2026-09-16 02:34:09 -07:00
proxy_client.py Merge pull request #42295 from BerriAI/litellm_fix_azure_cancellederror_cooldown 2026-09-21 17:24:22 -07:00
pytest.ini fix(otel v2): map completions, images, speech, transcription and moderation output onto the Langfuse generation output (#42394) 2026-09-21 21:57:44 -07:00
test_e2e_http.py test: bind management E2E callers and isolate JWT actors 2026-09-12 13:29:04 -07:00
test_fixture_bundle.py feat(e2e): record and replay streamed provider responses chunk-for-chunk 2026-08-24 12:51:44 -07:00
test_fixture_canonical.py test(e2e): pin query params and multipart form fields as replay match-key identity 2026-08-20 15:59:34 -04:00
test_fixture_mode.py feat(e2e): move record/replay to the provider edge (LIT-5745) 2026-08-19 18:39:15 -07:00
test_idp.py test: enforce isolated actors and stop OIDC process groups 2026-09-12 13:49:49 -07:00
test_junit_properties.py test(e2e): read JUnit properties off the real collected pytest Item 2026-09-01 19:12:07 -07:00
test_provider_edge.py fix(e2e): bind provider-cache recordings to the deployment's test, not the serving process 2026-09-16 17:08:17 -07:00
test_proxy_client.py fix(e2e): own a shared fixture's deployment by the fixture's node, not the first test 2026-09-16 17:35:05 -07:00
transport.py test(e2e): client disconnect must not bench the Azure deployment it cancelled 2026-09-21 18:39:37 +00:00