litellm/tests/test_litellm/proxy
devin-ai-integration[bot] 30d8b12512
test(e2e): add conversational matrix across chat, messages and responses (#42359)
* test(e2e): add conversational matrix across chat, messages and responses

Parameterizes one behavioral contract (reply, stream, cost log, tool call,
tool round trip) across /v1/chat/completions, /v1/messages and /v1/responses,
OpenAI and Anthropic models, and env-ref vs stored-credential auth, with
record/replay fixtures.

Adds general_settings.disable_model_info_refresh so the proxy fronting a
replay fixture does not poll every OpenAI-compatible deployment's /v1/models
in the background, which otherwise leaves unconsumed interactions in the
recorded bundle.

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(e2e): force the weather tool on the first turn and rename Provider to Deployment

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

---------

Co-authored-by: mateo <mateo@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-21 22:32:56 -07:00
..
_experimental/mcp_server fix(proxy): surface a database outage from the user read as 503 no_db_connection (#42399) 2026-09-21 20:52:41 -07:00
a2a
agent_endpoints fix(proxy): scope agent permissions to invoking caller 2026-09-21 22:32:32 +00:00
analytics_endpoints
anthropic_endpoints feat(proxy): opt-in litellm_call_id in JSON error bodies (#42391) 2026-09-21 19:17:18 -07:00
auth fix(proxy): surface a database outage from the user read as 503 no_db_connection (#42399) 2026-09-21 20:52:41 -07:00
batches_endpoints chore: merge main into fix/batch-retrieve-model-group 2026-09-19 16:25:39 -07:00
client fix(cli): serialize footer installs and tolerate unknown versions 2026-09-21 16:48:08 -07:00
common_utils feat(proxy): opt-in litellm_call_id in JSON error bodies (#42391) 2026-09-21 19:17:18 -07:00
config_resolvers test: drop two inert type: ignore comments 2026-09-19 14:22:19 -07:00
container_endpoints
credential_endpoints fix(credentials): answer 409 on a name collision, let PATCH resolve values from model_id 2026-09-15 10:41:43 -07:00
db Merge pull request #42026 from BerriAI/litellm_user_jwt_savings 2026-09-21 16:41:16 -07:00
discovery_endpoints
enterprise_billing
experimental/mcp_server
fine_tuning_endpoints
google_endpoints
guardrails fix(presidio): mask PII in streaming /v1/messages output (#42351) 2026-09-21 21:42:52 -07:00
health_endpoints
hooks Merge pull request #42062 from BerriAI/litellm_pr38499_batch_retrieve_model_group 2026-09-19 18:23:06 -07:00
image_endpoints Merge remote-tracking branch 'origin/main' into litellm_fix_image_edits_bracketed_alias 2026-09-17 14:49:54 -07:00
list_api
logging_endpoints
management_endpoints fix(proxy): drop cost-map metadata echoed back on model save (#41944) 2026-09-21 19:33:16 -07:00
management_helpers Merge remote-tracking branch 'origin/main' into litellm_jev_autorouter_launch_1789767495 2026-09-19 17:35:42 +00:00
memory
middleware fix(proxy): release unclaimed budget reservations at request end (#42304) 2026-09-21 19:51:12 -07:00
ocr_endpoints
openai_files_endpoint fix(files): keep an explicit target_storage on its old path and refuse litellm_db as a caller choice 2026-09-19 13:20:02 -07:00
pass_through_endpoints feat(proxy): add TinyFish Agent API passthrough with per-step billing (#41099) 2026-09-21 21:21:43 -07:00
policy_engine fix(policy_engine): resolve policies once and apply fallback semantics in get_matching_policies 2026-09-21 17:36:03 +00:00
prompts
proxy_server test(e2e): add conversational matrix across chat, messages and responses (#42359) 2026-09-21 22:32:56 -07:00
public_endpoints test: keep the Qwen brand missing-key tests off the network and split the ChatGPT fields test back out 2026-09-21 12:54:23 -07:00
rag_endpoints test(rag): drop the docstrings from the registered-store ingest tests 2026-09-19 04:10:49 -07:00
realtime_endpoints refactor(proxy): replace configurable model access denied message with a fixed clean client message 2026-09-16 01:44:36 +00:00
rerank_endpoints fix(proxy): carry litellm_call_id through endpoint specific error logs and failure responses 2026-09-16 02:29:26 +00:00
response_api_endpoints test(proxy): expect 422 for per-model budget rejections on cursor route 2026-09-20 06:10:16 +00:00
shutdown refactor(proxy): inject scheduled job shutdown timeouts 2026-09-21 19:37:28 +00:00
spend_tracking feat(tokenizer): preserve Python defaults with opt-in Rust dispatch (#42174) 2026-09-22 04:41:11 +00:00
test_configs
types_utils
ui_crud_endpoints Merge remote-tracking branch 'origin/main' into litellm_config_read_source 2026-09-21 15:54:52 -07:00
utils Merge pull request #41634 from BerriAI/litellm_agent_access_groups 2026-09-21 19:13:46 -05:00
vector_store_endpoints fix(vector_stores): run the full model grant check on caller-supplied model hints 2026-09-19 00:54:41 -07:00
vector_store_files_endpoints
video_endpoints
__init__.py
conftest.py test(proxy): isolate the agent read-through singleton between unknown-agent tests 2026-09-22 01:34:31 +00:00
test__types.py feat(auth): breached password detection, self-service change-password and forced password reset 2026-09-21 18:48:35 +00:00
test_aiohttp_cleanup_closed.py
test_aiohttp_session_recovery.py
test_api_key_masking_in_errors.py
test_audio_speech_prometheus_hooks.py
test_batch_expiry.py
test_batch_metadata_none_fix.py
test_batch_retrieve_bedrock.py
test_batch_x_litellm_model_encoding.py
test_blocked_response_usage.py
test_budget_reservation.py feat(tokenizer): preserve Python defaults with opt-in Rust dispatch (#42174) 2026-09-22 04:41:11 +00:00
test_caching_routes.py
test_chat_completion_metadata.py
test_claude_code_marketplace.py
test_collector.py
test_common_request_processing.py feat(errors): prefilled GitHub issue link on unmapped internal errors (#42065) 2026-09-21 22:30:41 -07:00
test_component_allowlists.py
test_conftest.py
test_cors_config.py
test_custom_proxy.py
test_dynamic_mcp_route.py
test_empty_model_list.py
test_enforce_user_param.py
test_fallback_management_endpoints.py
test_fastapi_offline_routes.py
test_filter_models_by_team_access_group.py
test_health_check_functions.py
test_health_check_max_tokens.py feat(auto-router): integrate JEV context and usage accounting 2026-09-18 21:38:15 +00:00
test_init_litellm_callbacks.py
test_langfuse_passthrough_security.py
test_lazy_openapi_snapshot.py
test_litellm_pre_call_utils.py fix(proxy): forward a client's anthropic-beta and anthropic-version headers to bedrock_mantle 2026-09-19 20:11:14 -07:00
test_max_budget_env_var.py
test_mcp_asgi_response.py
test_model_based_routing_files_batches.py
test_model_deprecations_endpoint.py
test_model_dump_with_preserved_fields.py
test_model_id_header_propagation.py
test_model_info_default_limits.py
test_model_level_guardrails.py
test_model_list_healthy_only.py
test_modify_response_streaming_passthrough.py
test_openai_ws_passthrough_routes.py
test_openapi_schema_validation.py
test_plugin_routes.py refactor(proxy): resolve config and DB settings precedence in one SettingsStore 2026-09-17 23:36:27 -07:00
test_pointfive_dashboard_config.py
test_pointfive_ui_callback.py
test_pricing_field_strip.py
test_prisma_engine_watchdog.py
test_prisma_migration.py
test_prometheus_cleanup.py
test_prometheus_metrics_server.py
test_provider_url_destination_guard.py
test_proxy_cli.py fix(proxy): only warn about the deprecated flag when it came from the CLI 2026-09-20 01:22:31 -07:00
test_proxy_logging_hook_detection.py fix(presidio): mask PII in streaming /v1/messages output (#42351) 2026-09-21 21:42:52 -07:00
test_proxy_server.py feat(tokenizer): preserve Python defaults with opt-in Rust dispatch (#42174) 2026-09-22 04:41:11 +00:00
test_proxy_types.py fix(router): honor team and key provider weights 2026-09-14 23:31:52 -07:00
test_proxy_utils.py feat(errors): prefilled GitHub issue link on unmapped internal errors (#42065) 2026-09-21 22:30:41 -07:00
test_pyroscope.py
test_read_model_list.py
test_redis_auth_cache_flag.py
test_response_model_sanitization.py
test_route_a2a_models.py
test_route_llm_request.py test(proxy): isolate the agent read-through singleton between unknown-agent tests 2026-09-22 01:34:31 +00:00
test_route_priority.py
test_sensitive_route_auth.py
test_shared_health_check.py
test_spend_log_cleanup.py Merge pull request #41213 from BerriAI/litellm_spend_log_cleanup_cancel_outcome 2026-09-21 17:03:25 -07:00
test_swagger_chat_completions.py
test_team_member_update.py fix(team): schedule membership audit writes after commit and lock the roster on role updates 2026-09-19 22:33:22 +00:00
test_team_org_move.py
test_tools_allowlist_enforcement.py
test_update_llm_router_resilience.py fix(proxy): keep the no-model_list guard to absent reads so model_list: [] still evicts 2026-09-16 23:14:52 +00:00