litellm/tests/test_litellm/proxy
yuneng-jiang c168199e33
test(ci): repair stale tests and move retired OpenAI text-completion fixtures (#43958)
* test(ci): repair stale request fakes, spend-log golden, auto-router labels, and Interactions spec lookups

Request fakes now carry the scope a real Starlette request has, the GCS pub/sub
spend-log golden gains the agent identity keys from #43722, the auto-router
session tests follow the baseline_models contract from #43348, and the
Interactions spec checks resolve the create body and resource paths from the
live spec instead of hardcoded names

* test(ci): move retired OpenAI text-completion fixtures to live vehicles

OpenAI still serves native /v1/completions on the gpt-5.4 family, so the
single-prompt cases move to text-completion-openai/gpt-5.4-nano. Multi-prompt
batches and echo with logprobs now 500 on every OpenAI model, so those cases
keep the same text-completion-openai transport pointed at Fireworks, which
documents both. The optional-params test asserts the request body actually
sent instead of a success callback whose assertions were swallowed

* test(ci): use a serverless Fireworks model for the text-completion batch and echo cases

gpt-oss-20b is on-demand only on Fireworks, so the CI key got 404 model not
deployed; glm-5p3-flash is listed as serverless

* test(ci): skip the ROI calculator repository listing in the security route sweep

GET /roi-calculator/repositories (#43669) lists repositories from the configured
GitHub API, api.github.com by default, so the S2 sweep's GET of every route made
the owned proxy reach an external host and failed the egress check in 31
integration-security tests. It joins /get/latest_release_info in the deny list
2026-09-30 19:19:59 -07:00
..
_experimental/mcp_server feat(ui): adopt the new LiteLLM logo and monogram (#43913) 2026-09-30 14:51:06 -07:00
a2a fix(a2a): reject malformed protocolVersion suffixes while keeping semver prereleases 2026-07-21 13:24:58 -07:00
agent_endpoints feat(agents): add identity registration and dashboard controls (#43723) 2026-09-30 15:21:21 -07:00
analytics_endpoints fix(proxy): attribute gate-rejected requests to their endpoint in cache analytics (#40824) 2026-09-12 16:03:38 -07:00
anthropic_endpoints feat(proxy): opt-in litellm_call_id in JSON error bodies (#42391) 2026-09-21 19:17:18 -07:00
auth refactor(proxy): answer every team access check with TeamAccess.allows (#43364) 2026-09-30 15:27:33 -07:00
batches_endpoints test(ci): repair stale tests and move retired OpenAI text-completion fixtures (#43958) 2026-09-30 19:19:59 -07:00
client feat(cli): reuse saved agent setup and add reconfigure (#43392) 2026-09-26 18:46:36 -07:00
common_utils fix(packaging): keep wheel paths under Windows MAX_PATH for Store Python (#43903) 2026-09-30 22:20:36 +00:00
config_resolvers feat(proxy): add maximum_daily_tag_spend_retention_period cleanup setting (#39221) 2026-09-26 15:07:56 -07:00
container_endpoints fix(containers): page upstream until a non-admin container list fills its limit 2026-09-02 21:26:34 -07:00
credential_endpoints fix(credentials): answer 409 on a name collision, let PATCH resolve values from model_id 2026-09-15 10:41:43 -07:00
db fix(proxy): attribute completed batch cost rows to /batches in daily activity (#43870) 2026-09-30 16:16:52 +00:00
discovery_endpoints chore(proxy): drop explanatory docstrings from credentials hint helper and tests 2026-09-14 19:29:30 +00:00
enterprise_billing feat(proxy): push-based OTLP billable-request metering for enterprise deployments (#31592) 2026-07-15 12:12:52 -07:00
experimental/mcp_server test: drop the cwd-relative sys.path.insert calls from the test suite (#37802) 2026-08-22 09:25:58 -07:00
fine_tuning_endpoints test: drop the cwd-relative sys.path.insert calls from the test suite (#37802) 2026-08-22 09:25:58 -07:00
google_endpoints test: drop the cwd-relative sys.path.insert calls from the test suite (#37802) 2026-08-22 09:25:58 -07:00
guardrails fix(guardrails): treat an unknown straiker api_version as unset instead of skipping the guardrail (#43956) 2026-09-30 18:20:57 -07:00
health_endpoints feat(langfuse): migrate the sdk callback to langfuse v4 (#36741) 2026-09-24 23:22:56 -07:00
hooks feat(proxy): record in spend logs whether a request used a client-forwarded Anthropic OAuth token (#43063) 2026-09-30 19:17:01 -07:00
image_endpoints ci: move tests/proxy_unit_tests to tests/unit/proxy and run the proxy-db shards from litellm-tests (#42903) 2026-09-24 22:59:11 +00:00
list_api feat: page the public model hub table off /public/v1/model_hub, keeping every filter (#39691) 2026-09-03 22:36:16 -07:00
logging_endpoints fix(usage): recover aliases for v1.99 double-hashed spend keys 2026-09-03 14:21:25 +00:00
management_endpoints refactor(proxy): answer every team access check with TeamAccess.allows (#43364) 2026-09-30 15:27:33 -07:00
management_helpers fix(s3_v2): upload fresh events first, drop terminal failures and hour-old retries by default, opt-in adaptive concurrency (#43022) 2026-09-26 14:58:28 -07:00
memory feat(proxy): add a search param to key, memory, audit, and spend log listings 2026-09-03 15:20:13 -07:00
middleware fix(proxy): release unclaimed budget reservations at request end (#42304) 2026-09-21 19:51:12 -07:00
ocr_endpoints refactor(ocr): remove the Python OCR execution path and require the Rust route (#43081) 2026-09-24 18:18:50 -07:00
openai_files_endpoint feat(vertex): native batch JSONL passthrough with cost tracking (#42810) 2026-09-24 12:35:34 -07:00
pass_through_endpoints fix(proxy): relay Azure passthrough body model groups through the router (#43896) 2026-09-30 14:57:31 -07:00
policy_engine fix(policy_engine): keep inherited parent guardrails when a child policy condition misses (#42548) 2026-09-22 23:27:38 -07:00
prompts fix(prompts): accept a string prompt_version and carry the viewed environment into code snippets 2026-08-29 12:55:34 -07:00
proxy_server feat(tracing): store spend in ClickHouse automatically (#43928) 2026-09-30 22:17:43 +00:00
public_endpoints fix(mcp): align hub publication status and controls (#43241) 2026-09-26 16:53:02 -07:00
rag_endpoints test(rag): drop the docstrings from the registered-store ingest tests 2026-09-19 04:10:49 -07:00
realtime_endpoints refactor(proxy): replace configurable model access denied message with a fixed clean client message 2026-09-16 01:44:36 +00:00
rerank_endpoints fix(proxy): carry litellm_call_id through endpoint specific error logs and failure responses 2026-09-16 02:29:26 +00:00
response_api_endpoints fix(responses): stream guardrail pre-call block as SSE with a typed output item (#42507) 2026-09-26 18:05:21 -07:00
shutdown refactor(proxy): inject scheduled job shutdown timeouts 2026-09-21 19:37:28 +00:00
spend_tracking feat(proxy): record in spend logs whether a request used a client-forwarded Anthropic OAuth token (#43063) 2026-09-30 19:17:01 -07:00
test_configs
types_utils feat(proxy): unified custom_key_policy hook for key generate, update and regenerate 2026-09-12 16:09:24 -07:00
ui_crud_endpoints fix(proxy): keep the submitted body out of 422 validation errors (#43231) 2026-09-25 17:05:53 -07:00
utils feat(agents): enforce authoritative agent permissions (#43721) 2026-09-30 11:11:37 -07:00
vector_store_endpoints fix(vector_stores): keep config-defined vector stores listed and read-only (#42574) 2026-09-23 04:02:16 +00:00
vector_store_files_endpoints test: drop the cwd-relative sys.path.insert calls from the test suite (#37802) 2026-08-22 09:25:58 -07:00
video_endpoints fix(videos): forward uploaded source file on /v1/videos/edits to the provider 2026-08-24 15:34:21 -07:00
__init__.py
conftest.py test(proxy): isolate the agent read-through singleton between unknown-agent tests 2026-09-22 01:34:31 +00:00
test__types.py feat(agents): add identity storage and validation contracts (#43720) 2026-09-29 17:59:21 -07:00
test_aiohttp_cleanup_closed.py
test_aiohttp_session_recovery.py
test_api_key_masking_in_errors.py
test_audio_speech_prometheus_hooks.py fix(proxy): match /v1/audio/speech content-type to the returned audio format 2026-08-29 13:44:43 -07:00
test_batch_expiry.py test: drop the cwd-relative sys.path.insert calls from the test suite (#37802) 2026-08-22 09:25:58 -07:00
test_batch_metadata_none_fix.py test: drop the cwd-relative sys.path.insert calls from the test suite (#37802) 2026-08-22 09:25:58 -07:00
test_batch_retrieve_bedrock.py test: drop the cwd-relative sys.path.insert calls from the test suite (#37802) 2026-08-22 09:25:58 -07:00
test_batch_x_litellm_model_encoding.py fix(proxy): apply model grants to unified file and batch ids on batch routes 2026-09-10 18:10:18 -04:00
test_blocked_response_usage.py fix(responses): stream guardrail pre-call block as SSE with a typed output item (#42507) 2026-09-26 18:05:21 -07:00
test_body_snapshot_callback_params.py test(integration): callback credential canary slots C1-C3 and D5 (#43630) 2026-09-29 10:49:30 -07:00
test_budget_reservation.py perf(proxy): hold one spend counter batch across admission and across post-call accounting (#43369) 2026-09-29 15:05:33 -07:00
test_bug_report_config.py feat(proxy): admin-only /debug/report sharing the bug report environment (#42440) 2026-09-22 12:05:23 -07:00
test_caching_routes.py test: drop the cwd-relative sys.path.insert calls from the test suite (#37802) 2026-08-22 09:25:58 -07:00
test_chat_completion_metadata.py
test_claude_code_marketplace.py test: run the 30 test files stranded in the second mirror (#37595) 2026-08-20 10:59:43 -07:00
test_collector.py feat(proxy): offload spend tracking to a pod-local collector sidecar (#40545) 2026-09-10 17:14:13 -07:00
test_common_request_processing.py fix(streaming): keep the served service_tier on streamed chunks and spend rows (#42870) 2026-09-29 12:54:17 -07:00
test_component_allowlists.py test(proxy): make two proxy-infra tests independent of sibling-test state (#42581) 2026-09-22 14:58:42 -07:00
test_conftest.py test(proxy): stop monkeypatch.undo re-planting fixture-mocked prisma_client 2026-08-13 20:11:06 -07:00
test_cors_config.py
test_custom_proxy.py test: drop the cwd-relative sys.path.insert calls from the test suite (#37802) 2026-08-22 09:25:58 -07:00
test_dynamic_mcp_route.py fix(proxy): serve aggregate MCP endpoint on bare /mcp instead of 307-redirecting (#34845) 2026-08-14 17:04:32 -07:00
test_empty_model_list.py test: drop the cwd-relative sys.path.insert calls from the test suite (#37802) 2026-08-22 09:25:58 -07:00
test_enforce_user_param.py test: require a match= on broad pytest.raises, and drop duplicate parametrize cases (#37769) 2026-08-20 20:24:49 -07:00
test_fallback_management_endpoints.py
test_fastapi_offline_routes.py test: drop the cwd-relative sys.path.insert calls from the test suite (#37802) 2026-08-22 09:25:58 -07:00
test_filter_models_by_team_access_group.py test: drop the cwd-relative sys.path.insert calls from the test suite (#37802) 2026-08-22 09:25:58 -07:00
test_health_check_functions.py fix(health): skip background health check DB writes when the latest-row read fails 2026-09-14 23:03:30 +00:00
test_health_check_max_tokens.py feat(auto-router): integrate JEV context and usage accounting 2026-09-18 21:38:15 +00:00
test_init_litellm_callbacks.py test: run the 30 test files stranded in the second mirror (#37595) 2026-08-20 10:59:43 -07:00
test_langfuse_passthrough_security.py
test_lazy_openapi_snapshot.py refactor(proxy): type the snapshot fragments and wrap a long test line 2026-08-26 14:39:51 -07:00
test_litellm_pre_call_utils.py feat(proxy): record in spend logs whether a request used a client-forwarded Anthropic OAuth token (#43063) 2026-09-30 19:17:01 -07:00
test_max_budget_env_var.py
test_mcp_asgi_response.py
test_model_based_routing_files_batches.py
test_model_deprecations_endpoint.py test: drop the cwd-relative sys.path.insert calls from the test suite (#37802) 2026-08-22 09:25:58 -07:00
test_model_dump_with_preserved_fields.py
test_model_id_header_propagation.py feat(proxy): add x-litellm-model-name response header with deployment model string (#33698) 2026-07-17 20:29:42 -07:00
test_model_info_default_limits.py perf(proxy): serialize /model/info listing once with orjson 2026-09-14 19:47:51 +00:00
test_model_level_guardrails.py fix(mcp): keep tool attribution on guardrail-blocked REST calls (#42790) 2026-09-23 15:40:56 -07:00
test_model_list_aliases.py fix(proxy): list key and team model aliases in GET /v1/models (#42908) 2026-09-24 06:41:13 -07:00
test_model_list_callback_filter.py feat(proxy): let callbacks filter the model listing routes per caller (#43027) 2026-09-24 15:52:44 -07:00
test_model_list_discoverable.py feat(proxy): honor model_info.discoverable on the model listing endpoints (#42825) 2026-09-23 17:51:16 -07:00
test_model_list_healthy_only.py perf(proxy): serialize /model/info listing once with orjson 2026-09-14 19:47:51 +00:00
test_modify_response_streaming_passthrough.py fix(proxy): use e.request_data for logging_obj in ModifyResponseException streaming passthrough (#30800) 2026-06-18 23:29:08 -07:00
test_native_compaction.py fix(gateway): expose /api/event_logging/batch on the gateway allowlist (#42572) 2026-09-22 14:09:40 -07:00
test_openai_ws_passthrough_routes.py fix(proxy): keep the token's team model list in the websocket passthrough gate without a database 2026-09-04 19:05:35 -07:00
test_openapi_schema_validation.py
test_plugin_routes.py refactor(proxy): resolve config and DB settings precedence in one SettingsStore 2026-09-17 23:36:27 -07:00
test_pointfive_dashboard_config.py feat(pointfive): add pointfive to the dashboard logging integrations 2026-09-10 14:02:35 +03:00
test_pointfive_ui_callback.py feat(pointfive): list pointfive in the proxy callback registry 2026-09-10 14:02:35 +03:00
test_pricing_field_strip.py fix(cost_calculator): bill chat per-second pricing once with a new cost_per_second field (#43614) 2026-09-29 11:27:14 -07:00
test_prisma_engine_watchdog.py fix(proxy): recreate the Prisma client when the writer session turns read-only (#40610) 2026-09-10 13:53:42 -07:00
test_prisma_migration.py fix(proxy): run migrations through python -m prisma when the prisma console script is not on PATH 2026-09-09 18:17:12 -07:00
test_prometheus_cleanup.py feat(deploy): metrics sidecar and separate metrics port in Helm and Terraform (#40163) 2026-09-08 13:31:07 -07:00
test_prometheus_metrics_server.py feat(deploy): metrics sidecar and separate metrics port in Helm and Terraform (#40163) 2026-09-08 13:31:07 -07:00
test_provider_url_destination_guard.py test: drop the cwd-relative sys.path.insert calls from the test suite (#37802) 2026-08-22 09:25:58 -07:00
test_proxy_cli.py feat(proxy_cli): add --validate_config dry-run flag (#41705) 2026-09-24 17:07:10 -05:00
test_proxy_logging_hook_detection.py fix(guardrails): store the masked output in spend logs when Presidio masks the response (#42441) 2026-09-22 01:19:22 -07:00
test_proxy_server.py test(proxy): scope user_api_key_auth overrides in proxy_server tests (#43952) 2026-10-01 00:27:53 +00:00
test_proxy_types.py fix(router): honor team and key provider weights 2026-09-14 23:31:52 -07:00
test_proxy_utils.py feat(guardrails): scan retrieved vector store chunks with the request's pre-call guardrails (#43271) 2026-09-26 14:18:38 -07:00
test_pyroscope.py
test_read_model_list.py feat: add minimal rust router + axum ai-gateway calling router.realtime (2/2) (#31135) 2026-06-23 19:16:34 -07:00
test_redis_auth_cache_flag.py fix(proxy): share per-model budget counters across replicas through the spend counter cache (#39375) 2026-09-02 12:40:59 -07:00
test_response_model_sanitization.py test: drop the cwd-relative sys.path.insert calls from the test suite (#37802) 2026-08-22 09:25:58 -07:00
test_route_a2a_models.py feat(agents): authenticate Entra identities and delegated requests (#43722) 2026-09-30 13:05:22 -07:00
test_route_llm_request.py test(proxy): isolate the agent read-through singleton between unknown-agent tests 2026-09-22 01:34:31 +00:00
test_route_priority.py perf(proxy): register liveness and core inference routes first (#40687) 2026-09-11 17:10:38 +00:00
test_sensitive_route_auth.py
test_shared_health_check.py fix(proxy): derive auto-router health from its underlying models (#38174) 2026-08-26 16:41:54 -07:00
test_spend_log_cleanup.py feat(proxy): add maximum_daily_tag_spend_retention_period cleanup setting (#39221) 2026-09-26 15:07:56 -07:00
test_swagger_chat_completions.py
test_team_member_update.py fix(team): schedule membership audit writes after commit and lock the roster on role updates 2026-09-19 22:33:22 +00:00
test_team_org_move.py test: require a match= on broad pytest.raises, and drop duplicate parametrize cases (#37769) 2026-08-20 20:24:49 -07:00
test_tools_allowlist_enforcement.py fix(anthropic): close hybrid tool-name allowlist gap and keep native tools through guardrails 2026-08-26 20:37:11 -07:00
test_tracing_endpoints.py feat(tracing): add OTLP trace ingestion and reads (#43915) 2026-09-30 21:12:29 +00:00
test_update_llm_router_resilience.py fix(proxy): keep the no-model_list guard to absent reads so model_list: [] still evicts 2026-09-16 23:14:52 +00:00
test_zerobus_dashboard_config.py feat(integrations): add Databricks Zerobus trace logging callback (#42013) 2026-09-25 15:29:23 -07:00