scriptoriumadmin/litellm
1
0
Fork 0
mirror of https://github.com/BerriAI/litellm.git synced 2026-09-28 01:32:17 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 43
litellm/tests/test_litellm/proxy
History
yuneng-jiang 65e42526d6
test(proxy): make two proxy-infra tests independent of sibling-test state (#42581) 
* test(proxy): make two proxy-infra tests independent of sibling-test state

Both tests read process-global state that another module in the same xdist
worker can change, so they passed or failed on shard scheduling rather than on
the behaviour they assert.

test_gateway_plus_backend_covers_full_app checked allowlist coverage against
the live route table. gateway/main.py trims routes once, inside the lifespan,
so it only ever sees what is registered at startup; a lazy feature appends its
router on demand afterwards and is never filtered. Those routes cannot be
dropped on the floor, but they do enter the assertion the moment a sibling test
warms the feature, and 110 of them sit in neither allowlist. Subtract exactly
the lazy features this process has loaded, which leaves the assertion at full
strength for every eagerly registered route.

test_real_proxy_child_auth_privacy_and_body_policy pins prisma_client to a bare
object(). litellm.max_budget is a module global that nothing restores between
tests; once a sibling leaves it above zero, user_api_key_auth takes the
global-spend branch, dereferences prisma_client.db, and the AttributeError
surfaces as HTTP 401. Pin max_budget next to the other globals the test already
controls.

* test(proxy): measure allowlist coverage in a pristine interpreter

The previous revision subtracted paths matching a loaded lazy feature's
prefixes. Those prefixes are broad enough to swallow eagerly registered routes:
31 of them, including /openai/deployments/*, /access_group/*, /cursor/* and
/mcp, which would have made a real allowlist regression invisible.

Run the coverage check in a fresh interpreter instead. No lazy feature is
loaded there, so the route table is exactly the one gateway/main.py's lifespan
trim sees, and nothing has to be subtracted for the result to be deterministic.
The probe also reports the lazy modules it loaded and its route count, so an
empty uncovered set cannot pass vacuously.

Drop _component_paths and the four allowlist constants it used; the probe
reproduces the predicate in the child process.
2026-09-22 14:58:42 -07:00
..
_experimental/mcp_server fix(mcp): preserve discovery attribution and sanitize logging headers (#42541) 2026-09-22 14:26:48 -07:00
a2a fix(a2a): reject malformed protocolVersion suffixes while keeping semver prereleases 2026-07-21 13:24:58 -07:00
agent_endpoints fix(proxy): scope agent permissions to invoking caller 2026-09-21 22:32:32 +00:00
analytics_endpoints fix(proxy): attribute gate-rejected requests to their endpoint in cache analytics (#40824) 2026-09-12 16:03:38 -07:00
anthropic_endpoints feat(proxy): opt-in litellm_call_id in JSON error bodies (#42391) 2026-09-21 19:17:18 -07:00
auth chore(cost-map): remove models past their deprecation date (#42435) 2026-09-22 21:19:26 +00:00
batches_endpoints chore: merge main into fix/batch-retrieve-model-group 2026-09-19 16:25:39 -07:00
client fix(cli): serialize footer installs and tolerate unknown versions 2026-09-21 16:48:08 -07:00
common_utils fix: enforce disable_custom_api_keys from general_settings (#42437) 2026-09-22 13:28:31 -07:00
config_resolvers test: drop two inert type: ignore comments 2026-09-19 14:22:19 -07:00
container_endpoints fix(containers): page upstream until a non-admin container list fills its limit 2026-09-02 21:26:34 -07:00
credential_endpoints fix(credentials): answer 409 on a name collision, let PATCH resolve values from model_id 2026-09-15 10:41:43 -07:00
db fix(proxy): honor DATABASE_DISABLE_PREPARED_STATEMENTS in the litellm CLI (#42556) 2026-09-22 16:10:49 -05:00
discovery_endpoints chore(proxy): drop explanatory docstrings from credentials hint helper and tests 2026-09-14 19:29:30 +00:00
enterprise_billing feat(proxy): push-based OTLP billable-request metering for enterprise deployments (#31592) 2026-07-15 12:12:52 -07:00
experimental/mcp_server test: drop the cwd-relative sys.path.insert calls from the test suite (#37802) 2026-08-22 09:25:58 -07:00
fine_tuning_endpoints test: drop the cwd-relative sys.path.insert calls from the test suite (#37802) 2026-08-22 09:25:58 -07:00
google_endpoints test: drop the cwd-relative sys.path.insert calls from the test suite (#37802) 2026-08-22 09:25:58 -07:00
guardrails fix(mcp): apply post-call rewrites without stale structured output (#41530) 2026-09-22 13:44:25 -07:00
health_endpoints Merge pull request #40814 from BerriAI/litellm_gate_health_services_alert_tests 2026-09-12 21:13:30 -07:00
hooks fix(proxy): share model rate-limit buckets between a model_group_alias and its target (#42516) 2026-09-22 13:31:58 -07:00
image_endpoints Merge remote-tracking branch 'origin/main' into litellm_fix_image_edits_bracketed_alias 2026-09-17 14:49:54 -07:00
list_api feat: page the public model hub table off /public/v1/model_hub, keeping every filter (#39691) 2026-09-03 22:36:16 -07:00
logging_endpoints fix(usage): recover aliases for v1.99 double-hashed spend keys 2026-09-03 14:21:25 +00:00
management_endpoints feat(proxy): configurable key_alias_pattern for key generate, update, and regenerate (#42553) 2026-09-22 14:52:01 -07:00
management_helpers fix(proxy): write key deleted audit logs for cascade and alias key deletions (#42446) 2026-09-22 11:52:08 -07:00
memory feat(proxy): add a search param to key, memory, audit, and spend log listings 2026-09-03 15:20:13 -07:00
middleware fix(proxy): release unclaimed budget reservations at request end (#42304) 2026-09-21 19:51:12 -07:00
ocr_endpoints fix(ocr): validate body req_format in the proxy endpoint and run its tests in CI 2026-08-17 18:29:35 +00:00
openai_files_endpoint fix(files): keep an explicit target_storage on its old path and refuse litellm_db as a caller choice 2026-09-19 13:20:02 -07:00
pass_through_endpoints chore(cost-map): remove models past their deprecation date (#42435) 2026-09-22 21:19:26 +00:00
policy_engine fix(policy_engine): resolve policies once and apply fallback semantics in get_matching_policies 2026-09-21 17:36:03 +00:00
prompts fix(prompts): accept a string prompt_version and carry the viewed environment into code snippets 2026-08-29 12:55:34 -07:00
proxy_server fix(ssrf): point the blocked-address remediation at litellm_settings (#42508) 2026-09-22 14:04:16 -07:00
public_endpoints test: keep the Qwen brand missing-key tests off the network and split the ChatGPT fields test back out 2026-09-21 12:54:23 -07:00
rag_endpoints test(rag): drop the docstrings from the registered-store ingest tests 2026-09-19 04:10:49 -07:00
realtime_endpoints refactor(proxy): replace configurable model access denied message with a fixed clean client message 2026-09-16 01:44:36 +00:00
rerank_endpoints fix(proxy): carry litellm_call_id through endpoint specific error logs and failure responses 2026-09-16 02:29:26 +00:00
response_api_endpoints test(proxy): expect 422 for per-model budget rejections on cursor route 2026-09-20 06:10:16 +00:00
shutdown refactor(proxy): inject scheduled job shutdown timeouts 2026-09-21 19:37:28 +00:00
spend_tracking chore(cost-map): remove models past their deprecation date (#42435) 2026-09-22 21:19:26 +00:00
test_configs
types_utils feat(proxy): unified custom_key_policy hook for key generate, update and regenerate 2026-09-12 16:09:24 -07:00
ui_crud_endpoints fix: enforce disable_custom_api_keys from general_settings (#42437) 2026-09-22 13:28:31 -07:00
utils fix(proxy): attribute provider and model_info on pre_call_hook rejections (#41077) 2026-09-22 14:14:13 -07:00
vector_store_endpoints fix(vector_stores): run the full model grant check on caller-supplied model hints 2026-09-19 00:54:41 -07:00
vector_store_files_endpoints test: drop the cwd-relative sys.path.insert calls from the test suite (#37802) 2026-08-22 09:25:58 -07:00
video_endpoints fix(videos): forward uploaded source file on /v1/videos/edits to the provider 2026-08-24 15:34:21 -07:00
__init__.py
conftest.py test(proxy): isolate the agent read-through singleton between unknown-agent tests 2026-09-22 01:34:31 +00:00
test__types.py feat(auth): breached password detection, self-service change-password and forced password reset 2026-09-21 18:48:35 +00:00
test_aiohttp_cleanup_closed.py
test_aiohttp_session_recovery.py
test_api_key_masking_in_errors.py
test_audio_speech_prometheus_hooks.py fix(proxy): match /v1/audio/speech content-type to the returned audio format 2026-08-29 13:44:43 -07:00
test_batch_expiry.py test: drop the cwd-relative sys.path.insert calls from the test suite (#37802) 2026-08-22 09:25:58 -07:00
test_batch_metadata_none_fix.py test: drop the cwd-relative sys.path.insert calls from the test suite (#37802) 2026-08-22 09:25:58 -07:00
test_batch_retrieve_bedrock.py test: drop the cwd-relative sys.path.insert calls from the test suite (#37802) 2026-08-22 09:25:58 -07:00
test_batch_x_litellm_model_encoding.py fix(proxy): apply model grants to unified file and batch ids on batch routes 2026-09-10 18:10:18 -04:00
test_blocked_response_usage.py fix(guardrails): remove the module-global translation mapping that leaked between tests 2026-09-03 03:28:52 -07:00
test_budget_reservation.py feat(tokenizer): preserve Python defaults with opt-in Rust dispatch (#42174) 2026-09-22 04:41:11 +00:00
test_bug_report_config.py feat(proxy): admin-only /debug/report sharing the bug report environment (#42440) 2026-09-22 12:05:23 -07:00
test_caching_routes.py test: drop the cwd-relative sys.path.insert calls from the test suite (#37802) 2026-08-22 09:25:58 -07:00
test_chat_completion_metadata.py
test_claude_code_marketplace.py test: run the 30 test files stranded in the second mirror (#37595) 2026-08-20 10:59:43 -07:00
test_collector.py feat(proxy): offload spend tracking to a pod-local collector sidecar (#40545) 2026-09-10 17:14:13 -07:00
test_common_request_processing.py feat(proxy): opt-in include_guardrail_response returns guardrail_information in the response (#42327) 2026-09-22 12:43:30 -07:00
test_component_allowlists.py test(proxy): make two proxy-infra tests independent of sibling-test state (#42581) 2026-09-22 14:58:42 -07:00
test_conftest.py test(proxy): stop monkeypatch.undo re-planting fixture-mocked prisma_client 2026-08-13 20:11:06 -07:00
test_cors_config.py
test_custom_proxy.py test: drop the cwd-relative sys.path.insert calls from the test suite (#37802) 2026-08-22 09:25:58 -07:00
test_dynamic_mcp_route.py fix(proxy): serve aggregate MCP endpoint on bare /mcp instead of 307-redirecting (#34845) 2026-08-14 17:04:32 -07:00
test_empty_model_list.py test: drop the cwd-relative sys.path.insert calls from the test suite (#37802) 2026-08-22 09:25:58 -07:00
test_enforce_user_param.py test: require a match= on broad pytest.raises, and drop duplicate parametrize cases (#37769) 2026-08-20 20:24:49 -07:00
test_fallback_management_endpoints.py
test_fastapi_offline_routes.py test: drop the cwd-relative sys.path.insert calls from the test suite (#37802) 2026-08-22 09:25:58 -07:00
test_filter_models_by_team_access_group.py test: drop the cwd-relative sys.path.insert calls from the test suite (#37802) 2026-08-22 09:25:58 -07:00
test_health_check_functions.py fix(health): skip background health check DB writes when the latest-row read fails 2026-09-14 23:03:30 +00:00
test_health_check_max_tokens.py feat(auto-router): integrate JEV context and usage accounting 2026-09-18 21:38:15 +00:00
test_init_litellm_callbacks.py test: run the 30 test files stranded in the second mirror (#37595) 2026-08-20 10:59:43 -07:00
test_langfuse_passthrough_security.py chore(callbacks): guard dynamic integration hosts 2026-04-30 14:27:19 -07:00
test_lazy_openapi_snapshot.py refactor(proxy): type the snapshot fragments and wrap a long test line 2026-08-26 14:39:51 -07:00
test_litellm_pre_call_utils.py fix(mcp): preserve discovery attribution and sanitize logging headers (#42541) 2026-09-22 14:26:48 -07:00
test_max_budget_env_var.py
test_mcp_asgi_response.py fix(mcp): surface upstream 401 for token-forwarding MCP servers (#27847) 2026-05-13 12:03:36 -07:00
test_model_based_routing_files_batches.py test(batches): move orphan tests into tests/test_litellm for CI coverage (#30510) 2026-06-16 10:20:59 -07:00
test_model_deprecations_endpoint.py test: drop the cwd-relative sys.path.insert calls from the test suite (#37802) 2026-08-22 09:25:58 -07:00
test_model_dump_with_preserved_fields.py
test_model_id_header_propagation.py feat(proxy): add x-litellm-model-name response header with deployment model string (#33698) 2026-07-17 20:29:42 -07:00
test_model_info_default_limits.py perf(proxy): serialize /model/info listing once with orjson 2026-09-14 19:47:51 +00:00
test_model_level_guardrails.py fix(proxy): apply key/team router_settings.model_group_alias (#35486) 2026-08-03 22:09:47 +00:00
test_model_list_healthy_only.py perf(proxy): serialize /model/info listing once with orjson 2026-09-14 19:47:51 +00:00
test_modify_response_streaming_passthrough.py fix(proxy): use e.request_data for logging_obj in ModifyResponseException streaming passthrough (#30800) 2026-06-18 23:29:08 -07:00
test_native_compaction.py fix(gateway): expose /api/event_logging/batch on the gateway allowlist (#42572) 2026-09-22 14:09:40 -07:00
test_openai_ws_passthrough_routes.py fix(proxy): keep the token's team model list in the websocket passthrough gate without a database 2026-09-04 19:05:35 -07:00
test_openapi_schema_validation.py fix(proxy): preserve HTTP operations when injecting WebSocket stubs into OpenAPI schema 2026-05-06 00:28:42 +02:00
test_plugin_routes.py refactor(proxy): resolve config and DB settings precedence in one SettingsStore 2026-09-17 23:36:27 -07:00
test_pointfive_dashboard_config.py feat(pointfive): add pointfive to the dashboard logging integrations 2026-09-10 14:02:35 +03:00
test_pointfive_ui_callback.py feat(pointfive): list pointfive in the proxy callback registry 2026-09-10 14:02:35 +03:00
test_pricing_field_strip.py fix(proxy): fold litellm_metadata into metadata on chat routes so tag routing sees merged tags 2026-08-29 00:42:43 -07:00
test_prisma_engine_watchdog.py fix(proxy): recreate the Prisma client when the writer session turns read-only (#40610) 2026-09-10 13:53:42 -07:00
test_prisma_migration.py fix(proxy): run migrations through python -m prisma when the prisma console script is not on PATH 2026-09-09 18:17:12 -07:00
test_prometheus_cleanup.py feat(deploy): metrics sidecar and separate metrics port in Helm and Terraform (#40163) 2026-09-08 13:31:07 -07:00
test_prometheus_metrics_server.py feat(deploy): metrics sidecar and separate metrics port in Helm and Terraform (#40163) 2026-09-08 13:31:07 -07:00
test_provider_url_destination_guard.py test: drop the cwd-relative sys.path.insert calls from the test suite (#37802) 2026-08-22 09:25:58 -07:00
test_proxy_cli.py fix(proxy_cli): import proxy_server once on script-style boot (#42584) 2026-09-22 14:29:50 -07:00
test_proxy_logging_hook_detection.py fix(guardrails): store the masked output in spend logs when Presidio masks the response (#42441) 2026-09-22 01:19:22 -07:00
test_proxy_server.py feat(proxy): configurable key_alias_pattern for key generate, update, and regenerate (#42553) 2026-09-22 14:52:01 -07:00
test_proxy_types.py fix(router): honor team and key provider weights 2026-09-14 23:31:52 -07:00
test_proxy_utils.py feat(errors): prefilled GitHub issue link on unmapped internal errors (#42065) 2026-09-21 22:30:41 -07:00
test_pyroscope.py Implement normalize_nonempty_secret_str function to trim whitespace from secrets and treat empty values as unset. Update proxy_server to use this function for Grafana credentials. Enhance tests to validate the new normalization behavior. 2026-05-04 18:17:31 +00:00
test_read_model_list.py feat: add minimal rust router + axum ai-gateway calling router.realtime (2/2) (#31135) 2026-06-23 19:16:34 -07:00
test_redis_auth_cache_flag.py fix(proxy): share per-model budget counters across replicas through the spend counter cache (#39375) 2026-09-02 12:40:59 -07:00
test_response_model_sanitization.py test: drop the cwd-relative sys.path.insert calls from the test suite (#37802) 2026-08-22 09:25:58 -07:00
test_route_a2a_models.py test: drop the cwd-relative sys.path.insert calls from the test suite (#37802) 2026-08-22 09:25:58 -07:00
test_route_llm_request.py test(proxy): isolate the agent read-through singleton between unknown-agent tests 2026-09-22 01:34:31 +00:00
test_route_priority.py perf(proxy): register liveness and core inference routes first (#40687) 2026-09-11 17:10:38 +00:00
test_sensitive_route_auth.py chore(proxy): guard sensitive public endpoints 2026-04-30 11:52:47 -07:00
test_shared_health_check.py fix(proxy): derive auto-router health from its underlying models (#38174) 2026-08-26 16:41:54 -07:00
test_spend_log_cleanup.py Merge pull request #41213 from BerriAI/litellm_spend_log_cleanup_cancel_outcome 2026-09-21 17:03:25 -07:00
test_swagger_chat_completions.py fix(proxy): avoid misleading multi-method operation ids 2026-04-30 20:44:14 -07:00
test_team_member_update.py fix(team): schedule membership audit writes after commit and lock the roster on role updates 2026-09-19 22:33:22 +00:00
test_team_org_move.py test: require a match= on broad pytest.raises, and drop duplicate parametrize cases (#37769) 2026-08-20 20:24:49 -07:00
test_tools_allowlist_enforcement.py fix(anthropic): close hybrid tool-name allowlist gap and keep native tools through guardrails 2026-08-26 20:37:11 -07:00
test_update_llm_router_resilience.py fix(proxy): keep the no-model_list guard to absent reads so model_list: [] still evicts 2026-09-16 23:14:52 +00:00