litellm/tests
shin-bot-litellm b609f5841b
fix: add missing OpenAI chat completion params to OPENAI_CHAT_COMPLETION_PARAMS (#21360)
* allow filtering by user in global usage

* add server root path test to github actions

* Update .github/workflows/test_server_root_path.yml

Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>

* address greptile review feedback (greploop iteration 1)

- Fix HTTPException swallowed by broad except block in get_user_daily_activity
  and get_user_daily_activity_aggregated: re-raise HTTPException before the
  generic handler so 403 status codes propagate correctly
- Add status_code assertions in non-admin access tests

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* address greptile review feedback (greploop iteration 2)

- Default user_id to caller's own ID for non-admins instead of 403 when
  omitted, preserving backward compatibility for API consumers
- Apply same fix to aggregated endpoint
- Update test to verify defaulting behavior instead of expecting 403
- Add useEffect to sync selectedUserId when auth state settles in
  UsagePageView to handle async auth initialization

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fixing syntax

* remove artifacts

* feat: guardrail tracing UI - policy, detection method, match details (#21349)

* feat: add GuardrailTracingDetail TypedDict and tracing fields to StandardLoggingGuardrailInformation

* feat: add policy_template field to Guardrail config TypedDict

* feat: accept GuardrailTracingDetail in base guardrail logging method

* feat: populate tracing fields in content filter guardrail

* test: add tracing fields tests for custom guardrail base class

* test: add tracing fields e2e tests for content filter guardrail

* feat: add guardrail tracing UI - policy badges, match details, timeline

* feat: redesign GuardrailViewer to Guardrails & Policy Compliance layout

Two-column layout with request lifecycle timeline on the left
and compact evaluation detail cards on the right. Header shows
guardrail count, pass/fail status, total overhead, policy info,
and an export button.

* feat: add clickable guardrail link in metrics + show policy names

* feat: add risk_score field to StandardLoggingGuardrailInformation

* feat: compute risk_score in content filter guardrail

* feat: display backend risk_score badge on evaluation cards

* fix: fallback to frontend risk score when backend doesn't provide one

* passing in masster key for api calls

* Fix: Add blog as incident report

* Fix: Add blog as incident report

* remove timeline

* feat(models): add github_copilot/gpt-5.3-codex and github_copilot/claude-opus-4.6-fast (#21316)

Add missing GitHub Copilot model entries for gpt-5.3-codex (GA) and
claude-opus-4.6-fast (Public Preview) to both the root and backup
model pricing JSON files.

* only tests for /ui

* bump: version 1.81.12 → 1.81.13

* Fixing mapped tests

* fixing no_config test

* fixing container tests

* fixing test_basic_openai_responses_api

* Adding bedrock thinking budget tokens to docs

* fixing regen key tests

* fix: add missing OpenAI chat completion params to OPENAI_CHAT_COMPLETION_PARAMS

Add store, prompt_cache_key, prompt_cache_retention, safety_identifier, and verbosity
to OPENAI_CHAT_COMPLETION_PARAMS list.

These params were already in DEFAULT_CHAT_COMPLETION_PARAM_VALUES but missing from
the OPENAI_CHAT_COMPLETION_PARAMS list, causing them to be dropped when passed to
OpenAI-compatible providers.

---------

Co-authored-by: yuneng-jiang <yuneng.jiang@gmail.com>
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-authored-by: Ishaan Jaff <ishaanjaffer0324@gmail.com>
Co-authored-by: Sameer Kankute <sameer@berri.ai>
Co-authored-by: Cesar Garcia <128240629+Chesars@users.noreply.github.com>
Co-authored-by: Krish Dholakia <krrishdholakia@gmail.com>
2026-02-16 20:31:21 -08:00
..
agent_tests [Fix] A2a Agent Gateway Fixes - A2A agents deployed with localhost/internal URLs in their agent cards (e.g., http://0.0.0.0:8001/) (#20604) 2026-02-06 15:02:34 -08:00
audio_tests [Feat] New provider TTS - Add AWS polly API for TTS (#18326) 2025-12-22 18:19:34 +05:30
basic_proxy_startup_tests Fix: remove merge markdown (#17586) 2025-12-06 05:38:16 -08:00
batches_tests fix: thread deployment model_info through batch cost calculation 2026-02-15 14:53:30 -05:00
code_coverage_tests IGNORE_FUNCTIONS 2026-02-14 12:59:15 -08:00
documentation_tests fix: test_env_keys 2026-01-20 18:37:56 +05:30
enterprise removed /models and v1/models from llm api routes (#20988) 2026-02-13 18:32:41 +05:30
guardrails_tests feat: EU AI Act Article 5 policy template for prohibited practices detection (#21342) 2026-02-16 15:33:07 -08:00
image_gen_tests [Release - 02/10/2026] v1.81.10-nightly 2026-02-10 16:26:30 -08:00
litellm fix(anthropic): filter unsupported JSON schema constraints for structured outputs (#20813) 2026-02-14 09:07:40 -08:00
litellm-proxy-extras fix: resolve 'does not exist' migration errors as applied in setup_database (#19281) 2026-01-26 22:11:36 -08:00
litellm_core_utils Fix: empty assistant message for converse API 2026-02-04 09:50:58 +05:30
litellm_utils_tests feat: hashicorp vault rotate support 2026-01-23 17:32:55 +09:00
llm_responses_api_testing fix: add missing OpenAI chat completion params to OPENAI_CHAT_COMPLETION_PARAMS (#21360) 2026-02-16 20:31:21 -08:00
llm_translation fix: add store to OPENAI_CHAT_COMPLETION_PARAMS (#21195) 2026-02-16 20:28:34 -08:00
load_tests Add memory leak detection tests with CI integration (#18881) 2026-01-09 17:36:10 -08:00
local_testing test_partner_models_httpx_streaming 2026-02-14 12:41:35 -08:00
logging_callback_tests fix(logging): avoid shared callback list references (#20984) 2026-02-13 18:32:41 +05:30
mcp_tests fix: import LiteLLM_ObjectPermissionTable from _types instead of proxy_server 2026-02-13 14:46:03 -03:00
multi_instance_e2e_tests fix: use fastuuid helper (#14903) 2025-09-25 15:47:01 -07:00
ocr_tests OCR test fixes 2026-01-11 08:00:31 -08:00
old_proxy_tests/tests remove prompt caching headers as the support has been removed 2026-01-02 11:08:35 +05:30
openai_endpoints_tests Fix: Managed Batches: Inconsistent State Management for list and cancel batches 2026-02-03 14:47:28 +05:30
otel_tests test fix 2026-01-31 19:08:07 -08:00
pass_through_tests test_anthropic_messages_to_wildcard_model 2026-01-07 15:03:06 +05:30
pass_through_unit_tests Fix beta header old tests 2026-02-11 18:13:36 +05:30
proxy_admin_ui_tests sorting spend logs in ui 2026-02-13 11:33:41 -08:00
proxy_e2e_anthropic_messages_tests test_anthropic_messages_with_all_beta_headers 2026-02-14 13:03:52 -08:00
proxy_security_tests test fix 2025-10-04 10:57:02 -07:00
proxy_unit_tests fix(test): mock environment variables for callback validation test 2026-02-15 20:44:17 -03:00
router_unit_tests fix(test): add router.acancel_batch coverage (#20183) 2026-01-31 12:39:19 -08:00
scim_tests
search_tests skip brave tests 2026-01-22 10:50:23 -08:00
spend_tracking_tests increasing time for spend tracking 2026-01-20 16:31:25 -08:00
store_model_in_db_tests fix: test mock 2026-01-02 17:38:52 +09:00
test_litellm fix: add missing OpenAI chat completion params to OPENAI_CHAT_COMPLETION_PARAMS (#21360) 2026-02-16 20:31:21 -08:00
unified_google_tests Revert "[Infra] Changing Google Tests to use Gemini 3 Flash Preview" 2026-01-20 17:32:10 -08:00
vector_store_tests Add support for vertex ai for rag/ingest 2026-02-13 20:11:10 +05:30
windows_tests
__init__.py
gettysburg.wav
large_text.py
openai_batch_completions.jsonl
README.MD [Feat] MCP Gateway Fine-grained Tools Addition (#15153) 2025-10-03 10:16:29 -07:00
test_budget_management.py fix: use fastuuid helper (#14903) 2025-09-25 15:47:01 -07:00
test_callbacks_on_proxy.py Fix CI: Revert security scan changes and add GitGuardian ignore rules (#18358) 2025-12-22 17:03:53 -08:00
test_config.py fix: use fastuuid helper (#14903) 2025-09-25 15:47:01 -07:00
test_debug_warning.py
test_default_encoding_non_root.py fix: resolve Read-only file system error in non-root images (#19449) 2026-01-20 19:00:52 -08:00
test_end_users.py fix: use fastuuid helper (#14903) 2025-09-25 15:47:01 -07:00
test_entrypoint.py
test_fallbacks.py Revert "fix: prevent error when max_fallbacks exceeds available models (#20071)" 2026-02-03 15:15:30 +05:30
test_gpt5_azure_temperature_support.py Fix: Azure GPT-5 incorrectly routed to O-series config (temperature parameter unsupported) (#16246) 2025-11-07 10:24:27 -08:00
test_health.py
test_keys.py Adding retries to flaky tests 2026-01-22 15:21:44 -08:00
test_litellm_proxy_responses_config.py Add native Responses API support for litellm_proxy provider (#15347) 2025-10-08 18:31:26 -07:00
test_logging.conf
test_models.py test_add_model_run_health 2025-09-27 10:59:25 -07:00
test_openai_endpoints.py test fix claude-sonnet-4-5-20250929 2025-10-28 19:05:13 -07:00
test_organizations.py Adding retries to flaky tests 2026-01-22 15:21:44 -08:00
test_otel_thread_leak.py Fix thread leak in OpenTelemetry dynamic header path (#19946) 2026-01-28 10:35:37 -08:00
test_passthrough_endpoints.py fix: update authorization header to use 'Bearer' instead of 'bearer' 2025-09-21 10:44:47 +00:00
test_presidio_latency.py fix(presidio): reuse HTTP connections to prevent OOMs (#19964) 2026-01-28 16:08:53 -08:00
test_proxy_server_non_root.py deactivating non root tests 2026-01-23 22:55:36 -08:00
test_ratelimit.py test_async_rate_limit 2025-11-15 10:34:24 -08:00
test_resource_cleanup.py
test_service_logger_otel.py fix(langfuse_otel): prevent empty proxy request spans from being sent to Langfuse 2026-01-28 15:35:35 +01:00
test_spend_logs.py Fix CI: Revert security scan changes and add GitGuardian ignore rules (#18358) 2025-12-22 17:03:53 -08:00
test_team.py Fix: test_team_update_sc_2 2026-02-05 09:40:21 +05:30
test_team_logging.py fix: use fastuuid helper (#14903) 2025-09-25 15:47:01 -07:00
test_team_members.py fix: use fastuuid helper (#14903) 2025-09-25 15:47:01 -07:00
test_users.py fix: use fastuuid helper (#14903) 2025-09-25 15:47:01 -07:00

In total litellm runs 1000+ tests

[02/20/2025] Update:

To make it easier to contribute and map what behavior is tested,

we've started mapping the litellm directory in tests/test_litellm

This folder can only run mock tests.