litellm/tests/test_litellm
Lior Drihem 62b84d6aad
Prompt security litellm (#16365)
* add prompt security guardrails provider

* cosmetic

* small

* add file sanitization and update context window

* add pdf and OOXML files support

* add system prompt support

* add tests and documentation

* remove print

* fix PLR0915 Too many statements (96 > 50)

* cosmetic

* fix mypy error

* Fix failed tests due to naming conflict of responses directory with same-named pip package

* Fix mypy error: use 'aembedding' instead of 'embeddings' for async embedding call type

* Fix: Install enterprise package into Poetry virtualenv for tests

The GitHub Actions workflow was installing litellm-enterprise to system Python
using 'python -m pip install -e .', but tests run in Poetry's virtualenv using
'poetry run pytest'. This caused ImportError for enterprise package types.

Changed to 'poetry run pip install -e .' so the package is available in the
same virtualenv where pytest executes.

Fixes enterprise test collection errors in GitHub Actions CI.

* Move Prompt Security guardrail tests to tests/test_litellm/

Per reviewer feedback, move test_prompt_security_guardrails.py from
tests/guardrails_tests/ to tests/test_litellm/proxy/guardrails/ so
it will be executed by GitHub Actions workflow test-litellm.yml.

This ensures the Prompt Security integration tests run in CI.

---------

Co-authored-by: Ori Tabac <oritabac@prompt.security>
Co-authored-by: Vitaly Neyman <vitaly@prompt.security>
2025-11-24 11:44:20 -08:00
..
caching fix(redis): handle float redis_version from AWS ElastiCache Valkey (#16207) 2025-11-04 19:20:00 -08:00
completion_extras/litellm_responses_transformation Fix optional param mapping (#16852) 2025-11-19 19:10:04 -08:00
containers Add E2E Container API Support (#16136) 2025-11-01 14:03:51 -07:00
enterprise/enterprise_callbacks [Feat] AI Gateway - Allow admins to disable, dynamic callback controls (#16750) 2025-11-17 18:29:07 -08:00
experimental_mcp_client refactor: drop MCPClient.connect and use run_with_session lifecycle (#16696) 2025-11-15 17:54:27 -08:00
google_genai [Oct Staging Branch] (#15460) 2025-10-17 17:52:25 -07:00
integrations Permission Management - disable global guardrails by key/team (#16983) 2025-11-22 15:43:50 -08:00
litellm_core_utils (fix) litellm_logging.py: fix mcp tool call response logging + (fix) responses_bridge: remove unmapped param error mid-stream - allows gpt-5 web search to work via responses api in .completion() (#16946) 2025-11-22 15:48:32 -08:00
llms Prompt security litellm (#16365) 2025-11-24 11:44:20 -08:00
passthrough fix test claude-sonnet-4-5-20250929 2025-10-31 18:13:29 -07:00
proxy Prompt security litellm (#16365) 2025-11-24 11:44:20 -08:00
responses fix: prevent duplicate spend logs in Responses API for non-OpenAI providers (#16992) 2025-11-23 21:57:01 -08:00
router_strategy fix: Prevent AttributeError for _get_tags_from_request_kwargs (#14735) 2025-09-19 15:21:02 -07:00
router_utils Add LLM provider response headers to Responses API (#16091) 2025-11-01 13:25:56 -07:00
secret_managers [Feat] Add Custom Secret Manager - Allow users to define and write a custom secret manager (#16297) 2025-11-05 17:20:26 -08:00
types Gemini models: capture image_tokens and support cost_per_output_image_token in costs calculations (#16912) 2025-11-21 19:59:24 -08:00
vector_stores Fix vector store create issue (#16804) 2025-11-19 16:53:20 -08:00
__init__.py [Feat] Add github co-pilot as a new LLM API provider (#12325) 2025-07-04 13:12:16 -07:00
conftest.py Perf speed up pytest (#15951) 2025-10-27 19:43:40 -07:00
log.txt fix: fix linting errors 2025-09-27 14:25:05 -07:00
readme.md Litellm fix GitHub action testing (#11163) 2025-05-26 14:41:42 -07:00
test_acompletion_session_reuse_e2e.py [Fix] - Sessions not being shared (#15388) 2025-10-09 16:57:40 -07:00
test_add_deployment_no_master_key.py fix: remove strict master_key check in add_deployment (#16453) 2025-11-10 19:23:13 -08:00
test_aembedding_session_reuse_e2e.py [Fix] - shared session parsing and usage issue (#15440) 2025-10-10 19:26:53 -07:00
test_azure_video_router.py Litellm sameer oct staging (#15806) 2025-10-24 12:17:22 -07:00
test_claude_haiku_4_5_config.py fix haiku-4-5 bedrock configs (#16732) 2025-11-17 19:52:01 -08:00
test_constants.py tes numeric constants 2025-10-29 18:23:38 -07:00
test_container_router.py Add E2E Container API Support (#16136) 2025-11-01 14:03:51 -07:00
test_cost_calculation_log_level.py Enhance logging in cost calculation tests to ensure DEBUG level captures are accurate. Updated tests to set logger level before assertions and restored original logger level after execution. This improves reliability of log level checks in both cost and batch cost calculation tests. 2025-08-14 14:21:22 -07:00
test_cost_calculator.py Fix audio transcription cost tracking (#16478) 2025-11-19 20:29:39 -08:00
test_exception_mapping_request_attribute.py Added test 2025-09-29 21:43:31 +05:30
test_filter_out_litellm_params.py [Bug Fix] Exa Search API - ensure request params are sent to Exa AI (#15855) 2025-10-23 11:56:30 -07:00
test_groq_streaming_encoding.py [Bug] Fix: Vertex Mistral not working for streaming (#13952) 2025-08-25 17:39:40 -07:00
test_logging.py [Bug Fix] StandardLoggingPayload on cache_hits should track custom llm provider + DD LLM Obs span type (#12652) 2025-07-16 15:43:15 -07:00
test_lowest_latency_zero_tokens.py fix: remove router inefficiencies (from O(M*N) to O(1)) - 62.5% faster P99 latency (#15046) 2025-09-29 15:49:46 -07:00
test_main.py test url with format 2025-11-22 10:10:08 -08:00
test_redis.py fix: Apply max_connections configuration to Redis async client (#15797) 2025-10-22 09:19:08 -07:00
test_responses_id_security.py test_encrypt_response_id_success 2025-11-14 17:28:15 -08:00
test_router.py router fallback for unknown models (#16419) 2025-11-10 19:33:01 -08:00
test_router_google_genai.py test(gemini): Add unit tests for Google GenAI adapter 2025-09-29 18:51:35 +08:00
test_shared_session_integration.py feat: Add shared_session parameter for aiohttp ClientSession reuse 2025-09-19 01:46:53 -07:00
test_system_message_format_bug.py Ensure that function_call_prompt extends system messages following its current schema 2025-08-03 17:47:13 +02:00
test_utils.py Gemini models: capture image_tokens and support cost_per_output_image_token in costs calculations (#16912) 2025-11-21 19:59:24 -08:00
test_uuid_helper.py Fix: revert fastuuid optional dependency, always use fastuuid in .__uid helper (#14941) 2025-09-26 09:14:20 -07:00
test_video_generation.py fix(gemini-video): inherit BaseVideoConfig to enable async content response (#16875) 2025-11-21 16:01:21 -08:00

Testing for litellm/

This directory 1:1 maps the the litellm/ directory, and can only contain mocked tests.

The point of this is to:

  1. Increase test coverage of litellm/
  2. Make it easy for contributors to add tests for the litellm/ package and easily run tests without needing LLM API keys.

File name conventions

  • litellm/proxy/test_caching_routes.py maps to litellm/proxy/caching_routes.py
  • test_<filename>.py maps to litellm/<filename>.py