scriptoriumadmin/litellm
1
0
Fork 0
mirror of https://github.com/BerriAI/litellm.git synced 2026-10-02 02:11:58 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 101
litellm/tests/test_litellm
History
Benjamin Bachmann 0d481ba6fc fix(guardrails): ContentFilter streaming hook respects event_hook 
async_post_call_streaming_iterator_hook previously applied regex
masking/blocking to every streaming response regardless of the
configured event_hook / mode. This contradicts the documented
contract where mode: pre_call scans only the request.

Impact: a pre_call regex intended to detect PII in user input
silently also redacted matching text from LLM output, masking legit
content (e.g. model identifiers, technical strings that incidentally
match the regex).

Fix: gate the streaming iterator on a new _runs_on_response()
helper. The hook only scans when event_hook is post_call or
during_call (including when either appears in a list). Otherwise the
iterator yields chunks unchanged.

Tests: six new cases covering pre_call, post_call, during_call,
default (unset → pre_call), list-with-post_call, list-only-pre_call.
Existing during_call streaming tests continue to pass.
2026-04-15 15:19:59 +02:00
..
a2a_protocol fix(a2a): preserve JSON-RPC envelope for AgentCore A2A-native agents (#25092) 2026-04-03 20:25:32 -07:00
anthropic_interface/exceptions
caching Litellm ishaan march 20 (#24303) 2026-03-21 12:40:11 -07:00
completion_extras feat(openai): round-trip Responses API reasoning_items in chat completions 2026-03-27 20:25:08 +05:30
containers feat(containers): Azure routing, managed container IDs, delete response parsing (#25287) 2026-04-11 09:21:01 -07:00
enterprise Revert "[Fix] Populate _hidden_params.model_id in batch terminal-state shortcut path" 2026-03-13 12:21:39 -07:00
expected_fine_tuning_api refactor: refactor testing 2026-03-28 18:39:32 -07:00
expected_responses_api_request [Feat] Adds support for server-side compaction on the OpenAI Responses API context_management (#21058) 2026-02-12 10:00:30 -08:00
experimental_mcp_client feat(mcp): add token authentication support for MCP servers 2026-03-10 18:33:08 +02:00
google_genai Revert "QA: improve gpt-5.4 code/bugs" 2026-03-13 10:15:47 -07:00
images Root cause fix - migrate all logging update to use 1 function - for centralized kwarg updates (#23659) 2026-03-15 23:21:01 -07:00
integrations Merge branch 'main' into litellm_oss_staging_04_04_2026 2026-04-14 20:23:06 +05:30
interactions [Staging] - Ishaan March 17th (#23903) 2026-03-18 15:09:01 -07:00
litellm_core_utils Merge pull request #25647 from BerriAI/litellm_yj_apr_11 2026-04-13 17:28:38 -07:00
llms Merge pull request #25348 from BerriAI/litellm_gemini-veo-video-resolution-pricing2 2026-04-14 10:23:22 -07:00
ocr Enable local file support for OCR (#22133) 2026-02-27 10:50:02 -08:00
passthrough Litellm fix update bedrock models (#24947) 2026-04-01 19:22:54 -07:00
proxy fix(guardrails): ContentFilter streaming hook respects event_hook 2026-04-15 15:19:59 +02:00
responses feat(containers): Azure routing, managed container IDs, delete response parsing (#25287) 2026-04-11 09:21:01 -07:00
router_strategy Litellm ishaan march30 (#24887) (#25151) 2026-04-04 14:44:07 -07:00
router_utils fix(router): don't create litellm_metadata for non-Responses API calls in encrypted_content_affinity_check (#25347) 2026-04-08 21:11:19 -07:00
secret_managers address review feedback 2026-04-11 21:52:39 -07:00
test_router fix: use atomic increment-first pattern for model RPM rate limiting 2026-02-24 09:55:07 -03:00
types feat(prometheus): add 7m and 10m latency histogram buckets (#25071) 2026-04-04 18:48:39 -07:00
vector_stores
__init__.py
conftest.py build: migrate packaging, CI, and Docker from Poetry to uv (#25007) 2026-04-09 11:46:23 -07:00
log.txt
readme.md
test_a2a_registry_lookup.py
test_acompletion_session_reuse_e2e.py
test_add_deployment_no_master_key.py
test_aembedding_session_reuse_e2e.py
test_anthropic_beta_headers_filtering.py Litellm fix update bedrock models (#24947) 2026-04-01 19:22:54 -07:00
test_anthropic_skills_transformation.py Clean up skills test: remove duplicate imports, parameterize mock HTTP method (#23360) 2026-03-11 22:10:27 +05:30
test_azure_video_router.py
test_chat_ui_responses_session.py [Docs] Fix "Page Not Found" link for Anthropic endpoint (#23349) 2026-03-11 20:17:41 +05:30
test_claude_haiku_4_5_config.py
test_claude_opus_4_6_config.py Fix the Pricing changes for claude models 2026-03-27 22:44:40 +05:30
test_constants.py Fix test 2026-03-27 21:21:43 +05:30
test_container_router.py
test_cost_calculation_log_level.py fix(tests): use record.getMessage() instead of record.message for LogRecord 2026-02-18 11:46:32 -03:00
test_cost_calculator.py feat(model):add wandb model offerings to include kimi-k2.5 and minimax-m2.5 (#25409) 2026-04-11 19:46:40 -07:00
test_count_tokens_public_api.py merge: resolve conflicts between main and litellm_oss_staging_03_11_2026 2026-03-12 09:38:31 -03:00
test_deepseek_model_metadata.py fix(model-info): sync DeepSeek model metadata and add bare-name fallback (#20885) 2026-02-11 12:48:10 +05:30
test_eager_tiktoken_load.py [Fix] CI: fix GHA timeouts and uv lock --check failures 2026-04-11 09:04:49 -07:00
test_exception_exports.py fix: export PermissionDeniedError from litellm.__init__ 2026-02-11 13:39:19 +01:00
test_exception_header_preservation.py Update test to righ place 2026-02-26 13:26:51 -08:00
test_exception_mapping_request_attribute.py
test_filter_out_litellm_params.py
test_get_blog_posts.py feat: fetch blog posts from docs RSS feed instead of static JSON on GitHub 2026-03-16 15:55:16 -07:00
test_gpt_image_cost_calculator.py
test_groq_streaming_encoding.py
test_lazy_imports.py
test_litellm_params_reserved_keys.py fix(snowflake): transform tool_choice string to object format (#23268) 2026-03-11 01:41:24 +05:30
test_logging.py Litellm oss staging 04 02 2026 p1 (#25055) 2026-04-08 21:37:10 -07:00
test_lowest_latency_zero_tokens.py
test_main.py build: migrate packaging, CI, and Docker from Poetry to uv (#25007) 2026-04-09 11:46:23 -07:00
test_model_cost_aliases.py [Feat] - Ishaan main merge branch (#23596) 2026-03-14 09:40:00 -07:00
test_model_param_helper.py
test_model_response_normalization.py fix(types): remove StreamingChoices from ModelResponse, use ModelResponseStream 2026-02-20 17:47:42 -03:00
test_nested_drop_params.py
test_project_alias_tracking.py feat(proxy): add project_alias tracking through callback metadata pipeline 2026-03-23 10:44:17 -07:00
test_project_tags_pydantic.py fix: req changes 2026-02-27 13:33:34 +05:30
test_redis.py Litellm ishaan march23 - MCP Toolsets + GCP Caching fix (#25146) (#25155) 2026-04-04 16:23:21 -07:00
test_register_model_custom_pricing.py test: fix misleading precedence test per review feedback 2026-03-02 08:31:33 +00:00
test_responses_api_bridge_non_stream.py
test_responses_id_security.py Fix responses ID security test for new request_cache parameter 2026-03-04 11:29:51 -03:00
test_router.py fix(router): pass custom_llm_provider to get_llm_provider for unprefixed model names (#25334) 2026-04-08 21:27:13 -07:00
test_router_google_genai.py
test_router_model_cost_isolation.py [Fix] prevent shared backend model key from being polluted by per-deployment custom pricing (#20679) 2026-02-09 19:38:44 -08:00
test_router_order_fallback.py Merge pull request #24611 from Sameerlite/Sameerlite/order-fallback2 2026-03-27 20:15:30 +05:30
test_router_per_deployment_num_retries.py
test_router_redis_init.py
test_router_retry_non_retryable_errors.py Reapply "feat: add model_cost aliases expansion support" 2026-03-12 13:36:57 -03:00
test_router_silent_experiment.py Update tests/test_litellm/test_router_silent_experiment.py 2026-03-14 16:47:56 +05:30
test_secret_redaction.py Litellm ishaan march30 (#24887) (#25151) 2026-04-04 14:44:07 -07:00
test_service_logger.py fix(proxy): fix master key rotation Prisma validation errors (#21330) 2026-02-16 15:13:05 -08:00
test_setup_wizard.py test: test 2026-03-28 19:17:38 -07:00
test_shared_session_integration.py
test_ssl_verify_unit.py BUMP Enterprise PIP 2026-02-14 13:40:48 -08:00
test_stream_chunk_builder_annotations.py fix: merge annotations from all streaming chunks in stream_chunk_builder 2026-03-15 14:20:45 +05:30
test_streaming_connection_cleanup.py fix: add debug logging to stream cleanup, improve tests 2026-02-14 17:31:39 -08:00
test_system_message_format_bug.py
test_utils.py fix tests and mypy 2026-04-08 22:30:39 +05:30
test_uuid_helper.py
test_video_generation.py feat(gemini): Veo Lite pricing, size→resolution, usage video_resolution for cost tiers 2026-04-08 19:50:50 +05:30
test_xai_responses_auto_routing.py

readme.md

Testing for litellm/

This directory 1:1 maps the the litellm/ directory, and can only contain mocked tests.

The point of this is to:

  1. Increase test coverage of litellm/
  2. Make it easy for contributors to add tests for the litellm/ package and easily run tests without needing LLM API keys.

File name conventions

  • litellm/proxy/test_caching_routes.py maps to litellm/proxy/caching_routes.py
  • test_<filename>.py maps to litellm/<filename>.py