scriptoriumadmin/litellm
1
0
Fork 0
mirror of https://github.com/BerriAI/litellm.git synced 2026-09-15 23:31:29 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 165
litellm/tests/test_litellm
History
Exact
mubashir1osmani de74191acb fix(guardrails): expand os.environ/ references in every litellm_params field 
initialize_guardrail only expanded os.environ/ for api_key and api_base, so
every other field kept the literal reference string. A guardrail registered
with aws_region_name="os.environ/AWS_REGION" handed boto3 the string
"os.environ/AWS_REGION", which failed the request with "Invalid AWS region
format: 'os.environ/AWS_REGION'. Region names must contain only lowercase
letters, digits, and hyphens." The same silently broke aws_access_key_id,
aws_secret_access_key, the presidio analyzer/anonymizer bases, and the rest of
the credential fields, none of which are reachable through api_key/api_base.

Resolve every string field that starts with os.environ/ instead of two named
ones. A reference that resolves to nothing now becomes None rather than the
string "None", so the downstream client falls back to its own credential chain
(pod identity, for example) instead of authenticating with a literal "None".

Reproduced against the live stage proxy before the fix: registering a bedrock
guardrail with the three os.environ/ AWS params returned 500 with the invalid
region message, while the same guardrail without them blocked correctly.
2026-07-24 12:56:56 -07:00
..
a2a_protocol fix(responses): stop scheduling sync success_handler concurrently with async_success_handler (#32239) 2026-07-07 09:13:50 -07:00
anthropic_interface fix(rust): route agentic-completion-hook /messages requests to Python for all stream modes (#34126) 2026-07-22 00:36:47 +00:00
batches Revert "chore(ci): sync litellm_internal_staging into daily OSS branch (#33337)" (#33339) 2026-07-14 19:32:25 -07:00
caching fix(cache): make in-memory and disk cache increments atomic (#34013) 2026-07-20 15:51:01 -07:00
completion_extras
containers
enterprise refactor(ui): point invitation links at the dedicated /onboarding route (#30857) 2026-07-08 19:47:05 -07:00
expected_fine_tuning_api
expected_responses_api_request test(responses): replace perma-skip azure shell e2e with offline coverage (#32444) 2026-07-08 10:01:41 -07:00
experimental_mcp_client fix(mcp): sanitize Anthropic tool schemas and stop encoding gateway names 2026-07-17 10:33:28 -07:00
google_genai chore: litellm oss staging 250626 (#31305) 2026-06-25 21:00:28 -07:00
images
integrations feat(guardrails): add only_scan_new_messages for per-session incremental scanning (#33278) 2026-07-22 09:56:59 -07:00
interactions fix(interactions): add queued to the Interaction status enum (#34135) 2026-07-21 12:36:21 -07:00
litellm_core_utils fix(logging): stop scheduling sync failure_handler concurrently with async_failure_handler (#34306) 2026-07-24 11:06:55 -07:00
llms fix(vertex): incrementally parse accumulated Gemini stream JSON to prevent multi-value wedge (#34320) 2026-07-24 11:07:02 -07:00
models
ocr test: litellm fix failing tests (#32577) 2026-07-09 13:54:45 -07:00
passthrough fix(passthrough): stream non-sse passthrough responses instead of buffering in memory (#32386) 2026-07-07 20:51:15 -07:00
proxy fix(guardrails): expand os.environ/ references in every litellm_params field 2026-07-24 12:56:56 -07:00
rag fix(rag): track LLM completion usage and spend for /v1/rag/query (#32438) 2026-07-17 17:45:27 +00:00
realtime_api fix(realtime): preserve nested transcription model and session-first model priority 2026-07-03 14:21:22 -07:00
repositories fix(team): make team member add atomic to prevent concurrent-add member loss (#34185) 2026-07-22 21:47:22 +00:00
rerank_api fix(rerank): log optional_rerank_params at debug to stop leaking request content (#32533) 2026-07-08 18:48:03 -07:00
responses fix(cost_tracking): map cache_write_tokens on Responses API usage path 2026-07-23 19:07:08 -07:00
router_strategy feat(complexity-router): add return_raw_model_name toggle for response model field (#33875) 2026-07-18 19:24:56 -07:00
router_utils fix(router): take the lowest minimum across a model group, not the highest 2026-07-16 19:53:10 -07:00
sandbox
secret_managers fix(proxy): harden secret name validation for external secret manager integrations (LIT-4201) (#32092) 2026-07-08 10:36:00 -07:00
test_router fix(logging): stop pinning large request payloads past request end (#33455) 2026-07-15 15:28:37 -07:00
types perf(core): fast-path SafeAttributeModel.__delattr__ for declared fields (#33993) 2026-07-21 00:06:56 +00:00
vector_stores
videos test(videos): add 1:1 test file scaffold for videos component paths (#30631) 2026-06-29 09:12:51 +05:30
__init__.py
conftest.py
log.txt
readme.md
test__types.py
test_a2a_registry_lookup.py
test_acompletion_session_reuse_e2e.py
test_add_deployment_no_master_key.py
test_aembedding_session_reuse_e2e.py
test_anthropic_beta_headers_filtering.py
test_anthropic_skills_transformation.py
test_anthropic_sonnet_1hr_cache_pricing.py
test_audio_transcription_rust_bridge.py perf(bedrock): audio transcription via rust core (py->rust bridge) (#33990) 2026-07-20 14:09:41 -07:00
test_azure_ad_token_credential_resolution.py
test_azure_video_router.py
test_batch_completion_models_all_responses.py
test_bedrock_anthropic_1hr_cache_pricing.py
test_bedrock_usgov_haiku_1hr_cache.py
test_bedrock_usgov_pricing.py
test_budget_ratchet_check.py refactor(lint): collapse type/lint budgets to a single per-rule limit (#31883) 2026-07-01 18:12:35 +03:00
test_chat_ui_responses_session.py
test_check_licenses.py
test_check_type_discipline.py chore(lint): zero out crash-class pyright rules and ban new type: ignore comments (#32152) 2026-07-04 16:56:12 -07:00
test_circleci_path_filter.py ci: gate CircleCI jobs on changed paths (#32080) 2026-07-04 19:15:08 -07:00
test_claude_fable_5_config.py Revert "chore(ci): sync litellm_internal_staging into daily OSS branch (#33337)" (#33339) 2026-07-14 19:32:25 -07:00
test_claude_haiku_4_5_config.py
test_claude_opus_4_6_config.py
test_claude_opus_4_8_config.py Revert "chore(ci): sync litellm_internal_staging into daily OSS branch (#33337)" (#33339) 2026-07-14 19:32:25 -07:00
test_claude_opus_5_config.py feat(anthropic): add Claude Opus 5 2026-07-24 10:43:49 -07:00
test_claude_sonnet_4_6_config.py
test_claude_sonnet_5_config.py fix(model_prices): apply claude-sonnet-5 introductory pricing through 2026-08-31 (#31917) 2026-07-01 17:45:57 -07:00
test_cloudflare_workers_ai_model_metadata.py
test_command_r7b_pricing.py
test_completion_timeout_resolution.py fix(router): honor litellm_settings.request_timeout as an independent per-attempt timeout (#31119) 2026-06-23 14:22:54 -07:00
test_compression.py
test_constants.py
test_container_router.py
test_cost_calculation_log_level.py
test_cost_calculator.py fix(cost_calculator): sum mirrored cache token fields once in combine_usage_objects 2026-07-23 19:07:09 -07:00
test_count_tokens_public_api.py
test_dashscope_image_generation.py
test_deepseek_model_metadata.py
test_dockerfile_non_root.py
test_eager_tiktoken_load.py
test_exception_exports.py
test_exception_header_preservation.py
test_exception_mapping_request_attribute.py
test_filter_out_litellm_params.py
test_get_blog_posts.py
test_git_hooks.py
test_github_close_low_quality_prs.py
test_github_review_gate.py
test_github_triage_with_llm.py
test_github_triage_workflows.py
test_gpt_5_5_model_metadata.py
test_gpt_5_6_model_metadata.py Revert "chore(ci): sync litellm_internal_staging into daily OSS branch (#33337)" (#33339) 2026-07-14 19:32:25 -07:00
test_gpt_image_cost_calculator.py chore: litellm oss staging (#31185) 2026-06-26 09:17:44 -07:00
test_gpt_realtime_mode.py test(pricing): pin the realtime mode assertion to the bundled cost map (#33806) 2026-07-18 02:52:55 +00:00
test_groq_streaming_encoding.py
test_guardrail_exception_status_codes.py
test_lazy_imports.py
test_litellm_params_reserved_keys.py
test_logging.py
test_lowest_latency_zero_tokens.py
test_main.py Revert "chore(ci): sync litellm_internal_staging into daily OSS branch (#33337)" (#33339) 2026-07-14 19:32:25 -07:00
test_main_module_header.py
test_mistral_medium_3_5_model_metadata.py fix(cost-map): retarget mistral-medium-latest to Medium 3.5 and add date-pinned aliases (#31373) 2026-06-25 18:27:18 -07:00
test_model_block_unblock.py
test_model_cost_aliases.py
test_model_param_helper.py
test_model_response_normalization.py
test_muse_spark_1_1_model_metadata.py Revert "chore(ci): sync litellm_internal_staging into daily OSS branch (#33337)" (#33339) 2026-07-14 19:32:25 -07:00
test_nested_drop_params.py
test_openai_embedding_encoding_format_default.py
test_prisma_generate_if_needed.py perf(lint): skip and cache base gate passes, parallelize make lint, skip redundant prisma generate (#32000) 2026-07-02 19:24:00 -07:00
test_project_alias_tracking.py
test_project_tags_pydantic.py
test_rag_openai_ingestion.py
test_rate_limit_error_unification.py
test_redact_string_in_error_paths.py
test_redis.py fix(redis): honor ssl value instead of key presence when building async connection pool (#32590) 2026-07-16 13:38:01 -07:00
test_register_model_custom_pricing.py fix(utils): resolve bedrock regional inference profiles to regional pricing in get_model_info (LIT-4056) (#32389) 2026-07-07 20:49:03 -07:00
test_register_model_zero_cost_persistence.py
test_responses_api_bridge_non_stream.py Revert "chore(ci): sync litellm_internal_staging into daily OSS branch (#33337)" (#33339) 2026-07-14 19:32:25 -07:00
test_responses_id_security.py fix(responses_id_security): decrypt response ids for input_items follow-ups (#32269) 2026-07-06 14:03:15 -07:00
test_responses_streaming_container_ownership.py
test_retrieve_batch_bedrock_dispatch.py
test_router.py Merge remote-tracking branch 'origin/litellm_internal_staging' into litellm_lit_4162_bedrock_batch_tags 2026-07-20 14:45:02 -07:00
test_router_block_helpers.py
test_router_exception_redaction.py fix(router): mask provider credentials embedded in fallback error messages (#32083) 2026-07-03 18:48:06 -07:00
test_router_google_genai.py
test_router_model_cost_isolation.py fix(router): propagate capability flags to shared backend cost map key 2026-07-20 16:46:30 -07:00
test_router_order_fallback.py
test_router_per_deployment_num_retries.py fix(router): honor request-level num_retries over global litellm_settings.num_retries (#34124) 2026-07-21 21:32:34 +00:00
test_router_redis_init.py
test_router_retry_non_retryable_errors.py
test_router_retry_policy_update.py fix(proxy): tighten role gating on /get/config/callbacks response (#31745) 2026-07-01 17:58:31 -07:00
test_router_silent_experiment.py
test_router_streaming_fallback_metadata.py
test_router_weighted_failover.py
test_ruff_strict_gate.py perf(lint): skip and cache base gate passes, parallelize make lint, skip redundant prisma generate (#32000) 2026-07-02 19:24:00 -07:00
test_secret_redaction.py fix(key management): enforce minimum custom key length and mask short keys in key_name (#33462) 2026-07-15 18:27:38 -07:00
test_service_logger.py
test_setup_wizard.py
test_shared_session_integration.py
test_ssl_verify_unit.py
test_stream_chunk_builder_annotations.py
test_streaming_connection_cleanup.py
test_system_message_format_bug.py
test_thinking_enabled.py
test_triage_rollout_heads_up.py
test_type_check_gate.py perf(lint): skip and cache base gate passes, parallelize make lint, skip redundant prisma generate (#32000) 2026-07-02 19:24:00 -07:00
test_type_discipline_gate.py refactor(lint): collapse type/lint budgets to a single per-rule limit (#31883) 2026-07-01 18:12:35 +03:00
test_utils.py fix(cost_tracking): map cache_write_tokens on Responses API usage path 2026-07-23 19:07:08 -07:00
test_utils_module_docstring.py
test_uuid_helper.py
test_vcr_safe_body_matcher.py
test_video_generation.py
test_xai_grok_4_3_model_metadata.py
test_xai_responses_auto_routing.py

readme.md

Testing for litellm/

This directory 1:1 maps the the litellm/ directory, and can only contain mocked tests.

The point of this is to:

  1. Increase test coverage of litellm/
  2. Make it easy for contributors to add tests for the litellm/ package and easily run tests without needing LLM API keys.

File name conventions

  • litellm/proxy/test_caching_routes.py maps to litellm/proxy/caching_routes.py
  • test_<filename>.py maps to litellm/<filename>.py