scriptoriumadmin/litellm
1
0
Fork 0
mirror of https://github.com/BerriAI/litellm.git synced 2026-09-14 23:21:35 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 102
litellm/tests/test_litellm
History
devin-ai-integration[bot] 78f512f5a8
fix(proxy): share CLI SSO login sessions across workers without enable_redis_auth_cache (#33261) 
* fix(proxy): share CLI SSO login sessions across workers without enable_redis_auth_cache

* fix(proxy): make CLI SSO flow state redis-authoritative across workers

The CLI SSO flow is stored in a DualCache whose get_cache is memory-first, so
the worker that served /sso/cli/start keeps serving its stale in-memory flow and
never observes the sso_complete/session_data update another worker writes during
the OAuth callback. Attaching Redis alone is not enough; poll on the original
worker returns pending forever.

Read and write the flow directly through the attached Redis backend when present
so every worker sees the same authoritative state, falling back to the in-memory
DualCache only when no Redis is configured.

* fix(proxy): serialize CLI SSO flow as JSON for the redis round trip

RedisCache stores values via str(value) and parses reads with
json.loads then ast.literal_eval. The completed flow contains a
LitellmUserRoles enum in session_data.user_role, whose repr is not a
parseable literal, so any worker reading the completed flow from redis
raised SyntaxError and returned 400 "CLI login session not found".
Writing the flow as json.dumps makes the round trip lossless (the enum
is a str subclass) and fails loudly at write time if a non-serializable
value is ever added to the flow.

* fix(proxy): point CLI SSO session-not-found hint at configuring Redis

The error message and warning still told users to set enable_redis_auth_cache,
but the CLI SSO session cache now gets Redis unconditionally whenever one is
configured, so that flag no longer affects CLI login

---------

Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: ryan-crabbe-berri <ryan@berri.ai>
(cherry picked from commit 17a83aa896)
2026-07-25 11:18:19 -07:00
..
a2a_protocol fix(responses): stop scheduling sync success_handler concurrently with async_success_handler (#32239) 2026-07-07 09:13:50 -07:00
anthropic_interface feat(messages): route Azure Anthropic /messages through Rust behind rust:true (#33616) 2026-07-18 11:56:25 -07:00
batches Revert "chore(ci): sync litellm_internal_staging into daily OSS branch (#33337)" (#33339) 2026-07-14 19:32:25 -07:00
caching fix(proxy): stop stale auth cache re-publish so key updates and deletes propagate across replicas (#33565) 2026-07-16 15:00:33 -07:00
completion_extras chore(oss): litellm oss staging 150626 (#30463) 2026-06-16 12:06:41 -07:00
containers fix(containers): record ownership for service-account keys + fix Prisma Json serialization (#28990) 2026-05-27 21:00:07 -07:00
enterprise refactor(ui): point invitation links at the dedicated /onboarding route (#30857) 2026-07-08 19:47:05 -07:00
expected_fine_tuning_api
expected_responses_api_request test(responses): replace perma-skip azure shell e2e with offline coverage (#32444) 2026-07-08 10:01:41 -07:00
experimental_mcp_client Revert "chore(ci): sync litellm_internal_staging into daily OSS branch (#33337)" (#33339) 2026-07-14 19:32:25 -07:00
google_genai chore: litellm oss staging 250626 (#31305) 2026-06-25 21:00:28 -07:00
images fix(image_edit): read vertex_project/location from litellm_params in Imagen get_complete_url 2026-04-21 15:03:40 +05:30
integrations chore(release): backport #33810, #33733 to rc/1.94.0 and bump litellm-proxy-extras to 0.4.79.post1 (#34215) 2026-07-21 19:53:06 -07:00
interactions fix(responses): stop scheduling sync success_handler concurrently with async_success_handler (#32239) 2026-07-07 09:13:50 -07:00
litellm_core_utils fix(logging): classify async anthropic_messages and generate_content as async (#33589) 2026-07-16 20:56:47 -07:00
llms chore(release): backport #33810, #33733 to rc/1.94.0 and bump litellm-proxy-extras to 0.4.79.post1 (#34215) 2026-07-21 19:53:06 -07:00
models feat(litellm): add models and repository layers (#29686) 2026-06-06 20:59:33 -07:00
ocr test: litellm fix failing tests (#32577) 2026-07-09 13:54:45 -07:00
passthrough fix(passthrough): stream non-sse passthrough responses instead of buffering in memory (#32386) 2026-07-07 20:51:15 -07:00
proxy fix(proxy): share CLI SSO login sessions across workers without enable_redis_auth_cache (#33261) 2026-07-25 11:18:19 -07:00
rag fix(rag): track LLM completion usage and spend for /v1/rag/query (#32438) 2026-07-17 17:45:27 +00:00
realtime_api fix(realtime): preserve nested transcription model and session-first model priority 2026-07-03 14:21:22 -07:00
repositories fix(proxy): count only active users toward license seat limit (#31227) 2026-06-29 18:01:02 -07:00
rerank_api fix(rerank): log optional_rerank_params at debug to stop leaking request content (#32533) 2026-07-08 18:48:03 -07:00
responses Merge origin/litellm_internal_staging into litellm_mcp_aggregate_outcomes 2026-07-17 10:19:36 -07:00
router_strategy feat(complexity-router): user-triggered escalation keywords (#33656) 2026-07-17 10:24:59 -07:00
router_utils fix(router): take the lowest minimum across a model group, not the highest 2026-07-16 19:53:10 -07:00
sandbox feat: add opensandbox sandbox provider (#31024) 2026-06-23 09:05:13 -07:00
secret_managers fix(proxy): harden secret name validation for external secret manager integrations (LIT-4201) (#32092) 2026-07-08 10:36:00 -07:00
test_router fix(logging): stop pinning large request payloads past request end (#33455) 2026-07-15 15:28:37 -07:00
types fix(anthropic): drop unsignable thinking blocks and allow null signature in logging (LIT-4007) (#31654) 2026-06-29 21:01:03 -07:00
vector_stores style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
videos test(videos): add 1:1 test file scaffold for videos component paths (#30631) 2026-06-29 09:12:51 +05:30
__init__.py
conftest.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
log.txt
readme.md
test__types.py Litellm OSS Staging 010626 (#29422) 2026-06-01 21:42:51 -07:00
test_a2a_registry_lookup.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
test_acompletion_session_reuse_e2e.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
test_add_deployment_no_master_key.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
test_aembedding_session_reuse_e2e.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
test_anthropic_beta_headers_filtering.py chore(oss): litellm oss staging 120626 (#30292) 2026-06-12 09:49:25 -07:00
test_anthropic_skills_transformation.py fix: encode upstream URL path identifiers 2026-04-29 22:02:39 -07:00
test_anthropic_sonnet_1hr_cache_pricing.py chore: litellm oss staging160626 (#30527) 2026-06-16 18:23:13 -07:00
test_azure_ad_token_credential_resolution.py chore(oss): litellm oss staging 150626 (#30463) 2026-06-16 12:06:41 -07:00
test_azure_video_router.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
test_batch_completion_models_all_responses.py test(batches): move orphan tests into tests/test_litellm for CI coverage (#30510) 2026-06-16 10:20:59 -07:00
test_bedrock_anthropic_1hr_cache_pricing.py Litellm oss staging 030626 (#29578) 2026-06-03 11:01:51 -07:00
test_bedrock_usgov_haiku_1hr_cache.py Litellm OSS Staging 010626 (#29422) 2026-06-01 21:42:51 -07:00
test_bedrock_usgov_pricing.py Litellm OSS Staging 010626 (#29422) 2026-06-01 21:42:51 -07:00
test_budget_ratchet_check.py refactor(lint): collapse type/lint budgets to a single per-rule limit (#31883) 2026-07-01 18:12:35 +03:00
test_chat_ui_responses_session.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
test_check_licenses.py fix(check_licenses): read PEP 639 license-expression metadata (#28529) 2026-05-22 11:22:38 -07:00
test_check_type_discipline.py chore(lint): zero out crash-class pyright rules and ban new type: ignore comments (#32152) 2026-07-04 16:56:12 -07:00
test_circleci_path_filter.py ci: gate CircleCI jobs on changed paths (#32080) 2026-07-04 19:15:08 -07:00
test_claude_fable_5_config.py Revert "chore(ci): sync litellm_internal_staging into daily OSS branch (#33337)" (#33339) 2026-07-14 19:32:25 -07:00
test_claude_haiku_4_5_config.py feat(anthropic): add Claude Opus 4.8 and prune reasoning-effort flags (#29238) 2026-05-28 18:50:33 -07:00
test_claude_opus_4_6_config.py feat(anthropic): add Claude Opus 4.8 and prune reasoning-effort flags (#29238) 2026-05-28 18:50:33 -07:00
test_claude_opus_4_8_config.py Revert "chore(ci): sync litellm_internal_staging into daily OSS branch (#33337)" (#33339) 2026-07-14 19:32:25 -07:00
test_claude_sonnet_4_6_config.py feat(anthropic): add Claude Opus 4.8 and prune reasoning-effort flags (#29238) 2026-05-28 18:50:33 -07:00
test_claude_sonnet_5_config.py fix(model_prices): apply claude-sonnet-5 introductory pricing through 2026-08-31 (#31917) 2026-07-01 17:45:57 -07:00
test_cloudflare_workers_ai_model_metadata.py feat(cloudflare): add current Workers AI text-generation models to the cost map (#31051) 2026-06-23 10:44:37 -07:00
test_command_r7b_pricing.py chore: litellm oss 170626 (#30637) 2026-06-17 21:11:12 -07:00
test_completion_timeout_resolution.py fix(router): honor litellm_settings.request_timeout as an independent per-attempt timeout (#31119) 2026-06-23 14:22:54 -07:00
test_compression.py Prompt Compression - add it to the proxy (#25729) 2026-04-20 15:08:00 -07:00
test_constants.py
test_container_router.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
test_cost_calculation_log_level.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
test_cost_calculator.py Revert "chore(ci): sync litellm_internal_staging into daily OSS branch (#33337)" (#33339) 2026-07-14 19:32:25 -07:00
test_count_tokens_public_api.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
test_dashscope_image_generation.py fix(dashscope): fail fast on image generation API errors 2026-04-23 18:41:01 +05:30
test_deepseek_model_metadata.py Revert "Merge pull request #16590 from Chesars/refactor/remove-backup-file-dry-principle" 2026-04-25 17:10:41 -03:00
test_dockerfile_non_root.py fix(docker.non_root): use numeric UID 65534 for K8s runAsNonRoot (#26268) 2026-04-22 18:00:04 -07:00
test_eager_tiktoken_load.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
test_exception_exports.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
test_exception_header_preservation.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
test_exception_mapping_request_attribute.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
test_filter_out_litellm_params.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
test_get_blog_posts.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
test_git_hooks.py chore(hooks): enforce Conventional Commits and Conventional Branches (#30174) 2026-06-11 10:00:23 -07:00
test_github_close_low_quality_prs.py feat(agent-shin): automated PR/issue triage, low-quality auto-close, and review-gate label lifecycle (#30433) 2026-06-17 20:42:27 -07:00
test_github_review_gate.py feat(agent-shin): automated PR/issue triage, low-quality auto-close, and review-gate label lifecycle (#30433) 2026-06-17 20:42:27 -07:00
test_github_triage_with_llm.py feat(agent-shin): automated PR/issue triage, low-quality auto-close, and review-gate label lifecycle (#30433) 2026-06-17 20:42:27 -07:00
test_github_triage_workflows.py fix(test): drop references to removed Agent Shin workflows (#30791) 2026-06-18 15:47:51 -07:00
test_gpt_5_5_model_metadata.py chore: litellm oss staging160626 (#30527) 2026-06-16 18:23:13 -07:00
test_gpt_5_6_model_metadata.py Revert "chore(ci): sync litellm_internal_staging into daily OSS branch (#33337)" (#33339) 2026-07-14 19:32:25 -07:00
test_gpt_image_cost_calculator.py chore: litellm oss staging (#31185) 2026-06-26 09:17:44 -07:00
test_gpt_realtime_mode.py test(pricing): pin the realtime mode assertion to the bundled cost map (#33806) 2026-07-18 02:52:55 +00:00
test_groq_streaming_encoding.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
test_guardrail_exception_status_codes.py Litellm oss staging (#28161) 2026-05-18 16:27:44 -07:00
test_lazy_imports.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
test_litellm_params_reserved_keys.py
test_logging.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
test_lowest_latency_zero_tokens.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
test_main.py Revert "chore(ci): sync litellm_internal_staging into daily OSS branch (#33337)" (#33339) 2026-07-14 19:32:25 -07:00
test_main_module_header.py Add main module header comment 2026-05-06 00:26:17 +00:00
test_mistral_medium_3_5_model_metadata.py fix(cost-map): retarget mistral-medium-latest to Medium 3.5 and add date-pinned aliases (#31373) 2026-06-25 18:27:18 -07:00
test_model_block_unblock.py feat: litellm oss 110626 (#30202) 2026-06-11 22:30:26 -07:00
test_model_cost_aliases.py
test_model_param_helper.py style: black format test_model_param_helper.py 2026-04-15 18:18:52 -07:00
test_model_response_normalization.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
test_muse_spark_1_1_model_metadata.py Revert "chore(ci): sync litellm_internal_staging into daily OSS branch (#33337)" (#33339) 2026-07-14 19:32:25 -07:00
test_nested_drop_params.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
test_openai_embedding_encoding_format_default.py fix(embeddings): allow omitting encoding_format via env sentinel none 2026-05-01 23:03:13 +05:30
test_prisma_generate_if_needed.py perf(lint): skip and cache base gate passes, parallelize make lint, skip redundant prisma generate (#32000) 2026-07-02 19:24:00 -07:00
test_project_alias_tracking.py
test_project_tags_pydantic.py
test_rag_openai_ingestion.py chore: litellm oss staging (#30745) 2026-06-18 13:55:35 -07:00
test_rate_limit_error_unification.py feat: standardize rate limit errors with category, rate_limit_type, model, and llm_provider fields (#27687) 2026-06-06 17:50:29 -07:00
test_redact_string_in_error_paths.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
test_redis.py fix(redis): honor ssl value instead of key presence when building async connection pool (#32590) 2026-07-16 13:38:01 -07:00
test_register_model_custom_pricing.py fix(utils): resolve bedrock regional inference profiles to regional pricing in get_model_info (LIT-4056) (#32389) 2026-07-07 20:49:03 -07:00
test_register_model_zero_cost_persistence.py chore(oss): litellm oss staging 150626 (#30463) 2026-06-16 12:06:41 -07:00
test_responses_api_bridge_non_stream.py Revert "chore(ci): sync litellm_internal_staging into daily OSS branch (#33337)" (#33339) 2026-07-14 19:32:25 -07:00
test_responses_id_security.py fix(responses_id_security): decrypt response ids for input_items follow-ups (#32269) 2026-07-06 14:03:15 -07:00
test_responses_streaming_container_ownership.py chore(oss): litellm oss staging 150626 (#30463) 2026-06-16 12:06:41 -07:00
test_retrieve_batch_bedrock_dispatch.py feat(batch-job): bedrock batch model invocation job retrieval (#26834) 2026-05-11 13:22:26 -07:00
test_router.py chore(release): backport #33810, #33733 to rc/1.94.0 and bump litellm-proxy-extras to 0.4.79.post1 (#34215) 2026-07-21 19:53:06 -07:00
test_router_block_helpers.py feat: litellm oss 110626 (#30202) 2026-06-11 22:30:26 -07:00
test_router_exception_redaction.py fix(router): mask provider credentials embedded in fallback error messages (#32083) 2026-07-03 18:48:06 -07:00
test_router_google_genai.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
test_router_model_cost_isolation.py fix(main): stop per-request custom pricing from clobbering shared model_cost pricing (#32163) 2026-07-07 10:25:31 -07:00
test_router_order_fallback.py chore: litellm oss staging160626 (#30527) 2026-06-16 18:23:13 -07:00
test_router_per_deployment_num_retries.py fix(router): guard num_retries=None in async_function_with_retries (#30036) 2026-06-23 10:16:15 -07:00
test_router_redis_init.py
test_router_retry_non_retryable_errors.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
test_router_retry_policy_update.py fix(proxy): tighten role gating on /get/config/callbacks response (#31745) 2026-07-01 17:58:31 -07:00
test_router_silent_experiment.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
test_router_streaming_fallback_metadata.py chore: litellm oss staging (#30968) 2026-06-23 07:31:44 -07:00
test_router_weighted_failover.py feat: add weighted-routing failover (#27980) 2026-05-15 17:28:54 +00:00
test_ruff_strict_gate.py perf(lint): skip and cache base gate passes, parallelize make lint, skip redundant prisma generate (#32000) 2026-07-02 19:24:00 -07:00
test_secret_redaction.py fix(key management): enforce minimum custom key length and mask short keys in key_name (#33462) 2026-07-15 18:27:38 -07:00
test_service_logger.py fix: missing span for guardrail passthrough (#29552) 2026-06-03 01:25:15 +00:00
test_setup_wizard.py
test_shared_session_integration.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
test_ssl_verify_unit.py Litellm OSS Staging (#29161) 2026-06-01 21:22:35 -07:00
test_stream_chunk_builder_annotations.py
test_streaming_connection_cleanup.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
test_system_message_format_bug.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
test_thinking_enabled.py Litellm oss staging 250526 (#28770) 2026-05-26 11:57:39 -07:00
test_triage_rollout_heads_up.py feat(agent-shin): automated PR/issue triage, low-quality auto-close, and review-gate label lifecycle (#30433) 2026-06-17 20:42:27 -07:00
test_type_check_gate.py perf(lint): skip and cache base gate passes, parallelize make lint, skip redundant prisma generate (#32000) 2026-07-02 19:24:00 -07:00
test_type_discipline_gate.py refactor(lint): collapse type/lint budgets to a single per-rule limit (#31883) 2026-07-01 18:12:35 +03:00
test_utils.py fix(fireworks_ai): correct glm-5p2 prompt-cache read price to $0.14/1M 2026-07-17 17:57:36 -07:00
test_utils_module_docstring.py Add utils module docstring 2026-05-06 00:42:49 +00:00
test_uuid_helper.py
test_vcr_safe_body_matcher.py test: stabilize batch VCR coverage and stop live upload/network leaks (#29477) 2026-06-02 16:11:52 -07:00
test_video_generation.py fix(vertex-ai): use DB credentials in video handlers + implement Veo video edit (#29098) 2026-05-28 11:45:41 -07:00
test_xai_grok_4_3_model_metadata.py feat(xai): add grok-4.3 and grok-4.3-latest to model_prices_and_conte… (#27154) 2026-05-07 09:06:56 -07:00
test_xai_responses_auto_routing.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00

readme.md

Testing for litellm/

This directory 1:1 maps the the litellm/ directory, and can only contain mocked tests.

The point of this is to:

  1. Increase test coverage of litellm/
  2. Make it easy for contributors to add tests for the litellm/ package and easily run tests without needing LLM API keys.

File name conventions

  • litellm/proxy/test_caching_routes.py maps to litellm/proxy/caching_routes.py
  • test_<filename>.py maps to litellm/<filename>.py