scriptoriumadmin/litellm
1
0
Fork 0
mirror of https://github.com/BerriAI/litellm.git synced 2026-10-09 03:18:44 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 108
litellm/tests/test_litellm
History
devin-ai-integration[bot] a3196907e4
feat(otel): emit gen_ai.conversation.id from the caller's session id on v2 LLM spans (#42486) 
* feat(otel): emit gen_ai.conversation.id from the caller's session id on v2 LLM spans

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(otel): keep the caller's header session under missing_session_id: generate and read replayed payload session ids

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(otel): drop only the proxy-minted session id so a caller id on the other metadata key survives

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(otel): keep a replayed session id hidden when it only echoes the payload trace id

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(otel): keep a replayed session id even when the payload trace id fell back to it

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(otel): stop reading the replayed payload's session id, the generated marker does not survive replay

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(integration): audit gen_ai.conversation.id on otel v2 spans through a real proxy, sink and postgres

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(integration): keep otel conversation rigs alive for the whole session so shuffled shards do not reboot the proxy per test

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(otel): stop the audit rig proxies from probing sibling test peers for model info

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(otel): record accepted OTLP batches in the sink instead of mutating the collector

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(otel): guard the accepted batch deque so snapshots cannot race sink appends

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

---------

Co-authored-by: mrinal <mrinal@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: yucheng <yucheng@berri.ai>
2026-09-23 00:49:29 -07:00
..
a2a_protocol fix(a2a): send message/stream for Bedrock AgentCore streaming requests 2026-09-21 13:39:32 +00:00
batches
caching fix(caching): keep embedding cache hits aligned with request inputs (#42571) 2026-09-22 17:14:44 -05:00
chat_completions
completion_extras fix(edenai): restore unrelated files the squashed PR commit had reverted 2026-09-21 19:42:37 +00:00
containers
endpoints
enterprise
expected_fine_tuning_api
expected_responses_api_request
experimental_mcp_client fix(mcp): restore legacy SSE and bounded cancellation cleanup (#42382) 2026-09-22 11:27:50 -07:00
fixtures/together_ai_sync
google_genai
images
integrations feat(otel): emit gen_ai.conversation.id from the caller's session id on v2 LLM spans (#42486) 2026-09-23 00:49:29 -07:00
interactions
litellm_core_utils fix: answer get_api_base for github_copilot and chatgpt without running the login flow (#42602) 2026-09-22 17:23:44 -07:00
llms chore(prices): sync AWS Bedrock prices and sources from the AWS price list (#42632) 2026-09-22 20:24:28 -07:00
messages
ocr
passthrough
proxy fix(proxy): publish auth cache invalidations in the background so a wedged coordination Redis cannot stall user updates (#42534) 2026-09-22 23:33:32 -07:00
rag
rerank_api fix: answer get_api_base for github_copilot and chatgpt without running the login flow (#42602) 2026-09-22 17:23:44 -07:00
responses fix(mcp): preserve discovery attribution and sanitize logging headers (#42541) 2026-09-22 14:26:48 -07:00
router_strategy feat(router): add group-scoped priority routing strategy (#42378) 2026-09-22 13:09:38 -07:00
router_utils feat(ui): simplify auto-router setup and clarify feature limits (#42625) 2026-09-22 18:03:32 -07:00
rust_bridge feat(logger): dispatch Python logging through the Rust diagnostics processor (#42616) 2026-09-22 18:44:15 -07:00
secret_managers fix(aws_secret_manager_v2): restore secret scheduled for deletion instead of failing CreateSecret (#42454) 2026-09-22 14:12:20 -05:00
types feat: add configurable provider affinity header mapping (#41033) 2026-09-22 13:31:33 -07:00
vector_stores fix(vector_stores): keep config-defined vector stores listed and read-only (#42574) 2026-09-23 04:02:16 +00:00
videos
__init__.py
conftest.py
log.txt
readme.md
test_a2a_registry_lookup.py
test_acompletion_session_reuse_e2e.py
test_add_deployment_no_master_key.py
test_aembedding_session_reuse_e2e.py
test_anthropic_beta_headers_filtering.py feat(router): native compact-to-fit across conversation APIs (#42074) 2026-09-21 22:52:29 -07:00
test_anthropic_skills_transformation.py
test_assert_ci_coverage.py
test_assert_workflow_dir_hygiene.py
test_audio_transcription_rust_bridge.py
test_auto_update_price_and_context_window_file.py
test_azure_ad_token_credential_resolution.py
test_azure_ai_grok_4_3_model_metadata.py
test_azure_ai_grok_4_6_model_metadata.py
test_baseten_glm_5_3_model_metadata.py
test_batch_completion_models_all_responses.py
test_bedrock_marengo_embed_3_model_metadata.py
test_budget_ratchet_check.py
test_chat_ui_responses_session.py
test_check_licenses.py
test_check_mcp_operation_boundary.py refactor(mcp): extract explicit operation context and dispatch 2026-09-21 12:24:15 -07:00
test_check_migrations_no_data_rewrites.py
test_check_py310_typing_imports.py
test_check_test_quality.py
test_check_type_discipline.py
test_circleci_path_filter.py
test_circleci_rust_toolchain.py
test_claude_fable_5_config.py
test_claude_opus_4_6_config.py
test_claude_opus_4_8_config.py
test_claude_opus_5_config.py feat(cost-map): add Claude Opus 5.5 for Vertex AI and Azure AI (#42599) 2026-09-22 21:48:07 +00:00
test_claude_sonnet_5_config.py
test_cloudflare_workers_ai_model_metadata.py
test_completion_timeout_resolution.py
test_component_entrypoint.py
test_compression.py
test_conftest.py
test_conftest_isolation.py
test_constants.py
test_container_router.py
test_cost_calculation_log_level.py
test_cost_calculator.py fix(pricing): drop the unpublished cached rate from the Gemini Live preview entries (#42651) 2026-09-22 20:35:34 -07:00
test_cost_map_guard.py ci: skip cost map file checks on PRs that leave the cost map untouched (#42406) 2026-09-21 21:40:48 -07:00
test_count_tokens_public_api.py chore(cost-map): remove models past their deprecation date (#42435) 2026-09-22 21:19:26 +00:00
test_dashscope_image_generation.py
test_daybreak_model_metadata.py
test_deepseek_model_metadata.py
test_default_branch.py
test_detect_changes.py
test_dockerfile_apk_repository.py
test_dockerfile_bedrock_realtime_extra.py
test_dockerfile_non_root.py
test_drop_params_env_var.py
test_e2e_egress_sentinel.py
test_eager_tiktoken_load.py
test_env_key_doc_gate.py
test_exception_exports.py
test_exception_header_preservation.py
test_exception_mapping_request_attribute.py
test_filter_out_litellm_params.py
test_fireworks_serverless_model_costs.py
test_gate_slot_lock.py
test_gemini_3_1_flash_lite_image_pricing.py
test_gemini_tts_native_audio_pricing.py
test_get_blog_posts.py
test_git_hooks.py
test_gpt_5_4_model_metadata.py
test_gpt_5_5_model_metadata.py
test_gpt_image_cost_calculator.py chore(cost-map): remove models past their deprecation date (#42435) 2026-09-22 21:19:26 +00:00
test_gpt_realtime_mode.py
test_groq_streaming_encoding.py
test_guardrail_exception_status_codes.py
test_lazy_imports.py
test_lint_workflow_diff_gates.py
test_litellm_params_reserved_keys.py
test_logging.py feat(logger): dispatch Python logging through the Rust diagnostics processor (#42616) 2026-09-22 18:44:15 -07:00
test_lowest_latency_zero_tokens.py
test_main.py fix(edenai): restore unrelated files the squashed PR commit had reverted 2026-09-21 19:42:37 +00:00
test_main_module_header.py
test_mistral_medium_3_5_model_metadata.py
test_mistral_small_4_0_model_metadata.py
test_mistral_zai_glm_5_2_model_metadata.py
test_model_block_unblock.py
test_model_cost_aliases.py
test_model_param_helper.py
test_model_prices_schema.py
test_model_response_normalization.py
test_muse_spark_1_1_model_metadata.py
test_muse_spark_1_2_model_metadata.py
test_muse_spark_1_3_model_metadata.py
test_mutation_report.py
test_nested_drop_params.py
test_non_chat_routes_open_llm_spans.py
test_openai_embedding_encoding_format_default.py
test_openai_service_tier_long_context_pricing.py feat(openai): add GPT-6 Sol and GPT-6 Luna (#42515) 2026-09-22 11:34:01 -07:00
test_pre_commit_lint.py test: count a zombie grandchild as gone in the migrate deploy timeout test (#42570) 2026-09-22 14:59:26 -07:00
test_prisma_generate_if_needed.py
test_process_helpers.py test: count a zombie grandchild as gone in the migrate deploy timeout test (#42570) 2026-09-22 14:59:26 -07:00
test_project_alias_tracking.py
test_project_tags_pydantic.py
test_proxy_auth.py
test_rag_openai_ingestion.py
test_rate_limit_error_unification.py
test_redact_string_in_error_paths.py test(realtime): drop legacy InvalidStatusCode tests and pin websockets imports (#42624) 2026-09-22 17:42:12 -07:00
test_redis.py
test_redis_credential_provider.py
test_register_model_custom_pricing.py
test_register_model_zero_cost_persistence.py
test_replicate_model_key_format.py
test_responses_api_bridge_non_stream.py
test_responses_id_security.py
test_responses_streaming_container_ownership.py
test_retrieve_batch_bedrock_dispatch.py
test_router.py chore(cost-map): remove models past their deprecation date (#42435) 2026-09-22 21:19:26 +00:00
test_router_block_helpers.py
test_router_exception_redaction.py fix(router): explain fallback outcome in plain words in the raised error (#42509) 2026-09-22 18:37:42 +00:00
test_router_google_genai.py
test_router_model_cost_isolation.py
test_router_order_fallback.py
test_router_per_deployment_num_retries.py
test_router_redis_init.py
test_router_retry_backoff_headers.py
test_router_retry_non_retryable_errors.py
test_router_retry_policy_update.py
test_router_silent_experiment.py
test_router_streaming_fallback_metadata.py
test_router_weighted_failover.py
test_ruff_strict_gate.py
test_sambanova_model_metadata.py
test_secret_redaction.py feat(logger): dispatch Python logging through the Rust diagnostics processor (#42616) 2026-09-22 18:44:15 -07:00
test_select_ui_test_scope.py
test_service_logger.py
test_setup_wizard.py
test_shared_session_integration.py
test_ssl_verify_unit.py
test_stream_chunk_builder_annotations.py
test_stream_chunk_builder_citations.py
test_stream_chunk_builder_images.py
test_streaming_connection_cleanup.py
test_sync_together_ai_models.py
test_system_message_format_bug.py
test_test_quality_gate.py
test_thinking_enabled.py
test_together_ai_model_metadata.py chore(cost-map): remove models past their deprecation date (#42435) 2026-09-22 21:19:26 +00:00
test_type_check_gate.py
test_type_discipline_gate.py
test_typesafe_model_metadata.py
test_unit_shard_missing_paths.py ci(test-unit): drop dead misc shard paths and skip missing paths with a warning (#42603) 2026-09-23 00:56:32 +00:00
test_unit_shard_per_test_timeout.py
test_utils.py test(utils): raise the post-success hook error from a guardrail in the failure-hook regression (#42646) 2026-09-23 02:11:19 +00:00
test_utils_module_docstring.py
test_uuid_helper.py
test_vcr_safe_body_matcher.py
test_vertex_ai_xai_grok_prompt_caching_metadata.py test(pricing): assert cache-priced vertex grok rows advertise supports_prompt_caching (#41526) 2026-09-21 20:58:11 -07:00
test_video_generation.py
test_with_dashboard_node.py
test_xai_grok_4_3_model_metadata.py
test_xai_responses_auto_routing.py chore(cost-map): remove models past their deprecation date (#42435) 2026-09-22 21:19:26 +00:00

readme.md

Testing for litellm/

This directory 1:1 maps the the litellm/ directory, and can only contain mocked tests.

The point of this is to:

  1. Increase test coverage of litellm/
  2. Make it easy for contributors to add tests for the litellm/ package and easily run tests without needing LLM API keys.

File name conventions

  • litellm/proxy/test_caching_routes.py maps to litellm/proxy/caching_routes.py
  • test_<filename>.py maps to litellm/<filename>.py