mirror of
https://github.com/BerriAI/litellm.git
synced 2026-09-15 23:31:29 +00:00
2cf209f265
* fix(mistral-ocr): forward the complete public parameter contract The Rust OCR path already lists id in the supported Mistral parameters, but litellm/ocr/main.py ran filter_out_litellm_params before handing optional_params to the bridge. That generic filter drops every key in all_litellm_params, which includes id, so the supported public Mistral OCR field id was silently stripped and never reached the provider while the rest of the contract went through. Restore id after the generic filter via an OCR-specific reserved set, keeping genuine internal params filtered. Add Rust core and ai-gateway contract tests that pin the exact serialized request body for the full supported contract and prove internal canaries are dropped, a Python regression test that id survives while internal params do not, and a deterministic capture E2E plus a live Mistral param test. Co-Authored-By: Ishaan Jaffer <155045088+ishaan-berri@users.noreply.github.com> * test(mistral-ocr): drop unnecessary comments from parity tests Co-Authored-By: Ishaan Jaffer <155045088+ishaan-berri@users.noreply.github.com> * fix(mistral-ocr): only forward reserved id when set; harden capture e2e Co-Authored-By: Ishaan Jaffer <155045088+ishaan-berri@users.noreply.github.com> * refactor(mistral-ocr): typed queue-injected capture harness for e2e Co-Authored-By: Ishaan Jaffer <155045088+ishaan-berri@users.noreply.github.com> * test(mistral-ocr): fully type E2E capture models and move gateway ocr tests Replace the remaining Any/coarse object wire types in the OCR E2E module with concrete frozen Pydantic models for the supported-param contract, internal canaries, upstream capture request, and normalized response, and build every request payload immutably per call. The capture fixture now cleans up on every failure path (terminate then always wait, kill then always wait, shutdown plus server_close, join the server thread) and requires an HTTP 200 liveness probe. The live Mistral test now exercises the complete public parameter contract with valid json_schema annotation formats. Move the ai-gateway OCR test module out of io/ocr.rs into a focused io/ocr/tests.rs. Co-Authored-By: Ishaan Jaffer <155045088+ishaan-berri@users.noreply.github.com> * fix(mistral-ocr): drop reserved id upstream when null and type the E2E harness Extend map_ocr_params so a null-valued reserved public id is omitted from the serialized request, matching the Python OCR boundary for the standalone Axum path where a caller can submit JSON id:null; add core map and serialized-body tests for both the absent and null id cases. Extract the deterministic capture harness into a typed ocr_capture_proxy helper module with an exception-safe fixture that cleans up on every failure path from capture-server creation onward (terminate then always wait, kill then always wait, shutdown plus server_close, join the server thread, close logs after the process exits) and requires an HTTP 200 liveness probe. Serialize wire payloads with by_alias so the annotation schema/additionalProperties keys match the provider contract. Replace the lambda internal canary in the Python bridge regression with a serializable value and cover the litellm_session_id and tags internal fields. Co-Authored-By: Ishaan Jaffer <155045088+ishaan-berri@users.noreply.github.com> --------- Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> Co-authored-by: Ishaan Jaffer <155045088+ishaan-berri@users.noreply.github.com> |
||
|---|---|---|
| .. | ||
| agent_tests | ||
| audio_tests | ||
| basic_proxy_startup_tests | ||
| batches_tests | ||
| benchmarks | ||
| code_coverage_tests | ||
| documentation_tests | ||
| e2e | ||
| enterprise | ||
| guardrails_tests | ||
| image_gen_tests | ||
| integration | ||
| litellm | ||
| litellm-proxy-extras | ||
| litellm_core_utils | ||
| litellm_utils_tests | ||
| llm_responses_api_testing | ||
| llm_translation | ||
| load_tests | ||
| local_testing | ||
| logging_callback_tests | ||
| mcp_tests | ||
| multi_instance_e2e_tests | ||
| ocr_tests | ||
| old_proxy_tests/tests | ||
| openai_endpoints_tests | ||
| otel_tests | ||
| pass_through_tests | ||
| pass_through_unit_tests | ||
| proxy_admin_ui_tests | ||
| proxy_behavior | ||
| proxy_e2e_anthropic_messages_tests | ||
| proxy_migration_tests | ||
| proxy_security_tests | ||
| proxy_unit_tests | ||
| router_unit_tests | ||
| scim_tests | ||
| search_tests | ||
| spend_tracking_tests | ||
| store_model_in_db_tests | ||
| test_litellm | ||
| unified_google_tests | ||
| vector_store_tests | ||
| windows_tests | ||
| __init__.py | ||
| _fake_openai_endpoint_server.py | ||
| _flush_vcr_cache.py | ||
| _live_test_helpers.py | ||
| _openai_record_replay_proxy.py | ||
| _vcr_conftest_common.py | ||
| _vcr_redis_persister.py | ||
| eval_swe_bench.py | ||
| fake_openai_endpoint.py | ||
| gettysburg.wav | ||
| large_text.py | ||
| openai_batch_completions.jsonl | ||
| pyrightconfig.json | ||
| README.MD | ||
| test_anthropic_compaction_usage.py | ||
| test_budget_management.py | ||
| test_callbacks_on_proxy.py | ||
| test_config.py | ||
| test_debug_warning.py | ||
| test_default_encoding_non_root.py | ||
| test_end_users.py | ||
| test_entrypoint.py | ||
| test_fallbacks.py | ||
| test_gpt5_azure_temperature_support.py | ||
| test_health.py | ||
| test_keys.py | ||
| test_litellm_proxy_responses_config.py | ||
| test_logging.conf | ||
| test_models.py | ||
| test_new_vector_store_endpoints.py | ||
| test_openai_endpoints.py | ||
| test_organizations.py | ||
| test_otel_thread_leak.py | ||
| test_passthrough_endpoints.py | ||
| test_presidio_latency.py | ||
| test_proxy_server_non_root.py | ||
| test_ratelimit.py | ||
| test_resource_cleanup.py | ||
| test_service_logger_otel.py | ||
| test_spend_logs.py | ||
| test_team.py | ||
| test_team_logging.py | ||
| test_team_members.py | ||
| test_users.py | ||
In total litellm runs 1000+ tests
[02/20/2025] Update:
To make it easier to contribute and map what behavior is tested,
we've started mapping the litellm directory in tests/test_litellm
This folder can only run mock tests.