mirror of
https://github.com/BerriAI/litellm.git
synced 2026-09-24 00:52:24 +00:00
1bc7d34a84
* chore(e2e): move the compat-matrix populator from a GCE VM to a Render cron job The daily Claude Code compatibility-matrix job ran as a systemd timer on the litellm-compatibility-matrix-populator VM in the vertex-check GCP project. Replace that with a Render Docker cron job built from a new Dockerfile in tests/e2e/claude_code/cron_vm: pinned and checksummed debian base, gh, uv, and Claude Code CLI, a non-root populator user, and run_daily.sh as the entrypoint. run_daily.sh now clones a fresh blobless checkout per run (Render cron disks are ephemeral), reads the publish PAT from the github-token secret file under CREDENTIALS_DIRECTORY, and its comments no longer describe systemd. The .service and .timer units are gone; README.md and the env example describe the Render service, its secret files, and the local docker build instead. * docs(e2e): name the plan and trigger route Render's cron-job API accepts Render answers a bare 404 for the legacy pro_max plan name on a cron job (4c-16g is the same 4 CPU / 16 GB size) and the manual trigger route is /v1/cron-jobs, not /v1/cronjobs. * fix(e2e): install the published litellm wheel instead of building the tag from source The tag builds a Rust extension through maturin, which needs a C and Rust toolchain the cron image does not carry, so the first Render run failed at uv sync with "linker cc not found". Sync the locked dependencies with --no-install-project, install the PyPI wheel (what users run) with --no-build, and pass --no-sync to every uv run so uv never puts the source build back. * fix(e2e): keep the SKIP_PUBLISH matrix where a Render run can read it The validation run wrote the matrix into the image checkout, which nobody can read once the container exits. Save it under HOME and print it at the end of the log instead. * fix(e2e): let the stale compat-matrix PR sweep see past the newest 100 docs PRs The docs repo has a few hundred open PRs, so a 100-item list never reached the week-old compat-matrix PR and the sweep left it open on every run. * docs(e2e): say the Render cron needs a manual deploy after each merge Pushes never started a deploy during setup because Render only hears about them through its GitHub app, which the org does not have, so the README now carries the deploy command and the wait-for-live rule * ci: build the compat-matrix cron image on pull requests The CI coverage gate requires every Dockerfile to be built by a job, and building this one on each PR that touches it also catches a broken pin or checksum before Render does * fix(e2e): shim the whole tests/e2e tree into the compat-matrix worktree The five-file helper allowlist missed fixture_mode, which e2e_config now imports, so the first Render run died at conftest load with ModuleNotFoundError. Copy the image's whole tests/e2e tree instead and keep pytest from loading the EKS-harness conftest with --confcutdir * fix(e2e): scope the compat-matrix sweep to the publishing account's own PRs The stale-PR sweep selected every open docs PR whose head branch starts with compat-matrix/, so a contributor's fork PR under that name would have been closed once a newer matrix PR existed. The sweep now resolves the publishing login from the token and only closes same-repo PRs that account opened --------- Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com> |
||
|---|---|---|
| .. | ||
| agent_tests | ||
| audio_tests | ||
| base_sdk_tests | ||
| basic_proxy_startup_tests | ||
| batches_tests | ||
| benchmarks | ||
| code_coverage_tests | ||
| documentation_tests | ||
| e2e | ||
| enterprise | ||
| guardrails_tests | ||
| image_gen_tests | ||
| integration | ||
| litellm-proxy-extras | ||
| litellm_utils_tests | ||
| llm_responses_api_testing | ||
| llm_translation | ||
| load_tests | ||
| local_testing | ||
| logging_callback_tests | ||
| mcp_tests | ||
| multi_instance_e2e_tests | ||
| ocr_tests | ||
| openai_endpoints_tests | ||
| otel_tests | ||
| pass_through_tests | ||
| pass_through_unit_tests | ||
| proxy_admin_ui_tests | ||
| proxy_behavior | ||
| proxy_e2e_anthropic_messages_tests | ||
| proxy_migration_tests | ||
| proxy_security_tests | ||
| proxy_unit_tests | ||
| router_unit_tests | ||
| rust-python-harness | ||
| search_tests | ||
| spend_tracking_tests | ||
| store_model_in_db_tests | ||
| test_gateway | ||
| test_litellm | ||
| test_litellm_rust | ||
| unified_google_tests | ||
| unit | ||
| vector_store_tests | ||
| windows_tests | ||
| __init__.py | ||
| _fake_openai_endpoint_server.py | ||
| _flush_vcr_cache.py | ||
| _live_test_helpers.py | ||
| _openai_record_replay_proxy.py | ||
| _process_helpers.py | ||
| _vcr_conftest_common.py | ||
| _vcr_redis_persister.py | ||
| _wait_helpers.py | ||
| _ws_vcr.py | ||
| AGENTS.md | ||
| capturing_transport.py | ||
| eval_swe_bench.py | ||
| fake_openai_endpoint.py | ||
| gettysburg.wav | ||
| large_text.py | ||
| openai_batch_completions.jsonl | ||
| pyrightconfig.json | ||
| README.MD | ||
| test_anthropic_compaction_usage.py | ||
| test_budget_management.py | ||
| test_callbacks_on_proxy.py | ||
| test_debug_warning.py | ||
| test_default_encoding_non_root.py | ||
| test_end_users.py | ||
| test_fallbacks.py | ||
| test_gpt5_azure_temperature_support.py | ||
| test_health.py | ||
| test_keys.py | ||
| test_litellm_proxy_responses_config.py | ||
| test_logging.conf | ||
| test_models.py | ||
| test_new_vector_store_endpoints.py | ||
| test_openai_endpoints.py | ||
| test_organizations.py | ||
| test_otel_thread_leak.py | ||
| test_presidio_latency.py | ||
| test_proxy_server_non_root.py | ||
| test_ratelimit.py | ||
| test_resource_cleanup.py | ||
| test_rust_python_harness.py | ||
| test_service_logger_otel.py | ||
| test_spend_logs.py | ||
| test_team.py | ||
| test_team_logging.py | ||
| test_team_members.py | ||
| test_users.py | ||
In total litellm runs 1000+ tests
[02/20/2025] Update:
To make it easier to contribute and map what behavior is tested,
we've started mapping the litellm directory in tests/test_litellm
This folder can only run mock tests.