mirror of
https://github.com/BerriAI/litellm.git
synced 2026-10-31 06:56:59 +00:00
ec7162aea2
* fix(cost_map): mark fireworks_ai/minimax-m3 as vision and pin capabilities verified against live calls Fireworks' serverless minimax-m3 answers image inputs, but its model listings say supports_image_input false, so the last two registry syncs flipped supports_vision back to false and the Fireworks pre-flight turned every image request into a 400 without ever calling the provider. Flip both minimax-m3 rows back to supports_vision true and record the live-call evidence in ci_cd/cost_map_pins.json. The cost map guard now fails any PR that lowers a pinned capability, quoting the evidence, and the sync bot may not touch the pins file. * fix(cost_map): fail the guard when the pins file is missing * fix(cost_map_guard): skip the pin checks on a branch cut before the pins file existed * fix(ci): check cost map pins against the merge result and keep price sync PRs off the pins file * fix(ci): run the cost map guard against the live base branch tip and re-run it when main's pins change * fix(ci): pass the live base tip to the cost map guard through env and re-run every guard run on a pin change main now runs the guard on pull_request with the base branch's copy of the script, so the workflow passes the live tip in COST_MAP_GUARD_TARGET, which the older script ignores. The recheck reads past GraphQL's 100-file cap, re-runs every guard run for a shared head, and fails when a re-run cannot start. The guard tells a merge conflict apart from a failed merge-tree call. * fix(ci): run the cost map pins recheck on litellm_ base branches too * fix(ci): bar bot-opened PRs from the pins, recheck pin PRs on cost map changes, and skip the merge for untouched PRs * fix(ci): read the pushed files from the compare API in the cost map pins recheck The push event GitHub hands to Actions leaves each commit's added, modified and removed lists null, so the recheck never saw a pins change and re-ran only the PRs that touch the pins. Compare the push's before and after commits instead, and re-check every cost map PR when the branch is new, the push was forced, the compare fails, or it lists 300 files, the most it returns. --------- Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com> |
||
|---|---|---|
| .. | ||
| _support | ||
| audio_tests | ||
| base_sdk_tests | ||
| benchmarks | ||
| code_coverage_tests | ||
| documentation_tests | ||
| e2e | ||
| e2e_harness | ||
| fixtures | ||
| harness_e2e | ||
| integration | ||
| rust-python-harness | ||
| test_litellm_rust | ||
| typing | ||
| unit | ||
| windows_tests | ||
| __init__.py | ||
| _fake_openai_endpoint_server.py | ||
| _flush_vcr_cache.py | ||
| _live_test_helpers.py | ||
| _master_key.py | ||
| _openai_record_replay_proxy.py | ||
| _process_helpers.py | ||
| _vcr_conftest_common.py | ||
| _vcr_redis_persister.py | ||
| _wait_helpers.py | ||
| _ws_vcr.py | ||
| AGENTS.md | ||
| capturing_transport.py | ||
| eval_swe_bench.py | ||
| fake_openai_endpoint.py | ||
| gettysburg.wav | ||
| large_text.py | ||
| openai_batch_completions.jsonl | ||
| pyrightconfig.json | ||
| README.MD | ||
| test_logging.conf | ||
| test_rust_python_harness.py | ||
| VCR_TESTING.md | ||
| white_100x100.png | ||
In total litellm runs 1000+ tests
[02/20/2025] Update:
To make it easier to contribute and map what behavior is tested,
we've started mapping the litellm directory in tests/unit
This folder can only run mock tests.