mirror of
https://github.com/BerriAI/litellm.git
synced 2026-10-21 05:15:06 +00:00
9e9dccb9d4
* fix(register_model): keep bedrock geo deployments priced across price data reloads A request-time pricing registration inserting into litellm.model_cost while a price data reload walked it could abort the reload (dictionary changed size during iteration in add_known_models), or trip the lookup map rebuild inside get_model_info so the deployment replay fell through to the unmapped path and wrote a raw bedrock/au.anthropic.<model> entry with no token pricing, a $0 shadow re-rolled on every later reload. Walk snapshots of the map, bind the case-insensitive lookup map locally, and let the unmapped path inherit the whole built-in entry of a provider get_model_info would accept. * fix(register_model): keep synthesized zero costs out of inherited built-in pricing An unmapped alias that inherits its built-in entry now takes only the token prices the catalogue really carries, so a synthesized zero never lands next to the caller's custom price. The router regression test reports the writer's failures instead of swallowing them, and the local names carry Final. * fix(register_model): match inherited built-ins by the key provider prefix when the value names none * fix(utils): resolve a tripped lookup from the key's own catalog row before its region sibling * fix(register_model): keep custom flat prices above inherited tier rates * fix(register_model): keep the other direction's tier rates when a custom flat price covers one * fix(register_model): keep a catalog row's tiers when its lookup trips, and snapshot request-path cost map walks A tripped get_model_info on a key that is itself a catalog row now inherits that row whole, so a custom flat price bills a long prompt at the row's above-threshold rate on both lookup paths instead of flipping until the next reload. The fal_ai, fireworks, ollama and gpt-5 request-path walks over litellm.model_cost iterate a snapshot so a concurrent reload cannot raise mid-request. * fix(register_model): price route-prefixed geo keys on a tripped lookup and replace cost map entries atomically * fix(register_model): move every alias of a cost map entry to its registered price * perf(register_model): read alias names from the shared entry instead of scanning the cost map Each registration scanned every cost map entry to find the names sharing it, about 1 ms on the 4,189-entry map, and a proxy request to a custom-priced deployment registers twice. Aliased entries now carry the names they are served under, so a registration reads them directly, and an alias the caller pointed at its own entry keeps that entry. The tests this PR added lose their docstrings * fix(utils): keep capability flags and write through on a tripped lookup in register_model A key the lookup cannot map now seeds its registered entry from the capability rules its unregistered lookup resolves to, so registering a region alias no longer turns off supports_system_messages and similar flags. A lookup that trips mid-walk on a key naming a catalog row writes into that row instead of leaving a shadow key that goes stale. Adds regression tests for both and for the request-path snapshot walks. * fix(utils): retry a tripped registration lookup and rebuild a registration a reload races A lookup a concurrent write trips is retried instead of writing into the catalog row its key names, so a per-request price never reprices a shared row. The inherited entry leaves the matched row's provider behind, so a provider-less azure_ai key stays priced, and a registration whose map a reload swapped mid-build is rebuilt from the new map. * fix(utils): write each registration into the map it was built from and rebuild it when a reload replaced that map * test(integration): price data reload keeps every registered deployment priced --------- Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com> |
||
|---|---|---|
| .. | ||
| _support | ||
| base_sdk_tests | ||
| benchmarks | ||
| code_coverage_tests | ||
| documentation_tests | ||
| e2e | ||
| e2e_harness | ||
| fixtures | ||
| harness_e2e | ||
| integration | ||
| rust-python-harness | ||
| test_litellm_rust | ||
| typing | ||
| unit | ||
| windows_tests | ||
| __init__.py | ||
| _fake_openai_endpoint_server.py | ||
| _flush_vcr_cache.py | ||
| _live_test_helpers.py | ||
| _master_key.py | ||
| _openai_record_replay_proxy.py | ||
| _process_helpers.py | ||
| _vcr_conftest_common.py | ||
| _vcr_redis_persister.py | ||
| _wait_helpers.py | ||
| _ws_vcr.py | ||
| AGENTS.md | ||
| capturing_transport.py | ||
| eval_swe_bench.py | ||
| fake_openai_endpoint.py | ||
| gettysburg.wav | ||
| large_text.py | ||
| openai_batch_completions.jsonl | ||
| pyrightconfig.json | ||
| README.MD | ||
| test_logging.conf | ||
| test_rust_python_harness.py | ||
| VCR_TESTING.md | ||
| white_100x100.png | ||
In total litellm runs 1000+ tests
[02/20/2025] Update:
To make it easier to contribute and map what behavior is tested,
we've started mapping the litellm directory in tests/unit
This folder can only run mock tests.