mirror of
https://github.com/BerriAI/litellm.git
synced 2026-11-03 07:23:33 +00:00
b89fb13d51
* fix(register_model): keep bedrock geo deployments priced across price data reloads
A request-time pricing registration inserting into litellm.model_cost while a
price data reload walked it could abort the reload (dictionary changed size
during iteration in add_known_models), or trip the lookup map rebuild inside
get_model_info so the deployment replay fell through to the unmapped path and
wrote a raw bedrock/au.anthropic.<model> entry with no token pricing, a $0
shadow re-rolled on every later reload. Walk snapshots of the map, bind the
case-insensitive lookup map locally, and let the unmapped path inherit the
whole built-in entry of a provider get_model_info would accept.
* fix(register_model): keep synthesized zero costs out of inherited built-in pricing
An unmapped alias that inherits its built-in entry now takes only the token
prices the catalogue really carries, so a synthesized zero never lands next to
the caller's custom price. The router regression test reports the writer's
failures instead of swallowing them, and the local names carry Final.
* fix(register_model): match inherited built-ins by the key provider prefix when the value names none
* fix(utils): resolve a tripped lookup from the key's own catalog row before its region sibling
* fix(register_model): keep custom flat prices above inherited tier rates
* fix(register_model): keep the other direction's tier rates when a custom flat price covers one
* fix(register_model): keep a catalog row's tiers when its lookup trips, and snapshot request-path cost map walks
A tripped get_model_info on a key that is itself a catalog row now inherits that row whole, so a custom flat price bills a long prompt at the row's above-threshold rate on both lookup paths instead of flipping until the next reload. The fal_ai, fireworks, ollama and gpt-5 request-path walks over litellm.model_cost iterate a snapshot so a concurrent reload cannot raise mid-request.
* fix(register_model): price route-prefixed geo keys on a tripped lookup and replace cost map entries atomically
* fix(register_model): move every alias of a cost map entry to its registered price
* perf(register_model): read alias names from the shared entry instead of scanning the cost map
Each registration scanned every cost map entry to find the names sharing it,
about 1 ms on the 4,189-entry map, and a proxy request to a custom-priced
deployment registers twice. Aliased entries now carry the names they are
served under, so a registration reads them directly, and an alias the caller
pointed at its own entry keeps that entry. The tests this PR added lose their
docstrings
* fix(utils): keep capability flags and write through on a tripped lookup in register_model
A key the lookup cannot map now seeds its registered entry from the capability
rules its unregistered lookup resolves to, so registering a region alias no
longer turns off supports_system_messages and similar flags. A lookup that
trips mid-walk on a key naming a catalog row writes into that row instead of
leaving a shadow key that goes stale. Adds regression tests for both and for
the request-path snapshot walks.
* fix(utils): retry a tripped registration lookup and rebuild a registration a reload races
A lookup a concurrent write trips is retried instead of writing into the
catalog row its key names, so a per-request price never reprices a shared
row. The inherited entry leaves the matched row's provider behind, so a
provider-less azure_ai key stays priced, and a registration whose map a
reload swapped mid-build is rebuilt from the new map.
* fix(utils): write each registration into the map it was built from and rebuild it when a reload replaced that map
* test(integration): price data reload keeps every registered deployment priced
---------
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
(cherry picked from commit
|
||
|---|---|---|
| .. | ||
| test_bundled_cost_map_price_wire.py | ||
| test_configured_prices.py | ||
| test_cost_margin_discount.py | ||
| test_databricks_cache_pricing.py | ||
| test_model_listing_token_limits.py | ||
| test_ocr_page_pricing.py | ||
| test_off_peak_pricing.py | ||
| test_per_second_pricing.py | ||
| test_price_data_reload_prices.py | ||
| test_price_precedence.py | ||
| test_realtime_cached_audio_pricing.py | ||
| test_service_tier_pricing.py | ||