The org cache entries written by prefetch_auth_objects carry the 5s
DEFAULT_IN_MEMORY_TTL. The first @log_db_metrics getter lazily imports
litellm.proxy.proxy_server, which on a cold CI runner can take longer
than 5s, so the org entry expired before get_org_object read it and the
getter fell through to the MagicMock database. Inject a frozen clock
into InMemoryCache so the test asserts the join, not import latency.
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
RouterRateLimitError now carries the model group's deployment ids so it
can tell when every deployment is cooled down, and exposes that as
type=all_deployments_in_cooldown with an explicit message. A partial
cooldown keeps type=rate_limit_error. Either way the proxy no longer
reports type=internal_server_error next to code 429
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
CodeQL resolves `from openai import Omit` in litellm/types/llms/openai.py to the
module itself, so every importer of a name whose definition line is in the diff
is reported as an unsafe cyclic import. 76ae35dfcd edited the line that defines
BaseLiteLLMOpenAIResponseObject there and got two alerts at files this PR does
not touch. That line is now byte-identical to main and CachedTokensDetails
arrives through a relative import isort keeps separate.
CodeQL flagged two module-level cyclic imports introduced by defining
CachedTokensDetails in litellm.types.llms.openai and importing it from
litellm.types.utils and litellm.cost_calculator. The class now lives in
litellm.types.llms.base, which imports nothing from litellm, and every
user imports it from there.
Also pins that combining realtime usages where only one response.done
carries cached_tokens_details keeps the earlier modality split in both
orders, and commits the regenerated dashboard API types.
CredentialLiteLLMParams omitted tenant_id, client_id, client_secret,
azure_scope, azure_username and azure_password, so the strict dump used
by credential reuse and Azure client init dropped them and the reused
credential ended with no auth at all
Co-authored-by: yassin <yassin@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
The e2e harness exists to prove product features end to end against a live
proxy. The prior Hard Rule carved out an exception for "tests that cover the
harness itself" and pointed at coverage_registry/test_collector.py, which in
practice invited unit tests of harness helpers to be staged alongside e2e
work. That is the wrong tool: harness logic that is worth locking down does
not need a mock-driven unit test living under tests/e2e.
Drop the carve-out. The Hard Rule now reads that no unit tests of any kind
belong under tests/e2e, and the passing mention of unmarked harness coverage
in the transport section is removed so the doc no longer contradicts itself.
coverage_registry/test_collector.py still exists on disk and is left in place
for now; whether to relocate or remove it is a separate decision.
Keeps the base's rule that a non-admin id lookup matching no spend-log row answers 403, so the detail route never consults cold storage without an owner row
* fix(proxy): bound tool and guardrail index create_many by the spend-log statement budgets
One flush drains up to MAX_LOGS_PER_INTERVAL source transactions or logs, but a
transaction fans out to one LiteLLM_SpendLogToolIndex row per tool and a log
to one LiteLLM_SpendLogGuardrailIndex row per guardrail, so the index
create_many payload was unbounded. Both index writes now go through
spend_log_write_batches(SPEND_LOG_WRITE_BATCH_MAX_BYTES, SPEND_LOG_WRITE_BATCH_MAX_ROWS).
The tool index write moves out of the rollup batch_() so the split reduces
the query-engine payload; replayed index rows are no-ops under
skip_duplicates, and the daily rollup upserts stay in one transaction
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(proxy): pin the row budget in the index fan-out tests
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: yucheng <yucheng@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Router.log_retry used to copy the failed attempt's kwargs and metadata into
metadata.previous_models. Nothing downstream read those copies, but they carried
client credentials into spend logs and grew the payload on every retry. Each
attempt now leaves a flat record (model group, deployment id, exception type and
string, attempt number), which drops RETRY_BREADCRUMB_EXCLUDED_KWARGS and the
per-retry credential masking.
num_retries_per_request was enforced from len(previous_models), which only
looked at the metadata bucket and never exceeded four records. The sync and
async client wrappers and the Rust lifecycle guard now read attempted_retries
from whichever metadata bucket the call carries.
Resolves LIT-7505
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
The bridge probe asked `responses_api_bridge_check` with the summary read straight off
the Responses object, but `litellm.completion` reads it from `optional_params` via
`peek_reasoning_summary_aliases`, which the bridged request never populated. So gpt-5,
gpt-5.1 and azure/gpt-5 answered "bridging" to the probe and "not bridging" for real,
and the object still landed on Chat Completions, which only takes a string
`reasoning_effort` is now always the effort string, and `summary` rides the
`reasoning_summary` alias that main.py already reassembles into `{effort, summary}` on
the bridged path. The alias is emitted only when the probe says the model bridges, so
no chat provider ever sees it, and the probe is now asked with the exact params this
transform emits
The debug call built its message with an f-string, which
test_logging_calls_do_not_build_their_message_eagerly rejects. Pass the exception
as a %-style argument so the message is only built when the log is emitted.
