Tin Chi Lo a4a1573e2d fix(cache_warming): let the concurrency bound also bound decompressed payloads ... Every session in a tick is started at once, so anything a session materializes before acquiring its replay slot scales with max_sessions rather than with the concurrency setting. The payload was inflated above the semaphore and held for the whole replay, and capture admits payloads up to eight times the compressed cap, so one tick could hold a decompressed payload per active session instead of per replay in flight. Decompression moves inside the slot, which is now held across the session's whole due set. The replay ceiling is unchanged, since a session's models are replayed in sequence inside its slot, and the CPU burst of inflating is now serialized to the same bound rather than running for every session at once. The bound had no test at all; the rig tracked peak in-flight replays and nothing asserted on it. One test now pins both halves through a real tick: replays in flight and payloads inflated must both respect max_concurrent_replays. It fails on the previous ordering with the payload count at the session count, not the bound.		2026-07-30 15:46:04 -07:00
..
a2a_protocol	fix(otel): label retrieval and agent metrics correctly and emit gen_ai.provider.name (#35151)	2026-07-30 13:48:59 -07:00
anthropic_interface	style: unify ruff format width on 120 (#31518)	2026-06-27 12:39:29 -07:00
assistants	style: unify ruff format width on 120 (#31518)	2026-06-27 12:39:29 -07:00
batch_completion	style: unify ruff format width on 120 (#31518)	2026-06-27 12:39:29 -07:00
batches	refactor(batches): aggregate batch output cost, usage, and models in a single pass	2026-07-29 22:00:01 -07:00
caching	fix(redis): stop an unreachable Redis from blocking every request (#35273)	2026-07-30 14:36:28 -07:00
completion_extras	fix(responses-bridge): return CustomStreamWrapper from the completed-response stream helper	2026-07-27 14:48:18 -07:00
compression	style: unify ruff format width on 120 (#31518)	2026-06-27 12:39:29 -07:00
containers	style: unify ruff format width on 120 (#31518)	2026-06-27 12:39:29 -07:00
endpoints/speech/speech_to_completion_bridge	style: unify ruff format width on 120 (#31518)	2026-06-27 12:39:29 -07:00
evals	style: unify ruff format width on 120 (#31518)	2026-06-27 12:39:29 -07:00
experimental_mcp_client	fix(mcp): sanitize Anthropic tool schemas and stop encoding gateway names	2026-07-17 10:33:28 -07:00
files	style: unify ruff format width on 120 (#31518)	2026-06-27 12:39:29 -07:00
fine_tuning	chore(lint): zero out crash-class pyright rules and ban new type: ignore comments (#32152)	2026-07-04 16:56:12 -07:00
google_genai	fix(logging): classify async anthropic_messages and generate_content as async (#33589)	2026-07-16 20:56:47 -07:00
images	style: unify ruff format width on 120 (#31518)	2026-06-27 12:39:29 -07:00
integrations	feat(complexity_router): multi-model prompt-cache warming for the auto-router	2026-07-30 15:46:04 -07:00
interactions	fix(responses): stop scheduling sync success_handler concurrently with async_success_handler (#32239)	2026-07-07 09:13:50 -07:00
litellm_core_utils	fix(complexity_router): scope keyless callers by their stamped tenancy, not one shared bucket	2026-07-30 15:46:04 -07:00
llms	chore(typing): clear basedpyright Any errors in proxy auth, repositories, and openai transforms	2026-07-30 13:48:43 +00:00
models	feat(mcp): issuer-anchored OAuth discovery (RFC 8414 §3.3) as the trust anchor	2026-07-15 16:59:01 -07:00
ocr	chore(typing): clear 2.7k basedpyright Any errors across 15 hotspot files	2026-07-26 18:39:26 -07:00
passthrough	fix(logging): classify allm_passthrough_route as async to prevent duplicate success callbacks (#32265)	2026-07-06 15:04:05 -07:00
proxy	refactor: extract user_is_scim_deactivated as one shared predicate	2026-07-30 15:46:04 -07:00
proxy_auth	style: unify ruff format width on 120 (#31518)	2026-06-27 12:39:29 -07:00
rag	fix(rag): track LLM completion usage and spend for /v1/rag/query (#32438)	2026-07-17 17:45:27 +00:00
realtime_api	fix(vertex): forward realtime health check params (#32550)	2026-07-08 17:30:46 -07:00
repositories	chore(typing): clear basedpyright Any errors in proxy auth, repositories, and openai transforms	2026-07-30 13:48:43 +00:00
rerank_api	fix(rerank): log optional_rerank_params at debug to stop leaking request content (#32533)	2026-07-08 18:48:03 -07:00
responses	feat(guardrails): scan and mask MCP tool results via post_mcp_call (#35155)	2026-07-30 14:10:26 -07:00
router_strategy	fix(cache_warming): let the concurrency bound also bound decompressed payloads	2026-07-30 15:46:04 -07:00
router_utils	feat(complexity_router): multi-model prompt-cache warming for the auto-router	2026-07-30 15:46:04 -07:00
rust_bridge	perf(bedrock): audio transcription via rust core (py->rust bridge) (#33990)	2026-07-20 14:09:41 -07:00
sandbox	style: unify ruff format width on 120 (#31518)	2026-06-27 12:39:29 -07:00
search	style: unify ruff format width on 120 (#31518)	2026-06-27 12:39:29 -07:00
secret_managers	fix(bedrock): stop replaying expired Google OIDC tokens to STS on guardrail auth (#34637)	2026-07-25 16:27:54 -07:00
skills	style: unify ruff format width on 120 (#31518)	2026-06-27 12:39:29 -07:00
types	feat(guardrails): scan and mask MCP tool results via post_mcp_call (#35155)	2026-07-30 14:10:26 -07:00
vector_store_files	style: unify ruff format width on 120 (#31518)	2026-06-27 12:39:29 -07:00
vector_stores	style: unify ruff format width on 120 (#31518)	2026-06-27 12:39:29 -07:00
videos	style: unify ruff format width on 120 (#31518)	2026-06-27 12:39:29 -07:00
__init__.py	feat(proxy): add overwrite_user_with_key_hash to stamp outgoing user param with key hash (#34417)	2026-07-23 16:38:01 -07:00
_internal_context.py	fix(proxy): harden request parameter handling	2026-04-16 01:38:12 +00:00
_lazy_imports.py	style: unify ruff format width on 120 (#31518)	2026-06-27 12:39:29 -07:00
_lazy_imports_registry.py	feat(tencent): add Tencent TokenHub as a provider (#31903)	2026-07-02 18:31:59 -07:00
_logging.py	style: unify ruff format width on 120 (#31518)	2026-06-27 12:39:29 -07:00
_redis.py	fix(redis): stop an unreachable Redis from blocking every request (#35273)	2026-07-30 14:36:28 -07:00
_redis_credential_provider.py	style: unify ruff format width on 120 (#31518)	2026-06-27 12:39:29 -07:00
_service_logger.py	style: unify ruff format width on 120 (#31518)	2026-06-27 12:39:29 -07:00
_uuid.py	build(deps-dev): bump black to 26.3.1 and apply formatting (#28525)	2026-05-21 17:24:18 -07:00
_version.py
anthropic_beta_headers_config.json	fix(bedrock-invoke): retain clear_tool_uses_20250919 context_management edits and emit context-management-2025-06-27 beta (LIT-3393) (#32658)	2026-07-09 14:31:27 -07:00
anthropic_beta_headers_manager.py	style: unify ruff format width on 120 (#31518)	2026-06-27 12:39:29 -07:00
blog_posts.json	docs(blog): add WebRTC blog post link	2026-03-13 17:54:33 +05:30
budget_manager.py	style: unify ruff format width on 120 (#31518)	2026-06-27 12:39:29 -07:00
constants.py	feat(complexity_router): multi-model prompt-cache warming for the auto-router	2026-07-30 15:46:04 -07:00
cost.json
cost_calculator.py	fix(cost_calculator): sum mirrored cache token fields once in combine_usage_objects	2026-07-23 19:07:09 -07:00
exceptions.py	feat(logging): add structured budget fields to budget rejection failure logs (#33460)	2026-07-16 12:39:04 -07:00
main.py	refactor(bedrock): remove the dead BedrockLLM invoke code path	2026-07-29 20:25:36 -07:00
model_prices_and_context_window_backup.json	fix(pricing): correct gpt-5.4-mini and gpt-5.4-nano token limits	2026-07-30 02:43:54 +00:00
policy_templates_backup.json
provider_endpoints_support_backup.json	chore: litellm oss staging (#30968)	2026-06-23 07:31:44 -07:00
py.typed
router.py	feat(router): record why the auto-router picked a tier and show it in the logs (#35016)	2026-07-30 11:55:10 -07:00
scheduler.py	style: unify ruff format width on 120 (#31518)	2026-06-27 12:39:29 -07:00
setup_wizard.py	feat(anthropic): add Claude Opus 5	2026-07-24 10:43:49 -07:00
timeout.py	style: unify ruff format width on 120 (#31518)	2026-06-27 12:39:29 -07:00
utils.py	fix: match exact class in callback dedup so a custom subclass does not block a built-in logger (#34804)	2026-07-27 12:18:42 -07:00