scriptoriumadmin/litellm
1
0
Fork 0
mirror of https://github.com/BerriAI/litellm.git synced 2026-09-12 23:01:41 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 87
litellm/litellm
History
Exact
mubashir1osmani 9426f7b9dd
fix(logging): route realtime success logging through the bounded worker (#31733) 
RealTimeStreaming.log_messages dispatched the success handler with a bare
asyncio.create_task, bypassing GLOBAL_LOGGING_WORKER (which gives a per-coroutine
timeout and a concurrency cap). On a long-lived realtime websocket a slow logging
callback left one suspended task per logged turn, each pinning that turn's
assembled response, accumulating without bound (~12-15k in-flight under load in a
repro) until OOM. Route realtime success logging through the bounded worker so
in-flight logging is capped and a hung callback is cancelled at the worker
timeout.

The chat and responses streaming success-logging paths are intentionally left
unchanged: their success callbacks must complete within the call's event-loop run
(the non-streaming path pairs the worker with a synchronous callback; the
streaming path has no such companion), so deferring them through the worker would
drop logs for one-shot SDK calls and breaks test_async_custom_handler_stream.
Bounding those paths needs a load-shedding approach and is left to a follow-up.

(cherry picked from commit d4c33b2b59)
2026-06-30 18:38:01 -07:00
..
a2a_protocol chore(lint): remove PLR0915 too-many-statements ruff rule (#30574) 2026-06-16 16:52:49 -07:00
anthropic_interface feat(context_management): compact_20260112 polyfill for non-Anthropic providers (#28868) 2026-05-30 09:20:05 -07:00
assistants
batch_completion fix: prompt registry 2026-02-18 00:34:54 +05:30
batches fix(vertex/files): stream OpenAI->Vertex batch JSONL uploads (#31036) 2026-06-29 17:56:01 -07:00
caching feat(caching): add valkey-semantic cache backend and fix semantic cache scope keys (#30675) 2026-06-19 17:09:17 -07:00
completion_extras chore(lint): remove PLR0915 too-many-statements ruff rule (#30574) 2026-06-16 16:52:49 -07:00
compression build(deps-dev): bump black to 26.3.1 and apply formatting (#28525) 2026-05-21 17:24:18 -07:00
containers style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
endpoints/speech/speech_to_completion_bridge
evals style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
experimental_mcp_client feat(mcp): per-server env vars with global + per-user scopes (#28917) 2026-06-05 20:15:11 -07:00
files fix(vertex/files): stream OpenAI->Vertex batch JSONL uploads (#31036) 2026-06-29 17:56:01 -07:00
fine_tuning address greptile review feedback (greploop iteration 1) 2026-03-27 20:04:41 +05:30
google_genai fix(google_genai): preserve complete SSE events in Vertex/Gemini image streaming (#30270) 2026-06-12 07:49:30 -07:00
images chore(lint): remove PLR0915 too-many-statements ruff rule (#30574) 2026-06-16 16:52:49 -07:00
integrations fix(otel): one v2 logger owns the global provider; scope tenant OTLP creds per exporter (#30590) 2026-06-19 11:15:29 -07:00
interactions fix(a2a): forward agent_extra_headers through completion bridge (#28277) 2026-06-11 21:56:18 -07:00
litellm_core_utils fix(logging): route realtime success logging through the bounded worker (#31733) 2026-06-30 18:38:01 -07:00
llms fix(realtime): stop second Gemini Live setup, retry hung handshake, close guardrail bypass (#31519) 2026-06-30 18:37:14 -07:00
models feat(litellm): add models and repository layers (#29686) 2026-06-06 20:59:33 -07:00
ocr chore: reject bare str at file-input sinks to prevent local-file read (#27762) 2026-05-12 16:40:07 -07:00
passthrough feat(passthrough): add configurable pass-through request timeouts (#30266) 2026-06-12 07:40:02 -07:00
proxy fix(proxy/client): redact api key from key/info client error messages (#31342) 2026-06-29 18:00:11 -07:00
proxy_auth fix: prompt registry 2026-02-18 00:34:54 +05:30
rag chore: litellm oss staging (#30745) 2026-06-18 13:55:35 -07:00
realtime_api chore(lint): remove PLR0915 too-many-statements ruff rule (#30574) 2026-06-16 16:52:49 -07:00
repositories feat(litellm): add models and repository layers (#29686) 2026-06-06 20:59:33 -07:00
rerank_api chore(lint): remove PLR0915 too-many-statements ruff rule (#30574) 2026-06-16 16:52:49 -07:00
responses chore(lint): remove PLR0915 too-many-statements ruff rule (#30574) 2026-06-16 16:52:49 -07:00
router_strategy chore(lint): remove PLR0915 too-many-statements ruff rule (#30574) 2026-06-16 16:52:49 -07:00
router_utils fix(vertex/files): stream OpenAI->Vertex batch JSONL uploads (#31036) 2026-06-29 17:56:01 -07:00
sandbox feat(sandbox): e2b code execution primitive (#30898) 2026-06-20 16:30:01 -07:00
search Litellm oss staging 050626 (#29774) 2026-06-05 13:51:51 -07:00
secret_managers ci: drop mypy entirely, standardize type checking on basedpyright (#30648) 2026-06-17 09:42:00 -07:00
skills chore(proxy): scope skills and container resources 2026-04-30 18:23:58 -07:00
types fix(vertex_ai/files): single media upload for batch files to fix 499s on large uploads (#31653) 2026-06-29 18:10:47 -07:00
vector_store_files build(deps-dev): bump black to 26.3.1 and apply formatting (#28525) 2026-05-21 17:24:18 -07:00
vector_stores feat(litellm): add models and repository layers (#29686) 2026-06-06 20:59:33 -07:00
videos ci(lint): add blanket-noqa, dataclass-default, and unused-noqa Ruff rules (#30516) 2026-06-16 12:07:22 -07:00
__init__.py feat(sandbox): e2b code execution primitive (#30898) 2026-06-20 16:30:01 -07:00
_internal_context.py fix(proxy): harden request parameter handling 2026-04-16 01:38:12 +00:00
_lazy_imports.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
_lazy_imports_registry.py chore(oss): litellm oss staging 150626 (#30463) 2026-06-16 12:06:41 -07:00
_logging.py ci(lint): add blanket-noqa, dataclass-default, and unused-noqa Ruff rules (#30516) 2026-06-16 12:07:22 -07:00
_redis.py chore(lint): remove PLR0915 too-many-statements ruff rule (#30574) 2026-06-16 16:52:49 -07:00
_redis_credential_provider.py feat: add ability to auth to azure with token (#27556) 2026-05-09 22:34:09 +00:00
_service_logger.py fix: missing span for guardrail passthrough (#29552) 2026-06-03 01:25:15 +00:00
_uuid.py build(deps-dev): bump black to 26.3.1 and apply formatting (#28525) 2026-05-21 17:24:18 -07:00
_version.py
anthropic_beta_headers_config.json chore(oss): litellm oss staging 120626 (#30292) 2026-06-12 09:49:25 -07:00
anthropic_beta_headers_manager.py Refactor: Filtering beta header after transformation 2026-03-16 10:47:15 +05:30
blog_posts.json docs(blog): add WebRTC blog post link 2026-03-13 17:54:33 +05:30
budget_manager.py docs(budget_manager): add docstring to BudgetManager.reset_cost (#27867) 2026-05-13 13:28:22 -07:00
constants.py chore: litellm oss staging (#30745) 2026-06-18 13:55:35 -07:00
cost.json
cost_calculator.py fix(cost): stop non-string response service_tier from dropping cost tracking (#30706) 2026-06-17 19:35:57 -07:00
exceptions.py feat: standardize rate limit errors with category, rate_limit_type, model, and llm_provider fields (#27687) 2026-06-06 17:50:29 -07:00
main.py fix(health): correct bedrock embedding health checks (#30583) 2026-06-17 14:34:09 -07:00
model_prices_and_context_window_backup.json chore: litellm oss staging (#30745) 2026-06-18 13:55:35 -07:00
policy_templates_backup.json feat(add-new-block_code_execution-guardrail): prevent agent from executing code (#22154) 2026-02-25 22:02:14 -08:00
provider_endpoints_support_backup.json chore(oss): litellm oss staging 150626 (#30463) 2026-06-16 12:06:41 -07:00
py.typed
router.py chore: litellm oss 170626 (#30637) 2026-06-17 21:11:12 -07:00
scheduler.py style: run black formatter on entire codebase 2026-03-11 17:07:57 -03:00
setup_wizard.py Add Claude Fable 5 across Anthropic, Bedrock, Vertex AI, and Azure AI (#30064) 2026-06-10 08:50:15 +05:30
timeout.py docs: add class docstring to _LoopWrapper (#27870) 2026-05-13 13:54:00 -07:00
utils.py feat(sandbox): e2b code execution primitive (#30898) 2026-06-20 16:30:01 -07:00