litellm/litellm/integrations
paveln83 4ce724e3df
fix(caching): key cached async clients by a per-loop token instead of id(loop) (#48877)
* fix(caching): key cached async clients by a per-loop token instead of id(loop)

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(caching): drop deferred closes queued on a collected event loop

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(caching): pin that a close queued on a live loop waits for that loop

* test(caching): run loop-reuse tests until a loop address repeats

The collected-loop tests ran a fixed 50 loops and passed with the fix reverted whenever no loop reused an earlier address. They now stop at the first reuse and skip if none happens, through a shared helper.

* test(caching): cover the cache key fallback for loops without weak references

Also corrects the evicted client closer docstring: a dropped bucket's clients are left to the garbage collector.

* fix(http): free short-lived event loops whose handler finalizer close finished as the loop stopped

* fix(caching): share sync clients across event loops and catch a failed finalizer close

* fix(agentops): stop closing the shared sync http client after the auth call

---------

Co-authored-by: mateo <mateo@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
2026-10-29 13:14:48 -07:00
..
_types style: unify ruff format width on 120 (#31518) 2026-06-27 12:39:29 -07:00
agentops fix(caching): key cached async clients by a per-loop token instead of id(loop) (#48877) 2026-10-29 13:14:48 -07:00
arize refactor(typing): clear 1181 basedpyright Any errors across 431 files (#47789) 2026-10-24 11:31:09 -07:00
azure_sentinel chore(lint): remove the LIT002 mutable-construction rule (#43971) 2026-10-01 12:24:02 -07:00
azure_storage feat(azure_storage): upload audit logs to Azure Storage via audit_log_callbacks (#43838) 2026-10-10 14:48:56 -07:00
bitbucket refactor(typing): clear 1143 basedpyright Any errors across 273 files (#46574) 2026-10-15 21:08:47 +00:00
clickhouse fix(proxy): honor disable_end_user_cost_tracking in spend logs and end-user tables (#40945) 2026-10-15 21:48:20 +00:00
cloudzero refactor(typing): clear 1181 basedpyright Any errors across 431 files (#47789) 2026-10-24 11:31:09 -07:00
code_interpreter_interception fix(websearch_interception): drop stream_options when forcing stream=False and replay it on the final stream (#45433) 2026-10-09 22:16:02 -07:00
compression_interception fix(proxy): run the remaining inline token counts off the event loop 2026-09-08 18:42:38 +00:00
datadog fix(logging): mark mirrored silent_model calls with is_silent_experiment in spend logs (#48888) 2026-10-29 05:12:50 -07:00
deepeval refactor(types): replace Any with proven types in 50 files (#47818) 2026-10-21 11:58:31 +00:00
dotprompt refactor(typing): clear 1181 basedpyright Any errors across 431 files (#47789) 2026-10-24 11:31:09 -07:00
email_templates feat(proxy): email alerts at configured percentages of a team member budget (#42665) 2026-09-26 02:42:45 +00:00
focus refactor(typing): clear 1181 basedpyright Any errors across 431 files (#47789) 2026-10-24 11:31:09 -07:00
gcs_bucket fix(security)!: build proxy-only failure logs from an allowlist and drop caller guardrail info (#48599) 2026-10-27 12:40:59 -07:00
gcs_pubsub refactor(typing): clear 1181 basedpyright Any errors across 431 files (#47789) 2026-10-24 11:31:09 -07:00
generic_api refactor(typing): clear 1143 basedpyright Any errors across 273 files (#46574) 2026-10-15 21:08:47 +00:00
generic_prompt_management refactor(llms): expose public names for private provider helpers (#45037) 2026-10-07 08:49:25 -07:00
gitlab fix(proxy)!: count endpoints return errors in their source API shape instead of silent local counts (#46815) 2026-10-16 00:11:39 -07:00
langfuse fix(otel)!: give key/team Langfuse destinations user, tags and session on OTel v2 without global Langfuse keys (#48718) 2026-10-29 12:19:08 -07:00
levo refactor(typing): widen delete_nested_value to Mapping and drop two unused Span aliases (#48001) 2026-10-24 19:39:27 +00:00
litellm_agent feat(lint): enforce Final on locals and freeze function parameters (LIT010, LIT011) 2026-08-04 12:54:39 -07:00
mavvrik_focus refactor(typing): clear 1143 basedpyright Any errors across 273 files (#46574) 2026-10-15 21:08:47 +00:00
newrelic refactor(typing): clear 1181 basedpyright Any errors across 431 files (#47789) 2026-10-24 11:31:09 -07:00
opentelemetry_utils refactor(types): replace Any with proven types in 14 files (#48928) 2026-10-29 07:26:07 -07:00
opik refactor(typing): clear 1181 basedpyright Any errors across 431 files (#47789) 2026-10-24 11:31:09 -07:00
otel fix(otel)!: print OTel v2 spans to stdout only when the operator asks (#48723) 2026-10-29 12:20:07 -07:00
pointfive perf(types): defer pydantic schema builds via shared LiteLLMBaseModel (#44720) 2026-10-05 22:10:08 -07:00
prometheus_helpers refactor: expose core private helpers under public names (#44871) 2026-10-07 00:05:27 +00:00
SlackAlerting refactor(router): move router callers onto the entry points and invert the proxy imports (#48847) 2026-10-29 17:26:33 +00:00
ternary refactor: clean up fresh tech debt in per-user Copilot auth, Ternary FOCUS export and telemetry (#45769) 2026-10-15 04:29:27 +00:00
vantage refactor(typing): clear 1143 basedpyright Any errors across 273 files (#46574) 2026-10-15 21:08:47 +00:00
vector_store_integrations refactor(types): replace Any with proven types in 137 files (#44478) 2026-10-05 11:01:45 -07:00
weave refactor: expose core private helpers under public names (#44871) 2026-10-07 00:05:27 +00:00
websearch_interception refactor(router): move router callers onto the entry points and invert the proxy imports (#48847) 2026-10-29 17:26:33 +00:00
zerobus perf(types): defer pydantic schema builds via shared LiteLLMBaseModel (#44720) 2026-10-05 22:10:08 -07:00
__init__.py add linting 2023-08-18 11:05:05 -07:00
adaptive_concurrency.py fix(s3_v2): upload fresh events first, drop terminal failures and hour-old retries by default, opt-in adaptive concurrency (#43022) 2026-09-26 14:58:28 -07:00
additional_logging_utils.py refactor(lint): apply every safe ruff autofix and zero 28 strict-rule budgets 2026-08-01 15:43:29 -07:00
anthropic_cache_control_hook.py fix(responses): drop a malformed prompt_cache_breakpoint under drop_params on the direct /v1/responses path (#45330) 2026-10-29 03:37:54 -07:00
argilla.py chore(typing): clear 670 basedpyright Any errors by validating provider payloads at the boundary (#45388) 2026-10-13 15:35:23 -07:00
athina.py feat(lint): enforce Final on locals and freeze function parameters (LIT010, LIT011) 2026-08-04 12:54:39 -07:00
batch_line_item_objects.py feat(batches): store each batch line's request and response in S3 and GCS (#41691) 2026-10-15 11:32:56 -07:00
batch_utils.py fix(azure_sentinel): split batches under the 1MB ingestion cap (#39880) 2026-09-05 17:15:36 -07:00
braintrust_logging.py refactor(typing): clear 1181 basedpyright Any errors across 431 files (#47789) 2026-10-24 11:31:09 -07:00
braintrust_mock_client.py chore(typing): clear 1.6k basedpyright Any errors across 56 files 2026-08-11 06:47:39 -07:00
callback_configs.json feat(s3_v2): add s3_partition_granularity option for hourly S3 folders (#43748) 2026-10-01 11:00:21 -07:00
custom_batch_logger.py feat(tracing): add OTLP trace ingestion and reads (#43915) 2026-09-30 21:12:29 +00:00
custom_guardrail.py refactor(typing): clear 1181 basedpyright Any errors across 431 files (#47789) 2026-10-24 11:31:09 -07:00
custom_logger.py refactor(types): replace Any with proven types in 14 files (#48928) 2026-10-29 07:26:07 -07:00
custom_prompt_management.py refactor(lint): apply every safe ruff autofix and zero 28 strict-rule budgets 2026-08-01 15:43:29 -07:00
custom_secret_manager.py refactor(typing): clear 1181 basedpyright Any errors across 431 files (#47789) 2026-10-24 11:31:09 -07:00
custom_sso_handler.py feat(lint): enforce Final on locals and freeze function parameters (LIT010, LIT011) 2026-08-04 12:54:39 -07:00
dynamodb.py refactor: expose core private helpers under public names (#44871) 2026-10-07 00:05:27 +00:00
email_alerting.py refactor(types): move slack alerting leaf types out of proxy/_types.py (#45752) 2026-10-10 19:06:22 +00:00
galileo.py perf(types): defer pydantic schema builds via shared LiteLLMBaseModel (#44720) 2026-10-05 22:10:08 -07:00
greenscale.py feat(lint): enforce Final on locals and freeze function parameters (LIT010, LIT011) 2026-08-04 12:54:39 -07:00
helicone.py feat(lint): enforce Final on locals and freeze function parameters (LIT010, LIT011) 2026-08-04 12:54:39 -07:00
helicone_mock_client.py feat(lint): enforce Final on locals and freeze function parameters (LIT010, LIT011) 2026-08-04 12:54:39 -07:00
humanloop.py refactor(llms): expose public names for private provider helpers (#45037) 2026-10-07 08:49:25 -07:00
lago.py refactor(typing): clear 1143 basedpyright Any errors across 273 files (#46574) 2026-10-15 21:08:47 +00:00
langsmith.py refactor(typing): clear 1181 basedpyright Any errors across 431 files (#47789) 2026-10-24 11:31:09 -07:00
langsmith_mock_client.py feat(lint): enforce Final on locals and freeze function parameters (LIT010, LIT011) 2026-08-04 12:54:39 -07:00
langtrace.py refactor(typing): clear 1181 basedpyright Any errors across 431 files (#47789) 2026-10-24 11:31:09 -07:00
literal_ai.py refactor(typing): clear 1143 basedpyright Any errors across 273 files (#46574) 2026-10-15 21:08:47 +00:00
logfire_logger.py refactor(typing): clear 1181 basedpyright Any errors across 431 files (#47789) 2026-10-24 11:31:09 -07:00
lunary.py refactor(typing): clear 1143 basedpyright Any errors across 273 files (#46574) 2026-10-15 21:08:47 +00:00
mlflow.py fix(mlflow): prevent _stream_id_to_span leak and mlflow 2.x end_trace TypeError (#39049) 2026-09-08 16:18:06 -07:00
mock_client_factory.py chore(typing): clear 1.6k basedpyright Any errors across 56 files 2026-08-11 06:47:39 -07:00
openmeter.py chore(lint): strip inert type: ignore comments and zero LIT009, LIT010, LIT011 headroom 2026-08-05 02:37:24 -07:00
opentelemetry.py fix(otel): put the error message on every failed span status (#44185) 2026-10-20 06:35:47 +00:00
posthog.py refactor(llms): expose public names for private provider helpers (#45037) 2026-10-07 08:49:25 -07:00
posthog_mock_client.py feat(lint): enforce Final on locals and freeze function parameters (LIT010, LIT011) 2026-08-04 12:54:39 -07:00
prometheus.py refactor(router): move router callers onto the entry points and invert the proxy imports (#48847) 2026-10-29 17:26:33 +00:00
prometheus_metrics_endpoint.py perf(prometheus): render /metrics off the event loop and coalesce concurrent scrapes (#37702) 2026-08-20 16:08:22 -07:00
prometheus_services.py fix(caching): report Redis circuit-breaker metrics through the Prometheus services logger (#46484) 2026-10-14 14:29:54 -07:00
prompt_layer.py feat(lint): enforce Final on locals and freeze function parameters (LIT010, LIT011) 2026-08-04 12:54:39 -07:00
prompt_management_base.py fix(prompts): apply prompt templates before routing on /v1/responses and honor ignore_prompt_manager_model 2026-08-26 14:12:28 -07:00
Readme.md (Feat) - Add GCS Pub/Sub Logging integration for sending DB SpendLogs to BigQuery (#7976) 2025-01-24 20:57:20 -08:00
rubrik.py refactor(sdk): import UserAPIKeyAuth from litellm.types.proxy.auth (#45806) 2026-10-10 21:27:48 +00:00
s3.py fix(s3): bound object key filenames (#40825) 2026-10-15 12:59:35 -07:00
s3_v2.py feat(batches): store each batch line's request and response in S3 and GCS (#41691) 2026-10-15 11:32:56 -07:00
shadow_eval_logger.py refactor(router): move router callers onto the entry points and invert the proxy imports (#48847) 2026-10-29 17:26:33 +00:00
sqs.py refactor(typing): clear 1181 basedpyright Any errors across 431 files (#47789) 2026-10-24 11:31:09 -07:00
supabase.py chore(lint): strip inert type: ignore comments and zero LIT009, LIT010, LIT011 headroom 2026-08-05 02:37:24 -07:00
test_httpx.py fix(utils.py): improved predibase exception mapping 2024-06-08 14:32:43 -07:00
traceloop.py chore(lint): strip inert type: ignore comments and zero LIT009, LIT010, LIT011 headroom 2026-08-05 02:37:24 -07:00
weights_biases.py refactor(types): replace Any with proven types in 32 files 2026-09-21 10:51:46 +00:00

Integrations

This folder contains logging integrations for litellm

eg. logging to Datadog, Langfuse, Prometheus, s3, GCS Bucket, etc.