scriptoriumadmin/litellm
1
0
Fork 0
mirror of https://github.com/BerriAI/litellm.git synced 2026-11-10 08:28:25 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 108
litellm/litellm/integrations
History
devin-ai-integration[bot] b55024658c
fix(proxy)!: stop update_cache from resurrecting evicted auth cache entries (#46997) 
* fix(proxy): stop update_cache from resurrecting evicted auth cache entries

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(proxy): update update_cache pins for synchronous write-back

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(proxy): wrap long assert in update_cache tag test

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(proxy): cover refill race and pin TTL with injected clock

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(proxy): count global proxy spend for every request with a user, cached on this worker or not

* fix(proxy): add global proxy spend with an atomic Redis increment that never re-creates a reset counter

The global proxy spend bump read the total, awaited, and wrote old + cost back
from a background task, so a budget reset that deleted the key in between got
the pre-reset total written back, and workers overwrote each other's totals.

RedisCache.async_increment_if_present runs INCRBYFLOAT only when the key
exists, keeping its TTL, and this worker's copy is then set from the shared
total with no await between its read and write. The bump now runs whenever a
proxy max_budget is set, matching the DB writer, and the armed post-call read
and the batch-read declaration it used are removed.

* fix(proxy): report the global spend Redis increment to service metrics and request traces

The global proxy spend increment now emits Redis service success and
failure events under the spend_counters target and passes the request
span through, so it shows as redis.incr spend_counters like the write
it replaced.

The unit tests run the real RedisCache increment against fakeredis
instead of mocks, and a new test checks the span lands under the
request span on success and on a Redis failure.

* test(proxy): keep the spend counter Redis fake off the network

* test(proxy): count increment_if_present as a shared-cache call in the key-family guard

* fix(proxy): keep the newer global spend total when Redis replies arrive out of order

---------

Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
2026-10-20 17:09:53 +00:00
..
_types style: unify ruff format width on 120 (#31518) 2026-06-27 12:39:29 -07:00
agentops refactor(llms): expose public names for private provider helpers (#45037) 2026-10-07 08:49:25 -07:00
arize fix(otel)!: leave out span cost when the price lookup failed (#44187) 2026-10-20 09:48:47 +00:00
azure_sentinel chore(lint): remove the LIT002 mutable-construction rule (#43971) 2026-10-01 12:24:02 -07:00
azure_storage feat(azure_storage): upload audit logs to Azure Storage via audit_log_callbacks (#43838) 2026-10-10 14:48:56 -07:00
bitbucket refactor(typing): clear 1143 basedpyright Any errors across 273 files (#46574) 2026-10-15 21:08:47 +00:00
clickhouse fix(proxy): honor disable_end_user_cost_tracking in spend logs and end-user tables (#40945) 2026-10-15 21:48:20 +00:00
cloudzero chore(typing): clear 670 basedpyright Any errors by validating provider payloads at the boundary (#45388) 2026-10-13 15:35:23 -07:00
code_interpreter_interception fix(websearch_interception): drop stream_options when forcing stream=False and replay it on the final stream (#45433) 2026-10-09 22:16:02 -07:00
compression_interception fix(proxy): run the remaining inline token counts off the event loop 2026-09-08 18:42:38 +00:00
datadog refactor(types): replace Any with proven types in 6 files (#47268) 2026-10-20 14:17:07 +00:00
deepeval perf(types): defer pydantic schema builds via shared LiteLLMBaseModel (#44720) 2026-10-05 22:10:08 -07:00
dotprompt chore(typing): clear 670 basedpyright Any errors by validating provider payloads at the boundary (#45388) 2026-10-13 15:35:23 -07:00
email_templates feat(proxy): email alerts at configured percentages of a team member budget (#42665) 2026-09-26 02:42:45 +00:00
focus refactor: clean up fresh tech debt in per-user Copilot auth, Ternary FOCUS export and telemetry (#45769) 2026-10-15 04:29:27 +00:00
gcs_bucket refactor(typing): clear 1143 basedpyright Any errors across 273 files (#46574) 2026-10-15 21:08:47 +00:00
gcs_pubsub refactor(proxy): expose public names for private proxy helpers (#45170) 2026-10-07 18:59:27 -07:00
generic_api refactor(typing): clear 1143 basedpyright Any errors across 273 files (#46574) 2026-10-15 21:08:47 +00:00
generic_prompt_management refactor(llms): expose public names for private provider helpers (#45037) 2026-10-07 08:49:25 -07:00
gitlab fix(proxy)!: count endpoints return errors in their source API shape instead of silent local counts (#46815) 2026-10-16 00:11:39 -07:00
langfuse fix(langfuse_otel)!: send the completion start time (#45597) 2026-10-20 13:52:26 +00:00
levo test: move tests/test_litellm integrations and secret_managers into tests/unit (#43194) 2026-09-25 12:57:07 -07:00
litellm_agent feat(lint): enforce Final on locals and freeze function parameters (LIT010, LIT011) 2026-08-04 12:54:39 -07:00
mavvrik_focus refactor(typing): clear 1143 basedpyright Any errors across 273 files (#46574) 2026-10-15 21:08:47 +00:00
newrelic chore(lint): remove the LIT002 mutable-construction rule (#43971) 2026-10-01 12:24:02 -07:00
opentelemetry_utils refactor(typing): clear 1143 basedpyright Any errors across 273 files (#46574) 2026-10-15 21:08:47 +00:00
opik refactor(llms): expose public names for private provider helpers (#45037) 2026-10-07 08:49:25 -07:00
otel fix(proxy)!: stop update_cache from resurrecting evicted auth cache entries (#46997) 2026-10-20 17:09:53 +00:00
pointfive perf(types): defer pydantic schema builds via shared LiteLLMBaseModel (#44720) 2026-10-05 22:10:08 -07:00
prometheus_helpers refactor: expose core private helpers under public names (#44871) 2026-10-07 00:05:27 +00:00
SlackAlerting refactor(sdk): import UserAPIKeyAuth from litellm.types.proxy.auth (#45806) 2026-10-10 21:27:48 +00:00
ternary refactor: clean up fresh tech debt in per-user Copilot auth, Ternary FOCUS export and telemetry (#45769) 2026-10-15 04:29:27 +00:00
vantage refactor(typing): clear 1143 basedpyright Any errors across 273 files (#46574) 2026-10-15 21:08:47 +00:00
vector_store_integrations refactor(types): replace Any with proven types in 137 files (#44478) 2026-10-05 11:01:45 -07:00
weave refactor: expose core private helpers under public names (#44871) 2026-10-07 00:05:27 +00:00
websearch_interception fix(exa): return highlights, highlight scores, score and the response envelope from Exa search (#47423) 2026-10-19 15:45:56 -07:00
zerobus perf(types): defer pydantic schema builds via shared LiteLLMBaseModel (#44720) 2026-10-05 22:10:08 -07:00
__init__.py add linting 2023-08-18 11:05:05 -07:00
adaptive_concurrency.py fix(s3_v2): upload fresh events first, drop terminal failures and hour-old retries by default, opt-in adaptive concurrency (#43022) 2026-09-26 14:58:28 -07:00
additional_logging_utils.py refactor(lint): apply every safe ruff autofix and zero 28 strict-rule budgets 2026-08-01 15:43:29 -07:00
anthropic_cache_control_hook.py fix(anthropic, bedrock)!: keep a redacted_thinking content item on every Claude route (#47489) 2026-10-20 15:00:31 +00:00
argilla.py chore(typing): clear 670 basedpyright Any errors by validating provider payloads at the boundary (#45388) 2026-10-13 15:35:23 -07:00
athina.py feat(lint): enforce Final on locals and freeze function parameters (LIT010, LIT011) 2026-08-04 12:54:39 -07:00
batch_line_item_objects.py feat(batches): store each batch line's request and response in S3 and GCS (#41691) 2026-10-15 11:32:56 -07:00
batch_utils.py fix(azure_sentinel): split batches under the 1MB ingestion cap (#39880) 2026-09-05 17:15:36 -07:00
braintrust_logging.py refactor: replace Any with precise types across 54 modules 2026-09-14 11:13:39 +00:00
braintrust_mock_client.py chore(typing): clear 1.6k basedpyright Any errors across 56 files 2026-08-11 06:47:39 -07:00
callback_configs.json feat(s3_v2): add s3_partition_granularity option for hourly S3 folders (#43748) 2026-10-01 11:00:21 -07:00
custom_batch_logger.py feat(tracing): add OTLP trace ingestion and reads (#43915) 2026-09-30 21:12:29 +00:00
custom_guardrail.py fix(guardrails)!: preserve Bedrock Converse redactions and honor tool-only scans (#45874) 2026-10-17 20:38:14 -07:00
custom_logger.py fix(logging): deliver completed batch success callbacks consistently (#47192) 2026-10-18 09:19:04 -07:00
custom_prompt_management.py refactor(lint): apply every safe ruff autofix and zero 28 strict-rule budgets 2026-08-01 15:43:29 -07:00
custom_secret_manager.py perf: build log messages lazily so filtered-out log records cost nothing (#35703) 2026-08-04 04:34:52 +00:00
custom_sso_handler.py feat(lint): enforce Final on locals and freeze function parameters (LIT010, LIT011) 2026-08-04 12:54:39 -07:00
dynamodb.py refactor: expose core private helpers under public names (#44871) 2026-10-07 00:05:27 +00:00
email_alerting.py refactor(types): move slack alerting leaf types out of proxy/_types.py (#45752) 2026-10-10 19:06:22 +00:00
galileo.py perf(types): defer pydantic schema builds via shared LiteLLMBaseModel (#44720) 2026-10-05 22:10:08 -07:00
greenscale.py feat(lint): enforce Final on locals and freeze function parameters (LIT010, LIT011) 2026-08-04 12:54:39 -07:00
helicone.py feat(lint): enforce Final on locals and freeze function parameters (LIT010, LIT011) 2026-08-04 12:54:39 -07:00
helicone_mock_client.py feat(lint): enforce Final on locals and freeze function parameters (LIT010, LIT011) 2026-08-04 12:54:39 -07:00
humanloop.py refactor(llms): expose public names for private provider helpers (#45037) 2026-10-07 08:49:25 -07:00
lago.py refactor(typing): clear 1143 basedpyright Any errors across 273 files (#46574) 2026-10-15 21:08:47 +00:00
langsmith.py refactor(typing): clear 1143 basedpyright Any errors across 273 files (#46574) 2026-10-15 21:08:47 +00:00
langsmith_mock_client.py feat(lint): enforce Final on locals and freeze function parameters (LIT010, LIT011) 2026-08-04 12:54:39 -07:00
langtrace.py refactor(types): move SpanAttributes, SpecialHeaders and AllowedModelRegion out of proxy._types (#44717) 2026-10-05 21:40:13 -07:00
literal_ai.py refactor(typing): clear 1143 basedpyright Any errors across 273 files (#46574) 2026-10-15 21:08:47 +00:00
logfire_logger.py feat(lint): enforce Final on locals and freeze function parameters (LIT010, LIT011) 2026-08-04 12:54:39 -07:00
lunary.py refactor(typing): clear 1143 basedpyright Any errors across 273 files (#46574) 2026-10-15 21:08:47 +00:00
mlflow.py fix(mlflow): prevent _stream_id_to_span leak and mlflow 2.x end_trace TypeError (#39049) 2026-09-08 16:18:06 -07:00
mock_client_factory.py chore(typing): clear 1.6k basedpyright Any errors across 56 files 2026-08-11 06:47:39 -07:00
openmeter.py chore(lint): strip inert type: ignore comments and zero LIT009, LIT010, LIT011 headroom 2026-08-05 02:37:24 -07:00
opentelemetry.py fix(otel): put the error message on every failed span status (#44185) 2026-10-20 06:35:47 +00:00
posthog.py refactor(llms): expose public names for private provider helpers (#45037) 2026-10-07 08:49:25 -07:00
posthog_mock_client.py feat(lint): enforce Final on locals and freeze function parameters (LIT010, LIT011) 2026-08-04 12:54:39 -07:00
prometheus.py fix(prometheus): preserve requested model after fallback (#45555) 2026-10-19 23:41:41 -07:00
prometheus_metrics_endpoint.py perf(prometheus): render /metrics off the event loop and coalesce concurrent scrapes (#37702) 2026-08-20 16:08:22 -07:00
prometheus_services.py fix(caching): report Redis circuit-breaker metrics through the Prometheus services logger (#46484) 2026-10-14 14:29:54 -07:00
prompt_layer.py feat(lint): enforce Final on locals and freeze function parameters (LIT010, LIT011) 2026-08-04 12:54:39 -07:00
prompt_management_base.py fix(prompts): apply prompt templates before routing on /v1/responses and honor ignore_prompt_manager_model 2026-08-26 14:12:28 -07:00
Readme.md (Feat) - Add GCS Pub/Sub Logging integration for sending DB SpendLogs to BigQuery (#7976) 2025-01-24 20:57:20 -08:00
rubrik.py refactor(sdk): import UserAPIKeyAuth from litellm.types.proxy.auth (#45806) 2026-10-10 21:27:48 +00:00
s3.py fix(s3): bound object key filenames (#40825) 2026-10-15 12:59:35 -07:00
s3_v2.py feat(batches): store each batch line's request and response in S3 and GCS (#41691) 2026-10-15 11:32:56 -07:00
shadow_eval_logger.py refactor(typing): clear 1143 basedpyright Any errors across 273 files (#46574) 2026-10-15 21:08:47 +00:00
sqs.py fix(bedrock): sign on a dedicated executor instead of the shared default one 2026-09-09 18:52:00 -07:00
supabase.py chore(lint): strip inert type: ignore comments and zero LIT009, LIT010, LIT011 headroom 2026-08-05 02:37:24 -07:00
test_httpx.py fix(utils.py): improved predibase exception mapping 2024-06-08 14:32:43 -07:00
traceloop.py chore(lint): strip inert type: ignore comments and zero LIT009, LIT010, LIT011 headroom 2026-08-05 02:37:24 -07:00
weights_biases.py refactor(types): replace Any with proven types in 32 files 2026-09-21 10:51:46 +00:00

Readme.md

Integrations

This folder contains logging integrations for litellm

eg. logging to Datadog, Langfuse, Prometheus, s3, GCS Bucket, etc.