scriptoriumadmin/litellm
1
0
Fork 0
mirror of https://github.com/BerriAI/litellm.git synced 2026-11-04 07:27:51 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 195
litellm/litellm/integrations
History
Vineeth Sai Varikuntla a7b66dd8b9
fix(openai): stop stripping cache_control from the caller's own messages and tools (#35602) 
* fix(openai): stop stripping cache_control from the caller's messages

Copy before removing the field, so the caller's own message objects are left alone.

* fix(fireworks_ai): strip cache_control on a copy so a fallback keeps the caller's hints

* fix: keep the caller's request intact on more providers and trim extra Claude cache hints under drop_params

* fix: drop only the oldest extra cache hints inside one message when trimming to the Claude limit

* fix: keep replayed session turns yielding their hints as whole turns while a request's own message keeps its newest hints

* fix(cache_control): count only hints that reach Claude and cap hints on OpenAI-compatible Claude routes under drop_params

* fix(cache_control): keep the capped Claude messages a list in OpenAI-compatible request bodies

* fix(caching): cap Claude hints for string drop_params, a Claude base_model, and Converse tool messages

The 4-hint cap now reads drop_params through drops_unsupported_params on chat, Converse, and Anthropic routes, names Claude by base_model on OpenAI-compatible routes, and counts a Converse tool message's hints as the one cache point it sends.

* fix(caching): trim Claude cache hints in one wire-order pass and reserve a top-level cache_control on OpenAI-compatible routes

Dropping the oldest hint one at a time re-walked the whole request per hint, which took seconds on long tool histories. The cap now drops each message's oldest hints in the order the provider counts them, so a string assistant turn loses its text hint before its tool call hints. OpenAI-compatible Claude routes also keep room for a request-level or extra_body cache_control under drop_params.

* test: drop narrating comments and stacked generator clauses from cache_control tests

* fix(vertex_ai): put every system message in the Gemini context cache so generateContent never sends it beside cachedContent

---------

Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
2026-10-31 00:29:10 -07:00
..
_types style: unify ruff format width on 120 (#31518) 2026-06-27 12:39:29 -07:00
agentops fix(caching): key cached async clients by a per-loop token instead of id(loop) (#48877) 2026-10-29 13:14:48 -07:00
arize refactor(types): replace Any with proven types in 21 files (#48921) 2026-10-29 18:43:24 -07:00
azure_sentinel chore(lint): remove the LIT002 mutable-construction rule (#43971) 2026-10-01 12:24:02 -07:00
azure_storage feat(azure_storage): upload audit logs to Azure Storage via audit_log_callbacks (#43838) 2026-10-10 14:48:56 -07:00
bitbucket refactor(typing): clear 1143 basedpyright Any errors across 273 files (#46574) 2026-10-15 21:08:47 +00:00
clickhouse fix(proxy): honor disable_end_user_cost_tracking in spend logs and end-user tables (#40945) 2026-10-15 21:48:20 +00:00
cloudzero refactor(typing): clear 1181 basedpyright Any errors across 431 files (#47789) 2026-10-24 11:31:09 -07:00
code_interpreter_interception fix(websearch_interception): drop stream_options when forcing stream=False and replay it on the final stream (#45433) 2026-10-09 22:16:02 -07:00
compression_interception fix(proxy): run the remaining inline token counts off the event loop 2026-09-08 18:42:38 +00:00
datadog fix(logging): mark mirrored silent_model calls with is_silent_experiment in spend logs (#48888) 2026-10-29 05:12:50 -07:00
deepeval refactor(types): replace Any with proven types in 50 files (#47818) 2026-10-21 11:58:31 +00:00
dotprompt refactor(typing): clear 1181 basedpyright Any errors across 431 files (#47789) 2026-10-24 11:31:09 -07:00
email_templates feat(proxy): email alerts at configured percentages of a team member budget (#42665) 2026-09-26 02:42:45 +00:00
focus refactor(typing): clear 1181 basedpyright Any errors across 431 files (#47789) 2026-10-24 11:31:09 -07:00
gcs_bucket fix(security)!: build proxy-only failure logs from an allowlist and drop caller guardrail info (#48599) 2026-10-27 12:40:59 -07:00
gcs_pubsub refactor(typing): clear 1181 basedpyright Any errors across 431 files (#47789) 2026-10-24 11:31:09 -07:00
generic_api refactor(typing): clear 1143 basedpyright Any errors across 273 files (#46574) 2026-10-15 21:08:47 +00:00
generic_prompt_management refactor(llms): expose public names for private provider helpers (#45037) 2026-10-07 08:49:25 -07:00
gitlab fix(proxy)!: count endpoints return errors in their source API shape instead of silent local counts (#46815) 2026-10-16 00:11:39 -07:00
langfuse fix(otel)!: give key/team Langfuse destinations user, tags and session on OTel v2 without global Langfuse keys (#48718) 2026-10-29 12:19:08 -07:00
levo refactor(typing): widen delete_nested_value to Mapping and drop two unused Span aliases (#48001) 2026-10-24 19:39:27 +00:00
litellm_agent feat(lint): enforce Final on locals and freeze function parameters (LIT010, LIT011) 2026-08-04 12:54:39 -07:00
mavvrik_focus refactor(typing): clear 1143 basedpyright Any errors across 273 files (#46574) 2026-10-15 21:08:47 +00:00
newrelic refactor(typing): clear 1181 basedpyright Any errors across 431 files (#47789) 2026-10-24 11:31:09 -07:00
opentelemetry_utils refactor(types): replace Any with proven types in 14 files (#48928) 2026-10-29 07:26:07 -07:00
opik refactor(typing): clear 1181 basedpyright Any errors across 431 files (#47789) 2026-10-24 11:31:09 -07:00
otel fix(router): budget classifier input by provider limits (#49306) 2026-10-30 14:38:14 -07:00
pointfive perf(types): defer pydantic schema builds via shared LiteLLMBaseModel (#44720) 2026-10-05 22:10:08 -07:00
prometheus_helpers refactor: expose core private helpers under public names (#44871) 2026-10-07 00:05:27 +00:00
SlackAlerting refactor(router): move router callers onto the entry points and invert the proxy imports (#48847) 2026-10-29 17:26:33 +00:00
ternary refactor: clean up fresh tech debt in per-user Copilot auth, Ternary FOCUS export and telemetry (#45769) 2026-10-15 04:29:27 +00:00
vantage refactor(typing): clear 1143 basedpyright Any errors across 273 files (#46574) 2026-10-15 21:08:47 +00:00
vector_store_integrations refactor(types): replace Any with proven types in 137 files (#44478) 2026-10-05 11:01:45 -07:00
weave refactor: expose core private helpers under public names (#44871) 2026-10-07 00:05:27 +00:00
websearch_interception fix(websearch-interception)!: stream native server-tool blocks to clients (#40967) 2026-10-30 00:06:57 -07:00
zerobus perf(types): defer pydantic schema builds via shared LiteLLMBaseModel (#44720) 2026-10-05 22:10:08 -07:00
__init__.py add linting 2023-08-18 11:05:05 -07:00
adaptive_concurrency.py fix(s3_v2): upload fresh events first, drop terminal failures and hour-old retries by default, opt-in adaptive concurrency (#43022) 2026-09-26 14:58:28 -07:00
additional_logging_utils.py refactor(lint): apply every safe ruff autofix and zero 28 strict-rule budgets 2026-08-01 15:43:29 -07:00
anthropic_cache_control_hook.py fix(openai): stop stripping cache_control from the caller's own messages and tools (#35602) 2026-10-31 00:29:10 -07:00
argilla.py chore(typing): clear 670 basedpyright Any errors by validating provider payloads at the boundary (#45388) 2026-10-13 15:35:23 -07:00
athina.py feat(lint): enforce Final on locals and freeze function parameters (LIT010, LIT011) 2026-08-04 12:54:39 -07:00
batch_line_item_objects.py feat(batches): store each batch line's request and response in S3 and GCS (#41691) 2026-10-15 11:32:56 -07:00
batch_utils.py fix(azure_sentinel): split batches under the 1MB ingestion cap (#39880) 2026-09-05 17:15:36 -07:00
braintrust_logging.py refactor(typing): clear 1181 basedpyright Any errors across 431 files (#47789) 2026-10-24 11:31:09 -07:00
braintrust_mock_client.py chore(typing): clear 1.6k basedpyright Any errors across 56 files 2026-08-11 06:47:39 -07:00
callback_configs.json feat(s3_v2): add s3_partition_granularity option for hourly S3 folders (#43748) 2026-10-01 11:00:21 -07:00
custom_batch_logger.py feat(tracing): add OTLP trace ingestion and reads (#43915) 2026-09-30 21:12:29 +00:00
custom_guardrail.py fix(guardrails): name only the request pipeline steps that ran in x-litellm-applied-guardrails (#49262) 2026-10-30 23:13:37 +00:00
custom_logger.py refactor(types): replace Any with proven types in 14 files (#48928) 2026-10-29 07:26:07 -07:00
custom_prompt_management.py refactor(lint): apply every safe ruff autofix and zero 28 strict-rule budgets 2026-08-01 15:43:29 -07:00
custom_secret_manager.py refactor(typing): clear 1181 basedpyright Any errors across 431 files (#47789) 2026-10-24 11:31:09 -07:00
custom_sso_handler.py feat(lint): enforce Final on locals and freeze function parameters (LIT010, LIT011) 2026-08-04 12:54:39 -07:00
dynamodb.py refactor: expose core private helpers under public names (#44871) 2026-10-07 00:05:27 +00:00
email_alerting.py refactor(types): move slack alerting leaf types out of proxy/_types.py (#45752) 2026-10-10 19:06:22 +00:00
galileo.py perf(types): defer pydantic schema builds via shared LiteLLMBaseModel (#44720) 2026-10-05 22:10:08 -07:00
greenscale.py feat(lint): enforce Final on locals and freeze function parameters (LIT010, LIT011) 2026-08-04 12:54:39 -07:00
helicone.py feat(lint): enforce Final on locals and freeze function parameters (LIT010, LIT011) 2026-08-04 12:54:39 -07:00
helicone_mock_client.py feat(lint): enforce Final on locals and freeze function parameters (LIT010, LIT011) 2026-08-04 12:54:39 -07:00
humanloop.py refactor(llms): expose public names for private provider helpers (#45037) 2026-10-07 08:49:25 -07:00
lago.py refactor(typing): clear 1143 basedpyright Any errors across 273 files (#46574) 2026-10-15 21:08:47 +00:00
langsmith.py refactor(typing): clear 1181 basedpyright Any errors across 431 files (#47789) 2026-10-24 11:31:09 -07:00
langsmith_mock_client.py feat(lint): enforce Final on locals and freeze function parameters (LIT010, LIT011) 2026-08-04 12:54:39 -07:00
langtrace.py refactor(typing): clear 1181 basedpyright Any errors across 431 files (#47789) 2026-10-24 11:31:09 -07:00
literal_ai.py refactor(typing): clear 1143 basedpyright Any errors across 273 files (#46574) 2026-10-15 21:08:47 +00:00
logfire_logger.py refactor(typing): clear 1181 basedpyright Any errors across 431 files (#47789) 2026-10-24 11:31:09 -07:00
lunary.py refactor(typing): clear 1143 basedpyright Any errors across 273 files (#46574) 2026-10-15 21:08:47 +00:00
mlflow.py fix(mlflow): prevent _stream_id_to_span leak and mlflow 2.x end_trace TypeError (#39049) 2026-09-08 16:18:06 -07:00
mock_client_factory.py chore(typing): clear 1.6k basedpyright Any errors across 56 files 2026-08-11 06:47:39 -07:00
openmeter.py chore(lint): strip inert type: ignore comments and zero LIT009, LIT010, LIT011 headroom 2026-08-05 02:37:24 -07:00
opentelemetry.py refactor(types): replace Any with proven types in 21 files (#48921) 2026-10-29 18:43:24 -07:00
posthog.py refactor(llms): expose public names for private provider helpers (#45037) 2026-10-07 08:49:25 -07:00
posthog_mock_client.py feat(lint): enforce Final on locals and freeze function parameters (LIT010, LIT011) 2026-08-04 12:54:39 -07:00
prometheus.py refactor(router): route the router family through RoutingContext and close the outside surface (#49118) 2026-10-30 19:18:54 -07:00
prometheus_metrics_endpoint.py perf(prometheus): render /metrics off the event loop and coalesce concurrent scrapes (#37702) 2026-08-20 16:08:22 -07:00
prometheus_services.py fix(caching): report Redis circuit-breaker metrics through the Prometheus services logger (#46484) 2026-10-14 14:29:54 -07:00
prompt_layer.py feat(lint): enforce Final on locals and freeze function parameters (LIT010, LIT011) 2026-08-04 12:54:39 -07:00
prompt_management_base.py fix(prompts): apply prompt templates before routing on /v1/responses and honor ignore_prompt_manager_model 2026-08-26 14:12:28 -07:00
Readme.md (Feat) - Add GCS Pub/Sub Logging integration for sending DB SpendLogs to BigQuery (#7976) 2025-01-24 20:57:20 -08:00
rubrik.py refactor(sdk): import UserAPIKeyAuth from litellm.types.proxy.auth (#45806) 2026-10-10 21:27:48 +00:00
s3.py fix(s3): bound object key filenames (#40825) 2026-10-15 12:59:35 -07:00
s3_v2.py feat(batches): store each batch line's request and response in S3 and GCS (#41691) 2026-10-15 11:32:56 -07:00
shadow_eval_logger.py refactor(router): move router callers onto the entry points and invert the proxy imports (#48847) 2026-10-29 17:26:33 +00:00
sqs.py refactor(typing): clear 1181 basedpyright Any errors across 431 files (#47789) 2026-10-24 11:31:09 -07:00
supabase.py chore(lint): strip inert type: ignore comments and zero LIT009, LIT010, LIT011 headroom 2026-08-05 02:37:24 -07:00
test_httpx.py fix(utils.py): improved predibase exception mapping 2024-06-08 14:32:43 -07:00
traceloop.py chore(lint): strip inert type: ignore comments and zero LIT009, LIT010, LIT011 headroom 2026-08-05 02:37:24 -07:00
weights_biases.py refactor(types): replace Any with proven types in 32 files 2026-09-21 10:51:46 +00:00

Readme.md

Integrations

This folder contains logging integrations for litellm

eg. logging to Datadog, Langfuse, Prometheus, s3, GCS Bucket, etc.