Commit graph

48684 commits

Author SHA1 Message Date
Joshua Valluru
e2d888bb91 docs: add MCP auth fix evidence 2026-09-12 16:48:20 -07:00
Joshua Valluru
94e4dd725f fix(mcp): require admission for delegated OAuth 2026-09-12 16:41:23 -07:00
Mateo Wang
566f026c1c
Merge pull request #40767 from BerriAI/litellm_ocr_custom_pricing
fix(cost): honour deployment custom pricing for OCR calls
2026-09-12 15:55:44 -07:00
mateo-berri
d7158ba795 Merge remote-tracking branch 'origin/litellm_internal_staging' into litellm_ocr_custom_pricing 2026-09-12 15:42:59 -07:00
Mateo Wang
426e675c49
Merge pull request #40764 from BerriAI/litellm_redis_pool_timeout_counts_as_timeout
fix(redis): count pool wait timeouts as breaker timeouts
2026-09-12 15:42:57 -07:00
Mateo Wang
fe5ff9d3b0
Merge pull request #40771 from BerriAI/litellm_regression_coverage_followup
test: tighten regression tests added in #37974
2026-09-12 15:29:36 -07:00
mateo-berri
51abb95de0 Merge remote-tracking branch 'origin/litellm_internal_staging' into litellm_redis_pool_timeout_counts_as_timeout
# Conflicts:
#	tests/test_litellm/caching/test_redis_cache.py
2026-09-12 15:26:08 -07:00
Mateo Wang
9276317b34
Merge pull request #40855 from BerriAI/litellm_registry_audit_2026_09_12
fix(registry): sync Azure/Together deprecation dates, fix computer-use-preview and OpenRouter metadata
2026-09-12 15:23:26 -07:00
Mateo Wang
e1bff56f0b
Merge pull request #40772 from BerriAI/litellm_lit6982_bucket_config_includes
fix(proxy): resolve config include directives for bucket-hosted configs
2026-09-12 15:21:56 -07:00
Mateo Wang
c040061f15
Merge pull request #40766 from BerriAI/litellm_fix_v1_messages_disconnect_partial_cost
fix(anthropic): price recovered tokens when a /v1/messages client disconnects mid-stream
2026-09-12 15:21:43 -07:00
Mateo Wang
d72ae3b9a2
Merge pull request #39895 from BerriAI/litellm_deflake_20260905
test: deflake redis semantic cache sys.modules leak, LangSmith init loop patch, wall-clock stagger assertion, and zombie grandchild check in the fake prisma cli
2026-09-12 15:20:26 -07:00
Mateo Wang
fc053dae3e
Merge pull request #40770 from BerriAI/litellm_lit6996_agent_skills_wellknown
feat(proxy): serve registered skills as an Agent Skills well-known index
2026-09-12 15:20:20 -07:00
Mateo Wang
f531537aa3
Merge pull request #40768 from BerriAI/litellm_prisma_cli_resolve
fix(proxy): run migrations through python -m prisma when the prisma console script is not on PATH
2026-09-12 15:20:06 -07:00
ryan-crabbe-berri
ba6a0c9fc6
Merge pull request #40652 from BerriAI/litellm_key_activity_search
feat(ui): search Key Activity by key alias, key hash, user id, or email
2026-09-12 15:05:53 -07:00
kerry-berri
0f03fc2985
Merge pull request #40901 from BerriAI/litellm_remove_fireworks_price_snapshot_tests
test(fireworks): stop pinning prices in the cost-map tests
2026-09-12 14:53:58 -07:00
yuneng-jiang
626d7aaf6a
Merge pull request #40905 from BerriAI/litellm_/release-version-bump-5f3482
chore: bump litellm-enterprise 0.1.66 -> 0.1.67, litellm-proxy-extras 0.4.96 -> 0.4.97
2026-09-12 14:49:46 -07:00
ryan-crabbe-berri
9a3f752724
Merge pull request #40656 from BerriAI/litellm_ui_table_search_pending_state
fix(ui): show loading state instead of stale rows while a table search is pending
2026-09-12 14:49:24 -07:00
Yuneng Jiang
f3ecdccb59
Merge remote-tracking branch 'origin/litellm_internal_staging' into litellm_/release-version-bump-5f3482 2026-09-12 14:40:07 -07:00
Yuneng Jiang
147eb23aab
bump: litellm-enterprise 0.1.66 -> 0.1.67, litellm-proxy-extras 0.4.96 -> 0.4.97 2026-09-12 14:40:01 -07:00
devin-ai-integration[bot]
5b36de4646
fix(guardrails): log mask when a guardrail adds request keys (#40882)
* fix(guardrails): log mask when a guardrail adds request keys

_inputs_were_modified only compared keys present in the pre-hook baseline, so a
guardrail that injected a new key such as tools was logged as allow. Compare over
the union of both key sets, and narrow the pre_call return value to the same
prompt-bearing keys the baseline holds so passthrough stays allow.

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(guardrails): snapshot apply_guardrail inputs before the hook mutates them

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

---------

Co-authored-by: yucheng <yucheng@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-12 14:39:40 -07:00
yuneng-jiang
883b722fd2
Merge pull request #40895 from BerriAI/litellm_budget_clear_persistence
fix(ui): persist cleared budgets and reset intervals
2026-09-12 14:22:24 -07:00
kerry
fbcc602122 test(fireworks): stop pinning prices in the cost-map tests
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-12 21:19:35 +00:00
devin-ai-integration[bot]
311d9bba37
perf(policy_engine): dedup attachments in one pass after sorting (#40883)
get_attached_policies_with_reasons rescanned the sorted matches with next() once
per distinct policy, which is quadratic and misses the one second budget past a
few thousand global attachments. Build a policy to broadest attachment map in one
pass instead, keeping the specificity sort and result order.

Co-authored-by: yucheng <yucheng@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-12 14:19:20 -07:00
kerry
09d63b259c test(fireworks): drop hardcoded price snapshot tests
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-12 21:11:02 +00:00
devin-ai-integration[bot]
261807114a
fix(proxy): accept both deferred stream logging arg shapes on native routes (#40869)
* fix(proxy): accept both deferred stream logging arg shapes on native routes

_arm_deferred_stream_dispatch armed a one-argument closure on every
anthropic_messages/aresponses stream that was not a CustomStreamWrapper or a
LiteLLMCompletionStreamingIterator. The bridged /v1/messages path returns a
plain SSE generator that shares its inner CustomStreamWrapper logging_obj, so
it stores (assembled_response, cache_hit) and _fire_deferred_stream_logging
raised TypeError, dropping spend logs and callbacks and ending the stream with
an error. The closure now dispatches on the stored args shape: a single
coroutine is enqueued, a two-tuple runs success handlers, anything else is
logged and dropped

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(proxy): assert dropped deferred payload via caplog instead of patching the logger

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

---------

Co-authored-by: yucheng <yucheng@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-12 14:10:58 -07:00
Yuneng Jiang
b24480941b
fix(ui): retain cleared user budget values after save 2026-09-12 13:43:55 -07:00
Yuneng Jiang
ad966d8340
fix(projects): persist explicit budget cap clears 2026-09-12 13:43:55 -07:00
Yuneng Jiang
f5f81e973a
fix(budgets): preserve explicit reset interval clears 2026-09-12 13:43:55 -07:00
mateo
414442cd06 fix(registry): mark computer-use-preview as supporting pdf input
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-12 19:46:13 +00:00
ryan-crabbe-berri
559247fa84
Merge pull request #40880 from BerriAI/litellm_tf_key_cascade_delete_recovery
fix(key): recover from a cascade-deleted key instead of failing the apply
2026-09-12 12:20:53 -07:00
mateo
9aa06cb4a3 fix(registry): correct computer-use-preview provider/schema flag and OpenRouter deepseek-v3.2 / claude-opus-4.6 metadata
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-12 19:20:05 +00:00
yuneng-jiang
5f73f837ec
Merge pull request #40774 from BerriAI/litellm_cache_response_semantics
test(e2e): verify cached answers and upstream request count
2026-09-12 12:11:57 -07:00
devin-ai-integration[bot]
49f94cf832
fix(ui): clarify blank TPM/RPM hint on budget modals (#40697)
Co-authored-by: jesus <jesus@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-12 12:08:08 -07:00
mateo
085db0b266 Merge litellm_internal_staging into rolling registry PR 2026-09-12 19:04:09 +00:00
ryan-crabbe-berri
a109553909 fix(key): recover from a cascade-deleted key instead of failing the apply
Deleting a team cascade-deletes its keys, so `terraform apply -replace` on a
team left the key's `/key/update` 404ing and aborted the apply with the key
resource stuck. The update now confirms the key is really gone and recreates it
under the new team; a `team_id` change between two live teams stays an in-place
update, and an unrelated failure still errors out.

Rebased onto current staging, which added a typed `apiError` and `isNotFound`,
so the recovery matches on the status code plus a re-read rather than on the
error string. The metadata pre-read, which fails before `/key/update` is ever
reached when the key is gone, routes through the same recovery.

Original work by @matthowardcohere in #39747.

Claude-Session: https://claude.ai/code/session_01XT1qsbjLwnhiN5sQ2hNUxr
2026-09-12 11:58:44 -07:00
yuneng-jiang
27f8d2a9ba
Merge pull request #40826 from BerriAI/litellm_local_form_clear_adapters
fix(ui): preserve clear and default semantics in local forms
2026-09-12 11:57:29 -07:00
yuneng-jiang
a73454b8fc
fix(ui): restore MCP catalog provider logos (#40781)
Resolves LIT-7390
2026-09-12 11:56:59 -07:00
yujonglee
347b642bdd
refactor(ocr): complete native lifecycle and preserve Azure auth (#40734)
* refactor(ocr): extract call completion boundary

* fix(ocr): release completion state after dispatch

* test(ocr): prove wrapper completion handoff

* test(ocr): narrow mapped failure assertion

* fix(ocr): preserve wrapper invocation kwargs

* fix(ocr): retain completion through finalization

* fix(ocr): make completion ownership explicit

* refactor(ocr): resolve logging executor explicitly

* fix(callbacks): preserve completion lifecycle behavior

* refactor(ocr): move public OCR into native lifecycle

* refactor(ocr): remove unused rust bridge capability

* wip

* wip

* refactor

* wip

* fix(ocr): preserve reducto native compatibility

* wip

* fix(ocr): document native callable casts

* perf(ocr): bound responses and reduce native scheduling overhead

* refactor(python-bridge): organize placeholder routes

* refactor test

* fix(ocr): normalize DeepSeek document content

* perf(ocr): skip unused callback work and benchmark callback overhead

* fix(ocr): align conversion contracts

* test(ocr): cover official provider response shapes

* fix(ocr): restore Python fallback and honor Rust opt-out

* fixes and refactor

* fix(ocr): preserve Azure Document Intelligence authentication

* fix(rust): enforce OCR response limits and lint contracts

* test(rust): align native OCR contract coverage

* test(ocr): isolate Azure auth precedence coverage
2026-09-12 11:56:49 -07:00
devin-ai-integration[bot]
0c98afa780
fix(mcp): enforce end user mcp_tool_permissions on tools/list and tools/call (#40865)
* fix(mcp): apply end user mcp_tool_permissions as a tool ceiling on tools/list and tools/call

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(mcp): restore scoped session admission coverage dropped by mistake

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

---------

Co-authored-by: yassin <yassin@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-12 10:33:09 -07:00
devin-ai-integration[bot]
e4f59a953c
feat(guardrails): add Conduct Guard integration with validated hooks and forwarded params (#40785)
* feat(guardrails): add ConductGuard integration

Adds Conduct Guard as a first-class LiteLLM guardrail. Point any
LiteLLM proxy at Conduct and every LLM call routed through it is
policy-checked before the upstream request goes out — block, warn,
audit, or trigger a human-in-the-loop approval, with the same signed
configuration + hash-chained audit log Conduct exposes on its native
enforcement surfaces.

- litellm/types/guardrails.py: add CONDUCT to SupportedGuardrailIntegrations.
- litellm/proxy/guardrails/guardrail_hooks/conduct/__init__.py: registration
  via guardrail_initializer_registry and guardrail_class_registry, picked
  up by the auto-discovery in guardrail_registry.py.
- litellm/proxy/guardrails/guardrail_hooks/conduct/conduct.py: the adapter.
  CustomGuardrail subclass, async_pre_call_hook, response envelope parser
  for the five Conduct verdicts (ok / advisory / WARNING / BLOCKED /
  PENDING approval), fail-mode logic, session-ID resolution chain
  (litellm_metadata.trace_id → X-Conduct-Session-Id → hash fallback).
- tests/test_litellm/proxy/guardrails/test_conduct_guardrail.py: envelope
  parsing, pre-call allow/block/approval, config precedence, missing-token
  construction error.

```yaml
guardrails:
  - guardrail_name: conduct-guard
    litellm_params:
      guardrail: conduct
      mode: pre_call
      api_base: https://api.conductai.ai      # optional, default
      api_key: os.environ/CONDUCT_AGENT_TOKEN # cond_agt_* token
      fail_mode: fail_closed                   # or fail_open
      tool_name: llm_call                      # scoped tool_name
```

A standalone PyPI package `conduct-litellm-guard` shipped ahead of this
PR for teams pinned to older LiteLLM versions. Once this integration
merges, the standalone README will point at the native support as the
preferred path.

- PyPI: https://pypi.org/project/conduct-litellm-guard/
- Product: https://conductai.ai/guard

Contact: sudhi@b2bsphere.com

* chore: ruff format for conduct guardrail

Fixes lint check on the upstream PR.

* chore: fix ruff lint errors

- Remove unused TYPE_CHECKING import (F401).
- Un-quote self-forward-ref type annotation (UP037).
- Suppress BLE001 on transport-fallback broad-except (intentional).

* chore: drop typing.Any to satisfy strict-rule budget

BerriAI's ruff strict-rule budget caps ANN401 (Any type annotation)
and TID251 (banned import) totals. Aligning with the CustomLogger
base signature (data: dict, cache: object, **kwargs untyped)
eliminates all Any uses in the module. Local tests still pass 15/15.

* chore: annotate **kwargs to satisfy ANN003 strict rule

Removing 'Any' in the prior commit left **kwargs untyped, which
tripped ANN003 (missing type annotation on **kwargs). Using 'object'
threads the strict-rule budget cleanly.

* refactor: slim upstream adapter — import from conduct-litellm-guard PyPI

The full adapter (response parser, session-ID chain, fail-mode logic,
HTTP client) lives in the conduct-litellm-guard package on PyPI. The
upstream tree hosts a thin re-export + the LiteLLM registration wiring.

Matches the Aporia / Lakera pattern — vendor SDK on PyPI, upstream
integration is a tiny adapter.

Benefits:
- Passes ruff-strict-budget and type-discipline-budget without new
  violations.
- Users get the same install experience as any other guardrail vendor:
    pip install conduct-litellm-guard
- Vendor keeps ownership of the parser + fail-mode semantics; upstream
  keeps a stable interface.

Tests slimmed to smoke coverage (imports work, class is a
CustomGuardrail, enum + registries wired, missing-package error path).
Full behavioural coverage stays in the PyPI package.

Local runs of both scripts/ruff_strict_gate.py and
scripts/type_discipline_gate.py against upstream/litellm_internal_staging:
both pass.

* test(conduct): skip smoke tests when conduct-litellm-guard not installed

The wrapper module imports its runtime from the conduct-litellm-guard
PyPI package. When the package is not installed in the CI environment,
the smoke tests can't verify wiring (the import raises before any test
runs). Use pytest.importorskip so BerriAI's default CI env doesn't
fail on this integration, while environments that do install the
package (via 'pip install conduct-litellm-guard[dev]' or similar)
still get the smoke coverage.

Full behavioural test coverage lives in the conduct-litellm-guard
package's own CI.

* test(conduct): cover initialize_guardrail to raise patch coverage

Codecov flagged the __init__.initialize_guardrail body as uncovered
(30% patch coverage on that file). Added a test that mocks
litellm.logging_callback_manager and calls initialize_guardrail with
a SimpleNamespace stand-in for LitellmParams — exercises the full
function body and confirms the callback is registered.

* address review findings on #38143 (yucheng-berri, cursor, veria-ai, devin)

Rename fail_mode → unreachable_fallback (typed field)
─────────────────────────────────────────────────────
The shim was reading a free-form ``fail_mode`` field; a typo silently
defaulted the plugin to fail-open behavior. Switch to the typed
``LitellmParams.unreachable_fallback`` field so Pydantic validates the
value at config load. The plugin's constructor kwarg stays as
``fail_mode`` — the initializer maps the typed field onto it.
(yucheng-berri, devin-ai-integration)

Fix timeout default (was silently discarded)
────────────────────────────────────────────
``getattr(litellm_params, "timeout", 8.0)`` only applied the default
when the attribute was missing; ``LitellmParams.timeout`` always
exists and defaults to ``None``, so the intended 8-second budget was
never used. Change to ``getattr(..., None) or 8.0`` so ``None`` (and
``0``) fall through to the default.
(cursor[bot])

Move ImportError from module-load to __init__
─────────────────────────────────────────────
Raising ImportError at module load caused the guardrail-hook
auto-discovery loop to silently drop the registration when
``conduct-litellm-guard`` was missing. Users saw configs load with
no guardrail active and no error. Import lazily; raise the friendly
``pip install`` error at ``ConductGuardrail.__init__`` when
actionable.
(cursor[bot])

Advertise only supported event hooks
────────────────────────────────────
``during_call`` mode was advertised in the guardrail config but the
class never overrode ``async_moderation_hook`` — every request in that
mode silently bypassed policy. Override ``get_supported_event_hooks``
to return only ``pre_call`` so LiteLLM validates configs against
supported modes at load time. ``during_call`` / ``post_call`` support
lands with plugin 0.3.x once the underlying response-gate is wired
through ``guard_check_response``.
(veria-ai)

Text-completion + full-turn prompt scanning
───────────────────────────────────────────
Fixed in the standalone package: ``conduct-litellm-guard 0.2.2``
(BerriAI/litellm PR #38143 companion, shipping to PyPI shortly).
Pinned in the docstring here as the minimum supported version.
(veria-ai — text_completion bypass + 4KB truncation)

Tests
─────
  * ``test_only_pre_call_event_hook_advertised`` — regression for
    ``during_call`` silent-bypass finding
  * ``test_initialize_prefers_typed_unreachable_fallback`` — regression
    for typo silent-fail-open finding
  * ``test_initialize_applies_timeout_default_when_field_is_none`` —
    regression for silently-discarded 8.0 default
  * ``test_missing_standalone_package_raises_at_construction`` —
    regression for silent-drop-on-import-failure finding (previous
    module-load raise replaced with lazy import + init-time raise)

* style: ruff format on the conduct guardrail shim + tests

Lint job on #38143 flagged three files as needing reformat. No
behavior change — just ruff-format's chosen line breaks and quoting.

* style: remove redundant noqa on re-exported GuardDecision

Ruff lint flagged this as unused because GuardDecision is re-exported
via __all__. Removing the noqa satisfies ruff without changing behavior.

* style: satisfy strict-rule budget (ANN201, ANN401, TID251)

BerriAI/litellm CI's ruff strict-rule budget check flagged four new
violations on the conduct shim. Fixes:

- __init__.py: add return type annotation on initialize_guardrail
  (ANN201)
- conduct.py: swap Any → object on __init__(*args, **kwargs) so the
  signature stays permissive without dynamically-typed Any (ANN401)
- conduct.py: drop the now-unused Any import (TID251)

Ruff --select ANN,TID passes locally.

* style: satisfy type-discipline budget (LIT008, LIT009)

BerriAI/litellm CI's type-discipline budget check flagged the
subclass __init__ shim. Fixes:

- Drop the __init__ override entirely — the subclass now inherits
  __init__ from _BaseConductGuard (when the standalone package is
  installed) or from CustomGuardrail (fallback). Removes both the
  banned **kwargs (LIT008) and all four inert # type: ignore markers
  (LIT009 x 4).
- Move the missing-package check into a dedicated
  raise_if_missing_package() helper called by
  initialize_guardrail before construction. Preserves the
  cursor[bot] fix (silent-drop-on-import-failure) without needing
  a custom __init__.
- Fallback branch aliases _BaseConductGuard = CustomGuardrail
  directly, no type-ignore comment needed.
- Test updated to exercise the helper instead of the removed
  __init__ path; new companion test asserts the helper is a no-op
  when the package IS installed.

Local: ruff --select ANN,TID passes clean. ruff format applied.

Same behavioral surface — user-visible error message unchanged.

* style: explicit assert on noop test (TQ001 zero-assert budget)

BerriAI/litellm CI's test-quality budget flagged
test_raise_if_missing_package_is_noop_when_present as a zero-assert
test (TQ001). Make the intent explicit: raise_if_missing_package()
must return None when the package IS installed.

* refactor: shim becomes a pure alias, hooks now on plugin's ConductGuard

Plugin conduct-litellm-guard 0.2.3 ships SUPPORTED_EVENT_HOOKS +
get_supported_event_hooks on ConductGuard directly. The upstream
shim's subclass wrapper is now redundant — dropping it clears every
strict-rule budget gate (ruff-strict / test-quality /
type-discipline / basedpyright) in one pass.

Changes:
- conduct.py: subclass removed; ConductGuardrail is now an alias for
  the plugin's ConductGuard (no dynamic base class, no reassignment,
  no # type: ignore). raise_if_missing_package helper unchanged.
- test file: _IMPORT_ERROR → _import_error rename to satisfy
  reportConstantRedefinition (basedpyright treats SCREAMING_CASE as
  constant). Also drops unused sys import.
- Pin bumped to conduct-litellm-guard>=0.2.3 in the module docstring.

Verified all four LiteLLM gate scripts locally against
upstream/litellm_internal_staging:
  ruff_strict_gate    OK
  test_quality_gate   OK
  type_discipline_gate OK
  type_check_gate     OK

* fix: real stub class in the missing-package fallback

Runtime regression in the previous simplification — the guardrail
registry iterates every registered class at load time and calls
get_supported_event_hooks(). Fallback of ConductGuardrail = None
crashed the whole registry with AttributeError, which cascaded into
unrelated guardrails' tests (noma_v2, repelloai, hide_secrets,
provider_specific_params, etc.).

Fallback now defines ConductGuardrail as a real subclass of
CustomGuardrail with the required class attrs (SUPPORTED_EVENT_HOOKS
+ get_supported_event_hooks). Matches the pattern the
guardrails_ai integration already uses in the same repo.
raise_if_missing_package still fires before instantiation so users
see the friendly pip install error.

All four budget gates re-verified locally against
upstream/litellm_internal_staging:
  ruff_strict_gate    OK
  test_quality_gate   OK
  type_discipline_gate OK
  type_check_gate     OK

* style: mutable-ok suppression on registry dicts + hook returns

* fix: SUPPORTED_EVENT_HOOKS must be GuardrailEventHooks enum, not str

LiteLLM's guardrail registry scans SUPPORTED_EVENT_HOOKS and calls
.value on each entry to build the mode allowlist. Plugin 0.2.3 shipped
bare strings, which raised AttributeError on three upstream tests
(same three as the pre-0.2.3 None-registration failure).

- Fallback stub now uses GuardrailEventHooks.pre_call.
- Docstring and pip install message updated to >=0.2.4.
- Test asserts against the enum member (which is what LiteLLM's
  registry scan actually sees).

Requires plugin conduct-litellm-guard >=0.2.4 (already tagged and
publishing).

All four budget gates verified locally green:
  ruff_strict, test_quality, type_discipline, type_check

* fix(guardrails): validate Conduct event hooks, forward tool_name, drop optional-package test skip

Pass the plugin's supported hook list into CustomGuardrail so unsupported modes
(during_call, post_call, logging_only) are rejected at config load instead of
silently doing nothing. Forward the configured tool_name to the plugin, and
replace the missing-package stub so the registry still discovers the guardrail
while construction raises an install hint.

The regression tests inject a recording guardrail class so they run without
conduct-litellm-guard installed; the previous module-level skip left the
adapter untested in CI.

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(guardrails): scan Responses API input through the unified Conduct bridge

The plugin's native pre_call hook only reads prompt and chat messages, so
/v1/responses requests reached Conduct with an empty prompt and were always
allowed. ConductGuardrail now implements apply_guardrail, which routes every
endpoint through LiteLLM's shared guardrail translation and feeds the
translated texts (or structured messages) to the plugin's check()

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(guardrails): log Conduct apply_guardrail decisions via log_guardrail_information

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* refactor(guardrails): move the Conduct apply_guardrail bridge into an injectable function

The bridge body only ran when conduct-litellm-guard was importable, which CI
never is, so codecov/patch reported it uncovered. apply_conduct_guardrail now
takes the plugin's check coroutine and blocked-error factory as parameters, so
the package-free tests exercise every verdict branch and the plugin-bound class
is a one-line delegate

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(guardrails): send tool-call-only turns to Conduct and test registry wiring through config load

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(guardrails): log non-blocking Conduct verdicts in standard guardrail information

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* feat(guardrails): add Conduct config model and Admin UI garden entry

Expose ConductGuardrailConfigModel through get_config_model() so
/guardrails/ui/provider_specific_params returns the api_key, api_base,
workspace_id, tool_name, timeout and unreachable_fallback fields, and
add the Conduct Guard partner card, preset and logo to the guardrail
garden so the integration can be created from the Admin UI

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(guardrails): pass unreachable_fallback directly to conduct-litellm-guard 0.2.5

The plugin renamed its constructor kwarg from fail_mode to unreachable_fallback in
0.2.5 and kept fail_mode only as a deprecated alias that warns on every init. Forward
the new kwarg and bump the documented pin to >=0.2.5. Mirrors 62325467 on #38143

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(guardrails): reject conduct-litellm-guard builds that swallow unreachable_fallback

Plugin 0.2.4 accepts **kwargs, so the renamed kwarg was silently dropped and a
configured fail_open became fail_closed. Fail at import with the install hint instead

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

---------

Co-authored-by: Sudhi Seshachala <sudhi@b2bsphere.com>
Co-authored-by: yucheng <yucheng@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-12 10:04:55 -07:00
yuneng-jiang
242c53c0c0
Merge pull request #40836 from BerriAI/litellm_key_project_detachment
fix(keys): support explicit project detachment
2026-09-12 10:00:09 -07:00
yuneng-jiang
15bcc5f6eb
Merge pull request #40839 from BerriAI/litellm_compression-inheritance
fix(router): restore compression inheritance when clearing overrides
2026-09-12 09:59:59 -07:00
tin-berri
2083e2a21f
feat(cli): configure Claude Code and Codex with a gateway key (#40829) 2026-09-12 09:59:13 -07:00
devin-ai-integration[bot]
eddfb5fb20
fix(responses): preserve hosted web search calls (#40828)
* fix(responses): preserve hosted web search calls

Co-Authored-By: Claude Code <noreply@anthropic.com>
(cherry picked from commit 09183b3346)

* chore: remove unrelated generated schema documentation changes

(cherry picked from commit ca6a860757)

* fix(responses): preserve hosted search context during replay

(cherry picked from commit 425f1e9b3a)

* chore: regenerate dashboard API types

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

---------

Co-authored-by: Tin Chi Lo <tin@berri.ai>
Co-authored-by: Claude Code <noreply@anthropic.com>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-12 09:52:19 -07:00
devin-ai-integration[bot]
1c61c2606e
perf(proxy): one MGET and one pipeline for post-call spend counters, no team/user/org refetch on the response path (#40841)
* perf(auth): prefetch user, team, membership, org and project in one MGET, one query and one pipeline

Auth read each object with its own Redis GET and, on a miss, its own DB
query, then the admission spend counters with one GET each. The prefetch
warms every entry the checks read with one MGET, one raw query for the
Redis misses and one pipeline write, and a per-request batch serves the
spend counter reads from one MGET. The per-object getters stay the
readers and the fallback, so enforcement does not depend on the prefetch

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* refactor(auth): keep prefetch and spend batch collections immutable

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* perf(auth): let the cold spend-counter reseed reuse the admission MGET instead of one GET per counter

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* perf(auth): prefetch referenced auth objects only after the key's model access check passes

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(auth): give the prefetch-ordering test's patches their test-quality reasons

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(auth): move the real-Postgres prefetch join test to the proxy_behavior shard

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(auth): read NULL nested permission and budget lists as [] in the prefetch join

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* perf(proxy): batch post-call spend counter reads and carry budget state through the request

Post-call warm checks, reservation reads and reconcile reads for one request now go through a task-local spend counter batch: one MGET answers every counter, successful increments write their result back into the batch so no second Redis read follows, and invalidation forgets the key. RedisCache.async_increment sends INCRBYFLOAT and its TTL command in one pipeline round trip.

Auth pins frozen team, user and org budget snapshots on UserAPIKeyAuth, the pre-call setup writes them into the request metadata, and Prometheus reads them back instead of calling get_key_object, get_team_object, get_user_object and get_org_object on the response path. The getters stay as the fallback for requests that carried nothing (custom auth, unauthenticated routes, skipped checks).

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* perf(proxy): reconcile the budget reservation and the post-call warm checks from one MGET and one pipeline

A scope opened inside an open spend counter batch binds into it instead of starting its own, so the reservation reconcile and the post-call warm checks share the request's single MGET. The reconcile reads every reserved counter concurrently, sends the consistent adjustments in one INCRBYFLOAT+EXPIRE pipeline and settles a flushed or reseeded counter on its own afterwards, keeping the pre-call resize fail-closed. PendingSpendIncrement moves to spend_counter_batch so budget_reservation can build a pipeline without importing a private name

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* chore(proxy): drop the dataclass import left behind by the PendingSpendIncrement move

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(types): import Self from typing_extensions so the proxy imports on Python 3.10

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(proxy): use a neutral organization alias in the carried budget state tests

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(proxy): cover recorded and forgotten spend counter values in the request batch

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(caching): assert async_set_cache_pipeline_with_ttls keeps per-entry TTLs

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* refactor(proxy): type the reservation entry carried through reconcile adjustments

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(auth): map the model table's aliases column to model_aliases in the prefetch join and read user memberships the way get_user_object does

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

---------

Co-authored-by: yassin <yassin@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-12 16:05:50 +00:00
devin-ai-integration[bot]
f2e0a5db1e
perf(auth): read user, team, membership, org, project and spend counters in one MGET, one query and one pipeline (#40834)
* perf(auth): prefetch user, team, membership, org and project in one MGET, one query and one pipeline

Auth read each object with its own Redis GET and, on a miss, its own DB
query, then the admission spend counters with one GET each. The prefetch
warms every entry the checks read with one MGET, one raw query for the
Redis misses and one pipeline write, and a per-request batch serves the
spend counter reads from one MGET. The per-object getters stay the
readers and the fallback, so enforcement does not depend on the prefetch

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* refactor(auth): keep prefetch and spend batch collections immutable

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* perf(auth): let the cold spend-counter reseed reuse the admission MGET instead of one GET per counter

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* perf(auth): prefetch referenced auth objects only after the key's model access check passes

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(auth): give the prefetch-ordering test's patches their test-quality reasons

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(auth): move the real-Postgres prefetch join test to the proxy_behavior shard

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(auth): read NULL nested permission and budget lists as [] in the prefetch join

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(caching): assert async_set_cache_pipeline_with_ttls keeps per-entry TTLs

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(auth): map the model table's aliases column to model_aliases in the prefetch join and read user memberships the way get_user_object does

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

---------

Co-authored-by: yassin <yassin@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-12 08:52:23 -07:00
mateo
db8201df8f fix(registry): sync azure/us and azure/eu o-series deprecation dates
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-12 14:44:52 +00:00
mateo
a6418b3ff5 fix(registry): keep gpt-5.4-mini/nano max_input_tokens at 272000
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-12 13:45:15 +00:00
mateo
6c974df9a0 fix(registry): sync Azure o-series and Together deprecation dates, gpt-5.4-mini/nano context window
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-12 13:24:00 +00:00
mateo
441f863abb Merge remote-tracking branch 'origin/litellm_internal_staging' into litellm_deflake_20260905
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

# Conflicts:
#	tests/test_litellm/proxy/db/test_pgbouncer.py
2026-09-12 09:18:42 +00:00