Commit graph

50180 commits

Author SHA1 Message Date
yassin
be1664a485 fix(proxy): rename AWS Secrets Manager secret when key alias changes
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-16 18:29:29 +00:00
ryan-crabbe-berri
ecd72c51da refactor(ui): register ssh skill sources verbatim and share the https host rules
The ssh parser rebuilt the clone url it was given, stripping a trailing .git and
appending one back. Git treats that suffix as optional, and the hosts whose clone
paths are not org/repo break when it is forced on, so an Azure DevOps v3 or a
CodeCommit v1/repos url registered through the form would no longer clone.

It also carried its own host pattern, which demanded an alphabetic final label and
so rejected internal hosts like gitlab.internal.k8s2 that the https path accepts.

Rewrite the scp form into an ssh:// url purely to validate it, reuse the https host
and credential checks through a shared isSafeHost, and store exactly what the user
typed. Only a url that survives the round trip unchanged is accepted, which is what
keeps traversal segments out of the feed, so the two ssh regexes, the dots-only
guard and the clone-url builders all collapse into one function.
2026-09-16 11:29:25 -07:00
mateo-berri
938702a3a7 Merge branch 'main' of https://github.com/BerriAI/litellm into litellm_anthropic_passthrough_strip_virtual_key 2026-09-16 11:28:09 -07:00
Yujong Lee
fc93c9ccba ci(rust): drop removed bedrock-auth feature steps
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-16 18:20:33 +00:00
yassin
8cab3a7846 refactor(otel): walk nested metadata iteratively instead of recursively
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-16 18:20:10 +00:00
yassin
4513f78df5 fix(migrations-check): read the table name past comments, ignore referential SET DEFAULT
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-16 18:14:36 +00:00
Yujong Lee
23633445d7 refactor(rust): extract auth and cache crates
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-16 18:12:11 +00:00
yassin
17059564a8 feat(otel): promote nested request metadata keys to litellm.metadata.* span attributes
baggage_metadata_keys entries such as requester_metadata.trace_id now resolve the caller's nested metadata.trace_id and stamp it on the LLM-call span as litellm.metadata.trace_id, in both the OTEL v2 logger and the legacy OpenTelemetry callback. Nested metadata mappings are flattened to dotted paths, only allowlisted leaves are promoted, and the requester_metadata blob itself is never promoted

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-16 18:08:14 +00:00
yucheng
1b5dacc717 fix(proxy): tolerate malformed auth spans and read the OTel span from request state for custom auth
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-16 18:05:38 +00:00
yassin
bf39aebcf1 ci(migrations): flag defaulted ADD COLUMN on request-log tables
Postgres 10 has no fast default path, so ADD COLUMN ... DEFAULT on
LiteLLM_SpendLogs rewrites the heap and every index under an ACCESS
EXCLUSIVE lock inside the boot-time migrate deploy. The checker now
reports it on LiteLLM_SpendLogs and LiteLLM_ErrorLogs; the two shipped
migrations that already do it are grandfathered

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-16 18:02:00 +00:00
yassin
ed18626291 fix(proxy): sync AWS Secrets Manager on body-less key regenerate
POST /key/{key}/regenerate with no request body reaches async_key_rotated_hook with data=None, and the secret manager sync was gated on data being present, so the rotated key never reached AWS Secrets Manager and the revoked key stayed stored. Gate on response.token_id only and read the requested alias null-safely

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-16 17:56:39 +00:00
Yujong Lee
3fcefa82f2 fix(rust): reconcile rebase with realtime and routing_utils removal on main
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-16 17:53:04 +00:00
berriai-litellm-provider-info-sync[bot]
f8fb31db3a
chore(prices): sync Azure prices: 2 models
azure_ai/grok-4.3: input_cost_per_token_above_200k_tokens, output_cost_per_token_above_200k_tokens, cache_read_input_token_cost_above_200k_tokens
azure_ai/grok-4.6: input_cost_per_token_above_200k_tokens, output_cost_per_token_above_200k_tokens, cache_read_input_token_cost_above_200k_tokens
2026-09-16 17:51:09 +00:00
Yujong Lee
26b5cd57a7 refactor(core): let the plan own delivery and gate sync leaves on the request kind
CallbackFamily::delivery is gone. plan_success, plan_failure and
plan_request return a Dispatch carrying family, delivery and gate, and
DispatchCursor takes that Dispatch plus CursorFacts. Delivery was never a
property of the family: legacy runs failure_handler inline from @client but
on the executor from dispatch_failure_handlers, and the streaming port will
need both.

object_target_eligible encodes the is_sync_request gate from
success_handler and failure_handler: on an async SDK request the sync
handler pass runs string integrations only, so CustomLogger and plain
callable targets are not logged twice alongside their async methods.
Parity-neutral for OCR because _is_sync_litellm_request does not know aocr,
but required before the first async route lands.
2026-09-16 17:50:11 +00:00
Yujong Lee
a28767d757 test(rust_bridge): bind every string-integration leaf to its real signature
dispatch_named_success, dispatch_named_failure and dispatch_named_request
now call the legacy integration singletons directly after isinstance
narrowing instead of through getattr-by-name casts, so basedpyright checks
arity and keyword names. None of those integrations run in the OCR suite,
so each branch is exercised against a stub of the real class and the
recorded call is bound with inspect.signature to the real method. Also
covers the logfire enum levels, the lunary end/error events, the supabase
pre-call-only guard, sentry hooks, openmeter and dynamodb awaitables, and
the langfuse trace-id cache path.
2026-09-16 17:50:11 +00:00
Yujong Lee
b204dec9f9 feat(python-bridge): run deployment hooks through the native cursor
async_pre_call_deployment_hook, async_post_call_success_deployment_hook and
async_post_call_failure_deployment_hook run on every SDK call, not only on
the proxy, and the native OCR path still delegated them to the Python chain
in litellm.utils. DeploymentBody now iterates litellm.callbacks through
CallbackFamily::Deployment* with Delivery::Await, chaining the replaced
kwargs or response through each CustomLogger and containing failure-hook
errors per target while pre and post hook errors propagate.

Runner::invoke returns an error when an awaited leaf yields a non-awaitable
instead of guessing from is_none. The legacy-disabled test registers a
deployment hook and forbids the three utils entry points.
2026-09-16 17:50:11 +00:00
Yujong Lee
aecc08050f style(rust_bridge): satisfy strict lint gates in the callback leaves
Type aliases carry TypeAlias, cast sites carry cast-ok on the call line,
dict fallbacks use frozen constants, and nested envelope seeding replaces a
None metadata value instead of returning it (legacy behaviour the setdefault
form missed). Fixes the PT012 shape in test_bindings and formats ocr/main.
2026-09-16 17:50:11 +00:00
Yujong Lee
316700aeb8 refactor(python-bridge): drop the finalize shim and prove native handles collect
Finalize calls update_response_metadata directly instead of through a
lifecycle.py passthrough. GC tests cover WorkerJob and DeferredSuccess
traversal through logger, targets and response, and DeferredSuccess close
and release idempotence.
2026-09-16 17:50:11 +00:00
Yujong Lee
f0b877532c feat(python-bridge): dispatch OCR callbacks through the native cursor
Success, failure and request callbacks on the native OCR path no longer go
through Logging.pre_call, post_call, success_handler or failure_handler.
Core's DispatchCursor selects each target and leaf method; the bridge
invokes it directly or through one labeled leaf in
litellm.rust_bridge.leaves. Delivery follows the family: request and sync
failure run inline, deployment hooks and async failure are awaited by
drive(), sync success runs as one grouped WorkerJob on the executor, async
success is a coroutine enqueued on the logging worker with the deferred gate
held natively in DeferredSuccess.

PrepareLogging (cost, standard payload, redaction) runs inside the dispatch
delivery and never fails the request; Finalize keeps only public response
metadata. Dedup markers stay on the shared logging object and are read as
an eligibility fact. Caller-supplied Logging instances take a separate
compat path that calls their own handlers.

Tests prove the two-pass hook order, per-target hook containment, hook
result replacement, marker write and honour, best-effort preparation, and a
positive run with every legacy orchestration entry point patched to raise.
2026-09-16 17:50:11 +00:00
Yujong Lee
f8190bbe80 refactor(python-bridge): classify callbacks natively instead of function_setup
Native OCR setup no longer calls utils.function_setup. The bridge reads
registration facts (coroutine-ness, CustomLogger, known names, existing list
membership) and core's plan_registration decides every public registry
mutation, which the bridge writes back through litellm.rust_bridge.setup.
The Logging object is built by a narrow Python factory with the same
constructor arguments. Caller-supplied Logging instances keep identity and
skip registration.

A differential test asserts registry side effects equal function_setup for
nine registration shapes; mutating the planner fails four of them.
2026-09-16 17:50:11 +00:00
Yujong Lee
54be2dcae1 feat(core): add native callback sequencing types and dispatch cursor
Adds CallbackId, Delivery, CallbackInvocation, CallbackFamily, LoggedMarker,
DispatchCursor, plan_success and plan_failure to call_lifecycle. Core now
owns target ordering, get_combined_callback_list dedup, the two-pass success
shape, marker placement, per-family delivery and error policy, and the
sync/async success dispatch plan. No host wiring yet.
2026-09-16 17:50:11 +00:00
Yujong Lee
1771e5bb68 refactor callback 2026-09-16 17:50:11 +00:00
Yujong Lee
75548d30f5 move io to core 2026-09-16 17:50:11 +00:00
Yujong Lee
5a7ea7ef71 refactor(python-bridge): make deferred logging callable 2026-09-16 17:50:11 +00:00
Yujong Lee
9bc6426ae1 refactor(python-bridge): bind OCR arguments once in Rust
The request crossed the boundary four times: Python bind_request into a
dataclass, Rust re-reading each field back with its own precedence rules,
a BridgeOcrRequest copy, then the core request. The dataclass was retained
for the whole call only so map_failure could read model and kwargs at the
end.

Add a route-agnostic Signature binder that reproduces Python's positional
and keyword binding rules and TypeError messages, and project straight
from the bound arguments into the core request. This removes the Python
LiteLLMOcrRequest dataclass, bind_request, PythonOcrInput, BridgeOcrRequest,
and the retained boundary_request, document, api_key and callback_inputs
Python objects. api_key for pre_call now comes from core's resolved
connection, matching the legacy handler, and core no longer carries
retained_fields or retains_document since the bridge stopped re-aliasing.
2026-09-16 17:50:11 +00:00
Yujong Lee
678e59c6b4 refactor(python-bridge): drop OCR callback fast path and body re-aliasing
The bridge kept a Rust-side shadow of Logging's callback registries and
elided pre_call/post_call/success_handler when it thought nothing was
listening. That forked the logging contract and hid a bug: with callbacks
present, the during-call hook re-inserted the caller's original document
and unmapped optional params over the provider-transformed body.

Call the real Logging handlers unconditionally, like the Python wrapper
does, and hand callbacks a fresh dict built from core's composed body
instead of re-aliasing caller objects into it.
2026-09-16 17:50:11 +00:00
Yujong Lee
7167bd0ff0 cleanup dead code 2026-09-16 17:50:11 +00:00
Yujong Lee
b9fca28c6e simplify 2026-09-16 17:50:11 +00:00
Yujong Lee
6d66be8bef refactor(python-bridge): make route bindings explicit 2026-09-16 17:50:11 +00:00
Yujong Lee
22a593d606 remove prepare_request 2026-09-16 17:50:11 +00:00
Yujong Lee
6c893a7dba remove input.py 2026-09-16 17:50:11 +00:00
Yujong Lee
2f25875ce6 refactor(ocr): separate unresolved connection state 2026-09-16 17:50:11 +00:00
Yujong Lee
056487a283 wip 2026-09-16 17:50:11 +00:00
Yujong Lee
5de196af63 wip 2026-09-16 17:50:11 +00:00
Yujong Lee
31b48f6191 more fixing 2026-09-16 17:50:11 +00:00
Yujong Lee
3a64b12911 more porting alignment 2026-09-16 17:50:10 +00:00
Yujong Lee
266ebcf4aa remove router from core 2026-09-16 17:50:10 +00:00
Yujong Lee
ec26f2a5a0 fix cohere 2026-09-16 17:50:10 +00:00
Yujong Lee
03a1c4a938 inlineing more errors 2026-09-16 17:50:10 +00:00
Yujong Lee
8cfb59082a refactor(rust): localize route errors 2026-09-16 17:50:10 +00:00
Yujong Lee
55fe8d40d0 refactor(ocr): flatten Azure transformation 2026-09-16 17:50:10 +00:00
Yujong Lee
0d63ecd931 wip 2026-09-16 17:50:10 +00:00
Yujong Lee
57e7bce5ed refactor(auth): localize AWS errors 2026-09-16 17:50:10 +00:00
Yujong Lee
3ef381a7b0 refactor(ocr): execute prepared request directly 2026-09-16 17:50:10 +00:00
Yujong Lee
9f13bf59b6 build(rust): scope provider auth dependencies 2026-09-16 17:50:10 +00:00
Yujong Lee
ddd101a780 wip 2026-09-16 17:50:10 +00:00
Yujong Lee
f4a6f695c9 refactor(rust): type provider model namespaces 2026-09-16 17:50:10 +00:00
Yujong Lee
c80617c4a6 wip 2026-09-16 17:49:20 +00:00
Yujong Lee
cc88a9479e wip 2026-09-16 17:49:20 +00:00
kerry
686556fffc test(anthropic): build served-model stream chunks immutably
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-16 17:49:15 +00:00