mirror of
https://github.com/BerriAI/litellm.git
synced 2026-10-23 05:34:41 +00:00
4bebcdf757
* fix(databricks): use the native Anthropic route for Claude /v1/messages Claude models on a Databricks workspace were bridged through chat completions on /v1/messages, which drops the safeguards field and never returns safeguard_results. They now call the workspace's /serving-endpoints/anthropic/v1/messages, the proxy forwards the client's anthropic-beta and anthropic-version headers to Databricks deployments, and the beta header map marks dangerous-tool-use-2026-09-03 as supported there. Any route whose platform does not take that beta drops safeguards together with it instead of answering 400. Non-Claude Databricks models keep the chat completions bridge * fix(databricks): derive the dangerous-tool-use beta from safeguards instead of forwarding client betas A Databricks Claude serving endpoint answers 400 to betas Claude Code sends on every turn, prompt-caching-scope-2026-01-05 among them, so the proxy no longer forwards the client's anthropic-beta header to Databricks deployments. The beta the classifier needs is derived from the safeguards body field, the same way the Azure AI route derives it. The Rust beta-header mirror stops rejecting the Databricks dangerous-tool-use cell so its config-cell test matches the JSON again * fix(databricks): keep the billing block, build the messages route once, and pass the partner user agent * fix(databricks): route Unity Catalog Claude names to the gateway messages route and keep env credentials Unity Catalog model-service names now reach the AI Gateway's native Anthropic route instead of a serving-endpoints path that answers 404 for them, and the native route resolves the key and base through the same order the chat route uses, so environment-only deployments keep working. The native route is limited to Databricks foundation-model Claude names, an unlisted provider forwards safeguards and the beta as sent, and the ten betas the workspace rejects are marked unsupported for Databricks in the beta map and its Rust mirror. * test(rust): expect Databricks to drop the advanced-tool-use beta * fix(databricks): keep passthrough safeguards, partner user agent, and classifier fallback Only the OpenAI-compatible JSON providers drop safeguards now, so a deployment opted into /v1/messages passthrough forwards the field and the beta as sent. When a route drops safeguards and the proxy has a classifier model, the proxy answers the verdict itself, as the chat completions bridge did. The Databricks Messages route also reads the databricks_user_agent param like the chat route. * fix(databricks): drop rejected server tools under drop_params, short-circuit web search, route UC names from a full Messages URL * fix(databricks): drop container on the native Messages route, as the chat completions bridge did * fix(databricks): keep every api_base on a route the workspace serves and gate the derived beta on the map Gate the dangerous-tool-use beta derivation on the beta map the way #43955 does, so GitHub Copilot, DeepSeek, and opt-in passthroughs keep the merge base wire. Recognize /serving-endpoints/anthropic api_bases, fall back to the merge base routing for an api_base path the workspace trim does not know, strip Bedrock invocation metrics from the streamed message_stop, forward client betas to Databricks filtered by the map, and keep tool_choice when a dropped server tool has no name * test(databricks): cover the native Messages route, derived beta and JSON provider safeguards drop on the wire * test(anthropic): expect the dangerous-tool-use beta derived from safeguards next to the client's betas * fix(databricks): send client anthropic headers only on the native Messages route and keep SDK auth Databricks chat completions, called directly or through the Messages bridge, no longer receive the client's anthropic-beta and anthropic-version, as at the merge base. The native Messages route still forwards them, filtered by the beta map, without counting them as caller credentials, so a deployment with no api_key keeps falling back to Databricks SDK auth. A pathless api_base goes to the native route only for Unity Catalog names, matching the chat route. * test(databricks): cover SDK auth with client anthropic headers and a Unity Catalog workspace root on the native route * fix(databricks): keep client anthropic headers off an opted-in passthrough api_base A databricks Claude deployment whose api_base is no workspace route and that opts into /v1/messages with supported_endpoints falls back to the OpenAI-like passthrough. That config does not filter betas through the beta map, yet it received the native-only Databricks header entry, so Claude Code's prompt-caching-scope beta reached the upstream raw and got a 400 where the merge base answered 200. Each Messages config now says whether it is the provider's native route, the OpenAI-like passthrough says no, and the handler passes that answer when it reads provider-specific headers, so native-only entries reach only native configs. * test(databricks): cover client anthropic betas on an opted-in passthrough api_base A databricks Claude deployment whose api_base is no workspace route and whose model_info.supported_endpoints opts into /v1/messages takes the OpenAI-like passthrough. The cell sends Claude Code's prompt-caching-scope and context-1m betas through the proxy and asserts the caller gets the upstream's message and the gateway receives no anthropic-beta, as on the merge base. The scripted gateway answers 400 on the prompt-caching-scope beta, as the real one does * fix(anthropic): keep beta map loads off shared threads and build the safeguards evaluator only when the proxy classifies Unfiltered configs such as first-party Anthropic never read the beta map to decide safeguards support. Filtered configs await the map on its dedicated loader before reading it, on both the native and the MCP paths. The native handler decides after the upstream call and builds the evaluator only when the proxy classifies, so malformed messages keep the upstream 400. The Databricks dispatch tests assert the dispatched route instead of the config class, and the sail and Eden AI wire rows expect safeguards dropped, as JSON providers do. * test(databricks): cover a loaded beta map that marks the safeguards beta unsupported A remote beta map that nulls dangerous-tool-use for databricks while the bundled map allows it must send neither safeguards nor the beta to the serving endpoint, with or without a client beta, and the caller gets a 200 with no verdict. * test(anthropic): annotate the MCP beta map recordings as Final * fix(anthropic): drop safeguards on routes without the classifier only under drop_params A route whose upstream does not answer safeguards (a JSON provider, or a map that marks the dangerous-tool-use beta unsupported) now forwards the field and the beta as the merge base did unless drop_params is set or the proxy runs its own classifier, matching how litellm treats any unsupported param. Databricks keeps dropping both, since its merge base bridge never forwarded the field. --------- Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com> |
||
|---|---|---|
| .. | ||
| _support | ||
| base_sdk_tests | ||
| benchmarks | ||
| code_coverage_tests | ||
| documentation_tests | ||
| e2e | ||
| e2e_harness | ||
| fixtures | ||
| harness_e2e | ||
| integration | ||
| rust-python-harness | ||
| test_litellm_rust | ||
| typing | ||
| unit | ||
| windows_tests | ||
| __init__.py | ||
| _fake_openai_endpoint_server.py | ||
| _flush_vcr_cache.py | ||
| _live_test_helpers.py | ||
| _master_key.py | ||
| _openai_record_replay_proxy.py | ||
| _process_helpers.py | ||
| _vcr_conftest_common.py | ||
| _vcr_redis_persister.py | ||
| _wait_helpers.py | ||
| _ws_vcr.py | ||
| AGENTS.md | ||
| capturing_transport.py | ||
| eval_swe_bench.py | ||
| fake_openai_endpoint.py | ||
| gettysburg.wav | ||
| large_text.py | ||
| openai_batch_completions.jsonl | ||
| pyrightconfig.json | ||
| README.MD | ||
| test_logging.conf | ||
| test_rust_python_harness.py | ||
| VCR_TESTING.md | ||
| white_100x100.png | ||
In total litellm runs 1000+ tests
[02/20/2025] Update:
To make it easier to contribute and map what behavior is tested,
we've started mapping the litellm directory in tests/unit
This folder can only run mock tests.