Vertex multi-region endpoints (e.g. us, eu) use the rep host pattern, not
{geo}-aiplatform.googleapis.com. Regional IDs still contain a hyphen.
common_utils.get_vertex_base_url centralizes the rule for SDK/API URL building.
Proxy pass-through duplicates the same branching in a local get_vertex_base_url
(with trailing slashes) to avoid importing from common_utils there; live
WebSocket passthrough uses the same multi-region host logic for wss://.
Tests cover us/eu for the common_utils helper.
Made-with: Cursor
The create-branch job in create-release.yml calls the reusable
create-release-branch.yml workflow, which requires contents: write.
The top-level permissions: {} blocks the inherited default, and only
the release job overrode it, so the nested call failed with:
The nested job 'create-branch' is requesting 'contents: write',
but is only allowed 'contents: none'.
Add the permission at the calling job level so the reusable
workflow is granted what it needs.
The previous `while not self._queue.empty(): await self._queue.join()`
pattern skipped the join entirely when the worker had already dequeued a
task but not yet called task_done(). asyncio.Queue.join() tracks
_unfinished_tasks (incremented by put, decremented by task_done), not
queue depth, so it already handles that case on its own.
The mocked async_increment_cache_pipeline is invoked from Router's
deployment_callback_on_success, registered as an async success callback.
Those callbacks are enqueued to GLOBAL_LOGGING_WORKER and run on a
background task, so the mock may not have been called yet when the test
asserts on it. Flush the worker before asserting.
Previous commit from greptile-apps added a new `when: always` teardown
step without removing the prior `name:`-only step, leaving a `- run`
block with no `command:` — CircleCI config validation rejects that.
Collapse back to a single teardown step that runs on success and
failure.
Two independent deflakes:
1. test_ui_view_spend_logs_unauthorized (unit) was returning 400 instead
of 401/403 when earlier tests in the file left proxy-auth globals
(prisma_client, master_key, user_custom_auth, general_settings,
user_api_key_cache) in a state that let invalid tokens pass auth and
fall through to the endpoint's own start_date/end_date validation.
Add an autouse fixture that pins those globals to their import-time
defaults for every test in the file. Harden the assertion to include
response body so future flakes are diagnosable.
2. test_basic_spend_accuracy (CI job proxy_spend_accuracy_tests) depends
on the Redis transaction buffer flushing spend to Postgres. The buffer
uses a single global pod-lock key (cronjob_lock:db_spend_update_job)
and a single global buffer list key. Pointing the proxy at the shared
remote Redis means concurrent CI pipelines contend for the same lock
and can drain each other's buffer into the wrong database. Add a
start_redis reusable command that boots a per-job redis:7-alpine
container (digest-pinned), and switch proxy_spend_accuracy_tests to
REDIS_HOST=host.docker.internal:6379 so lock and buffer state are
isolated per CI run.
* feat: add gpt-5.5 to model cost map
Add gpt-5.5 entry with pricing from OpenAI flagship page:
input $5/1M, cached input $0.50/1M, output $30/1M, 272K context.
* test: add gpt-5.5 coverage for model cost map and gpt-5 routing
- Add gpt-5.5 to GPT5_MODELS parametrized list so both OpenAIGPT5Config
and AzureOpenAIGPT5Config routing tests cover the new model.
- Add test_generic_cost_per_token_gpt55 verifying the new entry's
cost-map values ($5/$0.50/$30 per 1M) and that generic_cost_per_token
returns the expected prompt/completion costs.
Extracts release branch creation into a separate reusable workflow
(create-release-branch.yml) that can be triggered independently via
workflow_dispatch or called from other workflows via workflow_call.
create-release.yml now dispatches it as a dependent job after the
release publishes, keeping both workflows decoupled.
token_counter.py: the previous size-limit raises were inside except Exception: pass,
so they were silently swallowed. The post-read raise was worse — img_data was already
assigned the full body before the raise, so the oversized value was used downstream.
Restructured to only assign img_data when the body is within bounds.
vertex_ai/common_utils.py and llm_passthrough_endpoints.py: the is-not-None guard
skipped validation for None, falling through to produce "https://None-aiplatform..."
Added explicit None check that raises before the regex guard.
The rvm/install script sources scripts/functions/installer using
paths relative to the caller's working directory (not $0), so
invoking /tmp/rvm/install from /home/circleci/project fails with
'No such file or directory'. Switch to (cd /tmp/rvm && ./install).
Align the Ruby, Node.js, and npm install path with the rest of the
config. Three separate upstream installers were being invoked via
\`curl ... | bash\` or unlocked \`npm install\`:
- RVM's \`get.rvm.io/stable\` installer (mutable upstream script).
Replace with a shallow git clone of the rvm/rvm repo at tag 1.29.12
and verify HEAD matches the published commit SHA before running the
local \`./install\` script. Same pattern already used for the
helm-unittest plugin in .github/workflows/helm_unit_test.yml.
- NodeSource's \`deb.nodesource.com/setup_18.x\` piped into sudo bash.
Replace with a direct download of the Node.js 18.20.8 linux-x64
tarball from nodejs.org, verified against the published
SHASUMS256.txt digest before extraction.
- \`npm install @google-cloud/vertexai @google/generative-ai\` and
\`--save-dev jest\` resolved fresh from the npm registry on every
run. Add \`tests/pass_through_tests/package.json\` with pinned
direct-dep versions and commit the generated package-lock.json, then
switch CI to \`npm ci\` (exact lockfile install, fails on drift).
Also scopes the Ruby+JS test runners to \`tests/pass_through_tests/\`
so they pick up the committed package.json rather than writing
node_modules at repo root.
Stragglers from the 2026-04-21 Python 3.12 standardization:
- .github/workflows/check_duplicate_issues.yml (was 3.11)
- .github/workflows/llm-translation-testing.yml (was 3.11)
- .github/workflows/scan_duplicate_issues.yml (was 3.13)
- .circleci proxy_build_from_pip_tests (was 3.13)
The only intentional non-3.12 CI job is installing_litellm_on_python_3_13,
which exists as an explicit "latest supported Python" smoke matrix.
The Windows uv install step was piping a remote install.ps1 into
Invoke-Expression without any integrity check, while the Linux
install steps (install_uv command, line 89) download to a file,
verify SHA-256 against a hardcoded digest, and only then execute.
Bring the Windows path to the same pattern.
Also hardcode the kubectl v1.31.4 checksum in helm_chart_testing
instead of fetching kubectl.sha256 from the same origin as the
binary — if dl.k8s.io were ever to serve a tampered pair, a
co-hosted checksum provides no additional integrity.
The same branch filter block appeared 46 times in the workflow
declaration:
filters:
branches:
only:
- main
- /litellm_.*/
And the same pinned Python docker image appeared 29 times in jobs:
- image: cimg/python:3.12@sha256:9c796c...
auth:
username: ${DOCKERHUB_USERNAME}
password: ${DOCKERHUB_PASSWORD}
Replace with YAML anchors declared at first use:
- `&main_branches` on using_litellm_on_windows's filters block;
all other job entries reference it as `filters: *main_branches`.
- `&python312_image` on local_testing_part1's first docker image
entry; all other jobs reference `- *python312_image`, including
the multi-image jobs (auth_ui_unit_tests,
installing_litellm_on_python_v2_migration_resolver) which keep
their postgres sidecar entry inline afterwards.
Net result: one place to change when the image digest rolls or
the branch-filter convention changes. No behavior change — YAML
anchor resolution produces identical config at parse time.
Also adds Docker Hub auth block to upload-coverage (previously
pulled anonymously). No functional difference for a public
image, but avoids Docker Hub rate limits now that we reuse the
same entry.
Clean out copy-paste debug and workaround lines that serve no
purpose:
- `pwd && ls` echoes at the top of 30 "Run tests" steps (CCI
already logs working_directory on every step).
- "Show git commit hash" in local_testing_part1/part2 and
langfuse_logging_unit_tests (CCI shows the SHA in every job
header).
- "Verify Docker is available" stubs in 6 machine-executor
jobs (machine executors always have Docker).
- `sudo systemctl restart docker` in proxy_store_model_in_db_tests
(one-off workaround; not used anywhere else).
- Duplicated Black formatting step in local_testing_part1 and
local_testing_part2 — Black runs in the lint job, no reason to
run it again here.
- Second back-to-back `helm test litellm --logs` invocation in
helm_chart_testing (one call is enough).
No behavior change — these are all log-only or no-op steps.
The original check `"gpt-5-chat" not in model` already correctly
classifies all current gpt-5 variants (including gpt-5.3-chat and
gpt-5.1-chat, which do NOT contain the substring "gpt-5-chat"). This
change replaces it with an explicit `startswith("gpt-5-chat")` prefix
test on the provider-prefix-stripped model name.
The new check is functionally equivalent for all existing model names
but makes the classification boundary unambiguous and forward-safe:
future model names that might contain "gpt-5-chat" as an interior
substring won't accidentally be excluded from the GPT-5 reasoning path.
Also moves the new regression test from tests/ root to
tests/test_litellm/llms/openai/ so it is included in `make test-unit`.
The useEffect that re-fetches logs on sort/page/time changes:
useEffect(() => {
if (hasBackendFilters && accessToken) {
performSearch(filters, currentPage);
}
}, [sortBy, sortOrder, currentPage, startTime, endTime, isCustomDate]);
intentionally omits `filters` and `hasBackendFilters` from its dep array
to avoid double-fetches when a filter is applied. The side-effect is a
stale-closure bug: the effect captures `filters` and `hasBackendFilters`
from the render where its deps last changed, not from the render where
the user selected, e.g., a Key Alias.
Reproduce: set Key Alias → results appear correctly → change page or
sort → the effect fires with the OLD `filters` snapshot (no key_alias)
→ API request is sent without the filter → table shows unfiltered data.
Fix: store the latest `filters` and `hasBackendFilters` in refs that are
kept in sync on every render. The sort/page/time effect reads from the
refs instead of the closure so it always uses the current filter state
without altering the dep array.
Co-authored-by: Bytechoreographer <Bytechoreographer@users.noreply.github.com>
Co-authored-by: Claude Sonnet 4 (1M context) <noreply@anthropic.com>
When backend filters (e.g. Key Alias) are active on the Request Logs
page, the manual Fetch button called logs.refetch() which re-runs the
main TanStack Query. That query does not carry backend-only filter
params such as key_alias, so the button had two problems:
1. It fired a redundant API request without the active filters.
2. It did not refresh the filtered result set — backendFilteredLogs
stayed frozen at the last debounce-triggered fetch.
Fix: expose refetchWithFilters() from useLogFilterLogic and route the
Fetch button through it when hasBackendFilters is true. This cancels
any in-flight debounce and calls performSearch with the current filter
state, keeping all active filters intact.
Co-authored-by: Bytechoreographer <Bytechoreographer@users.noreply.github.com>
Co-authored-by: Claude Sonnet 4 (1M context) <noreply@anthropic.com>
* fix(anthropic): handle tool_choice type 'none' in messages API
* test(anthropic): add regression test for tool_choice type 'none'
---------
Co-authored-by: BillionClaw <267901332+BillionClaw@users.noreply.github.com>
Co-authored-by: Krrish Dholakia <krrish+github@berri.ai>
When reasoning_auto_summary is enabled (via litellm_settings or env var),
automatically set thinking.display="summarized" on native /v1/messages
requests. This ensures thinking content is returned in the response
instead of being omitted (the default on Claude 4.7+).
Only applies when thinking is enabled (type != "disabled").
The existing reasoning_auto_summary flag already handles the
/v1/responses path (summary="detailed") and the chat/completions
adapter path — this extends coverage to the native messages handler.
`get_file_ids_from_messages` and `update_messages_with_model_file_ids`
assume every content block with `type: "file"` has a nested `file` dict in
the OpenAI Chat Completions shape. That assumption is too strong: `type:
"file"` is a public content-block discriminator and several real producers
emit blocks that use it without the OpenAI `file` sub-dict. For example,
LangChain v1's `_normalize_messages` rewrites OpenAI file blocks into
`{"type":"file","id":"...","base64":"...","mime_type":"...","extras":{}}`
before they reach LiteLLM.
`AnthropicConfig.validate_environment` calls both helpers unconditionally
on every Anthropic (and Anthropic-via-Vertex) request, so any such block
raises `KeyError: 'file'` which the Vertex partner layer then wraps as a
`500 InternalServerError` before the LLM is even contacted.
This patch switches both helpers from `c["file"]` to a defensive
`c.get("file")` + dict check. When the block does not match the OpenAI
shape there is no file_id to extract or remap, so we skip it and leave
the block untouched for the downstream provider transformer to handle.
Adds 5 regression tests covering the LangChain v1 shape, the OpenAI
happy path, mixed shapes in one message, `file` set to a non-dict value,
and the remap path for non-OpenAI blocks.
Related to #24503, which proposed raising `BadRequestError` in the same
spots. For these two discovery functions specifically, the skip semantics
is strictly more permissive: well-formed OpenAI blocks still yield their
file_id, and legitimate non-OpenAI blocks stop crashing the request.
* fix(model-info): include reasoning effort support fields in get_model_info
_get_model_info_helper constructs ModelInfoBase explicitly but never
reads supports_xhigh/minimal/none_reasoning_effort from the cost map
JSON. Add the three fields so get_model_info() returns them correctly.
Also add supports_minimal_reasoning_effort to the ModelInfo TypedDict
(xhigh and none were already declared, minimal was missing).
* fix(model-registry): add missing reasoning effort fields for claude 4.6/4.7
Claude Opus 4.7 supports max reasoning effort (above xhigh).
The field was present for Opus 4.6 but missing for all Opus 4.7
entries (base, dated, Bedrock, Vertex AI, Azure AI).
All Claude 4.6/4.7 models (Opus 4.6, Sonnet 4.6, Opus 4.7) support
minimal reasoning effort via adaptive thinking. Add the field to all
provider variants.
* fix(adapter): map output_config.effort to reasoning_effort (#25079)
Anthropic's adaptive thinking (thinking.type="adaptive") and
output_config.effort were silently dropped when translating to
OpenAI format, resulting in no reasoning_effort on the outgoing
request.
Adapter changes (format translation):
- adapters/transformation.py: add "adaptive" branch to
translate_anthropic_thinking_to_reasoning_effort(); pass through
output_config.effort as-is in _translate_thinking_to_openai();
add "output_config" to translatable_anthropic_params
- adapters/handler.py: extract output_config from extra_kwargs into
request_data so it reaches the translation layer
- responses_adapters/transformation.py: add "adaptive" branch and
output_config param to translate_thinking_to_reasoning()
Handler changes (model-aware normalization):
- utils.py: add normalize_reasoning_effort_value() that uses
get_model_info() to map "max" → "xhigh"/"high" and
"minimal" → "minimal"/"low" based on model capabilities
- adapters/handler.py: call normalization before responses routing
- responses_adapters/handler.py: call normalization after translation
Relates to BerriAI/litellm#25079
* test(reasoning-effort): add tests for effort capability fields and normalize logic
Test coverage for:
- get_model_info returning supports_minimal/max_reasoning_effort fields
- JSON registry entries for claude 4.6/4.7 across all providers
- normalize_reasoning_effort_value degradation chains and exception fallback
- Adapter translation of adaptive thinking + output_config.effort
* fix: forward custom_llm_provider to normalize_reasoning_effort_value in responses adapter
* fix(mcp_semantic_tool_filter): match canonical tools that arrive with
a client-side namespace prefix.
`SemanticMCPToolFilter._get_tools_by_names` matched by exact equality
between the canonical name stored in the router
(`<server><MCP_TOOL_PREFIX_SEPARATOR><tool>`) and the name in the
incoming `tools[]` list. MCP clients such as opencode wrap every tool
name with their own additive alias prefix
(`<client_alias>_<canonical>`), so the two never matched, the filter
dropped every tool to zero, and the proxy forwarded `tools: []` with
`tool_choice: auto` — which strict upstream providers reject with a 400.
The fix adds anchored suffix matching with a separator check: the
canonical must form the complete tail of the incoming name and be
preceded by `_` or `-`. Exact matches still win over suffix matches,
incoming tools are returned at most once, and the original tool object
is passed through unchanged so the client-facing name survives for
tool-call round-trips.
Seven unit tests in a new TestGetToolsByNames class cover exact
match, underscore- and dash-prefixed variants, non-separator-anchored
suffixes (which must not match), exact-wins-over-prefixed precedence,
deduplication when two canonicals suffix-match the same incoming tool,
and ordering-follows-router-output.
Fixes#26078
* review: strengthen the suffix-fallback tie-breaker and the
deduplication regression test (Greptile comments on #26117)
- test_same_tool_not_returned_twice now passes two distinct canonicals
("read_file" and "file") that both suffix-match the same incoming
tool, rather than the same canonical twice, so the assertion
actually exercises the used_ids dedup path instead of the
duplicate-input-list path.
- The suffix fallback in _get_tools_by_names now prefers the shortest
incoming name that still qualifies under the separator-anchored
match. In the one-prefix-per-client opencode scenario this is a
no-op, but in multi-namespace configurations the shortest qualifying
name is the least-wrapped one and is the most defensible deterministic
choice, replacing the dict-insertion-order fallback.
- Adds test_suffix_fallback_prefers_shortest_candidate covering the
new tie-breaker directly.
Still 15 tests passing locally (was 14).
* review(#26117): gate suffix-matching on canonical containing MCP_TOOL_PREFIX_SEPARATOR
@krrish-berri-2 flagged a possible collision in the suffix fallback:
a local user function whose name happens to end in a bare canonical
substring (e.g. my_firecrawl_scrape vs canonical firecrawl_scrape)
would be spuriously selected.
Server-registered MCP tools are always emitted as
<server_name><MCP_TOOL_PREFIX_SEPARATOR><tool_name> via
add_server_prefix_to_name, so a canonical without the separator is
not a namespaced MCP tool and does not warrant suffix matching.
Added that guard to _name_matches_canonical with a regression test
(test_does_not_collide_with_local_function_on_unprefixed_canonical)
that reproduces the collision before the fix and is pinned after.
Pre-existing TestGetToolsByNames fixtures that relied on bare
canonicals (get_weather, search, read_file, write/delete/read) were
switched to realistic server-prefixed ones so they continue to
exercise the suffix-fallback path under the new guard. The opencode
scenario (client prefix on already-server-prefixed canonical) is
unchanged.
---------
Co-authored-by: sakenuGOD <sakenuGOD@users.noreply.github.com>
Co-authored-by: Krrish Dholakia <krrish+github@berri.ai>
Unit Tests: Proxy DB Operations / proxy-db (auth-checks, tests/proxy_unit_tests/test_auth_checks.py tests/proxy_unit_tests/test_user_api_key_auth.py, 20, 8) (push) Waiting to run
Unit Tests: Proxy DB Operations / proxy-db (remaining, tests/proxy_unit_tests --ignore=tests/proxy_unit_tests/test_key_generate_prisma.py --ignore=tests/proxy_unit_tests/test_auth_checks.py --ignore=tests/proxy_unit_tests/test_user_api_key_auth.py --ignore=tests/proxy_unit_tests/test_p… (push) Waiting to run
