* test: drop the cwd-relative sys.path.insert calls from the test suite
TQ003 stands at 1,077 across 1,058 files, and 1,015 of them are the same shape:
sys.path.insert(0, os.path.abspath("../..")) and its deeper siblings. The
argument resolves against the working directory rather than the file, so from
the repo root, where every job runs pytest, it inserts the directory two levels
above the checkout. It has never pointed at litellm. The package is installed
into the environment anyway, which is what actually makes the import work, and
what the rule's message has said all along.
Removing them leaves 1,634 imports of sys and os with no remaining reference,
and those go too, except where another test module imports the name back out of
the file. The rest of TQ003 is 62 call sites that resolve against __file__ or a
variable, which are a different question and are left alone.
Collection is identical either way: 45,871 tests and the same 51 pre-existing
collection errors before and after, and ruff reports no new undefined name.
* test: drop the duplicate imports the sys.path sweep exposed to F811
* test(pre-call-utils): restore the os import the new bedrock tests need
* fix(bedrock): normalize Messages system role and adaptive-thinking for Claude Invoke
* style(bedrock): use builtin generics in new Invoke helpers to clear UP006 gate
* fix(bedrock): honor explicit thinking budget_tokens=0 in clear_thinking conversion
The clear_thinking_20251015 -> adaptive conversion resolved the thinking
budget with `thinking.get("budget_tokens") or BEDROCK_MIN_THINKING_BUDGET_TOKENS`,
which treats a caller-supplied `budget_tokens=0` as missing and silently
substitutes the Bedrock minimum. Resolve the budget with an explicit
`is not None` check so an explicit 0 is honored.
* fix(bedrock): gate Fable 5 into clear_thinking adaptive injection on Invoke
_ensure_thinking_for_clear_thinking_context_management returns early when
_supports_extended_thinking_on_bedrock(model) is False, so the adaptive-thinking
injection never runs for models absent from that gate. Opus 4.8 slips through on
the incidental "opus-4" substring, but Fable 5 had no matching pattern, so a
clear_thinking_20251015 request on Fable 5 reached Bedrock with an unsupported
context-management edit and no thinking field; the exact 400 this path exists to
prevent. Add the fable-5 patterns to the gate so Fable 5 (mapped ids and unmapped
aliases) gets thinking.type=adaptive + output_config.effort like the other
adaptive models.
Extend the adaptive-injection regression test to cover Fable 5 (a mapped id and
an unmapped alias) so it fails without the gate entry, and add focused coverage
for the budget->effort tiers, the disabled/enabled/adaptive thinking branches,
output_config.effort preservation, and list/dict system-role normalization.
Also normalize the Invoke transformation module and its test to line-length 88
so ruff format --check (CI format-check) passes.
* refactor(anthropic): make supports_adaptive_thinking flag authoritative for thinking detection
Replace the per-version name helpers (_is_claude_4_6/4_7/4_8_model,
_is_claude_fable_5_model) with cost-map-flag-first detection. _is_adaptive_thinking_model
now reads supports_adaptive_thinking from the model cost map and falls back to a single
generalized family-version regex (_claude_version_at_least(model, 4, 6)) only when a model
is unmapped, instead of hard-coding each new Claude release.
Wire supports_adaptive_thinking through ProviderSpecificModelInfo and ModelInfo so the cost
map flag actually surfaces at lookup time. Reroute the Bedrock Invoke extended-thinking gate
and the two anthropic/chat/transformation.py call sites through _is_adaptive_thinking_model.
Known gap left to the fallback_generalizations work (#29718): unmapped Fable 5 aliases have
no parseable minor version, so they defer to the cost map and are not detected until a mapped
entry or a generalization rule exists. Covered by an explicit regression test.
* refactor(anthropic): drop name-based version fallback; resolve adaptive thinking from cost map only
The prior commit kept a regex (_claude_version_at_least) as a fallback when an id
resolved to no cost-map entry. Remove it: _is_adaptive_thinking_model now reads
supports_adaptive_thinking and nothing else, so "which Claude versions think
adaptively" lives entirely in the model cost map, and a new adaptive release is a
JSON edit rather than a Python edit.
To keep the flag authoritative across the id forms the Bedrock Invoke and anthropic
paths actually see, backfill supports_adaptive_thinking=true on every adaptive Claude
entry that was missing it (Opus 4.6/4.7 and Sonnet 4.6 across region/provider aliases)
in both the root and bundled cost maps, and generalize _model_map_lookup_candidates to
normalize an id to its base cost-map key: strip a Bedrock version suffix (-v1:0 fully,
or just the :0 inference-profile minor so the -v1-keyed 4.6 entries resolve), strip a
dated-release suffix (-20260219), and rewrite a dotted family version (4.6 -> 4-6).
This is id normalization feeding the lookup, not capability-by-name.
Tests load the PR-local cost map (the flags are not on main until merge) and cover each
normalization path plus the unmapped-alias deferral to fallback_generalizations (#29718).
* refactor(reasoning_effort): single-source effort<->thinking-budget mappings
Route every reasoning_effort <-> thinking-budget conversion through the DEFAULT_REASONING_EFFORT_*_THINKING_BUDGET constants so the numbers stay in sync across providers. The five constants are now 2000/5000/10000/20000/40000
Add reasoning_effort_from_thinking_budget() in litellm_core_utils/reasoning_effort_utils.py and route the three OpenAI-style forward maps (anthropic adapters, responses adapters, hosted_vllm) through it. The bedrock invoke and experimental messages adaptive maps now reference the constants directly; the only behavior change is the xhigh threshold moving from 24000 to 20000. Reverse maps and the cross-provider test grid read the same constants
* test(reasoning_effort): lift budget-mode max_tokens above the new high budget
The single-sourced DEFAULT_REASONING_EFFORT_*_THINKING_BUDGET thresholds moved
high from 4096 to 10000. The live reasoning_effort grid sends budget-mode
requests with max_tokens=8192, so reasoning_effort=high now produces
budget_tokens=10000 > max_tokens and every provider returns 'max_tokens must be
greater than thinking.budget_tokens'. Derive a shared BUDGET_MODE_MAX_TOKENS
(2x the high budget) for the spec and the request builder so the ceiling always
clears the largest 200-expected tier. Also resolve the inherited base
test_reasoning_effort assertion off the same high-budget constant instead of the
stale 4096 literal so it tracks the source of truth.
* fix(reasoning_effort): keep effort<->budget thresholds at pre-PR values
The single-sourcing refactor moved the shared effort<->budget thresholds up
(low 1024->2000, medium 2048->5000, high 4096->10000, xhigh 8192->20000,
max 16384->40000). That silently changes the effort->budget direction: a caller
who sets reasoning_effort together with a max_tokens that used to sit above the
old per-tier budget but below the new one now trips the provider's
"max_tokens must be greater than thinking.budget_tokens" 400. It spans every
backend that derives a budget from an effort (Anthropic, Gemini/Vertex,
hosted vLLM), not just Bedrock.
Restore the constants to their pre-PR values while keeping every backend reading
from the shared DEFAULT_REASONING_EFFORT_*_THINKING_BUDGET constants, so the
mapping stays single-sourced without the behavior change. Tests that pinned the
raised thresholds now derive their boundaries from the same constants.
* test(reasoning_effort): derive high effort->budget assertions from the shared constant
The cross-provider translation tests pinned reasoning_effort="high" to a literal
budget_tokens=10000, the raised value. Point them at
DEFAULT_REASONING_EFFORT_HIGH_THINKING_BUDGET so they track the single source
instead of a magic number.
* fix(anthropic): resolve adaptive flag for combined dated+versioned Bedrock ids
The model-map candidate normalization applied each suffix strip independently to
the original id, so the real Bedrock shape "<base>-<YYYYMMDD>-v1:0" never reduced
to its base cost-map key: stripping the version left the date, and the
dated-suffix regex is anchored to the end so it could not fire while the version
was still present. An adaptive Claude model invoked by its full dated+versioned
id (e.g. us.anthropic.claude-sonnet-4-6-20251101-v1:0) therefore resolved to
supports_adaptive_thinking=null and was treated as non-adaptive, reaching Bedrock
with the rejected thinking.type=enabled shape, the exact 400 this path prevents.
Add a composed normalization that rewrites the dotted family version, then peels
the -vN:rev version suffix, then the -YYYYMMDD dated suffix, so the combined form
resolves to its base key. Regression tests pin the combined suffix on sonnet-4-6
and opus-4-8 across provider/region prefixes.
* fix(reasoning_effort): align budget<->effort tests with reverted constants and format common_utils
The constant revert restored the effort<->budget thresholds to their pre-PR
values (1024/2048/4096/8192/16384) and single-sourced the reverse
budget->effort ladder through reasoning_effort_from_thinking_budget, but
several tests still pinned the briefly-raised values and the old hardcoded
reverse buckets, so the "All Other Providers" shard failed
Derive the anthropic chat effort->budget assertions from the shared
DEFAULT_REASONING_EFFORT_*_THINKING_BUDGET constants, and update the
experimental pass-through and responses adapter expectations to the
single-sourced reverse ladder (budget 1024 -> low, 5000 -> high)
Also run ruff format --line-length 88 over anthropic/common_utils.py so the
CI format-check, which checks the whole changed file, passes
* fix(proxy): optionally surface public team model name in /v1/models
Behind general_settings.use_team_public_model_name (default False). When
enabled, /v1/models and /models surface the public team_public_model_name
for team-scoped (BYOK) models instead of the internal routing key
model_name_{team_id}_{uuid} -- consistent with /v1/model/info and
OpenAI-compatible. Off by default so the listing's model ids stay
backward-compatible for callers that scripted against the internal name;
routing by the internal name is unchanged regardless of the flag.
Presentation-layer only: access-group, auth, and routing semantics are
unchanged; non-team models are pass-through.
* fix(proxy): default team model listings to public names
* test(proxy): cover team model listing metadata
* test(proxy): cover empty team listing deployments
* refactor(proxy): simplify team model listing translation
* fix(proxy): resolve public team model name on GET /v1/models/{id}
The listing endpoints advertise team_public_model_name, but the retrieve
endpoint validated and looked up by the raw id, so a public name 404'd.
Resolve the public name back to the internal routing key (scoped to the
caller's accessible models so colliding names never cross teams), look up
by it, and echo the public name back as the response id.
* test(proxy): cover public-name resolution on model retrieve
* refactor(proxy): extract team model-name translation into TeamModelNameTranslator
Move the team-scoped (BYOK) listing/retrieve name translation out of
proxy_server.py into a dedicated common_utils module. Static methods with
general_settings injected so the logic is unit-testable without globals and
proxy_server.py stays thin.
* refactor(proxy): use TeamModelNameTranslator in model_list and model_info
* test(proxy): target TeamModelNameTranslator for model-name translation
* fix(proxy): type create_model_info_response return as dict[str, object]
* fix(proxy): keep internal routing key for team model listing metadata lookup
Add listing_entries returning (public response id, internal lookup id) so
include_metadata=true resolves fallbacks against the routing key the router
indexes by, instead of the translated public name (which never matches).
* fix(proxy): build /v1/models metadata from internal key, show public id
* test(proxy): cover team listing fallback metadata via internal key
* fix(proxy): use builtin dict generics in create_model_info_response (UP006)
---------
Co-authored-by: Tushar More <tusharmore8408@gmail.com>
Co-authored-by: Ishaan Jaffer <ishaanjaffer0324@gmail.com>
* Fix remaining VCR live-call leaks
* test(vcr): dedupe live-test helpers and drop spurious kwargs
Extract the duplicated isVertexQuotaError/runVertexRequestOrSkip Vertex
quota-skip helpers into tests/pass_through_tests/vertex_test_helpers.js and the
duplicated _skip_live_prompt_caching_test guard into tests/_live_test_helpers.py
so each lives in one place. In test_aarun_thread_litellm, build a separate
message_data carrying role/content for add_message and a thread_data without
them for run_thread/run_thread_stream/get_messages, which no longer receive the
spurious message fields.
* test(overhead): assert mock transport is exercised in non-streaming and stream tests
- Add `_strip_image_b64_payloads` filter: rewrites `data[*].b64_json` in
image-gen responses to a 4-byte placeholder before the cassette is saved.
Image-edit and image-gen cassettes (193 MB / 184 MB / 104 MB / ...) will
shrink to <100 KB on next record. Tests assert response shape only, so
coverage is preserved.
- Add `_normalize_multipart_boundary` filter: replaces httpx's per-request
random multipart boundary with a fixed string in both Content-Type header
and body bytes. Audio-transcription / Whisper tests have been effectively
unmocked — every CI run hit live providers and was silently capped at
MAX_EPISODES_PER_CASSETTE=50. Both record and replay now see identical
bytes; the safe_body matcher works.
- Fix test_evals_api.py body poisoning: replace `int(time.time())` in eval
names with `hashlib.sha1(test_node_name)[:12]`, add a function-scoped
`managed_eval` fixture that creates and deletes the eval, and switch
`get_eval` / `update_eval` from `list_evals().data[0].id` (which made
the URL vary by run) to `managed_eval.id`. Net coverage gain: delete is
now actually exercised.
- Swap arxiv PDF URL in BaseOCRTest for the in-repo `dummy.pdf` (589 B)
served via sha-pinned jsdelivr.
- Swap etsystatic image URL in BaseLLMChatTest.test_image_url for the
in-repo LiteLLM logo (9.2 KB) served via the same jsdelivr pin.
- Add `tests/llm_translation/test_vcr_filters.py` with 14 unit tests
covering both new filters: replacement, idempotency, nesting, content-
length update, two-distinct-boundaries-converge-after-normalize, etc.
Cassettes recorded with the prior patterns will mismatch on the first CI
run after merge; recommend flushing the cassette Redis once (post-merge)
so re-records save under the new format from the start.
GitHub serves PDFs from raw.githubusercontent.com, github.com/.../raw/..., LFS, and Releases as application/octet-stream by deliberate anti-hotlinking policy. Anyone who passes a GitHub-hosted PDF URL as an OpenAI / Gemini / Bedrock file_id hits "unsupported MIME type 'application/octet-stream'" because _process_image_response inlines the URL with whatever Content-Type the server sent.
When the server-provided Content-Type is application/octet-stream or binary/octet-stream and the URL extension maps to a known MIME type (.pdf, .png, .jpg, etc.), trust the extension instead. Specific Content-Types (image/png, application/pdf) still win over the extension; the override only applies to generic binary types.
Also restores the Greptile SHA-pinned raw.githubusercontent.com URL on the file_id integration test so we test against the same hosting real users hit, no third-party CDN.
Resolve URL conflict: keep Greptile's commit-SHA pin for immutability, but route through jsDelivr (cdn.jsdelivr.net/gh/BerriAI/litellm@<sha>/...) so the response Content-Type is application/pdf instead of application/octet-stream. Without this, OpenAI / Gemini / Router PDF tests reject the inlined file_data with "unsupported MIME type 'application/octet-stream'".
The previous URL switch to raw.githubusercontent.com fixed Anthropic's "Unable to download" failure but caused OpenAI / Gemini / Router PDF tests to fail with "unsupported MIME type 'application/octet-stream'": those providers download the URL and inline it as data:<Content-Type>;base64,..., and raw.githubusercontent.com serves PDFs as application/octet-stream.
jsDelivr proxies the same in-repo fixture (cdn.jsdelivr.net/gh/BerriAI/litellm@main/...) and returns the correct Content-Type: application/pdf, so all providers (Anthropic forwards the URL natively; OpenAI/Gemini/Bedrock fetch and inline) get the right MIME type without changing transformer code.
The test was passing the Wikipedia URL https://upload.wikimedia.org/wikipedia/commons/2/20/Re_example.pdf as the file_id, which Anthropic's URL fetcher can no longer download (returns "Unable to download the file"). The URL is healthy for generic clients but Anthropic's fetcher fails on it deterministically, so the test has been red across PRs on litellm_internal_staging.
Switch to the in-repo fixture at tests/llm_translation/fixtures/dummy.pdf served via raw.githubusercontent.com - same fixture used elsewhere in the repo, no external CDN dependency that can block by user-agent.
* fix: use fastuuid helper across the codebase
First batch of changes, simple drop in replacement.
* second batch of changes
* fixed: script mistake on helper file
* fix(streaming_handler.py): maintain same 'created' across all chunks
Fixes https://github.com/BerriAI/litellm/issues/11437
* test: add unit test to ensure created is always the same across all chunks
* fix(types/utils.py): set a tool call id, if missing in delta tool call
Ensures stream chunk builder can reconstruct tool calls correctly
Fixes https://github.com/BerriAI/litellm/issues/11262
* fix(responses/transformation.py): support passing mcp server tool call to anthropic
allows switching between openai and anthropic for mcp tool calling
* fix(ollama/chat/transformation.py): set tool call id's when missing
* fix(anthropic/): fix regression when passing file url's to the 'file_id' parameter
add test and ensure anthropic file url's are correctly sent as 'document' blocks
* fix(vertex_and_google_ai_studio.py): Use same usage calculation function as non-streaming
Closes https://github.com/BerriAI/litellm/issues/10667
* test(test_vertex_and_google_ai_studio_gemini.py): update test
* feat(parallel_request_limiter_v2.py): add sliding window logic
allows rate limiting to work across minutes
* fix(parallel_request_limiter_v2.py): decrement usage on rate limit error
* fix(base_routing_strategy.py): fix merge from redis - preserve values in in-memory cache during gap b/w push to redis and read from redis
* fix(base_routing_strategy.py): catch the delta change during redis sync
ensures values are kept in sync
* fix(parallel_request_limiter_v2.py): update tpm tracking to use slot key logic
* fix: fix linting error
* test: update testing
* test: update tests
* test: skip on rate limit or internal server errors
* test: use pytest fixture instead
* test: bump mistral model
* fix(openai/gpt_transformation.py): handle missing filename for openai file data call
* fix(openai/gpt_transformation.py): clean handling for sync + async pdf url transformation flows
Fixes https://github.com/BerriAI/litellm/issues/10820
* build(model_prices_and_context_window.json): add 'supports_pdf_input' for all openai models which have 'vision' support
Follows openai guidelines
* feat(bedrock/chat): support cache pointing tool calls on Bedrock
Closes https://github.com/BerriAI/litellm/pull/10613
* fix: fix linting error
* add test_function_calling_with_tool_response to base llm tests
* run test suite for nova
* update test_function_calling_with_tool_response
* allowed ToolJsonSchemaBlock keys
* fix ToolJsonSchemaBlock
* add back pytest fixture
* test: test_prompt_caching
* test(base_llm_unit_tests.py): return '<thinking>' tag in response content
* fix(converse_transformation.py): extract `<thinking>` block from nova tool use response
Fixes https://github.com/BerriAI/litellm/issues/9063
* fix(factory.py): handle non-signature reasoning blocks to bedrock
pass as text input - bedrock raises ""User messages cannot contain reasoning content. Please remove the r
easoning content and try again." otherwise
* fix(main.py): Add drop params support for gpt
Fixes https://github.com/BerriAI/litellm/issues/10501
* fix(converse_transformation.py): fix linting error
* fix(utils.py): fix linting error
* test: cleanup test
* test: skip test until we have bedrock prompt caching permission
* fix(vertex_and_google_ai_studio.py): fix finish reason to be 'tool_calls' when tool call returned
Vertex returns 'Stop', openai format is 'tool calls'
* test(base_llm_unit_tests.py): bump test to assert tool calls in finish reason
* feat(fireworks_ai/chat): handle tool calling with fireworks ai correctly
Fixes https://github.com/BerriAI/litellm/issues/7209
* fix(utils.py): handle none type in message
* fix: fix model name in test
* fix(utils.py): fix validate check for openai messages
* fix: fix model returned
* fix(main.py): fix text completion routing
* test: update testing
* test: skip test - cohere having RBAC issues
* build(model_prices_and_context_window.json): add vertex ai gemini-2.5-flash pricing
* build(model_prices_and_context_window.json): add gemini reasoning token pricing
* fix(vertex_and_google_ai_studio_gemini.py): support counting thinking tokens for gemini
allows accurate cost calc
* fix(utils.py): add reasoning token cost calc to generic cost calc
ensures gemini-2.5-flash cost calculation is accurate
* build(model_prices_and_context_window.json): mark gemini-2.5-flash as 'supports_reasoning'
* feat(gemini/): support 'thinking' + 'reasoning_effort' params + new unit tests
allow controlling thinking effort for gemini-2.5-flash models
* test: update unit testing
* feat(vertex_and_google_ai_studio_gemini.py): return reasoning content if given in gemini response
* test: update model name
* fix: fix ruff check
* test(test_spend_management_endpoints.py): update tests to be less sensitive to new keys / updates to usage object
* fix(vertex_and_google_ai_studio_gemini.py): fix translation
* fix(litellm_proxy/chat/transformation.py): support 'thinking' param
Fixes https://github.com/BerriAI/litellm/issues/9380
* feat(azure/gpt_transformation.py): add azure audio model support
Closes https://github.com/BerriAI/litellm/issues/6305
* fix(utils.py): use provider_config in common functions
* fix(utils.py): add missing provider configs to get_chat_provider_config
* test: fix test
* fix: fix path
* feat(utils.py): make bedrock invoke nova config baseconfig compatible
* fix: fix linting errors
* fix(azure_ai/transformation.py): remove buggy optional param filtering for azure ai
Removes incorrect check for support tool choice when calling azure ai - prevented calling models with response_format unless on litell model cost map
* fix(amazon_cohere_transformation.py): fix bedrock invoke cohere transformation to inherit from coherechatconfig
* test: fix azure ai tool choice mapping
* fix: fix model cost map to add 'supports_tool_choice' to cohere models
* fix(get_supported_openai_params.py): check if custom llm provider in llm providers
* fix(get_supported_openai_params.py): fix llm provider in list check
* fix: fix ruff check errors
* fix: support defs when calling bedrock nova
* fix(factory.py): fix test
* test: move test to just checking async
* fix(transformation.py): handle function call with no schema
* fix(utils.py): handle pydantic base model in message tool calls
Fix https://github.com/BerriAI/litellm/issues/9321
* fix(vertex_and_google_ai_studio.py): handle tools=[]
Fixes https://github.com/BerriAI/litellm/issues/9080
* test: remove max token restriction
* test: fix basic test
* fix(get_supported_openai_params.py): fix check
* fix(converse_transformation.py): support fake streaming for meta.llama3-3-70b-instruct-v1:0
* fix: fix test
* fix: parse out empty dictionary on dbrx streaming + tool calls
* fix(handle-'strict'-param-when-calling-fireworks-ai): fireworks ai does not support 'strict' param
* fix: fix ruff check
'
* fix: handle no strict in function
* fix: revert bedrock change - handle in separate PR
* fix(internal_user_endpoints.py): cleanup unused variables on beta endpoint
no team/org split on daily user endpoint
* build(model_prices_and_context_window.json): gemini-2.0-flash supports audio input
* feat(gemini/transformation.py): support passing audio input to gemini
* test: fix test
* fix(gemini/transformation.py): support audio input as a url
enables passing google cloud bucket urls
* fix(gemini/transformation.py): support explicitly passing format of file
* fix(gemini/transformation.py): expand support for inferred file types from url
* fix(sagemaker/completion/transformation.py): fix special token error when counting sagemaker tokens
* test: fix import
* fix(anthropic/chat/transformation.py): Don't set tool choice on response_format conversion when thinking is enabled
Not allowed by Anthropic
Fixes https://github.com/BerriAI/litellm/issues/8901
* refactor: move test to base anthropic chat tests
ensures consistent behaviour across vertex/anthropic/bedrock
* fix(anthropic/chat/transformation.py): if thinking token is specified and max tokens is not - ensure max token to anthropic is higher than thinking tokens
* feat(converse_transformation.py): correctly handle thinking + response format on Bedrock Converse
Fixes https://github.com/BerriAI/litellm/issues/8901
* fix(converse_transformation.py): correctly handle adding max tokens
* test: handle service unavailable error
