* test: load the embedding base image from a committed 100x100 PNG instead of downloading it
* test: move the volcengine embedding test into tests/unit
* test: check gpt2 and r50k_base tokenizer parity against committed tiktoken reference files
* test: check hub tokenizer selection against an in-memory Hugging Face hub
* test: serve image URLs from respx in the gemini tool-result and format-param tests
* ci: drop the emptied legacy core-utils test path
* test: cover the cohere and anthropic tokenizer paths in the hub tokenizer test
* test: fetch every format-param image through respx and check its bytes reach the request
* test: drop the gpt2 and r50k_base parity tests, which no litellm path uses
* test: drop comments that restate assertions in the format-param test
* ci: run the unit_selection.sh shard files on every event instead of only fork pull requests
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* ci: rename fork-flag to unit-flag now that it applies on every event
* test: move tests/test_litellm root and small trees into tests/unit
Pure renames, no content changes. Follow-up commits in this PR fix
references, merge the three files that already existed in tests/unit,
keep live-provider tests in tests/test_litellm and wire CI.
* test: carry tests/test_litellm conftest isolation into tests/unit
Callback lists, routing fallbacks, cached HTTP clients, logger state, AWS,
proxy-URL and keychain env, and session-end client cleanup now reset for
unit tests too. The environment isolation owns its MonkeyPatch so a test's
own monkeypatch is undone before the model-cost teardown runs.
* test: merge, split and prune the moved root and small-tree tests
Merge batches/test_batch_utils.py and the chat_completions and messages
dispatch tests into the files that already existed in tests/unit. Keep
the live Gemini interactions tests, the async image-fetch format test and
the OpenAI embedding scorer test in tests/test_litellm since they need
real network or keys. Put test_router.py under tests/unit/test_router so
the existing package no longer shadows it. Delete eight tests the audit
found superseded by stronger ones kept in this move.
* ci: run the moved root and small-tree tests under their legacy flags
Add the misc and responses-caching-types flags to unit_selection.sh and
CircleCI, extend enterprise-routing and mcp-integration, and point the
legacy GHA shards, Makefile, redis-compat workflow, merge smoke manifest
and change classifier at the new paths.
* test: make the new tests/unit directories packages
tests/unit/test_package_layout.py requires every directory to carry an
__init__.py, and without one the moved and retained
test_litellm_responses_bridge.py modules collide on import.
* test: scope the unit socket block to tests/unit in shared sessions
The GHA shards collect the legacy test-path and the unit selection in one
pytest session. The unit conftest's loopback-only block leaked into legacy
modules that reach the network at import. The legacy conftest now lifts the
restriction at collect and setup time, and the unit conftest re-applies it
when collecting its own modules.
* test: move tests/test_litellm/llms into tests/unit/llms
Rename-only. Moves the provider tests and the fine-tuning fixtures they
load, mirroring the old paths. Follow-up commits merge, split and wire them.
* test: merge, split and prune the moved llms tests
Merges the Databricks chat transformation tests into the existing unit
file, keeps the tests that need real keys or the network in
tests/test_litellm, deletes the audited tests a stronger unit test
already covers, and points imports at tests.unit.llms.
* ci: run the moved llms tests under their legacy flags
The Vertex AI and All Other Providers shards keep their legacy test-path
for the retained files and add the llm-vertex-ai and llm-other-providers
unit selections. CircleCI gets matching unit jobs.
* test: make the tests/unit/llms directories packages
Adds __init__.py to the moved dirs and drops the legacy ones whose
directories no longer hold tests.
* test: drop script runners and path hacks the llms split left dangling
The __main__ runners in the split openai_like files and the Databricks e2e
runner called tests that now live in the other half of the split or were
deleted. The retained legacy halves also no longer need sys.path edits.
* test: give the shard-script tests their own GITHUB_OUTPUT
They only passed where the runner set it. The CircleCI unit job's env
allowlist drops it, so the script's redirect failed there.
* test: point the router and module-deletion checks at tests/unit
router_code_coverage and code_qa_check_tests only searched tests/test_litellm,
so the moved router tests no longer counted. The two silent-experiment tests
the audit deleted were the only direct callers of those methods; they are
replaced with tests that assert the forwarded shadow request and the
recursion guard.
* test: keep the Databricks manual e2e runner and fix the SageMaker Nova run path
The Databricks e2e file is a manual script whose main() calls the tests
that were pruned, so pruning them broke the documented run. It is back to
its main version. The SageMaker Nova docstring now points at the file's
real location in tests/local_testing.
* test: keep the job's UNIT_FLAG out of the shard-script tests
---------
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(anthropic): surface Responses bridge stream failures as Anthropic error events
The /v1/messages Responses bridge logged every upstream failure and ended the
SSE stream as if it had completed, so a rate limit, a provider 500, a dropped
connection, or a read timeout reached the client as HTTP 200 with a lone
message_start and no error event. Map response.failed and any raised upstream
exception to a redacted Anthropic error frame, stop pulling upstream after it,
and never fabricate end_turn or message_stop after a failure.
* fix(anthropic): close a Responses bridge stream that ends without a terminal event with an error event
Normalize the failure status behind the error type to an int or digit string
within 400..599, narrow the response.failed event through pydantic, reuse the
native Messages path's incomplete-stream message for a clean upstream EOF, and
cover the pydantic event, the unwrapped fallback error, and the EOF cases
---------
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
* feat(dd_span_tagger): emit litellm_user_email span tag for JWT-authenticated requests
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* refactor(dd_span_tagger): use dotted litellm.user_email tag for consistency
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* feat(model_hub): surface model_info.description in model group info and Model Hub UI
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* refactor(router): aggregate model group description without in-place mutation
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(health): skip background health check DB writes when the latest-row read fails
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* feat(providers): add Nadir intelligent-router provider (nadir/auto)
Nadir (https://getnadir.com) is an OpenAI-compatible intelligent router.
A single virtual model, nadir/auto, is classified server-side and routed to
the cheapest model that clears the quality bar. The response reports the
routed model in the model field, so LiteLLM cost tracking prices the real
underlying model.
- litellm/llms/nadir/chat/transformation.py: NadirConfig(OpenAIGPTConfig)
- register nadir across enum, provider lists, get_llm_provider, __init__,
lazy imports, utils, get_supported_openai_params
- add https://api.getnadir.com/v1 to openai_compatible_endpoints so base_url
only usage reverse-maps to the provider
- provider_endpoints_support.json entry (chat_completions only)
- docs page + unit tests (11 passing)
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* fix(nadir): scope credentials to the trusted base and validate reported cost
NADIR_API_KEY only loads for the https default base, the SDK no longer
falls back to litellm.api_key, the reported cost is validated before it
reaches the spend log, and streams price from the routed model.
---------
Co-authored-by: milan <milan@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: yassin <yassin@berri.ai>
Co-authored-by: ryan-crabbe-berri <ryan@berri.ai>
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
* feat(anthropic): placement policy for mid-conversation system messages
Pure functions over the OpenAI-format message list: split off the leading
system run, keep later system messages as role=system at a placement Anthropic
accepts on models flagged supports_mid_conversation_system (after a user turn,
before an assistant turn or the end, never adjacent), and convert them to user
turns in place elsewhere, keeping tool_result first in a merged user turn.
* fix(anthropic): keep mid-conversation system out of the chat completions system prompt
translate_system_message hoisted every role=system message, at any index, into
the top-level system block. On a conversation carrying a mid-session reminder
that rewrites the cached prefix, so the provider re-bills the whole history at
cache-write pricing on every turn (#36559). #36968 fixed this on /v1/messages;
the chat completions path, shared by first-party Anthropic, Vertex, Azure AI
and Bedrock Invoke, still hoisted.
Only the leading system run becomes the system prompt now. Later system
messages go through the placement policy, and anthropic_messages_pt emits a
system message instead of rejecting the role. The caller's message list is no
longer mutated. Tests pin the two-turn prefix invariant across all four chat
configs and both flag states.
* refactor(anthropic): single-source the converted system note
The /v1/messages pass-through and the chat completions path must prefix a
converted system turn with the same operator note.
* test(e2e): prove the prompt cache survives a mid-conversation system reminder on chat completions
Same priming and assertions as the /v1/messages cases, through
/v1/chat/completions with OpenAI-format messages, for first-party Anthropic
and Bedrock Invoke on a flagged (Opus 4.8) and an unflagged (Haiku 4.5) model.
The reminder sits between the assistant turn and the next user turn, the shape
OpenAI-style agent frameworks send, which is the placement the chat path has
to translate.
* test(anthropic): cover the cache_control rebuild shapes and type the test helpers
Codecov flagged the 5m ttl branch and the empty-system path of the wire
builder; both now have a test. Greptile asked for full typing on the new
test helpers.
* refactor(anthropic): read the mid-conversation flag through a public supports_ helper
supports_mid_conversation_system joins the other supports_* helpers in
litellm.utils, so the chat transformation stops importing the private
_supports_factory.
* chore(typing): declare the mid-conversation type aliases with TypeAlias
The Final sweep tightened LIT010, which exempts TypeAlias declarations but
counts a bare alias assignment as an unannotated binding.
* fix(anthropic): let add_code_execution_tool take the pass-through message union
The translator now emits role=system inside messages for models that accept it,
so anthropic_messages_pt returns the pass-through union. add_code_execution_tool
still declared the narrower user/assistant union while only ever reading
content, so upstream's strip_advisor_blocks_from_messages call in between made
the mismatch visible to the type checker.
* fix(bedrock): keep mid-conversation system messages in place on converse path
* fix: ruff format + multi tool_result order + regression test
* fix: satisfy type-discipline gate + update osv ignore for mlflow PYSEC-2026-3865
* fix(bedrock): restore role narrowing in hoisted system loop for basedpyright budget
* test(bedrock): cover mid-conversation system conversion branches
- non-dict guard in _opens_with_tool_result
- in-place conversion without tool context
- str/list cache_control preservation in mid-conversation path
- drop unreachable non-system guard in hoisted loop
* Place type-discipline suppressions on the lines the gate scans
* Narrow hoisted loop to system role so basedpyright sees the right TypedDict
* fix(anthropic): place mid-conversation system runs by their neighbours only
A run after an assistant turn now slides behind the user turn that
immediately follows it, and a run that ends the array or precedes an
assistant turn becomes a user turn in place. No later message can move
an earlier run, so a client that replays the conversation with more
turns appended sends a byte-identical prefix and preserved thinking
blocks keep their binding
* refactor(bedrock): share the converted system note with the anthropic module
Converse imports CONVERTED_SYSTEM_NOTE instead of carrying its own copy
of the same text, and the reordering helpers lose their comments
* test: pin the replayed request prefix across preserved-thinking turns
One test per audited feature, through the real entrypoint: the chat
transformations for anthropic, bedrock invoke, vertex and converse, the
modify_params dummy tool result, dotprompt with unchanged variables, and
Presidio masking against an in-process fake. Each serializes system,
tools and the earlier messages of turn N and N+1 and asserts they match.
The e2e mid-conversation system test imports its content blocks from
models.py again and is marked provider_live
* fix(anthropic): move mid-conversation system placement into prompt_templates
The prompt factory imported the placement helper from the Anthropic provider
package, whose common_utils reads a factory constant at import time, so loading
the factory first raised ImportError. The module now sits next to
anthropic_messages_pt and every consumer imports core utils
A user turn with content [] or None puts no block on the wire, so a system run
anchored to it landed first in messages or behind an assistant turn. Such a run
now converts in place; empty strings and empty text blocks still anchor because
the factory fills them with a placeholder
* fix(anthropic): anchor system messages only on user turns that reach the wire
* fix(bedrock): type the converse system-message helpers over the message TypedDicts
* fix(anthropic): read replayed pydantic messages in the Converse helpers and convert a system run whose assistant follower sends nothing
A history that replays the previous turn as the litellm.Message object
was invisible to the Converse system-message helpers, so a mid-conversation
system stayed between a tool call and its result or reached Converse as
role: system. The helpers now read fields through the shared
message_field and parts_of accessors and drop the local role predicate.
Flagged placement anchored a system run on any assistant follower, but
anthropic_messages_pt drops an assistant turn that puts no block on the
wire (content None, an empty list, an unsigned thinking part), so the
system landed directly before the next user turn, which Anthropic
rejects. Such a run now converts in place. An empty or whitespace text
turn still anchors, since the converter pads it with a placeholder.
* fix(anthropic): treat bridged encrypted reasoning as a vanishing assistant turn for system placement
An assistant turn whose only blocks carry Responses API encrypted reasoning is
dropped by anthropic_messages_pt, so a mid-conversation system run anchored
before it landed directly before the next user turn. The unsignable-thinking
predicate now lives in common_utils and both the factory and the placement
policy consult it.
* fix(anthropic): let an inline thinking part hide separate thinking_blocks in system placement
anthropic_messages_pt skips an assistant turn's separate thinking_blocks as soon
as its content list carries an inline thinking or redacted_thinking part, so a
turn whose inline part is unsigned puts nothing on the wire even when the
separate block is signed. The placement policy now mirrors that rule.
---------
Co-authored-by: Shifat Islam Santo <shifatislamsanto764@gmail.com>
Co-authored-by: ege-arhan <egearhany@gmail.com>
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
* fix(vertex_ai): stop advertising OpenAI platform-only params on Gemma and Llama routes
The Anthropic /v1/messages bridge derives prompt_cache_key from Claude Code's
session id whenever the provider config advertises it, and every Vertex
OpenAI-compatible route (gemma/, openai/<endpoint>, meta/) inherited the full
OpenAI list, so the Model Garden vLLM container rejected each turn with a
pydantic extra_forbidden 400. Vertex's Llama and Gemma configs now filter one
shared list of platform-only params (prompt_cache_key, prompt_cache_retention,
safety_identifier, service_tier, store, web_search_options, modalities,
prediction, audio, max_retries) out of their supported params, so the bridge
no longer derives the key and drop_params drops an explicit one.
* fix(vertex_ai): scope the platform-param filter to self-deployed Model Garden endpoints
---------
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
* fix(fal_ai): price nano-banana-2 and nano-banana-pro image generations by resolution
* fix(fal_ai): bill passthrough submits per requested image and register the resolution price keys
---------
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
* fix(vertex_ai): return chunk content, extractive text, and structData from search_api vector store hits
* fix(vertex_ai): report a chunk hit's relevanceScore as the search result score
* test(vertex_ai): type the search response helper and parametrized case
---------
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
* refactor(ocr): remove the Python OCR execution path and require the Rust route
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fmt
* refactor(ocr): tidy the native OCR passthrough binding
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* style(ocr): ruff format the azure passthrough transformation
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* refactor(ocr): resolve passthrough OCR costing in one Rust call
Replace passthrough_url/passthrough_transform with passthrough_response,
which matches the relayed endpoint against each Azure config's path
segments instead of building a fake request to call get_complete_url.
The binding drops the unused headers, status and api_base arguments.
Catch the ValueError/RuntimeError the binding raises so a relayed body
that is not OCR-shaped falls back to the passthrough object instead of
failing logging, and cover the relay against the real binding.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* fix(ocr): drop the unused LlmProviders import from health check helpers
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* ci: drop the ocr_testing job now that tests/ocr_tests is gone
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* test(ocr): restore the live OCR matrix and the ocr_testing job
The public litellm.ocr / aocr / Router interface is unchanged by the Rust
migration, so the live provider matrix still applies. Drops the stale VCR skip
list for the deleted test_rust_bridge.py.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* test(ocr): import Final in the health check helper tests
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
---------
Co-authored-by: Yujong Lee <yujong@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
* feat(spend): capture-rate check of LiteLLM spend against the OpenAI bill
* fix(spend): claim the alert lock after the check, NaN gauge on no rate, 180-day range cap, live settings, OpenAI adapter under llms
* fix(spend): chart the capture-rate gauge in the all-metrics dashboard and clear it when the check is removed
* fix(prometheus): record the capture-rate gauge when api_provider is an excluded label
---------
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
* fix(vertex_ai): keep batch output_file_id null until Vertex reports outputInfo
Vertex only sets outputInfo.gcsOutputDirectory once a batch job has written
output. Falling back to outputConfig's outputUriPrefix named the per-model
directory shared by every batch of the deployment, an object that never
exists, so the proxy minted a managed file for it under the first key and
every other key's file calls on that id were 403s
* fix(vertex_ai): treat a null gcsOutputDirectory as no output file yet
---------
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
* fix(bedrock): route unmapped openai family model ids to converse
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(bedrock): rename the e2e openai family backend constant
global.openai.gpt-6-sol has a cost-map row now, so the constant no longer
names an unmapped model
---------
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
* fix(logging): pass provider response headers to callbacks on every endpoint
Custom callbacks only received kwargs["response_headers"] for chat
completions. Responses, image generation and edit, speech, and
transcription calls either never recorded the provider's headers or
recorded them in one place and not the other.
Every handler now records the provider's httpx headers on the response's
hidden params as "headers" (raw) and "additional_headers" (processed,
with LiteLLM's own entries winning on a clash), and the logging object
derives model_call_details["response_headers"] from those hidden params
before cost calculation on the non-stream and both streaming success
paths, keeping a handler-set value authoritative. Binary speech responses
expose their hidden params to the standard logging payload, and the sync
OpenAI transcription request always fetches the raw response.
* test(images): point the legacy image and speech fakes at the raw response surface
Image generation now goes through the SDK's raw response so the provider headers can be read, and the speech binary response now carries hidden params. The unit fakes in the image generation, xinference, proxy provider, image edit, Vertex speech, and otel suites still pinned the old call surface and the old "no hidden params" assertion, so they read an uncalled mock or a fake response without headers.
* test(images): drop the rewritten mock comments and the generated edit PNGs
* test(images): move the llm-span test's image fake to the raw response surface
---------
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
* feat(vertex): native batch JSONL passthrough with cost tracking
Add a per-request `passthrough=true` multipart field on `POST /v1/files`
(and the same kwarg on `litellm.create_file`) that uploads a native
Vertex AI batch JSONL to the deployment's GCS bucket unchanged, so rows
using `googleSearch` and other Gemini-only features run as written and
the output, `groundingMetadata` included, comes back untouched.
Passthrough is sticky through the GCS object path
(`litellm-vertex-files/passthrough/...`), so batch create and output
retrieval inherit it without new state. Native output rows are costed
from their `usageMetadata` with the deployment's model and model_info,
in the polling and retrieve paths and for the existing global
`disable_vertex_batch_output_transformation` flag, which billed $0
before.
The proxy requires the target to resolve to vertex_ai deployments only,
refuses `passthrough` with a non-batch purpose, a non-default
`target_storage`, or pre-call guardrails, and validates native rows on
`request` instead of the OpenAI batch keys.
* refactor(vertex): keep native batch row pricing inside the Vertex adapter
Moves native Vertex batch row detection, response parsing, and per-row
pricing from litellm/batches/batch_utils.py into
litellm/llms/vertex_ai/batches/transformation.py, so batch_utils only
aggregates the rows it gets back. Adds tests/test_litellm/files to the
misc unit shard so the new test directory is claimed by a shard.
* fix(files): say what a passthrough batch upload takes when a row is not native
The missing-key 400 listed bare key names, so an OpenAI-shaped row under
passthrough=true read "Each line must be a JSON object with keys request".
The batch line shape now carries its own hint, and the passthrough one says
a passthrough upload takes native Vertex batch rows with a request key
* fix(batches): bill native Vertex embedding batch rows on the native cost path
A native Vertex output row whose response holds an embedding was validated as a
generateContent response, so the documented tokenCount-only shape counted as a failed
row. Price embedding rows from their own usage (promptTokenCount, else tokenCount) with
the helper the transformed embeddings path already used, and drop the prompt-details
helper nothing calls anymore.
* fix(batches): keep modality batch rates on native Vertex embedding rows
An embedding row that carries usageMetadata was billed from promptTokenCount alone, so
its promptTokensDetails no longer reached the audio, image, and video batch rates the
way it did before the native cost path. Run every row with usageMetadata through the
Gemini usage parser and keep the flat tokenCount fallback for embedding rows without it.
* fix(batches): price native Vertex batch rows by modelVersion under a wildcard deployment
A `vertex_ai/*` deployment hands the batch cost path `*` as the deployment model, which
no cost map resolves, so every native (passthrough or flag-on) row was billed at $0. A
wildcard deployment model now defers to the row's own `modelVersion`, the way the
transformed path already prices by the row's `model`.
Also moves the native passthrough tests under tests/test_litellm, the tree codecov
reads, and covers the raw upload chunking, the embedding output translation, the
unpriceable-row path, and the flag-on dispatch.
* fix(batches): keep explicit deployment prices for native Vertex rows without a modelVersion
Under a wildcard deployment a native batch row that carries no modelVersion (an embedding
row, or a generateContent row Vertex returned without one) was billed at $0 even when the
deployment's model_info sets explicit batch prices, because the cost calculator was never
called. The row now falls back to the wildcard name, which the cost calculator prices from
the explicit model_info, and only a row with neither a modelVersion nor a deployment model
is billed at $0 with the warning
---------
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
* fix(ollama): read the JSON thinking field on non-streaming completions
Ollama's /api/generate returns reasoning in a top-level `thinking` field, but
the completion transport only looked for inline <think> tags. reasoning_content
was therefore always null, and a model that spent its whole turn reasoning
returned an empty assistant message with tokens billed.
Port the precedence the ollama_chat transport already uses: the field wins and
inline tags stay the fallback. Applied to both non-streaming paths, including
the JSON-mode text fallback. The two fields are read through a small validated
model rather than off the untyped JSON, so absent and explicitly null
`response` stay distinct exactly as before.
* fix(ollama): keep the thinking field on JSON-mode completions
The first pass read `thinking` for plain replies and for JSON-mode text that
failed to parse, but the three JSON-mode branches that succeed still dropped
it: an empty `response`, a valid JSON object, and a function-call shaped one.
A model that spent its whole turn reasoning under `format: json` therefore
still came back blank with the tokens billed.
Carry the field on all three, type the new test helper's parameters, and cover
the null and malformed `response` fallbacks.
---------
Co-authored-by: Pawan-Shahane <shahanepawan511@gmail.com>
* feat(bedrock): serve the OpenAI models on bedrock-runtime's native Responses API
AWS serves the OpenAI models on bedrock-runtime through an OpenAI-compatible
surface at /openai/v1/responses, alongside Converse. LiteLLM had no Responses
config for the bedrock provider, so /v1/responses fell back to the Chat
Completions bridge and was translated into Converse. A realistic Codex session
does not survive that translation: its function_call / function_call_output
history becomes Converse toolUse / toolResult blocks with no toolConfig, and
Converse rejects the request outright.
Add a Responses config for that surface, opted into per model from the price-map
supported_endpoints so models without the signal keep the bridge exactly as
before. Auth is Bearer when a Bedrock API key is present, SigV4 otherwise.
Both Bedrock endpoints reject the Codex history item types agent_message,
context_compaction and local_shell_call, so the normalization bedrock_mantle
carried privately moves into a shared module and both providers use it. They are
history items, so they only bite from the second turn onward -- a first-turn
smoke test passes and hides the problem. Verified against bedrock-runtime with
global.openai.gpt-5.6-sol: additional_tools is accepted there (unlike on
bedrock-mantle) while those three types are rejected, so the two endpoints do
not share one validator and each provider opts in explicitly.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(bedrock): build the Responses endpoint from the region's partition suffix
get_complete_url hardcoded amazonaws.com in an f-string, so every non-commercial
partition got the wrong host: cn-north-1 resolved to amazonaws.com instead of
amazonaws.com.cn, and GovCloud/ISO regions were wrong the same way. Defer to
BaseAWSLLM._select_default_endpoint_url, which this config already inherits and
which resolves the suffix per partition.
test_no_fstring_hardcodes_the_commercial_dns_suffix scans the whole tree, so it
caught this even though it is not one of this PR's test files. Register the
config in ENDPOINT_BUILDERS so the cn/GovCloud endpoint sweep covers this
surface from now on rather than only the f-string guard.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* feat(bedrock): opt the gpt-6 family into the native Responses API
* fix(bedrock): drop the Responses tool types bedrock-runtime rejects
Codex sends a web_search tool on every turn. api.openai.com runs that tool
itself, and the Converse bridge dropped it silently, but bedrock-runtime's
native Responses endpoint rejects the whole request with 400 "web search is
not supported for this request". Filter the request's tools down to the
types bedrock-runtime's own validation error names, logging what was dropped,
through a helper shared with the Mantle route, which already did the same.
* fix(bedrock): emulate file_search and collapse custom Responses paths
* fix(bedrock): keep background and remote image inputs working on the native Responses route
* fix(bedrock): inline remote images inside tool outputs on the native Responses route
* fix(bedrock): inline remote computer screenshots on the native Responses route
---------
Co-authored-by: Leonardo Freitas dos Santos <leonardo.freitas.s@outlook.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
The tool_result user message built by the /v1/messages MCP loop used tuple
content, which the Messages to Chat Completions adapter silently dropped, so
non-Anthropic models re-requested the tool until the iteration cap or the
provider rejected the follow-up. Emit list content so the existing tool_result
branch translates it into a role tool message keyed by tool_call_id.
Resolves LIT-8474
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: bot_apk <apk@cognition.ai>
* ci: add dashboard and core smoke checks across supported Python versions
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* ci: tighten merge smoke harness and keep mapped test diffs additive
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* ci: terminate proxy on readiness timeout and use contextlib.suppress in teardown
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: yuneng <yuneng@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(bedrock): send json_schema as a forced tool on Claude Opus 4.7 and 4.8 Converse
Bedrock rejects outputConfig.textFormat on Opus 4.7 and 4.8 with
"output_config.format: Extra inputs are not permitted", and the AWS
model cards list structured outputs as not supported for both, so
their cost-map entries no longer claim supports_native_structured_output
and json_schema requests fall back to the json_tool_call tool.
Fixes#27846
* test(bedrock): assert Opus 4.7 and 4.8 inline the schema on Invoke, move the native case to Sonnet 4.6
---------
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
* fix(bedrock): stream /v1/messages Invoke bytes through instead of holding them in a 1024-byte chunker
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* style(bedrock): apply ruff format to invoke messages stream passthrough
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* style(bedrock): drop drive-by reformat of existing invoke messages tests
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(bedrock): collect streamed chunks into a tuple in passthrough regression test
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(bedrock): give the passthrough regression test a 10s first-chunk budget
* test(bedrock): type the eventstream frame helper's payload as Mapping[str, object]
* test(bedrock): take the gated byte stream's chunks as an immutable Sequence
---------
Co-authored-by: mateo <mateo@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
* fix: repair seven regressions caught by CircleCI on main
- vertex_ai: stop treating fine-tuned endpoint ids (numeric or
vertex_ai/gemini/<id>) and gemma models as Gemini 3+, which injected
temperature=1.0 and Gemini 3 thinking config into their requests (#42465)
- cost: price Azure DALL-E 3 from its azure/<quality>/<size>/dall-e-3 rows;
it only worked through the OpenAI rows that #42435 removed
- bedrock: stream bedrock/invoke/moonshot through an OpenAI-shaped chunk
decoder; the generic decoder dropped every chunk, which the
supports_response_schema flag from #42338 un-skipped in CI
- proxy: keep the public model_group on pre-routing rejections so the Usage
page groups them under the model name, not the deployment (#41077)
- cost map: mirror the base rows' capability flags onto Bedrock regional and
cross-region copies (#42254 and later syncs)
- whitelist the new regional Bedrock rows from #42543 and #42588 for the
converse routing check, following the existing regional-row convention
* fix(model-prices): mirror capability flags onto ap-southeast-3 bedrock rows
* refactor(bedrock): tighten types on the moonshot stream decoder and its tests
* test(e2e): pin bedrock batch create with blank S3 env vars
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(bedrock): treat blank S3 env vars as unset for batch jobs
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(e2e): trim blank S3 env gateway config
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(e2e): register blank_s3_env capability and clean gateway tempdir
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(e2e): move blank S3 env batch test to its own module
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: yucheng <yucheng@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
The two redaction tests raised the deprecated InvalidStatusCode, which the
websockets 15 asyncio client never raises, and asserted the raw 403 close
code that the handshake refusal path replaced with 1008. The refusal path
builds its close reason from the status code alone, so there is no secret
to redact there, and the handshake refusal tests already cover the error
event and the 1008 close.
Those refusal tests only passed when run after a sibling test had imported
websockets.asyncio.client, since websockets lazy-loads its exceptions
submodule. Importing InvalidStatus, Response, and Headers from their own
submodules makes them pass in any order.
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
* fix(ollama): send PNG and JPEG images without requiring Pillow
The ollama/ completion transport imported Pillow before it looked at the image, so every image request failed with a 500 on installs without Pillow. That includes the Docker image, where Pillow is only a CI dependency
Detect PNG and JPEG from their leading bytes and pass them through untouched. Pillow is now imported only when another format has to be re-encoded as JPEG, and that case still raises the same install hint
* fix(ollama): address Greptile findings on image conversion
Catch all exceptions on Pillow import, not just ImportError, so the helpful
install hint always appears. Break a line that exceeded 120 characters
* fix(anthropic): skip non-dict content items in beta-header and file-id helpers so malformed content lists return 400 instead of 500
Fixes#42094
Supersedes #42101
Co-authored-by: Pawan-Shahane <shahanepawan511@gmail.com>
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(e2e): spawn the DB-less regression proxy with -P so the cwd cannot shadow the pinned checkout
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(e2e): launch the DB-less proxy via -I -c with an explicit sys.path so python 3.10 works, drop DIRECT_URL, remove restating docstrings
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(e2e): gate the self-booted DB-less proxy behind the owned_gateway opt-in the Buildkite container cannot satisfy
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(anthropic): move the bare string content item repro to tests/integration
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* style(tests): wrap the anthropic bare string wire test to the 120 column limit
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* style(tests): wrap anthropic common_utils test literals to the 120 column limit
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* style: fix ruff findings in touched test files
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: kerry <kerry@berri.ai>
Co-authored-by: Pawan-Shahane <shahanepawan511@gmail.com>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* chore(cost-map): remove models past their deprecation date
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(cost-calc): drop the empty parametrize left behind by the gemini web search removal
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(cost-map): drop merge base block left by conflict resolution
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(cost-calc): drop gemini image cost tests pinned on removed model
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: kerry <kerry@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(fal_ai): honour global api_base for image generation and reject non-string reasoning_effort with 400
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(fal_ai): add return annotations to reasoning_effort regression tests
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(fal_ai): move the LIT-8340 repro from tests/e2e to tests/integration
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: kerry <kerry@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(fal_ai): reuse the status client and headers on the video result probe
The Fal result GET issued after a COMPLETED status poll built its own default
client and only carried Authorization and Content-Type, so an injected client,
a request-level ssl_verify and extra_headers were honored on the status GET but
not on the result GET, and a transport failure on that probe escaped as a 500.
The handler now hands the selected sync or async client to the provider status
transform, Fal reuses it with the full validated header set, and a probe
transport error stays non-terminal like the existing 429 and 5xx handling
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(fal_ai): mark the result probe header dict as mutable-ok for the type discipline gate
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(fal_ai): move the result probe repro to tests/integration
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(e2e): restore the shared e2e helpers to the merge base
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: kerry <kerry@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(fal_ai): align /fal_ai queue gate with the pricer and normalise resolution type
The /fal_ai gate admitted catalog keys the Fal pricer cannot price, so
those jobs were forwarded and logged at 0.0 spend. The gate now reuses
the pricer as its eligibility predicate. Resolution is normalised to a
string before the keyed price lookup so int and str spellings bill the
same.
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(fal_ai): assert passthrough pricing invariants on synthetic catalog entries
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(fal_ai): price the submitted body in the /fal_ai queue gate so keyed-only entries are admitted
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(fal_ai): move the queue gate repro to tests/integration
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: kerry <kerry@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(langsmith): json.dumps with default=str so non-serializable metadata does not crash batch flush
Serialize the runs/batch payload with json.dumps(default=str, allow_nan=False) and send it as content= with an explicit Content-Type, so datetime, Decimal and similar metadata values no longer raise TypeError and drop the batch. Forward content= on the AsyncHTTPHandler retry path so a retried batch re-sends the identical body
Replaces #39133, which was cut from the retired staging branch and conflicts with main
Co-authored-by: Damien Smrt <dsmrt@users.noreply.github.com>
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(langsmith): drop test docstrings and replace monkeypatch with a client-injecting handler
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(langsmith): add live e2e for non-native metadata reaching LangSmith
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(langsmith): scope the e2e docstring to the values the test injects
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(http_handler): close injected retry clients
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(e2e): deselect the LangSmith live e2e on the stage-mirror stack
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: yucheng <yucheng@berri.ai>
Co-authored-by: Damien Smrt <dsmrt@users.noreply.github.com>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(fal_ai): e2e for minimax h3 auto duration and oversized size
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(fal_ai): keep auto duration literal and make h3 tier lookup total
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(fal_ai): omit duration for h3 when seconds is auto
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(fal_ai): move the minimax h3 auto duration and oversized size repro to tests/integration
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: kerry <kerry@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix image cost: honor deployment pricing
* fix types: coerce fal deployment price, drop private import
* fix: forward every custom pricing field through get_litellm_params
* test: assert optional keys are absent, not merely None, in get_litellm_params
* test: type the deployment image pricing test parameters
* fix: bill deployment per-image and per-pixel prices on unlisted image models
* test: type the remaining image cost test parameters
---------
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
* fix(realtime): surface an upstream handshake refusal as an error event and policy close
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(realtime): tidy the handshake refusal e2e
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(realtime): keep upstream exception text out of the Azure client error
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* refactor(realtime): map handshake refusal close codes with a lookup
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(bedrock/claude_platform): strip body params the AWS endpoint rejects
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(bedrock/claude_platform): assert exact bodies through a strict fake gateway for every workspace alias and auth mode
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(proxy): block Claude Platform workspace id aliases in request bodies without admin opt-in
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: Venkata Donavalli <vdonavalli@live.com>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: shrey kharbanda <shreshth@berri.ai>
* fix(anthropic_adapter): keep reasoning_effort a string for targets that stay on chat completions
* fix(anthropic_adapter): judge the summary bridge with the deployment's api_base
The adapter's bridge check now resolves the provider and base the same
way completion() does, passing the deployment's api_base and api_key
into get_llm_provider and the resolved base into the bridge check, so a
Foundry deployment lands on the same route in both places and a bare
model name routed by its api_base still gets the plain tier.
A litellm_proxy target keeps the dict, since the upstream gateway makes
its own bridge decision and needs the summary to make it.
* refactor(anthropic_adapter): return the plain effort instead of writing it inside the helper
---------
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
* fix(responses): drop client_metadata before bridging to chat completions
Codex CLI sends client_metadata on every /v1/responses call. For a
provider with no native Responses config the chat-completions bridge
forwarded the raw kwargs, so client_metadata reached the provider as a
chat body field and Databricks rejected the request with an unknown
field 400. The bridge now drops the Responses-only request fields
before calling completion while still passing every other kwarg
through, so deployment-level params such as chat_template_kwargs keep
reaching providers without a native config.
* fix(databricks): merge consecutive system messages for chat-template models
Codex sends instructions plus a leading developer item, which the Responses
bridge and the developer-to-system translation turn into two consecutive
system messages that Databricks chat-template models reject with "System
message must be at the beginning". Each run of consecutive system messages
is now merged into one before the request is built for non-Claude models.
Also keep client_metadata out of the bridged chat request even when
allowed_openai_params names it, so both bridge branches drop the same set.
* fix(databricks): skip empty system messages when merging consecutive ones
Databricks drops empty content before the merge, so a system message in a
run could carry no content key and the merge iterated None. Those messages
are now skipped; a run with no content at all keeps its first message.
---------
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
* refactor(agentic-loop): build follow-up kwargs in one place so no executor can repeat a request param
The Responses and both chat completions follow-up executors each rebuilt the follow-up kwargs by hand and then expanded them next to the request params, so a plan whose kwargs repeated a request param raised a duplicate keyword TypeError. They now share build_agentic_followup_kwargs, which drops any key already sent as a request param (and the explicitly passed model/input/messages) from both the request kwargs and the plan kwargs. Each executor keeps its own internal-key filter unchanged, and the /v1/messages executor is untouched because it merges into a single dict and cannot hit this.
* test(agentic-loop): move follow-up regressions into their mapped test files
Greptile review: the executor regressions belong in test_llm_http_handler.py and test_chat_completion_agentic_loop.py rather than a split-off file, and the builder test helper returned a read-only mapping while promising a dict. The Responses overlap test is dropped because #41560 already added the same one to the mapped file.
---------
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
* fix(proxy): release unclaimed budget reservations at request end
* fix(proxy): release unclaimed budget reservations of websocket sessions too
* test(proxy): drop the structural middleware inheritance check
* fix(proxy): claim the budget reservation on streaming pass-through before its cost callback
The SSE chunk processor hands its success handler to the logging worker
after the response, so the request-end release freed the reservation
first and left the key unguarded until the worker drained. Claim it at
both end-of-stream hand-offs, the immediate enqueue and the coroutine
parked for deferred dispatch.
Give the xai realtime test double the litellm_params attribute every
real Logging object carries, since the wrapper now reads it.
* test(pass-through): give the vertex streaming test doubles a litellm_params dict
The spec'd Logging mocks in test_vertex_ai_anthropic_streaming_cost_injection.py
lacked the instance attribute the chunk processor now reads to claim the budget
reservation. Also restores main's _lazy_openapi_snapshot.json: the branch's copy
had been regenerated under Python 3.14, which dedents one docstring description
that the CI regeneration on Python 3.12 keeps indented, and the PR adds no lazily
loaded route, so main's file is the correct one.
* fix(pass-through): claim the budget reservation only after its cost callback is enqueued
Every pass-through success hand-off stamped callback_bound before handing the
coroutine to the logging worker. When that enqueue raised, the reservation stayed
claimed with no callback left to reconcile it, so the request-end release skipped it
and the reserved cost stayed pinned on the key's counter. Enqueue first, then claim,
so a failed hand-off leaves the reservation for the request-end release.
---------
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
* fix(fal_ai): price non-canonical image sizes from the nearest row and honour dump options
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(fal_ai): drop monkeypatched mixed pricing case
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* refactor(fal_ai): use the default dimensions constant directly
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(fal_ai): forward nested include and exclude when dumping image data
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(fal_ai): honour pydantic item selectors in image data serializer
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(fal_ai): match negative item selectors in image data serializer
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: kerry <kerry@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* feat(fal_ai): add flux-lora-depth image edits and moondream3 chat completions
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* chore(fal_ai): retrigger codecov processing
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(fal_ai): reject multi-turn and system messages for moondream3 chat
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(fal_ai): return 400 for invalid moondream3 chat requests
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(fal_ai): reject moondream3 responses missing output or usage
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(fal_ai): reject streaming moondream3 requests before dispatch
stream never reaches optional_params, so the transform_request check could not fire; reject in _complete_fal_ai on ctx.stream
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: kerry <kerry@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(bedrock): sign batch S3 requests with s3_access_key_id and s3_secret_access_key
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(bedrock): keep S3 signer test additions scoped to new cases
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(bedrock): drop e2e suite changes from the S3 signing fix
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(bedrock): build S3 credentials directly from the s3_* pair so ambient AWS_* env never mixes in
Restores the split-identity e2e coverage
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: yucheng <yucheng@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
