The gemini-live-2.5-flash-preview-native-audio-09-2025 model only works
with WebSocket (Live API), not REST endpoints. Changed supported_endpoints
from /v1/chat/completions to /vertex_ai/live to reflect the actual
passthrough endpoint available in LiteLLM proxy.
Match the error handling pattern used in the Anthropic count_tokens
endpoint: catch ProxyException separately to surface its status code
and message, and include error details in the generic 500 fallback.
The user-specified async client was being overwritten by
`litellm.module_level_aclient` in `streaming_handler.py` when using
async+streaming with Gemini.
This fix adds a `gemini_client` parameter to `make_call()` (matching
the existing pattern in `make_sync_call()`) so the user's custom client
is preserved and not overwritten.
Fixes#17148
- Log provider token counting failures instead of silently swallowing
- Fall back to local tokenizer when provider returns error response
- Map assistant tool_calls to Responses API function_call items
- Concatenate multiple system messages instead of overwriting
- Hide internal error details from proxy API responses
- Narrow exception catch in handler to network/JSON errors only
- Update test to match new fallback behavior
- Add OpenAITokenCounter using POST /v1/responses/input_tokens endpoint
- Add litellm.acount_tokens() public async API that auto-routes to provider APIs
- Add proxy endpoint POST /v1/responses/input_tokens for OpenAI-compatible counting
- Transform chat tools format to Responses API format for correct token counting
- Fall back to local tiktoken when provider API unavailable
Fixes#22302
- Remove token field from JWTKeyMappingResponse to prevent hashed key exposure
- Use _to_response() helper on all CRUD endpoints to control returned fields
- Return 409 for unique constraint violations, 400 for FK violations, 404 for not found
- Add response_model to endpoint decorators
- Add 8 new unit tests covering error handling and token redaction
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Kontext models (flux-kontext-pro, flux-kontext-max) support both
text-to-image and image editing. Add them to IMAGE_GENERATION_MODELS
and update supported_endpoints in model prices JSON.
- Change mode from "image_generation" to "image_edit" for all 4 BFL
image edit models (flux-kontext-pro, flux-kontext-max, flux-pro-1.0-fill,
flux-pro-1.0-expand)
- Rename shadowed api_base variable to complete_url in async handler
for consistency with sync path
- Remove response_format from supported params (BFL always returns URLs)
- Remove n and size from image edit supported params (not mapped)
- Raise ValueError on unknown model names instead of silently defaulting
- Create handler.py for image generation and image edit
- Move polling logic from transformation to handlers
- Handlers use _get_httpx_client() / get_async_httpx_client()
- Transformation files now only transform request/response data
- Follows Bedrock pattern for provider-specific handlers
Addresses feedback: transformation files should not make HTTP requests
Add native text-to-image generation for Black Forest Labs Flux models
(flux-pro-1.1, flux-pro-1.1-ultra, flux-dev, flux-pro).
- Polling-based async API with sync and async support
- OpenAI-compatible parameter mapping (size, n, quality)
- Reuses shared HTTP clients via _get_httpx_client()
- 39 unit tests added
Replace direct httpx.get() calls with _get_httpx_client() to reuse
cached HTTP client, following the pattern used by other providers
(RunwayML, Azure AI OCR, Sagemaker, etc.).
Add BFL models to model_prices_and_context_window.json with pricing:
- flux-kontext-pro: $0.04/image
- flux-kontext-max: $0.08/image
- flux-pro-1.0-fill: $0.05/image
- flux-pro-1.0-expand: $0.05/image
Add black_forest_labs_models set to __init__.py for model discovery.
Add native integration for Black Forest Labs image editing models
(flux-kontext-pro, flux-kontext-max, flux-pro-1.0-fill, flux-pro-1.0-expand).
Changes:
- Add BlackForestLabsImageEditConfig for BFL API transformation
- Add BLACK_FOREST_LABS to LlmProviders enum
- Add use_multipart_form_data() to BaseImageEditConfig for JSON vs form-data
- Modify image_edit_handler to support JSON request bodies
- Add comprehensive unit tests
Closes#11401
Providers like Cerebras return delta.reasoning in streaming responses
for gpt-oss models, but LiteLLM's Delta class expects reasoning_content.
This causes reasoning content to be silently dropped during streaming.
Fixes#13300
The AllModelsTab component was passing onRowClick to AllModelsDataTable
but the prop was not defined in the interface, causing the TypeScript
build to fail.
Add global media_resolution support for Gemini 2.x models (2.0, 2.5) when
using OpenAI's detail parameter on images. Previously, the detail parameter
was only working for Gemini 3+ models (per-part) and was silently ignored
for older Gemini models.
- Add _get_highest_media_resolution() and _extract_max_media_resolution_from_messages()
to extract highest detail from all images/files in a request
- Update _transform_request_body() to add mediaResolution to generationConfig
for Gemini 2.x models only (not 1.x which doesn't support it, not 3+ which
uses per-part)
- Add mediaResolution field to GenerationConfig TypedDict
- Support detail extraction from both image_url and file content types
- Add comprehensive unit tests and update documentation
