- Log provider token counting failures instead of silently swallowing
- Fall back to local tokenizer when provider returns error response
- Map assistant tool_calls to Responses API function_call items
- Concatenate multiple system messages instead of overwriting
- Hide internal error details from proxy API responses
- Narrow exception catch in handler to network/JSON errors only
- Update test to match new fallback behavior
- Add OpenAITokenCounter using POST /v1/responses/input_tokens endpoint
- Add litellm.acount_tokens() public async API that auto-routes to provider APIs
- Add proxy endpoint POST /v1/responses/input_tokens for OpenAI-compatible counting
- Transform chat tools format to Responses API format for correct token counting
- Fall back to local tiktoken when provider API unavailable
Fixes#22302
- Remove token field from JWTKeyMappingResponse to prevent hashed key exposure
- Use _to_response() helper on all CRUD endpoints to control returned fields
- Return 409 for unique constraint violations, 400 for FK violations, 404 for not found
- Add response_model to endpoint decorators
- Add 8 new unit tests covering error handling and token redaction
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Kontext models (flux-kontext-pro, flux-kontext-max) support both
text-to-image and image editing. Add them to IMAGE_GENERATION_MODELS
and update supported_endpoints in model prices JSON.
- Change mode from "image_generation" to "image_edit" for all 4 BFL
image edit models (flux-kontext-pro, flux-kontext-max, flux-pro-1.0-fill,
flux-pro-1.0-expand)
- Rename shadowed api_base variable to complete_url in async handler
for consistency with sync path
- Remove response_format from supported params (BFL always returns URLs)
- Remove n and size from image edit supported params (not mapped)
- Raise ValueError on unknown model names instead of silently defaulting
- Create handler.py for image generation and image edit
- Move polling logic from transformation to handlers
- Handlers use _get_httpx_client() / get_async_httpx_client()
- Transformation files now only transform request/response data
- Follows Bedrock pattern for provider-specific handlers
Addresses feedback: transformation files should not make HTTP requests
Add native text-to-image generation for Black Forest Labs Flux models
(flux-pro-1.1, flux-pro-1.1-ultra, flux-dev, flux-pro).
- Polling-based async API with sync and async support
- OpenAI-compatible parameter mapping (size, n, quality)
- Reuses shared HTTP clients via _get_httpx_client()
- 39 unit tests added
Replace direct httpx.get() calls with _get_httpx_client() to reuse
cached HTTP client, following the pattern used by other providers
(RunwayML, Azure AI OCR, Sagemaker, etc.).
Add BFL models to model_prices_and_context_window.json with pricing:
- flux-kontext-pro: $0.04/image
- flux-kontext-max: $0.08/image
- flux-pro-1.0-fill: $0.05/image
- flux-pro-1.0-expand: $0.05/image
Add black_forest_labs_models set to __init__.py for model discovery.
Add native integration for Black Forest Labs image editing models
(flux-kontext-pro, flux-kontext-max, flux-pro-1.0-fill, flux-pro-1.0-expand).
Changes:
- Add BlackForestLabsImageEditConfig for BFL API transformation
- Add BLACK_FOREST_LABS to LlmProviders enum
- Add use_multipart_form_data() to BaseImageEditConfig for JSON vs form-data
- Modify image_edit_handler to support JSON request bodies
- Add comprehensive unit tests
Closes#11401
Providers like Cerebras return delta.reasoning in streaming responses
for gpt-oss models, but LiteLLM's Delta class expects reasoning_content.
This causes reasoning content to be silently dropped during streaming.
Fixes#13300
The AllModelsTab component was passing onRowClick to AllModelsDataTable
but the prop was not defined in the interface, causing the TypeScript
build to fail.
Add global media_resolution support for Gemini 2.x models (2.0, 2.5) when
using OpenAI's detail parameter on images. Previously, the detail parameter
was only working for Gemini 3+ models (per-part) and was silently ignored
for older Gemini models.
- Add _get_highest_media_resolution() and _extract_max_media_resolution_from_messages()
to extract highest detail from all images/files in a request
- Update _transform_request_body() to add mediaResolution to generationConfig
for Gemini 2.x models only (not 1.x which doesn't support it, not 3+ which
uses per-part)
- Add mediaResolution field to GenerationConfig TypedDict
- Support detail extraction from both image_url and file content types
- Add comprehensive unit tests and update documentation
Add Mistral to the supported providers list in audio_transcription.md
and add Audio Transcription section to the Mistral provider page with
SDK usage, optional params, diarize support, and proxy configuration.
Add MistralAudioTranscriptionConfig for Mistral's /v1/audio/transcriptions
endpoint, enabling litellm.transcription() with mistral/voxtral-mini-latest
and other Voxtral models. Supports multipart form-data with OpenAI-compatible
params (language, temperature, response_format, timestamp_granularities)
plus Mistral-specific params like diarize.
* azure content enhancement...
* rafactored to increase confidence score
* improvements based on additional feedback
* removed unused import
* Force-split any word longer than max length allowed
* preserve whitespace in text splitting
* moving common initialization to base class
* consolidate enforcement into async_make_request as single point, remove redundant caller-side checks, extract shared init/HTTP logic into base, and fix stale log messages
* clean up
* clean up tests
1. Okta SSO docs (admin_ui_sso.md):
- Rewrite Step 3 to document both Org Auth Server (free) and
Custom Auth Server (paid SKU) as tabbed options
- Add Step 4 for GENERIC_CLIENT_STATE and PKCE configuration
(moved from troubleshooting into the main guide)
- Clarify no_matching_policy error only applies to Custom Auth Server
- Deduplicate troubleshooting section to reference Step 4
2. Custom SSO handler (custom_sso.py + custom_sso.md):
- Replace broken user_info() call with prisma_client.get_data()
- user_info() is a FastAPI route handler requiring Request and
UserAPIKeyAuth params, cannot be called directly
- Keep new_user/add_new_member as commented-out import references
in docs for customers who need them
