Wildcard openrouter/* deployments pass custom_llm_provider=openrouter with
the full openrouter/provider/model id; OpenRouter expects provider/model.
Strip the outer openrouter/ only when the remainder contains a slash so
native ids like openrouter/auto stay intact.
Adds regression test for proxy wildcard path.
Made-with: Cursor
- Cache LITELLM_ENABLE_TEAM_STALE_ALIAS_BYPASS at module level to avoid hot-path secret lookups
- Add clarifying comments for should_include_deployment team isolation logic
- Add negative assertion for update_team.assert_not_called() in test
- Add docstring clarification for _get_team_deployments helper pattern
- Add explicit assertion message in test_get_model_list_alias_optimization
Made-with: Cursor
- Use O(1) team index lookup instead of map_team_model in alias guard
- Fix MockPrismaClient to validate where clause filters
- Add comment explaining DB query trade-off for team deployments
Made-with: Cursor
- Skip model_aliases rewrite if model resolves to team deployments
- Add test coverage for sibling-preservation branch
- Update MockPrismaClient to support sibling deployment scenarios
Made-with: Cursor
- Add clarifying comments to test assertions
- Query prisma DB instead of in-memory router to avoid stale state
- Prevents incorrect deletion of old public name when siblings exist
Made-with: Cursor
Remove team model_alias rewrites and resolve team deployments by team_public_model_name with team_id so sibling deployments stay in the routing candidate pool, with explicit logs showing candidate selection before load balancing.
Made-with: Cursor
Use a deterministic internal model_name for team-scoped deployments so sibling deployments with the same public model share a routing group. This makes team alias writes idempotent and preserves multi-deployment failover/load balancing behavior.
Made-with: Cursor
- Add cancel/retrieve overrides in AzureOpenAIFineTuningAPI to normalize responses
- Expand _AZURE_STATUS_MAP to handle all known Azure statuses
- Add "pending" to OpenAIFileObject.status allowed values
- Fix async test mock to return awaitable LiteLLMFineTuningJob
- Add test_openai_file_object_accepts_pending_status
Made-with: Cursor
- Move trainingType injection to AzureOpenAIFineTuningAPI handler
- Guard normalization with is_azure flag to only apply to Azure responses
- Override acreate_fine_tuning_job in Azure handler to use is_azure=True
- Update test to directly test _ensure_training_type method
- Add test for OpenAI unchanged behavior
Made-with: Cursor
- Default trainingType=1 for Azure when omitted to avoid misleading "base model does not support fine-tuning" error
- Normalize Azure FineTuningJob responses (pending→queued, null fields→defaults) to match OpenAI schema
- Add pending status support to OpenAIFileObject for Azure file uploads
- Add test coverage for trainingType default and response normalization
Made-with: Cursor
Addresses Greptile review feedback: replace direct litellm.model_cost
lookup with the standard _supports_factory infrastructure used by
supports_reasoning, supports_native_streaming, etc.
- Add supports_native_structured_output() utility in litellm/utils.py
- Add supports_native_structured_output field to ModelInfoBase type
- Wire field into _get_model_info_helper return dict
- Delegate from Bedrock _supports_native_structured_outputs to utility
- Add field to JSON schema validator in test_utils.py
Wrap cost-map-dependent tests in try/finally to restore
os.environ["LITELLM_LOCAL_MODEL_COST_MAP"] and litellm.model_cost,
preventing test-ordering sensitivity.
Integration testing confirmed gemma-3 (4b/12b/27b) ignores the JSON
schema and returns free text, and nemotron-nano (9b/12b) errors with
"Tool calling is not supported in streaming mode" even on sync calls.
Remove the flag so these models fall back to the tool-call approach.
Also fix test assertions to match (nemotron-nano-3-30b is supported,
gemma-3 and nemotron-nano-12b are not).
Move the source of truth for which Bedrock models support native structured
outputs (outputConfig.textFormat) from a hardcoded substring set
(BEDROCK_NATIVE_STRUCTURED_OUTPUT_MODELS) to the cost JSON via a new
"supports_native_structured_output" flag. This makes it possible to add
support for new models (including Claude Sonnet 4.6, which was missing)
by updating the JSON alone, with no code changes needed.
Instead of returning a 400 error when return_to is passed without
control_plane_url configured, silently ignore it and proceed with
the normal same-origin SSO flow.
- Update test_anthropic_via_responses_api expected_events to include
CONTENT_PART_ADDED between OUTPUT_ITEM_ADDED and OUTPUT_TEXT_DELTA
- Add TestEnsureOutputItemContentPartAdded with 3 mock tests:
message item emits content_part.added, reasoning item does not,
and the event is only emitted once
Verify that spend_logs_metadata is correctly merged into combined_metadata
and flows through to Prometheus custom labels. Tests cover: basic extraction,
precedence when keys overlap, all three metadata sources combined, and None
handling.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Thread project_alias alongside project_id through the metadata pipeline so
callbacks receive the human-readable project name. DRY up duplicate metadata
dict construction in proxy_track_cost_callback and pass_through_endpoints by
reusing get_sanitized_user_information_from_key — future metadata fields only
need adding in one place.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Two independent bugs prevented post-call OpenAI Moderation guardrail
results from reaching downstream logging callbacks (Langfuse, Datadog).
Bug 1: process_output_response() created a throwaway request_data dict,
so guardrail info written by @log_guardrail_information was discarded.
Fixed by threading the real request_data from the unified guardrail
dispatcher through all 13 BaseTranslation handlers, with litellm_metadata
injection preserved for third-party guardrails (Zscaler, Prompt Security).
Also extended to process_output_streaming_response for consistency.
Bug 2: The @log_guardrail_information decorator collapsed the full
moderation API response (categories, scores, flagged status) to "allow".
Fixed by overriding _process_response/_process_error on
OpenAIModerationGuardrail to stash and log the full response, following
the established Model Armor pattern.
Both bridges (Responses→CC and CC→Responses) independently encoded the
same field mapping knowledge. This extracts 4 shared mappings into a
single module so future changes only need to happen in one place.
Shared mappings:
- status ↔ finish_reason bidirectional dicts and functions
- response_format ↔ text.format paired conversion functions
- provider_specific_fields normalization helper
- usage field name translation (input_tokens ↔ prompt_tokens, etc.)
No behavioral changes — bridge methods now delegate to the shared module.
The `dimensions` parameter was correctly mapped to `outputDimensionality`
in `optional_params` but never placed in the request body. The Vertex AI
predict endpoint expects it under a `parameters` field.
Add `parameters` dict to `VertexMultimodalEmbeddingRequest` TypedDict and
populate it from `optional_params` in `transform_embedding_request`.
Fixes#24392
Gemini API returns a DOCUMENT modality in promptTokensDetails for PDF
inputs, but the token parser only handled TEXT, IMAGE, AUDIO, and VIDEO.
DOCUMENT tokens were silently dropped, causing cost to be undercounted
by up to 99% for PDF-heavy requests.
Map DOCUMENT tokens to text_tokens since Gemini bills documents at the
text token rate. Applied to all four modality parser loops:
promptTokensDetails, cacheTokensDetails, responseTokensDetails, and
candidatesTokensDetails.
Fixes#24375
Replace _is_gemini_3_model() substring check with a
web_search_billing_unit field in model_prices JSON:
- "per_query": each search query billed individually (Gemini 3.x)
- "per_prompt" (default): flat fee per grounded API call (Gemini 2.x)
Add web_search_billing_unit to 23 Gemini 3.x model entries.
Update docs and tests accordingly.
Add tests for the gpt-5.1/5.2/5.4 reasoning.effort interaction:
- gpt-5.1 with no reasoning allows flexible temperature
- gpt-5.1 with effort='high' drops temperature
- gpt-5.4 with effort='none' allows flexible temperature
- Gemini 2.x charges per grounded prompt (flat $0.035), clamped to 1
regardless of internal query count
- Gemini 3.x charges per search query ($0.014 each)
- Extract web_search_requests from groundingMetadata in non-streaming
responses (parity with streaming path)
- Add search_context_cost_per_query to vertex_ai and base Gemini entries
- Move tests to tests/test_litellm/ (CI directory)
