- Add new SpendLogsMetadata keys to ignored_keys in spend logs tests
(regression from ccecc10c82 which intentionally includes all keys)
- Mock PrismaManager.setup_database and should_update_prisma_schema in
proxy CLI tests to prevent real DB migrations from running in CI
- Use CliRunner(mix_stderr=False) to fix Click stream lifecycle issues
- Use unique UUID suffix for Redis TTL test keys to avoid stale state
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
The previous approach tried to inject litellm_credential_name as a tag
in get_standard_logging_object_payload, but the credential name was never
available in litellm_params because the Logging object is created by the
proxy BEFORE the Router selects a deployment. The credential name only
exists in the deployment's litellm_params, which is resolved later.
This fix injects the credential name as a tag in
Router._update_kwargs_with_deployment(), right alongside the existing
deployment-level tags mechanism. This ensures the credential name flows
through the normal metadata.tags pipeline.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- RC1: Override _remove_key() in LLMClientCache to schedule aclose() on
evicted async clients instead of relying on GC
- RC2: Use passed connection_pool for URL configs instead of creating an
orphaned pool via from_url()
- RC3: Pass max_connections through to BlockingConnectionPool.from_url()
for URL configs, with input validation for invalid values
- RC5: Close sync redis_client in disconnect() with try/except guard
When a model has a litellm_credential_name, append it to request_tags during
logging so it flows into DailyTagSpend and becomes filterable in the Usage page.
- Add litellm_credential_name to _OPTIONAL_KWARGS_KEYS so it survives into
litellm_params during get_litellm_params() filtering
- Read credential name from litellm_params in get_standard_logging_object_payload()
and append to request_tags if not already present
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix(responses): fix O(n²) CPU overhead in reasoning streaming path
stream_chunk_builder was called on every reasoning chunk, rebuilding the
entire response from all collected chunks each time. Replace with
incremental accumulation of reasoning_content parts, only joining at
reasoning end.
* fix(responses): eliminate per-chunk thread spawning in async streaming path
_process_chunk() called run_async_function() on every SSE chunk, which
when invoked from an async context spawns a thread + event loop per call.
Move the hook call out of _process_chunk into the callers: async __anext__
directly awaits it, sync __next__ uses run_async_function.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* perf: reduce responses streaming CPU for text-only streams
* fix(test): replace deprecated claude-3-7-sonnet-latest in responses API test
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix(test): replace deprecated claude-3-7-sonnet-latest in tool result fix test
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix(test): replace deprecated claude-3-7-sonnet-latest in tool result empty call_id test
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Add attempted_retries and max_retries fields to SpendLogsMetadata so the
Logs page can display how many retries occurred for each request. The
router now injects retry tracking metadata before each make_call, which
flows through the logging pipeline into the spend logs metadata JSON.
The UI shows "Not Retried" when the first attempt succeeded, and
"N / M" (attempted / max) when retries occurred. The field is hidden
for requests that did not go through the router.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
ModelResponse.choices was typed as List[Union[Choices, StreamingChoices]] which
caused Pydantic serialization warnings and false linting errors. Now that
ModelResponseStream exists for streaming, narrow ModelResponse.choices to
List[Choices] and migrate all ModelResponse(stream=True) call sites to use
ModelResponseStream() instead.
Bedrock multimodal embedding models (Titan and Nova) were being costed
using the per-token text rate instead of the correct flat per-image rate
($0.00006/image). The pricing data was correct but never applied because
image_count was never populated in prompt_tokens_details.
Pass batch_data to Titan/Nova response transformers so they can count
image inputs and set PromptTokensDetailsWrapper(image_count=N) on Usage,
mirroring the existing Vertex AI pattern from PR #9623. Also fix the
text_tokens fallback in generic_cost_per_token to not override
text_tokens=0 when image_count > 0 (image-only requests).
* fix(model-pricing): add missing fireworks_ai model pricing for glm-4p7, minimax-m2p1, kimi-k2p5
Fireworks AI models called via short-form (fireworks_ai/<model>) were
reporting $0.00 cost because the pricing JSON lacked short-form entries.
The lookup fell through to the fireworks-ai-default bucket which has
zero cost.
Added 5 new entries to model_prices_and_context_window.json:
- fireworks_ai/accounts/fireworks/models/glm-4p7 (new long-form)
- fireworks_ai/accounts/fireworks/models/minimax-m2p1 (new long-form)
- fireworks_ai/glm-4p7 (new short-form)
- fireworks_ai/minimax-m2p1 (new short-form)
- fireworks_ai/kimi-k2p5 (new short-form; long-form already existed)
Pricing sourced from fireworks.ai model pages and pricing page.
* add cache_read_input_token_cost to kimi-k2p5 long-form entry for consistency
For gemini-3-pro-preview (not gemini-3-flash or gemini-3.1-pro-preview),
reasoning_effort="medium" maps to thinkingLevel="high" because the "medium"
thinking level is not available on that model variant.
Both test_reasoning_effort_maps_to_thinking_level_gemini_3 and
test_reasoning_effort_dict_format_gemini_3 had the correct comment
("medium -> high") but the wrong assertion (== "medium"). Fixed to
match the production code and the comments.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
The production error message was expanded when Sonnet 4.6 was also added as
a supported model for effort='max'. The test's match regex still referenced
the old "Claude Opus 4.6"-only message; update it to match the new
"Claude 4.6 models" wording.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Reset logo_path to default_logo when custom UI_LOGO_PATH file doesn't
exist, so the else branch at the bottom of get_image serves the default
logo instead of the non-existent custom path.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Add os.path.exists check before serving custom local logo so that a
non-existent UI_LOGO_PATH gracefully falls through to the cache/default
instead of causing a FileResponse error.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
The /get_image endpoint checked for cached_logo.jpg before reading the
UI_LOGO_PATH env var, so a pre-existing cache (e.g. baked into the base
Docker image) would always be served, ignoring the user's custom logo.
Move the UI_LOGO_PATH read before the cache check and serve local file
paths directly, bypassing the cache. The cache optimization is preserved
for HTTP URLs and the default logo where it is actually needed.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Empty system messages were skipped for Anthropic's system param but
not removed from the messages list, causing BadRequestError when
anthropic_messages_pt encountered the unsupported "system" role.
Fixes#21622
Pydantic v2's Union serializer for `List[Union[Choices, StreamingChoices]]`
tries both branches when serializing, emitting spurious
`PydanticSerializationUnexpectedValue` warnings (field count mismatch on
`Message` and type mismatch `Expected StreamingChoices but got Choices`).
Add a `WrapSerializer` on the `choices` field that serializes each item
individually via its own `model_dump()`, bypassing the Union dispatch
entirely while correctly propagating `exclude_none`, `exclude_unset`,
and `exclude_defaults` from the parent serialization context.
Regular team members could not see service accounts belonging to their
team. Additionally, former team members could still see service accounts
they created via the created_by filter after leaving the team.
- Add get_member_team_ids() to retrieve all teams a user belongs to
- Regular members now see team service accounts (user_id=NULL) but not
other members' personal keys
- Scope created_by filter to the user's current team memberships
- Add 7 unit tests covering the new visibility rules
Replace find_many + Python-side aggregation with a single SQL GROUP BY
query via query_raw in get_daily_activity_aggregated. This collapses
rows across entities (users/teams/orgs) in the database, reducing ~150k
rows to ~2-3k grouped rows before transfer to Python.
Also adds composite indexes (entity_id, date) to all 6 daily spend
tables for faster filtered queries.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
The admin bulk-delete path (prisma_client.delete_data) was never comparing
the DB return value against the requested tokens, so failed_tokens was
always empty for admins even when the DB silently skipped some tokens.
Adds the same mismatch check as the non-admin path, and a new test that
exercises admin bulk-delete returning fewer tokens than requested.
Fixes gap identified by Greptile review on #21609.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Instead of raising an exception when some requested tokens are not found
in the DB, populate `failed_tokens` in the return dict so callers can
inspect which keys were not deleted without catching a generic Exception.
Supersedes #12577.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
When extended thinking is enabled, the websearch interception agentic loop
builds a follow-up assistant message with only tool_use blocks. Anthropic's
API requires assistant messages to start with thinking/redacted_thinking
blocks when thinking is enabled, causing a 400 Bad Request.
Extract thinking blocks from the model's initial response, thread them
through the agentic loop, and prepend them to the follow-up assistant
message — matching the pattern used by anthropic_messages_pt in factory.py.
Fixes the error: "Expected 'thinking' or 'redacted_thinking', but found
'tool_use'"
* fix aviation safety topic filter: remove overly broad exceptions, add cockpit access block words
* fix airline brand protection filter: add identifier words, competitor/ops block words, tighten exceptions
* add example_sentences to all policy templates + topic-filtering and prompt-injection templates
* add policy_endpoints package with AI policy suggester
* update test patch targets for policy_endpoints package move
* add unit tests for AI policy suggester
* add suggestPolicyTemplates networking function
* add AI suggestion modal component
* add Use AI button and template loading callback to PolicyTemplates
* wire up AI suggestion modal in policies page
* fix policy_templates_backup.json path after package move
* add estimated_latency field to all policy templates
* use llm_router and accept model parameter in ai_policy_suggester
* add model param to suggest templates endpoint
* pass model param in suggestPolicyTemplates
* polish ai suggestion modal: model selector, auto-growing textareas, latency badges
* add template queue for processing multiple AI-suggested templates
* show template progress badge in guardrail selection modal
Moonshot's _transform_messages unconditionally flattened content arrays
to plain text, dropping image_url blocks. Vision models like kimi-k2.5
accept the standard OpenAI content array format.
Now checks for image_url blocks before flattening — if any message
contains images the content array is preserved intact.
Fixes#20862
AgentCore MCP server endpoints require the Accept header to contain
both application/json and text/event-stream per the MCP specification
(Streamable HTTP transport). Without this header, requests are rejected
with a 406 Not Acceptable error (JSON-RPC code -32011).
Sets the Accept header at the top of sign_request() so both JWT/Bearer
and SigV4 authentication paths include it.
* fix(scim): handle deprovisioning operations without path field
When SCIM providers send deprovisioning requests without a path field
(e.g., {"op": "replace", "value": {"active": false}}), the code was
storing the value under an empty string key in metadata.
This fix:
- Detects operations with no path where value is a dict
- Extracts and handles known fields like 'active' correctly
- Sets metadata["scim_active"] = false instead of metadata[""] = {"active": false}
Fixes: SCIM deprovisioning creating empty string keys in user metadata
* fix(scim): handle all known fields in operations without path
Extended the fix to handle all SCIM fields (not just active) when
operations have no path field:
- active -> scim_active
- displayName -> user_alias
- externalId -> user_id
- name.givenName/familyName -> scim_metadata
Added comprehensive test for multiple fields without path.
Addresses Greptile review feedback on RFC 7644 compliance.
* trigger PR update
