The response.output_item.done handler for function_call type was emitting
finish_reason='tool_calls' and a duplicate tool_call delta. This caused
premature stream termination after the first tool call in multi-tool
scenarios — downstream wrappers (e.g. AnthropicStreamWrapper) would close
the stream before subsequent tool calls arrived.
The response.completed event already inspects the response output list and
emits finish_reason='tool_calls' when function_call items are present, so
output_item.done does not need to (and must not) do so.
This mirrors the existing fix for message-type output_item.done (#17246).
Updated test_function_call_done_emits_is_finished (renamed) to assert
finish_reason=None and no duplicate delta. Updated test_text_plus_tool_calls_sequence
to match. Added test_multi_tool_call_stream_no_premature_finish which exercises
a synthetic 2-tool-call stream and verifies no premature termination.
Renamed test_build_custom_pricing_entry_kwargs_take_precedence_over_model_info
to test_build_custom_pricing_entry_setdefault_does_not_override_existing.
The original test claimed to verify kwargs precedence over model_info but
had no overlapping keys between the two sources. CustomPricingLiteLLMParams
fields and the model_info metadata keys (mode, supports_prompt_caching,
max_tokens) do not currently overlap. Updated the test to verify that
model_info fields are correctly merged, and added an explicit setdefault
assertion demonstrating that pre-existing keys would not be overwritten.
Co-authored-by: openhands <openhands@all-hands.dev>
Previously, register_model() was called with only input_cost_per_token,
output_cost_per_token, and litellm_provider. This dropped ~40+ other
pricing fields from CustomPricingLiteLLMParams (cache_read_input_token_cost,
cache_creation_input_token_cost, output_cost_per_reasoning_token, etc.)
as well as model_info metadata (mode, supports_prompt_caching, max_tokens).
For DB-sourced custom-priced models, the first request after a pod restart
would register a partial entry in litellm.model_cost, causing cost
calculations to miss cache token discounts and other extended pricing
until the entry was later enriched by deployment_callback_on_success
mutating the lru_cache.
Changes:
- Add _build_custom_pricing_entry() helper that iterates over all
CustomPricingLiteLLMParams.model_fields and merges model_info metadata
- Replace hardcoded 3-field dicts in both completion() and embedding()
with the new helper
- Add 7 tests covering field collection, model_info merging, precedence,
None skipping, and end-to-end register_model behavior
Co-authored-by: openhands <openhands@all-hands.dev>
- Parse VIDEO modality in promptTokensDetails → prompt_tokens_details.video_tokens
- Parse VIDEO modality in candidatesTokensDetails → completion_tokens_details.video_tokens
- Parse VIDEO modality in cacheTokensDetails and subtract from prompt video tokens
- Add video_tokens field to PromptTokensDetailsWrapper and CompletionTokensDetailsWrapper
- Fix implicit caching text-token fallback to not fire when cacheTokensDetails is present
- Add 4 unit tests covering: prompt video tokens, response video tokens,
auto-calculated text fallback with video, and explicit video cache subtraction
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Covers:
- Region + modelId correctly extracted for ap-northeast-1, us-east-1, us-west-2
- No region in path leaves modelId and optional_params unchanged
- Cross-region inference prefixes (us., eu., ap.) are not treated as region segments
- Explicitly set aws_region_name is not overridden by region in model path
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
When a user passes model="bedrock/ap-northeast-1/moonshotai.kimi-k2.5", get_llm_provider
strips the "bedrock/" prefix and passes "ap-northeast-1/moonshotai.kimi-k2.5" to the
converse handler. Two bugs occurred:
1. modelId was encoded as "ap-northeast-1%2Fmoonshotai.kimi-k2.5" (region included),
which AWS rejects as "not a valid model identifier"
2. The region ap-northeast-1 was never extracted, so the request went to the wrong
default region instead
Fix: after stripping routing prefixes in converse_handler.py completion(), check if the
remaining path starts with a known AWS region and strip it from modelId, injecting it
into optional_params so _get_aws_region_name picks it up.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Extract the repeated usage-combining block from both
_completion_streaming_iterator and _acompletion_streaming_iterator into a
shared static helper method _combine_fallback_usage. This brings both
functions under the PLR0915 50-statement limit, removing the noqa
suppressions.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Add `# noqa: PLR0915` suppression to match the async twin
`_acompletion_streaming_iterator` which already carries the same
suppression. The function's complexity is inherent (nested class,
generator with fallback logic, cleanup code).
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
The observatory test workflow failed because the "Verify tunnel
connectivity" step used a single curl with no retries. Cloudflare quick
tunnels need time for DNS propagation, and the first lookup can return
NXDOMAIN (curl exit 6). Replace with a retry loop (10 attempts, 5s
apart) matching the pattern already used in the health check step.
Also add `# noqa: PLR0915` to `_completion_streaming_iterator` in
router.py, matching the suppression already on its async twin.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
actions/checkout treats short commit hashes as branch names, causing
fetch failures. The checkout only needs the config file from the
repo, so use the default branch instead of a specific ref.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
The secrets context is not available in step-level if: conditions,
causing the workflow file to fail validation. Move the conditional
check into the shell script instead.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Pass AZURE_API_KEY, AZURE_API_BASE, OBSERVATORY_URL,
OBSERVATORY_API_KEY, and REQUEST_ID through step-level env
blocks so they are never interpolated directly into shell scripts.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Validate inputs.tag matches vX.Y.Z format to prevent script
injection via workflow_dispatch
- Pass tag via env var instead of direct interpolation in shell
- Add cleanup step to kill cloudflared and remove docker container
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Add timeout-minutes: 30 to prevent runaway jobs
- Build /run-test payload with jq --arg to safely escape
TUNNEL_URL and LITELLM_MASTER_KEY values
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Fail early if request_id is missing or null from the /run-test
response instead of polling /run-status/null for 15 minutes.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Avoids shell quoting issues with single quotes in JSON and
multi-line output truncation when using GITHUB_OUTPUT.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Add permissions block (contents: read) per GitHub security scan
- Poll /run-status/{request_id} instead of global /queue-status
to avoid race conditions with concurrent test runs
- Add result verification step that fails the workflow if tests
did not pass or the run errored
- Fix auth header to use X-LiteLLM-Observatory-API-Key
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- New reusable workflow that spins up a LiteLLM container from the
release image, exposes it via cloudflared tunnel, and triggers
test runs on the Railway-hosted observatory
- Integrates into ghcr_deploy.yml for RC and stable releases
- Can also be triggered manually via workflow_dispatch
- Add placeholder litellm_config.yaml for observatory test models
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
The transformation.py file was using FEATHERLESS_API_KEY (missing _AI_)
while the rest of the codebase (get_llm_provider_logic.py, utils.py)
correctly uses FEATHERLESS_AI_API_KEY. This caused 401 auth errors when
the user set FEATHERLESS_AI_API_KEY as documented.
Now checks FEATHERLESS_AI_API_KEY first (canonical name) with fallback
to FEATHERLESS_API_KEY (legacy compatibility). Same fix applied to
FEATHERLESS_AI_API_BASE.
Refs: #22490
- Add @log_db_metrics decorator to get_jwt_key_mapping_object for
consistent DB latency/error tracking with other helpers
- Move virtual key mapping lookup before auth_builder() to avoid
unnecessary team/user/org DB queries when mapping resolves
- JWT is decoded early; auth_builder only runs when no mapping found
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Persist description field on create (was silently dropped)
- Remove phantom key_alias from JWTKeyMappingResponse (not in schema)
- Populate created_by/updated_by audit fields from authenticated user
- Pass actual jwt_valid_token in admin path instead of empty dict
- Restore hash_token on create and fix duplicate try block
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Add page/size query params with take/skip to prevent unbounded queries.
Returns paginated response with total_count, current_page, total_pages.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Replace client-side full-fetch loop with single react-query call using
keepPreviousData; remove 5-second polling
- All filters (object ID, action, table, changed_by, team ID, key hash)
now passed as query params to the backend
- Add object_team_id and object_key_hash params to /audit endpoint using
Prisma JSON path filtering (PostgreSQL) to search inside before_value
and updated_values JSON columns
- Migrate table from custom TanStack DataTable to AntD Table with
server-side pagination
- Replace inline row expansion with a right-side AntD Drawer showing
metadata and before/after diff
- Refactor uiAuditLogsCall to accept a structured options object
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Add detailed UI walkthrough for Project Management feature including:
- Beta notice with link to API documentation
- Overview of projects and organizational hierarchy
- Prerequisites and setup instructions
- Separate section for enabling projects in UI settings
- Step-by-step guide for creating and managing projects
- Use cases for key organization within teams
- Next steps and related documentation links
- Proper sidebar navigation integration
Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>
A new QueryClient() was instantiated inside 5 component render functions
and at module-level in 3 more pages, with no shared QueryClientProvider
in either layout. This caused isolated, ephemeral caches with no
cross-page sharing and cache destruction on every re-render.
Moves QueryClient to a single module-level constant in AntdGlobalProvider
(the existing root-level "use client" wrapper in app/layout.tsx) and
removes the per-page QueryClient instantiations and QueryClientProvider
wrappers from all 8 affected pages.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Enables guardrail processing for OCR requests and responses. Adds OCR handler under litellm/llms/mistral/ocr/guardrail_translation/ to process document URLs on input and extracted page markdown on output. Includes route-to-call-type mappings for /ocr and /v1/ocr endpoints. Adds 14 unit tests and 4 e2e tests verifying handler discovery, input/output processing, and integration with UnifiedLLMGuardrails.
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Moves is_accepted=True from GET /onboarding/get_token to POST /onboarding/claim_token,
so the flag accurately reflects that a password has been set. Both endpoints now reject
already-used links, with get_token rejecting before any user data is returned.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
