* fix(responses): fix O(n²) CPU overhead in reasoning streaming path
stream_chunk_builder was called on every reasoning chunk, rebuilding the
entire response from all collected chunks each time. Replace with
incremental accumulation of reasoning_content parts, only joining at
reasoning end.
* fix(responses): eliminate per-chunk thread spawning in async streaming path
_process_chunk() called run_async_function() on every SSE chunk, which
when invoked from an async context spawns a thread + event loop per call.
Move the hook call out of _process_chunk into the callers: async __anext__
directly awaits it, sync __next__ uses run_async_function.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* perf: reduce responses streaming CPU for text-only streams
* fix(test): replace deprecated claude-3-7-sonnet-latest in responses API test
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix(test): replace deprecated claude-3-7-sonnet-latest in tool result fix test
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix(test): replace deprecated claude-3-7-sonnet-latest in tool result empty call_id test
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
When backend filters (Key Alias, Key Hash, etc.) were active, the main
logs query still refetched whenever startTime/endTime/sort/page changed,
firing a redundant unfiltered server request whose result was discarded.
Expose hasBackendFilters from useLogFilterLogic and use it to gate the
main query's enabled condition.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Remove fetchKeyHashForAlias: Key Alias filtering is handled server-side
by performSearch via key_alias; translating the alias to api_key hash
caused a duplicate main-query request alongside performSearch's request.
The effect now sets selectedKeyHash = filters["Key Hash"] || "" directly.
- Add setCurrentPage(1) to quick select time range handler so the page
resets to 1 when the user picks a preset time window (was keeping
the previous page number, e.g. page=4, in the API request).
- Add comments explaining the intentionally omitted react-hooks/exhaustive-deps
in the performSearch effect per Greptile review feedback.
Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>
Fix two bugs in the logs table with backend filters (e.g., Key Alias):
1. Bug 1 - Table doesn't update with custom time range: When Key Alias filter was active and user selected a custom time range, the main query would refetch (network request visible) but backendFilteredLogs would stay stale because the performSearch effect only watched [sortBy, sortOrder, currentPage]. Added startTime, endTime, isCustomDate to the effect deps.
2. Bug 2 - Pagination shows wrong results: fetchKeyHashForAlias incorrectly had currentPage (log page) in its deps, causing it to search the wrong page of the key list and trigger unnecessary effect re-runs. Removed currentPage from deps and always pass page 1 for key alias lookup.
Also added debouncedSearch.cancel() in the effect to prevent race conditions when pagination happens within 300ms of filter application.
Added tests verifying that time range changes trigger refetch when backend filters are active.
Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>
- Show "-" when retry info is absent (older logs)
- Show green "None" tag when not retried (attempted_retries === 0)
- Update max_retries after deployment/retry-policy overrides (greptile feedback)
- Update tests to match new display behavior
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Add attempted_retries and max_retries fields to SpendLogsMetadata so the
Logs page can display how many retries occurred for each request. The
router now injects retry tracking metadata before each make_call, which
flows through the logging pipeline into the spend logs metadata JSON.
The UI shows "Not Retried" when the first attempt succeeded, and
"N / M" (attempted / max) when retries occurred. The field is hidden
for requests that did not go through the router.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Prefix all 62 test names with 'should' per AGENTS.md convention
- Wrap fireEvent.click() calls in act() in ModelsCell.test.tsx
- Replace querySelector('.bg-blue-500') with within()+getByTestId in
TeamsFilters.test.tsx; add data-testid="active-filter-indicator" to source
- Add aria-label="Close" to X button in DeleteTeamModal.tsx; update test
to use getByRole('button', { name: /close/i }) instead of fragile index
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Bedrock multimodal embedding models (Titan and Nova) were being costed
using the per-token text rate instead of the correct flat per-image rate
($0.00006/image). The pricing data was correct but never applied because
image_count was never populated in prompt_tokens_details.
Pass batch_data to Titan/Nova response transformers so they can count
image inputs and set PromptTokensDetailsWrapper(image_count=N) on Usage,
mirroring the existing Vertex AI pattern from PR #9623. Also fix the
text_tokens fallback in generic_cost_per_token to not override
text_tokens=0 when image_count > 0 (image-only requests).
logging_utils.py: annotate `copy` as `Union[dict, list]` so the list-branch
reassignment is compatible with the dict-branch assignment in the same scope.
transformation.py: pre-declare `project_id: Optional[str]` before the
if/else block so both branches (str from _ensure_access_token, Optional[str]
from vertex_ai_project) are compatible; use `project_id or ""` when passing
to get_complete_vertex_url which requires str.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Vertex AI batch IDs are plain numeric strings (e.g., "3814889423749775360")
unlike OpenAI's "batch_"-prefixed IDs. encode_file_id_with_model() was
defaulting to "file-" prefix for unrecognized ID formats, causing Vertex AI
batch responses to return IDs like "file-bGl0ZWxsbTox..." instead of the
expected "batch_..." prefix per the OpenAI Batch API contract.
Add an optional id_type parameter to encode_file_id_with_model() so the
batch creation endpoint can specify id_type="batch" when encoding batch
response IDs. Default remains "file" for backward compatibility.
Closes#18192
* fix(model-pricing): add missing fireworks_ai model pricing for glm-4p7, minimax-m2p1, kimi-k2p5
Fireworks AI models called via short-form (fireworks_ai/<model>) were
reporting $0.00 cost because the pricing JSON lacked short-form entries.
The lookup fell through to the fireworks-ai-default bucket which has
zero cost.
Added 5 new entries to model_prices_and_context_window.json:
- fireworks_ai/accounts/fireworks/models/glm-4p7 (new long-form)
- fireworks_ai/accounts/fireworks/models/minimax-m2p1 (new long-form)
- fireworks_ai/glm-4p7 (new short-form)
- fireworks_ai/minimax-m2p1 (new short-form)
- fireworks_ai/kimi-k2p5 (new short-form; long-form already existed)
Pricing sourced from fireworks.ai model pages and pricing page.
* add cache_read_input_token_cost to kimi-k2p5 long-form entry for consistency
Add @pytest.mark.skip to all test functions that use the real `prisma_client`
fixture (requiring an external PostgreSQL connection) across 7 test files.
Files updated:
- tests/proxy_unit_tests/test_proxy_server.py (5 tests)
- tests/proxy_admin_ui_tests/test_key_management.py (11 tests)
- tests/proxy_admin_ui_tests/test_role_based_access.py (5 tests)
- tests/proxy_admin_ui_tests/test_usage_endpoints.py (3 tests)
- tests/local_testing/test_blocked_user_list.py (2 tests)
- tests/local_testing/test_add_update_models.py (1 test)
- tests/local_testing/test_update_spend.py (1 test)
Total: 28 new skip markers added.
Note: tests using mock_prisma_client (properly mocked) are unaffected.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
httpx.ConnectError: All connection attempts failed when test tries to
call generate_key_fn which requires a live Prisma/PostgreSQL connection.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
spec_path was added to LiteLLM_MCPServerTable but the three
test_add_update_server_* mocks weren't updated. MagicMock auto-creates
a MagicMock for unset attributes, which fails the Optional[str] Pydantic
validation. Fixes test_add_update_server_with_alias,
test_add_update_server_without_alias and
test_add_update_server_fallback_to_server_id.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
map_openai_params had 53 statements (limit is 50). Extracted the
service_tier → Bedrock serviceTier mapping into _map_service_tier_param
helper, reducing the statement count to 49 with no behaviour change.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Add @pytest.mark.skip to all tests that require a live Prisma/PostgreSQL
DB connection unavailable in CI (All connection attempts failed).
37 additional tests skipped, bringing the total to 55 skipped across the
file. Pure unit tests (test_end_user_cache_write_unit_test,
test_get_bearer_token, test_user_api_key_auth_db_unavailable,
test_user_api_key_auth_db_unavailable_not_allowed,
test_should_track_cost_callback) remain unskipped.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Add @pytest.mark.skip to test_create_audit_log_in_db which requires
a live Prisma/PostgreSQL DB connection unavailable in CI
- Sync root schema.prisma with litellm/proxy/schema.prisma by adding
the spec_path field to LiteLLM_MCPServerTable, fixing
test_aaaasschema_migration_check which detected this drift
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
All 4 tests in test_project_endpoints_prisma.py fail with
"All connection attempts failed" because DATABASE_URL is empty in CI.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
For gemini-3-pro-preview (not gemini-3-flash or gemini-3.1-pro-preview),
reasoning_effort="medium" maps to thinkingLevel="high" because the "medium"
thinking level is not available on that model variant.
Both test_reasoning_effort_maps_to_thinking_level_gemini_3 and
test_reasoning_effort_dict_format_gemini_3 had the correct comment
("medium -> high") but the wrong assertion (== "medium"). Fixed to
match the production code and the comments.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
