From 443f0ca4cd9396cadc624972aa8125a076f7011b Mon Sep 17 00:00:00 2001 From: ryan-crabbe-berri Date: Thu, 4 Jun 2026 07:41:31 -0700 Subject: [PATCH 1/4] ci(ui): frontend-lint job enforcing prettier + eslint on changed files (#29633) * ci(ui): add frontend-lint job enforcing prettier and eslint on changed files Lints only the files a PR adds or modifies under ui/litellm-dashboard, so new and touched code must be prettier-clean and eslint-clean while the existing tree is grandfathered. Skips cleanly when a PR touches no lintable UI files. This lets us adopt the formatters incrementally without a repo-wide reformat * ci(ui): write frontend-lint file lists to $RUNNER_TEMP Keep the prettier/eslint changed-file lists out of the checkout dir so they cannot collide with a future source file of the same name * lint(ui): baseline existing eslint findings so only new ones block Capture the current error-level eslint findings (318 across 183 files) in a committed suppressions baseline via eslint --suppress-all. Every rule stays at its error severity, so any newly introduced violation fails the frontend-lint gate, while the existing tree is grandfathered; touching a legacy file never forces fixing its pre-existing issues. CI runs eslint with --pass-on-unpruned-suppressions so that fixing a baselined issue does not fail on a now-stale suppression, and the generated baseline is prettier-ignored since eslint owns its format. Burn the baseline down over time with eslint --prune-suppressions * lint(ui): enforce a count budget for explicit any Make @typescript-eslint/no-explicit-any a warning and cap the total instead of hard-blocking each new one. A frontend-lint step counts the repo-wide explicit any and fails only when it exceeds the committed budget in eslint-any-budget.json. max starts at 2031, ten above the current 2021, so the next ten land as warnings and the build fails once that headroom is gone. Lower max over time toward target to ratchet the count down. New anys still surface as warnings on changed files via the normal eslint step * lint(ui): enable zero-cost rules no-var, no-self-assign, react/no-danger These have no existing violations, so they need no baseline; turning them on purely blocks new instances. react/no-danger guards against new dangerouslySetInnerHTML (XSS), no-var enforces let/const, and no-self-assign catches self-assignment typos. no-debugger is already enforced by the recommended preset * lint(ui): add baselined complexity rules Enable complexity:20, max-depth:4, max-params:4, max-nested-callbacks:4, with thresholds set near the codebase p99 so only genuine outliers are flagged. The 272 existing over-threshold functions are grandfathered in the suppressions baseline; new over-threshold functions block. Lower the thresholds over time to ratchet complexity down. max-lines-per-function is intentionally left off since React components are legitimately long * lint(ui): ban new raw fetch, standardize on React Query Add a no-restricted-syntax rule flagging bare fetch() calls, pointing contributors at React Query (@tanstack/react-query). The rule is not exempted anywhere, including the already-bloated networking.tsx, so all 331 existing fetch calls are grandfathered but no new ones can be added there or elsewhere. New data access goes through React Query, and the networking layer can be migrated out and pruned from the baseline over time * lint(ui): ban new @tremor/react imports Add a no-restricted-imports rule flagging imports from @tremor/react so tremor is phased out rather than spread further. The 232 existing tremor imports are grandfathered in the baseline; new ones block and point at antd. Migrate components off tremor and prune the baseline over time * lint(ui): widen explicit-any budget headroom to 2040 Raise max from 2031 to 2040, giving ~19 of slack over the current 2021 instead of 10 * style(ui): prettier-format eslint.config.mjs The frontend-lint gate flagged its own config file. Format it so the prettier check on this PR's changed files passes * lint(ui): soften complexity and max-depth to warnings These two are smell metrics with arbitrary thresholds where a legit new function can trip them, so make them advisory rather than hard-blocking. They drop out of the baseline (now 963). max-params, max-nested-callbacks, and the react-hooks rules stay strict since those are clear-cut * lint(ui): move complexity and max-depth to the count-budget pattern Generalize the explicit-any budget into a shared lint-budget mechanism: eslint-budgets.json maps a rule to {max, target} and check-lint-budgets.mjs counts each across the repo and fails when a count exceeds its max. complexity (129, max 140) and max-depth (61, max 70) now use the same slack-plus-counter model as explicit-any (2021, max 2040): they warn per-file and the build only fails if the repo-wide total crosses the ceiling. Lower each max toward its target over time * docs(ui): note pruning the eslint suppressions baseline when fixing lint debt --- .github/workflows/test-litellm-ui-build.yml | 76 + ui/litellm-dashboard/.prettierignore | 3 +- ui/litellm-dashboard/CLAUDE.md | 2 + ui/litellm-dashboard/eslint-budgets.json | 5 + ui/litellm-dashboard/eslint-suppressions.json | 2312 +++++++++++++++++ ui/litellm-dashboard/eslint.config.mjs | 28 +- .../scripts/check-lint-budgets.mjs | 30 + 7 files changed, 2453 insertions(+), 3 deletions(-) create mode 100644 ui/litellm-dashboard/eslint-budgets.json create mode 100644 ui/litellm-dashboard/eslint-suppressions.json create mode 100644 ui/litellm-dashboard/scripts/check-lint-budgets.mjs diff --git a/.github/workflows/test-litellm-ui-build.yml b/.github/workflows/test-litellm-ui-build.yml index 862f98e30f1..68497b10dbb 100644 --- a/.github/workflows/test-litellm-ui-build.yml +++ b/.github/workflows/test-litellm-ui-build.yml @@ -36,3 +36,79 @@ jobs: - name: Build run: npm run build + + frontend-lint: + runs-on: ubuntu-latest + timeout-minutes: 8 + defaults: + run: + working-directory: ui/litellm-dashboard + + steps: + - name: Checkout repository + uses: actions/checkout@08eba0b27e820071cde6df949e0beb9ba4906955 # v4.3.0 + with: + fetch-depth: 0 + persist-credentials: false + + - name: Collect changed files + id: changed + env: + BASE_SHA: ${{ github.event.pull_request.base.sha }} + run: | + : > "$RUNNER_TEMP/prettier_files.txt" + : > "$RUNNER_TEMP/eslint_files.txt" + while IFS= read -r f; do + [ -f "$f" ] || continue + case "$f" in + *.js | *.jsx | *.ts | *.tsx | *.mjs | *.cjs) + printf '%s\n' "$f" >> "$RUNNER_TEMP/prettier_files.txt" + printf '%s\n' "$f" >> "$RUNNER_TEMP/eslint_files.txt" ;; + *.json | *.css | *.scss | *.md | *.mdx | *.yml | *.yaml | *.html) + printf '%s\n' "$f" >> "$RUNNER_TEMP/prettier_files.txt" ;; + esac + done < <(git diff --name-only --diff-filter=ACMR --relative "$BASE_SHA"...HEAD -- .) + if [ -s "$RUNNER_TEMP/prettier_files.txt" ] || [ -s "$RUNNER_TEMP/eslint_files.txt" ]; then + echo "has_files=true" >> "$GITHUB_OUTPUT" + else + echo "has_files=false" >> "$GITHUB_OUTPUT" + echo "No lintable UI files changed in this PR; nothing to check." + fi + + - name: Setup Node.js + if: steps.changed.outputs.has_files == 'true' + uses: actions/setup-node@a0853c24544627f65ddf259abe73b1d18a591444 # v5.0 + with: + node-version: "20" + cache: "npm" + cache-dependency-path: ui/litellm-dashboard/package-lock.json + + - name: Install dependencies + if: steps.changed.outputs.has_files == 'true' + run: npm ci + + - name: Lint changed files (prettier + eslint) + if: steps.changed.outputs.has_files == 'true' + run: | + prettier_files=() + eslint_files=() + while IFS= read -r f; do prettier_files+=("$f"); done < "$RUNNER_TEMP/prettier_files.txt" + while IFS= read -r f; do eslint_files+=("$f"); done < "$RUNNER_TEMP/eslint_files.txt" + status=0 + if [ ${#prettier_files[@]} -gt 0 ]; then + echo "::group::Prettier (${#prettier_files[@]} files)" + npx prettier --check "${prettier_files[@]}" || { status=1; echo "::error::Unformatted files. Fix with: npm run format"; } + echo "::endgroup::" + fi + if [ ${#eslint_files[@]} -gt 0 ]; then + echo "::group::ESLint (${#eslint_files[@]} files)" + npx eslint --no-warn-ignored --pass-on-unpruned-suppressions "${eslint_files[@]}" || status=1 + echo "::endgroup::" + fi + exit $status + + - name: Check lint budgets + if: ${{ !cancelled() && steps.changed.outputs.has_files == 'true' }} + run: | + npx eslint . -f json -o "$RUNNER_TEMP/lint-report.json" || true + node scripts/check-lint-budgets.mjs "$RUNNER_TEMP/lint-report.json" eslint-budgets.json diff --git a/ui/litellm-dashboard/.prettierignore b/ui/litellm-dashboard/.prettierignore index ab37c884be1..d489a619838 100644 --- a/ui/litellm-dashboard/.prettierignore +++ b/ui/litellm-dashboard/.prettierignore @@ -8,4 +8,5 @@ build .turbo .next-static *.min.js -coverage/ \ No newline at end of file +coverage/ +eslint-suppressions.json \ No newline at end of file diff --git a/ui/litellm-dashboard/CLAUDE.md b/ui/litellm-dashboard/CLAUDE.md index 3d43019c749..7af913ab016 100644 --- a/ui/litellm-dashboard/CLAUDE.md +++ b/ui/litellm-dashboard/CLAUDE.md @@ -1 +1,3 @@ Never put LiteLLM tokens or API keys in `localStorage`. `localStorage` survives browser close. Prefer `httpOnly` cookies, or `sessionStorage` at most, understanding that any web storage is readable by injected scripts (XSS), and only httpOnly cookies are not + +When you fix lint violations that are grandfathered in `eslint-suppressions.json`, run `eslint . --prune-suppressions` and commit the updated baseline so the gate ratchets down instead of leaving a stale suppression diff --git a/ui/litellm-dashboard/eslint-budgets.json b/ui/litellm-dashboard/eslint-budgets.json new file mode 100644 index 00000000000..2139d177512 --- /dev/null +++ b/ui/litellm-dashboard/eslint-budgets.json @@ -0,0 +1,5 @@ +{ + "@typescript-eslint/no-explicit-any": { "max": 2040, "target": 1500 }, + "complexity": { "max": 140, "target": 80 }, + "max-depth": { "max": 70, "target": 30 } +} diff --git a/ui/litellm-dashboard/eslint-suppressions.json b/ui/litellm-dashboard/eslint-suppressions.json new file mode 100644 index 00000000000..1110403446c --- /dev/null +++ b/ui/litellm-dashboard/eslint-suppressions.json @@ -0,0 +1,2312 @@ +{ + "src/app/(dashboard)/api-reference/APIReferenceView.tsx": { + "no-restricted-imports": { + "count": 1 + } + }, + "src/app/(dashboard)/hooks/accessGroups/useAccessGroupDetails.ts": { + "no-restricted-syntax": { + "count": 1 + } + }, + "src/app/(dashboard)/hooks/accessGroups/useAccessGroups.ts": { + "no-restricted-syntax": { + "count": 1 + } + }, + "src/app/(dashboard)/hooks/accessGroups/useCreateAccessGroup.ts": { + "no-restricted-syntax": { + "count": 1 + } + }, + "src/app/(dashboard)/hooks/accessGroups/useDeleteAccessGroup.ts": { + "no-restricted-syntax": { + "count": 1 + } + }, + "src/app/(dashboard)/hooks/accessGroups/useEditAccessGroup.ts": { + "no-restricted-syntax": { + "count": 1 + } + }, + "src/app/(dashboard)/hooks/blogPosts/useBlogPosts.ts": { + "no-restricted-syntax": { + "count": 1 + } + }, + "src/app/(dashboard)/hooks/cloudzero/useCloudZeroCreate.ts": { + "no-restricted-syntax": { + "count": 1 + } + }, + "src/app/(dashboard)/hooks/cloudzero/useCloudZeroDryRun.ts": { + "no-restricted-syntax": { + "count": 1 + } + }, + "src/app/(dashboard)/hooks/cloudzero/useCloudZeroExport.ts": { + "no-restricted-syntax": { + "count": 1 + } + }, + "src/app/(dashboard)/hooks/cloudzero/useCloudZeroSettings.ts": { + "no-restricted-syntax": { + "count": 3 + } + }, + "src/app/(dashboard)/hooks/configOverrides/hashicorpVaultApi.ts": { + "no-restricted-syntax": { + "count": 4 + } + }, + "src/app/(dashboard)/hooks/guardrails/useRegisterGuardrail.ts": { + "no-restricted-syntax": { + "count": 1 + } + }, + "src/app/(dashboard)/hooks/healthReadiness/useHealthReadinessDetails.ts": { + "no-restricted-syntax": { + "count": 1 + } + }, + "src/app/(dashboard)/hooks/keys/useKeyAliases.test.ts": { + "react/display-name": { + "count": 1 + } + }, + "src/app/(dashboard)/hooks/keys/useKeys.ts": { + "no-restricted-syntax": { + "count": 1 + } + }, + "src/app/(dashboard)/hooks/keys/useResetKeySpend.ts": { + "no-restricted-syntax": { + "count": 1 + } + }, + "src/app/(dashboard)/hooks/models/useModels.ts": { + "max-params": { + "count": 1 + } + }, + "src/app/(dashboard)/hooks/projects/useCreateProject.test.ts": { + "react/display-name": { + "count": 1 + } + }, + "src/app/(dashboard)/hooks/projects/useCreateProject.ts": { + "no-restricted-syntax": { + "count": 1 + } + }, + "src/app/(dashboard)/hooks/projects/useDeleteProject.test.ts": { + "react/display-name": { + "count": 1 + } + }, + "src/app/(dashboard)/hooks/projects/useDeleteProject.ts": { + "no-restricted-syntax": { + "count": 1 + } + }, + "src/app/(dashboard)/hooks/projects/useProjectDetails.test.ts": { + "react/display-name": { + "count": 1 + } + }, + "src/app/(dashboard)/hooks/projects/useProjectDetails.ts": { + "no-restricted-syntax": { + "count": 1 + } + }, + "src/app/(dashboard)/hooks/projects/useProjects.test.ts": { + "react/display-name": { + "count": 1 + } + }, + "src/app/(dashboard)/hooks/projects/useProjects.ts": { + "no-restricted-syntax": { + "count": 1 + } + }, + "src/app/(dashboard)/hooks/projects/useUpdateProject.test.ts": { + "react/display-name": { + "count": 1 + } + }, + "src/app/(dashboard)/hooks/projects/useUpdateProject.ts": { + "no-restricted-syntax": { + "count": 1 + } + }, + "src/app/(dashboard)/hooks/proxyConfig/useProxyConfig.ts": { + "no-restricted-syntax": { + "count": 2 + } + }, + "src/app/(dashboard)/hooks/router/useRouterFields.ts": { + "no-restricted-syntax": { + "count": 1 + } + }, + "src/app/(dashboard)/hooks/storeModelInDB/useStoreModelInDB.ts": { + "no-restricted-syntax": { + "count": 1 + } + }, + "src/app/(dashboard)/hooks/storeRequestInSpendLogs/useStoreRequestInSpendLogs.ts": { + "no-restricted-syntax": { + "count": 1 + } + }, + "src/app/(dashboard)/hooks/teams/useTeams.ts": { + "no-restricted-syntax": { + "count": 2 + } + }, + "src/app/(dashboard)/layout.tsx": { + "react-hooks/set-state-in-effect": { + "count": 1 + } + }, + "src/app/(dashboard)/models-and-endpoints/ModelsAndEndpointsView.tsx": { + "no-restricted-imports": { + "count": 1 + }, + "react-hooks/preserve-manual-memoization": { + "count": 4 + } + }, + "src/app/(dashboard)/models-and-endpoints/components/AllModelsTab.test.tsx": { + "max-params": { + "count": 1 + }, + "unused-imports/no-unused-imports": { + "count": 1 + } + }, + "src/app/(dashboard)/models-and-endpoints/components/AllModelsTab.tsx": { + "no-restricted-imports": { + "count": 1 + }, + "react-hooks/set-state-in-effect": { + "count": 3 + } + }, + "src/app/(dashboard)/models-and-endpoints/components/ModelRetrySettingsTab.test.tsx": { + "react/display-name": { + "count": 1 + } + }, + "src/app/(dashboard)/models-and-endpoints/components/ModelRetrySettingsTab.tsx": { + "no-restricted-imports": { + "count": 1 + } + }, + "src/app/(dashboard)/models-and-endpoints/components/PriceDataManagementTab.tsx": { + "no-restricted-imports": { + "count": 1 + } + }, + "src/app/(dashboard)/playground/page.tsx": { + "no-restricted-imports": { + "count": 1 + } + }, + "src/app/login/LoginPage.tsx": { + "react-hooks/set-state-in-effect": { + "count": 2 + } + }, + "src/app/model_hub/page.tsx": { + "react-hooks/set-state-in-effect": { + "count": 1 + } + }, + "src/app/model_hub_table/page.tsx": { + "react-hooks/set-state-in-effect": { + "count": 1 + } + }, + "src/app/page.tsx": { + "unused-imports/no-unused-imports": { + "count": 2 + } + }, + "src/components/AIHub/AgentHubTableColumns.test.tsx": { + "unused-imports/no-unused-imports": { + "count": 1 + } + }, + "src/components/AIHub/AgentHubTableColumns.tsx": { + "no-restricted-imports": { + "count": 1 + } + }, + "src/components/AIHub/ClaudeCodeMarketplaceTab.tsx": { + "no-restricted-imports": { + "count": 1 + }, + "react-hooks/immutability": { + "count": 1 + } + }, + "src/components/AIHub/ModelHubTable.test.tsx": { + "max-params": { + "count": 1 + } + }, + "src/components/AIHub/ModelHubTable.tsx": { + "no-restricted-imports": { + "count": 1 + } + }, + "src/components/AIHub/SkillHubDashboard.tsx": { + "no-restricted-imports": { + "count": 1 + } + }, + "src/components/AIHub/UsefulLinksManagement.tsx": { + "no-restricted-imports": { + "count": 1 + }, + "react-hooks/set-state-in-effect": { + "count": 1 + } + }, + "src/components/AIHub/forms/MakeAgentPublicForm.tsx": { + "no-restricted-imports": { + "count": 1 + }, + "react-hooks/set-state-in-effect": { + "count": 1 + } + }, + "src/components/AIHub/forms/MakeMCPPublicForm.test.tsx": { + "react/display-name": { + "count": 1 + } + }, + "src/components/AIHub/forms/MakeMCPPublicForm.tsx": { + "no-restricted-imports": { + "count": 1 + }, + "react-hooks/set-state-in-effect": { + "count": 1 + } + }, + "src/components/AIHub/forms/MakeModelPublicForm.tsx": { + "no-restricted-imports": { + "count": 1 + }, + "react-hooks/set-state-in-effect": { + "count": 1 + } + }, + "src/components/AIHub/marketplace_table_columns.tsx": { + "no-restricted-imports": { + "count": 1 + } + }, + "src/components/AdminPanel.tsx": { + "no-restricted-imports": { + "count": 1 + }, + "react-hooks/set-state-in-effect": { + "count": 1 + } + }, + "src/components/CostTrackingSettings/add_margin_form.tsx": { + "no-restricted-imports": { + "count": 1 + } + }, + "src/components/CostTrackingSettings/add_provider_form.tsx": { + "no-restricted-imports": { + "count": 1 + } + }, + "src/components/CostTrackingSettings/cost_tracking_settings.tsx": { + "no-restricted-imports": { + "count": 1 + } + }, + "src/components/CostTrackingSettings/how_it_works.tsx": { + "no-restricted-imports": { + "count": 1 + } + }, + "src/components/CostTrackingSettings/pricing_calculator/multi_cost_results.test.tsx": { + "unused-imports/no-unused-imports": { + "count": 1 + } + }, + "src/components/CostTrackingSettings/pricing_calculator/multi_cost_results.tsx": { + "no-restricted-imports": { + "count": 1 + } + }, + "src/components/CostTrackingSettings/pricing_calculator/multi_export_dropdown.test.tsx": { + "unused-imports/no-unused-imports": { + "count": 1 + } + }, + "src/components/CostTrackingSettings/pricing_calculator/multi_export_dropdown.tsx": { + "no-restricted-imports": { + "count": 1 + } + }, + "src/components/CostTrackingSettings/pricing_calculator/use_multi_cost_estimate.ts": { + "no-restricted-syntax": { + "count": 1 + } + }, + "src/components/CostTrackingSettings/provider_discount_table.test.tsx": { + "unused-imports/no-unused-imports": { + "count": 1 + } + }, + "src/components/CostTrackingSettings/provider_discount_table.tsx": { + "no-restricted-imports": { + "count": 1 + } + }, + "src/components/CostTrackingSettings/provider_display_helpers.test.ts": { + "unused-imports/no-unused-imports": { + "count": 1 + } + }, + "src/components/CostTrackingSettings/provider_margin_table.tsx": { + "no-restricted-imports": { + "count": 1 + } + }, + "src/components/CostTrackingSettings/use_discount_config.ts": { + "no-restricted-syntax": { + "count": 2 + } + }, + "src/components/CostTrackingSettings/use_margin_config.ts": { + "no-restricted-syntax": { + "count": 2 + } + }, + "src/components/CreateUserButton.tsx": { + "no-restricted-imports": { + "count": 1 + }, + "react-hooks/set-state-in-effect": { + "count": 1 + } + }, + "src/components/DefaultUserSettings.tsx": { + "no-restricted-imports": { + "count": 1 + } + }, + "src/components/DeletedKeysPage/DeletedKeysTable/DeletedKeysTable.tsx": { + "no-restricted-imports": { + "count": 1 + } + }, + "src/components/DeletedTeamsPage/DeletedTeamsTable/DeletedTeamsTable.tsx": { + "no-restricted-imports": { + "count": 1 + } + }, + "src/components/EntityUsageExport/ExportSummary.tsx": { + "no-restricted-imports": { + "count": 1 + } + }, + "src/components/EntityUsageExport/UsageExportHeader.tsx": { + "no-restricted-imports": { + "count": 2 + } + }, + "src/components/EntityUsageExport/types.ts": { + "no-restricted-imports": { + "count": 1 + } + }, + "src/components/EntityUsageExport/utils.test.ts": { + "no-restricted-imports": { + "count": 1 + } + }, + "src/components/EntityUsageExport/utils.ts": { + "max-params": { + "count": 3 + }, + "no-restricted-imports": { + "count": 1 + } + }, + "src/components/GuardrailsMonitor/EvaluationSettingsModal.tsx": { + "react-hooks/set-state-in-effect": { + "count": 1 + } + }, + "src/components/GuardrailsMonitor/GuardrailsMonitorView.tsx": { + "no-restricted-imports": { + "count": 1 + } + }, + "src/components/GuardrailsMonitor/ScoreChart.test.tsx": { + "react/display-name": { + "count": 1 + } + }, + "src/components/GuardrailsMonitor/ScoreChart.tsx": { + "no-restricted-imports": { + "count": 1 + } + }, + "src/components/HelpLink.test.tsx": { + "unused-imports/no-unused-imports": { + "count": 1 + } + }, + "src/components/MemoryView/MemoryView.tsx": { + "react-hooks/set-state-in-effect": { + "count": 1 + } + }, + "src/components/Navbar/BlogDropdown/BlogDropdown.test.tsx": { + "max-nested-callbacks": { + "count": 12 + } + }, + "src/components/Navbar/UserDropdown/UserDropdown.tsx": { + "react-hooks/set-state-in-effect": { + "count": 1 + } + }, + "src/components/OldTeams.test.tsx": { + "max-nested-callbacks": { + "count": 4 + } + }, + "src/components/OldTeams.tsx": { + "no-restricted-imports": { + "count": 1 + }, + "react-hooks/set-state-in-effect": { + "count": 4 + } + }, + "src/components/Projects/ProjectDetailsPage.tsx": { + "no-restricted-imports": { + "count": 1 + } + }, + "src/components/Projects/ProjectKeysSection.tsx": { + "react-hooks/set-state-in-effect": { + "count": 1 + } + }, + "src/components/Projects/ProjectModals/ProjectBaseForm.tsx": { + "react-hooks/set-state-in-effect": { + "count": 2 + } + }, + "src/components/Projects/ProjectsPage.tsx": { + "react-hooks/set-state-in-effect": { + "count": 1 + } + }, + "src/components/SCIM.tsx": { + "no-restricted-imports": { + "count": 1 + }, + "react-hooks/set-state-in-effect": { + "count": 1 + } + }, + "src/components/SSOModals.tsx": { + "no-restricted-imports": { + "count": 1 + } + }, + "src/components/SearchTools/CreateSearchTools.tsx": { + "no-restricted-imports": { + "count": 1 + }, + "react-hooks/set-state-in-effect": { + "count": 1 + } + }, + "src/components/SearchTools/SearchToolTester.tsx": { + "no-restricted-imports": { + "count": 1 + } + }, + "src/components/SearchTools/SearchToolView.tsx": { + "no-restricted-imports": { + "count": 1 + } + }, + "src/components/SearchTools/SearchTools.tsx": { + "no-restricted-imports": { + "count": 1 + }, + "react-hooks/static-components": { + "count": 1 + } + }, + "src/components/Settings/AdminSettings/MCPSemanticFilterSettings/MCPSemanticFilterSettings.tsx": { + "react-hooks/set-state-in-effect": { + "count": 1 + } + }, + "src/components/Settings/AdminSettings/SSOSettings/Modals/BaseSSOSettingsForm.tsx": { + "no-restricted-imports": { + "count": 1 + } + }, + "src/components/Settings/AdminSettings/SSOSettings/Modals/EditSSOSettingsModal.test.tsx": { + "max-nested-callbacks": { + "count": 1 + } + }, + "src/components/Settings/AdminSettings/SSOSettings/SSOSettingsLoadingSkeleton.test.tsx": { + "max-nested-callbacks": { + "count": 4 + } + }, + "src/components/Settings/AdminSettings/UISettings/PageVisibilitySettings.tsx": { + "react-hooks/set-state-in-render": { + "count": 2 + } + }, + "src/components/Settings/LoggingAndAlerts/LoggingCallbacks/LoggingCallbacksTable.tsx": { + "no-restricted-imports": { + "count": 1 + } + }, + "src/components/Settings/RouterSettings/Fallbacks/AddFallbacks.tsx": { + "no-restricted-imports": { + "count": 1 + }, + "react-hooks/set-state-in-effect": { + "count": 1 + } + }, + "src/components/Settings/RouterSettings/Fallbacks/FallbackSelectionForm.tsx": { + "no-restricted-imports": { + "count": 1 + }, + "react-hooks/set-state-in-effect": { + "count": 1 + } + }, + "src/components/Settings/RouterSettings/Fallbacks/Fallbacks.tsx": { + "no-restricted-imports": { + "count": 1 + } + }, + "src/components/ToolDetail.tsx": { + "unused-imports/no-unused-imports": { + "count": 2 + } + }, + "src/components/ToolPolicies.tsx": { + "no-restricted-imports": { + "count": 1 + }, + "react-hooks/set-state-in-effect": { + "count": 1 + }, + "react-hooks/static-components": { + "count": 7 + }, + "unused-imports/no-unused-imports": { + "count": 1 + } + }, + "src/components/UIAccessControlForm.tsx": { + "no-restricted-imports": { + "count": 1 + } + }, + "src/components/UsageIndicator.tsx": { + "no-restricted-imports": { + "count": 1 + }, + "react-hooks/static-components": { + "count": 1 + } + }, + "src/components/UsagePage/components/EndpointUsage/components/EndpointUsageBarChart.tsx": { + "no-restricted-imports": { + "count": 1 + } + }, + "src/components/UsagePage/components/EndpointUsage/components/EndpointUsageLineChart.tsx": { + "no-restricted-imports": { + "count": 1 + } + }, + "src/components/UsagePage/components/EntityUsage/EntityUsage.tsx": { + "no-restricted-imports": { + "count": 1 + } + }, + "src/components/UsagePage/components/EntityUsage/SpendByProvider.tsx": { + "no-restricted-imports": { + "count": 1 + } + }, + "src/components/UsagePage/components/EntityUsage/TopKeyView.tsx": { + "no-restricted-imports": { + "count": 1 + } + }, + "src/components/UsagePage/components/EntityUsage/TopModelView.tsx": { + "no-restricted-imports": { + "count": 1 + } + }, + "src/components/UsagePage/components/KeyModelUsageView.tsx": { + "no-restricted-imports": { + "count": 1 + } + }, + "src/components/UsagePage/components/UsageAIChatPanel.tsx": { + "react-hooks/immutability": { + "count": 1 + } + }, + "src/components/UsagePage/components/UsagePageView.tsx": { + "no-restricted-imports": { + "count": 1 + }, + "react-hooks/purity": { + "count": 1 + }, + "react-hooks/set-state-in-effect": { + "count": 3 + } + }, + "src/components/UsagePage/hooks/usePaginatedDailyActivity.ts": { + "react-hooks/refs": { + "count": 1 + }, + "react-hooks/set-state-in-effect": { + "count": 1 + } + }, + "src/components/VirtualKeysPage/VirtualKeysTable.tsx": { + "no-restricted-imports": { + "count": 1 + } + }, + "src/components/WebRTCTester.jsx": { + "no-restricted-syntax": { + "count": 2 + }, + "react/no-unescaped-entities": { + "count": 2 + } + }, + "src/components/activity_metrics.tsx": { + "no-restricted-imports": { + "count": 1 + } + }, + "src/components/add_model/AddModelForm.tsx": { + "no-restricted-imports": { + "count": 1 + } + }, + "src/components/add_model/RouterConfigBuilder.tsx": { + "react-hooks/purity": { + "count": 1 + }, + "react-hooks/set-state-in-effect": { + "count": 1 + } + }, + "src/components/add_model/add_auto_router_tab.tsx": { + "no-restricted-imports": { + "count": 1 + } + }, + "src/components/add_model/add_model_tab.tsx": { + "no-restricted-imports": { + "count": 1 + } + }, + "src/components/add_model/advanced_settings.tsx": { + "no-restricted-imports": { + "count": 1 + } + }, + "src/components/add_model/conditional_public_model_name.tsx": { + "no-restricted-imports": { + "count": 1 + }, + "react-hooks/set-state-in-effect": { + "count": 2 + } + }, + "src/components/add_model/litellm_model_name.tsx": { + "no-restricted-imports": { + "count": 1 + } + }, + "src/components/add_model/provider_specific_fields.tsx": { + "no-restricted-imports": { + "count": 1 + }, + "react-hooks/immutability": { + "count": 3 + } + }, + "src/components/add_pass_through.tsx": { + "no-restricted-imports": { + "count": 2 + } + }, + "src/components/agent_management/AgentSelector.test.tsx": { + "react/display-name": { + "count": 1 + }, + "unused-imports/no-unused-imports": { + "count": 1 + } + }, + "src/components/agents.tsx": { + "no-restricted-imports": { + "count": 1 + }, + "react-hooks/set-state-in-effect": { + "count": 2 + } + }, + "src/components/agents/add_agent_form.tsx": { + "no-restricted-imports": { + "count": 1 + }, + "react-hooks/set-state-in-effect": { + "count": 2 + }, + "unused-imports/no-unused-imports": { + "count": 1 + } + }, + "src/components/agents/agent_card_discovery.tsx": { + "react-hooks/refs": { + "count": 3 + }, + "react-hooks/set-state-in-effect": { + "count": 1 + } + }, + "src/components/agents/agent_cost_view.tsx": { + "no-restricted-imports": { + "count": 1 + } + }, + "src/components/agents/agent_info.tsx": { + "no-restricted-imports": { + "count": 1 + }, + "react-hooks/immutability": { + "count": 1 + } + }, + "src/components/agents/agent_table.tsx": { + "no-restricted-imports": { + "count": 1 + } + }, + "src/components/alerting/dynamic_form.tsx": { + "no-restricted-imports": { + "count": 1 + } + }, + "src/components/budgets/budget_modal.tsx": { + "no-restricted-imports": { + "count": 1 + } + }, + "src/components/budgets/budget_panel.test.tsx": { + "unused-imports/no-unused-imports": { + "count": 2 + } + }, + "src/components/budgets/budget_panel.tsx": { + "no-restricted-imports": { + "count": 1 + } + }, + "src/components/budgets/edit_budget_modal.tsx": { + "no-restricted-imports": { + "count": 1 + } + }, + "src/components/bulk_create_users_button.tsx": { + "no-restricted-imports": { + "count": 1 + }, + "react-hooks/set-state-in-effect": { + "count": 1 + } + }, + "src/components/cache_dashboard.tsx": { + "no-restricted-imports": { + "count": 1 + }, + "react-hooks/purity": { + "count": 1 + }, + "react-hooks/set-state-in-effect": { + "count": 2 + } + }, + "src/components/cache_health.tsx": { + "no-restricted-imports": { + "count": 1 + } + }, + "src/components/cache_settings/CacheFieldRenderer.tsx": { + "no-restricted-imports": { + "count": 1 + } + }, + "src/components/cache_settings/RedisTypeSelector.tsx": { + "no-restricted-imports": { + "count": 1 + } + }, + "src/components/cache_settings/index.tsx": { + "no-restricted-imports": { + "count": 1 + }, + "react-hooks/set-state-in-effect": { + "count": 1 + } + }, + "src/components/chat/ChatMessages.tsx": { + "react-hooks/refs": { + "count": 1 + } + }, + "src/components/chat/ChatPage.tsx": { + "max-params": { + "count": 2 + }, + "react-hooks/set-state-in-effect": { + "count": 2 + }, + "unused-imports/no-unused-imports": { + "count": 1 + } + }, + "src/components/chat/ConversationList.tsx": { + "react-hooks/set-state-in-effect": { + "count": 1 + } + }, + "src/components/chat/MCPAppsPanel.tsx": { + "max-nested-callbacks": { + "count": 2 + }, + "react-hooks/set-state-in-effect": { + "count": 2 + } + }, + "src/components/chat/MCPCredentialsTab.tsx": { + "no-restricted-imports": { + "count": 1 + }, + "react-hooks/set-state-in-effect": { + "count": 1 + } + }, + "src/components/chat/useChatHistory.ts": { + "react-hooks/set-state-in-effect": { + "count": 3 + } + }, + "src/components/claude_code_plugins.tsx": { + "no-restricted-imports": { + "count": 1 + }, + "react-hooks/set-state-in-effect": { + "count": 1 + } + }, + "src/components/claude_code_plugins/MakeSkillPublicForm.tsx": { + "no-restricted-imports": { + "count": 1 + }, + "react-hooks/set-state-in-effect": { + "count": 1 + } + }, + "src/components/claude_code_plugins/add_plugin_form.tsx": { + "no-restricted-imports": { + "count": 1 + } + }, + "src/components/claude_code_plugins/helpers.test.ts": { + "unused-imports/no-unused-imports": { + "count": 1 + } + }, + "src/components/claude_code_plugins/plugin_info.tsx": { + "no-restricted-imports": { + "count": 1 + }, + "react-hooks/immutability": { + "count": 1 + } + }, + "src/components/claude_code_plugins/plugin_table.tsx": { + "no-restricted-imports": { + "count": 1 + } + }, + "src/components/cloudzero_export_modal.tsx": { + "no-restricted-imports": { + "count": 1 + }, + "no-restricted-syntax": { + "count": 3 + }, + "react-hooks/immutability": { + "count": 1 + } + }, + "src/components/common_components/AccessGroupSelector.tsx": { + "no-restricted-imports": { + "count": 1 + } + }, + "src/components/common_components/AutoRotationView.tsx": { + "no-restricted-imports": { + "count": 1 + } + }, + "src/components/common_components/DeleteResourceModal.tsx": { + "react-hooks/set-state-in-effect": { + "count": 1 + } + }, + "src/components/common_components/Filters/FilterInput.tsx": { + "react-hooks/set-state-in-effect": { + "count": 1 + } + }, + "src/components/common_components/IconActionButton/BaseActionButton.tsx": { + "no-restricted-imports": { + "count": 1 + } + }, + "src/components/common_components/KeyLifecycleSettings.tsx": { + "no-restricted-imports": { + "count": 1 + } + }, + "src/components/common_components/ModelAliasManager.tsx": { + "no-restricted-imports": { + "count": 1 + }, + "react-hooks/set-state-in-effect": { + "count": 1 + } + }, + "src/components/common_components/ModelSelector.tsx": { + "no-restricted-imports": { + "count": 1 + }, + "react-hooks/set-state-in-effect": { + "count": 1 + } + }, + "src/components/common_components/PassThroughGuardrailsSection.tsx": { + "no-restricted-imports": { + "count": 1 + }, + "react-hooks/set-state-in-effect": { + "count": 1 + } + }, + "src/components/common_components/PassThroughSecuritySection.tsx": { + "no-restricted-imports": { + "count": 1 + } + }, + "src/components/common_components/PremiumLoggingSettings.tsx": { + "no-restricted-imports": { + "count": 1 + } + }, + "src/components/common_components/RouterSettingsAccordion.tsx": { + "no-restricted-imports": { + "count": 1 + } + }, + "src/components/common_components/chartUtils.test.tsx": { + "no-restricted-imports": { + "count": 1 + } + }, + "src/components/common_components/chartUtils.tsx": { + "no-restricted-imports": { + "count": 1 + } + }, + "src/components/common_components/check_openapi_schema.tsx": { + "no-restricted-imports": { + "count": 1 + } + }, + "src/components/common_components/fetch_teams.tsx": { + "max-params": { + "count": 1 + } + }, + "src/components/common_components/simple_table.tsx": { + "no-restricted-imports": { + "count": 1 + } + }, + "src/components/common_components/user_search_modal.tsx": { + "react-hooks/use-memo": { + "count": 1 + } + }, + "src/components/constants.tsx": { + "react-hooks/set-state-in-effect": { + "count": 1 + } + }, + "src/components/edit_auto_router/edit_auto_router_modal.tsx": { + "no-restricted-imports": { + "count": 1 + }, + "react-hooks/immutability": { + "count": 1 + } + }, + "src/components/edit_user.tsx": { + "no-restricted-imports": { + "count": 1 + } + }, + "src/components/email_events/email_event_settings.tsx": { + "no-restricted-imports": { + "count": 1 + }, + "react-hooks/immutability": { + "count": 1 + } + }, + "src/components/email_settings.tsx": { + "no-restricted-imports": { + "count": 1 + } + }, + "src/components/general_settings.tsx": { + "no-restricted-imports": { + "count": 2 + } + }, + "src/components/guardrails.tsx": { + "react-hooks/set-state-in-effect": { + "count": 1 + } + }, + "src/components/guardrails/GuardrailTestPanel.tsx": { + "no-restricted-imports": { + "count": 1 + } + }, + "src/components/guardrails/GuardrailTestResults.tsx": { + "no-restricted-imports": { + "count": 1 + } + }, + "src/components/guardrails/TeamGuardrailsTab.tsx": { + "react-hooks/set-state-in-effect": { + "count": 1 + } + }, + "src/components/guardrails/add_guardrail_form.tsx": { + "react-hooks/set-state-in-effect": { + "count": 1 + }, + "react/no-unescaped-entities": { + "count": 2 + } + }, + "src/components/guardrails/content_filter/CompetitorIntentConfiguration.tsx": { + "react-hooks/set-state-in-effect": { + "count": 1 + } + }, + "src/components/guardrails/content_filter/ContentCategoryConfiguration.tsx": { + "react-hooks/set-state-in-effect": { + "count": 1 + } + }, + "src/components/guardrails/content_filter/ContentFilterDisplay.tsx": { + "no-restricted-imports": { + "count": 1 + } + }, + "src/components/guardrails/content_filter/ContentFilterManager.tsx": { + "max-params": { + "count": 2 + }, + "react-hooks/set-state-in-effect": { + "count": 1 + } + }, + "src/components/guardrails/custom_code/CustomCodeModal.tsx": { + "no-restricted-imports": { + "count": 1 + }, + "react-hooks/set-state-in-effect": { + "count": 1 + } + }, + "src/components/guardrails/edit_guardrail_form.tsx": { + "no-restricted-imports": { + "count": 1 + }, + "no-restricted-syntax": { + "count": 1 + }, + "react-hooks/set-state-in-effect": { + "count": 1 + } + }, + "src/components/guardrails/guardrail_info.tsx": { + "max-params": { + "count": 1 + }, + "no-restricted-imports": { + "count": 1 + }, + "react-hooks/set-state-in-effect": { + "count": 3 + } + }, + "src/components/guardrails/guardrail_optional_params.tsx": { + "react-hooks/set-state-in-effect": { + "count": 1 + } + }, + "src/components/guardrails/guardrail_provider_fields.tsx": { + "react-hooks/set-state-in-effect": { + "count": 1 + } + }, + "src/components/guardrails/guardrail_table.tsx": { + "no-restricted-imports": { + "count": 2 + } + }, + "src/components/guardrails/tool_permission/ToolPermissionRulesEditor.tsx": { + "no-restricted-imports": { + "count": 1 + }, + "react-hooks/purity": { + "count": 1 + } + }, + "src/components/key_team_helpers/filter_logic.tsx": { + "react-hooks/purity": { + "count": 1 + }, + "react-hooks/refs": { + "count": 1 + }, + "react-hooks/set-state-in-effect": { + "count": 3 + }, + "react-hooks/use-memo": { + "count": 1 + } + }, + "src/components/key_team_helpers/key_list.tsx": { + "react-hooks/set-state-in-effect": { + "count": 1 + } + }, + "src/components/key_value_input.tsx": { + "no-restricted-imports": { + "count": 1 + } + }, + "src/components/mcp_hub_table_columns.tsx": { + "no-restricted-imports": { + "count": 1 + } + }, + "src/components/mcp_server_management/MCPToolPermissions.tsx": { + "no-restricted-imports": { + "count": 1 + } + }, + "src/components/mcp_tools/ByokCredentialModal.tsx": { + "no-restricted-syntax": { + "count": 1 + } + }, + "src/components/mcp_tools/MCPLogoSelector.test.tsx": { + "unused-imports/no-unused-imports": { + "count": 1 + } + }, + "src/components/mcp_tools/MCPNetworkSettings.tsx": { + "react-hooks/immutability": { + "count": 2 + } + }, + "src/components/mcp_tools/MCPSubmissionsTab.tsx": { + "react-hooks/set-state-in-effect": { + "count": 1 + } + }, + "src/components/mcp_tools/MCPToolsetsTab.tsx": { + "no-restricted-imports": { + "count": 1 + }, + "react-hooks/set-state-in-effect": { + "count": 1 + }, + "unused-imports/no-unused-imports": { + "count": 2 + } + }, + "src/components/mcp_tools/McpCrudPermissionPanel.tsx": { + "no-restricted-imports": { + "count": 1 + } + }, + "src/components/mcp_tools/OAuthFormFields.tsx": { + "no-restricted-imports": { + "count": 1 + } + }, + "src/components/mcp_tools/OpenAPIQuickPicker.tsx": { + "react-hooks/set-state-in-effect": { + "count": 1 + } + }, + "src/components/mcp_tools/ToolTestPanel.tsx": { + "no-restricted-imports": { + "count": 1 + }, + "react-hooks/set-state-in-effect": { + "count": 1 + } + }, + "src/components/mcp_tools/create_mcp_server.tsx": { + "no-restricted-imports": { + "count": 1 + }, + "react-hooks/set-state-in-effect": { + "count": 5 + } + }, + "src/components/mcp_tools/mcp_connect.tsx": { + "no-restricted-imports": { + "count": 1 + }, + "react-hooks/static-components": { + "count": 4 + } + }, + "src/components/mcp_tools/mcp_connection_status.tsx": { + "no-restricted-imports": { + "count": 1 + } + }, + "src/components/mcp_tools/mcp_discovery.tsx": { + "react-hooks/set-state-in-effect": { + "count": 2 + } + }, + "src/components/mcp_tools/mcp_server_columns.tsx": { + "max-params": { + "count": 1 + }, + "no-restricted-imports": { + "count": 1 + } + }, + "src/components/mcp_tools/mcp_server_cost_config.tsx": { + "no-restricted-imports": { + "count": 1 + } + }, + "src/components/mcp_tools/mcp_server_cost_display.tsx": { + "no-restricted-imports": { + "count": 1 + } + }, + "src/components/mcp_tools/mcp_server_edit.test.tsx": { + "unused-imports/no-unused-imports": { + "count": 1 + } + }, + "src/components/mcp_tools/mcp_server_edit.tsx": { + "no-restricted-imports": { + "count": 1 + }, + "react-hooks/immutability": { + "count": 1 + }, + "react-hooks/set-state-in-effect": { + "count": 5 + } + }, + "src/components/mcp_tools/mcp_server_view.tsx": { + "no-restricted-imports": { + "count": 1 + } + }, + "src/components/mcp_tools/mcp_servers.tsx": { + "no-restricted-imports": { + "count": 1 + }, + "react-hooks/set-state-in-effect": { + "count": 2 + } + }, + "src/components/mcp_tools/mcp_tool_configuration.tsx": { + "no-restricted-imports": { + "count": 1 + } + }, + "src/components/mcp_tools/mcp_tools.tsx": { + "no-restricted-imports": { + "count": 1 + }, + "react-hooks/set-state-in-effect": { + "count": 2 + } + }, + "src/components/model_add/AddCredentialModal.tsx": { + "no-restricted-imports": { + "count": 1 + } + }, + "src/components/model_add/EditCredentialModal.tsx": { + "no-restricted-imports": { + "count": 1 + }, + "react-hooks/set-state-in-effect": { + "count": 1 + } + }, + "src/components/model_add/credentials.tsx": { + "no-restricted-imports": { + "count": 1 + } + }, + "src/components/model_add/reuse_credentials.tsx": { + "no-restricted-imports": { + "count": 1 + } + }, + "src/components/model_dashboard/HealthCheckComponent.tsx": { + "no-restricted-imports": { + "count": 1 + }, + "react-hooks/immutability": { + "count": 1 + } + }, + "src/components/model_dashboard/all_models_table.tsx": { + "no-restricted-imports": { + "count": 1 + } + }, + "src/components/model_dashboard/health_check_columns.tsx": { + "max-params": { + "count": 1 + }, + "no-restricted-imports": { + "count": 1 + } + }, + "src/components/model_dashboard/table.tsx": { + "no-restricted-imports": { + "count": 1 + } + }, + "src/components/model_filters.tsx": { + "no-restricted-imports": { + "count": 1 + } + }, + "src/components/model_group_alias_settings.tsx": { + "no-restricted-imports": { + "count": 1 + }, + "react-hooks/set-state-in-effect": { + "count": 1 + } + }, + "src/components/model_hub_table_columns.tsx": { + "no-restricted-imports": { + "count": 1 + } + }, + "src/components/model_info_view.tsx": { + "no-restricted-imports": { + "count": 1 + }, + "react-hooks/set-state-in-effect": { + "count": 1 + } + }, + "src/components/molecules/filter.tsx": { + "react-hooks/use-memo": { + "count": 1 + } + }, + "src/components/molecules/models/columns.test.tsx": { + "no-restricted-imports": { + "count": 1 + }, + "react/display-name": { + "count": 1 + } + }, + "src/components/molecules/models/columns.tsx": { + "max-params": { + "count": 1 + }, + "no-restricted-imports": { + "count": 1 + } + }, + "src/components/navbar.test.tsx": { + "unused-imports/no-unused-imports": { + "count": 1 + } + }, + "src/components/navbar.tsx": { + "react-hooks/set-state-in-effect": { + "count": 1 + } + }, + "src/components/networking.tsx": { + "max-params": { + "count": 23 + }, + "no-restricted-syntax": { + "count": 270 + } + }, + "src/components/object_permissions_view.tsx": { + "no-restricted-imports": { + "count": 1 + } + }, + "src/components/onboarding_link.tsx": { + "no-restricted-imports": { + "count": 1 + } + }, + "src/components/organisms/RegenerateKeyModal.tsx": { + "react-hooks/set-state-in-effect": { + "count": 1 + } + }, + "src/components/organisms/create_key_button.test.tsx": { + "@typescript-eslint/no-require-imports": { + "count": 2 + }, + "react/display-name": { + "count": 8 + } + }, + "src/components/organisms/create_key_button.tsx": { + "no-restricted-imports": { + "count": 1 + }, + "react-hooks/set-state-in-effect": { + "count": 4 + }, + "react-hooks/use-memo": { + "count": 1 + } + }, + "src/components/organization/organization_view.tsx": { + "no-restricted-imports": { + "count": 1 + }, + "unused-imports/no-unused-imports": { + "count": 1 + } + }, + "src/components/organizations.tsx": { + "no-restricted-imports": { + "count": 1 + } + }, + "src/components/page_utils.test.ts": { + "max-nested-callbacks": { + "count": 3 + } + }, + "src/components/pass_through_info.tsx": { + "no-restricted-imports": { + "count": 1 + } + }, + "src/components/pass_through_settings.tsx": { + "no-restricted-imports": { + "count": 1 + } + }, + "src/components/per_user_usage.tsx": { + "no-restricted-imports": { + "count": 1 + }, + "react-hooks/set-state-in-effect": { + "count": 1 + } + }, + "src/components/permissions/AgentPermissions.tsx": { + "no-restricted-imports": { + "count": 1 + } + }, + "src/components/permissions/MCPServerPermissions.tsx": { + "no-restricted-imports": { + "count": 1 + } + }, + "src/components/permissions/VectorStorePermissions.tsx": { + "no-restricted-imports": { + "count": 1 + } + }, + "src/components/playground/chat_ui/AdditionalModelSettings.tsx": { + "no-restricted-imports": { + "count": 1 + }, + "react-hooks/set-state-in-effect": { + "count": 2 + } + }, + "src/components/playground/chat_ui/AgentBuilderView.tsx": { + "react-hooks/set-state-in-effect": { + "count": 5 + } + }, + "src/components/playground/chat_ui/ChatImageUtils.test.tsx": { + "max-nested-callbacks": { + "count": 1 + } + }, + "src/components/playground/chat_ui/ChatUI.tsx": { + "no-restricted-imports": { + "count": 1 + }, + "react-hooks/set-state-in-effect": { + "count": 4 + }, + "unused-imports/no-unused-imports": { + "count": 13 + } + }, + "src/components/playground/chat_ui/CodeInterpreterOutput.tsx": { + "no-restricted-syntax": { + "count": 2 + } + }, + "src/components/playground/chat_ui/CodeInterpreterTool.tsx": { + "no-restricted-imports": { + "count": 1 + } + }, + "src/components/playground/chat_ui/RealtimePlayground.tsx": { + "react-hooks/immutability": { + "count": 2 + }, + "react-hooks/preserve-manual-memoization": { + "count": 1 + } + }, + "src/components/playground/compareUI/CompareUI.tsx": { + "react-hooks/set-state-in-effect": { + "count": 1 + } + }, + "src/components/playground/compareUI/components/ModelSelector.tsx": { + "no-restricted-imports": { + "count": 1 + } + }, + "src/components/playground/complianceUI/ComplianceUI.tsx": { + "react-hooks/preserve-manual-memoization": { + "count": 3 + } + }, + "src/components/playground/llm_calls/a2a_send_message.tsx": { + "max-params": { + "count": 2 + }, + "no-restricted-syntax": { + "count": 2 + } + }, + "src/components/playground/llm_calls/anthropic_messages.tsx": { + "max-params": { + "count": 1 + } + }, + "src/components/playground/llm_calls/audio_speech.tsx": { + "max-params": { + "count": 1 + } + }, + "src/components/playground/llm_calls/audio_transcriptions.tsx": { + "max-params": { + "count": 1 + } + }, + "src/components/playground/llm_calls/chat_completion.tsx": { + "max-params": { + "count": 1 + } + }, + "src/components/playground/llm_calls/embeddings_api.tsx": { + "max-params": { + "count": 1 + }, + "no-restricted-syntax": { + "count": 1 + } + }, + "src/components/playground/llm_calls/fetch_agents.tsx": { + "no-restricted-syntax": { + "count": 1 + } + }, + "src/components/playground/llm_calls/image_edits.tsx": { + "max-params": { + "count": 1 + } + }, + "src/components/playground/llm_calls/image_generation.tsx": { + "max-params": { + "count": 1 + } + }, + "src/components/playground/llm_calls/interactions_api.tsx": { + "max-params": { + "count": 1 + }, + "no-restricted-syntax": { + "count": 1 + } + }, + "src/components/playground/llm_calls/responses_api.tsx": { + "max-params": { + "count": 1 + } + }, + "src/components/policies/add_attachment_form.tsx": { + "no-restricted-imports": { + "count": 1 + }, + "react-hooks/immutability": { + "count": 1 + } + }, + "src/components/policies/add_policy_form.tsx": { + "no-restricted-imports": { + "count": 1 + }, + "react-hooks/immutability": { + "count": 2 + }, + "react-hooks/set-state-in-effect": { + "count": 1 + } + }, + "src/components/policies/ai_suggestion_modal.tsx": { + "no-restricted-imports": { + "count": 1 + }, + "react-hooks/immutability": { + "count": 1 + } + }, + "src/components/policies/attachment_table.test.tsx": { + "react/display-name": { + "count": 1 + } + }, + "src/components/policies/attachment_table.tsx": { + "no-restricted-imports": { + "count": 1 + } + }, + "src/components/policies/guardrail_selection_modal.tsx": { + "react-hooks/set-state-in-effect": { + "count": 1 + } + }, + "src/components/policies/impact_popover.test.tsx": { + "react/display-name": { + "count": 1 + } + }, + "src/components/policies/impact_popover.tsx": { + "no-restricted-imports": { + "count": 1 + } + }, + "src/components/policies/index.test.tsx": { + "react/display-name": { + "count": 1 + } + }, + "src/components/policies/index.tsx": { + "no-restricted-imports": { + "count": 1 + }, + "react-hooks/set-state-in-effect": { + "count": 1 + } + }, + "src/components/policies/pipeline_flow_builder.tsx": { + "no-restricted-imports": { + "count": 1 + }, + "react-hooks/set-state-in-effect": { + "count": 2 + } + }, + "src/components/policies/policy_info.tsx": { + "no-restricted-imports": { + "count": 1 + }, + "react-hooks/set-state-in-effect": { + "count": 1 + } + }, + "src/components/policies/policy_table.test.tsx": { + "react/display-name": { + "count": 1 + } + }, + "src/components/policies/policy_table.tsx": { + "no-restricted-imports": { + "count": 1 + } + }, + "src/components/policies/policy_test_panel.tsx": { + "no-restricted-imports": { + "count": 1 + }, + "react-hooks/immutability": { + "count": 1 + } + }, + "src/components/policies/template_parameter_modal.tsx": { + "no-restricted-imports": { + "count": 1 + }, + "react-hooks/immutability": { + "count": 1 + }, + "react-hooks/set-state-in-effect": { + "count": 1 + } + }, + "src/components/price_data_reload.tsx": { + "react-hooks/immutability": { + "count": 2 + } + }, + "src/components/prompts.tsx": { + "no-restricted-imports": { + "count": 1 + }, + "react-hooks/set-state-in-effect": { + "count": 1 + } + }, + "src/components/prompts/add_prompt_form.tsx": { + "no-restricted-imports": { + "count": 1 + } + }, + "src/components/prompts/prompt_editor_view/DeveloperMessageCard.tsx": { + "no-restricted-imports": { + "count": 1 + } + }, + "src/components/prompts/prompt_editor_view/ModelConfigCard.tsx": { + "no-restricted-imports": { + "count": 1 + } + }, + "src/components/prompts/prompt_editor_view/PromptCodeSnippets.tsx": { + "no-restricted-imports": { + "count": 1 + }, + "react-hooks/set-state-in-effect": { + "count": 1 + } + }, + "src/components/prompts/prompt_editor_view/PromptEditorHeader.tsx": { + "no-restricted-imports": { + "count": 1 + } + }, + "src/components/prompts/prompt_editor_view/PromptMessagesCard.tsx": { + "no-restricted-imports": { + "count": 1 + } + }, + "src/components/prompts/prompt_editor_view/PublishModal.tsx": { + "no-restricted-imports": { + "count": 1 + } + }, + "src/components/prompts/prompt_editor_view/ToolsCard.tsx": { + "no-restricted-imports": { + "count": 1 + } + }, + "src/components/prompts/prompt_editor_view/VersionHistorySidePanel.test.tsx": { + "max-nested-callbacks": { + "count": 1 + } + }, + "src/components/prompts/prompt_editor_view/VersionHistorySidePanel.tsx": { + "react-hooks/immutability": { + "count": 1 + } + }, + "src/components/prompts/prompt_editor_view/conversation_panel/MessageInput.tsx": { + "no-restricted-imports": { + "count": 1 + } + }, + "src/components/prompts/prompt_editor_view/conversation_panel/index.tsx": { + "no-restricted-imports": { + "count": 1 + } + }, + "src/components/prompts/prompt_editor_view/conversation_panel/useConversation.ts": { + "no-restricted-syntax": { + "count": 1 + } + }, + "src/components/prompts/prompt_info.tsx": { + "no-restricted-imports": { + "count": 1 + }, + "react-hooks/set-state-in-effect": { + "count": 2 + } + }, + "src/components/prompts/prompt_table.tsx": { + "no-restricted-imports": { + "count": 1 + } + }, + "src/components/public_model_hub.tsx": { + "no-restricted-imports": { + "count": 1 + } + }, + "src/components/query_param_input.tsx": { + "no-restricted-imports": { + "count": 1 + } + }, + "src/components/routing_groups/index.tsx": { + "react-hooks/preserve-manual-memoization": { + "count": 1 + } + }, + "src/components/settings.tsx": { + "no-restricted-imports": { + "count": 1 + } + }, + "src/components/shared/advanced_date_picker.tsx": { + "no-restricted-imports": { + "count": 1 + }, + "react-hooks/set-state-in-effect": { + "count": 3 + } + }, + "src/components/shared/numerical_input.tsx": { + "no-restricted-imports": { + "count": 1 + } + }, + "src/components/shared/usage_date_picker.tsx": { + "no-restricted-imports": { + "count": 1 + } + }, + "src/components/skill_hub_table_columns.tsx": { + "no-restricted-imports": { + "count": 1 + } + }, + "src/components/survey/NudgePrompt.tsx": { + "react-hooks/set-state-in-effect": { + "count": 1 + } + }, + "src/components/survey/SurveyModal.tsx": { + "no-restricted-syntax": { + "count": 1 + } + }, + "src/components/tag_management/TagTable.tsx": { + "no-restricted-imports": { + "count": 1 + } + }, + "src/components/tag_management/components/CreateTagModal.tsx": { + "no-restricted-imports": { + "count": 1 + } + }, + "src/components/tag_management/index.tsx": { + "no-restricted-imports": { + "count": 1 + }, + "react-hooks/set-state-in-effect": { + "count": 1 + } + }, + "src/components/tag_management/tag_info.tsx": { + "no-restricted-imports": { + "count": 1 + }, + "react-hooks/set-state-in-effect": { + "count": 1 + } + }, + "src/components/team/EditMembership.tsx": { + "no-restricted-imports": { + "count": 1 + } + }, + "src/components/team/LoggingSettings.tsx": { + "no-restricted-imports": { + "count": 1 + } + }, + "src/components/team/TeamInfo.tsx": { + "no-restricted-imports": { + "count": 1 + }, + "react-hooks/set-state-in-effect": { + "count": 1 + } + }, + "src/components/team/TeamVirtualKeysTable.tsx": { + "no-restricted-imports": { + "count": 1 + } + }, + "src/components/team/available_teams.tsx": { + "no-restricted-imports": { + "count": 1 + } + }, + "src/components/team/member_permissions.tsx": { + "no-restricted-imports": { + "count": 1 + }, + "react-hooks/set-state-in-effect": { + "count": 1 + } + }, + "src/components/team/useMyTeamMember.ts": { + "no-restricted-syntax": { + "count": 1 + } + }, + "src/components/templates/key_edit_view.tsx": { + "no-restricted-imports": { + "count": 1 + } + }, + "src/components/templates/key_info_view.test.tsx": { + "unused-imports/no-unused-imports": { + "count": 2 + } + }, + "src/components/templates/key_info_view.tsx": { + "no-restricted-imports": { + "count": 1 + }, + "react-hooks/set-state-in-effect": { + "count": 1 + } + }, + "src/components/transform_request.tsx": { + "no-restricted-imports": { + "count": 1 + } + }, + "src/components/ui_theme_settings.tsx": { + "no-restricted-imports": { + "count": 1 + }, + "no-restricted-syntax": { + "count": 3 + }, + "react-hooks/immutability": { + "count": 1 + } + }, + "src/components/usage.tsx": { + "no-restricted-imports": { + "count": 2 + }, + "react-hooks/immutability": { + "count": 1 + }, + "react-hooks/purity": { + "count": 1 + } + }, + "src/components/user_agent_activity.tsx": { + "no-restricted-imports": { + "count": 2 + }, + "react-hooks/set-state-in-effect": { + "count": 1 + } + }, + "src/components/user_dashboard.tsx": { + "no-restricted-imports": { + "count": 1 + }, + "react-hooks/set-state-in-effect": { + "count": 2 + } + }, + "src/components/user_edit_view.test.tsx": { + "react/display-name": { + "count": 1 + } + }, + "src/components/user_edit_view.tsx": { + "no-restricted-imports": { + "count": 1 + }, + "react-hooks/set-state-in-effect": { + "count": 1 + } + }, + "src/components/vector_store_management/CreateVectorStore.tsx": { + "no-restricted-imports": { + "count": 1 + } + }, + "src/components/vector_store_management/VectorStoreForm.tsx": { + "no-restricted-imports": { + "count": 1 + }, + "react/no-unescaped-entities": { + "count": 1 + } + }, + "src/components/vector_store_management/VectorStoreTable.tsx": { + "no-restricted-imports": { + "count": 1 + } + }, + "src/components/vector_store_management/index.tsx": { + "no-restricted-imports": { + "count": 1 + }, + "react-hooks/set-state-in-effect": { + "count": 1 + } + }, + "src/components/vector_store_management/vector_store_info.tsx": { + "no-restricted-imports": { + "count": 1 + }, + "react-hooks/set-state-in-effect": { + "count": 1 + } + }, + "src/components/view_logs/GuardrailViewer/CompliancePanel.tsx": { + "react-hooks/set-state-in-effect": { + "count": 1 + } + }, + "src/components/view_logs/LogDetailsDrawer/LogDetailsDrawer.tsx": { + "react-hooks/set-state-in-effect": { + "count": 2 + } + }, + "src/components/view_logs/LogDetailsDrawer/RealtimePrettyView.test.tsx": { + "unused-imports/no-unused-imports": { + "count": 2 + } + }, + "src/components/view_logs/LogDetailsDrawer/useKeyboardNavigation.ts": { + "react-hooks/immutability": { + "count": 2 + } + }, + "src/components/view_logs/columns.tsx": { + "no-restricted-imports": { + "count": 1 + } + }, + "src/components/view_logs/index.tsx": { + "no-restricted-imports": { + "count": 1 + }, + "react-hooks/set-state-in-effect": { + "count": 1 + } + }, + "src/components/view_logs/table.tsx": { + "no-restricted-imports": { + "count": 1 + } + }, + "src/components/view_user_spend.tsx": { + "react-hooks/set-state-in-effect": { + "count": 2 + } + }, + "src/components/view_users.tsx": { + "no-restricted-imports": { + "count": 1 + }, + "react-hooks/set-state-in-effect": { + "count": 1 + } + }, + "src/components/view_users/columns.tsx": { + "max-params": { + "count": 1 + }, + "no-restricted-imports": { + "count": 1 + } + }, + "src/components/view_users/table.tsx": { + "no-restricted-imports": { + "count": 1 + } + }, + "src/components/view_users/user_info_view.tsx": { + "no-restricted-imports": { + "count": 1 + }, + "react-hooks/set-state-in-effect": { + "count": 1 + } + }, + "src/components/workflow_runs/index.tsx": { + "no-restricted-syntax": { + "count": 3 + }, + "react-hooks/set-state-in-effect": { + "count": 1 + } + }, + "src/contexts/AuthContext.tsx": { + "react-hooks/set-state-in-effect": { + "count": 1 + } + }, + "src/contexts/ThemeContext.tsx": { + "no-restricted-syntax": { + "count": 1 + } + }, + "src/data/claimsCompliancePrompts.ts": { + "max-params": { + "count": 1 + } + }, + "src/data/codeExecutionCompliancePrompts.ts": { + "max-params": { + "count": 1 + } + }, + "src/data/compliancePrompts.ts": { + "max-params": { + "count": 1 + } + }, + "src/hooks/useMcpOAuthFlow.tsx": { + "react-hooks/set-state-in-effect": { + "count": 1 + } + }, + "src/hooks/useTestMCPConnection.tsx": { + "react-hooks/set-state-in-effect": { + "count": 1 + } + }, + "src/hooks/useToolsOAuthFlow.tsx": { + "react-hooks/refs": { + "count": 1 + }, + "react-hooks/set-state-in-effect": { + "count": 1 + } + }, + "src/hooks/useUserMcpOAuthFlow.tsx": { + "react-hooks/set-state-in-effect": { + "count": 1 + } + }, + "src/utils/dataUtils.test.ts": { + "max-nested-callbacks": { + "count": 1 + } + }, + "tailwind.config.js": { + "@typescript-eslint/no-require-imports": { + "count": 4 + } + }, + "tailwind.config.ts": { + "@typescript-eslint/no-require-imports": { + "count": 3 + } + }, + "tests/CreateKeyPage.expiredToken.test.tsx": { + "@typescript-eslint/no-require-imports": { + "count": 3 + }, + "react/display-name": { + "count": 1 + } + }, + "tests/setupTests.ts": { + "@typescript-eslint/no-this-alias": { + "count": 1 + }, + "react/display-name": { + "count": 1 + } + } +} \ No newline at end of file diff --git a/ui/litellm-dashboard/eslint.config.mjs b/ui/litellm-dashboard/eslint.config.mjs index 838cd4a069d..50d648feb81 100644 --- a/ui/litellm-dashboard/eslint.config.mjs +++ b/ui/litellm-dashboard/eslint.config.mjs @@ -16,7 +16,7 @@ const eslintConfig = [ plugins: { "unused-imports": unusedImports }, rules: { "unused-imports/no-unused-imports": "error", - "@typescript-eslint/no-explicit-any": "off", + "@typescript-eslint/no-explicit-any": "warn", "@typescript-eslint/no-unused-vars": "off", "@typescript-eslint/no-unused-expressions": "off", "@typescript-eslint/ban-ts-comment": "off", @@ -25,7 +25,31 @@ const eslintConfig = [ "no-prototype-builtins": "off", "no-useless-catch": "off", "no-useless-escape": "off", - "no-self-assign": "off", + "no-self-assign": "error", + "no-var": "error", + "react/no-danger": "error", + complexity: ["warn", 20], + "max-depth": ["warn", 4], + "max-params": ["error", 4], + "max-nested-callbacks": ["error", 4], + "no-restricted-syntax": [ + "error", + { + selector: "CallExpression[callee.name='fetch']", + message: "Use React Query (@tanstack/react-query) for data fetching instead of a raw fetch().", + }, + ], + "no-restricted-imports": [ + "error", + { + patterns: [ + { + group: ["@tremor/react", "@tremor/react/*"], + message: "@tremor/react is being phased out; build new UI with antd instead of adding tremor imports.", + }, + ], + }, + ], }, }, ]; diff --git a/ui/litellm-dashboard/scripts/check-lint-budgets.mjs b/ui/litellm-dashboard/scripts/check-lint-budgets.mjs new file mode 100644 index 00000000000..f6208f012bb --- /dev/null +++ b/ui/litellm-dashboard/scripts/check-lint-budgets.mjs @@ -0,0 +1,30 @@ +import { readFileSync } from "fs"; + +const [, , reportPath, budgetsPath] = process.argv; + +const report = JSON.parse(readFileSync(reportPath, "utf8")); +const budgets = JSON.parse(readFileSync(budgetsPath, "utf8")); + +const counts = {}; +for (const file of report) { + for (const message of file.messages) { + if (message.ruleId in budgets) { + counts[message.ruleId] = (counts[message.ruleId] || 0) + 1; + } + } +} + +let failed = false; +for (const [rule, { max, target }] of Object.entries(budgets)) { + const count = counts[rule] || 0; + const note = count > max ? "OVER BUDGET" : count <= target ? "at target" : `${max - count} of headroom`; + console.log(`${rule}: ${count} | max: ${max} | target: ${target} | ${note}`); + if (count > max) { + console.error( + `::error::${rule} budget exceeded (${count} > ${max}). Reduce usage; lower max in eslint-budgets.json as the count drops.`, + ); + failed = true; + } +} + +process.exit(failed ? 1 : 0); From 216c68db049d8d06a8fefc35aa4976f4dff8789f Mon Sep 17 00:00:00 2001 From: Sameer Kankute Date: Thu, 4 Jun 2026 20:13:30 +0530 Subject: [PATCH 2/4] fix(gemini): googleSearch + server-side tools and googleMaps JSON schema (#29582) * fix(gemini): keep googleSearch with server-side tools and googleMaps JSON schema Wire include_server_side_tool_invocations through completion() so mixed google_search and function tools are not dropped on Gemini 3+. Rewrite generationConfig to responseFormat when googleMaps is used with JSON schema. Fixes #27479 Fixes #29451 Co-authored-by: Cursor * address greptile review feedback (greploop iteration 1) * style: fix black formatting in main.py for py312 compat * Fix Gemini Google Maps extra_body JSON rewrite --------- Co-authored-by: Cursor --- litellm/constants.py | 2 + litellm/llms/gemini/chat/transformation.py | 1 + .../llms/vertex_ai/gemini/transformation.py | 56 ++++++++++++++ .../vertex_and_google_ai_studio_gemini.py | 23 ++++++ litellm/main.py | 8 ++ litellm/types/llms/vertex_ai.py | 1 + .../vertex_ai/gemini/test_transformation.py | 32 ++++++++ .../test_vertex_ai_gemini_transformation.py | 74 ++++++++++++++++++ ...test_vertex_and_google_ai_studio_gemini.py | 77 +++++++++++++++++++ 9 files changed, 274 insertions(+) diff --git a/litellm/constants.py b/litellm/constants.py index 26e25d0cef3..36e578bd323 100644 --- a/litellm/constants.py +++ b/litellm/constants.py @@ -678,6 +678,7 @@ OPENAI_CHAT_COMPLETION_PARAMS = [ "extra_headers", "thinking", "web_search_options", + "include_server_side_tool_invocations", "service_tier", "prompt_cache_key", "prompt_cache_retention", @@ -739,6 +740,7 @@ DEFAULT_CHAT_COMPLETION_PARAM_VALUES = { "verbosity": None, "thinking": None, "web_search_options": None, + "include_server_side_tool_invocations": None, "service_tier": None, "safety_identifier": None, "prompt_cache_key": None, diff --git a/litellm/llms/gemini/chat/transformation.py b/litellm/llms/gemini/chat/transformation.py index b69b7e1913e..4e9764446c9 100644 --- a/litellm/llms/gemini/chat/transformation.py +++ b/litellm/llms/gemini/chat/transformation.py @@ -93,6 +93,7 @@ class GoogleAIStudioGeminiConfig(VertexGeminiConfig): "modalities", "parallel_tool_calls", "web_search_options", + "include_server_side_tool_invocations", "service_tier", ] if supports_reasoning(model, custom_llm_provider="gemini"): diff --git a/litellm/llms/vertex_ai/gemini/transformation.py b/litellm/llms/vertex_ai/gemini/transformation.py index ef7bf82bfae..c578d6cd28b 100644 --- a/litellm/llms/vertex_ai/gemini/transformation.py +++ b/litellm/llms/vertex_ai/gemini/transformation.py @@ -1121,6 +1121,61 @@ def _pop_and_merge_extra_body(data: RequestBody, optional_params: dict) -> None: data_dict[k] = v +def _has_google_maps_tool(tools: Optional[Any]) -> bool: + """Return True if any tool object in the list has a 'googleMaps' key.""" + if not isinstance(tools, list): + return False + return any( + isinstance(t, dict) and VertexToolName.GOOGLE_MAPS.value in t for t in tools + ) + + +def _rewrite_mime_type_to_response_format(generation_config: GenerationConfig) -> None: + """ + Convert response_mime_type + response_json_schema/response_schema to the newer + responseFormat structure when googleMaps is present in tools. + + The Gemini API rejects the combination of googleMaps + response_mime_type: + 'application/json' with the error: + "Google Maps tool with a response mime type: 'application/json' is unsupported" + + The newer responseFormat field supports this combination on both the Gemini API + (generativelanguage.googleapis.com) and Vertex AI endpoints. + + Before: + generationConfig: { + response_mime_type: "application/json", + response_json_schema: {...} + } + + After: + generationConfig: { + responseFormat: { + "text": {"mimeType": "APPLICATION_JSON", "schema": {...}} + } + } + """ + schema = generation_config.pop("response_json_schema", None) # type: ignore[misc] + if schema is None: + schema = generation_config.pop("response_schema", None) # type: ignore[misc] + generation_config.pop("response_mime_type", None) # type: ignore[misc] + + response_format: Dict[str, Any] = {"text": {"mimeType": "APPLICATION_JSON"}} + if schema is not None: + response_format["text"]["schema"] = schema + generation_config["responseFormat"] = response_format # type: ignore[typeddict-unknown-key] + + +def _rewrite_google_maps_response_format(data: RequestBody) -> None: + generation_config = cast(Optional[GenerationConfig], data.get("generationConfig")) + if ( + isinstance(generation_config, dict) + and _has_google_maps_tool(data.get("tools")) + and generation_config.get("response_mime_type") == "application/json" + ): + _rewrite_mime_type_to_response_format(generation_config) + + def _transform_request_body( # noqa: PLR0915 messages: List[AllMessageValues], model: str, @@ -1246,6 +1301,7 @@ def _transform_request_body( # noqa: PLR0915 if labels and custom_llm_provider != LlmProviders.GEMINI: data["labels"] = labels _pop_and_merge_extra_body(data, optional_params) + _rewrite_google_maps_response_format(data) except Exception as e: raise e diff --git a/litellm/llms/vertex_ai/gemini/vertex_and_google_ai_studio_gemini.py b/litellm/llms/vertex_ai/gemini/vertex_and_google_ai_studio_gemini.py index 189ac7a7f6a..5cd02293f14 100644 --- a/litellm/llms/vertex_ai/gemini/vertex_and_google_ai_studio_gemini.py +++ b/litellm/llms/vertex_ai/gemini/vertex_and_google_ai_studio_gemini.py @@ -1147,6 +1147,26 @@ class VertexGeminiConfig(VertexAIBaseConfig, BaseConfig): return cast(dict, speech_config) + @staticmethod + def _apply_include_server_side_tool_invocations( + non_default_params: Dict, + optional_params: Dict, + ) -> None: + """ + Set include_server_side_tool_invocations before tools are mapped. + + map_openai_params iterates non_default_params in request order; if tools + appear before this flag, _resolve_search_tool_conflict would drop search + tools before the flag is applied. + """ + for key in ( + "include_server_side_tool_invocations", + "includeServerSideToolInvocations", + ): + if non_default_params.get(key) is True or optional_params.get(key) is True: + optional_params["include_server_side_tool_invocations"] = True + return + def map_openai_params( # noqa: PLR0915 self, non_default_params: Dict, @@ -1154,6 +1174,9 @@ class VertexGeminiConfig(VertexAIBaseConfig, BaseConfig): model: str, drop_params: bool, ) -> Dict: + self._apply_include_server_side_tool_invocations( + non_default_params, optional_params + ) gemini_sampling_params_warned: bool = False for param, value in non_default_params.items(): if param == "temperature": diff --git a/litellm/main.py b/litellm/main.py index da8624d11b8..96f81381c86 100644 --- a/litellm/main.py +++ b/litellm/main.py @@ -437,6 +437,7 @@ async def acompletion( # noqa: PLR0915 # Optional liteLLM function params thinking: Optional[AnthropicThinkingParam] = None, web_search_options: Optional[OpenAIWebSearchOptions] = None, + include_server_side_tool_invocations: Optional[bool] = None, # Session management shared_session: Optional["ClientSession"] = None, # Per-request JSON schema validation (overrides litellm.enable_json_schema_validation) @@ -584,6 +585,7 @@ async def acompletion( # noqa: PLR0915 "acompletion": True, # assuming this is a required parameter "thinking": thinking, "web_search_options": web_search_options, + "include_server_side_tool_invocations": include_server_side_tool_invocations, "shared_session": shared_session, "enable_json_schema_validation": enable_json_schema_validation, } @@ -1116,6 +1118,7 @@ def completion( # type: ignore # noqa: PLR0915 top_logprobs: Optional[int] = None, parallel_tool_calls: Optional[bool] = None, web_search_options: Optional[OpenAIWebSearchOptions] = None, + include_server_side_tool_invocations: Optional[bool] = None, deployment_id=None, extra_headers: Optional[dict] = None, safety_identifier: Optional[str] = None, @@ -1550,6 +1553,11 @@ def completion( # type: ignore # noqa: PLR0915 "reasoning_effort": reasoning_effort, "thinking": thinking, "web_search_options": web_search_options, + "include_server_side_tool_invocations": ( + include_server_side_tool_invocations + if include_server_side_tool_invocations is not None + else kwargs.get("include_server_side_tool_invocations") + ), "safety_identifier": safety_identifier, "service_tier": service_tier, "allowed_openai_params": kwargs.get("allowed_openai_params"), diff --git a/litellm/types/llms/vertex_ai.py b/litellm/types/llms/vertex_ai.py index b972ff3c538..51429d0769e 100644 --- a/litellm/types/llms/vertex_ai.py +++ b/litellm/types/llms/vertex_ai.py @@ -246,6 +246,7 @@ class GenerationConfig(TypedDict, total=False): response_mime_type: Literal["text/plain", "application/json"] response_schema: dict response_json_schema: dict + responseFormat: dict seed: int responseLogprobs: bool logprobs: int diff --git a/tests/litellm/llms/vertex_ai/gemini/test_transformation.py b/tests/litellm/llms/vertex_ai/gemini/test_transformation.py index 963e2d273a7..756923c5df6 100644 --- a/tests/litellm/llms/vertex_ai/gemini/test_transformation.py +++ b/tests/litellm/llms/vertex_ai/gemini/test_transformation.py @@ -246,6 +246,38 @@ async def test__transform_request_body_image_config_with_image_size(): assert rb["generationConfig"]["imageConfig"]["imageSize"] == "4K" +def test__transform_request_body_google_maps_json_schema_uses_response_format(): + """googleMaps + JSON schema must use responseFormat, not response_mime_type.""" + messages = [{"role": "user", "content": "Find restaurants in Mumbai"}] + schema = { + "type": "object", + "properties": {"places": {"type": "array"}}, + "required": ["places"], + } + optional_params = { + "tools": [{"googleMaps": {}}], + "response_mime_type": "application/json", + "response_json_schema": schema, + } + transform_request_params = { + "messages": messages, + "model": "gemini/gemini-3.1-flash-lite", + "optional_params": optional_params, + "custom_llm_provider": "gemini", + "litellm_params": {}, + "cached_content": None, + } + + rb: RequestBody = transformation._transform_request_body(**transform_request_params) + + gen = rb["generationConfig"] + assert "responseFormat" in gen + assert gen["responseFormat"]["text"]["mimeType"] == "APPLICATION_JSON" + assert gen["responseFormat"]["text"]["schema"] == schema + assert "response_mime_type" not in gen + assert "response_json_schema" not in gen + + def test_map_function_google_search_snake_case(): """ Test that google_search tool (snake_case) is properly mapped to googleSearch. diff --git a/tests/test_litellm/llms/vertex_ai/gemini/test_vertex_ai_gemini_transformation.py b/tests/test_litellm/llms/vertex_ai/gemini/test_vertex_ai_gemini_transformation.py index 628a6ed4cba..d99c190c6e5 100644 --- a/tests/test_litellm/llms/vertex_ai/gemini/test_vertex_ai_gemini_transformation.py +++ b/tests/test_litellm/llms/vertex_ai/gemini/test_vertex_ai_gemini_transformation.py @@ -285,6 +285,80 @@ def test_extra_body_tags_not_forwarded_to_vertex_ai(): assert result["custom_param"] == "allowed" +def test_extra_body_google_maps_rewrites_json_response_format(): + messages = [{"role": "user", "content": "test"}] + optional_params = { + "response_mime_type": "application/json", + "response_schema": { + "type": "object", + "properties": {"answer": {"type": "string"}}, + }, + "extra_body": { + "tools": [{"googleMaps": {}}], + }, + } + + result = _transform_request_body( + messages=messages, + model="gemini-2.5-pro", + optional_params=optional_params, + custom_llm_provider="vertex_ai", + litellm_params={}, + cached_content=None, + ) + + generation_config = result["generationConfig"] + assert "response_mime_type" not in generation_config + assert generation_config["responseFormat"] == { + "text": { + "mimeType": "APPLICATION_JSON", + "schema": { + "type": "object", + "properties": {"answer": {"type": "string"}}, + }, + } + } + + +def test_extra_body_generation_config_cannot_restore_google_maps_json_mime_type(): + messages = [{"role": "user", "content": "test"}] + optional_params = { + "tools": [{"googleMaps": {}}], + "response_mime_type": "application/json", + "extra_body": { + "generationConfig": { + "response_mime_type": "application/json", + "response_json_schema": { + "type": "object", + "properties": {"answer": {"type": "string"}}, + }, + }, + }, + } + + result = _transform_request_body( + messages=messages, + model="gemini-2.5-pro", + optional_params=optional_params, + custom_llm_provider="vertex_ai", + litellm_params={}, + cached_content=None, + ) + + generation_config = result["generationConfig"] + assert "response_mime_type" not in generation_config + assert "response_json_schema" not in generation_config + assert generation_config["responseFormat"] == { + "text": { + "mimeType": "APPLICATION_JSON", + "schema": { + "type": "object", + "properties": {"answer": {"type": "string"}}, + }, + } + } + + def test_metadata_to_labels_vertex_only(): """Test that metadata->labels conversion only happens for Vertex AI""" messages = [{"role": "user", "content": "test"}] diff --git a/tests/test_litellm/llms/vertex_ai/gemini/test_vertex_and_google_ai_studio_gemini.py b/tests/test_litellm/llms/vertex_ai/gemini/test_vertex_and_google_ai_studio_gemini.py index 45b9f4293fa..0d02521433a 100644 --- a/tests/test_litellm/llms/vertex_ai/gemini/test_vertex_and_google_ai_studio_gemini.py +++ b/tests/test_litellm/llms/vertex_ai/gemini/test_vertex_and_google_ai_studio_gemini.py @@ -3078,6 +3078,83 @@ def test_vertex_ai_gemini3_tool_combination_no_drop(): assert len(tools) == 3 +def test_get_optional_params_keeps_google_search_with_server_side_flag(): + """ + include_server_side_tool_invocations must be in non_default_params before + map_openai_params runs (not only via add_provider_specific_params after). + """ + from litellm.utils import get_optional_params + + optional_params = get_optional_params( + model="gemini-3.1-pro-preview", + custom_llm_provider="gemini", + tools=[ + {"google_search": {}}, + { + "type": "function", + "function": { + "name": "send_message", + "description": "Send a message back", + "parameters": { + "type": "object", + "properties": {"message": {"type": "string"}}, + "required": ["message"], + }, + }, + }, + ], + include_server_side_tool_invocations=True, + ) + + assert optional_params.get("include_server_side_tool_invocations") is True + tool_keys = set() + for tool in optional_params.get("tools", []): + tool_keys.update(tool.keys()) + assert "function_declarations" in tool_keys + assert "googleSearch" in tool_keys + + +def test_map_openai_params_tools_before_include_server_side_flag(): + """ + Request bodies often list tools before include_server_side_tool_invocations. + Search tools must not be dropped when the flag is present later in the dict. + """ + v = VertexGeminiConfig() + optional_params: dict = {} + non_default_params = { + "tools": [ + {"google_search": {}}, + { + "type": "function", + "function": { + "name": "send_message", + "description": "Send a message back", + "parameters": { + "type": "object", + "properties": {"message": {"type": "string"}}, + "required": ["message"], + }, + }, + }, + ], + "include_server_side_tool_invocations": True, + } + + result = v.map_openai_params( + non_default_params=non_default_params, + optional_params=optional_params, + model="gemini-3.1-pro-preview", + drop_params=True, + ) + + assert result.get("include_server_side_tool_invocations") is True + tool_keys = set() + for tool in result.get("tools", []): + tool_keys.update(tool.keys()) + assert "function_declarations" in tool_keys + assert "googleSearch" in tool_keys + + def test_vertex_ai_mixed_tools_and_web_search_options_drops_search(): """ When function tools and web_search_options are sent separately (Codex-style), From 20dc6dffa413535fcdc9ecb913cd6f020f67c3ad Mon Sep 17 00:00:00 2001 From: Sameer Kankute Date: Thu, 4 Jun 2026 20:14:51 +0530 Subject: [PATCH 3/4] fix(proxy): passthrough 404 when SERVER_ROOT_PATH is set (#29658) * fix(proxy): match passthrough registry routes bare-to-bare with SERVER_ROOT_PATH After #28547, get_request_route strips the deployment prefix while registry lookup still re-inflated stored paths via SERVER_ROOT_PATH, causing 404s under paths like /llmproxy/ml. Compare normalized bare routes in both is_registered_pass_through_route and get_registered_pass_through_route. Co-authored-by: Cursor * test(proxy): patch utils.get_server_root_path in passthrough auth tests After removing get_server_root_path from pass_through_endpoints, route and JWT tests must mock litellm.proxy.utils where normalization reads it. Co-authored-by: Cursor --------- Co-authored-by: Cursor --- .../pass_through_endpoints.py | 49 ++--- .../proxy/auth/test_handle_jwt.py | 2 +- .../proxy/auth/test_route_checks.py | 14 +- .../test_pass_through_endpoints.py | 190 +++++++----------- 4 files changed, 108 insertions(+), 147 deletions(-) diff --git a/litellm/proxy/pass_through_endpoints/pass_through_endpoints.py b/litellm/proxy/pass_through_endpoints/pass_through_endpoints.py index e49e1302ab5..6667010447b 100644 --- a/litellm/proxy/pass_through_endpoints/pass_through_endpoints.py +++ b/litellm/proxy/pass_through_endpoints/pass_through_endpoints.py @@ -59,7 +59,7 @@ from litellm.proxy.common_utils.http_parsing_utils import ( _safe_get_request_headers, ) from litellm.proxy.litellm_pre_call_utils import LiteLLMProxyRequestSetup -from litellm.proxy.utils import get_server_root_path, normalize_route_for_root_path +from litellm.proxy.utils import normalize_route_for_root_path from litellm.secret_managers.main import get_secret_str from litellm.types.llms.custom_http import httpxSpecialProvider from litellm.types.passthrough_endpoints.pass_through_endpoints import ( @@ -2499,20 +2499,16 @@ class InitPassThroughEndpointHelpers: return list(_registered_pass_through_routes.keys()) @staticmethod - def _build_full_path_with_root(path: str) -> str: + def _route_for_registry_lookup(route: str) -> str: """ - Build full path by prepending server root path if needed. + Normalize an incoming route to the bare path stored in the registry. - Args: - path: The relative path to build - - Returns: - Full path with server root prepended (if root is not "/") + Registry keys store root-stripped paths. Callers should pass routes from + ``get_request_route()`` (already stripped); prefixed ``request.url.path`` + values are stripped via ``normalize_route_for_root_path``. """ - root_path = get_server_root_path() - if root_path == "/": - return path - return f"{root_path}{path}" + normalized_route = normalize_route_for_root_path(route) + return normalized_route if normalized_route is not None else route @staticmethod def is_registered_pass_through_route(route: str) -> bool: @@ -2535,6 +2531,10 @@ class InitPassThroughEndpointHelpers: if normalized_route.startswith(mapped_route): return True + comparison_route = InitPassThroughEndpointHelpers._route_for_registry_lookup( + route + ) + # Fast path: check if any registered route key contains this path # Keys are in format: "{endpoint_id}:exact:{path}:{methods}" or "{endpoint_id}:subpath:{path}:{methods}" # For backward compatibility, also support old format: "{endpoint_id}:exact:{path}" or "{endpoint_id}:subpath:{path}" @@ -2543,14 +2543,13 @@ class InitPassThroughEndpointHelpers: parts = key.split(":", 3) # Split into [endpoint_id, type, path, methods?] if len(parts) >= 3: route_type = parts[1] - registered_path = ( - InitPassThroughEndpointHelpers._build_full_path_with_root(parts[2]) - ) - if route_type == "exact" and route == registered_path: + registered_path = parts[2] + if route_type == "exact" and comparison_route == registered_path: return True elif route_type == "subpath": - if route == registered_path or route.startswith( - registered_path + "/" + if ( + comparison_route == registered_path + or comparison_route.startswith(registered_path + "/") ): return True @@ -2561,13 +2560,14 @@ class InitPassThroughEndpointHelpers: route: str, method: Optional[str] = None ) -> Optional[Dict[str, Any]]: """Get passthrough params for a given route and optionally filter by HTTP method""" + comparison_route = InitPassThroughEndpointHelpers._route_for_registry_lookup( + route + ) for key in _registered_pass_through_routes.keys(): parts = key.split(":", 3) # Split into [endpoint_id, type, path, methods?] if len(parts) >= 3: route_type = parts[1] - registered_path = ( - InitPassThroughEndpointHelpers._build_full_path_with_root(parts[2]) - ) + registered_path = parts[2] # Get the methods for this route. Prefer the registered metadata, # but keep supporting test fixtures / older registry entries that @@ -2581,11 +2581,12 @@ class InitPassThroughEndpointHelpers: # Check if path matches path_matches = False - if route_type == "exact" and route == registered_path: + if route_type == "exact" and comparison_route == registered_path: path_matches = True elif route_type == "subpath": - if route == registered_path or route.startswith( - registered_path + "/" + if ( + comparison_route == registered_path + or comparison_route.startswith(registered_path + "/") ): path_matches = True diff --git a/tests/test_litellm/proxy/auth/test_handle_jwt.py b/tests/test_litellm/proxy/auth/test_handle_jwt.py index 41fc417d979..92bd7915152 100644 --- a/tests/test_litellm/proxy/auth/test_handle_jwt.py +++ b/tests/test_litellm/proxy/auth/test_handle_jwt.py @@ -233,7 +233,7 @@ async def test_find_team_with_model_access_uses_request_method_for_passthrough_a mock_registered_routes, ), patch( - "litellm.proxy.pass_through_endpoints.pass_through_endpoints.get_server_root_path", + "litellm.proxy.utils.get_server_root_path", return_value="/", ), ): diff --git a/tests/test_litellm/proxy/auth/test_route_checks.py b/tests/test_litellm/proxy/auth/test_route_checks.py index 197572216d0..63b61954cf6 100644 --- a/tests/test_litellm/proxy/auth/test_route_checks.py +++ b/tests/test_litellm/proxy/auth/test_route_checks.py @@ -733,7 +733,7 @@ def test_virtual_key_llm_api_routes_allows_registered_pass_through_endpoints(): mock_registered_routes, ), patch( - "litellm.proxy.pass_through_endpoints.pass_through_endpoints.get_server_root_path", + "litellm.proxy.utils.get_server_root_path", return_value="/", ), ): @@ -799,7 +799,7 @@ def test_virtual_key_llm_api_routes_allows_non_auth_enforced_pass_through_endpoi mock_registered_routes, ), patch( - "litellm.proxy.pass_through_endpoints.pass_through_endpoints.get_server_root_path", + "litellm.proxy.utils.get_server_root_path", return_value="/", ), ): @@ -849,7 +849,7 @@ def test_virtual_key_llm_api_routes_denies_auth_pass_through_without_allowlist() mock_registered_routes, ), patch( - "litellm.proxy.pass_through_endpoints.pass_through_endpoints.get_server_root_path", + "litellm.proxy.utils.get_server_root_path", return_value="/", ), ): @@ -893,7 +893,7 @@ def test_virtual_key_llm_api_routes_uses_method_specific_auth_setting(): mock_registered_routes, ), patch( - "litellm.proxy.pass_through_endpoints.pass_through_endpoints.get_server_root_path", + "litellm.proxy.utils.get_server_root_path", return_value="/", ), ): @@ -948,7 +948,7 @@ def test_non_proxy_admin_denies_auth_pass_through_without_allowlist(): mock_registered_routes, ), patch( - "litellm.proxy.pass_through_endpoints.pass_through_endpoints.get_server_root_path", + "litellm.proxy.utils.get_server_root_path", return_value="/", ), ): @@ -987,7 +987,7 @@ def test_non_proxy_admin_allows_auth_pass_through_with_team_allowlist(): mock_registered_routes, ), patch( - "litellm.proxy.pass_through_endpoints.pass_through_endpoints.get_server_root_path", + "litellm.proxy.utils.get_server_root_path", return_value="/", ), ): @@ -1021,7 +1021,7 @@ def test_virtual_key_without_llm_api_routes_cannot_access_pass_through(): mock_registered_routes, ), patch( - "litellm.proxy.pass_through_endpoints.pass_through_endpoints.get_server_root_path", + "litellm.proxy.utils.get_server_root_path", return_value="/", ), ): diff --git a/tests/test_litellm/proxy/pass_through_endpoints/test_pass_through_endpoints.py b/tests/test_litellm/proxy/pass_through_endpoints/test_pass_through_endpoints.py index 89c57bcced6..9b9d5e22a43 100644 --- a/tests/test_litellm/proxy/pass_through_endpoints/test_pass_through_endpoints.py +++ b/tests/test_litellm/proxy/pass_through_endpoints/test_pass_through_endpoints.py @@ -1386,10 +1386,6 @@ async def test_create_pass_through_endpoint_auth_true_enforces_allowlist(): "litellm.proxy.pass_through_endpoints.pass_through_endpoints._registered_pass_through_routes", registry, ), - patch( - "litellm.proxy.pass_through_endpoints.pass_through_endpoints.get_server_root_path", - return_value="/", - ), ): mock_get_config.return_value = ConfigFieldInfo( field_name="pass_through_endpoints", field_value=[] @@ -1485,10 +1481,6 @@ async def test_update_pass_through_endpoint_auth_true_enforces_allowlist(): "litellm.proxy.pass_through_endpoints.pass_through_endpoints._registered_pass_through_routes", registry, ), - patch( - "litellm.proxy.pass_through_endpoints.pass_through_endpoints.get_server_root_path", - return_value="/", - ), ): mock_get_config.return_value = ConfigFieldInfo( field_name="pass_through_endpoints", field_value=existing_endpoints @@ -1570,10 +1562,6 @@ async def test_update_pass_through_endpoint_preserves_auth_false(): "litellm.proxy.pass_through_endpoints.pass_through_endpoints._registered_pass_through_routes", registry, ), - patch( - "litellm.proxy.pass_through_endpoints.pass_through_endpoints.get_server_root_path", - return_value="/", - ), ): mock_get_config.return_value = ConfigFieldInfo( field_name="pass_through_endpoints", field_value=existing_endpoints @@ -2866,70 +2854,10 @@ async def test_create_pass_through_route_no_custom_body_falls_back(): assert call_kwargs["custom_body"] == request_parsed_body -def test_build_full_path_with_root_default(): - """ - Test _build_full_path_with_root with default root path (/) - """ - from litellm.proxy.pass_through_endpoints.pass_through_endpoints import ( - InitPassThroughEndpointHelpers, - ) - - with patch( - "litellm.proxy.pass_through_endpoints.pass_through_endpoints.get_server_root_path" - ) as mock_get_root: - # Test with default root path - mock_get_root.return_value = "/" - - result = InitPassThroughEndpointHelpers._build_full_path_with_root( - "/api/v1/endpoint" - ) - assert result == "/api/v1/endpoint" - - -def test_build_full_path_with_root_custom(): - """ - Test _build_full_path_with_root with custom root path - """ - from litellm.proxy.pass_through_endpoints.pass_through_endpoints import ( - InitPassThroughEndpointHelpers, - ) - - with patch( - "litellm.proxy.pass_through_endpoints.pass_through_endpoints.get_server_root_path" - ) as mock_get_root: - # Test with custom root path /proxy - mock_get_root.return_value = "/proxy" - - result = InitPassThroughEndpointHelpers._build_full_path_with_root( - "/api/v1/endpoint" - ) - assert result == "/proxy/api/v1/endpoint" - - -def test_build_full_path_with_root_nested(): - """ - Test _build_full_path_with_root with nested root path - """ - from litellm.proxy.pass_through_endpoints.pass_through_endpoints import ( - InitPassThroughEndpointHelpers, - ) - - with patch( - "litellm.proxy.pass_through_endpoints.pass_through_endpoints.get_server_root_path" - ) as mock_get_root: - # Test with nested root path /api/v2 - mock_get_root.return_value = "/api/v2" - - result = InitPassThroughEndpointHelpers._build_full_path_with_root("/endpoint") - assert result == "/api/v2/endpoint" - - def test_is_registered_pass_through_route_with_custom_root(): """ - Test is_registered_pass_through_route correctly handles server root path - - When server has a custom root path like /proxy, the registered path - should be constructed by prepending the root to match incoming routes. + Registry stores bare paths; incoming routes may be bare (get_request_route) + or prefixed (request.url.path). Both should resolve via normalization. """ from litellm.proxy.pass_through_endpoints.pass_through_endpoints import ( InitPassThroughEndpointHelpers, @@ -2948,32 +2876,13 @@ def test_is_registered_pass_through_route_with_custom_root(): "headers": {}, } - with patch( - "litellm.proxy.pass_through_endpoints.pass_through_endpoints.get_server_root_path" - ) as mock_get_root: - # Test with custom root path /proxy - mock_get_root.return_value = "/proxy" - - # Should match when request route includes the root path + with patch("litellm.proxy.utils.get_server_root_path", return_value="/proxy"): assert ( InitPassThroughEndpointHelpers.is_registered_pass_through_route( "/proxy/api/endpoint" ) is True ) - - # Should not match when request route doesn't include root path - assert ( - InitPassThroughEndpointHelpers.is_registered_pass_through_route( - "/api/endpoint" - ) - is False - ) - - # Test with default root path - mock_get_root.return_value = "/" - - # Should match with default root assert ( InitPassThroughEndpointHelpers.is_registered_pass_through_route( "/api/endpoint" @@ -2981,7 +2890,13 @@ def test_is_registered_pass_through_route_with_custom_root(): is True ) - # Should not match with root prepended when root is / + with patch("litellm.proxy.utils.get_server_root_path", return_value="/"): + assert ( + InitPassThroughEndpointHelpers.is_registered_pass_through_route( + "/api/endpoint" + ) + is True + ) assert ( InitPassThroughEndpointHelpers.is_registered_pass_through_route( "/proxy/api/endpoint" @@ -2995,10 +2910,8 @@ def test_is_registered_pass_through_route_with_custom_root(): def test_get_registered_pass_through_route_with_custom_root(): """ - Test get_registered_pass_through_route correctly handles server root path - - When server has a custom root path, the method should return the correct - endpoint configuration by matching the full path including the root. + get_registered_pass_through_route matches bare registry paths against + bare or SERVER_ROOT_PATH-prefixed incoming routes. """ from litellm.proxy.pass_through_endpoints.pass_through_endpoints import ( InitPassThroughEndpointHelpers, @@ -3019,13 +2932,8 @@ def test_get_registered_pass_through_route_with_custom_root(): route_key = f"{endpoint_id}:exact:{path}" _registered_pass_through_routes[route_key] = target_config - with patch( - "litellm.proxy.pass_through_endpoints.pass_through_endpoints.get_server_root_path" - ) as mock_get_root: - # Test with custom root path /litellm - mock_get_root.return_value = "/litellm" - - # Should return config when request route includes root path + with patch("litellm.proxy.utils.get_server_root_path", return_value="/litellm"): + # Prefixed incoming route result = InitPassThroughEndpointHelpers.get_registered_pass_through_route( "/litellm/chat/completions" ) @@ -3033,16 +2941,14 @@ def test_get_registered_pass_through_route_with_custom_root(): assert result["target"] == "http://api.example.com/v1/chat/completions" assert result["headers"]["Authorization"] == "Bearer token123" - # Should return None when route doesn't match + # Bare incoming route (get_request_route convention) result = InitPassThroughEndpointHelpers.get_registered_pass_through_route( "/chat/completions" ) - assert result is None + assert result is not None + assert result["target"] == "http://api.example.com/v1/chat/completions" - # Test with default root path - mock_get_root.return_value = "/" - - # Should return config with default root + with patch("litellm.proxy.utils.get_server_root_path", return_value="/"): result = InitPassThroughEndpointHelpers.get_registered_pass_through_route( "/chat/completions" ) @@ -3053,6 +2959,62 @@ def test_get_registered_pass_through_route_with_custom_root(): _registered_pass_through_routes.clear() +@pytest.mark.parametrize( + "server_root_path,route_type,incoming_route,should_match", + [ + ("", "subpath", "/ml/api/v1/time-series-forecast/predict", True), + ("", "exact", "/ml", True), + ("", "exact", "/ml/extra", False), + ("/llmproxy", "subpath", "/ml/api/v1/time-series-forecast/predict", True), + ( + "/llmproxy", + "subpath", + "/llmproxy/ml/api/v1/time-series-forecast/predict", + True, + ), + ("/llmproxy", "exact", "/ml", True), + ("/llmproxy", "exact", "/llmproxy/ml", True), + ("/llmproxy", "subpath", "/other/api", False), + ], +) +def test_db_registered_pass_through_route_bare_path_convention( + server_root_path, route_type, incoming_route, should_match +): + """ + Regression: #28547 / SERVER_ROOT_PATH — registry stores bare /ml paths; + get_request_route() supplies bare paths; prefixed url.path must still match. + """ + from litellm.proxy.pass_through_endpoints.pass_through_endpoints import ( + InitPassThroughEndpointHelpers, + _registered_pass_through_routes, + ) + + _registered_pass_through_routes.clear() + endpoint_id = "customer-ml" + path = "/ml" + route_key = f"{endpoint_id}:{route_type}:{path}:GET,POST" + _registered_pass_through_routes[route_key] = { + "endpoint_id": endpoint_id, + "path": path, + "type": route_type, + "target": "https://example.com", + "methods": ["GET", "POST"], + } + + with patch( + "litellm.proxy.utils.get_server_root_path", + return_value=server_root_path, + ): + assert ( + InitPassThroughEndpointHelpers.is_registered_pass_through_route( + incoming_route + ) + is should_match + ) + + _registered_pass_through_routes.clear() + + def test_mapped_pass_through_routes_with_server_root_path(): """ Mapped passthrough routes (vertex_ai, bedrock, etc) should match @@ -3064,9 +3026,7 @@ def test_mapped_pass_through_routes_with_server_root_path(): InitPassThroughEndpointHelpers, ) - with patch("litellm.proxy.utils.get_server_root_path") as mock_get_root: - mock_get_root.return_value = "/litellm" - + with patch("litellm.proxy.utils.get_server_root_path", return_value="/litellm"): # prefixed route should match mapped routes like /vertex_ai assert ( InitPassThroughEndpointHelpers.is_registered_pass_through_route( From ed073d382d75b42f62f600ce47a5fe0ffade70fa Mon Sep 17 00:00:00 2001 From: Sameer Kankute Date: Thu, 4 Jun 2026 20:33:24 +0530 Subject: [PATCH 4/4] fix(gemini-realtime): use GA event names for Pipecat 1.3.x compatibility (#29662) * fix(gemini-realtime): use GA event names for Pipecat 1.3.x compatibility Pipecat v1.3.0 adopted the OpenAI Realtime API GA event naming: response.audio.delta -> response.output_audio.delta response.text.delta -> response.output_text.delta response.audio.done -> response.output_audio.done response.text.done -> response.output_text.done The proxy was still emitting the old beta names; Pipecat's `parse_server_event` raises "Unimplemented server event type" for any unknown type, which killed the receive task handler and broke audio playback and tool-call delivery. Also: - conversation.item.created -> conversation.item.added (already handled) - client audio is buffered until backend setupComplete in deferred mode - call_id fallback UUID when Gemini returns empty id - status_details / token detail fields added to Pydantic-strict events The _GA_TO_BETA_EVENT_TYPES map in RealTimeStreaming already translates GA names back to beta for clients that opt in with the openai-beta header, so legacy clients are unaffected. Co-authored-by: Cursor * fix(gemini-realtime): address greptile review comments - emit outputTranscription as response.output_audio_transcript.delta instead of suppressing it; GA_TO_BETA map handles translation for legacy clients - cap pre-setup audio buffer at 200 frames to prevent memory exhaustion; log a warning when the limit is hit and additional frames are dropped - log remaining dropped message count on flush error Co-authored-by: Cursor * fix(gemini-realtime): address veria review comments - remove unused OpenAIRealtimeConversationItemCreated import - fix guardrail bypass: semantic_vad early-return now preserves create_response when set so a guardrail-injected create_response:false is not silently dropped - add per-connection 10 MB byte cap alongside the 200-frame count cap for the pre-setup audio buffer to prevent memory exhaustion Co-authored-by: Cursor * fix(gemini-realtime): fix mypy arg-type on _finalize_gemini_live_setup setup parameter typed as BidiGenerateContentSetup to match the TypedDict passed at both call sites; was dict which mypy rejected. Co-authored-by: Cursor * fix(gemini-realtime): widen _finalize_gemini_live_setup to Dict[str, Any] BidiGenerateContentSetup (TypedDict) is a subtype of Dict[str,Any] so both call sites (one passing a plain dict, one passing the TypedDict) satisfy mypy. Co-authored-by: Cursor * fix(gemini-realtime): cast BidiGenerateContentSetup to Dict at _finalize call site mypy rejects TypedDict as dict[str, Any] argument; cast at the call site where follow_up_setup is BidiGenerateContentSetup to satisfy the checker. Co-authored-by: Cursor * Fix Gemini realtime beta compatibility * Fix deferred Gemini setup audio ordering * fix: preserve Gemini audio transcript ids * fix(realtime): cap pre-setup client buffer on all append paths Route every append to the deferred-setup pending buffer through the per-connection message/byte caps. Previously only the audio-buffer fast path enforced the caps; once one frame was buffered, a client that withheld session.update could stream arbitrary frames into _pending_messages_until_setup unbounded and exhaust proxy memory. * style(gemini-realtime): apply black formatting to transformation.py * fix(gemini-realtime): log beta-translation fallback and name native-audio marker Surface the previously swallowed exception in _send_event_to_client so a failed GA->beta translation is observable instead of silently forwarding the untranslated event. Extract the native-audio model substring used by _finalize_gemini_live_setup into a named constant documenting why speechConfig is dropped on those setups. --------- Co-authored-by: Cursor Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com> --- .../litellm_core_utils/realtime_streaming.py | 142 +++++++- .../llms/gemini/realtime/transformation.py | 229 +++++++++--- .../test_realtime_streaming.py | 340 ++++++++++++++++++ .../test_gemini_realtime_transformation.py | 193 +++++++++- .../test_vertex_ai_realtime_transformation.py | 4 +- 5 files changed, 848 insertions(+), 60 deletions(-) diff --git a/litellm/litellm_core_utils/realtime_streaming.py b/litellm/litellm_core_utils/realtime_streaming.py index 33bb6d7d2ea..772f058d9bb 100644 --- a/litellm/litellm_core_utils/realtime_streaming.py +++ b/litellm/litellm_core_utils/realtime_streaming.py @@ -92,8 +92,27 @@ class RealTimeStreaming: # Track whether we have already sent the guardrail turn-detection update # that disables provider auto-response for transcription guardrails. self._guardrail_turn_detection_update_sent: bool = False + # Deferred Gemini Live setup: Pipecat may stream audio before session.update. + # Buffer client audio until the backend acknowledges setup (setupComplete). + self._backend_setup_complete: bool = ( + provider_config is None or provider_config.requires_session_configuration() + ) + self._flushing_pending_messages_until_setup: bool = False + self._pending_messages_until_setup: List[str] = [] + self._pending_messages_byte_total: int = 0 + + # Per-connection caps for pre-setup audio frames (message count + total bytes). + _MAX_BUFFERED_MESSAGES: int = 200 + _MAX_BUFFERED_BYTES: int = 10 * 1024 * 1024 # 10 MB _SESSION_EVENT_TYPES = frozenset(["session.created", "session.updated"]) + _CLIENT_AUDIO_BUFFER_TYPES = frozenset( + [ + "input_audio_buffer.append", + "input_audio_buffer.commit", + "input_audio_buffer.clear", + ] + ) _AUDIO_FORMAT_MAP: Dict[str, Dict[str, Any]] = { "pcm16": {"type": "audio/pcm", "rate": 24000}, "g711_ulaw": {"type": "audio/G711-ulaw", "rate": 8000}, @@ -285,6 +304,86 @@ class RealTimeStreaming: await self.backend_ws.send(message) # type: ignore[union-attr, attr-defined] return True + def _uses_deferred_backend_setup(self) -> bool: + """True when setup is deferred until the client's first session.update.""" + if self.provider_config is None: + return False + return not self.provider_config.requires_session_configuration() + + def _should_buffer_client_message_until_setup(self, message: str) -> bool: + if not self._uses_deferred_backend_setup(): + return False + if ( + self._backend_setup_complete + and not self._flushing_pending_messages_until_setup + ): + return False + try: + msg_obj = json.loads(message) + except (json.JSONDecodeError, TypeError): + return False + return msg_obj.get("type") in RealTimeStreaming._CLIENT_AUDIO_BUFFER_TYPES + + def _buffer_pending_message_until_setup(self, message: str) -> None: + msg_bytes = len(message.encode("utf-8")) + if ( + len(self._pending_messages_until_setup) + < RealTimeStreaming._MAX_BUFFERED_MESSAGES + and self._pending_messages_byte_total + msg_bytes + <= RealTimeStreaming._MAX_BUFFERED_BYTES + ): + self._pending_messages_until_setup.append(message) + self._pending_messages_byte_total += msg_bytes + else: + verbose_logger.warning( + "Pre-setup buffer full (%d messages / %d bytes); dropping frame", + len(self._pending_messages_until_setup), + self._pending_messages_byte_total, + ) + + async def _flush_pending_messages_until_setup(self) -> bool: + pending = self._pending_messages_until_setup + self._pending_messages_until_setup = [] + self._pending_messages_byte_total = 0 + for idx, message in enumerate(pending): + try: + await self._send_to_backend(message) + except Exception as e: + unsent = pending[idx:] + self._pending_messages_until_setup = ( + unsent + self._pending_messages_until_setup + ) + self._pending_messages_byte_total = sum( + len(msg.encode("utf-8")) + for msg in self._pending_messages_until_setup + ) + verbose_logger.debug( + "Failed to flush buffered client message after setup: %s " + "(%d buffered message(s) retained)", + e, + len(unsent), + ) + return False + return True + + async def _send_event_to_client(self, event: Any, event_str: str) -> bool: + if self._client_wants_beta and isinstance(event, dict): + try: + translated = self._translate_event_to_beta(event) + if translated is None: + return False + await self.websocket.send_text(json.dumps(translated)) + return True + except Exception as e: + verbose_logger.warning( + "Failed to translate %s to beta protocol, forwarding " + "untranslated event to client: %s", + event.get("type"), + e, + ) + await self.websocket.send_text(event_str) + return True + def _cache_session_configuration_request(self, transformed_message: str) -> None: """Store setup payload once sent to backend. @@ -547,6 +646,19 @@ class RealTimeStreaming: isinstance(event, dict) and event.get("type") == "session.created" ) if is_session_created_event: + if ( + self._uses_deferred_backend_setup() + and not self._backend_setup_complete + ): + self._backend_setup_complete = True + self._flushing_pending_messages_until_setup = True + try: + while self._pending_messages_until_setup: + flushed = await self._flush_pending_messages_until_setup() + if not flushed: + break + finally: + self._flushing_pending_messages_until_setup = False if self._session_created_sent_to_client: # A synthetic session.created (with placeholder defaults) was # already forwarded to the client when we connected. The @@ -569,7 +681,7 @@ class RealTimeStreaming: ## update if a prior attempt was dropped by the provider transform. if is_session_created_event and self._has_audio_transcription_guardrails(): self.store_message(event_str) - await self.websocket.send_text(event_str) + await self._send_event_to_client(event, event_str) await self._maybe_send_guardrail_turn_detection_update() continue ## GUARDRAIL: run on transcription events in provider_config path too @@ -581,7 +693,7 @@ class RealTimeStreaming: transcript = event.get("transcript", "") self._collect_user_input_from_backend_event(cast(dict, event)) self.store_message(event_str) - await self.websocket.send_text(event_str) + await self._send_event_to_client(event, event_str) blocked = await self.run_realtime_guardrails( cast(str, transcript), item_id=cast(Optional[str], event.get("item_id")), @@ -591,7 +703,7 @@ class RealTimeStreaming: continue ## LOGGING self.store_message(event_str) - await self.websocket.send_text(event_str) + await self._send_event_to_client(event, event_str) async def _handle_raw_backend_message(self, raw_response) -> bool: """Process a backend message without provider_config (raw path). @@ -880,6 +992,7 @@ class RealTimeStreaming: ## GUARDRAIL: intercept conversation.item.create for text-based injection. guardrail_turn_detection_injected = False + msg_type: Optional[str] = None try: msg_obj = json.loads(message) msg_type = msg_obj.get("type") @@ -1081,6 +1194,29 @@ class RealTimeStreaming: # actually forward to the backend. self.store_input(message=message) + if self._should_buffer_client_message_until_setup(message): + self._buffer_pending_message_until_setup(message) + continue + + if self._pending_messages_until_setup: + should_send_setup_before_buffered_messages = ( + not self._backend_setup_complete + and not self._flushing_pending_messages_until_setup + and msg_type == "session.update" + ) + if not should_send_setup_before_buffered_messages: + self._buffer_pending_message_until_setup(message) + if ( + self._backend_setup_complete + and not self._flushing_pending_messages_until_setup + ): + await self._flush_pending_messages_until_setup() + continue + + if self._flushing_pending_messages_until_setup: + self._buffer_pending_message_until_setup(message) + continue + ## FORWARD TO BACKEND # Only mark the guardrail turn_detection update as sent after the # backend actually accepted the message. Setting the flag earlier diff --git a/litellm/llms/gemini/realtime/transformation.py b/litellm/llms/gemini/realtime/transformation.py index cf1fc75ef10..212287fb7f8 100644 --- a/litellm/llms/gemini/realtime/transformation.py +++ b/litellm/llms/gemini/realtime/transformation.py @@ -27,7 +27,6 @@ from litellm.types.llms.gemini import ( ) from litellm.types.llms.openai import ( OpenAIRealtimeContentPartDone, - OpenAIRealtimeConversationItemCreated, OpenAIRealtimeDoneEvent, OpenAIRealtimeEvents, OpenAIRealtimeEventTypes, @@ -79,6 +78,12 @@ _KNOWN_GEMINI_TOP_LEVEL_KEYS: set = { map_key.split(".", 1)[0] for map_key in MAP_GEMINI_FIELD_TO_OPENAI_EVENT } +# Gemini Live native-audio model ids carry this marker (e.g. +# ``gemini-2.5-flash-native-audio-preview-09-2025``). These models reject a +# ``speechConfig`` on ``setup`` with a 1007 invalid-argument error, so it is +# stripped in ``_finalize_gemini_live_setup``. +_GEMINI_NATIVE_AUDIO_MODEL_MARKER = "native-audio" + class GeminiRealtimeConfig(BaseRealtimeConfig): # Cap the LRU of in-flight tool calls so long sessions with many tool @@ -98,6 +103,33 @@ class GeminiRealtimeConfig(BaseRealtimeConfig): # bypassing spend and budget accounting. self._pending_usage_metadata: Optional[dict] = None + @staticmethod + def _usage_detail_alias(details: Any, defaults: Dict[str, int]) -> Dict[str, Any]: + if not isinstance(details, dict): + return dict(defaults) + return { + **defaults, + **{key: value for key, value in details.items() if value is not None}, + } + + @staticmethod + def _add_pipecat_usage_detail_aliases(usage_dict: Dict[str, Any]) -> Dict[str, Any]: + usage_dict.setdefault( + "input_token_details", + GeminiRealtimeConfig._usage_detail_alias( + usage_dict.get("input_tokens_details"), + {"cached_tokens": 0, "text_tokens": 0, "audio_tokens": 0}, + ), + ) + usage_dict.setdefault( + "output_token_details", + GeminiRealtimeConfig._usage_detail_alias( + usage_dict.get("output_tokens_details"), + {"text_tokens": 0, "audio_tokens": 0}, + ), + ) + return usage_dict + def validate_environment( self, headers: dict, model: str, api_key: Optional[str] = None ) -> dict: @@ -173,9 +205,25 @@ class GeminiRealtimeConfig(BaseRealtimeConfig): def map_automatic_turn_detection( self, value: OpenAIRealtimeTurnDetection ) -> AutomaticActivityDetection: + """Map OpenAI ``server_vad`` to Gemini ``automaticActivityDetection``. + + OpenAI ``semantic_vad`` has no Gemini Live equivalent — return an empty + dict so callers omit ``realtimeInputConfig`` (mapping it with + ``disabled: true`` breaks native-audio sessions). + """ + if ( + isinstance(value, dict) + and value.get("type") == "semantic_vad" + and "create_response" not in value + ): + return AutomaticActivityDetection() + automatic_activity_dection = AutomaticActivityDetection() if "create_response" in value and isinstance(value["create_response"], bool): automatic_activity_dection["disabled"] = not value["create_response"] + elif isinstance(value, dict) and value.get("type") == "server_vad": + # OpenAI server VAD enables activity detection by default. + automatic_activity_dection["disabled"] = False else: automatic_activity_dection["disabled"] = True if "prefix_padding_ms" in value and isinstance(value["prefix_padding_ms"], int): @@ -197,6 +245,7 @@ class GeminiRealtimeConfig(BaseRealtimeConfig): "tools", "input_audio_transcription", "turn_detection", + "voice", ] def map_openai_params( @@ -231,17 +280,33 @@ class GeminiRealtimeConfig(BaseRealtimeConfig): optional_params["inputAudioTranscription"] = {} elif key == "turn_detection": value_typed = cast(OpenAIRealtimeTurnDetection, value) + if ( + isinstance(value_typed, dict) + and value_typed.get("type") == "semantic_vad" + and "create_response" not in value_typed + ): + # Pipecat/OpenAI GA semantic VAD — skip; Gemini uses its own VAD. + # Only skip when there is no create_response override so that + # a guardrail-injected create_response:false is not dropped. + continue transformed_audio_activity_config = self.map_automatic_turn_detection( value_typed ) - if ( - len(transformed_audio_activity_config) > 0 - ): # if the config is not empty, add it to the optional params + if transformed_audio_activity_config: optional_params["realtimeInputConfig"] = ( BidiGenerateContentRealtimeInputConfig( automaticActivityDetection=transformed_audio_activity_config ) ) + elif key == "voice": + from litellm.llms.vertex_ai.gemini.vertex_and_google_ai_studio_gemini import ( + VertexGeminiConfig, + ) + + vertex_gemini_config = VertexGeminiConfig() + speech_config = vertex_gemini_config._map_audio_params({"voice": value}) + if speech_config: + optional_params["generationConfig"]["speechConfig"] = speech_config if len(optional_params["generationConfig"]) == 0: optional_params.pop("generationConfig") return optional_params @@ -297,6 +362,9 @@ class GeminiRealtimeConfig(BaseRealtimeConfig): and "transcription" in input_cfg ): normalized["input_audio_transcription"] = input_cfg["transcription"] + output_cfg = audio.get("output") + if isinstance(output_cfg, dict) and output_cfg.get("voice"): + normalized["voice"] = output_cfg["voice"] extracted_turn_detection = GeminiRealtimeConfig._extract_turn_detection( normalized @@ -308,6 +376,18 @@ class GeminiRealtimeConfig(BaseRealtimeConfig): return normalized + @staticmethod + def _finalize_gemini_live_setup( + model: str, setup: Dict[str, Any] + ) -> Dict[str, Any]: + """Drop fields Gemini Live native-audio rejects on ``setup``.""" + if _GEMINI_NATIVE_AUDIO_MODEL_MARKER not in model.lower(): + return setup + generation_config = setup.get("generationConfig") + if isinstance(generation_config, dict): + generation_config.pop("speechConfig", None) + return setup + def _handle_session_update( self, json_message: dict, @@ -351,7 +431,11 @@ class GeminiRealtimeConfig(BaseRealtimeConfig): verbose_logger.debug( "Gemini Realtime: Sending initial setup with tools to backend" ) - return [json.dumps({"setup": new_overrides})] + return [ + json.dumps( + {"setup": self._finalize_gemini_live_setup(model, new_overrides)} + ) + ] if not new_overrides: verbose_logger.debug( @@ -420,7 +504,15 @@ class GeminiRealtimeConfig(BaseRealtimeConfig): verbose_logger.debug( "Gemini Realtime: Forwarding session.update as follow-up setup" ) - return [json.dumps({"setup": follow_up_setup})] + return [ + json.dumps( + { + "setup": self._finalize_gemini_live_setup( + model, cast(Dict[str, Any], follow_up_setup) + ) + } + ) + ] def _handle_conversation_item(self, json_message: dict) -> List[str]: """ @@ -666,6 +758,7 @@ class GeminiRealtimeConfig(BaseRealtimeConfig): "object": "realtime.response", "id": response_id, "status": "in_progress", + "status_details": None, "output": [], "conversation_id": conversation_id, "modalities": _modalities, @@ -675,9 +768,10 @@ class GeminiRealtimeConfig(BaseRealtimeConfig): ) response_items.append(response_created) - ## - return response.output_item.added ← adds ‘item_id’ same for all subsequent events + ## - return response.output_item.added response_output_item_added = OpenAIRealtimeStreamResponseOutputItemAdded( type="response.output_item.added", + event_id="event_{}".format(uuid.uuid4()), response_id=response_id, output_index=0, item={ @@ -690,20 +784,28 @@ class GeminiRealtimeConfig(BaseRealtimeConfig): }, ) response_items.append(response_output_item_added) - ## - return conversation.item.created - conversation_item_created = OpenAIRealtimeConversationItemCreated( - type="conversation.item.created", - event_id="event_{}".format(uuid.uuid4()), - item={ - "id": output_item_id, - "object": "realtime.item", - "type": "message", - "status": "in_progress", - "role": "assistant", - "content": [], - }, + ## - return conversation.item.added + # Pipecat 1.3.x handles "conversation.item.added" (not ".created"). + # Sending ".created" raises "Unimplemented server event type" which + # kills the receive task handler. + response_items.append( + cast( + OpenAIRealtimeEvents, + { + "type": "conversation.item.added", + "event_id": "event_{}".format(uuid.uuid4()), + "previous_item_id": None, + "item": { + "id": output_item_id, + "object": "realtime.item", + "type": "message", + "status": "in_progress", + "role": "assistant", + "content": [], + }, + }, + ) ) - response_items.append(conversation_item_created) ## - return response.content_part.added response_content_part_added = OpenAIRealtimeResponseContentPartAdded( type="response.content_part.added", @@ -749,9 +851,9 @@ class GeminiRealtimeConfig(BaseRealtimeConfig): return OpenAIRealtimeResponseDelta( type=( - "response.text.delta" + "response.output_text.delta" if delta_type == "text" - else "response.audio.delta" + else "response.output_audio.delta" ), content_index=0, event_id="event_{}".format(uuid.uuid4()), @@ -778,7 +880,7 @@ class GeminiRealtimeConfig(BaseRealtimeConfig): current_response_id = "resp_{}".format(uuid.uuid4()) if delta_type == "text": return OpenAIRealtimeResponseTextDone( - type="response.text.done", + type="response.output_text.done", content_index=0, event_id="event_{}".format(uuid.uuid4()), item_id=current_output_item_id, @@ -788,7 +890,7 @@ class GeminiRealtimeConfig(BaseRealtimeConfig): ) elif delta_type == "audio": return OpenAIRealtimeResponseAudioDone( - type="response.audio.done", + type="response.output_audio.done", content_index=0, event_id="event_{}".format(uuid.uuid4()), item_id=current_output_item_id, @@ -914,7 +1016,7 @@ class GeminiRealtimeConfig(BaseRealtimeConfig): events: List[OpenAIRealtimeFunctionCallArgumentsDone] = [] for idx, fc in enumerate(function_calls): - call_id = fc.get("id", "") + call_id = fc.get("id", "") or f"call_{uuid.uuid4().hex[:16]}" name = fc.get("name", "") # Store call_id → name mapping for round-trip. Use an LRU so @@ -962,7 +1064,7 @@ class GeminiRealtimeConfig(BaseRealtimeConfig): current_delta_chunks = [] any_delta_chunk = False for event in transformed_message: - if event["type"] == "response.text.delta": + if event["type"] == "response.output_text.delta": current_delta_chunks.append( cast(OpenAIRealtimeResponseDelta, event) ) @@ -973,7 +1075,7 @@ class GeminiRealtimeConfig(BaseRealtimeConfig): ) else: if ( - transformed_message["type"] == "response.text.delta" + transformed_message["type"] == "response.output_text.delta" ): # ONLY ACCUMULATE TEXT DELTA CHUNKS - AUDIO WILL CAUSE SERVER MEMORY ISSUES if current_delta_chunks is None: current_delta_chunks = [] @@ -1067,6 +1169,8 @@ class GeminiRealtimeConfig(BaseRealtimeConfig): responses_api_usage = LiteLLMCompletionResponsesConfig._transform_chat_completion_usage_to_responses_usage( _chat_completion_usage, ) + _usage_dict = responses_api_usage.model_dump() + self._add_pipecat_usage_detail_aliases(_usage_dict) response_done_event = OpenAIRealtimeDoneEvent( type="response.done", event_id="event_{}".format(uuid.uuid4()), @@ -1074,6 +1178,7 @@ class GeminiRealtimeConfig(BaseRealtimeConfig): object="realtime.response", id=current_response_id, status="completed", + status_details=None, # type: ignore[typeddict-item] output=( [output_item["item"] for output_item in output_items] if output_items @@ -1081,7 +1186,7 @@ class GeminiRealtimeConfig(BaseRealtimeConfig): ), conversation_id=current_conversation_id, modalities=_modalities, - usage=responses_api_usage.model_dump(), + usage=_usage_dict, ), ) if temperature is not None: @@ -1294,19 +1399,36 @@ class GeminiRealtimeConfig(BaseRealtimeConfig): output_tx = server_content.get("outputTranscription") if isinstance(output_tx, dict) and output_tx.get("text"): + if current_response_id is None: + current_response_id = "resp_{}".format(uuid.uuid4()) + if current_output_item_id is None: + current_output_item_id = "item_{}".format(uuid.uuid4()) + current_conversation_id = ( + current_conversation_id or "conv_{}".format(uuid.uuid4()) + ) + returned_message.extend( + self.return_new_content_delta_events( + session_configuration_request=session_configuration_request, + response_id=current_response_id, + output_item_id=current_output_item_id, + conversation_id=current_conversation_id, + delta_type="audio", + ) + ) + # Emit as the GA event name; _GA_TO_BETA_EVENT_TYPES translates + # this back to response.audio_transcript.delta for beta clients. returned_message.append( cast( OpenAIRealtimeEvents, { - "type": "response.audio_transcript.delta", + "type": "response.output_audio_transcript.delta", "event_id": "event_{}".format(uuid.uuid4()), - "delta": output_tx["text"], - "item_id": current_output_item_id - or "item_{}".format(uuid.uuid4()), - "response_id": current_response_id - or "resp_{}".format(uuid.uuid4()), - "output_index": 0, + "transcript": output_tx["text"], + "item_id": current_output_item_id, "content_index": 0, + "output_index": 0, + "response_id": current_response_id, + "delta": output_tx["text"], }, ) ) @@ -1416,6 +1538,7 @@ class GeminiRealtimeConfig(BaseRealtimeConfig): "object": "realtime.response", "id": current_response_id, "status": "in_progress", + "status_details": None, "output": [], "conversation_id": current_conversation_id, "modalities": tool_call_modalities, @@ -1460,6 +1583,29 @@ class GeminiRealtimeConfig(BaseRealtimeConfig): }, ) ) + # conversation.item.added — Pipecat 1.3.x registers the + # call_id into _pending_function_calls inside + # _handle_evt_conversation_item_added, which is triggered + # by this event (NOT by response.output_item.added and NOT + # by the old conversation.item.created which Pipecat 1.3.x + # does not handle). Without this event the subsequent + # response.function_call_arguments.done finds an empty + # pending-calls dict and drops the tool invocation silently. + returned_message.append( + cast( + OpenAIRealtimeEvents, + { + "type": "conversation.item.added", + "event_id": f"event_{uuid.uuid4()}", + "previous_item_id": None, + "item": { + **function_call_item, + "status": "in_progress", + "arguments": "", + }, + }, + ) + ) # response.function_call_arguments.delta — Gemini delivers # the full arguments string in a single toolCall frame # rather than streaming partial chunks, so emit one delta @@ -1496,14 +1642,6 @@ class GeminiRealtimeConfig(BaseRealtimeConfig): item={**function_call_item}, ) ) - # conversation.item.created - returned_message.append( - OpenAIRealtimeConversationItemCreated( - type="conversation.item.created", - event_id=f"event_{uuid.uuid4()}", - item={**function_call_item}, - ) - ) # response.done - close the response so clients can submit tool # results. Mirror the non-tool-call RESPONSE_DONE path: if Gemini @@ -1537,6 +1675,8 @@ class GeminiRealtimeConfig(BaseRealtimeConfig): tool_call_responses_api_usage = LiteLLMCompletionResponsesConfig._transform_chat_completion_usage_to_responses_usage( _tool_call_chat_completion_usage, ) + _tool_usage_dict = tool_call_responses_api_usage.model_dump() + self._add_pipecat_usage_detail_aliases(_tool_usage_dict) tool_call_done_event = OpenAIRealtimeDoneEvent( type="response.done", event_id=f"event_{uuid.uuid4()}", @@ -1544,6 +1684,7 @@ class GeminiRealtimeConfig(BaseRealtimeConfig): id=current_response_id, object="realtime.response", status="completed", + status_details=None, # type: ignore[typeddict-item] output=[ { "id": te["item_id"], @@ -1558,7 +1699,7 @@ class GeminiRealtimeConfig(BaseRealtimeConfig): ], conversation_id=current_conversation_id, modalities=tool_call_modalities, - usage=tool_call_responses_api_usage.model_dump(), + usage=_tool_usage_dict, ), ) tool_call_temperature = tool_call_generation_config.get("temperature") diff --git a/tests/test_litellm/litellm_core_utils/test_realtime_streaming.py b/tests/test_litellm/litellm_core_utils/test_realtime_streaming.py index 7913efe8294..3424bfd801c 100644 --- a/tests/test_litellm/litellm_core_utils/test_realtime_streaming.py +++ b/tests/test_litellm/litellm_core_utils/test_realtime_streaming.py @@ -293,6 +293,51 @@ def test_translate_event_to_beta_drops_conversation_item_done(): ) +@pytest.mark.asyncio +async def test_provider_config_path_translates_ga_events_for_beta_clients(): + client_ws = MagicMock() + client_ws.scope = {"headers": [(b"openai-beta", b"realtime=v1")]} + client_ws.send_text = AsyncMock() + backend_ws = MagicMock() + logging_obj = MagicMock() + + provider_config = MagicMock() + provider_config.transform_realtime_response = MagicMock( + return_value={ + "response": [ + { + "type": "response.output_text.delta", + "event_id": "event_1", + "delta": "hello", + }, + {"type": "conversation.item.done", "event_id": "event_2"}, + ], + "current_output_item_id": None, + "current_response_id": None, + "current_delta_chunks": [], + "current_conversation_id": None, + "current_item_chunks": [], + "current_delta_type": None, + "session_configuration_request": None, + } + ) + + streaming = RealTimeStreaming( + client_ws, + backend_ws, + logging_obj, + provider_config=provider_config, + model="gemini-2.5-flash", + ) + + await streaming._handle_provider_config_message("{}") + + assert client_ws.send_text.await_count == 1 + sent = json.loads(client_ws.send_text.await_args.args[0]) + assert sent["type"] == "response.text.delta" + assert sent["delta"] == "hello" + + def test_client_sent_openai_beta_realtime_header_detects_header(): ws = MagicMock() ws.scope = {"headers": [(b"openai-beta", b"realtime=v1")]} @@ -1770,3 +1815,298 @@ async def test_follow_up_setup_updates_cached_session_configuration_request(): await streaming.client_ack_messages() assert streaming.session_configuration_request == follow_up_setup + + +@pytest.mark.asyncio +async def test_deferred_setup_buffers_audio_until_backend_setup_complete(monkeypatch): + """Pipecat may send audio before session.update when setup is deferred.""" + monkeypatch.setattr(litellm, "gemini_live_defer_setup", True, raising=False) + from litellm.llms.gemini.realtime.transformation import GeminiRealtimeConfig + + client_ws = MagicMock() + audio_msg = json.dumps({"type": "input_audio_buffer.append", "audio": "AA=="}) + client_ws.receive_text = AsyncMock( + side_effect=[audio_msg, ConnectionClosed(None, None)] + ) + backend_ws = MagicMock() + backend_ws.send = AsyncMock() + logging_obj = MagicMock() + + config = GeminiRealtimeConfig() + streaming = RealTimeStreaming( + client_ws, + backend_ws, + logging_obj, + provider_config=config, + model="gemini-live-2.5-flash-native-audio", + ) + assert streaming._backend_setup_complete is False + + await streaming.client_ack_messages() + + backend_ws.send.assert_not_called() + assert len(streaming._pending_messages_until_setup) == 1 + + streaming._backend_setup_complete = True + await streaming._flush_pending_messages_until_setup() + + assert backend_ws.send.call_count == 1 + + +@pytest.mark.asyncio +async def test_deferred_setup_sends_session_update_before_buffered_audio(monkeypatch): + monkeypatch.setattr(litellm, "gemini_live_defer_setup", True, raising=False) + from litellm.llms.gemini.realtime.transformation import GeminiRealtimeConfig + + client_ws = MagicMock() + audio_msg = json.dumps({"type": "input_audio_buffer.append", "audio": "AA=="}) + session_update = json.dumps( + {"type": "session.update", "session": {"modalities": ["audio"]}} + ) + client_ws.receive_text = AsyncMock( + side_effect=[audio_msg, session_update, ConnectionClosed(None, None)] + ) + backend_ws = MagicMock() + backend_ws.send = AsyncMock() + logging_obj = MagicMock() + config = GeminiRealtimeConfig() + + streaming = RealTimeStreaming( + client_ws, + backend_ws, + logging_obj, + provider_config=config, + model="gemini-live-2.5-flash-native-audio", + ) + + await streaming.client_ack_messages() + + assert backend_ws.send.await_count == 1 + sent_payload = json.loads(backend_ws.send.await_args_list[0].args[0]) + assert "setup" in sent_payload + assert "realtimeInput" not in sent_payload + assert streaming._pending_messages_until_setup == [audio_msg] + + +@pytest.mark.asyncio +async def test_deferred_setup_flush_buffers_audio_received_during_flush(): + import asyncio + + client_ws = MagicMock() + client_ws.send_text = AsyncMock() + new_audio_msg = json.dumps( + {"type": "input_audio_buffer.append", "audio": "new-audio"} + ) + client_ws.receive_text = AsyncMock( + side_effect=[new_audio_msg, ConnectionClosed(None, None)] + ) + backend_ws = MagicMock() + logging_obj = MagicMock() + + provider_config = MagicMock() + provider_config.requires_session_configuration = MagicMock(return_value=False) + provider_config.transform_realtime_response = MagicMock( + return_value={ + "response": { + "type": "session.created", + "event_id": "event_1", + "session": {"id": "sess_1", "modalities": ["audio"]}, + }, + "current_output_item_id": None, + "current_response_id": None, + "current_delta_chunks": [], + "current_conversation_id": None, + "current_item_chunks": [], + "current_delta_type": None, + "session_configuration_request": None, + } + ) + + streaming = RealTimeStreaming( + websocket=client_ws, + backend_ws=backend_ws, + logging_obj=logging_obj, + provider_config=provider_config, + model="gemini-live-2.5-flash-native-audio", + ) + old_audio_msg = json.dumps( + {"type": "input_audio_buffer.append", "audio": "old-audio"} + ) + streaming._pending_messages_until_setup = [old_audio_msg] + streaming._pending_messages_byte_total = len(old_audio_msg.encode("utf-8")) + + first_flush_started = asyncio.Event() + release_flush = asyncio.Event() + sent_messages = [] + + async def send_to_backend(message): + sent_messages.append(message) + if message == old_audio_msg: + first_flush_started.set() + await release_flush.wait() + return True + + streaming._send_to_backend = send_to_backend # type: ignore[method-assign] + setup_task = asyncio.create_task( + streaming._handle_provider_config_message(json.dumps({"setupComplete": {}})) + ) + + await asyncio.wait_for(first_flush_started.wait(), timeout=1) + await streaming.client_ack_messages() + + assert sent_messages == [old_audio_msg] + assert streaming._pending_messages_until_setup == [new_audio_msg] + + release_flush.set() + await asyncio.wait_for(setup_task, timeout=1) + + assert sent_messages == [old_audio_msg, new_audio_msg] + assert streaming._pending_messages_until_setup == [] + + +@pytest.mark.asyncio +async def test_deferred_setup_flush_retains_unsent_messages_after_send_failure(): + client_ws = MagicMock() + backend_ws = MagicMock() + logging_obj = MagicMock() + streaming = RealTimeStreaming(client_ws, backend_ws, logging_obj) + buffered_messages = [ + json.dumps({"type": "input_audio_buffer.append", "audio": "AA=="}), + json.dumps({"type": "input_audio_buffer.commit"}), + ] + streaming._pending_messages_until_setup = list(buffered_messages) + streaming._pending_messages_byte_total = sum( + len(message.encode("utf-8")) for message in buffered_messages + ) + streaming._send_to_backend = AsyncMock( # type: ignore[method-assign] + side_effect=Exception("transient") + ) + + await streaming._flush_pending_messages_until_setup() + + assert streaming._pending_messages_until_setup == buffered_messages + assert streaming._pending_messages_byte_total == sum( + len(message.encode("utf-8")) for message in buffered_messages + ) + + streaming._send_to_backend = AsyncMock(return_value=True) # type: ignore[method-assign] + + await streaming._flush_pending_messages_until_setup() + + assert streaming._pending_messages_until_setup == [] + assert streaming._pending_messages_byte_total == 0 + assert streaming._send_to_backend.await_count == 2 + + +@pytest.mark.asyncio +async def test_deferred_setup_flushes_audio_on_backend_session_created(monkeypatch): + """Buffered audio is released when Gemini setupComplete becomes session.created.""" + monkeypatch.setattr(litellm, "gemini_live_defer_setup", True, raising=False) + from litellm.llms.gemini.realtime.transformation import GeminiRealtimeConfig + + client_ws = MagicMock() + client_ws.send_text = AsyncMock() + backend_ws = MagicMock() + backend_ws.send = AsyncMock() + backend_ws.recv = AsyncMock( + side_effect=[ + json.dumps({"setupComplete": {}}).encode(), + ConnectionClosed(None, None), + ] + ) + logging_obj = MagicMock() + logging_obj.litellm_trace_id = "trace_defer" + logging_obj.async_success_handler = AsyncMock() + logging_obj.success_handler = MagicMock() + config = GeminiRealtimeConfig() + + streaming = RealTimeStreaming( + client_ws, + backend_ws, + logging_obj, + provider_config=config, + model="gemini-live-2.5-flash-native-audio", + ) + streaming._pending_messages_until_setup.append( + json.dumps({"type": "input_audio_buffer.append", "audio": "AA=="}) + ) + + await streaming.backend_to_client_send_messages() + + assert streaming._backend_setup_complete is True + assert streaming._pending_messages_until_setup == [] + assert backend_ws.send.call_count == 1 + + +@pytest.mark.asyncio +async def test_deferred_setup_caps_non_audio_buffered_messages(monkeypatch): + """A client that withholds session.update cannot grow the pre-setup buffer + without bound by streaming non-audio frames after the first audio frame.""" + monkeypatch.setattr(litellm, "gemini_live_defer_setup", True, raising=False) + from litellm.llms.gemini.realtime.transformation import GeminiRealtimeConfig + + cap = RealTimeStreaming._MAX_BUFFERED_MESSAGES + audio_msg = json.dumps({"type": "input_audio_buffer.append", "audio": "AA=="}) + flood_msg = json.dumps({"type": "foo", "data": "x" * 1024}) + + client_ws = MagicMock() + client_ws.receive_text = AsyncMock( + side_effect=[audio_msg] + + [flood_msg] * (cap + 50) + + [ConnectionClosed(None, None)] + ) + backend_ws = MagicMock() + backend_ws.send = AsyncMock() + logging_obj = MagicMock() + + streaming = RealTimeStreaming( + client_ws, + backend_ws, + logging_obj, + provider_config=GeminiRealtimeConfig(), + model="gemini-live-2.5-flash-native-audio", + ) + assert streaming._backend_setup_complete is False + + await streaming.client_ack_messages() + + backend_ws.send.assert_not_called() + assert len(streaming._pending_messages_until_setup) == cap + assert ( + streaming._pending_messages_byte_total <= RealTimeStreaming._MAX_BUFFERED_BYTES + ) + + +@pytest.mark.asyncio +async def test_deferred_setup_caps_non_audio_buffered_bytes(monkeypatch): + """Non-audio frames appended after the first audio frame honor the byte budget.""" + monkeypatch.setattr(litellm, "gemini_live_defer_setup", True, raising=False) + from litellm.llms.gemini.realtime.transformation import GeminiRealtimeConfig + + audio_msg = json.dumps({"type": "input_audio_buffer.append", "audio": "AA=="}) + big_non_audio = json.dumps( + {"type": "foo", "data": "x" * (RealTimeStreaming._MAX_BUFFERED_BYTES + 1)} + ) + + client_ws = MagicMock() + client_ws.receive_text = AsyncMock( + side_effect=[audio_msg, big_non_audio, ConnectionClosed(None, None)] + ) + backend_ws = MagicMock() + backend_ws.send = AsyncMock() + logging_obj = MagicMock() + + streaming = RealTimeStreaming( + client_ws, + backend_ws, + logging_obj, + provider_config=GeminiRealtimeConfig(), + model="gemini-live-2.5-flash-native-audio", + ) + + await streaming.client_ack_messages() + + assert streaming._pending_messages_until_setup == [audio_msg] + assert ( + streaming._pending_messages_byte_total <= RealTimeStreaming._MAX_BUFFERED_BYTES + ) diff --git a/tests/test_litellm/llms/gemini/realtime/test_gemini_realtime_transformation.py b/tests/test_litellm/llms/gemini/realtime/test_gemini_realtime_transformation.py index cc0adc4277c..53f0766dbcb 100644 --- a/tests/test_litellm/llms/gemini/realtime/test_gemini_realtime_transformation.py +++ b/tests/test_litellm/llms/gemini/realtime/test_gemini_realtime_transformation.py @@ -235,12 +235,73 @@ def test_gemini_realtime_transformation_audio_delta(): contains_audio_delta = False for response in responses: - if response["type"] == OpenAIRealtimeEventTypes.RESPONSE_AUDIO_DELTA.value: + if ( + response["type"] + == OpenAIRealtimeEventTypes.RESPONSE_OUTPUT_AUDIO_DELTA.value + ): contains_audio_delta = True break assert contains_audio_delta, "Expected audio delta event" +def test_gemini_output_audio_transcript_delta_uses_active_response_ids(): + config = GeminiRealtimeConfig() + + session_configuration_request = { + "setup": { + "model": "gemini-1.5-flash", + "generationConfig": {"responseModalities": ["AUDIO"]}, + } + } + session_configuration_request_str = json.dumps(session_configuration_request) + event = { + "serverContent": { + "outputTranscription": {"text": "Hello from Gemini."}, + "modelTurn": { + "parts": [ + {"inlineData": {"mimeType": "audio/pcm", "data": "my-audio-data"}} + ] + }, + } + } + + result = config.transform_realtime_response( + json.dumps(event), + "gemini-1.5-flash", + MagicMock(), + realtime_response_transform_input={ + "session_configuration_request": session_configuration_request_str, + "current_output_item_id": None, + "current_response_id": None, + "current_conversation_id": None, + "current_delta_chunks": [], + "current_item_chunks": [], + "current_delta_type": None, + }, + ) + + responses = result["response"] + response_created = next( + response for response in responses if response["type"] == "response.created" + ) + transcript_delta = next( + response + for response in responses + if response["type"] == "response.output_audio_transcript.delta" + ) + audio_delta = next( + response + for response in responses + if response["type"] == "response.output_audio.delta" + ) + + assert transcript_delta["response_id"] == response_created["response"]["id"] + assert transcript_delta["response_id"] == audio_delta["response_id"] + assert transcript_delta["item_id"] == audio_delta["item_id"] + assert result["current_response_id"] == transcript_delta["response_id"] + assert result["current_output_item_id"] == transcript_delta["item_id"] + + def test_gemini_realtime_transformation_generation_complete(): from litellm.types.llms.openai import OpenAIRealtimeEventTypes @@ -278,7 +339,10 @@ def test_gemini_realtime_transformation_generation_complete(): contains_audio_done_event = False for response in responses: - if response["type"] == OpenAIRealtimeEventTypes.RESPONSE_AUDIO_DONE.value: + if ( + response["type"] + == OpenAIRealtimeEventTypes.RESPONSE_OUTPUT_AUDIO_DONE.value + ): contains_audio_done_event = True break assert contains_audio_done_event, "Expected audio done event" @@ -735,7 +799,14 @@ def test_gemini_tool_call_emits_response_created_preamble(): ) responses = result["response"] - # Should have: response.created, output_item.added, function_call_arguments.delta, function_call_arguments.done, output_item.done, conversation.item.created, response.done + # Expected sequence: + # 0: response.created + # 1: response.output_item.added (item status=in_progress) + # 2: conversation.item.added (registers call_id in Pipecat's _pending_function_calls) + # 3: response.function_call_arguments.delta + # 4: response.function_call_arguments.done + # 5: response.output_item.done + # 6: response.done assert len(responses) >= 7 assert responses[0]["type"] == "response.created" assert "response" in responses[0] @@ -749,14 +820,14 @@ def test_gemini_tool_call_emits_response_created_preamble(): assert responses[1]["type"] == "response.output_item.added" assert responses[1]["item"]["type"] == "function_call" assert responses[1]["item"]["status"] == "in_progress" - assert responses[2]["type"] == "response.function_call_arguments.delta" - assert responses[2]["call_id"] == "call_123" - assert responses[2]["delta"] == responses[3]["arguments"] - assert responses[3]["type"] == "response.function_call_arguments.done" - assert responses[4]["type"] == "response.output_item.done" - assert responses[4]["item"]["type"] == "function_call" - assert responses[4]["item"]["status"] == "completed" - assert responses[5]["type"] == "conversation.item.created" + assert responses[2]["type"] == "conversation.item.added" + assert responses[2]["item"]["type"] == "function_call" + assert responses[2]["item"]["call_id"] == "call_123" + assert responses[3]["type"] == "response.function_call_arguments.delta" + assert responses[3]["call_id"] == "call_123" + assert responses[3]["delta"] == responses[4]["arguments"] + assert responses[4]["type"] == "response.function_call_arguments.done" + assert responses[5]["type"] == "response.output_item.done" assert responses[5]["item"]["type"] == "function_call" assert responses[5]["item"]["status"] == "completed" assert responses[6]["type"] == "response.done" @@ -930,6 +1001,12 @@ def test_gemini_tool_call_response_done_includes_usage_from_sibling_metadata(): "promptTokenCount": 17, "responseTokenCount": 4, "totalTokenCount": 21, + "promptTokensDetails": [ + {"modality": "TEXT", "tokenCount": 17}, + ], + "responseTokensDetails": [ + {"modality": "TEXT", "tokenCount": 4}, + ], }, } ), @@ -953,6 +1030,8 @@ def test_gemini_tool_call_response_done_includes_usage_from_sibling_metadata(): assert usage["input_tokens"] == 17 assert usage["output_tokens"] == 4 assert usage["total_tokens"] == 21 + assert usage["input_token_details"]["text_tokens"] == 17 + assert usage["output_token_details"]["text_tokens"] == 4 def test_gemini_tool_call_response_done_falls_back_to_empty_usage(): @@ -1120,6 +1199,90 @@ def test_gemini_subsequent_session_update_forwards_tools_merged_with_original_se assert follow_up["inputAudioTranscription"] == {} +def test_gemini_realtime_pipecat_ga_session_voice_and_tools(): + """Pipecat OpenAIRealtimeSessionProperties: output_modalities, nested tools, + and audio.output.voice (e.g. Kore) must map into Gemini setup.""" + config = GeminiRealtimeConfig() + + session_update = { + "type": "session.update", + "session": { + "output_modalities": ["audio"], + "instructions": "Follow system instructions.", + "tools": [ + { + "type": "function", + "function": { + "name": "terminate_call", + "description": "End the call.", + "parameters": {"type": "object", "properties": {}}, + }, + } + ], + "audio": { + "input": { + "format": {"type": "audio/pcm", "rate": 24000}, + "turn_detection": {"type": "server_vad"}, + }, + "output": { + "format": {"type": "audio/pcm", "rate": 24000}, + "voice": "Kore", + }, + }, + "temperature": 0, + }, + } + + messages = config.transform_realtime_request( + json.dumps(session_update), + "gemini-2.5-flash-native-audio", + session_configuration_request=None, + ) + + assert len(messages) == 1 + setup = json.loads(messages[0])["setup"] + assert setup["generationConfig"]["responseModalities"] == ["AUDIO"] + # Native-audio Live rejects speechConfig on setup (see _finalize_gemini_live_setup). + assert "speechConfig" not in setup.get("generationConfig", {}) + assert setup["tools"][0]["function_declarations"][0]["name"] == "terminate_call" + assert ( + setup["realtimeInputConfig"]["automaticActivityDetection"]["disabled"] is False + ) + + +def test_gemini_realtime_pipecat_semantic_vad_omits_realtime_input_config(): + """Pipecat SemanticTurnDetection (semantic_vad) must not map to disabled VAD.""" + config = GeminiRealtimeConfig() + session_update = { + "type": "session.update", + "session": { + "output_modalities": ["audio"], + "instructions": "test", + "audio": { + "input": {"turn_detection": {"type": "semantic_vad"}}, + }, + "tools": [ + { + "type": "function", + "function": { + "name": "terminate_call", + "description": "End call.", + "parameters": {"type": "object", "properties": {}}, + }, + } + ], + }, + } + messages = config.transform_realtime_request( + json.dumps(session_update), + "gemini-live-2.5-flash-native-audio", + session_configuration_request=None, + ) + setup = json.loads(messages[0])["setup"] + assert "realtimeInputConfig" not in setup + assert setup["tools"][0]["function_declarations"][0]["name"] == "terminate_call" + + def test_gemini_subsequent_session_update_with_turn_detection_only_preserves_original_tools(): """A subsequent session.update carrying only turn_detection (the guardrail-injected disable) must keep the original tools/generationConfig.""" @@ -1407,6 +1570,12 @@ def test_gemini_standalone_usage_metadata_is_attributed_to_next_response_done(): "promptTokenCount": 5, "responseTokenCount": 11, "totalTokenCount": 16, + "promptTokensDetails": [ + {"modality": "TEXT", "tokenCount": 5}, + ], + "responseTokensDetails": [ + {"modality": "TEXT", "tokenCount": 11}, + ], } } ), @@ -1447,6 +1616,8 @@ def test_gemini_standalone_usage_metadata_is_attributed_to_next_response_done(): assert usage["input_tokens"] == 5 assert usage["output_tokens"] == 11 assert usage["total_tokens"] == 16 + assert usage["input_token_details"]["text_tokens"] == 5 + assert usage["output_token_details"]["text_tokens"] == 11 assert config._pending_usage_metadata is None diff --git a/tests/test_litellm/llms/vertex_ai/realtime/test_vertex_ai_realtime_transformation.py b/tests/test_litellm/llms/vertex_ai/realtime/test_vertex_ai_realtime_transformation.py index 0ad614099de..1ebd704be34 100644 --- a/tests/test_litellm/llms/vertex_ai/realtime/test_vertex_ai_realtime_transformation.py +++ b/tests/test_litellm/llms/vertex_ai/realtime/test_vertex_ai_realtime_transformation.py @@ -278,8 +278,8 @@ async def test_vertex_realtime_text_in_text_out(): assert session_created_msgs, "Expected session.created to be sent to client" # At least one text delta should have been forwarded - text_delta_msgs = [m for m in sent_to_client if '"response.text.delta"' in m] - assert text_delta_msgs, "Expected response.text.delta to be sent to client" + text_delta_msgs = [m for m in sent_to_client if '"response.output_text.delta"' in m] + assert text_delta_msgs, "Expected response.output_text.delta to be sent to client" # Verify the delta contains the model's text delta_obj = json.loads(text_delta_msgs[0])