The unified guardrail's streaming iterator hook processes every Nth
chunk (sampling_rate, default 5). On each sampled chunk it calls
process_output_streaming_response, which combines all accumulated text
into the first chunk and clears all subsequent chunks to "".
The hook then yielded `processed_items[-1]` — the last item, whose
content had been cleared to "". This permanently lost every Nth
chunk's content, causing random missing words/tokens in the client
output (observed in Roo Code, Open WebUI, etc.).
Fix: deep-copy the current chunk before guardrail processing runs,
then yield the original (unmodified) chunk. The guardrail validation
still executes and can block if it detects a problem, but the stream
content is preserved.
Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
* feat: add faster linting targets for development workflow
- Add lint-dev target that only checks changed files vs origin/main
- Add lint-format-changed to format only modified Python lines
- Add lint-ruff-dev using diff-quality for incremental lint checks
- Upgrade ruff from 0.1.x to 0.2.x for --range formatting support
- Add pylint and diff-cover as dev dependencies
- Use portable PIP variable for cross-platform compatibility
- Suppress poetry warnings in install-dev target
* fix(mypy): fix type: ignore placement for OTEL LogRecord import
The type: ignore[attr-defined] comment was on the import alias line
inside parentheses, but mypy reports the error on the `from` line.
Collapse to single-line imports so the suppression is on the correct
line. Also add no-redef to the fallback branch.
* fix: address review issues in faster linting PR
- Remove poetry lock/check from install-dev (slow, can mutate lockfile)
- Remove misplaced [virtualenvs] and [installer] from pyproject.toml
(these belong in poetry.toml, not project metadata)
- Remove unused pylint dev dependency (diff-quality uses pylint output
format, not the pylint package itself)
- Fix trailing whitespace in .PHONY declaration
- Use mktemp instead of hardcoded /tmp/ruff.txt in lint-ruff-dev
- Guard lint-ruff-FULL-dev against empty file list from git diff
- Fix incorrect comment on lint-dev target
- Regenerate poetry.lock
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* fix: address review issues in faster linting PR
- Remove poetry lock/check from install-dev (slow, can mutate lockfile)
- Remove misplaced [virtualenvs] and [installer] from pyproject.toml
(these belong in poetry.toml, not project metadata)
- Remove unused pylint dev dependency (diff-quality uses pylint output
format, not the pylint package itself)
- Fix trailing whitespace in .PHONY declaration
- Use mktemp instead of hardcoded /tmp/ruff.txt in lint-ruff-dev
- Guard lint-ruff-FULL-dev against empty file list from git diff
- Fix incorrect comment on lint-dev target
- Regenerate poetry.lock
---------
Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
* feat(guardrails/): allow custom code execution for guardrails
first step in allowing teams to submit custom code for guardrails
* feat: custom_code_guardrail.md
support passing custom code for guardrails
* feat: initial commit adding ui for custom code guardrails
allows users to write guardrails based on custom code
* feat: expose new test custom code guardrail endpoint
allows ui testing playground to sanity check if guardrail is working as expected
* fix: fix linting errors
* fix: fix max recursion check
* fix: fix linting error
* fix: enforce team MCP permissions when using JWT authentication
Root cause: When JWT auth was used with teams in groups (via team_ids_jwt_field),
the team's MCP permissions were not being enforced because:
1. The default team_allowed_routes did not include mcp_routes
2. allowed_routes_check() failed for MCP endpoints like /mcp/tools/list
3. find_team_with_model_access() skipped the team due to failed route check
4. team_id was None in UserAPIKeyAuth
5. MCPRequestHandler._get_allowed_mcp_servers_for_team() returned empty list
Fix: Add 'mcp_routes' to the default team_allowed_routes in LiteLLM_JWTAuth.
This ensures that teams can access MCP endpoints by default, allowing the
team's MCP server permissions to be properly enforced.
Added tests:
- test_reproduce_jwt_mcp_enforcement_issue: Reproduces the exact bug scenario
- test_verify_mcp_routes_in_default_team_allowed_routes: Verifies fix
- test_mcp_route_check_passes_for_team: Verifies route check works
Co-authored-by: ishaan <ishaan@berri.ai>
* test: add comprehensive E2E tests for JWT + team MCP permission enforcement
Added tests:
- test_e2e_jwt_team_mcp_permissions_enforced: Full E2E test verifying JWT auth
with teams in groups properly sets team_id and MCPRequestHandler returns
the team's MCP servers
- test_e2e_jwt_without_team_no_mcp_servers: Verifies no MCP servers returned
when JWT has no teams
- test_e2e_jwt_team_mcp_key_intersection: Verifies intersection logic when
both key and team have MCP permissions (result = intersection)
These tests verify the complete flow:
1. JWT token with team in groups field
2. JWT auth properly sets team_id on UserAPIKeyAuth
3. MCPRequestHandler.get_allowed_mcp_servers() returns team's MCP servers
4. Key/team permission intersection works correctly
Co-authored-by: ishaan <ishaan@berri.ai>
* test: add simple tests for JWT + MCP permission enforcement
Simple, focused tests that validate:
1. test_simple_jwt_mcp_permissions_enforced: JWT user with team gets team's MCP servers
2. test_simple_jwt_no_team_no_mcp_servers: JWT user without team gets no MCP servers
3. test_simple_jwt_team_id_required_for_mcp_permissions: Verifies team_id is required
4. test_jwt_auth_sets_team_id_for_mcp_route: JWT auth sets team_id for MCP routes
These tests directly verify the core MCP permission enforcement logic works
when using JWT authentication with teams.
Co-authored-by: ishaan <ishaan@berri.ai>
* Add test: MCP route without model still returns team_id
Co-authored-by: ishaan <ishaan@berri.ai>
* Add 2 debug logs for JWT+MCP troubleshooting
- handle_jwt.py: Log team route check result (team_id, route, is_allowed)
- user_api_key_auth_mcp.py: Log team_id when looking up MCP permissions
Co-authored-by: ishaan <ishaan@berri.ai>
---------
Co-authored-by: Cursor Agent <cursoragent@cursor.com>
Co-authored-by: ishaan <ishaan@berri.ai>
AsyncHTTPHandler.__del__ was closing httpx clients still in use by
AsyncOpenAI/AsyncAzureOpenAI due to independent cache lifecycles.
Restores standalone httpx client creation for OpenAI/Azure providers.
* fix fail-open for grayswan; pass metadata to cygnal api endpoint; update docs
* pass litellm_metadata to cygnal in payload
* switch error msg to const, and clean exception handling.
* update pyproject.toml as requested
* Revert "update pyproject.toml as requested"
This reverts commit 4eece154d0.
* fix: check for model_response_choices before guardrail input
* test: add tests for responses api translation
* fix: protect other guardrail translations
* refactor: remove type ignores
* anthropic request body got mutated fix
* add warning when extra_body is provided but user is non premium
* fix: resolve mypy union-attr errors in anthropic guardrail handler
Cast choices[0] to Choices type before accessing .message attribute
to satisfy mypy's union type checking for Choices | StreamingChoices.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* add logger when model response has no choices for streaming /response and /messages
* update pyproject.toml as requested
* Revert "update pyproject.toml as requested"
This reverts commit 541a2b075a.
* update pyproject.toml as requested
* Revert "update pyproject.toml as requested"
This reverts commit 716ea0caa1.
---------
Co-authored-by: Xiaohan Fu <xiaohan@grayswan.ai>
Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
* feat(ui): Add team-alias column to Models Health Status UI
- Added Team Alias column to the Models Health Status table
- Updated HealthCheckComponent to accept teams prop
- Updated health_check_columns to display team alias based on team_id
- Falls back to team_id if team alias not found, or shows '-' if no team
- Updated parent components to pass teams data to HealthCheckComponent
Co-authored-by: ishaan <ishaan@berri.ai>
* Update ui/litellm-dashboard/src/components/model_dashboard/health_check_columns.tsx
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
---------
Co-authored-by: Cursor Agent <cursoragent@cursor.com>
Co-authored-by: ishaan <ishaan@berri.ai>
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
