* Mark xAI models retiring on 2026-05-15 (#28788)
Per https://docs.x.ai/developers/migration/may-15-retirement, xAI is
retiring the following slugs on 2026-05-15 (auto-redirect to grok-4.3
with various reasoning efforts; callers continuing to use the old slugs
will be billed at grok-4.3 pricing):
grok-4-1-fast-reasoning{,-latest} -> grok-4.3 (low effort)
grok-4-1-fast-non-reasoning{,-latest} -> grok-4.3 (none)
grok-4-fast-reasoning -> grok-4.3 (low effort)
grok-4-fast-non-reasoning -> grok-4.3 (none)
grok-4-0709 -> grok-4.3 (low effort)
grok-code-fast-1{,-0825} -> grok-build-0.1
grok-3 -> grok-4.3 (none)
Only the direct xai/ slugs are tagged; third-party hosts (azure_ai,
oci, vercel_ai_gateway, perplexity/xai) run their own schedules. The
grok-3 retirement list explicitly names only the base grok-3 slug — the
-mini / -fast / -beta / -latest variants are not listed, so they remain
untouched.
* feat(moonshot): advertise json_schema response support on live models (#29683)
litellm.responses() already routes Moonshot through the responses->chat-completions
bridge, and Moonshot honors response_format json_schema on chat completions. The
cost-map entries left supports_response_schema unset, so discovery layers that gate
on that flag dropped Moonshot from structured-output / responses listings even though
the capability works end to end.
Set supports_response_schema on the nine models currently live on api.moonshot.ai:
kimi-k2.5, kimi-k2.6, the moonshot-v1 8k/32k/128k text and vision-preview variants,
and moonshot-v1-auto. Verified against the live API that each honors json_schema and
that litellm.responses() returns schema-valid structured output through the bridge.
* chore(moonshot): mark models retired from api.moonshot.ai as deprecated (#29685)
Thirteen Moonshot/Kimi models in the cost map no longer resolve on
api.moonshot.ai (all return 404). Stamp each with its deprecation_date from
platform.kimi.ai/docs/models rather than deleting the entries, so historical
cost calculation keeps resolving the names while tooling can surface the
retirement.
Dates: kimi-thinking-preview 2025-11-11; kimi-latest and its 8k/32k/128k context
variants 2026-01-28; the kimi-k2 preview/turbo/thinking series 2026-05-25; the
moonshot-v1 -0430 snapshots use their own 2024-04-30 snapshot date (Moonshot
publishes no discontinuation date for them).
* fix(moonshot): drop temperature for reasoning models (kimi-k2.5/k2.6) (#29687)
Kimi reasoning models reject every temperature except 1; a request with
temperature=0.2 returns "invalid temperature: only 1 is allowed for this model".
litellm only clamped temperature into [0.3, 1], so any value below 1 still 400'd.
Drop the temperature param entirely for reasoning models (gated on
supports_reasoning, the same signal transform_request already uses) so the model
default is used; the non-reasoning moonshot-v1 models keep the existing clamp.
Co-authored-by: Sameer Kankute <sameer@berri.ai>
* feat(mcp): add per-server timeout configuration (#29672)
* feat(mcp): add per-server timeout configuration
* fix(mcp): address timeout field review comments
- use is not None guard instead of or for 0.0 edge case
- copy timeout in both LiteLLM_MCPServerTable constructions (health check path + _build_mcp_server_table)
- add timeout Float? column to all three schema.prisma files
- extend round-trip test to cover _build_mcp_server_table direction
- add test for zero timeout not treated as falsy
* fix(mcp): forward timeout in _build_temporary_mcp_server_record
* fix(mcp): return 504 instead of 500 when per-server timeout fires
* test(mcp): add 504 timeout regression test; fix black formatting
* Add jp. Bedrock cross-region inference profile for claude-opus-4-7 (#28567)
* fix(thinking): handle None thinking param in is_thinking_enabled (#28598)
Squash-merged by litellm-agent from Terrajlz's PR.
* feat(helm): support tpl rendering in podAnnotations (#28609)
Squash-merged by litellm-agent from devauxbr's PR.
* Forward custom_llm_provider through the Responses API bridge (Fixes#28505) (#28575)
* Forward custom_llm_provider through the Responses API bridge (Fixes#28505)
When a Chat Completions request to a GPT-5.4+ model contains both
`tools` and `reasoning_effort`, `completion()` auto-routes through
`responses_api_bridge`. The bridge handler called
`litellm.responses()` / `litellm.aresponses()` without forwarding the
already-resolved `custom_llm_provider`, so the downstream call
re-invoked `get_llm_provider()` with `custom_llm_provider=None` and
stripped a second provider prefix from a `provider/provider/model`
deployment string.
For a deployment configured as `openai/openai/openai/gpt-5.5`,
the bridge flow sent `openai/gpt-5.5` to the upstream API instead of
the correct `openai/openai/gpt-5.5`. Upstream APIs that enforce
model-name allow-lists rejected this as `key_model_access_denied`.
Fix: pass the locally-resolved `custom_llm_provider` into both the
sync `responses()` and async `aresponses()` calls so the downstream
`_resolve_model_provider_for_responses` sees an explicit provider
and skips the second prefix-strip.
New regression test
`tests/test_litellm/completion_extras/test_responses_bridge_provider_propagation.py`
pins both call sites: each must forward `custom_llm_provider`.
* fix(28505): set custom_llm_provider on request_data instead of as duplicate kwarg
Greptile flagged that the previous patch passed custom_llm_provider as an
explicit kwarg to responses()/aresponses() while request_data already
carried it via the spread of sanitized_litellm_params, which would raise
TypeError: got multiple values for keyword argument on every real bridge
call.
Switches to assigning request_data['custom_llm_provider'] before the call
so the resolved provider wins over whatever sanitized_litellm_params spread
in, without duplicating the kwarg.
Updates the regression test to seed request_data with a sentinel
custom_llm_provider so it actually exercises the overwrite path (the
previous test mocked transform_request with a minimal dict and never hit
the conflict).
* chore: trigger shin-agent re-eval on retargeted staging base
* chore: trigger shin-agent re-eval against updated Greptile state
* Add jp. Bedrock cross-region inference profile for claude-opus-4-7
AWS Bedrock documents jp.anthropic.claude-opus-4-7 alongside the
existing us./eu./au./global. profiles for Claude Opus 4.7
(ap-northeast-1 Tokyo / ap-northeast-3 Osaka), but the entry is
missing from model_prices_and_context_window.json. Tokyo-region
users currently get an "unknown model" error when routing through
the JP geo profile.
Adds the entry to both the canonical file and the bundled backup,
mirroring the recent pattern for sonnet-4-6 (#27831). Pricing matches
the other regional profiles (10% premium over base/global).
Regression test pins all six documented profiles (base, global, us, eu,
au, jp) and asserts pricing parity between jp. and au. variants.
Source: https://docs.aws.amazon.com/bedrock/latest/userguide/model-card-anthropic-claude-opus-4-7.html
---------
Co-authored-by: Terrajlz <info@jouleselectrictech.com>
Co-authored-by: Bruno Devaux <devaux.br@gmail.com>
Co-authored-by: Sameer Kankute <sameer@berri.ai>
* feat(soniox): add soniox audio transcription integration (#29508)
* feat(openmeter): add OPENMETER_TRUST_REQUEST_USER to prevent forged attribution (#29650)
The OpenMeter callback resolves the CloudEvent subject from kwargs["user"]
first, then falls back to the key-bound user_api_key_user_id. For
multi-tenant proxy deployments, a client can set `"user": "..."` in the
request body and cause their usage to be attributed to that arbitrary
string — a billing-attribution forgery risk.
Adds OPENMETER_TRUST_REQUEST_USER env var (default "true" for backward
compatibility). When set to "false", the request-supplied `user` field is
ignored and the subject is resolved solely from user_api_key_user_id.
Matches the existing env-var-driven config pattern in this file
(OPENMETER_API_KEY, OPENMETER_API_ENDPOINT, OPENMETER_EVENT_TYPE).
* feat(search): add you_com as a search provider (#28370)
* feat(search): add you_com as a search provider
Registers You.com Search API as a first-class `search_provider` in the
`search_tools` registry, alongside Tavily, Exa, Perplexity, etc.
- New adapter: litellm/llms/you_com/search/transformation.py
- POSTs to https://ydc-index.io/v1/search
- Auth: X-API-Key from YOUCOM_API_KEY (or explicit api_key)
- Maps Perplexity unified spec: max_results -> count,
search_domain_filter -> include_domains, country -> country
- Flattens results.web + results.news into a single SearchResult list;
snippet prefers snippets[0], falls back to description; page_age -> date
- Registry: SearchProviders.YOU_COM in litellm/types/utils.py and wired
into ProviderConfigManager.get_provider_search_config()
- Pricing entry: model_prices_and_context_window.json (placeholder $0.0;
happy to adjust to maintainers' preferred public number)
- Docs: example router config snippet and example proxy yaml updated
- Tests: tests/search_tests/test_you_com_search.py - 5 mocked tests
(payload shape, domain filter mapping, snippet fallback, news flattening,
missing-api-key error)
Refs upstream expansion signal: #15942
* review fixups: normalize api_base, lowercase country, scope env-var to test
Addresses Greptile inline review comments on #28370:
- get_complete_url: strip trailing slashes from api_base *before* the
endswith("/v1/search") check, so a custom base like ".../v1/search/"
doesn't become ".../v1/search/v1/search".
- transform_search_request: .lower() country before sending, matching
Tavily's convention so callers using the unified spec form ("US") get
consistent behavior across providers.
- Tests: replace direct os.environ writes with an autouse monkeypatch
fixture so YOUCOM_API_KEY is set per-test and removed afterwards.
The missing-key test now uses monkeypatch.delenv. New test asserts the
trailing-slash normalization above.
Reverts the ARCHITECTURE.md / example yaml edits per the reviewer note
that documentation changes belong in the litellm-docs repo.
* support keyless free tier (api.you.com/v1/agents/search) as default
You.com offers an IP-throttled keyless endpoint that returns the same
response shape as the keyed one (~100 queries/day, no signup). This is a
significant onboarding lever - mirrors the keyless DuckDuckGo/SearXNG
providers already in the search_tools registry.
Behavior:
- YOUCOM_API_KEY set -> keyed: POST https://ydc-index.io/v1/search
(X-API-Key header)
- no key -> free: POST https://api.you.com/v1/agents/search
(no auth)
- YOUCOM_API_BASE override -> honored as-is
Tests:
- New: test_you_com_search_keyless_free_tier - asserts URL + absence of
X-API-Key when no key is configured.
- New: test_you_com_search_validate_environment_keyless - asserts the
config no longer raises when the key is absent.
- Removed: test_you_com_search_raises_without_api_key (the precondition
no longer holds).
- Existing payload/domain-filter/etc tests still cover keyed mode via
the autouse YOUCOM_API_KEY fixture.
Verified both endpoints accept POST + return identical JSON shape:
results.web[] / results.news[] with title, url, snippets, description,
page_age.
* register you_com in provider_endpoints_support.json
Adding `litellm/llms/you_com/` requires a corresponding entry in
provider_endpoints_support.json or the
code-quality/check_provider_folders_documented CI check fails.
Follows the compact tavily/serper pattern - endpoints: { search: true }.
Local run of the check now reports "All 114 provider folders are documented".
* move tests under tests/test_litellm/llms/ so CI exercises them
The litellm CI workflows scope unit tests to `tests/test_litellm/...`
(see test-unit-llm-providers.yml: `tests/test_litellm/llms` path), so
tests living under `tests/search_tests/` are never run in CI - which is
why codecov reports 0% patch coverage for the new adapter even though
the unit tests exist and pass locally.
Move test_you_com_search.py into `tests/test_litellm/llms/you_com/` so
the test-unit-llm-providers job picks it up. 7/7 tests still pass at
the new location.
(Sibling search-only providers - tavily, exa_ai, brave, etc. - still
live only in `tests/search_tests/` and would benefit from the same
move, but that is out of scope for this PR.)
* fix(you_com): pin Accept-Encoding: identity to dodge keyless gzip bug
The keyless free-tier endpoint (api.you.com/v1/agents/search) advertises
Content-Encoding: gzip but returns a body that httpx's decoder rejects
with `zlib.error: Error -3 while decompressing data: incorrect header
check`, surfacing as litellm.APIConnectionError in user code. curl works
because it doesn't request compression by default.
Pin Accept-Encoding: identity in validate_environment so the upstream
server skips compression entirely. Harmless on the keyed endpoint
(ydc-index.io/v1/search) which negotiates content-encoding correctly.
The header uses setdefault so a caller-supplied Accept-Encoding still
takes precedence. (Server-side bug has been flagged to the You.com team
separately - once fixed there, this workaround can be removed.)
New unit test: test_you_com_search_pins_identity_accept_encoding.
---------
Co-authored-by: Sameer Kankute <sameer@berri.ai>
* docs: fix README typo (#29419)
Correct clear spelling mistakes in documentation without changing behavior.
Confidence: high
Scope-risk: narrow
Tested: git diff --check; uvx codespell on changed files
Not-tested: Full docs build not run; text-only changes
* Fix(langfuse): pass httpx_client to Langfuse in langfuse_prompt_management to respect SSL_VERIFY (#29480)
* fix(langfuse): pass ssl_verify to Langfuse httpx client
* fix_langfuse_
* add unit tests
* addressed comments
---------
Co-authored-by: shin-berri <shin-laptop@berri.ai>
Co-authored-by: yuneng-jiang <yuneng@berri.ai>
* feat(models): add minimax/MiniMax-M3 to model cost map (#29412)
Add MiniMax's new flagship MiniMax-M3 to the native minimax provider:
512K context, 128K max output, native multimodal (supports_vision),
reasoning, prompt caching. Pricing (USD/M tokens): input 0.6 / output
2.4 / cache read 0.12. M3 has no active prompt-cache-write tier, so
cache_creation_input_token_cost is omitted.
Updated both the root model_prices_and_context_window.json (remote
source) and the bundled litellm/model_prices_and_context_window_backup.json
(local fallback), keeping them in sync.
* fix(logging): handle ResponseCompletedEvent in anthropic_messages streaming spend log (#29394)
* fix(logging): handle ResponseCompletedEvent in anthropic_messages streaming spend log
* fix(logging): extend terminal event handling to ResponseIncompleteEvent and ResponseFailedEvent; fix return type annotation
* feat(provider): Add Neosantara provider as OpenAI Compatible (#29646)
* Add Neosantara provider
* Register Neosantara provider enum
* Address Neosantara provider review feedback
* Add Neosantara packaged endpoint support
---------
Co-authored-by: shin-berri <shin-laptop@berri.ai>
Co-authored-by: yuneng-jiang <yuneng@berri.ai>
* fix: address greptile and veria review feedback
- langfuse: guard httpx_client injection behind version check (>= 2.7.3)
- soniox: propagate audio_transcription_duration in _hidden_params for spend tracking
- soniox: give SONIOX_API_BASE env var priority over caller-supplied api_base
- mcp: replace CancelledError catch with asyncio.wait_for + TimeoutError
* chore(mcp): add migration for per-server timeout column
* fix(test): add tool_use_system_prompt_tokens to model prices schema validator
* fix: mcp timeout test uses real asyncio.wait_for timeout; you_com get_complete_url respects resolved api_key
* fix: forward resolved api_key into you_com endpoint selection and apply timeout to soniox polling GETs
The search flow resolves api_key in validate_environment but never passed it
into get_complete_url, so a programmatic api_key (with no YOUCOM_API_KEY in the
env) set the X-API-Key header yet still selected the keyless free-tier endpoint.
Forward api_key through both the search entrypoint and the http handler so the
keyed endpoint is chosen.
HTTPHandler.get/AsyncHTTPHandler.get had no timeout parameter, so the Soniox
poll and transcript-fetch GETs silently used the client global default instead
of the caller timeout. Add a per-request timeout to get() and forward the
configured timeout from the Soniox handler.
* fix(soniox): price stt-async-v4 per second so transcriptions are billed
The handler stores audio_transcription_duration in _hidden_params, but the
model carried only token cost fields and the response has no token usage, so
the transcription cost path fell through to cost_per_second and returned $0.
An authenticated caller could transcribe Soniox audio without decrementing
their budget. Switch the entry to output_cost_per_second at Soniox's published
$0.10/hour async rate so the stored duration produces a real charge.
* fix(langfuse): use a dedicated httpx client for the SDK injection
The httpx_client handed to the Langfuse SDK came from _get_httpx_client(),
which returns LiteLLM's globally cached HTTPHandler. If Langfuse closed that
client on teardown it would invalidate the shared client used by every other
LiteLLM HTTP call. Build a dedicated httpx.Client instead, still resolving SSL
verification and client certificate from LiteLLM's configuration.
* fix(soniox): prefer caller-supplied api_base over SONIOX_API_BASE env var
* fix(cohere): support max_completion_tokens on cohere v2 chat (default route) (#29779)
* fix(cohere): support max_completion_tokens on cohere v2 chat
The default cohere_chat route resolves to CohereV2ChatConfig, which did not
list or map max_completion_tokens, so get_optional_params raised
UnsupportedParamsError for the standard OpenAI parameter (the modern
replacement for the deprecated max_tokens). The v1 config already maps it to
cohere's max_tokens; mirror that in v2 and add v2 regression tests.
* fix(cohere): make max_completion_tokens take precedence over max_tokens on v2
When both max_tokens and max_completion_tokens are supplied, prefer
max_completion_tokens explicitly rather than relying on dict iteration order,
and cover both orderings with a regression test.
---------
Co-authored-by: Daniel Yudelevich <4537920+yudelevi@users.noreply.github.com>
Co-authored-by: hectorc98 <hector.chamorroalvarez@adyen.com>
Co-authored-by: Filippo Menghi <113345637+Cyberfilo@users.noreply.github.com>
Co-authored-by: Terrajlz <info@jouleselectrictech.com>
Co-authored-by: Bruno Devaux <devaux.br@gmail.com>
Co-authored-by: Dan Lemon <dan@danlemon.com>
Co-authored-by: Saswat <saswatds@users.noreply.github.com>
Co-authored-by: Brian Sparker <brainsparker@users.noreply.github.com>
Co-authored-by: Zhao73 <156770117+Zhao73@users.noreply.github.com>
Co-authored-by: Urain Ahmad Shah <60431964+urainshah@users.noreply.github.com>
Co-authored-by: shin-berri <shin-laptop@berri.ai>
Co-authored-by: yuneng-jiang <yuneng@berri.ai>
Co-authored-by: kape <168134658+kapelame@users.noreply.github.com>
Co-authored-by: danisalvaa <159898202+danisalvaa@users.noreply.github.com>
Co-authored-by: Just R <remixingmagelang@gmail.com>
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
Co-authored-by: abhay23-AI <abhaytrivedi22@gmail.com>
* fix(galileo): use ingest traces API and standard logging payload
Switch hosted Galileo logging to /ingest/traces with nested trace/span payloads, read metrics from standard_logging_object, and include cost and total tokens on trace metrics.
Co-authored-by: Cursor <cursoragent@cursor.com>
* fix(galileo): route username/password auth to v2 traces ingest
Hosted Galileo no longer serves /observe/ingest; JWT login should post the same trace payload to /v2/projects/{project_id}/traces.
Co-authored-by: Cursor <cursoragent@cursor.com>
* fix(galileo): address Greptile review on logging and timestamps
Use debug-level logs for per-request Galileo callback messages and fall back to start_time/end_time when standard_logging_object omits startTime/endTime.
Co-authored-by: Cursor <cursoragent@cursor.com>
* feat(galileo): add Galileo to proxy UI callback configuration
Expose Galileo in the admin callback selector and config APIs so credentials can be configured through the dashboard instead of YAML only.
Co-authored-by: Cursor <cursoragent@cursor.com>
* fix(galileo): align response type logging with Langfuse
Mirror Langfuse input/output handling for rerank, speech, transcription,
realtime, pass-through, and other response types so Galileo ingest no longer
skips supported call types.
Co-authored-by: Cursor <cursoragent@cursor.com>
* fix(galileo): redact trace payload in debug logs and format with black
Avoid logging prompts and model responses in flush debug output while
keeping structural metadata for troubleshooting.
Co-authored-by: Cursor <cursoragent@cursor.com>
* fix(galileo): stop logging full trace payload in debug output
Log only flush URL and trace count so prompts and model responses are not
written to application logs when debug logging is enabled.
Co-authored-by: Cursor <cursoragent@cursor.com>
* Fix Galileo token totals and prompt messages
---------
Co-authored-by: Cursor <cursoragent@cursor.com>
* feat(agents): add LangFlow agent provider with A2A session bridging
Register LangFlow as a completion provider and agent type (UI + /api/v1/run),
and map A2A contextId to LangFlow session_id for multi-turn conversations.
Co-authored-by: Cursor <cursoragent@cursor.com>
* docs(providers): document langflow in provider_endpoints_support.json
Co-authored-by: Cursor <cursoragent@cursor.com>
* fix(agents): address Greptile review for LangFlow integration
Move A2A contextId→session_id mapping into LangFlow A2A provider config,
add langflow.svg logo, remove live integration test, use model for token count.
Co-authored-by: Cursor <cursoragent@cursor.com>
* fix(langflow): prevent flow_id override via request optional_params
Derive flow_id only from the authorized model name and reject flow_id
kwargs so callers cannot invoke a different LangFlow run endpoint.
Co-authored-by: Cursor <cursoragent@cursor.com>
* refactor(langflow): remove redundant flow_id branch in _get_flow_id
* fix(langflow): surface an error when the run response has no extractable message
Previously the response parser returned the raw JSON blob as the assistant
message when it could not find message text, silently presenting an
unparseable payload as a valid answer. It now returns None and the caller
raises a LangFlowError so the failure is visible to the client.
* fix(langflow): URL-encode flow_id path segment to prevent path injection
flow_id is taken from the model suffix and interpolated into
/api/v1/run/{flow_id}. Without path-segment encoding a model such as
langflow/../../x (or one containing ?) could move the request off the run
endpoint to another path on the configured LangFlow server using the
operator x-api-key. Encode the segment with quote(safe="") so it always
stays a single path segment.
* fix(langflow): reject empty flow_id from model name
* fix(langflow): return stripped flow_id so validation matches URL path
* fix(langflow): reject caller-supplied tweaks to prevent flow component override
* fix(langflow): reject caller-supplied tweaks injected via extra_body
The transform_request guard only inspected optional_params, but extra_body
is popped before transform_request runs and merged into the request body
afterward, letting a caller reintroduce tweaks and override the
operator-configured LangFlow flow components. Validate the final request
body in sign_request so tweaks cannot reach LangFlow through extra_body.
* test(langflow): move provider tests into mirrored coverage path
The langflow tests lived under tests/llm_translation/, whose CircleCI job
runs without --cov and uploads nothing to Codecov, so none of the new
langflow code counted toward patch coverage (codecov/patch reported 9.78%
of the diff hit against a 70.83% target).
Relocate them to tests/test_litellm/llms/langflow/, which the GitHub
Actions provider job runs with --cov=./litellm and uploads, and add
regression tests for the previously untested happy paths (transform_response
building the ModelResponse with usage, non-JSON body handling, last-user
message extraction, outputs-dict response shape, sign_request pass-through,
error class and stream flags). Patch coverage on the diff is now ~88%.
* fix(langflow): require litellm_params in A2A config instead of silent empty fallback
* fix(langflow): scope A2A session_id to the authenticated key
The LangFlow A2A bridge used the LangFlow session_id verbatim from the
client-controlled A2A contextId, so two distinct virtual keys authorized for
the same agent could read or append to each other's LangFlow conversation
memory by reusing a contextId.
Hand the authenticated key hash to the completion bridge through litellm_params
and namespace the forwarded session_id with it. The same key keeps a stable
session across turns, while different keys can no longer collide on a shared
contextId. The principal is hashed before it is embedded in the session_id, so
the stored token is never sent to the LangFlow backend; the original contextId
is preserved as a suffix for operator-side correlation.
* fix(langflow): wire authenticated key hash through A2A bridge and tests
Define A2A_USER_API_KEY_HASH_PARAM in the completion bridge handler, strip it
before litellm.acompletion, inject the authenticated key hash at the proxy A2A
endpoint, and add regression tests for per-key LangFlow session scoping.
---------
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
* Cato Networks guardrail, based on Aim (#26597)
* Aim was acquired by Cato Networks, creating Cato Networks guardrail based on Aim
* Add more tests
* Move test so they are reached by codecov coverage
* base URL trailing slashes
* Support Lemonade runtime context metadata (#28135)
* Support Lemonade runtime context metadata
* Add provider hook for runtime model metadata
* Address provider model info review feedback
Keep the runtime model info hook duck-typed instead of extending the base model-info class, and avoid importing ModelInfoBase from Ollama common utilities to reduce CodeQL cyclic-import noise.
Co-authored-by: openhands <openhands@all-hands.dev>
* Fix CI after staging rebase
Relax the Ollama runtime metadata return annotation to match the provider-hook dict response and update the Google Interactions OpenAPI status expectation for the current live spec.
Co-authored-by: openhands <openhands@all-hands.dev>
* Normalize Lemonade runtime model metadata
* Avoid leaking Ollama metadata auth
* Avoid leaking Lemonade metadata auth
---------
Co-authored-by: Graham Neubig <398875+neubig@users.noreply.github.com>
Co-authored-by: openhands <openhands@all-hands.dev>
* fix(cato): address guardrail review feedback
Use proxy-authenticated user identity, forward moderation hook return values,
and ensure streaming sender tasks are cancelled and awaited on exit.
Co-authored-by: Cursor <cursoragent@cursor.com>
* fix(vertex_ai): route google/gemma-*-maas through partner-models OpenAI path - clone of #28010 (#28846)
* fix(vertex_ai): route google/gemma-*-maas through partner-models OpenAI path
Fixes#26083
vertex_ai/google/gemma-4-26b-a4b-it-maas previously fell through to the
NON_GEMINI route. Per owtaylor's plan on #26083: add the google/gemma-
prefix to PartnerModelPrefixes so is_vertex_partner_model picks it up
and should_use_openai_handler routes it to the OpenAI-compatible
/endpoints/openapi/chat/completions URL. No gemma-detection exclusion
needed (the "gemma/" check uses a slash, which google/gemma-... doesn't
match). No OpenAIGPTConfig subclass needed — works with the base handler.
* fix(vertex_ai): mark gemma-4-26b-a4b-it-maas as vision-capable (empirically verified)
* fix(vertex_ai): address greptile feedback — provider category, canonical URL, sync backup
* test(vertex_ai): add function-calling and vision pass-through tests for Gemma MaaS
Addresses oss-pr-review-agent-shin feedback on PR #28010:
supports_function_calling, supports_tool_choice, and supports_vision were
marked true but had no tests proving the payloads actually reached the
OpenAI-compatible endpoint.
Added:
- test_gemma_maas_supports_function_calling — verifies the utility returns True
when the model_cost entry carries supports_function_calling=true
- test_gemma_maas_supports_vision — same for supports_vision
- test_vertex_ai_gemma_function_calling_passthrough — verifies tools + tool_choice
appear in the JSON body POSTed to /endpoints/openapi/chat/completions
- test_vertex_ai_gemma_vision_passthrough — verifies image_url content parts
survive transformation and reach the global endpoint URL
* fix: Delete uv.lock
* test(vertex_ai): add function-calling and vision pass-through tests for Gemma MaaS
Addresses oss-pr-review-agent-shin feedback on PR #28010:
P1 (patch target): Added a comment explaining why patching
litellm.llms.custom_httpx.http_handler.AsyncHTTPHandler is correct —
get_async_httpx_client() (defined in http_handler.py) instantiates
AsyncHTTPHandler within that module's scope, so the definition-site patch
intercepts it. Without the mock the test raises AuthenticationError,
confirming it never silently passes.
P2 (partner-provider regression guard): Added
test_gemma_routes_through_openai_handler() which calls
VertexAIPartnerModels.should_use_openai_handler() directly, so if Gemma's
routing to VertexPartnerProvider.llama ever changes the URL-shape tests
below it become a real regression guard rather than an unanchored unit test.
Also added:
- test_gemma_maas_supports_function_calling / supports_vision — capability
flag checks via patch.dict(litellm.model_cost)
- test_vertex_ai_gemma_function_calling_passthrough — tools + tool_choice
forwarded in the request body
- test_vertex_ai_gemma_vision_passthrough — image_url part survives
transformation to the global endpoint
Added:
- test_gemma_maas_supports_function_calling — verifies the utility returns True
when the model_cost entry carries supports_function_calling=true
- test_gemma_maas_supports_vision — same for supports_vision
- test_vertex_ai_gemma_function_calling_passthrough — verifies tools + tool_choice
appear in the JSON body POSTed to /endpoints/openapi/chat/completions
- test_vertex_ai_gemma_vision_passthrough — verifies image_url content parts
survive transformation and reach the global endpoint URL
* fix: proper patch for unit tests
---------
Co-authored-by: Iana <iana@Shivakumars-MacBook-Pro.local>
* fix(cato): guardrail all completion choices on output
When n > 1, only choices[0] was analyzed and redacted. Iterate every
Choices entry so block and anonymize actions apply to all completions.
Co-authored-by: Cursor <cursoragent@cursor.com>
* Fix review
* fix(cato_networks): harden output anonymize handling and restructure nested UI routes
Guard against empty redacted_output and empty all_redacted_messages from Cato.
Restructure nested admin UI HTML exports to index.html so extensionless routes work.
Co-authored-by: Cursor <cursoragent@cursor.com>
* Fix mypy
* fix(cato): guard missing policy_drill_down and all_redacted_messages keys
* fix(cato): avoid KeyError bypassing block action on missing analysis_result
* fix(cato): preserve non-text message fields during anonymize
Rebuild redacted messages from the original messages, overwriting only
content, so tool_calls, tool_call_id, name and multimodal fields survive
the anonymize action.
* fix(cato): preserve trailing messages when fewer redacted messages returned
Avoid silently truncating the conversation in _anonymize_request when Cato
returns fewer redacted messages than were sent, and isolate the no-api-key
config test from a pre-existing CATO_API_KEY environment variable.
* fix(cato,model-info): preserve stream block signal on sender teardown; forward api_key in dynamic model-info lookup
Suppress ConnectionClosed (alongside CancelledError) when tearing down the
Cato streaming sender task so a backend ConnectionClosed cannot mask the
original StreamingCallbackError (e.g. a guardrail block) raised by the
receive loop.
Thread api_key through get_model_info -> _get_model_info_helper so an
explicit key reaches a provider's dynamic get_model_info for a caller-supplied
api_base. Previously only api_base was forwarded, so authenticated Ollama and
Lemonade servers at a custom base could only be queried unauthenticated.
* fix(cato): surface mid-stream forwarding errors instead of blocking on recv
If the upstream LLM stream errors mid-flight, the sender task dies before
sending the terminal done frame, so the consumer would block on websocket.recv()
until Cato closes the connection. Race recv against the sender task and raise the
stored sender exception promptly as a StreamingCallbackError.
* fix(cato): drop spoofable end_user_id from guardrail user identity
Only the key/JWT-bound user_email is a trusted identity. end_user_id is
resolved from caller-supplied request fields (OpenAI user param, headers,
metadata), so an authenticated caller with no bound user_email could set it
to another user's email and have LiteLLM forward x-cato-user-email for that
victim, poisoning Cato audit and policy attribution. Forward only user_email
and omit the header otherwise.
* fix(cato): harden output anonymize path against missing content key
* fix(cato): fall back to original message when redacted content key is missing
* refactor(model-info): drop unused api_key from cached model-info helper
_cached_get_model_info_helper is only called by the cost-tracking hot path,
which never authenticates, so the api_key parameter was never populated.
Keeping it in the lru_cache key offered no benefit and risked fragmenting
the high-RPS cache and retaining credential strings per entry.
* fix(cato): preserve None content on tool-call-only choices in output hook
* fix(ollama): respect static-model guard in OllamaConfig.get_model_info
Delegate to OllamaModelInfo.get_model_info so statically-priced Ollama
models short-circuit before the /api/show network call instead of
hitting the server unconditionally.
* fix(lemonade,ollama): treat empty api_key as unset to avoid leaking server creds
An empty-string api_key was treated as an explicit key, so it passed the
guard meant to keep server-side credentials off caller-supplied bases and
then fell back through the env/global key chain. A caller could point
api_base at a server they control and send api_key="" to receive the
configured provider key in the Authorization header. Gate the credential
fallback on the api_key being truthy instead of merely not-None.
* fix(cato): inspect and redact Responses-API input, not just messages
The guardrail only read data["messages"], so /v1/responses requests, which
carry their text in data["input"], reached Cato as an empty message list
and bypassed inspection entirely. Send build_inspection_messages(data) so
both shapes are analyzed, and write anonymized results back with
apply_redacted_messages_back when the request used input.
* perf(utils): keep api_key out of get_model_info lru_cache key
* fix(cato): propagate ssl_verify to streaming WebSocket connection
The streaming hook applied ssl_verify only to the HTTP handler; the
websockets.connect() call used default verification, so a custom Cato
instance behind TLS with a self-signed cert worked for non-streaming
calls but failed every streaming request. Resolve the ssl_verify setting
into the connect() ssl argument, mirroring the HTTP handler.
* refactor(utils): rename shadowing local in _get_model_info_helper
* fix(cato): flatten multimodal chat content before inspection
Chat Completions requests whose message content is a multimodal parts
array were posted to Cato as the raw OpenAI parts, so text inside
content: [{"type":"text", ...}] reached the model without Cato ever
inspecting the string. Flatten each message's list content to plain text
while keeping the list 1:1 with the request so the index-based redaction
write-back stays valid; Responses-API input requests still go through
build_inspection_messages.
* test(lemonade): clear get_model_info cache around api_base test
* fix(cato): inspect and redact Responses-API input even when messages present
_inspection_messages returned early once messages was non-empty, so a
/v1/responses caller could place benign text in messages and disallowed
text in input and have only messages reach Cato while the model used
input. Inspect both fields and write anonymize redactions back to input
as well as the index-aligned messages.
* test(log_db_metrics): assert table_name event_metadata contract
log_db_metrics now emits minimal event_metadata via _safe_db_event_metadata
(table_name only, function_name/function_kwargs/function_args dropped as
redundant with call_type and unsafe to stamp on a span). The success-path
test still asserted function_name membership and crashed with TypeError on
the None metadata returned when no table_name is passed. Pass a table_name
and assert the surfaced contract instead.
* fix(cato): inspect and redact completion prompt and Responses-API instructions
The Cato guardrail only inspected chat messages and the Responses-API input field, so blocked text placed in the legacy /v1/completions prompt or the /v1/responses instructions field reached the model without ever being sent to Cato. Both fields are now appended as synthetic inspection messages, and the anonymize path slices Cato's redactions back to the field they came from.
* fix(cato): serialize non-str/bytes websocket chunks before forwarding
* fix(cato): inspect tool descriptions and tool-call arguments
* fix(cato): map redacted output by assistant index; restore get_model_info.cache_info
* fix(cato): block output even when detection_message is null/empty
A block_action returned by Cato on the output hook whose detection_message
was null or empty was let through to the caller: the truthiness guard on
detection_message skipped the HTTPException and the unblocked response was
returned. Raise the HTTPException directly in _handle_block_action_on_output
so the output path blocks unconditionally, mirroring the input path.
* fix(cato): inspect and redact nested tool param and legacy function descriptions
Tool/function parameter descriptions and the legacy functions[] array are
forwarded to the model but were not seen by Cato, so blocked text hidden there
bypassed inspection and anonymization. Recursively walk every description string
in tools[].function and functions[] schemas for both the analyze payload and the
anonymize write-back.
* fix(cato): traverse schema descriptions iteratively to satisfy recursive detector
The nested walk() generator recursed over tool/function JSON schemas with no
depth bound, which the recursive_detector code-quality gate rejects. Replace it
with an explicit-stack DFS that yields the same (container, key) refs in the
same pre-order, so schema description redaction is unchanged.
* fix(cato): inspect and redact response_format JSON schema descriptions
response_format json_schema descriptions are forwarded to the model, so
blocked text hidden in nested schema descriptions could bypass Cato
inspection and redaction. Extend the schema-description walk to cover
response_format alongside tools and legacy functions.
* fix(cato): skip output rewrite when Cato returns no redaction
Return None from call_cato_guardrail_on_output on monitor/no-action so the
post-call hook only mutates the message when there is an actual redaction,
instead of redundantly re-writing the original content.
* refactor(utils): resolve explicit api_key model info without the cache
Move the model-info build into a non-cached _build_model_info helper and drop
api_key from the lru-cached _cached_get_model_info signature. Both cached
helpers now take the same (model, provider, api_base) key and never forward
api_key, while explicit per-caller keys are resolved through the builder
directly instead of reaching into the cache wrapper's __wrapped__.
* fix(cato): inspect and redact non-description schema string values
Tool, function and response_format JSON schemas forward more than just
description text to the model. enum, const, default, examples and title
values are sent verbatim, so blocked content hidden in any of them
bypassed Cato inspection and redaction. Walk those schema string values
alongside descriptions on both the inspection and anonymize paths.
* fix(model-info): surface swallowed dynamic model-info errors
The provider-specific get_model_info dispatch falls back to the static cost
map when a provider's dynamic lookup raises, which is intentional graceful
degradation. Previously the exception was discarded with a bare debug line,
so a real failure (e.g. a provider whose get_model_info signature does not
accept api_key) was invisible. Log the exception at warning level with the
model and provider context so the fallback is diagnosable.
* fix(cato): inspect and redact Responses API output in post-call hook
The post-call success hook only handled ModelResponse, so /v1/responses
(which returns a ResponsesAPIResponse) bypassed the Cato output guardrail.
Extract and inspect/redact every output_text content block and function-call
arguments string, blocking on a block action, so generated text cannot escape
inspection by using the Responses API.
* chore: reset _experimental/out folder
* chore(ui): remove orphaned prebuilt dashboard chunk files
The _experimental/out manifests are byte-identical to the base branch, so the
served dashboard already matches base. 436 unreferenced Next.js chunk files had
accumulated in the directory and are not loaded by any manifest; removing them
restores the committed UI artifacts to the base build and drops the artifact
churn from this PR's diff.
* fix(guardrails,ollama): forward ssl_verify to Cato init and raise_for_status on /api/show
---------
Co-authored-by: Alex Yaroslavsky <trexinc@gmail.com>
Co-authored-by: Graham Neubig <neubig@gmail.com>
Co-authored-by: Graham Neubig <398875+neubig@users.noreply.github.com>
Co-authored-by: openhands <openhands@all-hands.dev>
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Piotr Placzko <piotr@icep-design.com>
Co-authored-by: Iana <iana@Shivakumars-MacBook-Pro.local>
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
* Add PromptGuard guardrail integration
Add PromptGuard as a first-class guardrail vendor in LiteLLM's proxy,
supporting prompt injection detection, PII redaction, topic filtering,
entity blocklists, and hallucination detection via PromptGuard's
/api/v1/guard API endpoint.
Backend:
- Add PROMPTGUARD to SupportedGuardrailIntegrations enum
- Implement PromptGuardGuardrail (CustomGuardrail subclass) with
apply_guardrail handling allow/block/redact decisions
- Add Pydantic config model with api_key, api_base, ui_friendly_name
- Auto-discovered via guardrail_hooks/promptguard/__init__.py registries
Frontend:
- Add PromptGuard partner card to Guardrail Garden with eval scores
- Add preset configuration for quick setup
- Add logo to guardrailLogoMap
Tests:
- 30 unit tests covering configuration, allow/block/redact actions,
request payload construction, error handling, config model, and
registry wiring
* Fix redact path and init ordering per review feedback
- P1: Update structured_messages (not just texts) when PromptGuard
returns a redact decision, so PII redaction is effective for the
primary LLM message path
- P2: Validate credentials before allocating the HTTPX client so
resources aren't acquired if PromptGuardMissingCredentials is raised
- Add tests for structured_messages redaction and texts-only redaction
* Harden PromptGuard integration: fail-open, event hooks, images, docs
- Add block_on_error config (default fail-closed, configurable fail-open)
- Declare supported_event_hooks (pre_call, post_call) like other vendors
- Forward images from GenericGuardrailAPIInputs to PromptGuard API
- Wrap API call in try/except for resilient error handling
- Add comprehensive documentation page with config examples
- Register docs page in sidebar alongside other guardrail providers
- Expand test suite from 32 to 40 tests covering new functionality
* Fix dict[str, Any] -> Dict[str, Any] for Python 3.8 compat
* Address remaining Greptile feedback: timeout, redact guard
- Add explicit 10s timeout to async_handler.post() to prevent
indefinite hangs when PromptGuard API is unresponsive
- Guard redact path: only update inputs["texts"] when the key
was originally present, avoiding phantom key injection
- Add test: redact with structured_messages only does not create
texts key (41 tests total)
* Fix CI lint: black formatting, add PromptGuardConfigModel to LitellmParams
- Reformat promptguard.py to match CI black version (parenthesization)
- Add PromptGuardConfigModel as base class of LitellmParams for proper
Pydantic schema validation, consistent with all other guardrail vendors
- Use litellm_params.block_on_error directly (now a typed field)
* Address Greptile review: redact path, null decision, error context
- P1: Filter _extract_texts_from_messages to user-role messages only,
preventing system/assistant content from being injected into texts
- P1: Strengthen test_redact_updates_structured_messages assertion from
weak `in` check to strict equality, catching the injection bug
- P2: Use `result.get("decision") or "allow"` to handle explicit null
decision values (not just absent keys)
- P2: Wrap bare exception re-raise in GuardrailRaisedException so the
caller knows which guardrail failed (block_on_error=True path)
- P2: Add static Promptguard entry in guardrail_provider_map so the
preset works before populateGuardrailProviderMap is called
- Add test for explicit null decision treated as allow
* Fix black formatting: collapse f-string in error message
* akto guardrails support in litellm
* docs(guardrails): add akto to supported values in types/guardrails.py
* frontend changes + fixes
* feat(akto): update Akto guardrail integration with new configuration options and modes
* docs(akto): enhance Akto documentation and configuration descriptions for clarity
* feat(tests): add proxy server request headers to sample request data
* refactor(akto): remove optional account and VXLAN IDs; update documentation and tests
* feat(akto): add event_type parameter for enhanced observability in guardrail logging
* refactor(akto): update environment variable references
* refactor the python codes
* refactor and fix linting
* refactor(akto): remove unused event hook and clean up imports
* refactor(akto): enhance AktoGuardrail with async support and improved logging
* fix: Register DynamoAI guardrail initializer and enum entry (#23752)
* fix: Register DynamoAI guardrail initializer and enum entry
Fix the "Unsupported guardrail: dynamoai" error by:
1. Adding DYNAMOAI to SupportedGuardrailIntegrations enum
2. Implementing initialize_guardrail() and registries in dynamoai/__init__.py
The DynamoAI guardrail was added in PR #15920 but never properly registered
in the initialization system. The __init__.py was missing the
guardrail_initializer_registry and guardrail_class_registry dictionaries
that the dynamic discovery mechanism looks for at module load time.
Fixes#22773
Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>
* Update litellm/proxy/guardrails/guardrail_hooks/dynamoai/__init__.py
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
* Update litellm/proxy/guardrails/guardrail_hooks/dynamoai/__init__.py
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
* test: Add tests for DynamoAI guardrail registration
Verifies enum entry, initializer registry, class registry,
instance creation, and global registry discovery.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
---------
Co-authored-by: Claude Haiku 4.5 <noreply@anthropic.com>
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
* docs: add v1.82.3 release notes and update provider_endpoints_support.json (#23816)
* Revert "docs: add v1.82.3 release notes and update provider_endpoints_support…" (#23817)
This reverts commit 966124966f.
* Refactor Akto guardrail configuration and tests; update UI description and tags
* add account and vxlan ID parameters to Akto guardrail initialization; update Akto logo format
* enhance Akto guardrail documentation and improve error handling for non-JSON responses
* address greptile issues
* fix: update payload handling to use 'data' instead of 'json' in AktoGuardrail and adjust tests accordingly
---------
Co-authored-by: Harshit Jain <48647625+Harshit28j@users.noreply.github.com>
Co-authored-by: Claude Haiku 4.5 <noreply@anthropic.com>
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
Co-authored-by: Joe Reyna <joseph.reyna@gmail.com>
Co-authored-by: Krish Dholakia <krrishdholakia@gmail.com>
* feat(ui): add MCP server logo support across admin and chat UIs
- New MCPLogoSelector component with grid of well-known logos (GitHub,
Slack, Notion, Linear, Jira, etc.) and custom URL input
- Create MCP Server form: logo picker with preview, OpenAPI presets
auto-fill logo from registry icon_url
- Edit MCP Server form: logo picker pre-populated from mcp_info.logo_url
- Admin table: logos rendered next to server name in Name column
- Chat MCPAppsPanel: logos on server cards (list + detail view) with
graceful fallback to letter avatars
- Chat MCPConnectPicker: logos next to server names in toggle list
- Fix pre-existing bug: setTools -> clearTools in create form cancel
- All 321 vitest files / 3211 tests pass
Co-authored-by: Ishaan Jaff <ishaan-jaff@users.noreply.github.com>
* feat(ui): use local SVG logos for MCP services, fix Chat UI rendering
- Add 15 new MCP service logo SVGs (Slack, Notion, Linear, Jira, Figma,
Gmail, Stripe, Salesforce, Shopify, HubSpot, Twilio, Sentry, Zapier,
GitLab, Google Drive) to both source and pre-built directories
- Switch MCPLogoSelector from CDN URLs (cdn.simpleicons.org) to local
asset paths (/ui/assets/logos/) for reliable rendering
- Logos now served by the proxy itself, working from any page path
including /ui/chat/ (absolute paths resolve correctly everywhere)
Co-authored-by: Ishaan Jaff <ishaan-jaff@users.noreply.github.com>
---------
Co-authored-by: Cursor Agent <cursoragent@cursor.com>
Co-authored-by: Ishaan Jaff <ishaan-jaff@users.noreply.github.com>
* feat(ui): add missing provider logos and map all backend providers to UI
- Downloaded 26 SVG logos from lobehub/lobe-icons for providers that were
missing visual branding (AI21, Baseten, Cloudflare, GitHub, Huggingface,
Hyperbolic, Lambda, LM Studio, Meta Llama, Moonshot, Nebius, Novita,
Nvidia NIM, Replicate, Recraft, Topaz, V0, Vercel, Watsonx/IBM,
Xinference, Friendli, Morph, Cometapi, Featherless, Langfuse, GitHub Copilot)
- Extended Providers enum from 47 to 107 entries to cover all backend
providers from provider_create_fields.json
- Extended provider_map to map all new enum keys to litellm_provider values
- Extended providerLogoMap to assign logos to all providers where available,
reusing parent logos for variants (e.g. Anthropic Text -> anthropic.svg)
- Fixed SVG currentColor issue: replaced fill='currentColor' with explicit
colors since CSS inheritance doesn't work in <img> elements
- Updated test reference from Providers.Watsonx to Providers.WATSONX
Co-authored-by: Ishaan Jaff <ishaan-jaff@users.noreply.github.com>
* docs(agents): add UI dashboard dev notes to Cursor Cloud instructions
Co-authored-by: Ishaan Jaff <ishaan-jaff@users.noreply.github.com>
* refactor(ui): remove non-LLM providers from Add Model dropdown
Remove Custom, Custom OpenAI, GitHub, Humanloop, Langfuse, Litellm Proxy,
and Milvus from the Providers enum, provider_map, and providerLogoMap.
These are not LLM API providers (they are internal tools, vector stores,
or observability platforms) and should not appear in the Add Model form.
Co-authored-by: Ishaan Jaff <ishaan-jaff@users.noreply.github.com>
---------
Co-authored-by: Cursor Agent <cursoragent@cursor.com>
Co-authored-by: Ishaan Jaff <ishaan-jaff@users.noreply.github.com>
- Downloaded official MiniMax logo from HuggingFace repository
- Added minimax.svg to assets/logos directory
- Updated providerLogoMap to reference the logo
- Logo source: https://huggingface.co/MiniMaxAI/MiniMax-VL-01
* add prompt security guardrails provider
* cosmetic
* small
* add file sanitization and update context window
* add pdf and OOXML files support
* add system prompt support
* add tests and documentation
* remove print
* fix PLR0915 Too many statements (96 > 50)
* cosmetic
* fix mypy error
* Fix failed tests due to naming conflict of responses directory with same-named pip package
* Fix mypy error: use 'aembedding' instead of 'embeddings' for async embedding call type
* Fix: Install enterprise package into Poetry virtualenv for tests
The GitHub Actions workflow was installing litellm-enterprise to system Python
using 'python -m pip install -e .', but tests run in Poetry's virtualenv using
'poetry run pytest'. This caused ImportError for enterprise package types.
Changed to 'poetry run pip install -e .' so the package is available in the
same virtualenv where pytest executes.
Fixes enterprise test collection errors in GitHub Actions CI.
* Move Prompt Security guardrail tests to tests/test_litellm/
Per reviewer feedback, move test_prompt_security_guardrails.py from
tests/guardrails_tests/ to tests/test_litellm/proxy/guardrails/ so
it will be executed by GitHub Actions workflow test-litellm.yml.
This ensures the Prompt Security integration tests run in CI.
---------
Co-authored-by: Ori Tabac <oritabac@prompt.security>
Co-authored-by: Vitaly Neyman <vitaly@prompt.security>
* add logos to callback list
* added logos
* minor
* cleanup console.log + remove unused functions + prettier
* more cleanup
* fix braintrust logo
* minor
* Add Deepgram provider to supported providers list and mappings
* add logo
* Add deepgram to model cost map
* ui - require api key for deepgram
* fix logo path
---------
Co-authored-by: Cursor Agent <cursoragent@cursor.com>
* ui fix bedrock guard
* polish: logo should appear after selecting provider
* fix ui config bedrock
* fix: refactor - use specific configs per provider
* fix: refactor - use specific configs per provider
* feat: ui, show provider specific params for guardrails
* fix: updated type of LiteLLM params for guardrails
* fix: updated type of LiteLLM params for guardrails
* ui, use endpoint for adding presidio, bedrock guardrails
* fix: linting error
* add llama guard and secret detector on UI
* add aim on ui
* allow adding lakera AI on litellm ui
* fix: fixes for params to init guardrails
* test: test_guardrail_info_response
* test: test_initialize_presidio_guardrail
* fix: init guardrails
* fix: init guardrails
* add showSearch
* working bedrock guard
