Address greptile's open thread on _async_refresh_locks growing
unboundedly in high-cardinality deployments.
- Add _maybe_prune_async_refresh_lock: drops the per-key Lock from
the registry once no coroutine holds it and no coroutine is queued
in lock._waiters. The check-then-pop sequence is safe under
asyncio's cooperative scheduler — a waiter that arrives after the
pop simply creates a fresh lock under the same key, which is fine
because the previous batch is already done.
- Wrap the slow-path async with lock in a try/finally so the prune
runs on every exit (return, exception, reauth retry).
- Extract the existing background-refresh task scheduling into
_schedule_background_refresh so get_access_token_async stays under
ruff's PLR0915 ("Too many statements") limit. No behaviour change.
- Regression tests cover both pruning after release (the dict
shrinks back to zero after each call) and the safeguard that
keeps the lock alive while a waiter is still queued.
The `allowed_tools` field on `BlockedToolsResult` was computed in
`_extract_blocked_tools` but never read by the only caller — when any
tool was blocked the integration unconditionally raised
`ModifyResponseException` to reject the full response, never doing
partial filtering. Drop the dataclass and return the blocking
explanation directly as `Optional[str]` so there's no misleading shape
hinting at unused partial-filter capability.
Co-authored-by: Greptile <greptile-apps[bot]@users.noreply.github.com>
- Fireworks _get_model_cost_capability: after exact-key lookups, fall back
to substring matching against fireworks_ai/* entries in model_cost so
model name variants (e.g. fine-tuned suffixes) continue to inherit
capability flags like supports_reasoning.
- Vertex vertex_llm_base: replace non-reentrant threading.Lock with RLock
on the sync refresh path so the reauthentication retry, which recurses
into get_access_token while still holding the lock, does not deadlock
when reloaded credentials are also expired.
Co-authored-by: Yassin Kortam <yassin@berri.ai>
Address bugbot review concerns:
- Sanitize proxy_server_request before forwarding to the Rubrik webhook.
The previous code passed the entire inbound HTTP context (Authorization,
Cookie, x-api-key, and the raw request body) through to a third-party
endpoint, which exfiltrates proxy credentials and upstream secrets. The
new _sanitize_proxy_server_request allowlists only url and method.
(Cursor Bugbot HIGH severity #3192354895)
- Treat a null choices[0].message.tool_calls as 'all blocked' rather than
letting iteration raise and silently fall through the outer except in
apply_guardrail (which would fail open). Iterate over a defensive
fallback list instead of relying on the dict default.
(Cursor Bugbot MEDIUM severity #3192349538)
Co-authored-by: Cursor Bugbot <bugbot@cursor.com>
Commit dfb2524 intentionally reverted _async_refresh_locks from a
WeakValueDictionary back to a regular Dict so the per-key asyncio.Lock
identity is stable across concurrent callers — preserving
single-flight semantics. The test asserting that the dict shrinks
back to 0 after refreshes was added when the WeakValueDictionary
backing was still in place; it now contradicts the deliberate design
and is failing CI.
- cost_calculator.ocr_cost: log a warning when pages_processed is reported
but no ocr_cost_per_page is configured, instead of silently billing zero
via an implicit '(... or 0.0) * pages_processed' fallback. Behavior is
preserved (zero cost) so free-tier / unpriced models still work, but
configuration gaps are now visible in logs.
- ChatGPTResponsesAPIConfig._extract_completed_response_from_sse: also
collect response.output_text.done events into a text-only items map and
merge them into the recovered output (OUTPUT_ITEM_DONE wins on duplicate
output_index), mirroring the LiteLLMResponses handler. This recovers
text content when a provider only emits OUTPUT_TEXT_DONE and the final
response.completed event has an empty output list.
Co-authored-by: Yassin Kortam <yassin@berri.ai>
- Guard background refresh done_callback with an identity check so a
stale callback cannot remove a newer task that already replaced it in
the tracking dict (done_callbacks are scheduled via call_soon, so a
fresh task can be stored for the same credential key before the old
callback fires).
- Replace WeakValueDictionary with a regular dict for
_async_refresh_locks so the per-key asyncio.Lock identity is stable
across concurrent callers; otherwise a lock can be GC'd between two
coroutines arriving for the same key, breaking single-flight.
Co-authored-by: Yassin Kortam <yassin@berri.ai>
- vertex: back `_async_refresh_locks` with a WeakValueDictionary so a per-key
Lock is auto-evicted once no coroutine holds it, preventing unbounded growth
in deployments with many credential combinations while keeping single-flight
semantics intact.
- rubrik: defer the periodic flush task to the first log event when the logger
is constructed without a running event loop, so low-traffic batches still
get drained instead of being silently stranded by a swallowed RuntimeError.
Pull in the base-branch fix from #28320 which (1) adds the
mistral/ministral-8b-2512 cost map entry on main and (2) backfills the
in-tree local backup into litellm.model_cost in tests/local_testing/conftest.py
so cost-calculator lookups succeed even before main's hosted cost map
catches up. This unblocks the failing test_completion_mistral_api and
test_completion_mistral_api_modified_input cases in local_testing_part1.
Co-authored-by: Claude <noreply@anthropic.com>
* feat(gemini): add gemini-3.1-flash-lite model cost map entries
Co-authored-by: Cursor <cursoragent@cursor.com>
* Update model_prices_and_context_window.json
* Update source URL for model pricing information
* Sync source URL for gemini-3.1-flash-lite in backup JSON
* fix(model_cost_map): add mistral/ministral-8b-2512 entry
Mistral rotated the 'mistral/mistral-tiny' alias to return
'ministral-8b-2512' as the response model, which is not in the cost map.
This caused test_completion_mistral_api and
test_completion_mistral_api_modified_input to fail in
completion_cost lookup. Add the entry mirroring the existing
openrouter/mistralai/ministral-8b-2512 pricing.
* test(cost_calculator): assert output_cost_per_reasoning_token for gemini-3.1-flash-lite
* fix(tests): backfill local backup entries into runtime model_cost
litellm.model_cost is loaded from LITELLM_MODEL_COST_MAP_URL (pinned to
main) at import time, so any pricing entries added to the in-tree backup
on this branch aren't visible at test runtime until they also land on
main. The Mistral cassette currently returns model=ministral-8b-2512
and the cost-calculator lookup in test_completion_mistral_api /
test_completion_mistral_api_modified_input fails despite the entry
existing in the local backup. Backfill missing backup entries into
litellm.model_cost in the local_testing conftest so these lookups
succeed against the cassette state the branch is being tested with.
* fix(tests): guard conftest backfill against empty local cost map
---------
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
Mistral's API now returns model='ministral-8b-2512' when 'mistral-tiny' is requested. Adding the entry so completion_cost can resolve the cost for that response.
- fireworks: resolve supports_reasoning lookup for short model names by also
trying the full accounts/fireworks/models/ path in model_cost
- ocr_cost: drop reducto-specific guard in shared utility; treat missing
pages_processed as zero cost when no per-page pricing is configured
- docs: remove reducto/rubrik markdown stubs from this repo (canonical docs
live in litellm-docs)
Use model-map lookup with _model_supports_effort_param fallback so Bedrock
Invoke keeps output_config for Claude 4.6/4.7 when pricing flags are missing.
Revert custom_llm_provider=bedrock for supports_output_config checks, fix
allowlist test model, and apply black to xai/vertex files failing lint CI.
Co-authored-by: Cursor <cursoragent@cursor.com>
Previously RubrikLogger relied on CustomBatchLogger.flush_queue, which
captured len(self.log_queue) separately from the snapshot taken inside
async_send_batch. Although both happen without an intervening await today
(so they agree in practice), they are semantically disconnected: a future
refactor that adds an await between the two captures, or that changes the
async_send_batch contract, could cause the parent to delete a different
number of items than were actually sent and trigger duplicate deliveries
to Rubrik.
Override flush_queue on RubrikLogger so a single snapshot drives both the
HTTP POST and the queue truncation. async_send_batch is preserved for
direct callers/tests but no longer participates in the canonical flush
path. Existing tests (including the one that explicitly invokes the base
CustomBatchLogger.flush_queue path) still pass.
Co-authored-by: Mateo Wang <mateo-berri@users.noreply.github.com>
_parse_sse_json_chunk in ChatGPTResponsesAPIConfig passed the raw chunk
directly to _strip_sse_data_from_chunk, which only matches the 'data:'
prefix at position 0. Chunks with leading whitespace (e.g. ' data: {...}')
were returned unchanged and silently failed JSON parsing, dropping the
contained event.
Mirror the existing fix in LiteLLMResponsesTransformationHandler._parse_raw_sse_chunk
by calling chunk.strip() before stripping the SSE prefix.
Adds a regression test using whitespace-padded data: lines and verifies
that the response.output_item.done payload is recovered into the final
ResponsesAPIResponse output.
Co-authored-by: Mateo Wang <mateo-berri@users.noreply.github.com>
Previously, _log_batch_to_rubrik swallowed all HTTP errors and exceptions,
and the parent flush_queue unconditionally drained the queue afterwards.
On Rubrik 5xx responses, network errors, or timeouts the in-flight events
were silently dropped without ever being delivered.
- Re-raise from _log_batch_to_rubrik so failures surface to the caller.
- In CustomBatchLogger.flush_queue, catch exceptions from async_send_batch
and leave the queue intact for retry on the next flush. Existing loggers
that override flush_queue (e.g. Datadog) or that swallow their own errors
inside async_send_batch (e.g. Langsmith, GCS, Argilla) are unaffected.
- Tests now assert events are preserved on HTTP errors, network errors,
and that mid-flush appended events are also preserved on failure.
Co-authored-by: Mateo Wang <mateo-berri@users.noreply.github.com>
When recovering output from raw SSE, OUTPUT_ITEM_DONE and OUTPUT_TEXT_DONE
events were treated as mutually exclusive fallbacks. If a stream emitted
OUTPUT_ITEM_DONE for some output indices and only OUTPUT_TEXT_DONE for
others, the text-only items at the missing indices were silently dropped.
Merge both dicts before returning, with OUTPUT_ITEM_DONE entries taking
precedence at any shared index (preserving the existing behavior covered
by test_transform_response_preserves_output_item_when_text_done_arrives_later).
Co-authored-by: Mateo Wang <mateo-berri@users.noreply.github.com>
asyncio.create_task() raises RuntimeError when called outside a running
event loop. Wrap the call in a try/except RuntimeError so that RubrikLogger
can be instantiated in synchronous contexts (e.g. during startup, testing)
without crashing. The periodic_flush background task simply won't start in
those cases; it starts normally when the constructor is called inside an
event loop.
Add a test that verifies instantiation outside an event loop does not raise
(does not patch asyncio.create_task).
Co-authored-by: Sameer Kankute <Sameerlite@users.noreply.github.com>
Completed asyncio.Task objects were never removed from
_background_refresh_tasks. In long-running proxies with many distinct
credential keys the dict grows indefinitely, retaining references to
finished tasks and their results.
Fix:
- Pop the existing (done) entry before creating a replacement task.
- Attach a done_callback to each new task that removes its entry from
the dict once the task finishes (success or failure).
Tests:
- test_background_refresh_task_removed_after_completion: verifies the
done-callback cleans up a single entry after the task completes.
- test_background_refresh_tasks_no_accumulation_across_many_keys:
drives 20 distinct credential keys and confirms the dict is empty
after all background refreshes finish.
Co-authored-by: Sameer Kankute <Sameerlite@users.noreply.github.com>
xAI can return total_tokens inconsistent with prompt_tokens +
completion_tokens when caching is enabled. Align with OpenAI-style
usage so shared LLM tests and downstream consumers see coherent totals.
Apply to non-streaming responses and streaming usage chunks.
Made-with: Cursor
* Add Rubrik as officially-supported guardrail plugin
Adds tool blocking and batch logging integration with an external Rubrik
webhook service. The plugin validates LLM tool calls against a policy
service (fail-open on errors) and batch-logs all requests/responses.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* Update Rubrik docs: config.yaml as primary, env vars as fallback
Restructures the Quick Start to present config.yaml as the recommended
approach with tabbed UI, and environment variables as an alternative
fallback.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* Add Rubrik env vars to config_settings reference
Fixes documentation validation by adding RUBRIK_API_KEY,
RUBRIK_BATCH_SIZE, RUBRIK_SAMPLING_RATE, and RUBRIK_WEBHOOK_URL
to the environment settings reference table.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* Add fallback message when blocking service returns empty explanation
Prevents whitespace-only violation message when the tool blocking
service blocks tools but returns an empty content field.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
When litellm migrated from Poetry to uv (PR #24905, v1.83.1), the core
dependency specifications in pyproject.toml changed from Poetry bare-version
strings (e.g. openai = "2.30.0") to PEP 621 exact pins (openai==2.24.0).
Poetry bare-version strings are actually caret ranges (^X.Y.Z == >=X.Y.Z,<X+1),
but PEP 621 == is exact. This means every downstream package that installs
litellm as a library dependency is now forced to downgrade aiohttp, pydantic,
openai, click, and 8 other common packages to exact old versions.
Fix: restore range specifiers for the 12 core runtime dependencies. The
optional extras (proxy, proxy-runtime, etc.) are consumed primarily by
Docker images where exact pins are appropriate and are left unchanged.
The uv.lock file continues to provide exact reproducibility for Docker
builds and CI.
Fixes: #26154
- preserve existing shared backend `mode` when router deployment registration
reuses a provider/model key already in `litellm.model_cost` (prevents alias
with `mode: chat` from downgrading shared `chatgpt/gpt-5.4` from `responses`
to `chat` and triggering 403s on /v1/chat/completions)
- teach the ChatGPT Responses parser to recover `response.output_item.done`
entries when `response.completed.output` is empty
- add defensive /responses -> /chat/completions bridge fallback that
reconstructs output items from raw SSE when `raw_response.output` is empty
- regression coverage for shared alias routing, empty completed.output
parsing, and SSE bridge recovery
Closes#25403
Co-authored-by: afoninsky <andrey.afoninsky@gmail.com>
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix: reuse cached credentials in VertexAIPartnerModels instead of creating new VertexLLM per request
VertexAIPartnerModels.completion() was creating a throwaway VertexLLM()
instance on every call to get an access token, bypassing the credential
cache inherited from VertexBase. This caused a fresh token fetch for
every single request, adding significant latency overhead.
Fix: call super().__init__() to initialize VertexBase's credential cache,
and use self._ensure_access_token() instead of a new VertexLLM instance.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: apply same credential caching fix to VertexAIGemmaModels and VertexAIModelGardenModels
Same bug as VertexAIPartnerModels: both classes had `pass` in __init__
instead of `super().__init__()`, and created throwaway VertexLLM()
instances per request instead of using self._ensure_access_token().
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix(vertex_ai): single-flight credential refresh to prevent thundering herd
When GCP credentials expire under high concurrency, all requests
simultaneously call credentials.refresh() via asyncify, saturating the
40-thread anyio pool and blocking the proxy for 20+ seconds.
This adds:
- Per-credential asyncio.Lock in get_access_token_async for single-flight
refresh (1 coroutine refreshes, others wait on the lock)
- Background refresh when token_state is STALE (usable but near expiry),
returning the current token immediately with zero added latency
- threading.Lock on the sync get_access_token path
- Uses google-auth's TokenState enum (FRESH/STALE/INVALID) instead of
reimplementing expiry logic
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: address PR review comments
- Use asyncio.create_task() instead of deprecated get_event_loop().create_task()
- Track in-flight background refresh tasks to prevent duplicate refreshes
when multiple STALE-path callers pass through the lock before the first
background task completes
- Add token validation in the STALE branch (consistent with FRESH/INVALID)
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: lazy-import TokenState to avoid breaking when google-auth is not installed
Also extract helper methods to bring get_access_token_async under the
PLR0915 statement limit (50).
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* chore: apply Black formatting to test file and update uv.lock
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: remove user-provided project_id from log messages (CodeQL log injection)
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: avoid leaking token value in error message, log type instead
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* chore: restore uv.lock to match litellm_oss_branch
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: remove project_id from remaining log message (CodeQL log injection)
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: remove remaining project_id from log and error messages
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Replace hardcoded _is_claude_4_6_model() string matching with
supports_output_config flag in model_prices_and_context_window.json,
accessed via _supports_factory(). This follows the project's established
pattern for model capability checks (per AGENTS.md rule #8).
Bedrock Invoke now conditionally preserves output_config for models
that declare supports_output_config=true (currently Claude 4.6 models),
while stripping it for older models to avoid request rejection.
Ref: https://github.com/BerriAI/litellm/issues/22797
* Add support for environment variable in interactions api
* Add sdk support for gemini create agent
* Add agents endpoint support via proxy
* Add outputs of each api
* Add routing for model and agents param
* Remove redundant condition in get_provider_agents_api_config
LlmProviders.GEMINI.value is literally the string "gemini", so the
second clause of the or was checking the exact same thing as the first.
Co-authored-by: Sameer Kankute <Sameerlite@users.noreply.github.com>
* fix: forward query-param credentials to list/get/delete/versions Gemini agent endpoints
The list_gemini_agents, get_gemini_agent, delete_gemini_agent, and
list_gemini_agent_versions endpoints previously constructed a hardcoded
data dict with no mechanism to pass provider credentials. Unlike
create_gemini_agent (POST, reads litellm_params_template from body),
these GET/DELETE endpoints gave no way for multi-tenant callers to
supply a per-request api_key or other LiteLLM params.
Fix:
- Add _merge_query_params_into_data() helper that reads query parameters
from the request and merges them into the data dict without overwriting
already-set keys (e.g. path params like 'name').
- Support a JSON-encoded litellm_params_template query parameter
(matching the POST body pattern) as well as flat key=value pairs
(e.g. api_key=AIza...).
- Apply the helper in all four affected endpoints.
- Add 13 unit tests covering the helper and each endpoint.
Co-authored-by: Sameer Kankute <Sameerlite@users.noreply.github.com>
* fix: pass model=None for managed agent proxy endpoints to prevent agent name polluting data["model"]
Endpoints acreate_agent, aget_agent, adelete_agent, and alist_agent_versions
were passing model=<agent_name> to base_process_llm_request. This caused
common_processing_pre_call_logic to write the agent name into self.data["model"],
which then triggered spurious model-alias mapping, rate-limiting lookups, and
logging tied to a non-existent model deployment.
The agent name is already carried in data["name"] and is passed correctly to
the SDK functions (litellm.interactions.agents.*). There is no reason to also
set model=<agent_name>; the correct value is model=None for all five managed-agent
management routes.
Adds tests/test_litellm/proxy/google_endpoints/test_managed_agents_model_param.py
to verify all five managed-agent endpoints pass model=None.
Co-authored-by: Sameer Kankute <Sameerlite@users.noreply.github.com>
* fix: address greptile P1/P2 review comments
P1 (router.py): Restore fallback/retry support for acreate_interaction
and create_interaction. Both were silently moved to _init_interactions_api_endpoints
(direct call, no fallbacks). Moved them back to _ageneric_api_call_with_fallbacks
so users with configured fallback models keep retry behaviour.
P1 security (agents_endpoints.py): Remove flat query-param credential
path (e.g. ?api_key=AIza...) from _merge_query_params_into_data.
Credentials in URL query strings appear verbatim in server access logs,
CDN edge logs, and browser history. Only the JSON-encoded
litellm_params_template query param (matching the POST body pattern) is
retained.
P2 (interactions/http_handler.py): Extract _BaseHTTPHandler with shared
_handle_error, _sync_client, and _async_client helpers. InteractionsHTTPHandler
now extends _BaseHTTPHandler. The _async_client reads the provider from
litellm_params instead of hardcoding GEMINI.
P2 (interactions/agents/http_handler.py): AgentsHTTPHandler now extends
InteractionsHTTPHandler (which inherits _BaseHTTPHandler) so all shared
HTTP infrastructure is reused rather than duplicated. Removes the
hardcoded LlmProviders.GEMINI from the async client path.
Co-authored-by: Cursor <cursoragent@cursor.com>
* fix: address CI failures from greptile review fixes
- black: format interactions/agents/main.py and utils.py
- tests: update test_gemini_agents_endpoints.py to match new
_merge_query_params_into_data behaviour (flat credential params are
rejected; only JSON-encoded litellm_params_template is accepted)
- ci: add test_gemini_agents_endpoints.py to endpoints-and-responses
shard in test-unit-proxy-db.yml so assert-shard-coverage passes
- tests: add _initialize_managed_agents_endpoints and
_init_managed_agents_api_endpoints test coverage so router_code_coverage
passes; also fix TestRouterCreateInteractionRouting to reflect that
acreate_interaction now correctly routes through
_ageneric_api_call_with_fallbacks (restoring fallback support)
Co-authored-by: Cursor <cursoragent@cursor.com>
* fix: remove InteractionsHTTPHandler._handle_error override to fix type errors
AgentsHTTPHandler extends InteractionsHTTPHandler and calls
self._handle_error(provider_config=agents_api_config) where
agents_api_config is BaseAgentsAPIConfig. Python MRO resolved _handle_error
to InteractionsHTTPHandler._handle_error which expected BaseInteractionsAPIConfig,
causing 10 mypy arg-type errors in interactions/agents/http_handler.py.
Removing the redundant override lets both classes inherit _BaseHTTPHandler._handle_error
(provider_config: Any) which is structurally correct for both config types.
Co-authored-by: Cursor <cursoragent@cursor.com>
* fix: agent-only interactions and managed agents provider routing
Resolve None custom_llm_provider in agents HTTP client lookup and set
custom_llm_provider on GenericLiteLLMParams for all agent CRUD paths.
Stop mapping agent names to proxy model routing; route interactions
through _init_interactions_api_endpoints with fallbacks only when model
is set. Consolidate duplicate router elif branches for interaction APIs.
Co-authored-by: Cursor <cursoragent@cursor.com>
* Fix greptile review
* test(agents): add unit tests for managed agents SDK and HTTP handler
Adds coverage for the new `litellm.interactions.agents` surface area:
- main.py: sync/async entry points (create/list/get/delete/list_versions),
provider config lookup, logging-obj helper, async error wrapping
- http_handler.py: every CRUD method (sync + async paths), `_is_async`
dispatch branches, and provider error mapping through GeminiAgentsConfig
- utils.py: get_provider_agents_api_config for supported / unsupported
providers
Brings patch coverage on these files from <25% to ~100% so codecov/patch
is satisfied.
Co-authored-by: Mateo Wang <mateo-berri@users.noreply.github.com>
* docs(gemini-agents): fix misleading credential-passing examples in GET/DELETE docstrings (#28293)
The four GET/DELETE endpoint docstrings (list_gemini_agents,
get_gemini_agent, delete_gemini_agent, list_gemini_agent_versions)
documented passing per-request credentials as flat query parameters
(e.g. ?api_key=AIza...). However, _merge_query_params_into_data only
reads the JSON-encoded litellm_params_template query parameter and
intentionally ignores flat params (URL query strings appear verbatim
in access logs, browser history, and Referer headers).
Callers following the documented curl examples would have their
credentials silently dropped and hit auth failures against Gemini.
Update the examples to use the supported JSON-encoded
litellm_params_template query parameter, matching _merge_query_params_into_data's own docstring.
Co-authored-by: Cursor Agent <cursoragent@cursor.com>
Co-authored-by: Mateo Wang <mateo-berri@users.noreply.github.com>
* refactor(agents): rename provider-agnostic agent response types
Move GeminiAgent{ListResponse,DeleteResult,VersionsResponse} to
provider-neutral names (AgentListResponse, AgentDeleteResult,
AgentVersionsResponse) so the BaseAgentsAPIConfig interface no longer
references Gemini-specific type names.
* fix(gemini-agents): close veria-flagged credential-escalation gaps
Two high-severity findings from the veria-ai PR review are addressed:
1. **api_base override could leak the shared Gemini key**
GeminiAgentsConfig.validate_environment falls back to GOOGLE_API_KEY /
GEMINI_API_KEY when no api_key is supplied. Combined with caller-controlled
api_base on the proxy CRUD endpoints, an authenticated user could redirect
the outbound request to an attacker-controlled host and capture the
operator's shared Gemini key from the x-goog-api-key header. The config
now refuses env-fallback whenever api_base is explicitly overridden.
2. **Managed-agent CRUD exposed to ordinary LLM keys**
The new /v1beta/agents routes live in google_routes (i.e. llm_api_routes),
so any non-admin LLM key can reach them. Unlike /v1beta/models/...:
generateContent these endpoints are NOT model-routed and have no
model_list-supplied credentials, so env-fallback would let any LLM key
list / create / delete agents inside the operator's Gemini project. Each
endpoint now calls _enforce_caller_supplied_provider_key, which requires
non-admin callers to supply their own Gemini api_key via
litellm_params_template. Proxy admins keep the env-fallback convenience.
Tests cover non-admin rejection, admin allow-through, the api_base override
guard, and SDK env-fallback when api_base is not overridden.
Co-authored-by: Mateo Wang <mateo-berri@users.noreply.github.com>
* test(router): restore strict assert_called_once_with on interactions default-provider test
---------
Co-authored-by: Cursor Agent <cursoragent@cursor.com>
Co-authored-by: Sameer Kankute <Sameerlite@users.noreply.github.com>
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
Co-authored-by: Mateo Wang <mateo-berri@users.noreply.github.com>
