* feat(azure): support credential chain for storage
* test(azure): clarify credential seam suppressions
* fix(azure): read chain tokens in a worker thread
The credential chain walk (IMDS probe, CLI subprocess) is blocking I/O,
so reading the provider inline in async set_valid_azure_ad_token stalls
every request on the worker's event loop
* fix(security): restrict and validate file uploads at /v1/files and /upload/logo
Extends fast-fail upload validation to every purpose at POST /v1/files,
not just purpose=batch: a configurable max_file_size_mb size cap and a
blocked_file_extensions denylist, plus rejection of filenames carrying a
directory-traversal component before anything is read, stored, or
forwarded to a provider.
Also fixes two concrete gaps found while auditing every upload surface:
the Azure Blob Storage backend derived a blob path's extension with
filename.split(".")[-1], which does not parse path structure and let a
crafted filename embed a directory traversal sequence into the stored
blob path; and POST /upload/logo (the admin UI logo upload) had no
role check at all, so any authenticated API key, not just a proxy
admin, could write a file to the server's disk.
* fix(lint): drop cast()/mutation from settings coercion, sync blocked_file_extensions on reload
Replaces the TypeAdapter+cast() reads of max_file_size_mb and
blocked_file_extensions with small isinstance-based validators, since the
codebase's cast() budget (LIT006) had no headroom left. Also adds the
blocked_file_extensions reload block that was missing from
_update_general_settings: it was registered as an editable setting but
never re-synced into runtime state, so a value set through the DB-backed
settings editor would silently never take effect (Greptile finding).
* fix(security): declare max_file_size_mb and blocked_file_extensions on ConfigGeneralSettings
The DB-backed general-settings update endpoints validate every field
through ConfigGeneralSettings.model_fields before persisting it, so
without these declarations an operator could never actually set either
setting through that path even though both were registered for the
Admin UI's settings editor and reloaded on config refresh (Greptile
finding). blocked_file_extensions is typed as a tuple, not a list, to
stay out of the immutable-collections lint budget; the stored JSON
value is unaffected since the raw request payload, not the validated
model, is what gets persisted.
* chore: regenerate schema.d.ts for the new ConfigGeneralSettings fields
* fix(security): normalize configured blocked_file_extensions casing
check_blocked_extension lowercased the uploaded filename's extension
before comparing but compared it against blocked_extensions verbatim,
so an admin-configured blocked_file_extensions: ['.EXE'] would never
match an uploaded payload.exe (Greptile finding). Normalizes the
configured values the same way at comparison time, and adds the
missing case (mismatched-case config, lowercase upload) as a
regression test, mutation-checked against the unfixed comparison.
* fix(security): restore caller-owned stream position after size inspection
_file_size_bytes unconditionally seeked back to 0 after measuring a
BinaryIO's length, discarding wherever the caller had actually
positioned it (Greptile finding). Saves and restores the original
position instead. Rewrites the existing test that had encoded the
old "always resets to 0" behavior as its expectation, and adds a
sibling case for the under-cap path; both are mutation-checked
against the unfixed always-reset-to-0 behavior.
#38318 taught exception_type to map upstream status codes for providers with
no branch of their own. It reads the status code off the exception, but
_handle_error stamps 500 onto every failure that never carried one, so a
refused connection reached the mapper wearing a status code nothing upstream
had sent, and came back as InternalServerError instead of APIConnectionError.
The two are not interchangeable to a caller: a 5xx says the provider answered
and failed, which the router treats as a reason to cool the deployment down,
while a connection error says the request never landed.
BaseLLMException now records whether its status code was received or
synthesized, _handle_error sets that when it invents the 500, and the status
mapper declines to act on a code litellm made up, so those failures fall
through to the APIConnectionError the branch was always meant to produce.
Genuine upstream 5xx responses are untouched, which the second test pins.
The search transformation assertion #38318 had loosened to InternalServerError
goes back to APIConnectionError for the same reason.
* test: drop the cwd-relative sys.path.insert calls from the test suite
TQ003 stands at 1,077 across 1,058 files, and 1,015 of them are the same shape:
sys.path.insert(0, os.path.abspath("../..")) and its deeper siblings. The
argument resolves against the working directory rather than the file, so from
the repo root, where every job runs pytest, it inserts the directory two levels
above the checkout. It has never pointed at litellm. The package is installed
into the environment anyway, which is what actually makes the import work, and
what the rule's message has said all along.
Removing them leaves 1,634 imports of sys and os with no remaining reference,
and those go too, except where another test module imports the name back out of
the file. The rest of TQ003 is 62 call sites that resolve against __file__ or a
variable, which are a different question and are left alone.
Collection is identical either way: 45,871 tests and the same 51 pre-existing
collection errors before and after, and ruff reports no new undefined name.
* test: drop the duplicate imports the sys.path sweep exposed to F811
* test(pre-call-utils): restore the os import the new bedrock tests need
* test(lint): ban blind pytest.raises(Exception) with ruff B017
A bare pytest.raises(Exception) accepts whatever the body throws. The TypeError
a refactor introduces satisfies it exactly as well as the rejection the test was
written for, so the crash reads as a pass and the test never goes red.
All 111 existing sites are narrowed here. A runtime probe recorded the concrete
exception each one actually catches, and each site now names that type. Where
the code under test genuinely raises a bare Exception, the site pins a stable
slice of the message with match= instead.
Two sites tell on themselves. The shared responses-API cancel test raises
"custom_llm_provider is required but passed as None" rather than talking to a
provider at all, because cancel_responses takes a provider, not a model. And
test_bedrock_guardrails_with_streaming was the only test in its file still
passing without AWS credentials, because the NoCredentialsError boto3 raised
long before the guardrail ran satisfied the blind raises.
* fix(test): widen the openai batch-dispatch assertion to OpenAIError
The narrowed NotFoundError only holds where OPENAI_API_KEY is set. Without one
the SDK raises OpenAIError while building the client, long before any 404, so CI
went red. OpenAIError covers both and still rejects a TypeError from a refactor.
* feat(search): add Nimble as a search provider
Adds `NimbleSearchConfig` so `search_provider: nimble` works across the SDK,
the proxy /v1/search endpoint, the Search Tools dashboard, and spend tracking.
Nimble's /v2/search already uses the Perplexity unified spec's parameter names,
so the request transform is close to a pass-through. `search_domain_filter`
splits into include_domains/exclude_domains on the spec's `-` prefix, `country`
is upper-cased to the ISO form Nimble documents, and everything else is
forwarded so focus, search_depth, time_range and the rest stay reachable. On the
response side, snippet prefers `content` and falls back to `description`, and a
malformed body raises an attributed error rather than reporting an empty search.
Also tightens `BaseSearchConfig.get_supported_perplexity_optional_params` to
return `frozenset[str]` instead of a bare mutable `set`, which every caller
already treats as read-only.
* fix(search): surface Nimble error bodies instead of empty results
Greptile flagged that a null or absent `results` degraded to a successful empty
search. A search with no hits comes back as `"results": []`, verified against the
live API, so the field is now required and anything else raises the attributed
schema error the other malformed bodies already take.
Also unwraps Nimble's second error envelope. Collection failures return
`{"success", "task_id", "message"}` rather than the `{"detail"}` shape validation
errors use, and only the latter was being read.
Drops comments that restated the adjacent code.
* docs(search): drop the Nimble param list from the transform docstring
It restated the vendor's API reference, which the module docstring already links,
and would go stale the moment Nimble adds a focus mode.
The azure_storage logging callback and the azure blob files backend built every
storage URL against the hardcoded commercial host, so an Azure Government account
was unreachable with no way to override it.
Read AZURE_STORAGE_ENDPOINT_SUFFIX (default core.windows.net) once in
AzureBlobStorageLogger and derive the Data Lake and Blob hosts from it, so all
seven previously hardcoded sites follow the configured cloud. Parse stored blob
URLs with urlparse instead of matching the commercial host, so URLs persisted
before the suffix was configured still resolve, and pin the resulting
host-validation boundary with tests.
* test(batches): add 1:1 test file scaffold for batches component paths
Co-authored-by: Cursor <cursoragent@cursor.com>
* Add harness test for create batch endpoint
* Add retrieve endpoint harness tests
* Add list endpoint harness tests
* Add cancel endpoint harness tests
* Add cancel endpoint harness tests
* Add test for litellm/batches/main.py
* Add test for litellm/tests/test_litellm/batches/test_batch_utils.py
* Add handler and transformation tests for all providers
* Fix: run batches tests in cicd
* fix(tests): remove azure/__init__.py that shadowed azure namespace package
Adding __init__.py to tests/test_litellm/llms/azure/ caused pytest to
insert tests/test_litellm/llms/ into sys.path[0], making our empty
azure/ dir shadow the real azure-identity namespace package. Any test
that patched azure.identity.* would then fail with AttributeError.
* style(tests): apply ruff format to test_batch_utils.py
Base migrated the formatter from black to ruff format (#31317); reformat the
batches scaffold test file to match.
---------
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
Search providers resolved the server-configured API key (e.g.
get_secret_str("SERPER_API_KEY")) in validate_environment whenever the
caller omitted api_key, while get_complete_url independently honored a
caller-supplied api_base. A caller who passes their own api_base and no
api_key therefore made the proxy send the operator's provider key to a
host they control; POST /search_tools/test_connection forwards
request-body api_base/api_key straight into asearch, so any authenticated
user could exfiltrate the server's search credentials.
Add a shared host-aware fallback in BaseSearchConfig.resolve_server_api_key
that only applies a server-managed secret when the caller-supplied
api_base is absent or resolves to a trusted host (the provider default or
the operator's own *_API_BASE env override); otherwise it refuses and asks
for an explicit api_key. The guard only triggers when a server secret
actually exists, so keyless and self-hosted providers (searxng, you.com
free tier) keep working. Every provider that carries a server secret is
migrated to the helper; dataforseo reuses the same guard for its
login:password basic-auth credentials.
This changes behavior for callers that previously passed a per-request
api_base while relying on a server-configured key: they must now pass an
explicit api_key, or the operator must configure the base via the
provider's *_API_BASE env var (which stays trusted).
* fix(pricing): add GitHub Copilot MAI Code Flash pricing (#30415)
* fix(pricing): add GitHub Copilot MAI Code Flash pricing
Add GitHub Copilot pricing entries for MAI-Code-1-Flash and the internal Copilot CLI model name so cost calculation can price input, cached input, and output tokens.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* test(pricing): cover GitHub Copilot MAI Code Flash pricing
Add regression coverage for both GitHub Copilot MAI-Code-1-Flash model names, including cached input pricing, chat endpoint metadata, and cost_per_token arithmetic.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
---------
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* fix(router/proxy): propagate completed_response through FallbackResponsesStreamWrapper for streaming /v1/responses container ownership (#30210) (#30213)
* fix(router/proxy): propagate completed_response through FallbackResponsesStreamWrapper for streaming /v1/responses container ownership (#30210)
#28990 added ownership recording for streaming /v1/responses via
_wrap_responses_stream_for_container_ownership, which reads
`getattr(stream_response, 'completed_response', None)` to extract the
ResponsesAPIResponse. The unit test bypassed the Router, so it never
exercised the production wrapping path.
Through the Router (every proxy deployment), the stream is wrapped by
FallbackResponsesStreamWrapper (router.py:2527). Its __init__ set
`self.completed_response = None` and __anext__ only forwarded chunks
— the inner source iterator's terminal event never bubbled up to the
attribute the ownership hook reads, so the hook silently recorded
nothing and every follow-up /v1/containers/<id>/files call returned
403 for non-admin keys.
This commit:
- router.py: pre-resolves the responses-API terminal event tuple
(response.completed / .incomplete / .failed) once per
_aresponses_streaming_iterator call, and has the wrapper's __anext__
sniff each forwarded chunk's .type. First terminal event hit gets
stored on the wrapper's completed_response. Iterator-agnostic — works
for source_iterator AND any future wrapper.
- common_request_processing.py: when _extract_completed_responses_response
returns None we now warn instead of silently skipping. Reporter on
#30210 lost a day to this exact silent skip; the warning surfaces
future regressions of the same shape directly in operator logs.
Fixes#30210
* fix(router): type-ignore wrapper getattr-defaults; broaden ownership-skip warning
CI lint (mypy) flagged the three pre-existing getattr(..., None) assignments
in FallbackResponsesStreamWrapper.__init__:
router.py:2564 self.response = getattr(source_iterator, 'response', None)
router.py:2565 self.model = getattr(source_iterator, 'model', None)
router.py:2566 self.logging_obj = getattr(..., None)
Those lines also exist on litellm_internal_staging and pass mypy there.
Adding the typed terminal-event tuple above the class made the function
body more narrowable, which surfaced the pre-existing mismatch — base
class declares non-Optional types but the bridge path
(LiteLLMCompletionStreamingIterator) legitimately omits these. Keep
the None fallback and silence with type: ignore[assignment].
Greptile 4/5 note: the ownership-skip warning hard-named code_interpreter
which misleads operators when a non-code_interpreter stream aborts.
Generalize to 'any tool container (e.g. code_interpreter)'.
* fix(register_model): drop synthesized zero costs to preserve sparse entries (#30198) (#30201)
* fix(register_model): drop synthesized zero costs to preserve sparse entries (#30198)
get_model_info synthesizes input_cost_per_token / output_cost_per_token = 0
when they are absent from the raw entry (the price-unknown and free cases
share the same representation). register_model then merges that result back
into litellm.model_cost, which flips a sparse entry from 'no cost keys'
(priced via model name) to 'cost keys = 0' (free).
That defeats _is_cost_explicitly_configured (#24949) on re-registration:
_is_model_cost_zero returns True, common_checks skips every tag / key /
team / user / org budget check for the group, and over-budget traffic
keeps returning 200. Spend keeps recording because cost calc still resolves
by model name, so the symptom is silent and only triggers on the second
register_model pass (router rebuild, /model/update, config sync).
Mirror the existing litellm_provider-None guard one block above and pop
the cost fields from the synthesized result when they are absent from the
raw entry and not in the caller's value. Caller-provided zeros (genuinely
free models, BYOK overrides) are preserved.
Fixes#30198
* fix(register_model): switch _raw_entry to is-None checks + drop dead test assertion
Greptile #30201 review notes:
- the `or`-chain in the raw-entry lookup treated an empty dict (a key
with no fields) as falsy and fell through to the second arm — replace
with explicit `is None` checks so a present-but-empty entry is still
taken at face value.
- the first assertion in `test_router_double_init_keeps_db_model_entry_sparse`
used `in (None, 0)` which passes under the bug condition (cost = 0
matches the tuple); the strong follow-up assertion already covers
every shape, so drop the dead branch.
* fix(bedrock mantle): use unique function-call id for responses->chat tool calls (#30426)
* fix(bedrock mantle): use unique function-call id for responses->chat tool calls
...
* fix(bedrock mantle): scope unique tool-call id fallback to degenerate call_id
The previous revision preferred the Responses item id for every tool call, which broke providers (and existing tests) where call_id is a unique, canonical correlation key. Restrict the fallback to the degenerate index-based call_id that Bedrock Mantle returns (call_0, call_1, ... resetting per response) and keep call_id otherwise. Revert the change to the OUTPUT_ITEM_DONE streaming handler, whose tool_call_chunk is never emitted (dead code, per review). Extend the regression tests to assert a normal call_id is preserved.
* fix(router): preserve azure_ad_token through CredentialLiteLLMParams for /v1/files + batches (#30235) (#30241)
* fix(router): preserve azure_ad_token through CredentialLiteLLMParams for /v1/files + batches (#30235)
Router.get_deployment_credentials_with_provider re-validates a
deployment's litellm_params through CredentialLiteLLMParams before
handing them to file/batch/passthrough callers:
return CredentialLiteLLMParams(
**deployment.litellm_params.model_dump(exclude_none=True)
).model_dump(exclude_none=True)
Any field NOT declared on CredentialLiteLLMParams gets silently dropped
on the way through. azure_ad_token was undeclared, so Azure deployments
using OAuth/M2M (azure_ad_token instead of a static api_key) silently
lost their token at the files endpoint and the proxy returned:
Missing credentials. Please pass one of api_key, azure_ad_token,
azure_ad_token_provider, ...
Declare azure_ad_token on CredentialLiteLLMParams alongside api_key /
api_base / api_version so it rides through the round-trip. Static-key
deployments stay unaffected (Optional, default None, dropped by
exclude_none=True). Provider-callable (azure_ad_token_provider) is a
separate concern and out of scope here.
Fixes#30235
* fix(ui-types): regenerate schema.d.ts for new azure_ad_token field
CI's 'Verify schema.d.ts matches the proxy OpenAPI spec' check
auto-detected the new field and emitted the exact diff to apply.
Two schemas had `aws_secret_access_key` from CredentialLiteLLMParams,
both get the new azure_ad_token marker next to it.
* fix(proxy): org_admin with own user_id now sees all org teams on /v2/team/list (#30247)
When the UI sends the callers own user_id (as it does for non-Admin
global roles), _enforce_list_team_v2_access now nulls it out for org
admins so _build_team_list_where_conditions scopes by organization_id
only -- matching the legacy /team/list behavior and the documented intent.
Fixes#30215
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
* test(vertex_ai): multi-region regression coverage for cachedContents host (#29571) (#29707)
litellm_internal_staging already routes the cachedContents URL through
get_vertex_base_url, fixing the multi-region 404 reported in #29571 —
but carries no test coverage for the actual regression scenario (eu/us
must resolve to the REP host aiplatform.{geo}.rep.googleapis.com).
Add TestContextCachingMultiRegionUrls: parametrized eu/us REP-host
assertions (including absence of the old broken {geo}-aiplatform host),
plus regional (us-central1) and global no-regression checks.
* fix(proxy): close upstream LLM stream when client disconnects mid-stream (#30245)
* fix(proxy): close upstream LLM stream when client disconnects mid-stream
When a streaming client disconnects, Starlette abandons the response
body iterator without calling aclose(), so the proxy's connection to
the upstream backend stays open until garbage collection, which may
never come. The backend (e.g. vLLM) keeps generating into a dead pipe:
small responses drain invisibly into TCP buffers while large ones block
the backend on a full send buffer indefinitely (observed via lsof as an
ESTABLISHED proxy->backend connection minutes after the client left)
create_response now returns a StreamingResponse subclass that closes
both its body iterator and the wrapped upstream-facing generator in a
shielded finally. The upstream generator is closed directly rather than
through a cascade because aclose() on a never-started generator skips
its body, which would make the cascade a no-op when the client
disconnects before the first chunk is sent.
async_streaming_data_generator also gains the same shielded
finally-aclose that async_data_generator in proxy_server.py already
had, covering the Anthropic and Google SSE paths
With this, killing a streaming client causes the backend to observe the
abort within about a second and free its slot, while completed streams
are unaffected. No flag is needed, unlike the non-streaming opt-in
cancel in #30223: this only releases resources after the client is
already gone and does not change any response a client can observe
Fixes#30244
* fix(proxy): close upstream even when body iterator aclose raises BaseException
Addresses the Greptile finding on #30245: the cleanup loop caught only
Exception while the generator-level cleanup catches BaseException, so a
CancelledError or GeneratorExit escaping body_iterator.aclose() would
skip closing the upstream generator. Both sites now use the same scope
and a regression test pins that the upstream is closed even when the
body iterator explodes with a BaseException
* fix(llms): expose aclose on BaseModelResponseIterator so stream close reaches the provider connection
The response-level close added for #30244 only worked for SDK-based
providers (e.g. openai), whose streams expose aclose all the way down.
Providers served by base_llm_http_handler (hosted_vllm and most modern
transformation-based providers) wrap a bare response.aiter_lines()
generator in BaseModelResponseIterator, which had no aclose or close at
all, and nothing retained the httpx response object; so
CustomStreamWrapper.aclose() silently did nothing and the upstream
connection stayed open. Verified with a vLLM-style mock: with
hosted_vllm/ the backend streamed all 100 chunks to completion after
the client disconnected, while openai/ aborted at chunk 6
BaseModelResponseIterator now carries an optional http_response and an
aclose() that closes it; make_async_call_stream_helper attaches the
response after building the iterator. With this, hosted_vllm aborts the
backend within ~1.6s of the client dropping, and completed streams are
unaffected
---------
Co-authored-by: kursad <kursad.lacin@brado.net>
* feat(anthropic): surface compaction usage iterations data (#27065)
* feat(anthropic): surface compaction usage iterations data
* style: apply black formatting to fix lint checks
* fix(usage): correct calculate usage with cached tokens when use ChatCompletionUsageBlock (#30422)
* fix(usage): correct calculate usage with cached tokens when use ChatCompletionUsageBlock
* fix(usage): optimize test imports
* feat: add fastCRW search provider (#30434)
* feat(provider): add LibertAI as a JSON-configured OpenAI-compatible provider (#30203)
* feat(provider): add LibertAI as a JSON-configured OpenAI-compatible provider
* libertai: update served endpoints backup + add mode/matrix tests
Addresses review feedback:
- Add libertai to litellm/provider_endpoints_support_backup.json, the file
actually served by GET /public/supported_endpoints (the root
provider_endpoints_support.json already had it).
- Add tests asserting bge-m3 normalizes to mode='embedding' and that the
served matrix lists libertai. embeddings stays false: the JSON-configured
provider path only wires chat routing (OpenAILike embedding handler is
reached only for literal openai_like/llamafile/lm_studio), matching the
llamagate precedent; bge-m3 remains in the cost map for metadata.
---------
Co-authored-by: Moshe Malawach <moshemalawach@users.noreply.github.com>
* feat(provider): add ModelScope as an OpenAI-compatible provider (#28460)
* add ModelScope API support
* add modelscope api support
* update modelscope model list
* add image-genetation support
* update test and multimodal
* fix: address PR review feedback for modelscope provider
* update README
* fix(customer_endpoints): restrict /customer/daily/activity to admin-only (#28849)
* fix(customer_endpoints): restrict /customer/daily/activity to admin-only
* fix(customer_endpoints): check role before prisma_client guard
* fix(custom_guardrail): key disable_global_guardrails takes precedence over team guardrail list (#28563)
* fix(fallbacks): preserve fallback model in SDK fallback responses (#28260)
* fix(fallbacks): preserve fallback model in response when using SDK-level fallbacks
* fix(fallbacks): gate x-litellm-* passthrough to trusted callers only
The previous patch unconditionally let `x-litellm-*` keys bypass the
`llm_provider-` prefix in `process_response_headers`. That function is
also called on raw upstream-provider response headers (e.g. from
`llm_http_handler.py`), so a malicious provider could return
`x-litellm-attempted-fallbacks` and spoof a LiteLLM-internal marker,
bypassing the proxy model-override guard.
Add a `preserve_litellm_internal_headers` flag (default False). Only
`response_metadata.py`, which re-processes the already-built
`_hidden_params["additional_headers"]` dict (LiteLLM-owned), passes
True. Raw provider header callsites keep the default False, so upstream
`x-litellm-*` still gets the `llm_provider-` prefix.
Adds a regression test for the spoofing case and renames the existing
preserve test to make the trusted-path semantics explicit.
* fix(fallbacks): ignore preserve_litellm_internal_headers for raw httpx.Headers inputs
* style(core_helpers): apply black formatting
* fix(lint): remove banned typing.List/Dict/Any imports and suppress PLR0913 on interface overrides
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* fix(lint): apply black formatting to modelscope chat transformation
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* fix(lint): replace noqa with proper fixes — use **kwargs and Awaitable instead of Any/List
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* fix(lint): remove unused AllMessageValues import
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* revert: restore base_model_iterator.py to original PR state
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* fix(lint): restore full method signatures for MyPy compatibility; bump PLR0913 budget for new provider files
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* fix(lint): use @override to suppress PLR0913 on inherited signatures instead of bumping budget
The overrides keep their full base-class signatures for MyPy compatibility, but those signatures carry more than five parameters, which tripped PLR0913 on each subclass redeclaration. Since the arity is dictated by the base class and cannot be reduced, decorate the overrides with typing_extensions.override; ruff treats that as the intended signal that the parameter count is not under the author's control and skips PLR0913. This restores the PLR0913 baseline to 1813.
* fix(lint): add @override to modelscope image generation overrides
Apply the same typing_extensions.override treatment to the image generation config so its inherited-signature overrides do not count against PLR0913.
---------
Co-authored-by: Joel Tony <github@jaytau.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: hcl <chenglunhu@gmail.com>
Co-authored-by: ztko <96878659+koztkozt@users.noreply.github.com>
Co-authored-by: Nahrin <nahrin@nahrinoda.com>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Co-authored-by: Humphrey <a739376838@gmail.com>
Co-authored-by: kursadlacin <kursadlacin@gmail.com>
Co-authored-by: kursad <kursad.lacin@brado.net>
Co-authored-by: Dushyant Acharya <dushyantacharya873@gmail.com>
Co-authored-by: Yuriy <yuriy.shuyskiy@gmail.com>
Co-authored-by: Recep S <22618852+us@users.noreply.github.com>
Co-authored-by: Moshe Malawach <moshe.malawach@protonmail.com>
Co-authored-by: Moshe Malawach <moshemalawach@users.noreply.github.com>
Co-authored-by: Rongkun Yan <2493404415@qq.com>
Co-authored-by: Varshith <kvarshithgowda@gmail.com>
Co-authored-by: Mateo Wang <277851410+mateo-berri@users.noreply.github.com>
* fix(team): keep team_alias cache in sync on _cache_team_object writes
_cache_team_object wrote only to the team_id:<id> cache key, but the
JWT auth path that uses team_alias_jwt_field reads from a separate
team_alias:<alias> key (get_team_object_by_alias caches under both
keys on miss, but reads only the alias-keyed one). After any
team-mutation endpoint (team_model_add, team_model_delete,
update_team, the two access-group writes) the team_id cache was
refreshed but the team_alias cache stayed stale until TTL — JWT
callers using team_alias_jwt_field kept seeing the pre-mutation
team for the full cache window.
Mirror the write under the alias key inside _cache_team_object so
every existing caller stays in sync without further changes. Skip
the alias write when team_alias is None/empty so we don't collide
across alias-less teams.
Surfaced testing the LIT-3244 cherry-pick on patch/1.86.0: the
LIT-3244 fix correctly invalidated the team_id cache but the
customer's JWT used team_alias_jwt_field, so they kept hitting the
stale alias-keyed entry.
* fix(team): delete (not overwrite) team_alias cache on _cache_team_object
The prior shape of this PR wrote both team_id:<id> AND team_alias:<alias>
from _cache_team_object. team_alias is NOT unique in the schema
(no @unique on LiteLLM_TeamTable.team_alias), and get_team_object_by_alias
enforces uniqueness on its own DB-fetch path (len(teams) > 1 raises).
Writing the alias-keyed cache from the generic refresh path bypassed
that check: a team admin renaming their team to collide with another
team's alias could silently overwrite the cached team for JWT-by-alias
auth, swapping the resolved team under that alias for the cache window.
Switch the alias-keyed operation from a write to a delete (mirroring
the dual-cache delete pattern in _delete_cache_key_object). After every
team write, the next JWT-by-alias reader cache-misses and falls through
to get_team_object_by_alias, which (a) re-fetches the fresh team from
DB, closing the LIT-3244 staleness gap that motivated this PR, and
(b) enforces alias uniqueness before populating either cache key.
team_id:<id> writes are unchanged — team_id is the table PK and is
guaranteed unique.
Surfaced in veria-ai review on #28739.
* fix(managed-files): anchor model_id regex so it doesn't match llm_output_file_model_id
extract_model_id_from_unified_id used `re.search(r"model_id,([^;]+)", ...)`
which substring-matches the `model_id,` inside the file-ID encoding's
`llm_output_file_model_id,<deployment_uuid>` field. parse_unified_id
then fed that deployment UUID back into the auth path as a model
candidate via _extract_models_from_managed_resource_id, and every
team-BYOK file attach 403'd with:
team not allowed to access model. This team can only access
models=['openai/*']. Tried to access <deployment-uuid>
The team's models list correctly contains the public name (`openai/*`)
that target_model_names matches, but the bogus UUID candidate fails
the wildcard check first.
Anchor the regex to a field boundary (`(?:^|;)model_id,`) so it
matches the legitimate top-level `model_id,<value>` field on
vector_store unified IDs and skips substring matches inside other
fields. File-IDs (which have no top-level `model_id` field) now
return None and contribute no spurious UUID candidate.
Surfaced reproducing LIT-3244 on patch/1.86.0 with the customer's
exact flow: team with openai/* BYOK deployment, JWT-scoped user,
POST /v1/vector_stores/{id}/files attaching a file uploaded with
target_model_names=openai/gpt-4o.
Two follow-ups to the managed-resource isolation fix:
1. Rename the new composite indexes to match Prisma's auto-generated naming
convention (`<Table>_created_by_team_id_created_at_idx`). The previous
`*_team_owner_created_at_idx` names left `prisma migrate diff` reporting
an outstanding `RENAME INDEX`, failing `test_aaaasschema_migration_check`.
2. Make `build_owner_filter` return an OR clause when the caller has both
a `user_id` and a `team_id`, so listings include team-shared resources
the same way `can_access_resource` already permits reading them. Without
this a user could fetch a team-shared resource by id but never see it
in their list view.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Service-account API keys are issued without a `user_id`, and managed
file/batch/vector-store ownership checks compared
`resource.created_by == user_api_key_dict.user_id`. Because Python
evaluates `None == None` as True, any service-account key passed
ownership checks for any resource also created without a user id, and
listing endpoints skipped the `created_by` filter entirely when the
caller had no user id — returning every tenant's records.
Replace the bare equality with an identity-aware helper:
- Admins (PROXY_ADMIN, PROXY_ADMIN_VIEW_ONLY) keep their unscoped view.
- Callers with a `user_id` are scoped to records they created.
- Callers without a `user_id` but with a `team_id` are scoped to records
created within their team via a new `created_by_team_id` column.
- Callers with no admin role and no identifying ids are denied — the
listing path returns an empty page without issuing a query.
Schema migration adds `created_by_team_id` to LiteLLM_ManagedFileTable,
LiteLLM_ManagedObjectTable, and LiteLLM_ManagedVectorStoreTable, plus
indexes for the new filter. Writes in BaseManagedResource and the
enterprise managed_files hook now stamp the column from
`user_api_key_dict.team_id`. Reads in `can_user_access_unified_resource_id`,
`can_user_call_unified_file_id`, `can_user_call_unified_object_id`,
`list_user_resources`, `list_user_batches`, and `get_user_created_file_ids`
all delegate to the new helper.
Tests cover the helper in isolation, the base-class listing/access paths,
and the enterprise file-access hook (including a regression test for the
original `None == None` bypass).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The empty-line filter in __next__/__anext__ called .strip() without
checking the type first. When the Responses API yields Pydantic
BaseModel events (e.g. ResponseCreatedEvent), this raises
AttributeError. Add an isinstance(str_line, str) guard so non-string
objects pass through to _handle_string_chunk as intended.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
