- upload_and_replace_file now compensates when Step 4 (old-file removal)
fails: the newly added file is purged from the KB collection, unlinked
from the KB, and routed through the file delete to remove storage, the
file row, and the per-file vector collection. This restores the pre-
replace state so the KB isn't left with duplicated entries drifting
from "replace" semantics. The 500 detail explicitly says the
replacement was rolled back so callers can retry cleanly.
- Directory-sync frontend now detects files that skipped browser hashing
(size > MAX_BROWSER_HASH_BYTES) and warns the user up front that those
files will be compared by size only — a content change that keeps the
same byte size will not be detected. Paths are also logged to the
console for troubleshooting.
- FileSyncCompareItem now rejects malformed payloads up front: file_path
must be non-empty, NUL-free, under 4096 chars, and must not contain
traversal segments; file_hash must be either empty (hashing skipped
signal) or a lowercase 64-char hex SHA-256; size must be >= 0. This
hardens the public API instead of relying on the frontend to behave.
- remove_file_from_knowledge_by_id with delete_file=True now hard-deletes
the file record, storage blob, and per-file vector collection only
when no other knowledge base still references that file. If another
KB still has the file attached, the request degrades to a KB-scoped
unlink so syncing KB-A can't silently wipe a file from KB-B. This
endpoint also now deletes the object-storage blob on hard-delete,
closing the previously-flagged storage leak.
- upload_and_replace_file Step 4 switches from the global file delete
route to this KB-scoped helper so the same cross-KB safeguard applies
to replace flows.
- upload_and_replace_file preserves HTTPException from upload and
remove steps instead of flattening every upstream status/detail into
a generic 400 via str(exc).
- Frontend syncDirectoryHandler remove loop switches back to
removeFileFromKnowledgeById now that the backend endpoint handles
shared files and storage cleanup safely.
- compare_files_for_sync duplicate detection uses collections.Counter
for O(n) dedup instead of list.count inside a set comprehension
(O(n^2)), which matters for large directory payloads.
- Empty incoming file_hash now falls through to the size-based
comparison branch instead of always being treated as changed, so the
browser can skip hashing large files without every such file being
flagged as modified.
- Browser calculateFileHash skips files above 100 MB (SubtleCrypto has
no streaming digest API) and returns an empty hash, letting the
server fall back to size comparison and avoiding tab OOM on very
large files.
- upload_and_replace_file now explicitly deletes the new file's vectors
from the KB collection in the Step 3 rollback path. process_file can
write embeddings before add_file_to_knowledge_by_id fails; without the
association the router delete can't discover those vectors, so they
would remain retrievable for a file record that no longer exists.
- syncDirectoryHandler switches the removed-files loop from
removeFileFromKnowledgeById (leaves storage blobs behind) to
deleteFileById, which also clears the object-storage blob and the
per-file vector collection. Each delete is guarded with try/catch and
a truthy-response check so silent null returns are counted as
failures instead of inflating the "removed" tally.
- Sync summary now reports the succeeded-removed count and rolls
removedFailed into totalFailed so the user sees real outcomes.
- compare_files_for_sync rejects payloads with duplicate file_path
entries up front, so the server no longer relies on the frontend to
dedupe — duplicates would otherwise schedule the same existing file
for replacement or removal twice.
- upload_and_replace_file now routes both the old-file removal and the
failure-path cleanup through the files router delete handler, so the
object-storage blob is removed alongside the DB row, vector entries,
and per-file vector collection. The KB-scoped remove_file helper left
storage objects behind and was accumulating orphans across syncs.
- On processing-status failure (Step 2), the newly uploaded artifact is
deleted instead of being left as an orphaned file row/blob.
- uploadFileHandler and addFileHandler now return explicit
success/failure signals so the directory sync loop can track real
outcomes. The sync summary reports succeeded counts (not planned
counts) and switches to a warning toast with a failure tally when any
upload or replace fails, instead of always reporting "Sync complete".
- Replace loop wraps uploadAndReplaceFile in try/catch so a single
failed replacement no longer aborts the whole sync, and shows a per-
file error toast with the server detail.
- Drop the unused has_access imports in files.py and knowledge.py
added by this PR (AccessGrants.has_access and has_access_to_file are
the actual call sites).
- upload_and_replace_file now awaits process_file; the missing await made
it a no-op coroutine, so the KB relation was added without embeddings
ever being generated.
- Replace the truthy-data check with an explicit status == 'completed'
gate so pending/failed files can't slip through as "processed"; surface
the stored processing error when available.
- Clean up via the files router delete handler on KB-add failure so
storage objects and the per-file vector collection are removed too;
Files.delete_file_by_id only drops the DB row and would orphan them.
- compare_files_for_sync keys existing files as a list per sync_path so
duplicate uploads no longer silently overwrite each other. The first
entry is treated as the canonical replace target; extras (both for
matched and unmatched paths) are scheduled for removal.
- compareFilesForSync / uploadAndReplaceFile wrappers fall back through
err.detail / err.message / stringification so non-API failures (network
TypeError, thrown string) produce a truthy error and actually throw,
instead of returning null and letting the sync flow report success for
failed replacements.
- Remove stray closing paren in compare_files_for_sync that broke module
import and prevented the router from loading.
- upload_and_replace_file now verifies the old file belongs to the target
knowledge base (not just that it exists globally), and propagates a 500
when the post-upload removal fails so callers can reconcile instead of
silently accumulating duplicates.
- syncDirectoryHandler splits directory-picker errors (routed through
handleUploadError) from API errors (now surfaced with the server's
detail message), so compare/upload/remove failures no longer show the
misleading "Error accessing directory" toast.
fix: add separate original_path field for directory sync
Previously, meta.name was overloaded to store the full path for directory
sync comparison (e.g., "docs/subfolder/readme.md"). This caused potential
downstream effects since meta.name is used for display and downloads.
Changes:
- Revert meta.name to store only the sanitized base filename
- Add new meta.original_path field that preserves the full upload path
- Update sync compare logic to prioritize original_path for matching
- Fallback chain for legacy files: original_path -> name -> filename
This maintains backwards compatibility with existing files while enabling
directory structure preservation for the sync feature.
fix: use size-based fallback for legacy files to prevent sync timeout
- Replace brittle __class__.__name__ check with isinstance(metadata, FormParam)
- For legacy files without stored hashes, use file size comparison instead
of downloading and hashing files on-demand during sync comparison
- This prevents HTTP timeouts for knowledge bases with 1000+ legacy files
- Remove unused get_file_hash function and calculate_sha256 import
The catch-all /{path:path} proxy forwards any request to the upstream OpenAI-compatible API with the admin's API key and no access control. This is an intentional proxy but should be opt-in.
Adds ENABLE_OPENAI_API_PASSTHROUGH env var (defaults to False). When disabled, the catch-all returns 403. No other routers (Ollama, responses) have catch-all proxies.
The GET /channels/{id}/members endpoint checked membership for group/dm channels but had no access gate for standard channels, allowing any authenticated user with channels permission to enumerate members of private standard channels by UUID.
The model name from user input was interpolated directly into Azure deployment URL paths without validation. A user could send a model name like '../../management/foo' to traverse the URL path and hit unintended Azure endpoints with the admin's API key.
Adds _sanitize_model_for_url that rejects path separators and traversal sequences, and percent-encodes the name. Applied at convert_to_azure_payload (covers chat completions + proxy) and the responses endpoint's direct URL construction.
These four endpoints checked model existence but never verified the user has read access via AccessGrants, allowing any authenticated user to use restricted models.
Uses the canonical check_model_access helper from utils.access_control.
Both LDAP and OAuth registration checked user count before insert to determine whether to assign admin role. With multiple workers, concurrent first-user registrations could each see zero users and both create admin accounts.
Applies the insert-first-check-after pattern already used by signup_handler: insert with DEFAULT_USER_ROLE, then atomically check get_num_users()==1 and promote only the sole user to admin.
is_user_channel_member and is_user_channel_manager did not filter on is_active, allowing deactivated members to retain read/write access to group channels via direct API calls.
The /responses proxy endpoint only required authentication via
get_verified_user but did not check per-model access grants. This
allowed any authenticated user to access any model through this
endpoint, bypassing the access control system.
Extract a shared check_model_access helper into utils/access_control
and replace all inline access control blocks across openai.py and
ollama.py (7 locations) with calls to this helper. This eliminates
code duplication and prevents future policy drift between endpoints.
CWE-862: Missing Authorization
CVSS:3.1/AV:N/AC:L/PR:L/UI:N/S:U/C:L/I:N/A:H (6.5 Medium)
Introduces REDIS_HEALTH_CHECK_INTERVAL and wires it through to every
Redis client created by get_redis_connection (plain, cluster and
sentinel paths, sync and async). When set, redis-py will PING any
connection idle longer than the interval on checkout, so dead sockets
are surfaced as reconnectable errors before a real command lands on
them.
Defaults to unset (empty string) so existing deployments see no
behavioural change. Operators who want the protection should set it
shorter than the Redis server `timeout` setting and any firewall/LB
idle timeout on the path to Redis.
Co-authored-by: Claude <noreply@anthropic.com>
Introduces REDIS_SOCKET_KEEPALIVE and wires socket_keepalive=True
through to every Redis client created by get_redis_connection
(plain, cluster and sentinel paths, sync and async). When enabled,
the kernel sends TCP keepalive probes on idle connections so
half-closed sockets (e.g. after a silent firewall/LB reset or a NIC
flap) are detected before the next command lands on them and the
request never sees a "Connection reset by peer" error.
Defaults to off so existing deployments see no behavioural change.
Operators who want the protection set REDIS_SOCKET_KEEPALIVE=true
in their environment.
Co-authored-by: Claude <noreply@anthropic.com>
* fix(redis): honor REDIS_SOCKET_CONNECT_TIMEOUT on non-sentinel clients
Previously only the sentinel path passed REDIS_SOCKET_CONNECT_TIMEOUT
through to the Redis client. Plain redis:// and cluster URLs fell back
to redis-py's default (no explicit connect timeout), so a hung Redis
or a black-holed network path could stall the whole worker until the
kernel gave up. Forwarding the same env var to from_url()/RedisCluster
keeps the behavior consistent across all deployment topologies.
* fix(redis): gate socket_connect_timeout on is-not-None, not truthiness
Addresses review feedback: the truthiness check on REDIS_SOCKET_CONNECT_TIMEOUT
silently dropped an explicit 0 value and was inconsistent with the sentinel
construction path, which forwards the value directly. Switch to `is not None`
so any user-configured value (including 0) is passed through to from_url()
and RedisCluster.from_url().
---------
Co-authored-by: Claude <noreply@anthropic.com>
Differentiate between "Allow File Upload" and "Allow Web Upload"
in Chinese translations to help administrators understand the
distinction:
- "Allow File Upload" = local file, cloud storage uploads
- "Allow Web Upload" = URL, YouTube, web content uploads
Using user.id as client_id causes WebSocket deadlocks when the same
user generates images concurrently (e.g., multi-model chat). ComfyUI
routes messages by clientId, so shared IDs mean only one connection
receives the completion — others hang forever.
Generate a unique UUID per request, matching ComfyUI's own examples.
