Commit graph

503 commits

Author SHA1 Message Date
Timothy Jaeryang Baek
b81627b2c9 refac 2026-07-26 18:46:39 -04:00
G30
db92ef292f
fix: unarchive chats moved into folders and refresh sidebar folders after menu moves (#27485) 2026-07-26 18:43:34 -04:00
G30
771540f3de
feat: sortable title and updated at columns on the knowledge workspace page (#27457) 2026-07-26 18:19:31 -04:00
Timothy Jaeryang Baek
1f0dc90abe refac 2026-07-26 18:06:03 -04:00
Classic298
e398ba3506
fix: don't seed non-persistent config keys (oauth.* with flag off) (#26928)
seed_defaults inserted a row for every key in DEFAULT_CONFIG regardless
of whether the DB is authoritative for it. With ENABLE_OAUTH_PERSISTENT_CONFIG
off, the oauth.* keys were seeded from the then-current (often empty) env
values. Enabling the flag later made those stale rows override live env vars
(e.g. ENABLE_OAUTH_SIGNUP=true stopped taking effect) and further env changes
were never picked up.

Skip keys where persistent_enabled_for() is false, matching the masking the
read paths (get/get_many/get_namespace/get_all) already apply.


Claude-Session: https://claude.ai/code/session_01Vr2RCYUTXCtgtV4WMUCK86

Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-24 00:58:30 -05:00
Timothy Jaeryang Baek
28bdcb063b refac 2026-07-24 01:54:36 -04:00
Timothy Jaeryang Baek
212eec408c refac 2026-07-24 01:44:30 -04:00
Timothy Jaeryang Baek
1f5b0d816f refac 2026-07-24 01:19:28 -04:00
Classic298
9b635d8f3d
fix: calendar attendee RSVP correctness — server-derived status and hide declined invites (#27007)
* fix: let only the attendee set their own calendar RSVP status

set_attendees took each attendee's status from the caller-supplied value, so an
event organiser could set another user's RSVP (for example to 'accepted') on
create or update. RSVP is meant to be self-service: the /events/{id}/rsvp
endpoint already scopes status changes to the calling user.

Derive attendee status server-side instead of from the request. An existing
attendee keeps the status they set via RSVP and a newly added attendee starts
'pending'; any caller-supplied status is ignored. Event edits no longer reset
attendees' existing responses.

Co-authored-by: legobattman <302282032+legobattman@users.noreply.github.com>

* fix: hide declined calendar invites from the attendee view

`get_events_by_range` surfaced every event where the user is an attendee regardless of their RSVP status, so declining an invite left it in the calendar with no way to remove it. Exclude `declined` attendee rows from the attendee branch, so a decline now removes the event from the user's own view while pending, accepted and tentative invitations still surface.

Co-Authored-By: legobattman <302282032+legobattman@users.noreply.github.com>

---------

Co-authored-by: legobattman <302282032+legobattman@users.noreply.github.com>
2026-07-24 00:01:16 -05:00
Classic298
48f78ca58d
fix: prevent startup crash when function/tool has null user_id (#26850)
The Function and Tool database columns declare user_id as a nullable
String column, but their Pydantic read-models required a non-null
string. A record with user_id NULL therefore raised a
pydantic ValidationError inside get_functions()/get_tools(), which run
during install_tool_and_function_dependencies() at app startup —
crashing the whole application and blocking all chat completions.

Make user_id Optional in the read/response models so such records
validate gracefully (user is already rendered as None downstream when
the id has no matching user) instead of taking down startup.


Claude-Session: https://claude.ai/code/session_01Y4RRUNq7ZUFkRWbWPkDw3m

Co-authored-by: Claude <noreply@anthropic.com>
2026-07-24 00:00:37 -05:00
Timothy Jaeryang Baek
4e869011cd refac 2026-07-24 00:42:11 -04:00
Timothy Jaeryang Baek
9a49b271aa refac 2026-07-23 19:17:19 -04:00
Classic298
699d512e2f
perf: drop redundant session.refresh calls after commit across the model layer (#27381)
Both session factories run with expire_on_commit=False, so ORM objects keep their attribute values after commit. Every session.refresh issued right after a commit therefore re-SELECTed a row whose values the session already held, including full chat JSON blobs and user settings, purely to overwrite identical data. Fifty such calls existed across the model layer, covering nearly every write path in the app (chat inserts, title updates, pin/archive toggles, user role and settings updates, tool, prompt, function, model, file, tag, feedback, memory, automation and grant writes).

All fifty are removed. The only refreshes with an actual job were the two update-then-reload paths in tools and skills, where a Core UPDATE statement bypasses the identity map; those now use session.get(..., populate_existing=True), which guarantees a fresh row in one SELECT whether or not the row was already present in the session (the previous code issued get plus refresh, two SELECTs, on the default configuration).

Benchmark (real SQLite DB, per write):

| write path | before | after |
| --- | --- | --- |
| chat title update, ~600 KB chat blob | 2.08 ms | 1.24 ms |
| user role update, small row | 1.21 ms | 0.68 ms |

On Postgres each removed refresh is additionally a network round trip. The chat-blob case also skips re-parsing the entire JSON document per write.

Functionally verified against a fresh database: user insert, role and settings updates, chat insert (including the server-default meta column, which is always provided client-side), title update and pin toggle, tool insert and the Core-update reload path, tag insert and the prompt insert flow that pins version_id after history creation all return correct values and persist correctly.
2026-07-23 18:08:00 -05:00
Classic298
6b655689cc perf: cut repeated per-model work out of model list assembly
get_all_models runs on every models refresh and, without the base-models cache (off by default), on every /api/models request. Several of its costs multiplied by the model count for no reason:

- The active action and filter id sets were derived from get_functions_by_type, which loads full function rows including plugin source and validates them, only for the ids and is_global flags. A generalized column-only query now returns (id, is_global) tuples; the existing filter-specific helper delegates to it.
- Action priorities were computed inside the per-model sort key, constructing a pydantic Valves object per action per model; with global actions in every model's list that was models x actions constructions per refresh. Priorities are now memoized per action.
- Global action and filter item dicts were rebuilt per model from the same modules. The item lists are now built once per function and shallow-copied per model, keeping per-model dicts independent exactly as before (nested values were already shared).
- Deactivated base-model overrides were dropped with models.remove, a linear scan and shift per removal; removals are now collected and filtered out in one identity-based pass, preserving list.remove's exact object semantics.
- RedisDict.set fingerprinted the payload by serializing the already-serialized mapping a second time plus a sha256; a direct dict comparison against the last written mapping has the same skip semantics without re-serializing anything.
- /api/models did tag normalization and profile-image stripping for every model before access filtering discarded the invisible ones, and always evaluated a json.dumps debug f-string; the work now runs only on visible models and the debug line is gated on the log level. The duplicate-id dedup keeps its position before filtering so the effective-model semantics are unchanged.

Benchmark:

| metric | before | after |
| --- | --- | --- |
| model-cache fingerprint, 200 models | 45 us | 1.4 us |
| action priority Valves builds, 200 models x 4 global actions | 0.37 ms (800 builds) | 0.002 ms (4 builds) |
| function-table payload for id sets | full rows incl. source | (id, is_global) tuples |

Functionally verified: the column-only id query matches the full-row query for actions and filters including inactive exclusion, and the fingerprint skip logic writes on first set, skips identical payloads, updates plus deletes stale keys on change and clears on empty, against a scripted fake Redis.
2026-07-23 19:05:32 -04:00
Classic298
310ae91302
perf: make tools defer_content real and batch tool and knowledge access filters (#27387)
Tools.get_tools(defer_content=True) contained the literal dead statement "stmt = stmt": the deferral was a no-op, so every tools listing loaded the full Python source of every tool (five caller sites pass defer_content=True expecting the optimization: the tools list endpoints and the user and group permission overviews). The listing now selects every column except content, and ToolModel.content becomes optional to represent deferred rows; router projections are content-less response models, so nothing downstream reads the source on these paths.

On top of that, get_tools_by_user_id issued one grant query per non-owned tool and Knowledges.get_knowledge_bases_by_user_id did the same per knowledge base (the latter also sits inside per-file access checks). Both now resolve grants for all non-owned rows in a single get_accessible_resource_ids call, the same batch helper the model listing already uses.

Benchmark (real SQLite DB):

| metric | before | after |
| --- | --- | --- |
| tools listing, 33 tools x ~200 KB source | 5.13 ms | 3.18 ms |
| grant queries per accessible-tools call, N non-owned tools | N | 1 |
| grant queries per accessible-KBs call, N non-owned KBs | N | 1 |

The listing row scales with source size; on Postgres the deferral additionally avoids shipping every tool's source over the wire per listing, and each removed grant query was a real round trip.

Functionally verified: deferred listings match full listings field for field with content None, grants included and router projections working; access filtering returns exactly owned plus granted tools and knowledge bases and nothing for strangers; full (non-deferred) reads still carry the source.
2026-07-23 17:51:07 -05:00
Classic298
5b518cbe43
perf: list knowledge base file metadata without loading extracted file contents (#27386)
Knowledges.get_file_metadatas_by_id fetched full File rows, whose data column carries the entire extracted text of each document, validated each into a FileModel and then threw everything except id, hash, meta and the timestamps away. The function backs every knowledge base detail view and runs again after every file add or remove (eight call sites in the knowledge router), so rendering a filename list for a 50-file knowledge base parsed tens of megabytes of JSON per request.

The listing now selects exactly the five columns the response needs, joined through KnowledgeFile, mirroring the column-only helper that already existed in the files model for id-based lookups.

Benchmark (real SQLite DB, 50 files with ~200 KB extracted text each):

| metric | before | after |
| --- | --- | --- |
| KB file metadata listing | 14.2 ms | 1.20 ms |

The gap widens linearly with file size and count since extracted contents no longer get read, parsed or validated at all.

Functionally verified: output matches the old implementation field for field on all 50 files and an unknown knowledge base still returns an empty list.
2026-07-23 17:50:55 -05:00
Classic298
d67bc4ffcd
perf: batch the file access check queries (#27383)
has_access_to_file runs for every non-owner file GET, per RAG file check and per shared-chat or model-attached file. Its final step called Models.get_models_by_user_id, which issued one grant query per non-owned workspace model, so a single file check on an instance with M workspace models cost M grant queries plus a group query, with the deny path always paying full price. Its collection_name step listed every knowledge base the user can access (itself one grant query per knowledge base) just to scan the list for one id. And get_accessible_folder_files repeated the whole pipeline per folder entry, refetching the caller's group memberships every time.

Three changes, all using parameters and helpers that already exist:
- Models.get_models_by_user_id resolves grants for all non-owned models in one get_accessible_resource_ids call and accepts prefetched user_group_ids.
- The collection_name check fetches the one referenced knowledge base and performs a single owner-or-grant check with the already-resolved group ids, preserving the write-requires-owner guard exactly (including its short-circuit before any grant query).
- get_accessible_folder_files resolves group ids once and threads them through every per-entry check.

Benchmark:

| metric | before | after |
| --- | --- | --- |
| filter loop CPU, 300 workspace models (queries stubbed) | 47 us | 19 us |
| grant queries per file-access check, M workspace models | M | 1 |
| group membership queries per folder listing, F files | F | 1 |

The stubbed CPU row understates the win: each removed query in the other two rows was a real database round trip.

Functionally verified with stubbed accessors: owned plus granted models are returned with owned ids excluded from the batch query; model-attached file access resolves through the batched path; the collection_name path does one KB fetch and one grant check with no full listing; a missing KB falls through; write access via a KB still requires the KB owner to own the file and short-circuits before the grant query; folder listings fetch groups exactly once.
2026-07-23 17:50:08 -05:00
Timothy Jaeryang Baek
c8f2e09fdc refac 2026-07-23 14:29:27 -04:00
Timothy Jaeryang Baek
25faa19941 refac 2026-07-23 14:27:55 -04:00
Timothy Jaeryang Baek
ca11bd90a7 chore: format 2026-07-23 13:41:16 -04:00
Timothy Jaeryang Baek
f9107edeeb refac 2026-07-23 12:48:14 -04:00
Timothy Jaeryang Baek
bb12b1a18b refac 2026-07-23 04:16:14 -04:00
Timothy Jaeryang Baek
cc9a44569e refac 2026-07-23 04:01:30 -04:00
Timothy Jaeryang Baek
48625e657f refac 2026-07-23 03:54:19 -04:00
Timothy Jaeryang Baek
cf887b68ea refac 2026-07-23 02:54:56 -04:00
Timothy Jaeryang Baek
409fb39717 refac 2026-07-21 13:53:30 -04:00
Timothy Jaeryang Baek
49e57f4e7e chore: format 2026-07-20 22:11:42 -04:00
Timothy Jaeryang Baek
af9a315ac3 refac 2026-07-20 01:33:47 -04:00
Timothy Jaeryang Baek
1a32d92d08 refac
Some checks failed
Create and publish Docker images with specific build args / merge (map[name:main suffix:]) (push) Blocked by required conditions
Create and publish Docker images with specific build args / merge (map[name:ollama suffix:-ollama]) (push) Blocked by required conditions
Create and publish Docker images with specific build args / merge (map[name:slim suffix:-slim]) (push) Blocked by required conditions
Create and publish Docker images with specific build args / notify-helm-charts (push) Blocked by required conditions
Create and publish Docker images with specific build args / copy-to-dockerhub (, main) (push) Blocked by required conditions
Create and publish Docker images with specific build args / copy-to-dockerhub (-cuda, cuda) (push) Blocked by required conditions
Create and publish Docker images with specific build args / copy-to-dockerhub (-cuda126, cuda126) (push) Blocked by required conditions
Create and publish Docker images with specific build args / copy-to-dockerhub (-ollama, ollama) (push) Blocked by required conditions
Create and publish Docker images with specific build args / copy-to-dockerhub (-slim, slim) (push) Blocked by required conditions
Create and publish Docker images with specific build args / build (map[arch:linux/amd64 runner:ubuntu-latest], map[build_args: free_disk:false name:main suffix:]) (push) Waiting to run
Create and publish Docker images with specific build args / build (map[arch:linux/amd64 runner:ubuntu-latest], map[build_args:USE_CUDA=true USE_CUDA_VER=cu126 free_disk:true name:cuda126 suffix:-cuda126]) (push) Waiting to run
Create and publish Docker images with specific build args / build (map[arch:linux/amd64 runner:ubuntu-latest], map[build_args:USE_CUDA=true free_disk:true name:cuda suffix:-cuda]) (push) Waiting to run
Create and publish Docker images with specific build args / build (map[arch:linux/amd64 runner:ubuntu-latest], map[build_args:USE_OLLAMA=true free_disk:false name:ollama suffix:-ollama]) (push) Waiting to run
Create and publish Docker images with specific build args / build (map[arch:linux/amd64 runner:ubuntu-latest], map[build_args:USE_SLIM=true free_disk:false name:slim suffix:-slim]) (push) Waiting to run
Create and publish Docker images with specific build args / build (map[arch:linux/arm64 runner:ubuntu-24.04-arm], map[build_args: free_disk:false name:main suffix:]) (push) Waiting to run
Create and publish Docker images with specific build args / build (map[arch:linux/arm64 runner:ubuntu-24.04-arm], map[build_args:USE_CUDA=true USE_CUDA_VER=cu126 free_disk:true name:cuda126 suffix:-cuda126]) (push) Waiting to run
Create and publish Docker images with specific build args / build (map[arch:linux/arm64 runner:ubuntu-24.04-arm], map[build_args:USE_CUDA=true free_disk:true name:cuda suffix:-cuda]) (push) Waiting to run
Create and publish Docker images with specific build args / build (map[arch:linux/arm64 runner:ubuntu-24.04-arm], map[build_args:USE_OLLAMA=true free_disk:false name:ollama suffix:-ollama]) (push) Waiting to run
Create and publish Docker images with specific build args / build (map[arch:linux/arm64 runner:ubuntu-24.04-arm], map[build_args:USE_SLIM=true free_disk:false name:slim suffix:-slim]) (push) Waiting to run
Create and publish Docker images with specific build args / merge (map[name:cuda suffix:-cuda]) (push) Blocked by required conditions
Create and publish Docker images with specific build args / merge (map[name:cuda126 suffix:-cuda126]) (push) Blocked by required conditions
Frontend Build / Format & Build (push) Waiting to run
Frontend Build / Unit Tests (push) Waiting to run
Python CI / Ruff Format (3.11) (push) Has been cancelled
Python CI / Ruff Format (3.12) (push) Has been cancelled
2026-07-17 06:50:16 -04:00
Timothy Jaeryang Baek
e5e2cd7876 refac 2026-07-17 04:48:21 -04:00
Timothy Jaeryang Baek
56f2cb5302 refac 2026-07-17 04:11:11 -04:00
Timothy Jaeryang Baek
743b9fd3ce refac 2026-07-16 21:57:43 -04:00
Timothy Jaeryang Baek
4d27bfff92 refac 2026-07-16 01:42:24 -04:00
Timothy Jaeryang Baek
185bca8552 refac 2026-07-15 22:34:52 -04:00
Timothy Jaeryang Baek
423cafd4e7 refac 2026-07-15 21:43:47 -04:00
Timothy Jaeryang Baek
7a9928ef17 refac 2026-07-14 23:08:41 -04:00
Timothy Jaeryang Baek
a9617ca218 refac 2026-07-14 01:15:29 -04:00
Timothy Jaeryang Baek
f1ded9409a refac 2026-07-14 01:13:40 -04:00
Timothy Jaeryang Baek
7088d245bb refac 2026-07-14 00:10:28 -04:00
Timothy Jaeryang Baek
73c0be8389 refac 2026-07-01 03:01:46 -05:00
Timothy Jaeryang Baek
ab22fe64bd refac
Co-Authored-By: Syed Osama Ali Shah <86572800+osamaali313@users.noreply.github.com>
2026-07-01 02:20:50 -05:00
Timothy Jaeryang Baek
403392b41b chore: format 2026-06-29 13:35:39 -05:00
Timothy Jaeryang Baek
22a44e67a8 refac 2026-06-29 13:15:29 -05:00
Timothy Jaeryang Baek
517cd8d102 refac 2026-06-29 13:03:14 -05:00
Timothy Jaeryang Baek
b4073f6378 refac 2026-06-29 12:29:10 -05:00
Timothy Jaeryang Baek
a146e17bdc refac 2026-06-29 11:33:32 -05:00
Timothy Jaeryang Baek
81e245548d refac 2026-06-29 11:00:53 -05:00
Timothy Jaeryang Baek
eb53281c9a refac
Co-Authored-By: Classic298 <27028174+Classic298@users.noreply.github.com>
2026-06-29 06:08:31 -05:00
Timothy Jaeryang Baek
b0487dd6dd refac 2026-06-29 05:49:54 -05:00
Timothy Jaeryang Baek
8927c9bb3d refac 2026-06-29 05:14:34 -05:00