mirror of
https://github.com/zotero/zotero.git
synced 2026-10-08 03:08:19 +00:00
Mirror of https://github.com/zotero/zotero.git
133e43ac87
Store each chunk's embedding as int8 instead of float32, cutting the vector column from 1536 to 384 bytes per row at about the same retrieval quality on the benchmark. Per-vector absmax scaling: the largest component becomes 127. Vectors are centered on the model's calibration mean before quantizing. The shared mean dominates a raw embedding, so quantizing raw would spend most of the 8 bits on it; centering first gives the int8 range to the informative residual, and drops the per-row vec_sub from the scoring SQL. The mean is now baked into stored rows, so a changed mean means a re-index, which the version-keyed calibration already implied. Quantized vectors have arbitrary norms, so scoring uses cosine rather than the dot product, in JS and in SQL (vec_distance_cosine over vec_int8(embedding) with the query quantized the same way). Calibration thresholds are measured on centered, quantized vectors so they describe what search computes. |
||
|---|---|---|
| .github/workflows | ||
| app | ||
| chrome | ||
| defaults/preferences | ||
| document-worker@6d0c0ce45d | ||
| js-build | ||
| note-editor@acec74d09b | ||
| reader@c6edbfff75 | ||
| resource | ||
| scripts | ||
| scss | ||
| styles@dff7452b24 | ||
| test | ||
| translators@60f2d542ff | ||
| types/gecko | ||
| .babelrc | ||
| .gitattributes | ||
| .gitignore | ||
| .gitmodules | ||
| chrome.manifest | ||
| CLAUDE.md | ||
| CONTRIBUTING.md | ||
| COPYING | ||
| eslint.config.mjs | ||
| package-lock.json | ||
| package.json | ||
| README.md | ||
| update.rdf | ||
| version | ||
Zotero
Zotero is a free, easy-to-use tool to help you collect, organize, cite, and share your research sources.
Please post feature requests or bug reports to the Zotero Forums. If you're having trouble with Zotero, see Getting Help.
For more information on how to use this source code, see the Zotero documentation.