mirror of
https://github.com/zotero/zotero.git
synced 2026-10-11 03:38:25 +00:00
Use token budget instead of character budget to enforce the limit on the memory usage during indexing. Tokens are the proper measure of embedding work, and different languages use different amount of tokens. e.g. English is ~4 characters per token and Chinese is ~1-2, which means that character budget can mean significantly more indexing work depending on user's library. |
||
|---|---|---|
| .. | ||
| components | ||
| content | ||
| resource | ||
| tests | ||
| chrome.manifest | ||
| runtests.sh | ||