diff --git a/benchmark/compaction/README.md b/benchmark/compaction/README.md index 9759eba9..06abf31a 100644 --- a/benchmark/compaction/README.md +++ b/benchmark/compaction/README.md @@ -73,4 +73,5 @@ For the recorded format amendment, native summaries used the pre-amendment sourc and permits only the held-out `exam` field to differ; the summary generator never receives that field. `costs.json` reports standalone summary/ranking input and output overhead separately from answer usage, including -recorded retries. Token counts do not apply model prices or cache discounts and are not monetary savings. +recorded retries. The chart converts summary + Jev + one Qwen answer usage to estimated USD with the dated rates in +[pricing.json](./pricing.json); cache discounts and trajectory collection are excluded. See [the formula](./SETTINGS.md#scoring-and-costs). diff --git a/benchmark/compaction/RESULTS.md b/benchmark/compaction/RESULTS.md index 92eb5989..61dbca3b 100644 --- a/benchmark/compaction/RESULTS.md +++ b/benchmark/compaction/RESULTS.md @@ -8,44 +8,62 @@ each summary remains byte-identical across its six conditions. ## Four primary tasks -| Method | Fact score | Answer input | Summary in / out | Jev in / out | -| --- | ---: | ---: | ---: | ---: | -| Full context | 100.0% | 4,014 | 0 / 0 | 0 / 0 | -| S | 82.0% | 2,480 | 5,379 / 200 | 0 / 0 | -| S + Jev Top 1 | 83.0% | 2,753 | 5,379 / 200 | 4,162 / 344 | -| S + Jev Top 2 | 94.5% | 2,974 | 5,379 / 200 | 4,162 / 344 | -| S + Jev Top 3 | 99.5% | 3,241 | 5,379 / 200 | 4,162 / 344 | -| S + Jev Top 4 | 98.1% | 3,496 | 5,379 / 200 | 4,162 / 344 | -| S + Jev Top 5 | 97.4% | 3,660 | 5,379 / 200 | 4,162 / 344 | -| H (Hermes, small budget) | 79.4% | 2,432 | 3,120 / 152 | 0 / 0 | -| H + Jev Top 1 | 85.3% | 2,705 | 3,120 / 152 | 4,162 / 344 | -| H + Jev Top 2 | 92.9% | 2,927 | 3,120 / 152 | 4,162 / 344 | -| H + Jev Top 3 | 97.9% | 3,193 | 3,120 / 152 | 4,162 / 344 | -| H + Jev Top 4 | 96.9% | 3,448 | 3,120 / 152 | 4,162 / 344 | -| H + Jev Top 5 | 96.9% | 3,613 | 3,120 / 152 | 4,162 / 344 | +| Method | Fact score | Answer input | Summary in / out | Jev in / out | Total USD | +| --- | ---: | ---: | ---: | ---: | ---: | +| Full context | 100.0% | 4,014 | 0 / 0 | 0 / 0 | $0.006992 | +| S | 82.0% | 2,480 | 5,379 / 200 | 0 / 0 | $0.014293 | +| S + Jev Top 1 | 83.0% | 2,753 | 5,379 / 200 | 4,162 / 344 | $0.014920 | +| S + Jev Top 2 | 94.5% | 2,974 | 5,379 / 200 | 4,162 / 344 | $0.015289 | +| S + Jev Top 3 | 99.5% | 3,241 | 5,379 / 200 | 4,162 / 344 | $0.015738 | +| S + Jev Top 4 | 98.1% | 3,496 | 5,379 / 200 | 4,162 / 344 | $0.016166 | +| S + Jev Top 5 | 97.4% | 3,660 | 5,379 / 200 | 4,162 / 344 | $0.016429 | +| H (Hermes, small budget) | 79.4% | 2,432 | 3,120 / 152 | 0 / 0 | $0.010245 | +| H + Jev Top 1 | 85.3% | 2,705 | 3,120 / 152 | 4,162 / 344 | $0.010877 | +| H + Jev Top 2 | 92.9% | 2,927 | 3,120 / 152 | 4,162 / 344 | $0.011244 | +| H + Jev Top 3 | 97.9% | 3,193 | 3,120 / 152 | 4,162 / 344 | $0.011697 | +| H + Jev Top 4 | 96.9% | 3,448 | 3,120 / 152 | 4,162 / 344 | $0.012114 | +| H + Jev Top 5 | 96.9% | 3,613 | 3,120 / 152 | 4,162 / 344 | $0.012388 | -![Quality, downstream input and Jev overhead by Top N](../../docs/figure/compaction-topn-results.svg) +![Quality, downstream input and estimated total fees by Top N](../../docs/figure/compaction-topn-results.svg) Both summary mechanisms benefit on this panel: S rises from **82.0% to 99.5%** at Top 3; H rises from **79.4% to 97.9%**. Mean downstream inputs at Top 3 are **3,241 / 3,193**, versus Full's **4,014**: reductions of **19.3% / 20.4%**. Top 4–5 do not improve these point estimates. This is an exploratory result on four synthetic task templates, not a general optimal-N claim, equivalence test or proof that Jev beats other equal-budget selectors. -Panel C plots Jev overhead against Top N: zero at Top 0, then the same **4,162 input + 344 output tokens** -(4,506 total) at Top 1–5 for either summary mechanism. Ranking scores every available unit once, irrespective of how -many units are subsequently retained. Summary-generation usage remains in the table. +Panel C plots **one-compression + one-answer total fees** against Top N, separately for S and H. +S increases from **$0.014293** at Top 0 to **$0.016429** at Top 5; +H increases from **$0.010245** to **$0.012388**. +Full's answering fee is **$0.006992**. More selected evidence increases downstream Qwen input cost; +Jev's ranking fee itself stays **$0.000175** for every N > 0. ## Token and cost accounting -All table values are mean provider tokens. Answer input includes the original system/task, summary/evidence and held-out -question. Summary and Jev columns are **one-time overhead per standalone condition**, including recorded length retries. -One frozen Jev ranking serves every N and either summary mechanism: Top 1–5 therefore have the same ranking cost; -it is neither multiplied by N nor divided by the number of sweep conditions. Summary-only and Full incur no Jev cost. -The two scoring repeats estimate one downstream answer; overhead is not amortized across those evaluation repeats. +Token columns are mean actual provider usage. Answer input includes the original system/task, summary/evidence and +held-out question. **Total USD** includes one summary generation, one complete Jev ranking for N > 0, and **one Qwen +answer (input + output)**. Recorded summary/ranking retries are included; trajectory collection and calibration are +excluded. Full pays its answering fee and has no compression preprocessing. The two scoring repeats estimate the +mean cost of one answer; preprocessing is charged once, not amortized across them. -These are token counts, **not currency**: model prices and cached-token discounts are not applied. Compression increases -one-off provider work here; lower downstream input alone does not demonstrate lower total cost. Repeated future turns -may amortize overhead, but that has not been measured. +For reference, preprocessing alone costs S $0.009864 / H $0.005902 at Top 0, +or S $0.010039 / H $0.006077 with Jev. Those preprocessing fees stay flat +at Top 1–5; the **overall** curve rises because the selected evidence must be read by Qwen when answering. + +Prices checked **2026-10-16**, per million tokens: + +| Model | Input USD | Output USD | Source | +| --- | ---: | ---: | --- | +| Qwen `qwen3.8-max`, China (Beijing) | $1.65 | $4.951 | [Alibaba Cloud](https://www.alibabacloud.com/help/en/model-studio/qwen3-8-max) | +| Jev `jev-1.13.0`, direct TypeSafe | $0.042 | $0 | [TypeSafe](https://docs.typesafe.ai/models) | + +`total_usd = ((summary_input + answer_input) × 1.65 + (summary_output + answer_output) × 4.951 ++ jev_input × 0.042) / 1,000,000`. +[pricing.json](./pricing.json) freezes the rates, sources and scope. These are **uncached public-list-price estimates**: +all prompt tokens are priced at the regular input rate, even where API usage reports cache hits. Free quotas, discounts, +credits, taxes and account billing differences are excluded. This is a one-answer list-price comparison, not invoice reconciliation. In this horizon every compressed method +costs more than Full; future-turn amortization or total savings have not been established. +One shared ranking serves Top 1–5 and both mechanisms in the experiment; each standalone fused method includes its full +cost, neither multiplied by N nor divided across sweep conditions. ## Setup and boundaries @@ -72,21 +90,21 @@ the other 38 comprise 9 primary and 29 control contexts. All remain scored. Aver
Ten short/query-only controls -| Method | Fact score | Answer input | Summary in / out | Jev in / out | -| --- | ---: | ---: | ---: | ---: | -| Full context | 100.0% | 2,727 | 0 / 0 | 0 / 0 | -| S | 67.1% | 2,315 | 1,099 / 74 | 0 / 0 | -| S + Jev Top 1 | 84.7% | 2,493 | 1,099 / 74 | 929 / 50 | -| S + Jev Top 2 | 87.5% | 2,573 | 1,099 / 74 | 929 / 50 | -| S + Jev Top 3 | 92.9% | 2,687 | 1,099 / 74 | 929 / 50 | -| S + Jev Top 4 | 92.9% | 2,722 | 1,099 / 74 | 929 / 50 | -| S + Jev Top 5 | 92.5% | 2,722 | 1,099 / 74 | 929 / 50 | -| H (Hermes, small budget) | 73.2% | 2,308 | 1,944 / 69 | 0 / 0 | -| H + Jev Top 1 | 80.1% | 2,486 | 1,944 / 69 | 929 / 50 | -| H + Jev Top 2 | 92.5% | 2,566 | 1,944 / 69 | 929 / 50 | -| H + Jev Top 3 | 99.2% | 2,680 | 1,944 / 69 | 929 / 50 | -| H + Jev Top 4 | 99.2% | 2,715 | 1,944 / 69 | 929 / 50 | -| H + Jev Top 5 | 98.8% | 2,715 | 1,944 / 69 | 929 / 50 | +| Method | Fact score | Answer input | Summary in / out | Jev in / out | Total USD | +| --- | ---: | ---: | ---: | ---: | ---: | +| Full context | 100.0% | 2,727 | 0 / 0 | 0 / 0 | $0.004692 | +| S | 67.1% | 2,315 | 1,099 / 74 | 0 / 0 | $0.006175 | +| S + Jev Top 1 | 84.7% | 2,493 | 1,099 / 74 | 929 / 50 | $0.006517 | +| S + Jev Top 2 | 87.5% | 2,573 | 1,099 / 74 | 929 / 50 | $0.006654 | +| S + Jev Top 3 | 92.9% | 2,687 | 1,099 / 74 | 929 / 50 | $0.006843 | +| S + Jev Top 4 | 92.9% | 2,722 | 1,099 / 74 | 929 / 50 | $0.006901 | +| S + Jev Top 5 | 92.5% | 2,722 | 1,099 / 74 | 929 / 50 | $0.006901 | +| H (Hermes, small budget) | 73.2% | 2,308 | 1,944 / 69 | 0 / 0 | $0.007532 | +| H + Jev Top 1 | 80.1% | 2,486 | 1,944 / 69 | 929 / 50 | $0.007871 | +| H + Jev Top 2 | 92.5% | 2,566 | 1,944 / 69 | 929 / 50 | $0.008019 | +| H + Jev Top 3 | 99.2% | 2,680 | 1,944 / 69 | 929 / 50 | $0.008204 | +| H + Jev Top 4 | 99.2% | 2,715 | 1,944 / 69 | 929 / 50 | $0.008263 | +| H + Jev Top 5 | 98.8% | 2,715 | 1,944 / 69 | 929 / 50 | $0.008264 | These controls are not ten additional long-context tasks. Their small source-unit counts can make Top N saturate. diff --git a/benchmark/compaction/SETTINGS.md b/benchmark/compaction/SETTINGS.md index 70654a54..325d3f1b 100644 --- a/benchmark/compaction/SETTINGS.md +++ b/benchmark/compaction/SETTINGS.md @@ -26,7 +26,8 @@ this comparison. This is more specific than calling H a modified version of all The Qwen alias is what the requests and captured answering responses identify; an immutable provider weight revision was not supplied. Service-side updates and nondeterminism can change new-run results even at temperature zero. Use the same `LLM_BASE_URL` service for the closest rerun; this endpoint and API keys are supplied privately in `.env`. -No portable public-provider identity or monetary price schedule is asserted. +The endpoint was verified as the public China (Beijing) DashScope endpoint; API keys remain private. +The chart uses the published USD list-price schedule below rather than an account invoice. ## Runtime dependencies @@ -118,7 +119,20 @@ generations, generations within tasks, then tasks equally. Report primary/contro Answer input includes system/task, compressed history and exam. Standalone-method overhead includes one summary and, for N > 0, one whole Jev ranking, including recorded retries. Ranking cost is identical across N and both mechanisms; do not multiply by N or divide by sweep size. Evaluation repeats do not amortize overhead. Report input/output tokens -by model; no prices/cache discounts are applied and no total monetary saving is claimed. +by model. Total fees use [pricing.json](./pricing.json), checked 2026-10-16: Qwen `qwen3.8-max` China (Beijing) +input **$1.65/M**, output **$4.951/M** ([official rate](https://www.alibabacloud.com/help/en/model-studio/qwen3-8-max)); +Jev `jev-1.13.0` input **$0.042/M**, output free ([official rate](https://docs.typesafe.ai/models)). + +`total_usd = ((summary_input + answer_input) * 1.65 + (summary_output + answer_output) * 4.951 ++ jev_input * 0.042) / 1e6`. +At Top 0 charge summary generation plus one Qwen answer; Top 1–5 add one identical whole Jev ranking +and charge the Qwen answer on the enlarged context. Mean usage is averaged across +the same tasks/generations; fees are computed from unrounded usage. Recorded retries are included. The plotted amount +includes one downstream answer and excludes collection/calibration. Full pays its actual mean answering fee and +no compression preprocessing. Two scoring repeats estimate the cost of one answer; they do not amortize overhead. +These are uncached list-price estimates: cache hits are deliberately charged at the regular input rate for comparison. +No free quotas, promotions, taxes or account-specific adjustments are applied. Prices are a dated snapshot, not an invoice; +no total monetary saving is claimed. ## Reproduction levels and artifact availability diff --git a/benchmark/compaction/STUDY.md b/benchmark/compaction/STUDY.md index 2899cdd9..bcfcf354 100644 --- a/benchmark/compaction/STUDY.md +++ b/benchmark/compaction/STUDY.md @@ -114,5 +114,7 @@ shorten 172/210 contexts (51/60 primary, 121/150 controls). Summary-only H still For costs, each independent method includes one summary generation when applicable, one complete Jev ranking when N > 0, and one downstream answer. Recorded retry usage is included. One ranking is shared across the Top-N sweep and mechanisms in the actual experiment; its standalone-method cost is neither multiplied by N nor divided by sweep size. -The two answer repeats do not amortize compression overhead. Report provider input/output tokens separately; do not -claim currency or total savings without prices, cache billing and an explicit continuation horizon. +The two answer repeats do not amortize compression overhead. Report provider input/output tokens separately. +The current chart estimates summary + Jev + one downstream answer fees from the dated public USD rates in [pricing.json](./pricing.json), +using uncached input pricing and excluding trajectory collection. [Settings](./SETTINGS.md#scoring-and-costs) define +the formula and exclusions; this is not an account invoice or a claim of total pipeline savings. diff --git a/benchmark/compaction/pricing.json b/benchmark/compaction/pricing.json new file mode 100644 index 00000000..1540a7ff --- /dev/null +++ b/benchmark/compaction/pricing.json @@ -0,0 +1,18 @@ +{ + "checked_on": "2026-10-16", + "currency": "USD", + "unit": "per_1000000_tokens", + "qwen3.8-max": { + "input": 1.65, + "output": 4.951, + "region": "China (Beijing)", + "source": "https://www.alibabacloud.com/help/en/model-studio/qwen3-8-max" + }, + "jev-1.13.0": { + "input": 0.042, + "output": 0.0, + "source": "https://docs.typesafe.ai/models" + }, + "scope": "One summary generation plus one complete Jev ranking when N > 0, plus one downstream Qwen answer; recorded summary/ranking retries included; trajectory collection excluded", + "billing_basis": "Public list-price estimate; all input charged at uncached rate; no credits, promotions, taxes, exchange conversion or invoice reconciliation" +} diff --git a/docs/en/context_compaction.md b/docs/en/context_compaction.md index 5b9565d4..d7b071d5 100644 --- a/docs/en/context_compaction.md +++ b/docs/en/context_compaction.md @@ -126,9 +126,11 @@ Jev-selected originals across mechanisms. ![Quality, downstream input and compression overhead](../figure/compaction-topn-results.svg) On four primary task templates, S improves from **82.0% to 99.5%** at Top 3; H improves from **79.4% to 97.9%**. -Their mean answer inputs are **3,241 / 3,193 tokens**, versus Full's **4,014 / 100%**. The figure separates downstream -input from Jev ranking overhead; summary-generation usage is in the results table. Top 0 has no Jev call; Top 1–5 share -one complete ranking and have identical Jev overhead. Fewer answer tokens do not establish lower total monetary cost. +Their mean answer inputs are **3,241 / 3,193 tokens**, versus Full's **4,014 / 100%**. Panel C shows **total fees for one compression plus one Qwen answer**: +summary generation + Jev ranking + downstream input/output. S increases **$0.014293 → $0.016429** from Top 0 to 5; +H increases **$0.010245 → $0.012388**. Full costs **$0.006992** for its answer. +Jev's fee stays flat after Top 1; additional retained evidence increases Qwen's input fee. These are uncached, +dated public-list-price estimates, not invoices or evidence of one-answer savings. The panel has 14 predefined query/answer templates, 546 contexts and 1,092 direct-Qwen answers. Ten short/query-only controls are reported separately. H reuses Hermes' compaction summary-generation pipeline with a smaller budget diff --git a/docs/figure/compaction-topn-results.svg b/docs/figure/compaction-topn-results.svg index adb8aa15..05be182d 100644 --- a/docs/figure/compaction-topn-results.svg +++ b/docs/figure/compaction-topn-results.svg @@ -6,7 +6,7 @@ - 2026-10-16T17:38:17.442645 + 2026-10-16T17:47:05.316892 image/svg+xml @@ -40,7 +40,7 @@ z +" clip-path="url(#p766555bb3d)" style="fill: none; stroke-dasharray: 7.5,6; stroke-dashoffset: 0; stroke: #7b8798; stroke-width: 1.5"/> @@ -88,7 +88,7 @@ L 420.832463 180.72 +" clip-path="url(#p766555bb3d)" style="fill: none; stroke: #e8edf3; stroke-width: 0.9; stroke-linecap: square"/> @@ -99,7 +99,7 @@ L 420.832463 434.52 +" clip-path="url(#p766555bb3d)" style="fill: none; stroke: #e8edf3; stroke-width: 0.9; stroke-linecap: square"/> @@ -110,7 +110,7 @@ L 420.832463 392.22 +" clip-path="url(#p766555bb3d)" style="fill: none; stroke: #e8edf3; stroke-width: 0.9; stroke-linecap: square"/> @@ -121,7 +121,7 @@ L 420.832463 349.92 +" clip-path="url(#p766555bb3d)" style="fill: none; stroke: #e8edf3; stroke-width: 0.9; stroke-linecap: square"/> @@ -132,7 +132,7 @@ L 420.832463 307.62 +" clip-path="url(#p766555bb3d)" style="fill: none; stroke: #e8edf3; stroke-width: 0.9; stroke-linecap: square"/> @@ -143,7 +143,7 @@ L 420.832463 265.32 +" clip-path="url(#p766555bb3d)" style="fill: none; stroke: #e8edf3; stroke-width: 0.9; stroke-linecap: square"/> @@ -154,7 +154,7 @@ L 420.832463 223.02 +" clip-path="url(#p766555bb3d)" style="fill: none; stroke: #e8edf3; stroke-width: 0.9; stroke-linecap: square"/> @@ -177,9 +177,9 @@ L 217.646476 227.42625 L 279.217987 185.12625 L 340.789498 196.87625 L 402.361009 202.75125 -" clip-path="url(#pf6d0324672)" style="fill: none; stroke: #dc8f20; stroke-width: 2.6; stroke-linecap: square"/> +" clip-path="url(#p766555bb3d)" style="fill: none; stroke: #dc8f20; stroke-width: 2.6; stroke-linecap: square"/> - - - - - - - - + + + + + + + @@ -216,9 +216,9 @@ L 217.646476 240.645 L 279.217987 198.345 L 340.789498 207.1575 L 402.361009 207.1575 -" clip-path="url(#pf6d0324672)" style="fill: none; stroke: #596fc4; stroke-width: 2.6; stroke-linecap: square"/> +" clip-path="url(#p766555bb3d)" style="fill: none; stroke: #596fc4; stroke-width: 2.6; stroke-linecap: square"/> - - - - - - - - + + + + + + + @@ -267,7 +267,7 @@ z +" clip-path="url(#pbc4e44716b)" style="fill: none; stroke-dasharray: 7.5,6; stroke-dashoffset: 0; stroke: #7b8798; stroke-width: 1.5"/> @@ -315,7 +315,7 @@ L 877.440222 195.184761 +" clip-path="url(#pbc4e44716b)" style="fill: none; stroke: #e8edf3; stroke-width: 0.9; stroke-linecap: square"/> @@ -326,7 +326,7 @@ L 877.440222 384.495652 +" clip-path="url(#pbc4e44716b)" style="fill: none; stroke: #e8edf3; stroke-width: 0.9; stroke-linecap: square"/> @@ -337,7 +337,7 @@ L 877.440222 321.965217 +" clip-path="url(#pbc4e44716b)" style="fill: none; stroke: #e8edf3; stroke-width: 0.9; stroke-linecap: square"/> @@ -348,7 +348,7 @@ L 877.440222 259.434783 +" clip-path="url(#pbc4e44716b)" style="fill: none; stroke: #e8edf3; stroke-width: 0.9; stroke-linecap: square"/> @@ -371,14 +371,14 @@ L 674.254235 325.195957 L 735.825746 291.856813 L 797.397258 259.935026 L 858.968769 239.393778 -" clip-path="url(#p6f3dc61b1a)" style="fill: none; stroke: #dc8f20; stroke-width: 2.6; stroke-linecap: square"/> - - - - - - - +" clip-path="url(#pbc4e44716b)" style="fill: none; stroke: #dc8f20; stroke-width: 2.6; stroke-linecap: square"/> + + + + + + + @@ -397,14 +397,14 @@ L 674.254235 331.136348 L 735.825746 297.797204 L 797.397258 265.875417 L 858.968769 245.33417 -" clip-path="url(#p6f3dc61b1a)" style="fill: none; stroke: #596fc4; stroke-width: 2.6; stroke-linecap: square"/> - - - - - - - +" clip-path="url(#pbc4e44716b)" style="fill: none; stroke: #596fc4; stroke-width: 2.6; stroke-linecap: square"/> + + + + + + + @@ -436,37 +436,37 @@ z - 0 + 0 - 1 + 1 - 2 + 2 - 3 + 3 - 4 + 4 - 5 + 5 @@ -478,305 +478,192 @@ z +" clip-path="url(#pc18ffa5c03)" style="fill: none; stroke: #e8edf3; stroke-width: 0.9; stroke-linecap: square"/> - 0 + $0.000 - + - 1k + $0.005 - + - 2k + $0.010 - + - 3k + $0.015 - + - 4k + $0.020 - - - - - - - 5k - - - - Mean Jev tokens per run + + Estimated USD: compress + answer + + + - - + + + + + + + + + + - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - + + $0.01429 - 0 + $0.01643 + + + + + + + + + + + - 4,506 + $0.01025 - 4,506 + $0.01239 - 4,506 + Summary + Jev + one Qwen answer - 4,506 + Full: $0.00699 - 4,506 - - - Same ranking cost for S and H - - - C Jev ranking overhead - - - - - - - - - - - - - - - Input (4,162) - - - - - - Output (344) - + C Total cost: one answer - + Jev across two summary strategies - + Same source trajectories. Frozen summaries. Identical Jev selections. - + At Top 3 - + S: 99.5% | 3,241 input tokens H: 97.9% | 3,193 input tokens - + Primary cohort: 4 task templates, 24 answers per method; 10 separate controls. Top 3–5 shorten 9/12 primary contexts per method. - - Jev overhead is zero at Top 0 and identical at Top 1–5 for S and H. Each fused method is charged one complete ranking, including retries. + + C includes one summary, one Jev ranking at N > 0, and one Qwen answer (input + output). Full includes its Qwen answer fee. - - Tokens are not currency; summary-generation overhead is listed in the results table. Direct Qwen evaluation; no agent continuation. + + Uncached list-price estimate (2026-10-16): Qwen $1.65 input / $4.951 output per M tokens; Jev $0.042 input / $0 output. Not an invoice. - - + + - + - + Generic summary S - + - + - + Hermes small-budget H - + - + Full context - + - + - + diff --git a/docs/zh/context_compaction.md b/docs/zh/context_compaction.md index ee71fd03..41abd7f7 100644 --- a/docs/zh/context_compaction.md +++ b/docs/zh/context_compaction.md @@ -117,8 +117,10 @@ Top 0 表示纯摘要；每轮摘要逐字固定，两种机制使用完全相 4 个主任务中，Top 3 将 S 从 **82.0% 提升至 99.5%**，将 H 从 **79.4% 提升至 97.9%**。 平均回答输入分别为 **3241 / 3193 tokens**，Full 为 **4014 tokens / 100%**。 -图中分别展示效果、回答输入和 Jev 排序开销；摘要生成用量见结果表。Top 0 不调用 Jev，Top 1～5 共用完整排名， -因此 Jev 开销相同。回答输入减少不等于总费用降低。 +C 图纵轴为**一次压缩后回答的总费用（USD）**：摘要生成 + Jev 排序 + Qwen 回答输入／输出。 +从 Top 0 到 Top 5，S 为 **$0.014293 → $0.016429**，H 为 **$0.010245 → $0.012388**； +Full 的回答费用为 **$0.006992**。Jev 排名费在 Top 1～5 不变，但保留更多原文会增加 Qwen 的输入费。 +按公开标价、不计缓存折扣估算，不等同于实际账单；当前单次回答口径下，压缩组总费用均高于 Full。 共 14 个预设 query/answer 模板、546 个上下文、1092 次直接 Qwen 回答；10 个短任务对照单独报告。 H 复用 Hermes 的上下文压缩摘要生成流程，并采用小预算与长度约束；实验直接生成摘要，不经过自动触发和首尾窗口选择。