mirror of
https://github.com/agentscope-ai/ReMe.git
synced 2026-10-23 05:35:30 +00:00
docs: plot Jev token overhead by retained Top N
This commit is contained in:
jinli.yl
parent
65ef735841
commit
e1a3b42579
4 changed files with 361 additions and 224 deletions
|
|
@ -24,13 +24,17 @@ each summary remains byte-identical across its six conditions.
|
|||
| H + Jev Top 4 | 96.9% | 3,448 | 3,120 / 152 | 4,162 / 344 |
|
||||
| H + Jev Top 5 | 96.9% | 3,613 | 3,120 / 152 | 4,162 / 344 |
|
||||
|
||||
|
||||
|
||||
|
||||
Both summary mechanisms benefit on this panel: S rises from **82.0% to 99.5%** at Top 3; H rises from **79.4% to 97.9%**.
|
||||
Mean downstream inputs at Top 3 are **3,241 / 3,193**, versus Full's **4,014**: reductions of **19.3% / 20.4%**.
|
||||
Top 4–5 do not improve these point estimates. This is an exploratory result on four synthetic task templates,
|
||||
not a general optimal-N claim, equivalence test or proof that Jev beats other equal-budget selectors.
|
||||
|
||||
Panel C plots Jev overhead against Top N: zero at Top 0, then the same **4,162 input + 344 output tokens**
|
||||
(4,506 total) at Top 1–5 for either summary mechanism. Ranking scores every available unit once, irrespective of how
|
||||
many units are subsequently retained. Summary-generation usage remains in the table.
|
||||
|
||||
## Token and cost accounting
|
||||
|
||||
All table values are mean provider tokens. Answer input includes the original system/task, summary/evidence and held-out
|
||||
|
|
|
|||
|
|
@ -127,7 +127,8 @@ Jev-selected originals across mechanisms.
|
|||
|
||||
On four primary task templates, S improves from **82.0% to 99.5%** at Top 3; H improves from **79.4% to 97.9%**.
|
||||
Their mean answer inputs are **3,241 / 3,193 tokens**, versus Full's **4,014 / 100%**. The figure separates downstream
|
||||
input from summary/ranking overhead; fewer answer tokens do not establish lower total monetary cost.
|
||||
input from Jev ranking overhead; summary-generation usage is in the results table. Top 0 has no Jev call; Top 1–5 share
|
||||
one complete ranking and have identical Jev overhead. Fewer answer tokens do not establish lower total monetary cost.
|
||||
|
||||
The panel has 14 predefined query/answer templates, 546 contexts and 1,092 direct-Qwen answers. Ten short/query-only
|
||||
controls are reported separately. H reuses Hermes' compaction summary-generation pipeline with a smaller budget
|
||||
|
|
@ -136,5 +137,5 @@ selection. All H-only contexts shorten Full; at Top 3–5 only 9/12 primary cont
|
|||
in scoring. This is an incremental, unequal-length comparison without agent continuation.
|
||||
|
||||
<!-- Payment has 10 units (maximum Top 10); actual N is min(N, available units). -->
|
||||
See [models, framework versions, parameters and reproduction steps](https://github.com/jinliyl/ReMe/blob/feat/decision-model-compaction/benchmark/compaction/SETTINGS.md),
|
||||
the [query/answer protocol](https://github.com/jinliyl/ReMe/blob/d2638b166f6daeec6f3e842750313526617786a1/benchmark/compaction/STUDY.md) and [complete results and token accounting](https://github.com/jinliyl/ReMe/blob/d2638b166f6daeec6f3e842750313526617786a1/benchmark/compaction/RESULTS.md).
|
||||
See [models, framework versions, parameters and reproduction steps](https://github.com/jinliyl/ReMe/blob/65ef7358417d21746cf9f95aaf9005aa521c5688/benchmark/compaction/SETTINGS.md),
|
||||
the [query/answer protocol](https://github.com/jinliyl/ReMe/blob/65ef7358417d21746cf9f95aaf9005aa521c5688/benchmark/compaction/STUDY.md) and [complete results and token accounting](https://github.com/jinliyl/ReMe/blob/65ef7358417d21746cf9f95aaf9005aa521c5688/benchmark/compaction/RESULTS.md).
|
||||
|
|
|
|||
|
|
@ -6,7 +6,7 @@
|
|||
<rdf:RDF xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:cc="http://creativecommons.org/ns#" xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#">
|
||||
<cc:Work>
|
||||
<dc:type rdf:resource="http://purl.org/dc/dcmitype/StillImage"/>
|
||||
<dc:date>2026-10-16T17:18:34.287481</dc:date>
|
||||
<dc:date>2026-10-16T17:38:17.442645</dc:date>
|
||||
<dc:format>image/svg+xml</dc:format>
|
||||
<dc:creator>
|
||||
<cc:Agent>
|
||||
|
|
@ -40,47 +40,47 @@ z
|
|||
<g id="line2d_1">
|
||||
<path d="M 76.032 180.72
|
||||
L 420.832463 180.72
|
||||
" clip-path="url(#p260a3f0845)" style="fill: none; stroke-dasharray: 7.5,6; stroke-dashoffset: 0; stroke: #7b8798; stroke-width: 1.5"/>
|
||||
" clip-path="url(#pf6d0324672)" style="fill: none; stroke-dasharray: 7.5,6; stroke-dashoffset: 0; stroke: #7b8798; stroke-width: 1.5"/>
|
||||
</g>
|
||||
<g id="matplotlib.axis_1">
|
||||
<g id="xtick_1">
|
||||
<g id="line2d_2"/>
|
||||
<g id="text_1">
|
||||
<text style="font-size: 12px; font-family: 'DejaVu Sans'; text-anchor: middle; fill: #58677b" x="94.503453" y="450.637187" transform="rotate(-0 94.503453 450.637187)">0</text>
|
||||
<text style="font-size: 12px; font-family: 'DejaVu Sans', Arial, sans-serif; text-anchor: middle; fill: #58677b" x="94.503453" y="450.637187" transform="rotate(-0 94.503453 450.637187)">0</text>
|
||||
</g>
|
||||
</g>
|
||||
<g id="xtick_2">
|
||||
<g id="line2d_3"/>
|
||||
<g id="text_2">
|
||||
<text style="font-size: 12px; font-family: 'DejaVu Sans'; text-anchor: middle; fill: #58677b" x="156.074965" y="450.637187" transform="rotate(-0 156.074965 450.637187)">1</text>
|
||||
<text style="font-size: 12px; font-family: 'DejaVu Sans', Arial, sans-serif; text-anchor: middle; fill: #58677b" x="156.074965" y="450.637187" transform="rotate(-0 156.074965 450.637187)">1</text>
|
||||
</g>
|
||||
</g>
|
||||
<g id="xtick_3">
|
||||
<g id="line2d_4"/>
|
||||
<g id="text_3">
|
||||
<text style="font-size: 12px; font-family: 'DejaVu Sans'; text-anchor: middle; fill: #58677b" x="217.646476" y="450.637187" transform="rotate(-0 217.646476 450.637187)">2</text>
|
||||
<text style="font-size: 12px; font-family: 'DejaVu Sans', Arial, sans-serif; text-anchor: middle; fill: #58677b" x="217.646476" y="450.637187" transform="rotate(-0 217.646476 450.637187)">2</text>
|
||||
</g>
|
||||
</g>
|
||||
<g id="xtick_4">
|
||||
<g id="line2d_5"/>
|
||||
<g id="text_4">
|
||||
<text style="font-size: 12px; font-family: 'DejaVu Sans'; text-anchor: middle; fill: #58677b" x="279.217987" y="450.637187" transform="rotate(-0 279.217987 450.637187)">3</text>
|
||||
<text style="font-size: 12px; font-family: 'DejaVu Sans', Arial, sans-serif; text-anchor: middle; fill: #58677b" x="279.217987" y="450.637187" transform="rotate(-0 279.217987 450.637187)">3</text>
|
||||
</g>
|
||||
</g>
|
||||
<g id="xtick_5">
|
||||
<g id="line2d_6"/>
|
||||
<g id="text_5">
|
||||
<text style="font-size: 12px; font-family: 'DejaVu Sans'; text-anchor: middle; fill: #58677b" x="340.789498" y="450.637187" transform="rotate(-0 340.789498 450.637187)">4</text>
|
||||
<text style="font-size: 12px; font-family: 'DejaVu Sans', Arial, sans-serif; text-anchor: middle; fill: #58677b" x="340.789498" y="450.637187" transform="rotate(-0 340.789498 450.637187)">4</text>
|
||||
</g>
|
||||
</g>
|
||||
<g id="xtick_6">
|
||||
<g id="line2d_7"/>
|
||||
<g id="text_6">
|
||||
<text style="font-size: 12px; font-family: 'DejaVu Sans'; text-anchor: middle; fill: #58677b" x="402.361009" y="450.637187" transform="rotate(-0 402.361009 450.637187)">5</text>
|
||||
<text style="font-size: 12px; font-family: 'DejaVu Sans', Arial, sans-serif; text-anchor: middle; fill: #58677b" x="402.361009" y="450.637187" transform="rotate(-0 402.361009 450.637187)">5</text>
|
||||
</g>
|
||||
</g>
|
||||
<g id="text_7">
|
||||
<text style="font-size: 12px; font-family: 'DejaVu Sans'; text-anchor: middle; fill: #43536b" x="248.432231" y="475.638125" transform="rotate(-0 248.432231 475.638125)">Jev Top N (0 = summary only)</text>
|
||||
<text style="font-size: 12px; font-family: 'DejaVu Sans', Arial, sans-serif; text-anchor: middle; fill: #43536b" x="248.432231" y="475.638125" transform="rotate(-0 248.432231 475.638125)">Jev Top N (0 = summary only)</text>
|
||||
</g>
|
||||
</g>
|
||||
<g id="matplotlib.axis_2">
|
||||
|
|
@ -88,81 +88,81 @@ L 420.832463 180.72
|
|||
<g id="line2d_8">
|
||||
<path d="M 76.032 434.52
|
||||
L 420.832463 434.52
|
||||
" clip-path="url(#p260a3f0845)" style="fill: none; stroke: #e8edf3; stroke-width: 0.9; stroke-linecap: square"/>
|
||||
" clip-path="url(#pf6d0324672)" style="fill: none; stroke: #e8edf3; stroke-width: 0.9; stroke-linecap: square"/>
|
||||
</g>
|
||||
<g id="line2d_9"/>
|
||||
<g id="text_8">
|
||||
<text style="font-size: 12px; font-family: 'DejaVu Sans'; text-anchor: end; fill: #58677b" x="69.032" y="439.078594" transform="rotate(-0 69.032 439.078594)">70</text>
|
||||
<text style="font-size: 12px; font-family: 'DejaVu Sans', Arial, sans-serif; text-anchor: end; fill: #58677b" x="69.032" y="439.078594" transform="rotate(-0 69.032 439.078594)">70</text>
|
||||
</g>
|
||||
</g>
|
||||
<g id="ytick_2">
|
||||
<g id="line2d_10">
|
||||
<path d="M 76.032 392.22
|
||||
L 420.832463 392.22
|
||||
" clip-path="url(#p260a3f0845)" style="fill: none; stroke: #e8edf3; stroke-width: 0.9; stroke-linecap: square"/>
|
||||
" clip-path="url(#pf6d0324672)" style="fill: none; stroke: #e8edf3; stroke-width: 0.9; stroke-linecap: square"/>
|
||||
</g>
|
||||
<g id="line2d_11"/>
|
||||
<g id="text_9">
|
||||
<text style="font-size: 12px; font-family: 'DejaVu Sans'; text-anchor: end; fill: #58677b" x="69.032" y="396.778594" transform="rotate(-0 69.032 396.778594)">75</text>
|
||||
<text style="font-size: 12px; font-family: 'DejaVu Sans', Arial, sans-serif; text-anchor: end; fill: #58677b" x="69.032" y="396.778594" transform="rotate(-0 69.032 396.778594)">75</text>
|
||||
</g>
|
||||
</g>
|
||||
<g id="ytick_3">
|
||||
<g id="line2d_12">
|
||||
<path d="M 76.032 349.92
|
||||
L 420.832463 349.92
|
||||
" clip-path="url(#p260a3f0845)" style="fill: none; stroke: #e8edf3; stroke-width: 0.9; stroke-linecap: square"/>
|
||||
" clip-path="url(#pf6d0324672)" style="fill: none; stroke: #e8edf3; stroke-width: 0.9; stroke-linecap: square"/>
|
||||
</g>
|
||||
<g id="line2d_13"/>
|
||||
<g id="text_10">
|
||||
<text style="font-size: 12px; font-family: 'DejaVu Sans'; text-anchor: end; fill: #58677b" x="69.032" y="354.478594" transform="rotate(-0 69.032 354.478594)">80</text>
|
||||
<text style="font-size: 12px; font-family: 'DejaVu Sans', Arial, sans-serif; text-anchor: end; fill: #58677b" x="69.032" y="354.478594" transform="rotate(-0 69.032 354.478594)">80</text>
|
||||
</g>
|
||||
</g>
|
||||
<g id="ytick_4">
|
||||
<g id="line2d_14">
|
||||
<path d="M 76.032 307.62
|
||||
L 420.832463 307.62
|
||||
" clip-path="url(#p260a3f0845)" style="fill: none; stroke: #e8edf3; stroke-width: 0.9; stroke-linecap: square"/>
|
||||
" clip-path="url(#pf6d0324672)" style="fill: none; stroke: #e8edf3; stroke-width: 0.9; stroke-linecap: square"/>
|
||||
</g>
|
||||
<g id="line2d_15"/>
|
||||
<g id="text_11">
|
||||
<text style="font-size: 12px; font-family: 'DejaVu Sans'; text-anchor: end; fill: #58677b" x="69.032" y="312.178594" transform="rotate(-0 69.032 312.178594)">85</text>
|
||||
<text style="font-size: 12px; font-family: 'DejaVu Sans', Arial, sans-serif; text-anchor: end; fill: #58677b" x="69.032" y="312.178594" transform="rotate(-0 69.032 312.178594)">85</text>
|
||||
</g>
|
||||
</g>
|
||||
<g id="ytick_5">
|
||||
<g id="line2d_16">
|
||||
<path d="M 76.032 265.32
|
||||
L 420.832463 265.32
|
||||
" clip-path="url(#p260a3f0845)" style="fill: none; stroke: #e8edf3; stroke-width: 0.9; stroke-linecap: square"/>
|
||||
" clip-path="url(#pf6d0324672)" style="fill: none; stroke: #e8edf3; stroke-width: 0.9; stroke-linecap: square"/>
|
||||
</g>
|
||||
<g id="line2d_17"/>
|
||||
<g id="text_12">
|
||||
<text style="font-size: 12px; font-family: 'DejaVu Sans'; text-anchor: end; fill: #58677b" x="69.032" y="269.878594" transform="rotate(-0 69.032 269.878594)">90</text>
|
||||
<text style="font-size: 12px; font-family: 'DejaVu Sans', Arial, sans-serif; text-anchor: end; fill: #58677b" x="69.032" y="269.878594" transform="rotate(-0 69.032 269.878594)">90</text>
|
||||
</g>
|
||||
</g>
|
||||
<g id="ytick_6">
|
||||
<g id="line2d_18">
|
||||
<path d="M 76.032 223.02
|
||||
L 420.832463 223.02
|
||||
" clip-path="url(#p260a3f0845)" style="fill: none; stroke: #e8edf3; stroke-width: 0.9; stroke-linecap: square"/>
|
||||
" clip-path="url(#pf6d0324672)" style="fill: none; stroke: #e8edf3; stroke-width: 0.9; stroke-linecap: square"/>
|
||||
</g>
|
||||
<g id="line2d_19"/>
|
||||
<g id="text_13">
|
||||
<text style="font-size: 12px; font-family: 'DejaVu Sans'; text-anchor: end; fill: #58677b" x="69.032" y="227.578594" transform="rotate(-0 69.032 227.578594)">95</text>
|
||||
<text style="font-size: 12px; font-family: 'DejaVu Sans', Arial, sans-serif; text-anchor: end; fill: #58677b" x="69.032" y="227.578594" transform="rotate(-0 69.032 227.578594)">95</text>
|
||||
</g>
|
||||
</g>
|
||||
<g id="ytick_7">
|
||||
<g id="line2d_20">
|
||||
<path d="M 76.032 180.72
|
||||
L 420.832463 180.72
|
||||
" clip-path="url(#p260a3f0845)" style="fill: none; stroke: #e8edf3; stroke-width: 0.9; stroke-linecap: square"/>
|
||||
" clip-path="url(#pf6d0324672)" style="fill: none; stroke: #e8edf3; stroke-width: 0.9; stroke-linecap: square"/>
|
||||
</g>
|
||||
<g id="line2d_21"/>
|
||||
<g id="text_14">
|
||||
<text style="font-size: 12px; font-family: 'DejaVu Sans'; text-anchor: end; fill: #58677b" x="69.032" y="185.278594" transform="rotate(-0 69.032 185.278594)">100</text>
|
||||
<text style="font-size: 12px; font-family: 'DejaVu Sans', Arial, sans-serif; text-anchor: end; fill: #58677b" x="69.032" y="185.278594" transform="rotate(-0 69.032 185.278594)">100</text>
|
||||
</g>
|
||||
</g>
|
||||
<g id="text_15">
|
||||
<text style="font-size: 12px; font-family: 'DejaVu Sans'; text-anchor: middle; fill: #43536b" x="33.244188" y="290.7" transform="rotate(-90 33.244188 290.7)">Exact-field score (%)</text>
|
||||
<text style="font-size: 12px; font-family: 'DejaVu Sans', Arial, sans-serif; text-anchor: middle; fill: #43536b" x="33.244188" y="290.7" transform="rotate(-90 33.244188 290.7)">Exact-field score (%)</text>
|
||||
</g>
|
||||
</g>
|
||||
<g id="patch_3">
|
||||
|
|
@ -177,9 +177,9 @@ L 217.646476 227.42625
|
|||
L 279.217987 185.12625
|
||||
L 340.789498 196.87625
|
||||
L 402.361009 202.75125
|
||||
" clip-path="url(#p260a3f0845)" style="fill: none; stroke: #dc8f20; stroke-width: 2.6; stroke-linecap: square"/>
|
||||
" clip-path="url(#pf6d0324672)" style="fill: none; stroke: #dc8f20; stroke-width: 2.6; stroke-linecap: square"/>
|
||||
<defs>
|
||||
<path id="m90dc9a1c0b" d="M 0 3
|
||||
<path id="mde656dc075" d="M 0 3
|
||||
C 0.795609 3 1.55874 2.683901 2.12132 2.12132
|
||||
C 2.683901 1.55874 3 0.795609 3 0
|
||||
C 3 -0.795609 2.683901 -1.55874 2.12132 -2.12132
|
||||
|
|
@ -191,23 +191,23 @@ C -1.55874 2.683901 -0.795609 3 0 3
|
|||
z
|
||||
" style="stroke: #dc8f20"/>
|
||||
</defs>
|
||||
<g clip-path="url(#p260a3f0845)">
|
||||
<use xlink:href="#m90dc9a1c0b" x="94.503453" y="332.8825" style="fill: #dc8f20; stroke: #dc8f20"/>
|
||||
<use xlink:href="#m90dc9a1c0b" x="156.074965" y="324.95125" style="fill: #dc8f20; stroke: #dc8f20"/>
|
||||
<use xlink:href="#m90dc9a1c0b" x="217.646476" y="227.42625" style="fill: #dc8f20; stroke: #dc8f20"/>
|
||||
<use xlink:href="#m90dc9a1c0b" x="279.217987" y="185.12625" style="fill: #dc8f20; stroke: #dc8f20"/>
|
||||
<use xlink:href="#m90dc9a1c0b" x="340.789498" y="196.87625" style="fill: #dc8f20; stroke: #dc8f20"/>
|
||||
<use xlink:href="#m90dc9a1c0b" x="402.361009" y="202.75125" style="fill: #dc8f20; stroke: #dc8f20"/>
|
||||
<g clip-path="url(#pf6d0324672)">
|
||||
<use xlink:href="#mde656dc075" x="94.503453" y="332.8825" style="fill: #dc8f20; stroke: #dc8f20"/>
|
||||
<use xlink:href="#mde656dc075" x="156.074965" y="324.95125" style="fill: #dc8f20; stroke: #dc8f20"/>
|
||||
<use xlink:href="#mde656dc075" x="217.646476" y="227.42625" style="fill: #dc8f20; stroke: #dc8f20"/>
|
||||
<use xlink:href="#mde656dc075" x="279.217987" y="185.12625" style="fill: #dc8f20; stroke: #dc8f20"/>
|
||||
<use xlink:href="#mde656dc075" x="340.789498" y="196.87625" style="fill: #dc8f20; stroke: #dc8f20"/>
|
||||
<use xlink:href="#mde656dc075" x="402.361009" y="202.75125" style="fill: #dc8f20; stroke: #dc8f20"/>
|
||||
</g>
|
||||
</g>
|
||||
<g id="text_16">
|
||||
<text style="font-weight: 700; font-size: 10.5px; font-family: 'DejaVu Sans'; text-anchor: middle; fill: #dc8f20" x="94.503453" y="322.8825" transform="rotate(-0 94.503453 322.8825)">82.0%</text>
|
||||
<text style="font-weight: 700; font-size: 10.5px; font-family: 'DejaVu Sans', Arial, sans-serif; text-anchor: middle; fill: #dc8f20" x="94.503453" y="322.8825" transform="rotate(-0 94.503453 322.8825)">82.0%</text>
|
||||
</g>
|
||||
<g id="text_17">
|
||||
<text style="font-weight: 700; font-size: 10.5px; font-family: 'DejaVu Sans'; text-anchor: middle; fill: #dc8f20" x="279.217987" y="175.12625" transform="rotate(-0 279.217987 175.12625)">99.5%</text>
|
||||
<text style="font-weight: 700; font-size: 10.5px; font-family: 'DejaVu Sans', Arial, sans-serif; text-anchor: middle; fill: #dc8f20" x="279.217987" y="175.12625" transform="rotate(-0 279.217987 175.12625)">99.5%</text>
|
||||
</g>
|
||||
<g id="text_18">
|
||||
<text style="font-weight: 700; font-size: 10.5px; font-family: 'DejaVu Sans'; text-anchor: middle; fill: #dc8f20" x="402.361009" y="192.75125" transform="rotate(-0 402.361009 192.75125)">97.4%</text>
|
||||
<text style="font-weight: 700; font-size: 10.5px; font-family: 'DejaVu Sans', Arial, sans-serif; text-anchor: middle; fill: #dc8f20" x="402.361009" y="192.75125" transform="rotate(-0 402.361009 192.75125)">97.4%</text>
|
||||
</g>
|
||||
<g id="line2d_23">
|
||||
<path d="M 94.503453 355.2075
|
||||
|
|
@ -216,9 +216,9 @@ L 217.646476 240.645
|
|||
L 279.217987 198.345
|
||||
L 340.789498 207.1575
|
||||
L 402.361009 207.1575
|
||||
" clip-path="url(#p260a3f0845)" style="fill: none; stroke: #596fc4; stroke-width: 2.6; stroke-linecap: square"/>
|
||||
" clip-path="url(#pf6d0324672)" style="fill: none; stroke: #596fc4; stroke-width: 2.6; stroke-linecap: square"/>
|
||||
<defs>
|
||||
<path id="m712a07e464" d="M 0 3
|
||||
<path id="m05292c0fc7" d="M 0 3
|
||||
C 0.795609 3 1.55874 2.683901 2.12132 2.12132
|
||||
C 2.683901 1.55874 3 0.795609 3 0
|
||||
C 3 -0.795609 2.683901 -1.55874 2.12132 -2.12132
|
||||
|
|
@ -230,29 +230,29 @@ C -1.55874 2.683901 -0.795609 3 0 3
|
|||
z
|
||||
" style="stroke: #596fc4"/>
|
||||
</defs>
|
||||
<g clip-path="url(#p260a3f0845)">
|
||||
<use xlink:href="#m712a07e464" x="94.503453" y="355.2075" style="fill: #596fc4; stroke: #596fc4"/>
|
||||
<use xlink:href="#m712a07e464" x="156.074965" y="304.97625" style="fill: #596fc4; stroke: #596fc4"/>
|
||||
<use xlink:href="#m712a07e464" x="217.646476" y="240.645" style="fill: #596fc4; stroke: #596fc4"/>
|
||||
<use xlink:href="#m712a07e464" x="279.217987" y="198.345" style="fill: #596fc4; stroke: #596fc4"/>
|
||||
<use xlink:href="#m712a07e464" x="340.789498" y="207.1575" style="fill: #596fc4; stroke: #596fc4"/>
|
||||
<use xlink:href="#m712a07e464" x="402.361009" y="207.1575" style="fill: #596fc4; stroke: #596fc4"/>
|
||||
<g clip-path="url(#pf6d0324672)">
|
||||
<use xlink:href="#m05292c0fc7" x="94.503453" y="355.2075" style="fill: #596fc4; stroke: #596fc4"/>
|
||||
<use xlink:href="#m05292c0fc7" x="156.074965" y="304.97625" style="fill: #596fc4; stroke: #596fc4"/>
|
||||
<use xlink:href="#m05292c0fc7" x="217.646476" y="240.645" style="fill: #596fc4; stroke: #596fc4"/>
|
||||
<use xlink:href="#m05292c0fc7" x="279.217987" y="198.345" style="fill: #596fc4; stroke: #596fc4"/>
|
||||
<use xlink:href="#m05292c0fc7" x="340.789498" y="207.1575" style="fill: #596fc4; stroke: #596fc4"/>
|
||||
<use xlink:href="#m05292c0fc7" x="402.361009" y="207.1575" style="fill: #596fc4; stroke: #596fc4"/>
|
||||
</g>
|
||||
</g>
|
||||
<g id="text_19">
|
||||
<text style="font-weight: 700; font-size: 10.5px; font-family: 'DejaVu Sans'; text-anchor: middle; fill: #596fc4" x="94.503453" y="373.2075" transform="rotate(-0 94.503453 373.2075)">79.4%</text>
|
||||
<text style="font-weight: 700; font-size: 10.5px; font-family: 'DejaVu Sans', Arial, sans-serif; text-anchor: middle; fill: #596fc4" x="94.503453" y="373.2075" transform="rotate(-0 94.503453 373.2075)">79.4%</text>
|
||||
</g>
|
||||
<g id="text_20">
|
||||
<text style="font-weight: 700; font-size: 10.5px; font-family: 'DejaVu Sans'; text-anchor: middle; fill: #596fc4" x="279.217987" y="216.345" transform="rotate(-0 279.217987 216.345)">97.9%</text>
|
||||
<text style="font-weight: 700; font-size: 10.5px; font-family: 'DejaVu Sans', Arial, sans-serif; text-anchor: middle; fill: #596fc4" x="279.217987" y="216.345" transform="rotate(-0 279.217987 216.345)">97.9%</text>
|
||||
</g>
|
||||
<g id="text_21">
|
||||
<text style="font-weight: 700; font-size: 10.5px; font-family: 'DejaVu Sans'; text-anchor: middle; fill: #596fc4" x="402.361009" y="225.1575" transform="rotate(-0 402.361009 225.1575)">96.9%</text>
|
||||
<text style="font-weight: 700; font-size: 10.5px; font-family: 'DejaVu Sans', Arial, sans-serif; text-anchor: middle; fill: #596fc4" x="402.361009" y="225.1575" transform="rotate(-0 402.361009 225.1575)">96.9%</text>
|
||||
</g>
|
||||
<g id="text_22">
|
||||
<text style="font-size: 11px; font-family: 'DejaVu Sans'; text-anchor: end; fill: #7b8798" x="413.936453" y="163.867481" transform="rotate(-0 413.936453 163.867481)">Full: 100%</text>
|
||||
<text style="font-size: 11px; font-family: 'DejaVu Sans', Arial, sans-serif; text-anchor: end; fill: #7b8798" x="413.936453" y="163.867481" transform="rotate(-0 413.936453 163.867481)">Full: 100%</text>
|
||||
</g>
|
||||
<g id="text_23">
|
||||
<text style="font-weight: 700; font-size: 15px; font-family: 'DejaVu Sans'; text-anchor: start; fill: #172d45" x="76.032" y="123.88" transform="rotate(-0 76.032 123.88)">A Answer quality</text>
|
||||
<text style="font-weight: 700; font-size: 15px; font-family: 'DejaVu Sans', Arial, sans-serif; text-anchor: start; fill: #172d45" x="76.032" y="123.88" transform="rotate(-0 76.032 123.88)">A Answer quality</text>
|
||||
</g>
|
||||
</g>
|
||||
<g id="axes_2">
|
||||
|
|
@ -267,47 +267,47 @@ z
|
|||
<g id="line2d_24">
|
||||
<path d="M 532.639759 195.184761
|
||||
L 877.440222 195.184761
|
||||
" clip-path="url(#pde7327f20e)" style="fill: none; stroke-dasharray: 7.5,6; stroke-dashoffset: 0; stroke: #7b8798; stroke-width: 1.5"/>
|
||||
" clip-path="url(#p6f3dc61b1a)" style="fill: none; stroke-dasharray: 7.5,6; stroke-dashoffset: 0; stroke: #7b8798; stroke-width: 1.5"/>
|
||||
</g>
|
||||
<g id="matplotlib.axis_3">
|
||||
<g id="xtick_7">
|
||||
<g id="line2d_25"/>
|
||||
<g id="text_24">
|
||||
<text style="font-size: 12px; font-family: 'DejaVu Sans'; text-anchor: middle; fill: #58677b" x="551.111213" y="450.637187" transform="rotate(-0 551.111213 450.637187)">0</text>
|
||||
<text style="font-size: 12px; font-family: 'DejaVu Sans', Arial, sans-serif; text-anchor: middle; fill: #58677b" x="551.111213" y="450.637187" transform="rotate(-0 551.111213 450.637187)">0</text>
|
||||
</g>
|
||||
</g>
|
||||
<g id="xtick_8">
|
||||
<g id="line2d_26"/>
|
||||
<g id="text_25">
|
||||
<text style="font-size: 12px; font-family: 'DejaVu Sans'; text-anchor: middle; fill: #58677b" x="612.682724" y="450.637187" transform="rotate(-0 612.682724 450.637187)">1</text>
|
||||
<text style="font-size: 12px; font-family: 'DejaVu Sans', Arial, sans-serif; text-anchor: middle; fill: #58677b" x="612.682724" y="450.637187" transform="rotate(-0 612.682724 450.637187)">1</text>
|
||||
</g>
|
||||
</g>
|
||||
<g id="xtick_9">
|
||||
<g id="line2d_27"/>
|
||||
<g id="text_26">
|
||||
<text style="font-size: 12px; font-family: 'DejaVu Sans'; text-anchor: middle; fill: #58677b" x="674.254235" y="450.637187" transform="rotate(-0 674.254235 450.637187)">2</text>
|
||||
<text style="font-size: 12px; font-family: 'DejaVu Sans', Arial, sans-serif; text-anchor: middle; fill: #58677b" x="674.254235" y="450.637187" transform="rotate(-0 674.254235 450.637187)">2</text>
|
||||
</g>
|
||||
</g>
|
||||
<g id="xtick_10">
|
||||
<g id="line2d_28"/>
|
||||
<g id="text_27">
|
||||
<text style="font-size: 12px; font-family: 'DejaVu Sans'; text-anchor: middle; fill: #58677b" x="735.825746" y="450.637187" transform="rotate(-0 735.825746 450.637187)">3</text>
|
||||
<text style="font-size: 12px; font-family: 'DejaVu Sans', Arial, sans-serif; text-anchor: middle; fill: #58677b" x="735.825746" y="450.637187" transform="rotate(-0 735.825746 450.637187)">3</text>
|
||||
</g>
|
||||
</g>
|
||||
<g id="xtick_11">
|
||||
<g id="line2d_29"/>
|
||||
<g id="text_28">
|
||||
<text style="font-size: 12px; font-family: 'DejaVu Sans'; text-anchor: middle; fill: #58677b" x="797.397258" y="450.637187" transform="rotate(-0 797.397258 450.637187)">4</text>
|
||||
<text style="font-size: 12px; font-family: 'DejaVu Sans', Arial, sans-serif; text-anchor: middle; fill: #58677b" x="797.397258" y="450.637187" transform="rotate(-0 797.397258 450.637187)">4</text>
|
||||
</g>
|
||||
</g>
|
||||
<g id="xtick_12">
|
||||
<g id="line2d_30"/>
|
||||
<g id="text_29">
|
||||
<text style="font-size: 12px; font-family: 'DejaVu Sans'; text-anchor: middle; fill: #58677b" x="858.968769" y="450.637187" transform="rotate(-0 858.968769 450.637187)">5</text>
|
||||
<text style="font-size: 12px; font-family: 'DejaVu Sans', Arial, sans-serif; text-anchor: middle; fill: #58677b" x="858.968769" y="450.637187" transform="rotate(-0 858.968769 450.637187)">5</text>
|
||||
</g>
|
||||
</g>
|
||||
<g id="text_30">
|
||||
<text style="font-size: 12px; font-family: 'DejaVu Sans'; text-anchor: middle; fill: #43536b" x="705.039991" y="475.638125" transform="rotate(-0 705.039991 475.638125)">Jev Top N (0 = summary only)</text>
|
||||
<text style="font-size: 12px; font-family: 'DejaVu Sans', Arial, sans-serif; text-anchor: middle; fill: #43536b" x="705.039991" y="475.638125" transform="rotate(-0 705.039991 475.638125)">Jev Top N (0 = summary only)</text>
|
||||
</g>
|
||||
</g>
|
||||
<g id="matplotlib.axis_4">
|
||||
|
|
@ -315,48 +315,48 @@ L 877.440222 195.184761
|
|||
<g id="line2d_31">
|
||||
<path d="M 532.639759 384.495652
|
||||
L 877.440222 384.495652
|
||||
" clip-path="url(#pde7327f20e)" style="fill: none; stroke: #e8edf3; stroke-width: 0.9; stroke-linecap: square"/>
|
||||
" clip-path="url(#p6f3dc61b1a)" style="fill: none; stroke: #e8edf3; stroke-width: 0.9; stroke-linecap: square"/>
|
||||
</g>
|
||||
<g id="line2d_32"/>
|
||||
<g id="text_31">
|
||||
<text style="font-size: 12px; font-family: 'DejaVu Sans'; text-anchor: end; fill: #58677b" x="525.639759" y="389.054246" transform="rotate(-0 525.639759 389.054246)">2500</text>
|
||||
<text style="font-size: 12px; font-family: 'DejaVu Sans', Arial, sans-serif; text-anchor: end; fill: #58677b" x="525.639759" y="389.054246" transform="rotate(-0 525.639759 389.054246)">2500</text>
|
||||
</g>
|
||||
</g>
|
||||
<g id="ytick_9">
|
||||
<g id="line2d_33">
|
||||
<path d="M 532.639759 321.965217
|
||||
L 877.440222 321.965217
|
||||
" clip-path="url(#pde7327f20e)" style="fill: none; stroke: #e8edf3; stroke-width: 0.9; stroke-linecap: square"/>
|
||||
" clip-path="url(#p6f3dc61b1a)" style="fill: none; stroke: #e8edf3; stroke-width: 0.9; stroke-linecap: square"/>
|
||||
</g>
|
||||
<g id="line2d_34"/>
|
||||
<g id="text_32">
|
||||
<text style="font-size: 12px; font-family: 'DejaVu Sans'; text-anchor: end; fill: #58677b" x="525.639759" y="326.523811" transform="rotate(-0 525.639759 326.523811)">3000</text>
|
||||
<text style="font-size: 12px; font-family: 'DejaVu Sans', Arial, sans-serif; text-anchor: end; fill: #58677b" x="525.639759" y="326.523811" transform="rotate(-0 525.639759 326.523811)">3000</text>
|
||||
</g>
|
||||
</g>
|
||||
<g id="ytick_10">
|
||||
<g id="line2d_35">
|
||||
<path d="M 532.639759 259.434783
|
||||
L 877.440222 259.434783
|
||||
" clip-path="url(#pde7327f20e)" style="fill: none; stroke: #e8edf3; stroke-width: 0.9; stroke-linecap: square"/>
|
||||
" clip-path="url(#p6f3dc61b1a)" style="fill: none; stroke: #e8edf3; stroke-width: 0.9; stroke-linecap: square"/>
|
||||
</g>
|
||||
<g id="line2d_36"/>
|
||||
<g id="text_33">
|
||||
<text style="font-size: 12px; font-family: 'DejaVu Sans'; text-anchor: end; fill: #58677b" x="525.639759" y="263.993376" transform="rotate(-0 525.639759 263.993376)">3500</text>
|
||||
<text style="font-size: 12px; font-family: 'DejaVu Sans', Arial, sans-serif; text-anchor: end; fill: #58677b" x="525.639759" y="263.993376" transform="rotate(-0 525.639759 263.993376)">3500</text>
|
||||
</g>
|
||||
</g>
|
||||
<g id="ytick_11">
|
||||
<g id="line2d_37">
|
||||
<path d="M 532.639759 196.904348
|
||||
L 877.440222 196.904348
|
||||
" clip-path="url(#pde7327f20e)" style="fill: none; stroke: #e8edf3; stroke-width: 0.9; stroke-linecap: square"/>
|
||||
" clip-path="url(#p6f3dc61b1a)" style="fill: none; stroke: #e8edf3; stroke-width: 0.9; stroke-linecap: square"/>
|
||||
</g>
|
||||
<g id="line2d_38"/>
|
||||
<g id="text_34">
|
||||
<text style="font-size: 12px; font-family: 'DejaVu Sans'; text-anchor: end; fill: #58677b" x="525.639759" y="201.462942" transform="rotate(-0 525.639759 201.462942)">4000</text>
|
||||
<text style="font-size: 12px; font-family: 'DejaVu Sans', Arial, sans-serif; text-anchor: end; fill: #58677b" x="525.639759" y="201.462942" transform="rotate(-0 525.639759 201.462942)">4000</text>
|
||||
</g>
|
||||
</g>
|
||||
<g id="text_35">
|
||||
<text style="font-size: 12px; font-family: 'DejaVu Sans'; text-anchor: middle; fill: #43536b" x="482.216947" y="290.7" transform="rotate(-90 482.216947 290.7)">Mean answer input tokens</text>
|
||||
<text style="font-size: 12px; font-family: 'DejaVu Sans', Arial, sans-serif; text-anchor: middle; fill: #43536b" x="482.216947" y="290.7" transform="rotate(-90 482.216947 290.7)">Mean answer input tokens</text>
|
||||
</g>
|
||||
</g>
|
||||
<g id="patch_5">
|
||||
|
|
@ -371,24 +371,24 @@ L 674.254235 325.195957
|
|||
L 735.825746 291.856813
|
||||
L 797.397258 259.935026
|
||||
L 858.968769 239.393778
|
||||
" clip-path="url(#pde7327f20e)" style="fill: none; stroke: #dc8f20; stroke-width: 2.6; stroke-linecap: square"/>
|
||||
<g clip-path="url(#pde7327f20e)">
|
||||
<use xlink:href="#m90dc9a1c0b" x="551.111213" y="387.007291" style="fill: #dc8f20; stroke: #dc8f20"/>
|
||||
<use xlink:href="#m90dc9a1c0b" x="612.682724" y="352.896939" style="fill: #dc8f20; stroke: #dc8f20"/>
|
||||
<use xlink:href="#m90dc9a1c0b" x="674.254235" y="325.195957" style="fill: #dc8f20; stroke: #dc8f20"/>
|
||||
<use xlink:href="#m90dc9a1c0b" x="735.825746" y="291.856813" style="fill: #dc8f20; stroke: #dc8f20"/>
|
||||
<use xlink:href="#m90dc9a1c0b" x="797.397258" y="259.935026" style="fill: #dc8f20; stroke: #dc8f20"/>
|
||||
<use xlink:href="#m90dc9a1c0b" x="858.968769" y="239.393778" style="fill: #dc8f20; stroke: #dc8f20"/>
|
||||
" clip-path="url(#p6f3dc61b1a)" style="fill: none; stroke: #dc8f20; stroke-width: 2.6; stroke-linecap: square"/>
|
||||
<g clip-path="url(#p6f3dc61b1a)">
|
||||
<use xlink:href="#mde656dc075" x="551.111213" y="387.007291" style="fill: #dc8f20; stroke: #dc8f20"/>
|
||||
<use xlink:href="#mde656dc075" x="612.682724" y="352.896939" style="fill: #dc8f20; stroke: #dc8f20"/>
|
||||
<use xlink:href="#mde656dc075" x="674.254235" y="325.195957" style="fill: #dc8f20; stroke: #dc8f20"/>
|
||||
<use xlink:href="#mde656dc075" x="735.825746" y="291.856813" style="fill: #dc8f20; stroke: #dc8f20"/>
|
||||
<use xlink:href="#mde656dc075" x="797.397258" y="259.935026" style="fill: #dc8f20; stroke: #dc8f20"/>
|
||||
<use xlink:href="#mde656dc075" x="858.968769" y="239.393778" style="fill: #dc8f20; stroke: #dc8f20"/>
|
||||
</g>
|
||||
</g>
|
||||
<g id="text_36">
|
||||
<text style="font-weight: 700; font-size: 10.5px; font-family: 'DejaVu Sans'; text-anchor: middle; fill: #dc8f20" x="551.111213" y="377.007291" transform="rotate(-0 551.111213 377.007291)">2,480</text>
|
||||
<text style="font-weight: 700; font-size: 10.5px; font-family: 'DejaVu Sans', Arial, sans-serif; text-anchor: middle; fill: #dc8f20" x="551.111213" y="377.007291" transform="rotate(-0 551.111213 377.007291)">2,480</text>
|
||||
</g>
|
||||
<g id="text_37">
|
||||
<text style="font-weight: 700; font-size: 10.5px; font-family: 'DejaVu Sans'; text-anchor: middle; fill: #dc8f20" x="735.825746" y="281.856813" transform="rotate(-0 735.825746 281.856813)">3,241</text>
|
||||
<text style="font-weight: 700; font-size: 10.5px; font-family: 'DejaVu Sans', Arial, sans-serif; text-anchor: middle; fill: #dc8f20" x="735.825746" y="281.856813" transform="rotate(-0 735.825746 281.856813)">3,241</text>
|
||||
</g>
|
||||
<g id="text_38">
|
||||
<text style="font-weight: 700; font-size: 10.5px; font-family: 'DejaVu Sans'; text-anchor: middle; fill: #dc8f20" x="858.968769" y="229.393778" transform="rotate(-0 858.968769 229.393778)">3,660</text>
|
||||
<text style="font-weight: 700; font-size: 10.5px; font-family: 'DejaVu Sans', Arial, sans-serif; text-anchor: middle; fill: #dc8f20" x="858.968769" y="229.393778" transform="rotate(-0 858.968769 229.393778)">3,660</text>
|
||||
</g>
|
||||
<g id="line2d_40">
|
||||
<path d="M 551.111213 392.947683
|
||||
|
|
@ -397,30 +397,30 @@ L 674.254235 331.136348
|
|||
L 735.825746 297.797204
|
||||
L 797.397258 265.875417
|
||||
L 858.968769 245.33417
|
||||
" clip-path="url(#pde7327f20e)" style="fill: none; stroke: #596fc4; stroke-width: 2.6; stroke-linecap: square"/>
|
||||
<g clip-path="url(#pde7327f20e)">
|
||||
<use xlink:href="#m712a07e464" x="551.111213" y="392.947683" style="fill: #596fc4; stroke: #596fc4"/>
|
||||
<use xlink:href="#m712a07e464" x="612.682724" y="358.83733" style="fill: #596fc4; stroke: #596fc4"/>
|
||||
<use xlink:href="#m712a07e464" x="674.254235" y="331.136348" style="fill: #596fc4; stroke: #596fc4"/>
|
||||
<use xlink:href="#m712a07e464" x="735.825746" y="297.797204" style="fill: #596fc4; stroke: #596fc4"/>
|
||||
<use xlink:href="#m712a07e464" x="797.397258" y="265.875417" style="fill: #596fc4; stroke: #596fc4"/>
|
||||
<use xlink:href="#m712a07e464" x="858.968769" y="245.33417" style="fill: #596fc4; stroke: #596fc4"/>
|
||||
" clip-path="url(#p6f3dc61b1a)" style="fill: none; stroke: #596fc4; stroke-width: 2.6; stroke-linecap: square"/>
|
||||
<g clip-path="url(#p6f3dc61b1a)">
|
||||
<use xlink:href="#m05292c0fc7" x="551.111213" y="392.947683" style="fill: #596fc4; stroke: #596fc4"/>
|
||||
<use xlink:href="#m05292c0fc7" x="612.682724" y="358.83733" style="fill: #596fc4; stroke: #596fc4"/>
|
||||
<use xlink:href="#m05292c0fc7" x="674.254235" y="331.136348" style="fill: #596fc4; stroke: #596fc4"/>
|
||||
<use xlink:href="#m05292c0fc7" x="735.825746" y="297.797204" style="fill: #596fc4; stroke: #596fc4"/>
|
||||
<use xlink:href="#m05292c0fc7" x="797.397258" y="265.875417" style="fill: #596fc4; stroke: #596fc4"/>
|
||||
<use xlink:href="#m05292c0fc7" x="858.968769" y="245.33417" style="fill: #596fc4; stroke: #596fc4"/>
|
||||
</g>
|
||||
</g>
|
||||
<g id="text_39">
|
||||
<text style="font-weight: 700; font-size: 10.5px; font-family: 'DejaVu Sans'; text-anchor: middle; fill: #596fc4" x="551.111213" y="410.947683" transform="rotate(-0 551.111213 410.947683)">2,432</text>
|
||||
<text style="font-weight: 700; font-size: 10.5px; font-family: 'DejaVu Sans', Arial, sans-serif; text-anchor: middle; fill: #596fc4" x="551.111213" y="410.947683" transform="rotate(-0 551.111213 410.947683)">2,432</text>
|
||||
</g>
|
||||
<g id="text_40">
|
||||
<text style="font-weight: 700; font-size: 10.5px; font-family: 'DejaVu Sans'; text-anchor: middle; fill: #596fc4" x="735.825746" y="315.797204" transform="rotate(-0 735.825746 315.797204)">3,193</text>
|
||||
<text style="font-weight: 700; font-size: 10.5px; font-family: 'DejaVu Sans', Arial, sans-serif; text-anchor: middle; fill: #596fc4" x="735.825746" y="315.797204" transform="rotate(-0 735.825746 315.797204)">3,193</text>
|
||||
</g>
|
||||
<g id="text_41">
|
||||
<text style="font-weight: 700; font-size: 10.5px; font-family: 'DejaVu Sans'; text-anchor: middle; fill: #596fc4" x="858.968769" y="263.33417" transform="rotate(-0 858.968769 263.33417)">3,613</text>
|
||||
<text style="font-weight: 700; font-size: 10.5px; font-family: 'DejaVu Sans', Arial, sans-serif; text-anchor: middle; fill: #596fc4" x="858.968769" y="263.33417" transform="rotate(-0 858.968769 263.33417)">3,613</text>
|
||||
</g>
|
||||
<g id="text_42">
|
||||
<text style="font-size: 11px; font-family: 'DejaVu Sans'; text-anchor: end; fill: #7b8798" x="870.544213" y="163.867481" transform="rotate(-0 870.544213 163.867481)">Full: 4,014</text>
|
||||
<text style="font-size: 11px; font-family: 'DejaVu Sans', Arial, sans-serif; text-anchor: end; fill: #7b8798" x="870.544213" y="163.867481" transform="rotate(-0 870.544213 163.867481)">Full: 4,014</text>
|
||||
</g>
|
||||
<g id="text_43">
|
||||
<text style="font-weight: 700; font-size: 15px; font-family: 'DejaVu Sans'; text-anchor: start; fill: #172d45" x="532.639759" y="123.88" transform="rotate(-0 532.639759 123.88)">B Downstream context</text>
|
||||
<text style="font-weight: 700; font-size: 15px; font-family: 'DejaVu Sans', Arial, sans-serif; text-anchor: start; fill: #172d45" x="532.639759" y="123.88" transform="rotate(-0 532.639759 123.88)">B Downstream context</text>
|
||||
</g>
|
||||
</g>
|
||||
<g id="axes_3">
|
||||
|
|
@ -434,218 +434,349 @@ z
|
|||
</g>
|
||||
<g id="matplotlib.axis_5">
|
||||
<g id="xtick_13">
|
||||
<g id="line2d_41">
|
||||
<path d="M 989.247519 434.52
|
||||
L 989.247519 146.88
|
||||
" clip-path="url(#p7ebe458925)" style="fill: none; stroke: #e8edf3; stroke-width: 0.9; stroke-linecap: square"/>
|
||||
</g>
|
||||
<g id="line2d_42"/>
|
||||
<g id="line2d_41"/>
|
||||
<g id="text_44">
|
||||
<text style="font-size: 12px; font-family: 'DejaVu Sans'; text-anchor: middle; fill: #58677b" x="989.247519" y="450.637187" transform="rotate(-0 989.247519 450.637187)">0</text>
|
||||
<text style="font-size: 12px; font-family: 'DejaVu Sans', Arial, sans-serif; text-anchor: middle; fill: #58677b" x="1013.978035" y="450.637187" transform="rotate(-0 1013.978035 450.637187)">0</text>
|
||||
</g>
|
||||
</g>
|
||||
<g id="xtick_14">
|
||||
<g id="line2d_43">
|
||||
<path d="M 1083.613961 434.52
|
||||
L 1083.613961 146.88
|
||||
" clip-path="url(#p7ebe458925)" style="fill: none; stroke: #e8edf3; stroke-width: 0.9; stroke-linecap: square"/>
|
||||
</g>
|
||||
<g id="line2d_44"/>
|
||||
<g id="line2d_42"/>
|
||||
<g id="text_45">
|
||||
<text style="font-size: 12px; font-family: 'DejaVu Sans'; text-anchor: middle; fill: #58677b" x="1083.613961" y="450.638125" transform="rotate(-0 1083.613961 450.638125)">2k</text>
|
||||
<text style="font-size: 12px; font-family: 'DejaVu Sans', Arial, sans-serif; text-anchor: middle; fill: #58677b" x="1075.804325" y="450.637187" transform="rotate(-0 1075.804325 450.637187)">1</text>
|
||||
</g>
|
||||
</g>
|
||||
<g id="xtick_15">
|
||||
<g id="line2d_45">
|
||||
<path d="M 1177.980404 434.52
|
||||
L 1177.980404 146.88
|
||||
" clip-path="url(#p7ebe458925)" style="fill: none; stroke: #e8edf3; stroke-width: 0.9; stroke-linecap: square"/>
|
||||
</g>
|
||||
<g id="line2d_46"/>
|
||||
<g id="line2d_43"/>
|
||||
<g id="text_46">
|
||||
<text style="font-size: 12px; font-family: 'DejaVu Sans'; text-anchor: middle; fill: #58677b" x="1177.980404" y="450.638125" transform="rotate(-0 1177.980404 450.638125)">4k</text>
|
||||
<text style="font-size: 12px; font-family: 'DejaVu Sans', Arial, sans-serif; text-anchor: middle; fill: #58677b" x="1137.630614" y="450.637187" transform="rotate(-0 1137.630614 450.637187)">2</text>
|
||||
</g>
|
||||
</g>
|
||||
<g id="xtick_16">
|
||||
<g id="line2d_47">
|
||||
<path d="M 1272.346846 434.52
|
||||
L 1272.346846 146.88
|
||||
" clip-path="url(#p7ebe458925)" style="fill: none; stroke: #e8edf3; stroke-width: 0.9; stroke-linecap: square"/>
|
||||
</g>
|
||||
<g id="line2d_48"/>
|
||||
<g id="line2d_44"/>
|
||||
<g id="text_47">
|
||||
<text style="font-size: 12px; font-family: 'DejaVu Sans'; text-anchor: middle; fill: #58677b" x="1272.346846" y="450.638125" transform="rotate(-0 1272.346846 450.638125)">6k</text>
|
||||
<text style="font-size: 12px; font-family: 'DejaVu Sans', Arial, sans-serif; text-anchor: middle; fill: #58677b" x="1199.456904" y="450.637187" transform="rotate(-0 1199.456904 450.637187)">3</text>
|
||||
</g>
|
||||
</g>
|
||||
<g id="text_48">
|
||||
<text style="font-size: 12px; font-family: 'DejaVu Sans'; text-anchor: middle; fill: #43536b" x="1168.543759" y="475.639063" transform="rotate(-0 1168.543759 475.639063)">Mean provider tokens per generation</text>
|
||||
<g id="xtick_17">
|
||||
<g id="line2d_45"/>
|
||||
<g id="text_48">
|
||||
<text style="font-size: 12px; font-family: 'DejaVu Sans', Arial, sans-serif; text-anchor: middle; fill: #58677b" x="1261.283194" y="450.637187" transform="rotate(-0 1261.283194 450.637187)">4</text>
|
||||
</g>
|
||||
</g>
|
||||
<g id="xtick_18">
|
||||
<g id="line2d_46"/>
|
||||
<g id="text_49">
|
||||
<text style="font-size: 12px; font-family: 'DejaVu Sans', Arial, sans-serif; text-anchor: middle; fill: #58677b" x="1323.109484" y="450.637187" transform="rotate(-0 1323.109484 450.637187)">5</text>
|
||||
</g>
|
||||
</g>
|
||||
<g id="text_50">
|
||||
<text style="font-size: 12px; font-family: 'DejaVu Sans', Arial, sans-serif; text-anchor: middle; fill: #43536b" x="1168.543759" y="475.638125" transform="rotate(-0 1168.543759 475.638125)">Jev Top N (0 = summary only)</text>
|
||||
</g>
|
||||
</g>
|
||||
<g id="matplotlib.axis_6">
|
||||
<g id="ytick_12">
|
||||
<g id="line2d_47">
|
||||
<path d="M 989.247519 434.52
|
||||
L 1347.84 434.52
|
||||
" clip-path="url(#p327e181977)" style="fill: none; stroke: #e8edf3; stroke-width: 0.9; stroke-linecap: square"/>
|
||||
</g>
|
||||
<g id="line2d_48"/>
|
||||
<g id="text_51">
|
||||
<text style="font-size: 12px; font-family: 'DejaVu Sans', Arial, sans-serif; text-anchor: end; fill: #58677b" x="982.247519" y="439.078594" transform="rotate(-0 982.247519 439.078594)">0</text>
|
||||
</g>
|
||||
</g>
|
||||
<g id="ytick_13">
|
||||
<g id="line2d_49">
|
||||
<path d="M 989.247519 382.221818
|
||||
L 1347.84 382.221818
|
||||
" clip-path="url(#p327e181977)" style="fill: none; stroke: #e8edf3; stroke-width: 0.9; stroke-linecap: square"/>
|
||||
</g>
|
||||
<g id="line2d_50"/>
|
||||
<g id="text_52">
|
||||
<text style="font-size: 12px; font-family: 'DejaVu Sans', Arial, sans-serif; text-anchor: end; fill: #58677b" x="982.247519" y="386.780881" transform="rotate(-0 982.247519 386.780881)">1k</text>
|
||||
</g>
|
||||
</g>
|
||||
<g id="ytick_14">
|
||||
<g id="line2d_51">
|
||||
<path d="M 989.247519 329.923636
|
||||
L 1347.84 329.923636
|
||||
" clip-path="url(#p327e181977)" style="fill: none; stroke: #e8edf3; stroke-width: 0.9; stroke-linecap: square"/>
|
||||
</g>
|
||||
<g id="line2d_52"/>
|
||||
<g id="text_53">
|
||||
<text style="font-size: 12px; font-family: 'DejaVu Sans', Arial, sans-serif; text-anchor: end; fill: #58677b" x="982.247519" y="334.482699" transform="rotate(-0 982.247519 334.482699)">2k</text>
|
||||
</g>
|
||||
</g>
|
||||
<g id="ytick_15">
|
||||
<g id="line2d_53">
|
||||
<path d="M 989.247519 277.625455
|
||||
L 1347.84 277.625455
|
||||
" clip-path="url(#p327e181977)" style="fill: none; stroke: #e8edf3; stroke-width: 0.9; stroke-linecap: square"/>
|
||||
</g>
|
||||
<g id="line2d_54"/>
|
||||
<g id="text_54">
|
||||
<text style="font-size: 12px; font-family: 'DejaVu Sans', Arial, sans-serif; text-anchor: end; fill: #58677b" x="982.247519" y="282.184517" transform="rotate(-0 982.247519 282.184517)">3k</text>
|
||||
</g>
|
||||
</g>
|
||||
<g id="ytick_16">
|
||||
<g id="line2d_55">
|
||||
<path d="M 989.247519 225.327273
|
||||
L 1347.84 225.327273
|
||||
" clip-path="url(#p327e181977)" style="fill: none; stroke: #e8edf3; stroke-width: 0.9; stroke-linecap: square"/>
|
||||
</g>
|
||||
<g id="line2d_56"/>
|
||||
<g id="text_55">
|
||||
<text style="font-size: 12px; font-family: 'DejaVu Sans', Arial, sans-serif; text-anchor: end; fill: #58677b" x="982.247519" y="229.886335" transform="rotate(-0 982.247519 229.886335)">4k</text>
|
||||
</g>
|
||||
</g>
|
||||
<g id="ytick_17">
|
||||
<g id="line2d_57">
|
||||
<path d="M 989.247519 173.029091
|
||||
L 1347.84 173.029091
|
||||
" clip-path="url(#p327e181977)" style="fill: none; stroke: #e8edf3; stroke-width: 0.9; stroke-linecap: square"/>
|
||||
</g>
|
||||
<g id="line2d_58"/>
|
||||
<g id="text_56">
|
||||
<text style="font-size: 12px; font-family: 'DejaVu Sans', Arial, sans-serif; text-anchor: end; fill: #58677b" x="982.247519" y="177.588153" transform="rotate(-0 982.247519 177.588153)">5k</text>
|
||||
</g>
|
||||
</g>
|
||||
<g id="text_57">
|
||||
<text style="font-size: 12px; font-family: 'DejaVu Sans', Arial, sans-serif; text-anchor: middle; fill: #43536b" x="954.780956" y="290.7" transform="rotate(-90 954.780956 290.7)">Mean Jev tokens per run</text>
|
||||
</g>
|
||||
</g>
|
||||
<g id="matplotlib.axis_6"/>
|
||||
<g id="patch_7">
|
||||
<path d="M 989.247519 434.52
|
||||
L 1347.84 434.52
|
||||
" style="fill: none; stroke: #dbe2ea; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square"/>
|
||||
</g>
|
||||
<g id="patch_8">
|
||||
<path d="M 989.247519 193.569391
|
||||
L 1243.046066 193.569391
|
||||
L 1243.046066 233.58887
|
||||
L 989.247519 233.58887
|
||||
<path d="M 996.048411 434.52
|
||||
L 1031.907659 434.52
|
||||
L 1031.907659 434.52
|
||||
L 996.048411 434.52
|
||||
z
|
||||
" clip-path="url(#p7ebe458925)" style="fill: #dc8f20"/>
|
||||
" clip-path="url(#p327e181977)" style="fill: #1a9985"/>
|
||||
</g>
|
||||
<g id="patch_9">
|
||||
<path d="M 1243.046066 193.569391
|
||||
L 1252.466982 193.569391
|
||||
L 1252.466982 233.58887
|
||||
L 1243.046066 233.58887
|
||||
<path d="M 1057.874701 434.52
|
||||
L 1093.733949 434.52
|
||||
L 1093.733949 216.868042
|
||||
L 1057.874701 216.868042
|
||||
z
|
||||
" clip-path="url(#p7ebe458925)" style="fill: #dc8f20; opacity: 0.3"/>
|
||||
</g>
|
||||
<g id="text_49">
|
||||
<text style="font-weight: 700; font-size: 11.5px; font-family: 'DejaVu Sans'; text-anchor: start; fill: #172d45" x="989.247519" y="182.469305" transform="rotate(-0 989.247519 182.469305)">S generation</text>
|
||||
</g>
|
||||
<g id="text_50">
|
||||
<text style="font-size: 10.5px; font-family: 'DejaVu Sans'; text-anchor: start; fill: #43536b" x="1256.949388" y="216.306669" transform="rotate(-0 1256.949388 216.306669)">5,379 / 200</text>
|
||||
" clip-path="url(#p327e181977)" style="fill: #1a9985"/>
|
||||
</g>
|
||||
<g id="patch_10">
|
||||
<path d="M 989.247519 276.943304
|
||||
L 1136.470965 276.943304
|
||||
L 1136.470965 316.962783
|
||||
L 989.247519 316.962783
|
||||
<path d="M 1119.70099 434.52
|
||||
L 1155.560239 434.52
|
||||
L 1155.560239 216.868042
|
||||
L 1119.70099 216.868042
|
||||
z
|
||||
" clip-path="url(#p7ebe458925)" style="fill: #596fc4"/>
|
||||
" clip-path="url(#p327e181977)" style="fill: #1a9985"/>
|
||||
</g>
|
||||
<g id="patch_11">
|
||||
<path d="M 1136.470965 276.943304
|
||||
L 1143.650678 276.943304
|
||||
L 1143.650678 316.962783
|
||||
L 1136.470965 316.962783
|
||||
<path d="M 1181.52728 434.52
|
||||
L 1217.386528 434.52
|
||||
L 1217.386528 216.868042
|
||||
L 1181.52728 216.868042
|
||||
z
|
||||
" clip-path="url(#p7ebe458925)" style="fill: #596fc4; opacity: 0.3"/>
|
||||
</g>
|
||||
<g id="text_51">
|
||||
<text style="font-weight: 700; font-size: 11.5px; font-family: 'DejaVu Sans'; text-anchor: start; fill: #172d45" x="989.247519" y="265.843218" transform="rotate(-0 989.247519 265.843218)">H generation</text>
|
||||
</g>
|
||||
<g id="text_52">
|
||||
<text style="font-size: 10.5px; font-family: 'DejaVu Sans'; text-anchor: start; fill: #43536b" x="1148.133084" y="299.680583" transform="rotate(-0 1148.133084 299.680583)">3,120 / 152</text>
|
||||
" clip-path="url(#p327e181977)" style="fill: #1a9985"/>
|
||||
</g>
|
||||
<g id="patch_12">
|
||||
<path d="M 989.247519 360.317217
|
||||
L 1185.61229 360.317217
|
||||
L 1185.61229 400.336696
|
||||
L 989.247519 400.336696
|
||||
<path d="M 1243.35357 434.52
|
||||
L 1279.212818 434.52
|
||||
L 1279.212818 216.868042
|
||||
L 1243.35357 216.868042
|
||||
z
|
||||
" clip-path="url(#p7ebe458925)" style="fill: #1a9985"/>
|
||||
" clip-path="url(#p327e181977)" style="fill: #1a9985"/>
|
||||
</g>
|
||||
<g id="patch_13">
|
||||
<path d="M 1185.61229 360.317217
|
||||
L 1201.855114 360.317217
|
||||
L 1201.855114 400.336696
|
||||
L 1185.61229 400.336696
|
||||
<path d="M 1305.17986 434.52
|
||||
L 1341.039108 434.52
|
||||
L 1341.039108 216.868042
|
||||
L 1305.17986 216.868042
|
||||
z
|
||||
" clip-path="url(#p7ebe458925)" style="fill: #1a9985; opacity: 0.3"/>
|
||||
" clip-path="url(#p327e181977)" style="fill: #1a9985"/>
|
||||
</g>
|
||||
<g id="text_53">
|
||||
<text style="font-weight: 700; font-size: 11.5px; font-family: 'DejaVu Sans'; text-anchor: start; fill: #172d45" x="989.247519" y="349.217131" transform="rotate(-0 989.247519 349.217131)">Jev ranking</text>
|
||||
<g id="patch_14">
|
||||
<path d="M 996.048411 434.52
|
||||
L 1031.907659 434.52
|
||||
L 1031.907659 434.52
|
||||
L 996.048411 434.52
|
||||
z
|
||||
" clip-path="url(#p327e181977)" style="fill: #b3ddd6"/>
|
||||
</g>
|
||||
<g id="text_54">
|
||||
<text style="font-size: 10.5px; font-family: 'DejaVu Sans'; text-anchor: start; fill: #43536b" x="1206.33752" y="383.054496" transform="rotate(-0 1206.33752 383.054496)">4,162 / 344</text>
|
||||
<g id="patch_15">
|
||||
<path d="M 1057.874701 216.868042
|
||||
L 1093.733949 216.868042
|
||||
L 1093.733949 198.864393
|
||||
L 1057.874701 198.864393
|
||||
z
|
||||
" clip-path="url(#p327e181977)" style="fill: #b3ddd6"/>
|
||||
</g>
|
||||
<g id="text_55">
|
||||
<text style="font-weight: 700; font-size: 15px; font-family: 'DejaVu Sans'; text-anchor: start; fill: #172d45" x="989.247519" y="123.88" transform="rotate(-0 989.247519 123.88)">C Compression overhead</text>
|
||||
<g id="patch_16">
|
||||
<path d="M 1119.70099 216.868042
|
||||
L 1155.560239 216.868042
|
||||
L 1155.560239 198.864393
|
||||
L 1119.70099 198.864393
|
||||
z
|
||||
" clip-path="url(#p327e181977)" style="fill: #b3ddd6"/>
|
||||
</g>
|
||||
<g id="patch_17">
|
||||
<path d="M 1181.52728 216.868042
|
||||
L 1217.386528 216.868042
|
||||
L 1217.386528 198.864393
|
||||
L 1181.52728 198.864393
|
||||
z
|
||||
" clip-path="url(#p327e181977)" style="fill: #b3ddd6"/>
|
||||
</g>
|
||||
<g id="patch_18">
|
||||
<path d="M 1243.35357 216.868042
|
||||
L 1279.212818 216.868042
|
||||
L 1279.212818 198.864393
|
||||
L 1243.35357 198.864393
|
||||
z
|
||||
" clip-path="url(#p327e181977)" style="fill: #b3ddd6"/>
|
||||
</g>
|
||||
<g id="patch_19">
|
||||
<path d="M 1305.17986 216.868042
|
||||
L 1341.039108 216.868042
|
||||
L 1341.039108 198.864393
|
||||
L 1305.17986 198.864393
|
||||
z
|
||||
" clip-path="url(#p327e181977)" style="fill: #b3ddd6"/>
|
||||
</g>
|
||||
<g id="text_58">
|
||||
<text style="font-weight: 700; font-size: 10.5px; font-family: 'DejaVu Sans', Arial, sans-serif; text-anchor: middle; fill: #172d45" x="1013.978035" y="425.198775" transform="rotate(-0 1013.978035 425.198775)">0</text>
|
||||
</g>
|
||||
<g id="text_59">
|
||||
<text style="font-weight: 700; font-size: 10.5px; font-family: 'DejaVu Sans', Arial, sans-serif; text-anchor: middle; fill: #172d45" x="1075.804325" y="189.543168" transform="rotate(-0 1075.804325 189.543168)">4,506</text>
|
||||
</g>
|
||||
<g id="text_60">
|
||||
<text style="font-weight: 700; font-size: 10.5px; font-family: 'DejaVu Sans', Arial, sans-serif; text-anchor: middle; fill: #172d45" x="1137.630614" y="189.543168" transform="rotate(-0 1137.630614 189.543168)">4,506</text>
|
||||
</g>
|
||||
<g id="text_61">
|
||||
<text style="font-weight: 700; font-size: 10.5px; font-family: 'DejaVu Sans', Arial, sans-serif; text-anchor: middle; fill: #172d45" x="1199.456904" y="189.543168" transform="rotate(-0 1199.456904 189.543168)">4,506</text>
|
||||
</g>
|
||||
<g id="text_62">
|
||||
<text style="font-weight: 700; font-size: 10.5px; font-family: 'DejaVu Sans', Arial, sans-serif; text-anchor: middle; fill: #172d45" x="1261.283194" y="189.543168" transform="rotate(-0 1261.283194 189.543168)">4,506</text>
|
||||
</g>
|
||||
<g id="text_63">
|
||||
<text style="font-weight: 700; font-size: 10.5px; font-family: 'DejaVu Sans', Arial, sans-serif; text-anchor: middle; fill: #172d45" x="1323.109484" y="189.543168" transform="rotate(-0 1323.109484 189.543168)">4,506</text>
|
||||
</g>
|
||||
<g id="text_64">
|
||||
<text style="font-size: 11px; font-family: 'DejaVu Sans', Arial, sans-serif; text-anchor: start; fill: #58677b" x="996.419368" y="163.867481" transform="rotate(-0 996.419368 163.867481)">Same ranking cost for S and H</text>
|
||||
</g>
|
||||
<g id="text_65">
|
||||
<text style="font-weight: 700; font-size: 15px; font-family: 'DejaVu Sans', Arial, sans-serif; text-anchor: start; fill: #172d45" x="989.247519" y="123.88" transform="rotate(-0 989.247519 123.88)">C Jev ranking overhead</text>
|
||||
</g>
|
||||
<g id="PathCollection_1">
|
||||
<defs>
|
||||
<path id="mce221bbee0" d="M 0 2.645751
|
||||
C 0.701661 2.645751 1.374679 2.366978 1.870829 1.870829
|
||||
C 2.366978 1.374679 2.645751 0.701661 2.645751 0
|
||||
C 2.645751 -0.701661 2.366978 -1.374679 1.870829 -1.870829
|
||||
C 1.374679 -2.366978 0.701661 -2.645751 0 -2.645751
|
||||
C -0.701661 -2.645751 -1.374679 -2.366978 -1.870829 -1.870829
|
||||
C -2.366978 -1.374679 -2.645751 -0.701661 -2.645751 0
|
||||
C -2.645751 0.701661 -2.366978 1.374679 -1.870829 1.870829
|
||||
C -1.374679 2.366978 -0.701661 2.645751 0 2.645751
|
||||
z
|
||||
" style="stroke: #1a9985"/>
|
||||
</defs>
|
||||
<g>
|
||||
<use xlink:href="#mce221bbee0" x="1013.978035" y="434.52" style="fill: #1a9985; stroke: #1a9985"/>
|
||||
</g>
|
||||
</g>
|
||||
<g id="legend_1">
|
||||
<g id="patch_14">
|
||||
<path d="M 1202.210312 492.151256
|
||||
L 1222.210312 492.151256
|
||||
L 1222.210312 485.151256
|
||||
L 1202.210312 485.151256
|
||||
<g id="patch_20">
|
||||
<path d="M 1065.389853 500.780456
|
||||
L 1085.389853 500.780456
|
||||
L 1085.389853 493.780456
|
||||
L 1065.389853 493.780456
|
||||
z
|
||||
" style="fill: #65758b"/>
|
||||
" style="fill: #1a9985"/>
|
||||
</g>
|
||||
<g id="text_56">
|
||||
<text style="font-size: 10px; font-family: 'DejaVu Sans'; text-anchor: start; fill: #172d45" x="1230.210312" y="492.151256" transform="rotate(-0 1230.210312 492.151256)">Input</text>
|
||||
<g id="text_66">
|
||||
<text style="font-size: 10px; font-family: 'DejaVu Sans', Arial, sans-serif; text-anchor: start; fill: #172d45" x="1093.389853" y="500.780456" transform="rotate(-0 1093.389853 500.780456)">Input (4,162)</text>
|
||||
</g>
|
||||
<g id="patch_15">
|
||||
<path d="M 1276.104062 492.151256
|
||||
L 1296.104062 492.151256
|
||||
L 1296.104062 485.151256
|
||||
L 1276.104062 485.151256
|
||||
<g id="patch_21">
|
||||
<path d="M 1178.892978 500.780456
|
||||
L 1198.892978 500.780456
|
||||
L 1198.892978 493.780456
|
||||
L 1178.892978 493.780456
|
||||
z
|
||||
" style="fill: #65758b; opacity: 0.3"/>
|
||||
" style="fill: #b3ddd6"/>
|
||||
</g>
|
||||
<g id="text_57">
|
||||
<text style="font-size: 10px; font-family: 'DejaVu Sans'; text-anchor: start; fill: #172d45" x="1304.104062" y="492.151256" transform="rotate(-0 1304.104062 492.151256)">Output</text>
|
||||
<g id="text_67">
|
||||
<text style="font-size: 10px; font-family: 'DejaVu Sans', Arial, sans-serif; text-anchor: start; fill: #172d45" x="1206.892978" y="500.780456" transform="rotate(-0 1206.892978 500.780456)">Output (344)</text>
|
||||
</g>
|
||||
</g>
|
||||
</g>
|
||||
<g id="text_58">
|
||||
<text style="font-weight: 700; font-size: 25px; font-family: 'DejaVu Sans'; text-anchor: start; fill: #172d45" x="76.032" y="36.72" transform="rotate(-0 76.032 36.72)">Jev across two summary strategies</text>
|
||||
<g id="text_68">
|
||||
<text style="font-weight: 700; font-size: 25px; font-family: 'DejaVu Sans', Arial, sans-serif; text-anchor: start; fill: #172d45" x="76.032" y="36.72" transform="rotate(-0 76.032 36.72)">Jev across two summary strategies</text>
|
||||
</g>
|
||||
<g id="text_59">
|
||||
<text style="font-size: 14px; font-family: 'DejaVu Sans'; text-anchor: start; fill: #58677b" x="76.032" y="70.38" transform="rotate(-0 76.032 70.38)">Same source trajectories. Frozen summaries. Identical Jev selections.</text>
|
||||
<g id="text_69">
|
||||
<text style="font-size: 14px; font-family: 'DejaVu Sans', Arial, sans-serif; text-anchor: start; fill: #58677b" x="76.032" y="70.38" transform="rotate(-0 76.032 70.38)">Same source trajectories. Frozen summaries. Identical Jev selections.</text>
|
||||
</g>
|
||||
<g id="text_60">
|
||||
<text style="font-weight: 700; font-size: 12px; font-family: 'DejaVu Sans'; text-anchor: start; fill: #172d45" x="76.032" y="514.08" transform="rotate(-0 76.032 514.08)">At Top 3</text>
|
||||
<g id="text_70">
|
||||
<text style="font-weight: 700; font-size: 12px; font-family: 'DejaVu Sans', Arial, sans-serif; text-anchor: start; fill: #172d45" x="76.032" y="514.08" transform="rotate(-0 76.032 514.08)">At Top 3</text>
|
||||
</g>
|
||||
<g id="text_61">
|
||||
<text style="font-size: 13px; font-family: 'DejaVu Sans'; text-anchor: start; fill: #172d45" x="186.624" y="514.08" transform="rotate(-0 186.624 514.08)">S: 99.5% | 3,241 input tokens H: 97.9% | 3,193 input tokens</text>
|
||||
<g id="text_71">
|
||||
<text style="font-size: 13px; font-family: 'DejaVu Sans', Arial, sans-serif; text-anchor: start; fill: #172d45" x="186.624" y="514.08" transform="rotate(-0 186.624 514.08)">S: 99.5% | 3,241 input tokens H: 97.9% | 3,193 input tokens</text>
|
||||
</g>
|
||||
<g id="text_62">
|
||||
<text style="font-size: 11px; font-family: 'DejaVu Sans'; text-anchor: start; fill: #58677b" x="76.032" y="547.74" transform="rotate(-0 76.032 547.74)">Primary cohort: 4 task templates, 24 answers per method. Another 10 short/query-only controls are reported separately.</text>
|
||||
<g id="text_72">
|
||||
<text style="font-size: 11px; font-family: 'DejaVu Sans', Arial, sans-serif; text-anchor: start; fill: #58677b" x="76.032" y="547.74" transform="rotate(-0 76.032 547.74)">Primary cohort: 4 task templates, 24 answers per method; 10 separate controls. Top 3–5 shorten 9/12 primary contexts per method.</text>
|
||||
</g>
|
||||
<g id="text_63">
|
||||
<text style="font-size: 11px; font-family: 'DejaVu Sans'; text-anchor: start; fill: #58677b" x="76.032" y="568.548" transform="rotate(-0 76.032 568.548)">Overhead includes recorded retries; one summary + one Jev ranking per standalone fusion run. Jev cost is unchanged for Top 1–5.</text>
|
||||
<g id="text_73">
|
||||
<text style="font-size: 11px; font-family: 'DejaVu Sans', Arial, sans-serif; text-anchor: start; fill: #58677b" x="76.032" y="568.548" transform="rotate(-0 76.032 568.548)">Jev overhead is zero at Top 0 and identical at Top 1–5 for S and H. Each fused method is charged one complete ranking, including retries.</text>
|
||||
</g>
|
||||
<g id="text_64">
|
||||
<text style="font-size: 11px; font-family: 'DejaVu Sans'; text-anchor: start; fill: #58677b" x="76.032" y="589.356" transform="rotate(-0 76.032 589.356)">Tokens are not currency. Top 3–5 shorten 9/12 primary contexts per method; all cases remain scored. Direct Qwen evaluation; no agent continuation.</text>
|
||||
<g id="text_74">
|
||||
<text style="font-size: 11px; font-family: 'DejaVu Sans', Arial, sans-serif; text-anchor: start; fill: #58677b" x="76.032" y="589.356" transform="rotate(-0 76.032 589.356)">Tokens are not currency; summary-generation overhead is listed in the results table. Direct Qwen evaluation; no agent continuation.</text>
|
||||
</g>
|
||||
<g id="legend_2">
|
||||
<g id="line2d_49">
|
||||
<g id="line2d_59">
|
||||
<path d="M 78.5376 97.114125
|
||||
L 90.5376 97.114125
|
||||
L 102.5376 97.114125
|
||||
" style="fill: none; stroke: #dc8f20; stroke-width: 2.5; stroke-linecap: square"/>
|
||||
<g>
|
||||
<use xlink:href="#m90dc9a1c0b" x="90.5376" y="97.114125" style="fill: #dc8f20; stroke: #dc8f20"/>
|
||||
<use xlink:href="#mde656dc075" x="90.5376" y="97.114125" style="fill: #dc8f20; stroke: #dc8f20"/>
|
||||
</g>
|
||||
</g>
|
||||
<g id="text_65">
|
||||
<text style="font-size: 12px; font-family: 'DejaVu Sans'; text-anchor: start; fill: #172d45" x="112.1376" y="101.314125" transform="rotate(-0 112.1376 101.314125)">Generic summary S</text>
|
||||
<g id="text_75">
|
||||
<text style="font-size: 12px; font-family: 'DejaVu Sans', Arial, sans-serif; text-anchor: start; fill: #172d45" x="112.1376" y="101.314125" transform="rotate(-0 112.1376 101.314125)">Generic summary S</text>
|
||||
</g>
|
||||
<g id="line2d_50">
|
||||
<g id="line2d_60">
|
||||
<path d="M 254.541975 97.114125
|
||||
L 266.541975 97.114125
|
||||
L 278.541975 97.114125
|
||||
" style="fill: none; stroke: #596fc4; stroke-width: 2.5; stroke-linecap: square"/>
|
||||
<g>
|
||||
<use xlink:href="#m712a07e464" x="266.541975" y="97.114125" style="fill: #596fc4; stroke: #596fc4"/>
|
||||
<use xlink:href="#m05292c0fc7" x="266.541975" y="97.114125" style="fill: #596fc4; stroke: #596fc4"/>
|
||||
</g>
|
||||
</g>
|
||||
<g id="text_66">
|
||||
<text style="font-size: 12px; font-family: 'DejaVu Sans'; text-anchor: start; fill: #172d45" x="288.141975" y="101.314125" transform="rotate(-0 288.141975 101.314125)">Hermes small-budget H</text>
|
||||
<g id="text_76">
|
||||
<text style="font-size: 12px; font-family: 'DejaVu Sans', Arial, sans-serif; text-anchor: start; fill: #172d45" x="288.141975" y="101.314125" transform="rotate(-0 288.141975 101.314125)">Hermes small-budget H</text>
|
||||
</g>
|
||||
<g id="line2d_51">
|
||||
<path d="M 457.06635 97.114125
|
||||
L 469.06635 97.114125
|
||||
L 481.06635 97.114125
|
||||
<g id="line2d_61">
|
||||
<path d="M 454.086975 97.114125
|
||||
L 466.086975 97.114125
|
||||
L 478.086975 97.114125
|
||||
" style="fill: none; stroke-dasharray: 5.55,2.4; stroke-dashoffset: 0; stroke: #7b8798; stroke-width: 1.5"/>
|
||||
</g>
|
||||
<g id="text_67">
|
||||
<text style="font-size: 12px; font-family: 'DejaVu Sans'; text-anchor: start; fill: #172d45" x="490.66635" y="101.314125" transform="rotate(-0 490.66635 101.314125)">Full context</text>
|
||||
<g id="text_77">
|
||||
<text style="font-size: 12px; font-family: 'DejaVu Sans', Arial, sans-serif; text-anchor: start; fill: #172d45" x="487.686975" y="101.314125" transform="rotate(-0 487.686975 101.314125)">Full context</text>
|
||||
</g>
|
||||
</g>
|
||||
</g>
|
||||
<defs>
|
||||
<clipPath id="p260a3f0845">
|
||||
<clipPath id="pf6d0324672">
|
||||
<rect x="76.032" y="146.88" width="344.800463" height="287.64"/>
|
||||
</clipPath>
|
||||
<clipPath id="pde7327f20e">
|
||||
<clipPath id="p6f3dc61b1a">
|
||||
<rect x="532.639759" y="146.88" width="344.800463" height="287.64"/>
|
||||
</clipPath>
|
||||
<clipPath id="p7ebe458925">
|
||||
<clipPath id="p327e181977">
|
||||
<rect x="989.247519" y="146.88" width="358.592481" height="287.64"/>
|
||||
</clipPath>
|
||||
</defs>
|
||||
|
|
|
|||
|
Before Width: | Height: | Size: 30 KiB After Width: | Height: | Size: 36 KiB |
|
|
@ -117,7 +117,8 @@ Top 0 表示纯摘要；每轮摘要逐字固定，两种机制使用完全相
|
|||
|
||||
4 个主任务中，Top 3 将 S 从 **82.0% 提升至 99.5%**，将 H 从 **79.4% 提升至 97.9%**。
|
||||
平均回答输入分别为 **3241 / 3193 tokens**，Full 为 **4014 tokens / 100%**。
|
||||
图中分别展示效果、回答输入和摘要／排序开销；回答输入减少不等于总费用降低。
|
||||
图中分别展示效果、回答输入和 Jev 排序开销；摘要生成用量见结果表。Top 0 不调用 Jev，Top 1～5 共用完整排名，
|
||||
因此 Jev 开销相同。回答输入减少不等于总费用降低。
|
||||
|
||||
共 14 个预设 query/answer 模板、546 个上下文、1092 次直接 Qwen 回答；10 个短任务对照单独报告。
|
||||
H 复用 Hermes 的上下文压缩摘要生成流程，并采用小预算与长度约束；实验直接生成摘要，不经过自动触发和首尾窗口选择。
|
||||
|
|
@ -125,5 +126,5 @@ H 复用 Hermes 的上下文压缩摘要生成流程，并采用小预算与长
|
|||
Top 3～5 每组主任务只有 9/12 个上下文实现缩短，全部样例均保留评分。这是非等长的增量比较，未继续运行 agent。
|
||||
|
||||
<!-- 支付轨迹有 10 个证据块，最多 Top 10；实际 N 为 min(N, 可用证据块数)。 -->
|
||||
模型、框架版本、参数和复现步骤见[实验设置](https://github.com/jinliyl/ReMe/blob/feat/decision-model-compaction/benchmark/compaction/SETTINGS.md)。
|
||||
详见[query、answer 与实验设定](https://github.com/jinliyl/ReMe/blob/d2638b166f6daeec6f3e842750313526617786a1/benchmark/compaction/STUDY.md)及[完整结果与 token 开销](https://github.com/jinliyl/ReMe/blob/d2638b166f6daeec6f3e842750313526617786a1/benchmark/compaction/RESULTS.md)。
|
||||
模型、框架版本、参数和复现步骤见[实验设置](https://github.com/jinliyl/ReMe/blob/65ef7358417d21746cf9f95aaf9005aa521c5688/benchmark/compaction/SETTINGS.md)。
|
||||
详见[query、answer 与实验设定](https://github.com/jinliyl/ReMe/blob/65ef7358417d21746cf9f95aaf9005aa521c5688/benchmark/compaction/STUDY.md)及[完整结果与 token 开销](https://github.com/jinliyl/ReMe/blob/65ef7358417d21746cf9f95aaf9005aa521c5688/benchmark/compaction/RESULTS.md)。
|
||||
|
|
|
|||
Loading…
Add table
Reference in a new issue