featurebase/ingest
Seebs e167f1c7fa fancier shard-sorting
This is a rework of Nia's radix sort. Still using stdlib sort for the
tail ends of things, and should probably replace it at some point
because it's still woefully inefficient, but this gets decent
performance, and lets us do the fancy thing of doing quick partial
sorting by record-key-only to get to shards, then deciding whether
to sort by value-then-record (as for a set field) or just by record
(as for int fields), which lets us reduce the amount of re-sorting
the same data by different criteria we do.

We also use a messy code-duplication basically-bubblesort for the
inner loops because it's much cheaper for small N.

This also lets us use field-aware sorting for shards, sorting them
correctly for a corresponding field type, and add corresponding API
support and fragment support for an option to tell the fragment
code that we already ordered things in the order that's most
efficient there, to avoid a second sort that we don't otherwise
need.
2021-08-19 09:50:59 -05:00
..
codec.go fancier shard-sorting 2021-08-19 09:50:59 -05:00
codec_test.go Prototype ingest API 2021-08-19 09:50:59 -05:00
doc.go Prototype ingest API 2021-08-19 09:50:59 -05:00
op.go fancier shard-sorting 2021-08-19 09:50:59 -05:00
op_test.go fancier shard-sorting 2021-08-19 09:50:59 -05:00
shard.go Prototype ingest API 2021-08-19 09:50:59 -05:00
sort.go Prototype ingest API 2021-08-19 09:50:59 -05:00
sort_test.go Prototype ingest API 2021-08-19 09:50:59 -05:00
translate.go Prototype ingest API 2021-08-19 09:50:59 -05:00
update.go Prototype ingest API 2021-08-19 09:50:59 -05:00
vec.go Prototype ingest API 2021-08-19 09:50:59 -05:00