Matt Jaffee
c172ca0680
combine fragment.rows and rowsForColumn with generalized filter
...
use filter funcs with closures for state instead of methods on structs. seems a
bit cleaner.
2018-10-08 16:16:39 -05:00
Matt Jaffee
8cd82af2e7
remove extraneous fragment.rows* methods
...
variadic filters makes separate methods unnecessary
2018-10-02 09:31:13 -05:00
Matt Jaffee
4f3f2e1a49
remove noFilter and filterWithOffsetLimit
...
can use an empty list of filters and a list of offsetFilter followed by limit
filter respectively
2018-10-02 09:25:07 -05:00
Matt Jaffee
f94cd8ae7d
add translation code for GroupBy "previous" arg
2018-09-28 14:30:18 -05:00
Matt Jaffee
61089981a2
remove check for column in shard in executeRowsShard
...
the check happens in executeRows and frag.rowsForColumn will panic if given a
column id not in its shard.
2018-09-28 10:20:48 -05:00
Matt Jaffee
ed1b09a1cd
fix columnID<>shard checks in executor and fragment
...
fragment panics if rowsForColumn is called with a column id not in the
fragment's shard. The justification for this is that we're wasting resources if
we're sending requests for a specific column to any shard other than the one
which contains that column.
2018-09-28 10:17:53 -05:00
Matt Jaffee
93e9f242fe
test Rows call with row keys, fix column id problem
2018-09-27 17:07:15 -05:00
Matt Jaffee
7d24276a98
rename "columns" to "rows" in Rows test so it makes sense
2018-09-27 16:18:37 -05:00
Matt Jaffee
78ff75690d
convert Rows to use previous/limit
...
pass previous+1 directly to fragment.rows so that the iterator can seek directly
to the start point. handle limit inside reduce so it can skip out early and
avoid extra allocation.
2018-09-27 16:14:40 -05:00
Matt Jaffee
ac98fcb6d4
rename RowIDs PQL to Rows
2018-09-26 16:09:05 -05:00
Matt Jaffee
4face45c2a
rename RowIDs methods to Rows
2018-09-26 16:03:26 -05:00
Matthew Jaffee
86d238edb0
Merge pull request #4 from travisturner/new-rows-iterate-cleanup
...
New rows iterate cleanup
2018-09-18 14:19:05 -05:00
Travis Turner
0ab3e72520
refactor mergeGroupCounts function
2018-09-18 14:09:52 -05:00
Travis Turner
71ad297450
change Rows() to RowIDs() and add RowIdentifiers return type to hold row keys
2018-09-18 12:45:09 -05:00
Travis Turner
f0666b2be0
change GroupByCounts to []GroupCount
2018-09-18 12:39:09 -05:00
Travis Turner
b6d386ba53
fix tests
2018-09-18 12:37:35 -05:00
Travis Turner
96b4086360
plug in in translation. adjust output format.
2018-09-18 12:37:26 -05:00
Travis Turner
de071d548a
remove additional decodeFieldRow (and hopefully allocation)
2018-09-18 12:37:10 -05:00
Travis Turner
e1b938e52c
add FieldRow struct to replace the groupBy string key
2018-09-18 12:36:46 -05:00
Matt Jaffee
54ce537327
copy non roaring-import code from Todds's row-iterate PR
...
tests passing
2018-09-17 16:01:51 -05:00
Yuce Tekol
74780528d7
Merge pull request #1646 from yuce/roaring-fast-add
...
Adds DirectAdd function to roaring.Bitmap
2018-09-17 22:04:30 +03:00
Yuce Tekol
e664a0e42f
rename add -> DirectAdd
2018-09-17 20:25:14 +03:00
Yuce Tekol
37fdd73a7f
DirectAdd adds a single value
2018-09-17 17:16:03 +03:00
Yuce Tekol
09c24cd3be
Changed the signature of Bitmap.DirectAdd function
2018-09-17 17:10:02 +03:00
Yuce Tekol
266051dd26
Adds DirectAdd function to roaring.Bitmap
2018-09-17 16:25:34 +03:00
Travis Turner
55bb12a434
Merge pull request #1643 from travisturner/generate-update
...
re-generate the apimethod stringer
2018-09-14 10:32:16 -05:00
Travis Turner
2223b23b95
re-generate the apimethod stringer
2018-09-14 09:40:48 -05:00
Travis Turner
734c31d963
Merge pull request #1622 from travisturner/roaring-import
...
Roaring import
2018-09-13 22:43:20 -05:00
Matt Jaffee
495af54c10
Merge branch 'master' into roaring-import
2018-09-13 15:50:16 -05:00
Matt Jaffee
870c5e131f
replace waitgroup with errgroup to avoid race on err
2018-09-13 15:27:00 -05:00
Matt Jaffee
f943c17e0b
ImportRoaring, add remote arg, fix data copy bug
...
also, handle err properly in client method instead of discarding.
2018-09-13 15:05:05 -05:00
Matt Jaffee
b412309447
rename standard roaring to "official" throughout
2018-09-13 13:38:18 -05:00
Matt Jaffee
b2ec5e373e
fixup explanatory comment
2018-09-13 13:33:51 -05:00
Matt Jaffee
7b0d4d75b4
remove validators on import-roaring and test handler
...
validators are for query args, not url vars. Also some misc cleanup and error
handling in the handler.
2018-09-13 13:14:35 -05:00
Matt Jaffee
fd655c998e
rename all instances of roaringbytes
...
function names now importRoaring and byte slice args are just called data. Also
renamed http endpoint from /importroaring to /import-roaring
2018-09-13 12:27:23 -05:00
Matt Jaffee
9e7cfeedc9
readability changes in fragment.go
2018-09-13 11:41:46 -05:00
Matt Jaffee
af7d405b11
add explanatory comment and move comment to correct spot
2018-09-13 10:41:36 -05:00
Matt Jaffee
f006df7758
re-order ImportRoaringBytes args to be more consistent
2018-09-13 10:39:00 -05:00
Matt Jaffee
c6817340c7
rename sliceWidth->shardWidth
2018-09-13 10:31:58 -05:00
Matt Jaffee
79c783eaf8
add nolint directive
...
there isn't much piont in making snapshot take a WriterTo, because the only
thing that's going to be written to a fragment is a roaring bitmap. making it
WriterTo just makes it slightly more annoying to jump into the Bitmap.WriterTo
implementation. since it's a private function, it should be straightforward to
change this in the future if the need arises.
2018-09-12 17:31:57 -05:00
Matt Jaffee
4a5f12631c
more naked returns
2018-09-12 17:29:35 -05:00
Matt Jaffee
d0d0bdd722
simplify fragment.unprotectedRow (unparam)
2018-09-12 17:25:12 -05:00
Matt Jaffee
3bc9c66f5e
remove some naked returns and simplify
2018-09-12 17:21:17 -05:00
Matt Jaffee
70e9da3b75
fmt -s fragment_internal_test
2018-09-12 17:13:07 -05:00
Matt Jaffee
f8569102f5
update some fragment comments re: the cache
2018-09-12 17:10:28 -05:00
Matt Jaffee
3ea07ae3a7
use cache.Recalculate instead of Invalidate for imports
...
Invalidate does not always rebuild the cache - if the last rebuild is < 10s ago,
it does nothing. We always want to rebuild the cache after imports.
Also updated the comments around recalculate/invalidate to clarify.
2018-09-12 17:07:46 -05:00
Matt Jaffee
14a8362261
roaring import - add cache invalidate and broken test
2018-09-12 16:48:07 -05:00
Matt Jaffee
b5a14de1fd
roaring import: fix api doc, refactor/rename, test merge
2018-09-12 16:11:51 -05:00
Matt Jaffee
a3243f99e1
linter fixes - reorder return vals, remove unused const
2018-09-12 13:24:59 -05:00
Todd Gruben
156260e68e
roaring allowed import to merge with existing data
2018-09-12 13:20:07 -05:00
