Commit graph

6674 commits

Author SHA1 Message Date
Maxton Huff
fb47cb875a remove error return type 2021-02-05 13:40:57 -06:00
Maxton Huff
fec0337630 add draft trial version of molecula 2021-02-05 12:16:08 -06:00
Maxton Huff
927db378b4 add linux OS check and the way mmap limit is read 2021-01-27 09:52:13 -06:00
Maxton Huff
8c36da9eeb Merge branch 'mmap' of github.com:Maxtonian/pilosa into mmap 2021-01-26 11:27:12 -06:00
Maxton Huff
3982a8e970 format messages and change mmap comparison logic 2021-01-26 11:26:36 -06:00
Maxton Huff
bd680ef144
Merge branch 'master' into mmap 2021-01-26 10:21:18 -06:00
seebs
89f1dfb239
Merge pull request #1367 from seebs/countRange
RBF countRange: Handle partial counts on bitmapPtr containers
2021-01-26 09:51:09 -06:00
Maxton Huff
3809fe6734 add error check 2021-01-26 09:35:51 -06:00
Maxton Huff
55a9952b92 mmap limit comparison error message 2021-01-25 16:19:12 -06:00
Seebs
a238afb21a Handle BitmapPtr cells in countRange
We need to be able to count bits in BitmapPtr containers. This only
comes up if you have a non-container-aligned range count, which we
never do in real production yet, but the API allows it so it should
work. In order to do this, we need to provide the tx to countRange
so it can grab pages as needed. Arguably, we should be able to avoid
actually creating/copying that page since we're only using it
internally, never returning it, but this is a pretty rare case
and probably not performance-critical.
2021-01-25 15:41:35 -06:00
Seebs
d1a9c91a96 Test CountRange on non-container ranges.
This test was supposed to check against all the container types,
but especially bitmaps, but turns out not to work because the
containers turn into RLE containers. Oops. Now, we start with
every-other-bit for the first 8k, then start filling in the holes,
so we get some bitmap containers and then start generating
RLE containers.
2021-01-25 15:41:35 -06:00
Nia
13c63baa14
Merge pull request #1365 from niaow/groupby-cluster-rows
CORE-28 Fix GroupBy with a global Rows filter
2021-01-25 15:37:48 -05:00
Nia
60ab9e84dd
Merge branch 'master' into groupby-cluster-rows 2021-01-25 15:26:13 -05:00
Nia
ed9fc962d5
Merge pull request #1359 from niaow/intersectany-single-word-run
CORE-31 Fix intersectionAnyRunBitmap when processing single-word runs
2021-01-25 11:16:30 -05:00
Nia
70d87a7950
Merge branch 'master' into intersectany-single-word-run 2021-01-25 11:03:51 -05:00
Nia Weiss
dde318ac8c
Move globally computed GroupBy rows calls into EmbeddedData
This fixes a bug where a globally computed Rows call would be computed with a subset of the shards.
2021-01-25 10:17:58 -05:00
Ben Johnson
66bc79634b
Merge pull request #1363 from molecula/pilosa-bench-vars
Add ingest time & latency stats to query benchmarks
2021-01-25 08:16:34 -07:00
Cody Soyland
649202b081
Fix cluster size setter: expose failures 2021-01-25 10:16:19 -05:00
Ben Johnson
a30fd9957e
Merge branch 'master' into pilosa-bench-vars 2021-01-25 07:56:20 -07:00
Cody Soyland
3da3d23b21
Merge pull request #1364 from codysoyland/lattice-submodule
Add git-submodule to manage UI version
2021-01-22 16:57:57 -06:00
Alan Bernstein
852057533b Add lattice submodule upgrade instructions 2021-01-22 16:51:17 -06:00
Cody Soyland
671c7262c9
Add lattice to PHONY 2021-01-22 16:26:09 -06:00
Cody Soyland
34f728521c
Add git-submodule to manage lattice version 2021-01-22 15:05:19 -06:00
Ben Johnson
acdff02fed Add ingest time & latency stats to query benchmarks 2021-01-22 11:41:44 -07:00
nagamocha3000
6c3c460c83
Merge pull request #1344 from nagamocha3000/bnm-fix-1080
Fix filtering on time fields in Rows() embedded within GroupBy
2021-01-22 19:56:08 +03:00
nagamocha3000
b5fdc6609a Update tests to remove sum column on GroupBy 2021-01-22 18:44:36 +03:00
Nia Weiss
0d9179f10c
roaring: fix intersectionAnyRunBitmap when processing single-word runs
When a run started and ended within a single word, the entirety of the word would be checked.
This would cause small runs to be processed incorrectly, and caused Distinct-on-sets to select rows that did not match the specified filter.
2021-01-21 18:09:27 -05:00
nagamocha3000
16b5fbe40b
Merge branch 'master' into bnm-fix-1080 2021-01-21 17:37:08 +03:00
nagamocha3000
6cbcde3a82 Add more tests for GroupBy on Time fields 2021-01-21 16:20:46 +03:00
nagamocha3000
113bec2474 Remove unnecessary nil check 2021-01-21 15:48:52 +03:00
Alan Bernstein
b273f3ba60
Merge pull request #1327 from alanbernstein/sql-history
Include SQL string in query-history
2021-01-20 19:05:36 -06:00
Alan Bernstein
eaf3ef153e Test SQL behavior 2021-01-20 15:58:55 -06:00
Alan Bernstein
bed03ba490 Update tests 2021-01-20 10:48:35 -06:00
Alan Bernstein
a0d6c253d1 Pass SQL query string from mapper to tracker 2021-01-20 10:48:35 -06:00
Cody Soyland
06241d1f84
Merge pull request #1258 from codysoyland/distinct-failure
Test to demonstrate failure running Distinct
2021-01-20 10:37:47 -06:00
Cody Soyland
0a325dcdb8
Merge branch 'master' into distinct-failure 2021-01-19 16:50:25 -06:00
seebs
38ded7963e
Merge pull request #1342 from seebs/aggregate
Handle aggregate functions better in sql/grpc/json
2021-01-19 16:28:32 -06:00
Seebs
932e84b681 handling aggregate types: add to protobuf, etc
We want to distinguish different *kinds* of GroupCounts, so we're
making the GroupCounts parent object track its type so we can keep that
correct.

Adding this to protobuf, etc, then creates some weird behaviors
because sometimes we expect []GroupCount, and sometimes we expect
*GroupCounts. This implies changes to test cases. Also, the
changes to test cases imply that some test cases are probably now
wrong; for instance, they're expecting a "sum" column, equal to zero,
when no sum was requested.

We try to make the encoder handle a []*GroupCount gotten from another
node without panicing, and avoid breaking the semantics of the existing
messages, renumbering messages or components, etc.

Since a previous version, the `.Groups` member has been privatized,
and the `.Get()` convenience accessor has been renamed `.Groups()`
and is now used consistently in a way that should reduce the risk
of nil pointers causing crashes. Also, NewGroupCounts is used in
a couple more places.
2021-01-19 16:23:15 -06:00
Cody Soyland
4ebf6f6ff7 Customize serialization of []GroupCount based on aggregate type/presence 2021-01-19 12:10:56 -06:00
nagamocha3000
0437f5d28a Handle case where row returned might be nil 2021-01-19 16:51:26 +03:00
Ben Johnson
56c001d288
Merge pull request #1341 from molecula/bench-vars
Update benchmarks to use expvar URL list
2021-01-18 16:06:43 -07:00
nagamocha3000
d26c6b048e Sort all rowIDs gathered before storing them 2021-01-19 02:00:50 +03:00
nagamocha3000
9b23dcdd0a Gather rows for each fragment in a much smarter way 2021-01-19 01:44:54 +03:00
Ben Johnson
01e6781abd Update benchmarks to use expvar URL list 2021-01-18 13:43:53 -07:00
nagamocha3000
f51ff4dc85 Add tests for Rows() on time fields 2021-01-18 20:54:31 +03:00
nagamocha3000
b77f9e8a43 Add timeFragments rowIterator 2021-01-15 23:36:54 +03:00
nagamocha3000
f91b4bc016 Add queries to test GroupBy Rows on time field 2021-01-15 22:58:18 +03:00
nagamocha3000
54d79d1e4d Populate 'places_visited' field for 'users' index 2021-01-15 22:50:54 +03:00
nagamocha3000
4dc3e49a0d Add test helper for inserting to time quantum fields 2021-01-15 22:42:37 +03:00
Ben Johnson
fbf546f131
Merge pull request #1319 from molecula/pilosa-bench 2021-01-12 16:51:27 -07:00