Commit graph

6650 commits

Author SHA1 Message Date
Cody Soyland
34f728521c
Add git-submodule to manage lattice version 2021-01-22 15:05:19 -06:00
nagamocha3000
6c3c460c83
Merge pull request #1344 from nagamocha3000/bnm-fix-1080
Fix filtering on time fields in Rows() embedded within GroupBy
2021-01-22 19:56:08 +03:00
nagamocha3000
b5fdc6609a Update tests to remove sum column on GroupBy 2021-01-22 18:44:36 +03:00
nagamocha3000
16b5fbe40b
Merge branch 'master' into bnm-fix-1080 2021-01-21 17:37:08 +03:00
nagamocha3000
6cbcde3a82 Add more tests for GroupBy on Time fields 2021-01-21 16:20:46 +03:00
nagamocha3000
113bec2474 Remove unnecessary nil check 2021-01-21 15:48:52 +03:00
Alan Bernstein
b273f3ba60
Merge pull request #1327 from alanbernstein/sql-history
Include SQL string in query-history
2021-01-20 19:05:36 -06:00
Alan Bernstein
eaf3ef153e Test SQL behavior 2021-01-20 15:58:55 -06:00
Alan Bernstein
bed03ba490 Update tests 2021-01-20 10:48:35 -06:00
Alan Bernstein
a0d6c253d1 Pass SQL query string from mapper to tracker 2021-01-20 10:48:35 -06:00
Cody Soyland
06241d1f84
Merge pull request #1258 from codysoyland/distinct-failure
Test to demonstrate failure running Distinct
2021-01-20 10:37:47 -06:00
Cody Soyland
0a325dcdb8
Merge branch 'master' into distinct-failure 2021-01-19 16:50:25 -06:00
seebs
38ded7963e
Merge pull request #1342 from seebs/aggregate
Handle aggregate functions better in sql/grpc/json
2021-01-19 16:28:32 -06:00
Seebs
932e84b681 handling aggregate types: add to protobuf, etc
We want to distinguish different *kinds* of GroupCounts, so we're
making the GroupCounts parent object track its type so we can keep that
correct.

Adding this to protobuf, etc, then creates some weird behaviors
because sometimes we expect []GroupCount, and sometimes we expect
*GroupCounts. This implies changes to test cases. Also, the
changes to test cases imply that some test cases are probably now
wrong; for instance, they're expecting a "sum" column, equal to zero,
when no sum was requested.

We try to make the encoder handle a []*GroupCount gotten from another
node without panicing, and avoid breaking the semantics of the existing
messages, renumbering messages or components, etc.

Since a previous version, the `.Groups` member has been privatized,
and the `.Get()` convenience accessor has been renamed `.Groups()`
and is now used consistently in a way that should reduce the risk
of nil pointers causing crashes. Also, NewGroupCounts is used in
a couple more places.
2021-01-19 16:23:15 -06:00
Cody Soyland
4ebf6f6ff7 Customize serialization of []GroupCount based on aggregate type/presence 2021-01-19 12:10:56 -06:00
nagamocha3000
0437f5d28a Handle case where row returned might be nil 2021-01-19 16:51:26 +03:00
Ben Johnson
56c001d288
Merge pull request #1341 from molecula/bench-vars
Update benchmarks to use expvar URL list
2021-01-18 16:06:43 -07:00
nagamocha3000
d26c6b048e Sort all rowIDs gathered before storing them 2021-01-19 02:00:50 +03:00
nagamocha3000
9b23dcdd0a Gather rows for each fragment in a much smarter way 2021-01-19 01:44:54 +03:00
Ben Johnson
01e6781abd Update benchmarks to use expvar URL list 2021-01-18 13:43:53 -07:00
nagamocha3000
f51ff4dc85 Add tests for Rows() on time fields 2021-01-18 20:54:31 +03:00
nagamocha3000
b77f9e8a43 Add timeFragments rowIterator 2021-01-15 23:36:54 +03:00
nagamocha3000
f91b4bc016 Add queries to test GroupBy Rows on time field 2021-01-15 22:58:18 +03:00
nagamocha3000
54d79d1e4d Populate 'places_visited' field for 'users' index 2021-01-15 22:50:54 +03:00
nagamocha3000
4dc3e49a0d Add test helper for inserting to time quantum fields 2021-01-15 22:42:37 +03:00
Ben Johnson
fbf546f131
Merge pull request #1319 from molecula/pilosa-bench 2021-01-12 16:51:27 -07:00
Ben Johnson
b7973e2612 Add read benchmarks 2021-01-12 13:48:09 -07:00
Matthew Jaffee
701d6448cc
Merge pull request #1307 from jaffee/1292-panic-groupby-int-distinct-aggregate
Fix potential panic when grouping on int field with Distinct aggregate
2021-01-08 16:38:00 -06:00
Matt Jaffee
48552553dc
guard against NPE when setting precomputed data
If a precomputed call returns a nil Row result somehow, that could
cause a nil pointer exception when handling the result in
handlePreCall.

In this particular case, A Distinct call on a BSI field with a filter
which returned no results could return a SignedRow{} with nil *Rows
inside of it. This only manifested if there was data in a single shard
as otherwise the reduce logic created a SignedRow with empty *Row
objects rather than nil ones. Isn't that fun?

Extra fun: the reason the filter was returning no results was not
because it was actually empty, but because of another bug where
constructing the Distinct calls to compute the aggregate of a GroupBy
doesn't take into account that the group might include an integer
field which means that the call needs to be constructed
differently. That bug is not fixed in this commit, hence the tests are
still failing, but not panicking.
2021-01-08 15:03:49 -06:00
Matthew Jaffee
616ba408f8
Merge pull request #1308 from codysoyland/groupby-distinct-int
Fix GroupBy Distinct aggregate on int field
2021-01-08 13:55:38 -06:00
Cody Soyland
83ce30f1c4
Fix GroupBy Distinct aggregate on int field 2021-01-08 13:35:58 -06:00
Ben Johnson
55fad52c59
Merge pull request #1306 from molecula/rbf-remove-cursor-arena
Remove RBF cursor arena.
2021-01-08 10:13:52 -07:00
Ben Johnson
489f33a173 Remove RBF cursor arena.
Previously there were two implementations of cursor reuse: sync pool &
an arena. This commit removes the arena in favor of the global pool.
2021-01-08 07:59:10 -07:00
Ben Johnson
1865de8248
Merge pull request #1301 from molecula/fix-rbf-cursor-close 2021-01-07 19:22:34 -07:00
jaten-molecula
b3854e867b
Merge branch 'master' into fix-rbf-cursor-close 2021-01-07 20:14:59 -06:00
Cody Soyland
476767f150
Merge pull request #1285 from codysoyland/grpc-crd
Add create, read, and delete index methods to gRPC interface
2021-01-07 15:24:13 -06:00
Cody Soyland
4f57b4d07b
Remove TrackExistence configuration from gRPC CreateIndex 2021-01-07 15:19:08 -06:00
Cody Soyland
d8ebfda1bd
Undo error cause changes due to broken logic in other places, check for ConflictError explicitly 2021-01-07 14:49:18 -06:00
Cody Soyland
219714a18e
Fix linter problems 2021-01-07 13:50:59 -06:00
Cody Soyland
daa8c9bd8b
Add tests for new gRPC create/get/delete calls 2021-01-07 13:05:48 -06:00
Cody Soyland
fdf5818fc2
Add create, read, and delete index methods to gRPC interface 2021-01-07 13:05:48 -06:00
Ben Johnson
93f06e0f9d Fix rbf.Cursor.Close() panic 2021-01-07 11:28:46 -07:00
Maxton Huff
17e010cca2
Merge pull request #1293 from Maxtonian/verbose-message
Fix formatting bug in verbose log message
2021-01-07 09:38:29 -06:00
Maxton Huff
d10e45648b change f.path to f.path() 2021-01-06 15:58:01 -06:00
Matthew Jaffee
d6cba17a01
Merge pull request #1274 from jaffee/generalized-groupby-sort-2
Add ability to sort on count or aggregate in GroupBy. Fix bug with offset being unsupported. Fix bugs with limit interacting poorly with other arguments.
2021-01-05 09:41:18 -07:00
Matt Jaffee
216e28a77e
add getSorter tests, fix bugs 2021-01-05 09:54:46 -06:00
Matt Jaffee
6dccb3d6be
remove (unused) sorting code related to fields, add comments 2021-01-03 08:34:53 -06:00
Matt Jaffee
6094663e7a
fix bug with "having" and "limit" in GroupBy
the limit could get applied before "having" in some cases which could
result in results being discarded which met the having condition while
results were kept which did not, ultimately resulting in GroupBy
falsely reporting fewer results than actually existed.
2021-01-01 21:56:59 -06:00
Matt Jaffee
ea539d8241
simplify groupby sorting and fix bugs
Back out support for sorting on fields (only count and aggregate
supported for now).

Fix bug where default return of "true" caused sort to be unstable. (If
they are equal, Less should return false)

Fix bug where limit was being applied before sorting.

Fix bug where offset was not actually allowed to be an argument to
GroupBy (weird! guess we weren't testing that very well)

Apply "having" after calculating Count(Distinct) aggregate so that
having can apply to that.

Switch to stable sort to make testing easier.
2020-12-31 14:42:03 -06:00
Matt Jaffee
5fdae74812
draft of sorting groupby results 2020-12-31 14:41:52 -06:00