This is pretty expensive even for default shard width, and very expensive
for ShardWidth = 1<<22, and we don't really get much extra benefit from
having a million values instead of a hundred or so.
If you try to Store to a nonexistent field, we create an automatic
Set field with no cache for it, assuming it won't be used for TopN
queries. If you want TopN to work, you need to actually create it
yourself.
If a shard has never had any decimal values in it at all for a
field, the ValCount object returned has no DecimalVal, which could
cause a segfault if we don't check for it. Add a test case which
sporadically triggers that behavior (it's timing/luck related,
unfortunately), and then also fix it.
If an index is provided to a bare distinct which happens
to be the index handling the query, then the query needs
to behave as if no index argument was provided.
For example:
When querying against index `i`,
```
Distinct(index="i", field="ints")`
```
should behave exactly like
```
Distinct(field="ints")
```
It turns out that it's not very useful to keep the sign
value as a separate argument in the pql.Decimal struct.
This commit incorporates it into Value, and makes Value
an `int64` (for some bone-headed reason I had made it a
`uint32` before which is just dumb).
this involved adding an optional float value to the ValCount struct
which complicated result types, necessitated grpc changes, and needed
quite a few tests at different layers.
This allows a BSI field to have an option indicating
that it is a foreign key to another index. If the foreign
index has column keys, then this field handles string values
by using the foreign index's translate store.
This PR is meant to get all columns from an index
based on the TrackExistence row.
`All()` is a PQL function that can be used as a typical
row object. Optional arguments are `limit` and `offset`.
This PR adds a field name (string) to the return types
which represent the values from a specific field. For example,
a TopN query on field `x` would be `TopN(x)` and have results
like:
```
[]Pair{
{ID: 14, Count: 10},
{ID: 3, Count: 8},
{ID: 7, Count: 3},
}
```
In order to know what field this result type refers to, we wrap
`[]Pair` in a new struct called `PairsField` which contains an
addition `Field` string where `x` is stored.
This is useful for informing the gRPC server how to construct
more appropriate headers for the result stream (in this case,
the column headers can now be "x" and "count").
Similar logic was applied to `RowIdentifiers` and `Pair` as well.
This PR adds support for a `having` argument in a `GroupBy` query.
Usage looks like this:
```
GroupBy(Rows(a), having=Condition(count > 10))
GroupBy(Rows(a), aggregate=Sum(field=b), having=Condition(sum > 100))
```
This change allows one to query Pilosa fields and indexes directly
with integer row and column ids even when key translation is
enabled. This was previously disallowed during query
translation... I'm not sure why, but it can be quite useful for
debugging and testing to be able to use IDs directly. I have a test in
go-pilosa which uses this functionality.
I also simplified a bunch of the test code which was of the form:
```
else {
if blah {
}
}
```
to be:
```
else if blah {
```
which I think is pretty harmless.
I also changed a snapshot log line that has been bugging me to be
Debug level so that it isn't generating lots of useless logs for long
running Pilosa instances.
