This modifies the parser to properly "unquote" incoming strings. So if
a string comes in double or single quoted, we approximately follow Go
rules for removing the quotes and processing escape sequences.
The differences from Go are:
1. we only support backslash, quote, tab and newline escape
sequenences.
2. Single quoted strings are supported and work just like double
quoted strings.
3. The peg parser won't actually accept backquoted strings (I don't
think)
Fixes: #411
This commit introduces a new type: pql.Decimal
We use that instead of float64 in order to ensure
that the string representation is consistent.
One unfortunate discovery during implementation is
that the RowAttrs and ColAttrs support floats, and
the PEG file was treating them as such. So I had
to split the PEG definitions into float-specific
items and decimal-specific items.
After a few experiments with pkg/plugin, I'm ready to concede that the
people warning me it was unsuitable for production use were in fact
correct.
In the brave new world, the "ext" package is moved to its own module
outside pilosa. This means that importing it doesn't imply any need to
version-check against pilosa; we can just use versioned copies of the
ext package, which can be public because it doesn't contain anything
we need to care about keeping proprietary.
Then we can, conditional on build tags, import modules from a
neighboring repo which contains the actual implementations, and if
they're imported, their init functions register them.
This PR is meant to get all columns from an index
based on the TrackExistence row.
`All()` is a PQL function that can be used as a typical
row object. Optional arguments are `limit` and `offset`.
There is a TODO in the `StringWithSubj` method because the value
types really depend on the subject type (for example, `count` uses
uint64, while `sum` uses int64). I'm waiting to address this
until we decide how to handle sums of floats (Decimal), because
that will affect this logic as well.
This PR adds support for a `having` argument in a `GroupBy` query.
Usage looks like this:
```
GroupBy(Rows(a), having=Condition(count > 10))
GroupBy(Rows(a), aggregate=Sum(field=b), having=Condition(sum > 100))
```
had to workaround some cruft in the parser that was trying to only
support a BETWEEN query as LTE, LTE. Now we have operations for all
combinations of LT and LTE.
unrelated - changed the port a test was binding to as it conflicted
with a port I was using locally.
So in some cases, when we do a query, the results of one
part of the query are innately shared-across-nodes; for
instance, a hypothetical Distinct query. More generally,
we allow cross-index queries; calls can have "index=foo"
in them.
This patch lets us handle that without duplicating that
query all over. Before we actually start doing the
separate calls, we run the query once from the coordinating
node, then patch the results in, and send relevant subsets
over to each client, etcetera. Also provides slightly
friendlier (and I hope faster) support for converting
bitmaps to/from sets of rows.
We also add an extension interface, and some fancy stuff
to let us define new calls, which use this. They're sort
of tied together because the first extension I wanted to
implement needed precomputed calls. The extension API
lets us create extensions using `pkg/plugin` (with all its
associated limitations, unfortunately), then query them
at load time for functionality.
This also implies some revamping of the argument
validation for PQL, like verifying that functions exist
and knowing things about their argument types.
So basically this is an overly intrusive patch, and would
be better as separate patches, but they're hard to detangle.
add trivial execution-time profiling
What if you could ?profile=true on a query and get some
numbers back? That'd be really cool.
We already have tracing/spans, but right now, those only generate
any data if you have something set up for them to trace to. Add a
fancy wrapper that lets us generate our own tracing data, and dump
it into the request response, if ?profile=true.
add a sample extension, add missing features to extension interface
Implement a naive probabilistic filter extension as an example of
what an extension looks like. In the process, discover multiple
omissions in the bitmap API. Well, I did *say* it was experimental.
The PQL looks like:
```
Range(frame=f, field0 >< [200,610])
```
One thing I noticed while implementing this is that it doesn't seem
like `FieldRange()` is used in either `Frame` or `View`; the Executor
calls `Fragment.FieldRange()` directly. The problem with this is that
the offset logic is calculated in the Frame, but since the Executor
doesn't go through Frame, then the Executor also has to calculate
the offset before calling `Fragment.FieldRange`. We should unify this
logic somewhere. Note, this applies to both `FieldRange` and
`FieldRangeBetween`.
