Commit graph

5250 commits

Author SHA1 Message Date
tgruben
ba70bf3079
Merge pull request #89 from tgruben/bug-q2-double-delete
bit remove leaves internals corrupt on empty edge case
2020-01-12 16:25:31 -06:00
tgruben
3261ec4dc4
Merge branch 'enterprise' into bug-q2-double-delete 2020-01-12 16:09:12 -06:00
Todd Gruben
99108a1c63 bit remove leaves internals corrupt on empty edge case 2020-01-12 12:00:00 -06:00
Travis Turner
9015c00da9
Merge pull request #84 from travisturner/foreign-index
Add FieldOption.ForeignIndex
2020-01-10 15:45:02 -06:00
Travis
a6a2f84bd5 During Holder.Open, apply foreign index after all indexes open
In the case where a field with a foreign index opens before the
foreign index has opened (and is available as a reference in the
holder), push the field into a queue to have its foreign index
applied once all indexes have opened.
2020-01-10 12:28:28 -06:00
Travis
3d3286a9ca fix an issue caused by empty column list defaulting to IDs 2020-01-10 12:28:28 -06:00
Travis
f79fde43e3 exclude internal fields (i.e. _exists) from Inspect output 2020-01-10 12:28:28 -06:00
Travis
742135dc10 Get ForeignIndex string value when reading BSI field.
In the `Inspect` function in `server/grpc.go`, getting
the value of an `int` field with a foreign index to
an index with `Keys()`, we need to return the string
key value instead of the BSI int value for the field.

This commit also changes the method `Field.keys()` to be
exported as `Field.Keys()` so that it's accessible in
the server package.
2020-01-10 12:28:28 -06:00
Travis
35f9dfa374 remove write portion of extension data race 2020-01-10 12:28:28 -06:00
Travis
881d3bef06 Adjust the FieldOption logic to be in place prior to field.Open().
This commit changes the order of FieldOption application so that
it's always set before field.Open() is called.

This was required because field.Open() now uses some of the values
from FieldOptions to determine if/when to use a particular
translateStore. For example, when FieldOptions.ForeignIndex is set,
the translateStore from the foreign index is retrieved during
field.Open().
2020-01-10 12:28:28 -06:00
Matt Jaffee
4d8f307c5e handle string values in ImportValueRequest sorting 2020-01-10 12:28:28 -06:00
Matt Jaffee
132cf7cc1c add StringValues to proto ImportValueRequest, update proto versions
I ran:

brew upgrade protobuf
GO111MODULE=off go get -u github.com/gogo/protobuf/protoc-gen-gofast

I'm not sure if everything is still going to work, but I'm excited to
find out!
2020-01-10 12:28:27 -06:00
Travis
b22d0143e4 loadNewExtensions is unused, but included for completeness 2020-01-10 12:28:27 -06:00
Travis
1542cbefc0 Add FieldOption.ForeignIndex
This allows a BSI field to have an option indicating
that it is a foreign key to another index. If the foreign
index has column keys, then this field handles string values
by using the foreign index's translate store.
2020-01-10 12:28:27 -06:00
Matthew Jaffee
7aef0743e1
Merge pull request #86 from jaffee/field-char-limit
allow field and index names up to 230 characters
2020-01-09 14:43:55 -06:00
Matt Jaffee
0a1de81441
allow field and index names up to 230 characters
The lowest limitation I've seen on any filesystem we care about is 255
characters. 230 leaves enough space that an index or field could be
backed up and have a timestamp and file extension appended while
still allowing for much longer index and field names.
2020-01-09 13:02:22 -06:00
tgruben
457789effd
Merge pull request #62 from tgruben/clearvalue
add support to clear value for column
2020-01-07 13:12:21 -06:00
Todd Gruben
643884aeb3 refactored clear to fix q2;removed unused comment 2020-01-07 12:55:32 -06:00
Todd Gruben
26e3460413 removed view creation from ClearValue 2020-01-07 10:55:45 -06:00
Todd Gruben
d843959904 fixed comments; removed create fragment 2020-01-07 10:39:56 -06:00
Todd Gruben
75e017cf4f Merge remote-tracking branch 'upstream/enterprise' into clearvalue 2020-01-07 10:28:47 -06:00
Cody Soyland
29f7448b89
Merge pull request #83 from codysoyland/expvar-compatibility
Initialize expvar lazily to prevent panic if importing both Pilosa v1 and v2
2020-01-02 17:56:42 -06:00
Cody Soyland
8d32005ede Initialize expvar lazily to prevent panic if importing both Pilosa v1 and v2. 2020-01-02 15:43:34 -06:00
Travis Turner
5643afac47
Merge pull request #82 from travisturner/row-response-sorter
WIP: RowResponseSorter for sorting a list of RowResponse based on sort params
2019-12-31 13:18:05 -06:00
Travis
0fffd9a0cb RowResponseSorter for sorting a list of RowResponse based on sort paraters 2019-12-31 11:52:01 -06:00
Travis Turner
c247805d96
Merge pull request #81 from travisturner/inspect-field-output
Remove empty field check in Inspect()
2019-12-27 13:42:36 -06:00
Travis Turner
98df5672e9
Merge branch 'enterprise' into inspect-field-output 2019-12-27 13:26:38 -06:00
Travis Turner
768de9dc3a
Merge pull request #80 from travisturner/row-response-error
Add StatusError to RowResponse for better error handling.
2019-12-27 13:25:38 -06:00
Travis
3be4141382 Return the correct data type label in grpc header
Based on the pilosa field type, return the correct data type
label in the gRPC column header.
2019-12-27 12:43:45 -06:00
Travis
73090e05c8 Add StatusError to RowResponse for better error handling.
This PR adds a `StatusError` to the `pproto.RowResponse` type, which
allows a stream to pass an error on the stream (encoded into
the `RowResponse.StatusError`). This can be checked downstream
for matching `EOF` or `err != nil` and handled appropriately.

This is helpful mainly with the `RowResponse` reducers which run in
goroutines. Instead of trying to manage a separate channel of errors
from those goroutines, we just follow the grpc model and send the
error with the stream.
2019-12-27 12:43:45 -06:00
Travis Turner
94f97297eb
Merge pull request #77 from travisturner/all-shard
Allow All() to be called at the shard level
2019-12-27 12:35:41 -06:00
Travis
d34e38f134 Remove empty field check in Inspect()
The check for field existence is not necessary; since we
add the `_id` field to every response then at the very
least that field will be returned.

This check was preventin a query like `select _id from ...`
from returning any results.
2019-12-26 22:27:46 -06:00
Travis
586a13e942 Allow All() to be called at the shard level 2019-12-20 22:44:46 -06:00
Cody Soyland
7b30b91448
Merge pull request #74 from codysoyland/docker-build-vendor
Vendor modules before building docker image so private modules can be downloaded
2019-12-20 17:52:26 -06:00
Cody Soyland
d4117f3137 Vendor modules before building docker image so private modules can be downloaded 2019-12-20 16:05:18 -06:00
seebs
47316d9f2f
Merge pull request #70 from seebs/unionAA
Simplify unionArrayArray
2019-12-20 13:49:51 -06:00
seebs
d4d3d75e28
Merge branch 'enterprise' into unionAA 2019-12-20 13:28:57 -06:00
Matthew Jaffee
d86a3c3f2f
Merge pull request #55 from seebs/pluginfix
Pluginfix
2019-12-20 12:53:09 -06:00
Matt Jaffee
fe0f57651e
build with distinct by default 2019-12-20 12:21:47 -06:00
Cody Soyland
49b2029656
Run "go mod vendor" outside of Docker so authenticated modules may use system credentials 2019-12-20 12:21:47 -06:00
Seebs
7e1fd8392f
go.mod/go.sum changes for using molecula/ext
This pins us to the initial external release of molecula/ext, which
with any luck will be the only one. (Narrator: It was not to be the
only one.) We also use GOPRIVATE so we don't need a replace directive.
2019-12-20 12:21:47 -06:00
Seebs
0eba050054
stop using pkg/plugin, start using build tags
After a few experiments with pkg/plugin, I'm ready to concede that the
people warning me it was unsuitable for production use were in fact
correct.

In the brave new world, the "ext" package is moved to its own module
outside pilosa. This means that importing it doesn't imply any need to
version-check against pilosa; we can just use versioned copies of the
ext package, which can be public because it doesn't contain anything
we need to care about keeping proprietary.

Then we can, conditional on build tags, import modules from a
neighboring repo which contains the actual implementations, and if
they're imported, their init functions register them.
2019-12-20 12:21:47 -06:00
Seebs
f51c2dbc42
use extensions through build tags 2019-12-20 12:21:47 -06:00
Travis Turner
179fb910f7
Merge pull request #71 from travisturner/all-limit-offset
Add All() support to PQL, including limit and offset
2019-12-18 22:10:42 -06:00
Travis
361e51cb41 Add All() support to PQL, including limit and offset
This PR is meant to get all columns from an index
based on the TrackExistence row.

`All()` is a PQL function that can be used as a typical
row object. Optional arguments are `limit` and `offset`.
2019-12-18 18:00:15 -06:00
Seebs
83aa505673 Simplify unionArrayArray
Also short-circuit it in some cases.
2019-12-17 15:30:56 -06:00
Travis Turner
1af016df84
Merge pull request #68 from travisturner/linter-fix
Fix an impossible code path raised by the linter
2019-12-17 09:04:37 -06:00
Travis
532caa0fbf fix an impossible code path raised by the linter 2019-12-16 21:56:57 -06:00
Travis Turner
6f556fb880
Merge pull request #63 from travisturner/row-field-label
Wrap return types: RowIdentifiers, Pair, and []Pair
2019-12-16 07:42:20 -06:00
Travis
4f7f4f58b1 add field to SignedRow, and implement its grpc response 2019-12-14 15:56:06 -06:00