scriptoriumadmin/featurebase
1
0
Fork 0
mirror of https://github.com/featurebasedb/featurebase.git synced 2026-08-28 10:54:59 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions

merge: featurebase merge updates for 2022-10-28 (#2188)

Browse source 
* use t.Fatal(f) to abort tests, not panic

* make perf_able run at all, make it debug a bit better

switch perf-able to using same node type we use for other spot instances,
because otherwise it never finds any available capacity.

we switch the perf-able script to use the standard get_value function
instead of direct jq calls.

we try to grab server logs if the restore fails in the hopes of finding
out why the restore very occasionally fails.

* Fix some issues with running IDK tests in docker. (#2248)

*Stop running TestKafkaSourceIntegration with t.Parallel()

This test can't be run in parallel as it's currently written. Doing so
allows for interleaving of messages to the same kafka topic between
tests.

I didn't attempt to modify the test so it could be run in parallel. That
could be done, but left for someone more ambitious.

* Remove idk/testenv/certs which got accidentally committed.

also update .gitignore to include those.

* changes to add bool support in idk (#2240)

* initial changes to add bool support in idk

* modifying some default parameters for testing, will revert them later

* adding support for bool in making fragments function

* boolean values implementation without supporting empty or null values at this point

* Implement bool support in batch using a map (and a slice for nulls) (#2247)

* Implement bool support in batch using a map (and a slice for nulls)

* Keep the PackBools default for now

But set it explicity in the ingest tests which rely on it.

* Modify batch to construct bool update like mutex

The code in API.ImportRoaringShard has a switch statement which causes
bool fields to be handled like mutex fields. This means, that the
viewUpdate.Clear value should only contain data in the first "row" of
the fragment, which it will treat as records to clear for *all* rows.
This makes more sense for mutex fields; for bool fields, there's only
one other row to clear. But since the code is currently handling them
the same, we need to construct viewUpdate.Clear such that it conforms to
that pattern.

This commit also adds a test which covers this logic.

* Remove commented code; revert config for testing

This commit also removes the DELETE_SENTINEL case for non-packed bools,
since that isn't supported anyway.

* Revert default setting

* remove inconsistent type scope

* correcting the logic of string converstion to bool

* resolving an error in a test

* adding tests to cover code related to bool support in batch.go file and interface.go files

* modifying interfaces test

* added one more test case

Co-authored-by: Travis Turner <travis@pilosa.com>
Co-authored-by: Travis Turner <travis@molecula.com>

* resolving bool null field ingestion error (#2254)

* resolving bool null field ingestion error

* testing issues

* adding null support for bools

* updating the null bool field ingestion

* trying to resolve issue when ingesting null value for bool type

* adding a clearing support for bool type

* resolving issues with bool null value ingestion

* updating the jwt go package version and removing changes made in docker compose file

* reverting jwt go version

* removing v4 of jwt

* adding a comment in test file to see if sonar cloud accepts this file

* don't obtain stack traces on rbf.Tx creation

We thought stack traces were mildly expensive. We were very wrong.
Due to a complicated issue in the Go runtime, simultaneous requests
for stack traces end up contending on a lock even when they're not
actually contending on any resources. I've filed a ticket in the
Go issue tracker for this:

	https://github.com/golang/go/issues/56400

In the mean time: Under some workloads, we were seeing 85% of all
CPU time go into the stack backtraces, of which 81% went into the
contention on those locks. But even if you take away the contention,
that leaves us with 4/19 of all CPU time in our code going into
building those stack backtraces. That's a lot of overhead for a
feature we virtually never use.

We might consider adding a backtrace functionality here, possibly
using `runtime.Callers` which is much lower overhead, and allows us
to generate a backtrace on demand (no argument values available,
but then, we never read those because they're unformatted hex
values), but I don't think it's actually very informative to know
what the stack traces were of the Tx; they don't necessarily reflect
the current state of any ongoing use of the Tx, so we can't necessarily
correlate them to goroutine stack dumps, and so on.

* fb-1729 Enriched Table Metadata (#2255)

enriched metadata for tables

added support for the concept of a table and field owners in metadata; mechanism to derive owner from http request metadata; metadata for table description

* tightened up is/is not null filter expressions (FB-1741) (#2260)

Covers tightening up handling filter expressions that contain is/is not null ops. These filters may have to be translated into PQL calls to be passed to the executor and even though sql3 language supports nullability for any data type, currently only BSI fields are nullable at the storage engine level (there is a ticket to add support for non-BSI field here FB-1689: IS SQL Argument returns incorrect error) so when these fields are used in filter conditions we need to handle BSI and non-BSI fields differently.

* added a test to cover the keyword replace as being synonymous with insert (#2261)

* update molecula references to featurebase (#2262)

Co-authored-by: Seebs <seebs@molecula.com>
Co-authored-by: Travis Turner <travis@pilosa.com>
Co-authored-by: Pranitha-malae <56414132+Pranitha-malae@users.noreply.github.com>
Co-authored-by: Travis Turner <travis@molecula.com>
Co-authored-by: pokeeffe-molecula <85502298+pokeeffe-molecula@users.noreply.github.com>
Co-authored-by: Stephanie Yang <stephanie@pilosa.com>
This commit is contained in:
CLoZengineer 2022-10-28 13:08:23 -04:00 • committed by GitHub
parent fc1a5dad53
commit daceee2ab6
No known key found for this signature in database
GPG key ID: 4AEE18F83AFDEB23
88 changed files with 3037 additions and 1804 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

3
.gitignore vendored
View file

@ -24,6 +24,9 @@ builds/
*.tfstate.backup
.vscode
idk/testdata/idk*.out
idk/testenv/certs/*
# copy of .gitignore from archived idk repo
# Compiled Object files, Static and Dynamic libs (Shared Objects)
*.o

2
Dockerfile
View file

@ -22,7 +22,7 @@ FROM golang:${GO_VERSION} as pilosa-builder
ARG MAKE_FLAGS
WORKDIR /pilosa
RUN go get github.com/rakyll/statik
RUN go install github.com/rakyll/statik@v0.1.7
COPY . ./
COPY --from=lattice-builder /lattice/build /lattice

8
Makefile
View file

@ -205,12 +205,8 @@ generate-pql: require-peg
generate-proto-grpc: require-protoc require-protoc-gen-go
protoc -I proto proto/pilosa.proto --go_out=plugins=grpc:proto
protoc -I proto proto/vdsm/vdsm.proto --go_out=plugins=grpc:proto
# TODO: Modify above commands and remove the below mv if possible.
# See https://go-review.googlesource.com/c/protobuf/+/219298/ for info on --go-opt
# I couldn't get it to work during development - Cody
cp -r proto/github.com/featurebasedb/featurebase/v3/proto/ proto/
rm -rf proto/github.com
# address re-generation here only if we need to
# protoc -I proto proto/vdsm.proto --go_out=plugins=grpc:proto
# `go generate` all needed packages
generate: generate-protoc generate-statik generate-stringer generate-pql

27
api.go
View file

@ -224,14 +224,23 @@ func (api *API) CreateIndex(ctx context.Context, indexName string, options Index
span, _ := tracing.StartSpanFromContext(ctx, "API.CreateIndex")
defer span.Finish()
// get the requestUserID from the context -- assumes the http handler has populated this from
// authN/Z info
requestUserID, ok := ctx.Value(ContextRequestUserIdKey).(string)
if !ok {
requestUserID = ""
}
if err := api.validate(apiCreateIndex); err != nil {
return nil, errors.Wrap(err, "validating api method")
}
// Populate the create index message.
ts := timestamp()
cim := &CreateIndexMessage{
Index: indexName,
CreatedAt: timestamp(),
CreatedAt: ts,
Owner: requestUserID,
Meta: options,
}
@ -307,6 +316,13 @@ func (api *API) CreateField(ctx context.Context, indexName string, fieldName str
return nil, errors.Wrap(err, "validating api method")
}
// get the requestUserID from the context -- assumes the http handler has populated this from
// authN/Z info
requestUserID, ok := ctx.Value(ContextRequestUserIdKey).(string)
if !ok {
requestUserID = ""
}
// Apply and validate functional options.
fo, err := newFieldOptions(opts...)
if err != nil {
@ -324,11 +340,12 @@ func (api *API) CreateField(ctx context.Context, indexName string, fieldName str
Index: indexName,
Field: fieldName,
CreatedAt: timestamp(),
Owner: requestUserID,
Meta: fo,
}
// Create field.
field, err := index.CreateField(fieldName, opts...)
field, err := index.CreateField(fieldName, requestUserID, opts...)
if err != nil {
return nil, errors.Wrap(err, "creating field")
}
@ -358,7 +375,11 @@ func (api *API) UpdateField(ctx context.Context, indexName, fieldName string, up
return newNotFoundError(ErrIndexNotFound, indexName)
}
cfm, err := index.UpdateField(ctx, fieldName, update)
// get the requestUserID from the context -- assumes the http handler has populated this from
// authN/Z info
requestUserID, _ := ctx.Value(ContextRequestUserIdKey).(string)
cfm, err := index.UpdateField(ctx, fieldName, requestUserID, update)
if err != nil {
return errors.Wrap(err, "updating field")
}

4
api_test.go
View file

@ -1388,11 +1388,11 @@ func TestVariousApiTranslateCalls(t *testing.T) {
// this should never actually get used because we're testing for errors here
r := strings.NewReader("")
// test index
idx, err := api.Holder().CreateIndex(c.Idx(), pilosa.IndexOptions{})
idx, err := api.Holder().CreateIndex(c.Idx(), "", pilosa.IndexOptions{})
if err != nil {
t.Fatalf("%v: could not create test index", err)
}
if _, err = idx.CreateFieldIfNotExistsWithOptions("field", &pilosa.FieldOptions{Keys: false}); err != nil {
if _, err = idx.CreateFieldIfNotExistsWithOptions("field", "", &pilosa.FieldOptions{Keys: false}); err != nil {
t.Fatalf("creating field: %v", err)
}
t.Run("translateIndexDbOnNilIndex",

146
client/batch.go
View file

@ -87,6 +87,7 @@ type agedTranslation struct {
// | uint64 | set | any |
// | int64 | int | any |
// | float64| decimal | scale |
// | bool | bool | any |
// | nil | any | |
//
// nil values are ignored.
@ -124,6 +125,15 @@ type Batch struct {
// values holds the values for each record of an int field
values map[string][]int64
// boolValues is a map[fieldName][idsIndex]bool, which holds the values for
// each record of a bool field. It is a map of maps in order to accomodate
// nil values (they just aren't recorded in the map[int]).
boolValues map[string]map[int]bool
// boolNulls holds a slice of indices into b.ids for each bool field which
// has nil values.
boolNulls map[string][]uint64
// times holds a time for each record. (if any of the fields are time fields)
times []QuantizedTime
@ -233,6 +243,8 @@ func NewBatch(client *Client, size int, index *Index, fields []*Field, opts ...B
headerMap := make(map[string]*Field, len(fields))
rowIDs := make(map[int][]uint64, len(fields))
values := make(map[string][]int64)
boolValues := make(map[string]map[int]bool)
boolNulls := make(map[string][]uint64)
tt := make(map[int]map[string][]int, len(fields))
ttSets := make(map[string]map[string][]int)
hasTime := false
@ -257,6 +269,8 @@ func NewBatch(client *Client, size int, index *Index, fields []*Field, opts ...B
tt[i] = make(map[string][]int)
}
rowIDs[i] = make([]uint64, 0, size)
case FieldTypeBool:
boolValues[field.Name()] = make(map[int]bool)
default:
return nil, errors.Errorf("field type '%s' is not currently supported through Batch", typ)
}
@ -273,6 +287,8 @@ func NewBatch(client *Client, size int, index *Index, fields []*Field, opts ...B
clearRowIDs: make(map[int]map[int]uint64),
rowIDSets: make(map[string][][]uint64),
values: values,
boolValues: boolValues,
boolNulls: boolNulls,
nullIndices: make(map[string][]uint64),
toTranslate: tt,
toTranslateClear: make(map[int]map[string][]int),
@ -480,28 +496,8 @@ func (b *Batch) Add(rec Row) error {
field := b.header[i]
switch val := rec.Values[i].(type) {
case string:
if field.Opts().Type() != FieldTypeInt {
// nil-extend
for len(b.rowIDs[i]) < curPos {
b.rowIDs[i] = append(b.rowIDs[i], nilSentinel)
}
rowIDs := b.rowIDs[i]
// empty string is not a valid value at this point (Pilosa refuses to translate it)
if val == "" { //
b.rowIDs[i] = append(rowIDs, nilSentinel)
} else if rowID, ok := b.getRowTranslation(field.Name(), val); ok {
b.rowIDs[i] = append(rowIDs, rowID)
} else {
ints, ok := b.toTranslate[i][val]
if !ok {
ints = make([]int, 0)
}
ints = append(ints, curPos)
b.toTranslate[i][val] = ints
b.rowIDs[i] = append(rowIDs, 0)
}
} else if field.Opts().Type() == FieldTypeInt {
switch field.Opts().Type() {
case FieldTypeInt:
if val == "" {
// copied from the `case nil:` section for ints and decimals
b.values[field.Name()] = append(b.values[field.Name()], 0)
@ -522,6 +518,30 @@ func (b *Batch) Add(rec Row) error {
b.toTranslate[i][val] = ints
b.values[field.Name()] = append(b.values[field.Name()], 0)
}
case FieldTypeBool:
// If we want to support bools as string values, we would do
// that here.
default:
// nil-extend
for len(b.rowIDs[i]) < curPos {
b.rowIDs[i] = append(b.rowIDs[i], nilSentinel)
}
rowIDs := b.rowIDs[i]
// empty string is not a valid value at this point (Pilosa refuses to translate it)
if val == "" { //
b.rowIDs[i] = append(rowIDs, nilSentinel)
} else if rowID, ok := b.getRowTranslation(field.Name(), val); ok {
b.rowIDs[i] = append(rowIDs, rowID)
} else {
ints, ok := b.toTranslate[i][val]
if !ok {
ints = make([]int, 0)
}
ints = append(ints, curPos)
b.toTranslate[i][val] = ints
b.rowIDs[i] = append(rowIDs, 0)
}
}
case uint64:
// nil-extend
@ -579,8 +599,8 @@ func (b *Batch) Add(rec Row) error {
}
b.rowIDSets[field.Name()] = append(rowIDSets, val)
case nil:
t := field.Opts().Type()
if t == FieldTypeInt || t == FieldTypeDecimal || t == FieldTypeTimestamp {
switch field.Opts().Type() {
case FieldTypeInt, FieldTypeDecimal, FieldTypeTimestamp:
b.values[field.Name()] = append(b.values[field.Name()], 0)
nullIndices, ok := b.nullIndices[field.Name()]
if !ok {
@ -589,7 +609,15 @@ func (b *Batch) Add(rec Row) error {
nullIndices = append(nullIndices, uint64(curPos))
b.nullIndices[field.Name()] = nullIndices
} else {
case FieldTypeBool:
boolNulls, ok := b.boolNulls[field.Name()]
if !ok {
boolNulls = make([]uint64, 0)
}
boolNulls = append(boolNulls, uint64(curPos))
b.boolNulls[field.Name()] = boolNulls
default:
// only append nil to rowIDs if this field already has
// rowIDs. Otherwise, this could be a []string or
// []uint64 field where we've only seen nil values so
@ -600,6 +628,10 @@ func (b *Batch) Add(rec Row) error {
b.rowIDs[i] = append(rowIDs, nilSentinel)
}
}
case bool:
b.boolValues[field.Name()][curPos] = val
default:
return errors.Errorf("Val %v Type %[1]T is not currently supported. Use string, uint64 (row id), or int64 (integer value)", val)
}
@ -637,7 +669,6 @@ func (b *Batch) Add(rec Row) error {
}
b.rowIDs[i][len(b.rowIDs[i])-1] = clearSentinel
}
default:
return errors.Errorf("Clearing a value '%v' Type %[1]T is not currently supported (field '%s')", val, field.Name())
}
@ -712,7 +743,6 @@ func (b *Batch) Import() error {
return errors.Wrap(err, "making fragments (flush)")
}
if b.useShardTransactionalEndpoint {
// TODO handle bool?
frags, clearFrags, err = b.makeSingleValFragments(frags, clearFrags)
if err != nil {
return errors.Wrap(err, "making single val fragments")
@ -1478,6 +1508,60 @@ func (b *Batch) makeSingleValFragments(frags, clearFrags fragments) (fragments,
}
}
}
// -------------------------
// Boolean fields
// -------------------------
falseRowOffset := 0 * shardWidth // fragment row 0
trueRowOffset := 1 * shardWidth // fragment row 1
// For bools which have been set to null, clear both the true and false
// values for the record. Because this ends up going through the
// API.ImportRoaringShard() method (which handles `bool` fields the same as
// `mutex` fields), we don't actually set the true and false rows of the
// boolean fragment; rather, we just set the first row to indicate which
// records (for all rows) to clear.
for fieldname, boolNulls := range b.boolNulls {
field := b.headerMap[fieldname]
if field.Opts().Type() != featurebase.FieldTypeBool {
continue
}
for _, pos := range boolNulls {
recID := b.ids[pos]
shard := recID / shardWidth
clearBM := clearFrags.GetOrCreate(shard, field.Name(), "standard")
fragmentColumn := recID % shardWidth
clearBM.Add(fragmentColumn)
}
}
// For bools which have been set to a non-nil value, set the appropriate
// value for the record, and unset the opposing values. For example, if the
// bool is set to `false`, then set the bit in the "false" row, and clear
// the bit in the "true" row.
for fieldname, boolMap := range b.boolValues {
field := b.headerMap[fieldname]
if field.Opts().Type() != featurebase.FieldTypeBool {
continue
}
for pos, boolVal := range boolMap {
recID := b.ids[pos]
shard := recID / shardWidth
bitmap := frags.GetOrCreate(shard, field.Name(), "standard")
clearBM := clearFrags.GetOrCreate(shard, field.Name(), "standard")
fragmentColumn := recID % shardWidth
clearBM.Add(fragmentColumn)
if boolVal {
bitmap.Add(trueRowOffset + fragmentColumn)
} else {
bitmap.Add(falseRowOffset + fragmentColumn)
}
}
}
return frags, clearFrags, nil
}
@ -1700,6 +1784,14 @@ func (b *Batch) reset() {
delete(clearMap, k)
}
}
for _, boolsMap := range b.boolValues {
for k := range boolsMap {
delete(boolsMap, k)
}
}
for k := range b.boolNulls {
delete(b.boolNulls, k) // TODO pool these slices
}
for i := range b.toTranslateID {
b.toTranslateID[i] = ""
}

43
client/batch_test.go
View file

@ -33,6 +33,7 @@ func TestAgainstCluster(t *testing.T) {
client := NewTestClient(t, c)
t.Run("string-slice-combos", func(t *testing.T) { testStringSliceCombos(t, c, client) })
t.Run("import-batch-ints", func(t *testing.T) { testImportBatchInts(t, c, client) })
t.Run("import-batch-bools", func(t *testing.T) { testImportBatchBools(t, c, client) })
t.Run("import-batch-sorting", func(t *testing.T) { testImportBatchSorting(t, c, client) })
t.Run("test-trim-null", func(t *testing.T) { testTrimNull(t, c, client) })
t.Run("test-string-slice-empty-and-nil", func(t *testing.T) { testStringSliceEmptyAndNil(t, c, client) })
@ -1842,3 +1843,45 @@ func mutexNilClearKey(t *testing.T, c *test.Cluster, client *Client) {
}
errorIfNotEqual(t, resp.Result().Row().Keys, []string{"0"})
}
func testImportBatchBools(t *testing.T, c *test.Cluster, client *Client) {
schema := NewSchema()
idx := schema.Index("test-import-batch-bools")
field := idx.Field("boolcol", OptFieldTypeBool())
err := client.SyncSchema(schema)
if err != nil {
t.Fatalf("syncing schema: %v", err)
}
b, err := NewBatch(client, 3, idx, []*Field{field}, OptUseShardTransactionalEndpoint(true))
if err != nil {
t.Fatalf("getting batch: %v", err)
}
r := Row{Values: make([]interface{}, 1)}
r.ID = uint64(0)
r.Values[0] = bool(false)
err = b.Add(r)
if err != nil {
t.Fatalf("adding after import: %v", err)
}
r.ID = uint64(1)
r.Values[0] = bool(true)
err = b.Add(r)
if err != nil {
t.Fatalf("adding second after import: %v", err)
}
err = b.Import()
if err != nil {
t.Fatalf("second import: %v", err)
}
resp, err := client.Query(idx.RawQuery("Count(All())"))
if err != nil {
t.Fatalf("querying: %v", err)
}
if res := resp.Results()[0]; res.Count() != 2 {
t.Fatalf("unexpected result: %+v", res)
}
}

5
cluster.go
View file

@ -879,7 +879,9 @@ type CreateShardMessage struct {
type CreateIndexMessage struct {
Index string
CreatedAt int64
Meta IndexOptions
Owner string
Meta IndexOptions
}
// DeleteIndexMessage is an internal message indicating index deletion.
@ -892,6 +894,7 @@ type CreateFieldMessage struct {
Index string
Field string
CreatedAt int64
Owner string
Meta *FieldOptions
}

4
dbshard_internal_test.go
View file

@ -188,7 +188,7 @@ func makeSampleRoaringDir(t *testing.T, root, index, backend string, minBytes in
}
func helperCreateDBShard(h *Holder, index string, shard uint64) *Index {
idx, err := h.CreateIndexIfNotExists(index, IndexOptions{})
idx, err := h.CreateIndexIfNotExists(index, "", IndexOptions{})
PanicOn(err)
// TODO: It's not clear that this is actually doing anything.
dbs, err := h.txf.dbPerShard.GetDBShard(index, shard, idx)
@ -253,7 +253,7 @@ func Test_DBPerShard_GetFieldView2Shards_map_from_RBF(t *testing.T) {
index := "rick"
field := "f"
idx, err := holder.CreateIndex(index, IndexOptions{})
idx, err := holder.CreateIndex(index, "", IndexOptions{})
if err != nil {
t.Fatalf("creating index: %v", err)
}

26
disco/disco.go
View file

@ -101,8 +101,8 @@ type Schemator interface {
CreateIndex(ctx context.Context, name string, val []byte) error
DeleteIndex(ctx context.Context, name string) error
Field(ctx context.Context, index, field string) ([]byte, error)
CreateField(ctx context.Context, index, field string, val []byte) error
UpdateField(ctx context.Context, index, field string, val []byte) error
CreateField(ctx context.Context, index, field string, fieldVal []byte) error
UpdateField(ctx context.Context, index, field string, fieldVal []byte) error
DeleteField(ctx context.Context, index, field string) error
View(ctx context.Context, index, field, view string) (bool, error)
CreateView(ctx context.Context, index, field, view string) error
@ -186,23 +186,27 @@ func (*nopSchemator) Index(ctx context.Context, name string) ([]byte, error) { r
func (*nopSchemator) CreateIndex(ctx context.Context, name string, val []byte) error { return nil }
// DeleteIndex is a no-op implementation of the Schemator DeleteIndex method.
func (*nopSchemator) DeleteIndex(ctx context.Context, name string) error { return nil }
func (*nopSchemator) DeleteIndex(ctx context.Context, name string) error {
return nil
}
// Field is a no-op implementation of the Schemator Field method.
func (*nopSchemator) Field(ctx context.Context, index, field string) ([]byte, error) { return nil, nil }
// CreateField is a no-op implementation of the Schemator CreateField method.
func (*nopSchemator) CreateField(ctx context.Context, index, field string, val []byte) error {
func (*nopSchemator) CreateField(ctx context.Context, index, field string, fieldVal []byte) error {
return nil
}
// UpdateField is a no-op implementation of the Schemator UpdateField method.
func (*nopSchemator) UpdateField(ctx context.Context, index, field string, val []byte) error {
func (*nopSchemator) UpdateField(ctx context.Context, index, field string, fieldVal []byte) error {
return nil
}
// DeleteField is a no-op implementation of the Schemator DeleteField method.
func (*nopSchemator) DeleteField(ctx context.Context, index, field string) error { return nil }
func (*nopSchemator) DeleteField(ctx context.Context, index, field string) error {
return nil
}
// View is a no-op implementation of the Schemator View method.
func (*nopSchemator) View(ctx context.Context, index, field, view string) (bool, error) {
@ -296,7 +300,7 @@ func (s *inMemSchemator) Field(ctx context.Context, index, field string) ([]byte
}
// CreateField is an in-memory implementation of the Schemator CreateField method.
func (s *inMemSchemator) CreateField(ctx context.Context, index, field string, val []byte) error {
func (s *inMemSchemator) CreateField(ctx context.Context, index, field string, fieldVal []byte) error {
s.mu.Lock()
defer s.mu.Unlock()
idx, ok := s.schema[index]
@ -307,17 +311,17 @@ func (s *inMemSchemator) CreateField(ctx context.Context, index, field string, v
// The current logic in pilosa doesn't allow us to return ErrFieldExists
// here, so for now we just update the Data value if the field already
// exists.
fld.Data = val
fld.Data = fieldVal
return nil
}
idx.Fields[field] = &Field{
Data: val,
Data: fieldVal,
Views: make(map[string]struct{}),
}
return nil
}
func (s *inMemSchemator) UpdateField(ctx context.Context, index, field string, val []byte) error {
func (s *inMemSchemator) UpdateField(ctx context.Context, index, field string, fieldVal []byte) error {
s.mu.Lock()
defer s.mu.Unlock()
idx, ok := s.schema[index]
@ -328,7 +332,7 @@ func (s *inMemSchemator) UpdateField(ctx context.Context, index, field string, v
// The current logic in pilosa doesn't allow us to return ErrFieldExists
// here, so for now we just update the Data value if the field already
// exists.
fld.Data = val
fld.Data = fieldVal
return nil
} else {
return ErrFieldDoesNotExist

6
encoding/proto/proto.go
View file

@ -681,12 +681,14 @@ func (s Serializer) encodeCreateIndexMessage(m *pilosa.CreateIndexMessage) *pb.C
return &pb.CreateIndexMessage{
Index: m.Index,
CreatedAt: m.CreatedAt,
Owner: m.Owner,
Meta: s.encodeIndexMeta(&m.Meta),
}
}
func (s Serializer) encodeIndexMeta(m *pilosa.IndexOptions) *pb.IndexMeta {
return &pb.IndexMeta{
Description: m.Description,
Keys: m.Keys,
TrackExistence: m.TrackExistence,
}
@ -703,6 +705,7 @@ func (s Serializer) encodeCreateFieldMessage(m *pilosa.CreateFieldMessage) *pb.C
Index: m.Index,
Field: m.Field,
CreatedAt: m.CreatedAt,
Owner: m.Owner,
Meta: s.encodeFieldOptions(m.Meta),
}
}
@ -1057,12 +1060,14 @@ func (s Serializer) decodeCreateShardMessage(pb *pb.CreateShardMessage, m *pilos
func (s Serializer) decodeCreateIndexMessage(pb *pb.CreateIndexMessage, m *pilosa.CreateIndexMessage) {
m.Index = pb.Index
m.CreatedAt = pb.CreatedAt
m.Owner = pb.Owner
m.Meta = pilosa.IndexOptions{}
s.decodeIndexMeta(pb.Meta, &m.Meta)
}
func (s Serializer) decodeIndexMeta(pb *pb.IndexMeta, m *pilosa.IndexOptions) {
if pb != nil {
m.Description = pb.Description
m.Keys = pb.Keys
m.TrackExistence = pb.TrackExistence
}
@ -1076,6 +1081,7 @@ func (s Serializer) decodeCreateFieldMessage(pb *pb.CreateFieldMessage, m *pilos
m.Index = pb.Index
m.Field = pb.Field
m.CreatedAt = pb.CreatedAt
m.Owner = pb.Owner
m.Meta = &pilosa.FieldOptions{}
s.decodeFieldOptions(pb.Meta, m.Meta)
}

35
etcd/embed.go
View file

@ -869,19 +869,20 @@ func (e *Etcd) Field(ctx context.Context, indexName string, name string) ([]byte
return e.getKeyBytes(ctx, key)
}
func (e *Etcd) CreateField(ctx context.Context, indexName string, name string, val []byte) error {
func (e *Etcd) CreateField(ctx context.Context, indexName string, name string, fieldVal []byte) error {
key := schemaPrefix + indexName + "/" + name
// Set up Op to write field value as bytes.
op := clientv3.OpPut(key, "")
op.WithValueBytes(val)
op.WithValueBytes(fieldVal)
// Check for key existence, and execute Op within a transaction.
var resp *clientv3.TxnResponse
err := e.retryClient(func(cli *clientv3.Client) (err error) {
resp, err = cli.Txn(ctx).
If(clientv3util.KeyMissing(key)).
If(
clientv3util.KeyMissing(key)).
Then(op).
Commit()
return err
@ -897,19 +898,20 @@ func (e *Etcd) CreateField(ctx context.Context, indexName string, name string, v
return nil
}
func (e *Etcd) UpdateField(ctx context.Context, indexName string, name string, val []byte) error {
func (e *Etcd) UpdateField(ctx context.Context, indexName string, name string, fieldVal []byte) error {
key := schemaPrefix + indexName + "/" + name
// Set up Op to write field value as bytes.
op := clientv3.OpPut(key, "")
op.WithValueBytes(val)
op.WithValueBytes(fieldVal)
// Check for key existence, and execute Op within a transaction.
var resp *clientv3.TxnResponse
err := e.retryClient(func(cli *clientv3.Client) (err error) {
resp, err = cli.Txn(ctx).
If(clientv3util.KeyExists(key)).
If(
clientv3util.KeyExists(key)).
Then(op).
Commit()
return err
@ -924,20 +926,31 @@ func (e *Etcd) UpdateField(ctx context.Context, indexName string, name string, v
return nil
}
func (e *Etcd) DeleteField(ctx context.Context, indexname string, name string) (err error) {
key := schemaPrefix + indexname + "/" + name
func (e *Etcd) DeleteField(ctx context.Context, indexName string, name string) (err error) {
key := schemaPrefix + indexName + "/" + name
var resp *clientv3.TxnResponse
// Deleting field and views in one transaction.
err = e.retryClient(func(cli *clientv3.Client) (err error) {
_, err = cli.Txn(ctx).
If(clientv3.Compare(clientv3.Version(key), ">", -1)).
resp, err = cli.Txn(ctx).
If(
clientv3.Compare(clientv3.Version(key), ">", -1)).
Then(
clientv3.OpDelete(key+"/", clientv3.WithPrefix()), // deleting field views
clientv3.OpDelete(key), // deleting field
).Commit()
return err
})
if err != nil {
return errors.Wrap(err, "executing transaction")
}
return errors.Wrap(err, "DeleteField")
if !resp.Succeeded {
return errors.New("deleting field from etcd failed")
}
return nil
}
func (e *Etcd) View(ctx context.Context, indexName, fieldName, name string) (bool, error) {

2
executor.go
View file

@ -6547,7 +6547,7 @@ func (e *executor) collectCallKeys(dst *keyCollector, c *pql.Call, index string)
if keyed {
opts = append(opts, OptFieldKeys())
}
if _, err := idx.CreateField(field, opts...); err != nil {
if _, err := idx.CreateField(field, "", opts...); err != nil {
// We wrap these because we want to indicate that it wasn't found,
// but also the problem we encountered trying to create it.
return newNotFoundError(errors.Wrap(err, "creating field"), field)

10
executor_internal_test.go
View file

@ -47,7 +47,7 @@ func TestExecutor_TranslateRowsOnBool(t *testing.T) {
Cluster: NewTestCluster(t, 1),
}
idx, err := e.Holder.CreateIndex("i", IndexOptions{})
idx, err := e.Holder.CreateIndex("i", "", IndexOptions{})
if err != nil {
t.Fatalf("creating index: %v", err)
}
@ -55,8 +55,8 @@ func TestExecutor_TranslateRowsOnBool(t *testing.T) {
qcx := holder.Txf().NewWritableQcx()
defer qcx.Abort()
fb, errb := idx.CreateField("b", OptFieldTypeBool())
_, errbk := idx.CreateField("bk", OptFieldTypeBool(), OptFieldKeys())
fb, errb := idx.CreateField("b", "", OptFieldTypeBool())
_, errbk := idx.CreateField("bk", "", OptFieldTypeBool(), OptFieldKeys())
if errb != nil || errbk != nil {
t.Fatalf("creating fields %v, %v", errb, errbk)
}
@ -563,12 +563,12 @@ func TestDistinctTimestampUnion(t *testing.T) {
func TestExecutor_DeleteRows(t *testing.T) {
holder := newTestHolder(t)
idx, err := holder.CreateIndex("i", IndexOptions{TrackExistence: true})
idx, err := holder.CreateIndex("i", "", IndexOptions{TrackExistence: true})
if err != nil {
t.Fatalf("creating index: %v", err)
}
f, err := idx.CreateField("f")
f, err := idx.CreateField("f", "")
if err != nil {
t.Fatalf("creating field: %v", err)
}

196
executor_test.go
View file

@ -1155,11 +1155,11 @@ func runCallTest(c *test.Cluster, t *testing.T, writeQuery string, readQueries [
}
hldr := c.GetHolder(0)
index, err := hldr.CreateIndex(indexName, *indexOptions)
index, err := hldr.CreateIndex(indexName, "", *indexOptions)
if err != nil {
t.Fatal(err)
}
_, err = index.CreateField("f", fieldOption...)
_, err = index.CreateField("f", "", fieldOption...)
if err != nil {
t.Fatal(err)
}
@ -1445,7 +1445,7 @@ func TestExecutor_Execute_Set(t *testing.T) {
t.Fatal(err)
}
if _, err := idx.CreateField("f"); err != nil {
if _, err := idx.CreateField("f", ""); err != nil {
t.Fatal(err)
}
@ -1462,7 +1462,7 @@ func TestExecutor_Execute_Set(t *testing.T) {
t.Run("ErrInvalidRowValueType", func(t *testing.T) {
idx := hldr.MustCreateIndexIfNotExists(c.Idx("inokey"), pilosa.IndexOptions{})
if _, err := idx.CreateField("f", pilosa.OptFieldKeys()); err != nil {
if _, err := idx.CreateField("f", "", pilosa.OptFieldKeys()); err != nil {
t.Fatal(err)
}
if _, err := cmd.API.Query(context.Background(), &pilosa.QueryRequest{Index: c.Idx("inokey"), Query: `Set(2, f=1.2)`}); err == nil || !strings.Contains(err.Error(), "invalid value") {
@ -1488,7 +1488,7 @@ func TestExecutor_Execute_SetBool(t *testing.T) {
// Create fields.
index := hldr.MustCreateIndexIfNotExists(c.Idx(), pilosa.IndexOptions{})
if _, err := index.CreateFieldIfNotExists("f", pilosa.OptFieldTypeBool()); err != nil {
if _, err := index.CreateFieldIfNotExists("f", "", pilosa.OptFieldTypeBool()); err != nil {
t.Fatal(err)
}
@ -1534,7 +1534,7 @@ func TestExecutor_Execute_SetBool(t *testing.T) {
// Create fields.
index := hldr.MustCreateIndexIfNotExists(c.Idx(), pilosa.IndexOptions{})
if _, err := index.CreateFieldIfNotExists("f", pilosa.OptFieldTypeBool()); err != nil {
if _, err := index.CreateFieldIfNotExists("f", "", pilosa.OptFieldTypeBool()); err != nil {
t.Fatal(err)
}
@ -1560,7 +1560,7 @@ func TestExecutor_Execute_SetDecimal(t *testing.T) {
// Create fields.
index := hldr.MustCreateIndexIfNotExists(c.Idx(), pilosa.IndexOptions{})
if _, err := index.CreateFieldIfNotExists("f", pilosa.OptFieldTypeDecimal(2)); err != nil {
if _, err := index.CreateFieldIfNotExists("f", "", pilosa.OptFieldTypeDecimal(2)); err != nil {
t.Fatal(err)
}
@ -1597,7 +1597,7 @@ func TestExecutor_Execute_SetDecimal(t *testing.T) {
// Create fields.
index := hldr.MustCreateIndexIfNotExists(c.Idx(), pilosa.IndexOptions{})
if _, err := index.CreateFieldIfNotExists("f", pilosa.OptFieldTypeDecimal(2)); err != nil {
if _, err := index.CreateFieldIfNotExists("f", "", pilosa.OptFieldTypeDecimal(2)); err != nil {
t.Fatal(err)
}
@ -1631,9 +1631,9 @@ func TestExecutor_Execute_SetValue(t *testing.T) {
// Create fields.
index := hldr.MustCreateIndexIfNotExists(c.Idx(), pilosa.IndexOptions{})
if _, err := index.CreateFieldIfNotExists("f", pilosa.OptFieldTypeInt(math.MinInt64, math.MaxInt64)); err != nil {
if _, err := index.CreateFieldIfNotExists("f", "", pilosa.OptFieldTypeInt(math.MinInt64, math.MaxInt64)); err != nil {
t.Fatal(err)
} else if _, err := index.CreateFieldIfNotExists("xxx"); err != nil {
} else if _, err := index.CreateFieldIfNotExists("xxx", ""); err != nil {
t.Fatal(err)
}
@ -1672,7 +1672,7 @@ func TestExecutor_Execute_SetValue(t *testing.T) {
hldr := c.GetHolder(0)
index := hldr.MustCreateIndexIfNotExists(c.Idx(), pilosa.IndexOptions{})
if _, err := index.CreateFieldIfNotExists("f", pilosa.OptFieldTypeInt(math.MinInt64, math.MaxInt64)); err != nil {
if _, err := index.CreateFieldIfNotExists("f", "", pilosa.OptFieldTypeInt(math.MinInt64, math.MaxInt64)); err != nil {
t.Fatal(err)
}
@ -1702,9 +1702,9 @@ func TestExecutor_Execute_SetValue(t *testing.T) {
// Create fields.
index := hldr.MustCreateIndexIfNotExists(c.Idx(), pilosa.IndexOptions{})
if _, err := index.CreateFieldIfNotExists("f", pilosa.OptFieldTypeTimestamp(pilosa.DefaultEpoch, pilosa.TimeUnitSeconds)); err != nil {
if _, err := index.CreateFieldIfNotExists("f", "", pilosa.OptFieldTypeTimestamp(pilosa.DefaultEpoch, pilosa.TimeUnitSeconds)); err != nil {
t.Fatal(err)
} else if _, err := index.CreateFieldIfNotExists("xxx"); err != nil {
} else if _, err := index.CreateFieldIfNotExists("xxx", ""); err != nil {
t.Fatal(err)
}
@ -1841,11 +1841,11 @@ func TestExecutor_Execute_TopN(t *testing.T) {
hldr := c.GetHolder(0)
// Set columns for rows 0, 10, & 20 across two shards.
if idx, err := hldr.CreateIndex(c.Idx(), pilosa.IndexOptions{}); err != nil {
if idx, err := hldr.CreateIndex(c.Idx(), "", pilosa.IndexOptions{}); err != nil {
t.Fatal(err)
} else if _, err := idx.CreateField("f"); err != nil {
} else if _, err := idx.CreateField("f", ""); err != nil {
t.Fatal(err)
} else if _, err := idx.CreateField("other"); err != nil {
} else if _, err := idx.CreateField("other", ""); err != nil {
t.Fatal(err)
} else if _, err := c.GetNode(0).API.Query(context.Background(), &pilosa.QueryRequest{Index: c.Idx(), Query: `
Set(0, f=0)
@ -1885,11 +1885,11 @@ func TestExecutor_Execute_TopN(t *testing.T) {
hldr := c.GetHolder(0)
// Set columns for rows 0, 10, & 20 across two shards.
if idx, err := hldr.CreateIndex(c.Idx(), pilosa.IndexOptions{Keys: true}); err != nil {
if idx, err := hldr.CreateIndex(c.Idx(), "", pilosa.IndexOptions{Keys: true}); err != nil {
t.Fatal(err)
} else if _, err := idx.CreateField("f"); err != nil {
} else if _, err := idx.CreateField("f", ""); err != nil {
t.Fatal(err)
} else if _, err := idx.CreateField("other"); err != nil {
} else if _, err := idx.CreateField("other", ""); err != nil {
t.Fatal(err)
} else if _, err := c.GetNode(0).API.Query(context.Background(), &pilosa.QueryRequest{Index: c.Idx(), Query: `
Set("zero", f=0)
@ -1929,11 +1929,11 @@ func TestExecutor_Execute_TopN(t *testing.T) {
hldr := c.GetHolder(0)
// Set columns for rows 0, 10, & 20 across two shards.
if idx, err := hldr.CreateIndex(c.Idx(), pilosa.IndexOptions{Keys: true}); err != nil {
if idx, err := hldr.CreateIndex(c.Idx(), "", pilosa.IndexOptions{Keys: true}); err != nil {
t.Fatal(err)
} else if _, err := idx.CreateField("f", pilosa.OptFieldKeys()); err != nil {
} else if _, err := idx.CreateField("f", "", pilosa.OptFieldKeys()); err != nil {
t.Fatal(err)
} else if _, err := idx.CreateField("other", pilosa.OptFieldKeys()); err != nil {
} else if _, err := idx.CreateField("other", "", pilosa.OptFieldKeys()); err != nil {
t.Fatal(err)
} else if _, err := c.GetNode(0).API.Query(context.Background(), &pilosa.QueryRequest{Index: c.Idx(), Query: `
Set("zero", f="zero")
@ -1975,11 +1975,11 @@ func TestExecutor_Execute_TopN(t *testing.T) {
hldr := c.GetHolder(0)
// Set columns for rows 0, 10, & 20 across two shards.
if idx, err := hldr.CreateIndex(c.Idx(), pilosa.IndexOptions{Keys: true}); err != nil {
if idx, err := hldr.CreateIndex(c.Idx(), "", pilosa.IndexOptions{Keys: true}); err != nil {
t.Fatal(err)
} else if _, err := idx.CreateField("f", pilosa.OptFieldKeys()); err != nil {
} else if _, err := idx.CreateField("f", "", pilosa.OptFieldKeys()); err != nil {
t.Fatal(err)
} else if _, err := idx.CreateField("other", pilosa.OptFieldKeys()); err != nil {
} else if _, err := idx.CreateField("other", "", pilosa.OptFieldKeys()); err != nil {
t.Fatal(err)
} else if _, err := c.GetNode(0).API.Query(context.Background(), &pilosa.QueryRequest{Index: c.Idx(), Query: `
Set("a", f="foo")
@ -2021,9 +2021,9 @@ func TestExecutor_Execute_TopN(t *testing.T) {
hldr := c.GetHolder(0)
// Set data on the "f" field.
if idx, err := hldr.CreateIndex(c.Idx(), pilosa.IndexOptions{}); err != nil {
if idx, err := hldr.CreateIndex(c.Idx(), "", pilosa.IndexOptions{}); err != nil {
t.Fatal(err)
} else if _, err := idx.CreateField("f"); err != nil {
} else if _, err := idx.CreateField("f", ""); err != nil {
t.Fatal(err)
} else if _, err := c.GetNode(0).API.Query(context.Background(), &pilosa.QueryRequest{Index: c.Idx(), Query: `
Set(0, f=0)
@ -2046,9 +2046,9 @@ func TestExecutor_Execute_TopN(t *testing.T) {
hldr := c.GetHolder(0)
// Create BSI "f" field.
if idx, err := hldr.CreateIndex(c.Idx(), pilosa.IndexOptions{}); err != nil {
if idx, err := hldr.CreateIndex(c.Idx(), "", pilosa.IndexOptions{}); err != nil {
t.Fatal(err)
} else if _, err := idx.CreateField("f", pilosa.OptFieldTypeInt(0, 100)); err != nil {
} else if _, err := idx.CreateField("f", "", pilosa.OptFieldTypeInt(0, 100)); err != nil {
t.Fatal(err)
} else if _, err := c.GetNode(0).API.Query(context.Background(), &pilosa.QueryRequest{Index: c.Idx(), Query: `TopN(f, n=2)`}); err == nil || !strings.Contains(err.Error(), `finding top results: mapping on primary node: cannot compute TopN() on integer, decimal, or timestamp field: "f"`) {
t.Fatalf("unexpected error: %v", err)
@ -2060,9 +2060,9 @@ func TestExecutor_Execute_TopN(t *testing.T) {
defer c.Close()
hldr := c.GetHolder(0)
if idx, err := hldr.CreateIndex(c.Idx(), pilosa.IndexOptions{}); err != nil {
if idx, err := hldr.CreateIndex(c.Idx(), "", pilosa.IndexOptions{}); err != nil {
t.Fatal(err)
} else if _, err := idx.CreateField("f", pilosa.OptFieldTypeSet(pilosa.CacheTypeNone, 0)); err != nil {
} else if _, err := idx.CreateField("f", "", pilosa.OptFieldTypeSet(pilosa.CacheTypeNone, 0)); err != nil {
t.Fatal(err)
} else if _, err := c.GetNode(0).API.Query(context.Background(), &pilosa.QueryRequest{Index: c.Idx(), Query: `
Set(0, f=0)
@ -2187,7 +2187,7 @@ func TestExecutor_Execute_MinMax(t *testing.T) {
defer c.Close()
hldr := c.GetHolder(0)
idx, err := hldr.CreateIndex(c.Idx(), pilosa.IndexOptions{})
idx, err := hldr.CreateIndex(c.Idx(), "", pilosa.IndexOptions{})
if err != nil {
t.Fatal(err)
}
@ -2205,7 +2205,7 @@ func TestExecutor_Execute_MinMax(t *testing.T) {
for i, test := range tests {
fld := fmt.Sprintf("f%d", i)
t.Run("MinMaxField_"+fld, func(t *testing.T) {
if _, err := idx.CreateField(fld, pilosa.OptFieldTypeInt(test.min, test.max)); err != nil {
if _, err := idx.CreateField(fld, "", pilosa.OptFieldTypeInt(test.min, test.max)); err != nil {
t.Fatal(err)
}
@ -2279,7 +2279,7 @@ func TestExecutor_Execute_MinMax(t *testing.T) {
defer c.Close()
hldr := c.GetHolder(0)
idx, err := hldr.CreateIndex(c.Idx(), pilosa.IndexOptions{})
idx, err := hldr.CreateIndex(c.Idx(), "", pilosa.IndexOptions{})
if err != nil {
t.Fatal(err)
}
@ -2323,7 +2323,7 @@ func TestExecutor_Execute_MinMax(t *testing.T) {
// This extra field exists to make there be shards which are present,
// but have no decimal values set, to make sure they don't break
// the results.
if _, err := idx.CreateFieldIfNotExists("z"); err != nil {
if _, err := idx.CreateFieldIfNotExists("z", ""); err != nil {
t.Fatal(err)
}
if res, err := c.GetNode(0).API.Query(context.Background(), &pilosa.QueryRequest{Index: c.Idx(), Query: `Set(1, z=0)`}); err != nil {
@ -2355,7 +2355,7 @@ func TestExecutor_Execute_MinMax(t *testing.T) {
for i, test := range tests {
fld := fmt.Sprintf("f%d", i)
t.Run("MinMaxField_"+fld, func(t *testing.T) {
if _, err := idx.CreateField(fld, pilosa.OptFieldTypeDecimal(test.scale, test.min, test.max)); err != nil {
if _, err := idx.CreateField(fld, "", pilosa.OptFieldTypeDecimal(test.scale, test.min, test.max)); err != nil {
t.Fatal(err)
}
@ -2411,7 +2411,7 @@ func TestExecutor_Execute_MinMax(t *testing.T) {
defer c.Close()
hldr := c.GetHolder(0)
idx, err := hldr.CreateIndex(c.Idx(), pilosa.IndexOptions{})
idx, err := hldr.CreateIndex(c.Idx(), "", pilosa.IndexOptions{})
if err != nil {
t.Fatal(err)
}
@ -2428,7 +2428,7 @@ func TestExecutor_Execute_MinMax(t *testing.T) {
for i, test := range tests {
fld := fmt.Sprintf("f%d", i)
t.Run("MinMaxField_"+fld, func(t *testing.T) {
if _, err := idx.CreateField(fld, pilosa.OptFieldTypeTimestamp(test.epoch, pilosa.TimeUnitSeconds)); err != nil {
if _, err := idx.CreateField(fld, "", pilosa.OptFieldTypeTimestamp(test.epoch, pilosa.TimeUnitSeconds)); err != nil {
t.Fatal(err)
} else if _, err := c.GetNode(0).API.Query(context.Background(), &pilosa.QueryRequest{Index: c.Idx(), Query: fmt.Sprintf(`Set(10, %s="%s")`, fld, test.set.Format(time.RFC3339))}); err != nil {
t.Fatal(err)
@ -2499,16 +2499,16 @@ func TestExecutor_Execute_MinMax(t *testing.T) {
defer c.Close()
hldr := c.GetHolder(0)
idx, err := hldr.CreateIndex(c.Idx(), pilosa.IndexOptions{})
idx, err := hldr.CreateIndex(c.Idx(), "", pilosa.IndexOptions{})
if err != nil {
t.Fatal(err)
}
if _, err := idx.CreateField("x"); err != nil {
if _, err := idx.CreateField("x", ""); err != nil {
t.Fatal(err)
}
if _, err := idx.CreateField("f", pilosa.OptFieldTypeInt(-1100, 1000)); err != nil {
if _, err := idx.CreateField("f", "", pilosa.OptFieldTypeInt(-1100, 1000)); err != nil {
t.Fatal(err)
}
@ -2563,16 +2563,16 @@ func TestExecutor_Execute_MinMax(t *testing.T) {
defer c.Close()
hldr := c.GetHolder(0)
idx, err := hldr.CreateIndex(c.Idx(), pilosa.IndexOptions{Keys: true})
idx, err := hldr.CreateIndex(c.Idx(), "", pilosa.IndexOptions{Keys: true})
if err != nil {
t.Fatal(err)
}
if _, err := idx.CreateField("x"); err != nil {
if _, err := idx.CreateField("x", ""); err != nil {
t.Fatal(err)
}
if _, err := idx.CreateField("f", pilosa.OptFieldTypeInt(-1110, 1000)); err != nil {
if _, err := idx.CreateField("f", "", pilosa.OptFieldTypeInt(-1110, 1000)); err != nil {
t.Fatal(err)
}
@ -2656,12 +2656,12 @@ func TestExecutor_Execute_MinMaxRow(t *testing.T) {
defer c.Close()
hldr := c.GetHolder(0)
idx, err := hldr.CreateIndex(c.Idx(), pilosa.IndexOptions{})
idx, err := hldr.CreateIndex(c.Idx(), "", pilosa.IndexOptions{})
if err != nil {
t.Fatal(err)
}
if _, err := idx.CreateField("f"); err != nil {
if _, err := idx.CreateField("f", ""); err != nil {
t.Fatal(err)
}
@ -2720,12 +2720,12 @@ func TestExecutor_Execute_MinMaxRow(t *testing.T) {
defer c.Close()
hldr := c.GetHolder(0)
idx, err := hldr.CreateIndex(c.Idx(), pilosa.IndexOptions{})
idx, err := hldr.CreateIndex(c.Idx(), "", pilosa.IndexOptions{})
if err != nil {
t.Fatal(err)
}
if _, err := idx.CreateField("f", pilosa.OptFieldKeys()); err != nil {
if _, err := idx.CreateField("f", "", pilosa.OptFieldKeys()); err != nil {
t.Fatal(err)
}
@ -2776,28 +2776,28 @@ func TestExecutor_Execute_Sum(t *testing.T) {
defer c.Close()
hldr := c.GetHolder(0)
idx, err := hldr.CreateIndex(c.Idx(), pilosa.IndexOptions{})
idx, err := hldr.CreateIndex(c.Idx(), "", pilosa.IndexOptions{})
if err != nil {
t.Fatal(err)
}
if _, err := idx.CreateField("x"); err != nil {
if _, err := idx.CreateField("x", ""); err != nil {
t.Fatal(err)
}
if _, err := idx.CreateField("foo", pilosa.OptFieldTypeInt(-990, 1000)); err != nil {
if _, err := idx.CreateField("foo", "", pilosa.OptFieldTypeInt(-990, 1000)); err != nil {
t.Fatal(err)
}
if _, err := idx.CreateField("bar", pilosa.OptFieldTypeInt(math.MinInt64, math.MaxInt64)); err != nil {
if _, err := idx.CreateField("bar", "", pilosa.OptFieldTypeInt(math.MinInt64, math.MaxInt64)); err != nil {
t.Fatal(err)
}
if _, err := idx.CreateField("other", pilosa.OptFieldTypeInt(math.MinInt64, math.MaxInt64)); err != nil {
if _, err := idx.CreateField("other", "", pilosa.OptFieldTypeInt(math.MinInt64, math.MaxInt64)); err != nil {
t.Fatal(err)
}
if _, err := idx.CreateField("dec", pilosa.OptFieldTypeDecimal(3)); err != nil {
if _, err := idx.CreateField("dec", "", pilosa.OptFieldTypeDecimal(3)); err != nil {
t.Fatal(err)
}
@ -2910,24 +2910,24 @@ func TestExecutor_Execute_Sum(t *testing.T) {
defer c.Close()
hldr := c.GetHolder(0)
idx, err := hldr.CreateIndex(c.Idx(), pilosa.IndexOptions{Keys: true})
idx, err := hldr.CreateIndex(c.Idx(), "", pilosa.IndexOptions{Keys: true})
if err != nil {
t.Fatal(err)
}
if _, err := idx.CreateField("x"); err != nil {
if _, err := idx.CreateField("x", ""); err != nil {
t.Fatal(err)
}
if _, err := idx.CreateField("foo", pilosa.OptFieldTypeInt(-990, 1000)); err != nil {
if _, err := idx.CreateField("foo", "", pilosa.OptFieldTypeInt(-990, 1000)); err != nil {
t.Fatal(err)
}
if _, err := idx.CreateField("bar", pilosa.OptFieldTypeInt(math.MinInt64, math.MaxInt64)); err != nil {
if _, err := idx.CreateField("bar", "", pilosa.OptFieldTypeInt(math.MinInt64, math.MaxInt64)); err != nil {
t.Fatal(err)
}
if _, err := idx.CreateField("other", pilosa.OptFieldTypeInt(math.MinInt64, math.MaxInt64)); err != nil {
if _, err := idx.CreateField("other", "", pilosa.OptFieldTypeInt(math.MinInt64, math.MaxInt64)); err != nil {
t.Fatal(err)
}
@ -2970,7 +2970,7 @@ func TestExecutor_DecimalArgs(t *testing.T) {
defer c.Close()
hldr := c.GetHolder(0)
idx, err := hldr.CreateIndex(c.Idx(), pilosa.IndexOptions{})
idx, err := hldr.CreateIndex(c.Idx(), "", pilosa.IndexOptions{})
if err != nil {
t.Fatal(err)
}
@ -2984,7 +2984,7 @@ func TestExecutor_DecimalArgs(t *testing.T) {
t.Fatal(err)
}
if _, err := idx.CreateField("f", pilosa.OptFieldTypeDecimal(2, min, max)); err != nil {
if _, err := idx.CreateField("f", "", pilosa.OptFieldTypeDecimal(2, min, max)); err != nil {
t.Fatal(err)
}
@ -3001,28 +3001,28 @@ func TestExecutor_Execute_Row_BSIGroup(t *testing.T) {
defer c.Close()
hldr := c.GetHolder(0)
idx, err := hldr.CreateIndex(c.Idx(), pilosa.IndexOptions{TrackExistence: true})
idx, err := hldr.CreateIndex(c.Idx(), "", pilosa.IndexOptions{TrackExistence: true})
if err != nil {
t.Fatal(err)
}
if _, err := idx.CreateField("f"); err != nil {
if _, err := idx.CreateField("f", ""); err != nil {
t.Fatal(err)
}
if _, err := idx.CreateField("foo", pilosa.OptFieldTypeInt(-990, 1000)); err != nil {
if _, err := idx.CreateField("foo", "", pilosa.OptFieldTypeInt(-990, 1000)); err != nil {
t.Fatal(err)
}
if _, err := idx.CreateField("bar", pilosa.OptFieldTypeInt(math.MinInt64, math.MaxInt64)); err != nil {
if _, err := idx.CreateField("bar", "", pilosa.OptFieldTypeInt(math.MinInt64, math.MaxInt64)); err != nil {
t.Fatal(err)
}
if _, err := idx.CreateField("other", pilosa.OptFieldTypeInt(math.MinInt64, math.MaxInt64)); err != nil {
if _, err := idx.CreateField("other", "", pilosa.OptFieldTypeInt(math.MinInt64, math.MaxInt64)); err != nil {
t.Fatal(err)
}
if _, err := idx.CreateField("edge", pilosa.OptFieldTypeInt(-900, 1000)); err != nil {
if _, err := idx.CreateField("edge", "", pilosa.OptFieldTypeInt(-900, 1000)); err != nil {
t.Fatal(err)
}
@ -3214,13 +3214,13 @@ func TestExecutor_Execute_Row_BSIGroupEdge(t *testing.T) {
defer c.Close()
hldr := c.GetHolder(0)
idx, err := hldr.CreateIndex(c.Idx(), pilosa.IndexOptions{})
idx, err := hldr.CreateIndex(c.Idx(), "", pilosa.IndexOptions{})
if err != nil {
t.Fatal(err)
}
t.Run("LT", func(t *testing.T) {
if _, err := idx.CreateField("f1", pilosa.OptFieldTypeInt(-2000, 2000)); err != nil {
if _, err := idx.CreateField("f1", "", pilosa.OptFieldTypeInt(-2000, 2000)); err != nil {
t.Fatal(err)
}
@ -3241,7 +3241,7 @@ func TestExecutor_Execute_Row_BSIGroupEdge(t *testing.T) {
})
t.Run("GT", func(t *testing.T) {
if _, err := idx.CreateField("f2", pilosa.OptFieldTypeInt(-2000, 2000)); err != nil {
if _, err := idx.CreateField("f2", "", pilosa.OptFieldTypeInt(-2000, 2000)); err != nil {
t.Fatal(err)
}
@ -3262,7 +3262,7 @@ func TestExecutor_Execute_Row_BSIGroupEdge(t *testing.T) {
})
t.Run("BTWN_LT_LT", func(t *testing.T) {
if _, err := idx.CreateField("f3", pilosa.OptFieldTypeInt(-2000, 2000)); err != nil {
if _, err := idx.CreateField("f3", "", pilosa.OptFieldTypeInt(-2000, 2000)); err != nil {
t.Fatal(err)
}
@ -3301,28 +3301,28 @@ func TestExecutor_Execute_Range_BSIGroup_Deprecated(t *testing.T) {
defer c.Close()
hldr := c.GetHolder(0)
idx, err := hldr.CreateIndex(c.Idx(), pilosa.IndexOptions{})
idx, err := hldr.CreateIndex(c.Idx(), "", pilosa.IndexOptions{})
if err != nil {
t.Fatal(err)
}
if _, err := idx.CreateField("f"); err != nil {
if _, err := idx.CreateField("f", ""); err != nil {
t.Fatal(err)
}
if _, err := idx.CreateField("foo", pilosa.OptFieldTypeInt(-990, 1000)); err != nil {
if _, err := idx.CreateField("foo", "", pilosa.OptFieldTypeInt(-990, 1000)); err != nil {
t.Fatal(err)
}
if _, err := idx.CreateField("bar", pilosa.OptFieldTypeInt(math.MinInt64, math.MaxInt64)); err != nil {
if _, err := idx.CreateField("bar", "", pilosa.OptFieldTypeInt(math.MinInt64, math.MaxInt64)); err != nil {
t.Fatal(err)
}
if _, err := idx.CreateField("other", pilosa.OptFieldTypeInt(math.MinInt64, math.MaxInt64)); err != nil {
if _, err := idx.CreateField("other", "", pilosa.OptFieldTypeInt(math.MinInt64, math.MaxInt64)); err != nil {
t.Fatal(err)
}
if _, err := idx.CreateField("edge", pilosa.OptFieldTypeInt(-1100, 1000)); err != nil {
if _, err := idx.CreateField("edge", "", pilosa.OptFieldTypeInt(-1100, 1000)); err != nil {
t.Fatal(err)
}
@ -3849,7 +3849,7 @@ func TestExecutor_Time_Clear_Quantums(t *testing.T) {
indexName := c.Idx(strings.ToLower(string(tt.quantum)))
index := hldr.MustCreateIndexIfNotExists(indexName, pilosa.IndexOptions{})
// Create field.
if _, err := index.CreateFieldIfNotExists("f", pilosa.OptFieldTypeTime(tt.quantum, "0")); err != nil {
if _, err := index.CreateFieldIfNotExists("f", "", pilosa.OptFieldTypeTime(tt.quantum, "0")); err != nil {
t.Fatal(err)
}
// Populate
@ -3933,7 +3933,7 @@ func TestExecutor_Execute_Existence(t *testing.T) {
hldr := c.GetHolder(0)
index := hldr.MustCreateIndexIfNotExists(c.Idx(), pilosa.IndexOptions{TrackExistence: true})
_, err := index.CreateField("f")
_, err := index.CreateField("f", "")
if err != nil {
t.Fatal(err)
}
@ -4335,7 +4335,7 @@ func TestExecutor_Execute_All(t *testing.T) {
defer c.Close()
hldr := c.GetHolder(0)
index := hldr.MustCreateIndexIfNotExists(c.Idx(), pilosa.IndexOptions{TrackExistence: true})
fld, err := index.CreateField("f")
fld, err := index.CreateField("f", "")
if err != nil {
t.Fatal(err)
}
@ -4427,7 +4427,7 @@ func TestExecutor_Execute_All(t *testing.T) {
defer c.Close()
hldr := c.GetHolder(0)
index := hldr.MustCreateIndexIfNotExists(c.Idx(), pilosa.IndexOptions{TrackExistence: true, Keys: true})
fld, err := index.CreateField("f")
fld, err := index.CreateField("f", "")
if err != nil {
t.Fatal(err)
}
@ -4492,7 +4492,7 @@ func TestExecutor_Execute_All(t *testing.T) {
defer c.Close()
hldr := c.GetHolder(0)
index := hldr.MustCreateIndexIfNotExists(c.Idx(), pilosa.IndexOptions{TrackExistence: true})
_, err := index.CreateField("f")
_, err := index.CreateField("f", "")
if err != nil {
t.Fatal(err)
}
@ -4520,7 +4520,7 @@ func TestExecutor_Execute_ClearRow(t *testing.T) {
defer c.Close()
hldr := c.GetHolder(0)
index := hldr.MustCreateIndexIfNotExists(c.Idx(), pilosa.IndexOptions{TrackExistence: true})
_, err := index.CreateField("f", pilosa.OptFieldTypeInt(math.MinInt64, math.MaxInt64))
_, err := index.CreateField("f", "", pilosa.OptFieldTypeInt(math.MinInt64, math.MaxInt64))
if err != nil {
t.Fatal(err)
}
@ -4536,7 +4536,7 @@ func TestExecutor_Execute_ClearRow(t *testing.T) {
defer c.Close()
hldr := c.GetHolder(0)
index := hldr.MustCreateIndexIfNotExists(c.Idx(), pilosa.IndexOptions{TrackExistence: true})
_, err := index.CreateField("f")
_, err := index.CreateField("f", "")
if err != nil {
t.Fatal(err)
}
@ -4616,10 +4616,10 @@ func TestExecutor_Execute_SetRow(t *testing.T) {
defer c.Close()
hldr := c.GetHolder(0)
index := hldr.MustCreateIndexIfNotExists(c.Idx(), pilosa.IndexOptions{TrackExistence: true})
if _, err := index.CreateField("f"); err != nil {
if _, err := index.CreateField("f", ""); err != nil {
t.Fatal(err)
}
if _, err := index.CreateField("tmp"); err != nil {
if _, err := index.CreateField("tmp", ""); err != nil {
t.Fatal(err)
}
@ -4671,7 +4671,7 @@ func TestExecutor_Execute_SetRow(t *testing.T) {
defer c.Close()
hldr := c.GetHolder(0)
idx := hldr.MustCreateIndexIfNotExists(c.Idx(), pilosa.IndexOptions{TrackExistence: true})
_, err := idx.CreateField("f")
_, err := idx.CreateField("f", "")
if err != nil {
t.Fatal(err)
}
@ -4724,7 +4724,7 @@ func TestExecutor_Execute_SetRow(t *testing.T) {
defer c.Close()
hldr := c.GetHolder(0)
index := hldr.MustCreateIndexIfNotExists(c.Idx(), pilosa.IndexOptions{TrackExistence: true})
_, err := index.CreateField("f")
_, err := index.CreateField("f", "")
if err != nil {
t.Fatal(err)
}
@ -4765,7 +4765,7 @@ func TestExecutor_Execute_SetRow(t *testing.T) {
defer c.Close()
hldr := c.GetHolder(0)
index := hldr.MustCreateIndexIfNotExists(c.Idx(), pilosa.IndexOptions{TrackExistence: true})
if _, err := index.CreateField("f", pilosa.OptFieldKeys()); err != nil {
if _, err := index.CreateField("f", "", pilosa.OptFieldKeys()); err != nil {
t.Fatal(err)
}
@ -4829,7 +4829,7 @@ func benchmarkExistence(nn bool, b *testing.B) {
index := hldr.MustCreateIndexIfNotExists(indexName, pilosa.IndexOptions{TrackExistence: nn})
// Create field.
if _, err := index.CreateFieldIfNotExists(fieldName); err != nil {
if _, err := index.CreateFieldIfNotExists(fieldName, ""); err != nil {
b.Fatal(err)
}
@ -6630,7 +6630,7 @@ func TestExecutor_Execute_IncludesColumn(t *testing.T) {
cmd := c.GetNode(0)
hldr := c.GetHolder(0)
index := hldr.MustCreateIndexIfNotExists(c.Idx(), pilosa.IndexOptions{Keys: true})
if _, err := index.CreateField("general", pilosa.OptFieldKeys()); err != nil {
if _, err := index.CreateField("general", "", pilosa.OptFieldKeys()); err != nil {
t.Fatal(err)
}
@ -6696,20 +6696,20 @@ func TestExecutor_Execute_MinMaxCountEqual(t *testing.T) {
defer c.Close()
hldr := c.GetHolder(0)
idx, err := hldr.CreateIndex(c.Idx(), pilosa.IndexOptions{})
idx, err := hldr.CreateIndex(c.Idx(), "", pilosa.IndexOptions{})
if err != nil {
t.Fatal(err)
}
if _, err := idx.CreateField("x"); err != nil {
if _, err := idx.CreateField("x", ""); err != nil {
t.Fatal(err)
}
if _, err := idx.CreateField("f", pilosa.OptFieldTypeInt(-1100, 1000)); err != nil {
if _, err := idx.CreateField("f", "", pilosa.OptFieldTypeInt(-1100, 1000)); err != nil {
t.Fatal(err)
}
if _, err := idx.CreateField("dec", pilosa.OptFieldTypeDecimal(3)); err != nil {
if _, err := idx.CreateField("dec", "", pilosa.OptFieldTypeDecimal(3)); err != nil {
t.Fatal(err)
}
@ -6869,7 +6869,7 @@ func TestExecutor_Execute_NoIndex(t *testing.T) {
defer c.Close()
hldr := c.GetHolder(0)
index := hldr.MustCreateIndexIfNotExists(c.Idx(), *indexOptions)
_, err := index.CreateField("f")
_, err := index.CreateField("f", "")
if err != nil {
t.Fatal("should work")
}
@ -9684,11 +9684,11 @@ func TestExecutorTimeRange(t *testing.T) {
}
indexName := c.Idx(t.Name())
hldr := c.GetHolder(0)
index, err := hldr.CreateIndex(indexName, pilosa.IndexOptions{})
index, err := hldr.CreateIndex(indexName, "", pilosa.IndexOptions{})
if err != nil {
t.Fatal(err)
}
_, err = index.CreateField("f")
_, err = index.CreateField("f", "")
if err != nil {
t.Fatal(err)
}

2
field.go
View file

@ -73,6 +73,7 @@ var availableShardFileFlushDuration = &protected{
type Field struct {
mu sync.RWMutex
createdAt int64
owner string
path string
index string
name string
@ -1972,6 +1973,7 @@ func (p fieldSlice) Less(i, j int) bool { return p[i].Name() < p[j].Name() }
type FieldInfo struct {
Name string `json:"name"`
CreatedAt int64 `json:"createdAt,omitempty"`
Owner string `json:"owner"`
Options FieldOptions `json:"options"`
Cardinality *uint64 `json:"cardinality,omitempty"`
Views []*ViewInfo `json:"views,omitempty"`

20
field_test.go
View file

@ -22,7 +22,7 @@ func TestField_SetValue(t *testing.T) {
t.Run("OK", func(t *testing.T) {
h, idx := test.MustOpenIndex(t)
f, err := idx.CreateField("f", pilosa.OptFieldTypeInt(math.MinInt64, math.MaxInt64))
f, err := idx.CreateField("f", "", pilosa.OptFieldTypeInt(math.MinInt64, math.MaxInt64))
if err != nil {
t.Fatal(err)
}
@ -63,7 +63,7 @@ func TestField_SetValue(t *testing.T) {
t.Run("Overwrite", func(t *testing.T) {
h, idx := test.MustOpenIndex(t)
f, err := idx.CreateField("f", pilosa.OptFieldTypeInt(math.MinInt64, math.MaxInt64))
f, err := idx.CreateField("f", "", pilosa.OptFieldTypeInt(math.MinInt64, math.MaxInt64))
if err != nil {
t.Fatal(err)
}
@ -98,7 +98,7 @@ func TestField_SetValue(t *testing.T) {
t.Run("ErrBSIGroupNotFound", func(t *testing.T) {
h, idx := test.MustOpenIndex(t)
f, err := idx.CreateField("f")
f, err := idx.CreateField("f", "")
if err != nil {
t.Fatal(err)
}
@ -115,7 +115,7 @@ func TestField_SetValue(t *testing.T) {
t.Run("ErrBSIGroupValueTooLow", func(t *testing.T) {
h, idx := test.MustOpenIndex(t)
f, err := idx.CreateField("f", pilosa.OptFieldTypeInt(20, 30))
f, err := idx.CreateField("f", "", pilosa.OptFieldTypeInt(20, 30))
if err != nil {
t.Fatal(err)
}
@ -130,7 +130,7 @@ func TestField_SetValue(t *testing.T) {
t.Run("ErrBSIGroupValueTooHigh", func(t *testing.T) {
h, idx := test.MustOpenIndex(t)
f, err := idx.CreateField("f", pilosa.OptFieldTypeInt(20, 30))
f, err := idx.CreateField("f", "", pilosa.OptFieldTypeInt(20, 30))
if err != nil {
t.Fatal(err)
}
@ -170,13 +170,13 @@ func TestField_NameValidation(t *testing.T) {
_, idx := test.MustOpenIndex(t)
for _, name := range validFieldNames {
_, err := idx.CreateField(name)
_, err := idx.CreateField(name, "")
if err != nil {
t.Fatalf("unexpected field name: %s %s", name, err)
}
}
for _, name := range invalidFieldNames {
_, err := idx.CreateField(name)
_, err := idx.CreateField(name, "")
if err == nil {
t.Fatalf("expected error on field name: %s", name)
}
@ -189,7 +189,7 @@ const includeRemote = false // for calls to Index.AvailableShards(localOnly bool
func TestField_AvailableShards(t *testing.T) {
h, idx := test.MustOpenIndex(t)
f, err := idx.CreateField("fld-shards")
f, err := idx.CreateField("fld-shards", "")
if err != nil {
t.Fatal(err)
}
@ -230,7 +230,7 @@ func TestField_ClearValue(t *testing.T) {
t.Run("OK", func(t *testing.T) {
h, idx := test.MustOpenIndex(t)
f, err := idx.CreateField("f", pilosa.OptFieldTypeInt(math.MinInt64, math.MaxInt64))
f, err := idx.CreateField("f", "", pilosa.OptFieldTypeInt(math.MinInt64, math.MaxInt64))
if err != nil {
t.Fatal(err)
}
@ -291,7 +291,7 @@ func TestFieldInfoMarshal(t *testing.T) {
if err != nil {
t.Fatalf("unexpected error marshalling index info, %v", err)
}
expected := []byte(`{"name":"timestamp","createdAt":1649270079233541000,"options":{"type":"timestamp","epoch":"1970-01-01T00:00:00Z","bitDepth":0,"min":-4294967296,"max":4294967296,"timeUnit":"s"}}`)
expected := []byte(`{"name":"timestamp","createdAt":1649270079233541000,"owner":"","options":{"type":"timestamp","epoch":"1970-01-01T00:00:00Z","bitDepth":0,"min":-4294967296,"max":4294967296,"timeUnit":"s"}}`)
if !bytes.Equal(a, expected) {
t.Fatalf("expected %s, got %s", expected, a)
}

6
fragment_internal_test.go
View file

@ -1331,7 +1331,7 @@ func TestFragment_TopN_CacheSize(t *testing.T) {
index := mustOpenIndex(t, IndexOptions{})
// Create field.
field, err := index.CreateFieldIfNotExists("f", OptFieldTypeSet(CacheTypeRanked, cacheSize))
field, err := index.CreateFieldIfNotExists("f", "", OptFieldTypeSet(CacheTypeRanked, cacheSize))
if err != nil {
t.Fatal(err)
}
@ -3087,11 +3087,11 @@ func newTestField(tb testing.TB, fieldOpts ...FieldOption) (*Holder, *Index, *Fi
fieldOpts = []FieldOption{OptFieldTypeDefault()}
}
h := newTestHolder(tb)
idx, err := h.CreateIndex("i", IndexOptions{})
idx, err := h.CreateIndex("i", "", IndexOptions{})
if err != nil {
tb.Fatalf("creating test index: %v", err)
}
fld, err := idx.CreateField("f", fieldOpts...)
fld, err := idx.CreateField("f", "", fieldOpts...)
if err != nil {
tb.Fatalf("creating test field: %v", err)
}

2
go.mod
View file

@ -82,6 +82,7 @@ require (
github.com/jaffee/commandeer v0.5.0
github.com/linkedin/goavro/v2 v2.11.1
google.golang.org/grpc v1.46.0
google.golang.org/protobuf v1.28.0
)
require (
@ -166,7 +167,6 @@ require (
golang.org/x/text v0.3.7 // indirect
google.golang.org/appengine v1.6.7 // indirect
google.golang.org/genproto v0.0.0-20220503193339-ba3ae3f07e29 // indirect
google.golang.org/protobuf v1.28.0 // indirect
gopkg.in/ini.v1 v1.62.0 // indirect
gopkg.in/natefinch/lumberjack.v2 v2.0.0 // indirect
gopkg.in/yaml.v3 v3.0.1 // indirect

33
holder.go
View file

@ -436,14 +436,17 @@ func (h *Holder) Open() error {
return errors.Wrap(err, "opening index")
}
// Since we don't have createdAt stored on disk within the data
// Since we don't have createdAt and the other metadata stored on disk within the data
// directory, we need to populate it from the etcd schema data.
// TODO: we may no longer need the createdAt value stored in memory on
// the index struct; it may only be needed in the schema return value
// from the API, which already comes from etcd. In that case, this logic
// could be removed, and the createdAt on the index struct could be
// removed.
index.createdAt = cim.CreatedAt
index.owner = cim.Owner
index.description = cim.Meta.Description
err = index.OpenWithSchema(idx)
if err != nil {
@ -678,10 +681,13 @@ func (h *Holder) schema(ctx context.Context, includeViews bool) ([]*IndexInfo, e
di := &IndexInfo{
Name: cim.Index,
CreatedAt: cim.CreatedAt,
Owner: cim.Owner,
Options: cim.Meta,
ShardWidth: ShardWidth,
Fields: make([]*FieldInfo, 0, len(index.Fields)),
}
updatedAt := cim.CreatedAt
lastUpdateUser := cim.Owner
for fieldName, field := range index.Fields {
if fieldName == existenceFieldName {
continue
@ -690,9 +696,14 @@ func (h *Holder) schema(ctx context.Context, includeViews bool) ([]*IndexInfo, e
if err != nil {
return nil, errors.Wrap(err, "decoding CreateFieldMessage")
}
if cfm.CreatedAt > updatedAt {
updatedAt = cfm.CreatedAt
lastUpdateUser = cfm.Owner
}
fi := &FieldInfo{
Name: cfm.Field,
CreatedAt: cfm.CreatedAt,
Owner: cfm.Owner,
Options: *cfm.Meta,
}
if includeViews {
@ -703,6 +714,8 @@ func (h *Holder) schema(ctx context.Context, includeViews bool) ([]*IndexInfo, e
}
di.Fields = append(di.Fields, fi)
}
di.UpdatedAt = updatedAt
di.LastUpdateUser = lastUpdateUser
sort.Sort(fieldInfoSlice(di.Fields))
a = append(a, di)
}
@ -716,14 +729,14 @@ func (h *Holder) applySchema(schema *Schema) error {
// We use h.CreateIndex() instead of h.CreateIndexIfNotExists() because we
// want to limit the use of this method for now to only new indexes.
for _, i := range schema.Indexes {
idx, err := h.CreateIndex(i.Name, i.Options)
idx, err := h.CreateIndex(i.Name, i.Owner, i.Options)
if err != nil {
return errors.Wrap(err, "creating index")
}
// Create fields that don't exist.
for _, f := range i.Fields {
fld, err := idx.CreateFieldIfNotExistsWithOptions(f.Name, &f.Options)
fld, err := idx.CreateFieldIfNotExistsWithOptions(f.Name, "", &f.Options)
if err != nil {
return errors.Wrap(err, "creating field")
}
@ -775,7 +788,7 @@ func (h *Holder) Indexes() []*Index {
// CreateIndex creates an index.
// An error is returned if the index already exists.
func (h *Holder) CreateIndex(name string, opt IndexOptions) (*Index, error) {
func (h *Holder) CreateIndex(name string, requestUserID string, opt IndexOptions) (*Index, error) {
h.mu.Lock()
defer h.mu.Unlock()
@ -784,9 +797,11 @@ func (h *Holder) CreateIndex(name string, opt IndexOptions) (*Index, error) {
return nil, newConflictError(ErrIndexExists)
}
ts := timestamp()
cim := &CreateIndexMessage{
Index: name,
CreatedAt: timestamp(),
CreatedAt: ts,
Owner: requestUserID,
Meta: opt,
}
@ -874,13 +889,15 @@ func (h *Holder) CreateIndexAndBroadcast(ctx context.Context, cim *CreateIndexMe
// CreateIndexIfNotExists returns an index by name.
// The index is created if it does not already exist.
func (h *Holder) CreateIndexIfNotExists(name string, opt IndexOptions) (*Index, error) {
func (h *Holder) CreateIndexIfNotExists(name string, requestUserID string, opt IndexOptions) (*Index, error) {
h.mu.Lock()
defer h.mu.Unlock()
ts := timestamp()
cim := &CreateIndexMessage{
Index: name,
CreatedAt: timestamp(),
CreatedAt: ts,
Owner: requestUserID,
Meta: opt,
}
@ -931,6 +948,8 @@ func (h *Holder) createIndex(cim *CreateIndexMessage, broadcast bool) (*Index, e
index.keys = cim.Meta.Keys
index.trackExistence = cim.Meta.TrackExistence
index.createdAt = cim.CreatedAt
index.owner = cim.Owner
index.description = cim.Meta.Description
if err = index.Open(); err != nil {
return nil, errors.Wrap(err, "opening")

4
holder_internal_test.go
View file

@ -7,11 +7,11 @@ import (
)
func setupTest(t *testing.T, h *Holder, rowCol []rowCols, indexName string) (*Index, *Field) {
idx, err := h.CreateIndexIfNotExists(indexName, IndexOptions{TrackExistence: true})
idx, err := h.CreateIndexIfNotExists(indexName, "", IndexOptions{TrackExistence: true})
if err != nil {
t.Fatalf("failed to create index %v: %v", indexName, err)
}
f, err := idx.CreateFieldIfNotExists("f")
f, err := idx.CreateFieldIfNotExists("f", "")
if err != nil {
t.Fatalf("failed to create field in index %v: %v", indexName, err)
}

26
holder_test.go
View file

@ -32,7 +32,7 @@ func TestHolder_Open(t *testing.T) {
// no automatic close here, because we manually close this, and then
// *fail* to reopen it.
if _, err := h.CreateIndex("test", pilosa.IndexOptions{}); err != nil {
if _, err := h.CreateIndex("test", "", pilosa.IndexOptions{}); err != nil {
t.Fatal(err)
} else if err := h.Close(); err != nil {
t.Fatal(err)
@ -51,10 +51,10 @@ func TestHolder_Open(t *testing.T) {
t.Run("ErrForeignIndexNotFound", func(t *testing.T) {
h := test.MustOpenHolder(t)
if idx, err := h.CreateIndex("foo", pilosa.IndexOptions{}); err != nil {
if idx, err := h.CreateIndex("foo", "", pilosa.IndexOptions{}); err != nil {
t.Fatal(err)
} else {
_, err := idx.CreateField("bar", pilosa.OptFieldTypeInt(0, 100), pilosa.OptFieldForeignIndex("nonexistent"))
_, err := idx.CreateField("bar", "", pilosa.OptFieldTypeInt(0, 100), pilosa.OptFieldForeignIndex("nonexistent"))
if err == nil {
t.Fatalf("expected error: %s", pilosa.ErrForeignIndexNotFound)
} else if errors.Cause(err) != pilosa.ErrForeignIndexNotFound {
@ -67,11 +67,11 @@ func TestHolder_Open(t *testing.T) {
t.Run("ForeignIndexNotOpenYet", func(t *testing.T) {
h := test.MustOpenHolder(t)
if _, err := h.CreateIndex("zzz", pilosa.IndexOptions{}); err != nil {
if _, err := h.CreateIndex("zzz", "", pilosa.IndexOptions{}); err != nil {
t.Fatal(err)
} else if idx, err := h.CreateIndex("foo", pilosa.IndexOptions{}); err != nil {
} else if idx, err := h.CreateIndex("foo", "", pilosa.IndexOptions{}); err != nil {
t.Fatal(err)
} else if _, err := idx.CreateField("bar", pilosa.OptFieldTypeInt(0, 100), pilosa.OptFieldForeignIndex("zzz")); err != nil {
} else if _, err := idx.CreateField("bar", "", pilosa.OptFieldTypeInt(0, 100), pilosa.OptFieldForeignIndex("zzz")); err != nil {
t.Fatal(err)
} else if err := h.Holder.Close(); err != nil {
t.Fatal(err)
@ -86,11 +86,11 @@ func TestHolder_Open(t *testing.T) {
t.Run("ForeignIndexIsOpen", func(t *testing.T) {
h := test.MustOpenHolder(t)
if _, err := h.CreateIndex("aaa", pilosa.IndexOptions{}); err != nil {
if _, err := h.CreateIndex("aaa", "", pilosa.IndexOptions{}); err != nil {
t.Fatal(err)
} else if idx, err := h.CreateIndex("foo", pilosa.IndexOptions{}); err != nil {
} else if idx, err := h.CreateIndex("foo", "", pilosa.IndexOptions{}); err != nil {
t.Fatal(err)
} else if _, err := idx.CreateField("bar", pilosa.OptFieldTypeInt(0, 100), pilosa.OptFieldForeignIndex("aaa")); err != nil {
} else if _, err := idx.CreateField("bar", "", pilosa.OptFieldTypeInt(0, 100), pilosa.OptFieldForeignIndex("aaa")); err != nil {
t.Fatal(err)
} else if err := h.Holder.Close(); err != nil {
t.Fatal(err)
@ -105,18 +105,18 @@ func TestHolder_Open(t *testing.T) {
t.Run("CreateIndexIfNotExists", func(t *testing.T) {
h := test.MustOpenHolder(t)
idx1, err := h.CreateIndexIfNotExists("aaa", pilosa.IndexOptions{})
idx1, err := h.CreateIndexIfNotExists("aaa", "", pilosa.IndexOptions{})
if err != nil {
t.Fatal(err)
}
if _, err = h.CreateIndex("aaa", pilosa.IndexOptions{}); err == nil {
if _, err = h.CreateIndex("aaa", "", pilosa.IndexOptions{}); err == nil {
t.Fatalf("expected: ConflictError, got: nil")
} else if _, ok := err.(pilosa.ConflictError); !ok {
t.Fatalf("expected: ConflictError, got: %s", err)
}
idx2, err := h.CreateIndexIfNotExists("aaa", pilosa.IndexOptions{})
idx2, err := h.CreateIndexIfNotExists("aaa", "", pilosa.IndexOptions{})
if err != nil {
t.Fatal(err)
}
@ -136,7 +136,7 @@ func TestHolder_HasData(t *testing.T) {
t.Fatal("expected HasData to return false, no err, but", ok, err)
}
if _, err := h.CreateIndex("test", pilosa.IndexOptions{}); err != nil {
if _, err := h.CreateIndex("test", "", pilosa.IndexOptions{}); err != nil {
t.Fatal(err)
}

81
http_handler.go
View file

@ -49,6 +49,18 @@ import (
"github.com/zeebo/blake3"
)
type ContextRequestUserIdKeyType string
const (
// ContextRequestUserIdKey is request userid key for a request ctx
ContextRequestUserIdKey = ContextRequestUserIdKeyType("request-user-id")
)
const (
// HeaderRequestUserID is request userid header
HeaderRequestUserID = "X-Request-Userid"
)
// Handler represents an HTTP handler.
type Handler struct {
Handler http.Handler
@ -692,34 +704,52 @@ func (h *Handler) chkAllowedNetworks(r *http.Request) (bool, context.Context) {
func (h *Handler) chkAuthN(handler http.HandlerFunc) http.HandlerFunc {
return func(w http.ResponseWriter, r *http.Request) {
ctx := r.Context()
if h.auth != nil {
// if IP is in allowed networks, then serve the request
allowedNetwork, ctx := h.chkAllowedNetworks(r)
if allowedNetwork {
handler.ServeHTTP(w, r.WithContext(ctx))
return
}
access, refresh := getTokens(r)
uinfo, err := h.auth.Authenticate(access, refresh)
if err != nil {
http.Error(w, errors.Wrap(err, "authenticating").Error(), http.StatusUnauthorized)
return
}
// just in case it got refreshed
ctx = context.WithValue(ctx, authn.ContextValueAccessToken, "Bearer "+access)
ctx = context.WithValue(ctx, authn.ContextValueRefreshToken, refresh)
h.auth.SetCookie(w, uinfo.Token, uinfo.RefreshToken, uinfo.Expiry)
//if the request is unauthenticated and we have the appropriate header get the userid from the header
requestUserID := r.Header.Get(HeaderRequestUserID)
ctx = context.WithValue(ctx, ContextRequestUserIdKey, requestUserID)
if h.auth == nil {
handler.ServeHTTP(w, r.WithContext(ctx))
return
}
// if IP is in allowed networks, then serve the request
allowedNetwork, ctx := h.chkAllowedNetworks(r)
if allowedNetwork {
handler.ServeHTTP(w, r.WithContext(ctx))
return
}
access, refresh := getTokens(r)
uinfo, err := h.auth.Authenticate(access, refresh)
if err != nil {
http.Error(w, errors.Wrap(err, "authenticating").Error(), http.StatusUnauthorized)
return
}
// prefer the user id from an authenticated request over one in a header
ctx = context.WithValue(ctx, ContextRequestUserIdKey, uinfo.UserID)
// just in case it got refreshed
ctx = context.WithValue(ctx, authn.ContextValueAccessToken, "Bearer "+access)
ctx = context.WithValue(ctx, authn.ContextValueRefreshToken, refresh)
h.auth.SetCookie(w, uinfo.Token, uinfo.RefreshToken, uinfo.Expiry)
handler.ServeHTTP(w, r.WithContext(ctx))
}
}
func (h *Handler) chkAuthZ(handler http.HandlerFunc, perm authz.Permission) http.HandlerFunc {
return func(w http.ResponseWriter, r *http.Request) {
// if auth isn't turned on, just serve the request
ctx := r.Context()
//if the request is unauthenticated and we have the appropriate header get the userid from the header
requestUserID := r.Header.Get(HeaderRequestUserID)
ctx = context.WithValue(ctx, ContextRequestUserIdKey, requestUserID)
// handle the case when auth is not turned on
if h.auth == nil {
handler.ServeHTTP(w, r)
handler.ServeHTTP(w, r.WithContext(ctx))
return
}
@ -738,16 +768,22 @@ func (h *Handler) chkAuthZ(handler http.HandlerFunc, perm authz.Permission) http
access, refresh := getTokens(r)
uinfo, err := h.auth.Authenticate(access, refresh)
if err != nil {
http.Error(w, errors.Wrap(err, "authenticating").Error(), http.StatusForbidden)
return
}
// prefer the user id from an authenticated request over one in a header
ctx = context.WithValue(ctx, ContextRequestUserIdKey, uinfo.UserID)
ctx = context.WithValue(ctx, authn.ContextValueAccessToken, "Bearer "+access)
ctx = context.WithValue(ctx, authn.ContextValueRefreshToken, refresh)
// just in case it got refreshed
h.auth.SetCookie(w, uinfo.Token, uinfo.RefreshToken, uinfo.Expiry)
// put the user's authN/Z info in the context
ctx = context.WithValue(r.Context(), contextKeyGroupMembership, uinfo.Groups)
ctx = context.WithValue(ctx, contextKeyGroupMembership, uinfo.Groups)
ctx = context.WithValue(ctx, authn.ContextValueAccessToken, "Bearer "+uinfo.Token)
ctx = context.WithValue(ctx, authn.ContextValueRefreshToken, uinfo.RefreshToken)
// unlikely h.permissions will be nil, but we'll check to be safe
@ -825,7 +861,6 @@ func (h *Handler) chkAuthZ(handler http.HandlerFunc, perm authz.Permission) http
}
}
handler.ServeHTTP(w, r.WithContext(ctx))
}
}
@ -1700,6 +1735,7 @@ func (p *postIndexRequest) UnmarshalJSON(b []byte) error {
// Unmarshal expected values.
_p := _postIndexRequest{
Options: IndexOptions{
Description: "",
Keys: false,
TrackExistence: true,
},
@ -1785,6 +1821,7 @@ func (h *Handler) handlePostIndex(w http.ResponseWriter, r *http.Request) {
// Decode request.
req := postIndexRequest{
Options: IndexOptions{
Description: "",
Keys: false,
TrackExistence: true,
},

2
http_translator_test.go
View file

@ -26,7 +26,7 @@ func TestTranslateStore_EntryReader(t *testing.T) {
hldr := test.Holder{Holder: primary.Server.Holder()}
index := hldr.MustCreateIndexIfNotExists("i", pilosa.IndexOptions{Keys: true})
_, err := index.CreateField("f")
_, err := index.CreateField("f", "")
if err != nil {
t.Fatal(err)
}

27
idk/ingest.go
View file

@ -1531,6 +1531,19 @@ func (m *Main) batchFromSchema(schema []Field) ([]Recordizer, pilosaclient.Recor
if hasMutex { //need to clear the mutex
rec.Clears[valIdx] = nil //? maybe
}
// else { //TODO(twg) set fields not supported
// }
default:
rec.Values[valIdx], err = idkField.PilosafyVal(rawRec[i])
}
return errors.Wrapf(err, "converting field %d:%+v, val:%+v", i, idkField, rawRec[i])
})
case BoolField:
recordizers = append(recordizers, func(rawRec []interface{}, rec *pilosaclient.Row) (err error) {
switch rawRec[i].(type) {
case DeleteSentinel:
rec.Values[valIdx] = nil
default:
rec.Values[valIdx], err = idkField.PilosafyVal(rawRec[i])
}
@ -1602,10 +1615,20 @@ func (m *Main) batchFromSchema(schema []Field) ([]Recordizer, pilosaclient.Recor
fields = append(fields, m.index.Field(fld.DestName(), pilosaclient.OptFieldTypeBool()))
valIdx := len(fields) - 1
recordizers = append(recordizers, func(rawRec []interface{}, rec *pilosaclient.Row) (err error) {
rec.Values[valIdx], err = idkField.PilosafyVal(rawRec[i])
switch rawRec[i].(type) {
case DeleteSentinel:
rec.Values[valIdx] = nil
default:
val, err := idkField.PilosafyVal(rawRec[i])
if err != nil {
return errors.Wrapf(err, "booling '%v' of %[1]T", val)
}
if b, ok := val.(bool); ok {
rec.Values[valIdx] = b
}
}
return errors.Wrapf(err, "converting field %d:%+v, val:%+v", i, idkField, rawRec[i])
})
// TODO, unpacked bools aren't actually supported by importbatch
} else {
fields = append(fields, boolField, boolFieldExists)
fieldIdx := len(fields) - 2

146
idk/ingest_test.go
View file

@ -20,6 +20,7 @@ import (
"github.com/featurebasedb/featurebase/v3/logger"
"github.com/golang-jwt/jwt"
"github.com/pkg/errors"
"github.com/stretchr/testify/assert"
)
func configureTestFlags(main *Main) {
@ -269,6 +270,7 @@ func TestSingleBoolClear(t *testing.T) {
ingester.Index = fmt.Sprintf("single_bool_clear%d", rand.Intn(100000))
ingester.BatchSize = 1
ingester.IDField = "id"
ingester.PackBools = "bools"
if err := ingester.Run(); err != nil {
t.Fatalf("%s: %v", idktest.ErrRunningIngest, err)
@ -300,6 +302,7 @@ func TestSingleBoolClear(t *testing.T) {
ingester2.NewSource = func() (Source, error) { return ts2, nil }
ingester2.Index = ingester.Index
ingester2.IDField = "id"
ingester2.PackBools = "bools"
if err := ingester2.Run(); err != nil {
t.Fatalf("running ingester2: %v", err)
@ -566,6 +569,7 @@ func TestDelete(t *testing.T) {
deleter := NewMain()
configureTestFlags(deleter)
deleter.Delete = true
deleter.PackBools = "bools"
deleter.NewSource = func() (Source, error) { return tsDelete, nil }
deleter.Index = indexName
deleter.BatchSize = 5
@ -579,6 +583,7 @@ func TestDelete(t *testing.T) {
ingester := NewMain()
configureTestFlags(ingester)
ingester.PackBools = "bools"
ingester.NewSource = func() (Source, error) { return tsWrite, nil }
ingester.PrimaryKeyFields = primaryKeyFields
ingester.Index = indexName
@ -874,7 +879,6 @@ func TestBatchFromSchema(t *testing.T) {
rawRec: []interface{}{true, uint64(7), false},
rowID: uint64(7),
rowVals: []interface{}{true, false},
err: "field type 'bool' is not currently supported through Batch",
},
{
name: "mutex field",
@ -1429,6 +1433,7 @@ func TestNilIngest(t *testing.T) {
err string
batchErr string
rdzErrs []string
packBools string
}
runTest := func(t *testing.T, test testcase, removeIndex bool, server serverInfo, rawRec []interface{}, clearmap map[int]interface{}, values []interface{}) {
m := NewMain()
@ -1437,6 +1442,7 @@ func TestNilIngest(t *testing.T) {
m.PrimaryKeyFields = test.pkFields
m.BatchSize = 2
m.Pprof = ""
m.PackBools = test.packBools
m.NewSource = func() (Source, error) { return nil, nil }
if server.AuthToken != "" {
m.AuthToken = server.AuthToken
@ -1522,12 +1528,13 @@ func TestNilIngest(t *testing.T) {
Vals2: []interface{}{nil, nil},
Vals3: []interface{}{nil, nil},
}, {
name: "bools null",
pkFields: []string{"user_id"},
rawRec1: []interface{}{"1a", true}, // bool and bool-exists
rawRec2: []interface{}{"1a", DELETE_SENTINEL},
rawRec3: []interface{}{"1a", nil},
rowID: "1a",
name: "bools null",
packBools: "bools",
pkFields: []string{"user_id"},
rawRec1: []interface{}{"1a", true}, // bool and bool-exists
rawRec2: []interface{}{"1a", DELETE_SENTINEL},
rawRec3: []interface{}{"1a", nil},
rowID: "1a",
schema: []Field{
StringField{NameVal: "user_id"},
BoolField{NameVal: "bool_val_1"},
@ -1587,3 +1594,128 @@ func TestNilIngest(t *testing.T) {
}
}
func TestBoolIngest(t *testing.T) {
rand.Seed(time.Now().UTC().UnixNano())
indexName := fmt.Sprintf("boolingest%d", rand.Intn(100000))
primaryKeyFields := []string{"user_id"}
tests := []struct {
src *testSource
expTrue []string
expFalse []string
expNull []string
}{
{
src: newTestSource(
[]Field{
StringField{NameVal: "user_id"},
BoolField{NameVal: "bool_val"},
},
[][]interface{}{
{"a1", true},
},
),
expTrue: []string{"a1"},
expFalse: nil,
expNull: nil,
},
{
src: newTestSource(
[]Field{
StringField{NameVal: "user_id"},
BoolField{NameVal: "bool_val"},
},
[][]interface{}{
{"a1", false},
},
),
expTrue: nil,
expFalse: []string{"a1"},
expNull: nil,
},
{
src: newTestSource(
[]Field{
StringField{NameVal: "user_id"},
BoolField{NameVal: "bool_val"},
},
[][]interface{}{
{"a1", nil},
},
),
expTrue: nil,
expFalse: nil,
expNull: []string{"a1"},
},
{
src: newTestSource(
[]Field{
StringField{NameVal: "user_id"},
BoolField{NameVal: "bool_val"},
},
[][]interface{}{
{"a1", DELETE_SENTINEL},
},
),
expTrue: nil,
expFalse: nil,
expNull: []string{"a1"},
},
}
var ing *Main
defer func() {
ing := ing
if err := ing.PilosaClient().DeleteIndexByName(ing.Index); err != nil {
t.Logf("%s for index %s: %v", idktest.ErrDeletingIndex, ing.Index, err)
}
}()
for i, test := range tests {
t.Run(fmt.Sprintf("test-%d", i), func(t *testing.T) {
ingester := NewMain()
configureTestFlags(ingester)
ingester.PackBools = ""
ingester.NewSource = func() (Source, error) { return test.src, nil }
ingester.PrimaryKeyFields = primaryKeyFields
ingester.Index = indexName
ingester.BatchSize = 1
ingester.UseShardTransactionalEndpoint = true
// Set ing so the defer can do cleanup.
if i == 0 {
ing = ingester
}
if err := ingester.Run(); err != nil {
t.Fatalf("%s: %v", idktest.ErrRunningIngest, err)
}
client := ingester.PilosaClient()
idx := ingester.index
fld := ingester.index.Field("bool_val")
// Check true.
{
resp, err := client.Query(fld.Row(true))
assert.NoError(t, err)
assert.Equal(t, test.expTrue, resp.Result().Row().Keys)
}
// Check false.
{
resp, err := client.Query(fld.Row(false))
assert.NoError(t, err)
assert.Equal(t, test.expFalse, resp.Result().Row().Keys)
}
// Check nil. This is used to test the ingestion of nil and null.
{
resp, err := client.Query(idx.Difference(idx.All(), idx.Union(fld.Row(true), fld.Row(false))))
assert.NoError(t, err)
assert.Equal(t, test.expNull, resp.Result().Row().Keys)
}
})
}
}

9
idk/interfaces.go
View file

@ -1131,11 +1131,16 @@ func toBool(val interface{}) (bool, error) {
}
return vt != 0, nil
case string:
switch strings.ToLower(vt) {
vt = strings.ToLower(vt)
vt = strings.TrimSpace(vt)
switch vt {
case "", "0", "f", "false":
return false, nil
case "1", "t", "true":
return true, nil
}
return true, nil
return false, errors.Errorf("couldn't convert %v of %[1]T to bool", vt)
default:
if vint, err := toInt64(val); err == nil {
return vint != 0, nil

23
idk/interfaces_test.go
View file

@ -15,6 +15,29 @@ func TestDecimalFieldPilosafy(t *testing.T) {
}
func TestBoolFieldPilosafy(t *testing.T) {
f := BoolField{NameVal: "boolcol"}
validTrue := []interface{}{true, 1, "t", "true", " T ", " True"}
validFalse := []interface{}{false, 0, "f", "false", " F ", " False"}
invalid := []interface{}{"boat", " test "}
for i, v := range validTrue {
if ret, err := f.PilosafyVal(v); err != nil || ret != true {
t.Errorf("test: %d, got: %v of %[1]T, err: %v", i, ret, err)
}
}
for i, v := range validFalse {
if ret, err := f.PilosafyVal(v); err != nil || ret != false {
t.Errorf("test: %d, got: %v of %[1]T, err: %v", i, ret, err)
}
}
for i, v := range invalid {
if ret, err := f.PilosafyVal(v); err == nil {
t.Errorf("test: %d, got: %v of %[1]T, err: %v", i, ret, err)
}
}
}
func TestPilosafyVal(t *testing.T) {
tests := []struct {
field Field

1
idk/kafka/cmd_delete_test.go
View file

@ -56,6 +56,7 @@ func TestDeleteConsumerCompoundStringKey(t *testing.T) {
m.Index = fmt.Sprintf("cmd_del_comp_indexij%s", topic)
m.PrimaryKeyFields = []string{"abc", "db", "user_id"}
m.Topics = []string{topic}
m.PackBools = "bools"
defer func() {
// TODO: for some reason (which I didn't dig into),

2
idk/kafka/source_test.go
View file

@ -549,8 +549,6 @@ func TestKafkaSourceTimeout(t *testing.T) {
// be, but I think it has something to do with kafka rebalancing
// itself when a new client joins.
func TestKafkaSourceIntegration(t *testing.T) {
t.Parallel()
if testing.Short() {
t.Skip()
}

16
idk/testenv/certs/ca.crl
View file

@ -1,16 +0,0 @@
-----BEGIN X509 CRL-----
MIICfjBoAgEBMA0GCSqGSIb3DQEBCwUAMA0xCzAJBgNVBAMTAmNhFw0yMjA3MjIx
OTUxNDVaGA8yMTIyMDcyMjE5NTE0MlowAKAjMCEwHwYDVR0jBBgwFoAU+NtDPKa1
UzeelNYEuhD5snOV+aowDQYJKoZIhvcNAQELBQADggIBAIfRwTYSQNDwnFB3YlB/
PoHhEWKdmyoFnN/Zax9FIa8/0F7K/mWwoCPH5xhdP9oo4MR3XR9R0A+iVX3Vohme
ku6XgL2YxGwYL5pZLNH1N34iRNF8R2TnLHMi8dF37JJcJ/HsuiGRqNnpnELlz1m8
/sxJVl8w3sZBsCmMsDI/lyzk1du+mKJk3bIjapefyT/cQGd4r8kbDyM4ZRzORiks
g4X6nXZK1jhBo6/7nzLsa9egHUGA4rleBleimCXcnaVCyQgVOXorhHdGPNIJQRZg
JGIzV1FdSp2ecrzZ0ERALtz4VkOyWK9Nfhy6TJBiKvc4gDS7M/meNl4GdIu8eV6W
BdL/VlR1JsUSBejd2vjdd5X0SZoMi6dJzUoU6EhmyRFnZMdLAlRcn7eHQqXW5bqP
ZqnGAbb68tVLZI0vt0QyYlmlaWKFkOMGSS+5Gu04uoYNDufHzMb0Nh5vt5MEEkNE
ZYERuzT/PcrI1dEq6di9iK6dRoeTERMgWcZoKg7Yx5jReks0v+Pae4d+B4dHdB52
LtmK9bn43/E2gbVhRYNDX7xi6rG9AxdshL3/IcLq11bWiDjPL7sUAcvJKk//KXEc
nA/p8H+ZHsD3E1Lln+/3Yf7nj/cxjbF46RRm0yzLdDuTVa+Ov8OB7DtGhT8wW+Ok
gEJXdXlcnzv51gGa2F35/DFi
-----END X509 CRL-----

28
idk/testenv/certs/ca.crt
View file

@ -1,28 +0,0 @@
-----BEGIN CERTIFICATE-----
MIIE3DCCAsSgAwIBAgIBATANBgkqhkiG9w0BAQsFADANMQswCQYDVQQDEwJjYTAg
Fw0yMjA3MjIxOTUxNDVaGA8yMTIyMDcyMjE5NTE0MlowDTELMAkGA1UEAxMCY2Ew
ggIiMA0GCSqGSIb3DQEBAQUAA4ICDwAwggIKAoICAQD3qe+w4WaeFtroEqBpxKJY
/a/IpC12Bg4HAYqkK3FwWhOVA+AQ1w2AA/BnLa3u8eszVsNdsnmo+qWZmI9fiJiR
sylJNbnATuWYus7rNNXWnhot3P6CJjEMs5phC0S9zaizFVzL7GFnpjfGgPrbZoQJ
Blmxb/IuEaaKGjUw0MSs83SuauRTGBjVSX80JYI61Wc/KO8hhmLEMne9WSD+Ow3i
GEt5RGRtN6HLkmCBPHieBsWtqToltEV5AqgOXiskGK16q13osBK6mg17QDqg3g7z
w2XSGQx/HSS8e16SqBBt3Ia43ZL7r8qgNxXA9wQP86RKI55ncR3Q7MGVcIMobYbh
gB3nlgqSIYkTs5+5Ffq3na+Fv7yjhw/4gl9UKb84wpZdgHyL4HFrkwdZ3k/go319
6n0gLpAsgruBF8MNRdQmyojFJX0NKgT7qW6FabpjZklLfzI8MWHX+2sCmICc8Ny5
imbdz9sJGJi9vEU+AT6HK0pNHQx+4aQyiq/Md3XFNJmkvt9m9vfE7IcTlV/1Pf7W
7Sj06q55k9lO+nmv6qfu3nO1EL+MIHDvObkqNrKiA4ynuBWa+ZIwjLvc/zs6w6or
l7qGSBZZCHv/e/cPrq/EIxUAvD9yhUUbpJY2FCTGv5U87vxWmqHGnoTviRbkg9u7
6mEwX84Xspg1bTgMbHs7OQIDAQABo0UwQzAOBgNVHQ8BAf8EBAMCAQYwEgYDVR0T
AQH/BAgwBgEB/wIBADAdBgNVHQ4EFgQU+NtDPKa1UzeelNYEuhD5snOV+aowDQYJ
KoZIhvcNAQELBQADggIBADLv2XnXmYCpHVjqVD4NHmg7iv1nitOPsgcqmIUlJ8ZA
iJKN7o5eZs1k26MQXFjcTtEqjV7+yzMfnH53bvD5oEzpxzK6IzQyhxOmz07h907w
/31GVy1VD9+hfUukRnwcVCabc8attyrSs1rI+4RmMdEBNrUEQkNnwWxwzgBMFaIM
P2uH8qen3EIJFxLMl5JJOYE2FzuXrTpvsA0jsCzxVI8bEHzBVaLiNqdMQedsdKAV
WLp64yW/ACnqadk1YIfb8S4sAx/HsLBsnz5+LHHoCzUcodSVODm/otBxJAB2vNUb
ulh3bSWZVdQ4SykMFLlDDd0l+gbgPCVsB8aKj5r6b4EuM6Bsq8GTG511f03vP96g
U4sUkZHssYxA7T5VzIHUtzSl0xC0JngVPVXAcUqGv0etNwlaL/zDotVC8mtPTF15
cTwy9upE/MHXrYegMTdi2t5ZvM9vVHWemSJ1J0X0jsVmJtT9hYuzPyFR8qAEpd06
XZD2upB8c3X+kyWdIV0oh2fk37glUU3JihZTRib24qBZ7TUlkGIM0K3PkUHlZ8+v
ljwouIa5Ghf+rYBA0ABJeYdFw6i9hRglWvBc/FHmpIBl9gLzB7+C9Yg20WXMZCvr
p7H/xSf5Dh8fpCO0yeGdpzfWy/3kpTmxg9/RPWyv8hTaB+4ur6CY3pVsVcshLhS3
-----END CERTIFICATE-----

51
idk/testenv/certs/ca.key
View file

@ -1,51 +0,0 @@
-----BEGIN RSA PRIVATE KEY-----
MIIJKgIBAAKCAgEA96nvsOFmnhba6BKgacSiWP2vyKQtdgYOBwGKpCtxcFoTlQPg
ENcNgAPwZy2t7vHrM1bDXbJ5qPqlmZiPX4iYkbMpSTW5wE7lmLrO6zTV1p4aLdz+
giYxDLOaYQtEvc2osxVcy+xhZ6Y3xoD622aECQZZsW/yLhGmiho1MNDErPN0rmrk
UxgY1Ul/NCWCOtVnPyjvIYZixDJ3vVkg/jsN4hhLeURkbTehy5JggTx4ngbFrak6
JbRFeQKoDl4rJBiteqtd6LASupoNe0A6oN4O88Nl0hkMfx0kvHtekqgQbdyGuN2S
+6/KoDcVwPcED/OkSiOeZ3Ed0OzBlXCDKG2G4YAd55YKkiGJE7OfuRX6t52vhb+8
o4cP+IJfVCm/OMKWXYB8i+Bxa5MHWd5P4KN9fep9IC6QLIK7gRfDDUXUJsqIxSV9
DSoE+6luhWm6Y2ZJS38yPDFh1/trApiAnPDcuYpm3c/bCRiYvbxFPgE+hytKTR0M
fuGkMoqvzHd1xTSZpL7fZvb3xOyHE5Vf9T3+1u0o9OqueZPZTvp5r+qn7t5ztRC/
jCBw7zm5KjayogOMp7gVmvmSMIy73P87OsOqK5e6hkgWWQh7/3v3D66vxCMVALw/
coVFG6SWNhQkxr+VPO78Vpqhxp6E74kW5IPbu+phMF/OF7KYNW04DGx7OzkCAwEA
AQKCAgEA6bBo4f8dhAhO0HJa+NI42j2t82WvG9GExOmYd2YiqP/lwFMixuEU9PlC
iykYGQIei5fPyoaQs3imb3L7vgo25CwoxdKzDhmkHWQPwPJe4B7Y1vPTFt2QpMqH
3g/y8iGvkCOVJzpJgrLdqCmmCwYFijp6wam+2+d9vIwUKpajLgqey8FC5oEzVXX/
WCYYZwlcXcVzNrAGW9i/EWSbdCgm5nNELA3zsPawbVUWSHvV2E7Nkhq/Kjqa3x85
ki0bCflbIAe5GcmiMn8QP38QkyBU/YAfgIpwNzU5p7vFNh7tOay/VoqEmYZ7bIft
t+Gc7KxaFjbbzgF3P99WnnrG14vZmuy4w/hCKsqDPNRefBTs6YQnEbfmu6CApuiP
Sog69F9n3oEQFzYbePeUDe9pOv8LpTDDkZL/gPW83Pp3C8LFtYhuXtGBWAvBj9P6
/ryG5pC9USk1gD6mJLBY+UbvEDsN7TKRLX8+Zx6M8YSoAMFceHNW91GGUd1I95s8
ZQqCNSJ8fR2jscpB8bs9pUs0pa2/WLG8EVaQ10H6nYn0aC/ySQAm92nG5dVnd9Ze
mXkmSOnWVMfLhw7h8vI9hv3E39EAYpwvCS/c4STNJX3dl5fsBq63UWBkHBW83mXR
4w/AT8jvHNpmLRumSBGULFZI9pAajI0ZFODL8JSySEE1KqvTLc0CggEBAP/VAze7
jAroP0N3icm1eZY0T2WaFqSOra3PO1Y/9NLi+wsteUnf1jMlvAO6qkHgV59RCjCu
X6fRCdiGGmUPS8ksPBzWshfUK9d+9ENIL3hj0gqNqpqDDqBNyqTICML1FWGLnVT7
Z5o0cPvqh0w+/Xv9A/67ryz9QpjaN8VsP2P/X7gBIgTZrv/yRy8oBI1kCebVdwfK
NEW6B8+yB67kt9yyRELPhXd67PqRwjTTkJkAvaghOJl9lQ7wm4qRyA2DHEoFuKK9
XPBWvQzBLtBPsVOPdyk9ENfYXIjoqkz4wyJKER4ltAEU+Eb2Daa1sjMUR94FneF6
redWZrAFHUeav18CggEBAPfTjRwmd1ROhoETMKdx8A1Q40Edy7FNdeVGPA7Evagn
77hmXbQlBHP1jCvAOisGJ3jG7L6mkpDltdsRUqHFnr3u3xV1KAK0kxMlefYKh6da
3/vDOd7xODed7QsFvmJH3YS0Sbjs1/C5WwGdqRopPzX73aS8Y+hJzhfDUsDV/PA4
PcHAT+Q86Rbr31XUURqP8+v5KEGkbHB0NBe/4t4sK8YY7KBbeLbB8zQR5pg4keKz
5M3lFIIpzcSsYVbObSTf1iBsMsGgTP26d+266qtmSwabBtD4t6Lye0uugLbAg6C+
FHk7cv0n3NxeoBFxkN0fFN4r8sKttFlMNgdbdo99RGcCggEBAMdduKQ2A2x8kNCC
113Zjmb/XOBQtFSI29AXLqaIMMInIHwVc++d/87cGSjOzt+HdMEy5j4JpYGh5YpW
9zrKMMUVM8NFjIQFVCUbSj8kgdnP8F9JByph94P1G3ZObIWBeKgLpRAU2yfQtjub
CNTiQafeBL7+hAULWFrFs7CmKsBCwtUiDOC6wyWyKMcW6HVHAHBi6d+oNBQiZPP2
SQaCNsZGJevqzJfPs92QuUdxl72kVigvu2vh8ccyugPCl6PwCJum2bv41fR1o93M
ri6n7AiyRKpebEHI2lPW/N/+/BI8phpYR26QaOrj1L+V2Mj1hQwAVIjikaM2Mc/6
LFcgwQECggEBAPBw75dAOsYw6Y6Rb8MFaEGDGPk0T0vWO2wWmG47s0ZSeOIPbxo4
T/mxYi2Qct8LOYCUf1z5f3nVZHsGc4VAlqLv/sByyj7Vs1wAIDP2Q2a7ZFR/NMZ4
gnJzZmgrSmtQDjuSL3SbVWbYbtNCg64+FnXpx0DtBoF4Gdw0RTWSZ7Uo28H/M4Pd
FMiIUEaVl5qNz7pCIiyNuifglin0ocesU0v/IpTyKkwmIC+ErpVBTUkGABdDCoK+
zETkegweewYrpTQPxa7WN9xz7adYkhV5SZE/F33n0ULmv3jN9VNZtFtgD70PUKEM
69ticyp5ZHMklLp5KS54GWIJ7uF0e7L7dkECggEAc2MBDPn0b5kb+7YTy5eayizM
n6OcUrTRpTpVGiRmeYkG25VNwH+P99dwbYV/+Im8TwNmlmfs4i1r0gx04lMDOiPr
eQ3c6LqI/N49QfFWOZLPjh5noZM7Rae/xkHTar7g49LUFGv1RiWNyIylzW931Ysc
/nwVY8ERIhNUp/7OTosIZDdF0TEnQYl2j3SvYd3SMqJpUBGX4ClEADzMil+hiD7w
DkLa9GFrGMAgGkcdp3bl1IUSipyI9zGdoUUMw/xXHdycm0MMbhhNn3M7j82bF6aP
OlirJSWDdOlclNXSpVTaNZ6zfBtaMfnZgUM2LF6U2ZtI0iC5ak/nDPo85Eiltw==
-----END RSA PRIVATE KEY-----

25
idk/testenv/certs/localhost.crt
View file

@ -1,25 +0,0 @@
-----BEGIN CERTIFICATE-----
MIIENjCCAh6gAwIBAgIQLtKA8VN9DP2u3nQt1rcZFzANBgkqhkiG9w0BAQsFADAN
MQswCQYDVQQDEwJjYTAgFw0yMjA3MjIxOTUxNDVaGA8yMTIyMDcyMjE5NTE0Mlow
FDESMBAGA1UEAxMJbG9jYWxob3N0MIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIB
CgKCAQEAsxpHn/bm5EwiYcKIKY3sKG5sByIJa6gG6MFnrBt2Urd9LlDEnyq6A38i
fhIwTqPUY4dvCXW6SNOqp/ugTCbp78F0GUPvJodK+IsrvvVjmQeXg/9ieN91VqY5
xaVkkaLOEWrcoTAD23+eO0Q3afG5+AFplFyrGwsCFIgz0iB8DMuixsObCao3EWA+
YwiFUTFE6gthq2rPPXzFtszlGFm6ZJAJOIfnVTgEjNMSc5KgkF39+1vL6URtybnk
3thhnsgSWQDoWgKiBxPt/IuTnJun5JdHkgKTgfO2PVczsFmMzg2e+LJjI8ds8GEY
T7P9jP5KjtzrZ8WMKw6BeeNzXCmjvQIDAQABo4GIMIGFMA4GA1UdDwEB/wQEAwID
uDAdBgNVHSUEFjAUBggrBgEFBQcDAQYIKwYBBQUHAwIwHQYDVR0OBBYEFB9m9jk7
qfCfPnHUcWDTf7b40ZH0MB8GA1UdIwQYMBaAFPjbQzymtVM3npTWBLoQ+bJzlfmq
MBQGA1UdEQQNMAuCCWxvY2FsaG9zdDANBgkqhkiG9w0BAQsFAAOCAgEAwwXgkVYu
VBq6Uxm+DORby4LiFmLwKIJFuO1UUlAvqNAmUyy6h3S/azb9qvfNiMmHe0+C9j2M
t4eb1KGbZaNiMr5bATFGkJzQl4bEsKZqLv23D6LDRV8D/DtZzNb6e2xhIx1uQYc8
J8oVlciTKVr5jI82ewvO4O4mu9MlFACva1Zvzk0CB4UZEq1QXpTAlNPG7e3PVT1u
wNZXJ/8kmE1/7OMfQ9PZBGtiYK3OAiTLc0GHFhNjnCFWvFq+sD70bSJCiy3UjMUq
GhcAVHsyzv+O2nmDX4G/QXn+An1wb6hfKYFsR4nWBXQn1HjaswXdw91O6t1UViTS
RittkL/jS3Pn1wzBl80zn+/oOcxSt/KKA07Wfi4JYk6KHWHMk6lTnvMn+csXiRpq
hvUd4gcr4fTYynaXWMP3Of+H1DyJ0edXHxrHjVRS7f9AOpQF1LaZSkWff9ety5Ir
omCvkmopsOeFxKvrbVvgkAi0r8rwfockSNxZI2tp2QFXu6FDywKtL/P/YBJujcsm
zm5XFrjnri/R+c9JQNbuctZtb3Y1WAXnz2RWeunNxxjyaYvEiWh6vXgu0QCXUbev
4wUOK+gFgyczHtbprph9a5iUjEA3P+wzp5u7zGpax07BXMEFSul0HEklc1YP3/ac
35SOBQlvo1oX0VJE0CZB8zlT7upPTuTjus0=
-----END CERTIFICATE-----

16
idk/testenv/certs/localhost.csr
View file

@ -1,16 +0,0 @@
-----BEGIN CERTIFICATE REQUEST-----
MIICgDCCAWgCAQAwFDESMBAGA1UEAxMJbG9jYWxob3N0MIIBIjANBgkqhkiG9w0B
AQEFAAOCAQ8AMIIBCgKCAQEAsxpHn/bm5EwiYcKIKY3sKG5sByIJa6gG6MFnrBt2
Urd9LlDEnyq6A38ifhIwTqPUY4dvCXW6SNOqp/ugTCbp78F0GUPvJodK+IsrvvVj
mQeXg/9ieN91VqY5xaVkkaLOEWrcoTAD23+eO0Q3afG5+AFplFyrGwsCFIgz0iB8
DMuixsObCao3EWA+YwiFUTFE6gthq2rPPXzFtszlGFm6ZJAJOIfnVTgEjNMSc5Kg
kF39+1vL6URtybnk3thhnsgSWQDoWgKiBxPt/IuTnJun5JdHkgKTgfO2PVczsFmM
zg2e+LJjI8ds8GEYT7P9jP5KjtzrZ8WMKw6BeeNzXCmjvQIDAQABoCcwJQYJKoZI
hvcNAQkOMRgwFjAUBgNVHREEDTALgglsb2NhbGhvc3QwDQYJKoZIhvcNAQELBQAD
ggEBAItPe0ClBPlTpkfmbTE0JWsai6Nh12bakigQePI58HYbeV2g0Ld9vpUNGE+i
esTnvDKSYEHKyscNRsehXU8XArKEj6MzUgicPxhqr0eow9O2kGanUz5/M4Y8w0Oa
Q2wUCpqS08jCT0bwdCtFY/y19DXJmwovkGa5+yB+UV5FKkA31l6X7c0vzrdQexAc
FOQjG/9TpvSWXkvkZOv9041f9xeg2g23lfx5gnDvef0hgKhbOsN9tr6tVfJ4zGDv
qC+PuKMTLTfJWKgc2wNLBevA3lYFoHchD+hZMGNktsMraS2Re0FC6MCqH9UMlSGd
T4FN0g5TdI1o5/Hcxjmeac+T444=
-----END CERTIFICATE REQUEST-----

27
idk/testenv/certs/localhost.key
View file

@ -1,27 +0,0 @@
-----BEGIN RSA PRIVATE KEY-----
MIIEowIBAAKCAQEAsxpHn/bm5EwiYcKIKY3sKG5sByIJa6gG6MFnrBt2Urd9LlDE
nyq6A38ifhIwTqPUY4dvCXW6SNOqp/ugTCbp78F0GUPvJodK+IsrvvVjmQeXg/9i
eN91VqY5xaVkkaLOEWrcoTAD23+eO0Q3afG5+AFplFyrGwsCFIgz0iB8DMuixsOb
Cao3EWA+YwiFUTFE6gthq2rPPXzFtszlGFm6ZJAJOIfnVTgEjNMSc5KgkF39+1vL
6URtybnk3thhnsgSWQDoWgKiBxPt/IuTnJun5JdHkgKTgfO2PVczsFmMzg2e+LJj
I8ds8GEYT7P9jP5KjtzrZ8WMKw6BeeNzXCmjvQIDAQABAoIBAFuT5meusWSuYS4n
b8P/o28Q9v+2saZwZggBTGc+K4Qj+kgCWGciS7aZ/SMFXdn3/xNdHrNxlOzD/hCi
otYNV7SV8C0zBUdoCjeHwbiEdIa80QZXQRyY9cB2gjfDV+b3nfJd964bhP1pBZk8
SjFLvlFHN5r6vyq2uDuTYRviLEvFwMqSf0avKTC5ZgNY5GqmL5tdHiFQHhKaCuTd
BdI1RJ4Hsmjo1GfOY2vQy1mzjeraHaDhmIN9v9mmyDr/aCVaKk7QVHvAiIG/E+T/
mQImA+1z/G6+BFxvdYu2e42N3jZvl2YELTcb6FnwIswG/1rQG2Cm6OPe2AeW1kBY
2ffdrqkCgYEA28tl3bhVUsA70+A3QS7xWuWuYuwIP1GnqWCP881G+uowC1tqVr2X
wrOEB+gF2v4JZybi6/yIE78LPf7z1pm9Oi1ZT8y/D1T+MjLux0CCG5tGWX26flPm
hFhCg2XvSpiTRnbwsbIoa3czjRxgvI1zpkLNkfV366ILOfiEGlvC+pcCgYEA0Jru
MESdulirmWWJMmIQxsGqMEdln2idh4r3csYHzUfEuQ7DSUunE9uB3ZcnsEzTbpPR
OP4+VdwfSinFA4EXIZ/kgOJMmdgWRWPMBpMkbRNOMigvy8K/k97JH8v0fV3wMa+U
5qK0O4EWthZjaJZtD2WyKoiwZDXXrJj8QsPeQssCgYEAsP529nglRXEF/JW3CSer
Nj51ErXt4kd4E15uLa8ltJL3w32HAXjFiAWVkeRXKsWdftKCs/R1BCm4/OCdLTg6
eGniXO+M/+S6mmQHgq7A10hP+2rSzj30CKkmTre8Xb7GAl1vcDj+caPZAI5UnCHw
enebISmhuz+PeMxWXP994NsCgYBM/0K3sjv90kjUf/PaEvRaSJWG6HOMGVM7+oFH
beznSnxESjyuyyvKWpVeszJFErZf9FOfk8hybKNn/m7n0jg1gw0WvXDEwEvIJQPj
0V/6msucG9U4gpOwAuV9xQxjUc8cp9BwKNIZvJYjc7QRoKR0sHbI2FczReRrorRJ
+H4LbQKBgDprPI55tyt15PivkyGJYjeHMr6bEk7bqHMDeloPwXsbZr4Esp/4PcYB
Js2BwC+PvXUG/37uk/8B9EdC/l0mbSWos0lhIkaORErnjwWOi1NBh8ZInbetUNIi
A+TeEEeSdnK7ou4s030krS/pG0ZsTUYVEMRFwHiIvoD6/G2wsJhi
-----END RSA PRIVATE KEY-----

25
idk/testenv/certs/pilosa-tls.crt
View file

@ -1,25 +0,0 @@
-----BEGIN CERTIFICATE-----
MIIEODCCAiCgAwIBAgIQbk/XbQqd/JZWsL2MQ2BmpjANBgkqhkiG9w0BAQsFADAN
MQswCQYDVQQDEwJjYTAgFw0yMjA3MjIxOTUxNDVaGA8yMTIyMDcyMjE5NTE0Mlow
FTETMBEGA1UEAxMKcGlsb3NhLXRsczCCASIwDQYJKoZIhvcNAQEBBQADggEPADCC
AQoCggEBAN++mmirorKBY7HNe+obmh/1od/5INgjIMobQ6BILGzKeWIH0BpEVR9w
dTUtGM2z/kct8Oi6RibOl99Lj0d/nseKn9kjqMXdJEnoyooQBI++QrWwyYHZ76ag
9H1gmpCqUQILibFGUy9WLRuJbbLAGYfFsb2aiISEIFwbhFg2uFFD8W7VGzOHCXfA
S+oiYSFP3E9pddN4bkurmeuiL+lXxFBTL+RhssdBNreiS4pJ21vbPJw0xpwU7RVB
2aSNVqfdVzXIMwnjPX9zqeGUcvJAcI+9kcMLwGmynjaZ1idVPbpmShhAHJqR/R/1
r20i32UE6l5SHBWeW+n/2Wb5ZXHT6lkCAwEAAaOBiTCBhjAOBgNVHQ8BAf8EBAMC
A7gwHQYDVR0lBBYwFAYIKwYBBQUHAwEGCCsGAQUFBwMCMB0GA1UdDgQWBBSpCzKw
Yt9vI7BsLAy43R9fGNLbjjAfBgNVHSMEGDAWgBT420M8prVTN56U1gS6EPmyc5X5
qjAVBgNVHREEDjAMggpwaWxvc2EtdGxzMA0GCSqGSIb3DQEBCwUAA4ICAQBPLSKl
LU3BenjGhIoPfz43odk+XWtcBWx2xxI0+RKtYMXA689lk9UO8ghtPfbhvovVzORF
Sveh1GC6n6kIwmA6v8svQewqWiKlcOglQttRuCdKPLF3y0OnyxW4ki4/nHOmKWvo
lUS+iB37HNMMkvAp9t19k3d18NVDWPpa1R8tukWYdsbeze/lznXO4lHL5AoRZqJP
h5ihCdW07spDX4N0cx09vfqCBGEMClYvnBZzIjiDdtEXjh9Ve81+kxqak6lX0nL0
p4ilqnWbz4zokTusJZCSfPslcq7V95+3EiBZjZ/NV9ZNjR9HCjfPCe5uQqYK5LJh
1/RG8gBUfcDLvm36c2cQI0ZrG9RMXKjBPlSv3OPd4vRlEMVHVtTyxP3FAMnhlYqp
kVmgoo0ambTxoYhdS3vKV/47KNnmWiJoPNZUZO+40zRfcCuBfuPQGlQgr3pctGZy
NQVp0eNz9xOH+UDrHhPPyDG0F+XcwgflBaIWSoyh4skESlzGqTBcsKGOkVCyrapq
a5jS1z8kdbunnwit8Di0OxXHxbjoIUAEpyP9fakGRcUD4LRZOUkHiR2Dci4hpzb3
6Xd4BtL0Va9uj//vcRQ+5nnO1PshOde6sQ7Kup2B4OmwyDPcm8Af3nViYQOt1hCc
5vO61aRQN6YmLO5i4CD+kjaurBn8ddTzbGJaaA==
-----END CERTIFICATE-----

16
idk/testenv/certs/pilosa-tls.csr
View file

@ -1,16 +0,0 @@
-----BEGIN CERTIFICATE REQUEST-----
MIICgjCCAWoCAQAwFTETMBEGA1UEAxMKcGlsb3NhLXRsczCCASIwDQYJKoZIhvcN
AQEBBQADggEPADCCAQoCggEBAN++mmirorKBY7HNe+obmh/1od/5INgjIMobQ6BI
LGzKeWIH0BpEVR9wdTUtGM2z/kct8Oi6RibOl99Lj0d/nseKn9kjqMXdJEnoyooQ
BI++QrWwyYHZ76ag9H1gmpCqUQILibFGUy9WLRuJbbLAGYfFsb2aiISEIFwbhFg2
uFFD8W7VGzOHCXfAS+oiYSFP3E9pddN4bkurmeuiL+lXxFBTL+RhssdBNreiS4pJ
21vbPJw0xpwU7RVB2aSNVqfdVzXIMwnjPX9zqeGUcvJAcI+9kcMLwGmynjaZ1idV
PbpmShhAHJqR/R/1r20i32UE6l5SHBWeW+n/2Wb5ZXHT6lkCAwEAAaAoMCYGCSqG
SIb3DQEJDjEZMBcwFQYDVR0RBA4wDIIKcGlsb3NhLXRsczANBgkqhkiG9w0BAQsF
AAOCAQEAauULC7xyYwTWpeM7b4I3LXMHAR3Q9/kQqyvBYkMUacKZRftzGZ2TwTwZ
6PXa3TI6zFQSL7Nb+Y7ybFhKWZ6jy8TQ8iXXLERxlk//Mg7+0EKNFIACfF7+iiuB
kOyfRSIYYQ3YhdoDUHAxe44eStPlvbMK+DR1SI96gC75Q3fxN73Tj45VCvvglSYt
tx3xPx6kQjseXodjzElYPEnjTSBADY0Ep8QPCZH1Q0fCWfh4xm/nScxNvvdlhc2q
uBV5mmciUrIbJycIcYKnlcUiWz0JkEPjEVu4LORlReFrKsVlTt7JYAuFAhOHxxP9
eroL8R2Fc5vQ/c/hM97oU2xdOrulTQ==
-----END CERTIFICATE REQUEST-----

27
idk/testenv/certs/pilosa-tls.key
View file

@ -1,27 +0,0 @@
-----BEGIN RSA PRIVATE KEY-----
MIIEpAIBAAKCAQEA376aaKuisoFjsc176huaH/Wh3/kg2CMgyhtDoEgsbMp5YgfQ
GkRVH3B1NS0YzbP+Ry3w6LpGJs6X30uPR3+ex4qf2SOoxd0kSejKihAEj75CtbDJ
gdnvpqD0fWCakKpRAguJsUZTL1YtG4ltssAZh8WxvZqIhIQgXBuEWDa4UUPxbtUb
M4cJd8BL6iJhIU/cT2l103huS6uZ66Iv6VfEUFMv5GGyx0E2t6JLiknbW9s8nDTG
nBTtFUHZpI1Wp91XNcgzCeM9f3Op4ZRy8kBwj72RwwvAabKeNpnWJ1U9umZKGEAc
mpH9H/WvbSLfZQTqXlIcFZ5b6f/ZZvllcdPqWQIDAQABAoIBADNQ7OsqymL0iCAE
h/IWqI/B74GGCwFndSLFhAoj44SEH7jbH+CWYYuXaFps4G51ZNGAeOt3pZVK9sww
8UitvYX4hlbv9cqDwMnuyV1G9TTep0AyrtTIXk2yRsDmwiyB05iLeuYcwgFuW2Qg
bfd8VZ6tOenJopv6Dc3yRbVRv45rgAfcwRxhDa8xaSVf1jAlXT0+8iPQFX0OHsv6
iWXk1c48BFEL4nQEtxcWquHqX676raDDizpeVUrnLhRzx0nYq8hiEdtm+O0VY77i
v4i9/vBuWEVVA3OkeufkkkiGuNYZyiDd1a8CZA07aJAMoSpK8pHBjT4N0j5z/lY9
+AvGV3ECgYEA6V2qkvwjyn+aQgU/Ma2q+mOWa/GdA13vsCFcRfh57UYL95QhVezu
HPk8KskzA6W2CwhG6nhUCfHLlYKqGvozRiaNsnp5dqPMOuLdNlo5ehziIp2FhSM+
2OmIdAHNzlWp0WaLBbNQQVddmQY1jJPmJitBmQuAs0k6g6uSaD++lfUCgYEA9XIL
d5dkKiK5w4pvAQ7tdszF2YRZvCY77J2cZKmNRO0OI10bU+m321xes+VuFap9VpRn
MTadYeZT9w3gyPX2rjU6bhKx1NKn8oqWOPmOsR+Rhc3OtEFt2fiMtIsNtH5Bd18f
VuSqri6NvGIJJjZ496gV7YtDr1VvxboaSpunoFUCgYEAgPdMnvJM7UnbfATeSZwK
U1nZ9UmPVh8BqTqmfdy1tRTy5B2u1oebh9ONFrAeSzO3CR3H4yggeWZw9e1CCKqg
z3Ha+PmKSrB220woqDLwdLWlmy+mbE9wGfvldwbQL+lTce/TgJD6F0Iq7bhqu4e4
tvtMw0GCHJKpjb6xqH8h2kkCgYAJ+xY+Sbi4Aet96R40c8/yOcd93eTsQr3DFHGs
ojxtZhpiE8Itul5QnEP5msXMOb+ZCrXbiXYC6iPV8wmGAoKCeQWkPjxfBCXyNfJ/
5J4J09fOEh1qtRJrf7DxEop/IXc2DHcwyGGsbZYz5+SpWiO3Jm2l2DjrY1UFPFkf
WY8KdQKBgQCtsRcM9Yhpid/wljem283Ea418nrIAY7zw4/JY4XRbaqxOW/YptQq3
YHQP/jQb1eFeszLwNAorczdXbms/bie2u1+S+vG13z4IgmMtUVXMn8OMN9XYRGXk
UZOtUsJtD8jEEivLoeUN2vdq4oQXWimjVPlh7KS4Qu/W2r0CO2VPBg==
-----END RSA PRIVATE KEY-----

25
idk/testenv/certs/theclient.crt
View file

@ -1,25 +0,0 @@
-----BEGIN CERTIFICATE-----
MIIENjCCAh6gAwIBAgIQSrfnXrPnUTXzmeGaoP31NjANBgkqhkiG9w0BAQsFADAN
MQswCQYDVQQDEwJjYTAgFw0yMjA3MjIxOTUxNDVaGA8yMTIyMDcyMjE5NTE0Mlow
FDESMBAGA1UEAxMJdGhlY2xpZW50MIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIB
CgKCAQEAudP6EFxKU3gcyd3NYIS7n5drg8Dwg6FyngyHsQM8l+EROAXep45iIJvu
QNiorUo+dUg5YlxRc1ElqpLi7oSrTZtRZ6EO7pyR44RJtZ9tJ6QjLSh9QaGFq0/2
GbWQKJJlPT4ftftTdk8ioe3TTDlSmZEGBY1+6Z3c8xxk98VoYJkB86KsLtC9b8q9
m4D3LWI4+J4echhM6x9YlHiq8HXsPHyBg51Kla3Q3jl1aDHfXuuvEiOVqxJEVx8e
11GPhYmFqQ561id9/S6uBc+u4DnNJUZ4xdA/uEo6PP4XgKBviAtaZvQGGBgtoKAr
tGBFF1QCDkq0QXvv65aArZnXyo+0ewIDAQABo4GIMIGFMA4GA1UdDwEB/wQEAwID
uDAdBgNVHSUEFjAUBggrBgEFBQcDAQYIKwYBBQUHAwIwHQYDVR0OBBYEFN7YKblI
oBtbbLrJaa4XNWrnykw3MB8GA1UdIwQYMBaAFPjbQzymtVM3npTWBLoQ+bJzlfmq
MBQGA1UdEQQNMAuCCXRoZWNsaWVudDANBgkqhkiG9w0BAQsFAAOCAgEA0usMDZ0b
Ktazcxfd7yDgmuIXy06R6fyt/wJjRdPo8oi96a0b9Zj0+BTP2HjkM8/lPO5F8HkX
Ct5tBvuVtwHiTK5fHVsAsSH9H/G9JbX+q7EF3JWx1DDYgXyhU25mcegEkyRZ+hUe
2T3nItwUYjntVoaw4zBS8awTKc78eDZQ3UV3GlGvY3nejYdhpMkGg9dJ4y4M/HGC
P1VuzEpvT7il0cCGMDpuXcdMKNvljmcZYUEC0LCAADTpgfBPGP6lZaB0kU0mpP+e
Lhl6gwZm/toT3u+MGBECUrzHhNvav9LHHBFkquhVopNEEfpBHPtMD8pYm8ULDd5w
2cy53B9m+p3a6jjB85SL9kYudrjMOCbKbABiLxUABfH+5cV9lITNm9GdwuqqtBqL
H4lBAe8wVTf+u4v14diIJXSIjw+mSf7+4htSTTB8BdBmNShmwlCfJA/PXmJqRkVl
I8KE0OTFxHM0rndawXJjFCeNpbHYlTmVW1jtgu3Oz3tqKNwZgHd8kFCwCEkJtno5
OkKamF1AAzAQELOqzIksIxRPde7egqQ3BfsHkOBbAdybBnUKGRhKf5/rA9SZFfe4
ncujFadlq+sUk60TTCZaCyMIxDgbb1vQk+7pf3RoB2BDLsfNCqo8rdAhcrB9/8sQ
04HBTyQF+FSJH4bStq5Zm+jMDcb9tx3EAVs=
-----END CERTIFICATE-----

16
idk/testenv/certs/theclient.csr
View file

@ -1,16 +0,0 @@
-----BEGIN CERTIFICATE REQUEST-----
MIICgDCCAWgCAQAwFDESMBAGA1UEAxMJdGhlY2xpZW50MIIBIjANBgkqhkiG9w0B
AQEFAAOCAQ8AMIIBCgKCAQEAudP6EFxKU3gcyd3NYIS7n5drg8Dwg6FyngyHsQM8
l+EROAXep45iIJvuQNiorUo+dUg5YlxRc1ElqpLi7oSrTZtRZ6EO7pyR44RJtZ9t
J6QjLSh9QaGFq0/2GbWQKJJlPT4ftftTdk8ioe3TTDlSmZEGBY1+6Z3c8xxk98Vo
YJkB86KsLtC9b8q9m4D3LWI4+J4echhM6x9YlHiq8HXsPHyBg51Kla3Q3jl1aDHf
XuuvEiOVqxJEVx8e11GPhYmFqQ561id9/S6uBc+u4DnNJUZ4xdA/uEo6PP4XgKBv
iAtaZvQGGBgtoKArtGBFF1QCDkq0QXvv65aArZnXyo+0ewIDAQABoCcwJQYJKoZI
hvcNAQkOMRgwFjAUBgNVHREEDTALggl0aGVjbGllbnQwDQYJKoZIhvcNAQELBQAD
ggEBAAqCJIhdEa+y7/KwjLWdHeG4+kE0ergbmIEab4rswrUxCNq/8wedAASXGJ/O
Fj3WgxnpNnIV8MazGDbiRke498EiTiUnGQ20t0o8NXYxYxkk9SWW+DzOTojRBRjx
0HLFCImymdu8B7S/pBdzLO3RcCpa6iRdwGd/9TJgxhKf0NBC/fyZJ+fHhYGZNdt9
U4g60/hJAD1XGBC3Ry0p8DfTpC3beb6RP0cJNT8YX4qMjq27nuQuaRKcdIV4jZa1
rms2VspHpOPt+w+XoGFul1lfUut/nFlu5X2aTgnFNHTPZ7eDBVTBCdX1puvTvslg
gzN8dqH97Qyg3lOLYtHUii9dm5M=
-----END CERTIFICATE REQUEST-----

27
idk/testenv/certs/theclient.key
View file

@ -1,27 +0,0 @@
-----BEGIN RSA PRIVATE KEY-----
MIIEogIBAAKCAQEAudP6EFxKU3gcyd3NYIS7n5drg8Dwg6FyngyHsQM8l+EROAXe
p45iIJvuQNiorUo+dUg5YlxRc1ElqpLi7oSrTZtRZ6EO7pyR44RJtZ9tJ6QjLSh9
QaGFq0/2GbWQKJJlPT4ftftTdk8ioe3TTDlSmZEGBY1+6Z3c8xxk98VoYJkB86Ks
LtC9b8q9m4D3LWI4+J4echhM6x9YlHiq8HXsPHyBg51Kla3Q3jl1aDHfXuuvEiOV
qxJEVx8e11GPhYmFqQ561id9/S6uBc+u4DnNJUZ4xdA/uEo6PP4XgKBviAtaZvQG
GBgtoKArtGBFF1QCDkq0QXvv65aArZnXyo+0ewIDAQABAoIBAGXNQh46YSGPGL5i
O4QX0y2f52+bgGesR71cAe4wZr6RtzwowwyqDQnuQrzVtLqShNr9gSpohtqU5vym
UZn3plzyrMsHycVgku/AZh0QxNzFp9uyZcRRt4fdW2ZEOEBxyvKdGQQ6NkQHIGLz
hiEo+h5U+6aJGLDlIckxkgLG6+KWoSUH1sm+Xl9CiP4st/m7gHiqBHz6dQTDVbmC
2uJJut3rlMjK2dzTxB+5mGvzknJfaqHg8ImXSv9kUpYBqYiDNQv2qNB2AolAKKt6
52Y8+T9iGrX78wrCfcal2cbhWe3lwnaIJ3ueeyt+5hPs8yvtq7vrrhG6d8KjiCZZ
oHPYVcECgYEA3D7jJ33oeX9/5iwBzMXAER6LFZlPDFN6LfwuvdrrO4UD4zw2NB9f
/yHXgpy1sgcSpweqEolTyQSUjqD5imJnIOujfYkrqmL6g1UlfKNwT/DtnCGAnnr2
fwq8RB/5omuuWaQn3a61RG7lXWTE2akFd0+u7bH1qYqADmMJLOu22JsCgYEA1/69
VnYETuETr8sCZe7Dj031w3Rc8VZWBgDO3G07tOPyadcAvAJWy/TxPQSDcKky2E0h
yO8Tw2mIxhYvLkyieok9hV4pckdaAgka8utOY0EzWTyJ64hQGRvbN6G+Tdqo2k/o
yTdhtRfYnRsZZcNdlQ0A4jkvzVAO8mU/lTSIoaECgYBDM9kh6rQtpm7DL7hY4BoL
3zL3IzFQhIQqVeF5Qrw2DA80neG4I76p3YyhqOlz6d6SIq+NOmPbWU9u+cVKqV3o
WJ4pJtqxAcSIoElw73nMmqoV0s8pDEH8X600aEFGgu7PIVycsrtSpzfLOH0sIPuu
/RxWIM/hrIYIXcWl9EJ1cwKBgGl9uOtpaGE9/Gd3NwyhuFFdOhLj2iDkkc8MxftS
Pj7ytYqLv4gTfGDQmKG8epWjjXmm8kE4wuZFvM9CNnjeZNyD+tAQVR3DS2qDF+5c
jkRbq8CBmLq4MKaSvFZHYLKVgLJfJMphig7wPpM8kQLqj+IkH7JdS8NyvloUqSEu
Ly0BAoGATwLAQ5bZ1zT+ul4zxR3vkIS7YgTcEHa5OzszKVQ6iDArRDa7NWC8R06j
1R2Ur+T3UVINVFsChDsbebb6AUoSCzM48ZDvnX6C12T0vx+wLziw29hN84gAa+1f
kgFJZ+uena+ZZc8MLyOcM6E1nW0BpAXE0yVF4Vn2Df/wNzm9cGY=
-----END RSA PRIVATE KEY-----

57
index.go
View file

@ -24,8 +24,11 @@ import (
// Index represents a container for fields.
type Index struct {
mu sync.RWMutex
createdAt int64
mu sync.RWMutex
createdAt int64
owner string
description string
path string
name string
qualifiedName string
@ -148,6 +151,7 @@ func (i *Index) Options() IndexOptions {
func (i *Index) options() IndexOptions {
return IndexOptions{
Description: i.description,
Keys: i.keys,
TrackExistence: i.trackExistence,
}
@ -348,6 +352,7 @@ func (i *Index) openExistenceField() error {
cfm := &CreateFieldMessage{
Index: i.name,
Field: existenceFieldName,
Owner: "",
CreatedAt: 0,
Meta: &FieldOptions{Type: FieldTypeSet, CacheType: CacheTypeNone, CacheSize: 0},
}
@ -524,7 +529,7 @@ func (i *Index) recalculateCaches() {
}
// CreateField creates a field.
func (i *Index) CreateField(name string, opts ...FieldOption) (*Field, error) {
func (i *Index) CreateField(name string, requestUserID string, opts ...FieldOption) (*Field, error) {
err := ValidateName(name)
if err != nil {
return nil, errors.Wrap(err, "validating name")
@ -553,10 +558,12 @@ func (i *Index) CreateField(name string, opts ...FieldOption) (*Field, error) {
return nil, errors.Wrap(err, "applying option")
}
ts := timestamp()
cfm := &CreateFieldMessage{
Index: i.name,
Field: name,
CreatedAt: timestamp(),
CreatedAt: ts,
Owner: requestUserID,
Meta: fo,
}
@ -585,7 +592,7 @@ func (i *Index) CreateField(name string, opts ...FieldOption) (*Field, error) {
}
// CreateFieldIfNotExists creates a field with the given options if it doesn't exist.
func (i *Index) CreateFieldIfNotExists(name string, opts ...FieldOption) (*Field, error) {
func (i *Index) CreateFieldIfNotExists(name string, requestUserID string, opts ...FieldOption) (*Field, error) {
err := ValidateName(name)
if err != nil {
return nil, errors.Wrap(err, "validating name")
@ -605,10 +612,12 @@ func (i *Index) CreateFieldIfNotExists(name string, opts ...FieldOption) (*Field
return nil, errors.Wrap(err, "applying option")
}
ts := timestamp()
cfm := &CreateFieldMessage{
Index: i.name,
Field: name,
CreatedAt: timestamp(),
CreatedAt: ts,
Owner: requestUserID,
Meta: fo,
}
@ -629,7 +638,7 @@ func (i *Index) CreateFieldIfNotExists(name string, opts ...FieldOption) (*Field
// function options, taking a *FieldOptions struct. TODO: This should
// definintely be refactored so we don't have these virtually equivalent
// methods, but I'm puttin this here for now just to see if it works.
func (i *Index) CreateFieldIfNotExistsWithOptions(name string, opt *FieldOptions) (*Field, error) {
func (i *Index) CreateFieldIfNotExistsWithOptions(name string, requestUserID string, opt *FieldOptions) (*Field, error) {
err := ValidateName(name)
if err != nil {
return nil, errors.Wrap(err, "validating name")
@ -680,10 +689,12 @@ func (i *Index) CreateFieldIfNotExistsWithOptions(name string, opt *FieldOptions
}
}
ts := timestamp()
cfm := &CreateFieldMessage{
Index: i.name,
Field: name,
CreatedAt: timestamp(),
CreatedAt: ts,
Owner: requestUserID,
Meta: opt,
}
@ -712,7 +723,7 @@ func (i *Index) persistField(ctx context.Context, cfm *CreateFieldMessage) error
}
if b, err := i.serializer.Marshal(cfm); err != nil {
return errors.Wrap(err, "marshaling")
return errors.Wrap(err, "marshaling field")
} else if err := i.holder.Schemator.CreateField(ctx, cfm.Index, cfm.Field, b); errors.Cause(err) == disco.ErrFieldExists {
return ErrFieldExists
} else if err != nil {
@ -729,7 +740,7 @@ func (i *Index) persistUpdateField(ctx context.Context, cfm *CreateFieldMessage)
}
if b, err := i.serializer.Marshal(cfm); err != nil {
return errors.Wrap(err, "marshaling")
return errors.Wrap(err, "marshaling field")
} else if err := i.holder.Schemator.UpdateField(ctx, cfm.Index, cfm.Field, b); errors.Cause(err) == disco.ErrFieldDoesNotExist {
return ErrFieldNotFound
} else if err != nil {
@ -738,7 +749,7 @@ func (i *Index) persistUpdateField(ctx context.Context, cfm *CreateFieldMessage)
return nil
}
func (i *Index) UpdateField(ctx context.Context, name string, update FieldUpdate) (*CreateFieldMessage, error) {
func (i *Index) UpdateField(ctx context.Context, name string, requestUserID string, update FieldUpdate) (*CreateFieldMessage, error) {
// Get field from etcd
buf, err := i.holder.Schemator.Field(ctx, i.name, name)
if err != nil {
@ -783,6 +794,9 @@ func (i *Index) UpdateField(ctx context.Context, name string, update FieldUpdate
return nil, errors.Wrap(err, "persisting updated field")
}
i.mu.Lock()
defer i.mu.Unlock()
return cfm, nil
}
@ -843,6 +857,7 @@ func (i *Index) createField(cfm *CreateFieldMessage) (*Field, error) {
return nil, errors.Wrap(err, "initializing")
}
f.createdAt = cfm.CreatedAt
f.owner = cfm.Owner
// Pass holder through to the field for use in looking
// up a foreign index.
@ -928,11 +943,14 @@ func (p indexSlice) Less(i, j int) bool { return p[i].Name() < p[j].Name() }
// IndexInfo represents schema information for an index.
type IndexInfo struct {
Name string `json:"name"`
CreatedAt int64 `json:"createdAt,omitempty"`
Options IndexOptions `json:"options"`
Fields []*FieldInfo `json:"fields"`
ShardWidth uint64 `json:"shardWidth"`
Name string `json:"name"`
CreatedAt int64 `json:"createdAt,omitempty"`
UpdatedAt int64 `json:"updatedAt"`
Owner string `json:"owner"`
LastUpdateUser string `json:"lastUpdatedUser"`
Options IndexOptions `json:"options"`
Fields []*FieldInfo `json:"fields"`
ShardWidth uint64 `json:"shardWidth"`
}
// Field returns the FieldInfo the provided field name. If the field does not
@ -954,9 +972,10 @@ func (p indexInfoSlice) Less(i, j int) bool { return p[i].Name < p[j].Name }
// IndexOptions represents options to set when initializing an index.
type IndexOptions struct {
Keys bool `json:"keys"`
TrackExistence bool `json:"trackExistence"`
PartitionN int `json:"partitionN"`
Keys bool `json:"keys"`
TrackExistence bool `json:"trackExistence"`
PartitionN int `json:"partitionN"`
Description string `json:"description"`
}
type importData struct {

2
index_internal_test.go
View file

@ -9,7 +9,7 @@ import (
// mustOpenIndex returns a new, opened index at a temporary path. Panic on error.
func mustOpenIndex(tb testing.TB, opt IndexOptions) *Index {
h := newTestHolder(tb)
index, err := h.CreateIndex("i", opt)
index, err := h.CreateIndex("i", "", opt)
if err != nil {
panic(err)

20
index_test.go
View file

@ -26,7 +26,7 @@ func TestIndex_CreateFieldIfNotExists(t *testing.T) {
_, index := test.MustOpenIndex(t)
// Create field.
f, err := index.CreateFieldIfNotExists("f")
f, err := index.CreateFieldIfNotExists("f", "")
if err != nil {
t.Fatal(err)
} else if f == nil {
@ -34,7 +34,7 @@ func TestIndex_CreateFieldIfNotExists(t *testing.T) {
}
// Retrieve existing field.
other, err := index.CreateFieldIfNotExists("f")
other, err := index.CreateFieldIfNotExists("f", "")
if err != nil {
t.Fatal(err)
} else if f.Field != other.Field {
@ -53,7 +53,7 @@ func TestIndex_CreateField(t *testing.T) {
_, index := test.MustOpenIndex(t)
// Create field with explicit quantum.
f, err := index.CreateField("f", pilosa.OptFieldTypeTime(pilosa.TimeQuantum("YMDH"), "0"))
f, err := index.CreateField("f", "", pilosa.OptFieldTypeTime(pilosa.TimeQuantum("YMDH"), "0"))
if err != nil {
t.Fatal(err)
} else if q := f.TimeQuantum(); q != pilosa.TimeQuantum("YMDH") {
@ -68,7 +68,7 @@ func TestIndex_CreateField(t *testing.T) {
_, index := test.MustOpenIndex(t)
// Create field with explicit quantum with no standard view
f, err := index.CreateField("f", pilosa.OptFieldTypeTime(pilosa.TimeQuantum("YMDH"), "0", true))
f, err := index.CreateField("f", "", pilosa.OptFieldTypeTime(pilosa.TimeQuantum("YMDH"), "0", true))
if err != nil {
t.Fatal(err)
} else if q := f.TimeQuantum(); q != pilosa.TimeQuantum("YMDH") {
@ -83,7 +83,7 @@ func TestIndex_CreateField(t *testing.T) {
_, index := test.MustOpenIndex(t)
// Create field with schema and verify it exists.
if f, err := index.CreateField("f", pilosa.OptFieldTypeInt(-990, 1000)); err != nil {
if f, err := index.CreateField("f", "", pilosa.OptFieldTypeInt(-990, 1000)); err != nil {
t.Fatal(err)
} else if !reflect.DeepEqual(f.Type(), pilosa.FieldTypeInt) {
t.Fatalf("unexpected type: %#v", f.Type())
@ -101,7 +101,7 @@ func TestIndex_CreateField(t *testing.T) {
_, index := test.MustOpenIndex(t)
// Create field with schema and verify it exists.
if f, err := index.CreateField("f", pilosa.OptFieldTypeTimestamp(pilosa.DefaultEpoch, pilosa.TimeUnitSeconds)); err != nil {
if f, err := index.CreateField("f", "", pilosa.OptFieldTypeTimestamp(pilosa.DefaultEpoch, pilosa.TimeUnitSeconds)); err != nil {
t.Fatal(err)
} else if !reflect.DeepEqual(f.Type(), pilosa.FieldTypeTimestamp) {
t.Fatalf("unexpected type: %#v", f.Type())
@ -193,7 +193,7 @@ func TestIndex_CreateField(t *testing.T) {
t.Run("IntField", func(t *testing.T) {
_, index := test.MustOpenIndex(t)
_, err := index.CreateField("f", pilosa.OptFieldTypeInt(-1, 1), pilosa.OptFieldKeys())
_, err := index.CreateField("f", "", pilosa.OptFieldTypeInt(-1, 1), pilosa.OptFieldKeys())
if errors.Cause(err) != pilosa.ErrIntFieldWithKeys {
t.Fatal("int field cannot be created with keys=true")
}
@ -203,7 +203,7 @@ func TestIndex_CreateField(t *testing.T) {
t.Run("DecimalField", func(t *testing.T) {
_, index := test.MustOpenIndex(t)
_, err := index.CreateField("f", pilosa.OptFieldTypeDecimal(1, pql.NewDecimal(-1, 0), pql.NewDecimal(1, 0)), pilosa.OptFieldKeys())
_, err := index.CreateField("f", "", pilosa.OptFieldTypeDecimal(1, pql.NewDecimal(-1, 0), pql.NewDecimal(1, 0)), pilosa.OptFieldKeys())
if errors.Cause(err) != pilosa.ErrDecimalFieldWithKeys {
t.Fatal("decimal field cannot be created with keys=true")
}
@ -216,7 +216,7 @@ func TestIndex_DeleteField(t *testing.T) {
_, index := test.MustOpenIndex(t)
// Create field.
if _, err := index.CreateFieldIfNotExists("f"); err != nil {
if _, err := index.CreateFieldIfNotExists("f", ""); err != nil {
t.Fatal(err)
}
@ -263,7 +263,7 @@ func TestIndex_RecreateFieldOnRestart(t *testing.T) {
// create index
indexName := fmt.Sprintf("idx_%d", rand.Uint64())
holder := c.GetHolder(0)
_, err := holder.CreateIndex(indexName, pilosa.IndexOptions{
_, err := holder.CreateIndex(indexName, "", pilosa.IndexOptions{
Keys: false,
})
if err != nil {

10
internal_client_test.go
View file

@ -854,7 +854,7 @@ func TestClient_ImportKeys(t *testing.T) {
// Load bitmap into cache to ensure cache gets updated.
index := hldr.MustCreateIndexIfNotExists(cluster.Idx(), pilosa.IndexOptions{Keys: true})
_, err := index.CreateFieldIfNotExists(fldName, pilosa.OptFieldTypeInt(-100, 100))
_, err := index.CreateFieldIfNotExists(fldName, "", pilosa.OptFieldTypeInt(-100, 100))
if err != nil {
t.Fatal(err)
}
@ -933,7 +933,7 @@ func TestClient_ImportIDs(t *testing.T) {
// Load bitmap into cache to ensure cache gets updated.
index := hldr.MustCreateIndexIfNotExists(idxName, pilosa.IndexOptions{Keys: false})
_, err := index.CreateFieldIfNotExists(fldName, pilosa.OptFieldTypeInt(-10000, 10000))
_, err := index.CreateFieldIfNotExists(fldName, "", pilosa.OptFieldTypeInt(-10000, 10000))
if err != nil {
t.Fatal(err)
}
@ -1001,7 +1001,7 @@ func TestClient_ImportValue(t *testing.T) {
// Load bitmap into cache to ensure cache gets updated.
index := hldr.MustCreateIndexIfNotExists(cluster.Idx(), pilosa.IndexOptions{})
_, err := index.CreateFieldIfNotExists(fldName, pilosa.OptFieldTypeInt(-100, 100))
_, err := index.CreateFieldIfNotExists(fldName, "", pilosa.OptFieldTypeInt(-100, 100))
if err != nil {
t.Fatal(err)
}
@ -1112,7 +1112,7 @@ func TestClient_ImportExistence(t *testing.T) {
fldName := "fset"
index := hldr.MustCreateIndexIfNotExists(idxName, pilosa.IndexOptions{TrackExistence: true})
_, err := index.CreateFieldIfNotExists(fldName)
_, err := index.CreateFieldIfNotExists(fldName, "")
if err != nil {
t.Fatal(err)
}
@ -1148,7 +1148,7 @@ func TestClient_ImportExistence(t *testing.T) {
fldName := "fint"
index := hldr.MustCreateIndexIfNotExists(idxName, pilosa.IndexOptions{TrackExistence: true})
_, err := index.CreateFieldIfNotExists(fldName, pilosa.OptFieldTypeInt(-100, 100))
_, err := index.CreateFieldIfNotExists(fldName, "", pilosa.OptFieldTypeInt(-100, 100))
if err != nil {
t.Fatal(err)
}

BIN
lattice/public/favicon.ico Executable file → Normal file
View file

Binary file not shown.
Side by side Swipe Overlay

Before

Width:  |  Height:  |  Size: 15 KiB

After

Width:  |  Height:  |  Size: 15 KiB

BIN
lattice/public/favicon.png
View file

Binary file not shown.
Side by side Swipe Overlay

Before

Width:  |  Height:  |  Size: 8 KiB

After

Width:  |  Height:  |  Size: 1.2 KiB

7
lattice/public/favicon.svg
View file

@ -1,7 +0,0 @@
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 50 50">
<style>.m-blue { fill: #004379; } @media (prefers-color-scheme: dark) { .m-blue { fill: #ffffff; } }
</style>
<path d="M25.46 1.32a2.38 2.38 0 10-2.38 4.13l17.62 10.2V36a2.38 2.38 0 004.76 0V12.9z" fill="#c3d3dc"/>
<path d="M12.78 7.62a2.37 2.37 0 00.85 3.24l17.64 10.22v20.37a2.39 2.39 0 004.77 0V18.33L16 6.75a2.4 2.4 0 00-3.22.87z" fill="#48b5cd"/>
<path d="M3.32 13.09A2.35 2.35 0 003 14.22V36a2.4 2.4 0 004.8 0V18.43l4.62 2.68v20.36a2.38 2.38 0 004.76 0v-17.6l4.66 2.7v20.32a2.39 2.39 0 004.77 0V23.82l-20-11.6a2.39 2.39 0 00-3.27.84z" class="m-blue"/>
</svg>
Side by side

Before

Width:  |  Height:  |  Size: 636 B

3
lattice/public/index.html
View file

@ -2,7 +2,6 @@
<html lang="en">
<head>
<meta charset="utf-8" />
<link rel="icon" href="%PUBLIC_URL%/favicon.svg" type="image/svg+xml" />
<link rel="icon" href="%PUBLIC_URL%/favicon.png" type="image/png" />
<meta
name="viewport"
@ -24,7 +23,7 @@
work correctly both with client-side routing and a non-root public URL.
Learn how to configure a non-root public URL by running `npm run build`.
-->
<title>Molecula</title>
<title>FeatureBase</title>
</head>
<body>
<noscript>You need to enable JavaScript to run this app.</noscript>

4
lattice/public/manifest.json
View file

@ -1,6 +1,6 @@
{
"short_name": "Molecula",
"name": "Molecula Demo App",
"short_name": "FeatureBase",
"name": "FeatureBase Demo App",
"icons": [
{
"src": "favicon.ico",

2
lattice/src/App/AuthFlow/AuthFlow.module.scss
View file

@ -16,7 +16,7 @@
}
.logo {
height: 85px;
height: 50px;
margin: 16px;
}

4
lattice/src/App/AuthFlow/Signin.tsx
View file

@ -2,7 +2,7 @@ import Card from '@material-ui/core/Card';
import CardContent from '@material-ui/core/CardContent';
import CardHeader from '@material-ui/core/CardHeader';
import { ReactComponent as MLogo } from 'assets/m-bug-alt.svg';
import { ReactComponent as FeatureBaseIcon } from 'assets/featurebase-icon.svg';
import css from './AuthFlow.module.scss';
import SignInButton from './SignInButton';
@ -20,7 +20,7 @@ function Signin(props) {
<div className={css.main}>
<div className={css.loginForm}>
<div className={css.logoContainer}>
<MLogo className={css.logo} />
<FeatureBaseIcon className={css.logo} />
</div>
{renderLoginForm()}
</div>

2
lattice/src/App/Home/ClusterHealth/Metrics/Metrics.tsx
View file

@ -239,7 +239,7 @@ export const Metrics: FC<MetricsProps> = ({ open, node, onClose }) => {
</DialogTitle>
<DialogContent>
<div className={css.infoMessage}>
Molecula provides metrics for use with Prometheus. Instantaneous
FeatureBase provides metrics for use with Prometheus. Instantaneous
metrics are shown here for convenience; to take full advantage of this
feature, consider using a Prometheus instance, perhaps with a{' '}
<Link

4
lattice/src/App/Home/Home.tsx
View file

@ -10,8 +10,8 @@ export const Home: FC = () => (
<Typography variant="h6" color="textPrimary" align="center" style={{padding: 30, paddingBottom: 0}}>
Try FeatureBase Cloud at
<span> </span>
<Link href="https://app.molecula.cloud/" underline="always">
app.molecula.cloud
<Link href="https://cloud.featurebase.com/" underline="always">
cloud.featurebase.com
</Link>
</Typography>
<ClusterHealth/>

2
lattice/src/App/QueryBuilder/QueryBuilderContainer.tsx
View file

@ -113,7 +113,7 @@ export const QueryBuilderContainer = () => {
type: 'text/plain;charset=utf-8',
});
element.href = URL.createObjectURL(file);
element.download = `molecula-${results?.index}-${dateTime}.csv`;
element.download = `featurebase-${results?.index}-${dateTime}.csv`;
document.body.appendChild(element);
element.click();
exportRows = [];

1
lattice/src/assets/darkTheme/MoleculaBug.svg
View file

@ -1 +0,0 @@
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 42.46 48.12"><defs><style>.mbug-cls-1{fill:#ffffff;}.mbug-cls-2{fill:#ffffff;}.mbug-cls-3{fill:#ffffff;}</style></defs><title>MoleculaBug</title><g id="Layer_2" data-name="Layer 2"><g id="Layer_1-2" data-name="Layer 1"><g id="art"><path class="mbug-cls-1" d="M22.46.32a2.38,2.38,0,1,0-2.38,4.13L37.7,14.65V35a2.38,2.38,0,0,0,4.76,0V11.9Z"/><path class="mbug-cls-2" d="M9.78,6.62a2.37,2.37,0,0,0,.85,3.24h0L28.27,20.08V40.45a2.39,2.39,0,0,0,4.77,0V17.33L13,5.75A2.39,2.39,0,0,0,9.78,6.62Z"/><path class="mbug-cls-3" d="M.32,12.09A2.35,2.35,0,0,0,0,13.22V35a2.4,2.4,0,0,0,4.8,0V17.43l4.62,2.68V40.47a2.38,2.38,0,0,0,4.76,0V22.87l4.66,2.7V45.89a2.39,2.39,0,0,0,4.77,0V22.82l-20-11.6a2.38,2.38,0,0,0-3.27.84Z"/></g></g></g></svg>
Side by side

Before

Width:  |  Height:  |  Size: 779 B

1
lattice/src/assets/darkTheme/MoleculaLogo.svg
View file

@ -1 +0,0 @@
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 265.39 48.27"><defs><style>.molecula-1{fill:#ffffff;}.molecula-2{fill:#c3d3dc;}.molecula-3{fill:#48b5cd;}</style></defs><title>Molecula Logo</title><g id="Layer_2" data-name="Layer 2"><g id="art"><path class="molecula-1" d="M55.15,13.89a2.18,2.18,0,0,1,2.19-2.2h.46a2.29,2.29,0,0,1,2,1.17l7.51,11.76,7.54-11.8a2.2,2.2,0,0,1,2-1.13h.46a2.18,2.18,0,0,1,2.19,2.2V34.64a2.15,2.15,0,0,1-2.19,2.16,2.19,2.19,0,0,1-2.16-2.16V19.77l-6.06,9.1a2,2,0,0,1-3.65,0l-6-9V34.68a2.11,2.11,0,0,1-2.16,2.12,2.08,2.08,0,0,1-2.12-2.12Z"/><path class="molecula-1" d="M85.68,24.3v-.07a13,13,0,0,1,25.92-.07v.07a13,13,0,0,1-25.92.07Zm21.35,0v-.07c0-4.82-3.5-8.82-8.42-8.82s-8.36,3.93-8.36,8.75v.07A8.46,8.46,0,0,0,98.68,33C103.6,33,107,29.12,107,24.3Z"/><path class="molecula-1" d="M117.77,13.82a2.18,2.18,0,0,1,4.35,0V32.66h11.23a2,2,0,0,1,0,4H120a2.16,2.16,0,0,1-2.19-2.2Z"/><path class="molecula-1" d="M140.58,34.43V14a2.16,2.16,0,0,1,2.19-2.2h14.42a2,2,0,0,1,1.95,1.95,2,2,0,0,1-1.95,2H144.93v6.44h10.66a2,2,0,0,1,1.95,2,1.93,1.93,0,0,1-1.95,1.91H144.93v6.66h12.43a1.95,1.95,0,0,1,0,3.9H142.77A2.16,2.16,0,0,1,140.58,34.43Z"/><path class="molecula-1" d="M164.09,24.3v-.07a12.55,12.55,0,0,1,12.79-12.82,13,13,0,0,1,8.5,2.8,2.16,2.16,0,0,1,.85,1.73,2.17,2.17,0,0,1-3.47,1.7,9,9,0,0,0-5.92-2.23c-4.71,0-8.18,3.9-8.18,8.75v.07c0,4.85,3.44,8.78,8.18,8.78a8.87,8.87,0,0,0,6.2-2.37,2,2,0,0,1,1.31-.5,2,2,0,0,1,1.35,3.58,12.64,12.64,0,0,1-9,3.33A12.47,12.47,0,0,1,164.09,24.3Z"/><path class="molecula-1" d="M191.83,26.11V13.82a2.18,2.18,0,0,1,4.35,0V25.93c0,4.6,2.38,7,6.27,7s6.24-2.3,6.24-6.87V13.82a2.18,2.18,0,0,1,4.35,0V25.89C213,33.3,208.86,37,202.38,37S191.83,33.3,191.83,26.11Z"/><path class="molecula-1" d="M219.91,13.82a2.18,2.18,0,0,1,4.36,0V32.66H235.5a2,2,0,0,1,0,4H222.11a2.17,2.17,0,0,1-2.2-2.2Z"/><path class="molecula-1" d="M241.31,33.79l9.21-20.54a2.75,2.75,0,0,1,2.62-1.77h.21a2.7,2.7,0,0,1,2.58,1.77l9.21,20.54a1.88,1.88,0,0,1,.25.92,2,2,0,0,1-2.05,2.09,2.25,2.25,0,0,1-2.13-1.52l-2-4.64h-12l-2.09,4.78a2.09,2.09,0,0,1-2,1.38,2,2,0,0,1-2-2A2.22,2.22,0,0,1,241.31,33.79Zm16.22-7-4.36-10-4.36,10Z"/><path class="molecula-1" d="M22.46.32a2.38,2.38,0,0,0-2.38,4.13L37.7,14.65V35a2.38,2.38,0,0,0,4.76,0V11.9Z"/><path class="molecula-1" d="M9.78,6.62a2.37,2.37,0,0,0,.87,3.25L28.27,20.08V40.45a2.39,2.39,0,0,0,4.77,0V17.33L13,5.75A2.39,2.39,0,0,0,9.78,6.62Z"/><path class="molecula-1" d="M.32,12.09A2.33,2.33,0,0,0,0,13.22C0,13.43,0,35,0,35A2.39,2.39,0,0,0,4.8,35V17.43l4.62,2.68V40.47a2.38,2.38,0,0,0,4.76,0V22.87l4.66,2.7V45.89a2.39,2.39,0,0,0,4.77,0V22.82l-20-11.6A2.39,2.39,0,0,0,.32,12.09Z"/></g></g></svg>
Side by side

Before

Width:  |  Height:  |  Size: 2.6 KiB

1
lattice/src/assets/darkTheme/featurebase-logo.svg Normal file
View file

File diff suppressed because one or more lines are too long
Side by side

After

Width:  |  Height:  |  Size: 7.4 KiB

1
lattice/src/assets/featurebase-icon.svg Normal file
View file

File diff suppressed because one or more lines are too long
Side by side

After

Width:  |  Height:  |  Size: 5.2 KiB

1
lattice/src/assets/lightTheme/MoleculaBug.svg
View file

@ -1 +0,0 @@
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 42.46 48.12"><defs><style>.cls-1{fill:#c3d3dc;}.cls-2{fill:#48b5cd;}.cls-3{fill:#004379;}</style></defs><title>MoleculaBug</title><g id="Layer_2" data-name="Layer 2"><g id="Layer_1-2" data-name="Layer 1"><g id="art"><path class="cls-1" d="M22.46.32a2.38,2.38,0,1,0-2.38,4.13L37.7,14.65V35a2.38,2.38,0,0,0,4.76,0V11.9Z"/><path class="cls-2" d="M9.78,6.62a2.37,2.37,0,0,0,.85,3.24h0L28.27,20.08V40.45a2.39,2.39,0,0,0,4.77,0V17.33L13,5.75A2.39,2.39,0,0,0,9.78,6.62Z"/><path class="cls-3" d="M.32,12.09A2.35,2.35,0,0,0,0,13.22V35a2.4,2.4,0,0,0,4.8,0V17.43l4.62,2.68V40.47a2.38,2.38,0,0,0,4.76,0V22.87l4.66,2.7V45.89a2.39,2.39,0,0,0,4.77,0V22.82l-20-11.6a2.38,2.38,0,0,0-3.27.84Z"/></g></g></g></svg>
Side by side

Before

Width:  |  Height:  |  Size: 749 B

1
lattice/src/assets/lightTheme/MoleculaLogo.svg
View file

@ -1 +0,0 @@
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 265.39 48.27"><defs><style>.molecula-1{fill:#004379;}.molecula-2{fill:#c3d3dc;}.molecula-3{fill:#48b5cd;}</style></defs><title>Molecula Logo</title><g id="Layer_2" data-name="Layer 2"><g id="art"><path class="molecula-1" d="M55.15,13.89a2.18,2.18,0,0,1,2.19-2.2h.46a2.29,2.29,0,0,1,2,1.17l7.51,11.76,7.54-11.8a2.2,2.2,0,0,1,2-1.13h.46a2.18,2.18,0,0,1,2.19,2.2V34.64a2.15,2.15,0,0,1-2.19,2.16,2.19,2.19,0,0,1-2.16-2.16V19.77l-6.06,9.1a2,2,0,0,1-3.65,0l-6-9V34.68a2.11,2.11,0,0,1-2.16,2.12,2.08,2.08,0,0,1-2.12-2.12Z"/><path class="molecula-1" d="M85.68,24.3v-.07a13,13,0,0,1,25.92-.07v.07a13,13,0,0,1-25.92.07Zm21.35,0v-.07c0-4.82-3.5-8.82-8.42-8.82s-8.36,3.93-8.36,8.75v.07A8.46,8.46,0,0,0,98.68,33C103.6,33,107,29.12,107,24.3Z"/><path class="molecula-1" d="M117.77,13.82a2.18,2.18,0,0,1,4.35,0V32.66h11.23a2,2,0,0,1,0,4H120a2.16,2.16,0,0,1-2.19-2.2Z"/><path class="molecula-1" d="M140.58,34.43V14a2.16,2.16,0,0,1,2.19-2.2h14.42a2,2,0,0,1,1.95,1.95,2,2,0,0,1-1.95,2H144.93v6.44h10.66a2,2,0,0,1,1.95,2,1.93,1.93,0,0,1-1.95,1.91H144.93v6.66h12.43a1.95,1.95,0,0,1,0,3.9H142.77A2.16,2.16,0,0,1,140.58,34.43Z"/><path class="molecula-1" d="M164.09,24.3v-.07a12.55,12.55,0,0,1,12.79-12.82,13,13,0,0,1,8.5,2.8,2.16,2.16,0,0,1,.85,1.73,2.17,2.17,0,0,1-3.47,1.7,9,9,0,0,0-5.92-2.23c-4.71,0-8.18,3.9-8.18,8.75v.07c0,4.85,3.44,8.78,8.18,8.78a8.87,8.87,0,0,0,6.2-2.37,2,2,0,0,1,1.31-.5,2,2,0,0,1,1.35,3.58,12.64,12.64,0,0,1-9,3.33A12.47,12.47,0,0,1,164.09,24.3Z"/><path class="molecula-1" d="M191.83,26.11V13.82a2.18,2.18,0,0,1,4.35,0V25.93c0,4.6,2.38,7,6.27,7s6.24-2.3,6.24-6.87V13.82a2.18,2.18,0,0,1,4.35,0V25.89C213,33.3,208.86,37,202.38,37S191.83,33.3,191.83,26.11Z"/><path class="molecula-1" d="M219.91,13.82a2.18,2.18,0,0,1,4.36,0V32.66H235.5a2,2,0,0,1,0,4H222.11a2.17,2.17,0,0,1-2.2-2.2Z"/><path class="molecula-1" d="M241.31,33.79l9.21-20.54a2.75,2.75,0,0,1,2.62-1.77h.21a2.7,2.7,0,0,1,2.58,1.77l9.21,20.54a1.88,1.88,0,0,1,.25.92,2,2,0,0,1-2.05,2.09,2.25,2.25,0,0,1-2.13-1.52l-2-4.64h-12l-2.09,4.78a2.09,2.09,0,0,1-2,1.38,2,2,0,0,1-2-2A2.22,2.22,0,0,1,241.31,33.79Zm16.22-7-4.36-10-4.36,10Z"/><path class="molecula-2" d="M22.46.32a2.38,2.38,0,0,0-2.38,4.13L37.7,14.65V35a2.38,2.38,0,0,0,4.76,0V11.9Z"/><path class="molecula-3" d="M9.78,6.62a2.37,2.37,0,0,0,.87,3.25L28.27,20.08V40.45a2.39,2.39,0,0,0,4.77,0V17.33L13,5.75A2.39,2.39,0,0,0,9.78,6.62Z"/><path class="molecula-1" d="M.32,12.09A2.33,2.33,0,0,0,0,13.22C0,13.43,0,35,0,35A2.39,2.39,0,0,0,4.8,35V17.43l4.62,2.68V40.47a2.38,2.38,0,0,0,4.76,0V22.87l4.66,2.7V45.89a2.39,2.39,0,0,0,4.77,0V22.82l-20-11.6A2.39,2.39,0,0,0,.32,12.09Z"/></g></g></svg>
Side by side

Before

Width:  |  Height:  |  Size: 2.6 KiB

1
lattice/src/assets/lightTheme/featurebase-logo.svg Normal file
View file

File diff suppressed because one or more lines are too long
Side by side

After

Width:  |  Height:  |  Size: 8.4 KiB

16
lattice/src/assets/m-bug-alt.svg
View file

@ -1,16 +0,0 @@
<?xml version="1.0" encoding="UTF-8"?>
<svg viewBox="0 0 276 313" version="1.1" xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink">
<defs>
<polygon id="path-1" points="0 0.35 275.231 0.35 275.231 312.94 0 312.94"></polygon>
</defs>
<g id="Group-3">
<mask id="mask-2" fill="white">
<use xlink:href="#path-1"></use>
</mask>
<g id="Clip-2"></g>
<path d="M275.231,86.005 C275.231,81.175 271.809,75.246 267.625,72.831 L145.222,2.162 C141.038,-0.254 134.193,-0.254 130.01,2.162 L7.606,72.831 C3.423,75.246 0,81.175 0,86.005 L0,227.345 C0,232.175 3.423,238.103 7.606,240.518 L130.01,311.188 C134.193,313.603 141.038,313.603 145.222,311.188 L267.625,240.518 C271.809,238.103 275.231,232.175 275.231,227.345 L275.231,86.005" id="Fill-1" fill="#084E83" mask="url(#mask-2)"></path>
</g>
<path d="M143.926,42.423 C138.489,39.277 131.526,41.13 128.376,46.569 C125.226,52.008 127.082,58.97 132.52,62.119 L216.669,110.849 L216.669,208.127 C216.669,214.412 221.764,219.507 228.049,219.507 C234.334,219.507 239.429,214.412 239.429,208.127 L239.429,97.728 L143.926,42.423" id="Fill-4" fill="#C3D4DD"></path>
<path d="M83.364,72.494 C80.213,77.933 82.07,84.895 87.508,88.044 L171.657,136.774 L171.657,234.053 C171.657,240.338 176.751,245.433 183.036,245.433 C189.321,245.433 194.416,240.338 194.416,234.053 L194.416,123.653 L98.913,68.349 C93.477,65.202 86.513,67.055 83.364,72.494" id="Fill-5" fill="#48B5CD"></path>
<path d="M38.185,98.634 C37.203,100.33 36.708,102.173 36.656,104.003 C36.633,105.008 36.819,208.157 36.819,208.157 C36.819,214.442 41.913,219.537 48.199,219.537 C54.484,219.537 59.578,214.442 59.578,208.157 L59.578,124.13 L81.632,136.915 L81.632,234.127 C81.632,240.412 86.727,245.507 93.012,245.507 C99.297,245.507 104.392,240.412 104.392,234.127 L104.392,150.11 L126.645,163.011 L126.645,260.017 C126.645,266.301 131.739,271.396 138.024,271.396 C144.309,271.396 149.404,266.301 149.404,260.017 L149.404,149.846 L53.735,94.489 C48.299,91.342 41.334,93.196 38.185,98.634" id="Fill-6" fill="#FFFFFF"></path>
</svg>
Side by side

Before

Width:  |  Height:  |  Size: 2.1 KiB

4
lattice/src/lattice.config.js
View file

@ -1,4 +1,4 @@
export const pilosaConfig = {
// hostname: 'localhost',
// httpPort: '10101'
hostname: 'localhost',
httpPort: '10101'
};

34
lattice/src/shared/Header/Header.tsx
View file

@ -1,17 +1,17 @@
import SignOutButton from "App/AuthFlow/SignOutButton";
import { ReactComponent as MoleculaLogoDark } from "assets/darkTheme/MoleculaLogo.svg";
import { ReactComponent as MoleculaLogo } from "assets/lightTheme/MoleculaLogo.svg";
import { FC } from "react";
import { Link } from "react-router-dom";
import { useAuth } from "services/useAuth";
import { ThemeToggle } from "shared/ThemeToggle";
import SignOutButton from 'App/AuthFlow/SignOutButton';
import { ReactComponent as FeatureBaseLogoDark } from 'assets/darkTheme/featurebase-logo.svg';
import { ReactComponent as FeatureBaseLogo } from 'assets/lightTheme/featurebase-logo.svg';
import { FC } from 'react';
import { Link } from 'react-router-dom';
import { useAuth } from 'services/useAuth';
import { ThemeToggle } from 'shared/ThemeToggle';
import AppBar from "@material-ui/core/AppBar";
import Button from "@material-ui/core/Button";
import { useTheme } from "@material-ui/core/styles";
import Toolbar from "@material-ui/core/Toolbar";
import AppBar from '@material-ui/core/AppBar';
import Button from '@material-ui/core/Button';
import { useTheme } from '@material-ui/core/styles';
import Toolbar from '@material-ui/core/Toolbar';
import css from "./Header.module.scss";
import css from './Header.module.scss';
type HeaderProps = {
onToggleTheme: () => void;
@ -19,7 +19,7 @@ type HeaderProps = {
export const Header: FC<HeaderProps> = ({ onToggleTheme }) => {
const theme = useTheme();
const isDark = theme.palette.type === "dark";
const isDark = theme.palette.type === 'dark';
const auth = useAuth();
return (
@ -32,8 +32,8 @@ export const Header: FC<HeaderProps> = ({ onToggleTheme }) => {
<Toolbar>
<div className={css.toolbarLayout}>
<Link to="/" className={css.homeLink}>
{!isDark && <MoleculaLogo className={css.logo} />}
{isDark && <MoleculaLogoDark className={css.logo} />}
{!isDark && <FeatureBaseLogo className={css.logo} />}
{isDark && <FeatureBaseLogoDark className={css.logo} />}
</Link>
<div className={css.toolbarActions}>
@ -50,8 +50,8 @@ export const Header: FC<HeaderProps> = ({ onToggleTheme }) => {
{auth.user && (
<Button
style={{
backgroundColor: "transparent",
pointerEvents: "none",
backgroundColor: 'transparent',
pointerEvents: 'none'
}}
>
{auth.user.username}

35
lattice/yarn.lock
View file

@ -2480,12 +2480,12 @@
dependencies:
"@types/react" "*"
"@types/react-dom@^16.9.8":
version "16.9.8"
resolved "https://registry.yarnpkg.com/@types/react-dom/-/react-dom-16.9.8.tgz#fe4c1e11dfc67155733dfa6aa65108b4971cb423"
integrity sha512-ykkPQ+5nFknnlU6lDd947WbQ6TE3NNzbQAkInC2EKY1qeYdTKp7onFusmYZb+ityzx2YviqT6BXSu+LyWWJwcA==
"@types/react-dom@^17.0.0":
version "17.0.18"
resolved "https://registry.yarnpkg.com/@types/react-dom/-/react-dom-17.0.18.tgz#8f7af38f5d9b42f79162eea7492e5a1caff70dc2"
integrity sha512-rLVtIfbwyur2iFKykP2w0pl/1unw26b5td16d5xMgp7/yjTHomkyxPYChFoCr/FtEX1lN9wY6lFj1qvKdS5kDw==
dependencies:
"@types/react" "*"
"@types/react" "^17"
"@types/react-router-dom@^5.1.5":
version "5.1.5"
@ -2511,13 +2511,14 @@
dependencies:
"@types/react" "*"
"@types/react@*", "@types/react@^16.9.41":
version "16.9.41"
resolved "https://registry.yarnpkg.com/@types/react/-/react-16.9.41.tgz#925137ee4d2ff406a0ecf29e8e9237390844002e"
integrity sha512-6cFei7F7L4wwuM+IND/Q2cV1koQUvJ8iSV+Gwn0c3kvABZ691g7sp3hfEQHOUBJtccl1gPi+EyNjMIl9nGA0ug==
"@types/react@*", "@types/react@^17", "@types/react@^17.0.0":
version "17.0.52"
resolved "https://registry.yarnpkg.com/@types/react/-/react-17.0.52.tgz#10d8b907b5c563ac014a541f289ae8eaa9bf2e9b"
integrity sha512-vwk8QqVODi0VaZZpDXQCmEmiOuyjEFPY7Ttaw5vjM112LOq37yz1CDJGrRJwA1fYEq4Iitd5rnjd1yWAc/bT+A==
dependencies:
"@types/prop-types" "*"
csstype "^2.2.0"
"@types/scheduler" "*"
csstype "^3.0.2"
"@types/resolve@0.0.8":
version "0.0.8"
@ -2526,6 +2527,11 @@
dependencies:
"@types/node" "*"
"@types/scheduler@*":
version "0.16.2"
resolved "https://registry.yarnpkg.com/@types/scheduler/-/scheduler-0.16.2.tgz#1a62f89525723dde24ba1b01b092bf5df8ad4d39"
integrity sha512-hppQEBDmlwhFAXKJX2KnWLYu5yMfi91yazPb2l+lbJiwW+wdo1gNeRA+3RgNSO39WYX2euey41KEwnqesU2Jew==
"@types/source-list-map@*":
version "0.1.2"
resolved "https://registry.yarnpkg.com/@types/source-list-map/-/source-list-map-0.1.2.tgz#0078836063ffaf17412349bba364087e0ac02ec9"
@ -4773,7 +4779,7 @@ cssstyle@^2.2.0:
dependencies:
cssom "~0.3.6"
csstype@^2.2.0, csstype@^2.5.2, csstype@^2.6.5, csstype@^2.6.7:
csstype@^2.5.2, csstype@^2.6.5, csstype@^2.6.7:
version "2.6.11"
resolved "https://registry.yarnpkg.com/csstype/-/csstype-2.6.11.tgz#452f4d024149ecf260a852b025e36562a253ffc5"
integrity sha512-l8YyEC9NBkSm783PFTvh0FmJy7s5pFKrDp49ZL7zBGX3fWkO+N4EEyan1qqp8cwPLDcD0OSdyY6hAMoxp34JFw==
@ -4783,6 +4789,11 @@ csstype@^2.5.5:
resolved "https://registry.yarnpkg.com/csstype/-/csstype-2.6.13.tgz#a6893015b90e84dd6e85d0e3b442a1e84f2dbe0f"
integrity sha512-ul26pfSQTZW8dcOnD2iiJssfXw0gdNVX9IJDH/X3K5DGPfj+fUYe3kB+swUY6BF3oZDxaID3AJt+9/ojSAE05A==
csstype@^3.0.2:
version "3.1.1"
resolved "https://registry.yarnpkg.com/csstype/-/csstype-3.1.1.tgz#841b532c45c758ee546a11d5bd7b7b473c8c30b9"
integrity sha512-DJR/VvkAvSZW9bTouZue2sSxDwdTN92uHjqeKVm+0dAqdfNykRzQ95tay8aXMBAAPpUiq4Qcug2L7neoRh2Egw==
currently-unhandled@^0.4.1:
version "0.4.1"
resolved "https://registry.yarnpkg.com/currently-unhandled/-/currently-unhandled-0.4.1.tgz#988df33feab191ef799a61369dd76c17adf957ea"
@ -8504,7 +8515,7 @@ lodash@^4.0.0, lodash@~4.17.10:
resolved "https://registry.yarnpkg.com/lodash/-/lodash-4.17.19.tgz#e48ddedbe30b3321783c5b4301fbd353bc1e4a4b"
integrity sha512-JNvd8XER9GQX0v2qJgsaN/mzFCNA5BRe/j8JN9d+tWyGLSodKQHKFicdwNYzWwI3wjRnaKPsGj1XkBjx/F96DQ==
lodash@^4.17.19:
lodash@^4.17.19, lodash@^4.17.21:
version "4.17.21"
resolved "https://registry.yarnpkg.com/lodash/-/lodash-4.17.21.tgz#679591c564c3bffaae8454cf0b3df370c3d6911c"
integrity sha512-v2kDEe57lecTulaDIuNTPy3Ry4gLGJ6Z1O3vE1krgXZNrsQ+LFTGHVxVjcXPs17LhbZVGedAJv8XZ1tvj5FvSg==

370
pb/private.pb.go
View file

@ -25,6 +25,7 @@ const _ = proto.ProtoPackageIsVersion3 // please upgrade the proto package
type IndexMeta struct {
Keys bool `protobuf:"varint,3,opt,name=Keys,proto3" json:"Keys,omitempty"`
TrackExistence bool `protobuf:"varint,4,opt,name=TrackExistence,proto3" json:"TrackExistence,omitempty"`
Description string `protobuf:"bytes,5,opt,name=Description,proto3" json:"Description,omitempty"`
XXX_NoUnkeyedLiteral struct{} `json:"-"`
XXX_unrecognized []byte `json:"-"`
XXX_sizecache int32 `json:"-"`
@ -77,6 +78,13 @@ func (m *IndexMeta) GetTrackExistence() bool {
return false
}
func (m *IndexMeta) GetDescription() string {
if m != nil {
return m.Description
}
return ""
}
type FieldOptions struct {
Type string `protobuf:"bytes,8,opt,name=Type,proto3" json:"Type,omitempty"`
CacheType string `protobuf:"bytes,3,opt,name=CacheType,proto3" json:"CacheType,omitempty"`
@ -633,6 +641,7 @@ type CreateIndexMessage struct {
Index string `protobuf:"bytes,1,opt,name=Index,proto3" json:"Index,omitempty"`
Meta *IndexMeta `protobuf:"bytes,2,opt,name=Meta,proto3" json:"Meta,omitempty"`
CreatedAt int64 `protobuf:"varint,3,opt,name=CreatedAt,proto3" json:"CreatedAt,omitempty"`
Owner string `protobuf:"bytes,5,opt,name=Owner,proto3" json:"Owner,omitempty"`
XXX_NoUnkeyedLiteral struct{} `json:"-"`
XXX_unrecognized []byte `json:"-"`
XXX_sizecache int32 `json:"-"`
@ -692,11 +701,19 @@ func (m *CreateIndexMessage) GetCreatedAt() int64 {
return 0
}
func (m *CreateIndexMessage) GetOwner() string {
if m != nil {
return m.Owner
}
return ""
}
type CreateFieldMessage struct {
Index string `protobuf:"bytes,1,opt,name=Index,proto3" json:"Index,omitempty"`
Field string `protobuf:"bytes,2,opt,name=Field,proto3" json:"Field,omitempty"`
Meta *FieldOptions `protobuf:"bytes,3,opt,name=Meta,proto3" json:"Meta,omitempty"`
CreatedAt int64 `protobuf:"varint,4,opt,name=CreatedAt,proto3" json:"CreatedAt,omitempty"`
Owner string `protobuf:"bytes,5,opt,name=Owner,proto3" json:"Owner,omitempty"`
XXX_NoUnkeyedLiteral struct{} `json:"-"`
XXX_unrecognized []byte `json:"-"`
XXX_sizecache int32 `json:"-"`
@ -763,6 +780,13 @@ func (m *CreateFieldMessage) GetCreatedAt() int64 {
return 0
}
func (m *CreateFieldMessage) GetOwner() string {
if m != nil {
return m.Owner
}
return ""
}
type UpdateFieldMessage struct {
CreateFieldMessage *CreateFieldMessage `protobuf:"bytes,1,opt,name=CreateFieldMessage,proto3" json:"CreateFieldMessage,omitempty"`
Update *FieldUpdate `protobuf:"bytes,2,opt,name=Update,proto3" json:"Update,omitempty"`
@ -2879,114 +2903,115 @@ func init() {
func init() { proto.RegisterFile("private.proto", fileDescriptor_d2a91b51c7bdc125) }
var fileDescriptor_d2a91b51c7bdc125 = []byte{
// 1701 bytes of a gzipped FileDescriptorProto
0x1f, 0x8b, 0x08, 0x00, 0x00, 0x00, 0x00, 0x00, 0x02, 0xff, 0xac, 0x58, 0x4b, 0x6f, 0x23, 0x4b,
0x15, 0xa6, 0x1f, 0xf1, 0xe3, 0x38, 0xce, 0x38, 0x75, 0xc3, 0xd0, 0x93, 0x3b, 0x44, 0x9e, 0x02,
0xcd, 0x98, 0x91, 0x08, 0x22, 0x77, 0x71, 0x11, 0x77, 0x73, 0x27, 0x76, 0x66, 0x30, 0xf7, 0xce,
0xe3, 0x56, 0x1e, 0x4b, 0x50, 0xa5, 0x5d, 0x4a, 0x5a, 0x69, 0x77, 0x9b, 0xee, 0x76, 0xc6, 0x9e,
0x05, 0x12, 0x08, 0x04, 0x1b, 0xf6, 0xac, 0xf8, 0x17, 0xfc, 0x01, 0x56, 0x6c, 0x90, 0xf8, 0x09,
0x68, 0xf8, 0x23, 0xa8, 0x4e, 0x55, 0x75, 0x97, 0x1d, 0x27, 0x86, 0x88, 0x5d, 0x9f, 0xef, 0x9c,
0x3a, 0xef, 0x3a, 0x75, 0x6c, 0x68, 0x4f, 0xb2, 0xe8, 0x9a, 0x17, 0x62, 0x7f, 0x92, 0xa5, 0x45,
0x4a, 0xdc, 0xc9, 0xf9, 0xee, 0xe6, 0x64, 0x7a, 0x1e, 0x47, 0xa1, 0x42, 0xe8, 0x2b, 0x68, 0x0e,
0x93, 0x91, 0x98, 0xbd, 0x16, 0x05, 0x27, 0x04, 0xfc, 0xaf, 0xc4, 0x3c, 0x0f, 0xbc, 0xae, 0xd3,
0x6b, 0x30, 0xfc, 0x26, 0x4f, 0x61, 0xeb, 0x24, 0xe3, 0xe1, 0xd5, 0xd1, 0x2c, 0xca, 0x0b, 0x91,
0x84, 0x22, 0xf0, 0x91, 0xbb, 0x84, 0xd2, 0xbf, 0x79, 0xb0, 0xf9, 0x32, 0x12, 0xf1, 0xe8, 0xed,
0xa4, 0x88, 0xd2, 0x24, 0x97, 0xca, 0x4e, 0xe6, 0x13, 0x11, 0x34, 0xba, 0x4e, 0xaf, 0xc9, 0xf0,
0x9b, 0x3c, 0x86, 0x66, 0x9f, 0x87, 0x97, 0x02, 0x19, 0x1e, 0x32, 0x2a, 0xa0, 0xe4, 0x1e, 0x47,
0x1f, 0x94, 0x95, 0x36, 0xab, 0x00, 0xd2, 0x85, 0xd6, 0x49, 0x34, 0x16, 0xdf, 0x4c, 0x79, 0x52,
0x4c, 0xc7, 0xc1, 0x06, 0x9e, 0xb6, 0x21, 0xf2, 0x10, 0x6a, 0x6f, 0xe3, 0xd1, 0xeb, 0x28, 0x09,
0x9a, 0x5d, 0xa7, 0xe7, 0x31, 0x4d, 0x19, 0x9c, 0xcf, 0x02, 0xa8, 0x70, 0x3e, 0x2b, 0xc3, 0x6d,
0x2d, 0x86, 0xfb, 0x26, 0x3d, 0x2e, 0x78, 0x32, 0xe2, 0xd9, 0xe8, 0x2c, 0x12, 0xef, 0x83, 0x4d,
0x15, 0xee, 0x22, 0x2a, 0xcf, 0x1e, 0xf2, 0x5c, 0x04, 0x6d, 0xd4, 0x88, 0xdf, 0x64, 0x17, 0x1a,
0x87, 0x51, 0x31, 0x10, 0x93, 0xe2, 0x32, 0xd8, 0xea, 0x3a, 0x3d, 0x9f, 0x95, 0x34, 0xd9, 0x81,
0x8d, 0xe3, 0x90, 0xc7, 0x22, 0x78, 0x80, 0x07, 0x14, 0x41, 0x28, 0x6c, 0xbe, 0x4c, 0x33, 0x11,
0x5d, 0x24, 0x58, 0x84, 0xa0, 0x83, 0x41, 0x2d, 0x60, 0xe4, 0xbb, 0xe0, 0xc9, 0x90, 0xb6, 0xbb,
0x4e, 0xaf, 0x75, 0xd0, 0xda, 0x9f, 0x9c, 0xef, 0x0f, 0x44, 0x18, 0x8d, 0x79, 0xcc, 0x24, 0x8e,
0x6c, 0x3e, 0x0b, 0xc8, 0x2a, 0x36, 0x9f, 0x49, 0x9f, 0x64, 0x8a, 0x4e, 0x93, 0xa8, 0x08, 0x3e,
0x41, 0xed, 0x25, 0x4d, 0x3a, 0xe0, 0x9d, 0x9c, 0x7c, 0x1d, 0xec, 0x20, 0x2c, 0x3f, 0x29, 0x85,
0xad, 0xe1, 0x78, 0x92, 0x66, 0x05, 0x13, 0xf9, 0x24, 0x4d, 0x72, 0x21, 0x65, 0x8e, 0xb2, 0x2c,
0x70, 0x94, 0xcc, 0x51, 0x96, 0xd1, 0x5f, 0x43, 0xe7, 0x30, 0x4e, 0xc3, 0xab, 0x01, 0x2f, 0x38,
0x13, 0xbf, 0x9a, 0x8a, 0xbc, 0x90, 0xd1, 0xa9, 0x00, 0x94, 0x9c, 0x22, 0x24, 0x8a, 0x1d, 0x11,
0xb8, 0x0a, 0x45, 0x42, 0x66, 0x0e, 0xf3, 0xaa, 0x0a, 0x88, 0xdf, 0x98, 0x9d, 0x4b, 0x9e, 0x8d,
0xb0, 0xea, 0x3e, 0x53, 0x84, 0x44, 0xd1, 0x12, 0x76, 0x8a, 0xcf, 0x14, 0x41, 0x87, 0xb0, 0x6d,
0xd9, 0xd7, 0x6e, 0x3e, 0x84, 0x1a, 0x4b, 0xdf, 0x0f, 0x07, 0x79, 0xe0, 0x74, 0xbd, 0x9e, 0xcf,
0x34, 0x85, 0x2d, 0x95, 0xc6, 0xd3, 0x71, 0x22, 0x59, 0x2e, 0xb2, 0x2a, 0x80, 0x3e, 0x82, 0x0d,
0xec, 0x2f, 0x19, 0x65, 0x75, 0x56, 0x7e, 0xd2, 0xdf, 0x38, 0xd0, 0x7c, 0xcd, 0x67, 0xe8, 0x48,
0x4e, 0x3e, 0x87, 0x86, 0xa9, 0x3e, 0x0a, 0xb5, 0x0e, 0x3e, 0x95, 0x99, 0x2e, 0x05, 0xf6, 0x0d,
0xf7, 0x28, 0x29, 0xb2, 0x39, 0x2b, 0x85, 0x77, 0xbf, 0x80, 0xf6, 0x02, 0x4b, 0x5a, 0xba, 0x12,
0x73, 0x93, 0xcf, 0x2b, 0x31, 0x97, 0x51, 0x5e, 0xf3, 0x78, 0x2a, 0x30, 0x4b, 0x3e, 0x53, 0xc4,
0x4f, 0xdd, 0x9f, 0x38, 0xf4, 0x0c, 0x48, 0x3f, 0x13, 0xbc, 0x10, 0x68, 0xe4, 0xb5, 0xc8, 0x73,
0x7e, 0x21, 0xd6, 0xe5, 0xda, 0xb3, 0x73, 0x5d, 0xe6, 0xd5, 0xb5, 0xf2, 0x4a, 0x9f, 0x03, 0x19,
0x88, 0x58, 0x14, 0x42, 0xdf, 0xfc, 0x3b, 0xf4, 0xd2, 0x2b, 0xe3, 0xc3, 0x7a, 0x59, 0xf2, 0x04,
0x7c, 0x39, 0x46, 0xd0, 0x58, 0xeb, 0xa0, 0x2d, 0x33, 0x54, 0xce, 0x16, 0x86, 0x2c, 0xac, 0x07,
0xaa, 0x1b, 0xbd, 0x28, 0xd0, 0x55, 0x8f, 0x55, 0x00, 0xfd, 0x9d, 0x63, 0xac, 0xa1, 0xfb, 0xff,
0x65, 0xc4, 0x0b, 0xdd, 0xf5, 0x7d, 0xed, 0x83, 0x87, 0x3e, 0x74, 0xa4, 0x0f, 0xf6, 0x54, 0x5a,
0xe5, 0x86, 0xbf, 0xec, 0xc6, 0xef, 0x1d, 0x20, 0xa7, 0x93, 0xd1, 0xb2, 0x1b, 0x2f, 0x57, 0x39,
0x87, 0x3e, 0xb5, 0x0e, 0x1e, 0x4a, 0x43, 0x37, 0xb9, 0x6c, 0x55, 0x38, 0xcf, 0xa0, 0xa6, 0xb4,
0xeb, 0x44, 0x3d, 0x28, 0x9d, 0x54, 0x30, 0xd3, 0x6c, 0xfa, 0x05, 0xb4, 0x2c, 0x18, 0xc7, 0x18,
0x46, 0xa1, 0xf3, 0xa0, 0x29, 0x99, 0x88, 0xb3, 0xb2, 0x81, 0x9a, 0x4c, 0x11, 0xf4, 0x4b, 0x53,
0xe4, 0xfb, 0xa6, 0x92, 0x86, 0xf0, 0xa9, 0xd2, 0xf0, 0xe2, 0x9a, 0x47, 0x31, 0x3f, 0x8f, 0xff,
0xa7, 0x3e, 0x5c, 0xa8, 0x4a, 0x00, 0x75, 0x3c, 0x3b, 0x1c, 0xe8, 0xbb, 0x6c, 0x48, 0x3a, 0x85,
0x6a, 0x2c, 0xbc, 0xe1, 0x63, 0xa1, 0xb5, 0xe1, 0x77, 0x59, 0x4c, 0xf7, 0xce, 0x62, 0xca, 0xf8,
0x23, 0xf1, 0x5e, 0x3e, 0x5b, 0x1e, 0xc6, 0x2f, 0x89, 0x35, 0x25, 0xfe, 0x21, 0xd4, 0x8e, 0xc3,
0x4b, 0x31, 0xe6, 0xe4, 0x7b, 0x50, 0x47, 0xcf, 0x45, 0xae, 0x6f, 0x76, 0xb3, 0xec, 0x5b, 0x66,
0x38, 0xb2, 0x23, 0x74, 0x7c, 0xab, 0xdc, 0x5c, 0x30, 0xe5, 0x2e, 0x99, 0x22, 0xcf, 0xa0, 0xae,
0xfd, 0xc5, 0x91, 0x77, 0xe3, 0x62, 0x18, 0x2e, 0x79, 0x02, 0x35, 0x8c, 0x2e, 0x0f, 0xfc, 0xca,
0x11, 0x44, 0x98, 0x66, 0xd0, 0x23, 0xf0, 0x4e, 0xd9, 0x50, 0x76, 0x02, 0x7a, 0x6f, 0xdc, 0xd0,
0x94, 0x74, 0xee, 0x67, 0x69, 0x5e, 0xe8, 0xdc, 0xe3, 0xb7, 0xc4, 0xde, 0xa5, 0x99, 0xba, 0x6c,
0x6d, 0x86, 0xdf, 0xf4, 0x8f, 0x0e, 0xf8, 0x6f, 0xd2, 0x91, 0x20, 0x5b, 0xe0, 0x0e, 0x07, 0x5a,
0x89, 0x3b, 0x1c, 0x90, 0x47, 0xa8, 0x5f, 0xe7, 0xbb, 0x2e, 0xed, 0x9f, 0xb2, 0x21, 0x43, 0x9b,
0x8f, 0xa1, 0x39, 0xcc, 0xdf, 0x65, 0xd1, 0x98, 0x67, 0x73, 0xbd, 0x20, 0x54, 0x00, 0x0e, 0x9a,
0x42, 0xb6, 0xb4, 0xaf, 0xca, 0x8e, 0x04, 0x79, 0x02, 0xf5, 0x57, 0xec, 0x5d, 0x5f, 0xaa, 0xdc,
0x58, 0x54, 0x69, 0x70, 0xfa, 0x25, 0x74, 0xa4, 0x27, 0x28, 0x6f, 0x3a, 0xeb, 0x21, 0xd4, 0x24,
0x56, 0x7a, 0xa6, 0xa9, 0xca, 0x88, 0x6b, 0x19, 0xa1, 0x2f, 0x95, 0x86, 0xa3, 0x6b, 0x91, 0x14,
0x56, 0x6f, 0x22, 0x8d, 0x0a, 0xda, 0x4c, 0x11, 0xe4, 0xb1, 0x8a, 0x5a, 0x87, 0xd7, 0x90, 0xbe,
0x48, 0x9a, 0x21, 0x4a, 0xe7, 0x00, 0xc6, 0x93, 0x69, 0x5e, 0xca, 0x3a, 0xab, 0x64, 0x09, 0x35,
0xed, 0xa3, 0xe7, 0x0c, 0x48, 0xbe, 0x42, 0x98, 0x69, 0xac, 0x1f, 0x54, 0x8d, 0xa5, 0xea, 0xf9,
0xa0, 0xac, 0xbb, 0xb2, 0x51, 0xb5, 0xd7, 0x25, 0xb4, 0x2c, 0x7c, 0x65, 0x8f, 0x3d, 0x2b, 0x9b,
0xc3, 0xad, 0x94, 0x21, 0xa2, 0x95, 0x69, 0xf6, 0x9a, 0x09, 0x1b, 0xe9, 0x91, 0x72, 0x87, 0xa5,
0x1e, 0x3c, 0x58, 0xbc, 0xf0, 0xe6, 0xe1, 0x5c, 0x86, 0xd7, 0x98, 0xfa, 0x83, 0x03, 0xed, 0x7e,
0x3c, 0xcd, 0x0b, 0x91, 0x95, 0x39, 0x6d, 0x6a, 0xa0, 0x2c, 0x6d, 0x05, 0xac, 0xae, 0x2e, 0xd9,
0x83, 0x0d, 0x99, 0x71, 0x75, 0xb9, 0xed, 0x42, 0x28, 0xd8, 0xaa, 0x84, 0x7f, 0x5b, 0x25, 0xe8,
0x19, 0x34, 0x0e, 0x8f, 0x87, 0xaf, 0xb2, 0x74, 0x3a, 0x59, 0x19, 0xb1, 0xd9, 0x54, 0x5d, 0x6b,
0x53, 0xed, 0xa8, 0xad, 0x4b, 0x45, 0x85, 0x8b, 0x56, 0x47, 0x2d, 0x5a, 0xbe, 0x46, 0xf8, 0x8c,
0x1e, 0xc3, 0xb6, 0x0a, 0x57, 0x4e, 0x9c, 0xfb, 0x8c, 0x45, 0xb3, 0x0a, 0x79, 0xd5, 0x2a, 0x24,
0x95, 0xaa, 0xa9, 0xfb, 0xff, 0x54, 0xfa, 0x0f, 0x17, 0xb6, 0x99, 0xc8, 0xa3, 0x0f, 0x62, 0x98,
0xe4, 0x45, 0x36, 0x0d, 0xcd, 0xc3, 0xf1, 0xf3, 0xf4, 0x5c, 0xd7, 0xc2, 0x63, 0x8a, 0xb8, 0xfb,
0x96, 0x10, 0x0a, 0x75, 0x7b, 0x08, 0xd8, 0x02, 0x86, 0x41, 0x9e, 0x43, 0xfd, 0x38, 0x9d, 0x66,
0x61, 0xd9, 0xf9, 0x38, 0xb9, 0x95, 0x7d, 0xc5, 0x60, 0x46, 0x80, 0x7c, 0x05, 0xe4, 0x24, 0xe3,
0x49, 0x1e, 0x73, 0xe9, 0x92, 0x39, 0xd6, 0xa8, 0x76, 0x2c, 0x8b, 0xbb, 0xa0, 0x61, 0xc5, 0x31,
0xb2, 0x6f, 0x5f, 0xe1, 0xa0, 0x8e, 0xfe, 0x6d, 0x19, 0xff, 0xf4, 0x3d, 0xb1, 0x2f, 0xf9, 0xe7,
0x4b, 0x1d, 0x1a, 0xd4, 0xf0, 0xc8, 0x36, 0x3e, 0xe6, 0x36, 0x83, 0x2d, 0xca, 0xd1, 0xdf, 0x3a,
0xb0, 0x69, 0x7b, 0xb3, 0x66, 0x5c, 0x94, 0xe5, 0x73, 0xd7, 0xaf, 0x6c, 0xa6, 0x7c, 0xfe, 0xaa,
0xf5, 0x78, 0xc3, 0x5e, 0xe3, 0x52, 0xf8, 0xce, 0x2d, 0xc9, 0xb9, 0x97, 0x3b, 0x5d, 0x68, 0xbd,
0xe3, 0x59, 0x11, 0x49, 0x65, 0xfa, 0x9d, 0xde, 0x60, 0x36, 0x44, 0x05, 0x3c, 0xba, 0xd1, 0x44,
0xfd, 0x74, 0x3c, 0x91, 0xdd, 0x7a, 0xaf, 0x66, 0x92, 0x63, 0x3a, 0xcb, 0xd2, 0xcc, 0x64, 0x00,
0x09, 0x7a, 0x08, 0x8d, 0x93, 0x74, 0x92, 0xc6, 0xe9, 0xc5, 0x7c, 0xcd, 0xc8, 0x08, 0xa0, 0xae,
0x9e, 0x06, 0x35, 0xa2, 0x9a, 0xcc, 0x90, 0xf4, 0x13, 0xd9, 0xef, 0x21, 0x8f, 0xc3, 0x69, 0xcc,
0x0b, 0x81, 0x4b, 0x3e, 0x82, 0x5f, 0xa7, 0x7c, 0xa4, 0xa6, 0x82, 0xbe, 0x5a, 0xf4, 0x97, 0xba,
0x01, 0x39, 0x86, 0x63, 0x3d, 0x41, 0x2f, 0x42, 0x7b, 0xd7, 0x52, 0x14, 0xf9, 0x31, 0xb4, 0x2c,
0x69, 0x7b, 0x81, 0xb3, 0x60, 0x66, 0xcb, 0xd0, 0xbf, 0x3a, 0x0b, 0x67, 0x6e, 0xbc, 0xb9, 0xda,
0xd4, 0xb5, 0x4a, 0x52, 0x83, 0x69, 0x4a, 0x86, 0x7e, 0x34, 0x0b, 0xe3, 0x69, 0x2e, 0x59, 0xfa,
0xc1, 0x2d, 0x01, 0x19, 0xba, 0xfc, 0x1d, 0x97, 0x4e, 0xcd, 0x72, 0x63, 0x48, 0xf9, 0x8b, 0x6f,
0x20, 0xf8, 0x28, 0x8e, 0x12, 0x81, 0xfd, 0xe2, 0xb1, 0x92, 0x26, 0xcf, 0xd5, 0x8c, 0x35, 0x8d,
0xbe, 0xb3, 0xe4, 0x38, 0xf2, 0xd4, 0xe4, 0xcd, 0x29, 0x81, 0xce, 0x32, 0x8b, 0xee, 0x00, 0x51,
0x1d, 0xf0, 0xe2, 0x3c, 0xcd, 0xcc, 0x6b, 0x4b, 0xfb, 0x66, 0xb8, 0xc8, 0xec, 0xaf, 0x7b, 0xc4,
0xab, 0xcc, 0xba, 0x76, 0x66, 0xe9, 0x2f, 0x60, 0x4b, 0xef, 0x76, 0x22, 0xc3, 0x86, 0x96, 0x09,
0x60, 0x22, 0x4c, 0xe5, 0x9a, 0x68, 0x7e, 0x9a, 0x55, 0x80, 0xd4, 0x83, 0x8b, 0xae, 0x79, 0x9d,
0x34, 0x85, 0xbb, 0x51, 0x74, 0x91, 0x88, 0x11, 0xbe, 0x18, 0x1e, 0xd3, 0x14, 0xfd, 0x93, 0x0b,
0x3b, 0x6a, 0xe9, 0x4c, 0x2e, 0x44, 0x5e, 0x54, 0x66, 0x70, 0xad, 0xc6, 0xf9, 0x5f, 0xae, 0xd5,
0xf8, 0x02, 0x3c, 0x85, 0xad, 0x7e, 0x2c, 0x78, 0x56, 0xf9, 0xa0, 0x0c, 0x2d, 0xa1, 0xf2, 0xde,
0x20, 0xa2, 0x9f, 0x67, 0xb5, 0x84, 0xda, 0x10, 0x39, 0x84, 0x86, 0x0e, 0xcd, 0x0c, 0xc4, 0xa7,
0xf8, 0x4a, 0xad, 0xf0, 0xc6, 0xec, 0xb7, 0xb9, 0xfe, 0x21, 0x69, 0xc8, 0xdd, 0xb7, 0xd0, 0x5e,
0x60, 0xad, 0xf8, 0x21, 0xd9, 0xb3, 0x7f, 0x48, 0xb6, 0x0e, 0x88, 0xb5, 0x2e, 0x6b, 0xed, 0xf6,
0x8f, 0xcb, 0x3e, 0x7c, 0x7b, 0x95, 0x03, 0x39, 0x79, 0x0e, 0x9e, 0x74, 0x54, 0x2d, 0xc3, 0xc1,
0x6d, 0x8e, 0x32, 0x29, 0x44, 0xff, 0xe2, 0xe8, 0xa4, 0x0a, 0xcd, 0x37, 0x7f, 0x08, 0x7c, 0x66,
0x2b, 0x79, 0x52, 0x2a, 0x59, 0x12, 0xdb, 0x2f, 0x03, 0x95, 0xd2, 0xbb, 0xdf, 0x40, 0x63, 0x55,
0x78, 0xbe, 0x0a, 0xef, 0x47, 0x8b, 0xe1, 0x3d, 0xba, 0xcd, 0xb3, 0xdc, 0x8a, 0xf2, 0xb0, 0xf3,
0xf7, 0x8f, 0x7b, 0xce, 0x3f, 0x3f, 0xee, 0x39, 0xff, 0xfa, 0xb8, 0xe7, 0xfc, 0xf9, 0xdf, 0x7b,
0xdf, 0x3a, 0xaf, 0xe1, 0xff, 0x5e, 0x9f, 0xfd, 0x27, 0x00, 0x00, 0xff, 0xff, 0x76, 0x11, 0x06,
0x2f, 0x1a, 0x13, 0x00, 0x00,
// 1728 bytes of a gzipped FileDescriptorProto
0x1f, 0x8b, 0x08, 0x00, 0x00, 0x00, 0x00, 0x00, 0x02, 0xff, 0xac, 0x58, 0x4f, 0x6f, 0x23, 0x49,
0x15, 0xa7, 0xff, 0xc4, 0x7f, 0x9e, 0xe3, 0x8c, 0x53, 0x1b, 0x86, 0x9e, 0xec, 0x10, 0x79, 0x0a,
0x34, 0x63, 0x46, 0x22, 0x88, 0xec, 0x61, 0x11, 0x7b, 0xd9, 0x89, 0x9d, 0x59, 0xcc, 0xee, 0xcc,
0x64, 0x2b, 0x7f, 0x8e, 0xa0, 0x4a, 0xbb, 0x94, 0xb4, 0xd2, 0xee, 0x36, 0xdd, 0xed, 0xc4, 0xde,
0x03, 0x12, 0x48, 0x08, 0x2e, 0xdc, 0x11, 0x07, 0xbe, 0x05, 0x5f, 0x80, 0x13, 0x17, 0x24, 0x3e,
0x02, 0x1a, 0xbe, 0x08, 0xaa, 0x57, 0x55, 0xdd, 0x65, 0xc7, 0x49, 0x86, 0x68, 0x6f, 0xfd, 0x7e,
0xaf, 0xfa, 0xd5, 0xef, 0xfd, 0xa9, 0x57, 0xaf, 0x1b, 0xda, 0x93, 0x2c, 0xba, 0xe2, 0x85, 0xd8,
0x9d, 0x64, 0x69, 0x91, 0x12, 0x77, 0x72, 0xb6, 0xbd, 0x3e, 0x99, 0x9e, 0xc5, 0x51, 0xa8, 0x10,
0x1a, 0x41, 0x73, 0x98, 0x8c, 0xc4, 0xec, 0x8d, 0x28, 0x38, 0x21, 0xe0, 0x7f, 0x29, 0xe6, 0x79,
0xe0, 0x75, 0x9d, 0x5e, 0x83, 0xe1, 0x33, 0x79, 0x0e, 0x1b, 0xc7, 0x19, 0x0f, 0x2f, 0x0f, 0x66,
0x51, 0x5e, 0x88, 0x24, 0x14, 0x81, 0x8f, 0xda, 0x25, 0x94, 0x74, 0xa1, 0x35, 0x10, 0x79, 0x98,
0x45, 0x93, 0x22, 0x4a, 0x93, 0x60, 0xad, 0xeb, 0xf4, 0x9a, 0xcc, 0x86, 0xe8, 0x3f, 0x3c, 0x58,
0x7f, 0x1d, 0x89, 0x78, 0xf4, 0x0e, 0xe5, 0x5c, 0x6e, 0x77, 0x3c, 0x9f, 0x88, 0xa0, 0x81, 0x6b,
0xf1, 0x99, 0x3c, 0x85, 0x66, 0x9f, 0x87, 0x17, 0x02, 0x15, 0x1e, 0x2a, 0x2a, 0xa0, 0xd4, 0x1e,
0x45, 0xdf, 0x28, 0x1e, 0x6d, 0x56, 0x01, 0x92, 0xc2, 0x71, 0x34, 0x16, 0x5f, 0x4f, 0x79, 0x52,
0x4c, 0xc7, 0x86, 0x82, 0x05, 0x91, 0xc7, 0x50, 0x7b, 0x17, 0x8f, 0xde, 0x44, 0x49, 0xd0, 0xec,
0x3a, 0x3d, 0x8f, 0x69, 0xc9, 0xe0, 0x7c, 0x16, 0x40, 0x85, 0xf3, 0x59, 0x19, 0x90, 0xd6, 0x62,
0x40, 0xde, 0xa6, 0x47, 0x05, 0x4f, 0x46, 0x3c, 0x1b, 0x9d, 0x46, 0xe2, 0x3a, 0x58, 0x57, 0x01,
0x59, 0x44, 0xe5, 0xbb, 0xfb, 0x3c, 0x17, 0x41, 0x1b, 0x2d, 0xe2, 0x33, 0xd9, 0x86, 0xc6, 0x7e,
0x54, 0x0c, 0xc4, 0xa4, 0xb8, 0x08, 0x36, 0xba, 0x4e, 0xcf, 0x67, 0xa5, 0x4c, 0xb6, 0x60, 0xed,
0x28, 0xe4, 0xb1, 0x08, 0x1e, 0xe1, 0x0b, 0x4a, 0x20, 0x14, 0xd6, 0x5f, 0xa7, 0x99, 0x88, 0xce,
0x13, 0x4c, 0x53, 0xd0, 0x41, 0xa7, 0x16, 0x30, 0xf2, 0x7d, 0xf0, 0xa4, 0x4b, 0x9b, 0x5d, 0xa7,
0xd7, 0xda, 0x6b, 0xed, 0x4e, 0xce, 0x76, 0x07, 0x22, 0x8c, 0xc6, 0x3c, 0x66, 0x12, 0x47, 0x35,
0x9f, 0x05, 0x64, 0x95, 0x9a, 0xcf, 0x24, 0x27, 0x19, 0xa2, 0x93, 0x24, 0x2a, 0x82, 0x8f, 0xd0,
0x7a, 0x29, 0x93, 0x0e, 0x78, 0xc7, 0xc7, 0x5f, 0x05, 0x5b, 0x08, 0xcb, 0x47, 0x4a, 0x61, 0x63,
0x38, 0x9e, 0xa4, 0x59, 0xc1, 0x44, 0x3e, 0x49, 0x93, 0x5c, 0xc8, 0x35, 0x07, 0x59, 0x16, 0x38,
0x6a, 0xcd, 0x41, 0x96, 0xd1, 0xdf, 0x42, 0x67, 0x3f, 0x4e, 0xc3, 0xcb, 0x01, 0x2f, 0x38, 0x13,
0xbf, 0x99, 0x8a, 0xbc, 0x90, 0xde, 0x29, 0x07, 0xd4, 0x3a, 0x25, 0x48, 0x14, 0x2b, 0x22, 0x70,
0x15, 0x8a, 0x82, 0x8c, 0x1c, 0xc6, 0x55, 0x25, 0x10, 0x9f, 0x31, 0x3a, 0x17, 0x3c, 0x1b, 0x61,
0xd6, 0x7d, 0xa6, 0x04, 0x89, 0xe2, 0x4e, 0x58, 0x29, 0x3e, 0x53, 0x02, 0x1d, 0xc2, 0xa6, 0xb5,
0xbf, 0xa6, 0xf9, 0x18, 0x6a, 0x2c, 0xbd, 0x1e, 0x0e, 0xf2, 0xc0, 0xe9, 0x7a, 0x3d, 0x9f, 0x69,
0x09, 0x4b, 0x2a, 0x8d, 0xa7, 0xe3, 0x44, 0xaa, 0x5c, 0x54, 0x55, 0x00, 0x7d, 0x02, 0x6b, 0x58,
0x5f, 0xd2, 0xcb, 0xea, 0x5d, 0xf9, 0x48, 0x7f, 0xe7, 0x40, 0xf3, 0x0d, 0x9f, 0x21, 0x91, 0x9c,
0x7c, 0x0a, 0x0d, 0x93, 0x7d, 0x5c, 0xd4, 0xda, 0xfb, 0x58, 0x46, 0xba, 0x5c, 0xb0, 0x6b, 0xb4,
0x07, 0x49, 0x91, 0xcd, 0x59, 0xb9, 0x78, 0xfb, 0x33, 0x68, 0x2f, 0xa8, 0xe4, 0x4e, 0x97, 0x62,
0x6e, 0xe2, 0x79, 0x29, 0xe6, 0xd2, 0xcb, 0x2b, 0x1e, 0x4f, 0x05, 0x46, 0xc9, 0x67, 0x4a, 0xf8,
0xb9, 0xfb, 0x33, 0x87, 0x9e, 0x02, 0xe9, 0x67, 0x82, 0x17, 0x02, 0x37, 0x79, 0x23, 0xf2, 0x9c,
0x9f, 0x8b, 0xfb, 0x62, 0xed, 0xd9, 0xb1, 0x2e, 0xe3, 0xea, 0x5a, 0x71, 0xa5, 0x2f, 0x81, 0x0c,
0x44, 0x2c, 0x0a, 0xa1, 0x7b, 0xc3, 0x1d, 0x76, 0x65, 0x1c, 0x34, 0x89, 0xfb, 0x17, 0x93, 0x67,
0xe0, 0xcb, 0x4e, 0x83, 0xbb, 0xb5, 0xf6, 0xda, 0x32, 0x44, 0x65, 0xfb, 0x61, 0xa8, 0xc2, 0x84,
0xa0, 0xb9, 0xd1, 0xab, 0x02, 0xb9, 0x7a, 0xac, 0x02, 0xa4, 0xd9, 0x77, 0xd7, 0x89, 0xc8, 0x74,
0x71, 0x28, 0x81, 0xfe, 0xb5, 0xe4, 0x80, 0x5e, 0x7d, 0x60, 0x20, 0x16, 0x8a, 0xee, 0x87, 0x9a,
0x99, 0x87, 0xcc, 0x3a, 0x92, 0x99, 0xdd, 0xac, 0x56, 0x91, 0xf3, 0x3f, 0x8c, 0xdc, 0x1f, 0x1c,
0x20, 0x27, 0x93, 0xd1, 0x32, 0xb9, 0xd7, 0xab, 0x28, 0x23, 0xd3, 0xd6, 0xde, 0x63, 0xb9, 0xfd,
0x4d, 0x2d, 0x5b, 0xe5, 0xe4, 0x0b, 0xa8, 0x29, 0xeb, 0x3a, 0xa8, 0x8f, 0x4a, 0xea, 0x0a, 0x66,
0x5a, 0x4d, 0x3f, 0x83, 0x96, 0x05, 0x63, 0xcf, 0x53, 0xbd, 0x5a, 0x45, 0x47, 0x4b, 0xd2, 0x89,
0xd3, 0xb2, 0xda, 0x9a, 0x4c, 0x09, 0xf4, 0x73, 0x53, 0x11, 0x0f, 0x0d, 0x30, 0x0d, 0xe1, 0x63,
0x65, 0xe1, 0xd5, 0x15, 0x8f, 0x62, 0x7e, 0x16, 0xff, 0x5f, 0x45, 0xbb, 0x90, 0xab, 0x00, 0xea,
0xf8, 0xee, 0x70, 0xa0, 0x0f, 0xbe, 0x11, 0xe9, 0x14, 0xaa, 0x1e, 0xf2, 0x96, 0x8f, 0x85, 0xb6,
0x86, 0xcf, 0x65, 0x8a, 0xdd, 0x3b, 0x53, 0x2c, 0xfd, 0x8f, 0xc4, 0xb5, 0xbc, 0x05, 0x3d, 0xf4,
0x5f, 0x0a, 0x77, 0x27, 0x9e, 0xfe, 0x18, 0x6a, 0x47, 0xe1, 0x85, 0x18, 0x73, 0xf2, 0x03, 0xa8,
0x23, 0x73, 0x91, 0xeb, 0x36, 0xd0, 0x2c, 0x6b, 0x9c, 0x19, 0x8d, 0xac, 0x08, 0xed, 0xdf, 0x2a,
0x9a, 0x0b, 0x5b, 0xb9, 0xcb, 0x35, 0xf6, 0x02, 0xea, 0x9a, 0x2f, 0x56, 0xd9, 0x8d, 0x43, 0x64,
0xb4, 0xe4, 0x19, 0xd4, 0xd0, 0xbb, 0x3c, 0xf0, 0x2b, 0x22, 0x88, 0x30, 0xad, 0xa0, 0x07, 0xe0,
0x9d, 0xb0, 0xa1, 0xac, 0x04, 0x64, 0x6f, 0x68, 0x68, 0x49, 0x92, 0xfb, 0x45, 0x9a, 0x17, 0x3a,
0xf6, 0xf8, 0x2c, 0xb1, 0xc3, 0x34, 0x53, 0x07, 0xb3, 0xcd, 0xf0, 0x99, 0xfe, 0xc9, 0x01, 0xff,
0x6d, 0x3a, 0x12, 0x64, 0x03, 0xdc, 0xe1, 0x40, 0x1b, 0x71, 0x87, 0x03, 0xf2, 0x04, 0xed, 0xeb,
0x78, 0xd7, 0xe5, 0xfe, 0x27, 0x6c, 0xc8, 0x70, 0xcf, 0xa7, 0xd0, 0x1c, 0xe6, 0x87, 0x59, 0x34,
0xe6, 0xd9, 0x5c, 0xcf, 0x1b, 0x15, 0x80, 0x5d, 0xa9, 0x90, 0x25, 0xed, 0xab, 0xb4, 0xa3, 0x40,
0x9e, 0x41, 0xfd, 0x0b, 0x76, 0xd8, 0x97, 0x26, 0xd7, 0x16, 0x4d, 0x1a, 0x9c, 0x7e, 0x0e, 0x1d,
0xc9, 0x04, 0xd7, 0x9b, 0xca, 0x7a, 0x0c, 0x35, 0x89, 0x95, 0xcc, 0xb4, 0x54, 0x6d, 0xe2, 0x5a,
0x9b, 0xd0, 0xd7, 0xca, 0xc2, 0xc1, 0x95, 0x48, 0x0a, 0xab, 0x36, 0x51, 0x46, 0x03, 0x6d, 0xa6,
0x04, 0xf2, 0x54, 0x79, 0xad, 0xdd, 0x6b, 0x48, 0x2e, 0x52, 0x66, 0x88, 0xd2, 0x39, 0x80, 0x61,
0x32, 0xcd, 0xcb, 0xb5, 0xce, 0xaa, 0xb5, 0x84, 0x9a, 0xf2, 0xd1, 0xdd, 0x07, 0xa4, 0x5e, 0x21,
0xcc, 0x14, 0xd6, 0x8f, 0xaa, 0xc2, 0x52, 0xf9, 0x7c, 0x54, 0xe6, 0x5d, 0xed, 0x51, 0x95, 0xd7,
0x05, 0xb4, 0x2c, 0x7c, 0x65, 0x8d, 0xbd, 0x28, 0x8b, 0xc3, 0xad, 0x8c, 0x21, 0xa2, 0x8d, 0x69,
0xf5, 0xdd, 0xdd, 0x98, 0x46, 0xba, 0xa5, 0xdc, 0xb1, 0x53, 0x0f, 0x1e, 0x2d, 0x1e, 0x78, 0x73,
0xcb, 0x2e, 0xc3, 0xf7, 0x6c, 0xf5, 0x47, 0x07, 0xda, 0xfd, 0x78, 0x9a, 0x17, 0x22, 0x2b, 0x63,
0xda, 0xd4, 0x40, 0x99, 0xda, 0x0a, 0x58, 0x9d, 0x5d, 0xb2, 0x03, 0x6b, 0x32, 0xe2, 0xea, 0x70,
0xdb, 0x89, 0x50, 0xb0, 0x95, 0x09, 0xff, 0xb6, 0x4c, 0xd0, 0x53, 0x68, 0xec, 0x1f, 0x0d, 0xbf,
0xc8, 0xd2, 0xe9, 0x64, 0xa5, 0xc7, 0x66, 0xac, 0x75, 0xad, 0xb1, 0xb6, 0xa3, 0x46, 0x34, 0xe5,
0x15, 0x4e, 0x65, 0x1d, 0x35, 0x95, 0xf9, 0x1a, 0xe1, 0x33, 0x7a, 0x04, 0x9b, 0xca, 0x5d, 0xd9,
0x71, 0x1e, 0xd2, 0x16, 0xcd, 0xdc, 0xe4, 0x55, 0x73, 0x93, 0x34, 0xaa, 0xba, 0xee, 0xb7, 0x69,
0xf4, 0x5f, 0x2e, 0x6c, 0x32, 0x91, 0x47, 0xdf, 0x88, 0x61, 0x92, 0x17, 0xd9, 0x34, 0x34, 0x17,
0xc7, 0x2f, 0xd3, 0x33, 0x9d, 0x0b, 0x8f, 0x29, 0xe1, 0xee, 0x53, 0x42, 0x28, 0xd4, 0xed, 0x26,
0x60, 0x2f, 0x30, 0x0a, 0xf2, 0x12, 0xea, 0x47, 0xe9, 0x34, 0x0b, 0xcb, 0xca, 0xc7, 0xce, 0xad,
0xf6, 0x57, 0x0a, 0x66, 0x16, 0x90, 0x2f, 0x81, 0x1c, 0x67, 0x3c, 0xc9, 0x63, 0x2e, 0x29, 0x99,
0xd7, 0x1a, 0xd5, 0x40, 0x66, 0x69, 0x17, 0x2c, 0xac, 0x78, 0x8d, 0xec, 0xda, 0x47, 0x38, 0xa8,
0x23, 0xbf, 0x0d, 0xc3, 0x4f, 0x9f, 0x13, 0xfb, 0x90, 0x7f, 0xba, 0x54, 0xa1, 0x41, 0x0d, 0x5f,
0xd9, 0xc4, 0xcb, 0xdc, 0x56, 0xb0, 0xc5, 0x75, 0xf4, 0xf7, 0x0e, 0xac, 0xdb, 0x6c, 0xee, 0x69,
0x17, 0x65, 0xfa, 0xdc, 0xfb, 0xe7, 0x3b, 0x93, 0x3e, 0x7f, 0xd5, 0x2c, 0xbd, 0x66, 0xcf, 0x7c,
0x29, 0x7c, 0xef, 0x96, 0xe0, 0x3c, 0x88, 0x4e, 0x17, 0x5a, 0x87, 0x3c, 0x2b, 0x22, 0x69, 0x4c,
0xdf, 0xd3, 0x6b, 0xcc, 0x86, 0xa8, 0x80, 0x27, 0x37, 0x8a, 0xa8, 0x9f, 0x8e, 0x27, 0xb2, 0x5a,
0x1f, 0x54, 0x4c, 0xb2, 0x4d, 0x67, 0x59, 0x9a, 0x99, 0x08, 0xa0, 0x40, 0xf7, 0xa1, 0x71, 0x9c,
0x4e, 0xd2, 0x38, 0x3d, 0x9f, 0xdf, 0xd3, 0x32, 0x02, 0xa8, 0xab, 0xab, 0x41, 0xb5, 0xa8, 0x26,
0x33, 0x22, 0xfd, 0x48, 0xd6, 0x7b, 0xc8, 0xe3, 0x70, 0x1a, 0xf3, 0x42, 0xe0, 0x17, 0x01, 0x82,
0x5f, 0xa5, 0x7c, 0xa4, 0xba, 0x82, 0x3e, 0x5a, 0xf4, 0xd7, 0xba, 0x00, 0x39, 0xba, 0x63, 0x5d,
0x41, 0xaf, 0x42, 0x7b, 0xd6, 0x52, 0x12, 0xf9, 0x29, 0xb4, 0xac, 0xd5, 0xf6, 0x00, 0x67, 0xc1,
0xcc, 0x5e, 0x43, 0xff, 0xee, 0x2c, 0xbc, 0x73, 0xe3, 0xce, 0xd5, 0x5b, 0x5d, 0xa9, 0x20, 0x35,
0x98, 0x96, 0xa4, 0xeb, 0x07, 0xb3, 0x30, 0x9e, 0xe6, 0x52, 0xa5, 0x2f, 0xdc, 0x12, 0x90, 0xae,
0xcb, 0x8f, 0xbe, 0x74, 0x6a, 0x86, 0x1b, 0x23, 0xca, 0xcf, 0xc3, 0x81, 0xe0, 0xa3, 0x38, 0x4a,
0x04, 0xd6, 0x8b, 0xc7, 0x4a, 0x99, 0xbc, 0x54, 0x3d, 0xd6, 0x14, 0xfa, 0xd6, 0x12, 0x71, 0xd4,
0xa9, 0xce, 0x9b, 0x53, 0x02, 0x9d, 0x65, 0x15, 0xdd, 0x02, 0xa2, 0x2a, 0xe0, 0xd5, 0x59, 0x9a,
0x99, 0xdb, 0x96, 0xf6, 0x4d, 0x73, 0x91, 0xd1, 0xbf, 0xef, 0x12, 0xaf, 0x22, 0xeb, 0xda, 0x91,
0xa5, 0xbf, 0x82, 0x0d, 0x3d, 0xdb, 0x89, 0x0c, 0x0b, 0x5a, 0x06, 0x80, 0x89, 0x30, 0x95, 0x63,
0xa2, 0xf9, 0x8e, 0xab, 0x00, 0x69, 0x07, 0x07, 0x5d, 0x73, 0x3b, 0x69, 0x09, 0x67, 0xa3, 0xe8,
0x3c, 0x11, 0x23, 0xbc, 0x31, 0x3c, 0xa6, 0x25, 0xfa, 0x67, 0x17, 0xb6, 0xd4, 0xd0, 0x99, 0x9c,
0x8b, 0xbc, 0xa8, 0xb6, 0xc1, 0xb1, 0x1a, 0xfb, 0x7f, 0x39, 0x56, 0xe3, 0x0d, 0xf0, 0x1c, 0x36,
0xfa, 0xb1, 0xe0, 0x59, 0xc5, 0x41, 0x6d, 0xb4, 0x84, 0xca, 0x73, 0x83, 0x88, 0xbe, 0x9e, 0xd5,
0x10, 0x6a, 0x43, 0x64, 0x1f, 0x1a, 0xda, 0x35, 0xd3, 0x10, 0x9f, 0xe3, 0x2d, 0xb5, 0x82, 0x8d,
0x99, 0x6f, 0x73, 0xfd, 0xd5, 0x69, 0xc4, 0xed, 0x77, 0xd0, 0x5e, 0x50, 0xad, 0xf8, 0xea, 0xec,
0xd9, 0x5f, 0x9d, 0xad, 0x3d, 0x62, 0x8d, 0xcb, 0xda, 0xba, 0xfd, 0x25, 0xda, 0x87, 0xef, 0xae,
0x22, 0x90, 0x93, 0x97, 0xe0, 0x49, 0xa2, 0x6a, 0x18, 0x0e, 0x6e, 0x23, 0xca, 0xe4, 0x22, 0xfa,
0x37, 0x47, 0x07, 0x55, 0x68, 0xbd, 0xf9, 0x7b, 0xf0, 0x89, 0x6d, 0xe4, 0x59, 0x69, 0x64, 0x69,
0xd9, 0x6e, 0xe9, 0xa8, 0x5c, 0xbd, 0xfd, 0x35, 0x34, 0x56, 0xb9, 0xe7, 0x2b, 0xf7, 0x7e, 0xb2,
0xe8, 0xde, 0x93, 0xdb, 0x98, 0xe5, 0x96, 0x97, 0xfb, 0x9d, 0x7f, 0xbe, 0xdf, 0x71, 0xfe, 0xfd,
0x7e, 0xc7, 0xf9, 0xcf, 0xfb, 0x1d, 0xe7, 0x2f, 0xff, 0xdd, 0xf9, 0xce, 0x59, 0x0d, 0x7f, 0xa3,
0x7d, 0xf2, 0xbf, 0x00, 0x00, 0x00, 0xff, 0xff, 0xa1, 0xcf, 0xef, 0xf0, 0x69, 0x13, 0x00, 0x00,
}
func (m *IndexMeta) Marshal() (dAtA []byte, err error) {
@ -3013,6 +3038,13 @@ func (m *IndexMeta) MarshalToSizedBuffer(dAtA []byte) (int, error) {
i -= len(m.XXX_unrecognized)
copy(dAtA[i:], m.XXX_unrecognized)
}
if len(m.Description) > 0 {
i -= len(m.Description)
copy(dAtA[i:], m.Description)
i = encodeVarintPrivate(dAtA, i, uint64(len(m.Description)))
i--
dAtA[i] = 0x2a
}
if m.TrackExistence {
i--
if m.TrackExistence {
@ -3537,6 +3569,13 @@ func (m *CreateIndexMessage) MarshalToSizedBuffer(dAtA []byte) (int, error) {
i -= len(m.XXX_unrecognized)
copy(dAtA[i:], m.XXX_unrecognized)
}
if len(m.Owner) > 0 {
i -= len(m.Owner)
copy(dAtA[i:], m.Owner)
i = encodeVarintPrivate(dAtA, i, uint64(len(m.Owner)))
i--
dAtA[i] = 0x2a
}
if m.CreatedAt != 0 {
i = encodeVarintPrivate(dAtA, i, uint64(m.CreatedAt))
i--
@ -3588,6 +3627,13 @@ func (m *CreateFieldMessage) MarshalToSizedBuffer(dAtA []byte) (int, error) {
i -= len(m.XXX_unrecognized)
copy(dAtA[i:], m.XXX_unrecognized)
}
if len(m.Owner) > 0 {
i -= len(m.Owner)
copy(dAtA[i:], m.Owner)
i = encodeVarintPrivate(dAtA, i, uint64(len(m.Owner)))
i--
dAtA[i] = 0x2a
}
if m.CreatedAt != 0 {
i = encodeVarintPrivate(dAtA, i, uint64(m.CreatedAt))
i--
@ -5435,6 +5481,10 @@ func (m *IndexMeta) Size() (n int) {
if m.TrackExistence {
n += 2
}
l = len(m.Description)
if l > 0 {
n += 1 + l + sovPrivate(uint64(l))
}
if m.XXX_unrecognized != nil {
n += len(m.XXX_unrecognized)
}
@ -5676,6 +5726,10 @@ func (m *CreateIndexMessage) Size() (n int) {
if m.CreatedAt != 0 {
n += 1 + sovPrivate(uint64(m.CreatedAt))
}
l = len(m.Owner)
if l > 0 {
n += 1 + l + sovPrivate(uint64(l))
}
if m.XXX_unrecognized != nil {
n += len(m.XXX_unrecognized)
}
@ -5703,6 +5757,10 @@ func (m *CreateFieldMessage) Size() (n int) {
if m.CreatedAt != 0 {
n += 1 + sovPrivate(uint64(m.CreatedAt))
}
l = len(m.Owner)
if l > 0 {
n += 1 + l + sovPrivate(uint64(l))
}
if m.XXX_unrecognized != nil {
n += len(m.XXX_unrecognized)
}
@ -6597,6 +6655,38 @@ func (m *IndexMeta) Unmarshal(dAtA []byte) error {
}
}
m.TrackExistence = bool(v != 0)
case 5:
if wireType != 2 {
return fmt.Errorf("proto: wrong wireType = %d for field Description", wireType)
}
var stringLen uint64
for shift := uint(0); ; shift += 7 {
if shift >= 64 {
return ErrIntOverflowPrivate
}
if iNdEx >= l {
return io.ErrUnexpectedEOF
}
b := dAtA[iNdEx]
iNdEx++
stringLen |= uint64(b&0x7F) << shift
if b < 0x80 {
break
}
}
intStringLen := int(stringLen)
if intStringLen < 0 {
return ErrInvalidLengthPrivate
}
postIndex := iNdEx + intStringLen
if postIndex < 0 {
return ErrInvalidLengthPrivate
}
if postIndex > l {
return io.ErrUnexpectedEOF
}
m.Description = string(dAtA[iNdEx:postIndex])
iNdEx = postIndex
default:
iNdEx = preIndex
skippy, err := skipPrivate(dAtA[iNdEx:])
@ -8183,6 +8273,38 @@ func (m *CreateIndexMessage) Unmarshal(dAtA []byte) error {
break
}
}
case 5:
if wireType != 2 {
return fmt.Errorf("proto: wrong wireType = %d for field Owner", wireType)
}
var stringLen uint64
for shift := uint(0); ; shift += 7 {
if shift >= 64 {
return ErrIntOverflowPrivate
}
if iNdEx >= l {
return io.ErrUnexpectedEOF
}
b := dAtA[iNdEx]
iNdEx++
stringLen |= uint64(b&0x7F) << shift
if b < 0x80 {
break
}
}
intStringLen := int(stringLen)
if intStringLen < 0 {
return ErrInvalidLengthPrivate
}
postIndex := iNdEx + intStringLen
if postIndex < 0 {
return ErrInvalidLengthPrivate
}
if postIndex > l {
return io.ErrUnexpectedEOF
}
m.Owner = string(dAtA[iNdEx:postIndex])
iNdEx = postIndex
default:
iNdEx = preIndex
skippy, err := skipPrivate(dAtA[iNdEx:])
@ -8353,6 +8475,38 @@ func (m *CreateFieldMessage) Unmarshal(dAtA []byte) error {
break
}
}
case 5:
if wireType != 2 {
return fmt.Errorf("proto: wrong wireType = %d for field Owner", wireType)
}
var stringLen uint64
for shift := uint(0); ; shift += 7 {
if shift >= 64 {
return ErrIntOverflowPrivate
}
if iNdEx >= l {
return io.ErrUnexpectedEOF
}
b := dAtA[iNdEx]
iNdEx++
stringLen |= uint64(b&0x7F) << shift
if b < 0x80 {
break
}
}
intStringLen := int(stringLen)
if intStringLen < 0 {
return ErrInvalidLengthPrivate
}
postIndex := iNdEx + intStringLen
if postIndex < 0 {
return ErrInvalidLengthPrivate
}
if postIndex > l {
return io.ErrUnexpectedEOF
}
m.Owner = string(dAtA[iNdEx:postIndex])
iNdEx = postIndex
default:
iNdEx = preIndex
skippy, err := skipPrivate(dAtA[iNdEx:])

3
pb/private.proto
View file

@ -7,6 +7,7 @@ import "public.proto";
message IndexMeta {
bool Keys = 3;
bool TrackExistence = 4;
string Description = 5;
}
message FieldOptions {
@ -67,6 +68,7 @@ message CreateIndexMessage {
string Index = 1;
IndexMeta Meta = 2;
int64 CreatedAt = 3;
string Owner = 5;
}
message CreateFieldMessage {
@ -74,6 +76,7 @@ message CreateFieldMessage {
string Field = 2;
FieldOptions Meta = 3;
int64 CreatedAt = 4;
string Owner = 5;
}
message UpdateFieldMessage {

2
pb/public.pb.go
View file

@ -465,6 +465,7 @@ func (m *KeyList) GetKeys() []string {
type ExtractedTableValue struct {
// Types that are valid to be assigned to Value:
//
// *ExtractedTableValue_IDs
// *ExtractedTableValue_Keys
// *ExtractedTableValue_BSIValue
@ -605,6 +606,7 @@ func (*ExtractedTableValue) XXX_OneofWrappers() []interface{} {
type ExtractedTableColumn struct {
// Types that are valid to be assigned to KeyOrID:
//
// *ExtractedTableColumn_Key
// *ExtractedTableColumn_ID
KeyOrID isExtractedTableColumn_KeyOrID `protobuf_oneof:"KeyOrID"`

2461
proto/pilosa.pb.go
View file

File diff suppressed because it is too large Load diff

7
proto/pilosa.proto
View file

@ -1,9 +1,7 @@
syntax = "proto3";
package pilosa;
package proto;
//import "public.proto";
option go_package = "github.com/pilosa/pilosa/v2/proto";
option go_package = "../proto";
message QueryPQLRequest {
string index = 1;
@ -93,6 +91,7 @@ message Index {
message CreateIndexRequest {
string name = 1;
bool keys = 2;
string description = 3;
}
message CreateIndexResponse {

47
qa/scripts/perf/able/ableSetup.sh
View file

@ -20,7 +20,35 @@ pushd ./qa/tf/perf/able
echo "Running terraform init..."
terraform init -input=false
echo "Running terraform apply..."
terraform apply -input=false -auto-approve
okay=false
# This logic is gratuitously complicated because I want visibility
# into how it's working, or not-working. The chances are this should
# just be a test against the exit status of terraform apply.
tries=1
while ! $okay && [ $tries -le $max_tries ] ; do
echo "Try $tries/$max_tries, running terraform..."
terraform apply -input=false -auto-approve
tf=$?
terraform output -json > outputs.json
echo "Outputs:"
cat outputs.json
must_get_value outputs.json TMP_I .ingest_ips 0 '"value"' 0
must_get_value outputs.json TMP_D .data_node_ips 0 '"value"' 0
echo "TF status: $tf, ingest_ips $TMP_I, data_node_ips $TMP_D"
case $TMP_I.$TMP_D in
*null*) echo >&2 "looks like we failed, null in IPs."
tries=$(expr $tries + 1)
;;
*) okay=true
;;
esac
done
echo "okay $okay, tries $tries"
if ! $okay; then
echo >&2 "didn't start terraform successfully, giving up"
exit 1
fi
terraform output -json > outputs.json
popd
@ -31,28 +59,33 @@ EBS_DEVICE_NAME=/dev/nvme1n1
FB_BINARY=featurebase_linux_arm64
# get the first ingest host
INGESTNODE0=$(cat ./qa/tf/perf/able/outputs.json | jq -r '[.ingest_ips][0]["value"][0]')
must_get_value ./qa/tf/perf/able/outputs.json INGESTNODE0 .ingest_ips 0 '"value"' 0
echo "using INGESTNODE0 ${INGESTNODE0}"
# get the first data host
DATANODE0=$(cat ./qa/tf/perf/able/outputs.json | jq -r '[.data_node_ips][0]["value"][0]')
must_get_value ./qa/tf/perf/able/outputs.json DATANODE0 .data_node_ips 0 '"value"' 0
echo "using DATANODE0 ${DATANODE0}"
case ${INGESTNODE0}${DATANODE0} in
*null*) echo >&2 "didn't get nodes, giving up early"
exit 1
;;
esac
DEPLOYED_CLUSTER_PREFIX=$(cat ./qa/tf/perf/able/outputs.json | jq -r '[.cluster_prefix][0]["value"]')
must_get_value ./qa/tf/perf/able/outputs.json DEPLOYED_CLUSTER_PREFIX .cluster_prefix 0 '"value"'
echo "Using DEPLOYED_CLUSTER_PREFIX: ${DEPLOYED_CLUSTER_PREFIX}"
DEPLOYED_CLUSTER_REPLICA_COUNT=$(cat ./qa/tf/perf/able/outputs.json | jq -r '[.fb_cluster_replica_count][0]["value"]')
must_get_value ./qa/tf/perf/able/outputs.json DEPLOYED_CLUSTER_REPLICA_COUNT .fb_cluster_replica_count 0 '"value"'
echo "Using DEPLOYED_CLUSTER_REPLICA_COUNT: ${DEPLOYED_CLUSTDEPLOYED_CLUSTER_REPLICA_COUNTER_PREFIX}"
DEPLOYED_DATA_IPS=$(cat ./qa/tf/perf/able/outputs.json | jq -r '[.data_node_ips][0]["value"][]')
must_get_value ./qa/tf/perf/able/outputs.json DEPLOYED_DATA_IPS .data_node_ips 0 '"value"' ""
echo "DEPLOYED_DATA_IPS: {"
echo "${DEPLOYED_DATA_IPS}"
echo "}"
DEPLOYED_DATA_IPS_LEN=`echo "$DEPLOYED_DATA_IPS" | wc -l`
DEPLOYED_INGEST_IPS=$(cat ./qa/tf/perf/able/outputs.json | jq -r '[.ingest_ips][0]["value"][]')
must_get_value ./qa/tf/perf/able/outputs.json DEPLOYED_INGEST_IPS .ingest_ips 0 '"value"' ""
echo "DEPLOYED_INGEST_IPS: {"
echo "${DEPLOYED_INGEST_IPS}"
echo "}"

13
qa/scripts/perf/able/ableTest.sh
View file

@ -1,12 +1,13 @@
#!/bin/bash
# get the first ingest host
INGESTNODE0=$(cat ./qa/tf/perf/able/outputs.json | jq -r '[.ingest_ips][0]["value"][0]')
must_get_value ./qa/tf/perf/able/outputs.json INGESTNODE0 .ingest_ips 0 '"value"' 0
echo "using INGESTNODE0 ${INGESTNODE0}"
# get the first data host
DATANODE0=$(cat ./qa/tf/perf/able/outputs.json | jq -r '[.data_node_ips][0]["value"][0]')
must_get_value ./qa/tf/perf/able/outputs.json DATANODE0 .data_node_ips 0 '"value"' 0
echo "using DATANODE0 ${DATANODE0}"
must_get_value ./qa/tf/perf/able/outputs.json DEPLOYED_DATA_IPS .data_node_ips 0 '"value"' ""
# leaving this here because K6 is timing out and need to work out why
# ssh -A -i ~/.ssh/gitlab-featurebase-ci.pem -o "StrictHostKeyChecking no" ec2-user@${INGESTNODE0} "wget https://github.com/grafana/k6/releases/download/v0.36.0/k6-v0.36.0-linux-arm64.tar.gz"
@ -45,7 +46,13 @@ echo "Restoring data"
ssh -A -i ~/.ssh/gitlab-featurebase-ci.pem -o "StrictHostKeyChecking no" ec2-user@${INGESTNODE0} "cd /data; featurebase restore --host http://${DATANODE0}:10101 -s /data/data/backup > restore.out"
if (( $? != 0 ))
then
echo "Restoring failed"
echo "restoring failed. trying to dump server logs in case server crashed:"
for i in $DEPLOYED_DATA_IPS; do
echo "BEGIN LOGS $i:"
ssh -A -i ~/.ssh/gitlab-featurebase-ci.pem -o "StrictHostKeyChecking no" ec2-user@${i} "cat /var/log/molecula/featurebase.log"
echo "END LOGS $i"
done
exit 1
fi

4
qa/tf/perf/able/main.tf
View file

@ -3,10 +3,10 @@ module "able-cluster" {
cluster_prefix = var.cluster_prefix
region = var.region
profile = var.profile
fb_data_node_type = "m6g.12xlarge"
fb_data_node_type = "c6g.16xlarge"
fb_data_disk_iops = 10000
fb_data_node_count = 3
fb_ingest_type = "m6g.2xlarge"
fb_ingest_type = "c6g.2xlarge"
fb_ingest_disk_iops = 10000
fb_ingest_disk_size_gb = 500
fb_ingest_node_count = 1

2
rbf/db.go
View file

@ -7,7 +7,6 @@ import (
"io"
"os"
"path/filepath"
"runtime/debug"
"sort"
"sync"
"syscall"
@ -669,7 +668,6 @@ func (db *DB) Begin(writable bool) (_ *Tx, err error) {
pageMap: db.pageMap,
walPageN: db.walPageN,
writable: writable,
stack: debug.Stack(), // DEBUG
DeleteEmptyContainer: true,
}

1
server.go
View file

@ -1016,6 +1016,7 @@ func (s *Server) receiveMessage(m Message) error {
if err := idx.UpdateFieldLocal(&obj.CreateFieldMessage, obj.Update); err != nil {
return err
}
case *DeleteFieldMessage:
idx := s.holder.Index(obj.Index)
if err := idx.DeleteField(obj.Field); err != nil {

90
server/handler_test.go
View file

@ -180,12 +180,12 @@ func TestHandler_Endpoints(t *testing.T) {
const shard = 0
tx0 := holder.Txf().NewWritableQcx()
defer tx0.Abort()
if f, err := i0.CreateFieldIfNotExists("f1", pilosa.OptFieldTypeDefault()); err != nil {
if f, err := i0.CreateFieldIfNotExists("f1", "", pilosa.OptFieldTypeDefault()); err != nil {
t.Fatal(err)
} else if _, err := f.SetBit(tx0, 0, 0, nil); err != nil {
t.Fatal(err)
}
if _, err := i0.CreateFieldIfNotExists("f0", pilosa.OptFieldTypeDefault()); err != nil {
if _, err := i0.CreateFieldIfNotExists("f0", "", pilosa.OptFieldTypeDefault()); err != nil {
t.Fatal(err)
}
if err := tx0.Finish(); err != nil {
@ -195,7 +195,7 @@ func TestHandler_Endpoints(t *testing.T) {
i1 := hldr.MustCreateIndexIfNotExists("i1", pilosa.IndexOptions{})
tx1 := holder.Txf().NewWritableQcx()
defer tx1.Abort()
if f, err := i1.CreateFieldIfNotExists("f0", pilosa.OptFieldTypeDefault()); err != nil {
if f, err := i1.CreateFieldIfNotExists("f0", "", pilosa.OptFieldTypeDefault()); err != nil {
t.Fatal(err)
} else if _, err := f.SetBit(tx1, 0, 0, nil); err != nil {
t.Fatal(err)
@ -216,9 +216,10 @@ func TestHandler_Endpoints(t *testing.T) {
&bodySchema); err != nil {
t.Fatalf("unexpected unmarshalling error: %v", err)
}
// DO NOT COMPARE `CreatedAt` - reset to 0
// DO NOT COMPARE `CreatedAt` & 'UpdatedAt' - reset to 0
for _, i := range bodySchema.Indexes {
i.CreatedAt = 0
i.UpdatedAt = 0
for _, f := range i.Fields {
f.CreatedAt = 0
}
@ -226,8 +227,27 @@ func TestHandler_Endpoints(t *testing.T) {
//
var targetSchema pilosa.Schema
if err := json.Unmarshal([]byte(fmt.Sprintf(`{"indexes":[{"name":"i0","options":{"keys":false,"trackExistence":false},"fields":[{"name":"f0","options":{"type":"set","cacheType":"ranked","cacheSize":50000,"keys":false}},{"name":"f1","options":{"type":"set","cacheType":"ranked","cacheSize":50000,"keys":false}}],"shardWidth":%d},{"name":"i1","options":{"keys":false,"trackExistence":false},"fields":[{"name":"f0","options":{"type":"set","cacheType":"ranked","cacheSize":50000,"keys":false}}],"shardWidth":%[1]d}]}`, pilosa.ShardWidth)),
&targetSchema); err != nil {
if err := json.Unmarshal([]byte(fmt.Sprintf(`{"indexes":[
{
"name":"i0",
"options":{"keys":false,"trackExistence":false},
"updatedAt": 0,
"description": "this is a description",
"fields":[
{"name":"f0","options":{"type":"set","cacheType":"ranked","cacheSize":50000,"keys":false}},
{"name":"f1","options":{"type":"set","cacheType":"ranked","cacheSize":50000,"keys":false}}
],
"shardWidth":%d
},
{
"name":"i1",
"options":{"keys":false,"trackExistence":false},
"updatedAt": 0,
"description": "this is a description",
"fields":[
{"name":"f0","options":{"type":"set","cacheType":"ranked","cacheSize":50000,"keys":false}}],"shardWidth":%[1]d}
]
}`, pilosa.ShardWidth)), &targetSchema); err != nil {
t.Fatalf("unexpected unmarshalling error: %v", err)
}
@ -240,13 +260,13 @@ func TestHandler_Endpoints(t *testing.T) {
i2 := hldr.MustCreateIndexIfNotExists("i2", pilosa.IndexOptions{})
tx2 := holder.Txf().NewWritableQcx()
defer tx2.Abort()
if f, err := i2.CreateFieldIfNotExists("f0", pilosa.OptFieldTypeSet(pilosa.CacheTypeRanked, 1000)); err != nil {
if f, err := i2.CreateFieldIfNotExists("f0", "", pilosa.OptFieldTypeSet(pilosa.CacheTypeRanked, 1000)); err != nil {
t.Fatal(err)
} else if _, err := f.SetBit(tx2, 0, 0, nil); err != nil {
t.Fatal(err)
}
f, err := i2.CreateFieldIfNotExists("f1", pilosa.OptFieldTypeInt(-100, 100))
f, err := i2.CreateFieldIfNotExists("f1", "", pilosa.OptFieldTypeInt(-100, 100))
if err != nil {
t.Fatal(err)
}
@ -257,7 +277,7 @@ func TestHandler_Endpoints(t *testing.T) {
}
}
f, err = i2.CreateFieldIfNotExists("f2", pilosa.OptFieldTypeDecimal(1, pql.NewDecimal(-10, 0), pql.NewDecimal(10, 0)))
f, err = i2.CreateFieldIfNotExists("f2", "", pilosa.OptFieldTypeDecimal(1, pql.NewDecimal(-10, 0), pql.NewDecimal(10, 0)))
if err != nil {
t.Fatal(err)
}
@ -268,17 +288,17 @@ func TestHandler_Endpoints(t *testing.T) {
}
}
if f, err := i2.CreateFieldIfNotExists("f3", pilosa.OptFieldTypeTime(pilosa.TimeQuantum("YMDH"), "0")); err != nil {
if f, err := i2.CreateFieldIfNotExists("f3", "", pilosa.OptFieldTypeTime(pilosa.TimeQuantum("YMDH"), "0")); err != nil {
t.Fatal(err)
} else if _, err := f.SetBit(tx2, 0, 0, nil); err != nil {
t.Fatal(err)
}
if f, err := i2.CreateFieldIfNotExists("f4", pilosa.OptFieldTypeMutex(pilosa.CacheTypeRanked, 5000)); err != nil {
if f, err := i2.CreateFieldIfNotExists("f4", "", pilosa.OptFieldTypeMutex(pilosa.CacheTypeRanked, 5000)); err != nil {
t.Fatal(err)
} else if _, err := f.SetBit(tx2, 0, 0, nil); err != nil {
t.Fatal(err)
}
if f, err := i2.CreateFieldIfNotExists("f5", pilosa.OptFieldTypeBool()); err != nil {
if f, err := i2.CreateFieldIfNotExists("f5", "", pilosa.OptFieldTypeBool()); err != nil {
t.Fatal(err)
} else if _, err := f.SetBit(tx2, 0, 0, nil); err != nil {
t.Fatal(err)
@ -299,9 +319,10 @@ func TestHandler_Endpoints(t *testing.T) {
if err := json.Unmarshal(w.Body.Bytes(), &bodySchema); err != nil {
t.Fatalf("unexpected unmarshalling error: %v", err)
}
// DO NOT COMPARE `CreatedAt` - reset to 0
// DO NOT COMPARE `CreatedAt` & `UpdatedAt`` - reset to 0
for _, i := range bodySchema.Indexes {
i.CreatedAt = 0
i.UpdatedAt = 0
for _, f := range i.Fields {
f.CreatedAt = 0
}
@ -309,7 +330,44 @@ func TestHandler_Endpoints(t *testing.T) {
//
var targetSchema pilosa.Schema
target := fmt.Sprintf(`{"indexes":[{"name":"i0","options":{"keys":false,"trackExistence":false},"fields":[{"name":"f0","options":{"type":"set","cacheType":"ranked","cacheSize":50000,"keys":false}},{"name":"f1","options":{"type":"set","cacheType":"ranked","cacheSize":50000,"keys":false},"views":[{"name":"standard"}]}],"shardWidth":%[1]d},{"name":"i1","options":{"keys":false,"trackExistence":false},"fields":[{"name":"f0","options":{"type":"set","cacheType":"ranked","cacheSize":50000,"keys":false},"views":[{"name":"standard"}]}],"shardWidth":%[1]d},{"name":"i2","options":{"keys":false,"trackExistence":false},"fields":[{"name":"f0","options":{"type":"set","cacheType":"ranked","cacheSize":1000,"keys":false},"views":[{"name":"standard"}]},{"name":"f1","options":{"type":"int","base":0,"bitDepth":0,"min":-100,"max":100,"keys":false,"foreignIndex":""},"views":[{"name":"bsig_f1"}]},{"name":"f2","options":{"type":"decimal","base":0,"scale":1,"bitDepth":0,"min":-10,"max":10,"keys":false},"views":[{"name":"bsig_f2"}]},{"name":"f3","options":{"type":"time","timeQuantum":"YMDH","keys":false,"noStandardView":false},"views":[{"name":"standard"}]},{"name":"f4","options":{"type":"mutex","cacheType":"ranked","cacheSize":5000,"keys":false},"views":[{"name":"standard"}]},{"name":"f5","options":{"type":"bool"},"views":[{"name":"standard"}]}],"shardWidth":%[1]d}]}`, pilosa.ShardWidth)
target := fmt.Sprintf(`{"indexes":[
{
"name":"i0",
"options":{"keys":false,"trackExistence":false},
"updatedAt": 0,
"description": "this is a description",
"fields":[
{"name":"f0","options":{"type":"set","cacheType":"ranked","cacheSize":50000,"keys":false}},
{"name":"f1","options":{"type":"set","cacheType":"ranked","cacheSize":50000,"keys":false},"views":[{"name":"standard"}]}
],
"shardWidth":%[1]d
},
{
"name":"i1",
"options":{"keys":false,"trackExistence":false},
"updatedAt": 0,
"description": "this is a description",
"fields":[
{"name":"f0","options":{"type":"set","cacheType":"ranked","cacheSize":50000,"keys":false},"views":[{"name":"standard"}]}
],
"shardWidth":%[1]d
},
{
"name":"i2",
"options":{"keys":false,"trackExistence":false},
"updatedAt": 0,
"description": "this is a description",
"fields":[
{"name":"f0","options":{"type":"set","cacheType":"ranked","cacheSize":1000,"keys":false},"views":[{"name":"standard"}]},
{"name":"f1","options":{"type":"int","base":0,"bitDepth":0,"min":-100,"max":100,"keys":false,"foreignIndex":""},"views":[{"name":"bsig_f1"}]},
{"name":"f2","options":{"type":"decimal","base":0,"scale":1,"bitDepth":0,"min":-10,"max":10,"keys":false},"views":[{"name":"bsig_f2"}]},
{"name":"f3","options":{"type":"time","timeQuantum":"YMDH","keys":false,"noStandardView":false},"views":[{"name":"standard"}]},
{"name":"f4","options":{"type":"mutex","cacheType":"ranked","cacheSize":5000,"keys":false},"views":[{"name":"standard"}]},
{"name":"f5","options":{"type":"bool"},"views":[{"name":"standard"}]}
],
"shardWidth":%[1]d}
]
}`, pilosa.ShardWidth)
if err := json.Unmarshal([]byte(target), &targetSchema); err != nil {
t.Fatalf("unexpected unmarshalling error: %v", err)
}
@ -417,7 +475,7 @@ func TestHandler_Endpoints(t *testing.T) {
})
t.Run("ImportRoaringOverwrite", func(t *testing.T) {
if _, err := i0.CreateFieldIfNotExists("int-field", pilosa.OptFieldTypeInt(0, 10)); err != nil {
if _, err := i0.CreateFieldIfNotExists("int-field", "", pilosa.OptFieldTypeInt(0, 10)); err != nil {
t.Fatal(err)
}
w := httptest.NewRecorder()
@ -958,7 +1016,7 @@ func TestHandler_Endpoints(t *testing.T) {
t.Run("Field delete", func(t *testing.T) {
i := hldr.MustCreateIndexIfNotExists("i", pilosa.IndexOptions{})
if _, err := i.CreateFieldIfNotExists("f1", pilosa.OptFieldTypeDefault()); err != nil {
if _, err := i.CreateFieldIfNotExists("f1", "", pilosa.OptFieldTypeDefault()); err != nil {
t.Fatal(err)
}
w := httptest.NewRecorder()

16
sql3/errors.go
View file

@ -46,6 +46,8 @@ const (
ErrLiteralEmptySetNotAllowed errors.Code = "ErrLiteralEmptySetNotAllowed"
ErrLiteralEmptyTupleNotAllowed errors.Code = "ErrLiteralEmptyTupleNotAllowed"
ErrSetLiteralMustContainIntOrString errors.Code = "ErrSetLiteralMustContainIntOrString"
ErrInvalidColumnInFilterExpression errors.Code = "ErrInvalidColumnInFilterExpression"
ErrInvalidTypeInFilterExpression errors.Code = "ErrInvalidTypeInFilterExpression"
ErrTypeAssignmentIncompatible errors.Code = "ErrTypeAssignmentIncompatible"
@ -198,6 +200,20 @@ func NewErrSetLiteralMustContainIntOrString(line, col int) error {
)
}
func NewErrInvalidColumnInFilterExpression(line, col int, column string, op string) error {
return errors.New(
ErrInvalidColumnInFilterExpression,
fmt.Sprintf("[%d:%d] '%s' column cannot be used in a %s filter expression", line, col, column, op),
)
}
func NewErrInvalidTypeInFilterExpression(line, col int, typeName string, op string) error {
return errors.New(
ErrInvalidTypeInFilterExpression,
fmt.Sprintf("[%d:%d] unsupported type '%s' for %s filter expression", line, col, typeName, op),
)
}
func NewErrLiteralEmptyTupleNotAllowed(line, col int) error {
return errors.New(
ErrLiteralEmptyTupleNotAllowed,

110
sql3/planner/executionplanner_test.go
View file

@ -22,25 +22,25 @@ func TestPlanner_Show(t *testing.T) {
c := test.MustRunCluster(t, 1)
defer c.Close()
index, err := c.GetHolder(0).CreateIndex(c.Idx("i"), pilosa.IndexOptions{TrackExistence: true})
index, err := c.GetHolder(0).CreateIndex(c.Idx("i"), "", pilosa.IndexOptions{TrackExistence: true})
if err != nil {
t.Fatal(err)
}
if _, err := index.CreateField("f", pilosa.OptFieldTypeInt(0, 1000)); err != nil {
if _, err := index.CreateField("f", "", pilosa.OptFieldTypeInt(0, 1000)); err != nil {
t.Fatal(err)
} else if _, err := index.CreateField("x", pilosa.OptFieldTypeInt(0, 1000)); err != nil {
} else if _, err := index.CreateField("x", "", pilosa.OptFieldTypeInt(0, 1000)); err != nil {
t.Fatal(err)
}
index2, err := c.GetHolder(0).CreateIndex(c.Idx("l"), pilosa.IndexOptions{TrackExistence: false})
index2, err := c.GetHolder(0).CreateIndex(c.Idx("l"), "", pilosa.IndexOptions{TrackExistence: false})
if err != nil {
t.Fatal(err)
}
if _, err := index2.CreateField("f", pilosa.OptFieldTypeInt(0, 1000)); err != nil {
if _, err := index2.CreateField("f", "", pilosa.OptFieldTypeInt(0, 1000)); err != nil {
t.Fatal(err)
} else if _, err := index2.CreateField("x", pilosa.OptFieldTypeInt(0, 1000)); err != nil {
} else if _, err := index2.CreateField("x", "", pilosa.OptFieldTypeInt(0, 1000)); err != nil {
t.Fatal(err)
}
@ -532,14 +532,14 @@ func TestPlanner_AlterTable(t *testing.T) {
c := test.MustRunCluster(t, 1)
defer c.Close()
index, err := c.GetHolder(0).CreateIndex(c.Idx("i"), pilosa.IndexOptions{TrackExistence: true})
index, err := c.GetHolder(0).CreateIndex(c.Idx("i"), "", pilosa.IndexOptions{TrackExistence: true})
if err != nil {
t.Fatal(err)
}
if _, err := index.CreateField("f", pilosa.OptFieldTypeInt(0, 1000)); err != nil {
if _, err := index.CreateField("f", "", pilosa.OptFieldTypeInt(0, 1000)); err != nil {
t.Fatal(err)
} else if _, err := index.CreateField("x", pilosa.OptFieldTypeInt(0, 1000)); err != nil {
} else if _, err := index.CreateField("x", "", pilosa.OptFieldTypeInt(0, 1000)); err != nil {
t.Fatal(err)
}
@ -594,14 +594,14 @@ func TestPlanner_DropTable(t *testing.T) {
c := test.MustRunCluster(t, 1)
defer c.Close()
index, err := c.GetHolder(0).CreateIndex(c.Idx("i"), pilosa.IndexOptions{TrackExistence: true})
index, err := c.GetHolder(0).CreateIndex(c.Idx("i"), "", pilosa.IndexOptions{TrackExistence: true})
if err != nil {
t.Fatal(err)
}
if _, err := index.CreateField("f", pilosa.OptFieldTypeInt(0, 1000)); err != nil {
if _, err := index.CreateField("f", "", pilosa.OptFieldTypeInt(0, 1000)); err != nil {
t.Fatal(err)
} else if _, err := index.CreateField("x", pilosa.OptFieldTypeInt(0, 1000)); err != nil {
} else if _, err := index.CreateField("x", "", pilosa.OptFieldTypeInt(0, 1000)); err != nil {
t.Fatal(err)
}
@ -617,25 +617,25 @@ func TestPlanner_ExpressionsInSelectListParen(t *testing.T) {
c := test.MustRunCluster(t, 1)
defer c.Close()
i0, err := c.GetHolder(0).CreateIndex(c.Idx("j"), pilosa.IndexOptions{TrackExistence: true})
i0, err := c.GetHolder(0).CreateIndex(c.Idx("j"), "", pilosa.IndexOptions{TrackExistence: true})
if err != nil {
t.Fatal(err)
}
if _, err := i0.CreateField("a", pilosa.OptFieldTypeInt(0, 1000)); err != nil {
if _, err := i0.CreateField("a", "", pilosa.OptFieldTypeInt(0, 1000)); err != nil {
t.Fatal(err)
} else if _, err := i0.CreateField("b", pilosa.OptFieldTypeInt(0, 1000)); err != nil {
} else if _, err := i0.CreateField("b", "", pilosa.OptFieldTypeInt(0, 1000)); err != nil {
t.Fatal(err)
}
i1, err := c.GetHolder(0).CreateIndex(c.Idx("k"), pilosa.IndexOptions{TrackExistence: true})
i1, err := c.GetHolder(0).CreateIndex(c.Idx("k"), "", pilosa.IndexOptions{TrackExistence: true})
if err != nil {
t.Fatal(err)
}
if _, err := i1.CreateField("x", pilosa.OptFieldTypeInt(0, 1000)); err != nil {
if _, err := i1.CreateField("x", "", pilosa.OptFieldTypeInt(0, 1000)); err != nil {
t.Fatal(err)
} else if _, err := i1.CreateField("y", pilosa.OptFieldTypeInt(0, 1000)); err != nil {
} else if _, err := i1.CreateField("y", "", pilosa.OptFieldTypeInt(0, 1000)); err != nil {
t.Fatal(err)
}
@ -698,20 +698,20 @@ func TestPlanner_ExpressionsInSelectListLiterals(t *testing.T) {
c := test.MustRunCluster(t, 1)
defer c.Close()
i0, err := c.GetHolder(0).CreateIndex(c.Idx("j"), pilosa.IndexOptions{TrackExistence: true})
i0, err := c.GetHolder(0).CreateIndex(c.Idx("j"), "", pilosa.IndexOptions{TrackExistence: true})
if err != nil {
t.Fatal(err)
}
if _, err := i0.CreateField("a", pilosa.OptFieldTypeInt(0, 1000)); err != nil {
if _, err := i0.CreateField("a", "", pilosa.OptFieldTypeInt(0, 1000)); err != nil {
t.Fatal(err)
} else if _, err := i0.CreateField("b", pilosa.OptFieldTypeInt(0, 1000)); err != nil {
} else if _, err := i0.CreateField("b", "", pilosa.OptFieldTypeInt(0, 1000)); err != nil {
t.Fatal(err)
} else if _, err := i0.CreateField("d", pilosa.OptFieldTypeDecimal(2)); err != nil {
} else if _, err := i0.CreateField("d", "", pilosa.OptFieldTypeDecimal(2)); err != nil {
t.Fatal(err)
} else if _, err := i0.CreateField("ts", pilosa.OptFieldTypeTimestamp(pilosa.DefaultEpoch, "s")); err != nil {
} else if _, err := i0.CreateField("ts", "", pilosa.OptFieldTypeTimestamp(pilosa.DefaultEpoch, "s")); err != nil {
t.Fatal(err)
} else if _, err := i0.CreateField("str", pilosa.OptFieldTypeMutex(pilosa.CacheTypeLRU, pilosa.DefaultCacheSize), pilosa.OptFieldKeys()); err != nil {
} else if _, err := i0.CreateField("str", "", pilosa.OptFieldTypeMutex(pilosa.CacheTypeLRU, pilosa.DefaultCacheSize), pilosa.OptFieldKeys()); err != nil {
t.Fatal(err)
}
@ -840,20 +840,20 @@ func TestPlanner_ExpressionsInSelectListCase(t *testing.T) {
c := test.MustRunCluster(t, 1)
defer c.Close()
i0, err := c.GetHolder(0).CreateIndex(c.Idx("j"), pilosa.IndexOptions{TrackExistence: true})
i0, err := c.GetHolder(0).CreateIndex(c.Idx("j"), "", pilosa.IndexOptions{TrackExistence: true})
if err != nil {
t.Fatal(err)
}
if _, err := i0.CreateField("a", pilosa.OptFieldTypeInt(0, 1000)); err != nil {
if _, err := i0.CreateField("a", "", pilosa.OptFieldTypeInt(0, 1000)); err != nil {
t.Fatal(err)
} else if _, err := i0.CreateField("b", pilosa.OptFieldTypeInt(0, 1000)); err != nil {
} else if _, err := i0.CreateField("b", "", pilosa.OptFieldTypeInt(0, 1000)); err != nil {
t.Fatal(err)
} else if _, err := i0.CreateField("d", pilosa.OptFieldTypeDecimal(2)); err != nil {
} else if _, err := i0.CreateField("d", "", pilosa.OptFieldTypeDecimal(2)); err != nil {
t.Fatal(err)
} else if _, err := i0.CreateField("ts", pilosa.OptFieldTypeTimestamp(pilosa.DefaultEpoch, "s")); err != nil {
} else if _, err := i0.CreateField("ts", "", pilosa.OptFieldTypeTimestamp(pilosa.DefaultEpoch, "s")); err != nil {
t.Fatal(err)
} else if _, err := i0.CreateField("str", pilosa.OptFieldTypeMutex(pilosa.CacheTypeLRU, pilosa.DefaultCacheSize), pilosa.OptFieldKeys()); err != nil {
} else if _, err := i0.CreateField("str", "", pilosa.OptFieldTypeMutex(pilosa.CacheTypeLRU, pilosa.DefaultCacheSize), pilosa.OptFieldKeys()); err != nil {
t.Fatal(err)
}
@ -921,25 +921,25 @@ func TestPlanner_Select(t *testing.T) {
c := test.MustRunCluster(t, 1)
defer c.Close()
i0, err := c.GetHolder(0).CreateIndex(c.Idx("j"), pilosa.IndexOptions{TrackExistence: true})
i0, err := c.GetHolder(0).CreateIndex(c.Idx("j"), "", pilosa.IndexOptions{TrackExistence: true})
if err != nil {
t.Fatal(err)
}
if _, err := i0.CreateField("a", pilosa.OptFieldTypeInt(0, 1000)); err != nil {
if _, err := i0.CreateField("a", "", pilosa.OptFieldTypeInt(0, 1000)); err != nil {
t.Fatal(err)
} else if _, err := i0.CreateField("b", pilosa.OptFieldTypeInt(0, 1000)); err != nil {
} else if _, err := i0.CreateField("b", "", pilosa.OptFieldTypeInt(0, 1000)); err != nil {
t.Fatal(err)
}
i1, err := c.GetHolder(0).CreateIndex(c.Idx("k"), pilosa.IndexOptions{TrackExistence: true})
i1, err := c.GetHolder(0).CreateIndex(c.Idx("k"), "", pilosa.IndexOptions{TrackExistence: true})
if err != nil {
t.Fatal(err)
}
if _, err := i1.CreateField("x", pilosa.OptFieldTypeInt(0, 1000)); err != nil {
if _, err := i1.CreateField("x", "", pilosa.OptFieldTypeInt(0, 1000)); err != nil {
t.Fatal(err)
} else if _, err := i1.CreateField("y", pilosa.OptFieldTypeInt(0, 1000)); err != nil {
} else if _, err := i1.CreateField("y", "", pilosa.OptFieldTypeInt(0, 1000)); err != nil {
t.Fatal(err)
}
@ -1120,14 +1120,14 @@ func TestPlanner_SelectOrderBy(t *testing.T) {
c := test.MustRunCluster(t, 1)
defer c.Close()
i0, err := c.GetHolder(0).CreateIndex(c.Idx("j"), pilosa.IndexOptions{TrackExistence: true})
i0, err := c.GetHolder(0).CreateIndex(c.Idx("j"), "", pilosa.IndexOptions{TrackExistence: true})
if err != nil {
t.Fatal(err)
}
if _, err := i0.CreateField("a", pilosa.OptFieldTypeInt(0, 1000)); err != nil {
if _, err := i0.CreateField("a", "", pilosa.OptFieldTypeInt(0, 1000)); err != nil {
t.Fatal(err)
} else if _, err := i0.CreateField("b", pilosa.OptFieldTypeInt(0, 1000)); err != nil {
} else if _, err := i0.CreateField("b", "", pilosa.OptFieldTypeInt(0, 1000)); err != nil {
t.Fatal(err)
}
@ -1170,14 +1170,14 @@ func TestPlanner_SelectSelectSource(t *testing.T) {
c := test.MustRunCluster(t, 1)
defer c.Close()
i0, err := c.GetHolder(0).CreateIndex(c.Idx("j"), pilosa.IndexOptions{TrackExistence: true})
i0, err := c.GetHolder(0).CreateIndex(c.Idx("j"), "", pilosa.IndexOptions{TrackExistence: true})
if err != nil {
t.Fatal(err)
}
if _, err := i0.CreateField("a", pilosa.OptFieldTypeInt(0, 1000)); err != nil {
if _, err := i0.CreateField("a", "", pilosa.OptFieldTypeInt(0, 1000)); err != nil {
t.Fatal(err)
} else if _, err := i0.CreateField("b", pilosa.OptFieldTypeInt(0, 1000)); err != nil {
} else if _, err := i0.CreateField("b", "", pilosa.OptFieldTypeInt(0, 1000)); err != nil {
t.Fatal(err)
}
@ -1242,23 +1242,23 @@ func TestPlanner_In(t *testing.T) {
c := test.MustRunCluster(t, 1)
defer c.Close()
i0, err := c.GetHolder(0).CreateIndex(c.Idx("j"), pilosa.IndexOptions{TrackExistence: true})
i0, err := c.GetHolder(0).CreateIndex(c.Idx("j"), "", pilosa.IndexOptions{TrackExistence: true})
if err != nil {
t.Fatal(err)
}
if _, err := i0.CreateField("a", pilosa.OptFieldTypeInt(0, 1000)); err != nil {
if _, err := i0.CreateField("a", "", pilosa.OptFieldTypeInt(0, 1000)); err != nil {
t.Fatal(err)
}
i1, err := c.GetHolder(0).CreateIndex(c.Idx("k"), pilosa.IndexOptions{TrackExistence: true})
i1, err := c.GetHolder(0).CreateIndex(c.Idx("k"), "", pilosa.IndexOptions{TrackExistence: true})
if err != nil {
t.Fatal(err)
}
if _, err := i1.CreateField("parentid", pilosa.OptFieldTypeInt(0, 1000)); err != nil {
if _, err := i1.CreateField("parentid", "", pilosa.OptFieldTypeInt(0, 1000)); err != nil {
t.Fatal(err)
} else if _, err := i1.CreateField("x", pilosa.OptFieldTypeInt(0, 1000)); err != nil {
} else if _, err := i1.CreateField("x", "", pilosa.OptFieldTypeInt(0, 1000)); err != nil {
t.Fatal(err)
}
@ -1373,23 +1373,23 @@ func TestPlanner_Distinct(t *testing.T) {
c := test.MustRunCluster(t, 1)
defer c.Close()
i0, err := c.GetHolder(0).CreateIndex(c.Idx("j"), pilosa.IndexOptions{TrackExistence: true})
i0, err := c.GetHolder(0).CreateIndex(c.Idx("j"), "", pilosa.IndexOptions{TrackExistence: true})
if err != nil {
t.Fatal(err)
}
if _, err := i0.CreateField("a", pilosa.OptFieldTypeInt(0, 1000)); err != nil {
if _, err := i0.CreateField("a", "", pilosa.OptFieldTypeInt(0, 1000)); err != nil {
t.Fatal(err)
}
i1, err := c.GetHolder(0).CreateIndex(c.Idx("k"), pilosa.IndexOptions{TrackExistence: true})
i1, err := c.GetHolder(0).CreateIndex(c.Idx("k"), "", pilosa.IndexOptions{TrackExistence: true})
if err != nil {
t.Fatal(err)
}
if _, err := i1.CreateField("parentid", pilosa.OptFieldTypeInt(0, 1000)); err != nil {
if _, err := i1.CreateField("parentid", "", pilosa.OptFieldTypeInt(0, 1000)); err != nil {
t.Fatal(err)
} else if _, err := i1.CreateField("x", pilosa.OptFieldTypeInt(0, 1000)); err != nil {
} else if _, err := i1.CreateField("x", "", pilosa.OptFieldTypeInt(0, 1000)); err != nil {
t.Fatal(err)
}
@ -1489,16 +1489,16 @@ func TestPlanner_SelectTop(t *testing.T) {
c := test.MustRunCluster(t, 1)
defer c.Close()
i0, err := c.GetHolder(0).CreateIndex(c.Idx("j"), pilosa.IndexOptions{TrackExistence: true})
i0, err := c.GetHolder(0).CreateIndex(c.Idx("j"), "", pilosa.IndexOptions{TrackExistence: true})
if err != nil {
t.Fatal(err)
}
if _, err := i0.CreateField("a", pilosa.OptFieldTypeInt(0, 1000)); err != nil {
if _, err := i0.CreateField("a", "", pilosa.OptFieldTypeInt(0, 1000)); err != nil {
t.Fatal(err)
}
if _, err := i0.CreateField("b", pilosa.OptFieldTypeInt(0, 1000)); err != nil {
if _, err := i0.CreateField("b", "", pilosa.OptFieldTypeInt(0, 1000)); err != nil {
t.Fatal(err)
}

32
sql3/planner/expressionpql.go
View file

@ -193,6 +193,38 @@ func (p *ExecutionPlanner) generatePQLCallFromBinaryExpr(ctx context.Context, ex
case parser.IN, parser.NOTIN:
return nil, sql3.NewErrInternal("IN operator is not supported")
case parser.IS, parser.ISNOT:
lhs, ok := expr.lhs.(*qualifiedRefPlanExpression)
if !ok {
return nil, sql3.NewErrInternalf("unexpected lhs %T", expr.lhs)
}
pqlOp := pql.EQ
if op == parser.ISNOT {
pqlOp = pql.NEQ
}
switch typ := expr.lhs.Type().(type) {
case *parser.DataTypeID:
if strings.EqualFold(lhs.columnName, "_id") {
return nil, sql3.NewErrInvalidColumnInFilterExpression(0, 0, "_id", "is/is not null")
}
return nil, sql3.NewErrInvalidTypeInFilterExpression(0, 0, typ.TypeName(), "is/is not null")
case *parser.DataTypeInt, *parser.DataTypeDecimal, *parser.DataTypeTimestamp:
return &pql.Call{
Name: "Row",
Args: map[string]interface{}{
lhs.columnName: &pql.Condition{
Op: pqlOp,
Value: nil,
},
},
}, nil
default:
return nil, sql3.NewErrInvalidTypeInFilterExpression(0, 0, typ.TypeName(), "is/is not null")
}
case parser.BETWEEN, parser.NOTBETWEEN:
return nil, sql3.NewErrInternal("BETWEEN operator is not supported")

1
sql3/planner/opcreatetable.go
View file

@ -109,6 +109,7 @@ func (i *createTableRowIter) Next(ctx context.Context) (types.Row, error) {
}
}
// TODO (pok) add ability to add description here
if err := i.planner.schemaAPI.CreateIndexAndFields(ctx, i.tableName, options, fields); err != nil {
if _, ok := errors.Cause(err).(pilosa.ConflictError); ok {
if i.failIfExists {

2
sql3/planner/opgroupby.go
View file

@ -6,7 +6,6 @@ import (
"context"
"fmt"
"hash/maphash"
"log"
"github.com/featurebasedb/featurebase/v3/errors"
"github.com/featurebasedb/featurebase/v3/sql3"
@ -336,6 +335,5 @@ func groupingKeyHash(ctx context.Context, groupByExprs []types.PlanExpression, r
rowKeys[i] = v
}
result := hash.Sum64()
log.Printf("Hash %v, %v", result, rowKeys)
return result, rowKeys, nil
}

12
sql3/sql_definitions_test.go
View file

@ -316,6 +316,9 @@ var tableTests []tableTest = []tableTest{
nullTests,
notNullTests,
//null filter tests
nullFilterTests,
//between tests
betweenTests,
notBetweenTests,
@ -372,6 +375,15 @@ var insertTest = tableTest{
expRows: rows(),
compare: compareExactUnordered,
},
{
// Replace
sqls: sqls(
"replace into testinsert (_id, a, b, s, bl, d, event, ievent) values (4, 40, 400, 'foo', false, 10.12, ['A', 'B', 'C'], [1, 2, 3])",
),
expHdrs: hdrs(),
expRows: rows(),
compare: compareExactUnordered,
},
{
// Insert with nulls
sqls: sqls(

173
sql3/sql_defs_null_test.go
View file

@ -1,6 +1,6 @@
package sql3_test
//NULL tests
// NULL tests
var nullTests = tableTest{
table: tbl(
"null_all_types",
@ -144,7 +144,7 @@ var nullTests = tableTest{
},
}
//NOT NULL tests
// NOT NULL tests
var notNullTests = tableTest{
table: tbl(
"not_null_all_types",
@ -275,3 +275,172 @@ var notNullTests = tableTest{
},
},
}
// NULL filter condition tests
var nullFilterTests = tableTest{
table: tbl(
"null_filter_all_types",
srcHdrs(
srcHdr("_id", fldTypeID),
srcHdr("i", fldTypeInt, "min 0", "max 1000"),
srcHdr("i1", fldTypeInt, "min 0", "max 1000"),
srcHdr("b1", fldTypeBool),
srcHdr("d1", fldTypeDecimal2),
srcHdr("id1", fldTypeID),
srcHdr("ids1", fldTypeIDSet),
srcHdr("s1", fldTypeString),
srcHdr("ss1", fldTypeStringSet),
srcHdr("t1", fldTypeTimestamp),
),
srcRows(
srcRow(int64(1), int64(1), nil, nil, nil, nil, nil, nil, nil, nil),
srcRow(int64(2), int64(1), int64(10), bool(true), float64(10), int64(20), []int64{101}, string("foo"), []string{"GET", "POST"}, knownTimestamp()),
),
),
sqlTests: []sqlTest{
{
sqls: sqls(
"select _id from null_filter_all_types where _id is null",
),
expErr: "'_id' column cannot be used in a is/is not null filter expression",
},
{
sqls: sqls(
"select _id from null_filter_all_types where _id is not null",
),
expErr: "'_id' column cannot be used in a is/is not null filter expression",
},
{
sqls: sqls(
"select _id from null_filter_all_types where i1 is null",
),
expHdrs: hdrs(
hdr("_id", fldTypeID),
),
expRows: rows(
row(int64(1)),
),
compare: compareExactUnordered,
},
{
sqls: sqls(
"select _id from null_filter_all_types where i1 is not null",
),
expHdrs: hdrs(
hdr("_id", fldTypeID),
),
expRows: rows(
row(int64(2)),
),
compare: compareExactUnordered,
},
{
sqls: sqls(
"select _id from null_filter_all_types where b1 is null",
),
expErr: "unsupported type 'BOOL' for is/is not null filter expression",
},
{
sqls: sqls(
"select _id from null_filter_all_types where b1 is not null",
),
expErr: "unsupported type 'BOOL' for is/is not null filter expression",
},
{
sqls: sqls(
"select _id from null_filter_all_types where d1 is null",
),
expHdrs: hdrs(
hdr("_id", fldTypeID),
),
expRows: rows(
row(int64(1)),
),
compare: compareExactUnordered,
},
{
sqls: sqls(
"select _id from null_filter_all_types where d1 is not null",
),
expHdrs: hdrs(
hdr("_id", fldTypeID),
),
expRows: rows(
row(int64(2)),
),
compare: compareExactUnordered,
},
{
sqls: sqls(
"select _id from null_filter_all_types where id1 is null",
),
expErr: "unsupported type 'ID' for is/is not null filter expression",
},
{
sqls: sqls(
"select _id from null_filter_all_types where id1 is not null",
),
expErr: "unsupported type 'ID' for is/is not null filter expression",
},
{
sqls: sqls(
"select _id from null_filter_all_types where ids1 is null",
),
expErr: "unsupported type 'IDSET' for is/is not null filter expression",
},
{
sqls: sqls(
"select _id from null_filter_all_types where ids1 is not null",
),
expErr: "unsupported type 'IDSET' for is/is not null filter expression",
},
{
sqls: sqls(
"select _id from null_filter_all_types where s1 is null",
),
expErr: "unsupported type 'STRING' for is/is not null filter expression",
},
{
sqls: sqls(
"select _id from null_filter_all_types where s1 is not null",
),
expErr: "unsupported type 'STRING' for is/is not null filter expression",
},
{
sqls: sqls(
"select _id from null_filter_all_types where ss1 is null",
),
expErr: "unsupported type 'STRINGSET' for is/is not null filter expression",
},
{
sqls: sqls(
"select _id from null_filter_all_types where ss1 is not null",
),
expErr: "unsupported type 'STRINGSET' for is/is not null filter expression",
},
{
sqls: sqls(
"select _id from null_filter_all_types where t1 is null",
),
expHdrs: hdrs(
hdr("_id", fldTypeID),
),
expRows: rows(
row(int64(1)),
),
compare: compareExactUnordered,
},
{
sqls: sqls(
"select _id from null_filter_all_types where t1 is not null",
),
expHdrs: hdrs(
hdr("_id", fldTypeID),
),
expRows: rows(
row(int64(2)),
),
compare: compareExactUnordered,
},
},
}

16
test/holder.go
View file

@ -65,7 +65,7 @@ func (h *Holder) Reopen() error {
// MustCreateIndexIfNotExists returns a given index. Panic on error.
func (h *Holder) MustCreateIndexIfNotExists(index string, opt pilosa.IndexOptions) *Index {
idx, err := h.Holder.CreateIndexIfNotExists(index, opt)
idx, err := h.Holder.CreateIndexIfNotExists(index, "", opt)
if err != nil {
h.tb.Fatalf("creating index: %v", err)
}
@ -75,7 +75,7 @@ func (h *Holder) MustCreateIndexIfNotExists(index string, opt pilosa.IndexOption
// Row returns a Row for a given field.
func (h *Holder) Row(index, field string, rowID uint64) *pilosa.Row {
idx := h.MustCreateIndexIfNotExists(index, pilosa.IndexOptions{})
f, err := idx.CreateFieldIfNotExists(field, pilosa.OptFieldTypeDefault())
f, err := idx.CreateFieldIfNotExists(field, "", pilosa.OptFieldTypeDefault())
if err != nil {
panic(err)
}
@ -117,7 +117,7 @@ func (h *Holder) ReadRow(index, field string, rowID uint64) *pilosa.Row {
func (h *Holder) RowTime(index, field string, rowID uint64, t time.Time, quantum string) *pilosa.Row {
idx := h.MustCreateIndexIfNotExists(index, pilosa.IndexOptions{})
f, err := idx.CreateFieldIfNotExists(field, pilosa.OptFieldTypeDefault())
f, err := idx.CreateFieldIfNotExists(field, "", pilosa.OptFieldTypeDefault())
if err != nil {
panic(err)
}
@ -142,7 +142,7 @@ func (h *Holder) SetBit(index, field string, rowID, columnID uint64) {
// SetBitTime sets a bit with timestamp on the given field.
func (h *Holder) SetBitTime(index, field string, rowID, columnID uint64, t *time.Time) {
idx := h.MustCreateIndexIfNotExists(index, pilosa.IndexOptions{})
f, err := idx.CreateFieldIfNotExists(field, pilosa.OptFieldTypeDefault())
f, err := idx.CreateFieldIfNotExists(field, "", pilosa.OptFieldTypeDefault())
if err != nil {
panic(err)
}
@ -163,7 +163,7 @@ func (h *Holder) SetBitTime(index, field string, rowID, columnID uint64, t *time
// ClearBit clears a bit on the given field.
func (h *Holder) ClearBit(index, field string, rowID, columnID uint64) {
idx := h.MustCreateIndexIfNotExists(index, pilosa.IndexOptions{})
f, err := idx.CreateFieldIfNotExists(field, pilosa.OptFieldTypeDefault())
f, err := idx.CreateFieldIfNotExists(field, "", pilosa.OptFieldTypeDefault())
if err != nil {
panic(err)
}
@ -192,7 +192,7 @@ func (h *Holder) MustSetBits(index, field string, rowID uint64, columnIDs ...uin
// SetValue sets an integer value on the given field.
func (h *Holder) SetValue(index, field string, columnID uint64, value int64) *Index {
idx := h.MustCreateIndexIfNotExists(index, pilosa.IndexOptions{})
f, err := idx.CreateFieldIfNotExists(field, pilosa.OptFieldTypeInt(math.MinInt64, math.MaxInt64))
f, err := idx.CreateFieldIfNotExists(field, "", pilosa.OptFieldTypeInt(math.MinInt64, math.MaxInt64))
if err != nil {
panic(err)
}
@ -214,7 +214,7 @@ func (h *Holder) SetValue(index, field string, columnID uint64, value int64) *In
// Value returns the integer value for a given column.
func (h *Holder) Value(index, field string, columnID uint64) (int64, bool) {
idx := h.MustCreateIndexIfNotExists(index, pilosa.IndexOptions{})
f, err := idx.CreateFieldIfNotExists(field, pilosa.OptFieldTypeInt(math.MinInt64, math.MaxInt64))
f, err := idx.CreateFieldIfNotExists(field, "", pilosa.OptFieldTypeInt(math.MinInt64, math.MaxInt64))
if err != nil {
panic(err)
}
@ -233,7 +233,7 @@ func (h *Holder) Value(index, field string, columnID uint64) (int64, bool) {
// on the given range.
func (h *Holder) Range(index, field string, op pql.Token, predicate int64) *pilosa.Row {
idx := h.MustCreateIndexIfNotExists(index, pilosa.IndexOptions{})
f, err := idx.CreateFieldIfNotExists(field, pilosa.OptFieldTypeInt(math.MinInt64, math.MaxInt64))
f, err := idx.CreateFieldIfNotExists(field, "", pilosa.OptFieldTypeInt(math.MinInt64, math.MaxInt64))
if err != nil {
panic(err)
}

10
test/index.go
View file

@ -21,7 +21,7 @@ func newIndex(tb testing.TB) (*Holder, *Index) {
testhook.Cleanup(tb, func() {
h.Close()
})
index, err := h.CreateIndex("i", pilosa.IndexOptions{})
index, err := h.CreateIndex("i", "", pilosa.IndexOptions{})
if err != nil {
panic(err)
}
@ -52,8 +52,8 @@ func (i *Index) Reopen() error {
}
// CreateField creates a field with the given options.
func (i *Index) CreateField(name string, opts ...pilosa.FieldOption) (*Field, error) {
f, err := i.Index.CreateField(name, opts...)
func (i *Index) CreateField(name string, requestUserID string, opts ...pilosa.FieldOption) (*Field, error) {
f, err := i.Index.CreateField(name, requestUserID, opts...)
if err != nil {
return nil, err
}
@ -61,8 +61,8 @@ func (i *Index) CreateField(name string, opts ...pilosa.FieldOption) (*Field, er
}
// CreateFieldIfNotExists creates a field with the given options if it doesn't exist.
func (i *Index) CreateFieldIfNotExists(name string, opts ...pilosa.FieldOption) (*Field, error) {
f, err := i.Index.CreateFieldIfNotExists(name, opts...)
func (i *Index) CreateFieldIfNotExists(name string, requestUserID string, opts ...pilosa.FieldOption) (*Field, error) {
f, err := i.Index.CreateFieldIfNotExists(name, requestUserID, opts...)
if err != nil {
return nil, err
}

56
tx_test.go
View file

@ -12,10 +12,12 @@ import (
. "github.com/featurebasedb/featurebase/v3/vprint" // nolint:staticcheck
)
func queryIRABit(m0api *pilosa.API, acctOwnerID uint64, iraField string, iraRowID uint64, index string) (bit bool) {
func queryIRABit(t *testing.T, m0api *pilosa.API, acctOwnerID uint64, iraField string, iraRowID uint64, index string) (bit bool) {
query := fmt.Sprintf("Row(%v=%v)", iraField, iraRowID) // acctOwnerID)
res, err := m0api.Query(context.Background(), &pilosa.QueryRequest{Index: index, Query: query})
PanicOn(err)
if err != nil {
t.Fatalf("querying IRA bit: %v", err)
}
cols := res.Results[0].(*pilosa.Row).Columns()
for i := range cols {
if cols[i] == acctOwnerID {
@ -25,10 +27,12 @@ func queryIRABit(m0api *pilosa.API, acctOwnerID uint64, iraField string, iraRowI
return false
}
func mustQueryAcct(m0api *pilosa.API, acctOwnerID uint64, fieldAcct0, index string) (acctBal int64) {
func mustQueryAcct(t *testing.T, m0api *pilosa.API, acctOwnerID uint64, fieldAcct0, index string) (acctBal int64) {
query := fmt.Sprintf("FieldValue(field=%v, column=%v)", fieldAcct0, acctOwnerID)
res, err := m0api.Query(context.Background(), &pilosa.QueryRequest{Index: index, Query: query})
PanicOn(err)
if err != nil {
t.Fatalf("querying account: %v", err)
}
if len(res.Results) == 0 {
return 0
@ -37,10 +41,10 @@ func mustQueryAcct(m0api *pilosa.API, acctOwnerID uint64, fieldAcct0, index stri
return valCount.Val
}
func queryBalances(m0api *pilosa.API, acctOwnerID uint64, fldAcct0, fldAcct1, index string) (acct0bal, acct1bal int64) {
func queryBalances(t *testing.T, m0api *pilosa.API, acctOwnerID uint64, fldAcct0, fldAcct1, index string) (acct0bal, acct1bal int64) {
acct0bal = mustQueryAcct(m0api, acctOwnerID, fldAcct0, index)
acct1bal = mustQueryAcct(m0api, acctOwnerID, fldAcct1, index)
acct0bal = mustQueryAcct(t, m0api, acctOwnerID, fldAcct0, index)
acct1bal = mustQueryAcct(t, m0api, acctOwnerID, fldAcct1, index)
return
}
@ -144,19 +148,19 @@ func TestAPI_ImportAtomicRecord(t *testing.T) {
//vv("AFTER the first ImportAtomicRecord!")
iraBit := queryIRABit(m0api, acctOwnerID, iraField, iraRowID, index)
iraBit := queryIRABit(t, m0api, acctOwnerID, iraField, iraRowID, index)
if !iraBit {
PanicOn("IRA bit should have been set")
t.Fatal("IRA bit should have been set")
}
startingBalanceAcct0, startingBalanceAcct1 := queryBalances(m0api, acctOwnerID, fieldAcct0, fieldAcct1, index)
startingBalanceAcct0, startingBalanceAcct1 := queryBalances(t, m0api, acctOwnerID, fieldAcct0, fieldAcct1, index)
//vv("starting balance: acct0=%v, acct1=%v", startingBalanceAcct0, startingBalanceAcct1)
if startingBalanceAcct0 != expectedBalStartingAcct0 {
PanicOn(fmt.Sprintf("expected %v, observed %v starting acct0 balance", expectedBalStartingAcct0, startingBalanceAcct0))
t.Fatalf("expected %v, observed %v starting acct0 balance", expectedBalStartingAcct0, startingBalanceAcct0)
}
if startingBalanceAcct1 != expectedBalStartingAcct1 {
PanicOn(fmt.Sprintf("expected %v, observed %v starting acct1 balance", expectedBalStartingAcct1, startingBalanceAcct1))
t.Fatalf("expected %v, observed %v starting acct1 balance", expectedBalStartingAcct1, startingBalanceAcct1)
}
//vv("sad path: transferUSD %v from %v -> %v, with power loss half-way through", transferUSD, fieldAcct0, fieldAcct1)
@ -175,20 +179,20 @@ func TestAPI_ImportAtomicRecord(t *testing.T) {
err = m0api.ImportAtomicRecord(ctx, qcx, air.Clone(), opt)
//err = m0api.ImportAtomicRecord(ctx, nil, air, opt)
if err != pilosa.ErrAborted {
PanicOn(fmt.Sprintf("expected ErrTxnAborted but got err='%#v'", err))
t.Fatalf("expected ErrTxnAborted but got err='%#v'", err)
}
// sad path, cleanup
qcx.Abort()
qcx = nil
b0, b1 := queryBalances(m0api, acctOwnerID, fieldAcct0, fieldAcct1, index)
b0, b1 := queryBalances(t, m0api, acctOwnerID, fieldAcct0, fieldAcct1, index)
//vv("after power failure tx, balance: acct0=%v, acct1=%v", b0, b1)
if b0 != expectedBalStartingAcct0 {
PanicOn(fmt.Sprintf("expected %v, observed %v starting acct0 balance", expectedBalStartingAcct0, b0))
t.Fatalf("expected %v, observed %v starting acct0 balance", expectedBalStartingAcct0, b0)
}
if b1 != expectedBalStartingAcct1 {
PanicOn(fmt.Sprintf("expected %v, observed %v starting acct1 balance", expectedBalStartingAcct1, b1))
t.Fatalf("expected %v, observed %v starting acct1 balance", expectedBalStartingAcct1, b1)
}
//vv("good: with power loss half-way, no change in account balances; acct0=%v; acct1=%v", b0, b1)
@ -199,16 +203,18 @@ func TestAPI_ImportAtomicRecord(t *testing.T) {
qcx = m0api.Txf().NewQcx()
err = m0api.ImportAtomicRecord(ctx, qcx, air.Clone())
PanicOn(err)
if err != nil {
t.Fatalf("importing record: %v", err)
}
if err := qcx.Finish(); err != nil {
t.Fatal(err)
}
eb0, eb1 := queryBalances(m0api, acctOwnerID, fieldAcct0, fieldAcct1, index)
eb0, eb1 := queryBalances(t, m0api, acctOwnerID, fieldAcct0, fieldAcct1, index)
// should have been applied this time.
if eb0 != expectedBalEndingAcct0 ||
eb1 != expectedBalEndingAcct1 {
PanicOn(fmt.Sprintf("problem: transaction did not get committed/applied. transferUSD=%v, but we see: startingBalanceAcct0=%v -> endingBalanceAcct0=%v; startingBalanceAcct1=%v -> endingBalanceAcct1=%v", transferUSD, startingBalanceAcct0, eb0, startingBalanceAcct1, eb1))
t.Fatalf("problem: transaction did not get committed/applied. transferUSD=%v, but we see: startingBalanceAcct0=%v -> endingBalanceAcct0=%v; startingBalanceAcct1=%v -> endingBalanceAcct1=%v", transferUSD, startingBalanceAcct0, eb0, startingBalanceAcct1, eb1)
}
//vv("ending balance: acct0=%v, acct1=%v", eb0, eb1)
@ -219,20 +225,22 @@ func TestAPI_ImportAtomicRecord(t *testing.T) {
qcx = m0api.Txf().NewQcx()
err = m0api.ImportAtomicRecord(ctx, qcx, air)
if err != nil {
t.Fatalf("importing record: %v", err)
}
if err := qcx.Finish(); err != nil {
t.Fatal(err)
}
PanicOn(err)
eb0, eb1 = queryBalances(m0api, acctOwnerID, fieldAcct0, fieldAcct1, index)
eb0, eb1 = queryBalances(t, m0api, acctOwnerID, fieldAcct0, fieldAcct1, index)
if eb0 != 0 ||
eb1 != 0 {
PanicOn("problem: bits did not clear")
t.Fatal("problem: bits did not clear")
}
//vv("cleared balances: acct0=%v, acct1=%v", eb0, eb1)
iraBit = queryIRABit(m0api, acctOwnerID, iraField, iraRowID, index)
iraBit = queryIRABit(t, m0api, acctOwnerID, iraField, iraRowID, index)
if iraBit {
PanicOn("IRA bit should have been cleared")
t.Fatal("IRA bit should have been cleared")
}
}