Commit graph

474 commits

Author SHA1 Message Date
Matt Jaffee
1c1204fc77
modify PQL parser to handle escapes in string values
This modifies the parser to properly "unquote" incoming strings. So if
a string comes in double or single quoted, we approximately follow Go
rules for removing the quotes and processing escape sequences.

The differences from Go are:
1. we only support backslash, quote, tab and newline escape
sequenences.
2. Single quoted strings are supported and work just like double
quoted strings.
3. The peg parser won't actually accept backquoted strings (I don't
think)

Fixes: #411
2020-05-29 07:58:50 -05:00
Kuba Podgórski
3fe85d0f91 Move grpc_internal_test to grpc_test 2020-05-15 19:52:03 +02:00
Kuba Podgórski
f40601f6f5 Wrap grpc resp.Err 2020-05-15 18:16:31 +02:00
Kuba Podgórski
7ee8f80012
Merge branch 'master' into opt-indiagnostics/310 2020-05-15 00:53:33 +02:00
Kuba Podgórski
fd27cbb886
Update server/config.go
Co-authored-by: Matthew Jaffee <matthew.jaffee@gmail.com>
2020-05-12 18:51:53 +02:00
Kuba Podgórski
889f79d11a Turn anti-entropy off by default 2020-05-12 16:49:43 +02:00
Kuba Podgórski
368cb8cb53 opt-in diagnostics 2020-05-12 15:37:47 +02:00
Travis
a91014c7bb
add FieldValue call 2020-05-06 11:52:27 -05:00
Travis
92a94c9ec1
remove noSleep option 2020-05-05 08:31:43 -05:00
Travis
7b36f417d8
avoid deadlock on translationSync.Reset during startup 2020-05-05 08:31:42 -05:00
Matt Jaffee
8aa7a76d31
change default tracing config to 'off'
also fix a typo
2020-04-30 15:02:15 -05:00
Travis
9405463911 suppress test config arguments from pilosa config output 2020-04-24 11:02:42 -05:00
Matt Jaffee
97ae8e0db7
add kuba testcase, fix race
we fix the race by not returning pointers to the things which we're
keeping in the in-memory store
2020-04-22 15:17:25 -05:00
Matt Jaffee
7c7836f16f
convert transactions to be pointers everywhere
I think this will improve the transaction response messages Kuba
mentioned where it was an empty transaction instead of a nil or not
there... if not it should make it easier to do that anyhow.
2020-04-22 14:01:24 -05:00
Matt Jaffee
c36952a0f1
propagate context throughout transaction stuff 2020-04-20 13:30:48 -05:00
Matt Jaffee
9ad1106647
implement transaction API layer and intra-cluster messaging
also adds a "noSleep" option to the server command to avoid the 5
second sleep we introduced on startup for non-coordinator cluster
nodes. The sleep doesn't seem to be needed in the tests and makes them
much slower.
2020-04-20 13:30:48 -05:00
Cody Soyland
5e1c72e6f2 Copy reopen.FileWriter into pilosa 2020-04-20 11:58:09 -05:00
Travis
7de8399b17 move gRPC listener creation outside of grpcServer
For tests, we need to create the grpc listener with port 0 in order to
automatically assign a port. This PR moves the lister creation outside
of the grcpServer itself so that we can access that auto-created port.
2020-04-18 11:38:15 -05:00
Cody Soyland
1058440cfe Do not reuse error object (data race) 2020-04-16 17:08:50 -05:00
Cody Soyland
2155d6cab8 Move some things around, fix linter warnings. 2020-04-16 17:08:50 -05:00
Cody Soyland
a66ee26a6b Reopen log file on SIGHUP 2020-04-16 17:08:50 -05:00
Cody Soyland
08fa90e0f3 Reduce duplication of setupLogger arch-specific code 2020-04-16 17:08:50 -05:00
Travis
c45a4bf3dc ToTable and ToRows interface for gRPC 2020-04-16 14:16:54 -05:00
Alan Bernstein
71b9762501
Address review feedback again 2020-04-10 20:59:20 -05:00
Alan Bernstein
9947c92e8e
Add stats labels and slow-query log in GRPC endpoints 2020-04-10 20:59:20 -05:00
Alan Bernstein
eaf21eb19b
Add metrics for GRPC request timing 2020-04-10 20:59:19 -05:00
Alan Bernstein
8b405c226c
Add 'prometheus' option in other help text/comments 2020-04-10 20:59:18 -05:00
Alan Bernstein
84e6a25bad
Update help message 2020-04-10 20:59:17 -05:00
Todd Gruben
e7bbc3a0a8 Added delay to allow cooridinator a head start in launch on multi node clusters.
considered using Cluster.Disabled to identify if pilosa was stand alone but settled
on using an empty gossip seeds list
2020-04-09 14:20:01 -05:00
Kuba Podgórski
1217deee1c Rebase 2020-04-08 01:06:18 +02:00
Travis
0e2bb550db add deleted (rebalanced) shards to remoteAvailableShards 2020-04-05 18:37:11 -05:00
Todd Gruben
40a3dce93c Co-authored-by: Travis Turner <github@calfrope.com> 2020-04-05 18:34:43 -05:00
Travis
182c1d3c42 alter tests to allow for shardwidth22
also, reset BitDepth on field and bsiGroup during
importRoaringOverwrite
2020-04-02 17:32:49 -05:00
Travis
4d985653ae convert grpc response to use pql.Decimal 2020-04-01 23:59:50 -05:00
Travis
22cca67d6a Revert "back out the pql.Decimal changes"
This reverts commit 741ba9b268.
2020-04-01 17:46:46 -05:00
Travis
741ba9b268 back out the pql.Decimal changes 2020-04-01 11:10:33 -05:00
Cody Soyland
3a7f385a01 Add updated CircleCI config
- Remove YAML magic
- Remove a lot of duplication
- Update linter
- Use parameterized jobs and matrix build
- Update Docker Hub CD to produce versioned and "latest" images
- Add custom shard width test to workflow
2020-03-31 15:17:11 -05:00
Travis
2724ecfd5f
WIP: include translate partitions in cluster resize instructions
This commit adds `TranslationSources` to the cluster
`ResizeInstruction`. These are the sources of translation
partitions which the receiving node needs in order to support
partition distribution in the new, resized cluster.

This also fixes a bug where index options were not being
encode in the proto Index object. That meant that the schema
transferred via protobuf was not correct. The reason why
things normally worked is because index creation typically
happens on the CreateIndex message, which does include the
options.

TODO:

- [ ] implement the TranslateStore interface for `InMemTranslateStore`
and `mock.TranslateStore`
- [ ] surely need some more tests around the `ReadFrom` and `WriteTo`
2020-03-30 21:24:16 -05:00
Travis
80da129861 require scale argument for decimal fields 2020-03-30 13:38:50 -05:00
Travis
1da7cf09bb support pql.Decimal for decimal field min/max arguments 2020-03-27 16:02:47 -05:00
Cory LaNou
391fda9849
Merge branch 'enterprise' into test-helpers 2020-03-24 15:51:01 -05:00
corylanou
f84185230d
make use of t.Helper 2020-03-24 15:00:32 -05:00
Cody Soyland
1aa1ec51aa Add QueryPQLUnary gRPC call 2020-03-23 16:24:31 -05:00
Travis
cbf80370cb support fragment sync for int and decimal fields
This PR adds support for anti-entropy syncing for integer
and decimal fields. It differs from the logic for other
field types in that it does not rely on a consensus to determine
what the value should be; instead, it considers the correct
values to be those of the primary replica. From there, data
is pushed to all non-primary replicas.
2020-03-13 12:15:54 -05:00
Travis
7aea54936e remove errant println from test 2020-03-03 11:27:36 -06:00
Seebs
cb686dcad0
make mmap test experiment with different amounts of mapping
This is sort of prototype-ish, but the idea is that we use SetMaxMapCount
from syswrap, which already exists, to let us test edge cases like
"what happens if you only sometimes have mapped data".
2020-02-21 16:38:35 -06:00
Matt Jaffee
fe46c84d19
also fix Sum query, but don't convert to float until the last step
this avoids compounding floating point errors while summing up the
numbers, and means less logic needs to change. Should probably convert
min and max to use this approach as well, though they don't suffer
from the compounding error issue, it is simpler.
2020-02-21 14:11:30 -06:00
Matt Jaffee
7321f9427c
min and max should properly scale their output for decimal fields
this involved adding an optional float value to the ValCount struct
which complicated result types, necessitated grpc changes, and needed
quite a few tests at different layers.
2020-02-21 14:11:29 -06:00
Travis Turner
52d450cbaf
Merge branch 'enterprise' into time-panic 2020-02-21 10:31:24 -06:00
Travis
13a1b3a2c1 send nil time value to prevent downstream panic 2020-02-18 15:55:40 -06:00