Commit graph

122 commits

Author SHA1 Message Date
Matt Jaffee
1c1204fc77
modify PQL parser to handle escapes in string values
This modifies the parser to properly "unquote" incoming strings. So if
a string comes in double or single quoted, we approximately follow Go
rules for removing the quotes and processing escape sequences.

The differences from Go are:
1. we only support backslash, quote, tab and newline escape
sequenences.
2. Single quoted strings are supported and work just like double
quoted strings.
3. The peg parser won't actually accept backquoted strings (I don't
think)

Fixes: #411
2020-05-29 07:58:50 -05:00
Matt Jaffee
97ae8e0db7
add kuba testcase, fix race
we fix the race by not returning pointers to the things which we're
keeping in the in-memory store
2020-04-22 15:17:25 -05:00
Matt Jaffee
7c7836f16f
convert transactions to be pointers everywhere
I think this will improve the transaction response messages Kuba
mentioned where it was an empty transaction instead of a nil or not
there... if not it should make it easier to do that anyhow.
2020-04-22 14:01:24 -05:00
Matt Jaffee
c36952a0f1
propagate context throughout transaction stuff 2020-04-20 13:30:48 -05:00
Matt Jaffee
9ad1106647
implement transaction API layer and intra-cluster messaging
also adds a "noSleep" option to the server command to avoid the 5
second sleep we introduced on startup for non-coordinator cluster
nodes. The sleep doesn't seem to be needed in the tests and makes them
much slower.
2020-04-20 13:30:48 -05:00
Travis
4d985653ae convert grpc response to use pql.Decimal 2020-04-01 23:59:50 -05:00
Travis
22cca67d6a Revert "back out the pql.Decimal changes"
This reverts commit 741ba9b268.
2020-04-01 17:46:46 -05:00
Travis
741ba9b268 back out the pql.Decimal changes 2020-04-01 11:10:33 -05:00
Travis
1da7cf09bb support pql.Decimal for decimal field min/max arguments 2020-03-27 16:02:47 -05:00
corylanou
f84185230d
make use of t.Helper 2020-03-24 15:00:32 -05:00
Matt Jaffee
fe46c84d19
also fix Sum query, but don't convert to float until the last step
this avoids compounding floating point errors while summing up the
numbers, and means less logic needs to change. Should probably convert
min and max to use this approach as well, though they don't suffer
from the compounding error issue, it is simpler.
2020-02-21 14:11:30 -06:00
Matt Jaffee
7321f9427c
min and max should properly scale their output for decimal fields
this involved adding an optional float value to the ValCount struct
which complicated result types, necessitated grpc changes, and needed
quite a few tests at different layers.
2020-02-21 14:11:29 -06:00
Kuba Podgórski
f3f11f4a44
Let translate keys as empty strings (#120) 2020-02-18 12:54:24 +01:00
Ben Johnson
c8cefea897
Fix test performance 2020-02-12 10:25:26 -06:00
Ben Johnson
c7c9c1e1d7
v2.0.0
Co-authored-by: Cody Soyland <codysoyland@gmail.com>
2019-10-08 14:56:17 -06:00
Matt Jaffee
ac8c4ed5fe
add concurrent field creation test 2019-09-23 07:51:04 -05:00
Matt Jaffee
9e6662fb00
send POSTed schema to all nodes in cluster
also fix a *bunch* of tests that weren't closing the clusters they
created. Cleaned up one test to use t.Run instead of just checking
everything in a loop
2019-04-29 19:31:23 -05:00
Seebs
ae17fcef7e refix a lint
Another test change made an `err :=` fail because it's no longer
declaring a new variable, but another one needed the :. Or a patch
applied incorrectly. It is a mystery.
2019-04-16 12:08:40 -05:00
Seebs
77d49ded64 so much lint
So with the switch to a new linter, we get a lot of new warnings,
and the majority of them are harmless probably, but a few might be
real. Variously just use _ to suppress warnings, or report errors.
There's probably things here that deserve better fixes, but we can
always revisit it.
2019-04-16 12:07:18 -05:00
Matt Jaffee
d5cfe880f7
address race condition by getting cluster nodes with lock
needed an unlocked version of sendsync for use within the cluster, so also
implemented that. Added a number of tests trying to reproduce the issue, but was
not able to. Not sure it's worth keeping the new tests.
2019-04-05 15:40:24 -05:00
Travis Turner
efb9f97e61
Advertise address and listen on 0.0.0.0
This commit adds support for advertise address by using a new config
option `advertise`, or by defaulting its value to that
specified in `bind`.

Also adds support for listening on 0.0.0.0 by trying to determine
the preferred outbound IP to use for the advertise address.
2019-01-21 22:36:55 -06:00
Yuce Tekol
dc735c17b8
Fixes #1805. Fixes Store call error messages 2019-01-16 20:52:33 +03:00
Yuce Tekol
65c30283d7
adds tests for GroupBy with keys; removes unused Bit message from proto 2019-01-03 15:20:02 +03:00
Matt Jaffee
31ab3d37b8
add stress tests
these produce an issue on master, but it is fixed on this branch
2018-12-19 17:21:57 -06:00
Matt Jaffee
ef6db5cc1a
propogate updates to node details (not just additions and deletions)
to all nodes in cluster, not just coordinator
2018-12-04 14:03:14 -06:00
Yuce Tekol
3d54f737cd
ditch OptFieldTypeTimeWithOptions 2018-11-20 23:21:09 +03:00
Yuce Tekol
7913a419ae
adds NoStandardView field option. Fixes #1710 2018-11-08 18:33:39 +03:00
Travis Turner
cdfcd6db5e
store remoteAvailableShards to file 2018-09-24 15:37:43 -05:00
Cody Soyland
e4a1281997 Merge branch 'master' into configurable-translate-map-size 2018-09-20 16:05:43 -05:00
Travis Turner
b86478c613
test to ensure views match UTC time 2018-09-19 16:17:09 -05:00
Cody Soyland
ebc4d7fc13 Set low map size for 32-bit 2018-09-19 13:37:42 -05:00
Cody Soyland
2394d36108 Adjust tests, fix 32-bit config 2018-09-19 13:30:22 -05:00
Matt Jaffee
1901ffada6
make sure cluster gets into DEGRADED state when adding nodes
previously, losing a node could cause the cluster to go from NORMAL->DEGRADED,
but adding a node would not cause it to go from STARTING->DEGRADED. Cody brought
this up in code review.
2018-08-08 19:42:30 -05:00
Matt Jaffee
bb32706cb6
support DEGRADED in removeNode
now, nodes which have failed and been removed from the running cluster state can
still be manually removed to trigger a cluster resize event. This is important
because otherwise there is no way to cause the cluster to resize itself if a
node fails and you don't want to add a node to replace it.
2018-08-06 16:55:23 -05:00
Matt Jaffee
ba5b46898f
fix determineClusterState bug and add more test cases 2018-08-06 14:55:38 -05:00
Matt Jaffee
48af3adc21
add degraded cluster state and handle node failure
cluster is in degraded state when some number of nodes greater than 0 but less
than replicaN have failed. This is sort of a hybrid of "STARTING" and "NORMAL"
states because we can still respond to queries as in the NORMAL state, but we
need to be alert to re-add nodes to the cluster if they come back online which
required some changes to the cluster logic.

In order to make debugging easier, the test.MustRunCluster functionality now
names the nodes in the cluster explicitly as "node0", "node1", etc. "node0" is
the coordinator.

A number of TODOs are left in the test for scenarios that need to be checked.
2018-08-06 11:57:24 -05:00
Cody Soyland
e9523063b5 Fix linter issues: unused 2018-07-20 10:33:29 -05:00
Cody Soyland
de99b5720f Fix linter issues: vet 2018-07-19 11:37:50 -05:00
Matt Jaffee
5ff77c816a
get rid of unecessary server stuff and export node and uri 2018-07-03 13:26:51 -05:00
Cody Soyland
824474160e Enhance test utilities (introduce Cluster type, improve naming) 2018-06-28 13:34:58 -05:00
Travis Turner
50794bf63b
move fieldOptions unmarshal to the handler
validate fieldOptions in http package
2018-06-25 15:14:22 -05:00
Alan Bernstein
35526dd0d7 Update to new PQL syntax beyond the parser 2018-06-22 08:11:27 -05:00
Travis Turner
fe167ea78c
un-export some top-level functions 2018-06-13 16:44:24 -05:00
Cody Soyland
daffa3b125 Rename InternalHTTPClient -> InternalClient 2018-06-13 09:04:04 -05:00
Cody Soyland
f2c104dfef Migrate HTTP handler and client into http subpackage. 2018-06-12 13:22:40 -05:00
Travis Turner
2a9b1e9e5b
final Frame to Field rename 2018-06-06 01:27:12 -05:00
Travis Turner
4c44c5f33a
fix Frame to Field in tests 2018-06-05 23:33:00 -05:00
Travis Turner
b88ebfb754
GoRename Frame to Field in diagnostics.go 2018-06-05 22:52:57 -05:00
Travis Turner
1d3c4d6fcb
first pass at GoRename Frame to Field in frame.go 2018-06-05 17:52:22 -05:00
Matt Jaffee
c1c89b9ef8
fix generate-config command, use single toml lib
The generate-config command was printing a fixed string rather than calling
NewConfig() which is the canonical source for default config. I also noticed
that we were depending on two different toml libraries, and so collapsed that to
a single one. We have to use pelletier rather than BurntSushi because the viper
library that we use depends on pelletier.
2018-06-01 09:21:38 -05:00