Commit graph

62 commits

Author SHA1 Message Date
Kuba Podgórski
ba7f039dd1 Rename etag to createdAt 2020-06-03 13:26:16 +02:00
Kuba Podgórski
3d270f45d2 Add (in memory) ETag to index and fields 2020-06-03 13:26:16 +02:00
Travis
182c1d3c42 alter tests to allow for shardwidth22
also, reset BitDepth on field and bsiGroup during
importRoaringOverwrite
2020-04-02 17:32:49 -05:00
Travis
22cca67d6a Revert "back out the pql.Decimal changes"
This reverts commit 741ba9b268.
2020-04-01 17:46:46 -05:00
Travis
741ba9b268 back out the pql.Decimal changes 2020-04-01 11:10:33 -05:00
Travis
80da129861 require scale argument for decimal fields 2020-03-30 13:38:50 -05:00
Travis
1da7cf09bb support pql.Decimal for decimal field min/max arguments 2020-03-27 16:02:47 -05:00
corylanou
f84185230d
make use of t.Helper 2020-03-24 15:00:32 -05:00
Travis
cbf80370cb support fragment sync for int and decimal fields
This PR adds support for anti-entropy syncing for integer
and decimal fields. It differs from the logic for other
field types in that it does not rely on a consensus to determine
what the value should be; instead, it considers the correct
values to be those of the primary replica. From there, data
is pushed to all non-primary replicas.
2020-03-13 12:15:54 -05:00
Travis
7aea54936e remove errant println from test 2020-03-03 11:27:36 -06:00
Kuba Podgórski
f3f11f4a44
Let translate keys as empty strings (#120) 2020-02-18 12:54:24 +01:00
Travis
d9ef4c0986
use OpenInMemTranslateStore by default in tests 2020-02-12 10:25:25 -06:00
Travis
49c8bf01a0
WIP: Thread OpenTranslateStore through Holder to Index 2020-02-12 10:25:25 -06:00
Ben Johnson
2f76283f03 fix replication errors & test races 2020-01-08 09:47:43 -07:00
Travis
5cb37834a0 Wrap return types: RowIdentifiers, Pair, and []Pair
This PR adds a field name (string) to the return types
which represent the values from a specific field. For example,
a TopN query on field `x` would be `TopN(x)` and have results
like:
```
[]Pair{
  {ID: 14, Count: 10},
  {ID: 3, Count: 8},
  {ID: 7, Count: 3},
}
```
In order to know what field this result type refers to, we wrap
`[]Pair` in a new struct called `PairsField` which contains an
addition `Field` string where `x` is stored.

This is useful for informing the gRPC server how to construct
more appropriate headers for the result stream (in this case,
the column headers can now be "x" and "count").

Similar logic was applied to `RowIdentifiers` and `Pair` as well.
2019-12-13 15:36:51 -06:00
Matt Jaffee
2651bfbd88
test and fix authentication of client certs btwn cluster nodes
This change should have been adding the "GetClientCertificate"
function in server/tlsconfig.go. This is in addition to the
GetCertificate func which is only used by servers. It ended up being
much more involved for a few reasons:

1. We had no way of passing a configured HTTP client into the
translate store stuff.

2. Our cluster tests assumed http, not HTTPS, and didn't have any way
to pass the necessary configuration in.

3. I encountered what turned out to be an unrelated bug in
cmd/server_test.go which is why I moved "close(m.Started)" in
server/server.go. Basically, I was running something on port 10111
which caused the test to fail (because it was trying to bind to that),
but the failure was not immediately caught during server startup
because the m.Started channel got closed which allowed the test code
to fall through to where it called m.Close() which then got a nil
pointer exception because m.Handler had never been set up.

4. Our test code was assuming that it could create clients that
ignored the config, which meant they didn't do TLS. I added an
InternalClient() method to pilosa.Server to expose the configured
client.
2019-10-18 15:17:35 -05:00
Ben Johnson
e844e1ad75
Translation store refactor 2019-10-09 08:59:41 -06:00
Ben Johnson
c7c9c1e1d7
v2.0.0
Co-authored-by: Cody Soyland <codysoyland@gmail.com>
2019-10-08 14:56:17 -06:00
Yuce Tekol
c8a3dc8c18
fix int min max test for 32bit 2019-05-28 14:25:46 +03:00
Yuce Tekol
5e102154ca
make linter happy 2019-05-28 14:12:04 +03:00
Yuce Tekol
5f4c5d4d35
added test for 1977 fix 2019-05-28 13:54:12 +03:00
Matt Jaffee
9e6662fb00
send POSTed schema to all nodes in cluster
also fix a *bunch* of tests that weren't closing the clusters they
created. Cleaned up one test to use t.Run instead of just checking
everything in a loop
2019-04-29 19:31:23 -05:00
Matt Jaffee
a6ee142403
update docs, add test 2019-04-26 18:27:10 -05:00
Seebs
77d49ded64 so much lint
So with the switch to a new linter, we get a lot of new warnings,
and the majority of them are harmless probably, but a few might be
real. Variously just use _ to suppress warnings, or report errors.
There's probably things here that deserve better fixes, but we can
always revisit it.
2019-04-16 12:07:18 -05:00
Matt Jaffee
27ff1b69c6
fix error messages in test. Fatalf=>Errorf to see more errors. 2019-04-15 08:41:13 -05:00
Cody Soyland
7ede65bf80
Merge branch 'master' into shardwidth22 2019-04-11 10:10:47 -05:00
Cody Soyland
7bb6fdffcb Add correct content type to query responses. Fixes #1873 2019-04-10 15:49:33 -05:00
Cody Soyland
992a075cfb Return empty result set when query empty. Fixes #1840 2019-04-10 15:27:12 -05:00
Matt Jaffee
836b467d3d
add support to modify shard width at build time
use "make <x> SHARD_WIDTH=nn"

fix tests to run and pass at different shardwidths

add shardwidth22 test to circle ci
2019-04-04 13:46:26 -05:00
Seebs
5ee87bbf7c add server stats to /info endpoint
report the approximate hardware specs (CPU speed, cores, memory)
of the server in the /info endpoint. This may be useful when
benchmarking.

We do some workarounds because gopsutil's core count output is
confusingly different between Linux and Darwin, and the MHz output
is usually wrong on Linux. Intel's app notes say to just parse
the model string. Whyyyyyyy.
2019-03-26 09:14:51 -05:00
Yuce Tekol
767062ab7c
Adds shardWidth to index info in schema 2019-03-01 14:57:46 +03:00
Yuce Tekol
d543689868
updated translate keys test to include new keys 2018-11-26 17:52:53 +03:00
Yuce Tekol
eb9e609794
Merged with master 2018-11-26 17:03:38 +03:00
Yuce Tekol
9c05155db4
Added /internal/translate/keys endpoint 2018-11-21 16:35:50 +03:00
Yuce Tekol
1faa789b31
remove ImportRoaringRequestView type 2018-11-21 14:53:16 +03:00
Yuce Tekol
1642e22872
fixed handler tests 2018-11-12 19:24:32 +03:00
Yuce Tekol
84c04900e1
Import roaring enpoint accepts a list of views 2018-11-12 18:43:59 +03:00
Yuce Tekol
066c3a4853
Fixes #1660 2018-10-01 14:15:51 +03:00
Matt Jaffee
7b0d4d75b4
remove validators on import-roaring and test handler
validators are for query args, not url vars. Also some misc cleanup and error
handling in the handler.
2018-09-13 13:14:35 -05:00
Matt Jaffee
a3243f99e1
linter fixes - reorder return vals, remove unused const 2018-09-12 13:24:59 -05:00
Cody Soyland
c44bb65163 Add more time to sleep in translator tests due to CI failures 2018-07-31 20:39:05 -05:00
Cody Soyland
0816ea8ecb Add sleep to test to wait for key replication 2018-07-31 12:11:21 -05:00
Cody Soyland
e79cd4b199 Add JSON parsing to translator test to verify keys 2018-07-31 11:40:30 -05:00
Cody Soyland
ca800c683e Add test for cluster translator 2018-07-24 14:57:06 -05:00
Travis Turner
babdb5ff4e
adjust test to account for new source of error downstream 2018-07-06 17:38:06 -05:00
Matt Jaffee
0a94d2f10d
Merge branch 'develop' into core-structs 2018-07-05 16:38:14 -05:00
Matt Jaffee
a164233c92
fix handler tests not to use internal and fix bug 2018-07-05 16:21:27 -05:00
Travis Turner
44f0b992f6
change all CreateField() methods to take functional options instead of FieldOptions 2018-07-04 21:22:35 -05:00
Travis Turner
cba91be126
move internal http endpoints under /internal 2018-07-02 08:48:23 -05:00
Travis Turner
3adc3f5978
add tests for index and field success responses 2018-07-01 19:54:09 -05:00