Commit graph

62 commits

Author SHA1 Message Date
Matt Jaffee
7c7836f16f
convert transactions to be pointers everywhere
I think this will improve the transaction response messages Kuba
mentioned where it was an empty transaction instead of a nil or not
there... if not it should make it easier to do that anyhow.
2020-04-22 14:01:24 -05:00
Matt Jaffee
89c1d48a0f
transaction deadline format UTC, lint
also change "deadlineSkew" comparison in tests to account for race tests in CI
seeing false differences
2020-04-20 14:37:28 -05:00
Matt Jaffee
41975de6b8
add transactions external documentation
- make sure client reads/closes all bodies
- support blank transaction ID in http handler
2020-04-20 13:30:48 -05:00
Matt Jaffee
210c7239ab
add HTTP handlers and client for transactions 2020-04-20 13:30:48 -05:00
Travis
22cca67d6a Revert "back out the pql.Decimal changes"
This reverts commit 741ba9b268.
2020-04-01 17:46:46 -05:00
Travis
741ba9b268 back out the pql.Decimal changes 2020-04-01 11:10:33 -05:00
Travis
1da7cf09bb support pql.Decimal for decimal field min/max arguments 2020-03-27 16:02:47 -05:00
corylanou
f84185230d
make use of t.Helper 2020-03-24 15:00:32 -05:00
Ben Johnson
9647d9b4bb fixing additional tests 2020-01-08 09:47:43 -07:00
Travis
5cb37834a0 Wrap return types: RowIdentifiers, Pair, and []Pair
This PR adds a field name (string) to the return types
which represent the values from a specific field. For example,
a TopN query on field `x` would be `TopN(x)` and have results
like:
```
[]Pair{
  {ID: 14, Count: 10},
  {ID: 3, Count: 8},
  {ID: 7, Count: 3},
}
```
In order to know what field this result type refers to, we wrap
`[]Pair` in a new struct called `PairsField` which contains an
addition `Field` string where `x` is stored.

This is useful for informing the gRPC server how to construct
more appropriate headers for the result stream (in this case,
the column headers can now be "x" and "count").

Similar logic was applied to `RowIdentifiers` and `Pair` as well.
2019-12-13 15:36:51 -06:00
Alan Bernstein
cf7d668b49
Support import column attrs in client 2019-12-01 07:33:24 -06:00
Travis
26fc621f09
Support integer predicates in Decimal field range queries. 2019-11-13 10:07:24 -06:00
Matt Jaffee
9c8ad727b5
allow floats in PQL queries for decimal fields
had to workaround some cruft in the parser that was trying to only
support a BETWEEN query as LTE, LTE. Now we have operations for all
combinations of LT and LTE.

unrelated - changed the port a test was binding to as it conflicted
with a port I was using locally.
2019-11-13 10:07:24 -06:00
Travis Turner
1a44f02e3c reset fragment.rowCache after importValue 2019-11-11 16:54:51 -06:00
Matt Jaffee
5dcabfcc7f
support for decimal fields
This commit adds a Decimal field type which is implemented mostly with
the Int field. It adds an optional "Scale" value to the Int field
which means that the values stored in that field are actually meant to
be divided by 10^Scale before being interpreted.

In order to make use of this functionality, we extend the importValue
request to allow a slice of floats rather than just int64. If the
slice of floats is present, each float in the slice is multiplied by
10^Scale and converted to an int64 before being imported. If a slice
of int64 is imported to a Decimal field, it is treated normally, and
scale is ignored. This allows the conversion to be handled at the
client side if desired.

Currently there are Field level methods for querying Float values out
of a decimal field, but no support in PQL or the executor for getting
float values. Going to wait until I can use the generic result type
before doing that, so for now, any values queried will be the scaled
integer values.

needed to add client support for importing float values, and did this
by adding a more general and simplified client method for value
imports.

rewrote api.ImportValue to use the new method which should be more
performant and efficient.

allow floats to be "pilosa import"ed into decimal fields
2019-10-29 16:36:14 -05:00
Ben Johnson
e844e1ad75
Translation store refactor 2019-10-09 08:59:41 -06:00
Ben Johnson
c7c9c1e1d7
v2.0.0
Co-authored-by: Cody Soyland <codysoyland@gmail.com>
2019-10-08 14:56:17 -06:00
Ben Johnson
40803372dd
Add min/max constraints; fix tests 2019-05-19 16:05:22 -06:00
Ben Johnson
d4de122549
Add min/max constraints 2019-05-17 15:52:17 -06:00
Matt Jaffee
9e6662fb00
send POSTed schema to all nodes in cluster
also fix a *bunch* of tests that weren't closing the clusters they
created. Cleaned up one test to use t.Run instead of just checking
everything in a loop
2019-04-29 19:31:23 -05:00
Seebs
77d49ded64 so much lint
So with the switch to a new linter, we get a lot of new warnings,
and the majority of them are harmless probably, but a few might be
real. Variously just use _ to suppress warnings, or report errors.
There's probably things here that deserve better fixes, but we can
always revisit it.
2019-04-16 12:07:18 -05:00
Ben Johnson
9d1e5ca8ce
Merge Range() into Row() call.
This commit refactors the `Range()` call and merges its functionality
into the `Row()` call.
2019-01-09 15:10:49 -07:00
Matt Jaffee
45cd48c1b7
change exists field to _exists 2018-12-17 15:44:50 -06:00
Yuce Tekol
1faa789b31
remove ImportRoaringRequestView type 2018-11-21 14:53:16 +03:00
Yuce Tekol
84c04900e1
Import roaring enpoint accepts a list of views 2018-11-12 18:43:59 +03:00
Travis Turner
e69a8f3db4
handle clear flag on import roaring (http package) 2018-10-24 15:16:08 -05:00
Travis Turner
f02605a528
consolidate ImportOptions setup. use url.Values{}. fix comments. 2018-10-23 17:37:57 -05:00
Travis Turner
f29b6e79ff
add import clear test coverage to http package 2018-10-23 17:37:57 -05:00
Matt Jaffee
495af54c10
Merge branch 'master' into roaring-import 2018-09-13 15:50:16 -05:00
Matt Jaffee
f943c17e0b
ImportRoaring, add remote arg, fix data copy bug
also, handle err properly in client method instead of discarding.
2018-09-13 15:05:05 -05:00
Travis Turner
9b4c67ee60
rename notnull to exists 2018-09-11 15:15:37 -05:00
Travis Turner
a433862c98
add tests for the notnull tracking 2018-09-11 15:14:57 -05:00
Travis Turner
6b75988b71
add key translation to exports 2018-08-20 13:40:58 -05:00
Travis Turner
f1a460aca7
adds view parameter to sync logic for syncing time fields 2018-08-16 11:27:08 -05:00
Travis Turner
177f25ee44
Add tests for importing value with column keys into integer fields.
Fix a bug in protofuf decoding of pilosa.Row.
2018-08-15 16:00:32 -05:00
Travis Turner
f547cf48ce
change t.Fatal() to t.Fatalf() 2018-08-15 15:36:55 -05:00
Travis Turner
1deb42a24a
adjust test to allow time for translation replication 2018-08-15 15:15:51 -05:00
Travis Turner
0a6f0e92d8
add tests to cover the coordinator logic for multi-node clusters 2018-08-14 13:58:37 -05:00
Travis Turner
1feebc890b
fix outdated test 2018-08-14 09:32:40 -05:00
Ben Johnson
a31a08c330
Support keys on import CLI. 2018-08-14 09:32:39 -05:00
Cody Soyland
990780bd93 Fix linter issues: staticcheck (covered by megacheck, along with gosimple and unused) 2018-07-20 11:19:24 -05:00
Cody Soyland
0b86bbb4f5 Fix linter issues: gochecknoinits 2018-07-18 11:58:27 -05:00
Matt Jaffee
0a94d2f10d
Merge branch 'develop' into core-structs 2018-07-05 16:38:14 -05:00
Matt Jaffee
e76a90e69b
change Query and QueryNode to use pilosa.* Query structs 2018-07-05 15:02:33 -05:00
Travis Turner
44f0b992f6
change all CreateField() methods to take functional options instead of FieldOptions 2018-07-04 21:22:35 -05:00
Matt Jaffee
a6a0c6a7c3
unexport Holder.view and prepare to unexport Holder.Fragment 2018-07-02 13:58:20 -05:00
Matt Jaffee
29ad1287d4
refactor executor_test.go 2018-06-30 08:41:19 -05:00
Matt Jaffee
ea77db895a
fix syncholder test and a few bugs
The http internal client's FragmentBlocks and Blockdata methods were being used
incorrectly, and incorrect respectively. One was not being passed a node URI by
monitorAntiEntropy, and the other was always using the defaultURI regardless of
what was passed to it. Antientropy was doubling not working because of this. I
think this crept in pretty recently, so hasn't actually affected anyone.

I exposed a SyncData method on Server so that we can invoke the anti entropy
task manually instead of trying to set up the interval so that it will run and
then sleeping and waiting for it to run. Now that this test works the way it
does, the other anti entropy test is obsolete, and I deleted it.
2018-06-29 14:48:13 -05:00
Matt Jaffee
5766f572b1
fix skipped client TopN test
had to remove the first check which was for the wrong result - because maxShard
is no longer wrong since the broadcaster is actually working.
2018-06-29 12:21:33 -05:00
Travis Turner
5dd7a9556a
rename slice to shard 2018-06-28 14:07:07 -05:00