Commit graph

3762 commits

Author SHA1 Message Date
Travis
01fa368c6a
Import FieldValues
This PR adds the ability to import field values into RangeEnabled frames
using the `pilosa import` command.

Example:
```
pilosa import -i i -f f --field foo sample-vals.csv
```
imports data from sample-vals.csv, which contains data in the format:
```
[ColumnID, Value]
```

Also fixes a bug where `frame.rangeEnabled` was not being set on frame creation.
2017-09-27 11:43:10 -05:00
Travis Turner
3bfa773b20 Merge pull request #842 from travisturner/roaring-optimizations
Roaring optimizations
2017-09-27 11:38:53 -05:00
Travis
c7e672e16f
cleaning up a few of the tests 2017-09-27 11:23:00 -05:00
Todd Gruben
de2f16b617
optimzed differenceRunBitmap 2017-09-27 11:23:00 -05:00
Todd Gruben
3f3cc60531
micro optimization to bitmap difference 2017-09-27 11:17:15 -05:00
Todd Gruben
50dab75c82
optimize unionArrayBitmap 2017-09-27 11:17:15 -05:00
Todd Gruben
48f34836e5
optimize differenceBitmapArray 2017-09-27 11:17:15 -05:00
Todd Gruben
08133368e3
optimize differenceArrayBitmap 2017-09-27 11:17:15 -05:00
Todd Gruben
71b7216c51
fix to to allocate properly for difference test 2017-09-27 11:17:15 -05:00
Todd Gruben
4487e6aadd
optimized differenceBitmapBitmap 2017-09-27 11:17:15 -05:00
Matthew Jaffee
de6a25cb8d
rewrite intersectArrayBitmap for perf 2017-09-27 11:17:15 -05:00
Todd Gruben
1736a40b74
check for array conversion to intersectbitmapbitmap 2017-09-27 11:17:14 -05:00
Todd Gruben
1db1ce63b7
optimize intersectionBitmapBitmap 2017-09-27 11:17:14 -05:00
Travis Turner
1fb540f40b Merge pull request #844 from travisturner/bug-fix-condition-string
Don't include an equal sign in the string representation of a Condition
2017-09-27 11:14:56 -05:00
Travis Turner
917f41e8eb Merge pull request #847 from travisturner/field-range-between
Implements BETWEEN for Range queries.
2017-09-27 11:14:31 -05:00
Travis
ab908227be
remove private function fieldRangeBetween 2017-09-27 11:02:32 -05:00
Linh Vo
7e53a4ed1d more test 2017-09-27 10:10:43 -05:00
Yuce Tekol
ce9033f828
updated with master 2017-09-27 16:28:27 +03:00
Yuce Tekol
89ecd5c0e8
more updates 2017-09-27 16:24:38 +03:00
Michael Baird
e87e7fda68 Add some new Diagnostics metrics 2017-09-26 22:42:35 -05:00
Michael Baird
566f4116f1 Using MultiStatsClient add Diagnostics 2017-09-26 22:42:21 -05:00
Michael Baird
68ce07b45d add Open/Close interface to Stats packages 2017-09-26 22:40:55 -05:00
Michael Baird
40f1457eef Diagnostics package 2017-09-26 22:39:57 -05:00
Travis Turner
26532b5753 Merge pull request #845 from travisturner/forgotten-error-check
forgotten error check which fooled the compiler but not guru/rename
2017-09-26 16:11:29 -05:00
Travis
7af8484d2e
handle another forgotten error check 2017-09-26 15:35:48 -05:00
Travis Turner
204898e612 Merge pull request #848 from travisturner/lock-to-read-lock
Lock to read lock
2017-09-26 15:29:58 -05:00
Linh Vo
a1a6c7d717 update test 2017-09-26 13:46:00 -05:00
Cody Soyland
be796fa2e0 Fix section heading name and incorrect flag. 2017-09-26 11:06:46 -05:00
Linh Vo
505363d8ed get fields 2017-09-26 10:57:22 -05:00
Travis Turner
046fb9b97e Merge pull request #846 from travisturner/reduce-executor-locks
micro optimization: reduce Lock calls in executor
2017-09-25 17:12:17 -05:00
Travis Turner
6066c29ea7 Merge pull request #849 from travisturner/flip-bitmap
implement container.flipBitmap() to improve differenceRunBitmap()
2017-09-25 17:10:37 -05:00
Travis
0d4f634dd8
implement container.flipBitmap() to improve differenceRunBitmap() 2017-09-25 15:48:40 -05:00
Travis
e53f218c49
consolidate the BaseValue code. Add tests for BaseValue ranges 2017-09-25 15:37:27 -05:00
Matthew Jaffee
f257be9926
FieldSum calls fragment.row which needs write lock
remove read lock and call fragment.Row
2017-09-25 15:24:52 -05:00
Matthew Jaffee
3aecb61285
more read-only locking for maxslice, timequantum, and Index.Frames 2017-09-25 15:20:17 -05:00
Matthew Jaffee
c5b6c39c56
read only lock in fragment.FieldSum 2017-09-25 15:20:00 -05:00
Matthew Jaffee
1210489f76
frame rowlabel and options read-only lock 2017-09-25 15:19:43 -05:00
Matthew Jaffee
023dbbcdfd
make index column label and options RLocks 2017-09-25 15:18:46 -05:00
Matthew Jaffee
ce7283ae1b
convert to RWMutex in index, holder, view, and frame
use RLock and RUnlock in mapperLocal hotpath
2017-09-25 15:16:40 -05:00
Travis
2b88d278bc
Implements BETWEEN for Range queries.
The PQL looks like:
```
Range(frame=f, field0 >< [200,610])
```

One thing I noticed while implementing this is that it doesn't seem
like `FieldRange()` is used in either `Frame` or `View`; the Executor
calls `Fragment.FieldRange()` directly. The problem with this is that
the offset logic is calculated in the Frame, but since the Executor
doesn't go through Frame, then the Executor also has to calculate
the offset before calling `Fragment.FieldRange`. We should unify this
logic somewhere. Note, this applies to both `FieldRange` and
`FieldRangeBetween`.
2017-09-25 15:11:04 -05:00
Todd Gruben
c8f2982e50
micro optimization: reduce Lock calls in executor 2017-09-25 15:02:49 -05:00
Matthew Jaffee
959d0c0ef9
forgotten error check which fooled the compiler but not guru/rename
see https://github.com/golang/go/issues/8560

also gofmting happening
2017-09-25 14:59:38 -05:00
Travis
51ca778d77
Don't include an equal sign in the string representation of a Condition 2017-09-25 14:51:15 -05:00
Yuce Tekol
b11b8b074d Merge pull request #837 from yuce/765-auto-create-frames-on-import
Enables auto-creating the schema on imports; Resolves #765
2017-09-25 21:55:19 +03:00
Matthew Jaffee
3e18cdecd3 Merge pull request #820 from tgruben/unmarshallbinary
Reuse container storage on UnmarshalBinary to improve memory utilization
2017-09-25 12:20:38 -05:00
Yuce Tekol
93ba7848fc Merge pull request #831 from yuce/client-libraries-doc
Updated client library docs
2017-09-25 19:57:47 +03:00
Yuce Tekol
5cc05cb000
yet another formatting fix 2017-09-25 17:32:38 +03:00
Yuce Tekol
e1d2bd1087
fix formatting 2017-09-25 17:23:11 +03:00
Yuce Tekol
24d9fa0086
deprecate labels 2017-09-25 17:22:21 +03:00
Yuce Tekol
ad05938a5e
Enables auto-creating the schema on imports; Resolves #765 2017-09-22 14:56:36 +03:00