scriptoriumadmin/featurebase
1
0
Fork 0
mirror of https://github.com/featurebasedb/featurebase.git synced 2026-08-28 10:54:59 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions
671 commits 45 branches 174 tags 85 MiB
Commit graph

671 commits

This branch
This branch
All branches
Author SHA1 Message Date
Ben Johnson
d5ad5fbb05 Add profile & bitmap attribute anti-entropy. 
Adds block-based anti-entropy for the attribute stores and hooks
into the existing `IndexSyncer` to regulate frequency of syncs.
 2016-09-30 11:47:43 -06:00
tgruben
c92f37ac3c Merge pull request #117 from umbel/bug 
Bug
 2016-09-27 17:20:59 -05:00
Todd Gruben
f2f7de4fd7 bug in difference 2016-09-27 16:07:48 -05:00
Todd Gruben
72db5500ca bug in Union 2016-09-27 16:04:20 -05:00
Todd Gruben
d699d41ea7 added docs for TopN attributes 2016-09-27 14:07:29 -05:00
tgruben
bd84fa3404 Merge pull request #116 from umbel/difference_bug 
Difference bug
 2016-09-23 16:05:23 -05:00
Todd Gruben
ad852fdbf1 cleaned up constant 2016-09-23 16:04:22 -05:00
Todd Gruben
5f65636199 fix roaring.Difference crash 2016-09-23 13:27:35 -05:00
Todd Gruben
6a8973f7c9 fix for intersect and updated tests 2016-09-22 16:47:39 -05:00
Todd Gruben
0a9dcee2f1 merge 2016-09-22 03:05:53 -05:00
Todd Gruben
f006270971 merge 2016-09-22 03:05:26 -05:00
tgruben
286762e804 Merge pull request #113 from benbjohnson/sort-import 
Add import sorting command
 2016-09-21 16:30:05 -05:00
Ben Johnson
46bef724a5 Add import sorting command. 
Introduces new `pilosactl sort` to sort import files by bit
position so they can be inserted faster. Also optimizes container
scanning and adds a `-buffer-size` flag to `import`.
 2016-09-21 14:33:24 -06:00
tgruben
c1fc750d21 Merge pull request #112 from benbjohnson/import-cache-2 
Update cache on Import (alternative)
 2016-09-16 15:54:13 -05:00
Ben Johnson
b40d4385e8 Remove import count optimization. 2016-09-16 11:36:54 -06:00
Todd Gruben
ec4b1352d1 cache update on import adjustment 2016-09-16 11:20:31 -06:00
Todd Gruben
ced399c6fc corrected ClearBit 2016-09-16 11:20:12 -06:00
Todd Gruben
4035763ef4 ignore checksum on single node clusters 2016-09-16 11:18:44 -06:00
Todd Gruben
8913216a6e cache update on import adjustment 2016-09-15 13:37:10 -05:00
Todd Gruben
46b3161c47 Merge branch 'master' of github.com:umbel/pilosa 2016-09-13 20:59:23 -05:00
tgruben
a89af88d47 Merge pull request #110 from benbjohnson/optimize-top-n 
Optimize TopN() w/ source query.
 2016-09-13 20:58:53 -05:00
Ben Johnson
cf4297a771 Optimize TopN() w/ source query. 
This commit makes several changes to optimize the TopN() query:

- Reduce highbits() back from 24-bits to 16-bits.
- Reduce MaxArraySize back from 2^20 to 4096.
- Optimize bitmap count invalidation.
- Parallelize TopN() across nodes.
- Parallelize TopN() across slices.
 2016-09-13 14:55:56 -06:00
Todd Gruben
244077aef7 Merge branch 'master' of github.com:umbel/pilosa 2016-09-07 14:15:10 -05:00
tgruben
9f345aec93 Merge pull request #109 from benbjohnson/opt-int-count 
Optimize IntersectionCount for Array+Bitmap
 2016-09-07 14:14:41 -05:00
Ben Johnson
39ed7de5c4 Optimize IntersectionCount for Array+Bitmap 
Reworks the `roaring.intersectionCountArrayBitmap()` call to avoid
using an iterator. Performance of the included benchmark went from
2.5ms to 1.1ms.

Some of the issue with intersectionCount is the increased size of
bitmaps and slices and I need to do additional testing with various
sizes.
 2016-09-07 10:13:31 -06:00
Todd Gruben
51560213f9 Merge branch 'master' of github.com:umbel/pilosa 2016-09-01 12:07:27 -05:00
tgruben
cc330f6c00 Merge pull request #108 from benbjohnson/remove-mmap-resizing 
Remove post-mmap resizing
 2016-09-01 12:06:38 -05:00
Ben Johnson
089650b845 Remove post-mmap resizing 2016-09-01 09:18:52 -06:00
Todd Gruben
c12d8e096b Merge branch 'master' of github.com:umbel/pilosa 2016-08-30 17:36:21 -05:00
Todd Gruben
f70a91c614 corrected ClearBit 2016-08-30 17:36:00 -05:00
tgruben
fdf4a35f6d Merge pull request #107 from benbjohnson/export 
Add export command to pilosactl
 2016-08-30 17:25:30 -05:00
Ben Johnson
39e0d22598 Add export command to pilosactl 
CSV exports can now be done with the pilosactl application:

	$ pilosactl -d mydb -f myframe -o MYFILE.csv

If `-o` is not specified then the CSV is written to STDOUT. The
exporter combines all slices for the db/frame to into a single
concatenated CSV file.
 2016-08-30 16:18:12 -06:00
Todd Gruben
32fec201c0 Merge branch 'master' of github.com:umbel/pilosa 2016-08-26 12:59:55 -05:00
tgruben
84bbf331a6 Merge pull request #105 from benbjohnson/optimize-bitmap 
Optimize bitmap with zero copy (WIP)
 2016-08-25 15:33:00 -05:00
Ben Johnson
04c814141a Optimize bitmap with zero copy 
Segments bitmaps by slice and changes `Fragment.Bitmap()` to return
bitmaps with segments which are offset ranges of mmapped bitmaps.
 2016-08-25 13:30:25 -06:00
Todd Gruben
838a7ee0f6 Merge branch 'master' of github.com:umbel/pilosa 2016-08-15 18:36:08 -05:00
tgruben
e4f245a490 Merge pull request #104 from benbjohnson/count-range 
CountRange()
 2016-08-15 18:35:58 -05:00
Ben Johnson
650e03cc79 add CountRange() 
This commit adds `roaring.Bitmap.CountRange()` to return the number
of bits in a subrange of a bitmap. This significantly improves
server start time and is needed for upcoming zero copy bitmap
optimizations.
 2016-08-15 16:03:25 -06:00
Todd Gruben
ee01ea624f Merge branch 'master' of github.com:umbel/pilosa 2016-08-12 15:10:57 -05:00
Todd Gruben
f5f3a58e96 cache loading 2016-08-12 15:10:50 -05:00
tgruben
5d48436e97 Merge pull request #103 from benbjohnson/fix-count-cache 
Fix count() cache retrieval
 2016-08-12 15:06:53 -05:00
Ben Johnson
6ad7d0bfab fix count() cache retrieval 2016-08-12 11:02:52 -06:00
Ben Johnson
96680a262c add http trace profiling 2016-08-12 10:58:45 -06:00
Ben Johnson
4883dea5f8 add CLI profiling 
This commit adds the `-cpuprofile` flag to the `pilosa` binary so
that startup can be profiled.
 2016-08-12 10:58:45 -06:00
Ben Johnson
3150471c3c fix bitmap reference bug 2016-08-09 14:57:37 -06:00
Todd Gruben
d79a5e7b0b Merge branch 'master' of github.com:umbel/pilosa 2016-08-08 09:35:43 -05:00
tgruben
301c3ec184 Merge pull request #102 from benbjohnson/serialize-cache-flush 
Serialize cache flush
 2016-08-08 09:30:47 -05:00
Ben Johnson
a063421467 serialize cache flush 
This commit moves the cache flush to the `Index` and only serializes
a single fragment at a time.

Also included in this commit is the `inspect` command for the
`pilsoactl` binary. This provides insight into pilosa data files.
 2016-08-05 14:58:37 -06:00
Todd Gruben
322da388f8 Merge branch 'master' of github.com:umbel/pilosa 2016-07-25 11:43:58 -05:00
tgruben
9732b55caa Merge pull request #100 from benbjohnson/cluster-attr 
Forward bitmap & profile attributes
 2016-07-25 11:42:52 -05:00