scriptoriumadmin/featurebase
1
0
Fork 0
mirror of https://github.com/featurebasedb/featurebase.git synced 2026-09-06 08:35:55 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions
668 commits 45 branches 174 tags 85 MiB
Commit graph

668 commits

This branch
This branch
All branches
Author SHA1 Message Date
Todd Gruben
d41f998932 cleaned up constant 2016-09-23 16:04:22 -05:00
Todd Gruben
11a2013f2d fix roaring.Difference crash 2016-09-23 13:27:35 -05:00
Todd Gruben
415507ffe3 fix for intersect and updated tests 2016-09-22 16:47:39 -05:00
Todd Gruben
57810f8ab6 merge 2016-09-22 03:05:53 -05:00
Todd Gruben
f53c2d07d8 merge 2016-09-22 03:05:26 -05:00
tgruben
7c1bf40ff5 Merge pull request #113 from benbjohnson/sort-import 
Add import sorting command
 2016-09-21 16:30:05 -05:00
Ben Johnson
432bf74368
Add import sorting command. 
Introduces new `pilosactl sort` to sort import files by bit
position so they can be inserted faster. Also optimizes container
scanning and adds a `-buffer-size` flag to `import`.
 2016-09-21 14:33:24 -06:00
tgruben
1071f193c0 Merge pull request #112 from benbjohnson/import-cache-2 
Update cache on Import (alternative)
 2016-09-16 15:54:13 -05:00
Ben Johnson
7c07cdf304
Remove import count optimization. 2016-09-16 11:36:54 -06:00
Todd Gruben
3907414a10
cache update on import adjustment 2016-09-16 11:20:31 -06:00
Todd Gruben
3009abbeef
corrected ClearBit 2016-09-16 11:20:12 -06:00
Todd Gruben
60853a6bef
ignore checksum on single node clusters 2016-09-16 11:18:44 -06:00
Todd Gruben
b7780518cc cache update on import adjustment 2016-09-15 13:37:10 -05:00
Todd Gruben
15e425ae8f Merge branch 'master' of github.com:umbel/pilosa 2016-09-13 20:59:23 -05:00
tgruben
84d010e66d Merge pull request #110 from benbjohnson/optimize-top-n 
Optimize TopN() w/ source query.
 2016-09-13 20:58:53 -05:00
Ben Johnson
947eea4668
Optimize TopN() w/ source query. 
This commit makes several changes to optimize the TopN() query:

- Reduce highbits() back from 24-bits to 16-bits.
- Reduce MaxArraySize back from 2^20 to 4096.
- Optimize bitmap count invalidation.
- Parallelize TopN() across nodes.
- Parallelize TopN() across slices.
 2016-09-13 14:55:56 -06:00
Todd Gruben
df2ac89558 Merge branch 'master' of github.com:umbel/pilosa 2016-09-07 14:15:10 -05:00
tgruben
7a349fca54 Merge pull request #109 from benbjohnson/opt-int-count 
Optimize IntersectionCount for Array+Bitmap
 2016-09-07 14:14:41 -05:00
Ben Johnson
44540abd1d
Optimize IntersectionCount for Array+Bitmap 
Reworks the `roaring.intersectionCountArrayBitmap()` call to avoid
using an iterator. Performance of the included benchmark went from
2.5ms to 1.1ms.

Some of the issue with intersectionCount is the increased size of
bitmaps and slices and I need to do additional testing with various
sizes.
 2016-09-07 10:13:31 -06:00
Todd Gruben
aea68cbbd8 Merge branch 'master' of github.com:umbel/pilosa 2016-09-01 12:07:27 -05:00
tgruben
361c5fbeb4 Merge pull request #108 from benbjohnson/remove-mmap-resizing 
Remove post-mmap resizing
 2016-09-01 12:06:38 -05:00
Ben Johnson
0879bab148
Remove post-mmap resizing 2016-09-01 09:18:52 -06:00
Todd Gruben
1968972a17 Merge branch 'master' of github.com:umbel/pilosa 2016-08-30 17:36:21 -05:00
Todd Gruben
d4067ef34e corrected ClearBit 2016-08-30 17:36:00 -05:00
tgruben
e130d99a18 Merge pull request #107 from benbjohnson/export 
Add export command to pilosactl
 2016-08-30 17:25:30 -05:00
Ben Johnson
157b7eb966
Add export command to pilosactl 
CSV exports can now be done with the pilosactl application:

	$ pilosactl -d mydb -f myframe -o MYFILE.csv

If `-o` is not specified then the CSV is written to STDOUT. The
exporter combines all slices for the db/frame to into a single
concatenated CSV file.
 2016-08-30 16:18:12 -06:00
Todd Gruben
758bed45ac Merge branch 'master' of github.com:umbel/pilosa 2016-08-26 12:59:55 -05:00
tgruben
f5a32395fc Merge pull request #105 from benbjohnson/optimize-bitmap 
Optimize bitmap with zero copy (WIP)
 2016-08-25 15:33:00 -05:00
Ben Johnson
54f12b655f
Optimize bitmap with zero copy 
Segments bitmaps by slice and changes `Fragment.Bitmap()` to return
bitmaps with segments which are offset ranges of mmapped bitmaps.
 2016-08-25 13:30:25 -06:00
Todd Gruben
6f7031652f Merge branch 'master' of github.com:umbel/pilosa 2016-08-15 18:36:08 -05:00
tgruben
9659f61420 Merge pull request #104 from benbjohnson/count-range 
CountRange()
 2016-08-15 18:35:58 -05:00
Ben Johnson
888040c26c
add CountRange() 
This commit adds `roaring.Bitmap.CountRange()` to return the number
of bits in a subrange of a bitmap. This significantly improves
server start time and is needed for upcoming zero copy bitmap
optimizations.
 2016-08-15 16:03:25 -06:00
Todd Gruben
5eabf67a11 Merge branch 'master' of github.com:umbel/pilosa 2016-08-12 15:10:57 -05:00
Todd Gruben
13f8ce673e cache loading 2016-08-12 15:10:50 -05:00
tgruben
d2106de4c0 Merge pull request #103 from benbjohnson/fix-count-cache 
Fix count() cache retrieval
 2016-08-12 15:06:53 -05:00
Ben Johnson
76af8b53a3
fix count() cache retrieval 2016-08-12 11:02:52 -06:00
Ben Johnson
07a815eb55
add http trace profiling 2016-08-12 10:58:45 -06:00
Ben Johnson
edce8c537c
add CLI profiling 
This commit adds the `-cpuprofile` flag to the `pilosa` binary so
that startup can be profiled.
 2016-08-12 10:58:45 -06:00
Ben Johnson
f50fc24749
fix bitmap reference bug 2016-08-09 14:57:37 -06:00
Todd Gruben
a833624e20 Merge branch 'master' of github.com:umbel/pilosa 2016-08-08 09:35:43 -05:00
tgruben
d354335374 Merge pull request #102 from benbjohnson/serialize-cache-flush 
Serialize cache flush
 2016-08-08 09:30:47 -05:00
Ben Johnson
15cc4d7d5a
serialize cache flush 
This commit moves the cache flush to the `Index` and only serializes
a single fragment at a time.

Also included in this commit is the `inspect` command for the
`pilsoactl` binary. This provides insight into pilosa data files.
 2016-08-05 14:58:37 -06:00
Todd Gruben
0fd6543497 Merge branch 'master' of github.com:umbel/pilosa 2016-07-25 11:43:58 -05:00
tgruben
916a22da29 Merge pull request #100 from benbjohnson/cluster-attr 
Forward bitmap & profile attributes
 2016-07-25 11:42:52 -05:00
Ben Johnson
eb8c0eee28
forward bitmap & profile attributes 
This commit adds forwarding of bitmap and profile attributes to
other servers in the cluster.
 2016-07-19 13:54:01 -06:00
Todd Gruben
0513dfd9ae Merge branch 'master' of github.com:umbel/pilosa 2016-07-07 11:23:50 -05:00
Todd Gruben
3b32db998d ignore checksum on single node clusters 2016-07-07 11:22:15 -05:00
tgruben
b189961f69 Merge pull request #99 from benbjohnson/optimize-block-checksumming 
Optimize sparse bitmap block checksums
 2016-07-07 11:18:58 -05:00
Ben Johnson
3fdf63f84f
optimize sparse bitmap block checksums 
This commit refactors the block checksumming by removing the
iteration over each block and instead only checking blocks which
have data. This requires merging the cache inspection with the
roaring iterator to reduce CPU time and memory allocations.
 2016-07-07 10:03:38 -06:00
tgruben
148c9105fb Merge pull request #95 from benbjohnson/sync-closing 
Check for early close during AAE
 2016-06-29 11:25:56 -05:00