scriptoriumadmin/featurebase
1
0
Fork 0
mirror of https://github.com/featurebasedb/featurebase.git synced 2026-09-06 16:45:55 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions
642 commits 45 branches 174 tags 85 MiB
Commit graph

642 commits

This branch
This branch
All branches
Author SHA1 Message Date
Todd Gruben
758bed45ac Merge branch 'master' of github.com:umbel/pilosa 2016-08-26 12:59:55 -05:00
tgruben
f5a32395fc Merge pull request #105 from benbjohnson/optimize-bitmap 
Optimize bitmap with zero copy (WIP)
 2016-08-25 15:33:00 -05:00
Ben Johnson
54f12b655f
Optimize bitmap with zero copy 
Segments bitmaps by slice and changes `Fragment.Bitmap()` to return
bitmaps with segments which are offset ranges of mmapped bitmaps.
 2016-08-25 13:30:25 -06:00
Todd Gruben
6f7031652f Merge branch 'master' of github.com:umbel/pilosa 2016-08-15 18:36:08 -05:00
tgruben
9659f61420 Merge pull request #104 from benbjohnson/count-range 
CountRange()
 2016-08-15 18:35:58 -05:00
Ben Johnson
888040c26c
add CountRange() 
This commit adds `roaring.Bitmap.CountRange()` to return the number
of bits in a subrange of a bitmap. This significantly improves
server start time and is needed for upcoming zero copy bitmap
optimizations.
 2016-08-15 16:03:25 -06:00
Todd Gruben
5eabf67a11 Merge branch 'master' of github.com:umbel/pilosa 2016-08-12 15:10:57 -05:00
Todd Gruben
13f8ce673e cache loading 2016-08-12 15:10:50 -05:00
tgruben
d2106de4c0 Merge pull request #103 from benbjohnson/fix-count-cache 
Fix count() cache retrieval
 2016-08-12 15:06:53 -05:00
Ben Johnson
76af8b53a3
fix count() cache retrieval 2016-08-12 11:02:52 -06:00
Ben Johnson
07a815eb55
add http trace profiling 2016-08-12 10:58:45 -06:00
Ben Johnson
edce8c537c
add CLI profiling 
This commit adds the `-cpuprofile` flag to the `pilosa` binary so
that startup can be profiled.
 2016-08-12 10:58:45 -06:00
Ben Johnson
f50fc24749
fix bitmap reference bug 2016-08-09 14:57:37 -06:00
Todd Gruben
a833624e20 Merge branch 'master' of github.com:umbel/pilosa 2016-08-08 09:35:43 -05:00
tgruben
d354335374 Merge pull request #102 from benbjohnson/serialize-cache-flush 
Serialize cache flush
 2016-08-08 09:30:47 -05:00
Ben Johnson
15cc4d7d5a
serialize cache flush 
This commit moves the cache flush to the `Index` and only serializes
a single fragment at a time.

Also included in this commit is the `inspect` command for the
`pilsoactl` binary. This provides insight into pilosa data files.
 2016-08-05 14:58:37 -06:00
Todd Gruben
0fd6543497 Merge branch 'master' of github.com:umbel/pilosa 2016-07-25 11:43:58 -05:00
tgruben
916a22da29 Merge pull request #100 from benbjohnson/cluster-attr 
Forward bitmap & profile attributes
 2016-07-25 11:42:52 -05:00
Ben Johnson
eb8c0eee28
forward bitmap & profile attributes 
This commit adds forwarding of bitmap and profile attributes to
other servers in the cluster.
 2016-07-19 13:54:01 -06:00
Todd Gruben
0513dfd9ae Merge branch 'master' of github.com:umbel/pilosa 2016-07-07 11:23:50 -05:00
Todd Gruben
3b32db998d ignore checksum on single node clusters 2016-07-07 11:22:15 -05:00
tgruben
b189961f69 Merge pull request #99 from benbjohnson/optimize-block-checksumming 
Optimize sparse bitmap block checksums
 2016-07-07 11:18:58 -05:00
Ben Johnson
3fdf63f84f
optimize sparse bitmap block checksums 
This commit refactors the block checksumming by removing the
iteration over each block and instead only checking blocks which
have data. This requires merging the cache inspection with the
roaring iterator to reduce CPU time and memory allocations.
 2016-07-07 10:03:38 -06:00
tgruben
148c9105fb Merge pull request #95 from benbjohnson/sync-closing 
Check for early close during AAE
 2016-06-29 11:25:56 -05:00
Ben Johnson
583aef6bbb
check for early close during AAE 
This commit adds a `Closing` channel to the `IndexSyncer` and the
`FragmentSyncer` that is periodically checked during execution.
If the channel is closed then an in-progress sync is immediately
stopped.
 2016-06-29 10:18:24 -06:00
Todd Gruben
e738804941 Merge branch 'master' of github.com:umbel/pilosa 2016-06-29 10:28:19 -05:00
tgruben
7ed40b4b4a Merge pull request #94 from benbjohnson/optimize-union-difference 
Optimize union() & difference()
 2016-06-28 15:34:38 -05:00
Ben Johnson
e4c7c9d1e8
optimize union() & difference() 
This commit moves the `Union()` and `Difference()` operations into
the `roaring` package to improve performance.
 2016-06-28 14:29:37 -06:00
Todd Gruben
23e9d7e159 Merge branch 'master' of github.com:umbel/pilosa 2016-06-23 09:05:21 -05:00
tgruben
da419863fc Merge pull request #93 from benbjohnson/set-bulk-attrs 
Optimize bulk bitmap attribute update
 2016-06-22 21:19:23 -05:00
Ben Johnson
b666f19fc7
optimize bulk bitmap attribute update 
This commit adds a fast path for optimizing queries that only
have `SetBitmapAttrs()` calls in them. It does this by grouping
the calls into a single Bolt transaction and also by merging
multiple attribute updates per ID into one update.
 2016-06-22 20:04:49 -06:00
Todd Gruben
e64ac426e9 Merge branch 'master' of github.com:umbel/pilosa 2016-06-21 08:45:10 -05:00
tgruben
c462b45a3a Merge pull request #91 from benbjohnson/bit-split 
roaring: change bit split to 40/24
 2016-06-21 08:30:36 -05:00
Ben Johnson
059f34788e
roaring: change bit split to 40/24 
This commit changes the `roaring.Bitmap` to use a 40/24 split in
the high and low bits. This change also requires the low bits
to use `uint32` instead of `uint16`.
 2016-06-20 20:37:05 -06:00
Todd Gruben
021032dc2d Merge branch 'master' of github.com:umbel/pilosa 2016-06-15 08:56:47 -05:00
tgruben
42400cd92b Merge pull request #90 from benbjohnson/optimize-intersection 
Optimize intersect()
 2016-06-14 16:57:52 -05:00
Ben Johnson
5a6d81543c
optimize intersect() 
This commit moves bitmap intersection into the `roaring` package
in order to reduce allocations.

Benchmarks against real world datasets show a speed improvement of
353%. Previously, intersection operated at approximately 5.6M
bits/sec against two 60K bit bitmaps with 30% overlap. The new
intersection operates at approximately 20M bits/sec.
 2016-06-14 15:19:21 -06:00
Todd Gruben
620a109046 Merge branch 'master' of github.com:umbel/pilosa 2016-06-10 10:21:27 -05:00
Todd Gruben
98c7f8d361 tuning slicewidth 2016-06-10 10:21:23 -05:00
Todd Gruben
0010ea14f8 added info to owns fragrment error message 2016-06-10 10:19:54 -05:00
tgruben
aaea7a16f9 Merge pull request #89 from benbjohnson/buffered-import 
Add buffered snapshot writer
 2016-06-10 10:00:43 -05:00
Ben Johnson
64c29d4b6e
add buffered snapshot writer 
This commit wraps the snapshot in a `bufio.Writer`.
 2016-06-10 08:58:37 -06:00
tgruben
882d105e99 Merge pull request #88 from benbjohnson/optimize-import 
Optimize Import
 2016-06-09 16:47:27 -05:00
Ben Johnson
c330c2e4c0
sort bits on import 
This commit sorts bits from import files before performing the input.
 2016-06-09 14:25:41 -06:00
tgruben
2359f3975e Merge pull request #87 from benbjohnson/fix-client-fragment-nodes 
Fix Client.FragmentNodes()
 2016-06-08 11:27:31 -05:00
Ben Johnson
2d69f2c989
fix Client.FragmentNodes() 
This commit fixes the hardcoded database field in
`Client.FragmentNodes()`.
 2016-06-08 10:23:44 -06:00
Todd Gruben
a204ae419a fixed typo 2016-06-07 15:29:28 -05:00
Todd Gruben
2145a568d6 Merge branch 'master' of github.com:umbel/pilosa 2016-06-07 15:25:42 -05:00
Todd Gruben
b140eb1c56 added logging on import;crashing bug on sync on empty fragment 2016-06-07 15:25:37 -05:00
tgruben
bce1e53313 Merge pull request #86 from benbjohnson/fragment-allocation 
Use db in fragment allocation
 2016-06-07 15:18:48 -05:00