scriptoriumadmin/featurebase
1
0
Fork 0
mirror of https://github.com/featurebasedb/featurebase.git synced 2026-10-07 11:27:50 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions
4641 commits 45 branches 174 tags 85 MiB
Commit graph

4641 commits

This branch
This branch
All branches
Author SHA1 Message Date
tgruben
fdf4a35f6d Merge pull request #107 from benbjohnson/export 
Add export command to pilosactl
 2016-08-30 17:25:30 -05:00
Ben Johnson
39e0d22598 Add export command to pilosactl 
CSV exports can now be done with the pilosactl application:

	$ pilosactl -d mydb -f myframe -o MYFILE.csv

If `-o` is not specified then the CSV is written to STDOUT. The
exporter combines all slices for the db/frame to into a single
concatenated CSV file.
 2016-08-30 16:18:12 -06:00
Todd Gruben
32fec201c0 Merge branch 'master' of github.com:umbel/pilosa 2016-08-26 12:59:55 -05:00
tgruben
84bbf331a6 Merge pull request #105 from benbjohnson/optimize-bitmap 
Optimize bitmap with zero copy (WIP)
 2016-08-25 15:33:00 -05:00
Ben Johnson
04c814141a Optimize bitmap with zero copy 
Segments bitmaps by slice and changes `Fragment.Bitmap()` to return
bitmaps with segments which are offset ranges of mmapped bitmaps.
 2016-08-25 13:30:25 -06:00
Todd Gruben
838a7ee0f6 Merge branch 'master' of github.com:umbel/pilosa 2016-08-15 18:36:08 -05:00
tgruben
e4f245a490 Merge pull request #104 from benbjohnson/count-range 
CountRange()
 2016-08-15 18:35:58 -05:00
Ben Johnson
650e03cc79 add CountRange() 
This commit adds `roaring.Bitmap.CountRange()` to return the number
of bits in a subrange of a bitmap. This significantly improves
server start time and is needed for upcoming zero copy bitmap
optimizations.
 2016-08-15 16:03:25 -06:00
Todd Gruben
ee01ea624f Merge branch 'master' of github.com:umbel/pilosa 2016-08-12 15:10:57 -05:00
Todd Gruben
f5f3a58e96 cache loading 2016-08-12 15:10:50 -05:00
tgruben
5d48436e97 Merge pull request #103 from benbjohnson/fix-count-cache 
Fix count() cache retrieval
 2016-08-12 15:06:53 -05:00
Ben Johnson
6ad7d0bfab fix count() cache retrieval 2016-08-12 11:02:52 -06:00
Ben Johnson
96680a262c add http trace profiling 2016-08-12 10:58:45 -06:00
Ben Johnson
4883dea5f8 add CLI profiling 
This commit adds the `-cpuprofile` flag to the `pilosa` binary so
that startup can be profiled.
 2016-08-12 10:58:45 -06:00
Ben Johnson
3150471c3c fix bitmap reference bug 2016-08-09 14:57:37 -06:00
Todd Gruben
d79a5e7b0b Merge branch 'master' of github.com:umbel/pilosa 2016-08-08 09:35:43 -05:00
tgruben
301c3ec184 Merge pull request #102 from benbjohnson/serialize-cache-flush 
Serialize cache flush
 2016-08-08 09:30:47 -05:00
Ben Johnson
a063421467 serialize cache flush 
This commit moves the cache flush to the `Index` and only serializes
a single fragment at a time.

Also included in this commit is the `inspect` command for the
`pilsoactl` binary. This provides insight into pilosa data files.
 2016-08-05 14:58:37 -06:00
Todd Gruben
322da388f8 Merge branch 'master' of github.com:umbel/pilosa 2016-07-25 11:43:58 -05:00
tgruben
9732b55caa Merge pull request #100 from benbjohnson/cluster-attr 
Forward bitmap & profile attributes
 2016-07-25 11:42:52 -05:00
Ben Johnson
c54047041d forward bitmap & profile attributes 
This commit adds forwarding of bitmap and profile attributes to
other servers in the cluster.
 2016-07-19 13:54:01 -06:00
Todd Gruben
8762cf3425 Merge branch 'master' of github.com:umbel/pilosa 2016-07-07 11:23:50 -05:00
Todd Gruben
3e288d6108 ignore checksum on single node clusters 2016-07-07 11:22:15 -05:00
tgruben
1adf899ea6 Merge pull request #99 from benbjohnson/optimize-block-checksumming 
Optimize sparse bitmap block checksums
 2016-07-07 11:18:58 -05:00
Ben Johnson
1ed2d54855 optimize sparse bitmap block checksums 
This commit refactors the block checksumming by removing the
iteration over each block and instead only checking blocks which
have data. This requires merging the cache inspection with the
roaring iterator to reduce CPU time and memory allocations.
 2016-07-07 10:03:38 -06:00
tgruben
9c38f199c6 Merge pull request #95 from benbjohnson/sync-closing 
Check for early close during AAE
 2016-06-29 11:25:56 -05:00
Ben Johnson
521b098c83 check for early close during AAE 
This commit adds a `Closing` channel to the `IndexSyncer` and the
`FragmentSyncer` that is periodically checked during execution.
If the channel is closed then an in-progress sync is immediately
stopped.
 2016-06-29 10:18:24 -06:00
Todd Gruben
50ee217b37 Merge branch 'master' of github.com:umbel/pilosa 2016-06-29 10:28:19 -05:00
tgruben
8ee0dcd95e Merge pull request #94 from benbjohnson/optimize-union-difference 
Optimize union() & difference()
 2016-06-28 15:34:38 -05:00
Ben Johnson
36cd2c84c1 optimize union() & difference() 
This commit moves the `Union()` and `Difference()` operations into
the `roaring` package to improve performance.
 2016-06-28 14:29:37 -06:00
Todd Gruben
46351c2f10 Merge branch 'master' of github.com:umbel/pilosa 2016-06-23 09:05:21 -05:00
tgruben
0a6147951c Merge pull request #93 from benbjohnson/set-bulk-attrs 
Optimize bulk bitmap attribute update
 2016-06-22 21:19:23 -05:00
Ben Johnson
1126e7b5f7 optimize bulk bitmap attribute update 
This commit adds a fast path for optimizing queries that only
have `SetBitmapAttrs()` calls in them. It does this by grouping
the calls into a single Bolt transaction and also by merging
multiple attribute updates per ID into one update.
 2016-06-22 20:04:49 -06:00
Todd Gruben
d76ff0964a Merge branch 'master' of github.com:umbel/pilosa 2016-06-21 08:45:10 -05:00
tgruben
2022e880b7 Merge pull request #91 from benbjohnson/bit-split 
roaring: change bit split to 40/24
 2016-06-21 08:30:36 -05:00
Ben Johnson
a8feb985e1 roaring: change bit split to 40/24 
This commit changes the `roaring.Bitmap` to use a 40/24 split in
the high and low bits. This change also requires the low bits
to use `uint32` instead of `uint16`.
 2016-06-20 20:37:05 -06:00
Todd Gruben
2c30a1dbf4 Merge branch 'master' of github.com:umbel/pilosa 2016-06-15 08:56:47 -05:00
tgruben
c93d94d4e2 Merge pull request #90 from benbjohnson/optimize-intersection 
Optimize intersect()
 2016-06-14 16:57:52 -05:00
Ben Johnson
89286ac23a optimize intersect() 
This commit moves bitmap intersection into the `roaring` package
in order to reduce allocations.

Benchmarks against real world datasets show a speed improvement of
353%. Previously, intersection operated at approximately 5.6M
bits/sec against two 60K bit bitmaps with 30% overlap. The new
intersection operates at approximately 20M bits/sec.
 2016-06-14 15:19:21 -06:00
Todd Gruben
a68975bb6f Merge branch 'master' of github.com:umbel/pilosa 2016-06-10 10:21:27 -05:00
Todd Gruben
7d8705868e tuning slicewidth 2016-06-10 10:21:23 -05:00
Todd Gruben
a2187b968c added info to owns fragrment error message 2016-06-10 10:19:54 -05:00
tgruben
5b40ebc036 Merge pull request #89 from benbjohnson/buffered-import 
Add buffered snapshot writer
 2016-06-10 10:00:43 -05:00
Ben Johnson
677824c065 add buffered snapshot writer 
This commit wraps the snapshot in a `bufio.Writer`.
 2016-06-10 08:58:37 -06:00
tgruben
c9f1e26143 Merge pull request #88 from benbjohnson/optimize-import 
Optimize Import
 2016-06-09 16:47:27 -05:00
Ben Johnson
64decdaf81 sort bits on import 
This commit sorts bits from import files before performing the input.
 2016-06-09 14:25:41 -06:00
tgruben
32697531fd Merge pull request #87 from benbjohnson/fix-client-fragment-nodes 
Fix Client.FragmentNodes()
 2016-06-08 11:27:31 -05:00
Ben Johnson
45a62c30bd fix Client.FragmentNodes() 
This commit fixes the hardcoded database field in
`Client.FragmentNodes()`.
 2016-06-08 10:23:44 -06:00
Todd Gruben
afb40fe8ef fixed typo 2016-06-07 15:29:28 -05:00
Todd Gruben
b40b2ff5ca Merge branch 'master' of github.com:umbel/pilosa 2016-06-07 15:25:42 -05:00