scriptoriumadmin/featurebase
1
0
Fork 0
mirror of https://github.com/featurebasedb/featurebase.git synced 2026-08-28 10:54:59 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions
685 commits 45 branches 174 tags 85 MiB
Commit graph

685 commits

This branch
This branch
All branches
Author SHA1 Message Date
Ben Johnson
c6ca86f2d9 Add ExpvarStatsClient and basic tracking. 
A `StatsClient` for `expvar` is added so we can track Stats through
the `/debug/vars` endpoint. Tags are nested inside maps so that we
can see stats for db, frame & slice.

Also added a `MultiStatsClient` for chaining multiple `StatsClient`
implementations together (e.g. `expvar` and DataDog).
 2016-10-20 14:35:14 -06:00
tgruben
53dc73a1ad Merge pull request #126 from umbel/clearbit_bug 
added remote termination clause for clearbit
 2016-10-18 15:54:42 -05:00
tgruben
084f08b7b8 Merge pull request #124 from benbjohnson/123-clear-bit-fix 
Fix ClearBit() file handle issue.
 2016-10-18 15:12:03 -05:00
Todd Gruben
fb7bd66bc2 added remote termination clause for clearbit 2016-10-18 15:10:02 -05:00
tgruben
c9c1ab0310 Merge pull request #125 from benbjohnson/123-fix-clear-bit-return 
Handle return data from remote ClearBit()
 2016-10-18 15:05:40 -05:00
Ben Johnson
aaf88563e3 Handle return data from remote ClearBit() 
Previously `ClearBit()` did not handle returned data correctly
and panicked. This change adds it to the list of available return
types.
 2016-10-18 11:54:00 -06:00
Ben Johnson
3edaa2e81d Fix ClearBit() file handle issue. 
Previously the index would create a fragment if it didn't exist
when clearing a bit. This causes additional file handles to be
created when it's already known that the bit is not set.
 2016-10-18 10:27:38 -06:00
tgruben
b8b5cb0337 Merge pull request #122 from benbjohnson/120-fix-topn 
Add slice to BitmapSegment operations.
 2016-10-13 13:04:27 -05:00
Ben Johnson
2322201bf4 Add slice to BitmapSegment operations. 
Previously the `slice` was not copied to each new segment when
`Intersect()`, `Union()`, and `Difference()` operations were performed.
This causes issues when those bitmaps were then operated on later such
as performing a `TopN()` on a source bitmap.
 2016-10-13 10:54:40 -06:00
tgruben
517097ddc3 Merge pull request #121 from benbjohnson/statsd 
Add StatsClient with DataDog implementation.
 2016-10-07 16:03:42 -05:00
Ben Johnson
8c878780e9 Add StatsClient with DataDog implementation. 2016-10-06 15:11:33 -06:00
tgruben
2e81850c03 Merge pull request #119 from benbjohnson/rmdb 
Add endpoint for database deletion.
 2016-10-04 13:03:03 -05:00
Ben Johnson
56aea09ce1 Add endpoint for database deletion. 
Executing a request against `DELETE /db` will now delete a database.

	curl -X DELETE "http://localhost:15000/db" -d '{"db":"d"}'
 2016-10-04 09:40:59 -06:00
tgruben
138355c155 Merge pull request #118 from benbjohnson/attr-sync 
Add profile & bitmap attribute anti-entropy
 2016-09-30 13:18:23 -05:00
Ben Johnson
d5ad5fbb05 Add profile & bitmap attribute anti-entropy. 
Adds block-based anti-entropy for the attribute stores and hooks
into the existing `IndexSyncer` to regulate frequency of syncs.
 2016-09-30 11:47:43 -06:00
tgruben
c92f37ac3c Merge pull request #117 from umbel/bug 
Bug
 2016-09-27 17:20:59 -05:00
Todd Gruben
f2f7de4fd7 bug in difference 2016-09-27 16:07:48 -05:00
Todd Gruben
72db5500ca bug in Union 2016-09-27 16:04:20 -05:00
Todd Gruben
d699d41ea7 added docs for TopN attributes 2016-09-27 14:07:29 -05:00
tgruben
bd84fa3404 Merge pull request #116 from umbel/difference_bug 
Difference bug
 2016-09-23 16:05:23 -05:00
Todd Gruben
ad852fdbf1 cleaned up constant 2016-09-23 16:04:22 -05:00
Todd Gruben
5f65636199 fix roaring.Difference crash 2016-09-23 13:27:35 -05:00
Todd Gruben
6a8973f7c9 fix for intersect and updated tests 2016-09-22 16:47:39 -05:00
Todd Gruben
0a9dcee2f1 merge 2016-09-22 03:05:53 -05:00
Todd Gruben
f006270971 merge 2016-09-22 03:05:26 -05:00
tgruben
286762e804 Merge pull request #113 from benbjohnson/sort-import 
Add import sorting command
 2016-09-21 16:30:05 -05:00
Ben Johnson
46bef724a5 Add import sorting command. 
Introduces new `pilosactl sort` to sort import files by bit
position so they can be inserted faster. Also optimizes container
scanning and adds a `-buffer-size` flag to `import`.
 2016-09-21 14:33:24 -06:00
tgruben
c1fc750d21 Merge pull request #112 from benbjohnson/import-cache-2 
Update cache on Import (alternative)
 2016-09-16 15:54:13 -05:00
Ben Johnson
b40d4385e8 Remove import count optimization. 2016-09-16 11:36:54 -06:00
Todd Gruben
ec4b1352d1 cache update on import adjustment 2016-09-16 11:20:31 -06:00
Todd Gruben
ced399c6fc corrected ClearBit 2016-09-16 11:20:12 -06:00
Todd Gruben
4035763ef4 ignore checksum on single node clusters 2016-09-16 11:18:44 -06:00
Todd Gruben
8913216a6e cache update on import adjustment 2016-09-15 13:37:10 -05:00
Todd Gruben
46b3161c47 Merge branch 'master' of github.com:umbel/pilosa 2016-09-13 20:59:23 -05:00
tgruben
a89af88d47 Merge pull request #110 from benbjohnson/optimize-top-n 
Optimize TopN() w/ source query.
 2016-09-13 20:58:53 -05:00
Ben Johnson
cf4297a771 Optimize TopN() w/ source query. 
This commit makes several changes to optimize the TopN() query:

- Reduce highbits() back from 24-bits to 16-bits.
- Reduce MaxArraySize back from 2^20 to 4096.
- Optimize bitmap count invalidation.
- Parallelize TopN() across nodes.
- Parallelize TopN() across slices.
 2016-09-13 14:55:56 -06:00
Todd Gruben
244077aef7 Merge branch 'master' of github.com:umbel/pilosa 2016-09-07 14:15:10 -05:00
tgruben
9f345aec93 Merge pull request #109 from benbjohnson/opt-int-count 
Optimize IntersectionCount for Array+Bitmap
 2016-09-07 14:14:41 -05:00
Ben Johnson
39ed7de5c4 Optimize IntersectionCount for Array+Bitmap 
Reworks the `roaring.intersectionCountArrayBitmap()` call to avoid
using an iterator. Performance of the included benchmark went from
2.5ms to 1.1ms.

Some of the issue with intersectionCount is the increased size of
bitmaps and slices and I need to do additional testing with various
sizes.
 2016-09-07 10:13:31 -06:00
Todd Gruben
51560213f9 Merge branch 'master' of github.com:umbel/pilosa 2016-09-01 12:07:27 -05:00
tgruben
cc330f6c00 Merge pull request #108 from benbjohnson/remove-mmap-resizing 
Remove post-mmap resizing
 2016-09-01 12:06:38 -05:00
Ben Johnson
089650b845 Remove post-mmap resizing 2016-09-01 09:18:52 -06:00
Todd Gruben
c12d8e096b Merge branch 'master' of github.com:umbel/pilosa 2016-08-30 17:36:21 -05:00
Todd Gruben
f70a91c614 corrected ClearBit 2016-08-30 17:36:00 -05:00
tgruben
fdf4a35f6d Merge pull request #107 from benbjohnson/export 
Add export command to pilosactl
 2016-08-30 17:25:30 -05:00
Ben Johnson
39e0d22598 Add export command to pilosactl 
CSV exports can now be done with the pilosactl application:

	$ pilosactl -d mydb -f myframe -o MYFILE.csv

If `-o` is not specified then the CSV is written to STDOUT. The
exporter combines all slices for the db/frame to into a single
concatenated CSV file.
 2016-08-30 16:18:12 -06:00
Todd Gruben
32fec201c0 Merge branch 'master' of github.com:umbel/pilosa 2016-08-26 12:59:55 -05:00
tgruben
84bbf331a6 Merge pull request #105 from benbjohnson/optimize-bitmap 
Optimize bitmap with zero copy (WIP)
 2016-08-25 15:33:00 -05:00
Ben Johnson
04c814141a Optimize bitmap with zero copy 
Segments bitmaps by slice and changes `Fragment.Bitmap()` to return
bitmaps with segments which are offset ranges of mmapped bitmaps.
 2016-08-25 13:30:25 -06:00
Todd Gruben
838a7ee0f6 Merge branch 'master' of github.com:umbel/pilosa 2016-08-15 18:36:08 -05:00