Matthew Jaffee
361a7b6d03
Merge pull request #199 from alanbernstein/benchmarks-readable
...
Use Duration's string representation in JSON output
2016-12-19 14:50:53 -06:00
Alan Bernstein
fad17ff9a1
Add -human flag to bagent
2016-12-19 14:49:09 -06:00
Matthew Jaffee
ffe9a89402
Merge pull request #200 from alanbernstein/benchmarks-zipf
...
Benchmarks zipf
2016-12-19 14:21:05 -06:00
jaffee
d649a48c21
creator can set GOMAXPROCS for remote pilosa nodes
2016-12-14 15:05:50 -06:00
jaffee
7b55adc39b
cleanup cluster shutdown and extra logging
2016-12-14 10:53:14 -06:00
jaffee
37d11b858e
clean up remote create, kill remote cluster
...
Had to jump through some hoops to kill the remote cluster reliably. Not all ssh
server implementations respect signals, so in order to kill the cluster, we
request a pty for the ssh session. This allows us to send the byte 0x03 which is
effectively the same has hitting Ctrl-c in an interactive ssh session. With a
pty, just closing the session also seems to kill the remote process.
2016-12-13 16:36:52 -06:00
Alan Bernstein
2becc6619d
Improve comment formatting and update usage
2016-12-13 13:25:31 -06:00
jaffee
bbb88e910f
create - start cluster over ssh when given hosts
2016-12-13 10:14:09 -06:00
jaffee
b97fb2b971
remove separate create() and get bspawn working again
2016-12-12 16:29:40 -06:00
Alan Bernstein
5c78164837
Add good defaults for zipf config
2016-12-12 16:11:23 -06:00
jaffee
e78fa1bc74
json output and local cluster log handling
2016-12-12 11:32:56 -06:00
Alan Bernstein
d58d1406f5
Add zipf benchmark with efficient random ID permutations
2016-12-08 19:14:24 -06:00
Alan Bernstein
76786e7ba0
Use Duration's string representation in JSON output
2016-12-08 15:06:02 -06:00
jaffee
33cbdba642
thread context through Benchmarks
...
Also fix a few ignored errors, an issue with pilosactl usage string, a broken
test, and a mistaken import.
2016-12-08 11:02:14 -06:00
jaffee
f9362c767d
make results nicer
2016-12-07 12:09:18 -06:00
jaffee
64b86f29cd
add slice height benchmark
2016-12-06 17:20:38 -06:00
jaffee
e5b54e7f4a
import benchmark sort by profile id after bitmap id
2016-12-06 16:11:00 -06:00
jaffee
aed735b85b
wire up import benchmark, fix bug with ImportCommand.Host
...
also add some config metadata to results in order to distinguish them
2016-12-06 10:16:28 -06:00
jaffee
4f9141b792
add import benchmark
2016-12-05 17:37:33 -06:00
Travis
dd6f41f551
standardizes on MaxSlice (instead of SliceN)
...
creates DB locally even if node doesn't have data for that DB
2016-11-30 12:59:17 -06:00
Travis
78666d6bba
Merge branch 'master' into db-specific-slices
2016-11-29 11:59:41 -06:00
jaffee
0588e0398c
rename imports umbel->pilosa
2016-11-28 15:45:18 -07:00
jaffee
e1e3222203
better documentation for bagent and bspawn
2016-11-28 15:45:18 -07:00
jaffee
8b23da6212
rename bench commands and args
...
lower case and dash separated
2016-11-28 15:45:18 -07:00
jaffee
c8b0fecd4a
add random query benchmark
2016-11-28 15:45:18 -07:00
jaffee
069484ab0e
bspawn can create and tear down cluster for benchmarks
2016-11-28 15:45:18 -07:00
jaffee
05a70d5fb2
refactor Init and provide ClientType option
...
Benchmark implementations can now use 'HasClient' as a component which provides
an Init method which sets up the pilosa client in a configurable way.
Implementations must still expose the ClientType argument through ConsumeFlags
and describe it in Usage()
2016-11-28 15:45:17 -07:00
jaffee
da024fb0d8
add agentNum to benchmark Init as well as run
2016-11-28 15:45:17 -07:00
jaffee
488ed1259c
fix broken context stuff after rebase
2016-11-28 15:45:17 -07:00
jaffee
85b4e83c2d
add MultiDBSetBits benchmark
2016-11-28 15:45:17 -07:00
jaffee
ce34faa9c9
forgot to check in randspawn benchmark file
2016-11-28 15:45:17 -07:00
jaffee
9cc3c76297
add RandomSetBits benchmark demonstrating seeds
2016-11-28 15:45:17 -07:00
jaffee
0d2a478153
add bspawn command and json config file
2016-11-28 15:45:17 -07:00
jaffee
72e8dba966
redirect pilosactl create -local output to file
2016-11-28 15:45:17 -07:00
jaffee
8c7e02b5ee
simplify and rename SetBits Benchmark
2016-11-28 15:45:17 -07:00
jaffee
7db4b55b49
renaming stuff to be more idiomatic
2016-11-28 15:45:17 -07:00
jaffee
92fcec82b3
setup and support BenchmarkCmd interface
2016-11-28 15:45:17 -07:00
jaffee
5ac7cab57e
move pilosa-benchmark-agent under pilosactl
2016-11-28 15:45:17 -07:00
jaffee
9dc50268c9
fix usage string for benchmark agent
2016-11-28 15:45:17 -07:00
jaffee
0df040c9df
add cluster creation to pilosactl
2016-11-28 15:45:17 -07:00
jaffee
423fdb3841
add benchmark agent
2016-11-28 15:45:17 -07:00
Cody Soyland
5206461d51
Migrate from Umbel to Pilosa organization on Github
2016-11-28 15:21:11 -06:00
Travis
b6f0a62fdd
changes MaxSlices logic to be DB-specific
2016-11-21 16:12:30 -06:00
Ben Johnson
6e304e8e7d
Add context to pilosactl.
2016-11-16 19:53:40 -07:00
Ben Johnson
3192897f55
Add pilosactl check.
2016-10-26 13:51:23 -06:00
Ben Johnson
c6ca86f2d9
Add ExpvarStatsClient and basic tracking.
...
A `StatsClient` for `expvar` is added so we can track Stats through
the `/debug/vars` endpoint. Tags are nested inside maps so that we
can see stats for db, frame & slice.
Also added a `MultiStatsClient` for chaining multiple `StatsClient`
implementations together (e.g. `expvar` and DataDog).
2016-10-20 14:35:14 -06:00
Ben Johnson
d5ad5fbb05
Add profile & bitmap attribute anti-entropy.
...
Adds block-based anti-entropy for the attribute stores and hooks
into the existing `IndexSyncer` to regulate frequency of syncs.
2016-09-30 11:47:43 -06:00
Todd Gruben
0a9dcee2f1
merge
2016-09-22 03:05:53 -05:00
Ben Johnson
46bef724a5
Add import sorting command.
...
Introduces new `pilosactl sort` to sort import files by bit
position so they can be inserted faster. Also optimizes container
scanning and adds a `-buffer-size` flag to `import`.
2016-09-21 14:33:24 -06:00
Ben Johnson
cf4297a771
Optimize TopN() w/ source query.
...
This commit makes several changes to optimize the TopN() query:
- Reduce highbits() back from 24-bits to 16-bits.
- Reduce MaxArraySize back from 2^20 to 4096.
- Optimize bitmap count invalidation.
- Parallelize TopN() across nodes.
- Parallelize TopN() across slices.
2016-09-13 14:55:56 -06:00
