jaffee
a03ecf92e9
fix rebase issues
2016-12-19 15:12:56 -06:00
jaffee
293621516e
have bagent add metadata to results
...
need this to differentiate multiple agents' output when it's all going to stdout.
2016-12-19 15:10:35 -06:00
jaffee
d196c39bce
basic implementation of remote agent spawning
2016-12-19 15:09:16 -06:00
Matthew Jaffee
e211dbd54a
Merge pull request #199 from alanbernstein/benchmarks-readable
...
Use Duration's string representation in JSON output
2016-12-19 14:50:53 -06:00
Alan Bernstein
8690157d3f
Add -human flag to bagent
2016-12-19 14:49:09 -06:00
Matthew Jaffee
58f697d814
Merge pull request #200 from alanbernstein/benchmarks-zipf
...
Benchmarks zipf
2016-12-19 14:21:05 -06:00
jaffee
0c6e6e4f44
creator can set GOMAXPROCS for remote pilosa nodes
2016-12-14 15:05:50 -06:00
jaffee
9d5acdeecd
cleanup cluster shutdown and extra logging
2016-12-14 10:53:14 -06:00
jaffee
968e82d1ad
clean up remote create, kill remote cluster
...
Had to jump through some hoops to kill the remote cluster reliably. Not all ssh
server implementations respect signals, so in order to kill the cluster, we
request a pty for the ssh session. This allows us to send the byte 0x03 which is
effectively the same has hitting Ctrl-c in an interactive ssh session. With a
pty, just closing the session also seems to kill the remote process.
2016-12-13 16:36:52 -06:00
Alan Bernstein
2968c15a37
Improve comment formatting and update usage
2016-12-13 13:25:31 -06:00
jaffee
6829f7cdd0
create - start cluster over ssh when given hosts
2016-12-13 10:14:09 -06:00
jaffee
e72e82ca7f
remove separate create() and get bspawn working again
2016-12-12 16:29:40 -06:00
Alan Bernstein
587d4259a1
Add good defaults for zipf config
2016-12-12 16:11:23 -06:00
jaffee
bd78a7c1b9
json output and local cluster log handling
2016-12-12 11:32:56 -06:00
Alan Bernstein
4de72e27dd
Add zipf benchmark with efficient random ID permutations
2016-12-08 19:14:24 -06:00
Alan Bernstein
7dfb53d08e
Use Duration's string representation in JSON output
2016-12-08 15:06:02 -06:00
jaffee
53171da640
thread context through Benchmarks
...
Also fix a few ignored errors, an issue with pilosactl usage string, a broken
test, and a mistaken import.
2016-12-08 11:02:14 -06:00
jaffee
4c67cb30c0
make results nicer
2016-12-07 12:09:18 -06:00
jaffee
b4bb9cf70d
add slice height benchmark
2016-12-06 17:20:38 -06:00
jaffee
af1d35affc
import benchmark sort by profile id after bitmap id
2016-12-06 16:11:00 -06:00
jaffee
838941b26a
wire up import benchmark, fix bug with ImportCommand.Host
...
also add some config metadata to results in order to distinguish them
2016-12-06 10:16:28 -06:00
jaffee
e5edf3e432
add import benchmark
2016-12-05 17:37:33 -06:00
jaffee
1d0a314287
rename imports umbel->pilosa
2016-11-28 15:45:18 -07:00
jaffee
86e226d574
better documentation for bagent and bspawn
2016-11-28 15:45:18 -07:00
jaffee
a3b69f9066
rename bench commands and args
...
lower case and dash separated
2016-11-28 15:45:18 -07:00
jaffee
aa2b4949d8
add random query benchmark
2016-11-28 15:45:18 -07:00
jaffee
df6e519222
bspawn can create and tear down cluster for benchmarks
2016-11-28 15:45:18 -07:00
jaffee
65ac486b4c
refactor Init and provide ClientType option
...
Benchmark implementations can now use 'HasClient' as a component which provides
an Init method which sets up the pilosa client in a configurable way.
Implementations must still expose the ClientType argument through ConsumeFlags
and describe it in Usage()
2016-11-28 15:45:17 -07:00
jaffee
2f4171bba8
add agentNum to benchmark Init as well as run
2016-11-28 15:45:17 -07:00
jaffee
256d516726
fix broken context stuff after rebase
2016-11-28 15:45:17 -07:00
jaffee
86d408310c
add MultiDBSetBits benchmark
2016-11-28 15:45:17 -07:00
jaffee
6d2cc4335f
forgot to check in randspawn benchmark file
2016-11-28 15:45:17 -07:00
jaffee
cb85882289
add RandomSetBits benchmark demonstrating seeds
2016-11-28 15:45:17 -07:00
jaffee
776da440f6
add bspawn command and json config file
2016-11-28 15:45:17 -07:00
jaffee
3bd5daa5f3
redirect pilosactl create -local output to file
2016-11-28 15:45:17 -07:00
jaffee
e6b31db90d
simplify and rename SetBits Benchmark
2016-11-28 15:45:17 -07:00
jaffee
6c42e8b7e7
renaming stuff to be more idiomatic
2016-11-28 15:45:17 -07:00
jaffee
7e524ed836
setup and support BenchmarkCmd interface
2016-11-28 15:45:17 -07:00
jaffee
5488c36527
move pilosa-benchmark-agent under pilosactl
2016-11-28 15:45:17 -07:00
jaffee
b0bf131eeb
fix usage string for benchmark agent
2016-11-28 15:45:17 -07:00
jaffee
983379928c
add cluster creation to pilosactl
2016-11-28 15:45:17 -07:00
jaffee
5d16dba5ec
add benchmark agent
2016-11-28 15:45:17 -07:00
Cody Soyland
a2d9a9a8d3
Migrate from Umbel to Pilosa organization on Github
2016-11-28 15:21:11 -06:00
Ben Johnson
7c1a39a02d
Add context to pilosactl.
2016-11-16 19:53:40 -07:00
Ben Johnson
748484040d
Add pilosactl check.
2016-10-26 13:51:23 -06:00
Ben Johnson
b00901d265
Add ExpvarStatsClient and basic tracking.
...
A `StatsClient` for `expvar` is added so we can track Stats through
the `/debug/vars` endpoint. Tags are nested inside maps so that we
can see stats for db, frame & slice.
Also added a `MultiStatsClient` for chaining multiple `StatsClient`
implementations together (e.g. `expvar` and DataDog).
2016-10-20 14:35:14 -06:00
Ben Johnson
d90a9b97c8
Add profile & bitmap attribute anti-entropy.
...
Adds block-based anti-entropy for the attribute stores and hooks
into the existing `IndexSyncer` to regulate frequency of syncs.
2016-09-30 11:47:43 -06:00
Todd Gruben
57810f8ab6
merge
2016-09-22 03:05:53 -05:00
Ben Johnson
432bf74368
Add import sorting command.
...
Introduces new `pilosactl sort` to sort import files by bit
position so they can be inserted faster. Also optimizes container
scanning and adds a `-buffer-size` flag to `import`.
2016-09-21 14:33:24 -06:00
Ben Johnson
947eea4668
Optimize TopN() w/ source query.
...
This commit makes several changes to optimize the TopN() query:
- Reduce highbits() back from 24-bits to 16-bits.
- Reduce MaxArraySize back from 2^20 to 4096.
- Optimize bitmap count invalidation.
- Parallelize TopN() across nodes.
- Parallelize TopN() across slices.
2016-09-13 14:55:56 -06:00
