Commit graph

262 commits

Author SHA1 Message Date
Nia Weiss
f4ba34247f
remove attributes
Attributes are unmaintained and unused.
They have become more of a liability than a benefit.
This change eliminates them from the codebase.
The only user-visible change (assuming that attrs are not used) is that the attrs field will no longer appear in row JSON.
2021-05-14 10:28:08 -04:00
Ben Johnson
7e804d7d08 Fix id alloc backup invocation 2021-05-07 10:59:01 -06:00
Ben Johnson
7284c4dd10 Add id alloc, col attrs, & row attrs backup 2021-05-07 10:59:01 -06:00
Ben Johnson
776b43a3cd Backup CLI 2021-05-07 10:59:01 -06:00
Seebs
e93d2fe06c bitmap unmarshalling and testing bug fixes
When unmarshalling ops, we weren't adding a meaningful OpN to them,
resulting in misleading reports from `pilosa inspect`. Also, we were
mistakenly reporting things as "mapped" when they were actually
using their internal storage (as with small array containers).

Add the "sanity check" to `pilosa inspect` so that errors like the
above get noticed more easily and corrected. Also, to make that work,
have roaring.InspectBinary actually put containers in the bitmap
it creates rather than just creating info entries for them.
2021-04-20 12:01:21 -05:00
Seebs
fd8a19278c add TestMain wrapper in ctl
The TestMain wrapper gets us the fancy testhook stuff tracking whether we're
deallocating things as expected, and we probably want that.
2021-04-13 12:37:07 -05:00
Alan Bernstein
285d0a0af8 Add log prefix levels 2021-04-12 20:33:39 -05:00
Nia
d5da6e229c
Merge branch 'master' into external-lookup 2021-03-31 08:12:18 -04:00
Nia Weiss
9bc1b23b7e
change "External" DB to "Lookup" DB 2021-03-29 12:54:29 -04:00
Nia Weiss
c4aad290ed
address ExternalLookup review comments 2021-03-29 08:22:41 -04:00
Antonio Navarro Perez
7ec85a1c0f Add at least 3 nodes on test clusters.
Signed-off-by: Antonio Navarro Perez <antnavper@gmail.com>
2021-03-26 10:16:41 +01:00
Nia Weiss
e9b92e1cd4
add ExternalLookup query 2021-03-25 13:21:28 -04:00
Kuba Podgórski
40337cff09
Merge branch 'master' into core-230/go-pilosa 2021-03-24 21:23:04 +01:00
Todd Gruben
6bafa989e0 adjust etcd config comments 2021-03-24 10:43:28 -05:00
Todd Gruben
8c4d7e09ff re-use pilosa tls settings 2021-03-22 16:48:09 -05:00
Kuba Podgórski
1e5388dbe9 Move internals proto files into separate package (pb) 2021-03-22 20:28:00 +01:00
Todd Gruben
f52b88a962 etcd tls configuration support 2021-03-22 09:10:32 -05:00
Seebs
1045268f01 use testhook to ensure temporary files and directories are cleaned up
In nearly all cases, we can just switch ioutil.TempDir->testhook.TempDir
and similarly for TempFile. There's one case where we can't because we
need files to be removed before tests are over.

Also in the process give identifiable names to a lot of temporary files
and make sure they're being cleaned up, and don't use "/tmp/foo" as a
file name in a test that could be running in more than one test process
at once. :)
2021-03-11 19:42:10 -06:00
Travis
42d69d6c5d
Remove gossip config 2021-03-04 13:51:18 -06:00
Travis
2bbe1fdde0
remove remaining references to "coordinator" 2021-02-23 17:23:09 -06:00
Antonio Navarro Perez
c45e21640c
Change coordinator to primary
Signed-off-by: Antonio Navarro Perez <antnavper@gmail.com>
2021-02-02 15:56:12 -06:00
Travis
457194f6a8
update config to support etcd arguments 2021-01-29 19:43:48 -06:00
Travis
e459c9a77b
change Config.DisCo to Config.Etcd 2021-01-29 14:11:51 -06:00
Travis
13984353e4
remove instances of os.Getenv("PILOSA_TXSRC") 2021-01-21 21:18:46 -06:00
Travis
f292d6061a
replace pilosa.DefaultTxsrc with storage.DefaultBackend 2021-01-20 22:06:13 -06:00
Travis
08fae2be4c
introduce storage.Config 2021-01-20 22:05:38 -06:00
Travis
f47800a920
add DisCo config to ctl/server 2021-01-18 23:47:35 -06:00
Maxton Huff
dee700741a deprecate cluster.long-query-time and create long-query-time
moved lonquerytime from cluster into server and moved cluster.longquerytime into top level config
kept cluster.longquerytime for backwards compatibility, favored if both longquerytime options are present
2020-12-10 10:51:31 -06:00
Cody Soyland
bc94c7bcdf Remove lmdb dependency and references, vendor Barrier 2020-12-09 08:43:23 -06:00
Jason E. Aten
03a54c6d5f rowcache off by default. pilosa server --rowcache-on turns it back on. 2020-12-08 23:18:45 +00:00
Alan Bernstein
2bc2a32263 Make query history length configurable 2020-11-23 20:23:27 -06:00
Jason E. Aten
524c78623f repair pilosa/ctl/server_test too 2020-11-18 18:53:22 +00:00
Jason E. Aten
06eb3afaaa use pilosa server --txsrc to prevent viper env var shadowing from stopping PILOSA_TXSRC getting through 2020-11-18 18:21:31 +00:00
Jason E. Aten
744827d5cc restore the env variable PILOSA_TXSRC's affect 2020-11-16 23:18:50 +00:00
Jason E. Aten
d46b603cd6 rbf: keep ElemN and BitN up to date.
- fix a bug in computing leafCell.BitN in a run after a bit Remove
- shrink bitmaps on remove
- util_test.go has Cursor.DebugSlowCheckAllPages to verify;
  used by cursor_test.go
2020-11-13 21:18:09 +00:00
Jason E. Aten
85f3fa9836 merge with the update to rbf to take a Config parameter 2020-10-27 15:11:45 +00:00
jaten-molecula
4543237e09
Merge branch 'master' into rbf-cli 2020-10-27 10:09:24 -05:00
Ben Johnson
2b1d790625 Fix tree key formatting; add --with-tree option 2020-10-27 08:24:56 -06:00
Ben Johnson
7fa1a5edd2 Add RBF CLI commands 2020-10-27 08:12:22 -06:00
Jason E. Aten
957cba1768 performance tuning: rbfcfg package, binary search for wal segment
- rbfcfg package holds Config for --rbf- command line flags
- wal.go: replace linear search with bisection for wal segment
2020-10-27 00:49:56 +00:00
Jason E. Aten
8d2ad048fa pilosa server --norowcache disables the row cache
- this can lessen memory pressure
- certain backends may not need it
- enables performance benchmarking and tuning
2020-10-22 22:20:48 +00:00
Alan Bernstein
5705b45864 Add a CLI flag for cluster name and include it in /status response 2020-10-01 10:43:49 -05:00
Ben Johnson
150c8a5b06 database per shard, HolderConfig, rbf bit-wise import speedups.
- introduce Query Context (Qcx) for managing database-per-shard.
- replaces the MultiTx, so mtx.go is retired and removed.
- introduces the HolderConfig struct and all Holders now have
  a path from birth.
- rbf speedups on bitwise writes
- badgerdb is removed due to unresolvable write conflicts.

fixes #703 #676
2020-09-04 13:00:33 -05:00
Alan Bernstein
4ce3879e2c Revert UI->Lattice name change, correct the ordering of error checks in statikHandler 2020-09-01 10:16:31 -05:00
Alan Bernstein
d912403eb2 Add missing file 2020-08-31 09:02:46 -05:00
Nia Weiss
f0f8336f57
rename postgres.addr to postgres.bind for consistency 2020-08-27 11:35:54 -04:00
Nia Weiss
288593231f
require TLS when configured 2020-08-27 10:59:40 -04:00
Nia Weiss
47dd6b5b8f
fix postgres endpoint config 2020-08-27 10:49:43 -04:00
Seebs
cecaf99ee4 testhook: leak auditing infrastructure
The testhook/ package provides an easy way to set up multiple
hooks to run before/after tests are run.

The audit hooks track open and closes of storage backends,
files, indexes, and holders, for example. A tempdir wrapper
creates temporary directories which are automatically cleaned up
when the test ends. Any kind of resource creation that
should be closed at test conclusion can be tracked. We
will complain at the end of the TestMain if resources are
leaking.

Leaks under go1.13:

We use a wrapper function which is a no-op for go 1.13, but actually
calls testing.TB.Cleanup in go1.14, so we can still build with 1.13 even though
tests will leak files all over the place there. Because of this,
don't run the testhook tests when using 1.13, as they'll always fail.

- the test/pilosa.go http client now times out after 10 seconds
to help diagnose hung server situations.

- Makefile targets added to get better progress reports.
2020-08-24 11:26:39 -05:00
Nia Weiss
128e02046a
add a postgres endpoint to pilosa 2020-08-20 11:29:16 -04:00