Commit graph

304 commits

Author SHA1 Message Date
Alan Bernstein
9565620f4d Disable /ui/usage endpoint completely when usage-duty-cycle is set to 0 2021-08-04 16:22:55 -05:00
Ben Johnson
60d534c505 Limit translation memory & add max query memory config 2021-08-02 15:28:00 -06:00
Mahesh Arumugam
858f889745 FeatureBase Renaming: changing go.mod module name for featurebase 2021-07-19 09:20:30 -07:00
Mahesh Arumugam
eb1cc304cf FeatureBase Renaming: EnvPrefix 2021-07-09 14:52:40 -07:00
Travis Turner
1809636319
improve help text
Co-authored-by: Alan Bernstein <alanaaronbernstein@gmail.com>
2021-07-08 13:53:06 -05:00
Travis
3c7ee11ff3
Add feature flag --future.rename to support rename to FeatureBase
This commit adds a `Future` scope to the configuration options, and for
the time being includes a single flag within that scope: `rename`.

Usage:
--future.rename

The value is a boolean available internally at: m.Config.Future.Rename
2021-07-08 13:18:43 -05:00
Nia
bc01e55280
Merge branch 'master' into keepInTranslation 2021-06-30 13:11:58 -04:00
Seebs
90c424a4d9 separate backup of index data and index translation keys
It's not enough to back up each index's translation keys after
backing up that index's data; we also have to back them up after
backing up any index data from indexes which have foreign key
references to that index. So we do the per-index passes separately.

Since the individual backup data files are being created separately,
the expected output is unchanged for a quiescent database, the only
difference is that the amount of translation info which might be
newer than the data stored for shards is potentially increased.
2021-06-30 11:46:47 -05:00
Nia Weiss
779e27dcf6
fsync all directories after completing a backup
Previously the backup tool only fsync'ed the files.
Since the directories were not synced, it was possible for the references to be lost.
Now we sync the entire output directory tree and its parent.
2021-06-30 12:39:17 -04:00
Todd Gruben
44281fa5c2 add chksum command for easier data validation 2021-06-23 16:10:52 -05:00
Todd Gruben
23ba86cbfc linter fix 2021-06-21 14:15:13 -05:00
Ben Johnson
dad2372a8c CORE-640: Parallelize restore command 2021-06-18 16:03:38 -06:00
Ben Johnson
3df1f2d094 CORE-639: Parallelize backup 2021-06-15 16:55:05 -06:00
Samir Patel
92ff74c72d
Merge branch 'master' into reset-cache-on-schema 2021-06-15 10:32:43 -05:00
Samir Patel
8bf472bb75 add duty cycle config flag 2021-06-14 10:48:52 -05:00
Ben Johnson
73f0668fe1 Refactor backup/restore to use directory archive 2021-06-14 09:43:20 -06:00
Samir Patel
b46b5fc134 remove usage-interval flag 2021-06-11 11:16:32 -05:00
Samir Patel
ae6687e71b change flag name to usage-interval 2021-06-01 10:47:09 -05:00
Samir Patel
f5cc179893 change flag from interval to duration 2021-06-01 10:47:09 -05:00
Samir Patel
8ce17f405b rename flag to disk-usage-interval 2021-06-01 10:47:09 -05:00
Samir Patel
827b125c3c rename and set default 2021-06-01 10:47:09 -05:00
Samir Patel
835c63011a add server flag 2021-06-01 10:47:08 -05:00
Todd Gruben
c16a7a06a0 applied bens suggestions mainly using errgroup 2021-05-28 11:02:31 -05:00
Todd Gruben
35672c53ce added node partition endpoint;cluster aware key restore 2021-05-28 09:27:42 -05:00
Kuba Podgórski
a1e7755fe7
Merge branch 'master' into fix-file-backup 2021-05-24 18:29:44 +02:00
Todd Gruben
94e66b74b5 tls config bug and arge handling 2021-05-24 11:00:37 -05:00
Kuba Podgórski
be06340869
Merge branch 'master' into fix-file-backup 2021-05-24 16:21:56 +02:00
Todd Gruben
69fda13ff4 cleanup arg processing 2021-05-24 08:48:32 -05:00
Kuba Podgórski
935edba1db Fix backup to a file 2021-05-24 10:36:58 +02:00
Todd Gruben
ff86f82ef2 applied ben's suggestions 2021-05-23 10:27:50 -05:00
Todd Gruben
3d577d762a cleanup 2021-05-21 09:27:08 -05:00
Todd Gruben
b25ad81e67 restore without restart 2021-05-21 09:27:08 -05:00
Todd Gruben
cc0c1b829a restore idalloc 2021-05-21 09:27:08 -05:00
Todd Gruben
a1a9103f62 restore column translate keys 2021-05-21 09:27:08 -05:00
Todd Gruben
329f86033d restore field translate keys 2021-05-21 09:27:08 -05:00
Todd Gruben
69245ee209 shard import 2021-05-21 09:27:08 -05:00
Todd Gruben
42b465b80c load schema 2021-05-21 09:27:08 -05:00
Todd Gruben
9d24fb07b7 wired in restore command 2021-05-21 09:27:08 -05:00
Todd Gruben
169dc30d62 api compiles 2021-05-21 09:27:08 -05:00
Todd Gruben
7c6423587a skeleton restore 2021-05-21 09:27:08 -05:00
Ben Johnson
a4f282c8e8 Allow backup to stdout 2021-05-19 15:04:36 -06:00
Ben Johnson
eb79c35cbd Add support for clustered backups 2021-05-18 15:17:28 -06:00
Nia Weiss
f4ba34247f
remove attributes
Attributes are unmaintained and unused.
They have become more of a liability than a benefit.
This change eliminates them from the codebase.
The only user-visible change (assuming that attrs are not used) is that the attrs field will no longer appear in row JSON.
2021-05-14 10:28:08 -04:00
Ben Johnson
7e804d7d08 Fix id alloc backup invocation 2021-05-07 10:59:01 -06:00
Ben Johnson
7284c4dd10 Add id alloc, col attrs, & row attrs backup 2021-05-07 10:59:01 -06:00
Ben Johnson
776b43a3cd Backup CLI 2021-05-07 10:59:01 -06:00
Seebs
e93d2fe06c bitmap unmarshalling and testing bug fixes
When unmarshalling ops, we weren't adding a meaningful OpN to them,
resulting in misleading reports from `pilosa inspect`. Also, we were
mistakenly reporting things as "mapped" when they were actually
using their internal storage (as with small array containers).

Add the "sanity check" to `pilosa inspect` so that errors like the
above get noticed more easily and corrected. Also, to make that work,
have roaring.InspectBinary actually put containers in the bitmap
it creates rather than just creating info entries for them.
2021-04-20 12:01:21 -05:00
Seebs
fd8a19278c add TestMain wrapper in ctl
The TestMain wrapper gets us the fancy testhook stuff tracking whether we're
deallocating things as expected, and we probably want that.
2021-04-13 12:37:07 -05:00
Alan Bernstein
285d0a0af8 Add log prefix levels 2021-04-12 20:33:39 -05:00
Nia
d5da6e229c
Merge branch 'master' into external-lookup 2021-03-31 08:12:18 -04:00