Commit graph

756 commits

Author SHA1 Message Date
Todd Gruben
2ddcbce8ad fix govet and gofmt errors in existing code 2021-10-29 13:14:27 -05:00
Seebs
ad30a926f4 Giant Commit: drop a bunch of stuff we don't use.
These commits are hard to disentagle, and doing them separately means
re-modifying the same chunks of code several times before removing it,
and similar things.

Basically:
(1) Drop the bolt backend storage.
(2) Drop the blue-green wrapper that compares two backends.
(3) Drop unused or barely-used Tx API components from all the
remaining backends.
(4) Minor related cleanup to simplify things related to these.

The boltdb backend existed only to verify RBF. The blue-green wrapper
was mostly used to verify RBF, but in practice we had to do a lot
of working around that, and it introduced a lot of special cases.

Types removed:

IteratorFinder: Used only to implement the roaring iterator
on top of boltdb, and to complicate the way it worked in roaring.
Reverted the complications. Also unexport NewSliceContainers
which is used only for that outside of roaring's internals.

PortMapper from cluster_internal_test.go: Used only for a test
we removed early this year. Never used for anything else.

RawRoaringData: Totally unused.

TxStore: Totally unused.

Functions removed from Tx API, and sometimes corresponding
members were removed from structs:

* Dump: debugging code, I don't think I found any actually reachable
  paths to it.
* Group: only used for debugging TxGroup stuff
* IncrementOpN: only used by fragment, fragment can increment its
  own opN.
* Options: unused?
* Pointer: debugging only
* Readonly: used only to decide how to handle Tx in a TxGrp,
  but we never add a non-readonly Tx to a TxGrp. Removed also all
  the corresponding write-aware stuff.
* RoaringBitmapReader: Used exactly once, can just be a bm.WriteTo.
* Sn (and OpenSnList): Unused
* UnionInPlace: unused and conceptually-invalid; it didn't write
  to storage and shouldn't have, and was just "create a bitmap
  then call union-in-place", which we can do directly.
* UseRowCache: just checked storage.UseRowCache.

Other things removed:

The SetRequiredForAtomicWriteTx and ClearRequiredForAtomicWriteTx
functions go away, since nothing now seems to be using them? Same
for holder_internal_test's `testHasBit` and `testMustNotHaveBit`,
which were unused.

The DBPerShard "DeleteDBPath" and "HasData" functions and related
parts were mostly unused; took out the parts that were never
actually being reached.

Changed the API of one function to simplify special cases and
remove things:
* ImportRoaringBits had a special "data" argument which gave it
  subtly different semantics for RBF and roaring (for roaring, it
  could produce a roaring bitmap *with ops log*), didn't seem to
  be adding much. Removed corresponding "readStorageFromArchive"
  which is not otherwise used.

Also took out various debugging/dumping functions that were unused
and may have bitrotted.

Dropped a test from txfactory_internal_test, and the "pjobs"
code, because those two were the only things that needed Barrier
and thus idem, which lets us drop two more dependencies. We already
have errgroup for grouping things which want to terminate as
soon as one of them errors, approximately. To do better we'd have
to have context-threading, really.

Unbroke the WriteFragment test for non-roaring tests and made it
not roaring-only.
2021-10-26 12:30:25 -05:00
Todd Gruben
4d4f64a339 address ben's comments 2021-10-12 11:21:14 -05:00
Todd Gruben
c4c7d91bf0 make linker happy 2021-10-12 10:24:19 -05:00
Todd Gruben
caee5680c3 wip 2021-10-12 10:20:24 -05:00
Todd Gruben
1a068cf0e2 added type support for looker; added intercept for yellowfin typelen query 2021-10-12 10:20:24 -05:00
Todd Gruben
51876b1821 addsql version to handler 2021-09-27 15:42:14 -05:00
Todd Gruben
0920c4c029 silence and rebase 2021-09-25 14:06:55 -05:00
Todd Gruben
d6dd1e025b removed extra command complete message
cleanup
2021-09-25 13:53:19 -05:00
Todd Gruben
5176b7ff03 added sqlversion config option 2021-09-25 13:53:19 -05:00
Todd Gruben
9c8538ec5a docker fix not really related to anything 2021-09-25 13:53:19 -05:00
Todd Gruben
c56fd33924 wip 2021-09-25 13:51:02 -05:00
Todd Gruben
95dc4a1a50 basic looker connection tests pass
sql1 pass through works
2021-09-25 13:51:02 -05:00
Alan Bernstein
9565620f4d Disable /ui/usage endpoint completely when usage-duty-cycle is set to 0 2021-08-04 16:22:55 -05:00
Ben Johnson
60d534c505 Limit translation memory & add max query memory config 2021-08-02 15:28:00 -06:00
Mahesh Arumugam
f51fd2351c Merge branch 'master' into ma/cloud-109 2021-07-19 12:40:35 -07:00
Mahesh Arumugam
858f889745 FeatureBase Renaming: changing go.mod module name for featurebase 2021-07-19 09:20:30 -07:00
Alan Bernstein
b7898b0a22 Allow usage-duty-cycle < 20%, and 0 disables 2021-07-15 14:22:17 -05:00
Mahesh Arumugam
2cec88e19d Merge branch 'master' into ma/cloud-110 2021-07-13 09:10:36 -07:00
Travis
4b494c3ec3
Update metric names to use "featurebase" prefix
If the `--future.rename` flag is set (to true), this commit will cause
metric names to be prefixed with "featurebase" instead of "pilosa".
2021-07-12 17:51:59 -05:00
Mahesh Arumugam
48fcb3a198 FeatureBase renaming: version info in server log 2021-07-12 12:25:43 -07:00
Travis
3c7ee11ff3
Add feature flag --future.rename to support rename to FeatureBase
This commit adds a `Future` scope to the configuration options, and for
the time being includes a single flag within that scope: `rename`.

Usage:
--future.rename

The value is a boolean available internally at: m.Config.Future.Rename
2021-07-08 13:18:43 -05:00
Samir Patel
8bf472bb75 add duty cycle config flag 2021-06-14 10:48:52 -05:00
Samir Patel
76b899ef82 Merge branch 'reset-cache-on-schema' of github.com:54mir/pilosa into reset-cache-on-schema 2021-06-11 11:24:49 -05:00
Samir Patel
b46b5fc134 remove usage-interval flag 2021-06-11 11:16:32 -05:00
Kuba Podgórski
b218901fae Add support for timestamps in pg writer 2021-06-10 21:58:08 +02:00
Samir Patel
9608c0c2cb move trigger to inside refresh fn 2021-06-01 20:48:12 -05:00
Samir Patel
edf71c1bf7 update comment and remote print 2021-06-01 10:47:10 -05:00
Samir Patel
5437a11216 remove prints 2021-06-01 10:47:10 -05:00
Samir Patel
4ac7f6e154 check error from ResetCache 2021-06-01 10:47:10 -05:00
Samir Patel
5f46635094 add channel to Refresh goroutine 2021-06-01 10:47:10 -05:00
Samir Patel
0810273ea1 reset cache before test and add test conditions 2021-06-01 10:47:10 -05:00
Samir Patel
3e81406ab0 clean test case and add comments 2021-06-01 10:47:10 -05:00
Samir Patel
eff3b25b97 change test case to reflect cache loading before test 2021-06-01 10:47:10 -05:00
Samir Patel
fe0bb20658 remove intentional failure 2021-06-01 10:47:10 -05:00
Samir Patel
a0ba9327f7 play with timing 2021-06-01 10:47:09 -05:00
Samir Patel
dbdd3c4998 see if this is the only test failing 2021-06-01 10:47:09 -05:00
Samir Patel
fd58fe1a7d test stuff 2021-06-01 10:47:09 -05:00
Samir Patel
108da005b7 test stuff 2021-06-01 10:47:09 -05:00
Samir Patel
f946528053 comment out debug statements 2021-06-01 10:47:09 -05:00
Samir Patel
0b083da126 debugging test 2021-06-01 10:47:09 -05:00
Samir Patel
1385cf61eb attempt to address missing node uri issue 2021-06-01 10:47:09 -05:00
Samir Patel
79f04f31cb see if this gets test passing 2021-06-01 10:47:09 -05:00
Samir Patel
ae6687e71b change flag name to usage-interval 2021-06-01 10:47:09 -05:00
Samir Patel
f5cc179893 change flag from interval to duration 2021-06-01 10:47:09 -05:00
Samir Patel
8ce17f405b rename flag to disk-usage-interval 2021-06-01 10:47:09 -05:00
Samir Patel
9beb3f0b3a rename flag and add default value for flag 2021-06-01 10:47:09 -05:00
Samir Patel
827b125c3c rename and set default 2021-06-01 10:47:09 -05:00
Samir Patel
03e0df389d use flag value as refresh value 2021-06-01 10:47:09 -05:00
Samir Patel
52418df4ba start periodic cache recalculation at startup 2021-06-01 10:47:08 -05:00