Commit graph

8920 commits

Author SHA1 Message Date
Matthew Jaffee
fe54cbf8ae remove other print and tweak backup test timings 2021-12-28 13:31:42 -06:00
Matthew Jaffee
bb39b05d05 remove leftover fmt.Println 2021-12-28 13:31:42 -06:00
Hoang Pham
936fb9e6bd UI - rename a unit test 2021-12-28 13:04:56 -06:00
Ben Johnson
f8e13f9629
Merge pull request #1829 from molecula/metrics
Add job & worker metrics
2021-12-28 11:31:13 -07:00
Ben Johnson
310584b0d8 Add job & worker metrics 2021-12-28 10:01:18 -07:00
Hoang Pham
0ad2bffd33 UI - added test files 2021-12-28 10:56:31 -06:00
Samir Patel
e5fa99a531 apply mw to handlers 2021-12-28 10:08:44 -05:00
Samir Patel
6d590581d4 apply mw to handlers 2021-12-28 10:06:40 -05:00
Samir Patel
7544e7d1cb extend mw 2021-12-28 09:24:46 -05:00
Samir Patel
b9961870c1 implement as mw 2021-12-27 18:22:53 -05:00
Samir Patel
7a6595d628 authorize few endpoints e.g. query 2021-12-27 16:43:47 -05:00
Samir Patel
0ef67fd699 move query logger option to auth 2021-12-27 16:42:13 -05:00
Travis Turner
145f65ab0e
Merge pull request #1827 from molecula/tlt/etcd-data-dir
[FB-1125] Expose `etcd.dir` configuration option
2021-12-27 11:42:59 -06:00
Travis
6638fa17ee
Expose etcd.dir configuration option
The goal is to allow a user to separate FeatureBase and etcd I/O.
2021-12-27 11:13:38 -06:00
Ben Johnson
4a5179f87f
Merge pull request #1824 from molecula/rbf-expvar 2021-12-27 10:02:37 -07:00
Ben Johnson
9367a62609 Add /debug/rbf endpoint for debugging 2021-12-27 09:34:43 -07:00
Samir Patel
49e9faa03b stub out checker 2021-12-22 16:49:43 -06:00
Samir Patel
684c408b93 Merge branch '54mir/protect-endpoints' into queryLoggerSetup 2021-12-22 15:12:31 -06:00
Samir Patel
9bf56188f7 Merge branch '54mir/authentication' of github.com:molecula/featurebase into 54mir/authentication 2021-12-22 14:08:21 -06:00
Samir Patel
8b40c6bf7b rm comments 2021-12-22 13:47:16 -06:00
Samir Patel
448289d609 add subcommand for key generation 2021-12-22 13:04:00 -06:00
Matthew Jaffee
f1b525fc74
Merge pull request #1807 from molecula/backup-http-retry
add exponential retry logic to internal http client, use in backup and restore
2021-12-22 13:00:24 -06:00
Matthew Jaffee
295fab4892 retry on >= 400, not just greater. good catch 2021-12-22 12:21:11 -06:00
Samir Patel
23a884b3a2 Merge branch '54mir/authentication' of github.com:molecula/featurebase into 54mir/authentication 2021-12-22 12:05:41 -06:00
Samir Patel
b8b4425d4f clean up 2021-12-22 12:05:24 -06:00
Hoang Pham
39ebbbd88e UI - formatted tsx files with prettier (single quote) 2021-12-22 11:40:01 -06:00
Samir Patel
9612096682 Merge branch '54mir/authentication' of github.com:molecula/featurebase into 54mir/authentication 2021-12-22 11:31:56 -06:00
Samir Patel
52d941d127 more tests 2021-12-22 11:31:37 -06:00
Matthew Jaffee
ea59f14d50 must use retryablehttp.NewClient to get defaults
otherwise it won't actually retry :(
2021-12-22 11:21:11 -06:00
Matthew Jaffee
640ba45129 use retryableHTTP in client, fix memory usage of restore
instead of awkwardly reading an entire file into a buffer, we use
retryablehttp's reader func to open the file fresh if we need to
retry, so a small fixed-size buffer can be used internally for copying
the contents onto the network.
2021-12-22 10:56:16 -06:00
Seebs
63d8686e22 scratch space -- need to finish updating deployPerf though 2021-12-21 16:20:25 -07:00
Seebs
b4304765e8 partial draft of perf/regression test script 2021-12-21 16:20:25 -07:00
Hoang Pham
c67ae54b1d UI - fixed Sign In button 2021-12-21 16:44:27 -06:00
Matthew Jaffee
cde3f6b5ea add profiling to backup/restore 2021-12-21 16:24:21 -06:00
Matthew Jaffee
2bce396445 add retry restore test and custom retry policy 2021-12-21 16:24:21 -06:00
Matthew Jaffee
d3b9193c8d try to fix data race with http lib
WARNING: DATA RACE
Write at 0x00c008121e80 by goroutine 235:
  bytes.(*Reader).WriteTo()
      /usr/local/go/src/bytes/reader.go:139 +0x45
  github.com/molecula/featurebase/v2/http.nopCloser.WriteTo()
      <autogenerated>:1 +0x5d
  io.copyBuffer()
      /usr/local/go/src/io/io.go:391 +0x482
  io.Copy()
      /usr/local/go/src/io/io.go:368 +0x78
  net/http.(*transferWriter).doBodyCopy()
      /usr/local/go/src/net/http/transfer.go:400 +0x2f
  net/http.(*transferWriter).writeBody()
      /usr/local/go/src/net/http/transfer.go:364 +0xc9a
  net/http.(*Request).write()
      /usr/local/go/src/net/http/request.go:682 +0x887
  net/http.(*persistConn).writeLoop()
      /usr/local/go/src/net/http/transport.go:2343 +0x349

Previous write at 0x00c008121e80 by goroutine 192:
  bytes.(*Reader).Seek()
      /usr/local/go/src/bytes/reader.go:118 +0x824
  github.com/molecula/featurebase/v2/http.(*InternalClient).doWithRetry()
      /go/src/github.com/molecula/featurebase/http/client.go:1773 +0x86d
  github.com/molecula/featurebase/v2/http.(*InternalClient).executeRequest()
      /go/src/github.com/molecula/featurebase/http/client.go:1806 +0x15b
  github.com/molecula/featurebase/v2/http.(*InternalClient).CreateIndex()
      /go/src/github.com/molecula/featurebase/http/client.go:433 +0xbf8
  github.com/molecula/featurebase/v2/server_test.TestMain_Set_Quick.func1()
      /go/src/github.com/molecula/featurebase/server/server_test.go:64 +0x624
  testing.tRunner()
      /usr/local/go/src/testing/testing.go:1123 +0x202

Goroutine 235 (running) created at:
  net/http.(*Transport).dialConn()
      /usr/local/go/src/net/http/transport.go:1709 +0xc30
  net/http.(*Transport).dialConnFor()
      /usr/local/go/src/net/http/transport.go:1421 +0x151

Goroutine 192 (running) created at:
  testing.(*T).Run()
      /usr/local/go/src/testing/testing.go:1168 +0x5bb
  github.com/molecula/featurebase/v2/server_test.TestMain_Set_Quick()
      /go/src/github.com/molecula/featurebase/server/server_test.go:45 +0x116
  testing.tRunner()
      /usr/local/go/src/testing/testing.go:1123 +0x202
2021-12-21 16:24:21 -06:00
Matthew Jaffee
f676fbfc51 add retryability to restore command 2021-12-21 16:24:20 -06:00
Matthew Jaffee
3105a24542 rewind Body on retry
this is really not ideal, and there are libraries for this kind of
thing, but I'd have to figure out how to make the libraries work with
everywhere we're already creating stdlib http clients.
2021-12-21 16:24:20 -06:00
Matthew Jaffee
cdf4bc4c88 add clustertests testing backup's retry 2021-12-21 16:24:20 -06:00
Matthew Jaffee
8486efaa79 add exponential retry logic to internal http client, use in backup 2021-12-21 16:24:20 -06:00
Hoang Pham
2bf1f0e4d4 Merge branch '54mir/authentication' of ssh://github.com/molecula/featurebase into 54mir/authentication 2021-12-21 16:06:11 -06:00
Hoang Pham
7de6c37b11 UI - cleaned up code, added comments 2021-12-21 16:06:02 -06:00
rachithrr
5650a24c9b query logger is set up. 2021-12-21 16:52:22 -05:00
Garrison Davis
d9dc613dd5
Merge pull request #1820 from molecula/pipeline-changes
Run integration on merge to default branch
2021-12-21 14:50:29 -07:00
Samir Patel
fd7d905be2 fix formatting issues 2021-12-21 15:49:51 -06:00
garrison.davis@molecula.com
15612b8b92 Run integration on merge to default branch
Additionally, the go version is using the GOVERSION build
variable instead.
2021-12-21 13:58:10 -07:00
Samir Patel
f011587d4e add tests 2021-12-21 10:08:49 -06:00
Samir Patel
1c907281bf authz changes 2021-12-20 18:03:18 -06:00
Samir Patel
10a7aa55ea same-site strict 2021-12-20 18:00:13 -06:00
Samir Patel
d9710cb7d7 remove auth struct from authorization 2021-12-20 16:45:17 -06:00