Samir Patel
41bde6ccba
don't write content to no content
2022-01-06 10:35:11 -06:00
Samir Patel
6e9efd0e09
Merge branch 'protect-endpoints' of github.com:molecula/featurebase into protect-endpoints
2022-01-05 17:30:18 -06:00
Samir Patel
3fe381ff22
address feeback
2022-01-05 17:29:59 -06:00
reesporte
fd896de270
rename CookieValue to AuthContext
...
because we're not using cookies anymore
2022-01-05 12:10:33 -06:00
reesporte
8d6490329b
Merge branch 'master' into protect-endpoints
2022-01-04 16:18:18 -06:00
Samir Patel
414dff1d22
revisions
2022-01-03 22:56:21 -06:00
Samir Patel
1b10f26258
fix permission stuff for write queries
2022-01-03 17:41:09 -06:00
Samir Patel
7834db2347
change write call detection
2022-01-03 16:24:14 -06:00
Samir Patel
d18b739402
add test cases
2022-01-03 11:49:38 -06:00
Samir Patel
c0fe253ce2
Merge branch 'protect-endpoints' of github.com:molecula/featurebase into protect-endpoints
2021-12-29 14:41:49 -05:00
Samir Patel
cf86be16c1
add authN only middleware for /internal
2021-12-29 14:41:31 -05:00
reesporte
8e697c0d8c
Merge branch 'protect-endpoints' of github.com:molecula/featurebase into protect-endpoints
2021-12-29 13:38:55 -06:00
reesporte
17679eb924
create a Permissions type
...
makes it nice to say p.Satisfies(otherPerm)
2021-12-29 13:38:11 -06:00
Samir Patel
7750900310
more logging
2021-12-29 13:18:42 -05:00
reesporte
42f3557c55
fix merge conflicts
2021-12-29 09:05:53 -06:00
Samir Patel
d95d4dac9d
pass group membership thru context
2021-12-28 17:36:52 -05:00
Matthew Jaffee
1a8c10d5f3
fix backup fail test so it actually fails
...
A few things were going wrong here.
First, we take a "RetryPeriod" option on backup and restore which is
meant to be roughly the total amount of time we spend retrying any
given request before failing. However we were incorrectly passing that
as the RetryMaxWait which is the maximum amount of time to sleep
between any two attempts. We now do some fuzzy math to figure out
approximately how many attempts we should make given a minimum sleep
of 100ms and the fact that we double the sleep time every attempt.
Second, during the backup test, if a host was totally stopped when we
started the request, it would fail immediately and then retry, but if
the host was stopped during the request (after DNS had resolved), then
the request would wait for the DialTimeout which we default to 30s, so
turning off the cluster for 5 seconds and turning it back on resulted
in the backup completing rather than failing. Because of this, we
change the commandClient to have a default dial timeout of 1 second.
I was tempted to change the global default to 1s which I think would
be fine, but didn't want to break anything too badly.
2021-12-28 13:31:42 -06:00
Matthew Jaffee
fe54cbf8ae
remove other print and tweak backup test timings
2021-12-28 13:31:42 -06:00
Matthew Jaffee
bb39b05d05
remove leftover fmt.Println
2021-12-28 13:31:42 -06:00
Samir Patel
e5fa99a531
apply mw to handlers
2021-12-28 10:08:44 -05:00
Samir Patel
6d590581d4
apply mw to handlers
2021-12-28 10:06:40 -05:00
Samir Patel
7544e7d1cb
extend mw
2021-12-28 09:24:46 -05:00
Samir Patel
b9961870c1
implement as mw
2021-12-27 18:22:53 -05:00
Samir Patel
7a6595d628
authorize few endpoints e.g. query
2021-12-27 16:43:47 -05:00
Ben Johnson
9367a62609
Add /debug/rbf endpoint for debugging
2021-12-27 09:34:43 -07:00
Samir Patel
49e9faa03b
stub out checker
2021-12-22 16:49:43 -06:00
Samir Patel
684c408b93
Merge branch '54mir/protect-endpoints' into queryLoggerSetup
2021-12-22 15:12:31 -06:00
Matthew Jaffee
295fab4892
retry on >= 400, not just greater. good catch
2021-12-22 12:21:11 -06:00
Samir Patel
b8b4425d4f
clean up
2021-12-22 12:05:24 -06:00
Samir Patel
52d941d127
more tests
2021-12-22 11:31:37 -06:00
Matthew Jaffee
ea59f14d50
must use retryablehttp.NewClient to get defaults
...
otherwise it won't actually retry :(
2021-12-22 11:21:11 -06:00
Matthew Jaffee
640ba45129
use retryableHTTP in client, fix memory usage of restore
...
instead of awkwardly reading an entire file into a buffer, we use
retryablehttp's reader func to open the file fresh if we need to
retry, so a small fixed-size buffer can be used internally for copying
the contents onto the network.
2021-12-22 10:56:16 -06:00
Matthew Jaffee
cde3f6b5ea
add profiling to backup/restore
2021-12-21 16:24:21 -06:00
Matthew Jaffee
d3b9193c8d
try to fix data race with http lib
...
WARNING: DATA RACE
Write at 0x00c008121e80 by goroutine 235:
bytes.(*Reader).WriteTo()
/usr/local/go/src/bytes/reader.go:139 +0x45
github.com/molecula/featurebase/v2/http.nopCloser.WriteTo()
<autogenerated>:1 +0x5d
io.copyBuffer()
/usr/local/go/src/io/io.go:391 +0x482
io.Copy()
/usr/local/go/src/io/io.go:368 +0x78
net/http.(*transferWriter).doBodyCopy()
/usr/local/go/src/net/http/transfer.go:400 +0x2f
net/http.(*transferWriter).writeBody()
/usr/local/go/src/net/http/transfer.go:364 +0xc9a
net/http.(*Request).write()
/usr/local/go/src/net/http/request.go:682 +0x887
net/http.(*persistConn).writeLoop()
/usr/local/go/src/net/http/transport.go:2343 +0x349
Previous write at 0x00c008121e80 by goroutine 192:
bytes.(*Reader).Seek()
/usr/local/go/src/bytes/reader.go:118 +0x824
github.com/molecula/featurebase/v2/http.(*InternalClient).doWithRetry()
/go/src/github.com/molecula/featurebase/http/client.go:1773 +0x86d
github.com/molecula/featurebase/v2/http.(*InternalClient).executeRequest()
/go/src/github.com/molecula/featurebase/http/client.go:1806 +0x15b
github.com/molecula/featurebase/v2/http.(*InternalClient).CreateIndex()
/go/src/github.com/molecula/featurebase/http/client.go:433 +0xbf8
github.com/molecula/featurebase/v2/server_test.TestMain_Set_Quick.func1()
/go/src/github.com/molecula/featurebase/server/server_test.go:64 +0x624
testing.tRunner()
/usr/local/go/src/testing/testing.go:1123 +0x202
Goroutine 235 (running) created at:
net/http.(*Transport).dialConn()
/usr/local/go/src/net/http/transport.go:1709 +0xc30
net/http.(*Transport).dialConnFor()
/usr/local/go/src/net/http/transport.go:1421 +0x151
Goroutine 192 (running) created at:
testing.(*T).Run()
/usr/local/go/src/testing/testing.go:1168 +0x5bb
github.com/molecula/featurebase/v2/server_test.TestMain_Set_Quick()
/go/src/github.com/molecula/featurebase/server/server_test.go:45 +0x116
testing.tRunner()
/usr/local/go/src/testing/testing.go:1123 +0x202
2021-12-21 16:24:21 -06:00
Matthew Jaffee
3105a24542
rewind Body on retry
...
this is really not ideal, and there are libraries for this kind of
thing, but I'd have to figure out how to make the libraries work with
everywhere we're already creating stdlib http clients.
2021-12-21 16:24:20 -06:00
Matthew Jaffee
cdf4bc4c88
add clustertests testing backup's retry
2021-12-21 16:24:20 -06:00
Matthew Jaffee
8486efaa79
add exponential retry logic to internal http client, use in backup
2021-12-21 16:24:20 -06:00
rachithrr
5650a24c9b
query logger is set up.
2021-12-21 16:52:22 -05:00
Samir Patel
f011587d4e
add tests
2021-12-21 10:08:49 -06:00
Samir Patel
3b374a62bf
Merge branch 'master' into 54mir/authentication
2021-12-20 16:42:11 -06:00
Samir Patel
602145b390
add to tests
2021-12-20 15:41:18 -06:00
Samir Patel
6faa889bfb
move logout url to conf
2021-12-20 14:30:19 -06:00
Samir Patel
e7f4eb1e36
response codes
2021-12-20 12:28:17 -06:00
Samir Patel
67d438aab5
clean up
2021-12-20 01:29:11 -06:00
Samir Patel
605d47702e
add handler tests
2021-12-20 00:15:48 -06:00
Samir Patel
0d52a952e0
resolve some comments
2021-12-19 23:33:44 -06:00
Samir Patel
3b58e887ed
add group lenth check
2021-12-17 11:34:48 -06:00
Samir Patel
b37f13e5c5
rename
2021-12-16 20:33:23 -06:00
Samir Patel
52625c70ab
check auth enabled before handling auth requests
2021-12-15 15:42:56 -06:00
Samir Patel
0cb27bfdd3
Merge branch '54mir/authentication' of github.com:molecula/featurebase into 54mir/authentication
2021-12-15 14:54:41 -06:00
