Commit graph

398 commits

Author SHA1 Message Date
Todd Gruben
35672c53ce added node partition endpoint;cluster aware key restore 2021-05-28 09:27:42 -05:00
Nia Weiss
a8f7ec4a12
execute like queries on the primary's key translation database
This works around an issue where unreplicated keys will not be matched everywhere.
This also avoids the cost of creating millions of bolt read transactions and allocating strings.
2021-05-27 14:45:10 -04:00
Alan Bernstein
58f0f42c1e Add /querybuilder path to statik lattice handler 2021-05-26 10:04:46 -05:00
Alan Bernstein
45dd6dd6bc Remove leftovers from lattice submodule 2021-05-26 10:04:46 -05:00
Kuba Podgórski
1ffafd70eb Add return to handleCommitIDs 2021-05-26 15:43:21 +02:00
Nia Weiss
bfe2383f65
correct handling of field translate data fetch in get translate data endpoint
After retrieving field translation, this attempted to fetch index translation which didn't work because this is not an index translation request.
2021-05-24 12:24:00 -04:00
Todd Gruben
ff86f82ef2 applied ben's suggestions 2021-05-23 10:27:50 -05:00
Todd Gruben
cc0c1b829a restore idalloc 2021-05-21 09:27:08 -05:00
Todd Gruben
42b465b80c load schema 2021-05-21 09:27:08 -05:00
Todd Gruben
169dc30d62 api compiles 2021-05-21 09:27:08 -05:00
Nia Weiss
f4ba34247f
remove attributes
Attributes are unmaintained and unused.
They have become more of a liability than a benefit.
This change eliminates them from the codebase.
The only user-visible change (assuming that attrs are not used) is that the attrs field will no longer appear in row JSON.
2021-05-14 10:28:08 -04:00
Maxton Huff
e21382fab8 remove handleInspect and inspect validator 2021-05-10 13:14:50 -05:00
Maxton Huff
882327b0a6 remove inspect router 2021-05-10 10:36:24 -05:00
Ben Johnson
7284c4dd10 Add id alloc, col attrs, & row attrs backup 2021-05-07 10:59:01 -06:00
Ben Johnson
776b43a3cd Backup CLI 2021-05-07 10:59:01 -06:00
Ben Johnson
ea01f7e37c Switch timestamp field to use epoch instead of min/max 2021-04-14 08:46:37 -06:00
Alan Bernstein
285d0a0af8 Add log prefix levels 2021-04-12 20:33:39 -05:00
Ben Johnson
fbd713d435
Merge branch 'master' into fix-int-fk 2021-04-09 11:28:07 -06:00
Ben Johnson
e1909661d5 Remove integer fk error check 2021-04-09 11:01:28 -06:00
Ben Johnson
5defbe3ef2 Fix timestamp value import 2021-04-09 10:56:25 -06:00
Nia
6140fc9d4c
Merge branch 'master' into id-alloc-desync-structured-error 2021-04-09 09:16:25 -04:00
Kuba Podgórski
c021b873d5 Fix panic on field not found on /import 2021-04-09 13:38:39 +02:00
Ben Johnson
cfc725e799 Add timestamp field type support 2021-04-06 10:50:10 -06:00
Nia Weiss
7734bcd53c
change ID allocation to return a structured error on offset desync
This is used to handle a possible case where a kafka partition is moved to another ingester while a previous ingester is still processing it, causing 2 ingesters to process it at the same time.
This allows a duplicate ingester to skip past messages which have already been ingested.
2021-04-06 10:39:22 -04:00
Alan Bernstein
2e93b2eea2 Add some context the longquerytime log message 2021-04-05 17:02:58 -05:00
Antonio Navarro Perez
03659d8d39 Review skipped tests, and try to execute them again.
Signed-off-by: Antonio Navarro Perez <antnavper@gmail.com>
2021-03-30 13:37:33 +02:00
Antonio Navarro Perez
7ec85a1c0f Add at least 3 nodes on test clusters.
Signed-off-by: Antonio Navarro Perez <antnavper@gmail.com>
2021-03-26 10:16:41 +01:00
Seebs
76e4181740 don't reuse sync.Mutex between translate readers
The functional option and returned closure combine to result in
us using the same sync.Mutex object for every TranslateReader on
a given server, which means that if one of them isn't producing anything,
we eventually end up waiting on that with all the others blocked
waiting for the lock. Use separate locks for each, of the same
type as the one initially provided as a template. This does mean
that multiple readers can be operating at once, but in theory
no two readers should ever be writing to the same stores, we
think.
2021-03-08 17:04:52 -06:00
Kuba Podgórski
d311b0cac4 Comment Status function
+ make waitForStatus more generic
2021-03-03 23:47:47 +01:00
Kuba Podgórski
a14baf8c15 waitForStatus for cluster test 2021-03-03 14:22:44 +01:00
Travis
ea8b07d380
Merge branch 'master' into disco 2021-03-02 22:11:04 -06:00
Seebs
4f5f3e30ea remove port_mapper because it can't work with our unrestartable server
Long story short: Once we create a server and start it, we can't start
it again. We can't close it and restart it, and we can't just start
it without closing it.

Unfortunately, if the server's config needs to change, we have a Problem
here.

This ultimately means that the retry logic for GetListeners can't actually
retry successfully; if we fail on the first attempt, we necessarily fail
on any later attempts also, and if we try to fix that, we get panics.

But!

We don't actually NEED to retry. We just need to ensure that we can
open a :0 port, extract the actual port number, and use that in places
where the port number mattered, without having to rebind it.

The only actual place we needed to rebind things was opening gRPC
servers, so we introduce a gRPC Listener that can be used instead of
trying to bind to a specified port.

In a bunch of other cases where we had similar logic to try to allocate
and then use a port, we can switch to just using a provided listener.
For instance, net/http has `Serve(net.Listener, handler)`, not just
ListenAndServe(addr, handler).

This should eliminate the weird CI failures from eaddrinuse.

NOT fixed: server/cluster_test.go/TestClusterResize_AddNode isn't working
right now. The new node isn't actually being added to the existing cluster.
I attempted this but was outsmarted by it, and I think fixing the
rest of this is worth it as a separate thing.
2021-02-24 11:25:46 -06:00
Travis
2bbe1fdde0
remove remaining references to "coordinator" 2021-02-23 17:23:09 -06:00
Antonio Navarro Perez
f0a5ca5d3a Change coordinator error to primary
Signed-off-by: Antonio Navarro Perez <antnavper@gmail.com>
2021-02-23 09:58:22 +01:00
Todd Gruben
4e3beb0d10 Provide option to update existence on import roaring 2021-02-19 18:08:25 -06:00
Todd Gruben
6219b4ca8b add optional UpdateExistence on importRoaring 2021-02-19 17:47:28 -06:00
Travis
5b237cae13
Merge branch 'master' into disco 2021-02-12 20:29:41 -06:00
Alan Bernstein
dc6c92771b Fail entire /schema/details request if one field query fails 2021-02-11 16:42:30 -06:00
Alan Bernstein
2773999190 Simplify response structs 2021-02-11 16:42:30 -06:00
Alan Bernstein
246c345c49 Eliminate omitempty from /schema/details response type 2021-02-11 16:42:30 -06:00
Alan Bernstein
56c940c6a0 Add /schema/details endpoint, which includes field cardinality computed via Count(Distinct()) 2021-02-11 16:42:30 -06:00
Antonio Navarro Perez
e0787ed8a8 Requested changes.
Signed-off-by: Antonio Navarro Perez <antnavper@gmail.com>
2021-02-11 16:37:41 +01:00
Antonio Navarro Perez
1f234df86e Really fix url on Web UI log
Signed-off-by: Antonio Navarro Perez <antnavper@gmail.com>
2021-02-11 11:10:38 +01:00
tgruben
a61ed011fc
Revert "Update existence field on import-roaring requests" 2021-02-10 14:51:11 -06:00
Todd Gruben
e7f272f37f updates existence field on importroaring fixes issue (1411) 2021-02-10 12:43:27 -06:00
Travis Turner
88f13d5e8b
Merge pull request #1398 from travisturner/disco-schema
Implement Schemator
2021-02-10 12:01:00 -06:00
Antonio Navarro Perez
484abe9cdb Fix enabled Web UI url log
Signed-off-by: Antonio Navarro Perez <antnavper@gmail.com>
2021-02-10 13:36:19 +01:00
Travis
114f6a8751
add withViews argument to api.Schema() method 2021-02-08 10:42:55 -06:00
Antonio Navarro Perez
b1ff8e55cd
Unify state
Signed-off-by: Antonio Navarro Perez <antnavper@gmail.com>
2021-02-06 16:53:42 -06:00
Kuba Podgórski
35b9d41da1
Merge branch 'disco-config-noder' into resizer-interface 2021-02-05 11:57:51 +01:00