Commit graph

11 commits

Author SHA1 Message Date
Garrison Davis
c07727548d Stop using string keys in contexts 2023-01-23 10:24:52 -07:00
Joe Friedrich
7da67caa99 fix go deps, add lattice 2023-01-20 02:11:00 +00:00
pokeeffe-molecula
a1fc6d04a1 introduce performance counters and system table fanout, plus refactor metrics (#2363)
* performance counters

* first cut of perf counters and system table fanout and a wire protocol
* significantly refactored prometheus support; removed statsd and exprvar

* removed node_id

* put dax subquery test back

* Change Translator.TranslateFieldIDs method to take a dax.TableKeyer

There are a bunch of other calls to the Translator interface methods
with currently take an `index string`, and those need to be converted to
dax.TableKeyer as well. But I need to review each call, because in at
least one place I noticed one being called with `result.Index` instead
of with the qtbl available. And I don't yet know how those could be
different.

Co-authored-by: Travis Turner <travis@molecula.com>
(cherry picked from commit 7f6ea0e6e5)
2023-01-19 21:35:02 +00:00
tgruben
2fc7abd72e Dataframe (#2241)
* Dataframe

(cherry picked from commit 2f1beaf119)
2022-12-12 09:01:20 -08:00
Garrison Davis
13a32409f3 Fix server.GetIndexes userInfo logic
userInfo at the relevant line can be nil here. We check later if the
userInfo != nil (and assure it passes authn/authz) or if userInfo == nil
then we return all the indexes we can find.

(cherry picked from commit 407d52baa0)
2022-11-15 11:32:03 -08:00
Garrison Davis
ab2b48da0d Stop using string keys in contexts
This fixes the OriginalIP and RequestUserID in the main featurebase
package, and the Access and Refresh tokens, the UserInfo, and the
[]string of Indexes passed with context.Context(s) in the authn package.

An empty struct was used for all of these keys (and relevant helper
functions we added) to avoid allocations where possible while still
using the context functionality.

Some of the logic in the server.GetIndexes function was fixed.

(cherry picked from commit 0f5a56c958)
2022-11-15 11:32:03 -08:00
Seebs
ea69b0637d significant refactor of test setup and teardown
We centralize the creation paths for test indexes, fields,
etcetera so they all have a common path, all using standard
test holders. There's still two versions, one for test.* functions
and one for internal. They do share a TestHolderConfig though.

Large hunks of the related APIs are simplified/streamlined.
* Fragments are always created with a Field and don't need
  a workaround in case they don't have it.
* Creation of test fragments, etc., use optional FieldOptions
  but don't specify names because they're all using new holders
  for each thing created anyway. This dramatically reduces
  the complexity of the calls.
* test fragments are created inside test views which are created
  inside test fields, etcetera, so everything is using the same
  logic; test views aren't bypassing the other layers, they're
  creating themselves normally within a field.
* Quite a few things now use the standard runtime/production
  logic instead of being custom workarounds; for instance, instead
  of `mustOpenMutexFragment` creating a fragment and then creating
  a mutex vector for it, we just create a mutex-typed field and
  have the normal runtime code do this.
* Similarly, we now use the same field creation logic that production
  does, instead of having our own test-only thing that validates
  field names directly, so our test that we're validating field names
  is actually testing the runtime code. Yay.
* fragSpec goes away. it was a replacement for fragProxy which existed
  to solve memory allocation problems but replaced them with interface
  overhead problems. Now we just have pointers to things and maintain
  valid data structures.
* Many panics are now Fatal or Fatalf calls.
* Some specific bugs fixed, like a cluster which was requested and
  then had its first node directly overwritten, which isn't valid with
  shared clusters.
* Drop the temp-dir test flag and TempDir variable, we can just use
  $TMPDIR.
* Drop a benchmark of "write file to disk" that was purely a benchmark
  of file write speed, not a benchmark of rendering the data that needs
  to be written.
* Drop the unused "flags" parameter to fragment creation, which was
  only used back when we changed the BSI format.
* Use holder.Txf() rather than index.Txf(). The TxFactory has to be
  holder-level anyway, referring to it via the index is misleading.
* Test holders automatically close themselves and delete themselves,
  we remove various other things that thought they were responsible
  for deleting themselves.
2022-09-30 11:25:27 -07:00
Seebs
964e7d86c8 kill everything that tries to pass nil tx to field/view things
The "field/view will just synthesize a tx" behavior is awful and
also hides a number of fundamental flaws. We distinguish between
"we really do mean to work on a single shard here" and "we intend
to work on the whole field or view", and the latter now take
Qcx instead of Tx.

This eliminates a lot of very weird cases where we checked for
nil Tx and synthesized them, and also gets us away from
field and view taking Tx parameters when no possible Tx
can be constructed which is valid, because Tx are inherently
shard-specific at this time.
2022-09-30 11:10:47 -07:00
CLoZengineer
f9ddb5d5c1
fix: updating code to meet linting requirements (#2171)
* removes unused filesize function

* removes ioutil usage

* updates ioutil.ReadAll to io.ReadAll

* updates ioutil.TempFile to os.CreateTemp

* updates ioutil.TempDir to os.MkdirTemp

* updates ioutil.ReadAll to os.ReadAll

* update ioutil.WriteFile to os.WriteFile

* updates ioutil.Discard to io.Discard

* updates ioutil.ReadDir to os.ReadDir where applicable

* removes unused code in idk

* creates type to use for context value keys

* replaces assert.Nil with assert.NoError for error checks
2022-09-29 12:34:29 -04:00
Fletcher Haynes
da9b57bd45 Updated dependency paths to reflect new repo location 2022-09-06 09:39:22 -07:00
Fletcher Haynes
eb06bb50ae Updated code to latest version for open-sourcing. 2022-09-02 13:23:39 -07:00