Commit graph

6339 commits

Author SHA1 Message Date
Cody Soyland
fc29bf5ec8 Fix Docker build target, enable when building Linux releases 2020-10-23 14:27:00 -05:00
Nia
c1db627593
Merge pull request #989 from niaow/translate-maybe
Allow querying without creating keys
2020-10-23 14:23:35 -04:00
Nia
3b93b767fa
Merge branch 'master' into translate-maybe 2020-10-23 13:59:10 -04:00
Nia Weiss
ddee7ae35e
address review comments for #989 translate-maybe 2020-10-23 10:59:08 -04:00
jaten-molecula
8f1aae9a69
Merge pull request #1016 from jaten-molecula/rowcache_optional
pilosa server --rowcache-off disables the row cache
2020-10-22 22:14:51 -05:00
Jason E. Aten
8d2ad048fa pilosa server --norowcache disables the row cache
- this can lessen memory pressure
- certain backends may not need it
- enables performance benchmarking and tuning
2020-10-22 22:20:48 +00:00
seebs
9eb251ec85
Merge pull request #69 from seebs/ctbench
Container op benchmarks
2020-10-22 15:03:09 -05:00
Seebs
8095455355 container performance benchmarking
This adds a series of archetypal containers that represent the
common use cases (arrays, bitmaps, or runs of various cardinalities)
and runs the basic operations against them for benchmarking purposes.

`benchpretty` is an app to snatch the BenchmarkCt* lines from
benchmark runs and display them in a possibly more usable form,
mostly as a precursor to cool analysis things.

This also adds a test to verify that intersectionCount(a, b)
is the same as intersect(a, b).N() for all the archetypal
containers.

This also includes a performance fix for intersectBitmapRun which
was spotted while running these tests.
2020-10-22 14:33:03 -05:00
Nia Weiss
afbbc93047
add basic tests for queries with missing keys 2020-10-22 13:19:05 -04:00
Nia Weiss
9204298d06
fix broken rebase on GetNodeUsage 2020-10-22 11:10:36 -04:00
Nia Weiss
8340b9ea64
re-apply single equals fix 2020-10-22 11:03:51 -04:00
Nia Weiss
b0a588f8dd
finish translate-if-exists 2020-10-22 10:50:09 -04:00
Nia Weiss
a74eda1eb0
cleanup of key translation fix 2020-10-22 10:50:08 -04:00
Nia Weiss
c60c699664
deal with linter false positives 2020-10-22 10:47:30 -04:00
Nia Weiss
da52b134bf
the tests pass now 2020-10-22 10:47:30 -04:00
Nia Weiss
5315579378
add some validation 2020-10-22 10:47:29 -04:00
Nia Weiss
1fe0edae08
fix find & create foreign index keys 2020-10-22 10:47:29 -04:00
Nia Weiss
a6e3723190
it mostly works now 2020-10-22 10:47:29 -04:00
Nia Weiss
13a19aeb42
query translation WIP 2020-10-22 10:47:29 -04:00
Nia Weiss
15036787a7
test new translation paths via API 2020-10-22 10:47:29 -04:00
Nia Weiss
8a130c150e
address review comments 2020-10-22 10:47:28 -04:00
Nia Weiss
22d6011d05
apply "maybe" key translation WIP 2020-10-22 10:47:25 -04:00
jaten-molecula
d12639b890
Merge pull request #1014 from jaten-molecula/log_in_utc
log in UTC in fixed width microseconds RFC3339 format
2020-10-21 20:01:36 -05:00
Jason E. Aten
aad38d1c60 log in UTC in fixed width microseconds 2020-10-21 18:36:28 +00:00
seebs
4cea813ae8
Merge pull request #999 from seebs/pqlCleanup
Fixes some minor PQL issues; adds support for case-insensitive PQL calls
2020-10-20 16:02:42 -05:00
jaten-molecula
fb6feb04f9
Merge branch 'master' into pqlCleanup 2020-10-20 15:21:54 -05:00
seebs
a1e1d86ae3
Merge pull request #1012 from seebs/ignoreCN
Supply "subject alternative name" for TLS certificates
2020-10-20 14:50:18 -05:00
Travis Turner
52b91b2df1
Merge branch 'master' into pqlCleanup 2020-10-20 14:19:45 -05:00
Seebs
5e36638e10 Supply "subject alternative name" for TLS certificates
Go killed off using the common name for hostnames starting with 1.15,
but this can be addressed by recreating the certs using a Subject
Alternative Name for the domain for "localhost". This allows tests
to pass without hanging, at least for me.
2020-10-20 14:14:26 -05:00
jaten-molecula
d929c945bc
Merge pull request #1011 from jaten-molecula/rbf_default
rbf is the default Tx type. Dogfood it.
2020-10-20 13:53:21 -05:00
Jason E. Aten
18ad4a8230 rename test-golang-1.14 -> test-golang-1.14.9 2020-10-20 13:37:13 -05:00
Travis Turner
4db93db0c1
Merge branch 'master' into pqlCleanup 2020-10-20 13:19:55 -05:00
Jason E. Aten
275779173d go1.15.3 support with GODEBUG=x509ignoreCN=0 2020-10-20 12:23:35 -05:00
Jason E. Aten
996b2ccb23 go1.13.15 -> go1.15.3 as our supported versions 2020-10-20 11:58:12 -05:00
Jason E. Aten
1216b93d73 rbf is the default Tx type. Dogfood it. 2020-10-20 11:07:07 -05:00
tgruben
c356498c19
Merge pull request #1010 from jaten-molecula/avoid_bg_deadlock
CI catches red blue-green tests. Qcx write flag
2020-10-20 11:01:53 -05:00
Travis Turner
7be3cd6a36
Merge branch 'master' into pqlCleanup 2020-10-20 10:59:48 -05:00
Jason E. Aten
2b4e6d25f5 CI catches red blue-green tests. Qcx write flag
- fix a CI/Makefile issue that was hiding red tests in CI.
 - the testv and testv-race targets now require /bin/bash
 - In executor.go, the top-level query context Qcx now
   has a write flag. It will upgrade read-Tx to write-Tx
   when Store() wraps some inner local-read operations,
   to avoid deadlocking against its own query. This deadlock
   happens in TestExecutor_Execute_SetRow/Set_NewRow
   under rbf_lmdb blue-green testing without the upgrade.
2020-10-20 10:38:40 -05:00
jaten-molecula
0dd3486b95
Merge pull request #1005 from jaten-molecula/bglimits
Fix blue-green Tx cleanup and document single import at once
2020-10-20 07:31:54 -05:00
Jason E. Aten
d9783406bd Fix blue-green Tx cleanup and document single import at once
- correct string constants for txtype so that
  blue-green cleanup correctly detects when
  2nd transaction in a pair has Committed and
  thus the blue-green RWMutex can be relased

- test that txtype.String() is consistent with
  the corresponding string constants.

- document in bluegreentx.go the current limitations
  of blue-green testing: only one github archive import
  (a single writing client) is supported by blue-green
  testing. Multiple importers will deadlock eventually
  on the DBShard.mut RWMutex. We could fix this by
  ordering the write locks and obtaining them in
  strictly increasing order (by shard number), but
  that would require alot of change to the executor
  and that would introduce more risk for a test-only
  pathway.
2020-10-20 07:09:08 -05:00
jaten-molecula
2ebff707a9
Merge pull request #1004 from jaten-molecula/bluegreenlock
introduce a per shard blue-green RWMutex
2020-10-19 18:02:29 -05:00
Jason E. Aten
81013999e5 introduce a per shard blue-green RWMutex
- allows blue-green testing with concurrent readers/writers.
 - otherwise we don't start/end the blue and green Tx
   together, and they get split by a read/write concurrently.
2020-10-19 17:28:35 -05:00
Travis Turner
f37a63e5cf
Merge branch 'master' into pqlCleanup 2020-10-19 15:45:04 -05:00
Cody Soyland
d3ae2614ce
Merge pull request #1001 from seebs/updateLint
uprev golangci-lint, fix a minor lint in rbf
2020-10-19 14:54:38 -05:00
Seebs
ed309821ae uprev golangci-lint, fix a minor lint in rbf
I have a newer staticcheck and golangci-lint on my laptop, and it started
complaining about something. The first comment added disables the check
in staticcheck-as-a-command, the second disables it when it's being done
by golangci-lint, which invokes the analysis passes directly and displays
the output differently, and also doesn't recognize the hints used by
staticcheck.

Newer golangci-lint doesn't find anything else that it wants to complain
about.
2020-10-19 14:25:11 -05:00
Seebs
6e725d50ce check error return from peg parser Init
The updated peg tool produces an Init that can return an error. As
of this writing, the error can't be non-nil unless you specified an
option which itself returned an error, but that could change later,
so let's be careful.
2020-10-19 13:47:42 -05:00
Seebs
e84d2d2a59 refactor number parsing a bit
There are subtle inconsistencies, like "01" being a valid decimal but not
a valid integer, which vaguely bug me. Cleaning this up, and the corresponding
parser logic.

A number can't have leading spaces because the grammar doesn't
put spaces in them in the first place, so stop accepting them in the
number syntax. This should never have any impact on anything,
it's just simpler.

Update a couple of test cases to reflect this -- no longer testing
that trailing spaces are okay, now testing that they're not, for
instance.
2020-10-19 13:37:21 -05:00
Seebs
c19d571112 more consistent spacing
We still prohibit a space before a leading '(', which maybe we shouldn't,
but we now allow spaces on both sides of a closing ')' more consistently.
Drop the unneeded "sp" before "close" in the special handling after
null, true, and false, because close now implies that.

Also, refactored the two instances of "sp '=' sp" into a thing called eq,
which may not be worth it.
2020-10-19 13:35:49 -05:00
Seebs
28d18920b9 Make PQL case-insensitive about call names.
Use "" strings for fixed string names. In startCall(), look up the
lowercase conversion of a call name in a table mapping all-lowercase
representations to canonical case, so we don't have to chase down
everyplace in the rest of the code base that assumes "Row" is
capitalized exactly like that.
2020-10-19 13:35:49 -05:00
Seebs
18ed02d7fb drop float/decimal distinction in PQL
PQL always produces decimals, which have effectively-arbitrary range,
but can convert them to floats when required; the executor then requests
this conversion in the handful of cases (SetRowAttrs and SetColumnAttrs)
where it wants floats rather than decimals.

Not yet fixed: The "Range" call may also be wrong now. It was specifying
an "fvalue" but is now effectively getting what used to be called a
"dvalue". However, so far as I can tell, that didn't work before either.
2020-10-19 13:35:49 -05:00