featurebase/client
Seebs 2052bb01d8 refactor testing to share clusters more often
When doing tests, we create a ton of one-off clusters. This
turns out to be expensive and slow. Fixing it is surprisingly hard.

Fundamentally: If we're sharing clusters, we need to use different
indexes for each test, to avoid clashes. This changes index names.
As a side-effect, this reorders many partition-based things, like
the order keys are returned in. Thus, to fix this, we change a lot
of tests to no longer depend on the *order* in which strings are
returned.

Having done that, we can also discard the ModHasher behavior, since
that only existed to allow us to reliably predict partitioning.

The basic design is as follows: Instead of a cluster being a
[]*Command, a "shareable" cluster is now a []*Command plus some
flags, and a "cluster" is a pointer to a possibly-shared cluster,
plus a link to the specific test using this specific cluster,
and correspondingly, its test name suitably coerced to be a valid
index name prefix.

The "test.Cluster" object now has methods to allow retrieving an
index name, and also implemnts fmt.Formatter to let you use,
e.g., `%i` with it in Sprintf to get "the index name, plus an i".
(This works for everything but %p and %T.)

This allows us to consistently rework all the many things that
use index names in a persistent way.

We also have `MustUnshared` and `MustRunUnsharedCluster` methods
which allow us to specify that a given test needs its own cluster
for some reason. For instance, the tests that want to run backups
need their own isolated cluster, and the tests that want to close
or reopen nodes need their own cluster because a reopened cluster
won't have working GRPC for some reason.

On "closing" a shared cluster (actually the test-specific wrapper
that reflects a given sharing), we delete any indexes starting with
that test's index name prefix. Otherwise, the huge pile of open
indexes prevents `go test -race` from working on MacOS, where we
run out of address space too quickly.

This is fairly enormous but most of the individual changes are
fairly trivial things like replacing the string "i" with "c.Idx()".

We also tweaked a test that failed for me a couple of times to
not depend on sort order.
2022-09-02 11:40:37 -05:00
..
csv Upgrade go.mod to featurebase/v3 2022-01-21 10:57:05 -07:00
docs remove attributes 2021-05-14 10:28:08 -04:00
egpool Upgrade go.mod to featurebase/v3 2022-01-21 10:57:05 -07:00
batch.go [FB-1587] crash would occur if missing nil-entry in batch (#2145) 2022-07-12 17:20:55 -05:00
batch_test.go add timestamp nils to nullIndices (#2095) 2022-06-02 15:54:24 -05:00
client.go [FB-1024 FB-1590] Increase timerange (un-revert) (#2174) 2022-08-04 15:20:45 -07:00
client_it_test.go refactor testing to share clusters more often 2022-09-02 11:40:37 -05:00
client_test.go Upgrade go.mod to featurebase/v3 2022-01-21 10:57:05 -07:00
cluster.go client side retry ingestAPI requests on primary host 2022-03-10 10:22:13 -06:00
cluster_test.go client side retry ingestAPI requests on primary host 2022-03-10 10:22:13 -06:00
doc.go Upgrade go.mod to featurebase/v3 2022-01-21 10:57:05 -07:00
error.go client side retry ingestAPI requests on primary host 2022-03-10 10:22:13 -06:00
ingest_api_batch.go Upgrade go.mod to featurebase/v3 2022-01-21 10:57:05 -07:00
ingest_api_batch_test.go client side retry ingestAPI requests on primary host 2022-03-10 10:22:13 -06:00
logimport.go add copyright notice back in 2021-12-10 11:01:04 -06:00
logimport_test.go add copyright notice back in 2021-12-10 11:01:04 -06:00
metrics.go "all bitmap" multi-field, single-shard ingest 2022-05-27 11:25:17 -05:00
orm.go don't encode 'base' in the field options json 2022-05-29 09:03:27 -05:00
orm_test.go Revert "FB-1491: Revert pql.Decimal commit for the 4.8.5 release (#2117)" 2022-06-17 14:58:30 -05:00
README.md remove trailing spaces en masse (#2024) 2022-04-18 09:31:35 -05:00
record.go add copyright notice back in 2021-12-10 11:01:04 -06:00
record_test.go Upgrade go.mod to featurebase/v3 2022-01-21 10:57:05 -07:00
response.go Upgrade go.mod to featurebase/v3 2022-01-21 10:57:05 -07:00
response_test.go Upgrade go.mod to featurebase/v3 2022-01-21 10:57:05 -07:00
shardnodes.go Upgrade go.mod to featurebase/v3 2022-01-21 10:57:05 -07:00
tracer.go add copyright notice back in 2021-12-10 11:01:04 -06:00
validate.go add copyright notice back in 2021-12-10 11:01:04 -06:00
validate_test.go add copyright notice back in 2021-12-10 11:01:04 -06:00
version.go add copyright notice back in 2021-12-10 11:01:04 -06:00

Go Client for Pilosa

Go client for Pilosa high performance distributed index.

Usage

If you have the pilosa repo in your GOPATH, you can import the library in your code using:

import "github.com/pilosa/pilosa/v2/client"

Quick overview

Assuming Pilosa server is running at localhost:10101 (the default):

package main

import (
	"fmt"

	"github.com/pilosa/pilosa/v2/client"
)

func main() {
	// Create the default client
	cli := client.DefaultClient()

	// Retrieve the schema
	schema, err := cli.Schema()

	// Create an Index object
	myindex := schema.Index("myindex")

	// Create a Field object
	myfield := myindex.Field("myfield")

	// make sure the index and the field exists on the server
	err := cli.SyncSchema(schema)

	// Send a Set query. If err is non-nil, response will be nil.
	response, err := cli.Query(myfield.Set(5, 42))

	// Send a Row query. If err is non-nil, response will be nil.
	response, err = cli.Query(myfield.Row(5))

	// Get the result
	result := response.Result()
	// Act on the result
	if result != nil {
		columns := result.Row().Columns
		fmt.Println("Got columns: ", columns)
	}

	// You can batch queries to improve throughput
	response, err = cli.Query(myindex.BatchQuery(
		myfield.Row(5),
		myfield.Row(10)))
	if err != nil {
		fmt.Println(err)
	}

	for _, result := range response.Results() {
		// Act on the result
		fmt.Println(result.Row().Columns)
	}
}

Documentation

Data Model and Queries

See: Data Model and Queries

Executing Queries

See: Server Interaction

Other Documentation