mirror of
https://github.com/featurebasedb/featurebase.git
synced 2026-08-28 10:54:59 +00:00
When doing tests, we create a ton of one-off clusters. This turns out to be expensive and slow. Fixing it is surprisingly hard. Fundamentally: If we're sharing clusters, we need to use different indexes for each test, to avoid clashes. This changes index names. As a side-effect, this reorders many partition-based things, like the order keys are returned in. Thus, to fix this, we change a lot of tests to no longer depend on the *order* in which strings are returned. Having done that, we can also discard the ModHasher behavior, since that only existed to allow us to reliably predict partitioning. The basic design is as follows: Instead of a cluster being a []*Command, a "shareable" cluster is now a []*Command plus some flags, and a "cluster" is a pointer to a possibly-shared cluster, plus a link to the specific test using this specific cluster, and correspondingly, its test name suitably coerced to be a valid index name prefix. The "test.Cluster" object now has methods to allow retrieving an index name, and also implemnts fmt.Formatter to let you use, e.g., `%i` with it in Sprintf to get "the index name, plus an i". (This works for everything but %p and %T.) This allows us to consistently rework all the many things that use index names in a persistent way. We also have `MustUnshared` and `MustRunUnsharedCluster` methods which allow us to specify that a given test needs its own cluster for some reason. For instance, the tests that want to run backups need their own isolated cluster, and the tests that want to close or reopen nodes need their own cluster because a reopened cluster won't have working GRPC for some reason. On "closing" a shared cluster (actually the test-specific wrapper that reflects a given sharing), we delete any indexes starting with that test's index name prefix. Otherwise, the huge pile of open indexes prevents `go test -race` from working on MacOS, where we run out of address space too quickly. This is fairly enormous but most of the individual changes are fairly trivial things like replacing the string "i" with "c.Idx()". We also tweaked a test that failed for me a couple of times to not depend on sort order. |
||
|---|---|---|
| .. | ||
| csv | ||
| docs | ||
| egpool | ||
| batch.go | ||
| batch_test.go | ||
| client.go | ||
| client_it_test.go | ||
| client_test.go | ||
| cluster.go | ||
| cluster_test.go | ||
| doc.go | ||
| error.go | ||
| ingest_api_batch.go | ||
| ingest_api_batch_test.go | ||
| logimport.go | ||
| logimport_test.go | ||
| metrics.go | ||
| orm.go | ||
| orm_test.go | ||
| README.md | ||
| record.go | ||
| record_test.go | ||
| response.go | ||
| response_test.go | ||
| shardnodes.go | ||
| tracer.go | ||
| validate.go | ||
| validate_test.go | ||
| version.go | ||
Go Client for Pilosa
Go client for Pilosa high performance distributed index.
Usage
If you have the pilosa repo in your GOPATH,
you can import the library in your code using:
import "github.com/pilosa/pilosa/v2/client"
Quick overview
Assuming Pilosa server is running at localhost:10101 (the default):
package main
import (
"fmt"
"github.com/pilosa/pilosa/v2/client"
)
func main() {
// Create the default client
cli := client.DefaultClient()
// Retrieve the schema
schema, err := cli.Schema()
// Create an Index object
myindex := schema.Index("myindex")
// Create a Field object
myfield := myindex.Field("myfield")
// make sure the index and the field exists on the server
err := cli.SyncSchema(schema)
// Send a Set query. If err is non-nil, response will be nil.
response, err := cli.Query(myfield.Set(5, 42))
// Send a Row query. If err is non-nil, response will be nil.
response, err = cli.Query(myfield.Row(5))
// Get the result
result := response.Result()
// Act on the result
if result != nil {
columns := result.Row().Columns
fmt.Println("Got columns: ", columns)
}
// You can batch queries to improve throughput
response, err = cli.Query(myindex.BatchQuery(
myfield.Row(5),
myfield.Row(10)))
if err != nil {
fmt.Println(err)
}
for _, result := range response.Results() {
// Act on the result
fmt.Println(result.Row().Columns)
}
}
Documentation
Data Model and Queries
Executing Queries
See: Server Interaction