enterprise/b: Ensure 64-bit range for btree keys.

The cmp() function used to compare btree keys was using the
trick of comparing unsigned values by coercing the result
of subtraction to a signed type. This is fine as long as
the range of differences never actually exceeds the limits
of the signed type. For instance, with int64, as long as
the magnitude of the difference is under 2^63 or so, it
works reasonably well.

Plain int, however, can be a 32-bit type, at which point
the magnitude of difference needed to break it is only
2^31 or so.

Subtraction and type conversion is enough cheaper than
branches that this is worth preserving, but it's worth
preserving by switching to an explicit int64 for the
operation and return type. This will not actually affect
performance except for people using the btree code on
32-bit machines, so it probably won't ever matter.
Which may also be true of the potential wrong answers,
but "might be slower" is a better risk than "might
crash".
This commit is contained in:
Seebs 2018-11-01 13:56:48 -05:00
parent 3a2c2c227d
commit 456c092930
2 changed files with 3 additions and 3 deletions

View file

@ -75,7 +75,7 @@ type (
// 0 if a == b
// > 0 if a > b
//
Cmp func(a, b uint64) int
Cmp func(a, b uint64) int64
d struct { // data page
c int

View file

@ -23,8 +23,8 @@ import (
"github.com/pilosa/pilosa/roaring"
)
func cmp(a, b uint64) int {
return int(a - b)
func cmp(a, b uint64) int64 {
return int64(a - b)
}
type bTreeContainers struct {