track field directly in view to prevent deadlocks

The central reason this exists:

**sync.RWMutex can block read locks even when no write lock is yet held.**

If a write lock is *requested*, this can block future read locks. In
particular, this means that recursive read locks are unsafe. But there's
additional problems.

The specific case that bit us involves not two, but *three* things
running at once.

Thing #1: executor doing AvailableShards. This RLocks the index, and
then each field, and then each view. To complete, it must be able to
obtain a read lock on each view in turn.

Thing #2: DeleteField. This Locks the index. Even if it is stuck
waiting for the lock (which it will be until AvailableShards completes),
it can prevent *additional* RLocks of the index.

Thing #3: CreateFragment. This Locks a view, then RLocks the index in
order to look up a field.

CreateFragment can't proceed until DeleteField completes. DeleteField
can't proceed until AvailableShards completes. And AvailableShards
can't proceed until CreateFragment completes.

Solution: Cache the *Field in the view, so we don't need a read lock
on the field or index to complete a CreateFragment.
This commit is contained in:
Seebs 2022-01-13 10:57:12 -06:00 • committed by Matthew Jaffee
parent 4a1e53421e
commit 6fba8aba8b
2 changed files with 3 additions and 1 deletions

View file

@ -1032,6 +1032,7 @@ func (f *Field) createViewIfNotExistsBase(cvm *CreateViewMessage) (*view, bool,
func (f *Field) newView(path, name string) *view {
view := newView(f.holder, path, f.index, f.name, name, f.options)
view.idx = f.idx
view.fld = f
view.stats = f.Stats
view.broadcaster = f.broadcaster
return view

View file

@ -40,6 +40,7 @@ type view struct {
holder *Holder
idx *Index
fld *Field
fieldType string
cacheType string
@ -363,7 +364,7 @@ func (v *view) notifyIfNewShard(shard uint64) {
}
func (v *view) newFragment(shard uint64) *fragment {
fld := v.idx.Field(v.field)
fld := v.fld
spec := fragSpec{
index: v.idx,
field: fld,