The internal/ingest and internal/schema endpoints were developed with
intent that they'd be the primary interface new users would work with,
because they were Easy To Use, and did not require any kind of setup,
the counterpoint being that ingest done this way had performance issues
because it ended up with huge amounts of JSON parsing to reformat
things into our native format. But this was understood to be the price
of providing a new-user-friendly JSON ingest experience.
A year later, we have no evidence that it's ever been used. We never
even moved it out of the `/internal` path. It's a lot of very complex
fiddly code and we don't seem to be using it, and at this point, our
anticipation is that if we really need something, we'll use CSV, which
we already have working, or something in the new SQL code. Either way,
we don't seem to be using this.
(cherry picked from commit 16ccbc461a)
also fix a *bunch* of tests that weren't closing the clusters they
created. Cleaned up one test to use t.Run instead of just checking
everything in a loop
This commit removes the previous `MaxShard` tracking and replaces
it with an `Available Shards` set tracking. This allows sparse shard
tracking without implicitly tracking all shards in between.
