Reduce heap memory usage

[wojas]

Several changes to reduce the maximum heap memory usage of LS.

The biggest change is to switch from a generated gogo-protobuf parser to a custom parser. This new parser uses a light low-level abstraction of the protobuf wire format to avoid allocating a large []KV slice with all items. Every KV item in this slice used 64 bytes, plus reallocation overhead as items are appended, plus (in 0.3.0) the copied key and value data as individual allocs, plus GC overhead. Instead, we now directly parse the protobuf wire data as we loop over the items one by one. We also take snapshots directly to protobuf wire format, without an intermediate slice representation.

We now limit the number of snapshots that can concurrently be held in memory using the new memory_downloaded_snapshots and memory_decompressed_snapshots configuration options. In the past this number could grow up to the number of unique instance names.

We now preallocate byte slices that are large enough to hold all the data that we will generate. This avoids many subsequent slice reallocations and copies to grow a slice, doubling the size every time.

Finally, we immediately force a garbage collection cycle as soon as we have released a large block of memory.

In local testing with 1M test domains and 6M records and snapshots from 3 instances, this reduced the peak RSS from over 10 GB (in v0.3.0) to under 3 GB. With more instances, the savings should be even higher with the new limits in place.

To verify that we are writing and reading the protobuf format correctly, this PR adds tests that perform roundtrips with the old gogo-protobuf code, and verify that the implementation is correct. The proto file will contrinue to be updated to reflect any new keys, and gogo can automatically generate code to fill fields with tests data, and methods that can compare all fields before and after a roundtrip.

Several other tests and benchmarks have been added.

[nvaatstra]

LGTM; Excellent optimisations. Only a few minor comments.

[nvaatstra]

Seems to cater to a future expansion to allow for multiple formats, perhaps have debug as its own flag so you can specify the level of detail (normal, debug) and the format (text, some other format)?

[wojas]

The reason for this switch was a bit different.

In 0.3.0, you could use two output formats:

• 'text' is a google protobuf encoding that is reversible to binary, but it makes all the key-values unreadable
• 'debug' shows the all the key-value entries in a more friendly way

Now that we drop the generated protobuf code, we also loose the 'text' encoder'. I decided to just make this the same as 'debug', instead of dropping the 'text' option, because I figured it would be less likely to break anything.

[nvaatstra]

This seems like it will not be reached, but if it ever does it looks like a potential risk of keeping a token without release()-ing it?

[wojas]

There are two ways we can handle calling Close() twice:

1. Ignore a second call.
2. panic()

The advantage of the first is that you can always defer u.Close(), even if you explicitly call it somewhere. Having this option may reduce the chance that you forget to call it in the first place.

The second one us strict about you calling it only once. This may detect incorrect logic in some cases, but actually increases the chance that you forget to call it in some code paths.

In this case I picked option 1, because it seemed like the least bad option.

The bigger issue happens when Close is not called. Perhaps we should add a runtime.SetFinalizer to detect that scenario, and then panic?