Add Zstd/LZ4 support for region files (1.16 edition)

[egg82]

Zstd using zstd-jni: https://github.com/luben/zstd-jni
LZ4 using lz4-java: https://github.com/lz4/lz4-java

Added new command-line option, --forceWrite, which will forcibly write the chunk/flush to disk regardless of whether or not the chunk has actually changed in a --forceUpgrade.
New compression only happens when a chunk is saved, so running a --forceUpgrade AND --forceWrite may be desirable.

New (hidden) config option for the compression method to use.
There's a new option in PaperConfig (compression level) for Zstd which defaults to 6.
There's a new option in PaperConfig (compression level) for Zlib which defaults to 5.
Added a getter to the API to retrieve the current chunk compression method being used.

Since the patch hooks Mojang's versioning system for new compression methods, it
should be internally consistent, easily-updatable, and reversible with a --forceUpgrade AND a --forceWrite.

This will likely conflict with plugins and programs expecting region files to be compressed using the current standard.

Supercedes #2814

TODO

• Test it to make sure it actually works
• Add API getter for current Zstd dict version
• Update to 1.16.4 (why does this happen every time?)

[schoentoon]

Perhaps it'd be an idea to have another look at #4192 around the same time this would get merged, as these two pull requests would likely benefit from each other.

[Titaniumtown]

Bump! Is there going to be more work on this any time soon? I'm really excited for the benefits a patch like this could add! Of course no rush, you guys are doing this for free of course, but I just wanted to check in and see if anything has changed :)

[egg82]

School took priority so it had to wait a bit, but yes there's plans to finish that to-do list.

[Titaniumtown]

@egg82 ok! Keep us posted :)

[mechoriet]

@egg82
maybe a not related question to this pr,
how much more compression/storage benefit will you get compared to extra processing power needed if any?
how would one generate a dict for zstd for minecraft? if there is a example for that, that your willing to share?
context:
looking into making a gamemode with lots of small worlds that could possibly benefit from this to lower storage and/or network traffic

thanks for the response in advance

[Proximyst]

You can generate the dictionary using the guide on another repository of his: https://github.com/egg82/CompressionTests

[egg82]

Closed in favor of #5029