Integrate compression into chainexchange topic #819
Assignees
Milestone
Comments
masih
added a commit
that referenced
this issue
Jan 22, 2025
The GPBFT message exchange over pubsub already uses zstd compression on top of CBOR encoded messages. The work here integrates the same style of compression for chain exchange messages, with additional unification of the encoding mechanism across the two. The work refactors the root level encoding implementation into a generic encoder decoder that both chain exchange and gpbft used. Tests and benchmarks are updated to reflect this. The benchmarking of partial gmessage encoding is also adjusted to fix a few redundant statements and bugs in testing. Fixes #819
masih
added a commit
that referenced
this issue
Jan 24, 2025
The GPBFT message exchange over pubsub already uses zstd compression on top of CBOR encoded messages. The work here integrates the same style of compression for chain exchange messages, with additional unification of the encoding mechanism across the two. The work refactors the root level encoding implementation into a generic encoder decoder that both chain exchange and gpbft used. Tests and benchmarks are updated to reflect this. The benchmarking of partial gmessage encoding is also adjusted to fix a few redundant statements and bugs in testing. Fixes #819
github-merge-queue bot
pushed a commit
that referenced
this issue
Jan 27, 2025
* Integrate zstd compression into chain exchange The GPBFT message exchange over pubsub already uses zstd compression on top of CBOR encoded messages. The work here integrates the same style of compression for chain exchange messages, with additional unification of the encoding mechanism across the two. The work refactors the root level encoding implementation into a generic encoder decoder that both chain exchange and gpbft used. Tests and benchmarks are updated to reflect this. The benchmarking of partial gmessage encoding is also adjusted to fix a few redundant statements and bugs in testing. Fixes #819 * Strictly Limit the size of decompressed values to 1 MiB The default message size limit in GossipSub is 1 MiB, which is unchanged in Lotus. This means when decompressing values, we can never have a valid compressed message that expands to larger than 1 MiB. Set this limit explicitly in the zstd decoder. * Massage the flaky test to submission
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add compression to messages published on chainexchange topic.
The text was updated successfully, but these errors were encountered: