Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ballet: Add Reed-Solomon module #244

Merged
merged 6 commits into from
Sep 11, 2023
Merged

ballet: Add Reed-Solomon module #244

merged 6 commits into from
Sep 11, 2023

Conversation

ptaffet-jump
Copy link
Contributor

@ptaffet-jump ptaffet-jump commented Apr 6, 2023

Adds high performance Reed-Solomon encoding and recovery, currently configured for up to 67 data shreds and 67 parity shreds (inclusive). Tuned for GFNI (~130 Gbps on the common 32/32 encoding case), but also relatively high performance on AVX2 (~50 Gbps for 32/32 encoding). Uses the FD_HAS_GFNI flag. Depends on the SIMD changes in #250, so we probably want to merge that PR first.

@ptaffet-jump ptaffet-jump force-pushed the ptaffet/reedsol branch 2 times, most recently from 9f4eb6c to 204b32a Compare April 13, 2023 18:34
@ptaffet-jump ptaffet-jump changed the base branch from main to ptaffet/simd_bytes April 13, 2023 18:34
@ptaffet-jump ptaffet-jump marked this pull request as ready for review April 13, 2023 18:35
@ripatel-fd ripatel-fd added crypto Cryptography modules turbine labels Apr 13, 2023
@ripatel-fd ripatel-fd added this to the [M1.3] Turbine Replacement milestone Apr 13, 2023
@ptaffet-jump ptaffet-jump changed the title ballet: Add Reed-Solomon Encoder ballet: Add Reed-Solomon Module May 4, 2023
@ptaffet-jump ptaffet-jump changed the title ballet: Add Reed-Solomon Module ballet: Add Reed-Solomon module May 4, 2023
@ptaffet-jump ptaffet-jump force-pushed the ptaffet/simd_bytes branch 2 times, most recently from d380bf9 to 3441519 Compare May 15, 2023 15:46
Base automatically changed from ptaffet/simd_bytes to main May 15, 2023 20:10
@ptaffet-jump ptaffet-jump force-pushed the ptaffet/reedsol branch 2 times, most recently from 60f93fb to ab1620c Compare June 6, 2023 22:13
kbowers-jump and others added 3 commits August 22, 2023 17:44
From review.  Mostly cosmetic stuff.  The most significant change is the
Galois field math implementation can be selected at compile time
independent of build target.  As part of this, fd_reedsol_internal.h and
fd_reedsol_pi.h were merged into an fd_reedsol_private.h.  The rest is
mostly naming tweaks, inclusion tweaks, macro robustness, error naming
strerror functions and the like.
/* Include the constants in one central spot */

#if FD_REEDSOL_ARITH_IMPL==0
FD_IMPORT_BINARY( fd_reedsol_arith_consts_generic_mul, "src/ballet/reedsol/constants/generic_constants.bin" );
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you encode these dependencies into the Makefile

@ptaffet-jump ptaffet-jump added this pull request to the merge queue Sep 11, 2023
Merged via the queue into main with commit 5a2c6f1 Sep 11, 2023
@ptaffet-jump ptaffet-jump deleted the ptaffet/reedsol branch September 11, 2023 20:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
crypto Cryptography modules turbine
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants