Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Adding more digest algorithm #78

Open
alvinsw opened this issue Aug 29, 2024 · 3 comments
Open

Adding more digest algorithm #78

alvinsw opened this issue Aug 29, 2024 · 3 comments

Comments

@alvinsw
Copy link

alvinsw commented Aug 29, 2024

I could not find any discussion or PR proposing the extension 0009, so I kind of missed it.
There is a need to pre-calculate CRC32 checksum for all the content files and I think the best way to do that is to add the checksums to the fixity section. In addition to the checksum, we also need the file size.
Can we revise extension 0009 so that it include both CRC32 and CRC32+size combination?

@rosy1280
Copy link
Contributor

@alvinsw ext 0009 already allows users to use the file size as a fixity algorithm in the fixity block. It's labeled as size in the table in ext 0009.

You are also permitted to list multiple algorithms in the fixity block. There is an example in section 3.5.4 of the specification that shows how you would do this.

Given that file size is already an option and that you are permitted to list more than one algorithm in the fixity block, it sounds like all we would need to do is add CRC32 as a fixity algorithm to ext 0009 to solve your use case. Is that correct?

@zimeon
Copy link
Contributor

zimeon commented Aug 29, 2024

History: Issue to add size #64 that lead to PR #65

I note that if we "update" 0009 we create a new extension that obsoletes 0009, as 0009 did for 0001.

@alvinsw - Can you say a bit more about your application of CRC32? Also, since there are many versions of the CRC checks, can you link to the specification of the one you propose for CRC32

@alvinsw
Copy link
Author

alvinsw commented Sep 3, 2024

@rosy1280 : Yes, that is correct . The size is already there, so we just need to add the crc32.

@zimeon : The use case that I encountered at the moment is to zip files dynamically using NGINX mod zip. The module requires the crc32 checksum to be known before the operation in order to support the Range header. I believe the best way to pre-calculate and store the checksum is to put them in the inventory.json. Yes, there are many versions of CRC so that's kind of problematic. Do we want to support all version of CRC? I could not find the official specification of the CRC, but the one that mod zip needs is the same as the one used by String::CRC32 perl module.
Do you know if there an authoritative site that listed all the crc variants spec?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants