js-fil-utils (npm: @rvagg/fil-utils)

Miscellaneous JavaScript Filecoin utilities implementing pieces of the Filecoin proofs based on the Rust implementations at https://github.com/filecoin-project/rust-fil-proofs/

• CommP calculation, using the following components which may be used separately:
  • Padded and piece size calculation
  • Zero padding stream conversion
  • Fr32 padding stream conversion
  • 32-byte chunked sha256 binary Merkle tree calculation

API

Contents

• async commp(stream)
• commp.fr32PadReader(stream)
• async commp.merkleRoot(stream)
• commp.pieceSizeFromRaw(size)
• commp.zeroPaddedSizeFromRaw(size)
• commp.zeroPadReader(stream)

async commp(stream)

Given an input stream, calculate the Filecoin CommP ("Piece Commitment") using stream processing.

Parameters:

• stream (Stream|AsyncIterator.<Buffer>)

Return value (Object): an object of the form { size, paddedSize, pieceSize, commp }

commp.fr32PadReader(stream)

Given a stream or async iterator (of Buffers), return a new async iterator that adds Fr32 padding. For every 254 bits, an additional 2 zero bits are added.

Parameters:

• stream (Stream|AsyncIterator.<Buffer>): a stream or async iterator of the original source, zero-padded to an appropriate length to fit in a pow2 sized piece after Fr32 padding.

Return value (AsyncIterator.<Buffer>)

async commp.merkleRoot(stream)

Given a stream or async iterator (of Buffers), return a merkle root as a Buffer using a 32-byte chunked sha256 binary tree.

Parameters:

• stream (Stream|AsyncIterator.<Buffer>): a stream or async iterator of raw bytes. The byte length is expected to be divisible by 64 (pairs of 32-bytes).

Return value (Buffer)

commp.pieceSizeFromRaw(size)

Determine the piece size for a given block of data. Does not account for Fr32 padding. A simple rounding up to the next pow2 size.

Parameters:

• size (number): the size of the original resource
• next (boolean): bump to the next pow2 piece size

Return value (number)

commp.zeroPaddedSizeFromRaw(size)

Determine the additional bytes of zeroed padding to append to the end of a resource of size length in order to fit within a pow2 piece while leaving enough room for Fr32 padding (2 bits per 254).

Parameters:

• size (number): the size of the original resource

Return value (number)

commp.zeroPadReader(stream)

Given a stream or async iterator (of Buffers), return a new async iterator that additional zero-padding at the end of an amount appropriate to fit in a pow2 Fr32 padded piece. See also {@zeroPaddedSize}.

Parameters:

• stream (Stream|AsyncIterator.<Buffer>): a stream or async iterator of the original source.
• size (number): the expected size, in bytes, of the original source.

Return value (AsyncIterator.<Buffer>)

License and Copyright

Copyright 2020 Rod Vagg

Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.