IPFS max block size and IPLD GIT

[cloutiertyler]

I'm a little bit confused about how go-ipld-git deals with sharding and block size. This repo seems to make the assumption that Blobs all fit within one block, but isn't it possible that a git Blob is larger than the maximum block size (1 MB I believe)?

[Stebalien]

Kind of; technically, IPLD doesn't specify a maximum block size. You shouldn't make blocks larger than 1MiB (2MiB? I can't recall) because you won't be able to transmit them over bitswap but that's a limitation of bitswap (imposed for security reasons) not IPLD itself.

Unfortunately, there's not much we can do about it without splitting up the blobs and changing all the hashes. You can find some discussion on possible solutions and workarounds here: https://discuss.ipfs.io/t/git-on-ipfs-links-and-references/730

[cloutiertyler]

Ah that makes sense! Thanks for the pointer!

One more question actually. The IPLD format size is meant to be the total size of an object and all of its children recursively summed, correct?

In the case of a Commit the size returned is 42.

go-ipld-git/commit.go

Line 208 in 58f2932

return 42, nil // close enough

Is that because it would be too difficult (impossible actually) to calculate it for a commit given the Git object data format? Does this mean that any merkledag based format that doesn't already have this calculated has to return bogus data for that field?

[Stebalien]

One more question actually. The IPLD format size is meant to be the total size of an object and all of its children recursively summed, correct?

Yes. The size function is a leftover from days gone by and needs to be removed (see ipfs/go-ipld-format#12).

[cloutiertyler]

Awesome, thanks so much for clarifying and for the quick replies!