-
Notifications
You must be signed in to change notification settings - Fork 1.2k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
fix: update maxHeaderSize #1800
Conversation
@jarifibrahim @manishrjain @MichelDiz PTAL, thanks. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The change looks good but I think this might break existing badger DBs because we use this value to determine where to write the data.
We'll need to bump the magicVersion
over here to prevent existing databases from breaking https://github.com/dgraph-io/badger/blob/2de177900e3ced14fdab91aa7c6a7e3e441f0192/manifest.go#L248
Also, once this PR is merged, we'll need a major version update.
e87c856
to
c09abc0
Compare
c09abc0
to
248f0b6
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks for the contribution. This does indeed look like a bug which could cause a panic in
func (h header) Encode(out []byte) int {
out[0], out[1] = h.meta, h.userMeta
index := 2
index += binary.PutUvarint(out[index:], uint64(h.klen))
index += binary.PutUvarint(out[index:], uint64(h.vlen))
index += binary.PutUvarint(out[index:], h.expiresAt)
return index
}
since the size of the out
buffer is allocated based on this parameter. We haven't seen this issue before since it would require each varInt encoded field to be its maximum size, but it certainly could happen. This doesn't require a magic number upgrade since this is fixing an edge case.
@joshua-goldstein could you write a test for it so that in case something changes, the test fails. We also should figure out what are the cases when the header size could become 22. |
Based on https://pkg.go.dev/encoding/binary#pkg-constants, the maxHeaderSize should be 22=5+5+10+1+1. Signed-off-by: Liang Zheng <[email protected]>
248f0b6
to
f7643cb
Compare
I am closing this PR in lieu of #1877. Once again, thank you for your contribution |
Based on pkg-constants, the
maxHeaderSize
should be 22=5+5+10+1+1, which is discussed on Why is themaxHeaderSize
21 and not 22?Signed-off-by: Liang Zheng [email protected]