Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Library is not flexible enough to handle Parity's custom compression #111

Open
lithp opened this issue Mar 26, 2019 · 0 comments
Open

Library is not flexible enough to handle Parity's custom compression #111

lithp opened this issue Mar 26, 2019 · 0 comments
Assignees

Comments

@lithp
Copy link

lithp commented Mar 26, 2019

Parity compresses and decompresses the rlp objects which it writes into the database. It has a hand-selected list of substitutions which it applies to strings (not lists). Attempting to rlp.decode one of the compressed objects results in rlp.exceptions.DecodingErrors. This is a good thing, because the compressed objects aren't valid rlp!

However, because of the way in which pyrlp is written it is very difficult to use pyrlp for reading these objects. consume_payment tries to recurse into rlp objects and doesn't provide any extension points for swapping out \x81\x02 (an example of an invalid sequence which appears in the compressed object) with the decompressed value. consume_length_prefix isn't usable either because it throws an exception on these objects, in order to read these objects I needed to implement my own.

I think this could be fixed by moving some exceptions around and breaking up a few functions such that users could consume just the logic they needed.

@lithp lithp self-assigned this Mar 26, 2019
@lithp lithp changed the title Library is not flexible to handle Parity's custom compression Library is not flexible enough to handle Parity's custom compression Mar 26, 2019
qoire added a commit to qoire/bcdbr that referenced this issue Jun 3, 2019
pivot away from rocksdb and parity due to issues listed in:
ethereum/pyrlp#111, for now fallback
to go-ethereum implementation.

New implementations:

* block_header decoding
* hash functions to generate keys for geth state
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant