Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

1.5.2 breaks serialization for formats like "bincode" #414

Closed
nullchinchilla opened this issue Jul 15, 2024 · 4 comments
Closed

1.5.2 breaks serialization for formats like "bincode" #414

nullchinchilla opened this issue Jul 15, 2024 · 4 comments

Comments

@nullchinchilla
Copy link

nullchinchilla commented Jul 15, 2024

Serializing a Hash produces a bytestring instead of an array in
formats that support bytestrings (like CBOR). Deserialization is
backwards-compatible with the array format.

This is not backwards compatible for formats like "bincode", where it's not possible to first try to deserialize as an array and then try deserialization as a bytestring. Previously serialized and saved Hashes will now fail on deserialization.

This has actually broken a large production system I run that currently must be pegged to 1.5.1 as a result. Please at least make this feature optional, or enable it only for human-readable serde formats like JSON.

@oconnor663
Copy link
Member

I'll revert this change tonight, and then we can consider whether there's a better way to accomplish the same thing. Do you have time to put together a small program that reproduces the break you're seeing?

oconnor663 added a commit that referenced this issue Jul 15, 2024
Changes since 1.5.2:
- Revert the serialization change. It was intended to be backwards
  compatible, but that didn't hold for non-self-describing serialization
  formats like bincode. See #414.
@oconnor663
Copy link
Member

I've published v1.5.3 with the revert. Let me know if you can confirm that it fixes your system.

@nullchinchilla
Copy link
Author

Yes it does fix it!

I think a better way of doing this is probably to check whether the format is human readable, which might be a good proxy for whether it's self-describing.

But I think probably the best way is to release 2.0 instead.

@BurningEnlightenment
Copy link
Collaborator

I think a better way of doing this is probably to check whether the format is human readable, which might be a good proxy for whether it's self-describing.

CBOR, JSONB and MessagePack would be missed by that check, right?

jefferyq2 pushed a commit to jefferyq2/BLAKE3 that referenced this issue Jul 23, 2024
This mostly reverts commits 8416b16 and
dd0afd6.

Changing the serialization of Hash can only be backwards-compatible in
self-describing formats like CBOR. In non-self-describing formats like
bincode, the deserializer has to know in advance which serialization
format was used.

Fixes BLAKE3-team#414.

Reopens BLAKE3-team#412.
jefferyq2 pushed a commit to jefferyq2/BLAKE3 that referenced this issue Jul 23, 2024
Changes since 1.5.2:
- Revert the serialization change. It was intended to be backwards
  compatible, but that didn't hold for non-self-describing serialization
  formats like bincode. See BLAKE3-team#414.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants