Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use msgpack for storage instead of gob #52

Merged
merged 4 commits into from
Jul 13, 2020

Conversation

patrick-ogrady
Copy link
Contributor

@patrick-ogrady patrick-ogrady commented Jul 13, 2020

Fixes #49

Motivation

The gob package requires all types to be registered. When Rosetta API implementations populate metadata (on any struct) with non-primitive values, it caused gob to throw an error:

gob: type not registered for interface: *

Solution

Instead of registering all non-primitive types we could expect to see in Metadata (map[string]interface{}, map[int]interface{}, []interface{}, etc), the internal/storage package has been modified to store data as msgpack using this package.

Not only will this change resolve #49, it also improves performance (compared to gob). Alternatively, we could have replaced gob with json but this would've caused significant performance degradation.

NOTE: THIS CHANGE BREAKS ALL EXISTING STATE STORAGE! YOU WILL NEED TO CREATE A NEW DATASTORE WHEN UPDATING THE ROSETTA-CLI!

@coveralls
Copy link

coveralls commented Jul 13, 2020

Pull Request Test Coverage Report for Build 1420

  • 18 of 18 (100.0%) changed or added relevant lines in 1 file are covered.
  • No unchanged relevant lines lost coverage.
  • Overall coverage increased (+0.5%) to 75.385%

Totals Coverage Status
Change from base Build 1385: 0.5%
Covered Lines: 392
Relevant Lines: 520

💛 - Coveralls

@patrick-ogrady patrick-ogrady changed the title [WIP] Fix Gob Encoding Error Use JSON for storage instead of gob Jul 13, 2020
@patrick-ogrady patrick-ogrady force-pushed the patrick/fix-gob-encoding-issue branch from 9db935a to ff6bbd4 Compare July 13, 2020 18:28
@patrick-ogrady patrick-ogrady changed the title Use JSON for storage instead of gob Use msgpack for storage instead of gob Jul 13, 2020
@patrick-ogrady patrick-ogrady requested review from niallo, juliankoh and a team July 13, 2020 18:47
@patrick-ogrady patrick-ogrady merged commit 7856d74 into master Jul 13, 2020
@patrick-ogrady patrick-ogrady deleted the patrick/fix-gob-encoding-issue branch July 13, 2020 18:50
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Development

Successfully merging this pull request may close these issues.

Error when supplying AccountIdentifier's metadata field
3 participants