Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

remove no-LMDB fallback from cosmic-swingset #3131

Closed
warner opened this issue May 19, 2021 · 2 comments · Fixed by #3211
Closed

remove no-LMDB fallback from cosmic-swingset #3131

warner opened this issue May 19, 2021 · 2 comments · Fixed by #3211
Assignees
Labels
cosmic-swingset package: cosmic-swingset SwingSet package: SwingSet

Comments

@warner
Copy link
Member

warner commented May 19, 2021

cosmic-swingset currently accomodates platforms which do not support LMDB (https://github.com/Agoric/agoric-sdk/blob/1de9ee6101b0a33b3f0fca853bf02d8a446cbe5a/packages/cosmic-swingset/src/check-lmdb.js) by falling back to the "jsonlines" persistence mode of "swing-store-simple" (which otherwise keeps all of its state in RAM, in one big Map). In this mode, commit() serializes the entire Map and writes it to a file, and open() reads that entire file back into memory. Obviously, this only performs acceptably for toy examples or unit tests, but it's the one we started with, and we didn't remove it after getting LMDB.

I think we added this fallback when we identified LMDB problems on WSL, but I also think we subsequently found a better workaround (tell LMDB to not use sparse files or mmap()).

We're adding (#3065, #3122) a feature to the kernelDB that moves large transcript entries out of the key-value store and into linear auxilliary files. The LMDB implementation is the most important one, and we're adding a comparable API to the in-RAM non-persistent flavor of swing-store-simple. But we don't want to implement persistence for the new API, and in fact we intend to get rid of the persistent mode altogether.

So the task is to change cosmic-swingset to remove this fallback to swing-store-simple. If LMDB doesn't work, cosmic-swingset cannot be run on that platform (since it couldn't be run remotely efficiently there anyways).

Originally posted by @warner in #3122 (comment)

@warner warner added the cosmic-swingset package: cosmic-swingset label May 19, 2021
@warner warner added the SwingSet package: SwingSet label May 19, 2021
@michaelfig
Copy link
Member

I think we added this fallback when we identified LMDB problems on WSL, but I also think we subsequently found a better workaround (tell LMDB to not use sparse files or mmap()).

I find it funny that the #950 discussion that is linked just above the fallback code in question definitively resolves the problem for which this fallback was built!

Good thing we have github as memory. I wasn't aware of the LMDB workaround in #985, and neither @warner nor @FUDCo seemed to know that #985 meant the current issue's fallback can be removed.

@warner
Copy link
Member Author

warner commented May 19, 2021

Heh, yeah. I think we wrote the "use simplestore" workaround as a quick hack while @FUDCo was locked in a cave diagnosing the mmap problem, and we wound up with both, and then moved on to the next fire without removing the redundant workaround.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
cosmic-swingset package: cosmic-swingset SwingSet package: SwingSet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants