Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Flat db healing failed with MerkleTrieException on 24.12.0-RC2 #8015

Open
jframe opened this issue Dec 11, 2024 · 4 comments
Open

Flat db healing failed with MerkleTrieException on 24.12.0-RC2 #8015

jframe opened this issue Dec 11, 2024 · 4 comments
Labels
bonsai bug Something isn't working P3 Medium (ex: JSON-RPC request not working with a specific client library due to loose spec assumtion) syncing

Comments

@jframe
Copy link
Contributor

jframe commented Dec 11, 2024

During testing of 24.12.0-RC2, one node failed the sync during the flat db healing step with a MerkleTrieException. There are no other exceptions in the log.

{"@timestamp":"2024-12-09T09:43:38,974","level":"INFO","thread":"EthScheduler-Services-34 (batchHealAndPersistFlatStorageData)","class":"Pipeline","message":"Unexpected exception in pipeline. Aborting.
Throwable summary: org.hyperledger.besu.ethereum.trie.MerkleTrieException: Unable to load trie node value for hash 0x7c645ffe7048d604cc92a50f9e033b98e05ff48fe07e6d0223abf58fe274d8ce location 0x040005
	at: org.hyperledger.besu.ethereum.trie.StoredNode.lambda$load$0(StoredNode.java:135)","throwable":""}
@jframe
Copy link
Contributor Author

jframe commented Dec 11, 2024

dev-elc-bu-nb-sepolia-burn-snap-24.12.0-RC2_besu.log

@jframe
Copy link
Contributor Author

jframe commented Dec 11, 2024

I've taken a snapshot of node so we can inspect the database if needed

@macfarla macfarla added the bug Something isn't working label Jan 6, 2025
@macfarla
Copy link
Contributor

macfarla commented Jan 6, 2025

mitigation here would be to have a fallback to restart the pipeline if this error occurs

@macfarla macfarla added P2 High (ex: Degrading performance issues, unexpected behavior of core features (DevP2P, syncing, etc)) P3 Medium (ex: JSON-RPC request not working with a specific client library due to loose spec assumtion) and removed P2 High (ex: Degrading performance issues, unexpected behavior of core features (DevP2P, syncing, etc)) labels Jan 6, 2025
@macfarla macfarla mentioned this issue Jan 6, 2025
Open
@macfarla
Copy link
Contributor

macfarla commented Jan 6, 2025

Created #8081 for the mitigation and leaving this issue as P3 for the root cause investigation

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bonsai bug Something isn't working P3 Medium (ex: JSON-RPC request not working with a specific client library due to loose spec assumtion) syncing
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@jframe @macfarla