Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fold objects by_sqn continues to fold after the Journal snapshot SQN #465

Open
martinsumner opened this issue Jan 9, 2025 · 1 comment

Comments

@martinsumner
Copy link
Owner

martinsumner commented Jan 9, 2025

Although the fold will not any objects after the Journal snapshot SQN:

IsValidFun =
fun(Bucket, Key, SQN) ->
LedgerKey = leveled_codec:to_objectkey(Bucket, Key, Tag),
CheckSQN =
leveled_penciller:pcl_checksequencenumber(
LedgerSnapshot, LedgerKey, SQN),
% Need to check that we have not folded past the point
% at which the snapshot was taken
(JournalSQN >= SQN) and (CheckSQN == current)
end,

The fold will continue to load batches until the end of the last CDB file in the manifest. This may cover > 100K objects, and so the end of the fold is delayed.

This caused a Riak issue with handoffs where the remainder after the last batch is only returned to the riak_core_handoff_sender after the fold has completed - and if this more than 5 minutes after the last batch was sent, the riak_core_handoff_receiver will have timed out, and the handoff does not now complete. The remainder is not received, and if it is an ownership handoff it has to be started from scratch.

@martinsumner
Copy link
Owner Author

#466

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant