[Merged by Bors] - Pre-allocate vectors in SSZ decoding #3417
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
michaelsproul
added
work-in-progress
|
Running this for 24h on two similar Ropsten nodes didn't yield any significant change. If anything the vanilla v2.5.1 (light blue) code seems to be using slightly less free memory than this PR (dark blue)
I'm on the fence about whether we should bother merging it. On one hand explicit is better than implicit, so maybe we should be explicit about the allocation behaviour we want here rather than relying on the optimiser or the allocator to smooth it out for us (i.e. merge). On the other hand, if it ain't broke maybe we shouldn't meddle with it unnecessarily. |
michaelsproul
added
ready-for-review
realbigsean
reviewed
Sep 13, 2022
realbigsean
added
waiting-on-author
michaelsproul
added
ready-for-review
realbigsean
approved these changes
Sep 15, 2022
realbigsean
added
ready-for-merge
|
bors r+ |
bors bot
pushed a commit
that referenced
this pull request
Sep 16, 2022
## Issue Addressed Fixes a potential regression in memory fragmentation identified by @paulhauner here: #3371 (comment). ## Proposed Changes Immediately allocate a vector with sufficient size to hold all decoded elements in SSZ decoding. The `size_hint` is derived from the range iterator here: https://github.com/sigp/lighthouse/blob/2983235650811437b44199f9c94e517e948a1e9b/consensus/ssz/src/decode/impls.rs#L489 ## Additional Info I'd like to test this out on some infra for a substantial duration to see if it affects total fragmentation.
|
Build failed (retrying...): |
bors bot
pushed a commit
that referenced
this pull request
Sep 16, 2022
## Issue Addressed Fixes a potential regression in memory fragmentation identified by @paulhauner here: #3371 (comment). ## Proposed Changes Immediately allocate a vector with sufficient size to hold all decoded elements in SSZ decoding. The `size_hint` is derived from the range iterator here: https://github.com/sigp/lighthouse/blob/2983235650811437b44199f9c94e517e948a1e9b/consensus/ssz/src/decode/impls.rs#L489 ## Additional Info I'd like to test this out on some infra for a substantial duration to see if it affects total fragmentation.
|
This PR was included in a batch that was canceled, it will be automatically retried |
|
bors r- |
|
Canceled. |
|
bors r+ |
|
bors r- |
|
Canceled. |
|
bors r+ |
bors bot
pushed a commit
that referenced
this pull request
Sep 16, 2022
## Issue Addressed Fixes a potential regression in memory fragmentation identified by @paulhauner here: #3371 (comment). ## Proposed Changes Immediately allocate a vector with sufficient size to hold all decoded elements in SSZ decoding. The `size_hint` is derived from the range iterator here: https://github.com/sigp/lighthouse/blob/2983235650811437b44199f9c94e517e948a1e9b/consensus/ssz/src/decode/impls.rs#L489 ## Additional Info I'd like to test this out on some infra for a substantial duration to see if it affects total fragmentation.
|
Build failed (retrying...): |
bors bot
pushed a commit
that referenced
this pull request
Sep 16, 2022
## Issue Addressed Fixes a potential regression in memory fragmentation identified by @paulhauner here: #3371 (comment). ## Proposed Changes Immediately allocate a vector with sufficient size to hold all decoded elements in SSZ decoding. The `size_hint` is derived from the range iterator here: https://github.com/sigp/lighthouse/blob/2983235650811437b44199f9c94e517e948a1e9b/consensus/ssz/src/decode/impls.rs#L489 ## Additional Info I'd like to test this out on some infra for a substantial duration to see if it affects total fragmentation.
|
Build failed (retrying...): |
bors bot
pushed a commit
that referenced
this pull request
Sep 16, 2022
## Issue Addressed Fixes a potential regression in memory fragmentation identified by @paulhauner here: #3371 (comment). ## Proposed Changes Immediately allocate a vector with sufficient size to hold all decoded elements in SSZ decoding. The `size_hint` is derived from the range iterator here: https://github.com/sigp/lighthouse/blob/2983235650811437b44199f9c94e517e948a1e9b/consensus/ssz/src/decode/impls.rs#L489 ## Additional Info I'd like to test this out on some infra for a substantial duration to see if it affects total fragmentation.
bors
bot
changed the title
Woodpile37
pushed a commit
to Woodpile37/lighthouse
that referenced
this pull request
Jan 6, 2024
## Issue Addressed Fixes a potential regression in memory fragmentation identified by @paulhauner here: sigp#3371 (comment). ## Proposed Changes Immediately allocate a vector with sufficient size to hold all decoded elements in SSZ decoding. The `size_hint` is derived from the range iterator here: https://github.com/sigp/lighthouse/blob/2983235650811437b44199f9c94e517e948a1e9b/consensus/ssz/src/decode/impls.rs#L489 ## Additional Info I'd like to test this out on some infra for a substantial duration to see if it affects total fragmentation.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Issue Addressed
Fixes a potential regression in memory fragmentation identified by @paulhauner here: #3371 (comment).
Proposed Changes
Immediately allocate a vector with sufficient size to hold all decoded elements in SSZ decoding. The
size_hintis derived from the range iterator here:lighthouse/consensus/ssz/src/decode/impls.rs
Line 489 in 2983235
Additional Info
I'd like to test this out on some infra for a substantial duration to see if it affects total fragmentation.