[Hexagon] Use single allocation to back 2-d arrays #10903
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
To maintain alignment of each individual region, padding may be added to the single allocation. As a result, this PR is dependent on functionality introduced in #10878 |
|
Current CI failures are the expected differences at the C++ side, which are resolved in #10878. No changes expected to be needed for them, but will need to relaunch CI once it lands. |
Currently, each allocation allocates an entire page, so even a relatively small number of allocations can use very large amounts of VTCM. This commit changes calls to `AllocVtcmWorkspace` of shape `[N,M]` from performing `N` allocations of size `M`, to 1 allocation of size `N*M`. Since `N` is usually much smaller than a page, this reduces the total amount of memory required. This is an intermediate step, where the long-term solution is to use static planning for VTCM allocations. This returns the same `void**` type as the static planning eventually will, but avoids excess memory use in the meantime.
Previously, when a single monolithic allocation is used to back a 2-d Hexagon buffer of shape `[nallocs, nbytes_per_allocation]`, the allocation itself is aligned, but each individual region is not. This commit ensures that each individual region also followed the alignment specified.
Lunderberg
force-pushed
the
hexagon_buffer_single_alloc
branch
from
2db1475 to
42e6e85
Compare
|
Rebased onto main to restart CI. |
csullivan
approved these changes
Apr 6, 2022
There was a problem hiding this comment.
This is a great way to handle separate allocations, thanks @Lunderberg
pfk-beta
pushed a commit
to pfk-beta/tvm
that referenced
this pull request
Apr 11, 2022
* [Hexagon] Use single allocation to back 2-d arrays Currently, each allocation allocates an entire page, so even a relatively small number of allocations can use very large amounts of VTCM. This commit changes calls to `AllocVtcmWorkspace` of shape `[N,M]` from performing `N` allocations of size `M`, to 1 allocation of size `N*M`. Since `N` is usually much smaller than a page, this reduces the total amount of memory required. This is an intermediate step, where the long-term solution is to use static planning for VTCM allocations. This returns the same `void**` type as the static planning eventually will, but avoids excess memory use in the meantime. * [Hexagon] Maintain alignment of allocations Previously, when a single monolithic allocation is used to back a 2-d Hexagon buffer of shape `[nallocs, nbytes_per_allocation]`, the allocation itself is aligned, but each individual region is not. This commit ensures that each individual region also followed the alignment specified.
mehrdadh
pushed a commit
to mehrdadh/tvm
that referenced
this pull request
Apr 11, 2022
* [Hexagon] Use single allocation to back 2-d arrays Currently, each allocation allocates an entire page, so even a relatively small number of allocations can use very large amounts of VTCM. This commit changes calls to `AllocVtcmWorkspace` of shape `[N,M]` from performing `N` allocations of size `M`, to 1 allocation of size `N*M`. Since `N` is usually much smaller than a page, this reduces the total amount of memory required. This is an intermediate step, where the long-term solution is to use static planning for VTCM allocations. This returns the same `void**` type as the static planning eventually will, but avoids excess memory use in the meantime. * [Hexagon] Maintain alignment of allocations Previously, when a single monolithic allocation is used to back a 2-d Hexagon buffer of shape `[nallocs, nbytes_per_allocation]`, the allocation itself is aligned, but each individual region is not. This commit ensures that each individual region also followed the alignment specified.
Lucien0
pushed a commit
to Lucien0/tvm
that referenced
this pull request
Apr 19, 2022
* [Hexagon] Use single allocation to back 2-d arrays Currently, each allocation allocates an entire page, so even a relatively small number of allocations can use very large amounts of VTCM. This commit changes calls to `AllocVtcmWorkspace` of shape `[N,M]` from performing `N` allocations of size `M`, to 1 allocation of size `N*M`. Since `N` is usually much smaller than a page, this reduces the total amount of memory required. This is an intermediate step, where the long-term solution is to use static planning for VTCM allocations. This returns the same `void**` type as the static planning eventually will, but avoids excess memory use in the meantime. * [Hexagon] Maintain alignment of allocations Previously, when a single monolithic allocation is used to back a 2-d Hexagon buffer of shape `[nallocs, nbytes_per_allocation]`, the allocation itself is aligned, but each individual region is not. This commit ensures that each individual region also followed the alignment specified.
altanh
pushed a commit
to altanh/tvm
that referenced
this pull request
Apr 28, 2022
* [Hexagon] Use single allocation to back 2-d arrays Currently, each allocation allocates an entire page, so even a relatively small number of allocations can use very large amounts of VTCM. This commit changes calls to `AllocVtcmWorkspace` of shape `[N,M]` from performing `N` allocations of size `M`, to 1 allocation of size `N*M`. Since `N` is usually much smaller than a page, this reduces the total amount of memory required. This is an intermediate step, where the long-term solution is to use static planning for VTCM allocations. This returns the same `void**` type as the static planning eventually will, but avoids excess memory use in the meantime. * [Hexagon] Maintain alignment of allocations Previously, when a single monolithic allocation is used to back a 2-d Hexagon buffer of shape `[nallocs, nbytes_per_allocation]`, the allocation itself is aligned, but each individual region is not. This commit ensures that each individual region also followed the alignment specified.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Currently, each allocation allocates an entire page, so even a relatively small number of allocations can use very large amounts of VTCM. This commit changes calls to
AllocVtcmWorkspaceof shape[N,M]from performingNallocations of sizeM, to 1 allocation of sizeN*M. SinceNis usually much smaller than a page, this reduces the total amount of memory required.This is an intermediate step, where the long-term solution is to use static planning for VTCM allocations. This returns the same
void**type as the static planning eventually will, but avoids excess memory use in the meantime.