[TIR][USMP] adding the pass to convert to pool offsets #9418

manupak · 2021-11-01T19:27:52Z

This commit adds a transform pass that consumes the planned pool allocations using memory planning algorithm that convertes them to pool offsets.

* adds two test cases for a linear structure with two pools
* adds test case with a single pool for residual structures

manupak · 2021-11-01T19:32:12Z

This is blocked on #8468 and then #9214. This contains the contents of the dependent PRs.

manupak · 2021-11-01T19:34:27Z

a friendly ping @mbs-octoml !

mbs-octoml · 2021-11-22T21:22:26Z

Hi @manupa-arm, I was waiting for #8468 to go in and #9214 to come out of DRAFT. Is that ok?

manupak · 2021-11-22T22:13:00Z

@mbs-octoml , yes Im also waiting for #8468 to go in :D -- so when #9214 comes out of draft it doesn't include changes belonging to #8468

manupak · 2021-12-03T09:33:25Z

Ping for reviews : @mbaret @mbs-octoml

mbs-octoml

Thanks for including the comments. Made the mistake of looking at this late on a Fri afternoon and my eyes started glazing over, but seems the main things are the new load_node in the AllocateNode visitor and capturing AllocatedPoolInfo on func/global params.

I'm still finding my way in TIR land so defer to say @junrushao1994 or similar for proper review.

include/tvm/tir/usmp/utils.h

mbs-octoml · 2021-12-04T00:41:54Z

src/tir/ir/stmt.cc

@@ -35,7 +35,12 @@ namespace tir {
 LetStmt::LetStmt(Var var, PrimExpr value, Stmt body, Span span) {
  ICHECK(value.defined());
  ICHECK(body.defined());
-  ICHECK_EQ(value.dtype(), var.dtype());
+  auto vdtype = value.dtype();
+  if (var->type_annotation.as<PointerTypeNode>()) {


could you add a comment? how did this ever work?

Added a comment.

src/tir/usmp/transform/convert_pool_allocations_to_offsets.cc

manupak · 2021-12-06T17:21:29Z

Thanks @mbs-octoml ! I've done the necessary changes.

Pinging folks @lhutton1 @ekalda @leandron @NicolaLancellotti for review/approval

lhutton1

Thanks @manupa-arm, I'm not all that familiar with USMP but gave this a quick look over. Just a few minor things - feel free to address them in a follow up.

python/tvm/tir/usmp/transform/transform.py

src/tir/usmp/transform/convert_pool_allocations_to_offsets.cc

tests/python/unittest/test_tir_usmp_transform_convert_pool_allocations_to_offsets.py

manupak · 2021-12-08T15:27:27Z

Thanks! @NicolaLancellotti @lhutton1 . I've addressed your comments.

lhutton1

LGTM

This commit adds a transform pass that consumes the planned pool allocations using memory planning algorithm that convertes them to pool offsets. * adds two test cases for a linear structure with two pools * adds test case with a single pool for residual structures Change-Id: I9d31e854461b5c21df72d1452120d286b96791c0

* Adding a toggle to produce TIR that is TVMScript printable for unit testing * Fixing the unit tests * Ensure deterministic pool variable ordering. Change-Id: I317675df03327b0ebbf4ca074255384e63f07cd6

Fixing the references after changes in the memory planning algorithm. Change-Id: Id7c22356fd5de43d10a2b4fc70e978af2c6d599d

* fixing the lint Change-Id: I7ff920b92d14a9919c930a4b35a2169c77a57dd1

* removing unnecessary defitinitions * remove global var map * adding explaination for let bindings to pointer type Change-Id: I31bd1a9f3057ee7f06252263565b0f75c51e6d13

* rebase changes * making imports absolute * fixing typos and removing unnecesary lines Change-Id: I4c94b9955b001513fecb39ca94f81b1ad99c7bfc

* fixing typos Change-Id: I42c557fd394aefdf8c2e825c4e88770eb0732f9b

manupak · 2021-12-09T13:19:31Z

@leandron can we get this in ?

leandron

LGTM

* [TIR][USMP] adding the pass to convert to pool offsets This commit adds a transform pass that consumes the planned pool allocations using memory planning algorithm that convertes them to pool offsets. * adds two test cases for a linear structure with two pools * adds test case with a single pool for residual structures Change-Id: I9d31e854461b5c21df72d1452120d286b96791c0 * [TIR][USMP] adding the pass to convert to pool offsets * Adding a toggle to produce TIR that is TVMScript printable for unit testing * Fixing the unit tests * Ensure deterministic pool variable ordering. Change-Id: I317675df03327b0ebbf4ca074255384e63f07cd6 * [TIR][USMP] adding the pass to convert to pool offsets Fixing the references after changes in the memory planning algorithm. Change-Id: Id7c22356fd5de43d10a2b4fc70e978af2c6d599d * [TIR][USMP] adding the pass to convert to pool offsets * fixing the lint Change-Id: I7ff920b92d14a9919c930a4b35a2169c77a57dd1 * [TIR][USMP] adding the pass to convert to pool offsets * removing unnecessary defitinitions * remove global var map * adding explaination for let bindings to pointer type Change-Id: I31bd1a9f3057ee7f06252263565b0f75c51e6d13 * [TIR][USMP] adding the pass to convert to pool offsets * rebase changes * making imports absolute * fixing typos and removing unnecesary lines Change-Id: I4c94b9955b001513fecb39ca94f81b1ad99c7bfc * [TIR][USMP] adding the pass to convert to pool offsets * fixing typos Change-Id: I42c557fd394aefdf8c2e825c4e88770eb0732f9b

manupak requested review from areusch, comaniac, Hzfengsy, jroesch, junrushao, kparzysz-quic, masahi, merrymercy, tqchen, vinx13, yzhliu, zhiics and ZihengJiang as code owners November 1, 2021 19:27

manupak marked this pull request as draft November 1, 2021 19:28

manupak mentioned this pull request Nov 1, 2021

[RFC] Unified Static Memory Planning (USMP) Tracking Issue #8404

Closed

21 tasks

manupak force-pushed the usmp_pool_allocations_to_offsets branch from 93a81d8 to f95f642 Compare November 22, 2021 18:36

manupak mentioned this pull request Nov 23, 2021

[TIR][USMP] Integrating USMP to AoT Executor #9565

Merged

manupak force-pushed the usmp_pool_allocations_to_offsets branch from f95f642 to f2841f5 Compare November 23, 2021 17:53

manupak force-pushed the usmp_pool_allocations_to_offsets branch 2 times, most recently from e5f1867 to 508b61d Compare December 2, 2021 18:05

manupak marked this pull request as ready for review December 2, 2021 18:06

manupak requested a review from icemelon as a code owner December 2, 2021 18:06

mbs-octoml approved these changes Dec 4, 2021

View reviewed changes

manupak force-pushed the usmp_pool_allocations_to_offsets branch from 508b61d to 977608e Compare December 6, 2021 16:54

lhutton1 approved these changes Dec 7, 2021

View reviewed changes

NicolaLancellotti approved these changes Dec 7, 2021

View reviewed changes

manupak force-pushed the usmp_pool_allocations_to_offsets branch from 977608e to 52a4f12 Compare December 8, 2021 14:46

lhutton1 approved these changes Dec 8, 2021

View reviewed changes

manupak added 7 commits December 8, 2021 15:41

[TIR][USMP] adding the pass to convert to pool offsets

2b318cf

* Adding a toggle to produce TIR that is TVMScript printable for unit testing * Fixing the unit tests * Ensure deterministic pool variable ordering. Change-Id: I317675df03327b0ebbf4ca074255384e63f07cd6

[TIR][USMP] adding the pass to convert to pool offsets

4b34dd7

Fixing the references after changes in the memory planning algorithm. Change-Id: Id7c22356fd5de43d10a2b4fc70e978af2c6d599d

[TIR][USMP] adding the pass to convert to pool offsets

673b298

* fixing the lint Change-Id: I7ff920b92d14a9919c930a4b35a2169c77a57dd1

[TIR][USMP] adding the pass to convert to pool offsets

2ff702a

* removing unnecessary defitinitions * remove global var map * adding explaination for let bindings to pointer type Change-Id: I31bd1a9f3057ee7f06252263565b0f75c51e6d13

[TIR][USMP] adding the pass to convert to pool offsets

3ac8379

* rebase changes * making imports absolute * fixing typos and removing unnecesary lines Change-Id: I4c94b9955b001513fecb39ca94f81b1ad99c7bfc

[TIR][USMP] adding the pass to convert to pool offsets

2c6cf6c

* fixing typos Change-Id: I42c557fd394aefdf8c2e825c4e88770eb0732f9b

manupak force-pushed the usmp_pool_allocations_to_offsets branch from 9ce59ee to 2c6cf6c Compare December 8, 2021 15:49

leandron approved these changes Dec 9, 2021

View reviewed changes

leandron merged commit 3ce4fe4 into apache:main Dec 9, 2021

driazati mentioned this pull request Jul 14, 2022

TVM v0.9.0.rc0 Release Candidate Notes #12102

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[TIR][USMP] adding the pass to convert to pool offsets #9418

[TIR][USMP] adding the pass to convert to pool offsets #9418

manupak commented Nov 1, 2021

manupak commented Nov 1, 2021 •

edited

Loading

manupak commented Nov 1, 2021

mbs-octoml commented Nov 22, 2021

manupak commented Nov 22, 2021

manupak commented Dec 3, 2021

mbs-octoml left a comment

mbs-octoml Dec 4, 2021

manupak Dec 6, 2021

manupak commented Dec 6, 2021

lhutton1 left a comment

manupak commented Dec 8, 2021

lhutton1 left a comment

manupak commented Dec 9, 2021

leandron left a comment

[TIR][USMP] adding the pass to convert to pool offsets #9418

[TIR][USMP] adding the pass to convert to pool offsets #9418

Conversation

manupak commented Nov 1, 2021

manupak commented Nov 1, 2021 • edited Loading

manupak commented Nov 1, 2021

mbs-octoml commented Nov 22, 2021

manupak commented Nov 22, 2021

manupak commented Dec 3, 2021

mbs-octoml left a comment

Choose a reason for hiding this comment

mbs-octoml Dec 4, 2021

Choose a reason for hiding this comment

manupak Dec 6, 2021

Choose a reason for hiding this comment

manupak commented Dec 6, 2021

lhutton1 left a comment

Choose a reason for hiding this comment

manupak commented Dec 8, 2021

lhutton1 left a comment

Choose a reason for hiding this comment

manupak commented Dec 9, 2021

leandron left a comment

Choose a reason for hiding this comment

manupak commented Nov 1, 2021 •

edited

Loading