【PIR】Add AutoLayoutPass and AutoLayoutSimplifyPass #67576

liuruyan · 2024-08-20T08:11:10Z

PR Category

Performance Optimization

PR Types

Performance

Description

The two passes are used to greedily insert transpose into the IR and remove consecutive redundant transposes to ensure that the Operation works on their appropriate Layout.

Pcard-67164

paddle-bot · 2024-08-20T08:11:16Z

你的PR提交成功，感谢你对开源项目的贡献!
请关注后续CI自动化测试结果，详情请参考Paddle-CI手册。
Your PR has been submitted. Thanks for your contribution!
Please wait for the result of CI firstly. See Paddle CI Manual for details.

paddle-ci-bot · 2024-08-31T03:06:09Z

Sorry to inform you that 70f53ac's CIs have passed for more than 7 days. To prevent PR conflicts, you need to re-run all CIs manually.

zyfncg · 2024-09-05T11:34:15Z

paddle/common/flags.cc

@@ -1368,6 +1368,10 @@ PHI_DEFINE_EXPORTED_bool(
    false,
    "EinsumOp backward will be speedup at the expense of more gpu memory.");

+PHI_DEFINE_EXPORTED_bool(enable_auto_layout_pass,


flag需要加一些类似的注释说明

zyfncg · 2024-09-05T11:40:04Z

paddle/fluid/inference/api/analysis_predictor.cc

+            if (gpu_pass == "transfer_layout_pass" &&
+                config_.autolayout_enabled())
+              continue;


这样的逻辑有些trick，而且不止一处，不了解背景的开发者看到会感到迷茫，最好想办法优化下

拆分pr，关于推理的flag拆除一个独立pr进行处理

zyfncg · 2024-09-05T11:44:43Z

paddle/fluid/pir/transforms/general/auto_layout_pass.cc

+  void TransferLayout(pir::Builder builder, pir::Block* block) {
+    for (auto& op_item : *block) {
+      auto op = &op_item;
+      auto op_name = op->name();


const auto&

由于DoTransposeOpOperand、RewriteLayout、DoTransposeOpResult，中要求 pir::Operation* 因此优化为 auto&&

zyfncg · 2024-09-05T11:45:45Z

paddle/fluid/pir/transforms/general/auto_layout_pass.cc

+      if (op_name == "builtin.parameter" || op_name == "pd_op.feed" ||
+          op_name == "builtin.shadow_output")


这里的判断可以看看 ImmutableLayoutTrait 这个trait

确实可以使用 ImmutableLayoutTrait 辅助判断

zyfncg · 2024-09-05T12:07:51Z

paddle/fluid/pir/transforms/general/auto_layout_pass.cc

+          RewriteLayout(op, op->operands_source());
+          DoTransposeOpResult(op, builder);
+        }
+      } else if (IsInsertTransposeOpBefore(op)) {


这个else分支怎么理解？

True: 当前 Op 不是 NHWC 并且该 Op 的 operand 中至少一个被插入了 transpose 而为 NCHW。但是该 OP 对 layout 不敏感，可以运行在 NHWC 下，因此需在其前插入 Transpose 以消除 NCHW 的 Transpose

zyfncg · 2024-09-05T12:13:20Z

paddle/fluid/pir/transforms/general/auto_layout_pass.cc

+    if (op->HasAttribute("data_format")) {
+      op->set_attribute("data_format", pir::StrAttribute::get(ctx_, "NHWC"));
+    }
+    auto p_attribute_map = op->attributes();


const auto&

zyfncg · 2024-09-05T12:21:15Z

paddle/fluid/pir/transforms/general/auto_layout_pass.cc

+  // Skip the operand which is not dense tensor or not 4-D tensor, they don't
+  // need transpose.
+  bool JudgeValue(const pir::Value& value) {
+    if (auto type = value.type().dyn_cast<paddle::dialect::DenseTensorType>()) {


最好先判断下value和value.type()是否为空

zyfncg · 2024-09-05T12:22:18Z

paddle/fluid/pir/transforms/general/auto_layout_pass.cc

+    builder.SetInsertionPointAfter(op);
+    for (auto& result : op->results()) {
+      // Canbe optimize with cache when not eliminate the transpose op.
+      if (!JudgeValue(result)) continue;


可以再加个过滤条件，value的user为空时就不插transpose了

zyfncg · 2024-09-05T12:27:07Z

paddle/fluid/pir/transforms/general/auto_layout_simplify_pass.cc

+
+  bool Match(paddle::dialect::TransposeOp op) const override {
+    auto before_transpose = op.x().defining_op();
+    if (before_transpose->dyn_cast<paddle::dialect::TransposeOp>()) {


这里可以写成

if(!before_transpose->isa<paddle::dialect::TransposeOp>()) { return false; }

从而避免后面大量代码的缩进

zyfncg · 2024-09-05T12:28:06Z

paddle/fluid/pir/transforms/general/auto_layout_simplify_pass.cc

+      std::vector<int32_t> before_perm;
+      for (size_t i = 0; i < before_perm_attr.size(); ++i) {
+        auto attr = before_perm_attr.at(i);
+        before_perm.push_back(attr.dyn_cast<pir::Int32Attribute>().data());
+      }
+
+      const auto after_perm_attr = op.attribute<pir::ArrayAttribute>("perm");
+      std::vector<int32_t> after_perm;
+      for (size_t i = 0; i < after_perm_attr.size(); ++i) {
+        auto attr = after_perm_attr.at(i);
+        after_perm.push_back(attr.dyn_cast<pir::Int32Attribute>().data());
+      }
+
+      if (before_perm[0] == after_perm[0] && before_perm[1] == after_perm[3] &&
+          before_perm[2] == after_perm[1] && before_perm[3] == after_perm[2] &&
+          before_perm == NCHW2NHWC_) {
+        return true;
+      }
+
+      if (before_perm[0] == after_perm[0] && before_perm[1] == after_perm[2] &&
+          before_perm[2] == after_perm[3] && before_perm[3] == after_perm[1] &&
+          before_perm == NHWC2NCHW_) {
+        return true;
+      }


这里的判断逻辑是不可以化简下？

Aurelius84

建议对这layout的调整和冗余transpose的化简分别构造一个单测case，来确保功能始终是正常的。

Aurelius84 · 2024-09-09T10:55:56Z

paddle/common/flags.cc

+/**
+ * Performance related FLAG
+ * Name: enable_auto_layout_pass
+ * Since Version: 2.6.0


Suggested change

* Since Version: 2.6.0

* Since Version: 3.0.0

准确的讲，这里应该写下一个要发布的正式版本

Aurelius84 · 2024-09-09T10:57:25Z

paddle/fluid/pir/transforms/general/auto_layout_pass.cc

+#include <unordered_set>
+
+#include "paddle/common/enforce.h"
+#include "paddle/common/errors.h"


只需包含enforce.h即可，不需要显式包含errors.h

Aurelius84 · 2024-09-09T10:58:55Z

paddle/fluid/pir/transforms/general/auto_layout_pass.cc

+
+class AutoLayoutPass : public pir::Pass {
+ public:
+  AutoLayoutPass() : pir::Pass("auto_layout_pass", 1) {}


// opt_level=1: constant fold, cse, memory optimize, etc. // opt_level=2: the fusion logical pass. // opt_level=3: layout, etc. // opt_level=4: the radical optimization, maybe affect precision, etc. uint8_t opt_level;

see opt_level in paddle/pir/include/pass/pass.h

Suggested change

AutoLayoutPass() : pir::Pass("auto_layout_pass", 1) {}

AutoLayoutPass() : pir::Pass("auto_layout_pass", 3) {}

Aurelius84 · 2024-09-09T11:07:12Z

paddle/fluid/pir/transforms/general/auto_layout_pass.cc

+    builder.set_insertion_point(op);
+
+    // For conv2d, only transpose the input.
+    if (op->name() == "pd_op.conv2d") {


一般我们更建议用 op->isa()

… autolayoutpass

paddle-ci-bot · 2024-10-02T03:15:38Z

Sorry to inform you that b871d4a's CIs have passed for more than 7 days. To prevent PR conflicts, you need to re-run all CIs manually.

… autolayoutpass

add two pass.

b7f1732

paddle-bot bot added the contributor External developers label Aug 20, 2024

skip filter transpose in conv2d.

70f53ac

add flag.

2df3030

zyfncg reviewed Sep 5, 2024

View reviewed changes

liuruyan added 3 commits September 9, 2024 03:12

fix comment and split part of infer into a new PR.

b604612

extract func for not implement infermeta.

624e6db

fix

37c80de

Aurelius84 previously approved these changes Sep 9, 2024

View reviewed changes

liuruyan added 3 commits September 18, 2024 07:20

Merge branch 'develop' of https://github.com/PaddlePaddle/Paddle into…

55b87d1

… autolayoutpass

Merge branch 'develop' of https://github.com/PaddlePaddle/Paddle into…

7bc17a8

… autolayoutpass

add test.

67ff503

liuruyan dismissed Aurelius84’s stale review via 67ff503 September 23, 2024 08:29

fix coverage

b871d4a

Merge branch 'develop' of https://github.com/PaddlePaddle/Paddle into…

5a50223

… autolayoutpass

phlrain approved these changes Oct 11, 2024

View reviewed changes

zyfncg approved these changes Oct 11, 2024

View reviewed changes

zyfncg merged commit c0b11bb into PaddlePaddle:develop Oct 11, 2024
26 of 27 checks passed

liuruyan deleted the autolayoutpass branch October 11, 2024 04:33

liuruyan mentioned this pull request Oct 11, 2024

Add Flag of AutoLayoutPass in inference process #68632

Closed

liuruyan changed the title ~~Add AutoLayoutPass and AutoLayoutSimplifyPass~~ 【PIR】Add AutoLayoutPass and AutoLayoutSimplifyPass Oct 16, 2024

liuruyan mentioned this pull request Oct 16, 2024

【FLAG】Add Flag of AutoLayoutPass in d2s process #68759

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

【PIR】Add AutoLayoutPass and AutoLayoutSimplifyPass #67576

【PIR】Add AutoLayoutPass and AutoLayoutSimplifyPass #67576

liuruyan commented Aug 20, 2024 •

edited

Loading

paddle-bot bot commented Aug 20, 2024

paddle-ci-bot bot commented Aug 31, 2024

zyfncg Sep 5, 2024

liuruyan Sep 9, 2024

zyfncg Sep 5, 2024

liuruyan Sep 8, 2024

zyfncg Sep 5, 2024

liuruyan Sep 8, 2024 •

edited

Loading

zyfncg Sep 5, 2024

liuruyan Sep 8, 2024

zyfncg Sep 5, 2024

liuruyan Sep 8, 2024

zyfncg Sep 5, 2024

liuruyan Sep 8, 2024

zyfncg Sep 5, 2024

liuruyan Sep 8, 2024

zyfncg Sep 5, 2024

liuruyan Sep 8, 2024

zyfncg Sep 5, 2024

liuruyan Sep 8, 2024

zyfncg Sep 5, 2024

liuruyan Sep 8, 2024

Aurelius84 left a comment

Aurelius84 Sep 9, 2024

Aurelius84 Sep 9, 2024

Aurelius84 Sep 9, 2024

Aurelius84 Sep 9, 2024

paddle-ci-bot bot commented Oct 2, 2024

		if (op_name == "builtin.parameter" \|\| op_name == "pd_op.feed" \|\|
		op_name == "builtin.shadow_output")

	AutoLayoutPass() : pir::Pass("auto_layout_pass", 1) {}
	AutoLayoutPass() : pir::Pass("auto_layout_pass", 3) {}

【PIR】Add AutoLayoutPass and AutoLayoutSimplifyPass #67576

【PIR】Add AutoLayoutPass and AutoLayoutSimplifyPass #67576

Conversation

liuruyan commented Aug 20, 2024 • edited Loading

PR Category

PR Types

Description

paddle-bot bot commented Aug 20, 2024

paddle-ci-bot bot commented Aug 31, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

liuruyan Sep 8, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Aurelius84 left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

paddle-ci-bot bot commented Oct 2, 2024

liuruyan commented Aug 20, 2024 •

edited

Loading

liuruyan Sep 8, 2024 •

edited

Loading