[Compat][3.11] support for-loop #371

zrr1999 · 2023-09-03T16:02:03Z

前移 _current_line 初始化为 -1 的语句，使得当 get_instructions 函数中发生错误时依然可以正确返回报错栈。
实现 gen_jump、gen_pop_jump，简化后期适配操作。
fix _break_graph_in_for_loop in python3.11, 将以前固定的jump（大多数为绝对跳转），改为根据方向调整jump指令（python3.11新指令）。
修复 relocate_jump_target，使其可以正确计算JUMP_BACKWARD的参数。
启用如下单测
- test_12_for_loop ❌ -> ✅
- test_15_slice ❌ -> ✅
- test_21_global ❌ -> ✅
- test_enumerate ❌ -> ✅
- test_inplace_api ❌ -> ✅
- test_range ❌ -> ✅
- test_resnet50_backward ❌ -> ✅
- test_side_effects 🚧 -> ✅

paddle-bot · 2023-09-03T16:02:08Z

Thanks for your contribution!

SigureMo · 2023-09-05T02:51:43Z

sot/opcode_translator/executor/opcode_executor.py

+        if sys.version_info >= (3, 11):
+            jump_if_break = self._graph.pycode_gen._add_instr(
+                "POP_JUMP_FORWARD_IF_FALSE"
+            )
+        else:
+            jump_if_break = self._graph.pycode_gen._add_instr(
+                "POP_JUMP_IF_FALSE"
+            )


这里是否可以直接用 gen_pop_jump 呢？

SigureMo · 2023-09-05T02:53:35Z

sot/opcode_translator/instruction_utils/instruction_utils.py

+            elif instr.opname == "JUMP_BACKWARD":
+                new_arg = instr.offset - jump_target + 2
+                if new_arg < 0:
+                    # NOTE(zrr1999): in Python 3.11, JUMP_ABSOLUTE is removed, so we need to use JUMP_FORWARD instead,
+                    # but in for loop breakgraph, we reuse JUMP_BACKWARD to jump forward, so we need to change it to JUMP_FORWARD.
+                    instr.opname = "JUMP_FORWARD"
+                    instr.opcode = dis.opmap["JUMP_FORWARD"]
+                    new_arg = jump_target - instr.offset - 2
+            else:
+                # other REL_JUMP, such as JUMP_FORWARD, FOR_ITER
+                new_arg = jump_target - instr.offset - 2


是否可以写个函数来自动修正 JUMP 呢（Callable[[Instruction], Instruction]）？包括 BACKWARD -> FORWARD 和 FORWARD -> BACKWARD

这段逻辑就可以封装起来了

feifei-111 · 2023-09-05T03:05:19Z

sot/opcode_translator/executor/opcode_executor.py


        # 7. add JUMP_ABSOLUTE to FOR_ITER
-        self._graph.pycode_gen._add_instr("JUMP_ABSOLUTE", jump_to=for_iter)
+        self._graph.pycode_gen.gen_jump(cur_instr, for_iter)


cur_instr 只是栈分析时候使用的临时变量，这样传可能不太对

这里一定是往回跳（跳回FOR_ITER）

feifei-111 · 2023-09-05T03:14:49Z

sot/opcode_translator/executor/opcode_executor.py

-        break_jump = pycode_gen._add_instr(
-            "JUMP_ABSOLUTE", jump_to=out_loop_instr
-        )
+        cur_instr = self._instructions[self._lasti]


这个不就是for_iter这个输入吗

feifei-111 · 2023-09-05T03:17:11Z

sot/opcode_translator/executor/opcode_executor.py

-            "JUMP_ABSOLUTE", jump_to=out_loop_instr
-        )
+        cur_instr = self._instructions[self._lasti]
+        pycode_gen.gen_jump(cur_instr, out_loop_instr)


这里jump的起点也不是self._instructions[self._lasti]吧，应该是我们生成的最新代码的起点
如果是这样，那么起点和终点应该都到 pycode gen 里面拿啊，怎么去 self._instruction 里面拿，那个是原始的code啊

feifei-111 · 2023-09-05T03:17:55Z

sot/opcode_translator/executor/opcode_executor.py

        nop_for_continue = pycode_gen._add_instr("NOP")
+        jump = pycode_gen.gen_jump(cur_instr, for_iter_instr)


feifei-111 · 2023-09-05T03:20:27Z

sot/opcode_translator/executor/pycode_generator.py

@@ -871,6 +871,38 @@ def gen_swap(self, n):
        else:
            raise NotImplementedError("swap is not supported before python3.11")

+    def gen_jump(self, instr: Instruction, jump_to: Instruction) -> Instruction:


不需要传instr吧？没有用而且是错的，因为跳转起点不应该是待会要插入的jump语句吗？那么应该用下一个待插入的位置的offset来比吧？

feifei-111 · 2023-09-05T03:21:40Z

sot/opcode_translator/executor/pycode_generator.py

+
+    def gen_pop_jump(
+        self, instr: Instruction, jump_to: Instruction, *, suffix: str = ""
+    ):


SigureMo · 2023-09-05T06:16:45Z

sot/opcode_translator/instruction_utils/instruction_utils.py

+            instr.opname = "JUMP_BACKWARD"
+            instr.opcode = dis.opmap["JUMP_BACKWARD"]


这貌似不太对？这里是不是应该将 FORWARD 改成 BACKWARD，BACKWARD 改成 FORWARD？

另外 else 分支是不是不用动？

SigureMo · 2023-09-05T06:17:55Z

sot/opcode_translator/executor/pycode_generator.py

@@ -871,6 +871,35 @@ def gen_swap(self, n):
        else:
            raise NotImplementedError("swap is not supported before python3.11")

+    def gen_forward_jump(self, jump_to: Instruction) -> Instruction:
+        return self._add_instr("JUMP_BACKWARD", jump_to=jump_to)


emmm？FORWARD？

feifei-111

LGTM

SigureMo

LGTMeow 🐾

paddle-bot bot added the contributor External developers label Sep 3, 2023

zrr1999 added 7 commits September 4, 2023 13:30

fix bug

ca9b8e9

impl gen_jump

2e60b9b

fix relocate_jump_target

4f4a648

fix relocate_jump_target

64f75b8

gen_pop_jump and fix _break_graph_in_for_loop

7e79198

fix bug

51d2953

enable test14

36922fe

zrr1999 force-pushed the py3.11_bg_jump branch from 34aa1b0 to 36922fe Compare September 4, 2023 13:30

zrr1999 changed the title ~~Py3.11 bg jump~~ [Compat][3.11] support jump breakgraph Sep 4, 2023

zrr1999 marked this pull request as ready for review September 4, 2023 15:00

zrr1999 requested review from SigureMo and gouzil September 4, 2023 15:00

SigureMo added 2 commits September 5, 2023 10:49

enable more tests blocked by for-loop

11448db

Merge branch 'develop' into py3.11_bg_jump

e2735ff

SigureMo changed the title ~~[Compat][3.11] support jump breakgraph~~ [Compat][3.11] support for-loop Sep 5, 2023

SigureMo reviewed Sep 5, 2023

View reviewed changes

feifei-111 reviewed Sep 5, 2023

View reviewed changes

zrr1999 added 4 commits September 5, 2023 04:27

add ROT_N

218157b

remove some unused

79258f4

improve

68f09cb

improve

4144be5

SigureMo reviewed Sep 5, 2023

View reviewed changes

zrr1999 added 5 commits September 5, 2023 06:19

fix bug

5d38135

fix correct_jump_direction

2e148bc

fix bug

5620879

fix bug

50c5eaa

fix bug

066157c

SigureMo added 3 commits September 5, 2023 17:02

refine the code

562a1b1

add default value for gen_jump

d88751d

enable test_resnet50_backward

9087e16

feifei-111 approved these changes Sep 5, 2023

View reviewed changes

SigureMo approved these changes Sep 5, 2023

View reviewed changes

SigureMo merged commit 694aa4b into PaddlePaddle:develop Sep 5, 2023

SigureMo mentioned this pull request Sep 5, 2023

Python3.11 支持任务汇总 #360

Closed

zrr1999 mentioned this pull request Dec 1, 2023

WAVE SUMMIT+2023下半年飞桨开源之星评选-信息征集 PaddlePaddle/community#765

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Compat][3.11] support for-loop #371

[Compat][3.11] support for-loop #371

zrr1999 commented Sep 3, 2023 •

edited by SigureMo

Loading

paddle-bot bot commented Sep 3, 2023

SigureMo Sep 5, 2023

SigureMo Sep 5, 2023

feifei-111 Sep 5, 2023

feifei-111 Sep 5, 2023

feifei-111 Sep 5, 2023

feifei-111 Sep 5, 2023

feifei-111 Sep 5, 2023

feifei-111 Sep 5, 2023

feifei-111 Sep 5, 2023

SigureMo Sep 5, 2023

SigureMo Sep 5, 2023

feifei-111 left a comment

SigureMo left a comment

		nop_for_continue = pycode_gen._add_instr("NOP")
		jump = pycode_gen.gen_jump(cur_instr, for_iter_instr)

		instr.opname = "JUMP_BACKWARD"
		instr.opcode = dis.opmap["JUMP_BACKWARD"]

[Compat][3.11] support for-loop #371

[Compat][3.11] support for-loop #371

Conversation

zrr1999 commented Sep 3, 2023 • edited by SigureMo Loading

paddle-bot bot commented Sep 3, 2023

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

feifei-111 left a comment

Choose a reason for hiding this comment

SigureMo left a comment

Choose a reason for hiding this comment

zrr1999 commented Sep 3, 2023 •

edited by SigureMo

Loading