Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Opt: Faster Symbolic Lowerer #275

Merged
merged 5 commits into from
Nov 30, 2024
Merged

Opt: Faster Symbolic Lowerer #275

merged 5 commits into from
Nov 30, 2024

Conversation

hikettei
Copy link
Owner

@hikettei hikettei commented Nov 30, 2024

Workload

  • Improve the speed of lowering full symbolic schedule-item (remove expr-scalar-equivalent-p)
  • Improve the scheduling/simplifying time for the full symbolic
  • optimize remove-unused-blueprint
  • improve the simplifying time
  • CI: Benchmark with symbolic
  • Tests: having decent tests for full symbolic (for gemm, attention, etc...)
  • Repro and test for BugFix: Symbolic is loaded as tensor? #272
  • NOOPT=1 produces a warning with Transformer?

@hikettei hikettei changed the title Opt: Fast and safe symbolic compilation Opt: Fast and Safe symbolic compilation Nov 30, 2024
@hikettei
Copy link
Owner Author

Lowerer benchmark with full symbolic

  seconds  |     gc     |     consed    | calls |  sec/call  |  name  
-----------------------------------------------------------
    13.181 |      0.000 | 1,076,014,848 |    59 |   0.223401 | CATEN/CODEGEN/BLUEPRINT:LOWER-SCHEDULE-ITEM
     0.989 |      0.032 | 2,568,178,736 |    59 |   0.016765 | CATEN/CODEGEN/BLUEPRINT::GET-GROUPED-DIMS
     0.845 |      0.000 | 1,445,614,320 |    59 |   0.014330 | CATEN/CODEGEN/BLUEPRINT::SCHEDULE-ITEM-GATHER-DYNAMIC-SHAPES
     0.486 |      0.000 |     3,645,744 |   352 |   0.001381 | CATEN/CODEGEN/BLUEPRINT::TRY-INSERT-NODE
     0.358 |      0.000 |     2,270,288 |    31 |   0.011549 | CATEN/CODEGEN/BLUEPRINT::FIND-CACHE-BASE-SCHEDULE-ITEM
     0.246 |      0.000 |   739,971,072 |    59 |   0.004174 | CATEN/CODEGEN/BLUEPRINT::FIXUP-GRAPH-ITERATION-SPACE
     0.046 |      0.000 |    45,684,400 |    59 |   0.000778 | CATEN/CODEGEN/BLUEPRINT:PRINT-BLUEPRINT
     0.008 |      0.000 |     6,921,824 |    31 |   0.000272 | CATEN/CODEGEN/BLUEPRINT:LOWER-CACHED-SCHEDULE-ITEM
     0.007 |      0.000 |     7,950,208 |   257 |   0.000028 | CATEN/CODEGEN/BLUEPRINT::%MAKE-FOR
     0.001 |      0.000 |        75,312 |   355 |   0.000002 | CATEN/CODEGEN/BLUEPRINT::NODE-DEPEND-IDX-LIST
     0.001 |      0.000 |       196,400 |    59 |   0.000009 | CATEN/CODEGEN/BLUEPRINT::GRAPH-REDUCED-AXES
     0.000 |      0.000 |       393,120 |    59 |   0.000008 | CATEN/CODEGEN/BLUEPRINT::GRAPH-SWIZZLE-SPACE
     0.000 |      0.000 |             0 |    59 |   0.000005 | CATEN/CODEGEN/BLUEPRINT::INITIAL-BP
     0.000 |      0.000 |             0 |    31 |   0.000008 | CATEN/CODEGEN/BLUEPRINT::COMPARE-AND-MAKE-NAMESPACE-MAP
     0.000 |      0.000 |             0 |    59 |   0.000004 | CATEN/CODEGEN/BLUEPRINT::SCHEDULE-ITEM-INFER-IO-BUFFERS
     0.000 |      0.000 |       157,424 |   392 |   0.000001 | CATEN/CODEGEN/BLUEPRINT::NODE-REDUCED-GIDS
     0.000 |      0.000 |             0 |   441 |   0.000000 | CATEN/CODEGEN/BLUEPRINT::NODE-REDUCED-AXES
     0.000 |      0.000 |         7,328 |    59 |   0.000002 | CATEN/CODEGEN/BLUEPRINT::SIMPLIFY-POINTER-AND-CONSTANT
     0.000 |      0.000 |       188,144 |   257 |   0.000000 | CATEN/CODEGEN/BLUEPRINT::%MAKE-ENDFOR
     0.000 |      0.000 |       209,200 |    59 |   0.000001 | CATEN/CODEGEN/BLUEPRINT::INITIAL-LOOP-PERMUTATION
     0.000 |      0.000 |             0 |    37 |   0.000001 | CATEN/CODEGEN/BLUEPRINT::REDUCE-BP
     0.000 |      0.000 |         6,688 |    59 |   0.000000 | CATEN/CODEGEN/BLUEPRINT::MAKE-CTX
     0.000 |      0.000 |             0 |    47 |   0.000000 | CATEN/CODEGEN/BLUEPRINT::RECURSIVE-SCALAR-P
     0.000 |      0.000 |        49,136 |   353 |   0.000000 | CATEN/CODEGEN/BLUEPRINT::RECURSIVE-LOWER-INTO-BP
-----------------------------------------------------------
    16.171 |      0.032 | 5,897,534,192 | 3,292 |            | Total

@hikettei
Copy link
Owner Author

Also for exprify

  seconds  |     gc     |     consed     |  calls |  sec/call  |  name  
-------------------------------------------------------------
    13.179 |      0.000 |  1,076,298,880 |    118 |   0.111690 | CATEN/CODEGEN/BLUEPRINT:LOWER-SCHEDULE-ITEM
     9.390 |      0.000 |     66,548,320 |     59 |   0.159146 | CATEN/CODEGEN/EXPRIFY::REMOVE-UNUSED-BLUEPRINT
     1.981 |      0.032 |  5,136,625,280 |    118 |   0.016787 | CATEN/CODEGEN/BLUEPRINT::GET-GROUPED-DIMS
     1.768 |      0.049 |  2,891,099,936 |    118 |   0.014982 | CATEN/CODEGEN/BLUEPRINT::SCHEDULE-ITEM-GATHER-DYNAMIC-SHAPES
     1.541 |      0.000 |    984,390,032 |    118 |   0.013056 | CATEN/CODEGEN/EXPRIFY::SCHEDULE-OUTPUTS
     1.277 |      0.000 |     16,265,008 |    118 |   0.010824 | CATEN/CODEGEN/EXPRIFY::BLUEPRINT-TMP-BUFFERS
     0.955 |      0.000 |      7,559,344 |    704 |   0.001357 | CATEN/CODEGEN/BLUEPRINT::TRY-INSERT-NODE
     0.691 |      0.000 |      5,264,880 |     62 |   0.011141 | CATEN/CODEGEN/BLUEPRINT::FIND-CACHE-BASE-SCHEDULE-ITEM
     0.509 |      0.000 |  1,479,418,464 |    118 |   0.004312 | CATEN/CODEGEN/BLUEPRINT::FIXUP-GRAPH-ITERATION-SPACE
     0.090 |      0.000 |     91,717,424 |    118 |   0.000765 | CATEN/CODEGEN/BLUEPRINT:PRINT-BLUEPRINT
     0.015 |      0.000 |     13,126,720 |     62 |   0.000237 | CATEN/CODEGEN/BLUEPRINT:LOWER-CACHED-SCHEDULE-ITEM
     0.014 |      0.000 |     16,127,264 |    514 |   0.000028 | CATEN/CODEGEN/BLUEPRINT::%MAKE-FOR
     0.004 |      0.000 |      5,630,944 |     59 |   0.000073 | CATEN/CODEGEN/EXPRIFY:EXPR-SET-ITERATIONS
     0.003 |      0.000 |      1,646,576 |    315 |   0.000008 | CATEN/CODEGEN/EXPRIFY::EXPRIFY
     0.002 |      0.000 |        402,912 |    710 |   0.000002 | CATEN/CODEGEN/BLUEPRINT::NODE-DEPEND-IDX-LIST
     0.001 |      0.000 |        247,968 |    118 |   0.000008 | CATEN/CODEGEN/BLUEPRINT::GRAPH-REDUCED-AXES
     0.001 |      0.000 |        239,696 |    665 |   0.000001 | CATEN/CODEGEN/EXPRIFY::FORCE-REALIZE-P
     0.001 |      0.000 |        524,160 |    118 |   0.000007 | CATEN/CODEGEN/BLUEPRINT::GRAPH-SWIZZLE-SPACE
     0.001 |      0.000 |        767,152 |     59 |   0.000013 | CATEN/CODEGEN/EXPRIFY:GRAPH-EXPRIFY
     0.001 |      0.000 |        248,752 |     59 |   0.000011 | CATEN/CODEGEN/EXPRIFY:GRAPH-SCALARIFY
     0.001 |      0.000 |         17,184 |    118 |   0.000005 | CATEN/CODEGEN/BLUEPRINT::INITIAL-BP
     0.000 |      0.000 |        174,768 |    315 |   0.000001 | CATEN/CODEGEN/EXPRIFY::MERGE-AND-GRAFT-EXPR
     0.000 |      0.000 |        461,504 |    784 |   0.000001 | CATEN/CODEGEN/BLUEPRINT::NODE-REDUCED-GIDS
     0.000 |      0.000 |              0 |    118 |   0.000004 | CATEN/CODEGEN/BLUEPRINT::SCHEDULE-ITEM-INFER-IO-BUFFERS
     0.000 |      0.000 |              0 |     62 |   0.000006 | CATEN/CODEGEN/BLUEPRINT::COMPARE-AND-MAKE-NAMESPACE-MAP
     0.000 |      0.000 |            128 |    882 |   0.000000 | CATEN/CODEGEN/BLUEPRINT::NODE-REDUCED-AXES
     0.000 |      0.000 |          1,120 |    328 |   0.000001 | CATEN/CODEGEN/EXPRIFY::REWRITE-EXPR-AREF
     0.000 |      0.000 |         26,528 |     59 |   0.000004 | CATEN/CODEGEN/EXPRIFY::EXPR-ONLY-LEAF-ARE-ARGUMENTS
     0.000 |      0.000 |         47,040 |     59 |   0.000004 | CATEN/CODEGEN/EXPRIFY:GRAPH-PROPAGATE-POINTER-ID-TYPE
     0.000 |      0.000 |          7,328 |    118 |   0.000002 | CATEN/CODEGEN/BLUEPRINT::SIMPLIFY-POINTER-AND-CONSTANT
     0.000 |      0.000 |         51,568 |    826 |   0.000000 | CATEN/CODEGEN/EXPRIFY::PROPAGATE-LOAD-CONST
     0.000 |      0.000 |        295,232 |    514 |   0.000000 | CATEN/CODEGEN/BLUEPRINT::%MAKE-ENDFOR
     0.000 |      0.000 |         65,520 |    525 |   0.000000 | CATEN/CODEGEN/EXPRIFY::MEMORY-ACCESS-LOCAL-P
     0.000 |      0.000 |        260,768 |    118 |   0.000001 | CATEN/CODEGEN/BLUEPRINT::INITIAL-LOOP-PERMUTATION
     0.000 |      0.000 |          9,792 |     74 |   0.000001 | CATEN/CODEGEN/BLUEPRINT::REDUCE-BP
     0.000 |      0.000 |         26,544 |    425 |   0.000000 | CATEN/CODEGEN/EXPRIFY::REWRITE-AS-SCALAR
     0.000 |      0.000 |         19,936 |    118 |   0.000000 | CATEN/CODEGEN/BLUEPRINT::MAKE-CTX
     0.000 |      0.000 |         51,568 |     59 |   0.000001 | CATEN/CODEGEN/EXPRIFY:EXPR-REWRITE-EDGE-WITH-POINTER-ID
     0.000 |      0.000 |              0 |    168 |   0.000000 | CATEN/CODEGEN/EXPRIFY::EXPR-WRITES
     0.000 |      0.000 |              0 |     94 |   0.000000 | CATEN/CODEGEN/BLUEPRINT::RECURSIVE-SCALAR-P
     0.000 |      0.000 |         83,376 |    706 |   0.000000 | CATEN/CODEGEN/BLUEPRINT::RECURSIVE-LOWER-INTO-BP
-------------------------------------------------------------
    31.427 |      0.081 | 11,795,749,616 | 10,800 |            | Total

@hikettei hikettei changed the title Opt: Fast and Safe symbolic compilation Opt: Faster Symbolic Lowerer Nov 30, 2024
@hikettei hikettei marked this pull request as ready for review November 30, 2024 06:33
@hikettei hikettei merged commit dfddd5c into main Nov 30, 2024
6 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant