Code

Tianxing/moe quantization#693

Code

Draft

Chi-Chu319 wants to merge 56 commits intomain_perfROCm/triton:main_perffrom tianxing/moe-quantizationROCm/triton:tianxing/moe-quantizationCopy head branch name to clipboard

+319-29

Commits

Commits on Dec 2, 2024

• moe_gemm_kernel implemented

  Chi-Chu319

  committed
  a8e7ccf

  Copy full SHA for a8e7ccf

Commits on Dec 9, 2024

• added kernel caller

  Chi-Chu319

  committed
  a706c5d

  Copy full SHA for a706c5d

Commits on Dec 10, 2024

• test

  Chi-Chu319

  committed
  fa489bd

  Copy full SHA for fa489bd

• wil fix test

  Chi-Chu319

  committed
  0a2158d

  Copy full SHA for 0a2158d

Commits on Dec 11, 2024

• pdated autotune setting

  Chi-Chu319

  committed
  a6495a5

  Copy full SHA for a6495a5

Commits on Dec 13, 2024

• reformatted the code

  Chi-Chu319

  committed
  15b959d

  Copy full SHA for 15b959d

Commits on Dec 16, 2024

• moe_align_block_size impl

  Chi-Chu319

  committed
  60d8059

  Copy full SHA for 60d8059

• correct impl

  Chi-Chu319

  committed
  7e1beb9

  Copy full SHA for 7e1beb9

Commits on Dec 18, 2024

• added weight, test

  Chi-Chu319

  committed
  9cdaee6

  Copy full SHA for 9cdaee6

• updated readme and git action work flow

  Chi-Chu319

  committed
  fe44173

  Copy full SHA for fe44173

• pre-commit

  Chi-Chu319

  committed
  6f737c6

  Copy full SHA for 6f737c6

Commits on Dec 19, 2024

• benchmark

  Chi-Chu319

  committed
  4e17aba

  Copy full SHA for 4e17aba

Commits on Dec 20, 2024

• moved moe align block size outside the gemm function

  Chi-Chu319

  committed
  da1e148

  Copy full SHA for da1e148

• pre-commit

  Chi-Chu319

  committed
  f7ffc17

  Copy full SHA for f7ffc17

• Removed unnecessary comments

  Chi-Chu319

  committed
  9a43c1c

  Copy full SHA for 9a43c1c

• Merge branch 'main_perf' into tianxing/moe-gemm

  Chi-Chu319

  authored
  cefc74e

  Copy full SHA for cefc74e

Commits on Dec 23, 2024

• add dtype support, set the default dtype to fp16

  Chi-Chu319

  committed
  c13b572

  Copy full SHA for c13b572

• pre-commit

  Chi-Chu319

  committed
  b9a9504

  Copy full SHA for b9a9504

Commits on Dec 30, 2024

• tunnable benchmark

  Chi-Chu319

  committed
  1beeb14

  Copy full SHA for 1beeb14

• pre commit

  Chi-Chu319

  committed
  16c08d8

  Copy full SHA for 16c08d8

Commits on Dec 31, 2024

• code clean up

  Chi-Chu319

  committed
  e5ac604

  Copy full SHA for e5ac604

• Implemented moe gemm, test and benchmarking.

  Show description for 71649a6
  Chi-Chu319

  committed
  71649a6

  Copy full SHA for 71649a6

• Merge branch 'tianxing/moe-gemm' of github.com:ROCm/triton into tianxing/moe-gemm

  Chi-Chu319

  committed
  3d044a2

  Copy full SHA for 3d044a2

• removed benchmark files

  Chi-Chu319

  committed
  5adb971

  Copy full SHA for 5adb971

• Merge branch 'main_perf' into tianxing/moe-gemm

  Chi-Chu319

  authored
  6852a96

  Copy full SHA for 6852a96

• removed all the benchmark files

  Chi-Chu319

  committed
  ce4c4e8

  Copy full SHA for ce4c4e8

• updated readme

  Chi-Chu319

  committed
  826cf5d

  Copy full SHA for 826cf5d

Commits on Jan 2, 2025

• quantization impl

  Chi-Chu319

  committed
  1e1b66b

  Copy full SHA for 1e1b66b

• Quantization implementation fp8 int8. TODO check the b_scale shape for fp8

  Chi-Chu319

  committed
  121f922

  Copy full SHA for 121f922

Commits on Jan 3, 2025

• corrected moe scale int8 dim

  Chi-Chu319

  committed
  e73becc

  Copy full SHA for e73becc

• remove the -tune option, and consolidated the config files

  Chi-Chu319

  committed
  c06c61e

  Copy full SHA for c06c61e

• pre commit

  Chi-Chu319

  committed
  34130b9

  Copy full SHA for 34130b9

Commits on Jan 7, 2025

• Merge branch 'main_perf' into tianxing/moe-gemm

  Chi-Chu319

  committed
  c2b85ff

  Copy full SHA for c2b85ff

• updated M_THRESHOLD and configs after tunnig

  Chi-Chu319

  committed
  dc68ce1

  Copy full SHA for dc68ce1

• Merge branch 'main_perf' into tianxing/moe-gemm

  Chi-Chu319

  committed
  04a1629

  Copy full SHA for 04a1629

• mistral model benchmarking

  Chi-Chu319

  committed
  53fa702

  Copy full SHA for 53fa702

• pre commit

  Chi-Chu319

  committed
  1698d79

  Copy full SHA for 1698d79

• noqa: E402

  Chi-Chu319

  committed
  42d8dbc

  Copy full SHA for 42d8dbc

• pre commit

  Chi-Chu319

  committed
  3a04e90

  Copy full SHA for 3a04e90

Commits on Jan 8, 2025

• more fine tuned model config. show mem throught put in benchmark

  Chi-Chu319

  committed
  9c83fd7

  Copy full SHA for 9c83fd7

• pre-commit

  Chi-Chu319

  committed
  e9d3dc2

  Copy full SHA for e9d3dc2

• Merge branch 'tianxing/moe-gemm' into tianxing/moe-quantization

  Chi-Chu319

  committed
  03f43d7

  Copy full SHA for 03f43d7

• quantization benchmark support

  Chi-Chu319

  committed
  78c50a4

  Copy full SHA for 78c50a4

• pre commit

  Chi-Chu319

  committed
  f474a63

  Copy full SHA for f474a63

• fixed bandwidth computation

  Chi-Chu319

  committed
  ad2daad

  Copy full SHA for ad2daad

• fixed bandwidth calculation and config matmul intra dim

  Chi-Chu319

  committed
  26b17e4

  Copy full SHA for 26b17e4

Commits on Jan 10, 2025

• First and second gemm odel benchmarking

  Chi-Chu319

  committed
  30a488c

  Copy full SHA for 30a488c

• Merge branch 'tianxing/moe-gemm' into tianxing/moe-quantization

  Chi-Chu319

  committed
  2eddc52

  Copy full SHA for 2eddc52

• reversed k n

  Chi-Chu319

  committed
  39eca09

  Copy full SHA for 39eca09

• pre commit

  Chi-Chu319

  committed
  0da016e

  Copy full SHA for 0da016e

• pre-commit fix format

  Chi-Chu319

  committed
  be6520b

  Copy full SHA for be6520b

• pre commit fix

  Chi-Chu319

  committed
  54d207d

  Copy full SHA for 54d207d

• pre commit fix

  Chi-Chu319

  committed
  2fe49dd

  Copy full SHA for 2fe49dd

• pre commit

  Chi-Chu319

  committed
  6959222

  Copy full SHA for 6959222

• Merge branch 'tianxing/moe-gemm' into tianxing/moe-quantization

  Chi-Chu319

  committed
  4469329

  Copy full SHA for 4469329

Commits on Jan 15, 2025

• Merge branch 'main_perf' into tianxing/moe-quantization

  Chi-Chu319

  committed
  44baf29

  Copy full SHA for 44baf29