Code

Support for flash attention 2.0#7063

Code

Merged

hsiehjackson merged 10 commits intomainNVIDIA/NeMo:mainfrom sandeepsub/flash_attn_2NVIDIA/NeMo:sandeepsub/flash_attn_2Copy head branch name to clipboardSep 1, 2023

+45-22

Commits

Commits on Jul 18, 2023

• Add flash attn 2

  Show description for 0fad969
  MaximumEntropy

  committed
  0fad969

  Copy full SHA for 0fad969

• [pre-commit.ci] auto fixes from pre-commit.com hooks

  Show description for f09051c
  pre-commit-ci[bot]

  committed
  f09051c

  Copy full SHA for f09051c

Commits on Jul 25, 2023

• Merge branch 'main' into sandeepsub/flash_attn_2

  okuchaiev

  authored
  2c34e68

  Copy full SHA for 2c34e68

Commits on Aug 9, 2023

• Merge branch 'main' into sandeepsub/flash_attn_2

  hsiehjackson

  authored
  cdf2b71

  Copy full SHA for cdf2b71

• Add FA2 feature

  Show description for b98b285
  hsiehjackson

  committed
  b98b285

  Copy full SHA for b98b285

• Remove debugging

  Show description for aa40a67
  hsiehjackson

  committed
  aa40a67

  Copy full SHA for aa40a67

• [pre-commit.ci] auto fixes from pre-commit.com hooks

  Show description for 5964e43
  pre-commit-ci[bot]

  committed
  5964e43

  Copy full SHA for 5964e43

• Merge branch 'main' into sandeepsub/flash_attn_2

  hsiehjackson

  authored
  efc311b

  Copy full SHA for efc311b

Commits on Aug 11, 2023

• Merge branch 'main' into sandeepsub/flash_attn_2

  Show description for 83dc43c
  hsiehjackson

  authored
  83dc43c

  Copy full SHA for 83dc43c

Commits on Aug 31, 2023

• Merge branch 'main' into sandeepsub/flash_attn_2

  hsiehjackson

  authored
  aced0a5

  Copy full SHA for aced0a5