Code

model: support arch DbrxForCausalLM#6515

Code

Merged

phymbert merged 81 commits intomasterggerganov/llama.cpp:masterfrom hp/model/support-dbrxggerganov/llama.cpp:hp/model/support-dbrxCopy head branch name to clipboardApr 13, 2024

+427-147

Commits

Commits on Apr 6, 2024

• model: dbrx convert to gguf

  Show description for 1d8de31
  phymbert

  committed
  1d8de31

• llama: support dbrx

  Show description for ed582c1
  phymbert

  committed
  ed582c1

• gguf-py: remove wrong clip -> clamp

  phymbert

  committed
  3e3d2d1

• model: dbrx, trust remote code

  phymbert

  committed
  3937100

• llama: add label for model 132B

  phymbert

  committed
  c0beb3c

• model: dbrx fix python linter in convert-hf-to-gguf.py

  phymbert

  committed
  0921033

• llama: support dbrx fix norm type

  phymbert

  committed
  e4f8ee4

• dbrx: minor

  phymbert

  committed
  a7f9a3e

• convert: dbrx: fix mixed up and down expert tensors

  phymbert

  committed
  e3c1e81

• convert: dbrx: fix mixed up and down expert tensors

  Show description for 0a35f58
  phymbert

  committed
  0a35f58

• doc: dbrx: add the model as supported

  phymbert

  committed
  c8e6f90

• convert: dbrx: fix remove wrong ATTN_OUT_NORM tensor, add output layer mapping

  phymbert

  committed
  916b918

• llama: dbrx: remove wrong attn output layer in model arch

  phymbert

  committed
  03da419

• scripts: get-wikitext-2 add unzip

  phymbert

  committed
  76f266b

• llama: dbrx: no attention output layer

  phymbert

  committed
  9c7dedb

• model: dbrx: fix missing embedding tensor, mix with output layer

  phymbert

  committed
  fe80898

• llama: dbrx: remove not existing condition on empty output layer

  phymbert

  committed
  4f12a58

• Merge remote-tracking branch 'origin/master' into hp/model/support-dbrx

  phymbert

  committed
  6985629

• llama: dbrx: remove unnecessary optional tensor on FFN_GATE_EXPS

  phymbert

  committed
  7e7cd53

Commits on Apr 7, 2024

• llama: increase maximum experts allowed

  phymbert

  committed
  52c4033

• model: dbrx: convert add n_ff

  phymbert

  committed
  06a59ab

• llama: dbrx: quantize fix n_attention_wv tensor name

  phymbert

  committed
  305ac3b

• model: dbrx: convert fix tokenizer

  phymbert

  committed
  b6522a9

• llama: dbrx: quantize fix n_attention_wv tensor name

  phymbert

  committed
  dccb012

• model: convert-hf-to-gguf.py add _set_vocab_tiktoken gpt2 backed on llama.cpp

  phymbert

  committed
  61be4b9

• model: convert-hf-to-gguf.py fix classname conflict with qwen2

  phymbert

  committed
  1fb6d95

• model: dbrx: convert-hf-to-gguf.py fix fix ftype missing, fix tensor names does not suffix with .weight

  phymbert

  committed
  200ce21

• model: dbrx: convert-hf-to-gguf.py add chat template

  phymbert

  committed
  9e17dad

• llama: quantize: remove wrong look for tensor qkv name as it was badly missing the .weight suffix

  phymbert

  committed
  d7546fd

• model: dbrx: convert-hf-to-gguf.py fix 'token_embd.weight' has wrong shape, fix special tokens

  phymbert

  committed
  3a9dc2e

• model: dbrx: convert-hf-to-gguf.py support python 3.8

  phymbert

  committed
  8154617

• llama: dbrx: no weight suffix in ffn_gate_exps, ffn_up_exps and ffn_down_exps. Output tensor not optional.

  phymbert

  committed
  2449ef4

• llama: quantize: remove wrong look for tensor qkv name as it was badly missing the .weight suffix

  Show description for 1bd9427
  phymbert

  committed
  1bd9427

• llama: dbrx: fix tensor qkv number of elements

  phymbert

  committed
  e9987c6

• model: dbrx: convert reshape expert tensors to 3D

  phymbert

  committed
  d151d8f

• model: dbrx: convert experts to f16

  phymbert

  committed
  f062b83

• model: dbrx: fix tensor names mapping broken

  phymbert

  committed
  dbfd591

• model: dbrx: fix expert reshape

  phymbert

  committed
  7dd84b0

• model: dbrx: fix expert reshape

  phymbert

  committed
  c9bddbf

• model: dbrx: fix again sic expert reshape

  phymbert

  committed
  e2c9199

• model: dbrx: weird fix expert reshape

  phymbert

  committed
  50b4373

• llama: dbrx: output norm dim

  phymbert

  committed
  0ab1bae

• llama: dbrx: fix last normalization

  phymbert

  committed
  830e46d

• llama: dbrx: revert

  phymbert

  committed
  2897aa6

• llama: dbrx: move norm2 after attention, fix build kv

  phymbert

  committed
  993f836

• llama: dbrx: fix build kv att out

  phymbert

  committed
  b01b062

• llama: dbrx: fix build kv att out tensor name

  phymbert

  committed
  74e6d87

• llama: dbrx: hardcode nn.LayerNorm epsilon

  phymbert

  committed
  f8f97e7

• llama: dbrx: Try another rope type

  phymbert

  committed
  71f9e47

Commits on Apr 8, 2024

• llama: dbrx: fix k scale

  phymbert

  committed
  52c6276

• llama: dbrx: move norm epsilon to convert. Fix missing normalization.

  phymbert

  committed
  8e22688

• llama: dbrx: rename tensor to actual meaning. Fix normalization in graph. Permute expert tensors to the llama.cpp layout

  phymbert

  committed
  35dce3e

• llama: dbrx: convert remove previous reverse

  phymbert

  committed
  506cc2e

• llama: dbrx: load norm eps in hparams

  phymbert

  committed
  eb0847e

• llama: dbrx: fix experts tensor layout

  phymbert

  committed
  81f308a

• model: dbrx: convert-hf-to-gguf.py fix experts tensors shapes

  phymbert

  committed
  21fb24a

• llama: factorize moe graph implementation between grok, mixtral and dbrx

  phymbert

  committed
  f20c04f

• model: dbrx convert permute experts directly torch, log shape

  phymbert

  committed
  48909ed

• llama: dbrx: fix experts 3D tensor layout (again)

  phymbert

  committed
  18a84fe

• llama: dbrx: fix experts 3D tensor layout (again)

  phymbert

  committed
  9968952

• llama: dbrx: document changes, permute only FFN_DOWN_EXPS. Add a check for ftype

  phymbert

  committed
  e66f1e3

• llama: dbrx: rename layer_out_norm to attn_out_norm

  phymbert

  committed
  f30a73b

• llama: dbrx: first add the residuals and then do the norm

  phymbert

  committed
  ea8b58c

• model: dbrx: convert fix mixed ffn_gate_exps and ffn_down_exps

  phymbert

  committed
  55943a2

• llama: dbrx: fix ggml context of the attention outputs weight

  phymbert

  committed
  c7b9a2e

• gguf-py: revert spaces

  phymbert

  committed
  ac82aa0

Commits on Apr 9, 2024

• gguf-py: dbrx: reverse again the MOE tensors mapping:

  Show description for ac75fbd
  phymbert

  committed
  ac75fbd

• Merge remote-tracking branch 'origin/master' into hp/model/support-dbrx

  phymbert

  committed
  e5631cf

Commits on Apr 10, 2024

• Merge remote-tracking branch 'origin/master' into hp/model/support-dbrx

  phymbert

  committed
  6f813dc

• llama: dbrx: use the MOE naming convention for model type

  phymbert

  committed
  74529e5

Commits on Apr 11, 2024

• Merge remote-tracking branch 'origin/master' into hp/model/support-dbrx

  phymbert

  committed
  06527c6

Commits on Apr 12, 2024

• model: convert-hf-to-gguf.py remove tiktoken

  phymbert

  committed
  fc89fee

• Is silu activation function applied to MODEL_TENSOR.FFN_GATE_EXP here? If so, we must change this to w1 for DBRX. Each expert in DBRX has 3 linear layers: w1, v1 and w2. For an input tensor x, outp…

  Show description for bdc4efe
  

  phymbert

  and

  megha95

  authored
  bdc4efe

• Is silu activation function applied to MODEL_TENSOR.FFN_GATE_EXP here? If so, we must change this to w1 for DBRX. Each expert in DBRX has 3 linear layers: w1, v1 and w2. For an input tensor x, outp…

  Show description for 542585f
  

  phymbert

  and

  megha95

  authored
  542585f

• Wrong input was being fed to moe layer. This needs to be corrected

  Show description for ecbfb1b
  

  phymbert

  and

  megha95

  authored
  ecbfb1b

• eval-callback: also print last n elements of each dimension

  phymbert

  committed
  647a11b

• minor spaces

  phymbert

  committed
  03bdc36

• convert: update comment of MOE tensors mapping

  phymbert

  committed
  8e6758f

• llama: rename build_moe to build_moe_ffn and fix grok is using gelu instead of silu.

  Show description for f1256dc
  phymbert

  committed
  f1256dc

• convert-hf-to-gguf.py: fix python linter

  phymbert

  committed
  e517585

Commits on Apr 13, 2024

• minor: fix indent in llama_build_graph

  phymbert

  committed
  9f77484