Code

llama : pad KV cache size#4280

Code

Merged

ggerganov merged 2 commits intomasterggerganov/llama.cpp:masterfrom gg/pad-kv-cacheggerganov/llama.cpp:gg/pad-kv-cacheCopy head branch name to clipboardDec 3, 2023

+2-3

Commits

Commits on Dec 1, 2023

• llama : pad KV cache size to 32

  ggerganov

  committed
  3e68df8

• metal : try to improve batched decoding

  ggerganov

  committed
  3cb1c34