Code

Make loading weights 10-100x faster #613

Code

Merged

jart merged 9 commits intoggml-org:masterggml-org/llama.cpp:masterfrom jart:loaderjart/llama.cpp:loaderCopy head branch name to clipboardMar 30, 2023

+717-328

Commits

Commits on Mar 29, 2023

• Add mmap support for model files

  

  slaren

  authored and

  jart

  committed
  2a6cef6

  Copy full SHA for 2a6cef6

• Fix ggml_init_params in quantize

  

  slaren

  authored and

  jart

  committed
  a1e0f17

  Copy full SHA for a1e0f17

• Make mmap_file static

  

  slaren

  authored and

  jart

  committed
  4ae12d0

  Copy full SHA for 4ae12d0

• Unmap the file in llama_free

  

  slaren

  authored and

  jart

  committed
  4daaa5e

  Copy full SHA for 4daaa5e

• Always initialize mm_addr and mm_length in llama_model

  

  slaren

  authored and

  jart

  committed
  812cfa1

  Copy full SHA for 812cfa1

• Initial windows support (untested)

  

  slaren

  authored and

  jart

  committed
  80c2178

  Copy full SHA for 80c2178

Commits on Mar 30, 2023

• Make loading weights 10-100x faster

  Show description for 75d1e55
  jart

  committed
  75d1e55

  Copy full SHA for 75d1e55

• Ensure --mlock works properly with mmap() support

  jart

  committed
  a45e843

  Copy full SHA for a45e843

• Introduce GGML migration tool for new file format

  Show description for 516474b
  jart

  committed
  516474b

  Copy full SHA for 516474b