Skip to content

Commit

Permalink
Add info about CUDA_VISIBLE_DEVICES (#1682)
Browse files Browse the repository at this point in the history
  • Loading branch information
SlyEcho authored Jun 3, 2023
1 parent b5c8546 commit d8bd001
Showing 1 changed file with 3 additions and 1 deletion.
4 changes: 3 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -310,6 +310,8 @@ Building the program with BLAS support may lead to some performance improvements
```
Note: Because llama.cpp uses multiple CUDA streams for matrix multiplication results [are not guaranteed to be reproducible](https://docs.nvidia.com/cuda/cublas/index.html#results-reproducibility). If you need reproducibility, set `GGML_CUDA_MAX_STREAMS` in the file `ggml-cuda.cu` to 1.
The environment variable [`CUDA_VISIBLE_DEVICES`](https://docs.nvidia.com/cuda/cuda-c-programming-guide/index.html#env-vars) can be used to specify which GPU(s) will be used.
- **CLBlast**
OpenCL acceleration is provided by the matrix multiplication kernels from the [CLBlast](https://github.com/CNugteren/CLBlast) project and custom kernels for ggml that can generate tokens on the GPU.
Expand Down Expand Up @@ -348,7 +350,7 @@ Building the program with BLAS support may lead to some performance improvements
cmake --install . --prefix /some/path
```

Where `/some/path` is where the built library will be installed (default is `/usr/loca`l`).
Where `/some/path` is where the built library will be installed (default is `/usr/local`).
</details>

Building:
Expand Down

0 comments on commit d8bd001

Please sign in to comment.