Skip to content

Commit

Permalink
Remove DPSII patches
Browse files Browse the repository at this point in the history
  • Loading branch information
sasha0552 authored Dec 28, 2024
1 parent c0c47c9 commit abc1f2e
Show file tree
Hide file tree
Showing 4 changed files with 11 additions and 272 deletions.

This file was deleted.

This file was deleted.

This file was deleted.

22 changes: 11 additions & 11 deletions airootfs/home/tori/.local/share/tori/patches/README.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
> [!IMPORTANT]
> The DPSII patches have been superseded by the `nvidia-pstated` daemon, which can be found [here](https://github.com/sasha0552/nvidia-pstated). The daemon does not require application patching and switches performance states automatically.
> **Patches in this directory will be removed at the end of this year.**
> **DPSII patches in this directory will be removed at the end of this year.**
# ToriLinux patches

Expand All @@ -23,22 +23,22 @@ Don't forget to recompile the project if necessary!

## Patch types

### Drop PState In Idle (DPSII)
### ~~Drop PState In Idle (DPSII)~~

These patches can be used to automatically manage the performance states of NVIDIA GPUs using the [nvidia-pstate](https://github.com/sasha0552/nvidia-pstate).
~~These patches can be used to automatically manage the performance states of NVIDIA GPUs using the [nvidia-pstate](https://github.com/sasha0552/nvidia-pstate).~~

Remember that you need to install `nvidia-pstate` in the virtual environment (if applicable), and/or add the executable to the system PATH.
*Currently only `llama.cpp` uses `nvidia-pstate` executables from PATH. Other projects use `nvidia-pstate` as a library.*
~~Remember that you need to install `nvidia-pstate` in the virtual environment (if applicable), and/or add the executable to the system PATH.
*Currently only `llama.cpp` uses `nvidia-pstate` executables from PATH. Other projects use `nvidia-pstate` as a library.*~~

This can reduce power consumption/temperatures, especially on server GPUs (note that SXM versions of server GPUs may not work).
~~This can reduce power consumption/temperatures, especially on server GPUs (note that SXM versions of server GPUs may not work).~~

For example, on my NVIDIA Tesla P40, these patches reduce power consumption from 50W to 10W (and temperature from 50C to ~25C (room temperature)) by setting the performance state to 8 when the application is idle. When a request comes in, it restores the driver behavior, so inference can run as usual.
~~For example, on my NVIDIA Tesla P40, these patches reduce power consumption from 50W to 10W (and temperature from 50C to ~25C (room temperature)) by setting the performance state to 8 when the application is idle. When a request comes in, it restores the driver behavior, so inference can run as usual.~~

This type of patch is currently available for these projects:
~~This type of patch is currently available for these projects:~~

* [automatic](https://github.com/vladmandic/automatic)
* [llama.cpp](https://github.com/ggerganov/llama.cpp)
* [vllm](https://github.com/vllm-project/vllm)
* ~~[automatic](https://github.com/vladmandic/automatic)~~
* ~~[llama.cpp](https://github.com/ggerganov/llama.cpp)~~
* ~~[vllm](https://github.com/vllm-project/vllm)~~

### There were a few other patches earlier ...

Expand Down

0 comments on commit abc1f2e

Please sign in to comment.