High latency when using PulseAudio, due to missing pa_buffer_attr

[nyanpasu64]

I'm using RtAudio (set to have 1-2 blocks of 64 samples) to output to PulseAudio (set to have two 6ms buffers), but getting bad latency (I estimate to be around 100 milliseconds).

RtAudio mutexes

#133 (comment)

The Pulse support in RtAudio has been fairly minimal. It was contributed by a user and has never been extensively investigated by myself or others to bring it up to what I might consider "full" support. The ALSA support, on the other hand, is (or at least was) robust and includes "realtime" functionality that was tested and verified.

The PulseAudio backend uses mutexes in the audio generation callback, which I've heard is unacceptable in real-time audio. It would probably take days for me to understand RtAudio, understand why the mutexes are needed, and remove them. So I'll focus on the latency for now. OpenMPT's PulseAudio support doesn't use mutexes, but tends to lock up the entire program.

My application

My own application generates audio in blocks of 64 samples. When I check pa_simple_get_latency() before calling pa_simple_write(), the latency rises monotonically from 0 to around 230000 μs in steps of 1333µs (the sound callback is being called nonstop for 64 samples at a time), before it stabilizes (pa_simple_write blocks and waits for PulseAudio to drain audio). For some reason, pa_simple_write won't block before then.

RtApiPulse sets stream_.nBuffers = 1; and writes that to options->numberOfBuffers. This is a lie; in reality, at 64-sample blocks, the number fluctuates around 170 (around 230000 / 1333). options->numberOfBuffers's initial value (set by the user) is ignored in all backends except ALSA.

OpenMPT PulseAudio

OpenMPT with native PulseAudio behaves weirdly. (source code)

Apparently in PulseAudio mode, OpenMPT's "exclusive mode" checkbox is renamed to "Adjust latency" but still controls m_Settings.ExclusiveMode (which no longer toggles exclusive mode).

The OpenMPT Setup dialog has a Sound Card tab which displays current latency (source code).

With "Adjust latency" checked, m_Settings.ExclusiveMode is true, pa_simple_new(...pa_buffer_attr) is passed a non-null pointer, and subjective latency is fairly low. The dialog shows "current: 40ms" or less of latency.

With "Adjust latency" unchecked, m_Settings.ExclusiveMode is false, pa_simple_new(...pa_buffer_attr) is passed NULL, and subjective latency is as bad as RtAudio. The dialog shows "current: 250ms" of latency. This equals pa_simple_get_latency plus the buffer size (20ms). 230ms matches the latency in RtAudio.

Problem and Solution

If you pass a null pa_buffer_attr* to pa_simple_new(), then PulseAudio will run your callback too many times and buffer nearly 250ms of input in the steady-state.

To avoid this latency, RtAudio should create a pa_buffer_attr, and populate it using options->numberOfBuffers. I haven't figured out the details yet. Can I submit a PR once I've figured it out?

Discussion

PulseAudio is designed to make low latency tricky to achieve and easy to break by mistake (you can get working sound with terrible latency). To get low latency, you have to reduce the callback chunk size in your app, reduce the output chunk size in the pulseaudio server, and turn off excessive queueing in the pulseaudio api.

Someone mentioned, "Pulseaudio is widely criticized for its complexity and latency."

Is alsa dmix usable for sharing one device among multiple apps? Does it "Just Work" without careful tweaking? When I kill PulseAudio and try using RtAudio in ALSA mode, it connects to the wrong device and won't output audio.

Is PulseAudio inherently inappropriate for low-latency or real-time audio (locks or allocations)? https://gavv.github.io/articles/pulseaudio-under-the-hood/#native-protocol describes that it uses a "Unix domain or TCP stream socket", and has a "zero-copy mode" based on shared memory file descriptors and mmap.