Implement VirtIO sound device playback

[Cuda-Chen]

Implement VirtIO sound device supporting these operations (the item with checked box checked means it is implemented right now):

• For setting up the device:
  • VIRTIO_SND_R_JACK_INFO
  • VIRTIO_SND_R_PCM_INFO
  • VIRTIO_SND_R_CHMAP_INFO
• For playing the sound (PCM):
  • VIRTIO_SND_R_PCM_SET_PARAMS
  • VIRTIO_SND_R_PCM_PREPARE
  • VIRTIO_SND_R_PCM_RELEASE
  • VIRTIO_SND_R_PCM_START
  • VIRTIO_SND_R_PCM_STOP

Limitations

1. The emulator will hang if PulseAudio is enabled on host.
   It causes to CNFA and CNFA cannot initialize if PulseAudio
   is disabled, What's worse, it needs re-working as CNFA uses
   different threading models between pure ALSA and PulseAudio environment.
   Therefore, users need mitigations (i.e., restart the emulator after
   playing sound) or just wait for the future release.

2. The playback may plays with repeating artifact (for example,
   A "front center" ALSA example sound will sound like "front front
   center"). The root cause is the Linux Kernel and it hasn't got
   fixed even in mainline version.

Test Cases

boot up test

test procedures

1. Execute make check to run semu.
2. Check kernel message (dmesg).

expected results

1. The following message should appear while booting up:

[    4.011962] ALSA device list:
[    4.015962]   #0: Loopback 1
[    4.015962]   #1: VirtIO SoundCard at platform/f4400000.virtio/virtio2

check driver configuration

test procedures

1. Execute aplay -l in emulator.
2. Check the output messages.

expected results

1. The following message should appear after Step 1:

$ aplay -l          
**** List of PLAYBACK Hardware Devices ****

<other sound device here>

card 1: SoundCard [VirtIO SoundCard], device 0: virtio-snd [VirtIO PCM 0]
  Subdevices: 1/1
  Subdevice #0: subdevice #0

play sound

test procedures

1. Execute speaker-test in emulator.
2. Check the host speaker.

expected results

1. A white noise will be played by host speaker while speaker-test is executing.

play sequence sound

test procedures

1. Execute aplay /usr/share/sounds/alsa/Front_Center.wav in emulator.
2. Check the host speaker.

expected results

1. A sound with "Front Center" speaking should be played.

[jserv]

Run clang-format before submitting.

[jserv]

Consult https://github.com/cntools/cnfa/blob/master/.github/workflows/build-cnfa.yml and mention the build dependency in top-level README.md.

[Cuda-Chen]

Hi @jserv ,
After playing the sound for a whole month, I consider that there might be a pitfall of my implementation of playing sound or the Linux Kernel version this project is using.

The virtio-snd plays the sound with repeating artifact (taking "front center" as example, it sounds like "front front center").

Kindly leave some thoughts for some references or some folks of dealing with repeating artifact if I had a chance, and it will be a great thanks for your help!

If you would like to get the file, I will convert the file into base64 encoded text in order to upload to here.

[jserv]

If you would like to get the file, I will convert the file into base64 encoded text in order to upload to here.

Simply share the prebuilt Linux kernel image file, rootfs, and test files via Google Drive, Dropbox, or similar services as they are intended to be temporary.

[Cuda-Chen]

Hi @jserv ,

This Dropbox share link contains the files that reviewers will need: https://www.dropbox.com/scl/fo/vv3luiccfco5tjwt8tqxm/AANAyYgL9qvVPhuT4dtkkIA?rlkey=dv84qtw9y4e0ea8go03q5qwf8&st=73j9agts&dl=0

The usage of files are described as follows:

1. Image: guest Linux OS image.
2. rootfs.cpio: disk image.
3. out_stream.bin: a binary dump of output stream (i.e., the Front_Center.wav of ALSA sound example). You can use aplay to play the file like this: cat out_stream.bin | aplay --format=S16_LE --rate=48000 -.

[jserv]

Tested on macOS + Apple M1:

# aplay -l
**** List of PLAYBACK Hardware Devices ****
card 0: SoundCard [VirtIO SoundCard], device 0: virtio-snd [VirtIO PCM 0]
  Subdevices: 1/1
  Subdevice #0: subdevice #0
# aplay /usr/share/sounds/alsa/Front_Center.wav
Playing WAVE '/usr/share/sounds/alsa/Front_Center.wav' : Signed 16 bit Little Endian, Rate 48000 Hz, Mono
CoreAudio: from: [O/I] (null)/(null) (semu-virtio-snd) / (48000,0)x(1,0) (4800)

Then, it got stuck without progress.

[Cuda-Chen]

Then, it got stuck without progress.

Thanks for helping the tests on macOS. I guess it is some kind of locking issue.
While trying to fix, I would like to ask whether a "front center" sound comes out from host or not while executing aplay /usr/share/sounds/alsa/Front_Center.wav?

For comparison on Linux + X86 it runs like this:

# aplay /usr/share/sounds/alsa/Front_Center.wav 
Playing WAVE '/usr/share/sounds/alsa/Front_Center.wav' : Signed 16 bit Little Endian, Rate 48000 Hz, Mono
CNFA Alsa Init (nil) (nil)  (48000 0) 1 0
0x55f41abed1d0 (nil)
PERIOD: 4800  BUFFER: 14400
CNFA Alsa Init Out -> 0x55f41abed1d0 (nil)  (48000 0) 1 0
(... play "front front center" then stuck in aplay program)

[jserv]

Consider the change below to suppress compilation warnings raised by CNFA:

--- a/Makefile
+++ b/Makefile
@@ -85,6 +85,7 @@ CNFA_LIB := cnfa/CNFA_sf.h
 $(CNFA_LIB): cnfa/Makefile cnfa/os_generic
        $(MAKE) -C $(dir $<) CNFA_sf.h
 main.o: $(CNFA_LIB)
+virtio-snd.o: CFLAGS += -Wno-unused-parameter
 endif
 
 # .DEFAULT_GOAL should be set to all since the very first target is not all

[Cuda-Chen]

Hi @jserv ,

Then, it got stuck without progress.

After checking the CNFA part, the coreaudio just like the PulseAudio/PipeWire part: no need to use a dedicated thread for pushing PCM frames.

I would like to ask which one you prefer for resolving this issue:

1. Exclude macOS support.
   • No need to change the threading model of pushing PCM.
2. Use PulseAudio/PipeWire solution.
   • Threading module of pushing PCM frames need to be re-designed.