This post continues my adventure with KVM/QEMU and PCI passthrough on Debian. The previous post from 2018 „Using a PCI graphics card in KVM/QEMU on Debian Stretch“ contains more information on the whole process, while this post has a stronger focus on QEMU compilation and sound with PulseAudio.
According to various forums posts, discussions and wiki articles, KVM/QEMU setups have a long history of problems with sound. In my 2018 configuration passing through the PulseAudio socket worked wonders with a Windows guest. I still don't know why it worked relatively good, given reviews from others who used the same feature. Yet, with the upgrade to Debian Buster (Debian 10), sound problems came back. I could kinda ease crackling sound by experimenting with buffer sizes and closing any CPU-intensive tasks on the host, but the quality was way worse than on Stretch.
So, recently I finally had enough and looked into the issue again. To my surprise, with the release of QEMU 4.2 in December 2019 came the integration of the new audiodev patches and new PulseAudio parameters. More on that in Gerd Hoffmann's post „sound configuration changes in qemu“. Due to their age these binaries are not included in Buster. What is left to do? Compile QEMU ourselves! And not to spoil too much – it's an easy task and the result works perfectly! Going on, the setup consists of Debian Buster, source files for QEMU 4.2 and a Docker container.
Building QEMU v4.2.0 from tar or git
You can either choose to build QEMU on your native host system, or use a container as I did. I prefer to use containers for compiling, because dependency management and traceability of new files are vastly improved in encapsulated environments.
Since my target system will be Debian, the process is done in a
debian:buster-slim Docker container. This way, all libraries are in sync with the Debian-based host. In an earlier trial run, an Arch Linux image was used but resulted in a library error – Arch referenced a newer dynamic library located in
/usr/lib inside the build.
As recommended in the QEMU wiki, we will generate an out-of-tree build.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23
Note that instead of downloading the .tar.xz you can also clone the repository with git.
1 2 3 4 5 6 7
By default, the cloned repository uses the master branch, which contains the latest commits to QEMU. Switching to the v4.2.0 tag produces the same files as the downloaded .tar.xz. But using git also offers you to compile the most recent development version, if you wish to experiment further. Given that full support for audiodev was finalized in v4.2.0 there will surely be new improvements coming in in later releases. Staying on the bleeding edge gives direct access to fresh commits.
Your usage of configure flags might vary. Use
configure -h to get the full list. Here, the following features are enabled explicitly:
- kvm: enable KVM support
- tools: build tools like qemu-img
- spice: enable SPICE support
- vnc: enable VNC support
- sdl: enable SDL support
- libusb and usb-redir: enable USB passthrough support
- seccomp: enable seccomp
- linux-aio: enable Linux async I/O support (here: used for LVM)
- avx2: enable AVX2 CPU flag support
For further information about the used configure flags above, read the QEMU docs and the Gentoo QEMU wiki page.
Create new emulator wrapper and add AppArmor rules
Now the new binary must be integrated into libvirt. The following script is saved as
/usr/local/bin/kvm-custom and made executable with
chmod +x. It's a copy of
/usr/bin/kvm which comes with the qemu-kvm package on Debian, adapted to use the compiled binary instead of the one in /usr/bin.
If you do not use the qemu-kvm compatibility package, your emulator binary might be referenced in other ways, for example as
<emulator>/usr/bin/qemu-system-x86_64</emulator> in libvirt. In that case, change the path or create the wrapper as well, same if you wish to remove your packaged QEMU installation. The correct configuration depends on your individual setup. AFAIK libvirt will enable KVM support to QEMU automatically if the domain is of type kvm.
Regarding executables in Debian Buster, AppArmor now watches over applications by default so we must configure the libvirt profile with new permissions. For this, edit
/etc/apparmor.d/abstractions/libvirt-qemu as root. Add the share folder with read permission (BIOS files etc.) and the two binaries kvm-custom and qemu-system-x86_64 with read/mmap/inherit/execute permissions in between the existing entries.
1 2 3 4 5 6 7 8 9 10 11
Reload apparmor to apply the changes.
Updating libvirt's virtual machine config
In the libvirt domain config, remove all existing
QEMU_ env variables which are related to audio (for example QEMU_AUDIO_DRV, QEMU_PA_SERVER), remove any sound device (ICH6/ICH9) and then add the new commandline arguments below. The -audiodev parameter is what we are after in our mission to fix PulseAudio sound in VMs. Here's what needs to be changed in the libvirt domain XML configuration for the guest machine:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23
Starting the VM should then add a new client in your PulseAudio panel (most probably called snd0, can be changed with the audiodev parameter
out.stream-name=<name>), which volume level is adjustable independently. Inside the guest, check that the sound device uses 48kHz to match the audiodev. Other issues might be worked out by using other parameters like out.frequency, out.channels and out.buffer-length. See the docs below for more information.
Good luck and I hope this process improves your audio experience as well!
Sources for this article:
- QEMU wiki: QEMU on Linux hosts and Changelog 4.2
- QEMU docs: QEMU System Emulation User’s Guide
- Stefan Weil: QEMU version 4.2.0 User Documentation
- gentoo wiki: QEMU/Options
- kraxel.org: sound configuration changes in qemu (Jan 2020 for QEMU 4.2)
- heiko-sieger.info: #Qemu 4.0 Audio Improvements (Updated 2020)
- /r/VFIO: QEMU 4.0 due soon - might bring superb audio - test now! (by u/spheenik)
- Arch Linux Wiki: Passing VM audio to host via PulseAudio
- KVM Forum 2012: VFIO: A user's perspective (historic)
Other useful tools and future work:
- Looking Glass, „Looking Glass is an open-source application that allows the use of a KVM (Kernel-based Virtual Machine) configured for VGA PCI Pass-through without an attached physical monitor, keyboard or mouse. This is the final step required to move away from dual booting with other operating systems for legacy programs that require high-performance graphics.“
- ddcutil, „ddcutil is a Linux program for managing monitor settings, such as brightness, color levels, and input source. Generally speaking, any settings that can be changed by pressing buttons on the monitor can be modified by ddcutil.“, can be used to automatically switch monitor input on VM start.