Pulseaudio and latency
If you are a Linux enthusiast and are playing fps games or record audio of some musical instrument then you have probably encountered issues with Pulseaudio latency.
This blog post describes how to live with and manage Pulseaudio latency problems. For fast answer scroll directly to chapter Positive effect on latency as bellow is some leading information about Linux sound architecture
end of TL;DR so now go and read…
Linux sound architecture
Linux kernel has snd base module and on top of that other sound related modules can be built.
$ lsmod|grep ^snd|cut -d' ' -f1 snd_usb_audio snd_usbmidi_lib snd_rawmidi snd_seq_device snd_hda_codec_hdmi snd_hda_codec_conexant snd_hda_codec_generic snd_hda_intel snd_hda_controller snd_hda_codec snd_hwdep snd_pcm snd_timer snd
Some of those modules creates log entries during activity or when loaded.
$ dmesg|grep 'snd\|card'|cut -d']' -f2 snd_hda_intel 0000:00:1b.0: irq 45 for MSI/MSI-X input: HDA Digital PCBeep as /devices/pci0000:00/0000:00:1b.0/sound/card0/hdaudioC0D0/input12 input: HDA Intel PCH Mic as /devices/pci0000:00/0000:00:1b.0/sound/card0/input13 input: HDA Intel PCH Headphone as /devices/pci0000:00/0000:00:1b.0/sound/card0/input14 input: HDA Intel PCH HDMI/DP,pcm=3 as /devices/pci0000:00/0000:00:1b.0/sound/card0/input15 usbcore: registered new interface driver snd-usb-audio
Basically ALSA is just a bunch of kernel modules and sound card drivers. There is also useful command line utilities to make life easier with those kernel modules.
Note: There is also other sound systems present in Linux world but ALSA is the most used currently.
Outputs are called as playback devices in ALSA.
$ aplay -l **** List of PLAYBACK Hardware Devices **** card 0: PCH [HDA Intel PCH], device 0: CX20590 Analog [CX20590 Analog] Subdevices: 1/1 Subdevice #0: subdevice #0 card 0: PCH [HDA Intel PCH], device 3: HDMI 0 [HDMI 0] Subdevices: 1/1 Subdevice #0: subdevice #0 card 1: CODEC [USB Audio CODEC], device 0: USB Audio [USB Audio] Subdevices: 1/1 Subdevice #0: subdevice #0 card 2: Device [USB Sound Device], device 0: USB Audio [USB Audio] Subdevices: 0/1 Subdevice #0: subdevice #0
Inputs are called as capture devices in ALSA.
$ arecord -l **** List of CAPTURE Hardware Devices **** card 0: PCH [HDA Intel PCH], device 0: CX20590 Analog [CX20590 Analog] Subdevices: 1/1 Subdevice #0: subdevice #0 card 1: CODEC [USB Audio CODEC], device 0: USB Audio [USB Audio] Subdevices: 0/1 Subdevice #0: subdevice #0 card 2: Device [USB Sound Device], device 0: USB Audio [USB Audio] Subdevices: 1/1 Subdevice #0: subdevice #0
Streams in general
Streams in computing are constructed using sources, flows, and sinks.
Source > Flow > Sink
For example audio data stream has a source (like sound card input) and sink (like sound card output) and between there is flow which modifies audio data. This modifying in flow can be as simple as gain control or delay control or more advanced like DSP effects.
Pulseaudio is stream based. Both sources and sinks are present in terminology as well. Pulseaudio is also module based.
As Pulseaudio is not kernel based and does not introduce new kernel modules, some other way of managing audio data streams should be present. One example of these modules in this example setup is module-alsa-card which provides capability of interacting with ALSA hardware sound cards.
ALSA card’s inputs are presented as sources in Pulseaudio.
$ pactl list sources short 0 alsa_input.usb-Burr-Brown_from_TI_USB_Audio_CODEC-00-CODEC.analog-stereo module-alsa-card.c s16le 2ch 44100Hz RUNNING 1 alsa_output.usb-0d8c_USB_Sound_Device-00-Device.iec958-stereo.monitor module-alsa-card.c s16le 2ch 44100Hz IDLE 2 alsa_output.pci-0000_00_1b.0.analog-stereo.monitor module-alsa-card.c s16le 2ch 44100Hz SUSPENDED
ALSA card’s outputs are presented as sinks in Pulseaudio.
$ pactl list sinks short 0 alsa_output.usb-0d8c_USB_Sound_Device-00-Device.iec958-stereo module-alsa-card.c s16le 2ch 44100Hz RUNNING 1 alsa_output.pci-0000_00_1b.0.analog-stereo module-alsa-card.c s16le 2ch 44100Hz SUSPENDED
Latency and buffering
Every party described above adds own latency footprint into system. Linux kernel modules are generally well performing and kernel-level latency is often low enough. It is buffering between parties which causes major impact to latencies.
There is several ways to measure Pulseaudio latency. You cannot get rid of latency completely but sometimes default buffer sizes are too large and therefore cause worthless latency.
$ pactl list sinks Sink #0 State: RUNNING Name: alsa_output.usb-0d8c_USB_Sound_Device-00-Device.iec958-stereo Description: CM106 Like Sound Device Digital Stereo (IEC958) Driver: module-alsa-card.c Sample Specification: s16le 2ch 48000Hz Channel Map: front-left,front-right Owner Module: 7 Mute: no Volume: front-left: 65536 / 100% / 0.00 dB, front-right: 65536 / 100% / 0.00 dB balance 0.00 Base Volume: 65536 / 100% / 0.00 dB Monitor Source: alsa_output.usb-0d8c_USB_Sound_Device-00-Device.iec958-stereo.monitor Latency: 103758 usec, configured 100000 usec Flags: HARDWARE DECIBEL_VOLUME LATENCY SET_FORMATS
Above sink details shows there is some latency present.
Latency: 103758 usec, configured 100000 usec
There is current latency and configured latency present in every source and sink. You cannot configure latency directly but you can modify variables that affect latency positively.
Modules and scheduling
Several other Pulseaudio modules (other than just module-alsa-card) are present in this example setup.
$ pactl list modules short|cut -d$'\t' -f2 module-device-restore module-stream-restore module-card-restore module-augment-properties module-switch-on-port-available module-udev-detect module-alsa-card module-native-protocol-unix module-default-device-restore module-rescue-streams module-always-sink module-intended-roles module-suspend-on-idle module-position-event-sounds module-role-cork module-filter-heuristics module-filter-apply
Every one of these modules has it’s own configuration parameters. Read more from Pulseaudio modules documentation
Some of the parameters affecting positively to latency are:
- A boolean. If “true”, the sinks and sources of this card will use the timer-based scheduling.
- The number of fragments to be used in the sink and source buffers. Only effective if the timer-based scheduling is disabled.
- The size of one fragment (in bytes) to be used in the sink and source buffers. Only effective if the timer-based scheduling is disabled.
- The total sink and source buffer size in bytes. Only effective if the timer-based scheduling is enabled.
- The buffer fill level (in bytes) at which the sinks must refill the buffer. Only effective if the timer-based scheduling is enabled.
- A boolean. Normally when there’s an alsa underrun or overrun, and timer-based scheduling is used, the alsa sink or source will raise the minimum latency that applications can get to avoid further underruns or overruns. If this option is enabled, the minimum latency will stay constant even if underruns or overruns occur.
Positive effect on latency
There is module module-udev-detect which searches ALSA sound cards present in system and adds own instance of module-alsa-card for each of these cards. Unfortunately default parameter values are not so nice for low-latency setup.
For example one of my USB sound cards has.
$ pactl list modules ... Module #7 Name: module-alsa-card Argument: device_id="2" name="usb-0d8c_USB_Sound_Device-00-Device" card_name="alsa_card.usb-0d8c_USB_Sound_Device-00-Device" namereg_fail=false tsched=yes fixed_latency_range=no ignore_dB=no deferred_volume=yes use_ucm=yes card_properties="module-udev-detect.discovered=1"
I would suggest for you to try.
tsched=no fixed_latency_range=yes fragments=1 fragment_size=15
First, unload the module corresponding your desired sound card, #7 in my setup.
$ pactl unload-module 7
Then load it again with new parameter values.
$ pactl load-module module-alsa-card \ device_id="2" \ name="usb-0d8c_USB_Sound_Device-00-Device" \ card_name="alsa_card.usb-0d8c_USB_Sound_Device-00-Device" \ namereg_fail=false \ tsched=no \ fixed_latency_range=yes \ ignore_dB=no \ deferred_volume=yes \ use_ucm=yes \ card_properties="module-udev-detect.discovered=1" \ fragments=1 \ fragment_size=15
My latency with default values.
Latency: 103444 usec, configured 100000 usec
which is ~103ms, and next after changes.
Latency: 19369 usec, configured 20000 usec
which is ~19ms.
If you audio starts to chop or stutter then your frame size is too small to keep sound card buffer saturated enough. You can play with
fragment_size parameter values to find the best for your setup. Smaller gives lower latency, higher gives better chance to have unchoppy audio playback.
One way to connect a source directly to a sink is using module-loopback.
$ pactl load-module module-loopback latency_msec=1
And for unloading previous module.
$ pactl unload-module module-loopback
The other (and often better what comes to overall latency) way is to use pacat command and piping.
$ pacat -r --latency-msec=1 -d SOURCE-NAME | pacat -p --latency-msec=1 -d SINK-NAME
Use complete source/sink names.
$ pacat -r \ --latency-msec=1 \ -d alsa_input.usb-Burr-Brown_from_TI_USB_Audio_CODEC-00-CODEC.analog-stereo \ | pacat -p \ --latency-msec=1 \ -d alsa_output.usb-0d8c_USB_Sound_Device-00-Device.iec958-stereo