PipeWire support

[Be-ing]

PipeWire will soon be replacing both JACK and PulseAudio on Linux starting with Fedora 34 which will be released in a few weeks. Arch currently has packages that make it easy to switch to PipeWire. I am unclear about Ubuntu's plans and just asked them about that.

PipeWire reimplements the JACK and PulseAudio APIs. Currently the PipeWire developer is not recommending to use the new PipeWire API if applications already work well with PipeWire via JACK or PulseAudio. I am not familiar enough with cpal to know, but there might be some benefits to using the PipeWire API instead of using PipeWire via the JACK API.

PipeWire upstream already provides Rust bindings.

I have attempted to test cpal's JACK backend with PipeWire but have not gotten it to work. The pw-uninstalled.sh script sets the LD_LIBRARY_PATH environment variable so PipeWire's reimplementation of libjack is loaded before JACK's original library. I know Rust links statically to other Rust libraries but I am unclear how Rust links to dynamic C libraries. Maybe the LD_LIBRARY_PATH trick does not work with Rust?

cpal on  master is 📦 v0.13.2 via 🦀 v1.49.0 
❯ cargo run --release --example beep --features jack
    Finished release [optimized] target(s) in 0.03s
     Running `target/release/examples/beep`
Output device: default
Default output config: SupportedStreamConfig { channels: 2, sample_rate: SampleRate(44100), buffer_size: Range { min: 170, max: 1466015503 }, sample_format: F32 }
memory allocation of 5404319552844632832 bytes failed
fish: “cargo run --release --example b…” terminated by signal SIGABRT (Abort)

cpal on  master is 📦 v0.13.2 via 🦀 v1.49.0 
❯ ~/sw/pipewire/pw-uninstalled.sh
Using default build directory: /home/be/sw/pipewire/build
Welcome to fish, the friendly interactive shell
Type `help` for instructions on how to use fish

cpal on  master is 📦 v0.13.2 via 🦀 v1.49.0 
❯ cargo run --release --example beep --features jack
    Finished release [optimized] target(s) in 0.03s
     Running `target/release/examples/beep`
Output device: default
Default output config: SupportedStreamConfig { channels: 2, sample_rate: SampleRate(44100), buffer_size: Range { min: 170, max: 1466015503 }, sample_format: F32 }
memory allocation of 5404319552844632832 bytes failed
fish: “cargo run --release --example b…” terminated by signal SIGABRT (Abort)

Related discussions regarding PipeWire and PortAudio: https://gitlab.freedesktop.org/pipewire/pipewire/-/issues/130 https://github.com/PortAudio/portaudio/issues/425

[Be-ing]

The PipeWire developer is very responsive and working hard on PipeWire every day. If anyone is interested in working on this for cpal, I recommend getting in touch with him through PipeWire's issue tracker and/or #pipewire on Freenode.

[ishitatsuyuki]

I'm running Arch + pipewire{,-jack,-pulse} and the jack example works here. What distro do you use?

     Running `target/release/examples/beep`
Output device: default
Default output config: SupportedStreamConfig { channels: 2, sample_rate: SampleRate(44100), buffer_size: Range { min: 3, max: 4194304 }, sample_format: F32 }

[ishitatsuyuki]

Oh, actually I made a mistake; the above one is probably using PipeWire's ALSA integration.

When testing JACK you should run with cargo run --release --example beep --features jack -- --jack.

[ishitatsuyuki]

When I run with pipewire-jack it doesn't crash but doesn't make sound either. Strange.

[ishitatsuyuki]

Seems that we have the same regex problem as PortAudio. Fix incoming.

[Be-ing]

Hopefully it doesn't take 3 days to figure out how to find and replace a string in Rust like it did for me in C :laughing:

[Be-ing]

I recommend reading the PipeWire wiki. There is a lot of good information there.

[ishitatsuyuki]

FYI it's https://github.com/RustAudio/cpal/blob/5cfa09042c629c1ba39bbac1a6ae9278847a9ac1/src/host/jack/stream.rs#L152 and https://github.com/RustAudio/cpal/blob/5cfa09042c629c1ba39bbac1a6ae9278847a9ac1/src/host/jack/stream.rs#L178. But I'm planning to change to a different logic based on mpv's.

[Be-ing]

Hardcoding using the system ports in JACK is definitely not right and won't work with PipeWire at all. I am wondering why that was done. Was it just a quick hack to get something working or does this reflect a bigger problem in the cpal API?

PortAudio's JACK implementation is quite quirky. Normally JACK applications create all their ports which any application communicating with the server can connect to arbitrarily. But PortAudio's abstractions flip that relationship upside down so PortAudio creates a port when the application asks to start a stream; all of the PortAudio application's ports are not available for other applications to connect to until the PortAudio application opens the stream. Does cpal's API have the same problem?

[ishitatsuyuki]

As usual, our development effort is limited and the JACK backend is somewhat unfinished. We do care about making the abstractions not leaky, though.

We register a port when you create a Stream... I hope it's fine? You can then start or stop the Stream through the methods, and stopping isn't tied to closing the port.

https://github.com/RustAudio/cpal/blob/5cfa09042c629c1ba39bbac1a6ae9278847a9ac1/src/host/jack/stream.rs#L37

[Be-ing]

As long as the application can create JACK/PipeWire ports for other applications or the session manager to connect to without the cpal application necessarily using all of them, I think it can work. There may need to be a mechanism for signaling the cpal application when another application or hardware device has connected to/disconnected from the cpal application's port, but I'm not familiar enough with any of these APIs to be sure about that. That may be related to hotplug support (#373).

[Be-ing]

I have identified at least one good reason for a new PipeWire backend rather than relying on PipeWIre's reimplementation of the JACK API. From the PipeWire wiki:

PipeWire can adapt the latency dynamically, which is important for power usage on a laptop. When low latency is required, the system can switch automatically and seamlessly to smaller buffer sizes.

This means that a CPAL application using PipeWire could request a small buffer size for low latency. When the application goes idle, it could call Stream::pause and yield control of the buffer size back to PipeWire. This would allow keeping a low latency application open in the background, pausing it, then watching a YouTube video in Firefox without having to close any applications or mess with reconfiguring anything. If I force a JACK application using the PipeWire JACK reimplementation to use a small buffer size with the PIPEWIRE_LATENCY environment variable, YouTube videos in Firefox have glitchy audio while the other application remains running.

JACK has an API somewhat like this with int jack_deactivate (jack_client_t * client), but this does not fit with CPAL's abstractions. To use this, CPAL would need to track the state of each Device's streams and call jack_deactivate when all the CPAL Streams for a Device are paused.

Related discussion on the PipeWire issue tracker: https://gitlab.freedesktop.org/pipewire/pipewire/-/issues/722

[Be-ing]

@ishitatsuyuki if you or anyone else works on a new PipeWire backend for CPAL, I would be happy to help test it. I am afraid writing it myself would be above my skill level though.

[Ralith]

It kinda sounds like you have the most domain expertise here. Why not have a go?

[Be-ing]

I think that would be a bit ambitious for a first Rust project.

[Be-ing]

FWIW, here is SDL's PipeWire backend: https://github.com/libsdl-org/SDL/pull/4094

[Be-ing]

This would be great for Whisperfish!

[kamiyaa]

Any updates on this?

[ishitatsuyuki]

As far as I have tested, it seems the JACK backend (+pw-jack) works fine for low-latency use cases on PipeWire. If you have any specific thing missing that requires directly integrating with PipeWire, then please let me know with a comment.

[ethindp]

I've been looking at developing a PW backend, but one major problem I see is the main loop.

A PipeWire application requires a main loop object to be created. An application calls the run method, which never returns (it can be terminated but you do it through sources). This would be a major problem with CPALand anything consuming it since it would never allow anything to run beyond the PW main loop. I'm guessingother audio backendsdo this, but I can't seem to find any way of stepping through the loop without using sources (e.g. timers). Thoughts about getting around this?

[ishitatsuyuki]

I don't really know, but can you run that loop on another thread? It's common to create additional helper threads in cpal backends.

[ethindp]

@ishitatsuyuki Yep, that's exactly what the PipeWire documentationstates:

The pipewire library is not really thread-safe, so pipewire objects do not implement Send or Sync. However, you can spawn a MainLoop in another thread and do bidirectional communication using two channels. To send messages to the main thread, we can easily use a std::sync::mpsc. Because we are stuck in the main loop in the pipewire thread and can’t just block on receiving a message, we use a pipewire::channel instead.

I'll look at doing that -- I didn't think of that lol.

[Be-ing]

With https://github.com/RustAudio/rust-jack/pull/154 I confirm that the cpal examples work with PipeWire's implementation of JACK.

The enumerate example is only listing the ports cpal crates. I don't know if that's an issue with the enumerate example or cpal:

cpal on  HEAD (59ee8ee) [!] is 📦 v0.13.4 via 🦀 v1.56.1 
❯ cargo run --release --example enumerate --features jack
    Finished release [optimized] target(s) in 0.05s
     Running `target/release/examples/enumerate`
Supported hosts:
  [Jack, Alsa]
Available hosts:
  [Jack, Alsa]
JACK
  Default Input Device:
    Some("cpal_client_in")
  Default Output Device:
    Some("cpal_client_out")
  Devices: 
  1. "cpal_client_in"
    Default input stream config:
      SupportedStreamConfig { channels: 2, sample_rate: SampleRate(48000), buffer_size: Range { min: 1024, max: 1024 }, sample_format: F32 }
    All supported input stream configs:
      1.1. SupportedStreamConfigRange { channels: 1, min_sample_rate: SampleRate(48000), max_sample_rate: SampleRate(48000), buffer_size: Range { min: 1024, max: 1024 }, sample_format: F32 }
      1.2. SupportedStreamConfigRange { channels: 2, min_sample_rate: SampleRate(48000), max_sample_rate: SampleRate(48000), buffer_size: Range { min: 1024, max: 1024 }, sample_format: F32 }
      1.3. SupportedStreamConfigRange { channels: 4, min_sample_rate: SampleRate(48000), max_sample_rate: SampleRate(48000), buffer_size: Range { min: 1024, max: 1024 }, sample_format: F32 }
      1.4. SupportedStreamConfigRange { channels: 6, min_sample_rate: SampleRate(48000), max_sample_rate: SampleRate(48000), buffer_size: Range { min: 1024, max: 1024 }, sample_format: F32 }
      1.5. SupportedStreamConfigRange { channels: 8, min_sample_rate: SampleRate(48000), max_sample_rate: SampleRate(48000), buffer_size: Range { min: 1024, max: 1024 }, sample_format: F32 }
      1.6. SupportedStreamConfigRange { channels: 16, min_sample_rate: SampleRate(48000), max_sample_rate: SampleRate(48000), buffer_size: Range { min: 1024, max: 1024 }, sample_format: F32 }
      1.7. SupportedStreamConfigRange { channels: 24, min_sample_rate: SampleRate(48000), max_sample_rate: SampleRate(48000), buffer_size: Range { min: 1024, max: 1024 }, sample_format: F32 }
      1.8. SupportedStreamConfigRange { channels: 32, min_sample_rate: SampleRate(48000), max_sample_rate: SampleRate(48000), buffer_size: Range { min: 1024, max: 1024 }, sample_format: F32 }
      1.9. SupportedStreamConfigRange { channels: 48, min_sample_rate: SampleRate(48000), max_sample_rate: SampleRate(48000), buffer_size: Range { min: 1024, max: 1024 }, sample_format: F32 }
      1.10. SupportedStreamConfigRange { channels: 64, min_sample_rate: SampleRate(48000), max_sample_rate: SampleRate(48000), buffer_size: Range { min: 1024, max: 1024 }, sample_format: F32 }
    Default output stream config:
      SupportedStreamConfig { channels: 2, sample_rate: SampleRate(48000), buffer_size: Range { min: 1024, max: 1024 }, sample_format: F32 }
    All supported output stream configs:
      1.1. SupportedStreamConfigRange { channels: 1, min_sample_rate: SampleRate(48000), max_sample_rate: SampleRate(48000), buffer_size: Range { min: 1024, max: 1024 }, sample_format: F32 }
      1.2. SupportedStreamConfigRange { channels: 2, min_sample_rate: SampleRate(48000), max_sample_rate: SampleRate(48000), buffer_size: Range { min: 1024, max: 1024 }, sample_format: F32 }
      1.3. SupportedStreamConfigRange { channels: 4, min_sample_rate: SampleRate(48000), max_sample_rate: SampleRate(48000), buffer_size: Range { min: 1024, max: 1024 }, sample_format: F32 }
      1.4. SupportedStreamConfigRange { channels: 6, min_sample_rate: SampleRate(48000), max_sample_rate: SampleRate(48000), buffer_size: Range { min: 1024, max: 1024 }, sample_format: F32 }
      1.5. SupportedStreamConfigRange { channels: 8, min_sample_rate: SampleRate(48000), max_sample_rate: SampleRate(48000), buffer_size: Range { min: 1024, max: 1024 }, sample_format: F32 }
      1.6. SupportedStreamConfigRange { channels: 16, min_sample_rate: SampleRate(48000), max_sample_rate: SampleRate(48000), buffer_size: Range { min: 1024, max: 1024 }, sample_format: F32 }
      1.7. SupportedStreamConfigRange { channels: 24, min_sample_rate: SampleRate(48000), max_sample_rate: SampleRate(48000), buffer_size: Range { min: 1024, max: 1024 }, sample_format: F32 }
      1.8. SupportedStreamConfigRange { channels: 32, min_sample_rate: SampleRate(48000), max_sample_rate: SampleRate(48000), buffer_size: Range { min: 1024, max: 1024 }, sample_format: F32 }
      1.9. SupportedStreamConfigRange { channels: 48, min_sample_rate: SampleRate(48000), max_sample_rate: SampleRate(48000), buffer_size: Range { min: 1024, max: 1024 }, sample_format: F32 }
      1.10. SupportedStreamConfigRange { channels: 64, min_sample_rate: SampleRate(48000), max_sample_rate: SampleRate(48000), buffer_size: Range { min: 1024, max: 1024 }, sample_format: F32 }
  2. "cpal_client_out"
    Default input stream config:
      SupportedStreamConfig { channels: 2, sample_rate: SampleRate(48000), buffer_size: Range { min: 1024, max: 1024 }, sample_format: F32 }
    All supported input stream configs:
      2.1. SupportedStreamConfigRange { channels: 1, min_sample_rate: SampleRate(48000), max_sample_rate: SampleRate(48000), buffer_size: Range { min: 1024, max: 1024 }, sample_format: F32 }
      2.2. SupportedStreamConfigRange { channels: 2, min_sample_rate: SampleRate(48000), max_sample_rate: SampleRate(48000), buffer_size: Range { min: 1024, max: 1024 }, sample_format: F32 }
      2.3. SupportedStreamConfigRange { channels: 4, min_sample_rate: SampleRate(48000), max_sample_rate: SampleRate(48000), buffer_size: Range { min: 1024, max: 1024 }, sample_format: F32 }
      2.4. SupportedStreamConfigRange { channels: 6, min_sample_rate: SampleRate(48000), max_sample_rate: SampleRate(48000), buffer_size: Range { min: 1024, max: 1024 }, sample_format: F32 }
      2.5. SupportedStreamConfigRange { channels: 8, min_sample_rate: SampleRate(48000), max_sample_rate: SampleRate(48000), buffer_size: Range { min: 1024, max: 1024 }, sample_format: F32 }
      2.6. SupportedStreamConfigRange { channels: 16, min_sample_rate: SampleRate(48000), max_sample_rate: SampleRate(48000), buffer_size: Range { min: 1024, max: 1024 }, sample_format: F32 }
      2.7. SupportedStreamConfigRange { channels: 24, min_sample_rate: SampleRate(48000), max_sample_rate: SampleRate(48000), buffer_size: Range { min: 1024, max: 1024 }, sample_format: F32 }
      2.8. SupportedStreamConfigRange { channels: 32, min_sample_rate: SampleRate(48000), max_sample_rate: SampleRate(48000), buffer_size: Range { min: 1024, max: 1024 }, sample_format: F32 }
      2.9. SupportedStreamConfigRange { channels: 48, min_sample_rate: SampleRate(48000), max_sample_rate: SampleRate(48000), buffer_size: Range { min: 1024, max: 1024 }, sample_format: F32 }
      2.10. SupportedStreamConfigRange { channels: 64, min_sample_rate: SampleRate(48000), max_sample_rate: SampleRate(48000), buffer_size: Range { min: 1024, max: 1024 }, sample_format: F32 }
    Default output stream config:
      SupportedStreamConfig { channels: 2, sample_rate: SampleRate(48000), buffer_size: Range { min: 1024, max: 1024 }, sample_format: F32 }
    All supported output stream configs:
      2.1. SupportedStreamConfigRange { channels: 1, min_sample_rate: SampleRate(48000), max_sample_rate: SampleRate(48000), buffer_size: Range { min: 1024, max: 1024 }, sample_format: F32 }
      2.2. SupportedStreamConfigRange { channels: 2, min_sample_rate: SampleRate(48000), max_sample_rate: SampleRate(48000), buffer_size: Range { min: 1024, max: 1024 }, sample_format: F32 }
      2.3. SupportedStreamConfigRange { channels: 4, min_sample_rate: SampleRate(48000), max_sample_rate: SampleRate(48000), buffer_size: Range { min: 1024, max: 1024 }, sample_format: F32 }
      2.4. SupportedStreamConfigRange { channels: 6, min_sample_rate: SampleRate(48000), max_sample_rate: SampleRate(48000), buffer_size: Range { min: 1024, max: 1024 }, sample_format: F32 }
      2.5. SupportedStreamConfigRange { channels: 8, min_sample_rate: SampleRate(48000), max_sample_rate: SampleRate(48000), buffer_size: Range { min: 1024, max: 1024 }, sample_format: F32 }
      2.6. SupportedStreamConfigRange { channels: 16, min_sample_rate: SampleRate(48000), max_sample_rate: SampleRate(48000), buffer_size: Range { min: 1024, max: 1024 }, sample_format: F32 }
      2.7. SupportedStreamConfigRange { channels: 24, min_sample_rate: SampleRate(48000), max_sample_rate: SampleRate(48000), buffer_size: Range { min: 1024, max: 1024 }, sample_format: F32 }
      2.8. SupportedStreamConfigRange { channels: 32, min_sample_rate: SampleRate(48000), max_sample_rate: SampleRate(48000), buffer_size: Range { min: 1024, max: 1024 }, sample_format: F32 }
      2.9. SupportedStreamConfigRange { channels: 48, min_sample_rate: SampleRate(48000), max_sample_rate: SampleRate(48000), buffer_size: Range { min: 1024, max: 1024 }, sample_format: F32 }
      2.10. SupportedStreamConfigRange { channels: 64, min_sample_rate: SampleRate(48000), max_sample_rate: SampleRate(48000), buffer_size: Range { min: 1024, max: 1024 }, sample_format: F32 }
ALSA
  Default Input Device:
    Some("default")
  Default Output Device:
    Some("default")
  Devices:
...

[Be-ing]

https://github.com/RustAudio/cpal/pull/624 fixes the build using Pipewire for the jack feature.

[m00nwtchr]

I'm started working on a PipeWire host here: https://github.com/m00nwtchr/cpal/tree/pipewire-host

Any help/thoughts would be appreciated.

I think I'm fairly far along, but I can't find any way aside from the PipeWire Stream api to actually write/read data, and I'm not sure if my node creation code is correct.

[Be-ing]

Cool! Could you make a draft pull request?

[Be-ing]

I can't find any way aside from the PipeWire Stream api to actually write/read data

The pw_filter API does not have Rust bindings yet. This is in progress: https://gitlab.freedesktop.org/pipewire/pipewire-rs/-/merge_requests/112

[m00nwtchr]

Is pw_filter needed for this, though?

https://docs.pipewire.org/page_streams.html

For more complicated nodes such as filters or ports with multiple inputs and/or outputs you will need to use the pw_filter or make a pw_node yourself and export it with pw_core_export.

Also, opened the draft PR: #651

[Be-ing]

If you're creating pw_nodes manually then I don't think you strictly need pw_filter, but my understanding is that using pw_filter would be easier to implement.