Debian unstable user here (and also Gentoo on a different PC) - I've removed PulseAudio for a couple of months now, and everything works nicely (including a Bluetooth headset and HDMI audio out) with PipeWire and WirePlumber - the only hiccup was an old manual configuration I had for ALSA to use PulseAudio as default audio sink, which needed to be removed.

I haven't tested Jack compatibility yet.