Looking for Options: How do you get low latency remote GUIs out of your Linux VMs, with sound?

[SInisterPisces]

So, I'm struggling a bit with my Linux-based GUI OS VMs, even with GPU acceleration via VirGL. In particular, I'm looking for low latency input (c.f., pressing a key and waiting seconds to see a response in the OS), low latency audio (I'm working with software defined radio right now, and need to be able to hear what's actually happening when it's happening, without distortion), and enough GPU acceleration that 80 percent of my CPU isn't trying to draw the GUI.

All I've got is a Ryzen 5900HX with iGPU, and until I can figure out how to do the sorcery that lets you actually PCIe passthrough one of those (there's a thread here where it's Mostly Working®), I'm using VirGL to take a slice of it. It works well enough, but I think it'd work better if I could give the VM more than 512 MB of VRAM. Separate issue.

The Proxmox node has a 2.5GbE LAN connection, and my client machine's got a 10GbE LAN connection, so I shouldn't be having network issues.

So, right now, I'm using VirGL (512 MB) + SPICE. VirGL's managing to take my CPU load down to something reasonable, and SPICE is getting me audio, but my biggest issue is still UI latency. It goes from "annoying" to "completely unusable" depending on what I'm doing. I can't even use the terminal most of the time. I was using the i440fx machine type and have since switched to q35, but in the event that doesn't fix it, I'm interested in what other people have had good success with. I'm looking for a solution I can standardize on in my prebuilt linux VMs (This one is a prebuilt Ubuntu Focal-based VM set up for SDR tasks).

I've tried xRDP. Great, low UI latency using Microsoft's Remote Desktop client on a Mac. Absolutely no audio passthrough. This is a known issue on Ubuntus of a certain age. I tried to install a custom version of xRDP with audio passthrough enabled--it got me working audio, but it was crackly and lagged and I couldn't figure out how to fix it, as the Mac Microsoft RDP client doesn't expose any performance settings that I could find.

NoMachine might work, if it'll recognize the SPICE audio driver as a sound device and pass through. I'm still testing that one. But even it wasn't as low latency on input as i would like.

I haven't tested Midnight yet, but if it works I think it might be the way to go, since it's meant for gaming and I'd not be pushing as hard as that.

I'd also like to use Batocera or RetroPie in a VM at some point, which is never going to happen if I can't get something like DragonOS working usably.

So, question: what does everyone else use to get low latency UI + audio?
Handicap: "Get a server with an actual dedicated GPU" is not an option. Yet.

 

[BobhWasatch]

Note that most Linux remote desktop solutions do not depend on the QEMU sound driver. That's more for getting sound out of the host machine. Xrdp and the like generally use pulseaudio to send the sound over the network to the client machine. Spice seems to be the exception here I guess because it is built in to QEMU. Also, I use XFCE so remote desktops don't need a GPU to perform well.

Note also that my client is usually a laptop running Linux. So some of my issues with RDP might have been due to the RDP client I was using (xfreerdp).

I used xrdp on my VM's for a long time and just put up with the sound lagging since I didn't watch many videos that way. If I access xrdp through Apache Guacamole then the sound is more in sync for some reason but playing videos was still not a happy experience. The other thing is every time you do a major upgrade you get to rebuild the xrdp sound system.

Anyhow, at one point I set up an SDR which for various reasons having to do with connecting the antennas had to be operated remotely. For a while I used Windows with Remote Desktop but that was a really annoying solution since remote admin on a single instance of Windows is pretty weak IMO. Basically it is just fragile and easy to break such that you need physical access. Maybe if I gave MS more money it would be better.

I tried Linux with xrdp on the SDR machine and it just didn't work very well. Sound lagged more and more over time to where eventually I would change to another radio station and it would take like several seconds to catch up. It made tuning in SSB and CW stations really painful. Click-wait-click-wait, etc.

I eventually found NoMachine and it worked great for the SDR application. Sound stays in sync and the lag between tuning and the sound changing is quite tolerable. The waterfall works nicely and is in real-time. Videos play acceptably for my limited use. It is good enough that sometimes I even forget that I'm using a remote machine. That machine runs on real hardware and NoMachine does make use of the integrated GPU.

After that I started using NoMachine on my Linux VM's and been pretty happy with it. With XFCE performance is decent even without a GPU. The only downside I've had is that real multi-monitor support is a paid feature. The free version just spreads the desktop between the monitors as one big screen so windows pop up in the middle and so forth. It also seems to work better in a GPU-less environment if you do not enable a desktop manager (e.g. sddm, gdm) and just use virtual desktops.