Wayland RDP Server with audio using Microsoft’s WSLg stack
I spent some time playing with the Microsoft’s modified Weston/PulseAudio/FreeRDP stack. This is what Windows is using to display GUI Linux apps under WSL. I primarily use Linux and I use Linux remote desktops quite a bit. I use xrdp, freerdp, x11vnc, vinagre, guacamole for different things. I’ve tried GNOME’s built in RDP server and freerdp2-shadow-x11. They all work well enough. xrdp is best so far, but it’s a PITA to set up and the stack is kind of heavy. I know it’s possible to get audio and H264 video working with xrdp, but I have not tried it yet. I am curious about a Wayland RDP server. I installed Weston (the reference Wayland compositor) on Debian and was shocked at how easy it was to get an RDP server up and running. Weston pulls in few dependencies, it’s quite small, and has an RDP backend baked in that works quite well out of the box. Just run «weston —backend=rdp-backend.so». Depending on the version, it may be necessary to generate a key file, which is easy. There is no authentication, it just fires up an RDP server logged in as the current user. This is perfect for me since I’m using wireguard for network level authentication. It supports the RemoteFX codec, but not the progressive codec, and no H264 codec, no sound. Not bad.
It annoys me to say so, but Windows has the best remote desktop, MS designed RDP after all. It’s awesome that MS is integrating Linux these days, and I was curious how WSL works. It turns out they are using RDP. They designed a new extension called rail that send each window individually, to make it seamless. They have forks of Weston, PulseAudio, and FreeRDP up on github, with all of their modifications to the Linux side of the RDP stack. Interesting.
After poking around in my free time last week, I got them up and running on a minimal Debian bullseye install. Microsoft compiles minimal binaries, without some of the goodies that I want, so after trying those I decided to compile from source. Their stack is designed to be run on the same host, so they aren’t worried about network latency at all. I had to make some small tweaks to get it working well with FreeRDP. Audio is uncompressed PCM (no easy fix, afaik). Audio didn’t play well with the FreeRDP pulse client plugin (easy fix). RemoteFX is hard-coded to disabled (easy fix).
Firefox works great. I can stream youtube videos in 1080p with audio on the same network and can’t even tell it’s remote.
Run it as root on Debian bullseye. It downloads, tweaks, compiles, and installs Microsoft’s forks of weston, pulseaudio, and FreeRDP and sets them up to run as root on boot. You could also just use it as a guide, line-by-line, that’s probably a good idea anyway.
MS uses a supervisor process that sets everything up, keeps it connected, and restarts things as needed. That won’t work for us since we aren’t connecting to Windows. During testing Weston stopped accepting clients a couple times when trying unsupported client options and it was necessary to killall -9 weston. This is just for testing, it is not meant as a production environtment, espeically because it’s running as root; but you could totally build a production worthy terminal server around these tools.
To connect using freerdp, I’ve had the best results with this command:
xfreerdp /v:10.0.1.135 /cert-ignore /size:1024×768 /f /floatbar:sticky:off,show:always /sound:latency:200 -gfx /rfx
The thing that bothers me the most is the inability to resize the client desktop size after connection. Of everything that I have tested, that only works with xrdp and Windows. It would also be nice if the audio stream was compressed and if progressive or H264 video encoding was supported. It would be cool to get rail support in the freerdp client, that would open up some interesting options for virtualizing apps. Hopefully the MS patches get merged upstream and more community work can be done on this.
Use a laptop as a 2nd display on Linux using FreeRDP
Dual monitors using a laptop as a second display. This guide assumes that you are using a Debian based distro (X11 only). Both devices must be connected to the same network. 1. Define an output to be a used as a dummy monitor a. The simple method: select an unused
zswap: improve memory performance on Linux
Zwap is a Linux kernel module that will attempt to compress memory pages and keep them in ram before swapping to disk. This is helpful on low memory systems and will result in a noticeable performance improvement in most cases. It is required that a swap device exists. Here are
XFINITY Hotspots
Connect to XFINITY hotspots from Android and Linux without an app. Android 11 EAP method: TTLS Identity: your-username Password: your-password CA certificate: Use system certificates Online certificate status: Don’t validate Domain: secure.aaa.wifi.comcast.com Phase 2 authentication: PAP The others require a certificate Get the certificate from FireFox:
Wayland, Remote Desktop Sharing and Ubuntu 22.04
Desktop sharing is one of the essential things I do every day. For almost 10 years now, I’m using x11vnc to get GUI access on remote Ubuntu desktops. There are many VNC servers, but x11vnc is the best I have found so far, as it works well even under extremely severe bandwidth constraints, i.e., it can easily live with 1 Mbit/s in the uplink when using the highest compression scheme. And if nothing happens on the desktop, there is almost no data transmitted. As the name implies, x11vnc is based on the x11 display server. That’s a bit of a problem these days, as all Linux distributions that use the Gnome desktop, including Ubuntu, are trying very hard for years now to switch to the newer Wayland display server. One of the problems that have plagued all attempts so far was the lack of a usable remote desktop solution. But now, it looks like Ubuntu will make Wayland the default compositor in the upcoming 22.04 Long Term Support (LTS) version. X11 is still supported, but once Wayland becomes the default, I wonder how much love x11 will still get!? So long story short, I was wondering what kind of remote desktop sharing solution Gnome and Ubuntu are proposing for 22.04 and whether it works as well as x11vnc. The answer was not as bad as I thought, but also not as good as I hoped.
Broken in the Daily Build
It’s still early April 2022 so Ubuntu LTS 22.04 has not yet been released. However, there are daily builds available, so I download the latest one to have a closer look at the built-in desktop sharing. To my surprise, the functionality is totally broken in my daily build, but I’m not the first to notice. More about that later. So I assume for now that in whatever way the current problem will be fixed before 22.04 is released, the result will pretty much look like what is currently built into Ubuntu 21.10. So I got the latest image of that release and continued there.
Update 11. May 2022: It turned out that my assumption that 22.04 LTS would work the same as 21.10 was wrong. The final version of 22.04 LTS uses RDP instead of VNC for remote desktop sharing in combination with Wayland. Therefore, head over to my later review of the final RDP based solution in Ubuntu 22.04.
The remainder of this post has a look of the VNC backend of the remote desktop solution of Ubuntu 21.04 and also of the “legacy fallback option” in Ubuntu 22.04.
Wayland, Desktop Sharing and Ubuntu 21.10
In Ubuntu 21.10, desktop sharing works in Wayland mode and I could activate it in the settings. I then used Remmina and its VNC plugin to connect remotely. In the local network, this works very nicely and the remote UI is responsive to mouse clicks and key presses. A positive surprise!
However, I could already see there and then that the bandwidth requirements are huge. Double digit Mbit/s datarates were not out of the ordinary. While this is no problem in the local network, such data rates are a total showstopper when connecting to a device over a slow Internet connection with 1-2 Mbit/s. Also, connecting to moving devices and quickly changing backhaul speeds, e.g. in trains (yes, I do that) is just out of the question at such speeds.
Minimum Data Rates
So what’s the slowest Internet connection Ubuntu’s Wayland based remote desktop sharing is still usable with? I experimented for quite a while and in the highest compression setting, I could get down as low as 2 Mbit/s. Anything less and the session becomes unresponsive. Even at 2 Mbit/s, responsiveness is considerably worse compared to x11vnc. In addition, desktop sharing takes the full 2 Mbit/s of bandwidth at all times, even if nothing changes on the screen. This significantly slows down other applications running over the slow remote line at the same time. So while I’m glad it works at all, I’m not so happy that some of my major use cases can’t be covered with that solution.
Other Tidbits
In addition, there are a number of other things that would make my live difficult with Ubuntu’s Wayland desktop sharing. Fortunately, I found a workaround for each of them:
No Localhost Option
The first thing that bugged me was that when desktop sharing is activated, it is open to all incoming IP addresses. That’s a problem, because VNC only uses 8 character passwords. That’s not strong enough by far. X11vnc shares the same problem, but I usually run the vnc server in ‘localhost’ mode, i.e. it only accepts incoming connections from 127.0.0.1. To access desktop sharing, I establish an ssh session and activate port forwarding. The way around this with Ubuntu’s built in Wayland desktop sharing is to use the Universal Firewall (ufw) and limit incoming access to port 5900 to localhost. I haven’t tried if this works, but I’m pretty confident that this can be made to work.
Network Dependencies
In Ubuntu’s settings menu, Remote Desktop Sharing can and has to be activated per Wi-Fi SSID. That’s nice but many devices I support frequently change Wi-Fi networks and I can’t get to the GUI remotely to activate desktop sharing in a new Wi-Fi. Fortunately, I usually do have SSH connections to those machines, even if they change networks frequently, and running
systemctl —user start gnome-remote-desktop
on the shell activates the remote desktop server if it is not running.
Restarting Hung Sessions
You might have noticed that nothing is perfect. Internet connectivity on the move in particular. If a VNC connection is interrupted, the vnc server hangs, and it’s not possible to log-in again. Also, there’s no way on the GUI to get things going again. But once the ssh link is up again, killing the remote desktop server and starting it again on the shell gets things running again.
Repeating Keys
Over slow connections, VNC suffers from characters that keep repeating. I didn’t have much of a problem with that, as x11vnc can handle very slow connections. With gnome-remote-desktop with the VNC backend, this is much more of a problem over slow connections. The fix: On the remote host, disable ‘repeat keys’ in the ‘Universal Access’ settings. Cool, I can use this for my current x11 setup as well in special situations.
Summary and Way Forward
Over relatively fast and un-metered Internet connections, Ubuntu’s Wayland desktop sharing with gnome-remote-desktop and the VNC backend works quite well. Unfortunately, my low bandwidth use-case is the norm rather than the exception for me, so it’s not a solution for me. Fortunately, it’s also possible to switch from Wayland to x11 for a user when logging in, and Ubuntu remembers this decision across reboots. In the next episode I’ll therefore have a closer look how x11vnc behaves in Ubuntu 22.04 LTS. Also, there is still the possibility that Ubuntu will switch to the RDP backend of the gnome-remote-desktop package they use for remote desktop sharing, which might change things again. If that happens, I’ll have a closer again.
And to check if the current GUI session runs over X11 or Wayland, the following shell command comes in handy: