February 9, 2016 - Derek Foreman
WOW, Wayland Over Wire!
A common complaint about Wayland is that it isn’t network transparent. X allows you to run an application on one computer and display its output on a different computer as long as the application doesn’t depend on certain “modern” features (such as the shared memory extension). Applications are forwarded individually and are indistinguishable from apps on the local desktop (network performance considerations aside). This is different than remote desktop protocols like VNC or RDP which provide control of an entire operating system.
The Hurdles to Wayland Network Transparency
Obviously, if we intend to make Wayland a replacement for X, we need to duplicate this functionality. However, some Wayland design decisions make it difficult to implement this kind of network transparency:
- File descriptor passing is used extensively in the protocol. For example, keymaps are passed from the compositor to the client as file descriptors; the client is supposed to mmap() the file descriptor and treat it like an array. This sounds a little silly at first, but a keymap is actually a fairly large chunk of data that’s too large to fit in a single Wayland message packet (they can’t be larger than 4Kb). Obviously you can’t (usefully) pass a file descriptor over the network.
- Keyboard repeat is handled on the client side, so if you have a dodgy network connection and a key press packet arrives but the key release packet is delayed, the client would start repeating keys.
- Buffers are shared between the client and compositor… somehow. They could be shared memory through mmap or dmabuf, or it could be mesa’s buffer extension. In any event, image data is never pushed to the compositor through a network socket, it always takes the form of some kind of handle that both the client and compositor understand. These handles are local only and are meaningless over a network.
Otherwise, there’s really not a lot about the Wayland protocol that makes it local only. In fact, it’s designed to be asynchronous and minimize round trips; this is great for remote connections.
Wow! Introducing Wayland Over Wire!
So, I’ve prepared a really simplistic implementation of Wayland network transparency, or Wayland Over Wire, which essentially spits the Wayland protocol over a TCP/IP socket. Since Wayland only carries handles to image data in its messages, I’ve added some new messages to take care of this. I’ve also taught libwayland-client about buffers, which is probably a controversial move. I’ve done this by adding client side “hooks” to certain requests.
A hook is a function that’s called just before the request is sent, and passed the same parameters. This lets libwayland-client itself track the damage instead of just being a transparent proxy between the compositor and client. The end result is that any compositor (I’ve tried both Weston and Enlightenment) can use this to provide remote apps with only 1 line of code added to the compositor:
Since this is an epic, gaping security hole, I’m not going to publish a patch to enable this, but feel free to add that line to your compositor and give it a try (hint: in Weston it’s right before load_modules in main). Just keep in mind that this opens a network port that lets anyone connect directly to your compositor: something that could potentially create major security risks.
Work Left to be Completed
Since this is a proof of concept/request for comments, I’ve bailed on some of the less interesting (though hugely important) problems like authentication, port selection, ipv6 support. Additionally, cut and paste is not yet implemented. There’s also a list of other minor (and not so minor) issues, including:
- Clients must use the wl_shm buffer type to work, meaning EGL and dmabuf won’t work.
- Damage tracking is terrible. Instead of doing anything at all intelligent I simply use a rectangle that contains all damage. If two points at opposite corners of the window change at the same time, the whole frame will be sent.
- There’s no compression on image data, so this will only work reasonably on relatively fast network connections. I must confess to have only tested it on Gigabit-Ethernet.
Those issues can be resolved and refined, but there’s still something that’s always going to suck with this approach: I’m not sure how to get around the client side key repeat problems. Very clever toolkits could hold off on generating repeated keys until they get a frame callback, but this could result in deferring a lot of processing until it’s too late to finish the rendering before the next screen refresh.
I’ve just recently posted the patch series on the Wayland mailing list, so feel free to give it a try, but please keep in mind this is definitely “use at your own risk” stuff. As I said, anyone can connect to that network port so be careful how you use it, and I don’t recommend using it in production. If you have any feedback to provide, feel free to share it on the mailing list, or chat with us in #wayland on Freenode.
About Derek Foreman
Derek Foreman is a Senior Open Source Developer with Samsung's Open Source Group, specializing in graphics work. Previously, he worked on the graphics team at an open source consultancy where his work primarily focused on hardware enablement and software optimization for embedded systems. His career started at a biomedical institute where he developed analysis and control software for medical imaging equipment.
Image Credits: Kristian Høgsberg