June 23, 2016 - Bryce Harrington
Introducing Wayland Screen Inhibition
The Inhibited Wayland Desktop – Part 1
This article is part one of a two part series on screen inhibition in Wayland.
Picture this: you’re giving a presentation where you’re digging deep into some fascinating detail, when suddenly, your screensaver pops on. Quickly, you rush back to your laptop to tap the keyboard or wave the mouse to hide the pictures of your cat and bring your presentation back. Now, where were you? You’ve lost your train of thought, to the audience’s bemusement and to the detriment of your presentation.
Or, imagine this: you’re watching a movie on your cell phone or tablet while on battery power, and you’re constantly frustrated when the screen dims or goes black every 5 minutes.
I’m sure many of you have had problems just like these, and the generally accepted solution for these issues is to temporarily inhibit power saving mode as well as any active screensavers while the movie or presentation is running. On the desktop it’s possible to do this manually (or via helper tools like xdg-screensaver or caffeine), but it’s better if the presentation application could simply disable the screensaver automatically when it begins displaying, and then restore the screensaver settings when it pauses or exits (or crashes).
In the past, each desktop environment provided its own interface to do this. Client applications that wished to be used on multiple desktop environments needed to carry routines to handle each individual environment. Some apps brute forced the screensaver off (with no provision to re-enable it in the case of a crash); many simply didn’t bother. Today, most desktop environments support freedesktop’s ScreenSaver dbus protocol, and can reliably suspend the screensaver for the life of the process (or until re-enabled by the client). However, this technology is for X11; what about Wayland?
In this first blog post I’ll take an in-depth look into how screen locking/blanking/saving and idle timing works. Then, in the next post of this series I’ll dig into the new idle inhibition protocol and how it works in Weston.
Locking on Idle
As part of a security-focused design philosophy, Wayland limits the control given to client applications over system resources; screen blanking/locking is considered system-level functionality. Unlike X11, it doesn’t allow any random screensaver application to futz with the screen willy-nilly.
Screen-locking is secured by limiting access to the locking/unlocking API of a known authenticated client. The compositor executes the client directly and sets up a secure communication channel to it. With this entire privileged interface protected, the unlock request can’t be abused, and as a consequence of this design the compositor will typically own the responsibility for starting up the locking app. In the case of the Weston compositor, for example, the desktop-shell client handles screen locking and only the compositor itself can start it.
The unlock dialog is exported to the desktop-shell client. When the idle timeout occurs, the screen fades to black and locks. Input events cause the compositor to wake and send a ‘prepare_lock_surface’ event to the desktop-shell; this replies with a set_lock_surface that specifies the unlock dialog’s surface. The compositor fades in and transitions to this dialog; when the user has dismissed it successfully, desktop-shell sends an unlock request and all the surfaces are brought back.
When locking occurs, the shell plugin moves all the surfaces from the compositor’s rendering list aside, with only the background surface and pointer cursor remaining. Nothing else can receive input, be activated, run animations, nor send frame events, but the surface objects themselves are still alive.
The client can register a surface as an idle animation for a given output; this remains hidden until an input timeout occurs and the compositor fades to black. If there’s a screensaver surface, it fades in to show the idle animation. Additionally, this process can start a screensaver client automatically.
While the idle animation runs, the unlock dialog will appear as soon as an input event registers; once this is cleared successfully the screensaver client is killed. Otherwise, eventually a second timeout is hit and the compositor blanks the screen entirely and stops all screen rendering.
Idle States in Weston
The mechanics of idling and screen blanking is left as internal details of the given compositor, and actual implementation will vary. Now, I’ll offer an explanation of how it works in the Weston compositor to provide an example.
Weston tracks a global state for the monitors, which can be active, sleeping, idle, or offscreen. The active state indicates normal rendering behavior and uses a global idle timer to track when to enter idle mode (5 minutes by default); the input system resets this timer each time there is an input event (a mouse move/click, a keyboard press or release, etc.). When the system enters idle mode, the idle animation or lock screen is started up and faded in. The idle animation maintains another timeout that, when triggered, tells the compositor to fade the idle animation out and enter sleep state. A wake action via weston_compositor_wake() will return the Idle system to active state; for example, a lock screen can call this when a correct password is entered.
In sleep state, the compositor ceases rendering, stops sending frame events, and powers down the monitors by sending them OFF commands via DPMS. It is theoretically possible to hit zero CPU usage at this point, even with an active screensaver. In Weston’s desktop shell, surfaces are organized into layers; when sleep state is entered, a new locking layer is added and all other layers are removed. Thus, the client stops further rendering activity.
Input activity will cue Weston to return to active state. If the system is in sleep state, the monitors are sent DPMS ON commands and the outputs begin repainting again. Only the locking layer is present at this point, so only the lock screen is shown (with a nice fade-in animation). When the user interacts appropriately with the lock screen to unlock it, the other layers are re-added and thus resume rendering activity.
There is also another state for the outputs, offscreen, which leaves the monitors powered on but simply skips rendering to them. There are a few special cases where this behavior is needed, such as during display system shutdown.
At this point, let’s take a brief aside to discuss screensavers. We’ve been using this as a catch-all term for various things triggered through idle timeout, including blanking, dimming, locking, and powering down the display. Weston also includes a snazzy, animated ‘fade-out’ effect that kicks in when the idle counter runs out. Specifically, “screensaver” refers to a client application that draws graphics over the top of the screen, sometimes also presenting a password dialog to terminate it. The animation functionality was originally used to prevent phosphor burn-in, but modern display hardware is mostly no longer at risk of this. The animated eye-candy aspect of screensavers primarily has amusement value these days, but the locking functionality is still relevant, indeed even critical in these days of intense computer security.
So let’s take a look at locking first. Unlike X, locking is not left to the client to enforce, but is instead implemented in the compositor itself. This means that screen locking will be simpler and much more reliable if the compositor implements it properly; Wayland provides an API specifically for establishing locking. While the compositor is in its ‘locked’ state, it only sends rendering commands to the surface designated as the lock surface; this is where our eye-candy animations can come in. Weston’s desktop-shell provides for two types of animations, a preliminary one to fade the screen out, and one for the lock screen itself.
The fade-out animation kicks in when the idle counter reaches zero. A surface is created that spans all outputs and overlays all the clients with a transparency that gradually shifts to opaque black over a short period of time. When this animation sequence completes, Weston then removes the fullscreen, panel, and top-level layers; this prevents anything from showing or receiving input events and causes the render loop to skip over all client applications. The compositor is then put into sleep mode, and all seats lose their keyboard focus.
After that, once the system is locked, any input activity will be directed to the lock screen animation. In Weston, this simply displays a colored circle that you click on. Then, this invokes the unlock functionality which reverses the changes the lock put into effect.
If you’ve made it through this blog post, you now have a reasonably good idea of how idle behavior is managed at the protocol level in Wayland and implemented in Weston. In part 2, we’ll examine the newly-proposed idle inhibit protocol, and how it changes the behaviors.
About Bryce Harrington
Bryce Harrington is a Senior Open Source Developer at the Samsung Open Source Group focusing on Open Source Graphics. Prior to Samsung, he lead Canonical, Ltd.'s Ubuntu X.org team, and focused on stabilization of the graphics and input infrastructures for the Ubuntu distribution. Bryce began his career in the aerospace industry as a spacecraft propulsions engineer at The Aerospace Corporation, Hughes Space and Communications and TRW. Later, he joined the Open Source Development Labs as a Senior Performance Engineer working on NFSv4 testing and development of automated test systems. He is a founder and developer of the Inkscape project and serves as Chairman of the Inkscape Board. Bryce has a BS-AE from USC and an MS-AE from Caltech.