January 8, 2016 - Mike Blumenkrantz
Desktop Compositing: How Much Overdraw Is Too Much?
In my previous post on compositing, I covered the overall methodology with which I implemented Compiz plugin support in the Enlightenment compositor. Now I’m going to go into some detail about the rendering portion.
The first thing that should be reiterated is Compiz only performs damage calculations for the entire screen and does not track them per-window. This is, to say the least, problematic. There’s no way to easily predict where a window will draw at any given time either: there are window effects that cause clients to zoom in/out to/from the mouse cursor, and others that cause the client to bounce around outside of its frame region. The compositor must be prepared to draw things for each window at any geometry on the screen at any time, regardless of common sense clipping.
Rendering Outside the Box
The first attempt I made at getting things to work was to just get anything to render. This involved a lot of code-surgery inside Compiz to remove overall screen effect drawing, lighting effects, and some GL calls that were almost as old as I am: things of this nature. Compiz was allowed to render window contents and only window contents; whenever it had updates to render the sandbox would be called into, and all the effect plugins would get their turns to throw pixels around. Finally, I added a method for overriding the GL surface of a client’s rendered compositor image and stuck the texture into it for testing. Predictably, the client rendered on-screen upside down and without any effects.
The next step was equally predictable: putting the entire Compiz GL surface into the compositor object for rendering. As expected, the effects looked terrible because they were clipped by the client’s geometry. A wobbly window that cannot freely wobble is no wobbly window at all, says I!
This posed a bit of a technical problem, however. Evas has no mechanism (that I’m aware of) for saying “this is the image’s geometry, but also draw outside that geometry whenever I tell you to.” Rather, all images are clipped to their bounding box. There was only one option: use all of the GPU’s available power.
Given that the window damages only exist for the entire screen, the easiest choice, and the only choice that I could think of at the time, was to make an image object the size of the screen for every window and do fullscreen renders; always. The image ignores mouse events and is added inside the overall client’s compositor object to preserve stacking. Then, it’s just up to the Compiz integration to ensure the window renders in the “right” place in order to fool the user when, in fact, the rendered image for the window is in no way tethered to or inside of its frame, despite the appearance.
Wobbling Our Way to Functional Effects Modules
With this done, only some visual artifacts remained. The issue here was that when an effect draws outside the boundary of the window geometry, at what point do you clear the screen of the past renders? For example, if a window is animating a slide in from off screen, how can I determine when to clean up the “effect” renders so that only the normal window content remains? This turned out to be a big blocker: every time I thought I’d figured out a reasonable algorithm for detecting effect draws vs client draws I’d find a corner case where something didn’t work as expected.
In the end, there were a number of changes that needed to be made. For the specific case of the wobbly plugin, issues arose when releasing a window that had achieved a high velocity, and the window would get stuck in a “wobbled” state. To avoid this, the Compiz integration layer tells Compiz that the window has stopped being dragged only after all the rendering has completed, then it performs a full redraw for that window’s full-screen image object.
Another case was the animation plugin where windows zoom in and out of the cursor position. This was more difficult to track down as it is important to clear out the animation frames in order to have an effect that doesn’t look like garbage. In order to solve the issue, I ended up tracking the GL drawing vertices for the window on each frame; if they changed between the pre-draw and draw phases then an effect was mangling things and the entire window should be redrawn. Probably.
More Work for Another Time
Some other smaller hacks were done here and there, but this is the gist of how things work under the hood. The last known issue that remains, aside from the lack of capabilities to draw any of the screen effects, is that compositor object mirrors are broken for windows. This means that the compositor is unable to create copies of window contents for reuse in other places, such as the pager gadget. I’m not entirely certain why this is happening. The image mirrors don’t render anything at all when the bound texture is passed to them, so attempting to scissor anything for a drawable wasn’t worth my time. It’s confusing, and I lacked the motivation to really dig into this any more than I already had.
I imagine that screen effects would not be terribly difficult to add; another full screen image object could be used for the whole thing, and it would probably render okay like that. Something for a rainy day, I suppose.
The downside of this method is obvious, since it means that for many frames the compositor is doing fullscreen GL draws, potentially multiple times. Using a powerful GPU it’s unlikely that anyone will notice much slowdown, but this is obviously not anything suitable for embedded systems. Then again, Compiz uses GL calls from ancient versions of (desktop) OpenGL, so it would need a full API interception layer in order to get this working on an embedded system in any form, or even on Wayland.
About Mike Blumenkrantz
Mike is the release manager for Enlightenment as well as a core developer of the EFL toolkit. He sometimes contributes to the Servo application embedding API/ABI, and in his free time he attempts to write desktop applications.