← Otto Developer GuideDRM Planes and Direct Scanout

How Otto hands parts of the screen straight to the display hardware, so the GPU does not have to composite them.

Structure and rationale live here. The full behavioural contract — per-buffer damage rules, blur-composite invalidation, promotion hysteresis — is specs/plane-scanout.md.

Reparenting a layer raises it

add_sublayer on a layer that is already a child moves it to the end of its parent’s children — which is exactly how raise_window_to_front raises a window. Anything that reparents a window layer therefore raises it as a side effect, whatever its place in the stack was.

Promotion moves a window’s subtree out of the workspace’s windows_layer and into the output’s promoted-plane container; demotion moves it back. That move back is a raise, so a window demoted after another one was mapped came back on top of it and stayed there — the workspace’s windows_list is the stack, and WorkspaceView::restack_windows re-applies it after every demotion.

The promoted plane also scans out above the windows plane, so anything left promoted while it is no longer the topmost window covers the window that is. Promotion waits for its candidate to settle (PROMOTE_STABLE) to avoid moving a live subtree between containers every frame; demotion must not wait, or a newly mapped window comes up behind the one it should be covering.

What a plane is, and why you would want one

A display controller can read from more than one buffer and blend the results itself, on the way to the cable. Each of those inputs is a plane: one primary plane (always present), zero or more overlay planes, and usually a dedicated cursor plane.

The analogy is a physical animation cel stack. Instead of repainting one sheet every frame, you draw the background once, the characters on a second sheet, and the caption on a third — then let the camera stack them. Moving the caption means moving one sheet, not repainting the picture.

Concretely: if the dock is on its own plane and only the dock is animating, the GPU renders a small strip and the display engine composites it over untouched buffers. The background, the windows and everything else are not touched at all, and often no GPU work happens for that frame whatsoever.

What Otto actually does

Most compositors use planes opportunistically — “if a fullscreen window happens to be scanout-compatible, promote it”. Otto goes further and deliberately splits its own scene into per-purpose buffers so there is something for the planes to take.

Plane decomposition

For the same decomposition seen from the other end — which scene subtree each of these buffers is rendered from — see the overview in The Scene Graph.

Per output, front to back: the dock strip, the app-switcher strip, overlay UI, exposé, the promoted client window, the windows buffer, and the background. Each of those is a subtree of the lay-rs scene — see The Scene Graph for how the tree is shaped and why. (When the session is locked, only the lock plane is composited — nothing that could hold a window is even consulted.)

Two details are load-bearing:

The rest are ARGB8888 and non-opaque, since they have to blend.

A buffer is re-rendered only when damage lands under its subtree. An idle desktop produces no re-renders and no page flips.

When decomposition is enabled

SurfaceData::planes_enabled (src/udev/device.rs) requires all three:

When any fails, the output renders as a single scene element — the ordinary path — and Otto logs why under the otto::planes target.

NVIDIA: overlay planes are cleared entirely at surface creation, because overlay usage on those drivers is broken. That also disables decomposition, by way of the overlay count check.

How assignment actually happens

Otto does not assign planes. It hands Smithay’s DrmCompositor a list of render elements, and render_frame() tries each one on a plane, front-most first, every frame. For each element it:

  1. Calls element.underlying_storage(renderer). None means the element has no importable buffer and must be GPU-composited.
  2. Exports a dmabuf and adds a framebuffer via the framebuffer exporter.
  3. Tests plane compatibility — format, transform, z-order, size — with try_assign_plane().
  4. Assigns it if compatible; otherwise the element is rendered by the GPU into the primary swapchain, exactly as a plane-less compositor would.

So plane usage is best-effort and re-decided per frame. There is no probing step and no cached tier: acceptance is delegated to the kernel, and rejection is a normal, silent fallback.

For an element to be eligible it must provide underlying_storage(), report Kind::ScanoutCandidate, be backed by a dmabuf (not CPU memory or an anonymous GPU texture), and use a format and transform the plane supports.

Direct scanout of a client window

The topmost window can be promoted: its own client buffer goes straight to a plane, so its pixels never pass through Otto’s renderer at all. This is the big win for video players and games.

Promotion has hysteresis: the candidate set must stay unchanged for 500 ms (PROMOTE_STABLE) before anything new is promoted, because flapping between promoted and composited is visible as a flicker. Its shadow is drawn separately by the windows buffer.

Otto also publishes dmabuf feedback to clients, with a scanout tranche alongside the render tranche, so a client can allocate buffers in formats that are promotable in the first place (get_surface_dmabuf_feedback in src/udev/feedback.rs).

The blur problem

Planes composite in the display engine, which means a translucent plane cannot see the planes below it — but Otto’s dock, menus and OSD are frosted glass and must show what is behind them.

Otto solves this by compositing the lower planes into a separate, downscaled image, blurring it once as a whole, and letting each blur-bearing layer seed itself from that pre-blurred image (src/udev/backdrop.rs). Blurring once per frame rather than once per consumer is both cheaper and more correct: blurring inside a layer’s rounded clip samples transparent pixels at the shape edge and leaves a faded rim, while a whole-image blur has no edge to sample across.

Promoted client windows are folded into that composite by blitting their dmabuf — the same buffer KMS scans out — so a shared window does not vanish from the blur behind the dock.

Debugging

# What Otto decided, per output
RUST_LOG=otto::planes=info cargo run -- --tty-udev

# What Smithay decided, per element per frame
RUST_LOG=smithay::backend::drm::compositor=trace cargo run -- --tty-udev

Useful trace lines:

Build with --features debug-kms for extra KMS logging.