← Otto Developer GuideFile Previews

How otto-files shows what is inside a file: grid and list thumbnails, the preview column at the end of column view, the Peek panel, and video playback in the last two.

Behaviour is specified in specs/peek.md and specs/file-browser.md. This page describes the structure as it is built today, and it disagrees with the Peek spec in several places; see Where the spec has drifted. Three neighbouring pieces have their own pages: the video worker, otto-media-kit; the words recognised in a picture, Text in Pictures; and the icon a preview stands in for, File Icons.

The pieces

components/
├── otto-kit/src/preview/          Preview payload types, layout, draw, zoom maths (pure Skia)
├── otto-kit/src/filetype/         name → type of record, bytes → sniffed type
├── otto-peek/                     decode and present: the worker and the session hosts embed
│   ├── src/spawn.rs               open a file, run a worker on it, read the payload back
│   ├── src/sandbox.rs             rlimits, namespaces, fd hygiene, --sandbox-selftest
│   ├── src/payload.rs             the wire format ("OQV2") and its validation
│   ├── src/decode/                one decoder per content type, run in the worker
│   ├── src/opening.rs             entrance/exit geometry for the panel
│   ├── src/session.rs             Session, zoom/pan, Video, decode request sizing
│   ├── src/decode_cache.rs        recent decodes, so reopening a file skips the worker
│   ├── src/ocrcache.rs            words recognised in pictures, kept between runs
│   ├── src/draw.rs                a Session's content and badge, the same in every host
│   ├── src/badge.rs               the text-recognition badge in the preview's corner
│   ├── src/imagesize.rs           a picture's size from its header
│   ├── src/render.rs              PNG rendering for the standalone CLI only
│   └── src/main.rs                `otto-peek` CLI: --describe, --render, --filmstrip
├── otto-media-kit/                video: Player (library) + otto-media-worker (GStreamer)
├── otto-preview/                  one file in a window, around the same Session
└── otto-files/src/
    ├── thumbnails.rs              in-memory thumbnail store (no threads, no I/O)
    ├── thumbcache.rs              read-only freedesktop thumbnail cache
    ├── pane_surfaces.rs           Peek's overlay and the preview video's subsurfaces
    ├── view.rs                    draw_peek, preview column stage, draw_thumbnail
    └── app.rs                     wiring: sync_thumbnails, start_preview, start_peek

Three paths use this code. Every file byte is interpreted in the same sandboxed decode worker, which is the host binary run again with --decode-worker:

                         ┌─────────────── otto-files process ───────────────┐
 grid/list/column rows ─▶│ thumbnails::Store ─▶ thumbcache::lookup ─(miss)─┐│
 preview column ────────▶│ start_preview ───────────────────────────────── ┤│
 Peek (Space) ────▶│ start_peek ─────────────────────────────── ┤│
                         │               spawn_blocking: decode_path ◀──────┘│
                         └──────────────────────────┬────────────────────────┘
                                   fd 3 = the file   │  stdout = OQV2 payload
                                                     ▼
                         otto-files --decode-worker  (sandboxed, one per decode)
                                                     │
                     video/* card + media worker available?
                                                     ▼
                         otto-media-worker  (GStreamer, one per playback)

Every host must call otto_peek::run_worker_if_requested() as the first thing in main, before any thread or Wayland connection exists (otto-files does it in main.rs). If it doesn’t, the worker is the host re-executed as an ordinary host, which starts a second file browser instead of decoding anything.

The payload

Decoders don’t draw. A decoder produces an otto_kit::preview::Preview, and every host draws that with otto_kit::preview::draw. The set of variants is closed:

A new content type adds a decoder that returns one of these, not a new variant.

The icon chain is filled in two places. The worker fills it from the sniffed MIME type (decode::decode). The parent fills in a chain derived from the file name only when the chain is still empty (payload::with_icon), which happens for payloads no decoder produced: a crash, a timeout, a file that couldn’t be opened. When the two disagree, the worker’s chain is used, because the name is only a guess.

Wire format (payload.rs). The format is written by hand, without serde: the magic OQV2, a one-byte tag, then little-endian integers and strings prefixed with their length. On the host side payload::decode:

• never panics on truncated input (Cursor::take uses checked arithmetic);
• rejects any length above 512 MiB, and any string that isn’t valid UTF-8;
• caps preallocation, so a hostile length can’t reserve memory up front;
• refuses a pixel buffer whose length isn’t w × h × 4;
• turns a bad magic or an unknown tag into None, which spawn::run turns into Unavailable.

Pixels::to_image checks the length once more before handing the buffer to Skia. Tests at the bottom of payload.rs reject every truncated prefix of a valid payload.

The decode worker

Spawning (spawn.rs)

open(path) opens the file with O_NONBLOCK, calls fstat, and refuses FIFOs, sockets and devices, which could block on open or on read. decode_path wraps the whole sequence and always returns a Preview, even when the open fails.

run then:

1. Starts current_exe() --decode-worker --width W --height H --page P --zoom Z --name NAME with a cleared environment. Only PATH, RUST_LOG (default error) and LANGUAGE are passed, the last so that reasons and card facts come back in the user’s language.
2. In pre_exec: dup2s the file to fd 3, closes every other inherited descriptor (close_range), and applies the sandbox.
3. Reads the worker’s stdout to the end on a helper thread, waiting at most 8 s (DEADLINE). A worker that overruns is killed and the result is Unavailable.
4. Ignores the exit status. Only the bytes on stdout count: a crashed worker produces a truncated payload, and truncated payloads fail validation.

The worker never learns a path. Its name argument is used for titles and to narrow a sniffed type to a subtype, never to choose a decoder.

Containment (sandbox.rs)

sandbox::apply runs twice: once in pre_exec and again at the top of the worker. It does the following, in order:

1. chdir("/");
2. PR_SET_NO_NEW_PRIVS;
3. RLIMIT_AS 1 GiB, RLIMIT_CPU 10 s, RLIMIT_FSIZE 0, RLIMIT_NOFILE 64, RLIMIT_CORE 0;
4. unshare(CLONE_NEWUSER | CLONE_NEWNET), falling back to CLONE_NEWNET. This is best effort and failures are ignored.

If apply fails inside the worker, the worker answers Unavailable("peek-error-sandbox") rather than decoding without containment. Decoders also read through read_capped (512 MiB by default) and apply their own limits, listed below.

This is not a filesystem jail. There is no seccomp filter and no mount namespace, so a compromised decoder can still open and read any file the user can read. The code says so, and otto-peek --sandbox-selftest reports it (can_still_open_other_files) together with the rlimits and a network probe. The self-test runs the real sandbox in a real child with a piped stdout, because RLIMIT_FSIZE = 0 would raise SIGXFSZ on a file-backed stdout. Keep its output in step with any change to sandbox.rs.

Dispatch and decoders (decode/)

decode::previewed:

• sends a directory to listing::directory;
• answers a zero-length file with an “empty file” card;
• otherwise reads 4 KiB and calls filetype::refine_opt(filetype::sniff(head), name). The content decides the type, and the name may only narrow it to a subtype (an SVG behind an XML declaration, for example).

dispatch then tries these in order:

The only dependencies are libc and otto-kit (Skia included): there is no image crate, PDF library, archive crate or syntax highlighter. That is a deliberate choice. PDF rasterisers are run as programs rather than linked, so a new format can often be supported by adding a command to a table.

Editing a PDF is apart from previewing one. Studio fills a PDF’s form and marks it up through the otto-pdf crate, which loads PDFium (libpdfium.so) when first asked for and never links it; see specs/preview-app.md, Form and markup. Peek and the decoders above don’t use it.

Request sizes are in physical pixels. The host already asks for about twice the panel size, so decoders must not double it again.

Thumbnails

The store (thumbnails.rs)

thumbnails::Store has no threads and does no work itself. The host asks it which thumbnails are wanted, runs those jobs, and reports each result back:

• Requests. Browser::sync_thumbnails runs once per update and considers only the active column. Parent Miller columns draw thumbnails the store already holds but never ask for new ones. It works out the visible range for the current mode (grid, list or Miller pane; search results and Recent are ordinary columns) and skips directories.
• Key. Absolute path plus mtime. A different mtime makes the entry refetchable, and image() won’t serve a stale one.
• Throttling. At most MAX_IN_FLIGHT = 4 jobs at a time, taken top to bottom from the visible range, with no other prioritisation.
• No cancellation. Jobs for rows that scrolled away still run to completion, and changing folder doesn’t reset the store (Store::clear has no callers). Entries live until evicted.
• Eviction. At most 512 entries or 96 MiB (w × h × 4 for ready images), whichever is hit first. The oldest insertion goes first, and pending slots are never evicted.

Fetching (thumbnails::fetch)

App::start_thumbnail runs each job with tokio::task::spawn_blocking:

1. thumbcache::lookup reads the freedesktop cache in-process.
   • Root and size. $XDG_CACHE_HOME/thumbnails. It picks the smallest size bucket at or above the request, otherwise the largest smaller one.
   • Name. md5(file URI).png, with MD5 and the URI escaping written by hand.
   • Validation. Thumb::MTime is compared in whole seconds, read by a small PNG text-chunk walker, and the image is loaded with Image::from_encoded.
   • Failures. fail/otto-files/ markers are honoured on read.
2. On a miss, it generates a thumbnail only when may_generate is true. That is Kind::Image today, so PDF and video thumbnails come from the shared cache or not at all. Generating calls otto_peek::decode_path with a thumbnail-sized Request: one sandboxed worker per thumbnail.
3. Only Preview::Pixels becomes a thumbnail. Cards, text and listings leave the type icon in place.

The cache is read-only. Nothing is written, neither thumbnails nor failure markers. A miss is remembered only in memory, as State::Absent.

Sizes. Size::for_box(edge, scale) rounds up to a 128/256/512/1024 bucket. The box edge is 64 pt in grid and 18 pt in list and column views. AppContext::scale_factor() is an integer, the drawing surface’s scale rounded up, so a fractional scale asks for the next whole one.

Back on the UI thread. Store::finish bumps epoch when a picture lands, and the worker calls request_wakeup. The Miller pane picture keys include the epoch. That epoch is global, so every landing invalidates every pane, not only the one showing that file.

Drawing

All three views go through view::draw_thumbnail, which aspect-fits the image (never enlarging it), aligns it to the bottom of the icon box, and adds a hairline:

• list and grid call it from their immediate-mode row and cell drawing;
• column view uses scene.rs rows, which prefer thumb over icon.

A thumbnail replaces the type icon on the next repaint, with no placeholder or fade. The draw uses SamplingOptions::default(), so a 128 px bucket drawn into an 18 pt row is sampled nearest-neighbour.

The preview column

The trailing pane in column view (view::PREVIEW_W = 280) is shown when Browser::preview_visible holds: column view, exactly one selected entry, and that entry is not a directory. It does not use thumbnails: it runs a full Peek decode at panel size.

• sync_preview_target replaces PreviewPaneState and bumps its generation on every selection change. There is no cache, and an in-flight decode is not cancelled.

• App::start_preview runs peek::decode(path, panel, scale) on a blocking task. The width is 280 × scale × 2 and the height is the window height × scale × 2, both clamped to 64–4096. finish_preview drops results from an old generation.

• Drawing. The stage (draw_preview_stage) draws:

  • the video, if there is one;
  • otherwise a card’s hero, or the large type icon while the decode is pending or unavailable;
  • otherwise otto_kit::preview::draw at Zoom::FIT.

  The caption (preview_info) shows kind, size and modified time.

• The whole pane is a picture_cached scene layer (files-preview). Its key covers the name, scroll, whether a result has arrived, the video, the theme and the size.

• A video opens paused on its first frame and plays on click, because the column follows the arrow keys; see Video.

Peek

A session

peek::Session is the whole state of one open panel: the current Preview, first_row, Zoom, the two pan ScrollViews, an optional Video, expanded, opened_at, and whether it is closing.

Opening and moving between files work like this:

1. Space, or the palette’s Quick Look, calls start_peek. The palette can’t run it itself: run_request returns Followup::Peek, and the host drains that in follow_peek.
2. begin_peek bumps peek_generation and replaces the session with Session::awaiting(name, is_dir, anchor). The old preview, zoom, pan and video are dropped, while opened_at and expanded are kept, so moving to another file doesn’t replay the entrance. The panel shows “Opening preview…” immediately, never the previous file’s content.
3. A blocking task runs peek::decode, sized at panel × scale × 2 and clamped to 64–4096. finish_peek discards a result whose generation is stale.
4. An arrow key, Home, End or Page Up/Down with the panel open moves the selection and calls start_peek again.

Stale decodes are ignored, not killed. The superseded worker keeps running until it finishes or reaches the 8 s deadline. close_peek also bumps the generation, so a slow decode can’t reopen a panel that was dismissed.

The surface

The panel is a child of an overlay layer-shell surface (namespace otto-peek), created and synced by PaneSurfaces::sync_peek and destroyed when the exit finishes:

• Overlay. Anchored to every edge of the output the window last entered, exclusive zone -1, exclusive keyboard. Its transparent ground is painted at the configured size, because Otto hit-tests a layer surface against its buffer; a press on it closes the panel. Destroying it hands the keyboard back to the window, or the desk, it was taken from.
• Position. Centred on the usable area of the display, worked out in window points. request_output_frame is asked of the window’s own style (once when the panel opens and once when the exit starts) and of the overlay’s; the difference between the two answers moves window-point rects onto the overlay. The panel stays hidden until both are in.
• Material. Set through otto_surface_style_v1: corner radius 12, a shadow, and BackgroundBlur when frosting is on (OTTO_FROSTING), Normal otherwise. The client draws no shadow of its own.
• Stacking. Above everything, as an overlay. restack orders only the window’s own subsurfaces: columns, preview video, pan bar, palette, catcher.
• Repaints. The panel is repainted only when peek_key changes (rect, generation, first_row, zoom, scrollbars, loading, the video’s frame sequence), and not while a frame is in flight. A resize is claimed only once the matching buffer has been painted (otto-kit’s PlacedSurface::set_rect holds it for the next paint), because the style protocol applies a size immediately and would stretch the old buffer. A hidden panel drops its input region.

Entrance and exit use the geometry in otto_peek::opening:

• entrance. A single uniform scale from the anchor (the cursor entry’s icon rect, view::peek_anchor) to the resting rect, clamped to 0.04–1. With no anchor it swells in place from 0.96.
• Timing. sample is a spring with a small overshoot and runs 300 ms in; sample_out is a smoothstep and runs 180 ms out.
• Who animates. The client does, resizing and moving the surface every frame (Session::panel), with the frame loop kept alive by peek_animating(). Comments in opening.rs and lib.rs still say the compositor runs it through surface-style transactions; it doesn’t.
• Chrome. The 30 pt title strip, close dot and expand dot fade in with the card’s size (peek_chrome_opacity), so they don’t fill the first frames.
• Closing. close_peek moves the session to peek_closing, re-reads the anchor, and tick_peek_exit retires it. A playing video keeps being drawn through the exit.

Expand (Session::toggle_expanded) swaps the resting rect: normally 72% of the space with a 420×320 minimum, expanded it fills the space less a 12 pt margin. The setting survives moving to another file.

Drawing and input

view::draw_peek draws the card background and hairline, the title strip, and then otto_peek::draw::content: either otto_media_kit::view::draw for a video or otto_kit::preview::draw(content, preview, first_row, zoom, icons) with the picture edges and word selection, followed by the pan scrollbars. otto_peek::draw::badge adds the text-recognition badge. Studio (otto-preview/src/content.rs) calls the same two, laying its PDF form fields and marks over the picture through the overlay hook.

otto_kit::preview owns layout and the zoom geometry, so every host clamps the same way:

• Scale. Zoom { scale, offset, band } runs from fit to MAX = 8×, and snaps back to fit below SNAP = 1.02.
• Pinch. zoom_about keeps the focal point fixed during a pinch. Hosts track the pointer position themselves, because the pinch protocol reports only the drift since the gesture began.
• Limits. clamp_zoom holds offset inside the picture’s slack and passes band (the rubber-band stretch) through untouched. Only zoomed adds it at draw time.
• Resampling. Pictures are drawn with the Mitchell cubic resampler over a checkerboard.

Input in otto-files:

The pointer reaches the panel two ways: on its own surface (install_peek_pointer, surface-local) and on the toplevel (a press outside closes). The two share hit rects.

Zoom does not decode again. Request.zoom exists and image::raster supports it, but peek::decode always asks for zoom 1. An 8× zoom enlarges the panel-sized decode, which is about 2× the panel’s pixels.

Words. A picture that has been read carries its words on Preview::Pixels, and draw_selection paints the selection over the picture after it is drawn. The recogniser, the cache and the badge are on Text in Pictures.

Animation. first_row doubles as the frame index for an animated picture, and tick_animation advances it from the update tick for as long as the panel is open. Because first_row is part of peek_key, each frame repaints the panel’s surface alone. The strip itself is built by the decoder; see File Icons.

Accessibility. An open session is published with A11yTree::preview and takes focus.

Dragging the panel

The panel is dragged by its title strip. view::peek_grip_rect is that strip minus the buttons, with 4 pt of clearance before the expand dot; the content is not a handle.

peek_grip records the press point and the panel’s top-left at the press, and takes a double click inside the double-click window as expand instead. drag_peek_to then places the panel at origin + point - grab.

Nothing is read back from the render path. A pointer holding a button stays in a grab whose focus surface and coordinate frame are fixed, so measuring against a per-frame rect would make the drag run away from the cursor.

place_peek clamps the left edge inside the bounds and the top so that at least the title strip stays on screen. The strip is the only thing that can bring the panel back. clamp_peek re-clamps when the window or the display changes, against where the panel is asking to be rather than where it was last put. Bounds are the display once the compositor has answered request_output_frame, and the window until then.

The offset is folded into the resting rect, so surface placement, drawing and hit rects cannot disagree, and the panel’s content key is its size rather than its rect. Dragging repaints nothing and only moves the panel’s surface. Expanding recentres. The offset survives arrow-keying to another file, but not closing the panel, since a session is built afresh.

app/peek_drag_tests.rs names the rules it holds: the_title_strip_takes_hold_of_the_panel, the_content_does_not, a_drag_over_the_panels_own_surface_does_not_run_away, a_panel_cannot_be_dragged_off_the_display, clamping_every_frame_does_not_walk_the_panel.

Video

Playback is not a decoder. The decode worker returns a video/* Card as usual, and the host decides whether to play it:

1. payload::is_video(preview) must be true. That means a card whose first icon is a specific video-* name, so the choice follows the sniffed bytes and not the file extension.
2. otto_media_kit::player::available() must find otto-media-worker, looking at OTTO_MEDIA_WORKER, then next to the executable, then PATH.
3. peek::Video::open calls Player::open(path, options, wake), with the frame size limited to panel × scale (64–3840 × 64–2160). The card stays underneath as the fallback. Its hero would be the poster, but media::video never produces one, so a video shows its icon until the first frame arrives.

In the Peek panel the video autoplays and is drawn into the panel’s own surface. Session::video_key is part of peek_key, and the player’s wake requests a frame. Video::pointer handles input:

• a click on the picture or the play button toggles playback;
• mute switches the volume between 0 and 1;
• scrubbing pauses, runs fast keyframe seeks while dragging, then one accurate seek on release, and resumes playback if it was playing.

Moving to another file drops the session and with it the Player, which kills the worker.

In the preview column the video opens paused. The worker delivers the preroll frame so it isn’t black. PaneSurfaces::sync_preview_video gives it its own subsurface, so a 30 fps clip repaints only that surface and not the toplevel or the cached files-preview picture, whose key leaves the video out when it is on a surface. That surface:

• exists only in column view, and only while Peek is closed;
• stays hidden until VideoSnapshot::aspect() is known (frame, then announced size, then poster);
• is sized by view::preview_video_box to width / aspect + transport height, and hidden if that would spill out of the viewport;
• has an empty input region. The toplevel hit-tests the same box through Browser::preview_video_pointer.

otto-media-kit in brief

The full description is on otto-media-kit. What a host needs to know:

• Two halves. The library (default-features = false in hosts, no GStreamer) and otto-media-worker (feature worker, the default). The worker needs Rust 1.92 or later because of the gstreamer 0.25 crates; the library alone doesn’t.
• Descriptors. The file on fd 3 and a memfd frame ring on fd 4: a 4 KiB header, then three RGBx slots. Commands go as lines on stdin, events come back as lines on stdout.
  • Commands: play, pause, seek <ns> accurate|fast, volume <0..1>, quit.
  • Events: ready <w> <h> <duration_ns|-1>, frame <slot> <seq> <pts_ns>, position <ns>, playing, paused, ended, error <text>.
• Threads. Player reads events on its own thread (otto-media-events), copies each announced slot out immediately, and calls wake after every event, so wake has to be cheap. Hosts draw from Player::state() and frame() snapshots, or pass them to view::draw_frame on another thread.
• Lifecycle. Dropping a Player sends quit and then kills the worker straight away. A worker that exits while loading, playing or paused turns the state into Failed. There is no PR_SET_PDEATHSIG: a worker whose host dies exits when stdin reaches EOF or a write to stdout fails.
• Pipeline. playbin3 (or playbin as a fallback) with videoconvert ! videoscale ! capsfilter(RGBx, [1,max], par 1/1) ! appsink sync=true max-buffers=2 drop=true, playing file:///proc/self/fd/3. Audio goes through playbin’s default sink inside the worker.

Debugging

Unit tests cover:

• the payload round trip and rejection of bad input;
• decoder limits, text encodings and PDF page counts;
• opening.rs geometry;
• zoom and pan in both otto_kit::preview and otto_peek::session;
• the thumbnail store’s throttling and eviction;
• cache naming and mtime checks;
• the media-kit protocol and transport layout.

Nothing runs a real decode worker or media worker in cargo test. The CLI above is the way to check them.

Known gaps

Thumbnails

• Cache writes. Nothing writes to the shared thumbnail cache, including the failure markers it reads. The spec’s “Peek is a thumbnail producer” is not built (Opened::mtime and len are marked dead_code).
• Cancellation. Neither thumbnails nor previews are cancelled. Scrolling fast or arrow-keying quickly leaves workers running up to their deadline.
• Invalidation. The thumbnail epoch is global, so a thumbnail landing invalidates every Miller pane picture.
• Loading cost. Cached thumbnails load lazily with Image::from_encoded, so the PNG is decoded on first draw, on the render thread.
• Scale and sampling. Thumbnail sizing truncates fractional scales, and thumbnails are drawn with nearest sampling.
• Failure markers. is_known_failure reads its marker on the UI thread, once per file the store hasn’t seen before.

Decoding

• Sandbox. The decode worker can still read the user’s files: there is no seccomp filter or mount namespace.
• Payload size. The total payload isn’t bounded, only each field of it.
• Dispatch. may_generate is decided by the entry’s Kind, but the worker dispatches on the sniffed content. A file named like an image whose bytes are a PDF therefore reaches the PDF rasteriser while thumbnailing.
• Formats. Video has no poster frame, and only MP4/QuickTime headers are read; a moov atom at the end of a large file isn’t found. Folders get no preview column. Text has no highlighting.

Peek and media kit

• Zoom. Zooming never decodes again, so detail stops at about 2× the panel’s pixels.
• Frame ring. otto-media-kit resizes the ring with set_len before the host has read the matching ready. If a stream’s frame size shrinks mid-playback, the host can read a stale frame through its old mapping past the end of the memfd and get SIGBUS. The ring is also remapped on every ready, and ready follows every flushing seek, so scrubbing costs an mmap/munmap per seek.
• Worker path. OTTO_MEDIA_WORKER isn’t checked for existence, so available() can report true and Player::open then fail.

Where the spec has drifted

specs/peek.md still carries parts of earlier designs:

• Status. It says “draft — nothing implemented”.
• Descriptors. It describes three descriptors (file, memfd, status pipe). The code uses fd 3 and a stdout pipe.
• Animation. It says the compositor runs the animation through surface-style transactions. The client animates.
• Dropped designs. It keeps D-Bus-era text (SetIndex, SetUris, a five-second service grace, layer-shell overlays).
• Zoom. It promises that zooming decodes again.

Several code comments are stale for the same reasons: opening.rs and lib.rs on the animation; pane_surfaces.rs claiming the panel’s input region is empty; comments in view.rs and app.rs about the panel being drawn into the window’s own surface; and app.rs mentioning D-Bus activation. If you change either document, check this list first.