From 61005f3dc119625608d357a1dbb57d4f67ce3ccb Mon Sep 17 00:00:00 2001 From: OneSlip Date: Thu, 5 Nov 2026 17:32:26 +1000 Subject: [PATCH] Culling: the loupe uses built previews; photos prepared ahead and in the background (#3077) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * PERF-CULL-0: ui.inspect reports how the loupe showed each photo it was stepped to perf.loupeSteps records, per step, the ms from the step to the first frame with anything of the photo, with a full-size picture (camera JPEG, cached or rendered) and with LightCraft's own render, measured on the frames the app draws, so culling benchmarks exclude the control channel's round trip. Co-Authored-By: Claude Opus 5.5 (cherry picked from commit 95f0343848fc0a238f80f07ceb292eaf6779aeaa) * PERF-CULL-1: browsing a folder shows its first photos within a second The Local view read and hashed every file of the folder before showing any of them (2,933 EOS R5 C CR3s: 30.6 s to the first photo). It now hands the files the library lacks straight to the import, ordered the way the view sorts them (file time for the date sorts), which adds them batch by batch: first photo 0.9 s, twelve thumbnails 1.4 s, the whole folder in the same ~34 s as before, in the background. Co-Authored-By: Claude Opus 5.5 (cherry picked from commit b766558e7ef4f802919a144da53e444f70e0186b) * PERF-CULL-2: the loupe makes the next photos' camera previews ahead of the step Culling steps through photos, and each step waited for the next raw's embedded JPEG to be read and decoded (p50 187 ms, p95 332 ms on EOS R5 C CR3s) and then for a full raw develop. Now: - the loupe's stand-ins for the next five photos in the direction of travel and the one behind are made ahead (Slot::QuickAhead, priority just below the loupe's own render) and taken over at the step, so a step shows the camera JPEG at once (p50 0 ms); - a slot keeps its photo while the photo stays wanted, so photos that shift places as the user steps are not made twice; - decoded camera JPEGs stay in the memory LRU (never on disk), so going back shows them again without a decode; - while an arrow key is held (a photo left within 0.25 s), the raw develop waits until the photo has stayed 0.12 s, and a render queued for a photo already left is dropped, so the workers make the previews instead of renders nobody sees. Co-Authored-By: Claude Opus 5.5 (cherry picked from commit 0430b95636d2b6adce8887af7b9464687c95549e) * PERF-CULL-3: CR3 previews take the smallest embedded JPEG that covers the view; plain sRGB JPEGs decode straight to 8-bit Upstream's embedded_preview_covering (14e0d128) picks the smallest covering JPEG of a TIFF-based raw; this extends it to Canon CR3 and adds the 8-bit decode: - A CR3 keeps an 8192 px JPEG track, a 1620 px PRVW box and a 160 px THMB. embedded_preview_covering now picks among the three (cr3_boxes lists the boxes), so a grid thumbnail or a loupe up to 1620 px decodes the PRVW. Loader at 1280 px on 20 EOS R5 C CR3s: 168-322 ms -> 31 ms a photo. - lightcraft_codecs::decode_jpeg_srgb8_fit decodes a plain 8-bit sRGB JPEG (DCT-scaled) and box-averages it in linear light straight into 8-bit sRGB, without the float image the general path builds; anything with an ICC profile, an Adobe RGB declaration, CMYK or 12-bit takes the general path. 45 MP preview fitted to 2560 px: 182 -> 151 ms (min of 15, loaded machine). Co-Authored-By: Claude Opus 5.5 (cherry picked from commit a67bb398c2f438cbe16f4db1431ae0270f1c8b04) * PERF-CULL-4: Camera preview label; thumbnails ahead of the step; 100 % shows the camera JPEG first - While the loupe shows a raw's embedded JPEG (or its thumbnail), a small 'Camera preview' label at the top right says so, with what that means on hover (widget loupe:cameraPreview; nine languages), so colour and tone are not judged on the camera's rendering. - The look-ahead also asks for the next photos' thumbnails (a small embedded JPEG each), so a held arrow key has something to show in full screen too, where no filmstrip asks for them (2400x1500 window, 40 held steps: camera JPEG on 36, nothing on 2; before 1 and 4). Stepping through a whole wedding keeps at most 600 thumbnails, those around the open photo. - Zoomed in on an unedited raw, its camera JPEG at full size (8192 px for an EOS R5 C) is drawn under the window render until that arrives: 100 % in ~0.9 s instead of ~5.6 s; released when not zoomed. Co-Authored-By: Claude Opus 5.5 (cherry picked from commit 9f212b1e16815c3c0ca33130319a7ef41fb05a26) * PERF-CULL-5: lightcraft-cli bench-cull and tools/cull-bench.py time culling a real shoot bench-cull FOLDER (headless) times reading the folder into the Local view, the first screen of grid thumbnails on the app's render threads, camera previews at 1280 / 2560 px and full size, a cold 2560 px loupe render, rating 50 photos in a row and importing N files in place / copying M, as 'name: X ms wall' lines (p50, p95, n) and JSON. Libraries are made under --work; nothing is written next to the photos. tools/cull-bench.py drives the running app over the control channel (browse, first photo, steps at rest, a held arrow key, 100 % zoom, rating) and reads the app's own frame timings (perf.loupeSteps). Co-Authored-By: Claude Opus 5.5 (cherry picked from commit bd2eaac591b72f8258acae4389c67aa4d1da3b27) * PERF-CULL-5: bench-cull --files K samples a slow folder; e2e test; cull-bench usage Co-Authored-By: Claude Opus 5.5 (cherry picked from commit c32c9556924ccdee6f8c103864df51c3e170df98) * PERF-CULL: docs for culling speed (parity rows, gap 26, preview reads, opening a photo) Co-Authored-By: Claude Opus 5.5 (cherry picked from commit fb69b5975b51f882d82b2ca684e0f7ddc8373908) * PERF-CULL: the whole-file preview decodes a plain sRGB camera JPEG the quick path's way Upstream's quick-path test wants the bounded read and the whole file to give the same picture byte for byte; with the branch's straight-to-8-bit decode on the quick path only, a 49 px preview came out differently. Both paths now take it. Co-Authored-By: Claude Opus 5.5 (cherry picked from commit 0c8acab4c2f3e85d1c51fab5faf62b6e997d7c73) * PERF-THUMB-FIRST: CR3 thumbnails decode the 1620 px PRVW copy, and grid thumbnails from camera JPEGs are kept in the thumbnail cache A CR3 grid thumbnail decoded the full-size JPEG track (45 MP, ~150 ms each) whatever size was asked; it now takes the smallest embedded copy that covers the cell (PRVW, same shape), and the box search stops at mdat (was a scan of the whole file). The decoded camera JPEG of an unedited raw is written to the library's thumbnail cache, so a folder seen before fills from small files. Measured on 250 R5 C raws (loaded machine): first screen fills in 294 ms (was 553), on a second visit 53 ms (was 610). Co-Authored-By: Claude Opus 5.5 (cherry picked from commit 12162f966dd859b7e53ac65b5c4564a8e837b7eb) * PERF-CULL: cargo fmt Co-Authored-By: Claude Opus 5.5 * PERF-CULL: docs for culling speed (parity rows LR-IMP-LOCAL, LR-IMP-PREVIEW-READ, LR-VIEW-DETAIL; gap 26) Co-Authored-By: Claude Opus 5.5 * PERF-CULL: review fixes (bench work folder, 8-bit APP1 cap, size sort) - bench-cull makes its libraries in a new folder of the run's own under --work and never deletes what is already there (it removed lib-browse, lib-add and lib-copy under a folder the caller named). - The 8-bit sRGB JPEG path refuses more than 128 APP1 segments before its decoders walk them again, as the general decode does. - Browsing by file size keeps unreadable files last in both directions (they were sized 0, so first when ascending). - --files K in the CLI synopsis and help. Co-Authored-By: Claude Opus 5.5 * PERF-PREVIEW-PREP: the loupe uses built previews; photos are prepared ahead and in the background Built standard previews were never what the loupe showed: it rendered every photo again (~1.8 s on a raw), the rendered previews in memory were capped at 128 MB (~84 photos) whatever the budget, the disk cache was a fixed 2 GB (a wedding's previews pruned each other), the cache's writer skipped disk writes when it fell behind, and Build Previews with one photo open built only that photo though the menu said "Visible". - The loupe's render job takes a cached view render of the same settings that is at least as big as asked for, fitted down (no render). - Memory for rendered previews is a quarter of the cache share; the disk cache's automatic size makes room for every photo (about 1.5 MB each); Settings offers Automatic, 50 GB and 200 GB. - A build skips photos whose previews are there, writes each to disk before counting it, builds at least the loupe's size, and with one photo selected builds the view. - Prepare photos in the background (Settings > Performance), per mode (culling, library, editing), app-wide (kept in ui.json) or per library: photos ahead rendered for the loupe (a few at a time, 0 = off) and the whole view's previews built in the spare time once the user steps through photos: two lanes, waiting while the photo on screen renders, nearest first in the direction of travel, one cancellable row, replaced by a build that is asked for. Commands prefs.backgroundPrep / prefs.setBackgroundPrep; ui.inspect perf.previewCache. Co-Authored-By: Claude Opus 5.5 * PERF-CULL: the bench work-folder test removes only a folder it made Co-Authored-By: Claude Opus 5.5 * PERF-CULL: browsing keeps the whole listing for the folder bookkeeping; cull-bench stops on small folders The browse worker queues only the files the library lacks, but the folder bookkeeping (browse times, records of files gone) needs every file listed: a subfolder of known photos only was not stamped as browsed. The known files now come along marked as already in the library. tools/cull-bench.py stops waiting at min(12, photos) thumbnails. Co-Authored-By: Claude Opus 5.5 * PERF-PREVIEW-PREP: review fixes (smart previews honour id, editing builds by the Library row, lenient prep settings) - library.smartPreviews with an explicit id works on that photo, not the whole view. - While editing, the whole view's background build follows the Library row (its switch and which photos); the Editing row's scope is for the photos kept ready ahead. - The app-wide and per-library preparation settings are read leniently: a value this build doesn't know is the defaults (app) or none (library), and the other settings are kept. Co-Authored-By: Claude Opus 5.5 * PERF-CULL: cull-bench counts twelve thumbnails while the folder is still loading Co-Authored-By: Claude Opus 5.5 * PERF-PREVIEW-PREP: a view change stops the background build for the old view The running background build was only re-aimed at the photo on screen; after another folder, filter or library it went on building photos no longer shown. It is now stopped when the view's signature changes (not counted as stopped by the user), and the view now shown is prepared. Co-Authored-By: Claude Opus 5.5 * PERF-CULL: cull-bench picks the thumbnail threshold from the final photo count Co-Authored-By: Claude Opus 5.5 * PERF-PREVIEW-PREP: the view signature covers every photo; the view-change test holds the build explicitly Co-Authored-By: Claude Opus 5.5 * PERF-CULL: cull-bench stops waiting for twelve thumbnails 30 s after the folder loaded Co-Authored-By: Claude Opus 5.5 * PERF-PREVIEW-PREP: in the marked scope, marking a photo changes what the view's build prepares Co-Authored-By: Claude Opus 5.5 * PERF-CULL: cull-bench times only the thumbnails the browse adds Thumbnails already cached before library.browse no longer count towards the first-thumbnail and twelve-thumbnail times. Co-Authored-By: Claude Opus 5.5 * PERF-CULL: count every thumbnail made, for the browse benchmark thumbTextures counts resident thumbnails, and evicting old ones can hide new ones. The renderer now also counts each grid thumbnail it stores (perf.thumbsMade, never lowered by eviction), and cull-bench times the browse from that. Co-Authored-By: Claude Opus 5.5 * PERF-PREVIEW-PREP: the marked-scope test waits for its builds before cleaning up Both preview builds finish before the test library is removed, and the removal is checked, so no worker writes into a folder being deleted. Co-Authored-By: Claude Opus 5.5 * PERF-CULL: stand-ins made ahead keep their order, size and photo Review fixes for the loupe's stand-ins and the browse benchmark: - a stand-in still queued ahead moves to its new place in the order when the user steps; - one made ahead still shows when the loupe's own request failed; - one made ahead for a smaller loupe isn't taken over by a larger one; - quick and full-size camera-preview keys include the photo, so a virtual copy gets its own; - perf.thumbsMade counts photos given a thumbnail, not a render replacing a stand-in. Each has a test that fails without it. Co-Authored-By: Claude Opus 5.5 * PERF-CULL: a late stand-in never takes another photo's place in the loupe A stand-in made ahead takes over after the loupe's own request failed only while the loupe still wants that photo and has no other request pending; the test covers stepping on before it arrives. Co-Authored-By: Claude Opus 5.5 * PERF-CULL: a stand-in made ahead and no longer wanted is dropped when it arrives Freeing a look-ahead slot whose job is already running now also forgets its request, so the late result isn't kept as a hidden texture, unless the loupe is waiting for exactly that stand-in. Co-Authored-By: Claude Opus 5.5 * PERF-CULL: a running stand-in the loupe waits for keeps its look-ahead slot When the next look-ahead no longer wants the photo the loupe has just stepped to, the slot whose job is still making that photo's stand-in stays reserved instead of going to a new photo, so the result still reaches the loupe. Co-Authored-By: Claude Opus 5.5 * PERF-CULL: CodeRabbit round 10 (browse order ties, library skip, CR3 boxes after mdat, badge once, look-ahead crop key, cli test binary) - browse_order: import- and edit-date sorts queue new files by name (they tie on those keys); ties go by name in the view's direction, as the catalog breaks them. - Local browsing never lists the library's own folder (browse_files, as scan_with skips it). - CR3 PRVW/THMB search skips only the mdat payload: boxes after it (a moov holding THMB) are still found. - The camera-preview badge and the loupe timing are settled once, after the window renders (no double badge with the full-size camera JPEG). - The loupe look-ahead key includes the crop. - bench-cull CLI tests use binary() after upstream's test split. Co-Authored-By: Claude Opus 5.5 * PERF-CULL: CodeRabbit round 11 (browse listing errors and library root, rating ties, every CR3 PRVW, checked CR3 box math, look-ahead view key, windows wait while settling) - browse_files: a folder that can't be listed is an error (the scan ends as failed, nothing is forgotten as gone), and the library's folder or one inside it lists nothing. - New photos tie on rating: queued by name in the view's direction. - Every valid CR3 PRVW/THMB is a candidate (the smallest covering one is picked); the box offsets use checked arithmetic. - The loupe look-ahead is keyed to the visible list's generation (a new sort, filter or source changes the neighbours). - While stepping fast, zoomed-in window renders wait like the frame, and a window drawn before the frame is timed as the render. Co-Authored-By: Claude Opus 5.5 * PERF-CULL: CodeRabbit round 12 (loupe timing: the After window, over any stand-in) In a split view only the After side's window says the raw's render is on screen; and a window drawn over the camera JPEG counts as the render too (the source shown is a render only when it is "render" or "cached"). Co-Authored-By: Claude Opus 5.5 * PERF-CULL: CodeRabbit round 13 (loupe timing: the Before window when the Before is shown alone) Co-Authored-By: Claude Opus 5.5 * PERF-CULL: CodeRabbit round 14 (loupe timing: a Before window side by side with \ held is not the shown view) Co-Authored-By: Claude Opus 5.5 * PERF-CULL: CodeRabbit round 15 (the split-view timing test asserts its precondition) Co-Authored-By: Claude Opus 5.5 * PERF-CULL: CodeRabbit round 16 (side by side with \ held, each window stays in its pane) Co-Authored-By: Claude Opus 5.5 * PERF-CULL: a cancelled browse stops ordering its files; browsing hands back earlier probes Co-Authored-By: Claude Opus 5.5 * PERF-CULL: the web build runs the browse job in place as FnMut (it hands the probes back) Co-Authored-By: Claude Opus 5.5 * PERF-CULL: a scan hands the probes it took back however it ends (CodeRabbit, #3077 / #3080) - The browse worker sends a result on every path: the listing, or `Stopped` (the folder couldn't be listed, or another folder was chosen) with the probes `ScanInput::new` took from the session. A listing failure is never sent as an empty listing. - A scan stopped without waiting (its cross, or a browse replaced by another) keeps its channel until the worker ends, so the probes still come back; a finished scan's probes join the session's instead of replacing them. - A cancelled scan whose worker ended without a result is not reported as "Scan failed". Co-Authored-By: Claude Opus 5.5 * PERF-PREVIEW-PREP: built previews are the ones on disk; stand-ins ahead follow the preparation settings (CodeRabbit, #3077) - A build counts a photo as built only when its previews are in the disk cache (when it has one): a loupe render kept only in memory, its deferred write skipped, is built again and written (`stored_dims`). - `DiskCache::put` and `PreviewCache::put_at` say whether the file was written; a view preview the cache folder can't take fails the photo instead of counting it as built. - The loupe's stand-ins ahead honour Settings > Performance for the mode: none for "Only the photo on screen" or no photos ahead (the queued ones are dropped), only marked photos in the marked scope. The settings are part of what the look-ahead was made for. Co-Authored-By: Claude Opus 5.5 * PERF-PREVIEW-PREP: a preview for a discarded cache fails the photo; cache file sizes are read from 4 KiB first (CodeRabbit, #3077 / #3080) - `PreviewCache::put_at` returns false for a job made before the cache was cleared or retired (nothing is kept), so a build that overlaps Discard Preview Cache counts that photo as failed, not built. The warning says which of the two happened. - `DiskCache::dims` reads a file's first 4 KiB, and up to 64 KiB only when the size marker isn't there. Co-Authored-By: Claude Opus 5.5 * PERF-PREVIEW-PREP: a photo another build renders counts once it is built; a write pruned away is not kept (CodeRabbit, #3077 / #3080) - A build that meets a photo another build is rendering waits for that render, then counts the photo as built only when its previews are there, else renders it itself (it was counted as shared at once). - `DiskCache::put` returns false when the pruning its own write starts removes that file (a picture bigger than the room pruning leaves). Co-Authored-By: Claude Opus 5.5 * PERF-CULL: put_memory holds the cache generation through its insertion (CodeRabbit, #3080) A clear can no longer finish between put_memory's look at the generation and its insertion. Co-Authored-By: Claude Opus 5.5 * PERF-CULL: browsing skips the library's folder by any name it has (a symlink, a path through ..) (CodeRabbit, #3077) Co-Authored-By: Claude Opus 5.5 * PERF-CULL: rustfmt the library-folder check Co-Authored-By: Claude Opus 5.5 * PERF-PREVIEW-PREP: cancelled builds stop before their test library is removed (CodeRabbit, #3077) Co-Authored-By: Claude Opus 5.5 * PERF-CULL: the library folder is skipped by its resolved path only; lib/../outside is browsed, an unresolvable folder reports its error (CodeRabbit, #3077) Co-Authored-By: Claude Opus 5.5 * PERF-PREVIEW-PREP: the replaced background build is held by TestHold, not a 250 ms deadline (CodeRabbit, #3077) Co-Authored-By: Claude Opus 5.5 * PERF-CULL: a browsed folder has one spelling (. and .. taken out by name) for the view, its listing and the catalog's records; an unresolvable root inside the library is still skipped (CodeRabbit, #3055 / #3077 / #3080) Co-Authored-By: Claude Opus 5.5 * PERF-PREVIEW-PREP: TestHold counts its holders: one test's release doesn't free a build another test holds (CodeRabbit, #3077 / #3080) Co-Authored-By: Claude Opus 5.5 * PERF-PREVIEW-PREP: TestHold's release without the deprecated fetch_update Co-Authored-By: Claude Opus 5.5 * PERF-CULL: browsing resolves `..` the way the system does, and `.` is the current folder `lexical_path` took `..` out by name, which is wrong on POSIX when the folder before it is a symlink (`link/..` is the folder above where the link points). `browse_path` makes the root absolute first (so `.` and `a/..` name the current folder, not an empty path), resolves a link just before a `..` on POSIX, and keeps every other spelling as written (a mapped drive stays a drive letter, matching the catalog's records). Windows takes `..` out by name, as before. Tests: a folder link followed by `..` lists what the system lists there; `.` is the current folder. Co-Authored-By: Claude Opus 5.5 * PERF-CULL: browsing resolves the folder on its worker, never on the window's thread `browse()` called `browse_path` (which may canonicalize a link) and `is_dir()` before starting the worker, so a link into a stalled network mount could freeze the window. The UI now spells the folder with `browse_spelling` (`.`/`..` by name, no disk access) and shows it as given; the worker resolves it with `browse_path`, checks it is a folder (else the scan ends as failed, with the usual toast), and sends the resolved spelling, which the view takes before the listing arrives. Tests: a browse whose resolution is held returns at once and waits, then reads the folder; a missing folder fails on the worker; `browse_spelling` matches `browse_path` without links. Co-Authored-By: Claude Opus 5.5 * PERF-CULL: the browse worker resolves the path as given, not the view's spelling (CodeRabbit, #3055 / #3077 / #3080) On Unix `photos/link/../select` with `link` pointing at `real/day` is `real/select`; the spelling with `..` taken out by name read `photos/select`. The view keeps the spelling until the worker answers. Co-Authored-By: Claude Opus 5.5 * PERF-PREVIEW-PREP: bench-cull measures small folders and never writes its results among the photos (CodeRabbit, #3077) A folder with no more photos than the grid sample was reported with empty camera-preview and loupe timings: the samples now fall back to the photos it has. A --json path in the source folder is refused before the run. Co-Authored-By: Claude Opus 5.5 * PERF-PREVIEW-PREP: editing renders the loupe from its source; the loupe holds background work back only while its render is on the way (CodeRabbit, #3077) In Develop a cached view render left the source undecoded until the first slider draft. foreground() was renewed every loupe frame even with the render shown, so background builds mostly waited. Co-Authored-By: Claude Opus 5.5 * PERF-PREVIEW-PREP: loupeSteps.total, a step cursor that survives the 256-step list (CodeRabbit, #3077) tools/cull-bench.py sliced the list by its length, which stops growing at 256 steps, and reported no steps on repeated runs. Also documents perf.thumbsMade and perf.previewCache. Co-Authored-By: Claude Opus 5.5 * PERF-PREVIEW-PREP: the camera preview is located once when the fast decode declines (CodeRabbit, #3077) The general decode takes the stand-in JPEG the fast path already found instead of parsing the container again. Co-Authored-By: Claude Opus 5.5 * PERF-PREVIEW-PREP: the uncapped preview cache warns once per session when its drive runs low The automatic preview cache budget grows with the library and has no ceiling. When the library opens, and at most every five minutes while previews are built, the engine reads the free space of the drive holding the cache (fs4, MIT OR Apache-2.0, pure Rust over rustix / windows-sys); under 10 % or under 20 GB free it raises one alert per session. The desktop app shows it as a notice with the drive's free space and the cache's size and a button to Settings > Performance, where a size limit can be set. A drive that can't say its free space gives no alert (logged). library.info reports lowDiskSpace; the notice is in ui.inspect's notices like the others. Co-Authored-By: Claude Opus 5.5 * PERF-PREVIEW-PREP: the free-space read runs off the UI thread; a queued low-disk notice goes with its library; --work outside the source folder (CodeRabbit, #3077) - Session::check_disk_space reads the drive on a worker thread and takes the answer on a later call (take_disk_space_alert, library.info, the next check): a slow network share no longer holds up a library open or a UI frame. Test: a_slow_drive_does_not_hold_up_the_check. - The UI keeps the low-disk notice with the library generation it was raised for and drops it when another library opens. Test: a_queued_notice_goes_when_another_library_opens. - The real-drive test runs on native targets only (read is None on wasm32). - bench-cull refuses a --work folder inside the measured folder, also one not made yet, before anything is created. Test: the_work_folder_never_goes_in_the_source_folder. Co-Authored-By: Claude Opus 5.5 * PERF-PREVIEW-PREP: one free-space read at a time across library switches, the cache counted on its worker; a --json link into the source folder is refused (CodeRabbit, #3077/#3080) - A library opened while the drive's free-space read still runs keeps that read (marked stale) instead of starting a second worker beside it; its answer is dropped and the current library's drive is read once it has come. Switching libraries while a share hangs no longer piles up workers. Test: switching_libraries_while_a_drive_hangs_starts_no_second_read. - When the drive is low, the preview cache is counted on the read's worker and the count travels with the answer: taking it on the UI thread no longer scans a big cache on a slow share. DiskCache::known_size reads the count without touching the disk. Test: the_preview_cache_is_counted_off_the_ui_thread. - bench-cull: a --json path that exists is resolved itself (a link to a photo in the source folder is that photo), a link that points nowhere yet by the file writing through it would create, a loop of links is refused. Test: a_json_link_into_the_source_folder_is_refused. Co-Authored-By: Claude Opus 5.5 * PERF-PREVIEW-PREP: bench-cull --json replaces its file instead of writing into it, so a hard link to a photo can't overwrite the photo (CodeRabbit, #3077) The results go to a new file next to the destination, renamed over it; the temporary file is removed on error. Test: a_json_hard_link_to_a_photo_leaves_the_photo_alone (fails with the old write). Co-Authored-By: Claude Opus 5.5 * PERF-PREVIEW-PREP: bench-cull refuses a --json entry in the source folder, also a link there to a file outside (CodeRabbit, #3080) Writing the results replaces the --json entry itself, so a link sitting in the source folder that points outside it would be replaced, removing that entry. check_outputs now checks where the entry sits (its folder resolved, the final link not followed) as well as where it resolves to. Co-Authored-By: Claude Opus 5.5 * PERF-CULL: a frame without the loupe ends its step timing, so time away from Detail never counts (CodeRabbit, #3055) LoupeTiming::left was never called: a photo's record stayed open while the grid, Compare, Survey, Reference, People or the slideshow was up, so dwell / any / full / sharp ms counted time outside Detail. observe() marks the frame; finish_ui_pass ends the step when the loupe drew nothing. Co-Authored-By: Claude Opus 5.5 * PERF-PREVIEW-PREP: Build Standard-Sized Previews keeps 2560 px before a loupe was drawn (CodeRabbit, #3077) On Automatic with no loupe size known yet (a fresh session) the command left `edge` out, so the engine built its 2048 px minimum and a typical fit view rendered again. The loupe-aware engine size still applies once a loupe was drawn; a chosen limit is used as is. Co-Authored-By: Claude Opus 5.5 * PERF-CULL: main's merge keeps one Russian "★ Favorites" and its browse test uses the scan's end channel Both sides had added the same key; main's new Auto Import browse test sends through the scan channel, which this branch carries as `ScanEnd`. Co-Authored-By: Claude Opus 5.5 * PERF-CULL: stepping fast drops the queued Before render too; the thumbnail count is per photo across eviction (CodeRabbit, #3055) While an arrow is held the Before view's raw render waits like the loupe's own, and one still queued for a photo left behind is dropped. `thumbsMade` counts each photo once per library, so a thumbnail evicted and made again, or a library left and opened again, doesn't count twice. Co-Authored-By: Claude Opus 5.5 * PERF-PREVIEW-PREP: the free space is read every few minutes, not only while previews are built (CodeRabbit, #3080) Thumbnails and the loupe's previews fill the cache too, so a drive could run low after the library opened without the notice: the low-disk check now runs whatever fills the cache (still at most every five minutes). The engine gains two hidden test hooks (a reader and forgetting the last read). Co-Authored-By: Claude Opus 5.5 * PERF-PREVIEW-PREP: a slow free-space read shows in the activity stack and an idle window wakes for the disk check (CodeRabbit, #3077) Co-Authored-By: Claude Opus 5.5 * PERF-PREVIEW-PREP: background preparation advances on every logic pass, hidden ones too (CodeRabbit, #3077) Co-Authored-By: Claude Opus 5.5 * PERF-CULL: a photo the loupe opens on is never a fast step; only a step inside the loupe has a predecessor (CodeRabbit, #3077) Co-Authored-By: Claude Opus 5.5 * PERF-CULL: the cull benchmark rates one named photo and puts its rating back (CodeRabbit, #3077) Co-Authored-By: Claude Opus 5.5 * PERF-CULL: the cull benchmark refuses a full step history, says when thumbnails were in memory, fails on a lost key and ends a zoom that needs no region (CodeRabbit, #3055) Co-Authored-By: Claude Opus 5.5 * PERF-PREVIEW-PREP: a forced build renders a photo another build was rendering itself (CodeRabbit, #3080) Co-Authored-By: Claude Opus 5.5 * PERF-CULL: the cull benchmark stops on a folder with no photos and times a zoom by the region that ended it (CodeRabbit, #3055/#3080) - browse: a folder that stays loaded with no supported photo ends the browse loop after 3 s instead of waiting 30 minutes, and the run stops with a clear message before the loupe section instead of failing on an empty selection; - zoom to 100 %: the time reported is the moment the loop ended on (the region's arrival, or the settled whole frame when no region is asked for), so a whole frame that settled before a region was requested no longer gives a too-short zoom time. Co-Authored-By: Claude Opus 5.5 * PERF-CULL: the cull benchmark says no photos are visible, keeps an earlier result, and records an unfinished zoom as missing (CodeRabbit, #3055/#3077/#3080) - an empty selection after browsing means no photo is visible (a filter can hide them), not that the folder has none: say so and point at the filters; - LABEL.json is no longer overwritten: an existing one stops the run before it starts unless --overwrite is given (the final write opens with "x" then); - a zoom that did not finish within 60 s is written as null, not as a 60 s zoom time. Co-Authored-By: Claude Opus 5.5 * PERF-CULL: a raw's preview pointer is read with checked bounds on its offset and size (FredApps, #3080) The JPEGInterchangeFormat pointer and the undefined-tag JPEGs were sliced with `as usize` casts of the file's own u64 numbers: safe on 64-bit (the slice is `get`, its end cut at the file's end), but an offset past 4 GiB would wrap to a small, wrong one on a 32-bit build (wasm32). `file_span` now converts with `try_from` and saturates the end. Test: `a_preview_pointer_past_the_end_of_the_file_finds_no_preview` (pointers past the end, lengths past it, u32::MAX for both; a real strip preview beside the lying pointer is still found; every truncation of the file is safe). Co-Authored-By: Claude Opus 5.5 * PERF-CULL: bench-cull refuses a --work or --json inside the photo folder (FredApps, #3055) Nothing is ever written next to the photos: a --work folder or a --json file inside FOLDER (the folder itself, a subfolder, or a path that goes there through `..`) is refused before anything is made. Paths are compared resolved (links followed as far as they exist). The cleanup already removes only what the run made (its own `bench-cull-run-…` folder's `lib-copy`), never --work. Test: `work_and_json_inside_the_photo_folder_are_refused`. Co-Authored-By: Claude Opus 5.5 * PERF-PREVIEW-PREP: a test that building previews writes nothing next to the photos (FredApps, #3077) `building_previews_writes_nothing_next_to_the_photos`: Build Previews and the background preparation while culling leave the shoot folder with the same files, byte for byte; previews go only to the library's own cache. Co-Authored-By: Claude Opus 5.5 * PERF-CULL: cull-bench.py prints its usage on missing arguments and refuses a dangling LABEL.json link up front (CodeRabbit, #3055/#3077) - PORT / FOLDER missing (or PORT not a number), or an option with no value or an unreadable one, stop with the usage line instead of a traceback; - the early "LABEL.json exists" check uses `os.path.lexists`, so a dangling link there is refused before the run, not at the final exclusive write. Co-Authored-By: Claude Opus 5.5 * PERF-CULL: a preview cut short by the end of the file is not taken (CodeRabbit, #3077) `file_span` keeps a span the file's end cuts short only while it still holds a JPEG end marker (an overstated length with the whole JPEG inside). A truncated JPEG passed the frame-header check, won as the largest preview and then failed to decode, so no preview was shown. Test: `a_preview_cut_short_by_the_end_of_the_file_is_not_taken`. Co-Authored-By: Claude Opus 5.5 * PERF-PREVIEW-PREP: the shoot-folder test waits for the background build to finish (CodeRabbit, #3077) `building_previews_writes_nothing_next_to_the_photos` used a 5 s capped poll and then went on; it now asserts `wait_finished` (30 s), so a late write by a slow build can't slip past the check. Co-Authored-By: Claude Opus 5.5 * PERF-CULL: a cut-short preview must reach its own JPEG end marker; cull-bench refuses bad options up front (CodeRabbit, #3055/#3077/#3080) - raw: file_span keeps a span the file's end cuts short only when walking the JPEG's marker segments reaches its end marker inside it, and cuts it there. An FF D9 inside an APP payload no longer counts (it is skipped by the segment length), and bytes that don't start with FF D8 are refused without scanning the rest of the file. Tests: the cut-short test gains an APP1 payload holding FF D9 (taken before this fix) and a whole JPEG followed by other bytes; the_end_of_a_jpeg_is_its_own_end_marker covers stuffed bytes, restart markers, bad segment lengths and every truncation. - tools/cull-bench.py: an unknown option, an option where a value belongs (--label --overwrite), --steps/--hold-ms below 1, or a port outside 1-65535 stop with the usage line before the run. Co-Authored-By: Claude Opus 5.5 * PERF-CULL: cull-bench refuses a run whose steps wouldn't fit the app's loupe history (CodeRabbit, #3055) The app keeps its last 256 loupe steps. A --steps that can't fit a run (1 + 11 + STEPS + 1) stops before the run, and each phase checks the room left before it starts, so a hold whose first steps were dropped is never reported as the whole hold. Co-Authored-By: Claude Opus 5.5 * PERF-CULL: bench-cull refuses ".." in --json and --work (CodeRabbit, #3544) A ".." after a folder the run hasn't made yet can't be resolved by the source-folder check, and the run then makes that folder: --json NEW/../FOLDER/r.json would land in the source folder. Test: a_dot_dot_in_the_outputs_is_refused. Co-Authored-By: Claude Opus 5.5 * PERF-CULL: a cut-short preview needs a scan before its end and the walks share a budget; cull-bench catches dropped steps and odd arguments (CodeRabbit, #3055/#3077/#3080) - raw: jpeg_end accepts an end marker only after a start of scan (an end with no image data before it is refused), and the walks of a file's pointers past its end share scan_budget, as scan_for_jpeg's do. Tests: a whole-looking JPEG with no scan is not taken; the_walks_to_a_jpegs_end_share_a_budget. - tools/cull-bench.py: the port must be a decimal number int() takes (isdecimal, guarded); a FOLDER that looks like an option stops with usage; --steps goes up to 254 (only the hold's STEPS + 1 must fit, with one to spare); each phase starts with room to spare, so a full history means other steps came in and the phase stops instead of reporting a shifted window. Co-Authored-By: Claude Opus 5.5 * PERF-CULL: the camera JPEG goes only under the window render, and a stand-in decoded across a cache clear isn't kept (CodeRabbit, #3544) - ui-egui: zoomed in, the full-size camera JPEG is painted only where the window render will go (window_cover), so a window capped smaller than the view leaves the developed whole-frame render in the strips beside it instead of an unlabelled camera JPEG. Test: the_camera_jpeg_covers_only_where_the_window_goes. - engine/preview: QuickJob's camera-JPEG cache carries the cache generation it was made in and writes with put_deferred_at / the new put_memory_at, so a job decoding when Discard Preview Cache clears the cache can't put its picture back (as render jobs already do). Test: a_camera_jpeg_decoded_across_a_cache_clear_is_not_kept. Co-Authored-By: Claude Opus 5.5 * PERF-CULL: a bad build request leaves the background build alone, and preparation settings that can't be saved aren't kept (CodeRabbit, #3544) - library.buildPreviews stops a running background build only after its own ids are known to be good, and a build whose thread can't start is marked finished instead of left "running". - prep_tick clears started_for when the background build fails to start, so the view retries. - set_prep_settings / clear_library_prep put the previous settings back when prefs.json can't be written. Tests: a_bad_build_request_leaves_the_background_build_running, a_prep_setting_that_cant_be_saved_is_not_kept. Co-Authored-By: Claude Opus 5.5 * PERF-CULL: a background build that can't start waits 30 s before the view tries again Clearing `started_for` on a failed start made every tick call library.buildPreviews again while the cause lasted (CodeRabbit on #3080). The view now waits RETRY_AFTER first. Test `a_build_that_cant_start_waits_before_trying_again`. Co-Authored-By: Claude Opus 5.5 * PERF-CULL: a build waiting to retry asks for a frame at its time; a build whose worker can't start leaves the background one running CodeRabbit on #3077/#3080: a background build that couldn't start retried only on the next input. The session now says how long until the retry (prep_retry_in) and the app asks for a frame then. CodeRabbit on #3544: a build asked for whose worker thread couldn't start cancelled the background preparation and left nothing running. The background build is now stopped only once the new one has started; otherwise it stays the session's build. Co-Authored-By: Claude Opus 5.5 * PERF-CULL: a retry time that has passed asks for no more frames CodeRabbit on #3077 / #3080: once the retry deadline passed, prep_retry_in returned zero, so a view that couldn't be tried (grid, preparation off) asked for a frame at once on every pass and kept an idle window drawing. It returns None then. Co-Authored-By: Claude Opus 5.5 * PERF-CULL: CodeRabbit items from #3577 that live in #3077 and #3055 - disk space: a read whose thread can't start (or whose worker ends without answering) is tried again after 10 s, not after the 5-minute check interval; a hung read is polled every 5 s after its first 5 s instead of every 250 ms - the low-disk notice of the library before is not drawn in the frame after a library switch - library.smartPreviews reads its ids before stopping the background build - a forced build renders the grid thumbnail again instead of reading its cache - the loupe's look-ahead asks for its stand-ins again after a renderer reset - browse: a broken link before `..` is an error, not the link's parent folder - the probes of two stopped scans: the later scan's wins whichever ends first - camera JPEG fast path: a JPEG with an incomplete ICC sequence goes the general way; a CR3 without a JPEG track picks its box preview by size too - tests: bench-cull compares the source photos' bytes, the disk notice test waits for the open's read Co-Authored-By: Claude Opus 5.5 * PERF-CULL: CodeRabbit round 4 on #3077 and the #3583 items in its code - A background smart-preview request cancels the running build only once its worker started; a worker that can't start leaves the earlier build running and the session's (test hook). - The notice logic looks again at the next disk check (10 s after a read that couldn't start). - CR3 boxes: largest by pixels; every top-level mdat left out; whole-file fallback takes the first box of a kind; a copy without an end marker isn't chosen to cover a size; a test decodes the chosen copy. - The fitted JPEG decode reads a SEF file as the image before its trailer; its doc says a header that can't be read is Ok(None). - Settings: a saved cache size the list doesn't offer shows as its own option. - bench-cull rejects a second FOLDER; a browse through `..` reads again. - cull-bench.py: --size checked, baseline waits for renders, --skip-browse claims no folder, a short rest phase warns. Co-Authored-By: Claude Opus 5.5 * Format the quick-full draw call * Loupe stand-in choice moves to its own function * Render results: the newer-draft and display checks move to their own functions * Render results: a frame left on the GPU is shown by its own function * Format the frame's texture entry * Docs: one row each for the import and performance entries * Format the quick view's key --------- Co-authored-by: Claude Opus 5.5 Co-authored-by: FredApps <153173854+FredApps@users.noreply.github.com> --- Cargo.lock | 11 + Cargo.toml | 2 + apps/lightcraft-cli/src/bench_cull.rs | 585 ++++++++++++++++ apps/lightcraft-cli/src/main.rs | 9 + apps/lightcraft-cli/tests/it/cli.rs | 69 ++ apps/lightcraft/src/main.rs | 1 + crates/codecs/src/jpeg.rs | 207 ++++++ crates/codecs/src/lib.rs | 15 + crates/codecs/tests/it/header.rs | 4 + crates/engine/Cargo.toml | 2 + crates/engine/src/cmd/library/specs.rs | 4 + crates/engine/src/cmd/previews.rs | 408 +++++++++-- crates/engine/src/disk_space.rs | 577 ++++++++++++++++ crates/engine/src/files/fs.rs | 6 + crates/engine/src/files/preview.rs | 23 +- crates/engine/src/import.rs | 4 +- crates/engine/src/import/job.rs | 2 + crates/engine/src/import/scan.rs | 181 ++++- crates/engine/src/lib.rs | 13 + crates/engine/src/library.rs | 45 +- crates/engine/src/media.rs | 9 +- crates/engine/src/media/cache.rs | 8 +- crates/engine/src/media/quick.rs | 43 +- crates/engine/src/media/render.rs | 107 ++- crates/engine/src/media/session.rs | 66 +- crates/engine/src/media/source.rs | 15 +- .../engine/src/media/thumbnail_hash_tests.rs | 101 ++- crates/engine/src/prep.rs | 532 +++++++++++++++ crates/engine/src/tests.rs | 1 + crates/engine/src/tests/import.rs | 209 ++++++ crates/engine/src/tests/lens_refresh.rs | 2 +- crates/engine/src/tests/preview_prep.rs | 635 ++++++++++++++++++ crates/preview/src/disk.rs | 63 +- crates/preview/src/lib.rs | 74 +- crates/preview/src/tests.rs | 60 ++ crates/raw/src/preview.rs | 220 +++++- crates/raw/src/preview/tests.rs | 350 ++++++++++ crates/ui-egui/locales/cs.json | 1 + crates/ui-egui/locales/de.json | 3 + crates/ui-egui/locales/es.json | 3 + crates/ui-egui/locales/fi.json | 1 + crates/ui-egui/locales/fr.json | 3 + crates/ui-egui/locales/id.json | 1 + crates/ui-egui/locales/it.json | 1 + crates/ui-egui/locales/ja.json | 3 + crates/ui-egui/locales/ka.json | 1 + crates/ui-egui/locales/pt-br.json | 3 + crates/ui-egui/locales/ru.json | 3 + crates/ui-egui/locales/sv.json | 1 + crates/ui-egui/locales/uk.json | 3 + crates/ui-egui/locales/vi.json | 1 + crates/ui-egui/locales/zh-hans.json | 3 + crates/ui-egui/locales/zh-hant.json | 3 + crates/ui-egui/src/app.rs | 44 ++ crates/ui-egui/src/app/frame.rs | 5 +- crates/ui-egui/src/control.rs | 19 +- crates/ui-egui/src/dialogs/import.rs | 5 +- crates/ui-egui/src/dialogs/import/scan.rs | 249 +++++-- crates/ui-egui/src/dialogs/settings.rs | 4 +- .../src/dialogs/settings/performance.rs | 131 +++- crates/ui-egui/src/headless/tests.rs | 140 ++++ crates/ui-egui/src/lib.rs | 23 + crates/ui-egui/src/loupe_timing.rs | 269 ++++++++ crates/ui-egui/src/menus/menubar.rs | 20 +- crates/ui-egui/src/panels/detail.rs | 305 ++++++++- crates/ui-egui/src/panels/detail/info.rs | 18 + .../ui-egui/src/panels/detail/look_ahead.rs | 119 ++++ crates/ui-egui/src/panels/detail/window.rs | 7 +- crates/ui-egui/src/panels/notices.rs | 128 +++- crates/ui-egui/src/render.rs | 337 ++++++++-- crates/ui-egui/src/render/thumbnail_tests.rs | 404 +++++++++++ crates/ui-egui/src/state/settings.rs | 16 + crates/ui-egui/src/tests.rs | 2 + crates/ui-egui/src/tests/disk_space.rs | 151 +++++ crates/ui-egui/src/tests/drop_import.rs | 5 +- crates/ui-egui/src/tests/lifecycle_037.rs | 40 ++ crates/ui-egui/src/tests/preview_limit.rs | 72 ++ crates/ui-egui/src/tests/scan_probes.rs | 204 ++++++ docs/control-protocol.md | 2 +- docs/gaps.md | 6 +- docs/gpu-pipeline.md | 5 + docs/parity-checklist.md | 22 +- docs/raw-preview-reads.md | 18 + tools/cull-bench.py | 336 +++++++++ 84 files changed, 7508 insertions(+), 295 deletions(-) create mode 100644 apps/lightcraft-cli/src/bench_cull.rs create mode 100644 crates/engine/src/disk_space.rs create mode 100644 crates/engine/src/prep.rs create mode 100644 crates/engine/src/tests/preview_prep.rs create mode 100644 crates/ui-egui/src/loupe_timing.rs create mode 100644 crates/ui-egui/src/panels/detail/look_ahead.rs create mode 100644 crates/ui-egui/src/tests/disk_space.rs create mode 100644 crates/ui-egui/src/tests/scan_probes.rs create mode 100644 tools/cull-bench.py diff --git a/Cargo.lock b/Cargo.lock index 1d0bb2bdc2..1b870d1c39 100644 --- a/Cargo.lock +++ b/Cargo.lock @@ -2139,6 +2139,16 @@ version = "0.3.1" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "aa9a19cbb55df58761df49b23516a86d432839add4af60fc256da840f66ed35b" +[[package]] +name = "fs4" +version = "1.1.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "7e72ed92b67c146290f88e9c89d60ca163ea417a446f61ffd7b72df3e7f1dfd5" +dependencies = [ + "rustix 1.1.5", + "windows-sys 0.61.2", +] + [[package]] name = "futures-core" version = "0.3.34" @@ -3732,6 +3742,7 @@ dependencies = [ "ab_glyph", "chrono", "craft-docpath", + "fs4", "harfrust 0.13.3", "jxl-oxide", "libc", diff --git a/Cargo.toml b/Cargo.toml index 061507c999..25b742c1b7 100644 --- a/Cargo.toml +++ b/Cargo.toml @@ -58,6 +58,8 @@ log = "0.4" # All date/time conversion goes through chrono (no hand-rolled calendar code beyond the 400-year range extension; AGENTS.md). chrono = { version = "0.4.45", default-features = false, features = ["std"] } rayon = "1" +# free/total space of a drive (engine's low-disk alert); MIT OR Apache-2.0, pure Rust (rustix / windows-sys) +fs4 = { version = "1.1", default-features = false } winapi-util = "0.1" # lightcraft-catalog reads a file's volume serial number and file index with it on Windows proptest = "1" bytemuck = { version = "1", features = ["derive"] } diff --git a/apps/lightcraft-cli/src/bench_cull.rs b/apps/lightcraft-cli/src/bench_cull.rs new file mode 100644 index 0000000000..9107a94553 --- /dev/null +++ b/apps/lightcraft-cli/src/bench_cull.rs @@ -0,0 +1,585 @@ +//! `lightcraft-cli bench-cull FOLDER`: time what culling a shoot costs, headless, on real files. +//! +//! What a wedding photographer does most is open a folder of a few thousand raws, look at each +//! one, rate it and move on. This times the engine side of each step on the files themselves (the +//! desktop app's own frame timings come from `ui.inspect` → `perf.loupeSteps`, see +//! `tools/cull-bench.py`): +//! +//! - browse: reading the folder into the Local view (`library.browse`, the whole folder); +//! - grid thumbnails: the camera JPEG at 512 px for the first screen, on as many threads as the +//! app renders with; +//! - camera preview at loupe sizes (1280, 2560) and at full size (the 100 % view), one at a time; +//! - LightCraft's render of the loupe (2560 px) from a cold source; +//! - rating: `photo.rate` on many photos in a row, journal included; +//! - import: adding N files in place to a fresh library, and copying M of them into it (`--n 0`: +//! no import). +//! +//! `--files K` reads only the first K files (by name) of the folder, as browsing reads them: a +//! sample of a folder on a network share, where reading thousands of raws takes hours. +//! +//! Each number is printed as `name: X ms wall` (median, with p95 and n), the format +//! `cargo xtask bench` parses, and the lot as one JSON object with `--json FILE`. Nothing is +//! written next to the photos: every library is made in a new folder of the run's own under +//! `--work` (default: a folder in the system temp directory; nothing already there is deleted), +//! XMP auto-write stays off, and the source folder is only read. + +use std::path::{Path, PathBuf}; +use std::sync::Arc; +use std::sync::atomic::{AtomicUsize, Ordering}; +use std::time::Instant; + +use lightcraft_engine::Session; +use lightcraft_engine::catalog::PhotoId; +use serde_json::{Value, json}; + +const USAGE: &str = "lightcraft-cli bench-cull FOLDER [--files K] [--n N] [--copy M] [--work DIR] [--json FILE]"; + +/// Threads the app renders thumbnails with (`Renderer::default`: cores − 1, at most 6). +fn render_threads() -> usize { + std::thread::available_parallelism().map(|n| n.get().saturating_sub(1)).unwrap_or(2).clamp(2, 6) +} + +/// Median, p95 and count of `ms` (milliseconds). +fn stats(ms: &[f64]) -> Value { + if ms.is_empty() { + return Value::Null; + } + let mut v = ms.to_vec(); + v.sort_by(f64::total_cmp); + let at = |q: f64| v.get(((v.len() - 1) as f64 * q).round() as usize).copied().unwrap_or(0.0); + json!({"p50": round(at(0.5)), "p95": round(at(0.95)), "max": round(at(1.0)), "n": v.len()}) +} + +fn round(x: f64) -> f64 { + (x * 10.0).round() / 10.0 +} + +fn ms_since(t: Instant) -> f64 { + t.elapsed().as_secs_f64() * 1000.0 +} + +/// Print one result line the way `xtask bench` parses it (`name: X ms wall`). +fn line(name: &str, s: &Value) { + if let Some(p50) = s.get("p50").and_then(Value::as_f64) { + println!("{name}: {p50:.1} ms wall (p95 {} ms, n {})", s["p95"], s["n"]); + } +} + +/// A new, empty folder `name` in this run's own folder; never deletes anything already there. +fn fresh_dir(root: &Path, name: &str) -> Result { + let d = root.join(name); + std::fs::create_dir(&d).map_err(|e| format!("{}: {e}", d.display()))?; + Ok(d) +} + +/// A folder of this run's own under `work`, which must not exist yet: a run never deletes or +/// reuses what is already under `--work`. +fn run_dir(work: &Path) -> Result { + std::fs::create_dir_all(work).map_err(|e| format!("{}: {e}", work.display()))?; + let stamp = std::time::SystemTime::now().duration_since(std::time::UNIX_EPOCH).map_or(0, |d| d.as_nanos()); + static RUNS: AtomicUsize = AtomicUsize::new(0); + let n = RUNS.fetch_add(1, Ordering::Relaxed); + let d = work.join(format!("bench-cull-run-{}-{stamp}-{n}", std::process::id())); + std::fs::create_dir(&d).map_err(|e| format!("{}: {e}", d.display()))?; + Ok(d) +} + +/// `take` items of `v` after the first `skip` (photos the stages before have read, so cold in the +/// file cache), or, when `v` has no more than `skip`, its first `take`: a small folder is measured +/// on photos already read rather than not at all. +fn sample_after(v: &[T], skip: usize, take: usize) -> &[T] { + let from = if v.len() > skip { skip } else { 0 }; + let rest = v.get(from..).unwrap_or_default(); + rest.get(..take.min(rest.len())).unwrap_or_default() +} + +/// Whether `path` names a file in `folder` (the source folder, which a run only reads), checked +/// twice: where its own entry sits (the folder holding it, the final link not followed: writing the +/// results replaces that entry, so a link to a file elsewhere that sits in the source folder, maybe +/// standing for a photo, would be replaced; CodeRabbit on #3080), and where it resolves to: a file +/// that is there by itself (a link to a photo there is the photo), a link that points nowhere yet +/// by where writing through it would create the file, and anything else by its nearest folder that +/// exists (a `--work` folder may not be there yet). +fn inside_source(path: &Path, folder: &Path) -> bool { + let Ok(folder) = std::fs::canonicalize(folder) else { return false }; + entry_inside(path, &folder) || resolves_inside(path, &folder, 0) +} + +/// Whether the folder holding `path`'s own entry (its nearest folder that exists, resolved) is in +/// `folder` (already resolved). The entry itself is not followed. +fn entry_inside(path: &Path, folder: &Path) -> bool { + let mut p = path.parent().filter(|p| !p.as_os_str().is_empty()).unwrap_or(Path::new(".")); + loop { + if let Ok(c) = std::fs::canonicalize(p) { + return c.starts_with(folder); + } + match p.parent() { + Some(up) if !up.as_os_str().is_empty() => p = up, + Some(_) => p = Path::new("."), + None => return false, + } + } +} + +/// [`inside_source`] for a `folder` already resolved; `hops`: links followed so far (a loop of +/// links counts as inside: refused). +fn resolves_inside(path: &Path, folder: &Path, hops: u32) -> bool { + if let Ok(c) = std::fs::canonicalize(path) { + return c.starts_with(folder); + } + if let Ok(target) = std::fs::read_link(path) { + if hops >= 40 { + return true; + } + let target = path.parent().map_or_else(|| target.clone(), |dir| dir.join(&target)); + return resolves_inside(&target, folder, hops + 1); + } + entry_inside(path, folder) +} + +/// Refuse `--json` or `--work` in the folder being measured: nothing is written next to the photos, +/// the results and the run's libraries included. Checked before anything is made. +fn check_outputs(folder: &Path, json: Option<&Path>, work: &Path) -> Result<(), String> { + // a `..` after a folder not made yet can't be resolved now, and the run makes that folder: + // `--json NEW/../FOLDER/r.json` would pass the checks below and land in the source folder + // (CodeRabbit on #3544) + for (name, p) in [("--json", json), ("--work", Some(work))] { + if let Some(p) = p.filter(|p| p.components().any(|c| c == std::path::Component::ParentDir)) { + return Err(format!("{}: {name} must not contain \"..\"; give the folder without it", p.display())); + } + } + if let Some(j) = json.filter(|j| inside_source(j, folder)) { + return Err(format!("{}: --json must not be in the folder being measured", j.display())); + } + if inside_source(&work.join("run"), folder) { + return Err(format!("{}: --work must not be in the folder being measured", work.display())); + } + Ok(()) +} + +fn library(dir: &Path) -> Result { + let mut s = Session::new().with_fs().with_system_clock(); + s.open_library(dir, false).map_err(|e| format!("{}: {e}", dir.display()))?; + // nothing is ever written next to the photos + s.execute("library.xmpPreferences", &json!({"autoWrite": false})).map_err(|e| e.to_string())?; + Ok(s) +} + +pub fn run(args: &[String]) -> Result<(), String> { + let mut folder = None; + let mut n = 2000usize; + let mut copy = 200usize; + let mut work = None; + let mut json_out = None; + let mut files = None; + let mut i = 0; + while i < args.len() { + let value = |i: usize| args.get(i + 1).cloned().ok_or_else(|| format!("{} needs a value\n{USAGE}", args[i])); + match args[i].as_str() { + "--n" => { + n = value(i)?.parse().map_err(|_| "--n needs a number".to_string())?; + i += 1; + } + "--copy" => { + copy = value(i)?.parse().map_err(|_| "--copy needs a number".to_string())?; + i += 1; + } + "--files" => { + files = Some(value(i)?.parse::().map_err(|_| "--files needs a number".to_string())?); + i += 1; + } + "--work" => { + work = Some(PathBuf::from(value(i)?)); + i += 1; + } + "--json" => { + json_out = Some(PathBuf::from(value(i)?)); + i += 1; + } + "-h" | "--help" => { + println!("{USAGE}"); + return Ok(()); + } + a if a.starts_with("--") => return Err(format!("unknown option {a}\n{USAGE}")), + // (a second folder would replace the first and be timed in its place: CodeRabbit on #3583) + a if folder.is_some() => return Err(format!("{a}: one FOLDER only\n{USAGE}")), + a => folder = Some(PathBuf::from(a)), + } + i += 1; + } + let folder = folder.ok_or_else(|| USAGE.to_string())?; + if !folder.is_dir() { + return Err(format!("{}: not a folder", folder.display())); + } + let work = work.unwrap_or_else(|| std::env::temp_dir().join(format!("lightcraft-bench-cull-{}", std::process::id()))); + check_outputs(&folder, json_out.as_deref(), &work)?; + let work = run_dir(&work)?; + let mut out = json!({"folder": folder.display().to_string(), "threads": render_threads()}); + + // --- browse: the whole folder into the Local view ------------------------------------------- + let mut s = library(&fresh_dir(&work, "lib-browse")?)?; + // `--files K`: only the first K files by name, read as browsing reads them (a sample of a slow + // network folder) + let t = Instant::now(); + let (photos, ids): (u64, Vec) = match files { + None => { + let r = s.execute("library.browse", &json!({"path": folder.display().to_string()})).map_err(|e| e.to_string())?; + (r["photos"].as_u64().unwrap_or(0), s.visible_cloned()) + } + Some(k) => { + let mut list: Vec = std::fs::read_dir(&folder) + .map_err(|e| format!("{}: {e}", folder.display()))? + .flatten() + .map(|e| e.path()) + .filter(|p| p.is_file() && lightcraft_engine::import::is_supported(p)) + .map(|p| p.display().to_string()) + .collect(); + list.sort(); + list.truncate(k); + let r = s.execute("library.import", &json!({"paths": list, "mode": "add", "local": true})).map_err(|e| e.to_string())?; + let ids: Vec = r["imported"].as_array().into_iter().flatten().filter_map(Value::as_u64).map(PhotoId).collect(); + (ids.len() as u64, ids) + } + }; + let browse_ms = ms_since(t); + let what = if files.is_some() { "first files" } else { "whole folder" }; + println!("browse {photos} photos ({what}): {browse_ms:.1} ms wall"); + out["browse"] = json!({"ms": round(browse_ms), "photos": photos, "perPhotoMs": round(browse_ms / photos.max(1) as f64)}); + let paths: Vec = ids + .iter() + .filter_map(|id| match &s.catalog.photo(*id)?.source { + lightcraft_engine::catalog::Source::File { path } => Some(path.clone()), + _ => None, + }) + .collect(); + if paths.is_empty() { + return Err(format!("{}: no photos", folder.display())); + } + let loader = s.media.preview_loader.clone().ok_or("no preview loader")?; + + // --- grid thumbnails: the first screen, in parallel ------------------------------------------ + let screen: Vec = paths.iter().take(48).cloned().collect(); + let next = Arc::new(AtomicUsize::new(0)); + let t = Instant::now(); + let per: Vec<(f64, f64)> = std::thread::scope(|sc| { + let workers: Vec<_> = (0..render_threads()) + .map(|_| { + let (next, loader, screen) = (next.clone(), loader.clone(), &screen); + sc.spawn(move || { + let mut done = Vec::new(); + while let Some(p) = screen.get(next.fetch_add(1, Ordering::Relaxed)) { + let a = Instant::now(); + let _ = loader(p, 512); + done.push((ms_since(a), ms_since(t))); + } + done + }) + }) + .collect(); + workers.into_iter().flat_map(|w| w.join().unwrap_or_default()).collect() + }); + let first = per.iter().map(|p| p.1).fold(f64::INFINITY, f64::min); + let all = per.iter().map(|p| p.1).fold(0.0, f64::max); + let each = stats(&per.iter().map(|p| p.0).collect::>()); + println!("grid: first thumbnail: {first:.1} ms wall"); + println!("grid: {} thumbnails: {all:.1} ms wall", per.len()); + line("grid: one 512 px thumbnail", &each); + out["thumbs"] = json!({"firstMs": round(first), "allMs": round(all), "count": per.len(), "each": each}); + + // --- camera previews at loupe sizes, one at a time (what a step costs without look-ahead) ----- + let sample = sample_after(&paths, 48, 20); + for edge in [1280usize, 2560, usize::MAX] { + let ms: Vec = sample + .iter() + .map(|p| { + let a = Instant::now(); + let _ = loader(p, edge); + ms_since(a) + }) + .collect(); + let name = if edge == usize::MAX { "camera preview full size (100 %)".to_string() } else { format!("camera preview {edge} px") }; + let st = stats(&ms); + line(&name, &st); + out[if edge == usize::MAX { "cameraFull".to_string() } else { format!("camera{edge}") }] = st; + } + + // --- LightCraft's loupe render (2560 px) from a cold source -------------------------------- + let ms: Vec = sample_after(&ids, 80, 5) + .iter() + .filter_map(|id| { + let job = s.loupe_job(*id, 2560, 1708, true)?; + let a = Instant::now(); + let r = job.run(); + let t = ms_since(a); + r.rendered.is_ok().then_some(t) + }) + .collect(); + let st = stats(&ms); + line("loupe render 2560 px (cold)", &st); + out["loupeRender"] = st; + + // --- rating: a culling pass of keypresses --------------------------------------------------- + let ms: Vec = ids + .iter() + .take(50) + .enumerate() + .filter_map(|(k, id)| { + // the key acts on the selected photo, as in the loupe + s.execute("library.select", &json!({"ids": [id.0], "active": id.0})).ok()?; + let a = Instant::now(); + s.execute("photo.rate", &json!({"rating": (k % 5) + 1})).ok()?; + Some(ms_since(a)) + }) + .collect(); + let st = stats(&ms); + line("rate one photo (journal written)", &st); + out["rate"] = st; + drop(s); + + // --- import: add in place, then copy ---------------------------------------------------------- + if n == 0 { + return finish(out, json_out); + } + let add: Vec = paths.iter().take(n).cloned().collect(); + let mut s = library(&fresh_dir(&work, "lib-add")?)?; + let t = Instant::now(); + let r = s.execute("library.import", &json!({"paths": add, "mode": "add"})).map_err(|e| e.to_string())?; + let add_ms = ms_since(t); + let added = r["imported"].as_array().map(Vec::len).unwrap_or(0); + println!("import {added} photos in place: {add_ms:.1} ms wall"); + out["importAdd"] = json!({"ms": round(add_ms), "photos": added, "perPhotoMs": round(add_ms / added.max(1) as f64)}); + drop(s); + if copy > 0 { + let some: Vec = paths.iter().take(copy).cloned().collect(); + let lib = fresh_dir(&work, "lib-copy")?; + let mut s = library(&lib)?; + let t = Instant::now(); + let r = s + .execute("library.import", &json!({"paths": some, "mode": "copy", "destination": lib.join("Originals").display().to_string()})) + .map_err(|e| e.to_string())?; + let copy_ms = ms_since(t); + let copied = r["imported"].as_array().map(Vec::len).unwrap_or(0); + println!("import {copied} photos by copying: {copy_ms:.1} ms wall"); + out["importCopy"] = json!({"ms": round(copy_ms), "photos": copied, "perPhotoMs": round(copy_ms / copied.max(1) as f64)}); + drop(s); + let _ = std::fs::remove_dir_all(&lib); + } + finish(out, json_out) +} + +fn finish(out: Value, json_out: Option) -> Result<(), String> { + if let Some(p) = json_out { + write_replacing(&p, serde_json::to_string_pretty(&out).unwrap_or_default().as_bytes())?; + } + println!("{}", serde_json::to_string(&out).unwrap_or_default()); + Ok(()) +} + +/// Write `bytes` to `path` through a new file next to it, renamed over `path`: an existing `path` +/// is replaced as a name, never written into. A hard link there shares its data with another file +/// (maybe a photo in the source folder, which [`inside_source`] can't tell), and writing into it +/// would overwrite that file too (CodeRabbit on #3077). The temporary file is removed on error. +fn write_replacing(path: &Path, bytes: &[u8]) -> Result<(), String> { + use std::io::Write; + let err = |e: std::io::Error| format!("{}: {e}", path.display()); + let dir = path.parent().filter(|d| !d.as_os_str().is_empty()).unwrap_or(Path::new(".")); + let name = path.file_name().ok_or_else(|| format!("{}: not a file name", path.display()))?; + static TEMPS: AtomicUsize = AtomicUsize::new(0); + let n = TEMPS.fetch_add(1, Ordering::Relaxed); + let mut tmp_name = std::ffi::OsString::from("."); + tmp_name.push(name); + tmp_name.push(format!(".{}-{n}.tmp", std::process::id())); + let tmp = dir.join(tmp_name); + let written = + std::fs::OpenOptions::new().write(true).create_new(true).open(&tmp).and_then(|mut f| f.write_all(bytes).and_then(|()| f.sync_all())); + if let Err(e) = written.and_then(|()| std::fs::rename(&tmp, path)) { + let _ = std::fs::remove_file(&tmp); + return Err(err(e)); + } + Ok(()) +} + +#[cfg(test)] +mod tests { + use super::*; + + #[test] + fn a_second_folder_is_an_error() { + let e = run(&["a".to_string(), "b".to_string()]).unwrap_err(); + assert!(e.starts_with("b: one FOLDER only"), "{e}"); + } + + /// `--work` may hold anything: a run makes a new folder of its own there and never deletes + /// what is already in it (it used to remove `lib-browse`, `lib-add` and `lib-copy` there). + #[test] + fn a_run_never_deletes_what_is_under_work() { + // a folder this test owns (made new), so the cleanup below removes nothing else + let work = run_dir(&std::env::temp_dir()).unwrap(); + let keep = work.join("lib-browse").join("keep.txt"); + std::fs::create_dir_all(keep.parent().unwrap()).unwrap(); + std::fs::write(&keep, b"mine").unwrap(); + let a = run_dir(&work).unwrap(); + let b = run_dir(&work).unwrap(); + assert_ne!(a, b); + fresh_dir(&a, "lib-browse").unwrap(); + assert!(fresh_dir(&a, "lib-browse").is_err(), "an existing folder is never reused"); + assert_eq!(std::fs::read(&keep).unwrap(), b"mine"); + let _ = std::fs::remove_dir_all(&work); + } + + /// A folder too small for the disjoint samples is measured on the photos it has (CodeRabbit on #3077). + #[test] + fn a_small_folder_is_sampled_from_its_start() { + let v: Vec = (0..100).collect(); + assert_eq!(sample_after(&v, 48, 20), &v[48..68]); + assert_eq!(sample_after(&v, 90, 20), &v[90..100]); + assert_eq!(sample_after(&v[..3], 48, 20), &v[..3]); + assert_eq!(sample_after(&v[..48], 48, 5), &v[..5]); + assert!(sample_after::(&[], 48, 20).is_empty()); + } + + /// A file link at `at` to `target`. + #[cfg(unix)] + fn link(target: &Path, at: &Path) -> std::io::Result<()> { + std::os::unix::fs::symlink(target, at) + } + + /// A file link at `at` to `target` (Windows: needs Developer Mode or the right to make links). + #[cfg(windows)] + fn link(target: &Path, at: &Path) -> std::io::Result<()> { + std::os::windows::fs::symlink_file(target, at) + } + + #[cfg(not(any(unix, windows)))] + fn link(_: &Path, _: &Path) -> std::io::Result<()> { + Err(std::io::Error::other("no file links here")) + } + + /// `--json` naming a link (made elsewhere) to a file in the source folder is refused: writing + /// the results would follow it and overwrite a photo (CodeRabbit on #3080). So is a link that + /// points to a file not made yet in the source folder, and a loop of links. + #[test] + fn a_json_link_into_the_source_folder_is_refused() { + let work = run_dir(&std::env::temp_dir()).unwrap(); + let src = fresh_dir(&work, "src").unwrap(); + let photo = src.join("a.jpg"); + std::fs::write(&photo, b"photo").unwrap(); + let to_photo = work.join("report.json"); + if let Err(e) = link(&photo, &to_photo) { + let _ = std::fs::remove_dir_all(&work); + lightcraft_testkit::skip!(System, "can't make a file link here ({e}); Windows needs Developer Mode"); + return; + } + assert!(inside_source(&to_photo, &src), "a link to a photo in the source folder"); + let e = check_outputs(&src, Some(&to_photo), &work.join("elsewhere")).unwrap_err(); + assert!(e.contains("--json"), "{e}"); + let to_new = work.join("new.json"); + link(&src.join("not-yet.json"), &to_new).unwrap(); + assert!(inside_source(&to_new, &src), "a link to a file not made yet in the source folder"); + let (x, y) = (work.join("x.json"), work.join("y.json")); + link(&y, &x).unwrap(); + link(&x, &y).unwrap(); + assert!(inside_source(&x, &src), "a loop of links"); + let to_elsewhere = work.join("fine.json"); + link(&work.join("out.json"), &to_elsewhere).unwrap(); + assert!(!inside_source(&to_elsewhere, &src), "a link to a file outside is fine"); + assert_eq!(std::fs::read(&photo).unwrap(), b"photo"); + let _ = std::fs::remove_dir_all(&work); + } + + /// `--json` naming a link that sits in the source folder and points to a file outside it is + /// refused: writing the results would replace the link itself, an entry of the source folder + /// (CodeRabbit on #3080). The link stays as it was. + #[test] + fn a_json_link_in_the_source_folder_to_a_file_outside_is_refused() { + let work = run_dir(&std::env::temp_dir()).unwrap(); + let src = fresh_dir(&work, "src").unwrap(); + let outside = work.join("out.json"); + std::fs::write(&outside, b"outside").unwrap(); + let in_src = src.join("a.jpg"); + if let Err(e) = link(&outside, &in_src) { + let _ = std::fs::remove_dir_all(&work); + lightcraft_testkit::skip!(System, "can't make a file link here ({e}); Windows needs Developer Mode"); + return; + } + assert!(inside_source(&in_src, &src), "a link in the source folder, wherever it points"); + let e = check_outputs(&src, Some(&in_src), &work.join("elsewhere")).unwrap_err(); + assert!(e.contains("--json"), "{e}"); + assert!(std::fs::symlink_metadata(&in_src).unwrap().file_type().is_symlink(), "the link stays"); + assert_eq!(std::fs::read_link(&in_src).unwrap(), outside); + assert_eq!(std::fs::read(&outside).unwrap(), b"outside"); + // a new file in the source folder (nothing there yet) is refused too + assert!(check_outputs(&src, Some(&src.join("new.json")), &work.join("elsewhere")).is_err()); + let _ = std::fs::remove_dir_all(&work); + } + + /// `--json` naming a hard link (made elsewhere) to a photo in the source folder passes + /// [`inside_source`] (a hard link has no target to resolve), so the results replace the link + /// instead of writing into it: the photo keeps its bytes (CodeRabbit on #3077). + #[test] + fn a_json_hard_link_to_a_photo_leaves_the_photo_alone() { + let work = run_dir(&std::env::temp_dir()).unwrap(); + let src = fresh_dir(&work, "src").unwrap(); + let photo = src.join("a.jpg"); + std::fs::write(&photo, b"photo").unwrap(); + let report = work.join("report.json"); + if let Err(e) = std::fs::hard_link(&photo, &report) { + let _ = std::fs::remove_dir_all(&work); + lightcraft_testkit::skip!(System, "can't make a hard link here ({e})"); + return; + } + finish(json!({"browse": 1}), Some(report.clone())).unwrap(); + assert_eq!(std::fs::read(&photo).unwrap(), b"photo", "the photo is unchanged"); + let written: Value = serde_json::from_slice(&std::fs::read(&report).unwrap()).unwrap(); + assert_eq!(written["browse"], 1); + let left: Vec<_> = std::fs::read_dir(&work).unwrap().flatten().map(|e| e.file_name()).collect(); + assert_eq!(left.len(), 2, "no temporary file is left: {left:?}"); + // a folder that isn't there: an error, nothing panics + assert!(write_replacing(&work.join("missing").join("r.json"), b"x").is_err()); + let _ = std::fs::remove_dir_all(&work); + } + + /// `--json` naming a file in the source folder is refused (CodeRabbit on #3077); elsewhere it is fine. + #[test] + fn the_results_never_go_in_the_source_folder() { + let work = run_dir(&std::env::temp_dir()).unwrap(); + let src = fresh_dir(&work, "src").unwrap(); + let sub = fresh_dir(&src, "sub").unwrap(); + assert!(inside_source(&src.join("a.png"), &src)); + assert!(inside_source(&sub.join("x.json"), &src)); + assert!(inside_source(&src.join("sub").join("..").join("a.png"), &src)); + assert!(!inside_source(&work.join("out.json"), &src)); + let _ = std::fs::remove_dir_all(&work); + } + + /// `--work` in the source folder is refused before anything is made there, also when it doesn't + /// exist yet (CodeRabbit on #3077); elsewhere it is fine. + #[test] + fn the_work_folder_never_goes_in_the_source_folder() { + let work = run_dir(&std::env::temp_dir()).unwrap(); + let src = fresh_dir(&work, "src").unwrap(); + let inside = src.join("out").join("deeper"); + let e = check_outputs(&src, None, &inside).unwrap_err(); + assert!(e.contains("--work"), "{e}"); + assert!(check_outputs(&src, None, &src).is_err(), "the source folder itself"); + assert!(!src.join("out").exists(), "nothing was made in the source folder"); + assert!(check_outputs(&src, None, &work.join("elsewhere")).is_ok()); + assert!(check_outputs(&src, Some(&src.join("r.json")), &work.join("elsewhere")).is_err()); + let _ = std::fs::remove_dir_all(&work); + } + + /// A `..` in `--json` or `--work` is refused: after a folder the run hasn't made yet it can't be + /// resolved now, and would lead into the source folder once the run makes it (CodeRabbit on #3544). + #[test] + fn a_dot_dot_in_the_outputs_is_refused() { + let work = run_dir(&std::env::temp_dir()).unwrap(); + let src = fresh_dir(&work, "src").unwrap(); + let sneaky = work.join("new").join("..").join("src").join("r.json"); + let e = check_outputs(&src, Some(&sneaky), &work.join("elsewhere")).unwrap_err(); + assert!(e.contains("--json") && e.contains(".."), "{e}"); + let e = check_outputs(&src, None, &work.join("new").join("..").join("src")).unwrap_err(); + assert!(e.contains("--work"), "{e}"); + assert!(!work.join("new").exists(), "nothing was made"); + let _ = std::fs::remove_dir_all(&work); + } +} diff --git a/apps/lightcraft-cli/src/main.rs b/apps/lightcraft-cli/src/main.rs index 7822bbed9a..d384229558 100644 --- a/apps/lightcraft-cli/src/main.rs +++ b/apps/lightcraft-cli/src/main.rs @@ -11,11 +11,13 @@ //! lightcraft-cli commands [--json] //! lightcraft-cli controls [--json] //! lightcraft-cli calibrate [--max N] [--out DIR] FOLDERS/FILES… +//! lightcraft-cli bench-cull FOLDER [--files K] [--n N] [--copy M] [--work DIR] [--json FILE] //! ``` #![forbid(unsafe_code)] #![deny(clippy::unwrap_used, clippy::expect_used, clippy::panic, clippy::unimplemented, clippy::todo, clippy::unreachable)] mod alloc_release; +mod bench_cull; use std::io::Write; use std::path::Path; @@ -110,6 +112,12 @@ USAGE: model, written as .json to DIR (default: the profiles folder LightCraft reads, /camera-profiles, or $LIGHTCRAFT_CAMERA_PROFILES). Raws of a profiled model then take their colour from the profile and only their tone from their own JPEG. + lightcraft-cli bench-cull FOLDER [--files K] [--n N] [--copy M] [--work DIR] [--json FILE] + Time what culling a shoot costs on its own files: reading the folder (browse), the first + screen of grid thumbnails, camera previews at loupe sizes and full size, a loupe render, + rating, and importing N files in place (default 2000) and copying M (default 200). Libraries + are made in a new folder under DIR (default: the temp folder); nothing is written next to the + photos. --files K reads only the first K files by name (a sample of a slow network folder). lightcraft-cli --version | --help "; @@ -176,6 +184,7 @@ fn main() -> ExitCode { Some("controls") => controls(&args[1..]), Some("denoise-serve") => denoise_serve(&args[1..]), Some("calibrate") => calibrate(&args[1..]), + Some("bench-cull") => bench_cull::run(&args[1..]), Some("--version" | "-V" | "version") => { println!("lightcraft-cli {}", env!("CARGO_PKG_VERSION")); Ok(()) diff --git a/apps/lightcraft-cli/tests/it/cli.rs b/apps/lightcraft-cli/tests/it/cli.rs index 613197459c..2a052fc2d8 100644 --- a/apps/lightcraft-cli/tests/it/cli.rs +++ b/apps/lightcraft-cli/tests/it/cli.rs @@ -101,6 +101,75 @@ fn gradient_png(path: &std::path::Path) { std::fs::write(path, png).unwrap(); } +/// PERF-CULL-5: `bench-cull` times a folder end to end and writes nothing next to the photos. +#[test] +fn bench_cull_times_a_folder_and_leaves_it_untouched() { + let dir = tmp("bench-cull-src"); + std::fs::create_dir_all(&dir).unwrap(); + for (seed, name) in ["a.png", "b.png", "c.png"].iter().enumerate() { + // three different pictures (the same bytes would import as one photo) + let img = lightcraft_raster::Rgba8::from_fn(120, 80, |x, y| [(x * 2) as u8, (y * 3) as u8, (seed * 80) as u8, 255]); + let png = lightcraft_codecs::encode_png(&lightcraft_codecs::EncodeImage::rgba8(&img), &Default::default()).unwrap(); + std::fs::write(dir.join(name), png).unwrap(); + } + // every file's name and bytes: nothing added, and no photo written into (CodeRabbit on #3577) + let listing = |d: &std::path::Path| { + let mut v: Vec<(String, Vec)> = std::fs::read_dir(d) + .unwrap() + .map(|e| e.unwrap()) + .map(|e| (e.file_name().to_string_lossy().to_string(), std::fs::read(e.path()).unwrap())) + .collect(); + v.sort(); + v + }; + let before = listing(&dir); + let work = tmp("bench-cull-work"); + let json_out = tmp("bench-cull.json"); + let r = Command::new(binary()) + .arg("bench-cull") + .arg(&dir) + .args(["--n", "3", "--copy", "1", "--work"]) + .arg(&work) + .arg("--json") + .arg(&json_out) + .output() + .unwrap(); + let stdout = String::from_utf8_lossy(&r.stdout); + assert!(r.status.success(), "{stdout}{}", String::from_utf8_lossy(&r.stderr)); + assert!(stdout.contains("browse 3 photos") && stdout.contains("ms wall"), "{stdout}"); + let v: Value = serde_json::from_str(&std::fs::read_to_string(&json_out).unwrap()).unwrap(); + assert_eq!(v["browse"]["photos"], 3); + assert_eq!(v["importAdd"]["photos"], 3); + assert_eq!(v["importCopy"]["photos"], 1); + assert_eq!(v["rate"]["n"], 3); + // a folder smaller than the samples is measured on the photos it has, not reported empty + for stage in ["camera1280", "camera2560", "cameraFull", "loupeRender"] { + assert_eq!(v[stage]["n"], 3, "{stage}: {}", v[stage]); + } + assert_eq!(listing(&dir), before, "nothing written next to the photos"); + // results asked for in the source folder: refused, the photo untouched + let photo = std::fs::read(dir.join("a.png")).unwrap(); + let r = Command::new(binary()) + .arg("bench-cull") + .arg(&dir) + .args(["--n", "0", "--work"]) + .arg(&work) + .arg("--json") + .arg(dir.join("a.png")) + .output() + .unwrap(); + assert!(!r.status.success() && String::from_utf8_lossy(&r.stderr).contains("--json"), "{}", String::from_utf8_lossy(&r.stderr)); + assert_eq!(std::fs::read(dir.join("a.png")).unwrap(), photo); + assert_eq!(listing(&dir), before); + // a sample of the folder, no import + let r = Command::new(binary()).arg("bench-cull").arg(&dir).args(["--files", "2", "--n", "0", "--work"]).arg(&work).output().unwrap(); + let stdout = String::from_utf8_lossy(&r.stdout); + assert!(r.status.success() && stdout.contains("browse 2 photos (first files)") && !stdout.contains("import"), "{stdout}"); + // a bad folder is an error, not a panic + let r = Command::new(binary()).args(["bench-cull", "/no/such/folder"]).output().unwrap(); + assert!(!r.status.success() && String::from_utf8_lossy(&r.stderr).contains("not a folder")); +} + #[test] fn calibrate_reads_raf_inputs_and_reports_bad_files() { let input = tmp("invalid.RAF"); diff --git a/apps/lightcraft/src/main.rs b/apps/lightcraft/src/main.rs index 31ae7229f5..d5dcbc7360 100644 --- a/apps/lightcraft/src/main.rs +++ b/apps/lightcraft/src/main.rs @@ -371,6 +371,7 @@ fn main() -> eframe::Result { app.notices.extend(launch_notices); app.notices.extend(window_notice); app.gpu_notice = true; + app.disk_space_notice = true; // an empty library opens where Import is offered (a grid, with the sidebar); before // the snapshot below, so this alone isn't written to the settings file if problem.is_none() && files.is_empty() { diff --git a/crates/codecs/src/jpeg.rs b/crates/codecs/src/jpeg.rs index 89af1372bf..3e09ae1c4e 100644 --- a/crates/codecs/src/jpeg.rs +++ b/crates/codecs/src/jpeg.rs @@ -862,3 +862,210 @@ mod tests { } } } + +/// [`crate::decode_jpeg_srgb8_fit`]: a plain 8-bit sRGB JPEG decoded (DCT-scaled when that still +/// covers the box) and area-averaged in linear light straight into 8-bit sRGB, without the +/// float image the general path builds (for a 45 MP camera preview fitted to 2560 px that float +/// image and its conversions were half the time). `Ok(None)` for anything else. +pub(crate) fn srgb8_fit(bytes: &[u8], max_w: u32, max_h: u32) -> Result> { + // (a Samsung SEF trailer read as the file it stands for, as the general decode does: CodeRabbit on #3583) + let bytes = &*without_sef_trailer(bytes); + let Some(m) = parse_markers(bytes) else { return Ok(None) }; + let plain = matches!(m.components, 1 | 3) + && m.precision == 8 + && matches!(m.sof, 0xC0..=0xC2) + && m.icc.is_none() + && !m.icc_invalid + && m.adobe_transform.is_none(); + let summary = m.exif.as_deref().map(exif::summarize).unwrap_or_default(); + if !plain || summary.adobe_rgb_hint || max_w == 0 || max_h == 0 { + return Ok(None); + } + let budget = DecodeOptions::default(); + check_size(F, m.width as u64, m.height as u64, &budget)?; + // as in the general decode: the decoders below walk every APP1 again + if m.app1 > MAX_APP1 { + return Err(Error::Malformed(F, "too many APP1 markers".into())); + } + let scale_to = (m.width >= max_w.saturating_mul(2) || m.height >= max_h.saturating_mul(2)).then_some((max_w, max_h)); + let non_interleaved = matches!(m.sof, 0xC0 | 0xC1) && m.first_scan_components < m.components; + let raw = + if scale_to.is_some() || non_interleaved { decode_jpeg_decoder(bytes, &m, scale_to, &budget)? } else { decode_zune(bytes, &m, &budget)? }; + let Buf::U8(px) = &raw.buf else { return Ok(None) }; + let ch = match raw.model { + Model::Rgb => 3, + Model::Gray => 1, + Model::Cmyk => return Ok(None), + }; + let (w, h) = (raw.width, raw.height); + if w == 0 || h == 0 || px.len() < w.saturating_mul(h).saturating_mul(ch) { + return Err(Error::Malformed(F, "short output".into())); + } + // the size `lightcraft_raster::resample::fit` gives: within the box, aspect kept, never larger + let s = (max_w as f64 / w as f64).min(max_h as f64 / h as f64).min(1.0); + let tw = ((w as f64 * s).round() as usize).clamp(1, w); + let th = ((h as f64 * s).round() as usize).clamp(1, h); + let orientation = summary.orientation.filter(|o| (1..=8).contains(o)).unwrap_or(1); + Ok(Some((area_average_srgb8(px, w, h, ch, tw, th), orientation))) +} + +/// Source pixels and weights each output pixel along one axis covers: a box of the scale's width +/// (`n` source pixels onto `out`), partly covered pixels at its ends weighted by their share. +fn box_taps(n: usize, out: usize) -> Vec<(usize, Vec)> { + let scale = n as f64 / out as f64; + (0..out) + .map(|o| { + let (a, b) = (o as f64 * scale, ((o + 1) as f64 * scale).min(n as f64)); + let first = (a.floor() as usize).min(n - 1); + let last = ((b.ceil() as usize).max(first + 1)).min(n); + let ws: Vec = + (first..last).map(|j| ((b.min(j as f64 + 1.0) - a.max(j as f64)).max(0.0) / (b - a).max(f64::MIN_POSITIVE)) as f32).collect(); + (first, ws) + }) + .collect() +} + +/// `px` (`w × h`, `ch` = 1 or 3 interleaved 8-bit sRGB samples) averaged over boxes in linear +/// light into `tw × th` opaque 8-bit sRGB pixels. A copy when the size doesn't change. +fn area_average_srgb8(px: &[u8], w: usize, h: usize, ch: usize, tw: usize, th: usize) -> lightcraft_raster::Rgba8 { + use lightcraft_color::transfer::{decode_srgb8, encode_srgb8}; + let mut out = lightcraft_raster::Rgba8::new(tw, th); + let sample = |x: usize, y: usize, c: usize| px.get((y * w + x) * ch + if ch == 3 { c } else { 0 }).copied().unwrap_or(0); + if (tw, th) == (w, h) { + lightcraft_raster::par_rows(&mut out.data, tw, |y, row| { + for (x, o) in row.iter_mut().enumerate() { + *o = [sample(x, y, 0), sample(x, y, 1), sample(x, y, 2), 255]; + } + }); + return out; + } + let lut: [f32; 256] = std::array::from_fn(|i| decode_srgb8(i as u8)); + let xs = box_taps(w, tw); + let ys = box_taps(h, th); + // one source row filtered across into linear RGB at the output width + let across = |sy: usize, buf: &mut Vec<[f32; 3]>| { + buf.clear(); + let Some(src) = px.get(sy * w * ch..(sy + 1) * w * ch) else { + buf.resize(tw, [0.0; 3]); + return; + }; + for (x0, wx) in &xs { + let mut a = [0f32; 3]; + if let Some(span) = src.get(x0 * ch..(x0 + wx.len()) * ch) { + for (p, &k) in span.chunks_exact(ch).zip(wx) { + match p { + [r, g, b] => { + a[0] += lut[*r as usize] * k; + a[1] += lut[*g as usize] * k; + a[2] += lut[*b as usize] * k; + } + [v] => { + let l = lut[*v as usize] * k; + a = [a[0] + l, a[1] + l, a[2] + l]; + } + _ => {} + } + } + } + buf.push(a); + } + }; + lightcraft_raster::par_rows(&mut out.data, tw, |y, row| { + let Some((y0, wy)) = ys.get(y) else { return }; + let mut acc = vec![[0f32; 3]; tw]; + let mut line = Vec::with_capacity(tw); + for (dy, &k) in wy.iter().enumerate() { + across(y0 + dy, &mut line); + for (a, l) in acc.iter_mut().zip(&line) { + a[0] += l[0] * k; + a[1] += l[1] * k; + a[2] += l[2] * k; + } + } + for (o, a) in row.iter_mut().zip(&acc) { + *o = [encode_srgb8(a[0]), encode_srgb8(a[1]), encode_srgb8(a[2]), 255]; + } + }); + out +} + +#[cfg(test)] +mod srgb8_fit_tests { + use super::*; + + fn jpeg(w: usize, h: usize, f: impl Fn(usize, usize) -> [u8; 3]) -> Vec { + let data: Vec<[u8; 4]> = (0..w * h) + .map(|i| { + let [r, g, b] = f(i % w, i / w); + [r, g, b, 255] + }) + .collect(); + let img = lightcraft_raster::Rgba8 { width: w, height: h, data }; + crate::encode_jpeg(&crate::EncodeImage::rgba8(&img), 95, crate::ChromaSubsampling::S420, &crate::EncodeMeta::default()).unwrap() + } + + /// The fast path shows what the general decode shows for a plain sRGB JPEG, at the same size. + #[test] + fn matches_the_general_decode_within_a_few_levels() { + let b = jpeg(1203, 801, |x, y| [((x * 7) ^ (y * 3)) as u8, (x / 5 + y / 3) as u8, ((x * y) % 251) as u8]); + for (mw, mh) in [(300u32, 300u32), (640, 640), (1203, 801), (5000, 5000), (500, 120)] { + let (fast, o) = srgb8_fit(&b, mw, mh).unwrap().unwrap(); + let slow = crate::decode(&b, DecodeOptions::fit(mw, mh)).unwrap().to_srgb8(); + assert_eq!((fast.width, fast.height), (slow.width, slow.height), "box {mw}×{mh}"); + assert_eq!(o, 1); + let diff: Vec = + fast.data.iter().zip(&slow.data).flat_map(|(a, b)| (0..3).map(move |c| (a[c] as i32 - b[c] as i32).unsigned_abs())).collect(); + let mean = diff.iter().sum::() as f64 / diff.len() as f64; + assert!(mean < 3.0, "box {mw}×{mh}: mean difference {mean}"); + } + } + + /// Anything but plain 8-bit sRGB goes the general way; junk is an error or `None`, never a panic. + #[test] + fn declines_what_it_does_not_cover_and_never_panics() { + let b = jpeg(64, 48, |x, _| [x as u8, 0, 0]); + assert!(srgb8_fit(&b, 0, 10).unwrap().is_none()); + assert!(srgb8_fit(b"not a jpeg", 10, 10).unwrap().is_none()); + for cut in [0, 2, 20, b.len() / 2, b.len() - 3] { + let _ = srgb8_fit(&b[..cut], 32, 32); + } + // a 1-pixel-wide image into a box much smaller in one axis + let thin = jpeg(1, 300, |_, y| [y as u8, y as u8, y as u8]); + let (img, _) = srgb8_fit(&thin, 10, 10).unwrap().unwrap(); + assert_eq!((img.width, img.height), (1, 10)); + } + + /// An incomplete ICC_PROFILE sequence (chunk 1 of 2) is "has a profile we can't read", not + /// "plain sRGB": the general decode handles it, as the doc says (CodeRabbit on #3577). + #[test] + fn an_incomplete_icc_sequence_goes_the_general_way() { + let b = jpeg(16, 16, |x, y| [x as u8, y as u8, 0]); + let mut seg = vec![0xFF, 0xE2, 0, 0]; + seg.extend_from_slice(b"ICC_PROFILE"); + seg.extend_from_slice(&[1, 2]); + seg.extend_from_slice(&[0u8; 64]); + let len = u16::try_from(seg.len() - 2).unwrap(); + seg[2..4].copy_from_slice(&len.to_be_bytes()); + let mut with = b[..2].to_vec(); + with.extend_from_slice(&seg); + with.extend_from_slice(&b[2..]); + assert!(parse_markers(&with).unwrap().icc_invalid); + assert!(srgb8_fit(&b, 8, 8).unwrap().is_some()); + assert!(srgb8_fit(&with, 8, 8).unwrap().is_none()); + } + + /// The 8-bit path refuses an APP1 flood before its decoders walk it again, as the general one does. + #[test] + fn an_app1_flood_is_refused_by_the_8_bit_path_too() { + let b = jpeg(16, 16, |x, y| [x as u8, y as u8, 0]); + let mut flood = b[..2].to_vec(); + for _ in 0..200 { + flood.extend_from_slice(&[0xFF, 0xE1, 0x00, 0x02]); + } + flood.extend_from_slice(&b[2..]); + match srgb8_fit(&flood, 8, 8) { + Err(Error::Malformed(_, m)) => assert!(m.contains("APP1"), "{m}"), + other => panic!("unexpected: {other:?}"), + } + } +} diff --git a/crates/codecs/src/lib.rs b/crates/codecs/src/lib.rs index 8210f33f01..ad918ddd02 100644 --- a/crates/codecs/src/lib.rs +++ b/crates/codecs/src/lib.rs @@ -169,6 +169,21 @@ pub fn decode(bytes: &[u8], opts: DecodeOptions) -> Result { std::panic::catch_unwind(|| decode_unguarded(bytes, format, &opts)).unwrap_or_else(|_| Err(Error::Malformed(format, "decoder panicked".into()))) } +/// A JPEG for display, fast: fitted within `max_w × max_h` (never enlarged) as 8-bit sRGB with its +/// EXIF orientation (1..=8, not applied), the picture `decode(bytes, DecodeOptions::fit(..)) +/// .to_srgb8()` gives within a few levels, in about half the time for a large camera preview. +/// `Ok(None)` when the file is not a plain 8-bit sRGB JPEG (an ICC profile or an Adobe RGB EXIF +/// hint, CMYK, 12-bit, lossless or arithmetic coding) or its headers can't be read: use +/// [`decode`], which reports a malformed header as an error. Bad scan data is an error here, and +/// nothing is ever a panic. +pub fn decode_jpeg_srgb8_fit(bytes: &[u8], max_w: u32, max_h: u32) -> Result> { + if sniff(bytes) != Some(Format::Jpeg) { + return Ok(None); + } + std::panic::catch_unwind(|| jpeg::srgb8_fit(bytes, max_w, max_h)) + .unwrap_or_else(|_| Err(Error::Malformed(Format::Jpeg, "decoder panicked".into()))) +} + /// What a file's headers say about its image: enough to catalogue it without decoding pixels. #[derive(Clone, Copy, Debug, PartialEq, Eq)] pub struct Header { diff --git a/crates/codecs/tests/it/header.rs b/crates/codecs/tests/it/header.rs index f6ce7d3216..7fec902228 100644 --- a/crates/codecs/tests/it/header.rs +++ b/crates/codecs/tests/it/header.rs @@ -208,6 +208,10 @@ fn samsung_sef_trailer_without_eoi() { assert_eq!(full.image.data, reference.image.data, "the same pixels as the file with its EOI"); let small = decode(&pano, DecodeOptions::fit(32, 32)).unwrap(); assert_eq!((small.source_width, small.source_height), (160, 96)); + // the fast display decode too, scaled (where the trailer after the scan used to fail it) + let (fit, _) = lightcraft_codecs::decode_jpeg_srgb8_fit(&pano, 32, 32).unwrap().unwrap(); + let (fit_ref, _) = lightcraft_codecs::decode_jpeg_srgb8_fit(&jpeg, 32, 32).unwrap().unwrap(); + assert_eq!((fit.width, fit.height, &fit.data), (fit_ref.width, fit_ref.height, &fit_ref.data)); // an FF D9 inside the SEF data is not the image's end let eoi_in_data = sef_file(scan, block, b"SEFH", 0x08E0, &[0xFF, 0xC0, 0, 0x11, 0xFF, 0xD9, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12], None); assert_eq!(read_header(&eoi_in_data).unwrap().width, 160); diff --git a/crates/engine/Cargo.toml b/crates/engine/Cargo.toml index 0d40c73184..a01f28510b 100644 --- a/crates/engine/Cargo.toml +++ b/crates/engine/Cargo.toml @@ -49,6 +49,8 @@ lightcraft-segment = { workspace = true, optional = true } lightcraft-fetch = { workspace = true } # capture-time parsing for face recognition (native only) chrono = { workspace = true } +# free space of the drive holding the preview cache (the low-disk alert; pure Rust: rustix / windows-sys) +fs4 = { workspace = true } [features] # AI masks with SAM 3 (Object / Describe). Off for the web and CLI builds. diff --git a/crates/engine/src/cmd/library/specs.rs b/crates/engine/src/cmd/library/specs.rs index b99e97c054..4274e689c9 100644 --- a/crates/engine/src/cmd/library/specs.rs +++ b/crates/engine/src/cmd/library/specs.rs @@ -1367,6 +1367,8 @@ pub fn specs() -> Vec { ), // ---- persistence cmd!(query "library.info", "Library Info", [], None, "{}", always, |s, _| { + // the answer of a free-space read that ran on a worker (lowDiskSpace below) + s.poll_disk_space(); let photos = s.catalog.len(); let albums = s.catalog.albums().count(); let (rendered_n, rendered_bytes) = s.media.rendered.mem_usage(); @@ -1402,6 +1404,8 @@ pub fn specs() -> Vec { "lastError": lib.last_error, // settings files that were unreadable or damaged at open (kept, defaults used) "settingsWarnings": lib.settings_warnings, + // the drive holding the preview cache ran low this session (warned once) + "lowDiskSpace": s.disk_space_alert_raised(), // changes applied in memory whose write failed (retried by every save) "unsavedOps": s.unsaved().map_or(0, |u| u.0), "unsavedError": s.unsaved().map(|u| u.1), diff --git a/crates/engine/src/cmd/previews.rs b/crates/engine/src/cmd/previews.rs index 95aff90b79..e9c40232e9 100644 --- a/crates/engine/src/cmd/previews.rs +++ b/crates/engine/src/cmd/previews.rs @@ -31,6 +31,20 @@ pub struct PreviewBuild { pub what: &'static str, /// Why the whole build failed (e.g. the smart previews folder isn't there). pub error: std::sync::Mutex>, + /// Photos whose previews were there already (from an earlier build, the loupe or the photos + /// prepared ahead): counted as done, not rendered again (PERF-PREVIEW-REUSE). + pub skipped: AtomicUsize, + /// Photos left alone because another job was building them at that moment. + pub shared: AtomicUsize, + /// Started by the background preparation ([`crate::prep`]): it yields to the photo on screen, + /// and a build the user (or an import) asks for replaces it. + pub background: bool, + /// The photos in build order (ids), and where the user is among them: the next photo built + /// is the nearest one not built yet from there, in the direction of travel + /// ([`PreviewBuild::focus_on`]). + pub order: Vec, + pub focus: AtomicUsize, + pub backward: AtomicBool, /// Test-only: the gate this build's lanes wait at before a photo's first render (see /// [`test_gate`]). Captured by `build` on the calling thread; production builds never have one. #[cfg(test)] @@ -47,10 +61,38 @@ impl PreviewBuild { "cancelled": self.cancel.load(Ordering::Relaxed), "what": if self.what.is_empty() { "previews" } else { self.what }, "repaired": self.repaired.load(Ordering::Relaxed), + "skipped": self.skipped.load(Ordering::Relaxed), + "shared": self.shared.load(Ordering::Relaxed), + "background": self.background, "error": self.error(), }) } + pub fn running(&self) -> bool { + !self.finished.load(Ordering::Relaxed) + } + + /// The user is at photo `id`, moving `backward` or forward: build from there on. + pub fn focus_on(&self, id: lightcraft_catalog::PhotoId, backward: bool) { + if let Some(i) = self.order.iter().position(|p| *p == id.0) { + self.focus.store(i, Ordering::Relaxed); + self.backward.store(backward, Ordering::Relaxed); + } + } + + /// Index of the next photo to build in `queue` (`None`: none left): the first one still there + /// from the focus on in the direction of travel, then the rest the other way. + pub(crate) fn next_index(&self, queue: &[Option]) -> Option { + let n = queue.len(); + let at = self.focus.load(Ordering::Relaxed).min(n.saturating_sub(1)); + let there = |i: &usize| queue.get(*i).is_some_and(Option::is_some); + if self.backward.load(Ordering::Relaxed) { + (0..=at).rev().find(there).or_else(|| (at..n).find(there)) + } else { + (at..n).find(there).or_else(|| (0..at).rev().find(there)) + } + } + pub fn error(&self) -> Option { self.error.lock().unwrap_or_else(std::sync::PoisonError::into_inner).clone() } @@ -69,9 +111,93 @@ pub(crate) fn jobs_for(s: &mut Session, id: lightcraft_catalog::PhotoId, edge: O if e >= thumb { view.into_iter().chain(grid).collect() } else { grid.into_iter().chain(view).collect() } } +/// The previews the build of one photo would make are all there already, big enough: its view +/// render at least as large as asked for (`view_cache` of the view job) and its grid thumbnail +/// (`cache` of the thumbnail job), on disk when the cache has one: a render the loupe kept only in +/// memory (its deferred disk write skipped) is built again, and written. Only file headers are read. +pub(crate) fn already_built(jobs: &[RenderJob]) -> bool { + !jobs.is_empty() + && jobs.iter().all(|j| { + if let Some((cache, key)) = &j.view_cache { + let want = j.request.max_w.max(j.request.max_h) as f64; + cache.stored_dims(*key).is_some_and(|(w, h)| w.max(h) as f64 >= want * 0.99) + } else if let Some((cache, key)) = &j.cache { + cache.stored_dims(*key).is_some() + } else { + false + } + }) +} + +/// Photos some preview build is rendering right now, by (the cache they go to, photo id): two +/// builds (a background one being replaced, and the one replacing it) never render the same photo +/// at once. +static BUILDING: std::sync::Mutex> = std::sync::Mutex::new(Vec::new()); + +/// Holds a photo in [`BUILDING`] while it is rendered. +struct Building((usize, u64)); + +impl Building { + /// `id` of `jobs` (the cache their results go to tells sessions apart). + fn take(id: u64, jobs: &[RenderJob]) -> Option { + let cache = jobs.iter().find_map(|j| j.view_cache.as_ref().or(j.cache.as_ref())).map_or(0, |(c, _)| Arc::as_ptr(c) as usize); + let id = (cache, id); + let mut b = BUILDING.lock().unwrap_or_else(std::sync::PoisonError::into_inner); + if b.contains(&id) { + return None; + } + b.push(id); + Some(Building(id)) + } + + /// [`Self::take`], waiting while another build renders the photo (a build asked for replacing the + /// background one may meet it on the same photo): the photo is not counted until that render is over, as + /// built when it made the previews, else rendered here. With whether it had to wait; `None` when `cancel` + /// was asked for meanwhile. (On the web, with one thread, a photo held is not waited for: `Some((None, true))`.) + fn wait(id: u64, jobs: &[RenderJob], cancel: &AtomicBool) -> Option<(Option, bool)> { + if let Some(b) = Self::take(id, jobs) { + return Some((Some(b), false)); + } + Self::wait_held(id, jobs, cancel) + } + + /// [`Self::wait`] for a photo another build holds. + #[cfg(not(target_arch = "wasm32"))] + fn wait_held(id: u64, jobs: &[RenderJob], cancel: &AtomicBool) -> Option<(Option, bool)> { + loop { + if cancel.load(Ordering::Relaxed) { + return None; + } + std::thread::sleep(std::time::Duration::from_millis(10)); + if let Some(b) = Self::take(id, jobs) { + return Some((Some(b), true)); + } + } + } + + /// [`Self::wait`] for a photo another build holds: on the web, with one thread, it is not waited for. + #[cfg(target_arch = "wasm32")] + fn wait_held(_id: u64, _jobs: &[RenderJob], cancel: &AtomicBool) -> Option<(Option, bool)> { + (!cancel.load(Ordering::Relaxed)).then_some((None, true)) + } +} + +/// Tests: hold `id` of `jobs` as a build rendering it would, until the returned value is dropped. +#[cfg(test)] +pub(crate) fn hold_building(id: u64, jobs: &[RenderJob]) -> Option> { + Building::take(id, jobs).map(|b| Box::new(b) as Box) +} + +impl Drop for Building { + fn drop(&mut self) { + BUILDING.lock().unwrap_or_else(std::sync::PoisonError::into_inner).retain(|x| *x != self.0); + } +} + /// The build's row in the activity stack (`what` as in [`PreviewBuild::what`]). fn task_for(s: &Session, state: &PreviewBuild) -> TaskGuard { let (kind, label) = match state.what { + "" if state.background => ("previews", "Preparing photos in the background"), "" => ("previews", "Building previews"), "smart previews" => ("smartPreviews", "Building smart previews"), _ => ("smartPreviews", "Discarding smart previews"), @@ -84,7 +210,8 @@ fn task_for(s: &Session, state: &PreviewBuild) -> TaskGuard { /// Render one photo's jobs in order. The source the first job decodes serves the ones after it /// (the grid thumbnail is made from the view's source instead of decoding the file again). /// `None` when `cancel` was asked for before a job started: that photo is not counted as done. -fn run_photo_cancellable(jobs: Vec, cancel: Option<&AtomicBool>) -> Option { +/// `force`: the grid thumbnail is rendered too, not taken from the cache it is written to. +fn run_photo_cancellable(jobs: Vec, cancel: Option<&AtomicBool>, force: bool) -> Option { let mut decoded: Option = None; let mut ok = true; for mut j in jobs { @@ -97,7 +224,33 @@ fn run_photo_cancellable(jobs: Vec, cancel: Option<&AtomicBool>) -> O { j.source = crate::media::SourceRef::Loaded(Box::new(src.clone())); } + // The view render is written to the disk cache here, before the photo counts as built, + // not left to the cache's writer (which skips writes when it falls behind): a built + // preview is on disk, for this session and the next. (Taken out of the job: it would only + // queue the write, and would serve a smaller cached view.) + // A preview that isn't kept fails the photo: it is not built (a full or unwritable cache folder, or the + // cache discarded while the photo rendered). + let keep = j.view_cache.take().map(|v| (v, j.cache_generation)); + // forced: the thumbnail job's cache is only written to, after the render (a job reads its cache first; + // CodeRabbit on #3577) + let rewrite = if force { j.cache.take().map(|c| (c, j.cache_generation)) } else { None }; let r = j.run(); + if let (Some(((cache, key), generation)), Ok(img)) = (rewrite, r.rendered.as_ref()) + && !cache.put_at(generation, key, Arc::new(img.image.clone())) + { + log::warn!("building previews: the grid thumbnail could not be kept"); + ok = false; + } + if let (Some(((cache, key), generation)), Ok(img)) = (keep, r.rendered.as_ref()) + && !cache.put_at(generation, key, Arc::new(img.image.clone())) + { + if cache.generation() == generation { + log::warn!("building previews: the view preview could not be written to the cache folder"); + } else { + log::warn!("building previews: the preview cache was discarded while the photo rendered"); + } + ok = false; + } ok &= r.rendered.is_ok(); if decoded.is_none() { decoded = r.loaded; @@ -109,7 +262,7 @@ fn run_photo_cancellable(jobs: Vec, cancel: Option<&AtomicBool>) -> O /// [`run_photo_cancellable`] for callers that render a photo without a cancel flag. #[cfg(test)] pub(crate) fn run_photo(jobs: Vec) -> bool { - run_photo_cancellable(jobs, None) == Some(true) + run_photo_cancellable(jobs, None, false) == Some(true) } /// How many photos render side by side: as many as an export batch for standard previews (each @@ -224,34 +377,83 @@ pub(crate) mod test_gate { pub(crate) fn take() -> Option> { NEXT_GATE.with(|g| g.borrow_mut().take()) } + + thread_local! { + /// The next build started on this thread fails to start its worker (a thread the system + /// won't give), taken by `build`. + static FAIL_SPAWN: std::cell::Cell = const { std::cell::Cell::new(false) }; + } + + /// Have the next build started on this thread fail to start its worker. + pub(crate) fn fail_next_spawn() { + FAIL_SPAWN.with(|f| f.set(true)); + } + + /// Whether this build's worker is to fail to start (once). + pub(crate) fn take_fail_spawn() -> bool { + FAIL_SPAWN.with(std::cell::Cell::take) + } } -fn run_all(jobs: Vec>, state: &PreviewBuild, task: &TaskGuard, lanes: usize) { +/// One photo's work in a build: its id and its render jobs ([`jobs_for`]). +type PhotoJobs = (u64, Vec); + +/// Build every photo of `jobs` on `lanes` threads. `force`: render even the photos whose previews +/// are there. `task`: the build's own row (`None`: another job's row shows it, e.g. the import's +/// Getting photos ready). +fn run_all(jobs: Vec, state: &PreviewBuild, task: Option<&TaskGuard>, lanes: usize, force: bool) { let total = jobs.len(); - let queue = std::sync::Mutex::new(jobs.into_iter()); + let queue = std::sync::Mutex::new(jobs.into_iter().map(Some).collect::>()); let work = || { loop { if state.cancel.load(Ordering::Relaxed) { break; } - let next = queue.lock().unwrap_or_else(std::sync::PoisonError::into_inner).next(); - let Some(photo_jobs) = next else { break }; + // the background preparation waits while the photo on screen is being made + if state.background && crate::prep::wait_for_foreground(&state.cancel) { + break; + } + let next = { + let mut q = queue.lock().unwrap_or_else(std::sync::PoisonError::into_inner); + state.next_index(&q).and_then(|i| q.get_mut(i)).and_then(Option::take) + }; + let Some((id, photo_jobs)) = next else { break }; // A cancel accepted while this lane waited for the queue (or for another lane's // photo) stops before this photo's first render; a cancelled photo is not counted. if state.cancel.load(Ordering::Relaxed) { break; } - #[cfg(test)] - if let Some(gate) = &state.gate { - gate.enter(); - } - let outcome = crate::guard::catch("building previews", || run_photo_cancellable(photo_jobs, Some(&state.cancel))).unwrap_or(Some(false)); - let Some(ok) = outcome else { break }; - if !ok { - state.failed.fetch_add(1, Ordering::Relaxed); + if !force && already_built(&photo_jobs) { + state.skipped.fetch_add(1, Ordering::Relaxed); + } else if let Some((held, shared)) = Building::wait(id, &photo_jobs, &state.cancel) { + // another build was rendering this photo: what it made counts, if it made it (not + // for a forced build, which renders every photo itself) + if !force && shared && (held.is_none() || already_built(&photo_jobs)) { + state.shared.fetch_add(1, Ordering::Relaxed); + let done = state.done.fetch_add(1, Ordering::Relaxed).saturating_add(1); + if let Some(task) = task { + task.progress(done as u64, total as u64); + } + continue; + } + #[cfg(test)] + if let Some(gate) = &state.gate { + gate.enter(); + } + let outcome = + crate::guard::catch("building previews", || run_photo_cancellable(photo_jobs, Some(&state.cancel), force)).unwrap_or(Some(false)); + let Some(ok) = outcome else { break }; + if !ok { + state.failed.fetch_add(1, Ordering::Relaxed); + } + } else { + // cancelled while another build was rendering it: not counted + break; } let done = state.done.fetch_add(1, Ordering::Relaxed).saturating_add(1); - task.progress(done as u64, total as u64); + if let Some(task) = task { + task.progress(done as u64, total as u64); + } } }; #[cfg(not(target_arch = "wasm32"))] @@ -272,43 +474,99 @@ fn run_all(jobs: Vec>, state: &PreviewBuild, task: &TaskGuard, la state.finished.store(true, Ordering::Relaxed); } +/// The long edge a standard preview is built at: [`STANDARD_EDGE`], or what the loupe asks for +/// on this screen when that is more ([`Session::loupe_edge`]), so the loupe finds what was built. +pub fn standard_edge(s: &Session) -> usize { + STANDARD_EDGE.max(s.loupe_edge.min(8192)) +} + fn build(s: &mut Session, p: &Value) -> Result { const C: &str = "library.buildPreviews"; // tests: this invocation's gate, consumed here so an early error cannot attach it to a later build #[cfg(test)] let gate = test_gate::take(); let edge = match str_param(p, "size").unwrap_or("standard") { - "standard" => Some(p.get("edge").and_then(Value::as_u64).map_or(STANDARD_EDGE, |e| e.clamp(256, 8192) as usize)), + "standard" => Some(p.get("edge").and_then(Value::as_u64).map_or_else(|| standard_edge(s), |e| e.clamp(256, 8192) as usize)), "full" | "1:1" => None, other => return Err(bad(C, format!("unknown size `{other}` (standard|full)"))), }; - if s.preview_build.as_ref().is_some_and(|b| !b.finished.load(Ordering::Relaxed)) { - return Err(bad(C, "a preview build is already running")); + let background = p.get("background").and_then(Value::as_bool).unwrap_or(false); + let force = p.get("force").and_then(Value::as_bool).unwrap_or(false); + // (`row: false`: the job that asked shows the progress in its own row) + let row = p.get("row").and_then(Value::as_bool).unwrap_or(true); + // the background preparation gives way to a build that was asked for; nothing replaces a build + // that was asked for (it carries on, and the photos it built are skipped by the next). The + // background one is stopped only once this request is known to be good (below): a request + // with bad ids leaves it running (CodeRabbit on #3544) + let mut gives_way = None; + if let Some(b) = s.preview_build.as_ref().filter(|b| b.running()) { + if b.background && !background { + gives_way = Some(b.cancel.clone()); + } else { + return Err(bad(C, "a preview build is already running")); + } } - // explicit ids, else the selection, else everything in view - let ids = if p.get("ids").is_some() || p.get("id").is_some() || !s.selection.ids.is_empty() { s.targets(p)? } else { s.visible_cloned() }; - let jobs: Vec> = ids.iter().map(|id| jobs_for(s, *id, edge)).filter(|j| !j.is_empty()).collect(); + s.fit_disk_cache(); + // explicit ids, else the selected photos (more than one: as the menu says), else everything in + // view (a single photo selected is the one open, not a choice of photos: "Visible" in the menu) + let ids = if p.get("ids").is_some() || p.get("id").is_some() || (!background && s.selection.ids.len() > 1) { + s.targets(p)? + } else { + s.visible_cloned() + }; + let jobs: Vec = ids.iter().map(|id| (id.0, jobs_for(s, *id, edge))).filter(|j| !j.1.is_empty()).collect(); + // the background build stays the session's until this one has started (a worker that can't + // start leaves it running and in place: CodeRabbit on #3544) + let previous = s.preview_build.clone(); + let order = jobs.iter().map(|j| j.0).collect(); + #[cfg_attr(not(test), allow(unused_mut))] + let mut build_state = PreviewBuild { total: jobs.len(), background, order, ..Default::default() }; #[cfg(test)] - let state = Arc::new(PreviewBuild { total: jobs.len(), gate, ..Default::default() }); - #[cfg(not(test))] - let state = Arc::new(PreviewBuild { total: jobs.len(), ..Default::default() }); + { + build_state.gate = gate; + } + let state = Arc::new(build_state); + if let Some(at) = p.get("from").and_then(Value::as_u64) { + state.focus_on(lightcraft_catalog::PhotoId(at), p.get("backward").and_then(Value::as_bool).unwrap_or(false)); + } s.preview_build = Some(state.clone()); - let task = task_for(s, &state); - let lanes = lanes(jobs.len(), edge.is_none()); + let task = row.then(|| task_for(s, &state)); + // the background preparation leaves most of the machine to the photo on screen and the user + let lanes = if background { crate::prep::background_lanes() } else { lanes(jobs.len(), edge.is_none()) }; let wait = p.get("wait").and_then(Value::as_bool).unwrap_or(false) || cfg!(target_arch = "wasm32"); if wait { - run_all(jobs, &state, &task, lanes); + if let Some(cancel) = gives_way { + cancel.store(true, Ordering::Relaxed); + } + run_all(jobs, &state, task.as_ref(), lanes, force); return Ok(state.json()); } #[cfg(not(target_arch = "wasm32"))] { // the row stays while the worker runs (it is dropped with the closure, however that ends) let st = state.clone(); - std::thread::Builder::new() - .name("lc-build-previews".into()) - .spawn(move || run_all(jobs, &st, &task, lanes)) - .map_err(|e| bad(C, format!("could not start: {e}")))?; + #[cfg(test)] + let started = if test_gate::take_fail_spawn() { + drop(task); + Err(std::io::Error::other("no thread (test)")) + } else { + std::thread::Builder::new().name("lc-build-previews".into()).spawn(move || run_all(jobs, &st, task.as_ref(), lanes, force)) + }; + #[cfg(not(test))] + let started = std::thread::Builder::new().name("lc-build-previews".into()).spawn(move || run_all(jobs, &st, task.as_ref(), lanes, force)); + if let Err(e) = started { + // never started: not left "running", which would hold every later build back, and the + // build it was to replace carries on + state.finished.store(true, Ordering::Relaxed); + s.preview_build = previous; + return Err(bad(C, format!("could not start: {e}"))); + } + if let Some(cancel) = gives_way { + cancel.store(true, Ordering::Relaxed); + } } + #[cfg(target_arch = "wasm32")] + let _ = previous; Ok(state.json()) } @@ -476,10 +734,18 @@ fn smart(s: &mut Session, p: &Value) -> Result { let dir = s.media.smart_dir.clone().ok_or_else(|| bad(C, "smart previews need a library on disk"))?; let discard = p.get("discard").and_then(Value::as_bool).unwrap_or(false); let background = p.get("background").and_then(Value::as_bool).unwrap_or(false); - if background && s.preview_build.as_ref().is_some_and(|b| !b.finished.load(Ordering::Relaxed)) { - return Err(bad(C, "a preview build is already running")); + // (as Build Previews: the photos named, or the selected ones when more than one is, else + // everything in view; read first, so a request with bad ids stops nothing: CodeRabbit on #3577) + let ids = if p.get("ids").is_some() || p.get("id").is_some() || s.selection.ids.len() > 1 { s.targets(p)? } else { s.visible_cloned() }; + // (the background preparation gives way, as to Build Previews: once this one's worker has started) + let mut gives_way = None; + if background && let Some(b) = s.preview_build.as_ref().filter(|b| b.running()) { + if b.background { + gives_way = Some(b.cancel.clone()); + } else { + return Err(bad(C, "a preview build is already running")); + } } - let ids = if p.get("ids").is_some() || !s.selection.ids.is_empty() { s.targets(p)? } else { s.visible_cloned() }; let mut jobs: Vec = Vec::new(); for id in ids { let Some(ph) = s.catalog.photo(id).cloned() else { continue }; @@ -496,19 +762,35 @@ fn smart(s: &mut Session, p: &Value) -> Result { if background { let what = if discard { "discarding smart previews" } else { "smart previews" }; let state = Arc::new(PreviewBuild { total: jobs.len(), what, ..Default::default() }); - s.preview_build = Some(state.clone()); + let previous = s.preview_build.replace(state.clone()); let task = task_for(s, &state); let st = state.clone(); - std::thread::Builder::new() - .name("lc-smart-previews".into()) - .spawn(move || { - if let Err(e) = smart_run(&dir, custom, discard, jobs, Some((&st, &task))) { - log::warn!("{C}: {e}"); - *st.error.lock().unwrap_or_else(std::sync::PoisonError::into_inner) = Some(e); - } - st.finished.store(true, Ordering::Relaxed); - }) - .map_err(|e| bad(C, format!("could not start: {e}")))?; + let work = move || { + if let Err(e) = smart_run(&dir, custom, discard, jobs, Some((&st, &task))) { + log::warn!("{C}: {e}"); + *st.error.lock().unwrap_or_else(std::sync::PoisonError::into_inner) = Some(e); + } + st.finished.store(true, Ordering::Relaxed); + }; + #[cfg(test)] + let started = if test_gate::take_fail_spawn() { + drop(work); + Err(std::io::Error::other("no thread (test)")) + } else { + std::thread::Builder::new().name("lc-smart-previews".into()).spawn(work) + }; + #[cfg(not(test))] + let started = std::thread::Builder::new().name("lc-smart-previews".into()).spawn(work); + if let Err(e) = started { + // never started: not left "running", and the background build it was to replace carries on + // (CodeRabbit on #3077) + state.finished.store(true, Ordering::Relaxed); + s.preview_build = previous; + return Err(bad(C, format!("could not start: {e}"))); + } + if let Some(cancel) = gives_way { + cancel.store(true, Ordering::Relaxed); + } return Ok(state.json()); } let r = smart_run(&dir, custom, discard, jobs, None).map_err(|e| bad(C, e))?; @@ -647,7 +929,7 @@ pub fn specs() -> Vec { "Build Smart Previews", [], None, - "{ids?, discard?: bool, background?: bool} — build (or discard) the smart previews of the selected photos (else all in view): compact proxies in the library that keep photos editable and exportable (at proxy size) while their originals are offline; a damaged proxy (cut short) is rebuilt → {built, repaired, removed, failed}; with background (the app's menu) the files are read and written on a worker thread → {total, done, failed, repaired, running, what} like library.buildPreviews (progress: library.previewProgress, stop: library.cancelPreviews)", + "{ids?, discard?: bool, background?: bool} — build (or discard) the smart previews of the selected photos (when more than one is selected; else all in view): compact proxies in the library that keep photos editable and exportable (at proxy size) while their originals are offline; a damaged proxy (cut short) is rebuilt → {built, repaired, removed, failed}; with background (the app's menu) the files are read and written on a worker thread → {total, done, failed, repaired, running, what} like library.buildPreviews (progress: library.previewProgress, stop: library.cancelPreviews)", always, smart ), @@ -674,13 +956,43 @@ pub fn specs() -> Vec { "Build Previews", [], None, - "{size?: standard (2048 px, or `edge`) | full (1:1), ids?, wait?: bool} — render the grid thumbnail and loupe view of the selected photos (else all in view) into the preview cache, in the background unless `wait` → {total, done, failed, running}", + "{size?: standard (2048 px, or the loupe's size when bigger, or `edge`) | full (1:1), ids?, wait?: bool, force?: bool, from?: id, backward?: bool, background?: bool, row?: bool} — render the grid thumbnail and loupe view of the selected photos (when more than one is selected; else all in view) into the preview cache (memory and disk), in the background unless `wait`. Photos whose previews are there already (big enough) are skipped unless `force`; the photos nearest `from` go first. `background`: the background preparation's build (two lanes, waits while the photo on screen renders; a build asked for replaces it). `row: false`: no row of its own in the activity stack (the caller's row shows it) → {total, done, failed, skipped, shared, running}", always, build ), - cmd!(query "library.previewProgress", "Preview Build Progress", [], None, "{} → {total, done, failed, repaired, running, cancelled, what, error} | null", always, |s, _| { + cmd!(query "library.previewProgress", "Preview Build Progress", [], None, "{} → {total, done, failed, repaired, skipped, shared, background, running, cancelled, what, error} | null", always, |s, _| { Ok(s.preview_build.as_ref().map_or(Value::Null, |b| b.json())) }), + cmd!(query "prefs.backgroundPrep", "Background Preparation Settings", [], None, "{} → {app, library|null, settings (in use), own: bool (this library keeps its own), building} — what is prepared in the background per mode (culling | library: previews of every photo in view, ahead = photos ahead rendered for the loupe, eyeAhead = photos ahead whose eye focus strip is worked out, scope all|marked|onScreen; develop: ahead = photos kept decoded ready to edit, scope)", always, |s, _| { + Ok(s.prep_json()) + }), + cmd!( + "prefs.setBackgroundPrep", + "Set Background Preparation", + [], + None, + "{settings?: partial {culling?, library?, develop?}, library?: bool (this library's own settings; else the app-wide ones), reset?: bool (this library goes back to the app-wide settings)} → as prefs.backgroundPrep", + always, + |s, p| { + const C: &str = "prefs.setBackgroundPrep"; + let library = p.get("library").and_then(Value::as_bool).unwrap_or(false); + if p.get("reset").and_then(Value::as_bool).unwrap_or(false) { + s.clear_library_prep()?; + return Ok(s.prep_json()); + } + if library && s.library.is_none() { + return Err(bad(C, "no library is open")); + } + let base = if library { s.prep.library.unwrap_or(s.prep.app) } else { s.prep.app }; + let next = match p.get("settings") { + Some(patch) if patch.is_object() => base.patched(patch).map_err(|e| bad(C, e))?, + Some(_) => return Err(bad(C, "settings must be an object")), + None => base, + }; + s.set_prep_settings(next, library)?; + Ok(s.prep_json()) + } + ), cmd!("library.cancelPreviews", "Cancel Preview Build", [], None, "{}", always, |s, _| { if let Some(b) = &s.preview_build { b.cancel.store(true, Ordering::Relaxed); diff --git a/crates/engine/src/disk_space.rs b/crates/engine/src/disk_space.rs new file mode 100644 index 0000000000..eb7f29e967 --- /dev/null +++ b/crates/engine/src/disk_space.rs @@ -0,0 +1,577 @@ +//! A once-per-session alert when the drive holding the library's preview cache runs low. +//! +//! The automatic preview cache budget grows with the library and has no ceiling +//! ([`crate::media::default_disk_cache_bytes`]): a large library can fill its drive. Rather than cap +//! it, the session looks at the drive's free space when the library opens and, at most every +//! [`CHECK_EVERY`] while the app runs (thumbnails and previews fill the cache); when it is under [`LOW_FREE_PERCENT`] of the drive or +//! under [`LOW_FREE_BYTES`], it hands out one [`LowDiskAlert`] ([`crate::Session::take_disk_space_alert`]) +//! that the UI shows with a way to Settings ▸ Performance, where the cache can be given a limit. It is +//! not repeated until the next library open or app run. Reading the free space can fail (a network +//! share that doesn't say, the web build): then there is no alert, only a log line. +//! Natively the read runs on a worker thread and its answer is taken on a later call, so a slow +//! network share never holds up the UI thread that asks. + +use std::path::Path; + +/// The drive is low when less than this share of it (percent) is free… +pub const LOW_FREE_PERCENT: u64 = 10; +/// …or when less than this is free (20 GB), whichever comes first. +pub const LOW_FREE_BYTES: u64 = 20 << 30; +/// While a library is open, the free space is read at most this often (statvfs on the cache's +/// drive: cheap, but not something for every frame or every write). +pub const CHECK_EVERY: std::time::Duration = std::time::Duration::from_secs(5 * 60); +/// A read when the library opens that hasn't answered after this long shows in the activity stack. +const SHOW_ROW_AFTER: std::time::Duration = std::time::Duration::from_millis(200); +/// A read whose thread couldn't start (the system out of threads for a moment) is tried again this +/// soon, not after [`CHECK_EVERY`]. +pub const RETRY_AFTER: std::time::Duration = std::time::Duration::from_secs(10); + +/// What a read's worker answers: the drive's space and the cache's size when low (`None`: the drive +/// didn't say), or that the read couldn't start. +type Answer = Result, NotStarted>; + +/// A read's thread couldn't be started. +struct NotStarted; + +/// Space on one drive, in bytes. +#[derive(Clone, Copy, Debug, PartialEq, Eq)] +pub struct DiskSpace { + /// Free for this user (what can still be written). + pub available: u64, + /// The drive's size. + pub total: u64, +} + +impl DiskSpace { + /// Under [`LOW_FREE_PERCENT`] of the drive free, or under [`LOW_FREE_BYTES`]. A drive that + /// reports no size isn't judged by its share. + pub fn is_low(&self) -> bool { + let under_share = self.total > 0 && u128::from(self.available) * 100 < u128::from(self.total) * u128::from(LOW_FREE_PERCENT); + under_share || self.available < LOW_FREE_BYTES + } +} + +/// What the UI tells the user once: the drive's free space and size, and how big the preview +/// cache is now. +#[derive(Clone, Debug, PartialEq, Eq)] +pub struct LowDiskAlert { + pub space: DiskSpace, + /// Bytes the preview cache takes on that drive. + pub cache_bytes: u64, +} + +/// The free space of the drive holding `dir` (or its parent when `dir` isn't there yet). `None` +/// (logged) when it can't be read. +pub fn read(dir: &Path) -> Option { + #[cfg(not(target_arch = "wasm32"))] + { + let probe = if dir.exists() { dir } else { dir.parent().unwrap_or(dir) }; + match fs4::statvfs(probe) { + Ok(s) => Some(DiskSpace { available: s.available_space(), total: s.total_space() }), + Err(e) => { + log::info!("disk space: can't read the free space of {}: {e}", probe.display()); + None + } + } + } + #[cfg(target_arch = "wasm32")] + { + let _ = dir; + None + } +} + +/// Reads the free space of a drive ([`read`]; a test can put a slow one in its place). +pub type Reader = std::sync::Arc Option + Send + Sync>; + +/// The session's state of the alert (reset when a library opens). +#[derive(Default)] +pub(crate) struct DiskAlertState { + /// The alert was handed out (or is waiting to be) this session: never again until reset. + pub(crate) raised: bool, + /// The alert waiting for the UI. + pub(crate) pending: Option, + /// When the free space was last read. + pub(crate) checked_at: Option, + /// The read running on a worker (natively): a network share can take its time to answer, and + /// the check runs on the UI thread. It answers with the drive's space and, when that is low, + /// the preview cache's size (counting the cache can mean reading every file's size: that too + /// stays off the UI thread). + probe: Option>, + /// The running read was started for the library open before this one: its answer is ignored, + /// and the drive is read again for the current library once it has come. Until then no second + /// read starts, so switching libraries while a drive hangs never piles up workers. + stale: bool, + /// In place of [`read`] (tests). + pub(crate) reader: Option, + /// Tests: the read's thread that fails to start (`"disk-space"`, `"disk-space-read"`). + pub(crate) fail_spawn: Option<&'static str>, +} + +/// Start a named thread; `fail` stands for a system that can't (tests). +#[cfg(not(target_arch = "wasm32"))] +fn spawn(name: &str, fail: bool, f: impl FnOnce() + Send + 'static) -> std::io::Result<()> { + if fail { + return Err(std::io::Error::other("no thread (test)")); + } + std::thread::Builder::new().name(name.into()).spawn(f).map(drop) +} + +impl DiskAlertState { + /// A fresh state for a newly opened library (the reader stays; a read still running is kept, + /// marked stale). + pub(crate) fn reset(&mut self) { + let probe = self.probe.take(); + let stale = probe.is_some(); + *self = DiskAlertState { reader: self.reader.take(), fail_spawn: self.fail_spawn, probe, stale, ..Default::default() }; + } +} + +impl crate::Session { + /// Read the free space of the drive holding the preview cache, at most every [`CHECK_EVERY`] + /// (`now`: whatever the last read, as when the library opens), and raise the alert if it is low + /// ([`Session::note_disk_space`](crate::Session::note_disk_space)). Nothing once the alert was + /// raised this session, without an on-disk library, or when the space can't be read. + /// The read itself runs on a worker (natively) and its answer is taken by a later call or + /// [`Session::poll_disk_space`]: a slow drive never holds up the caller. + pub fn check_disk_space(&mut self, now: bool) { + self.poll_disk_space(); + // (a read still running for an earlier library open starts this library's read when it answers) + if self.disk_alert.raised || self.disk_alert.probe.is_some() { + return; + } + let Some(dir) = self.library.as_ref().filter(|l| l.on_disk).map(crate::library::Library::thumbs_dir) else { return }; + let at = web_time::Instant::now(); + if !now && self.disk_alert.checked_at.is_some_and(|t| at.saturating_duration_since(t) < CHECK_EVERY) { + return; + } + self.disk_alert.checked_at = Some(at); + let reader = self.disk_alert.reader.clone(); + let rendered = self.media.rendered.clone(); + let read_now = move || { + let space = match reader { + Some(r) => r(&dir), + None => read(&dir), + }?; + // the cache is counted only when the alert will tell its size + let cache_bytes = if space.is_low() { rendered.disk().map_or(0, |d| d.size()) } else { 0 }; + Some((space, cache_bytes)) + }; + #[cfg(not(target_arch = "wasm32"))] + { + // a read when the library opens that takes more than a moment (a network share slow to + // answer, or the cache counted when the drive is low) shows in the activity stack; a quick + // one, and the quiet periodic ones every few minutes, don't + let activity = now.then(|| self.activity.clone()); + let fail = self.disk_alert.fail_spawn; + let (tx, rx) = std::sync::mpsc::channel(); + let started = spawn("disk-space", fail == Some("disk-space"), move || { + let answer = match activity { + None => Ok(read_now()), + Some(activity) => { + let (done, answered) = std::sync::mpsc::channel(); + let reading = spawn("disk-space-read", fail == Some("disk-space-read"), move || { + let _ = done.send(read_now()); + }); + match reading { + Err(e) => { + log::info!("disk space: can't start the free-space read: {e}"); + Err(NotStarted) + } + Ok(()) => match answered.recv_timeout(SHOW_ROW_AFTER) { + Ok(answer) => Ok(answer), + Err(_) => { + let _row = activity.start("cache", "Checking free disk space", crate::activity::Cancel::No); + Ok(answered.recv().ok().flatten()) + } + }, + } + } + }; + let _ = tx.send(answer); + }); + match started { + Ok(()) => self.disk_alert.probe = Some(rx), + Err(e) => { + log::info!("disk space: can't start the free-space read: {e}"); + self.retry_disk_space_soon(); + } + } + } + #[cfg(target_arch = "wasm32")] + if let Some((space, cache_bytes)) = read_now() { + self.raise_disk_alert(space, cache_bytes); + } + } + + /// Take the answer of the free-space read running on a worker, if it has come, and raise the + /// alert when the drive is low. An answer for an earlier library open is dropped and the drive + /// of the current one is read instead. Called by [`Session::check_disk_space`], + /// [`Session::take_disk_space_alert`] and `library.info`. + pub fn poll_disk_space(&mut self) { + let Some(rx) = self.disk_alert.probe.as_ref() else { return }; + let answer = match rx.try_recv() { + Ok(answer) => answer, + Err(std::sync::mpsc::TryRecvError::Empty) => return, + // (a worker that ended without answering: tried again soon too) + Err(std::sync::mpsc::TryRecvError::Disconnected) => Err(NotStarted), + }; + self.disk_alert.probe = None; + if std::mem::take(&mut self.disk_alert.stale) { + self.check_disk_space(true); + } else { + match answer { + Ok(Some((space, cache_bytes))) => { + self.raise_disk_alert(space, cache_bytes); + } + Ok(None) => {} + Err(NotStarted) => self.retry_disk_space_soon(), + } + } + } + + /// A read that couldn't start counts as made [`RETRY_AFTER`] less than [`CHECK_EVERY`] ago: the next check + /// reads the drive after [`RETRY_AFTER`] (CodeRabbit on #3577: it waited the whole five minutes). + fn retry_disk_space_soon(&mut self) { + let now = web_time::Instant::now(); + self.disk_alert.checked_at = Some(now.checked_sub(CHECK_EVERY.saturating_sub(RETRY_AFTER)).unwrap_or(now)); + } + + /// Raise the low-disk alert for `space` when it is low and the alert wasn't raised this session + /// yet (the numbers come from [`read`], or from a test). Returns whether it was raised now. + /// Counts the preview cache here, on the caller's thread: the session's own reads count it on + /// their worker. + pub fn note_disk_space(&mut self, space: DiskSpace) -> bool { + if self.disk_alert.raised || !space.is_low() { + return false; + } + let cache_bytes = self.media.rendered.disk().map_or(0, |d| d.size()); + self.raise_disk_alert(space, cache_bytes) + } + + /// [`Session::note_disk_space`] with the cache's size already counted. + fn raise_disk_alert(&mut self, space: DiskSpace, cache_bytes: u64) -> bool { + if self.disk_alert.raised || !space.is_low() { + return false; + } + log::warn!("disk space: {} of {} bytes free; the preview cache takes {cache_bytes} bytes", space.available, space.total); + self.disk_alert.raised = true; + self.disk_alert.pending = Some(LowDiskAlert { space, cache_bytes }); + true + } + + /// The low-disk alert not shown yet, if any (handed out once per session). + pub fn take_disk_space_alert(&mut self) -> Option { + self.poll_disk_space(); + self.disk_alert.pending.take() + } + + /// Read the drive's space with `reader` instead of asking the system (tests of the app). + #[doc(hidden)] + pub fn set_disk_space_reader(&mut self, reader: Option) { + self.disk_alert.reader = reader; + } + + /// Forget when the space was last read, as if [`CHECK_EVERY`] had passed (tests of the app). + #[doc(hidden)] + pub fn forget_disk_space_check(&mut self) { + self.disk_alert.checked_at = None; + } + + /// Whether a free-space read is running on its worker (its answer not taken yet). + pub fn disk_space_reading(&self) -> bool { + self.disk_alert.probe.is_some() + } + + /// How long until [`Session::check_disk_space`] reads the drive again (zero when it is due, `None` + /// when it won't: the alert was raised, a read runs, or nothing has been read yet). + pub fn disk_space_next_check_in(&self) -> Option { + let a = &self.disk_alert; + if a.raised || a.probe.is_some() { + return None; + } + Some(CHECK_EVERY.saturating_sub(a.checked_at?.elapsed())) + } + + /// How long the free-space read running on its worker has run (`Duration::MAX` for one started for the + /// library before), or `None` when none runs. + pub fn disk_space_read_running_for(&self) -> Option { + let a = &self.disk_alert; + a.probe.as_ref()?; + Some(a.checked_at.filter(|_| !a.stale).map_or(std::time::Duration::MAX, |t| t.elapsed())) + } + + /// Whether the low-disk alert was raised since the library opened (`library.info` → + /// `lowDiskSpace`). + pub fn disk_space_alert_raised(&self) -> bool { + self.disk_alert.raised + } +} + +#[cfg(test)] +mod tests { + use super::{DiskSpace, LOW_FREE_BYTES}; + use crate::Session; + + const GB: u64 = 1 << 30; + + #[test] + fn plenty_of_room_on_both_counts_raises_no_alert() { + let mut s = Session::new(); + assert!(!s.note_disk_space(DiskSpace { available: 200 * GB, total: 1000 * GB }), "20 % and 200 GB free"); + assert!(!s.note_disk_space(DiskSpace { available: 25 * GB, total: 100 * GB }), "25 % and 25 GB free"); + assert!(s.take_disk_space_alert().is_none()); + assert!(!s.disk_space_alert_raised()); + } + + #[test] + fn under_ten_percent_free_raises_the_alert() { + let mut s = Session::new(); + // 300 GB is far above 20 GB, but only 7.5 % of a 4 TB drive + let space = DiskSpace { available: 300 * GB, total: 4000 * GB }; + assert!(s.note_disk_space(space)); + let alert = s.take_disk_space_alert().expect("the alert is handed out"); + assert_eq!(alert.space, space); + assert!(s.disk_space_alert_raised()); + } + + #[test] + fn under_twenty_gb_free_raises_the_alert() { + let mut s = Session::new(); + // 15 % of a 100 GB drive is free, but that is under 20 GB + let space = DiskSpace { available: 15 * GB, total: 100 * GB }; + assert!(space.available < LOW_FREE_BYTES); + assert!(s.note_disk_space(space)); + assert_eq!(s.take_disk_space_alert().map(|a| a.space), Some(space)); + } + + #[test] + fn the_alert_is_raised_once_per_session() { + let mut s = Session::new(); + let low = DiskSpace { available: GB, total: 500 * GB }; + assert!(s.note_disk_space(low)); + assert!(s.take_disk_space_alert().is_some()); + assert!(s.take_disk_space_alert().is_none(), "handed out once"); + assert!(!s.note_disk_space(DiskSpace { available: GB / 2, ..low }), "lower still: not again this session"); + assert!(s.take_disk_space_alert().is_none()); + s.check_disk_space(true); + assert!(s.take_disk_space_alert().is_none(), "a check after the alert reads nothing"); + } + + #[test] + fn opening_a_library_starts_a_new_session_for_the_alert() { + let dir = std::env::temp_dir().join(format!("lc-disk-alert-{}", std::process::id())); + let _ = std::fs::remove_dir_all(&dir); + std::fs::create_dir_all(&dir).unwrap(); + let mut s = Session::new().with_fs(); + s.open_library(&dir, false).unwrap(); + // whatever this machine's drive holds, raise it by hand + let _ = s.take_disk_space_alert(); + let low = DiskSpace { available: GB, total: 500 * GB }; + if !s.disk_space_alert_raised() { + assert!(s.note_disk_space(low)); + } + assert!(!s.note_disk_space(low), "once per session"); + s.close_library().unwrap(); + s.open_library(&dir, false).unwrap(); + let _ = s.take_disk_space_alert(); + let raised_by_open = s.disk_space_alert_raised(); + assert!(raised_by_open || s.note_disk_space(low), "a new library open may warn again"); + s.close_library().unwrap(); + let _ = std::fs::remove_dir_all(&dir); + } + + /// A drive that takes its time to answer (a network share) doesn't hold up the library open + /// or the checks on the UI thread: the read runs on a worker and its answer is taken later. + #[test] + fn a_slow_drive_does_not_hold_up_the_check() { + use std::sync::{Arc, Mutex, mpsc}; + let dir = std::env::temp_dir().join(format!("lc-disk-alert-slow-{}", std::process::id())); + let _ = std::fs::remove_dir_all(&dir); + std::fs::create_dir_all(&dir).unwrap(); + let (release, wait) = mpsc::channel::<()>(); + let wait = Mutex::new(wait); + let mut s = Session::new().with_fs(); + s.disk_alert.reader = Some(Arc::new(move |_: &std::path::Path| { + // answers once the test lets it (or after 10 s, so a broken test can't hang) + let _ = wait.lock().unwrap().recv_timeout(std::time::Duration::from_secs(10)); + Some(DiskSpace { available: GB, total: 500 * GB }) + })); + s.open_library(&dir, false).unwrap(); + assert!(s.take_disk_space_alert().is_none(), "the open didn't wait for the drive's answer"); + // the read the open started shows in the activity stack once it takes more than a moment + // (CodeRabbit on #3077) + let reading = |s: &Session| s.activity.list().iter().any(|t| t.kind == "cache" && t.label == "Checking free disk space"); + let until = web_time::Instant::now() + std::time::Duration::from_secs(10); + while !reading(&s) { + assert!(web_time::Instant::now() < until, "{:?}", s.activity.list()); + std::thread::sleep(std::time::Duration::from_millis(5)); + } + assert!(s.disk_space_reading()); + s.check_disk_space(true); + assert!(s.take_disk_space_alert().is_none(), "nor does a check while the read runs"); + release.send(()).unwrap(); + let until = std::time::Instant::now() + std::time::Duration::from_secs(10); + let alert = loop { + if let Some(a) = s.take_disk_space_alert() { + break a; + } + assert!(std::time::Instant::now() < until, "the drive's answer was never taken"); + std::thread::sleep(std::time::Duration::from_millis(5)); + }; + assert_eq!(alert.space.available, GB); + assert!(s.disk_space_alert_raised()); + assert!(!reading(&s) && !s.disk_space_reading(), "the row goes with the read"); + s.close_library().unwrap(); + let _ = std::fs::remove_dir_all(&dir); + } + + /// A drive that hangs while the user switches libraries: the read still running is kept, no + /// second worker starts beside it, its answer (for the old library) is dropped, and the current + /// library's drive is read once it has come. + #[cfg(not(target_arch = "wasm32"))] + #[test] + fn switching_libraries_while_a_drive_hangs_starts_no_second_read() { + use std::sync::{Arc, Mutex, mpsc}; + use std::time::{Duration, Instant}; + let root = std::env::temp_dir().join(format!("lc-disk-alert-switch-{}", std::process::id())); + let _ = std::fs::remove_dir_all(&root); + let (a, b) = (root.join("a"), root.join("b")); + std::fs::create_dir_all(&a).unwrap(); + std::fs::create_dir_all(&b).unwrap(); + let (release, wait) = mpsc::channel::<()>(); + let (started_tx, started) = mpsc::channel::(); + let wait = Mutex::new(wait); + let started_tx = Mutex::new(started_tx); + let mut s = Session::new().with_fs(); + s.disk_alert.reader = Some(Arc::new(move |dir: &std::path::Path| { + let _ = started_tx.lock().unwrap().send(dir.to_path_buf()); + let _ = wait.lock().unwrap().recv_timeout(Duration::from_secs(10)); + Some(DiskSpace { available: GB, total: 500 * GB }) + })); + s.open_library(&a, false).unwrap(); + let first = started.recv_timeout(Duration::from_secs(10)).expect("the open reads the drive"); + assert!(first.starts_with(&a), "{first:?}"); + s.close_library().unwrap(); + s.open_library(&b, false).unwrap(); + s.check_disk_space(true); + assert!(started.recv_timeout(Duration::from_millis(300)).is_err(), "no second read beside the hanging one"); + release.send(()).unwrap(); + let until = Instant::now() + Duration::from_secs(10); + let second = loop { + assert!(s.take_disk_space_alert().is_none(), "the old library's answer is dropped"); + if let Ok(dir) = started.try_recv() { + break dir; + } + assert!(Instant::now() < until, "the current library's drive was never read"); + std::thread::sleep(Duration::from_millis(5)); + }; + assert!(second.starts_with(&b), "{second:?}"); + assert!(!s.disk_space_alert_raised()); + release.send(()).unwrap(); + let until = Instant::now() + Duration::from_secs(10); + while s.take_disk_space_alert().is_none() { + assert!(Instant::now() < until, "the current library's answer was never taken"); + std::thread::sleep(Duration::from_millis(5)); + } + s.close_library().unwrap(); + let _ = std::fs::remove_dir_all(&root); + } + + /// When the drive is low, the preview cache is counted on the read's worker too: counting it can + /// mean reading the size of every file (a big cache on a slow share), never on the UI thread + /// that takes the answer. + #[cfg(not(target_arch = "wasm32"))] + #[test] + fn the_preview_cache_is_counted_off_the_ui_thread() { + use std::sync::{Arc, Mutex, mpsc}; + use std::time::{Duration, Instant}; + let dir = std::env::temp_dir().join(format!("lc-disk-alert-count-{}", std::process::id())); + let _ = std::fs::remove_dir_all(&dir); + std::fs::create_dir_all(&dir).unwrap(); + let (release, wait) = mpsc::channel::<()>(); + let wait = Mutex::new(wait); + let mut s = Session::new().with_fs(); + s.disk_alert.reader = Some(Arc::new(move |_: &std::path::Path| { + let _ = wait.lock().unwrap().recv_timeout(Duration::from_secs(10)); + Some(DiskSpace { available: GB, total: 500 * GB }) + })); + s.open_library(&dir, false).unwrap(); + let disk = || s.media.rendered.disk().map(lightcraft_preview::DiskCache::known_size); + assert_eq!(disk(), Some(None), "an on-disk library has a preview cache, not counted yet"); + release.send(()).unwrap(); + // no poll here: only the worker can count the cache + let until = Instant::now() + Duration::from_secs(10); + let counted = loop { + if let Some(Some(bytes)) = disk() { + break bytes; + } + assert!(Instant::now() < until, "the worker never counted the cache"); + std::thread::sleep(Duration::from_millis(5)); + }; + let until = Instant::now() + Duration::from_secs(10); + let alert = loop { + if let Some(a) = s.take_disk_space_alert() { + break a; + } + assert!(Instant::now() < until, "the drive's answer was never taken"); + std::thread::sleep(Duration::from_millis(5)); + }; + assert_eq!(alert.cache_bytes, counted); + s.close_library().unwrap(); + let _ = std::fs::remove_dir_all(&dir); + } + + #[cfg(not(target_arch = "wasm32"))] + #[test] + fn the_free_space_of_a_real_drive_is_read() { + let space = super::read(&std::env::temp_dir().join("lc-disk-alert-not-there")).expect("the temp folder's drive says its size"); + assert!(space.total > 0 && space.available <= space.total, "{space:?}"); + } + + /// A read whose thread can't start is tried again after [`super::RETRY_AFTER`], not after the five minutes + /// of [`super::CHECK_EVERY`] (CodeRabbit on #3577); for either of its two threads. + #[cfg(not(target_arch = "wasm32"))] + #[test] + fn a_read_that_could_not_start_is_tried_again_soon() { + use std::time::{Duration, Instant}; + for thread in ["disk-space", "disk-space-read"] { + let dir = std::env::temp_dir().join(format!("lc-disk-alert-nothread-{thread}-{}", std::process::id())); + let _ = std::fs::remove_dir_all(&dir); + std::fs::create_dir_all(&dir).unwrap(); + let mut s = Session::new().with_fs(); + s.disk_alert.reader = Some(std::sync::Arc::new(|_: &std::path::Path| Some(DiskSpace { available: GB, total: 500 * GB }))); + s.disk_alert.fail_spawn = Some(thread); + s.open_library(&dir, false).unwrap(); + let until = Instant::now() + Duration::from_secs(10); + while s.disk_space_reading() { + assert!(Instant::now() < until, "{thread}: the read never ended"); + s.poll_disk_space(); + std::thread::sleep(Duration::from_millis(5)); + } + let since = s.disk_alert.checked_at.expect("a check was made").elapsed(); + assert!( + since + super::RETRY_AFTER + Duration::from_secs(1) >= super::CHECK_EVERY, + "{thread}: next read in {:?}", + super::CHECK_EVERY.saturating_sub(since) + ); + // the thread can start again: the retry reads the drive and raises the alert + s.disk_alert.fail_spawn = None; + s.disk_alert.checked_at = s.disk_alert.checked_at.and_then(|t| t.checked_sub(super::RETRY_AFTER)); + s.check_disk_space(false); + let until = Instant::now() + Duration::from_secs(10); + while s.take_disk_space_alert().is_none() { + assert!(Instant::now() < until, "{thread}: the retry never read the drive"); + std::thread::sleep(Duration::from_millis(5)); + } + s.close_library().unwrap(); + let _ = std::fs::remove_dir_all(&dir); + } + } + + #[test] + fn a_drive_that_reports_no_size_is_judged_by_its_free_bytes() { + assert!(!DiskSpace { available: 50 * GB, total: 0 }.is_low()); + assert!(DiskSpace { available: 5 * GB, total: 0 }.is_low()); + } +} diff --git a/crates/engine/src/files/fs.rs b/crates/engine/src/files/fs.rs index 05784d542b..ce8347d701 100644 --- a/crates/engine/src/files/fs.rs +++ b/crates/engine/src/files/fs.rs @@ -37,6 +37,12 @@ pub(super) fn raf_preview_extent(prefix: &[u8]) -> Option { /// `adobe_rgb`: the container declares the preview Adobe RGB ([`PreviewRead::adobe_rgb`]), applied /// as [`decode_raw_preview`] applies it. `None` when the bytes are not a decodable image. pub(super) fn quick_image_srgb(header: &[u8], window: &[u8], adobe_rgb: bool, max_edge: usize) -> Option { + // a plain sRGB camera JPEG (nearly all of them) straight to 8-bit (PERF-CULL-3: 45 MP preview + // of an EOS R5 C fitted to 2560 px, 194 → ~110 ms); anything else takes the general decode + let edge = u32::try_from(max_edge).unwrap_or(u32::MAX); + if !adobe_rgb && let Ok(Some((image, orientation))) = lightcraft_codecs::decode_jpeg_srgb8_fit(window, edge, edge) { + return Some(image.oriented(preview_orientation(header, orientation))); + } let decoded = if adobe_rgb && window.starts_with(&[0xff, 0xd8]) { lightcraft_codecs::decode_jpeg_with_fallback(window, fit_box(max_edge), lightcraft_codecs::NamedSpace::AdobeRgb) } else { diff --git a/crates/engine/src/files/preview.rs b/crates/engine/src/files/preview.rs index 825167f5c1..85ee2c5fb8 100644 --- a/crates/engine/src/files/preview.rs +++ b/crates/engine/src/files/preview.rs @@ -33,7 +33,11 @@ pub(crate) fn decode_raw_preview( opts: lightcraft_codecs::DecodeOptions, covering: Option, ) -> Option { - let jpeg = stand_in_image(bytes, covering)?; + decode_stand_in(bytes, &stand_in_image(bytes, covering)?, opts) +} + +/// [`decode_raw_preview`] of the stand-in `jpeg` already found in the raw file `bytes` (by [`stand_in_image`]). +fn decode_stand_in(bytes: &[u8], jpeg: &[u8], opts: lightcraft_codecs::DecodeOptions) -> Option { // the colour space is a property of the JPEG; a TIFF stand-in (the file itself, or a Hasselblad RGB preview // wrapped as a TIFF) and a JPEG XL preview are read by their own decoders let is_jpeg = jpeg.starts_with(&[0xff, 0xd8]); @@ -43,9 +47,9 @@ pub(crate) fn decode_raw_preview( lightcraft_raw::PreviewColorSpace::Srgb => lightcraft_codecs::NamedSpace::Srgb, lightcraft_raw::PreviewColorSpace::AdobeRgb => lightcraft_codecs::NamedSpace::AdobeRgb, }; - lightcraft_codecs::decode_jpeg_with_fallback(&jpeg, opts, space).ok() + lightcraft_codecs::decode_jpeg_with_fallback(jpeg, opts, space).ok() } - None => lightcraft_codecs::decode(&jpeg, opts).ok(), + None => lightcraft_codecs::decode(jpeg, opts).ok(), } } @@ -80,7 +84,18 @@ pub fn load_embedded_preview(bytes: &[u8], max_edge: usize) -> Option<(Rgb32f, S /// loupe and grid show it until the raw itself has been developed ([`crate::media::QuickJob`]). The smallest /// embedded JPEG that covers `max_edge` is decoded, not the largest. pub fn embedded_preview_srgb(bytes: &[u8], max_edge: usize) -> Option { - let mut d = decode_raw_preview(bytes, fit_box(max_edge), Some(max_edge))?; + // a plain sRGB camera JPEG straight to 8-bit, as the quick path decodes it (PERF-CULL-3), so + // both paths give the same picture; anything else takes the general decode + let edge = u32::try_from(max_edge).unwrap_or(u32::MAX); + // (found once: the general decode below takes the same stand-in when the fast path declines) + let jpeg = stand_in_image(bytes, Some(max_edge))?; + if lightcraft_raw::embedded_preview_color_space(bytes) != Some(lightcraft_raw::PreviewColorSpace::AdobeRgb) + && matches!(jpeg, std::borrow::Cow::Owned(_)) + && let Ok(Some((image, orientation))) = lightcraft_codecs::decode_jpeg_srgb8_fit(&jpeg, edge, edge) + { + return Some(image.oriented(preview_orientation(bytes, orientation))); + } + let mut d = decode_stand_in(bytes, &jpeg, fit_box(max_edge))?; if d.image.width.max(d.image.height) > max_edge { d.image = fit(&d.image, max_edge, max_edge, Filter::Box); d.alpha = None; diff --git a/crates/engine/src/import.rs b/crates/engine/src/import.rs index 6a6b6e6a70..b4ee766048 100644 --- a/crates/engine/src/import.rs +++ b/crates/engine/src/import.rs @@ -53,7 +53,9 @@ pub(crate) use files::{expand_noting, probe_all, probe_paths, probe_with}; use job::PreparedItem; pub use job::{ImportJob, Prepared, candidate_thumb_key, import, import_with}; pub(crate) use scan::scan_files; -pub use scan::{ScanInput, ScanOutput, ScanProgress, scan, scan_with}; +pub use scan::{ + ScanInput, ScanOutput, ScanProgress, browse_files, browse_order, browse_path, browse_queue, browse_queue_until, browse_spelling, scan, scan_with, +}; #[cfg(test)] use scan::{library_files, library_hashes}; pub use types::{ diff --git a/crates/engine/src/import/job.rs b/crates/engine/src/import/job.rs index ae635cb10d..e8ed759dd2 100644 --- a/crates/engine/src/import/job.rs +++ b/crates/engine/src/import/job.rs @@ -528,6 +528,8 @@ impl Session { key, cached: Vec::new(), embedded, + embedded_cache: None, + embedded_on_disk: false, minimum_embedded_edge, small: Some(Box::new(small)), }) diff --git a/crates/engine/src/import/scan.rs b/crates/engine/src/import/scan.rs index 57ee995f26..ad247c22c6 100644 --- a/crates/engine/src/import/scan.rs +++ b/crates/engine/src/import/scan.rs @@ -1,4 +1,4 @@ -use super::files::{Seen, expand, probe_all, reusable_probes}; +use super::files::{Seen, expand, is_supported, probe_all, reusable_probes}; use super::types::ImportCandidate; use super::{HashMap, Path, PathBuf, PhotoId, ProbeInfo, Session, Source}; @@ -98,6 +98,185 @@ impl ScanInput { pub(crate) fn library(&self) -> std::sync::Arc { std::sync::Arc::clone(&self.library) } + + /// The probes of earlier scans this input took from the session ([`ScanInput::new`]), to hand back with a + /// scan's output that probes nothing itself (browsing), so a later import doesn't read those files again. + pub fn take_probes(&mut self) -> HashMap { + std::mem::take(&mut self.cache) + } +} + +/// What browsing a folder (Local) has to read: the files of `files` the library has no photo at, +/// in the order the view will show them (see [`browse_order`]). Unlike [`scan_with`] nothing is +/// read, so the import that follows adds the photos batch by batch as it reads them, and the first +/// screen of a 3,000-raw wedding shows within a second instead of after every file was read +/// (measured on 2,933 EOS R5 C CR3s: 30.6 s to the first photo before). +pub fn browse_queue(input: &ScanInput, files: Vec, sort: &lightcraft_catalog::Sort) -> Vec { + browse_queue_until(input, files, sort, &std::sync::atomic::AtomicBool::new(false)).unwrap_or_default() +} + +/// [`browse_queue`] that stops reading the files' times or sizes once `cancel` is set (another folder was +/// chosen): `None` then. +pub fn browse_queue_until( + input: &ScanInput, + files: Vec, + sort: &lightcraft_catalog::Sort, + cancel: &std::sync::atomic::AtomicBool, +) -> Option> { + let known = library_files(&input.library, &files); + let mut files: Vec = files.into_iter().filter(|f| !known.contains_key(Path::new(f))).collect(); + browse_order_until(&mut files, sort, cancel).then_some(files) +} + +/// The files browsing `root` (Local) lists: the supported, non-hidden files in it, or below it +/// with `subfolders`, never inside the library's own folder (as [`scan_with`] skips it). +/// +/// A folder that can't be listed whole (gone, no access, a network share dropping out) is an +/// error, not an empty or partial listing: the browse bookkeeping would take the files missing +/// from it for files gone. (With `subfolders` the walk skips the folders below it that it can't +/// read, as the import's scan does.) +pub fn browse_files(input: &ScanInput, root: &str, subfolders: bool) -> std::io::Result> { + // one spelling for the folder, and so for its files: `lib/../outside` lists `outside`'s files as + // the catalog stores them, not as new photos under another name + let root_path = browse_path(Path::new(root))?; + let root = root_path.to_string_lossy().to_string(); + // the library's folder itself, or a folder inside it: nothing to browse + if input.skip.as_deref().is_some_and(|lib| inside(&root_path, lib)) { + return Ok(Vec::new()); + } + if subfolders { + // (the root itself must be there to be walked) + std::fs::read_dir(&root)?; + return Ok(expand(&[root], input.skip.as_deref())); + } + let mut v = Vec::new(); + for entry in std::fs::read_dir(&root)? { + let entry = entry?; + if crate::walk::is_hidden_entry(&entry) { + continue; + } + let f = entry.path(); + if f.is_file() && is_supported(&f) { + v.push(f.to_string_lossy().to_string()); + } + } + v.sort(); + Ok(v) +} + +/// The folder `path` names, as one spelling for the view, its listing and the catalog's records: +/// absolute (`.` is the current folder), with `.` and `..` taken out the way the system reads them. +/// Windows takes `..` out by name; elsewhere `link/..` is the folder above where the link points, +/// so a link before a `..` is resolved first. Everything else keeps its spelling (a mapped drive +/// stays a drive letter, a link not followed by `..` stays the link), as the records were made. +/// +/// It may read the disk (a link on a stalled network mount can block it): call it off the UI thread. +/// [`browse_spelling`] is the UI's spelling meanwhile. +pub fn browse_path(path: &Path) -> std::io::Result { + normalized(path, true) +} + +/// [`browse_path`] without reading the disk: `.` and `..` taken out by name. What the view shows +/// until the worker has resolved the folder; the same folder unless a link comes before a `..` +/// (not on Windows, which takes `..` out by name too). +pub fn browse_spelling(path: &Path) -> std::io::Result { + normalized(path, false) +} + +fn normalized(path: &Path, resolve_links: bool) -> std::io::Result { + use std::path::Component; + // (no disk access: the current folder is the process's) + let path = std::path::absolute(path)?; + let mut out = PathBuf::new(); + for c in path.components() { + match c { + Component::CurDir => {} + Component::ParentDir => { + // a link before `..` is left through where it points; a broken one is an error, not + // its parent folder by name (CodeRabbit on #3577) + #[cfg(not(windows))] + if resolve_links && std::fs::symlink_metadata(&out).is_ok_and(|m| m.file_type().is_symlink()) { + out = std::fs::canonicalize(&out)?; + } + #[cfg(windows)] + let _ = resolve_links; + // (above the root: still the root) + if matches!(out.components().next_back(), Some(Component::Normal(_))) { + out.pop(); + } + } + c => out.push(c.as_os_str()), + } + } + Ok(out) +} + +/// Whether `root` (a [`browse_path`]) is `lib` or a folder inside it: as written, or once both are +/// resolved (a symlink to the library names the same folder), as the recursive walk tells. A root +/// that can't be resolved (gone, or a network share that won't say) is compared as written when it +/// is there; one that isn't there is not inside, and listing it reports why. +fn inside(root: &Path, lib: &Path) -> bool { + let as_written = || root.starts_with(browse_path(lib).unwrap_or_else(|_| lib.to_path_buf())); + match std::fs::canonicalize(root) { + Ok(resolved) => as_written() || std::fs::canonicalize(lib).is_ok_and(|lib| resolved.starts_with(lib)), + Err(_) => root.exists() && as_written(), + } +} + +/// Order `files` the way a view sorted by `sort` will show them, as far as the file system can +/// tell before the files are read: by modification time for the capture date (a camera's file +/// time is its capture time, near enough to fill the first screen first), by name, or by size. +/// New photos share one import time and have no edit time, so the import- and edit-date sorts +/// show them by name. Ties go by name in the view's direction, as the catalog breaks them +/// ([`lightcraft_catalog::Sort::order`]); new photos tie on rating too. Random keeps the given order. Files whose +/// time or size can't be read go last. +pub fn browse_order(files: &mut [String], sort: &lightcraft_catalog::Sort) { + browse_order_until(files, sort, &std::sync::atomic::AtomicBool::new(false)); +} + +/// [`browse_order`] that stops reading the files' times or sizes once `cancel` is set: `false` then, and +/// `files` is left as it was. +fn browse_order_until(files: &mut [String], sort: &lightcraft_catalog::Sort, cancel: &std::sync::atomic::AtomicBool) -> bool { + use lightcraft_catalog::SortKey; + use lightcraft_catalog::query::cmp_name; + use std::cmp::Ordering; + use std::sync::atomic::Ordering::Relaxed; + let asc = sort.ascending; + let name = |f: &str| Path::new(f).file_name().map(|n| n.to_string_lossy().into_owned()).unwrap_or_default(); + let directed = |o: Ordering| if asc { o } else { o.reverse() }; + // `key` per file (`None`: unreadable, last in either direction), then the name, in the view's direction + let by_key = |files: &mut [String], key: &dyn Fn(&str) -> Option| { + let mut keyed: Vec<(Option, String, String)> = Vec::with_capacity(files.len()); + for f in files.iter() { + if cancel.load(Relaxed) { + return false; + } + keyed.push((key(f), name(f), f.clone())); + } + keyed.sort_by(|a, b| match (a.0, b.0) { + (Some(x), Some(y)) => directed(x.cmp(&y).then_with(|| cmp_name(&a.1, &b.1))), + (Some(_), None) => Ordering::Less, + (None, Some(_)) => Ordering::Greater, + (None, None) => directed(cmp_name(&a.1, &b.1)), + }); + for (slot, (_, _, f)) in files.iter_mut().zip(keyed) { + *slot = f; + } + true + }; + match sort.key { + SortKey::CaptureDate => by_key(files, &|f| { + let t = std::fs::metadata(f).and_then(|m| m.modified()).ok()?; + // (nanoseconds since the epoch fit a u64 until 2554; earlier times count as the epoch) + Some(t.duration_since(std::time::UNIX_EPOCH).map_or(0, |d| u64::try_from(d.as_nanos()).unwrap_or(u64::MAX))) + }), + SortKey::ImportDate | SortKey::EditDate | SortKey::Rating | SortKey::FileName => { + files.sort_by(|a, b| directed(cmp_name(&name(a), &name(b)))); + !cancel.load(Relaxed) + } + SortKey::FileSize => by_key(files, &|f| std::fs::metadata(f).ok().map(|m| m.len())), + SortKey::Random => !cancel.load(Relaxed), + } } /// Find and probe what importing `paths` would add, marking duplicates (by path, or by content diff --git a/crates/engine/src/lib.rs b/crates/engine/src/lib.rs index f71f9becd1..bade3892d3 100644 --- a/crates/engine/src/lib.rs +++ b/crates/engine/src/lib.rs @@ -28,6 +28,7 @@ pub mod crs_masks; pub mod demo; pub mod denoise; pub mod devices; +pub mod disk_space; pub mod display; pub mod doc_path; mod errors; @@ -61,6 +62,7 @@ mod model_download; pub mod open_path; pub mod originals; mod pdf; +pub mod prep; pub mod preset_import; pub mod preset_luminar; pub mod presets; @@ -398,10 +400,18 @@ pub struct Session { pub slideshow_export: Option>, /// The last (or running) Build Previews. pub preview_build: Option>, + /// Preparing photos in the background: settings (app-wide and this library's) and the build + /// it runs ([`prep`]). + pub prep: prep::PrepState, + /// The long edge (px) the loupe renders at on this screen, as the frontend last said + /// (0: not known). Standard previews are built at least this big, so the loupe can use them. + pub loupe_edge: usize, /// Develop defaults applied on import (persisted in prefs.json). pub import_defaults: import::ImportDefaults, /// Disk budget of the library's thumbnail cache in MB (0 = default; persisted in prefs.json). pub cache_mb: u32, + /// The once-per-session alert for a drive running low under the preview cache ([`disk_space`]). + pub(crate) disk_alert: disk_space::DiskAlertState, /// Slideshow playback settings (`slideshow.playback`; persisted in prefs.json). pub slideshow: slideshow::PlaybackSettings, /// Slideshow slide layout: Layout, Options and Backdrop (`slideshow.layout`; persisted in @@ -537,9 +547,12 @@ impl Session { folder_changes: None, activity: Default::default(), preview_build: None, + prep: Default::default(), + loupe_edge: 0, slideshow_export: None, import_defaults: import::ImportDefaults::default(), cache_mb: 0, + disk_alert: Default::default(), slideshow: slideshow::PlaybackSettings::default(), slide_layout: slide::SlideSettings::default(), slide_overlays: slide_overlay::OverlaySettings::default(), diff --git a/crates/engine/src/library.rs b/crates/engine/src/library.rs index 23e333c4ca..8cd242face 100644 --- a/crates/engine/src/library.rs +++ b/crates/engine/src/library.rs @@ -258,6 +258,13 @@ impl Library { } } +/// [`PrefsFile`]'s `prep`, read leniently (see there). +fn lenient_prep<'de, D: serde::Deserializer<'de>>(d: D) -> std::result::Result, D::Error> { + use serde::Deserialize; + let value = serde_json::Value::deserialize(d)?; + Ok(serde_json::from_value(value).ok()) +} + #[derive(Default, Serialize, Deserialize)] #[serde(default, rename_all = "camelCase")] struct PrefsFile { @@ -295,6 +302,11 @@ struct PrefsFile { import: crate::import::ImportDefaults, /// Thumbnail disk cache budget (MB, 0 = default). cache_mb: u32, + /// This library's own background preparation settings (`None`: the app-wide ones). Read + /// leniently: a value a newer build wrote that this one doesn't know is `None`, and the rest of + /// the file is kept. + #[serde(skip_serializing_if = "Option::is_none", deserialize_with = "lenient_prep")] + prep: Option, /// Folder for smart previews (default: `Smart Previews` in the library). smart_previews_dir: Option, /// Days after which untouched Local records of unbrowsed folders are forgotten (missing = @@ -585,6 +597,7 @@ impl Session { self.recent_keywords = prefs.recent_keywords; self.import_defaults = prefs.import; self.cache_mb = prefs.cache_mb; + self.prep.library = prefs.prep.map(crate::prep::PrepSettings::sanitized); self.slideshow = prefs.slideshow.sanitized(); self.slide_layout = prefs.slide_layout.sanitized(); self.slide_overlays = prefs.slide_overlays.sanitized(); @@ -688,6 +701,9 @@ impl Session { crate::cmd::previews::refresh_stale_smart_previews(self); } self.library_identity = std::sync::Arc::new(()); + // a library open is a new session for the low-disk alert: look at the drive now + self.disk_alert.reset(); + self.check_disk_space(true); Ok(()) } @@ -900,6 +916,7 @@ impl Session { recent_keywords: self.recent_keywords.clone(), import: self.import_defaults.clone(), cache_mb: self.cache_mb, + prep: self.prep.library, smart_previews_dir: self.smart_previews_dir.as_ref().map(|d| d.to_string_lossy().to_string()), forget_local_days: Some(self.forget_local_days), slideshow: self.slideshow.clone(), @@ -927,9 +944,23 @@ impl Session { new } - /// The thumbnail disk cache budget in bytes ([`Session::cache_mb`], else the default). + /// The thumbnail disk cache budget in bytes ([`Session::cache_mb`], else the default, which + /// grows with the library: [`crate::media::default_disk_cache_bytes`]). pub fn cache_bytes(&self) -> u64 { - if self.cache_mb == 0 { crate::media::DISK_CACHE_BYTES } else { u64::from(self.cache_mb) << 20 } + if self.cache_mb == 0 { crate::media::default_disk_cache_bytes(self.catalog.len()) } else { u64::from(self.cache_mb) << 20 } + } + + /// Bring the disk cache's budget up to date with the library (photos were added since it was + /// set): cheap, nothing is read or written. Called before previews are built, so it also + /// looks at the drive's free space when that is due ([`Session::check_disk_space`]). + pub fn fit_disk_cache(&mut self) { + let want = self.cache_bytes(); + if let Some(d) = self.media.rendered.disk() + && d.budget() != want + { + d.set_budget(want); + } + self.check_disk_space(false); } /// Change the thumbnail disk cache budget (MB, 0 = default) in place: the cached thumbnails @@ -991,6 +1022,16 @@ fn unlocked_warning(lock: Option<&LibraryLock>) -> Option { mod resolve_tests { use super::*; + /// A library's background preparation value this build doesn't know reads as none (the + /// app-wide settings apply) and keeps the rest of `prefs.json`. + #[test] + fn an_unknown_background_preparation_value_keeps_the_other_prefs() { + let v = serde_json::json!({"cacheMb": 77, "prep": {"culling": {"scope": "someday"}}}); + let read: PrefsFile = serde_json::from_value(v).unwrap(); + assert!(read.prep.is_none()); + assert_eq!(read.cache_mb, 77); + } + /// `--library /catalog.snap` failed with the OS's "File exists" (creating a folder over /// the file); it opens the library that file belongs to. #[test] diff --git a/crates/engine/src/media.rs b/crates/engine/src/media.rs index a4528795b1..9c6de8dd2c 100644 --- a/crates/engine/src/media.rs +++ b/crates/engine/src/media.rs @@ -35,14 +35,17 @@ mod settings_hash; mod source; pub use cache::DenoiseIndex; +#[cfg(test)] +pub(crate) use cache::rendered_budget; pub use quick::{QuickJob, QuickSource, content_key, source_info}; -pub use render::{RenderJob, RenderResult, ScreenFrame, develop, develop_reporting, fit_for_display}; +pub use render::{RenderJob, RenderResult, ScreenFrame, develop, develop_reporting, fit_cached, fit_for_display}; #[cfg(test)] use settings_hash::SETTINGS_HASH_ENTRIES; use settings_hash::SettingsHashes; pub use source::{ - DISK_CACHE_BYTES, DecodedSource, DenoiseSpec, FACE_EDGE_STEP, FileLoader, FileProbe, MakeProduct, PairLoader, PreviewLoader, ProbeInfo, - RENDER_CACHE_VERSION, ReducedLoader, SourceLevel, SourceRef, THUMB_SIZES, Twin, face_edge, thumb_bucket, + DISK_CACHE_BYTES, DISK_CACHE_PER_PHOTO, DecodedSource, DenoiseSpec, FACE_EDGE_STEP, FileLoader, FileProbe, MakeProduct, PairLoader, + PreviewLoader, ProbeInfo, RENDER_CACHE_VERSION, ReducedLoader, SourceLevel, SourceRef, THUMB_SIZES, Twin, default_disk_cache_bytes, face_edge, + thumb_bucket, }; pub struct MediaCache { diff --git a/crates/engine/src/media/cache.rs b/crates/engine/src/media/cache.rs index 234efb1485..036b4730a4 100644 --- a/crates/engine/src/media/cache.rs +++ b/crates/engine/src/media/cache.rs @@ -75,9 +75,11 @@ impl Default for MediaCache { } } -/// Memory for rendered previews out of the cache share. -fn rendered_budget(share: usize) -> usize { - RENDERED_MEM_BYTES.min(share / 4) +/// Memory for rendered previews out of the cache share: a quarter of it, so a big budget keeps +/// a whole shoot's previews (a 20 GB budget holds about 230 standard previews, or 1 500 loupe +/// renders at 1 200 px), and at least [`RENDERED_MEM_BYTES`] where the share allows. +pub(crate) fn rendered_budget(share: usize) -> usize { + (share / 4).max(RENDERED_MEM_BYTES.min(share / 2)) } impl MediaCache { diff --git a/crates/engine/src/media/quick.rs b/crates/engine/src/media/quick.rs index 335a118ca6..25c93e9f81 100644 --- a/crates/engine/src/media/quick.rs +++ b/crates/engine/src/media/quick.rs @@ -25,6 +25,15 @@ pub struct QuickJob { pub cached: Vec<(Arc, Hash128)>, /// Embedded preview of an unedited raw: (path, loader, max edge). pub embedded: Option<(String, PreviewLoader, usize)>, + /// Where a decoded [`Self::embedded`] preview is kept in memory, so stepping back to a photo + /// shows its camera JPEG again without reading and decoding it (culling goes back and forth): + /// the cache (memory only, never its disk), the key for this photo and edge, and the cache's + /// generation when the job was made (a Discard Preview Cache since then keeps the picture out). + pub embedded_cache: Option<(Arc, Hash128, u64)>, + /// Whether [`Self::embedded_cache`] keeps the decoded preview on disk too: a grid thumbnail, + /// so a folder seen before fills in from small files at once instead of every raw being read + /// and its camera JPEG decoded again (the loupe's large stand-ins stay in memory). + pub embedded_on_disk: bool, /// Import tiles need this many pixels when a render can provide them. `None` accepts any /// camera JPEG immediately (ordinary transient stand-ins and preview-only imports). /// An undersized JPEG remains the fallback if the render fails or is unavailable. @@ -79,13 +88,30 @@ impl QuickJob { } } let mut embedded_fallback = None; - if let Some((path, loader, edge)) = &self.embedded - && let Some(image) = loader(path, *edge) - { - if self.minimum_embedded_edge.is_none_or(|minimum| image.width.max(image.height) >= minimum) { - return done(rendered(image), Some(QuickSource::Embedded)); + if let Some((path, loader, edge)) = &self.embedded { + let kept = + self.embedded_cache.as_ref().and_then(|(cache, k, _)| if self.embedded_on_disk { cache.get(*k) } else { cache.get_memory(*k) }); + let image = match kept { + Some(img) => Some(Arc::unwrap_or_clone(img)), + None => { + let image = loader(path, *edge); + // written for the generation the job was made in: a cache cleared meanwhile stays clear + if let (Some(image), Some((cache, k, generation))) = (&image, &self.embedded_cache) { + if self.embedded_on_disk { + cache.put_deferred_at(*generation, *k, Arc::new(image.clone())); + } else { + cache.put_memory_at(*generation, *k, Arc::new(image.clone())); + } + } + image + } + }; + if let Some(image) = image { + if self.minimum_embedded_edge.is_none_or(|minimum| image.width.max(image.height) >= minimum) { + return done(rendered(image), Some(QuickSource::Embedded)); + } + embedded_fallback = Some(image); } - embedded_fallback = Some(image); } if let Some(job) = self.small { let r = job.run(); @@ -100,6 +126,11 @@ impl QuickJob { } } +/// The thumbnail cache's key for the camera JPEG standing for the thumbnail of cache key `k`. +pub(super) fn embedded_thumb_key(k: Hash128) -> Hash128 { + Hasher128::new().str("embedded-thumb").u64((k.0 >> 64) as u64).u64(k.0 as u64).finish() +} + /// What identifies a photo's pixels for caching: its content hash, else its source. /// Which decode of a photo a loaded source is: its file's content and, when it has one, its database lens profile /// ([`lens_key`]; the decode fits the starting colour through it). A job's source is kept only when this still diff --git a/crates/engine/src/media/render.rs b/crates/engine/src/media/render.rs index 56dc3b7cb9..bd8ee5292f 100644 --- a/crates/engine/src/media/render.rs +++ b/crates/engine/src/media/render.rs @@ -1,5 +1,5 @@ use super::quick::QuickSource; -use super::source::{DecodedSource, SourceLevel, SourceRef}; +use super::source::{DecodedSource, SourceLevel, SourceRef, VIEW_REUSE_SLACK}; use super::{ Arc, DevelopSettings, Hash128, Histogram, PhotoId, PreviewCache, Quality, RenderRequest, Rendered, Rgb32f, Rgba8, Source, SourceInfo, StageCache, }; @@ -241,8 +241,58 @@ impl RenderJob { r } + /// The view render kept for this job's photo and settings ([`Self::view_cache`]), fitted to the + /// size asked for, when it is at least that big, or unchanged when it covers the job's display + /// fit (else `None`: render). Only for what the cached + /// view is: a full-quality, 8-bit, whole-frame SDR render (the cache holds sRGB; a display + /// profile is applied to it like to any cached picture, [`crate::display::present`]). + pub fn cached_view(&self) -> Option { + let (cache, key) = self.view_cache.as_ref()?; + if self.request.quality != Quality::Full + || self.request.depth != lightcraft_pipeline::OutputDepth::U8 + || self.request.window.is_some() + || self.settings.hdr.enabled + || self.slide.is_some() + { + return None; + } + let img = cache.get_at(self.cache_generation, *key)?; + fit_cached(&img, self.request.max_w, self.request.max_h).or_else(|| { + // the loupe at rest asks for at least the preview level and shows the render shrunk to + // the drawn size (the display fit): a cached view that covers the drawn size is enough + let (fw, fh) = self.display_fit?; + let (w, h) = (img.width, img.height); + let s = (fw as f64 / w.max(1) as f64).min(fh as f64 / h.max(1) as f64); + (w > 0 && h > 0 && fw > 0 && fh > 0 && s <= 1.0 + VIEW_REUSE_SLACK).then(|| Arc::unwrap_or_clone(img)) + }) + } + /// The photo's render (without slide composition). fn run_photo(self) -> RenderResult { + // the loupe: a cached view render of these settings at least this big is the render (built + // previews and the photos prepared ahead are used, not rendered again; PERF-PREVIEW-REUSE) + if let Some(mut image) = self.cached_view() { + // as for a render: the histogram of the full picture, then the display fit + let histogram = Histogram::of_srgb8(&image); + if let Some((fw, fh)) = self.display_fit + && (image.width > fw.max(1) || image.height > fh.max(1)) + { + image = fit_for_display(&image, fw, fh); + } + return RenderResult { + request_id: self.request_id, + source_key: self.source_key, + photo: self.photo, + level: self.level, + key: self.key, + rendered: Ok(Rendered { image, histogram, deep: None }), + gpu_fallback: None, + loaded: None, + quick: Some(QuickSource::Cached), + display: None, + frame: None, + }; + } if let Some((cache, key)) = &self.cache && let Some(img) = cache.get_at(self.cache_generation, *key) { @@ -395,6 +445,61 @@ impl RenderJob { } } +/// `img` fitted into `max_w × max_h` (its aspect kept), when it is big enough to need no +/// enlarging (within [`VIEW_REUSE_SLACK`]): as it is, or shrunk by averaging. +pub fn fit_cached(img: &Rgba8, max_w: usize, max_h: usize) -> Option { + let (w, h) = (img.width, img.height); + if w == 0 || h == 0 || max_w == 0 || max_h == 0 { + return None; + } + let s = (max_w as f64 / w as f64).min(max_h as f64 / h as f64); + if !s.is_finite() || s > 1.0 + VIEW_REUSE_SLACK { + return None; + } + let (nw, nh) = (((w as f64 * s).round() as usize).clamp(1, w), ((h as f64 * s).round() as usize).clamp(1, h)); + if nw == w && nh == h { + return Some(img.clone()); + } + Some(shrink_rgba(img, nw, nh)) +} + +/// `img` shrunk to `nw × nh` (no larger than it), each pixel the average of the ones it covers. +fn shrink_rgba(img: &Rgba8, nw: usize, nh: usize) -> Rgba8 { + let (w, h) = (img.width, img.height); + let (nw, nh) = (nw.clamp(1, w.max(1)), nh.clamp(1, h.max(1))); + // the source columns / rows each output column / row covers: [start, end) + let spans = |n: usize, of: usize| -> Vec<(usize, usize)> { + (0..n) + .map(|i| { + let a = i.saturating_mul(of) / n; + let b = (i.saturating_add(1).saturating_mul(of) / n).max(a.saturating_add(1)).min(of); + (a, b) + }) + .collect() + }; + let (xs, ys) = (spans(nw, w), spans(nh, h)); + let mut data = Vec::with_capacity(nw.saturating_mul(nh)); + let mut row = vec![[0u32; 4]; nw]; + for &(y0, y1) in &ys { + row.iter_mut().for_each(|r| *r = [0; 4]); + for y in y0..y1 { + let Some(line) = img.data.get(y * w..(y + 1) * w) else { continue }; + for (acc, &(x0, x1)) in row.iter_mut().zip(&xs) { + for px in line.get(x0..x1).unwrap_or(&[]) { + for (a, v) in acc.iter_mut().zip(px) { + *a += u32::from(*v); + } + } + } + } + for (acc, &(x0, x1)) in row.iter().zip(&xs) { + let n = (x1.saturating_sub(x0).saturating_mul(y1.saturating_sub(y0))).max(1) as u32; + data.push(acc.map(|a| ((a + n / 2) / n).min(255) as u8)); + } + } + Rgba8 { width: nw, height: nh, data } +} + /// Render `src`: on the GPU when `gpu` is set and a GPU is available (`lightcraft_gpu`), else on /// the CPU, reusing `stages` either way. Both produce the same image within 1–3 LSB (see /// `docs/gpu-pipeline.md`); `LIGHTCRAFT_GPU=0` or [`lightcraft_gpu::set_enabled`] forces the CPU. diff --git a/crates/engine/src/media/session.rs b/crates/engine/src/media/session.rs index ac1b8145a9..ed472972bd 100644 --- a/crates/engine/src/media/session.rs +++ b/crates/engine/src/media/session.rs @@ -1,4 +1,4 @@ -use super::quick::{QuickJob, content_key, source_info}; +use super::quick::{QuickJob, content_key, embedded_thumb_key, source_info}; use super::render::{RenderJob, RenderResult}; use super::settings_hash::with_process; use super::source::{PreviewLoader, RENDER_CACHE_VERSION, SourceLevel, SourceRef, THUMB_SIZES, face_edge, source_edge_needed, thumb_bucket}; @@ -264,12 +264,19 @@ impl crate::Session { Some(crate::face_tiles::PreviewCut { path, loader, want, min, rect, aspect, edge }) } - /// Size-independent cache key of a photo's view render (loupe) for its current settings, its lens profile - /// included as in [`Self::variant_key`]. `profile_salt`: [`MediaCache::camera_profile_salt`] of the photo's profile. - pub(crate) fn view_key(p: &Photo, apply_crop: bool, profile_salt: u64) -> Hash128 { + /// Size-independent cache key of a photo's view render (loupe) for its current settings. The + /// denoised picture, camera profile and lens profile in use are part of it (as of the render's own + /// key, [`Self::variant_key`]): a view rendered before the photo's AI Denoise result was there is + /// not shown as the denoised one. + pub(crate) fn view_key(&self, p: &Photo, apply_crop: bool) -> Hash128 { + let salt = self.media.denoise_salt(p.id, &p.develop); + let profile_salt = self.media.camera_profile_salt(&p.develop.profile.id); let mut k = Hasher128::new(); k.str(&content_key(p)).str("view").u64(p.develop.hash64()).u64(apply_crop as u64).u64(RENDER_CACHE_VERSION); with_process(&mut k, &p.develop).u64(crate::camera_profiles::cache_key()); + if salt != 0 { + k.u64(salt); + } if profile_salt != 0 { k.str("dcp").u64(profile_salt); } @@ -285,8 +292,7 @@ impl crate::Session { pub fn loupe_job(&mut self, id: PhotoId, max_w: usize, max_h: usize, apply_crop: bool) -> Option { let mut job = self.render_job(id, max_w, max_h, false, apply_crop)?; let p = self.catalog.photo(id)?; - let profile_salt = self.media.camera_profile_salt(&p.develop.profile.id); - job.view_cache = Some((self.media.rendered.clone(), Self::view_key(p, apply_crop, profile_salt))); + job.view_cache = Some((self.media.rendered.clone(), self.view_key(p, apply_crop))); Some(job) } @@ -396,11 +402,49 @@ impl crate::Session { let small = self.thumb_job(id, THUMB_SIZES[THUMB_SIZES.len() - 1])?; // (the thumbnail job itself starts with the cached thumbnail, after the sharper embedded preview) let profile_salt = self.media.camera_profile_salt(&p.develop.profile.id); - let cached = vec![(self.media.rendered.clone(), Self::view_key(&p, apply_crop, profile_salt))]; - let h = Hasher128::new().str(&content_key(&p)).str("quick").u64(p.develop.hash64()).u64(apply_crop as u64).u64(profile_salt).finish(); + let cached = vec![(self.media.rendered.clone(), self.view_key(&p, apply_crop))]; + // (the photo too: a virtual copy shares the file and, unedited, the settings) + let h = + Hasher128::new().str(&content_key(&p)).str("quick").u64(p.develop.hash64()).u64(apply_crop as u64).u64(id.0).u64(profile_salt).finish(); let key = h.0 as u64; let embedded = self.embedded_of(&p).map(|(path, l)| (path, l, max_edge.clamp(1, SourceLevel::Preview.max_edge()))); - Some(QuickJob { request_id: 0, photo: id, key, cached, embedded, minimum_embedded_edge: None, small: apply_crop.then(|| Box::new(small)) }) + let embedded_cache = embedded.as_ref().map(|(_, _, edge)| { + let cache = self.media.rendered.clone(); + let generation = cache.generation(); + (cache, Hasher128::new().str(&content_key(&p)).str("embedded").u64(*edge as u64).finish(), generation) + }); + Some(QuickJob { + request_id: 0, + photo: id, + key, + cached, + embedded, + embedded_cache, + embedded_on_disk: false, + minimum_embedded_edge: None, + small: apply_crop.then(|| Box::new(small)), + }) + } + + /// The camera's embedded JPEG of an unedited raw at up to `max_edge` pixels (its full size for + /// a camera that stores one), for the loupe zoomed to 100 % while LightCraft renders the window + /// on screen: a focus check needs the pixels at once, and a 45 MP render of the window takes + /// seconds. Not kept in any cache (a full-size picture is 180 MB). `None` for other photos. + pub fn embedded_full_job(&mut self, id: PhotoId, max_edge: usize) -> Option { + let p = self.catalog.photo(id)?.clone(); + let (path, l) = self.embedded_of(&p)?; + let key = Hasher128::new().str(&content_key(&p)).str("embedded-full").u64(id.0).u64(max_edge as u64).finish().0 as u64; + Some(QuickJob { + request_id: 0, + photo: id, + key, + cached: Vec::new(), + embedded: Some((path, l, max_edge.max(1))), + embedded_cache: None, + embedded_on_disk: false, + minimum_embedded_edge: None, + small: None, + }) } /// A first grid thumbnail for an unedited raw: the cached render for `job` (the real @@ -419,6 +463,10 @@ impl crate::Session { key: job.key, cached: job.cache.clone().into_iter().collect(), embedded: Some((path, l, edge)), + // the camera JPEG is this thumbnail (an unedited raw): kept with the rendered ones, under + // a key of its own, so the folder's next visit reads it from the thumbnail cache + embedded_cache: job.cache.as_ref().map(|(c, k)| (c.clone(), embedded_thumb_key(*k), c.generation())), + embedded_on_disk: true, minimum_embedded_edge: None, small: Some(Box::new(job.clone())), }) diff --git a/crates/engine/src/media/source.rs b/crates/engine/src/media/source.rs index f13349f18c..faec08ca98 100644 --- a/crates/engine/src/media/source.rs +++ b/crates/engine/src/media/source.rs @@ -24,10 +24,23 @@ pub fn thumb_bucket(long_edge: usize) -> usize { /// Memory budgets. pub(super) const THUMB_SOURCE_BYTES: usize = 384 << 20; +/// The least memory kept for rendered previews (when the budget allows it). pub(super) const RENDERED_MEM_BYTES: usize = 128 << 20; -/// Disk budget for the thumbnail cache. +/// The smallest disk budget for the thumbnail and preview cache. pub const DISK_CACHE_BYTES: u64 = 2 << 30; +/// What one photo's previews take on disk, about: its standard preview (a 2048 px JPEG, ~1 MB) +/// and its grid thumbnails. The default disk budget grows by this per photo of the library, so +/// built previews of a whole wedding are not pruned while they are being made (PERF-PREVIEW-REUSE). +pub const DISK_CACHE_PER_PHOTO: u64 = 3 << 19; +/// The default disk budget for a library of `photos`: [`DISK_CACHE_BYTES`], or room for every +/// photo's previews ([`DISK_CACHE_PER_PHOTO`]) when that is more. +pub fn default_disk_cache_bytes(photos: usize) -> u64 { + DISK_CACHE_BYTES.max((photos as u64).saturating_mul(DISK_CACHE_PER_PHOTO)) +} +/// A cached view render this much smaller than asked for still serves the loupe (rounding: a +/// fit of the same frame into a box one pixel different). +pub(super) const VIEW_REUSE_SLACK: f64 = 0.01; #[derive(Clone, Copy, Debug, PartialEq, Eq, Hash, Serialize, Deserialize)] #[serde(rename_all = "lowercase")] diff --git a/crates/engine/src/media/thumbnail_hash_tests.rs b/crates/engine/src/media/thumbnail_hash_tests.rs index 85089d231d..6f9fcab339 100644 --- a/crates/engine/src/media/thumbnail_hash_tests.rs +++ b/crates/engine/src/media/thumbnail_hash_tests.rs @@ -6,6 +6,60 @@ fn session() -> crate::Session { s.catalog.apply(Op::AddPhoto { photo: Box::new(p) }).unwrap(); s } +/// PERF-CULL: an unedited raw's camera JPEG is read once for the loupe and kept in memory for a +/// step back; the full-size one (zoomed in) is made at the size asked for and kept nowhere; an +/// edited raw or another file has none. +#[test] +fn camera_previews_are_kept_for_a_revisit_and_full_size_ones_are_not() { + let reads = Arc::new(std::sync::atomic::AtomicUsize::new(0)); + let mut s = crate::Session::new(); + let r = reads.clone(); + s.media.preview_loader = Some(Arc::new(move |_: &str, edge: usize| { + r.fetch_add(1, std::sync::atomic::Ordering::SeqCst); + Some(Rgba8 { width: edge.min(64), height: 2, data: vec![[9, 8, 7, 255]; edge.min(64) * 2] }) + })); + let mut raw = Photo::new(PhotoId(1), Source::File { path: "a.cr3".into() }, "a.cr3", "CR3", 6000, 4000, "2026-01-01"); + raw.kind = MediaKind::Raw; + raw.develop = Arc::new(raw.camera_defaults()); + s.catalog.apply(Op::AddPhoto { photo: Box::new(raw) }).unwrap(); + let jpeg = Photo::new(PhotoId(2), Source::File { path: "b.jpg".into() }, "b.jpg", "JPEG", 600, 400, "2026-01-01"); + s.catalog.apply(Op::AddPhoto { photo: Box::new(jpeg) }).unwrap(); + for _ in 0..3 { + let q = s.quick_view_job(PhotoId(1), 48, true).unwrap(); + let r = q.run(); + assert_eq!(r.quick, Some(QuickSource::Embedded)); + } + assert_eq!(reads.load(std::sync::atomic::Ordering::SeqCst), 1, "read once, then from memory"); + let full = s.embedded_full_job(PhotoId(1), 8192).unwrap(); + assert!(full.embedded_cache.is_none() && full.cached.is_empty() && full.small.is_none()); + assert_eq!(full.embedded.as_ref().map(|e| e.2), Some(8192)); + assert_ne!(full.key, s.quick_view_job(PhotoId(1), 48, true).unwrap().key); + assert_eq!(full.run().quick, Some(QuickSource::Embedded)); + assert!(s.embedded_full_job(PhotoId(2), 8192).is_none(), "not a raw"); + let mut edited = (*s.catalog.photo(PhotoId(1)).unwrap().develop).clone(); + edited.light.exposure = 1.0; + s.catalog.apply(Op::SetDevelop { id: PhotoId(1), settings: Arc::new(edited), label: "edit".into(), edited: None }).unwrap(); + assert!(s.embedded_full_job(PhotoId(1), 8192).is_none(), "an edit is shown rendered"); +} + +/// A virtual copy shares its master's file and, unedited, its settings; its loupe stand-ins +/// are its own all the same (CodeRabbit on #3080). +#[test] +fn a_virtual_copy_has_its_own_stand_in_keys() { + let mut s = crate::Session::new(); + s.media.preview_loader = Some(Arc::new(|_: &str, _: usize| None::)); + for id in [1, 2] { + let mut raw = Photo::new(PhotoId(id), Source::File { path: "a.cr3".into() }, "a.cr3", "CR3", 6000, 4000, "2026-01-01"); + raw.kind = MediaKind::Raw; + raw.develop = Arc::new(raw.camera_defaults()); + s.catalog.apply(Op::AddPhoto { photo: Box::new(raw) }).unwrap(); + } + let quick = |s: &mut crate::Session, id: u64| s.quick_view_job(PhotoId(id), 48, true).unwrap().key; + assert_ne!(quick(&mut s, 1), quick(&mut s, 2)); + let full = |s: &mut crate::Session, id: u64| s.embedded_full_job(PhotoId(id), 8192).unwrap().key; + assert_ne!(full(&mut s, 1), full(&mut s, 2)); +} + /// Settings saved by a newer LightCraft (a process number this build doesn't know) render with /// this build's latest process, and a build that resolved the number to an older process /// cached other pixels under the same settings hash: the persistent keys (thumbnails, the @@ -37,7 +91,7 @@ fn persistent_keys_name_the_process_the_settings_render_with() { let resolved = u64::from(lightcraft_develop::Process::LATEST.version().0); let keys = [ (s.thumb_job(PhotoId(1), 256).unwrap().cache.unwrap().1, None, 256), - (crate::Session::view_key(&p, true, 0), Some("view"), 1), + (s.view_key(&p, true), Some("view"), 1), (crate::Session::variant_key(&p, &newer, 256, 0), Some("variant"), 256), ]; for (key, kind, size) in keys { @@ -69,7 +123,7 @@ fn v1_and_v2_photos_keep_the_render_cache_keys_they_had() { }; let keys = [ s.thumb_job(PhotoId(1), 256).unwrap().cache.unwrap().1 == before_v3(None, 256), - crate::Session::view_key(&p, true, 0) == before_v3(Some("view"), 1), + s.view_key(&p, true) == before_v3(Some("view"), 1), crate::Session::variant_key(&p, &settings, 256, 0) == before_v3(Some("variant"), 256), ]; let old = process <= Process::V2; @@ -217,6 +271,8 @@ fn a_render_that_panics_fails_instead_of_never_answering() { key: 9, cached: Vec::new(), embedded: Some(("x.arw".into(), Arc::new(|_: &str, _: usize| -> Option { panic!("preview reader bug") }), 256)), + embedded_cache: None, + embedded_on_disk: false, minimum_embedded_edge: None, small: None, }; @@ -224,3 +280,44 @@ fn a_render_that_panics_fails_instead_of_never_answering() { assert_eq!((r.request_id, r.key), (7, 9)); assert_eq!(r.rendered.err().as_deref(), Some("the render failed unexpectedly: preview reader bug")); } + +/// CodeRabbit on #3544: a stand-in job decoding a camera JPEG when Discard Preview Cache clears the +/// cache doesn't put the picture back: it writes for the generation it was made in (memory and +/// disk alike), as render jobs do. +#[test] +fn a_camera_jpeg_decoded_across_a_cache_clear_is_not_kept() { + for on_disk in [false, true] { + let cache = Arc::new(PreviewCache::memory(64 << 20)); + let made_in = cache.generation(); + let key = Hash128(5); + let quick = QuickJob { + request_id: 1, + photo: PhotoId(1), + key: 2, + cached: Vec::new(), + embedded: Some(("x.arw".into(), Arc::new(|_: &str, _: usize| Some(Rgba8::new(16, 16))), 256)), + embedded_cache: Some((cache.clone(), key, made_in)), + embedded_on_disk: on_disk, + minimum_embedded_edge: None, + small: None, + }; + // the user discards the cache while the job is on its way + cache.clear(); + assert!(quick.run().rendered.is_ok(), "the stand-in is still shown"); + assert!(cache.get_memory(key).is_none(), "on disk {on_disk}: nothing kept in the cleared cache"); + // a job made after the clear keeps it as before + let quick = QuickJob { + request_id: 1, + photo: PhotoId(1), + key: 2, + cached: Vec::new(), + embedded: Some(("x.arw".into(), Arc::new(|_: &str, _: usize| Some(Rgba8::new(16, 16))), 256)), + embedded_cache: Some((cache.clone(), key, cache.generation())), + embedded_on_disk: on_disk, + minimum_embedded_edge: None, + small: None, + }; + assert!(quick.run().rendered.is_ok()); + assert!(cache.get_memory(key).is_some(), "on disk {on_disk}: kept when made after the clear"); + } +} diff --git a/crates/engine/src/prep.rs b/crates/engine/src/prep.rs new file mode 100644 index 0000000000..bb53d24418 --- /dev/null +++ b/crates/engine/src/prep.rs @@ -0,0 +1,532 @@ +//! Preparing photos in the background (PERF-PREVIEW-PREP, notes 47 and 48): while the user culls, +//! the photos ahead are made ready before they are reached, and with the machine otherwise idle +//! the previews of every photo in view are built, so stepping never waits for a render. +//! +//! What is prepared is set per mode, because each mode needs something else: +//! - **Culling** (a folder browsed before import, the Import tab): previews of every photo in the +//! folder, the next photos' loupe renders and their eye focus strips. +//! - **Library**: the same for the photos in the library's view. +//! - **Develop** (editing): the next photos decoded and kept ready to edit. +//! +//! Each mode has its own switch for the whole-view build, how many photos ahead to prepare (0 = +//! none) and which photos ([`Scope`]). The settings are app-wide (the desktop app keeps them in `ui.json`) +//! and a library can keep its own (its `prefs.json`): a timelapse library prepares nothing while a +//! wedding library prepares everything. +//! +//! The whole-view build is the preview build ([`crate::cmd::previews`]) in its background form: two +//! lanes, waiting whenever the photo on screen is being made ([`foreground`]), nearest photos to the +//! user first and re-aimed as they move ([`crate::cmd::previews::PreviewBuild::focus_on`]), photos +//! with previews skipped, one row in the activity stack that can be cancelled. It starts once the +//! user is culling (a few photos stepped through in the loupe), never just because a library was +//! opened (note 36: opening an old library to look at two photos must not start rebuilding). + +use std::sync::Arc; +use std::sync::atomic::{AtomicBool, AtomicU64, Ordering}; + +use lightcraft_catalog::PhotoId; +use serde::{Deserialize, Serialize}; +use serde_json::{Value, json}; + +use crate::Session; +use crate::cmd::previews::PreviewBuild; + +/// Which photos a mode prepares. +#[derive(Clone, Copy, Debug, Default, PartialEq, Eq, Serialize, Deserialize)] +#[serde(rename_all = "camelCase")] +pub enum Scope { + /// Every photo in view (the folder, the collection, the filter). + #[default] + All, + /// Only picks, and photos with stars or a colour label. + Marked, + /// Only the photo on screen: nothing ahead, nothing in the background. + OnScreen, +} + +impl Scope { + pub fn parse(s: &str) -> Option { + match s { + "all" => Some(Scope::All), + "marked" => Some(Scope::Marked), + "onScreen" => Some(Scope::OnScreen), + _ => None, + } + } +} + +/// The most photos prepared ahead of the one on screen. +pub const MAX_AHEAD: u8 = 20; + +/// What one mode prepares. +#[derive(Clone, Copy, Debug, PartialEq, Eq, Serialize, Deserialize)] +#[serde(default, rename_all = "camelCase")] +pub struct ModePrep { + /// Build the previews of every photo in scope in the background. + pub previews: bool, + /// Photos ahead (in the direction of travel) whose loupe render is made before they are reached (0 = none). + pub ahead: u8, + /// Photos ahead whose eye focus strip is worked out before they are reached (0 = only the one on + /// screen), for a frontend that shows one (the desktop app doesn't yet). + pub eye_ahead: u8, + pub scope: Scope, +} + +impl Default for ModePrep { + fn default() -> Self { + ModePrep { previews: true, ahead: 5, eye_ahead: 8, scope: Scope::All } + } +} + +/// Editing: what is kept ready for the next photos. +#[derive(Clone, Copy, Debug, PartialEq, Eq, Serialize, Deserialize)] +#[serde(default, rename_all = "camelCase")] +pub struct DevelopPrep { + /// Photos ahead decoded and kept ready to edit (0 = none). + pub ahead: u8, + pub scope: Scope, +} + +impl Default for DevelopPrep { + fn default() -> Self { + DevelopPrep { ahead: 2, scope: Scope::All } + } +} + +/// The background preparation settings. +#[derive(Clone, Copy, Debug, Default, PartialEq, Eq, Serialize, Deserialize)] +#[serde(default, rename_all = "camelCase")] +pub struct PrepSettings { + pub culling: ModePrep, + pub library: ModePrep, + pub develop: DevelopPrep, +} + +impl PrepSettings { + /// Values from files or agents kept in range. + pub fn sanitized(mut self) -> Self { + for m in [&mut self.culling, &mut self.library] { + m.ahead = m.ahead.min(MAX_AHEAD); + m.eye_ahead = m.eye_ahead.min(MAX_AHEAD); + } + self.develop.ahead = self.develop.ahead.min(MAX_AHEAD); + self + } + + /// The settings of a browsing mode ([`Mode::Culling`] or [`Mode::Library`]; Develop prepares + /// views the way the Library does). + pub fn mode(&self, mode: Mode) -> ModePrep { + match mode { + Mode::Culling => self.culling, + Mode::Library | Mode::Develop => self.library, + } + } + + /// Photos ahead to render for the loupe in `mode` (0 when its scope is the photo on screen). + pub fn ahead(&self, mode: Mode) -> usize { + match mode { + Mode::Develop if self.develop.scope == Scope::OnScreen => 0, + Mode::Develop => usize::from(self.develop.ahead), + m => { + let p = self.mode(m); + if p.scope == Scope::OnScreen { 0 } else { usize::from(p.ahead) } + } + } + } + + /// Photos ahead whose eye focus strip is worked out in `mode`. + pub fn eye_ahead(&self, mode: Mode) -> usize { + let p = self.mode(mode); + if p.scope == Scope::OnScreen { 0 } else { usize::from(p.eye_ahead) } + } + + /// The scope that applies in `mode`. + pub fn scope(&self, mode: Mode) -> Scope { + match mode { + Mode::Develop => self.develop.scope, + m => self.mode(m).scope, + } + } + + /// Apply the fields of `patch` (a partial object, as `prefs.setBackgroundPrep` takes it). + pub fn patched(&self, patch: &Value) -> Result { + let mut v = serde_json::to_value(self).map_err(|e| e.to_string())?; + merge(&mut v, patch, 0); + let p: PrepSettings = serde_json::from_value(v).map_err(|e| format!("background preparation settings: {e}"))?; + Ok(p.sanitized()) + } +} + +/// `patch` merged into `into`, objects field by field (two levels deep: mode → field). +fn merge(into: &mut Value, patch: &Value, depth: usize) { + match (into, patch) { + (Value::Object(a), Value::Object(b)) if depth < 3 => { + for (k, v) in b { + match a.get_mut(k) { + Some(old) => merge(old, v, depth + 1), + None => { + a.insert(k.clone(), v.clone()); + } + } + } + } + (a, b) => *a = b.clone(), + } +} + +/// What the user is doing, as far as preparation goes. +#[derive(Clone, Copy, Debug, PartialEq, Eq, Serialize, Deserialize)] +#[serde(rename_all = "camelCase")] +pub enum Mode { + Culling, + Library, + Develop, +} + +impl Mode { + pub fn parse(s: &str) -> Option { + match s { + "culling" => Some(Mode::Culling), + "library" => Some(Mode::Library), + "develop" => Some(Mode::Develop), + _ => None, + } + } + + /// The mode of what `s` shows when the frontend doesn't say: a folder browsed before import is + /// culling, anything else the library. + pub fn of(s: &Session) -> Mode { + if s.source == crate::LibrarySource::Folder { Mode::Culling } else { Mode::Library } + } +} + +/// Photos the user steps through in the loupe before the whole view is prepared: a library opened +/// to look at one photo starts nothing. +pub const STEPS_BEFORE_BUILD: usize = 3; +/// How long a view whose build couldn't start waits before trying again. +pub const RETRY_AFTER: std::time::Duration = std::time::Duration::from_secs(30); + +/// The background preparation's state in a session. +#[derive(Default)] +pub struct PrepState { + /// App-wide settings. + pub app: PrepSettings, + /// This library's own (`None`: it uses the app-wide ones). + pub library: Option, + /// The background build this preparation started. + build: Option>, + /// The view (its signature) whose build was started, or that the user stopped. + started_for: Option, + declined_for: Option, + /// A build that couldn't start: the view (its signature), and when it may try again (not on + /// every tick while the cause lasts; CodeRabbit on #3080). + pub retry_at: Option<(u64, web_time::Instant)>, + /// Tests: the next starts to fail, as a build thread that can't be spawned does. + #[cfg(test)] + pub(crate) fail_starts: usize, + /// The view the steps were counted in, the photo last seen, and the steps. + steps_in: Option, + last_at: Option, + steps: usize, +} + +impl std::fmt::Debug for PrepState { + fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result { + f.debug_struct("PrepState").field("app", &self.app).field("library", &self.library).field("steps", &self.steps).finish() + } +} + +impl PrepState { + /// The settings in use. + pub fn settings(&self) -> PrepSettings { + self.library.unwrap_or(self.app) + } +} + +/// Until when (ms on [`clock_ms`]) the photo on screen is being made: background work waits. +static FOREGROUND_UNTIL: AtomicU64 = AtomicU64::new(0); + +/// How long a foreground signal holds background work back (ms): a little more than a frame +/// apart, so a frontend saying it every frame holds it back the whole time. +const FOREGROUND_HOLD_MS: u64 = 250; + +fn clock_ms() -> u64 { + static START: std::sync::OnceLock = std::sync::OnceLock::new(); + u64::try_from(START.get_or_init(web_time::Instant::now).elapsed().as_millis()).unwrap_or(u64::MAX) +} + +/// The frontend says the photo on screen is being made (a render it waits for is on its way): +/// background preparation waits until it has been quiet for a moment. +pub fn foreground() { + FOREGROUND_UNTIL.store(clock_ms().saturating_add(FOREGROUND_HOLD_MS), Ordering::Relaxed); +} + +/// Whether the photo on screen is being made now. +pub fn foreground_busy() -> bool { + #[cfg(test)] + if TEST_HOLD.load(Ordering::Relaxed) > 0 { + return true; + } + clock_ms() < FOREGROUND_UNTIL.load(Ordering::Relaxed) +} + +/// Tests: hold every background build at its foreground wait until released (a synchronization +/// point that doesn't depend on the hold's 250 ms). A count, not a switch: tests running side by +/// side each hold until their own release, and one test's release doesn't free another's build. +#[cfg(test)] +static TEST_HOLD: std::sync::atomic::AtomicUsize = std::sync::atomic::AtomicUsize::new(0); + +/// Holds background builds while it lives (also when the test fails), whatever other holds do. +#[cfg(test)] +pub(crate) struct TestHold; + +#[cfg(test)] +impl TestHold { + pub(crate) fn new() -> Self { + TEST_HOLD.fetch_add(1, Ordering::Relaxed); + TestHold + } +} + +#[cfg(test)] +impl Drop for TestHold { + fn drop(&mut self) { + // (one release per hold made: never below 0) + TEST_HOLD.fetch_sub(1, Ordering::Relaxed); + } +} + +/// Wait while the photo on screen is being made. `true`: `cancel` was set meanwhile. +pub fn wait_for_foreground(cancel: &AtomicBool) -> bool { + #[cfg(not(target_arch = "wasm32"))] + while foreground_busy() { + if cancel.load(Ordering::Relaxed) { + return true; + } + std::thread::sleep(std::time::Duration::from_millis(30)); + } + cancel.load(Ordering::Relaxed) +} + +/// Photos the background build renders side by side: each render spreads over the cores already, +/// so two keep the machine busy while leaving room for the photo on screen. +pub fn background_lanes() -> usize { + let cores = std::thread::available_parallelism().map_or(2, |n| n.get()); + if cores >= 8 { 2 } else { 1 } +} + +/// A cheap identity of the photos a build prepares: a build is started once per view. `scope` +/// [`Scope::Marked`]: which of them are marked counts too (marking a photo changes the set). +fn signature(ids: &[PhotoId], s: &Session, scope: Scope) -> u64 { + let mut h = lightcraft_preview::Hasher128::new(); + h.u64(ids.len() as u64).u64(s.catalog.len() as u64); + // every id: a sort that moves photos in the middle is another view + for id in ids { + h.u64(id.0); + if scope == Scope::Marked { + h.u64(u64::from(marked(s, *id))); + } + } + h.finish().0 as u64 +} + +/// Whether `id` is a pick, or has stars or a colour label (the [`Scope::Marked`] photos). +pub fn marked(s: &Session, id: PhotoId) -> bool { + s.catalog.photo(id).is_some_and(|p| p.flag == lightcraft_catalog::Flag::Pick || p.rating > 0 || p.label.is_some()) +} + +impl Session { + /// How long until a view whose build couldn't start tries again ([`RETRY_AFTER`]), while one + /// waits: the app asks for a frame then, so the retry doesn't wait for the next input + /// (CodeRabbit on #3077/#3080). `None` once the time has come. + pub fn prep_retry_in(&self) -> Option { + // (a deadline that has passed asks for nothing: the pass at it tried again, or the view + // can't be tried now and a frame at once on every pass would keep an idle window drawing: + // CodeRabbit on #3077 / #3080) + let wait = self.prep.retry_at.map(|(_, at)| at.saturating_duration_since(web_time::Instant::now()))?; + (!wait.is_zero()).then_some(wait) + } + + /// The background preparation settings in use (this library's own, else the app-wide ones). + pub fn prep_settings(&self) -> PrepSettings { + self.prep.settings() + } + + /// Advance the background preparation (cheap; call every frame). `mode`: what the user is doing; + /// `at`: the photo in the loupe (`None` in the grid), `backward`: stepping back. + pub fn prep_tick(&mut self, mode: Mode, at: Option, backward: bool) { + let settings = self.prep.settings(); + let want = settings.mode(mode); + // the whole view's build follows its own row (Culling, or Library also while editing): its + // switch and its "which photos"; the Editing row's scope is for the photos kept ready ahead + let wanted = want.previews && want.scope != Scope::OnScreen; + // the running build: re-aimed at the user, stopped when switched off + if let Some(b) = self.prep.build.clone() { + if b.running() { + if !wanted { + b.cancel.store(true, Ordering::Relaxed); + } else if self.prep.started_for.is_some_and(|sig| sig != signature(&self.visible_cloned(), self, want.scope)) { + // another view (a folder, a filter, another library): this build's photos are + // not the ones shown; stopped, and the view now shown is prepared in its place + // (not counted as stopped by the user) + b.cancel.store(true, Ordering::Relaxed); + self.prep.started_for = None; + } else if let Some(id) = at { + b.focus_on(id, backward); + } + return; + } + self.prep.build = None; + let mine = self.preview_build.as_ref().is_some_and(|p| Arc::ptr_eq(p, &b)); + if b.cancel.load(Ordering::Relaxed) { + if mine && wanted { + // stopped from its row: not again for this view + self.prep.declined_for = self.prep.started_for; + } else { + // replaced by a build that was asked for, or switched off: may start again later + self.prep.started_for = None; + } + } + } + if !wanted { + return; + } + let Some(at) = at else { return }; + let ids = self.visible_cloned(); + let sig = signature(&ids, self, want.scope); + if self.prep.steps_in != Some(sig) { + self.prep.steps_in = Some(sig); + self.prep.steps = 0; + self.prep.last_at = None; + } + if self.prep.last_at != Some(at) { + self.prep.last_at = Some(at); + self.prep.steps = self.prep.steps.saturating_add(1); + } + let busy = self.preview_build.as_ref().is_some_and(|b| b.running()); + if busy + || self.prep.steps < STEPS_BEFORE_BUILD + || self.prep.started_for == Some(sig) + || self.prep.declined_for == Some(sig) + || self.prep.retry_at.is_some_and(|(view, at)| view == sig && web_time::Instant::now() < at) + || self.library.as_ref().is_none_or(|l| !l.on_disk) + { + return; + } + let ids: Vec = match want.scope { + Scope::Marked => ids.into_iter().filter(|id| marked(self, *id)).map(|id| id.0).collect(), + _ => ids.into_iter().map(|id| id.0).collect(), + }; + self.prep.started_for = Some(sig); + if ids.is_empty() { + return; + } + let p = json!({"ids": ids, "background": true, "from": at.0, "backward": backward}); + // (tests: a start made to fail, as a build thread that can't be spawned fails) + #[cfg(test)] + let fail = self.prep.fail_starts.checked_sub(1).map(|left| self.prep.fail_starts = left).is_some(); + #[cfg(not(test))] + let fail = false; + let started = if fail { + Err(crate::EngineError::Disabled("library.buildPreviews".into(), "test: the build couldn't start".into())) + } else { + self.execute("library.buildPreviews", &p) + }; + match started { + Ok(_) => { + self.prep.build = self.preview_build.clone(); + self.prep.retry_at = None; + } + Err(e) => { + // not started: this view tries again after a while (not on every tick) + self.prep.started_for = None; + self.prep.retry_at = web_time::Instant::now().checked_add(RETRY_AFTER).map(|at| (sig, at)); + log::info!("background preparation: {e}"); + } + } + } + + /// Change the background preparation settings: the app-wide ones, or (`library`) this library's + /// own. Saved at once. + pub fn set_prep_settings(&mut self, settings: PrepSettings, library: bool) -> crate::Result<()> { + let settings = settings.sanitized(); + if library { + let previous = self.prep.library; + self.prep.library = Some(settings); + self.save_library_prep(previous) + } else { + let previous = self.prep.app; + self.prep.app = settings; + let saved = self.save_prefs(); + if saved.is_err() { + self.prep.app = previous; + } + saved + } + } + + /// Back to the app-wide settings for this library. + pub fn clear_library_prep(&mut self) -> crate::Result<()> { + let previous = self.prep.library; + self.prep.library = None; + self.save_library_prep(previous) + } + + /// Save this library's preparation settings; when that fails, `previous` is put back, so the + /// session doesn't go on with settings the command reported as not changed (CodeRabbit on #3544). + fn save_library_prep(&mut self, previous: Option) -> crate::Result<()> { + let saved = self.save_prefs(); + if saved.is_err() { + self.prep.library = previous; + } + saved + } + + pub(crate) fn prep_json(&self) -> Value { + json!({ + "app": self.prep.app, + "library": self.prep.library, + "settings": self.prep.settings(), + "own": self.prep.library.is_some(), + "building": self.prep.build.as_ref().filter(|b| b.running()).map(|b| b.json()), + }) + } +} + +#[cfg(test)] +mod tests { + use super::*; + + #[test] + fn a_patch_changes_only_what_it_names_and_is_kept_in_range() { + let p = PrepSettings::default().patched(&json!({"culling": {"ahead": 99}, "develop": {"scope": "onScreen"}})).unwrap(); + assert_eq!(p.culling.ahead, MAX_AHEAD); + assert_eq!(p.culling.eye_ahead, 8, "untouched"); + assert!(p.library.previews); + assert_eq!(p.develop.scope, Scope::OnScreen); + assert_eq!(p.ahead(Mode::Develop), 0); + assert!(PrepSettings::default().patched(&json!({"culling": {"scope": "sometimes"}})).is_err()); + } + + /// Tests side by side: one test's release doesn't free the build another test holds (CodeRabbit on #3077). + #[test] + fn one_tests_release_does_not_free_another_tests_hold() { + let a = TestHold::new(); + let b = TestHold::new(); + drop(a); + assert!(TEST_HOLD.load(Ordering::Relaxed) > 0 && foreground_busy(), "the other hold still holds"); + drop(b); + } + + #[test] + fn the_photo_on_screen_holds_background_work_back() { + let cancel = AtomicBool::new(false); + foreground(); + assert!(foreground_busy()); + let t = web_time::Instant::now(); + assert!(!wait_for_foreground(&cancel)); + assert!(t.elapsed().as_millis() >= 100, "waited for the foreground to go quiet"); + cancel.store(true, Ordering::Relaxed); + foreground(); + assert!(wait_for_foreground(&cancel), "a cancel ends the wait"); + } +} diff --git a/crates/engine/src/tests.rs b/crates/engine/src/tests.rs index a22547ec4a..3393eca3fc 100644 --- a/crates/engine/src/tests.rs +++ b/crates/engine/src/tests.rs @@ -44,6 +44,7 @@ mod organize; mod persist; mod prefs; mod presets_scale; +mod preview_prep; mod print; mod process; mod recheck; diff --git a/crates/engine/src/tests/import.rs b/crates/engine/src/tests/import.rs index e1c6dacd62..54f3382014 100644 --- a/crates/engine/src/tests/import.rs +++ b/crates/engine/src/tests/import.rs @@ -303,6 +303,149 @@ fn browsing_a_folder_lists_its_photos_without_adding_them() { let _ = std::fs::remove_dir_all(&dir); } +/// PERF-CULL-1: the Local view reads only what the library lacks, newest file first for the default +/// (newest capture first) sort, so the first screen is read first; other sorts follow their key. +#[test] +fn browsing_queues_unknown_files_in_the_order_the_view_shows_them() { + use lightcraft_catalog::{Sort, SortKey}; + let dir = temp_dir("browse-order"); + let set_time = |p: &Path, secs: u64| { + let t = std::time::UNIX_EPOCH + std::time::Duration::from_secs(secs); + std::fs::File::options().write(true).open(p).unwrap().set_modified(t).unwrap(); + }; + for (name, seed, secs) in [("a.png", 1u8, 300u64), ("b.png", 2, 100), ("c.png", 3, 200)] { + write_png(&dir.join(name), seed); + set_time(&dir.join(name), 1_700_000_000 + secs); + } + let mut s = Session::new().with_fs(); + let known = dir.join("b.png").to_string_lossy().to_string(); + s.execute("library.import", &json!({"paths": [known.clone()]})).unwrap(); + let files: Vec = ["a.png", "b.png", "c.png", "gone.png"].iter().map(|n| dir.join(n).to_string_lossy().to_string()).collect(); + let (input, _) = crate::import::ScanInput::new(&mut s, &[]); + let names = |v: Vec| v.iter().map(|f| Path::new(f).file_name().unwrap().to_string_lossy().to_string()).collect::>(); + // newest first, the file already in the library left out, an unreadable one last + let q = crate::import::browse_queue(&input, files.clone(), &Sort::default()); + assert_eq!(names(q), ["a.png", "c.png", "gone.png"]); + let q = crate::import::browse_queue(&input, files.clone(), &Sort { ascending: true, ..Sort::default() }); + assert_eq!(names(q), ["c.png", "a.png", "gone.png"]); + let q = crate::import::browse_queue(&input, files.clone(), &Sort { key: SortKey::FileName, ascending: false, ..Sort::default() }); + assert_eq!(names(q), ["gone.png", "c.png", "a.png"]); + // by size an unreadable file is last in both directions (it was sized 0, first when ascending) + for ascending in [true, false] { + let q = crate::import::browse_queue(&input, files.clone(), &Sort { key: SortKey::FileSize, ascending, ..Sort::default() }); + assert_eq!(names(q).last().map(String::as_str), Some("gone.png"), "ascending {ascending}"); + } + let q = crate::import::browse_queue(&input, files, &Sort { key: SortKey::Rating, ..Sort::default() }); + assert_eq!(names(q), ["gone.png", "c.png", "a.png"], "new photos tie on rating: by name, descending as the view"); + let _ = std::fs::remove_dir_all(&dir); +} + +/// Choosing another folder while a browse orders its files (`cancel` set) stops the ordering: no queue comes +/// back, and nothing more is read (CodeRabbit on #3055). Browsing hands back the probes of earlier scans it took. +#[test] +fn a_cancelled_browse_stops_ordering_and_keeps_earlier_probes() { + use lightcraft_catalog::{Sort, SortKey}; + use std::sync::atomic::AtomicBool; + let dir = temp_dir("browse-cancel"); + write_png(&dir.join("a.png"), 1); + write_png(&dir.join("b.png"), 2); + let files: Vec = ["a.png", "b.png"].iter().map(|n| dir.join(n).to_string_lossy().to_string()).collect(); + let mut s = Session::new().with_fs(); + s.import_probes.insert(files[0].clone(), crate::media::ProbeInfo { file_size: 1, ..Default::default() }); + let (mut input, _) = crate::import::ScanInput::new(&mut s, &[]); + for key in [SortKey::CaptureDate, SortKey::FileSize, SortKey::FileName, SortKey::Random] { + let sort = Sort { key, ..Sort::default() }; + assert_eq!(crate::import::browse_queue_until(&input, files.clone(), &sort, &AtomicBool::new(true)), None, "{key:?}"); + assert_eq!(crate::import::browse_queue_until(&input, files.clone(), &sort, &AtomicBool::new(false)).map(|q| q.len()), Some(2), "{key:?}"); + } + let probes = input.take_probes(); + assert!(probes.contains_key(&files[0]), "the earlier scan's probe comes back"); + let _ = std::fs::remove_dir_all(&dir); +} + +/// Browsing a folder with its subfolders lists nothing inside the library's own folder (the +/// import that follows would read it as photos), as the import's scan skips it. +#[test] +fn browsing_with_subfolders_skips_the_librarys_own_folder() { + let dir = temp_dir("browse-skip-lib"); + std::fs::create_dir_all(dir.join("lib")).unwrap(); + let mut s = Session::new().with_fs(); + s.open_library(dir.join("lib"), false).unwrap(); + write_png(&dir.join("a.png"), 1); + write_png(&dir.join("lib").join("inside.png"), 2); + let (input, _) = crate::import::ScanInput::new(&mut s, &[]); + let root = dir.to_string_lossy().to_string(); + let names = |v: Vec| v.iter().map(|f| Path::new(f).file_name().unwrap().to_string_lossy().to_string()).collect::>(); + assert_eq!(names(crate::import::browse_files(&input, &root, true).unwrap()), ["a.png"]); + assert_eq!(names(crate::import::browse_files(&input, &root, false).unwrap()), ["a.png"]); + // the library's folder chosen itself, with or without its subfolders + let lib = dir.join("lib").to_string_lossy().to_string(); + assert!(crate::import::browse_files(&input, &lib, false).unwrap().is_empty()); + assert!(crate::import::browse_files(&input, &lib, true).unwrap().is_empty()); + // the library's folder by another name: through `..`, or a symlink to it + std::fs::create_dir_all(dir.join("other")).unwrap(); + let around = dir.join("other").join("..").join("lib").to_string_lossy().to_string(); + assert!(crate::import::browse_files(&input, &around, false).unwrap().is_empty(), "{around}"); + assert!(crate::import::browse_files(&input, &around, true).unwrap().is_empty(), "{around}"); + #[cfg(unix)] + { + let alias = dir.join("alias"); + std::os::unix::fs::symlink(dir.join("lib"), &alias).unwrap(); + let alias = alias.to_string_lossy().to_string(); + assert!(crate::import::browse_files(&input, &alias, false).unwrap().is_empty(), "{alias}"); + assert!(crate::import::browse_files(&input, &alias, true).unwrap().is_empty(), "{alias}"); + } + // a folder next to the library named through it is outside it: browsed (CodeRabbit on #3077) + std::fs::create_dir_all(dir.join("outside")).unwrap(); + write_png(&dir.join("outside").join("next.png"), 3); + let next = dir.join("lib").join("..").join("outside").to_string_lossy().to_string(); + assert_eq!(names(crate::import::browse_files(&input, &next, false).unwrap()), ["next.png"], "{next}"); + assert_eq!(names(crate::import::browse_files(&input, &next, true).unwrap()), ["next.png"], "{next}"); + // under the folder's one spelling: the paths match the folder's own, so the catalog's records are found + // (CodeRabbit on #3055) + let direct = dir.join("outside").to_string_lossy().to_string(); + for sub in [false, true] { + assert_eq!(crate::import::browse_files(&input, &next, sub).unwrap(), crate::import::browse_files(&input, &direct, sub).unwrap()); + } + // a folder inside the library that isn't there is an error too, not an empty folder + let gone = dir.join("lib").join("not-there").to_string_lossy().to_string(); + assert!(crate::import::browse_files(&input, &gone, false).is_err(), "{gone}"); + // a folder that can't be listed is an error, not an empty folder (its photos aren't "gone") + let missing = dir.join("not-there").to_string_lossy().to_string(); + assert!(crate::import::browse_files(&input, &missing, false).is_err()); + assert!(crate::import::browse_files(&input, &missing, true).is_err()); + let _ = std::fs::remove_dir_all(&dir); +} + +/// Browsing's queue breaks ties the way the catalog does (by name, in the view's direction), and +/// the import- and edit-date sorts, where new photos tie, go by name rather than file time. +#[test] +fn browsing_breaks_ties_by_name_in_the_views_direction() { + use lightcraft_catalog::{Sort, SortKey}; + let dir = temp_dir("browse-ties"); + // names and file times run in opposite directions; x and y share a time and a size + for (name, secs) in [("x.png", 100u64), ("y.png", 100), ("z.png", 50)] { + write_png(&dir.join(name), 7); + let t = std::time::UNIX_EPOCH + std::time::Duration::from_secs(1_700_000_000 + secs); + std::fs::File::options().write(true).open(dir.join(name)).unwrap().set_modified(t).unwrap(); + } + let files: Vec = ["y.png", "z.png", "x.png"].iter().map(|n| dir.join(n).to_string_lossy().to_string()).collect(); + let order = |key: SortKey, ascending: bool| { + let mut f = files.clone(); + crate::import::browse_order(&mut f, &Sort { key, ascending, ..Sort::default() }); + f.iter().map(|f| Path::new(f).file_name().unwrap().to_string_lossy().to_string()).collect::>() + }; + assert_eq!(order(SortKey::CaptureDate, false), ["y.png", "x.png", "z.png"]); + assert_eq!(order(SortKey::CaptureDate, true), ["z.png", "x.png", "y.png"]); + for key in [SortKey::ImportDate, SortKey::EditDate, SortKey::Rating] { + assert_eq!(order(key, true), ["x.png", "y.png", "z.png"], "{key:?}"); + assert_eq!(order(key, false), ["z.png", "y.png", "x.png"], "{key:?}"); + } + assert_eq!(order(SortKey::FileSize, true), ["x.png", "y.png", "z.png"]); + assert_eq!(order(SortKey::FileSize, false), ["z.png", "y.png", "x.png"]); + let _ = std::fs::remove_dir_all(&dir); +} + /// Photos a.png, b.png, c.png imported from `old`, then: a renamed into `moved`; b renamed, with /// an impostor (same name and size, other bytes) under its old name; c gone, with two impostors. fn renamed_and_ambiguous_scene(tag: &str) -> (std::path::PathBuf, Session) { @@ -2785,3 +2928,69 @@ fn a_file_arriving_beside_its_raws_stem_reads_its_own_sidecar() { assert_eq!(read(&s, "IMG_3.TIF"), 0, "the catalogued raw owns IMG_3.xmp"); let _ = std::fs::remove_dir_all(&src); } + +/// A folder's one spelling: `.` is the current folder, `.` and `..` are taken out, never above the +/// root (CodeRabbit on #3055 / #3080). +#[test] +fn a_browsed_folder_has_one_spelling() { + use crate::import::browse_path; + let root = std::env::temp_dir(); + assert_eq!(browse_path(&root.join("lib").join("..").join("outside")).unwrap(), root.join("outside")); + assert_eq!(browse_path(&root.join(".").join("a").join(".").join("b")).unwrap(), root.join("a").join("b")); + let top: PathBuf = root.ancestors().last().map(Path::to_path_buf).unwrap_or_default(); + assert_eq!(browse_path(&top.join("..").join("x")).unwrap(), top.join("x"), "never above the root"); + assert_eq!(browse_path(Path::new(".")).unwrap(), std::env::current_dir().unwrap(), "`.` is the current folder"); + assert_eq!(browse_path(Path::new("a/..")).unwrap(), std::env::current_dir().unwrap()); + // the UI's spelling, without the disk: the same where no link comes before a `..` + let by_name = crate::import::browse_spelling(&root.join("lib").join("..").join("outside")).unwrap(); + assert_eq!(by_name, root.join("outside")); + assert_eq!(crate::import::browse_spelling(Path::new(".")).unwrap(), std::env::current_dir().unwrap()); +} + +/// `link/..` is the folder the system reads (POSIX: above where the link points; Windows: the +/// folder holding the link): browsing it lists what the system lists there (CodeRabbit on #3055 / +/// #3077 / #3080). +#[test] +fn a_link_before_dotdot_names_the_folder_the_system_reads() { + let dir = temp_dir("browse-link-dotdot"); + std::fs::create_dir_all(dir.join("real").join("sub")).unwrap(); + write_png(&dir.join("real").join("in-real.png"), 1); + write_png(&dir.join("beside-link.png"), 2); + let link = dir.join("link"); + #[cfg(unix)] + let made = std::os::unix::fs::symlink(dir.join("real").join("sub"), &link); + #[cfg(windows)] + let made = std::os::windows::fs::symlink_dir(dir.join("real").join("sub"), &link); + if let Err(e) = made { + lightcraft_testkit::skip!(System, "can't make a folder link here: {e}"); + let _ = std::fs::remove_dir_all(&dir); + return; + } + let mut s = Session::new().with_fs(); + let (input, _) = crate::import::ScanInput::new(&mut s, &[]); + let through = link.join(".."); + let names = |v: Vec| v.iter().map(|f| Path::new(f).file_name().unwrap().to_string_lossy().to_string()).collect::>(); + let listed = names(crate::import::browse_files(&input, &through.to_string_lossy(), false).unwrap()); + let mut system: Vec = + std::fs::read_dir(&through).unwrap().flatten().map(|e| e.file_name().to_string_lossy().to_string()).filter(|n| n.ends_with(".png")).collect(); + system.sort(); + assert_eq!(listed, system, "{}", through.display()); + #[cfg(unix)] + assert_eq!(listed, ["in-real.png"]); + let _ = std::fs::remove_dir_all(&dir); +} + +/// A broken link before `..` is an error, not the link's own folder by name: the system can't read +/// `broken/..` either (CodeRabbit on #3577). +#[cfg(unix)] +#[test] +fn a_broken_link_before_dotdot_is_an_error() { + let dir = temp_dir("browse-broken-link-dotdot"); + std::fs::create_dir_all(&dir).unwrap(); + let link = dir.join("broken"); + std::os::unix::fs::symlink(dir.join("nowhere"), &link).unwrap(); + assert!(crate::import::browse_path(&link.join("..")).is_err()); + // without reading the disk the spelling is still the folder by name + assert_eq!(crate::import::browse_spelling(&link.join("..")).unwrap(), dir); + let _ = std::fs::remove_dir_all(&dir); +} diff --git a/crates/engine/src/tests/lens_refresh.rs b/crates/engine/src/tests/lens_refresh.rs index 7678c93a3d..d3bf3a3574 100644 --- a/crates/engine/src/tests/lens_refresh.rs +++ b/crates/engine/src/tests/lens_refresh.rs @@ -237,7 +237,7 @@ fn a_new_database_profile_changes_the_render_and_cache_keys() { let render = s.render_job(id, 800, 600, false, true).unwrap().key; let thumb = s.thumb_job(id, 256).unwrap().cache.unwrap().1; let p = s.catalog.photo(id).unwrap().clone(); - (render, thumb, Session::variant_key(&p, &p.develop, 128, 0), Session::view_key(&p, true, 0)) + (render, thumb, Session::variant_key(&p, &p.develop, 128, 0), s.view_key(&p, true)) }; let before = keys(&mut s); s.catalog.apply(Op::SetEmbeddedLens { id, lens: Some(Box::new(lens(-0.03))) }).unwrap(); diff --git a/crates/engine/src/tests/preview_prep.rs b/crates/engine/src/tests/preview_prep.rs new file mode 100644 index 0000000000..0981c2cf96 --- /dev/null +++ b/crates/engine/src/tests/preview_prep.rs @@ -0,0 +1,635 @@ +//! Previews are made once and used (notes 47–49): the loupe shows a built or prepared preview +//! instead of rendering the photo again, a build skips photos whose previews are there, built +//! previews are on disk when the build says it is done, the import's queue shows one row, and the +//! background preparation follows the user and gives way to a build that was asked for. + +use std::path::{Path, PathBuf}; +use std::sync::atomic::Ordering; + +use lightcraft_catalog::PhotoId; +use serde_json::json; + +use crate::Session; +use crate::media::QuickSource; + +fn temp_dir(tag: &str) -> PathBuf { + let d = std::env::temp_dir().join(format!("lc-preview-prep-{tag}-{}", std::process::id())); + let _ = std::fs::remove_dir_all(&d); + std::fs::create_dir_all(&d).unwrap(); + d +} + +fn png(p: &Path, seed: u8) { + let img = lightcraft_raster::Rgba8::from_fn(600, 400, |x, y| [(x % 251) as u8, (y % 241) as u8, seed, 255]); + let b = + crate::export::encode_image(&img, &crate::export::ExportOptions { format: crate::export::ExportFormat::Png, ..Default::default() }).unwrap(); + std::fs::create_dir_all(p.parent().unwrap()).unwrap(); + std::fs::write(p, b).unwrap(); +} + +/// A library on disk with `n` photos (600 × 400 PNGs) imported, in name order. +fn library(tag: &str, n: usize) -> (Session, PathBuf, Vec) { + let root = temp_dir(tag); + let paths: Vec = (0..n) + .map(|i| { + let p = root.join("shoot").join(format!("p{i:02}.png")); + png(&p, (i * 20) as u8); + p.to_string_lossy().to_string() + }) + .collect(); + let mut s = Session::new().with_fs(); + s.open_library(root.join("lib"), false).unwrap(); + s.execute("library.import", &json!({"paths": paths})).unwrap(); + let mut ids: Vec<(String, PhotoId)> = s.catalog.photos().map(|p| (p.file_name.clone(), p.id)).collect(); + ids.sort(); + (s, root, ids.into_iter().map(|(_, id)| id).collect()) +} + +fn ids_json(ids: &[PhotoId]) -> Vec { + ids.iter().map(|i| i.0).collect() +} + +/// FredApps on #3077: building previews (asked for, and the background preparation while culling) +/// writes only into the library's own cache, never next to the photos: the shoot folder holds the +/// same files afterwards, byte for byte. +#[test] +fn building_previews_writes_nothing_next_to_the_photos() { + let (mut s, root, ids) = library("prep-beside", 4); + let shoot = root.join("shoot"); + let listing = |dir: &std::path::Path| { + let mut v: Vec<(String, Vec)> = std::fs::read_dir(dir) + .unwrap() + .map(|e| e.unwrap().path()) + .map(|p| (p.file_name().unwrap().to_string_lossy().to_string(), std::fs::read(&p).unwrap())) + .collect(); + v.sort(); + v + }; + let before = listing(&shoot); + assert_eq!(before.len(), 4); + let r = s.execute("library.buildPreviews", &json!({"ids": ids_json(&ids), "edge": 300, "wait": true})).unwrap(); + assert_eq!(r["done"], 4, "{r}"); + for id in &ids { + s.prep_tick(crate::prep::Mode::Library, Some(*id), false); + } + if let Some(b) = s.preview_build.clone() { + assert!(wait_finished(&b), "the background build stops (a late write would slip past the check)"); + } + assert_eq!(listing(&shoot), before, "nothing was written next to the photos"); + let _ = std::fs::remove_dir_all(&root); +} + +/// Note 47: a built preview at least as big as the loupe's size is what the loupe shows (fitted +/// down, no render); a smaller one is not (the photo renders). +#[test] +fn the_loupe_uses_a_built_preview_that_is_big_enough() { + let mut s = Session::with_demo(); + let id = s.active().unwrap(); + s.execute("library.buildPreviews", &json!({"ids": [id.0], "edge": 512, "wait": true})).unwrap(); + let r = s.loupe_job(id, 300, 300, true).unwrap().run(); + assert_eq!(r.quick, Some(QuickSource::Cached), "taken from the built preview"); + let img = r.rendered.unwrap().image; + assert_eq!(img.width.max(img.height), 300, "fitted to the loupe's size"); + // bigger than what was built: rendered + let r = s.loupe_job(id, 900, 900, true).unwrap().run(); + assert_eq!(r.quick, None, "a preview too small for the loupe is not used"); + assert!(r.rendered.is_ok()); + // a draft (a slider held) never comes from the cache + let r = s.loupe_job(id, 300, 300, true).unwrap().draft().run(); + assert_eq!(r.quick, None); +} + +/// Shrinking a cached view keeps its colour (an average of the pixels it covers) and its aspect, +/// and refuses to enlarge. +#[test] +fn a_cached_view_is_fitted_down_never_up() { + let img = lightcraft_raster::Rgba8::from_fn(400, 200, |x, _| if x % 2 == 0 { [200, 100, 0, 255] } else { [100, 50, 0, 255] }); + let f = crate::media::fit_cached(&img, 100, 100).unwrap(); + assert_eq!((f.width, f.height), (100, 50)); + assert!(f.data.iter().all(|p| p[0] == 150 && p[1] == 75), "averaged"); + assert!(crate::media::fit_cached(&img, 800, 800).is_none(), "never enlarged"); + assert_eq!(crate::media::fit_cached(&img, 401, 999).map(|f| f.width), Some(400), "a pixel short is the same size"); + assert!(crate::media::fit_cached(&img, 0, 10).is_none()); +} + +/// Note 47: the rendered previews in memory follow the memory budget (they were capped at +/// 128 MB, about 84 loupe renders, whatever the budget). +#[test] +fn the_memory_for_rendered_previews_grows_with_the_budget() { + let share = crate::memory::cache_share(20 << 30); + assert!(crate::media::rendered_budget(share) >= 2 << 30, "a 20 GB budget keeps gigabytes of previews"); + assert_eq!(crate::media::rendered_budget(256 << 20), 128 << 20, "a small budget keeps the old floor where it can"); + assert!(crate::media::rendered_budget(64 << 20) <= 32 << 20, "and never more than the share allows"); +} + +/// Note 47: the disk cache's default room grows with the library, so a wedding's previews are not +/// pruned while they are built (the fixed 2 GB held about 2 000 standard previews). +#[test] +fn the_disk_cache_default_grows_with_the_library() { + assert_eq!(crate::media::default_disk_cache_bytes(10), crate::media::DISK_CACHE_BYTES); + assert!(crate::media::default_disk_cache_bytes(4000) >= 4000 * (1 << 20)); + let (mut s, root, ids) = library("diskroom", 2); + assert_eq!(s.cache_bytes(), crate::media::DISK_CACHE_BYTES); + s.cache_mb = 700; + assert_eq!(s.cache_bytes(), 700 << 20, "a size the user chose is kept"); + assert_eq!(ids.len(), 2); + let _ = std::fs::remove_dir_all(root); +} + +/// Notes 47 and 49: a build skips photos whose previews are there (built before, or by the loupe), +/// renders those whose preview is too small, and its previews are on disk when it is done (they +/// were left to a writer that skips writes when it falls behind). +#[test] +fn a_build_skips_what_is_there_and_leaves_it_on_disk() { + let (mut s, root, ids) = library("skip", 4); + let r = s.execute("library.buildPreviews", &json!({"ids": ids_json(&ids), "edge": 300, "wait": true})).unwrap(); + assert_eq!((r["done"].as_u64(), r["skipped"].as_u64(), r["failed"].as_u64()), (Some(4), Some(0), Some(0)), "{r}"); + // on disk now (no waiting for a writer), at the size built + for id in &ids { + let p = s.catalog.photo(*id).unwrap().clone(); + let key = s.view_key(&p, true); + let disk = s.media.rendered.disk().expect("a library has a disk cache"); + let (w, h) = disk.dims(key).expect("the view preview is on disk"); + assert_eq!(w.max(h), 300); + } + // again: nothing is rendered + let r = s.execute("library.buildPreviews", &json!({"ids": ids_json(&ids), "edge": 300, "wait": true})).unwrap(); + assert_eq!((r["done"].as_u64(), r["skipped"].as_u64()), (Some(4), Some(4)), "{r}"); + // forced, or bigger: rendered + let r = s.execute("library.buildPreviews", &json!({"ids": [ids[0].0], "edge": 300, "wait": true, "force": true})).unwrap(); + assert_eq!(r["skipped"], 0, "{r}"); + let r = s.execute("library.buildPreviews", &json!({"ids": ids_json(&ids), "edge": 500, "wait": true})).unwrap(); + assert_eq!(r["skipped"], 0, "too small for what is asked: built again {r}"); + // a library opened again finds them on disk (memory is empty) + drop(s); + let mut s = Session::new().with_fs(); + s.open_library(root.join("lib"), false).unwrap(); + let r = s.execute("library.buildPreviews", &json!({"ids": ids_json(&ids), "edge": 500, "wait": true})).unwrap(); + assert_eq!(r["skipped"], 4, "{r}"); + let r = s.loupe_job(ids[1], 400, 400, true).unwrap().run(); + assert_eq!(r.quick, Some(QuickSource::Cached), "the loupe uses the preview built in the last session"); + let _ = std::fs::remove_dir_all(root); +} + +/// A forced build renders the grid thumbnail again too, not the one its cache holds (CodeRabbit on #3577: +/// the thumbnail job read its cache first, so `force` rebuilt only the view). +#[test] +fn a_forced_build_renders_the_thumbnail_again() { + let (mut s, root, ids) = library("force-thumb", 1); + let build = |s: &mut Session, force: bool| { + s.execute("library.buildPreviews", &json!({"ids": [ids[0].0], "edge": 300, "wait": true, "force": force})).unwrap() + }; + build(&mut s, false); + let jobs = crate::cmd::previews::jobs_for(&mut s, ids[0], Some(300)); + let thumb = jobs.iter().find(|j| j.view_cache.is_none() && j.cache.is_some()).expect("a thumbnail job"); + let ((cache, key), generation) = (thumb.cache.clone().unwrap(), thumb.cache_generation); + let stand_in = lightcraft_raster::Rgba8 { width: 1, height: 1, data: vec![[1, 2, 3, 255]] }; + assert!(cache.put_at(generation, key, std::sync::Arc::new(stand_in))); + let r = build(&mut s, false); + assert_eq!(r["skipped"], 1, "{r}"); + assert_eq!(cache.get_at(generation, key).map(|i| i.width), Some(1), "unforced: kept"); + let r = build(&mut s, true); + assert_eq!(r["failed"], 0, "{r}"); + assert!(cache.get_at(generation, key).is_some_and(|i| i.width > 1), "forced: rendered again"); + let _ = std::fs::remove_dir_all(root); +} + +/// Note 47: standard previews are built at least as big as the loupe on this screen asks for. +#[test] +fn standard_previews_are_as_big_as_the_loupe() { + let mut s = Session::with_demo(); + assert_eq!(crate::cmd::previews::standard_edge(&s), crate::cmd::previews::STANDARD_EDGE); + s.loupe_edge = 3000; + assert_eq!(crate::cmd::previews::standard_edge(&s), 3000); + s.loupe_edge = 1200; + assert_eq!(crate::cmd::previews::standard_edge(&s), crate::cmd::previews::STANDARD_EDGE); +} + +/// Note 48: a background build goes from where the user is, in the direction they step, and +/// follows them when they jump. +#[test] +fn the_background_build_starts_where_the_user_is() { + use crate::cmd::previews::PreviewBuild; + let b = PreviewBuild { order: (10..20).collect(), ..Default::default() }; + let mut queue: Vec> = (10..20).map(Some).collect(); + b.focus_on(PhotoId(14), false); + let mut took = Vec::new(); + for _ in 0..3 { + let i = b.next_index(&queue).unwrap(); + took.push(queue[i].take().unwrap()); + } + assert_eq!(took, vec![14, 15, 16], "forward from the photo on screen"); + b.focus_on(PhotoId(12), true); + let i = b.next_index(&queue).unwrap(); + assert_eq!(queue[i], Some(12), "jumped back: from there, backwards"); + queue[i] = None; + assert_eq!(b.next_index(&queue).map(|i| queue[i]), Some(Some(11))); + // the end reached: the rest the other way + b.focus_on(PhotoId(19), false); + for _ in 0..6 { + let i = b.next_index(&queue).unwrap(); + queue[i] = None; + } + assert!(b.next_index(&queue).is_none(), "all taken"); +} + +/// Note 49: the background preparation gives way to a build that is asked for (no error, no +/// two builds), and is not started again for a view the user stopped it in. +#[test] +fn a_build_asked_for_replaces_the_background_one() { + let (mut s, root, ids) = library("replace", 3); + // held back: the photo on screen is being made all along (held until released below, not for a + // deadline a slow test machine could pass before the build asked for comes) + let hold = crate::prep::TestHold::new(); + s.execute("library.buildPreviews", &json!({"ids": ids_json(&ids), "edge": 300, "background": true})).unwrap(); + let bg = s.preview_build.clone().unwrap(); + assert!(bg.background && bg.running()); + assert!(s.activity.list().iter().any(|t| t.label == "Preparing photos in the background"), "{:?}", s.activity.list()); + // a second background build is refused; one that is asked for replaces it + assert!(s.execute("library.buildPreviews", &json!({"ids": ids_json(&ids), "background": true})).is_err()); + let r = s.execute("library.buildPreviews", &json!({"ids": ids_json(&ids), "edge": 300, "wait": true})).unwrap(); + assert_eq!(r["done"], 3, "{r}"); + assert!(bg.cancel.load(Ordering::Relaxed), "the background build was stopped"); + drop(hold); + // its workers stop before their library is removed under them + assert!(wait_finished(&bg), "a cancelled build stops"); + drop(s); + std::fs::remove_dir_all(&root).unwrap(); +} + +/// Note 48: the background preparation starts once the user steps through photos in the loupe +/// (not when a library opens), builds the view's previews in one cancellable row, and a stop +/// from that row holds for the view. +#[test] +fn the_background_preparation_starts_when_culling_and_can_be_stopped() { + let (mut s, root, ids) = library("prep", 4); + let mode = crate::prep::Mode::Library; + s.prep_tick(mode, None, false); + s.prep_tick(mode, Some(ids[0]), false); + assert!(s.preview_build.is_none(), "opening and looking at one photo starts nothing"); + s.prep_tick(mode, Some(ids[1]), false); + s.prep_tick(mode, Some(ids[2]), false); + let b = s.preview_build.clone().expect("culling: the view is prepared"); + assert!(b.background); + assert_eq!(b.total, 4); + for _ in 0..600 { + s.prep_tick(mode, Some(ids[2]), false); + if !b.running() { + break; + } + std::thread::sleep(std::time::Duration::from_millis(20)); + } + assert!(!b.running(), "built"); + assert_eq!(b.done.load(Ordering::Relaxed), 4); + s.prep_tick(mode, Some(ids[3]), false); + assert!(same_build(&s, &b), "not started again for the same view"); + // switched off: nothing starts in a new view either + s.execute("prefs.setBackgroundPrep", &json!({"settings": {"library": {"previews": false}}})).unwrap(); + let off = s.execute("prefs.backgroundPrep", &json!({})).unwrap(); + assert_eq!(off["settings"]["library"]["previews"], false, "{off}"); + assert_eq!(off["settings"]["culling"]["previews"], true, "each mode has its own"); + let _ = std::fs::remove_dir_all(root); +} + +/// While editing, the whole view's build follows the Library row (its switch and "which photos"); +/// the Editing row's scope is only for the photos kept ready ahead. +#[test] +fn editing_builds_the_view_by_the_library_rows_scope() { + let (mut s, root, ids) = library("prep-develop", 4); + s.execute("prefs.setBackgroundPrep", &json!({"settings": {"develop": {"scope": "onScreen"}, "library": {"scope": "marked"}}})).unwrap(); + s.execute("photo.rate", &json!({"ids": [ids[1].0], "rating": 3})).unwrap(); + let mode = crate::prep::Mode::Develop; + for id in &ids { + s.prep_tick(mode, Some(*id), false); + } + let b = s.preview_build.clone().expect("the Library row asks for the view while editing"); + assert_eq!(b.total, 1, "only the marked photos, as the Library row says"); + b.cancel.store(true, Ordering::Relaxed); + // its workers stop before their library is removed under them + assert!(wait_finished(&b), "a cancelled build stops"); + drop(s); + std::fs::remove_dir_all(&root).unwrap(); +} + +/// With "which photos" = marked, marking another photo changes what the view's build prepares: a +/// build for the newly marked set follows (it wasn't, as the view itself stayed the same). +#[test] +fn marking_a_photo_prepares_it_in_the_marked_scope() { + let (mut s, root, ids) = library("prep-marked", 4); + s.execute("prefs.setBackgroundPrep", &json!({"settings": {"library": {"scope": "marked"}}})).unwrap(); + s.execute("photo.rate", &json!({"ids": [ids[1].0], "rating": 3})).unwrap(); + let mode = crate::prep::Mode::Library; + let tick = |s: &mut Session, k: usize| s.prep_tick(mode, Some(ids[k % 3]), false); + for k in 0..3 { + tick(&mut s, k); + } + let first = s.preview_build.clone().expect("the marked photos are prepared"); + assert_eq!(first.total, 1); + s.execute("photo.rate", &json!({"ids": [ids[3].0], "rating": 2})).unwrap(); + let mut next = None; + for k in 0..600 { + tick(&mut s, k); + if let Some(b) = s.preview_build.clone().filter(|b| !std::sync::Arc::ptr_eq(b, &first)) { + next = Some(b); + break; + } + std::thread::sleep(std::time::Duration::from_millis(20)); + } + let next = next.expect("a build for the newly marked set"); + assert_eq!(next.total, 2); + next.cancel.store(true, Ordering::Relaxed); + // both builds' workers stop before their library is removed under them + for b in [&first, &next] { + assert!(wait_finished(b), "a cancelled build stops"); + } + drop(s); + std::fs::remove_dir_all(&root).unwrap(); +} + +/// Waits (up to 30 s) for a build's workers to finish. +fn wait_finished(b: &crate::cmd::previews::PreviewBuild) -> bool { + for _ in 0..1500 { + if b.finished.load(Ordering::Relaxed) { + return true; + } + std::thread::sleep(std::time::Duration::from_millis(20)); + } + false +} + +/// A background build started for one view is stopped when another view is shown (here the +/// library changes under it), not left building photos no longer shown; it doesn't count as +/// stopped by the user, so the view now shown is prepared. +#[test] +fn a_view_change_stops_the_background_build_for_the_old_view() { + let (mut s, root, ids) = library("prep-view", 6); + let mode = crate::prep::Mode::Library; + // the build's lanes wait at their foreground check until released below + let hold = crate::prep::TestHold::new(); + for id in &ids[..3] { + s.prep_tick(mode, Some(*id), false); + } + let b = s.preview_build.clone().expect("the view is prepared"); + assert!(b.running()); + s.catalog.apply(lightcraft_catalog::Op::RemovePhoto { id: ids[5] }).unwrap(); + s.prep_tick(mode, Some(ids[2]), false); + let stopped = b.cancel.load(Ordering::Relaxed); + drop(hold); + assert!(stopped, "the old view's build is stopped"); + // the user steps on in the view now shown + for k in 0..600 { + s.prep_tick(mode, Some(ids[k % 3]), false); + if s.preview_build.as_ref().is_some_and(|p| !std::sync::Arc::ptr_eq(p, &b)) { + break; + } + std::thread::sleep(std::time::Duration::from_millis(20)); + } + let next = s.preview_build.clone().expect("a build for the view now shown"); + assert!(!std::sync::Arc::ptr_eq(&next, &b)); + assert_eq!(next.total, 5); + next.cancel.store(true, Ordering::Relaxed); + // both builds' workers stop before their library is removed under them + for b in [&b, &next] { + assert!(wait_finished(b), "a cancelled build stops"); + } + drop(s); + std::fs::remove_dir_all(&root).unwrap(); +} + +fn same_build(s: &Session, b: &std::sync::Arc) -> bool { + s.preview_build.as_ref().is_some_and(|p| std::sync::Arc::ptr_eq(p, b)) +} + +/// Note 48: a library keeps its own background settings (a timelapse library prepares nothing), +/// saved with it; the app-wide ones apply elsewhere. +#[test] +fn a_library_can_keep_its_own_background_settings() { + let (mut s, root, _ids) = library("own", 1); + let r = s.execute("prefs.setBackgroundPrep", &json!({"library": true, "settings": {"culling": {"previews": false, "ahead": 0}}})).unwrap(); + assert_eq!(r["own"], true, "{r}"); + assert_eq!(r["settings"]["culling"]["ahead"], 0); + assert_eq!(r["app"]["culling"]["ahead"], 5, "the app-wide ones are unchanged"); + drop(s); + let mut s = Session::new().with_fs(); + s.open_library(root.join("lib"), false).unwrap(); + assert_eq!(s.prep_settings().culling.ahead, 0, "kept with the library"); + assert_eq!(s.prep_settings().ahead(crate::prep::Mode::Culling), 0); + s.execute("prefs.setBackgroundPrep", &json!({"reset": true})).unwrap(); + assert_eq!(s.prep_settings().culling.ahead, 5, "back to the app-wide settings"); + assert!(s.execute("prefs.setBackgroundPrep", &json!({"settings": 3})).is_err()); + let _ = std::fs::remove_dir_all(root); +} + +/// Note 47: Build Previews with one photo open (selected) builds every photo in view, as its menu +/// item says ("Visible"); it built only the open photo, so culling after it found nothing built. +#[test] +fn build_previews_with_one_photo_open_builds_the_view() { + let mut s = Session::with_demo(); + let n = s.visible_cloned().len(); + assert!(n > 2); + let one = s.visible_cloned()[0]; + s.selection = crate::Selection::single(one); + let r = s.execute("library.buildPreviews", &json!({"edge": 256, "wait": true})).unwrap(); + assert_eq!(r["total"], n, "{r}"); + let two: Vec = s.visible_cloned().into_iter().take(2).collect(); + s.selection = crate::Selection { ids: two.clone(), active: two.first().copied() }; + let r = s.execute("library.buildPreviews", &json!({"edge": 256, "wait": true})).unwrap(); + assert_eq!(r["total"], 2, "a choice of photos is built alone {r}"); +} + +/// A view the loupe kept only in memory (its deferred disk write skipped when the writer fell behind) is not +/// counted as built: the build renders it and writes it to disk (CodeRabbit on #3077). +#[test] +fn a_preview_kept_only_in_memory_is_built_and_written() { + let (mut s, root, ids) = library("memonly", 1); + let jobs = crate::cmd::previews::jobs_for(&mut s, ids[0], Some(300)); + assert!(!jobs.is_empty()); + for j in &jobs { + let img = std::sync::Arc::new(lightcraft_raster::Rgba8::new(300, 200)); + if let Some((cache, key)) = &j.view_cache { + cache.put_memory(*key, img); + } else if let Some((cache, key)) = &j.cache { + cache.put_memory(*key, img); + } + } + assert!(!crate::cmd::previews::already_built(&jobs), "a copy only in memory is not a built preview"); + let r = s.execute("library.buildPreviews", &json!({"ids": ids_json(&ids), "edge": 300, "wait": true})).unwrap(); + assert_eq!((r["skipped"].as_u64(), r["failed"].as_u64()), (Some(0), Some(0)), "{r}"); + let p = s.catalog.photo(ids[0]).unwrap().clone(); + let key = s.view_key(&p, true); + assert!(s.media.rendered.disk().and_then(|d| d.dims(key)).is_some(), "the view preview is on disk"); + let _ = std::fs::remove_dir_all(root); +} + +/// A view preview the cache folder can't take (full, or not writable) fails the photo instead of counting it +/// as built (CodeRabbit on #3077). +#[test] +fn a_preview_that_cannot_be_written_fails_the_build() { + let (mut s, root, ids) = library("unwritable", 2); + let dir = s.media.rendered.disk().expect("a library has a disk cache").dir().to_path_buf(); + let _ = std::fs::remove_dir_all(&dir); + // a file where the cache folder goes: nothing can be written under it + std::fs::write(&dir, b"not a folder").unwrap(); + let r = s.execute("library.buildPreviews", &json!({"ids": ids_json(&ids), "edge": 300, "wait": true})).unwrap(); + assert_eq!(r["failed"].as_u64(), Some(2), "{r}"); + let _ = std::fs::remove_file(&dir); + let _ = std::fs::remove_dir_all(root); +} + +/// A photo whose preview cache is discarded while it renders is not counted as built: its preview isn't kept +/// (CodeRabbit on #3077). +#[test] +fn a_preview_rendered_for_a_discarded_cache_fails_the_photo() { + let (mut s, root, ids) = library("discarded", 1); + let jobs = crate::cmd::previews::jobs_for(&mut s, ids[0], Some(300)); + s.execute("library.clearPreviews", &json!({})).unwrap(); + assert!(!crate::cmd::previews::run_photo(jobs), "nothing was kept: the photo is not built"); + let jobs = crate::cmd::previews::jobs_for(&mut s, ids[0], Some(300)); + assert!(crate::cmd::previews::run_photo(jobs), "built for the cache as it is now"); + let _ = std::fs::remove_dir_all(root); +} + +/// A forced build renders a photo itself even when another build was rendering it and made its previews +/// (CodeRabbit on #3080): the other build's result is not taken as this one's. +#[test] +fn a_forced_build_renders_a_photo_another_build_was_rendering() { + let (mut s, root, ids) = library("forced-shared", 1); + s.execute("library.buildPreviews", &json!({"ids": ids_json(&ids), "edge": 300, "wait": true})).unwrap(); + let jobs = crate::cmd::previews::jobs_for(&mut s, ids[0], Some(300)); + let held = crate::cmd::previews::hold_building(ids[0].0, &jobs).expect("nobody renders it now"); + s.execute("library.buildPreviews", &json!({"ids": ids_json(&ids), "edge": 300, "force": true})).unwrap(); + let b = s.preview_build.clone().expect("the build"); + std::thread::sleep(std::time::Duration::from_millis(300)); + assert_eq!(b.done.load(Ordering::Relaxed), 0, "not counted while the other build holds it"); + drop(held); + assert!(wait_finished(&b)); + let counts = + (b.done.load(Ordering::Relaxed), b.failed.load(Ordering::Relaxed), b.shared.load(Ordering::Relaxed), b.skipped.load(Ordering::Relaxed)); + assert_eq!(counts, (1, 0, 0, 0), "rendered here, not taken from the other build"); + let _ = std::fs::remove_dir_all(root); +} + +/// A photo another build is rendering is not counted until that render is over: here the other build never +/// made its previews, so this build renders it (CodeRabbit on #3080). +#[test] +fn a_photo_another_build_is_rendering_is_counted_only_once_it_is_built() { + let (mut s, root, ids) = library("shared", 1); + let jobs = crate::cmd::previews::jobs_for(&mut s, ids[0], Some(300)); + let held = crate::cmd::previews::hold_building(ids[0].0, &jobs).expect("nobody renders it yet"); + s.execute("library.buildPreviews", &json!({"ids": ids_json(&ids), "edge": 300})).unwrap(); + let b = s.preview_build.clone().expect("the build"); + std::thread::sleep(std::time::Duration::from_millis(300)); + assert_eq!(b.done.load(Ordering::Relaxed), 0, "not counted while the other build holds it"); + drop(held); + assert!(wait_finished(&b)); + assert_eq!((b.done.load(Ordering::Relaxed), b.failed.load(Ordering::Relaxed), b.shared.load(Ordering::Relaxed)), (1, 0, 0)); + let p = s.catalog.photo(ids[0]).unwrap().clone(); + let key = s.view_key(&p, true); + assert!(s.media.rendered.disk().and_then(|d| d.dims(key)).is_some(), "rendered here, on disk"); + let _ = std::fs::remove_dir_all(root); +} + +/// CodeRabbit on #3544: a library preparation setting that can't be saved is not kept, nor is +/// going back to the app-wide settings: the session stays as the command's error says. +#[test] +fn a_prep_setting_that_cant_be_saved_is_not_kept() { + use crate::prep::PrepSettings; + let (mut s, root, _) = library("prep-unsaved", 1); + let mut own = PrepSettings::default(); + own.culling.ahead = 3; + s.set_prep_settings(own, true).unwrap(); + // prefs.json can't be written: a folder stands in its place + let prefs = root.join("lib").join("prefs.json"); + let _ = std::fs::remove_file(&prefs); + std::fs::create_dir_all(prefs.join("in-the-way")).unwrap(); + let mut other = own; + other.culling.ahead = 5; + assert!(s.set_prep_settings(other, true).is_err()); + assert_eq!(s.prep_json()["library"], json!(own), "the setting that couldn't be saved is not kept"); + assert!(s.clear_library_prep().is_err()); + assert_eq!(s.prep_json()["library"], json!(own), "nor going back to the app-wide settings"); + let _ = std::fs::remove_dir_all(&root); +} + +/// CodeRabbit on #3544: a build asked for with malformed ids fails without stopping the +/// background preparation that is running. +#[test] +fn a_bad_build_request_leaves_the_background_build_running() { + let _hold = crate::prep::TestHold::new(); + let (mut s, root, ids) = library("prep-bad-ids", 3); + s.execute("library.buildPreviews", &json!({"ids": ids_json(&ids), "background": true})).unwrap(); + let b = s.preview_build.clone().expect("the background build"); + assert!(b.running()); + assert!(s.execute("library.buildPreviews", &json!({"ids": ["x"]})).is_err(), "ids that are not photo ids"); + assert!(!b.cancel.load(std::sync::atomic::Ordering::Relaxed), "the background build was not stopped"); + let _ = std::fs::remove_dir_all(&root); +} + +/// CodeRabbit on #3577: smart previews asked for in the background with malformed ids fail without +/// stopping the background preparation either. +#[test] +fn a_bad_smart_preview_request_leaves_the_background_build_running() { + let _hold = crate::prep::TestHold::new(); + let (mut s, root, ids) = library("prep-bad-smart-ids", 3); + s.execute("library.buildPreviews", &json!({"ids": ids_json(&ids), "background": true})).unwrap(); + let b = s.preview_build.clone().expect("the background build"); + assert!(b.running()); + assert!(s.execute("library.smartPreviews", &json!({"ids": ["x"], "background": true})).is_err(), "ids that are not photo ids"); + assert!(!b.cancel.load(std::sync::atomic::Ordering::Relaxed), "the background build was not stopped"); + // a smart-preview worker that can't start: the background build carries on, and stays the session's + crate::cmd::previews::test_gate::fail_next_spawn(); + assert!(s.execute("library.smartPreviews", &json!({"ids": ids_json(&ids), "background": true})).is_err(), "the worker didn't start"); + assert!(!b.cancel.load(std::sync::atomic::Ordering::Relaxed), "the background build was not stopped"); + assert!(s.preview_build.as_ref().is_some_and(|now| std::sync::Arc::ptr_eq(now, &b)), "and is still the session's build"); + // (stopped and finished before its folder goes) + b.cancel.store(true, std::sync::atomic::Ordering::Relaxed); + drop(_hold); + let until = std::time::Instant::now() + std::time::Duration::from_secs(20); + while b.running() && std::time::Instant::now() < until { + std::thread::sleep(std::time::Duration::from_millis(5)); + } + let _ = std::fs::remove_dir_all(&root); +} + +/// CodeRabbit on #3544: a build asked for whose worker can't start fails without stopping or +/// replacing the background preparation that is running. +#[test] +fn a_build_that_cant_start_leaves_the_background_build_running() { + let _hold = crate::prep::TestHold::new(); + let (mut s, root, ids) = library("prep-no-thread", 3); + s.execute("library.buildPreviews", &json!({"ids": ids_json(&ids), "background": true})).unwrap(); + let b = s.preview_build.clone().expect("the background build"); + assert!(b.running()); + crate::cmd::previews::test_gate::fail_next_spawn(); + assert!(s.execute("library.buildPreviews", &json!({"ids": ids_json(&ids)})).is_err(), "the worker didn't start"); + assert!(!b.cancel.load(std::sync::atomic::Ordering::Relaxed), "the background build was not stopped"); + assert!(s.preview_build.as_ref().is_some_and(|now| std::sync::Arc::ptr_eq(now, &b)), "and is still the session's build"); + let _ = std::fs::remove_dir_all(&root); +} + +/// CodeRabbit on #3080: a background build that can't start isn't tried again on every tick; the +/// view tries again once [`crate::prep::RETRY_AFTER`] has passed. +#[test] +fn a_build_that_cant_start_waits_before_trying_again() { + let (mut s, root, ids) = library("prep-retry", 3); + let mode = crate::prep::Mode::Library; + s.prep.fail_starts = 1; + for id in &ids { + s.prep_tick(mode, Some(*id), false); + } + assert!(s.preview_build.is_none(), "the start failed"); + assert!(s.prep.retry_at.is_some()); + // the app is told when to look again (a frame is asked for then) + assert!(s.prep_retry_in().is_some_and(|d| d <= crate::prep::RETRY_AFTER && d > crate::prep::RETRY_AFTER / 2)); + s.prep_tick(mode, Some(ids[2]), false); + assert!(s.preview_build.is_none(), "not tried again at once"); + // the wait is over: no more frames asked for it (CodeRabbit on #3077 / #3080) + s.prep.retry_at = s.prep.retry_at.map(|(view, _)| (view, web_time::Instant::now())); + assert_eq!(s.prep_retry_in(), None, "a deadline that has passed asks for no frame"); + s.prep_tick(mode, Some(ids[2]), false); + let b = s.preview_build.clone().expect("tried again"); + assert!(b.background && s.prep.retry_at.is_none()); + b.cancel.store(true, Ordering::Relaxed); + let _ = wait_finished(&b); + let _ = std::fs::remove_dir_all(&root); +} diff --git a/crates/preview/src/disk.rs b/crates/preview/src/disk.rs index ca82870483..a1395a04ca 100644 --- a/crates/preview/src/disk.rs +++ b/crates/preview/src/disk.rs @@ -172,15 +172,43 @@ impl DiskCache { } } - pub fn put(&self, key: Hash128, img: &Rgba8) { - let Some(bytes) = encode_jpeg(img) else { return }; + /// The size (width, height) of the picture kept under `key`, read from the start of its file + /// only (no decode, no hit counted, nothing written): to tell whether a preview is there and + /// big enough without loading it. + pub fn dims(&self, key: Hash128) -> Option<(usize, usize)> { + use std::io::Read; let p = self.path(key); - let Some(parent) = p.parent() else { return }; + if !p.parent().is_some_and(is_real_dir) { + return None; + } + let mut f = std::fs::File::open(&p).or_else(|_| std::fs::File::open(self.named(key, OLD_EXT))).ok()?; + // the size is normally in the first few KiB: more is read only when that is not enough + let mut head = Vec::new(); + for limit in [FIRST_HEADER_BYTES, HEADER_BYTES] { + let more = limit.saturating_sub(head.len() as u64); + (&mut f).take(more).read_to_end(&mut head).ok()?; + if let Some(dims) = jpeg_dims(&head) { + return Some(dims); + } + if (head.len() as u64) < limit { + // the whole file was read + break; + } + } + None + } + + /// Write `img` under `key`. Returns whether the file was written (a full or unwritable cache folder + /// gives `false`; the cache is unchanged then). + pub fn put(&self, key: Hash128, img: &Rgba8) -> bool { + let Some(bytes) = encode_jpeg(img) else { return false }; + let p = self.path(key); + let Some(parent) = p.parent() else { return false }; // a folder the cache makes for itself says that it is a cache; one that was there // already may hold somebody's files, and is not declared one let made = !self.dir.exists(); if std::fs::create_dir_all(parent).is_err() || !is_real_dir(parent) { - return; + return false; } if made { let _ = std::fs::write(self.dir.join(TAG), TAG_TEXT); @@ -197,7 +225,7 @@ impl DiskCache { let written = std::fs::OpenOptions::new().write(true).create_new(true).open(&tmp).and_then(|mut f| std::io::Write::write_all(&mut f, &bytes)); if written.is_err() || std::fs::rename(&tmp, &p).is_err() { let _ = std::fs::remove_file(&tmp); - return; + return false; } self.writes.fetch_add(1, Ordering::Relaxed); *used = used.saturating_sub(old_len).saturating_add(bytes.len() as u64); @@ -205,7 +233,10 @@ impl DiskCache { drop(total); if over { self.prune(); + // (a picture bigger than the room pruning leaves goes too: it was not kept) + return p.is_file(); } + true } /// All cache files (only names of the cache's own shape, see [`is_cache_file`]): @@ -374,6 +405,12 @@ impl DiskCache { *t.get_or_insert_with(|| self.scan().iter().map(|f| f.1).sum()) } + /// Bytes on disk when already known, without reading the folder (`None` until [`Self::size`], + /// a write or a prune has counted them). + pub fn known_size(&self) -> Option { + *self.total.lock().unwrap_or_else(|e| e.into_inner()) + } + pub fn clear(&self) { for (p, _, _) in self.scan() { let _ = std::fs::remove_file(p); @@ -432,6 +469,22 @@ pub fn encode_jpeg(img: &Rgba8) -> Option> { Some(out) } +/// How much of a cached file [`DiskCache::dims`] reads: the JPEG headers this cache writes come +/// first and are a few hundred bytes. +const HEADER_BYTES: u64 = 64 << 10; +/// What [`DiskCache::dims`] reads first, which holds the size of the JPEGs this cache writes. +const FIRST_HEADER_BYTES: u64 = 4 << 10; + +/// A JPEG's (width, height) from its headers (`None`: not a JPEG, or cut short before its frame header). +pub fn jpeg_dims(bytes: &[u8]) -> Option<(usize, usize)> { + use zune_core::bytestream::ZCursor; + let mut d = zune_jpeg::JpegDecoder::new(ZCursor::new(bytes)); + d.decode_headers().ok()?; + let info = d.info()?; + let (w, h) = (info.width as usize, info.height as usize); + (w > 0 && h > 0).then_some((w, h)) +} + /// Decode a cached thumbnail (see [`encode_jpeg`]). pub fn decode_jpeg(bytes: &[u8]) -> Option { use zune_core::bytestream::ZCursor; diff --git a/crates/preview/src/lib.rs b/crates/preview/src/lib.rs index 14bd50a1b3..3c5485ae8f 100644 --- a/crates/preview/src/lib.rs +++ b/crates/preview/src/lib.rs @@ -21,7 +21,7 @@ use std::path::Path; use std::sync::atomic::{AtomicUsize, Ordering}; use std::sync::{Arc, Mutex, RwLock}; -pub use disk::{DiskCache, decode_jpeg, encode_jpeg}; +pub use disk::{DiskCache, decode_jpeg, encode_jpeg, jpeg_dims}; pub use hash::{Hash128, Hasher128, hash_bytes}; use lightcraft_raster::Rgba8; pub use lru::{Lru, next_tick}; @@ -108,20 +108,80 @@ impl PreviewCache { Some(img) } - pub fn put(&self, key: Hash128, img: Arc) { - self.put_at(self.generation(), key, img); + /// The size of the image kept under `key` (memory, else disk: read from its file's header), without + /// loading it or changing what is kept. + pub fn dims(&self, key: Hash128) -> Option<(usize, usize)> { + if self.generation.read().unwrap_or_else(std::sync::PoisonError::into_inner).is_none() { + return None; + } + if let Some(v) = self.mem.lock().unwrap_or_else(|e| e.into_inner()).peek(&key) { + return Some((v.width, v.height)); + } + self.disk.as_ref()?.dims(key) } - pub fn put_at(&self, generation: u64, key: Hash128, img: Arc) { + /// The size of the image kept under `key` where it lasts: on disk when the cache has a disk (a copy only + /// in memory, whose deferred write may have been skipped, doesn't count), else in memory. Nothing is + /// loaded or changed. + pub fn stored_dims(&self, key: Hash128) -> Option<(usize, usize)> { + match &self.disk { + Some(d) if self.generation.read().unwrap_or_else(std::sync::PoisonError::into_inner).is_some() => d.dims(key), + Some(_) => None, + None => self.dims(key), + } + } + + /// The memory LRU's copy of `key`, without looking on disk. + pub fn get_memory(&self, key: Hash128) -> Option> { + if self.generation.read().unwrap_or_else(std::sync::PoisonError::into_inner).is_none() { + return None; + } + self.mem.lock().unwrap_or_else(|e| e.into_inner()).get(&key).cloned() + } + + /// Keep `img` in memory only (it is never written to disk): for pictures that are cheap + /// enough to make again, such as a raw's decoded camera JPEG, but worth keeping for a revisit. + pub fn put_memory(&self, key: Hash128, img: Arc) { + // (the guard is held through the insertion: a clear can't come between the look and the insertion) + let current = self.generation.read().unwrap_or_else(std::sync::PoisonError::into_inner); + if current.is_none() { + return; + } + let c = cost(&img); + self.mem.lock().unwrap_or_else(|e| e.into_inner()).insert(key, img, c); + drop(current); + } + + /// [`Self::put_memory`] for a job that captured `generation`: nothing is kept once the cache + /// was cleared (or retired) after that, so a job started before Discard Preview Cache can't + /// put its picture back. + pub fn put_memory_at(&self, generation: u64, key: Hash128, img: Arc) { + // (the guard is held through the insertion: a clear can't come between the look and the insertion) let current = self.generation.read().unwrap_or_else(std::sync::PoisonError::into_inner); if *current != Some(generation) { return; } - if let Some(d) = &self.disk { - d.put(key, &img); - } let c = cost(&img); self.mem.lock().unwrap_or_else(|e| e.into_inner()).insert(key, img, c); + drop(current); + } + + pub fn put(&self, key: Hash128, img: Arc) { + let _ = self.put_at(self.generation(), key, img); + } + + /// [`Self::put`] for a job of `generation`. Returns whether the image is kept where it lasts: `false` when + /// the cache was cleared or retired since the job was made (nothing is kept; see [`Self::generation`] to + /// tell), or when the disk cache could not write the file (the image is kept in memory all the same). + pub fn put_at(&self, generation: u64, key: Hash128, img: Arc) -> bool { + let current = self.generation.read().unwrap_or_else(std::sync::PoisonError::into_inner); + if *current != Some(generation) { + return false; + } + let written = self.disk.as_ref().is_none_or(|d| d.put(key, &img)); + let c = cost(&img); + self.mem.lock().unwrap_or_else(|e| e.into_inner()).insert(key, img, c); + written } /// Like [`Self::put`], but the disk write (a JPEG encode) happens on a background thread, so diff --git a/crates/preview/src/tests.rs b/crates/preview/src/tests.rs index 2f07e927fb..975c2ddf79 100644 --- a/crates/preview/src/tests.rs +++ b/crates/preview/src/tests.rs @@ -792,3 +792,63 @@ fn deferred_writes_are_bounded_and_drain_to_disk() { assert!(keys.iter().all(|k| cache.get(*k).is_some())); let _ = std::fs::remove_dir_all(&dir); } + +/// `put_at` says whether the image is kept where it lasts: not for a job made before the cache was cleared or +/// retired, nor when the disk cache can't write it (CodeRabbit on #3077). +#[test] +fn put_at_says_whether_the_image_was_kept() { + let dir = temp_dir("put-at-kept"); + let cache = PreviewCache::with_disk(1 << 20, &dir, 1 << 20); + let (key, img) = (hash_bytes(b"kept"), Arc::new(gradient(8, 8, 3))); + let old = cache.generation(); + assert!(cache.put_at(old, key, img.clone()), "kept"); + cache.clear(); + assert!(!cache.put_at(old, key, img.clone()), "a job from before the clear keeps nothing"); + assert!(cache.put_at(cache.generation(), key, img.clone())); + let generation = cache.generation(); + cache.retire(); + assert!(!cache.put_at(generation, key, img.clone()), "nor one for a retired cache"); + // a file where the cache folder goes: the write fails + let blocked = temp_dir("put-at-blocked"); + std::fs::write(&blocked, b"not a folder").unwrap(); + let cache = PreviewCache::with_disk(1 << 20, &blocked, 1 << 20); + assert!(!cache.put_at(cache.generation(), key, img.clone()), "not written"); + let memory = PreviewCache::memory(1 << 20); + assert!(memory.put_at(memory.generation(), key, img), "a memory-only cache keeps it"); + let _ = std::fs::remove_file(&blocked); + let _ = std::fs::remove_dir_all(&dir); +} + +/// `dims` reads a cached file's first few KiB, and further when its size marker comes later (CodeRabbit on #3080). +#[test] +fn dims_finds_a_size_marker_past_the_first_few_kib() { + let dir = temp_dir("dims-far"); + let d = DiskCache::new(&dir, 1 << 20); + let key = hash_bytes(b"far"); + assert!(d.put(key, &gradient(40, 30, 5))); + assert_eq!(d.dims(key), Some((40, 30))); + // the same picture with a 10 KiB application segment before its frame header + let file = files_under(&dir).into_iter().find(|f| f.ends_with(disk::EXT)).expect("the cached file"); + let path = dir.join(file); + let jpeg = std::fs::read(&path).unwrap(); + let mut padded = jpeg[..2].to_vec(); + let pad = 10 << 10; + padded.extend_from_slice(&[0xFF, 0xEF, ((pad + 2) >> 8) as u8, ((pad + 2) & 0xFF) as u8]); + padded.extend(std::iter::repeat_n(0u8, pad)); + padded.extend_from_slice(&jpeg[2..]); + std::fs::write(&path, padded).unwrap(); + assert_eq!(d.dims(key), Some((40, 30)), "found past the first read"); + let _ = std::fs::remove_dir_all(&dir); +} + +/// A picture bigger than the room pruning leaves is removed by the pruning its own write starts: `put` says it +/// was not kept (CodeRabbit on #3077). +#[test] +fn a_write_the_pruning_removes_is_not_reported_as_kept() { + let dir = temp_dir("pruned-away"); + let d = DiskCache::new(&dir, 64); + assert!(!d.put(hash_bytes(b"too big"), &gradient(64, 64, 9)), "pruned away at once"); + let roomy = DiskCache::new(&dir, 1 << 20); + assert!(roomy.put(hash_bytes(b"fits"), &gradient(64, 64, 9))); + let _ = std::fs::remove_dir_all(&dir); +} diff --git a/crates/raw/src/preview.rs b/crates/raw/src/preview.rs index 2b16c441ad..d661439780 100644 --- a/crates/raw/src/preview.rs +++ b/crates/raw/src/preview.rs @@ -138,12 +138,68 @@ fn is_jxl(b: &[u8]) -> bool { b.starts_with(&[0xff, 0x0a]) || b.starts_with(&[0, 0, 0, 0x0c, b'J', b'X', b'L', b' ', 0x0d, 0x0a, 0x87, 0x0a]) } -fn candidates<'a>(data: &'a [u8], ifd: &Ifd, base: u64, out: &mut Vec<&'a [u8]>) { +/// The bytes a file's own (offset, length) pair names, cut at the file's end; `None` when the offset +/// lies past it. Both numbers come from the file: never cast them (a `u64` offset past 4 GiB would +/// wrap to a small, wrong one on a 32-bit build) nor add them unchecked. A span the file's end cuts +/// short is kept only when it is a JPEG whose own end marker is inside it, after its scan (an +/// overstated length with the whole JPEG inside), cut at that marker: a truncated JPEG passes the +/// frame-header check, would win as the largest preview and then fail to decode (CodeRabbit on +/// #3077). An `FF D9` inside an APP payload doesn't count, bytes that don't start a JPEG are refused +/// without a scan, and the walks share `budget` (bytes, as [`scan_with_budget`]'s do), so a file of +/// many such pointers stays linear. +fn file_span<'a>(data: &'a [u8], offset: u64, len: u64, budget: &mut usize) -> Option<&'a [u8]> { + let start = usize::try_from(offset).ok()?; + let wanted = usize::try_from(offset.saturating_add(len)).unwrap_or(usize::MAX); + let span = data.get(start..wanted.min(data.len()))?; + if wanted > data.len() { + return span.get(..jpeg_end(span, budget)?); + } + Some(span) +} + +/// Where the JPEG at the start of `s` ends (just past its end marker), walking its marker segments: +/// a segment's payload is skipped by its stated length (so an `FF D9` inside APP data isn't taken +/// for the end), entropy-coded bytes one at a time (stuffed `FF 00` and restart markers included). +/// `None` when `s` isn't a JPEG, ends first, reaches its end marker before a start of scan (no image +/// data), or the walk spends `budget` (one per step, shared by a file's walks). Each step moves +/// forward, so it ends within `s`. +fn jpeg_end(s: &[u8], budget: &mut usize) -> Option { + if !s.starts_with(&[0xff, 0xd8]) { + return None; + } + let (mut i, mut scanned) = (2usize, false); + loop { + *budget = budget.checked_sub(1)?; + match (s.get(i), s.get(i.checked_add(1)?)) { + (None, _) | (Some(&0xff), None) => return None, + (Some(&0xff), Some(&m)) => match m { + 0xd9 => return if scanned { i.checked_add(2) } else { None }, + // fill byte + 0xff => i += 1, + // stuffed zero, restart marker, TEM: no length follows + 0x00 | 0x01 | 0xd0..=0xd7 => i = i.checked_add(2)?, + _ => { + let (&hi, &lo) = (s.get(i.checked_add(2)?)?, s.get(i.checked_add(3)?)?); + let len = usize::from(u16::from_be_bytes([hi, lo])); + if len < 2 { + return None; + } + scanned |= m == 0xda; + i = i.checked_add(2)?.checked_add(len)?; + } + }, + // entropy-coded data + (Some(_), _) => i += 1, + } + } +} + +fn candidates<'a>(data: &'a [u8], ifd: &Ifd, base: u64, out: &mut Vec<&'a [u8]>, budget: &mut usize) { // whole JPEG files stored as an undefined-type tag value (e.g. Panasonic `JpgFromRaw` 0x002e) for e in &ifd.entries { if matches!(e.value, lightcraft_tiff::Value::Undefined(_)) && e.count() > 1024 - && let Some(s) = data.get(e.offset as usize..(e.offset.saturating_add(e.count() as u64) as usize).min(data.len())) + && let Some(s) = file_span(data, e.offset, e.count() as u64, budget) && s.starts_with(&[0xff, 0xd8]) { out.push(s); @@ -151,7 +207,7 @@ fn candidates<'a>(data: &'a [u8], ifd: &Ifd, base: u64, out: &mut Vec<&'a [u8]>) } if let (Some(off), Some(len)) = (ifd.u64(t::JPEG_INTERCHANGE_FORMAT), ifd.u64(t::JPEG_INTERCHANGE_FORMAT_LENGTH)) { let off = off.saturating_add(base); - if let Some(s) = data.get(off as usize..(off.saturating_add(len) as usize).min(data.len())) { + if let Some(s) = file_span(data, off, len, budget) { out.push(s); } } @@ -196,8 +252,12 @@ fn smallest_covering<'a>(found: &[&'a [u8]], largest: &[u8], min_edge: u32) -> O let (a, b) = (u64::from(w) * u64::from(bh), u64::from(h) * u64::from(bw)); w > 0 && h > 0 && c == bc && a.abs_diff(b) * 100 <= a }; + // (one cut short before its end marker would fail to decode, and nothing would try another: CodeRabbit + // on #3583) + let whole = |s: &[u8]| trim_eoi(s).ends_with(&[0xff, 0xd9]); found .iter() + .filter(|s| whole(s)) .filter_map(|&s| jpeg_frame(s).filter(|&f| like(f) && f.0.max(f.1) >= min_edge).map(|(w, h, _)| (s, w.max(h)))) .min_by_key(|&(_, long)| long) .map(|(s, _)| s) @@ -210,6 +270,20 @@ fn preview_covering(bytes: &[u8], min_edge: Option) -> Option> { } let format = crate::probe(bytes); if format == Some(crate::RawFormat::Cr3) { + // a Canon CR3 keeps its full-size JPEG beside a 1620 px `PRVW` (and a 160 px `THMB`): a + // thumbnail decodes the small copy, not 45 MP of JPEG (about 150 ms a photo, the grid's + // "popcorn" on a folder of R5 / R5 C raws) + // (without a JPEG track, the largest box is the picture's shape: the boxes alone may hold several) + if let Some(min) = min_edge { + let mut found = cr3_preview_box_list(bytes); + let full = cr3_full_jpeg(bytes); + found.extend(full); + if let Some(largest) = full.or_else(|| largest_by_pixels(&found)) + && let Some(j) = smallest_covering(&found, largest, min) + { + return Some(trim_eoi(j).to_vec()); + } + } return cr3_preview(bytes).map(|j| trim_eoi(j).to_vec()); } if matches!(format, Some(crate::RawFormat::Crw | crate::RawFormat::Mrw | crate::RawFormat::X3f)) { @@ -217,8 +291,10 @@ fn preview_covering(bytes: &[u8], min_edge: Option) -> Option> { } let tiff = Tiff::parse(bytes).ok()?; let mut found: Vec<&[u8]> = Vec::new(); + // the walks of pointers past the file's end share one budget (see file_span) + let mut budget = scan_budget(bytes.len()); for ifd in tiff.all_ifds() { - candidates(bytes, ifd, 0, &mut found); + candidates(bytes, ifd, 0, &mut found, &mut budget); } // maker-note preview IFDs (e.g. Nikon PreviewIFD 0x0011 holds JPEGInterchangeFormat relative to the note base) if let Some(exif) = tiff.exif() @@ -226,11 +302,11 @@ fn preview_covering(bytes: &[u8], min_edge: Option) -> Option> { { let make = tiff.find(t::MAKE).and_then(|e| e.value.as_str()).unwrap_or_default(); if let Some(mn) = makernote::parse_makernote(bytes, e.offset, e.count() as u64, tiff.order, &make) { - candidates(bytes, &mn.ifd, mn.base, &mut found); + candidates(bytes, &mn.ifd, mn.base, &mut found, &mut budget); if let Some(off) = mn.ifd.u64(0x0011) && let Ok((pifd, _)) = lightcraft_tiff::parse_ifd_at(bytes, mn.base + off, mn.order, mn.base, false, &Default::default()) { - candidates(bytes, &pifd, mn.base, &mut found); + candidates(bytes, &pifd, mn.base, &mut found, &mut budget); } } } @@ -442,53 +518,123 @@ fn scan_with_budget(data: &[u8], budget: usize) -> (Option>, u64) { /// Canon CR3: the full-size JPEG track (see [`lightcraft_meta::cr3`]), else the `PRVW` / `THMB` boxes. fn cr3_preview(bytes: &[u8]) -> Option<&[u8]> { + cr3_full_jpeg(bytes).or_else(|| cr3_preview_boxes(bytes)) +} + +/// A CR3's full-size JPEG track, when it holds a DCT JPEG. +fn cr3_full_jpeg(bytes: &[u8]) -> Option<&[u8]> { let full = lightcraft_meta::cr3::parse_cr3(bytes) .and_then(|c| c.tracks.iter().find(|t| t.kind == lightcraft_meta::cr3::Cr3TrackKind::Jpeg).and_then(|t| t.data)); - if let Some(j) = full.and_then(|(at, len)| bytes.get(at..at.checked_add(len)?)).filter(|j| is_dct_jpeg(j)) { - return Some(j); - } - cr3_preview_boxes(bytes) + full.and_then(|(at, len)| bytes.get(at..at.checked_add(len)?)).filter(|j| is_dct_jpeg(j)) } -/// Canon CR3 (ISO base media file): the `PRVW` box (Laurent Clévy's CR3 notes; layout confirmed on a CC0 -/// sample) is `u32 size, "PRVW", u32 0, u16 ?, u16 width, u16 height, u16 ?, u32 jpeg length, JPEG`; the smaller -/// `THMB` box has the same shape. Returns the larger valid one. /// Marker-walk budget of the CR3 fallback: candidates can overlap (each claiming the rest of the file) and /// [`is_dct_jpeg`] walks each marker by marker, so walking every candidate would be quadratic in file size. /// The `THMB`/`PRVW` previews this finds are far below the cap together; once it is spent the scan stops. const CR3_MAX_WALK: usize = 8 << 20; +/// Canon CR3 (ISO base media file): the `PRVW` box (Laurent Clévy's CR3 notes; layout confirmed on a CC0 +/// sample) is `u32 size, "PRVW", u32 0, u16 ?, u16 width, u16 height, u16 ?, u32 jpeg length, JPEG`; the smaller +/// `THMB` box has the same shape. Returns the larger valid one. fn cr3_preview_boxes(bytes: &[u8]) -> Option<&[u8]> { - let mut best: Option<&[u8]> = None; + largest_by_pixels(&cr3_preview_box_list(bytes)) +} + +/// Of `found`, the JPEG with the longest edge in pixels (a smaller picture can take more bytes); the +/// first of equals. CodeRabbit on #3077. +fn largest_by_pixels<'a>(found: &[&'a [u8]]) -> Option<&'a [u8]> { + let long = |j: &[u8]| jpeg_frame(j).map_or(0, |(w, h, _)| w.max(h)); + found.iter().copied().rev().max_by_key(|j| long(j)) +} + +/// The valid JPEGs of a CR3's `PRVW` and `THMB` boxes, every one: a file may carry more than one +/// `PRVW`, and the smallest that covers a size is picked from them all. +fn cr3_preview_box_list(bytes: &[u8]) -> Vec<&[u8]> { + let mut out: Vec<&[u8]> = Vec::new(); let mut walked = 0usize; + // the boxes sit outside the media data: only the rest of the file is searched (the media data + // is most of the file, and a thumbnail is asked for every photo in view) + let file = bytes; + // (every top-level `mdat` is left out, not only the first: a PRVW-shaped JPEG in media data is not a + // preview; CodeRabbit on #3583) + let media = cr3_media_data(file); + let whole_file = media.is_empty(); + let mut regions: Vec<&[u8]> = Vec::new(); + let mut from = 0usize; + for (start, end) in media { + if start >= from { + regions.extend(file.get(from..start)); + } + from = from.max(end); + } + regions.extend(file.get(from..)); for tag in [b"PRVW", b"THMB"] { - let mut from = 0; - while let Some(i) = bytes.get(from..).and_then(|s| s.windows(4).position(|w| w == tag)).map(|p| p + from) { - from = i + 4; - let Some(start) = i.checked_sub(4) else { continue }; - let be32 = |at: usize| bytes.get(at..at + 4).map(|s| u32::from_be_bytes([s[0], s[1], s[2], s[3]]) as usize); - let (Some(size), Some(len)) = (be32(start), be32(start + 20)) else { continue }; - let j = start + 24; - // 32-bit box lengths can wrap `usize` on wasm32: check the sums, then slice with `get` - let (Some(end), Some(lim)) = (j.checked_add(len), start.checked_add(size.max(24))) else { continue }; - if len < 4 || end > lim { - continue; - } - let Some(jpeg) = bytes.get(j..end) else { continue }; - // One that doesn't fit the budget left is skipped; a smaller one later may still fit. - if walked + jpeg.len() > CR3_MAX_WALK { - continue; - } - walked += jpeg.len(); - if is_dct_jpeg(jpeg) { - if best.is_none_or(|b| b.len() < jpeg.len()) { - best = Some(jpeg); - } + // the whole file searched (its media data not found): the first valid box of a kind only, as + // the declared one comes before the media data and a later look-alike must not win + let mut taken = false; + for &bytes in ®ions { + if whole_file && taken { break; } + let mut from = 0; + while let Some(i) = bytes.get(from..).and_then(|s| s.windows(4).position(|w| w == tag)).map(|p| p + from) { + from = i + 4; + let Some(start) = i.checked_sub(4) else { continue }; + // (the sizes are the file's: checked, a 32-bit build must not wrap them) + let be32 = |at: usize| bytes.get(at..at.checked_add(4)?).map(|s| u32::from_be_bytes([s[0], s[1], s[2], s[3]]) as usize); + let (Some(size), Some(len)) = (be32(start), start.checked_add(20).and_then(be32)) else { continue }; + let (Some(j), Some(box_end)) = (start.checked_add(24), start.checked_add(size.max(24))) else { continue }; + let Some(end) = j.checked_add(len).filter(|e| len >= 4 && *e <= box_end && *e <= bytes.len()) else { continue }; + let Some(jpeg) = bytes.get(j..end) else { continue }; + // One that doesn't fit the budget left is skipped; a smaller one later may still fit. + if walked.saturating_add(jpeg.len()) > CR3_MAX_WALK { + continue; + } + walked = walked.saturating_add(jpeg.len()); + if is_dct_jpeg(jpeg) { + out.push(jpeg); + taken = true; + if whole_file { + break; + } + // (past it: a tag inside its JPEG is not another box) + from = end; + } + } } } - best + out +} + +/// Where a CR3's top-level `mdat` boxes lie (`start..end` each, `end` clamped to the file, in file +/// order): the media data, most of the file. Empty when there is none, or the boxes stop making sense +/// before the first. The metadata and the small previews are in the other top-level boxes. +fn cr3_media_data(bytes: &[u8]) -> Vec<(usize, usize)> { + let mut out = Vec::new(); + let _ = cr3_media_walk(bytes, &mut out); + out +} + +/// [`cr3_media_data`]'s walk of the top-level boxes; `None` where they stop making sense. +fn cr3_media_walk(bytes: &[u8], out: &mut Vec<(usize, usize)>) -> Option<()> { + let be32 = |at: usize| bytes.get(at..at.checked_add(4)?).map(|s| u64::from(u32::from_be_bytes([s[0], s[1], s[2], s[3]]))); + let be64 = |at: usize| bytes.get(at..at.checked_add(8)?).map(|s| u64::from_be_bytes([s[0], s[1], s[2], s[3], s[4], s[5], s[6], s[7]])); + let mut at = 0usize; + for _ in 0..64 { + let size = be32(at)?; + let len = match size { + // (a box running to the end of the file) + 0 => u64::try_from(bytes.len().saturating_sub(at)).unwrap_or(u64::MAX), + 1 => be64(at.checked_add(8)?)?, + n => n, + }; + let next = usize::try_from(len).ok().filter(|l| *l >= 8).and_then(|l| at.checked_add(l)); + if bytes.get(at.checked_add(4)?..at.checked_add(8)?) == Some(b"mdat".as_slice()) { + out.push((at, next.unwrap_or(bytes.len()).min(bytes.len()))); + } + at = next.filter(|n| *n < bytes.len())?; + } + None } /// Trim trailing garbage after the last EOI when a stored length over-reports. diff --git a/crates/raw/src/preview/tests.rs b/crates/raw/src/preview/tests.rs index aa2dc0da74..561d39645d 100644 --- a/crates/raw/src/preview/tests.rs +++ b/crates/raw/src/preview/tests.rs @@ -296,6 +296,98 @@ fn cr3_prefers_the_full_size_jpeg_track() { assert_eq!(embedded_preview(&file).unwrap(), small); } +/// A CR3 whose `PRVW` box holds `prvw` and whose JPEG track holds `big`. +fn cr3_with(prvw_jpeg: &[u8], big: &[u8]) -> Vec { + let bx = |kind: &[u8; 4], body: &[u8]| -> Vec { [&((body.len() + 8) as u32).to_be_bytes()[..], kind, body].concat() }; + let full = |kind: &[u8; 4], body: &[u8]| bx(kind, &[&[0u8; 4][..], body].concat()); + let mut file = bx(b"ftyp", b"crx \0\0\0\x01crx isom"); + let mut prvw = b"\0\0\0\0\0\x01\x06T\x048\0\x01".to_vec(); + prvw.extend_from_slice(&(prvw_jpeg.len() as u32).to_be_bytes()); + prvw.extend_from_slice(prvw_jpeg); + file.extend(bx(b"PRVW", &prvw)); + let mut craw = vec![0u8; 82]; + craw.extend(bx(b"JPEG", &[0; 4])); + let at = 4096u64; + let stsd = full(b"stsd", &[&1u32.to_be_bytes()[..], &bx(b"CRAW", &craw)].concat()); + let stsz = full(b"stsz", &[0u32.to_be_bytes(), 1u32.to_be_bytes(), (big.len() as u32).to_be_bytes()].concat()); + let co64 = full(b"co64", &[&1u32.to_be_bytes()[..], &at.to_be_bytes()].concat()); + let trak = bx(b"trak", &bx(b"mdia", &bx(b"minf", &bx(b"stbl", &[stsd, stsz, co64].concat())))); + file.extend(bx(b"moov", &trak)); + file.resize(at as usize, 0); + file.extend_from_slice(big); + file +} + +/// A CR3 thumbnail decodes the 1620 px `PRVW` copy, not the full-size JPEG (PERF-THUMB-FIRST: +/// 150 ms a photo for the full one); a size the copy can't cover, and the largest preview, are +/// still the full JPEG; a `PRVW` of another shape is never taken for the picture. +#[test] +fn cr3_thumbnails_take_the_smallest_copy_that_covers_them() { + let (prvw, big) = (sized_jpeg(1620, 1080, 3, 300), sized_jpeg(6000, 4000, 3, 3000)); + let file = cr3_with(&prvw, &big); + assert_eq!(embedded_preview_covering(&file, 512).unwrap(), prvw); + assert_eq!(embedded_preview_covering(&file, 1620).unwrap(), prvw); + assert_eq!(embedded_preview_covering(&file, 2000).unwrap(), big); + assert_eq!(embedded_preview(&file).unwrap(), big); + let letterboxed = cr3_with(&sized_jpeg(1620, 1215, 3, 300), &big); + assert_eq!(embedded_preview_covering(&letterboxed, 512).unwrap(), big, "another shape is not a copy"); + for n in (0..file.len()).step_by(13) { + let _ = embedded_preview_covering(&file[..n], 512); + } +} + +/// PERF-CULL-4: the smallest preview that covers the size asked for, of the largest one's shape; +/// the largest when none covers it. A CR3's 8192 px track, 1620 px `PRVW` and 160 px `THMB`. +#[test] +fn the_smallest_preview_covering_the_size_is_chosen() { + let bx = |kind: &[u8; 4], body: &[u8]| -> Vec { [&((body.len() + 8) as u32).to_be_bytes()[..], kind, body].concat() }; + let full = |kind: &[u8; 4], body: &[u8]| bx(kind, &[&[0u8; 4][..], body].concat()); + let boxed = |j: &[u8]| { + let mut b = b"\0\0\0\0\0\x01\x06\x54\x04\x38\0\x01".to_vec(); + b.extend_from_slice(&(j.len() as u32).to_be_bytes()); + b.extend_from_slice(j); + b + }; + let (thmb, prvw, big) = (sized_jpeg(160, 107, 1, 50), sized_jpeg(1620, 1080, 1, 300), sized_jpeg(8192, 5464, 1, 3000)); + let mut file = bx(b"ftyp", b"crx \0\0\0\x01crx isom"); + file.extend(bx(b"THMB", &boxed(&thmb))); + file.extend(bx(b"PRVW", &boxed(&prvw))); + let mut craw = vec![0u8; 82]; + craw.extend(bx(b"JPEG", &[0; 4])); + let at = 4096u64; + let stsd = full(b"stsd", &[&1u32.to_be_bytes()[..], &bx(b"CRAW", &craw)].concat()); + let stsz = full(b"stsz", &[0u32.to_be_bytes(), 1u32.to_be_bytes(), (big.len() as u32).to_be_bytes()].concat()); + let co64 = full(b"co64", &[&1u32.to_be_bytes()[..], &at.to_be_bytes()].concat()); + let trak = bx(b"trak", &bx(b"mdia", &bx(b"minf", &bx(b"stbl", &[stsd, stsz, co64].concat())))); + file.extend(bx(b"moov", &trak)); + file.resize(at as usize, 0); + file.extend_from_slice(&big); + assert_eq!(embedded_preview(&file).unwrap(), big); + assert_eq!(embedded_preview_covering(&file, 512).unwrap(), prvw, "a grid thumbnail"); + assert_eq!(embedded_preview_covering(&file, 1620).unwrap(), prvw); + assert_eq!(embedded_preview_covering(&file, 100).unwrap(), thmb); + assert_eq!(embedded_preview_covering(&file, 2560).unwrap(), big, "a large loupe"); + assert_eq!(embedded_preview_covering(&file, u32::MAX).unwrap(), big); + // a smaller preview of another shape (letterboxed 16:9) is never a stand-in for the 3:2 one + let mut wide = file.clone(); + let p = wide.windows(4).position(|w| w == b"PRVW").unwrap(); + // the JPEG starts 20 bytes after the tag; its frame height 7 bytes into it + let height = p + 20 + 7; + wide[height..height + 2].copy_from_slice(&911u16.to_be_bytes()); + assert_eq!(embedded_preview_covering(&wide, 512).unwrap(), big); + // without the JPEG track the boxes are chosen from too: the smallest that covers, else the largest + // (CodeRabbit on #3577: a box-only CR3 always gave its largest box) + let boxes_only = &file[..file.windows(4).position(|w| w == b"moov").unwrap() - 4]; + assert_eq!(embedded_preview_covering(boxes_only, 100).unwrap(), thmb); + assert_eq!(embedded_preview_covering(boxes_only, 512).unwrap(), prvw); + assert_eq!(embedded_preview_covering(boxes_only, 2560).unwrap(), prvw, "none covers: the largest"); + assert_eq!(embedded_preview(boxes_only).unwrap(), prvw); + // damaged files never panic + for n in (0..file.len()).step_by(7) { + let _ = embedded_preview_covering(&file[..n], 512); + } +} + /// A minimal CR3 whose Canon box holds `CMT2` (EXIF ColorSpace `exif`) and `CMT3` (maker-note ColorSpace `note`). fn cr3_with_color_space(exif: Option, note: Option) -> Vec { let bx = |kind: &[u8; 4], body: &[u8]| -> Vec { [&((body.len() + 8) as u32).to_be_bytes()[..], kind, body].concat() }; @@ -502,3 +594,261 @@ fn scanned_containers_give_the_largest_colour_jpeg() { f.extend_from_slice(&mosaic); assert_eq!(embedded_preview(&f), None); } + +/// A CR3's `PRVW` / `THMB` boxes are searched outside the media data only, but before and after +/// it: a `moov` after the `mdat` (its Canon `uuid` holding the `THMB`) still gives its preview, and +/// bytes inside the media data that look like a preview box are never taken for one. +#[test] +fn cr3_preview_boxes_after_the_media_data_are_found() { + let bx = |kind: &[u8; 4], body: &[u8]| -> Vec { [&((body.len() + 8) as u32).to_be_bytes()[..], kind, body].concat() }; + let boxed = |j: &[u8]| { + let mut b = b"\0\0\0\0\0\x01\x06\x54\x04\x38\0\x01".to_vec(); + b.extend_from_slice(&(j.len() as u32).to_be_bytes()); + b.extend_from_slice(j); + b + }; + let (thmb, in_media) = (sized_jpeg(160, 107, 1, 50), sized_jpeg(1620, 1080, 1, 300)); + let mut file = bx(b"ftyp", b"crx \0\0\0\x01crx isom"); + file.extend(bx(b"mdat", &bx(b"PRVW", &boxed(&in_media)))); + file.extend(bx(b"moov", &bx(b"uuid", &[&[0x85u8; 16][..], &bx(b"THMB", &boxed(&thmb))].concat()))); + assert_eq!(embedded_preview(&file).unwrap(), thmb); + for n in (0..file.len()).step_by(5) { + let _ = embedded_preview(&file[..n]); + } +} + +/// Every valid `PRVW` of a CR3 is a candidate (CodeRabbit on #3055): with a small one first and +/// one that covers the size after it, a thumbnail takes the second, not the full-size track. +#[test] +fn a_cr3_with_two_prvw_boxes_offers_both() { + let bx = |kind: &[u8; 4], body: &[u8]| -> Vec { [&((body.len() + 8) as u32).to_be_bytes()[..], kind, body].concat() }; + let full = |kind: &[u8; 4], body: &[u8]| bx(kind, &[&[0u8; 4][..], body].concat()); + let boxed = |j: &[u8]| { + let mut b = b"\0\0\0\0\0\x01\x06\x54\x04\x38\0\x01".to_vec(); + b.extend_from_slice(&(j.len() as u32).to_be_bytes()); + b.extend_from_slice(j); + b + }; + let (small, covering, big) = (sized_jpeg(300, 200, 1, 50), sized_jpeg(1620, 1080, 1, 300), sized_jpeg(8192, 5464, 1, 3000)); + let mut file = bx(b"ftyp", b"crx \0\0\0\x01crx isom"); + file.extend(bx(b"PRVW", &boxed(&small))); + file.extend(bx(b"PRVW", &boxed(&covering))); + let mut craw = vec![0u8; 82]; + craw.extend(bx(b"JPEG", &[0; 4])); + let at = 4096u64; + let stsd = full(b"stsd", &[&1u32.to_be_bytes()[..], &bx(b"CRAW", &craw)].concat()); + let stsz = full(b"stsz", &[0u32.to_be_bytes(), 1u32.to_be_bytes(), (big.len() as u32).to_be_bytes()].concat()); + let co64 = full(b"co64", &[&1u32.to_be_bytes()[..], &at.to_be_bytes()].concat()); + let trak = bx(b"trak", &bx(b"mdia", &bx(b"minf", &bx(b"stbl", &[stsd, stsz, co64].concat())))); + file.extend(bx(b"moov", &trak)); + // (the track in the media data, as a camera writes it, after a `free` box up to it) + let pad = at as usize - 16 - file.len(); + file.extend(bx(b"free", &vec![0u8; pad])); + file.extend(bx(b"mdat", &big)); + assert_eq!(embedded_preview_covering(&file, 1024).unwrap(), covering); + assert_eq!(embedded_preview_covering(&file, 200).unwrap(), small); +} + +/// A CR3 preview box claiming the largest sizes a file can give is skipped, never wrapped or +/// panicked on (CodeRabbit on #3080: on a 32-bit build the plain sums could overflow). +#[test] +fn a_cr3_preview_box_with_hostile_sizes_is_skipped() { + let mut file = b"\0\0\0\x18ftypcrx \0\0\0\x01crx isom".to_vec(); + file.extend_from_slice(&u32::MAX.to_be_bytes()); + file.extend_from_slice(b"PRVW\0\0\0\0\0\x01\x06\x54\x04\x38\0\x01"); + file.extend_from_slice(&u32::MAX.to_be_bytes()); + file.extend_from_slice(&fake_jpeg(64)); + assert!(embedded_preview(&file).is_none()); + assert!(embedded_preview_covering(&file, 64).is_none()); +} + +/// A corrupt raw whose preview pointer (JPEGInterchangeFormat and its length) lies past the end +/// of the file, or whose length runs past it, finds no preview and never panics; a real preview +/// beside the lying pointer is still found (FredApps on #3080: the preview must be read with +/// bounds on its offset and size). +#[test] +fn a_preview_pointer_past_the_end_of_the_file_finds_no_preview() { + let real = sized_jpeg(640, 480, 3, 200); + let lying = |off: u32, len: u32, with_real: bool| { + let mut ifd = IfdBuilder::new(); + ifd.set(t::JPEG_INTERCHANGE_FORMAT, Value::Long(vec![off])); + ifd.set(t::JPEG_INTERCHANGE_FORMAT_LENGTH, Value::Long(vec![len])); + if with_real { + ifd.set(t::COMPRESSION, Value::Short(vec![6])); + ifd.set(t::IMAGE_WIDTH, Value::Long(vec![1])); + ifd.set(t::IMAGE_LENGTH, Value::Long(vec![1])); + ifd.set_image(ImageData::Strips { rows_per_strip: 1, strips: vec![real.clone()] }); + } + TiffWriter::default().write(&[ifd]).unwrap() + }; + let short = lying(0, 0, false).len() as u32; + for (off, len) in [(u32::MAX, u32::MAX), (u32::MAX, 16), (short + 10, 4096), (8, u32::MAX), (short - 2, u32::MAX - 1)] { + let file = lying(off, len, false); + assert_eq!(embedded_preview(&file), None, "pointer {off:#x}+{len:#x} in a {} byte file", file.len()); + assert_eq!(embedded_preview_covering(&file, 64), None); + assert_eq!(embedded_preview_color_space(&file), None); + let with_real = lying(off, len, true); + assert_eq!(embedded_preview(&with_real).as_deref(), Some(&real[..]), "the strip preview beside pointer {off:#x}+{len:#x}"); + // and every truncation of the file (a raw cut short by a failed copy) is safe too + for n in 0..with_real.len() { + let _ = embedded_preview_covering(&with_real[..n], 64); + } + } +} + +/// A preview pointer whose length runs past the end of a file that cuts its JPEG short (no end +/// marker) finds no preview, so a smaller whole one wins; an overstated length whose JPEG is whole +/// inside the file still finds it (CodeRabbit on #3077). +#[test] +fn a_preview_cut_short_by_the_end_of_the_file_is_not_taken() { + let big = sized_jpeg(1600, 1200, 3, 4000); + let small = sized_jpeg(160, 120, 3, 100); + // the small JPEG as a strip, the big one pointed to at the end of the file + let with_tail = |tail: &[u8], declared: u32| { + let probe = { + let mut ifd = IfdBuilder::new(); + ifd.set(t::JPEG_INTERCHANGE_FORMAT, Value::Long(vec![0])); + ifd.set(t::JPEG_INTERCHANGE_FORMAT_LENGTH, Value::Long(vec![0])); + ifd.set(t::COMPRESSION, Value::Short(vec![6])); + ifd.set(t::IMAGE_WIDTH, Value::Long(vec![1])); + ifd.set(t::IMAGE_LENGTH, Value::Long(vec![1])); + ifd.set_image(ImageData::Strips { rows_per_strip: 1, strips: vec![small.clone()] }); + ifd + }; + let at = TiffWriter::default().write(&[probe]).unwrap().len() as u32; + let mut ifd = IfdBuilder::new(); + ifd.set(t::JPEG_INTERCHANGE_FORMAT, Value::Long(vec![at])); + ifd.set(t::JPEG_INTERCHANGE_FORMAT_LENGTH, Value::Long(vec![declared])); + ifd.set(t::COMPRESSION, Value::Short(vec![6])); + ifd.set(t::IMAGE_WIDTH, Value::Long(vec![1])); + ifd.set(t::IMAGE_LENGTH, Value::Long(vec![1])); + ifd.set_image(ImageData::Strips { rows_per_strip: 1, strips: vec![small.clone()] }); + let mut file = TiffWriter::default().write(&[ifd]).unwrap(); + assert_eq!(file.len() as u32, at, "the pointer lands right after the TIFF"); + file.extend_from_slice(tail); + file + }; + // the big one with a start of scan after its frame header (sized_jpeg has none), as a real JPEG + let mut big = big; + big.splice(21..21, [0xff, 0xda, 0x00, 0x02]); + let declared = big.len() as u32 + 64; + // cut short: half of the big JPEG, no end marker + let cut = with_tail(&big[..big.len() / 2], declared); + assert_eq!(embedded_preview(&cut).as_deref(), Some(&small[..]), "the truncated JPEG is not taken"); + // whole, its length overstated past the file's end: still the largest + let whole = with_tail(&big, declared); + assert_eq!(embedded_preview(&whole).as_deref(), Some(&big[..]), "a whole JPEG with an overstated length is kept"); + // an end marker inside an APP1 payload is not the JPEG's end: cut short, still not taken + let mut with_app = vec![0xff, 0xd8, 0xff, 0xe1, 0x00, 0x08, 0x12, 0xff, 0xd9, 0x34, 0x56, 0x78]; + with_app.extend_from_slice(&big[2..]); + let declared = with_app.len() as u32 + 64; + let cut = with_tail(&with_app[..with_app.len() / 2], declared); + assert_eq!(embedded_preview(&cut).as_deref(), Some(&small[..]), "an FF D9 in APP1 doesn't make a cut JPEG whole"); + // whole, with other bytes after it up to the file's end: kept, cut at its own end marker + let mut tail = with_app.clone(); + tail.extend_from_slice(&[0x11, 0xff, 0xd9, 0x22]); + let whole = with_tail(&tail, declared); + assert_eq!(embedded_preview(&whole).as_deref(), Some(&with_app[..]), "a whole JPEG is cut at its own end"); + // an end marker before any start of scan: no image data, not taken (CodeRabbit on #3080) + let unscanned = sized_jpeg(1600, 1200, 3, 4000); + let whole = with_tail(&unscanned, unscanned.len() as u32 + 64); + assert_eq!(embedded_preview(&whole).as_deref(), Some(&small[..]), "a JPEG without a scan is not taken"); +} + +/// The walk to a JPEG's end refuses what isn't one and never runs past the bytes it has. +#[test] +fn the_end_of_a_jpeg_is_its_own_end_marker() { + let jpeg_end = |s: &[u8]| super::jpeg_end(s, &mut (1usize << 20)); + assert_eq!(jpeg_end(&[0x12, 0x34, 0xff, 0xd9]), None, "not a JPEG"); + assert_eq!(jpeg_end(&[0xff, 0xd8, 0xff, 0xd9]), None, "an end before any scan: no image data"); + // a stuffed zero and a restart marker in the scan data, then the end + assert_eq!(jpeg_end(&[0xff, 0xd8, 0xff, 0xda, 0x00, 0x02, 0x55, 0xff, 0x00, 0xff, 0xd3, 0x66, 0xff, 0xd9]), Some(14)); + // a segment length of 0 or 1, or one running past the end + assert_eq!(jpeg_end(&[0xff, 0xd8, 0xff, 0xe1, 0x00, 0x01, 0xff, 0xd9]), None); + assert_eq!(jpeg_end(&[0xff, 0xd8, 0xff, 0xe1, 0xff, 0xff, 0xff, 0xd9]), None); + for n in 0..12 { + assert_eq!(jpeg_end(&[0xff, 0xd8, 0xff, 0xe1, 0x00, 0x04, 0xff, 0xd9, 0x55, 0xff, 0xff, 0xff][..n]), None, "{n} bytes"); + } +} + +/// CodeRabbit on #3080: the walks to a JPEG's end share one budget, so a file of many pointers past +/// its end can't make each walk run the whole file: a walk that would overspend it gives up. +#[test] +fn the_walks_to_a_jpegs_end_share_a_budget() { + use super::jpeg_end; + let mut j = vec![0xff, 0xd8, 0xff, 0xda, 0x00, 0x02]; + j.extend(std::iter::repeat_n(0x55u8, 1000)); + j.extend_from_slice(&[0xff, 0xd9]); + let mut budget = 1500; + assert_eq!(jpeg_end(&j, &mut budget), Some(j.len()), "the first walk fits"); + assert!(budget < 1500 - 1000, "and is charged for its steps: {budget} left"); + assert_eq!(jpeg_end(&j, &mut budget), None, "the second would overspend what is left"); + assert_eq!(budget, 0); +} + +/// A CR3 without a JPEG track whose smaller `PRVW` (another shape) takes more bytes: the picture is +/// the box with the most pixels, not the most bytes (CodeRabbit on #3077). +#[test] +fn a_box_only_cr3_is_judged_by_pixels_not_bytes() { + let bx = |kind: &[u8; 4], body: &[u8]| -> Vec { [&((body.len() + 8) as u32).to_be_bytes()[..], kind, body].concat() }; + let boxed = |j: &[u8]| { + let mut b = b"\0\0\0\0\0\x01\x06\x54\x04\x38\0\x01".to_vec(); + b.extend_from_slice(&(j.len() as u32).to_be_bytes()); + b.extend_from_slice(j); + b + }; + let (big, square) = (sized_jpeg(1620, 1080, 3, 300), sized_jpeg(800, 800, 3, 5000)); + let mut file = bx(b"ftyp", b"crx \0\0\0\x01crx isom"); + file.extend(bx(b"PRVW", &boxed(&square))); + file.extend(bx(b"PRVW", &boxed(&big))); + // no media data found: the whole file is searched, and only its first box counts (a later look-alike could be + // in media data; CodeRabbit on #3583) + assert_eq!(embedded_preview(&file).unwrap(), square); + file.extend(bx(b"mdat", &[0u8; 16])); + assert_eq!(embedded_preview(&file).unwrap(), big); + assert_eq!(embedded_preview_covering(&file, 100).unwrap(), big, "the square is another picture's shape"); +} + +/// Every top-level `mdat` is media data, not only the first: a PRVW-shaped JPEG in a second one is +/// never the preview; and a `PRVW` cut short before its end marker isn't chosen to cover a size +/// (CodeRabbit on #3583). +#[test] +fn cr3_boxes_in_any_media_data_or_cut_short_are_not_previews() { + let bx = |kind: &[u8; 4], body: &[u8]| -> Vec { [&((body.len() + 8) as u32).to_be_bytes()[..], kind, body].concat() }; + let boxed = |j: &[u8]| { + let mut b = b"\0\0\0\0\0\x01\x06\x54\x04\x38\0\x01".to_vec(); + b.extend_from_slice(&(j.len() as u32).to_be_bytes()); + b.extend_from_slice(j); + b + }; + let (real, fake) = (sized_jpeg(1620, 1080, 3, 300), sized_jpeg(3000, 2000, 3, 300)); + let mut file = bx(b"ftyp", b"crx \0\0\0\x01crx isom"); + file.extend(bx(b"PRVW", &boxed(&real))); + file.extend(bx(b"mdat", &[0u8; 64])); + file.extend(bx(b"mdat", &bx(b"PRVW", &boxed(&fake)))); + assert_eq!(embedded_preview(&file).unwrap(), real, "the look-alike in the second mdat is media data"); + // a PRVW cut short (no end marker): the full-size track covers a thumbnail instead + let mut short = real.clone(); + short.truncate(real.len() - 2); + let big = sized_jpeg(6000, 4000, 3, 3000); + assert_eq!(embedded_preview_covering(&cr3_with(&short, &big), 512).unwrap(), big); + assert_eq!(embedded_preview_covering(&cr3_with(&real, &big), 512).unwrap(), real, "a whole one is still taken"); +} + +/// The copy a CR3 thumbnail is given decodes, whichever is chosen (CodeRabbit on #3583: the other CR3 fixtures have +/// no scan): real encoded JPEGs, the `PRVW` for a small size, the full-size track past it. +#[test] +fn the_cr3_copy_chosen_for_a_thumbnail_decodes() { + let encode = |w: u16, h: u16| { + let px: Vec = (0..usize::from(w) * usize::from(h)).flat_map(|i| [(i % 251) as u8, 90, 160]).collect(); + let mut j = Vec::new(); + jpeg_encoder::Encoder::new(&mut j, 85).encode(&px, w, h, jpeg_encoder::ColorType::Rgb).unwrap(); + j + }; + let file = cr3_with(&encode(96, 64), &encode(384, 256)); + for (min, want) in [(64u32, (96, 64)), (200, (384, 256))] { + let j = embedded_preview_covering(&file, min).unwrap(); + let (_, w, h, c) = crate::tiffraw::lossy_jpeg(&j, 3, 1 << 20).unwrap(); + assert_eq!(((w, h), c), (want, 3), "covering {min} px"); + } +} diff --git a/crates/ui-egui/locales/cs.json b/crates/ui-egui/locales/cs.json index 0b142df5e6..a8d925d799 100644 --- a/crates/ui-egui/locales/cs.json +++ b/crates/ui-egui/locales/cs.json @@ -1245,6 +1245,7 @@ "Specific folder": "Konkrétní složka", "Same folder as original photo": "Stejná složka jako původní fotografie", "Tidying the thumbnail cache": "Úklid mezipaměti miniatur", + "Checking free disk space": "Kontrola volného místa na disku", "No masks yet. Choose a mask type above.": "Zatím žádné masky. Vyberte typ masky nahoře.", "No matching commands": "Žádné odpovídající příkazy", "No matching photos": "Žádné odpovídající fotografie", diff --git a/crates/ui-egui/locales/de.json b/crates/ui-egui/locales/de.json index 552ce68016..5a5b8e9a2b 100644 --- a/crates/ui-egui/locales/de.json +++ b/crates/ui-egui/locales/de.json @@ -2329,6 +2329,7 @@ "Specific folder": "Bestimmter Ordner", "Same folder as original photo": "Im selben Ordner wie das Originalfoto", "Tidying the thumbnail cache": "Miniaturcache wird aufgeräumt", + "Checking free disk space": "Freier Speicherplatz wird geprüft", "Reset all edits (Cmd+Shift+R)": "Alle Bearbeitungen zurücksetzen (⌘⇧R)", "Smart albums update automatically: change their rules, or use Photo ▸ Delete Photo": "Smart-Alben aktualisieren sich selbst: Ändere ihre Regeln oder verwende Foto → Foto löschen", "LightCraft stopped": "LightCraft wurde beendet", @@ -3459,6 +3460,8 @@ "The photos are imported once it ends": "Die Fotos werden danach importiert", "An import is running. Drop photos when it has finished": "Ein Import läuft. Lege die Fotos ab, wenn er fertig ist", "Finish Lightroom catalog import before adding photos": "Schließe den Import des Lightroom-Katalogs ab, bevor du Fotos hinzufügst", + "Camera preview": "Kameravorschau", + "The camera's own JPEG, shown until LightCraft has rendered the raw. Judge colour and tone on the rendered photo.": "Das eigene JPEG der Kamera, angezeigt, bis LightCraft die Raw-Datei gerendert hat. Farbe und Tonwerte am gerenderten Foto beurteilen.", "Profile: from the file (the camera's own correction)": "Profil: aus der Datei (Korrektur der Kamera)", "Downloading the lens database…": "Objektivdatenbank wird geladen…", "This file names no lens.": "Diese Datei nennt kein Objektiv.", diff --git a/crates/ui-egui/locales/es.json b/crates/ui-egui/locales/es.json index b82a261256..ba1a5ec747 100644 --- a/crates/ui-egui/locales/es.json +++ b/crates/ui-egui/locales/es.json @@ -2333,6 +2333,7 @@ "Specific folder": "Carpeta específica", "Same folder as original photo": "Misma carpeta que la foto original", "Tidying the thumbnail cache": "Ordenando la caché de miniaturas", + "Checking free disk space": "Comprobando el espacio libre en disco", "Reset all edits (Cmd+Shift+R)": "Restablecer todas las ediciones (Cmd+Mayús+R)", "Smart albums update automatically: change their rules, or use Photo ▸ Delete Photo": "Los álbumes inteligentes se actualizan solos: cambie sus reglas o use Foto → Eliminar foto", "LightCraft stopped": "LightCraft se detuvo", @@ -2378,6 +2379,8 @@ "The photos are imported once it ends": "Las fotos se importarán después", "An import is running. Drop photos when it has finished": "Hay una importación en curso. Suelta las fotos cuando termine", "Finish Lightroom catalog import before adding photos": "Termina la importación del catálogo de Lightroom antes de añadir fotos", + "Camera preview": "Vista previa de la cámara", + "The camera's own JPEG, shown until LightCraft has rendered the raw. Judge colour and tone on the rendered photo.": "El JPEG propio de la cámara, mostrado hasta que LightCraft haya renderizado el RAW. Juzga el color y el tono en la foto renderizada.", "Profile: from the file (the camera's own correction)": "Perfil: del archivo (corrección de la propia cámara)", "Downloading the lens database…": "Descargando la base de datos de objetivos…", "This file names no lens.": "Este archivo no indica ningún objetivo.", diff --git a/crates/ui-egui/locales/fi.json b/crates/ui-egui/locales/fi.json index df80342abe..b99329eb84 100644 --- a/crates/ui-egui/locales/fi.json +++ b/crates/ui-egui/locales/fi.json @@ -2065,6 +2065,7 @@ "Thursday": "torstai", "Tick the box to accept the model's terms first": "Hyväksy ensin mallin ehdot valitsemalla ruutu", "Tidying the thumbnail cache": "Siivotaan pikkukuvien välimuistia", + "Checking free disk space": "Tarkistetaan vapaata levytilaa", "Time zone": "Aikavyöhyke", "Tint": "Värisävy", "Title": "Otsikko", diff --git a/crates/ui-egui/locales/fr.json b/crates/ui-egui/locales/fr.json index e513e217de..2d1c623e17 100644 --- a/crates/ui-egui/locales/fr.json +++ b/crates/ui-egui/locales/fr.json @@ -2328,6 +2328,7 @@ "Specific folder": "Dossier spécifique", "Same folder as original photo": "Même dossier que la photo d’origine", "Tidying the thumbnail cache": "Rangement du cache des miniatures", + "Checking free disk space": "Vérification de l'espace disque libre", "Reset all edits (Cmd+Shift+R)": "Réinitialiser toutes les retouches (Cmd+Maj+R)", "Smart albums update automatically: change their rules, or use Photo ▸ Delete Photo": "Les albums intelligents se mettent à jour d'eux-mêmes : modifiez leurs règles ou utilisez Photo → Supprimer la photo", "LightCraft stopped": "LightCraft s’est arrêté", @@ -2409,6 +2410,8 @@ "The photos are imported once a library is open": "Les photos seront importées dès qu'une bibliothèque sera ouverte", "Open a library first, then drop the presets again": "Ouvrez d'abord une bibliothèque, puis déposez à nouveau les préréglages", "This library couldn't be checked:": "Impossible de vérifier cette bibliothèque :", + "Camera preview": "Aperçu de l'appareil", + "The camera's own JPEG, shown until LightCraft has rendered the raw. Judge colour and tone on the rendered photo.": "Le JPEG de l'appareil photo, affiché jusqu'à ce que LightCraft ait rendu le fichier RAW. Jugez la couleur et les tons sur la photo rendue.", "Profile: from the file (the camera's own correction)": "Profil : du fichier (correction de l'appareil)", "Downloading the lens database…": "Téléchargement de la base de données d'objectifs…", "This file names no lens.": "Ce fichier n'indique aucun objectif.", diff --git a/crates/ui-egui/locales/id.json b/crates/ui-egui/locales/id.json index 7f5e56e1ce..8889444a08 100644 --- a/crates/ui-egui/locales/id.json +++ b/crates/ui-egui/locales/id.json @@ -2062,6 +2062,7 @@ "Thursday": "Kamis", "Tick the box to accept the model's terms first": "Centang kotak untuk menyetujui ketentuan model terlebih dahulu", "Tidying the thumbnail cache": "Merapikan cache thumbnail", + "Checking free disk space": "Memeriksa ruang disk kosong", "Time zone": "Zona waktu", "Tint": "Rona", "Title": "Judul", diff --git a/crates/ui-egui/locales/it.json b/crates/ui-egui/locales/it.json index c77a85b41f..b7a617a5af 100644 --- a/crates/ui-egui/locales/it.json +++ b/crates/ui-egui/locales/it.json @@ -937,6 +937,7 @@ "Specific folder": "Cartella specifica", "Same folder as original photo": "Stessa cartella della foto originale", "Tidying the thumbnail cache": "Riordino della cache delle miniature", + "Checking free disk space": "Controllo dello spazio libero su disco", "No masks yet. Choose a mask type above.": "Ancora nessuna maschera. Scegli un tipo di maschera qui sopra.", "No matching commands": "Nessun comando corrispondente", "No matching photos": "Nessuna foto corrispondente", diff --git a/crates/ui-egui/locales/ja.json b/crates/ui-egui/locales/ja.json index 72ed66ad20..a07bb9de25 100644 --- a/crates/ui-egui/locales/ja.json +++ b/crates/ui-egui/locales/ja.json @@ -2331,6 +2331,7 @@ "Specific folder": "指定フォルダー", "Same folder as original photo": "元の写真と同じフォルダー", "Tidying the thumbnail cache": "サムネールキャッシュを整理中", + "Checking free disk space": "ディスクの空き容量を確認中", "Reset all edits (Cmd+Shift+R)": "すべての編集をリセット(⌘⇧R)", "Smart albums update automatically: change their rules, or use Photo ▸ Delete Photo": "スマートアルバムは自動で更新されます。ルールを変更するか、写真 → 写真を削除を使ってください", "LightCraft stopped": "LightCraftが終了しました", @@ -2376,6 +2377,8 @@ "The photos are imported once it ends": "終わりしだい写真を読み込みます", "An import is running. Drop photos when it has finished": "読み込み中です。完了してから写真をドロップしてください", "Finish Lightroom catalog import before adding photos": "写真を追加する前に Lightroom カタログの読み込みを完了してください", + "Camera preview": "カメラのプレビュー", + "The camera's own JPEG, shown until LightCraft has rendered the raw. Judge colour and tone on the rendered photo.": "LightCraft が RAW を現像するまで表示される、カメラ自身の JPEG です。色とトーンは現像後の写真で判断してください。", "Profile: from the file (the camera's own correction)": "プロファイル: ファイル内(カメラ自身の補正)", "Downloading the lens database…": "レンズデータベースをダウンロード中…", "This file names no lens.": "このファイルにはレンズ名がありません。", diff --git a/crates/ui-egui/locales/ka.json b/crates/ui-egui/locales/ka.json index e9d0ba8b05..b994b84144 100644 --- a/crates/ui-egui/locales/ka.json +++ b/crates/ui-egui/locales/ka.json @@ -2945,6 +2945,7 @@ "This row covers more than one disk: drop on a folder inside it": "ეს სტრიქონი ერთზე მეტ დისკს მოიცავს: ჩააგდეთ მის შიგნით მყოფ საქაღალდეზე", "This screen is off: turn it on under Titles to play it.": "ეს ეკრანი გამორთულია: დასაკრავად ჩართეთ „ტიტრებში“.", "Tidying the thumbnail cache": "ესკიზების ქეში სუფთავდება", + "Checking free disk space": "დისკზე თავისუფალი ადგილის შემოწმება", "Title text": "ტიტრის ტექსტი", "Titles": "ტიტრები", "Tokens: {file}, {name}, {title}, {caption}, {date}, {exposure}, {iso}, {focal}, {camera}, {lens}, {rating}, {creator}, {copyright}. Other text shows as typed.": "ტეგები: {file}, {name}, {title}, {caption}, {date}, {exposure}, {iso}, {focal}, {camera}, {lens}, {rating}, {creator}, {copyright}. სხვა ტექსტი ისე ჩანს, როგორც აკრიფეთ.", diff --git a/crates/ui-egui/locales/pt-br.json b/crates/ui-egui/locales/pt-br.json index 821bc56ff3..285b84f48d 100644 --- a/crates/ui-egui/locales/pt-br.json +++ b/crates/ui-egui/locales/pt-br.json @@ -2171,6 +2171,7 @@ "Specific folder": "Pasta específica", "Same folder as original photo": "Mesma pasta da foto original", "Tidying the thumbnail cache": "Organizando o cache de miniaturas", + "Checking free disk space": "Verificando o espaço livre em disco", "Reset all edits (Cmd+Shift+R)": "Redefinir todas as edições (Cmd+Shift+R)", "Smart albums update automatically: change their rules, or use Photo ▸ Delete Photo": "Álbuns inteligentes se atualizam sozinhos: altere as regras ou use Foto → Excluir foto", "LightCraft stopped": "O LightCraft parou", @@ -2222,6 +2223,8 @@ "The photos are imported once it ends": "As fotos serão importadas depois", "An import is running. Drop photos when it has finished": "Uma importação está em andamento. Solte as fotos quando ela terminar", "Finish Lightroom catalog import before adding photos": "Conclua a importação do catálogo do Lightroom antes de adicionar fotos", + "Camera preview": "Prévia da câmera", + "The camera's own JPEG, shown until LightCraft has rendered the raw. Judge colour and tone on the rendered photo.": "O JPEG da própria câmera, mostrado até o LightCraft renderizar o RAW. Avalie cor e tom na foto renderizada.", "Profile: from the file (the camera's own correction)": "Perfil: do arquivo (correção da própria câmera)", "Downloading the lens database…": "Baixando o banco de dados de lentes…", "This file names no lens.": "Este arquivo não indica nenhuma lente.", diff --git a/crates/ui-egui/locales/ru.json b/crates/ui-egui/locales/ru.json index db51744bfa..79b8ced04d 100644 --- a/crates/ui-egui/locales/ru.json +++ b/crates/ui-egui/locales/ru.json @@ -2539,6 +2539,7 @@ "Specific folder": "Указанная папка", "Same folder as original photo": "Та же папка, что и у оригинала", "Tidying the thumbnail cache": "Упорядочивание кэша миниатюр", + "Checking free disk space": "Проверка свободного места на диске", "Reset all edits (Cmd+Shift+R)": "Сбросить все правки (Cmd+Shift+R)", "Smart albums update automatically: change their rules, or use Photo ▸ Delete Photo": "Смарт-альбомы обновляются сами: измените их правила или выберите Фото → Удалить фото", "LightCraft stopped": "LightCraft остановился", @@ -2584,6 +2585,8 @@ "The photos are imported once it ends": "Фото будут импортированы после этого", "An import is running. Drop photos when it has finished": "Импорт уже выполняется. Перетащите фото, когда он закончится", "Finish Lightroom catalog import before adding photos": "Завершите импорт каталога Lightroom, прежде чем добавлять фото", + "Camera preview": "Превью камеры", + "The camera's own JPEG, shown until LightCraft has rendered the raw. Judge colour and tone on the rendered photo.": "Собственный JPEG камеры, показанный, пока LightCraft не обработает RAW. Оценивайте цвет и тон по обработанному фото.", "Profile: from the file (the camera's own correction)": "Профиль: из файла (коррекция самой камеры)", "Downloading the lens database…": "Загружается база объективов…", "This file names no lens.": "В этом файле не указан объектив.", diff --git a/crates/ui-egui/locales/sv.json b/crates/ui-egui/locales/sv.json index 4c4267d523..1b35ecccde 100644 --- a/crates/ui-egui/locales/sv.json +++ b/crates/ui-egui/locales/sv.json @@ -2242,6 +2242,7 @@ "Thursday": "Torsdag", "Tick the box to accept the model's terms first": "Markera rutan för att godkänna modellens villkor först", "Tidying the thumbnail cache": "Städar miniatyrcachen", + "Checking free disk space": "Kontrollerar ledigt diskutrymme", "Time zone": "Tidszon", "Tint": "Färgton", "Title": "Titel", diff --git a/crates/ui-egui/locales/uk.json b/crates/ui-egui/locales/uk.json index 085be4f7e4..7aafd1059a 100644 --- a/crates/ui-egui/locales/uk.json +++ b/crates/ui-egui/locales/uk.json @@ -2329,6 +2329,7 @@ "Specific folder": "Вказана папка", "Same folder as original photo": "Та сама папка, що й оригінал", "Tidying the thumbnail cache": "Упорядкування кешу мініатюр", + "Checking free disk space": "Перевірка вільного місця на диску", "Reset all edits (Cmd+Shift+R)": "Скинути всі зміни (Cmd+Shift+R)", "Smart albums update automatically: change their rules, or use Photo ▸ Delete Photo": "Розумні альбоми оновлюються самі: змініть їхні правила або виберіть Фото → Видалити фото", "LightCraft stopped": "LightCraft зупинився", @@ -2374,6 +2375,8 @@ "The photos are imported once it ends": "Фото буде імпортовано після цього", "An import is running. Drop photos when it has finished": "Імпорт уже виконується. Перетягніть фото, коли він завершиться", "Finish Lightroom catalog import before adding photos": "Завершіть імпорт каталогу Lightroom, перш ніж додавати фото", + "Camera preview": "Прев'ю камери", + "The camera's own JPEG, shown until LightCraft has rendered the raw. Judge colour and tone on the rendered photo.": "Власний JPEG камери, показаний, доки LightCraft не обробить RAW. Оцінюйте колір і тон за обробленим фото.", "Profile: from the file (the camera's own correction)": "Профіль: з файлу (корекція самої камери)", "Downloading the lens database…": "Завантажується база об'єктивів…", "This file names no lens.": "У цьому файлі не вказано об'єктив.", diff --git a/crates/ui-egui/locales/vi.json b/crates/ui-egui/locales/vi.json index dd93d0485c..07a4ca85f1 100644 --- a/crates/ui-egui/locales/vi.json +++ b/crates/ui-egui/locales/vi.json @@ -2568,6 +2568,7 @@ "Thumbnail size": "Kích thước ảnh thu nhỏ", "Import Thumbnail Size": "Kích thước ảnh thu nhỏ khi nhập", "Tidying the thumbnail cache": "Đang dọn dẹp bộ nhớ đệm ảnh thu nhỏ", + "Checking free disk space": "Đang kiểm tra dung lượng đĩa trống", "Title bar": "Thanh tiêu đề", "Title text": "Văn bản tiêu đề", "Titles": "Tiêu đề", diff --git a/crates/ui-egui/locales/zh-hans.json b/crates/ui-egui/locales/zh-hans.json index 5f138b6f9c..a199e84fe0 100644 --- a/crates/ui-egui/locales/zh-hans.json +++ b/crates/ui-egui/locales/zh-hans.json @@ -2164,6 +2164,7 @@ "Specific folder": "指定文件夹", "Same folder as original photo": "与原始照片相同的文件夹", "Tidying the thumbnail cache": "正在整理缩略图缓存", + "Checking free disk space": "正在检查可用磁盘空间", "Reset all edits (Cmd+Shift+R)": "重置所有编辑 (Cmd+Shift+R)", "Smart albums update automatically: change their rules, or use Photo ▸ Delete Photo": "智能相册会自动更新:请更改其规则,或使用“照片 → 删除照片”", "LightCraft stopped": "LightCraft 已停止", @@ -2221,6 +2222,8 @@ "The photos are imported once it ends": "结束后即导入这些照片", "An import is running. Drop photos when it has finished": "正在导入。导入完成后再拖放照片", "Finish Lightroom catalog import before adding photos": "添加照片前,请先完成 Lightroom 目录的导入", + "Camera preview": "相机预览", + "The camera's own JPEG, shown until LightCraft has rendered the raw. Judge colour and tone on the rendered photo.": "相机自己的 JPEG,在 LightCraft 渲染出原始文件之前显示。请在渲染后的照片上判断颜色和影调。", "Profile: from the file (the camera's own correction)": "配置文件:来自文件(相机自身的校正)", "Downloading the lens database…": "正在下载镜头数据库…", "This file names no lens.": "此文件未注明镜头。", diff --git a/crates/ui-egui/locales/zh-hant.json b/crates/ui-egui/locales/zh-hant.json index 275accbeb0..d5cb5462cc 100644 --- a/crates/ui-egui/locales/zh-hant.json +++ b/crates/ui-egui/locales/zh-hant.json @@ -2132,6 +2132,7 @@ "Specific folder": "指定資料夾", "Same folder as original photo": "與原始照片相同的資料夾", "Tidying the thumbnail cache": "正在整理縮圖快取", + "Checking free disk space": "正在檢查可用磁碟空間", "Reset all edits (Cmd+Shift+R)": "重設所有編輯(Cmd+Shift+R)", "Smart albums update automatically: change their rules, or use Photo ▸ Delete Photo": "智慧型相簿會自動更新:請變更其規則,或使用「照片 → 刪除照片」", "LightCraft stopped": "LightCraft 已停止", @@ -2177,6 +2178,8 @@ "The photos are imported once it ends": "結束後即匯入這些照片", "An import is running. Drop photos when it has finished": "正在匯入。匯入完成後再拖放照片", "Finish Lightroom catalog import before adding photos": "新增照片前,請先完成 Lightroom 目錄的匯入", + "Camera preview": "相機預覽", + "The camera's own JPEG, shown until LightCraft has rendered the raw. Judge colour and tone on the rendered photo.": "相機自己的 JPEG,在 LightCraft 算繪出原始檔之前顯示。請在算繪後的照片上判斷色彩與色調。", "Profile: from the file (the camera's own correction)": "設定檔:來自檔案(相機本身的校正)", "Downloading the lens database…": "正在下載鏡頭資料庫…", "This file names no lens.": "此檔案未註明鏡頭。", diff --git a/crates/ui-egui/src/app.rs b/crates/ui-egui/src/app.rs index bba91d2f71..34ffc3811b 100644 --- a/crates/ui-egui/src/app.rs +++ b/crates/ui-egui/src/app.rs @@ -65,6 +65,8 @@ impl LightcraftApp { #[cfg(test)] library_passes: vec![], #[cfg(test)] + prep_ticks: 0, + #[cfg(test)] missing_count_test_delay: None, shadow: None, synthetic: vec![], @@ -94,6 +96,9 @@ impl LightcraftApp { space_hand: None, brush_live: None, loupe_shown: None, + loupe_timing: Default::default(), + prep_synced: false, + look_ahead: Default::default(), region_view: None, region_before_view: None, size_hold: Default::default(), @@ -106,6 +111,7 @@ impl LightcraftApp { merge: merge::MergeState::default(), import: None, scan: None, + scans_stopping: Default::default(), waiting_import: Default::default(), sync: None, sync_owns_changes: false, @@ -123,6 +129,8 @@ impl LightcraftApp { window_is_fullscreen: false, gpu_applied: None, gpu_notice: false, + disk_space_notice: false, + disk_notice: None, memory_applied: None, display_applied: None, display_error: None, @@ -488,6 +496,41 @@ impl LightcraftApp { } } + /// The app-wide background preparation settings live with the app settings (`ui.json`): taken + /// into the session once, then a change made through `prefs.setBackgroundPrep` is saved with + /// them. Run at the start of every logic pass, before any command of that pass (also before + /// the fonts are ready and the window is drawn). + fn sync_prep_settings(&mut self) { + if !self.prep_synced { + self.session.prep.app = self.ui.settings.background_prep.sanitized(); + self.prep_synced = true; + } else if self.session.prep.app != self.ui.settings.background_prep { + self.ui.settings.background_prep = self.session.prep.app; + } + } + + /// Preparing photos in the background ([`lightcraft_engine::prep`]): the whole view's previews + /// once the user is culling in the loupe, built from the photo on screen in the direction of travel. + /// Advanced on every logic pass, the hidden ones too (a minimised window keeps preparing). + fn background_prep(&mut self, ctx: &egui::Context) { + #[cfg(test)] + { + self.prep_ticks += 1; + } + let mode = panels::detail::prep_mode(self); + let loupe = self.ui.fullscreen || self.ui.view == state::ViewMode::Detail; + let at = if loupe { self.session.active() } else { None }; + let backward = self.look_ahead.backward(); + self.session.prep_tick(mode, at, backward); + if self.session.preview_build.as_ref().is_some_and(|b| b.running()) { + ctx.request_repaint_after(std::time::Duration::from_millis(500)); + } + // a build that couldn't start tries again at its time, also when nothing else draws a frame + if let Some(wait) = self.session.prep_retry_in() { + ctx.request_repaint_after(wait); + } + } + /// AI masks: apply finished background requests (clicks, descriptions, detail passes) and /// show their errors; start a zoomed-in detail pass once the clicking stops /// (`ui.detail_due`); watch the model download. Never waits for the model. @@ -584,6 +627,7 @@ impl LightcraftApp { /// hidden, from `Context::run_logic`. It may read the refreshed viewport fields only: never /// `i.time`, events, keys or dropped files. pub fn logic(&mut self, ctx: &egui::Context) { + self.sync_prep_settings(); i18n::set_language(self.ui.language); // The fork's AccessKit root language and bidi runs follow the locale; presentation only. // Compare against the cached string: `Context::language()` allocates on every call, and diff --git a/crates/ui-egui/src/app/frame.rs b/crates/ui-egui/src/app/frame.rs index 91eebe1cee..224a993484 100644 --- a/crates/ui-egui/src/app/frame.rs +++ b/crates/ui-egui/src/app/frame.rs @@ -44,7 +44,8 @@ impl LightcraftApp { /// monotonic app clock, so a hidden `run_logic` keeps it advancing and a second layout pass is /// a no-op rather than a duplicate. pub(super) fn logic_background(&mut self, ctx: &egui::Context) { - panels::notices::logic(self); + self.background_prep(ctx); + panels::notices::logic(self, ctx); // closing the window (or Quit) with changes only in memory: retry, else ask first if ctx.input(|i| i.viewport().close_requested()) && !panels::notices::may_close(self) { ctx.send_viewport_cmd(egui::ViewportCommand::CancelClose); @@ -358,6 +359,8 @@ impl LightcraftApp { } self.widgets = std::mem::take(&mut self.next_widgets); self.widget_hits = std::mem::take(&mut self.next_widget_hits); + // a frame without the loupe (grid, Compare, Survey, slideshow…) ends the loupe's step timing + self.loupe_timing.end_frame(ctx.input(|i| i.time)); panels::notices::finish_ui_pass(self, ctx); panels::library_problem::finish_ui_pass(self, ctx); self.ui_passes = self.ui_passes.wrapping_add(1); diff --git a/crates/ui-egui/src/control.rs b/crates/ui-egui/src/control.rs index 3c6c2e1576..63403650b2 100644 --- a/crates/ui-egui/src/control.rs +++ b/crates/ui-egui/src/control.rs @@ -133,7 +133,7 @@ pub fn inspect(app: &LightcraftApp, ctx: &egui::Context) -> Value { "selection": app.session.selection.ids.iter().map(|p| p.0).collect::>(), "activeMask": app.session.active_mask, "widgetCount": app.widgets.len(), - "perf": {"viewFollowed": app.session.view_followed, "viewWorkedOut": app.session.view_worked_out, "frameMs": app.perf.frame_ms, "logicMs": app.perf.logic_ms, "updateMs": app.perf.update_ms, "maxUpdateMs": app.perf.max_update_ms, "fps": app.perf.fps, "lastRenderMs": app.renderer.last_main_ms, "renderQueue": app.renderer.queued(), "rendersInFlight": app.renderer.in_flight(), "pendingSlots": app.renderer.pending_slots(), "mergeRunning": app.merge.busy(), "lastMerge": app.merge.last_result, "rendersDone": app.renderer.completed, "thumbTextures": app.renderer.thumb_textures(), "variantTextures": app.renderer.variant_textures(), "gpu": (lightcraft_engine::gpu::ready() && lightcraft_engine::gpu::available()).then(lightcraft_engine::gpu::adapter_name).flatten(), "gpuReason": lightcraft_engine::gpu::unavailable_reason(), "gpuFallback": lightcraft_engine::gpu::last_fallback(), "gpuWindowDevice": lightcraft_engine::gpu::on_window_device(), "screenTextures": app.renderer.has_screen_textures(), "mainOnGpu": app.renderer.textures.get(&crate::render::Slot::Main).is_some_and(|t| t.native.is_some())}, + "perf": {"viewFollowed": app.session.view_followed, "viewWorkedOut": app.session.view_worked_out, "frameMs": app.perf.frame_ms, "logicMs": app.perf.logic_ms, "updateMs": app.perf.update_ms, "maxUpdateMs": app.perf.max_update_ms, "fps": app.perf.fps, "lastRenderMs": app.renderer.last_main_ms, "renderQueue": app.renderer.queued(), "rendersInFlight": app.renderer.in_flight(), "pendingSlots": app.renderer.pending_slots(), "mergeRunning": app.merge.busy(), "lastMerge": app.merge.last_result, "rendersDone": app.renderer.completed, "thumbTextures": app.renderer.thumb_textures(), "thumbsMade": app.renderer.thumbs_made, "variantTextures": app.renderer.variant_textures(), "gpu": (lightcraft_engine::gpu::ready() && lightcraft_engine::gpu::available()).then(lightcraft_engine::gpu::adapter_name).flatten(), "gpuReason": lightcraft_engine::gpu::unavailable_reason(), "gpuFallback": lightcraft_engine::gpu::last_fallback(), "gpuWindowDevice": lightcraft_engine::gpu::on_window_device(), "screenTextures": app.renderer.has_screen_textures(), "mainOnGpu": app.renderer.textures.get(&crate::render::Slot::Main).is_some_and(|t| t.native.is_some()), "loupeSteps": app.loupe_timing.to_json(), "previewCache": preview_cache_json(app)}, "loupe": app.loupe_shown.map(|(p, src)| json!({ "photo": p.0, "source": src, @@ -894,3 +894,20 @@ pub fn save_screenshot(app: &mut LightcraftApp, image: &egui::ColorImage, target } } } + +/// The preview cache's numbers for `ui.inspect` → `perf.previewCache`: what memory holds, and the +/// disk cache's hits, misses and writes this session (notes 47 and 48: are built previews used?). +fn preview_cache_json(app: &crate::LightcraftApp) -> serde_json::Value { + use std::sync::atomic::Ordering; + let (entries, bytes) = app.session.media.rendered.mem_usage(); + let disk = app.session.media.rendered.disk(); + serde_json::json!({ + "memEntries": entries, + "memBytes": bytes, + "diskHits": disk.map(|d| d.hits.load(Ordering::Relaxed)), + "diskMisses": disk.map(|d| d.misses.load(Ordering::Relaxed)), + "diskWrites": disk.map(|d| d.writes.load(Ordering::Relaxed)), + "diskBudget": disk.map(|d| d.budget()), + "loupeEdge": app.session.loupe_edge, + }) +} diff --git a/crates/ui-egui/src/dialogs/import.rs b/crates/ui-egui/src/dialogs/import.rs index e5bfab8817..c927608e01 100644 --- a/crates/ui-egui/src/dialogs/import.rs +++ b/crates/ui-egui/src/dialogs/import.rs @@ -34,11 +34,12 @@ pub(crate) use body::thumb_edge; use body::{DEFAULT_BELOW_GRID, badge_rect, dup_counts}; pub use body::{DEFAULT_FOLDER_TEMPLATE, body, example_destination, source_summary}; pub(crate) use body::{DIALOG_SIZE, THUMB_SIZE_DEFAULT, note_dialog_bottom, thumb_size}; +pub(crate) use scan::StoppingScans; #[cfg(not(target_arch = "wasm32"))] pub(crate) use scan::cards_seen; -pub use scan::{ScanTask, browse, open, poll_scan, running}; #[cfg(test)] -pub(crate) use scan::{stall_scan, stall_scan_of}; +pub(crate) use scan::{HELD_RESOLVES, ScanEnd, scan_by_hand, stall_scan, stall_scan_of}; +pub use scan::{ScanTask, browse, open, poll_scan, running}; pub(crate) use tags::{tag_field_width, tag_help, tag_toggle, unknown_tags_warning}; pub use task::{ImportTask, start, start_paths, tick}; diff --git a/crates/ui-egui/src/dialogs/import/scan.rs b/crates/ui-egui/src/dialogs/import/scan.rs index 67b1fa176b..c82f67123c 100644 --- a/crates/ui-egui/src/dialogs/import/scan.rs +++ b/crates/ui-egui/src/dialogs/import/scan.rs @@ -5,7 +5,7 @@ use super::{ImportDialog, LightcraftApp, Ordering, ScanInput, ScanOutput, ScanPr /// window must keep answering meanwhile). pub struct ScanTask { progress: std::sync::Arc, - rx: std::sync::mpsc::Receiver, + rx: std::sync::mpsc::Receiver, /// Open the review with "copy into the library" checked (a camera / card). pub copy: bool, /// Open the review with this album chosen (an album's context menu ▸ Import…). @@ -16,6 +16,91 @@ pub struct ScanTask { sources: Vec, /// The scan's row in the activity stack (its ✕ and `activity.cancel` set `progress.cancel`). guard: lightcraft_engine::activity::TaskGuard, + /// Browsing: the folder as the worker resolved it, once it has. The view shows it as given until then + /// (resolving reads the disk, which a stalled network mount can block: never on this thread). + resolved: Option>, +} + +/// What a scan's worker sends back. Every way it ends sends one, so the probes of earlier scans that +/// [`ScanInput::new`] took from the session always go back to it (a later import doesn't read those files again). +pub(crate) enum ScanEnd { + /// The scan finished. + Done(ScanOutput), + /// Browsing stopped without a listing: the folder couldn't be listed (`failed`), or another folder was + /// chosen. Never an empty listing, which the browse bookkeeping would take for every file gone. + Stopped { probes: std::collections::HashMap, failed: bool }, +} + +impl ScanEnd { + /// The probes this end hands back to the session. + fn into_probes(self) -> std::collections::HashMap { + match self { + ScanEnd::Done(out) => out.probes, + ScanEnd::Stopped { probes, .. } => probes, + } + } +} + +/// Scans stopped without waiting for their workers ([`stop_scan`]), in the order they were stopped, which is +/// the order they started (one scan runs at a time), and which of their probes went back to the session. +#[derive(Default)] +pub(crate) struct StoppingScans { + /// How many scans were stopped: each one's number. + stopped: u64, + /// Each stopped scan's number and channel, kept until its worker ends. + pending: Vec<(u64, std::sync::mpsc::Receiver)>, + /// The probes a stopped scan put back: by whom, and what they said (to tell they are still the ones there). + from: std::collections::HashMap, +} + +/// What tells two probes of one file apart: a file that changed but kept its size has another content hash. +type ProbeMark = (u64, Option, Option); + +fn mark(p: &lightcraft_engine::media::ProbeInfo) -> ProbeMark { + (p.file_size, p.content_hash.clone(), p.file_modified.clone()) +} + +/// Stop the running scan (if any) without waiting for its worker: the scan is closed at once, and its +/// channel is kept until the worker ends, so the probes it holds still come back ([`collect_stopped`]). +fn stop_scan(app: &mut LightcraftApp) { + if let Some(t) = app.scan.take() { + t.progress.cancel.store(true, Ordering::Relaxed); + let s = &mut app.scans_stopping; + s.stopped = s.stopped.saturating_add(1); + s.pending.push((s.stopped, t.rx)); + } +} + +/// The probes of stopped scans whose workers have ended go back to the session (called every frame). A probe +/// the session got since a scan started is newer than that scan's and stays, except one that a scan stopped +/// before it put back: a later scan read that file again, so its probe wins, whichever worker ends first +/// (an older probe of a file changed at the same size would hand the copy check a stale content hash). +fn collect_stopped(app: &mut LightcraftApp, ctx: &egui::Context) { + let probes = &mut app.session.import_probes; + let StoppingScans { pending, from, .. } = &mut app.scans_stopping; + pending.retain(|(seq, rx)| match rx.try_recv() { + Ok(end) => { + for (k, v) in end.into_probes() { + let replace = match probes.get(&k) { + None => true, + Some(have) => from.get(&k).is_some_and(|(by, m)| by < seq && *m == mark(have)), + }; + if replace { + from.insert(k.clone(), (*seq, mark(&v))); + probes.insert(k, v); + } + } + false + } + Err(std::sync::mpsc::TryRecvError::Empty) => true, + Err(std::sync::mpsc::TryRecvError::Disconnected) => false, + }); + if pending.is_empty() { + // no stopped scan left to compare with + from.clear(); + } else { + ctx.request_repaint_after(std::time::Duration::from_millis(100)); + } } /// The activity row of a scan: the review's ("Scanning folder") or the Local view's ("Reading folder"). @@ -66,14 +151,14 @@ pub fn open(app: &mut LightcraftApp, paths: Vec) -> Result) -> Result) -> Result { - let dir = std::path::absolute(std::path::Path::new(path)).map_err(|e| e.to_string())?; - if !dir.is_dir() { - return Err(format!("{path}: not a folder")); - } + // one spelling for the view, the listing and the catalog's records: `a/lib/../outside` is `a/outside`. Taken out + // by name here, without reading the disk (a link on a stalled network mount would freeze the window): the worker + // resolves the folder, checks it is one, and the view takes its spelling when it says (see `poll_scan`) + let dir = lightcraft_engine::import::browse_spelling(std::path::Path::new(path)).map_err(|e| e.to_string())?; if app.scan.as_ref().is_some_and(|t| !t.browse) || app.import.as_ref().is_some_and(|t| !t.browse) { return Err("an import is running".into()); } let dir_s = dir.to_string_lossy().trim_end_matches(['/', '\\']).to_string(); let subfolders = subfolders.unwrap_or_else(|| app.session.browse.as_ref().is_some_and(|b| b.subfolders)); let running = app.scan.as_ref().is_some_and(|t| t.browse) || app.import.as_ref().is_some_and(|t| t.browse); - if running && app.session.browse.as_ref().is_some_and(|b| b.path == dir_s && b.subfolders == subfolders) { + // (a path through `..` can name another folder than its spelling, past a link: only the worker can tell, so it + // browses again; CodeRabbit on #3583) + let through_dotdot = std::path::Path::new(path).components().any(|c| c == std::path::Component::ParentDir); + if running && !through_dotdot && app.session.browse.as_ref().is_some_and(|b| b.path == dir_s && b.subfolders == subfolders) { // already reading this folder: clicking it again must not restart the progress app.session.source = lightcraft_engine::LibrarySource::Folder; app.session.keep_active_in_view(); return Ok(json!({"path": dir_s, "subfolders": subfolders, "scanning": true})); } - if let Some(t) = app.scan.take() { - t.progress.cancel.store(true, Ordering::Relaxed); - } + stop_scan(app); app.import = None; app.session.browse = Some(lightcraft_engine::Browse { path: dir_s.clone(), subfolders }); app.session.source = lightcraft_engine::LibrarySource::Folder; // the photos the library already knows from this folder show at once: keep or pick the active one app.session.keep_active_in_view(); - let (input, _) = ScanInput::new(&mut app.session, std::slice::from_ref(&dir_s)); + let (mut input, _) = ScanInput::new(&mut app.session, std::slice::from_ref(&dir_s)); let progress = std::sync::Arc::new(ScanProgress::default()); let (tx, rx) = std::sync::mpsc::channel(); let p = progress.clone(); - let root = dir_s.clone(); + // the worker resolves the path as given: on Unix `link/..` is the folder above where the link points, which the + // spelling (`..` taken out by name) can't tell + let given = path.to_owned(); + let sort = app.session.sort; + let (resolved_tx, resolved) = std::sync::mpsc::channel(); let job = move || { - let files: Vec = if subfolders { - lightcraft_engine::import::expand(&[root], None) - } else { - let mut v: Vec = std::fs::read_dir(&root) - .map(|rd| { - rd.flatten() - .filter(|e| !lightcraft_engine::walk::is_hidden_entry(e)) - .map(|e| e.path()) - .filter(|f| f.is_file() && lightcraft_engine::import::is_supported(f)) - .map(|f| f.to_string_lossy().to_string()) - .collect() - }) - .unwrap_or_default(); - v.sort(); - v + let root = match resolve_browse_root(&given) { + Ok(root) => root, + Err(e) => { + log::warn!("browse: can't read {given}: {e}"); + let _ = tx.send(ScanEnd::Stopped { probes: input.take_probes(), failed: true }); + return; + } }; - let _ = tx.send(scan_with(input, &files, &p)); + let _ = resolved_tx.send(root.clone()); + // (never the library's own folder, which the import would otherwise read as photos) + let files = match lightcraft_engine::import::browse_files(&input, &root, subfolders) { + Ok(files) => files, + Err(e) => { + // no listing is sent: the scan ends as failed, and no record is taken for a file gone + log::warn!("browse: can't list {root}: {e}"); + let _ = tx.send(ScanEnd::Stopped { probes: input.take_probes(), failed: true }); + return; + } + }; + // Nothing is read here: the import that follows reads the files and adds them batch by + // batch, first screen first, so the view fills while the rest is read (PERF-CULL-1). + // Another folder chosen meanwhile (`p.cancel`) stops the ordering, and no listing is sent. + let Some(queue) = lightcraft_engine::import::browse_queue_until(&input, files.clone(), &sort, &p.cancel) else { + let _ = tx.send(ScanEnd::Stopped { probes: input.take_probes(), failed: false }); + return; + }; + p.total.store(queue.len(), Ordering::Relaxed); + p.done.store(queue.len(), Ordering::Relaxed); + // the files the library knows come too, marked as such: the folder bookkeeping (browse + // times, records of files gone) needs the whole listing; only the queue is imported + let queued: std::collections::HashSet<&String> = queue.iter().collect(); + let known: Vec = files.iter().filter(|f| !queued.contains(f)).cloned().collect(); + let known = + known.into_iter().map(|path| lightcraft_engine::import::ImportCandidate { path, duplicate: Some("path".into()), ..Default::default() }); + let candidates = + queue.iter().cloned().map(|path| lightcraft_engine::import::ImportCandidate { path, ..Default::default() }).chain(known).collect(); + // the probes of earlier scans go back to the session with the result, for the import that follows + let _ = tx.send(ScanEnd::Done(lightcraft_engine::import::ScanOutput { candidates, probes: input.take_probes() })); }; #[cfg(not(target_arch = "wasm32"))] std::thread::spawn(job); + // the job takes the probes back out of `input`, so it is an FnMut where it runs in place #[cfg(target_arch = "wasm32")] - job(); + { + let mut job = job; + job(); + } let guard = scan_guard(app, &progress, true); - app.scan = Some(ScanTask { progress, rx, copy: false, album: None, browse: true, sources: Vec::new(), guard }); + app.scan = Some(ScanTask { progress, rx, copy: false, album: None, browse: true, sources: Vec::new(), guard, resolved: Some(resolved) }); app.renderer.forget_imports(); Ok(json!({"path": dir_s, "subfolders": subfolders, "scanning": true})) } +/// The browsed folder `given` (the path as `browse` received it, not its [`lightcraft_engine::import::browse_spelling`]) +/// as [`lightcraft_engine::import::browse_path`] resolves it, checked to be a folder: on the browse worker, as it reads the disk. +fn resolve_browse_root(given: &str) -> Result { + #[cfg(test)] + held_resolve(given); + let dir = lightcraft_engine::import::browse_path(std::path::Path::new(given)).map_err(|e| e.to_string())?; + if !dir.is_dir() { + return Err(format!("{given}: not a folder")); + } + Ok(dir.to_string_lossy().trim_end_matches(['/', '\\']).to_string()) +} + +/// Tests: browsed folders whose resolution waits (as on a stalled network mount) while they are listed here. +#[cfg(test)] +pub(crate) static HELD_RESOLVES: std::sync::Mutex> = std::sync::Mutex::new(Vec::new()); + +#[cfg(test)] +fn held_resolve(given: &str) { + let held = || HELD_RESOLVES.lock().unwrap_or_else(std::sync::PoisonError::into_inner).iter().any(|p| p == given); + let start = std::time::Instant::now(); + while held() && start.elapsed() < std::time::Duration::from_secs(30) { + std::thread::sleep(std::time::Duration::from_millis(5)); + } +} + /// Collect a finished scan and open the review (called every frame). pub fn poll_scan(app: &mut LightcraftApp, ctx: &egui::Context) { show_pending_review(app, ctx); + collect_stopped(app, ctx); let Some(task) = app.scan.as_ref() else { return }; ctx.request_repaint_after(std::time::Duration::from_millis(100)); - let out = match task.rx.try_recv() { + // the worker resolved the browsed folder: the view takes its spelling (before its listing, sent after it) + if let Some(root) = task.resolved.as_ref().and_then(|r| r.try_recv().ok()) + && let Some(b) = app.session.browse.as_mut().filter(|b| b.path != root) + { + b.path = root; + app.session.keep_active_in_view(); + } + let Some(task) = app.scan.as_ref() else { return }; + let end = match task.rx.try_recv() { Ok(o) => o, Err(std::sync::mpsc::TryRecvError::Empty) => { if task.guard.is_cancelled() { // ✕: the worker stops at its next file; don't wait for it (a NAS read can take a while) - app.scan = None; + stop_scan(app); return; } let (total, done) = (task.progress.total.load(Ordering::Relaxed), task.progress.done.load(Ordering::Relaxed)); @@ -160,16 +309,32 @@ pub fn poll_scan(app: &mut LightcraftApp, ctx: &egui::Context) { return; } Err(_) => { + // a cancelled scan whose worker ended without a result was not a failure + let cancelled = task.guard.is_cancelled() || task.progress.cancel.load(Ordering::Relaxed); app.scan = None; - app.toast(ctx, crate::i18n::tr("Scan failed")); + if !cancelled { + app.toast(ctx, crate::i18n::tr("Scan failed")); + } return; } }; let Some(task) = app.scan.take() else { return }; - if task.progress.cancel.load(Ordering::Relaxed) { + let cancelled = task.progress.cancel.load(Ordering::Relaxed); + let out = match end { + ScanEnd::Done(out) => out, + ScanEnd::Stopped { probes, failed } => { + app.session.import_probes.extend(probes); + if failed && !cancelled { + app.toast(ctx, crate::i18n::tr("Scan failed")); + } + return; + } + }; + // the probes go back to the session however the scan ended (the ones of a scan stopped earlier may be there) + app.session.import_probes.extend(out.probes); + if cancelled { return; } - app.session.import_probes = out.probes; if task.browse { // the folder and the ones its photos are in count as browsed now, as `library.browse` records them, so // their untouched Local photos aren't forgotten while the folder is in use @@ -247,11 +412,17 @@ pub(crate) fn stall_scan(app: &mut LightcraftApp) -> Box { /// Tests: the same, for a Local folder browse scan (`browse`) instead of the review's scan. #[cfg(test)] pub(crate) fn stall_scan_of(app: &mut LightcraftApp, browse: bool) -> Box { + Box::new(scan_by_hand(app, browse)) +} + +/// Tests: a scan whose worker is the test: what it sends through the returned channel is the scan's end. +#[cfg(test)] +pub(crate) fn scan_by_hand(app: &mut LightcraftApp, browse: bool) -> std::sync::mpsc::Sender { let progress = std::sync::Arc::new(ScanProgress::default()); - let (tx, rx) = std::sync::mpsc::channel::(); + let (tx, rx) = std::sync::mpsc::channel::(); let guard = scan_guard(app, &progress, false); - app.scan = Some(ScanTask { progress, rx, copy: false, album: None, browse, sources: Vec::new(), guard }); - Box::new(tx) + app.scan = Some(ScanTask { progress, rx, copy: false, album: None, browse, sources: Vec::new(), guard, resolved: None }); + tx } /// Whether an import or any scan (the review's, or a Local folder browse) is running, or a finished scan's diff --git a/crates/ui-egui/src/dialogs/settings.rs b/crates/ui-egui/src/dialogs/settings.rs index 271b6e7c91..cada6c7fba 100644 --- a/crates/ui-egui/src/dialogs/settings.rs +++ b/crates/ui-egui/src/dialogs/settings.rs @@ -57,8 +57,8 @@ pub const TABS: &[(&str, &str)] = &[ ("denoise", "AI Denoise"), ]; -/// Thumbnail cache sizes offered (MB). -const CACHE_SIZES: [u32; 5] = [512, 1024, 2048, 4096, 8192]; +/// Preview cache sizes offered (MB; 0 = automatic: room for every photo of the library). +const CACHE_SIZES: [u32; 8] = [512, 1024, 2048, 4096, 8192, 51_200, 204_800, 0]; const LABEL_W: f32 = 150.0; diff --git a/crates/ui-egui/src/dialogs/settings/performance.rs b/crates/ui-egui/src/dialogs/settings/performance.rs index 370cf08bac..214dd38243 100644 --- a/crates/ui-egui/src/dialogs/settings/performance.rs +++ b/crates/ui-egui/src/dialogs/settings/performance.rs @@ -49,15 +49,30 @@ pub(super) fn performance_tab(app: &mut LightcraftApp, ui: &mut egui::Ui, t: &To let opts = [(0u32, auto.as_str()), (512, "512 MB"), (1024, "1 GB"), (2048, "2 GB"), (4096, "4 GB")]; choices(ui, "settingsMemory", &opts, &mut app.ui.settings.memory_mb); }); - heading(ui, t, crate::i18n::tr("Thumbnail cache")); - let cur = (app.session.cache_bytes() >> 20) as u32; + heading(ui, t, crate::i18n::tr("Preview cache")); + let cur = app.session.cache_mb; row(ui, t, crate::i18n::tr("Size limit"), |ui| { - let opts = [(512u32, "512 MB"), (1024, "1 GB"), (2048, "2 GB"), (4096, "4 GB"), (8192, "8 GB")]; - let mut v = if CACHE_SIZES.contains(&cur) { cur } else { 2048 }; + let gb = lightcraft_engine::media::default_disk_cache_bytes(app.session.catalog.len()).div_ceil(1 << 30); + let auto = format!("{} ({gb} GB)", crate::i18n::tr("Automatic")); + let custom = custom_cache_option(cur); + let mut opts = + vec![(512u32, "512 MB"), (1024, "1 GB"), (2048, "2 GB"), (4096, "4 GB"), (8192, "8 GB"), (51_200, "50 GB"), (204_800, "200 GB")]; + opts.extend(custom.as_ref().map(|(mb, l)| (*mb, l.as_str()))); + opts.push((0, auto.as_str())); + // (a saved size the list doesn't offer shows as its own option, not as Automatic: picking + // Automatic then changes it; CodeRabbit on #3583) + let mut v = if CACHE_SIZES.contains(&cur) || custom.is_some() { cur } else { 0 }; if choices(ui, "settingsCache", &opts, &mut v) { let _ = app.run("library.preferences", json!({"cacheMb": v})); } }); + hint( + ui, + t, + crate::i18n::tr( + "Grid thumbnails and the previews the loupe shows, kept on disk so photos open at once next time. Automatic makes room for every photo in this library (about 1.5 MB a photo). 50 GB: normal · 200 GB: better, for several weddings or large shoots.", + ), + ); let used = app.session.media.rendered.disk().map(|d| d.size()); row(ui, t, crate::i18n::tr("In use"), |ui| { ui.label( @@ -70,6 +85,7 @@ pub(super) fn performance_tab(app: &mut LightcraftApp, ui: &mut egui::Ui, t: &To let _ = app.run("library.clearPreviews", json!({})); } }); + background_prep(app, ui, t); heading(ui, t, crate::i18n::tr("Local folders")); row(ui, t, crate::i18n::tr("Forget unchanged photos"), |ui| { let opts = [(0u32, "Never"), (7, "After 7 days"), (30, "After 30 days"), (90, "After 90 days"), (365, "After a year")]; @@ -89,6 +105,92 @@ pub(super) fn performance_tab(app: &mut LightcraftApp, ui: &mut egui::Ui, t: &To smart_previews(app, ui, t); } +/// Settings ▸ Performance ▸ Prepare photos in the background (notes 47 and 48): per mode, what +/// is made ready before the user gets to it (`prefs.backgroundPrep`), app-wide or for this library. +fn background_prep(app: &mut LightcraftApp, ui: &mut egui::Ui, t: &Tokens) { + use lightcraft_engine::prep::{Mode, Scope}; + heading(ui, t, crate::i18n::tr("Prepare photos in the background")); + hint( + ui, + t, + crate::i18n::tr( + "While you go through photos in the loupe, the next ones are made ready before you reach them, and when the computer is otherwise idle the previews of every photo in view are built. The photo on screen always comes first; the work shows in the activity list, where it can be stopped.", + ), + ); + let own = app.session.prep.library.is_some(); + let lib_open = app.session.library.is_some(); + if lib_open { + let mut v = own; + if check(ui, "settings.prepOwn", &mut v, "Use different settings for this library") { + let r = if v { + let cur = app.session.prep_settings(); + app.run("prefs.setBackgroundPrep", json!({"library": true, "settings": cur})) + } else { + app.run("prefs.setBackgroundPrep", json!({"reset": true})) + }; + if let Err(e) = r { + log::warn!("background preparation: {e}"); + } + } + hint(ui, t, crate::i18n::tr("For example, a timelapse library can prepare nothing while a wedding library prepares everything.")); + } + let cur = app.session.prep_settings(); + let mut patch: Option = None; + let scopes = [(Scope::All, "Every photo"), (Scope::Marked, "Picks, rated and labelled"), (Scope::OnScreen, "Only the photo on screen")]; + for (mode, key, title) in [(Mode::Culling, "culling", "Culling (a folder before import)"), (Mode::Library, "library", "Library")] { + let m = cur.mode(mode); + ui.add_space(4.0); + ui.label(RichText::new(crate::i18n::tr(title)).font(t.semibold(12.0)).color(t.text)); + let mut previews = m.previews; + row(ui, t, crate::i18n::tr("Previews"), |ui| { + if check(ui, &format!("settings.prep.{key}.previews"), &mut previews, "Build previews for every photo in view") { + patch = Some(json!({key: {"previews": previews}})); + } + }); + hint(ui, t, crate::i18n::tr("Starts once you step through a few photos. Uses about 1.5 MB of disk a photo, and the computer's spare time.")); + let mut ahead = m.ahead; + row(ui, t, crate::i18n::tr("Photos ahead"), |ui| { + let opts = [(0u8, "Off"), (3, "3"), (5, "5"), (10, "10"), (20, "20")]; + if choices(ui, &format!("settingsPrepAhead-{key}"), &opts, &mut ahead) { + patch = Some(json!({key: {"ahead": ahead}})); + } + }); + let mut scope = m.scope; + row(ui, t, crate::i18n::tr("Which photos"), |ui| { + if choices(ui, &format!("settingsPrepScope-{key}"), &scopes, &mut scope) { + patch = Some(json!({key: {"scope": scope}})); + } + }); + } + ui.add_space(4.0); + ui.label(RichText::new(crate::i18n::tr("Editing")).font(t.semibold(12.0)).color(t.text)); + let mut ahead = cur.develop.ahead; + row(ui, t, crate::i18n::tr("Photos ahead"), |ui| { + let opts = [(0u8, "Off"), (1, "1"), (2, "2"), (3, "3"), (5, "5")]; + if choices(ui, "settingsPrepAhead-develop", &opts, &mut ahead) { + patch = Some(json!({"develop": {"ahead": ahead}})); + } + }); + hint( + ui, + t, + crate::i18n::tr( + "The next photos are opened and kept ready to edit, so the first slider move on them is quick. Uses memory: a few hundred MB a photo.", + ), + ); + let mut scope = cur.develop.scope; + row(ui, t, crate::i18n::tr("Which photos"), |ui| { + if choices(ui, "settingsPrepScope-develop", &scopes, &mut scope) { + patch = Some(json!({"develop": {"scope": scope}})); + } + }); + if let Some(patch) = patch + && let Err(e) = app.run("prefs.setBackgroundPrep", json!({"library": own, "settings": patch})) + { + log::warn!("background preparation: {e}"); + } +} + /// Where this library keeps its smart previews (the offline-editing proxies, which can be large): /// the effective folder, what is in it, and choosing another one. #[cfg(not(target_arch = "wasm32"))] @@ -193,3 +295,24 @@ fn smart_previews(app: &mut LightcraftApp, ui: &mut egui::Ui, t: &Tokens) { } hint(ui, t, crate::i18n::tr("Keep it on a drive with room: smart previews are about 1 MB per photo. The thumbnail cache stays in the library.")); } + +/// The cache size limit saved in the settings, as an option of its own when the list doesn't offer +/// it (set through `library.preferences`, or by an older version): its value and label. +fn custom_cache_option(cur: u32) -> Option<(u32, String)> { + if cur == 0 || CACHE_SIZES.contains(&cur) { + return None; + } + let label = if cur.is_multiple_of(1024) { format!("{} GB", cur / 1024) } else { format!("{cur} MB") }; + Some((cur, label)) +} + +#[cfg(test)] +mod tests { + #[test] + fn a_saved_cache_size_the_list_does_not_offer_is_an_option() { + assert_eq!(super::custom_cache_option(0), None, "Automatic"); + assert_eq!(super::custom_cache_option(51_200), None, "offered"); + assert_eq!(super::custom_cache_option(10_240), Some((10_240, "10 GB".to_string()))); + assert_eq!(super::custom_cache_option(3000), Some((3000, "3000 MB".to_string()))); + } +} diff --git a/crates/ui-egui/src/headless/tests.rs b/crates/ui-egui/src/headless/tests.rs index 7ba584faf0..43465fee78 100644 --- a/crates/ui-egui/src/headless/tests.rs +++ b/crates/ui-egui/src/headless/tests.rs @@ -527,6 +527,10 @@ fn demo_grid_snapshot_has_ui_pixels() { assert!(bright > 30, "no text in the top bar ({bright} bright px)"); // thumbnails arrived (photo textures were drawn) assert!(h.app.renderer.thumb_textures() > 0); + // and are counted for benchmarks, a count that eviction doesn't lower + let r = h.request("ui.inspect", json!({}), SETTLE); + let made = r["result"]["perf"]["thumbsMade"].as_u64().unwrap_or(0); + assert!(made >= h.app.renderer.thumb_textures() as u64, "{}", r["result"]["perf"]); } #[test] @@ -3416,6 +3420,15 @@ fn browsing_from_the_ui_records_the_folder() { assert_eq!(h.app.session.visible_cloned().len(), 1); assert!(h.app.session.catalog.last_browsed(&root).is_some(), "the browsed folder"); assert!(h.app.session.catalog.last_browsed(&day_s).is_some(), "the folder its photo is in"); + // browsed again: a subfolder with only photos the library knows (nothing to import) still + // counts as browsed + h.app.session.catalog.apply(lightcraft_catalog::Op::SetBrowsed { folder: day_s.clone(), at: None }).unwrap(); + assert!(h.app.session.catalog.last_browsed(&day_s).is_none()); + let r = h.request("engine.execute", json!({"command": "library.browse", "params": {"path": root, "subfolders": true}}), t); + assert_eq!(r["result"]["scanning"], true, "{r}"); + h.settle(SETTLE); + assert_eq!(h.app.session.visible_cloned().len(), 1); + assert!(h.app.session.catalog.last_browsed(&day_s).is_some(), "a folder of known photos only"); let _ = std::fs::remove_dir_all(&dir); } @@ -5057,6 +5070,133 @@ fn screenshot_request_is_answered_without_a_window() { assert_eq!(r["result"]["width"], 640, "{r}"); } +/// Note 47: stepping in the loupe to a photo whose preview was built shows that preview as the +/// photo (fitted to the loupe, no render), and standard previews are built at the loupe's size. +#[test] +fn the_loupe_shows_built_previews_without_rendering_again() { + use crate::render::Slot; + use lightcraft_engine::media::QuickSource; + let mut h = demo([1000.0, 700.0]); + let t = Duration::from_secs(10); + // nothing prepared ahead here: what the loupe shows comes from the build alone + h.request( + "engine.execute", + json!({"command": "prefs.setBackgroundPrep", "params": {"settings": {"library": {"ahead": 0, "previews": false}}}}), + t, + ); + h.request("ui.set", json!({"view": "detail", "right": "none"}), t); + h.settle(SETTLE); + let edge = h.app.session.loupe_edge; + assert!(edge > 100, "the loupe said its size: {edge}"); + let ids = h.app.session.visible_cloned(); + let all: Vec = ids.iter().map(|i| i.0).collect(); + let r = h.request("engine.execute", json!({"command": "library.buildPreviews", "params": {"wait": true, "edge": edge, "ids": all}}), t); + assert_eq!(r["result"]["done"], ids.len(), "{r}"); + let next = ids.get(1).copied().expect("a second photo"); + h.request("engine.execute", json!({"command": "library.select", "params": {"ids": [next.0], "active": next.0}}), t); + assert!(h.step_until(SETTLE, |h| h.app.renderer.textures.get(&Slot::Main).is_some_and(|t| t.photo == next))); + let main = h.app.renderer.textures.get(&Slot::Main).unwrap(); + assert_eq!(main.quick, Some(QuickSource::Cached), "the built preview is the loupe's picture"); + let r = h.request("ui.inspect", json!({}), t); + assert!(r["result"]["perf"]["previewCache"]["memEntries"].as_u64().unwrap_or(0) > 0, "{}", r["result"]["perf"]["previewCache"]); +} + +/// Editing, the loupe renders the photo from its source even when a built preview of it is cached +/// (CodeRabbit on #3077): the cached view would show at once but leave the source undecoded, and +/// the first slider draft would wait for it. +#[test] +fn editing_renders_the_loupe_from_its_source_not_a_built_preview() { + use crate::render::Slot; + use lightcraft_engine::media::QuickSource; + let mut h = demo([1000.0, 700.0]); + let t = Duration::from_secs(10); + h.request( + "engine.execute", + json!({"command": "prefs.setBackgroundPrep", "params": {"settings": {"library": {"ahead": 0, "previews": false}, "develop": {"ahead": 0}}}}), + t, + ); + h.request("ui.set", json!({"view": "detail", "right": "none"}), t); + h.settle(SETTLE); + let edge = h.app.session.loupe_edge; + let ids = h.app.session.visible_cloned(); + let all: Vec = ids.iter().map(|i| i.0).collect(); + let r = h.request("engine.execute", json!({"command": "library.buildPreviews", "params": {"wait": true, "edge": edge, "ids": all}}), t); + assert_eq!(r["result"]["done"], ids.len(), "{r}"); + h.request("ui.set", json!({"right": "edit"}), t); + let next = ids.get(1).copied().expect("a second photo"); + h.request("engine.execute", json!({"command": "library.select", "params": {"ids": [next.0], "active": next.0}}), t); + assert!(h.step_until(SETTLE, |h| h.app.renderer.textures.get(&Slot::Main).is_some_and(|t| t.photo == next))); + let main = h.app.renderer.textures.get(&Slot::Main).unwrap(); + assert_ne!(main.quick, Some(QuickSource::Cached), "rendered from the source while editing"); +} + +/// The loupe holds background preparation back only while its render is on the way (CodeRabbit on +/// #3077): once the photo is shown, the frames that follow don't renew the hold. +#[test] +fn a_shown_loupe_render_does_not_hold_background_preparation() { + use crate::render::Slot; + let mut h = demo([1000.0, 700.0]); + let t = Duration::from_secs(10); + h.request("ui.set", json!({"view": "detail", "right": "none"}), t); + let id = h.app.session.active().expect("an active photo"); + let signals = || crate::panels::detail::FOREGROUND_SIGNALS.with(std::cell::Cell::get); + let before = signals(); + assert!( + h.step_until(SETTLE, |h| h.app.renderer.textures.get(&Slot::Main).is_some_and(|t| t.photo == id) && !h.app.renderer.is_pending(Slot::Main)) + ); + assert!(signals() > before, "the hold was asked for while the render was on its way"); + let shown = signals(); + for _ in 0..10 { + h.step(); + } + assert_eq!(signals(), shown, "no hold while the render is on screen"); +} + +/// The app-wide background preparation settings are kept with the app settings (`ui.json`): +/// the saved ones reach the session at launch, and a change made through the command is saved. +#[test] +fn app_wide_preparation_settings_are_saved_with_the_app_settings() { + let mut h = demo([1000.0, 700.0]); + let t = Duration::from_secs(10); + let mut saved = lightcraft_engine::prep::PrepSettings::default(); + saved.library.ahead = 3; + // (as if read from ui.json before the first frame) + h.app.ui.settings.background_prep = saved; + h.app.prep_synced = false; + h.step(); + assert_eq!(h.app.session.prep.app, saved, "the saved settings reach the session"); + h.request("engine.execute", json!({"command": "prefs.setBackgroundPrep", "params": {"settings": {"library": {"ahead": 10}}}}), t); + h.step(); + assert_eq!(h.app.ui.settings.background_prep.library.ahead, 10, "a change is kept with the app settings"); + assert_eq!(h.app.session.prep.app.library.ahead, 10); +} + +/// Note 48: the loupe prepares as many photos ahead as the settings say, a few at a time, and +/// none when that is off. +#[test] +fn the_loupe_prepares_the_photos_ahead_the_settings_ask_for() { + let mut h = demo([1000.0, 700.0]); + let t = Duration::from_secs(10); + let n = h.app.session.visible_cloned().len(); + assert!(n >= 4, "the demo has photos to step through"); + h.request("engine.execute", json!({"command": "prefs.setBackgroundPrep", "params": {"settings": {"library": {"ahead": 3}}}}), t); + h.request("ui.set", json!({"view": "detail", "right": "none"}), t); + h.settle(SETTLE); + for _ in 0..40 { + h.step(); + } + h.settle(SETTLE); + // the first photo: three ahead (no photo behind it) + assert_eq!(h.app.renderer.prefetch_slots_asked(), 3.min(n - 1)); + h.app.renderer.forget_all(); + h.request("engine.execute", json!({"command": "prefs.setBackgroundPrep", "params": {"settings": {"library": {"ahead": 0}}}}), t); + for _ in 0..40 { + h.step(); + } + h.settle(SETTLE); + assert_eq!(h.app.renderer.prefetch_slots_asked(), 0, "off: nothing prepared ahead"); +} + /// A size typed in the Export dialog is the size exported: the pixel fields kept only multiples /// of 16, so 1080 became 1088 and 3000 became 3008 (issue #3300). #[test] diff --git a/crates/ui-egui/src/lib.rs b/crates/ui-egui/src/lib.rs index 00311afd33..009238d2bb 100644 --- a/crates/ui-egui/src/lib.rs +++ b/crates/ui-egui/src/lib.rs @@ -33,6 +33,7 @@ pub mod icons; pub mod lens_db; pub mod lightroom_import; pub mod links; +pub mod loupe_timing; pub mod menus; mod missing_sweep; mod model_setup; @@ -223,6 +224,9 @@ pub struct LightcraftApp { /// effects (which the global shortcut guard also hides). #[cfg(test)] pub(crate) library_passes: Vec, + /// Test-only: how many times background preparation was advanced (once per logic pass). + #[cfg(test)] + pub(crate) prep_ticks: u32, /// Test-only: a delay for the sidebar's Missing Photos count worker (a slow disk). Kept here, /// not in `ui`, because `ui.set` rebuilds `ui` and would drop it. #[cfg(test)] @@ -289,6 +293,14 @@ pub struct LightcraftApp { /// What the loupe drew last frame: photo and source ("render", "cached", "embedded", "small", /// "thumb", "none"). pub loupe_shown: Option<(lightcraft_catalog::PhotoId, &'static str)>, + /// When each photo stepped to in the loupe first showed (`ui.inspect` → `perf.loupeSteps`). + pub loupe_timing: loupe_timing::LoupeTiming, + /// The app-wide background preparation settings were handed from the saved app settings to + /// the session ([`Self::background_prep`]). + prep_synced: bool, + /// The loupe's look-ahead: the photo it was made for, the direction of travel (+1 / -1), the + /// catalog revision and the edge, so it is worked out again only when one of them changes. + pub(crate) look_ahead: panels::detail::LookAhead, /// The window render the loupe asked for last, by job key (see [`region`]); kept for the few /// windows whose textures can be on screen, and for the inspector. pub region_view: Option, @@ -314,6 +326,9 @@ pub struct LightcraftApp { pub import: Option, /// A folder scan in progress (feeds the import review). pub scan: Option, + /// Scans stopped without waiting for their workers: each channel is kept until its worker ends, so + /// the probes it took from the session come back. + pub(crate) scans_stopping: import::StoppingScans, /// Files that waited for a library and wait for the running import or Synchronize Folder too. pub waiting_import: panels::library_problem::WaitingImport, /// A Synchronize Folder scan in progress (feeds its dialog). @@ -350,6 +365,14 @@ pub struct LightcraftApp { /// Say once, when the GPU device has been tried, that no graphics processor was found /// ([`panels::notices::gpu_notice`]). The desktop app sets it; headless sessions and tests don't. pub gpu_notice: bool, + /// Tell the user once per session when the drive holding the preview cache runs low + /// ([`panels::notices::disk_space_notice`]). The desktop app sets it; headless sessions and tests + /// don't (their drive is whatever the machine has). + pub disk_space_notice: bool, + /// The low-disk notice's text while it is queued in [`LightcraftApp::notices`]: it is shown with a + /// button to Settings ▸ Performance. With the library it was raised for + /// ([`lightcraft_engine::Session::library_generation`]): another library open drops it. + pub(crate) disk_notice: Option<(u64, String)>, /// The memory budget setting last applied (MB, 0 = automatic). memory_applied: Option, /// The display profile setting last applied (`app.displayProfile`). diff --git a/crates/ui-egui/src/loupe_timing.rs b/crates/ui-egui/src/loupe_timing.rs new file mode 100644 index 0000000000..e0df23768f --- /dev/null +++ b/crates/ui-egui/src/loupe_timing.rs @@ -0,0 +1,269 @@ +//! How long the loupe takes to show each photo it is stepped to, measured on the frames it draws. +//! +//! Culling is stepping through photos, so the number that matters is the time from the step (the +//! first frame that draws the new photo) to the first frame that shows something of it, to the +//! first frame with a full-size picture (the camera's embedded JPEG or a cached render) and to the +//! first frame with LightCraft's own render. `ui.inspect` → `perf.loupeSteps` reports the latest +//! steps; `perf-scratch`-style drivers turn them into medians and p95 without the control +//! channel's own round trip in the numbers. + +use std::collections::VecDeque; + +use lightcraft_catalog::PhotoId; +use serde_json::{Value, json}; + +/// Steps kept for `ui.inspect`. +const KEEP: usize = 256; + +/// A photo left sooner than this (s) after it was stepped to was skipped past: the user is +/// holding an arrow key or tapping through. +pub const FAST_STEP_SECS: f64 = 0.25; +/// While stepping fast, how long a photo has to stay before its render is started (s). Until then +/// the loupe shows its camera JPEG or thumbnail, and the workers make the next ones' instead of +/// starting a full raw develop that the next step throws away. +pub const SETTLE_SECS: f64 = 0.12; + +/// One photo's way onto the screen (times in ms since the frame it was stepped to). +#[derive(Clone, Copy, Debug, PartialEq)] +pub struct StepTiming { + pub photo: PhotoId, + /// First frame that drew anything of the photo (often the grid thumbnail, magnified). + pub any_ms: Option, + /// What that was. + pub any_source: &'static str, + /// First frame with a full-size stand-in or better: the camera's JPEG, a cached render, the render. + pub full_ms: Option, + pub full_source: &'static str, + /// First frame with LightCraft's own render of the photo (`render` or `cached`). + pub sharp_ms: Option, + /// How long the photo stayed in the loupe (ms): a step taken before `sharp_ms` was skipped past. + pub dwell_ms: f64, +} + +impl StepTiming { + fn to_json(self) -> Value { + json!({ + "photo": self.photo.0, + "anyMs": self.any_ms, + "anySource": self.any_source, + "fullMs": self.full_ms, + "fullSource": self.full_source, + "sharpMs": self.sharp_ms, + "dwellMs": self.dwell_ms, + }) + } +} + +/// The loupe's step timings. +#[derive(Debug, Default)] +pub struct LoupeTiming { + current: Option<(f64, StepTiming)>, + done: VecDeque, + /// Steps finished since the app started (never lowered when old ones leave `done`): a driver's + /// cursor for the steps made after it looked. + total: u64, + /// Whether the loupe drew a photo in the frame being drawn ([`LoupeTiming::end_frame`]). + drawn: bool, + /// How long the photo before the current one stayed (ms), when the loupe stepped to it from + /// that photo; `None` when the loupe opened on it (from the grid, or coming back to the loupe). + stepped_from: Option, +} + +/// Sources that show the photo at full size: the camera's JPEG, a cached or a fresh render. +fn full_size(source: &str) -> bool { + matches!(source, "embedded" | "cached" | "render" | "preview") +} + +impl LoupeTiming { + /// The loupe drew `photo` from `source` (what [`crate::panels::detail`] reports in + /// `loupe_shown`) in the frame that started at `now` (s). + pub fn observe(&mut self, photo: PhotoId, source: &'static str, now: f64) { + self.drawn = true; + if self.current.as_ref().is_none_or(|(_, t)| t.photo != photo) { + let stepped = self.current.is_some(); + self.finish(now); + self.stepped_from = if stepped { self.done.back().map(|t| t.dwell_ms) } else { None }; + let t = StepTiming { photo, any_ms: None, any_source: "none", full_ms: None, full_source: "none", sharp_ms: None, dwell_ms: 0.0 }; + self.current = Some((now, t)); + } + let Some((since, t)) = self.current.as_mut() else { return }; + let ms = ((now - *since) * 1000.0).max(0.0); + if !ms.is_finite() { + return; + } + t.dwell_ms = ms; + if source != "none" && t.any_ms.is_none() { + t.any_ms = Some(ms); + t.any_source = source; + } + if full_size(source) && t.full_ms.is_none() { + t.full_ms = Some(ms); + t.full_source = source; + } + if matches!(source, "render" | "cached") && t.sharp_ms.is_none() { + t.sharp_ms = Some(ms); + } + } + + /// Whether the user is stepping fast and `photo` (being drawn in the frame at `now`) has not + /// stayed [`SETTLE_SECS`] yet: the photo before it was left within [`FAST_STEP_SECS`]. + pub fn settling(&self, photo: PhotoId, now: f64) -> bool { + let (since, prev_dwell) = match &self.current { + // the first frame of a step: `observe` has not seen this photo yet + Some((since, t)) if t.photo != photo => (now, (now - since) * 1000.0), + // (a photo the loupe opened on, not stepped to, never settles: the step that ended by + // leaving the loupe is not its predecessor) + Some((since, _)) => (*since, self.stepped_from.unwrap_or(f64::INFINITY)), + None => return false, + }; + prev_dwell.is_finite() && prev_dwell < FAST_STEP_SECS * 1000.0 && now - since < SETTLE_SECS + } + + /// The loupe is not showing a photo any more (the grid is up). + pub fn left(&mut self, now: f64) { + self.finish(now); + } + + /// A frame (started at `now`) was drawn: when the loupe drew no photo in it (the grid, Compare, Survey, + /// Reference, People or the slideshow is up), the photo it showed was left then, so time away from the + /// loupe never counts as time on that photo. + pub fn end_frame(&mut self, now: f64) { + if !std::mem::take(&mut self.drawn) { + self.left(now); + } + } + + fn finish(&mut self, now: f64) { + if let Some((since, mut t)) = self.current.take() { + let dwell = (now - since) * 1000.0; + if dwell.is_finite() { + t.dwell_ms = dwell.max(t.dwell_ms); + } + if self.done.len() >= KEEP { + self.done.pop_front(); + } + self.done.push_back(t); + self.total = self.total.saturating_add(1); + } + } + + /// The finished steps, oldest first (the latest [`KEEP`]), how many have finished in all, and the photo on screen now. + pub fn to_json(&self) -> Value { + json!({ + "steps": self.done.iter().map(|t| t.to_json()).collect::>(), + "total": self.total, + "current": self.current.as_ref().map(|(_, t)| t.to_json()), + }) + } + + #[cfg(test)] + fn steps(&self) -> Vec { + self.done.iter().copied().collect() + } +} + +#[cfg(test)] +mod tests { + use super::*; + + #[test] + fn a_step_records_when_each_kind_of_picture_first_showed() { + let mut l = LoupeTiming::default(); + l.observe(PhotoId(1), "none", 10.0); + l.observe(PhotoId(1), "thumb", 10.016); + l.observe(PhotoId(1), "embedded", 10.040); + l.observe(PhotoId(1), "embedded", 10.056); + l.observe(PhotoId(1), "render", 11.0); + l.observe(PhotoId(2), "thumb", 12.0); + let s = l.steps(); + assert_eq!(s.len(), 1); + let t = s[0]; + assert_eq!(t.photo, PhotoId(1)); + assert_eq!((t.any_source, t.full_source), ("thumb", "embedded")); + assert!((t.any_ms.unwrap() - 16.0).abs() < 0.01); + assert!((t.full_ms.unwrap() - 40.0).abs() < 0.01); + assert!((t.sharp_ms.unwrap() - 1000.0).abs() < 0.01); + assert!((t.dwell_ms - 2000.0).abs() < 0.01); + assert_eq!(l.to_json()["current"]["photo"], 2); + } + + #[test] + fn stepping_fast_holds_the_render_back_until_the_photo_stays() { + let mut l = LoupeTiming::default(); + // the first photo opened: no step before it + assert!(!l.settling(PhotoId(1), 0.0)); + l.observe(PhotoId(1), "embedded", 0.0); + // a deliberate step (the photo before stayed 2 s): render at once + assert!(!l.settling(PhotoId(2), 2.0)); + l.observe(PhotoId(2), "embedded", 2.0); + // held arrow: the photo before stayed 33 ms + assert!(l.settling(PhotoId(3), 2.033)); + l.observe(PhotoId(3), "thumb", 2.033); + assert!(l.settling(PhotoId(3), 2.1)); + // …until this one has stayed long enough + assert!(!l.settling(PhotoId(3), 2.033 + SETTLE_SECS + 0.001)); + } + + /// Leaving the loupe soon after a step and coming back on another photo is not a fast step + /// (CodeRabbit on #3077): the photo the loupe opens on renders at once, on every frame. + #[test] + fn coming_back_to_the_loupe_on_another_photo_is_not_a_fast_step() { + let mut l = LoupeTiming::default(); + l.observe(PhotoId(1), "embedded", 0.0); + l.end_frame(0.0); + // the grid is up after 100 ms on the photo + l.end_frame(0.1); + assert!(!l.settling(PhotoId(2), 0.2), "the first frame back"); + l.observe(PhotoId(2), "thumb", 0.2); + l.end_frame(0.2); + assert!(!l.settling(PhotoId(2), 0.25), "nor the frames after it"); + // a fast step from there still settles + l.observe(PhotoId(2), "embedded", 0.25); + assert!(l.settling(PhotoId(3), 0.3)); + l.observe(PhotoId(3), "thumb", 0.3); + assert!(l.settling(PhotoId(3), 0.35)); + } + + /// A frame without the loupe finishes the photo it showed: coming back to it later starts a new step, and + /// neither counts the time away (CodeRabbit on #3055). + #[test] + fn leaving_the_loupe_finishes_its_photo() { + let mut l = LoupeTiming::default(); + l.observe(PhotoId(1), "embedded", 10.0); + l.end_frame(10.0); + l.observe(PhotoId(1), "embedded", 10.5); + l.end_frame(10.5); + assert!(l.steps().is_empty(), "frames that draw the loupe keep the step open"); + // the grid is up from 11 s to 60 s + l.end_frame(11.0); + l.end_frame(60.0); + let s = l.steps(); + assert_eq!(s.len(), 1); + assert!((s[0].dwell_ms - 1000.0).abs() < 0.01, "dwell ends when the loupe went: {}", s[0].dwell_ms); + // back in the loupe on the same photo: a new step, timed from its return + l.observe(PhotoId(1), "render", 60.0); + l.observe(PhotoId(1), "render", 60.2); + l.end_frame(60.2); + assert_eq!(l.steps().len(), 1); + assert_eq!(l.to_json()["current"]["sharpMs"], 0.0); + assert!((l.to_json()["current"]["dwellMs"].as_f64().unwrap() - 200.0).abs() < 0.01); + } + + #[test] + fn a_photo_stepped_past_has_no_sharp_time_and_the_list_is_bounded() { + let mut l = LoupeTiming::default(); + for i in 0..(KEEP as u64 + 10) { + l.observe(PhotoId(i), "thumb", i as f64 * 0.033); + } + l.left(1000.0); + let s = l.steps(); + assert_eq!(s.len(), KEEP); + // the count of steps goes on past the kept list: a driver's cursor (CodeRabbit on #3077) + assert_eq!(l.to_json()["total"], KEEP as u64 + 10); + assert!(s.iter().all(|t| t.sharp_ms.is_none() && t.full_ms.is_none())); + // a non-finite clock never poisons the record + l.observe(PhotoId(9999), "render", f64::NAN); + l.observe(PhotoId(9999), "render", f64::INFINITY); + assert!(l.to_json().is_object()); + } +} diff --git a/crates/ui-egui/src/menus/menubar.rs b/crates/ui-egui/src/menus/menubar.rs index 672c7cbccc..b73729a913 100644 --- a/crates/ui-egui/src/menus/menubar.rs +++ b/crates/ui-egui/src/menus/menubar.rs @@ -465,17 +465,31 @@ fn live_label(app: &LightcraftApp, id: &str, label: &str) -> String { } /// The parameterized submenus. +/// The params of Build Standard-Sized Previews: automatic builds as big as the loupe on this screen shows photos +/// (the engine's standard size, once a loupe has been drawn); before that, in a fresh session, the standard +/// 2560 px, not the engine's 2048 px minimum, so a typical fit view needn't render again; a chosen limit is used as is. +pub(crate) fn standard_build_params(settings: &crate::state::AppSettings, loupe_edge: usize) -> Value { + match settings.preview_limit { + 0 if loupe_edge == 0 => json!({"size": "standard", "edge": settings.standard_preview_edge()}), + 0 => json!({"size": "standard"}), + edge => json!({"size": "standard", "edge": edge}), + } +} + pub(crate) fn expanded(app: &LightcraftApp, name: &str) -> Option> { let active = app.session.active().and_then(|id| app.session.catalog.photo(id).cloned()); let has = active.is_some(); Some(match name { "Previews" => { - let running = app.session.preview_build.as_ref().is_some_and(|b| !b.finished.load(std::sync::atomic::Ordering::Relaxed)); + // (the background preparation's build gives way to one asked for here) + let running = app.session.preview_build.as_ref().is_some_and(|b| b.running() && !b.background); + let stoppable = app.session.preview_build.as_ref().is_some_and(|b| b.running()); + let standard = standard_build_params(&app.ui.settings, app.session.loupe_edge); let scope = if app.session.selection.ids.len() > 1 { "Selected" } else { "Visible" }; vec![ item( "library.buildPreviews", - json!({"size": "standard", "edge": app.ui.settings.standard_preview_edge()}), + standard, crate::i18n::tr_format!("Build Standard-Sized Previews ({scope})", scope = crate::i18n::tr(scope)), None, !running, @@ -489,7 +503,7 @@ pub(crate) fn expanded(app: &LightcraftApp, name: &str) -> Option> !running, None, ), - item("library.cancelPreviews", Value::Null, "Stop Building Previews", None, running, None), + item("library.cancelPreviews", Value::Null, "Stop Building Previews", None, stoppable, None), MenuNode::Separator, // (read and written on a worker thread: the originals may be on a slow drive) item( diff --git a/crates/ui-egui/src/panels/detail.rs b/crates/ui-egui/src/panels/detail.rs index 3e321304d2..534949ab00 100644 --- a/crates/ui-egui/src/panels/detail.rs +++ b/crates/ui-egui/src/panels/detail.rs @@ -18,6 +18,7 @@ mod crop; mod filmstrip; mod geometry; mod info; +mod look_ahead; mod masks; mod retouch; mod window; @@ -41,11 +42,15 @@ use geometry::{animated_rect, middle_pan}; pub(crate) use geometry::{fit_rect, fit_texture_rect, focus_pan, navigate_gesture, output_px, pan_image, snap_fit_rect, texture_side}; #[cfg(test)] use info::RENDERING_AFTER_SECS; +use info::camera_preview_badge; use info::{ FILTER_PILL_HEIGHT, FILTER_PILL_INSET, brief_info, filter_pill, filter_pill_shown, info_overlay, info_overlay_extent, label_badge_rect, preview_only_pill, quick_name, }; pub(crate) use info::{draw_focus_point, failed_pill, loupe_pill, rendering_overdue}; +use look_ahead::BEHIND; +pub use look_ahead::LookAhead; +pub(crate) use look_ahead::{Prepare, look_ahead}; use masks::{RegionContext, RegionEdit, color_range_pick, mask_overlay, region_overlay}; use retouch::{eye_overlay, remove_overlay}; pub(crate) use window::overlay_supports_window; @@ -229,11 +234,12 @@ pub fn show(app: &mut LightcraftApp, ui: &mut egui::Ui) { // one axis only. A box that matches the source to rounding is rendered at exactly its size. let short = |e: usize, a: f32| ((e as f32 / a).round() as usize).max(1); let (rw, rh) = if aspect >= 1.0 { (want, short(want, aspect)) } else { (short(want, 1.0 / aspect), want) }; + note_loupe_edge(app, sizes.canvas_long, !interacting && !split && !split_view); // hover and before renders are stand-ins: the preview size is plenty and keeps them cheap let se = app.ui.settings.stand_in_edge(want, texture_side); let (sw, sh) = if aspect >= 1.0 { (se, short(se, aspect)) } else { (short(se, 1.0 / aspect), se) }; let ask = LoupeAsk { id, with_crop: !crop_tool, interacting, fit_px, overlay }; - request_main(app, ui, &ask, [rw, rh], plan.window_edge.is_none()); + let settling = request_main_or_settle(app, ui, &ask, [rw, rh], plan.window_edge.is_none(), now); // hovering a preset or profile: the photo with that look, shown instead of the loupe render // once it is ready (nothing is committed) let hover_key = request_hover(app, &ask, [sw, sh]); @@ -242,8 +248,14 @@ pub fn show(app: &mut LightcraftApp, ui: &mut egui::Ui) { } // until the loupe has this photo: show its cached render / embedded preview / a thumbnail request_stand_in(app, id, want, !crop_tool); + if !interacting { + // (what Settings ▸ Performance lets this mode prepare ahead) + let prepare = prepare_ahead(app); + look_ahead(app, id, want.max(8), !crop_tool, prepare); + } let show_before = app.ui.before_after == BeforeAfter::Original || ui.input(|i| i.key_down(egui::Key::Backslash)); - if split || show_before || split_view { + // (the Before render waits for a settled photo like the loupe's own) + if !settling && (split || show_before || split_view) { request_before(app, &ask, [sw, sh]); } // zoomed past what the whole-frame render holds: also render the part on screen, at no more @@ -263,13 +275,16 @@ pub fn show(app: &mut LightcraftApp, ui: &mut egui::Ui) { look, holding: navigating || app.size_hold.holding(now), }; - let window_views = if window_ctx.frame_edge.is_some() { + // (stepping fast: no window is asked for until the photo stays, as for the frame above) + let window_views = if window_ctx.frame_edge.is_some() && !settling { let mode = ViewMode { split, show_before, split_view }; window_views(mode, &Views { img_rect, target_rect, before_img, before_target, areas: &areas, main_area, canvas }) } else { Vec::new() }; let windows = request_windows(app, &window_ctx, window_views); + let single = hover_key.is_none() && !split && !show_before && !split_view && !interacting; + let camera_full = request_camera_full(app, &photo, single && !windows.is_empty(), native_long.min(texture_side)); let p = ui.painter_at(canvas); let draw = |slot: Slot, r: Rect, clip: Rect| draw_slot(app, &p, id, ppp, slot, r, clip); // soft proofing: a paper-white surround and the proof's name, as Lightroom shows it @@ -306,16 +321,21 @@ pub fn show(app: &mut LightcraftApp, ui: &mut egui::Ui) { none_message(app, &p, &t, canvas, id); } } - app.loupe_shown = Some((id, shown)); + // a wipe shows the Before whole-frame render on its side of the line… let line = crate::region::wipe_line(img_rect, canvas); draw_wipe_before(app, &p, id, img_rect, line); + let camera_full_drawn = camera_full && draw_camera_full(app, &p, id, img_rect, display_aspect, window_cover(&windows)); // …and the window renders go over the whole-frame ones, all of them, where they belong in the // frame (clipped to their pane, or their side of the line) - for (v, view) in &windows { - // (holding `\` in a wipe shows the Before everywhere: it is the Before view alone then) - let mode = if show_before { BeforeAfter::Original } else { app.ui.before_after }; - let clip = crate::region::window_clip(mode, v.before, v.visible, canvas, img_rect); - draw_window(&p.with_clip_rect(clip), app, &window_ctx, v, view); + let drawn = draw_windows(app, &p, &window_ctx, &windows, [canvas, img_rect], (show_before, split)); + let shown = shown_after_windows(shown, camera_full_drawn, drawn); + app.loupe_shown = Some((id, shown)); + app.loupe_timing.observe(id, shown, now); + // the camera's own JPEG is on screen, not LightCraft's rendering of the raw: say so (once), so + // colour and tone are not judged on it (PERF-CULL-4; Lightroom Classic badges its embedded + // previews). Not over a window render, which is LightCraft's own. + if camera_jpeg_shown(app, &photo, shown, drawn) { + camera_preview_badge(ui, canvas, img_rect.intersect(canvas)); } if let Some((bg, g)) = before_badge { paint_badge(&p, bg, g); @@ -462,21 +482,69 @@ fn prefetch_neighbours(app: &mut LightcraftApp, id: PhotoId, main_area: Rect, pp return; } let ids = app.session.visible_cloned(); - if let Some(i) = ids.iter().position(|p| *p == id) { - let next = ids.get(i + 1).copied(); - let prev = i.checked_sub(1).and_then(|j| ids.get(j)).copied(); - for (n, nid) in [next, prev].into_iter().enumerate() { - let Some(nid) = nid else { continue }; - let Some(np) = app.session.catalog.photo(nid).cloned() else { continue }; - let nf = Frame::with_lens(np.width.max(1) as usize, np.height.max(1) as usize, &np.develop, with_crop, np.embedded_lens.as_ref()); - let na = nf.aspect() as f32; - let nr = fit_rect(main_area, na, app.ui.zoom, output_px(&nf), ppp, app.ui.pan); - let nw = app.ui.settings.prefetch_edge(nr.width().max(nr.height()) * ppp, output_px(&nf).into_iter().max().unwrap_or(1), texture_side); - let (w, h) = if na >= 1.0 { (nw, (nw as f32 / na) as usize) } else { ((nw as f32 * na) as usize, nw) }; - if let Some(job) = app.session.loupe_job(nid, w.max(8), h.max(8), with_crop) { - app.renderer.prefetch(Slot::Prefetch(n as u8), job, PREFETCH_PRIORITY); - } + let mode = prep_mode(app); + let prep = app.session.prep_settings(); + let ahead = prep.ahead(mode); + // editing: the next photos are decoded and kept, ready for their edits (their view render is not + // what is wanted, and a cached one would leave the photo undecoded) + let develop = mode == lightcraft_engine::prep::Mode::Develop; + let Some(i) = ids.iter().position(|p| *p == id).filter(|_| ahead > 0) else { return }; + let mut targets = app.look_ahead.targets_n(&ids, i, ahead, BEHIND); + if prep.scope(mode) == lightcraft_engine::prep::Scope::Marked { + targets.retain(|t| lightcraft_engine::prep::marked(&app.session, *t)); + } + // a few at a time, nearest first (the pool keeps workers free for the photo the user goes to) + let mut room = PREFETCH_AT_ONCE.saturating_sub(app.renderer.prefetches_pending()); + let todo: Vec<(usize, PhotoId)> = targets.into_iter().filter_map(|nid| ids.iter().position(|p| *p == nid).map(|at| (at, nid))).collect(); + for (at, nid) in todo { + if room == 0 { + break; } + let n = prefetch_slot(at); + let Some(mut job) = neighbour_job(app, nid, main_area, ppp, texture_side, with_crop) else { continue }; + if develop { + job.view_cache = None; + } + // (asked for already, in this slot: queued, running or made) + if app.renderer.prefetch_asked(n, job.key) { + continue; + } + room -= 1; + app.renderer.prefetch(n, job, PREFETCH_PRIORITY); + } +} + +/// The render a neighbour is prepared with: its loupe render at the size it will be drawn at. +fn neighbour_job( + app: &mut LightcraftApp, + nid: PhotoId, + main_area: Rect, + ppp: f32, + texture_side: usize, + with_crop: bool, +) -> Option { + let np = app.session.catalog.photo(nid)?; + let nf = Frame::with_lens(np.width.max(1) as usize, np.height.max(1) as usize, &np.develop, with_crop, np.embedded_lens.as_ref()); + let na = nf.aspect() as f32; + let nr = fit_rect(main_area, na, app.ui.zoom, output_px(&nf), ppp, app.ui.pan); + let nw = app.ui.settings.prefetch_edge(nr.width().max(nr.height()) * ppp, output_px(&nf).into_iter().max().unwrap_or(1), texture_side); + let (w, h) = if na >= 1.0 { (nw, (nw as f32 / na) as usize) } else { ((nw as f32 * na) as usize, nw) }; + app.session.loupe_job(nid, w.max(8), h.max(8), with_crop) +} + +/// What Settings ▸ Performance lets the mode the user is in prepare ahead. +fn prepare_ahead(app: &LightcraftApp) -> Prepare { + let mode = prep_mode(app); + let prep = app.session.prep_settings(); + Prepare::of(prep.ahead(mode), prep.scope(mode)) +} + +/// The most the loupe shows a whole photo at (the canvas's long side; a photo of another shape than +/// this one may fill it), noted while it shows one view fitted (`single`): standard previews are +/// built at least this big. +fn note_loupe_edge(app: &mut LightcraftApp, canvas_long: f32, single: bool) { + if single && matches!(app.ui.zoom, Zoom::Fit) && canvas_long.is_finite() && canvas_long >= 1.0 { + app.session.loupe_edge = canvas_long.ceil() as usize; } } @@ -733,10 +801,138 @@ struct LoupeAsk { overlay: lightcraft_pipeline::Overlay, } +/// Stepping fast (an arrow held down): the render waits until the photo stays, the camera JPEG or +/// thumbnail shows meanwhile, and a render queued for a photo already left is dropped. Otherwise +/// the loupe's own render is requested. Whether the photo is still settling. +fn request_main_or_settle(app: &mut LightcraftApp, ui: &egui::Ui, a: &LoupeAsk, size: [usize; 2], whole_draft: bool, now: f64) -> bool { + let settling = !a.interacting && app.loupe_timing.settling(a.id, now); + if !settling { + request_main(app, ui, a, size, whole_draft); + return false; + } + drop_unsettled(&mut app.renderer, a.id); + ui.ctx().request_repaint_after(std::time::Duration::from_secs_f64(crate::loupe_timing::SETTLE_SECS / 2.0)); + true +} + +/// While the photo settles: drop the raw renders still queued for a photo the user is stepping +/// past (one already running finishes). Its whole-frame render unless that is on screen already, +/// the zoomed-in windows (a region render decodes the raw as much as the frame) and the Before +/// view, which is a raw render too. +pub(crate) fn drop_unsettled(renderer: &mut crate::render::Renderer, photo: PhotoId) { + if renderer.textures.get(&Slot::Main).is_none_or(|t| t.photo != photo) { + renderer.drop_queued(Slot::Main); + } + for slot in [Slot::Region, Slot::RegionBefore, Slot::Before] { + renderer.drop_queued(slot); + } +} + +/// Zoomed in on an unedited raw (`zoomed`: one view, window renders up): its camera JPEG at full +/// size under the window render, so a focus check doesn't wait for the raw (PERF-CULL-5; Capture +/// One's Cull view shows the embedded JPEG at 100 % too). Released when the view is not zoomed any +/// more (180 MB). Whether it is wanted. +fn request_camera_full(app: &mut LightcraftApp, photo: &lightcraft_catalog::Photo, zoomed: bool, edge: usize) -> bool { + let wanted = zoomed && app.session.embedded_thumb_is_final(photo); + if wanted { + if let Some(q) = app.session.embedded_full_job(photo.id, edge) { + app.renderer.request_quick(Slot::QuickFull, q, 98); + } + } else if app.renderer.textures.contains_key(&Slot::QuickFull) || app.renderer.is_pending(Slot::QuickFull) { + app.renderer.forget(Slot::QuickFull); + } + wanted +} + +/// The camera JPEG at full size, magnified like the whole-frame render, until the window is +/// there: only where the window will go (`cover`, see [`window_cover`]). A window capped smaller +/// than the view leaves strips around it, and those keep the developed whole-frame render rather +/// than an unlabelled camera JPEG (CodeRabbit on #3544). Whether it was drawn. +fn draw_camera_full(app: &LightcraftApp, p: &egui::Painter, id: PhotoId, img_rect: Rect, display_aspect: f32, cover: Option) -> bool { + let Some(cover) = cover else { return false }; + let Some(tex) = app.renderer.textures.get(&Slot::QuickFull).filter(|t| t.photo == id) else { return false }; + if !crate::region::same_aspect(tex.size[0].max(1) as f32 / tex.size[1].max(1) as f32, display_aspect) { + return false; + } + p.with_clip_rect(p.clip_rect().intersect(cover)).image(tex.id(), img_rect, Rect::from_min_max(pos2(0.0, 0.0), pos2(1.0, 1.0)), Color32::WHITE); + true +} + +/// Where on screen the requested window renders go (each its window's share of its view's drawn +/// rect), joined: `None` without any. +fn window_cover(windows: &[(WindowView, crate::region::RegionView)]) -> Option { + windows + .iter() + .map(|(v, w)| { + let (fw, fh) = (w.full.0.max(1) as f32, w.full.1.max(1) as f32); + let at = v.img.min + vec2(w.window.x as f32 / fw * v.img.width(), w.window.y as f32 / fh * v.img.height()); + Rect::from_min_size(at, vec2(w.window.w as f32 / fw * v.img.width(), w.window.h as f32 / fh * v.img.height())) + }) + .reduce(|a, b| a.union(b)) +} + +/// Which window renders [`draw_windows`] drew. +#[derive(Clone, Copy)] +struct WindowsDrawn { + /// Any of them. + any: bool, + /// The window of the view shown whole: the After side's in a split view, the Before's when + /// the Before is shown alone; what the timing and the source shown are about. Holding `\` + /// side by side or top/bottom keeps both panes, and the After pane is still the one shown. + shown: bool, +} + +/// Draw the window renders over the whole-frame ones, each where it belongs in the frame +/// (clipped to its pane, or its side of the line). +fn draw_windows( + app: &LightcraftApp, + p: &egui::Painter, + ctx: &WindowCtx, + windows: &[(WindowView, crate::region::RegionView)], + [canvas, img_rect]: [Rect; 2], + (show_before, split): (bool, bool), +) -> WindowsDrawn { + let mut out = WindowsDrawn { any: false, shown: false }; + // (holding `\` in a wipe shows the Before everywhere: it is the Before view alone then; side by + // side or top/bottom, both panes stay, and each window stays in its own) + let before_alone = show_before && !split; + for (v, view) in windows { + let mode = if before_alone { BeforeAfter::Original } else { app.ui.before_after }; + let clip = crate::region::window_clip(mode, v.before, v.visible, canvas, img_rect); + let drawn = draw_window(&p.with_clip_rect(clip), app, ctx, v, view); + out.any |= drawn; + out.shown |= drawn && (!v.before || before_alone); + } + out +} + +/// What is on screen once the window renders are drawn: the camera JPEG at full size stands in +/// for the whole-frame render until a window covers it, and a window rendered over a stand-in +/// (thumbnail or camera JPEG) before the whole frame is the raw's own render. +fn shown_after_windows(shown: &'static str, camera_full_drawn: bool, drawn: WindowsDrawn) -> &'static str { + if camera_full_drawn && !drawn.any { + "embedded" + } else if drawn.shown && !matches!(shown, "render" | "cached") { + "render" + } else { + shown + } +} + +/// Whether the camera's own JPEG is what is on screen (not under a window render). +fn camera_jpeg_shown(app: &LightcraftApp, photo: &lightcraft_catalog::Photo, shown: &str, drawn: WindowsDrawn) -> bool { + !drawn.any && (shown == "embedded" || (shown == "thumb" && app.session.embedded_thumb_is_final(photo))) +} + /// Request the loupe's own render at `[rw, rh]` (drafts during drags). fn request_main(app: &mut LightcraftApp, ui: &egui::Ui, a: &LoupeAsk, [rw, rh]: [usize; 2], whole_draft: bool) { let (id, interacting, fit_px, overlay) = (a.id, a.interacting, a.fit_px, a.overlay); - if let Some(job) = app.session.loupe_job(id, rw.max(8), rh.max(8), a.with_crop) { + if let Some(mut job) = app.session.loupe_job(id, rw.max(8), rh.max(8), a.with_crop) { + // editing: the photo is rendered from its source, which stays decoded for the first slider + // draft (a cached view render would show at once but leave the source unread until then) + if prep_mode(app) == lightcraft_engine::prep::Mode::Develop { + job.view_cache = None; + } let job = if interacting { job.draft() } else { job }; // at rest the render is shrunk to exactly the size it is drawn at with a Lanczos filter on // the worker, so the window draws it texel for texel instead of shrinking it with 2-tap @@ -745,6 +941,11 @@ fn request_main(app: &mut LightcraftApp, ui: &egui::Ui, a: &LoupeAsk, [rw, rh]: let job = job.with_overlay(overlay).with_proof(app.ui.soft_proof.then_some(app.ui.proof)); let job = job.with_hidden_grading(app.ui.grading_hidden_for(id)); app.renderer.request(Slot::Main, job, 100); + // the photo on screen comes first: background preparation waits while it is being made + // (only then: this runs every loupe frame, and a render already shown asks for nothing) + if app.renderer.is_pending(Slot::Main) { + signal_foreground(); + } // a drag's draft that renders in a few ms (GPU) is drawn in this frame, not the next (zoomed // in, what changes on screen is the window's draft) #[cfg(not(target_arch = "wasm32"))] @@ -757,6 +958,19 @@ fn request_main(app: &mut LightcraftApp, ui: &egui::Ui, a: &LoupeAsk, [rw, rh]: } } +/// Tell background preparation the loupe is waiting for its render ([`lightcraft_engine::prep::foreground`]). +fn signal_foreground() { + #[cfg(test)] + FOREGROUND_SIGNALS.with(|n| n.set(n.get() + 1)); + lightcraft_engine::prep::foreground(); +} + +#[cfg(test)] +thread_local! { + /// Tests: how often this thread's loupe told background preparation to wait. + pub(crate) static FOREGROUND_SIGNALS: std::cell::Cell = const { std::cell::Cell::new(0) }; +} + /// Hovering a preset or profile: the photo with that look, shown instead of the loupe render once /// it is ready (nothing is committed). The key of its render and the label to show. fn request_hover(app: &mut LightcraftApp, a: &LoupeAsk, [sw, sh]: [usize; 2]) -> Option<(u64, String)> { @@ -887,6 +1101,26 @@ fn request_stand_in(app: &mut LightcraftApp, id: PhotoId, edge: usize, with_crop /// Neighbour prefetch: below on-screen thumbnails, above background thumbnail refreshes. const PREFETCH_PRIORITY: u32 = 4; +/// Photos prepared ahead at once (each render spreads over the cores already): the others wait, +/// so the photo the user steps to next finds a worker free. +const PREFETCH_AT_ONCE: usize = 2; +/// Prepared-ahead slots, by the photo's place in the view: a step keeps the photos still ahead in +/// the slots they have (nothing is asked for twice), and the jump to another part of the shoot +/// replaces what was waiting. +const PREFETCH_SLOTS: usize = 32; + +fn prefetch_slot(at: usize) -> Slot { + Slot::Prefetch((at % PREFETCH_SLOTS) as u8) +} + +/// What the user is doing, for the background preparation: editing (a tool panel open), else +/// culling a folder or browsing the library. +pub fn prep_mode(app: &LightcraftApp) -> lightcraft_engine::prep::Mode { + match app.ui.right { + RightPanel::Edit | RightPanel::Crop | RightPanel::Remove | RightPanel::Masking | RightPanel::RedEye => lightcraft_engine::prep::Mode::Develop, + _ => lightcraft_engine::prep::Mode::of(&app.session), + } +} /// Targeted adjustment tool: dragging up/down on the photo raises/lowers the tone-curve region or /// the colour-mixer bands under the press point (`develop.targeted`, one call per whole step, all @@ -1075,6 +1309,29 @@ pub(crate) fn click_zoom_and_pan(app: &mut LightcraftApp, resp: &egui::Response, #[cfg(test)] mod tests { + /// The camera JPEG goes only where the window render will: a window of the left half of the + /// frame covers the left half of the drawn photo, not the strips beside it. + #[test] + fn the_camera_jpeg_covers_only_where_the_window_goes() { + use super::{WindowView, window_cover}; + use crate::region::RegionView; + use egui::{Rect, pos2}; + use lightcraft_pipeline::PixelWindow; + assert_eq!(window_cover(&[]), None, "no window: no camera JPEG"); + let img = Rect::from_min_max(pos2(100.0, 50.0), pos2(700.0, 450.0)); + let v = WindowView { slot: crate::render::Slot::Region, before: false, img, target: img, visible: img }; + let w = RegionView { + photo: lightcraft_catalog::PhotoId(1), + before: false, + key: 1, + full: (6000, 4000), + window: PixelWindow { x: 0, y: 1000, w: 3000, h: 2000 }, + settings: 0, + }; + let r = window_cover(&[(v, w)]).unwrap(); + assert_eq!(r, Rect::from_min_max(pos2(100.0, 150.0), pos2(400.0, 350.0)), "the window's share of the drawn rect"); + } + /// The rotation pointer's arc bulges away from the side of the crop box it is on, and follows /// a box that is itself rotated. #[test] diff --git a/crates/ui-egui/src/panels/detail/info.rs b/crates/ui-egui/src/panels/detail/info.rs index da5ae1bed6..99a0fc1108 100644 --- a/crates/ui-egui/src/panels/detail/info.rs +++ b/crates/ui-egui/src/panels/detail/info.rs @@ -201,6 +201,24 @@ pub(super) fn filter_pill(app: &mut LightcraftApp, ui: &mut egui::Ui, canvas: Re } } +/// A small "Camera preview" label at the top right of the photo while the loupe shows the camera's +/// embedded JPEG (widget `loupe:cameraPreview`), with what that means on hover. +pub(super) fn camera_preview_badge(ui: &egui::Ui, canvas: Rect, over: Rect) { + let t = Tokens::get(ui.ctx()); + let p = ui.painter_at(canvas); + let text = crate::i18n::tr("Camera preview"); + let g = p.layout_no_wrap(text.to_string(), t.font(11.5), Color32::from_gray(225)); + let size = vec2(g.size().x + 16.0, 18.0); + let over = if over.is_positive() { over } else { canvas }; + let r = Rect::from_min_size(pos2(over.right() - size.x - 8.0, over.top() + 8.0), size); + p.rect_filled(r, 9.0, Color32::from_black_alpha(165)); + p.galley(pos2(r.left() + 8.0, r.center().y - g.size().y / 2.0), g, Color32::from_gray(225)); + register(ui, "loupe:cameraPreview", r); + let tip = crate::i18n::tr("The camera's own JPEG, shown until LightCraft has rendered the raw. Judge colour and tone on the rendered photo."); + let resp = ui.interact(r, egui::Id::unique("camera-preview-badge"), Sense::hover()).on_hover_text(tip); + resp.widget_info(|| egui::WidgetInfo::labeled(egui::Role::Label, true, format!("{text}: {tip}"))); +} + /// A raw shown from its embedded JPEG (issue #10): a pill at the canvas' top centre saying so; /// hovering it explains (registered as `notice:previewOnly:loupe`). pub(super) fn preview_only_pill(ui: &mut egui::Ui, canvas: Rect, reason: &str) { diff --git a/crates/ui-egui/src/panels/detail/look_ahead.rs b/crates/ui-egui/src/panels/detail/look_ahead.rs new file mode 100644 index 0000000000..839bf9fe92 --- /dev/null +++ b/crates/ui-egui/src/panels/detail/look_ahead.rs @@ -0,0 +1,119 @@ +//! The loupe's look-ahead: stand-ins and thumbnails for the photos it is likely to be stepped to +//! next (PERF-CULL-2). + +use super::{LightcraftApp, PhotoId}; + +/// How many photos ahead (in the direction of travel) the loupe makes its stand-in for, and how +/// many behind (a step back is common when culling). +const AHEAD: usize = 5; +pub(super) const BEHIND: usize = 1; +/// The look-ahead's priority: below the loupe's own render (100), above everything else. +const AHEAD_PRIORITY: u32 = 96; +/// Thumbnails the loupe keeps while stepping (as many as the grid keeps), and how many photos on +/// either side of the open one stay when it trims them. +const MAX_LOUPE_THUMBS: usize = 600; +const KEEP_AROUND: usize = 64; +/// The thumbnail size the look-ahead asks for (the filmstrip's at 2x). +const AHEAD_THUMB_EDGE: usize = 256; + +/// What [`look_ahead`] last worked out (see `LightcraftApp::look_ahead`). +#[derive(Debug, Default)] +pub struct LookAhead { + /// The last photo the loupe showed, for the direction of travel. + last: Option, + /// +1 forward (the default), -1 backward. + dir: i8, + /// (photo, direction, catalog revision, visible list generation, edge, crop, which photos) the + /// requests were made for: a new source, filter or sort changes the neighbours without an edit, + /// and a change of the preparation settings what may be made. + made_for: Option<(PhotoId, i8, u64, u64, usize, bool, Prepare, u64)>, +} + +/// Which photos the look-ahead may prepare, from Settings ▸ Performance for what the user is doing +/// ([`lightcraft_engine::prep::Mode`]): none when its "Photos ahead" is 0 or it prepares only the photo +/// on screen, else the photos of its "which photos". +#[derive(Clone, Copy, Debug, PartialEq, Eq)] +pub(crate) enum Prepare { + Nothing, + Photos(lightcraft_engine::prep::Scope), +} + +impl Prepare { + /// For `ahead` photos ahead (the mode's count, 0 for the photo on screen only) of `scope`. + pub(crate) fn of(ahead: usize, scope: lightcraft_engine::prep::Scope) -> Prepare { + if ahead == 0 || scope == lightcraft_engine::prep::Scope::OnScreen { Prepare::Nothing } else { Prepare::Photos(scope) } + } +} + +impl LookAhead { + /// The photos to prepare around `id` at position `i` of `ids`: [`AHEAD`] in the direction of + /// travel, nearest first, then [`BEHIND`] the other way, of those `prepare` allows. (The + /// stand-ins are cheap: as many as [`AHEAD`] whatever the mode's count, which is for the full + /// renders made ahead.) + fn targets(&self, app: &LightcraftApp, ids: &[PhotoId], i: usize, prepare: Prepare) -> Vec { + let Prepare::Photos(scope) = prepare else { return Vec::new() }; + let mut targets = self.targets_n(ids, i, AHEAD, BEHIND); + if scope == lightcraft_engine::prep::Scope::Marked { + targets.retain(|t| lightcraft_engine::prep::marked(&app.session, *t)); + } + targets + } + + /// `ahead` photos in the direction of travel from position `i` of `ids`, nearest first, then + /// `behind` the other way. + pub fn targets_n(&self, ids: &[PhotoId], i: usize, ahead: usize, behind: usize) -> Vec { + let dir: isize = if self.dir < 0 { -1 } else { 1 }; + let at = |k: isize| i.checked_add_signed(k).and_then(|j| ids.get(j)).copied(); + let cap = |n: usize| n.min(ids.len()) as isize; + let ahead = (1..=cap(ahead)).filter_map(|k| at(dir * k)); + let behind = (1..=cap(behind)).filter_map(|k| at(-dir * k)); + ahead.chain(behind).collect() + } + + /// Whether the user is stepping backwards. + pub fn backward(&self) -> bool { + self.dir < 0 + } +} + +/// Culling: make the loupe's stand-ins (the camera JPEG of an unedited raw, or a cached render) of +/// the next photos in the direction of travel, so that a step shows the next one at once instead +/// of after reading and decoding it (see [`Slot::QuickAhead`]). +/// Only the photos `prepare` allows (Settings ▸ Performance): none at all asks for nothing, and drops the +/// stand-ins still queued. +pub(crate) fn look_ahead(app: &mut LightcraftApp, id: PhotoId, edge: usize, crop: bool, prepare: Prepare) { + let rev = app.session.catalog.revision; + if app.look_ahead.last != Some(id) { + let ids = app.session.visible_cloned(); + let pos = |p: PhotoId| ids.iter().position(|q| *q == p); + if let (Some(prev), Some(i)) = (app.look_ahead.last, pos(id)) + && let Some(j) = pos(prev) + { + app.look_ahead.dir = if j > i { -1 } else { 1 }; + } + app.look_ahead.last = Some(id); + } + let (visible_gen, ids) = app.session.visible_shared(); + // (the renderer's epoch: a reset drops the stand-ins made, which are then asked for again) + let want = (id, app.look_ahead.dir, rev, visible_gen, edge, crop, prepare, app.renderer.epoch()); + if app.look_ahead.made_for == Some(want) { + return; + } + app.look_ahead.made_for = Some(want); + let Some(i) = ids.iter().position(|p| *p == id) else { return }; + let targets = app.look_ahead.targets(app, &ids, i, prepare); + // stepping through a whole wedding in the loupe would keep every thumbnail it ever drew: keep + // the ones around the photo (what the filmstrip can show) once there are many + if app.renderer.thumb_textures() > MAX_LOUPE_THUMBS { + let keep = ids.iter().skip(i.saturating_sub(KEEP_AROUND)).take(2 * KEEP_AROUND + 1).copied().collect(); + app.renderer.evict_thumbs(&keep, MAX_LOUPE_THUMBS); + } + // their thumbnails first (a small embedded JPEG each, a few ms): something to show at once + // even when a step outruns the full-size previews, and in full screen, where no filmstrip + // asks for them + for &nid in &targets { + crate::panels::grid::request_thumb(app, nid, AHEAD_THUMB_EDGE, AHEAD_PRIORITY + 2); + } + let jobs = targets.into_iter().filter_map(|nid| app.session.quick_view_job(nid, edge, crop)).collect(); + app.renderer.look_ahead(jobs, AHEAD_PRIORITY); +} diff --git a/crates/ui-egui/src/panels/detail/window.rs b/crates/ui-egui/src/panels/detail/window.rs index 3351fab289..5dbaa55f4a 100644 --- a/crates/ui-egui/src/panels/detail/window.rs +++ b/crates/ui-egui/src/panels/detail/window.rs @@ -110,13 +110,14 @@ pub(super) fn request_window(app: &mut LightcraftApp, c: &WindowCtx, v: &WindowV /// Draw `v`'s window texture over the whole-frame render, if it is the one asked for (or, in a /// drag, an earlier draft of it). -pub(super) fn draw_window(p: &egui::Painter, app: &LightcraftApp, c: &WindowCtx, v: &WindowView, wanted: &crate::region::RegionView) { - let Some(tex) = app.renderer.textures.get(&v.slot).filter(|t| t.photo == c.id) else { return }; +pub(super) fn draw_window(p: &egui::Painter, app: &LightcraftApp, c: &WindowCtx, v: &WindowView, wanted: &crate::region::RegionView) -> bool { + let Some(tex) = app.renderer.textures.get(&v.slot).filter(|t| t.photo == c.id) else { return false }; let Some(tile) = app.region_tiles.get(&(v.before, tex.key)).filter(|t| t.is_current(c.id, wanted.full, wanted.settings, c.interacting)) else { - return; + return false; }; let (fw, fh) = (tile.full.0 as f32, tile.full.1 as f32); let at = v.img.min + vec2(tile.window.x as f32 / fw * v.img.width(), tile.window.y as f32 / fh * v.img.height()); let dst = Rect::from_min_size(at, vec2(tile.window.w as f32 / fw * v.img.width(), tile.window.h as f32 / fh * v.img.height())); p.image(tex.id(), dst, Rect::from_min_max(pos2(0.0, 0.0), pos2(1.0, 1.0)), Color32::WHITE); + true } diff --git a/crates/ui-egui/src/panels/notices.rs b/crates/ui-egui/src/panels/notices.rs index cc01701932..7a95a423e0 100644 --- a/crates/ui-egui/src/panels/notices.rs +++ b/crates/ui-egui/src/panels/notices.rs @@ -1,6 +1,7 @@ //! Things the user must not miss (issue #103): settings files that were damaged or unreadable when //! the library opened (and host warnings such as a damaged `ui.json`), and quitting while changes -//! are only in memory or while tasks that would be cut short are running (issue #345). +//! are only in memory or while tasks that would be cut short are running (issue #345). Also, once +//! per session, that the drive holding the preview cache is running low ([`disk_space_notice`]). use egui::{Align2, RichText, vec2}; @@ -8,11 +9,35 @@ use crate::theme::Tokens; use crate::widgets::WidgetHit; use crate::{LightcraftApp, QuitPrompt}; -/// Collect the library's settings-file warnings (each is shown once), and the no-GPU notice once the GPU device has -/// been tried. -pub fn logic(app: &mut LightcraftApp) { +/// Collect the library's settings-file warnings (each is shown once), the no-GPU notice once the GPU device has +/// been tried, and the low-disk notice (once per session). +pub fn logic(app: &mut LightcraftApp, ctx: &egui::Context) { let new = app.session.take_library_warnings(); app.notices.extend(new); + // a low-disk notice still queued from the library that was open before is not this one's + if let Some((generation, text)) = app.disk_notice.as_ref() + && *generation != app.session.library_generation() + { + let text = text.clone(); + app.notices.retain(|n| *n != text); + app.disk_notice = None; + } + if app.disk_space_notice { + // the cache grows with every thumbnail and preview, not only while previews are built: the + // engine reads the free space at most every few minutes (a clock check otherwise) + app.session.check_disk_space(false); + // an idle window still takes the read's answer and makes the next check: a pass soon while + // the read runs, and one when the next check is due + if !app.session.disk_space_alert_raised() { + let s = &app.session; + ctx.request_repaint_after(disk_check_wait(s.disk_space_read_running_for(), s.disk_space_next_check_in())); + } + if let Some(alert) = app.session.take_disk_space_alert() { + let text = disk_space_notice(&alert); + app.disk_notice = Some((app.session.library_generation(), text.clone())); + app.notices.push(text); + } + } use lightcraft_engine::gpu; if app.gpu_notice && app.gpu_applied == Some(true) && gpu::ready() { app.gpu_notice = false; @@ -22,6 +47,18 @@ pub fn logic(app: &mut LightcraftApp) { } } +/// How soon the notice logic looks again for the free-space read: soon while a read runs, less often once it has +/// run for a while (a drive that hangs must not keep an idle window drawing four times a second for good), and +/// when the next check is due (`next_in`: sooner after a read that couldn't start) when none runs. +fn disk_check_wait(running_for: Option, next_in: Option) -> std::time::Duration { + use std::time::Duration; + match running_for { + Some(d) if d < Duration::from_secs(5) => Duration::from_millis(250), + Some(_) => Duration::from_secs(5), + None => next_in.unwrap_or(lightcraft_engine::disk_space::CHECK_EVERY), + } +} + /// What to tell the user when GPU rendering is on (`enabled`) but the GPU could not be used for `reason`: only when no /// graphics adapter was found at all ("no GPU adapter among …", e.g. no Vulkan loader on Linux), which makes every /// render much slower without saying so. Nothing when it was turned off (preference, `LIGHTCRAFT_GPU`), a software @@ -40,6 +77,34 @@ pub fn gpu_notice(reason: Option<&str>, enabled: bool) -> Option { Some(text) } +/// Bytes in plain words: "12.3 GB", "850 MB" (binary units, as the cache size choices). +fn size_words(bytes: u64) -> String { + const GB: f64 = (1u64 << 30) as f64; + const MB: f64 = (1u64 << 20) as f64; + let b = bytes as f64; + if b >= 100.0 * GB { + format!("{:.0} GB", b / GB) + } else if b >= GB { + format!("{:.1} GB", b / GB) + } else { + format!("{:.0} MB", b / MB) + } +} + +/// What to tell the user when the drive holding the preview cache runs low: its free space, the +/// cache's size, and where a limit can be set (the notice's button opens Settings ▸ Performance). +pub fn disk_space_notice(alert: &lightcraft_engine::disk_space::LowDiskAlert) -> String { + let free = size_words(alert.space.available); + let total = size_words(alert.space.total); + let cache = size_words(alert.cache_bytes); + crate::i18n::tr_format!( + "The drive with this library's previews is almost full: {free} free of {total}. The preview cache takes {cache} and grows as the library does. To keep it smaller, set a size limit in Settings → Performance.", + free = free, + total = total, + cache = cache + ) +} + /// The quit prompt's text while `count` cancellable tasks run (`names` lists them). fn tasks_message(count: usize, names: &str) -> String { if count == 1 { @@ -113,6 +178,7 @@ enum Choice { QuitAnyway, Cancel, Ok, + Settings, } /// The prompt (quit or settings notice) that owns input this frame, if any. @@ -597,11 +663,27 @@ pub fn show(app: &mut LightcraftApp, ctx: &egui::Context) { app.quit_confirmed = true; app.ui.quit = true; } - Some(Choice::Cancel | Choice::Ok) | None => app.quit_prompt = None, + Some(Choice::Cancel | Choice::Ok | Choice::Settings) | None => app.quit_prompt = None, } return; } if let Some(text) = app.notices.first().cloned() { + let disk = app.disk_notice.as_ref().filter(|(_, t)| *t == text).map(|(generation, _)| *generation); + if let Some(generation) = disk { + // another library opened since `logic` ran this frame: the notice is the previous library's, + // and the next `logic` takes it away + if generation != app.session.library_generation() { + return; + } + let buttons = [("noticeSettings", "Open Performance Settings", Choice::Settings), ("noticeOk", "OK", Choice::Ok)]; + let Some(i) = window(ctx, "settings-notice", "Drive almost full", &text, &buttons, &mut app.notice_input) else { return }; + app.notices.remove(0); + app.disk_notice = None; + if buttons.get(i).is_some_and(|(_, _, choice)| *choice == Choice::Settings) { + app.ui.dialog = Some(crate::state::Dialog::Settings { tab: "performance".into() }); + } + return; + } let buttons = [("noticeOk", "OK", Choice::Ok)]; if window(ctx, "settings-notice", "Settings file problem", &text, &buttons, &mut app.notice_input).is_some() { app.notices.remove(0); @@ -613,6 +695,42 @@ pub fn show(app: &mut LightcraftApp, ctx: &egui::Context) { mod tests { use super::{PromptKey, gpu_notice, prepare_ui_pass, prompt_allows_event, prompt_key}; + /// A library switch between `logic` and drawing leaves the previous library's low-disk notice queued for + /// that frame: it is not drawn, nor answered (CodeRabbit on #3577). + #[test] + fn a_low_disk_notice_of_the_library_before_is_not_drawn() { + let ctx = egui::Context::default(); + let mut app = crate::LightcraftApp::new(lightcraft_engine::Session::with_demo(), crate::Services::default()); + let text = "Drive almost full".to_string(); + app.notices.push(text.clone()); + app.disk_notice = Some((app.session.library_generation().wrapping_add(1), text)); + // the prompt's first button pressed, as Enter would + app.notice_input.activate = Some(0); + let mut out = ctx.run_ui(egui::RawInput::default(), |ui| super::show(&mut app, ui.ctx())); + out.textures_delta.clear(); + assert_eq!(app.notices.len(), 1, "not answered"); + assert!(app.ui.dialog.is_none(), "Settings didn't open"); + // the current library's: drawn and answered + app.disk_notice = app.disk_notice.take().map(|(_, t)| (app.session.library_generation(), t)); + let mut out = ctx.run_ui(egui::RawInput::default(), |ui| super::show(&mut app, ui.ctx())); + out.textures_delta.clear(); + assert!(app.notices.is_empty(), "answered"); + } + + /// The window wakes four times a second while a free-space read runs, then every few seconds once it hangs + /// (CodeRabbit on #3577: it asked for 250 ms passes for as long as the drive hung). + #[test] + fn a_hanging_free_space_read_is_looked_at_less_often() { + use lightcraft_engine::disk_space::{CHECK_EVERY, RETRY_AFTER}; + use std::time::Duration; + assert_eq!(super::disk_check_wait(Some(Duration::ZERO), None), Duration::from_millis(250)); + assert_eq!(super::disk_check_wait(Some(Duration::from_secs(60)), None), Duration::from_secs(5)); + assert_eq!(super::disk_check_wait(Some(Duration::MAX), None), Duration::from_secs(5)); + assert_eq!(super::disk_check_wait(None, None), CHECK_EVERY); + // after a read that couldn't start: at the retry, not five minutes on (CodeRabbit on #3077) + assert_eq!(super::disk_check_wait(None, Some(RETRY_AFTER)), RETRY_AFTER); + } + /// The prompt barrier keeps its own keys and pointer input, drops text/wheel input, and drops /// the start/move half of a new touch gesture while keeping End/Cancel to clear egui's private /// touch state. No prompt means everything survives. diff --git a/crates/ui-egui/src/render.rs b/crates/ui-egui/src/render.rs index 9237e43066..e8c3a90788 100644 --- a/crates/ui-egui/src/render.rs +++ b/crates/ui-egui/src/render.rs @@ -45,6 +45,13 @@ pub enum Slot { /// Background preparation of a neighbouring photo (no texture: its decoded source and view /// render are cached by the engine). 0 = next, 1 = previous. Prefetch(u8), + /// The loupe's stand-in for a photo the user is likely to step to next (the camera JPEG of an + /// unedited raw, or a cached render), made ahead in the direction of travel so a step shows it + /// at once: [`Renderer::request_quick`] takes the texture over into [`Slot::Preview`]. + QuickAhead(u8), + /// The loupe zoomed in on an unedited raw: its camera JPEG at full size, drawn under the window + /// render until that arrives (a focus check at 100 % without waiting for the raw). + QuickFull, /// A variant thumbnail (profile / preset browsers), by its job key ([`Session::variant_job`]). /// Kept in a bounded LRU ([`VARIANT_TEXTURES`]). Variant(u64), @@ -242,6 +249,11 @@ pub const MAX_VARIANT_TEXTURES: usize = 800; /// Priority of variant thumbnail jobs: below on-screen grid thumbnails and the loupe. const VARIANT_PRIORITY: u32 = 6; +/// The camera-preview edge a loupe stand-in is made at (0: one that doesn't depend on the size). +fn quick_edge(job: &QuickJob) -> usize { + job.embedded.as_ref().map_or(0, |e| e.2) +} + pub struct Renderer { #[cfg(test)] pub(crate) thumb_jobs_built: usize, @@ -283,6 +295,14 @@ pub struct Renderer { pub completed: u64, /// Finished jobs thrown away because a newer request for their slot had replaced them. pub discarded: u64, + /// Photos given a grid thumbnail since start (a render replacing a stand-in isn't counted again): + /// unlike [`Renderer::thumb_textures`] it never goes down when old thumbnails are evicted (for + /// benchmarks). + pub thumbs_made: u64, + /// The photos [`Self::thumbs_made`] counted, by library (a hash of its folder) and id: a + /// thumbnail made again after eviction, or after going to another library and back, is the + /// same photo (photo ids are a library's own). Never cleared. + thumbs_counted: std::collections::HashSet<(u64, PhotoId)>, /// Keep a CPU copy of every texture so the UI can be rasterized headlessly (set when the app /// is driven by the control channel). pub keep_pixels: bool, @@ -290,6 +310,10 @@ pub struct Renderer { stages: HashMap>, /// Slot → key of the last quick job requested for it (each is tried once). quick_tried: HashMap, + /// The camera-preview edge each loupe stand-in ([`Slot::Preview`], [`Slot::QuickAhead`]) was + /// asked for (0: one that doesn't depend on the size): a stand-in made ahead for a smaller loupe + /// isn't taken over by a larger one. + quick_edge: HashMap, /// Jobs that failed (unreadable or missing file…), by slot and job key, with the error and /// the photo: not requested again until the key changes (an edit, another size) — the grid /// asks every frame. @@ -327,6 +351,9 @@ pub struct Renderer { native_frames: HashMap, } +/// How many [`Slot::QuickAhead`] stand-ins the loupe keeps ready. +pub const AHEAD_SLOTS: u8 = 6; + /// After this long without renders the GPU renderer's pool of recycled buffers is freed. #[cfg(not(target_arch = "wasm32"))] const IDLE_TRIM: std::time::Duration = std::time::Duration::from_secs(3); @@ -395,9 +422,12 @@ impl Renderer { last_main_ms: 0.0, completed: 0, discarded: 0, + thumbs_made: 0, + thumbs_counted: Default::default(), keep_pixels: false, stages: HashMap::new(), quick_tried: HashMap::new(), + quick_edge: HashMap::new(), failed: HashMap::new(), catalog_rev: 0, prefetched: HashMap::new(), @@ -731,7 +761,9 @@ impl Renderer { } } - /// Request a stand-in for `slot` (once per job key). + /// Request a stand-in for `slot` (once per job key). The loupe's [`Slot::Preview`] takes over + /// a stand-in already made ahead for the same photo ([`Slot::QuickAhead`]) instead of making it + /// again: shown in this frame when it is ready, or as soon as that job finishes. pub fn request_quick(&mut self, slot: Slot, mut job: QuickJob, priority: u32) { if self.quick_tried.get(&slot) == Some(&job.key) { if self.pending.get(&slot).is_some_and(|p| p.0 == job.key && p.1 != priority) { @@ -758,6 +790,14 @@ impl Renderer { return; } self.quick_tried.insert(slot, job.key); + if matches!(slot, Slot::Preview | Slot::QuickAhead(_)) { + self.quick_edge.insert(slot, quick_edge(&job)); + } + if slot == Slot::Preview && self.take_ahead(job.photo, job.key, quick_edge(&job)) { + self.pending.remove(&slot); + self.requests.remove(&slot); + return; + } let source = job .small .as_ref() @@ -772,7 +812,84 @@ impl Renderer { ); let key = job.key; let display = self.display.clone(); - self.pool.submit(slot, key, priority, Box::new(move || Self::run_for(|| job.run(), &display))); + let background = matches!(slot, Slot::QuickAhead(_)); + self.pool.submit( + slot, + key, + priority, + Box::new(move || { + let run = || if background { lightcraft_engine::memory::in_background(|| job.run()) } else { job.run() }; + Self::run_for(run, &display) + }), + ); + } + + /// Whether the stand-in asked for in `slot` is at least `edge` pixels (see `quick_edge`). + fn ahead_covers(&self, slot: Slot, edge: usize) -> bool { + self.quick_edge.get(&slot).copied().unwrap_or(0) >= edge + } + + /// Move a [`Slot::QuickAhead`] texture of `photo` made for `key`, at least `edge` pixels, into + /// [`Slot::Preview`]. + fn take_ahead(&mut self, photo: PhotoId, key: u64, edge: usize) -> bool { + let Some(slot) = self + .textures + .iter() + .find(|(s, t)| matches!(s, Slot::QuickAhead(_)) && t.photo == photo && t.key == key && self.ahead_covers(**s, edge)) + .map(|(s, _)| *s) + else { + return false; + }; + let Some(tex) = self.textures.remove(&slot) else { return false }; + self.quick_tried.remove(&slot); + self.textures.insert(Slot::Preview, tex); + true + } + + /// Make the loupe's stand-ins for the photos it is likely to be stepped to next (see + /// [`Slot::QuickAhead`]), most likely first: `jobs[0]` gets `priority`, each next one less. + /// A slot keeps a photo for as long as the photo stays wanted (made or being made, it is not + /// made again when the user steps and the photos shift places); slots holding photos not + /// wanted any more are freed for the new ones, and their queued jobs dropped. + pub fn look_ahead(&mut self, jobs: Vec, priority: u32) { + let slots: Vec = (0..AHEAD_SLOTS).map(Slot::QuickAhead).collect(); + let held = |r: &Self, s: Slot| r.textures.get(&s).map(|t| t.key).or_else(|| r.quick_tried.get(&s).copied()); + // (a stand-in asked for at a smaller loupe size isn't kept for a larger one) + let wanted: HashMap = jobs.iter().map(|j| (j.key, quick_edge(j))).collect(); + let mut free = Vec::new(); + for &s in &slots { + let key = held(self, s); + if key.is_none_or(|k| wanted.get(&k).is_none_or(|&edge| !self.ahead_covers(s, edge))) { + self.textures.remove(&s); + // a job already running: its result is thrown away when it arrives (not left as a + // hidden stand-in), unless the loupe is waiting for just that one (a step to the + // photo): then the slot stays reserved for it until it arrives + let preview_edge = self.quick_edge.get(&Slot::Preview).copied().unwrap_or(0); + let loupe_waits = + key.is_some_and(|k| self.pending.get(&Slot::Preview).is_some_and(|p| p.0 == k)) && self.ahead_covers(s, preview_edge); + if !self.drop_queued(s) && loupe_waits { + continue; + } + self.quick_tried.remove(&s); + self.pending.remove(&s); + self.requests.remove(&s); + free.push(s); + } + } + let have: HashMap = slots.iter().filter_map(|&s| held(self, s).map(|k| (k, s))).collect(); + let mut free = free.into_iter(); + for (rank, job) in jobs.into_iter().enumerate() { + let priority = priority.saturating_sub(rank as u32); + if let Some(&s) = have.get(&job.key) { + // kept: a job still queued moves to its new place in the order + if self.quick_tried.get(&s) == Some(&job.key) { + self.request_quick(s, job, priority); + } + continue; + } + let Some(slot) = free.next() else { break }; + self.request_quick(slot, job, priority); + } } /// Render a slideshow title screen into `slot` (once per screen key, like [`Self::request`]): @@ -817,6 +934,21 @@ impl Renderer { ); } + /// Prepared-ahead jobs ([`Slot::Prefetch`]) queued or running. + pub fn prefetches_pending(&self) -> usize { + self.pending.keys().filter(|s| matches!(s, Slot::Prefetch(_))).count() + } + + /// Prepared-ahead slots asked for since the renderer last forgot everything. + pub fn prefetch_slots_asked(&self) -> usize { + self.prefetched.keys().filter(|s| matches!(s, Slot::Prefetch(_))).count() + } + + /// Whether `slot` was last asked for the job of `key` (queued, running or done). + pub fn prefetch_asked(&self, slot: Slot, key: u64) -> bool { + self.prefetched.get(&slot) == Some(&key) + } + /// Drop the slideshow slides other than `keep`: their textures and their requests that have /// not started (a slide that is no longer near is not worth rendering). pub fn release_slides(&mut self, keep: &[lightcraft_engine::slideshow::Item]) { @@ -869,6 +1001,11 @@ impl Renderer { self.textures.retain(|_, t| t.native.is_none()); } + /// Changes when [`Self::forget_all`] drops every texture and request: what was asked for before is gone. + pub fn epoch(&self) -> u64 { + self.epoch + } + pub fn is_pending(&self, slot: Slot) -> bool { self.pending.contains_key(&slot) } @@ -1132,6 +1269,89 @@ impl Renderer { finished } + /// The slot a finished job goes to. Made ahead for the photo the loupe is now waiting for, it is + /// the loupe's stand-in (the loupe's own request for it, if one was started, is dropped when it + /// arrives); also when the loupe's own request failed and it still wants that photo: the + /// stand-in made ahead still shows, but never over another photo's request. + fn stand_in_slot(&mut self, slot: Slot, r: &RenderResult) -> Slot { + let waiting = self.pending.get(&Slot::Preview).is_some_and(|p| p.0 == r.key); + let failed = !self.pending.contains_key(&Slot::Preview) + && self.quick_tried.get(&Slot::Preview) == Some(&r.key) + && self.failed.get(&Slot::Preview).is_some_and(|f| f.0 == r.key && f.3 == r.photo); + if !matches!(slot, Slot::QuickAhead(_)) + || r.rendered.is_err() + || !(waiting || failed) + || !self.ahead_covers(slot, self.quick_edge.get(&Slot::Preview).copied().unwrap_or(0)) + { + return slot; + } + self.pending.remove(&Slot::Preview); + self.requests.remove(&Slot::Preview); + self.quick_tried.remove(&slot); + Slot::Preview + } + + /// A superseded result an interactive view still takes: a draft newer than what it shows. + fn is_newer_draft(&self, slot: Slot, r: &RenderResult) -> bool { + slot.is_view() + && self.requests.contains_key(&slot) + && r.request_id >= self.epoch + && self.shown_ids.get(&slot).is_none_or(|shown| r.request_id > *shown) + && r.rendered.is_ok() + } + + /// Convert `r` to the display profile in use; whether that changed it. + fn convert_for_display(&self, r: &mut RenderResult) -> bool { + let Some(d) = &self.display else { return false }; + let before = r.display; + lightcraft_engine::display::present(r, d); + r.display != before + } + + /// Show a render that stayed on the GPU (`r.frame`) in `slot`; whether it could be. When it + /// can't, views come back as pixels from then on. + fn show_frame(&mut self, ctx: &egui::Context, slot: Slot, r: &RenderResult, look: Option<(u64, bool)>, ms: f64) -> bool { + let (Some(frame), Ok(rendered)) = (r.frame.as_ref(), r.rendered.as_ref()) else { return false }; + self.failed.remove(&slot); + let shown = self.screen.as_mut().and_then(|s| s.show(frame, self.natives.get(&slot).copied())); + let Some(id) = shown else { + // can't be shown: from now on views come back as pixels + log::warn!("render {slot:?}: a GPU-resident render can't be shown; rendering pixels"); + self.free_natives(); + self.screen = None; + return false; + }; + self.natives.insert(slot, id); + if self.keep_pixels { + self.native_frames.insert(slot, frame.clone()); + } + let (size, histogram) = ([frame.width, frame.height], rendered.histogram.clone()); + match self.textures.get_mut(&slot) { + Some(t) => { + t.show_native(id); + t.look = look; + t.key = r.key; + t.photo = r.photo; + t.size = size; + t.histogram = Some(histogram); + t.ms = ms; + t.quick = None; + } + None => { + // (`tex` is never drawn while `native` is set) + let blank = egui::ColorImage::filled([1, 1], egui::Color32::BLACK); + let tex = ctx.load_texture(format!("{slot:?}"), blank, egui::TextureOptions::LINEAR); + let t = + Tex { look, key: r.key, photo: r.photo, tex, size, histogram: Some(histogram), ms, quick: None, pixels: None, native: Some(id) }; + self.textures.insert(slot, t); + } + } + if slot == Slot::Main { + self.last_main_ms = ms; + } + true + } + /// Turn finished jobs into textures. Returns true if anything changed. fn collect(&mut self, ctx: &egui::Context, session: &mut Session, inline_ms: f64) -> bool { let finished = self.take_finished(inline_ms); @@ -1152,12 +1372,7 @@ impl Renderer { // shows, and the source it decoded: a drag supersedes every request before it // finishes, and a source thrown away with them is decoded again for every tick. let (source, look) = if self.requests.get(&slot).map(|request| request.id) != Some(r.request_id) { - let newer_draft = slot.is_view() - && self.requests.contains_key(&slot) - && r.request_id >= self.epoch - && self.shown_ids.get(&slot).is_none_or(|shown| r.request_id > *shown) - && r.rendered.is_ok(); - if !newer_draft { + if !self.is_newer_draft(slot, &r) { self.discarded += 1; let replaced = self.requests.contains_key(&slot); self.decided(|| { @@ -1180,66 +1395,16 @@ impl Renderer { self.shown_ids.insert(slot, r.request_id); } // (results from an offload, which doesn't know the display profile: converted here) - if let Some(d) = &self.display { - let before = r.display; - lightcraft_engine::display::present(&mut r, d); - if r.display != before { - color = None; // converted after the worker made its texture pixels - } + if self.convert_for_display(&mut r) { + color = None; // converted after the worker made its texture pixels } if self.pending.get(&slot).is_some_and(|p| p.0 == r.key) { self.pending.remove(&slot); } + slot = self.stand_in_slot(slot, &r); // a render that stayed on the GPU: drawn from its texture - if let (Some(frame), Ok(rendered)) = (r.frame.as_ref(), r.rendered.as_ref()) { - self.failed.remove(&slot); - let shown = self.screen.as_mut().and_then(|s| s.show(frame, self.natives.get(&slot).copied())); - let Some(id) = shown else { - // can't be shown: from now on views come back as pixels - log::warn!("render {slot:?}: a GPU-resident render can't be shown; rendering pixels"); - self.free_natives(); - self.screen = None; - continue; - }; - self.natives.insert(slot, id); - if self.keep_pixels { - self.native_frames.insert(slot, frame.clone()); - } - let (size, histogram) = ([frame.width, frame.height], rendered.histogram.clone()); - match self.textures.get_mut(&slot) { - Some(t) => { - t.show_native(id); - t.look = look; - t.key = r.key; - t.photo = r.photo; - t.size = size; - t.histogram = Some(histogram); - t.ms = ms; - t.quick = None; - } - None => { - // (`tex` is never drawn while `native` is set) - let blank = egui::ColorImage::filled([1, 1], egui::Color32::BLACK); - let tex = ctx.load_texture(format!("{slot:?}"), blank, egui::TextureOptions::LINEAR); - let t = Tex { - look, - key: r.key, - photo: r.photo, - tex, - size, - histogram: Some(histogram), - ms, - quick: None, - pixels: None, - native: Some(id), - }; - self.textures.insert(slot, t); - } - } - if slot == Slot::Main { - self.last_main_ms = ms; - } - changed = true; + if r.frame.is_some() && r.rendered.is_ok() { + changed |= self.show_frame(ctx, slot, &r, look, ms); continue; } let rendered = match r.rendered { @@ -1259,7 +1424,7 @@ impl Renderer { x } Err(e) => { - if !matches!(slot, Slot::Prefetch(_)) { + if !matches!(slot, Slot::Prefetch(_) | Slot::QuickAhead(_)) { let message = render_failure_message(slot, r.photo, source.as_ref(), &e); #[cfg(test)] self.failure_messages.push(message.clone()); @@ -1291,6 +1456,11 @@ impl Renderer { slot = Slot::Thumb(id); } } + // (a photo's first thumbnail: not a render replacing its stand-in, nor one made again) + let first_thumb = match slot { + Slot::Thumb(id) | Slot::ThumbQuick(id) => self.thumbs_counted.insert((library_key(session), id)), + _ => false, + }; if let Slot::Thumb(id) = slot { self.textures.remove(&Slot::ThumbQuick(id)); } @@ -1332,7 +1502,11 @@ impl Renderer { ); } } - if slot == Slot::Main { + if first_thumb { + self.thumbs_made += 1; + } + // (a preview taken from the cache says nothing about how long a render takes) + if slot == Slot::Main && r.quick.is_none() { self.last_main_ms = ms; } changed = true; @@ -1413,6 +1587,29 @@ impl Renderer { .collect() } + /// Forget everything about `slot`: its texture, its queued job, and that its stand-in was asked + /// for (so asking again makes it again). + pub fn forget(&mut self, slot: Slot) { + self.drop_queued(slot); + self.pending.remove(&slot); + self.requests.remove(&slot); + self.quick_tried.remove(&slot); + self.release(slot); + } + + /// Drop `slot`'s job if it is still queued (a running one finishes). Returns whether one was. + pub fn drop_queued(&mut self, slot: Slot) -> bool { + let dropped = self.pool.reprioritize(|s, p| if *s == slot { None } else { Some(p) }); + let before = self.queue.len(); + self.queue.retain(|q| q.slot != slot); + let any = !dropped.is_empty() || self.queue.len() != before; + if any { + self.pending.remove(&slot); + self.requests.remove(&slot); + } + any + } + /// Drop the import review thumbnails (a new review, or the dialog closed). pub fn forget_imports(&mut self) { self.retain_imports(&Default::default()); @@ -1773,3 +1970,13 @@ mod color_image_tests { assert!(color_image(&img).pixels.is_empty()); } } + +/// Which library `session` has open, for [`Renderer::thumbs_made`]: a hash of its folder (the same +/// library opened again is the same), 0 with none. +fn library_key(session: &Session) -> u64 { + use std::hash::{Hash, Hasher}; + let Some(lib) = session.library.as_ref() else { return 0 }; + let mut h = std::collections::hash_map::DefaultHasher::new(); + lib.dir.hash(&mut h); + h.finish() +} diff --git a/crates/ui-egui/src/render/thumbnail_tests.rs b/crates/ui-egui/src/render/thumbnail_tests.rs index 7a2cb1a6a9..b2a87e6522 100644 --- a/crates/ui-egui/src/render/thumbnail_tests.rs +++ b/crates/ui-egui/src/render/thumbnail_tests.rs @@ -1541,3 +1541,407 @@ fn undo_to_embedded_thumb_rejects_late_full_render() { assert_eq!(quick.pixels.as_ref().unwrap().pixels, pixels_a); assert!(!app.renderer.is_pending(full_slot)); } + +/// PERF-CULL-2: the loupe's stand-ins made ahead are taken over at the step (no job, the texture +/// at once), a photo still wanted keeps its slot as the photos shift, and a step to a photo whose +/// stand-in is still being made shows it when it arrives. +#[test] +fn a_step_takes_over_the_camera_preview_made_ahead() { + let gate = Arc::new(std::sync::Mutex::new(())); + let mut s = Session::new(); + let g = gate.clone(); + s.media.preview_loader = Some(Arc::new(move |_: &str, _| { + drop(g.lock().unwrap_or_else(std::sync::PoisonError::into_inner)); + Some(lightcraft_raster::Rgba8 { width: 4, height: 4, data: vec![[200, 100, 50, 255]; 16] }) + })); + let raw = |id: u64| { + let mut p = Photo::new(PhotoId(id), Source::File { path: format!("r{id}.cr3") }, &format!("r{id}.cr3"), "CR3", 6000, 4000, "2026-01-01"); + p.kind = lightcraft_catalog::MediaKind::Raw; + p.develop = Arc::new(p.camera_defaults()); + p + }; + let (mut app, _, ctx) = fixture_with(s, raw(1)); + for id in 2..=4 { + app.session.catalog.apply(Op::AddPhoto { photo: Box::new(raw(id)) }).unwrap(); + } + let job = |app: &mut crate::LightcraftApp, id: u64| app.session.quick_view_job(PhotoId(id), 64, true).unwrap(); + let wait = |app: &mut crate::LightcraftApp, done: &dyn Fn(&Renderer) -> bool| { + let t0 = std::time::Instant::now(); + while !done(&app.renderer) { + assert!(t0.elapsed() < std::time::Duration::from_secs(20), "the stand-ins never arrived"); + std::thread::sleep(std::time::Duration::from_millis(5)); + app.renderer.poll(&ctx, &mut app.session); + } + }; + let ahead_of = |r: &Renderer, id: u64| r.textures.iter().any(|(s, t)| matches!(s, Slot::QuickAhead(_)) && t.photo == PhotoId(id)); + let jobs = vec![job(&mut app, 2), job(&mut app, 3)]; + app.renderer.look_ahead(jobs, 96); + wait(&mut app, &|r| ahead_of(r, 2) && ahead_of(r, 3)); + // the step to photo 2: its stand-in is the loupe's at once, nothing is made again + let q = job(&mut app, 2); + app.renderer.request_quick(Slot::Preview, q, 110); + assert!(!app.renderer.is_pending(Slot::Preview)); + assert_eq!(app.renderer.textures.get(&Slot::Preview).map(|t| t.photo), Some(PhotoId(2))); + assert!(!ahead_of(&app.renderer, 2)); + // the next look-ahead (3 and 4): 3 keeps its slot and is not made again + let held = gate.lock().unwrap(); + let jobs = vec![job(&mut app, 3), job(&mut app, 4)]; + app.renderer.look_ahead(jobs, 96); + assert!(ahead_of(&app.renderer, 3), "a photo still wanted keeps what was made for it"); + let pending_ahead: Vec = app.renderer.pending.keys().copied().filter(|s| matches!(s, Slot::QuickAhead(_))).collect(); + assert_eq!(pending_ahead.len(), 1, "only photo 4 is being made: {pending_ahead:?}"); + // the step to photo 4 while its stand-in is still being made: shown when it arrives + let q = job(&mut app, 4); + app.renderer.request_quick(Slot::Preview, q, 110); + drop(held); + wait(&mut app, &|r| r.textures.get(&Slot::Preview).is_some_and(|t| t.photo == PhotoId(4)) && !r.is_pending(Slot::Preview)); + wait(&mut app, &|r| !r.pending.keys().any(|s| matches!(s, Slot::QuickAhead(_) | Slot::Preview))); + assert_eq!(app.renderer.textures.get(&Slot::Preview).map(|t| t.photo), Some(PhotoId(4))); +} + +/// A loupe with unedited raws 1..=n whose camera previews are read at once, on a renderer +/// that runs nothing until asked (`run_inline`). +fn raw_loupe(n: u64) -> (crate::LightcraftApp, egui::Context) { + let mut s = Session::new(); + s.media.preview_loader = Some(Arc::new(|_: &str, _| Some(lightcraft_raster::Rgba8 { width: 4, height: 4, data: vec![[200, 100, 50, 255]; 16] }))); + let raw = |id: u64| { + let mut p = Photo::new(PhotoId(id), Source::File { path: format!("r{id}.cr3") }, &format!("r{id}.cr3"), "CR3", 6000, 4000, "2026-01-01"); + p.kind = lightcraft_catalog::MediaKind::Raw; + p.develop = Arc::new(p.camera_defaults()); + p + }; + let (mut app, _, ctx) = fixture_with(s, raw(1)); + for id in 2..=n { + app.session.catalog.apply(Op::AddPhoto { photo: Box::new(raw(id)) }).unwrap(); + } + app.renderer.pool = JobPool::new(0); + (app, ctx) +} + +/// The look-ahead with every photo allowed (the default preparation settings). +const ALL: crate::panels::detail::Prepare = crate::panels::detail::Prepare::Photos(lightcraft_engine::prep::Scope::All); + +fn quick(app: &mut crate::LightcraftApp, id: u64, edge: usize) -> QuickJob { + app.session.quick_view_job(PhotoId(id), edge, true).unwrap() +} + +/// The loupe's stand-ins ahead follow Settings ▸ Performance: none when the mode prepares only the photo on +/// screen (or none ahead), and the ones still queued are dropped then; only marked photos in the marked scope +/// (CodeRabbit on #3077). +#[test] +fn the_look_ahead_prepares_only_what_the_preparation_settings_allow() { + use crate::panels::detail::{Prepare, look_ahead}; + use lightcraft_engine::prep::Scope; + let (mut app, _ctx) = raw_loupe(4); + app.session.sort.key = lightcraft_catalog::SortKey::FileName; + app.session.sort.ascending = true; + // (asked for: queued, running or made) + let ahead = |r: &Renderer| r.quick_tried.keys().filter(|s| matches!(s, Slot::QuickAhead(_))).count(); + look_ahead(&mut app, PhotoId(1), 64, true, Prepare::of(5, Scope::All)); + assert_eq!(ahead(&app.renderer), 3, "photos 2 to 4 are made ahead"); + // the photo on screen only: nothing ahead, and what was queued goes + look_ahead(&mut app, PhotoId(1), 64, true, Prepare::of(0, Scope::OnScreen)); + assert_eq!(ahead(&app.renderer), 0, "nothing is made ahead for the photo on screen only"); + look_ahead(&mut app, PhotoId(1), 64, true, Prepare::of(0, Scope::All)); + assert_eq!(ahead(&app.renderer), 0, "nor with no photos ahead"); + // picks, rated and labelled: only those + app.session.execute("library.select", &serde_json::json!({"ids": [3]})).unwrap(); + app.session.execute("photo.rate", &serde_json::json!({"rating": 2})).unwrap(); + look_ahead(&mut app, PhotoId(1), 64, true, Prepare::of(5, Scope::Marked)); + let k3 = quick(&mut app, 3, 64).key; + let queued: Vec = app.renderer.quick_tried.iter().filter(|(s, _)| matches!(s, Slot::QuickAhead(_))).map(|(_, k)| *k).collect(); + assert_eq!(queued, vec![k3], "only the rated photo is made ahead"); +} + +/// A renderer reset (`forget_all`: a library switch, a colour-management change) drops the stand-ins asked +/// for ahead; the same photo asks for them again (CodeRabbit on #3577: the look-ahead took them for made). +#[test] +fn the_look_ahead_asks_again_after_a_renderer_reset() { + use crate::panels::detail::look_ahead; + let (mut app, _ctx) = raw_loupe(4); + let ahead = |r: &Renderer| r.quick_tried.keys().filter(|s| matches!(s, Slot::QuickAhead(_))).count(); + look_ahead(&mut app, PhotoId(1), 64, true, ALL); + let asked = ahead(&app.renderer); + assert!(asked > 0); + app.renderer.forget_all(); + assert_eq!(ahead(&app.renderer), 0); + look_ahead(&mut app, PhotoId(1), 64, true, ALL); + assert_eq!(ahead(&app.renderer), asked, "asked for again"); +} + +/// A stand-in still queued ahead whose photo moves up in the order (the user stepped) is +/// queued at its new place, not left behind photos now further away (CodeRabbit on #3055). +#[test] +fn a_kept_ahead_job_moves_to_its_new_place_in_the_order() { + let (mut app, _ctx) = raw_loupe(3); + let jobs = vec![quick(&mut app, 2, 64), quick(&mut app, 3, 64)]; + let (k2, k3) = (jobs[0].key, jobs[1].key); + app.renderer.look_ahead(jobs, 96); + let priority_of = |r: &Renderer, key: u64| { + let slot = r.quick_tried.iter().find(|(s, k)| matches!(s, Slot::QuickAhead(_)) && **k == key).map(|(s, _)| *s).unwrap(); + r.pending[&slot].1 + }; + assert_eq!((priority_of(&app.renderer, k2), priority_of(&app.renderer, k3)), (96, 95)); + let jobs = vec![quick(&mut app, 3, 64), quick(&mut app, 2, 64)]; + app.renderer.look_ahead(jobs, 96); + assert_eq!((priority_of(&app.renderer, k3), priority_of(&app.renderer, k2)), (96, 95)); + assert_eq!(app.renderer.pool.queued(), 2, "nothing is queued twice"); +} + +/// A stand-in made ahead that is no longer wanted while it is being made is thrown away when it +/// arrives, not kept as a hidden texture (CodeRabbit on #3077); one the loupe waits for is kept. +#[test] +fn an_ahead_stand_in_no_longer_wanted_is_dropped_when_it_arrives() { + let gate = Arc::new(std::sync::Mutex::new(())); + let mut s = Session::new(); + let g = gate.clone(); + s.media.preview_loader = Some(Arc::new(move |_: &str, _| { + drop(g.lock().unwrap_or_else(std::sync::PoisonError::into_inner)); + Some(lightcraft_raster::Rgba8 { width: 4, height: 4, data: vec![[200, 100, 50, 255]; 16] }) + })); + let raw = |id: u64| { + let mut p = Photo::new(PhotoId(id), Source::File { path: format!("r{id}.cr3") }, &format!("r{id}.cr3"), "CR3", 6000, 4000, "2026-01-01"); + p.kind = lightcraft_catalog::MediaKind::Raw; + p.develop = Arc::new(p.camera_defaults()); + p + }; + let (mut app, _, ctx) = fixture_with(s, raw(1)); + app.session.catalog.apply(Op::AddPhoto { photo: Box::new(raw(2)) }).unwrap(); + let wait = |app: &mut crate::LightcraftApp, done: &dyn Fn(&Renderer) -> bool, what: &str| { + let t0 = std::time::Instant::now(); + while !done(&app.renderer) { + assert!(t0.elapsed() < std::time::Duration::from_secs(20), "{what}"); + std::thread::sleep(std::time::Duration::from_millis(5)); + app.renderer.poll(&ctx, &mut app.session); + } + }; + let any_ahead = |r: &Renderer| r.textures.keys().any(|s| matches!(s, Slot::QuickAhead(_))); + let held = gate.lock().unwrap(); + let job = quick(&mut app, 2, 64); + app.renderer.look_ahead(vec![job], 96); + wait(&mut app, &|r| r.pool.queued() == 0, "the stand-in never started"); + // not wanted any more (the user went elsewhere) while it is being made + app.renderer.look_ahead(Vec::new(), 96); + assert!(!app.renderer.pending.keys().any(|s| matches!(s, Slot::QuickAhead(_)))); + let done = app.renderer.completed; + drop(held); + wait(&mut app, &|r| r.completed > done, "the stand-in never finished"); + assert!(!any_ahead(&app.renderer), "a stand-in nobody wants is left as a hidden texture"); + // the step to photo 2 while its stand-in is being made: the loupe takes it when it arrives + let held = gate.lock().unwrap(); + let job = quick(&mut app, 2, 32); + app.renderer.look_ahead(vec![job], 96); + wait(&mut app, &|r| r.pool.queued() == 0, "the stand-in never started"); + let job = quick(&mut app, 2, 32); + app.renderer.request_quick(Slot::Preview, job, 110); + app.renderer.look_ahead(Vec::new(), 96); + drop(held); + wait(&mut app, &|r| r.textures.get(&Slot::Preview).is_some_and(|t| t.photo == PhotoId(2)), "the loupe never got its stand-in"); +} + +/// The loupe waits for a stand-in made ahead that is still being made when the next look-ahead +/// brings new photos: its slot stays reserved, so the result still reaches the loupe (CodeRabbit +/// on #3055 and #3080). +#[test] +fn a_running_stand_in_the_loupe_waits_for_keeps_its_slot() { + // the first camera preview read (photo 2's, made ahead) waits for `first`, every later one + // for `later` + let first = Arc::new(std::sync::Mutex::new(())); + let later = Arc::new(std::sync::Mutex::new(())); + let calls = Arc::new(std::sync::atomic::AtomicUsize::new(0)); + let mut s = Session::new(); + let (f, l, c) = (first.clone(), later.clone(), calls.clone()); + s.media.preview_loader = Some(Arc::new(move |_: &str, _| { + let gate = if c.fetch_add(1, std::sync::atomic::Ordering::SeqCst) == 0 { &f } else { &l }; + drop(gate.lock().unwrap_or_else(std::sync::PoisonError::into_inner)); + Some(lightcraft_raster::Rgba8 { width: 4, height: 4, data: vec![[200, 100, 50, 255]; 16] }) + })); + let raw = |id: u64| { + let mut p = Photo::new(PhotoId(id), Source::File { path: format!("r{id}.cr3") }, &format!("r{id}.cr3"), "CR3", 6000, 4000, "2026-01-01"); + p.kind = lightcraft_catalog::MediaKind::Raw; + p.develop = Arc::new(p.camera_defaults()); + p + }; + let (mut app, _, ctx) = fixture_with(s, raw(1)); + for id in 2..=3 { + app.session.catalog.apply(Op::AddPhoto { photo: Box::new(raw(id)) }).unwrap(); + } + app.renderer.pool = JobPool::new(4); + let wait = |app: &mut crate::LightcraftApp, done: &dyn Fn(&Renderer) -> bool, what: &str| { + let t0 = std::time::Instant::now(); + while !done(&app.renderer) { + assert!(t0.elapsed() < std::time::Duration::from_secs(20), "{what}"); + std::thread::sleep(std::time::Duration::from_millis(5)); + app.renderer.poll(&ctx, &mut app.session); + } + }; + let held_first = first.lock().unwrap(); + let held_later = later.lock().unwrap(); + let job = quick(&mut app, 2, 64); + app.renderer.look_ahead(vec![job], 96); + wait(&mut app, &|r| r.pool.queued() == 0, "photo 2's stand-in never started"); + wait(&mut app, &|_| calls.load(std::sync::atomic::Ordering::SeqCst) == 1, "photo 2's preview was never read"); + // the step to photo 2, then the next look-ahead, for photos 3 and 1 + let job = quick(&mut app, 2, 64); + app.renderer.request_quick(Slot::Preview, job, 110); + assert!(app.renderer.is_pending(Slot::Preview)); + let jobs = vec![quick(&mut app, 3, 64), quick(&mut app, 1, 64)]; + app.renderer.look_ahead(jobs, 96); + let done = app.renderer.completed; + drop(held_first); + wait(&mut app, &|r| r.completed > done, "photo 2's stand-in never finished"); + assert_eq!(app.renderer.textures.get(&Slot::Preview).map(|t| t.photo), Some(PhotoId(2)), "the loupe lost the stand-in it waited for"); + drop(held_later); + wait(&mut app, &|r| r.pool.queued() == 0 && !r.pending.keys().any(|s| matches!(s, Slot::QuickAhead(_))), "the look-ahead never finished"); +} + +/// The stand-in made ahead still shows when the loupe's own request for it failed first +/// (CodeRabbit on #3055). +#[test] +fn a_stand_in_made_ahead_shows_after_the_loupes_own_request_failed() { + let (mut app, ctx) = raw_loupe(2); + let job = quick(&mut app, 2, 64); + let key = job.key; + app.renderer.look_ahead(vec![job], 96); + let job = quick(&mut app, 2, 64); + app.renderer.request_quick(Slot::Preview, job, 110); + assert!(app.renderer.is_pending(Slot::Preview)); + // the loupe's request fails (its job never runs) + assert!(app.renderer.drop_queued(Slot::Preview)); + app.renderer.failed.insert(Slot::Preview, (key, "unreadable".into(), 0, PhotoId(2))); + assert_eq!(app.renderer.pool.run_inline(1), 1); + app.renderer.poll(&ctx, &mut app.session); + assert_eq!(app.renderer.textures.get(&Slot::Preview).map(|t| t.photo), Some(PhotoId(2))); + assert!(!app.renderer.failed.contains_key(&Slot::Preview)); + // the user has stepped on to photo 3 when photo 2's stand-in arrives: 3's request stays + let (mut app, ctx) = raw_loupe(3); + let job = quick(&mut app, 2, 64); + let key = job.key; + app.renderer.look_ahead(vec![job], 96); + let job = quick(&mut app, 2, 64); + app.renderer.request_quick(Slot::Preview, job, 110); + assert!(app.renderer.drop_queued(Slot::Preview)); + app.renderer.failed.insert(Slot::Preview, (key, "unreadable".into(), 0, PhotoId(2))); + let job = quick(&mut app, 3, 64); + let key3 = job.key; + // (queued behind the stand-in made ahead, so that one arrives first) + app.renderer.request_quick(Slot::Preview, job, 50); + let ahead = app.renderer.pool.run_inline(1); + assert_eq!(ahead, 1); + app.renderer.poll(&ctx, &mut app.session); + assert!(app.renderer.textures.get(&Slot::Preview).is_none_or(|t| t.photo != PhotoId(2)), "photo 2 took photo 3's place"); + assert_eq!(app.renderer.pending.get(&Slot::Preview).map(|p| p.0), Some(key3), "photo 3's request is kept"); +} + +/// A stand-in made ahead for a smaller loupe isn't taken over when the loupe has grown: the +/// larger one is made (CodeRabbit on #3077). +#[test] +fn a_stand_in_made_ahead_for_a_smaller_loupe_is_not_taken_over() { + let (mut app, ctx) = raw_loupe(2); + let small = quick(&mut app, 2, 32); + let large = quick(&mut app, 2, 64); + assert_eq!(small.key, large.key, "the same stand-in, at another size"); + app.renderer.look_ahead(vec![small], 96); + assert_eq!(app.renderer.pool.run_inline(1), 1); + app.renderer.poll(&ctx, &mut app.session); + assert!(app.renderer.textures.keys().any(|s| matches!(s, Slot::QuickAhead(_)))); + app.renderer.request_quick(Slot::Preview, large, 110); + assert!(app.renderer.is_pending(Slot::Preview), "the larger stand-in is made"); + assert!(!app.renderer.textures.contains_key(&Slot::Preview)); + // the same size is taken over at once + let (mut app, ctx) = raw_loupe(2); + let job = quick(&mut app, 2, 64); + app.renderer.look_ahead(vec![job], 96); + assert_eq!(app.renderer.pool.run_inline(1), 1); + app.renderer.poll(&ctx, &mut app.session); + let job = quick(&mut app, 2, 64); + app.renderer.request_quick(Slot::Preview, job, 110); + assert!(!app.renderer.is_pending(Slot::Preview)); + assert_eq!(app.renderer.textures.get(&Slot::Preview).map(|t| t.photo), Some(PhotoId(2))); +} + +/// The benchmark's thumbnail count is per photo: a render replacing a photo's stand-in isn't +/// counted again (CodeRabbit on #3055). +#[test] +fn a_thumbnail_replacing_its_stand_in_is_not_counted_again() { + let mut s = Session::new(); + s.media.preview_loader = Some(Arc::new(|_, _| Some(lightcraft_raster::Rgba8 { width: 4, height: 4, data: vec![[200, 100, 50, 255]; 16] }))); + let mut raw = (*photo()).clone(); + raw.kind = lightcraft_catalog::MediaKind::Raw; + raw.develop = Arc::new(raw.camera_defaults()); + let (mut app, work, ctx) = fixture_with(s, raw); + app.renderer.pool = JobPool::new(0); + let id = PhotoId(1); + crate::panels::grid::request_thumb(&mut app, id, 384, 10); + assert_eq!(app.renderer.pool.run_inline(1), 1); + app.renderer.poll(&ctx, &mut app.session); + assert!(app.renderer.textures.contains_key(&Slot::ThumbQuick(id))); + assert_eq!(app.renderer.thumbs_made, 1); + let mut edited = (*app.session.catalog.photo(id).unwrap().develop).clone(); + edited.light.exposure += 0.5; + app.session.commit("Exposure", Op::SetDevelop { id, settings: Arc::new(edited), label: "Exposure".into(), edited: None }).unwrap(); + crate::panels::grid::request_thumb(&mut app, id, 384, 10); + let (slot, job) = take_job(&work); + assert_eq!(slot, Slot::Thumb(id)); + finish(&mut app, &work, &ctx, slot, job.run()); + assert!(app.renderer.textures.contains_key(&Slot::Thumb(id)), "the render replaced the stand-in"); + assert_eq!(app.renderer.thumbs_made, 1, "still one photo"); + // evicted and made again (scrolled away and back): the same photo (CodeRabbit on #3055) + app.renderer.evict_thumbs(&Default::default(), 0); + assert!(!app.renderer.textures.contains_key(&Slot::Thumb(id)), "evicted"); + crate::panels::grid::request_thumb(&mut app, id, 384, 10); + let (slot, job) = take_job(&work); + finish(&mut app, &work, &ctx, slot, job.run()); + assert!(app.renderer.textures.contains_key(&slot), "made again"); + assert_eq!(app.renderer.thumbs_made, 1, "an evicted photo made again is not counted again"); +} + +/// The loupe's look-ahead asks again when only the crop changes (its stand-ins' keys include the +/// crop), instead of keeping stand-ins the loupe can no longer use (CodeRabbit on #3077). +#[test] +fn the_look_ahead_asks_again_when_only_the_crop_changes() { + let (mut app, _ctx) = raw_loupe(3); + let ahead_keys = |r: &Renderer| r.quick_tried.iter().filter(|(s, _)| matches!(s, Slot::QuickAhead(_))).map(|(_, k)| *k).collect::>(); + crate::panels::detail::look_ahead(&mut app, PhotoId(1), 64, true, ALL); + let cropped = app.session.quick_view_job(PhotoId(2), 64, true).unwrap().key; + assert!(ahead_keys(&app.renderer).contains(&cropped)); + crate::panels::detail::look_ahead(&mut app, PhotoId(1), 64, false, ALL); + let uncropped = app.session.quick_view_job(PhotoId(2), 64, false).unwrap().key; + assert_ne!(cropped, uncropped); + assert!(ahead_keys(&app.renderer).contains(&uncropped), "the stand-ins are made again without the crop"); +} + +/// The loupe's look-ahead asks again when the view's order changes around the same photo (a new +/// sort, filter or source changes its neighbours without an edit; CodeRabbit on #3055). +#[test] +fn the_look_ahead_follows_a_new_order_around_the_same_photo() { + let (mut app, _ctx) = raw_loupe(5); + let ahead_keys = |r: &Renderer| r.quick_tried.iter().filter(|(s, _)| matches!(s, Slot::QuickAhead(_))).map(|(_, k)| *k).collect::>(); + app.session.sort.key = lightcraft_catalog::SortKey::FileName; + app.session.sort.ascending = true; + crate::panels::detail::look_ahead(&mut app, PhotoId(3), 64, true, ALL); + let one = quick(&mut app, 1, 64).key; + assert!(!ahead_keys(&app.renderer).contains(&one), "r1 is two behind: not prepared"); + // reversed: r1 is two ahead now + app.session.sort.ascending = false; + crate::panels::detail::look_ahead(&mut app, PhotoId(3), 64, true, ALL); + assert!(ahead_keys(&app.renderer).contains(&one), "the new neighbours are prepared"); +} + +/// Stepping fast (an arrow held down), the raw renders still queued for the photo left behind are +/// dropped, the Before view's too (CodeRabbit on #3055); the whole-frame render on screen stays. +#[test] +fn stepping_fast_drops_the_queued_before_render() { + let (mut app, _work, ctx) = fixture(); + app.renderer.set_offload(Box::new(Blocked)); + for slot in [Slot::Main, Slot::Before, Slot::Region, Slot::RegionBefore] { + let job = app.session.render_job(PhotoId(1), 64, 64, false, true).unwrap(); + app.renderer.request(slot, job, 100); + } + app.renderer.poll(&ctx, &mut app.session); + assert!(app.renderer.is_pending(Slot::Before), "queued behind the blocked worker"); + crate::panels::detail::drop_unsettled(&mut app.renderer, PhotoId(2)); + for slot in [Slot::Main, Slot::Before, Slot::Region, Slot::RegionBefore] { + assert!(!app.renderer.is_pending(slot), "{slot:?} still queued"); + } +} diff --git a/crates/ui-egui/src/state/settings.rs b/crates/ui-egui/src/state/settings.rs index b858b9a55f..0fae2ed8e7 100644 --- a/crates/ui-egui/src/state/settings.rs +++ b/crates/ui-egui/src/state/settings.rs @@ -166,6 +166,11 @@ pub struct AppSettings { /// View ▸ View Options ▸ Show message when loading or rendering photos: the loupe says /// "Rendering…" while the photo it shows is not the finished render. pub loupe_messages: bool, + /// Settings ▸ Performance ▸ Prepare photos in the background, app-wide (a library may keep its + /// own in its `prefs.json`): handed to the session at launch and kept in step with it. Read + /// leniently: a value this build doesn't know is the defaults, never the other settings lost. + #[serde(deserialize_with = "lenient")] + pub background_prep: lightcraft_engine::prep::PrepSettings, /// Shortcuts the user changed (Help ▸ Keyboard Shortcuts): command id → shortcut, `""` = none. pub keymap: crate::shortcuts::Keymap, /// Auto (follow the system), Dark or Light. @@ -238,6 +243,7 @@ impl Default for AppSettings { film_badges: true, grid_badges: GridBadges::Auto, loupe_messages: true, + background_prep: Default::default(), keymap: Default::default(), appearance_mode: AppearanceMode::Dark, dark_theme: DarkTheme::Charcoal, @@ -430,6 +436,16 @@ fn deserialize_favorite_themes<'de, D: serde::Deserializer<'de>>(d: D) -> Result mod loupe_info_tests { use super::*; + /// A background preparation value this build doesn't know (a newer build's scope) reads as the + /// defaults and keeps the rest of the settings. + #[test] + fn an_unknown_background_preparation_value_costs_nothing_else() { + let v = serde_json::json!({"filmNames": false, "backgroundPrep": {"culling": {"scope": "someday"}}}); + let read: AppSettings = serde_json::from_value(v).unwrap(); + assert_eq!(read.background_prep, lightcraft_engine::prep::PrepSettings::default()); + assert!(!read.film_names); + } + /// The lines chosen in View Options are saved with the app's settings; a settings file from /// before they existed, or with a line this build doesn't know, gets the default lines and /// keeps the rest of its settings. diff --git a/crates/ui-egui/src/tests.rs b/crates/ui-egui/src/tests.rs index 4401d5d179..19717ef817 100644 --- a/crates/ui-egui/src/tests.rs +++ b/crates/ui-egui/src/tests.rs @@ -24,6 +24,7 @@ mod dialog_settings; mod dialog_settings_text; mod dialog_tab; mod dialogs; +mod disk_space; mod docking_acceptance; mod drop_import; mod edit_layout; @@ -80,6 +81,7 @@ mod recheck; mod rename_person; mod rules_editor; mod sam_setup; +mod scan_probes; mod scroll; mod slider_reset; mod slideshow; diff --git a/crates/ui-egui/src/tests/disk_space.rs b/crates/ui-egui/src/tests/disk_space.rs new file mode 100644 index 0000000000..c2434afedf --- /dev/null +++ b/crates/ui-egui/src/tests/disk_space.rs @@ -0,0 +1,151 @@ +//! The low-disk notice: once per session, when the drive holding the preview cache runs low, with a +//! button to Settings ▸ Performance (where the cache's size limit is). + +use std::time::Duration; + +use lightcraft_engine::disk_space::DiskSpace; +use serde_json::json; + +use crate::headless::Headless; +use crate::state::Dialog; +use crate::{LightcraftApp, Services}; + +const T: Duration = Duration::from_secs(20); +const GB: u64 = 1 << 30; + +fn has(h: &Headless, id: &str) -> bool { + h.app.widgets.iter().any(|(w, _)| w == id) +} + +#[test] +fn a_low_drive_is_told_once_with_a_way_to_the_performance_settings() { + let mut app = LightcraftApp::new(lightcraft_engine::Session::with_demo(), Services::default()); + app.disk_space_notice = true; + let mut h = Headless::new(app, [1300.0, 900.0], 1.0); + h.step(); + assert!(h.app.notices.is_empty(), "no notice before the drive is low"); + // 12 GB free of a 500 GB drive: under 20 GB and under 10 % + assert!(h.app.session.note_disk_space(DiskSpace { available: 12 * GB, total: 500 * GB })); + h.step(); + h.step(); + assert_eq!(h.app.notices.len(), 1, "{:?}", h.app.notices); + let text = h.app.notices[0].clone(); + assert!(text.contains("12.0 GB free of 500 GB") && text.contains("Settings → Performance"), "{text}"); + let inspect = h.request("ui.inspect", json!({}), T); + assert_eq!(inspect["result"]["notices"][0], text.as_str(), "{inspect}"); + assert!(has(&h, "button:noticeSettings") && has(&h, "button:noticeOk"), "the notice offers the settings"); + let r = h.request("ui.clickWidget", json!({"id": "button:noticeSettings"}), T); + assert_eq!(r["ok"], true, "{r}"); + assert!(h.app.notices.is_empty(), "the button dismisses the notice"); + assert_eq!(h.app.ui.dialog, Some(Dialog::Settings { tab: "performance".into() }), "and opens Settings ▸ Performance"); + // lower still, later in the same session: not again + assert!(!h.app.session.note_disk_space(DiskSpace { available: 2 * GB, total: 500 * GB })); + h.step(); + h.step(); + assert!(h.app.notices.is_empty(), "once per session"); +} + +#[test] +fn without_the_desktop_flag_no_notice_is_shown() { + let mut h = Headless::new(LightcraftApp::new(lightcraft_engine::Session::with_demo(), Services::default()), [1300.0, 900.0], 1.0); + assert!(h.app.session.note_disk_space(DiskSpace { available: GB, total: 500 * GB })); + h.step(); + assert!(h.app.notices.is_empty(), "headless sessions and tests don't show it"); +} + +/// A notice still queued when another library opens belongs to the library before: it goes, and +/// the new library's drive is judged on its own. +#[test] +fn a_queued_notice_goes_when_another_library_opens() { + let dir = std::env::temp_dir().join(format!("lc-disk-notice-switch-{}", std::process::id())); + let _ = std::fs::remove_dir_all(&dir); + std::fs::create_dir_all(&dir).unwrap(); + let mut app = LightcraftApp::new(lightcraft_engine::Session::with_demo().with_fs(), Services::default()); + app.disk_space_notice = true; + let mut h = Headless::new(app, [1300.0, 900.0], 1.0); + h.step(); + assert!(h.app.session.note_disk_space(DiskSpace { available: GB, total: 500 * GB })); + h.step(); + assert_eq!(h.app.notices.len(), 1, "{:?}", h.app.notices); + // another library opens before the notice was answered + h.app.session.open_library(&dir, false).unwrap(); + h.step(); + h.step(); + assert!(h.app.notices.iter().all(|n| !n.contains("1.0 GB free of 500 GB")), "the previous library's notice is gone: {:?}", h.app.notices); + let _ = h.app.session.close_library(); + let _ = std::fs::remove_dir_all(&dir); +} + +/// The drive is read again every few minutes whatever is filling the cache, not only while +/// previews are built (CodeRabbit on #3080): thumbnails and the loupe's previews alone can take a +/// drive under the line after the library opened. +#[test] +fn a_drive_that_runs_low_without_a_preview_build_is_told() { + use std::sync::atomic::{AtomicBool, Ordering}; + let dir = std::env::temp_dir().join(format!("lc-disk-notice-later-{}", std::process::id())); + let _ = std::fs::remove_dir_all(&dir); + std::fs::create_dir_all(&dir).unwrap(); + let low = std::sync::Arc::new(AtomicBool::new(false)); + let mut session = lightcraft_engine::Session::with_demo().with_fs(); + let l = low.clone(); + session.set_disk_space_reader(Some(std::sync::Arc::new(move |_: &std::path::Path| { + Some(DiskSpace { available: if l.load(Ordering::Relaxed) { GB } else { 400 * GB }, total: 500 * GB }) + }))); + let mut app = LightcraftApp::new(session, Services::default()); + app.disk_space_notice = true; + let mut h = Headless::new(app, [1300.0, 900.0], 1.0); + h.app.session.open_library(&dir, false).unwrap(); + // (the open's read answered: a pass before that would see no alert yet whatever the drive holds) + assert!(h.step_until(T, |h| !h.app.session.disk_space_reading() && h.app.session.preview_build.is_none())); + assert!(!h.app.session.disk_space_alert_raised()); + for _ in 0..5 { + h.step(); + } + assert!(h.app.notices.is_empty(), "plenty of space at open: {:?}", h.app.notices); + // later, the drive fills up while nothing builds previews + low.store(true, Ordering::Relaxed); + h.app.session.forget_disk_space_check(); + assert!(h.app.session.preview_build.as_ref().is_none_or(|b| !b.running())); + assert!(h.step_until(T, |h| !h.app.notices.is_empty()), "the low drive is told"); + assert!(h.app.notices[0].contains("1.0 GB free of 500 GB"), "{:?}", h.app.notices); + let _ = h.app.session.close_library(); + let _ = std::fs::remove_dir_all(&dir); +} + +/// An idle window still takes the drive's answer and makes the next check (CodeRabbit on #3077): +/// while a read runs the notice logic asks for a pass soon, and otherwise for one when the next +/// check is due; once the alert was raised it asks for nothing. +#[test] +fn an_idle_window_wakes_for_the_disk_check() { + use std::sync::{Arc, Mutex, mpsc}; + let dir = std::env::temp_dir().join(format!("lc-disk-notice-idle-{}", std::process::id())); + let _ = std::fs::remove_dir_all(&dir); + std::fs::create_dir_all(&dir).unwrap(); + let (release, wait) = mpsc::channel::<()>(); + let wait = Mutex::new(wait); + let mut session = lightcraft_engine::Session::with_demo().with_fs(); + session.set_disk_space_reader(Some(Arc::new(move |_: &std::path::Path| { + let _ = wait.lock().unwrap().recv_timeout(Duration::from_secs(10)); + Some(DiskSpace { available: GB, total: 500 * GB }) + }))); + let mut app = LightcraftApp::new(session, Services::default()); + app.disk_space_notice = true; + app.session.open_library(&dir, false).unwrap(); + assert!(app.session.disk_space_reading()); + let ctx = egui::Context::default(); + let pass = |app: &mut LightcraftApp| { + let mut out = ctx.run_ui(egui::RawInput::default(), |_| crate::panels::notices::logic(app, &ctx)); + out.textures_delta.clear(); + out.viewport_output.values().map(|v| v.repaint_delay).min().unwrap_or(Duration::MAX) + }; + assert!(pass(&mut app) <= Duration::from_millis(250), "a pass soon while the read runs"); + release.send(()).unwrap(); + let until = std::time::Instant::now() + T; + while app.notices.is_empty() { + assert!(std::time::Instant::now() < until, "the answer was never taken"); + pass(&mut app); + } + assert!(pass(&mut app) > Duration::from_secs(60), "raised: nothing more to wake for"); + let _ = app.session.close_library(); + let _ = std::fs::remove_dir_all(&dir); +} diff --git a/crates/ui-egui/src/tests/drop_import.rs b/crates/ui-egui/src/tests/drop_import.rs index e5d427d65e..6136ccf93f 100644 --- a/crates/ui-egui/src/tests/drop_import.rs +++ b/crates/ui-egui/src/tests/drop_import.rs @@ -1193,13 +1193,12 @@ fn auto_import_waits_for_a_browse_that_starts_during_its_listing() { let dir = Scratch::new("auto-browse"); let mut h = app(&dir); second_auto_import_listing(&mut h, &dir); - let scan = crate::import::stall_scan_of(&mut h.app, true); + let tx = crate::import::scan_by_hand(&mut h.app, true); apply_auto_import_listing(&mut h); assert!(h.app.import.is_none(), "no Auto Import for the browse to replace"); // the browse finds one photo of its own and imports it - let Ok(tx) = scan.downcast::>() else { panic!("the stalled scan's sender") }; let found = ImportCandidate { path: dir.path("trip/a.png"), ..Default::default() }; - tx.send(ScanOutput { candidates: vec![found], probes: Default::default() }).unwrap(); + tx.send(crate::import::ScanEnd::Done(ScanOutput { candidates: vec![found], probes: Default::default() })).unwrap(); drop(tx); assert!(h.step_until(T, |h| h.app.import.is_some()), "the browse starts its import"); assert!(h.step_until(T, |h| h.app.import.is_none()), "and it ends"); diff --git a/crates/ui-egui/src/tests/lifecycle_037.rs b/crates/ui-egui/src/tests/lifecycle_037.rs index 7b51cb3feb..dbff222044 100644 --- a/crates/ui-egui/src/tests/lifecycle_037.rs +++ b/crates/ui-egui/src/tests/lifecycle_037.rs @@ -2332,3 +2332,43 @@ fn synthetic_modifier_ownership_follows_host_changes_across_frames() { assert_eq!(app.synthetic_mods, egui::Modifiers::NONE); assert!(!app.synthetic_mods_release); } + +/// Background preparation is advanced on every logic pass, the hidden ones too (CodeRabbit on +/// #3077): a minimised window keeps preparing the view's previews, and a shown frame ticks it once. +#[test] +fn background_preparation_advances_while_the_window_is_hidden() { + let mut app = app(); + let ctx = egui::Context::default(); + let hidden = viewport_raw(egui::vec2(800.0, 600.0), false); + let _ = ctx.run_logic(&hidden, |ctx| app.logic(ctx)); + let _ = ctx.run_logic(&hidden, |ctx| app.logic(ctx)); + assert_eq!(app.prep_ticks, 2, "each hidden pass advances it"); + let visible = viewport_raw(egui::vec2(800.0, 600.0), true); + let _ = ctx.run_logic(&visible, |ctx| app.logic(ctx)); + assert_eq!(app.prep_ticks, 3, "a shown pass advances it once"); +} +/// CodeRabbit on #3077/#3080: a background build that couldn't start asks for a frame at its retry +/// time, so the retry happens with no input and no other timed frame. +#[test] +fn a_build_waiting_to_retry_asks_for_a_frame_at_its_time() { + let mut app = app(); + let ctx = egui::Context::default(); + let delays = std::sync::Arc::new(std::sync::Mutex::new(Vec::new())); + let seen = delays.clone(); + ctx.set_request_repaint_callback(move |info| seen.lock().unwrap().push(info.delay)); + let wait = std::time::Duration::from_secs(7); + let hidden = viewport_raw(egui::vec2(800.0, 600.0), false); + // (the first passes ask for frames at once, which would hide a later request) + for _ in 0..3 { + let _ = ctx.run_logic(&hidden, |ctx| app.logic(ctx)); + } + delays.lock().unwrap().clear(); + app.session.prep.retry_at = Some((0, std::time::Instant::now() + wait)); + let _ = ctx.run_logic(&hidden, |ctx| app.logic(ctx)); + let got = delays.lock().unwrap().clone(); + assert!(got.iter().any(|d| *d > std::time::Duration::from_secs(6) && *d <= wait), "a frame at the retry time: {got:?}"); + app.session.prep.retry_at = None; + delays.lock().unwrap().clear(); + let _ = ctx.run_logic(&hidden, |ctx| app.logic(ctx)); + assert!(!delays.lock().unwrap().iter().any(|d| *d > std::time::Duration::from_secs(6) && *d <= wait), "none without a retry waiting"); +} diff --git a/crates/ui-egui/src/tests/preview_limit.rs b/crates/ui-egui/src/tests/preview_limit.rs index ce5a5aba42..ddd6c74790 100644 --- a/crates/ui-egui/src/tests/preview_limit.rs +++ b/crates/ui-egui/src/tests/preview_limit.rs @@ -1290,6 +1290,38 @@ mod in_the_loupe { assert!(window[0].1.right() >= canvas.right() - 0.5, "the Before window stops at the wipe line: {:?}", window[0].1); } + // Given Before and After side by side at 1:1 with `\` held, the After pane is still the view + // shown: a Before window drawn while the After pane shows a stand-in does not make the loupe + // say the render is on screen + #[test] + fn holding_backslash_side_by_side_the_before_window_is_not_the_shown_render() { + let (_fx, mut h, _) = detail_in([1800.0, 900.0]); + h.app.ui.before_after = crate::state::BeforeAfter::SideBySide; + h.request("engine.execute", json!({"command": "view.zoom100"}), T); + h.settle(SETTLE); + h.events.push(egui::Event::Key { + key: egui::Key::Backslash, + physical_key: Some(egui::Key::Backslash), + pressed: true, + repeat: false, + modifiers: egui::Modifiers::NONE, + }); + h.settle(SETTLE); + // (the key's own shortcut switches to the Before view; side by side is chosen again with the key down) + h.app.ui.before_after = crate::state::BeforeAfter::SideBySide; + h.settle(SETTLE); + assert!(h.app.renderer.textures.contains_key(&Slot::RegionBefore), "the Before window is rendered"); + // the After pane back to a stand-in: no whole-frame render and no window of its own + h.app.renderer.textures.remove(&Slot::Main); + h.app.renderer.textures.remove(&Slot::Region); + h.step(); + assert!(!drawn(&h, Slot::RegionBefore).is_empty(), "the Before window is painted"); + // (the step must not bring the After pane's render back, or the check below proves nothing) + assert!(!h.app.renderer.textures.contains_key(&Slot::Main), "the whole-frame render came back during the step"); + assert!(!h.app.renderer.textures.contains_key(&Slot::Region), "the After window came back during the step"); + assert_ne!(h.app.loupe_shown.map(|l| l.1), Some("render"), "the After pane shows a stand-in, not the render"); + } + // Given Before and After side by side at 1:1, each window is painted inside its own pane: its // margin (a window is wider than what its pane shows) does not spill into the neighbour #[test] @@ -1306,6 +1338,33 @@ mod in_the_loupe { assert!(after.iter().all(|(_, clip, _)| clip.left() >= canvas.center().x), "the After window spills into the Before pane: {after:?}"); } + // Given Before and After side by side at 1:1 with `\` held, both panes stay, so each window + // stays clipped to its own pane: the Before window's margin does not cover the After pane + #[test] + fn holding_backslash_side_by_side_keeps_each_window_in_its_pane() { + let (_fx, mut h, _) = detail_in([1800.0, 900.0]); + h.app.ui.before_after = crate::state::BeforeAfter::SideBySide; + h.request("engine.execute", json!({"command": "view.zoom100"}), T); + h.settle(SETTLE); + h.events.push(egui::Event::Key { + key: egui::Key::Backslash, + physical_key: Some(egui::Key::Backslash), + pressed: true, + repeat: false, + modifiers: egui::Modifiers::NONE, + }); + h.settle(SETTLE); + // (the key's own shortcut switches to the Before view; side by side is chosen again with the key down) + h.app.ui.before_after = crate::state::BeforeAfter::SideBySide; + h.settle(SETTLE); + h.step(); + let canvas = h.app.canvas_rect.unwrap(); + let (before, after) = (drawn(&h, Slot::RegionBefore), drawn(&h, Slot::Region)); + assert!(!before.is_empty() && !after.is_empty()); + assert!(before.iter().all(|(_, clip, _)| clip.right() <= canvas.center().x), "the Before window spills into the After pane: {before:?}"); + assert!(after.iter().all(|(_, clip, _)| clip.left() >= canvas.center().x), "the After window spills into the Before pane: {after:?}"); + } + // Given a Before and an After with the same look (an unedited photo), both windows are kept // apart: the job key is the same, and one tile must not replace the other #[test] @@ -1583,3 +1642,16 @@ mod in_the_loupe { assert_ne!(after.settings, before.settings); } } + +// Build Standard-Sized Previews on Automatic: the loupe's size once one was drawn (the engine's standard), the +// standard 2560 px in a fresh session, not the engine's 2048 px minimum a fit view would render again over +// (CodeRabbit on #3077); a chosen limit as is +#[test] +fn standard_build_keeps_the_standard_size_before_a_loupe_was_drawn() { + use crate::menubar::standard_build_params; + use serde_json::json; + assert_eq!(standard_build_params(&with_limit(0), 0), json!({"size": "standard", "edge": STANDARD_PREVIEW_EDGE})); + assert_eq!(standard_build_params(&with_limit(0), 2400), json!({"size": "standard"})); + assert_eq!(standard_build_params(&with_limit(3840), 0), json!({"size": "standard", "edge": 3840})); + assert_eq!(standard_build_params(&with_limit(3840), 2400), json!({"size": "standard", "edge": 3840})); +} diff --git a/crates/ui-egui/src/tests/scan_probes.rs b/crates/ui-egui/src/tests/scan_probes.rs new file mode 100644 index 0000000000..04c7ad10d0 --- /dev/null +++ b/crates/ui-egui/src/tests/scan_probes.rs @@ -0,0 +1,204 @@ +//! A scan takes the probes of earlier scans from the session (`ScanInput::new`) and hands them back +//! however it ends: finished, stopped by another folder or by ✕, or unable to list its folder, so a later +//! import doesn't read those files again. + +use std::time::Duration; + +use crate::LightcraftApp; +use crate::headless::Headless; +use crate::import::ScanEnd; + +const T: Duration = Duration::from_secs(10); +const KEPT: &str = "/nowhere/kept.cr3"; + +fn demo() -> Headless { + let services = crate::Services { png: None, ..Default::default() }; + Headless::new(LightcraftApp::new(lightcraft_engine::Session::with_demo(), services), [1000.0, 700.0], 1.0) +} + +fn probes() -> std::collections::HashMap { + [(KEPT.to_string(), lightcraft_engine::media::ProbeInfo { file_size: 7, ..Default::default() })].into_iter().collect() +} + +fn kept(h: &Headless) -> bool { + h.app.session.import_probes.get(KEPT).is_some_and(|p| p.file_size == 7) +} + +fn failed_toast(h: &Headless) -> bool { + h.app.ui.toast.as_ref().is_some_and(|t| t.0.contains("Scan failed")) +} + +fn cancel_scan(h: &mut Headless) { + let id = h.app.session.activity.list().iter().find(|t| t.kind == "scan").map(|t| t.id).expect("the scan's row"); + h.app.session.activity.cancel(id).expect("the scan can be cancelled"); +} + +#[test] +fn a_browse_replaced_by_another_folder_hands_back_its_probes() { + let mut h = demo(); + let base = std::env::temp_dir().join(format!("lc-ui-scan-probes-{}", std::process::id())); + let (a, b) = (base.join("a"), base.join("b")); + std::fs::create_dir_all(&a).unwrap(); + std::fs::create_dir_all(&b).unwrap(); + h.app.session.import_probes = probes(); + crate::import::browse(&mut h.app, &a.to_string_lossy(), None).unwrap(); + assert!(h.app.session.import_probes.is_empty(), "the first browse took the probes"); + // another folder at once: the first browse is stopped (or its result left unread) + crate::import::browse(&mut h.app, &b.to_string_lossy(), None).unwrap(); + assert!(h.step_until(T, |h| h.app.scan.is_none() && kept(h)), "the first browse's probes came back"); + let _ = std::fs::remove_dir_all(&base); +} + +#[test] +fn a_browse_that_cannot_list_its_folder_hands_back_its_probes() { + let mut h = demo(); + let tx = crate::import::scan_by_hand(&mut h.app, true); + tx.send(ScanEnd::Stopped { probes: probes(), failed: true }).unwrap(); + h.step(); + assert!(h.app.scan.is_none()); + assert!(kept(&h), "the probes came back"); + assert!(failed_toast(&h), "the failure is reported: {:?}", h.app.ui.toast); +} + +#[test] +fn a_scan_closed_with_its_cross_hands_back_its_probes_when_the_worker_ends() { + let mut h = demo(); + let tx = crate::import::scan_by_hand(&mut h.app, false); + cancel_scan(&mut h); + h.step(); + assert!(h.app.scan.is_none(), "✕ closes the scan at once"); + // the worker ends later, with the probes it took + tx.send(ScanEnd::Done(lightcraft_engine::import::ScanOutput { candidates: Vec::new(), probes: probes() })).unwrap(); + assert!(h.step_until(T, kept), "the probes came back"); + assert!(h.app.ui.dialog.is_none(), "no review opened"); + assert!(!failed_toast(&h)); +} + +#[test] +fn a_cancelled_scan_whose_worker_ends_without_a_result_is_not_a_failure() { + let mut h = demo(); + let tx = crate::import::scan_by_hand(&mut h.app, true); + cancel_scan(&mut h); + drop(tx); + h.step(); + assert!(h.app.scan.is_none()); + assert!(!failed_toast(&h), "a cancelled scan says nothing: {:?}", h.app.ui.toast); +} + +/// Two stopped scans hand back a probe of one file: the later scan read it again, so its probe stays, +/// whichever worker ends first (CodeRabbit on #3577: the older one won when its worker ended first, +/// and its content hash is stale for a file changed at the same size). +#[test] +fn the_later_stopped_scans_probe_wins_whichever_worker_ends_first() { + let probe = |hash: &str| { + let p = lightcraft_engine::media::ProbeInfo { file_size: 7, content_hash: Some(hash.into()), ..Default::default() }; + [(KEPT.to_string(), p)].into_iter().collect() + }; + for later_ends_first in [false, true] { + let mut h = demo(); + let older = crate::import::scan_by_hand(&mut h.app, true); + cancel_scan(&mut h); + h.step(); + let newer = crate::import::scan_by_hand(&mut h.app, true); + cancel_scan(&mut h); + h.step(); + assert!(h.app.scan.is_none(), "both stopped"); + let a = ScanEnd::Stopped { probes: probe("older"), failed: false }; + let b = ScanEnd::Stopped { probes: probe("newer"), failed: false }; + if later_ends_first { + newer.send(b).unwrap(); + h.step(); + older.send(a).unwrap(); + } else { + older.send(a).unwrap(); + h.step(); + newer.send(b).unwrap(); + } + h.step(); + let hash = h.app.session.import_probes.get(KEPT).and_then(|p| p.content_hash.clone()); + assert_eq!(hash.as_deref(), Some("newer"), "later worker ends first: {later_ends_first}"); + } +} + +/// Browsing reads the disk only on its worker (CodeRabbit on #3077): a folder whose resolution is held (a link on +/// a stalled network mount) still lets `browse` return at once, the view shows the folder as given, and the scan +/// waits; once resolved, the folder is read. +#[test] +fn a_browse_whose_folder_resolves_slowly_returns_at_once_and_waits() { + let mut h = demo(); + let dir = std::env::temp_dir().join(format!("lc-ui-browse-held-{}", std::process::id())); + std::fs::create_dir_all(&dir).unwrap(); + // (spelled with `sub/..`: the worker is handed the path as given, not the view's spelling of it) + let given = dir.join("sub").join("..").to_string_lossy().to_string(); + let spelled = lightcraft_engine::import::browse_spelling(&dir).unwrap().to_string_lossy().trim_end_matches(['/', '\\']).to_string(); + crate::import::HELD_RESOLVES.lock().unwrap().push(given.clone()); + let start = std::time::Instant::now(); + let r = crate::import::browse(&mut h.app, &given, None).unwrap(); + assert!(start.elapsed() < Duration::from_secs(2), "returned after {:?}", start.elapsed()); + assert_eq!(r["scanning"], true, "{r}"); + assert_eq!(h.app.session.browse.as_ref().map(|b| b.path.clone()), Some(spelled), "shown as spelled meanwhile"); + for _ in 0..5 { + h.step(); + } + assert!(h.app.scan.is_some(), "still waiting for the folder"); + crate::import::HELD_RESOLVES.lock().unwrap().retain(|p| *p != given); + assert!(h.step_until(T, |h| h.app.scan.is_none()), "read once resolved"); + assert!(!failed_toast(&h), "{:?}", h.app.ui.toast); + let _ = std::fs::remove_dir_all(&dir); +} + +/// A path that isn't a folder is found out by the worker, not by the window's thread: `browse` returns, and the scan +/// then ends as failed. +#[test] +fn a_browse_of_a_missing_folder_fails_on_its_worker() { + let mut h = demo(); + let gone = std::env::temp_dir().join(format!("lc-ui-browse-gone-{}", std::process::id())).join("not-there"); + assert!(crate::import::browse(&mut h.app, &gone.to_string_lossy(), None).is_ok()); + assert!(h.step_until(T, |h| h.app.scan.is_none())); + assert!(failed_toast(&h), "the failure is reported: {:?}", h.app.ui.toast); +} + +/// The worker resolves the path as given (CodeRabbit on #3055 / #3077 / #3080): on Unix `photos/link/../select`, with +/// `link` pointing at `real/day`, is `real/select`, not the `photos/select` that taking `..` out by name would read. +#[cfg(unix)] +#[test] +fn a_browse_through_a_link_and_dot_dot_reads_the_folder_the_system_names() { + let mut h = demo(); + let base = std::env::temp_dir().join(format!("lc-ui-browse-link-{}", std::process::id())); + let (day, real_select, photos) = (base.join("real/day"), base.join("real/select"), base.join("photos")); + for d in [&day, &real_select, &photos.join("select")] { + std::fs::create_dir_all(d).unwrap(); + } + std::os::unix::fs::symlink(&day, photos.join("link")).unwrap(); + let given = photos.join("link").join("..").join("select"); + crate::import::browse(&mut h.app, &given.to_string_lossy(), None).unwrap(); + assert!(h.step_until(T, |h| h.app.scan.is_none()), "the browse ended"); + assert!(!failed_toast(&h), "{:?}", h.app.ui.toast); + let want = std::fs::canonicalize(&real_select).unwrap(); + let shown = h.app.session.browse.as_ref().map(|b| std::fs::canonicalize(&b.path).unwrap()); + assert_eq!(shown, Some(want), "the folder the system names, not photos/select"); + let _ = std::fs::remove_dir_all(&base); +} + +/// Browsing a folder that is being read again keeps its progress, but not through `..` (CodeRabbit on #3583): past +/// a link, `x/link/../y` can be another folder than the `x/y` it is spelled as, so the worker reads it again. +#[test] +fn a_browse_through_dot_dot_reads_again_while_its_spelling_is_being_read() { + let mut h = demo(); + let dir = std::env::temp_dir().join(format!("lc-ui-browse-dotdot-{}", std::process::id())); + std::fs::create_dir_all(dir.join("sub")).unwrap(); + let plain = dir.to_string_lossy().to_string(); + crate::import::HELD_RESOLVES.lock().unwrap().push(plain.clone()); + let row = |h: &Headless| h.app.session.activity.list().iter().find(|t| t.kind == "scan").map(|t| t.id); + crate::import::browse(&mut h.app, &plain, None).unwrap(); + let first = row(&h); + assert!(first.is_some(), "the browse's row"); + crate::import::browse(&mut h.app, &plain, None).unwrap(); + assert_eq!(row(&h), first, "the same folder again: the same read"); + let given = dir.join("sub").join("..").to_string_lossy().to_string(); + crate::import::browse(&mut h.app, &given, None).unwrap(); + assert_ne!(row(&h), first, "through `..`: read again"); + crate::import::HELD_RESOLVES.lock().unwrap().retain(|p| *p != plain); + assert!(h.step_until(T, |h| h.app.scan.is_none()), "the browse ended"); + let _ = std::fs::remove_dir_all(&dir); +} diff --git a/docs/control-protocol.md b/docs/control-protocol.md index 43ff7a7175..4f25582349 100644 --- a/docs/control-protocol.md +++ b/docs/control-protocol.md @@ -89,7 +89,7 @@ filmstrip is visible. | `engine.commands` | — | Engine + UI commands: id, label, menu, shortcut, params doc, enabled | | `ui.menu.list` | — | Menu entries (flat: id, label, menu path, shortcut, enabled) | | `ui.menu.tree` | — | The menu bar as shown (File … Help): items `{id, params?, label, shortcut?, enabled, checked?}`, separators, submenus — the model behind the native macOS menu bar and the in-window menus | -| `ui.inspect` | — | UI state, window, `loupe` (`source`, `pending`, `failed` = why its render failed, else `null`, and `region: {full, window, pending}` when a zoomed view also renders the window on screen at no more than 100 % — `full` is the frame it is cut from, `window` its `[x, y, w, h]` — and `regionBefore` for the Before side of a Before/After view), canvas/image rects, `scroll: {grid, filmstrip}` (scroll offsets in points, `null` until drawn), active photo, selection, perf (`frameMs` = layout, `logicMs` = per-frame logic before it, `updateMs` = both, `maxUpdateMs`, `fps`, render queue …, `gpu` = adapter in use, `gpuReason` = why renders don't use the GPU, `gpuFallback` = latest render redone on the CPU and why, `gpuWindowDevice` / `screenTextures` / `mainOnGpu` = renders run on the window's device / can be drawn from the GPU / the loupe shows one now — see `docs/gpu-pipeline.md`), status, memory (bytes per cache, see `library.memory`; plus stage caches — `budgetBytes`, `trimmed`, `sharedSourceBytes` (the one device copy of a big original) — and textures), `export: {running: {total, done, current} \| null, last}`, `notices` (warnings waiting to be shown, e.g. a damaged settings file; OK = `button:noticeOk`), `quitPrompt` (why quitting was stopped: tasks that would be cut short are running — `button:quitAnyway` cancels them and quits, `button:quitCancel` keeps working — or unsaved changes: `button:quitRetry` / `button:quitAnyway` / `button:quitCancel`), `import: {done, total, imported, cancelled} \| null` (an import runs on a worker thread), `tasks` (other background work: `Find Missing Photos`, `Auto Import`), `activity` (every long-running task in flight as the activity stack in the top-left corner shows it, same shape as `activity.list`; a row appears once its task is 0.5 s old, ids `activity:row:`, ✕ = `activity:cancel:`, `activity:more` expands past 3 rows), `fileDialogs` (commands waiting on a native file dialog, which runs off the UI thread; they run again with the answer when it closes), `copied` (the text the UI last put on the clipboard, `null` until something is copied) | +| `ui.inspect` | — | UI state, window, `loupe` (`source`, `pending`, `failed` = why its render failed, else `null`, and `region: {full, window, pending}` when a zoomed view also renders the window on screen at no more than 100 % — `full` is the frame it is cut from, `window` its `[x, y, w, h]` — and `regionBefore` for the Before side of a Before/After view), canvas/image rects, `scroll: {grid, filmstrip}` (scroll offsets in points, `null` until drawn), active photo, selection, perf (`frameMs` = layout, `logicMs` = per-frame logic before it, `updateMs` = both, `maxUpdateMs`, `fps`, render queue …, `gpu` = adapter in use, `gpuReason` = why renders don't use the GPU, `gpuFallback` = latest render redone on the CPU and why, `gpuWindowDevice` / `screenTextures` / `mainOnGpu` = renders run on the window's device / can be drawn from the GPU / the loupe shows one now — see `docs/gpu-pipeline.md`, `loupeSteps` = how the loupe showed each photo it was stepped to, measured on the frames it drew: `{steps: [{photo, anyMs, anySource, fullMs, fullSource, sharpMs, dwellMs}], total, current}` — ms from the step to the first frame with anything of the photo, with a full-size picture (`embedded` camera JPEG, `cached` or `render`), and with LightCraft's own render; `dwellMs` is how long it stayed, the latest 256 steps; `total` counts every step finished since launch, a cursor that keeps rising when old steps leave the list; `thumbsMade` = photos given a grid thumbnail since launch (never lowered when old thumbnails are evicted, unlike `thumbTextures`); `previewCache` = the rendered-preview cache: `memEntries` / `memBytes` in memory, the disk cache's `diskHits`, `diskMisses`, `diskWrites` and `diskBudget` (bytes; all `null` without a library on disk), and `loupeEdge` (px, the loupe's longest edge, the size standard previews are built at)), status, memory (bytes per cache, see `library.memory`; plus stage caches — `budgetBytes`, `trimmed`, `sharedSourceBytes` (the one device copy of a big original) — and textures), `export: {running: {total, done, current} \| null, last}`, `notices` (warnings waiting to be shown, e.g. a damaged settings file, or once per session the drive holding the preview cache running low; OK = `button:noticeOk`, and the low-drive notice's `button:noticeSettings` opens Settings ▸ Performance), `quitPrompt` (why quitting was stopped: tasks that would be cut short are running — `button:quitAnyway` cancels them and quits, `button:quitCancel` keeps working — or unsaved changes: `button:quitRetry` / `button:quitAnyway` / `button:quitCancel`), `import: {done, total, imported, cancelled} \| null` (an import runs on a worker thread), `tasks` (other background work: `Find Missing Photos`, `Auto Import`), `activity` (every long-running task in flight as the activity stack in the top-left corner shows it, same shape as `activity.list`; a row appears once its task is 0.5 s old, ids `activity:row:`, ✕ = `activity:cancel:`, `activity:more` expands past 3 rows), `fileDialogs` (commands waiting on a native file dialog, which runs off the UI thread; they run again with the answer when it closes), `copied` (the text the UI last put on the clipboard, `null` until something is copied) | | `ui.widgets` | `{filter?}` | On-screen widgets `{id, rect: [x, y, w, h]}` (screen points) | | `ui.clickWidget` / `ui.dragWidget` / `ui.hoverWidget` | `{id, count?, button?, fx?, fy?}` (`button: "right"` right-clicks; `count` is capped at 16) / `{id, toX?, toY?, dx?, dy?, steps?}` (steps are capped at 1000) / `{id, fx?, fy?}` | Real egui input on a widget (hover: the pointer rests on it, e.g. for preset/profile previews). `fx`/`fy` are fractions of the registered rect and may lie outside `[0,1]`; the resulting point must still be inside egui's clipped interaction rect, and a non-finite fraction or resolved point is rejected before any input is queued. Geometry comes from the latest completed layout; if the widget moves before the first event or before a press, or its latest clipped rect no longer contains the point, the request fails with an error instead of clicking another widget or retargeting. A target below an ordinary modal dialog may succeed as a no-op or a backdrop click, because egui routes that input to the dialog; a target covered by a non-modal overlay fails. The blocking library-problem/launch-list window is explicit: its dim takes the modal layer, but a background target fails instead of reporting a no-op success, and only that window's own controls (`button:library…`) pass. Its creation frame is shielded and redrawn with empty input like a prompt's, so background text/pointer input cannot act while the new modal areas settle. A quit prompt or settings notice is a real egui modal scope with a foreground backdrop. Only its own buttons (`button:quit…`, `button:noticeOk`) accept a `ui.clickWidget`; background targets fail, and background keystrokes, text/clipboard input, wheel/zoom input and AccessKit actions are dropped before panels see them. The prompt itself takes Tab/Shift+Tab, Enter/Space on the focused button, and Escape/Enter for the safe default Cancel; AccessKit Focus/Click works when targeted at a prompt button. Its first creation frame (including a Tasks→Unsaved or notice→next-notice transition) is shielded and redrawn with empty input, so a press/release in that frame, or a held press from before it, cannot act underneath it; a synthetic drag already in progress is canceled. Once settled, raw `ui.click`/`ui.drag`/`ui.pointer` and backdrop clicks are still delivered to the prompt backdrop as no-ops, and the prompt's own button clicks work normally. The area/layer check cannot see a different widget covering the point in the same layer. A `ui.clickWidget` whose widget disappears before its release also fails, after canceling the pointer press without a click or drag (the same rules apply when a click spans several host frames) | | `ui.move` / `ui.click` / `ui.drag` | `{x, y, count?, button?}` / `{x, y, toX, toY, steps?}` | Raw pointer input, screen points (the same click and step caps apply). In every pointer and scroll gesture a number other than `count` and `steps` must lie between −1,000,000 and 1,000,000, or the request is refused and queues nothing | diff --git a/docs/gaps.md b/docs/gaps.md index 566786a5f0..e89a6162c0 100644 --- a/docs/gaps.md +++ b/docs/gaps.md @@ -1,6 +1,6 @@ # Gaps: where LightCraft falls short of Lightroom -> **Last reviewed:** 2026-10-11 · **Last updated:** 2026-11-05 · **Change:** minor (Contrast and Blacks calibrated in process V4, #523; PERF-STEP-STALL: flags, ratings and edits never wait for the drive; before: gap 1: the Canon EOS D60 is the first chart-calibrated model, its matrices from our own ColorChecker shots give its white balance a Kelvin scale; gap 20: Canon CRW decodes; LR-LIB-VIRTUALCOPY: default number exhaustion is validated before copy creation; gap 8: lens profile work is tracked in #329; gap 8: lensfun's lens database offered on the first launch and downloaded on request, looked up for the library; gap 30: the Info panel shows the selected photos' shared metadata and marks mixed fields; gap 28: ⌘⌥⇧C / ⌘⌥⇧V copy and paste metadata; gap 13: before a Lightroom catalog import the app asks merge or new library, #2672; gap 28: F8 shows / hides the right panel; import review album hierarchy, #2362; gap 20: Minolta MRW decodes; gap 10: 557 models verified with CRW and MRW; gap 15: Print Page Options — page numbers, page info, crop marks; gap 28: `\` shows / hides the Filter Bar in the grids; gap 21: Indonesian interface shipped, every message but What's New; gap 27: the Windows x64 installer verified on Windows 11; the Settings → Apps uninstall confirmation remains; gap 28: ⌘L Enable Filters switches the filter off and on; gap 15: Japanese photo and postcard paper sizes in the Print page setup and PDF; typed GPS input now rejects malformed coordinates before metadata changes; LR-MAP-TRACKLOG decodes GPX character references in coordinates, elevations and timestamps; malformed references error; CDATA stays literal; ranked gaps unchanged; gap 8: lens profile work is tracked in #329; gap 30: the Info panel shows the selected photos' shared metadata and marks mixed fields; gap 28: ⌘⌥⇧C / ⌘⌥⇧V copy and paste metadata; gap 13: before a Lightroom catalog import the app asks merge or new library, #2672; gap 28: F8 shows / hides the right panel; import review album hierarchy, #2362; gap 20: Minolta MRW decodes; gap 10: 557 models verified with CRW and MRW; gap 15: Print Page Options — page numbers, page info, crop marks; gap 28: `\` shows / hides the Filter Bar in the grids; gap 21: Indonesian interface shipped, every message but What's New; gap 27: the Windows x64 installer verified on Windows 11; the Settings → Apps uninstall confirmation remains; gap 28: ⌘L Enable Filters switches the filter off and on; gap 15: Japanese photo and postcard paper sizes in the Print page setup and PDF; import: checked rename-sequence exhaustion; import review: remembered thumbnail size and density-aware previews (#2237, part of #2056); pre-import loupe/keyboard culling remains open; no ranked gap closes; gap 8: lens profile work is tracked in #329; gap 30: the Info panel shows the selected photos' shared metadata and marks mixed fields; gap 28: ⌘⌥⇧C / ⌘⌥⇧V copy and paste metadata; gap 13: before a Lightroom catalog import the app asks merge or new library, #2672; gap 28: F8 shows / hides the right panel; import review album hierarchy, #2362; gap 20: Minolta MRW decodes; gap 10: 557 models verified with CRW and MRW; gap 15: Print Page Options — page numbers, page info, crop marks; gap 28: `\` shows / hides the Filter Bar in the grids; gap 21: Indonesian interface shipped, every message but What's New; gap 27: the Windows x64 installer verified on Windows 11; the Settings → Apps uninstall confirmation remains; gap 28: ⌘L Enable Filters switches the filter off and on; gap 15: Japanese photo and postcard paper sizes in the Print page setup and PDF; gap 9: CI runs the tests on Linux x64 since #2667; macOS, Windows and the GPU equivalence tests still don't; gap 8: lens profile work is tracked in #329; gap 30: the Info panel shows the selected photos' shared metadata and marks mixed fields; gap 28: ⌘⌥⇧C / ⌘⌥⇧V copy and paste metadata; gap 13: before a Lightroom catalog import the app asks merge or new library, #2672; gap 28: F8 shows / hides the right panel; import review album hierarchy, #2362; gap 20: Minolta MRW decodes; gap 10: 557 models verified with CRW and MRW; gap 15: Print Page Options — page numbers, page info, crop marks; gap 28: `\` shows / hides the Filter Bar in the grids; gap 21: Indonesian interface shipped, every message but What's New; gap 27: the Windows x64 installer verified on Windows 11; the Settings → Apps uninstall confirmation remains; gap 28: ⌘L Enable Filters switches the filter off and on; gap 15: Japanese photo and postcard paper sizes in the Print page setup and PDF; gap 8: Sony ILCE-7RM6 in-file distortion conversion extends to the APS-C ILCE-6700; LR-META-EXIF-FULL rejects extra EXIF clock components and preserves date fallbacks; gap 8: lens profile work is tracked in #329; gap 30: the Info panel shows the selected photos' shared metadata and marks mixed fields; gap 28: ⌘⌥⇧C / ⌘⌥⇧V copy and paste metadata; gap 13: before a Lightroom catalog import the app asks merge or new library, #2672; gap 28: F8 shows / hides the right panel; import review album hierarchy, #2362; gap 20: Minolta MRW decodes; gap 10: 557 models verified with CRW and MRW; gap 15: Print Page Options — page numbers, page info, crop marks; gap 28: `\` shows / hides the Filter Bar in the grids; gap 21: Indonesian interface shipped, every message but What's New; gap 27: the Windows x64 installer verified on Windows 11; the Settings → Apps uninstall confirmation remains; gap 28: ⌘L Enable Filters switches the filter off and on; gap 15: Japanese photo and postcard paper sizes in the Print page setup and PDF; JPEG ICC APP2 sequence validation; gap 8: lens profile work is tracked in #329; gap 30: the Info panel shows the selected photos' shared metadata and marks mixed fields; gap 28: ⌘⌥⇧C / ⌘⌥⇧V copy and paste metadata; gap 13: before a Lightroom catalog import the app asks merge or new library, #2672; gap 28: F8 shows / hides the right panel; import review album hierarchy, #2362; gap 20: Minolta MRW decodes; gap 10: 557 models verified with CRW and MRW; gap 15: Print Page Options — page numbers, page info, crop marks; gap 28: `\` shows / hides the Filter Bar in the grids; gap 21: Indonesian interface shipped, every message but What's New; gap 27: the Windows x64 installer verified on Windows 11; the Settings → Apps uninstall confirmation remains; gap 28: ⌘L Enable Filters switches the filter off and on; gap 15: Japanese photo and postcard paper sizes in the Print page setup and PDF; gap 8: lens profile work is tracked in #329; gap 30: the Info panel shows the selected photos' shared metadata and marks mixed fields; gap 28: ⌘⌥⇧C / ⌘⌥⇧V copy and paste metadata; gap 13: before a Lightroom catalog import the app asks merge or new library, #2672; gap 28: F8 shows / hides the right panel; import review album hierarchy, #2362; gap 20: Minolta MRW decodes; gap 10: 557 models verified with CRW and MRW; gap 15: Print Page Options — page numbers, page info, crop marks; gap 28: `\` shows / hides the Filter Bar in the grids; gap 21: Indonesian interface shipped, every message but What's New; gap 27: the Windows x64 installer verified on Windows 11; the Settings → Apps uninstall confirmation remains; gap 28: ⌘L Enable Filters switches the filter off and on; gap 15: Japanese photo and postcard paper sizes in the Print page setup and PDF; gap 24: extreme brush segments skip the intermediate dabs; **Last reviewed:** 2026-10-11 · **Last updated:** 2026-10-20 · **Change:** minor (gap 15: Print Page Options — page numbers, page info, crop marks) +> **Last reviewed:** 2026-10-11 · **Last updated:** 2026-11-05 · **Change:** minor (gap 26: culling uses built previews, background preparation per mode; before: Contrast and Blacks calibrated in process V4, #523; PERF-STEP-STALL: flags, ratings and edits never wait for the drive; gap 1: the Canon EOS D60 is the first chart-calibrated model, its matrices from our own ColorChecker shots give its white balance a Kelvin scale; gap 20: Canon CRW decodes; LR-LIB-VIRTUALCOPY: default number exhaustion is validated before copy creation; gap 8: lens profile work is tracked in #329; gap 8: lensfun's lens database offered on the first launch and downloaded on request, looked up for the library; gap 30: the Info panel shows the selected photos' shared metadata and marks mixed fields; gap 28: ⌘⌥⇧C / ⌘⌥⇧V copy and paste metadata; gap 13: before a Lightroom catalog import the app asks merge or new library, #2672; gap 28: F8 shows / hides the right panel; import review album hierarchy, #2362; gap 20: Minolta MRW decodes; gap 10: 557 models verified with CRW and MRW; gap 15: Print Page Options — page numbers, page info, crop marks; gap 28: `\` shows / hides the Filter Bar in the grids; gap 21: Indonesian interface shipped, every message but What's New; gap 27: the Windows x64 installer verified on Windows 11; the Settings → Apps uninstall confirmation remains; gap 28: ⌘L Enable Filters switches the filter off and on; gap 15: Japanese photo and postcard paper sizes in the Print page setup and PDF; typed GPS input now rejects malformed coordinates before metadata changes; LR-MAP-TRACKLOG decodes GPX character references in coordinates, elevations and timestamps; malformed references error; CDATA stays literal; ranked gaps unchanged; gap 8: lens profile work is tracked in #329; gap 30: the Info panel shows the selected photos' shared metadata and marks mixed fields; gap 28: ⌘⌥⇧C / ⌘⌥⇧V copy and paste metadata; gap 13: before a Lightroom catalog import the app asks merge or new library, #2672; gap 28: F8 shows / hides the right panel; import review album hierarchy, #2362; gap 20: Minolta MRW decodes; gap 10: 557 models verified with CRW and MRW; gap 15: Print Page Options — page numbers, page info, crop marks; gap 28: `\` shows / hides the Filter Bar in the grids; gap 21: Indonesian interface shipped, every message but What's New; gap 27: the Windows x64 installer verified on Windows 11; the Settings → Apps uninstall confirmation remains; gap 28: ⌘L Enable Filters switches the filter off and on; gap 15: Japanese photo and postcard paper sizes in the Print page setup and PDF; import: checked rename-sequence exhaustion; import review: remembered thumbnail size and density-aware previews (#2237, part of #2056); pre-import loupe/keyboard culling remains open; no ranked gap closes; gap 8: lens profile work is tracked in #329; gap 30: the Info panel shows the selected photos' shared metadata and marks mixed fields; gap 28: ⌘⌥⇧C / ⌘⌥⇧V copy and paste metadata; gap 13: before a Lightroom catalog import the app asks merge or new library, #2672; gap 28: F8 shows / hides the right panel; import review album hierarchy, #2362; gap 20: Minolta MRW decodes; gap 10: 557 models verified with CRW and MRW; gap 15: Print Page Options — page numbers, page info, crop marks; gap 28: `\` shows / hides the Filter Bar in the grids; gap 21: Indonesian interface shipped, every message but What's New; gap 27: the Windows x64 installer verified on Windows 11; the Settings → Apps uninstall confirmation remains; gap 28: ⌘L Enable Filters switches the filter off and on; gap 15: Japanese photo and postcard paper sizes in the Print page setup and PDF; gap 9: CI runs the tests on Linux x64 since #2667; macOS, Windows and the GPU equivalence tests still don't; gap 8: lens profile work is tracked in #329; gap 30: the Info panel shows the selected photos' shared metadata and marks mixed fields; gap 28: ⌘⌥⇧C / ⌘⌥⇧V copy and paste metadata; gap 13: before a Lightroom catalog import the app asks merge or new library, #2672; gap 28: F8 shows / hides the right panel; import review album hierarchy, #2362; gap 20: Minolta MRW decodes; gap 10: 557 models verified with CRW and MRW; gap 15: Print Page Options — page numbers, page info, crop marks; gap 28: `\` shows / hides the Filter Bar in the grids; gap 21: Indonesian interface shipped, every message but What's New; gap 27: the Windows x64 installer verified on Windows 11; the Settings → Apps uninstall confirmation remains; gap 28: ⌘L Enable Filters switches the filter off and on; gap 15: Japanese photo and postcard paper sizes in the Print page setup and PDF; gap 8: Sony ILCE-7RM6 in-file distortion conversion extends to the APS-C ILCE-6700; LR-META-EXIF-FULL rejects extra EXIF clock components and preserves date fallbacks; gap 8: lens profile work is tracked in #329; gap 30: the Info panel shows the selected photos' shared metadata and marks mixed fields; gap 28: ⌘⌥⇧C / ⌘⌥⇧V copy and paste metadata; gap 13: before a Lightroom catalog import the app asks merge or new library, #2672; gap 28: F8 shows / hides the right panel; import review album hierarchy, #2362; gap 20: Minolta MRW decodes; gap 10: 557 models verified with CRW and MRW; gap 15: Print Page Options — page numbers, page info, crop marks; gap 28: `\` shows / hides the Filter Bar in the grids; gap 21: Indonesian interface shipped, every message but What's New; gap 27: the Windows x64 installer verified on Windows 11; the Settings → Apps uninstall confirmation remains; gap 28: ⌘L Enable Filters switches the filter off and on; gap 15: Japanese photo and postcard paper sizes in the Print page setup and PDF; JPEG ICC APP2 sequence validation; gap 8: lens profile work is tracked in #329; gap 30: the Info panel shows the selected photos' shared metadata and marks mixed fields; gap 28: ⌘⌥⇧C / ⌘⌥⇧V copy and paste metadata; gap 13: before a Lightroom catalog import the app asks merge or new library, #2672; gap 28: F8 shows / hides the right panel; import review album hierarchy, #2362; gap 20: Minolta MRW decodes; gap 10: 557 models verified with CRW and MRW; gap 15: Print Page Options — page numbers, page info, crop marks; gap 28: `\` shows / hides the Filter Bar in the grids; gap 21: Indonesian interface shipped, every message but What's New; gap 27: the Windows x64 installer verified on Windows 11; the Settings → Apps uninstall confirmation remains; gap 28: ⌘L Enable Filters switches the filter off and on; gap 15: Japanese photo and postcard paper sizes in the Print page setup and PDF; gap 8: lens profile work is tracked in #329; gap 30: the Info panel shows the selected photos' shared metadata and marks mixed fields; gap 28: ⌘⌥⇧C / ⌘⌥⇧V copy and paste metadata; gap 13: before a Lightroom catalog import the app asks merge or new library, #2672; gap 28: F8 shows / hides the right panel; import review album hierarchy, #2362; gap 20: Minolta MRW decodes; gap 10: 557 models verified with CRW and MRW; gap 15: Print Page Options — page numbers, page info, crop marks; gap 28: `\` shows / hides the Filter Bar in the grids; gap 21: Indonesian interface shipped, every message but What's New; gap 27: the Windows x64 installer verified on Windows 11; the Settings → Apps uninstall confirmation remains; gap 28: ⌘L Enable Filters switches the filter off and on; gap 15: Japanese photo and postcard paper sizes in the Print page setup and PDF; gap 24: extreme brush segments skip the intermediate dabs; **Last reviewed:** 2026-10-11 · **Last updated:** 2026-10-20 · **Change:** minor (gap 15: Print Page Options — page numbers, page info, crop marks) Every known shortfall, one entry each, ranked by user impact (how many Lightroom users hit it, and how hard it blocks them), then by effort. **Agents pick work from the top** (the first entry nobody is @@ -40,7 +40,7 @@ Hours are Opus 5.5 agent wall-clock hours (see [target-app-parity.md](target-app | 22 | **Pen pressure on macOS / Linux tablets** (brush size and flow follow pressure where winit reports a pen force), OS monitor profiles on Windows (Linux reads colord's, macOS the screen's), follow-window colour on Linux / Windows | hardware-parity.md | brush users with tablets; multi-monitor setups | 11–22 | [hardware-parity](hardware-parity.md) | | 24 | **Brush live preview and masking feel** (#517, #518); Feather / Edge mask refinement (15.5) | issues | masking feels slower than Lightroom; extreme brush segments now skip the intermediate dabs | 8–12 | [ui-parity](ui-parity.md) | | 25 | **File writes**: AVIF import; Render to DNG as a catalog command (lossy DNG and the Render to DNG export format landed) (PSD export landed with Edit in PhotoCraft, #953) | file-format-parity.md | interchange edge cases | 19–34 | [file-format-parity](file-format-parity.md) | -| 26 | **Performance at scale**: flags, ratings and edits are written and synced to the catalog log by a thread of its own, so a key never waits for a busy drive (a 300 ms sync: 0.2 ms per flag, was 305 ms), and a write that fails is held, retried and counted as unsaved, never dropped (PERF-STEP-STALL); 85k-photo library opens in 1.7–4.7 s; 100k grid test pending; colour NR at half resolution; at 100k photos stack and album lookups are indexed (an "In a Stack" or "Album" smart album counts in ~14 ms, was 0.6–2.7 s) | ROADMAP log, gpu-pipeline.md | large-catalog users | 15–30 | [target-app-parity](target-app-parity.md#by-dimension) | +| 26 | **Performance at scale**: flags, ratings and edits are written and synced to the catalog log by a thread of its own, so a key never waits for a busy drive (a 300 ms sync: 0.2 ms per flag, was 305 ms), and a write that fails is held, retried and counted as unsaved, never dropped (PERF-STEP-STALL); 85k-photo library opens in 1.7–4.7 s; 100k grid test pending; colour NR at half resolution; at 100k photos stack and album lookups are indexed (an "In a Stack" or "Album" smart album counts in ~14 ms, was 0.6–2.7 s); culling a wedding: first photos of a 2,933-raw folder in 0.9 s and camera JPEGs ready ahead of each step, but a full-size 45 MP camera JPEG still takes ~150 ms to decode (held-arrow stepping at a 2,560 px loupe can outrun it), the 100 % view of the raw itself still takes ~5 s to render (the camera JPEG is shown at 100 % meanwhile, ~0.9 s), and browsing or importing still reads and hashes every raw whole (~10 ms a file on SSD, ~1 s a 25 MB file from a NAS); the loupe uses built previews and photos are prepared ahead and the whole view in the background (face detection, AI masks and denoise are not yet part of that preparation) | ROADMAP log, gpu-pipeline.md | large-catalog users | 15–30 | [target-app-parity](target-app-parity.md#by-dimension) | | 27 | **Windows / Linux runtime coverage**: the Windows x64 installer is verified on Windows 11 (0.9.1: install and upgrade end on the completion page, a custom folder is kept, Launch works, removal is clean), but uninstalling from Settings → Apps ends with no confirmation (#892, #931); GPU path proven on Apple and on one AMD GPU on Linux (Radeon 8060S, Vulkan); NVIDIA, Intel and DX12 unverified; Windows DirectComposition VRR workaround implemented, monitor verification pending; the Linux riscv64 tarball (new in 0.6.0, #844) is cross-compiled and never run beyond a QEMU smoke test | ROADMAP, issues, `release.yml` | non-Mac users find bugs first | 13–27 | [hardware-parity](hardware-parity.md) | | 29 | **HDR merge leftovers** (LR-MERGE-HDR): AI Denoise doesn't run on merged (linear) DNGs; JPEG brackets are treated as linear (no camera response curve); every frame is held as 32-bit float, ~0.86 GB per 67 MP frame (half-float frames would halve it) | tested on Sony A7R VI brackets 2026-10-14 ([merge.md](merge.md)); Lightroom's Denoise supports HDR DNGs | HDR shooters who denoise their merges, merge JPEGs, or merge 7+ frames of high-resolution raws on 16 GB machines | 10–20 (+ denoise linear-RGB contract, gap 11) | [merge](merge.md) | | 28 | **Small Classic shortcuts and keys**: Tab / ⇧Tab (hide the side panels / all panels; Tab moves the keyboard focus here) and F5. Rating `[` `]`, flag cycling (`` ` ``), `\` (Filter Bar), ⌘L (Enable Filters), ⌘⌥⇧C / ⌘⌥⇧V (Copy / Paste Metadata), F1 help, F6 / F7 / F8 (filmstrip, left panel, right panel) are bound; the grid-cycle key is shown beside both grid rows in View (#2500) | [ui-parity](ui-parity.md#shortcuts) | Classic muscle memory | 2–4 | [ui-parity](ui-parity.md) | @@ -90,6 +90,8 @@ Beta-blocking gaps (B) total **~230–430 h**; with the stability and platform w | Date | Change | Summary | |---|---|---| +| 2026-10-27 | minor | Gap 26: the loupe shows built previews (they were rendered again), photos ahead and the whole view are prepared in the background with settings per mode; gap not closed | +| 2026-10-27 | minor | Gap 26: culling speed measured and improved (progressive Local browse, camera previews made ahead, a CR3's smallest covering JPEG, camera JPEG first at 100 %); what is left listed; gap not closed | | 2026-11-02 | minor | PERF-STEP-STALL: the catalog log is written and synced off the UI thread, so stepping and flagging never wait for the drive (a 300 ms sync: slowest flag 0.2 ms, was 305 ms); a failed write is held and retried, counted as unsaved, never dropped; gap 26 not closed | | 2026-10-28 | minor | Gap 8 shrinks: the desktop app offers lensfun's lens database (CC BY-SA, pinned) once on the first launch and downloads it on request and looks the library's raws up when it arrives; validation, a profile picker and own calibration remain | | 2026-10-23 | patch | Gap 8: the Sigma 10–18 mm on the ILCE-6700 gets a measured lens exception at 10–10.5 mm (0.55 px worst over 99 photos, was about 1.4 px) | diff --git a/docs/gpu-pipeline.md b/docs/gpu-pipeline.md index 6e063e6f90..a21ef36801 100644 --- a/docs/gpu-pipeline.md +++ b/docs/gpu-pipeline.md @@ -428,6 +428,11 @@ view shows a GPU-resident draft reads that texture back for the picture (`Screen - The next and previous photos in filmstrip order are prepared in the background once the current one is rendered (decoded source kept, view render cached): stepping through photos shows the developed image in ~50 ms. +- Culling (PERF-CULL): the stand-ins of the next five photos in the direction of travel and the one + behind are made ahead (`render::Slot::QuickAhead`) and taken over at the step, so a step shows + the camera JPEG at once; while an arrow key is held (a photo left within 0.25 s) the raw render + of a photo waits until it has stayed 0.12 s, and a render queued for a photo already left is + dropped. `ui.inspect` → `perf.loupeSteps` times each step on the frames drawn. ## Memory (M5.6) A view's stages keep an uploaded source only up to 96 MB (a Preview-level source); a bigger one (the original a zoom window is cut from) is kept once, outside the views' stages, for all views of the photo (the Before and After windows and every pan share it; the Before window of a Before/After waits at first open until the After's job has handed the original over, or each would decode and upload its own copy) and given back when the app idles with no zoom window on screen (a zoomed view keeps it for its next pan: a 64 MP original is 800 MB to upload again; issue #323: kept per view it counted against the stage budget on every tick of a drag, and not kept every new window uploaded 288 MB again). diff --git a/docs/parity-checklist.md b/docs/parity-checklist.md index 2758969b26..fb535f1c43 100644 --- a/docs/parity-checklist.md +++ b/docs/parity-checklist.md @@ -1,6 +1,6 @@ # Lightroom parity checklist -> **Last reviewed:** 2026-10-10 · **Last updated:** 2026-11-05 · **Change:** minor (LR-PROF-IMPORT: the user's own .dcp camera profiles import, checked on a real D750 profile against Lightroom; before: LRC-DEV-AUTOSYNC: white balance syncs on each photo's own scale, As Shot and Auto per photo; LR-LIB-FLAG: a flag or rating never waits for the drive, PERF-STEP-STALL; LRC-LIB-VISUALSEARCH: Find Similar Photos reads the photos on a worker thread, so a large library no longer freezes the window; LR-MASK-LINEAR: masks stay on the photo's content under lens corrections, Upright, Transform and Rotate; LR-IMP-FORMATS: the Canon CRW decoder is the shared craft-libs crate `craft-crw`; LR-PROF-CAMERACOLOR: chart-measured matrices for the Canon EOS D60, its white balance in Kelvin; CRW imports name their camera; LR-BEHAV-ACCESS: the Flatpak may talk to the accessibility bus; LR-LIB-ALBUM: album commands leave out photos the library no longer has and record nothing when they add nothing; LR-BEHAV-DRAGDROP: a right-click on a text field while photos are carried from the grid opens no Cut / Copy / Paste menu; LR-BEHAV-DRAGDROP: a right-click while photos are carried from the grid opens no context menu; LR-LIB-VIRTUALCOPY: default numbering is preflighted across the requested batch and explicit names bypass suffix allocation; LR-BEHAV-DRAGDROP: only the primary button drags photos out of the grid and drops them; KEYC-RATING: deleting a keyword while Enable Filters has the filter put aside no longer leaves an empty one there; LR-BEHAV-DRAGDROP: a drop's destination is named with its parent when another folder or album has the same name; LR-VIEW-FILMSTRIP: photos drag from the filmstrip to the same sidebar targets as from the grid; LRC-LIB-LIBFOLDERS: a deep row's count follows its name in a narrow panel; LR-FILT-FLAG / LRC-LIB-FILTERBAR: the flag toggles combine, Picked + Unflagged, #2588; LR-VIEW-PHOTOGRID, LR-VIEW-FILMSTRIP, LR-VIEW-COMPARE, LR-VIEW-SURVEY, LRC-LIB-REFVIEW, LRC-LIB-PEOPLE: the active photo's selection ring shows on light themes and in Compare, Survey and the face tiles, and needs a loaded thumbnail; LR-FILT-SEARCH-META: `>=` / `<=` in fielded search tokens; MENU-FILE-CLOSE, KEY-CLOSEWINDOW: the notes say Close Window quits the app, after the same check as Quit; MENU-EDIT-SELECTNONE, MENU-EDIT-PASTESELECTED: Edit lists Select Only Active Photo after Deselect All and Paste Settings from Previous after Paste Selected Settings…; on macOS no doubled separators are left where Settings… moved to the app menu; LR-BEHAV-TITLEBAR: a double-click follows the system setting, and a panel over the bar keeps the window still; KEYC-METADATA, LRC-LIB-KEYWORDS: ⌥0 / ⌥⇧0 go to the next / previous keyword set; LRC-DEV-HISTOGRAM-DRAG: the histogram's five tonal zones — hover for the value, drag to adjust the Basic control the press locked, double-click to reset; LR-VIEW-HISTOGRAM note updated; MENU-BAR: on macOS a menu item that yields its key to a focused text field or to shortcut recording really lets it go; LR-MERGE-HDR / LR-MERGE-PANO: Lanczos-3 warps; LR-IMP-LOCAL: browsing from the sidebar records the folder; LR-VIEW-ZOOM: only a full-size drag within 20 ms clears a slow-drag verdict; LR-IMP-ADD-DIALOG: hierarchical import album destination reconciled with main, #2362; KEY-SLIDER-NUDGE: vertical wheel nudges need a click or drag in the row; leaving it disarms them; LR-EDIT-OPTICS-PROFILE: Fujifilm nine-knot lens tables read at the plain radius; LR-VIEW-ZOOM: only a full-size drag within 20 ms clears a slow-drag verdict; LRC-LIB-LIBFOLDERS, LR-BEHAV-BG-TASKS: Synchronize Folder's scan shows its folder listing and stops it on ✕; LRC-LIB-LIBFOLDERS: Synchronize Folder says a folder is up to date; LR-LIB-STACK: filmstrip stack badge click; LRC-LIB-REFVIEW / CATALOG-SWITCH: Escape preserves edits; catalog-local pins; deleted and malformed references refused; LRC-LIB-COMPARE: the Info panel labels the active photo while culling, #2398; KEYC-RATING: `\` shows / hides the Filter Bar in the grids; LR-VIEW-PHOTOGRID: the grid reads only the photos an import batch or an edit touched, not the whole list; earlier: LRC-PRINT-PAGE: Print Page Options; LR-META-XMP: fewer false unimported-XMP warnings; KEYC-METADATA: ⌘K / Ctrl+K Add Keywords; LRC-DEV-HISTOGRAM-DRAG: the histogram's five tonal zones — hover for the value, drag to adjust the Basic control the press locked, double-click to reset; LR-VIEW-HISTOGRAM note updated; MENU-BAR: on macOS a menu item that yields its key to a focused text field or to shortcut recording really lets it go; LR-MERGE-HDR / LR-MERGE-PANO: Lanczos-3 warps; LR-IMP-LOCAL: browsing from the sidebar records the folder; LR-VIEW-ZOOM: only a full-size drag within 20 ms clears a slow-drag verdict; LR-IMP-ADD-DIALOG: hierarchical import album destination reconciled with main, #2362; KEY-SLIDER-NUDGE: vertical wheel nudges need a click or drag in the row; leaving it disarms them; LR-EDIT-OPTICS-PROFILE: Fujifilm nine-knot lens tables read at the plain radius; LR-VIEW-ZOOM: only a full-size drag within 20 ms clears a slow-drag verdict; LRC-LIB-LIBFOLDERS, LR-BEHAV-BG-TASKS: Synchronize Folder's scan shows its folder listing and stops it on ✕; LRC-LIB-LIBFOLDERS: Synchronize Folder says a folder is up to date; LR-LIB-STACK: filmstrip stack badge click; LRC-LIB-REFVIEW / CATALOG-SWITCH: Escape preserves edits; catalog-local pins; deleted and malformed references refused; LRC-LIB-COMPARE: the Info panel labels the active photo while culling, #2398; KEYC-RATING: `\` shows / hides the Filter Bar in the grids; LR-VIEW-PHOTOGRID: the grid reads only the photos an import batch or an edit touched, not the whole list; earlier: LRC-PRINT-PAGE: Print Page Options; LR-META-XMP: fewer false unimported-XMP warnings; LR-META-LOCATION: Info GPS text is fully validated before metadata changes; LR-MAP-TRACKLOG decodes GPX character references in point fields; malformed references error; CDATA stays literal; LR-BEHAV-ACCESS: the Thumbnail size control is a slider to screen readers; LR-BEHAV-ACCESS: the Detail photo is named with its file name; LR-VIEW-PHOTOGRID: the grid's empty-view message names its shortcut as this platform writes it and as it is bound; LR-BEHAV-ACCESS: the grid's date headers are named as they read; LR-BEHAV-ACCESS: the sidebar's disclosure triangles name their row; KEYC-METADATA, LRC-LIB-KEYWORDS: ⌘⌥⇧K Set Keyword Shortcut and ⇧K; KEYC-METADATA: ⌘⇧K / Ctrl+Shift+K Edit Keywords; LR-FILT-SEARCH-META: `>=` / `<=` in fielded search tokens; MENU-FILE-CLOSE, KEY-CLOSEWINDOW: the notes say Close Window quits the app, after the same check as Quit; MENU-EDIT-SELECTNONE, MENU-EDIT-PASTESELECTED: Edit lists Select Only Active Photo after Deselect All and Paste Settings from Previous after Paste Selected Settings…; on macOS no doubled separators are left where Settings… moved to the app menu; LR-BEHAV-TITLEBAR: a double-click follows the system setting, and a panel over the bar keeps the window still; KEYC-METADATA, LRC-LIB-KEYWORDS: ⌥0 / ⌥⇧0 go to the next / previous keyword set; LRC-DEV-HISTOGRAM-DRAG: the histogram's five tonal zones — hover for the value, drag to adjust the Basic control the press locked, double-click to reset; LR-VIEW-HISTOGRAM note updated; MENU-BAR: on macOS a menu item that yields its key to a focused text field or to shortcut recording really lets it go; LR-MERGE-HDR / LR-MERGE-PANO: Lanczos-3 warps; LR-IMP-LOCAL: browsing from the sidebar records the folder; LR-VIEW-ZOOM: only a full-size drag within 20 ms clears a slow-drag verdict; LR-IMP-ADD-DIALOG: hierarchical import album destination reconciled with main, #2362; KEY-SLIDER-NUDGE: vertical wheel nudges need a click or drag in the row; leaving it disarms them; LR-EDIT-OPTICS-PROFILE: Fujifilm nine-knot lens tables read at the plain radius; LR-VIEW-ZOOM: only a full-size drag within 20 ms clears a slow-drag verdict; LRC-LIB-LIBFOLDERS, LR-BEHAV-BG-TASKS: Synchronize Folder's scan shows its folder listing and stops it on ✕; LRC-LIB-LIBFOLDERS: Synchronize Folder says a folder is up to date; LR-LIB-STACK: filmstrip stack badge click; LRC-LIB-REFVIEW / CATALOG-SWITCH: Escape preserves edits; catalog-local pins; deleted and malformed references refused; LRC-LIB-COMPARE: the Info panel labels the active photo while culling, #2398; KEYC-RATING: `\` shows / hides the Filter Bar in the grids; LR-VIEW-PHOTOGRID: the grid reads only the photos an import batch or an edit touched, not the whole list; earlier: LRC-PRINT-PAGE: Print Page Options; LR-META-XMP: fewer false unimported-XMP warnings; KEYC-METADATA: ⌘K / Ctrl+K Add Keywords; LRC-DEV-HISTOGRAM-DRAG: the histogram's five tonal zones — hover for the value, drag to adjust the Basic control the press locked, double-click to reset; LR-VIEW-HISTOGRAM note updated; MENU-BAR: on macOS a menu item that yields its key to a focused text field or to shortcut recording really lets it go; LR-MERGE-HDR / LR-MERGE-PANO: Lanczos-3 warps; LR-IMP-LOCAL: browsing from the sidebar records the folder; LR-VIEW-ZOOM: only a full-size drag within 20 ms clears a slow-drag verdict; LR-IMP-ADD-DIALOG: hierarchical import album destination reconciled with main, #2362; KEY-SLIDER-NUDGE: vertical wheel nudges need a click or drag in the row; leaving it disarms them; LR-EDIT-OPTICS-PROFILE: Fujifilm nine-knot lens tables read at the plain radius; LR-VIEW-ZOOM: only a full-size drag within 20 ms clears a slow-drag verdict; LRC-LIB-LIBFOLDERS, LR-BEHAV-BG-TASKS: Synchronize Folder's scan shows its folder listing and stops it on ✕; LRC-LIB-LIBFOLDERS: Synchronize Folder says a folder is up to date; LR-LIB-STACK: filmstrip stack badge click; LRC-LIB-REFVIEW / CATALOG-SWITCH: Escape preserves edits; catalog-local pins; deleted and malformed references refused; LRC-LIB-COMPARE: the Info panel labels the active photo while culling, #2398; KEYC-RATING: `\` shows / hides the Filter Bar in the grids; LR-VIEW-PHOTOGRID: the grid reads only the photos an import batch or an edit touched, not the whole list; earlier: LRC-PRINT-PAGE: Print Page Options; LR-META-XMP: fewer false unimported-XMP warnings; LRC-LIB-FOLDERS: Local's built-in places (Pictures, Desktop, Downloads, Home) offer no Rename Folder… or Move Folder To…, and `folder.rename` / `folder.move` refuse the home folder and its Pictures, Desktop and Downloads; LRC-LIB-CATALOG-SWITCH: a library's catalog.snap or folder given at launch opens that library; LR-IMP-ADD-DIALOG / LR-IMP-MOVE: checked rename-sequence exhaustion; LR-IMP-ADD-DIALOG: an import asks the disk about the probes a scan left, and for each photo's sidecar, up to 16 files at a time; LRC-LIB-LIBFOLDERS: Don't ask again in the Move Photos question is kept only when the move started; a refused move leaves the question on; LR-IMP-ADD-DIALOG: remembered import thumbnail size, density-aware bounded previews and one job per candidate/size bucket (#2237, part of #2056); import loupe remains open; counts unchanged; LR-FILT-SEARCH-META: `>=` / `<=` in fielded search tokens; MENU-FILE-CLOSE, KEY-CLOSEWINDOW: the notes say Close Window quits the app, after the same check as Quit; MENU-EDIT-SELECTNONE, MENU-EDIT-PASTESELECTED: Edit lists Select Only Active Photo after Deselect All and Paste Settings from Previous after Paste Selected Settings…; on macOS no doubled separators are left where Settings… moved to the app menu; LR-BEHAV-TITLEBAR: a double-click follows the system setting, and a panel over the bar keeps the window still; KEYC-METADATA, LRC-LIB-KEYWORDS: ⌥0 / ⌥⇧0 go to the next / previous keyword set; LRC-DEV-HISTOGRAM-DRAG: the histogram's five tonal zones — hover for the value, drag to adjust the Basic control the press locked, double-click to reset; LR-VIEW-HISTOGRAM note updated; MENU-BAR: on macOS a menu item that yields its key to a focused text field or to shortcut recording really lets it go; LR-MERGE-HDR / LR-MERGE-PANO: Lanczos-3 warps; LR-IMP-LOCAL: browsing from the sidebar records the folder; LR-VIEW-ZOOM: only a full-size drag within 20 ms clears a slow-drag verdict; LR-IMP-ADD-DIALOG: hierarchical import album destination reconciled with main, #2362; KEY-SLIDER-NUDGE: vertical wheel nudges need a click or drag in the row; leaving it disarms them; LR-EDIT-OPTICS-PROFILE: Fujifilm nine-knot lens tables read at the plain radius; LR-VIEW-ZOOM: only a full-size drag within 20 ms clears a slow-drag verdict; LRC-LIB-LIBFOLDERS, LR-BEHAV-BG-TASKS: Synchronize Folder's scan shows its folder listing and stops it on ✕; LRC-LIB-LIBFOLDERS: Synchronize Folder says a folder is up to date; LR-LIB-STACK: filmstrip stack badge click; LRC-LIB-REFVIEW / CATALOG-SWITCH: Escape preserves edits; catalog-local pins; deleted and malformed references refused; LRC-LIB-COMPARE: the Info panel labels the active photo while culling, #2398; KEYC-RATING: `\` shows / hides the Filter Bar in the grids; LR-VIEW-PHOTOGRID: the grid reads only the photos an import batch or an edit touched, not the whole list; earlier: LRC-PRINT-PAGE: Print Page Options; LR-META-XMP: fewer false unimported-XMP warnings; KEYC-METADATA: ⌘K / Ctrl+K Add Keywords; LRC-DEV-HISTOGRAM-DRAG: the histogram's five tonal zones — hover for the value, drag to adjust the Basic control the press locked, double-click to reset; LR-VIEW-HISTOGRAM note updated; MENU-BAR: on macOS a menu item that yields its key to a focused text field or to shortcut recording really lets it go; LR-MERGE-HDR / LR-MERGE-PANO: Lanczos-3 warps; LR-IMP-LOCAL: browsing from the sidebar records the folder; LR-VIEW-ZOOM: only a full-size drag within 20 ms clears a slow-drag verdict; LR-IMP-ADD-DIALOG: hierarchical import album destination reconciled with main, #2362; KEY-SLIDER-NUDGE: vertical wheel nudges need a click or drag in the row; leaving it disarms them; LR-EDIT-OPTICS-PROFILE: Fujifilm nine-knot lens tables read at the plain radius; LR-VIEW-ZOOM: only a full-size drag within 20 ms clears a slow-drag verdict; LRC-LIB-LIBFOLDERS, LR-BEHAV-BG-TASKS: Synchronize Folder's scan shows its folder listing and stops it on ✕; LRC-LIB-LIBFOLDERS: Synchronize Folder says a folder is up to date; LR-LIB-STACK: filmstrip stack badge click; LRC-LIB-REFVIEW / CATALOG-SWITCH: Escape preserves edits; catalog-local pins; deleted and malformed references refused; LRC-LIB-COMPARE: the Info panel labels the active photo while culling, #2398; KEYC-RATING: `\` shows / hides the Filter Bar in the grids; LR-VIEW-PHOTOGRID: the grid reads only the photos an import batch or an edit touched, not the whole list; earlier: LRC-PRINT-PAGE: Print Page Options; LR-META-XMP: fewer false unimported-XMP warnings; LR-MERGE-HDR / LR-MERGE-PANO / LR-MERGE-HDRPANO: the dialog makes a new preview each time it opens, also after a failed one, #2708; LR-FILT-SEARCH-META: `>=` / `<=` in fielded search tokens; MENU-FILE-CLOSE, KEY-CLOSEWINDOW: the notes say Close Window quits the app, after the same check as Quit; MENU-EDIT-SELECTNONE, MENU-EDIT-PASTESELECTED: Edit lists Select Only Active Photo after Deselect All and Paste Settings from Previous after Paste Selected Settings…; on macOS no doubled separators are left where Settings… moved to the app menu; LR-BEHAV-TITLEBAR: a double-click follows the system setting, and a panel over the bar keeps the window still; KEYC-METADATA, LRC-LIB-KEYWORDS: ⌥0 / ⌥⇧0 go to the next / previous keyword set; LRC-DEV-HISTOGRAM-DRAG: the histogram's five tonal zones — hover for the value, drag to adjust the Basic control the press locked, double-click to reset; LR-VIEW-HISTOGRAM note updated; MENU-BAR: on macOS a menu item that yields its key to a focused text field or to shortcut recording really lets it go; LR-MERGE-HDR / LR-MERGE-PANO: Lanczos-3 warps; LR-IMP-LOCAL: browsing from the sidebar records the folder; LR-VIEW-ZOOM: only a full-size drag within 20 ms clears a slow-drag verdict; LR-IMP-ADD-DIALOG: hierarchical import album destination reconciled with main, #2362; KEY-SLIDER-NUDGE: vertical wheel nudges need a click or drag in the row; leaving it disarms them; LR-EDIT-OPTICS-PROFILE: Fujifilm nine-knot lens tables read at the plain radius; LR-VIEW-ZOOM: only a full-size drag within 20 ms clears a slow-drag verdict; LRC-LIB-LIBFOLDERS, LR-BEHAV-BG-TASKS: Synchronize Folder's scan shows its folder listing and stops it on ✕; LRC-LIB-LIBFOLDERS: Synchronize Folder says a folder is up to date; LR-LIB-STACK: filmstrip stack badge click; LRC-LIB-REFVIEW / CATALOG-SWITCH: Escape preserves edits; catalog-local pins; deleted and malformed references refused; LRC-LIB-COMPARE: the Info panel labels the active photo while culling, #2398; KEYC-RATING: `\` shows / hides the Filter Bar in the grids; LR-VIEW-PHOTOGRID: the grid reads only the photos an import batch or an edit touched, not the whole list; earlier: LRC-PRINT-PAGE: Print Page Options; LR-META-XMP: fewer false unimported-XMP warnings; KEYC-METADATA: ⌘K / Ctrl+K Add Keywords; LRC-DEV-HISTOGRAM-DRAG: the histogram's five tonal zones — hover for the value, drag to adjust the Basic control the press locked, double-click to reset; LR-VIEW-HISTOGRAM note updated; MENU-BAR: on macOS a menu item that yields its key to a focused text field or to shortcut recording really lets it go; LR-MERGE-HDR / LR-MERGE-PANO: Lanczos-3 warps; LR-IMP-LOCAL: browsing from the sidebar records the folder; LR-VIEW-ZOOM: only a full-size drag within 20 ms clears a slow-drag verdict; LR-IMP-ADD-DIALOG: hierarchical import album destination reconciled with main, #2362; KEY-SLIDER-NUDGE: vertical wheel nudges need a click or drag in the row; leaving it disarms them; LR-EDIT-OPTICS-PROFILE: Fujifilm nine-knot lens tables read at the plain radius; LR-VIEW-ZOOM: only a full-size drag within 20 ms clears a slow-drag verdict; LRC-LIB-LIBFOLDERS, LR-BEHAV-BG-TASKS: Synchronize Folder's scan shows its folder listing and stops it on ✕; LRC-LIB-LIBFOLDERS: Synchronize Folder says a folder is up to date; LR-LIB-STACK: filmstrip stack badge click; LRC-LIB-REFVIEW / CATALOG-SWITCH: Escape preserves edits; catalog-local pins; deleted and malformed references refused; LRC-LIB-COMPARE: the Info panel labels the active photo while culling, #2398; KEYC-RATING: `\` shows / hides the Filter Bar in the grids; LR-VIEW-PHOTOGRID: the grid reads only the photos an import batch or an edit touched, not the whole list; earlier: LRC-PRINT-PAGE: Print Page Options; LR-META-XMP: fewer false unimported-XMP warnings; LRC-LIB-MISSING: `library.missing` from the control channel checks the files on a worker thread instead of the UI thread; LR-FILT-SEARCH-META: `>=` / `<=` in fielded search tokens; MENU-FILE-CLOSE, KEY-CLOSEWINDOW: the notes say Close Window quits the app, after the same check as Quit; MENU-EDIT-SELECTNONE, MENU-EDIT-PASTESELECTED: Edit lists Select Only Active Photo after Deselect All and Paste Settings from Previous after Paste Selected Settings…; on macOS no doubled separators are left where Settings… moved to the app menu; LR-BEHAV-TITLEBAR: a double-click follows the system setting, and a panel over the bar keeps the window still; KEYC-METADATA, LRC-LIB-KEYWORDS: ⌥0 / ⌥⇧0 go to the next / previous keyword set; LRC-DEV-HISTOGRAM-DRAG: the histogram's five tonal zones — hover for the value, drag to adjust the Basic control the press locked, double-click to reset; LR-VIEW-HISTOGRAM note updated; MENU-BAR: on macOS a menu item that yields its key to a focused text field or to shortcut recording really lets it go; LR-MERGE-HDR / LR-MERGE-PANO: Lanczos-3 warps; LR-IMP-LOCAL: browsing from the sidebar records the folder; LR-VIEW-ZOOM: only a full-size drag within 20 ms clears a slow-drag verdict; LR-IMP-ADD-DIALOG: hierarchical import album destination reconciled with main, #2362; KEY-SLIDER-NUDGE: vertical wheel nudges need a click or drag in the row; leaving it disarms them; LR-EDIT-OPTICS-PROFILE: Fujifilm nine-knot lens tables read at the plain radius; LR-VIEW-ZOOM: only a full-size drag within 20 ms clears a slow-drag verdict; LRC-LIB-LIBFOLDERS, LR-BEHAV-BG-TASKS: Synchronize Folder's scan shows its folder listing and stops it on ✕; LRC-LIB-LIBFOLDERS: Synchronize Folder says a folder is up to date; LR-LIB-STACK: filmstrip stack badge click; LRC-LIB-REFVIEW / CATALOG-SWITCH: Escape preserves edits; catalog-local pins; deleted and malformed references refused; LRC-LIB-COMPARE: the Info panel labels the active photo while culling, #2398; KEYC-RATING: `\` shows / hides the Filter Bar in the grids; LR-VIEW-PHOTOGRID: the grid reads only the photos an import batch or an edit touched, not the whole list; earlier: LRC-PRINT-PAGE: Print Page Options; LR-META-XMP: fewer false unimported-XMP warnings; KEYC-METADATA: ⌘K / Ctrl+K Add Keywords; LRC-DEV-HISTOGRAM-DRAG: the histogram's five tonal zones — hover for the value, drag to adjust the Basic control the press locked, double-click to reset; LR-VIEW-HISTOGRAM note updated; MENU-BAR: on macOS a menu item that yields its key to a focused text field or to shortcut recording really lets it go; LR-MERGE-HDR / LR-MERGE-PANO: Lanczos-3 warps; LR-IMP-LOCAL: browsing from the sidebar records the folder; LR-VIEW-ZOOM: only a full-size drag within 20 ms clears a slow-drag verdict; LR-IMP-ADD-DIALOG: hierarchical import album destination reconciled with main, #2362; KEY-SLIDER-NUDGE: vertical wheel nudges need a click or drag in the row; leaving it disarms them; LR-EDIT-OPTICS-PROFILE: Fujifilm nine-knot lens tables read at the plain radius; LR-VIEW-ZOOM: only a full-size drag within 20 ms clears a slow-drag verdict; LRC-LIB-LIBFOLDERS, LR-BEHAV-BG-TASKS: Synchronize Folder's scan shows its folder listing and stops it on ✕; LRC-LIB-LIBFOLDERS: Synchronize Folder says a folder is up to date; LR-LIB-STACK: filmstrip stack badge click; LRC-LIB-REFVIEW / CATALOG-SWITCH: Escape preserves edits; catalog-local pins; deleted and malformed references refused; LRC-LIB-COMPARE: the Info panel labels the active photo while culling, #2398; KEYC-RATING: `\` shows / hides the Filter Bar in the grids; LR-VIEW-PHOTOGRID: the grid reads only the photos an import batch or an edit touched, not the whole list; earlier: LRC-PRINT-PAGE: Print Page Options; LR-META-XMP: fewer false unimported-XMP warnings; LR-CROP-AUTO: Auto straighten reads the vertical vanishing point and the horizon; LR-META-EXIF-FULL: reject extra EXIF clock components so invalid capture-time candidates fall through to valid metadata; LR-FILT-SEARCH-META: `>=` / `<=` in fielded search tokens; MENU-FILE-CLOSE, KEY-CLOSEWINDOW: the notes say Close Window quits the app, after the same check as Quit; MENU-EDIT-SELECTNONE, MENU-EDIT-PASTESELECTED: Edit lists Select Only Active Photo after Deselect All and Paste Settings from Previous after Paste Selected Settings…; on macOS no doubled separators are left where Settings… moved to the app menu; LR-BEHAV-TITLEBAR: a double-click follows the system setting, and a panel over the bar keeps the window still; KEYC-METADATA, LRC-LIB-KEYWORDS: ⌥0 / ⌥⇧0 go to the next / previous keyword set; LRC-DEV-HISTOGRAM-DRAG: the histogram's five tonal zones — hover for the value, drag to adjust the Basic control the press locked, double-click to reset; LR-VIEW-HISTOGRAM note updated; MENU-BAR: on macOS a menu item that yields its key to a focused text field or to shortcut recording really lets it go; LR-MERGE-HDR / LR-MERGE-PANO: Lanczos-3 warps; LR-IMP-LOCAL: browsing from the sidebar records the folder; LR-VIEW-ZOOM: only a full-size drag within 20 ms clears a slow-drag verdict; LR-IMP-ADD-DIALOG: hierarchical import album destination reconciled with main, #2362; KEY-SLIDER-NUDGE: vertical wheel nudges need a click or drag in the row; leaving it disarms them; LR-EDIT-OPTICS-PROFILE: Fujifilm nine-knot lens tables read at the plain radius; LR-VIEW-ZOOM: only a full-size drag within 20 ms clears a slow-drag verdict; LRC-LIB-LIBFOLDERS, LR-BEHAV-BG-TASKS: Synchronize Folder's scan shows its folder listing and stops it on ✕; LRC-LIB-LIBFOLDERS: Synchronize Folder says a folder is up to date; LR-LIB-STACK: filmstrip stack badge click; LRC-LIB-REFVIEW / CATALOG-SWITCH: Escape preserves edits; catalog-local pins; deleted and malformed references refused; LRC-LIB-COMPARE: the Info panel labels the active photo while culling, #2398; KEYC-RATING: `\` shows / hides the Filter Bar in the grids; LR-VIEW-PHOTOGRID: the grid reads only the photos an import batch or an edit touched, not the whole list; earlier: LRC-PRINT-PAGE: Print Page Options; LR-META-XMP: fewer false unimported-XMP warnings; KEYC-METADATA: ⌘K / Ctrl+K Add Keywords; LRC-DEV-HISTOGRAM-DRAG: the histogram's five tonal zones — hover for the value, drag to adjust the Basic control the press locked, double-click to reset; LR-VIEW-HISTOGRAM note updated; MENU-BAR: on macOS a menu item that yields its key to a focused text field or to shortcut recording really lets it go; LR-MERGE-HDR / LR-MERGE-PANO: Lanczos-3 warps; LR-IMP-LOCAL: browsing from the sidebar records the folder; LR-VIEW-ZOOM: only a full-size drag within 20 ms clears a slow-drag verdict; LR-IMP-ADD-DIALOG: hierarchical import album destination reconciled with main, #2362; KEY-SLIDER-NUDGE: vertical wheel nudges need a click or drag in the row; leaving it disarms them; LR-EDIT-OPTICS-PROFILE: Fujifilm nine-knot lens tables read at the plain radius; LR-VIEW-ZOOM: only a full-size drag within 20 ms clears a slow-drag verdict; LRC-LIB-LIBFOLDERS, LR-BEHAV-BG-TASKS: Synchronize Folder's scan shows its folder listing and stops it on ✕; LRC-LIB-LIBFOLDERS: Synchronize Folder says a folder is up to date; LR-LIB-STACK: filmstrip stack badge click; LRC-LIB-REFVIEW / CATALOG-SWITCH: Escape preserves edits; catalog-local pins; deleted and malformed references refused; LRC-LIB-COMPARE: the Info panel labels the active photo while culling, #2398; KEYC-RATING: `\` shows / hides the Filter Bar in the grids; LR-VIEW-PHOTOGRID: the grid reads only the photos an import batch or an edit touched, not the whole list; earlier: LRC-PRINT-PAGE: Print Page Options; LR-META-XMP: fewer false unimported-XMP warnings; LR-META-XMP / LRC-MAP-TRACKLOG: malformed ISO times are not projected or matched; LR-IMP-FORMATS: JPEG ICC APP2 chunk validation; LR-BEHAV-TITLEBAR: a dialog leaves the title bar undimmed, as a macOS sheet does; LR-BEHAV-ACCESS: the bars are toolbars and the panels landmarks for screen readers; LR-BEHAV-ACCESS: every dialog and prompt is exposed as a dialog (a modal one, an alert dialog for the prompts); LR-BEHAV-ACCESS: icon-only buttons are named by their label, with the shortcut as the description; MENU-HELP-HELP: on macOS the Help menu is registered with the application only once the menu bar is installed (muda ignores the call before), so AppKit knows which menu is Help; LR-BEHAV-ACCESS: a slider's value field is named ` value`, with how to use it as its description; MENU-BAR: every shortcut the app shows on macOS lists its modifiers ⌃⌥⇧⌘, as the menu bar does; MENU-BAR: on macOS two menu rows that run the same command each get their own native row and label (File ▸ Export… and File ▸ Export with Preset ▸ Custom…; File ▸ New Album… and Photo ▸ Add to Album ▸ New Album…); LR-BEHAV-DIALOGS: a one-line prompt that is refused stays open and says why, and a New Folder on Disk says it was made; LR-FILT-SEARCH-META: `>=` / `<=` in fielded search tokens; MENU-FILE-CLOSE, KEY-CLOSEWINDOW: the notes say Close Window quits the app, after the same check as Quit; MENU-EDIT-SELECTNONE, MENU-EDIT-PASTESELECTED: Edit lists Select Only Active Photo after Deselect All and Paste Settings from Previous after Paste Selected Settings…; on macOS no doubled separators are left where Settings… moved to the app menu; LR-BEHAV-TITLEBAR: a double-click follows the system setting, and a panel over the bar keeps the window still; KEYC-METADATA, LRC-LIB-KEYWORDS: ⌥0 / ⌥⇧0 go to the next / previous keyword set; LRC-DEV-HISTOGRAM-DRAG: the histogram's five tonal zones — hover for the value, drag to adjust the Basic control the press locked, double-click to reset; LR-VIEW-HISTOGRAM note updated; MENU-BAR: on macOS a menu item that yields its key to a focused text field or to shortcut recording really lets it go; LR-MERGE-HDR / LR-MERGE-PANO: Lanczos-3 warps; LR-IMP-LOCAL: browsing from the sidebar records the folder; LR-VIEW-ZOOM: only a full-size drag within 20 ms clears a slow-drag verdict; LR-IMP-ADD-DIALOG: hierarchical import album destination reconciled with main, #2362; KEY-SLIDER-NUDGE: vertical wheel nudges need a click or drag in the row; leaving it disarms them; LR-EDIT-OPTICS-PROFILE: Fujifilm nine-knot lens tables read at the plain radius; LR-VIEW-ZOOM: only a full-size drag within 20 ms clears a slow-drag verdict; LRC-LIB-LIBFOLDERS, LR-BEHAV-BG-TASKS: Synchronize Folder's scan shows its folder listing and stops it on ✕; LRC-LIB-LIBFOLDERS: Synchronize Folder says a folder is up to date; LR-LIB-STACK: filmstrip stack badge click; LRC-LIB-REFVIEW / CATALOG-SWITCH: Escape preserves edits; catalog-local pins; deleted and malformed references refused; LRC-LIB-COMPARE: the Info panel labels the active photo while culling, #2398; KEYC-RATING: `\` shows / hides the Filter Bar in the grids; LR-VIEW-PHOTOGRID: the grid reads only the photos an import batch or an edit touched, not the whole list; earlier: LRC-PRINT-PAGE: Print Page Options; LR-META-XMP: fewer false unimported-XMP warnings; KEYC-METADATA: ⌘K / Ctrl+K Add Keywords; LRC-DEV-HISTOGRAM-DRAG: the histogram's five tonal zones — hover for the value, drag to adjust the Basic control the press locked, double-click to reset; LR-VIEW-HISTOGRAM note updated; MENU-BAR: on macOS a menu item that yields its key to a focused text field or to shortcut recording really lets it go; LR-MERGE-HDR / LR-MERGE-PANO: Lanczos-3 warps; LR-IMP-LOCAL: browsing from the sidebar records the folder; LR-VIEW-ZOOM: only a full-size drag within 20 ms clears a slow-drag verdict; LR-IMP-ADD-DIALOG: hierarchical import album destination reconciled with main, #2362; KEY-SLIDER-NUDGE: vertical wheel nudges need a click or drag in the row; leaving it disarms them; LR-EDIT-OPTICS-PROFILE: Fujifilm nine-knot lens tables read at the plain radius; LR-VIEW-ZOOM: only a full-size drag within 20 ms clears a slow-drag verdict; LRC-LIB-LIBFOLDERS, LR-BEHAV-BG-TASKS: Synchronize Folder's scan shows its folder listing and stops it on ✕; LRC-LIB-LIBFOLDERS: Synchronize Folder says a folder is up to date; LR-LIB-STACK: filmstrip stack badge click; LRC-LIB-REFVIEW / CATALOG-SWITCH: Escape preserves edits; catalog-local pins; deleted and malformed references refused; LRC-LIB-COMPARE: the Info panel labels the active photo while culling, #2398; KEYC-RATING: `\` shows / hides the Filter Bar in the grids; LR-VIEW-PHOTOGRID: the grid reads only the photos an import batch or an edit touched, not the whole list; earlier: LRC-PRINT-PAGE: Print Page Options; LR-META-XMP: fewer false unimported-XMP warnings; LR-VIEW-HISTOGRAM: the shadow and highlight clipping buttons take their tile and chevron from the theme, so the chevron reads in every theme; LR-FILT-SEARCH-META: `>=` / `<=` in fielded search tokens; MENU-FILE-CLOSE, KEY-CLOSEWINDOW: the notes say Close Window quits the app, after the same check as Quit; MENU-EDIT-SELECTNONE, MENU-EDIT-PASTESELECTED: Edit lists Select Only Active Photo after Deselect All and Paste Settings from Previous after Paste Selected Settings…; on macOS no doubled separators are left where Settings… moved to the app menu; LR-BEHAV-TITLEBAR: a double-click follows the system setting, and a panel over the bar keeps the window still; KEYC-METADATA, LRC-LIB-KEYWORDS: ⌥0 / ⌥⇧0 go to the next / previous keyword set; LRC-DEV-HISTOGRAM-DRAG: the histogram's five tonal zones — hover for the value, drag to adjust the Basic control the press locked, double-click to reset; LR-VIEW-HISTOGRAM note updated; MENU-BAR: on macOS a menu item that yields its key to a focused text field or to shortcut recording really lets it go; LR-MERGE-HDR / LR-MERGE-PANO: Lanczos-3 warps; LR-IMP-LOCAL: browsing from the sidebar records the folder; LR-VIEW-ZOOM: only a full-size drag within 20 ms clears a slow-drag verdict; LR-IMP-ADD-DIALOG: hierarchical import album destination reconciled with main, #2362; KEY-SLIDER-NUDGE: vertical wheel nudges need a click or drag in the row; leaving it disarms them; LR-EDIT-OPTICS-PROFILE: Fujifilm nine-knot lens tables read at the plain radius; LR-VIEW-ZOOM: only a full-size drag within 20 ms clears a slow-drag verdict; LRC-LIB-LIBFOLDERS, LR-BEHAV-BG-TASKS: Synchronize Folder's scan shows its folder listing and stops it on ✕; LRC-LIB-LIBFOLDERS: Synchronize Folder says a folder is up to date; LR-LIB-STACK: filmstrip stack badge click; LRC-LIB-REFVIEW / CATALOG-SWITCH: Escape preserves edits; catalog-local pins; deleted and malformed references refused; LRC-LIB-COMPARE: the Info panel labels the active photo while culling, #2398; KEYC-RATING: `\` shows / hides the Filter Bar in the grids; LR-VIEW-PHOTOGRID: the grid reads only the photos an import batch or an edit touched, not the whole list; earlier: LRC-PRINT-PAGE: Print Page Options; LR-META-XMP: fewer false unimported-XMP warnings; KEYC-METADATA: ⌘K / Ctrl+K Add Keywords; LRC-DEV-HISTOGRAM-DRAG: the histogram's five tonal zones — hover for the value, drag to adjust the Basic control the press locked, double-click to reset; LR-VIEW-HISTOGRAM note updated; MENU-BAR: on macOS a menu item that yields its key to a focused text field or to shortcut recording really lets it go; LR-MERGE-HDR / LR-MERGE-PANO: Lanczos-3 warps; LR-IMP-LOCAL: browsing from the sidebar records the folder; LR-VIEW-ZOOM: only a full-size drag within 20 ms clears a slow-drag verdict; LR-IMP-ADD-DIALOG: hierarchical import album destination reconciled with main, #2362; KEY-SLIDER-NUDGE: vertical wheel nudges need a click or drag in the row; leaving it disarms them; LR-EDIT-OPTICS-PROFILE: Fujifilm nine-knot lens tables read at the plain radius; LR-VIEW-ZOOM: only a full-size drag within 20 ms clears a slow-drag verdict; LRC-LIB-LIBFOLDERS, LR-BEHAV-BG-TASKS: Synchronize Folder's scan shows its folder listing and stops it on ✕; LRC-LIB-LIBFOLDERS: Synchronize Folder says a folder is up to date; LR-LIB-STACK: filmstrip stack badge click; LRC-LIB-REFVIEW / CATALOG-SWITCH: Escape preserves edits; catalog-local pins; deleted and malformed references refused; LRC-LIB-COMPARE: the Info panel labels the active photo while culling, #2398; KEYC-RATING: `\` shows / hides the Filter Bar in the grids; LR-VIEW-PHOTOGRID: the grid reads only the photos an import batch or an edit touched, not the whole list; earlier: LRC-PRINT-PAGE: Print Page Options; LR-META-XMP: fewer false unimported-XMP warnings; LR-FILT-SEARCH-META: `>=` / `<=` in fielded search tokens; MENU-FILE-CLOSE, KEY-CLOSEWINDOW: the notes say Close Window quits the app, after the same check as Quit; MENU-EDIT-SELECTNONE, MENU-EDIT-PASTESELECTED: Edit lists Select Only Active Photo after Deselect All and Paste Settings from Previous after Paste Selected Settings…; on macOS no doubled separators are left where Settings… moved to the app menu; LR-BEHAV-TITLEBAR: a double-click follows the system setting, and a panel over the bar keeps the window still; KEYC-METADATA, LRC-LIB-KEYWORDS: ⌥0 / ⌥⇧0 go to the next / previous keyword set; LRC-DEV-HISTOGRAM-DRAG: the histogram's five tonal zones — hover for the value, drag to adjust the Basic control the press locked, double-click to reset; LR-VIEW-HISTOGRAM note updated; MENU-BAR: on macOS a menu item that yields its key to a focused text field or to shortcut recording really lets it go; LR-MERGE-HDR / LR-MERGE-PANO: Lanczos-3 warps; LR-IMP-LOCAL: browsing from the sidebar records the folder; LR-VIEW-ZOOM: only a full-size drag within 20 ms clears a slow-drag verdict; LR-IMP-ADD-DIALOG: hierarchical import album destination reconciled with main, #2362; KEY-SLIDER-NUDGE: vertical wheel nudges need a click or drag in the row; leaving it disarms them; LR-EDIT-OPTICS-PROFILE: Fujifilm nine-knot lens tables read at the plain radius; LR-VIEW-ZOOM: only a full-size drag within 20 ms clears a slow-drag verdict; LRC-LIB-LIBFOLDERS, LR-BEHAV-BG-TASKS: Synchronize Folder's scan shows its folder listing and stops it on ✕; LRC-LIB-LIBFOLDERS: Synchronize Folder says a folder is up to date; LR-LIB-STACK: filmstrip stack badge click; LRC-LIB-REFVIEW / CATALOG-SWITCH: Escape preserves edits; catalog-local pins; deleted and malformed references refused; LRC-LIB-COMPARE: the Info panel labels the active photo while culling, #2398; KEYC-RATING: `\` shows / hides the Filter Bar in the grids; LR-VIEW-PHOTOGRID: the grid reads only the photos an import batch or an edit touched, not the whole list; earlier: LRC-PRINT-PAGE: Print Page Options; LR-META-XMP: fewer false unimported-XMP warnings; KEYC-METADATA: ⌘K / Ctrl+K Add Keywords; LRC-DEV-HISTOGRAM-DRAG: the histogram's five tonal zones — hover for the value, drag to adjust the Basic control the press locked, double-click to reset; LR-VIEW-HISTOGRAM note updated; MENU-BAR: on macOS a menu item that yields its key to a focused text field or to shortcut recording really lets it go; LR-MERGE-HDR / LR-MERGE-PANO: Lanczos-3 warps; LR-IMP-LOCAL: browsing from the sidebar records the folder; LR-VIEW-ZOOM: only a full-size drag within 20 ms clears a slow-drag verdict; LR-IMP-ADD-DIALOG: hierarchical import album destination reconciled with main, #2362; KEY-SLIDER-NUDGE: vertical wheel nudges need a click or drag in the row; leaving it disarms them; LR-EDIT-OPTICS-PROFILE: Fujifilm nine-knot lens tables read at the plain radius; LR-VIEW-ZOOM: only a full-size drag within 20 ms clears a slow-drag verdict; LRC-LIB-LIBFOLDERS, LR-BEHAV-BG-TASKS: Synchronize Folder's scan shows its folder listing and stops it on ✕; LRC-LIB-LIBFOLDERS: Synchronize Folder says a folder is up to date; LR-LIB-STACK: filmstrip stack badge click; LRC-LIB-REFVIEW / CATALOG-SWITCH: Escape preserves edits; catalog-local pins; deleted and malformed references refused; LRC-LIB-COMPARE: the Info panel labels the active photo while culling, #2398; KEYC-RATING: `\` shows / hides the Filter Bar in the grids; LR-VIEW-PHOTOGRID: the grid reads only the photos an import batch or an edit touched, not the whole list; earlier: LRC-PRINT-PAGE: Print Page Options; LR-META-XMP: fewer false unimported-XMP warnings; LR-MASK-BRUSH: extreme brush segments skip the intermediate dabs; LR-MASK-BRUSH: with the mask overlay hidden, the stroke being painted no longer tints the photo) · **Target:** Adobe Lightroom Classic 15.5 / Lightroom 9.6 +> **Last reviewed:** 2026-10-10 · **Last updated:** 2026-11-05 · **Change:** minor (LR-BEHAV-ACCESS: the Flatpak may talk to the accessibility bus; before: LR-PROF-IMPORT: the user's own .dcp camera profiles import, checked on a real D750 profile against Lightroom; LRC-DEV-AUTOSYNC: white balance syncs on each photo's own scale, As Shot and Auto per photo; LR-LIB-FLAG: a flag or rating never waits for the drive, PERF-STEP-STALL; LRC-LIB-VISUALSEARCH: Find Similar Photos reads the photos on a worker thread, so a large library no longer freezes the window; LR-MASK-LINEAR: masks stay on the photo's content under lens corrections, Upright, Transform and Rotate; LR-IMP-FORMATS: the Canon CRW decoder is the shared craft-libs crate `craft-crw`; LR-PROF-CAMERACOLOR: chart-measured matrices for the Canon EOS D60, its white balance in Kelvin; CRW imports name their camera; LR-BEHAV-ACCESS: the Flatpak may talk to the accessibility bus; LR-LIB-ALBUM: album commands leave out photos the library no longer has and record nothing when they add nothing; LR-BEHAV-DRAGDROP: a right-click on a text field while photos are carried from the grid opens no Cut / Copy / Paste menu; LR-BEHAV-DRAGDROP: a right-click while photos are carried from the grid opens no context menu; LR-LIB-VIRTUALCOPY: default numbering is preflighted across the requested batch and explicit names bypass suffix allocation; LR-BEHAV-DRAGDROP: only the primary button drags photos out of the grid and drops them; KEYC-RATING: deleting a keyword while Enable Filters has the filter put aside no longer leaves an empty one there; LR-BEHAV-DRAGDROP: a drop's destination is named with its parent when another folder or album has the same name; LR-VIEW-FILMSTRIP: photos drag from the filmstrip to the same sidebar targets as from the grid; LRC-LIB-LIBFOLDERS: a deep row's count follows its name in a narrow panel; LR-FILT-FLAG / LRC-LIB-FILTERBAR: the flag toggles combine, Picked + Unflagged, #2588; LR-VIEW-PHOTOGRID, LR-VIEW-FILMSTRIP, LR-VIEW-COMPARE, LR-VIEW-SURVEY, LRC-LIB-REFVIEW, LRC-LIB-PEOPLE: the active photo's selection ring shows on light themes and in Compare, Survey and the face tiles, and needs a loaded thumbnail; LR-FILT-SEARCH-META: `>=` / `<=` in fielded search tokens; MENU-FILE-CLOSE, KEY-CLOSEWINDOW: the notes say Close Window quits the app, after the same check as Quit; MENU-EDIT-SELECTNONE, MENU-EDIT-PASTESELECTED: Edit lists Select Only Active Photo after Deselect All and Paste Settings from Previous after Paste Selected Settings…; on macOS no doubled separators are left where Settings… moved to the app menu; LR-BEHAV-TITLEBAR: a double-click follows the system setting, and a panel over the bar keeps the window still; KEYC-METADATA, LRC-LIB-KEYWORDS: ⌥0 / ⌥⇧0 go to the next / previous keyword set; LRC-DEV-HISTOGRAM-DRAG: the histogram's five tonal zones — hover for the value, drag to adjust the Basic control the press locked, double-click to reset; LR-VIEW-HISTOGRAM note updated; MENU-BAR: on macOS a menu item that yields its key to a focused text field or to shortcut recording really lets it go; LR-MERGE-HDR / LR-MERGE-PANO: Lanczos-3 warps; LR-IMP-LOCAL: browsing from the sidebar records the folder; LR-VIEW-ZOOM: only a full-size drag within 20 ms clears a slow-drag verdict; LR-IMP-ADD-DIALOG: hierarchical import album destination reconciled with main, #2362; KEY-SLIDER-NUDGE: vertical wheel nudges need a click or drag in the row; leaving it disarms them; LR-EDIT-OPTICS-PROFILE: Fujifilm nine-knot lens tables read at the plain radius; LR-VIEW-ZOOM: only a full-size drag within 20 ms clears a slow-drag verdict; LRC-LIB-LIBFOLDERS, LR-BEHAV-BG-TASKS: Synchronize Folder's scan shows its folder listing and stops it on ✕; LRC-LIB-LIBFOLDERS: Synchronize Folder says a folder is up to date; LR-LIB-STACK: filmstrip stack badge click; LRC-LIB-REFVIEW / CATALOG-SWITCH: Escape preserves edits; catalog-local pins; deleted and malformed references refused; LRC-LIB-COMPARE: the Info panel labels the active photo while culling, #2398; KEYC-RATING: `\` shows / hides the Filter Bar in the grids; LR-VIEW-PHOTOGRID: the grid reads only the photos an import batch or an edit touched, not the whole list; earlier: LRC-PRINT-PAGE: Print Page Options; LR-META-XMP: fewer false unimported-XMP warnings; KEYC-METADATA: ⌘K / Ctrl+K Add Keywords; LRC-DEV-HISTOGRAM-DRAG: the histogram's five tonal zones — hover for the value, drag to adjust the Basic control the press locked, double-click to reset; LR-VIEW-HISTOGRAM note updated; MENU-BAR: on macOS a menu item that yields its key to a focused text field or to shortcut recording really lets it go; LR-MERGE-HDR / LR-MERGE-PANO: Lanczos-3 warps; LR-IMP-LOCAL: browsing from the sidebar records the folder; LR-VIEW-ZOOM: only a full-size drag within 20 ms clears a slow-drag verdict; LR-IMP-ADD-DIALOG: hierarchical import album destination reconciled with main, #2362; KEY-SLIDER-NUDGE: vertical wheel nudges need a click or drag in the row; leaving it disarms them; LR-EDIT-OPTICS-PROFILE: Fujifilm nine-knot lens tables read at the plain radius; LR-VIEW-ZOOM: only a full-size drag within 20 ms clears a slow-drag verdict; LRC-LIB-LIBFOLDERS, LR-BEHAV-BG-TASKS: Synchronize Folder's scan shows its folder listing and stops it on ✕; LRC-LIB-LIBFOLDERS: Synchronize Folder says a folder is up to date; LR-LIB-STACK: filmstrip stack badge click; LRC-LIB-REFVIEW / CATALOG-SWITCH: Escape preserves edits; catalog-local pins; deleted and malformed references refused; LRC-LIB-COMPARE: the Info panel labels the active photo while culling, #2398; KEYC-RATING: `\` shows / hides the Filter Bar in the grids; LR-VIEW-PHOTOGRID: the grid reads only the photos an import batch or an edit touched, not the whole list; earlier: LRC-PRINT-PAGE: Print Page Options; LR-META-XMP: fewer false unimported-XMP warnings; LR-META-LOCATION: Info GPS text is fully validated before metadata changes; LR-MAP-TRACKLOG decodes GPX character references in point fields; malformed references error; CDATA stays literal; LR-BEHAV-ACCESS: the Thumbnail size control is a slider to screen readers; LR-BEHAV-ACCESS: the Detail photo is named with its file name; LR-VIEW-PHOTOGRID: the grid's empty-view message names its shortcut as this platform writes it and as it is bound; LR-BEHAV-ACCESS: the grid's date headers are named as they read; LR-BEHAV-ACCESS: the sidebar's disclosure triangles name their row; KEYC-METADATA, LRC-LIB-KEYWORDS: ⌘⌥⇧K Set Keyword Shortcut and ⇧K; KEYC-METADATA: ⌘⇧K / Ctrl+Shift+K Edit Keywords; LR-FILT-SEARCH-META: `>=` / `<=` in fielded search tokens; MENU-FILE-CLOSE, KEY-CLOSEWINDOW: the notes say Close Window quits the app, after the same check as Quit; MENU-EDIT-SELECTNONE, MENU-EDIT-PASTESELECTED: Edit lists Select Only Active Photo after Deselect All and Paste Settings from Previous after Paste Selected Settings…; on macOS no doubled separators are left where Settings… moved to the app menu; LR-BEHAV-TITLEBAR: a double-click follows the system setting, and a panel over the bar keeps the window still; KEYC-METADATA, LRC-LIB-KEYWORDS: ⌥0 / ⌥⇧0 go to the next / previous keyword set; LRC-DEV-HISTOGRAM-DRAG: the histogram's five tonal zones — hover for the value, drag to adjust the Basic control the press locked, double-click to reset; LR-VIEW-HISTOGRAM note updated; MENU-BAR: on macOS a menu item that yields its key to a focused text field or to shortcut recording really lets it go; LR-MERGE-HDR / LR-MERGE-PANO: Lanczos-3 warps; LR-IMP-LOCAL: browsing from the sidebar records the folder; LR-VIEW-ZOOM: only a full-size drag within 20 ms clears a slow-drag verdict; LR-IMP-ADD-DIALOG: hierarchical import album destination reconciled with main, #2362; KEY-SLIDER-NUDGE: vertical wheel nudges need a click or drag in the row; leaving it disarms them; LR-EDIT-OPTICS-PROFILE: Fujifilm nine-knot lens tables read at the plain radius; LR-VIEW-ZOOM: only a full-size drag within 20 ms clears a slow-drag verdict; LRC-LIB-LIBFOLDERS, LR-BEHAV-BG-TASKS: Synchronize Folder's scan shows its folder listing and stops it on ✕; LRC-LIB-LIBFOLDERS: Synchronize Folder says a folder is up to date; LR-LIB-STACK: filmstrip stack badge click; LRC-LIB-REFVIEW / CATALOG-SWITCH: Escape preserves edits; catalog-local pins; deleted and malformed references refused; LRC-LIB-COMPARE: the Info panel labels the active photo while culling, #2398; KEYC-RATING: `\` shows / hides the Filter Bar in the grids; LR-VIEW-PHOTOGRID: the grid reads only the photos an import batch or an edit touched, not the whole list; earlier: LRC-PRINT-PAGE: Print Page Options; LR-META-XMP: fewer false unimported-XMP warnings; KEYC-METADATA: ⌘K / Ctrl+K Add Keywords; LRC-DEV-HISTOGRAM-DRAG: the histogram's five tonal zones — hover for the value, drag to adjust the Basic control the press locked, double-click to reset; LR-VIEW-HISTOGRAM note updated; MENU-BAR: on macOS a menu item that yields its key to a focused text field or to shortcut recording really lets it go; LR-MERGE-HDR / LR-MERGE-PANO: Lanczos-3 warps; LR-IMP-LOCAL: browsing from the sidebar records the folder; LR-VIEW-ZOOM: only a full-size drag within 20 ms clears a slow-drag verdict; LR-IMP-ADD-DIALOG: hierarchical import album destination reconciled with main, #2362; KEY-SLIDER-NUDGE: vertical wheel nudges need a click or drag in the row; leaving it disarms them; LR-EDIT-OPTICS-PROFILE: Fujifilm nine-knot lens tables read at the plain radius; LR-VIEW-ZOOM: only a full-size drag within 20 ms clears a slow-drag verdict; LRC-LIB-LIBFOLDERS, LR-BEHAV-BG-TASKS: Synchronize Folder's scan shows its folder listing and stops it on ✕; LRC-LIB-LIBFOLDERS: Synchronize Folder says a folder is up to date; LR-LIB-STACK: filmstrip stack badge click; LRC-LIB-REFVIEW / CATALOG-SWITCH: Escape preserves edits; catalog-local pins; deleted and malformed references refused; LRC-LIB-COMPARE: the Info panel labels the active photo while culling, #2398; KEYC-RATING: `\` shows / hides the Filter Bar in the grids; LR-VIEW-PHOTOGRID: the grid reads only the photos an import batch or an edit touched, not the whole list; earlier: LRC-PRINT-PAGE: Print Page Options; LR-META-XMP: fewer false unimported-XMP warnings; LRC-LIB-FOLDERS: Local's built-in places (Pictures, Desktop, Downloads, Home) offer no Rename Folder… or Move Folder To…, and `folder.rename` / `folder.move` refuse the home folder and its Pictures, Desktop and Downloads; LRC-LIB-CATALOG-SWITCH: a library's catalog.snap or folder given at launch opens that library; LR-IMP-ADD-DIALOG / LR-IMP-MOVE: checked rename-sequence exhaustion; LR-IMP-ADD-DIALOG: an import asks the disk about the probes a scan left, and for each photo's sidecar, up to 16 files at a time; LRC-LIB-LIBFOLDERS: Don't ask again in the Move Photos question is kept only when the move started; a refused move leaves the question on; LR-IMP-ADD-DIALOG: remembered import thumbnail size, density-aware bounded previews and one job per candidate/size bucket (#2237, part of #2056); import loupe remains open; counts unchanged; LR-FILT-SEARCH-META: `>=` / `<=` in fielded search tokens; MENU-FILE-CLOSE, KEY-CLOSEWINDOW: the notes say Close Window quits the app, after the same check as Quit; MENU-EDIT-SELECTNONE, MENU-EDIT-PASTESELECTED: Edit lists Select Only Active Photo after Deselect All and Paste Settings from Previous after Paste Selected Settings…; on macOS no doubled separators are left where Settings… moved to the app menu; LR-BEHAV-TITLEBAR: a double-click follows the system setting, and a panel over the bar keeps the window still; KEYC-METADATA, LRC-LIB-KEYWORDS: ⌥0 / ⌥⇧0 go to the next / previous keyword set; LRC-DEV-HISTOGRAM-DRAG: the histogram's five tonal zones — hover for the value, drag to adjust the Basic control the press locked, double-click to reset; LR-VIEW-HISTOGRAM note updated; MENU-BAR: on macOS a menu item that yields its key to a focused text field or to shortcut recording really lets it go; LR-MERGE-HDR / LR-MERGE-PANO: Lanczos-3 warps; LR-IMP-LOCAL: browsing from the sidebar records the folder; LR-VIEW-ZOOM: only a full-size drag within 20 ms clears a slow-drag verdict; LR-IMP-ADD-DIALOG: hierarchical import album destination reconciled with main, #2362; KEY-SLIDER-NUDGE: vertical wheel nudges need a click or drag in the row; leaving it disarms them; LR-EDIT-OPTICS-PROFILE: Fujifilm nine-knot lens tables read at the plain radius; LR-VIEW-ZOOM: only a full-size drag within 20 ms clears a slow-drag verdict; LRC-LIB-LIBFOLDERS, LR-BEHAV-BG-TASKS: Synchronize Folder's scan shows its folder listing and stops it on ✕; LRC-LIB-LIBFOLDERS: Synchronize Folder says a folder is up to date; LR-LIB-STACK: filmstrip stack badge click; LRC-LIB-REFVIEW / CATALOG-SWITCH: Escape preserves edits; catalog-local pins; deleted and malformed references refused; LRC-LIB-COMPARE: the Info panel labels the active photo while culling, #2398; KEYC-RATING: `\` shows / hides the Filter Bar in the grids; LR-VIEW-PHOTOGRID: the grid reads only the photos an import batch or an edit touched, not the whole list; earlier: LRC-PRINT-PAGE: Print Page Options; LR-META-XMP: fewer false unimported-XMP warnings; KEYC-METADATA: ⌘K / Ctrl+K Add Keywords; LRC-DEV-HISTOGRAM-DRAG: the histogram's five tonal zones — hover for the value, drag to adjust the Basic control the press locked, double-click to reset; LR-VIEW-HISTOGRAM note updated; MENU-BAR: on macOS a menu item that yields its key to a focused text field or to shortcut recording really lets it go; LR-MERGE-HDR / LR-MERGE-PANO: Lanczos-3 warps; LR-IMP-LOCAL: browsing from the sidebar records the folder; LR-VIEW-ZOOM: only a full-size drag within 20 ms clears a slow-drag verdict; LR-IMP-ADD-DIALOG: hierarchical import album destination reconciled with main, #2362; KEY-SLIDER-NUDGE: vertical wheel nudges need a click or drag in the row; leaving it disarms them; LR-EDIT-OPTICS-PROFILE: Fujifilm nine-knot lens tables read at the plain radius; LR-VIEW-ZOOM: only a full-size drag within 20 ms clears a slow-drag verdict; LRC-LIB-LIBFOLDERS, LR-BEHAV-BG-TASKS: Synchronize Folder's scan shows its folder listing and stops it on ✕; LRC-LIB-LIBFOLDERS: Synchronize Folder says a folder is up to date; LR-LIB-STACK: filmstrip stack badge click; LRC-LIB-REFVIEW / CATALOG-SWITCH: Escape preserves edits; catalog-local pins; deleted and malformed references refused; LRC-LIB-COMPARE: the Info panel labels the active photo while culling, #2398; KEYC-RATING: `\` shows / hides the Filter Bar in the grids; LR-VIEW-PHOTOGRID: the grid reads only the photos an import batch or an edit touched, not the whole list; earlier: LRC-PRINT-PAGE: Print Page Options; LR-META-XMP: fewer false unimported-XMP warnings; LR-MERGE-HDR / LR-MERGE-PANO / LR-MERGE-HDRPANO: the dialog makes a new preview each time it opens, also after a failed one, #2708; LR-FILT-SEARCH-META: `>=` / `<=` in fielded search tokens; MENU-FILE-CLOSE, KEY-CLOSEWINDOW: the notes say Close Window quits the app, after the same check as Quit; MENU-EDIT-SELECTNONE, MENU-EDIT-PASTESELECTED: Edit lists Select Only Active Photo after Deselect All and Paste Settings from Previous after Paste Selected Settings…; on macOS no doubled separators are left where Settings… moved to the app menu; LR-BEHAV-TITLEBAR: a double-click follows the system setting, and a panel over the bar keeps the window still; KEYC-METADATA, LRC-LIB-KEYWORDS: ⌥0 / ⌥⇧0 go to the next / previous keyword set; LRC-DEV-HISTOGRAM-DRAG: the histogram's five tonal zones — hover for the value, drag to adjust the Basic control the press locked, double-click to reset; LR-VIEW-HISTOGRAM note updated; MENU-BAR: on macOS a menu item that yields its key to a focused text field or to shortcut recording really lets it go; LR-MERGE-HDR / LR-MERGE-PANO: Lanczos-3 warps; LR-IMP-LOCAL: browsing from the sidebar records the folder; LR-VIEW-ZOOM: only a full-size drag within 20 ms clears a slow-drag verdict; LR-IMP-ADD-DIALOG: hierarchical import album destination reconciled with main, #2362; KEY-SLIDER-NUDGE: vertical wheel nudges need a click or drag in the row; leaving it disarms them; LR-EDIT-OPTICS-PROFILE: Fujifilm nine-knot lens tables read at the plain radius; LR-VIEW-ZOOM: only a full-size drag within 20 ms clears a slow-drag verdict; LRC-LIB-LIBFOLDERS, LR-BEHAV-BG-TASKS: Synchronize Folder's scan shows its folder listing and stops it on ✕; LRC-LIB-LIBFOLDERS: Synchronize Folder says a folder is up to date; LR-LIB-STACK: filmstrip stack badge click; LRC-LIB-REFVIEW / CATALOG-SWITCH: Escape preserves edits; catalog-local pins; deleted and malformed references refused; LRC-LIB-COMPARE: the Info panel labels the active photo while culling, #2398; KEYC-RATING: `\` shows / hides the Filter Bar in the grids; LR-VIEW-PHOTOGRID: the grid reads only the photos an import batch or an edit touched, not the whole list; earlier: LRC-PRINT-PAGE: Print Page Options; LR-META-XMP: fewer false unimported-XMP warnings; KEYC-METADATA: ⌘K / Ctrl+K Add Keywords; LRC-DEV-HISTOGRAM-DRAG: the histogram's five tonal zones — hover for the value, drag to adjust the Basic control the press locked, double-click to reset; LR-VIEW-HISTOGRAM note updated; MENU-BAR: on macOS a menu item that yields its key to a focused text field or to shortcut recording really lets it go; LR-MERGE-HDR / LR-MERGE-PANO: Lanczos-3 warps; LR-IMP-LOCAL: browsing from the sidebar records the folder; LR-VIEW-ZOOM: only a full-size drag within 20 ms clears a slow-drag verdict; LR-IMP-ADD-DIALOG: hierarchical import album destination reconciled with main, #2362; KEY-SLIDER-NUDGE: vertical wheel nudges need a click or drag in the row; leaving it disarms them; LR-EDIT-OPTICS-PROFILE: Fujifilm nine-knot lens tables read at the plain radius; LR-VIEW-ZOOM: only a full-size drag within 20 ms clears a slow-drag verdict; LRC-LIB-LIBFOLDERS, LR-BEHAV-BG-TASKS: Synchronize Folder's scan shows its folder listing and stops it on ✕; LRC-LIB-LIBFOLDERS: Synchronize Folder says a folder is up to date; LR-LIB-STACK: filmstrip stack badge click; LRC-LIB-REFVIEW / CATALOG-SWITCH: Escape preserves edits; catalog-local pins; deleted and malformed references refused; LRC-LIB-COMPARE: the Info panel labels the active photo while culling, #2398; KEYC-RATING: `\` shows / hides the Filter Bar in the grids; LR-VIEW-PHOTOGRID: the grid reads only the photos an import batch or an edit touched, not the whole list; earlier: LRC-PRINT-PAGE: Print Page Options; LR-META-XMP: fewer false unimported-XMP warnings; LRC-LIB-MISSING: `library.missing` from the control channel checks the files on a worker thread instead of the UI thread; LR-FILT-SEARCH-META: `>=` / `<=` in fielded search tokens; MENU-FILE-CLOSE, KEY-CLOSEWINDOW: the notes say Close Window quits the app, after the same check as Quit; MENU-EDIT-SELECTNONE, MENU-EDIT-PASTESELECTED: Edit lists Select Only Active Photo after Deselect All and Paste Settings from Previous after Paste Selected Settings…; on macOS no doubled separators are left where Settings… moved to the app menu; LR-BEHAV-TITLEBAR: a double-click follows the system setting, and a panel over the bar keeps the window still; KEYC-METADATA, LRC-LIB-KEYWORDS: ⌥0 / ⌥⇧0 go to the next / previous keyword set; LRC-DEV-HISTOGRAM-DRAG: the histogram's five tonal zones — hover for the value, drag to adjust the Basic control the press locked, double-click to reset; LR-VIEW-HISTOGRAM note updated; MENU-BAR: on macOS a menu item that yields its key to a focused text field or to shortcut recording really lets it go; LR-MERGE-HDR / LR-MERGE-PANO: Lanczos-3 warps; LR-IMP-LOCAL: browsing from the sidebar records the folder; LR-VIEW-ZOOM: only a full-size drag within 20 ms clears a slow-drag verdict; LR-IMP-ADD-DIALOG: hierarchical import album destination reconciled with main, #2362; KEY-SLIDER-NUDGE: vertical wheel nudges need a click or drag in the row; leaving it disarms them; LR-EDIT-OPTICS-PROFILE: Fujifilm nine-knot lens tables read at the plain radius; LR-VIEW-ZOOM: only a full-size drag within 20 ms clears a slow-drag verdict; LRC-LIB-LIBFOLDERS, LR-BEHAV-BG-TASKS: Synchronize Folder's scan shows its folder listing and stops it on ✕; LRC-LIB-LIBFOLDERS: Synchronize Folder says a folder is up to date; LR-LIB-STACK: filmstrip stack badge click; LRC-LIB-REFVIEW / CATALOG-SWITCH: Escape preserves edits; catalog-local pins; deleted and malformed references refused; LRC-LIB-COMPARE: the Info panel labels the active photo while culling, #2398; KEYC-RATING: `\` shows / hides the Filter Bar in the grids; LR-VIEW-PHOTOGRID: the grid reads only the photos an import batch or an edit touched, not the whole list; earlier: LRC-PRINT-PAGE: Print Page Options; LR-META-XMP: fewer false unimported-XMP warnings; KEYC-METADATA: ⌘K / Ctrl+K Add Keywords; LRC-DEV-HISTOGRAM-DRAG: the histogram's five tonal zones — hover for the value, drag to adjust the Basic control the press locked, double-click to reset; LR-VIEW-HISTOGRAM note updated; MENU-BAR: on macOS a menu item that yields its key to a focused text field or to shortcut recording really lets it go; LR-MERGE-HDR / LR-MERGE-PANO: Lanczos-3 warps; LR-IMP-LOCAL: browsing from the sidebar records the folder; LR-VIEW-ZOOM: only a full-size drag within 20 ms clears a slow-drag verdict; LR-IMP-ADD-DIALOG: hierarchical import album destination reconciled with main, #2362; KEY-SLIDER-NUDGE: vertical wheel nudges need a click or drag in the row; leaving it disarms them; LR-EDIT-OPTICS-PROFILE: Fujifilm nine-knot lens tables read at the plain radius; LR-VIEW-ZOOM: only a full-size drag within 20 ms clears a slow-drag verdict; LRC-LIB-LIBFOLDERS, LR-BEHAV-BG-TASKS: Synchronize Folder's scan shows its folder listing and stops it on ✕; LRC-LIB-LIBFOLDERS: Synchronize Folder says a folder is up to date; LR-LIB-STACK: filmstrip stack badge click; LRC-LIB-REFVIEW / CATALOG-SWITCH: Escape preserves edits; catalog-local pins; deleted and malformed references refused; LRC-LIB-COMPARE: the Info panel labels the active photo while culling, #2398; KEYC-RATING: `\` shows / hides the Filter Bar in the grids; LR-VIEW-PHOTOGRID: the grid reads only the photos an import batch or an edit touched, not the whole list; earlier: LRC-PRINT-PAGE: Print Page Options; LR-META-XMP: fewer false unimported-XMP warnings; LR-CROP-AUTO: Auto straighten reads the vertical vanishing point and the horizon; LR-META-EXIF-FULL: reject extra EXIF clock components so invalid capture-time candidates fall through to valid metadata; LR-FILT-SEARCH-META: `>=` / `<=` in fielded search tokens; MENU-FILE-CLOSE, KEY-CLOSEWINDOW: the notes say Close Window quits the app, after the same check as Quit; MENU-EDIT-SELECTNONE, MENU-EDIT-PASTESELECTED: Edit lists Select Only Active Photo after Deselect All and Paste Settings from Previous after Paste Selected Settings…; on macOS no doubled separators are left where Settings… moved to the app menu; LR-BEHAV-TITLEBAR: a double-click follows the system setting, and a panel over the bar keeps the window still; KEYC-METADATA, LRC-LIB-KEYWORDS: ⌥0 / ⌥⇧0 go to the next / previous keyword set; LRC-DEV-HISTOGRAM-DRAG: the histogram's five tonal zones — hover for the value, drag to adjust the Basic control the press locked, double-click to reset; LR-VIEW-HISTOGRAM note updated; MENU-BAR: on macOS a menu item that yields its key to a focused text field or to shortcut recording really lets it go; LR-MERGE-HDR / LR-MERGE-PANO: Lanczos-3 warps; LR-IMP-LOCAL: browsing from the sidebar records the folder; LR-VIEW-ZOOM: only a full-size drag within 20 ms clears a slow-drag verdict; LR-IMP-ADD-DIALOG: hierarchical import album destination reconciled with main, #2362; KEY-SLIDER-NUDGE: vertical wheel nudges need a click or drag in the row; leaving it disarms them; LR-EDIT-OPTICS-PROFILE: Fujifilm nine-knot lens tables read at the plain radius; LR-VIEW-ZOOM: only a full-size drag within 20 ms clears a slow-drag verdict; LRC-LIB-LIBFOLDERS, LR-BEHAV-BG-TASKS: Synchronize Folder's scan shows its folder listing and stops it on ✕; LRC-LIB-LIBFOLDERS: Synchronize Folder says a folder is up to date; LR-LIB-STACK: filmstrip stack badge click; LRC-LIB-REFVIEW / CATALOG-SWITCH: Escape preserves edits; catalog-local pins; deleted and malformed references refused; LRC-LIB-COMPARE: the Info panel labels the active photo while culling, #2398; KEYC-RATING: `\` shows / hides the Filter Bar in the grids; LR-VIEW-PHOTOGRID: the grid reads only the photos an import batch or an edit touched, not the whole list; earlier: LRC-PRINT-PAGE: Print Page Options; LR-META-XMP: fewer false unimported-XMP warnings; KEYC-METADATA: ⌘K / Ctrl+K Add Keywords; LRC-DEV-HISTOGRAM-DRAG: the histogram's five tonal zones — hover for the value, drag to adjust the Basic control the press locked, double-click to reset; LR-VIEW-HISTOGRAM note updated; MENU-BAR: on macOS a menu item that yields its key to a focused text field or to shortcut recording really lets it go; LR-MERGE-HDR / LR-MERGE-PANO: Lanczos-3 warps; LR-IMP-LOCAL: browsing from the sidebar records the folder; LR-VIEW-ZOOM: only a full-size drag within 20 ms clears a slow-drag verdict; LR-IMP-ADD-DIALOG: hierarchical import album destination reconciled with main, #2362; KEY-SLIDER-NUDGE: vertical wheel nudges need a click or drag in the row; leaving it disarms them; LR-EDIT-OPTICS-PROFILE: Fujifilm nine-knot lens tables read at the plain radius; LR-VIEW-ZOOM: only a full-size drag within 20 ms clears a slow-drag verdict; LRC-LIB-LIBFOLDERS, LR-BEHAV-BG-TASKS: Synchronize Folder's scan shows its folder listing and stops it on ✕; LRC-LIB-LIBFOLDERS: Synchronize Folder says a folder is up to date; LR-LIB-STACK: filmstrip stack badge click; LRC-LIB-REFVIEW / CATALOG-SWITCH: Escape preserves edits; catalog-local pins; deleted and malformed references refused; LRC-LIB-COMPARE: the Info panel labels the active photo while culling, #2398; KEYC-RATING: `\` shows / hides the Filter Bar in the grids; LR-VIEW-PHOTOGRID: the grid reads only the photos an import batch or an edit touched, not the whole list; earlier: LRC-PRINT-PAGE: Print Page Options; LR-META-XMP: fewer false unimported-XMP warnings; LR-META-XMP / LRC-MAP-TRACKLOG: malformed ISO times are not projected or matched; LR-IMP-FORMATS: JPEG ICC APP2 chunk validation; LR-BEHAV-TITLEBAR: a dialog leaves the title bar undimmed, as a macOS sheet does; LR-BEHAV-ACCESS: the bars are toolbars and the panels landmarks for screen readers; LR-BEHAV-ACCESS: every dialog and prompt is exposed as a dialog (a modal one, an alert dialog for the prompts); LR-BEHAV-ACCESS: icon-only buttons are named by their label, with the shortcut as the description; MENU-HELP-HELP: on macOS the Help menu is registered with the application only once the menu bar is installed (muda ignores the call before), so AppKit knows which menu is Help; LR-BEHAV-ACCESS: a slider's value field is named ` value`, with how to use it as its description; MENU-BAR: every shortcut the app shows on macOS lists its modifiers ⌃⌥⇧⌘, as the menu bar does; MENU-BAR: on macOS two menu rows that run the same command each get their own native row and label (File ▸ Export… and File ▸ Export with Preset ▸ Custom…; File ▸ New Album… and Photo ▸ Add to Album ▸ New Album…); LR-BEHAV-DIALOGS: a one-line prompt that is refused stays open and says why, and a New Folder on Disk says it was made; LR-FILT-SEARCH-META: `>=` / `<=` in fielded search tokens; MENU-FILE-CLOSE, KEY-CLOSEWINDOW: the notes say Close Window quits the app, after the same check as Quit; MENU-EDIT-SELECTNONE, MENU-EDIT-PASTESELECTED: Edit lists Select Only Active Photo after Deselect All and Paste Settings from Previous after Paste Selected Settings…; on macOS no doubled separators are left where Settings… moved to the app menu; LR-BEHAV-TITLEBAR: a double-click follows the system setting, and a panel over the bar keeps the window still; KEYC-METADATA, LRC-LIB-KEYWORDS: ⌥0 / ⌥⇧0 go to the next / previous keyword set; LRC-DEV-HISTOGRAM-DRAG: the histogram's five tonal zones — hover for the value, drag to adjust the Basic control the press locked, double-click to reset; LR-VIEW-HISTOGRAM note updated; MENU-BAR: on macOS a menu item that yields its key to a focused text field or to shortcut recording really lets it go; LR-MERGE-HDR / LR-MERGE-PANO: Lanczos-3 warps; LR-IMP-LOCAL: browsing from the sidebar records the folder; LR-VIEW-ZOOM: only a full-size drag within 20 ms clears a slow-drag verdict; LR-IMP-ADD-DIALOG: hierarchical import album destination reconciled with main, #2362; KEY-SLIDER-NUDGE: vertical wheel nudges need a click or drag in the row; leaving it disarms them; LR-EDIT-OPTICS-PROFILE: Fujifilm nine-knot lens tables read at the plain radius; LR-VIEW-ZOOM: only a full-size drag within 20 ms clears a slow-drag verdict; LRC-LIB-LIBFOLDERS, LR-BEHAV-BG-TASKS: Synchronize Folder's scan shows its folder listing and stops it on ✕; LRC-LIB-LIBFOLDERS: Synchronize Folder says a folder is up to date; LR-LIB-STACK: filmstrip stack badge click; LRC-LIB-REFVIEW / CATALOG-SWITCH: Escape preserves edits; catalog-local pins; deleted and malformed references refused; LRC-LIB-COMPARE: the Info panel labels the active photo while culling, #2398; KEYC-RATING: `\` shows / hides the Filter Bar in the grids; LR-VIEW-PHOTOGRID: the grid reads only the photos an import batch or an edit touched, not the whole list; earlier: LRC-PRINT-PAGE: Print Page Options; LR-META-XMP: fewer false unimported-XMP warnings; KEYC-METADATA: ⌘K / Ctrl+K Add Keywords; LRC-DEV-HISTOGRAM-DRAG: the histogram's five tonal zones — hover for the value, drag to adjust the Basic control the press locked, double-click to reset; LR-VIEW-HISTOGRAM note updated; MENU-BAR: on macOS a menu item that yields its key to a focused text field or to shortcut recording really lets it go; LR-MERGE-HDR / LR-MERGE-PANO: Lanczos-3 warps; LR-IMP-LOCAL: browsing from the sidebar records the folder; LR-VIEW-ZOOM: only a full-size drag within 20 ms clears a slow-drag verdict; LR-IMP-ADD-DIALOG: hierarchical import album destination reconciled with main, #2362; KEY-SLIDER-NUDGE: vertical wheel nudges need a click or drag in the row; leaving it disarms them; LR-EDIT-OPTICS-PROFILE: Fujifilm nine-knot lens tables read at the plain radius; LR-VIEW-ZOOM: only a full-size drag within 20 ms clears a slow-drag verdict; LRC-LIB-LIBFOLDERS, LR-BEHAV-BG-TASKS: Synchronize Folder's scan shows its folder listing and stops it on ✕; LRC-LIB-LIBFOLDERS: Synchronize Folder says a folder is up to date; LR-LIB-STACK: filmstrip stack badge click; LRC-LIB-REFVIEW / CATALOG-SWITCH: Escape preserves edits; catalog-local pins; deleted and malformed references refused; LRC-LIB-COMPARE: the Info panel labels the active photo while culling, #2398; KEYC-RATING: `\` shows / hides the Filter Bar in the grids; LR-VIEW-PHOTOGRID: the grid reads only the photos an import batch or an edit touched, not the whole list; earlier: LRC-PRINT-PAGE: Print Page Options; LR-META-XMP: fewer false unimported-XMP warnings; LR-VIEW-HISTOGRAM: the shadow and highlight clipping buttons take their tile and chevron from the theme, so the chevron reads in every theme; LR-FILT-SEARCH-META: `>=` / `<=` in fielded search tokens; MENU-FILE-CLOSE, KEY-CLOSEWINDOW: the notes say Close Window quits the app, after the same check as Quit; MENU-EDIT-SELECTNONE, MENU-EDIT-PASTESELECTED: Edit lists Select Only Active Photo after Deselect All and Paste Settings from Previous after Paste Selected Settings…; on macOS no doubled separators are left where Settings… moved to the app menu; LR-BEHAV-TITLEBAR: a double-click follows the system setting, and a panel over the bar keeps the window still; KEYC-METADATA, LRC-LIB-KEYWORDS: ⌥0 / ⌥⇧0 go to the next / previous keyword set; LRC-DEV-HISTOGRAM-DRAG: the histogram's five tonal zones — hover for the value, drag to adjust the Basic control the press locked, double-click to reset; LR-VIEW-HISTOGRAM note updated; MENU-BAR: on macOS a menu item that yields its key to a focused text field or to shortcut recording really lets it go; LR-MERGE-HDR / LR-MERGE-PANO: Lanczos-3 warps; LR-IMP-LOCAL: browsing from the sidebar records the folder; LR-VIEW-ZOOM: only a full-size drag within 20 ms clears a slow-drag verdict; LR-IMP-ADD-DIALOG: hierarchical import album destination reconciled with main, #2362; KEY-SLIDER-NUDGE: vertical wheel nudges need a click or drag in the row; leaving it disarms them; LR-EDIT-OPTICS-PROFILE: Fujifilm nine-knot lens tables read at the plain radius; LR-VIEW-ZOOM: only a full-size drag within 20 ms clears a slow-drag verdict; LRC-LIB-LIBFOLDERS, LR-BEHAV-BG-TASKS: Synchronize Folder's scan shows its folder listing and stops it on ✕; LRC-LIB-LIBFOLDERS: Synchronize Folder says a folder is up to date; LR-LIB-STACK: filmstrip stack badge click; LRC-LIB-REFVIEW / CATALOG-SWITCH: Escape preserves edits; catalog-local pins; deleted and malformed references refused; LRC-LIB-COMPARE: the Info panel labels the active photo while culling, #2398; KEYC-RATING: `\` shows / hides the Filter Bar in the grids; LR-VIEW-PHOTOGRID: the grid reads only the photos an import batch or an edit touched, not the whole list; earlier: LRC-PRINT-PAGE: Print Page Options; LR-META-XMP: fewer false unimported-XMP warnings; KEYC-METADATA: ⌘K / Ctrl+K Add Keywords; LRC-DEV-HISTOGRAM-DRAG: the histogram's five tonal zones — hover for the value, drag to adjust the Basic control the press locked, double-click to reset; LR-VIEW-HISTOGRAM note updated; MENU-BAR: on macOS a menu item that yields its key to a focused text field or to shortcut recording really lets it go; LR-MERGE-HDR / LR-MERGE-PANO: Lanczos-3 warps; LR-IMP-LOCAL: browsing from the sidebar records the folder; LR-VIEW-ZOOM: only a full-size drag within 20 ms clears a slow-drag verdict; LR-IMP-ADD-DIALOG: hierarchical import album destination reconciled with main, #2362; KEY-SLIDER-NUDGE: vertical wheel nudges need a click or drag in the row; leaving it disarms them; LR-EDIT-OPTICS-PROFILE: Fujifilm nine-knot lens tables read at the plain radius; LR-VIEW-ZOOM: only a full-size drag within 20 ms clears a slow-drag verdict; LRC-LIB-LIBFOLDERS, LR-BEHAV-BG-TASKS: Synchronize Folder's scan shows its folder listing and stops it on ✕; LRC-LIB-LIBFOLDERS: Synchronize Folder says a folder is up to date; LR-LIB-STACK: filmstrip stack badge click; LRC-LIB-REFVIEW / CATALOG-SWITCH: Escape preserves edits; catalog-local pins; deleted and malformed references refused; LRC-LIB-COMPARE: the Info panel labels the active photo while culling, #2398; KEYC-RATING: `\` shows / hides the Filter Bar in the grids; LR-VIEW-PHOTOGRID: the grid reads only the photos an import batch or an edit touched, not the whole list; earlier: LRC-PRINT-PAGE: Print Page Options; LR-META-XMP: fewer false unimported-XMP warnings; LR-FILT-SEARCH-META: `>=` / `<=` in fielded search tokens; MENU-FILE-CLOSE, KEY-CLOSEWINDOW: the notes say Close Window quits the app, after the same check as Quit; MENU-EDIT-SELECTNONE, MENU-EDIT-PASTESELECTED: Edit lists Select Only Active Photo after Deselect All and Paste Settings from Previous after Paste Selected Settings…; on macOS no doubled separators are left where Settings… moved to the app menu; LR-BEHAV-TITLEBAR: a double-click follows the system setting, and a panel over the bar keeps the window still; KEYC-METADATA, LRC-LIB-KEYWORDS: ⌥0 / ⌥⇧0 go to the next / previous keyword set; LRC-DEV-HISTOGRAM-DRAG: the histogram's five tonal zones — hover for the value, drag to adjust the Basic control the press locked, double-click to reset; LR-VIEW-HISTOGRAM note updated; MENU-BAR: on macOS a menu item that yields its key to a focused text field or to shortcut recording really lets it go; LR-MERGE-HDR / LR-MERGE-PANO: Lanczos-3 warps; LR-IMP-LOCAL: browsing from the sidebar records the folder; LR-VIEW-ZOOM: only a full-size drag within 20 ms clears a slow-drag verdict; LR-IMP-ADD-DIALOG: hierarchical import album destination reconciled with main, #2362; KEY-SLIDER-NUDGE: vertical wheel nudges need a click or drag in the row; leaving it disarms them; LR-EDIT-OPTICS-PROFILE: Fujifilm nine-knot lens tables read at the plain radius; LR-VIEW-ZOOM: only a full-size drag within 20 ms clears a slow-drag verdict; LRC-LIB-LIBFOLDERS, LR-BEHAV-BG-TASKS: Synchronize Folder's scan shows its folder listing and stops it on ✕; LRC-LIB-LIBFOLDERS: Synchronize Folder says a folder is up to date; LR-LIB-STACK: filmstrip stack badge click; LRC-LIB-REFVIEW / CATALOG-SWITCH: Escape preserves edits; catalog-local pins; deleted and malformed references refused; LRC-LIB-COMPARE: the Info panel labels the active photo while culling, #2398; KEYC-RATING: `\` shows / hides the Filter Bar in the grids; LR-VIEW-PHOTOGRID: the grid reads only the photos an import batch or an edit touched, not the whole list; earlier: LRC-PRINT-PAGE: Print Page Options; LR-META-XMP: fewer false unimported-XMP warnings; KEYC-METADATA: ⌘K / Ctrl+K Add Keywords; LRC-DEV-HISTOGRAM-DRAG: the histogram's five tonal zones — hover for the value, drag to adjust the Basic control the press locked, double-click to reset; LR-VIEW-HISTOGRAM note updated; MENU-BAR: on macOS a menu item that yields its key to a focused text field or to shortcut recording really lets it go; LR-MERGE-HDR / LR-MERGE-PANO: Lanczos-3 warps; LR-IMP-LOCAL: browsing from the sidebar records the folder; LR-VIEW-ZOOM: only a full-size drag within 20 ms clears a slow-drag verdict; LR-IMP-ADD-DIALOG: hierarchical import album destination reconciled with main, #2362; KEY-SLIDER-NUDGE: vertical wheel nudges need a click or drag in the row; leaving it disarms them; LR-EDIT-OPTICS-PROFILE: Fujifilm nine-knot lens tables read at the plain radius; LR-VIEW-ZOOM: only a full-size drag within 20 ms clears a slow-drag verdict; LRC-LIB-LIBFOLDERS, LR-BEHAV-BG-TASKS: Synchronize Folder's scan shows its folder listing and stops it on ✕; LRC-LIB-LIBFOLDERS: Synchronize Folder says a folder is up to date; LR-LIB-STACK: filmstrip stack badge click; LRC-LIB-REFVIEW / CATALOG-SWITCH: Escape preserves edits; catalog-local pins; deleted and malformed references refused; LRC-LIB-COMPARE: the Info panel labels the active photo while culling, #2398; KEYC-RATING: `\` shows / hides the Filter Bar in the grids; LR-VIEW-PHOTOGRID: the grid reads only the photos an import batch or an edit touched, not the whole list; earlier: LRC-PRINT-PAGE: Print Page Options; LR-META-XMP: fewer false unimported-XMP warnings; LR-MASK-BRUSH: extreme brush segments skip the intermediate dabs; LR-MASK-BRUSH: with the mask overlay hidden, the stroke being painted no longer tints the photo) · **Target:** Adobe Lightroom Classic 15.5 / Lightroom 9.6 The measured, row-by-row checklist: one row per Lightroom feature, menu item and shortcut, what we have and what is missing. It answers *does the feature exist?* (feature breadth). Whether it is good @@ -52,10 +52,10 @@ cited here still exists. Formerly docs/parity.md. | X. Cross-cutting behaviours (BEHAV) | 23 | 6 | 1 | 1 | 8/8 (100%) | 7/10 (70%) | | Y. Menus | 85 | 0 | 4 | 8 | 47/47 (100%) | 23/23 (100%) | | Z. Keyboard shortcuts (desktop) | 78 | 4 | 2 | 1 | 49/52 (94%) | 24/26 (92%) | -| Lightroom Classic extras | 41 | 25 | 21 | 8 | — | 24/26 (92%) | -| **Total** | 417 | 69 | 57 | 35 | 192/203 (95%) | 146/161 (91%) | +| Lightroom Classic extras | 41 | 26 | 21 | 8 | — | 24/27 (89%) | +| **Total** | 417 | 70 | 57 | 35 | 192/203 (95%) | 146/162 (90%) | -Weighted completion (✅ = 1, 🟡 = ½, 🚫 left out): **83.1%** of 543 in-scope rows — P0 97.3% of 203 · P1 94.4% of 161 · P2 56.7% of 178. +Weighted completion (✅ = 1, 🟡 = ½, 🚫 left out): **83.1%** of 544 in-scope rows — P0 97.3% of 203 · P1 94.1% of 162 · P2 56.7% of 178. ## Top gaps and shortcut notes @@ -78,11 +78,11 @@ differences and missing bindings are in [ui-parity.md](ui-parity.md#shortcuts). | LR-IMP-RAWDEFAULT | Raw defaults | P1 | ✅ | `cmd:library.preferences`, `crates/engine/src/import.rs`, `crates/ui-egui/src/dialogs/settings.rs` | LightCraft default / a preset / per camera (make + model); non-raw default too; the preset look counts as unedited and Reset returns to it | | LR-IMP-MIGRATE | Migrate other catalogs | OOS | 🚫 | | | | LR-IMP-PROFILES | Import profiles & presets | P1 | ✅ | `cmd:file.importPresets`, `cmd:preset.import`, `cmd:profile.import`, `crates/engine/src/preset_import.rs`, `crates/engine/src/preset_luminar.rs` | presets: .lcpreset, XMP, classic .lrtemplate (local adjustment templates become mask effects, LR-MASK-EFFECTS), photos carrying edits ("DNG presets"), Luminar looks (.lmp, .mplumpack collections; sliders with a counterpart), .zip bundles, folders (folder → group), drag & drop (an `.xmp` dropped together with its photo is that photo's sidecar and is not imported as a preset, issue #936); masks carried over; unmapped settings reported (fields that only describe the preset or photo, e.g. `Cluster`, `SortName`, `SupportsAmount2`, the as-shot white, empty Point Color slots, are not); profiles: .cube 3D LUTs (one toast reports both halves; a LUT that fails is named with its reason); a preset that names a profile (`crs:Look`) applies the imported profile of that name, imported before or after it, and Lightroom's basic profiles map to ours (`crates/engine/src/profile_refs.rs`, issue #1001); Adobe profile formats deliberately unsupported (so `.xmp` creative profiles' colour tables aren't read) | -| LR-IMP-LOCAL | Work on files in place | P0 | ✅ | `cmd:library.browse`, `cmd:photo.addToLibrary`, `cmd:library.import` (mode add), `cmd:local.addRoot`, `cmd:local.hide`, `cmd:local.restoreHidden`, `crates/engine/src/cmd/browse.rs`, `crates/ui-egui/src/panels/left.rs` (`local_section`), `crates/ui-egui/src/panels/grid.rs` (`folder_header`) | Local: browse Pictures / Desktop / Downloads / Home or any folder without adding it (breadcrumb, Include subfolders, Add N to My Photos); browsed photos stay out of All Photos, albums and counts (album commands skip them and the album menus say to Add to My Photos first, issue #684); folders picked with Browse Folder… (or Keep in Local / Add to Local on a folder row) stay listed across restarts, and browsing a folder inside a listed one highlights it inside that tree (opened down to it, siblings reachable) instead of adding a root; edits go to XMP sidecars; importing promotes them; right-click a top-level location → Remove from Local hides the shortcut only (nothing on disk or in the catalog changes), saved with the UI state, and “Show N hidden locations” (or Browse Folder… on it) brings it back, each on its own from the row's list when several are hidden, or all at once with Restore All (issue #2114); a folder is one location however its path is spelled (`/` or `\`, trailing separator, `.`/`..`, drive-letter case; `folder_key` in `crates/catalog/src/query.rs`); browsed photos nobody changed are forgotten once their folder has not been browsed for 30 days (Settings → Performance → Local folders, 0 = never; checked when the library opens, or `cmd:library.forgetLocal` with `dryRun`; files and sidecars stay, browsing again brings them back; see `crates/catalog/src/local.rs` for what counts as a change); browsing a folder again forgets at once the untouched records of its files that another app deleted or moved, so they no longer fail to load on every visit (issue #1392; changed ones stay, shown as missing); browsing a folder keeps the active photo when the folder shows it, else makes the folder's first photo active as it arrives, the same rule as albums and the other sources (`Session::keep_active_in_view`; issue #1447: before, a library photo stayed active, so Photo Grid marked nothing and Detail, the histogram and the info showed a photo the folder doesn't hold) | +| LR-IMP-LOCAL | Work on files in place | P0 | ✅ | `cmd:library.browse`, `cmd:photo.addToLibrary`, `cmd:library.import` (mode add), `cmd:local.addRoot`, `cmd:local.hide`, `cmd:local.restoreHidden`, `crates/engine/src/cmd/browse.rs`, `crates/ui-egui/src/panels/left.rs` (`local_section`), `crates/ui-egui/src/panels/grid.rs` (`folder_header`), `crates/engine/src/import.rs` (`browse_queue`) | Local: browse Pictures / Desktop / Downloads / Home or any folder without adding it (breadcrumb, Include subfolders, Add N to My Photos); the folder's photos join the view batch by batch while they are read, first screen first (`browse_queue`: newest file first for the default sort), so a 2,933-raw wedding folder shows its first photos in 0.9 s instead of after every file was read (30.6 s; the whole folder still takes ~30 s); browsed photos stay out of All Photos, albums and counts (album commands skip them and the album menus say to Add to My Photos first, issue #684); folders picked with Browse Folder… (or Keep in Local / Add to Local on a folder row) stay listed across restarts, and browsing a folder inside a listed one highlights it inside that tree (opened down to it, siblings reachable) instead of adding a root; edits go to XMP sidecars; importing promotes them; right-click a top-level location → Remove from Local hides the shortcut only (nothing on disk or in the catalog changes), saved with the UI state, and “Show N hidden locations” (or Browse Folder… on it) brings it back, each on its own from the row's list when several are hidden, or all at once with Restore All (issue #2114); a folder is one location however its path is spelled (`/` or `\`, trailing separator, `.`/`..`, drive-letter case; `folder_key` in `crates/catalog/src/query.rs`); browsed photos nobody changed are forgotten once their folder has not been browsed for 30 days (Settings → Performance → Local folders, 0 = never; checked when the library opens, or `cmd:library.forgetLocal` with `dryRun`; files and sidecars stay, browsing again brings them back; see `crates/catalog/src/local.rs` for what counts as a change); browsing a folder again forgets at once the untouched records of its files that another app deleted or moved, so they no longer fail to load on every visit (issue #1392; changed ones stay, shown as missing); browsing a folder keeps the active photo when the folder shows it, else makes the folder's first photo active as it arrives, the same rule as albums and the other sources (`Session::keep_active_in_view`; issue #1447: before, a library photo stayed active, so Photo Grid marked nothing and Detail, the histogram and the info showed a photo the folder doesn't hold) | | LR-IMP-SIDECAR-SPLIT | Separate XMP sidecar variants | P2 | ⬜ | `cmd:library.xmpPreferences` | sidecar naming option exists (stem/full), no split sidecars | +| LR-IMP-PREVIEW-READ | Preview reads the header, not the raw | P0 | ✅ | `crates/raw/src/preview.rs` (`bounded_preview`, `embedded_preview_from_range`, `embedded_preview_covering`), `crates/engine/src/files.rs` (`preview_window`), `docs/raw-preview-reads.md`, `crates/engine/src/files.rs` (`preview_window`, `planned_preview`), `crates/raw/src/sparse.rs` (`preview_plan`), `crates/raw/src/dng.rs` (`decode_reduced`), `crates/codecs/src/lib.rs` (`decode_jpeg_srgb8_fit`) | a grid thumbnail and a loupe preview of an unedited raw show the camera's embedded JPEG, which sits near the start of the file (a 24 MP Fuji RAF keeps its 5.6 MB JPEG in a 56 MB file), so the preview loader reads the header and that JPEG instead of the whole file: 56.5 MB → 5.9 MB and ~10 s → 0.96 s measured on one real RAF over a GVFS/SMB Wi-Fi share (byte-identical preview pixels; `LIGHTCRAFT_PROFILE=1` reports the bytes); the RAF pointer is read from its fixed header fields, a CR3's boxes and a TIFF raw whose preview tag sits past the prefix are read through `lightcraft_raw::sparse::preview_plan`, which names the ranges `embedded_preview` reads and nothing else; a cached render as stand-in, an edited photo and a raw without an embedded JPEG still render; the formats it cannot describe (CRW/MRW/X3F are found by scanning, NEF/ORF/PEF probe from samples) keep the whole-file path, and a lying pointer, a short read or a file that changes underneath us falls back to it rather than losing the preview. An unedited raw's **grid thumbnail is that camera JPEG itself**, so no sensor decode is scheduled behind it (`Session::embedded_thumb_is_final`, `grid::skips_the_real_render`): drawing a 512 px thumbnail used to decode the whole raw — 45 s a photo at 1.2 MB/s, 8.3 GB for a 149-raw folder — and the queue it built delayed the render the user was waiting for. Stand-ins and grid thumbnails decode the smallest embedded JPEG that covers the requested size and is a copy of the largest (aspect ratio within 1 %, same component count), not always the largest, on the whole-file and the plan route alike: an A7R VI's 1616 × 1080 JPEG (0.3 MB) instead of its 9984 × 6656 one (5.6 MB), mid-sized NEF / RW2 JPEGs, and a Canon CR3's 1620 px `PRVW` (or 160 px `THMB`) instead of its 8192 px JPEG track (`cr3_preview_box_list`; EOS R5 C at 1280 px: 168–322 → 31 ms a photo); a plain 8-bit sRGB camera JPEG decodes straight to 8-bit (`decode_jpeg_srgb8_fit`, no float image), and the decoded camera JPEG of an unedited raw is kept in the library's thumbnail cache, so a folder seen before fills from small files (250 R5 C raws: first screen 553 → 294 ms, second visit 610 → 53 ms); a DNG that keeps reduced-resolution copies of its LinearRaw image (Lightroom HDR/panorama merges) develops a thumbnail or preview from the smallest copy that covers it (512 px of a 23836 × 7078 pano: 38 s → 1.1 s) | | LR-IMP-FORMATS | Supported formats | P0 | 🟡 | `crates/codecs/src/lib.rs`, `crates/raw/src/lib.rs`, `crates/raw/src/jxl.rs` | JPEG (ICC APP2 sequences require unique complete chunks with consistent totals; malformed profiles use the existing unsupported-profile sRGB fallback; a Samsung panorama whose scan data runs into its SEF trailer with no end-of-image marker reads as the image before the trailer, while a JPEG cut short is still refused, issue #3350, `crates/codecs/src/jpeg.rs`), PNG, TIFF (1- to 64-bit integer, 16- to 64-bit float, incl. bilevel and CCITT fax), WebP, JXL, PSD, GIF, BMP (lossy wide-gamut JXL decodes to the right colours: jxl-oxide desaturated XYB rendered to P3/Rec. 2020, worked around in `crates/codecs/src/jxl.rs`); DNG (incl. lossy / Smart Preview DNG and DNG 1.7 JPEG XL tiles/strips: lossless ones decode bit-exactly on synthetic CFA and LinearRaw files, lossy ones are restored to the declared primaries and transfer curve (linear, sRGB, BT.709 or a power; PQ / HLG stay preview-only) without clipping or gamut mapping, checked on one real file with 108 lossy tiles and on 2 Lightroom Classic 15.1 lossy LinearRaw DNGs (ILCE-7CM2) whose tiles declare Rec. 2100 primaries with the sRGB curve, which were preview-only before, and any tile that fails to decode makes the file preview-only; DNG 1.7 row/column-interleaved rasters (`RowInterleaveFactor` / `ColumnInterleaveFactor`) are undone before the mosaic is read, so a file from Adobe's converter with a 2×2 interleave no longer decodes as four scaled-down copies in the wrong colours; a dimension the factor does not divide decodes with short trailing fields; verified on 4 real ILCE-6400 JPEG XL DNGs rendering bit-identical to their non-interleaved twins, factors above 2 rest on the plain reading of the tag; a single-chunk JPEG XL preview IFD is offered as the embedded preview; a DNG whose samples pile up at one value well below its stated `WhiteLevel`, as a camera app's 12-bit Bayer DNG that states 4095 and clips at 3567, has those samples raised to the white level on decode, so highlight reconstruction sees the clipped areas instead of white balance tinting them magenta, issue #548, `crates/raw/src/saturation.rs`), CR2 (colour-filter layout from the file's `CR2CFAPattern` tag, issue #85), CRW (CIFF, the 2000–2004 Canon bodies: clean-room-derived decode frozen in `craft-libs crates/crw/clean-room/02-crw-format.md`, `crates/raw/src/vendor/crw.rs`; EOS D30/D60/10D/300D, PowerShot G1 (10-bit GBRG) and G6 pixel-exact against a black-box reference on 28 of 29 corpus files incl. 18 private D60s; as-shot white balance from the ColorBalance record's ShotInfo-selected slot; black level from the darkest flat masked-border strip; rotation from ImageInfo; starting look fitted from the file's own embedded JPEG like CR2/CR3 (`camera_preview::file_local_look`), accepted on all 10 decodable public corpus files (holdout RMS 0.009–0.047), with the fitted matrix absorbing the white balance on the bodies without a ColorBalance record (D30, PowerShots); the PowerShot G5's anomalous entropy coding and the pre-CIFF compacts stay preview-only, and G2/G3/G4/Pro1/S-series decode unverified for lack of redistributable samples), ARW (pre-2017 bodies: white balance and black level from the encrypted `SR2SubIFD`, issue #148, its `WB_RGGBLevels` being the white balance the camera applied, issue #535; the DSLR-A900's packed 12-bit strip, two pixels per three bytes, read only when the strip is exactly 1.5 bytes per pixel, with its camera-JPEG window centred, checked on one CC0 file (A850 unmeasured); downsized Sony lossless YCbCr 4:2:0 / 4:2:2 tiles decoded to linear RGB, verified on private ILCE-7M4 3:2 & 4:3 examples including 16-bit full-resolution export; Sony DSC-R1 .SR2 decodes, its big-endian 16-bit words read in the order that fits the sample range, checked on the CC0 3221 sample; its crop and white balance live in the SR2SubIFD and are not read yet; ARW6 Compressed / Compressed (HQ) of the ILCE-7RM6 (`crates/raw/src/vendor/arw6.rs`, `crates/raw/src/llvc.rs`; 12-bit codes exact against a black-box oracle on 8 CC0 files, FF over the full frame, APS-C in the interior (LibRaw's own edge rows differ); output through our own companding curve, measured from CC0 lossless / ARW6 pairs; geometry checked against the camera JPEG, `docs/arw6-compression.md`; the ILCE-7M5 writes the same codec, unverified)), NEF (uncompressed, lossless and lossy compressed 12/14-bit, `crates/raw/src/vendor/nefc.rs`), RAF (uncompressed, lossless/lossy compressed Bayer and X-Trans; `crates/raw/src/vendor/rafc.rs`, full sensor verification on 13 bodies, `docs/raf-compression.md`), RW2 / Leica RWL / Panasonic RAW in every raw format (compressed 4 and 6, the prefix-coded strips of format 8 from the GH6 on, packed 2/5/7, the 16-bit words of the 2005–2007 bodies; black levels, in-camera aspect crops and mapped-out defect pixels; 178 CC0 files from 118 bodies checked, `crates/raw/src/vendor/rw2.rs`), PEF, ORF (uncompressed; colour-filter layout from the file's Exif `CFAPattern`; 12.8 bit/px 16-byte blocks of ten 12-bit samples + a zero pad byte, E-300 / E-330 / E-500 verified on CC0 files, E-M5 Mark II / PEN-F high-res files decode smooth but CFA and black unverified), Kodak DCS520C TIFF (a two-component 12-bit lossless JPEG strip, the two components the even and odd columns; the file's `GrayResponseCurve` table linearises the companded samples to 12 bits, black measured per CFA position, white balance from the `KodakIFD`; `RawFormat::KodakDcs`, `crates/raw/src/vendor/kodak.rs`, checked on the one CC0 sample against its IFD0 thumbnail), Hasselblad 3FR and FFF (a TIFF without a DNG version tag that describes its CFA IFD, levels, crop and white balance with DNG's tags; `RawFormat::CfaTiff`, read by `dng::decode_as`, RGGB when the file states no `CFAPattern`, checked against the camera preview on the CC0 H4D-40 FFF and CFV 3FR; an FFF has no colour matrix of its own and takes the camera's from the original 3FR header it embeds in tag `0xC51B`; neither carries a JPEG, so their uncompressed 8-bit RGB preview (1217 × 913 in the H4D-40 FFF, 218 × 218 in the CFV 3FR) is the embedded preview, handed on as a baseline TIFF, `crates/raw/src/vendor/hasselblad.rs`): uncompressed (X1D, X1D II, X2D, CFV 100C) and Hasselblad's lossless JPEG with scan selection value 8 (every FFF; H3D, H4D and CFV-50c 3FR), `ljpeg::decode_hasselblad`, its rule measured on 14 CC0 files with the ljpeg-explorer tool, Minolta / Konica Minolta MRW (the `\0MRM` block container: `PRD` sensor size, 12-bit packed or 16-bit-word storage and the RGGB / GBRG cell, `WBG` gains in cell order, the `TTW` TIFF for Exif and orientation, black 0 and white from the saturation plateau; `crates/raw/src/vendor/mrw.rs`, DiMAGE 7, 7Hi, A1, A2, A200 and Dynax / Maxxum 7D checked on CC0 files against their previews), Samsung SRW without compression (NX5, NX10, NX11 and the NX20-generation bodies NX20, NX200, NX210, NX1000 and NX1100 as 12-bit packed samples in either bit order, EX1 and WB2000 as 16-bit words; layout, camera-JPEG framing, levels and white balance from the maker note, `crates/raw/src/vendor/srw.rs`; median ΔE vs the camera JPEG 2.2–7.6 on 11 of the 12 files from 10 bodies checked, 10.7 (5.7 registered) on the backlit WB2000 sample). 12-bit NEF black level: maker note `0x003d` is in 14-bit units (D750, D780, D850, D7500, Z 50 checked). Lossy type 2 NEF curves of version `0x44 0x40` (D780, D6, Z 5, Z 6, Z 7, Z 7_2) span a quarter of the sample range, read from the table's own point spacing. Import probes (`probe_info`) of compressed NEFs decode only the first 128 rows, the ones the masked-trailing-column test reads in both the probe and the full decode (the D5100 and D7000 write 8 masked columns at 0 with no maker-note tag for them, compressed and uncompressed alike), instead of the whole Huffman stream: ~3 ms instead of the full decode time per file (386 ms on a 24 MP D750 file), issue #708. CR3 lossless CRX Bayer and version 0x100/0x200 C-RAW decodes to sensor pixels with timed white balance, black/white levels and file crop (EOS R6 Mark III `ColorData` version 66: as-shot white balance and black level from the file, white level from the data, `docs/cr3.md`) (the shot's aspect ratio from the maker note's `AspectInfo`); full sensor samples match the reference on M50/R100/R8/RP, incl. RP 1.6× crop mode (`crates/raw/tests/it/cr3_corpus.rs`, `docs/cr3.md`). EOS R5 Mark II C-RAW and RAW decode too (the QP map's first-symbol literal and 23-zero escape, odd-height Bayer planes): 4 CC0 files pinned, every QP map of 33,665 private C-RAW files decodes exactly, and full-frame images match the camera JPEG. Unsupported coding variants retain metadata and use the embedded JPEG when available; HEVC preview decoding remains unsupported. Missing: CR3 unverified CRX variants, compressed ORF, compressed Samsung SRW (NX1, NX30, NX300, NX500, NX2000, NX3000, NX3300, NX mini: preview only with the reason), Nikon High Efficiency NEF (HE / HE★ from the Z 8, Z 9, Z 6III and Z f: compression 34713 with a JPEG XS codestream and no `0x0096` table; no public specification, so preview only with the reason "Nikon High Efficiency NEF", issue #193; Lossless compressed NEFs from the same bodies decode), NEF "lossy after split" (preview only: imported with `preview_only` = the decoder's reason, rendered as a rendered JPEG, flagged by a grid badge and loupe/Edit/Info notices, `previewOnly` in `catalog.query`; Reload clears it once the file decodes, and the notices carry a Reload from Disk button; each flag is stamped with the app version that set it, and a library opened in another version rechecks its preview-only raws in the background, so a raw a newer release decodes is cleared without a manual Reload, issue #662, `library.recheckPreviewOnly`), TIFF-based raws whose raw data sits in a private block (Phase One / Leaf IIQ, Canon EOS-1D / 1Ds and the Kodak DCS Pro / DCS760C TIFFs (compression 65000): IFD0 is only a reduced RGB image next to a much larger declared picture) are recognised from that structure and import as `preview_only` with the reason, instead of opening as a thumbnail-sized ordinary image (`thumbnail_shell` in `crates/raw/src/lib.rs`), Recognised raws LightCraft cannot decode open as `preview_only` with any usable embedded JPEG, also Sigma X3F (container recognised by magic, preview found by a structural JPEG scan), the Canon CRWs the decoder refuses (PowerShot G5, pre-CIFF compacts), Kodak KDC (IFD0 without `ImageWidth`), Leaf MOS and Epson ERF (scan; ones with no JPEG still fail with "without an embedded preview", AVIF decode (HEIC decodes only behind the non-default `heif` feature, which the release packages enable: `crates/heif`, heic-rs for the container and HEVC (bit-exact against libde265 on iPhone tiles) plus our own colour conversion, matching libheif within 1 code value on iPhone-style grids (VUI full range, Display P3 ICC, nclx, 10-bit, irot/imir/clap, gain map/depth ignored); 4:2:0/monochrome only, 4:2:2/4:4:4 Canon/Sony HIF refused; same policy as PhotoCraft), palette (indexed-colour) TIFF (refused by the `tiff` crate); folder discovery and file pickers include Sony SR2 and SRF, with the same extension list used by CLI/MCP: DSC-R1 SR2 decodes from sensor data; DSC-F828 SRF stays preview-only; transparency is flattened onto white, in linear light, where the working image is built (`to_working` in `crates/codecs/src/lib.rs`): renders, previews, exports and contact sheets no longer turn transparent areas black, issue #1212; median ΔE vs the camera JPEG 2.2–7.6 on 11 of the 12 files from 10 bodies checked, 10.7 (5.7 registered) on the backlit WB2000 sample). 12-bit NEF black level: maker note `0x003d` is in 14-bit units (D750, D780, D850, D7500, Z 50 checked). Lossy type 2 NEF curves of version `0x44 0x40` (D780, D6, Z 5, Z 6, Z 7, Z 7_2) span a quarter of the sample range, read from the table's own point spacing. Import probes (`probe_info`) of compressed NEFs decode only the first 128 rows, the ones the masked-trailing-column test reads in both the probe and the full decode (the D5100 and D7000 write 8 masked columns at 0 with no maker-note tag for them, compressed and uncompressed alike), instead of the whole Huffman stream: ~3 ms instead of the full decode time per file (386 ms on a 24 MP D750 file), issue #708. CR3 lossless CRX Bayer and version 0x100/0x200 C-RAW decodes to sensor pixels with timed white balance, black/white levels and file crop (the shot's aspect ratio from the maker note's `AspectInfo`); the Hasselblad Lunar, a Sony NEX-7 under Make HASSELBLAD, decodes as an ARW: a HASSELBLAD file with Sony's raw coding (compression 32767 or tag `0x7010`) goes to the ARW reader, and the body for its `Tag2010` layout comes from the maker note's `SonyModelID`, checked on the CC0 4051 file, issue #1275; an uncompressed MOS, whose IFD0 is the 16-bit mosaic with no Make tag, is recognised by Leaf's data block `0x8606` instead of opening as a greyscale image of its mosaic, issue #1275), and any decode failure of a recognised raw, not only an unsupported variant Nikon 12-bit small RAW (D810 / D4S): packed Y0/Y1/Cb/Cr rows decode to linear RGB; header probes retain geometry without decoding pixels, and truncated strips are refused. Chroma gains are approximate. | | LR-IMP-CAMERA-COVERAGE | Camera coverage, verified per model | P0 | 🟡 | `crates/raw/tests/it/corpus.rs`, `crates/raw/tests/it/coverage_sweep_corpus.rs`, `docs/raw/evidence.tsv`, `docs/raw/coverage-sweep.tsv`, `xtask/src/main.rs` | a supported container is not the same as every camera that writes it decoding correctly: ~55 CC0 corpus files plus 6 exact-sensor CR3 cases and 4 pinned EOS R5 Mark II files (plus ~60 NEFs checked by hand and 178 Panasonic / Leica RW2, RWL and RAW files from 118 bodies, checked against each file's own JPEG) against the >1,000 models Lightroom lists; per-model differences still surface (e.g. the CR2 colour-filter layout differed by model until it was read from the file's own tag, issue #85, checked on 35 CR2 bodies; CR2 black levels were read from border columns light already reaches, left of `SensorInfo`'s border, so CR2s rendered dark with crushed shadows (EOS 6D: 2591 instead of 2047), now measured from the masked columns only, checked on 8 corpus bodies; pre-2017 Sony ARWs opened green until their scrambled white balance / black level were read, issue #148, checked on 16 ARWs from 15 Sony bodies; their white balance now comes from the encrypted `SR2SubIFD`'s `WB_RGGBLevels`, the gains the camera applied, which 14 bodies (DSLR-A450 … A700, NEX-3/5/C3, SLT-A33/A35/A55, ILCE-6001, UMC-R10C) had no other source for and which the maker-note estimate read before missed by up to 100 % on preset shots, issue #535, checked against ExifTool on 100 decoded ARWs from 87 Sony bodies; the encrypted SR2 black level is now read from its directory entry (the DSLR-A450/A500/A550/A700 read 354–975 instead of 512), checked on 126 ARWs from 97 Sony bodies, and downsized lossless M/S ARWs use their measured black level 1024 instead of 512, checked on 12 files from 4 bodies, issue #535, and on a fifth, the ILCE-7CM2 (issue #697: 0.5.0 rendered its lossless M file with that 512 pedestal, the darkest half of a dusk scene +11 L* and blue, the lamplight white; its darkest 0.1 % per channel sits at 1017/1020/1024, and with 1024 the render is within +3 L* of the camera JPEG; the body's compressed and uncompressed "M-size" files are Bayer at 4736×3132 with black 512, two public ones in the corpus); pre-2017 Sony bodies shot in the camera's 16:9 mode (ILCE-7SM2, DSLR-A580) now get the 16:9 crop of their camera JPEG instead of the whole 3:2 frame with padding columns, issue #535; Canon PowerShot and EOS M CR2s whose `ColorData` is stored as bytes opened green until the bytes were read as 16-bit words, 25 files from 25 bodies checked; private Z 8 crop verified; the Nikon D4, which records no CropArea, gets its image area and white level measured on 148 NEFs against the camera JPEG (optical-black stripes and magenta clipped highlights gone); the D7100, whose data holds picture beyond the camera's frame, develops at Lightroom's frame (the camera JPEG's size) in full frame and 1.3x crop, so Lightroom crops line up (#1248).; CR2 white levels came from masked rows holding samples at 16383, so on the EOS 50D, 60D, 450D (Rebel XSi), 500D (Rebel T1i), 550D (Rebel T2i), 1200D (Rebel T5) and 100D (Rebel SL1) clipped highlights sat at 81–98% of the range and were not treated as clipped, now taken from the active area only, issue #1272). Canon CRW verified pixel-exact on 6 bodies (D30, D60, 10D, 300D, G1, G6; 10 CC0 files in the corpus, `crates/raw/tests/it/corpus.rs`, plus 18 maintainer D60 files via the digest-pinned sample test of craft-crw in craft-libs (crates/crw/tests/samples.rs there); `craft-libs crates/crw/clean-room/03-verification.md`). Compressed RAF sensor arrays verified on 13 Fujifilm bodies (`docs/raf-compression.md`). ILCE-7RM6 ARW6 12-bit codes verified on 8 CC0 files (FF and APS-C, both compression modes). Sony ILCE-7CR (61 MP): lossless compressed L and M and compressed ARW2 samples of one scene in the corpus (black level, default crop, as-shot gains, lossless vs compressed mosaic agreement, fitted camera look), and all nine public codings (also uncompressed, small and APS-C crop) decode and fit. Grow the CC0 corpus per model and test each sample decodes with plausible colour. Nikon CropArea and untagged-preview colour-space regression tests; the Nikon D4, which records no CropArea, gets its image area and white level measured on 148 NEFs against the camera JPEG (optical-black stripes and magenta clipped highlights gone). Pentax PEFs without area tags (KP, KF, K-3 Mark III, K-70, K-1, K-1 Mark II, K-S1, K-S2, 645Z) no longer keep their masked border: black, empty and saturated lines are trimmed against the maker-note BlackPoint, issue #1271, measured on 38 CC0 PEFs from 30 bodies.; Canon PowerShot and EOS M CR2s whose `ColorData` is stored as bytes opened green until the bytes were read as 16-bit words, 25 files from 25 bodies checked). Compressed RAF sensor arrays verified on 13 Fujifilm bodies (`docs/raf-compression.md`). ILCE-7RM6 ARW6 12-bit codes verified on 8 CC0 files (FF and APS-C, both compression modes). Sony ILCE-7CR (61 MP): lossless compressed L and M and compressed ARW2 samples of one scene in the corpus (black level, default crop, as-shot gains, lossless vs compressed mosaic agreement, fitted camera look), and all nine public codings (also uncompressed, small and APS-C crop) decode and fit. Grow the CC0 corpus per model and test each sample decodes with plausible colour. Nikon CropArea and untagged-preview colour-space regression tests; the Nikon D4, which records no CropArea, gets its image area and white level measured on 148 NEFs against the camera JPEG (optical-black stripes and magenta clipped highlights gone). The Canon PowerShot G9, which writes no `ColorData`, takes its as-shot white balance from the WBInfo row (`0x0029`) of the preset ShotInfo names, issue #1273, checked on 1 CC0 file against neutral areas of the camera JPEG (within 0.1 %).; the Nikon D4, which records no CropArea, gets its image area and white level measured on 148 NEFs against the camera JPEG (optical-black stripes and magenta clipped highlights gone). The Olympus E-10 (no ImageProcessing directory) takes its as-shot gains and its black level (`0x1012` in `0x102c` ValidBits units: 128 at 12 bits) from the main maker note, issue #1273: its render against the camera thumbnail goes from ΔE76 18.5 with a rejected fit to 11.5 with the fit accepted (measured by AlexChambers).; the Nikon D4, which records no CropArea, gets its image area and white level measured on 148 NEFs against the camera JPEG (optical-black stripes and magenta clipped highlights gone), and the Nikon D90 likewise on 10 of 550 NEFs (42 filler columns on the right that developed as a pink band; green clips below red and blue).; the Nikon D4, which records no CropArea, gets its image area and white level measured on 148 NEFs against the camera JPEG (optical-black stripes and magenta clipped highlights gone). Public raw.pixls.us coverage sweep (#1327): 30 more models verified on 40 CC0 files, with SHA-256 and FNV-1a sensor hashes pinned (`crates/raw/tests/it/coverage_sweep_corpus.rs`, `docs/raw/coverage-sweep.tsv`; `docs/raw/coverage.py`: 515 → 545 verified rows). Render/reference luminance correlation is 0.6801 to 0.9998; eleven candidates excluded for identity, provenance or missing/mismatched reference images. Nikon D6 / Z 6 / Z 7 uncompressed and Z 8 HE variants remain preview-only. Own-matrix DNG tone differs from the camera JPEG (#1277); sensor verification is not colour calibration. | -| LR-IMP-PREVIEW-READ | Preview reads the header, not the raw | P0 | ✅ | `crates/raw/src/preview.rs` (`bounded_preview`, `embedded_preview_from_range`, `embedded_preview_covering`), `crates/engine/src/files.rs` (`preview_window`), `docs/raw-preview-reads.md`, `crates/engine/src/files.rs` (`preview_window`, `planned_preview`), `crates/raw/src/sparse.rs` (`preview_plan`), `crates/raw/src/dng.rs` (`decode_reduced`) | a grid thumbnail and a loupe preview of an unedited raw show the camera's embedded JPEG, which sits near the start of the file (a 24 MP Fuji RAF keeps its 5.6 MB JPEG in a 56 MB file), so the preview loader reads the header and that JPEG instead of the whole file: 56.5 MB → 5.9 MB and ~10 s → 0.96 s measured on one real RAF over a GVFS/SMB Wi-Fi share (byte-identical preview pixels; `LIGHTCRAFT_PROFILE=1` reports the bytes); the RAF pointer is read from its fixed header fields, a CR3's boxes and a TIFF raw whose preview tag sits past the prefix are read through `lightcraft_raw::sparse::preview_plan`, which names the ranges `embedded_preview` reads and nothing else; a cached render as stand-in, an edited photo and a raw without an embedded JPEG still render; the formats it cannot describe (CRW/MRW/X3F are found by scanning, NEF/ORF/PEF probe from samples) keep the whole-file path, and a lying pointer, a short read or a file that changes underneath us falls back to it rather than losing the preview. An unedited raw's **grid thumbnail is that camera JPEG itself**, so no sensor decode is scheduled behind it (`Session::embedded_thumb_is_final`, `grid::skips_the_real_render`): drawing a 512 px thumbnail used to decode the whole raw — 45 s a photo at 1.2 MB/s, 8.3 GB for a 149-raw folder — and the queue it built delayed the render the user was waiting for. Stand-ins and grid thumbnails decode the smallest embedded JPEG that covers the requested size and is a copy of the largest (aspect ratio within 1 %, same component count), not always the largest, on the whole-file and the plan route alike: an A7R VI's 1616 × 1080 JPEG (0.3 MB) instead of its 9984 × 6656 one (5.6 MB), and mid-sized NEF / RW2 JPEGs; a DNG that keeps reduced-resolution copies of its LinearRaw image (Lightroom HDR/panorama merges) develops a thumbnail or preview from the smallest copy that covers it (512 px of a 23836 × 7078 pano: 38 s → 1.1 s) | | LR-IMP-CULL-AT-IMPORT | Culling analysis at import | P2 | 🟡 | `cmd:photo.analyze` | run Assisted Culling on the imported photos (they're selected after an import); not automatic | | LR-IMP-MOVE | Move on import [Classic] | P1 | ✅ | `cmd:library.import` (`mode` move), `crates/engine/src/import_move.rs`, `crates/engine/src/tests/import_move.rs`, `crates/ui-egui/src/dialogs/import.rs` | Transfer ▸ Move with Copy's destination, folders (day / month / one folder / custom template), rename template and example destination; XMP sidecars (both namings) move along; each source is removed only after its destination is written (hard link on the same volume, else copied, synced and compared byte for byte) and its catalog record is saved; failed, duplicate, unchecked files keep their sources, taken names get -1, -2…, never overwritten; if the rename sequence runs out, later files are reported as failed and earlier completed moves remain catalogued; files already in the destination/library are added in place; sources that can't be removed (read-only card) are kept and reported (`kept`). Undo removes the photos from the library but leaves the moved files at the destination. No Copy as DNG while moving | | LR-IMP-DNG-CONVERT | Convert to DNG on import [Classic] | P2 | ✅ | `cmd:library.import` (`dng`), `crates/ui-egui/src/dialogs/import.rs` | copy imports: Raw files ▸ Copy as DNG (lossless; the card is untouched); the raw copy is removed only once its DNG is verified and on disk (issue #106) | @@ -125,7 +125,7 @@ differences and missing bindings are in [ui-parity.md](ui-parity.md#shortcuts). |---|---|---|---|---|---| | LR-VIEW-PHOTOGRID | Justified photo grid | P0 | ✅ | `cmd:view.photoGrid`, `cmd:library.groups`, `cmd:library.sort` (`group`), `crates/ui-egui/src/panels/grid.rs`, `crates/catalog/src/dates.rs` | grouped by capture / import / edit date (day headers “Wednesday, 30 September 2026 · 12 photos”, months or years when zoomed out, or a fixed day/month/year/none choice in the sort menu); sticky header; clicking a header selects its photos; stacks never split; scrolls to the active photo only when it changes (or the grid comes back on screen), never on its own; a mouse-wheel notch scrolls a third of the grid's height, in both grids, as in Lightroom Classic (`theme::WHEEL_STRIP_SHARE`; trackpads scroll the points they report); queued embedded thumbnail stand-ins follow viewport priority changes without another extraction (`crates/ui-egui/src/render.rs`, issue #348; full-app visual/timing verification pending); selected photos are ringed in the theme's colours (`Tokens::ring_active` / `ring_selected`, 3:1 or more against the grid in every theme): the active photo gets a 3 pt two-tone ring (a thin inner line against the photo, so it shows on a white photo too), the other selected photos a 1.5 pt ring; the empty view (an empty album or other source in a library with photos, or an empty Recently Deleted) names Import Photos' shortcut as the menu bar writes it and as bound (⌘⇧I on macOS, Ctrl+Shift+I elsewhere), and none when it is unbound | | LR-VIEW-SQUAREGRID | Square grid | P0 | ✅ | `cmd:view.squareGrid` | same date headers and selection rings as the photo grid | -| LR-VIEW-DETAIL | Single-photo view | P0 | ✅ | `cmd:view.detail`, `crates/ui-egui/src/panels/detail.rs` | a “Rendering…” pill at the bottom of the photo (widget `loupe:rendering`; above the Navigator when that is in the way) once a render has been on its way for 0.5 s with nothing new on screen: over a stand-in (the raw's embedded JPEG, a thumbnail), the previous render after an edit, or, zoomed in, the magnified frame waiting for its window; Before views watch their own renders. Always on (Lightroom Classic makes it a View Options choice). When the render fails behind a stand-in, a caution pill says “This photo can't be opened” (“Original offline · no smart preview” when that is why) with the reason on hover and for screen readers (widget `loupe:renderFailed`; `ui.inspect` → `loupe.failed`) | +| LR-VIEW-DETAIL | Single-photo view | P0 | ✅ | `cmd:view.detail`, `crates/ui-egui/src/panels/detail.rs` | a “Rendering…” pill at the bottom of the photo (widget `loupe:rendering`; above the Navigator when that is in the way) once a render has been on its way for 0.5 s with nothing new on screen: over a stand-in (the raw's embedded JPEG, a thumbnail), the previous render after an edit, or, zoomed in, the magnified frame waiting for its window; Before views watch their own renders. Always on (Lightroom Classic makes it a View Options choice). When the render fails behind a stand-in, a caution pill says “This photo can't be opened” (“Original offline · no smart preview” when that is why) with the reason on hover and for screen readers (widget `loupe:renderFailed`; `ui.inspect` → `loupe.failed`). While the camera's embedded JPEG is on screen a “Camera preview” label at the top right says so, with what that means on hover (widget `loupe:cameraPreview`), so colour isn't judged on it. Culling: the stand-ins of the next five photos in the direction of travel and the one behind are made ahead and taken over at the step (`render::Slot::QuickAhead`; EOS R5 C wedding: step to the camera JPEG p50 187 → 0 ms), decoded camera JPEGs stay in memory for a step back, zoomed in on an unedited raw the camera JPEG is shown at full size under the window render until that arrives (100 % in ~0.9 s instead of ~5.6 s, `render::Slot::QuickFull`), and while an arrow key is held the raw render waits until a photo stays 0.12 s (`crates/ui-egui/src/loupe_timing.rs`); `ui.inspect` → `perf.loupeSteps` times each step on the frames drawn | | LR-VIEW-EDIT | Edit view | P0 | ✅ | `cmd:panel.edit` | | | LR-VIEW-FULLSCREEN | Full-screen preview | P1 | ✅ | `cmd:view.fullScreenPreview`, `cmd:view.enterFullScreen` | photo on black, arrows step, Esc exits; ⇧⌘F window full screen | | LR-VIEW-FILMSTRIP | Filmstrip | P0 | ✅ | `cmd:view.filmstrip`, `crates/ui-egui/src/panels/detail.rs` | the photo context menu on right-click; the mouse wheel scrolls it sideways, a third of its width per notch as in Lightroom Classic (trackpads scroll the points they report); it follows the active photo only when that changes (centred if off screen), so a scrolled strip stays put; every selected photo has the selected background; selected photos with a loaded thumbnail also show a ring, the active one in the heavier two-tone ring and the others in the lighter one, in the theme's colours, as in the grid (a selected cell with no thumbnail, still loading or failed, has the background and no ring); a drag from a cell carries the selection (a photo outside it drags alone and replaces it; in Compare it drags alone and leaves the selection, the active photo and the pair as they are) to the same sidebar rows as a drag from the grid, and a click without movement works as before (it selects the photo; in Compare it goes to the active side, see LR-VIEW-COMPARE) | @@ -430,7 +430,7 @@ differences and missing bindings are in [ui-parity.md](ui-parity.md#shortcuts). | Id | Feature | Tier | Status | Evidence | Missing / notes | |---|---|---|---|---|---| | LR-PREF-GENERAL | General settings | P0 | ✅ | `cmd:app.settings`, `cmd:app.openLibrary`, `crates/ui-egui/src/dialogs/settings.rs`, `crates/ui-egui/src/panels/notices.rs` | General / Import / Performance / Display / Interface tabs; app settings in ui.json (written atomically, saved when the library changes and every few seconds), library settings in prefs.json; At launch: open the last library or ask (see LRC-LIB-CATALOG-SWITCH); a damaged settings file is kept as `.corrupt-