mirror of
https://github.com/storytold/lightcraft.git
synced 2026-11-21 01:43:51 +01:00
cb91d69eec7e4dcb1d63c42657bf93b0f2bb7413
29
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
|
|
25ffe3e5d4 |
codecs,raw,preview,meta,engine: cut over TIFF/JPEG/focus to craft-libs and fix CI (#4244)
* codecs,raw,preview,meta,engine: cut over TIFF/JPEG/focus to craft-libs and fix CI * docs: update parity-checklist paths after craft-raw/craft-metadata cutover * Remove unused codec workspace deps and cut examples over to craft-image-codecs * chore: update Cargo.lock after removing unused codec deps |
||
|
|
0d2dd9a974 |
A library's files were written by path, so a path that came to lead
somewhere else (a share mounted over the folder, the folder moved away and another put in its place) made the session write into that other folder: its catalog.snap and catalog.log were replaced, and dropping the lock removed the owner note of the program that had the library open. OpenedFolder records the folder's device and inode number (and when it was made, where the file system keeps that) when a library opens, and check() asks, with one stat, whether the path still leads to it. A drive or share mounted again gets another device number, and some file systems number their folders anew at every mount, so a folder also counts when it holds exactly the owner note this session's lock wrote (the lock vouches for it, and the note holds a value no other lock's note has), or when it has the same inode number and both creation times are known and equal; on the same device two different creation times are two folders, since an inode number is given again to a folder made later. FsStore (both stores of a library, the snapshot worker), the catalog log's thread and the lock's owner note ask before they write; a refusal is an ordinary write error that names the path, so the changes stay queued, the app says why they are unsaved, and saving goes on once the folder is back. The thumbnail cache gets a gate that the library closes the same way, and its takeover, prune and clear passes look at it every 256 files. The catalog log is opened by its path for each batch and appended to only while it is as long as what the thread wrote. A copy of the folder put at the path holds the note and counts as the library, but its log lacks what was written after the copy was made: it is refused like a failed write (reported, with later changes taken meanwhile), and the snapshot the app takes for held records replaces it, instead of going on writing into the folder that was moved away. A longer log is cut back only after a write of the thread's own failed. The lock removes the owner note only when it is the one it wrote, in the folder it locked. Reopening a path whose folder was replaced takes that folder's own lock instead of handing the old one over, so does the check_library preflight, and a lock taken in another folder than the stores recorded is refused. A Move import removes its originals only after the catalog log is on disk, not when it was handed to the log's thread. Windows reports no folder identity through std (nightly only): there the check only asks that a folder is still at the path. |
||
|
|
61005f3dc1 |
Culling: the loupe uses built previews; photos prepared ahead and in the background (#3077)
* PERF-CULL-0: ui.inspect reports how the loupe showed each photo it was stepped to
perf.loupeSteps records, per step, the ms from the step to the first frame
with anything of the photo, with a full-size picture (camera JPEG, cached
or rendered) and with LightCraft's own render, measured on the frames the
app draws, so culling benchmarks exclude the control channel's round trip.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
(cherry picked from commit 95f0343848fc0a238f80f07ceb292eaf6779aeaa)
* PERF-CULL-1: browsing a folder shows its first photos within a second
The Local view read and hashed every file of the folder before showing any
of them (2,933 EOS R5 C CR3s: 30.6 s to the first photo). It now hands the
files the library lacks straight to the import, ordered the way the view
sorts them (file time for the date sorts), which adds them batch by batch:
first photo 0.9 s, twelve thumbnails 1.4 s, the whole folder in the same
~34 s as before, in the background.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
(cherry picked from commit b766558e7ef4f802919a144da53e444f70e0186b)
* PERF-CULL-2: the loupe makes the next photos' camera previews ahead of the step
Culling steps through photos, and each step waited for the next raw's
embedded JPEG to be read and decoded (p50 187 ms, p95 332 ms on EOS R5 C
CR3s) and then for a full raw develop. Now:
- the loupe's stand-ins for the next five photos in the direction of
travel and the one behind are made ahead (Slot::QuickAhead, priority
just below the loupe's own render) and taken over at the step, so a
step shows the camera JPEG at once (p50 0 ms);
- a slot keeps its photo while the photo stays wanted, so photos that
shift places as the user steps are not made twice;
- decoded camera JPEGs stay in the memory LRU (never on disk), so going
back shows them again without a decode;
- while an arrow key is held (a photo left within 0.25 s), the raw
develop waits until the photo has stayed 0.12 s, and a render queued
for a photo already left is dropped, so the workers make the previews
instead of renders nobody sees.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
(cherry picked from commit 0430b95636d2b6adce8887af7b9464687c95549e)
* PERF-CULL-3: CR3 previews take the smallest embedded JPEG that covers the view; plain sRGB JPEGs decode straight to 8-bit
Upstream's embedded_preview_covering (
|
||
|
|
f618fc06a7 | security: staging and probe files are created new, never written through an existing entry (#3673) | ||
|
|
dd119ca007 |
Reject signed strings in Hash128::parse
Found while testing crates/preview/src/hash.rs:21 Co-Authored-By: Claude Haiku 5.5 <noreply@anthropic.com> Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> |
||
|
|
260d03ced6 |
Merge PR #1777
# Conflicts: # docs/whats-new.md |
||
|
|
1f87328c6d |
Never crash: Write preview cache files from one bounded writer (#1522)
Each deferred cache write started its own thread holding its image, with no limit. One writer thread per cache now takes them in turn, and a write is skipped once 128 MB is waiting. |
||
|
|
d2523e8544 |
Preview cache: cached thumbnails are no longer taken for the person's photos
The thumbnail cache wrote .jpg files into the library's thumbs folder, which is usually inside Pictures: photo galleries listed every thumbnail as a photo (issues #610, #1066), and every read brought the file's date forward, so each launch made them all look new. - Cache files are named <32 hex>.lcthumb (JPEG data as before). A .jpg of before is taken over under the new name when it is asked for; the rest when the cache is tidied, which the engine does once per cache folder, on a thread of its own with a row in the activity stack that shows its progress and can stop it. Nothing is rendered again. - Reading writes nothing: what was read is remembered for the session (a full cache lets it go last), and only a file that went unused for a week has its date brought forward. - A folder that holds nothing but the cache gets a CACHEDIR.TAG, when the cache makes it and when it is tidied; one that also holds somebody's files is not tagged (issue #98). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
|
935fabeb18 | Preview disk cache: ignore symlinked shard directories (#1504) | ||
|
|
702f90bfc9 | Batch 32: close the tests two merges added at one place; cargo fmt | ||
|
|
251397694e |
Merge PR #1302
# Conflicts: # crates/preview/src/tests.rs |
||
|
|
a681a3c667 | Merge PR #1076 | ||
|
|
c8f676252f |
M5.6: with a GPU, a slider drag's preview is drawn in the frame of the drag tick
A drag tick requested the loupe's draft and the frame ended; the draft, done in 1.5-3 ms on the GPU, was picked up at the start of the next frame, a frame after the slider moved. Now, in a drag, the loupe waits up to 40 % of a frame for its draft when the last loupe render took well under that (not with a render offload, nor zoomed in, where what changes on screen is the window's draft). A CPU render, which would only make the frame late, is not waited for. JobPool::wait waits for the next finished job, which try_recv then hands out; Renderer::poll's collection of finished jobs is split out so the wait can turn the draft into the texture. Co-Authored-By: Claude <noreply@anthropic.com> |
||
|
|
d7037d117f |
M5.6: a view slot renders one job at a time, so a drag's renders don't pile up
The job pool replaced a slot's queued job, but a newer job for a slot whose job was already running started beside it on another worker. A slider drag submits faster than renders finish, so one view had several renders on the GPU at once and about 30 % of finished renders were thrown away (61 MP raw, 1:1 Exposure drag). Now a slot's next job waits until its running one ends (a newer submission still replaces it in the queue); other slots are not held up, and a job that panics frees its slot. Measured on a 61 MP raw at 1:1, 8 s Exposure drags (M4 Max): thrown-away renders roughly halved, and frames over 20 ms 0 % in every one of 4 runs (one run of the overlapping pool dropped 10 %). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
|
87be53279b | fix(preview): account for replaced thumbnail cache entries | ||
|
|
5a537d5d39 |
LR-BEHAV-GPU: no render is inside the GPU driver when the app quits (issue #620)
A render still running on a worker thread while the process exits crashes in the driver (SIGSEGV in the NVIDIA Vulkan driver on Linux), where neither the panic hook nor catch_unwind helps: pool workers were detached, the devices are statics that are never dropped, and nothing on the way out waited for GPU work. - gpu: every entry into a device (render, device creation, pool trim, denoise runner and tiles) is counted as work in flight (exit.rs). begin_shutdown() lets nothing new in; wait_idle(timeout) waits for what is inside. - preview: JobPool keeps its workers. shutdown(timeout) and Drop discard the queued jobs and wait, bounded, for the running ones (2 s on drop); never for the calling thread, and not twice. - engine: a render job caught by the shutdown gives up instead of rendering on the CPU, so quitting doesn't wait for it. - app: on_exit ends with LightcraftApp::shutdown (2 s for everything), after settings and library are saved; the macOS menu-bar Quit goes through on_exit. Not changed: the export, merge, denoise and import threads are still detached; their GPU work goes through the gate, their CPU work is cut off at exit as before. Tests: pool drop waits / is bounded / survives a panicked job and its own job dropping it; the gate; GPU renders refused and the device not created after begin_shutdown; the app's shutdown leaves no render worker. The crash itself needs Linux + NVIDIA and was not reproduced here (Windows, DX12): 9 of 9 quits of the app mid-render exit with code 0 in under 0.8 s. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
|
62325bd45e |
Thumbnails: retire the replaced preview cache so its old jobs can't write
Reopening a library (or opening another one, or set_disk_cache_bytes falling back to attaching) replaced MediaCache::rendered with a new PreviewCache, but render jobs still in flight kept the old object. Each PreviewCache checked only its own generation, so after a clear on the new cache the old one's generation was still current: a finishing thumbnail job wrote its pixels through it into the directory the clear had just emptied. The UI rejected the result, but the JPEG stayed. attach_disk_cache now retires the cache it replaces before swapping it: PreviewCache::retire takes the generation write lock (so it waits for a write in progress, like clear) and marks the object retired; every get/put/put_deferred through it is then refused and its clear() is a no-op, as its directory may now belong to the replacement. Files on disk are left alone, so a plain reopen still finds its thumbnails, and resizing the budget in place keeps the same cache object as before. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
|
73cf0d50bb |
Thumbnails: keep textures when the disk cache is resized
Changing the disk cache size (library.preferences cacheMb) re-attached a new PreviewCache object, and the renderer treats a new cache object as a library reset: every texture, the loupe included, was dropped and re-rendered. Thumbnails are keyed by content, so a new budget leaves them valid. The budget now changes in place on the attached cache (same object, same generation); the next write trims to it, as before. A preview clear, a generation bump and a new library still invalidate. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
|
f2f7a1cb26 |
T0: optimize thumbnail requests and reject obsolete preview work
Skip unchanged thumbnail jobs and reuse bounded settings hashes (both maps capped at 1024 entries). Reject stale render results and cache writes after edits, library changes and preview clears: render requests carry a request id and cache generation, PreviewCache writes and reads hold the generation lock that clear() takes, and library.clearPreviews also drops cached sources. The web worker index follows the same generation rule. The decoder guards, Windows path fixes and test fixture changes that were bundled in a327b98 are separate commits; the benchmark follows this one. Co-authored-by: Codex <noreply@openai.com> Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
|
0fd0ffb783 |
Issue #98: thumbnail cache only lists, counts and deletes its own files
scan() now accepts only <2 hex>/<32 hex>.jpg (matching shard) and the cache's own <32 hex>.tmp<id> temp files, so prune/clear never delete and size() never counts foreign files in a shared thumbs/ folder. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
|
3f20f98931 |
Merge pull request #56 from storytold/pr/never-crash-core
Never crash: color, geom, raster, develop, scenes, preview, pipeline |
||
|
|
fc13807f91 |
Never crash: color, geom, raster, develop, scenes, preview, pipeline
- color: RgbSpace::to_xyz/from_xyz and bradford() no longer expect a
matrix inverse. Degenerate (collinear) primaries, which readers reject
before building a space from a file, leave the primaries unscaled /
give the identity; the constant Bradford and OKLab matrices fall back
to the identity, which can't happen (round trips are tested).
- pipeline geometry: sample() matches (mode, warp) instead of
expect("warp"); a warp plan without a warp (sample_plan never makes
one) samples through the equivalent affine.
- color, geom, raster, develop, scenes, preview and pipeline now #![deny]
unwrap/expect/panic/unimplemented/todo/unreachable; clippy.toml as in
the steering PR.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
||
|
|
a8c3f01e1e |
CI: clean under clippy 1.99 (as_chunks for constant chunk sizes, ? in export resize parsing)
Rust 1.99 (stable since 2026-09-28) adds clippy::chunks_exact_to_as_chunks and widens clippy::question_mark, which failed `cargo xtask ci` on main. Same behaviour: the remainder that chunks_exact skipped is the remainder as_chunks returns separately. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
|
6a0d5346e8 |
LRC-LIB-PREVIEWS: build standard-sized / 1:1 previews in the background
library.buildPreviews {size: standard (Settings → Performance preview size) | full, ids?,
wait?} renders each photo's grid thumbnail and loupe view into the memory + disk preview cache
(selected photos, else all in view) on a background thread; library.previewProgress /
cancelPreviews. File ▸ Previews: Build Standard-Sized / 1:1, Stop, Discard Preview Cache;
toasts when a build starts and finishes. Also silences the wasm-only unused `threads` warning.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
||
|
|
7266f4153f |
M5.6c: one memory budget for caches, decode gate and GPU pool; peak RSS 1.9-2.3 GB -> 0.83-0.99 GB
- memory::budget (default min(25% RAM, 1.5 GiB); LIGHTCRAFT_MEMORY_MB, app.memoryBudget): half for engine caches (sources + rendered previews, LRU across them via a global use tick; the photo on screen stays), a quarter for decodes in flight (WorkGate: thumbnails, prefetch and import probes wait; loupe/export loads never wait), an eighth for the GPU free pool. - GPU: best-fit buffer reuse (<= 25% slack), buffers released outside a render are pooled at once (no stranded retired lists), pool trimmed after 3 s idle. - Decodes drop file bytes and raw samples early, colour/crop/orient in place. - macOS: apps install malloc_zone_pressure_relief as the release hook (after each decode, import and idle trim): freed large blocks no longer stay resident. 14 raws + loupe + 12 steps (M4 Pro): peak RSS 1.9-2.3 GB -> 0.83-0.99 GB, footprint 2.5-2.6 GB -> 0.91-0.96 GB, heap peak 930 -> 656 MB; stepping median ~13 ms. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
|
9886d49823 |
Merge branch 'worktree-agent-a69dcec0f0c61bc39'
# Conflicts: # crates/engine/src/media.rs # crates/ui-egui/src/render.rs |
||
|
|
5fd5384bfb |
M5.4a: instant first view — cached view render / embedded raw preview, then the full render
- engine: QuickJob — what to show the moment a photo opens: the loupe's last render for the
current settings (kept by Session::loupe_job under a size-independent "view" key in the
preview cache, memory + disk with a library; the disk JPEG is written off the render
thread via PreviewCache::put_deferred), else the embedded camera JPEG of a raw that still
has its import look (import::has_import_look), else a thumbnail-level render.
- Grid / filmstrip: an unedited raw without a thumbnail shows its cached thumbnail, else its
embedded preview (Slot::ThumbQuick) at once; the real rendered thumbnail follows at
background priority (same cache keys, so it still ends up matching the develop render;
edited photos render directly).
- Loupe (detail, compare, filmstrip selection): Slot::Preview holds the stand-in until
Slot::Main has the photo; the full render then replaces it in place (no blank frame).
- files::embedded_preview_srgb + PreviewLoader hook (desktop/CLI/MCP and web).
- ui.inspect → loupe {photo, source: render|cached|embedded|small|thumb|none, pending}.
Real app (24 MP raws, loaded machine): first image 15–90 ms (thumbnail, then embedded
preview), full render 0.2–0.5 s (was 1.0–2.4 s with nothing shown before it).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
||
|
|
8f2058a124 |
M0.13: web render workers — pipeline renders off the main thread
Renders run in dedicated module Web Workers, each a second instance of the app's wasm module (index.html compiles it once and posts the WebAssembly.Module to worker.js). A job crosses as JSON (WireJob: source, settings, request); the worker reads originals and cached thumbnails from browser storage itself, keeps decoded sources and per-view stage caches, and transfers RGBA back. Jobs follow the worker that decoded the photo; the loupe waits for it. No wasm threads or SharedArrayBuffer needed; falls back to inline rendering if no worker starts (?workers=0). Thumbnail disk cache on the web: thumbs/<key>.jpg in OPFS/IndexedDB with an index on the main thread, pruned to the desktop budget (2 GB → 80 %, LRU). ui-egui: RenderOffload hook in the Renderer (queue/priorities/dedup stay in the UI). engine: RenderJob.origin + MediaCache::origin_ref; preview: encode_jpeg/decode_jpeg public. xtask web: copies worker.js; --serve sends COOP/COEP/CORP (documented in docs/web.md). Headless Chrome, cold load + grid scroll, 24 demo photos: main-thread busy 12.4 s → 0.7–1.1 s, long tasks 23 (max 1.1 s) → 0; grid filled in 6.3–8.4 s vs 13–16 s; slider job 4 ms. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
|
bcecab7d84 |
M1.5: thumbnail/preview cache (disk + memory LRU, prioritized job pool)
- new crate lightcraft-preview (L3): 128-bit content hashing (SipHash-1-3), cost-bounded Lru, DiskCache (JPEG q90 files keyed by content hash + settings hash + size + RENDER_CACHE_VERSION; temp+rename writes; LRU prune by mtime to 80% of budget; corrupt files dropped), PreviewCache (memory LRU over disk), JobPool (per-slot dedupe, priorities, reprioritize/drop, inline mode for wasm). - engine: Session::thumb_job buckets thumbnails to 128/256/384/512 and serves them from the cache without decoding the source; decoded thumbnail sources now in a 384 MB LRU (was unbounded); library attaches <library>/thumbs; library.info cache stats; library.clearPreviews. - ui-egui: Renderer runs on JobPool; thumbnails scrolled out of the prefetch area are dropped from the queue; on-screen thumbnails get re-prioritized; ui.inspect reports in-flight renders + thumb textures. - Demo library (24 photos): grid fully drawn 4.4 s cold -> 0.21 s warm (24/24 disk hits, no source decoded). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> |