mirror of
https://github.com/storytold/lightcraft.git
synced 2026-11-21 01:43:51 +01:00
cb91d69eec7e4dcb1d63c42657bf93b0f2bb7413
6
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
|
|
1a301fc2d8 |
LR-EDIT-LENSBLUR: camera-like Lens Blur with AI subject focus and per-mask bokeh
Part of the Lens Blur row (`LR-EDIT-LENSBLUR`, ⬜ → 🟡: no depth map yet). Related: #377 (SAM 3 subject selection, used here for the Lens Blur focus).
## Problem
Lens Blur settings existed but nothing rendered them: there was no way to add a shallow depth of field, keep the subject sharp while the rest falls out of focus, or blur only part of a photo through a mask.
## Change
**Edit → Lens Blur** (`section.lensBlur`, ⌘6)
- **Apply**, **Amount**, **Boost** (brighter bokeh highlights) and **Transition** (how gradually the blur grows away from what is in focus, so the subject doesn't look cut out).
- **In Focus**: *Subject* (a mask) or a fixed shape: **Auto / Round / Horizontal / Vertical**, buttons drawn in code that work on the light themes.
- **Move & Resize** (tool `lensBlurShape`) drags the ellipse on the photo and stretches it by its handles. Each drag is one undo step; a shape button puts the ellipse back in the middle.
- **Find Subject** (`lensBlur.findSubject`) runs only when clicked, never automatically on Apply.
- It starts from the focus point the camera recorded, when the file has one (Sony ILCE-1 and ILCE-7SM3, the models `lightcraft_meta` reads today). Otherwise it starts from the subject estimate.
- SAM 3 keeps the prominent people and animals: those at the focus point and those of a similar size. When the focus point falls on neither, it falls back to an object click at that point.
- The result is a normal Object mask ("Lens Blur Subject") that can be refined in Masking.
- It runs on the background segment worker, and on remote SAM 3 too (`Operation::Subject`). While the mask is pending, the UI shows a spinner and the render keeps the subject estimate in focus, so the photo is never blurred whole.
- Without the model, it returns the model-missing error, changes nothing, and the UI offers the SAM 3 download.
- **Brush Refinement**: Focus and Blur brushes painted on the photo (`lensBlur.brushStroke`, `lensBlur.brushReset`), with Size, Feather, Flow, Amount and Auto Mask.
- While Apply is off, In Focus, the shapes, Move & Resize and Find Subject are disabled.
**Masking: a Lens Blur block in every mask** (`section:maskLensBlur`, under Refine Edges)
- **Blur: Selection / Outside**. Outside blurs what the mask leaves out, without inverting the mask's other adjustments.
- **Amount**, **Boost** and **Transition**, per mask. The header's eye switches the block off and keeps its values; the header folds the block.
- Stored as `lens_blur`, `lens_blur_outside`, `lens_blur_boost`, `lens_blur_transition` and `lens_blur_off` on `LocalAdjustments`, set through `mask.adjust`.
**Rendering** (`crates/pipeline/src/lens_blur.rs`)
- Runs in the scene-linear stage after denoise; its settings are part of `pre_nr_key`.
- Disc (bokeh) kernel at four levels on coarse grids, with a normalized convolution weighted by the circle of confusion, so sharp areas don't bleed halos into the blur. Transition uses a chamfer distance to the in-focus area.
- The radius is relative to the long edge, so the blur doesn't depend on resolution. The work stays bounded on large exports: the disc kernel has a fixed size, and each level's coarse grid stays within a cell budget.
- Zoomed windows, both 8-bit and deep (16-bit/float), read around them and are cut back to the request. The grids follow the whole output's origin, and the automatic subject estimate is made once from the whole photo. A window therefore matches the whole render.
- GPU path: read back, apply on the CPU, upload (`crates/gpu/src/render.rs`).
- Hostile values (NaN or infinite sizes, out-of-range areas and focus points, oversized stroke lists) are sanitized or rejected, never a panic.
All new strings are in the 17 locales.
**Not in this PR:**
- Real depth. The falloff is the image distance from the focus, not scene depth; a depth model (e.g. Depth Anything V2 Small on candle) would be a separate PR.
- Rotating the ellipse.
- Aperture shapes.
- Focus points for more camera makes and models.
- Find Subject doesn't resume by itself after the SAM 3 download (the Masking tools do). Like the existing Object / Describe jobs, it shows no row in the activity stack.
## How it was checked
- [x] Full `cargo xtask ci` on Windows 11 x86_64 after the latest merge of `main` (
|
||
|
|
f3bb7a9f30 |
CI: one test binary per crate, line tables only, one WASM check (#2710)
* CI: dev profile keeps line tables for workspace crates and no debug info for dependencies * Control server tests: persistent_requests_and_slow_replies_outlive_the_idle_read_timeout no longer races the idle timeout Each request (and the closing half-shutdown) reaches the socket before the worker reads, so on a slow machine only the reply waits outlast the 100 ms idle timeout, as the test intends. The idle test sends its input the same way, and the no-frames test allows 5 s instead of 2 s to give up. * CI: one integration-test binary per crate Each crate's tests/*.rs become modules of tests/it/main.rs (51 test files into 13 binaries, plus 6 that stay their own). Inner #![cfg] attributes move to the mod lines. Files that change state for the whole process keep a binary of their own: codecs jxl_tail_heap and raw sr2_heap (dhat global allocator), gpu shutdown, shutdown_first and memory, ui-egui shutdown. Docs, comments and the GPU probe name the new paths and targets (--test it <module>::); the CI report names an it binary's skips after the crate. * CI: tests under nextest, doctests apart, one wasm check xtask ci runs the tests with cargo nextest run --lib --bins --tests when cargo-nextest is installed, else cargo test with the same targets and a one-line notice; doctests run after them with cargo test --workspace --doc. The heif and hardware GPU steps use the same runner. Examples stay out of the test build (clippy --all-targets still checks them). Concurrent tests under nextest follow the RUST_TEST_THREADS cap (NEXTEST_TEST_THREADS). The wasm check is one cargo check --keep-going over the same crates, and clippy runs with --keep-going. * CI: tests run under cargo test by default, nextest with LIGHTCRAFT_NEXTEST=1 (per-process scene cache) |
||
|
|
90a248edad |
M12: Describe selections from SAM 3's semantic mask: sure inside, no lines between instances
Describe merged the detector's per-instance masks. Those stay near 0 even well inside what they found: a sky came out about 87 % selected, a volleyball net about 70 %, a person about 94 %. Two instances that touch both fade out where they meet, so a line showed between people standing side by side. - The detector's semantic head is now ported: semantic_projection over the pixel decoder, the transformers implementation's semantic_seg. It matches the reference to about 1e-3, like the instance masks. tools/sam3_reference.py writes it and the reference test checks it; older reference files still pass. - Each instance found (score above the threshold, as before) takes the semantic logits where it is the strongest instance and near (above -4). The semantic mask is sure inside and runs on across the boundary between two instances. Far from every instance nothing changes, so Describe selects nothing it didn't find before. On 16 prompts over three photos (a mountain landscape, two gym photos), the median coverage inside each selection rises, e.g.: - sky 0.87 -> 1.00 - mountains 0.81 -> 0.99 - net 0.71 -> 0.99 - floor 0.92 -> 1.00 - person 0.94 -> 0.99 Each selection gains 0 to 1.9 % of the photo at its soft rim and loses none. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
|
3e42f2e05c |
feat(testkit): #1037 tests record why they skip
libtest has no skipped status and hides the output of passing tests, so a test that returns early without a GPU, a corpus file, a craft-fonts face or a model passed silently. `lightcraft-testkit` (the testkit slot the layering table already reserves, dev-dependency only, std only) adds `skip!(Need, "reason")`: it prints the note as before and, when LIGHTCRAFT_SKIP_LOG names a file, appends one JSON line (need, reason, test, binary, file:line). Plain `cargo test` is unchanged. The 61 existing skip sites use it. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
|
0613f925de | Support remote SAM3 inference through a private Metal worker | ||
|
|
6468bfc06b |
M12: SAM 3 Object and Describe masks (pure Rust, Metal)
AI masks from Meta's SAM 3, run in LightCraft itself: a new
`lightcraft-segment` crate ports the Hugging Face transformers
implementation (Apache-2.0) to candle, on the GPU through Metal on macOS
(CPU elsewhere), with no C/C++ and no `unsafe`.
- Object: click to include, ⌥-click to leave a part out (SAM 3's
interactive prompt encoder + two-way mask decoder).
- Describe: a text prompt selects every instance ("sky", "the red car";
"car, road" merges several) via the CLIP text encoder and DETR detector.
- Both as new masks and as Add / Subtract / Intersect components; + / −
buttons under the selected mask combine selections.
- The segmentation is stored with the mask (288² quantized logits, ~10-30
KB), so renders, exports and the web build never need the model.
- A zoomed-in detail pass (background, ~5 s on an M4 Pro) re-segments the
area around a selection from a sharper source: 5-10x finer edges on
small objects.
- Per-selection Edge slider (hard ↔ soft); hovering a mask in the list
shows it in red (the always-on overlay is now off by default, O toggles).
- The engine feature `sam` (on in the desktop app) gates the model; the
image is encoded once per photo in the background, clicks take ~35 ms.
- Weights are not bundled (SAM License): tools/install-sam3.sh / .ps1
download and verify facebook/sam3; docs/ai-masks.md.
- Clearer errors (missing file, missing model) and a stderr logger in the
app (LIGHTCRAFT_LOG=info).
candle is pinned at 0.9.2: later releases make candle-core depend on
tokenizers with the Oniguruma C library. Every stage is checked against
the Python reference (tests/reference.rs, tools/sam3_reference.py): on
Metal the backbone matches to 7e-5 relative error, masks to ~5e-5.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01C8BkS7JDqa1EMcjW6spDGA
|