Commit Graph
184 Commits
Author SHA1 Message Date
Dylan M. Taylor 5a24289898 raw: consume merged DJI DNG and Sony ZV-E10 fixes (#4532)
* raw: consume merged DJI DNG and Sony ZV-E10 fixes (#587)

Pin craft-raw to merged f589ff48, carrying craft-libs #2042 and #2039. Preserve the Fuji Kelvin adapter and other application codec pins. Bridge the RAW writer TIFF identity explicitly and regenerate Flatpak sources.

* raw: align engine preview fallback test TIFF error identity
2026-11-18 23:27:04 -05:00
Dylan M. Taylor 27b755a93c raw: cut DCP camera-profile files over to craft-raw (#4332) 2026-11-17 08:14:04 -05:00
FredApps 207c20f558 Merge remote-tracking branch 'origin/main' into t4298
# Conflicts:
#	Cargo.lock
#	Cargo.toml
#	packaging/linux/flatpak/cargo-sources.json
2026-11-16 10:33:04 +01:00
Dylan M. Taylor 0466978870 test(codecs): preserve two-plane JXL fixture and large AV1 coverage 2026-11-16 01:34:37 -05:00
Dylan M. Taylor 2ed4595663 Include sample-to-chunk mappings in CR3 fixtures 2026-11-15 17:37:16 -05:00
Dylan M. Taylor 8c013aec83 Keep RAW fixtures valid under shared container and strip bounds 2026-11-15 18:54:10 +00:00
Dylan M. Taylor b757d74a31 test(raw): keep DNG JXL tile fixtures single-layer 2026-11-15 01:36:18 +00:00
Dylan M. Taylor 54a6ddd856 test(raw): encode single-plane JPEG XL fixtures as gray 2026-11-14 23:41:03 +00:00
Dylan M. Taylor a4d64049b5 fix(codecs): enable av1-encode feature and pass EncodeOptions by value 2026-11-14 21:29:53 +00:00
Dylan M. Taylor 5d22eff36c chore(codecs): cut over AVIF, AV1, and JXL tests to craft-libs 2026-11-14 21:19:39 +00:00
Dylan M. Taylor 25ffe3e5d4 codecs,raw,preview,meta,engine: cut over TIFF/JPEG/focus to craft-libs and fix CI (#4244)
* codecs,raw,preview,meta,engine: cut over TIFF/JPEG/focus to craft-libs and fix CI

* docs: update parity-checklist paths after craft-raw/craft-metadata cutover

* Remove unused codec workspace deps and cut examples over to craft-image-codecs

* chore: update Cargo.lock after removing unused codec deps
2026-11-14 13:49:38 -05:00
Dylan M. Taylor 6f012c10b7 chore: finish craft-raw/craft-libs cutover after merge with main 2026-11-14 05:38:31 +00:00
Dylan M. Taylor 34161c9a8c Use craft-tiff from craft-libs; remove crates/tiff
lightcraft-tiff moved to craft-libs unchanged as craft-tiff
(storytold/craft-libs#783), the first step of sharing RAW support with
PhotoCraft (storytold/craft-libs#587). The layer table drops tiff and its
intra-layer ordering before raster.
2026-11-11 01:25:32 +00:00
FredApps 10db03b091 LR-IMP-FORMATS: withdraw compressed ORF decoding pending provenance review (#3916)
* LR-IMP-FORMATS: withdraw compressed ORF decoding

Remove the decoder introduced in #3775 pending provenance review while preserving uncompressed ORF layouts and embedded previews. Add unsupported probe/decode regressions, remove the prior provenance narrative, and restore camera coverage records to preview-only.

* LR-IMP-FORMATS: refresh ORF support status and render caches

* LR-IMP-FORMATS: document ORF withdrawal regressions
2026-11-07 15:56:03 +01:00
547fd29f3c Extend CR3 white level to actual sensor saturation (#3805)
* Extend CR3 white level to actual sensor saturation

The white level comes from Canon's stated value, not from the sensor data, so highlights between the stated level and true saturation are misclassified as clipped. Extending the white level to the real pile-up and folding the delta into baseline_exposure leaves everything below the old white unchanged while recovering the lost highlight range.

Add RawImage::extend_white_to_saturation() to saturation.rs that scans for a saturation pile-up above the stated white (coarse 1/128 bins, requires >= 1 sample in 100,000 and >= 64 samples above), sets white to the lowest colour's pile-up edge or the bit-depth top, and returns the EV delta. Call it from cr3.rs decode() on Mode::Full and add the delta to ColorData::baseline_exposure. Bump RENDER_CACHE_VERSION in media.rs.

Test: extend_white_to_saturation_above_stated_white failed before the change and passes after.

* Preserve CR3 highlight headroom and exposure

Assisted-by: OpenAI Codex

* Tests: a probe does not know the white headroom a CR3 decode finds

The header probe stays a header probe, so the comparison leaves out
baseline_exposure, which a full decode adjusts from the samples.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: eliets2 <287273517+eliets2@users.noreply.github.com>
Co-authored-by: FredApps <FredApps@users.noreply.github.com>
Co-authored-by: FredApps <153173854+FredApps@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-05 23:01:16 +01:00
ba282d3637 LR-IMP-FORMATS: pure-Rust clean-room compressed Olympus ORF bitstream decoder (#3775)
* LR-IMP-FORMATS: pure-Rust compressed Olympus ORF bitstream decoder

* LR-IMP-FORMATS: address CodeRabbit review feedback on PR #3747

* orf: bound predictor arithmetic, bit widths, and stream consumption

* orf: use unsigned magnitude abs_diff for predictor differences

* docs: correct Olympus verified-body count in parity header

* orf: saturating wbits derivation, widen pixel shift to i64 before clamp

* docs: sync raw parity headline figures with coverage tables

* docs: record clean-room provenance and report 58 newly unlocked bodies for compressed ORF

* docs: note LibRaw derivation and pending clean-room reimplementation for compressed ORF

* LR-IMP-FORMATS: pure-Rust clean-room compressed Olympus ORF bitstream decoder

* LR-IMP-FORMATS: sync orfc path in parity docs and correct mainstream weighted depth sum

* LR-IMP-FORMATS: address CodeRabbit review feedback on ORF gap and evidence sync

* Convert Negative 1/4: lightcraft-film, the negative conversion maths from Film Lab (#2490)

The film model, levels in density space, film presets, colour and tone of
Film Lab's core (same author, MIT), without the PhotoCraft-only sharpening
and JSON scanning, plus a whole-scan measurement (base, black and white
points, neutral balance) a host stores with a photo's settings. A library
with no caller yet.

Co-authored-by: FredApps <153173854+FredApps@users.noreply.github.com>

* LR-IMP-FORMATS: address CodeRabbit PR #3775 review (round 2)

- orf.rs: scope sample-based bit-depth inference to Word16 branch only
  (from_word16 flag prevents heuristic from running on compressed ORF data)
- orfc.rs: reject truncated compressed streams before allocating the frame
  (minimum 1 bit per pixel check)
- orfc.rs: remove feedback_shift_644 == 15 special case (ordinary exponent)
- ROADMAP/README/target-app-parity: fix '78 bodies' to '58 more bodies
  (2 verified)' — 78 is total Olympus+OM count, 58 were newly unlocked
- target-app-parity: RAW readiness 80% -> 87%, recompute weighted sums
- parity-checklist: refresh summary via cargo xtask parity --write

Generated with Antigravity

* LR-IMP-FORMATS: address CodeRabbit PR #3775 review (round 3)

- target-app-parity: RAW camera decoding 85% -> 89% (559+721 of 1446)
- target-app-parity: mainstream practitioner row lists ORF as per-model
  verification, not as unfinished decoder alongside CR3/HE NEF
- log_writer: check_regular_directory_entry before truncate open
  (prevents symlink-following truncation of files outside the library)

Generated with Antigravity

* LR-IMP-FORMATS: regenerate parity summary after film crate merge

cargo xtask parity --write: 83.2% of 543 in-scope rows (was 83.5%/532);
sync target-app-parity.md checklist references to match.

Generated with Antigravity

---------

Co-authored-by: Yorgos S. <smyrgeorge@gmail.com>
Co-authored-by: FredApps <153173854+FredApps@users.noreply.github.com>
2026-11-05 22:23:31 +01:00
8972565249 LR-PROF-IMPORT: import the user's own DNG camera profiles (.dcp) (#1063)
* LR-PROF-IMPORT: import the user's own DNG camera profiles (.dcp)

profile.import (File > Import Profiles & Presets, drag & drop) now also
reads DNG Camera Profiles: the user's own .dcp files, e.g. made with a
colour-checker tool. They appear in the profile browser and menu under
Camera Profiles with dcp:<camera>-<file> ids; profiles.camera lists them
and profile.deleteCamera / profile.deleteImported remove them. A library
on disk keeps a copy in Profiles/. No profile ships with the app and the
repository holds none: tests build theirs in memory (write_dcp).

lightcraft_raw::dcp parses the file per DNG spec ch. 6 (magic RC in
place of 42; IFD 0 read through the existing DNG colour builders) and
DcpProfile::applied_to puts its matrices, illuminants and look in place
of a raw file's, keeping the shot's white balance, analog balance and
baseline exposure.

Choosing one (develop.profile) replaces the raw file's colour model when
it is decoded, before the starting colour and the white-balance scale
(WbScaleOf) are worked out, so the matrix path takes over from the fitted
JPEG look (CR3/CR2/ARW...) and white balance follows the profile. A
profile without a tone curve of its own (most chart-made ones) keeps the
photo's contrast: the DNG's tone curve, else the tone of the camera-JPEG
fit. The profile travels with SourceRef::File into FileLoader /
PairLoader (new Option<&DcpProfile> argument); a decoded source records
the profile it was made with and is not reused for another. Smart
previews and the web render worker keep the file's own colour; rendered
(non-raw) photos ignore it.

This reverses the documented "Adobe profile formats deliberately
unsupported" stance for user-supplied files only; docs updated.

Split out of #1012.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* LR-PROF-IMPORT: build against upstream main (kelvin_presets, 3-arg loaders)

DcpProfile::applied_to keeps the file's white-balance presets (new
ColorData::kelvin_presets upstream): they are camera facts like as-shot
neutral. Upstream's new FileLoader test closures take the profile argument.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* LR-PROF-IMPORT: hover previews and browser thumbnails develop the hovered camera profile

The loupe's hover preview (preview_job) and the profile browser's thumbnails (variant_job) took
the photo's own decoded source, so hovering an imported camera profile showed the built-in colour
(and hovering a built-in profile while one was applied kept the camera profile's). Both now decode
as the hovered settings develop: the source follows the settings' camera profile, as the main
render does, and a variant's cache key already follows its settings.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* LR-PROF-IMPORT: build against upstream main (face-tile test loader and render_bench take the camera profile)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* CI: the catalog's staging-link test module is Unix-only (unused import broke clippy on Windows)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* CI: same-folder export test expects the '/'-joined subfolder (failed on Windows)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* CI: folder-drop tests build widget ids with the Folders tree's '/' paths (failed on Windows)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* LR-PROF-IMPORT: rustfmt after merging main

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* LR-PROF-IMPORT: a real D750 camera profile as a pinned test corpus, checked against Lightroom

dieli's CC0 Nikon D750 ColorChecker set (Calibrite PROFILER dual-illuminant DCP,
matrix-only; the two chart NEFs; Lightroom Classic 15.6 renders with the profile)
is fetched by cargo xtask corpus --download into corpus/d750-colorchecker/, pinned
to commit f37f4054 and by sha256; nothing is committed.

Tests (skip without the corpus): the profile parses as matrix-only with
Cool White Fluorescent + D55; the matrices blend by inverse temperature; the
matrix-only path inverts the colour matrix and adapts to D50; it takes over the
chart NEFs' colour; imported and applied end to end, the charts develop within
mean dE2000 3.84 (flash) / 3.35 (indoor) of Lightroom's renders.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* LR-PROF-IMPORT: a single .dcp file is a camera profile only, not also an unreadable LUT

profile.import read every plain file that was neither a folder nor a .zip as a .cube LUT, so importing
one .dcp imported the camera profile and also reported it as failed ("stream did not contain valid
UTF-8"). Only .cube files are read as LUTs now.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* LR-PROF-IMPORT: review fixes for imported camera profiles

- A library moved or restored elsewhere finds its camera profiles in its own Profiles/ folder.
- A grid thumbnail is cut from the decoded preview only when that preview used the same camera profile.
- Render and source cache keys carry a fingerprint of the profile file, so a deleted or replaced profile never
  has its old renders stand in.
- Camera profiles can be favourites, and favourites/recent entries survive reopening the library.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* LR-PROF-IMPORT: smart previews without the camera profile, current-profile source facts, unsupported files reported

CodeRabbit review of 4ef025d9:
- Build Smart Previews develops the proxy without the photo's imported camera profile and never takes a cached
  source decoded with it (the denoised picture is kept); before, only the stale-proxy refresh left it out.
- Session::source_info reports decoder facts only from a cached source developed with the camera profile the
  photo uses now, else the catalog's.
- profile.import reports a file that is neither .cube, .dcp nor .zip as failed instead of passing over it.

Each with a regression test in tests_dcp.rs.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* LR-PROF-IMPORT: the Before side decodes with its own camera profile; a failed profile copy fails that file only

- build_job (and region_job_before through it) picks the Before render's source with the Before
  settings, so it never reuses or decodes a source developed with the edit's camera profile.
- profile.import: a .dcp whose copy into the library fails is reported in `failed`; the profiles
  imported with it stay imported and are saved, and nothing half-written is left.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* LR-PROF-IMPORT: a cached smart preview stands in while the original is offline

A smart preview carries no camera profile, so with an imported profile the profile check
rejected the cached proxy and every render decoded it again (it gives the same picture). It
now stands in, for its own level and for the thumbnail cut from it, while the original is
known to be offline; otherwise the profile check is as before, so once the original is back
the photo is decoded from it with its profile. Photos without an imported profile keep main's
behaviour. Test `tests_dcp::a_cached_smart_preview_stands_in_while_the_original_is_offline`
(fails without the fix).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* LR-PROF-IMPORT: deleting a camera profile removes the library's copy before forgetting it

profile.deleteCamera now removes the library's copy of the profile file
first. A copy that is already gone is fine; any other error (file in use,
no access) fails the command and keeps the profile listed, so the library
never holds a profile file it no longer lists. Test:
a_profile_whose_copy_cannot_be_deleted_stays.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* LR-PROF-IMPORT: deleting a camera profile never follows `..` out of the profiles folder

A listed path with a `..` component can start with the profiles folder and
still lead out of it; such a path is not treated as the library's copy, so
deleting the profile leaves that file alone. Test:
deleting_a_profile_never_follows_dot_dot_out_of_the_profiles_folder.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* LR-PROF-IMPORT: both tone fits of a DNG see its lens opcodes; no side effect in the profile-tag comparison

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* LR-PROF-IMPORT: a hover preview's key carries the camera profile file's fingerprint

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* LR-PROF-IMPORT: unusable camera profiles not offered; file's WB scale and the profile's tone kept

Review of #1063 (CodeRabbit, outside-diff findings):
- a camera profile whose file is missing or damaged when the library opens
  is no longer offered in the profile browser or accepted by develop.profile
  (it silently developed with the raw's own colour); the entry stays, so it is
  offered again once the file is back.
- a raw on the relative white-balance scale stays relative with a camera
  profile (its matrices turned it to Kelvin, so existing edits were read as
  absolute temperatures and the catalog and quick previews disagreed).
- with a camera profile its tone curve, then the tone kept for a curve-less
  profile, come before the fitted camera look's tone.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* LR-PROF-IMPORT: one Camera Profiles group; the web loader keeps a raw's own colour

Review of #1063 (CodeRabbit):
- a LUT group named "Camera Profiles" that didn't sort last made the profile
  menu list that group twice; it is now one group, last, with both kinds.
- without web workers a render ran inline and the web loader applied the
  selected camera profile, unlike the worker (the documented web behaviour
  keeps the raw's own colour); the loader now leaves it out.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* LR-PROF-IMPORT: camera profile files are read at most to the 64 MiB limit

A .dcp larger than the parser accepts was read whole into memory before
being refused. One bounded read (lightcraft_raw::dcp::read_dcp) stops one
byte past the limit and is used for import, reopening a library and the
duplicate check.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* LR-PROF-IMPORT: a test that a hostile hue/saturation map header in a camera profile drops the map (FredApps, #1063)

Dimensions far past the data, past u32 products or zero, an entry count of 4 billion values and a
data offset past the end each drop the map while the profile's matrix still reads.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* Docs: parity checklist keeps both sides of the merge

* LR-PROF-IMPORT: tests added on main take the file loader's camera-profile argument

* Merge main; a thumbnail test takes the file loader's camera-profile argument

* Docs: one camera colour row, with main's notes and the import note

* Process key tests take the profile salt argument

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: FredApps <153173854+FredApps@users.noreply.github.com>
2026-11-05 07:09:30 +01:00
FredApps 0852c21a9d Merge remote-tracking branch 'origin/main' into pr-3144
# Conflicts:
#	crates/engine/src/media/source.rs
#	docs/gaps.md
#	docs/parity-checklist.md
#	docs/raw-parity.md
#	docs/whats-new.md
2026-11-04 12:07:38 +01:00
06c1b9ae07 Use crw craftlib (#3503)
* CRW-LIB: decode Canon CRW through the shared craft-crw crate

LightCraft decodes Canon CRW (the CIFF raws of the EOS D30, D60, 10D, 300D and the
PowerShot G1/G6) through craft-libs' `craft-crw`, the clean-room decoder derived and
verified on the `clean-room-crw-format-decode` branch and moved there with its frozen
tables and derivation record. `crates/raw/src/vendor/crw.rs` is now the adapter: it maps
the crate's header and samples onto `RawImage` (CFA, levels, active area, orientation,
as-shot white balance, the make/model/time records a CRW keeps instead of Exif) and
converts the capture time through chrono. The dependency is a local path
(`../craft-libs/crates/crw`) until the craft-libs branch merges; then it becomes a git
revision.

Everything LightCraft-side from that branch comes along unchanged: `probe_bytes` fills
metadata gaps from the decoder's records so CRW imports name their camera; the EOS D60
bundled camera profile and its chart-measured dual-illuminant matrices (`chart_table.rs`,
`spectral::Camera` carries its illuminant pair) give the D60 a Kelvin white-balance scale;
CRW joins the JPEG-fitted starting look, the catalog's raw formats and `lightcraft-cli
calibrate`; `RENDER_CACHE_VERSION` 71; the CC0 CRW corpus entries for `cargo xtask corpus`;
coverage, parity, gaps, raw-parity, target-app-parity and ROADMAP updates. The G5 stays
preview-only (the crate refuses it with the reason) and the robustness test keeps its CRW
sample.

What does not come along: the decoder, its tables and `crw_oracle.rs` (now craft-crw's
digest-pinned sample test) and the derivation record, which lives in craft-libs
`crates/crw/clean-room/`. `clean-room/crw/` here keeps only the EOS D60 chart-calibration
document and its capture list, with a README pointing at the record.

Gates: fmt, clippy -D warnings (raw all targets; engine, catalog, cli), lightcraft-raw tests
(475 + 134, the corpus files decoding through the adapter), catalog tests, engine lib tests
(1,300 pass; the two failures, `tests_import::recursive_discovery_…` and
`originals::tests::write_targets_pin_absent_paths_…`, are in files unchanged from main),
`cargo xtask parity`, `assets`, `layers`, `coverage.py --check`.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

* no message

* no message

* Update documentation after moving code to craft-libs

* Review coment fixes

* CRW: pin craft-crw f3b62ec, a truncated entropy stream is Corrupt

craft-libs #472 makes craft_crw::decode refuse a stream that ends more than
Limits::max_stream_end_fill (64 samples, one block) before the frame does,
instead of filling the rest with predictor values and returning success.
The adapter already maps craft_crw::Error::Corrupt to RawError::Corrupt;
a regression test proves a truncated CRW now fails at this boundary (it
decoded as a flat frame on the old pin) while its header still reads.

Only craft-crw moves; craft-ui/docpath/update/window stay on c5a1cbf.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* Code review comment fix

* Add some tests

---------

Co-authored-by: Robert Woldberg <314159+rwoldberg@users.noreply.github.com>
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
2026-11-02 20:41:34 -05:00
samueliliandAntigravity 8ffe73cc3c fix raw lens and vignette corrections
- X-S20 lens distortion and vignetting scale fixes
- X-S20 / X-S10 / X-M5 spectral sensor mappings
- Add Samuel Li to contributors/people.toml

Co-Authored-By: Antigravity <noreply@google.com>

- address review concerns

- update RENDER_CACHE_VERSION value in docs/camera-preview-colour.md

- made sure diagonal scales are exact match
2026-10-30 08:40:55 -07:00
Dylan M. Taylor 2c7d98b5e2 refactor(raw): separate DNG metadata and opcode parsing 2026-10-28 03:24:18 +00:00
Claude 4db57c976b LR-EDIT-OPTICS-PROFILE: enable Sony ILCE-7M3 embedded distortion correction
The A7 III's raw-IFD 0x7037 tables were refused until their geometry was checked. Measured against
the camera's own JPEG of the same exposure (Lens Comp. Distortion: Auto) on 27 RAW+JPEG pairs from
one body with the FE 16-35 F4 ZA, FE 28-70 and FE 50 F1.8 (strong barrel to strong pincushion,
compressed and uncompressed, TV and monitor checkerboards): corrected board corners sit 0.06-0.44 px
(median, 6000 px frame) from the JPEG's, against 0.8-26 px uncorrected; LightCraft applies the
camera's amount of correction within 2 %; corrected rows and columns are as straight as the JPEG's.
Sony's crop and the default crop agree on this body, so the warp is centred on the image.

A unit test pins the warp from three of these tables to the camera's fitted geometry; the model
boundary tests now accept ILCE-7M3 and still refuse its neighbours. RENDER_CACHE_VERSION is bumped.
The measurement script handles screen moiré (X-corner check, duplicate corners, consensus lattice
vectors). The corpus test's two ILCE-7M3 samples now expect one warp; raw.pixls.us is not reachable
from this environment, so that test was skipped here.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015gRtdWF1E9358FfzhzYqCS
2026-10-27 10:52:36 +01:00
aiongg 59d1d6f0f8 refactor(xtask): point the raw corpus test and ci-quick map at the moved sources 2026-10-26 21:39:56 +00:00
Brandon Thomas 0605272e69 Merge PR #2620 2026-10-23 17:21:16 -04:00
aiongg f3bb7a9f30 CI: one test binary per crate, line tables only, one WASM check (#2710)
* CI: dev profile keeps line tables for workspace crates and no debug info for dependencies

* Control server tests: persistent_requests_and_slow_replies_outlive_the_idle_read_timeout no longer races the idle timeout

Each request (and the closing half-shutdown) reaches the socket before the
worker reads, so on a slow machine only the reply waits outlast the 100 ms
idle timeout, as the test intends. The idle test sends its input the same
way, and the no-frames test allows 5 s instead of 2 s to give up.

* CI: one integration-test binary per crate

Each crate's tests/*.rs become modules of tests/it/main.rs (51 test files
into 13 binaries, plus 6 that stay their own). Inner #![cfg] attributes
move to the mod lines. Files that change state for the whole process keep
a binary of their own: codecs jxl_tail_heap and raw sr2_heap (dhat global
allocator), gpu shutdown, shutdown_first and memory, ui-egui shutdown.
Docs, comments and the GPU probe name the new paths and targets
(--test it <module>::); the CI report names an it binary's skips after
the crate.

* CI: tests under nextest, doctests apart, one wasm check

xtask ci runs the tests with cargo nextest run --lib --bins --tests when
cargo-nextest is installed, else cargo test with the same targets and a
one-line notice; doctests run after them with cargo test --workspace --doc.
The heif and hardware GPU steps use the same runner. Examples stay out of
the test build (clippy --all-targets still checks them). Concurrent tests
under nextest follow the RUST_TEST_THREADS cap (NEXTEST_TEST_THREADS).
The wasm check is one cargo check --keep-going over the same crates, and
clippy runs with --keep-going.

* CI: tests run under cargo test by default, nextest with LIGHTCRAFT_NEXTEST=1 (per-process scene cache)
2026-10-23 13:34:30 -04:00
francoisgnandClaude Opus 5.5 98189d5311 Corpus tests: the Z 8 / Z 6II profile tests pair through the lens warp; Hasselblad TIFF previews pass
Two corpus tests fail on main with the CC0 corpus present (they skip without it, so no CI
has run them):

- camera_preview::tests::corpus_z8_uses_the_bundled_profile / corpus_z6ii_…: the test fitted
  the camera look with `lens: None`. Since 1ee751ebd (Nikon's in-file distortion for the
  Z 6II and Z 8, the same day as the tests) the camera's JPEG is matched through that warp,
  which files.rs reads with `lens_geometry` and removes from the opcodes before binning the
  proxy. The test now does the same; the app path was right all along.
- corpus_raw_decodes: coverage-1639.fff has no JPEG and hands on its 8-bit RGB image as a
  baseline TIFF (`rgb_preview`, b5dc500c5); the test only accepted JPEG / JPEG XL previews.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-22 23:30:04 +03:00
aiongg 70e9338085 Merge PR #1040: LR-IMP-FORMATS: decode Minolta MRW
docs/parity-checklist.md: header keeps the newer date; LR-IMP-FORMATS row keeps both sides' changes.
docs/raw-parity.md: header change note names both sides' changes.
docs/raw/coverage.tsv: merged cleanly; same rows as both sides, no duplicates (regenerated on the combined branch).
2026-10-22 02:42:05 +00:00
aiongg 04b099a4f5 Merge PR #1901: LR-EDIT-OPTICS-PROFILE: apply A7CR embedded distortion for corrected full-frame raws
crates/engine/src/camera_preview.rs: LOOK_VERSION keeps both history lines and the higher value.
crates/engine/src/media.rs: RENDER_CACHE_VERSION keeps the higher value.
docs/whats-new.md: both 2026-10-14 entries under one heading.
2026-10-22 02:40:09 +00:00
aiongg 758f2e2f21 Merge PR #671: LR-IMP-FORMATS: decode table-less Nikon 34713 NEFs stored as packed 12/14-bit data
crates/engine/src/media.rs: RENDER_CACHE_VERSION keeps the higher value.
docs/parity-checklist.md: LR-IMP-FORMATS row keeps both sides' changes.
2026-10-22 02:39:20 +00:00
Andrew FrahnandClaude Sonnet 5.5 5e1bac1264 Merge upstream/main into pr/longtail-raws
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-20 09:58:17 +11:00
Pierre-Edouard HonnetandClaude Opus 5.5 c806debada LR-IMP-FORMATS: CRX high-frequency band width from the tile geometry (#899)
The decoder tried each high-frequency band at its synthesis width and up to
two more columns, keeping the first width whose entropy stream ended exactly.
Two widths can both do that with different coefficients (#899).

Measured on every C-RAW plane in cr3_corpus.rs (CC0 raw.pixls.us files):
each band decodes at exactly one width. A tile with a neighbour on its right
codes bands 5 and 8 (the vertical high-pass of the level-2 and level-1
low-pass columns) at an odd width, so an even band gains one unused column;
no other band is extended. The width now comes from that rule, and a stream
that does not fit it is corrupt (the CR3 falls back to its preview).

All corpus sensor hashes are unchanged. The property-test encoder now codes
bands 5 and 8 the same way. Middle tiles (three or more C-RAW tiles) follow
the left-tile rule; no public file has one.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-15 20:12:46 +02:00
radialmonster 37fea78c14 LR-IMP-FORMATS: decode Coolpix NRWs stored as 16-bit big-endian words under 34713
The Coolpix P330, P7700 and P7800 NRWs carry compression 34713 with no maker
note 0x0096 table, 12 bits, one sample per pixel and a single strip of exactly
width * height * 2 bytes. The strip is one big-endian 16-bit word per pixel,
row-major with no header or padding; the sample is the word >> 4.

The layout was found by black-box analysis of the CC0 raw.pixls.us files
(strip byte-count identity, same-colour smoothness, correlation with the
embedded JPEG), not from any other converter's code.

The group is tried after the other packed layouts, so it cannot take a strip
they handle. Black level: these files have no 0x003d and no NRW block and no
masked pixels, so it is 200 (the floor of the P7800 frame is 201; the other
Coolpix NRWs state 200), never above the darkest sample. White comes from the
data. Other Nikon NEF/NRW files decode identically (278 checked).

Bumps RENDER_CACHE_VERSION to 52.
2026-10-15 08:25:08 -04:00
Alex ChambersandClaude Opus 5.5 1be3b3d743 LR-EDIT-OPTICS-PROFILE: apply A7CR embedded distortion for corrected full-frame raws
Use the corrected-radius table convention for full-frame Bayer ILCE-7CR files with distortion Auto or forced Auto. Preserve Off files, unvalidated modes and the existing ILCE-7RM4A conversion.

Add mapping, setting, scope, padding, orientation, resizing and DNG checks. Record camera-JPEG geometry and color measurements, including the remaining wide-end edge residual. Refresh stored lens data with LOOK_VERSION 11 and rendered results with RENDER_CACHE_VERSION 68.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-15 10:21:19 +02:00
Brandon Thomas 51a784cd95 Merge PR #1475 (rebased)
# Conflicts:
#	crates/raw/examples/rawinfo.rs
#	crates/raw/src/lib.rs
#	crates/raw/src/vendor/arw.rs
#	crates/raw/src/vendor/erf.rs
#	crates/raw/src/vendor/mef.rs
#	crates/raw/src/vendor/nef.rs
#	crates/raw/tests/lossy_dng.rs
#	crates/raw/tests/robust.rs
2026-10-15 01:13:36 -04:00
Brandon Thomas 66a0cd826c Merge PR #1475
# Conflicts:
#	crates/merge/src/output.rs
#	crates/raw/src/vendor/nef.rs
2026-10-15 01:05:31 -04:00
Brandon Thomas 9ca113defa Merge PR #1135 (rebased)
# Conflicts:
#	xtask/src/main.rs
2026-10-15 00:53:45 -04:00
Brandon Thomas 76881c977e Merge PR #1327 (rebased)
# Conflicts:
#	ROADMAP.md
#	docs/gaps.md
#	docs/parity-checklist.md
#	docs/target-app-parity.md
2026-10-15 00:53:22 -04:00
Brandon Thomas f7204aa8dd Merge PR #1176 (rebased) 2026-10-15 00:52:31 -04:00
Brandon Thomas f27ca795c7 Merge PR #1327
# Conflicts:
#	ROADMAP.md
#	crates/raw/tests/corpus.rs
#	docs/gaps.md
#	docs/parity-checklist.md
#	docs/raw-parity.md
#	docs/raw/coverage.tsv
#	docs/raw/evidence.tsv
#	docs/target-app-parity.md
2026-10-15 00:44:42 -04:00
Brandon Thomas f613aeba59 Merge PR #1176
# Conflicts:
#	crates/raw/src/vendor/arw.rs
#	docs/parity-checklist.md
#	docs/sony-lens-corrections.md
2026-10-14 23:54:06 -04:00
Brandon ThomasandClaude Opus 5.5 851aedc64d Batch 36: tests and tracker after the merges
- Visualize HDR (#1555) has no default key: ⇧H cycles the Tool Overlay (#1473, issue #224)
- CR3 tests (#1461, #1462) accept adaptive QP 131–175, as #1694 decodes; cr3.md says the R6 Mark II files decode
- English fallbacks in the Czech and Polish catalogs use → for the menu triangle; Polish in the keyword-label test
- canvas handle tests show pins (Tool Overlay Always); a focused handle keeps its pins shown in Auto
- the model-install test and demo deadlock guards wait on deadlines long enough for a busy machine

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-14 23:25:29 -04:00
Brandon Thomas 9ca3cd0926 Merge PR #1135
# Conflicts:
#	Cargo.lock
#	Cargo.toml
#	README.md
#	crates/raw/Cargo.toml
#	crates/ui-egui/src/theme.rs
#	xtask/src/main.rs
2026-10-14 23:23:18 -04:00
Brandon Thomas 60696fe7c1 Merge PR #1462 2026-10-14 22:29:12 -04:00
Brandon Thomas 0f842c8f87 Merge PR #1461
# Conflicts:
#	crates/raw/tests/robust.rs
2026-10-14 22:29:11 -04:00
Brandon Thomas d8b52d9405 Merge PR #1105
# Conflicts:
#	ROADMAP.md
#	docs/gaps.md
#	docs/parity-checklist.md
#	docs/raw-parity.md
2026-10-14 21:59:32 -04:00
Brandon Thomas aba04aaf5c Merge PR #1087
# Conflicts:
#	ROADMAP.md
#	docs/gaps.md
#	docs/parity-checklist.md
#	docs/raw-parity.md
#	docs/raw/coverage.tsv
#	docs/raw/evidence.tsv
#	xtask/src/main.rs
2026-10-14 21:59:31 -04:00
Brandon Thomas d3edca8000 Merge PR #1081
# Conflicts:
#	ROADMAP.md
#	docs/gaps.md
#	docs/parity-checklist.md
#	docs/raw-parity.md
#	docs/raw/coverage.tsv
#	xtask/src/main.rs
2026-10-14 21:59:29 -04:00
Brandon Thomas 6ab5e2b0d4 Merge PR #1108
# Conflicts:
#	ROADMAP.md
#	docs/gaps.md
#	docs/parity-checklist.md
#	docs/raw-parity.md
#	docs/raw/evidence.tsv
#	xtask/src/main.rs
2026-10-14 21:59:28 -04:00
Brandon Thomas 381b57dd0b Merge PR #1096
# Conflicts:
#	ROADMAP.md
#	docs/gaps.md
#	docs/parity-checklist.md
#	docs/raw-parity.md
#	xtask/src/main.rs
2026-10-14 21:59:10 -04:00
Brandon Thomas a19cbdb17b Merge PR #1045
# Conflicts:
#	ROADMAP.md
#	docs/gaps.md
#	docs/parity-checklist.md
#	docs/raw-parity.md
#	docs/target-app-parity.md
2026-10-14 21:59:07 -04:00