* CRW-LIB: decode Canon CRW through the shared craft-crw crate
LightCraft decodes Canon CRW (the CIFF raws of the EOS D30, D60, 10D, 300D and the
PowerShot G1/G6) through craft-libs' `craft-crw`, the clean-room decoder derived and
verified on the `clean-room-crw-format-decode` branch and moved there with its frozen
tables and derivation record. `crates/raw/src/vendor/crw.rs` is now the adapter: it maps
the crate's header and samples onto `RawImage` (CFA, levels, active area, orientation,
as-shot white balance, the make/model/time records a CRW keeps instead of Exif) and
converts the capture time through chrono. The dependency is a local path
(`../craft-libs/crates/crw`) until the craft-libs branch merges; then it becomes a git
revision.
Everything LightCraft-side from that branch comes along unchanged: `probe_bytes` fills
metadata gaps from the decoder's records so CRW imports name their camera; the EOS D60
bundled camera profile and its chart-measured dual-illuminant matrices (`chart_table.rs`,
`spectral::Camera` carries its illuminant pair) give the D60 a Kelvin white-balance scale;
CRW joins the JPEG-fitted starting look, the catalog's raw formats and `lightcraft-cli
calibrate`; `RENDER_CACHE_VERSION` 71; the CC0 CRW corpus entries for `cargo xtask corpus`;
coverage, parity, gaps, raw-parity, target-app-parity and ROADMAP updates. The G5 stays
preview-only (the crate refuses it with the reason) and the robustness test keeps its CRW
sample.
What does not come along: the decoder, its tables and `crw_oracle.rs` (now craft-crw's
digest-pinned sample test) and the derivation record, which lives in craft-libs
`crates/crw/clean-room/`. `clean-room/crw/` here keeps only the EOS D60 chart-calibration
document and its capture list, with a README pointing at the record.
Gates: fmt, clippy -D warnings (raw all targets; engine, catalog, cli), lightcraft-raw tests
(475 + 134, the corpus files decoding through the adapter), catalog tests, engine lib tests
(1,300 pass; the two failures, `tests_import::recursive_discovery_…` and
`originals::tests::write_targets_pin_absent_paths_…`, are in files unchanged from main),
`cargo xtask parity`, `assets`, `layers`, `coverage.py --check`.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* no message
* no message
* Update documentation after moving code to craft-libs
* Review coment fixes
* CRW: pin craft-crw f3b62ec, a truncated entropy stream is Corrupt
craft-libs #472 makes craft_crw::decode refuse a stream that ends more than
Limits::max_stream_end_fill (64 samples, one block) before the frame does,
instead of filling the rest with predictor values and returning success.
The adapter already maps craft_crw::Error::Corrupt to RawError::Corrupt;
a regression test proves a truncated CRW now fails at this boundary (it
decoded as a flat frame on the old pin) while its header still reads.
Only craft-crw moves; craft-ui/docpath/update/window stay on c5a1cbf.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Code review comment fix
* Add some tests
---------
Co-authored-by: Robert Woldberg <314159+rwoldberg@users.noreply.github.com>
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
* security: harden dev-tool scripts and the testkit skip-log test (CodeRabbit)
- tools/install-sam3.sh: the HF bearer token went through curl's argv, where
`ps` exposes it to other local accounts (CWE-214); it now travels through
a private 0600 temp config file removed on exit.
- packaging/macos/import-cert.sh: the decoded .p12 was written to a
predictable path in a shared TMPDIR, where a precreated attacker-readable
file would receive it (CWE-377); both cert and keychain are created inside
a mktemp -d 0700 directory now.
- docs/showcase/run.py: the reply reader copied whole responses with no size
cap (obvious quadratic-copy and unbounded-memory DoS). It accumulates in a
bytearray now and closes the connection past 128 MiB.
- tools/spectral_camera_table.py: alias/make/model strings and comments from
the rawtoaces-data checkout moved directly into Rust source; one newline in
a comment became Rust code. External text is now escaped as Rust literal
content and comments are normalized to one printable line.
- crates/testkit: appended_lines_accumulate reused a pip-predictable temp
dir by creating it with create_dir_all and carried on after failing to
remove its log path, so the append would follow an attacker-planted
symlink (CWE-59); it now fails the test if the directory exists instead of
testing through it, and removes any existing log path only after clearly
checking it, always failing the test instead of proceeding when removal fails.
Findings #10080/#10078 PR (rows 2, 6, 8, 9, 15 of the
storytold-lightcraft-agent-findings report).
🤖 Generated with Codebuff
Co-Authored-By: Codebuff <noreply@codebuff.com>
* security: bounded catalog keyword, repair, journal recovery work (CodeRabbit)
- keywords: from_file searched every hierarchical level for each flat keyword
and `same` lower-cased two strings per comparison — one lowered level set
instead (CWE-407, row 97).
- keywords: parse_keyword_list scanned each keyword's growing synonym list per
line — a lowered-synonym set per keyword instead (CWE-407, row 100).
- keywords: keyword_tree stored every level's prefix twice, so a photo XMP
keyword nested deeper than a list file accepts made stored bytes quadratic
in the level count; the tree caps at MAX_LEVELS, and keyword_list_text names
whatever lies deeper from the photos' and list's own spellings, so the file
still reads back (CWE-400, row 103).
- catalog: repair validated a folder chain once per album — memoized by chain
now, so a valid chain in a flat snapshot costs one walk overall (CWE-407,
row 101).
- store: journal append and truncate follow a planted catalog.log symlink;
both refuse any non-regular file at the name before writing through it
(CWE-59, row 107), like the staging writes since #1502.
- journal: resync re-validated the whole remaining suffix UTF-8 for every
candidate record start; a whole line is validated once (covering every
candidate in it), and an invalid line's re-checks stop at a 64 MiB budget,
with the tail then staying a torn one the damaged path keeps and reports
(CWE-407, row 109).
New tests: from_file case-insensitive level match, synonyms kept once, a
4096-level keyword cut at MAX_LEVELS, a link at catalog.log refused, and
resync over a fragment of 1024 record starts.
Findings rows 97, 100, 101, 103, 107, 109 of the
storytold-lightcraft-agent-findings report.
🤖 Generated with Codebuff
Co-Authored-By: Codebuff <noreply@codebuff.com>
---------
Co-authored-by: Codebuff <noreply@codebuff.com>
Resolve and validate command-line path arguments before use, enforce HTTPS at
download sinks, and confirm path containment. Hardens local codegen/fetch tools
against path-traversal and command-argument-injection findings.
A raw without colour matrices of its own now gets DNG ColorMatrix1/2 and
ForwardMatrix1/2 for its model when the model is one of the 52 cameras
in rawtoaces-data (Academy Software Foundation, Apache-2.0): 22 Canon,
11 Nikon, 11 Sony, 3 Fujifilm and 5 others, plus the other names its
alias list gives (Rebel T3i and Kiss X5 for the EOS 600D; bodies it
lists as sharing a sensor). tools/spectral_camera_table.py fits, under
Standard Illuminant A and D65, a white-preserving 3x3 to the measured
spectral sensitivities over rawtoaces-data's 190 training reflectances
and writes crates/raw/src/spectral_table.rs. No number comes from Adobe
or from a raw decoder. lightcraft_raw::spectral looks a camera up by
Exif make and model; the DNG colour model does the rest.
camera_preview::starting_colour takes the first available of: the
file's own matrices, a camera profile for the model (fitted to the
JPEG for tone), the spectral matrices, the file-local JPEG fit, the
neutral fallback. A profiled model whose profile fit fails takes its
spectral matrices. White balance stays relative to the as-shot look for
every raw without matrices of its own, so Photo::relative_wb and the
controls are unchanged; calibrate still pools covered models.
Tests: one per precedence step (synthetic), the table's lookup, aliases
and DNG shape, and a corpus test of the order on ILCE-7M4 (profile),
EOS 5D Mark III and D5100 (spectral), EOS 7D (own fit) and a DNG. The
ILCE-7RM2 and GFX100S corpus tests now expect the spectral matrices.
Attribution in assets/ATTRIBUTION.md and NOTICE. RENDER_CACHE_VERSION 23.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>