Files
lightcraft/crates/codecs/README.md
T
cf3d0205ef Decode AVIF through shared image codecs (#4312)
* Decode AVIF through shared image codecs

* Include JPEG encoder distribution notices

* Exercise AVIF HDR decoding through public API

* Preserve AVIF source bit depth

* Test AVIF alpha through public entrypoints

* Update AVIF round trip for supported decode

* Document native AVIF decode support

* Clarify import feature gates

* Keep RAW fixtures valid under shared container and strip bounds

* Include sample-to-chunk mappings in CR3 fixtures

* Find Similar test: a photo read from memory isn't a missing read (#3823)

The second search after a stop plans its reads after frames in which the
loupe and filmstrip decoded photos; those are measured from memory
(source_ref takes the cached source), so the drive sees fewer than 12 reads
on a loaded machine. Check instead that no file is read twice, the searched
photo above all; the answer still needs every photo's signature.

* Find Similar test: every photo is measured after the resumed search

* Bound AVIF padded picture storage and preserve integer source flag

* Size the AVIF picture-store budget for the stored canvas, not the clap-cropped size

* Attribute the AVIF lossless decode fixture

* Keep the AVIF clap test helpers free of unwrap for the codecs lint gate

* lightcraft: keep the newer craft-cms pin after merging main

---------

Co-authored-by: Bruno-Alumn <171815746+Bruno-Alumn@users.noreply.github.com>
Co-authored-by: FredApps <153173854+FredApps@users.noreply.github.com>
2026-11-18 01:58:41 -05:00

132 lines
9.7 KiB
Markdown
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
# lightcraft-codecs (L1)
Standard image formats for LightCraft: format sniffing, decoding to **linear light** with colour
management, fast thumbnails, and encoders with metadata embedding. Pure Rust, no C dependencies,
builds for `wasm32-unknown-unknown`.
## API at a glance
```rust
let fmt: Option<Format> = sniff(&bytes); // JPEG/PNG/TIFF/WebP/AVIF/HEIF/JXL/GIF/BMP/PSD/raw
let d: Decoded = decode(&bytes, DecodeOptions::default())?;
// d.image: Rgb32f, linear light in the source primaries (d.space: SourceSpace)
// d.alpha, d.bit_depth, d.float, d.orientation (EXIF value, NOT applied), d.exif/xmp/icc (raw blobs)
let working: Rgb32f = to_working(&d); // linear Rec.2020 D65 (Bradford), alpha flattened onto white
let fast = decode(&bytes, DecodeOptions::fit(2048, 2048))?; // JPEG: DCT-domain 1/2..1/8 scaling
let t: Thumbnail = decode_thumbnail(&bytes, 256)?; // embedded EXIF/MPF preview or scaled decode
let h: Header = read_header(&bytes)?; // stored width/height + orientation, no pixels decoded
let icc = icc::write_named(NamedSpace::DisplayP3); // or icc::write_matrix_trc(&space, &Trc::…)
let meta = EncodeMeta { icc: Some(&icc), exif: Some(&exif), xmp: Some(&xmp) };
encode_jpeg(&EncodeImage::rgba8(&img), 90, ChromaSubsampling::S444, &meta)?;
encode_png(&EncodeImage::new(w, h, 4, Samples::U16(&px)), &meta)?;
encode_tiff(&EncodeImage::new(w, h, 3, Samples::F32(&px)), TiffCompression::Deflate, &meta)?;
encode_webp_lossless(&img, &meta)?;
encode_avif(&img, 70, 6, &meta)?; // native + feature `avif`
encode_jxl(&img, &JxlOptions { quality: 90, ..Default::default() }, &meta)?; // native + feature `jxl`
// HDR gain map JPEG (ISO 21496-1): SDR + HDR renditions → base + gain map, and back
let (map, gm) = gainmap::compute(&sdr_linear, &hdr_linear, w, h, luma, &GainMapOptions::default())?;
gainmap::encode_jpeg(&EncodeImage::new(w, h, 3, Samples::U8(&base8)), &map, &gm, 90, ChromaSubsampling::S444, &meta)?;
let found = gainmap::read_jpeg(&bytes); // ours, cameras', Lightroom's
gainmap::apply(&mut base_linear, w, h, &found.gain, &found.meta, 1.0)?;
```
Thumbnail fallback decoding limits the source to 64 million pixels by default.
Non-JPEG decoders allocate at source resolution before resizing, even for a tiny
thumbnail. Use `decode_thumbnail_with` and `ThumbnailOptions::max_pixels` to set
a different source budget. Oversized sources return `Error::TooLarge`; embedded
JPEG previews can still be used without decoding the full source image.
`Format::RawTiffLike` (DNG, CR2, NEF, ARW, PEF, ORF, RW2, SRW, …) and `Format::RawOther` (CR3, RAF,
CRW, MRW, X3F) are detected so the engine can route them to `lightcraft-raw`; `decode` returns
`Error::Unsupported` for them.
## Formats
| Format | Decode | Encode | Notes |
|---|---|---|---|
| JPEG | yes (zune-jpeg; jpeg-decoder for DCT scaling, CMYK/YCCK, 12-bit, non-interleaved scans) | yes (shared `craft-jpeg`, optimized baseline, 4:4:4/4:2:2/4:2:0) | EXIF (APP1), XMP (APP1, ≤ 64 KiB), ICC (APP2; only complete, consistently counted, uniquely numbered chunk sequences are assembled; malformed profiles use the existing unsupported-profile fallback), Adobe APP14, MPF previews |
| PNG | yes, 1–16-bit, palette, tRNS, interlaced | 8/16-bit, gray/GA/RGB/RGBA | iCCP, eXIf, iTXt XMP; sRGB/cICP/cHRM/gAMA honoured when no ICC |
| TIFF | 8/16-bit int (32/64-bit int kept to 16 bits), 16/32/64-bit float, gray/RGB/CMYK/palette, alpha (assoc./unassoc.), planar, LZW/Deflate/PackBits/JPEG | 8/16-bit, 32-bit float; none/LZW/Deflate/PackBits | ICC (34675), XMP (700), Orientation; first IFD only; EXIF not written (use `craft-tiff`) |
| WebP | lossy + lossless, alpha, first frame of animations | lossless only | ICCP/EXIF/XMP chunks |
| GIF / BMP | yes (first GIF frame) | — | via `image` |
| PSD / PSB | merged composite: 8/16/32-bit gray, RGB, CMYK, indexed, duotone (as gray); raw/RLE | — | ICC (1039), EXIF (1058), XMP (1060); ZIP-compressed composite, Lab and 1-bit unsupported |
| JPEG XL | yes (`craft-jxl`, feature `jxl`, default on) | yes (`craft-jxl`, `encode_jxl`, feature `jxl`, native only): lossy VarDCT / lossless Modular, 8/16-bit, effort (fast / normal), progressive; HDR as 16-bit Rec. 2020 PQ | decode: enum colour → rendered straight to linear Rec.2020 (XYB via linear sRGB primaries); ICC → our ICC path; orientation applied by the shared decoder (reported as 1). Encode: lossy files declare an enum encoding (sRGB, Display P3, Rec. 2020 BT.709 curve, Rec. 2020 PQ), lossless ones embed the ICC profile; EXIF / XMP in `Exif` / `xml ` boxes |
| AVIF | **no** | yes (ravif/rav1e, native only, feature `avif`) | 8/10-bit sRGB, EXIF; no ICC in the muxer. HDR: `encode_avif_hdr` writes 10-bit BT.2020 PQ (SDR white at 203 cd/m², CICP `nclx` + `clli`) or HLG (`hlg_signal`, SDR white at 0.75) |
| HEIC/HEIF | yes with feature `heif` (craft-heif: heic-rs + libheif-matching colour; 4:2:0/mono, 8–10-bit, grids, irot/imir/clap, ICC/nclx/VUI, EXIF/XMP, thumbnail item) | — | 4:2:2/4:4:4 (Canon/Sony HIF) unsupported |
| Gain map JPEG (HDR) | gain map found and decoded (`gainmap::read_jpeg`): ISO 21496-1 metadata, else Adobe `hdrgm` XMP | yes (`gainmap::encode_jpeg`) | CIPA DC-007 MPF index (gain map typed `0x050000`), ISO 21496-1 APP2 on both images, `hdrgm` + Container XMP for Android/Chrome, Apple `HDRGainMap`/`HDRToneMap` XMP; gain maps are never used as thumbnails |
## Colour
Decoders never guess silently; `SourceSpace::origin` says where the interpretation came from:
`IccMatrixTrc` (profile applied exactly: TRC linearization + colorants → XYZ D50), `IccCms` (LUT-based
or CMYK profile converted by the CMS straight to linear Rec.2020), `Container` (PNG chunks, JXL enum,
EXIF `R03` Adobe RGB hint), `Untagged` (assumed sRGB; untagged float data assumed linear),
`IccUnsupported` (profile present but unusable → **sRGB fallback**), `Naive` (CMYK without profile).
`decode_jpeg_with_fallback(bytes, opts, space)` lets an enclosing RAW container supply the colour
space of a JPEG with no ICC or EXIF metadata. It applies the transfer curve before linear-light
resizing; a JPEG carrying its own metadata retains its existing interpretation.
ICC profiles are parsed with `moxcms` (v2/v4, `curv`/`para` TRCs, `chad`); sRGB, Display P3,
Adobe RGB (1998), ProPhoto (ROMM) and Rec.2020 are recognised by colorants. `icc::write_matrix_trc`
emits v4 display profiles (D50 PCS, Bradford `chad`) for export embedding.
Before colour conversion, positive-infinity float samples become 1.0; NaN and negative infinity
become 0.0. This also applies to alpha. Finite samples keep their existing range: matrix/TRC paths
preserve values above 1.0 and below 0.0, while CMS inputs and alpha remain bounded to 0..1.
## Dependencies and licences
All pure Rust, verified with `cargo info` / `cargo tree`:
| Crate | Licence | Use |
|---|---|---|
| zune-jpeg, zune-core | MIT OR Apache-2.0 OR Zlib | JPEG decode |
| jpeg-decoder | MIT OR Apache-2.0 | scaled / CMYK JPEG decode |
| craft-jpeg | MIT OR Apache-2.0 | baseline JPEG encode (shared craft-libs) |
| jpeg-encoder | (MIT OR Apache-2.0) AND IJG | 0.6 still ships inside `craft-image-codecs` (IJG: permissive, attribution in docs); 0.7 only in tests and the JPEG benchmark (reference CMYK fixture, prior 4:2:2 output) |
| png | MIT OR Apache-2.0 | PNG |
| tiff | MIT | TIFF |
| image-webp | MIT OR Apache-2.0 | WebP |
| image (gif, bmp only) | MIT OR Apache-2.0 | GIF/BMP |
| moxcms | BSD-3-Clause OR Apache-2.0 | ICC parse/write, CMS |
| craft-jxl | MIT OR Apache-2.0 | JPEG XL decode/encode (shared craft-libs) |
| jxl-oxide | MIT OR Apache-2.0 | JPEG XL test oracle only |
| ravif (+ rav1e, avif-serialize) | BSD-3-Clause / BSD-2-Clause | AVIF encode (native only; heavy to compile) |
Deliberately **not** used: anything AGPL from imazen (heic, zenjpeg, jxl-encoder, rav1d-safe),
jpegxl-rs / imagequant (GPL / C), dav1d / libheif (C).
## Performance (Apple M-series, release, synthetic 24 MP JPEG q90 4:2:0)
Run `cargo test --release -p lightcraft-codecs --test it bench:: -- --ignored --nocapture`.
| Operation | Time |
|---|---|
| full decode → linear `Rgb32f` | ~92 ms (≈ 260 MP/s) |
| decode fit 2048 (DCT 1/2) | ~81 ms |
| thumbnail 256, scaled decode (DCT 1/8) | ~54 ms (entropy decoding bound) |
| thumbnail 256 from embedded EXIF/MPF preview | ~0.6 ms |
| `to_working` matrix | ~23 ms |
| PNG / TIFF-deflate encode (fast levels) | ~430 ms / ~900 ms |
## Gaps / limitations
- **AVIF decode** uses the shared pure-Rust AV1 decoder on native builds with the `avif` feature.
Animated AVIF, depth and gain-map auxiliary images, and progressive/layered images are unsupported.
- **HEIC** needs the `heif` feature (off by default: HEVC patents are the distributor's call);
without it, `Error::Unsupported` says the build has no HEIC support. 4:2:2 and 4:4:4 HEIF
(Canon/Sony HIF) is refused: heic-rs 0.1.1 loses sync on those bitstreams.
- JPEG: arithmetic coding unsupported (as in both decoders); extended XMP (> 64 KiB) neither read
nor written; EXIF > 64 KiB not written. The encoder uses standard Huffman tables (interleaved
baseline) for maximum compatibility.
- Thumbnails under 15 ms need an embedded preview ≥ the requested edge (EXIF IFD1 or MPF). Most
cameras store a 160 px EXIF thumbnail: use `decode_thumbnail_with` with `min_embedded_edge: 160`
for an instant placeholder, otherwise the DCT-scaled path takes ~50 ms for 24 MP.
- TIFF: first image only; EXIF is not re-emitted on encode; Lab/uncompressed-YCbCr unsupported.
- PSD: only the merged composite (no layers); ZIP-compressed composites unsupported.
- Panics inside third-party decoders are caught (`catch_unwind`) on unwinding targets; on wasm32
(`panic = abort`) we rely on the property tests of our own code plus the decoders' own hardening.