Port samhyun’s progressive JPEG sampling guard from storytold/figcraft#1491,
source commit 9aac391e045c3c5894c1112e537d8ea2da382521.
Shared API and existing frame-walker adaptation by Dylan M. Taylor.
Code and base are identical to storytold/craft-libs#2294 at
7f868f0a109aa1c301b79b9aa2c06626689fcd55; this replacement corrects attribution.
Co-authored-by: samhyun <rnflal31@gmail.com>
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
* image-codecs: find the JPEG-LS frame header past 64 segments, and add detect_prefix
detect walked at most 64 marker segments looking for the SOF55 frame
header, so a JPEG-LS file with more segments before its frame header,
such as an ICC profile split over 255 APP2 segments, was reported as
JPEG. The walk now runs until it reaches a frame or scan header or the
end of the bytes. Each step moves forward at least one byte, so the walk
takes at most one step per byte.
detect also reported JPEG for a buffer that ended before the frame
header, which is what a caller that reads only the start of a file
passes. The new detect_prefix returns Detection::NeedBytes(n) in that
case, with the length the next step reads, and otherwise the answer
detect gives. The doc comment on detect now says that it needs the bytes
up to the frame header.
* image-codecs: walk each JPEG marker segment once when detecting in steps
detect_prefix walks the marker segments from the SOI on every call, so a
caller that reads exactly the length each Detection::NeedBytes asks for
walks the earlier segments again on each call. Over a file with many
small segments before its frame header, the total walk grows with the
square of the segment count.
The new PrefixDetector keeps the offset of the marker where its last
walk stopped and starts the next walk there. Its detect gives the same
answer and asks for the same lengths as detect_prefix, and
detect_prefix now runs a single step of a new PrefixDetector.
* image-codecs: read past a JPEG-LS restart interval segment and stop at FF 00
The JPEG-LS frame walk treated a DRI segment before the SOF55 frame header
as the end of the walk, so a file with one was detected and decoded as
JPEG. DRI now takes the length-prefixed path. The decoder accepts a 2-, 3-
or 4-byte interval of 0, reports a nonzero interval as unsupported, and
rejects other segment sizes as malformed.
`FF 00` after the SOI is a stuffed zero, not a marker, so the walk now ends
there instead of asking for a length field.
A PrefixDetector whose walk stopped at another frame header answers JPEG
for bytes that start with `FF D8 FF`, without walking from the SOI again.
* image-codecs: apply the JPEG-LS restart interval in effect at each scan
A later DRI with an interval of 0 turns restarts off again, and an
interval of at least the frame height puts a non-interleaved scan in
one interval with no restart marker. Both decode again. A nonzero
interval shorter than the frame height is still unsupported.
* image-codecs: reject a restart marker after a JPEG-LS scan of one interval
A restart marker comes only between two restart intervals. When the
interval in effect is at least the frame height, the scan is one
interval, so a restart marker after it, with or without fill bytes
before it, is now an error instead of being skipped.
Adds a test that a PrefixDetector starts each call at the marker where
its previous walk stopped.
The VP8 encoder cloned the whole luma reconstruction before each
macroblock's B_PRED trial and restored it when a whole-block mode won,
so encoding P pixels copied about P^2/256 bytes. The trial writes only
the current macroblock, and the whole-block path writes every pixel of
that macroblock again without reading the trial's values, so the copy
and the restore are removed.
The output is byte-identical: a new test pins the SHA-256 of ten
encoded files (1x1 to 640x480, sizes that are not multiples of 16,
qualities 10 to 95) taken before the change, and decodes each with
image-webp. An ignored test prints encode times for 512 to 4096
squares; in release on one machine, 4096x4096 took 162.9 s before and
1.36 s after.
* craft-docx: keep a table style's w:tblInd on read and save
A table style's own indent was dropped on read, so it was gone on save.
pandoc's Table style sets it to 0, and Word for Mac places a table whose
style chain sets no indent about 5.4 pt further right than one set to 0.
craft-wordmodel gains TableStyleParts::indent, merged through basedOn.
The reader takes a dxa indent (nil is 0; pct and auto set none) and the
writer emits it in schema order, 0 included.
Part of storytold/wordcraft#5767.
* craft-docx: a table style's w:tblInd without w:w is 0
CT_TblWidth's w:w defaults to 0, so the style resets a based-on style's
indent instead of inheriting it. The test helper now fails when a saved
style is missing rather than matching an empty string.
* craft-docx: the tblInd override test checks the base sets an indent
Without it the test would pass if the reader dropped the base style's
indent, since there would be nothing to override.
* craft-docx: theme font slots survive a save
w:rFonts' w:eastAsiaTheme and w:cstheme slots are kept on read and written
back. Word picks the East Asian and complex-script fonts through them, so
dropping them changed which font Word used for East Asian punctuation and
its line breaks. The Latin slots (w:asciiTheme, w:hAnsiTheme) are kept when
the run's font came from them and written back only while they still name
that font in the document's theme, so a font chosen later is not hidden.
craft-wordmodel: a one-byte ThemeFont and CharProps font_ascii_theme,
font_hansi_theme, font_ea_theme and font_cs_theme. Overlaying a named font
without its slot clears the slot; minus keeps a slot whose font stays.
Ported from storytold/wordcraft#5772.
* craft-docx: keep the theme's East Asian and script fonts; exact slot replacement
The saved theme wrote empty a:ea and a:cs faces and no per-script fonts, so
a kept w:eastAsiaTheme or w:cstheme slot named no font in Word. The reader
now keeps the theme's ea, cs and a:font script entries (THEME_FONTS) and
the writer puts them back.
craft-wordmodel: a named font replaces both Latin slots with its own, so a
patch with one slot no longer leaves the other stale. minus keeps a font
equal to the base's when its slots differ, so a font that clears the
base's slots still clears them when overlaid. latin_slot compares the
theme face without cloning it.
* craft-docx: theme fonts keep faces and scripts apart; limit only usable scripts
THEME_FONTS lines now say whether they hold a default face (a:ea, a:cs) or a
per-script font, so an a:font script="cs" is no longer written back as the
a:cs face. The 256-entry limit applies to usable per-script fonts only, so
empty default faces no longer push the last one out.
A test pins down Latin slots that name different faces: the model has one
Latin font, so only the slot that still names it is written.
* craft-docx: a long theme script id never pushes out one font resolution uses
The kept per-script fonts count script ids of 8 characters or fewer (the
ones font resolution takes) and longer ones (language tags such as
zh-Hant-TW) apart, 256 each, so a long one can't use up the place of a
mapping resolution keeps.
* craft-docx: THEME_FONTS names both per-script caps
* craft-docx: a DOCX border without w:sz is an eighth of a point
* craft-docx: a table border without w:sz is width 0, and ODT and RTF keep it
A table, cell or table-style border without w:sz gives the same output in
Word as w:sz="0" and takes no room; read as half a point, rows below a
pandoc table's header rule moved down half a point. PropCtx::table_borders
now reads it as 0, as PropCtx::borders does since #1488.
ODT and RTF have no border that is drawn and takes no room, so a drawn
width-0 cell side is written at Border::THINNEST instead of being dropped.
Where two ODT cells share an edge, a line with a width wins over a width-0
one, whichever cell states it.
Table half of storytold/wordcraft#5764.
---------
Co-authored-by: Nghiathan13 <201675788+Nghiathan13@users.noreply.github.com>
The VML reader took a shape's outline from its stroked and strokecolor
attributes only, so a v:stroke child with on="f" or opacity="0" was
ignored and the shape got a black 0.75 pt outline. Google Docs writes
its horizontal rules this way (a 0.05 pt v:rect with a transparent
stroke): Word shows nothing, and after a save every rule became a
visible line.
Each stroke property (on, color, opacity, endcap) now resolves as Word
does: the shape's own v:stroke, then the shape's attribute (stroked,
strokecolor), then the same two on its shapetype. A fully transparent
line is kept as a shape without an outline, and freeforms take the
resolved opacity, inherited ones included. Opacity is read in all three
forms a file can use (0, 0% and 65536ths such as 0f).
Moved from storytold/wordcraft#3829, where it was reviewed, now that
the DOCX reader lives here.
Co-authored-by: ARC <261443955+arc-uri-el@users.noreply.github.com>
From 45° up to 135° and from 225° up to 315°, Word measures the room a
turned object's rotated bounds take against its frame with width and
height exchanged, so a quarter turn records no effect extent. craft-docx
measured it against the unturned frame, so saving a picture turned 90°
wrote half the difference between its sides as negative top and bottom
extents, which Word draws as a crop, and the text after it moves up.
Reading had the same mismatch the other way: a picture Word turned 60°
gained phantom effects room above and below.
Float::effect_spin_pad gives the overhang in Word's measure, choosing the
frame from the angle as it is saved (60000ths of a degree), so an angle
such as 44.999996° is measured as the 45° it is written as. The docx
reader and both writers use it; a bare turn (Float::is_bare_turn) still
records none, and layout keeps spin_pad and line_pad.
The test reads and saves a 254 x 337 pt picture with the extents Word
for Microsoft 365 (build 16.0.20527) writes at 60° and 90° and on both
sides of each boundary (44/45, 134/135, 224/225, 314/315), and reads the
saved file back. Two existing tests asserted the old values and now use
Word's.
Moved from storytold/wordcraft#3828, where it was reviewed, now that the
DOCX reader and writer live here.
Co-authored-by: ARC <261443955+arc-uri-el@users.noreply.github.com>
* Add craft-color-transform: RGB spaces, white balance, OkLab, tone curves and PQ tone mapping
craft-color-transform builds on craft-color. It holds RGB spaces from
primaries and a white point with Bradford adaptation, white balance by
correlated color temperature and tint, OkLab and OkLCh for any RGB space,
monotone tone curves and 1D lookup tables, and the BT.2408-8 Annex 5 EETF
with a maxRGB tone map. The per-pixel functions take f32 and do not
allocate, and the crate builds for wasm32-unknown-unknown.
Moved from StudioCraft's craft-color-transform
(blamejs/StudioCraft crates/craft-color-transform). The README names the
LightCraft and FilmCraft modules parts of it started from.
* Color transform: compose, interpolate and invert at extreme values
Lut1::compose samples the densest grid of at most MAX_LUT entries that holds both tables' points, so a short inner table, or one that crosses an outer peak between its own points, keeps the outer table's points to within one grid step. Lut1::eval weights the two entries in f64, so entries as far apart as -f32::MAX and f32::MAX no longer give NaN or infinity. MonotoneCurve::new compares the tangent pair's length with 3|d|, so a subnormal secant next to a normal one no longer gives a NaN tangent. Mat3::inverse divides the matrix by its largest entry before taking the determinant, so diag(1e110) and diag(1e-200) invert. OkLab::for_space falls back to XYZ_TO_LMS when a matrix has no inverse or an entry beyond f32.
* Color transform: build the matrix arithmetic on craft_cms::math
craft_cms::math gains the two hardened functions the color transforms
need, and craft-color-transform calls them instead of carrying its own
copies.
- math::inverse divides the matrix by its largest entry before taking
the determinant, so a matrix whose own determinant overflows or
underflows still inverts, and it returns None for NaN, infinite or
all-zero input and for an inverse beyond f64. math::invert keeps its
absolute 1e-12 floor, which profile construction uses to validate ICC
matrices.
- math::bradford_checked returns None when a white's cone response is
not finite or not above 0, or the ratios overflow. math::bradford
still divides without the check.
- math::BRADFORD is public.
- Mat3::mul, apply, determinant and inverse, Xy::to_xyz, BRADFORD and
bradford_xyz call craft_cms::math.
- oklab::lms_matrices gives OkLab::for_space's matrices in f64.
* Color transform: keep the EETF's black lift rising for a high target black
The BT.2408 Annex 5 lift minLum * (1 - E2)^4 has a slope of 1 - 4 minLum
at E2 = 0, so a target black above a quarter of the normalized range
made the output fall near black, and it raised the peak well above the
target peak.
The lift is now minLum * (1 - E2 / r)^k with r = max(T, min(1, 2 -
4 minLum)) and k = min(4, r / minLum), where T is the top of the output
(the smaller of maxLum and 1). It equals the Annex's lift up to minLum
0.25, keeps the output rising, changes continuously with the target
black, and from minLum 0.5 reaches 0 at the top of the output. A target
black above the source peak reads as the source peak.
* Color transform: state the composition bound for the fallback grid
* craft-geom and craft-develop-model: develop settings and crs: import, from LightCraft
craft-develop-model is lightcraft-develop (settings, controls, presets,
segmentation masks, process versions) plus the crs: import from
lightcraft-engine (crs.rs, crs_masks.rs). craft-geom is lightcraft-geom,
which the model and the coming develop pipeline both need. Step 3 of #587.
* craft-develop-model, craft-geom: review fixes to copy, presets, section reset and crs: import
Copy and presets: to_json_for_merge states every setting the stored JSON
leaves out at its default, so copying a group turns HDR, the camera's local
tone and Lens Blur's optional settings off on the target, and a preset at an
amount starts from them. A preset at no amount adds no mask, and its Lens
Blur focus mask follows its mask to the id the photo gives it.
Reset and range: reset_section resets the sliders of point colours and red
eyes, Upright with its guides, and the two Optics switches. Settings from
outside the app keep the list sliders in range. is_unedited counts a chosen
white balance as an edit.
crs: import: a brush dab that is not a finite number is dropped instead of
failing the whole packet, brush runs that differ in MaskInverted stay two
components, RetouchInfo is read only without RetouchAreas, and an XMP preset
keeps its masks. SegMask::from_logits does not overflow on an absurd side.
craft-geom: Homography::from_quads solves a mapping that sends the origin
to infinity, by solving again about the middle of the source quad.
Docs: the links to LightCraft's light-v3.md and grain.md point at that
repository, and xmp-interop.md says when the older fields are read. The
README lists what now differs from LightCraft.
* craft-develop-model, craft-geom: second round of review fixes to crs: import, mask grids, crop and quads
A range mask with a number that is not one failed the whole merge; a straight legacy ToneCurve and a
photo's switched-off masks or spots were left unsaid, so the photo kept what it had; a mask grid of a
side past the limit was cut to 32 bits; a crop restored under a new angle could reach outside the
photo; a quad mapping onto a collapsed quad solved to a matrix that does not map it.
Ported from storytold/wordcraft#6242 (issue storytold/wordcraft#6212), written before the RTF reader
moved here. Rectangles, ellipses, triangles, diamonds, block arrows and text boxes are read from
the shape's properties (position, size, wrap, fill, line, text); the \defshp picture Word writes
after a shape it read, also inside \shppict, is not imported as a second copy.
Co-authored-by: Mauro Mazzieri <mauro.mazzieri@talete.net>
R16G16_UNORM stores a red and a green channel
(https://learn.microsoft.com/en-us/windows/win32/api/dxgiformat/ne-dxgiformat-dxgi_format),
but craft-dds read it as gray and alpha, so a two-channel normal map
opened with its green channel as transparency. It now reads as RGB with
blue 0, as BC5 and G16R16 channel masks already do. encode16 still
writes two-channel 16-bit pixels as R16G16; they read back as red and
green.
craft-image-codecs decoded every DDS at 8 bits. Texture::bits_per_channel
gives the widest stored channel, and textures wider than 8 bits (R16,
R16G16, R16G16B16A16, legacy FourCC 36, and channel masks with a 10- or
16-bit field) now open as U16 through decode16. craft-raster-codecs'
decode_levels does the same for every mip level.
When the cross-reference data is rebuilt from object headers, the index
was keyed by object number alone, so a later `N 1 obj` replaced
`N 0 obj` while the renderer still drew `N 0 obj` under `N 0 R`. cos
read the page's object as null and a full save wrote null
(storytold/pdfcraft#6759).
Each number now keeps its object by the renderer's rule: the last real
`N 0 obj`, else the last header as a generation-0 stand-in. Where a
number has headers of several generations, a bounded walk from the
trailer gives the slot to the one generation the document reaches,
when it reaches one only, so a reused number such as PDFBOX-3785's
`202097.pdf` keeps its page. A trailer /Root is ranked by its own
generation's header, and a slot whose object stream supplies indexed
members stays. What one slot cannot hold goes to the repair log, and
Document::rebuilt_with_unsettled_generations() says when a save could
differ from what the renderer drew.
Port of storytold/pdfcraft#6767, which can't land in pdfcraft since cos
moved here.
* image-codecs: read JPEG XL through craft-jxl behind a jxl feature
Format::Jxl reads when the non-default `jxl` feature is on (decode only, so the
encoder's jixel dependency is not built); without it JPEG XL is still detected and
opening one is an Unsupported error that says the build lacks it. Orientation is
applied by the decoder, so the Exif and XMP orientation become 1. ICC, Exif and XMP
come through; associated alpha is divided out; float files open as F32.
* image-codecs: skip JPEG XL in the one-pixel round trip
The encoder refuses images narrower than two pixels; with the jxl feature JPEG XL is a
read-and-write format, so the generic round trip reached it.
jxl-oxide decompresses brob (Brotli Exif/XMP) boxes into a buffer its AllocTracker
does not count, so a 54-byte box expanded to 64 MiB of XMP and a larger one can ask for
gigabytes before any pixel limit applies. decode now inflates every brob box under one
8 MiB budget per file, rewrites it as the plain box it wraps and drops boxes past the
budget, nested or broken ones. Files without a brob box are not copied.
For an animation frame that starts with an `ALPH` chunk, image-webp 0.2.4 decodes
the next chunk as VP8 whatever its FourCC, skips the size check it makes for frames
without alpha, and indexes the frame's buffers by the picture's own size. A picture
larger than its `ANMF` header, or a non-key frame (decoded as 0x0), panicked with
out-of-bounds indexing in `read_frame` and `fill_rgba`. image-webp also steps from one
frame to the next without the padding byte, so after an odd-sized `ANMF` chunk it
reads the next frame one byte early.
Before decoding, the container is walked for the first frame with alpha whose picture
is not a VP8 key frame of its header's size, and for the first odd-sized `ANMF` chunk.
That frame is refused like a damaged frame: `decode` (which reads frame 0) errors only
when it is the first, and `decode_webp_frames` keeps the frames before it with a
`Truncated` warning.
Found by fuzzing `decode_webp`; ported from storytold/photocraft#10473, which was
written before PhotoCraft's codecs moved here. The regression test panics without the
fix.
Co-authored-by: DJHoltkamp <4955429+DJHoltkamp@users.noreply.github.com>
take_out searched the whole rest of the document for the next picture and the next inline
object on every item, so a document with many of one kind and none (or few) of the other cost
items × size. It now keeps where the next of each is and searches for a kind again only once
it has been passed. The output is the same.
* image-codecs: PNG bounds its image on the IDAT data and reserves only what it can make (lightcraft#3912)
The plausibility check measured the stored samples against the whole file
length, so ancillary chunks lifted the bound, and the decoder then reserved
the whole expanded output up front. A 1-bit 20000x20000 image is 50 MB
stored but 400 MB to 1.6 GB once expanded to 8-bit grey, RGB or RGBA, so
about 60 KB of IDAT reserved gigabytes.
- Bound the stored samples on the bytes of the first IDAT run (1032:1 plus
1 MiB), not the file length.
- Reserve up front only what the IDAT data can inflate to; past that, grow
the buffer as rows decode, at most doubling and never past the full
output. A forged file fails on its first row; a real solid or sparse
1-bit image that is past the ratio once expanded still decodes.
- Interlaced images decode into the whole zero-filled output in one go,
so their expanded size is checked against the IDAT data.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Fnx9MwPUug72xHRgg993uc
* Fix Adam7 palette expansion bounds without allocating unvalidated output
---------
Co-authored-by: Claude Sonnet 5.5 <noreply@anthropic.com>
Co-authored-by: Dylan M. Taylor <dylan@dylanmtaylor.com>
Ported from storytold/lightcraft#672, whose decoder code now lives here.
Some CCD Coolpix bodies store two interlaced fields one after the other in
one tight 12-bit strip. For the measured bodies, in their measured legacy
layout (binary maker-note version 0.2.0.0, compression 1, one strip of
w*h*1.5 bytes), the fields are interleaved in the model's field order. The
E5700's CYGM mosaic becomes approximate linear RGB with fixed gains.
Masked columns are judged on the stored rows, so the header probe and the
full decode agree. Other files keep their stored rows.
Checked on 527 NEF/NRW samples: only the six E5400/E5700/E8400/E8800 files
(and four synthetic legacy-layout fixtures) change; the other 517 decode
bit-identically.
Claude-Session: https://claude.ai/code/session_01Fnx9MwPUug72xHRgg993uc
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Reads the camera's white-balance presets as a Kelvin curve where the
decoders gave none:
- CR2 ColorData versions 1 to 12 (5-word records), version 10 at either
of its two layouts, picked by the Daylight record's 5200 K;
- CR3 ColorData versions 16-19, 32-34, 48 and 64-66;
- PEF maker note 0x0221 (entries of 53190 - K and the red/blue
multipliers);
- RW2/RWL/RAW tag 0x0027: the D55 and 3200 K studio tungsten
multipliers as a neutral pair with their chromaticities
(KelvinPresets::from_neutral_lights / neutral_lights).
Ported from storytold/lightcraft#2527.
Claude-Session: https://claude.ai/code/session_01Fnx9MwPUug72xHRgg993uc
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
A DNG tile's frame may hold at most twice the tile's samples and a Sony
quad tile's at most the tile's samples, with no 65,536-sample floor. The
floor let each of many tiny tiles decode and keep a frame far larger than
the tile budget counts (storytold/lightcraft#4024). A DNG tile also drops
the surplus capacity of a larger frame after the resize.
Claude-Session: https://claude.ai/code/session_01Fnx9MwPUug72xHRgg993uc
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>