PaintColrLayers::closurev1() recorded firstLayerIndex ..
firstLayerIndex + numLayers - 1 as retained layers without checking
the range against LayerList.len (PaintColrLayers::sanitize is a plain
check_struct), and _colr_closure() then creates one colrv1_layers map
entry per retained index. A crafted font whose leaf paints point past
the end of the LayerList could therefore add up to 255 entries each for
layers that do not exist, driving subset-plan memory to roughly 1200x
the font size (585 MiB for a 512 KB font).
Clamp the range to the LayerList at the one place the retained-layer set
is built, before add_layer_indices() and for the dispatch loop alike;
this also removes the firstLayerIndex + numLayers wrap-around in the loop
bound. Readers (paint_glyph, dependv1) never record indices and already
tolerate out-of-range ones via LayerList::get_paint() returning Null, so
they are left alone. A PaintColrLayers whose range lies past the source
LayerList now has no mapping and subset writes 0xFFFFFFFF; both that and
a renumbered index past the new LayerList are out of range and draw
nothing.
Adds a regression test that builds such a font inline and pins the
output; it fails without the fix.
Discovered by Anthropic's OSS Scanner, as vulnerability ANT-2026-ROQFK0RA.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>