In a well-formed gvar the per-glyph data ranges [offset[i],
offset[i+1]) are disjoint, so the retained glyphs' data add up to at
most the table. gvar::subset()'s avar2 culling pass used the offsets
as they are, so malformed offsets (0, D, 0, D, ...) could point
thousands of glyphs at one 128 KB block; the pass parsed, rewrote and
kept a heap copy of it once per glyph, repeated on each of the
serializer's buffer-doubling retries (26 s and 1.3 GiB from a 251 KB
font before it was rejected).
Restore the invariant where the pass is driven: charge each retained
glyph's byte length to a budget equal to the source blob length with
hb_budget_spend(), and on exhaustion raise HB_SERIALIZE_ERROR_OTHER
so _hb_subset_table_try() does not retry. A spec-conforming table
can never reach the limit, so there is no tuning constant and valid
fonts subset byte-identically; a font whose retained glyphs' gvar
ranges overlap by more than the table length now fails avar2 partial
instancing.
Adds test_avar2_culling_aliased_gvar_data to test/api/test-subset-avar2.c.
The non-avar2 instantiate() branch trusts the same offsets and is
not touched here.
Discovered by Anthropic's OSS Scanner, as vulnerability ANT-2026-NJZQ4XFP.
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>