[subset] Check the VARC axis-index allocation size before narrowing (#6477)

The saturating size helper returns size_t, so assigning its product to
unsigned can truncate it before the UINT_MAX check. Check multiplication
at unsigned width before resizing the buffer used by compile_unsafe().

Validated the reported high-count case under ASan/UBSan: it fails cleanly
after this change. A small control produces byte-identical output. The
ASan/UBSan Meson suite passes; check-symbols also passes after generating
its symbol manifests.
The large input remains a manual regression because it requires several
GB of memory.

Discovered by Anthropic's OSS Scanner, as vulnerability ANT-2026-V0UYCX4C.

Assisted-by: OpenAI Codex
This commit is contained in:
Behdad Esfahbod authored and GitHub committed 2026-10-29 20:55:45 -06:00
1 parent d0b38a793b
commit fdd29aa9a9
1 file changed
+4 -2
+4 -2
View File
@@ -178,8 +178,10 @@ struct varc_subset_plan_t
}
auto &data = axis_indices_data[i];
/* One control byte and at most four data bytes per value. */
unsigned size = hb_unsigned_mul_saturate (indices.length, 5u);
if (unlikely (size == UINT_MAX || !data.resize_dirty (size))) return false;
unsigned size;
if (unlikely (hb_unsigned_mul_overflows (indices.length, 5u, &size) ||
!data.resize_dirty (size)))
return false;
data.shrink (TupleValues::compile_unsafe (indices.as_array (), data.arrayZ));
bytes = data.as_array ();
}