* [subset] Return size_t rather than unsigned from get_size() methods
It's possible some get_size() computations during serialization
may overflow 32-bit arithmetic.
(The result of get_size() is used as size_t when allocating via
hb_serialize_context_t::extend(), but if the return value was
declared as unsigned int, the value may have wrapped resulting
in under-allocation.)
* [subset] Use size_t when accumulating subset_data_size
* [graph] Use size_t (rather than signed int64_t) for vertex_len variables
* Use size_t for sanitize_closure_t::stride to avoid narrowing
* Fix: Algorithmic Complexity DoS in HarfBuzz Repacker
Introduce total iteration limits in overflow resolution loop and physical growth limits (vertices and spaces) to prevent Hang/OOM during subsetting of malicious fonts.
* Repacker: Move safety limits to hb-limits.hh as requested by Behdad
---------
Co-authored-by: default <default@zimbra.0day.lab>
Prior to this change the object id for a vertex in the repacker graph is it's position in the topological ordering. As a result after each sorting the object ids change. This decouples the topological sorting from the object id assignments. A separate ordering list is retained. This simplifies handling around object ids which no longer needs to account for them shifting, and improves performance by eliminating the step of reassigned link object ids after each sort.
Move child is O(n) where n = number of links on the parent object. The batch version is also O(n). Use this in ligature splitting clone range to avoid O(n^2) behaviour.
When attempting to fix overflows nodes that have incoming edges from more than one parent can be duplicated. However, if all of these incoming links are virtual except for one duplicating it will effectively orphan it (one of the copies will be pointed to only by virtual links). Since parent book keeping is expensive we don't want to separately track virtual/real parents so instead just don't duplicate nodes that have at least one incoming virtual link.
Otherwise nags about things like this,
In member function ‘int64_t graph::graph_t::vertex_t::modified_distance(unsigned int) const’,
inlined from ‘void graph::graph_t::sort_shortest_distance()’ at ../src/graph/graph.hh:626:24:
../src/graph/graph.hh:371:20: warning: dangling pointer to an unnamed temporary may be used [-Wdangling-pointer=]
371 | hb_clamp (distance + distance_modifier (), (uint64_t) 0, (uint64_t) 0x7FFFFFFFFFF);
| ~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
And some of the CI bots fail also like this https://github.com/harfbuzz/harfbuzz/actions/runs/9838686960/job/27159310858?pr=4793
But it probably something else also can be improved which maybe is out of scope for this particular change.
The old approach considered only one class at a time, which in some cases can generate the wrong answer. This change updates the estimation to consider how all classes in the current split would end up encoded in a single ClassDef table. Additionally compute whether glyphs are consecutive only for the current split (instead of the fully mapping).
Prior to this change overflows to shared nodes were handled by duplicating the link from only a single parent on each iteration. However, I've encountered fonts where there is a large number of parents sharing a single child. Using the prior strategy requires large number of overflow resolution iterations to resolve overflows. This changes shared overflow resolution to duplicate the shared child and re-assign multiple overflowing parents in a single iteration. This reduces total packing size in these cases and allows resolution to complete in far fewer iterations.
when instantiating varstore, we need to pop a tuple like
(combined_gain, i, j), if combined gain is the same then we compare the
value of i, and then j. So we'd like to use custom type as the key when
popping from the queue. This would match fonttool's algorithm cause it
uses heappop().