Commit Graph
15530 Commits
Author SHA1 Message Date
Skef Iterum 9101d242b3 Update depend API docs and add explicit test mode to fuzzer
Documentation updates:
- Add comprehensive context_set documentation to depend-api.md with
  encoding details and example code for check_context_satisfied()
- Update depend-for-closure.md: remove obsolete over-approximation
  analysis, reflect that recent testing shows no over-approximation
  with proper context_set filtering
- Update depend-implementation.md: correct descriptions of active
  glyph filtering, sequential accumulation, ligature sets, and
  context recording to match current implementation
- Fix terminology: change 'binning' to 'segmentation' throughout
- Fix hb-depend-data.hh:557: use HB_CODEPOINT_INVALID instead of 0
  for layout_tag default value

Fuzzer enhancements (hb-depend-closure-parity):
- Add explicit test mode with command-line options similar to
  hb-subset conventions:
    --unicodes=LIST       Unicode codepoints (U+XXXX, decimal, ranges)
    --gids=LIST           Glyph IDs (decimal, ranges)
    --no-layout-closure   Skip GSUB closure
    --layout-features=LIST GSUB features (comma-separated or *)
- Support multiple input formats:
    --unicodes=U+0041,U+0042,U+0043
    --unicodes=65-90
    --gids=10-50 --no-layout-closure
    --layout-features=liga,calt or --layout-features='*'
- Add robust input parsing with validation to prevent infinite loops
  on invalid input (e.g., --gids=abc or --unicodes=U+GGGG)
- Add error checking for conflicting options

This allows reproducing random test cases manually and testing
specific scenarios without finding the right seed.

Tested: All modes produce identical results to seed-based tests.
2026-02-05 19:11:40 -08:00
Skef Iterum e214465565 Add depend API: graph extraction with contextual filtering
This commit implements the experimental depend API for extracting glyph
dependency graphs from OpenType fonts. The depend API provides the foundation
for computing glyph closures and analyzing font structure.

The implementation significantly improves upon the initial draft (commit
3f59433d5) by adding contextual filtering to reduce over-approximation while
still capturing complete dependency graphs.

Core Architecture
-----------------

The depend implementation operates in graph extraction mode: it walks all font
tables once to build a complete dependency graph. The key innovation is
contextual filtering that reduces over-approximation compared to the naive
approach.

Contextual Filtering for Reduced Over-Approximation
----------------------------------------------------

The initial implementation (3f59433d5) recorded dependencies for ALL glyphs in
every lookup's coverage, leading to significant over-approximation. The current
implementation filters dependencies based on context:

**Individual Lookups**: Each lookup's depend() method filters by active glyphs:

  // Old approach (3f59433d5)
  + hb_iter (this+coverage)  // ALL coverage glyphs
  | hb_apply (record_dependency)

  // New approach
  hb_set_t intersection;
  (this+coverage).intersect_set (c->parent_active_glyphs(), intersection);
  + hb_iter (intersection)  // ONLY contextually relevant glyphs
  | hb_apply (record_dependency)

**Contextual Lookups**: ContextFormat1/2/3 and ChainContextFormat1/2/3 set up
active glyph contexts for nested lookups:

  hb_set_t* cur_active_glyphs = c->push_cur_active_glyphs();
  get_coverage().intersect_set(*c->glyphs, *cur_active_glyphs);

  // Process rules with filtered coverage
  + hb_zip (this+coverage, ruleSet)
  | hb_filter ([&] (hb_codepoint_t g) {
      return c->parent_active_glyphs().has(g);
    }, hb_first)
  | hb_apply (process_rule)

This significantly reduces over-approximation for simple cases while maintaining
complete graph coverage.

**Intentional Over-Approximation at Nested Boundaries**: When contextual rules
invoke nested lookups, we pass ALL glyphs to ensure no edges are missed:

  // In context_depend_recurse_lookups()
  cur_active_glyphs->set (*c->glyphs);  // Universe for nested lookups

This design choice prioritizes completeness over precision at contextual
boundaries, which is appropriate for graph extraction.

Edge Deduplication and Performance
-----------------------------------

Dependency extraction can generate millions of potential edges. To maintain
acceptable performance:

- Edge hashing (compute_edge_hash): O(1) duplicate detection via hash table
  keyed on (source, table_tag, layout_tag, dependent, ligature_set_contents)
- Hash collision handling: Falls back to O(n) linear search through glyph's
  existing dependencies only on collision (rare)
- Component vectors: Stores edge components for collision detection
- Post-extraction cleanup: Frees temporary structures (edge_hashes,
  lookup_features) to reduce memory footprint

Table-Specific Extraction
--------------------------

- **cmap**: Unicode → glyph mappings including UVS (Unicode Variation Sequences)
- **GSUB**: All substitution rules with feature tags and ligature set tracking
  - Handles all 8 GSUB lookup types including Extension (type 7)
  - Records which features reference each dependency
  - Groups ligature alternatives via ligature_set indices
- **glyf**: TrueType composite glyph components
- **CFF**: CFF1 SEAC (Standard Encoding Accented Character) composites
- **COLR**: Color layer dependencies for both COLRv0 and COLRv1
  - Filters self-referential PaintGlyph to prevent trivial cycles
- **MATH**: Mathematical variant glyphs and assembly components

CFF SEAC dependencies use table_tag='CFF ' with layout_tag=0 and
ligature_set=HB_CODEPOINT_INVALID, matching the pattern used for glyf.

Data Structure: hb_depend_data_record_t
----------------------------------------

Each dependency edge contains:
- table_tag: Source table (GSUB, glyf, CFF, COLR, MATH, cmap)
- dependent: Target glyph ID
- layout_tag: Feature tag (for GSUB) or UVS codepoint (for cmap), else 0
- ligature_set: Index into sets array for ligature alternatives, else INVALID
- context_set: Reserved for future contextual filtering (currently unused)

Graph stored as vector indexed by glyph ID, each containing vector of records.

Testing and Documentation
--------------------------

- test/api/test-ot-depend.c: Comprehensive test suite (645 lines)
  - Tests all table types and GSUB substitution formats
  - Includes regression tests for edge cases
  - Uses ContextFormat2-Depend-Test.ttf (minimized from 486KB to 85KB)

- test/fuzzing/hb-depend-fuzzer.cc: Production-quality closure implementation
  - Demonstrates correct ligature set handling
  - Includes feature filtering and over-approximation analysis
  - Reference implementation for depend-based closure computation

- docs/depend-api.md: Public API documentation with usage examples
- docs/depend-for-closure.md: Guide to using depend for closure computation
- docs/depend-implementation.md: Implementation details and architecture

Build System
-------------

- Meson option: -Ddepend_api=true
- Compile flag: -DHB_DEPEND_API
- Symbols in libharfbuzz-subset.dylib (applications must link both
  libharfbuzz and libharfbuzz-subset)

Status: Experimental API, subject to change.
2026-02-05 19:11:40 -08:00
Skef Iterum 0b16d2f98a Add comprehensive documentation for depend API
This commit adds complete documentation for the dependency analysis API
at all levels: public API, internal structures, and implementation.

Public API documentation (src/hb-depend.cc, src/hb-depend.h):
- Replaced "XXX Todo" placeholder in SECTION comment with detailed
  module description explaining dependency graphs and use cases
- Fixed misleading hb_depend_t description (was only mentioning UVS)
- Added full documentation for hb_depend_print() with debugging notes
- Added usage examples to all three public functions
- Updated all version tags from "?.0.0" to "REPLACEME"
- Added experimental API warnings throughout

Internal documentation (src/hb-depend-data.hh, src/hb-depend.hh):
- Documented 5 internal structures with purpose and ownership semantics
- Added comments explaining data relationships and lifecycle

Algorithm documentation (src/hb-depend.cc):
- Added detailed algorithm comments to 5 extraction methods explaining
  how dependencies are collected from each OpenType table (cmap, GSUB,
  glyf, COLR, MATH)

User guide (docs/depend-api.md):
- Created comprehensive 250-line API guide with:
  * Introduction with mechanism overview
  * Basic usage example (iterating dependencies)
  * Advanced examples (computing reachable glyphs, ligature sets)
  * API details and compilation instructions
  * Implementation notes including ownership semantics
  * Use cases (subsetting, coverage analysis, binning, testing)
- Prominent experimental API warnings

Build configuration (meson_options.txt):
- Updated option description to emphasize experimental status

All documentation follows HarfBuzz conventions and includes gtk-doc
formatting for API reference generation.
2026-02-05 19:11:40 -08:00
Skef Iterum a80a42725f Fix depend code for wrapped format structs
After rebasing onto main, the depend code needs to use format.v
instead of format directly, due to main's "Wrap formats in struct"
change (346b8dcf9).

Updated:
- hb-ot-cmap-table.hh: _.u.format == 14 → _.u.format.v == 14
- COLR.hh: switch (u.format) → switch (u.format.v)
2026-02-05 19:11:40 -08:00
Skef Iterum 3b38a53db0 Finishing up draft 2026-02-05 19:11:40 -08:00
Skef Iterum 4eeb54bafa Add colrv1-depend
first try making support conditional on HB_DEPEND_API
2026-02-05 19:11:40 -08:00
Skef Iterum 05d9221ee4 extraction API 2026-02-05 19:11:40 -08:00
Skef Iterum d3c4eb9031 Math, COLR, and glyf dependencies 2026-02-05 19:11:40 -08:00
Skef Iterum 3a439be382 Initial playing around with dependencies 2026-02-05 19:11:40 -08:00
Behdad Esfahbod c1e94eb08e [gvar] Remove unused variable assignment 2026-02-05 15:33:27 -07:00
Behdad Esfahbod b863a8f462 Revert "[rust] Use HarfBuzz-style glyph outline"
This reverts commit 6ba28949f5.
2026-02-05 15:32:46 -07:00
Behdad Esfahbod 6ba28949f5 [rust] Use HarfBuzz-style glyph outline
Slightly slower, but matches our point order better.
2026-02-05 15:24:53 -07:00
Behdad Esfahbod ccaef949c4 [gvar] Speed up by using a fused delta application codepath (#5730)
* [gvar] Speed up by using a fused delta application codepath

6% speedup in hanzi VARC font rendering.

* [gvar] Improve fused delta application

Another 1%.
2026-02-05 14:20:19 -07:00
Behdad Esfahbod c42f4b5218 [VARC] Implement lazy skipping 2026-02-05 12:44:49 -07:00
Behdad Esfahbod 0334f2e38d [rust] Add a couple of compiler flags (#5729)
Makes binary 100kb smaller!
2026-02-04 05:07:31 -07:00
Behdad Esfahbod b1c71874a4 [rust] Use panic="abort" in debugoptimized builds as well 2026-02-03 23:43:00 -07:00
Behdad Esfahbod 8c8f76d690 [scalar-cache] Enlarge, and adjust scratch logic (#5728)
No need to clear on default construction, or set length even.
2026-02-03 23:40:55 -07:00
Khaled Hosny b42511e071 12.3.2 2026-01-24 14:09:52 +02:00
Qunxin Liu e601560b77 fix fuzzer issue: https://oss-fuzz.com/testcase-detail/6005602106277888
Do not use pointers that may point outside of the allocated memory region
2026-01-23 11:44:39 -07:00
Behdad Esfahbod e365cd0e2e [cmap] Another null check
https://github.com/harfbuzz/harfbuzz/security/advisories/GHSA-pgp8-9vq3-fq2r#advisory-comment-154898
2026-01-22 19:01:39 -07:00
Behdad Esfahbod dc73849f56 [gvar] Fix padding size calcs (#5718)
The previous state of is_long_offset() is irrelevant here.

Fixes https://oss-fuzz.com/testcase-detail/5920156651159552
2026-01-22 18:08:51 -07:00
Qunxin Liu b17fa8ef39 Fix fuzzer found heap-use-after-free crash (#5717)
Fixes https://oss-fuzz.com/testcase-detail/6005602106277888
2026-01-21 14:42:29 -07:00
Qunxin Liu 6e37798a16 Merge pull request #5716 from qxliu76/gvar_fix
[subset] fix padding in gvar table
2026-01-20 15:04:11 -08:00
Qunxin Liu 1c0a3a14d3 [subset] fix padding in gvar table
Add necessary padding if original font is using long format and
subsetted font is using short format
2026-01-20 14:47:54 -08:00
Khaled Hosny abcb75a5d1 12.3.1 2026-01-20 23:59:40 +02:00
Behdad Esfahbod 4ca351535b [rust] Roll to new HarfRust 2026-01-10 07:41:54 -07:00
Bastien Dejean 8f5abdc42f GCC 4.9.4: Keep template names unique (#5713)
Fixes #5712.
2026-01-10 05:02:17 -07:00
Behdad Esfahbod 0f58c68c6e Remove use of std::is_trivially_copyable (#5711)
Fixes https://github.com/harfbuzz/harfbuzz/issues/5709
2026-01-09 04:54:49 -07:00
Behdad Esfahbod 1265ff8d99 [cmap] malloc fail test (#5710)
Fixes https://github.com/harfbuzz/harfbuzz/security/advisories/GHSA-xvjr-f2r9-c7ww
2026-01-09 04:54:42 -07:00
Qunxin Liu fa2908bf16 [subset] avar mapping: fix return value for avar1 fonts (#5705) 2026-01-07 12:49:37 -07:00
Behdad Esfahbod e8dd3cd376 Add a couple inline's
Try to fix https://github.com/harfbuzz/harfbuzz/issues/5704
2026-01-07 12:30:30 -07:00
Dominik Röttsches e1f2565db0 Build fix for avar table with HB_NO_AVAR2 2026-01-07 11:45:16 +02:00
Behdad Esfahbod 5163127641 [matcher] Micro-optimize
Some 2% in Duployan & Roboto.
2026-01-05 17:47:17 -07:00
Behdad Esfahbod 120a38a55e [GDEF] Inline a function (#5701)
17% speedup in Duployan benchmark!
2026-01-05 15:52:54 -07:00
Behdad Esfahbod 18e46b1ae5 [GDEF] Speed up mark set filtering
Remove one bounds check.
2026-01-03 17:08:46 -07:00
Khaled HosnyandBehdad Esfahbod c7ac9e4345 [kbts] Update to version 2.07 (#5697)
* [kbts] Update to version 2.07

* [kbts] More error checking

* [kbts] Update to the latest commit

* [kbts] Fix destroy order

Apparently it has to be done this way. Otherwise, freeing
the config frees the scratchpad as well, or something to that
extent, and makes valgrind very unhappy.

* [kbts] Hook objects to shape-plan for reuse

---------

Co-authored-by: Behdad Esfahbod <behdad@behdad.org>
2026-01-01 21:01:30 -07:00
Behdad Esfahbod 29a47dcd83 Enlarge MappingCache to 256 entries again
Matches HR.
2025-12-31 16:35:58 -07:00
Behdad Esfahbod d1f58e5762 Tweak an unlikely
This doesn't seem to slowdown benchmarks much, but speeds up Duployan
by ~8%.
2025-12-31 16:28:56 -07:00
Behdad Esfahbod 1c3633e121 [GDEF] Use set-digest, not bit-set, for mark coverages again
This seems to be slightly faster, more memory efficient, and
matches HarfRust.
2025-12-31 15:56:43 -07:00
Behdad Esfahbod d58bc8c94d Remove a couple inlines
Speeds up things a bit.
2025-12-31 15:45:58 -07:00
Behdad Esfahbod a789943958 [binary-cache] Enlarge to 256 entries
2% speedup in NotoNastaliqUrdu benchmark.
2025-12-30 12:15:12 -07:00
Behdad Esfahbod a4757aefc2 Remove unused default value
In the HarfRust port, compiler warns about unused value. lol
2025-12-27 17:22:46 -07:00
Behdad Esfahbod aaf22616ba [contextual] Fix typo
This involves a very edge case, where there are no rules in the ruleset.
One copy of the code was correct, other not. Fix.
2025-12-27 17:20:25 -07:00
Khaled Hosny 5a31dd02f0 12.3.0 2025-12-27 21:58:17 +02:00
Behdad Esfahbod 635cd0e854 [calculate-scalar] Another tiny speedup
~1% in varc-hanzi benchmark.
2025-12-24 15:15:06 -07:00
Behdad Esfahbod 17092a67cc [calculate-scalar] A bit more speedup
5% speedup in varc-hanzi drawing.
No slowdown on GoogleSansFlex.
2025-12-24 14:17:38 -07:00
Behdad Esfahbod 21d15507fc [calculate-scalar] Another path we can use the cache
If the scalar=1.0 even if we have interims, we can
still use the cache.
2025-12-24 12:13:54 -07:00
Behdad Esfahbod 1a94172282 [calculate-scalar] Use cache sometimes even if has_interm
More involved logic.
2025-12-24 01:41:40 -07:00
Behdad Esfahbod 0f6fecfaf4 [calculate_scalar] Only jumpskip if num-coords >= 16
Otherwise, it was slowing down GoogleSansFlex rendering.
2025-12-24 01:41:36 -07:00
Behdad Esfahbod 22c876331a [calculate-scalar] Jump-skip 4 zeros at a time 2025-12-23 17:49:37 -07:00