Commit Graph
513 Commits
Author SHA1 Message Date
Skef IterumandClaude 7ad4fa1e6b Address code review feedback for depend API
This commit addresses multiple code review comments:

1. Switch RNG to std::mt19937 for deterministic tests
   - Replace std::rand/srand with std::mt19937 in parity checker
   - Use std::uniform_int_distribution for all random number generation
   - Ensures test determinism across platforms for non-error-injection tests

2. Convert debug fprintf to DEBUG_MSG_LEVEL
   - Convert GSUB lookup processing prints to DEBUG_MSG_LEVEL(DEPEND, nullptr, 1, 0, ...)
   - Visible at HB_DEBUG_DEPEND=2 or higher for detailed per-lookup debugging

3. Rename function for consistency
   - hb_depend_from_face → hb_depend_from_face_or_fail
   - Follows HarfBuzz convention for functions returning nullptr on failure
   - Update all call sites, documentation, and code comments

4. Adopt established set deduplication pattern
   - Change from hb_vector_t<hb_set_t> to hb_vector_t<hb::unique_ptr<hb_set_t>>
   - Use pointer-based hashmap: hb_hashmap_t<const hb_set_t*, hb_codepoint_t>
   - Leverages hb_hash's content-based deduplication via pointer dereferencing
   - Follows pattern established elsewhere in HarfBuzz codebase

5. Separate ligature and context set semantics
   - Rename free_set → free_ligature_set (no hashmap manipulation)
   - Rename new_set → new_ligature_set (no hashmap manipulation)
   - Rename find_or_create_set → find_or_create_context_set (only function using hashmap)
   - Makes it explicit which sets are deduplicated (context) vs unique (ligature)
   - Update all call sites in Ligature.hh, ReverseChainSingleSubstFormat1.hh, hb-ot-layout-gsubgpos.hh

6. Code cleanup
   - Remove unused add_to_ligature_set function (dead code)
   - Refactor new_ligature_set(cp) to delegate to new_ligature_set(set)
   - Move set_to_index declaration near sets/free_set_list for better organization

All tests pass.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2026-02-05 20:58:13 -08:00
Skef IterumandClaude Opus 4.5 c32a6ca5bd Fix under-approximation in context set filtering and update edge flag docs
Two related fixes:

1. Context set under-approximation fix (hb-ot-layout-gsubgpos.hh):
   When a direct requirement (from input positions) overlapped with a
   disjunctive set (backtrack/lookahead), the code incorrectly added
   the filtered (remaining) set as an additional requirement. Fix:
   if position_context intersects with original_set at all, the
   disjunctive requirement is already satisfied - don't add anything.

2. Parity checker ligature fix (hb-depend-closure-parity.cc):
   When processing deferred ligatures, store {dependent, ligature_set,
   flags} at deferral time instead of re-scanning edges later. This
   preserves the feature filtering that was already done and correctly
   propagates edge flags.

Also updates edge flag documentation to correctly explain the mechanism:
- Depend extraction records edges based on static input coverage/class
- During closure, lookups within a rule are applied sequentially
- Lookups at earlier positions may transform later positions
- Multiple lookups at same position may interact (one produces a glyph,
  another immediately consumes it as an "intermediate")
- So a glyph matching coverage might not persist when closure traverses

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-02-05 19:11:40 -08:00
Skef IterumandClaude b8060c3398 Add edge flags to detect over-approximation from contextual positions
Edges created from non-zero positions in multi-position contextual rules
can consume intermediate glyphs (glyphs produced then immediately consumed,
never settling in closure). This causes depend's closure to include extra
glyphs compared to subset's closure, which is expected behavior.

Added HB_DEPEND_EDGE_FLAG_FROM_CONTEXT_POSITION flag to mark these edges
at extraction time. Updated parity checker to track when such edges are
hit during closure computation and distinguish between:
- Expected over-approximation (hit flagged edge): informational only
- Unexpected over-approximation (no flagged edge): critical error

This enables detection of genuine bugs (unexpected over-approximation)
while allowing expected cases from intermediate glyphs.

Updated all API callers and documentation to include nullable flags
parameter. Renamed parity checker terminology from "fuzzer" to "parity"
for clarity.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2026-02-05 19:11:40 -08:00
Skef Iterum 9101d242b3 Update depend API docs and add explicit test mode to fuzzer
Documentation updates:
- Add comprehensive context_set documentation to depend-api.md with
  encoding details and example code for check_context_satisfied()
- Update depend-for-closure.md: remove obsolete over-approximation
  analysis, reflect that recent testing shows no over-approximation
  with proper context_set filtering
- Update depend-implementation.md: correct descriptions of active
  glyph filtering, sequential accumulation, ligature sets, and
  context recording to match current implementation
- Fix terminology: change 'binning' to 'segmentation' throughout
- Fix hb-depend-data.hh:557: use HB_CODEPOINT_INVALID instead of 0
  for layout_tag default value

Fuzzer enhancements (hb-depend-closure-parity):
- Add explicit test mode with command-line options similar to
  hb-subset conventions:
    --unicodes=LIST       Unicode codepoints (U+XXXX, decimal, ranges)
    --gids=LIST           Glyph IDs (decimal, ranges)
    --no-layout-closure   Skip GSUB closure
    --layout-features=LIST GSUB features (comma-separated or *)
- Support multiple input formats:
    --unicodes=U+0041,U+0042,U+0043
    --unicodes=65-90
    --gids=10-50 --no-layout-closure
    --layout-features=liga,calt or --layout-features='*'
- Add robust input parsing with validation to prevent infinite loops
  on invalid input (e.g., --gids=abc or --unicodes=U+GGGG)
- Add error checking for conflicting options

This allows reproducing random test cases manually and testing
specific scenarios without finding the right seed.

Tested: All modes produce identical results to seed-based tests.
2026-02-05 19:11:40 -08:00
Skef Iterum e214465565 Add depend API: graph extraction with contextual filtering
This commit implements the experimental depend API for extracting glyph
dependency graphs from OpenType fonts. The depend API provides the foundation
for computing glyph closures and analyzing font structure.

The implementation significantly improves upon the initial draft (commit
3f59433d5) by adding contextual filtering to reduce over-approximation while
still capturing complete dependency graphs.

Core Architecture
-----------------

The depend implementation operates in graph extraction mode: it walks all font
tables once to build a complete dependency graph. The key innovation is
contextual filtering that reduces over-approximation compared to the naive
approach.

Contextual Filtering for Reduced Over-Approximation
----------------------------------------------------

The initial implementation (3f59433d5) recorded dependencies for ALL glyphs in
every lookup's coverage, leading to significant over-approximation. The current
implementation filters dependencies based on context:

**Individual Lookups**: Each lookup's depend() method filters by active glyphs:

  // Old approach (3f59433d5)
  + hb_iter (this+coverage)  // ALL coverage glyphs
  | hb_apply (record_dependency)

  // New approach
  hb_set_t intersection;
  (this+coverage).intersect_set (c->parent_active_glyphs(), intersection);
  + hb_iter (intersection)  // ONLY contextually relevant glyphs
  | hb_apply (record_dependency)

**Contextual Lookups**: ContextFormat1/2/3 and ChainContextFormat1/2/3 set up
active glyph contexts for nested lookups:

  hb_set_t* cur_active_glyphs = c->push_cur_active_glyphs();
  get_coverage().intersect_set(*c->glyphs, *cur_active_glyphs);

  // Process rules with filtered coverage
  + hb_zip (this+coverage, ruleSet)
  | hb_filter ([&] (hb_codepoint_t g) {
      return c->parent_active_glyphs().has(g);
    }, hb_first)
  | hb_apply (process_rule)

This significantly reduces over-approximation for simple cases while maintaining
complete graph coverage.

**Intentional Over-Approximation at Nested Boundaries**: When contextual rules
invoke nested lookups, we pass ALL glyphs to ensure no edges are missed:

  // In context_depend_recurse_lookups()
  cur_active_glyphs->set (*c->glyphs);  // Universe for nested lookups

This design choice prioritizes completeness over precision at contextual
boundaries, which is appropriate for graph extraction.

Edge Deduplication and Performance
-----------------------------------

Dependency extraction can generate millions of potential edges. To maintain
acceptable performance:

- Edge hashing (compute_edge_hash): O(1) duplicate detection via hash table
  keyed on (source, table_tag, layout_tag, dependent, ligature_set_contents)
- Hash collision handling: Falls back to O(n) linear search through glyph's
  existing dependencies only on collision (rare)
- Component vectors: Stores edge components for collision detection
- Post-extraction cleanup: Frees temporary structures (edge_hashes,
  lookup_features) to reduce memory footprint

Table-Specific Extraction
--------------------------

- **cmap**: Unicode → glyph mappings including UVS (Unicode Variation Sequences)
- **GSUB**: All substitution rules with feature tags and ligature set tracking
  - Handles all 8 GSUB lookup types including Extension (type 7)
  - Records which features reference each dependency
  - Groups ligature alternatives via ligature_set indices
- **glyf**: TrueType composite glyph components
- **CFF**: CFF1 SEAC (Standard Encoding Accented Character) composites
- **COLR**: Color layer dependencies for both COLRv0 and COLRv1
  - Filters self-referential PaintGlyph to prevent trivial cycles
- **MATH**: Mathematical variant glyphs and assembly components

CFF SEAC dependencies use table_tag='CFF ' with layout_tag=0 and
ligature_set=HB_CODEPOINT_INVALID, matching the pattern used for glyf.

Data Structure: hb_depend_data_record_t
----------------------------------------

Each dependency edge contains:
- table_tag: Source table (GSUB, glyf, CFF, COLR, MATH, cmap)
- dependent: Target glyph ID
- layout_tag: Feature tag (for GSUB) or UVS codepoint (for cmap), else 0
- ligature_set: Index into sets array for ligature alternatives, else INVALID
- context_set: Reserved for future contextual filtering (currently unused)

Graph stored as vector indexed by glyph ID, each containing vector of records.

Testing and Documentation
--------------------------

- test/api/test-ot-depend.c: Comprehensive test suite (645 lines)
  - Tests all table types and GSUB substitution formats
  - Includes regression tests for edge cases
  - Uses ContextFormat2-Depend-Test.ttf (minimized from 486KB to 85KB)

- test/fuzzing/hb-depend-fuzzer.cc: Production-quality closure implementation
  - Demonstrates correct ligature set handling
  - Includes feature filtering and over-approximation analysis
  - Reference implementation for depend-based closure computation

- docs/depend-api.md: Public API documentation with usage examples
- docs/depend-for-closure.md: Guide to using depend for closure computation
- docs/depend-implementation.md: Implementation details and architecture

Build System
-------------

- Meson option: -Ddepend_api=true
- Compile flag: -DHB_DEPEND_API
- Symbols in libharfbuzz-subset.dylib (applications must link both
  libharfbuzz and libharfbuzz-subset)

Status: Experimental API, subject to change.
2026-02-05 19:11:40 -08:00
Skef Iterum 0b16d2f98a Add comprehensive documentation for depend API
This commit adds complete documentation for the dependency analysis API
at all levels: public API, internal structures, and implementation.

Public API documentation (src/hb-depend.cc, src/hb-depend.h):
- Replaced "XXX Todo" placeholder in SECTION comment with detailed
  module description explaining dependency graphs and use cases
- Fixed misleading hb_depend_t description (was only mentioning UVS)
- Added full documentation for hb_depend_print() with debugging notes
- Added usage examples to all three public functions
- Updated all version tags from "?.0.0" to "REPLACEME"
- Added experimental API warnings throughout

Internal documentation (src/hb-depend-data.hh, src/hb-depend.hh):
- Documented 5 internal structures with purpose and ownership semantics
- Added comments explaining data relationships and lifecycle

Algorithm documentation (src/hb-depend.cc):
- Added detailed algorithm comments to 5 extraction methods explaining
  how dependencies are collected from each OpenType table (cmap, GSUB,
  glyf, COLR, MATH)

User guide (docs/depend-api.md):
- Created comprehensive 250-line API guide with:
  * Introduction with mechanism overview
  * Basic usage example (iterating dependencies)
  * Advanced examples (computing reachable glyphs, ligature sets)
  * API details and compilation instructions
  * Implementation notes including ownership semantics
  * Use cases (subsetting, coverage analysis, binning, testing)
- Prominent experimental API warnings

Build configuration (meson_options.txt):
- Updated option description to emphasize experimental status

All documentation follows HarfBuzz conventions and includes gtk-doc
formatting for API reference generation.
2026-02-05 19:11:40 -08:00
Khaled Hosny a790c38b78 [docs] Fix build 2025-10-01 08:26:58 +03:00
Khaled Hosny cb59b347ca 12.1.0 2025-10-01 08:24:34 +03:00
Behdad Esfahbod 4a26e289f7 Add hb_ot_layout_lookup_collect_glyph_alternates() (#5367)
* [map] Massage operator << overloads

* [ot-layout] Add +hb_ot_layout_lookup_collect_glyph_alternates

To collect all glyph mapping from SingleSubst or AlternateSubst
lookups in one call.  Needed by FreeType autohinter for performance.

New API:
+hb_ot_layout_lookup_collect_glyph_alternates()

* [layout] Change hb_ot_layout_lookup_collect_glyph_alternates() API

https://github.com/harfbuzz/harfbuzz/pull/5367#discussion_r2149019638
2025-09-30 13:31:04 -06:00
Count_MHM 025f72b240 Make example variable names consistent 2025-08-03 18:30:54 -06:00
Khaled Hosny 30e7af11ee 11.3.0 2025-07-20 21:40:37 +03:00
Behdad Esfahbod 1835c088db [font] Add get_glyph_[hv]_origins callbacks (plural)
Unused so far, and unimplemented in backends.
2025-07-20 07:19:23 -06:00
Corentin Noël 37627c6756 [docs] Specify the online location of the documentation
Move the releaseinfo under the bookinfo as it is what gtk-doc expects.

This specifically allows the online location to also be exported into the devhelp2
file.
2025-06-13 11:27:56 -06:00
Khaled Hosny 63cdd745ff 11.2.0 2025-04-28 15:56:32 +03:00
Behdad Esfahbod 38db1facf2 [paint] Change back return type of paint_glyph() to void
Add new "_or_fail" variants of draw_glyph and paint_glyph

New API:
+hb_font_draw_glyph_or_fail_func_t
+hb_font_paint_glyph_or_fail_func_t
+hb_font_funcs_set_draw_glyph_or_fail_func()
+hb_font_funcs_set_paint_glyph_or_fail_func()
+hb_font_draw_glyph_or_fail()
+hb_font_paint_glyph_or_fail()

Deprecated API:
-hb_font_draw_glyph_func_t
-hb_font_paint_glyph_func_t
-hb_font_funcs_set_draw_glyph_func()
-hb_font_funcs_set_paint_glyph_func()
2025-04-23 18:23:51 -06:00
Behdad Esfahbod 853720ead4 [font] Add hb_font_is_synthetic()
New API:
+hb_font_is_synthetic()
2025-04-22 22:51:40 -06:00
Garret Rieger a1e587b75a [subset] Add hb_subset_cff_get_charstrings_index and hb_subset_cff2_get_charstrings_index.
These methods allow retrieving the entire charstrings index structure from a CFF or CFF2 table.
2025-03-31 16:28:28 -06:00
Garret Rieger ae8b288db7 [subset] Correctly handle lifetime of charstrings data blob returned by hb_subset_cff_get_charstring_data. 2025-03-27 22:07:06 +00:00
Garret Rieger 0d4053b11d [subset] Add hb-subset.h API for retrieving the raw per glyph outline data from CFF and CFF2. 2025-03-27 20:20:52 +00:00
Khaled Hosny ea6a172f84 11.0.0 2025-03-24 00:33:53 +02:00
Behdad Esfahbod c274ee7b3e [directwrite] Add hb_font_t setter/getter from IDWriteFontFace
Changed API:
+hb_directwrite_font_create()

New API:
+hb_directwrite_font_get_dw_font_face()
2025-03-22 16:02:39 -06:00
Behdad Esfahbod f0b0d92ab5 [directwrite] Deprecate hb_directwrite_font_get_dw_font() 2025-03-22 15:44:50 -06:00
Behdad Esfahbod 8d8fe19099 [dwrite] Add font-funcs sketch
Small steps.

Fixes https://github.com/harfbuzz/harfbuzz/issues/5144
2025-03-15 21:03:30 -06:00
Behdad Esfahbod 000a0ad7a6 [paint] Add push_font_transform() / push_inverse_font_transform()
Fixes https://github.com/harfbuzz/harfbuzz/issues/5146
2025-03-15 18:35:32 -06:00
Behdad Esfahbod 1e9d101ed7 [directwrite] Add hb_directwrite_face_create_from_file_or_fail()
Just loads the blob from file and creates a face from it.

New API:
+hb_directwrite_face_create_from_file_or_fail()
2025-03-14 02:14:23 -06:00
Behdad Esfahbod b0a5920d1a [directwrite] Add hb_directwrite_face_create_from_blob_or_fail()
Part of https://github.com/harfbuzz/harfbuzz/issues/5082

New API:
+hb_directwrite_face_create_from_blob_or_fail()
2025-03-13 14:27:37 -06:00
Behdad Esfahbod 0b2a0bac47 Merge pull request #5130 from harfbuzz/malloc-rust
[fontations] Make fontations use hb_malloc et al
2025-03-11 23:04:11 -06:00
Behdad Esfahbod 5bf81c3758 [common] Make hb_malloc() et al public
New API:
+hb_malloc()
+hb_calloc()
+hb_realloc()
+hb_free()
2025-03-11 22:52:34 -06:00
Behdad Esfahbod 8e8a9f6f40 [face] Add hb_face_create_or_fail_using()
Fixes https://github.com/harfbuzz/harfbuzz/issues/5117

Untested.

New API:
+hb_face_create_or_fail_using
2025-03-11 11:56:00 -06:00
Behdad Esfahbod f98c203e7a [coretext] Add hb_coretext_face_create_from_blob_or_fail()
Part of https://github.com/harfbuzz/harfbuzz/issues/5117

Untested.

New API:
+hb_coretext_face_create_from_blob_or_fail()
2025-03-11 11:56:00 -06:00
Behdad Esfahbod e4fe8bf95e [ft] Add hb_ft_face_create_from_blob_or_fail()
Part of https://github.com/harfbuzz/harfbuzz/issues/5117

Untested.

New API:
+hb_ft_face_create_from_blob_or_fail()
2025-03-11 11:56:00 -06:00
Behdad Esfahbod 1531be8a1d [buffer] Add docs for the new cluster level 2025-03-10 01:02:11 -06:00
Behdad Esfahbod 5cf1fa3add [face] Fix docs 2025-03-09 01:47:23 -07:00
Behdad Esfahbod 2a878b1b76 [font-funcs] Add two new API
+ hb_font_set_funcs_using()
+ hb_font_list_funcs()

Part of https://github.com/harfbuzz/harfbuzz/issues/5117
2025-03-08 11:50:53 -07:00
Behdad Esfahbod bed809bc6b [fontations] Add to docs
Module itself undocumented so far.
2025-03-06 04:23:24 +02:00
Khaled Hosny 3ef8709829 10.4.0 2025-03-01 00:59:49 +02:00
Khaled Hosny 532ed228e7 [docs] Move all deprecated APIs to the deprecated section 2025-02-28 23:56:03 +02:00
Khaled Hosny 69e1c3c352 [ft] Add hb_ft_font_get_ft_face()
A new name for hb_ft_font_get_face(), which is now deprecated.

Fixes  https://github.com/harfbuzz/harfbuzz/issues/5084
2025-02-28 23:56:03 +02:00
Khaled Hosny 484e8d4997 [directwrite] Add hb_directwrite_face_get_dw_font_face()
A new name for hb_directwrite_face_get_font_face(), which is now
deprecated.

Part of https://github.com/harfbuzz/harfbuzz/issues/5084
2025-02-28 23:56:03 +02:00
Khaled Hosny 694ffa8747 10.3.0 2025-02-20 23:56:59 +02:00
Khaled Hosny 03cf1565f5 [directwrite] Rename font_get_font to font_get_dw_font
https://github.com/harfbuzz/harfbuzz/pull/5079#pullrequestreview-2627407692
2025-02-20 22:16:22 +02:00
Khaled Hosny a8fd29718a [directwrite] Add hb_directwrite_font_create()/font_get_font()
Uses hb_directwrite_face_create() under the hood, but copies variations
from directwrite to hb-font.
2025-02-20 21:40:27 +02:00
Khaled Hosny cf336f17b2 [shape-plan] Rename hb_ot_shape_plan_get_features to get_feature_tags
Simplify the API by returning only feature tags. The users of this API
would be interested only in feature enabled by default and whether the
feature is globally or partially enabled wouldn’t be of much interest in
that case. For user features, the user of the API already have full
access to them.
2025-02-14 21:54:30 +02:00
Khaled Hosny a7d7715f10 [shape-plan] Add hb_ot_shape_plan_get_features
This should get the features on a shape plan after executing it.

Initially I wanted to return an array of tags, but then there can be
user features that are not enabled globally, so I thought returning
hb_feature_t with value and range would be better. There is a TODO since
I couldn’t figure out how to get the value and range from the feature
mask. But also it may be overkill and a simple boolean indicating wither
it is a global feature or not would be enough.

I wounder also what should happen to non-user features that are applied
selectively, like init or medi, does ot make sense to indicate whether
they are global or not?

This is inspired by the discussion in:
https://github.com/fontforge/fontforge/pull/5522#pullrequestreview-2574321449,
but it might be useful to other HarfBuzz users.
2025-02-14 21:54:30 +02:00
Tor Arne Vestbø 04b2006fc9 Remove outdated documentation note about CT operating in 96 DPI
As extensively discussed and documented in #1484, Core Text does
not operate in 96 DPI. Core Text doesn't actually have a concept of
DPI internally, as it doesn't rasterize anything by itself, it just
generates vector paths that get passed along to Core Graphics.

In practice this means Core Text operates in the classical macOS
logical DPI of 72, with one typographic point corresponding to one
point in the Core Graphics coordinate system, which for a normal
bitmap context then corresponds to one pixel -- or two pixels for
a "retina" context with a 2x scale transform.

As of f401f85a5a, we no longer apply
any assumptions in HB about the target DPI being different than the
72 DPI used by CT, for example to account for the Web's standard of
96 DPI, so let's remove the documentation that still indicated this
was necessary.
2025-02-03 04:28:02 -07:00
Khaled Hosny 7b27c8edd4 10.2.0 2025-01-11 18:13:18 +02:00
Garret Rieger bdb50f1c6e [repacker] Also rename api method from hb_subset_repack_or_fail -> hb_subset_serialize_or_fail. 2025-01-07 13:39:51 -07:00
Garret Rieger 056504168c [repacker] rename hb-subset-repacker -> hb-subset-serialize.
Also hb_link_t and hb_object_t to hb_subset_serialize_link_t and hb_subset_serialize_object_t.
2025-01-07 13:39:51 -07:00
Behdad Esfahbod c1f5b780f3 Fix docs 2025-01-07 11:31:05 -07:00
Garret Rieger 5a6f5922dd [repacker] Promote repacking API to stable.
For #227.
2025-01-07 11:28:38 -07:00