* [buffer] Fix a typo and doc string * [buffer] Add merge_grapheme_clusters / merge_out_grapheme_clusters Cluster merges serve two distinct purposes that require different level gates: - Grapheme grouping: combining marks merged into their base cluster. Correct gate: IS_GRAPHEMES (levels 0 and 3). New functions: merge_grapheme_clusters / merge_out_grapheme_clusters. - Monotonicity maintenance: clusters merged to prevent non-monotone output after ligature formation, reordering, etc. Correct gate: IS_MONOTONE (levels 0 and 1). Existing functions: merge_clusters / merge_out_clusters. Previously both paths shared merge_clusters_impl's single IS_MONOTONE gate, so GRAPHEMES (level 3) was incorrectly blocked from merging grapheme clusters. This also caused a regression in the Thai and Hangul shapers for MONOTONE_CHARACTERS (level 1): when their explicit == MONOTONE_GRAPHEMES call-site guards were removed in #5069, the impls' IS_MONOTONE gate no longer protected them. Gates are now in the inline wrappers in hb-buffer.hh; the _impl functions are gate-free. hb_form_clusters, CoreText grapheme detection, and the Thai/Hangul shapers are updated to call the grapheme-semantic variants. Fixes #5750. Fixes https://github.com/harfbuzz/harfbuzz/issues/5751 Fixes https://github.com/harfbuzz/harfbuzz/issues/5750 Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * [thai] Adjust grapheme merging logic * [hangul] Tweak grapheme merging --------- Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
Adding tests
You can test shaping of a unicode sequence against a font like this:
$ ./hb-unicode-encode 41 42 43 627 | ../../util/hb-shape font.ttf
assuming an in-tree build. The 41 42 43 627 here is a sequence of
Unicode codepoints: U+0041,0042,0043,0627. When you are happy with
the shape results, you can use the record-test.sh script to add
this to the test suite. record-test.sh requires pyftsubset to
be installed. You can get pyftsubset by installing
FontTools from https://github.com/behdad/fonttools.
To use record-test.sh, just put it right before the hb-shape invocation:
$ ./hb-unicode-encode 41 42 43 627 | ./record-test.sh ../../util/hb-shape font.ttf
hb-subset and hb-view are resolved as sibling executables next to the
provided hb-shape binary.
what this does is:
- Subset the font for the sequence of Unicode characters requested,
- Compare the
hb-shapeoutput of the original font versus the subset font for the input sequence, - If the outputs differ, perhaps it is because the font does not have
glyph names; it then compares the output of
hb-viewfor both fonts. - If the outputs differ, recording fails. Otherwise, it will move the
subset font file into
data/in-house/fontsand name it after its hash, and print out the test case input, which you can then redirect to an existing or new test file indata/in-house/testsusing-o, e.g.:
$ ./hb-unicode-encode 41 42 43 627 | ./record-test.sh -o data/in-house/tests/test-name.tests ../../util/hb-shape font.ttf
If you created a new test file, add it to data/in-house/meson.build
so it is run. Check that make check does indeed run it, and that the
test passes. When everything looks good, git add the new font as well
as the new test file if you created any. You can see what new files are
there by running git status data/in-house. And commit!
Note! Please only add tests using Open Source fonts, preferably under OFL or similar license.