Files
harfbuzz/test/shape
Dylan PulverandDylan Pulver 2688e6c579 [unicode] Treat U+180F FVS4 as Default_Ignorable (#6229)
U+180F MONGOLIAN FREE VARIATION SELECTOR FOUR was added in Unicode
14.0 and has Default_Ignorable_Code_Point=Yes, but
is_default_ignorable() only covered U+180B..180E, so FVS4 was never
hidden and leaked a .notdef into the output.

8c65442069 ("[Unicode 14] Handle U+180F FVS4") added the U+180F case
to _hb_glyph_info_set_unicode_props() and to the Arabic shaper, but
that branch sits inside "if (unicode->is_default_ignorable (u))", so
it has been unreachable since it was written.  Extending the range to
U+180B..180F makes it live.

U+180B..180F are all Default_Ignorable_Code_Point per
DerivedCoreProperties.txt, so the range stays contiguous.

Before, with a font with no Mongolian coverage:

  $ hb-shape Roboto-Regular.abc.ttf -u U+0061,U+180F
  [1=0+1114|0=0@-908,0+0]

Every other default ignorable (U+180B..180E, U+FE00, U+200B) on that
font shapes to [1=0+1114].  After the change U+180F matches them.

Added two cases to mongolian-variation-selector.tests, which already
covered FVS1, FVS2, FVS3 and MVS but not FVS4: U+180F, and U+1810
MONGOLIAN DIGIT ZERO to pin the top of the range so it cannot be
widened by accident.

Tested with "meson test -C build": 254/254 OK, 0 failures.  Restoring
the old U+180B..180E fails in-house 6174/6176, and widening it to
U+180B..1810 fails the same way.

Assisted-by: claude-opus-5

Co-authored-by: Dylan Pulver <dylanpulver@users.noreply.github.com>
2026-09-03 16:26:30 -06:00
..
2021-08-11 18:49:59 -06:00
2021-08-11 18:49:59 -06:00
2025-06-14 11:02:37 -06:00

Adding tests

You can test shaping of a unicode sequence against a font like this:

$ ./hb-unicode-encode 41 42 43 627 | ../../util/hb-shape font.ttf

assuming an in-tree build. The 41 42 43 627 here is a sequence of Unicode codepoints: U+0041,0042,0043,0627. When you are happy with the shape results, you can use the record-test.sh script to add this to the test suite. record-test.sh requires pyftsubset to be installed. You can get pyftsubset by installing FontTools from https://github.com/behdad/fonttools.

To use record-test.sh, just put it right before the hb-shape invocation:

$ ./hb-unicode-encode 41 42 43 627 | ./record-test.sh ../../util/hb-shape font.ttf

hb-subset and hb-view are resolved as sibling executables next to the provided hb-shape binary.

what this does is:

  • Subset the font for the sequence of Unicode characters requested,
  • Compare the hb-shape output of the original font versus the subset font for the input sequence,
  • If the outputs differ, perhaps it is because the font does not have glyph names; it then compares the output of hb-view for both fonts.
  • If the outputs differ, recording fails. Otherwise, it will move the subset font file into data/in-house/fonts and name it after its hash, and print out the test case input, which you can then redirect to an existing or new test file in data/in-house/tests using -o, e.g.:
$ ./hb-unicode-encode 41 42 43 627 | ./record-test.sh -o data/in-house/tests/test-name.tests ../../util/hb-shape font.ttf

If you created a new test file, add it to data/in-house/meson.build so it is run. Check that make check does indeed run it, and that the test passes. When everything looks good, git add the new font as well as the new test file if you created any. You can see what new files are there by running git status data/in-house. And commit!

Note! Please only add tests using Open Source fonts, preferably under OFL or similar license.