feat: convert composite fonts, kinsoku tables and mojikumi tables

Read the composite font and its entries (classes 0xCB02 and 0xCB03):
name, font, style and custom characters, and the kinsoku and mojikumi
tables with the character lists of a custom kinsoku table.
This commit is contained in:
aiongg committed 2026-10-07 01:40:31 +00:00
1 parent 3f4a68dfe6
commit 9eba4eea66
6 files changed
+348 -1

No files matched your search

+38 -1
View File
@@ -92,7 +92,44 @@ u32 count *n*, then *n* entries: u8 1 if the string is a built-in key
- `NumDesignAxes`, `DesignAxesName` and `DesignAxesValues` (131 fonts of
variable font families) are not in the records above, and were not
found elsewhere.
- `CompositeFont` elements are not written.
## Composite fonts
Every INDD file has one object of class 0xCB02, and every IDML one
`CompositeFont`, `[No composite font]` (240 of 240 IDML files). Its
entries are objects of class 0xCB03; IDML writes them as
`CompositeFontEntry` with `Self="u<UID>"`, six per file. Implemented in
`src/model/cjk.rs`.
**Composite font (chunk 0xCB02).** A flag byte (1 = built-in key, `$ID/`)
and the name, fields not identified, then a u16 count and the UIDs of
the entries, which end the chunk. The list is the IDML entry order in
76 of 78 pairs; the other two are pairs whose IDML was exported from
another save (their entry UIDs have no INDD object). No sample has
another composite font, so the converter writes only this one.
**Entry (chunk 0xCB03).**
| Field | IDML | Evidence |
|---|---|---|
| Flag byte, string | `Name` | 456 of 456 |
| u32 font family UID | `AppliedFont` (Properties): the family's name | 454 of 456; the other 2 are in the pair whose IDML names the family `Minion Pro (OTF)` (above) |
| Flag byte, string | `FontStyle` | 456 of 456 (`$ID/R`, `$ID/Regular`, `$ID/Roman`) |
| Four f64 | `RelativeSize`, `HorizontalScale`, `VerticalScale` 100, `BaselineShift` 0 | (100, 0, 100, 100) in all 1,500 entries of the 250 distinct little-endian files |
| u16 1, u16 count *n*, *n* ranges | `CustomCharacters` | 380 of 380 |
| Four u16 | `ScaleOption`: all 1 `true`, all 0 `false` | 456 of 456 |
Each range is three code points (first, last, first again), each one
UTF-16 unit or a surrogate pair. `CustomCharacters` is every character
from first to last of each range, in order. IDML never writes
`CustomCharacters` for the `$ID/Kanji` entry (240 of 240 files), whose
ranges cover most of Unicode; the converter leaves it out for that entry.
The four numbers have one value in every sample, so only the 0 is told
apart (`BaselineShift`); the converter writes the four attributes only
when the numbers are (100, 0, 100, 100). Which of the four u16 is
`ScaleOption` is not known; the converter writes it only when all four
agree. `Locked="true"` is in every IDML entry and is written from that
observation (`idml-values.md`).
`PlatformName` is `$ID/` in all 4,464 IDML fonts; the converter writes
that value (`idml-values.md`).
+6
View File
@@ -163,6 +163,12 @@ column, inset, footnote and column rule values of
many of these from the INDD. Values that only the 2 files with DOM 20
have (18) are left out as for the preferences.
## Composite font entries
`Locked="true"` is in all 1,440 `CompositeFontEntry` elements of the
240 corpus IDML files. No INDD field for it is identified; the converter
writes it on every entry (`fonts.md`).
## Column direction of pages
All 832 `MarginPreference` elements in the 240 corpus IDML files have
+28
View File
@@ -723,6 +723,34 @@ lines with several different leadings. Of the values in the IDML schema
also has code 3, with no evidence; the converter leaves out codes other
than 0, 1 and 2.
## Kinsoku and mojikumi tables
Kinsoku tables are objects of classes 0x4209 (hard), 0x420A (soft),
0x42B4 (Korean), 0x42B5 (Simplified Chinese), 0x42B6 (Traditional
Chinese) and 0x4204 (custom); mojikumi tables are class 0x4206. IDML
writes one `KinsokuTable` or `MojikumiTable` per object in
`designmap.xml`, kinsoku tables first, each in UID order. Chunk 0x100B is
a flag byte (1 = built-in key, `$ID/`) and the name; `Self` is
`KinsokuTable/<Name>` or `MojikumiTable/<Name>`. Implemented in
`src/model/cjk.rs`.
Evidence: 8 pairs have such objects, and their IDML lists exactly them,
by name and in that order (6 with one table of each kind, 2 with 6
kinsoku and 16 mojikumi tables). The other 70 pairs have neither objects
nor elements.
**Custom kinsoku table (chunk 0x4214).** Five u16 counts, then that many
UTF-16 units for each of five lists: `CantBeginLineChars`,
`CantEndLineChars`, a list not identified, `HangingPunctuationChars` and
`CantBeSeparatedChars`. The evidence is one table in two copies of one
document: the four non-empty lists have different lengths (66, 22, 4
and 3), and each matches the IDML attribute of that length. The layout
parses in all 85 objects of class 0x4204 in the 250 distinct
little-endian files, and the third count is 0 in all of them.
Built-in tables have only `Self` and `Name` in IDML. Mojikumi chunk
0x421E (a u16 from 1 to 16) is not used.
## Colours (0x1F05)
| Chunk | Contents |
+67
View File
@@ -877,6 +877,26 @@ impl Writer<'_> {
}
x.empty("idPkg:Graphic", &[("src", "Resources/Graphic.xml".into())]);
x.empty("idPkg:Fonts", &[("src", "Resources/Fonts.xml".into())]);
// Kinsoku tables, then mojikumi tables, as the schema orders them.
for mojikumi in [false, true] {
for t in doc.cjk_tables.iter().filter(|t| t.mojikumi == mojikumi) {
let tag = if mojikumi {
"MojikumiTable"
} else {
"KinsokuTable"
};
x.start(tag)
.attr("Self", format!("{tag}/{}", self_name(&t.name)))
.attr("Name", &t.name);
if let Some([begin, end, _, hanging, together]) = &t.chars {
x.attr("CantBeginLineChars", begin)
.attr("CantEndLineChars", end)
.attr("HangingPunctuationChars", hanging)
.attr("CantBeSeparatedChars", together);
}
x.end();
}
}
x.empty("idPkg:Styles", &[("src", "Resources/Styles.xml".into())]);
// Present, with these values, in every corpus IDML; the root
// paragraph style refers to it. See docs/format/idml-values.md.
@@ -1720,6 +1740,53 @@ impl Writer<'_> {
}
x.end();
}
// Only the built-in composite font occurs in the corpus.
for cf in self
.doc
.composite_fonts
.iter()
.filter(|c| c.name == "$ID/[No composite font]")
{
x.start("CompositeFont")
.attr("Self", format!("CompositeFont/{}", self_name(&cf.name)))
.attr("Name", &cf.name);
for e in &cf.entries {
// The four numbers are the same in every sample.
let usual = e.numbers == [100.0, 0.0, 100.0, 100.0];
x.start("CompositeFontEntry")
.attr("Self", uref(Some(e.uid)))
.attr("Name", &e.name)
.attr("FontStyle", &e.font_style);
if usual {
x.attr("RelativeSize", "100")
.attr("HorizontalScale", "100")
.attr("VerticalScale", "100");
}
// IDML gives no characters for the Kanji entry.
if e.name != "$ID/Kanji" {
x.attr("CustomCharacters", e.characters());
}
// In every CompositeFontEntry of the corpus IDML files.
x.attr("Locked", "true");
match e.scale {
[1, 1, 1, 1] => {
x.attr("ScaleOption", "true");
}
[0, 0, 0, 0] => {
x.attr("ScaleOption", "false");
}
_ => {}
}
if usual {
x.attr("BaselineShift", "0");
}
if let Some(f) = self.doc.fonts.get(&e.font_family) {
Self::properties(&mut x, &[("AppliedFont", "string", f.name.clone().into())]);
}
x.end();
}
x.end();
}
x.end();
x.finish()
}
+174
View File
@@ -0,0 +1,174 @@
//! Composite fonts, kinsoku tables and mojikumi tables (CJK typography).
//! See `docs/format/fonts.md` and `docs/format/objects.md`.
use crate::Error;
use crate::object::Cursor;
pub mod class {
pub const COMPOSITE_FONT: u32 = 0xCB02;
pub const COMPOSITE_FONT_ENTRY: u32 = 0xCB03;
/// Kinsoku tables: hard, soft, Korean, Simplified Chinese, Traditional
/// Chinese and custom.
pub const KINSOKU: [u32; 6] = [0x4209, 0x420A, 0x42B4, 0x42B5, 0x42B6, CUSTOM_KINSOKU];
pub const CUSTOM_KINSOKU: u32 = 0x4204;
pub const MOJIKUMI: u32 = 0x4206;
}
pub mod chunk {
pub const COMPOSITE_FONT: u32 = 0xCB02;
pub const COMPOSITE_FONT_ENTRY: u32 = 0xCB03;
/// Name of a kinsoku or mojikumi table: flag byte and string.
pub const TABLE_NAME: u32 = 0x100B;
/// Character lists of a custom kinsoku table.
pub const KINSOKU_CHARS: u32 = 0x4214;
}
/// A name stored as a flag byte (1 = built-in key, `$ID/` in IDML) and an
/// in-object string.
fn flagged_name(c: &mut Cursor) -> Result<String, Error> {
let builtin = c.u8()? == 1;
let name = c.string()?;
Ok(if builtin { format!("$ID/{name}") } else { name })
}
#[derive(Debug, Clone, PartialEq)]
pub struct CompositeFont {
pub uid: u32,
/// The IDML name, with `$ID/` for a built-in name.
pub name: String,
/// Entries, in IDML order.
pub entries: Vec<CompositeFontEntry>,
}
#[derive(Debug, Clone, PartialEq)]
pub struct CompositeFontEntry {
pub uid: u32,
pub name: String,
pub font_family: u32,
pub font_style: String,
/// Four f64; (100, 0, 100, 100) in every sample, the 0 being the
/// baseline shift.
pub numbers: [f64; 4],
/// Character ranges as (first, last) code points.
pub ranges: Vec<(u32, u32)>,
/// The four u16 after the ranges: all 1 for `ScaleOption="true"`, all 0
/// for false.
pub scale: [u16; 4],
}
/// A code point stored as one UTF-16 unit or a surrogate pair.
fn code_point(c: &mut Cursor) -> Result<u32, Error> {
let hi = c.u16()? as u32;
if (0xD800..0xDC00).contains(&hi) {
let lo = c.u16()? as u32;
Ok(0x10000 + ((hi - 0xD800) << 10) + (lo.wrapping_sub(0xDC00) & 0x3FF))
} else {
Ok(hi)
}
}
impl CompositeFontEntry {
/// Chunk 0xCB03: name, u32 font family, font style, four f64, u16 1,
/// u16 range count, ranges (first, last, first), four u16.
pub fn read(uid: u32, obj: &crate::Object) -> Result<Option<CompositeFontEntry>, Error> {
let Some(d) = obj.chunk(chunk::COMPOSITE_FONT_ENTRY) else {
return Ok(None);
};
let mut c = Cursor::new(d);
let name = flagged_name(&mut c)?;
let font_family = c.u32()?;
let font_style = flagged_name(&mut c)?;
let numbers = [c.f64()?, c.f64()?, c.f64()?, c.f64()?];
c.u16()?;
let n = c.u16()?;
let mut ranges = Vec::new();
for _ in 0..n {
let first = code_point(&mut c)?;
let last = code_point(&mut c)?;
code_point(&mut c)?;
ranges.push((first, last));
}
let scale = [c.u16()?, c.u16()?, c.u16()?, c.u16()?];
Ok(Some(CompositeFontEntry {
uid,
name,
font_family,
font_style,
numbers,
ranges,
scale,
}))
}
/// The characters of the ranges, in order (IDML `CustomCharacters`).
pub fn characters(&self) -> String {
self.ranges
.iter()
.flat_map(|&(a, b)| (a..=b).filter_map(char::from_u32))
.collect()
}
}
impl CompositeFont {
/// Chunk 0xCB02: the name, fields not identified, then a u16 count and
/// the entry UIDs, which end the chunk. Returns the name and the
/// entry UIDs.
pub fn read(obj: &crate::Object) -> Result<Option<(String, Vec<u32>)>, Error> {
let Some(d) = obj.chunk(chunk::COMPOSITE_FONT) else {
return Ok(None);
};
let mut c = Cursor::new(d);
let name = flagged_name(&mut c)?;
let start = c.pos();
let Some(p) = (start..d.len().saturating_sub(1)).find(|&p| {
let n = u16::from_le_bytes([d[p], d[p + 1]]) as usize;
n > 0 && p + 2 + 4 * n == d.len()
}) else {
return Ok(Some((name, Vec::new())));
};
let mut c = Cursor::new(&d[p..]);
let n = c.u16()?;
let entries = (0..n).map(|_| c.u32()).collect::<Result<Vec<_>, _>>()?;
Ok(Some((name, entries)))
}
}
/// A kinsoku or mojikumi table.
#[derive(Debug, Clone, PartialEq)]
pub struct CjkTable {
pub uid: u32,
pub mojikumi: bool,
pub name: String,
/// The character lists of a custom kinsoku table: cannot begin a line,
/// cannot end a line, (not identified), hanging punctuation, cannot be
/// separated.
pub chars: Option<[String; 5]>,
}
impl CjkTable {
pub fn read(uid: u32, cls: u32, obj: &crate::Object) -> Result<Option<CjkTable>, Error> {
let Some(d) = obj.chunk(chunk::TABLE_NAME) else {
return Ok(None);
};
let name = flagged_name(&mut Cursor::new(d))?;
let chars = match obj.chunk(chunk::KINSOKU_CHARS) {
Some(d) if cls == class::CUSTOM_KINSOKU => {
let mut c = Cursor::new(d);
let counts = [c.u16()?, c.u16()?, c.u16()?, c.u16()?, c.u16()?];
let mut lists: [String; 5] = Default::default();
for (list, n) in lists.iter_mut().zip(counts) {
let units = (0..n).map(|_| c.u16()).collect::<Result<Vec<_>, _>>()?;
*list = String::from_utf16_lossy(&units);
}
Some(lists)
}
_ => None,
};
Ok(Some(CjkTable {
uid,
mojikumi: cls == class::MOJIKUMI,
name,
chars,
}))
}
}
+35
View File
@@ -2,6 +2,7 @@
//! `docs/format/objects.md` for the class and chunk layouts used here.
pub mod attrs;
pub mod cjk;
pub mod color;
pub mod font;
pub mod hyperlink;
@@ -12,6 +13,7 @@ pub mod xref;
use std::collections::{BTreeMap, HashMap};
pub use attrs::{Attrs, Value};
pub use cjk::{CjkTable, CompositeFont, CompositeFontEntry};
pub use color::{Color, Gradient, Tint};
pub use font::{Font, FontFamily};
pub use hyperlink::{Bookmark, Destination, DestinationKind, Hyperlink, SourceRange, TextSource};
@@ -458,6 +460,9 @@ pub struct Document {
/// Problems that did not stop the conversion (content left out).
pub warnings: Vec<String>,
pub preferences: Option<DocumentPreferences>,
pub composite_fonts: Vec<CompositeFont>,
/// Kinsoku and mojikumi tables, in UID order.
pub cjk_tables: Vec<CjkTable>,
}
/// Document setup, from chunk 0x533 of the preferences object.
@@ -655,6 +660,9 @@ impl<'a> Reader<'a> {
let mut destinations = Vec::new();
let mut bookmarks = BTreeMap::new();
let mut cross_reference_formats = BTreeMap::new();
let mut composite_fonts = Vec::new();
let mut composite_entries = HashMap::new();
let mut cjk_tables = Vec::new();
for &(uid, cls) in self.db.classes() {
if self.db.object(uid)?.is_none() {
continue;
@@ -838,6 +846,21 @@ impl<'a> Reader<'a> {
bookmarks.insert(uid, b);
}
}
cjk::class::COMPOSITE_FONT => {
if let Some((name, entries)) = CompositeFont::read(&*self.object(uid)?)? {
composite_fonts.push((uid, name, entries));
}
}
cjk::class::COMPOSITE_FONT_ENTRY => {
if let Some(e) = CompositeFontEntry::read(uid, &*self.object(uid)?)? {
composite_entries.insert(uid, e);
}
}
c if c == cjk::class::MOJIKUMI || cjk::class::KINSOKU.contains(&c) => {
if let Some(t) = CjkTable::read(uid, c, &*self.object(uid)?)? {
cjk_tables.push(t);
}
}
class::LANGUAGE => {
if let Some(d) = self.chunk(uid, chunk::LANGUAGE_NAME)?
&& d.len() > 1
@@ -900,6 +923,18 @@ impl<'a> Reader<'a> {
cross_reference_formats,
warnings: self.warnings.borrow().clone(),
preferences: self.document_preferences()?,
composite_fonts: composite_fonts
.into_iter()
.map(|(uid, name, entries)| CompositeFont {
uid,
name,
entries: entries
.iter()
.filter_map(|e| composite_entries.get(e).cloned())
.collect(),
})
.collect(),
cjk_tables,
})
}