Commit Graph
151 Commits
Author SHA1 Message Date
aiongg 39e81d0247 feat: generate the root style values from the compared pairs
tools/root_values.py collects the root styles from the pairs that
compare.py compares (little-endian, same major version, without the
privately held samples): 794 distinct IDML files, DOM 7 to 21, instead
of the 88 the corpus had first. The pair rule leaves out an IDML with no
root styles beside a newer INDD and a big-endian pair. The other object
styles and the preferences stay pinned to SOURCE; their files are
unchanged. The root table style writes every mapped attribute of its
list, since the stroke weights, spacing and end tints now vary between
files and are no longer in the values file. The root object style's
varying values (named grid, optical margin size, baseline frame grid
colour, corner radii, gutter, first baseline offset) are read from the
INDD. Evidence in idml-values.md and tables.md.

compare.py --exclude own/, trustworthy pairs: coverage 98.53 % ->
98.56 % (+2,942 values whose first DOM version is now earlier:
IncludeClass, EpubAriaRole, paragraph border options,
ParagraphKashidaWidth, shading overprint, TransformAttributeOption and
FlexLayoutAttributeOption), documents >= 99 % 278 -> 313; all pairs
96.65 % -> 96.67 %. No pair loses a reproduced value; extras unchanged
(8,968 trustworthy, 278,333 all). No new failures.
2026-10-09 19:46:58 +00:00
aiongg 7085facf9d feat: write fill, stroke and corner values that items take from the base list
Every document has one object of class 0x6E02 whose chunk 0x6E03 is a
page item attribute list. An item that does not store a fill, stroke or
corner attribute has the value of this list; IDML writes it where it
differs from the object style's effective value. When the object style
turns the attribute's category off, IDML writes the item's value even
where it equals the style's (EnableFill: fill; EnableStroke: stroke
colour; EnableStrokeAndCornerOptions: the other stroke and corner
attributes). This replaces StrokeWeight="1" for items with [None]: the
base list has 0.2835 in 125 of 606 trustworthy documents, which
explains the 0.1 mm stroke weights, and corner radii of 14.1732 in the
same documents. Groups are left out; EPS text takes no corner values.
Evidence in attributes.md.

compare.py --exclude own/, trustworthy pairs: coverage 98.41 % ->
98.53 % (StrokeColor of rectangles, text frames, ovals, polygons and
lines 3,964 missing -> 4; StrokeWeight 256 wrong and 200 missing -> 0;
corner radii and options about 10,000 missing -> 24; fill, miter and
category-off values), documents >= 99 % 268 -> 278; all pairs 96.54 %
-> 96.65 %. Extras unchanged on trustworthy pairs (8,968); over all
pairs +93: 4 StrokeWeight values in 3 stale pairs and attributes of
items the reference does not have. No new failures.
2026-10-09 19:33:37 +00:00
aiongg 15d6af5eee feat: write the grid starting point, TOC story direction and object style optical margin
- Page GridStartingPoint from the u32 at 0 of chunk 0xCD04 (0
  TopOutside, 4 CenterVertical; other codes are in no pair and are
  left out).
- TOCStyle SetStoryDirection from the first u16 field of chunk 0x11605
  (0 Horizontal, 1 Vertical). It is no longer a WHEN_WRITTEN key of
  element_values.py (the regenerated file is unchanged).
- ObjectStyle StoryPreference OpticalMarginSize and
  OpticalMarginAlignment from the f64 at 2 and u16 at 12 of chunk
  0x285B, as in the story chunk 0x2EE.

Evidence in objects.md.

compare.py --exclude own/, trustworthy pairs: coverage 98.32 % ->
98.41 % (Page GridStartingPoint 10,080 missing -> 0, TOCStyle
SetStoryDirection 620 missing -> 0, object style OpticalMarginSize 261
and OpticalMarginAlignment 4 wrong -> 0), documents >= 99 % 225 -> 268;
all pairs 96.46 % -> 96.54 %. Extras unchanged on trustworthy pairs
(8,968); over all pairs +182, the new attribute on pages of stale pairs
that the reference does not have. No new failures.
2026-10-09 18:51:48 +00:00
aiongg 29b8757ef6 feat: write the named grid of stories and object styles
Read chunk 0xCD32 of stories and object styles (u16 1 and a named grid
UID, or u16 0 for none) and write AppliedNamedGrid as the grid's Self,
or n. Evidence in objects.md, named grids.

compare.py --exclude own/, trustworthy pairs: coverage 98.16 % ->
98.32 % (Story AppliedNamedGrid 19,424 missing -> 0, ObjectStyle
AppliedNamedGrid 37 wrong -> 0), documents >= 99 % 187 -> 225; all
pairs 96.30 % -> 96.46 % (26,805 of 26,805 stories, 3,515 of 3,515
object styles). Extras unchanged (8,968), no new failures.
2026-10-09 18:37:34 +00:00
aiongg 5ee17cb100 feat: write the baseline frame grid colour and two text preferences
Write TextPreference UseCidMojikumi and UseNewVerticalScaling from
bytes 168 and 170 of preferences chunk 0x280 when they are equal (803
of 803 pairs). Write BaselineFrameGridColor in the preferences and in
every object style by an observed rule over stored data: the interface
colour of chunk 0x2834 (u32 at 20) when not 0, otherwise Charcoal if
byte 168 of chunk 0x280 is 1 and LightBlue if it is 0 (803 of 803
pairs). Evidence in preferences.md.

compare.py --exclude own/, trustworthy pairs: coverage 98.15 % ->
98.16 % (wrong values fixed: object style colours 581, preference
colour 132, UseCidMojikumi and UseNewVerticalScaling 129 each),
documents >= 99 % 184 -> 187; all pairs 96.29 % -> 96.30 %. Extras
unchanged (8,968), no new failures or warnings, schema failures 0 on
pairs.

diff_outputs.py: 4,077 outputs differ, only in Resources/Preferences.xml
and Resources/Styles.xml: the two text preferences now come from the
INDD (order of attributes; values change where byte 168 is 1), and the
colour changes where the rule gives Charcoal.
2026-10-08 17:38:34 +00:00
aiongg a5de84e81f feat: write frame grids and the layout grid of their stories
A text frame whose multi-column frame object has chunk 0xCD41 starting
with u16 1 is a frame grid: write its GridDataInformation (grid view,
character count at the bottom, the document's character count size).
A frame grid story writes its own chunk 0xCD02 as the story's
GridDataInformation. Story FrameType already comes from chunk 0x2EE
and is unchanged. Evidence in objects.md.

compare.py --exclude own/, trustworthy pairs: coverage 98.14 % ->
98.15 % (+1,600 values: 246 TextFrame and 56 Story grid elements with
all their attributes); all pairs 96.28 % -> 96.29 %, 248 of 248 frame
grids and 57 of 57 story grids. Extras unchanged (8,968), no new
failures or warnings, schema failures 0 on pairs.

diff_outputs.py: 99 outputs differ, in the spreads and stories of
documents with frame grids.
2026-10-08 17:23:05 +00:00
aiongg 7aa63682b8 feat: write layout grid sizes and named grid settings
Write PointSize, CharacterAki and LineAki of layout grids whatever
their value: the corpus now has pages and named grids with other
values, all equal to the IDML. Read the flag before the grid's font
style (1 = built-in key, $ID/). Named grids take their own chunk 0xCD02
when they have one (grids made by the user), otherwise the document's
layout grid preference, which the built-in [Page Grid] equals in 803 of
803 pairs; before, they took the pages' grid. One reader now decodes
the grid layout for pages, named grids and the preferences. Evidence
in objects.md.

compare.py --exclude own/, trustworthy pairs: coverage 98.12 % ->
98.14 % (+2,418 values: page PointSize and LineAki 875 each, page
FontStyle 44, named grids +624 including 32 more elements, 47 wrong
values fixed), documents >= 99 % 182 -> 184; all pairs 96.26 % ->
96.28 %. Extras unchanged (8,968), no new failures or warnings, schema
failures 0 on pairs.

diff_outputs.py: 663 outputs differ, in the grids of pages, master
pages, named grids and the two grid preferences.
2026-10-08 17:08:10 +00:00
aiongg a958e55d82 feat: write PrimaryTextFrame of master spreads
Read master spread chunk 0x140A (u32 story) and write PrimaryTextFrame
as the last MasterSpread attribute from DOM 8: n without a story,
otherwise the spread's text frame of that story that has no previous
frame. Evidence in objects.md.

compare.py --exclude own/, trustworthy pairs: coverage 98.11 % ->
98.12 % (MasterSpread PrimaryTextFrame 0 -> 1,025 of 1,025), documents
>= 99 % 179 -> 182; all pairs 96.25 % -> 96.26 %, 1,521 of 1,521
values. Trustworthy extras unchanged (8,968); over all pairs +9, one
attribute on each of 9 master spreads that the stale IDMLs lack. No new
failures or warnings, schema failures 0 on pairs.

diff_outputs.py: 3,620 outputs differ, only in MasterSpreads/*.
2026-10-08 16:43:09 +00:00
aiongg 3a8718d9db fix: accept any second byte in searched in-object strings
The byte after the string tag is a script code in some strings, not
always 0. Read a layer name at offset 19 of chunk 0x304 when the tag is
there, accept any second byte in find_string and in the style GUID
pattern. Evidence in objects.md.

compare.py --exclude own/, trustworthy pairs: coverage 98.10 % ->
98.11 % (+912 values): Layer Name 188 wrong -> 0, StyleUniqueId +479
paragraph and +245 character styles, 4 more styles whose IDML GUID
differs from the stored one (a known kind). All pairs 96.24 % ->
96.25 %. Extras unchanged (8,968), no new failures or warnings, schema
failures 0 on pairs.

diff_outputs.py: 383 outputs differ: 455 layer names that were empty
now have their name (346 files), and 1,538 styles gain StyleUniqueId.
No other change.
2026-10-08 16:27:10 +00:00
aiongg c43a16815b feat: write custom mojikumi tables
Read class 0x4203 (name in chunk 0x100B, settings in chunk 0x420A) and
write each as a MojikumiTable with BasedOnMojikumiSet and its
OverrideMojikumiAkiList, together with the built-in mojikumi tables in
UID order. Mojikumi references to these tables become
MojikumiTable/<name> objects. Evidence in objects.md: 185 of 185 tables
and 62,439 of 62,439 entries over all pairs.

compare.py --exclude own/, trustworthy pairs: coverage 98.09 % ->
98.10 % (+1,193 values: 97 tables, 576 paragraph ranges, 105 paragraph
styles, 94 cell ranges, 19 story ranges, 11 text defaults); all pairs
96.23 % -> 96.24 %. No wrong values, extras unchanged (8,968), no new
failures or warnings, schema failures 0 on pairs.

diff_outputs.py: 132 outputs differ, all files with custom mojikumi
tables (designmap.xml, and the styles and stories that refer to them).
2026-10-08 16:09:58 +00:00
aiongg 53fff7650f feat: write ruby, kenten and warichu attributes
Map 14 more text attribute IDs (ruby alignment, spacing, overhang,
position, offset, scaling, auto align, tate-chu-yoko digits, X scales,
font styles; warichu on/off, lines, size, line spacing, alignment), add
KinsokuHangType code 1 and KentenKind code 5, and skip the ruby strand
(run kind 0x420D), a copy that IDML does not use. Evidence in
attributes.md and objects.md.

compare.py --exclude own/, trustworthy pairs: coverage 98.07 % ->
98.09 % (+3,312 values), documents >= 99 % 176 -> 179; all pairs
96.21 % -> 96.23 %. No wrong values among the new ones, extras
unchanged (8,968), no new failures, schema failures 0 on pairs.

diff_outputs.py: 4,433 outputs differ. In 4,083 only the order of
attributes changes: root styles and text defaults now take these values
from the INDD instead of the observed values. The other 350 gain the
attributes on ranges and styles.
2026-10-08 15:50:08 +00:00
aiongg 2541186464 docs: correct the kinds of LAB colours in the corpus 2026-10-08 15:06:27 +00:00
aiongg 236ab10dea docs: add the LAB and HSB spaces to the colour value chunk 2026-10-08 15:06:13 +00:00
aiongg a0da69e463 feat: write outer and inner glow, isolate blending and more blend modes
Add the transparency attribute IDs of the outer glow (applied, blend
mode, opacity, colour, spread, size), the inner glow (applied, blend
mode, size), isolate blending and the fill opacity, and every blend
mode code as its position in the schema's enumeration (codes 2, 4, 5,
7, 14 and 15 are seen besides 0, 1, 3 and 9).

compare.py --exclude own/:
- trustworthy 98.06% -> 98.07% (+1,055 values: OuterGlowSetting of
  text frames 108 of 108, of rectangles 56 of 56; BlendMode of
  rectangles 271 -> 337 of 337)
- all pairs 96.21% (+1,247)
- extra values unchanged (8,968; all pairs 277,399)
2026-10-08 15:05:23 +00:00
aiongg b83a11c923 fix: number bullets in hexadecimal
IDML writes the index in the Self of ABullet in hexadecimal
(dABullet9, dABulleta); the converter wrote it in decimal from the
tenth bullet on.

compare.py --exclude own/: trustworthy 98.06% (+25 values, ABullet
3,365 of 3,365); extra values 8,993 -> 8,968.
2026-10-08 14:57:59 +00:00
aiongg 58ccd9970e feat: write the gradient that the page item defaults name
Chunk 0x6E06 of the page item defaults object names an object of class
0x5533, a gradient and a colour. IDML writes that gradient in every
document even when nothing refers to it. The converter now treats the
swatches of the chunk as named by the page item defaults, as it does
for the entries of the two tables around the attribute list.

compare.py --exclude own/:
- trustworthy 98.05% -> 98.06% (+413 values: Gradient 756 -> 783 of
  783 elements, with their stops)
- all pairs 96.20% (+815)
- extra values unchanged (8,993; all pairs 277,424)
2026-10-08 14:53:44 +00:00
aiongg 1844a83671 fix: leave out spread guides outside the spread's pages
IDML leaves out a guide owned by the spread when it lies outside the
bounding box of the spread's pages (84 of 85 such guides in the
trustworthy pairs measured before; all spread guides inside and all
page guides are written). The one exception, on a master spread, is
lost.

compare.py --exclude own/:
- trustworthy 98.05% (-12 values: the exception guide)
- extra values 10,001 -> 8,993 (no extra Guide left in trustworthy
  pairs); all pairs 279,404 -> 277,424 (-48 values)
2026-10-08 14:46:42 +00:00
aiongg dccfc4a5ce feat: convert placed InDesign pages
Read class 0x6607 (a page of another InDesign document placed in a
frame) as a placed graphic and write it as ImportedPage: PageNumber
(chunk 0x2505 u32 + 1), the attributes of placed graphics,
GraphicBounds, the text wrap and the link. ImportedPageCrop, the XMP
packet and the PDF-style children are not decoded and left out.

compare.py --exclude own/:
- trustworthy 97.91% -> 97.92% (+1,475 values; ImportedPage 38 of
  38 with transform, bounds and page number, and their links)
- all pairs 96.02% -> 96.03%; 9 more links resolved to another file
  at export (45 values leave the counts)
- extra values unchanged
- compare.py --all with schemas: 0 conversion and 0 schema failures
  on pairs; outputs without a reference: the same 11 endnote-markup
  files
2026-10-08 14:24:50 +00:00
aiongg 1842f3568d feat: convert text on a path
Chunk 0xB30A of a shape or frame names its text path object (class
0xB320) and holds the flip, the start and end brackets and three codes.
The text path's multi-column frame holds a column of class 0xB318 that
gives the story and the thread, as for text frames. The converter
writes TextPath right after the item's Properties.

compare.py --exclude own/:
- trustworthy 97.90% -> 97.91% (+396 values; TextPath 36 of 36)
- all pairs 96.02% (+462; 42 of 42 with an INDD object)
- extra values unchanged in trustworthy pairs; all pairs +153: text
  paths in 2 stale pairs whose IDML does not have them
2026-10-08 14:09:49 +00:00
aiongg 83a6a7137a feat: convert EPS text
Read class 0x660B (text kept from a placed EPS or PDF graphic) as a
page item and write it as EPSText: ItemTransform, PathBoundingBox
(chunk 0x154), EPSTextData (chunk 0x6611 in big-endian order without
two tail fields, base64), EPSTextAttributeBounds (chunk 0x6612) and
the text wrap. element_values.py collects the EPSText values every
element has.

compare.py --exclude own/:
- trustworthy 97.72% -> 97.90% (+22,108 values; EPSText 633 of 633
  elements, with transform, bounds, text data and text wrap all equal)
- all pairs 95.89% -> 96.02%
- extra values unchanged in trustworthy pairs (26,256); all pairs
  304,533 -> 305,985: 1 stale pair whose IDML lacks its EPS text
- not yet reproduced: EPSText StrokeType (634), OverprintFill (33),
  StrokeDashAndGap (7)
2026-10-08 14:02:35 +00:00
aiongg 0a01b3c6d4 fix: write numbers as IDML does at every length
num() rounded ties to even only at 17 significant digits. IDML takes
the decimal nearest to the exact value at the shortest length, with
ties to even, at any length, and writes numbers below 1e-4 with an
exponent (1e-05). The rule gives the IDML text for all but 67 of the
1,090,937 distinct numeric texts of the reference IDMLs (those 67 are
text values).

diff_outputs.py: 2,672 outputs differ, no change in failures or
warnings.

compare.py --exclude own/:
- trustworthy 97.72% (+77 values): PDF GraphicBounds 849 -> 913 of
  913, EPS 323 -> 329, SVG 140 -> 146
- all pairs +99 values: PDF, EPS and SVG GraphicBounds all equal
- extra values unchanged
2026-10-08 13:51:34 +00:00
aiongg d2ad1e611a fix: read font families that refer to another family
Record kind 2 of chunk 0x3E05 is u8 2, u8 0, the u32 UID of another
font family and a u16 count of font entries. The converter read it as
a family record and left the fonts out with a warning ("string tag 5
at 3, expected 2" and similar), in 14 Japanese pairs and 5 other files.

IDML writes a family with a count of 0 as a copy of the family it
refers to, with FontType="Unknown", Version="" and TypekitID="$ID/"
(15 of 15 pairs). The converter now does the same. For a record with
entries (4 families in 2 files, no IDML) it keeps the name and leaves
the fonts out with a warning.

compare.py --exclude own/:
- trustworthy 97.72% (+234 values: 16 fonts, FontFamily Name 15,
  AppliedFont of styles and text 11); all pairs 95.89% (+234)
- extra values unchanged (26,256; all pairs 304,533)
2026-10-08 13:42:42 +00:00
aiongg e792768818 fix: read the composite font of InDesign 3.0 files
In InDesign 3.0 files chunk 0xCB02 starts with four zero bytes before
the flagged name (9 of 9 distinct 3.0 files, both byte orders). The
converter left the composite font out with a warning ("string tag 0
at 1, expected 2"); it now reads the name after the four bytes when it
does not parse at 0. The 3.0 files convert without that warning.

No pair is from 3.0: compare.py --exclude own/ is unchanged
(trustworthy 97.72%, all pairs 95.89%).
2026-10-08 13:42:42 +00:00
aiongg 0c560ea96b feat: write the values that follow the language of the last session
Decode the save history of the document (chunk 0x1D8): one record per
session, with a u16 language code. The layout parses 4,431 of the
4,439 distinct valid corpus files (not the six InDesign 2.x files and
two without the chunk) and all 803 pairs. Code 0x0101 in the last
record marks a session of a Japanese or Chinese edition: all 149
Japanese pairs have it.

The converter uses it for values that IDML writes in the exporting
InDesign's language:

- spread FlattenerPreference without chunk 0x10833: 400/400 for
  0x0101, otherwise 300/150, none when the history cannot be read
  (7,481 of 7,490 spreads of all pairs; the 9 others are one pair
  exported in a session the INDD does not record)
- IndexOptions without chunk 0x13010: BetweenEntriesSeparator "、" or
  "; " (789 of 791 pairs) and Title "索引" for 0x0101 (158 of 159);
  no title for other codes, whose titles vary by language
- EndnoteOption without chunk 0x2261E: separator U+3000 and
  RubyMarker for 0x0101, tab and SuperscriptMarker otherwise (140 of
  141)

The endnote title and the Assignment Name do not follow the code and
stay out (objects.md, save history, has the counts).

compare.py --exclude own/:
- trustworthy coverage 97.71% -> 97.72% (+1,119 values: flattener
  resolutions 5,065 -> 5,404 of 5,413 each, index title 10 -> 136,
  separator 481 -> 604 of 606, endnote separator and marker position
  301 -> 397 of 398)
- all pairs 95.88% -> 95.89% (+1,679)
- extra values unchanged (26,256; all pairs 304,533)
- compare.py --all with schemas: 0 conversion and 0 schema failures
  on pairs
2026-10-08 13:34:24 +00:00
aiongg fb283101c3 feat: read the frame type and orientation of stories
Story chunk 0x2EE also holds the text orientation (u16 at 0: 0
Horizontal, 1 Vertical) and the frame type (u16 at 14: 0
TextFrameType, 1 FrameGridType). Over the 28,669 stories of all corpus
pairs both match the IDML in every story; stories without the chunk
are all TextFrameType and Horizontal. The converter writes
StoryPreference FrameType and StoryOrientation from it; the
orientation of the story's frames is used only for an unknown code.

compare.py --exclude own/:
- trustworthy coverage 97.55% -> 97.71% (11,986,627 -> 12,006,265);
  StoryPreference FrameType 0 -> 19,424 of 19,424, StoryOrientation
  19,210 -> 19,424 of 19,424
- all pairs 95.72% -> 95.88%
- extra values 26,256 (unchanged); all pairs 303,866 -> 304,533: the
  two attributes on the stories of stale pairs that the reference does
  not have
2026-10-08 13:15:38 +00:00
aiongg d34e22bea2 chore: regenerate the element values from the larger corpus
Regenerate src/idml/element_values.xml from the 827 distinct IDML files
of the corpus (683 before). Values that now vary across the IDML files
drop out:

- Story AppliedNamedGrid, StoryPreference FrameType and
  StoryOrientation (stories with named grids, frame grids or vertical
  text)
- Page GridStartingPoint (28 pages of DOM 18)
- TOCStyle SetStoryDirection (3 styles are Vertical)
- 9 TextDefault, 2 CjkGridPreference and 1 IndexOptions preference values

element_values.py now collects ClippingPathSettings only for
ClippingType="None", the only case in which the converter writes the
element, and leaves user-named trap presets out of the TrapPreset list.
Without these two rules a DOM 12 file with its own trap preset and 8
images with a clipping path would have removed the values for every
document.

root_values.py read the text attribute table from src/idml/mod.rs,
which moved to attrs.rs; it now runs again. Its files stay pinned and
are not regenerated (idml-values.md says why).

compare.py --exclude own/ (803 pairs, 606 trustworthy):
- trustworthy coverage 97.95% -> 97.55% (12,036,295 -> 11,986,627 of
  12,287,602); all pairs 96.12% -> 95.72%
- the loss is the dropped Story AppliedNamedGrid (19,416), FrameType
  (19,368), Page GridStartingPoint (10,052), TOCStyle
  SetStoryDirection (618) and StoryOrientation (214)
- extra values 26,257 -> 26,256 (all pairs 305,366 -> 303,866)
- conversion failures 0 of 803
2026-10-08 13:04:04 +00:00
aiongg b0c886f77d feat: read the column direction of pages
The last u16 of page chunk 0x528 is the column direction (1 vertical,
0 horizontal), inherited from the master page with the columns. It
replaces the constant Horizontal the writer wrote on every page.

compare.py --exclude own/:
- trustworthy value coverage 98.03% -> 98.03% (+14 values); all pairs
  +14
- Page/MarginPreference ColumnDirection 9,274 of 9,274 (14 vertical)
- extra values unchanged; no key lost values
2026-10-08 08:35:25 +00:00
aiongg 64f08256e3 feat: take story, frame, clipping and ink constants from the value files
The writer wrote these as literals. element_values.py now collects
Story, XmlStory, Story/StoryPreference, the InCopyExportOption of
stories, rectangles, ovals and polygons, the ClippingPathSettings of
images, PDF and EPS graphics, and Ink; the writer reads them from
element_values.xml after the values it reads from the INDD.

- UserText (a WhenWritten value) is written from version 11.2, where
  IDML starts to have it (was 12).
- Ink ConvertToProcess is read from byte 56 after the ink name; it was
  the only ink value that varies (3 inks true).
- InCopyExportOption of ovals and the EPS clipping values follow the
  versions the corpus shows (ovals from DOM 8, EPS up to DOM 20).

compare.py --exclude own/:
- trustworthy value coverage 98.03% -> 98.03% (+771 values: UserText
  768, ConvertToProcess 3); all pairs 96.12% -> 96.13% (+1,211)
- extra values: trustworthy unchanged; all pairs +43 (UserText in
  stale pairs whose IDML is older)
- compare.py --all with schemas: 0 schema failures on pairs; files
  without a reference as before
2026-10-08 08:21:32 +00:00
aiongg c98436d3b2 docs: note the KerningMethod runs the new codes do not explain 2026-10-08 08:03:17 +00:00
aiongg af9f21700a feat: add the text enumeration codes found in the corpus
New codes for Position, Justification, Composer, KerningMethod,
ParagraphBreakType, StartParagraph, OTFFigureStyle, SpanColumnType and
BulletsAndNumberingListType, each found by aligning INDD runs with IDML
ranges and styles by name; no run or style contradicts them.

compare.py --exclude own/:
- trustworthy value coverage 98.01% -> 98.03% (+1,859 values); all
  pairs 96.10% -> 96.12% (+2,700)
- extra values: trustworthy unchanged; all pairs +13 (stale pairs)
- no key lost values
2026-10-08 08:03:04 +00:00
aiongg 4633fdeec5 feat: write 90 more paragraph and character attributes
Hyphenation, glyph and word spacing, keeps, rules above, underline and
strikethrough types, colours and tints, numbering format, alignment and
level, paragraph and character direction, OpenType features, CJK
settings (aki, kinsoku, ruby, kenten) and paragraph border and shading
widths, offsets and weights. Each ID is present on a style exactly when
the IDML style has the attribute. Rule, underline and strikethrough
stroke types are a reference and a built-in code (new layout
RefOrCode); NumberingFormat is written as a string.
MergeConsecutiveParaBorders is written from version 13.1, as in IDML.

compare.py --exclude own/:
- trustworthy value coverage 97.66% -> 98.01% (+41,082 values); all
  pairs 95.64% -> 96.10% (+74,262)
- of 121,196 values of these attributes, 120,409 reproduced and 3 wrong
  (all 3 were wrong before)
- extra values: trustworthy unchanged; all pairs +218 (ranges and
  styles of stale pairs that their IDML lacks)
- compare.py --all with schemas: 0 schema failures on pairs; files
  without a reference as before
2026-10-08 07:58:48 +00:00
aiongg cd7116ff8d feat: read the numbering lists and AppliedNumberingList
Preferences chunk 0x1A49B lists the default numbering list and the
others; each list (class 0x1A483) has its name and two flags in chunk
0x1A49A. The converter writes every list in designmap.xml, the default
first, in place of the constant default list, which it keeps only for a
file whose lists it cannot read. Paragraph attribute 0x1A417
(AppliedNumberingList) names a list by UID.

compare.py --exclude own/:
- trustworthy value coverage 97.65% -> 97.66% (+896 values); all pairs
  95.63% -> 95.64% (+1,231)
- NumberingList 705 of 705 (216 more), AppliedNumberingList +32
- extra values unchanged; no key lost values
- diff_outputs.py: other outputs differ only in the position of the
  AppliedNumberingList property
2026-10-08 07:38:55 +00:00
aiongg 98889a8bfc feat: read story title, direction and optical margin from their chunks
Story chunks 0xA44C (StoryTitle), 0x50F96 (StoryDirection, 1 right to
left) and 0x2EE (OpticalMarginSize and OpticalMarginAlignment) replace
the constant values the writer wrote for every story. Without a chunk
the story keeps the value of every story without it.

compare.py --exclude own/:
- trustworthy value coverage 97.64% -> 97.65% (+1,566 values); all
  pairs 95.61% -> 95.63% (+3,461)
- StoryDirection 780, OpticalMarginAlignment 441, OpticalMarginSize 315
  and StoryTitle 30 more stories right
- extra values unchanged; no key lost values
2026-10-08 07:31:48 +00:00
aiongg 53746658d9 feat: decode style export flags, AllowOverrides and the document label
- Paragraph and character styles: SplitDocument, EmitCss and
  IncludeClass from the u16 values after the tag map count of chunk
  0x28F0 (when the style has no tag maps); false, true, true without
  the chunk.
- Page items of master spreads: AllowOverrides from chunk 0x142D
  (1 false, else true). Placed graphics and items of document spreads
  have none.
- Document/Properties/Label from document chunk 0x1630B.

compare.py --exclude own/:
- trustworthy value coverage 97.48% -> 97.64% (+18,499 values); all
  pairs 95.46% -> 95.61% (+22,885)
- AllowOverrides 11,791 of 11,791; paragraph style SplitDocument +3,403
  and IncludeClass +2,869; document label 436 of 438
- extra values +3 (ParagraphStyle SplitDocument); no key lost values
- compare.py --all with schemas: 0 schema failures on pairs; files
  without a reference as before
2026-10-08 07:25:44 +00:00
aiongg f0bd7f1e3b feat: decode default styles, grids, guide colours and other preferences
From the preferences object:
- TextDefault AppliedParagraphStyle and AppliedCharacterStyle (chunks
  0x28D4, 0x28D5); PageItemDefault graphic, text and grid object styles
  (chunk 0x1B959).
- LayoutGridDataInformation and StoryGridDataInformation (chunks 0xCD2F,
  0xCD2E): font, font style, sizes, akis, scales, character count size.
- MasterTextFrame and CreatePrimaryTextFrame (chunk 0x59C),
  BlendingSpace (0x1081F), AllowPageShuffle, GuidesShown, snapping,
  GuidesInBack, BaselineGridRelativeOption, ZeroPoint, layout
  adjustment, ShowTextThreads (from version 21.1), AbutTextToTextWrap
  and HonourTextIndentsWithTextWrap.
- Interface colours of guides, grids, margins, columns and the
  pasteboard, with four more named colours (LightGray, GridBlue, Peach,
  Burgundy).
Node::merge no longer adds observed text or children to a typed value,
so a colour list is not merged with an observed colour name.

compare.py --exclude own/:
- trustworthy value coverage 97.42% -> 97.48% (+6,332 values); all
  pairs 95.41% -> 95.46% (+8,463)
- extra values: trustworthy unchanged; all pairs +1 (ShowTextThreads in
  one pair whose IDML is from an earlier version)
- compare.py --all with schemas: 0 schema failures on pairs; files
  without a reference as before
2026-10-08 07:00:27 +00:00
aiongg b8ef0d51d8 feat: decode font name flags, all font types and missing-font records
- FontType codes 0 Type1, 3 ATC and 0xFFFFFFFF Unknown.
- TypekitID from INDD version 9.2 (was 12).
- The key flag of the native style name gives FontStyleNameNative a
  $ID/ prefix; the key flag of the family name gives $ID/ to the family
  name in FontFamily, Font Name and every AppliedFont.
- Missing-font records (kind 3) parse: no fonts, or short font records
  with style, platform name and Unknown type.
- The family name in the font's own script replaces the name when the
  family's writing script is the script byte of the document users.

compare.py --exclude own/:
- trustworthy value coverage 97.36% -> 97.42% (+7,627 values); all
  pairs 95.35% -> 95.41% (+9,808)
- TypekitID +3,483, FontStyleNameNative +1,262, FontType +664, 196
  more Font elements matched, AppliedFont and BulletsFont +235
- extra values 26,348 -> 24,144 (Font elements whose Self now matches)
- warnings 413 -> 406 (missing-font records now parse)
- compare.py --all with schemas: 0 schema failures on pairs; files
  without a reference as before
2026-10-08 06:27:03 +00:00
aiongg 6f27a77b9b refactor: parse the index groups chunk by its layout
Chunk 0x1307E is now read field by field: each group's name, include
byte and header variant, then its header variants and their sections.
This replaces the search for group names. objects.md gives the layout.

diff_outputs.py: 0 differing outputs, 0 changed failures and warnings
over 4,274 files.
2026-10-08 06:10:28 +00:00
aiongg f7fe41e65d feat: write the index, its topics and the page reference topics
The index object (class 0x13004) lists its sections (class 0x13005);
chunk 0x13007 of a section holds its topic records depth first, with
their level, name, sort order and page references. The converter
writes the Index element in designmap.xml with its nested Topic
elements, and gives each PageReference its ReferencedTopic and
PageReferenceType. Evidence: docs/format/index.md.

compare.py --exclude own/:
- trustworthy value coverage 97.17% -> 97.36% (+21,617 values); all
  pairs 95.21% -> 95.35% (+21,808)
- Topic 2,185 of 2,185 with Name and SortOrder; Index 16 of 16;
  PageReference ReferencedTopic and PageReferenceType 7,523 of 7,523
- extra values +15: the 3 topics and 3 page references one INDD has
  and its earlier IDML lacks
- compare.py --all with schemas: 0 schema failures on pairs; files
  without a reference as before. One of them (DOM 19) has topic
  records of another length; its sections are left out with a warning
  (21 warnings).
2026-10-08 06:09:17 +00:00
aiongg 450fd1f5ac feat: decode the index, chapter, dictionary and EPUB preferences
- IndexHeaderSetting (chunk 0x1300E): its five attributes and the whole
  Properties/ListOfIndexHeaderGroup tree.
- IndexOptions (chunk 0x13010): title, title and level styles,
  separators and flags. Styles are matched by name among the styles
  outside groups. Without the chunk no Title is written: it follows the
  exporting application's language.
- ChapterNumberPreference (chunk 0x1A4C4): format, source, number.
- DictionaryPreference Composition: UseDocument with chunk 0x2806,
  Both without.
- EPubExportPreference (DOM 8 on): Version and Id from chunk 0x21A1A;
  without it the observed Id, TocStyleName $ID/, and Version Epub3
  from INDD version 18.1.

compare.py --exclude own/:
- trustworthy value coverage 97.15% -> 97.17% (+2,465 values); all
  pairs 95.19% -> 95.21% (+3,281)
- ListOfIndexHeaderGroup 489 of 489; IndexOptions Title 10 of 10 with
  the chunk; Composition wrong in 5 documents (Both with the chunk);
  TocStyleName with the chunk is not written (13)
- extra values unchanged; no key lost values; 0 failures
- compare.py --all with schemas: 0 schema failures on pairs; the files
  without a reference fail as before (22 conversions, 11 schemas)
2026-10-08 05:45:37 +00:00
aiongg 651cc4604e feat: keep the XML import and dictionary constants in the value files
XMLImportPreference AllowTransform, ImportCALSTables,
ImportTextIntoTables, ImportToSelected, RemoveUnmatchedExisting,
RepeatTextElements and DictionaryPreference MergeUserDictionary are the
same in all 654 pairs. Two of the 683 IDML files differ; both were
exported by an earlier version than their INDD was last saved with, so
they are not pairs. element_values.py now leaves those files out for
these keys (LATER_INDD_EXCEPTIONS); idml-values.md records why.

compare.py --exclude own/: trustworthy value coverage 97.12% -> 97.15%
(+3,423 values, 7 keys x 489 pairs); all pairs 95.16% -> 95.19%
(+4,578). No key lost values; extra values unchanged.
2026-10-08 05:45:37 +00:00
aiongg 78b59b6319 feat: write the spread flattener resolutions without the flattener chunk
Spreads without chunk 0x10833 get LineArtAndTextResolution 300 and
GradientAndMeshResolution 150. No INDD field decides these values:
5,045 of the 5,074 trustworthy spreads without the chunk have them; the
other 29 (all spreads of 9 documents) have 400 and 400. The evidence is
in objects.md and idml-values.md.

compare.py --exclude own/:
- trustworthy value coverage 97.03% -> 97.12% (+10,090 values); all
  pairs 95.07% -> 95.16% (+13,836)
- Spread/FlattenerPreference resolutions: 5,074 missing -> 29 wrong
- extra values: trustworthy unchanged; all pairs +306 (the attributes
  on spreads of stale pairs that their IDML no longer has)
2026-10-08 05:13:20 +00:00
aiongg c6e943e187 feat: decode the whole print settings chunks and booklet margins
Chunks 0xA4C and 0xAF2 are read to their end: device type, paper
offset range, colour output, screening, scaling, tiling and thumbnails,
graphics and font download, printer marks, bleed, slug, profile, ink
screens, flattener preset and the paper size selector. AllPrinterMarks
and PreserveColorNumbers are derived from them. Printer and PPD follow
the corrected rules (built-in empty printer, device-independent PPD
only for PostScript file output). Chunk 0xAF0 gives the four
PrintBookletOption margins. Booklets leave out the attributes their
element does not have.

compare.py --exclude own/ (trustworthy pairs):
- value coverage 96.85% -> 97.03% (+21,049 values; all pairs
  94.89% -> 95.07%, +28,090)
- PrintPreference and PrintBookletPrintPreference: every attribute
  reproduced in all 489 pairs except PreserveColorNumbers (2) and
  MarkType (1); PrintBookletOption margins 11 -> 0 documents wrong
- extra values unchanged; no key lost values; 0 failures
2026-10-08 05:13:12 +00:00
aiongg 261e378d07 feat: write notes, page references, tracked changes and hidden XML markers
- Notes (class 0xA429): their text and a Note element in place of the
  U+FEFF, with dates by the link time rule, user name and document user.
- Index markers (class 0x13006): a PageReference with Self and Id in
  place of the U+FEFF (topics are not written yet).
- Tracked changes: deleted text (class 0xA40A) as Change DeletedText
  before the following character; inserted runs (strand run kind
  0xA466) wrapped in Change InsertedText inside each character range,
  closed at tables, anchored items and XML markers.
- XML markers are left out of the text when the node store cannot be
  read.
- Character-level sources open before an element character (note,
  destination), apart from paragraph destinations, which the schema
  does not allow in them.

Trustworthy pairs: reproduced 10,772,444 -> 11,266,452 (92.60 % ->
96.85 %); stories with identical text 18,574 -> 18,602 of 18,610;
PageReference 7,523 with Id; 1,709 more hyperlink sources and
hyperlinks (in deleted text). Notes 211 of 211 and changes 6,024 of
6,027 match the reference attributes (multisets per pair, no Self).
Extra values 26,107 -> 26,333: 220 from Change elements written inside
sources where InDesign writes an empty Change before them, 9 from
PageReference. HyperlinkTextSource/Note loses 3 elements for the same
reason (the source's first child is now the Change). Schema
validation: 0 failures.
2026-10-08 04:15:15 +00:00
aiongg 76069876a4 feat: write TextFrameFootnoteOptionsObject from DOM 14
From DOM 14 IDML repeats a frame's footnote options as
TextFrameFootnoteOptionsObject on the frames whose TextFramePreference
has FootnotesEnableOverrides. The converter writes it on those frames,
with the values of that preference.

Trustworthy pairs: reproduced 10,650,374 -> 10,669,264 (92.54 % ->
92.71 %); 3,778 elements with all four values, no extra value (26,107).
No key reproduces fewer values. Schema validation: 0 failures.
2026-10-08 03:08:46 +00:00
aiongg 649e80ea6f feat: write footnotes, endnotes and their options
- Footnotes: text owned by a footnote (class 0x24F) is read into the
  footnote and written as a Footnote element in place of its U+0004;
  hyperlink sources inside footnotes are placed like the others.
- FootnoteOption from preferences chunk 0x2820 (styles, numbering,
  marker, spacing, strings, splitting, straddling, both rules); without
  the chunk, the text style and straddling every IDML has. The value
  file reader decodes numeric character references, which fixes the
  SeparatorText tab.
- EndnoteOption from chunk 0x2261E; default marker and text styles
  without it. IsEndnoteStory from DOM 13 instead of 15.
- Endnotes: the endnote story (class 0x2801) is converted, references
  (class 0x2805) become Endnote elements, ranges (class 0x2804, in the
  range tree) EndnoteRange, and its frames EndnoteTextFrame, as InDesign
  writes them.
- compare.py accepts the schema errors of that endnote markup when the
  reference IDML of the pair has the same errors (docs/measurement.md).

Trustworthy pairs: reproduced 10,579,111 -> 10,650,374 (91.92 % ->
92.54 %); stories with identical text 18,565 -> 18,574 (0 missing);
documents >= 99 % 20 -> 21; TextRange +62,059, FootnoteOption and
IsEndnoteStory complete, EndnoteOption +1,356, hyperlinks in footnotes
+271. Extra values 26,179 -> 26,107. No key reproduces fewer values.
Schema validation: 0 failures; 1 pair accepted by the endnote rule; 11
files without a reference fail only on endnote markup.
2026-10-08 02:56:03 +00:00
aiongg 64e5667dfd feat: read hyperlinks of InDesign 5.0 files
InDesign 5.0 destinations have no key, hyperlinks name their destination
by UID, text destination owners use chunk 0x1351E, and text sources are
placed by start and end markers owned by the story (chunk 0x13524)
instead of a range tree. These are read by version; keys are left out
where there are none.

No 5.0 file has an IDML. Over the 17 5.0 files with hyperlinks: 121
hyperlinks, 118 text sources, 7 text destinations and 45 bookmarks are
now written, the 116 warnings about their destination objects are gone,
and every output validates. Pair results unchanged (91.92 %, extra
values 26,516). Schema validation: 0 failures.
2026-10-08 02:22:33 +00:00
aiongg 18449cc430 feat: write external page destinations
Class 0x13552 objects are written as HyperlinkExternalPageDestination:
hidden and key from chunk 0x13580, page index from chunk 0x1B8, view
from chunk 0x13527, and the name IDML derives from the linked
document's file name. DocumentPath is left out (it starts with a volume
name the INDD does not hold). Hyperlinks of kind 2004 and bookmarks now
find them.

Trustworthy pairs: reproduced 10,566,138 -> 10,578,767 (91.81 % ->
91.92 %); 643 of 643 external page destinations with all values but
DocumentPath, 576 more hyperlinks, bookmarks 452 -> 1,027 of 1,027.
Extra values unchanged (26,516); no key reproduces fewer values. Schema
validation: 0 failures.
2026-10-08 02:09:58 +00:00
aiongg dd15594885 feat: write paragraph-level text sources and cross-reference sources
- A text source over several character ranges of one paragraph range is
  written as a child of the ParagraphStyleRange: the ranges at its ends
  are split and the parts inside become its CharacterStyleRange
  children. Sources across paragraph ranges, or holding an XML marker,
  are left out with a warning.
- Text sources with chunk 0x135A0 are written as CrossReferenceSource
  with AppliedFormat. One that holds more than a text variable instance
  is written at paragraph level, since the schema allows no Content in
  it.

Trustworthy pairs: reproduced 10,530,141 -> 10,566,138 (91.50 % ->
91.81 %); HyperlinkTextSource 3,449 -> 5,475, CrossReferenceSource 116,
hyperlinks 3,097 -> 5,149, all with their values. Extra values 26,634 ->
26,516: the 332 values of cross-reference sources written as hyperlink
sources go, and cross-reference sources written around a character
range add their inner ranges (one pair). No key reproduces fewer values.
"spans several text ranges" warnings over all files 9,703 -> 3,277.
Schema validation: 0 failures.
2026-10-08 01:55:40 +00:00
aiongg bd8044dfd9 feat: write text and paragraph destinations in story text
Class 0x13504 objects are read as text or paragraph destinations (kind
in chunk 0x13526). The story's owned items of class 0x1353C name them at
their U+FEFF, which is now written as an empty HyperlinkTextDestination
or ParagraphDestination element instead of the character. Hyperlinks and
bookmarks can point at them.

Trustworthy pairs: reproduced 10,513,964 -> 10,530,141 (91.36 % ->
91.50 %); stories with identical text 18,517 -> 18,565; documents at
>= 99 % 18 -> 20; HyperlinkTextDestination 685 of 685,
ParagraphDestination 119 of 119, bookmarks 373 -> 452. Extra values
unchanged (26,634); no key reproduces fewer values. Schema validation: 0
failures. The 7 text destinations of InDesign 5.0 files (no key) are
left out with a warning, like their page and URL destinations.
2026-10-08 01:39:43 +00:00
aiongg 5da32d1925 feat: write alternative destinations and page item sources
- Text source chunk 0x135B7 is decoded (type, anchor name, index marker,
  page number string, level) and written as AlternativeDestination;
  control characters in the page number become <?AID 00xx?> text.
- Page item sources (class 0x13503) are written as
  HyperlinkPageItemSource when their page item is written, and their
  hyperlinks with them.

Trustworthy pairs: reproduced 10,511,339 -> 10,513,964 (91.36 %);
AlternativeDestination 463 -> 1,048 of 1,048, HyperlinkPageItemSource
136 of 136, 136 more hyperlinks. Extra values unchanged (26,634); no key
reproduces fewer values. Schema validation: 0 failures.
2026-10-08 01:21:35 +00:00