Commit Graph
168 Commits
Author SHA1 Message Date
aiongg f7fe41e65d feat: write the index, its topics and the page reference topics
The index object (class 0x13004) lists its sections (class 0x13005);
chunk 0x13007 of a section holds its topic records depth first, with
their level, name, sort order and page references. The converter
writes the Index element in designmap.xml with its nested Topic
elements, and gives each PageReference its ReferencedTopic and
PageReferenceType. Evidence: docs/format/index.md.

compare.py --exclude own/:
- trustworthy value coverage 97.17% -> 97.36% (+21,617 values); all
  pairs 95.21% -> 95.35% (+21,808)
- Topic 2,185 of 2,185 with Name and SortOrder; Index 16 of 16;
  PageReference ReferencedTopic and PageReferenceType 7,523 of 7,523
- extra values +15: the 3 topics and 3 page references one INDD has
  and its earlier IDML lacks
- compare.py --all with schemas: 0 schema failures on pairs; files
  without a reference as before. One of them (DOM 19) has topic
  records of another length; its sections are left out with a warning
  (21 warnings).
2026-10-08 06:09:17 +00:00
aiongg 450fd1f5ac feat: decode the index, chapter, dictionary and EPUB preferences
- IndexHeaderSetting (chunk 0x1300E): its five attributes and the whole
  Properties/ListOfIndexHeaderGroup tree.
- IndexOptions (chunk 0x13010): title, title and level styles,
  separators and flags. Styles are matched by name among the styles
  outside groups. Without the chunk no Title is written: it follows the
  exporting application's language.
- ChapterNumberPreference (chunk 0x1A4C4): format, source, number.
- DictionaryPreference Composition: UseDocument with chunk 0x2806,
  Both without.
- EPubExportPreference (DOM 8 on): Version and Id from chunk 0x21A1A;
  without it the observed Id, TocStyleName $ID/, and Version Epub3
  from INDD version 18.1.

compare.py --exclude own/:
- trustworthy value coverage 97.15% -> 97.17% (+2,465 values); all
  pairs 95.19% -> 95.21% (+3,281)
- ListOfIndexHeaderGroup 489 of 489; IndexOptions Title 10 of 10 with
  the chunk; Composition wrong in 5 documents (Both with the chunk);
  TocStyleName with the chunk is not written (13)
- extra values unchanged; no key lost values; 0 failures
- compare.py --all with schemas: 0 schema failures on pairs; the files
  without a reference fail as before (22 conversions, 11 schemas)
2026-10-08 05:45:37 +00:00
aiongg 651cc4604e feat: keep the XML import and dictionary constants in the value files
XMLImportPreference AllowTransform, ImportCALSTables,
ImportTextIntoTables, ImportToSelected, RemoveUnmatchedExisting,
RepeatTextElements and DictionaryPreference MergeUserDictionary are the
same in all 654 pairs. Two of the 683 IDML files differ; both were
exported by an earlier version than their INDD was last saved with, so
they are not pairs. element_values.py now leaves those files out for
these keys (LATER_INDD_EXCEPTIONS); idml-values.md records why.

compare.py --exclude own/: trustworthy value coverage 97.12% -> 97.15%
(+3,423 values, 7 keys x 489 pairs); all pairs 95.16% -> 95.19%
(+4,578). No key lost values; extra values unchanged.
2026-10-08 05:45:37 +00:00
aiongg f6dc474e61 chore: count indentation-only property text as empty in compare.py
InDesign writes some empty Properties children open and closed on two
lines, so their text is a line feed and tabs
(XMLImportPreference/Properties/TransformParameters). compare.py now
treats such text as empty. measurement.md says so.

Effect (measurement only, same output): P.TransformParameters
reproduced in 489 of 489 trustworthy pairs. Trustworthy value coverage
97.12% -> 97.12% (+489 values); all pairs +654.
2026-10-08 05:45:37 +00:00
aiongg 78b59b6319 feat: write the spread flattener resolutions without the flattener chunk
Spreads without chunk 0x10833 get LineArtAndTextResolution 300 and
GradientAndMeshResolution 150. No INDD field decides these values:
5,045 of the 5,074 trustworthy spreads without the chunk have them; the
other 29 (all spreads of 9 documents) have 400 and 400. The evidence is
in objects.md and idml-values.md.

compare.py --exclude own/:
- trustworthy value coverage 97.03% -> 97.12% (+10,090 values); all
  pairs 95.07% -> 95.16% (+13,836)
- Spread/FlattenerPreference resolutions: 5,074 missing -> 29 wrong
- extra values: trustworthy unchanged; all pairs +306 (the attributes
  on spreads of stale pairs that their IDML no longer has)
2026-10-08 05:13:20 +00:00
aiongg c6e943e187 feat: decode the whole print settings chunks and booklet margins
Chunks 0xA4C and 0xAF2 are read to their end: device type, paper
offset range, colour output, screening, scaling, tiling and thumbnails,
graphics and font download, printer marks, bleed, slug, profile, ink
screens, flattener preset and the paper size selector. AllPrinterMarks
and PreserveColorNumbers are derived from them. Printer and PPD follow
the corrected rules (built-in empty printer, device-independent PPD
only for PostScript file output). Chunk 0xAF0 gives the four
PrintBookletOption margins. Booklets leave out the attributes their
element does not have.

compare.py --exclude own/ (trustworthy pairs):
- value coverage 96.85% -> 97.03% (+21,049 values; all pairs
  94.89% -> 95.07%, +28,090)
- PrintPreference and PrintBookletPrintPreference: every attribute
  reproduced in all 489 pairs except PreserveColorNumbers (2) and
  MarkType (1); PrintBookletOption margins 11 -> 0 documents wrong
- extra values unchanged; no key lost values; 0 failures
2026-10-08 05:13:12 +00:00
aiongg 930b7a88cc chore: keep only differing outputs in diff_outputs.py
Each run kept every converted package for both revisions, about 18 GB
over the corpus. Now the outputs of a file are deleted once they compare
equal, each run starts with empty output folders, and a commit's build
directory is deleted after its binary is copied.
2026-10-08 04:55:08 +00:00
aiongg 020d0beb7a docs: state only what the samples show about byte order 2026-10-08 04:36:23 +00:00
aiongg e453d1e8e8 docs: simplify the supported versions table 2026-10-08 04:36:15 +00:00
aiongg 95e74255ba docs: update the README status, coverage tables and numbers 2026-10-08 04:29:31 +00:00
aiongg 4ce9073dfa chore: refresh the README numbers from compare.py's summary
compare.py now also writes target/compare/summary.json: the options of
the run (--all, schema validation, --exclude, --limit, --file), the
files, conversion failures (error messages) and schema validation
failures of the paired and the other files, and the values, reproduced
values and story counts of the trustworthy and of all pairs.

tools/readme_numbers.py rewrites the "Current numbers" table of the
README, now between `<!-- numbers:start -->` and `<!-- numbers:end -->`,
from that summary. It refuses a summary from a run that included own/,
covered part of the corpus, or ran without --all or schema validation.
With --schemas and --jing it first builds the converter and runs
`compare.py --all --exclude own/` with validation, so one command
refreshes the README; --check only reports whether it is current.

The numbers in the README are unchanged.
2026-10-08 04:18:22 +00:00
aiongg c010bedef4 chore: check the WebAssembly build of the library
tools/check_wasm.sh builds the library for wasm32-unknown-unknown,
then builds a probe module (a cdylib generated under the target
directory that exports a function calling `indd::convert`) and checks
with Node.js that the module imports nothing: a conversion needs no
file system, clock or other host function. It then converts the
fetched fixtures, or the files given, in the probe module and compares
each package byte for byte with the native `indd convert` output.

No library change was needed: only `convert_file` and `read_header`
use `std::fs`, and `convert` does not call them. The 5 fixtures and 80
corpus files give the same packages in WebAssembly as natively. The
check takes about 10 s; it is documented in CLAUDE.md, CONTRIBUTING.md,
README.md and ARCHITECTURE.md. There is no hosted CI.

Idea from storytold/designcraft#98.
2026-10-08 04:18:22 +00:00
aiongg 057a4b934d test: convert damaged documents without panics or hangs
`src/hostile.rs` converts damaged inputs and fails if a conversion
panics or takes longer than 20 s:

- Synthetic documents built from a seed with `database::synthetic`: a
  document object with well-formed lists, spreads that hold spread
  layers, pages and page items, and objects of the model's classes with
  chunks of the model's IDs (taken from the hexadecimal constants of the
  model sources). Chunk data is shaped like UID lists, child lists,
  attribute lists, strings and numbers, or random. Each document is
  converted as built, with one object truncated, with bytes flipped, and
  with the file image truncated, through the model and the IDML writer.
- The fixture files, if fetched: truncated at structure boundaries and
  random lengths, and with 1 to 64 bytes flipped, a quarter of the cases
  in the header and master pages.

The cases come from a fixed seed and run on all cores, with a watchdog
for hangs. The default sweep has about 1,600 cases and takes 2 s in a
debug build and 0.5 s in a release build (16 cores).
INDD_HOSTILE_SCALE multiplies the cases and INDD_HOSTILE_SEED changes
the seed; sweeps at scale 40 with seeds 0x1DD and 7 (about 63,000 cases
each, debug build with overflow checks) found no panic or hang.

Idea from storytold/designcraft#98.
2026-10-08 04:18:22 +00:00
aiongg 8b4e4a8e90 refactor: deny unwrap, expect and panic in library and binary code
The library and the binary now deny clippy's unwrap_used, expect_used,
panic, unimplemented, todo and unreachable lints outside unit tests, and
forbid unsafe code. Malformed input must give an error or a warning,
never a panic.

The 36 offending calls are replaced without changing behaviour:

- Byte reads that converted a slice of checked length to an array now
  use the `Encoding::*_at` decoders, `first_chunk`, or a `Cursor`
  helper that returns `Result`; the fallbacks are the existing results
  for data of the wrong length (`Value::Other`, a skipped value, a
  short-data error).
- The fixed-offset readers of the header and the master pages build the
  array with `array::from_fn`; they index as before.
- `Node::set` returns the node it set, so `TextFramePreference` insets
  no longer look it up again with `expect`.
- The embedded value files fall back to no values if one fails to
  parse, and a block with a version bound that is not a number does not
  apply. A new unit test checks that every committed file parses and
  that every bound is a number.
- `Xml::end` without an open element fails a debug assertion and
  writes nothing in a release build.

diff_outputs.py over the corpus (without own/): 0 differing outputs,
0 changed failures, 0 changed warnings.

Idea from storytold/designcraft#98.
2026-10-08 04:18:22 +00:00
aiongg 261e378d07 feat: write notes, page references, tracked changes and hidden XML markers
- Notes (class 0xA429): their text and a Note element in place of the
  U+FEFF, with dates by the link time rule, user name and document user.
- Index markers (class 0x13006): a PageReference with Self and Id in
  place of the U+FEFF (topics are not written yet).
- Tracked changes: deleted text (class 0xA40A) as Change DeletedText
  before the following character; inserted runs (strand run kind
  0xA466) wrapped in Change InsertedText inside each character range,
  closed at tables, anchored items and XML markers.
- XML markers are left out of the text when the node store cannot be
  read.
- Character-level sources open before an element character (note,
  destination), apart from paragraph destinations, which the schema
  does not allow in them.

Trustworthy pairs: reproduced 10,772,444 -> 11,266,452 (92.60 % ->
96.85 %); stories with identical text 18,574 -> 18,602 of 18,610;
PageReference 7,523 with Id; 1,709 more hyperlink sources and
hyperlinks (in deleted text). Notes 211 of 211 and changes 6,024 of
6,027 match the reference attributes (multisets per pair, no Self).
Extra values 26,107 -> 26,333: 220 from Change elements written inside
sources where InDesign writes an empty Change before them, 9 from
PageReference. HyperlinkTextSource/Note loses 3 elements for the same
reason (the source's first child is now the Change). Schema
validation: 0 failures.
2026-10-08 04:15:15 +00:00
aiongg b278998860 fix(compare): count the text ranges of footnotes, table cells and notes
Ranges inside Footnote, Cell and Note elements were not counted: the
element counted as one position of the story text and its own ranges
were skipped. Each now forms a text flow of its own, with offsets from
0, compared when the story's text is identical (it already includes
their text). The headline counts everything InDesign writes.

Trustworthy pairs: reference values 11,508,691 -> 11,632,968,
reproduced 10,669,264 -> 10,772,444, coverage 92.71 % -> 92.60 %; extra
values unchanged (26,107); no key reproduces fewer values.
2026-10-08 03:12:18 +00:00
aiongg 76069876a4 feat: write TextFrameFootnoteOptionsObject from DOM 14
From DOM 14 IDML repeats a frame's footnote options as
TextFrameFootnoteOptionsObject on the frames whose TextFramePreference
has FootnotesEnableOverrides. The converter writes it on those frames,
with the values of that preference.

Trustworthy pairs: reproduced 10,650,374 -> 10,669,264 (92.54 % ->
92.71 %); 3,778 elements with all four values, no extra value (26,107).
No key reproduces fewer values. Schema validation: 0 failures.
2026-10-08 03:08:46 +00:00
aiongg 649e80ea6f feat: write footnotes, endnotes and their options
- Footnotes: text owned by a footnote (class 0x24F) is read into the
  footnote and written as a Footnote element in place of its U+0004;
  hyperlink sources inside footnotes are placed like the others.
- FootnoteOption from preferences chunk 0x2820 (styles, numbering,
  marker, spacing, strings, splitting, straddling, both rules); without
  the chunk, the text style and straddling every IDML has. The value
  file reader decodes numeric character references, which fixes the
  SeparatorText tab.
- EndnoteOption from chunk 0x2261E; default marker and text styles
  without it. IsEndnoteStory from DOM 13 instead of 15.
- Endnotes: the endnote story (class 0x2801) is converted, references
  (class 0x2805) become Endnote elements, ranges (class 0x2804, in the
  range tree) EndnoteRange, and its frames EndnoteTextFrame, as InDesign
  writes them.
- compare.py accepts the schema errors of that endnote markup when the
  reference IDML of the pair has the same errors (docs/measurement.md).

Trustworthy pairs: reproduced 10,579,111 -> 10,650,374 (91.92 % ->
92.54 %); stories with identical text 18,565 -> 18,574 (0 missing);
documents >= 99 % 20 -> 21; TextRange +62,059, FootnoteOption and
IsEndnoteStory complete, EndnoteOption +1,356, hyperlinks in footnotes
+271. Extra values 26,179 -> 26,107. No key reproduces fewer values.
Schema validation: 0 failures; 1 pair accepted by the endnote rule; 11
files without a reference fail only on endnote markup.
2026-10-08 02:56:03 +00:00
aiongg d4f3fd5a64 fix(compare): count no text positions outside character ranges
The walker of the previous commit counted every element outside a
character range, such as StoryPreference, as a text position. A story
whose reference has MetadataPacketPreference, which the converter does
not write, then had all its range offsets shifted by one. Only Content
and elements inside a character range count now.

Trustworthy pairs at HEAD: reproduced 10,578,767 -> 10,579,111 (91.92 %),
extra values 26,516 -> 26,179; reference values unchanged. The numbers
in docs/measurement.md for the measurement change are re-measured with
the corrected rule (10,505,425 reproduced, 26,297 extra values). No key
reproduces fewer values.
2026-10-08 02:41:23 +00:00
aiongg 64e5667dfd feat: read hyperlinks of InDesign 5.0 files
InDesign 5.0 destinations have no key, hyperlinks name their destination
by UID, text destination owners use chunk 0x1351E, and text sources are
placed by start and end markers owned by the story (chunk 0x13524)
instead of a range tree. These are read by version; keys are left out
where there are none.

No 5.0 file has an IDML. Over the 17 5.0 files with hyperlinks: 121
hyperlinks, 118 text sources, 7 text destinations and 45 bookmarks are
now written, the 116 warnings about their destination objects are gone,
and every output validates. Pair results unchanged (91.92 %, extra
values 26,516). Schema validation: 0 failures.
2026-10-08 02:22:33 +00:00
aiongg 18449cc430 feat: write external page destinations
Class 0x13552 objects are written as HyperlinkExternalPageDestination:
hidden and key from chunk 0x13580, page index from chunk 0x1B8, view
from chunk 0x13527, and the name IDML derives from the linked
document's file name. DocumentPath is left out (it starts with a volume
name the INDD does not hold). Hyperlinks of kind 2004 and bookmarks now
find them.

Trustworthy pairs: reproduced 10,566,138 -> 10,578,767 (91.81 % ->
91.92 %); 643 of 643 external page destinations with all values but
DocumentPath, 576 more hyperlinks, bookmarks 452 -> 1,027 of 1,027.
Extra values unchanged (26,516); no key reproduces fewer values. Schema
validation: 0 failures.
2026-10-08 02:09:58 +00:00
aiongg dd15594885 feat: write paragraph-level text sources and cross-reference sources
- A text source over several character ranges of one paragraph range is
  written as a child of the ParagraphStyleRange: the ranges at its ends
  are split and the parts inside become its CharacterStyleRange
  children. Sources across paragraph ranges, or holding an XML marker,
  are left out with a warning.
- Text sources with chunk 0x135A0 are written as CrossReferenceSource
  with AppliedFormat. One that holds more than a text variable instance
  is written at paragraph level, since the schema allows no Content in
  it.

Trustworthy pairs: reproduced 10,530,141 -> 10,566,138 (91.50 % ->
91.81 %); HyperlinkTextSource 3,449 -> 5,475, CrossReferenceSource 116,
hyperlinks 3,097 -> 5,149, all with their values. Extra values 26,634 ->
26,516: the 332 values of cross-reference sources written as hyperlink
sources go, and cross-reference sources written around a character
range add their inner ranges (one pair). No key reproduces fewer values.
"spans several text ranges" warnings over all files 9,703 -> 3,277.
Schema validation: 0 failures.
2026-10-08 01:55:40 +00:00
aiongg bd8044dfd9 feat: write text and paragraph destinations in story text
Class 0x13504 objects are read as text or paragraph destinations (kind
in chunk 0x13526). The story's owned items of class 0x1353C name them at
their U+FEFF, which is now written as an empty HyperlinkTextDestination
or ParagraphDestination element instead of the character. Hyperlinks and
bookmarks can point at them.

Trustworthy pairs: reproduced 10,513,964 -> 10,530,141 (91.36 % ->
91.50 %); stories with identical text 18,517 -> 18,565; documents at
>= 99 % 18 -> 20; HyperlinkTextDestination 685 of 685,
ParagraphDestination 119 of 119, bookmarks 373 -> 452. Extra values
unchanged (26,634); no key reproduces fewer values. Schema validation: 0
failures. The 7 text destinations of InDesign 5.0 files (no key) are
left out with a warning, like their page and URL destinations.
2026-10-08 01:39:43 +00:00
aiongg 5da32d1925 feat: write alternative destinations and page item sources
- Text source chunk 0x135B7 is decoded (type, anchor name, index marker,
  page number string, level) and written as AlternativeDestination;
  control characters in the page number become <?AID 00xx?> text.
- Page item sources (class 0x13503) are written as
  HyperlinkPageItemSource when their page item is written, and their
  hyperlinks with them.

Trustworthy pairs: reproduced 10,511,339 -> 10,513,964 (91.36 %);
AlternativeDestination 463 -> 1,048 of 1,048, HyperlinkPageItemSource
136 of 136, 136 more hyperlinks. Extra values unchanged (26,634); no key
reproduces fewer values. Schema validation: 0 failures.
2026-10-08 01:21:35 +00:00
aiongg 3ac6275173 feat: write hyperlink appearance, page destination bounds and other destinations
- Hyperlink: Visible, Width, BorderStyle and BorderColor whenever chunk
  0x13553 has the corpus pattern, Highlight from its byte 10, no
  BorderColor for hyperlinks to a page of another document.
- Hyperlink destination: left out for hyperlinks into another document
  (chunk 0x1359F) instead of a wrong local destination; `n` for kind
  2000 without a destination; hyperlinks without a source are left out.
- Page destination: ViewBounds from chunk 0x13527 and NameManually.

Trustworthy pairs: reproduced 10,505,081 -> 10,511,339 (91.28 % ->
91.33 %); 22 more hyperlinks, Hyperlink appearance keys 1,982 -> 2,878
of 2,878, page destination ViewBounds and NameManually 831 of 831,
Destination wrong 0. Extra values unchanged (26,634); no key reproduces
fewer values. Schema validation: 0 failures.
2026-10-08 01:06:17 +00:00
aiongg 52a32e9b85 fix(compare): count text ranges inside wrapper elements
text_ranges read only ParagraphStyleRange children of the story and their
CharacterStyleRange children. Ranges inside HyperlinkTextSource,
CrossReferenceSource, XMLElement, Change and EndnoteRange were skipped,
and their text was left out of the offsets, so every later range of the
story started earlier in the reference than in the output. It now walks
through those wrappers at any depth.

Trustworthy pairs (compare.py --exclude own/):
- hyperlink and cross-reference sources alone: reference values
  11,142,835 -> 11,158,348, reproduced 10,501,458 -> 10,505,081,
  extra values 30,237 -> 26,634
- with XMLElement, Change and EndnoteRange (counted as the headline must
  include everything InDesign writes): reference values 11,508,691,
  coverage 94.24 % -> 91.28 %; 348,306 of the 350,343 new values are in
  one document
- story text unchanged (18,517 of 18,610); documents >= 99 %: 18
- three keys reproduce one value fewer (CSR.Underline, PSR.Justification,
  PSR.P.TabList): ranges that matched at a shifted offset before
2026-10-08 00:51:12 +00:00
aiongg adc397f871 feat: write cell and table style groups
Groups (class 0xCA8B) listed by the root cell or table style group are
written as CellStyleGroup or TableStyleGroup elements holding their
styles, in the order of the group's children (objects.md).

compare.py --exclude own/, trustworthy pairs: CellStyleGroup elements
0 -> 2 of 2 (4 values); no other change, no new extras, no conversion
or schema failures.
2026-10-08 00:32:07 +00:00
aiongg 01113d1c21 feat: write the table, table style and root table style attributes
New table and table style attributes: stroke and fill counts, end row
and start/end column stroke colours, weights, types and tints, column
fill colours and tints, skip counts, graphic insets and
ClipContentToGraphicCell; on tables only LeftInset etc. and
BreakHeaders; on styles only the footer region style and the region
flags. Groups whose IDs are always equal (border weights, colours,
types, tints and gap colours; StartColumnStrokeType/EndColumnLineStyle;
SkipFirstHeader/SkipLastFooter) are written only when all IDs agree.
Table stroke types are decoded as two words. The root table style takes
its INDD values where the observed root values have the attribute, and
from DOM 11 its text and graphic cell values. A cell style without its
own paragraph style gets [No paragraph style] (tables.md).

compare.py --exclude own/, trustworthy pairs: value coverage 94.23 % ->
94.24 %; every Table key above is now right on every table, TableStyle
Text*/Graphic*Inset and clip flags 341-348 -> 375 of 375, border and
region keys up, CellStyle AppliedParagraphStyle 507 -> 547 of 547. No
new extras in trustworthy pairs; 98 in a stale pair's table that its
reference lacks.
2026-10-08 00:18:50 +00:00
aiongg 2319223d3a feat: write cell edges along merged cells and the other cell attributes
An edge value is written when every grid position along that edge has
it with the same value; the priority comes from the cell's own set, or
is 1 for the right (bottom) edge of a merged cell whose edge positions
have one. New edge groups: gap tint, gap colour, overprint and gap
overprint. Cell edge tints -1 are written as 100 and a weight is 0 when
the edge colour is None. Stroke codes 0x5A3F (Japanese Dots) and 0xB007
(ThickThick) for cells and tables. New cell attributes: graphic insets,
OverprintFill, VerticalJustification 0 and 3, FirstBaselineOffset,
WritingDirection (not on cell styles: the schema lacks it),
RotationAngle, DiagonalLineStrokeOverprint. Text*Inset only from DOM 11.
Cell style edge IDs are rotated against the cell IDs (tables.md).

compare.py --exclude own/, trustworthy pairs: value coverage 93.13 % ->
94.23 %, median 96.7 % -> 96.8 %, documents >= 99 % 14 -> 18; extra
values 30,379 -> 30,237. Every Cell edge key is now right on every cell
except one bottom weight (tables.md); CellStyle edge keys rise too.
One unnamed colour (14 values) is no longer written: its only reference
was a wrong BottomEdgeStrokeColor that is now right, and IDML writes the
colour though nothing refers to it (objects.md). In stale pairs the
extras of a table missing from its reference grow by 2,590.
2026-10-07 23:49:40 +00:00
aiongg 32749c0bb5 feat: resolve cell spans and the text cell values in effect
Cells start at positions with attribute 0xB666 (text) or 0x10469
(graphic); positions past a short row record are covered. The layout
record is parsed by its flags, so the width is read where it is, and
cells never laid out are kept. Spans follow a two-pass rule: the column
span whose widths best match the layout width plus the insets in effect,
then the free covered rows below. From DOM 11 every Table, Row, Column
and Cell gets TextTop/Left/Bottom/RightInset and ClipContentToTextCell
in effect: cell set, cell style chain, table-style region cell style,
table, table style chain, root table style; rows and columns copy their
first cell (tables.md).

compare.py --exclude own/, trustworthy pairs: value coverage 92.51 % ->
93.13 %, median 96.5 % -> 96.7 %; story text 18,509 -> 18,517. All
Cell, Row, Column and Table text insets and ClipContentToTextCell are
now right (Cell 11,097 of 11,097 each), Cell ColumnSpan 11,304 -> 11,550
and RowSpan 11,389 -> 11,550 of 11,550; 144 overset cells are no longer
dropped. No new extras in trustworthy pairs; the extras of one stale
pair's table that its reference lacks grow by 1,052.
2026-10-07 23:16:50 +00:00
aiongg e9a6a3cdf7 feat: write table direction and the row group attributes
Chunk 0x50F65 gives TableDirection (written on every table); the row
group attributes 0xB69F, 0x10407 and 0xB6A1 give AutoGrow (now also
true), StartRow and KeepWithNextRow (tables.md).

compare.py --exclude own/, trustworthy pairs: value coverage 92.50 % ->
92.51 %; Table TableDirection 0 -> 326 of 326, Row AutoGrow 63 -> 1,128
of 1,128, StartRow 10 of 10, KeepWithNextRow 8 of 8. No new extras in
trustworthy pairs; 7 in one stale pair's table that the reference lacks.
2026-10-07 23:00:13 +00:00
aiongg 758a12a92e docs: list the applied object style module in ARCHITECTURE.md 2026-10-07 22:34:22 +00:00
aiongg 5fc388f409 style: format grouped_name 2026-10-07 22:34:16 +00:00
aiongg eacc1be120 fix: name grouped styles with their group path and escape colons
A style's IDML Name is its group path and its own name joined by ":",
with a ":" inside a name written "\:" and "$ID/" before the whole path
of a built-in style; Self is that name escaped (objects.md, style
groups). Object and cell styles now include their group path too, and
every reference uses the same name. A font with an empty style name is
named "<family> Regular" with the family alone in Self (fonts.md).

compare.py, trustworthy pairs: extra values 33,181 -> 30,379 (0.27 %);
value coverage 92.45 % -> 92.50 % (10,307,358 values; grouped style
names, styles now matched); no value lost; 14 documents >= 99 %,
median 96.5 %. No conversion or schema failures; story text
18,509/18,610.
2026-10-07 22:34:08 +00:00
aiongg 8b0f46837b feat: write unnamed colours and gradients only when something refers to them
IDML writes a colour or gradient without a name only if a value in the
package refers to it, a gradient stop or tint uses the colour, or the
page item defaults name it in one of the two entry tables of chunk
0x6E07 (objects.md, colours; preferences.md). The graphic part is now
written last, from the references in the other parts; the second table
of chunk 0x6E07 is read.

compare.py, trustworthy pairs: extra values 155,956 -> 33,181 (unnamed
Color -114,734, Gradient -3,498, GradientStop -4,543). 463 values are
no longer reproduced: 10 unnamed colours and 22 unnamed gradients with
their stops that IDML writes though nothing refers to them (open, see
objects.md). Value coverage 92.46 % -> 92.45 %. No conversion or schema
failures; story text 18,509/18,610.

The previous commit (09e87fa), measured alone: extra values 635,511 ->
155,956, value coverage 92.41 % -> 92.46 %; 39 values lost (7 opacity
stops and 8 drop shadows IDML writes with values equal to the style,
5 StrokeWeight, 1 TextColumnFixedWidth).
2026-10-07 22:33:58 +00:00
aiongg 09e87fadea feat: write page item values only where they differ from the object style
IDML leaves out the values of a page item that equal those of its
applied object style. The writer now compares with the style's
effective values (its own, then the styles it is based on, then [None]):

- TextFramePreference: an attribute is written when its category is off
  in the style's category list (chunk 0x1B92E, also for [None]) or its
  value differs; TextColumnMaxWidth (DOM 8), TextColumnCount (DOM 10)
  and InsetSpacing (DOM 11) are always written; footnote values from
  13.1 (objects.md, text frame preferences).
- Fill, stroke, miter and corner attributes (attributes.md).
- Transparency effects: only differing values, and an effect element
  only when it has one or its opacity stops differ (transparency.md).

New decoding: object style chunk 0x1B924 fields for balance, fixed
width, justification, first baseline offset (code 4 = XHeight, also for
frames) and auto-sizing; chunk 0x1B92C (style transparency). Object
styles get the DOM 12 text frame values from DOM 8 and
FootnotesEnableOverrides from 13.1 (idml-values.md).
2026-10-07 22:19:20 +00:00
aiongg 17c6ace935 feat: leave out values IDML does not write (layers, root styles, version gates)
Rules from the corpus, each documented in docs/format/:

- ItemLayer only on items directly under a spread (objects.md).
- The root object style [None] gets no Enable… attributes and no effects
  category settings; EnableTextFrameColumnRuleOptions from 15.1, and the
  footnote values of an object style's TextFramePreference from 13.1.
- The root paragraph style has no NextStyle and no AllNestedStyles;
  TextDefault has AllNestedStyles only when the list has items; the
  empty TabList is written from DOM 8 (attributes.md).
- DOM 7 has no EPubExportPreference or HTMLExportPreference.
- Row and column Self use a hexadecimal index (tables.md).
- Cell CellType from DOM 11; Document Name from DOM 13; ColorGroup Self
  is the UID before 11.3.
- The internal layer is the first of the document's layer list, whatever
  its name.
- A CR in a name is %0d in Self (colours, gradients, tints, hyperlink
  destinations, styles).

compare.py, trustworthy pairs (after the compare fix of 48d184f):
extra values 705,523 -> 635,511; value coverage 92.25 % -> 92.41 %
(10,296,599 values); 2 ItemLayer values lost (the 2 grouped items whose
IDML has one); 12 documents >= 99 %. No conversion or schema failures.
2026-10-07 22:01:28 +00:00
aiongg 48d184f225 fix(compare): match fonts whose family has the technology suffix
IDML adds " (OTF)", " (TT)" or " (T1)" to some family names depending
on the exporting computer. compare.py removed it from family names but
not from Font Self keys and Font Name, so each such font counted as
missing and the converter's font as extra. The suffix is now removed
from the family part of both.

Trustworthy pairs: extra Font values 17,889 -> 5,146 (-12,743); the
same fonts now count as reproduced.
2026-10-07 21:53:14 +00:00
aiongg 14b36ee317 docs: list the graphic, export option and XMP modules in ARCHITECTURE.md 2026-10-07 21:26:39 +00:00
aiongg 6511128e7f feat: write per-corner attributes and paragraph border and shading corners
Page items, object styles and PageItemDefault: each corner radius and
corner option from its own attribute ID (top left 0x6E70/0x6E6F, top
right 0x6E94/0x6E91, bottom left 0x6E92/0x6E8F, bottom right
0x6E93/0x6E90), with the corner codes InsetCorner, BevelCorner and
FancyCorner. Object styles no longer need all four corners equal.

Paragraph styles and text: ParagraphShadingTopLeft/BottomLeftCornerRadius
(0x1DF12, 0x1DF14); the tied groups of border corner radii, shading
right corner radii and shading corner options when their IDs agree; the
shading and border origin pairs for the known code pairs.

compare.py --all --exclude own/ (trustworthy pairs): value coverage
91.61% -> 92.25% (+71,545 values); documents >= 99 %: 5 -> 11; median
95.7% -> 96.3%; no key lower. Extra values 709,654 -> 718,266
(+8,612): the per-corner attributes follow the presence of CornerRadius
and CornerOption in the item's list, so each of the existing 2,055
extra CornerRadius values (items whose IDML leaves the value to the
object style) now has four per-corner extras, and likewise the
CornerOption extras. 0 conversion and 0 schema failures.
2026-10-07 21:26:29 +00:00
aiongg 5b7e0ecf1f feat: write link attributes and placed graphic settings
Links: LinkResourceModified, ShowInUI, LinkImportStamp and the two link
times from chunk 0x8C9B, PDFIdentifier from chunk 0x1B6, the size and
format name from the resource's chunk 0x8C92, and the values every link
has. Link times are written in the time zone of the XMP date nearest in
day of the year (the INDD has no time zone; compare.py does not score
them): 3,886 of 4,446 import times and 3,316 of 4,434 modification
times right.

Images, PDF, EPS and SVG graphics: the page item settings and object
style, Space, ActualPpi and EffectivePpi (chunk 0x1708 and the scale on
the spread), ImageTypeName, Profile (chunk 0x7C0F), fill colour and
gradients, vector colour policies (chunk 0x7C42 or the document's
policies), PDFAttribute (chunk 0x251B), GraphicLayerOption with the
GraphicLayer tree (chunk 0x177A) and LayerCompOption (chunk 0x9209),
and ImageIOPreference from chunk 0x1714 in place of chunk 0x8C39.
The placed graphic writer moves to idml/graphic.rs.

compare.py --all --exclude own/ (trustworthy pairs): value coverage
90.00% -> 91.61% (+179,834 values); documents >= 99 %: 1 -> 5; median
94.2% -> 95.7%; no key lower. Extra values 709,643 -> 709,654 (+11 in
SVG elements that are not in the reference). New warnings: 178 layer
chunks in 3 InDesign 3.0 files, whose layout differs, are left out.
0 conversion and 0 schema failures.
2026-10-07 21:13:31 +00:00
aiongg 9de24aa28f feat: read object export options of page items (chunk 0x1E206)
ObjectExportOption now takes its alternative text, actual text and
tagging values from chunk 0x1E206 of the item, and its other attributes
from a table by INDD major and minor version (8.0-9.1, 9.2, 10.0, 10.1
on, 21.0, 21.1, 21.4), for page items from version 8 and for every
object style, [None] included, from version 9. Page items of version 7
get no ObjectExportOption, as in their IDML.

compare.py --all --exclude own/ (trustworthy pairs): value coverage
88.72% -> 90.00% (+142,324 values); documents >= 99 %: 0 -> 1; median
92.4% -> 94.2%; no key lower. Extra values 713,688 -> 709,643: -4,319
from the DOM 7 rectangles that no longer get the element; +274 more
values in GraphicLine, Oval and TextFrame elements that are not in the
reference (more attributes per element). 0 conversion and 0 schema
failures.
2026-10-07 20:51:44 +00:00
aiongg 50638a4760 fix: write 17-digit numbers rounded to the nearest, ties to even
IDML writes the shortest decimal that reads back as the same value; when
that needs 17 digits it takes the 17-digit decimal nearest to the exact
value, with ties to even (150.23599243164062). Rust's shortest form can
end in the other digit (…063). num() now re-rounds 17-digit output with
{:.16e}.

compare.py --all --exclude own/ (trustworthy pairs): value coverage
88.72% (9,885,604 -> 9,885,761 values); PDF P.GraphicBounds +148,
EPS P.GraphicBounds +9; no key lower; extra values unchanged (713,688);
0 conversion and 0 schema failures.
2026-10-07 20:37:04 +00:00
aiongg ae5a44db3a docs: add ARCHITECTURE.md and CONTRIBUTING.md; bring the README up to date
The README listed implemented features as not started, an old attribute
count, corpus size and numbers, and no library use. The numbers are from
compare.py --all with schema validation at this revision.
2026-10-07 20:07:28 +00:00
aiongg f314de906d docs: fix stale attribute notes and misplaced doc comments 2026-10-07 20:00:23 +00:00
aiongg bb6fb3ef61 test: move writer tests into their modules; unit tests for tables and CJK objects
Tables and composite fonts, entries and kinsoku tables are tested on
synthetic objects, including big-endian composite font entries.
2026-10-07 19:59:06 +00:00
aiongg ece97ec5f1 fix: leave out an unreadable story, spread or page item with a warning
One story, spread or page item that cannot be read no longer fails the
whole document; it is left out with a warning, like other objects. An
attribute list that cannot be parsed gives a warning instead of being
dropped silently. Corpus output is unchanged; on 400 mutated copies of a
fixture, 28 conversions fail instead of 33 (the rest are damaged
database structures).
2026-10-07 19:57:10 +00:00
aiongg 206d1c157d refactor: keep start-tag attributes until the tag closes, so none is written twice
Values set with Xml::attr now replace observed values whatever the order
of the calls; the writer no longer has to write INDD values before
attrs_missing.
2026-10-07 19:54:14 +00:00
aiongg d205f6c851 fix: one corner option map for page items, defaults and object styles
Page items and page item defaults knew only code 0x5A15; object styles
also knew 0 (None) and 0x5A16 (InverseRoundedCorner). Using the object
style codes everywhere reproduces 486 more reference values in the
trustworthy pairs (489 of 489 page item defaults, 4 rectangles, 2 text
frames) and adds no extra values. Object styles write CornerOption in
the page item attribute order.
2026-10-07 19:50:06 +00:00
aiongg 1a88b36ac8 refactor: one Name type and builtin_key for built-in $ID/ names 2026-10-07 19:38:25 +00:00