Commit Graph
295 Commits
Author SHA1 Message Date
aiongg 76afa7e239 docs: update the README numbers 2026-10-11 18:38:47 +00:00
aiongg 7fcdc80b61 feat: read IncludeSectionPrefix and name pages and sections by it
Section chunk 0x4C02 stores IncludeSectionPrefix as a u16 after a u16 0.
The section Name is its prefix; pages of a section that includes the
prefix are named with it. The first section starts at its stored start
even when it continues numbering, and the page descriptor writes a
return in the marker as U+2029.
2026-10-11 18:01:55 +00:00
aiongg 867c967a5f docs: update the README numbers 2026-10-11 17:53:52 +00:00
aiongg 1149d01150 feat: write a paragraph range per paragraph-style record
IDML starts a ParagraphStyleRange at every paragraph-style run record,
also between neighbouring records with equal formatting. Document the
rule and the paragraph length list of the records in objects.md, and
the remaining splits inside one record in unknowns.md.
2026-10-11 17:24:08 +00:00
aiongg 29c22fde72 docs: update the README numbers 2026-10-11 17:08:35 +00:00
aiongg 088e636e2b feat: write XML elements around paragraph and character ranges
An element whose start marker and last character lie in different
paragraph ranges wraps the paragraph ranges, which split after its start
marker and at its end marker; one within a paragraph range whose markers
lie in different character ranges wraps character ranges. Start markers
sit in a character range of the parent and end markers in none. Before
version 11 an element with a story as content ends its character range.
Elements are no longer left out when they span paragraphs.
2026-10-11 16:35:42 +00:00
aiongg bff0fd4dc6 docs: update the README numbers 2026-10-11 16:02:14 +00:00
aiongg 86c512235f feat: write the page items of graphic table cells
A grid position's 0x10469 record names a graphic cell object (class
0x1046B, u32 at byte 70 of its first value) whose chunk 0x10466 lists
the cell's page items. Write them inside the Cell like nested items.
2026-10-11 15:29:17 +00:00
aiongg 9b1eb43798 docs: point CLAUDE.md at the moved DesignCraft checkout 2026-10-11 15:01:00 +00:00
aiongg d6b9e3cb6e docs: update the README numbers 2026-10-11 15:01:00 +00:00
aiongg 60928833fe docs: update the InsetSpacing counts 2026-10-11 15:01:00 +00:00
aiongg 35f15b7556 feat: write InsetSpacing as the f64 at 0 of chunk 0x3723 for frames that are not 4-point rectangles
A plain rectangle has exactly four straight points along the axes. Other
frames take the one number from offset 0 of chunk 0x3723, whatever their
four insets, and before DOM 11 always have InsetSpacing.
2026-10-11 14:22:55 +00:00
aiongg 83994d99be docs: update the README numbers 2026-10-11 14:06:50 +00:00
aiongg f941f0fd0e docs: update the InsetSpacing counts 2026-10-11 14:06:50 +00:00
aiongg 3d47ce310b feat: write InsetSpacing as one number for frames that are not plain rectangles 2026-10-11 13:07:02 +00:00
aiongg 0042c35734 docs: update the README numbers 2026-10-11 05:33:25 +00:00
aiongg 0a27d17fce feat: write the transparency of object styles 2026-10-11 04:36:33 +00:00
aiongg 08c6b78b28 docs: update the README numbers 2026-10-11 03:40:36 +00:00
aiongg 5d9dc8f9e1 test: allow a container without contiguous objects 2026-10-11 03:40:36 +00:00
aiongg ea8fe69f34 feat: write the PhotoshopPath and GraphicFrame contour types 2026-10-11 02:49:55 +00:00
aiongg 68308f6f02 docs: update the README numbers 2026-10-11 02:27:02 +00:00
aiongg 8b2597426c feat: write the ContourPathName of contours
Every ContourOption has ContourPathName: `$ID/`, or for an AlphaChannel
contour the first channel name of image chunk 0x177C. Frames take their
graphic's name; frames whose object style turns the text wrap category
on get a ContourOption with only the name. Object styles write `$ID/`.
2026-10-11 01:33:13 +00:00
aiongg dd2b3f07f3 feat: write the custom text wrap path
A wrap path object (class 0x3702) flagged custom in chunk 0x3709 has its
0x162B paths written as TextWrapPreference/Properties/PathGeometry, before
TextWrapOffset.
2026-10-11 01:25:42 +00:00
aiongg bf7c2c4081 docs: list render_compare.py in CLAUDE.md 2026-10-11 01:12:16 +00:00
aiongg 6230fc0cbf docs: update the README numbers 2026-10-11 00:17:19 +00:00
aiongg 483442bd9e fix(compare): give the nested option table its own name
The table of nested option elements reused the name of the table of
story elements that text ranges skip, so text ranges inside footnotes,
table cells and notes were counted as story ranges.
2026-10-10 23:56:54 +00:00
aiongg 2dbec6a041 feat: write the text wrap contour of graphics, frames and object styles
Decode the 29-byte contour chunk (0x373D on placed graphics, 0x373F in
the preferences, 0x3777 on object styles): u32 contour type at 0 and
IncludeInsideEdges at 17. Graphics without the chunk take the
preferences' contour, else SameAsClipping; a frame with one graphic
takes its graphic's contour unless its object style turns the text wrap
category on; object styles without the chunk have SameAsClipping.

Over the trustworthy pairs every ContourType and IncludeInsideEdges the
converter writes equals the reference (graphics 9,535, frames 8,965,
object styles 5,556, preferences 1,251); 14 graphics get a ContourType
the reference leaves out.
2026-10-10 22:36:25 +00:00
aiongg a99131da31 fix(compare): count the ContourOption of text wrap settings
compare.py compared a child element's own children only when the child
had no attributes, so the ContourOption inside TextWrapPreference (which
has attributes) was never counted. It is the only such grandchild outside
stories; count it as Parent/TextWrapPreference/ContourOption, and the
one in Preferences.xml as TextWrapPreference/ContourOption.
2026-10-10 21:58:14 +00:00
aiongg 0c0fb6f996 feat: write GlyphForm ExpertForm and OTFFigureStyle TabularOldstyle 2026-10-10 21:48:04 +00:00
aiongg 08b846f831 feat: name the built-in mojikumi sets 10 and 14
kMojikumiDefaultName10 is LineEndAllOneEmEnum: the BasedOnMojikumiSet of
7 custom mojikumi tables in 7 trustworthy pairs (code 10).
kMojikumiDefaultName14 is LineEndPeriodOneEmEnum: the Mojikumi of one
paragraph range in one trustworthy pair. Codes 3, 5, 6, 9 and 11 have
no pair and stay unwritten (docs/format/unknowns.md).

compare.py --exclude own/: trustworthy values reproduced 28,699,017 ->
28,699,026 (+7 BasedOnMojikumiSet, +2 Mojikumi); coverage 99.484 % ->
99.485 %; extras 92,883 unchanged; no pair loses values.
2026-10-10 21:20:47 +00:00
aiongg 8c4cf32d51 feat: write alternate glyphs (text attribute 0x42AE)
Text attribute 0x42AE (value type 0x42DF) holds the OpenType feature of
a glyph chosen from its alternates: u16 count, then per feature four tag
bytes and a u32 number. IDML writes it as Properties/OpenTypeFeatures,
a list of [$ID/<tag>, number]. The converter writes values with one
feature and leaves out the empty lists that styles store.

Evidence (docs/format/attributes.md, alternate glyphs): 127 of 127 runs
in the trustworthy pairs, 11 tags; tag byte order checked on 7
big-endian files.

compare.py --exclude own/: trustworthy values reproduced 28,698,890 ->
28,699,017 (+127, all CSR.P.OpenTypeFeatures gaps); extras 92,883
unchanged; no pair loses values; 0 conversion failures.
2026-10-10 21:20:38 +00:00
aiongg d46a160c6b docs: update the README numbers 2026-10-10 19:22:07 +00:00
aiongg 5fb4df3857 docs: state only the measured exceptions of the column and watermark rules
Leaves out details of the column direction exceptions and the chunk size that were not measured, and updates the counts of the open items from the current comparison.
2026-10-10 19:09:21 +00:00
aiongg 86f2a4f17a fix: read the TOC entry level as a u16
The u32 read as Level is a u16 level and a u16 that is 0 in every entry
of the pairs and 1 in some entries of files without one, where the u32
gave levels such as 65537 that the schema does not allow. Outputs of the
pairs do not change.
2026-10-10 19:09:21 +00:00
aiongg c87e41bcb8 docs: list the open format items
docs/format/unknowns.md lists what is not decoded after the next-gaps work, with the reason (no field found, never varies, no pair, not explained, not needed) and the page that describes the rest of each item.
2026-10-10 18:34:24 +00:00
aiongg a299387eb0 feat: decode page item, preference and resource values
Page items:
- drop shadow KnockedOut (0x1EB6F), true where neither item nor style
  stores it
- text wrap: the u32 at 40 of 0x3703 is the master page flag and the
  wrap side (two u16, as the big-endian pair shows)
- groups get the fill, stroke and corner values all their children have
- dashes (0x5A35: list and corner code) on items whose stroke type in
  effect is Dashed
- BeforeGroupingLayerPosition (0x324) from 21.4; export SizeType
  (0x1E22B); no generated alt text attributes on groups
- anchored settings on items inside anchored items
- IgnoreWrap false without the chunk, MinimumFirstBaselineOffset 0 by
  the baseline category, InsetSpacing before DOM 11 by the general
  category; EPS contents from 0x8C23
- custom striped and dashed stroke styles (classes 0xB016, 0x5A43) and
  StrokeType code 0x5A42

Preferences: column direction (0x555, else the story orientation),
optical margin size by the text size unit, highlight flags false without
their chunk, grid and gutter defaults and print marks by the edition
code, watermark font by the assignment name, EPUB ViewDocumentAfterExport
for DOM 8 and 9, HTMLFXLExportPreference from DOM 10.1, MathML document
values from DOM 20, index header type by variant name, three more
built-in kinsoku and mojikumi tables.

Resources and text: the saving edition from the XMP black name (tints
of black, unknown users, endnote and index titles, endnote marker), TOC
style entries, PageNumberType (0x1B09), CustomGlyph (0x1B5E), alignment
codes of 0x4266, 0x1B3E and 0xB676, spot colour alias and DOM 8 section
pagination (observed values), text variable settings and result texts,
object styles' AppliedParagraphStyle without a chunk and flex spacing,
stories of frames anchored in the XML backing story.

Fixes: flattener resolutions of 0x10833 were swapped; colour group Self
follows the DOM version.

compare.py, trustworthy pairs: value coverage 98.96 % -> 99.48 %
(+150,288 values), documents >= 99 % 914 -> 1,021, story text 37,129 ->
37,147 of 37,150 (0 missing), extras 92,838 -> 92,883 (+788 from the
KnockedOut and group-corner exceptions, -744 from fixed extras). No pair
loses values against f21267f. diff_outputs.py: 5,743 of 5,765 outputs
differ, no failure or warning differs.
2026-10-10 18:34:10 +00:00
aiongg 6c76bb7451 feat: write tracked changes at the level that holds their extent
Inserted text over several text runs is written as a child of its
paragraph range, or of the story when it spans paragraph ranges, with
the ranges split at its ends, instead of one Change per character range.
A character-level hyperlink source whose whole extent is inserted text
is written inside the Change. Changes over tables, anchored items or
XML markers, and changes that do not nest with a paragraph-level
source, keep the per-range form.

compare.py, trustworthy pairs: 3,960 of the 3,962 reference Change
elements have an equal one in the output (3,969 written);
HyperlinkTextSource/Change extras fall from 220 values in 10 documents
to 0; 13 more text range starts. Only stories with tracked changes or
sources in changes differ in the output.
2026-10-10 18:33:45 +00:00
aiongg f21267f1a8 fix(render_compare): correct line-box offsets per page
The document-wide median per pair of box heights follows pagination
drift on other pages. Each page uses its own median for a pair with at
least 3 matched lines and falls back to the document's median otherwise.
Add --reuse DIR to measure the tool on the renders of an earlier run.
2026-10-10 15:13:03 +00:00
aiongg 3122adfee5 docs: use a generic example of a family name escape 2026-10-10 14:56:50 +00:00
aiongg 0a0c464995 docs: update the README numbers 2026-10-10 14:56:05 +00:00
aiongg dec1d99ce4 feat: write an empty font style as the family's Regular or first style
A text attribute list that stores an empty FontStyle is written with the
style of its AppliedFont family's Regular font, or of the family's first
font if it has no Regular one.

compare.py --exclude own/, trustworthy pairs: ParagraphStyle FontStyle
4,017 -> 4,080/4,086, CharacterStyle 1,310 -> 1,334/1,345, TextDefault
1,240 -> 1,250/1,251, text ranges 76,473 -> 76,619/76,652. No value
that matched before changes; no extra values.
2026-10-10 14:11:48 +00:00
aiongg 129f5b1dca feat: write the transform attributes of object styles
Chunk 0x1E234 of an object style holds four u16 flags and four f64:
X, Y, height and width of TransformAttributeOption. IDML writes each
value whose flag is 1 (36 styles; 2,390 with all flags 0 have none).

compare.py --exclude own/, trustworthy pairs: TransformAttrWidth
0 -> 30/30, TransformAttrHeight 0 -> 11/11, TransformAttrX 0 -> 3/3,
TransformAttrY 0 -> 1/1; no extra values.
2026-10-10 13:23:29 +00:00
aiongg 3ba72f54ce feat: write DesiredGlyphScaling of paragraph ranges
Text attribute 0x1B33 is DesiredGlyphScaling, a fraction written x100.
Paragraph run overrides in 10 documents store 0.96 to 0.99, matching the
22 IDML ParagraphStyleRange values; root styles and text defaults store
1.0 (IDML 100).

compare.py --exclude own/, trustworthy pairs: TextRange
PSR.DesiredGlyphScaling 0 -> 89/89, Cell/ParagraphStyleRange 0 -> 1/1;
no extra values.
2026-10-10 13:23:29 +00:00
aiongg c513cdf03d feat: write the drop shadow colour and ColumnRuleOverride of page items
- Item attribute 0x10837 is the drop shadow EffectColor, written by the
  object style comparison of the other transparency values. Where the
  style stores no swatch, Black is left out: one item has it, and its
  IDML has no EffectColor.
- ColumnRuleOverride is the u32 at 0 of chunk 0x22646 and is written as
  true or false.

compare.py --exclude own/, trustworthy pairs: DropShadowSetting
EffectColor 0 -> 11/11, TextFramePreference ColumnRuleOverride
21,165 -> 21,167/21,167, 4 more Color elements (swatches the shadows
use). No new extra values; no pair loses values.
2026-10-10 12:51:16 +00:00
aiongg bdf62e7646 feat: write GREP and line styles, more nested style codes and composite fonts
- GREP styles (0x1BBA) and line styles (0x1BBB) are decoded and written
  as AllGREPStyles and AllLineStyles on styles, text defaults and
  paragraph ranges; empty lists are not written.
- Nested style delimiters (^c), ^t, ^L, ^p, ^9 and ^S in brackets map to
  Dropcap, Tabs, Repeat, EndNestedStyle, Digits and Sentence.
- An AppliedFont whose family name (with <hhhh> escapes decoded) is a
  composite font's name is written as CompositeFont/<name>, except in
  character styles, which keep the name.

compare.py --exclude own/, trustworthy pairs: ParagraphStyle
AllGREPStyles 152/152, TextDefault 48/48, text ranges 320/320,
AllLineStyles 6/6 and 9/9, ParagraphStyle AllNestedStyles 54 -> 124/124,
TextDefault 4 -> 10/10, ParagraphStyle AppliedFont 4,664 -> 4,803/4,804,
TextDefault 1,232 -> 1,250/1,251, text ranges 84,909 -> 85,587/85,587.
No new extra values; no pair loses values.
2026-10-10 12:51:16 +00:00
aiongg f1a9abeb42 fix: bound the capacity of text segments by the bytes left
A damaged length could reserve up to 8 GB before the segments were read.
Each code unit takes at least one byte, so the capacity is now at most
the bytes left in the value. No output changes.
2026-10-10 12:51:06 +00:00
aiongg 58da17b51e feat: add tools/render_compare.py, a render comparison of same-save triples
For each INDD with a reference IDML and a PDF of the same save, compare
the InDesign PDF (R), DesignCraft's render of the reference IDML (A) and
DesignCraft's render of the converter's IDML (B) page by page. A vs B
measures the converter: pixel-identical documents and pages, and the
first differing page with compare.py's gap keys on it. R vs A measures
DesignCraft: issue kinds counted on font-complete documents and on all.

DesignCraft is an input program (--designcraft or DESIGNCRAFT_CLI).
Needs poppler-utils and the Python standard library only. Documented in
docs/measurement.md, "Render comparison", and in CONTRIBUTING.md.
2026-10-10 11:17:52 +00:00
aiongg 57b84c7687 refactor: share XMP reading in compare.py and add the same-save rule for PDFs
xmp_descriptions and xmp_value hold the packet parsing that modify_date
used. xmp_instances reads the instance IDs, and pdf_stale_reasons
applies the same-save rule for a PDF beside an INDD (source instance in
the INDD history, ModifyDate within an hour, page count, InDesign
creator). compare.py's own results do not change.
2026-10-10 11:17:52 +00:00
aiongg 468b6389e1 docs: update the README numbers
Trustworthy coverage 98.46 % -> 98.96 % (28,546,513 of 28,847,647
values), all pairs 97.28 % -> 97.77 %, documents at 99 % or more 193 ->
901 of 1,251, extra values 93,502 -> 92,838, story text unchanged
(37,129 of 37,150 exact); no pair loses a value, 0 conversion and 0
paired schema failures.
2026-10-10 10:11:37 +00:00
aiongg 03763dafd5 style: format the preference, colour and language code 2026-10-10 09:40:30 +00:00