Commit Graph
71 Commits
Author SHA1 Message Date
aiongg 27cfef8fd7 feat: write named grids and object style export options
designmap.xml gets the document's named grids with the layout grid
settings all its pages share. Object styles other than [None] get the
export options every IDML of their version has, now also before
InDesign 12.

Value coverage on trustworthy pairs: 86.72 % to 86.83 %.
2026-10-07 13:29:58 +00:00
aiongg 5344dc2148 fix: tell polygons from ovals and rectangles by the stored shape code
Chunk 0x6204 of a frame holds a shape code that gives the IDML element
(line, rectangle, oval or polygon). Without a code, a closed four-point
path is an oval only when all its points are of type 0. About 5,000
shapes of the trustworthy pairs that were written as ovals are polygons,
with all their attributes and children.

Value coverage on trustworthy pairs: 84.50 % to 86.72 %.
2026-10-07 13:11:59 +00:00
aiongg 2d52b399c4 feat: write table of contents styles and the stories they made
Styles.xml gets a TOCStyle for each TOC style object, with its name,
title, title style and numbering, anchor and line break settings, and
stories get AppliedTOCStyle through the table of contents object that
made them. Entries are not decoded yet.

Value coverage on trustworthy pairs: 84.29 % to 84.50 %.
2026-10-07 12:41:09 +00:00
aiongg e6dd2d629f feat: write style import flags, unique IDs and common style values
Paragraph and character styles get Imported (the u16 after the style
kind) and StyleUniqueId (the GUID after the name) from the INDD, and the
values every non-root style has in IDML, such as PreviewColor, from
observation. The values generator leaves root styles out of the style
paths.

Value coverage on trustworthy pairs: 84.07 % to 84.29 %.
2026-10-07 12:14:48 +00:00
aiongg 463fef540a feat: write swatch library, alternate colour and guide settings
Colours get SwatchCreatorID and their alternate colour from the INDD,
HSB colours are written as RGB with ConvertToHsb, and every swatch names
its colour group swatch from DOM 12 on. Guides get Locked, GuideZone,
OverriddenPageItemProps and PageIndex, which counts pages from the
spine. Tints, gradients and the None swatch get the values every IDML
has on them.

Value coverage on trustworthy pairs: 83.37 % to 84.07 %.
2026-10-07 11:54:33 +00:00
aiongg 09782f6af0 feat: write more text frame settings
TextFramePreference gets the maximum column width, fixed column width,
auto-sizing minimums, text wrap override, column rule width and colour,
footnote spacings and the inset spacing (from the frame), read from
their chunks. Attributes that IDML writes only on some frames, with one
value wherever it does, are generated as WhenWritten values and also
give the versions from which column rules and footnotes exist.

Value coverage on trustworthy pairs: 82.03 % to 83.37 %.
2026-10-07 11:34:20 +00:00
aiongg 722fddfe97 feat: write languages, document users and document settings
designmap.xml gets the document's languages from their INDD objects,
with the quotes every IDML gives each language, the document users from
the document object, and the settings elements every IDML of the version
has (watermark, linked story and page item options, footnote and
endnote options and others) with their common values. User colours are
set by the exporting InDesign, not the INDD, and are left out of value
coverage.

Value coverage on trustworthy pairs: 81.65 % to 82.03 %.
2026-10-07 11:08:03 +00:00
aiongg fbe1e7f6ca feat: write spread, page, section and layer settings
Spreads get AllowPageShuffle and their FlattenerPreference, master
spreads ShowMasterItems, pages their tab order, overrides, layout grid
use, layout rule, colour, alternate layout and numbering descriptor,
sections their prefix, marker and alternate layout, and layers their
printing, guide, UI, wrap and colour settings. Interface colours are
named from one table shared with XML tags. Values every IDML has on
these elements, and the two built-in trap presets, are written from
observation.

Value coverage on trustworthy pairs: 80.03 % to 81.65 %.
2026-10-07 10:45:11 +00:00
aiongg 3d968bea8a feat: write the settings every page item has
Page items get Name, Visible, Locked, the interface change counts,
OverriddenPageItemProps, the layout constraints and the gradient angles
and highlights, read from their chunks and attribute lists, with the IDML
defaults where a chunk is absent. Groups take the gradient values their
children share. tools/element_values.py collects the values every IDML
of a version has on an element; the converter writes those of page items
and their export options, now for every version.

Value coverage on trustworthy pairs: 62.62 % to 80.03 %.
2026-10-07 10:09:02 +00:00
aiongg c9528b10ff feat(tools): list reproduced values per key and dump whole chunks
compare.py writes values.tsv and values-all.tsv with every key's values and
how many are reproduced, so a change can be checked for keys that lose
reproduced values. indd dump --full prints every byte of each chunk.
root_values.py reads the current inventory layout and keeps to the pairs
its generated files were made from.
2026-10-07 09:39:02 +00:00
aiongg f5c73f9460 fix: convert or clearly reject every file in the fetched sample batch
- InDesign 5.0: read document chunk 0x13501 with the 3.0/4.0 layout
  (four UID lists); the 6.0 layout failed on 17 files.
- Reject files that are shorter than their database pages as truncated,
  and check the signature before the size so that short files of other
  kinds are reported as not INDD files. The corpus tests accept both.
- Hyperlink page destinations: write view setting 0 as Fixed (567 of 567
  in the pairs); leave out a zoom outside the schema's 5-4000 %.
- Leave out, with a warning, values the IDML schema does not allow: ink
  neutral density -1, writing script 0xFFFFFFFF, negative drop shadow
  size and other out-of-range transparency values. The IDML writer now
  returns its own warnings.
2026-10-07 07:59:23 +00:00
aiongg 3e89cad3bd fix: convert or clearly reject every file in the new sample batch
- Bullet list: the font style can be followed by 1 and four bytes.
- InDesign 2.0: in-object strings start with 1, not 2.
- InDesign 2.0-5.0: XML tag names are a flag byte and a string, in
  either byte order.
- InDesign 3.0 and 4.0: the document's hyperlink lists start at offset
  0 and include page destinations.
- XMP: the packet can be shorter than its object; take its length.
- InDesign 1.x files (no signature, DOCUMENT at 0x5C) are reported as
  not supported; files whose page directory lies beyond the database
  pages as having no object database.
- Schema: leave out hyperlink sources that hold an anchored object, and
  the characters U+FFFE and U+FFFF, which XML does not allow.

tests/corpus.rs accepts the two rejections above and skips the paths
listed in the local corpus/exclude.txt.
2026-10-07 06:27:26 +00:00
aiongg bf72b93d23 test: download fixtures from pinned sources instead of committing them
Remove the sample documents, PDF and licence copies from tests/fixtures/.
tests/fixtures/manifest.json lists each file's download URL (pinned to a
commit for GitHub sources), size and SHA-256. tools/fetch_fixtures.py
downloads them into the git-ignored tests/fixtures/files/ and verifies
them. The fixture tests read from there and skip when it is absent.
2026-10-07 05:47:17 +00:00
aiongg e21d73aa0a docs: never commit downloaded sample files 2026-10-07 05:35:59 +00:00
aiongg fe0a20fc0e feat: convert tate-chu-yoko (Tatechuyoko) 2026-10-07 05:34:59 +00:00
aiongg 24c959bf94 feat: convert manual kerning (KerningValue) 2026-10-07 05:29:41 +00:00
aiongg 790d22d483 feat: write first baseline offset code 3 as EmboxHeight 2026-10-07 05:17:22 +00:00
aiongg 4bf1094489 feat: convert ruby text
Character runs that set attribute 0x422D to 1 carry their ruby text in
0x422E (u32 length, then text segments). In a vertically typeset
sample's print PDF that text is set small beside the run's characters.
The converter writes RubyFlag and RubyString.
2026-10-07 04:31:04 +00:00
aiongg 4bc6768fb0 feat: write vertical story orientation
A text frame's multi-column frame stores a matrix (chunk 0x2DE) that is
the identity for horizontal text and a quarter turn for vertical text.
A story whose frames all have the quarter turn is written Vertical
instead of the Horizontal every sample's IDML has.
2026-10-07 04:27:07 +00:00
aiongg 6a357f4287 feat: write the Kanji page number style
Section style code 0x4C12 is Kanji, shown by the Chinese-numeral folios
of a vertically typeset sample's print PDF. Pages in such a section are
named with Chinese digits, as the folios show.
2026-10-07 04:26:16 +00:00
aiongg 9fef3b0516 feat: read the page binding
DocumentPreference PageBinding comes from the document preferences
(u16 at 64) instead of the value every sample's IDML has. A vertically
typeset sample bound on the right stores 1, RightToLeft.
2026-10-07 04:25:39 +00:00
aiongg 7c89bdfe8a fix: write only colour models the samples show
Code 1 is Spot in two pairs; code 3 has no sample, so its Model is now
left out with a warning instead of written as MixedInkModel.
2026-10-07 04:15:57 +00:00
aiongg a9985b3775 fix: validate every corpus output and fix the two schema failures
compare.py --all now also converts the INDD and INDT files that have no
same-version IDML (any version, either byte order), validates them and
counts converter warnings by kind. validate.sh runs Jing once per schema
instead of once per file, which makes validating the whole corpus take
about 3 minutes.

Two schema failures showed up in files without an IDML:

- Gradient swatch midpoints are stored as positions from the start of
  the gradient. IDML wants them relative to the gap between two stops,
  as for opacity gradient stops. Values outside 13-87 are left out with
  a warning.
- The style kind is a u16, not a u32. One InDesign 7.5 template has a
  non-zero u16 after it, which made 151 character styles paragraph
  styles.
2026-10-07 03:50:08 +00:00
aiongg 1d0e27db32 feat: convert big-endian files from InDesign 3.0 and 4.0
The byte order flag applies to object data only: database pages stay
little-endian. Object data is decoded in the file's byte order, and in
big-endian data the string tag and the flag byte before it are swapped.

The two big-endian samples are also the only files from before
InDesign 7.0, and some of their layouts differ: page transform and
bounds chunks, font records, XML tag names and nodes, story strands and
40-byte guide records are read; text frame settings, document
preferences, a composite font and a section style are left out with a
warning. The 40-byte guides of an InDesign 7.0 file are now read too.

docs/format/big-endian.md records the evidence. The corpus tests now
cover all byte orders.
2026-10-07 03:11:04 +00:00
aiongg 496c3e5d65 feat: convert the XML structure
Read the XML nodes stored with each story, place their markers in the
story text and write the backing story as XmlStory, with the elements
in it and in the stories they tag. Element Self names follow the node
IDs from the root down. The backing story is added to StoryList.
2026-10-07 02:42:17 +00:00
aiongg b0e0d7f4f6 docs: record why the font family technology suffix is not predicted 2026-10-07 02:27:40 +00:00
aiongg 56b59e1319 fix: count story positions in characters, not UTF-16 units
Text records count UTF-16 code units, but style runs, owned items and
text owners count a surrogate pair as one position. Convert those
positions to UTF-16 offsets so characters outside the Basic Multilingual
Plane keep their formatting and are not split into two replacement
characters.
2026-10-07 02:27:40 +00:00
aiongg 201697c779 feat: convert XML tags
XML tags (class 0xBF19) hold their name in chunk 0xBF2F and refer to an
interface colour in chunk 0x117. Write them in XML/Tags.xml, sorted by
name as IDML lists them.
2026-10-07 01:44:20 +00:00
aiongg 7bde5098a6 feat: convert anchored object offset and vertical alignment
Anchors and object styles hold anchored object settings in chunk
0x2800: the Y offset at 0 and the vertical alignment at 52. Write them
on object styles, and on anchored items where they differ from the
item's object style.
2026-10-07 01:43:24 +00:00
aiongg 537af3af4d feat: convert the bullet character list
The preferences object lists the bullet characters offered for lists in
chunk 0x1A488. Write them as ABullet elements.
2026-10-07 01:42:37 +00:00
aiongg 44c85c5a54 feat: write export options of page items
Every page item in the corpus IDML files from DOM 12 on has the same
ObjectExportOption, and every rectangle, oval and polygon the same
InCopyExportOption. Write them from that observation.
2026-10-07 01:41:55 +00:00
aiongg 32c74f5c6b feat: convert inks and colour groups
Write an Ink for each ink object (class 0x1F07) with its neutral
density, trap order, frequency and angle, and the colour groups listed
in the preferences (chunk 0x1F61) with their swatches.
2026-10-07 01:41:10 +00:00
aiongg 9eba4eea66 feat: convert composite fonts, kinsoku tables and mojikumi tables
Read the composite font and its entries (classes 0xCB02 and 0xCB03):
name, font, style and custom characters, and the kinsoku and mojikumi
tables with the character lists of a custom kinsoku table.
2026-10-07 01:40:31 +00:00
aiongg 3f4a68dfe6 feat: write object style settings for every object style
Object styles other than [None] now get their text frame, story, text
wrap and frame fitting settings, fill, stroke and corner attributes,
applied paragraph style and some category flags from the INDD (chunks
0x1B92B, 0x1B924, 0x285B, 0x50F28, 0x3776, 0x3777, 0x1B92E, 0x1B946),
and the values every such style has in the corpus IDML files, collected
by tools/root_values.py into src/idml/object_style_values.xml. The root
style gets the same INDD values in place of observed ones.
2026-10-07 01:33:21 +00:00
aiongg 4ff4dd03e0 feat: convert frame fitting and clipping path settings
Frame fitting attributes are in the page item attribute list and, for
object styles, in chunk 0x1B956. Write FrameFittingOption on rectangles,
ovals, polygons and object styles, ClippingPathSettings on images, PDF
and EPS graphics from chunk 0x2C1A, and ImageIOPreference on images from
chunk 0x8C39.
2026-10-07 01:26:02 +00:00
aiongg 2694a9ea43 feat: write story preferences and story values every IDML has
Every story in the corpus IDML files has the same StoryPreference and
InCopyExportOption, and the same TrackChanges, StoryTitle,
AppliedNamedGrid, UserText and IsEndnoteStory (the last two from DOM 12
and 15). No INDD field for them is identified, so they are written from
IDML observation.
2026-10-07 01:21:28 +00:00
aiongg c938df6f0b feat: convert page margins, columns and layout grid
Pages store margins (chunk 0x51A) and columns (0x528) with a flag that
says whether they are the page's own; otherwise the master page at the
same position supplies them. Write MarginPreference from the values in
effect and GridDataInformation from chunk 0xCD02.
2026-10-07 01:13:34 +00:00
aiongg 0dc40563ab feat: write preference values that every exported IDML has 2026-10-07 00:35:38 +00:00
aiongg c976451c2f feat: convert cell formatting, cell and table styles, and table attributes 2026-10-07 00:24:59 +00:00
aiongg 9594ac557b feat: convert tint swatches 2026-10-07 00:04:23 +00:00
aiongg ccda8898e9 feat: convert page item transparency: opacity, blend mode and effects 2026-10-06 23:55:15 +00:00
aiongg 146afc2cf7 feat: convert cross-reference formats
Read cross-reference formats (class 0x1355E) and their building blocks
and write them to designmap. Evidence is in
docs/format/cross-references.md.
2026-10-06 23:23:45 +00:00
aiongg 78b05c7687 feat: convert hyperlinks, text sources, destinations and bookmarks
Read hyperlinks (class 0x13501), text sources (0x13502) with their text
ranges from the story's range tree, page and URL destinations and
bookmarks, and write them to designmap and the stories. Evidence is in
docs/format/hyperlinks.md.
2026-10-06 23:16:28 +00:00
aiongg 3ca668e96e feat: convert text variables and their instances
Read text variable definitions (class 0xCAB4) into designmap TextVariable
elements with their settings, and write instances (class 0xCA64) as
TextVariableInstance elements in place of U+0018. Evidence and the
settings left out are in docs/format/text-variables.md.
2026-10-06 23:03:09 +00:00
aiongg 4014eea106 feat: write font families and fonts in Fonts.xml
Read the font records of class 0x3E03 (chunk 0x3E05) and the Typekit IDs
(chunk 0x3EEB), and write FontFamily and Font elements. Evidence and the
fields left out are in docs/format/fonts.md.
2026-10-06 22:55:45 +00:00
aiongg 3490507fbf feat: convert stroke type and stroke alignment
Decode page item attribute 0x6E6E (a reference followed by a built-in
stroke style code) as StrokeType and 0x6E8C as StrokeAlignment, with
evidence from items and from object style attribute lists. Write the 18
built-in stroke styles every exported IDML lists.
2026-10-06 22:28:29 +00:00
aiongg 358fb2dbe1 feat: convert ruler guides
Read guide objects (class 0x3301, chunk 0x3308) from spread layers and
write them as Guide elements in their page: orientation, location, fit to
page, guide type, item layer, and the view threshold and colour values the
samples show. Locations are measured from the spread, and ViewPreference
states RulerOrigin SpreadOrigin to match.
2026-10-06 22:20:45 +00:00
aiongg 36b7ae9eaf feat: write root style values that every exported IDML has
Collect the root style attributes, Properties and child elements that are
the same in every corpus IDML (tools/root_values.py) and write them on the
root paragraph, character, object, cell and table styles. Values read from
the INDD take precedence; values that first appear after DOM 7 are gated
by InDesign version. Also write the default numbering list, and write a
style only once when two style objects share a name.
2026-10-06 22:04:05 +00:00
aiongg 327eaee967 docs: state nested-style evidence in general terms; stop tracking __pycache__ 2026-10-06 21:15:29 +00:00
aiongg d5d2e1451e feat: convert keep lines, nested styles, tab stops, languages and more text attributes
Adds KeepLinesTogether, AllNestedStyles, TabList, AppliedLanguage,
RightIndent, BaselineShift, VerticalScale, NoBreak, OTFFigureStyle,
Position, SingleWordJustification, ParagraphBreakType, rule above
indents and width, RuleBelow and its colour, span columns, paragraph
shading and border on/colour/top/bottom offsets, SameParaStyleSpacing,
grid alignment, text gradient fill, EndJoin, and bullet and numbering
attributes. Four-byte values of numeric attributes are now written,
which fixes DropcapDetail. Evidence is in docs/format/attributes.md.
2026-10-06 21:14:52 +00:00