A text frame's multi-column frame stores a matrix (chunk 0x2DE) that is
the identity for horizontal text and a quarter turn for vertical text.
A story whose frames all have the quarter turn is written Vertical
instead of the Horizontal every sample's IDML has.
Section style code 0x4C12 is Kanji, shown by the Chinese-numeral folios
of a vertically typeset sample's print PDF. Pages in such a section are
named with Chinese digits, as the folios show.
DocumentPreference PageBinding comes from the document preferences
(u16 at 64) instead of the value every sample's IDML has. A vertically
typeset sample bound on the right stores 1, RightToLeft.
compare.py --all now also converts the INDD and INDT files that have no
same-version IDML (any version, either byte order), validates them and
counts converter warnings by kind. validate.sh runs Jing once per schema
instead of once per file, which makes validating the whole corpus take
about 3 minutes.
Two schema failures showed up in files without an IDML:
- Gradient swatch midpoints are stored as positions from the start of
the gradient. IDML wants them relative to the gap between two stops,
as for opacity gradient stops. Values outside 13-87 are left out with
a warning.
- The style kind is a u16, not a u32. One InDesign 7.5 template has a
non-zero u16 after it, which made 151 character styles paragraph
styles.
The byte order flag applies to object data only: database pages stay
little-endian. Object data is decoded in the file's byte order, and in
big-endian data the string tag and the flag byte before it are swapped.
The two big-endian samples are also the only files from before
InDesign 7.0, and some of their layouts differ: page transform and
bounds chunks, font records, XML tag names and nodes, story strands and
40-byte guide records are read; text frame settings, document
preferences, a composite font and a section style are left out with a
warning. The 40-byte guides of an InDesign 7.0 file are now read too.
docs/format/big-endian.md records the evidence. The corpus tests now
cover all byte orders.
Read the XML nodes stored with each story, place their markers in the
story text and write the backing story as XmlStory, with the elements
in it and in the stories they tag. Element Self names follow the node
IDs from the root down. The backing story is added to StoryList.
Text records count UTF-16 code units, but style runs, owned items and
text owners count a surrogate pair as one position. Convert those
positions to UTF-16 offsets so characters outside the Basic Multilingual
Plane keep their formatting and are not split into two replacement
characters.
XML tags (class 0xBF19) hold their name in chunk 0xBF2F and refer to an
interface colour in chunk 0x117. Write them in XML/Tags.xml, sorted by
name as IDML lists them.
Anchors and object styles hold anchored object settings in chunk
0x2800: the Y offset at 0 and the vertical alignment at 52. Write them
on object styles, and on anchored items where they differ from the
item's object style.
Write an Ink for each ink object (class 0x1F07) with its neutral
density, trap order, frequency and angle, and the colour groups listed
in the preferences (chunk 0x1F61) with their swatches.
Read the composite font and its entries (classes 0xCB02 and 0xCB03):
name, font, style and custom characters, and the kinsoku and mojikumi
tables with the character lists of a custom kinsoku table.
Object styles other than [None] now get their text frame, story, text
wrap and frame fitting settings, fill, stroke and corner attributes,
applied paragraph style and some category flags from the INDD (chunks
0x1B92B, 0x1B924, 0x285B, 0x50F28, 0x3776, 0x3777, 0x1B92E, 0x1B946),
and the values every such style has in the corpus IDML files, collected
by tools/root_values.py into src/idml/object_style_values.xml. The root
style gets the same INDD values in place of observed ones.
Frame fitting attributes are in the page item attribute list and, for
object styles, in chunk 0x1B956. Write FrameFittingOption on rectangles,
ovals, polygons and object styles, ClippingPathSettings on images, PDF
and EPS graphics from chunk 0x2C1A, and ImageIOPreference on images from
chunk 0x8C39.
Pages store margins (chunk 0x51A) and columns (0x528) with a flag that
says whether they are the page's own; otherwise the master page at the
same position supplies them. Write MarginPreference from the values in
effect and GridDataInformation from chunk 0xCD02.
Read text variable definitions (class 0xCAB4) into designmap TextVariable
elements with their settings, and write instances (class 0xCA64) as
TextVariableInstance elements in place of U+0018. Evidence and the
settings left out are in docs/format/text-variables.md.
Read guide objects (class 0x3301, chunk 0x3308) from spread layers and
write them as Guide elements in their page: orientation, location, fit to
page, guide type, item layer, and the view threshold and colour values the
samples show. Locations are measured from the spread, and ViewPreference
states RulerOrigin SpreadOrigin to match.
Map paragraph attributes 0x1B37 (StartParagraph), 0x1B0D and 0x1B0E
(drop cap characters and lines), 0x1B31 and 0x1B32 (maximum and minimum
letter spacing), and the text frame's first baseline offset code in
chunk 0x2CE. Evidence is recorded in docs/format.
Capitalization codes 1 (SmallCaps) and 3 (CapToSmallCap), text wrap
modes 1 (JumpObjectTextWrap) and 3 (BoundingBoxTextWrap) with offsets
stored as left, top, right, bottom, and page number style 0x4C17
(LowerRoman) with Roman page names. Evidence in docs/format.
Read chunk 0x3703 (mode, offsets, flags) and the contour type in chunk 0x373D, and write TextWrapPreference for page items, groups and placed graphics. Modes 0 (None) and 6 (Contour) are written; other modes are left out with a warning. Offsets are written only when all are 0, since their order is not known yet.
Read each section's first page and page number style from chunk 0x4C02, sort sections by their first page, and write one Section per section with its length. Write PageNumberStart only when numbering restarts, and PageNumberStyle only for the Arabic style seen in the corpus. Pages are named by their number in their section.
Master pages use the same chunk 0x140F as document pages. Write AppliedMaster and MasterPageTransform for them too: "n" and the identity matrix when the chunk is absent or empty, as in all corpus pairs, and the named master spread otherwise.
Read gradient swatches (stops, locations, midpoints, type) and write them
to Graphic.xml after the swatches, as the schema requires. Write
CornerOption, CornerRadius and gradient start and length on page items.
Decode unknown 4-byte attribute values as numbers.
Write DocumentPreference (page size, facing pages, bleeds, intent) and
TextFramePreference (columns, gutter, column width, vertical
justification, auto-sizing). compare.py now also compares child elements
without Self and the preference elements.
Write the style hierarchy with root and nested groups and path-qualified
style references, object styles with AppliedObjectStyle on page items,
master spread names and prefixes, and sections with their numbering.
Keep negative zero in numbers, as IDML does.
Read graphics inside frames (transform, bounds) and their links and link
resource URIs, and write them as Image/PDF/EPS/SVG elements with Link
children. Frames with graphics get ContentType GraphicType.
Read the owned-items strand of stories and write anchored page items in
place of their U+FFFC character. Record that the third object tree lists
deleted UIDs, which no live hierarchy references.
Decode colour swatches (name, model, space, values, override, flags) and
the None swatch, and write them to Graphic.xml. Parse attribute lists and
write page item fill colour and tint, stroke colour and weight, and miter
limit. Write BasedOn of root styles as a string, as IDML does. Write forced
line breaks as U+2028 and keep surrogate pairs intact across text runs.
Document the object, chunk and attribute formats with their evidence.