mirror of
https://github.com/storytold/indd-utils.git
synced 2026-11-20 17:20:49 +01:00
312 lines
13 KiB
Markdown
312 lines
13 KiB
Markdown
# INDD objects
|
||
|
||
How the byte stream of a database object (see `database.md`) is
|
||
structured, and what the classes and chunks used by the converter mean.
|
||
Implemented in `src/object.rs` and `src/model/`.
|
||
|
||
## How this was learned
|
||
|
||
IDML exported by InDesign names most objects `Self="u<hex>"`, and that hex
|
||
number is the object's UID in the INDD file. So each corpus pair gives, for
|
||
every such IDML element, the INDD object, its class, and the attribute
|
||
values its chunks must encode. Facts below were found by comparing the two,
|
||
and are checked by `tools/compare.py`, which converts every pair (75 unique
|
||
pairs, InDesign 12–21) and compares the output with the reference IDML
|
||
attribute by attribute.
|
||
|
||
## Chunks
|
||
|
||
Most objects (99.9 % of 773,753 in the little-endian corpus) are a sequence
|
||
of chunks:
|
||
|
||
| Size | Field |
|
||
|---|---|
|
||
| 4 | Chunk ID |
|
||
| 4 | Data length *n* |
|
||
| *n* | Data |
|
||
|
||
The rest are plain byte streams: objects of class 0x129 (raw data, below)
|
||
and the XMP packet (UID 0x80000001). In the 245 distinct little-endian
|
||
files, all 1,171 class 0x129 objects are plain byte streams. Chunk IDs are in the same number space as
|
||
class IDs; a chunk often lists objects of the class with the same number
|
||
(for example chunk 0x501 of the document lists spreads, class 0x501).
|
||
|
||
## Primitive encodings
|
||
|
||
All little-endian.
|
||
|
||
- **UID lists:** u32 count, then that many u32 UIDs.
|
||
- **Matrices:** six f64, in IDML `ItemTransform` order (a b c d tx ty).
|
||
- **Text segments:** a u16 header, flags in bits 14–15 and a count in bits
|
||
0–13. 0x4000: *count* single-byte characters follow. 0x8000: *count*
|
||
UTF-16LE code units follow. A text of *n* code units is a sequence of
|
||
segments adding up to *n*. Example: "WOMEN’S\r" is
|
||
`05 40 "WOMEN" 01 80 19 20 02 40 "S\r"`.
|
||
- **In-object strings:** u8 2, u8 (usually 0, meaning unknown), u16 length
|
||
in code units, then segments. 4,135 occurrences in three sample files.
|
||
|
||
## Document (class 0xE01, always UID 1)
|
||
|
||
| Chunk | Contents |
|
||
|---|---|
|
||
| 0x501 | UID list: spreads, in order |
|
||
| 0x1401 | UID list: master spreads |
|
||
| 0x301 | UID list: layers (the first is an internal layer that holds pages) |
|
||
| 0x313 | u32: active layer (IDML `ActiveLayer`) |
|
||
| 0x222 | UID lists: stories (IDML `StoryList`) |
|
||
| 0x4C01 | UID list: sections |
|
||
|
||
## Spreads (0x501) and master spreads (0x1401)
|
||
|
||
| Chunk | Contents |
|
||
|---|---|
|
||
| 0x503 | u32 self, u32 0, UID list of spread layers |
|
||
| 0x56E | Matrix: `ItemTransform` |
|
||
| 0x1B8 | u32: `BindingLocation` |
|
||
|
||
**Spread layers (0x301)** hold the items of one document layer on one
|
||
spread. Chunk 0x302: u32 document layer UID, u16 1 for the layer's guide
|
||
part. Chunk 0x303: u32 spread, u32 spread, UID list of children. Pages are
|
||
children of the spread layer for the internal pages layer.
|
||
|
||
## Layers (0x302)
|
||
|
||
Chunk 0x304: u16 locked, u16 visible, then fields not yet identified, then
|
||
the name as an in-object string. The internal layer is named
|
||
`Internal_pages_layer_name` and has no IDML element.
|
||
|
||
## Pages (0x50F)
|
||
|
||
| Chunk | Contents |
|
||
|---|---|
|
||
| 0x140F | u32 applied master spread, u16 (unknown), matrix `MasterPageTransform` |
|
||
| 0x5CC | Matrix: `ItemTransform` |
|
||
| 0x5DD | Four f64: left, top, right, bottom. IDML `GeometricBounds` is top, left, bottom, right. |
|
||
| 0x51A | Four f64: margins (36 in the blank document) |
|
||
|
||
## Page items
|
||
|
||
Frames, shapes and lines are all class 0x6201; groups are 0x401.
|
||
|
||
| Chunk | Contents |
|
||
|---|---|
|
||
| 0x151 | Matrix: `ItemTransform` |
|
||
| 0x162B | Path geometry (below) |
|
||
| 0x15B | Hierarchy: u32 spread, u32 parent, UID list of children |
|
||
| 0x6E03 | Attribute list: local formatting (see `attributes.md`) |
|
||
| 0x1B916 | u32: applied object style |
|
||
|
||
**Path geometry:** u32 path count; per path: u32 point count, points, u16
|
||
1 if the path is open. Each point is a u32 type, then f64 values: type 2 =
|
||
anchor (x, y) only; types 0 and 1 = left direction, anchor, right
|
||
direction. All 3,191 path chunks in the samples parse exactly, and the
|
||
points of every page item path match the IDML `PathPointType` values.
|
||
|
||
**IDML element type** is not stored. A frame is a `TextFrame` if a child of
|
||
class 0x263 exists. Otherwise the converter uses the path: two points and
|
||
open → `GraphicLine`; four corner points on two distinct x and y values →
|
||
`Rectangle`; four smooth points → `Oval`; anything else → `Polygon`. In the
|
||
samples this gives 123/123 lines, 671/671 rectangles, 10/10 ovals and
|
||
396/402 polygons (6 polygons are ellipse-like).
|
||
|
||
**Text frames:** the frame's child of class 0x263 (multi-column frame) has
|
||
a child of class 0x227 (frame column). Chunk 0x220 of the column holds the
|
||
frame list (class 0x228). Its chunk 0x205: u32 story, then a UID list of
|
||
the columns of all threaded frames in order, which gives
|
||
`PreviousTextFrame` and `NextTextFrame`.
|
||
|
||
## Placed graphics
|
||
|
||
Images (0x1702), PDF (0x2501), EPS (0x6601) and SVG (0x6639) are children
|
||
of a frame (in its 0x15B list) and share these chunks:
|
||
|
||
| Chunk | Contents |
|
||
|---|---|
|
||
| 0x151 | Matrix: `ItemTransform` (437/437 match) |
|
||
| 0x1633 | Four f64: `GraphicBounds` left, top, right, bottom |
|
||
| 0x8CBC | u32, u32, u32 link UID |
|
||
|
||
**Links (0x8C42).** Chunk 0x8C9B: u32 0, u32, u32 link resource UID, u32,
|
||
u32 graphic UID, fields, then the import stamp as a u32 length and text
|
||
segments, then two 8-byte timestamps. **Link resources (0x8C41)**: chunk
|
||
0x8C92 is a flag byte, u32 length, then the URI as bytes
|
||
(`LinkResourceURI`; 335/340 match, the rest were relinked after export).
|
||
|
||
After the URI, chunk 0x8C92 continues:
|
||
|
||
| Offset after the URI | Field |
|
||
|---|---|
|
||
| 0 | u8 1, u8 1, u16 0 (all resources) |
|
||
| 4 | u32 0 or 1, not identified |
|
||
| 8 | u32 0, 2 or 3 |
|
||
| 12 | u32 UID of a raw data object (class 0x129), or 0 |
|
||
|
||
**Embedded links.** A link is embedded (`StoredState="Embedded"`) when its
|
||
resource names a raw data object at offset 12. That object holds the
|
||
linked file's bytes, which IDML writes as the graphic's `Contents`.
|
||
|
||
Evidence:
|
||
|
||
- In the 245 distinct little-endian files there are 883 link resources.
|
||
The field at offset 8 is 2 in 123 of them, and exactly those 123 name a
|
||
raw data object at offset 12. The other 760 have 0 at offset 12 (742
|
||
with 0 at offset 8, 18 with 3; no pair shows what 3 means).
|
||
- In the corpus pairs, all 10 resources of embedded links (12 links) name
|
||
a raw data object, and none of the 369 resources of normal links do.
|
||
- For all 12 embedded links, the raw data object's bytes equal the
|
||
base64-decoded `Contents` of the IDML graphic (JPEG, PNG, PDF, EPS and
|
||
SVG files, 2 KB to 11 MB).
|
||
|
||
## Graphics pasted without a link
|
||
|
||
A graphic with no link (its chunk 0x8CBC names no link) can hold its file
|
||
itself:
|
||
|
||
| Class | Chunk | Contents |
|
||
|---|---|---|
|
||
| Image (0x1702) | 0x8C23 | u32 raw data object (0 = none) |
|
||
| PDF (0x2501) | 0x2521 | u32 raw data object (0 = none) |
|
||
|
||
Evidence from the corpus pairs: the raw data object equals the IDML
|
||
`Contents` for 124 of 124 PDFs and 2 of 2 images without a link. One
|
||
more image without a link has 0x8C23 = 0; its IDML `Contents` equals the
|
||
preview data described below, which the converter does not use. In the
|
||
whole corpus, two linked and embedded PDFs also have a 0x2521 object,
|
||
different from the link resource's; with no pair to compare, the
|
||
converter uses the link resource's.
|
||
|
||
## Graphic previews
|
||
|
||
Placed graphics also have chunk 0x170D, the UID of an object of class
|
||
0x1708, whose chunk 0x119 names a raw data object. In the 245 distinct
|
||
little-endian files, these objects are TIFF (613), JPEG (411), PNG (307),
|
||
GIF (27) and one other. In the same-version pairs they equal the IDML
|
||
`Contents` for only 3 of 109 embedded or pasted graphics, so they are
|
||
most likely screen previews. The converter does not use them.
|
||
|
||
## Embedded data in IDML
|
||
|
||
The reference IDML files write `Contents` inside the graphic's
|
||
`Properties` as base64 (with `=` padding) in lines of 76 characters joined
|
||
by a line feed, with no line feed at the end. The text is split into
|
||
CDATA sections of 262,144 characters. The converter writes the same form.
|
||
|
||
## Stories (0x201)
|
||
|
||
Chunk 0x223: u32 length, u16, u32 first strand, UID list of more strands,
|
||
then other fields. Each strand has chunk 0x261: u16 count, then (u32
|
||
length, u32 data object UID) pairs. Each data object has chunk 0x262:
|
||
|
||
| Field | Contents |
|
||
|---|---|
|
||
| u32 | Kind: 0x202 text, 0x203 character style runs, 0x204 paragraph style runs |
|
||
| u32 | Owning strand |
|
||
| u16 | Run count |
|
||
| runs | Each: u32 record size, then the record |
|
||
| u32 | Total length |
|
||
|
||
Each record starts with a u32 run length in UTF-16 code units. Text
|
||
records continue with text segments. Style records continue with the
|
||
style UID.
|
||
|
||
**Owned items (run kind 0x209).** Each record: u32 run length, u16
|
||
count, then that many (u32 class, u32 UID) pairs, the objects owned by
|
||
the text position at the start of the run. An item anchored in text has
|
||
the character U+FFFC at that position and is owned through an object of
|
||
class 0x262, whose chunk 0x2C8 (u32, u32, UID list) lists the anchored page
|
||
item. IDML writes the page item element in place of the U+FFFC.
|
||
|
||
INDD stores a forced line break as U+000A; IDML writes it as U+2028. The
|
||
last paragraph return of a story is not written to IDML.
|
||
|
||
## Styles (0x205)
|
||
|
||
Paragraph and character styles share the class. Chunk 0x230: u32 next
|
||
style (0 = itself), u32 based-on style, fields not yet identified, u32 1
|
||
for paragraph styles or 0 for character styles, u8 1 if the name is a
|
||
built-in key (`$ID/` in IDML), the name as an in-object string, then a GUID
|
||
string in newer files. The name's offset varies (23–26 bytes), so the
|
||
converter locates it as a flag byte followed by a valid in-object string.
|
||
All 486 style names and 291 `NextStyle` values in the pairs match.
|
||
|
||
IDML writes a `BasedOn` of the root `[No paragraph style]` or
|
||
`[No character style]` as a string (`$ID/[No paragraph style]`), and any
|
||
other base as an object reference.
|
||
|
||
## Style groups and object styles
|
||
|
||
**Root groups** (class 0xCA8C for paragraph and character styles, 0x1B972
|
||
object styles, 0x20241 cell styles, 0x1044F table styles) are named in
|
||
IDML by UID (`RootParagraphStyleGroup Self="u7e"`). Chunk 0x28C2 says
|
||
which kind: 0xCA0C paragraph, 0xCA0D character, 0x1B924 object, 0xB669
|
||
cell, 0xB668 table (all 75 pairs). Children: chunk 0x28DC (text styles)
|
||
or 0x1B95A (object styles), each u32, u32, UID list. A built-in root style
|
||
such as `[No character style]` need not be listed.
|
||
|
||
**Groups** (class 0xCA8B): chunk 0x28D3 is parent, root, UID list of
|
||
children; chunk 0x28D2 a flag byte and the name. IDML names every group
|
||
`$ID/<name>` (20 of 21 groups), and a style inside groups is referenced as
|
||
`ParagraphStyle/<group>:<group>:<name>` with `:` escaped as `%3a`. All 412
|
||
paragraph styles and 21 groups in the pairs match.
|
||
|
||
**Object styles** (class 0x1B901): chunk 0x1B907 is u32 based-on style,
|
||
u8 1 for a built-in name, then the name. A page item's chunk 0x1B916 is
|
||
its applied object style (`AppliedObjectStyle`, 671/671 rectangles). A
|
||
style based on the root `[None]` has its `BasedOn` written as a string.
|
||
|
||
## Master spread names and sections
|
||
|
||
Master spread chunk 0x1402: a flag byte and the name prefix (`A`), a flag
|
||
byte and the base name (`Master`). IDML `Name` is `<prefix>-<base>`, and
|
||
the master's pages are named by the prefix (99/99 match).
|
||
|
||
Section chunk 0x4C02: u8, string, u8, string, u32, u32 page number start,
|
||
u32, u32 continue numbering (1 = true). All 73 sections match. Every
|
||
section in the corpus uses Arabic numbering.
|
||
|
||
## Document preferences (class 0x2202)
|
||
|
||
One object of class 0x2202 holds document-wide preferences. Chunk 0x533
|
||
(`DocumentPreference`): f64 page width at 0, f64 page height at 8, byte 58
|
||
2 for facing pages and 1 otherwise, four f64 bleeds from offset 70, u32
|
||
intent at 142 (0 print, 1 web, 2 mobile). All values match the 75 pairs.
|
||
The four bleeds are equal in every sample, so their order (written as top,
|
||
bottom, inside, outside) is not verified.
|
||
|
||
## Text frame preferences
|
||
|
||
From the frame's multi-column frame object (class 0x263):
|
||
|
||
| Chunk | Offset | `TextFramePreference` attribute |
|
||
|---|---|---|
|
||
| 0x2D1 | u32 0 | `TextColumnCount` (1,052/1,052) |
|
||
| 0x2D1 | f64 4 | `TextColumnGutter` (93/93) |
|
||
| 0x2D1 | f64 14 | `TextColumnFixedWidth` (1,045/1,045) |
|
||
| 0x2CE | u16 2 | `VerticalJustification`: 0 Top, 1 Center, 2 Bottom, 3 Justify (350/350) |
|
||
| 0x2CE | u16 20 | `VerticalBalanceColumns` (18/18) |
|
||
| 0x2CE | u16 22 | `AutoSizingType`: 0 Off, 1 HeightOnly, 2 WidthOnly, 3 HeightAndWidth (61/61) |
|
||
| 0x2CE | u32 24 | `AutoSizingReferencePoint`: 0–8, top-left to bottom-right by rows (58/58) |
|
||
|
||
Inset spacing is 0 in every sample and not located.
|
||
|
||
## Colours (0x1F05)
|
||
|
||
| Chunk | Contents |
|
||
|---|---|
|
||
| 0x1F10 | u8 1 if the name is a built-in key; name; u32 flags: bit 0 removable, bit 1 visible, bit 2 editable |
|
||
| 0x1F01 | u32 space (5 RGB, 6 CMYK), u16 count, f64 components as fractions |
|
||
| 0x1F09 | u32 model: 0 Process, 2 Registration |
|
||
| 0x1F24 | f64 (−1), then u32 `ColorOverride`: 0 Normal, 1 Specialpaper, 2 Specialblack, 3 Specialregistration, 4 Hiddenreserved |
|
||
|
||
Unnamed colours are referenced by UID (`Color/u93`). All 1,296 colours in
|
||
the pairs match on `Model`, `Space`, `ColorValue`, `ColorOverride`,
|
||
`Name` and the three flags.
|
||
|
||
The `None` swatch is class 0x6E0B (name in chunk 0x1F30).
|
||
|
||
**Gradients (0x5503).** Chunk 0x5503: u16 stop count *n*, *n* u32 stop
|
||
colour UIDs, *n* f64 locations (0–1), *n* f64 midpoints (the midpoint
|
||
between stops i and i+1 is stored with stop i), u32 type (1 linear).
|
||
Chunk 0x5505: flag byte, name, u32 flags as for colours. All 99 gradients
|
||
and 198 stops in the pairs match.
|