Files
indd-utils/CONTRIBUTING.md
T
aiongg ae5a44db3a docs: add ARCHITECTURE.md and CONTRIBUTING.md; bring the README up to date
The README listed implemented features as not started, an old attribute
count, corpus size and numbers, and no library use. The numbers are from
compare.py --all with schema validation at this revision.
2026-10-07 20:07:28 +00:00

6.9 KiB

Contributing

This project learns the INDD format only from sample files, under the clean-room rules in CLEANROOM.md. Read that file first. ARCHITECTURE.md explains where the code for each step lives.

Clean-room rules

Every contributor follows these rules. A change that breaks one is not accepted.

  • No Adobe software. Do not install or run InDesign, and do not use the InDesign SDK, its documentation or its binaries.
  • No knowledge from elsewhere. Do not use what you know or remember of InDesign internals (class names, IDs, persistence details). Use only what the samples show and the references CLEANROOM.md permits.
  • No made-to-order samples. Do not create INDD files for this project or ask anyone to make them.
  • Every fact has evidence. A fact about the INDD format goes in docs/format/ with what supports it: which files, how many, what was compared. Code relies only on documented facts.
  • No sample files in the repository. Never commit INDD, IDML, PDF, images or archives, whatever their licence. Open-licensed test fixtures are downloaded by tools/fetch_fixtures.py; other samples stay in the git-ignored corpus/.
  • No golden files from samples. Tests compare with the reference IDML at run time; they do not store output derived from third-party files.
  • Do not name the corpus sources in committed files. The local corpus/SOURCES.md records where the samples came from.
  • Privately held samples stay private. Leave them out of committed numbers (--exclude own/ in the tools) and never describe their content.
  • Treat sample files as untrusted. Run Python tools with -I, keep downloads in their own directory, and keep scratch scripts out of it.

Setting up

  1. Install a recent stable Rust and Python 3.
  2. python3 -I tools/fetch_fixtures.py downloads the test fixtures and checks their SHA-256.
  3. Put your samples in corpus/ and run python3 -I tools/inventory.py corpus/ to list their versions and pairs. A pair is an INDD file with an IDML exported from the same document.
  4. For schema validation, get the IDML RelaxNG schemas and Jing and keep them outside the repository (see tools/validate.sh).

Checks

There is no hosted CI. Run the checks on your own machine before every commit:

cargo fmt
cargo clippy --all-targets
cargo test --release
cargo doc --no-deps
python3 -I tools/diff_outputs.py              # output of HEAD vs the working tree
cargo build --release && python3 -I tools/compare.py --exclude own/

After a change to the IDML writer, also validate every output: python3 -I tools/compare.py --all --exclude own/ --schemas <dir> --jing <dir> (about 5 minutes on the full corpus).

Kind of change Required result
Refactoring diff_outputs.py: 0 differing outputs, 0 changed failures and warnings
Behaviour change compare.py: no new conversion or schema validation failures; value coverage does not fall; the diff of diff_outputs.py is explained in the commit message

Adding something end to end

The same six steps apply whether you add a text attribute, a page item setting, an element or a constant value.

1. Find it

  • indd audit <file> lists what the converter does not read in one document: classes, chunks, attribute IDs, unknown codes.
  • python3 -I tools/audit_corpus.py --exclude own/ ranks those over the corpus by files affected (tables in target/audit/).
  • compare.py prints the biggest value gaps and writes gaps.tsv and values.tsv to target/compare/; --detail TAG --show N shows the differing values of one element.

2. Prove it

  • Pair an INDD object with its IDML element: the element's Self is the object's UID in hexadecimal (Self="u1a2" is UID 0x1A2).
  • Look at the bytes with indd dump <file> <uid>... and indd objects.
  • Write a scratch script (outside the repository) that compares the candidate field with the IDML value over every pair, and count the matches. Use trustworthy pairs (docs/measurement.md).
  • A field whose value is the same in every sample cannot be told apart from its neighbours. Do not map it.

3. Document it

Add the fact to the matching file in docs/format/: the layout, the mapping, and the evidence (files, counts such as "171 of 171").

4. Decode it

All INDD decoding is in src/model/.

What Where
A class or chunk ID src/model/ids.rs (or the area module's class / chunk)
A value in an attribute list nothing for a number; for a structured value, a Layout in text_layout and a Value variant (src/model/attrs.rs)
A setting in its own chunk a field on the model struct, read in the area module (item.rs, style.rs, …) with self.chunk and self.cursor
A new class a struct, a reader function, and an arm in Reader::class_object (src/model/document.rs)

Read numbers and strings with a Cursor from self.cursor(&data) (or obj.cursor(data)), never with from_le_bytes: object data is in the file's byte order. Add a unit test that builds the bytes in code; the test-only database::synthetic module builds objects, flagged strings and whole databases, in either byte order.

5. Write it

The writer (src/idml/) maps model types to IDML and never reads INDD bytes.

What Where
An attribute in a text, page item, cell or table list a row (ID, IDML name, Kind) in the tables of src/idml/attrs.rs
A new kind of value a Kind variant and its arm in Writer::value (src/idml/kind.rs)
A setting or element the writer function for its element, at the position the schema requires

Write values read from the INDD with Xml::attr. They take precedence over observed values in any order.

6. Measure it

Run compare.py and diff_outputs.py. Check that value coverage rose, that no extra values appeared (extras.tsv), and that every output still validates. Put the numbers in the commit message.

Constant values

A value that every InDesign export has, but that has no INDD field, comes from the generated value files, not from a literal in the writer:

  1. Add the element path to ELEMENTS (or LISTS, KEYED, WHEN_WRITTEN) in tools/element_values.py.
  2. Run python3 -I tools/element_values.py --write and review the diff of src/idml/element_values.xml.
  3. Read the values with values::element (or element_attrs, when_written, list, keyed) where the writer writes the element.
  4. Record the evidence in docs/format/idml-values.md.

Commits

  • Use conventional commit messages: feat:, fix:, refactor:, test:, docs:, chore:.
  • One change per commit. A refactoring commit changes no output.
  • For a behaviour change, say in the body what changed in the output and why, with the numbers from compare.py.
  • Stage only the files you changed; never commit corpus/, fixture files, schemas, Jing, or tool output.
  • Do not add hosted CI workflows.