mirror of
https://github.com/storytold/indd-utils.git
synced 2026-11-20 17:20:49 +01:00
Pure moves: IDs (ids), the Reader core (reader), Document and Reader::document (document), spreads and pages (spread), page items and graphics (item), stories (story), styles (style), document-level settings (settings) and name searches (strings). tools/diff_outputs.py: 0 of 4,274 outputs differ.
5.9 KiB
5.9 KiB
indd-utils
Rust reader for INDD files (Adobe InDesign's native format) and converter
to IDML. Crate and command: indd. Repo: aiongg/indd-utils (private for
now). Licence: MIT OR Apache-2.0.
Clean-room rules (read CLEANROOM.md)
- Never commit downloaded sample files of any kind (INDD, IDML, PDF,
images, archives), whatever their licence. They can be downloaded
elsewhere; the repo records only how to get them.
corpus/is git-ignored and stays local. - Every fact about the INDD format goes in
docs/format/with the evidence that supports it (which files, how many, what was compared). Code relies only on documented facts. - Do not use knowledge of InDesign internals from memory (InDesign SDK
class names, IDs, persistence details). If it can't be shown from the
corpus or a permitted reference in
CLEANROOM.md, it doesn't go in. - Do not install or run InDesign, and do not ask anyone to create INDD files for this project.
- No golden output files derived from third-party samples. Corpus tests compare against the sibling IDML at run time.
- Openly licensed corpus sources may be listed in a committed fetch
manifest (pinned URL, hash, licence) so contributors can rebuild that
part of the corpus. Do not name any other source in committed files.
The full provenance record is
corpus/SOURCES.md(local).
Layout
src/header.rs,src/container.rs: decoded layers (seedocs/format/).src/audit.rs: records what a conversion reads, forindd audit.src/model/: the document model.reader(typed object access),document(Reader::document), and one module per area:spread,item,story,style,settings,table,color,font, …src/idml/: the IDML writer.mod.rshaswrite; one module per package part (designmap,resources,styles,spread,story), plusattrs(attribute tables),format,pages,values.docs/measurement.md: howcompare.pyand the audit measure the converter (stale pairs, value coverage, exclusions).tests/fixtures.rs: smoke tests on open-licensed samples listed intests/fixtures/manifest.json; they skip iftests/fixtures/files/is absent.tools/fetch_fixtures.py: downloads the fixtures into the git-ignoredtests/fixtures/files/and checks their SHA-256.tests/corpus.rs: tests over the localcorpus/; they skip if absent.tools/inventory.py: corpus inventory (python3 -I tools/inventory.py corpus/).tools/root_values.py [--write]: root style values that every corpus IDML has; regeneratessrc/idml/root_values.xml(seedocs/format/idml-values.md).- Analysis scratch scripts go in the session scratchpad, not the repo.
Anything worth keeping becomes an
inddsubcommand.
Corpus
corpus/ (git-ignored, ~3.4 GB): 357 third-party INDD files, 240 with a
sibling IDML, plus privately held samples in corpus/own/.
corpus/inventory.tsv lists version and pairing per file. Priority:
InDesign 18–21, little-endian. Some IDMLs show a different save than their
INDD; compare.py marks those pairs stale and reports trustworthy pairs
separately (rule and evidence: docs/measurement.md).
Commands
cargo test(all tests),cargo clippy --all-targets,cargo fmt.python3 -I tools/fetch_fixtures.py: fetch the fixture files beforecargo test. To add a fixture, add its pinned URL, size and SHA-256 totests/fixtures/manifest.jsonand its licence and source totests/fixtures/README.md.cargo run -q -- info <file>,indd convert in.indd out.idml,indd objects <file>,indd dump <file> <uid>...,indd uids <file>,indd xmp <file>.python3 -I tools/compare.py [--detail TAG --show N]: convert every corpus pair and compare with the reference IDML. This is the main measure of progress; run it after every change. It reports all pairs and the trustworthy ones (--stale Nlists stale pairs,--trustedlimits the tables to trustworthy pairs).- Headline numbers (value coverage, extra values, document scores, ranked
gaps; defined in
docs/measurement.md):cargo build --release && python3 -I tools/compare.py --exclude own/. It prints them last and writespairs.tsv,gaps*.tsv,values*.tsvandextras*.tsvtotarget/compare/. python3 -I tools/diff_outputs.py [OLD [NEW]]: build two revisions (defaultHEADand the working tree.) under~/.cache/indd-diff/, convert every corpus file with both and report which outputs differ (package entries compared byte for byte), plus differences in failures and warnings. Takes about a minute. A refactoring must show 0 differing outputs. Full list:~/.cache/indd-diff/diff.tsv.indd audit <file>: what the converter does not read in one document (classes, chunks, attribute IDs, unknown codes, strand kinds) and its warnings.python3 -I tools/audit_corpus.py --exclude own/ranks these over the whole corpus by files affected (tables intarget/audit/). Use it to pick what to decode next.compare.py --allalso converts every other INDD/INDT undercorpus/(any version, either byte order) and reports their failures; every run counts converter warnings by kind (--warnings Nlists N kinds).--exclude PREFIXleaves out files whose path undercorpus/starts with PREFIX; committed numbers must not include the privately held samples (--exclude own/).- Schema validation:
tools/validate.sh a.idml [b.idml...] <schemas> <jing>, orcompare.py --schemas <dir> --jing <dir>. validate.sh validates many packages in one run (one JVM per schema and 400 parts,VALIDATE_JOBSin parallel); compare.py validates its outputs 600 at a time. Runcompare.py --all --schemas … --jing …after changes to the IDML writer: it takes about 5 minutes. The IDML RelaxNG schemas (generated by InDesign, published in the metanorma/idml repo) and Jing (Maven Central) are kept outside the repo, for example in the session scratchpad; do not commit them.