Commit Graph
10 Commits
Author SHA1 Message Date
aiongg 071c0f6237 docs: update the README numbers
Trustworthy coverage 97.91 % -> 98.46 % (28,404,742 of 28,847,647
values), all pairs 96.64 % -> 97.28 %, documents at 99 % or more
186 -> 193 of 1,251, story text 37,112 -> 37,129 of 37,150 exact,
extra values 93,520 -> 93,502. The form field commit gains about
133,000 values, not 150,000 as its message says.
2026-10-10 08:15:41 +00:00
aiongg cbf12fbed2 feat: write user composite fonts and the paragraph border gap colour
Every composite font is written, in UID order. Chunk 0xCB02 is read as
four flagged strings, a u32, a u16 and the entry list, which ends the
chunk; the count search remains for other layouts (it took a wrong
position in one 12.1 document). Entries get RelativeSize and
BaselineShift (the first two f64), HorizontalScale and VerticalScale,
Locked from the u16 after the numbers (204 false), and no
CustomCharacters on the first entry of each font, the base font (1,434
of 1,434). Text attribute 0x1BFA is ParagraphBorderGapColor, 0 being
Swatch/None. Evidence in fonts.md and attributes.md.

compare.py --exclude own/, trustworthy pairs: 97.84 % -> 97.91 %
(+17,964 values, 0 lost on any key); CompositeFontEntry 7,306 -> 8,589
of 8,589 elements (AppliedFont 8,553: one document whose IDML names its
families in English), CompositeFont 1,434 of 1,434, TextRange
PSR.P.ParagraphBorderGapColor 0 -> 2,745 of 2,745. Extras 93,569 ->
93,520: the 49 CustomCharacters extras are gone.
2026-10-10 05:47:59 +00:00
aiongg 10b008e77a feat: regenerate the value files over the larger corpus
tools/root_values.py collects the object styles other than [None] and
the preferences from the pairs compare.py compares, like the root
styles, instead of the part of the corpus it had first: 1,585 pairs,
1,448 distinct IDML files. tools/element_values.py reads 1,495 distinct
IDML files. The DOM 8 to 11 merge of the DOM 12 object style frame
values is gone: the file has those versions itself.

Values that vary in the larger corpus are dropped, also where only one
document differs and the converter does not read the value from the
INDD: 209 keys, most of them preferences (EPubExportPreference,
TextDefault, PageItemDefault, IndexOptions, TextPreference), object
style values (line ends, gap colour, stroke tint, anchored object and
text wrap settings, baseline frame grid before DOM 18), Spread
PageTransitionType and PageTransitionDirection, and character style
PreviewColor. Evidence in idml-values.md.

compare.py --exclude own/, trustworthy pairs: coverage 97.79 % ->
97.09 % (-212,444 values on dropped keys, +11,501 on keys whose first
version is now earlier); documents >= 99 % 514 -> 106 of 1,251; all
pairs 96.55 % -> 95.85 %. Extras 94,610 -> 94,328. Every lost value is
on a regenerated key. No new failures; story text unchanged.
2026-10-10 02:08:29 +00:00
aiongg 39e81d0247 feat: generate the root style values from the compared pairs
tools/root_values.py collects the root styles from the pairs that
compare.py compares (little-endian, same major version, without the
privately held samples): 794 distinct IDML files, DOM 7 to 21, instead
of the 88 the corpus had first. The pair rule leaves out an IDML with no
root styles beside a newer INDD and a big-endian pair. The other object
styles and the preferences stay pinned to SOURCE; their files are
unchanged. The root table style writes every mapped attribute of its
list, since the stroke weights, spacing and end tints now vary between
files and are no longer in the values file. The root object style's
varying values (named grid, optical margin size, baseline frame grid
colour, corner radii, gutter, first baseline offset) are read from the
INDD. Evidence in idml-values.md and tables.md.

compare.py --exclude own/, trustworthy pairs: coverage 98.53 % ->
98.56 % (+2,942 values whose first DOM version is now earlier:
IncludeClass, EpubAriaRole, paragraph border options,
ParagraphKashidaWidth, shading overprint, TransformAttributeOption and
FlexLayoutAttributeOption), documents >= 99 % 278 -> 313; all pairs
96.65 % -> 96.67 %. No pair loses a reproduced value; extras unchanged
(8,968 trustworthy, 278,333 all). No new failures.
2026-10-09 19:46:58 +00:00
aiongg ea0751f1b2 docs: list the observed values the writer still writes as literals 2026-10-08 08:35:34 +00:00
aiongg c010bedef4 chore: check the WebAssembly build of the library
tools/check_wasm.sh builds the library for wasm32-unknown-unknown,
then builds a probe module (a cdylib generated under the target
directory that exports a function calling `indd::convert`) and checks
with Node.js that the module imports nothing: a conversion needs no
file system, clock or other host function. It then converts the
fetched fixtures, or the files given, in the probe module and compares
each package byte for byte with the native `indd convert` output.

No library change was needed: only `convert_file` and `read_header`
use `std::fs`, and `convert` does not call them. The 5 fixtures and 80
corpus files give the same packages in WebAssembly as natively. The
check takes about 10 s; it is documented in CLAUDE.md, CONTRIBUTING.md,
README.md and ARCHITECTURE.md. There is no hosted CI.

Idea from storytold/designcraft#98.
2026-10-08 04:18:22 +00:00
aiongg 057a4b934d test: convert damaged documents without panics or hangs
`src/hostile.rs` converts damaged inputs and fails if a conversion
panics or takes longer than 20 s:

- Synthetic documents built from a seed with `database::synthetic`: a
  document object with well-formed lists, spreads that hold spread
  layers, pages and page items, and objects of the model's classes with
  chunks of the model's IDs (taken from the hexadecimal constants of the
  model sources). Chunk data is shaped like UID lists, child lists,
  attribute lists, strings and numbers, or random. Each document is
  converted as built, with one object truncated, with bytes flipped, and
  with the file image truncated, through the model and the IDML writer.
- The fixture files, if fetched: truncated at structure boundaries and
  random lengths, and with 1 to 64 bytes flipped, a quarter of the cases
  in the header and master pages.

The cases come from a fixed seed and run on all cores, with a watchdog
for hangs. The default sweep has about 1,600 cases and takes 2 s in a
debug build and 0.5 s in a release build (16 cores).
INDD_HOSTILE_SCALE multiplies the cases and INDD_HOSTILE_SEED changes
the seed; sweeps at scale 40 with seeds 0x1DD and 7 (about 63,000 cases
each, debug build with overflow checks) found no panic or hang.

Idea from storytold/designcraft#98.
2026-10-08 04:18:22 +00:00
aiongg 758a12a92e docs: list the applied object style module in ARCHITECTURE.md 2026-10-07 22:34:22 +00:00
aiongg 14b36ee317 docs: list the graphic, export option and XMP modules in ARCHITECTURE.md 2026-10-07 21:26:39 +00:00
aiongg ae5a44db3a docs: add ARCHITECTURE.md and CONTRIBUTING.md; bring the README up to date
The README listed implemented features as not started, an old attribute
count, corpus size and numbers, and no library use. The numbers are from
compare.py --all with schema validation at this revision.
2026-10-07 20:07:28 +00:00