diff --git a/CLAUDE.md b/CLAUDE.md index 0493fad..4668e0e 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -28,6 +28,8 @@ now). Licence: MIT. - `tests/fixtures.rs`: smoke tests on committed open-licensed samples. - `tests/corpus.rs`: tests over the local `corpus/`; they skip if absent. - `tools/inventory.py`: corpus inventory (`python3 -I tools/inventory.py corpus/`). +- `tools/root_values.py [--write]`: root style values that every corpus IDML + has; regenerates `src/idml/root_values.xml` (see `docs/format/idml-values.md`). - Analysis scratch scripts go in the session scratchpad, not the repo. Anything worth keeping becomes an `indd` subcommand. diff --git a/docs/format/attributes.md b/docs/format/attributes.md index e44320b..e1ac2b1 100644 --- a/docs/format/attributes.md +++ b/docs/format/attributes.md @@ -336,7 +336,8 @@ One public template without an IDML has a style with code 4; with no reference, the converter leaves out codes other than 0 and 2. Attributes whose value never varies in the corpus cannot be located this -way and are not written; InDesign then uses its defaults. These include +way. The converter writes the root styles' values for them from IDML +observation (`idml-values.md`); styles and ranges inherit them. These include `KeepFirstLines`, `KeepLastLines`, `KeepAllLinesTogether`, `KeepWithNext`, `KeepWithPrevious`, `Skew`, `StrikeThru`, `LastLineIndent`, `RuleAbove`, `RuleAboveColor`, the `RuleBelow` indents diff --git a/docs/format/idml-values.md b/docs/format/idml-values.md new file mode 100644 index 0000000..c265c9f --- /dev/null +++ b/docs/format/idml-values.md @@ -0,0 +1,96 @@ +# Values written from IDML observation + +Some values the converter writes do not come from the INDD file. They are +values that every IDML exported by InDesign in the corpus has. They are +facts about InDesign's IDML output, not about the INDD format, and the +evidence is the corpus IDML files alone. + +Why they are written: IDML consumers apply their own defaults to +attributes that are missing. When those defaults differ from InDesign's, +the layout differs. For example, a consumer that defaults to keeping all +lines of a paragraph together breaks pages differently from a document +whose root style says `KeepAllLinesTogether="false"`. + +## Root style values + +The root styles are `[No paragraph style]`, `[No character style]`, the +object style `[None]`, the cell style `[None]` and the table style +`[No table style]`. Every corpus IDML has all five. + +`tools/root_values.py` reads `Resources/Styles.xml` of every corpus IDML +and collects each attribute of the root styles, each `Properties` child, +and each attribute of their child elements (for example +`TextFramePreference` in the object style). It keeps a value when: + +- every IDML from some DOM version on has it, with the same value, and +- the converter does not read it from the INDD. + +The kept values are in `src/idml/root_values.xml`, generated by that +tool. The converter writes them on its root styles. A value the +converter reads from the INDD takes precedence: for the paragraph and +character root styles, attributes in the text attribute table +(`attributes.md`) are left out of the file, and any attribute the INDD +supplies is written from the INDD. + +The corpus has 240 INDD files with an IDML, which are 88 distinct IDML +files. Their DOM versions are 7 (1 file), 12 (7 files) and 15 to 20 (80 +files). Values that first appear after DOM 7 are written only for +documents of that InDesign version or later (`MinimumVersion` in the +file). + +| Root style | Values | Same in | Written for version | +|---|---|---|---| +| `[No paragraph style]` | 180 | 88 of 88 files | all | +| | 14 | 87 of 87 files with DOM 12 or later | 12 and later | +| | 30 | 80 of 80 files with DOM 15 or later | 15 and later | +| `[No character style]` | 1 | 88 of 88 | all | +| | 2 | 87 of 87 (DOM 12 or later) | 12 and later | +| | 2 | 80 of 80 (DOM 15 or later) | 15 and later | +| Object style `[None]` | 75 | 88 of 88 | all | +| | 43 | 87 of 87 (DOM 12 or later) | 12 and later | +| | 16 | 80 of 80 (DOM 15 or later) | 15 and later | +| Cell style `[None]` | 1 | 88 of 88 | all | +| Table style `[No table style]` | 105 | 88 of 88 | all | +| | 10 | 87 of 87 (DOM 12 or later) | 12 and later | + +Among them are the keep settings of `[No paragraph style]`: +`KeepAllLinesTogether="false"`, `KeepFirstLines="2"`, `KeepLastLines="2"`, +`KeepWithNext="0"` and `KeepWithPrevious="false"`, in 88 of 88 files. + +**Left out because they vary between files:** + +- `[No paragraph style]`: `DiacriticPosition`, `KinsokuSet`, `Mojikumi`, + `StyleUniqueId`, `TreatIdeographicSpaceAsSpace`, and the eight corner + radii of paragraph borders and shading (12 or 1). +- `[No character style]`: `StyleUniqueId`. +- Object style `[None]`: `FrameFittingOption/FittingAlignment`, + `StoryPreference/StoryDirection`, + `TextFramePreference/ColumnRuleStrokeColor` and + `TextFramePreference/ColumnRuleStrokeWidth`. + +**Left out because the converter reads them from the INDD:** 85 +attributes of `[No paragraph style]`, listed in `attributes.md`. Some of +these are also the same in every IDML; the INDD value is written. In +every compared pair the converted root styles have all of them. + +The object, cell and table root styles get only observed values: the +converter reads no attributes of these styles from the INDD yet. The cell +and table root styles are written as new elements in +`RootCellStyleGroup` and `RootTableStyleGroup`. + +## Default numbering list + +The root paragraph style refers to `NumberingList/$ID/[Default]`. All 240 +corpus IDML files have this element in `designmap.xml`, with the same +attributes: `Name="$ID/[Default]"`, +`ContinueNumbersAcrossStories="false"` and +`ContinueNumbersAcrossDocuments="false"`. The converter writes it after +`idPkg:Styles`, where the IDML schema puts it. + +## Not yet written + +`Resources/Preferences.xml` has more elements that every corpus IDML has +(`TextDefault`, `TextPreference`, `PageItemDefault`, `StoryPreference`, +`FootnoteOption` and others). Many of their attributes are the same in +all 88 files (for example 196 of the 276 attributes of `TextDefault`), +but others vary and may be stored in the INDD. They are not written yet. diff --git a/src/idml/mod.rs b/src/idml/mod.rs index d33566a..693af2c 100644 --- a/src/idml/mod.rs +++ b/src/idml/mod.rs @@ -1,5 +1,6 @@ //! IDML package writer. +mod values; mod xml; pub mod zip; @@ -306,6 +307,7 @@ const ITEM_ATTRS: &[(u32, &str, AttrKind)] = &[ (0x5525, "GradientStrokeLength", AttrKind::Number), (0x5526, "GradientStrokeStart", AttrKind::Point), ]; +use values::Node; use xml::Xml; use crate::object::Cursor; @@ -664,6 +666,17 @@ impl Writer<'_> { x.empty("idPkg:Graphic", &[("src", "Resources/Graphic.xml".into())]); x.empty("idPkg:Fonts", &[("src", "Resources/Fonts.xml".into())]); x.empty("idPkg:Styles", &[("src", "Resources/Styles.xml".into())]); + // Present, with these values, in every corpus IDML; the root + // paragraph style refers to it. See docs/format/idml-values.md. + x.empty( + "NumberingList", + &[ + ("Self", "NumberingList/$ID/[Default]".into()), + ("Name", "$ID/[Default]".into()), + ("ContinueNumbersAcrossStories", "false".into()), + ("ContinueNumbersAcrossDocuments", "false".into()), + ], + ); x.empty( "idPkg:Preferences", &[("src", "Resources/Preferences.xml".into())], @@ -928,7 +941,12 @@ impl Writer<'_> { } fn properties(x: &mut Xml, props: &[Property]) { - if props.is_empty() { + Self::properties_with(x, props, &[]); + } + + /// `` with `props`, then the elements in `extra`. + fn properties_with(x: &mut Xml, props: &[Property], extra: &[Node]) { + if props.is_empty() && extra.is_empty() { return; } x.start("Properties"); @@ -958,6 +976,56 @@ impl Writer<'_> { } x.end(); } + for n in extra { + n.write(x); + } + x.end(); + } + + /// Observed values of a root style (see `values`) that are not in + /// `written` (attributes) or `props` (Properties children): attributes, + /// Properties children and other child elements. + fn root_values( + &self, + tag: &str, + written: &[&str], + props: &[Property], + ) -> (Vec<(String, String)>, Vec, Vec) { + let node = values::root_style(tag, self.doc.version.major); + let attrs = node + .attrs + .into_iter() + .filter(|(k, _)| !written.contains(&k.as_str())) + .collect(); + let mut extra_props = Vec::new(); + let mut children = Vec::new(); + for c in node.children { + if c.tag == "Properties" { + extra_props.extend( + c.children + .into_iter() + .filter(|p| !props.iter().any(|(n, _, _)| *n == p.tag)), + ); + } else { + children.push(c); + } + } + (attrs, extra_props, children) + } + + /// A root cell or table style, from observed values only. + fn root_table_style(&self, x: &mut Xml, tag: &str, name: &str) { + let (attrs, props, children) = self.root_values(tag, &[], &[]); + x.start(tag) + .attr("Self", format!("{tag}/$ID/{name}")) + .attr("Name", format!("$ID/{name}")); + for (k, v) in &attrs { + x.attr(k, v); + } + Self::properties_with(x, &[], &props); + for c in &children { + c.write(x); + } x.end(); } @@ -1045,19 +1113,35 @@ impl Writer<'_> { if let Some(root) = root { self.style_group_children(&mut x, root, tag, sub_tag, &mut written); } + // Styles not listed in a group (such as a built-in root + // style). Some documents have two style objects with the same + // name, which would give two elements with the same `Self`; + // the first is written. + let mut refs: std::collections::HashSet = written + .iter() + .map(|&u| self.style_ref(Some(u), paragraph)) + .collect(); for s in doc.styles.values().filter(|s| s.paragraph == paragraph) { - if !written.contains(&s.uid) { + if !written.contains(&s.uid) && refs.insert(self.style_ref(Some(s.uid), paragraph)) + { self.style_element(&mut x, s, tag); } } x.end(); } - for (tag, kind) in [ - ("RootCellStyleGroup", root_kind::CELL), - ("RootTableStyleGroup", root_kind::TABLE), + for (tag, kind, style, name) in [ + ("RootCellStyleGroup", root_kind::CELL, "CellStyle", "[None]"), + ( + "RootTableStyleGroup", + root_kind::TABLE, + "TableStyle", + "[No table style]", + ), ] { let id = root_of(kind).map_or(tag.to_string(), |g| uref(Some(g.uid))); - x.empty(tag, &[("Self", id)]); + x.start(tag).attr("Self", id); + self.root_table_style(&mut x, style, name); + x.end(); } let object_root = root_of(root_kind::OBJECT); x.start("RootObjectStyleGroup").attr( @@ -1073,7 +1157,16 @@ impl Writer<'_> { x.start("ObjectStyle") .attr("Self", format!("ObjectStyle/{}", self_name(&name))) .attr("Name", &name); - if let Some(base) = os.based_on.and_then(|b| doc.object_styles.get(&b)) { + if os.builtin && os.name == "[None]" && os.based_on.is_none() { + let (attrs, props, children) = self.root_values("ObjectStyle", &[], &[]); + for (k, v) in &attrs { + x.attr(k, v); + } + Self::properties_with(&mut x, &[], &props); + for c in &children { + c.write(&mut x); + } + } else if let Some(base) = os.based_on.and_then(|b| doc.object_styles.get(&b)) { let base_name = if base.builtin { format!("$ID/{}", base.name) } else { @@ -1138,6 +1231,17 @@ impl Writer<'_> { for (k, v) in &plain { x.attr(k, v); } + // Root styles also get the values every exported IDML has on them; + // values read from the INDD take precedence. + let mut extra = Vec::new(); + if s.builtin && s.based_on.is_none() && s.name.starts_with("[No ") { + let written: Vec<&str> = plain.iter().map(|(k, _)| *k).collect(); + let (attrs, more, _) = self.root_values(tag, &written, &props); + for (k, v) in &attrs { + x.attr(k, v); + } + extra = more; + } if let Some(base) = s.based_on.and_then(|b| doc.styles.get(&b)) { // The root "[No ... style]" is written as a string. let root = base.builtin && base.name.starts_with("[No "); @@ -1154,7 +1258,7 @@ impl Writer<'_> { ); } } - Self::properties(x, &props); + Self::properties_with(x, &props, &extra); x.end(); } diff --git a/src/idml/root_values.xml b/src/idml/root_values.xml new file mode 100644 index 0000000..fc18f30 --- /dev/null +++ b/src/idml/root_values.xml @@ -0,0 +1,90 @@ + + + + + + + Text Color + StrokeStyle/$ID/Solid + StrokeStyle/$ID/Solid + NoBalancing + Swatch/None + Swatch/None + Swatch/None + StrokeStyle/$ID/Solid + Text Color + Swatch/None + StrokeStyle/$ID/Solid + $ID/ + Nothing + Text Color + Text Color + $ID/ + Nothing + Text Color + Text Color + 1, 2, 3, 4... + NumberingList/$ID/[Default] + + + + + + + + 0000 + + + + + LightBlue + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + Swatch/None + StrokeStyle/$ID/Solid + + + + + + + + + diff --git a/src/idml/values.rs b/src/idml/values.rs new file mode 100644 index 0000000..30fa296 --- /dev/null +++ b/src/idml/values.rs @@ -0,0 +1,193 @@ +//! Values written from IDML observation: what every InDesign-exported IDML +//! in the corpus has on the root styles (`root_values.xml`, generated by +//! `tools/root_values.py`). See `docs/format/idml-values.md`. + +use super::xml::Xml; + +/// An element of the values file. +#[derive(Debug, Clone, PartialEq, Default)] +pub struct Node { + pub tag: String, + pub attrs: Vec<(String, String)>, + pub text: Option, + pub children: Vec, +} + +impl Node { + pub fn child(&self, tag: &str) -> Option<&Node> { + self.children.iter().find(|c| c.tag == tag) + } + + pub fn attr(&self, name: &str) -> Option<&str> { + self.attrs + .iter() + .find(|(k, _)| k == name) + .map(|(_, v)| v.as_str()) + } + + /// Add the attributes and children of `other` that `self` lacks. + /// Children with the same tag are merged. + fn merge(&mut self, other: &Node) { + for (k, v) in &other.attrs { + if self.attr(k).is_none() { + self.attrs.push((k.clone(), v.clone())); + } + } + if self.text.is_none() { + self.text = other.text.clone(); + } + for c in &other.children { + match self.children.iter_mut().find(|m| m.tag == c.tag) { + Some(m) => m.merge(c), + None => self.children.push(c.clone()), + } + } + } + + /// Write this element and its children. + pub fn write(&self, x: &mut Xml) { + x.start(&self.tag); + for (k, v) in &self.attrs { + x.attr(k, v); + } + if let Some(t) = &self.text { + x.text(t); + } + for c in &self.children { + c.write(x); + } + x.end(); + } +} + +const ROOT_VALUES: &str = include_str!("root_values.xml"); + +/// The observed values of root style element `tag` (`ParagraphStyle`, +/// `ObjectStyle`, ...) for a document of InDesign version `major`. +pub fn root_style(tag: &str, major: u32) -> Node { + let mut out = Node { + tag: tag.to_string(), + ..Node::default() + }; + let root = parse(ROOT_VALUES).expect("root_values.xml is well-formed"); + for block in &root.children { + let since = block + .attr("MinimumVersion") + .map_or(0, |v| v.parse().expect("MinimumVersion is a number")); + if since <= major + && let Some(n) = block.child(tag) + { + out.merge(n); + } + } + out +} + +fn unescape(s: &str) -> String { + s.replace("<", "<") + .replace(">", ">") + .replace(""", "\"") + .replace("'", "'") + .replace("&", "&") +} + +/// Parse the small, trusted XML subset of the values file: elements, +/// attributes in single or double quotes, text, comments and the XML +/// declaration. +fn parse(s: &str) -> Option { + let mut stack: Vec = vec![Node::default()]; + let mut rest = s; + while !rest.is_empty() { + if let Some(r) = rest.strip_prefix("")? + 2..]; + } else if let Some(r) = rest.strip_prefix("")? + 3..]; + } else if let Some(r) = rest.strip_prefix("')? + 1..]; + let done = stack.pop()?; + stack.last_mut()?.children.push(done); + } else if let Some(r) = rest.strip_prefix('<') { + let end = r.find('>')?; + let (mut head, empty) = match r[..end].strip_suffix('/') { + Some(h) => (h.trim_end(), true), + None => (&r[..end], false), + }; + rest = &r[end + 1..]; + let name_end = head.find(char::is_whitespace).unwrap_or(head.len()); + let mut node = Node { + tag: head[..name_end].to_string(), + ..Node::default() + }; + head = head[name_end..].trim_start(); + while !head.is_empty() { + let eq = head.find('=')?; + let key = head[..eq].trim(); + let value = head[eq + 1..].trim_start(); + let quote = value.chars().next()?; + let close = value[1..].find(quote)? + 1; + node.attrs + .push((key.to_string(), unescape(&value[1..close]))); + head = value[close + 1..].trim_start(); + } + if empty { + stack.last_mut()?.children.push(node); + } else { + stack.push(node); + } + } else { + let end = rest.find('<').unwrap_or(rest.len()); + let text = &rest[..end]; + if !text.trim().is_empty() { + stack.last_mut()?.text = Some(unescape(text)); + } + rest = &rest[end..]; + } + } + let mut top = stack.pop()?; + stack.is_empty().then_some(())?; + top.children.pop() +} + +#[cfg(test)] +mod tests { + use super::*; + + #[test] + fn parses_values_subset() { + let n = parse( + "\n\n\n\t\n\tt < u\n\n", + ) + .unwrap(); + assert_eq!(n.tag, "A"); + assert_eq!(n.attr("x"), Some("1 & 2")); + assert_eq!(n.attr("y"), Some("q\"")); + assert_eq!(n.children.len(), 2); + assert_eq!(n.child("C").unwrap().text.as_deref(), Some("t < u")); + } + + #[test] + fn gates_values_by_version() { + let old = root_style("ParagraphStyle", 7); + let new = root_style("ParagraphStyle", 21); + assert_eq!(old.attr("KeepFirstLines"), Some("2")); + assert_eq!(old.attr("KeepAllLinesTogether"), Some("false")); + assert!(old.attr("EmitCss").is_none()); + assert_eq!(new.attr("EmitCss"), Some("true")); + let props = new.child("Properties").unwrap(); + assert!(props.child("RuleAboveType").is_some()); + assert!(props.child("ParagraphBorderType").is_some()); + let os = root_style("ObjectStyle", 21); + assert!( + os.child("TextFramePreference") + .unwrap() + .attr("TextColumnCount") + .is_some() + ); + assert!( + os.child("TextFramePreference") + .unwrap() + .attr("AutoSizingType") + .is_some() + ); + } +} diff --git a/tools/root_values.py b/tools/root_values.py new file mode 100644 index 0000000..65bf3b9 --- /dev/null +++ b/tools/root_values.py @@ -0,0 +1,183 @@ +#!/usr/bin/env python3 +"""Find the values that every corpus IDML has on its root styles. + +Reads Resources/Styles.xml of every IDML in the corpus pairs (distinct +files only) and collects each attribute, Properties child and child +element attribute of the root styles. A value is kept when it is the same +in every IDML from some DOM version on and present in all of them; the +earliest such version is its minimum version. Attributes the converter +reads from the INDD (the TEXT_ATTRS table in src/idml/mod.rs) are left +out for the paragraph and character root styles. + +Usage: python3 -I tools/root_values.py [--write] +Prints the evidence counts. With --write, regenerates +src/idml/root_values.xml. See docs/format/idml-values.md. +""" + +import hashlib +import re +import sys +import xml.etree.ElementTree as ET +import zipfile +from collections import OrderedDict, defaultdict +from pathlib import Path +from xml.sax.saxutils import quoteattr + +ROOT = Path(__file__).resolve().parent.parent +OUT = ROOT / "src" / "idml" / "root_values.xml" + +TARGETS = OrderedDict([ + ("ParagraphStyle", "ParagraphStyle/$ID/[No paragraph style]"), + ("CharacterStyle", "CharacterStyle/$ID/[No character style]"), + ("ObjectStyle", "ObjectStyle/$ID/[None]"), + ("CellStyle", "CellStyle/$ID/[None]"), + ("TableStyle", "TableStyle/$ID/[No table style]"), +]) +# Written by the converter itself. +SKIP = {"Self", "Name"} + +HEADER = """ + +""" + + +def decoded_names(): + src = (ROOT / "src" / "idml" / "mod.rs").read_text() + table = src[src.index("const TEXT_ATTRS"):src.index("/// An attribute written as a")] + return set(re.findall(r'0x[0-9A-F]+,\s*"(\w+)"', table)) + + +def corpus_idmls(): + rows = (ROOT / "corpus" / "inventory.tsv").read_text().splitlines()[1:] + seen, out, pairs = set(), [], 0 + for row in rows: + path, _size, _valid, _order, _ver, _creator, dom = row.split("\t") + if not dom: + continue + pairs += 1 + idml = (ROOT / "corpus" / path).with_suffix(".idml") + digest = hashlib.md5(idml.read_bytes()).hexdigest() + if digest not in seen: + seen.add(digest) + out.append((int(dom.split(".")[0]), idml)) + return pairs, out + + +def leaves(el, path, out, order): + """Flatten an element: (path, '@attr') and (path, 'P/tag') keys.""" + for k, v in el.attrib.items(): + out[(path, "@" + k)] = v + order.setdefault((path, "@" + k), len(order)) + for c in el: + if c.tag == "Properties": + for p in c: + key = (path, "P/" + p.tag) + out[key] = re.sub(r">\s+<", "><", ET.tostring(p, encoding="unicode").strip()) + order.setdefault(key, len(order)) + else: + order.setdefault((path + (c.tag,), ""), len(order)) + leaves(c, path + (c.tag,), out, order) + + +def analyse(): + decoded = decoded_names() + pairs, idmls = corpus_idmls() + doms = [d for d, _ in idmls] + values = defaultdict(dict) + order = defaultdict(dict) + for i, (_dom, idml) in enumerate(idmls): + root = ET.fromstring(zipfile.ZipFile(idml).read("Resources/Styles.xml")) + for el in root.iter(): + if el.tag in TARGETS and TARGETS[el.tag] == el.get("Self"): + out = {} + leaves(el, (), out, order[el.tag]) + for k, v in out.items(): + values[(el.tag, k)][i] = v + kept = defaultdict(list) + left_out = defaultdict(lambda: defaultdict(list)) + for (tag, key), by_file in values.items(): + path, name = key + if not path and name[1:] in SKIP: + continue + plain = name.split("/", 1)[1] if name.startswith("P/") else name[1:] + if not path and tag in ("ParagraphStyle", "CharacterStyle") and plain in decoded: + left_out[tag]["read from the INDD"].append(plain) + continue + since = None + for m in sorted({doms[i] for i in by_file}): + files = [i for i, d in enumerate(doms) if d >= m] + if all(i in by_file for i in files): + if len({by_file[i] for i in files}) == 1: + since = (m, len(files)) + break + if since is None: + left_out[tag]["varies"].append("/".join(path + (plain,))) + continue + kept[tag].append((key, next(iter(by_file.values())), since)) + return pairs, idmls, doms, kept, left_out, order + + +def element(tag, node, indent): + attrs = "".join(f" {k}={quoteattr(v)}" for k, v in node["attrs"]) + if not node["props"] and not node["children"]: + return f"{indent}<{tag}{attrs} />\n" + s = f"{indent}<{tag}{attrs}>\n" + if node["props"]: + s += f"{indent}\t\n" + s += "".join(f"{indent}\t\t{p}\n" for p in node["props"]) + s += f"{indent}\t\n" + for t, c in node["children"].items(): + s += element(t, c, indent + "\t") + return s + f"{indent}\n" + + +def write(kept, order, first): + blocks = [] + for since in sorted({s for items in kept.values() for _, _, (s, _) in items}): + gate = "" if since == first else f' MinimumVersion="{since}"' + s = f"\t\n" + for tag in TARGETS: + items = sorted((it for it in kept[tag] if it[2][0] == since), + key=lambda it: order[tag][it[0]]) + if not items: + continue + tree = {"attrs": [], "props": [], "children": OrderedDict()} + for (path, name), v, _ in items: + node = tree + for p in path: + node = node["children"].setdefault( + p, {"attrs": [], "props": [], "children": OrderedDict()}) + if name.startswith("@"): + node["attrs"].append((name[1:], v)) + else: + node["props"].append(v) + s += element(tag, tree, "\t\t") + blocks.append(s + "\t\n") + OUT.write_text(HEADER + "\n" + "".join(blocks) + "\n") + + +def main(): + pairs, idmls, doms, kept, left_out, order = analyse() + print(f"{pairs} pairs, {len(idmls)} distinct IDML files, DOM versions {sorted(set(doms))}") + for tag in TARGETS: + counts = defaultdict(int) + for _, _, since in kept[tag]: + counts[since] += 1 + print(f"\n{tag}:") + for (since, n), k in sorted(counts.items()): + print(f" {k:4} values, the same in {n} of {n} IDML files with DOM >= {since}") + for why, names in left_out[tag].items(): + print(f" {len(names):4} left out ({why}): {', '.join(sorted(names))}") + if "--write" in sys.argv: + write(kept, order, min(doms)) + print(f"\nwrote {OUT.relative_to(ROOT)}") + + +if __name__ == "__main__": + main()