feat: write root style values that every exported IDML has

Collect the root style attributes, Properties and child elements that are
the same in every corpus IDML (tools/root_values.py) and write them on the
root paragraph, character, object, cell and table styles. Values read from
the INDD take precedence; values that first appear after DOM 7 are gated
by InDesign version. Also write the default numbering list, and write a
style only once when two style objects share a name.
This commit is contained in:
aiongg committed 2026-10-06 22:04:05 +00:00
1 parent 327eaee967
commit 36b7ae9eaf
7 files changed
+678 -9

No files matched your search

+2
View File
@@ -28,6 +28,8 @@ now). Licence: MIT.
- `tests/fixtures.rs`: smoke tests on committed open-licensed samples.
- `tests/corpus.rs`: tests over the local `corpus/`; they skip if absent.
- `tools/inventory.py`: corpus inventory (`python3 -I tools/inventory.py corpus/`).
- `tools/root_values.py [--write]`: root style values that every corpus IDML
has; regenerates `src/idml/root_values.xml` (see `docs/format/idml-values.md`).
- Analysis scratch scripts go in the session scratchpad, not the repo.
Anything worth keeping becomes an `indd` subcommand.
+2 -1
View File
@@ -336,7 +336,8 @@ One public template without an IDML has a style with code 4; with no
reference, the converter leaves out codes other than 0 and 2.
Attributes whose value never varies in the corpus cannot be located this
way and are not written; InDesign then uses its defaults. These include
way. The converter writes the root styles' values for them from IDML
observation (`idml-values.md`); styles and ranges inherit them. These include
`KeepFirstLines`, `KeepLastLines`, `KeepAllLinesTogether`,
`KeepWithNext`, `KeepWithPrevious`, `Skew`, `StrikeThru`,
`LastLineIndent`, `RuleAbove`, `RuleAboveColor`, the `RuleBelow` indents
+96
View File
@@ -0,0 +1,96 @@
# Values written from IDML observation
Some values the converter writes do not come from the INDD file. They are
values that every IDML exported by InDesign in the corpus has. They are
facts about InDesign's IDML output, not about the INDD format, and the
evidence is the corpus IDML files alone.
Why they are written: IDML consumers apply their own defaults to
attributes that are missing. When those defaults differ from InDesign's,
the layout differs. For example, a consumer that defaults to keeping all
lines of a paragraph together breaks pages differently from a document
whose root style says `KeepAllLinesTogether="false"`.
## Root style values
The root styles are `[No paragraph style]`, `[No character style]`, the
object style `[None]`, the cell style `[None]` and the table style
`[No table style]`. Every corpus IDML has all five.
`tools/root_values.py` reads `Resources/Styles.xml` of every corpus IDML
and collects each attribute of the root styles, each `Properties` child,
and each attribute of their child elements (for example
`TextFramePreference` in the object style). It keeps a value when:
- every IDML from some DOM version on has it, with the same value, and
- the converter does not read it from the INDD.
The kept values are in `src/idml/root_values.xml`, generated by that
tool. The converter writes them on its root styles. A value the
converter reads from the INDD takes precedence: for the paragraph and
character root styles, attributes in the text attribute table
(`attributes.md`) are left out of the file, and any attribute the INDD
supplies is written from the INDD.
The corpus has 240 INDD files with an IDML, which are 88 distinct IDML
files. Their DOM versions are 7 (1 file), 12 (7 files) and 15 to 20 (80
files). Values that first appear after DOM 7 are written only for
documents of that InDesign version or later (`MinimumVersion` in the
file).
| Root style | Values | Same in | Written for version |
|---|---|---|---|
| `[No paragraph style]` | 180 | 88 of 88 files | all |
| | 14 | 87 of 87 files with DOM 12 or later | 12 and later |
| | 30 | 80 of 80 files with DOM 15 or later | 15 and later |
| `[No character style]` | 1 | 88 of 88 | all |
| | 2 | 87 of 87 (DOM 12 or later) | 12 and later |
| | 2 | 80 of 80 (DOM 15 or later) | 15 and later |
| Object style `[None]` | 75 | 88 of 88 | all |
| | 43 | 87 of 87 (DOM 12 or later) | 12 and later |
| | 16 | 80 of 80 (DOM 15 or later) | 15 and later |
| Cell style `[None]` | 1 | 88 of 88 | all |
| Table style `[No table style]` | 105 | 88 of 88 | all |
| | 10 | 87 of 87 (DOM 12 or later) | 12 and later |
Among them are the keep settings of `[No paragraph style]`:
`KeepAllLinesTogether="false"`, `KeepFirstLines="2"`, `KeepLastLines="2"`,
`KeepWithNext="0"` and `KeepWithPrevious="false"`, in 88 of 88 files.
**Left out because they vary between files:**
- `[No paragraph style]`: `DiacriticPosition`, `KinsokuSet`, `Mojikumi`,
`StyleUniqueId`, `TreatIdeographicSpaceAsSpace`, and the eight corner
radii of paragraph borders and shading (12 or 1).
- `[No character style]`: `StyleUniqueId`.
- Object style `[None]`: `FrameFittingOption/FittingAlignment`,
`StoryPreference/StoryDirection`,
`TextFramePreference/ColumnRuleStrokeColor` and
`TextFramePreference/ColumnRuleStrokeWidth`.
**Left out because the converter reads them from the INDD:** 85
attributes of `[No paragraph style]`, listed in `attributes.md`. Some of
these are also the same in every IDML; the INDD value is written. In
every compared pair the converted root styles have all of them.
The object, cell and table root styles get only observed values: the
converter reads no attributes of these styles from the INDD yet. The cell
and table root styles are written as new elements in
`RootCellStyleGroup` and `RootTableStyleGroup`.
## Default numbering list
The root paragraph style refers to `NumberingList/$ID/[Default]`. All 240
corpus IDML files have this element in `designmap.xml`, with the same
attributes: `Name="$ID/[Default]"`,
`ContinueNumbersAcrossStories="false"` and
`ContinueNumbersAcrossDocuments="false"`. The converter writes it after
`idPkg:Styles`, where the IDML schema puts it.
## Not yet written
`Resources/Preferences.xml` has more elements that every corpus IDML has
(`TextDefault`, `TextPreference`, `PageItemDefault`, `StoryPreference`,
`FootnoteOption` and others). Many of their attributes are the same in
all 88 files (for example 196 of the 276 attributes of `TextDefault`),
but others vary and may be stored in the INDD. They are not written yet.
+112 -8
View File
@@ -1,5 +1,6 @@
//! IDML package writer.
mod values;
mod xml;
pub mod zip;
@@ -306,6 +307,7 @@ const ITEM_ATTRS: &[(u32, &str, AttrKind)] = &[
(0x5525, "GradientStrokeLength", AttrKind::Number),
(0x5526, "GradientStrokeStart", AttrKind::Point),
];
use values::Node;
use xml::Xml;
use crate::object::Cursor;
@@ -664,6 +666,17 @@ impl Writer<'_> {
x.empty("idPkg:Graphic", &[("src", "Resources/Graphic.xml".into())]);
x.empty("idPkg:Fonts", &[("src", "Resources/Fonts.xml".into())]);
x.empty("idPkg:Styles", &[("src", "Resources/Styles.xml".into())]);
// Present, with these values, in every corpus IDML; the root
// paragraph style refers to it. See docs/format/idml-values.md.
x.empty(
"NumberingList",
&[
("Self", "NumberingList/$ID/[Default]".into()),
("Name", "$ID/[Default]".into()),
("ContinueNumbersAcrossStories", "false".into()),
("ContinueNumbersAcrossDocuments", "false".into()),
],
);
x.empty(
"idPkg:Preferences",
&[("src", "Resources/Preferences.xml".into())],
@@ -928,7 +941,12 @@ impl Writer<'_> {
}
fn properties(x: &mut Xml, props: &[Property]) {
if props.is_empty() {
Self::properties_with(x, props, &[]);
}
/// `<Properties>` with `props`, then the elements in `extra`.
fn properties_with(x: &mut Xml, props: &[Property], extra: &[Node]) {
if props.is_empty() && extra.is_empty() {
return;
}
x.start("Properties");
@@ -958,6 +976,56 @@ impl Writer<'_> {
}
x.end();
}
for n in extra {
n.write(x);
}
x.end();
}
/// Observed values of a root style (see `values`) that are not in
/// `written` (attributes) or `props` (Properties children): attributes,
/// Properties children and other child elements.
fn root_values(
&self,
tag: &str,
written: &[&str],
props: &[Property],
) -> (Vec<(String, String)>, Vec<Node>, Vec<Node>) {
let node = values::root_style(tag, self.doc.version.major);
let attrs = node
.attrs
.into_iter()
.filter(|(k, _)| !written.contains(&k.as_str()))
.collect();
let mut extra_props = Vec::new();
let mut children = Vec::new();
for c in node.children {
if c.tag == "Properties" {
extra_props.extend(
c.children
.into_iter()
.filter(|p| !props.iter().any(|(n, _, _)| *n == p.tag)),
);
} else {
children.push(c);
}
}
(attrs, extra_props, children)
}
/// A root cell or table style, from observed values only.
fn root_table_style(&self, x: &mut Xml, tag: &str, name: &str) {
let (attrs, props, children) = self.root_values(tag, &[], &[]);
x.start(tag)
.attr("Self", format!("{tag}/$ID/{name}"))
.attr("Name", format!("$ID/{name}"));
for (k, v) in &attrs {
x.attr(k, v);
}
Self::properties_with(x, &[], &props);
for c in &children {
c.write(x);
}
x.end();
}
@@ -1045,19 +1113,35 @@ impl Writer<'_> {
if let Some(root) = root {
self.style_group_children(&mut x, root, tag, sub_tag, &mut written);
}
// Styles not listed in a group (such as a built-in root
// style). Some documents have two style objects with the same
// name, which would give two elements with the same `Self`;
// the first is written.
let mut refs: std::collections::HashSet<String> = written
.iter()
.map(|&u| self.style_ref(Some(u), paragraph))
.collect();
for s in doc.styles.values().filter(|s| s.paragraph == paragraph) {
if !written.contains(&s.uid) {
if !written.contains(&s.uid) && refs.insert(self.style_ref(Some(s.uid), paragraph))
{
self.style_element(&mut x, s, tag);
}
}
x.end();
}
for (tag, kind) in [
("RootCellStyleGroup", root_kind::CELL),
("RootTableStyleGroup", root_kind::TABLE),
for (tag, kind, style, name) in [
("RootCellStyleGroup", root_kind::CELL, "CellStyle", "[None]"),
(
"RootTableStyleGroup",
root_kind::TABLE,
"TableStyle",
"[No table style]",
),
] {
let id = root_of(kind).map_or(tag.to_string(), |g| uref(Some(g.uid)));
x.empty(tag, &[("Self", id)]);
x.start(tag).attr("Self", id);
self.root_table_style(&mut x, style, name);
x.end();
}
let object_root = root_of(root_kind::OBJECT);
x.start("RootObjectStyleGroup").attr(
@@ -1073,7 +1157,16 @@ impl Writer<'_> {
x.start("ObjectStyle")
.attr("Self", format!("ObjectStyle/{}", self_name(&name)))
.attr("Name", &name);
if let Some(base) = os.based_on.and_then(|b| doc.object_styles.get(&b)) {
if os.builtin && os.name == "[None]" && os.based_on.is_none() {
let (attrs, props, children) = self.root_values("ObjectStyle", &[], &[]);
for (k, v) in &attrs {
x.attr(k, v);
}
Self::properties_with(&mut x, &[], &props);
for c in &children {
c.write(&mut x);
}
} else if let Some(base) = os.based_on.and_then(|b| doc.object_styles.get(&b)) {
let base_name = if base.builtin {
format!("$ID/{}", base.name)
} else {
@@ -1138,6 +1231,17 @@ impl Writer<'_> {
for (k, v) in &plain {
x.attr(k, v);
}
// Root styles also get the values every exported IDML has on them;
// values read from the INDD take precedence.
let mut extra = Vec::new();
if s.builtin && s.based_on.is_none() && s.name.starts_with("[No ") {
let written: Vec<&str> = plain.iter().map(|(k, _)| *k).collect();
let (attrs, more, _) = self.root_values(tag, &written, &props);
for (k, v) in &attrs {
x.attr(k, v);
}
extra = more;
}
if let Some(base) = s.based_on.and_then(|b| doc.styles.get(&b)) {
// The root "[No ... style]" is written as a string.
let root = base.builtin && base.name.starts_with("[No ");
@@ -1154,7 +1258,7 @@ impl Writer<'_> {
);
}
}
Self::properties(x, &props);
Self::properties_with(x, &props, &extra);
x.end();
}
+90
View File
@@ -0,0 +1,90 @@
<?xml version="1.0" encoding="UTF-8"?>
<!--
Values that every IDML exported by InDesign in the corpus has on the root
styles. The converter writes them on its root styles; values it reads from
the INDD take precedence. Values in a block with MinimumVersion are written
only for documents of that InDesign version or later. Generated by
tools/root_values.py; evidence in docs/format/idml-values.md.
-->
<RootStyleValues>
<Values>
<ParagraphStyle Imported="false" PageNumberType="AutoPageNumber" HyphenateLadderLimit="3" HyphenateAfterFirst="2" HyphenateBeforeLast="2" HyphenateCapitalizedWords="true" HyphenateWordsLongerThan="5" DesiredWordSpacing="100" DesiredLetterSpacing="0" DesiredGlyphScaling="100" MaximumGlyphScaling="100" MinimumGlyphScaling="100" KeepAllLinesTogether="false" KeepWithNext="0" KeepFirstLines="2" KeepLastLines="2" StrikeThru="false" CharacterAlignment="AlignEmCenter" StrokeTint="-1" OverprintStroke="false" OverprintFill="false" GradientStrokeAngle="0" GradientStrokeLength="-1" GradientStrokeStart="0 0" Skew="0" RuleAboveTint="-1" RuleBelowLeftIndent="0" RuleBelowRightIndent="0" RuleBelowWidth="ColumnWidth" RuleAboveOverprint="false" RuleBelowOverprint="false" RuleAbove="false" LastLineIndent="0" HyphenateLastWord="true" OTFOrdinal="false" OTFFraction="false" OTFDiscretionaryLigature="false" OTFTitling="false" RuleAboveGapTint="-1" RuleAboveGapOverprint="false" RuleBelowGapTint="-1" RuleBelowGapOverprint="false" PositionalForm="None" OTFMark="true" HyphenWeight="5" OTFLocale="true" HyphenateAcrossColumns="true" KeepRuleAboveInFrame="false" IgnoreEdgeAlignment="false" OTFSlashedZero="false" OTFStylisticSets="0" OTFHistorical="false" UnderlineGapOverprint="false" UnderlineGapTint="-1" UnderlineOverprint="false" UnderlineTint="-1" StrikeThroughGapOverprint="false" StrikeThroughGapTint="-1" StrikeThroughOffset="-9999" StrikeThroughOverprint="false" StrikeThroughTint="-1" StrikeThroughWeight="-9999" StrokeAlignment="OutsideAlignment" SplitColumnOutsideGutter="0" KeepWithPrevious="false" SpanColumnMinSpaceBefore="0" OTFSwash="false" Tsume="0" LeadingAki="-1" TrailingAki="-1" KinsokuType="KinsokuPushInFirst" KinsokuHangType="None" BunriKinshi="true" RubyOpenTypePro="true" RubyFontSize="-1" RubyAlignment="RubyJIS" RubyType="PerCharacterRuby" RubyParentSpacing="RubyParent121Aki" RubyXScale="100" RubyYScale="100" RubyXOffset="0" RubyYOffset="0" RubyPosition="AboveRight" RubyAutoAlign="true" RubyParentOverhangAmount="RubyOverhangOneRuby" RubyOverhang="false" RubyAutoScaling="false" RubyParentScalingPercent="66" RubyTint="-1" RubyOverprintFill="Auto" RubyStrokeTint="-1" RubyOverprintStroke="Auto" RubyWeight="-1" KentenKind="None" KentenFontSize="-1" KentenXScale="100" KentenYScale="100" KentenPlacement="0" KentenAlignment="AlignKentenCenter" KentenPosition="AboveRight" KentenCustomCharacter="" KentenCharacterSet="CharacterInput" KentenTint="-1" KentenOverprintFill="Auto" KentenStrokeTint="-1" KentenOverprintStroke="Auto" KentenWeight="-1" Tatechuyoko="false" TatechuyokoXOffset="0" TatechuyokoYOffset="0" AutoTcy="0" AutoTcyIncludeRoman="false" Jidori="0" GridGyoudori="0" CharacterRotation="0" RotateSingleByteCharacters="false" Rensuuji="true" ShataiMagnification="0" ShataiDegreeAngle="4500" ShataiAdjustTsume="true" ShataiAdjustRotation="false" Warichu="false" WarichuLines="2" WarichuSize="50" WarichuLineSpacing="0" WarichuAlignment="Auto" WarichuCharsBeforeBreak="2" WarichuCharsAfterBreak="2" OTFHVKana="false" OTFProportionalMetrics="false" OTFRomanItalics="false" LeadingModel="LeadingModelAkiBelow" ScaleAffectsLineHeight="false" ParagraphGyoudori="false" CjkGridTracking="false" GlyphForm="None" RubyAutoTcyDigits="0" RubyAutoTcyIncludeRoman="false" RubyAutoTcyAutoScale="true" AllowArbitraryHyphenation="false" NumberingStartAt="1" NumberingLevel="1" NumberingApplyRestartPolicy="true" BulletsAlignment="LeftAlign" NumberingAlignment="LeftAlign" BulletsTextAfter="^t" DigitsType="DefaultDigits" Kashidas="DefaultKashidas" CharacterDirection="DefaultDirection" ParagraphDirection="LeftToRightDirection" ParagraphJustification="DefaultJustification" XOffsetDiacritic="0" YOffsetDiacritic="0" OTFOverlapSwash="false" OTFStylisticAlternate="false" OTFJustificationAlternate="false" OTFStretchedAlternate="false" KeyboardDirection="DefaultDirection">
<Properties>
<RuleAboveColor type="string">Text Color</RuleAboveColor>
<RuleAboveType type="object">StrokeStyle/$ID/Solid</RuleAboveType>
<RuleBelowType type="object">StrokeStyle/$ID/Solid</RuleBelowType>
<BalanceRaggedLines type="enumeration">NoBalancing</BalanceRaggedLines>
<RuleAboveGapColor type="object">Swatch/None</RuleAboveGapColor>
<RuleBelowGapColor type="object">Swatch/None</RuleBelowGapColor>
<UnderlineGapColor type="object">Swatch/None</UnderlineGapColor>
<UnderlineType type="object">StrokeStyle/$ID/Solid</UnderlineType>
<StrikeThroughColor type="string">Text Color</StrikeThroughColor>
<StrikeThroughGapColor type="object">Swatch/None</StrikeThroughGapColor>
<StrikeThroughType type="object">StrokeStyle/$ID/Solid</StrikeThroughType>
<RubyFont type="string">$ID/</RubyFont>
<RubyFontStyle type="enumeration">Nothing</RubyFontStyle>
<RubyFill type="string">Text Color</RubyFill>
<RubyStroke type="string">Text Color</RubyStroke>
<KentenFont type="string">$ID/</KentenFont>
<KentenFontStyle type="enumeration">Nothing</KentenFontStyle>
<KentenFillColor type="string">Text Color</KentenFillColor>
<KentenStrokeColor type="string">Text Color</KentenStrokeColor>
<NumberingFormat type="string">1, 2, 3, 4...</NumberingFormat>
<AppliedNumberingList type="object">NumberingList/$ID/[Default]</AppliedNumberingList>
<NumberingRestartPolicies RestartPolicy="AnyPreviousLevel" LowerLevel="0" UpperLevel="0" />
</Properties>
</ParagraphStyle>
<CharacterStyle Imported="false" />
<ObjectStyle TopLeftCornerOption="None" TopRightCornerOption="None" BottomLeftCornerOption="None" BottomRightCornerOption="None" TopLeftCornerRadius="12" TopRightCornerRadius="12" BottomLeftCornerRadius="12" BottomRightCornerRadius="12" AppliedParagraphStyle="ParagraphStyle/$ID/[No paragraph style]" CornerRadius="12" FillColor="Swatch/None" FillTint="-1" StrokeWeight="0" MiterLimit="4" EndCap="ButtEndCap" EndJoin="MiterEndJoin" StrokeType="StrokeStyle/$ID/Solid" LeftLineEnd="None" RightLineEnd="None" StrokeColor="Swatch/None" StrokeTint="-1" GapColor="Swatch/None" GapTint="-1" StrokeAlignment="CenterAlignment" Nonprinting="false" GradientFillAngle="0" GradientStrokeAngle="0" AppliedNamedGrid="n" CornerOption="None">
<TextFramePreference TextColumnCount="1" TextColumnGutter="12" TextColumnFixedWidth="144" UseFixedColumnWidth="false" FirstBaselineOffset="AscentOffset" MinimumFirstBaselineOffset="0" VerticalJustification="TopAlign" VerticalThreshold="0" IgnoreWrap="false" VerticalBalanceColumns="false">
<Properties>
<InsetSpacing type="list"><ListItem type="unit">0</ListItem><ListItem type="unit">0</ListItem><ListItem type="unit">0</ListItem><ListItem type="unit">0</ListItem></InsetSpacing>
</Properties>
</TextFramePreference>
<BaselineFrameGridOption UseCustomBaselineFrameGrid="false" StartingOffsetForBaselineFrameGrid="0" BaselineFrameGridRelativeOption="TopOfInset" BaselineFrameGridIncrement="12">
<Properties>
<BaselineFrameGridColor type="enumeration">LightBlue</BaselineFrameGridColor>
</Properties>
</BaselineFrameGridOption>
<AnchoredObjectSetting AnchoredPosition="InlinePosition" SpineRelative="false" LockPosition="false" PinPosition="true" AnchorPoint="BottomRightAnchor" HorizontalAlignment="LeftAlign" HorizontalReferencePoint="TextFrame" VerticalAlignment="BottomAlign" VerticalReferencePoint="LineBaseline" AnchorXoffset="0" AnchorYoffset="0" AnchorSpaceAbove="0" />
<TextWrapPreference Inverse="false" ApplyToMasterPageOnly="false" TextWrapSide="BothSides" TextWrapMode="None">
<Properties>
<TextWrapOffset Top="0" Left="0" Bottom="0" Right="0" />
</Properties>
<ContourOption ContourType="SameAsClipping" IncludeInsideEdges="false" ContourPathName="$ID/" />
</TextWrapPreference>
<StoryPreference OpticalMarginAlignment="false" OpticalMarginSize="12" FrameType="TextFrameType" StoryOrientation="Horizontal" />
<FrameFittingOption AutoFit="false" LeftCrop="0" TopCrop="0" RightCrop="0" BottomCrop="0" FittingOnEmptyFrame="None" />
</ObjectStyle>
<CellStyle AppliedParagraphStyle="ParagraphStyle/$ID/[No paragraph style]" />
<TableStyle StrokeOrder="BestJoins" TopBorderStrokeWeight="1" TopBorderStrokeType="StrokeStyle/$ID/Solid" TopBorderStrokeColor="Color/Black" TopBorderStrokeTint="100" TopBorderStrokeOverprint="false" TopBorderStrokeGapColor="Color/Paper" TopBorderStrokeGapTint="100" TopBorderStrokeGapOverprint="false" LeftBorderStrokeWeight="1" LeftBorderStrokeType="StrokeStyle/$ID/Solid" LeftBorderStrokeColor="Color/Black" LeftBorderStrokeTint="100" LeftBorderStrokeOverprint="false" LeftBorderStrokeGapColor="Color/Paper" LeftBorderStrokeGapTint="100" LeftBorderStrokeGapOverprint="false" BottomBorderStrokeWeight="1" BottomBorderStrokeType="StrokeStyle/$ID/Solid" BottomBorderStrokeColor="Color/Black" BottomBorderStrokeTint="100" BottomBorderStrokeOverprint="false" BottomBorderStrokeGapColor="Color/Paper" BottomBorderStrokeGapTint="100" BottomBorderStrokeGapOverprint="false" RightBorderStrokeWeight="1" RightBorderStrokeType="StrokeStyle/$ID/Solid" RightBorderStrokeColor="Color/Black" RightBorderStrokeTint="100" RightBorderStrokeOverprint="false" RightBorderStrokeGapColor="Color/Paper" RightBorderStrokeGapTint="100" RightBorderStrokeGapOverprint="false" SpaceBefore="4" SpaceAfter="-4" SkipFirstAlternatingStrokeRows="0" SkipLastAlternatingStrokeRows="0" StartRowStrokeCount="0" StartRowStrokeColor="Color/Black" StartRowStrokeWeight="1" StartRowStrokeType="StrokeStyle/$ID/Solid" StartRowStrokeTint="100" StartRowStrokeGapOverprint="false" StartRowStrokeGapColor="Color/Paper" StartRowStrokeGapTint="100" StartRowStrokeOverprint="false" EndRowStrokeCount="0" EndRowStrokeColor="Color/Black" EndRowStrokeWeight="0.25" EndRowStrokeType="StrokeStyle/$ID/Solid" EndRowStrokeTint="100" EndRowStrokeOverprint="false" EndRowStrokeGapColor="Color/Paper" EndRowStrokeGapTint="100" EndRowStrokeGapOverprint="false" SkipFirstAlternatingStrokeColumns="0" SkipLastAlternatingStrokeColumns="0" StartColumnStrokeCount="0" StartColumnStrokeColor="Color/Black" StartColumnStrokeWeight="1" StartColumnStrokeType="StrokeStyle/$ID/Solid" StartColumnStrokeTint="100" StartColumnStrokeOverprint="false" StartColumnStrokeGapColor="Color/Paper" StartColumnStrokeGapTint="100" StartColumnStrokeGapOverprint="false" EndColumnStrokeCount="0" EndColumnStrokeColor="Color/Black" EndColumnStrokeWeight="0.25" EndColumnLineStyle="StrokeStyle/$ID/Solid" EndColumnStrokeTint="100" EndColumnStrokeOverprint="false" EndColumnStrokeGapColor="Color/Paper" EndColumnStrokeGapTint="100" EndColumnStrokeGapOverprint="false" ColumnFillsPriority="false" SkipFirstAlternatingFillRows="0" SkipLastAlternatingFillRows="0" StartRowFillColor="Color/Black" StartRowFillCount="0" StartRowFillTint="20" StartRowFillOverprint="false" EndRowFillCount="0" EndRowFillColor="Swatch/None" EndRowFillTint="100" EndRowFillOverprint="false" SkipFirstAlternatingFillColumns="0" SkipLastAlternatingFillColumns="0" StartColumnFillCount="0" StartColumnFillColor="Color/Black" StartColumnFillTint="20" StartColumnFillOverprint="false" EndColumnFillCount="0" EndColumnFillColor="Swatch/None" EndColumnFillTint="100" EndColumnFillOverprint="false" HeaderRegionSameAsBodyRegion="true" FooterRegionSameAsBodyRegion="true" LeftColumnRegionSameAsBodyRegion="true" RightColumnRegionSameAsBodyRegion="true" HeaderRegionCellStyle="n" FooterRegionCellStyle="n" LeftColumnRegionCellStyle="n" RightColumnRegionCellStyle="n" BodyRegionCellStyle="CellStyle/$ID/[None]" />
</Values>
<Values MinimumVersion="12">
<ParagraphStyle SplitDocument="false" EmitCss="true" EmptyNestedStyles="true" EmptyLineStyles="true" EmptyGrepStyles="true" ParagraphShadingOverprint="false" ParagraphShadingWidth="ColumnWidth" ParagraphShadingClipToFrame="false" ParagraphShadingSuppressPrinting="false" ParagraphShadingLeftOffset="0" ParagraphShadingRightOffset="0" ParagraphShadingTopOrigin="AscentTopOrigin" ParagraphShadingBottomOrigin="DescentBottomOrigin" ParagraphKashidaWidth="2" />
<CharacterStyle SplitDocument="false" EmitCss="true" />
<ObjectStyle EmitCss="true" ArrowHeadAlignment="InsidePath" LeftArrowHeadScale="100" RightArrowHeadScale="100">
<ObjectExportOption AltTextSourceType="SourceXMLStructure" ActualTextSourceType="SourceXMLStructure" CustomAltText="$ID/" CustomActualText="$ID/" ApplyTagType="TagFromStructure" ImageConversionType="JPEG" ImageExportResolution="Ppi300" GIFOptionsPalette="AdaptivePalette" GIFOptionsInterlaced="true" JPEGOptionsQuality="High" JPEGOptionsFormat="BaselineEncoding" ImageAlignment="AlignLeft" ImageSpaceBefore="0" ImageSpaceAfter="0" UseImagePageBreak="false" ImagePageBreak="PageBreakBefore" CustomImageAlignment="false" SpaceUnit="CssPixel" CustomLayout="false" CustomLayoutType="AlignmentAndSpacing" EpubType="$ID/" SizeType="DefaultSize" CustomSize="$ID/" PreserveAppearanceFromLayout="PreserveAppearanceDefault">
<Properties>
<AltMetadataProperty NamespacePrefix="$ID/" PropertyPath="$ID/" />
<ActualMetadataProperty NamespacePrefix="$ID/" PropertyPath="$ID/" />
</Properties>
</ObjectExportOption>
<TextFramePreference UseFlexibleColumnWidth="false" TextColumnMaxWidth="0" AutoSizingType="Off" AutoSizingReferencePoint="CenterPoint" UseMinimumHeightForAutoSizing="false" MinimumHeightForAutoSizing="0" UseMinimumWidthForAutoSizing="false" MinimumWidthForAutoSizing="0" UseNoLineBreaksForAutoSizing="false" />
<TextFrameFootnoteOptionsObject EnableOverrides="false" SpanFootnotesAcross="false" MinimumSpacingOption="12" SpaceBetweenFootnotes="6" />
</ObjectStyle>
<TableStyle GraphicLeftInset="0" GraphicTopInset="0" GraphicRightInset="0" GraphicBottomInset="0" ClipContentToGraphicCell="false" TextTopInset="4" TextLeftInset="4" TextBottomInset="4" TextRightInset="4" ClipContentToTextCell="false" />
</Values>
<Values MinimumVersion="15">
<ParagraphStyle IncludeClass="true" ExtendedKeyboardShortcut="0 0 0" ParagraphBorderTint="-1" ParagraphBorderOverprint="false" ParagraphBorderGapTint="-1" ParagraphBorderGapOverprint="false" ParagraphBorderLeftOffset="0" ParagraphBorderRightOffset="0" ParagraphBorderStrokeEndJoin="MiterEndJoin" ParagraphBorderTopLeftCornerOption="None" ParagraphBorderTopRightCornerOption="None" ParagraphBorderBottomLeftCornerOption="None" ParagraphBorderBottomRightCornerOption="None" ParagraphShadingTopLeftCornerOption="None" ParagraphShadingTopRightCornerOption="None" ParagraphShadingBottomLeftCornerOption="None" ParagraphShadingBottomRightCornerOption="None" ParagraphBorderStrokeEndCap="ButtEndCap" ParagraphBorderWidth="ColumnWidth" ParagraphBorderTopOrigin="AscentTopOrigin" ParagraphBorderBottomOrigin="DescentBottomOrigin" ParagraphBorderTopLineWeight="1" ParagraphBorderBottomLineWeight="1" ParagraphBorderLeftLineWeight="1" ParagraphBorderRightLineWeight="1" ParagraphBorderDisplayIfSplits="false" MergeConsecutiveParaBorders="true" ProviderHyphenationStyle="HyphAll">
<Properties>
<ParagraphBorderGapColor type="object">Swatch/None</ParagraphBorderGapColor>
<ParagraphBorderType type="object">StrokeStyle/$ID/Solid</ParagraphBorderType>
</Properties>
</ParagraphStyle>
<CharacterStyle IncludeClass="true" ExtendedKeyboardShortcut="0 0 0" />
<ObjectStyle IncludeClass="true">
<TransformAttributeOption TransformAttrLeftReference="PageEdgeReference" TransformAttrTopReference="PageEdgeReference" TransformAttrRefAnchorPoint="TopLeftAnchor" />
<TextFramePreference FootnotesEnableOverrides="false" FootnotesSpanAcrossColumns="false" FootnotesMinimumSpacing="12" FootnotesSpaceBetween="6" ColumnRuleOverride="false" ColumnRuleOffset="0" ColumnRuleTopInset="0" ColumnRuleInsetChainOverride="true" ColumnRuleBottomInset="0" ColumnRuleStrokeType="StrokeStyle/$ID/Solid" ColumnRuleStrokeTint="100" ColumnRuleOverprintOverride="false" />
</ObjectStyle>
</Values>
</RootStyleValues>
+193
View File
@@ -0,0 +1,193 @@
//! Values written from IDML observation: what every InDesign-exported IDML
//! in the corpus has on the root styles (`root_values.xml`, generated by
//! `tools/root_values.py`). See `docs/format/idml-values.md`.
use super::xml::Xml;
/// An element of the values file.
#[derive(Debug, Clone, PartialEq, Default)]
pub struct Node {
pub tag: String,
pub attrs: Vec<(String, String)>,
pub text: Option<String>,
pub children: Vec<Node>,
}
impl Node {
pub fn child(&self, tag: &str) -> Option<&Node> {
self.children.iter().find(|c| c.tag == tag)
}
pub fn attr(&self, name: &str) -> Option<&str> {
self.attrs
.iter()
.find(|(k, _)| k == name)
.map(|(_, v)| v.as_str())
}
/// Add the attributes and children of `other` that `self` lacks.
/// Children with the same tag are merged.
fn merge(&mut self, other: &Node) {
for (k, v) in &other.attrs {
if self.attr(k).is_none() {
self.attrs.push((k.clone(), v.clone()));
}
}
if self.text.is_none() {
self.text = other.text.clone();
}
for c in &other.children {
match self.children.iter_mut().find(|m| m.tag == c.tag) {
Some(m) => m.merge(c),
None => self.children.push(c.clone()),
}
}
}
/// Write this element and its children.
pub fn write(&self, x: &mut Xml) {
x.start(&self.tag);
for (k, v) in &self.attrs {
x.attr(k, v);
}
if let Some(t) = &self.text {
x.text(t);
}
for c in &self.children {
c.write(x);
}
x.end();
}
}
const ROOT_VALUES: &str = include_str!("root_values.xml");
/// The observed values of root style element `tag` (`ParagraphStyle`,
/// `ObjectStyle`, ...) for a document of InDesign version `major`.
pub fn root_style(tag: &str, major: u32) -> Node {
let mut out = Node {
tag: tag.to_string(),
..Node::default()
};
let root = parse(ROOT_VALUES).expect("root_values.xml is well-formed");
for block in &root.children {
let since = block
.attr("MinimumVersion")
.map_or(0, |v| v.parse().expect("MinimumVersion is a number"));
if since <= major
&& let Some(n) = block.child(tag)
{
out.merge(n);
}
}
out
}
fn unescape(s: &str) -> String {
s.replace("&lt;", "<")
.replace("&gt;", ">")
.replace("&quot;", "\"")
.replace("&apos;", "'")
.replace("&amp;", "&")
}
/// Parse the small, trusted XML subset of the values file: elements,
/// attributes in single or double quotes, text, comments and the XML
/// declaration.
fn parse(s: &str) -> Option<Node> {
let mut stack: Vec<Node> = vec![Node::default()];
let mut rest = s;
while !rest.is_empty() {
if let Some(r) = rest.strip_prefix("<?") {
rest = &r[r.find("?>")? + 2..];
} else if let Some(r) = rest.strip_prefix("<!--") {
rest = &r[r.find("-->")? + 3..];
} else if let Some(r) = rest.strip_prefix("</") {
rest = &r[r.find('>')? + 1..];
let done = stack.pop()?;
stack.last_mut()?.children.push(done);
} else if let Some(r) = rest.strip_prefix('<') {
let end = r.find('>')?;
let (mut head, empty) = match r[..end].strip_suffix('/') {
Some(h) => (h.trim_end(), true),
None => (&r[..end], false),
};
rest = &r[end + 1..];
let name_end = head.find(char::is_whitespace).unwrap_or(head.len());
let mut node = Node {
tag: head[..name_end].to_string(),
..Node::default()
};
head = head[name_end..].trim_start();
while !head.is_empty() {
let eq = head.find('=')?;
let key = head[..eq].trim();
let value = head[eq + 1..].trim_start();
let quote = value.chars().next()?;
let close = value[1..].find(quote)? + 1;
node.attrs
.push((key.to_string(), unescape(&value[1..close])));
head = value[close + 1..].trim_start();
}
if empty {
stack.last_mut()?.children.push(node);
} else {
stack.push(node);
}
} else {
let end = rest.find('<').unwrap_or(rest.len());
let text = &rest[..end];
if !text.trim().is_empty() {
stack.last_mut()?.text = Some(unescape(text));
}
rest = &rest[end..];
}
}
let mut top = stack.pop()?;
stack.is_empty().then_some(())?;
top.children.pop()
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn parses_values_subset() {
let n = parse(
"<?xml version=\"1.0\"?>\n<!-- c -->\n<A x=\"1 &amp; 2\" y='q\"'>\n\t<B />\n\t<C type=\"string\">t &lt; u</C>\n</A>\n",
)
.unwrap();
assert_eq!(n.tag, "A");
assert_eq!(n.attr("x"), Some("1 & 2"));
assert_eq!(n.attr("y"), Some("q\""));
assert_eq!(n.children.len(), 2);
assert_eq!(n.child("C").unwrap().text.as_deref(), Some("t < u"));
}
#[test]
fn gates_values_by_version() {
let old = root_style("ParagraphStyle", 7);
let new = root_style("ParagraphStyle", 21);
assert_eq!(old.attr("KeepFirstLines"), Some("2"));
assert_eq!(old.attr("KeepAllLinesTogether"), Some("false"));
assert!(old.attr("EmitCss").is_none());
assert_eq!(new.attr("EmitCss"), Some("true"));
let props = new.child("Properties").unwrap();
assert!(props.child("RuleAboveType").is_some());
assert!(props.child("ParagraphBorderType").is_some());
let os = root_style("ObjectStyle", 21);
assert!(
os.child("TextFramePreference")
.unwrap()
.attr("TextColumnCount")
.is_some()
);
assert!(
os.child("TextFramePreference")
.unwrap()
.attr("AutoSizingType")
.is_some()
);
}
}
+183
View File
@@ -0,0 +1,183 @@
#!/usr/bin/env python3
"""Find the values that every corpus IDML has on its root styles.
Reads Resources/Styles.xml of every IDML in the corpus pairs (distinct
files only) and collects each attribute, Properties child and child
element attribute of the root styles. A value is kept when it is the same
in every IDML from some DOM version on and present in all of them; the
earliest such version is its minimum version. Attributes the converter
reads from the INDD (the TEXT_ATTRS table in src/idml/mod.rs) are left
out for the paragraph and character root styles.
Usage: python3 -I tools/root_values.py [--write]
Prints the evidence counts. With --write, regenerates
src/idml/root_values.xml. See docs/format/idml-values.md.
"""
import hashlib
import re
import sys
import xml.etree.ElementTree as ET
import zipfile
from collections import OrderedDict, defaultdict
from pathlib import Path
from xml.sax.saxutils import quoteattr
ROOT = Path(__file__).resolve().parent.parent
OUT = ROOT / "src" / "idml" / "root_values.xml"
TARGETS = OrderedDict([
("ParagraphStyle", "ParagraphStyle/$ID/[No paragraph style]"),
("CharacterStyle", "CharacterStyle/$ID/[No character style]"),
("ObjectStyle", "ObjectStyle/$ID/[None]"),
("CellStyle", "CellStyle/$ID/[None]"),
("TableStyle", "TableStyle/$ID/[No table style]"),
])
# Written by the converter itself.
SKIP = {"Self", "Name"}
HEADER = """<?xml version="1.0" encoding="UTF-8"?>
<!--
Values that every IDML exported by InDesign in the corpus has on the root
styles. The converter writes them on its root styles; values it reads from
the INDD take precedence. Values in a block with MinimumVersion are written
only for documents of that InDesign version or later. Generated by
tools/root_values.py; evidence in docs/format/idml-values.md.
-->
"""
def decoded_names():
src = (ROOT / "src" / "idml" / "mod.rs").read_text()
table = src[src.index("const TEXT_ATTRS"):src.index("/// An attribute written as a")]
return set(re.findall(r'0x[0-9A-F]+,\s*"(\w+)"', table))
def corpus_idmls():
rows = (ROOT / "corpus" / "inventory.tsv").read_text().splitlines()[1:]
seen, out, pairs = set(), [], 0
for row in rows:
path, _size, _valid, _order, _ver, _creator, dom = row.split("\t")
if not dom:
continue
pairs += 1
idml = (ROOT / "corpus" / path).with_suffix(".idml")
digest = hashlib.md5(idml.read_bytes()).hexdigest()
if digest not in seen:
seen.add(digest)
out.append((int(dom.split(".")[0]), idml))
return pairs, out
def leaves(el, path, out, order):
"""Flatten an element: (path, '@attr') and (path, 'P/tag') keys."""
for k, v in el.attrib.items():
out[(path, "@" + k)] = v
order.setdefault((path, "@" + k), len(order))
for c in el:
if c.tag == "Properties":
for p in c:
key = (path, "P/" + p.tag)
out[key] = re.sub(r">\s+<", "><", ET.tostring(p, encoding="unicode").strip())
order.setdefault(key, len(order))
else:
order.setdefault((path + (c.tag,), ""), len(order))
leaves(c, path + (c.tag,), out, order)
def analyse():
decoded = decoded_names()
pairs, idmls = corpus_idmls()
doms = [d for d, _ in idmls]
values = defaultdict(dict)
order = defaultdict(dict)
for i, (_dom, idml) in enumerate(idmls):
root = ET.fromstring(zipfile.ZipFile(idml).read("Resources/Styles.xml"))
for el in root.iter():
if el.tag in TARGETS and TARGETS[el.tag] == el.get("Self"):
out = {}
leaves(el, (), out, order[el.tag])
for k, v in out.items():
values[(el.tag, k)][i] = v
kept = defaultdict(list)
left_out = defaultdict(lambda: defaultdict(list))
for (tag, key), by_file in values.items():
path, name = key
if not path and name[1:] in SKIP:
continue
plain = name.split("/", 1)[1] if name.startswith("P/") else name[1:]
if not path and tag in ("ParagraphStyle", "CharacterStyle") and plain in decoded:
left_out[tag]["read from the INDD"].append(plain)
continue
since = None
for m in sorted({doms[i] for i in by_file}):
files = [i for i, d in enumerate(doms) if d >= m]
if all(i in by_file for i in files):
if len({by_file[i] for i in files}) == 1:
since = (m, len(files))
break
if since is None:
left_out[tag]["varies"].append("/".join(path + (plain,)))
continue
kept[tag].append((key, next(iter(by_file.values())), since))
return pairs, idmls, doms, kept, left_out, order
def element(tag, node, indent):
attrs = "".join(f" {k}={quoteattr(v)}" for k, v in node["attrs"])
if not node["props"] and not node["children"]:
return f"{indent}<{tag}{attrs} />\n"
s = f"{indent}<{tag}{attrs}>\n"
if node["props"]:
s += f"{indent}\t<Properties>\n"
s += "".join(f"{indent}\t\t{p}\n" for p in node["props"])
s += f"{indent}\t</Properties>\n"
for t, c in node["children"].items():
s += element(t, c, indent + "\t")
return s + f"{indent}</{tag}>\n"
def write(kept, order, first):
blocks = []
for since in sorted({s for items in kept.values() for _, _, (s, _) in items}):
gate = "" if since == first else f' MinimumVersion="{since}"'
s = f"\t<Values{gate}>\n"
for tag in TARGETS:
items = sorted((it for it in kept[tag] if it[2][0] == since),
key=lambda it: order[tag][it[0]])
if not items:
continue
tree = {"attrs": [], "props": [], "children": OrderedDict()}
for (path, name), v, _ in items:
node = tree
for p in path:
node = node["children"].setdefault(
p, {"attrs": [], "props": [], "children": OrderedDict()})
if name.startswith("@"):
node["attrs"].append((name[1:], v))
else:
node["props"].append(v)
s += element(tag, tree, "\t\t")
blocks.append(s + "\t</Values>\n")
OUT.write_text(HEADER + "<RootStyleValues>\n" + "".join(blocks) + "</RootStyleValues>\n")
def main():
pairs, idmls, doms, kept, left_out, order = analyse()
print(f"{pairs} pairs, {len(idmls)} distinct IDML files, DOM versions {sorted(set(doms))}")
for tag in TARGETS:
counts = defaultdict(int)
for _, _, since in kept[tag]:
counts[since] += 1
print(f"\n{tag}:")
for (since, n), k in sorted(counts.items()):
print(f" {k:4} values, the same in {n} of {n} IDML files with DOM >= {since}")
for why, names in left_out[tag].items():
print(f" {len(names):4} left out ({why}): {', '.join(sorted(names))}")
if "--write" in sys.argv:
write(kept, order, min(doms))
print(f"\nwrote {OUT.relative_to(ROOT)}")
if __name__ == "__main__":
main()