mirror of
https://github.com/storytold/indd-utils.git
synced 2026-11-20 01:00:46 +01:00
fix(compare): count no text positions outside character ranges
The walker of the previous commit counted every element outside a character range, such as StoryPreference, as a text position. A story whose reference has MetadataPacketPreference, which the converter does not write, then had all its range offsets shifted by one. Only Content and elements inside a character range count now. Trustworthy pairs at HEAD: reproduced 10,578,767 -> 10,579,111 (91.92 %), extra values 26,516 -> 26,179; reference values unchanged. The numbers in docs/measurement.md for the measurement change are re-measured with the corrected rule (10,505,425 reproduced, 26,297 extra values). No key reproduces fewer values.
This commit is contained in:
1 parent
64e5667dfd
commit
d4f3fd5a64
2 files changed
+17
-13
No files matched your search
+6
-4
@@ -140,8 +140,10 @@ IDML puts some elements around text ranges or around text inside a range:
|
||||
`compare.py` looks through them: every `ParagraphStyleRange` and
|
||||
`CharacterStyleRange` of the story counts, and the text inside a wrapper
|
||||
counts towards the offsets. Offsets count `Content` in UTF-16 code units
|
||||
and every other element (`Br`, `Table`, `Footnote`, an anchored frame) as
|
||||
one position; `Properties` and `XMLAttribute` count nothing. When a range
|
||||
and every other element inside a character range (`Br`, `Table`,
|
||||
`Footnote`, an anchored frame) as one position. `Properties`,
|
||||
`XMLAttribute` and elements outside character ranges (`StoryPreference`,
|
||||
`MetadataPacketPreference`, …) count nothing. When a range
|
||||
nested in a wrapper starts at the same offset as the range around it, the
|
||||
nested range is the one compared. The wrapper elements themselves have a
|
||||
`Self` and count as elements.
|
||||
@@ -151,8 +153,8 @@ Counting them changed the trustworthy totals as follows:
|
||||
| Ranges counted | Reference values | Reproduced | Coverage | Extra values |
|
||||
|---|---:|---:|---:|---:|
|
||||
| Only `ParagraphStyleRange` children of the story and their `CharacterStyleRange` children | 11,142,835 | 10,501,458 | 94.24 % | 30,237 |
|
||||
| Also inside `HyperlinkTextSource` and `CrossReferenceSource` | 11,158,348 | 10,505,081 | 94.15 % | 26,634 |
|
||||
| Also inside `XMLElement`, `Change` and `EndnoteRange` | 11,508,691 | 10,505,081 | 91.28 % | 26,634 |
|
||||
| Also inside `HyperlinkTextSource` and `CrossReferenceSource` | 11,158,348 | 10,505,425 | 94.15 % | 26,297 |
|
||||
| Also inside `XMLElement`, `Change` and `EndnoteRange` | 11,508,691 | 10,505,425 | 91.28 % | 26,297 |
|
||||
|
||||
Before, the skipped ranges were not counted, and every later range of the
|
||||
story started at a smaller offset in the reference than in the output: the
|
||||
|
||||
+11
-9
@@ -224,35 +224,37 @@ def text_ranges(story):
|
||||
"""Map text offset -> attributes of the paragraph and character range
|
||||
starting there, with keys prefixed PSR. and CSR. Ranges inside wrapper
|
||||
elements count, at whatever depth. Offsets count Content in UTF-16 code
|
||||
units and every other element (Br, Table, Footnote, an anchored frame) as
|
||||
one position."""
|
||||
units and every other element inside a character range (Br, Table,
|
||||
Footnote, an anchored frame) as one position; elements outside
|
||||
character ranges (StoryPreference, ...) are not text."""
|
||||
out = {}
|
||||
pos = 0
|
||||
|
||||
def walk(el, pa):
|
||||
"""Walk the children of `el` inside the paragraph attributes `pa`."""
|
||||
def walk(el, pa, in_csr):
|
||||
"""Walk the children of `el` inside the paragraph attributes `pa`;
|
||||
`in_csr` tells whether `el` is inside a character range."""
|
||||
nonlocal pos
|
||||
for ch in el:
|
||||
if ch.tag in NOT_TEXT:
|
||||
continue
|
||||
if ch.tag == "ParagraphStyleRange":
|
||||
walk(ch, {"PSR." + k: v for k, v in props(ch).items()})
|
||||
walk(ch, {"PSR." + k: v for k, v in props(ch).items()}, False)
|
||||
elif ch.tag == "CharacterStyleRange":
|
||||
start = pos
|
||||
walk(ch, pa)
|
||||
walk(ch, pa, True)
|
||||
if pos > start:
|
||||
ca = {"CSR." + k: v for k, v in props(ch).items()}
|
||||
# A range nested at the same offset (Change > range at
|
||||
# the start of this one) keeps the offset.
|
||||
out.setdefault(start, {**pa, **ca})
|
||||
elif ch.tag in WRAPPERS:
|
||||
walk(ch, pa)
|
||||
walk(ch, pa, in_csr)
|
||||
elif ch.tag == "Content":
|
||||
pos += len((ch.text or "").encode("utf-16-le")) // 2
|
||||
else:
|
||||
elif in_csr:
|
||||
pos += 1
|
||||
|
||||
walk(story, {})
|
||||
walk(story, {}, False)
|
||||
return out
|
||||
|
||||
|
||||
|
||||
Reference in new issue
Block a user