Files
pdfcraft/crates/redact/src/tests.rs
Michael Higgins 6da8fa27ed fix(redact): redact an XObject or font whose /Subtype is a reference (#6750)
## What this changes

Fixes #6750 (audit F-9).

Redaction kept an image under a mark when the image's `/Subtype` was an indirect reference (`/Subtype 8 0 R`, object 8 being `/Image`). `xobject()` in `crates/redact/src/interp.rs` matched on `s.dict.name(b"Subtype")`, and `Dict::name` does not follow references. The image fell through to `_ => None`, so the `Do`, the resource entry and the samples stayed in the saved file under the black box. The renderer still draws such an image.

- `xobject()` resolves `/Subtype` before matching, so an image or form named by reference is redacted like a direct one.
- `is_vertical()` does the same for a Type0 font, so vertical text in a font with a referenced `/Subtype` still makes Apply refuse rather than pass.
- The `_` arm now fails closed. An XObject with no `/Subtype`, or one the redactor doesn't read, is removed in Apply when its placement hits a mark. Verify counts it as residue there. Its placement is the unit square an image fills, or the `/BBox` under its `/Matrix` when it has one. Apply counts it in `forms_removed` and sets `removed_cut`, as it does for a form it cannot read. A placement outside every mark is left alone.

Related open PRs, checked for overlap: #5849 resolves `/Subtype` on annotations in `lib.rs`, and #5662 resolves an indirect `/Subtype /Form` in `prune.rs`. Neither touches `xobject()` or `is_vertical()`.

## How you verified it

- [x] `cargo fmt --check` (clean)
- [ ] clippy with `-D warnings` on `pdfcraft-redact`: **not run locally**
- [ ] tests pass on `pdfcraft-redact`: **not run locally with the fix**, see below

**What ran.** With the three new or extended tests in place and `interp.rs` reverted to `main`, I ran `cargo test -p pdfcraft-redact --lib -- an_image_whose_subtype an_xobject_of_unknown vertical_writing_text`. All three failed on the expected assertions:
- `an_image_whose_subtype_is_a_reference_is_redacted`: the direct `/Subtype /Image` control passed, then the `8 0 R` case failed with `images_removed: 0`.
- `an_xobject_of_unknown_subtype_under_a_mark_goes`: the verifier counted 0, expected 1.
- `vertical_writing_text_under_a_mark_fails_closed`: the referenced `/Subtype /Type0` case did not refuse.

**What did not run.** Local builds were stopped for this change before the run with the fix completed. I have **not** seen these tests pass with the fix, nor run the rest of the crate's tests or clippy. I'm relying on CI for those. This PR is a draft until CI passes.

## Never-crash

- [x] No new `unwrap`/`expect`/`panic!`/`unreachable!`/`todo!` outside tests
- [x] No indexing with input-derived positions (the `/BBox` is read with a slice pattern)
- [x] No `unsafe`

## Parity and reach

No parity change: this hardens the existing Apply Redactions behaviour.

## Anything a reviewer should look at first

The fail-closed `_` arm. It removes an unclassifiable XObject only where a placement hits a mark, never elsewhere. Verify counts it as residue, so the post-apply check would refuse if Apply ever left one.
2026-11-18 16:35:49 +07:00

7364 lines
405 KiB
Rust
Raw Permalink Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
use std::sync::Arc;
use pdfcraft_annot::{Meta, NewAnnotation, Shape, Style, add_annotation, rect_quad};
use pdfcraft_cos::{Document, SaveOptions, write_full, write_incremental};
use pdfcraft_testkit::{Pdf, stream_obj as stream};
use super::*;
fn pdf(objs: Vec<Vec<u8>>) -> Document {
Document::open(Arc::new(Pdf::new().objs(objs).build())).unwrap()
}
/// A font where every glyph is 500 units wide (so a 10 pt glyph advances 5 pt).
const FONT: &str = "<< /Type /Font /Subtype /TrueType /BaseFont /Arial /FirstChar 32 /LastChar 126 /Widths 95 0 R /FontDescriptor << /Ascent 800 /Descent -200 >> >>";
fn widths() -> Vec<u8> {
format!("[{}]", vec!["500"; 95].join(" ")).into_bytes()
}
/// One 300×300 page with `content`, font /F1, and extra resources/objects.
fn one_page(content: &[u8], extra_res: &str, extra: Vec<Vec<u8>>) -> Document {
let mut objs: Vec<Vec<u8>> = vec![
b"<< /Type /Catalog /Pages 2 0 R >>".to_vec(),
b"<< /Type /Pages /Kids [3 0 R] /Count 1 >>".to_vec(),
format!("<< /Type /Page /Parent 2 0 R /MediaBox [0 0 300 300] /Contents 4 0 R /Resources << /Font << /F1 5 0 R >> {extra_res} >> >>")
.into_bytes(),
stream("", content),
FONT.replace("95 0 R", "6 0 R").into_bytes(),
widths(),
];
objs.extend(extra);
pdf(objs)
}
fn mark(doc: &mut Document, page: usize, rects: &[[f64; 4]], overlay: &str) {
let shape = Shape::Redact { quads: rects.iter().map(|r| rect_quad(*r)).collect(), overlay: overlay.into(), look: Default::default() };
let style = Style::default_for(&shape);
add_annotation(doc, &NewAnnotation { page, shape, style, contents: String::new(), author: "Tester".into() }, &Meta::default()).unwrap();
}
/// The decoded content streams of a page, joined.
fn content(doc: &Document, page: usize) -> String {
let p = &pdfcraft_model::pages(doc)[page];
let (_, data) = page_streams(doc, &p.dict, page).unwrap();
data.iter().map(|d| String::from_utf8_lossy(d).into_owned()).collect::<Vec<_>>().join("\n")
}
/// Glyphs (and inline images) of page `page` under `rects` (the verifier's count).
fn under(doc: &mut Document, page: usize, rects: &[[f64; 4]]) -> usize {
let p = pdfcraft_model::pages(doc).swap_remove(page);
let mut budget = pdfcraft_cos::DecodeBudget::page();
let (_, data) = page_streams_within(doc, &p.dict, page, &mut budget).unwrap();
let res = p.dict.get(b"Resources").and_then(|r| doc.resolve(r).as_dict().cloned()).unwrap_or_default();
let mut rep = Report::default();
let mut images = pdfcraft_cos::ImageBudget::with_work(0);
let mut scope = Scope::new(rects, Mode::Verify, &mut rep, budget, &mut images);
process(doc, &mut scope, &data, &res, pdfcraft_content::Matrix::IDENTITY).residue
}
fn reopen(doc: &Document) -> Document {
Document::open(Arc::new(write_incremental(doc, &SaveOptions::default()).unwrap())).unwrap()
}
/// A word drawn as outlines names itself in /ActualText. A mark over part of it removes the
/// outlines (a shading painting them is clipped instead), and the name, which would still read the
/// whole word, goes with them. A word the mark misses keeps its own.
#[test]
fn redacted_paint_loses_the_actual_text_that_names_it() {
let mut doc = one_page(
b"/Span <</ActualText (SECRET)>> BDC 100 100 m 160 100 l 160 120 l 100 120 l h f EMC /Span <</ActualText (KEPT)>> BDC 100 200 60 20 re f EMC",
"",
vec![],
);
mark(&mut doc, 0, &[[90.0, 95.0, 130.0, 125.0]], "");
let r = apply(&mut doc, None).unwrap();
assert_eq!((r.paths_clipped, r.paths_removed), (0, 1));
let c = content(&doc, 0);
assert!(!c.contains("SECRET") && c.contains("KEPT"), "{c}");
let shading =
"<< /ShadingType 2 /ColorSpace /DeviceGray /Coords [0 0 300 0] /Function << /FunctionType 2 /Domain [0 1] /C0 [0] /C1 [1] /N 1 >> >>";
let mut doc = one_page(b"/Span <</ActualText (SHADED)>> BDC /Sh0 sh EMC", "/Shading << /Sh0 7 0 R >>", vec![shading.as_bytes().to_vec()]);
mark(&mut doc, 0, &[[90.0, 95.0, 130.0, 125.0]], "");
apply(&mut doc, None).unwrap();
let c = content(&doc, 0);
// Drawn after a clip that cuts the region out.
assert!(!c.contains("SHADED") && c.find("W*").zip(c.find(" sh")).is_some_and(|(w, s)| w < s), "{c}");
// A path that clips as it paints.
let mut doc = one_page(b"q /Span <</ActualText (CLIPS)>> BDC 100 100 m 160 100 l 160 120 l 100 120 l h W f EMC Q", "", vec![]);
mark(&mut doc, 0, &[[90.0, 95.0, 130.0, 125.0]], "");
assert_eq!(apply(&mut doc, None).unwrap().paths_removed, 1);
let c = content(&doc, 0);
assert!(!c.contains("CLIPS"), "{c}");
}
/// A CMap with a codespace range the font parser can't keep (five bytes) makes the page's text a
/// guess: apply refuses while a mark is pending.
#[test]
fn a_cmap_with_a_dropped_codespace_range_fails_closed() {
let program = "/CIDInit /ProcSet findresource begin 12 dict begin begincmap 2 begincodespacerange <0000> <FFFF> <0000000000> <FFFFFFFFFF> endcodespacerange 1 begincidrange <0000> <FFFF> 0 endcidrange endcmap CMapName currentdict /CMap defineresource pop end end";
let mut doc = pdf(vec![
b"<< /Type /Catalog /Pages 2 0 R >>".to_vec(),
b"<< /Type /Pages /Kids [3 0 R] /Count 1 >>".to_vec(),
b"<< /Type /Page /Parent 2 0 R /MediaBox [0 0 300 300] /Contents 4 0 R /Resources << /Font << /F2 5 0 R >> >> >>".to_vec(),
stream("", b"BT /F2 10 Tf 100 50 Td <0001000200030004> Tj ET"),
b"<< /Type /Font /Subtype /Type0 /BaseFont /X /Encoding 6 0 R /DescendantFonts [<< /Type /Font /Subtype /CIDFontType2 /BaseFont /X /DW 1000 >>] >>".to_vec(),
stream("/Type /CMap /CMapName /Custom", program.as_bytes()),
]);
mark(&mut doc, 0, &[[95.0, 40.0, 140.0, 70.0]], "");
assert_eq!(apply(&mut doc, None), Err(RedactError::DroppedCodespace(1)));
}
/// Nested names around redacted paint all go, the innermost too, whichever scope was cut first.
#[test]
fn redacted_paint_loses_every_name_around_it() {
let mut doc = one_page(
b"/Span <</ActualText (OUTER)>> BDC 100 100 m 160 100 l 160 120 l 100 120 l h f /Span <</ActualText (INNER)>> BDC 100 130 m 160 130 l 160 150 l 100 150 l h f EMC EMC",
"",
vec![],
);
mark(&mut doc, 0, &[[90.0, 95.0, 130.0, 155.0]], "");
assert_eq!(apply(&mut doc, None).unwrap().paths_removed, 2);
let c = content(&doc, 0);
assert!(!c.contains("OUTER") && !c.contains("INNER"), "{c}");
}
/// Removing hidden text clips nothing, so a shading on the same page keeps the words that name it.
#[test]
fn removing_hidden_text_keeps_the_alt_text_of_a_shading() {
let shading =
"<< /ShadingType 2 /ColorSpace /DeviceGray /Coords [0 0 300 0] /Function << /FunctionType 2 /Domain [0 1] /C0 [0] /C1 [1] /N 1 >> >>";
let mut doc = one_page(
b"BT /F1 10 Tf 3 Tr 10 200 Td (SECRET) Tj ET /Figure <</Alt (chart)>> BDC /Sh0 sh EMC",
"/Shading << /Sh0 7 0 R >>",
vec![shading.as_bytes().to_vec()],
);
sanitize::remove_hidden(&mut doc, &[sanitize::Hidden::HiddenText]).unwrap();
let c = content(&doc, 0);
assert!(!c.contains("SECRET") && c.contains("(chart)"), "{c}");
}
#[test]
fn glyphs_under_a_mark_go_and_the_rest_stays_put() {
// "AB1234CD" from x = 10 at 10 pt: each glyph 5 pt wide; 1234 spans x 20–40.
let mut doc = one_page(b"BT /F1 10 Tf 10 100 Td (AB1234CD) Tj ET", "", vec![]);
assert_eq!(under(&mut doc, 0, &[[40.0, 95.0, 50.0, 110.0]]), 2, "C and D before");
mark(&mut doc, 0, &[[20.0, 95.0, 40.0, 110.0]], "");
let r = apply(&mut doc, None).unwrap();
assert_eq!((r.marks, r.glyphs), (1, 4));
let c = content(&doc, 0);
assert!(c.contains("[(AB) -2000 (CD)] TJ"), "{c}");
assert!(!c.contains("1234"));
assert_eq!(under(&mut doc, 0, &[[40.0, 95.0, 50.0, 110.0]]), 2, "C and D are where they were");
assert_eq!(under(&mut doc, 0, &[[20.0, 95.0, 40.0, 110.0]]), 0);
// The mark became a black box drawn into the page; the annotation is gone.
assert!(marks(&doc).is_empty());
assert!(c.contains("0 0 0 rg") && c.contains("20 95 20 15 re f"), "{c}");
let doc = reopen(&doc);
assert!(!content(&doc, 0).contains("1234"));
}
/// `content` behind a PNG Up predictor whose `/DecodeParms` is an indirect object added to `doc`,
/// in a stream with `dict`'s entries too.
fn predicted(doc: &mut Document, dict: pdfcraft_cos::Dict, content: &[u8]) -> pdfcraft_cos::Stream {
use pdfcraft_cos::{Dict, Object, Stream};
// One column: each byte a row of its own, tagged 2.
let mut rows = Vec::new();
let mut above = 0u8;
for &b in content {
rows.extend([2, b.wrapping_sub(above)]);
above = b;
}
let parms = doc.add(Object::Dict([(b"Predictor".to_vec(), Object::Int(12))].into_iter().collect::<Dict>()));
let mut s = Stream::flate(Dict::new(), &rows);
for (k, v) in dict.iter() {
s.dict.set(k.to_vec(), v.clone());
}
s.dict.set(b"DecodeParms".to_vec(), Object::Ref(parms));
s
}
/// Content whose `/DecodeParms` is given by reference is read through the reference, on the page
/// and in a form: the text under a mark goes. Read with the predictor's tag bytes left in, no glyph
/// was found under the mark, and the redaction left the text in the file and said it was done.
#[test]
fn content_with_parameters_given_by_reference_is_redacted() {
use pdfcraft_cos::{Dict, ObjRef, Object};
let secret = b"BT /F1 10 Tf 10 100 Td (SECRET) Tj ET";
let marked = [[5.0, 90.0, 120.0, 115.0]];
let mut page = one_page(b"", "", vec![]);
let s = predicted(&mut page, Dict::new(), secret);
page.set(ObjRef::new(4, 0), s);
// The parameters take object 7 and the form 8.
let mut form = one_page(b"/Fm Do", "/XObject << /Fm 8 0 R >>", vec![]);
let mut dict = Dict::new();
dict.set(b"Subtype".to_vec(), Object::name("Form"));
dict.set(b"BBox".to_vec(), Object::Array([0, 0, 300, 300].into_iter().map(Object::Int).collect()));
dict.set(
b"Resources".to_vec(),
Object::Dict(
[(b"Font".to_vec(), Object::Dict([(b"F1".to_vec(), Object::Ref(ObjRef::new(5, 0)))].into_iter().collect()))].into_iter().collect(),
),
);
let s = predicted(&mut form, dict, secret);
form.set(ObjRef::new(8, 0), s);
for (mut doc, what) in [(page, "page"), (form, "form")] {
assert_eq!(under(&mut doc, 0, &marked), 6, "{what}");
mark(&mut doc, 0, &marked, "");
let r = apply(&mut doc, None).unwrap();
assert_eq!((r.marks, r.glyphs), (1, 6), "{what}");
let mut saved = reopen(&doc);
assert_eq!(under(&mut saved, 0, &marked), 0, "{what}");
// Nor is the text left in a stream the page no longer draws, read with its parameters.
for (r, o) in saved.scan_objects() {
let Object::Stream(s) = &*o else { continue };
let data = pdfcraft_cos::DecodeBudget::new(usize::MAX, 0).decode_with_damage(&saved, s, usize::MAX).map(|(d, _)| d).unwrap_or_default();
assert!(!data.windows(6).any(|w| w == b"SECRET"), "{what}: {r:?}");
}
}
}
/// An image removed under a mark stays in the page's resources while another of its content
/// streams, with `/DecodeParms` given by reference, still draws it. Read without the parameters,
/// that stream seemed not to draw it, and the page lost the image there.
#[test]
fn an_image_drawn_again_in_content_with_parameters_by_reference_stays() {
use pdfcraft_cos::{Dict, ObjRef, Object};
let img = stream("/Type /XObject /Subtype /Image /Width 4 /Height 1 /ColorSpace /DeviceGray /BitsPerComponent 8", [255, 255, 255, 255]);
let mut doc = one_page(b"q 10 0 0 10 10 10 cm /Im1 Do Q", "/XObject << /Im1 7 0 R >>", vec![img]);
let again = predicted(&mut doc, Dict::new(), b"q 10 0 0 10 200 200 cm /Im1 Do Q");
let again = doc.add(Object::Stream(again));
doc.update_dict(ObjRef::new(3, 0), |d| d.set(b"Contents".to_vec(), Object::Array(vec![Object::Ref(ObjRef::new(4, 0)), Object::Ref(again)])))
.unwrap();
mark(&mut doc, 0, &[[5.0, 5.0, 25.0, 25.0]], "");
let r = apply(&mut doc, None).unwrap();
assert_eq!(r.images_removed, 1, "{r:?}");
let doc = reopen(&doc);
assert!(content(&doc, 0).contains("200 200 cm /Im1 Do"), "{}", content(&doc, 0));
let p = pdfcraft_model::pages(&doc).swap_remove(0);
let res = doc.resolve(p.dict.get(b"Resources").unwrap()).as_dict().cloned().unwrap();
let xo = doc.resolve(res.get(b"XObject").unwrap()).as_dict().cloned().unwrap();
// The save numbers objects afresh: the image is told by what it is.
let Some(Object::Stream(image)) = xo.reference(b"Im1").map(|r| (*doc.get(r)).clone()) else { panic!("no /Im1: {xo:?}") };
assert_eq!((image.dict.name(b"Subtype"), image.decoded().ok()), (Some(&b"Image"[..]), Some(vec![255; 4])));
}
/// #2130: a font with no /Widths is drawn with its embedded program's advances. Measured with a
/// guess from its name instead (Courier's 600 units for "Mono"), the end of the line seemed to
/// lie past a mark that covers all of it on screen, and was kept.
#[test]
fn a_whole_line_under_a_mark_goes_even_without_widths() {
let text = b"illicit little lilies";
let font = "<< /Type /Font /Subtype /TrueType /BaseFont /HouseMono /Encoding /WinAnsiEncoding /FontDescriptor 6 0 R >>";
let mut doc = pdf(vec![
b"<< /Type /Catalog /Pages 2 0 R >>".to_vec(),
b"<< /Type /Pages /Kids [3 0 R] /Count 1 >>".to_vec(),
b"<< /Type /Page /Parent 2 0 R /MediaBox [0 0 300 300] /Contents 4 0 R /Resources << /Font << /F1 5 0 R >> >> >>".to_vec(),
stream("", [b"BT /F1 10 Tf 10 100 Td (".as_slice(), text, b") Tj ET"].concat()),
font.as_bytes().to_vec(),
b"<< /Type /FontDescriptor /FontName /HouseMono /Flags 32 /Ascent 900 /Descent -200 /FontFile2 7 0 R >>".to_vec(),
stream(&format!("/Length1 {}", pdfcraft_fonts::INTER_REGULAR.len()), pdfcraft_fonts::INTER_REGULAR),
]);
// Inter draws these 21 narrow glyphs in about 60 pt; Courier's guess would need 126.
let line = [5.0, 95.0, 90.0, 112.0];
mark(&mut doc, 0, &[line], "");
let r = apply(&mut doc, None).unwrap();
assert_eq!(r.glyphs, text.len(), "every glyph of the line goes");
assert_eq!(under(&mut doc, 0, &[[0.0, 0.0, 300.0, 300.0]]), 0, "{}", content(&doc, 0));
}
/// Issue #2448: the redacted words are often in the document information dictionary and the XMP
/// metadata too. Applying removes the entries and streams that repeat them, and keeps the rest.
#[test]
fn redacted_text_does_not_survive_in_info_or_xmp() {
let xmp = |body: &str| {
stream(
"/Type /Metadata /Subtype /XML",
format!(
r#"<x:xmpmeta xmlns:x="adobe:ns:meta/"><rdf:RDF xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#">{body}</rdf:RDF></x:xmpmeta>"#
)
.as_bytes(),
)
};
let objs: Vec<Vec<u8>> = vec![
b"<< /Type /Catalog /Pages 2 0 R /Metadata 7 0 R >>".to_vec(),
b"<< /Type /Pages /Kids [3 0 R] /Count 1 >>".to_vec(),
b"<< /Type /Page /Parent 2 0 R /MediaBox [0 0 300 300] /Contents 4 0 R /Resources << /Font << /F1 5 0 R >> >> /Metadata 9 0 R >>".to_vec(),
// "BLUEFIN" spans x 55-90.
stream("", b"BT /F1 10 Tf 10 100 Td (Codename BLUEFIN report) Tj ET"),
FONT.replace("95 0 R", "6 0 R").into_bytes(),
widths(),
// 7: the document's XMP names it in an attribute, through a character reference.
xmp(r#"<rdf:Description rdf:about="" xmlns:pdf="http://ns.adobe.com/pdf/1.3/" pdf:Keywords="tuna, Blue&#x46;in"/>"#),
// 8: the information dictionary.
b"<< /Title (Codename BLUEFIN report) /Keywords (tuna, bluefin) /Author (Ada Lovelace) /Producer (Test) >>".to_vec(),
// 9: the page's XMP doesn't mention it.
xmp(r#"<rdf:Description rdf:about="" xmlns:dc="http://purl.org/dc/elements/1.1/"><dc:title>Annual figures</dc:title></rdf:Description>"#),
];
let mut doc = Document::open(Arc::new(Pdf::new().objs(objs).trailer_entry("/Info 8 0 R").build())).unwrap();
mark(&mut doc, 0, &[[55.0, 95.0, 90.0, 110.0]], "");
let r = apply(&mut doc, None).unwrap();
assert_eq!((r.glyphs, r.metadata), (7, 3), "{r:?}");
// Without object streams, so the byte search below can see the `/Info` dictionary.
let bytes = write_full(&doc, &SaveOptions { object_streams: false, ..SaveOptions::default() }).unwrap();
let has = |needle: &[u8]| bytes.windows(needle.len()).any(|w| w.eq_ignore_ascii_case(needle));
assert!(has(b"Ada Lovelace"), "the kept /Info entry is visible to the search");
assert!(!has(b"bluefin") && !has(b"Blue&#x46;in"), "the redacted word survived in the metadata");
let saved = Document::open(Arc::new(bytes)).unwrap();
let info = saved.trailer().get(b"Info").and_then(|i| saved.resolve(i).as_dict().cloned()).unwrap();
let text = |k: &[u8]| info.get(k).and_then(|v| saved.resolve(v).as_string().map(|s| s.to_text()));
assert_eq!((text(b"Title"), text(b"Keywords")), (None, None));
assert_eq!((text(b"Author").as_deref(), text(b"Producer").as_deref()), (Some("Ada Lovelace"), Some("Test")));
let catalog = saved.get(saved.root().unwrap()).as_dict().cloned().unwrap();
assert!(!catalog.contains(b"Metadata"), "the document XMP named the redacted word");
assert!(pdfcraft_model::pages(&saved)[0].dict.contains(b"Metadata"), "the page XMP didn't, and stays");
}
/// An `/Info` value that is itself part of the removed text goes, though its words are too short
/// to be terms ("J. Li"); a value with fewer than three letters or digits stays.
#[test]
fn an_info_value_inside_the_redacted_text_goes() {
let objs: Vec<Vec<u8>> = vec![
b"<< /Type /Catalog /Pages 2 0 R >>".to_vec(),
b"<< /Type /Pages /Kids [3 0 R] /Count 1 >>".to_vec(),
b"<< /Type /Page /Parent 2 0 R /MediaBox [0 0 300 300] /Contents 4 0 R /Resources << /Font << /F1 5 0 R >> >> >>".to_vec(),
stream("", b"BT /F1 10 Tf 10 100 Td (Prepared by J. Li) Tj ET"),
FONT.replace("95 0 R", "6 0 R").into_bytes(),
widths(),
b"<< /Author (J. li) /Subject (by) /Producer (Test) >>".to_vec(),
];
let mut doc = Document::open(Arc::new(Pdf::new().objs(objs).trailer_entry("/Info 7 0 R").build())).unwrap();
mark(&mut doc, 0, &[[5.0, 95.0, 200.0, 110.0]], "");
let r = apply(&mut doc, None).unwrap();
assert_eq!(r.metadata, 1, "{r:?}");
let info = doc.trailer().get(b"Info").and_then(|i| doc.resolve(i).as_dict().cloned()).unwrap();
assert!(!info.contains(b"Author"), "the author's name was redacted from the page");
assert!(info.contains(b"Subject") && info.contains(b"Producer"));
}
/// Removed glyphs whose text isn't known (no `/ToUnicode`, glyph names outside the glyph list)
/// leave nothing to search the metadata for, so every `/Info` string goes, as when the collected
/// text was cut short (review on #2539). Glyphs whose text is known keep the targeted check.
#[test]
fn removed_glyphs_with_no_known_text_clear_the_metadata() {
let page = |codes: &[u8], font: &str| -> Document {
let content = [&b"BT /F1 10 Tf 10 100 Td ("[..], codes, b") Tj ET"].concat();
let objs: Vec<Vec<u8>> = vec![
b"<< /Type /Catalog /Pages 2 0 R >>".to_vec(),
b"<< /Type /Pages /Kids [3 0 R] /Count 1 >>".to_vec(),
b"<< /Type /Page /Parent 2 0 R /MediaBox [0 0 300 300] /Contents 4 0 R /Resources << /Font << /F1 5 0 R >> >> >>".to_vec(),
stream("", &content),
font.replace("95 0 R", "6 0 R").into_bytes(),
widths(),
b"<< /Title (Project Nightjar) /Producer (Test) >>".to_vec(),
];
Document::open(Arc::new(Pdf::new().objs(objs).trailer_entry("/Info 7 0 R").build())).unwrap()
};
let unnamed = FONT.replace(" /Widths", " /Encoding << /Differences [65 /g17 /g18 /g19] >> /Widths");
let mut doc = page(b"ABC", &unnamed);
mark(&mut doc, 0, &[[5.0, 95.0, 200.0, 110.0]], "");
let r = apply(&mut doc, None).unwrap();
let info = doc.trailer().get(b"Info").and_then(|i| doc.resolve(i).as_dict().cloned()).unwrap();
assert!(!info.contains(b"Title") && !info.contains(b"Producer"), "fail closed: {r:?}");
// Control: the same glyphs with known text don't touch unrelated metadata.
let mut doc = page(b"ABC", FONT);
mark(&mut doc, 0, &[[5.0, 95.0, 200.0, 110.0]], "");
apply(&mut doc, None).unwrap();
let info = doc.trailer().get(b"Info").and_then(|i| doc.resolve(i).as_dict().cloned()).unwrap();
assert!(info.contains(b"Title") && info.contains(b"Producer"));
}
/// One page showing `content`, with `/Info` `info` and catalog XMP whose `rdf:RDF` holds
/// `xmp_body`.
fn with_metadata(content: &[u8], info: &str, xmp_body: &str) -> Document {
let xmp = format!(
r#"<?xpacket begin="" id="W5M0MpCehiHzreSzNTczkc9d"?><x:xmpmeta xmlns:x="adobe:ns:meta/"><rdf:RDF xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#">{xmp_body}</rdf:RDF></x:xmpmeta><?xpacket end="w"?>"#
);
let objs: Vec<Vec<u8>> = vec![
b"<< /Type /Catalog /Pages 2 0 R /Metadata 7 0 R >>".to_vec(),
b"<< /Type /Pages /Kids [3 0 R] /Count 1 >>".to_vec(),
b"<< /Type /Page /Parent 2 0 R /MediaBox [0 0 300 300] /Contents 4 0 R /Resources << /Font << /F1 5 0 R >> >> >>".to_vec(),
stream("", content),
FONT.replace("95 0 R", "6 0 R").into_bytes(),
widths(),
stream("/Type /Metadata /Subtype /XML", xmp.as_bytes()),
info.as_bytes().to_vec(),
];
Document::open(Arc::new(Pdf::new().objs(objs).trailer_entry("/Info 8 0 R").build())).unwrap()
}
fn info_keys(doc: &Document) -> Vec<String> {
let info = doc.trailer().get(b"Info").and_then(|i| doc.resolve(i).as_dict().cloned()).unwrap();
info.iter().map(|(k, _)| String::from_utf8_lossy(k).into_owned()).collect()
}
fn has_xmp(doc: &Document) -> bool {
doc.get(doc.root().unwrap()).as_dict().is_some_and(|c| c.contains(b"Metadata"))
}
/// Letter-spaced text (character spacing wider than a word gap between plain glyphs) is still
/// one word, so `/Title (Bluefin)` goes when "B L U E F I N" is redacted (#2539 review).
#[test]
fn letter_spaced_redacted_text_still_matches_metadata() {
let body = r#"<rdf:Description rdf:about="" xmlns:dc="http://purl.org/dc/elements/1.1/"><dc:title>Bluefin</dc:title></rdf:Description>"#;
let mut doc = with_metadata(b"BT /F1 10 Tf 3 Tc 10 100 Td (BLUEFIN) Tj ET", "<< /Title (Bluefin) /Producer (Test) >>", body);
mark(&mut doc, 0, &[[5.0, 95.0, 200.0, 110.0]], "");
let r = apply(&mut doc, None).unwrap();
assert_eq!(r.metadata, 2, "{r:?}");
assert_eq!(info_keys(&doc), ["Producer"]);
assert!(!has_xmp(&doc));
}
/// An XMP value that is part of the removed text goes, as an `/Info` value does, though its
/// words are too short to be terms ("J. Li"). Values are compared as whole words, so "True"
/// doesn't go with "untrue", and namespace URIs and the `xpacket` wrapper aren't searched, so
/// redacting "Adobe … end" keeps a packet declared under `adobe:ns:meta/` and closed by
/// `<?xpacket end="w"?>`.
#[test]
fn xmp_values_inside_the_redacted_text_go_as_whole_words() {
let creator = r#"<rdf:Description rdf:about="" xmlns:dc="http://purl.org/dc/elements/1.1/"><dc:creator>J. Li</dc:creator></rdf:Description>"#;
let mut doc = with_metadata(b"BT /F1 10 Tf 10 100 Td (Prepared by J. Li) Tj ET", "<< /Producer (Test) >>", creator);
mark(&mut doc, 0, &[[5.0, 95.0, 200.0, 110.0]], "");
let r = apply(&mut doc, None).unwrap();
assert_eq!(r.metadata, 1, "{r:?}");
assert!(!has_xmp(&doc), "the packet named J. Li");
let trapped = r#"<rdf:Description rdf:about="" xmlns:pdf="http://ns.adobe.com/pdf/1.3/" pdf:Trapped="True"/>"#;
let mut doc = with_metadata(b"BT /F1 10 Tf 10 100 Td (Adobe said untrue at the end) Tj ET", "<< /Subject (True) /Producer (Test) >>", trapped);
mark(&mut doc, 0, &[[5.0, 95.0, 200.0, 110.0]], "");
let r = apply(&mut doc, None).unwrap();
assert_eq!(r.metadata, 0, "{r:?}");
assert!(has_xmp(&doc));
assert_eq!(info_keys(&doc), ["Subject", "Producer"]);
}
/// When the removed text was too much to collect, what wasn't collected can't be searched for:
/// every `/Info` string and the XMP go (fail-closed), even though the runs that were kept match
/// nothing.
#[test]
fn metadata_is_cleared_when_the_removed_text_was_cut_short() {
// More separate runs of one letter than are kept: each `A` is too far from the last to join.
let mut content = b"BT /F1 10 Tf 10 100 Td".to_vec();
for _ in 0..50_001 {
content.extend_from_slice(b" (A) Tj 30 0 Td (A) Tj -30 0 Td");
}
content.extend_from_slice(b" ET");
let body = r#"<rdf:Description rdf:about="" xmlns:dc="http://purl.org/dc/elements/1.1/"><dc:title>Annual figures</dc:title></rdf:Description>"#;
let mut doc = with_metadata(&content, "<< /Title (Annual figures) /Producer (Test) /Trapped /False >>", body);
mark(&mut doc, 0, &[[5.0, 95.0, 200.0, 110.0]], "");
let r = apply(&mut doc, None).unwrap();
assert_eq!(r.metadata, 3, "{r:?}");
assert_eq!(info_keys(&doc), ["Trapped"], "strings go, the name stays");
assert!(!has_xmp(&doc));
}
/// #2539 review: the removed text kept for the metadata check is bounded for the whole document,
/// not per page. One glyph code can stand for 256 UTF-16 units of `/ToUnicode` text (768 bytes of
/// UTF-8), so here each page makes 150 runs of 80 such glyphs (9.2 MB, under the page's run and
/// byte limits); two pages pass `MAX_REMOVED_BYTES`, so the rest isn't kept and the check fails
/// closed.
#[test]
fn removed_text_kept_for_the_metadata_check_is_bounded_per_document() {
let long = "4E00".repeat(256);
let to_unicode = format!("/CIDInit /ProcSet findresource begin 1 beginbfchar <41> <{long}> endbfchar end");
let run = "A".repeat(80);
let mut content = b"BT /F1 1 Tf 10 100 Td".to_vec();
for _ in 0..75 {
content.extend_from_slice(format!(" ({run}) Tj 30 0 Td ({run}) Tj -30 0 Td").as_bytes());
}
content.extend_from_slice(b" ET");
let page = "/Type /Page /Parent 2 0 R /MediaBox [0 0 300 300] /Contents 4 0 R /Resources << /Font << /F1 5 0 R >> >>";
let objs: Vec<Vec<u8>> = vec![
b"<< /Type /Catalog /Pages 2 0 R >>".to_vec(),
b"<< /Type /Pages /Kids [3 0 R 8 0 R] /Count 2 >>".to_vec(),
format!("<< {page} >>").into_bytes(),
stream("", &content),
FONT.replace("95 0 R", "6 0 R").replace(" /Widths", " /ToUnicode 7 0 R /Widths").into_bytes(),
widths(),
stream("", to_unicode.as_bytes()),
format!("<< {page} >>").into_bytes(),
];
let mut doc = pdf(objs);
let mut applying = Applying {
report: Report::default(),
doomed_fields: Vec::new(),
doomed_xobjects: Vec::new(),
retired_properties: Vec::new(),
widget_owner: Vec::new(),
removed_text: Vec::new(),
removed_cut: false,
removed_room: interp::MAX_REMOVED_BYTES,
removed_everywhere: HashSet::new(),
images: pdfcraft_cos::ImageBudget::new(&doc),
removed_appearances: HashSet::new(),
appearance_scan_limit: false,
pruned_patterns: Vec::new(),
};
let rects = [[5.0, 99.0, 200.0, 102.0]];
let kept = |a: &Applying| a.removed_text.iter().map(String::len).sum::<usize>();
let first = pdfcraft_model::pages(&doc).swap_remove(0);
redact_page_content(&mut doc, &mut applying, 0, &first, &rects).unwrap();
assert_eq!((kept(&applying), applying.removed_cut), (9_216_000, false), "one page fits");
let second = pdfcraft_model::pages(&doc).swap_remove(1);
redact_page_content(&mut doc, &mut applying, 1, &second, &rects).unwrap();
assert!(applying.removed_cut, "the second page passes the document's limit");
assert!(kept(&applying) <= interp::MAX_REMOVED_BYTES, "kept {} bytes", kept(&applying));
}
/// #2539 review: the searching a scrub may do is one budget for all of it, not a check per
/// value: very many small `/Info` values each fit, but together they don't, and past the budget
/// the rest counts as matching (fail-closed). With room to search, nothing that doesn't mention
/// the removed text goes.
#[test]
fn metadata_search_has_one_budget_for_every_value() {
let entries: String = (0..50).map(|i| format!("/K{i} (an unrelated value number {i:02} in the list) ")).collect();
let info = format!("<< {entries}/Producer (Test) >>");
let fresh = || with_metadata(b"BT /F1 10 Tf 10 100 Td (Bluefin) Tj ET", &info, "");
let mut doc = fresh();
assert_eq!(metadata::scrub_within(&mut doc, &["Bluefin".into()], true, 1 << 28).unwrap().removed, 0, "room to search: nothing goes");
let mut doc = fresh();
// Each value costs about 40; fifty of them don't fit in 1000.
let n = metadata::scrub_within(&mut doc, &["Bluefin".into()], true, 1000).unwrap().removed;
assert!(n > 20, "past the budget the rest goes ({n} went)");
}
/// #2539 review: an entity is looked for only in the 12 bytes after its `&`, so a packet of bare
/// `&` characters is read in linear time, and decoding is unchanged.
#[test]
fn xml_entities_are_found_near_their_ampersand_only() {
assert_eq!(metadata::xml_values("<a>R&#233;sum&eacute; &amp; &#x41;&#65; &#1234567890123;</a>"), ["Résum&eacute; & AA &#1234567890123;"]);
// #2539 review: a character reference with many leading zeros still decodes, so it can't
// hide a redacted word from the search; a reference that never ends stays as written.
assert_eq!(metadata::xml_values("<a>&#00000000000066;LUEFIN &#x000000000000000042;LUEFIN &#0000000000;</a>"), ["BLUEFIN BLUEFIN &#0000000000;"]);
let bare = "&".repeat(200_000);
let started = std::time::Instant::now();
let values = metadata::xml_values(&format!("<a>{bare}</a>"));
assert_eq!(values.concat().len(), bare.len());
let zeros = format!("&#{}", "0".repeat(10)).repeat(20_000);
assert_eq!(metadata::xml_values(&format!("<a>{zeros}</a>")).concat().len(), zeros.len());
eprintln!("200000 bare ampersands, 20000 unended references: {:?}", started.elapsed());
}
/// #2539 review: Apply removes a form XObject whole when it is nested past the depth limit (or
/// its stream can't be decoded), without reading its text, so what it held is unknown: the
/// metadata check fails closed.
#[test]
fn a_form_removed_unread_clears_the_metadata() {
// Forms 5, 6, ... each draw the next, one more than the depth limit deep, and the last one
// paints. The page draws form 5.
let last = 5 + crate::interp::MAX_DEPTH;
let mut objs: Vec<Vec<u8>> = vec![
b"<< /Type /Catalog /Pages 2 0 R >>".to_vec(),
b"<< /Type /Pages /Kids [3 0 R] /Count 1 >>".to_vec(),
b"<< /Type /Page /Parent 2 0 R /MediaBox [0 0 300 300] /Contents 4 0 R /Resources << /XObject << /Fm 5 0 R >> >> >>".to_vec(),
stream("", b"q /Fm Do Q"),
];
for n in 5..=last {
objs.push(stream(
&format!("/Type /XObject /Subtype /Form /BBox [0 0 300 300] /Resources << /XObject << /Fm {} 0 R >> >>", n + 1),
b"q /Fm Do Q",
));
}
objs.push(stream("/Type /XObject /Subtype /Form /BBox [0 0 300 300]", b"0 g 10 10 50 50 re f"));
objs.push(b"<< /Title (Annual figures) /Producer (Test) >>".to_vec());
let info = objs.len();
let mut doc = Document::open(Arc::new(Pdf::new().objs(objs).trailer_entry(&format!("/Info {info} 0 R")).build())).unwrap();
mark(&mut doc, 0, &[[0.0, 0.0, 300.0, 300.0]], "");
let r = apply(&mut doc, None).unwrap();
assert!(r.forms_removed >= 1, "{r:?}");
assert!(!info_keys(&doc).contains(&"Title".to_string()), "the deep form's text was never read: {:?}", info_keys(&doc));
}
/// A value shorter than the redacted sentence goes from XMP too, in a custom schema's element or
/// attribute, as from `/Info` (the regression #2598 suggested keeping): the packets holding it are
/// removed, one that doesn't stays.
#[test]
fn a_shorter_value_in_custom_xmp_goes() {
let xmp = |body: &str| {
stream(
"/Type /Metadata /Subtype /XML",
format!(
r#"<x:xmpmeta xmlns:x="adobe:ns:meta/"><rdf:RDF xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#">{body}</rdf:RDF></x:xmpmeta>"#
)
.as_bytes(),
)
};
let objs: Vec<Vec<u8>> = vec![
b"<< /Type /Catalog /Pages 2 0 R /Metadata 7 0 R >>".to_vec(),
b"<< /Type /Pages /Kids [3 0 R] /Count 1 >>".to_vec(),
b"<< /Type /Page /Parent 2 0 R /MediaBox [0 0 300 300] /Contents 4 0 R /Resources << /Font << /F1 5 0 R >> >> /Metadata 8 0 R >>".to_vec(),
stream("", b"BT /F1 10 Tf 10 100 Td (JAMES BOND IS A SPY) Tj ET"),
FONT.replace("95 0 R", "6 0 R").into_bytes(),
widths(),
// 7: an element of a custom schema.
xmp(r#"<rdf:Description rdf:about="" xmlns:acme="https://example.org/acme/"><acme:Agent>James Bond</acme:Agent></rdf:Description>"#),
// 8: an attribute of a custom schema.
xmp(r#"<rdf:Description rdf:about="" xmlns:acme="https://example.org/acme/" acme:code="James Bond"/>"#),
// 9: a third packet, on no holder yet; swapped onto the page below to show a clean one stays.
xmp(r#"<rdf:Description rdf:about="" xmlns:dc="http://purl.org/dc/elements/1.1/"><dc:title>Annual figures</dc:title></rdf:Description>"#),
b"<< /Author (James Bond) /Subject (Annual figures) >>".to_vec(),
];
for (holder_of_8, expected) in [("8 0 R", 3), ("9 0 R", 2)] {
let objs: Vec<Vec<u8>> =
objs.iter().map(|o| String::from_utf8_lossy(o).replace("/Metadata 8 0 R", &format!("/Metadata {holder_of_8}")).into_bytes()).collect();
let mut doc = Document::open(Arc::new(Pdf::new().objs(objs).trailer_entry("/Info 10 0 R").build())).unwrap();
mark(&mut doc, 0, &[[5.0, 95.0, 200.0, 110.0]], "");
let r = apply(&mut doc, None).unwrap();
assert_eq!(r.metadata, expected, "{holder_of_8}: {r:?}");
let catalog = doc.get(doc.root().unwrap()).as_dict().cloned().unwrap();
assert!(!catalog.contains(b"Metadata"), "the custom element named him");
let page_keeps = pdfcraft_model::pages(&doc)[0].dict.contains(b"Metadata");
assert_eq!(page_keeps, holder_of_8 == "9 0 R", "the attribute packet goes, a clean one stays");
let info = doc.trailer().get(b"Info").and_then(|i| doc.resolve(i).as_dict().cloned()).unwrap();
assert!(!info.contains(b"Author") && info.contains(b"Subject"), "the author goes, the unrelated subject stays");
}
}
/// The removed text is collected in runs by position, not by operator: glyphs shown one operator
/// at a time still make one word, a gap makes a space, and separate marks make separate runs.
#[test]
fn removed_text_is_collected_in_runs() {
let mut doc = one_page(
b"BT /F1 10 Tf 10 100 Td (BLU) Tj 15 0 Td (E) Tj 5 0 Td [(F) -300 (IN)] TJ ( 7) Tj ET BT /F1 10 Tf 10 200 Td (Gary) Tj ET",
"",
vec![],
);
let page = pdfcraft_model::pages(&doc).swap_remove(0);
let (_, data) = page_streams(&doc, &page.dict, 0).unwrap();
let res = page.dict.get(b"Resources").and_then(|r| doc.resolve(r).as_dict().cloned()).unwrap_or_default();
let mut rep = Report::default();
let rects = [[10.0, 95.0, 300.0, 110.0], [10.0, 195.0, 30.0, 210.0]];
let mut images = pdfcraft_cos::ImageBudget::new(&doc);
let mut scope = Scope::new(&rects, Mode::Apply, &mut rep, pdfcraft_cos::DecodeBudget::page(), &mut images);
process(&mut doc, &mut scope, &data, &res, pdfcraft_content::Matrix::IDENTITY);
scope.flush();
assert_eq!(scope.removed_text, ["BLUEF IN 7", "Gary"]);
}
#[test]
fn metadata_terms_and_xmp_text() {
// Runs of two letters or digits or more are matched whole, and by their words of three or
// more, ignoring case and spacing.
let t = metadata::terms(&["John Smith".into(), "7".into(), "of".into(), "A-12".into()]);
assert_eq!(t.as_deref(), Some(&["john smith", "john", "smith", "of", "a-12"].map(String::from)[..]));
// A run too long to search for whole can't be matched: the caller clears everything.
assert_eq!(metadata::terms(&["x".repeat(5000)]), None);
// XMP values: element content, attribute values, CDATA, comments and processing instructions,
// entities decoded; not names, and not namespace declarations.
let x = metadata::xml_values(
r#"<?xpacket begin="" id="W5M0MpCehiHzreSzNTczkc9d"?><?audit James Bond?><?xpacketAudit SECRET?><x:xmpmeta xmlns:x="adobe:ns:meta/" xmlns="http://example.org/" xmlns:y = "adobe:ns:y/" b = "spaced"><dc:title a="x &amp; y"><!-- hidden --><rdf:li>R&#233;sum&eacute; <![CDATA[<raw>]]></rdf:li></dc:title></x:xmpmeta>"#,
);
// Only the `xpacket` wrapper itself is skipped, and a declaration is one with spaces around
// `=` too.
assert_eq!(x, ["audit James Bond", "xpacketAudit SECRET", "spaced", "x & y", " hidden ", "Résum&eacute; ", "<raw>"]);
}
/// XMP is decoded from UTF-8 or UTF-16, the latter also without a byte-order mark at the start
/// (a packet's mark sits inside `<?xpacket begin="…"`); anything that isn't valid is not read.
#[test]
fn xmp_packets_decode_or_fail_closed() {
let text = "<?xpacket begin=\"\u{feff}\"?><a>Bond</a>";
let utf16 = |be: bool| text.encode_utf16().flat_map(|u| if be { u.to_be_bytes() } else { u.to_le_bytes() }).collect::<Vec<u8>>();
assert_eq!(metadata::decode_xmp(text.as_bytes()).as_deref(), Some(text));
assert_eq!(metadata::decode_xmp(&utf16(true)).as_deref(), Some(text));
assert_eq!(metadata::decode_xmp(&utf16(false)).as_deref(), Some(text));
assert_eq!(metadata::decode_xmp(&[[0xFF, 0xFE].as_slice(), &utf16(false)].concat()).as_deref(), Some(text));
assert_eq!(metadata::decode_xmp(b"<a>Bond \xE9</a>"), None, "Latin-1 isn't searched lossily");
assert_eq!(metadata::decode_xmp(&utf16(false)[1..]), None, "odd length");
}
#[test]
fn added_text_parameters_do_not_keep_redacted_text() {
// An Edit ▸ Add content item: its stream dictionary keeps the source text under /PCAdded.
let mut doc = pdf(vec![
b"<< /Type /Catalog /Pages 2 0 R >>".to_vec(),
b"<< /Type /Pages /Kids [3 0 R] /Count 1 >>".to_vec(),
b"<< /Type /Page /Parent 2 0 R /MediaBox [0 0 300 300] /Contents [4 0 R] /Resources << /Font << /F1 5 0 R >> >> >>".to_vec(),
stream("/PCMark /Added /PCAdded << /Kind /Text /Text (AB1234CD) >>", b"BT /F1 10 Tf 10 100 Td (AB1234CD) Tj ET"),
FONT.replace("95 0 R", "6 0 R").into_bytes(),
widths(),
]);
mark(&mut doc, 0, &[[20.0, 95.0, 40.0, 110.0]], "");
apply(&mut doc, None).unwrap();
assert!(!content(&doc, 0).contains("1234"));
let bytes = write_full(&doc, &SaveOptions::default()).unwrap();
assert!(!bytes.windows(4).any(|w| w == b"1234"), "redacted text left in the saved file");
}
#[test]
fn operators_split_across_content_streams_are_redacted_whole() {
// One content stream in three pieces, split between tokens: a marked-content dictionary
// ends in the second piece, and the TJ array that ends it shows its glyphs with the
// operator in the third.
let mut doc = pdf(vec![
b"<< /Type /Catalog /Pages 2 0 R >>".to_vec(),
b"<< /Type /Pages /Kids [3 0 R] /Count 1 >>".to_vec(),
b"<< /Type /Page /Parent 2 0 R /MediaBox [0 0 300 300] /Contents [4 0 R 7 0 R 8 0 R] /Resources << /Font << /F1 5 0 R >> >> >>".to_vec(),
stream("", b"/P << /MCID 0"),
FONT.replace("95 0 R", "6 0 R").into_bytes(),
widths(),
stream("", b">> BDC BT /F1 10 Tf 10 100 Td [(AB1234CD)]"),
stream("", b"TJ ET EMC BT /F1 10 Tf 10 200 Td (KEEP) Tj ET"),
]);
assert_eq!(under(&mut doc, 0, &[[20.0, 95.0, 40.0, 110.0]]), 4, "the verifier sees the split TJ");
mark(&mut doc, 0, &[[20.0, 95.0, 40.0, 110.0]], "");
let r = apply(&mut doc, None).unwrap();
assert_eq!(r.glyphs, 4, "{r:?}");
let c = content(&doc, 0);
assert!(!c.contains("1234"), "redacted glyphs left in the page: {c}");
assert!(c.contains("KEEP"), "{c}");
assert_eq!(under(&mut doc, 0, &[[40.0, 95.0, 50.0, 110.0]]), 2, "C and D are where they were");
// Every operator still has its operands: none was cut off from them.
for op in pdfcraft_content::parse(c.as_bytes()).ops {
let want = match op.op.as_slice() {
b"TJ" | b"Tj" => 1,
b"BDC" | b"Tf" | b"Td" => 2,
_ => continue,
};
assert_eq!(op.operands.len(), want, "{} lost its operands in {c}", String::from_utf8_lossy(&op.op));
}
let doc = reopen(&doc);
assert!(!content(&doc, 0).contains("1234"));
}
#[test]
fn kerning_spacing_scaling_and_line_operators_are_honoured() {
// TJ kerning, character and word spacing, 50% horizontal scaling, ' and ".
let src = b"BT /F1 10 Tf 2 Tc 4 Tw 50 Tz 12 TL 0 200 Td [(AB) -1000 (C D)] TJ (EF) ' 1 0 (GH) \" ET";
let mut doc = one_page(src, "", vec![]);
// Line 1 (y 200): A at 0, B at 3.5 (advance (5+2)×0.5), kern +5 → C at 12, space at 15.5,
// D at 21 (space advance (5+2+4)×0.5 = 5.5). Remove C only.
mark(&mut doc, 0, &[[12.2, 195.0, 14.8, 210.0]], "");
// Line 2 (y 188) "EF" and line 3 (y 176) "GH" (Tw 1, Tc 0): remove F and G.
mark(&mut doc, 0, &[[3.8, 183.0, 6.0, 198.0 - 10.0], [0.2, 171.0, 2.3, 180.0]], "");
let r = apply(&mut doc, None).unwrap();
assert_eq!(r.glyphs, 3, "{r:?}");
let c = content(&doc, 0);
assert!(c.contains("[(AB) -1700 ( D)] TJ"), "{c}");
assert!(c.contains("T*\n[(E) -700] TJ"), "{c}");
assert!(c.contains("1 Tw\n0 Tc\nT*\n[-500 (H)] TJ"), "{c}");
// Everything else stays readable where it was.
assert_eq!(under(&mut doc, 0, &[[20.0, 195.0, 30.0, 210.0]]), 1, "D");
assert_eq!(under(&mut doc, 0, &[[2.6, 171.0, 6.0, 180.0]]), 1, "H");
}
#[test]
fn rotated_text_and_composite_fonts() {
// A Type0 Identity-H font (two-byte codes) with /W widths, and a 90° text matrix.
let t0 = b"<< /Type /Font /Subtype /Type0 /BaseFont /X /Encoding /Identity-H /DescendantFonts [<< /Type /Font /Subtype /CIDFontType2 /BaseFont /X /DW 1000 /W [1 [500 500] 3 4 250] >>] >>".to_vec();
let src = b"BT /F2 10 Tf 0 1 -1 0 100 50 Tm <0001000200030004> Tj ET";
let mut doc = one_page(src, "", vec![]);
let font_ref = doc.add(Object::Dict(Dict::new()));
let parsed = {
let mut lx = pdfcraft_cos::Lexer::new(&t0, 0);
lx.object().unwrap()
};
doc.set(font_ref, parsed);
let page = pdfcraft_model::pages(&doc)[0].obj;
doc.update_dict(page, |d| {
let mut res = d.get(b"Resources").and_then(Object::as_dict).cloned().unwrap();
let mut fonts = res.get(b"Font").and_then(Object::as_dict).cloned().unwrap();
fonts.set(b"F2".to_vec(), Object::Ref(font_ref));
res.set(b"Font".to_vec(), Object::Dict(fonts));
d.set(b"Resources".to_vec(), Object::Dict(res));
})
.unwrap();
// Glyphs run upwards from y 50: CID1 50–55, CID2 55–60, CID3 60–62.5, CID4 62.5–65.
mark(&mut doc, 0, &[[85.0, 55.5, 105.0, 61.0]], "");
let r = apply(&mut doc, None).unwrap();
assert_eq!(r.glyphs, 2, "CIDs 2 and 3");
let c = content(&doc, 0);
assert!(c.contains("[<0001> -750 <0004>] TJ") || c.contains("[(\\000\\001) -750 (\\000\\004)] TJ"), "{c}");
}
#[test]
fn vertical_writing_text_under_a_mark_fails_closed() {
// A Type0 font laid out vertically (Identity-V / CMap WMode 1) cannot have its glyphs placed
// under a mark by the horizontal interpreter, so Apply Redactions must fail rather than ship
// the text under the box. The horizontal control (Identity-H) is redacted normally.
let make = |encoding: &str| -> Document {
let t0 = format!(
"<< /Type /Font /Subtype /Type0 /BaseFont /X /Encoding /{encoding} /DescendantFonts [<< /Type /Font /Subtype /CIDFontType2 /BaseFont /X /DW 1000 /W [1 [500 500] 3 4 250] >>] >>"
)
.into_bytes();
let src = b"BT /F2 10 Tf 0 1 -1 0 100 50 Tm <0001000200030004> Tj ET";
let mut doc = one_page(src, "", vec![]);
let font_ref = doc.add(Object::Dict(Dict::new()));
let parsed = {
let mut lx = pdfcraft_cos::Lexer::new(&t0, 0);
lx.object().unwrap()
};
doc.set(font_ref, parsed);
let page = pdfcraft_model::pages(&doc)[0].obj;
doc.update_dict(page, |d| {
let mut res = d.get(b"Resources").and_then(Object::as_dict).cloned().unwrap();
let mut fonts = res.get(b"Font").and_then(Object::as_dict).cloned().unwrap();
fonts.set(b"F2".to_vec(), Object::Ref(font_ref));
res.set(b"Font".to_vec(), Object::Dict(fonts));
d.set(b"Resources".to_vec(), Object::Dict(res));
})
.unwrap();
doc
};
// Identity-V: apply must refuse rather than report success with the text still in the file.
let mut vdoc = make("Identity-V");
mark(&mut vdoc, 0, &[[85.0, 55.5, 105.0, 61.0]], "");
let err = apply(&mut vdoc, None).unwrap_err();
assert_eq!(err, RedactError::VerticalText(1), "vertical text with a pending mark must not silently pass");
let msg = err.to_string();
assert!(msg.contains("cannot tell whether"), "{msg}");
assert!(!msg.contains("text under a redaction mark"), "{msg}");
// Identity-H control: the same glyphs and mark are redacted normally.
let mut hdoc = make("Identity-H");
mark(&mut hdoc, 0, &[[85.0, 55.5, 105.0, 61.0]], "");
let r = apply(&mut hdoc, None).unwrap();
assert_eq!(r.glyphs, 2, "the horizontal control is redacted: {r:?}");
// A Type0 font whose /Subtype is a reference still writes vertically (#6750).
let mut rdoc = make("Identity-V");
let page = pdfcraft_model::pages(&rdoc)[0].obj;
let res = rdoc.get(page).as_dict().and_then(|d| d.get(b"Resources")).and_then(Object::as_dict).cloned().unwrap();
let font_ref = res.get(b"Font").and_then(Object::as_dict).and_then(|f| f.get(b"F2")).and_then(Object::as_ref).unwrap();
let name_ref = rdoc.add(Object::Name(b"Type0".to_vec()));
rdoc.update_dict(font_ref, |d| d.set(b"Subtype".to_vec(), Object::Ref(name_ref))).unwrap();
mark(&mut rdoc, 0, &[[85.0, 55.5, 105.0, 61.0]], "");
assert_eq!(apply(&mut rdoc, None).unwrap_err(), RedactError::VerticalText(1), "a referenced /Subtype /Type0 is still vertical");
}
#[test]
fn images_are_removed_or_have_their_pixels_cleared() {
// Image 1 is fully covered; image 2 (4×1 gray, all white) is half covered.
let img = stream("/Type /XObject /Subtype /Image /Width 4 /Height 1 /ColorSpace /DeviceGray /BitsPerComponent 8", [255, 255, 255, 255]);
let src = b"q 10 0 0 10 10 10 cm /Im1 Do Q q 40 0 0 10 100 100 cm /Im2 Do Q";
let mut doc = one_page(src, "/XObject << /Im1 7 0 R /Im2 7 0 R >>", vec![img]);
mark(&mut doc, 0, &[[5.0, 5.0, 25.0, 25.0], [95.0, 95.0, 120.0, 115.0]], "");
let r = apply(&mut doc, None).unwrap();
assert_eq!((r.images_removed, r.images_cleared), (1, 1), "{r:?}");
let c = content(&doc, 0);
assert!(!c.contains("/Im1 Do") && !c.contains("/Im2 Do"), "{c}");
let p = pdfcraft_model::pages(&doc).swap_remove(0);
let res = doc.resolve(p.dict.get(b"Resources").unwrap()).as_dict().cloned().unwrap();
let xo = doc.resolve(res.get(b"XObject").unwrap()).as_dict().cloned().unwrap();
let new = xo.iter().find(|(k, _)| k.starts_with(b"PCRedacted")).map(|(_, v)| v.clone()).expect("a cleared copy");
let Object::Stream(s) = &*doc.resolve(&new) else { panic!() };
assert_eq!(s.decoded().unwrap(), [0, 0, 255, 255], "pixels 1–2 (x 100–120) cleared");
let Object::Stream(orig) = &*doc.get(ObjRef::new(7, 0)) else { panic!() };
assert_eq!(orig.decoded().unwrap(), [255; 4], "the shared original is untouched");
}
/// `/Subtype 8 0 R` naming `/Image` is still an image, and the renderer draws it. Redaction read
/// only a direct name, so it kept the `Do` and the samples under the box (#6750). The direct
/// `/Subtype /Image` is the control.
#[test]
fn an_image_whose_subtype_is_a_reference_is_redacted() {
const MARKER: &[u8; 16] = b"F9-SECRET-PIXELS";
for subtype in ["/Image", "8 0 R"] {
let img = stream(&format!("/Type /XObject /Subtype {subtype} /Width 16 /Height 1 /ColorSpace /DeviceGray /BitsPerComponent 8"), MARKER);
let mut doc = one_page(b"q 10 0 0 10 10 10 cm /Im1 Do Q", "/XObject << /Im1 7 0 R >>", vec![img, b"/Image".to_vec()]);
mark(&mut doc, 0, &[[5.0, 5.0, 25.0, 25.0]], "");
let r = apply(&mut doc, None).unwrap();
assert_eq!(r.images_removed, 1, "{subtype}: {r:?}");
let c = content(&doc, 0);
assert!(!c.contains("/Im1 Do"), "{subtype}: {c}");
let bytes = write_full(&doc, &SaveOptions { object_streams: false, ..SaveOptions::default() }).unwrap();
assert!(!bytes.windows(MARKER.len()).any(|w| w == MARKER), "{subtype}: the image samples stay in the saved file");
}
}
/// An XObject the redactor cannot classify (no `/Subtype`, or one it does not know) may still draw
/// something under a mark. Apply removes it there and verification counts it; one elsewhere stays.
#[test]
fn an_xobject_of_unknown_subtype_under_a_mark_goes() {
for subtype in ["", "/Subtype /PS", "/Subtype 42"] {
let x = stream(&format!("/Type /XObject {subtype} /Width 1 /Height 1 /ColorSpace /DeviceGray /BitsPerComponent 8"), [0]);
let src = b"q 10 0 0 10 10 10 cm /X1 Do Q q 10 0 0 10 200 200 cm /X1 Do Q";
let mut doc = one_page(src, "/XObject << /X1 7 0 R >>", vec![x]);
let rects = [[5.0, 5.0, 25.0, 25.0]];
assert_eq!(under(&mut doc, 0, &rects), 1, "{subtype:?}: verification counts it");
mark(&mut doc, 0, &rects, "");
let r = apply(&mut doc, None).unwrap();
assert_eq!(r.forms_removed, 1, "{subtype:?}: {r:?}");
let c = content(&doc, 0);
assert_eq!(c.matches("/X1 Do").count(), 1, "{subtype:?}: only the placement outside the mark stays: {c}");
assert!(c.contains("200 200 cm"), "{subtype:?}: {c}");
}
}
/// An unclassifiable XObject's `/Matrix` and `/BBox` are read with their entries by reference, as
/// the proof reads a form's. Read only directly, a box moved onto the mark was taken for the
/// identity or for no box, and the placement was kept and not counted. Present but malformed, its
/// place can't be told, so it counts. The direct entries are the control.
#[test]
fn an_xobject_of_unknown_subtype_is_placed_by_referenced_geometry() {
for geometry in [
"/Matrix [1 0 0 1 100 100] /BBox [0 0 10 10]",
"/Matrix [1 0 0 1 8 0 R 9 0 R] /BBox [0 0 10 10]",
"/BBox [100 100 10 0 R 10 0 R]",
"/Matrix [1 0 0 1 100] /BBox [0 0 10 10]",
"/BBox [0 0 10]",
] {
let x = stream(&format!("/Type /XObject /Subtype /PS {geometry}"), [0]);
let mut doc = one_page(b"/X1 Do", "/XObject << /X1 7 0 R >>", vec![x, b"100".to_vec(), b"100".to_vec(), b"110".to_vec()]);
let rects = [[105.0, 105.0, 110.0, 110.0]];
assert_eq!(under(&mut doc, 0, &rects), 1, "{geometry}: verification counts it");
mark(&mut doc, 0, &rects, "");
let r = apply(&mut doc, None).unwrap();
assert_eq!(r.forms_removed, 1, "{geometry}: {r:?}");
let c = content(&doc, 0);
assert!(!c.contains("/X1 Do"), "{geometry}: {c}");
}
}
/// A `/BBox` of null is no entry (ISO 32000-2 §7.3.9), directly or by reference. It was read as a
/// malformed box, so an unclassifiable XObject placed outside every mark was removed and counted.
/// No `/BBox` at all is the control.
#[test]
fn an_xobject_of_unknown_subtype_with_a_null_bbox_is_placed_as_without_one() {
for bbox in ["", "/BBox null", "/BBox 8 0 R"] {
let x = stream(&format!("/Type /XObject /Subtype /PS {bbox}"), [0]);
let src = b"q 10 0 0 10 200 200 cm /X1 Do Q";
let mut doc = one_page(src, "/XObject << /X1 7 0 R >>", vec![x, b"null".to_vec()]);
let rects = [[5.0, 5.0, 25.0, 25.0]];
assert_eq!(under(&mut doc, 0, &rects), 0, "{bbox:?}: verification leaves it out");
mark(&mut doc, 0, &rects, "");
let r = apply(&mut doc, None).unwrap();
assert_eq!(r.forms_removed, 0, "{bbox:?}: {r:?}");
let c = content(&doc, 0);
assert!(c.contains("/X1 Do"), "{bbox:?}: the placement outside the mark stays: {c}");
}
}
/// A `/BBox` is read only when it has four entries; any other length is malformed whatever its
/// entries, so the placement's place can't be told and it counts as under a mark.
#[test]
fn an_xobject_of_unknown_subtype_with_a_bbox_of_the_wrong_length_counts() {
for bbox in ["/BBox []", "/BBox [0 0 10 10 10]", "/BBox [0 0 10 10 8 0 R]"] {
let x = stream(&format!("/Type /XObject /Subtype /PS {bbox}"), [0]);
let mut doc = one_page(b"q 10 0 0 10 200 200 cm /X1 Do Q", "/XObject << /X1 7 0 R >>", vec![x, b"10".to_vec()]);
assert_eq!(under(&mut doc, 0, &[[5.0, 5.0, 25.0, 25.0]]), 1, "{bbox:?}: verification counts it");
}
}
#[test]
fn vectors_are_removed_or_clipped_and_inline_images_go() {
let src = b"0 g 10 10 20 20 re f 0 0 300 300 re f q 10 0 0 10 50 50 cm BI /W 1 /H 1 /CS /G /BPC 8 ID \x80 EI Q 1 0 0 RG 150 150 m 160 160 l S";
let mut doc = one_page(src, "", vec![]);
mark(&mut doc, 0, &[[5.0, 5.0, 35.0, 35.0], [45.0, 45.0, 65.0, 65.0]], "");
let r = apply(&mut doc, None).unwrap();
assert_eq!((r.paths_removed, r.paths_clipped, r.images_removed), (2, 0, 1), "{r:?}");
let c = content(&doc, 0);
assert!(!c.contains("10 10 20 20 re"), "{c}");
assert!(!c.contains("300 300 re"), "the intersecting rect is removed: {c}");
assert!(!c.contains("BI"), "{c}");
assert!(c.contains("150 150 m"), "the line elsewhere stays");
}
/// A horizontal stroke's construction box has no area, so it overlapped nothing and stayed under
/// the mark. The painted stroke is as wide as the line: a 12 pt line centred 5 pt above the mark
/// reaches into it and goes; a 1 pt line there does not and stays.
#[test]
fn a_stroke_is_removed_by_its_painted_width() {
let src = b"12 w 10 60 m 100 60 l S 1 w 10 60 m 100 60 l S 4 w 10 50 m 100 50 l S";
let mut doc = one_page(src, "", vec![]);
mark(&mut doc, 0, &[[40.0, 45.0, 60.0, 55.0]], "");
let r = apply(&mut doc, None).unwrap();
assert_eq!(r.paths_removed, 2, "{r:?}");
let c = content(&doc, 0);
assert!(!c.contains("12 w\n10 60 m"), "the wide line reaches the mark: {c}");
assert!(c.contains("1 w\n10 60 m\n100 60 l\nS"), "the thin line beside it stays: {c}");
assert!(!c.contains("10 50 m"), "the line through the mark goes: {c}");
}
/// A tiling pattern's cell is a content stream of its own and can draw text. Redacting a shape
/// filled with such a pattern removed the fill from the page but left the pattern in the page's
/// resources, so its cell -- with the text that was under the mark -- stayed in the saved file
/// (#3647). A pattern nothing paints with any more now leaves the resources; one some surviving
/// paint still uses stays.
#[test]
fn a_pattern_whose_only_painting_was_redacted_leaves_the_page() {
fn page_with(fills: &str) -> Document {
let cell = stream(
"/Type /Pattern /PatternType 1 /PaintType 1 /TilingType 1 /BBox [0 0 300 300] /XStep 300 /YStep 300 /Resources << /Font << /F1 5 0 R >> >>",
b"BT /F1 10 Tf 10 150 Td (SECRET) Tj ET",
);
let content = format!("BT /F1 10 Tf 10 280 Td (PUBLIC) Tj ET {fills}");
one_page(content.as_bytes(), "/Pattern << /P0 7 0 R >>", vec![cell])
}
let has_pattern = |doc: &Document| {
pdfcraft_model::pages(doc)[0]
.dict
.get(b"Resources")
.and_then(|r| doc.resolve(r).as_dict().cloned())
.is_some_and(|res| res.get(b"Pattern").is_some())
};
// One box, marked: the pattern is no longer painted, so it goes with the fill.
let mut doc = page_with("/Pattern cs /P0 scn 20 140 100 30 re f");
mark(&mut doc, 0, &[[15.0, 135.0, 130.0, 175.0]], "");
let report = apply(&mut doc, None).unwrap();
assert_eq!(report.patterns_removed, 1, "the pattern left the page");
assert!(!has_pattern(&doc), "the page has no /Pattern left");
let saved = write_incremental(&doc, &SaveOptions::default()).unwrap();
assert!(!saved.windows(8).any(|w| w == b"(SECRET)"), "the cell's text is gone from the saved file");
// Two boxes share a text-bearing cell. The second still needs its pattern, but it is not
// safe to infer that it visibly displays every byte of the cell.
let mut doc = page_with("/Pattern cs /P0 scn 20 140 100 30 re f /Pattern cs /P0 scn 20 40 100 30 re f");
mark(&mut doc, 0, &[[15.0, 135.0, 130.0, 175.0]], "");
let before = write_full(&doc, &SaveOptions::default()).unwrap();
assert!(matches!(apply(&mut doc, None), Err(RedactError::Pattern { reason: PatternProblem::Content, .. })));
assert_eq!(write_full(&doc, &SaveOptions::default()).unwrap(), before, "refusal keeps the pending mark and original bytes");
// Paint is not only path painting: text shows with the fill pattern too, and the text here
// survives the mark. Dropping the pattern would take the fill away from text still on the
// page, so a pattern is kept for every operator that can paint with it.
let mut doc = page_with("/Pattern cs /P0 scn 20 140 100 30 re f BT /F1 10 Tf 10 240 Td (KEEP) Tj ET");
mark(&mut doc, 0, &[[15.0, 135.0, 130.0, 175.0]], "");
assert!(matches!(apply(&mut doc, None), Err(RedactError::Pattern { reason: PatternProblem::Content, .. })));
assert!(has_pattern(&doc), "the refused operation keeps the original pattern");
assert!(content(&doc, 0).contains("(KEEP)"), "the text is still there: {}", content(&doc, 0));
// Text in render mode 3 paints nothing, so it keeps no pattern: the unpainted pattern goes.
let mut doc = page_with("/Pattern cs /P0 scn 20 140 100 30 re f BT 3 Tr /F1 10 Tf 10 240 Td (KEEP) Tj ET");
mark(&mut doc, 0, &[[15.0, 135.0, 130.0, 175.0]], "");
assert_eq!(apply(&mut doc, None).unwrap().patterns_removed, 1, "invisible text paints no pattern");
// `/Pattern` may be an indirect object, like any value. Reading the reference itself as a
// dictionary found none, so nothing was pruned and the cell stayed in the saved file.
let cell = stream(
"/Type /Pattern /PatternType 1 /PaintType 1 /TilingType 1 /BBox [0 0 300 300] /XStep 300 /YStep 300 /Resources << /Font << /F1 5 0 R >> >>",
b"BT /F1 10 Tf 10 150 Td (SECRET) Tj ET",
);
let mut doc = one_page(b"/Pattern cs /P0 scn 20 140 100 30 re f", "/Pattern 8 0 R", vec![cell, b"<< /P0 7 0 R >>".to_vec()]);
mark(&mut doc, 0, &[[15.0, 135.0, 130.0, 175.0]], "");
assert_eq!(apply(&mut doc, None).unwrap().patterns_removed, 1, "an indirect /Pattern is resolved before it is pruned");
assert!(!has_pattern(&doc), "the page has no /Pattern left");
let saved = write_incremental(&doc, &SaveOptions::default()).unwrap();
assert!(!saved.windows(8).any(|w| w == b"(SECRET)"), "the cell's text is gone from the saved file");
}
/// Pruning takes the pattern out of `/Resources` but leaves the `/P0 scn` that selected it, so
/// the saved page names a pattern that is no longer there. ISO 32000-1 (8.6.6.2) says that
/// operand is a key of the `/Pattern` subdictionary, so this is what a reader is handed: the
/// question is whether it stops reading the stream there. Asked in the review of #3647.
#[test]
fn a_page_whose_pruned_pattern_is_still_selected_renders() {
let cell = stream(
"/Type /Pattern /PatternType 1 /PaintType 1 /TilingType 1 /BBox [0 0 300 300] /XStep 300 /YStep 300 /Resources << /Font << /F1 5 0 R >> >>",
b"BT /F1 10 Tf 10 150 Td (SECRET) Tj ET",
);
// `0 g` after the fill puts the text back on a plain colour: with the pattern still selected
// the text would paint with it, and a pattern something still paints with is never pruned.
// The text is *after* the selector on purpose -- that is what a reader would lose if it gave
// up on the stale `/P0`.
let content_stream = b"/Pattern cs /P0 scn 20 140 100 30 re f 0 g BT /F1 24 Tf 10 40 Td (PUBLIC) Tj ET";
let mut doc = one_page(content_stream, "/Pattern << /P0 7 0 R >>", vec![cell]);
mark(&mut doc, 0, &[[15.0, 135.0, 130.0, 175.0]], "");
assert_eq!(apply(&mut doc, None).unwrap().patterns_removed, 1, "the pattern is pruned");
// The state this test is about: the selector outlived the resource it names.
let after = content(&doc, 0);
assert!(after.contains("/P0 scn"), "the selector is still in the content: {after}");
assert!(!after.contains("20 140 100 30 re"), "the fill that used it is gone: {after}");
let bytes = write_full(&doc, &SaveOptions::default()).unwrap();
assert!(!bytes.windows(8).any(|w| w == b"(SECRET)"), "the cell's text is gone from the saved file");
let mut r = pdfcraft_render::PageRenderer::new(std::sync::Arc::new(bytes), pdfcraft_render::RenderConfig::default());
let out = r.render(pdfcraft_render::RenderRequest { page: 0, scale: 1.0, ..Default::default() });
assert!(out.error.is_none(), "the page still renders: {:?}, warnings {:?}", out.error, out.warnings);
assert_eq!((out.width, out.height), (300, 300), "the page box is unchanged");
// The text after the stale selector is still drawn: 24 pt at y 40, so the top-left origin
// puts it in the rows around 240..265. Without it the reader gave up at the bad operand.
let painted = |rows: std::ops::Range<u32>, cols: std::ops::Range<u32>| {
rows.flat_map(|y| cols.clone().map(move |x| (y, x))).any(|(y, x)| {
let i = ((y * out.width + x) * 4) as usize;
out.rgba[i..i + 3] != [255, 255, 255]
})
};
assert!(painted(236..268, 8..120), "the text after the stale selector is still painted");
// The mark itself is drawn over the area the pattern filled -- `apply` writes a black
// overlay there (`q 0 0 0 rg 15 135 115 40 re f Q`), so the cell cannot show through it
// even if a reader kept the stale selector.
let black = |y: u32, x: u32| {
let i = ((y * out.width + x) * 4) as usize;
out.rgba[i..i + 3] == [0, 0, 0]
};
assert!(black(145, 70), "the redaction overlay is painted over the pattern's box");
}
#[test]
fn redaction_cannot_leave_hidden_cell_text_when_pattern_is_used_elsewhere() {
let cell = stream(
"/Type /Pattern /PatternType 1 /PaintType 1 /TilingType 1 /BBox [0 0 300 300] /XStep 300 /YStep 300 /Resources << /Font << /F1 5 0 R >> >>",
b"0.9 g 0 0 300 300 re f 0 g BT /F1 10 Tf 30 150 Td (SECRET) Tj ET",
);
let content = b"/Pattern cs /P0 scn 20 140 100 30 re f 20 40 100 30 re f";
let mut doc = one_page(content, "/Pattern << /P0 7 0 R >>", vec![cell]);
mark(&mut doc, 0, &[[15.0, 135.0, 130.0, 175.0]], "");
let before = write_full(&doc, &SaveOptions::default()).unwrap();
assert!(matches!(apply(&mut doc, None), Err(RedactError::Pattern { reason: PatternProblem::Content, .. })));
assert_eq!(write_full(&doc, &SaveOptions::default()).unwrap(), before, "refusal must be atomic");
}
#[test]
fn redaction_cannot_leave_hidden_cell_text_on_an_unselected_page() {
let cell = stream(
"/Type /Pattern /PatternType 1 /PaintType 1 /TilingType 1 /BBox [0 0 300 300] /XStep 300 /YStep 300 /Resources << /Font << /F1 5 0 R >> >>",
b"BT /F1 10 Tf 30 150 Td (SECRET) Tj ET",
);
let mut doc = pdf(vec![
b"<< /Type /Catalog /Pages 2 0 R >>".to_vec(),
b"<< /Type /Pages /Kids [3 0 R 8 0 R] /Count 2 >>".to_vec(),
b"<< /Type /Page /Parent 2 0 R /MediaBox [0 0 300 300] /Contents 4 0 R /Resources << /Pattern << /P0 7 0 R >> >> >>".to_vec(),
stream("", b"/Pattern cs /P0 scn 20 140 100 30 re f"),
FONT.replace("95 0 R", "6 0 R").into_bytes(),
widths(),
cell,
b"<< /Type /Page /Parent 2 0 R /MediaBox [0 0 300 300] /Contents 9 0 R /Resources << /Pattern << /P0 7 0 R >> >> >>".to_vec(),
stream("", b"/Pattern cs /P0 scn 20 40 100 30 re f"),
]);
mark(&mut doc, 0, &[[15.0, 135.0, 130.0, 175.0]], "");
let before = write_full(&doc, &SaveOptions::default()).unwrap();
match apply(&mut doc, Some(&[0])) {
Ok(_) => {
let saved = write_full(&doc, &SaveOptions::default()).unwrap();
assert!(!saved.windows(8).any(|w| w == b"(SECRET)"), "the unselected page kept the hidden cell text");
}
Err(RedactError::Pattern { reason: PatternProblem::Content, .. }) => {
assert_eq!(write_full(&doc, &SaveOptions::default()).unwrap(), before, "refusal must keep both pages and the mark");
}
Err(error) => panic!("unexpected refusal: {error}"),
}
}
#[test]
fn redacted_form_must_not_retain_its_own_pattern_cell_text() {
let cell = stream(
"/Type /Pattern /PatternType 1 /PaintType 1 /TilingType 1 /BBox [0 0 300 300] /XStep 300 /YStep 300 /Resources << /Font << /F1 5 0 R >> >>",
b"BT /F1 10 Tf 30 150 Td (SECRET) Tj ET",
);
let form = stream(
"/Type /XObject /Subtype /Form /BBox [0 0 300 300] /Resources << /Pattern << /P0 7 0 R >> >>",
b"/Pattern cs /P0 scn 20 140 100 30 re f",
);
let mut doc = one_page(b"/Fm Do", "/XObject << /Fm 8 0 R >>", vec![cell, form]);
mark(&mut doc, 0, &[[15.0, 135.0, 130.0, 175.0]], "");
let before = write_full(&doc, &SaveOptions::default()).unwrap();
assert!(matches!(apply(&mut doc, None), Err(RedactError::Pattern { reason: PatternProblem::Content, .. })));
assert_eq!(write_full(&doc, &SaveOptions::default()).unwrap(), before, "form-owned pattern refusal must be atomic");
}
#[test]
fn redacted_form_must_not_retain_inherited_pattern_cell_text() {
let cell = stream(
"/Type /Pattern /PatternType 1 /PaintType 1 /TilingType 1 /BBox [0 0 300 300] /XStep 300 /YStep 300 /Resources << /Font << /F1 5 0 R >> >>",
b"BT /F1 10 Tf 30 150 Td (SECRET) Tj ET",
);
let form = stream("/Type /XObject /Subtype /Form /BBox [0 0 300 300]", b"20 140 100 30 re f");
let mut doc = one_page(b"/Pattern cs /P0 scn /Fm Do", "/Pattern << /P0 7 0 R >> /XObject << /Fm 8 0 R >>", vec![cell, form]);
mark(&mut doc, 0, &[[15.0, 135.0, 130.0, 175.0]], "");
let before = write_full(&doc, &SaveOptions::default()).unwrap();
assert!(matches!(apply(&mut doc, None), Err(RedactError::Pattern { reason: PatternProblem::Content, .. })));
assert_eq!(write_full(&doc, &SaveOptions::default()).unwrap(), before, "inherited pattern refusal must be atomic");
}
#[test]
fn partially_covered_vector_paths_do_not_leak_drawing_commands() {
// A vector path with detailed drawing commands (e.g. outlined text or confidential geometry)
// that is only partially covered by a redaction mark must be completely removed, not retained
// behind a clipping path (F-008).
let src = b"0 g 100 100 m 150 150 l 200 100 l h f 10 10 m 20 20 l W b 300 300 m 350 350 l S";
let mut doc = one_page(src, "", vec![]);
// Redaction mark covers only part of the triangle (100..200, 100..150):
// mark is [120, 120, 160, 160], partially intersecting the triangle.
// Also partially intersect the clip-and-stroke path [10..20, 10..20] with [15, 15, 25, 25].
mark(&mut doc, 0, &[[120.0, 120.0, 160.0, 160.0], [15.0, 15.0, 25.0, 25.0]], "");
let r = apply(&mut doc, None).unwrap();
assert_eq!(r.paths_removed, 2, "both partially intersected paths removed");
assert_eq!(r.paths_clipped, 0, "no paths retained via clipping");
let c = content(&doc, 0);
// Neither the drawing operators nor the coordinates should leak in the content stream.
assert!(!c.contains("100 100 m"), "underlying geometry must not leak: {c}");
assert!(!c.contains("150 150 l"), "underlying geometry must not leak: {c}");
assert!(!c.contains("200 100 l"), "underlying geometry must not leak: {c}");
assert!(!c.contains("10 10 m"), "partial clip geometry must not leak: {c}");
assert!(!c.contains("20 20 l"), "partial clip geometry must not leak: {c}");
// Unrelated non-intersecting path remains untouched.
assert!(c.contains("300 300 m\n350 350 l\nS"), "unaffected vector path must be kept: {c}");
}
#[test]
fn a_covered_clip_and_paint_path_leaves_an_empty_clip_not_its_outline() {
// The outline of a fully covered path that clips and paints is confidential geometry: only
// an empty clip may stay, and what follows is hidden as the covered clip would have hidden it.
let src = b"q 100 100 m 150 180 l 200 100 l h W f 300 300 50 50 re f Q";
let mut doc = one_page(src, "", vec![]);
mark(&mut doc, 0, &[[90.0, 90.0, 210.0, 190.0]], "");
let r = apply(&mut doc, None).unwrap();
assert_eq!(r.paths_removed, 1);
let c = content(&doc, 0);
assert!(!c.contains("150 180 l") && !c.contains("100 100 m"), "outline must not stay: {c}");
assert!(c.contains("0 0 0 0 re\nW\nn"), "an empty clip stays: {c}");
}
/// A chain of nested forms deeper than the pass can walk was removed from the page, taking
/// whatever else it drew with it even when nothing in it was marked. The limit now reaches as
/// deep as the renderer and Edit text read (50), so anything a reader can see can be walked;
/// past that nothing is drawn either, so removing it costs nothing visible (#3645).
#[test]
fn a_deeply_nested_form_keeps_the_text_nobody_marked() {
/// A page that draws `HEAD` at the top, then a chain of `depth` forms whose innermost draws
/// `DEEP`, then `TAIL`. Only `HEAD` is marked.
fn page_with_chain(depth: usize) -> Document {
let first = 7;
let mut objs: Vec<Vec<u8>> = vec![
b"<< /Type /Catalog /Pages 2 0 R >>".to_vec(),
b"<< /Type /Pages /Kids [3 0 R] /Count 1 >>".to_vec(),
format!(
"<< /Type /Page /Parent 2 0 R /MediaBox [0 0 300 300] /Contents 4 0 R /Resources << /Font << /F1 5 0 R >> /XObject << /Fm1 {first} 0 R >> >> >>"
)
.into_bytes(),
stream("", b"BT /F1 10 Tf 10 280 Td (HEAD) Tj ET /Fm1 Do BT /F1 10 Tf 10 10 Td (TAIL) Tj ET"),
FONT.replace("95 0 R", "6 0 R").into_bytes(),
widths(),
];
for k in 1..=depth {
let res = if k < depth {
format!("/Resources << /Font << /F1 5 0 R >> /XObject << /Fm{} {} 0 R >> >>", k + 1, first + k)
} else {
"/Resources << /Font << /F1 5 0 R >> >>".to_string()
};
let body: Vec<u8> = if k < depth { format!("/Fm{} Do", k + 1).into_bytes() } else { b"BT /F1 10 Tf 10 150 Td (DEEP) Tj ET".to_vec() };
objs.push(stream(&format!("/Type /XObject /Subtype /Form /BBox [0 0 300 300] {res}"), &body));
}
pdf(objs)
}
// Deeper than the old limit of 12 and within the new one: the innermost text survives.
for depth in [1, 12, 13, 20, 50] {
let mut doc = page_with_chain(depth);
mark(&mut doc, 0, &[[5.0, 275.0, 60.0, 295.0]], "");
let report = apply(&mut doc, None).unwrap();
assert_eq!(report.forms_removed, 0, "depth {depth}: a form was removed");
let c = content(&doc, 0);
assert!(c.contains("(TAIL)"), "depth {depth}: {c}");
// The innermost form's text is still in the saved file.
let saved = write_incremental(&doc, &SaveOptions::default()).unwrap();
assert!(saved.windows(6).any(|w| w == b"(DEEP)"), "depth {depth}: the innermost form's text was deleted");
}
// Past what any reader follows, the form is still removed: the fail-safe, and nothing the
// renderer draws is lost because it stops at 50 too.
let mut doc = page_with_chain(crate::interp::MAX_DEPTH + 1);
mark(&mut doc, 0, &[[5.0, 275.0, 60.0, 295.0]], "");
assert_eq!(apply(&mut doc, None).unwrap().forms_removed, 1, "a chain past the limit is removed");
}
/// Depth alone does not bound the walk: a form that draws the next level twice, or a cycle of two
/// forms, costs 2^depth interpretations, which at a depth limit of 50 is a hang on a file anyone
/// can send. The pass also counts the forms it enters, and a page past that budget is treated as
/// one it cannot read.
#[test]
fn a_form_that_fans_out_or_loops_is_bounded_by_the_page_budget() {
/// A page drawing `Fm1`, where each of `n` forms draws the next **twice** and the last draws
/// text: 2^n paths if nothing stops the walk. With `looping`, the last form draws the first
/// one again instead, so the chain has no end at all.
fn fan_out(n: usize, looping: bool) -> Document {
let first = 7;
let mut objs: Vec<Vec<u8>> = vec![
b"<< /Type /Catalog /Pages 2 0 R >>".to_vec(),
b"<< /Type /Pages /Kids [3 0 R] /Count 1 >>".to_vec(),
format!(
"<< /Type /Page /Parent 2 0 R /MediaBox [0 0 300 300] /Contents 4 0 R /Resources << /Font << /F1 5 0 R >> /XObject << /Fm1 {first} 0 R >> >> >>"
)
.into_bytes(),
stream("", b"BT /F1 10 Tf 10 280 Td (HEAD) Tj ET /Fm1 Do"),
FONT.replace("95 0 R", "6 0 R").into_bytes(),
widths(),
];
for k in 1..=n {
let (next, body) = if k < n {
(first + k, format!("/Fm{0} Do /Fm{0} Do", k + 1))
} else if looping {
(first, "/Fm1 Do /Fm1 Do".to_string())
} else {
(first, "BT /F1 10 Tf 10 150 Td (DEEP) Tj ET".to_string())
};
let name = if k < n { format!("/Fm{}", k + 1) } else { "/Fm1".to_string() };
let res = format!("/Resources << /Font << /F1 5 0 R >> /XObject << {name} {next} 0 R >> >>");
objs.push(stream(&format!("/Type /XObject /Subtype /Form /BBox [0 0 300 300] {res}"), body.as_bytes()));
}
pdf(objs)
}
for looping in [false, true] {
let start = std::time::Instant::now();
let mut doc = fan_out(crate::interp::MAX_DEPTH, looping);
mark(&mut doc, 0, &[[5.0, 275.0, 60.0, 295.0]], "");
let result = apply(&mut doc, None);
let took = start.elapsed();
// An unbounded walk would never end; the limit leaves room for a loaded test machine.
assert!(took < std::time::Duration::from_secs(120), "looping {looping}: the walk took {took:?}");
// The page's form budget stops the walk, and Apply refuses the page rather than
// removing forms it could not read.
assert!(result.is_err(), "looping {looping}: the page past its form budget is refused");
}
}
#[test]
fn shared_form_xobjects_are_copied_not_changed() {
let form =
stream("/Type /XObject /Subtype /Form /BBox [0 0 300 300] /Resources << /Font << /F1 5 0 R >> >>", b"BT /F1 10 Tf 10 100 Td (SECRET) Tj ET");
let mut objs: Vec<Vec<u8>> = vec![
b"<< /Type /Catalog /Pages 2 0 R >>".to_vec(),
b"<< /Type /Pages /Kids [3 0 R 8 0 R] /Count 2 /MediaBox [0 0 300 300] /Resources << /XObject << /Fm 7 0 R >> >> >>".to_vec(),
b"<< /Type /Page /Parent 2 0 R /Contents 4 0 R >>".to_vec(),
stream("", b"/Fm Do"),
FONT.replace("95 0 R", "6 0 R").into_bytes(),
widths(),
form,
];
objs.push(b"<< /Type /Page /Parent 2 0 R /Contents 4 0 R >>".to_vec());
let mut doc = pdf(objs);
mark(&mut doc, 0, &[[0.0, 90.0, 300.0, 120.0]], "");
let r = apply(&mut doc, None).unwrap();
assert_eq!((r.glyphs, r.forms_rewritten), (6, 1), "{r:?}");
assert!(content(&doc, 0).contains("/PCRedacted1 Do"));
assert_eq!(content(&doc, 1), "/Fm Do", "page 2 shares the stream and the form: untouched");
assert_eq!(under(&mut doc, 1, &[[0.0, 90.0, 300.0, 120.0]]), 6);
assert_eq!(under(&mut doc, 0, &[[0.0, 90.0, 300.0, 120.0]]), 0);
}
#[test]
fn redacted_images_and_forms_do_not_survive_a_full_rewrite_save() {
// Page 1 draws a fully covered image, a partly covered image and a form XObject whose text
// is under the mark. After Apply Redactions and the forced full rewrite, none of their secret
// bytes may stay in the saved file, while an unrelated image drawn only on page 2 survives.
let im1 = b"REDIMGONECOVERED"; // 16 bytes, one gray row
let impart = b"REDIMGPARTSECRETPUBPARTBOTTOMROW"; // 16-byte secret top row + 16-byte bottom row
let imkeep = b"PUBIMGKEEPSURVIVE"; // 17 bytes, used only on page 2
let objs: Vec<Vec<u8>> = vec![
b"<< /Type /Catalog /Pages 2 0 R >>".to_vec(),
b"<< /Type /Pages /Kids [3 0 R 10 0 R] /Count 2 >>".to_vec(),
b"<< /Type /Page /Parent 2 0 R /MediaBox [0 0 300 300] /Contents 4 0 R /Resources << /Font << /F1 5 0 R >> /XObject << /Im1 7 0 R /ImPart 8 0 R /FmSecret 9 0 R >> >> >>".to_vec(),
stream("", b"q 50 0 0 50 20 230 cm /Im1 Do Q q 50 0 0 100 100 180 cm /ImPart Do Q /FmSecret Do"),
FONT.replace("95 0 R", "6 0 R").into_bytes(),
widths(),
stream("/Type /XObject /Subtype /Image /Width 16 /Height 1 /ColorSpace /DeviceGray /BitsPerComponent 8", im1),
stream("/Type /XObject /Subtype /Image /Width 16 /Height 2 /ColorSpace /DeviceGray /BitsPerComponent 8", impart),
stream(
"/Type /XObject /Subtype /Form /BBox [0 0 300 300] /Resources << /Font << /F1 5 0 R >> >>",
b"BT /F1 10 Tf 20 230 Td (REDFORMSECRETTEXT) Tj ET",
),
b"<< /Type /Page /Parent 2 0 R /MediaBox [0 0 300 300] /Contents 11 0 R /Resources << /XObject << /ImKeep 12 0 R >> >> >>".to_vec(),
stream("", b"q 50 0 0 50 20 20 cm /ImKeep Do Q"),
stream("/Type /XObject /Subtype /Image /Width 17 /Height 1 /ColorSpace /DeviceGray /BitsPerComponent 8", imkeep),
];
let mut doc = pdf(objs);
mark(&mut doc, 0, &[[0.0, 200.0, 300.0, 300.0]], "");
let r = apply(&mut doc, None).unwrap();
assert!(r.images_removed >= 1 && r.forms_rewritten >= 1, "{r:?}");
// The full rewrite that redaction forces must not carry the removed content forward.
let bytes = write_full(&doc, &SaveOptions::default()).unwrap();
let has = |needle: &[u8]| bytes.windows(needle.len()).any(|w| w == needle);
assert!(!has(b"REDIMGONECOVERED"), "covered image samples survived the save");
assert!(!has(b"REDIMGPARTSECRET"), "partly covered image's secret row survived the save");
assert!(!has(b"REDFORMSECRETTEXT"), "the form's redacted text survived the save");
// The unrelated image drawn only on page 2 stays referenced and intact.
let saved = Document::open(Arc::new(bytes)).unwrap();
let keep = pdfcraft_model::pages(&saved).swap_remove(1);
let res = saved.resolve(keep.dict.get(b"Resources").unwrap()).as_dict().cloned().unwrap();
let xo = saved.resolve(res.get(b"XObject").unwrap()).as_dict().cloned().unwrap();
let kref = xo.get(b"ImKeep").and_then(Object::as_ref).expect("ImKeep still referenced");
let Object::Stream(s) = &*saved.get(kref) else { panic!("ImKeep is a stream") };
assert_eq!(s.decoded().unwrap(), imkeep, "the unrelated image survives intact");
}
/// Two pages that draw through one resource dictionary: inherited from the /Pages node, or one
/// dictionary object both pages refer to. Page 1 draws a covered image, a partly covered image
/// and a form whose text is under the mark; page 2 draws /ImKeep, and /Im1 too when `again`.
fn two_pages_one_resource_dictionary(inherited: bool, again: bool) -> Document {
let res = "<< /Font << /F1 5 0 R >> /XObject << /Im1 7 0 R /ImPart 8 0 R /FmSecret 9 0 R /ImKeep 12 0 R >> >>";
let (pages_res, page_res) = if inherited { (format!("/Resources {res}"), "") } else { (String::new(), "/Resources 13 0 R") };
let page2: &[u8] =
if again { b"q 50 0 0 50 20 20 cm /ImKeep Do Q q 50 0 0 50 120 20 cm /Im1 Do Q" } else { b"q 50 0 0 50 20 20 cm /ImKeep Do Q" };
pdf(vec![
b"<< /Type /Catalog /Pages 2 0 R >>".to_vec(),
format!("<< /Type /Pages /Kids [3 0 R 10 0 R] /Count 2 {pages_res} >>").into_bytes(),
format!("<< /Type /Page /Parent 2 0 R /MediaBox [0 0 300 300] /Contents 4 0 R {page_res} >>").into_bytes(),
stream("", b"q 50 0 0 50 20 230 cm /Im1 Do Q q 50 0 0 100 100 180 cm /ImPart Do Q /FmSecret Do"),
FONT.replace("95 0 R", "6 0 R").into_bytes(),
widths(),
stream("/Type /XObject /Subtype /Image /Width 16 /Height 1 /ColorSpace /DeviceGray /BitsPerComponent 8", b"REDIMGONECOVERED"),
stream("/Type /XObject /Subtype /Image /Width 16 /Height 2 /ColorSpace /DeviceGray /BitsPerComponent 8", b"REDIMGPARTSECRETPUBPARTBOTTOMROW"),
stream(
"/Type /XObject /Subtype /Form /BBox [0 0 300 300] /Resources << /Font << /F1 5 0 R >> >>",
b"BT /F1 10 Tf 20 230 Td (REDFORMSECRETTEXT) Tj ET",
),
format!("<< /Type /Page /Parent 2 0 R /MediaBox [0 0 300 300] /Contents 11 0 R {page_res} >>").into_bytes(),
stream("", page2),
stream("/Type /XObject /Subtype /Image /Width 17 /Height 1 /ColorSpace /DeviceGray /BitsPerComponent 8", b"PUBIMGKEEPSURVIVE"),
// 13: the dictionary both pages refer to (unused when the resources are inherited).
if inherited { b"<< >>".to_vec() } else { res.as_bytes().to_vec() },
])
}
/// What image `s` decodes to, its fax or JBIG2 data included, with no limit on the work: the
/// samples an operation would read through its `ImageBudget`.
fn decoded_image(doc: &Document, s: &pdfcraft_cos::Stream) -> Result<Vec<u8>, pdfcraft_cos::CosError> {
pdfcraft_cos::ImageBudget::with_work(u64::MAX).decode(doc, s, pdfcraft_cos::MAX_DECODED)
}
/// The samples of the image a page's (possibly inherited) resources name `name`, decoded through
/// its image codec too ([`decoded_image`]).
fn named_image(doc: &Document, page: usize, name: &[u8]) -> Option<Vec<u8>> {
named_stream(doc, page, name).and_then(|s| decoded_image(doc, &s).ok())
}
/// The image a page's (possibly inherited) resources name `name`.
fn named_stream(doc: &Document, page: usize, name: &[u8]) -> Option<Stream> {
let p = pdfcraft_model::pages(doc).swap_remove(page);
let res = doc.resolve(p.dict.get(b"Resources")?).as_dict().cloned()?;
let xo = doc.resolve(res.get(b"XObject")?).as_dict().cloned()?;
match &*doc.get(xo.get(name)?.as_ref()?) {
Object::Stream(s) => Some(s.clone()),
_ => None,
}
}
#[test]
fn redaction_clears_pixels_intersected_without_covering_their_centres() {
for (content, rect) in
[(&b"q 200 0 0 100 0 0 cm /Im Do Q"[..], [10.0, 10.0, 20.0, 90.0]), (&b"q 0 100 -200 0 200 0 cm /Im Do Q"[..], [10.0, 5.0, 190.0, 10.0])]
{
let mut doc = one_page(
content,
"/XObject << /Im 7 0 R >>",
vec![stream("/Type /XObject /Subtype /Image /Width 2 /Height 1 /ColorSpace /DeviceGray /BitsPerComponent 8", [123, 234])],
);
mark(&mut doc, 0, &[rect], "");
let report = apply(&mut doc, None).unwrap();
assert_eq!(report.images_cleared, 1);
let saved = Document::open(Arc::new(write_full(&doc, &SaveOptions::default()).unwrap())).unwrap();
assert_eq!(named_image(&saved, 0, b"Im"), None);
assert_eq!(named_image(&saved, 0, b"PCRedacted1"), Some(vec![0, 234]));
}
}
/// An image whose PNG predictor is given by reference has its covered pixel cleared, and only
/// that one, whichever part is the reference: the dictionary, an array's entry, or the number
/// itself. `Stream::decoded` reads only a direct `/DecodeParms` with direct numbers, so the row
/// tags stayed in the data, the cleared byte drifted off the covered pixel and the covered sample
/// survived.
#[test]
fn an_image_whose_predictor_is_given_by_reference_has_the_covered_pixel_cleared() {
// Two rows of four grey samples, each behind a PNG "None" row tag.
let predicted = pdfcraft_filters::encode_flate(&[0, 10, 20, 30, 40, 0, 50, 60, 70, 80]);
let parms = "/Predictor 15 /Colors 1 /BitsPerComponent 8 /Columns 4";
for (decode_parms, object8) in [
("8 0 R".to_owned(), format!("<< {parms} >>")),
("[8 0 R]".to_owned(), format!("<< {parms} >>")),
("<< /Predictor 8 0 R /Colors 1 /BitsPerComponent 8 /Columns 4 >>".to_owned(), "15".to_owned()),
] {
let image = format!(
"/Type /XObject /Subtype /Image /Width 4 /Height 2 /ColorSpace /DeviceGray /BitsPerComponent 8 /Filter /FlateDecode /DecodeParms {decode_parms}"
);
let mut doc = one_page(b"q 40 0 0 20 0 0 cm /Im Do Q", "/XObject << /Im 7 0 R >>", vec![stream(&image, &predicted), object8.into_bytes()]);
// The top row's second sample.
mark(&mut doc, 0, &[[12.0, 12.0, 18.0, 18.0]], "");
let report = apply(&mut doc, None).unwrap();
assert_eq!(report.images_cleared, 1, "{decode_parms}");
let saved = Document::open(Arc::new(write_full(&doc, &SaveOptions::default()).unwrap())).unwrap();
assert_eq!(named_image(&saved, 0, b"PCRedacted1"), Some(vec![10, 0, 30, 40, 50, 60, 70, 80]), "{decode_parms}");
}
}
/// A CCITT fax image (a scan the optimizer or Create PDF stored as Group 4) has its covered
/// pixels cleared in a copy, like any image PdfCraft can decode, instead of being removed whole:
/// whether it sits behind another filter, with its parameters given by reference, or is a mask,
/// its decode array inverting it given directly, by reference or with its first number so.
#[test]
fn a_fax_image_has_its_covered_pixels_cleared_instead_of_being_removed() {
// 16 × 2, all 1 bits (white), or all 0 bits (a mask painting everywhere); the left half of
// the top row is covered.
let g4 = pdfcraft_filters::encode_g4(&[0xFF; 4], 16, 2).unwrap();
let painted = pdfcraft_filters::encode_g4(&[0x00; 4], 16, 2).unwrap();
let hex: Vec<u8> = g4.iter().map(|b| format!("{b:02x}")).collect::<String>().bytes().chain(*b">").collect();
let image = "/Type /XObject /Subtype /Image /Width 16 /Height 2 /BitsPerComponent 1";
let parms = "<< /K -1 /Columns 16 /Rows 2 >>";
for (dict, data, extra, cleared) in [
(format!("{image} /ColorSpace /DeviceGray /Filter /CCITTFaxDecode /DecodeParms {parms}"), g4.clone(), vec![], [0x00, 0xFF, 0xFF, 0xFF]),
(
format!("{image} /ColorSpace /DeviceGray /Filter [/ASCIIHexDecode /CCITTFaxDecode] /DecodeParms [null 8 0 R]"),
hex,
vec![parms.as_bytes().to_vec()],
[0x00, 0xFF, 0xFF, 0xFF],
),
// A mask's 0 bits paint: cleared bits are set to 1, which paints nothing.
(format!("{image} /ImageMask true /Filter /CCITTFaxDecode /DecodeParms {parms}"), painted, vec![], [0xFF, 0x00, 0x00, 0x00]),
// With /Decode [1 0] a mask's 1 bits paint, and cleared bits are 0.
(format!("{image} /ImageMask true /Decode [1 0] /Filter /CCITTFaxDecode /DecodeParms {parms}"), g4.clone(), vec![], [0x00, 0xFF, 0xFF, 0xFF]),
(
format!("{image} /ImageMask true /Decode 8 0 R /Filter /CCITTFaxDecode /DecodeParms {parms}"),
g4.clone(),
vec![b"[1 0]".to_vec()],
[0x00, 0xFF, 0xFF, 0xFF],
),
(
format!("{image} /ImageMask true /Decode [8 0 R 0] /Filter /CCITTFaxDecode /DecodeParms {parms}"),
g4.clone(),
vec![b"1".to_vec()],
[0x00, 0xFF, 0xFF, 0xFF],
),
// Through a reference to a reference: the array, its first number, and a colour space.
(
format!("{image} /ImageMask true /Decode 8 0 R /Filter /CCITTFaxDecode /DecodeParms {parms}"),
g4.clone(),
vec![b"9 0 R".to_vec(), b"[1 0]".to_vec()],
[0x00, 0xFF, 0xFF, 0xFF],
),
(
format!("{image} /ImageMask true /Decode [8 0 R 0] /Filter /CCITTFaxDecode /DecodeParms {parms}"),
g4.clone(),
vec![b"9 0 R".to_vec(), b"1".to_vec()],
[0x00, 0xFF, 0xFF, 0xFF],
),
(
format!("{image} /ColorSpace 8 0 R /Filter /CCITTFaxDecode /DecodeParms {parms}"),
g4.clone(),
vec![b"9 0 R".to_vec(), b"/DeviceGray".to_vec()],
[0x00, 0xFF, 0xFF, 0xFF],
),
] {
let mut objs = vec![stream(&dict, data)];
objs.extend(extra);
let mut doc = one_page(b"q 160 0 0 20 0 0 cm /Im Do Q", "/XObject << /Im 7 0 R >>", objs);
mark(&mut doc, 0, &[[0.0, 12.0, 75.0, 18.0]], "");
let report = apply(&mut doc, None).unwrap();
assert_eq!((report.images_cleared, report.images_removed), (1, 0), "{dict}");
let saved = Document::open(Arc::new(write_full(&doc, &SaveOptions::default()).unwrap())).unwrap();
assert_eq!(named_image(&saved, 0, b"PCRedacted1").as_deref(), Some(&cleared[..]), "{dict}");
}
}
/// Images whose data is more than twice their raster before its last filter are cleared, not
/// taken for decompression bombs: grey stored as ASCIIHex text broken into lines under Flate (two
/// digits a byte and a line break every 64), and RGB behind an array of filters, which a fax
/// image's bound (a byte a pixel) would refuse.
#[test]
fn images_behind_ascii_hex_in_lines_or_a_filter_array_are_cleared() {
let (w, h) = (1000usize, 200usize);
let grey: Vec<u8> = (0..w * h).map(|i| (i % 251) as u8).collect();
let digits: Vec<u8> = grey.iter().flat_map(|b| format!("{b:02X}").into_bytes()).collect();
let mut lines: Vec<u8> = digits.chunks(64).flat_map(|line| line.iter().copied().chain(*b"\n")).collect();
lines.push(b'>');
let rgb: Vec<u8> = (0..100 * 100 * 3).map(|i| (i % 253) as u8).collect();
let rgb_hex: Vec<u8> = pdfcraft_filters::encode_flate(&rgb).iter().flat_map(|b| format!("{b:02x}").into_bytes()).chain(*b">").collect();
for (dict, data) in [
(
format!(
"/Type /XObject /Subtype /Image /Width {w} /Height {h} /ColorSpace /DeviceGray /BitsPerComponent 8 /Filter [/FlateDecode /ASCIIHexDecode]"
),
pdfcraft_filters::encode_flate(&lines),
),
(
"/Type /XObject /Subtype /Image /Width 100 /Height 100 /ColorSpace /DeviceRGB /BitsPerComponent 8 /Filter [/ASCIIHexDecode /FlateDecode]"
.to_string(),
rgb_hex,
),
] {
let mut doc = one_page(b"q 100 0 0 20 0 0 cm /Im Do Q", "/XObject << /Im 7 0 R >>", vec![stream(&dict, data)]);
mark(&mut doc, 0, &[[0.0, 0.0, 10.0, 10.0]], "");
let report = apply(&mut doc, None).unwrap();
assert_eq!((report.images_cleared, report.images_removed), (1, 0), "{dict}");
}
}
/// A fax image that can't be decoded in full is removed whole, as any other image is: one whose
/// code is cut short, and one behind a filter that inflates past what a code for its pixels can
/// take (a decompression bomb, which `decoded_image` alone would undo up to a gigabyte).
#[test]
fn a_fax_image_that_cannot_be_decoded_in_full_is_removed_whole() {
let g4 = pdfcraft_filters::encode_g4(&[0xFF; 4], 16, 2).unwrap();
// A busy picture's code, cut off in its first row.
let busy = pdfcraft_filters::encode_g4(&[0xAA, 0xAA, 0x55, 0x55], 16, 2).unwrap();
let short = busy.get(..1).unwrap().to_vec();
// The real code, then four megabytes of padding the fax decoder would never read.
let mut padded = g4.clone();
padded.resize(4 << 20, 0);
let bomb = pdfcraft_filters::encode_flate(&padded);
let image = "/Type /XObject /Subtype /Image /Width 16 /Height 2 /ColorSpace /DeviceGray /BitsPerComponent 1";
let parms = "<< /K -1 /Columns 16 /Rows 2 >>";
for (filters, data) in [("/CCITTFaxDecode /DecodeParms", short), ("[/FlateDecode /CCITTFaxDecode] /DecodeParms [null", bomb)] {
let close = if filters.ends_with("[null") { "]" } else { "" };
let dict = format!("{image} /Filter {filters} {parms}{close}");
let mut doc = one_page(b"q 160 0 0 20 0 0 cm /Im Do Q", "/XObject << /Im 7 0 R >>", vec![stream(&dict, data)]);
mark(&mut doc, 0, &[[0.0, 12.0, 75.0, 18.0]], "");
let report = apply(&mut doc, None).unwrap();
assert_eq!((report.images_cleared, report.images_removed), (0, 1), "{dict}");
}
}
/// Group 3 on bytes is removed whole: the renderer draws some of its layouts as other rows than
/// the decoder reads, so the pixels a mark covers on screen needn't be the ones cleared. Group 3
/// off bytes and Group 4 on bytes are cleared. Each picture is 16 × 2 white, each row a white run
/// of 16 (`101010`): off bytes; on bytes with no end-of-line codes; on bytes after codes whose fill
/// ends each on a byte, at `/K 0` and at `/K 1` (a 1 after each code), `/EncodedByteAlign` given by
/// reference there; and Group 4 rows on bytes, each the one code 1 (vertical, no change).
#[test]
fn group_3_on_bytes_is_removed_whole_and_other_fax_images_cleared() {
let image = "/Type /XObject /Subtype /Image /Width 16 /Height 2 /ColorSpace /DeviceGray /BitsPerComponent 1 /Filter /CCITTFaxDecode";
for (k, on_bytes, parms, data, extra, removed) in [
(0, false, "<< /K 0 /Columns 16 >>", vec![0xAA, 0xA0], vec![], false),
(0, true, "<< /K 0 /Columns 16 /EncodedByteAlign true >>", vec![0xA8, 0xA8], vec![], true),
(0, true, "<< /K 0 /Columns 16 /EncodedByteAlign true >>", vec![0x00, 0x01, 0xA8, 0x00, 0x01, 0xA8], vec![], true),
(1, true, "<< /K 1 /Columns 16 /EncodedByteAlign 8 0 R >>", vec![0x00, 0x01, 0xD4, 0x00, 0x01, 0xD4], vec![b"true".to_vec()], true),
(-1, true, "<< /K -1 /Columns 16 /EncodedByteAlign true >>", vec![0x80, 0x80], vec![], false),
] {
let p = pdfcraft_filters::CcittParams { k, columns: 16, encoded_byte_align: on_bytes, ..Default::default() };
let mut work = u64::MAX;
assert_eq!(pdfcraft_filters::decode_ccitt_within(&data, &p, 2, &mut work).unwrap(), [0xFF; 4], "{parms}: the picture is white");
let dict = format!("{image} /DecodeParms {parms}");
let mut objs = vec![stream(&dict, data)];
objs.extend(extra);
let mut doc = one_page(b"q 160 0 0 20 0 0 cm /Im Do Q", "/XObject << /Im 7 0 R >>", objs);
mark(&mut doc, 0, &[[0.0, 12.0, 75.0, 18.0]], "");
let report = apply(&mut doc, None).unwrap();
if removed {
assert_eq!((report.images_cleared, report.images_removed), (0, 1), "{parms}");
} else {
assert_eq!((report.images_cleared, report.images_removed), (1, 0), "{parms}");
let saved = Document::open(Arc::new(write_full(&doc, &SaveOptions::default()).unwrap())).unwrap();
assert_eq!(named_image(&saved, 0, b"PCRedacted1").as_deref(), Some(&[0x00, 0xFF, 0xFF, 0xFF][..]), "{parms}");
}
}
}
/// A busy fax image is cleared, though its code is bigger than three times its raster (the bound
/// for other images): a code takes up to about a byte per pixel, and the fax image gets that much.
#[test]
fn a_busy_fax_image_behind_another_filter_is_cleared() {
// 256 × 256, rows of a one-pixel checkerboard between white ones: each checkered row is coded
// afresh against the white one above it, about four bits a pixel.
let rows: Vec<u8> = (0..256).flat_map(|y| [if y % 2 == 0 { 0xAA } else { 0xFF }; 32]).collect();
let g4 = pdfcraft_filters::encode_g4(&rows, 256, 256).unwrap();
let raster = 32 * 256;
assert!(g4.len() > (raster + 256) * 3 + 4096, "{} bytes: within the bound for other images", g4.len());
let dict = "/Type /XObject /Subtype /Image /Width 256 /Height 256 /ColorSpace /DeviceGray /BitsPerComponent 1 \
/Filter [/FlateDecode /CCITTFaxDecode] /DecodeParms [null << /K -1 /Columns 256 /Rows 256 >>]";
let mut doc = one_page(b"q 256 0 0 256 0 0 cm /Im Do Q", "/XObject << /Im 7 0 R >>", vec![stream(dict, pdfcraft_filters::encode_flate(&g4))]);
mark(&mut doc, 0, &[[0.0, 0.0, 10.0, 10.0]], "");
let report = apply(&mut doc, None).unwrap();
assert_eq!((report.images_cleared, report.images_removed), (1, 0));
}
/// After a save no stream holds the pixels under a mark, read as it would be shown, JBIG2 included,
/// on a scan coded with symbols and on one coded as a generic region. `COVERED` and `KEPT` are pixels
/// found nowhere else in the image, one lot under the mark and one beside it, so the search is seen
/// to find what stays. The one letter over and over, lower down, is what makes symbols worth coding.
/// The picture saved is the one with every pixel under the mark cleared and no other changed, so a
/// search that one cleared byte would blind isn't all that holds this.
#[test]
fn no_stream_holds_the_pixels_under_a_mark_on_a_jbig2_scan() {
const COVERED: [u8; 8] = [0x0F, 0x3C, 0x5A, 0xC3, 0x96, 0x69, 0xA5, 0x1E];
const KEPT: [u8; 8] = [0x87, 0x2D, 0x4B, 0xE1, 0x78, 0xD2, 0xB4, 0x1F];
const LETTER: [&str; 7] = ["#####", "#....", "####.", "#....", "#....", "#....", "#####"];
let (w, h) = (512usize, 64usize);
let stride = w / 8;
let mut bits = vec![0xFFu8; stride * h];
bits[stride * 2..stride * 2 + 8].copy_from_slice(&COVERED);
bits[stride * 2 + 16..stride * 2 + 24].copy_from_slice(&KEPT);
for top in (16..h - 8).step_by(10) {
for x in (4..w - 8).step_by(8) {
for (dy, row) in LETTER.iter().enumerate() {
for (dx, _) in row.bytes().enumerate().filter(|(_, c)| *c == b'#') {
bits[(top + dy) * stride + (x + dx) / 8] &= !(0x80 >> ((x + dx) % 8));
}
}
}
}
let holds = |doc: &Document, pixels: &[u8]| {
doc.scan_objects().any(|(_, object)| match &*object {
Object::Stream(s) => decoded_image(doc, s).is_ok_and(|d| d.windows(pixels.len()).any(|w| w == pixels)),
_ => false,
})
};
// Columns 0 to 99 of rows 0 to 7.
let mut cleared = bits.clone();
for row in cleared.chunks_mut(stride).take(8) {
row[..12].fill(0);
row[12] &= 0x0F;
}
let symbols = pdfcraft_filters::encode_jbig2(&bits, w as u32, h as u32).unwrap();
let generic = pdfcraft_filters::encode_jbig2_generic(&bits, w as u32, h as u32).unwrap();
assert!(symbols.len() < generic.len(), "symbol coding has to be what this tests");
for coded in [symbols, generic] {
let dict = format!("/Type /XObject /Subtype /Image /Width {w} /Height {h} /BitsPerComponent 1 /ColorSpace /DeviceGray /Filter /JBIG2Decode");
// Half a point a pixel, its top row at y 132.
let mut doc = one_page(b"q 256 0 0 32 0 100 cm /Im Do Q", "/XObject << /Im 7 0 R >>", vec![stream(&dict, coded)]);
assert!(holds(&doc, &COVERED) && holds(&doc, &KEPT), "the pixels are there to be found");
mark(&mut doc, 0, &[[0.25, 128.25, 49.75, 132.0]], "");
assert_eq!(apply(&mut doc, None).unwrap().images_cleared, 1);
let saved = Document::open(Arc::new(write_full(&doc, &SaveOptions::default()).unwrap())).unwrap();
assert!(!holds(&saved, &COVERED), "the covered pixels stayed in the file");
assert!(holds(&saved, &KEPT), "the pixels beside them stay");
assert!(holds(&saved, &cleared), "every pixel under the mark is cleared, and no other changed");
}
}
/// `/JBIG2Globals` data a decoder skips: one extension segment (ISO/IEC 14492 7.4.14) of a type it
/// needn't read, holding `secret`; its 11-byte header gives the length of the data after it.
fn jbig2_globals(secret: &[u8]) -> Vec<u8> {
let mut g = vec![0, 0, 0, 0, 0x3E, 0, 0];
g.extend_from_slice(&u32::try_from(4 + secret.len()).unwrap().to_be_bytes());
g.extend_from_slice(&[0, 0, 0, 1]);
g.extend_from_slice(secret);
g
}
/// An image the run's allowance can't decode can't be cleared. Removed whole, like an image that
/// can't be read, it would take what no mark covers with it without a word, so the run stops there,
/// and [`apply`] leaves the document as it was.
#[test]
fn a_jbig2_image_past_the_runs_allowance_stops_the_run() {
let mut dict = Dict::new();
dict.set(b"Width".to_vec(), Object::Int(16));
dict.set(b"Height".to_vec(), Object::Int(2));
dict.set(b"BitsPerComponent".to_vec(), Object::Int(1));
dict.set(b"ColorSpace".to_vec(), Object::name("DeviceGray"));
dict.set(b"Filter".to_vec(), Object::name("JBIG2Decode"));
let image = Stream::from_raw(dict, pdfcraft_filters::encode_jbig2_generic(&[0xFF; 4], 16, 2).unwrap());
let doc = Document::new_empty();
let half = [[0.0, 0.0, 0.5, 1.0]];
let clear = |work| crate::image::clear(&doc, &image, &pdfcraft_content::Matrix::IDENTITY, &half, &mut pdfcraft_cos::ImageBudget::with_work(work));
assert!(matches!(clear(u64::MAX), Ok(Some(_))), "cleared in a copy");
assert!(matches!(clear(0), Err(())), "it can't be cleared");
let coded = pdfcraft_filters::encode_jbig2_generic(&[0xFF; 4], 16, 2).unwrap();
let dict = "/Type /XObject /Subtype /Image /Width 16 /Height 2 /BitsPerComponent 1 /ColorSpace /DeviceGray /Filter /JBIG2Decode";
let mut doc = one_page(b"q 160 0 0 20 0 0 cm /Im Do Q", "/XObject << /Im 7 0 R >>", vec![stream(dict, coded)]);
mark(&mut doc, 0, &[[0.0, 12.0, 75.0, 18.0]], "");
let refused = crate::apply_within(&mut doc, None, pdfcraft_cos::ImageBudget::with_work(0));
assert!(matches!(refused, Err(crate::RedactError::ImagesPastAllowance)), "{refused:?}");
let mut doc = one_page(
b"q 160 0 0 20 0 0 cm /Im Do Q",
"/XObject << /Im 7 0 R >>",
vec![stream(dict, pdfcraft_filters::encode_jbig2_generic(&[0xFF; 4], 16, 2).unwrap())],
);
mark(&mut doc, 0, &[[0.0, 12.0, 75.0, 18.0]], "");
assert!(crate::apply_within(&mut doc, None, pdfcraft_cos::ImageBudget::with_work(u64::MAX)).is_ok());
}
/// A Flate image past the run's allowance stops the run as a JBIG2 image does: removed whole, it
/// would take what no mark covers with it, unreported. A JPEG, which clearing doesn't read, is
/// removed whole as it always was, and spends nothing.
#[test]
fn a_flate_image_past_the_runs_allowance_stops_the_run() {
let dict = "/Type /XObject /Subtype /Image /Width 2 /Height 2 /BitsPerComponent 8 /ColorSpace /DeviceGray /Filter /FlateDecode";
let page = || one_page(b"q 160 0 0 20 0 0 cm /Im Do Q", "/XObject << /Im 7 0 R >>", vec![stream(dict, pdfcraft_filters::encode_flate(&[9; 4]))]);
let mut doc = page();
mark(&mut doc, 0, &[[0.0, 12.0, 75.0, 18.0]], "");
let refused = crate::apply_within(&mut doc, None, pdfcraft_cos::ImageBudget::with_work(0));
assert!(matches!(refused, Err(crate::RedactError::ImagesPastAllowance)), "{refused:?}");
let mut doc = page();
mark(&mut doc, 0, &[[0.0, 12.0, 75.0, 18.0]], "");
assert!(crate::apply_within(&mut doc, None, pdfcraft_cos::ImageBudget::with_work(u64::MAX)).is_ok());
let dict = "/Type /XObject /Subtype /Image /Width 2 /Height 2 /BitsPerComponent 8 /ColorSpace /DeviceGray /Filter /DCTDecode";
let mut doc = one_page(b"q 160 0 0 20 0 0 cm /Im Do Q", "/XObject << /Im 7 0 R >>", vec![stream(dict, b"not read")]);
mark(&mut doc, 0, &[[0.0, 12.0, 75.0, 18.0]], "");
let report = crate::apply_within(&mut doc, None, pdfcraft_cos::ImageBudget::with_work(0)).unwrap();
assert_eq!(report.images_removed, 1);
}
/// A JBIG2 image (a scan the optimizer stored as JBIG2) has its covered pixels cleared in a copy,
/// like a fax image, instead of being removed whole: with `/JBIG2Globals`, its `/DecodeParms` by
/// reference, behind another filter with its globals at its own place, or as a mask, its decode
/// array inverting it or not.
#[test]
fn a_jbig2_image_has_its_covered_pixels_cleared_instead_of_being_removed() {
// 16 × 2, all 1 bits (white), or all 0 bits (a mask painting everywhere); the left half of
// the top row is covered.
let white = pdfcraft_filters::encode_jbig2_generic(&[0xFF; 4], 16, 2).unwrap();
let painted = pdfcraft_filters::encode_jbig2_generic(&[0x00; 4], 16, 2).unwrap();
let hex: Vec<u8> = white.iter().map(|b| format!("{b:02x}")).collect::<String>().bytes().chain(*b">").collect();
let globals = stream("", jbig2_globals(b"shared"));
let image = "/Type /XObject /Subtype /Image /Width 16 /Height 2 /BitsPerComponent 1";
for (dict, data, extra, cleared) in [
(format!("{image} /ColorSpace /DeviceGray /Filter /JBIG2Decode"), white.clone(), vec![], [0x00, 0xFF, 0xFF, 0xFF]),
(
format!("{image} /ColorSpace /DeviceGray /Filter /JBIG2Decode /DecodeParms 8 0 R"),
white.clone(),
vec![b"<< /JBIG2Globals 9 0 R >>".to_vec(), globals.clone()],
[0x00, 0xFF, 0xFF, 0xFF],
),
(
format!("{image} /ColorSpace /DeviceGray /Filter [/ASCIIHexDecode /JBIG2Decode] /DecodeParms [null << /JBIG2Globals 8 0 R >>]"),
hex,
vec![globals.clone()],
[0x00, 0xFF, 0xFF, 0xFF],
),
// `/Crypt` was undone on load, and takes a place in the parameters.
(
format!("{image} /ColorSpace /DeviceGray /Filter [/Crypt /JBIG2Decode] /DecodeParms [null << /JBIG2Globals 8 0 R >>]"),
white.clone(),
vec![globals],
[0x00, 0xFF, 0xFF, 0xFF],
),
// A mask's 0 bits paint: cleared bits are set to 1, which paints nothing.
(format!("{image} /ImageMask true /Filter /JBIG2Decode"), painted, vec![], [0xFF, 0x00, 0x00, 0x00]),
// With /Decode [1 0] a mask's 1 bits paint, and cleared bits are 0.
(format!("{image} /ImageMask true /Decode [1 0] /Filter /JBIG2Decode"), white, vec![], [0x00, 0xFF, 0xFF, 0xFF]),
] {
let mut objs = vec![stream(&dict, data)];
objs.extend(extra);
let mut doc = one_page(b"q 160 0 0 20 0 0 cm /Im Do Q", "/XObject << /Im 7 0 R >>", objs);
mark(&mut doc, 0, &[[0.0, 12.0, 75.0, 18.0]], "");
let report = apply(&mut doc, None).unwrap();
assert_eq!((report.images_cleared, report.images_removed), (1, 0), "{dict}");
let saved = Document::open(Arc::new(write_full(&doc, &SaveOptions::default()).unwrap())).unwrap();
assert_eq!(named_image(&saved, 0, b"PCRedacted1").as_deref(), Some(&cleared[..]), "{dict}");
}
}
/// A JBIG2 image is removed whole when its covered pixels can't be cleared for certain: a JBIG2
/// page taller than the image (the renderer draws the whole page into the image, so the rows cleared
/// wouldn't be the ones shown)
/// or shorter, or of another width; globals that can't be read; data cut short; an image of more
/// than one bit or component, which JBIG2 doesn't code, even when its rows take as many bytes; and
/// a page far past the image's raster, which isn't decoded.
#[test]
fn a_jbig2_image_that_cannot_be_cleared_for_certain_is_removed_whole() {
let white = |rows: u32| pdfcraft_filters::encode_jbig2_generic(&vec![0xFF; 2 * rows as usize], 16, rows).unwrap();
// The page information (the first segment's data, after its 11-byte header) claiming 20,000
// rows of 16.
let mut tall = white(2);
tall[15..19].copy_from_slice(&20_000u32.to_be_bytes());
let image = |size: &str, more: &str| {
format!("/Type /XObject /Subtype /Image {size} /BitsPerComponent 1 /ColorSpace /DeviceGray /Filter /JBIG2Decode {more}")
};
let size = "/Width 16 /Height 2";
for (dict, data, extra) in [
(image(size, ""), white(3), vec![]),
(image("/Width 16 /Height 3", ""), white(2), vec![]),
(image("/Width 15 /Height 2", ""), white(2), vec![]),
(image(size, "/DecodeParms << /JBIG2Globals 8 0 R >>"), white(2), vec![stream("", b"not JBIG2")]),
(image(size, ""), white(2)[..20].to_vec(), vec![]),
(image(size, "").replace("/BitsPerComponent 1", "/BitsPerComponent 8"), white(2), vec![]),
(image(size, "").replace("/DeviceGray", "/DeviceRGB"), white(2), vec![]),
// Two pixels of RGB at a bit take a byte a row, as JBIG2's rows do.
(
image("/Width 2 /Height 2", "").replace("/DeviceGray", "/DeviceRGB"),
pdfcraft_filters::encode_jbig2_generic(&[0xFF; 2], 2, 2).unwrap(),
vec![],
),
(image(size, ""), tall, vec![]),
] {
let mut objs = vec![stream(&dict, data)];
objs.extend(extra);
let mut doc = one_page(b"q 160 0 0 20 0 0 cm /Im Do Q", "/XObject << /Im 7 0 R >>", objs);
mark(&mut doc, 0, &[[0.0, 12.0, 75.0, 18.0]], "");
let report = apply(&mut doc, None).unwrap();
assert_eq!((report.images_cleared, report.images_removed), (0, 1), "{dict}");
}
}
/// A scan stored as JBIG2 symbols, with globals, keeps every pixel outside the area once it is
/// redacted and saved, and the pixels under it are cleared; neither its JBIG2 code nor its globals,
/// which no other image uses, stay in the file. The cleared copy is coded afresh as JBIG2, smaller
/// than Flate, and refers to no globals.
#[test]
fn a_redacted_jbig2_scan_keeps_neither_its_code_nor_its_globals() {
const LETTERS: [[&str; 7]; 3] = [
["#####", "#....", "####.", "#....", "#....", "#....", "#####"],
[".###.", "#...#", "#....", "#....", "#....", "#...#", ".###."],
["#...#", "#...#", "#####", "#...#", "#...#", "#...#", "#...#"],
];
let (w, h) = (400usize, 40usize);
let stride = w.div_ceil(8);
let mut bits = vec![0xFFu8; stride * h];
for (line, top) in [2usize, 12, 22, 32].into_iter().enumerate() {
for (k, x) in (3..w - 6).step_by(7).enumerate() {
for (dy, row) in LETTERS[(k + line) % 3].iter().enumerate() {
for (dx, _) in row.bytes().enumerate().filter(|(_, c)| *c == b'#') {
bits[(top + dy) * stride + (x + dx) / 8] &= !(0x80 >> ((x + dx) % 8));
}
}
}
}
let coded = pdfcraft_filters::encode_jbig2(&bits, w as u32, h as u32).unwrap();
let original = coded.clone();
assert!(coded.len() < pdfcraft_filters::encode_jbig2_generic(&bits, w as u32, h as u32).unwrap().len(), "symbol coded");
let dict = format!(
"/Type /XObject /Subtype /Image /Width {w} /Height {h} /BitsPerComponent 1 /ColorSpace /DeviceGray /Filter /JBIG2Decode /DecodeParms << /JBIG2Globals 8 0 R >>"
);
// Half a point a pixel, its top row at y 120.
let mut doc = one_page(
b"q 200 0 0 20 0 100 cm /Im Do Q",
"/XObject << /Im 7 0 R >>",
vec![stream(&dict, coded), stream("", jbig2_globals(b"LEAK_GLOBALS"))],
);
// Columns 0 to 99 of rows 0 to 12.
mark(&mut doc, 0, &[[0.25, 113.75, 49.75, 120.0]], "");
let report = apply(&mut doc, None).unwrap();
assert_eq!((report.images_cleared, report.images_removed), (1, 0));
let mut cleared = bits.clone();
for y in 0..=12 {
for x in 0..=99 {
cleared[y * stride + x / 8] &= !(0x80 >> (x % 8));
}
}
let saved = Document::open(Arc::new(write_full(&doc, &SaveOptions::default()).unwrap())).unwrap();
assert_eq!(named_image(&saved, 0, b"PCRedacted1"), Some(cleared));
assert_no_secret(&doc, b"LEAK_GLOBALS");
let copy = named_stream(&saved, 0, b"PCRedacted1").unwrap();
assert_eq!(copy.dict.name(b"Filter"), Some(&b"JBIG2Decode"[..]));
assert!(!copy.dict.contains(b"DecodeParms"), "the copy refers to no globals");
let flate = pdfcraft_filters::encode_flate(&named_image(&saved, 0, b"PCRedacted1").unwrap());
assert!(copy.raw.len() < flate.len(), "JBIG2 {} bytes, Flate {}", copy.raw.len(), flate.len());
for (r, object) in saved.scan_objects() {
if let Object::Stream(s) = &*object {
assert_ne!(&s.raw[..], &original[..], "the original JBIG2 code stays as {r:?}");
}
}
}
/// `w` × `h` rows, `w` a multiple of 8: lines of word-sized dark blocks 14 rows tall, their edges
/// ragged by a sample from row to row, as a scan's are; Group 4 codes such rows from the row above.
fn ragged_scan(w: usize, h: usize) -> Vec<u8> {
let stride = w / 8;
let mut seed = 0x9E37_79B9_7F4A_7C15u64;
let mut next = move || {
seed ^= seed << 13;
seed ^= seed >> 7;
seed ^= seed << 17;
seed
};
let mut bits = vec![0xFFu8; stride * h];
for top in (3..h - 14).step_by(20) {
let mut x = 4;
while x + 70 < w {
let len = 20 + (next() % 40) as usize;
for y in top..top + 14 {
let (left, right) = (x + (next() % 2) as usize, x + len - (next() % 2) as usize);
for xx in left..right {
bits[y * stride + xx / 8] &= !(0x80 >> (xx % 8));
}
}
x += len + 6 + (next() % 6) as usize;
}
}
bits
}
/// A cleared one-bit image of one component, a scan as a rule, is stored as CCITT Group 4 when
/// that is smaller than Flate, whatever it was stored as (Flate here), and never as JBIG2 unless it
/// was JBIG2; an eight-bit image stays Flate. The copy decodes to the cleared samples.
#[test]
fn a_cleared_black_and_white_scan_is_stored_as_group_4() {
let (w, h) = (800usize, 200usize);
let stride = w / 8;
let bits = ragged_scan(w, h);
let g4 = pdfcraft_filters::encode_g4(&bits, w as u32, h as u32).unwrap();
assert!(g4.len() < pdfcraft_filters::encode_flate(&bits).len(), "Group 4 is smaller than Flate here");
let grey: Vec<u8> = (0..w * h).map(|i| (i % 251) as u8).collect();
for (dict, data, filter) in [
(
format!("/Type /XObject /Subtype /Image /Width {w} /Height {h} /BitsPerComponent 1 /ColorSpace /DeviceGray /Filter /FlateDecode"),
pdfcraft_filters::encode_flate(&bits),
&b"CCITTFaxDecode"[..],
),
(
format!("/Type /XObject /Subtype /Image /Width {w} /Height {h} /BitsPerComponent 8 /ColorSpace /DeviceGray /Filter /FlateDecode"),
pdfcraft_filters::encode_flate(&grey),
&b"FlateDecode"[..],
),
] {
let mut doc = one_page(b"q 200 0 0 50 0 100 cm /Im Do Q", "/XObject << /Im 7 0 R >>", vec![stream(&dict, data)]);
// The left quarter of the top half.
mark(&mut doc, 0, &[[0.0, 125.0, 50.0, 150.0]], "");
assert_eq!(apply(&mut doc, None).unwrap().images_cleared, 1, "{dict}");
let saved = Document::open(Arc::new(write_full(&doc, &SaveOptions::default()).unwrap())).unwrap();
let copy = named_stream(&saved, 0, b"PCRedacted1").unwrap();
assert_eq!(copy.dict.name(b"Filter"), Some(filter), "{dict}");
// Covered: rows 0 to 99 (y 150 down to 125, a quarter point a row), samples 0 to 199.
let mut expected = if filter == b"FlateDecode" { grey.clone() } else { bits.clone() };
for y in 0..100 {
for x in 0..200 {
if filter == b"FlateDecode" {
expected[y * w + x] = 0;
} else {
expected[y * stride + x / 8] &= !(0x80 >> (x % 8));
}
}
}
assert_eq!(named_image(&saved, 0, b"PCRedacted1").unwrap(), expected, "{dict}");
}
}
/// A cleared one-bit image is stored as the smallest stream, its dictionary counted: a tiny one,
/// whose Group 4 code is shorter than Flate's but whose `/DecodeParms` aren't, stays Flate. Rows
/// wider than Group 4 codes (`MAX_G4_COLUMNS`) stay Flate too, and still clear.
#[test]
fn a_cleared_image_group_4_wouldnt_shrink_stays_flate() {
let wide = pdfcraft_filters::MAX_G4_COLUMNS as usize + 1;
for (w, h) in [(3usize, 2usize), (wide, 2)] {
let stride = w.div_ceil(8);
assert_eq!(pdfcraft_filters::encode_g4(&vec![0xFF; stride * h], w as u32, h as u32).is_ok(), w != wide, "{w} × {h}");
let dict = format!("/Type /XObject /Subtype /Image /Width {w} /Height {h} /BitsPerComponent 1 /ColorSpace /DeviceGray /Filter /FlateDecode");
let mut doc = one_page(
b"q 200 0 0 20 0 100 cm /Im Do Q",
"/XObject << /Im 7 0 R >>",
vec![stream(&dict, pdfcraft_filters::encode_flate(&vec![0xFF; stride * h]))],
);
// The top row's left half.
mark(&mut doc, 0, &[[0.0, 112.0, 99.0, 118.0]], "");
assert_eq!(apply(&mut doc, None).unwrap().images_cleared, 1, "{w} × {h}");
let saved = Document::open(Arc::new(write_full(&doc, &SaveOptions::default()).unwrap())).unwrap();
assert_eq!(named_stream(&saved, 0, b"PCRedacted1").unwrap().dict.name(b"Filter"), Some(&b"FlateDecode"[..]), "{w} × {h}");
let samples = named_image(&saved, 0, b"PCRedacted1").unwrap();
let sample = |x: usize| samples.get(x / 8).is_some_and(|b| b & (0x80 >> (x % 8)) != 0);
assert!(!sample(0) && sample(w - 1), "{w} × {h}: the top row's first sample is cleared and its last kept");
assert!(samples.get(stride..).is_some_and(|row| row.iter().all(|b| *b == 0xFF)), "{w} × {h}: the bottom row is kept");
}
}
/// Only one bit of one component goes to Group 4 or JBIG2. An 8-bit grey image here starts as white
/// as a bilevel scan and goes on as noise Flate can't shrink, so Group 4, which would read only the
/// first rows' worth of bytes, would look far smaller while losing the picture.
#[test]
fn a_cleared_image_of_more_than_one_bit_stays_flate() {
let (w, h) = (256usize, 256usize);
let mut data = vec![0xFF; w * h];
let mut state = 0x1234_5678u32;
for b in data.iter_mut().skip(w * h / 8) {
state ^= state << 13;
state ^= state >> 17;
state ^= state << 5;
*b = state.to_le_bytes()[0];
}
let dict = format!("/Type /XObject /Subtype /Image /Width {w} /Height {h} /BitsPerComponent 8 /ColorSpace /DeviceGray /Filter /FlateDecode");
let mut doc = one_page(b"q 200 0 0 20 0 100 cm /Im Do Q", "/XObject << /Im 7 0 R >>", vec![stream(&dict, pdfcraft_filters::encode_flate(&data))]);
mark(&mut doc, 0, &[[0.0, 112.0, 99.0, 118.0]], "");
assert_eq!(apply(&mut doc, None).unwrap().images_cleared, 1);
let saved = Document::open(Arc::new(write_full(&doc, &SaveOptions::default()).unwrap())).unwrap();
assert_eq!(named_stream(&saved, 0, b"PCRedacted1").unwrap().dict.name(b"Filter"), Some(&b"FlateDecode"[..]));
assert_eq!(named_image(&saved, 0, b"PCRedacted1").unwrap().get(h * w / 2..), data.get(h * w / 2..), "the rows below the mark are kept");
}
/// A copy stored as Group 4 or JBIG2 is read like any image: a second redaction, across a save,
/// clears it again rather than removing it whole, keeps what the first cleared, and stores it
/// smaller than Flate again.
#[test]
fn a_cleared_copy_stored_as_group_4_or_jbig2_can_be_redacted_again() {
let (w, h) = (800usize, 200usize);
let stride = w / 8;
let bits = ragged_scan(w, h);
let image = format!("/Type /XObject /Subtype /Image /Width {w} /Height {h} /BitsPerComponent 1 /ColorSpace /DeviceGray");
for (filter, data, wanted) in [
("/FlateDecode", pdfcraft_filters::encode_flate(&bits), &b"CCITTFaxDecode"[..]),
("/JBIG2Decode", pdfcraft_filters::encode_jbig2(&bits, w as u32, h as u32).unwrap(), &b"JBIG2Decode"[..]),
] {
let dict = format!("{image} /Filter {filter}");
let mut doc = one_page(b"q 200 0 0 50 0 100 cm /Im Do Q", "/XObject << /Im 7 0 R >>", vec![stream(&dict, data)]);
mark(&mut doc, 0, &[[0.0, 125.0, 50.0, 150.0]], "");
assert_eq!(apply(&mut doc, None).unwrap().images_cleared, 1, "{filter}");
let mut saved = Document::open(Arc::new(write_full(&doc, &SaveOptions::default()).unwrap())).unwrap();
assert_eq!(named_stream(&saved, 0, b"PCRedacted1").unwrap().dict.name(b"Filter"), Some(wanted), "{filter}");
mark(&mut saved, 0, &[[150.0, 100.0, 200.0, 125.0]], "");
let report = apply(&mut saved, None).unwrap();
assert_eq!((report.images_cleared, report.images_removed), (1, 0), "{filter}: cleared again, not removed");
let again = Document::open(Arc::new(write_full(&saved, &SaveOptions::default()).unwrap())).unwrap();
// Rows 0 to 99 of the left quarter, then 100 to 199 of the right.
let mut expected = bits.clone();
for (rows, columns) in [(0..100, 0..200), (100..200, 600..800)] {
for y in rows {
for x in columns.clone() {
expected[y * stride + x / 8] &= !(0x80 >> (x % 8));
}
}
}
let names: Vec<Vec<u8>> = (1..4).map(|n| format!("PCRedacted{n}").into_bytes()).collect();
let stored = names.iter().find_map(|n| named_image(&again, 0, n));
assert_eq!(stored, Some(expected), "{filter}");
}
}
/// The segments of a JBIG2 stream `encode_jbig2` wrote, each with its header: a header is the
/// segment's number (4 bytes), type, flags whose top three bits count the segments it refers to,
/// those (a byte each), its page (a byte) and its data's length (4 bytes; ISO/IEC 14492 7.2).
fn jbig2_segments(mut coded: &[u8]) -> Vec<Vec<u8>> {
let mut segments = Vec::new();
while !coded.is_empty() {
let refers = usize::from(coded[5] >> 5);
let header = 4 + 1 + 1 + refers + 1 + 4;
let length = u32::from_be_bytes(coded[header - 4..header].try_into().unwrap()) as usize;
segments.push(coded[..header + length].to_vec());
coded = &coded[header + length..];
}
segments
}
/// Two pages of a scan share one `/JBIG2Globals` stream holding the symbol dictionary, as
/// `jbig2enc -s` and most scanners write them, and the first has a mark found nowhere else (a
/// signature, say) under the area. The dictionary holds that mark's bitmap, so once the first page
/// is redacted the second page's image is stored again without the globals, and neither the
/// dictionary nor any stream naming globals stays in the saved file; both pages keep every pixel
/// outside the area.
#[test]
fn globals_a_redacted_jbig2_image_shares_with_another_page_leave_the_file() {
const LETTER: [&str; 7] = ["#####", "#....", "####.", "#....", "#....", "#....", "#####"];
const SIGNATURE: [&str; 7] = ["#..#.##", ".##.#..", "#.#..##", "..##.#.", "##..#.#", ".#.##..", "#..#..#"];
let (w, h) = (400usize, 40usize);
let stride = w.div_ceil(8);
let draw = |bits: &mut [u8], shape: &[&str], top: usize, left: usize| {
for (dy, row) in shape.iter().enumerate() {
for (dx, _) in row.bytes().enumerate().filter(|(_, c)| *c == b'#') {
bits[(top + dy) * stride + (left + dx) / 8] &= !(0x80 >> ((left + dx) % 8));
}
}
};
// The letter over and over lower down, and the signature once, top left, under the mark.
let mut first = vec![0xFFu8; stride * h];
for top in [14usize, 24, 32] {
for x in (3..w - 6).step_by(7) {
draw(&mut first, &LETTER, top, x);
}
}
draw(&mut first, &SIGNATURE, 2, 10);
// The second page is a scan's ragged text, which Group 4 or JBIG2 stores smaller than Flate.
let second = ragged_scan(w, h);
// The first page's symbol dictionary (its second segment) goes to the globals, on no page; the
// second page's segments are numbered past it.
let mut segments = jbig2_segments(&pdfcraft_filters::encode_jbig2(&first, w as u32, h as u32).unwrap());
assert_eq!(segments[1][4], 0, "the second segment is the symbol dictionary");
let mut dictionary = segments.remove(1);
dictionary[6] = 0;
let page_one: Vec<u8> = segments.concat();
let page_two: Vec<u8> = jbig2_segments(&pdfcraft_filters::encode_jbig2_generic(&second, w as u32, h as u32).unwrap())
.into_iter()
.enumerate()
.flat_map(|(k, mut s)| {
s[..4].copy_from_slice(&(100 + k as u32).to_be_bytes());
s
})
.collect();
let image = format!(
"/Type /XObject /Subtype /Image /Width {w} /Height {h} /BitsPerComponent 1 /ColorSpace /DeviceGray /Filter /JBIG2Decode /DecodeParms << /JBIG2Globals 9 0 R >>"
);
// Half a point a pixel, the image's top row at y 120.
let page = |content: u32, im: u32| {
format!("<< /Type /Page /Parent 2 0 R /MediaBox [0 0 300 300] /Contents {content} 0 R /Resources << /XObject << /Im {im} 0 R >> >> >>")
.into_bytes()
};
let build = |second_image: &str| {
pdf(vec![
b"<< /Type /Catalog /Pages 2 0 R >>".to_vec(),
b"<< /Type /Pages /Kids [3 0 R 4 0 R] /Count 2 >>".to_vec(),
page(5, 7),
page(6, 8),
stream("", b"q 200 0 0 20 0 100 cm /Im Do Q".as_slice()),
stream("", b"q 200 0 0 20 0 100 cm /Im Do Q".as_slice()),
stream(&image, page_one.clone()),
stream(second_image, page_two.clone()),
stream("", dictionary.clone()),
])
};
let mut doc = build(&image);
// A page's picture as it is drawn, JBIG2 read with its globals.
let picture = |doc: &Document, page: usize, name: &[u8]| {
let p = pdfcraft_model::pages(doc).swap_remove(page);
let res = doc.resolve(p.dict.get(b"Resources")?).as_dict().cloned()?;
let xo = doc.resolve(res.get(b"XObject")?).as_dict().cloned()?;
match &*doc.get(xo.get(name)?.as_ref()?) {
Object::Stream(s) => decoded_image(doc, s).ok(),
_ => None,
}
};
assert_eq!(picture(&doc, 0, b"Im"), Some(first.clone()), "the first page reads its symbols from the globals");
assert_eq!(picture(&doc, 1, b"Im"), Some(second.clone()));
// Columns 0 to 99 of rows 0 to 12.
mark(&mut doc, 0, &[[0.25, 113.75, 49.75, 120.0]], "");
let report = apply(&mut doc, None).unwrap();
assert_eq!((report.images_cleared, report.images_removed, report.images_restored), (1, 0, 1));
let mut cleared = first.clone();
for y in 0..=12 {
for x in 0..=99 {
cleared[y * stride + x / 8] &= !(0x80 >> (x % 8));
}
}
let saved = reopen(&doc);
assert_eq!(picture(&saved, 0, b"PCRedacted1"), Some(cleared));
assert_eq!(picture(&saved, 1, b"Im"), Some(second.clone()), "the second page keeps its picture");
// Stored again as a scan is: Group 4 or JBIG2 when smaller than Flate.
let restored = named_stream(&saved, 1, b"Im").unwrap();
let flate = pdfcraft_filters::encode_flate(&second);
assert_ne!(restored.dict.name(b"Filter"), Some(&b"FlateDecode"[..]));
assert!(restored.raw.len() < flate.len(), "{:?} {} bytes, Flate {}", restored.dict.name(b"Filter"), restored.raw.len(), flate.len());
// The dictionary's data (after its 11-byte header) codes the signature's bitmap.
let coded = &dictionary[11..];
for (r, object) in saved.scan_objects() {
let mut bytes = Vec::new();
pdfcraft_cos::serialize(&object, &mut bytes);
assert!(!bytes.windows(b"JBIG2Globals".len()).any(|w| w == b"JBIG2Globals"), "{r:?} still names globals");
let data = match &*object {
Object::Stream(s) => s.decoded().unwrap_or_default(),
_ => Vec::new(),
};
assert!(!bytes.windows(coded.len()).chain(data.windows(coded.len())).any(|w| w == coded), "the symbol dictionary stays as {r:?}");
}
// The second page's image can't be read for certain (its JBIG2 page is shorter than the
// image), so it can't be stored without the globals: the redaction is refused.
let mut doc = build(&image.replace(&format!("/Height {h}"), &format!("/Height {}", h + 1)));
mark(&mut doc, 0, &[[0.25, 113.75, 49.75, 120.0]], "");
assert!(matches!(apply(&mut doc, None), Err(RedactError::SharedJbig2Globals(1))));
// Reading the dictionary to know copies of it, and storing the second page's image again,
// decode within what is left of the run's allowance: past it the redaction is refused as one
// past the allowance, not reported as an image that can't be read. The first page's image
// leaves (its page no longer lists it).
let mut doc = build(&image);
let before = crate::globals::kept(&doc).unwrap();
let first = pdfcraft_model::pages(&doc)[0].obj;
doc.update_dict(first, |d| {
d.remove(b"Resources");
})
.unwrap();
let refused = crate::globals::unshare(&mut doc.clone(), &before, &mut pdfcraft_cos::ImageBudget::with_work(0));
assert!(matches!(refused, Err(RedactError::ImagesPastAllowance)), "{refused:?}");
// Enough to read the dictionary, and nothing left for the image.
let mut reading = pdfcraft_cos::ImageBudget::with_work(u64::MAX);
crate::globals::content(&mut crate::globals::Run::new(crate::globals::RUN_BUDGET), &doc, 9, &mut reading).unwrap();
let read = u64::MAX - reading.work_left();
let refused = crate::globals::unshare(&mut doc.clone(), &before, &mut pdfcraft_cos::ImageBudget::with_work(read));
assert!(matches!(refused, Err(RedactError::ImagesPastAllowance)), "the image: {refused:?}");
// Removing hidden information chose no pages, so fewer of them is no way out: it is refused as
// finding what shares the dictionary is.
let refused = crate::sanitize::unshare_unmarked(&mut doc.clone(), &before, &mut pdfcraft_cos::ImageBudget::with_work(0));
assert!(matches!(refused, Err(RedactError::SharedJbig2Budget)), "{refused:?}");
let restored = crate::globals::unshare(&mut doc, &before, &mut pdfcraft_cos::ImageBudget::with_work(u64::MAX));
assert_eq!(restored.ok(), Some(1));
}
/// A scan's parts when it keeps its symbols in `/JBIG2Globals`: the dictionary, moved out as segment
/// data belonging to no page, and the page information and regions that are left.
struct SharedScan {
bits: Vec<u8>,
symbols: Vec<u8>,
page: Vec<u8>,
}
fn shared_scan() -> SharedScan {
const LETTERS: [[&str; 7]; 3] = [
["#####", "#....", "####.", "#....", "#....", "#....", "#####"],
[".###.", "#...#", "#....", "#....", "#....", "#...#", ".###."],
["#...#", "#...#", "#####", "#...#", "#...#", "#...#", "#...#"],
];
let (w, h) = (400usize, 40usize);
let stride = w / 8;
let mut bits = vec![0xFFu8; stride * h];
for (line, top) in [2usize, 12, 22, 32].into_iter().enumerate() {
for (k, x) in (3..w - 6).step_by(7).enumerate() {
for (dy, row) in LETTERS[(k + line) % 3].iter().enumerate() {
for (dx, _) in row.bytes().enumerate().filter(|(_, c)| *c == b'#') {
bits[(top + dy) * stride + (x + dx) / 8] &= !(0x80 >> ((x + dx) % 8));
}
}
}
}
let coded = pdfcraft_filters::encode_jbig2(&bits, w as u32, h as u32).unwrap();
// The end of the segment at `at`, and where its page association is (7.2.2 to 7.2.7).
let end = |at: usize| {
let page = at + 6 + usize::from(coded[at + 5] >> 5);
let length = u32::from_be_bytes(coded[page + 1..page + 5].try_into().unwrap());
(page + 5 + length as usize, page)
};
let (dictionary, _) = end(0);
assert_eq!(coded[dictionary + 4], 0, "the second segment is the symbol dictionary");
let (rest, page) = end(dictionary);
let mut symbols = coded[dictionary..rest].to_vec();
symbols[page - dictionary] = 0;
SharedScan { page: [&coded[..dictionary], &coded[rest..]].concat(), symbols, bits }
}
/// An image of [`SharedScan`]'s size that uses the dictionary, object 9, by `parms`.
fn shared_image(parms: &str) -> String {
format!("/Type /XObject /Subtype /Image /Width 400 /Height 40 /BitsPerComponent 1 /ColorSpace /DeviceGray /Filter /JBIG2Decode {parms}")
}
/// Two pages of a scan sharing object 9: its symbols, and `LEAK_GLOBALS`, for the shape of a mark found
/// only under the area. The first page draws image 7 unless told otherwise; the second draws image 8.
/// `extra` are objects 10 on.
struct SharedPages {
/// What the first page draws, its `/Resources` entries and anything else in its dictionary.
content: Vec<u8>,
resources: String,
page: String,
/// Image 7's dictionary and data; image 8's.
first: String,
first_data: Vec<u8>,
other: String,
other_data: Vec<u8>,
extra: Vec<Vec<u8>>,
}
const DRAWS: &[u8] = b"q 200 0 0 20 0 100 cm /Im Do Q";
impl SharedPages {
fn new(scan: &SharedScan) -> Self {
let image = shared_image(SHARED);
Self {
content: DRAWS.to_vec(),
resources: "/XObject << /Im 7 0 R >>".into(),
page: String::new(),
first: image.clone(),
first_data: scan.page.clone(),
other: image,
other_data: scan.page.clone(),
extra: vec![],
}
}
fn build(self, scan: &SharedScan) -> Document {
let Self { content, resources, page, first, first_data, other, other_data, extra } = self;
let mut objs: Vec<Vec<u8>> = vec![
b"<< /Type /Catalog /Pages 2 0 R >>".to_vec(),
b"<< /Type /Pages /Kids [3 0 R 4 0 R] /Count 2 >>".to_vec(),
format!("<< /Type /Page /Parent 2 0 R /MediaBox [0 0 300 300] /Contents 5 0 R /Resources << {resources} >> {page} >>").into_bytes(),
b"<< /Type /Page /Parent 2 0 R /MediaBox [0 0 300 300] /Contents 6 0 R /Resources << /XObject << /Im 8 0 R >> >> >>".to_vec(),
stream("", content),
stream("", DRAWS),
stream(&first, first_data),
stream(&other, other_data),
stream("", [jbig2_globals(b"LEAK_GLOBALS"), scan.symbols.clone()].concat()),
];
objs.extend(extra);
pdf(objs)
}
}
/// The samples of the image the page draws as `/Im`.
fn drawn_samples(doc: &Document, page: usize) -> Vec<u8> {
let p = pdfcraft_model::pages(doc).swap_remove(page);
let res = doc.resolve(p.dict.get(b"Resources").unwrap()).as_dict().cloned().unwrap();
let xo = doc.resolve(res.get(b"XObject").unwrap()).as_dict().cloned().unwrap();
match &*doc.get(xo.get(b"Im").unwrap().as_ref().unwrap()) {
Object::Stream(s) => decoded_image(doc, s).unwrap(),
other => panic!("not an image: {other:?}"),
}
}
const SHARED: &str = "/DecodeParms << /JBIG2Globals 9 0 R >>";
const CLEARS: [f64; 4] = [0.25, 113.75, 49.75, 120.0];
const COVERS: [f64; 4] = [0.0, 100.0, 200.0, 120.0];
/// The ways image 7 can leave the document while taking the dictionary's only reason to stay with it.
fn ways_out(scan: &SharedScan) -> Vec<(&'static str, [f64; 4], SharedPages)> {
let base = || SharedPages::new(scan);
let in_form = |filter: &str, data: &[u8]| {
stream(&format!("/Type /XObject /Subtype /Form /BBox [0 0 300 300] /Resources << /XObject << /Im 7 0 R >> >> {filter}"), data)
};
let plain = "/Type /XObject /Subtype /Image /Width 1 /Height 1 /BitsPerComponent 8 /ColorSpace /DeviceGray";
let through_form = |filter: &str, data: &[u8]| SharedPages {
content: b"/Fm Do".to_vec(),
resources: "/XObject << /Fm 10 0 R >>".into(),
extra: vec![in_form(filter, data)],
..base()
};
let annotated = |subtype: &str| SharedPages {
content: Vec::new(),
resources: String::new(),
page: "/Annots [10 0 R]".into(),
extra: vec![
format!("<< /Type /Annot /Subtype /{subtype} /T (Field) /FT /Tx /Rect [0 100 200 120] /AP << /N 11 0 R >> >>").into_bytes(),
in_form("", DRAWS),
],
..base()
};
let glyph = SharedPages {
content: b"BT /T3 200 Tf 0 100 Td (a) Tj ET".to_vec(),
resources: "/Font << /T3 10 0 R >>".into(),
extra: vec![
b"<< /Type /Font /Subtype /Type3 /FontBBox [0 0 1000 1000] /FontMatrix [0.001 0 0 0.001 0 0] /CharProcs << /a 11 0 R >> \
/Encoding << /Differences [97 /a] >> /FirstChar 97 /LastChar 97 /Widths [1000] /Resources << /XObject << /Im 7 0 R >> >> >>"
.to_vec(),
stream("", b"1000 0 d0 q 1000 0 0 100 0 0 cm /Im Do Q"),
],
..base()
};
vec![
("cleared", CLEARS, base()),
("removed", COVERS, base()),
("in a form", COVERS, through_form("", DRAWS)),
("in a form that can't be read", COVERS, through_form("/Filter /FlateDecode", b"not Flate")),
("a stamp's appearance", COVERS, annotated("Stamp")),
("a widget's appearance", COVERS, annotated("Widget")),
("a Type 3 glyph", COVERS, glyph),
(
"a mask",
COVERS,
SharedPages { first: format!("{plain} /SMask 10 0 R"), extra: vec![stream(&shared_image(SHARED), scan.page.clone())], ..base() },
),
(
"a stencil mask",
COVERS,
SharedPages {
first: format!("{plain} /Mask 10 0 R"),
extra: vec![stream(
&shared_image(SHARED).replace("/BitsPerComponent 1 /ColorSpace /DeviceGray", "/ImageMask true"),
scan.page.clone(),
)],
..base()
},
),
(
"parameters for data in another file",
COVERS,
SharedPages { first: shared_image("/F (scan.jb2) /FFilter /JBIG2Decode /FDecodeParms << /JBIG2Globals 9 0 R >>"), ..base() },
),
(
"an alternate",
COVERS,
SharedPages {
first: format!("{plain} /Alternates [<< /Image 10 0 R >>]"),
extra: vec![stream(&shared_image(SHARED), scan.page.clone())],
..base()
},
),
(
"/Filter by reference",
COVERS,
SharedPages {
first: shared_image("/DecodeParms [null << /JBIG2Globals 9 0 R >>]").replace("/Filter /JBIG2Decode", "/Filter 10 0 R"),
extra: vec![b"[/ASCIIHexDecode /JBIG2Decode]".to_vec()],
..base()
},
),
(
"a reference to it",
COVERS,
SharedPages { first: shared_image("/DecodeParms << /JBIG2Globals 10 0 R >>"), extra: vec![b"9 0 R".to_vec()], ..base() },
),
(
"parameters through a reference to them",
COVERS,
SharedPages {
first: shared_image("/DecodeParms 10 0 R"),
extra: vec![b"11 0 R".to_vec(), b"<< /JBIG2Globals 9 0 R >>".to_vec()],
..base()
},
),
(
"an entry through a reference to it",
COVERS,
SharedPages {
first: shared_image("/DecodeParms [10 0 R]"),
extra: vec![b"11 0 R".to_vec(), b"<< /JBIG2Globals 9 0 R >>".to_vec()],
..base()
},
),
(
"an array of them through a reference to it",
COVERS,
SharedPages {
first: shared_image("/DecodeParms 10 0 R"),
extra: vec![b"11 0 R".to_vec(), b"[<< /JBIG2Globals 9 0 R >>]".to_vec()],
..base()
},
),
("/DP", COVERS, SharedPages { first: shared_image("/DP << /JBIG2Globals 9 0 R >>"), ..base() }),
// Cleared, the copy keeps no parameters of the old data, so none names the dictionary.
("/DP, cleared", CLEARS, SharedPages { first: format!("{} /DP << /JBIG2Globals 9 0 R >>", shared_image(SHARED)), ..base() }),
(
"parameters for data in another file, with none, cleared",
CLEARS,
SharedPages { first: format!("{} /FFilter /JBIG2Decode /FDecodeParms << /JBIG2Globals 9 0 R >>", shared_image(SHARED)), ..base() },
),
("another generation", COVERS, SharedPages { first: shared_image("/DecodeParms << /JBIG2Globals 9 1 R >>"), ..base() }),
// Cos reads a reference of another generation as null (#1143); a reader that doesn't compare
// generations draws with what it names. Read first, it mustn't hide what the other names.
(
"parameters through another generation, then their own",
COVERS,
SharedPages {
first: shared_image("/DecodeParms [10 1 R 10 0 R]").replace("/Filter /JBIG2Decode", "/Filter [/ASCIIHexDecode /JBIG2Decode]"),
extra: vec![b"<< /JBIG2Globals 9 0 R >>".to_vec()],
..base()
},
),
(
"parameters through another generation",
COVERS,
SharedPages { first: shared_image("/DecodeParms 10 1 R"), extra: vec![b"<< /JBIG2Globals 9 0 R >>".to_vec()], ..base() },
),
(
"a reference to it of another generation",
COVERS,
SharedPages { first: shared_image("/DecodeParms << /JBIG2Globals 10 1 R >>"), extra: vec![b"9 0 R".to_vec()], ..base() },
),
]
}
/// Scanners and `jbig2enc -s` share one symbol dictionary across a scan's pages, and it holds the
/// shape of every distinct mark, one found only under a redaction included: a signature, a stamp.
/// The dictionary stayed in the file while any other image used it. Now the image left on the second
/// page is stored again without it, whichever way the first one left the document.
#[test]
fn a_redacted_scans_shared_jbig2_dictionary_leaves_the_file() {
let scan = shared_scan();
for (way, rect, pages) in ways_out(&scan) {
let mut doc = pages.build(&scan);
assert_eq!(drawn_samples(&doc, 1), scan.bits, "{way}: the second page reads through the dictionary");
mark(&mut doc, 0, &[rect], "");
let report = apply(&mut doc, None).unwrap();
assert_eq!(report.images_restored, 1, "{way}");
assert_gone(&doc, b"LEAK_GLOBALS");
assert_gone(&doc, b"JBIG2Globals");
assert_eq!(drawn_samples(&reopen(&doc), 1), scan.bits, "{way}: its pixels are unchanged");
}
}
/// A size given through a reference of another generation is null, as cos reads it (#1143), whatever
/// another image read through the reference of that number's own: the image can't be stored again, so
/// the redaction refuses, rather than storing it with the size the other reference read.
#[test]
fn a_size_through_another_generation_isnt_the_one_its_own_reference_reads() {
let scan = shared_scan();
let stale = shared_image(SHARED).replace("/Width 400", "/Width 11 1 R");
let mut doc = SharedPages {
content: [DRAWS, b" q 200 0 0 20 0 200 cm /Stale Do Q".as_slice()].concat(),
resources: "/XObject << /Im 7 0 R /Stale 12 0 R >>".into(),
other: shared_image(SHARED).replace("/Width 400", "/Width 11 0 R"),
extra: vec![b"null".to_vec(), b"400".to_vec(), stream(&stale, scan.page.clone())],
..SharedPages::new(&scan)
}
.build(&scan);
mark(&mut doc, 0, &[COVERS], "");
let result = apply(&mut doc, None);
assert!(matches!(result, Err(RedactError::SharedJbig2Globals(1))), "{result:?}");
}
/// An image whose cross-reference row and object header give different generations is read through
/// either (#1143), and page 2 names it by the header's. Stored again without the dictionary, it keeps
/// both, so that page still draws it, in the document and in a full save.
#[test]
fn an_image_stored_again_keeps_the_generation_its_header_gives() {
let scan = shared_scan();
let bytes = SharedPages::new(&scan).build(&scan).bytes().to_vec();
let patch = |bytes: Vec<u8>, from: &[u8], to: &[u8]| {
let at = bytes.windows(from.len()).position(|w| w == from).unwrap();
assert_eq!(bytes.windows(from.len()).filter(|w| *w == from).count(), 1);
[&bytes[..at], to, &bytes[at + from.len()..]].concat()
};
let bytes = patch(patch(bytes, b"\n8 0 obj", b"\n8 1 obj"), b"/Im 8 0 R", b"/Im 8 1 R");
let mut doc = Document::open(Arc::new(bytes)).unwrap();
assert_eq!(drawn_samples(&doc, 1), scan.bits, "the second page reads its image through the header's generation");
mark(&mut doc, 0, &[COVERS], "");
assert_eq!(apply(&mut doc, None).unwrap().images_restored, 1);
assert_eq!(drawn_samples(&doc, 1), scan.bits, "the image stored again is still drawn");
let full = write_full(&doc, &SaveOptions::default()).unwrap();
assert!(!full.windows(12).any(|w| w == b"LEAK_GLOBALS"), "the dictionary left the full save");
assert_eq!(drawn_samples(&Document::open(Arc::new(full)).unwrap(), 1), scan.bits, "and is drawn after it");
assert_gone(&doc, b"LEAK_GLOBALS");
}
/// However the image that stays names the dictionary, it comes out without it: its parameters by
/// reference, behind another filter, or a copy of the dictionary, as a file put back together from
/// pages split apart has, each page with its own. It is also read when its size or filter is given
/// by reference, which `decoded_image` alone refuses.
#[test]
fn what_stays_is_stored_without_the_dictionary_however_it_names_it() {
let scan = shared_scan();
let hex: Vec<u8> = scan.page.iter().map(|b| format!("{b:02x}")).collect::<String>().bytes().chain(*b">").collect();
let copy = stream("", [jbig2_globals(b"LEAK_GLOBALS"), scan.symbols.clone()].concat());
let image = shared_image(SHARED);
for (other, data, extra) in [
(shared_image("/DecodeParms 10 0 R"), scan.page.clone(), vec![b"<< /JBIG2Globals 9 0 R >>".to_vec()]),
(
shared_image("/DecodeParms [null << /JBIG2Globals 9 0 R >>]").replace("/Filter /JBIG2Decode", "/Filter [/ASCIIHexDecode /JBIG2Decode]"),
hex.clone(),
vec![],
),
(shared_image("/DecodeParms << /JBIG2Globals 10 0 R >>"), scan.page.clone(), vec![copy]),
(image.replace("/Width 400", "/Width 10 0 R"), scan.page.clone(), vec![b"400".to_vec()]),
(image.replace("/Height 40", "/Height 10 0 R"), scan.page.clone(), vec![b"40".to_vec()]),
(image.replace("/Filter /JBIG2Decode", "/Filter 10 0 R"), scan.page.clone(), vec![b"/JBIG2Decode".to_vec()]),
(
shared_image("/DecodeParms [null << /JBIG2Globals 9 0 R >>]").replace("/Filter /JBIG2Decode", "/Filter [/ASCIIHexDecode 10 0 R]"),
hex.clone(),
vec![b"/JBIG2Decode".to_vec()],
),
(image.replace("/ColorSpace /DeviceGray ", ""), scan.page.clone(), vec![]),
(
image.replace("/ColorSpace /DeviceGray", "/ColorSpace [/DeviceN [/Black] /DeviceGray 10 0 R]"),
scan.page.clone(),
vec![b"<< /FunctionType 2 /Domain [0 1] /N 1 >>".to_vec()],
),
// Parameters no filter reads, on a picture that needs no dictionary.
(shared_image("/DP << /JBIG2Globals 9 0 R >>"), pdfcraft_filters::encode_jbig2_generic(&scan.bits, 400, 40).unwrap(), vec![]),
// Those for data in another file, with none: beside the parameters it reads, and alone.
(format!("{image} /FFilter /JBIG2Decode /FDecodeParms << /JBIG2Globals 9 0 R >>"), scan.page.clone(), vec![]),
(
shared_image("/FFilter /JBIG2Decode /FDecodeParms << /JBIG2Globals 9 0 R >>"),
pdfcraft_filters::encode_jbig2_generic(&scan.bits, 400, 40).unwrap(),
vec![],
),
// A `/F` that is null, directly, by reference, or as a reference to no object, is none.
(format!("{image} /F null"), scan.page.clone(), vec![]),
(format!("{image} /F 10 0 R"), scan.page.clone(), vec![b"null".to_vec()]),
(format!("{image} /F 99 0 R"), scan.page.clone(), vec![]),
// Through a reference to a reference: /F to null, the size, the filter, the colour space.
(format!("{image} /F 10 0 R"), scan.page.clone(), vec![b"11 0 R".to_vec(), b"null".to_vec()]),
(shared_image("/DecodeParms 10 0 R"), scan.page.clone(), vec![b"11 0 R".to_vec(), b"<< /JBIG2Globals 9 0 R >>".to_vec()]),
(image.replace("/Width 400", "/Width 10 0 R"), scan.page.clone(), vec![b"11 0 R".to_vec(), b"400".to_vec()]),
(image.replace("/Height 40", "/Height 10 0 R"), scan.page.clone(), vec![b"11 0 R".to_vec(), b"40".to_vec()]),
(image.replace("/Filter /JBIG2Decode", "/Filter 10 0 R"), scan.page.clone(), vec![b"11 0 R".to_vec(), b"/JBIG2Decode".to_vec()]),
(
shared_image("/DecodeParms [null << /JBIG2Globals 9 0 R >>]").replace("/Filter /JBIG2Decode", "/Filter [/ASCIIHexDecode 10 0 R]"),
hex.clone(),
vec![b"11 0 R".to_vec(), b"/JBIG2Decode".to_vec()],
),
(image.replace("/ColorSpace /DeviceGray", "/ColorSpace 10 0 R"), scan.page.clone(), vec![b"11 0 R".to_vec(), b"/DeviceGray".to_vec()]),
// An ICC profile of one component is one.
(format!("{} /ColorSpace [/ICCBased 10 0 R]", image.replace("/ColorSpace /DeviceGray ", "")), scan.page.clone(), vec![stream("/N 1", b"")]),
] {
let mut doc = SharedPages { other: other.clone(), other_data: data, extra, ..SharedPages::new(&scan) }.build(&scan);
mark(&mut doc, 0, &[CLEARS], "");
let report = apply(&mut doc, None).unwrap();
assert_eq!(report.images_restored, 1, "{other}");
assert_gone(&doc, b"LEAK_GLOBALS");
assert_eq!(drawn_samples(&reopen(&doc), 1), scan.bits, "{other}: its pixels are unchanged");
}
}
/// JBIG2 is drawn as one bit of one component whatever the dictionary says, so the Flate copy has to
/// say so too: a stream with no `/BitsPerComponent`, or 8, shows the same picture stored again.
#[test]
fn a_stored_jbig2_image_is_one_bit_whatever_its_dictionary_said() {
let scan = shared_scan();
let hex: Vec<u8> = scan.page.iter().map(|b| format!("{b:02x}")).collect::<String>().bytes().chain(*b">").collect();
let chained =
shared_image("/DecodeParms [null << /JBIG2Globals 9 0 R >>]").replace("/Filter /JBIG2Decode", "/Filter [/ASCIIHexDecode /JBIG2Decode]");
let lone = shared_image(SHARED).replace("/Filter /JBIG2Decode", "/Filter [/JBIG2Decode]");
for (other, other_data, bits) in [
(shared_image(SHARED), scan.page.clone(), ""),
(shared_image(SHARED), scan.page.clone(), "/BitsPerComponent 8"),
(chained, hex, ""),
(lone, scan.page.clone(), ""),
] {
let other = other.replace("/BitsPerComponent 1", bits);
let mut doc = SharedPages { other, other_data, ..SharedPages::new(&scan) }.build(&scan);
mark(&mut doc, 0, &[CLEARS], "");
apply(&mut doc, None).unwrap();
let saved = reopen(&doc);
assert_eq!(drawn_samples(&saved, 1), scan.bits, "{bits}");
let p = pdfcraft_model::pages(&saved).swap_remove(1);
let res = saved.resolve(p.dict.get(b"Resources").unwrap()).as_dict().cloned().unwrap();
let xo = saved.resolve(res.get(b"XObject").unwrap()).as_dict().cloned().unwrap();
let image = saved.get(xo.get(b"Im").unwrap().as_ref().unwrap());
let Object::Stream(s) = &*image else { panic!("not an image") };
assert_eq!(s.dict.int(b"BitsPerComponent"), Some(1), "{bits}");
}
}
/// A mark that takes nothing using the dictionary leaves it, and what uses it, alone: here it covers
/// text in a form that also draws the scan, away from the scan. A stream nothing reaches isn't decoded
/// for nothing either: this one can't be, and it uses the dictionary.
#[test]
fn a_dictionary_stays_when_nothing_using_it_left() {
let scan = shared_scan();
let text = b"BT /F1 10 Tf 10 250 Td (SECRET) Tj ET q 200 0 0 20 0 100 cm /Im Do Q";
let pages = SharedPages {
content: b"/Fm Do".to_vec(),
resources: "/XObject << /Fm 10 0 R >>".into(),
extra: vec![
stream(
"/Type /XObject /Subtype /Form /BBox [0 0 300 300] /Resources << /Font << /F1 << /Type /Font /Subtype /Type1 /BaseFont /Helvetica >> >> /XObject << /Im 7 0 R >> >>",
text,
),
stream(&shared_image(SHARED), b"not JBIG2"),
],
..SharedPages::new(&scan)
};
let mut doc = pages.build(&scan);
mark(&mut doc, 0, &[[0.0, 240.0, 200.0, 270.0]], "");
let report = apply(&mut doc, None).unwrap();
assert_eq!((report.images_removed, report.images_cleared, report.images_restored), (0, 0, 0));
assert_eq!(drawn_samples(&reopen(&doc), 1), scan.bits);
}
/// A dictionary of other bytes is another dictionary: the scan using it keeps it and isn't stored again.
#[test]
fn another_jbig2_dictionary_is_left_alone() {
let scan = shared_scan();
let own = stream("", [jbig2_globals(b"OWN_GLOBALS"), scan.symbols.clone()].concat());
let other = shared_image("/DecodeParms << /JBIG2Globals 10 0 R >>");
let mut doc = SharedPages { other, extra: vec![own], ..SharedPages::new(&scan) }.build(&scan);
mark(&mut doc, 0, &[CLEARS], "");
assert_eq!(apply(&mut doc, None).unwrap().images_restored, 0);
let saved = reopen(&doc);
let holds = |secret: &[u8]| {
saved.scan_objects().any(|(_, o)| matches!(&*o, Object::Stream(s) if s.decoded().is_ok_and(|d| d.windows(secret.len()).any(|w| w == secret))))
};
assert!(!holds(b"LEAK_GLOBALS") && holds(b"OWN_GLOBALS"));
assert_eq!(drawn_samples(&saved, 1), scan.bits);
}
/// An image that can't be stored without the dictionary may still be drawn: the renderer draws a
/// JBIG2 page of another size than its image, which `decoded_image` refuses, and draws three components
/// of one bit as grey. Removing it would lose a page no mark reached, and keeping the dictionary would
/// keep what the mark covered, so the redaction refuses and the document stays as it was. So it does
/// for anything else that refers to the dictionary, a reference that holds a reference to it among them.
#[test]
fn a_shared_jbig2_dictionary_that_cant_leave_refuses_the_redaction() {
let scan = shared_scan();
let image = shared_image(SHARED);
let elsewhere = "/Type /XObject /Subtype /Image /Width 1 /Height 1 /BitsPerComponent 1 /ColorSpace /DeviceGray /Note 9 0 R";
let not_jbig2 = image.replace("/Filter /JBIG2Decode", "/Filter /FlateDecode");
for (other, data, extra) in [
(image.replace("/Height 40", "/Height 41"), scan.page.clone(), vec![]),
(image.replace("/DeviceGray", "/DeviceRGB"), scan.page.clone(), vec![]),
(image.replace("/DeviceGray", "/DeviceCMYK"), scan.page.clone(), vec![]),
(image.replace("/ColorSpace /DeviceGray", "/ColorSpace 10 0 R"), scan.page.clone(), vec![b"11 0 R".to_vec(), b"/DeviceRGB".to_vec()]),
(
format!("{image} /ColorSpace [/ICCBased 10 0 R]").replace("/ColorSpace /DeviceGray ", ""),
scan.page.clone(),
vec![b"11 0 R".to_vec(), stream("/N 3", b"")],
),
// Through a reference to a reference, which now reads as what it names (#5548): stored again
// without the dictionary, or refused (see below).
(shared_image("/DecodeParms << /JBIG2Globals 10 0 R >>"), scan.page.clone(), vec![b"9 0 R".to_vec()]),
(elsewhere.to_string(), vec![0], vec![]),
(not_jbig2, pdfcraft_filters::encode_flate(&scan.bits), vec![]),
// A copy with its filter by reference is known for a copy, and the decoder doesn't read it.
(
shared_image("/DecodeParms << /JBIG2Globals 10 0 R >>"),
scan.page.clone(),
vec![
stream("/Filter 11 0 R", pdfcraft_filters::encode_flate(&[jbig2_globals(b"LEAK_GLOBALS"), scan.symbols.clone()].concat())),
b"/FlateDecode".to_vec(),
],
),
] {
let through_reference = other.contains("/JBIG2Globals 10 0 R");
let mut doc = SharedPages { other: other.clone(), other_data: data, extra, ..SharedPages::new(&scan) }.build(&scan);
mark(&mut doc, 0, &[CLEARS], "");
let before = doc.clone();
match apply(&mut doc, None) {
Err(RedactError::SharedJbig2Globals(_)) => assert_eq!(drawn_samples(&doc, 0), drawn_samples(&before, 0), "{other}: nothing was redacted"),
Ok(_) if through_reference => {
// Object 10 too, the copy or the reference to the dictionary: `assert_gone` can't
// decode the copy, its `/Filter` given by reference, nor see its bytes, which are Flate.
let kept = pdfcraft_cos::reachable_objects(&doc).unwrap();
assert!(!kept.contains(&9) && !kept.contains(&10), "{other}: the dictionary or its copy stays");
assert_gone(&doc, b"LEAK_GLOBALS");
}
result => panic!("{other}: {result:?}"),
}
}
}
/// What is stored again is bounded as what is cleared is: an image of [`image::MAX_PIXELS`] is, and one
/// past it, which decodes, refuses the redaction.
#[test]
fn an_image_past_the_size_redaction_clears_refuses_it() {
let scan = shared_scan();
let h = 8192u32;
for (w, stored) in [(8192u32, true), (8200, false)] {
let blank = vec![0xFFu8; (w as usize).div_ceil(8) * h as usize];
let other = shared_image(SHARED).replace("/Width 400", &format!("/Width {w}")).replace("/Height 40", &format!("/Height {h}"));
let data = pdfcraft_filters::encode_jbig2_generic(&blank, w, h).unwrap();
let mut doc = SharedPages { other, other_data: data, ..SharedPages::new(&scan) }.build(&scan);
mark(&mut doc, 0, &[CLEARS], "");
let run = apply(&mut doc, None);
assert_eq!(run.as_ref().is_ok_and(|r| r.images_restored == 1), stored, "{w}");
assert_eq!(matches!(run, Err(RedactError::SharedJbig2Globals(1))), !stored, "{w}");
}
}
/// An alternate of an image stored again stays with it, stored without the dictionary in turn: a page
/// no mark reached keeps what it had.
#[test]
fn an_alternate_of_what_stays_stays() {
let scan = shared_scan();
let other = format!("{} /Alternates [<< /Image 10 0 R >>]", shared_image(SHARED));
let mut doc = SharedPages { other, extra: vec![stream(&shared_image(SHARED), scan.page.clone())], ..SharedPages::new(&scan) }.build(&scan);
mark(&mut doc, 0, &[CLEARS], "");
assert_eq!(apply(&mut doc, None).unwrap().images_restored, 2);
assert_gone(&doc, b"LEAK_GLOBALS");
let saved = reopen(&doc);
let kept = saved.scan_objects().filter(|(_, o)| matches!(&**o, Object::Stream(s) if s.dict.contains(b"Alternates"))).count();
assert_eq!(kept, 1);
}
/// The refusal counts the images that can't be stored, and says so; when only something else refers
/// to the dictionary, it counts none and says that instead, however many such references there are.
#[test]
fn a_refused_redaction_counts_what_cant_be_stored() {
let scan = shared_scan();
let wrong = shared_image(SHARED).replace("/Height 40", "/Height 41");
let other = format!("{wrong} /SMask 10 0 R");
let mut doc = SharedPages { other, extra: vec![stream(&wrong, scan.page.clone())], ..SharedPages::new(&scan) }.build(&scan);
mark(&mut doc, 0, &[CLEARS], "");
let refused = apply(&mut doc, None);
assert!(matches!(refused, Err(RedactError::SharedJbig2Globals(2))));
assert!(refused.unwrap_err().to_string().starts_with("2 item(s) share JBIG2 symbol data with something being removed and can't be stored"));
let note = "/Type /XObject /Subtype /Image /Width 1 /Height 1 /BitsPerComponent 1 /ColorSpace /DeviceGray /Note 9 0 R";
for (other, extra, stuck) in [
(note.to_string(), vec![], 0),
(format!("{note} /SMask 10 0 R"), vec![stream(note, vec![0])], 0),
(format!("{note} /SMask 10 0 R"), vec![stream(&wrong, scan.page.clone())], 1),
] {
let mut doc = SharedPages { other: other.clone(), other_data: vec![0], extra, ..SharedPages::new(&scan) }.build(&scan);
mark(&mut doc, 0, &[CLEARS], "");
let refused = apply(&mut doc, None);
assert!(matches!(refused, Err(RedactError::SharedJbig2Globals(n)) if n == stuck), "{other}");
let text = refused.unwrap_err().to_string();
assert_eq!(text.starts_with("something else in the file refers to JBIG2 symbol data"), stuck == 0, "{other}: {text}");
assert_eq!(text.contains("can't be stored"), stuck > 0, "{other}: {text}");
}
// An image whose data is in another file can't be stored here, though what is here decodes: readers
// draw the other file, with the dictionary its own parameters name.
let elsewhere = format!("{} /F (page2.jb2) /FFilter /JBIG2Decode /FDecodeParms << /JBIG2Globals 9 0 R >>", shared_image(SHARED));
let mut doc = SharedPages { other: elsewhere, ..SharedPages::new(&scan) }.build(&scan);
mark(&mut doc, 0, &[CLEARS], "");
assert!(matches!(apply(&mut doc, None), Err(RedactError::SharedJbig2Globals(1))));
}
/// A dictionary that names itself as its own globals (malformed) can't be stored without them, but
/// leaves with what used it, so the run goes on; it isn't counted as stored, nor, when the run is
/// refused, as something that uses the dictionary.
#[test]
fn a_dictionary_that_names_itself_isnt_counted_as_stored() {
let scan = shared_scan();
let names_itself = |pages: SharedPages| {
let mut doc = pages.build(&scan);
let dictionary = pdfcraft_cos::ObjRef::new(9, 0);
let Object::Stream(mut s) = (*doc.get(dictionary)).clone() else { panic!("object 9 is the dictionary") };
let mut parms = pdfcraft_cos::Dict::new();
parms.set(b"JBIG2Globals".to_vec(), Object::Ref(dictionary));
s.dict.set(b"DecodeParms".to_vec(), Object::Dict(parms));
doc.set(dictionary, Object::Stream(s));
mark(&mut doc, 0, &[CLEARS], "");
doc
};
let mut doc = names_itself(SharedPages::new(&scan));
assert_eq!(apply(&mut doc, None).unwrap().images_restored, 1);
assert_gone(&doc, b"LEAK_GLOBALS");
assert_eq!(drawn_samples(&reopen(&doc), 1), scan.bits);
// The second page's image has its data in another file, so it can't be stored: one item.
let mut doc = names_itself(SharedPages { other: format!("{} /F (p.jb2)", shared_image(SHARED)), ..SharedPages::new(&scan) });
assert!(matches!(apply(&mut doc, None), Err(RedactError::SharedJbig2Globals(1))));
}
/// What a stream's size, filters and parameters are made direct to is small, however often they
/// refer to a large object: only numbers, names and the parameters decoders read are copied.
#[test]
fn what_is_made_direct_is_small_however_often_a_large_object_is_named() {
let big = format!("[{}]", "1 ".repeat(100_000));
let keys: String = (0..50).map(|k| format!(" /K{k} 2 0 R")).collect();
// Three filters, and parameters for 2,001: the entries past the filters are dropped. Two name
// object 6, whose `/JBIG2Globals` is the large array given directly, which no decoder reads.
let (filters, entries) = (" 2 0 R".repeat(2), format!(" 6 0 R 6 0 R{}", " 2 0 R".repeat(1998)));
let dict =
format!("/Width 2 0 R /Height 2 0 R /Filter [/FlateDecode{filters}] /DecodeParms [<< /Predictor 3 0 R /Columns 2 0 R{keys} >>{entries}]");
let globals = format!("<< /JBIG2Globals {big} >>");
// A chain of more filters than any decoder takes is no filter, and its parameters aren't read:
// ten here, against eight and a `/Crypt`.
let long = format!("/Filter [{}] /DecodeParms 2 0 R", "/FlateDecode ".repeat(10));
let doc = pdf(vec![
b"<< /Type /Catalog >>".to_vec(),
big.into_bytes(),
b"4 0 R".to_vec(),
b"12".to_vec(),
stream(&dict, b""),
globals.into_bytes(),
stream(&long, b""),
stream(&format!("/Filter [{}]", "/FlateDecode ".repeat(9)), b""),
// A name no decoder knows, long or short, is no filter, and isn't copied however often.
format!("/{}", "A".repeat(2000)).into_bytes(),
format!("[{} /Foo]", " 9 0 R".repeat(8)).into_bytes(),
stream("/Filter 10 0 R", b""),
]);
let Object::Stream(s) = &*doc.get(ObjRef::new(5, 0)) else { panic!("object 5 is the stream") };
let made = crate::globals::Run::new(crate::globals::RUN_BUDGET).readable(&doc, s).unwrap();
let mut bytes = Vec::new();
pdfcraft_cos::serialize(&Object::Dict(made.dict.clone()), &mut bytes);
assert!(bytes.len() < 4096, "{} bytes made direct", bytes.len());
let parms = made.dict.get(b"DecodeParms").and_then(Object::as_array).unwrap();
assert_eq!(parms.first().and_then(Object::as_dict).and_then(|d| d.get(b"Predictor")), Some(&Object::Int(12)));
assert_eq!(parms.len(), 3, "an entry for each filter");
assert!(parms[1].as_dict().is_some_and(|d| !d.contains(b"JBIG2Globals")), "{:?}", parms[1]);
let Object::Stream(s) = &*doc.get(ObjRef::new(7, 0)) else { panic!("object 7 is the stream") };
let made = crate::globals::Run::new(crate::globals::RUN_BUDGET).readable(&doc, s).unwrap();
assert_eq!(made.dict.get(b"Filter"), Some(&Object::Null));
assert_eq!(made.dict.get(b"DecodeParms").and_then(Object::as_array).map(Vec::len), Some(0));
// Eight filters and a `/Crypt` are as many as a decoder takes: those are read.
let Object::Stream(s) = &*doc.get(ObjRef::new(8, 0)) else { panic!("object 8 is the stream") };
let made = crate::globals::Run::new(crate::globals::RUN_BUDGET).readable(&doc, s).unwrap();
assert_eq!(made.dict.get(b"Filter").and_then(Object::as_array).map(Vec::len), Some(9));
let Object::Stream(s) = &*doc.get(ObjRef::new(11, 0)) else { panic!("object 11 is the stream") };
let made = crate::globals::Run::new(crate::globals::RUN_BUDGET).readable(&doc, s).unwrap();
assert_eq!(made.dict.get(b"Filter"), Some(&Object::Array(vec![Object::Null; 9])));
}
/// A `/JBIG2Globals` reference in parameters given directly is fetched once a run, however many
/// entries and streams name it, and what it leads to is held once.
#[test]
fn a_dictionary_named_in_direct_entries_is_fetched_once() {
let entries = " << /JBIG2Globals 2 0 R >>".repeat(9);
let mut objects = vec![b"<< /Type /Catalog >>".to_vec(), stream("", b"symbols")];
objects.extend((0..50).map(|_| stream(&format!("/Filter /JBIG2Decode /DecodeParms [{entries}]"), b"")));
let doc = pdf(objects);
let mut run = crate::globals::Run::new(crate::globals::RUN_BUDGET);
for n in 3..53 {
let Object::Stream(s) = &*doc.get(ObjRef::new(n, 0)) else { panic!("object {n} is a stream") };
let lists = run.names_of(&doc, s).unwrap();
assert_eq!(lists.iter().map(|(list, _)| list.to_vec()).collect::<Vec<_>>(), [vec![2]]);
}
assert_eq!((run.fetched, run.spent), (1, 2), "one fetch, and the one number held");
}
/// A `/JBIG2Globals` reference leading through a chain names each reference on the way, up to the
/// limit of 33, whether the chain is met first part way or from its start.
#[test]
fn a_chain_of_references_is_named_to_the_limit() {
// Objects 2 to 41 each refer to the next, and 42 is the dictionary; 43 names the chain from 20,
// and 44 from its start.
let mut objects = vec![b"<< /Type /Catalog >>".to_vec()];
objects.extend((3..=42).map(|n| format!("{n} 0 R").into_bytes()));
objects.push(stream("", b"symbols"));
objects.push(stream("/Filter /JBIG2Decode /DecodeParms << /JBIG2Globals 20 0 R >>", b""));
objects.push(stream("/Filter /JBIG2Decode /DecodeParms << /JBIG2Globals 2 0 R >>", b""));
let doc = pdf(objects);
let mut run = crate::globals::Run::new(crate::globals::RUN_BUDGET);
let named = |run: &mut crate::globals::Run, n: u32| {
let Object::Stream(s) = &*doc.get(ObjRef::new(n, 0)) else { panic!("object {n} is a stream") };
run.names_of(&doc, s).unwrap().iter().flat_map(|(list, _)| list.to_vec()).collect::<Vec<_>>()
};
assert_eq!(named(&mut run, 43), (20..=42).collect::<Vec<u32>>());
assert_eq!(named(&mut run, 44), (2..=34).collect::<Vec<u32>>());
}
/// An image is charged the decoded size of each dictionary it names, as decoding it may undo that
/// dictionary's filters again.
#[test]
fn an_image_is_charged_the_dictionaries_it_names() {
let image = "/Type /XObject /Subtype /Image /Width 8 /Height 8 /BitsPerComponent 1 /Filter /JBIG2Decode /DecodeParms << /JBIG2Globals 2 0 R >>";
let doc = pdf(vec![b"<< /Type /Catalog >>".to_vec(), stream("", vec![0u8; 40_000]), stream(image, b"not JBIG2")]);
let mut run = crate::globals::Run::new(crate::globals::RUN_BUDGET);
crate::globals::content(&mut run, &doc, 2, &mut pdfcraft_cos::ImageBudget::with_work(u64::MAX)).unwrap();
assert_eq!(run.spent, 10, "40,000 bytes: a unit and nine whole 4 KiB");
let Object::Stream(s) = &*doc.get(ObjRef::new(3, 0)) else { panic!("object 3 is the image") };
assert!(matches!(crate::globals::without_dictionary(&mut run, &doc, s, &mut pdfcraft_cos::ImageBudget::with_work(u64::MAX)), Ok(None)));
// The chain to 2 (a fetch and the number held), the dictionary's 10, the image's allowance 2.
assert_eq!(run.spent, 24);
}
/// A dictionary pays the run's image allowance for what every stage of its filters reads and makes,
/// not for what the last one makes: two Flates inflate a MiB of zero bytes, which hex skips as white
/// space, so it decodes to nothing. Past the allowance the run is refused, through `unshare` too.
#[test]
fn a_dictionary_pays_for_every_stage_of_its_filters() {
let hollow = pdfcraft_filters::encode_flate(&pdfcraft_filters::encode_flate(&vec![0u8; 1 << 20]));
let dict = || stream("/Filter [/FlateDecode /FlateDecode /ASCIIHexDecode]", hollow.clone());
let doc = pdf(vec![b"<< /Type /Catalog >>".to_vec(), dict()]);
let mut run = crate::globals::Run::new(crate::globals::RUN_BUDGET);
let mut images = pdfcraft_cos::ImageBudget::with_work(u64::MAX);
assert!(crate::globals::content(&mut run, &doc, 2, &mut images).unwrap().is_some());
assert_eq!(run.spent, 1, "it decodes to nothing: a unit");
let paid = u64::MAX - images.work_left();
assert!(paid > 1 << 20, "the MiB the second Flate made is paid for: {paid} steps");
let mut run = crate::globals::Run::new(crate::globals::RUN_BUDGET);
let refused = crate::globals::content(&mut run, &doc, 2, &mut pdfcraft_cos::ImageBudget::with_work(1 << 20));
assert_eq!(refused, Err(RedactError::ImagesPastAllowance));
// An image naming it leaves its page: finding copies of the dictionary reads it.
let image = "/Type /XObject /Subtype /Image /Width 8 /Height 8 /BitsPerComponent 1 /Filter /JBIG2Decode /DecodeParms << /JBIG2Globals 8 0 R >>";
let mut doc = one_page(b"q 8 0 0 8 0 0 cm /Im Do Q", "/XObject << /Im 7 0 R >>", vec![stream(image, b""), dict()]);
let before = crate::globals::kept(&doc).unwrap();
let page = pdfcraft_model::pages(&doc)[0].obj;
doc.update_dict(page, |d| {
d.remove(b"Resources");
})
.unwrap();
let refused = crate::globals::unshare(&mut doc.clone(), &before, &mut pdfcraft_cos::ImageBudget::with_work(1 << 20));
assert_eq!(refused, Err(RedactError::ImagesPastAllowance));
assert_eq!(crate::globals::unshare(&mut doc, &before, &mut pdfcraft_cos::ImageBudget::with_work(u64::MAX)), Ok(0));
}
/// What can't be decoded is charged as much as it was allowed to decode, so going past a limit
/// costs as much as reaching it: a dictionary `MAX_JBIG2_GLOBALS` (here one behind a filter that
/// isn't decoded here), an image its allowance.
#[test]
fn a_decode_that_fails_is_charged_what_it_was_allowed() {
let image = "/Type /XObject /Subtype /Image /Width 8 /Height 8 /BitsPerComponent 1 /Filter /JBIG2Decode";
let doc = pdf(vec![b"<< /Type /Catalog >>".to_vec(), stream("/Filter /DCTDecode", b"not a JPEG"), stream(image, b"not JBIG2")]);
let units = pdfcraft_cos::MAX_JBIG2_GLOBALS / 4096 + 1;
let run = |budget| crate::globals::Run::new(budget);
let images = || pdfcraft_cos::ImageBudget::with_work(u64::MAX);
assert!(crate::globals::content(&mut run(units), &doc, 2, &mut images()).is_ok());
assert_eq!(crate::globals::content(&mut run(units - 1), &doc, 2, &mut images()), Err(RedactError::SharedJbig2Budget));
// The image's allowance: (one byte a row of 8, and a byte a row) × 3 a byte, and 4096: 4144
// bytes, two units.
let Object::Stream(s) = &*doc.get(ObjRef::new(3, 0)) else { panic!("object 3 is the image") };
assert!(matches!(crate::globals::without_dictionary(&mut run(2), &doc, s, &mut pdfcraft_cos::ImageBudget::with_work(u64::MAX)), Ok(None)));
assert!(matches!(
crate::globals::without_dictionary(&mut run(1), &doc, s, &mut pdfcraft_cos::ImageBudget::with_work(u64::MAX)),
Err(RedactError::SharedJbig2Budget)
));
}
/// What streams share by reference is read once a run, however many share it and however they
/// refer to it: a parameters array of 5,000 entries named by 100 streams, directly, each through an
/// object of its own that refers to it, or through a chain of two, costs the array and its one
/// entry, once.
#[test]
fn a_shared_parameters_array_is_read_once_however_many_streams_share_it() {
let entries = " 4 0 R".repeat(5000);
let mut objects =
vec![b"<< /Type /Catalog >>".to_vec(), format!("[{entries}]").into_bytes(), stream("", b"symbols"), b"<< /JBIG2Globals 3 0 R >>".to_vec()];
// Objects 5 to 54 refer to the array, and 55 to 5.
objects.extend((0..50).map(|_| b"2 0 R".to_vec()));
objects.push(b"5 0 R".to_vec());
// Streams 56 to 155: fifty name the array directly, forty-nine through objects 5 to 53, one
// through the chain from 55.
objects.extend((0..50).map(|_| stream("/Filter /JBIG2Decode /DecodeParms 2 0 R", b"")));
objects.extend((5..54).map(|n| stream(&format!("/Filter /JBIG2Decode /DecodeParms {n} 0 R"), b"")));
objects.push(stream("/Filter /JBIG2Decode /DecodeParms 55 0 R", b""));
let doc = pdf(objects);
let mut run = crate::globals::Run::new(crate::globals::RUN_BUDGET);
for n in 56..156 {
let Object::Stream(s) = &*doc.get(ObjRef::new(n, 0)) else { panic!("object {n} is a stream") };
let lists = run.names_of(&doc, s).unwrap();
assert_eq!(lists.iter().map(|(list, _)| list.to_vec()).collect::<Vec<_>>(), [vec![3]]);
}
assert_eq!(run.read, 2);
// Each object once: the array, its entry, the dictionary it names, each of the 49 objects that
// refer to the array, and 55.
assert_eq!(run.fetched, 53);
// A run whose budget that is past is refused.
let mut run = crate::globals::Run::new(20);
let refused = (56..156).try_for_each(|n| {
let Object::Stream(s) = &*doc.get(ObjRef::new(n, 0)) else { panic!("object {n} is a stream") };
run.names_of(&doc, s).map(|_| ())
});
assert_eq!(refused, Err(RedactError::SharedJbig2Budget));
}
/// A run's budget is spent to the unit: a run may spend all of it, and is refused past it. What is
/// decoded spends a unit, and one more for each whole 4 KiB.
#[test]
fn a_runs_budget_is_spent_to_the_unit() {
let mut run = crate::globals::Run::new(10);
assert_eq!(run.decoded(4095), Ok(()));
assert_eq!(run.decoded(8 * 4096), Ok(()));
assert_eq!(run.decoded(0), Err(RedactError::SharedJbig2Budget));
}
/// What a stream's size, filters and parameters lead to is fetched once a run, however many streams
/// share them by reference: fifty images naming the same objects fetch each once for each way it is
/// read.
#[test]
fn what_streams_share_is_fetched_once_a_run() {
let mut objects =
vec![b"<< /Type /Catalog >>".to_vec(), b"/FlateDecode".to_vec(), b"<< /Predictor 4 0 R /Columns 4 0 R >>".to_vec(), b"1".to_vec()];
objects.extend(
(0..50).map(|_| stream("/Width 4 0 R /Height 4 0 R /Filter [2 0 R 2 0 R] /DecodeParms [3 0 R 3 0 R] /ColorSpace 2 0 R /F 4 0 R", b"")),
);
let doc = pdf(objects);
let mut run = crate::globals::Run::new(crate::globals::RUN_BUDGET);
for n in 5..55 {
let Object::Stream(s) = &*doc.get(ObjRef::new(n, 0)) else { panic!("object {n} is a stream") };
let made = run.readable(&doc, s).unwrap();
assert_eq!(made.dict.get(b"Width"), Some(&Object::Int(1)));
assert_eq!(made.dict.get(b"Filter").and_then(Object::as_array).map(Vec::len), Some(2));
}
// Object 4 as a size and as a parameter, 2 as a filter, 3 as an entry.
assert_eq!(run.fetched, 4);
}
/// A copy of the dictionary stored with a predictor is found whatever way its parameters are given,
/// through a chain of references among them, as a reader that follows them decodes it. Through a
/// chain the run either stores the image again with the copy's symbols, now that a reference to a
/// reference reads as what it names (#5548), or is refused; it never goes on with the copy left in
/// the file.
#[test]
fn a_copy_is_found_however_its_parameters_are_given() {
let scan = shared_scan();
let globals = [jbig2_globals(b"LEAK_GLOBALS"), scan.symbols.clone()].concat();
// PNG rows of one byte, each with no prediction: a byte for the row's tag, then the byte.
let predicted: Vec<u8> = globals.iter().flat_map(|b| [0u8, *b]).collect();
let flate = pdfcraft_filters::encode_flate(&predicted);
let parms = b"<< /Predictor 12 /Columns 1 >>".to_vec();
for (spelling, extra, stored) in [
("/DecodeParms << /Predictor 12 /Columns 1 >>", vec![], true),
("/DecodeParms 11 0 R", vec![parms.clone()], true),
("/DecodeParms 11 0 R", vec![b"12 0 R".to_vec(), parms.clone()], false),
("/DecodeParms [11 0 R]", vec![b"12 0 R".to_vec(), parms.clone()], false),
("/DecodeParms << /Predictor 11 0 R /Columns 1 >>", vec![b"12 0 R".to_vec(), b"12".to_vec()], false),
// `false` above: stored again or refused, either is safe (see the doc).
] {
let mut objects = vec![stream(&format!("/Filter /FlateDecode {spelling}"), flate.clone())];
objects.extend(extra.clone());
let other = shared_image("/DecodeParms << /JBIG2Globals 10 0 R >>");
let mut doc = SharedPages { other, extra: objects, ..SharedPages::new(&scan) }.build(&scan);
mark(&mut doc, 0, &[CLEARS], "");
let at = format!("{spelling} {}", extra.len());
match apply(&mut doc, None) {
Ok(report) => {
assert_eq!(report.images_restored, 1, "{at}");
let kept = pdfcraft_cos::reachable_objects(&doc).unwrap();
assert!(!kept.contains(&10) && !kept.contains(&9), "{at}: the copy or the dictionary stays");
assert_eq!(drawn_samples(&reopen(&doc), 1), scan.bits, "{at}");
}
Err(e) => assert!(!stored && matches!(e, RedactError::SharedJbig2Globals(1)), "{at}: {e:?}"),
}
}
}
/// An image that names the dictionary and another one is stored again without both, but the other
/// was shared with nothing that left: an image that uses only it stays as it was, whichever of the
/// two comes first in the file, and so does that dictionary.
#[test]
fn another_dictionary_named_beside_it_stays_whatever_the_order() {
let scan = shared_scan();
let other_dictionary = stream("", [jbig2_globals(b"OTHER_GLOBALS"), scan.symbols.clone()].concat());
let only_other = shared_image("/DecodeParms << /JBIG2Globals 11 0 R >>");
let chained = "/Type /XObject /Subtype /Image /Width 400 /Height 40 /BitsPerComponent 1 /ColorSpace /DeviceGray /Filter [/FlateDecode /JBIG2Decode] /DecodeParms [<< /JBIG2Globals 11 0 R >> << /JBIG2Globals 9 0 R >>]";
let both_spellings = [
(chained.to_string(), pdfcraft_filters::encode_flate(&scan.page)),
(format!("{} /DP << /JBIG2Globals 11 0 R >>", shared_image(SHARED)), scan.page.clone()),
];
let other_stays = |doc: &Document| {
let saved = reopen(doc);
saved.scan_objects().any(|(_, o)| matches!(&*o, Object::Stream(s) if s.raw.windows(13).any(|w| w == b"OTHER_GLOBALS")))
};
for (both, data) in both_spellings {
// Image 8 is drawn on the second page, and image 10 is its alternate.
let only_other_first = SharedPages {
other: format!("{only_other} /Alternates [<< /Image 10 0 R >>]"),
extra: vec![stream(&both, data.clone()), other_dictionary.clone()],
..SharedPages::new(&scan)
};
let both_first = SharedPages {
other: format!("{both} /Alternates [<< /Image 10 0 R >>]"),
other_data: data.clone(),
extra: vec![stream(&only_other, scan.page.clone()), other_dictionary.clone()],
..SharedPages::new(&scan)
};
for (pages, only_other_at) in [(only_other_first, 8), (both_first, 10)] {
let mut doc = pages.build(&scan);
let untouched = doc.get(pdfcraft_cos::ObjRef::new(only_other_at, 0));
mark(&mut doc, 0, &[CLEARS], "");
assert_eq!(apply(&mut doc, None).unwrap().images_restored, 1, "{both}");
assert_gone(&doc, b"LEAK_GLOBALS");
assert!(other_stays(&doc), "{both}");
assert_eq!(doc.get(pdfcraft_cos::ObjRef::new(only_other_at, 0)), untouched, "{both}");
}
}
}
/// Remove Hidden Information leaves the same way: comments go, with what their appearances drew.
#[test]
fn removing_hidden_information_takes_a_shared_jbig2_dictionary_too() {
let scan = shared_scan();
let form = stream("/Type /XObject /Subtype /Form /BBox [0 0 300 300] /Resources << /XObject << /Im 7 0 R >> >>", DRAWS);
let annotation = b"<< /Type /Annot /Subtype /Stamp /Rect [0 100 200 120] /AP << /N 11 0 R >> >>".to_vec();
let mut doc = SharedPages {
content: Vec::new(),
resources: String::new(),
page: "/Annots [10 0 R]".into(),
extra: vec![annotation, form],
..SharedPages::new(&scan)
}
.build(&scan);
crate::sanitize::remove_hidden(&mut doc, &[crate::sanitize::Hidden::Comments]).unwrap();
assert_gone(&doc, b"LEAK_GLOBALS");
assert_eq!(drawn_samples(&reopen(&doc), 1), scan.bits);
}
/// Remove Hidden Information refuses as a redaction does when the dictionary can't leave, but it has
/// changed the document by then, so the refusal doesn't say nothing was changed.
#[test]
fn a_refused_removal_of_hidden_information_doesnt_say_nothing_changed() {
let scan = shared_scan();
let form = stream("/Type /XObject /Subtype /Form /BBox [0 0 300 300] /Resources << /XObject << /Im 7 0 R >> >>", DRAWS);
let annotation = b"<< /Type /Annot /Subtype /Stamp /Rect [0 100 200 120] /AP << /N 11 0 R >> >>".to_vec();
let mut doc = SharedPages {
content: Vec::new(),
resources: String::new(),
page: "/Annots [10 0 R]".into(),
other: shared_image(SHARED).replace("/Filter /JBIG2Decode", "/Filter /FlateDecode"),
other_data: pdfcraft_filters::encode_flate(&scan.bits),
extra: vec![annotation, form],
..SharedPages::new(&scan)
}
.build(&scan);
let annotations = |doc: &Document| crate::annots_of(doc, &pdfcraft_model::pages(doc)[0].dict).len();
assert_eq!(annotations(&doc), 1);
let refused = crate::sanitize::remove_hidden(&mut doc, &[crate::sanitize::Hidden::Comments]);
assert!(matches!(refused, Err(RedactError::SharedJbig2Globals(1))));
assert_eq!(annotations(&doc), 0, "the comment went before the refusal");
let text = refused.unwrap_err().to_string();
assert!(text.ends_with("this was refused") && !text.contains("nothing was changed"), "{text}");
}
/// Remove Hidden Information also unlinks an image its page lists but no longer draws (left there by
/// Delete Image, say): that is a way out too, so the dictionary goes with it.
#[test]
fn an_undrawn_image_unlinked_takes_a_shared_jbig2_dictionary_too() {
let scan = shared_scan();
let mut doc = SharedPages { content: Vec::new(), ..SharedPages::new(&scan) }.build(&scan);
crate::sanitize::remove_hidden(&mut doc, &[crate::sanitize::Hidden::Comments]).unwrap();
assert!(!crate::globals::kept(&doc).unwrap().contains(&7), "the image nothing drew is unlinked");
assert_gone(&doc, b"LEAK_GLOBALS");
assert_eq!(drawn_samples(&reopen(&doc), 1), scan.bits);
}
/// A redaction also removes the catalog's XMP when it repeats the removed text (#2448): an image
/// only that packet still refers to leaves the file then, and the dictionary with it.
#[test]
fn an_image_only_the_scrubbed_metadata_refers_to_takes_a_shared_jbig2_dictionary_too() {
let scan = shared_scan();
let mut doc = SharedPages {
content: [DRAWS, b" BT /F1 10 Tf 10 105 Td (Bluefin) Tj ET".as_slice()].concat(),
resources: "/XObject << /Im 7 0 R >> /Font << /F1 11 0 R >>".into(),
extra: vec![
stream("/Type /Metadata /Subtype /XML /Thumb 7 0 R", b"<x:xmpmeta><dc:title>Bluefin</dc:title></x:xmpmeta>"),
FONT.replace("95 0 R", "12 0 R").into_bytes(),
widths(),
],
..SharedPages::new(&scan)
}
.build(&scan);
let root = doc.root().unwrap();
doc.update_dict(root, |d| d.set(b"Metadata".to_vec(), Object::Ref(ObjRef::new(10, 0)))).unwrap();
mark(&mut doc, 0, &[COVERS], "");
let report = apply(&mut doc, None).unwrap();
assert_eq!((report.metadata, report.images_restored), (1, 1), "the packet went, and the image on page 2 was stored again");
assert_gone(&doc, b"LEAK_GLOBALS");
assert_eq!(drawn_samples(&reopen(&doc), 1), scan.bits);
}
/// A redaction that leaves a pattern unpainted unlinks it from a `/Pages` node no page inherits
/// from any more, here one the page's own parent shadowed: an image its cell drew leaves the file
/// then, and the dictionary with it.
#[test]
fn an_image_a_pruned_patterns_cell_drew_takes_a_shared_jbig2_dictionary_too() {
let scan = shared_scan();
let cell = "/Type /Pattern /PatternType 1 /PaintType 1 /TilingType 1 /BBox [0 0 300 300] /XStep 300 /YStep 300 /Resources << /XObject << /Im 7 0 R >> >>";
let image = shared_image(SHARED);
let mut doc = pdf(vec![
b"<< /Type /Catalog /Pages 2 0 R >>".to_vec(),
b"<< /Type /Pages /Kids [11 0 R 4 0 R] /Count 2 /Resources << /Pattern << /P0 10 0 R >> >> >>".to_vec(),
b"<< /Type /Page /Parent 11 0 R /MediaBox [0 0 300 300] /Contents 5 0 R >>".to_vec(),
b"<< /Type /Page /Parent 2 0 R /MediaBox [0 0 300 300] /Contents 6 0 R /Resources << /XObject << /Im 8 0 R >> >> >>".to_vec(),
stream("", b"/Pattern cs /P0 scn 0 100 200 20 re f"),
stream("", DRAWS),
stream(&image, &scan.page),
stream(&image, &scan.page),
stream("", [jbig2_globals(b"LEAK_GLOBALS"), scan.symbols.clone()].concat()),
stream(cell, DRAWS),
b"<< /Type /Pages /Parent 2 0 R /Kids [3 0 R] /Count 1 /Resources << /Pattern << /P0 10 0 R >> >> >>".to_vec(),
]);
mark(&mut doc, 0, &[[-5.0, 95.0, 205.0, 125.0]], "");
let report = apply(&mut doc, None).unwrap();
assert_eq!((report.patterns_removed, report.images_restored), (1, 1));
assert!(!crate::globals::kept(&doc).unwrap().contains(&10), "the cell is unlinked");
assert_gone(&doc, b"LEAK_GLOBALS");
assert_eq!(drawn_samples(&reopen(&doc), 1), scan.bits);
}
/// [`assert_no_secret`] for a file that may hold JBIG2 streams, which `decoded` doesn't read: a stream
/// stored again can be coded alone.
fn assert_gone(doc: &Document, secret: &[u8]) {
for (r, object) in reopen(doc).scan_objects() {
let mut bytes = Vec::new();
pdfcraft_cos::serialize(&object, &mut bytes);
assert!(!bytes.windows(secret.len()).any(|w| w == secret), "secret remains in object {r:?}");
if let Object::Stream(s) = &*object
&& let Ok(data) = s.decoded()
{
assert!(!data.windows(secret.len()).any(|w| w == secret), "secret remains in stream {r:?}");
}
}
}
/// Apply Redactions gives the page its own pruned /Resources, but the dictionary it drew through
/// before (inherited from the /Pages node, or shared with another page) still named the removed
/// image and form, so a full-rewrite save kept them.
#[test]
fn redacted_xobjects_do_not_survive_through_inherited_or_shared_resources() {
for inherited in [true, false] {
let mut doc = two_pages_one_resource_dictionary(inherited, false);
mark(&mut doc, 0, &[[0.0, 200.0, 300.0, 300.0]], "");
let r = apply(&mut doc, None).unwrap();
assert!(r.images_removed >= 1 && r.forms_rewritten >= 1, "{r:?}");
let bytes = write_full(&doc, &SaveOptions::default()).unwrap();
let has = |needle: &[u8]| bytes.windows(needle.len()).any(|w| w == needle);
for secret in [&b"REDIMGONECOVERED"[..], b"REDIMGPARTSECRET", b"REDFORMSECRETTEXT"] {
assert!(!has(secret), "{} survived the save (inherited: {inherited})", String::from_utf8_lossy(secret));
}
// Page 2 still draws its own image through the same dictionary.
let saved = Document::open(Arc::new(bytes)).unwrap();
assert_eq!(named_image(&saved, 1, b"ImKeep").as_deref(), Some(&b"PUBIMGKEEPSURVIVE"[..]), "inherited: {inherited}");
}
}
/// The same image shown on another page is not under a mark there: that page keeps it.
#[test]
fn an_image_another_page_still_draws_through_the_same_resources_is_kept() {
for inherited in [true, false] {
let mut doc = two_pages_one_resource_dictionary(inherited, true);
mark(&mut doc, 0, &[[0.0, 200.0, 300.0, 300.0]], "");
apply(&mut doc, None).unwrap();
let saved = Document::open(Arc::new(write_full(&doc, &SaveOptions::default()).unwrap())).unwrap();
assert_eq!(named_image(&saved, 1, b"Im1").as_deref(), Some(&b"REDIMGONECOVERED"[..]), "inherited: {inherited}");
assert_eq!(named_image(&saved, 0, b"Im1"), None, "the redacted page no longer names it (inherited: {inherited})");
assert!(content(&saved, 1).contains("/Im1 Do"), "page 2 is untouched");
}
}
/// A form without /Resources draws through its caller's: the image it drew under the mark is
/// named by the page's own dictionary, which no `Do` of the page itself ever used.
#[test]
fn an_image_a_form_drew_through_the_pages_resources_does_not_survive() {
let mut doc = one_page(
b"/FmBare Do",
"/XObject << /FmBare 7 0 R /Im1 8 0 R >>",
vec![
stream("/Type /XObject /Subtype /Form /BBox [0 0 300 300]", b"q 50 0 0 50 20 230 cm /Im1 Do Q"),
stream("/Type /XObject /Subtype /Image /Width 16 /Height 1 /ColorSpace /DeviceGray /BitsPerComponent 8", b"REDIMGONECOVERED"),
],
);
mark(&mut doc, 0, &[[0.0, 200.0, 300.0, 300.0]], "");
let r = apply(&mut doc, None).unwrap();
assert_eq!(r.images_removed, 1, "{r:?}");
let bytes = write_full(&doc, &SaveOptions::default()).unwrap();
assert!(!bytes.windows(16).any(|w| w == b"REDIMGONECOVERED"), "the image the form drew survived the save");
}
/// How a form can have no resources of its own: no /Resources, a null one, or a reference to an
/// object that isn't there (which is null too).
const NO_RESOURCES: [&str; 3] = ["", "/Resources null", "/Resources 99 0 R"];
/// The page draws an image under the mark and, outside it, a form without /Resources of its own
/// that draws the same image through the page's: the page keeps the name, whether the form lies
/// wholly outside the mark or is entered and left as it was.
#[test]
fn an_image_a_form_draws_again_through_the_pages_resources_is_kept() {
for bbox in ["[0 0 100 100]", "[0 0 300 300]"] {
for none in NO_RESOURCES {
let mut doc = one_page(
b"q 50 0 0 50 20 230 cm /Im1 Do Q /FmBare Do",
"/XObject << /FmBare 7 0 R /Im1 8 0 R >>",
vec![
stream(&format!("/Type /XObject /Subtype /Form /BBox {bbox} {none}"), b"q 50 0 0 50 20 20 cm /Im1 Do Q"),
stream("/Type /XObject /Subtype /Image /Width 16 /Height 1 /ColorSpace /DeviceGray /BitsPerComponent 8", b"PUBIMGSHOWNAGAIN"),
],
);
mark(&mut doc, 0, &[[0.0, 200.0, 300.0, 300.0]], "");
let r = apply(&mut doc, None).unwrap();
assert_eq!((r.images_removed, r.forms_rewritten), (1, 0), "{bbox} {none}: {r:?}");
let saved = Document::open(Arc::new(write_full(&doc, &SaveOptions::default()).unwrap())).unwrap();
assert!(content(&saved, 0).contains("/FmBare Do"), "{bbox} {none}");
assert_eq!(named_image(&saved, 0, b"Im1").as_deref(), Some(&b"PUBIMGSHOWNAGAIN"[..]), "{bbox} {none}: the form still draws it");
}
}
}
/// The same one level down: a form with resources of its own draws the image under the mark and,
/// outside it, a form without any that draws it again through the first form's.
#[test]
fn an_image_a_nested_form_draws_again_through_its_callers_resources_is_kept() {
for none in NO_RESOURCES {
let mut doc = one_page(
b"/FmOwn Do",
"/XObject << /FmOwn 7 0 R >>",
vec![
stream(
"/Type /XObject /Subtype /Form /BBox [0 0 300 300] /Resources << /XObject << /FmBare 8 0 R /Im1 9 0 R >> >>",
b"q 50 0 0 50 20 230 cm /Im1 Do Q /FmBare Do",
),
stream(&format!("/Type /XObject /Subtype /Form /BBox [0 0 100 100] {none}"), b"q 50 0 0 50 20 20 cm /Im1 Do Q"),
stream("/Type /XObject /Subtype /Image /Width 16 /Height 1 /ColorSpace /DeviceGray /BitsPerComponent 8", b"PUBIMGSHOWNAGAIN"),
],
);
mark(&mut doc, 0, &[[0.0, 200.0, 300.0, 300.0]], "");
let r = apply(&mut doc, None).unwrap();
assert_eq!((r.images_removed, r.forms_rewritten), (1, 1), "{none}: {r:?}");
let saved = Document::open(Arc::new(write_full(&doc, &SaveOptions::default()).unwrap())).unwrap();
assert!(content(&saved, 0).contains("/PCRedacted1 Do"), "{none}");
let form = named_stream(&saved, 0, b"PCRedacted1").expect("the rewritten form");
assert!(form.decoded().unwrap().windows(10).any(|w| w == b"/FmBare Do"), "{none}");
let res = saved.resolve(form.dict.get(b"Resources").unwrap()).as_dict().cloned().unwrap();
let xo = saved.resolve(res.get(b"XObject").unwrap()).as_dict().cloned().unwrap();
let image = xo.get(b"Im1").and_then(Object::as_ref).unwrap_or_else(|| panic!("{none}: the rewritten form still names /Im1"));
let Object::Stream(image) = &*saved.get(image) else { panic!("/Im1 is a stream") };
assert_eq!(decoded_image(&saved, image).unwrap(), b"PUBIMGSHOWNAGAIN", "{none}");
}
}
/// A form without /Resources that won't decode may draw the image under the mark through the
/// page's: the redaction is refused. A page with fonts or properties to clean up refused it
/// already; this one has neither.
#[test]
fn a_form_that_cannot_be_read_refuses_the_redaction_of_an_image_it_may_draw() {
let mut doc = pdf(vec![
b"<< /Type /Catalog /Pages 2 0 R >>".to_vec(),
b"<< /Type /Pages /Kids [3 0 R] /Count 1 >>".to_vec(),
b"<< /Type /Page /Parent 2 0 R /MediaBox [0 0 300 300] /Contents 4 0 R /Resources << /XObject << /FmBroken 5 0 R /Im1 6 0 R >> >> >>"
.to_vec(),
stream("", b"q 50 0 0 50 20 230 cm /Im1 Do Q /FmBroken Do"),
stream("/Type /XObject /Subtype /Form /BBox [0 0 100 100] /Filter /DCTDecode", b"q 50 0 0 50 20 20 cm /Im1 Do Q"),
stream("/Type /XObject /Subtype /Image /Width 16 /Height 1 /ColorSpace /DeviceGray /BitsPerComponent 8", b"PUBIMGSHOWNAGAIN"),
]);
let Object::Stream(broken) = &*doc.get(ObjRef::new(5, 0)) else { panic!("the form is a stream") };
assert!(broken.decoded().is_err(), "the fixture's form must not decode");
mark(&mut doc, 0, &[[0.0, 200.0, 300.0, 300.0]], "");
assert!(matches!(apply(&mut doc, None), Err(RedactError::Cos(pdfcraft_cos::CosError::Filter(_)))));
}
/// A page of two content streams, with no fonts or properties: the first draws an image at
/// (20, 230), 50 pt square, named `image` in the page's resources; the second is `tail`, which
/// draws nothing. `catalog` goes in the catalog, and `image_dict` in the image's dictionary.
fn image_and_a_tail(catalog: &str, image: &str, image_dict: &str, tail: &[u8]) -> Document {
pdf(vec![
format!("<< /Type /Catalog /Pages 2 0 R {catalog} >>").into_bytes(),
b"<< /Type /Pages /Kids [3 0 R] /Count 1 >>".to_vec(),
format!("<< /Type /Page /Parent 2 0 R /MediaBox [0 0 300 300] /Contents [4 0 R 5 0 R] /Resources << /XObject << /{image} 6 0 R >> >> >>")
.into_bytes(),
stream("", format!("q 50 0 0 50 20 230 cm /{image} Do Q").as_bytes()),
stream("", tail),
stream(
&format!("/Type /XObject /Subtype /Image /Width 16 /Height 1 /ColorSpace /DeviceGray /BitsPerComponent 8 {image_dict}"),
b"REDIMGONECOVERED",
),
b"<< /Type /OCG /Name (Secret layer) >>".to_vec(),
])
}
/// What the page draws can't be told when content the redaction leaves as it was holds bytes the
/// parser can't read: the redaction of an image is refused. In a content stream of the page, the
/// walk reads past them and refuses the page itself (`Unreadable`). In a form without resources of
/// its own that lies away from the mark, which the walk doesn't enter, the check of what the page
/// still draws through its resources refuses it.
#[test]
fn unreadable_bytes_in_a_stream_left_as_it_was_refuse_the_redaction_of_an_image() {
for (tail, refused) in [(&b"q Q )"[..], true), (b"q Q", false)] {
let mut doc = image_and_a_tail("", "Im1", "", tail);
mark(&mut doc, 0, &[[0.0, 200.0, 300.0, 300.0]], "");
let result = apply(&mut doc, None).map(|r| r.images_removed);
assert_eq!(result, if refused { Err(RedactError::Unreadable(1)) } else { Ok(1) });
}
let mut doc = image_and_a_tail("", "Im1", "", b"/Fm Do");
let bbox = Object::Array(vec![0.into(), 0.into(), 300.into(), 100.into()]);
let form = Dict::from_iter([(b"Type".to_vec(), Object::name("XObject")), (b"Subtype".to_vec(), Object::name("Form")), (b"BBox".to_vec(), bbox)]);
let form = doc.add(Object::Stream(Stream::from_raw(form, b"q Q )".to_vec())));
doc.update_dict(ObjRef::new(3, 0), |d| {
if let Some(Object::Dict(res)) = d.get_mut(b"Resources")
&& let Some(Object::Dict(xo)) = res.get_mut(b"XObject")
{
xo.set(b"Fm".to_vec(), Object::Ref(form));
}
})
.unwrap();
mark(&mut doc, 0, &[[0.0, 200.0, 300.0, 300.0]], "");
let result = apply(&mut doc, None);
assert!(
matches!(&result, Err(RedactError::Cos(pdfcraft_cos::CosError::Syntax { detail, .. })) if detail == "redaction could not fully inspect inherited form resources"),
"{result:?}"
);
}
/// Remove Hidden Information refuses the same page: removing an off layer's image is refused on a
/// page whose drawing can't be told. Its walk reads every content stream, and refuses the page
/// itself.
#[test]
fn unreadable_bytes_in_a_stream_left_as_it_was_refuse_removing_a_hidden_image() {
use crate::sanitize::{Hidden, remove_hidden};
for (tail, refused) in [(&b"q Q )"[..], true), (b"q Q", false)] {
let mut doc = image_and_a_tail("/OCProperties << /OCGs [7 0 R] /D << /OFF [7 0 R] >> >>", "ImOC", "/OC 7 0 R", tail);
let result = remove_hidden(&mut doc, &[Hidden::HiddenLayers]);
if refused {
assert_eq!(result, Err(RedactError::Unreadable(1)));
} else {
assert!(result.unwrap().iter().any(|(h, n)| *h == Hidden::HiddenLayers && *n > 0));
}
}
}
/// Remove Hidden Information keeps a name that a form without resources still draws, too. Hidden
/// text includes text wholly off the page, so what it removes from a form depends on where it is
/// drawn: the page's /Fm, drawn off the page, is rewritten, while /G draws /Fm again on the page,
/// where its text shows, and stays as it was. The page still names /Fm, with its text.
#[test]
fn removing_hidden_text_keeps_a_form_a_form_without_resources_still_draws() {
use crate::sanitize::{Hidden, remove_hidden};
for none in NO_RESOURCES {
let mut doc = pdf(vec![
b"<< /Type /Catalog /Pages 2 0 R >>".to_vec(),
b"<< /Type /Pages /Kids [3 0 R] /Count 1 >>".to_vec(),
b"<< /Type /Page /Parent 2 0 R /MediaBox [0 0 300 300] /Contents 4 0 R /Resources << /XObject << /Fm 5 0 R /G 6 0 R >> >> >>".to_vec(),
stream("", b"q 1 0 0 1 1000 0 cm /Fm Do Q /G Do"),
stream(
"/Type /XObject /Subtype /Form /BBox [0 0 300 300] /Resources << /Font << /F1 7 0 R >> >>",
b"BT /F1 10 Tf 20 20 Td (KEEPTEXT) Tj ET",
),
stream(&format!("/Type /XObject /Subtype /Form /BBox [0 0 300 300] {none}"), b"/Fm Do"),
FONT.replace("95 0 R", "8 0 R").into_bytes(),
widths(),
]);
let removed = remove_hidden(&mut doc, &[Hidden::HiddenText]).unwrap();
assert!(removed.iter().any(|(h, n)| *h == Hidden::HiddenText && *n > 0), "{none:?}: nothing was removed");
let saved = Document::open(Arc::new(write_full(&doc, &SaveOptions::default()).unwrap())).unwrap();
assert!(content(&saved, 0).contains("/G Do"), "{none:?}: /G was rewritten");
let fm = named_stream(&saved, 0, b"Fm").and_then(|s| s.decoded().ok());
assert!(fm.is_some_and(|d| d.windows(8).any(|w| w == b"KEEPTEXT")), "{none:?}: the page lost /Fm, which /G still draws");
}
}
/// The page's own check follows only forms without resources of their own: a form that names a
/// different image /Im1 in resources of its own doesn't keep the page's redacted /Im1.
#[test]
fn the_pages_check_does_not_follow_a_form_with_resources_of_its_own() {
let image = "/Type /XObject /Subtype /Image /Width 1 /Height 1 /ColorSpace /DeviceGray /BitsPerComponent 8";
let doc = pdf(vec![
b"<< /Type /Catalog /Pages 2 0 R >>".to_vec(),
b"<< /Type /Pages /Kids [] /Count 0 >>".to_vec(),
stream("/Type /XObject /Subtype /Form /BBox [0 0 1 1] /Resources << /XObject << /Im1 4 0 R >> >>", b"/Im1 Do"),
stream(image, b"o"),
stream(image, b"p"),
]);
let mut xo = Dict::from_iter([(b"FmOwn".to_vec(), Object::Ref(ObjRef::new(3, 0))), (b"Im1".to_vec(), Object::Ref(ObjRef::new(5, 0)))]);
crate::interp::prune(&doc, &mut pdfcraft_cos::DecodeBudget::page(), &mut xo, &[b"Im1".to_vec()], &[b"/FmOwn Do".to_vec()]).unwrap();
assert!(!xo.contains(b"Im1") && xo.contains(b"FmOwn"), "{xo:?}");
}
/// What content draws through its resources is read up to 2,000,000 operators, those of the forms
/// it draws without resources of their own included; past that it is refused.
#[test]
fn what_content_draws_through_its_resources_is_read_up_to_two_million_operators() {
let doc = Document::new_empty();
for (ops, read) in [(2_000_000, true), (2_000_001, false)] {
let content = b"n ".repeat(ops);
assert_eq!(
crate::interp::resource_ops(&doc, &mut pdfcraft_cos::DecodeBudget::page(), &Dict::new(), &[content]).is_ok(),
read,
"{ops} operators"
);
}
// The content's own `/Fm Do` is one of them.
for (ops, read) in [(1_999_999, true), (2_000_000, false)] {
let doc = pdf(vec![
b"<< /Type /Catalog /Pages 2 0 R >>".to_vec(),
b"<< /Type /Pages /Kids [] /Count 0 >>".to_vec(),
stream("/Type /XObject /Subtype /Form /BBox [0 0 1 1]", b"n ".repeat(ops)),
]);
let res = Dict::from_iter([(b"XObject".to_vec(), Object::Dict(Dict::from_iter([(b"Fm".to_vec(), Object::Ref(ObjRef::new(3, 0)))])))]);
assert_eq!(
crate::interp::resource_ops(&doc, &mut pdfcraft_cos::DecodeBudget::page(), &res, &[b"/Fm Do".to_vec()]).is_ok(),
read,
"a form of {ops} operators"
);
}
}
/// `content` behind a PNG predictor (Up, one column) whose `/DecodeParms` are given by reference,
/// in a stream with `entries` too: read without the parameters, its bytes are the predictor's tags
/// and differences, not the operators.
fn behind_parms_by_reference(doc: &mut Document, entries: &[(&str, Object)], content: &[u8]) -> Stream {
let mut rows = Vec::new();
let mut above = 0u8;
for &b in content {
rows.extend([2, b.wrapping_sub(above)]);
above = b;
}
let parms = doc.add(Object::Dict(Dict::from_iter([(b"Predictor".to_vec(), Object::Int(12))])));
let mut s = Stream::flate(Dict::new(), &rows);
for (key, value) in entries {
s.dict.set(key.as_bytes().to_vec(), value.clone());
}
s.dict.set(b"DecodeParms".to_vec(), Object::Ref(parms));
s
}
/// A form's entries: `/Subtype /Form` and `bbox`.
fn form_entries(bbox: [i64; 4]) -> Vec<(&'static str, Object)> {
vec![("Subtype", Object::name("Form")), ("BBox", Object::Array(bbox.into_iter().map(Object::Int).collect()))]
}
/// A form outside the mark draws the image again through the page's resources, its content behind
/// a predictor whose `/DecodeParms` are given by reference. Both checks of whether the page still
/// draws it read the form through them, the page's (`resource_ops`) and the unlinking one at the
/// end (`names_content_draws`): the page keeps the image for the form.
#[test]
fn an_image_a_form_with_parameters_by_reference_draws_again_is_kept() {
let mut doc = one_page(
b"q 50 0 0 50 20 230 cm /Im1 Do Q /FmBare Do",
"/XObject << /FmBare 7 0 R /Im1 8 0 R >>",
vec![
stream("/Type /XObject /Subtype /Form /BBox [0 0 100 100]", b""),
stream("/Type /XObject /Subtype /Image /Width 16 /Height 1 /ColorSpace /DeviceGray /BitsPerComponent 8", b"PUBIMGSHOWNAGAIN"),
],
);
let form = behind_parms_by_reference(&mut doc, &form_entries([0, 0, 100, 100]), b"q 50 0 0 50 20 20 cm /Im1 Do Q");
doc.set(ObjRef::new(7, 0), form);
mark(&mut doc, 0, &[[0.0, 200.0, 300.0, 300.0]], "");
let r = apply(&mut doc, None).unwrap();
assert_eq!(r.images_removed, 1, "{r:?}");
let saved = Document::open(Arc::new(write_full(&doc, &SaveOptions::default()).unwrap())).unwrap();
assert_eq!(named_image(&saved, 0, b"Im1").as_deref(), Some(&b"PUBIMGSHOWNAGAIN"[..]), "the form still draws it");
}
/// A surviving form outside the mark shares the page's resources and draws the image itself, its
/// content behind a predictor whose `/DecodeParms` are given by reference: unlinking the redacted
/// image from the forms' resources reads it through them, and keeps the image the form draws.
#[test]
fn an_image_a_surviving_form_with_parameters_by_reference_draws_is_kept() {
let mut doc = pdf(vec![
b"<< /Type /Catalog /Pages 2 0 R >>".to_vec(),
b"<< /Type /Pages /Kids [3 0 R] /Count 1 >>".to_vec(),
b"<< /Type /Page /Parent 2 0 R /MediaBox [0 0 300 300] /Contents 4 0 R /Resources 5 0 R >>".to_vec(),
stream("", b"q 100 0 0 100 0 0 cm /Im Do Q /Fm Do"),
b"<< /XObject << /Im 6 0 R /Fm 7 0 R >> >>".to_vec(),
stream("/Type /XObject /Subtype /Image /Width 11 /Height 1 /ColorSpace /DeviceGray /BitsPerComponent 8", b"IMAGESECRET"),
stream("/Type /XObject /Subtype /Form /BBox [150 150 300 300]", b""),
]);
let mut entries = form_entries([150, 150, 300, 300]);
entries.push(("Resources", Object::Ref(ObjRef::new(5, 0))));
let form = behind_parms_by_reference(&mut doc, &entries, b"q 50 0 0 50 200 200 cm /Im Do Q");
doc.set(ObjRef::new(7, 0), form);
mark(&mut doc, 0, &[[0.0, 0.0, 120.0, 120.0]], "");
assert_eq!(apply(&mut doc, None).unwrap().images_removed, 1);
assert!(surviving_references::full_save_holds(&doc, b"IMAGESECRET"), "the image the form still draws outside the mark");
}
/// A form without resources of its own crosses the mark and draws an image, or text, under it,
/// its content behind a predictor whose `/DecodeParms` are given by reference. The walk reads it
/// through them and removes what lies under the mark: the image, whether the page draws it under
/// the mark too or only the form does, and the text's glyphs. Read without them, the form looked
/// empty and was left as it was, still drawing them, and the page's check, which reads it with
/// them, kept the image's name for it.
#[test]
fn what_a_form_with_parameters_by_reference_draws_under_the_mark_is_removed() {
let mut doc = one_page(b"/FmBare Do", "/XObject << /FmBare 7 0 R >>", vec![stream("/Type /XObject /Subtype /Form /BBox [0 0 100 100]", b"")]);
let mut entries = form_entries([0, 0, 100, 100]);
// Its drawing moved up by 210: the text lands at y 250, under the mark.
entries.push(("Matrix", Object::Array([1, 0, 0, 1, 0, 210].into_iter().map(Object::Int).collect())));
let form = behind_parms_by_reference(&mut doc, &entries, b"BT /F1 10 Tf 20 40 Td (SECRETVALUE) Tj ET");
doc.set(ObjRef::new(7, 0), form);
mark(&mut doc, 0, &[[0.0, 200.0, 300.0, 300.0]], "");
let r = apply(&mut doc, None).unwrap();
assert_eq!((r.glyphs, r.forms_rewritten), (11, 1), "the text the form draws under the mark: {r:?}");
for page in [&b"q 50 0 0 50 20 230 cm /Im1 Do Q /FmBare Do"[..], b"/FmBare Do"] {
let mut doc = one_page(
page,
"/XObject << /FmBare 7 0 R /Im1 8 0 R >>",
vec![
stream("/Type /XObject /Subtype /Form /BBox [0 0 100 100]", b""),
stream("/Type /XObject /Subtype /Image /Width 16 /Height 1 /ColorSpace /DeviceGray /BitsPerComponent 8", b"REDIMGONECOVERED"),
],
);
let mut entries = form_entries([0, 0, 100, 100]);
// Its drawing moved up by 210: the image lands at y 230 to 280, under the mark.
entries.push(("Matrix", Object::Array([1, 0, 0, 1, 0, 210].into_iter().map(Object::Int).collect())));
let form = behind_parms_by_reference(&mut doc, &entries, b"q 50 0 0 50 20 20 cm /Im1 Do Q");
doc.set(ObjRef::new(7, 0), form);
mark(&mut doc, 0, &[[0.0, 200.0, 300.0, 300.0]], "");
let r = apply(&mut doc, None).unwrap();
assert_eq!(r.forms_rewritten, 1, "{}: {r:?}", String::from_utf8_lossy(page));
let bytes = write_full(&doc, &SaveOptions::default()).unwrap();
assert!(!bytes.windows(16).any(|w| w == b"REDIMGONECOVERED"), "{}: the image the form drew under the mark", String::from_utf8_lossy(page));
}
}
/// A form without resources of its own that the redaction rewrites gets a new name. It keeps none
/// of its own either when its copy of the page's loses nothing, which takes a form drawing itself
/// (here off the page, so outside the mark), and then goes on drawing through the page's: itself
/// and, through a form it draws unchanged, the image under the mark again outside it. The page's
/// check of what it still draws follows the rewritten form under its new name, so the page keeps
/// both names for it.
#[test]
fn an_image_a_rewritten_form_draws_again_through_the_pages_resources_is_kept() {
let mut doc = one_page(
b"q 50 0 0 50 20 230 cm /Im1 Do Q /FmBare Do",
"/XObject << /FmBare 7 0 R /Inner 8 0 R /Im1 9 0 R >>",
vec![
stream(
"/Type /XObject /Subtype /Form /BBox [0 0 300 300]",
b"BT /F1 10 Tf 20 250 Td (SECRET) Tj ET /Inner Do q 1 0 0 1 0 -1000 cm /FmBare Do Q",
),
stream("/Type /XObject /Subtype /Form /BBox [0 0 100 100]", b"q 50 0 0 50 20 20 cm /Im1 Do Q"),
stream("/Type /XObject /Subtype /Image /Width 16 /Height 1 /ColorSpace /DeviceGray /BitsPerComponent 8", b"PUBIMGSHOWNAGAIN"),
],
);
mark(&mut doc, 0, &[[0.0, 200.0, 300.0, 300.0]], "");
let r = apply(&mut doc, None).unwrap();
assert_eq!((r.images_removed, r.forms_rewritten), (1, 1), "{r:?}");
let saved = Document::open(Arc::new(write_full(&doc, &SaveOptions::default()).unwrap())).unwrap();
let page = content(&saved, 0);
let drawn: Vec<Vec<u8>> =
pdfcraft_content::parse(page.as_bytes()).ops.iter().filter(|op| op.is("Do")).filter_map(|op| op.last_name().map(<[u8]>::to_vec)).collect();
assert!(drawn.len() == 1 && drawn[0] != b"FmBare", "the form is drawn under its new name: {page}");
let rewritten = named_stream(&saved, 0, &drawn[0]).unwrap();
assert!(!rewritten.dict.contains(b"Resources"), "the rewritten form keeps no resources of its own");
assert!(named_stream(&saved, 0, b"FmBare").is_some(), "the rewritten form still draws the original");
assert_eq!(named_image(&saved, 0, b"Im1").as_deref(), Some(&b"PUBIMGSHOWNAGAIN"[..]), "the unchanged form still draws it");
}
/// The walk of what content draws through its resources charges the forms it reads to the
/// allowance it is given: a form decoding to more than is left is refused, one within it read.
#[test]
fn what_content_draws_through_its_resources_is_read_within_its_allowance() {
let doc = pdf(vec![
b"<< /Type /Catalog /Pages 2 0 R >>".to_vec(),
b"<< /Type /Pages /Kids [] /Count 0 >>".to_vec(),
stream("/Type /XObject /Subtype /Form /BBox [0 0 1 1]", b"n ".repeat(1000)),
]);
let res = Dict::from_iter([(b"XObject".to_vec(), Object::Dict(Dict::from_iter([(b"Fm".to_vec(), Object::Ref(ObjRef::new(3, 0)))])))]);
for (allowance, read) in [(2000, true), (1999, false)] {
let mut budget = pdfcraft_cos::DecodeBudget::new(allowance, 0);
assert_eq!(crate::interp::resource_ops(&doc, &mut budget, &res, &[b"/Fm Do".to_vec()]).is_ok(), read, "{allowance} bytes");
}
}
/// An unused Type 3 font, listed as /T3 beside /F1 in the `/Font` of `holder`'s resources (a page's
/// or a form's), so that the font cleanup, which prunes only Type 3 fonts, charges its read of what
/// they draw to the page's allowance (without one it has an allowance of its own).
fn list_type3(doc: &mut Document, holder: ObjRef) {
let font = Object::Dict(Dict::from_iter([
(b"Type".to_vec(), Object::name("Font")),
(b"Subtype".to_vec(), Object::name("Type3")),
(b"FontBBox".to_vec(), Object::Array(vec![0.into(), 0.into(), 1000.into(), 1000.into()])),
(b"FontMatrix".to_vec(), Object::Array(vec![Object::Real(0.001), 0.into(), 0.into(), Object::Real(0.001), 0.into(), 0.into()])),
(b"CharProcs".to_vec(), Object::Dict(Dict::new())),
]));
let font = Object::Ref(doc.add(font));
let list = |d: &mut Dict| {
let Some(Object::Dict(res)) = d.get_mut(b"Resources") else { panic!("{holder:?} has resources") };
let Some(Object::Dict(fonts)) = res.get_mut(b"Font") else { panic!("{holder:?} lists fonts") };
fonts.set(b"T3".to_vec(), font);
};
match &*doc.get(holder) {
Object::Stream(s) => {
let mut s = s.clone();
list(&mut s.dict);
doc.set(holder, Object::Stream(s));
}
_ => doc.update_dict(holder, list).unwrap(),
}
}
/// The cleanups after the content walk spend what the walk left of the page's allowance, one
/// allowance between them. A form without resources of its own that crosses the mark is read by
/// the walk, then again by the check of what the page still draws and by the font cleanup, which
/// charges the page's allowance when the page lists a Type 3 font: at 100 MiB, three reads are more
/// than the page's 256 MiB, and the redaction is refused; at 50 MiB it goes on. With no Type 3 font
/// the font cleanup's read has an allowance of its own, so the page's pays for two reads of
/// 100 MiB, and the redaction goes on.
#[test]
fn the_cleanups_after_the_content_walk_spend_what_it_left_of_the_pages_allowance() {
for (mib, type3, refused) in [(100, true, true), (50, true, false), (100, false, false)] {
let blank = pdfcraft_filters::encode_flate(&vec![b' '; mib << 20]);
let mut doc = one_page(
b"q 50 0 0 50 20 230 cm /Im1 Do Q /FmBare Do",
"/XObject << /FmBare 7 0 R /Im1 8 0 R >>",
vec![
stream("/Type /XObject /Subtype /Form /BBox [0 0 300 300] /Filter /FlateDecode", &blank),
stream("/Type /XObject /Subtype /Image /Width 1 /Height 1 /ColorSpace /DeviceGray /BitsPerComponent 8", b"o"),
],
);
if type3 {
list_type3(&mut doc, ObjRef::new(3, 0));
}
mark(&mut doc, 0, &[[0.0, 200.0, 300.0, 300.0]], "");
assert_eq!(apply(&mut doc, None).is_err(), refused, "{mib} MiB, Type 3: {type3}");
}
}
/// Text under the mark in a content stream that holds bytes the parser can't read, inside a `TJ`
/// array, is read past by the walk and would stay. The font cleanup's read of what the content
/// draws refuses such bytes in the streams and forms the walk leaves as they were, and makes that
/// read with any font listed, a Type 3 font or not (with none, on an allowance of its own). Here
/// the walk rewrites the text in the page's first stream and leaves the broken `TJ` as it was:
/// Apply Redactions refuses the page, whether that `TJ` is in the page's second stream or in a
/// form without resources of its own, and so does Remove Hidden Information. The cleanups run only
/// when the walk rewrites some of the page, and read what it rewrote as rewritten, so such bytes in
/// a stream it rewrites still go by, and so do those on a page it rewrites nothing of, unless the
/// document still lists a Type 3 font (`unlink_type3` then reads every page).
#[test]
fn text_beside_bytes_the_parser_cannot_read_refuses_the_redaction() {
use crate::sanitize::{Hidden, remove_hidden};
let page_of = |second: &[u8]| {
pdf(vec![
b"<< /Type /Catalog /Pages 2 0 R >>".to_vec(),
b"<< /Type /Pages /Kids [3 0 R] /Count 1 >>".to_vec(),
b"<< /Type /Page /Parent 2 0 R /MediaBox [0 0 300 300] /Contents [4 0 R 5 0 R] /Resources << /Font << /F1 6 0 R >> >> >>".to_vec(),
stream("", b"BT /F1 10 Tf 20 230 Td (SECRET) Tj ET"),
stream("", second),
FONT.replace("95 0 R", "7 0 R").into_bytes(),
widths(),
])
};
for tail in [
&b"BT /F1 10 Tf 20 260 Td [(TAIL) ) (SECRET)] TJ ET"[..],
b"BT /F1 10 Tf 20 260 Td [(TAIL) } (SECRET)] TJ ET",
b"BT /F1 10 Tf 20 260 Td [(TAILSECRET) )] TJ ET",
b"BT /F1 10 Tf 20 260 Td [) (TAILSECRET)] TJ ET",
] {
let mut doc = page_of(tail);
mark(&mut doc, 0, &[[0.0, 200.0, 300.0, 300.0]], "");
assert!(apply(&mut doc, None).is_err(), "{}", String::from_utf8_lossy(tail));
}
let mut doc = one_page(
b"BT /F1 10 Tf 20 230 Td (SECRET) Tj ET /FmBare Do",
"/XObject << /FmBare 7 0 R >>",
vec![stream("/Type /XObject /Subtype /Form /BBox [0 0 300 300]", b"BT /F1 10 Tf 20 260 Td [(TAIL) ) (SECRET)] TJ ET")],
);
mark(&mut doc, 0, &[[0.0, 200.0, 300.0, 300.0]], "");
assert!(apply(&mut doc, None).is_err(), "a form's text");
let mut doc = page_of(b"BT /F1 10 Tf 3 Tr 20 260 Td [(TAIL) ) (HIDDEN)] TJ ET");
doc.set(ObjRef::new(4, 0), Object::Stream(Stream::from_raw(Dict::new(), b"BT /F1 10 Tf 3 Tr 20 230 Td (HIDDEN) Tj ET".to_vec())));
assert!(remove_hidden(&mut doc, &[Hidden::HiddenText]).is_err(), "hidden text");
}
/// The cleanups of a form the redaction rewrites spend the page's allowance too. The rewritten
/// form has resources of its own, so the page's cleanups don't follow it, and draws through them a
/// form without any that crosses the mark: the walk reads that form, then the check of what the
/// rewritten form still draws and its font cleanup (when it lists a Type 3 font) read it again. At
/// 100 MiB, three reads are more than the page's 256 MiB, and the redaction is refused; at 50 MiB
/// it goes on. With no Type 3 font the font cleanup's read has an allowance of its own, so the
/// page's pays for two reads of 100 MiB, and the redaction goes on.
#[test]
fn the_cleanups_of_a_rewritten_form_spend_the_pages_allowance() {
for (mib, type3, refused) in [(100, true, true), (50, true, false), (100, false, false)] {
let blank = pdfcraft_filters::encode_flate(&vec![b' '; mib << 20]);
let mut doc = one_page(
b"/FmText Do",
"/XObject << /FmText 7 0 R >>",
vec![
stream(
"/Type /XObject /Subtype /Form /BBox [0 0 300 300] /Resources << /Font << /F1 5 0 R >> /XObject << /FmBare 8 0 R >> >>",
b"BT /F1 10 Tf 20 250 Td (SECRET) Tj ET /FmBare Do",
),
stream("/Type /XObject /Subtype /Form /BBox [0 0 300 300] /Filter /FlateDecode", &blank),
],
);
if type3 {
list_type3(&mut doc, ObjRef::new(7, 0));
}
mark(&mut doc, 0, &[[0.0, 200.0, 300.0, 300.0]], "");
assert_eq!(apply(&mut doc, None).is_err(), refused, "{mib} MiB, Type 3: {type3}");
}
}
/// Removing hidden text spends what its content walk left of the page's allowance too. The page
/// draws a form holding hidden text, which is rewritten, and a form without resources of its own:
/// the walk reads the second, then the check of what the page still draws and the font cleanup
/// (when the page lists a Type 3 font) read it again. At 100 MiB, three reads are more than the
/// page's 256 MiB, and the removal is refused; at 50 MiB it goes on. With no Type 3 font the font
/// cleanup's read has an allowance of its own, so the page's pays for two reads of 100 MiB, and the
/// removal goes on.
#[test]
fn removing_hidden_text_spends_what_the_content_walk_left_of_the_pages_allowance() {
use crate::sanitize::{Hidden, remove_hidden};
for (mib, type3, refused) in [(100, true, true), (50, true, false), (100, false, false)] {
let blank = pdfcraft_filters::encode_flate(&vec![b' '; mib << 20]);
let mut doc = one_page(
b"/FmHidden Do /FmBare Do",
"/XObject << /FmHidden 7 0 R /FmBare 8 0 R >>",
vec![
stream("/Type /XObject /Subtype /Form /BBox [0 0 300 300]", b"BT /F1 10 Tf 3 Tr 20 250 Td (HIDDEN) Tj ET"),
stream("/Type /XObject /Subtype /Form /BBox [0 0 300 300] /Filter /FlateDecode", &blank),
],
);
if type3 {
list_type3(&mut doc, ObjRef::new(3, 0));
}
assert_eq!(remove_hidden(&mut doc, &[Hidden::HiddenText]).is_err(), refused, "{mib} MiB, Type 3: {type3}");
}
}
/// A form with resources of its own draws through those, not its caller's: one on another page
/// that names a different image /Im1 doesn't keep the redacted /Im1 in the dictionary both pages
/// inherit.
#[test]
fn a_form_with_resources_of_its_own_does_not_keep_its_callers_image() {
let image = "/Type /XObject /Subtype /Image /Width 16 /Height 1 /ColorSpace /DeviceGray /BitsPerComponent 8";
let mut doc = pdf(vec![
b"<< /Type /Catalog /Pages 2 0 R >>".to_vec(),
b"<< /Type /Pages /Kids [3 0 R 5 0 R] /Count 2 /Resources << /XObject << /Im1 7 0 R /FmOwn 8 0 R >> >> >>".to_vec(),
b"<< /Type /Page /Parent 2 0 R /MediaBox [0 0 300 300] /Contents 4 0 R >>".to_vec(),
stream("", b"q 50 0 0 50 20 230 cm /Im1 Do Q"),
b"<< /Type /Page /Parent 2 0 R /MediaBox [0 0 300 300] /Contents 6 0 R >>".to_vec(),
stream("", b"/FmOwn Do"),
stream(image, b"REDIMGONECOVERED"),
stream("/Type /XObject /Subtype /Form /BBox [0 0 300 300] /Resources << /XObject << /Im1 9 0 R >> >>", b"q 50 0 0 50 20 20 cm /Im1 Do Q"),
stream(image, b"PUBIMGSHOWNAGAIN"),
]);
mark(&mut doc, 0, &[[0.0, 200.0, 300.0, 300.0]], "");
let r = apply(&mut doc, None).unwrap();
assert_eq!(r.images_removed, 1, "{r:?}");
let bytes = write_full(&doc, &SaveOptions::default()).unwrap();
assert!(!bytes.windows(16).any(|w| w == b"REDIMGONECOVERED"), "the redacted image survived the save");
let saved = Document::open(Arc::new(bytes)).unwrap();
let form = named_stream(&saved, 1, b"FmOwn").expect("page 2 still draws its form");
let res = saved.resolve(form.dict.get(b"Resources").unwrap()).as_dict().cloned().unwrap();
let xo = saved.resolve(res.get(b"XObject").unwrap()).as_dict().cloned().unwrap();
let Object::Stream(own) = &*saved.get(xo.get(b"Im1").and_then(Object::as_ref).unwrap()) else { panic!("/Im1 is a stream") };
assert_eq!(decoded_image(&saved, own).unwrap(), b"PUBIMGSHOWNAGAIN", "the form's own image stays");
}
/// Remove Hidden Information has the same rewrite: an off layer's image named by an inherited
/// /Resources must leave the saved file too.
#[test]
fn a_hidden_layers_image_does_not_survive_through_inherited_resources() {
use crate::sanitize::{Hidden, remove_hidden};
let mut doc = pdf(vec![
b"<< /Type /Catalog /Pages 2 0 R /OCProperties << /OCGs [6 0 R] /D << /OFF [6 0 R] >> >> >>".to_vec(),
b"<< /Type /Pages /Kids [3 0 R] /Count 1 /Resources << /XObject << /ImOC 5 0 R >> >> >>".to_vec(),
b"<< /Type /Page /Parent 2 0 R /MediaBox [0 0 300 300] /Contents 4 0 R >>".to_vec(),
stream("", b"q 50 0 0 50 20 20 cm /ImOC Do Q"),
stream("/Type /XObject /Subtype /Image /Width 16 /Height 1 /ColorSpace /DeviceGray /BitsPerComponent 8 /OC 6 0 R", b"HIDDENLAYERIMAGE"),
b"<< /Type /OCG /Name (Secret layer) >>".to_vec(),
]);
let done = remove_hidden(&mut doc, &[Hidden::HiddenLayers]).unwrap();
assert!(done.iter().any(|(h, n)| *h == Hidden::HiddenLayers && *n > 0), "{done:?}");
let bytes = write_full(&doc, &SaveOptions::default()).unwrap();
assert!(!bytes.windows(16).any(|w| w == b"HIDDENLAYERIMAGE"), "the off layer's image survived the save");
}
#[test]
fn hidden_ocmd_all_on_content_is_removed_before_its_off_group_is_unlinked() {
use crate::sanitize::{Hidden, remove_hidden};
let mut doc = one_page(
b"BT /F1 10 Tf 10 200 Td (PUBLIC) Tj ET /OC /M BDC BT /F1 10 Tf 10 100 Td (SECRET) Tj ET EMC",
"/Properties << /M 9 0 R >>",
vec![
b"<< /Type /OCG /Name (On) >>".to_vec(),
b"<< /Type /OCG /Name (Off) >>".to_vec(),
b"<< /Type /OCMD /OCGs [7 0 R 8 0 R] /P /AllOn >>".to_vec(),
],
);
let root = doc.root().unwrap();
doc.update_dict(root, |d| {
d.set(
b"OCProperties".to_vec(),
Object::Dict(pdfcraft_cos::Dict::from_iter([
(b"OCGs".to_vec(), Object::Array(vec![Object::Ref(ObjRef::new(7, 0)), Object::Ref(ObjRef::new(8, 0))])),
(
b"D".to_vec(),
Object::Dict(pdfcraft_cos::Dict::from_iter([
(b"Order".to_vec(), Object::Array(vec![Object::Ref(ObjRef::new(7, 0)), Object::Ref(ObjRef::new(8, 0))])),
(b"OFF".to_vec(), Object::Array(vec![Object::Ref(ObjRef::new(8, 0))])),
])),
),
])),
);
})
.unwrap();
let before = pdfcraft_edit::text_lines(&doc, 0).unwrap();
assert!(before.iter().any(|line| line.text.contains("PUBLIC")));
assert!(before.iter().any(|line| line.text.contains("SECRET")), "extractor sees source text before sanitization");
remove_hidden(&mut doc, &[Hidden::HiddenLayers]).unwrap();
let saved = Document::open(Arc::new(write_full(&doc, &SaveOptions::default()).unwrap())).unwrap();
let lines = pdfcraft_edit::text_lines(&saved, 0).unwrap();
assert!(lines.iter().any(|line| line.text.contains("PUBLIC")), "visible content remains: {lines:?}");
assert!(!lines.iter().any(|line| line.text.contains("SECRET")), "hidden text is extractable: {lines:?}");
assert!(!write_full(&saved, &SaveOptions::default()).unwrap().windows(6).any(|w| w == b"SECRET"));
}
#[test]
fn hidden_ocmd_content_in_a_form_and_nested_marked_blocks_is_removed() {
use crate::sanitize::{Hidden, remove_hidden};
let mut doc = one_page(
b"/Fm Do",
"/Properties << /M 9 0 R >> /XObject << /Fm 10 0 R >>",
vec![
b"<< /Type /OCG /Name (On) >>".to_vec(),
b"<< /Type /OCG /Name (Off) >>".to_vec(),
b"<< /Type /OCMD /OCGs [7 0 R 8 0 R] /P /AllOn >>".to_vec(),
stream(
"/Type /XObject /Subtype /Form /BBox [0 0 300 300] /Resources << /Font << /F1 5 0 R >> /Properties << /M 9 0 R >> >>",
b"/OC /M BDC /OC /M BDC BT /F1 10 Tf 10 100 Td (SECRET) Tj ET EMC EMC",
),
],
);
let root = doc.root().unwrap();
doc.update_dict(root, |d| {
d.set(
b"OCProperties".to_vec(),
Object::Dict(Dict::from_iter([
(b"OCGs".to_vec(), Object::Array(vec![Object::Ref(ObjRef::new(7, 0)), Object::Ref(ObjRef::new(8, 0))])),
(b"D".to_vec(), Object::Dict(Dict::from_iter([(b"OFF".to_vec(), Object::Array(vec![Object::Ref(ObjRef::new(8, 0))]))]))),
])),
);
})
.unwrap();
remove_hidden(&mut doc, &[Hidden::HiddenLayers]).unwrap();
let saved = Document::open(Arc::new(write_full(&doc, &SaveOptions::default()).unwrap())).unwrap();
assert!(!pdfcraft_edit::text_lines(&saved, 0).unwrap().iter().any(|line| line.text.contains("SECRET")));
}
#[test]
fn hidden_ocmd_alloff_is_sanitized_when_every_group_is_on() {
use crate::sanitize::{Hidden, remove_hidden, scan};
let mut doc = one_page(
b"/OC /M BDC BT /F1 10 Tf 10 100 Td (SECRET) Tj ET EMC",
"/Properties << /M 9 0 R >>",
vec![
b"<< /Type /OCG /Name (On) >>".to_vec(),
b"<< /Type /OCG /Name (Unused) >>".to_vec(),
b"<< /Type /OCMD /OCGs [7 0 R] /P /AllOff >>".to_vec(),
],
);
let counts = scan(&doc);
assert!(counts.iter().any(|(h, n)| *h == Hidden::HiddenLayers && *n > 0), "{counts:?}");
remove_hidden(&mut doc, &[Hidden::HiddenLayers]).unwrap();
let saved = Document::open(Arc::new(write_full(&doc, &SaveOptions::default()).unwrap())).unwrap();
assert!(!pdfcraft_edit::text_lines(&saved, 0).unwrap().iter().any(|line| line.text.contains("SECRET")));
}
#[test]
fn hidden_ve_not_is_sanitized_when_its_group_is_on() {
use crate::sanitize::{Hidden, remove_hidden, scan};
let mut doc = one_page(
b"/OC /M BDC BT /F1 10 Tf 10 100 Td (SECRET) Tj ET EMC",
"/Properties << /M 9 0 R >>",
vec![b"<< /Type /OCG /Name (On) >>".to_vec(), b"<< /Type /OCG /Name (Unused) >>".to_vec(), b"<< /Type /OCMD /VE [/Not 7 0 R] >>".to_vec()],
);
let counts = scan(&doc);
assert!(counts.iter().any(|(h, n)| *h == Hidden::HiddenLayers && *n > 0), "{counts:?}");
remove_hidden(&mut doc, &[Hidden::HiddenLayers]).unwrap();
let saved = Document::open(Arc::new(write_full(&doc, &SaveOptions::default()).unwrap())).unwrap();
assert!(!pdfcraft_edit::text_lines(&saved, 0).unwrap().iter().any(|line| line.text.contains("SECRET")));
}
fn all_on_ocmd_xobject(subtype: &str, policy: &str) -> Document {
let membership = match policy {
"AllOff" => "<< /Type /OCMD /OCGs [7 0 R] /P /AllOff >>",
"Not" => "<< /Type /OCMD /VE [/Not 7 0 R] >>",
_ => "<< /Type /OCMD /OCGs [7 0 R] /P /AllOff >>",
};
let (object_number, oc) = if policy == "DirectAllOff" { (8, membership) } else { (9, "8 0 R") };
let xobject = match subtype {
"Image" => stream(
&format!("/Type /XObject /Subtype /Image /Width 4 /Height 1 /ColorSpace /DeviceGray /BitsPerComponent 8 /OC {oc}"),
b"PRIVATE_HIDDEN_XOBJECT_IMAGE",
),
_ => stream(
&format!("/Type /XObject /Subtype /Form /BBox [0 0 300 300] /Resources << /Font << /F1 5 0 R >> >> /OC {oc}"),
b"BT /F1 10 Tf 10 100 Td (PRIVATE_HIDDEN_XOBJECT_FORM) Tj ET",
),
};
let mut extras = vec![b"<< /Type /OCG /Name (On) >>".to_vec()];
if object_number == 9 {
extras.push(membership.as_bytes().to_vec());
}
extras.push(xobject);
let mut doc = one_page(b"/X Do", &format!("/XObject << /X {object_number} 0 R >>"), extras);
let root = doc.root().unwrap();
doc.update_dict(root, |d| {
d.set(
b"OCProperties".to_vec(),
Object::Dict(Dict::from_iter([
(b"OCGs".to_vec(), Object::Array(vec![Object::Ref(ObjRef::new(7, 0))])),
(b"D".to_vec(), Object::Dict(Dict::from_iter([(b"BaseState".to_vec(), Object::name("ON"))]))),
])),
);
})
.unwrap();
doc
}
#[test]
fn hidden_all_on_ocmd_image_and_form_xobjects_are_counted_and_removed() {
use crate::sanitize::{Hidden, remove_hidden, scan};
let mut failures = Vec::new();
for subtype in ["Image", "Form"] {
for policy in ["AllOff", "Not"] {
let mut doc = all_on_ocmd_xobject(subtype, policy);
let before = scan(&doc);
let hidden_count = before.iter().find(|(h, _)| *h == Hidden::HiddenLayers).map_or(0, |(_, n)| *n);
if hidden_count == 0 {
failures.push(format!("{subtype} /{policy} was not counted: {before:?}"));
}
match remove_hidden(&mut doc, &[Hidden::HiddenLayers]) {
Ok(removed) if removed.iter().any(|(h, n)| *h == Hidden::HiddenLayers && *n > 0) => {}
Ok(removed) => failures.push(format!("{subtype} /{policy} remove report is empty: {removed:?}")),
Err(error) => failures.push(format!("{subtype} /{policy} remove failed: {error}")),
}
let bytes = write_full(&doc, &SaveOptions::default()).unwrap();
if bytes.windows(b"PRIVATE_HIDDEN_XOBJECT_IMAGE".len()).any(|w| w == b"PRIVATE_HIDDEN_XOBJECT_IMAGE") {
failures.push(format!("{subtype} /{policy} image payload remains in full save"));
}
if bytes.windows(b"PRIVATE_HIDDEN_XOBJECT_FORM".len()).any(|w| w == b"PRIVATE_HIDDEN_XOBJECT_FORM") {
failures.push(format!("{subtype} /{policy} Form payload remains in full save"));
}
let saved = Document::open(Arc::new(bytes)).unwrap();
if pdfcraft_edit::text_lines(&saved, 0).unwrap().iter().any(|line| line.text.contains("PRIVATE_HIDDEN_XOBJECT_FORM")) {
failures.push(format!("{subtype} /{policy} Form text remains extractable"));
}
if !pdfcraft_edit::page_images(&saved, 0).unwrap().is_empty() {
failures.push(format!("{subtype} /{policy} image remains on the page"));
}
}
}
assert!(failures.is_empty(), "{}", failures.join("\n"));
}
#[test]
fn repeated_image_with_direct_alloff_membership_is_counted_per_draw() {
use crate::sanitize::{Hidden, scan};
let mut doc = all_on_ocmd_xobject("Image", "DirectAllOff");
let page = pdfcraft_model::pages(&doc)[0].obj;
let contents = doc.add(Object::Stream(pdfcraft_cos::Stream::flate(Dict::new(), b"/X Do /X Do /X Do")));
doc.update_dict(page, |d| d.set(b"Contents".to_vec(), Object::Ref(contents))).unwrap();
let original = write_full(&doc, &SaveOptions::default()).unwrap();
let counts = scan(&doc);
assert_eq!(counts.iter().find(|(h, _)| *h == Hidden::HiddenLayers).map(|(_, n)| *n), Some(3), "{counts:?}");
assert_eq!(write_full(&doc, &SaveOptions::default()).unwrap(), original, "Verify scan must not mutate the document");
}
#[test]
fn undecodable_form_is_counted_and_removed_by_hidden_layer_sanitization() {
use crate::sanitize::{Hidden, remove_hidden, scan};
let mut doc = one_page(
b"/Bad Do",
"/XObject << /Bad 7 0 R >>",
vec![stream("/Type /XObject /Subtype /Form /BBox [0 0 300 300] /Filter /UnknownFilter", b"PRIVATE_UNDECODABLE_FORM")],
);
let counts = scan(&doc);
assert!(counts.iter().any(|(h, n)| *h == Hidden::HiddenLayers && *n > 0), "{counts:?}");
remove_hidden(&mut doc, &[Hidden::HiddenLayers]).unwrap();
let bytes = write_full(&doc, &SaveOptions::default()).unwrap();
assert!(!bytes.windows(b"PRIVATE_UNDECODABLE_FORM".len()).any(|w| w == b"PRIVATE_UNDECODABLE_FORM"));
}
#[test]
fn form_chain_at_max_depth_is_counted_and_removed_by_hidden_layer_sanitization() {
use crate::sanitize::{Hidden, remove_hidden, scan};
const PRIVATE: &[u8] = b"PRIVATE_FORM_BEYOND_MAX_DEPTH";
// The final Form is encountered at MAX_DEPTH and is removed conservatively without being
// entered. Verify must count that removal or sanitize can skip the HiddenLayers pass entirely.
let max = crate::interp::MAX_DEPTH;
let mut forms = Vec::new();
for depth in 0..=max {
let content = if depth == max {
b"BT /F1 10 Tf 10 100 Td (PRIVATE_FORM_BEYOND_MAX_DEPTH) Tj ET".to_vec()
} else {
format!("/F{} Do", depth + 1).into_bytes()
};
forms.push(stream("/Type /XObject /Subtype /Form /BBox [0 0 300 300]", &content));
}
let xobjects = (0..=max).map(|depth| format!("/F{depth} {} 0 R", 7 + depth)).collect::<Vec<_>>().join(" ");
let mut doc = one_page(b"/F0 Do", &format!("/XObject << {xobjects} >>"), forms);
assert!(
write_full(&doc, &SaveOptions::default()).unwrap().windows(PRIVATE.len()).any(|w| w == PRIVATE),
"fixture lacked nested payload before sanitization"
);
let counts = scan(&doc);
assert!(counts.iter().any(|(hidden, count)| *hidden == Hidden::HiddenLayers && *count > 0), "depth-limited Form was not counted: {counts:?}");
let removed = remove_hidden(&mut doc, &[Hidden::HiddenLayers]).unwrap();
assert!(
removed.iter().any(|(hidden, count)| *hidden == Hidden::HiddenLayers && *count > 0),
"depth-limited Form was not reported removed: {removed:?}"
);
let bytes = write_full(&doc, &SaveOptions::default()).unwrap();
assert!(!bytes.windows(PRIVATE.len()).any(|w| w == PRIVATE), "depth-limited Form payload remains in full save");
}
#[test]
fn hidden_ocmd_resolves_an_indirect_policy_name() {
use crate::sanitize::{Hidden, remove_hidden};
let mut doc = one_page(
b"/OC /M BDC BT /F1 10 Tf 10 100 Td (VISIBLE) Tj ET EMC",
"/Properties << /M 9 0 R >>",
vec![
b"<< /Type /OCG /Name (On) >>".to_vec(),
b"<< /Type /OCG /Name (Off) >>".to_vec(),
b"<< /Type /OCMD /OCGs [7 0 R 8 0 R] /P 12 0 R >>".to_vec(),
],
);
doc.set(ObjRef::new(12, 0), Object::name("AnyOn"));
let root = doc.root().unwrap();
doc.update_dict(root, |d| {
d.set(
b"OCProperties".to_vec(),
Object::Dict(Dict::from_iter([
(b"OCGs".to_vec(), Object::Array(vec![Object::Ref(ObjRef::new(7, 0)), Object::Ref(ObjRef::new(8, 0))])),
(b"D".to_vec(), Object::Dict(Dict::from_iter([(b"OFF".to_vec(), Object::Array(vec![Object::Ref(ObjRef::new(8, 0))]))]))),
])),
);
})
.unwrap();
remove_hidden(&mut doc, &[Hidden::HiddenLayers]).unwrap();
let saved = Document::open(Arc::new(write_full(&doc, &SaveOptions::default()).unwrap())).unwrap();
assert!(pdfcraft_edit::text_lines(&saved, 0).unwrap().iter().any(|line| line.text.contains("VISIBLE")));
}
#[test]
fn surviving_anyoff_and_not_content_keeps_its_off_group_state() {
use crate::sanitize::{Hidden, remove_hidden};
for use_not in [false, true] {
let mut doc = one_page(
b"/OC /M BDC BT /F1 10 Tf 10 100 Td (VISIBLE) Tj ET EMC",
"/Properties << /M 9 0 R >>",
vec![
b"<< /Type /OCG /Name (On) >>".to_vec(),
b"<< /Type /OCG /Name (Off) >>".to_vec(),
if use_not { b"<< /Type /OCMD /VE [/Not 8 0 R] >>".to_vec() } else { b"<< /Type /OCMD /OCGs [8 0 R] /P /AnyOff >>".to_vec() },
],
);
let root = doc.root().unwrap();
doc.update_dict(root, |d| {
d.set(
b"OCProperties".to_vec(),
Object::Dict(Dict::from_iter([
(b"OCGs".to_vec(), Object::Array(vec![Object::Ref(ObjRef::new(7, 0)), Object::Ref(ObjRef::new(8, 0))])),
(b"D".to_vec(), Object::Dict(Dict::from_iter([(b"OFF".to_vec(), Object::Array(vec![Object::Ref(ObjRef::new(8, 0))]))]))),
])),
);
})
.unwrap();
remove_hidden(&mut doc, &[Hidden::HiddenLayers]).unwrap();
let root = doc.root().unwrap();
let catalog = doc.get(root);
let oc = catalog.as_dict().and_then(|d| d.get(b"OCProperties")).map(|o| doc.resolve(o));
let default = oc.as_deref().and_then(|o| o.as_dict()).and_then(|d| d.get(b"D")).map(|o| doc.resolve(o));
let off = default.as_deref().and_then(|o| o.as_dict()).and_then(|d| d.get(b"OFF")).map(|o| doc.resolve(o));
assert!(
off.as_deref().and_then(Object::as_array).is_some_and(|a| a.iter().any(|o| o.as_ref() == Some(ObjRef::new(8, 0)))),
"off group lost for /VE Not={use_not}"
);
let saved = Document::open(Arc::new(write_full(&doc, &SaveOptions::default()).unwrap())).unwrap();
assert!(pdfcraft_edit::text_lines(&saved, 0).unwrap().iter().any(|line| line.text.contains("VISIBLE")));
}
}
#[test]
fn hidden_ocmd_policy_matches_the_default_configuration() {
use crate::sanitize::{Hidden, remove_hidden};
let cases = [
(None, false, None, false),
(Some("AnyOn"), false, None, false),
(Some("AllOn"), false, None, true),
(Some("AnyOff"), false, None, false),
(Some("AllOff"), false, None, true),
(Some("UnknownPolicy"), false, None, true),
(None, true, None, true),
(None, false, Some("and"), true),
(None, false, Some("and_single_not"), true),
(None, false, Some("and_single_off"), true),
(None, false, Some("or_single_on"), false),
(None, false, Some("or_single_off"), true),
(Some("AllOn"), false, Some("not_off"), false),
(None, false, Some("malformed_not"), true),
(None, false, Some("malformed_without_ocgs"), true),
// `/VE` names the hidden group through an alias object (10 0 R is a reference to 8 0 R).
(None, false, Some("alias"), true),
];
for (policy, malformed_policy, visibility, should_remove) in cases {
let mut doc = one_page(
b"/OC /M BDC BT /F1 10 Tf 10 100 Td (SECRET) Tj ET EMC",
"/Properties << /M 9 0 R >>",
vec![
b"<< /Type /OCG /Name (On) >>".to_vec(),
b"<< /Type /OCG /Name (Off) >>".to_vec(),
b"<< /Type /OCMD /OCGs [7 0 R 8 0 R] /P /AllOn >>".to_vec(),
],
);
let mut membership = Dict::new();
membership.set(b"Type".to_vec(), Object::name("OCMD"));
if visibility != Some("malformed_without_ocgs") {
membership.set(b"OCGs".to_vec(), Object::Array(vec![Object::Ref(ObjRef::new(7, 0)), Object::Ref(ObjRef::new(8, 0))]));
}
if let Some(policy) = policy {
membership.set(b"P".to_vec(), Object::name(policy));
} else if malformed_policy {
membership.set(b"P".to_vec(), Object::Int(17));
}
match visibility {
Some("and") => membership
.set(b"VE".to_vec(), Object::Array(vec![Object::name("And"), Object::Ref(ObjRef::new(7, 0)), Object::Ref(ObjRef::new(8, 0))])),
Some("and_single_not") => membership.set(
b"VE".to_vec(),
Object::Array(vec![Object::name("Not"), Object::Array(vec![Object::name("And"), Object::Ref(ObjRef::new(7, 0))])]),
),
Some("and_single_off") => membership.set(b"VE".to_vec(), Object::Array(vec![Object::name("And"), Object::Ref(ObjRef::new(8, 0))])),
Some("or_single_on") => membership.set(b"VE".to_vec(), Object::Array(vec![Object::name("Or"), Object::Ref(ObjRef::new(7, 0))])),
Some("or_single_off") => membership.set(b"VE".to_vec(), Object::Array(vec![Object::name("Or"), Object::Ref(ObjRef::new(8, 0))])),
Some("not_off") => membership.set(b"VE".to_vec(), Object::Array(vec![Object::name("Not"), Object::Ref(ObjRef::new(8, 0))])),
Some("malformed_not") => membership
.set(b"VE".to_vec(), Object::Array(vec![Object::name("Not"), Object::Ref(ObjRef::new(7, 0)), Object::Ref(ObjRef::new(8, 0))])),
Some("malformed_without_ocgs") => {
membership.set(b"VE".to_vec(), Object::Array(vec![Object::name("UnknownOperator"), Object::Ref(ObjRef::new(8, 0))]))
}
Some("alias") => membership.set(b"VE".to_vec(), Object::Array(vec![Object::name("Or"), Object::Ref(ObjRef::new(10, 0))])),
_ => {}
}
doc.set(ObjRef::new(9, 0), Object::Dict(membership));
doc.set(ObjRef::new(10, 0), Object::Ref(ObjRef::new(8, 0)));
let root = doc.root().unwrap();
doc.update_dict(root, |d| {
d.set(
b"OCProperties".to_vec(),
Object::Dict(Dict::from_iter([
(b"OCGs".to_vec(), Object::Array(vec![Object::Ref(ObjRef::new(7, 0)), Object::Ref(ObjRef::new(8, 0))])),
(
b"D".to_vec(),
Object::Dict(Dict::from_iter([
(b"Order".to_vec(), Object::Array(vec![Object::Ref(ObjRef::new(7, 0)), Object::Ref(ObjRef::new(8, 0))])),
(b"OFF".to_vec(), Object::Array(vec![Object::Ref(ObjRef::new(8, 0))])),
])),
),
])),
);
})
.unwrap();
remove_hidden(&mut doc, &[Hidden::HiddenLayers]).unwrap();
let saved = Document::open(Arc::new(write_full(&doc, &SaveOptions::default()).unwrap())).unwrap();
let has_secret = pdfcraft_edit::text_lines(&saved, 0).unwrap().iter().any(|line| line.text.contains("SECRET"));
assert_eq!(has_secret, !should_remove, "policy {policy:?}, malformed={malformed_policy}, VE={visibility:?}");
}
}
#[test]
fn redacting_an_xfa_field_drops_the_xfa_packet_so_its_value_cannot_survive() {
// A static AcroForm+XFA form whose field value lives both in the widget /V and in the XFA
// datasets packet (Designer-style /XFA array). Redacting the widget must delete the field and
// stop the form being XFA, so neither the saved file nor a later relayout keeps the value.
let objs: Vec<Vec<u8>> = vec![
b"<< /Type /Catalog /Pages 2 0 R /NeedsRendering true /AcroForm 7 0 R >>".to_vec(),
b"<< /Type /Pages /Kids [3 0 R] /Count 1 >>".to_vec(),
b"<< /Type /Page /Parent 2 0 R /MediaBox [0 0 300 300] /Contents 4 0 R /Resources << /Font << /F1 5 0 R >> >> /Annots [8 0 R] >>".to_vec(),
stream("", b"BT ET"),
FONT.replace("95 0 R", "6 0 R").into_bytes(),
widths(),
b"<< /Fields [8 0 R] /XFA 9 0 R /PCXfaLayout << /Pages 1 /Fields 1 >> >>".to_vec(),
b"<< /Type /Annot /Subtype /Widget /FT /Tx /T (ssn) /V (SSN-987-65-4321) /Rect [72 200 272 220] /P 3 0 R >>".to_vec(),
b"[(datasets) 10 0 R (template) 11 0 R]".to_vec(),
stream(
"",
b"<xfa:datasets xmlns:xfa=\"http://www.xfa.org/schema/xfa-data/1.0/\"><xfa:data><ssn>SSN-987-65-4321</ssn></xfa:data></xfa:datasets>",
),
stream(
"",
b"<template xmlns=\"http://www.xfa.org/schema/xfa-template/3.3/\"><subform name=\"form1\"><field name=\"ssn\"/></subform></template>",
),
];
let mut doc = pdf(objs);
assert_eq!(pdfcraft_forms::fields(&doc).len(), 1, "the XFA-bound field is present before");
mark(&mut doc, 0, &[[72.0, 200.0, 272.0, 220.0]], "");
let r = apply(&mut doc, None).unwrap();
assert!(r.fields >= 1 && r.xfa_removed, "{r:?}");
let bytes = write_full(&doc, &SaveOptions::default()).unwrap();
assert!(!bytes.windows(15).any(|w| w == b"SSN-987-65-4321"), "the redacted value survived in the datasets packet");
let saved = Document::open(Arc::new(bytes)).unwrap();
let cat = saved.get(saved.root().unwrap()).as_dict().cloned().unwrap();
let acro = saved.resolve(cat.get(b"AcroForm").unwrap()).as_dict().cloned().unwrap();
assert!(acro.get(b"XFA").is_none(), "/XFA is gone from the saved AcroForm");
assert!(acro.get(b"PCXfaLayout").is_none(), "the cached layout mark is cleared");
assert!(cat.get(b"NeedsRendering").is_none(), "/NeedsRendering is cleared");
}
#[test]
fn replies_to_a_redacted_comment_go_with_it() {
// #2659: a highlight under the mark has a reply placed elsewhere, and that reply a reply of
// its own. The replies' /IRT would keep the highlight (its comment and QuadPoints) in the file.
let objs: Vec<Vec<u8>> = vec![
b"<< /Type /Catalog /Pages 2 0 R >>".to_vec(),
b"<< /Type /Pages /Kids [3 0 R] /Count 1 >>".to_vec(),
b"<< /Type /Page /Parent 2 0 R /MediaBox [0 0 300 300] /Contents 4 0 R /Annots [5 0 R 6 0 R 7 0 R 8 0 R] >>".to_vec(),
stream("", b"BT ET"),
b"<< /Type /Annot /Subtype /Highlight /Rect [20 20 60 30] /QuadPoints [20 30 60 30 20 20 60 20] /Contents (HIGHLIGHT-NOTE) >>".to_vec(),
b"<< /Type /Annot /Subtype /Text /Rect [250 250 270 270] /IRT 5 0 R /Contents (FIRST-REPLY) >>".to_vec(),
b"<< /Type /Annot /Subtype /Text /Rect [250 200 270 220] /IRT 6 0 R /Contents (SECOND-REPLY) >>".to_vec(),
b"<< /Type /Annot /Subtype /Text /Rect [250 100 270 120] /Contents (UNRELATED) >>".to_vec(),
];
let mut doc = pdf(objs);
mark(&mut doc, 0, &[[10.0, 10.0, 70.0, 40.0]], "");
let r = apply(&mut doc, None).unwrap();
assert_eq!(r.annotations, 3, "the highlight and both replies: {r:?}");
let p = pdfcraft_model::pages(&doc).swap_remove(0);
assert_eq!(annots_of(&doc, &p.dict).len(), 1, "only the unrelated note stays");
// A classic table, so the objects' strings are plain in the file.
let bytes = write_full(&doc, &SaveOptions { object_streams: false, ..SaveOptions::default() }).unwrap();
for gone in [&b"HIGHLIGHT-NOTE"[..], b"FIRST-REPLY", b"SECOND-REPLY"] {
assert!(!bytes.windows(gone.len()).any(|w| w == gone), "{} survived the save", String::from_utf8_lossy(gone));
}
assert!(bytes.windows(9).any(|w| w == b"UNRELATED"));
}
/// The replies listed before the annotations they answer, deepest first, and a pop-up on a
/// reply: one pass in `/Annots` order would keep them, the repeated passes don't (review of
/// #2697).
#[test]
fn replies_listed_before_their_parent_and_their_pop_ups_go_too() {
let objs: Vec<Vec<u8>> = vec![
b"<< /Type /Catalog /Pages 2 0 R >>".to_vec(),
b"<< /Type /Pages /Kids [3 0 R] /Count 1 >>".to_vec(),
b"<< /Type /Page /Parent 2 0 R /MediaBox [0 0 300 300] /Contents 4 0 R /Annots [7 0 R 9 0 R 6 0 R 5 0 R 8 0 R] >>".to_vec(),
stream("", b"BT ET"),
b"<< /Type /Annot /Subtype /Highlight /Rect [20 20 60 30] /QuadPoints [20 30 60 30 20 20 60 20] /Contents (HIGHLIGHT-NOTE) >>".to_vec(),
b"<< /Type /Annot /Subtype /Text /Rect [250 250 270 270] /IRT 5 0 R /Contents (FIRST-REPLY) /Popup 9 0 R >>".to_vec(),
b"<< /Type /Annot /Subtype /Text /Rect [250 200 270 220] /IRT 6 0 R /Contents (SECOND-REPLY) >>".to_vec(),
b"<< /Type /Annot /Subtype /Text /Rect [250 100 270 120] /Contents (UNRELATED) >>".to_vec(),
b"<< /Type /Annot /Subtype /Popup /Rect [200 150 290 190] /Parent 6 0 R >>".to_vec(),
];
let mut doc = pdf(objs);
mark(&mut doc, 0, &[[10.0, 10.0, 70.0, 40.0]], "");
let r = apply(&mut doc, None).unwrap();
assert_eq!(r.annotations, 3, "the highlight and both replies: {r:?}");
let p = pdfcraft_model::pages(&doc).swap_remove(0);
let left = annots_of(&doc, &p.dict);
assert_eq!(left, vec![Object::Ref(ObjRef::new(8, 0))], "only the unrelated note stays, the reply's pop-up goes too");
}
#[test]
fn comments_links_and_fields_under_a_mark_go() {
let mut doc = one_page(b"", "", vec![]);
// A comment under the mark, a link elsewhere, and a text field under the mark.
let square = Shape::Rectangle { rect: [20.0, 20.0, 40.0, 40.0] };
add_annotation(
&mut doc,
&NewAnnotation { page: 0, style: Style::default_for(&square), shape: square, contents: "x".into(), author: "a".into() },
&Meta::default(),
)
.unwrap();
let link = Shape::Rectangle { rect: [200.0, 200.0, 220.0, 220.0] };
add_annotation(
&mut doc,
&NewAnnotation { page: 0, style: Style::default_for(&link), shape: link, contents: "keep".into(), author: "a".into() },
&Meta::default(),
)
.unwrap();
pdfcraft_forms::add_field(&mut doc, 0, [10.0, 50.0, 100.0, 70.0], &pdfcraft_forms::NewField::Text { multiline: false }, Some("ssn")).unwrap();
mark(&mut doc, 0, &[[0.0, 0.0, 120.0, 80.0]], "REDACTED");
let r = apply(&mut doc, None).unwrap();
assert_eq!((r.annotations, r.fields), (1, 1), "{r:?}");
assert!(pdfcraft_forms::fields(&doc).is_empty());
let p = pdfcraft_model::pages(&doc).swap_remove(0);
assert_eq!(annots_of(&doc, &p.dict).len(), 1, "only the far rectangle stays");
let c = content(&doc, 0);
assert!(c.contains("(REDACTED) Tj"), "overlay text: {c}");
}
#[test]
fn nothing_to_apply_and_clearing_marks() {
let mut doc = one_page(b"BT /F1 10 Tf 10 100 Td (AB) Tj ET", "", vec![]);
assert_eq!(apply(&mut doc, None), Err(RedactError::NothingToApply));
mark(&mut doc, 0, &[[0.0, 0.0, 50.0, 50.0]], "");
assert_eq!(marks(&doc).len(), 1);
assert_eq!(clear_marks(&mut doc, None), Ok(1));
assert!(marks(&doc).is_empty());
assert!(content(&doc, 0).contains("(AB) Tj"));
}
/// Clearing marks puts a page's annotation list back as it was: a page that had none has no
/// `/Annots` again, replies to the marks included, and one listed through an indirect array keeps
/// pointing at that array.
#[test]
fn clearing_marks_leaves_the_annotation_list_as_it_was() {
let mut doc = one_page(b"BT /F1 10 Tf 10 100 Td (AB) Tj ET", "", vec![]);
let page = pdfcraft_model::pages(&doc)[0].obj;
let before = doc.get(page);
mark(&mut doc, 0, &[[0.0, 0.0, 50.0, 50.0]], "");
pdfcraft_annot::add_reply(&mut doc, 0, 0, "a reply", "Tester", &Meta::default()).unwrap();
// And a reply written inline, as a file can.
let mut inline = Dict::new();
inline.set(b"Subtype".to_vec(), Object::name("Text"));
inline.set(b"IRT".to_vec(), Object::Ref(marks(&doc)[0].obj));
let mut listed = annots_of(&doc, doc.get(page).as_dict().unwrap());
listed.push(Object::Dict(inline));
doc.update_dict(page, |d| d.set(b"Annots".to_vec(), Object::Array(listed))).unwrap();
assert_eq!(clear_marks(&mut doc, None), Ok(1), "one mark");
assert_eq!(doc.get(page), before, "the page has no /Annots again: both replies went with their mark");
let mut doc = pdf(vec![
b"<< /Type /Catalog /Pages 2 0 R >>".to_vec(),
b"<< /Type /Pages /Kids [3 0 R] /Count 1 >>".to_vec(),
b"<< /Type /Page /Parent 2 0 R /MediaBox [0 0 300 300] /Annots 4 0 R >>".to_vec(),
b"[5 0 R]".to_vec(),
b"<< /Type /Annot /Subtype /Link /Rect [0 0 10 10] >>".to_vec(),
]);
let (page, list) = (ObjRef::new(3, 0), ObjRef::new(4, 0));
let before = (doc.get(page), doc.get(list));
mark(&mut doc, 0, &[[0.0, 0.0, 50.0, 50.0]], "");
assert_eq!(doc.get(list).as_array().map(|a| a.len()), Some(2), "the mark is listed in the shared array");
clear_marks(&mut doc, None).unwrap();
assert_eq!((doc.get(page), doc.get(list)), before, "the page points at its array, which lists the link alone");
}
/// A group subordinate (`/RT /Group`) of a mark stays when the marks are cleared, as one stays
/// when its primary comment is deleted, and loses its `/IRT`: nothing then refers to the mark,
/// so a full save leaves the mark's text out.
#[test]
fn clearing_marks_releases_a_group_member_of_a_mark() {
let mut doc = one_page(b"BT /F1 10 Tf 10 100 Td (AB) Tj ET", "", vec![]);
mark(&mut doc, 0, &[[0.0, 0.0, 50.0, 50.0]], "");
let m = marks(&doc)[0].obj;
doc.update_dict(m, |d| d.set(b"Contents".to_vec(), pdfcraft_cos::PdfString::text("MARK_SECRET_TEXT"))).unwrap();
let mut member = Dict::new();
member.set(b"Type".to_vec(), Object::name("Annot"));
member.set(b"Subtype".to_vec(), Object::name("Square"));
member.set(b"Rect".to_vec(), Object::Array([60, 60, 90, 90].map(Object::Int).to_vec()));
member.set(b"IRT".to_vec(), Object::Ref(m));
member.set(b"RT".to_vec(), Object::name("Group"));
let member = doc.add(Object::Dict(member));
let page = pdfcraft_model::pages(&doc)[0].obj;
let mut listed = annots_of(&doc, doc.get(page).as_dict().unwrap());
listed.push(Object::Ref(member));
doc.update_dict(page, |d| d.set(b"Annots".to_vec(), Object::Array(listed))).unwrap();
assert_eq!(clear_marks(&mut doc, None), Ok(1));
assert_eq!(annots_of(&doc, doc.get(page).as_dict().unwrap()), [Object::Ref(member)], "the member stays");
let d = doc.get(member).as_dict().cloned().unwrap();
assert!(d.get(b"IRT").is_none() && d.name(b"RT") == Some(b"Group".as_slice()), "{d:?}");
let full = write_full(&doc, &SaveOptions { object_streams: false, ..SaveOptions::default() }).unwrap();
assert!(!full.windows(16).any(|w| w == b"MARK_SECRET_TEXT"), "the cleared mark is not in a full save");
}
#[test]
fn redacted_copies_are_named_with_redacted_before_the_extension() {
assert_eq!(redacted_file_name("contract.pdf"), "contract_Redacted.pdf");
assert_eq!(redacted_file_name("SCAN.PDF"), "SCAN_Redacted.pdf");
assert_eq!(redacted_file_name("Report.Pdf"), "Report_Redacted.pdf");
// A document without the extension (opened from bytes) gets it.
assert_eq!(redacted_file_name("Untitled"), "Untitled_Redacted.pdf");
// Redacting a redacted copy again does not add a second suffix.
assert_eq!(redacted_file_name("contract_Redacted.pdf"), "contract_Redacted.pdf");
}
#[test]
fn unreadable_content_fails_closed() {
let mut doc = one_page(b"", "", vec![]);
let page = pdfcraft_model::pages(&doc)[0].obj;
let bad = doc.add(Object::Stream(Stream::from_raw(
{
let mut d = Dict::new();
d.set(b"Filter".to_vec(), Object::name("DCTDecode"));
d
},
b"garbage".to_vec(),
)));
doc.update_dict(page, |d| d.set(b"Contents".to_vec(), Object::Ref(bad))).unwrap();
mark(&mut doc, 0, &[[0.0, 0.0, 50.0, 50.0]], "");
assert_eq!(apply(&mut doc, None), Err(RedactError::Unreadable(1)));
}
/// Leaves a mark with nothing readable: /QuadPoints becomes a name and /Rect is removed.
fn break_area(doc: &mut Document, obj: ObjRef) {
doc.update_dict(obj, |d| {
d.set(b"QuadPoints".to_vec(), Object::name("oops"));
d.remove(b"Rect");
})
.unwrap();
}
/// A Redact annotation with no readable area (neither usable QuadPoints nor a usable Rect) used to
/// vanish: `marks` skipped it, so applying the readable mark next to it reported success and left
/// the text under the broken mark in place. Now the whole operation is refused before anything
/// changes, and the count is of the unreadable marks.
#[test]
fn a_mark_with_no_readable_area_is_refused_instead_of_vanishing() {
let mut doc = one_page(b"BT /F1 10 Tf 10 100 Td (AB1234CD) Tj ET", "", vec![]);
mark(&mut doc, 0, &[[40.0, 95.0, 50.0, 110.0]], "");
mark(&mut doc, 0, &[[0.0, 95.0, 10.0, 110.0]], "");
for m in marks(&doc) {
break_area(&mut doc, m.obj);
}
mark(&mut doc, 0, &[[20.0, 95.0, 40.0, 110.0]], "");
assert_eq!(marks(&doc).len(), 1, "only the readable mark is listed");
assert_eq!(apply(&mut doc, None), Err(RedactError::UnreadableMark(2)));
// Nothing changed: the readable mark is still pending and its text is still drawn.
assert_eq!(marks(&doc).len(), 1);
assert!(content(&doc, 0).contains("(AB1234CD) Tj"), "{}", content(&doc, 0));
}
/// A mark whose QuadPoints can't be used still redacts through its Rect, as it always did.
#[test]
fn a_mark_with_only_a_usable_rect_still_redacts() {
let mut doc = one_page(b"BT /F1 10 Tf 10 100 Td (AB1234CD) Tj ET", "", vec![]);
mark(&mut doc, 0, &[[20.0, 95.0, 40.0, 110.0]], "");
let m = marks(&doc).swap_remove(0);
doc.update_dict(m.obj, |d| {
d.set(b"QuadPoints".to_vec(), Object::name("oops"));
d.set(b"Rect".to_vec(), Object::Array(vec![Object::Int(20), Object::Int(95), Object::Int(40), Object::Int(110)]));
})
.unwrap();
assert_eq!(marks(&doc).len(), 1, "the Rect keeps the mark readable");
let r = apply(&mut doc, None).unwrap();
assert_eq!((r.marks, r.glyphs), (1, 4));
assert!(!content(&doc, 0).contains("1234"));
}
/// Only the pages being applied decide the refusal: an unreadable mark on another page does not
/// stop the marks of the selected page from being applied.
#[test]
fn an_unreadable_mark_on_another_page_does_not_stop_the_selected_page() {
let mut doc = pdf(vec![
b"<< /Type /Catalog /Pages 2 0 R >>".to_vec(),
b"<< /Type /Pages /Kids [3 0 R 7 0 R] /Count 2 >>".to_vec(),
b"<< /Type /Page /Parent 2 0 R /MediaBox [0 0 300 300] /Contents 4 0 R /Resources << /Font << /F1 5 0 R >> >> >>".to_vec(),
stream("", b"BT /F1 10 Tf 10 100 Td (AB1234CD) Tj ET"),
FONT.replace("95 0 R", "6 0 R").into_bytes(),
widths(),
b"<< /Type /Page /Parent 2 0 R /MediaBox [0 0 300 300] /Contents 8 0 R /Resources << /Font << /F1 5 0 R >> >> >>".to_vec(),
stream("", b"BT /F1 10 Tf 10 100 Td (XY) Tj ET"),
]);
mark(&mut doc, 1, &[[10.0, 95.0, 30.0, 110.0]], "");
let broken = marks(&doc).swap_remove(0);
break_area(&mut doc, broken.obj);
mark(&mut doc, 0, &[[20.0, 95.0, 40.0, 110.0]], "");
assert_eq!(apply(&mut doc, None), Err(RedactError::UnreadableMark(1)));
assert_eq!(apply(&mut doc, Some(&[0])).map(|r| r.glyphs), Ok(4));
assert!(!content(&doc, 0).contains("1234"));
}
/// The quad over "1234" in `AB1234CD` (x 20–40) and the one over "CD" (x 40–50).
const OVER_1234: [f64; 8] = [20.0, 110.0, 40.0, 110.0, 20.0, 95.0, 40.0, 95.0];
const OVER_CD: [f64; 8] = [40.0, 110.0, 50.0, 110.0, 40.0, 95.0, 50.0, 95.0];
/// `AB1234CD` with one mark whose `/QuadPoints` is `quads`, whose `/Rect` is `rect` (removed when
/// `None`) and which has a fill (`/IC`) only when `fill` is set.
fn quad_mark(quads: Vec<Object>, rect: Option<[i64; 4]>, fill: bool) -> (Document, ObjRef) {
let mut doc = one_page(b"BT /F1 10 Tf 10 100 Td (AB1234CD) Tj ET", "", vec![]);
mark(&mut doc, 0, &[[20.0, 95.0, 40.0, 110.0]], "");
let m = marks(&doc).swap_remove(0);
doc.update_dict(m.obj, |d| {
d.set(b"QuadPoints".to_vec(), Object::Array(quads));
if let Some(r) = rect {
d.set(b"Rect".to_vec(), Object::Array(r.iter().map(|v| Object::Int(*v)).collect()));
} else {
d.remove(b"Rect");
}
if !fill {
d.remove(b"IC");
}
})
.unwrap();
(doc, m.obj)
}
fn reals(v: &[f64]) -> Vec<Object> {
v.iter().map(|x| Object::Real(*x)).collect()
}
/// An all-NaN quad folded to `[MAX, MAX, MIN, MIN]`: a finite, inverted box that passed as a
/// readable area. Without a fill the mark was applied and removed nothing; with one the overlay
/// failed. With no `/Rect` either, the mark has no area and apply refuses it.
#[test]
fn an_all_nan_quad_without_a_rect_is_refused_not_applied_as_nothing() {
for fill in [false, true] {
let (mut doc, _) = quad_mark(reals(&[f64::NAN; 8]), None, fill);
assert!(marks(&doc).is_empty(), "fill {fill}: the NaN quad is no area");
assert_eq!(apply(&mut doc, None), Err(RedactError::UnreadableMark(1)), "fill {fill}");
assert!(content(&doc, 0).contains("(AB1234CD) Tj"), "fill {fill}: nothing changed");
}
}
/// With a usable `/Rect`, an all-NaN mark redacts exactly the `/Rect`.
#[test]
fn an_all_nan_quad_falls_back_to_the_rect() {
for fill in [false, true] {
let (mut doc, _) = quad_mark(reals(&[f64::NAN; 8]), Some([20, 95, 40, 110]), fill);
assert_eq!(marks(&doc)[0].rects, vec![[20.0, 95.0, 40.0, 110.0]], "fill {fill}");
let r = apply(&mut doc, None).unwrap();
assert_eq!((r.marks, r.glyphs), (1, 4), "fill {fill}");
assert!(!content(&doc, 0).contains("1234"), "fill {fill}");
}
}
/// One NaN among finite entries made `f64::min`/`max` ignore it and fold the other three corners:
/// a quad the document never described. Such a quad is unusable.
#[test]
fn a_quad_with_one_nan_entry_is_unusable() {
let mut q = OVER_1234;
q[0] = f64::NAN;
let (mut doc, _) = quad_mark(reals(&q), None, false);
assert!(marks(&doc).is_empty());
assert_eq!(apply(&mut doc, None), Err(RedactError::UnreadableMark(1)));
// Next to a usable quad, only that one applies.
let (mut doc, _) = quad_mark(reals(&[q, OVER_CD].concat()), None, false);
assert_eq!(marks(&doc)[0].rects, vec![[40.0, 95.0, 50.0, 110.0]]);
assert_eq!(apply(&mut doc, None).map(|r| r.glyphs), Ok(2));
}
/// A name in the middle of `/QuadPoints` was dropped, which moved every later coordinate into the
/// wrong slot and folded the first quad to a box over most of the page.
#[test]
fn a_non_number_entry_drops_its_quad_without_shifting_the_rest() {
let mut quads = reals(&[OVER_1234, OVER_CD].concat());
quads[2] = Object::name("oops");
let (mut doc, _) = quad_mark(quads, None, false);
assert_eq!(marks(&doc)[0].rects, vec![[40.0, 95.0, 50.0, 110.0]]);
assert_eq!(apply(&mut doc, None).map(|r| r.glyphs), Ok(2));
}
/// Twelve entries are one quad and half of another: only the whole one applies.
#[test]
fn a_trailing_partial_quad_is_ignored() {
let (mut doc, _) = quad_mark(reals(&[&OVER_1234[..], &OVER_CD[..4]].concat()), None, false);
assert_eq!(marks(&doc)[0].rects, vec![[20.0, 95.0, 40.0, 110.0]]);
assert_eq!(apply(&mut doc, None).map(|r| r.glyphs), Ok(4));
}
/// Finite quads redact what they always did, one box each, and the `/Rect` is not consulted.
#[test]
fn finite_quads_redact_their_areas_as_before() {
let (mut doc, _) = quad_mark(reals(&[OVER_1234, OVER_CD].concat()), Some([0, 0, 1, 1]), true);
assert_eq!(marks(&doc)[0].rects, vec![[20.0, 95.0, 40.0, 110.0], [40.0, 95.0, 50.0, 110.0]]);
let r = apply(&mut doc, None).unwrap();
assert_eq!((r.marks, r.glyphs), (1, 6));
assert!(!content(&doc, 0).contains("1234"));
}
/// The boxes a reader's quads outline, or its `/Rect` when it has none (as the comment outlines
/// fall back to it).
fn outline(quads: &[[f32; 8]], rect: [f32; 4]) -> Vec<[f32; 4]> {
if quads.is_empty() {
return vec![rect];
}
quads
.iter()
.map(|q| {
let (xs, ys) = ([q[0], q[2], q[4], q[6]], [q[1], q[3], q[5], q[7]]);
let (lo, hi) = (|v: [f32; 4]| v.into_iter().fold(f32::MAX, f32::min), |v: [f32; 4]| v.into_iter().fold(f32::MIN, f32::max));
[lo(xs), lo(ys), hi(xs), hi(ys)]
})
.collect()
}
/// The outline the comment list draws (`pdfcraft_annot::summaries`, and the inspector's listing
/// of the saved file) is the area apply removes, for malformed `/QuadPoints` too. The readers
/// used to disagree: the summaries and the inspector read non-numbers as zeros, apply dropped
/// them.
#[test]
fn the_outline_shown_is_the_area_apply_removes() {
// (entries, whether a saved file can say them: PDF syntax has no NaN). `forty` is an indirect
// number, which every reader follows.
let cases = |forty: ObjRef| -> Vec<(Vec<Object>, bool)> {
let mut name_inside = reals(&[OVER_1234, OVER_CD].concat());
name_inside[2] = Object::name("oops");
let mut by_reference = reals(&OVER_1234);
by_reference[2] = Object::Ref(forty);
let mut one_nan = OVER_1234;
one_nan[5] = f64::NAN;
let mut one_huge = OVER_1234;
one_huge[5] = 4e38;
vec![
(reals(&[OVER_1234, OVER_CD].concat()), true),
(name_inside, true),
(reals(&[&OVER_1234[..], &OVER_CD[..4]].concat()), true),
(by_reference, true),
(vec![Object::name("oops"); 8], true),
(reals(&[one_nan, OVER_CD].concat()), false),
(reals(&[one_huge, OVER_CD].concat()), true),
(reals(&[f64::NAN; 8]), false),
]
};
for i in 0.. {
let (mut doc, obj) = quad_mark(Vec::new(), Some([20, 95, 40, 110]), true);
let forty = doc.add(Object::Int(40));
let Some((quads, saved)) = cases(forty).into_iter().nth(i) else { break };
doc.update_dict(obj, |d| d.set(b"QuadPoints".to_vec(), Object::Array(quads))).unwrap();
let applied: Vec<[f32; 4]> = marks(&doc)[0].rects.iter().map(|r| r.map(|v| v as f32)).collect();
let summary = pdfcraft_annot::summaries(&doc).into_iter().find(|s| s.subtype == "Redact").unwrap();
assert_eq!(outline(&summary.quads, summary.rect), applied, "case {i}: comment summaries");
if saved {
let bytes = write_incremental(&doc, &SaveOptions::default()).unwrap();
let info = pdfcraft_render::inspect(Arc::new(bytes), None).unwrap();
let listed = info.annotations.iter().find(|a| a.subtype == "Redact").unwrap();
assert_eq!(outline(&listed.quads, listed.rect), applied, "case {i}: inspector");
}
}
}
/// A document with something in every hidden-information category.
fn hidden_fixture() -> Document {
let content = b"BT /F1 10 Tf 10 100 Td (Visible) Tj 3 Tr (Hidden) Tj 0 Tr ET BT /F1 10 Tf 500 500 Td (Offpage) Tj ET /OC /L1 BDC BT /F1 10 Tf 10 50 Td (Layer) Tj ET EMC";
let objs: Vec<Vec<u8>> = vec![
// 1 catalog
b"<< /Type /Catalog /Pages 2 0 R /Metadata 7 0 R /Names << /EmbeddedFiles << /Names [(a.txt) 8 0 R] >> /JavaScript << /Names [(init) 9 0 R] >> >> /OpenAction 9 0 R /Outlines 10 0 R /PageMode /UseOutlines /PieceInfo << /App << /Private 1 >> >> /OCProperties << /OCGs [13 0 R 14 0 R] /D << /OFF [13 0 R] /Order [13 0 R 14 0 R] >> >> /AcroForm << /Fields [17 0 R] >> >>".to_vec(),
b"<< /Type /Pages /Kids [3 0 R] /Count 1 >>".to_vec(),
// 3 page
b"<< /Type /Page /Parent 2 0 R /MediaBox [0 0 300 300] /Contents 4 0 R /Resources << /Font << /F1 5 0 R >> /Properties << /L1 13 0 R >> >> /Annots [15 0 R 16 0 R 17 0 R 18 0 R 19 0 R] /PieceInfo << /App << /Private 2 >> >> /AA << /O 9 0 R >> >>".to_vec(),
stream("", content),
FONT.replace("95 0 R", "6 0 R").into_bytes(),
widths(),
stream("/Type /Metadata /Subtype /XML", b"<x:xmpmeta/>"),
b"<< /Type /Filespec /F (a.txt) /EF << /F 20 0 R >> >>".to_vec(),
b"<< /S /JavaScript /JS (app.alert(1)) >>".to_vec(),
b"<< /Type /Outlines /First 11 0 R /Last 12 0 R /Count 2 >>".to_vec(),
b"<< /Title (One) /Parent 10 0 R /Next 12 0 R >>".to_vec(),
b"<< /Title (Two) /Parent 10 0 R /Prev 11 0 R >>".to_vec(),
b"<< /Type /OCG /Name (Secret layer) >>".to_vec(),
b"<< /Type /OCG /Name (Shown layer) >>".to_vec(),
// 15 comment + 16 its pop-up, 17 widget, 18 link, 19 attachment
b"<< /Type /Annot /Subtype /Square /Rect [10 10 50 50] /C [1 0 0] /Popup 16 0 R >>".to_vec(),
b"<< /Type /Annot /Subtype /Popup /Rect [60 10 160 60] /Parent 15 0 R >>".to_vec(),
b"<< /Type /Annot /Subtype /Widget /FT /Tx /T (name) /V (Ada) /Rect [10 200 150 220] /A 9 0 R >>".to_vec(),
b"<< /Type /Annot /Subtype /Link /Rect [10 230 50 240] /A << /S /URI /URI (https://example.org) >> >>".to_vec(),
b"<< /Type /Annot /Subtype /FileAttachment /Rect [200 10 210 20] /FS 8 0 R >>".to_vec(),
stream("", b"attached"),
];
let mut doc = pdf(objs);
let info = doc.add(Object::Dict({
let mut d = Dict::new();
d.set(b"Title".to_vec(), pdfcraft_cos::PdfString::text("Secret plan"));
d.set(b"Author".to_vec(), pdfcraft_cos::PdfString::text("Ada"));
d
}));
doc.trailer_mut().set(b"Info".to_vec(), Object::Ref(info));
doc
}
#[test]
fn hidden_information_is_counted_and_removed() {
use crate::sanitize::{Hidden, remove_hidden, sanitize, scan};
let mut doc = hidden_fixture();
let counts: std::collections::HashMap<Hidden, usize> = scan(&doc).into_iter().collect();
assert_eq!(counts[&Hidden::Metadata], 3, "two Info entries and the XMP stream");
assert_eq!(counts[&Hidden::Attachments], 2, "the embedded file and the attachment annotation");
assert_eq!(counts[&Hidden::Comments], 1);
assert_eq!(counts[&Hidden::FormFields], 1);
assert_eq!(counts[&Hidden::HiddenText], 6 + 7, "Hidden (render mode 3) and Offpage (off the page)");
assert_eq!(counts[&Hidden::HiddenLayers], 2, "one off layer and its block");
assert_eq!(counts[&Hidden::Bookmarks], 2);
assert_eq!(counts[&Hidden::LinksActionsScripts], 5, "link, open action, page actions, widget action, document script");
assert_eq!(counts[&Hidden::PrivateData], 2);
// Only hidden text first.
let done = remove_hidden(&mut doc, &[Hidden::HiddenText]).unwrap();
assert_eq!(done, [(Hidden::HiddenText, 13)]);
let c = content(&doc, 0);
assert!(c.contains("(Visible)") && !c.contains("Hidden") && !c.contains("Offpage") && c.contains("(Layer)"), "{c}");
assert!(doc.full_save_required());
// Then everything.
let done = sanitize(&mut doc).unwrap();
assert!(done.iter().all(|(h, _)| *h != Hidden::HiddenText));
let c = content(&doc, 0);
assert!(!c.contains("Layer") && c.contains("(Visible)"), "{c}");
let doc = reopen(&doc);
let after: Vec<(Hidden, usize)> = scan(&doc).into_iter().filter(|(_, n)| *n > 0).collect();
// A full save writes a fresh /Info with the modification date only.
assert!(after.iter().all(|(h, _)| *h == Hidden::Metadata), "{after:?}");
let cat = doc.get(doc.root().unwrap()).as_dict().cloned().unwrap();
for k in [&b"Outlines"[..], b"OpenAction", b"PieceInfo", b"AcroForm", b"Metadata"] {
assert!(!cat.contains(k), "{}", String::from_utf8_lossy(k));
}
assert!(pdfcraft_forms::fields(&doc).is_empty());
let p = pdfcraft_model::pages(&doc).swap_remove(0);
assert!(annots_of(&doc, &p.dict).is_empty());
let shown = doc.object_numbers().into_iter().any(|n| match &*doc.get(ObjRef::new(n, doc.generation(n))) {
Object::Stream(s) => s.decoded().is_ok_and(|d| d.windows(5).any(|w| w == b"(Ada)")),
_ => false,
});
assert!(shown, "the field's value stays visible as page content");
}
/// A one-page document whose catalog names an outline root (object 5). `items` are the outline
/// items, objects 6 onward.
fn outlined_document(items: Vec<Vec<u8>>) -> Document {
let mut objs: Vec<Vec<u8>> = vec![
b"<< /Type /Catalog /Pages 2 0 R /Outlines 5 0 R >>".to_vec(),
b"<< /Type /Pages /Kids [3 0 R] /Count 1 >>".to_vec(),
b"<< /Type /Page /Parent 2 0 R /MediaBox [0 0 300 300] /Contents 4 0 R >>".to_vec(),
stream("", b"BT ET"),
b"<< /Type /Outlines /First 6 0 R >>".to_vec(),
];
objs.extend(items);
pdf(objs)
}
/// An outline of `levels` levels with two items each. Both items of a level name the first item
/// of the next level as their /First, so each of those items has two parents, and a walk that
/// follows every path takes 2^levels steps. Items a and b of level k are objects 6 + 2k and 7 + 2k.
fn hostile_outline(levels: usize) -> Document {
let mut items = Vec::new();
for k in 0..levels {
let (a, b) = (6 + 2 * k, 7 + 2 * k);
let child = if k + 1 < levels { format!(" /First {} 0 R", a + 2) } else { String::new() };
items.push(format!("<< /Title (a) /Next {b} 0 R{child} >>").into_bytes());
items.push(format!("<< /Title (b){child} /Parent {a} 0 R >>").into_bytes());
}
outlined_document(items)
}
/// Runs `work` on another thread and returns its result. A walk that spins fails the test after
/// `bound` instead of hanging the whole run; the thread stays behind and ends with the process.
fn finishes_within<T: Send + 'static>(bound: std::time::Duration, work: impl FnOnce() -> T + Send + 'static) -> T {
let (tx, rx) = std::sync::mpsc::channel();
std::thread::spawn(move || {
let _ = tx.send(work());
});
match rx.recv_timeout(bound) {
Ok(out) => out,
Err(std::sync::mpsc::RecvTimeoutError::Timeout) => {
panic!("the walk did not finish within {bound:?}: it is spinning on shared outline items")
}
Err(std::sync::mpsc::RecvTimeoutError::Disconnected) => panic!("the walk panicked"),
}
}
/// Generous for a slow machine: the bounded walk takes a few milliseconds.
const WALK_BOUND: std::time::Duration = std::time::Duration::from_secs(10);
/// An outline whose items name shared children is counted item by item, each item once, and the
/// walk ends. Counting along every path instead takes 2^60 steps.
#[test]
fn an_outline_whose_items_share_children_is_counted_without_spinning() {
use crate::sanitize::{Hidden, scan};
let doc = hostile_outline(60);
let counts = finishes_within(WALK_BOUND, move || scan(&doc));
assert_eq!(counts.iter().find(|c| c.0 == Hidden::Bookmarks), Some(&(Hidden::Bookmarks, 120)), "60 levels of two distinct items");
}
/// Removing the bookmarks counts them through the same walk first, so it must finish too, and
/// it must take the outline out of the catalog.
#[test]
fn removing_bookmarks_from_an_outline_whose_items_share_children_finishes() {
use crate::sanitize::{Hidden, remove_hidden};
let mut doc = hostile_outline(60);
let (done, outlines_left) = finishes_within(WALK_BOUND, move || {
let done = remove_hidden(&mut doc, &[Hidden::Bookmarks]);
let root = doc.root().unwrap();
let left = doc.get(root).as_dict().is_some_and(|c| c.contains(b"Outlines"));
(done, left)
});
assert_eq!(done.unwrap(), [(Hidden::Bookmarks, 120)]);
assert!(!outlines_left, "the outline is removed from the catalog");
}
/// Nesting and siblings: every item of an ordinary outline is counted once, as before.
#[test]
fn a_nested_outline_is_counted_item_by_item() {
use crate::sanitize::{Hidden, scan};
let doc = outlined_document(vec![
b"<< /Title (Chapter 1) /Parent 5 0 R /First 7 0 R /Last 8 0 R /Next 9 0 R >>".to_vec(),
b"<< /Title (1.1) /Parent 6 0 R /Next 8 0 R >>".to_vec(),
b"<< /Title (1.2) /Parent 6 0 R /Prev 7 0 R >>".to_vec(),
b"<< /Title (Chapter 2) /Parent 5 0 R /Prev 6 0 R /First 10 0 R >>".to_vec(),
b"<< /Title (2.1) /Parent 9 0 R >>".to_vec(),
]);
assert_eq!(scan(&doc).iter().find(|c| c.0 == Hidden::Bookmarks), Some(&(Hidden::Bookmarks, 5)));
}
/// Two levels of 60,000 distinct items: each level is under the per-level cap, so only the total
/// cap stops the count at 100,000 (without it the count would be 120,000).
#[test]
fn an_outline_counts_no_more_items_than_its_total_budget() {
use crate::sanitize::{Hidden, scan};
let per_level = 60_000;
let mut items = Vec::with_capacity(2 * per_level);
// Item j is object 6 + j. Level one is items 0..per_level; its first item leads to level two.
for i in 0..per_level {
let next = if i + 1 < per_level { format!(" /Next {} 0 R", 7 + i) } else { String::new() };
let first = if i == 0 { format!(" /First {} 0 R", 6 + per_level) } else { String::new() };
items.push(format!("<< /Title (a){next}{first} >>").into_bytes());
}
for i in 0..per_level {
let next = if i + 1 < per_level { format!(" /Next {} 0 R", 7 + per_level + i) } else { String::new() };
items.push(format!("<< /Title (b){next} >>").into_bytes());
}
let doc = outlined_document(items);
assert_eq!(scan(&doc).iter().find(|c| c.0 == Hidden::Bookmarks), Some(&(Hidden::Bookmarks, 100_000)));
}
/// Items written inline (not as indirect objects) count towards the total budget too. Each of
/// 60,000 indirect siblings has one inline child, so 120,000 items are reached, and the total must
/// still stop at 100,000.
#[test]
fn inline_outline_children_count_towards_the_total_budget() {
use crate::sanitize::{Hidden, scan};
let siblings = 60_000;
let mut items = Vec::with_capacity(siblings);
for i in 0..siblings {
let next = if i + 1 < siblings { format!(" /Next {} 0 R", 7 + i) } else { String::new() };
items.push(format!("<< /Title (a){next} /First << /Title (c) >> >>").into_bytes());
}
let doc = outlined_document(items);
assert_eq!(scan(&doc).iter().find(|c| c.0 == Hidden::Bookmarks), Some(&(Hidden::Bookmarks, 100_000)));
}
/// A redacted copy and a sanitized copy are different files from their source: neither may keep
/// the source's file identifier (ISO 32000-2 §14.4). A save without the rewrite keeps it.
#[test]
fn a_redacted_or_sanitized_copy_gets_a_new_identifier() {
fn with_id(mut doc: Document) -> Document {
let id = Object::Array(vec![
Object::String(pdfcraft_cos::PdfString::literal(vec![0x11; 16])),
Object::String(pdfcraft_cos::PdfString::literal(vec![0x22; 16])),
]);
doc.trailer_mut().set(b"ID".to_vec(), id);
doc
}
fn saved_ids(doc: &Document) -> Vec<Vec<u8>> {
let saved = Document::open(Arc::new(write_full(doc, &SaveOptions::default()).unwrap())).unwrap();
match saved.trailer().get(b"ID") {
Some(Object::Array(a)) => a.iter().map(|s| s.as_string().map(|s| s.bytes.clone()).unwrap_or_default()).collect(),
_ => Vec::new(),
}
}
let source = vec![vec![0x11; 16], vec![0x22; 16]];
let assert_new = |ids: &[Vec<u8>], what: &str| {
assert_eq!(ids.len(), 2, "{what}");
for (new, old) in ids.iter().zip(&source) {
assert_eq!(new.len(), 16, "{what}");
assert_ne!(new, old, "{what}: the copy keeps the source's identifier");
}
};
// Redaction: the same save keeps the identifier until the redaction asks for a new one.
let mut doc = with_id(one_page(b"BT /F1 10 Tf 10 100 Td (Secret) Tj ET", "", vec![]));
mark(&mut doc, 0, &[[0.0, 95.0, 300.0, 110.0]], "");
assert_eq!(saved_ids(&doc), source, "before the redaction");
apply(&mut doc, None).unwrap();
assert!(doc.new_id_required());
assert_new(&saved_ids(&doc), "redaction");
// Sanitizing with hidden information to remove.
let mut doc = with_id(hidden_fixture());
crate::sanitize::sanitize(&mut doc).unwrap();
assert!(doc.new_id_required());
assert_new(&saved_ids(&doc), "sanitizing");
// Sanitizing a file with nothing hidden still rewrites it, so it gets a new identifier too.
let mut doc = with_id(one_page(b"BT ET", "", vec![]));
assert!(crate::sanitize::sanitize(&mut doc).unwrap().is_empty());
assert_new(&saved_ids(&doc), "sanitizing without hidden information");
}
/// A copy that lost hidden information through `remove_hidden` itself is a different file from its
/// source, so it does not keep the source's identifier either: the remove-hidden edit and the
/// optimizer's discard option call `remove_hidden` directly, not through `sanitize`.
#[test]
fn a_remove_hidden_copy_gets_a_new_identifier() {
use crate::sanitize::{Hidden, remove_hidden};
let with_id = |mut doc: Document| {
doc.trailer_mut().set(
b"ID".to_vec(),
Object::Array(vec![
Object::String(pdfcraft_cos::PdfString::literal(vec![0x11; 16])),
Object::String(pdfcraft_cos::PdfString::literal(vec![0x22; 16])),
]),
);
doc
};
let saved_ids = |doc: &Document| {
let saved = Document::open(Arc::new(write_full(doc, &SaveOptions::default()).unwrap())).unwrap();
match saved.trailer().get(b"ID") {
Some(Object::Array(a)) => a.iter().map(|s| s.as_string().map(|s| s.bytes.clone()).unwrap_or_default()).collect::<Vec<_>>(),
_ => Vec::new(),
}
};
let mut doc = with_id(hidden_fixture());
assert_eq!(remove_hidden(&mut doc, &[Hidden::Metadata]).unwrap().len(), 1, "the fixture has metadata to remove");
assert!(doc.new_id_required());
let ids = saved_ids(&doc);
assert_eq!(ids.len(), 2);
assert_ne!(ids[0], vec![0x11; 16], "the copy keeps the source's first identifier string");
assert_ne!(ids[1], vec![0x22; 16], "the copy keeps the source's second identifier string");
}
/// Links, actions and JavaScripts takes the scripts out of bookmarks that stay: an item that runs
/// JavaScript loses its action, a jump with a script chained after it keeps only the jump, and a
/// plain jump is left alone.
#[test]
fn bookmarks_that_stay_lose_their_scripts() {
use crate::sanitize::{Hidden, remove_hidden, scan};
let mut doc = pdf(vec![
b"<< /Type /Catalog /Pages 2 0 R /Outlines 5 0 R >>".to_vec(),
b"<< /Type /Pages /Kids [3 0 R] /Count 1 >>".to_vec(),
b"<< /Type /Page /Parent 2 0 R /MediaBox [0 0 300 300] /Contents 4 0 R >>".to_vec(),
stream("", b""),
b"<< /Type /Outlines /First 6 0 R /Last 8 0 R /Count 3 >>".to_vec(),
// 6 runs a script, 7 jumps and then runs one, 8 only jumps; 9 the script
b"<< /Title (Runs) /Parent 5 0 R /Next 7 0 R /A 9 0 R >>".to_vec(),
b"<< /Title (Jumps then runs) /Parent 5 0 R /Prev 6 0 R /Next 8 0 R /A << /S /GoTo /D [3 0 R /Fit] /Next 9 0 R >> >>".to_vec(),
b"<< /Title (Jumps) /Parent 5 0 R /Prev 7 0 R /A << /S /GoTo /D [3 0 R /Fit] >> >>".to_vec(),
b"<< /S /JavaScript /JS (app.alert('OUTLINESCRIPT')) >>".to_vec(),
]);
assert_eq!(scan(&doc).into_iter().find(|c| c.0 == Hidden::LinksActionsScripts), Some((Hidden::LinksActionsScripts, 2)));
assert_eq!(remove_hidden(&mut doc, &[Hidden::LinksActionsScripts]).unwrap(), [(Hidden::LinksActionsScripts, 2)]);
let item = |n: u32| doc.get(ObjRef::new(n, 0)).as_dict().cloned().unwrap();
assert!(!item(6).contains(b"A"), "{:?}", item(6));
let jump = |n: u32| item(n).get(b"A").and_then(|a| doc.resolve(a).as_dict().cloned()).unwrap();
assert_eq!((jump(7).name(b"S"), jump(7).contains(b"D"), jump(7).contains(b"Next")), (Some(&b"GoTo"[..]), true, false));
assert_eq!(jump(8).name(b"S"), Some(&b"GoTo"[..]));
let bytes = write_full(&doc, &SaveOptions { object_streams: false, ..SaveOptions::default() }).unwrap();
assert!(bytes.windows(7).any(|w| w == b"(Jumps)"), "the bookmarks stay");
assert!(!bytes.windows(13).any(|w| w == b"OUTLINESCRIPT"), "the bookmark's script survived the save");
}
/// A damaged outline may reach an item only backwards (`/Last`, `/Prev`, `/Parent`). The full save
/// keeps that item all the same, so its script goes too.
#[test]
fn bookmarks_reached_only_backwards_lose_their_scripts() {
use crate::sanitize::{Hidden, remove_hidden, scan};
// /First and /Next reach only 6; 7, 8 and 9 hang off its /Last, /Prev and /Parent and share script 10
let mut doc = outlined_document(vec![
b"<< /Title (Reached) /Last 7 0 R /Prev 8 0 R /Parent 9 0 R >>".to_vec(),
b"<< /Title (Last) /A 10 0 R >>".to_vec(),
b"<< /Title (Prev) /A 10 0 R >>".to_vec(),
b"<< /Title (Parent) /A 10 0 R >>".to_vec(),
b"<< /S /JavaScript /JS (app.alert('OUTLINESCRIPT')) >>".to_vec(),
]);
assert_eq!(scan(&doc).into_iter().find(|c| c.0 == Hidden::LinksActionsScripts), Some((Hidden::LinksActionsScripts, 3)));
assert_eq!(remove_hidden(&mut doc, &[Hidden::LinksActionsScripts]).unwrap(), [(Hidden::LinksActionsScripts, 3)]);
let bytes = write_full(&doc, &SaveOptions { object_streams: false, ..SaveOptions::default() }).unwrap();
assert!(bytes.windows(8).any(|w| w == b"(Parent)"), "the bookmarks stay");
assert!(!bytes.windows(13).any(|w| w == b"OUTLINESCRIPT"), "a script reached only backwards survived the save");
}
/// Outline items written in place count towards the outline budget too: 60,000 items with an inline
/// child each are 120,000 items, so the outline is refused rather than walked past the budget.
#[test]
fn an_outline_past_the_budget_with_inline_items_is_refused() {
use crate::sanitize::{Hidden, remove_hidden, scan};
let siblings = 60_000;
let mut items = Vec::with_capacity(siblings);
for i in 0..siblings {
let next = if i + 1 < siblings { format!(" /Next {} 0 R", 7 + i) } else { String::new() };
items.push(format!("<< /Title (a){next} /First << /Title (c) >> >>").into_bytes());
}
let mut doc = outlined_document(items);
assert_eq!(scan(&doc).into_iter().find(|c| c.0 == Hidden::LinksActionsScripts), Some((Hidden::LinksActionsScripts, 1)));
let out = remove_hidden(&mut doc, &[Hidden::LinksActionsScripts]);
assert!(matches!(out, Err(RedactError::TooDeep)), "{out:?}");
}
/// Issue #2447: in a tagged PDF the structure tree points at the link and the comment it tags
/// (`/OBJR`). Removing them must drop those references too, or the full rewrite keeps the
/// annotations, the URL and the comment text included, while the page no longer lists them.
#[test]
fn removed_links_and_comments_leave_the_structure_tree_and_the_saved_file() {
use crate::sanitize::{Hidden, remove_hidden};
let objs: Vec<Vec<u8>> = vec![
// 1 catalog, with a structure tree
b"<< /Type /Catalog /Pages 2 0 R /MarkInfo << /Marked true >> /StructTreeRoot 8 0 R >>".to_vec(),
b"<< /Type /Pages /Kids [3 0 R] /Count 1 >>".to_vec(),
// 3 page: a tagged paragraph (MCID 0), a link and a comment, both tagged
b"<< /Type /Page /Parent 2 0 R /MediaBox [0 0 300 300] /Contents 4 0 R /Resources << /Font << /F1 5 0 R >> >> /Annots [6 0 R 7 0 R] /StructParents 0 >>".to_vec(),
stream("", b"/P << /MCID 0 >> BDC BT /F1 10 Tf 10 230 Td (See the site) Tj ET EMC"),
FONT.replace("95 0 R", "13 0 R").into_bytes(),
// 6 link, 7 comment
b"<< /Type /Annot /Subtype /Link /Rect [10 228 80 240] /StructParent 1 /A << /S /URI /URI (https://secret.example/token=ZX81) >> >>".to_vec(),
b"<< /Type /Annot /Subtype /Text /Rect [200 200 220 220] /StructParent 2 /Contents (COMMENTSECRETNOTE) >>".to_vec(),
// 8 struct tree root, 9 document element, 10 paragraph, 11 link element (direct OBJR),
// 12 annotation element (OBJR by reference, as K's only kid)
b"<< /Type /StructTreeRoot /K 9 0 R /ParentTree << /Nums [0 [10 0 R] 1 11 0 R 2 12 0 R] >> /ParentTreeNextKey 3 >>".to_vec(),
b"<< /Type /StructElem /S /Document /P 8 0 R /K [10 0 R 11 0 R 12 0 R] >>".to_vec(),
b"<< /Type /StructElem /S /P /P 9 0 R /Pg 3 0 R /K 0 >>".to_vec(),
// the link element's alternate text repeats the URL
b"<< /Type /StructElem /S /Link /P 9 0 R /Pg 3 0 R /Alt (secret.example link) /K [<< /Type /OBJR /Obj 6 0 R /Pg 3 0 R >>] >>".to_vec(),
b"<< /Type /StructElem /S /Annot /P 9 0 R /Pg 3 0 R /K 14 0 R >>".to_vec(),
widths(),
b"<< /Type /OBJR /Obj 7 0 R /Pg 3 0 R >>".to_vec(),
];
let mut doc = pdf(objs);
let done = remove_hidden(&mut doc, &[Hidden::LinksActionsScripts, Hidden::Comments]).unwrap();
assert!(done.contains(&(Hidden::LinksActionsScripts, 1)) && done.contains(&(Hidden::Comments, 1)), "{done:?}");
// Without object streams, whose compression would hide the strings from the search.
let bytes = write_full(&doc, &SaveOptions { object_streams: false, ..SaveOptions::default() }).unwrap();
let has = |needle: &[u8]| bytes.windows(needle.len()).any(|w| w == needle);
assert!(has(b"(See the site)"), "the search sees the saved objects");
assert!(!has(b"secret.example"), "the removed link's URL survived the save");
assert!(!has(b"COMMENTSECRETNOTE"), "the removed comment's text survived the save");
// The tagged paragraph and its content stay; the elements lose only the references.
let saved = Document::open(Arc::new(bytes)).unwrap();
assert!(content(&saved, 0).contains("(See the site)"));
let objrs = saved.object_numbers().into_iter().filter(|&n| {
let o = saved.get(ObjRef::new(n, saved.generation(n)));
let direct = o.as_dict().is_some_and(|d| d.name(b"Type") == Some(b"OBJR"));
let nested = o.as_dict().and_then(|d| d.get(b"K")).is_some_and(|k| match &*saved.resolve(k) {
Object::Array(a) => a.iter().any(|x| saved.resolve(x).as_dict().is_some_and(|d| d.name(b"Type") == Some(b"OBJR"))),
_ => false,
});
direct || nested
});
assert_eq!(objrs.count(), 0, "no object reference to a removed annotation is left");
let root = saved.get(saved.root().unwrap()).as_dict().cloned().unwrap();
assert!(root.contains(b"StructTreeRoot"), "the structure tree itself stays");
}
/// #2532 review: a tagged form's widgets, flattened when form fields are removed, leave the
/// structure tree too (their field value with them), and a pop-up listed on another page than
/// its removed comment goes with it.
#[test]
fn flattened_widgets_and_far_pop_ups_leave_the_structure_tree() {
use crate::sanitize::{Hidden, remove_hidden};
let objs: Vec<Vec<u8>> = vec![
// 1 catalog with a form and a structure tree
b"<< /Type /Catalog /Pages 2 0 R /AcroForm << /Fields [6 0 R] >> /StructTreeRoot 8 0 R >>".to_vec(),
b"<< /Type /Pages /Kids [3 0 R 13 0 R] /Count 2 >>".to_vec(),
// 3 page 1: a text field's widget and a comment
b"<< /Type /Page /Parent 2 0 R /MediaBox [0 0 300 300] /Contents 4 0 R /Resources << /Font << /F1 5 0 R >> >> /Annots [6 0 R 7 0 R] /StructParents 0 >>".to_vec(),
stream("", b"BT /F1 10 Tf 10 230 Td (Name:) Tj ET"),
FONT.replace("95 0 R", "12 0 R").into_bytes(),
// 6 the field and its widget, with an appearance
b"<< /Type /Annot /Subtype /Widget /FT /Tx /T (name) /V (FIELDSECRETVALUE) /Rect [50 225 200 240] /StructParent 1 /AP << /N 11 0 R >> >>".to_vec(),
// 7 comment, its pop-up (15) listed on page 2
b"<< /Type /Annot /Subtype /Text /Rect [200 200 220 220] /Contents (note) /Popup 15 0 R >>".to_vec(),
// 8 struct tree root, 9 document, 10 form element tagging the widget
b"<< /Type /StructTreeRoot /K 9 0 R /ParentTree << /Nums [1 10 0 R] >> /ParentTreeNextKey 2 >>".to_vec(),
b"<< /Type /StructElem /S /Document /P 8 0 R /K [10 0 R] >>".to_vec(),
b"<< /Type /StructElem /S /Form /P 9 0 R /Pg 3 0 R /K [<< /Type /OBJR /Obj 6 0 R /Pg 3 0 R >>] >>".to_vec(),
stream("/Type /XObject /Subtype /Form /BBox [0 0 150 15] /Resources << /Font << /F1 5 0 R >> >>", b"BT /F1 10 Tf 2 3 Td (FIELDSECRETVALUE) Tj ET"),
widths(),
// 13 page 2, listing the pop-up of page 1's comment
b"<< /Type /Page /Parent 2 0 R /MediaBox [0 0 300 300] /Annots [15 0 R 16 0 R] >>".to_vec(),
b"<< >>".to_vec(),
b"<< /Type /Annot /Subtype /Popup /Rect [0 0 100 50] /Parent 7 0 R >>".to_vec(),
// 16 an unrelated link on page 2 stays
b"<< /Type /Annot /Subtype /Link /Rect [0 0 10 10] >>".to_vec(),
];
let mut doc = pdf(objs);
remove_hidden(&mut doc, &[Hidden::FormFields, Hidden::Comments]).unwrap();
let page2 = pdfcraft_model::pages(&doc)[1].dict.clone();
let left: Vec<Object> = page2.get(b"Annots").map(|a| doc.resolve(a).as_array().map(|a| a.to_vec()).unwrap_or_default()).unwrap_or_default();
assert_eq!(left, [Object::Ref(ObjRef::new(16, 0))], "the far pop-up went with its comment");
let form = doc.get(ObjRef::new(10, 0)).as_dict().cloned().unwrap();
let kids = form.get(b"K").map(|k| doc.resolve(k).as_array().map(Vec::len).unwrap_or(1));
assert!(kids.unwrap_or(0) == 0, "the form element no longer references the flattened widget");
let bytes = write_full(&doc, &SaveOptions { object_streams: false, ..SaveOptions::default() }).unwrap();
let saved = Document::open(Arc::new(bytes)).unwrap();
let field_left = saved.object_numbers().into_iter().any(|n| {
saved.get(ObjRef::new(n, saved.generation(n))).as_dict().is_some_and(|d| d.name(b"Subtype") == Some(b"Widget") || d.contains(b"FT"))
});
assert!(!field_left, "the flattened field didn't survive the save through the structure tree");
}
/// Issue #2874: applying redactions removes the comment and the form field under a mark, and
/// with them the structure tree's object references (`/OBJR`) to them, so a full rewrite of a
/// tagged PDF can't keep the comment's text or the field's value. The field's widget on another
/// page goes with it; a comment outside the mark keeps its reference.
#[test]
fn applied_redactions_drop_structure_tree_references_to_what_they_remove() {
let objs: Vec<Vec<u8>> = vec![
// 1 catalog with a form and a structure tree
b"<< /Type /Catalog /Pages 2 0 R /AcroForm << /Fields [6 0 R] >> /MarkInfo << /Marked true >> /StructTreeRoot 9 0 R >>".to_vec(),
b"<< /Type /Pages /Kids [3 0 R 16 0 R] /Count 2 >>".to_vec(),
// 3 page 1: a label, a text field's widget and a comment under the mark, a comment outside
b"<< /Type /Page /Parent 2 0 R /MediaBox [0 0 300 300] /Contents 4 0 R /Resources << /Font << /F1 5 0 R >> >> /Annots [7 0 R 8 0 R 15 0 R] /StructParents 0 >>".to_vec(),
stream("", b"BT /F1 10 Tf 10 230 Td (Name:) Tj ET"),
FONT.replace("95 0 R", "14 0 R").into_bytes(),
// 6 the field, 7 its widget on page 1 (17 is its widget on page 2), 8 the comment
b"<< /FT /Tx /T (name) /V (FIELDSECRETVALUE) /Kids [7 0 R 17 0 R] >>".to_vec(),
b"<< /Type /Annot /Subtype /Widget /Parent 6 0 R /P 3 0 R /Rect [50 225 200 240] /StructParent 1 /AP << /N 13 0 R >> >>".to_vec(),
b"<< /Type /Annot /Subtype /Text /P 3 0 R /Rect [200 200 220 220] /StructParent 2 /Contents (COMMENTSECRETNOTE) >>".to_vec(),
// 9 struct tree root, 10 document, 11 form element (direct OBJRs to both widgets),
// 12 annotation element (OBJR to the comment), 18 annotation element for the kept comment
b"<< /Type /StructTreeRoot /K 10 0 R /ParentTree << /Nums [1 11 0 R 2 12 0 R 3 18 0 R 4 11 0 R] >> /ParentTreeNextKey 5 >>".to_vec(),
b"<< /Type /StructElem /S /Document /P 9 0 R /K [11 0 R 12 0 R 18 0 R] >>".to_vec(),
b"<< /Type /StructElem /S /Form /P 10 0 R /Pg 3 0 R /Alt (FIELDSECRETVALUE) /K [<< /Type /OBJR /Obj 7 0 R /Pg 3 0 R >> << /Type /OBJR /Obj 17 0 R /Pg 16 0 R >>] >>".to_vec(),
b"<< /Type /StructElem /S /Annot /P 10 0 R /Pg 3 0 R /K [<< /Type /OBJR /Obj 8 0 R /Pg 3 0 R >>] >>".to_vec(),
stream("/Type /XObject /Subtype /Form /BBox [0 0 150 15] /Resources << /Font << /F1 5 0 R >> >>", b"BT /F1 10 Tf 2 3 Td (FIELDSECRETVALUE) Tj ET"),
widths(),
// 15 the comment outside the mark
b"<< /Type /Annot /Subtype /Text /P 3 0 R /Rect [250 10 270 30] /StructParent 3 /Contents (Kept note) >>".to_vec(),
// 16 page 2, with the field's second widget
b"<< /Type /Page /Parent 2 0 R /MediaBox [0 0 300 300] /Annots [17 0 R] /StructParents 4 >>".to_vec(),
b"<< /Type /Annot /Subtype /Widget /Parent 6 0 R /P 16 0 R /Rect [50 50 200 65] /StructParent 4 /AP << /N 13 0 R >> >>".to_vec(),
b"<< /Type /StructElem /S /Annot /P 10 0 R /Pg 3 0 R /K [<< /Type /OBJR /Obj 15 0 R /Pg 3 0 R >>] >>".to_vec(),
];
let mut doc = pdf(objs);
mark(&mut doc, 0, &[[40.0, 190.0, 230.0, 245.0]], "");
let report = apply(&mut doc, None).unwrap();
assert_eq!((report.annotations, report.fields), (1, 1), "{report:?}");
assert!(report.tags >= 3, "the three references to the comment and both widgets are counted: {report:?}");
// Without object streams, whose compression would hide the strings from the search.
let bytes = write_full(&doc, &SaveOptions { object_streams: false, ..SaveOptions::default() }).unwrap();
let has = |needle: &[u8]| bytes.windows(needle.len()).any(|w| w == needle);
assert!(has(b"(Name:)") && has(b"(Kept note)"), "the search sees the saved objects");
assert!(!has(b"COMMENTSECRETNOTE"), "the redacted comment's text survived the save through the structure tree");
assert!(!has(b"FIELDSECRETVALUE"), "the redacted field's value survived the save through the structure tree");
let saved = Document::open(Arc::new(bytes)).unwrap();
let widget_left = saved
.object_numbers()
.into_iter()
.any(|n| saved.get(ObjRef::new(n, saved.generation(n))).as_dict().is_some_and(|d| d.name(b"Subtype") == Some(b"Widget")));
assert!(!widget_left, "neither of the field's widgets survived the save");
// The kept comment is still tagged.
let kept = saved.object_numbers().into_iter().any(|n| {
saved.get(ObjRef::new(n, saved.generation(n))).as_dict().and_then(|d| d.get(b"K")).is_some_and(|k| match &*saved.resolve(k) {
Object::Array(a) => a.iter().any(|x| {
saved.resolve(x).as_dict().and_then(|d| d.get(b"Obj")).and_then(Object::as_ref).is_some_and(|r| {
saved.get(r).as_dict().and_then(|d| d.get(b"Contents")).and_then(Object::as_string).is_some_and(|s| s.to_text() == "Kept note")
})
}),
_ => false,
})
});
assert!(kept, "the comment outside the mark keeps its structure reference");
}
/// #2874: a structure element written in place that tags a comment under a mark can't be edited
/// without rewriting what holds it, so applying is refused rather than leaving the comment
/// reachable (fail-closed, as in Sanitize).
#[test]
fn applying_redactions_refuses_an_in_place_element_tagging_a_removed_comment() {
let objs: Vec<Vec<u8>> = vec![
b"<< /Type /Catalog /Pages 2 0 R /StructTreeRoot 6 0 R >>".to_vec(),
b"<< /Type /Pages /Kids [3 0 R] /Count 1 >>".to_vec(),
b"<< /Type /Page /Parent 2 0 R /MediaBox [0 0 300 300] /Annots [4 0 R] >>".to_vec(),
b"<< /Type /Annot /Subtype /Text /Rect [200 200 220 220] /StructParent 0 /Contents (COMMENTSECRETNOTE) >>".to_vec(),
b"<< >>".to_vec(),
// 6 the root's only kid, an annotation element written in place
b"<< /Type /StructTreeRoot /K [<< /Type /StructElem /S /Annot /K [<< /Type /OBJR /Obj 4 0 R >>] >>] >>".to_vec(),
];
let mut doc = pdf(objs);
mark(&mut doc, 0, &[[190.0, 190.0, 230.0, 230.0]], "");
assert!(matches!(apply(&mut doc, None), Err(RedactError::Tags)));
}
/// #2532 review: a structure tree larger than the walk's limit can't be searched in full, so a
/// reference the walk never reached could keep a removed annotation. Then it is an error, and
/// nothing is changed, rather than success with references left behind (removing only the tree
/// would leave an outline's `/SE` reaching them, review on #3285); under the limit only the
/// reference goes.
#[test]
fn a_structure_tree_too_large_to_walk_is_refused() {
let objs: Vec<Vec<u8>> = vec![
b"<< /Type /Catalog /Pages 2 0 R /MarkInfo << /Marked true >> /StructTreeRoot 5 0 R >>".to_vec(),
b"<< /Type /Pages /Kids [3 0 R] /Count 1 >>".to_vec(),
b"<< /Type /Page /Parent 2 0 R /MediaBox [0 0 300 300] >>".to_vec(),
// 4: the removed link
b"<< /Type /Annot /Subtype /Link /Rect [0 0 10 10] /A << /S /URI /URI (https://secret.example) >> >>".to_vec(),
// 5 root, 6 document, 7 section, 8 link element with the OBJR
b"<< /Type /StructTreeRoot /K 6 0 R >>".to_vec(),
b"<< /Type /StructElem /S /Document /P 5 0 R /K [7 0 R] >>".to_vec(),
b"<< /Type /StructElem /S /Sect /P 6 0 R /K [8 0 R] >>".to_vec(),
b"<< /Type /StructElem /S /Link /P 7 0 R /K [<< /Type /OBJR /Obj 4 0 R >>] >>".to_vec(),
];
let gone: std::collections::HashSet<ObjRef> = [ObjRef::new(4, 0)].into();
let catalog = |doc: &Document| doc.get(doc.root().unwrap()).as_dict().cloned().unwrap();
let mut doc = pdf(objs.clone());
assert_eq!(crate::tags::drop_object_refs_within(&mut doc, &gone, 100).unwrap(), 1);
assert!(catalog(&doc).contains(b"StructTreeRoot") && catalog(&doc).contains(b"MarkInfo"), "under the limit the tree stays");
let mut doc = pdf(objs);
assert!(matches!(crate::tags::drop_object_refs_within(&mut doc, &gone, 2), Err(RedactError::Tags)), "a tree the walk couldn't finish is refused");
assert!(catalog(&doc).contains(b"StructTreeRoot"), "nothing changed");
}
/// #3302: an element listed only in the parent tree (not under the root's /K) still owns the
/// marked content redaction changed, so its /ActualText and /Alt, which repeat the redacted
/// words, go as they do for an element under /K.
#[test]
fn an_element_listed_only_in_the_parent_tree_loses_its_redacted_text() {
let objs: Vec<Vec<u8>> = vec![
b"<< /Type /Catalog /Pages 2 0 R /StructTreeRoot 5 0 R /MarkInfo << /Marked true >> >>".to_vec(),
b"<< /Type /Pages /Kids [3 0 R] /Count 1 >>".to_vec(),
b"<< /Type /Page /Parent 2 0 R /MediaBox [0 0 300 300] /Contents 4 0 R /StructParents 0 /Resources << /Font << /F1 9 0 R >> >> >>".to_vec(),
stream("", b"/P <</MCID 0>> BDC BT /F1 14 Tf 20 150 Td (Codename Bluefin) Tj ET EMC"),
// The root's /K reaches only the empty document element 6; the paragraph 7 is in the
// parent tree only.
b"<< /Type /StructTreeRoot /K 6 0 R /ParentTree << /Nums [0 [7 0 R]] >> /ParentTreeNextKey 1 >>".to_vec(),
b"<< /Type /StructElem /S /Document /P 5 0 R >>".to_vec(),
b"<< /Type /StructElem /S /P /P 6 0 R /Pg 3 0 R /K 0 /ActualText (Codename Bluefin) /Alt (Codename Bluefin) >>".to_vec(),
b"<< /Type /StructElem /S /Note /P 6 0 R >>".to_vec(),
b"<< /Type /Font /Subtype /Type1 /BaseFont /Helvetica /Encoding /WinAnsiEncoding >>".to_vec(),
];
let mut doc = pdf(objs);
mark(&mut doc, 0, &[[10.0, 140.0, 290.0, 170.0]], "");
let r = apply(&mut doc, None).unwrap();
assert!(r.glyphs > 0 && r.tags >= 1, "{r:?}");
let element = doc.get(ObjRef::new(7, 0)).as_dict().cloned().unwrap();
assert!(!element.contains(b"ActualText") && !element.contains(b"Alt"), "{element:?}");
}
/// #3302: a structure tree too large to walk in full fails closed: an error, so the redaction
/// isn't made, instead of the walk stopping and reporting success with elements never checked.
#[test]
fn a_tag_walk_cut_short_is_refused() {
let objs: Vec<Vec<u8>> = vec![
b"<< /Type /Catalog /Pages 2 0 R /StructTreeRoot 5 0 R >>".to_vec(),
b"<< /Type /Pages /Kids [3 0 R] /Count 1 >>".to_vec(),
b"<< /Type /Page /Parent 2 0 R /MediaBox [0 0 300 300] /StructParents 0 >>".to_vec(),
b"<< >>".to_vec(),
b"<< /Type /StructTreeRoot /K 6 0 R >>".to_vec(),
b"<< /Type /StructElem /S /Document /P 5 0 R /K [7 0 R 8 0 R] >>".to_vec(),
b"<< /Type /StructElem /S /P /P 6 0 R /Pg 3 0 R /K 0 /ActualText (Codename Bluefin) >>".to_vec(),
b"<< /Type /StructElem /S /Figure /P 6 0 R /Alt (A chart) /E (chart) >>".to_vec(),
];
let page = ObjRef::new(3, 0);
let changed: std::collections::HashSet<i64> = [0].into();
let mut doc = pdf(objs.clone());
assert_eq!(crate::tags::clean_within(&mut doc, page, None, &changed, &Default::default(), 100).unwrap(), 1);
let figure = doc.get(ObjRef::new(8, 0)).as_dict().cloned().unwrap();
assert!(figure.contains(b"Alt"), "a walk that finishes keeps an unrelated element's text");
let mut doc = pdf(objs);
assert!(matches!(crate::tags::clean_within(&mut doc, page, None, &changed, &Default::default(), 1), Err(RedactError::Tags)));
}
/// #2532 follow-up: an element reached only through the parent tree or the ID tree, not from
/// the root's /K, still points at a removed annotation, and the root keeps it reachable in a full
/// rewrite. Both trees are walked too, written in place or as their own objects.
#[test]
fn elements_listed_only_in_the_parent_tree_or_the_id_tree_lose_their_references() {
let objs: Vec<Vec<u8>> = vec![
b"<< /Type /Catalog /Pages 2 0 R /MarkInfo << /Marked true >> /StructTreeRoot 5 0 R >>".to_vec(),
b"<< /Type /Pages /Kids [3 0 R] /Count 1 >>".to_vec(),
b"<< /Type /Page /Parent 2 0 R /MediaBox [0 0 300 300] >>".to_vec(),
// 4: the removed link; 8: the removed comment
b"<< /Type /Annot /Subtype /Link /Rect [0 0 10 10] /A << /S /URI /URI (https://secret.example) >> >>".to_vec(),
// 5: the root; its /K reaches only the empty document element 6. The parent tree is
// written in place, the ID tree's kid (9) as its own object.
b"<< /Type /StructTreeRoot /K 6 0 R /ParentTree << /Nums [0 7 0 R] >> /IDTree << /Kids [9 0 R] >> >>".to_vec(),
b"<< /Type /StructElem /S /Document /P 5 0 R >>".to_vec(),
// 7: a link element only the parent tree lists
b"<< /Type /StructElem /S /Link /P 6 0 R /K << /Type /OBJR /Obj 4 0 R >> /Alt (secret.example link) >>".to_vec(),
b"<< /Type /Annot /Subtype /Text /Rect [0 0 10 10] /Contents (hushhush note) >>".to_vec(),
b"<< /Names [(note1) 10 0 R] >>".to_vec(),
// 10: an annotation element only the ID tree lists
b"<< /Type /StructElem /S /Annot /P 6 0 R /ID (note1) /K [<< /Type /OBJR /Obj 8 0 R >>] >>".to_vec(),
];
let gone: std::collections::HashSet<ObjRef> = [ObjRef::new(4, 0), ObjRef::new(8, 0)].into();
let mut doc = pdf(objs);
assert_eq!(crate::tags::drop_object_refs(&mut doc, &gone).unwrap(), 2, "both references go");
let bytes = write_full(&doc, &SaveOptions { object_streams: false, ..SaveOptions::default() }).unwrap();
for secret in [&b"secret.example"[..], b"hushhush"] {
assert!(!bytes.windows(secret.len()).any(|w| w == secret), "{} survived the rewrite", String::from_utf8_lossy(secret));
}
}
/// #3285 review: an element without its own /Pg takes its ancestor's page, also when the parent
/// tree lists it. One under /K must not be reached first through the parent tree without that
/// page, and one only the parent tree lists finds it through /P.
#[test]
fn an_element_inheriting_its_page_loses_its_redacted_text_when_the_parent_tree_lists_it() {
let objs: Vec<Vec<u8>> = vec![
b"<< /Type /Catalog /Pages 2 0 R /StructTreeRoot 5 0 R /MarkInfo << /Marked true >> >>".to_vec(),
b"<< /Type /Pages /Kids [3 0 R] /Count 1 >>".to_vec(),
b"<< /Type /Page /Parent 2 0 R /MediaBox [0 0 300 300] /Contents 4 0 R /StructParents 0 /Resources << /Font << /F1 9 0 R >> >> >>".to_vec(),
stream("", b"/P <</MCID 0>> BDC BT /F1 14 Tf 20 150 Td (Codename Bluefin) Tj ET EMC /P <</MCID 1>> BDC BT /F1 14 Tf 20 100 Td (Codename Bluefin) Tj ET EMC"),
// The document element 6 carries the page; 7 sits under it, 8 only points at it.
b"<< /Type /StructTreeRoot /K 6 0 R /ParentTree << /Nums [0 [7 0 R 8 0 R]] >> /ParentTreeNextKey 1 >>".to_vec(),
b"<< /Type /StructElem /S /Document /P 5 0 R /Pg 3 0 R /K [7 0 R] >>".to_vec(),
b"<< /Type /StructElem /S /P /P 6 0 R /K 0 /ActualText (Codename Bluefin) >>".to_vec(),
b"<< /Type /StructElem /S /P /P 6 0 R /K 1 /Alt (Codename Bluefin) >>".to_vec(),
b"<< /Type /Font /Subtype /Type1 /BaseFont /Helvetica /Encoding /WinAnsiEncoding >>".to_vec(),
];
let mut doc = pdf(objs);
mark(&mut doc, 0, &[[10.0, 90.0, 290.0, 170.0]], "");
let r = apply(&mut doc, None).unwrap();
assert!(r.glyphs > 0, "{r:?}");
for n in [7, 8] {
let element = doc.get(ObjRef::new(n, 0)).as_dict().cloned().unwrap();
assert!(!element.contains(b"ActualText") && !element.contains(b"Alt"), "{n}: {element:?}");
}
}
/// #3285 review: a structure element written in place, here as an ID tree value, can't be
/// edited without rewriting what holds it; when it tags a removed annotation, removing hidden
/// information is refused instead of keeping the annotation reachable in a full rewrite.
/// Direct callers discard the failed working copy to preserve the original document.
#[test]
fn an_element_written_in_place_that_tags_a_removed_annotation_is_refused() {
let objs: Vec<Vec<u8>> = vec![
b"<< /Type /Catalog /Pages 2 0 R /MarkInfo << /Marked true >> /StructTreeRoot 5 0 R >>".to_vec(),
b"<< /Type /Pages /Kids [3 0 R] /Count 1 >>".to_vec(),
b"<< /Type /Page /Parent 2 0 R /MediaBox [0 0 300 300] /Annots [4 0 R] >>".to_vec(),
b"<< /Type /Annot /Subtype /Text /Rect [0 0 10 10] /Contents (hushhush note) >>".to_vec(),
b"<< /Type /StructTreeRoot /K 6 0 R /IDTree << /Names [(note1) << /Type /StructElem /S /Annot /P 6 0 R /K [<< /Type /OBJR /Obj 4 0 R >>] >>] >> >>".to_vec(),
b"<< /Type /StructElem /S /Document /P 5 0 R >>".to_vec(),
];
let original = pdf(objs);
let before = write_full(&original, &SaveOptions::default()).unwrap();
let mut working = original.clone();
assert!(matches!(sanitize::remove_hidden(&mut working, &[sanitize::Hidden::Comments]), Err(RedactError::Tags)));
assert!(!working.get(ObjRef::new(3, 0)).as_dict().unwrap().contains(b"Annots"), "the failed working copy was partially changed");
drop(working);
assert_eq!(write_full(&original, &SaveOptions::default()).unwrap(), before, "discarding the working copy preserves the original");
}
/// Whether a full rewrite of `doc` still holds the redacted word.
fn bluefin_survives(doc: &Document) -> bool {
let bytes = write_full(doc, &SaveOptions { object_streams: false, ..SaveOptions::default() }).unwrap();
bytes.windows(7).any(|w| w == b"Bluefin")
}
/// #3285 review (the maintainer's probes): an element written in place in the parent tree that
/// owns the redacted text can't be changed, so applying is refused; the same element as an object
/// of its own is cleaned, and nothing of the word is left in a full rewrite. A refusal leaves the
/// document as it was.
#[test]
fn an_element_written_in_place_owning_redacted_text_is_refused() {
let objs = |element: &str| -> Vec<Vec<u8>> {
vec![
b"<< /Type /Catalog /Pages 2 0 R /StructTreeRoot 5 0 R /MarkInfo << /Marked true >> >>".to_vec(),
b"<< /Type /Pages /Kids [3 0 R] /Count 1 >>".to_vec(),
b"<< /Type /Page /Parent 2 0 R /MediaBox [0 0 300 300] /Contents 4 0 R /StructParents 0 /Resources << /Font << /F1 9 0 R >> >> >>"
.to_vec(),
stream("", b"/P <</MCID 0>> BDC BT /F1 14 Tf 20 150 Td (Codename Bluefin) Tj ET EMC"),
format!("<< /Type /StructTreeRoot /K 6 0 R /ParentTree << /Nums [0 [{element}]] >> /ParentTreeNextKey 1 >>").into_bytes(),
b"<< /Type /StructElem /S /Document /P 5 0 R >>".to_vec(),
b"<< /Type /StructElem /S /P /P 6 0 R /Pg 3 0 R /K 0 /ActualText (Codename Bluefin) >>".to_vec(),
b"<< >>".to_vec(),
b"<< /Type /Font /Subtype /Type1 /BaseFont /Helvetica /Encoding /WinAnsiEncoding >>".to_vec(),
]
};
let mut doc = pdf(objs("<< /Type /StructElem /S /P /P 6 0 R /Pg 3 0 R /K 0 /ActualText (Codename Bluefin) >>"));
mark(&mut doc, 0, &[[10.0, 140.0, 290.0, 170.0]], "");
let before = write_full(&doc, &SaveOptions::default()).unwrap();
let error = apply(&mut doc, None).unwrap_err();
assert!(matches!(error, RedactError::Tags));
assert_eq!(
error.to_string(),
"the document's tags can't be cleaned completely (an element written in place, or a structure tree too large or too deep to follow); the operation was refused"
);
assert_eq!(write_full(&doc, &SaveOptions::default()).unwrap(), before, "the refused document is unchanged");
let mut doc = pdf(objs("7 0 R"));
mark(&mut doc, 0, &[[10.0, 140.0, 290.0, 170.0]], "");
let r = apply(&mut doc, None).unwrap();
assert!(r.glyphs > 0 && !bluefin_survives(&doc), "{r:?}");
}
/// #3285 review: an element whose inherited page can't be found (more ancestors than the lookup
/// follows) isn't taken for one with no page: applying is refused. Within the limit the page is
/// found and the text goes.
#[test]
fn an_element_whose_inherited_page_is_out_of_reach_is_refused() {
let objs = |depth: u32| -> Vec<Vec<u8>> {
let mut objs: Vec<Vec<u8>> = vec![
b"<< /Type /Catalog /Pages 2 0 R /StructTreeRoot 5 0 R /MarkInfo << /Marked true >> >>".to_vec(),
b"<< /Type /Pages /Kids [3 0 R] /Count 1 >>".to_vec(),
b"<< /Type /Page /Parent 2 0 R /MediaBox [0 0 300 300] /Contents 4 0 R /StructParents 0 /Resources << /Font << /F1 9 0 R >> >> >>"
.to_vec(),
stream("", b"/P <</MCID 0>> BDC BT /F1 14 Tf 20 150 Td (Codename Bluefin) Tj ET EMC"),
b"<< /Type /StructTreeRoot /K 6 0 R /ParentTree << /Nums [0 [7 0 R]] >> /ParentTreeNextKey 1 >>".to_vec(),
b"<< /Type /StructElem /S /Document /P 5 0 R /Pg 3 0 R >>".to_vec(),
format!("<< /Type /StructElem /S /P /P {} 0 R /K 0 /ActualText (Codename Bluefin) >>", 10 + depth - 1).into_bytes(),
b"<< >>".to_vec(),
b"<< /Type /Font /Subtype /Type1 /BaseFont /Helvetica /Encoding /WinAnsiEncoding >>".to_vec(),
];
for i in 0..depth {
let parent = if i == 0 { 6 } else { 10 + i - 1 };
objs.push(format!("<< /Type /StructElem /S /Div /P {parent} 0 R >>").into_bytes());
}
objs
};
let mut doc = pdf(objs(300));
mark(&mut doc, 0, &[[10.0, 140.0, 290.0, 170.0]], "");
assert!(matches!(apply(&mut doc, None), Err(RedactError::Tags)));
let mut doc = pdf(objs(100));
mark(&mut doc, 0, &[[10.0, 140.0, 290.0, 170.0]], "");
apply(&mut doc, None).unwrap();
assert!(!bluefin_survives(&doc));
}
/// Issue #1325: removing comments keeps the annotations that are not comments: the multimedia
/// and print-production ones (Watermark, PrinterMark, TrapNet, Screen, Movie, RichMedia, 3D).
#[test]
fn removing_comments_keeps_annotations_that_are_not_comments() {
use crate::sanitize::{Hidden, remove_hidden, scan};
let kept = ["Link", "Watermark", "PrinterMark", "TrapNet", "Screen", "Movie", "RichMedia", "3D"];
let annots: Vec<String> = (0..=kept.len()).map(|i| format!("{} 0 R", i + 4)).collect();
let mut objs = vec![
b"<< /Type /Catalog /Pages 2 0 R >>".to_vec(),
b"<< /Type /Pages /Kids [3 0 R] /Count 1 >>".to_vec(),
format!("<< /Type /Page /Parent 2 0 R /MediaBox [0 0 300 300] /Resources << >> /Annots [{}] >>", annots.join(" ")).into_bytes(),
b"<< /Type /Annot /Subtype /Square /Rect [10 10 50 50] >>".to_vec(),
];
objs.extend(kept.iter().map(|s| format!("<< /Type /Annot /Subtype /{s} /Rect [60 60 90 90] >>").into_bytes()));
let mut doc = pdf(objs);
let counts: std::collections::HashMap<Hidden, usize> = scan(&doc).into_iter().collect();
assert_eq!(counts[&Hidden::Comments], 1, "only the Square");
assert_eq!(remove_hidden(&mut doc, &[Hidden::Comments]).unwrap(), [(Hidden::Comments, 1)]);
let doc = reopen(&doc);
let p = pdfcraft_model::pages(&doc).swap_remove(0);
let left: Vec<String> = annots_of(&doc, &p.dict)
.iter()
.filter_map(|a| doc.resolve(a).as_dict().and_then(|d| d.name(b"Subtype")).map(|s| String::from_utf8_lossy(s).into_owned()))
.collect();
assert_eq!(left, kept);
}
#[test]
fn overlay_text_takes_its_font_size_colour_alignment_and_repeats() {
let mut doc = one_page(b"BT /F1 12 Tf 20 250 Td (Secret salary figures) Tj ET", "", Vec::new());
let look = pdfcraft_annot::OverlayLook { font: pdfcraft_annot::OverlayFont::Courier, size: 8.0, color: [0.0, 0.0, 1.0], align: 0, repeat: true };
let shape = Shape::Redact { quads: vec![rect_quad([10.0, 200.0, 290.0, 270.0])], overlay: "REDACTED".into(), look };
let style = Style::default_for(&shape);
add_annotation(&mut doc, &NewAnnotation { page: 0, shape, style, contents: String::new(), author: "T".into() }, &Meta::default()).unwrap();
// Written as Acrobat writes it.
let m = &marks(&doc)[0];
assert_eq!(m.look, look, "the look round-trips through /DA, /Q and /Repeat");
apply(&mut doc, None).unwrap();
let doc = reopen(&doc);
let c = content(&doc, 0);
assert!(c.contains("0 0 1 rg /PCCour 8 Tf"), "{c}");
// 70 pt high at 8 pt × 1.2 leading: several lines, each the word repeated.
assert!(c.matches(" Tm (REDACTED REDACTED").count() >= 5, "{c}");
assert!(c.contains("1 0 0 1 11 "), "left aligned at the area's edge: {c}");
let p = &pdfcraft_model::pages(&doc)[0];
let fonts = p
.dict
.get(b"Resources")
.and_then(|r| doc.resolve(r).as_dict().cloned())
.and_then(|r| r.get(b"Font").and_then(|f| doc.resolve(f).as_dict().cloned()))
.unwrap();
assert!(fonts.contains(b"PCCour") && !fonts.contains(b"PCTimes"));
}
fn repeated_overlay_document(width: f64, height: f64, repeat: bool) -> Document {
pdf(vec![
b"<< /Type /Catalog /Pages 2 0 R >>".to_vec(),
b"<< /Type /Pages /Kids [3 0 R] /Count 1 >>".to_vec(),
b"<< /Type /Page /Parent 2 0 R /MediaBox [0 0 600 800] /Contents 4 0 R /Annots [5 0 R] >>".to_vec(),
stream("", b"0 0 20 20 re f"),
format!("<< /Type /Annot /Subtype /Redact /Rect [0 0 {width} {height}] /IC [0 0 0] /OverlayText (X) /DA (/Helv 2 Tf) /Repeat {repeat} >>")
.into_bytes(),
])
}
#[test]
fn oversized_repeated_overlay_is_refused_before_changing_the_document() {
// About 11 MiB of generated content before the fix; a bounded reproduction, not an OOM test.
let mut doc = repeated_overlay_document(16000.0, 1600.0, true);
let before = write_full(&doc, &SaveOptions::default()).unwrap();
assert_eq!(apply(&mut doc, None), Err(RedactError::OverlayLimit));
assert_eq!(write_full(&doc, &SaveOptions::default()).unwrap(), before);
assert_eq!(marks(&doc).len(), 1);
let mut single = repeated_overlay_document(16000.0, 1600.0, false);
apply(&mut single, None).unwrap();
assert_eq!(content(&reopen(&single), 0).matches("(X) Tj").count(), 1);
}
#[test]
fn overlay_budget_is_shared_by_pages_and_respects_the_page_selection() {
let mut doc = repeated_overlay_document(8000.0, 800.0, true);
let page = (*doc.get(ObjRef::new(3, 0))).clone();
let mut kids = vec![Object::Ref(ObjRef::new(3, 0))];
for _ in 0..2 {
let mut page = page.as_dict().unwrap().clone();
let annotation = doc.add((*doc.get(ObjRef::new(5, 0))).clone());
page.set(b"Annots".to_vec(), Object::Array(vec![Object::Ref(annotation)]));
kids.push(Object::Ref(doc.add(Object::Dict(page))));
}
doc.update_dict(ObjRef::new(2, 0), |d| {
d.set(b"Kids".to_vec(), Object::Array(kids));
d.set(b"Count".to_vec(), Object::Int(3));
})
.unwrap();
let before = write_full(&doc, &SaveOptions::default()).unwrap();
// Each appearance fits on its own; their sum exceeds the operation's budget.
assert_eq!(apply(&mut doc, None), Err(RedactError::OverlayLimit));
assert_eq!(write_full(&doc, &SaveOptions::default()).unwrap(), before);
assert_eq!(marks(&doc).len(), 3);
apply(&mut doc, Some(&[1])).unwrap();
assert_eq!(marks(&doc).iter().map(|m| m.page).collect::<Vec<_>>(), [0, 2]);
}
#[test]
fn grouped_marks_keep_overlay_order_and_ignore_duplicate_page_selections() {
let mut doc = repeated_overlay_document(60.0, 30.0, false);
let page = doc.get(ObjRef::new(3, 0)).as_dict().unwrap().clone();
let original_mark = (*doc.get(ObjRef::new(5, 0))).clone();
let mut kids = vec![Object::Ref(ObjRef::new(3, 0))];
for _ in 0..2 {
let mut copy = page.clone();
let mark = doc.add(original_mark.clone());
copy.set(b"Annots".to_vec(), Object::Array(vec![Object::Ref(mark)]));
kids.push(Object::Ref(doc.add(Object::Dict(copy))));
}
doc.update_dict(ObjRef::new(2, 0), |d| {
d.set(b"Kids".to_vec(), Object::Array(kids));
d.set(b"Count".to_vec(), Object::Int(3));
})
.unwrap();
let mut second = original_mark.as_dict().unwrap().clone();
second.set(b"OverlayText".to_vec(), Object::String(pdfcraft_cos::PdfString::literal(b"Y".to_vec())));
let second = doc.add(Object::Dict(second));
doc.update_dict(ObjRef::new(3, 0), |d| {
d.set(b"Annots".to_vec(), Object::Array(vec![Object::Ref(ObjRef::new(5, 0)), Object::Ref(second)]));
})
.unwrap();
let untouched = content(&doc, 1);
let report = apply(&mut doc, Some(&[2, 0, 2])).unwrap();
assert_eq!((report.pages, report.marks), (2, 3));
let saved = reopen(&doc);
assert_eq!(marks(&saved).iter().map(|m| m.page).collect::<Vec<_>>(), [1]);
assert_eq!(content(&saved, 1), untouched);
let first = content(&saved, 0);
assert!(first.find("(X) Tj").unwrap() < first.find("(Y) Tj").unwrap());
assert_eq!(content(&saved, 2).matches("(X) Tj").count(), 1);
}
#[test]
fn overlay_extreme_counts_and_invalid_geometry_are_refused_without_large_allocations() {
let doc = repeated_overlay_document(20.0, 10.0, true);
let original = marks(&doc).remove(0);
for rect in [[0.0, 0.0, 1e100, 10.0], [0.0, 0.0, 20.0, 1e100], [-f64::MAX, 0.0, f64::MAX, 10.0], [0.0, 0.0, f64::NAN, 10.0]] {
let mut mark = original.clone();
mark.rects = vec![rect];
let mut remaining = overlay::MAX_BYTES;
assert_eq!(overlay::content(&[&mark], &mut remaining), Err(RedactError::OverlayLimit));
}
}
#[test]
fn repeated_overlay_keeps_its_exact_appearance_and_counts_escaped_bytes() {
let doc = repeated_overlay_document(20.0, 10.0, true);
let mut mark = marks(&doc).remove(0);
let expected = b"q 0 0 0 rg\n0 0 20 10 re f\nQ\nq 0 0 20 10 re W n BT 1 0 0 rg /PCHelv 2 Tf 1 0 0 1 1.773 7.9 Tm (X X X X X X X X X) Tj 1 0 0 1 1.773 5.5 Tm (X X X X X X X X X) Tj 1 0 0 1 1.773 3.1 Tm (X X X X X X X X X) Tj 1 0 0 1 1.773 0.7 Tm (X X X X X X X X X) Tj ET Q\n";
let mut remaining = expected.len();
assert_eq!(overlay::content(&[&mark], &mut remaining).unwrap(), expected);
assert_eq!(remaining, 0);
mark.overlay = "é(\\)".into();
let mut remaining = overlay::MAX_BYTES;
let encoded = overlay::content(&[&mark], &mut remaining).unwrap();
let escaped = b"\xe9\\(\\\\\\)";
assert!(encoded.windows(escaped.len()).any(|w| w == escaped));
assert_eq!(overlay::MAX_BYTES - remaining, encoded.len());
let mut too_small = encoded.len() - 1;
assert_eq!(overlay::content(&[&mark], &mut too_small), Err(RedactError::OverlayLimit));
}
#[test]
#[ignore = "manual overlay timing probe"]
fn repeated_overlay_timing_probe() {
use std::hash::{Hash, Hasher};
for (w, h) in [(20.0, 10.0), (2000.0, 200.0), (4000.0, 400.0), (8000.0, 800.0)] {
let doc = repeated_overlay_document(w, h, true);
let marks = marks(&doc);
let refs: Vec<_> = marks.iter().collect();
let start = std::time::Instant::now();
let mut bytes = Vec::new();
for _ in 0..20 {
let mut remaining = overlay::MAX_BYTES;
bytes = std::hint::black_box(overlay::content(std::hint::black_box(&refs), &mut remaining).unwrap());
}
let elapsed = start.elapsed().as_secs_f64() * 1000.0 / 20.0;
let mut hash = std::collections::hash_map::DefaultHasher::new();
bytes.hash(&mut hash);
println!("{w}x{h}: bytes={} hash={} ms={elapsed:.3}", bytes.len(), hash.finish());
}
}
/// Overlay text the standard fonts can't draw: refused when it is marked, and for a mark another
/// program made, refused when it is applied, before anything is redacted, rather than burnt into
/// the page as `?`.
#[test]
fn overlay_text_the_standard_fonts_cannot_draw_is_refused() {
let mut doc = one_page(b"BT /F1 12 Tf 20 250 Td (Secret salary figures) Tj ET", "", Vec::new());
let look = pdfcraft_annot::OverlayLook::default();
let mark = |overlay: &str| Shape::Redact { quads: vec![rect_quad([10.0, 200.0, 290.0, 270.0])], overlay: overlay.into(), look };
let add = |doc: &mut Document, shape: Shape| {
let style = Style::default_for(&shape);
add_annotation(doc, &NewAnnotation { page: 0, shape, style, contents: String::new(), author: "T".into() }, &Meta::default())
};
let err = add(&mut doc, mark("机密")).unwrap_err().to_string();
assert!(err.contains("\"机\" (U+673A)") && err.contains("overlay text"), "{err}");
assert!(marks(&doc).is_empty());
// A mark whose overlay text was written by another program.
add(&mut doc, mark("REDACTED")).unwrap();
let obj = marks(&doc)[0].obj;
doc.update_dict(obj, |d| d.set(b"OverlayText".to_vec(), pdfcraft_cos::PdfString::text("机密"))).unwrap();
let before = content(&doc, 0);
let err = apply(&mut doc, None).unwrap_err();
assert!(
matches!(&err, RedactError::Undrawable(m) if m.contains("page 1") && m.contains("U+673A") && m.contains("nothing was redacted")),
"{err:?}"
);
assert_eq!(content(&doc, 0), before, "nothing was redacted");
assert_eq!(marks(&doc).len(), 1, "the mark is still there");
// Cleared, it applies.
doc.update_dict(obj, |d| d.set(b"OverlayText".to_vec(), pdfcraft_cos::PdfString::text(""))).unwrap();
apply(&mut doc, None).unwrap();
assert!(!content(&reopen(&doc), 0).contains("Secret"));
}
#[test]
fn redaction_codes_join_in_set_order_and_reject_strangers() {
use crate::codes::{CODE_SETS, CodeSet};
let foia = CodeSet::from_id("foia").unwrap();
assert_eq!(foia.overlay(&["(b)(6)", "(b)(1)(A)", "(b)(6)"]), Ok("(b)(1)(A), (b)(6)".into()));
assert_eq!(foia.overlay(&[]), Ok(String::new()));
assert_eq!(foia.overlay(&["(b)(6)", "(k)(1)"]), Err("(k)(1)"));
let privacy = CodeSet::from_id("privacy-act").unwrap();
assert_eq!(privacy.overlay(&["(k)(7)", "(d)(5)"]), Ok("(d)(5), (k)(7)".into()));
assert_eq!(CodeSet::from_id("gdpr"), None);
assert!(CODE_SETS.iter().all(|s| !s.codes.is_empty() && s.codes.iter().all(|c| !c.is_empty())));
}
#[test]
fn tags_lose_what_redaction_removed() {
let content = b"/P <</MCID 0>> BDC BT /F1 10 Tf 10 200 Td (SECRET) Tj ET EMC /P <</MCID 1>> BDC BT /F1 10 Tf 10 100 Td (Public) Tj ET EMC";
let mut doc = pdf(vec![
b"<< /Type /Catalog /Pages 2 0 R /StructTreeRoot 7 0 R /MarkInfo << /Marked true >> >>".to_vec(),
b"<< /Type /Pages /Kids [3 0 R] /Count 1 >>".to_vec(),
b"<< /Type /Page /Parent 2 0 R /MediaBox [0 0 300 300] /Contents 4 0 R /StructParents 0 /Resources << /Font << /F1 5 0 R >> >> >>".to_vec(),
stream("", content),
FONT.replace("95 0 R", "6 0 R").into_bytes(),
widths(),
b"<< /Type /StructTreeRoot /K 8 0 R >>".to_vec(),
b"<< /S /Document /P 7 0 R /K [9 0 R 10 0 R] >>".to_vec(),
b"<< /S /P /P 8 0 R /Pg 3 0 R /ActualText (SECRET) /Alt (the secret) /K 0 >>".to_vec(),
b"<< /S /P /P 8 0 R /Pg 3 0 R /ActualText (Public) /K 1 >>".to_vec(),
]);
mark(&mut doc, 0, &[[5.0, 195.0, 60.0, 212.0]], "");
let report = apply(&mut doc, None).unwrap();
assert_eq!(report.tags, 1);
let secret = doc.get(pdfcraft_cos::ObjRef::new(9, 0)).as_dict().cloned().unwrap();
assert!(secret.get(b"ActualText").is_none() && secret.get(b"Alt").is_none());
assert!(secret.get(b"K").is_none(), "its marked content is empty now");
let public = doc.get(pdfcraft_cos::ObjRef::new(10, 0)).as_dict().cloned().unwrap();
assert_eq!(public.get(b"K").and_then(Object::as_int), Some(1));
assert!(public.get(b"ActualText").is_some(), "untouched content keeps its tags");
}
fn assert_no_secret(doc: &Document, secret: &[u8]) {
let saved = reopen(doc);
for (r, object) in saved.scan_objects() {
let mut bytes = Vec::new();
pdfcraft_cos::serialize(&object, &mut bytes);
assert!(!bytes.windows(secret.len()).any(|w| w == secret), "secret remains in object {r:?}");
if let Object::Stream(s) = &*object {
// Image codecs, a cleared scan coded as Group 4 or JBIG2, decode through the document.
let data = s.decoded().or_else(|_| decoded_image(&saved, s)).unwrap();
assert!(!data.windows(secret.len()).any(|w| w == secret), "secret remains in stream {r:?}");
}
}
}
#[test]
fn named_mcids_and_indirect_content_references_lose_descriptions() {
let mut doc = one_page(
b"/P /P0 BDC BT /F1 10 Tf 10 200 Td (SECRET) Tj ET EMC /P <</MCID 1>> BDC BT /F1 10 Tf 10 100 Td (Public) Tj ET EMC",
"/Properties << /P0 7 0 R >>",
vec![
b"<< /MCID 0 >>".to_vec(),
b"<< /Type /StructTreeRoot /K 9 0 R >>".to_vec(),
b"<< /S /Document /P 8 0 R /K [10 0 R 11 0 R] >>".to_vec(),
b"<< /S /P /P 9 0 R /Pg 3 0 R /ActualText (LEAK_TAG) /K 12 0 R >>".to_vec(),
b"<< /S /P /P 9 0 R /Pg 3 0 R /ActualText (Public) /K 1 >>".to_vec(),
b"<< /Type /MCR /Pg 3 0 R /MCID 0 >>".to_vec(),
],
);
doc.update_dict(doc.root().unwrap(), |d| d.set(b"StructTreeRoot".to_vec(), Object::Ref(ObjRef::new(8, 0)))).unwrap();
mark(&mut doc, 0, &[[5.0, 195.0, 60.0, 212.0]], "");
assert_eq!(apply(&mut doc, None).unwrap().tags, 1);
assert_no_secret(&doc, b"LEAK_");
assert!(!doc.get(ObjRef::new(10, 0)).as_dict().unwrap().contains(b"K"));
assert_eq!(doc.get(ObjRef::new(11, 0)).as_dict().unwrap().get(b"ActualText").unwrap().as_string().unwrap().to_text(), "Public");
}
#[test]
fn form_tags_are_cleaned_and_surviving_references_point_at_the_copy() {
let mut doc = one_page(
b"/Fm Do",
"/XObject << /Fm 7 0 R >>",
vec![
stream(
"/Type /XObject /Subtype /Form /BBox [0 0 300 300] /StructParents 0 /Resources << /Font << /F1 5 0 R >> /Properties << /P0 << /MCID 0 >> >> >>",
b"/P /P0 BDC BT /F1 10 Tf 10 200 Td (SECRET) Tj ET EMC /P <</MCID 1>> BDC BT /F1 10 Tf 10 100 Td (Public) Tj ET EMC",
),
b"<< /Type /StructTreeRoot /K 9 0 R /ParentTree 14 0 R >>".to_vec(),
b"<< /S /Document /P 8 0 R /K [10 0 R 11 0 R] >>".to_vec(),
b"<< /S /P /P 9 0 R /Pg 3 0 R /ActualText (LEAK_FORM_TAG) /K 12 0 R >>".to_vec(),
b"<< /S /P /P 9 0 R /Pg 3 0 R /ActualText (Public) /K 13 0 R >>".to_vec(),
b"<< /Type /MCR /Pg 3 0 R /Stm 7 0 R /MCID 0 >>".to_vec(),
b"<< /Type /MCR /Pg 3 0 R /Stm 7 0 R /StmOwn 7 0 R /MCID 1 >>".to_vec(),
b"<< /Nums [0 [10 0 R 11 0 R]] >>".to_vec(),
],
);
doc.update_dict(doc.root().unwrap(), |d| d.set(b"StructTreeRoot".to_vec(), Object::Ref(ObjRef::new(8, 0)))).unwrap();
mark(&mut doc, 0, &[[5.0, 195.0, 60.0, 212.0]], "");
apply(&mut doc, None).unwrap();
assert_no_secret(&doc, b"LEAK_");
assert_no_secret(&doc, b"SECRET");
assert!(!doc.get(ObjRef::new(10, 0)).as_dict().unwrap().contains(b"K"));
let reference = doc.get(ObjRef::new(13, 0)).as_dict().unwrap().get(b"Stm").unwrap().as_ref().unwrap();
assert_ne!(reference, ObjRef::new(7, 0));
assert_eq!(doc.get(ObjRef::new(13, 0)).as_dict().unwrap().get(b"StmOwn").unwrap().as_ref(), Some(reference));
let Object::Stream(s) = &*doc.get(reference) else { panic!() };
assert!(String::from_utf8(s.decoded().unwrap()).unwrap().contains("Public"));
assert!(doc.get(ObjRef::new(11, 0)).as_dict().unwrap().contains(b"ActualText"));
}
#[test]
fn inherited_resources_do_not_keep_the_original_form() {
let form = stream("/Type /XObject /Subtype /Form /BBox [0 0 300 300]", b"BT /F1 10 Tf 10 200 Td (LEAK_INHERITED) Tj ET");
let mut doc = one_page(b"/Fm Do", "/XObject << /Fm 7 0 R >>", vec![form]);
let resources = doc.get(ObjRef::new(3, 0)).as_dict().unwrap().get(b"Resources").unwrap().clone();
doc.update_dict(ObjRef::new(2, 0), |d| d.set(b"Resources".to_vec(), resources)).unwrap();
doc.update_dict(ObjRef::new(3, 0), |d| {
d.remove(b"Resources");
})
.unwrap();
mark(&mut doc, 0, &[[0.0, 190.0, 300.0, 220.0]], "");
apply(&mut doc, None).unwrap();
assert_no_secret(&doc, b"LEAK_");
assert!(!doc.get(ObjRef::new(2, 0)).as_dict().unwrap().contains(b"Resources"));
}
#[test]
fn unselected_pages_keep_inherited_forms() {
let form = stream("/Type /XObject /Subtype /Form /BBox [0 0 300 300]", b"BT /F1 10 Tf 10 200 Td (SECRET) Tj ET");
let mut doc = one_page(b"/Fm Do", "/XObject << /Fm 7 0 R >>", vec![form]);
let mut page = doc.get(ObjRef::new(3, 0)).as_dict().unwrap().clone();
let resources = page.remove(b"Resources").unwrap();
let second = doc.add(Object::Dict(page));
doc.update_dict(ObjRef::new(2, 0), |d| {
d.set(b"Resources".to_vec(), resources);
d.set(b"Kids".to_vec(), Object::Array(vec![Object::Ref(ObjRef::new(3, 0)), Object::Ref(second)]));
d.set(b"Count".to_vec(), Object::Int(2));
})
.unwrap();
doc.update_dict(ObjRef::new(3, 0), |d| {
d.remove(b"Resources");
})
.unwrap();
mark(&mut doc, 0, &[[0.0, 190.0, 300.0, 220.0]], "");
apply(&mut doc, Some(&[0])).unwrap();
let mut saved = reopen(&doc);
assert_eq!(under(&mut saved, 0, &[[0.0, 190.0, 300.0, 220.0]]), 0);
assert_eq!(under(&mut saved, 1, &[[0.0, 190.0, 300.0, 220.0]]), 6);
}
#[test]
fn nested_resource_less_forms_keep_live_resources_after_redacting_another_page() {
for own_resources in [false, true] {
for still_drawn in [false, true] {
let original = b"BT /F1 10 Tf 10 200 Td (SECRET) Tj ET";
let mut extra = vec![
stream("/Type /XObject /Subtype /Form /BBox [0 0 300 300]", original),
stream("/Type /XObject /Subtype /Form /BBox [0 0 300 300]", if still_drawn { b"/Fm Do".as_slice() } else { b"q Q".as_slice() }),
];
if own_resources {
extra.push(stream(
"/Type /XObject /Subtype /Form /BBox [0 0 300 300] /Resources << /Font << /F1 5 0 R >> /XObject << /Fm 7 0 R /Relay 8 0 R >> >>",
b"/Relay Do",
));
}
let resources =
if own_resources { "/XObject << /Fm 7 0 R /Relay 8 0 R /Wrapper 9 0 R >>" } else { "/XObject << /Fm 7 0 R /Relay 8 0 R >>" };
let mut doc = one_page(b"/Fm Do", resources, extra);
let mut page = doc.get(ObjRef::new(3, 0)).as_dict().unwrap().clone();
let inherited = page.remove(b"Resources").unwrap();
doc.set(ObjRef::new(3, 0), Object::Dict(page.clone()));
let second_content =
doc.add(Object::Stream(Stream::from_raw(Dict::new(), if own_resources { b"/Wrapper Do".to_vec() } else { b"/Relay Do".to_vec() })));
page.set(b"Contents".to_vec(), Object::Ref(second_content));
let second = doc.add(Object::Dict(page));
doc.update_dict(ObjRef::new(2, 0), |d| {
d.set(b"Resources".to_vec(), inherited);
d.set(b"Kids".to_vec(), Object::Array(vec![Object::Ref(ObjRef::new(3, 0)), Object::Ref(second)]));
d.set(b"Count".to_vec(), Object::Int(2));
})
.unwrap();
let region = [[0.0, 190.0, 300.0, 220.0]];
let surviving = if still_drawn { 6 } else { 0 };
assert_eq!(under(&mut doc, 0, &region), 6, "the marked occurrence is real text");
assert_eq!(under(&mut doc, 1, &region), surviving, "the relay's original nested content is visible");
mark(&mut doc, 0, &region, "");
let report = apply(&mut doc, Some(&[0])).unwrap();
assert_eq!((report.marks, report.glyphs), (1, 6));
assert_eq!(under(&mut doc, 0, &region), 0);
assert_eq!(under(&mut doc, 1, &region), surviving, "redacting another page must preserve live nested content");
// A full saved document must retain both the reachable resource and its content,
// without relying on an old incremental revision's otherwise-unreachable objects.
let mut saved = Document::open(Arc::new(write_full(&doc, &SaveOptions::default()).unwrap())).unwrap();
assert_eq!(under(&mut saved, 0, &region), 0);
assert_eq!(under(&mut saved, 1, &region), surviving);
let page = pdfcraft_model::pages(&saved).swap_remove(1);
let res = saved.resolve(page.dict.get(b"Resources").unwrap());
let xo = saved.resolve(res.as_dict().unwrap().get(b"XObject").unwrap());
let mut holder = xo.as_dict().unwrap().clone();
if own_resources {
let wrapper = saved.resolve(holder.get(b"Wrapper").unwrap());
let Object::Stream(wrapper) = &*wrapper else { panic!("a real wrapper form") };
let res = saved.resolve(wrapper.dict.get(b"Resources").unwrap());
let xo = saved.resolve(res.as_dict().unwrap().get(b"XObject").unwrap());
holder = xo.as_dict().unwrap().clone();
}
assert_eq!(holder.contains(b"Fm"), still_drawn, "keep only resources a surviving nested form actually uses");
if still_drawn {
let form = saved.resolve(holder.get(b"Fm").unwrap());
let Object::Stream(form) = &*form else { panic!("the preserved original form") };
assert_eq!(form.decoded().unwrap(), original);
}
}
}
}
/// A page tree of `n` pages that all draw form /Fm and inherit a direct `/Resources` from the root.
fn inheriting_pages(n: usize) -> (Document, Vec<ObjRef>) {
let form = stream("/Type /XObject /Subtype /Form /BBox [0 0 300 300]", b"BT /F1 10 Tf 10 200 Td (SECRET) Tj ET");
let mut doc = one_page(b"/Fm Do", "/XObject << /Fm 7 0 R >>", vec![form]);
let mut page = doc.get(ObjRef::new(3, 0)).as_dict().unwrap().clone();
let resources = page.remove(b"Resources").unwrap();
doc.set(ObjRef::new(3, 0), Object::Dict(page.clone()));
let mut kids = vec![ObjRef::new(3, 0)];
kids.extend((1..n).map(|_| doc.add(Object::Dict(page.clone()))));
doc.update_dict(ObjRef::new(2, 0), |d| {
d.set(b"Resources".to_vec(), resources);
d.set(b"Kids".to_vec(), Object::Array(kids.iter().map(|r| Object::Ref(*r)).collect()));
d.set(b"Count".to_vec(), Object::Int(n as i64));
})
.unwrap();
let saved = reopen(&doc);
(saved, kids)
}
#[test]
fn redacting_one_page_leaves_the_other_pages_untouched() {
let (mut doc, kids) = inheriting_pages(50);
mark(&mut doc, 0, &[[0.0, 190.0, 300.0, 220.0]], "");
let before = doc.modified_objects();
apply(&mut doc, Some(&[0])).unwrap();
let changed: Vec<u32> = doc.modified_objects().into_iter().filter(|n| !before.contains(n)).collect();
for kid in &kids[1..] {
assert!(!changed.contains(&kid.num), "unredacted page {kid:?} was rewritten");
}
// The page, its new content and resources, the cleaned form, the parent and one shared
// resources object: not one object per page.
assert!(changed.len() < 10, "{changed:?}");
// The parent keeps the resources the other pages draw with, through a single shared object.
let parent = doc.get(ObjRef::new(2, 0)).as_dict().unwrap().clone();
assert!(parent.get(b"Resources").and_then(Object::as_ref).is_some());
let mut saved = reopen(&doc);
assert_eq!(under(&mut saved, 0, &[[0.0, 190.0, 300.0, 220.0]]), 0);
assert_eq!(under(&mut saved, 49, &[[0.0, 190.0, 300.0, 220.0]]), 6);
}
#[test]
fn redacting_every_inheriting_page_retires_the_parent_resources() {
let (mut doc, kids) = inheriting_pages(20);
for page in 0..kids.len() {
mark(&mut doc, page, &[[0.0, 190.0, 300.0, 220.0]], "");
}
apply(&mut doc, None).unwrap();
assert!(!doc.get(ObjRef::new(2, 0)).as_dict().unwrap().contains(b"Resources"));
assert_no_secret(&doc, b"SECRET");
let mut saved = reopen(&doc);
for page in 0..kids.len() {
assert!(pdfcraft_model::pages(&saved)[page].dict.contains(b"Resources"));
assert_eq!(under(&mut saved, page, &[[0.0, 190.0, 300.0, 220.0]]), 0);
}
}
#[test]
fn replacement_text_and_property_resources_do_not_survive_redaction() {
let mut doc = one_page(
b"/Span /P0 BDC /Span <</ActualText (LEAK_INLINE)>> BDC BT /F1 10 Tf 10 200 Td (SECRET)",
"/Properties << /P0 7 0 R >>",
vec![
b"<< /MCID 0 /ActualText (LEAK_PROPERTY) /Alt (LEAK_ALT) /E (LEAK_EXPANSION) >>".to_vec(),
stream("", b"Tj ET EMC EMC /Span <</ActualText (Public point)>> DP BT /F1 10 Tf 10 100 Td (Public) Tj ET"),
],
);
doc.update_dict(ObjRef::new(3, 0), |d| {
d.set(b"Contents".to_vec(), Object::Array(vec![Object::Ref(ObjRef::new(4, 0)), Object::Ref(ObjRef::new(8, 0))]));
})
.unwrap();
mark(&mut doc, 0, &[[5.0, 195.0, 60.0, 212.0]], "");
apply(&mut doc, None).unwrap();
assert_no_secret(&doc, b"LEAK_");
let saved = reopen(&doc);
assert!(content(&saved, 0).contains("Public"));
let p = pdfcraft_model::pages(&saved).swap_remove(0);
assert!(!saved.resolve(p.dict.get(b"Resources").unwrap()).as_dict().unwrap().contains(b"Properties"));
}
#[test]
fn inherited_form_properties_lose_replacement_text() {
let mut doc = one_page(
b"/Fm Do",
"/XObject << /Fm 7 0 R >> /Properties << /P0 8 0 R >>",
vec![
stream(
"/Type /XObject /Subtype /Form /BBox [0 0 300 300]",
b"/Span /P0 BDC BT /F1 10 Tf 10 200 Td (SECRET) Tj ET EMC BT /F1 10 Tf 10 100 Td (Public) Tj ET",
),
b"<< /ActualText (LEAK_FORM_PROPERTY) >>".to_vec(),
],
);
mark(&mut doc, 0, &[[5.0, 195.0, 60.0, 212.0]], "");
apply(&mut doc, None).unwrap();
assert_no_secret(&doc, b"LEAK_");
assert_no_secret(&doc, b"SECRET");
let mut saved = reopen(&doc);
assert_eq!(under(&mut saved, 0, &[[0.0, 90.0, 100.0, 110.0]]), 6);
}
#[test]
fn properties_used_by_unmodified_inheriting_forms_stay() {
let mut doc = one_page(
b"BT /F1 10 Tf 10 200 Td (SECRET) Tj ET /Fm Do",
"/XObject << /Fm 7 0 R >> /Properties << /P0 8 0 R >>",
vec![stream("/Type /XObject /Subtype /Form /BBox [0 0 10 10]", b"/Span /P0 BDC 0 0 10 10 re f EMC"), b"<< /ActualText (Public) >>".to_vec()],
);
mark(&mut doc, 0, &[[5.0, 195.0, 60.0, 212.0]], "");
apply(&mut doc, None).unwrap();
let saved = reopen(&doc);
let page = pdfcraft_model::pages(&saved).swap_remove(0);
let resources = saved.resolve(page.dict.get(b"Resources").unwrap());
let properties = saved.resolve(resources.as_dict().unwrap().get(b"Properties").unwrap());
let property = saved.resolve(properties.as_dict().unwrap().get(b"P0").unwrap());
assert_eq!(property.as_dict().unwrap().get(b"ActualText").unwrap().as_string().unwrap().to_text(), "Public");
}
#[test]
fn untouched_replacement_descriptions_survive_redaction_and_hidden_text_cleanup() {
for hidden in [false, true] {
let mode = if hidden { 3 } else { 0 };
let data = format!(
"/Span <</ActualText (LEAK_REMOVED)>> BDC BT /F1 10 Tf {mode} Tr 10 200 Td (SECRET) Tj ET EMC \
/Span /Public BDC BT /F1 10 Tf 0 Tr 10 100 Td (Public) Tj ET EMC"
);
let mut doc = one_page(
data.as_bytes(),
"/Properties << /Public 7 0 R >>",
vec![b"<< /ActualText (PUBLIC_ACTUAL) /Alt (PUBLIC_ALT) /E (PUBLIC_EXPANSION) /T (PUBLIC_TITLE) >>".to_vec()],
);
if hidden {
sanitize::remove_hidden(&mut doc, &[sanitize::Hidden::HiddenText]).unwrap();
} else {
mark(&mut doc, 0, &[[5.0, 195.0, 60.0, 212.0]], "");
apply(&mut doc, None).unwrap();
}
assert_no_secret(&doc, b"LEAK_");
assert_no_secret(&doc, b"SECRET");
let saved = reopen(&doc);
let page = pdfcraft_model::pages(&saved).swap_remove(0);
let resources = saved.resolve(page.dict.get(b"Resources").unwrap());
let properties = saved.resolve(resources.as_dict().unwrap().get(b"Properties").unwrap());
let property = saved.resolve(properties.as_dict().unwrap().get(b"Public").unwrap());
let dict = property.as_dict().unwrap();
for key in [&b"ActualText"[..], b"Alt", b"E", b"T"] {
assert!(dict.contains(key), "surviving description removed: {key:?}, hidden={hidden}");
}
}
}
/// An image drawn through an ICC profile has as many components as the profile's `/N` says, and
/// is cleared in place like any other, the profile given through a chain of references too.
#[test]
fn an_icc_based_image_is_cleared_in_place_through_any_references() {
for (n, chained) in [(1, false), (3, false), (1, true), (3, true)] {
let img = stream("/Type /XObject /Subtype /Image /Width 2 /Height 1 /ColorSpace [/ICCBased 8 0 R] /BitsPerComponent 8", vec![255u8; 2 * n]);
let icc = stream(&format!("/N {n}"), b"");
let objects = if chained { vec![img, b"9 0 R".to_vec(), icc] } else { vec![img, icc] };
let mut doc = one_page(b"q 20 0 0 10 100 100 cm /Im Do Q", "/XObject << /Im 7 0 R >>", objects);
// The left pixel.
mark(&mut doc, 0, &[[95.0, 95.0, 109.0, 115.0]], "");
let r = apply(&mut doc, None).unwrap();
assert_eq!((r.images_removed, r.images_cleared), (0, 1), "/N {n}, chained {chained}: {r:?}");
let samples = [vec![0u8; n], vec![255u8; n]].concat();
assert_eq!(named_image(&doc, 0, b"PCRedacted1"), Some(samples), "/N {n}, chained {chained}");
}
}
/// A cleared copy keeps the parameters for data in another file only while it has one: `/FFilter`
/// and `/FDecodeParms` mean nothing without a `/F` that isn't null, and may name a JBIG2
/// dictionary that would then stay.
#[test]
fn a_cleared_copy_keeps_parameters_for_data_in_another_file_only_with_one() {
for (f, kept) in [("/F (scan.jb2)", true), ("/F 9 0 R", true), ("/F null", false), ("/F 8 0 R", false), ("/F 99 0 R", false), ("", false)] {
let dict = format!(
"/Type /XObject /Subtype /Image /Width 2 /Height 1 /ColorSpace /DeviceGray /BitsPerComponent 8 {f} /FFilter /JBIG2Decode /FDecodeParms << /JBIG2Globals 10 0 R >>"
);
let objects = vec![stream(&dict, [255u8, 255]), b"null".to_vec(), b"(scan.jb2)".to_vec()];
let doc = one_page(b"q 20 0 0 10 100 100 cm /Im Do Q", "/XObject << /Im 7 0 R >>", objects);
let Object::Stream(s) = &*doc.get(ObjRef::new(7, 0)) else { panic!("object 7 is the image") };
let left = [[0.0, 0.0, 0.4, 1.0]];
let copy = crate::image::clear(&doc, s, &pdfcraft_content::Matrix::IDENTITY, &left, &mut pdfcraft_cos::ImageBudget::with_work(u64::MAX));
let Ok(Some(copy)) = copy else { panic!("{f}: cleared in a copy") };
assert_eq!((copy.dict.contains(b"FFilter"), copy.dict.contains(b"FDecodeParms")), (kept, kept), "{f}");
}
}
#[test]
fn a_hostile_icc_component_count_is_refused_not_multiplied_into_a_crash() {
// /N comes from the file. 2^60 × 8 bpc × width 2 overflows the row size; a negative /N wrapped
// to a huge usize. Either way the half-covered image can't be cleared, so it is removed whole.
for n in ["1152921504606846976", "-1", "2"] {
let img = stream("/Type /XObject /Subtype /Image /Width 2 /Height 1 /ColorSpace [/ICCBased 8 0 R] /BitsPerComponent 8", [255, 255]);
let icc = stream(&format!("/N {n}"), b"");
let mut doc = one_page(b"q 20 0 0 10 100 100 cm /Im Do Q", "/XObject << /Im 7 0 R >>", vec![img, icc]);
mark(&mut doc, 0, &[[95.0, 95.0, 110.0, 115.0]], "");
let r = apply(&mut doc, None).unwrap();
assert_eq!((r.images_removed, r.images_cleared), (1, 0), "/N {n}: {r:?}");
assert!(!content(&doc, 0).contains("/Im Do"), "/N {n}");
}
}
#[test]
fn image_alternatives_and_masks_cannot_keep_redacted_pixels() {
for extra in ["/Alternates [<< /Image 8 0 R >>]", "/SMask 8 0 R", "/Mask 8 0 R", "/Mask [0 0]"] {
let mut doc = one_page(
b"q 0 0 300 300 re W n 8 0 0 10 10 10 cm /Im Do Q",
"/XObject << /Im 7 0 R >>",
vec![
stream(
&format!("/Type /XObject /Subtype /Image /Width 8 /Height 2 /ColorSpace /DeviceGray /BitsPerComponent 8 {extra}"),
b"LEAK_TOPPUBLIC__",
),
stream("/Type /XObject /Subtype /Image /Width 8 /Height 1 /ColorSpace /DeviceGray /BitsPerComponent 8", b"LEAK_ALT"),
],
);
mark(&mut doc, 0, &[[10.0, 15.0, 18.0, 20.0]], "");
let report = apply(&mut doc, None).unwrap();
assert_no_secret(&doc, b"LEAK_");
assert_eq!((report.images_cleared, report.images_removed), if extra.starts_with("/Alternates") { (1, 0) } else { (0, 1) });
}
}
#[test]
fn masks_and_alternates_outside_redaction_stay() {
let mut doc = one_page(
b"q 8 0 0 10 10 10 cm /Im Do Q",
"/XObject << /Im 7 0 R >>",
vec![
stream(
"/Type /XObject /Subtype /Image /Width 8 /Height 2 /ColorSpace /DeviceGray /BitsPerComponent 8 /SMask 8 0 R /Alternates [<< /Image 8 0 R >>]",
b"PUBLIC_TOPBOTTOM",
),
stream("/Type /XObject /Subtype /Image /Width 8 /Height 1 /ColorSpace /DeviceGray /BitsPerComponent 8", b"PUBLIC__"),
],
);
mark(&mut doc, 0, &[[100.0, 100.0, 120.0, 120.0]], "");
let report = apply(&mut doc, None).unwrap();
assert_eq!((report.images_removed, report.images_cleared), (0, 0));
let saved = reopen(&doc);
let page = pdfcraft_model::pages(&saved).swap_remove(0);
let resources = saved.resolve(page.dict.get(b"Resources").unwrap());
let xobjects = saved.resolve(resources.as_dict().unwrap().get(b"XObject").unwrap());
let image = saved.resolve(xobjects.as_dict().unwrap().get(b"Im").unwrap());
let Object::Stream(image) = &*image else { panic!() };
assert!(image.dict.contains(b"SMask") && image.dict.contains(b"Alternates"));
assert_eq!(image.decoded().unwrap(), b"PUBLIC_TOPBOTTOM");
}
#[test]
fn stream_boundary_comment_stays_inert_after_unrelated_redaction() {
for tail in [b"BT /F1 10 Tf 10 200 Td (HIDDEN) Tj ET".as_slice(), b"BT /F1 10 Tf 10 200 Td (HIDDEN) Tj ET\nBT /F1 10 Tf 10 250 Td (LIVE) Tj ET"] {
let mut doc = one_page(b"BT /F1 10 Tf 10 100 Td (REMOVE) Tj ET % note", "", vec![]);
let hidden = doc.add(Object::Stream(Stream::from_raw(Dict::new(), tail.to_vec())));
let page = pdfcraft_model::pages(&doc)[0].obj;
doc.update_dict(page, |d| d.set(b"Contents".to_vec(), Object::Array(vec![Object::Ref(ObjRef::new(4, 0)), Object::Ref(hidden)]))).unwrap();
mark(&mut doc, 0, &[[9.0, 95.0, 50.0, 110.0]], "");
assert_eq!(apply(&mut doc, None).unwrap().glyphs, 6);
let reopened = reopen(&doc);
let page = &pdfcraft_model::pages(&reopened)[0];
let (_, data) = page_streams(&reopened, &page.dict, 0).unwrap();
// Independent of Pieces::join: the renderer concatenates streams with spaces.
let ops = pdfcraft_content::parse(&data.join(&b' ')).ops;
assert!(!ops.iter().any(|o| o.operands.iter().any(|v| v.as_string().is_some_and(|s| s.bytes == b"HIDDEN"))), "hidden text became executable");
let live = ops.iter().any(|o| o.operands.iter().any(|v| v.as_string().is_some_and(|s| s.bytes == b"LIVE")));
assert_eq!(live, tail.contains(&b'\n'));
}
}
#[test]
fn rewriting_a_piece_keeps_its_leading_comment_terminator() {
for eol in [b"\n".as_slice(), b"\r", b"\r\n"] {
let mut doc = one_page(b"q Q % note", "", vec![]);
let mut tail = eol.to_vec();
tail.extend(b"BT /F1 10 Tf 10 100 Td (REMOVE) Tj ET BT /F1 10 Tf 10 200 Td (KEEP) Tj ET");
let second = doc.add(Object::Stream(Stream::from_raw(Dict::new(), tail)));
let page = pdfcraft_model::pages(&doc)[0].obj;
doc.update_dict(page, |d| d.set(b"Contents".to_vec(), Object::Array(vec![Object::Ref(ObjRef::new(4, 0)), Object::Ref(second)]))).unwrap();
mark(&mut doc, 0, &[[9.0, 95.0, 50.0, 110.0]], "");
assert_eq!(apply(&mut doc, None).unwrap().glyphs, 6);
let p = &pdfcraft_model::pages(&doc)[0];
let (_, data) = page_streams(&doc, &p.dict, 0).unwrap();
let ops = pdfcraft_content::parse(&data.join(&b' ')).ops;
assert_eq!(ops.iter().filter(|o| o.is("BT")).count(), 2, "leading EOL lost; first live operator swallowed");
assert!(ops.iter().any(|o| o.operands.iter().any(|v| v.as_string().is_some_and(|s| s.bytes == b"KEEP"))));
}
}
#[test]
fn named_hidden_text_descriptions_are_removed_from_saved_bytes() {
let mut doc = one_page(
b"/Span /P0 BDC BT /F1 10 Tf 3 Tr 10 200 Td (SECRET) Tj ET EMC",
"/Properties << /P0 7 0 R >>",
vec![b"<< /ActualText (LEAK_ACTUAL) /Alt (LEAK_ALT) /E (LEAK_EXPANSION) /T (LEAK_TITLE) >>".to_vec()],
);
sanitize::remove_hidden(&mut doc, &[sanitize::Hidden::HiddenText]).unwrap();
assert_no_secret(&doc, b"SECRET");
assert_no_secret(&doc, b"LEAK_");
let saved = reopen(&doc);
let page = pdfcraft_model::pages(&saved).swap_remove(0);
assert!(!saved.resolve(page.dict.get(b"Resources").unwrap()).as_dict().unwrap().contains(b"Properties"));
}
#[test]
fn unused_descriptions_in_shared_or_inherited_resources_are_retired() {
for hidden in [false, true] {
for inherited in [false, true] {
let mode = if hidden { 3 } else { 0 };
let data = format!("/Span /P0 BDC BT /F1 10 Tf {mode} Tr 10 200 Td (SECRET) Tj ET EMC /Fm Do");
let mut doc = one_page(
data.as_bytes(),
"/Properties << /P0 7 0 R /Public 8 0 R >> /XObject << /Fm 9 0 R >>",
vec![
b"<< /ActualText (LEAK_ACTUAL) /Alt (LEAK_ALT) /E (LEAK_EXPANSION) /T (LEAK_TITLE) >>".to_vec(),
b"<< /ActualText (PUBLIC_ACTUAL) /Alt (PUBLIC_ALT) /E (PUBLIC_EXPANSION) /T (PUBLIC_TITLE) >>".to_vec(),
stream("/Type /XObject /Subtype /Form /BBox [0 0 100 120]", b"/Span /Public BDC BT /F1 10 Tf 0 Tr 10 100 Td (Public) Tj ET EMC"),
],
);
let first = doc.get(ObjRef::new(3, 0)).as_dict().unwrap().clone();
let resources = doc.add(first.get(b"Resources").unwrap().clone());
let second_content = doc.add(Object::Stream(Stream::from_raw(Dict::new(), b"/Fm Do".to_vec())));
let mut second = first;
second.set(b"Contents".to_vec(), Object::Ref(second_content));
if inherited {
second.remove(b"Resources");
} else {
second.set(b"Resources".to_vec(), Object::Ref(resources));
}
let second = doc.add(Object::Dict(second));
doc.update_dict(ObjRef::new(3, 0), |d| {
if inherited {
d.remove(b"Resources");
} else {
d.set(b"Resources".to_vec(), Object::Ref(resources));
}
})
.unwrap();
doc.update_dict(ObjRef::new(2, 0), |d| {
d.set(b"Kids".to_vec(), Object::Array(vec![Object::Ref(ObjRef::new(3, 0)), Object::Ref(second)]));
d.set(b"Count".to_vec(), Object::Int(2));
if inherited {
d.set(b"Resources".to_vec(), Object::Ref(resources));
}
})
.unwrap();
if hidden {
sanitize::remove_hidden(&mut doc, &[sanitize::Hidden::HiddenText]).unwrap();
} else {
mark(&mut doc, 0, &[[5.0, 195.0, 60.0, 212.0]], "");
apply(&mut doc, Some(&[0])).unwrap();
}
assert_no_secret(&doc, b"LEAK_");
assert_no_secret(&doc, b"SECRET");
let mut saved = reopen(&doc);
for page_index in 0..2 {
assert_eq!(under(&mut saved, page_index, &[[0.0, 90.0, 100.0, 110.0]]), 6);
let page = pdfcraft_model::pages(&saved).swap_remove(page_index);
let resources = saved.resolve(page.dict.get(b"Resources").unwrap());
let properties = saved.resolve(resources.as_dict().unwrap().get(b"Properties").unwrap());
assert!(!properties.as_dict().unwrap().contains(b"P0"));
let property = saved.resolve(properties.as_dict().unwrap().get(b"Public").unwrap());
for key in [&b"ActualText"[..], b"Alt", b"E", b"T"] {
assert!(property.as_dict().unwrap().contains(key), "shared form lost {key:?}: hidden={hidden}, inherited={inherited}");
}
}
}
}
}
mod descriptions;
mod elsewhere;
mod form_matrix;
mod hidden_scope;
mod identification;
mod inline_images;
mod limits;
mod metadata_fail_closed;
mod metadata_matching;
mod operands_from_the_end;
mod property_retirement;
mod surviving_references;
mod text_state;
mod unparsed;
/// `[(FIRSTARR)] [(LASTARR)] TJ` is malformed (a surplus array operand), but the operator still
/// draws only the one immediately before it (LASTARR, 7 glyphs, 35 pt wide from x = 10) -- not
/// the first (FIRSTARR, 8 glyphs), which is never drawn at all, as the renderer agrees (#3569).
#[test]
fn a_tj_surplus_arrays_undrawn_glyphs_are_not_seen_under_a_mark() {
let mut doc = one_page(b"BT /F1 10 Tf 10 100 Td [(FIRSTARR)] [(LASTARR)] TJ ET", "", vec![]);
assert_eq!(under(&mut doc, 0, &[[45.0, 95.0, 50.0, 110.0]]), 0, "nothing is drawn past LASTARR's 35pt width");
assert_eq!(under(&mut doc, 0, &[[10.0, 95.0, 45.0, 110.0]]), 7, "LASTARR's 7 glyphs, the one the page draws");
}
#[test]
fn form_entries_share_one_budget_per_page() {
// 20 draws of a form that draws the next 20 times, four levels down: 160,000 glyphs from a
// few hundred bytes.
let draws = "/X Do ".repeat(20);
let mut objs = vec![stream("/Type /XObject /Subtype /Form /Resources << /Font << /F1 5 0 R >> >>", b"BT /F1 10 Tf 10 100 Td (A) Tj ET")];
for inner in 7..10 {
objs.push(stream(&format!("/Type /XObject /Subtype /Form /Resources << /XObject << /X {inner} 0 R >> >>"), draws.as_bytes()));
}
let mut doc = one_page(draws.as_bytes(), "/XObject << /X 10 0 R >>", objs);
let residue = under(&mut doc, 0, &[[0.0, 90.0, 50.0, 120.0]]);
assert!(residue > 0 && residue <= pdfcraft_cos::MAX_PAGE_FORMS, "{residue} glyphs counted");
// Applying refuses the page rather than leave forms past the budget unread.
mark(&mut doc, 0, &[[0.0, 90.0, 50.0, 120.0]], "");
assert!(apply(&mut doc, None).is_err());
}
/// A name tree whose levels each list the next level twice: a walk that follows every path
/// visits 2^levels nodes.
#[test]
fn a_name_tree_whose_nodes_share_kids_counts_each_entry_once() {
use crate::sanitize::{Hidden, scan};
let levels = 12;
let mut objs: Vec<Vec<u8>> = vec![
b"<< /Type /Catalog /Pages 2 0 R /Names << /EmbeddedFiles 5 0 R >> >>".to_vec(),
b"<< /Type /Pages /Kids [3 0 R] /Count 1 >>".to_vec(),
b"<< /Type /Page /Parent 2 0 R /MediaBox [0 0 300 300] /Contents 4 0 R >>".to_vec(),
stream("", b"BT ET"),
];
for k in 0..levels {
let next = 6 + k;
objs.push(format!("<< /Kids [{next} 0 R {next} 0 R] >>").into_bytes());
}
objs.push(b"<< /Names [(a.txt) << /Type /Filespec /F (a.txt) >>] >>".to_vec());
let doc = pdf(objs);
let counts: std::collections::HashMap<Hidden, usize> = scan(&doc).into_iter().collect();
assert_eq!(counts[&Hidden::Attachments], 1, "one entry, reached through 2^{levels} paths");
}
/// Page content that stops decoding at its start (`z` inside an ASCII85 group) while the rest of
/// the stream draws hidden text. A tolerant decode reads nothing, counts nothing and keeps the
/// stream; sanitizing must refuse instead.
#[test]
fn hidden_text_in_content_that_stops_decoding_is_refused() {
use crate::sanitize::{Hidden, remove_hidden, sanitize};
let encoded: &[u8] = br"!!z6<#'\7PQ#?0Ha>,+>Y->EZd(j+>PW)+B2ko-rWQ%;FOP6+B3(u78s~>";
let fresh = || {
let mut doc = one_page(b"", "", vec![stream("/Filter /ASCII85Decode", encoded)]);
doc.update_dict(ObjRef::new(3, 0), |d| d.set(b"Contents".to_vec(), Object::Ref(ObjRef::new(7, 0)))).unwrap();
doc
};
let mut doc = fresh();
let out = remove_hidden(&mut doc, &[Hidden::HiddenText]);
assert!(matches!(out, Err(RedactError::Unreadable(1))), "{out:?}");
let mut doc = fresh();
let out = sanitize(&mut doc);
assert!(matches!(out, Err(RedactError::Unreadable(1))), "{out:?}");
}
/// An array of references to objects `nums` (generation 0).
fn ref_array(nums: &[u32]) -> Object {
Object::Array(nums.iter().map(|&n| Object::Ref(ObjRef::new(n, 0))).collect())
}
/// An interactive form dictionary listing fields `nums`.
fn fields_dict(nums: &[u32]) -> Object {
let mut d = Dict::new();
d.set(b"Fields".to_vec(), ref_array(nums));
Object::Dict(d)
}
/// Flattening keeps hidden widgets (they draw nothing); removing form fields must still take them
/// and their values, and their parent field's value, out of the file.
#[test]
fn removing_form_fields_takes_hidden_widgets_and_their_values_out_of_the_file() {
use crate::sanitize::{Hidden, remove_hidden};
let mut doc = one_page(
b"BT /F1 10 Tf 10 100 Td (Visible) Tj ET",
"",
vec![
// 7 a hidden terminal field merged with its widget, 8 a parent field, 9 its hidden widget
b"<< /Type /Annot /Subtype /Widget /FT /Tx /T (own) /V (LEAK_WIDGET_VALUE) /F 2 /Rect [10 200 150 220] /AP << /N 10 0 R >> >>".to_vec(),
b"<< /FT /Tx /T (parent) /V (LEAK_PARENT_VALUE) /Kids [9 0 R] >>".to_vec(),
b"<< /Type /Annot /Subtype /Widget /Parent 8 0 R /F 32 /Rect [10 230 150 250] /AP << /N 10 0 R >> >>".to_vec(),
stream("/Type /XObject /Subtype /Form /BBox [0 0 140 20]", b"0 g"),
],
);
doc.update_dict(doc.root().unwrap(), |d| d.set(b"AcroForm".to_vec(), fields_dict(&[7, 8]))).unwrap();
doc.update_dict(ObjRef::new(3, 0), |d| d.set(b"Annots".to_vec(), ref_array(&[7, 9]))).unwrap();
remove_hidden(&mut doc, &[Hidden::FormFields]).unwrap();
assert_no_secret(&doc, b"LEAK_");
assert!(annots_of(&doc, &pdfcraft_model::pages(&doc).swap_remove(0).dict).is_empty());
}
/// F-021: removing form fields on a document with an AcroForm dictionary (even one with no
/// enumerable terminal fields) removes the AcroForm dictionary, processes widgets, and requires full save.
#[test]
fn removing_form_fields_removes_acroform_even_without_terminal_fields() {
use crate::sanitize::{Hidden, remove_hidden, scan};
let mut doc =
one_page(b"BT /F1 10 Tf 10 100 Td (Visible) Tj ET", "", vec![b"<< /Type /Annot /Subtype /Widget /Rect [10 200 150 220] >>".to_vec()]);
doc.update_dict(doc.root().unwrap(), |d| d.set(b"AcroForm".to_vec(), fields_dict(&[]))).unwrap();
doc.update_dict(ObjRef::new(3, 0), |d| d.set(b"Annots".to_vec(), ref_array(&[7]))).unwrap();
let counts = scan(&doc);
assert_eq!(counts.iter().find(|c| c.0 == Hidden::FormFields).map(|c| c.1), Some(1));
assert!(doc.get(doc.root().unwrap()).as_dict().is_some_and(|d| d.contains(b"AcroForm")));
assert!(!doc.full_save_required());
let done = remove_hidden(&mut doc, &[Hidden::FormFields]).unwrap();
assert_eq!(done, [(Hidden::FormFields, 1)]);
assert!(!doc.get(doc.root().unwrap()).as_dict().is_some_and(|d| d.contains(b"AcroForm")));
assert!(annots_of(&doc, &pdfcraft_model::pages(&doc).swap_remove(0).dict).is_empty());
assert!(doc.full_save_required());
}
/// Removing hidden text cleans the structure tree as redaction does: the elements describing the
/// removed text lose their descriptions, and a reference into a rewritten Form points at the copy.
#[test]
fn removing_hidden_text_cleans_the_structure_tree() {
use crate::sanitize::{Hidden, remove_hidden};
let mut doc = one_page(
b"/P <</MCID 0>> BDC BT /F1 10 Tf 3 Tr 10 200 Td (SECRET) Tj ET EMC /P <</MCID 1>> BDC BT /F1 10 Tf 0 Tr 10 100 Td (Public) Tj ET EMC /Fm Do",
"/XObject << /Fm 7 0 R >>",
vec![
stream(
"/Type /XObject /Subtype /Form /BBox [0 0 300 300] /Resources << /Font << /F1 5 0 R >> >>",
b"/P <</MCID 0>> BDC BT /F1 10 Tf 3 Tr 10 150 Td (FORMSECRET) Tj ET EMC",
),
b"<< /Type /StructTreeRoot /K 9 0 R >>".to_vec(),
b"<< /S /Document /P 8 0 R /K [10 0 R 11 0 R 12 0 R] >>".to_vec(),
b"<< /S /P /P 9 0 R /Pg 3 0 R /ActualText (LEAK_PAGE_TAG) /K 0 >>".to_vec(),
b"<< /S /P /P 9 0 R /Pg 3 0 R /ActualText (Public) /K 1 >>".to_vec(),
b"<< /S /P /P 9 0 R /Pg 3 0 R /Alt (LEAK_FORM_TAG) /K << /Type /MCR /Pg 3 0 R /Stm 7 0 R /MCID 0 >> >>".to_vec(),
],
);
doc.update_dict(doc.root().unwrap(), |d| d.set(b"StructTreeRoot".to_vec(), Object::Ref(ObjRef::new(8, 0)))).unwrap();
doc.update_dict(ObjRef::new(3, 0), |d| d.set(b"StructParents".to_vec(), Object::Int(0))).unwrap();
remove_hidden(&mut doc, &[Hidden::HiddenText]).unwrap();
assert_no_secret(&doc, b"LEAK_");
assert_no_secret(&doc, b"SECRET");
assert!(doc.get(ObjRef::new(11, 0)).as_dict().unwrap().contains(b"ActualText"), "untouched content keeps its tags");
}
/// One page drawing image /Im (object 7) and form /Fm (object 8), with `extra` objects from 9 on.
fn owners_fixture(image: &str, form: &str, extra: Vec<Vec<u8>>) -> Document {
let mut objs = vec![
stream(&format!("/Type /XObject /Subtype /Image /Width 1 /Height 1 /ColorSpace /DeviceGray /BitsPerComponent 8 {image}"), b"\x80"),
stream(&format!("/Type /XObject /Subtype /Form /BBox [0 0 10 10] {form}"), b"0 0 5 5 re f"),
];
objs.extend(extra);
one_page(b"BT /F1 10 Tf 10 100 Td (Visible) Tj ET q 10 0 0 10 0 0 cm /Im Do Q /Fm Do", "/XObject << /Im 7 0 R /Fm 8 0 R >>", objs)
}
/// Removing one category and saving leaves none of `secret` in the file, and the scan counted it.
fn removes_from_every_owner(mut doc: Document, h: crate::sanitize::Hidden, secret: &[u8]) {
use crate::sanitize::{remove_hidden, scan};
let counts: std::collections::HashMap<_, usize> = scan(&doc).into_iter().collect();
assert!(counts[&h] > 0, "{h:?} is counted");
remove_hidden(&mut doc, &[h]).unwrap();
assert_no_secret(&doc, secret);
assert!(scan(&reopen(&doc)).into_iter().all(|(k, n)| k != h || n == 0 || h == crate::sanitize::Hidden::Metadata));
}
#[test]
fn private_data_of_a_form_xobject_is_removed() {
let doc = owners_fixture("", "/PieceInfo << /App << /Private (LEAK_PRIVATE) >> >>", vec![]);
removes_from_every_owner(doc, crate::sanitize::Hidden::PrivateData, b"LEAK_");
}
#[test]
fn metadata_of_an_image_is_removed() {
let doc = owners_fixture("/Metadata 9 0 R", "", vec![stream("/Type /Metadata /Subtype /XML", b"<x:xmpmeta>LEAK_XMP</x:xmpmeta>")]);
removes_from_every_owner(doc, crate::sanitize::Hidden::Metadata, b"LEAK_");
}
#[test]
fn a_pages_associated_files_are_removed_with_attachments() {
let mut doc = owners_fixture(
"",
"",
vec![b"<< /Type /Filespec /F (a.txt) /AFRelationship /Data /EF << /F 10 0 R >> >>".to_vec(), stream("/Type /EmbeddedFile", b"LEAK_ATTACHED")],
);
doc.update_dict(ObjRef::new(3, 0), |d| d.set(b"AF".to_vec(), ref_array(&[9]))).unwrap();
removes_from_every_owner(doc, crate::sanitize::Hidden::Attachments, b"LEAK_");
}
#[test]
fn a_fields_own_actions_are_removed_with_links_actions_and_scripts() {
let mut doc = owners_fixture(
"",
"",
vec![
b"<< /FT /Tx /T (total) /AA << /C << /S /JavaScript /JS (LEAK_SCRIPT) >> >> /Kids [10 0 R] >>".to_vec(),
b"<< /Type /Annot /Subtype /Widget /Parent 9 0 R /Rect [10 200 150 220] >>".to_vec(),
],
);
doc.update_dict(doc.root().unwrap(), |d| d.set(b"AcroForm".to_vec(), fields_dict(&[9]))).unwrap();
doc.update_dict(ObjRef::new(3, 0), |d| d.set(b"Annots".to_vec(), ref_array(&[10]))).unwrap();
removes_from_every_owner(doc, crate::sanitize::Hidden::LinksActionsScripts, b"LEAK_");
}
/// Resource names are chosen by the producer: a font named like an owned entry stays.
#[test]
fn a_resource_named_like_an_owned_entry_is_kept() {
let mut doc = one_page(
b"BT /AF 10 Tf 10 100 Td (Visible) Tj ET",
"",
vec![b"<< /Type /Filespec /F (a.txt) /EF << /F 8 0 R >> >>".to_vec(), stream("/Type /EmbeddedFile", b"LEAK_ATTACHED")],
);
doc.update_dict(ObjRef::new(3, 0), |d| {
let mut fonts = Dict::new();
fonts.set(b"AF".to_vec(), Object::Ref(ObjRef::new(5, 0)));
let mut res = Dict::new();
res.set(b"Font".to_vec(), Object::Dict(fonts));
d.set(b"Resources".to_vec(), Object::Dict(res));
d.set(b"AF".to_vec(), ref_array(&[7]));
})
.unwrap();
removes_from_every_owner(doc.clone(), crate::sanitize::Hidden::Attachments, b"LEAK_");
crate::sanitize::remove_hidden(&mut doc, &[crate::sanitize::Hidden::Attachments]).unwrap();
let page = pdfcraft_model::pages(&doc).swap_remove(0);
let res = doc.resolve(page.dict.get(b"Resources").unwrap()).as_dict().cloned().unwrap();
assert!(res.get(b"Font").and_then(|f| f.as_dict()).is_some_and(|f| f.contains(b"AF")));
assert!(!page.dict.contains(b"AF"));
}
/// Owners written in place deeper than the sweep follows are refused, not skipped.
#[test]
fn owners_nested_too_deeply_are_refused() {
use crate::sanitize::{Hidden, remove_hidden};
let nested = format!("{}<< /AA << /O << /S /JavaScript /JS (LEAK_DEEP) >> >> >>{}", "[".repeat(100), "]".repeat(100));
let mut doc = one_page(b"BT ET", "", vec![format!("<< /Deep {nested} >>").into_bytes()]);
doc.update_dict(ObjRef::new(3, 0), |d| d.set(b"PieceInfo".to_vec(), Object::Ref(ObjRef::new(7, 0)))).unwrap();
let out = remove_hidden(&mut doc, &[Hidden::LinksActionsScripts]);
assert!(matches!(out, Err(RedactError::TooDeep)), "{out:?}");
}
/// A name tree node that lists the same kid more times than the node budget: each repeat costs a
/// step, so the walk stops at the budget instead of scanning every repeat, and the distinct kid
/// listed after them is never reached.
#[test]
fn repeated_name_tree_kids_count_towards_the_node_budget() {
use crate::sanitize::{Hidden, scan};
let repeats = "6 0 R ".repeat(100_010);
let doc = pdf(vec![
b"<< /Type /Catalog /Pages 2 0 R /Names << /EmbeddedFiles 5 0 R >> >>".to_vec(),
b"<< /Type /Pages /Kids [3 0 R] /Count 1 >>".to_vec(),
b"<< /Type /Page /Parent 2 0 R /MediaBox [0 0 300 300] /Contents 4 0 R >>".to_vec(),
stream("", b"BT ET"),
format!("<< /Kids [{repeats}7 0 R] >>").into_bytes(),
b"<< /Names [(a.txt) << /Type /Filespec /F (a.txt) >>] >>".to_vec(),
b"<< /Names [(b.txt) << /Type /Filespec /F (b.txt) >>] >>".to_vec(),
]);
let counts: std::collections::HashMap<Hidden, usize> = scan(&doc).into_iter().collect();
assert_eq!(counts[&Hidden::Attachments], 1, "the walk went past its node budget to the last kid");
}
/// Content streams and the forms they draw spend one page allowance: 130 MiB of content and a
/// 130 MiB form are each under it, together past it.
#[test]
fn content_and_forms_share_one_budget_per_page() {
use pdfcraft_cos::{Dict, Stream};
let mut doc = one_page(b"", "", vec![]);
let mut content = b"BT /F1 10 Tf 10 100 Td (SECRET) Tj ET /X Do\n%".to_vec();
content.resize(130 << 20, b'x');
let content = doc.add(Object::Stream(Stream::flate(Dict::new(), &content)));
let mut form = Dict::new();
form.set(b"Type".to_vec(), Object::name("XObject"));
form.set(b"Subtype".to_vec(), Object::name("Form"));
let mut body = b"%".to_vec();
body.resize(130 << 20, b'x');
let form = doc.add(Object::Stream(Stream::flate(form, &body)));
doc.update_dict(ObjRef::new(3, 0), |d| {
let mut xobjects = Dict::new();
xobjects.set(b"X".to_vec(), Object::Ref(form));
let mut fonts = Dict::new();
fonts.set(b"F1".to_vec(), Object::Ref(ObjRef::new(5, 0)));
let mut res = Dict::new();
res.set(b"Font".to_vec(), Object::Dict(fonts));
res.set(b"XObject".to_vec(), Object::Dict(xobjects));
d.set(b"Resources".to_vec(), Object::Dict(res));
d.set(b"Contents".to_vec(), Object::Ref(content));
})
.unwrap();
mark(&mut doc, 0, &[[0.0, 90.0, 50.0, 120.0]], "");
assert!(apply(&mut doc, None).is_err(), "the form past the page's allowance was read");
}
/// A page that fails to read after another page was already redacted leaves the whole document as
/// it was, the shared resources the first page's rewrite localised included.
#[test]
fn a_failure_on_a_later_page_leaves_the_document_unchanged() {
let mut doc = two_pages_one_resource_dictionary(true, false);
let all = pdfcraft_model::pages(&doc);
let (first, second, parent) = (all[0].obj, all[1].obj, doc.get(all[0].obj).as_dict().unwrap().reference(b"Parent").unwrap());
let mut dct = Dict::new();
dct.set(b"Filter".to_vec(), Object::name("DCTDecode"));
let bad = doc.add(Object::Stream(Stream::from_raw(dct, b"garbage".to_vec())));
doc.update_dict(second, |d| d.set(b"Contents".to_vec(), Object::Ref(bad))).unwrap();
mark(&mut doc, 0, &[[0.0, 0.0, 300.0, 300.0]], "");
mark(&mut doc, 1, &[[0.0, 0.0, 50.0, 50.0]], "");
let before: Vec<Object> = [first, second, parent].iter().map(|r| (*doc.get(*r)).clone()).collect();
let text = content(&doc, 0);
assert_eq!(apply(&mut doc, None), Err(RedactError::Unreadable(2)));
let after: Vec<Object> = [first, second, parent].iter().map(|r| (*doc.get(*r)).clone()).collect();
assert_eq!(after, before);
assert_eq!(content(&doc, 0), text);
}
#[test]
fn a_reply_and_pop_up_on_another_page_go_with_the_removed_comment() {
let objs: Vec<Vec<u8>> = vec![
b"<< /Type /Catalog /Pages 2 0 R >>".to_vec(),
b"<< /Type /Pages /Kids [3 0 R 4 0 R] /Count 2 >>".to_vec(),
b"<< /Type /Page /Parent 2 0 R /MediaBox [0 0 300 300] /Contents 5 0 R /Annots [6 0 R] >>".to_vec(),
b"<< /Type /Page /Parent 2 0 R /MediaBox [0 0 300 300] /Contents 5 0 R /Annots [7 0 R 8 0 R] >>".to_vec(),
stream("", b"BT ET"),
b"<< /Type /Annot /Subtype /Text /Rect [20 20 40 40] /Contents (PARENT-NOTE) /Popup 8 0 R >>".to_vec(),
b"<< /Type /Annot /Subtype /Text /Rect [250 250 270 270] /IRT 6 0 R /Contents (OTHER-PAGE-REPLY) >>".to_vec(),
b"<< /Type /Annot /Subtype /Popup /Rect [200 150 290 190] /Parent 6 0 R >>".to_vec(),
];
let mut doc = pdf(objs);
mark(&mut doc, 0, &[[10.0, 10.0, 70.0, 50.0]], "");
apply(&mut doc, None).unwrap();
let second = pdfcraft_model::pages(&doc).swap_remove(1);
assert!(annots_of(&doc, &second.dict).is_empty(), "the reply and the pop-up leave page 2");
let bytes = write_full(&doc, &SaveOptions { object_streams: false, ..SaveOptions::default() }).unwrap();
assert!(!bytes.windows(10).any(|w| w == b"PARENT-NOT"), "the removed comment survived the save");
}
#[test]
fn a_redact_mark_written_in_place_is_refused() {
let objs: Vec<Vec<u8>> = vec![
b"<< /Type /Catalog /Pages 2 0 R >>".to_vec(),
b"<< /Type /Pages /Kids [3 0 R] /Count 1 >>".to_vec(),
b"<< /Type /Page /Parent 2 0 R /MediaBox [0 0 300 300] /Contents 4 0 R /Annots [<< /Type /Annot /Subtype /Redact /Rect [10 10 70 40] >>] >>"
.to_vec(),
stream("", b"BT ET"),
];
let mut doc = pdf(objs);
mark(&mut doc, 0, &[[100.0, 100.0, 120.0, 120.0]], "");
assert!(matches!(apply(&mut doc, None), Err(RedactError::UnreadableMark(1))));
}
#[test]
fn do_acts_on_the_operand_before_the_operator_like_the_renderer() {
// `/Missing /Im Do`: the renderer draws /Im, so redaction must find the image there.
let img = stream("/Type /XObject /Subtype /Image /Width 1 /Height 1 /ColorSpace /DeviceGray /BitsPerComponent 8", b"\x00");
let mut doc = one_page(b"q 100 0 0 100 10 10 cm /Missing /Im Do Q", "/XObject << /Im 7 0 R >>", vec![img]);
mark(&mut doc, 0, &[[0.0, 0.0, 300.0, 300.0]], "");
let r = apply(&mut doc, None).unwrap();
assert_eq!(r.images_removed, 1, "{r:?}");
}
#[test]
fn hidden_text_removal_drops_the_source_text_of_added_content() {
let mut doc = one_page(b"BT /F1 12 Tf 3 Tr 20 20 Td (SECRET) Tj ET", "", vec![]);
let content_ref = pdfcraft_model::pages(&doc)[0].dict.get(b"Contents").and_then(Object::as_ref).unwrap();
doc.update_dict(content_ref, |d| {
d.set(b"PCAdded".to_vec(), Object::Dict(Dict::new()));
})
.unwrap();
sanitize::remove_hidden(&mut doc, &[sanitize::Hidden::HiddenText]).unwrap();
let p = &pdfcraft_model::pages(&doc)[0];
let (list, _) = page_streams(&doc, &p.dict, 0).unwrap();
assert!(list.iter().all(|o| doc.resolve(o).as_dict().is_some_and(|d| !d.contains(b"PCAdded"))));
}
#[test]
fn a_reply_on_another_page_to_a_removed_popup_goes_too() {
let objs: Vec<Vec<u8>> = vec![
b"<< /Type /Catalog /Pages 2 0 R >>".to_vec(),
b"<< /Type /Pages /Kids [3 0 R 4 0 R] /Count 2 >>".to_vec(),
b"<< /Type /Page /Parent 2 0 R /MediaBox [0 0 300 300] /Contents 5 0 R /Annots [6 0 R 8 0 R] >>".to_vec(),
b"<< /Type /Page /Parent 2 0 R /MediaBox [0 0 300 300] /Contents 5 0 R /Annots [7 0 R] >>".to_vec(),
stream("", b"BT ET"),
b"<< /Type /Annot /Subtype /Text /Rect [20 20 40 40] /Contents (PARENT-NOTE) /Popup 8 0 R >>".to_vec(),
b"<< /Type /Annot /Subtype /Text /Rect [250 250 270 270] /IRT 8 0 R /Contents (POPUP-REPLY) >>".to_vec(),
b"<< /Type /Annot /Subtype /Popup /Rect [200 150 290 190] /Parent 6 0 R /Contents (POPUP-TEXT) >>".to_vec(),
];
let mut doc = pdf(objs);
mark(&mut doc, 0, &[[10.0, 10.0, 70.0, 50.0]], "");
apply(&mut doc, None).unwrap();
let second = pdfcraft_model::pages(&doc).swap_remove(1);
assert!(annots_of(&doc, &second.dict).is_empty());
let bytes = write_full(&doc, &SaveOptions { object_streams: false, ..SaveOptions::default() }).unwrap();
for gone in [&b"POPUP-REPLY"[..], b"POPUP-TEXT", b"PARENT-NOTE"] {
assert!(!bytes.windows(gone.len()).any(|w| w == gone), "{} survived", String::from_utf8_lossy(gone));
}
}
/// A hairline under a shrinking CTM still paints about a device pixel: its pad must not shrink
/// with the CTM, or a mark on that ink misses it.
#[test]
fn a_hairline_under_a_shrinking_ctm_is_padded_by_half_a_unit() {
// Device line from (10, 10) to (50, 10); the mark lies on its painted pixel, above its axis.
let mut doc = one_page(b"0.01 0 0 0.01 0 0 cm 0 w 1000 1000 m 5000 1000 l S", "", vec![]);
mark(&mut doc, 0, &[[20.0, 10.2, 30.0, 10.4]], "");
assert_eq!(apply(&mut doc, None).unwrap().paths_removed, 1);
}
/// One `re` has four mitred corners: it is padded by the miter limit like any joined path.
#[test]
fn a_rectangle_stroke_is_padded_for_its_mitred_corners() {
let mut doc = one_page(b"2 w 100 100 50 50 re S", "", vec![]);
// Past half the line width from the rectangle, inside the miter reach.
mark(&mut doc, 0, &[[155.0, 120.0, 160.0, 130.0]], "");
assert_eq!(apply(&mut doc, None).unwrap().paths_removed, 1);
}
/// A projecting square cap's outer corner reaches lw/2 x sqrt(2) from the endpoint. Under a shear
/// it maps past a pad of lw/2 times the CTM's stretch (the review's example: corner at x = 12).
#[test]
fn a_square_cap_corner_under_a_shear_is_inside_the_pad() {
let mut doc = one_page(b"1 0 1 1 0 0 cm 2 J 2 w 0 0 m 10 0 l S", "", vec![]);
mark(&mut doc, 0, &[[11.8, 0.8, 11.9, 0.95]], "");
assert_eq!(apply(&mut doc, None).unwrap().paths_removed, 1);
}
/// A line width set between the path and its `S` still applies to that stroke, so the stroke is
/// tested at the width it is painted with; the width itself stays, the path goes.
#[test]
fn a_width_set_between_the_path_and_its_stroke_does_not_hide_it_from_the_test() {
let mut doc = one_page(b"10 60 m 100 60 l 12 w S 5 5 m 9 9 l S", "", vec![]);
mark(&mut doc, 0, &[[40.0, 45.0, 60.0, 55.0]], "");
let r = apply(&mut doc, None).unwrap();
assert_eq!(r.paths_removed, 1, "{r:?}");
let c = content(&doc, 0);
assert!(!c.contains("10 60 m") && !c.contains("100 60 l"), "the wide line reaches the mark: {c}");
assert!(c.contains("12 w"), "the width stays: {c}");
assert!(c.contains("5 5 m"), "the line elsewhere stays: {c}");
}
/// The renderer takes a line-state operator's operand from the end: `1 12 w` strokes 12 points wide,
/// so the stroke is tested at that width.
#[test]
fn a_surplus_operand_before_the_line_width_does_not_hide_the_stroke() {
let mut doc = one_page(b"1 12 w 10 60 m 100 60 l S", "", vec![]);
mark(&mut doc, 0, &[[40.0, 45.0, 60.0, 55.0]], "");
let r = apply(&mut doc, None).unwrap();
assert_eq!(r.paths_removed, 1, "{r:?}");
assert!(!content(&doc, 0).contains("100 60 l"));
}
/// One page drawing `SECRET` in the `/OC /L` block, with layer 7 0 R in the default
/// configuration `config` (objects 8 and up are `extra`).
fn layer_state_page(config: Vec<(&str, Object)>, resource: &str, extra: Vec<Vec<u8>>) -> Document {
let mut objects = vec![b"<< /Type /OCG /Name (Secret) >>".to_vec()];
objects.extend(extra);
let mut doc = one_page(b"/OC /L BDC BT /F1 10 Tf 10 100 Td (SECRET) Tj ET EMC", &format!("/Properties << /L {resource} >>"), objects);
let config = Object::Dict(Dict::from_iter(config.into_iter().map(|(k, v)| (k.as_bytes().to_vec(), v))));
let root = doc.root().unwrap();
doc.update_dict(root, |d| {
d.set(
b"OCProperties".to_vec(),
Object::Dict(Dict::from_iter([(b"OCGs".to_vec(), Object::Array(vec![Object::Ref(ObjRef::new(7, 0))])), (b"D".to_vec(), config)])),
);
})
.unwrap();
doc
}
fn assert_layer_secret_removed(mut doc: Document) {
use crate::sanitize::{Hidden, remove_hidden, scan};
let counts = scan(&doc);
assert!(counts.iter().any(|(h, n)| *h == Hidden::HiddenLayers && *n > 0), "{counts:?}");
remove_hidden(&mut doc, &[Hidden::HiddenLayers]).unwrap();
let saved = Document::open(Arc::new(write_full(&doc, &SaveOptions::default()).unwrap())).unwrap();
assert!(!pdfcraft_edit::text_lines(&saved, 0).unwrap().iter().any(|line| line.text.contains("SECRET")));
}
#[test]
fn an_indirect_base_state_off_hides_the_layer() {
assert_layer_secret_removed(layer_state_page(vec![("BaseState", Object::Ref(ObjRef::new(8, 0)))], "7 0 R", vec![b"/OFF".to_vec()]));
}
#[test]
fn a_chained_base_state_off_hides_the_layer() {
assert_layer_secret_removed(layer_state_page(
vec![("BaseState", Object::Ref(ObjRef::new(8, 0)))],
"7 0 R",
vec![b"9 0 R".to_vec(), b"/OFF".to_vec()],
));
}
#[test]
fn a_chained_membership_type_is_evaluated() {
assert_layer_secret_removed(layer_state_page(
vec![("BaseState", Object::name("ON"))],
"8 0 R",
vec![b"<< /Type 9 0 R /OCGs [7 0 R] /P /AllOff >>".to_vec(), b"10 0 R".to_vec(), b"/OCMD".to_vec()],
));
}
#[test]
fn off_wins_over_on_when_the_base_state_is_off() {
assert_layer_secret_removed(layer_state_page(
vec![
("BaseState", Object::name("OFF")),
("ON", Object::Array(vec![Object::Ref(ObjRef::new(7, 0))])),
("OFF", Object::Array(vec![Object::Ref(ObjRef::new(7, 0))])),
],
"7 0 R",
Vec::new(),
));
}
#[test]
fn a_membership_dictionary_with_an_indirect_type_is_evaluated() {
assert_layer_secret_removed(layer_state_page(
vec![("BaseState", Object::name("ON"))],
"8 0 R",
vec![b"<< /Type 9 0 R /OCGs [7 0 R] /P /AllOff >>".to_vec(), b"/OCMD".to_vec()],
));
}
/// An invisible OCR-style text layer drawn in a Type3 font (what the OCR output writes for
/// characters beyond WinAnsi) is redacted like any other text: its glyphs are hit-tested by
/// their real advances, removed, and the verifier passes.
#[test]
fn a_type3_text_layer_is_redacted_like_any_other() {
// One glyph (code 1): a filled box, 500 units wide, with a ToUnicode CMap that reads it
// back as U+3042.
let t3 = b"<< /Type /Font /Subtype /Type3 /Name /PCOcr /FontBBox [0 0 700 700] /FontMatrix [0.001 0 0 0.001 0 0] \
/FirstChar 1 /LastChar 1 /Widths [500] /Encoding << /Type /Encoding /Differences [1 /g01] >> \
/CharProcs << /g01 8 0 R >> /ToUnicode 9 0 R >>";
let charproc = b"500 0 0 0 0 700 700 d1\n0 0 700 700 re f\n";
let cmap = b"/CIDInit /ProcSet findresource begin\n12 dict begin\nbegincmap\n/CMapType 2 def\n1 begincodespacerange\n<01> <01>\nendcodespacerange\n1 beginbfchar\n<01> <3042>\nendbfchar\nendcmap\nCMapName currentdict /CMap defineresource pop\nend\nend\n";
let layer = b"BT 3 Tr /T3 10 Tf 100 100 Td (\x01) Tj ET";
let mut doc = pdf(vec![
b"<< /Type /Catalog /Pages 2 0 R >>".to_vec(),
b"<< /Type /Pages /Kids [3 0 R] /Count 1 >>".to_vec(),
b"<< /Type /Page /Parent 2 0 R /MediaBox [0 0 300 300] /Contents 4 0 R /Resources << /Font << /T3 5 0 R >> >> >>".to_vec(),
stream("", layer),
t3.to_vec(),
FONT.replace("95 0 R", "6 0 R").into_bytes(),
widths(),
stream("", charproc),
stream("", cmap),
]);
// The glyph box at size 10: 5 pt wide, from the font's descent to its ascent.
assert_eq!(under(&mut doc, 0, &[[98.0, 96.0, 112.0, 110.0]]), 1, "the Type3 glyph is found under the mark");
assert_eq!(under(&mut doc, 0, &[[200.0, 200.0, 250.0, 250.0]]), 0, "elsewhere the page is clean");
mark(&mut doc, 0, &[[98.0, 96.0, 112.0, 110.0]], "");
let report = apply(&mut doc, None).unwrap();
assert_eq!(report.glyphs, 1, "{report:?}");
// The verified apply left no text-showing operator: the re-read page is empty of them.
assert!(!content(&doc, 0).contains("Tj"), "{}", content(&doc, 0));
assert_eq!(under(&mut doc, 0, &[[98.0, 96.0, 112.0, 110.0]]), 0, "nothing is left under the mark");
}
#[test]
fn a_mark_whose_subtype_is_an_indirect_name_is_applied() {
// `/Subtype` may be an indirect reference (ISO 32000-2 §7.3.10); the audit reads it that way.
// Passing over the mark would leave the text under it while the others report success.
let objs: Vec<Vec<u8>> = vec![
b"<< /Type /Catalog /Pages 2 0 R >>".to_vec(),
b"<< /Type /Pages /Kids [3 0 R] /Count 1 >>".to_vec(),
b"<< /Type /Page /Parent 2 0 R /MediaBox [0 0 300 300] /Contents 4 0 R /Resources << /Font << /F1 5 0 R >> >> /Annots [7 0 R] >>".to_vec(),
stream("", b"BT /F1 10 Tf 20 20 Td (SECRET) Tj ET"),
FONT.replace("95 0 R", "6 0 R").into_bytes(),
widths(),
b"<< /Type /Annot /Subtype 8 0 R /Rect [10 10 90 40] >>".to_vec(),
b"/Redact".to_vec(),
];
let mut doc = pdf(objs);
assert_eq!(marks(&doc).len(), 1, "the mark is found");
let r = apply(&mut doc, None).unwrap();
assert_eq!(r.marks, 1);
assert!(r.glyphs >= 6, "the text under the mark went: {r:?}");
assert!(marks(&doc).is_empty(), "the mark itself went");
}
#[test]
fn a_mark_written_in_place_with_an_indirect_subtype_is_refused() {
let objs: Vec<Vec<u8>> = vec![
b"<< /Type /Catalog /Pages 2 0 R >>".to_vec(),
b"<< /Type /Pages /Kids [3 0 R] /Count 1 >>".to_vec(),
b"<< /Type /Page /Parent 2 0 R /MediaBox [0 0 300 300] /Contents 4 0 R /Annots [<< /Type /Annot /Subtype 5 0 R /Rect [10 10 70 40] >>] >>"
.to_vec(),
stream("", b"BT ET"),
b"/Redact".to_vec(),
];
let mut doc = pdf(objs);
mark(&mut doc, 0, &[[100.0, 100.0, 120.0, 120.0]], "");
assert!(matches!(apply(&mut doc, None), Err(RedactError::UnreadableMark(1))));
}
#[test]
fn annotations_written_in_place_go_when_covered_or_hanging_off_a_removed_one() {
// `/Annots` may hold dictionaries rather than references. One under the mark, a reply to a
// removed highlight on the same page and on another, and a reply to that inline comment's
// pop-up must all go; an inline comment elsewhere stays.
let objs: Vec<Vec<u8>> = vec![
b"<< /Type /Catalog /Pages 2 0 R >>".to_vec(),
b"<< /Type /Pages /Kids [3 0 R 4 0 R] /Count 2 >>".to_vec(),
b"<< /Type /Page /Parent 2 0 R /MediaBox [0 0 300 300] /Contents 5 0 R /Annots [6 0 R \
<< /Type /Annot /Subtype /Text /Rect [30 15 40 25] /Contents (INLINE-UNDER) /Popup 7 0 R >> \
<< /Type /Annot /Subtype /Text /Rect [250 250 270 270] /IRT 6 0 R /Contents (INLINE-REPLY) >> \
<< /Type /Annot /Subtype /Text /Rect [250 100 270 120] /Contents (INLINE-KEPT) >> 7 0 R] >>"
.to_vec(),
b"<< /Type /Page /Parent 2 0 R /MediaBox [0 0 300 300] /Contents 5 0 R /Annots [\
<< /Type /Annot /Subtype /Text /Rect [250 250 270 270] /IRT 6 0 R /Contents (OTHER-PAGE-INLINE-REPLY) >> \
<< /Type /Annot /Subtype /Text /Rect [200 200 220 220] /IRT 7 0 R /Contents (POPUP-INLINE-REPLY) >>] >>"
.to_vec(),
stream("", b"BT ET"),
b"<< /Type /Annot /Subtype /Highlight /Rect [20 20 60 30] /QuadPoints [20 30 60 30 20 20 60 20] /Contents (HIGHLIGHT-NOTE) >>".to_vec(),
b"<< /Type /Annot /Subtype /Popup /Rect [200 150 290 190] /Contents (INLINE-POPUP) >>".to_vec(),
];
let mut doc = pdf(objs);
mark(&mut doc, 0, &[[10.0, 10.0, 70.0, 40.0]], "");
let r = apply(&mut doc, None).unwrap();
assert_eq!(r.annotations, 5, "the highlight, the inline comment and the three replies: {r:?}");
let pages = pdfcraft_model::pages(&doc);
assert_eq!(annots_of(&doc, &pages[0].dict).len(), 1, "only the inline comment elsewhere stays");
assert!(annots_of(&doc, &pages[1].dict).is_empty(), "the replies leave page 2");
let bytes = write_full(&doc, &SaveOptions { object_streams: false, ..SaveOptions::default() }).unwrap();
for gone in [&b"HIGHLIGHT-NOTE"[..], b"INLINE-UNDER", b"INLINE-REPLY", b"OTHER-PAGE-INLINE-REPLY", b"POPUP-INLINE-REPLY", b"INLINE-POPUP"] {
assert!(!bytes.windows(gone.len()).any(|w| w == gone), "{} survived the save", String::from_utf8_lossy(gone));
}
assert!(bytes.windows(11).any(|w| w == b"INLINE-KEPT"));
}
#[test]
fn a_rewritten_form_xobject_drops_the_source_text_of_added_content() {
// A form drawing invisible text whose dictionary repeats it in /PCAdded.
let form = stream(
"/Type /XObject /Subtype /Form /BBox [0 0 300 300] /Resources << /Font << /F1 5 0 R >> >> /PCMark /Added /PCAdded << /Kind /Text /Text (LEAK_FORM_SOURCE) >>",
b"BT /F1 10 Tf 3 Tr 10 100 Td (HIDDEN) Tj ET BT /F1 10 Tf 10 200 Td (Shown) Tj ET",
);
let mut doc = one_page(b"/X Do", "/XObject << /X 7 0 R >>", vec![form]);
sanitize::remove_hidden(&mut doc, &[sanitize::Hidden::HiddenText]).unwrap();
let bytes = write_full(&doc, &SaveOptions { object_streams: false, ..SaveOptions::default() }).unwrap();
assert!(!bytes.windows(16).any(|w| w == b"LEAK_FORM_SOURCE"), "the form's added-content source text survived");
}
#[test]
fn a_redacted_form_xobject_drops_the_source_text_of_added_content() {
let form = stream(
"/Type /XObject /Subtype /Form /BBox [0 0 300 300] /Resources << /Font << /F1 5 0 R >> >> /PCMark /Added /PCAdded << /Kind /Text /Text (AB1234CD) >>",
b"BT /F1 10 Tf 10 100 Td (AB1234CD) Tj ET",
);
let mut doc = one_page(b"/X Do", "/XObject << /X 7 0 R >>", vec![form]);
mark(&mut doc, 0, &[[20.0, 95.0, 40.0, 110.0]], "");
apply(&mut doc, None).unwrap();
let bytes = write_full(&doc, &SaveOptions { object_streams: false, ..SaveOptions::default() }).unwrap();
assert!(!bytes.windows(4).any(|w| w == b"1234"), "redacted text left in the saved file");
}
#[test]
fn an_inline_widget_takes_its_field_even_with_no_referenced_widget() {
// The field's only widget is written in place in `/Annots`, so no referenced widget names it:
// the field (and its value) must still go with the widget under the mark.
let objs: Vec<Vec<u8>> = vec![
b"<< /Type /Catalog /Pages 2 0 R /AcroForm << /Fields [6 0 R] >> >>".to_vec(),
b"<< /Type /Pages /Kids [3 0 R] /Count 1 >>".to_vec(),
b"<< /Type /Page /Parent 2 0 R /MediaBox [0 0 300 300] /Contents 4 0 R /Annots [<< /Type /Annot /Subtype /Widget /Rect [20 20 60 30] /Parent 6 0 R >>] >>"
.to_vec(),
stream("", b"BT ET"),
b"<< >>".to_vec(),
b"<< /FT /Tx /T (name) /V (FIELD-SECRET) >>".to_vec(),
];
let mut doc = pdf(objs);
mark(&mut doc, 0, &[[10.0, 10.0, 70.0, 40.0]], "");
let r = apply(&mut doc, None).unwrap();
assert_eq!(r.fields, 1, "{r:?}");
let bytes = write_full(&doc, &SaveOptions { object_streams: false, ..SaveOptions::default() }).unwrap();
assert!(!bytes.windows(12).any(|w| w == b"FIELD-SECRET"), "the field value survived the save");
}
#[test]
fn an_inline_widget_of_the_field_on_another_page_goes_with_it() {
// Two pages: the widget under the mark, and a sibling widget in place on the second page with the
// same `/Parent`. Deleting the field must not leave the sibling naming it.
let objs: Vec<Vec<u8>> = vec![
b"<< /Type /Catalog /Pages 2 0 R /AcroForm << /Fields [7 0 R] >> >>".to_vec(),
b"<< /Type /Pages /Kids [3 0 R 5 0 R] /Count 2 >>".to_vec(),
b"<< /Type /Page /Parent 2 0 R /MediaBox [0 0 300 300] /Contents 4 0 R /Annots [<< /Type /Annot /Subtype /Widget /Rect [20 20 60 30] /Parent 7 0 R >>] >>"
.to_vec(),
stream("", b"BT ET"),
b"<< /Type /Page /Parent 2 0 R /MediaBox [0 0 300 300] /Contents 4 0 R /Annots [<< /Type /Annot /Subtype /Widget /Rect [200 200 260 230] /Parent 7 0 R >>] >>"
.to_vec(),
b"<< >>".to_vec(),
b"<< /FT /Tx /T (name) /V (SIBLING-SECRET) >>".to_vec(),
];
let mut doc = pdf(objs);
mark(&mut doc, 0, &[[10.0, 10.0, 70.0, 40.0]], "");
apply(&mut doc, None).unwrap();
let bytes = write_full(&doc, &SaveOptions { object_streams: false, ..SaveOptions::default() }).unwrap();
assert!(!bytes.windows(14).any(|w| w == b"SIBLING-SECRET"), "the field value survived through the sibling widget");
}
/// Private data takes an image's prepress data too: its OPI link to the original file and its
/// alternate images, which can be the unedited original.
#[test]
fn private_data_takes_opi_links_and_alternate_images() {
use crate::sanitize::{Hidden, remove_hidden, scan};
let mut doc = pdf(vec![
b"<< /Type /Catalog /Pages 2 0 R >>".to_vec(),
b"<< /Type /Pages /Kids [3 0 R] /Count 1 >>".to_vec(),
b"<< /Type /Page /Parent 2 0 R /MediaBox [0 0 300 300] /Contents 4 0 R /Resources << /XObject << /Im 5 0 R /Fm 6 0 R >> >> >>".to_vec(),
stream("", b"q 50 0 0 50 20 20 cm /Im Do Q /Fm Do"),
stream(
"/Type /XObject /Subtype /Image /Width 11 /Height 1 /ColorSpace /DeviceGray /BitsPerComponent 8 /Alternates [<< /Image 7 0 R /DefaultForPrinting true >>]",
b"SHOWNIMAGE!",
),
stream("/Type /XObject /Subtype /Form /BBox [0 0 300 300] /OPI << /1.3 << /Type /OPI /Version 1.3 /F (OPISERVERPATH.tif) >> >>", b""),
stream("/Type /XObject /Subtype /Image /Width 17 /Height 1 /ColorSpace /DeviceGray /BitsPerComponent 8", b"ALTERNATEORIGINAL"),
]);
assert_eq!(scan(&doc).into_iter().find(|c| c.0 == Hidden::PrivateData), Some((Hidden::PrivateData, 2)));
assert_eq!(remove_hidden(&mut doc, &[Hidden::PrivateData]).unwrap(), [(Hidden::PrivateData, 2)]);
let bytes = write_full(&doc, &SaveOptions { object_streams: false, ..SaveOptions::default() }).unwrap();
let has = |needle: &[u8]| bytes.windows(needle.len()).any(|w| w == needle);
assert!(has(b"SHOWNIMAGE!"), "the image the page shows stays");
assert!(!has(b"ALTERNATEORIGINAL"), "the alternate image survived the save");
assert!(!has(b"OPISERVERPATH"), "the OPI link survived the save");
}
/// The full rewrite of the document made of `objs` after Remove Hidden Information, run for its
/// metadata so it has something to do.
fn hidden_information_removed(objs: Vec<Vec<u8>>) -> Vec<u8> {
use crate::sanitize::{Hidden, remove_hidden};
let mut doc = pdf(objs);
let info = doc.add(Object::Dict({
let mut d = Dict::new();
d.set(b"Title".to_vec(), pdfcraft_cos::PdfString::text("Plan"));
d
}));
doc.trailer_mut().set(b"Info".to_vec(), Object::Ref(info));
remove_hidden(&mut doc, &[Hidden::Metadata]).unwrap();
write_full(&doc, &SaveOptions { object_streams: false, ..SaveOptions::default() }).unwrap()
}
fn gray_image(data: &[u8]) -> Vec<u8> {
stream(&format!("/Type /XObject /Subtype /Image /Width {} /Height 1 /ColorSpace /DeviceGray /BitsPerComponent 8", data.len()), data)
}
/// An image Delete Image took off the page is still listed in its resources: Remove Hidden
/// Information unlinks it, so the full rewrite leaves it out, as it does one only an unused form
/// draws. Images a page still draws stay, directly, through a form without /Resources, or on
/// another page sharing the dictionary, its name and `Do` in two content streams.
#[test]
fn images_no_page_draws_leave_with_hidden_information() {
let bytes = hidden_information_removed(vec![
b"<< /Type /Catalog /Pages 2 0 R >>".to_vec(),
b"<< /Type /Pages /Kids [3 0 R 4 0 R] /Count 2 >>".to_vec(),
b"<< /Type /Page /Parent 2 0 R /MediaBox [0 0 300 300] /Contents 5 0 R /Resources 7 0 R >>".to_vec(),
b"<< /Type /Page /Parent 2 0 R /MediaBox [0 0 300 300] /Contents [6 0 R 15 0 R] /Resources 7 0 R >>".to_vec(),
stream("", b"q 50 0 0 50 20 20 cm /Shown Do Q /Fm Do"),
stream("", b"q 50 0 0 50 20 20 cm /Other"),
b"<< /XObject << /Shown 8 0 R /Deleted 9 0 R /Nested 10 0 R /Other 11 0 R /Fm 12 0 R /Unused 13 0 R /Orphan 14 0 R >> >>".to_vec(),
gray_image(b"SHOWNIMAGEBYTES"),
gray_image(b"DELETEDIMAGEBYTES"),
gray_image(b"NESTEDIMAGEBYTES"),
gray_image(b"OTHERPAGEIMAGE"),
stream("/Type /XObject /Subtype /Form /BBox [0 0 300 300]", b"q 10 0 0 10 200 200 cm /Nested Do Q"),
stream("/Type /XObject /Subtype /Form /BBox [0 0 300 300]", b"/Orphan Do"),
gray_image(b"ORPHANIMAGEBYTES"),
stream("", b"Do Q"),
]);
let has = |needle: &[u8]| bytes.windows(needle.len()).any(|w| w == needle);
assert!(!has(b"DELETEDIMAGEBYTES"), "the deleted image survived the save");
assert!(!has(b"ORPHANIMAGEBYTES"), "an image only an unused form draws survived the save");
for kept in [&b"SHOWNIMAGEBYTES"[..], b"NESTEDIMAGEBYTES", b"OTHERPAGEIMAGE"] {
assert!(has(kept), "{} is drawn and must stay", String::from_utf8_lossy(kept));
}
}
/// A page listed under two /Pages nodes draws through either one's resources, whichever a reader
/// comes to it from: neither loses the image it draws.
#[test]
fn a_page_listed_twice_keeps_both_parents_images() {
let bytes = hidden_information_removed(vec![
b"<< /Type /Catalog /Pages 2 0 R >>".to_vec(),
b"<< /Type /Pages /Kids [3 0 R 4 0 R] /Count 2 >>".to_vec(),
b"<< /Type /Pages /Parent 2 0 R /Kids [5 0 R] /Count 1 /Resources << /XObject << /Im 7 0 R >> >> >>".to_vec(),
b"<< /Type /Pages /Parent 2 0 R /Kids [5 0 R] /Count 1 /Resources << /XObject << /Im 8 0 R >> >> >>".to_vec(),
b"<< /Type /Page /Parent 3 0 R /MediaBox [0 0 300 300] /Contents 6 0 R >>".to_vec(),
stream("", b"q 50 0 0 50 20 20 cm /Im Do Q"),
gray_image(b"FIRSTPARENTIMAGE"),
gray_image(b"SECONDPARENTIMAGE"),
]);
let has = |needle: &[u8]| bytes.windows(needle.len()).any(|w| w == needle);
for kept in [&b"FIRSTPARENTIMAGE"[..], b"SECONDPARENTIMAGE"] {
assert!(has(kept), "{} is drawn and must stay", String::from_utf8_lossy(kept));
}
}
/// Unlinking an image nothing draws is a removal of its own: with nothing else to remove, it is
/// still applied, not reported as nothing to do.
#[test]
fn unlinking_alone_still_applies() {
use crate::sanitize::{Hidden, remove_hidden};
let mut doc = pdf(vec![
b"<< /Type /Catalog /Pages 2 0 R >>".to_vec(),
b"<< /Type /Pages /Kids [3 0 R] /Count 1 >>".to_vec(),
b"<< /Type /Page /Parent 2 0 R /MediaBox [0 0 300 300] /Resources << /XObject << /Deleted 4 0 R >> >> >>".to_vec(),
gray_image(b"DELETEDIMAGEBYTES"),
]);
assert_eq!(remove_hidden(&mut doc, &[Hidden::Metadata]), Ok(Vec::new()));
let bytes = write_full(&doc, &SaveOptions { object_streams: false, ..SaveOptions::default() }).unwrap();
assert!(!bytes.windows(17).any(|w| w == b"DELETEDIMAGEBYTES"), "the deleted image survived the save");
}
/// A name a pattern, a form, an appearance or a soft mask doesn't define is looked up in the
/// page's resources, /Resources of its own or not: an image drawn only that way is still shown,
/// so it stays, as one drawn by a form named through an object that holds only a reference. One
/// no content names goes.
#[test]
fn images_drawn_through_the_page_resources_stay() {
let bytes = hidden_information_removed(vec![
b"<< /Type /Catalog /Pages 2 0 R >>".to_vec(),
b"<< /Type /Pages /Kids [3 0 R] /Count 1 >>".to_vec(),
b"<< /Type /Page /Parent 2 0 R /MediaBox [0 0 300 300] /Contents 4 0 R /Resources 5 0 R /Annots [6 0 R] >>".to_vec(),
stream("", b"q /Pattern cs /Pt scn 0 0 10 10 re f Q /Own Do /Loop Do q /Gs gs 0 0 5 5 re f Q /Chained Do"),
b"<< /XObject << /ByPattern 7 0 R /ByOwnForm 8 0 R /ByAppearance 9 0 R /ByMask 10 0 R /Deleted 11 0 R /Own 12 0 R /Loop 16 0 R \
/Chained 17 0 R /ByChain 19 0 R >> /Pattern << /Pt 13 0 R >> /ExtGState << /Gs << /SMask << /S /Luminosity /G 14 0 R >> >> >> >>"
.to_vec(),
b"<< /Type /Annot /Subtype /Square /Rect [0 0 50 50] /AP << /N 15 0 R >> >>".to_vec(),
gray_image(b"BYPATTERNIMAGE"),
gray_image(b"BYOWNFORMIMAGE"),
gray_image(b"BYAPPEARANCEIMAGE"),
gray_image(b"BYMASKIMAGE"),
gray_image(b"DELETEDIMAGEBYTES"),
stream("/Type /XObject /Subtype /Form /BBox [0 0 10 10] /Resources << /ProcSet [/PDF] >>", b"/ByOwnForm Do"),
stream("/Type /Pattern /PatternType 1 /PaintType 1 /TilingType 1 /BBox [0 0 10 10] /XStep 10 /YStep 10", b"/ByPattern Do"),
stream("/Type /XObject /Subtype /Form /BBox [0 0 10 10] /Group << /S /Transparency /CS /DeviceGray >>", b"/ByMask Do"),
stream("/Type /XObject /Subtype /Form /BBox [0 0 50 50]", b"/ByAppearance Do"),
// Draws itself: read once.
stream("/Type /XObject /Subtype /Form /BBox [0 0 10 10] /Resources << /XObject << /Loop 16 0 R >> >>", b"/Loop Do"),
b"18 0 R".to_vec(),
stream("/Type /XObject /Subtype /Form /BBox [0 0 10 10]", b"/ByChain Do"),
gray_image(b"BYCHAINIMAGE"),
]);
let has = |needle: &[u8]| bytes.windows(needle.len()).any(|w| w == needle);
assert!(!has(b"DELETEDIMAGEBYTES"), "the deleted image survived the save");
for kept in [&b"BYPATTERNIMAGE"[..], b"BYOWNFORMIMAGE", b"BYAPPEARANCEIMAGE", b"BYMASKIMAGE", b"BYCHAINIMAGE"] {
assert!(has(kept), "{} is drawn and must stay", String::from_utf8_lossy(kept));
}
}
/// A Type 3 glyph draws through the page's resources too, as does a form in the font's own
/// resources: their images stay, and the dictionary is still cleaned of what nothing draws.
#[test]
fn images_drawn_by_type3_glyphs_stay() {
let bytes = hidden_information_removed(vec![
b"<< /Type /Catalog /Pages 2 0 R >>".to_vec(),
b"<< /Type /Pages /Kids [3 0 R] /Count 1 >>".to_vec(),
b"<< /Type /Page /Parent 2 0 R /MediaBox [0 0 300 300] /Contents 4 0 R /Resources 5 0 R >>".to_vec(),
stream("", b"BT /T3 12 Tf (a) Tj ET"),
b"<< /XObject << /ByGlyph 7 0 R /Deleted 8 0 R /ByFontForm 11 0 R >> /Font << /T3 6 0 R >> >>".to_vec(),
b"<< /Type /Font /Subtype /Type3 /FontBBox [0 0 10 10] /FontMatrix [0.1 0 0 0.1 0 0] /CharProcs << /a 9 0 R >> \
/Encoding << /Differences [97 /a] >> /FirstChar 97 /LastChar 97 /Widths [10] /Resources << /XObject << /Fx 10 0 R >> >> >>"
.to_vec(),
gray_image(b"BYGLYPHIMAGE"),
gray_image(b"DELETEDIMAGEBYTES"),
stream("", b"10 0 d0 /ByGlyph Do /Fx Do"),
stream("/Type /XObject /Subtype /Form /BBox [0 0 10 10]", b"/ByFontForm Do"),
gray_image(b"BYFONTFORMIMAGE"),
]);
let has = |needle: &[u8]| bytes.windows(needle.len()).any(|w| w == needle);
assert!(!has(b"DELETEDIMAGEBYTES"), "the deleted image survived the save");
assert!(has(b"BYGLYPHIMAGE"), "the glyph's image is drawn and must stay");
assert!(has(b"BYFONTFORMIMAGE"), "the image a form in the font's resources draws must stay");
}
/// Layer lists that name one array many times, through arrays that do the same, are cleaned once
/// where each array is, not copied once per path: the file stays as small as it was.
#[test]
fn a_shared_layer_list_is_cleaned_once() {
use crate::sanitize::{Hidden, remove_hidden};
let mut objs = vec![
b"<< /Type /Catalog /Pages 2 0 R /OCProperties << /OCGs [6 0 R 7 0 R] /D << /OFF [6 0 R] /Order [9 0 R 9 0 R] >> >> >>".to_vec(),
b"<< /Type /Pages /Kids [3 0 R] /Count 1 >>".to_vec(),
b"<< /Type /Page /Parent 2 0 R /MediaBox [0 0 300 300] /Contents 4 0 R /Resources << /Font << /F1 5 0 R >> >> >>".to_vec(),
stream("", b"BT /F1 10 Tf 10 100 Td (Kept) Tj ET"),
FONT.replace("95 0 R", "8 0 R").into_bytes(),
b"<< /Type /OCG /Name (SECRETLAYERNAME) >>".to_vec(),
b"<< /Type /OCG /Name (Shown layer) >>".to_vec(),
widths(),
];
// Objects 9 to 22 each name the next one twice; 23 names both layers.
objs.extend((10..24).map(|n| format!("[{n} 0 R {n} 0 R]").into_bytes()));
objs.push(b"[6 0 R 7 0 R]".to_vec());
let mut doc = pdf(objs);
remove_hidden(&mut doc, &[Hidden::HiddenLayers]).unwrap();
let bytes = write_full(&doc, &SaveOptions { object_streams: false, ..SaveOptions::default() }).unwrap();
let has = |needle: &[u8]| bytes.windows(needle.len()).any(|w| w == needle);
assert!(has(b"(Shown layer)") && !has(b"SECRETLAYERNAME"), "the removed layer survived the save");
assert!(bytes.len() < 32 * 1024, "the shared lists were copied once per path: {} bytes", bytes.len());
}
/// A removed off layer leaves every list that names it, or its dictionary, name included, stays in
/// the file: a sub-list of the default `/Order` (its heading too, once it names no layer), a
/// radio-button group (a group left empty goes), a usage rule (one that names it directly too), and
/// the alternate configurations in `/Configs`.
#[test]
fn a_removed_layer_leaves_every_configuration() {
use crate::sanitize::{Hidden, remove_hidden};
let mut doc = pdf(vec![
b"<< /Type /Catalog /Pages 2 0 R /OCProperties << /OCGs [6 0 R 7 0 R] /D << /OFF [6 0 R] /Order [7 0 R [(Group) 6 0 R]] /RBGroups [[6 0 R 7 0 R] [6 0 R]] /AS [<< /Event /Print /OCGs [6 0 R] /Category [/Print] >> << /Event /View /OCGs 6 0 R /Category [/View] >>] >> /Configs [8 0 R] >> >>".to_vec(),
b"<< /Type /Pages /Kids [3 0 R] /Count 1 >>".to_vec(),
b"<< /Type /Page /Parent 2 0 R /MediaBox [0 0 300 300] /Contents 4 0 R /Resources << /Font << /F1 5 0 R >> /Properties << /L1 6 0 R >> >> >>".to_vec(),
stream("", b"/OC /L1 BDC BT /F1 10 Tf 10 50 Td (Gone) Tj ET EMC BT /F1 10 Tf 10 100 Td (Kept) Tj ET"),
FONT.replace("95 0 R", "9 0 R").into_bytes(),
b"<< /Type /OCG /Name (SECRETLAYERNAME) >>".to_vec(),
b"<< /Type /OCG /Name (Shown layer) >>".to_vec(),
b"<< /Name (Other view) /OFF [7 0 R] /ON [6 0 R] /Order [6 0 R 7 0 R] >>".to_vec(),
widths(),
]);
remove_hidden(&mut doc, &[Hidden::HiddenLayers]).unwrap();
let bytes = write_full(&doc, &SaveOptions { object_streams: false, ..SaveOptions::default() }).unwrap();
let has = |needle: &[u8]| bytes.windows(needle.len()).any(|w| w == needle);
assert!(has(b"(Shown layer)") && has(b"(Other view)"), "the layer that stays and the configuration stay");
assert!(!has(b"SECRETLAYERNAME"), "the removed layer survived the save");
assert!(!has(b"(Group)"), "the heading of a sub-list left with no layer stayed");
let catalog = doc.get(doc.root().unwrap());
let config = catalog.as_dict().and_then(|c| c.get(b"OCProperties")).and_then(Object::as_dict).and_then(|p| p.get(b"D")).and_then(Object::as_dict);
let group = Object::Array(vec![Object::Ref(ObjRef::new(7, 0))]);
assert_eq!(config.and_then(|d| d.get(b"RBGroups")), Some(&Object::Array(vec![group])), "a radio-button group left empty stayed");
}
/// A layer the default configuration lists on but a View usage rule turns off is hidden on screen
/// (ISO 32000-2, 8.11.4.4): Hidden layers removes it. A Print rule leaves its layer as it is.
#[test]
fn a_layer_a_view_rule_turns_off_is_hidden() {
use crate::sanitize::{Hidden, remove_hidden, scan};
let mut doc = pdf(vec![
b"<< /Type /Catalog /Pages 2 0 R /OCProperties << /OCGs [6 0 R 7 0 R] /D << /AS [<< /Event /View /OCGs [6 0 R] /Category [/View] >> << /Event /Print /OCGs [7 0 R] /Category [/Print] >>] >> >> >>".to_vec(),
b"<< /Type /Pages /Kids [3 0 R] /Count 1 >>".to_vec(),
b"<< /Type /Page /Parent 2 0 R /MediaBox [0 0 300 300] /Contents 4 0 R /Resources << /Font << /F1 5 0 R >> /Properties << /L1 6 0 R /L2 7 0 R >> >> >>".to_vec(),
stream("", b"/OC /L1 BDC BT /F1 10 Tf 10 50 Td (ScreenOff) Tj ET EMC /OC /L2 BDC BT /F1 10 Tf 10 100 Td (PrintOff) Tj ET EMC"),
FONT.replace("95 0 R", "8 0 R").into_bytes(),
b"<< /Type /OCG /Name (Screen) /Usage << /View << /ViewState /OFF >> >> >>".to_vec(),
b"<< /Type /OCG /Name (Print) /Usage << /Print << /PrintState /OFF >> >> >>".to_vec(),
widths(),
]);
assert!(scan(&doc).into_iter().any(|(h, n)| h == Hidden::HiddenLayers && n > 0));
remove_hidden(&mut doc, &[Hidden::HiddenLayers]).unwrap();
let c = content(&doc, 0);
assert!(!c.contains("ScreenOff") && c.contains("(PrintOff)"), "{c}");
}