Include explicitly referenced production module files in policy checks

This commit is contained in:
Oliver Mee committed 2026-11-16 18:44:09 +08:00
1 parent 59941e75af
commit 41397fa761
5 files changed
+142 -49

No files matched your search

+1 -1
View File
@@ -40,7 +40,7 @@ coverage and secrets passed. The companion CLI was built with locked dependencie
| Objective-C registration identity | Classes and categories listed by both ordinary and non-lazy tables appear once per decoded address, in first-seen order. Distinct definitions with the same name remain separate. | Three original Mach-O regressions fail before the correction; class/category overlap, chained-pointer aliases and same-named distinct definitions pass afterward, with an ordinary class/protocol control. No Objective-C runtime was executed. |
| Paged byte searches | The match cap keeps the earliest file offsets across hex patterns, UTF-8 and UTF-16LE queries. Counts at the exact cap remain complete; excess matches are disclosed as partial. | Engine fixtures cover early text before a full later pattern stream, early wide text before a full narrow stream, the final capped page across all three queries, exact-cap counts and overlapping ties. |
| CArchive filename boundaries | Member names require a NUL terminator inside their TOC record. A damaged name stops the listing with a truncation note; preceding members remain readable. | Old and new cookies, valid listings, a damaged second name and extraction of the retained first member. |
| Source policy gates | Process, UI-side-effect and isolated-session checks inspect Rust paths and explicit import aliases; test exemptions follow cfg predicates and explicit module declarations. Hidden module directories remain in scope. Source read/parse/list failures fail visibly, including in the lock check; symlinked source directories are rejected visibly instead of skipped (linked Rust files are read) | Synthetic source fixtures and `cargo xtask layers` / `locks`; syntax checks only, without macro expansion, re-export resolution or compiler-wide enforcement. The UI gate starts at `ui-egui/src`; explicit UI module paths outside it remain pending |
| Source policy gates | Process, UI-side-effect and isolated-session checks inspect Rust paths and explicit import aliases; test exemptions follow cfg predicates and explicit module declarations. Hidden module directories remain in scope. Source read/parse/list failures fail visibly, including in the lock check; symlinked source directories are rejected visibly instead of skipped (linked Rust files are read) | Synthetic source fixtures and `cargo xtask layers` / `locks`; syntax checks only, without macro expansion, re-export resolution or compiler-wide enforcement. Explicit module files reachable from each source root are included within the workspace, including UI modules outside `src`; missing cfg-dependent files still need compiler checks |
| PyInstaller archives | CArchive members cannot read the table, cookie or appended signing data. Legacy PYZ data members unpack with their original bytes and names. Empty namespace packages are listed as metadata; older namespaces with code still unpack as packages. | CArchive range fixtures cover both cookies, cross-boundary members, signed tails and empty options; PYZ fixtures cover mixed member types. Verified source `a5bcfaed`: [formats tests, fmt and clippy](https://github.com/Afloat16/claude-mem/actions/runs/42636776430/job/129226891917); [layers, locks, secrets, wasm and unchanged Cargo.lock](https://github.com/Afloat16/claude-mem/actions/runs/42640688213/job/129240669663); [unchanged-code regression failures](https://github.com/Afloat16/claude-mem/actions/runs/42636776430/job/129226895582). |
| Expanded PYZ paths | Nested module, resource and unknown-member paths share an 8 MiB retained-byte budget with namespace names across one listing. Complete identities and declared module counts remain available in partial results. | Shared-prefix, multiple-PYZ, Unicode namespace and normal extraction regressions. A native PyInstaller 6.16.0 archive writer produced a 5.04 MB CArchive whose unbounded listing retained 146 MB of path text and aborted under a 128 MiB address-space limit. The bounded reader completes under that limit, retaining 8,386,347 expanded-path bytes plus the existing 3,626-byte outer member name, with truncation reported. |
| Binary property-list object types | Accepts compatible version-zero minor signatures and rejects unsupported major versions, invalid simple/date markers and unrepresentable non-string dictionary keys. | Typed binary fixtures with valid null, string-key and date controls. |
+3 -1
View File
@@ -100,7 +100,9 @@ isn't enough to exempt it. The source-policy checks follow explicit imports and
not macro expansion, globs, re-exports or all Rust name resolution; compiler checks still matter.
Hidden module directories are source too. Symlinked source directories fail visibly rather than
being omitted or followed into cycles; ordinary Rust file links are read and checked. Git metadata
stays out of the scan.
stays out of the directory walk. Module-level declarations explicitly referencing a file
include it even under an otherwise-exempt directory; discovery is cycle-safe and bounded to the
workspace. Missing cfg-dependent files still require compiler validation.
**A heartbeat sampled before shutdown raced a successful stop.** A child could write between
the baseline read and `stop()`, failing the descendant cleanup test even when cleanup worked.
+9 -2
View File
@@ -205,7 +205,7 @@ pub fn check(root: &Path) -> bool {
continue;
}
};
let files = match super::source_policy::read_all(files) {
let files = match super::source_policy::read_all(files, root) {
Ok(files) => files,
Err(e) => {
eprintln!("locks: {e}");
@@ -213,7 +213,14 @@ pub fn check(root: &Path) -> bool {
continue;
}
};
let tests = super::source_policy::test_files(&files);
let tests = match super::source_policy::test_files(&files) {
Ok(tests) => tests,
Err(e) => {
eprintln!("locks: {e}");
bad += 1;
continue;
}
};
for (file, syntax) in files {
if file.canonicalize().is_ok_and(|p| tests.contains(&p)) {
continue;
+61 -7
View File
@@ -9,8 +9,8 @@ mod i18n_coverage;
mod ico;
mod locks;
mod secrets;
mod status;
mod source_policy;
mod status;
/// Workspace crates and their layer. A crate may depend only on strictly lower layers
/// (L0 format crates depend on nothing in the workspace).
@@ -249,7 +249,7 @@ fn process_calls_only_in_platform(root: &Path) -> bool {
}
/// R1.6: UI src paths may read files, but writes and launches belong behind tools.
/// Explicit module paths outside this source root remain a documented follow-up.
/// Explicit module files reachable from this source root are included too.
fn ui_has_no_side_effects(root: &Path) -> bool {
check_source_policy(
root,
@@ -271,7 +271,7 @@ fn check_source_policy(root: &Path, dirs: &[&str], skip: &[&str], policy: source
continue;
}
};
let files = match source_policy::read_all(files) {
let files = match source_policy::read_all(files, root) {
Ok(files) => files,
Err(e) => {
eprintln!("layers: {e}");
@@ -279,7 +279,14 @@ fn check_source_policy(root: &Path, dirs: &[&str], skip: &[&str], policy: source
continue;
}
};
let tests = source_policy::test_files(&files);
let tests = match source_policy::test_files(&files) {
Ok(tests) => tests,
Err(e) => {
eprintln!("layers: {e}");
ok = false;
continue;
}
};
for (file, syntax) in files {
if !matches!(policy, source_policy::Policy::Session) && file.canonicalize().is_ok_and(|p| tests.contains(&p)) {
continue;
@@ -445,6 +452,7 @@ mod tests {
let status = readme.split("\n## Status").nth(1).unwrap();
let status = status.split("\n**Responsible use.**").next().unwrap().replace("\n> ", " ");
assert!(!status.contains("**Next:** the first public release"), "{status}");
}
}
#[cfg(test)]
@@ -638,9 +646,7 @@ mod policy_tests {
std::fs::create_dir_all(source.parent().unwrap()).unwrap();
std::fs::write(&source, "fn f() { std::process::Command::new(\"app\"); std::fs::write(\"out\", b\"x\"); let _ = Session::default(); let n = e.session().a() + e.session().b(); }").unwrap();
assert!(!process_calls_only_in_platform(d.path()), "{relative}");
if !relative.starts_with("..") {
assert!(!ui_has_no_side_effects(d.path()), "{relative}");
}
assert!(!ui_has_no_side_effects(d.path()), "{relative}");
assert!(!no_real_machine_sessions(d.path()), "{relative}");
assert!(!locks::check(d.path()), "{relative}");
std::fs::write(&source, "fn allowed() {}").unwrap();
@@ -692,6 +698,54 @@ mod policy_tests {
assert!(!process_calls_only_in_platform(d.path()), "a linked Rust file is still checked");
}
#[test]
fn explicit_production_modules_override_directory_exemptions() {
let d = fixture("crates/ui-egui/src/lib.rs", "#[path = \"../tests/runtime.rs\"] mod runtime;");
let dir = d.path().join("crates/ui-egui/tests");
std::fs::create_dir_all(&dir).unwrap();
let source = dir.join("runtime.rs");
std::fs::write(&source, "fn f() { std::process::Command::new(\"app\"); std::fs::write(\"out\", b\"x\"); let n = e.session().a() + e.session().b(); }")
.unwrap();
assert!(!process_calls_only_in_platform(d.path()));
assert!(!ui_has_no_side_effects(d.path()));
assert!(!locks::check(d.path()));
std::fs::write(d.path().join("crates/ui-egui/src/lib.rs"), "#[cfg(test)] #[path = \"../tests/runtime.rs\"] mod runtime;").unwrap();
assert!(process_calls_only_in_platform(d.path()));
assert!(ui_has_no_side_effects(d.path()));
assert!(locks::check(d.path()));
std::fs::write(d.path().join("crates/ui-egui/src/lib.rs"), "fn allowed() {}").unwrap();
assert!(process_calls_only_in_platform(d.path()), "unreferenced integration tests stay exempt");
assert!(locks::check(d.path()));
std::fs::write(d.path().join("crates/ui-egui/src/lib.rs"), "#[path = \"../tests/runtime.rs\"] mod runtime;").unwrap();
std::fs::write(&source, [0xff]).unwrap();
assert!(!process_calls_only_in_platform(d.path()));
assert!(!ui_has_no_side_effects(d.path()));
assert!(!no_real_machine_sessions(d.path()));
assert!(!locks::check(d.path()));
}
#[test]
fn referenced_source_discovery_stays_within_the_workspace() {
let outside = tempfile::tempdir().unwrap();
let source = outside.path().join("module.rs");
std::fs::write(&source, "fn allowed() {}").unwrap();
let d = fixture("crates/ui-egui/src/lib.rs", format!("#[path = {source:?}] mod external;"));
let files = source_policy::rust_files(&d.path().join("crates"), &[]).unwrap();
let error = source_policy::read_all(files, d.path()).err().expect("external sources are unavailable");
assert!(error.contains("outside the workspace"), "{error}");
assert!(!process_calls_only_in_platform(d.path()));
assert!(!ui_has_no_side_effects(d.path()));
assert!(!no_real_machine_sessions(d.path()));
assert!(!locks::check(d.path()));
// A missing file under an inactive cfg is legal; the compiler handles active missing
// modules rather than this syntax check guessing the full build configuration.
std::fs::write(d.path().join("crates/ui-egui/src/lib.rs"), "#[cfg(any())] mod absent; fn allowed() {}").unwrap();
assert!(process_calls_only_in_platform(d.path()));
assert!(ui_has_no_side_effects(d.path()));
assert!(no_real_machine_sessions(d.path()));
assert!(locks::check(d.path()));
}
#[test]
fn source_policies_fail_on_unreadable_rust() {
let d = fixture("crates/ui-egui/src/main.rs", [0xff]);
+68 -38
View File
@@ -281,42 +281,62 @@ pub fn problems(file: &syn::File, policy: Policy) -> Vec<(usize, String)> {
finder.found
}
/// Track explicit module declarations so an out-of-line test module has the same exemption as
/// an inline one. A file also used by production stays in scope. Unreferenced files are checked
/// conservatively; macro-generated modules and cross-crate inclusion aren't inferred.
pub fn test_files(files: &[(PathBuf, syn::File)]) -> std::collections::BTreeSet<PathBuf> {
fn modules(items: &[syn::Item], dir: &Path, attr_dir: &Path, test: bool, out: &mut Vec<(PathBuf, bool)>) {
for item in items {
let syn::Item::Mod(module) = item else { continue };
let test = test || test_only(&module.attrs);
let name = module.ident.to_string();
if let Some((_, items)) = &module.content {
modules(items, &dir.join(&name), &attr_dir.join(&name), test, out);
} else {
let custom = module.attrs.iter().find_map(|a| {
if !a.path().is_ident("path") {
return None;
/// Resolve only module-level conventional files and literal `#[path]` declarations. Missing files
/// may belong to an inactive cfg; the compiler decides that. Other inspection errors fail.
fn modules(items: &[syn::Item], dir: &Path, attr_dir: &Path, test: bool, out: &mut Vec<(PathBuf, bool)>) -> Result<(), String> {
for item in items {
let syn::Item::Mod(module) = item else { continue };
let test = test || test_only(&module.attrs);
let name = module.ident.to_string();
if let Some((_, items)) = &module.content {
modules(items, &dir.join(&name), &attr_dir.join(&name), test, out)?;
} else {
let custom = module.attrs.iter().find_map(|a| {
if !a.path().is_ident("path") {
return None;
}
let Meta::NameValue(value) = &a.meta else { return None };
let syn::Expr::Lit(value) = &value.value else { return None };
let syn::Lit::Str(value) = &value.lit else { return None };
Some(attr_dir.join(value.value()))
});
let candidates = custom.map_or_else(|| vec![dir.join(format!("{name}.rs")), dir.join(name).join("mod.rs")], |path| vec![path]);
for path in candidates {
match path.canonicalize() {
Ok(path) => {
out.push((path, test));
break;
}
let Meta::NameValue(value) = &a.meta else { return None };
let syn::Expr::Lit(value) = &value.value else { return None };
let syn::Lit::Str(value) = &value.lit else { return None };
Some(attr_dir.join(value.value()))
});
let path = custom.unwrap_or_else(|| {
let flat = dir.join(format!("{name}.rs"));
if flat.is_file() { flat } else { dir.join(name).join("mod.rs") }
});
if let Ok(path) = path.canonicalize() {
out.push((path, test));
Err(e) if e.kind() == std::io::ErrorKind::NotFound => {}
Err(e) => return Err(format!("cannot inspect module {}: {e}", path.display())),
}
}
}
}
Ok(())
}
fn references(path: &Path, file: &syn::File) -> Result<Vec<(PathBuf, bool)>, String> {
let parent = path.parent().unwrap_or(Path::new(""));
let dir = if path.file_stem().is_some_and(|n| n == "lib" || n == "main" || n == "mod") {
parent.to_path_buf()
} else {
parent.join(path.file_stem().unwrap_or_default())
};
let mut refs = Vec::new();
modules(&file.items, &dir, parent, false, &mut refs)?;
Ok(refs)
}
/// Track explicit module declarations so an out-of-line test module has the same exemption as
/// an inline one. A file also used by production stays in scope. Unreferenced files are checked
/// conservatively; macro-generated modules and cross-crate imports aren't inferred.
pub fn test_files(files: &[(PathBuf, syn::File)]) -> Result<std::collections::BTreeSet<PathBuf>, String> {
let mut incoming: BTreeMap<PathBuf, Vec<(PathBuf, bool)>> = BTreeMap::new();
let mut tests = std::collections::BTreeSet::new();
let mut roots = std::collections::BTreeSet::new();
for (path, file) in files {
let path = path.canonicalize().unwrap_or_else(|_| path.clone());
let path = path.canonicalize().map_err(|e| format!("cannot inspect {}: {e}", path.display()))?;
if test_only(&file.attrs) {
tests.insert(path.clone());
}
@@ -330,14 +350,7 @@ pub fn test_files(files: &[(PathBuf, syn::File)]) -> std::collections::BTreeSet<
if entry || bin {
roots.insert(path.clone());
}
let dir = if path.file_stem().is_some_and(|n| n == "lib" || n == "main" || n == "mod") {
parent.to_path_buf()
} else {
parent.join(path.file_stem().unwrap_or_default())
};
let mut refs = Vec::new();
modules(&file.items, &dir, parent, false, &mut refs);
for (target, test) in refs {
for (target, test) in references(&path, file)? {
incoming.entry(target).or_default().push((path.clone(), test));
}
}
@@ -349,11 +362,28 @@ pub fn test_files(files: &[(PathBuf, syn::File)]) -> std::collections::BTreeSet<
}
}
if tests.len() == before {
return tests;
return Ok(tests);
}
}
}
pub fn read_all(paths: Vec<PathBuf>) -> Result<Vec<(PathBuf, syn::File)>, String> {
paths.into_iter().map(|path| read(&path).map(|file| (path, file))).collect()
/// Include explicit module files even beneath otherwise-exempt directories. Canonical paths
/// bound discovery to the workspace and stop cyclic or shared module references repeating.
pub fn read_all(mut paths: Vec<PathBuf>, root: &Path) -> Result<Vec<(PathBuf, syn::File)>, String> {
let workspace = root.canonicalize().map_err(|e| format!("cannot inspect workspace {}: {e}", root.display()))?;
let mut seen = std::collections::BTreeSet::new();
let mut files = Vec::new();
while let Some(path) = paths.pop() {
let canonical = path.canonicalize().map_err(|e| format!("cannot inspect source {}: {e}", path.display()))?;
let relative =
canonical.strip_prefix(&workspace).map_err(|_| format!("source {} is outside the workspace; source policies cannot inspect it", path.display()))?;
if !seen.insert(canonical.clone()) {
continue;
}
let path = root.join(relative);
let file = read(&path)?;
paths.extend(references(&canonical, &file)?.into_iter().map(|(path, _)| path));
files.push((path, file));
}
Ok(files)
}