Commit Graph
76 Commits
Author SHA1 Message Date
RRFZ a39f0bf2f8 Cover terminal parent segments in both HTML reference types (R3.12) 2026-11-18 07:48:05 -05:00
RRFZ a640dc31b8 Resolve local file URL authorities from the application root (R3.12) 2026-11-18 07:15:44 -05:00
RRFZ 8616a48dc8 fix(analysis): read file://localhost as this computer and keep refs ending in a dot segment off files (R3.12) 2026-11-18 06:48:15 -05:00
BFlat e364d88b1a Merge pull request #2113 from Plastic-time/bugfix/r3.12-js-app-tail-resolution
Follow-up to #2100. A page script written as a `data:` or `blob:` URL was read as a local path, so `data:text/javascript,0//../../boot.js` became a dependency on the app's own `boot.js`. Anything with a URL scheme is now left alone, while a local path whose query holds a URL stays local. A path written with a `./` in the middle reaches the same file as one without. A dependency's file under `node_modules/` no longer wins over the app's own file of the same name. These lookups now use one index built once per app, where each used to scan every file.
2026-11-18 18:43:32 +07:00
RRFZ 99516f7e99 fix(analysis): drop tabs inside HTML URLs, read FILE: in any case and keep trailing-slash paths off files
HTML refs lose interior tabs and newlines before the scheme check, file: is stripped case-insensitively, and a path ending in / resolves to no file in resolve_path or a modulepreload.
2026-11-18 06:02:13 -05:00
RRFZ 957b5b6f69 test(analysis): pin the ledger's tie gap and next-build acceptance
The evidence cache comment says a result counts in this build when it was published before its first check.
2026-11-18 05:25:58 -05:00
RRFZ 1d7011b602 fix(analysis): trim HTML URL attributes, match node_modules segments and keep remote loads as URLs
Spaced HTML refs are classified as a browser reads them, only a whole node_modules directory segment marks dependency code, and remote loadURL targets stay URL nodes.
2026-11-18 05:24:18 -05:00
RRFZ 56e3517f61 fix(analysis): treat any URL scheme as external and index js_app exact tails
data: HTML refs no longer resolve to local files, equivalent tail spellings reach the same file, node_modules listing entries never win a tail, and tails come from one index per analysis.
2026-11-18 05:07:15 -05:00
RRFZ 422e43b5df fix(analysis): check each ledger evidence id once and break verifier ties fully
A result that starts to verify mid-build can no longer prove a row with no selected result, proven needs a selected verifier result, and tied verifiers report the same gap in either order.
2026-11-18 05:00:34 -05:00
BFlat 8f97eebfc1 Merge pull request #2105 from Plastic-time/bugfix/r3.12-js-ledger-storage-and-bindings
The ledger of what a rebuilt JavaScript app must do names, for each entry, the place in the app's map it came from. For storage uses, requests with no method and native modules listed as files, that name didn't match any node in the map, so those entries led nowhere; they now use the map's own ids. The same entry could also be listed twice, and no longer is. And when one owner in the rebuild had several records for an entry, only the first was read: test cases in the later ones were lost, and a failed check after a passing one went unnoticed. All of an owner's records are now read together, and one failed check is enough to mark the entry disproved.
2026-11-18 16:54:39 +07:00
BFlat a33263a523 Merge pull request #2103 from Plastic-time/bugfix/r3.12-gitignore-globstar-escaped-slash
Follow-up to #2087. In a `.gitignore` pattern such as `a/**\/b.js`, the slash after `**` is written with a backslash, and `**` was then read as a single `*`, so `a/x/y/b.js` wasn't ignored. `**` followed by an escaped slash now crosses folders as git reads it, and still needs at least one folder in between.
2026-11-18 16:50:08 +07:00
RRFZ 58d5d35781 Recognize an escaped slash after a gitignore globstar (R3.12) 2026-11-18 04:09:29 -05:00
RRFZ d04467530b Check exact tails before TypeScript fallbacks and skip external HTML references in js_app (R3.12) 2026-11-18 04:00:28 -05:00
BFlat f21b14d001 Merge pull request #2097 from Plastic-time/bugfix/r3.12-js-app-typescript-resolution
In a TypeScript project that hasn't been built, `import "./util.js"` means the file `util.ts`: the import names what the file will be called after building. Mapping such a folder left those imports unresolved, so the app's modules looked unconnected. An import or preload path ending in `.js`, `.mjs`, `.cjs` or `.jsx` now falls back to the TypeScript file beside it when no file of that exact name exists, and `.mts`/`.cts` files and more kinds of `index` file are found. The declaration files `.d.mts` and `.d.cts` are no longer taken for code, as `.d.ts` already wasn't.
2026-11-18 15:47:04 +07:00
RRFZ bdf0854760 Fix js_ledger expectation source_node IDs, candidate deduplication, and same-owner binding aggregation (R3.12) 2026-11-18 03:24:45 -05:00
RRFZ 8213aecf60 Exclude .d.mts/.d.cts declarations and resolve TypeScript source specifiers in js_app (R3.12) 2026-11-18 03:08:03 -05:00
RRFZ 5dc919fd74 Strip trailing dots from recovered module and web-script segments and match known module extensions case-insensitively (R3.12) 2026-11-18 01:46:44 -05:00
BFlat d2f31c11ac Merge pull request #2087 from Plastic-time/bugfix/r3.12-gitignore-pattern-rules
Reading a source folder with its `.gitignore` files applied differed from git in four ways. A pattern starting with a backslash to mean a literal `*`, `[` or `?` was read as a wildcard. A name ending in an escaped space lost the space. `**` inside a name (`a**b.js`) crossed folders, where git treats it as one `*`. And a file under an ignored folder could be brought back by a later `!` rule, which git doesn't allow. Each now matches what `git check-ignore` reports.
2026-11-18 13:04:11 +07:00
RRFZ e113600242 Normalize base_dir segments in Ignore::add_file and cover globstar mutation cases 2026-11-18 00:37:15 -05:00
RRFZ a0ea1bc443 Honour gitignore backslash escapes, quoted trailing spaces, globstar segments, and excluded parent directories in Ignore (R3.12) 2026-11-18 00:09:09 -05:00
RRFZ 2dbed823bc Normalize ancestor directory segments and cover closed Hangul syllables 2026-11-17 23:53:02 -05:00
RRFZ 1904af648c Share full case-folded path and parent-directory collision keys across JS recovery and web script layout (R3.13) 2026-11-17 23:18:44 -05:00
BFlat b9f9a2a9ea Merge pull request #2078 from Plastic-time/bugfix/r3.13-share-secret-redaction-helpers
The live browser and Electron capture tools had their own copy of the code that hides declared secrets, and only that copy hid two secrets that overlap: in a network recording or a stand-in built from one, `Bearer abc` and `abc123456789` in `Bearer abc123456789` left `123456789` showing. All three now use one implementation, in the analysis crate, which hides overlapping matches as one.
2026-11-18 10:39:21 +07:00
BFlat 1e55067102 Merge pull request #2077 from Plastic-time/bugfix/r3.13-web-scripts-layout-collisions
Saving the scripts of a web capture planned two files for URLs that differed only in letter case; on Windows and on a default Mac disk those are one file, so the second script was skipped. Paths are now compared without regard to case. A script whose URL ends in `.JS`, `.MJS` or `.CJS` no longer gets a second `.js` added, and a script captured many times is numbered `.hash.2`, `.hash.3` and so on, where each copy used to add to the last one's name until it was too long to write.
2026-11-18 10:26:27 +07:00
RRFZ 0aa9c1559c Share span-merging secret redaction helpers across HAR, stand-in, and engine (R3.13)
- Update `analysis::web_har::replace_needles_with` to scan each UTF-8 character
  boundary for the longest matching needle and merge overlapping matched byte
  spans before replacing, closing the suffix leak on partially overlapping
  declared secrets in HAR inspection and stand-in builds while keeping distinct
  non-overlapping adjacent matches separate for stand-in deduplication budgets.
- Export `encode_query`, `contains_ascii_ci`, and `replace_needles_with` from
  `analysis::web_har` and call them directly from `engine::tools::web_common`,
  removing the duplicate implementations so all three redactors stay in sync.
2026-11-17 22:05:17 -05:00
RRFZ 5296e50f2b Avoid case collisions, double extensions, and chained suffixes in web_scripts::plan_layout (R3.13) 2026-11-17 21:52:01 -05:00
RRFZ e6199c8ba5 Preserve non-empty vendor versions across duplicate markers in compare_versions 2026-11-17 20:25:49 -05:00
BFlat 9db8830a1a Merge pull request #2063 from Plastic-time/bugfix/r3.12-indexed-sourcemap-content-counts
An indexed source map (one made of sections) always reported 0 sources with embedded content, because only the top level was counted; the count now includes every inline section, nested ones too. A stand-alone map named like `app.map.map` also lost both suffixes when its script name was worked out; only the last `.map` is removed now.
2026-11-18 07:53:23 +07:00
BFlat 83580ca3f4 Merge pull request #2059 from Plastic-time/bugfix/r3.12-js-compare-exclusive-tiers
Comparing two versions of a JavaScript app matched each old module against every rule before moving to the next, so an earlier module's weaker match (same structure, or same id) could take a new module that a later one matched exactly, and one new module could be reported twice. Modules are now matched one rule at a time across all of them, strongest first, and each new module is claimed once.
2026-11-18 07:44:11 +07:00
RRFZ 351444ab7b Count indexed source map content across sections and strip literal map suffix 2026-11-17 19:27:00 -05:00
BFlat 5a86212ea4 Merge pull request #2053 from Plastic-time/bugfix/r3.13-har-secret-encodings
A value the person declared as hidden could still appear in a network capture view or a stand-in when it was written the way a form posts it (spaces as `+` with other characters percent-encoded), with mixed-case percent escapes, or as an encoded form field name. Those spellings are now matched, without regard to letter case, with the longest spelling replaced first.
2026-11-18 07:26:26 +07:00
RRFZ e3ffe6b79e Match comparison modules across exclusive priority passes (R3.12) 2026-11-17 19:00:10 -05:00
RRFZ 53d3ccc768 Retain quote_plus-style form needles alongside WHATWG form encoding 2026-11-17 18:54:08 -05:00
RRFZ 51a4dde273 Separate form and generic safe chars in HAR encode_query 2026-11-17 18:27:14 -05:00
RRFZ 17bf314643 Apply case-insensitive and decoded secret scrubbing in standin 2026-11-17 17:57:21 -05:00
RRFZ 15f2ed0825 Redact form-encoded and mixed-case percent-escaped secrets in HAR views (R3.13) 2026-11-17 16:36:47 -05:00
RRFZ 3ef52a2f07 Read section source content and ignore lists when tracing indexed source maps (R3.13) 2026-11-17 16:36:47 -05:00
RRFZ 7ad785b788 Keep the request content type in recorded HAR bodies (R3.13) 2026-11-16 22:07:09 -05:00
muraa-p 8374ad7154 Record the site's start page the way the walkthrough takes it (#1857)
The Deep website study from #1838 sent `web_walkthrough_record` a step shape it doesn't accept, so no recording was ever written. The step is now the tool's own (a test reads it as the tool's params), request bodies are kept so two operations to one endpoint are told apart, a body cut at its cap is left out rather than replayed incomplete, headers are read from the shape the browser keeps them in, and the file is written with private permissions.
2026-11-14 17:42:46 +07:00
muraa-p 3394cad58f Record a website's traffic in Deep mode and tell the builder about the stand-in backend (#1838)
A Deep website study now records the start page's requests and answers to `capture/original.har` (bodies within their cap, credential headers and declared secrets left out as in every capture; a failure is noted in how-it-works.md, not fatal). When that recording is there, the brief tells the builder that `web_standin_build` and `web_standin_start` can serve those answers to its own front end. Clean room is unchanged. Starting a stand-in automatically stays open (R3.11).
2026-11-14 16:30:07 +07:00
Alex Joaquim Pereira 26ea91abd3 Resolve and disclose application graph import facts (R3.12) (#1795) (#1795)
An application graph resolved only the first 2,000 imports of a module and dropped which per-file fact lists were cut, so later dependencies lost their edge and role and nothing said a list was cut. Every retained import is now resolved, the cut lists are named in a facts_truncated limitation, and js_app_map reports partial coverage for such a graph.
2026-11-13 00:47:39 +07:00
Alex Joaquim Pereira c64b74ec83 Report complete comparison row totals (R3.12) (#1792) (#1792)
app_graphs_compare and js_exports_compare cut added, removed, bundle-module and vendor rows at 2,000 without saying so, and the app comparison's summary reported the cut lengths as totals. Each list now carries its complete total beside the capped rows, the summary uses those totals, and a cut comparison is marked rows_capped with a limitation and partial coverage.
2026-11-13 00:07:18 +07:00
Alex Joaquim Pereira 1fb6cf697a Mark archive comparison rows capped (R3.11) (#1782) (#1782)
core_archives_compare counted every added, removed, changed and moved file but cut each list at max_rows without saying so. It now marks a cut comparison rows_capped, with a limitation that the counts are complete and partial coverage; a comparison that fits keeps its evidence as before.
2026-11-12 21:11:00 +07:00
Alex Joaquim Pereira 71dda0be00 Disclose application-graph edges dropped at the cap (R3.12) (#1780) (#1780)
An application graph dropped edges past 4 x max_nodes without saying so, and js_app_map reported complete coverage even for a graph cut by its node or edge cap. The builder now counts edges the cap turns away and records an edge_cap limitation beside node_cap, and js_app_map marks coverage partial when either cap cut the graph, on every module page.
2026-11-12 20:34:52 +07:00
Alex Joaquim Pereira 664ce030d1 Keep source-bundle comparison totals honest under the row cap (R3.12) (#1778) (#1778)
A source-to-bundle comparison with more scripts than max_rows reported the examined window's counts as if they covered the whole tree, and its count of bundle units without a source was the capped row count. It now says how many scripts were examined beside the total, counts every unclaimed unit while still listing at most max_rows, and marks a capped result with a rows_capped limitation and partial coverage.
2026-11-12 19:46:46 +07:00
Alex Joaquim Pereira d56fa23860 Mark sources of unread external source-map sections uncertain (R3.12) (#1767) (#1767)
An index-map section that points at an external map, which is never fetched, adds no sources, so an application graph treated the map's list as complete. Such sections are now counted (through nested index maps), the graph names them in a limitation, and comparing a reference source tree with the bundle reports an otherwise unmatched source as uncertain with partial coverage instead of absent.
2026-11-12 18:12:13 +07:00
Nicolas Bohn 12aad97757 Keep the sources of a source map refused for bad mappings or section offsets (R3.12) (#1717)
A map with malformed mappings or section offsets was reported unreadable, although its sources list is usually whole, and the list is all an application graph keeps. A new sourcemap::parse_sources reads the sources, content, names and ignore list without decoding mappings or checking section offsets; the application graph re-reads a map parse refuses for those reasons this way, lists its sources with a source_map_mappings_unreadable limitation, and compares them as usual. A map whose decodable mappings name a source past the end of its list, or with a malformed section, is still unreadable.
2026-11-12 09:26:47 +07:00
Nicolas Bohn 438c80082a Report sources of unreadable source maps as uncertain, not missing (R3.12) (#1708)
A source map that was malformed, had an undecodable data URL or was listed but not loaded left its sources looking absent, so a source comparison reported them as confirmed missing. Such a map is now marked unreadable with a limitation, and the comparison reports otherwise unmatched sources as uncertain with partial coverage. A stand-alone .map beside an analysed script is read like a referenced one; other .map files stay listed and unread, so they can't make a comparison uncertain or match code that doesn't ship.
2026-11-12 08:50:04 +07:00
Nicolas Bohn ab79ec87a4 Count source-map sections, check skipped values and keep refused maps' sources uncertain (R3.12) (#1705)
Follow-up to #1703. A sections array of non-objects could cost several times the old parser's memory, skipped values weren't checked as a full read checks them so some broken maps read as valid, entries a repeated key replaces still used the budget, and the analyzer read a map refused at a limit as one with no sources. Index-map sections now count against the entry budget and non-map sections stay small, skipped values get the same string, number and nesting checks, a replaced list is freed and gives its entries back, the default limit is 4,000,000 so a million sources with content fit, and a map refused at any of the parser's limits leaves its sources uncertain with a limitation, while a malformed map is read as before.
2026-11-12 02:43:59 +07:00
Afloat 3a9ad9c6c5 Report uncertain source comparisons after source-map truncation (R3.12) (#1694)
When a source map's inventory was cut by its limits, the source-to-bundle comparison searched only the retained names and reported an omitted source as confirmed unmatched. Sources with no observed match now go in a separate uncertain_sources list with the reason source_map_inventory_truncated, the result says the inventory was truncated, and the evidence carries a limitation and partial coverage; positive matches are unchanged.
2026-11-11 22:17:23 +07:00