Commit Graph
1584 Commits
Author SHA1 Message Date
Afloat 503e80fa81 Report incomplete SQLite b-tree walks (R3.12) (#1608)
Null, cyclic and reused b-tree children were skipped while row counts and row pages still read as complete, so a readable sibling could look like the whole table. Such skips, along with missing pages, wrong page kinds, skipped rows and walk limits, now mark a count as a lower bound and a row page as incomplete, while the readable rows are kept.
2026-11-11 12:39:45 +07:00
Afloat d4a39569e3 Report incomplete firmware archive member scans (R3.12) (#1607)
ZIP, tar and cpio readers can stop their member listings early, but firmware extraction ignored their truncation flags, so directory entries could use up the member limit and later files were left out while the result reported complete coverage. A cut-short member listing, in any region or nested archive, now marks the extraction truncated with a limitation saying what to raise or inspect.
2026-11-11 12:39:39 +07:00
Afloat 2fb99e2ba3 Keep nested ZIP members inside the outer firmware archive (R3.12) (#1606)
A ZIP stored inside a firmware ZIP member could supply the first plausible end record, so the scanner cut the outer archive short at the inner one: extraction left out outer members and wrote the inner contents at the wrong path. Each end record and central directory is now matched to the archive whose extent it describes, so a nested ZIP stays a member of the outer one and its siblings are all extracted.
2026-11-11 12:39:34 +07:00
Afloat a67ba2caa9 Bound firmware ZIP decoding by the remaining output budget (R3.12) (#1605)
Firmware extraction decoded ZIP members into a queue before charging their output bytes, so each member got the same remaining budget and a small compressed archive could hold many times the output limit; a member that hit the decode limit also left the truncation flag unset. Paths and the remaining file and byte limits are now checked before a member is decoded, each member is decoded only up to what's left, and a member stopped by the limit marks the extraction truncated.
2026-11-11 12:38:25 +07:00
Afloat 0028b7cbd8 Reuse recovered filesystem bytes during firmware extraction (R3.12) (#1604)
Filesystem recovery already owns each extracted file's bytes, but firmware extraction copied them into its output queue and kept the originals, so a large recovered file took two full copies in memory before it was written. The recovered buffers are now moved into the queue. Path checks, byte and file limits, nested scans and the report are unchanged.
2026-11-11 12:38:18 +07:00
Afloat f57fa23780 Disclose incomplete asset catalog tree reads (R3.12) (#1603)
A truncated asset catalog tree or node header could give no entries and no warning, and apple_assets_list then reported a complete empty listing; other interrupted tree walks kept what they read but called the result complete. Tree completeness is now tracked through the headers, child and leaf links, missing blocks and walk limits. Readable entries are kept, and an incomplete tree marks coverage partial, sets tree_incomplete and says so in the summary. Legal empty trees still read as complete.
2026-11-11 12:36:40 +07:00
Afloat 150fabc9c0 Apply keyed archive pagination before copying object rows (R3.12) (#1602)
apple_archive_decode built every object row before paging and copied a shared class name once per object while counting classes, so a 97 KB archive with 1,024 objects sharing one 64 KiB class name held 64 MiB of class text even when one row was asked for. Only the requested rows are built now, and class names are borrowed from their shared class object while the class counts are taken. Totals, order, offsets, limits and the output shape are unchanged.
2026-11-11 12:33:30 +07:00
Afloat 6b3cb629de Mark shortened Python code inventories as partial (R3.12) (#1596)
The Python bytecode reader shortened long constants and names to its per-string limit and marked them with an ellipsis, but still reported the code inventory as complete, and the ellipsis it added wasn't counted against the overall text budget. A shortened constant, name, file name or qualified name now marks the inventory cut short, the budget counts the marker, and shortened text still ends on a whole character.
2026-11-11 12:31:53 +07:00
Afloat 9c4f081e92 Bound native value-flow work before decoding instructions (R3.11) (#1598)
The value-flow graph decoded every instruction of every block and only then applied its node limit, so a huge function did all that work before most of it was thrown away. The node budget is now spent as nodes are made: parameters first, then instructions, and decoding stops once it runs out. Blocks that weren't decoded in full are tracked so their edges and definitions aren't treated as complete, and the graph is marked truncated.
2026-11-11 12:28:36 +07:00
Afloat 221d9c1911 Reject duplicate keys before summarizing safetensors files (R3.12) (#1597)
The safetensors reader kept the last value for a repeated JSON key, so two tensors with the same name, or a repeated dtype, shape, offset or metadata field, produced a complete-looking report while earlier entries were silently lost. The header is now read through a duplicate-checking JSON reader, and a header with a repeated key in any object is refused as malformed, as the safetensors format requires distinct keys.
2026-11-11 11:51:38 +07:00
Afloat b92931196d Preserve AArch64 exclusive-memory results and semantics (R3.11) (#1592)
AArch64 exclusive stores treated their status result as an input and never defined it, exclusive pair loads defined only their first result, and acquire-exclusive single loads were unknown operations. The value-flow graph lost the status edge into a retry loop's condition and the second loaded value, and pseudo-C could show an exclusive store as an assignment from its status register, or drop an exclusive load from a known import slot. Exclusive stores now write their status (unless it's discarded to WZR) and read their values, exclusive and acquire-exclusive loads are loads, pair forms define both results, and pseudo-C keeps them as memory operations.
2026-11-11 11:51:17 +07:00
김준호 4f61975f25 Pair Rust functions across toolchains by their names without the crate hash (R5.7) (#1589)
app_builds_line_up paired only 1,448 of about 36,000 functions across two Rust toolchains, because every v0 crate hash changed, so exact names rarely matched. A new name_without_hash stage, after exact names, pairs Rust names (legacy and v0) that are equal once demangled without the compiler's hashes, and keeps the instantiating crate so each crate's copy of a generic impl stays separate. Called names are compared the same way, so the structure fingerprint survives a toolchain change, and a pair whose names differ only by hash isn't counted as renamed. On the same two cargo builds this pairs 14,036 functions (13,356 by name), with masked-bytes pairs down from 888 to 233. The crate lookup reads only the end of a name, so a long crafted symbol can't make it slow.
2026-11-11 07:29:35 +07:00
Dylan M. Taylor 55aa8d36f6 packaging: use [[ for bash tests (SonarCloud S7688) (#1588)
* packaging: use [[ for bash tests (SonarCloud S7688)

Behaviour-preserving: `[[ ]]` in place of `[ ]` for bash tests (SonarCloud shell:S7688). Every script has a bash shebang; no test uses `-a`/`-o`, escaped parentheses or an unquoted pattern on the right of `=`. `bash -n` passes on each script; the rest is left to CI.

Clears all 81 open issues: `packaging/{linux,macos,freebsd,web}/*.sh`.

* packaging/web: parse the wasm size override as base 10 under [[

Inside [[ ]], -gt evaluates its operands arithmetically: a leading zero reads as octal and other text as an expression. Read the override as a base-10 number, as photocraft does.
2026-11-10 18:56:01 -05:00
The viewer 3483d7cc6e Keep build activity visible with long waiting notes (R2.9) (#1587)
A long follow-up pasted into the prompt box during a build made its waiting note taller than the window: the note was drawn outside the conversation's scroll area, so it pushed the prompt box off screen and left no room for the live activity. Waiting notes now sit inside the conversation's scroll area, above the live activity, with their Take back and Send now controls scrolling with them. Up arrow still brings back the last waiting note. A smoke test checks a 160-line note in a 600-point window.
2026-11-11 06:27:16 +07:00
RRFZ 3695f540a5 Report contradictory DOS executable size declarations (R3.12) (#1532)
An MZ header with zero pages, a last-page size above 512, a header shorter than its fixed fields or a header larger than the declared image was read into clamped sizes with nothing said. Each of these is now reported as a problem, so the DOS tool marks the file inconsistent and the binary listing warns, while the entry point, relocations and load module still read. Full pages written with a last-page size of 0 or 512 stay clean.
2026-11-11 05:34:19 +07:00
RRFZ 0ba60eb4e2 Reject source map mappings with missing source or name entries (R3.12) (#1526)
A source map segment could point past the end of the sources or names arrays, as ACAA does with one declared source, and the index was kept. Such mappings are now refused as bad mappings. An explicitly unknown source (null) is still accepted.
2026-11-11 05:25:22 +07:00
RRFZ 5eda19a052 Read ROMFS names at the configured name-length limit (R3.12) (#1528)
A ROMFS name exactly as long as the name limit stopped the directory walk, because the scan for its terminator didn't look one byte past the limit: with max_name = 12, readme12.txt was refused. The scan now includes the terminator's byte, and a name one byte longer is still refused.
2026-11-11 05:12:25 +07:00
FredApps 6e4220632a Merge pull request #1483 from storytold/sonar/analysis-scope
sonar: leave out vendor/ and record why SHA-1 stays in the WebSocket handshake and signature checks
2026-11-10 21:12:01 +01:00
FredApps 4659873974 Merge branch 'main' into sonar/analysis-scope 2026-11-10 20:53:54 +01:00
Cr 994d0e1b30 Decode Swift one-time initialization symbols (R3.12) (#1583)
A Swift symbol ending in WZ is a stored variable's one-time initialization function, and one ending in Wz its one-time initialization token. The demangler reported both as "unsupported witness kind", including merged forms such as WZTm. They now read as swift-demangle prints them: the kind and the variable's name, with the surrounding module, type or extension left out and a private variable keeping its discriminator.
2026-11-11 02:53:35 +07:00
FredApps ba36805a61 Merge branch 'main' into sonar/analysis-scope 2026-11-10 20:48:18 +01:00
Afloat e7c1980a19 Keep Unity bundle members within their uncompressed stream (R3.12) (#1574)
The UnityFS directory reader checked that compressed blocks fit the file but returned member offsets and sizes without checking the uncompressed stream they point into, so a member could lie past all data, overflow, or claim bytes in an empty stream. Block sizes are now summed with checked arithmetic, and a member whose range doesn't fit the stream is marked as unreadable. It keeps its directory entry, but its contents aren't read. Listings say how many such members there are, mark coverage partial and count objects as "at least", and the other members stay readable. An overflowing total of block sizes still makes the table unreadable.
2026-11-11 02:47:44 +07:00
Afloat 2d4f83ebf4 Recover ordinary uncompressed cramfs blocks (R3.12) (#1577)
The cramfs reader read the uncompressed-block flag (bit 31) as part of a block's end offset, so valid files and symlinks with blocks stored uncompressed were reported as corrupt. The flag is now separated from the end pointer, in either byte order, before the span is checked, and an uncompressed block is taken as is when its length matches the full or final partial block. Compressed runs, zero-filled holes and the size budgets are unchanged, and direct-pointer blocks are reported as unsupported.
2026-11-11 02:47:30 +07:00
FredApps 8f927018d0 Merge branch 'main' into sonar/analysis-scope 2026-11-10 20:41:49 +01:00
Cr f790c3ae9e Decode Swift outlined variable symbols (R3.12) (#1581)
Tv with an optional index is an outlined variable, the compiler's copy of a value named after the symbol before it (Tv_ is #0, Tv0_ is #1). The demangler reported every one as "unsupported thunk or specialization". They now read as swift-demangle prints them, "outlined variable #N of" a function, type, protocol witness or other global. A bare name, a module, an outlined variable of an outlined variable, or an index of 2^31 or more is still refused.
2026-11-11 02:41:44 +07:00
FredApps 4e3ff581b8 Merge branch 'main' into sonar/analysis-scope 2026-11-10 20:40:52 +01:00
Afloat 675a7d7724 Recognize required volatile modifiers in CLI signatures (R3.12) (#1580)
The .NET signature reader called a type volatile when any of its first eight custom modifiers contained IsVolatile, so optional modifiers, a Custom.IsVolatile type and IsVolatileExtra all counted, and a real required marker after eight other modifiers was missed. A type is now volatile only when a required modifier is exactly System.Runtime.CompilerServices.IsVolatile, checked across every modifier.
2026-11-11 02:40:34 +07:00
FredApps 5419e8e013 Merge branch 'main' into sonar/analysis-scope 2026-11-10 20:39:55 +01:00
Cr 5cb423e814 Decode Swift function type attributes (R3.12) (#1582)
A Swift function type can carry markers after its parameters and result, and the demangler read only Ya (async): Yb (@Sendable), Yc (a global actor such as @Swift.MainActor), YA (@isolated(any)) and YK (typed throws) were reported invalid, the first failure for 19,217 symbols in macOS and Xcode system binaries, mostly closures and functions that take a @Sendable or @MainActor closure. Function types now keep these attributes and print them as swift-demangle does, with typed throws as throws(E).
2026-11-11 02:39:45 +07:00
FredApps 0488575bf6 Merge branch 'main' into sonar/analysis-scope 2026-11-10 20:37:05 +01:00
Cr 0f2f4d7d2d Decode Swift key path getter and setter symbols (R3.12) (#1579)
A Swift symbol ending in TK is a key path's getter and Tk its setter, with the property or subscript the path reads under the type it starts from. The demangler reported them as "unsupported thunk or specialization". They now read as swift-demangle prints them: key path getter (or setter) for the property with its type, then the base type. A global, module or bare name in either place is still refused.
2026-11-11 02:36:34 +07:00
FredApps 4815705755 Merge branch 'main' into sonar/analysis-scope 2026-11-10 20:34:16 +01:00
Cr 3fbd47a6b3 Name Swift noncanonical specialized metadata symbols (R3.12) (#1578)
Three Swift metadata kinds read with the wrong phrase: MN said "type metadata for" where swift-demangle prints "noncanonical specialized generic type metadata for", Mj said "module descriptor" for what is an "opaque type descriptor accessor key for" a type, and Mz said "type metadata heap reference for" for the "flag for loading of canonical specialized generic type metadata for". They now read as swift-demangle prints them.
2026-11-11 02:34:09 +07:00
FredApps 47ff685d85 Merge branch 'main' into sonar/analysis-scope 2026-11-10 20:32:24 +01:00
Afloat 5120403cb8 Read requested ranges from stored Unity bundle blocks (R3.12) (#1576)
A small metadata or object read from a Unity bundle read and copied every block it touched in full, stored blocks included, so a valid stored block over the 512 MiB read limit made ordinary bundle and object listing fail even for a 496-byte member, and smaller stored blocks allocated bytes nobody asked for. For a stored block the reader now checks that its stored and uncompressed sizes agree and reads just the requested part from the file. Compressed blocks keep the existing decompress-and-cache path, and the 256 MiB per-request limit stays.
2026-11-11 02:32:09 +07:00
FredApps 3c642f6e64 Merge branch 'main' into sonar/analysis-scope 2026-11-10 20:29:37 +01:00
Afloat ffaef8564d Apply global PAX attributes to TAR members (R3.12) (#1570)
The tar reader handled per-member PAX extensions but ignored global ones, so later members could get the wrong path, link target or stored size, and a global size override left the next header at the wrong offset. Global path, link and size attributes are now kept until a later global record replaces or deletes them, applied after per-member PAX values and before GNU long names or the plain header. Each metadata header is read at its own stored length, inherited sizes must be unsigned decimals, and inherited sparse attributes are reported as unsupported. Copies of inherited names share a 64 MiB budget; engine installs refuse an archive that exceeds it before writing anything.
2026-11-11 02:29:14 +07:00
FredApps 08a88e3baf Merge branch 'main' into sonar/analysis-scope 2026-11-10 20:24:06 +01:00
Afloat 1fc942a04f Recover files from big-endian cramfs images (R3.12) (#1575)
The firmware scanner recognised both cramfs byte orders, but the filesystem reader refused big-endian images before reading anything; their inodes also pack the size, name length and offset into different bit positions, so the magic alone wasn't enough. The byte order is now taken from the magic and used for inode modes, sizes, the name-length and offset bitfields (size in the upper 24 bits, name length in the upper 6, offset in the lower 26) and the compressed-block pointers. Directory traversal keeps the same span checks and recovery budgets.
2026-11-11 02:23:22 +07:00
FredApps 4ab1a26727 Merge branch 'main' into sonar/analysis-scope 2026-11-10 20:23:09 +01:00
Afloat 36ff598a44 Bound expanded PYZ identities across archive listings (R3.12) (#1567)
PYZ name limits applied to the names stored in its table, but each nested entry also repeats the surrounding CArchive member path, so a short table under a long shared prefix could hold far more path text than its size suggests: a valid 5 MB archive made with PyInstaller 6.16 kept 146 MB of paths for 40,001 entries. Expanded module, package, resource and unknown-member paths, and namespace-package names, now share an 8 MiB budget per listing, across all nested PYZ members and in standalone PYZ listings. Each complete path is checked before it's built; when the budget runs out, the entries read so far are kept and the listing is marked truncated.
2026-11-11 02:22:32 +07:00
Afloat e957c1d8cb Use version-specific Python headers for extracted PYZ code (R3.12) (#1563)
Code taken out of a PyInstaller PYZ archive was always given a 16-byte .pyc header, although Python 2.5 to 3.2 use 8 bytes and 3.3 to 3.6 use 12, so older code had extra zeros between the header and the marshalled code and couldn't be read at its own header boundary. The output limit reserved 16 bytes too, refusing an exact limit for 2.7 or 3.6 code. Wrapping, header parsing and the PYZ output limit now share one header-size lookup from the release table, and unknown magic numbers keep the 16-byte form. Tests cover standalone and nested PYZ archives, modules and packages, and exact limits.
2026-11-11 02:22:26 +07:00
Afloat 893134c342 Bound expanded ASAR paths and say when an extraction is incomplete (#1558)
ASAR entry and depth limits didn't bound the combined size of expanded paths: a compact directory tree could repeat one long parent prefix across thousands of short names, and a valid 1.5 MB archive kept 145 MB of path text. Expanded entry paths and rejected names now share an 8 MiB budget, checked before each path is built; when it runs out, the entries read so far are kept and the listing is marked truncated. core_archive_extract used to ignore a truncated listing and call the extraction complete; for ASAR and ZIP it now extracts what was listed, adds a limitation that the listing was cut short, and marks coverage partial with no claimed total. The JavaScript app analysis carries the same flag.
2026-11-11 02:22:14 +07:00
FredApps 3b92cedf4e Merge branch 'main' into sonar/analysis-scope 2026-11-10 20:10:10 +01:00
Cr be003e1224 Decode Swift outlined value operation symbols (R3.12) (#1572)
The Swift compiler writes outlined value operations (a copy, move, destroy or enum tag operation for one type) as the type followed by WO and a letter. The demangler read WO as an unsupported witness kind, so these were invalid: 28,456 symbols in macOS and Xcode system binaries, 28,391 of which now decode. Each letter reads as swift-demangle prints it, such as outlined destroy of T, including the indexed enum operations and merged ones.
2026-11-11 02:09:21 +07:00
FredApps c4b96ad81c Merge branch 'main' into sonar/analysis-scope 2026-11-10 20:05:31 +01:00
Cr 80c96e205b Decode Swift opaque result types (R3.12) (#1569)
A Swift function, variable or subscript that returns some P writes that result as Qr (a later opaque type in a tuple is QR with an index), and elsewhere the same opaque type is named by its declaration with QO, generic arguments and Qo. The demangler knew only Qz and Qy, so all of these were invalid ("unsupported associated type"), the first reason for 29,707 symbols in macOS and Xcode system binaries. They now read as swift-demangle prints them.
2026-11-11 02:05:02 +07:00
FredApps d9d121da28 Merge branch 'main' into sonar/analysis-scope 2026-11-10 20:04:33 +01:00
Afloat 6f0092949c Bound retained PE import and export names (R3.12) (#1571)
PE import descriptors and export aliases can point at the same strings again and again, and the row limits didn't bound the text kept: a 526 KB import fixture and a 1.2 MB export fixture each ran out of memory under a 128 MiB limit, and export indexing decoded every name before deciding which rows fit. Retained names (library names, import names, export aliases and forwarders) now draw on a 64 MiB budget, one shared by the normal, delay-load and bound import directories and one for exports. Export names are decoded only for rows that are kept. When the budget runs out, the rows read so far and the declared export count are kept, and the table is marked truncated with a warning.
2026-11-11 02:04:00 +07:00
Afloat 71d97a481b Merge repeated JAR entry sections before checking file digests (R3.12) (#1568)
A JAR manifest may describe one file in several sections, whose attributes merge with later values replacing earlier ones. The integrity check verified each section on its own, so a valid signed manifest could report the same file as both matching and mismatching, and the ZIP entry was read more than once. Digest attributes are now merged by exact entry path before checking: a later section replaces only the attributes it has, the existing digest preference applies, and each file is checked and counted once. Attribute names still match case-insensitively and file paths case-sensitively.
2026-11-11 02:03:54 +07:00