Files
craft-ai-integrations/tools/mutation
JeremyandClaude Opus 5.5 db30646d9d fix(ai-client): ask before running a ComfyUI workflow with add-on steps not known to stay local (#191)
* test(ai-native): read access lists in-process in the Windows tests

The owner-only tests read each file's access list back with PowerShell's
Get-Acl. That call failed for every test on the windows-latest CI runner
after #184 merged, while the library code itself was not involved. The
tests now read the owner and list with GetNamedSecurityInfoW, through a
test-only helper in the named unsafe module, and the owner parser no
longer assumes a group section.

* test(ai-native): accept the LA alias for the built-in Administrator

CI runners use the machine's built-in Administrator account (RID 500),
which SDDL writes as LA instead of its full SID. The owner-only tests
compared against the full SID and failed on windows-latest; the files
themselves were correct.

* fix(ai-client): ask before running a ComfyUI workflow with add-on steps not known to stay local

A connection to ComfyUI on this computer is local, but an imported workflow
can pass the picture and prompt to an add-on step that sends them to another
host. Such workflows ran with no consent prompt.

ai-comfyui now keeps a short list of ComfyUI's own steps that keep data on
this computer (local_steps.rs). When the chosen library workflow uses any
other step, the plan marks the job as possibly leaving the computer and names
those steps in the new Disclosure::unknown_steps, so the existing consent
path asks. An agreement records the steps it named and covers only those, so
a workflow that adds a step asks again. start re-reads the workflow and
refuses if its steps changed since the plan. Nothing is refused outright: a
workflow with add-on steps still runs once the person agrees.

The shared disclosure_lines adds a plain line naming the steps, and the
adopting guide says what apps show.

* fix(ai-comfyui): agree to a workflow's exact step names, shorten only the label

Step names were cleaned and cut to 80 characters before consent and the
changed-workflow check compared them, so two steps sharing their first 80
characters looked like one. unknown_steps now returns the exact names; the
consent line in ai-egui cleans and shortens them for display only.

* fix(ai-client): check workflow consent for every provider spelling

* fix: freeze approved workflows and scope consent grants

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-15 00:14:36 +00:00
..

Deliberate-bug checks

Each case breaks one safety rule in the source, runs the one test that should notice, and puts the source back. A case is caught only if the mutated code compiled and the test run failed. A mutant that does not compile counts as not caught, so every survivor is either a test gap (fix the test) or a bad mutant (fix the mutant). Cargo runs serially, targeted with -p and --locked, honouring CARGO_BUILD_JOBS from the build resource.

Script Groups
mutation_check.py u1 u3 u4 p4 p10 v1 v2 v3 sd vx lc lr nr ct u2 wf (connections, model sources, node packs, media, client flows, declarative adapter, workflow library). Cases for u2 are in mutation_u2.py.
mutation_check.py seg Shared selection contract: points, box, description and checked result metadata.
mutation_check.py rt rten model runner, bounded picture decoding, resampling and prompt assembly.
mutation_check.py rt rten engine picture handling: bounded PNG and JPEG decoding, mask encoding, resampling.
mutation_check.py bp Local Burn weight preparation (ai-burn-setup): source hashes, cache key, damaged saved weights, cancel.
mutation_check.py be Burn engine: saved-weights loading, the panic guard, answer checks, file names, its catalogue route, the .bpk binding rule and the shared architecture revision.
mutation_check.py bo A recipe only built-in runtimes run is never offered, planned, installed, registered or advised as a ComfyUI model.
mutation_check.py fr First-run setup flow: the licence tick only where needed, the one start button, the suggested folder, adopting checked files, the route-aware try-it sample, status and picker when a file has gone, route notes and engine advice. Cases in mutation_first_run.py.
mutation_check.py at AppTools (ai-client/src/app_tools.rs): an unknown tool never becomes another tool, custom tools are listed, a request carries the tool's capability, run returns outputs and reports a failed or cancelled job as an error, one cancel reaches every waiting job, a finished job removes only itself, a job past its wait ends cancelled, a job that will not stop is reported as possibly still running, an unreadable answer asks the job to stop, a text reply is kept, an online job is not asked many times a second, an unreadable store is not an unknown tool, a cancel does not take a finished job's result away, and a zero wait starts nothing. Cases in mutation_app_tools.py.
mutation_check.py gs Generic selection: scratch sinks, the refusal of clicks and mapped selection workflows.
mutation_check.py gs Generic selection workflows: the scratch sinks, the refusal of clicks and mapped selection workflows.
mutation_check.py gu Generic selection uploads: one upload per picture, the bounded memory, the resend after a cleared scratch area or a failed job.
mutation_check.py gm Generic selection masks: size, one channel, the empty-mask answer and the picture header.
mutation_check.py s21 SAM 2.1 selection (sam21_session.rs, detector.rs): the Florence-2-large preflight, the detector name and the choice check.
mutation_check.py ca Upstream contract alignment: model-test address provenance, required inputs, and app-facing refusal before key access. Cases in mutation_contract.py.
mutation_check.py ps Node pack status report (pack/status.rs): links, hidden folders, names, the install record.
mutation_check.py vx Vertex express adapter (ai-vertex/src/lib.rs): the API key travels in the x-goog-api-key header, never in the request URL.
mutation_check.py lg Remembered licences (registry/licences.rs, install.rs): terms, versions, local files, damaged record.
mutation_check.py eg Connection panel lifecycle (ai-egui/src/panel.rs): the same-frame Save dedup and the stale-answer generation guard. Cases in mutation_egui.py; receipt and logs in verification/eg-mutations.json.
mutation_check.py mg MCP gateway to the Craft apps: the app list's checks (absolute command, id, cap), the line caps, a timed-out, failed or dead connection, unknown tools refused, a partial or changed tool list, the stdio line cap. Cases in mutation_mcp_gateway.py.
mutation_check.py kf Standard secret chain (secrets/standard.rs): a locked secret file is reported, not read as empty; an empty passphrase counts as none; when to ask for a passphrase. Cases in mutation_keys.py.
mutation_check.py jx The JPEG Extended XMP fragment cap (ai-core/src/jpeg.rs) and its check in the rten and candle engines and the returned-picture decode (ai-media/src/result_image.rs). Cases in mutation_jpeg.py.
mutation_check.py mm Models kept in memory (ai-engine/src/memory.rs, provider.rs): an unload never interrupts a run and is never lost, idle and single-model unloads, the free-memory check before a load. Cases in mutation_engine_memory.py.
mutation_check.py cm The same through the app's client (ai-client/src/engines.rs, factory.rs, client.rs): unloading one model needs its connection, a replaced engine's models are not listed, a free-memory reader applies in any build order. Cases in mutation_engine_memory.py.
mutation_check.py rr Run records (ai-client/src/run_record.rs): error and notice text is cleaned of keys, request text, text parameter values (nested ones too, and each in JSON-escaped form) and links, a line holding a key (plain or JSON-escaped) is refused, an empty secret does not refuse everything, a result link is never kept, results are hashed, and outcome, notices, usage and start time are kept. Cases in mutation_run_record.py.
mutation_check.py rp A reply's cost once parsed (ai-http/src/json.rs): each charge of the estimate, and the byte cap and parse budget of every OpenAI, Gemini, Vertex, declarative and ComfyUI reply reader. Cases in mutation_reply_parse.py.
sam3_mutation_check.py sg1 sg2 sg2b sg2c sg3 (SAM 3 masks, sessions, slots, cancel).
python tools/mutation/mutation_check.py u2            # one group
python tools/mutation/mutation_check.py u2 stale      # cases whose name contains "stale"

The "Deliberate-bug checks" workflow runs a group on Ubuntu and macOS. It takes the group and an optional case-name filter, so one case can be checked on both systems in minutes instead of waiting for the group. A filter that names no case in the group fails instead of reporting success:

group:  p4
filter: input-folder-read

Group ids are historical labels. The header of each test file says what it covers.

Run them exclusively. They snapshot and restore source files, so do not edit, format or build those files while one runs. Afterwards read the restored source and rerun the focused tests.

Logs go to tools/mutation/verification/ (not committed) after passing through scrub.py, which replaces a Windows or Unix profile folder with %USERPROFILE% so no account name is written.

gi checks that Gemini retains unread terminal states and that a running poll does not mark a result delivered. It uses seeded job tracking and a closed loopback endpoint, with no live provider.