mirror of
https://github.com/storytold/effectcraft.git
synced 2026-11-20 17:31:55 +01:00
* Plug-ins: API 2 (PluginHost, padding, layer_span) Split out of the OpenFX work so it can be reviewed on its own. Additive; the WebAssembly ABI stays at 1. - point3 / layer / text / hidden parameters, integer sliders, parameter groups. - EffectPlugin::padding (transparent pixels around the buffer, at most 4096 per side). - EffectPlugin::render_v2 with a PluginHost: the input at other times, other layers, parameters at other times, frame rate, per-instance key, the layer's source range. - EffectEnv::layer_span from the renderer; MAX_PLUGINS 1024. - plugin_error: a plug-in effect's last render error, for the UI. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01KxacwTpUwqyfpcLgKW8yjS * Plug-ins: external effect helpers (crates/helper-host) Native plug-in formats run in separate helper programs, never in EffectCraft's process (docs/adr/0001-external-effect-helpers.md). New safe-Rust crate effectcraft-helper-host (L3, native only, unsafe forbidden): it finds helper programs, starts each as a child process and registers its effects through plug-in API 2, so the UI and command surface need nothing new. - Protocol v1 (docs/external-helpers.md): JSON lines over stdin / stdout (helper replies are prefixed so plug-in chatter is ignored); pixels go through a frame exchange file (/dev/shm on Linux), never through the pipe. Render-time callbacks for the input at other times, other layers and parameters at other times, with nested renders through the same helper. - A helper that exits, hangs past its timeout or breaks the protocol is stopped. The render fails with a message, the frame is left as it was, and the helper restarts for the next render. An effect that stops its helper 3 times is turned off (helper_blocklist.txt). - Opt-in: nothing starts helpers unless the user asks (Effect > Scan for Effect Helpers), or opts in to starting them at launch with EFFECTCRAFT_HELPERS=1, an environment variable, so writing to the settings folder alone can't run code. effect.helpers.scan / load are refused for confined agent sessions (confine.rs), like file.executeFile. - Discovery: helpers/ next to the app (Contents/Helpers on macOS), helpers/ in the settings folder, EFFECTCRAFT_HELPER_PATH. - Commands effect.helpers.list / scan / load; Effect menu entries (translated); the last render error of an effect shows as a warning sign after its name in Effect Controls. - serve(): the helper side for Rust helpers. tests/it/main.rs runs the whole protocol (render, padding, callbacks, nesting, threads, crash, restart, block list, hang) against a fake helper, with nothing to install. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01KxacwTpUwqyfpcLgKW8yjS * Effect helpers: address review (start failures, frame file mode, nextest listing) - A helper that can't start (not installed any more, restarting too often, no frame exchange file) is reported but no longer charged to the effect being rendered: it is Failure::Refused, not Failure::Broken, so it never counts toward turning that effect off. New check in tests/it: once the helper is out of restarts, renders of an effect fail without blocking it (fails without the fix: the effect was turned off after 3 such renders). - The frame exchange file is created owner-only (0o600) on Unix: it holds the user's frames and /dev/shm is shared. Asserted in the frames unit test. - tests/it (harness = false) speaks the libtest listing protocol cargo-nextest uses: `--list --format terse` prints `helper_protocol: test` (nothing with `--ignored`), and `helper_protocol --exact` runs the checks. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01KxacwTpUwqyfpcLgKW8yjS * Effect helpers: bound writes to a helper, drain stderr, keep callbacks in step - Writes to a helper go through a writer thread with the request timeout: a helper that stops reading its stdin (hung with a large callback answer or render request pending) no longer blocks the render thread forever while holding the helper. - stderr is drained as bytes (lossy, in 64 KiB pieces) to the end: output that isn't UTF-8 stopped the reader, closing the pipe under the helper (EPIPE / SIGPIPE on its next write); lines were unbounded. - Callback images that can't be written are answered with `null` instead of failing the render while the helper waits (which left the protocol out of step); a frame that can't be written before the request is sent is not charged to the effect. - Mid-render calls use the render's own process: no respawn or heartbeat ping while the helper is waiting for an answer. - Waiting for a busy helper is bounded (load timeout) instead of forever; status() no longer blocks on a running render. - Reap the child when start-up fails after spawning; cap the block list read. - Scripts start helpers only with Allow Scripts to Write Files and Access Network on (they run native code), as for File.execute(). - helper-host declares the workspace's no-panic crate lints; the desktop main() keeps its doc comment. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Effect helpers: rustfmt the fake provider, build the test session in its initializer (clippy) Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Plug-in Library: offer WebAssembly API 1 releases only; a helper already stopped mid-render isn't charged again - WasmPlugin loads only modules at the WebAssembly ABI version, which stays 1 while PLUGIN_API_VERSION is 2: share it as effects::plugin::WASM_API_VERSION and have the library's compatibility check use it, so it never installs an API 2 release that can't load. - with_running: a process that is already gone was stopped by what broke it (a nested render, charged there), so this render fails without charging its own effect. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * frames: plug-in effects count as reading other frames (plug-in API 2) With main's narrowed frame invalidation (#2795), an edit inside a precomp under an effect that reads its layer at other times must reach the layer's whole span. Plug-in API 2 gives every plug-in effect the input at other times, so any effect outside ec. counts. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> Co-authored-by: FredApps <153173854+FredApps@users.noreply.github.com>
114 lines
5.7 KiB
Rust
114 lines
5.7 KiB
Rust
//! # effectcraft-plugin (L3)
|
||
//!
|
||
//! Loads **WebAssembly effect plug-ins** (plug-in ABI v1, see `docs/plugins.md`; the native
|
||
//! plug-in API 2 additions such as layer parameters and host services are not part of it) into the effect
|
||
//! registry ([`effectcraft_effects::plugin`]). A plug-in is a `.wasm` (or `.wat`) module with
|
||
//! no imports that exports:
|
||
//!
|
||
//! | export | signature | |
|
||
//! |---|---|---|
|
||
//! | `memory` | memory | linear memory shared with the host |
|
||
//! | `ec_api_version` | `() -> i32` | must return 1 |
|
||
//! | `ec_manifest_ptr`, `ec_manifest_len` | `() -> i32` | a UTF-8 JSON [`PluginManifest`] in memory |
|
||
//! | `ec_alloc` | `(bytes: i32) -> i32` | a buffer of at least `bytes` bytes, 8-byte aligned; it may reuse the previous one |
|
||
//! | `ec_render` | `(pixels: i32, width: i32, height: i32, params: i32, nparams: i32, time: f64, scale: f64) -> i32` | process the pixels in place; 0 = ok |
|
||
//!
|
||
//! For each render the host asks `ec_alloc` for one buffer holding the parameters (`nparams`
|
||
//! little-endian `f64`s) followed by the pixels (`width × height × 4` little-endian `f32`s,
|
||
//! premultiplied RGBA, row-major), calls `ec_render`, and reads the pixels back.
|
||
//!
|
||
//! Modules run in the [wasmi](https://github.com/wasmi-labs/wasmi) interpreter (pure Rust,
|
||
//! MIT/Apache-2.0) with deterministic floats, no imports (no files, network or clock) and a
|
||
//! fuel limit per frame, so a plug-in can't escape its sandbox, hang the app, or render
|
||
//! differently on another machine. The runtime is the `wasm` feature (on for the desktop app and
|
||
//! the CLI); without it [`load_wasm`] says plug-ins are unavailable.
|
||
|
||
#![deny(clippy::unwrap_used, clippy::expect_used, clippy::panic, clippy::unimplemented, clippy::todo, clippy::unreachable)]
|
||
|
||
use effectcraft_effects::EffectSpec;
|
||
pub use effectcraft_effects::plugin::{EffectPlugin, PLUGIN_API_VERSION, PluginFrame, PluginManifest, PluginParams, WASM_API_VERSION, register_plugin};
|
||
|
||
#[cfg(feature = "wasm")]
|
||
mod wasm;
|
||
|
||
/// Whether this build can load WebAssembly plug-ins.
|
||
pub fn wasm_available() -> bool {
|
||
cfg!(feature = "wasm")
|
||
}
|
||
|
||
/// Load and register a WebAssembly plug-in (`bytes` = a `.wasm` binary or `.wat` text;
|
||
/// `source` = where it came from, for listings). Returns the registered effect.
|
||
pub fn load_wasm(bytes: &[u8], source: &str) -> Result<&'static EffectSpec, String> {
|
||
#[cfg(feature = "wasm")]
|
||
{
|
||
let p = wasm::WasmPlugin::new(bytes, source)?;
|
||
register_plugin(std::sync::Arc::new(p))
|
||
}
|
||
#[cfg(not(feature = "wasm"))]
|
||
{
|
||
let _ = (bytes, source);
|
||
Err("WebAssembly plug-ins are not available in this build".into())
|
||
}
|
||
}
|
||
|
||
/// [`load_wasm`] for the engine's plug-in loader hook: `{id, name, category, version, api}`.
|
||
pub fn loader(bytes: &[u8], source: &str) -> Result<serde_json::Value, String> {
|
||
let spec = load_wasm(bytes, source)?;
|
||
let m = effectcraft_effects::plugin::plugin(spec.id).map(|p| p.manifest().clone());
|
||
Ok(serde_json::json!({
|
||
"id": spec.id,
|
||
"name": spec.name,
|
||
"category": spec.category,
|
||
"version": m.as_ref().map(|m| m.version.clone()).unwrap_or_default(),
|
||
"api": m.as_ref().map(|m| m.api).unwrap_or(PLUGIN_API_VERSION),
|
||
"params": spec.params.len(),
|
||
}))
|
||
}
|
||
|
||
/// Render a module's effect without registering it (the Plug-in Library's preview): default
|
||
/// parameters, `pixels` (premultiplied RGBA, `width × height`) at each of `times` (seconds).
|
||
/// Returns the manifest as JSON and one frame per time.
|
||
pub fn preview(bytes: &[u8], width: u32, height: u32, pixels: &[[f32; 4]], times: &[f64]) -> Result<(serde_json::Value, Vec<Vec<[f32; 4]>>), String> {
|
||
#[cfg(feature = "wasm")]
|
||
{
|
||
use effectcraft_effects::plugin::{EffectPlugin, PluginFrame, PluginParamKind, PluginParams};
|
||
let p = wasm::WasmPlugin::new(bytes, "preview")?;
|
||
let m = p.manifest();
|
||
let mut flat = Vec::with_capacity(m.flat_len());
|
||
for param in &m.params {
|
||
match ¶m.kind {
|
||
PluginParamKind::Slider { default, .. } | PluginParamKind::Angle { default } => flat.push(*default),
|
||
PluginParamKind::Checkbox { default } => flat.push(if *default { 1.0 } else { 0.0 }),
|
||
PluginParamKind::Popup { default, .. } => flat.push(*default as f64),
|
||
// Points default to fractions of the layer: the frame is the layer.
|
||
PluginParamKind::Point { default } => flat.extend([default[0] * f64::from(width), default[1] * f64::from(height)]),
|
||
PluginParamKind::Color { default } => flat.extend(default.iter().copied()),
|
||
// API 2 kinds: a WebAssembly module can't declare them (`WasmPlugin::new` refuses them).
|
||
PluginParamKind::Point3 { default } => flat.extend(default.iter().copied()),
|
||
PluginParamKind::Layer => flat.push(-1.0),
|
||
PluginParamKind::Text { .. } | PluginParamKind::Hidden { .. } => {}
|
||
}
|
||
}
|
||
let params = PluginParams { manifest: m, flat: &flat, texts: &[] };
|
||
let mut frames = Vec::with_capacity(times.len());
|
||
for &t in times {
|
||
let mut px = pixels.to_vec();
|
||
let mut frame = PluginFrame { width, height, pixels: &mut px, scale: 1.0, origin: [0.0; 2], layer_size: [f64::from(width), f64::from(height)] };
|
||
p.render(&mut frame, ¶ms, t)?;
|
||
frames.push(px);
|
||
}
|
||
let manifest = serde_json::to_value(m).map_err(|e| e.to_string())?;
|
||
Ok((manifest, frames))
|
||
}
|
||
#[cfg(not(feature = "wasm"))]
|
||
{
|
||
let _ = (bytes, width, height, pixels, times);
|
||
Err("WebAssembly plug-ins are not available in this build".into())
|
||
}
|
||
}
|
||
|
||
// Two attributes, so clippy sees `cfg(test)` and treats the module as test code.
|
||
#[cfg(test)]
|
||
#[cfg(feature = "wasm")]
|
||
mod tests;
|