Auto: Reuse SBS parsing and fix stale preview cache failures

Integrate main's read limits and preserve the font, evaluator, converter,
and measured preview repairs. Complete the scoped type, comment, and
spacing cleanup.

Reuse one validated XML document for library layout and graph reads.
Associate cache maintenance completion with its submitted revision so
old failures cannot consume new settings or clear requests.
This commit is contained in:
chasse20 committed 2026-11-14 22:52:26 +00:00
commit 1a7225c8c2
158 files changed
+2536 -788

No files matched your search

+1 -2
View File
@@ -22,9 +22,8 @@ jobs:
uses: storytold/craft-common-actions/.github/workflows/gate.yml@main
with:
# Same craft-fonts commit as release.yml.
craft-fonts-ref: 3d9ab3a132ed0eff457e424226a435a901da30d3
craft-fonts-ref: c1db6d1a8ed772191241c3d185e2104be84d8691
os: '["ubuntu-24.04","windows-latest"]'
toolchain: "1.96.0"
# xtask ci runs cargo test (including doctests), not cargo nextest.
tools: ""
packages: >-
+1 -1
View File
@@ -24,5 +24,5 @@ jobs:
# Intentionally tracked at @main (not SHA-pinned) so org-wide CI improvements, security fixes, and build optimizations propagate without PR churn across all craftapp repos.
uses: storytold/craft-common-actions/.github/workflows/freebsd.yml@main
with:
craft-fonts-ref: 3d9ab3a132ed0eff457e424226a435a901da30d3
craft-fonts-ref: c1db6d1a8ed772191241c3d185e2104be84d8691
run-package-check: true
+1 -1
View File
@@ -33,7 +33,7 @@ jobs:
dry-run: ${{ github.event_name == 'schedule' }}
app-name: materialcraft
app-title: "MaterialCraft"
craft-fonts-ref: 3d9ab3a132ed0eff457e424226a435a901da30d3
craft-fonts-ref: c1db6d1a8ed772191241c3d185e2104be84d8691
web-craft-fonts: true
version: ${{ inputs.version }}
secrets:
+77
View File
@@ -0,0 +1,77 @@
# Code style
Style conventions for MaterialCraft's Rust code.
Read alongside [AGENTS.md](AGENTS.md) for repository requirements.
## Naming and Rust conventions
- Use descriptive names that explain purpose; include units where useful. Avoid misleading names
and abbreviations whose meaning is unclear in context.
- Follow Rust naming conventions: `snake_case` for functions, variables, fields, and modules;
`UpperCamelCase` for types and traits; `SCREAMING_SNAKE_CASE` for constants and statics.
- Conventional Rust names are not cryptic abbreviations. Keep names such as `fmt`, `iter`, `len`,
`args`, `err`, `id`, and `ui` when their meaning is clear. Do not expand them just to make them
longer.
- Keep standard file and directory names such as `lib.rs`, `main.rs`, `mod.rs`, `src`, `tests`,
`examples`, and `benches`. Rust keywords such as `mut`, `ref`, `impl`, and `mod` are language
syntax, not naming problems.
- Short names are appropriate in a clear local context: `i` for an index, `x` and `y` for
coordinates, `w` and `h` for image dimensions, and conventional generic parameters such as `T`.
Use more descriptive names when scope or ambiguity makes those hard to follow.
## Magic numbers
- Give non-obvious limits, thresholds, sizes, and tuning values descriptive named constants
instead of leaving unexplained numeric literals in the code.
- Name constants for their purpose, with units where useful, rather than for their literal value.
Keep them in the narrowest useful scope.
- Reuse an existing constant when it represents the same concept. Equal numeric values alone do
not mean two concepts should share a constant.
- Explain the origin or rationale of non-obvious coefficients and tolerances in a short comment.
Use standard library constants for standard mathematical values.
- Keep literals inline when their role is clear, such as zero initialization, incrementing by one,
or dividing by two to find a midpoint. Do not introduce names such as `ZERO` or `TWO` that add
no meaning.
- Apply this guidance to tests too; clear fixture values and expected results can remain literal.
## Formatting and spacing
- Let `cargo fmt` and the repository's [rustfmt.toml](rustfmt.toml) handle mechanical formatting.
Do not introduce a competing indentation or line-wrapping convention.
- Use one blank line between distinct logical steps: input preparation, validation, computation,
mutation, and result construction.
- Keep closely related statements together. Do not add a blank line after every declaration or
fragment a single operation with extra spacing.
- Put a blank line after a multiline early-exit guard, including `if` and `let ... else`, before
the next statement.
- Put a blank line before a comment introducing a new step, with no blank line between the
comment and the code it explains.
- Keep documentation and attributes attached to their declarations.
- Separate function definitions with one blank line. Avoid extra empty lines just inside braces
or between every field, argument, or statement.
- In tests, separate preparation, action, and verification with blank lines when those phases
are distinct; keep related assertions together.
## Source organization
- Give each struct or enum its own source file, with its implementation beside it. Use the full
type name in `snake_case`, such as `mesh_curvature_settings.rs` for `MeshCurvatureSettings`.
- Preserve conventional crate roots and module entry points such as `lib.rs`, `main.rs`, and
`mod.rs`. Follow the command and node module placement required by `AGENTS.md`.
- Prefer singular folder names for new domain modules, such as `map`, `setting`, or `tracing`.
Keep standard Rust directories and repository-mandated paths unchanged.
- Group related types into focused modules rather than collecting unrelated types in one large file.
- Apply these preferences to new types and source organization changes within scope. Do not
reorganize unrelated existing code to enforce them.
## Comments
- Explain intent or a non-obvious constraint rather than narrating the code.
- Use short `//` comments. Avoid block comments and wrapped prose in ordinary implementation
comments; adjacent comment lines should each express a concise point.
- Do not add trailing punctuation to ordinary code comments.
- Preserve explanations of guards and invariants. Remove stale or redundant comments without
losing necessary rationale.
- API documentation, required license headers, and examples may use conventional multiline
formatting and punctuation.
- Apply the same comment guidance to production code and tests.
+7
View File
@@ -141,6 +141,13 @@ opt-level = 1
lto = "thin"
codegen-units = 1
# Fast optimized verification: each test executable otherwise repeats cross-crate LTO
# `false` keeps thin LTO within each crate; release packaging keeps the profile above
[profile.verify]
inherits = "release"
lto = false
codegen-units = 16
[profile.wasm-release]
inherits = "release"
opt-level = "s"
+21 -6
View File
@@ -68,6 +68,7 @@ pub fn server_token(port: u16) -> Result<(String, Option<PathBuf>), AutomationEr
validate_token(t.trim())?;
return Ok((t.trim().to_string(), None));
}
let token = generate_token()?;
let path = token_file(port);
write_token_file(&path, &token).map_err(|e| {
@@ -111,12 +112,15 @@ fn read_token_file(path: &Path) -> std::io::Result<String> {
if !meta.is_file() {
return refuse("not a regular file");
}
if meta.uid() != rustix::process::geteuid().as_raw() {
return refuse("owned by another user");
}
if meta.mode() & 0o077 != 0 {
return refuse("readable or writable by other users");
}
file
};
#[cfg(not(unix))]
@@ -146,12 +150,14 @@ pub fn read_bounded_line(r: &mut impl BufRead, line: &mut String) -> std::io::Re
if n == 0 {
return Ok(LineRead::Eof);
}
if n > MAX_REQUEST_BYTES {
if bytes.last() != Some(&b'\n') {
discard_rest_of_line(r)?;
}
return Ok(LineRead::TooLong);
}
*line = String::from_utf8(bytes).map_err(|e| std::io::Error::new(std::io::ErrorKind::InvalidData, e))?;
Ok(LineRead::Line)
}
@@ -162,10 +168,12 @@ fn discard_rest_of_line(r: &mut impl BufRead) -> std::io::Result<()> {
if buf.is_empty() {
return Ok(());
}
if let Some(end) = buf.iter().position(|&b| b == b'\n') {
r.consume(end + 1);
return Ok(());
}
let n = buf.len();
r.consume(n);
}
@@ -179,7 +187,7 @@ pub struct Permit(Arc<AtomicUsize>);
impl ConnectionLimiter {
pub fn try_acquire(&self, max: usize) -> Option<Permit> {
self.0.fetch_update(Ordering::AcqRel, Ordering::Acquire, |n| (n < max).then_some(n + 1)).ok().map(|_| Permit(self.0.clone()))
self.0.try_update(Ordering::AcqRel, Ordering::Acquire, |n| (n < max).then_some(n + 1)).ok().map(|_| Permit(self.0.clone()))
}
}
@@ -201,7 +209,7 @@ impl AuthDeadline {
let stream = stream.try_clone()?;
let (tx, rx) = channel::<()>();
std::thread::Builder::new().name("materialcraft-auth-deadline".into()).spawn(move || {
// Dropping the deadline disconnects the channel and ends this thread at once.
// Dropping the deadline disconnects the channel and ends this thread at once
if let Err(RecvTimeoutError::Timeout) = rx.recv_timeout(timeout) {
let _ = stream.shutdown(Shutdown::Both);
}
@@ -274,24 +282,31 @@ mod tests {
fn unsafe_token_files_are_refused() {
use std::os::unix::fs::PermissionsExt;
let dir = scratch("refuse");
// Readable by others: another user may have planted or read it.
// Readable by others: another user may have planted or read it
let open = dir.join("open.token");
std::fs::write(&open, "planted-token-0123456789").unwrap();
std::fs::set_permissions(&open, std::fs::Permissions::from_mode(0o644)).unwrap();
assert!(read_token_file(&open).is_err());
// A symlink, even to an owner-only file: a client must not send that file as its token.
// A symlink, even to an owner-only file: a client must not send that file as its token
let secret = dir.join("secret");
write_token_file(&secret, "private-0123456789abcdef").unwrap();
let link = dir.join("link.token");
std::os::unix::fs::symlink(&secret, &link).unwrap();
assert!(read_token_file(&link).is_err());
// A FIFO is refused without blocking on open or read.
// A FIFO is refused without blocking on open or read
let fifo = dir.join("fifo.token");
if std::process::Command::new("mkfifo").arg(&fifo).status().is_ok_and(|s| s.success()) {
assert!(read_token_file(&fifo).is_err());
}
// A directory is not a token.
// A directory is not a token
assert!(read_token_file(&dir).is_err());
let _ = std::fs::remove_dir_all(dir);
}
@@ -19,6 +19,7 @@ fn main() -> Result<(), Box<dyn std::error::Error>> {
missing_resources.push(format!("{path}#{}", resource.sbs_path()));
}
}
for graph in &package.graphs {
graphs += 1;
if let Some(function) = &graph.function {
@@ -31,6 +32,7 @@ fn main() -> Result<(), Box<dyn std::error::Error>> {
}
}
}
let result = serde_json::json!({
"files": files.len(), "loaded_packages": library.packages.len(), "graphs": graphs,
"standalone_functions": functions, "resources": resources, "resource_declarations_without_bytes": missing_resources,
+2
View File
@@ -103,6 +103,7 @@ pub fn execute(s: &mut Session, id: &str, params: &J) -> Result<J> {
if let Some(result) = crate::selection_commands::execute(s, id, params) {
return result;
}
(spec.enabled)(s).map_err(|why| EngineError::Disabled { cmd: id.to_string(), why })?;
let result = (spec.run)(s, params)?;
if !s.selection.is_empty() || s.annotation_selection.is_some_and(|selection| !s.annotation_target.items(s).is_ok_and(|items| selection.exists(&items))) {
@@ -179,6 +180,7 @@ pub(crate) fn ids_param(s: &Session, cmd: &str, p: &J) -> Result<Vec<NodeId>> {
for id in &ids {
g.node(*id).map_err(|_| bad(cmd, format!("no node {}", id.0)))?;
}
Ok(ids)
}
+21 -3
View File
@@ -2,6 +2,9 @@ use std::collections::{HashMap, HashSet};
use materialcraft_image::Image;
const DEFAULT_LIMIT_BYTES: usize = 128 << 20;
const MAX_ENTRIES: usize = 128;
/// Decoded bitmaps are expendable: resources and active cooks own their source and output data.
pub(crate) struct DecodedImages {
images: HashMap<(u64, bool), (Image, u64)>,
@@ -12,7 +15,7 @@ pub(crate) struct DecodedImages {
impl Default for DecodedImages {
fn default() -> Self {
Self::new(128 << 20)
Self::new(DEFAULT_LIMIT_BYTES)
}
}
@@ -33,17 +36,21 @@ impl DecodedImages {
if let Some((old, _)) = self.images.remove(&key) {
self.bytes = self.bytes.saturating_sub(bytes(&old));
}
let size = bytes(&image);
// A large decode can serve its current cook without occupying the persistent cache.
// A large decode can serve its current cook without occupying the persistent cache
if size == 0 || size > self.limit {
return;
}
while self.bytes > self.limit - size || self.images.len() >= 128 {
while self.bytes > self.limit - size || self.images.len() >= MAX_ENTRIES {
let Some(key) = self.images.iter().min_by_key(|(_, (_, used))| *used).map(|(key, _)| *key) else { break };
if let Some((old, _)) = self.images.remove(&key) {
self.bytes = self.bytes.saturating_sub(bytes(&old));
}
}
self.tick = self.tick.saturating_add(1);
self.images.insert(key, (image, self.tick));
self.bytes += size;
@@ -78,12 +85,16 @@ mod tests {
cache.insert((0, false), image);
for key in 1..1_000 {
cache.insert((key, key % 2 == 0), Image::filled(4, 4, Kind::Gray, [0.5; 4]));
assert!(cache.bytes <= 128);
assert!(cache.images.len() <= 2);
}
assert!(weak.upgrade().is_none());
assert!(cache.get(&(999, false)).is_some());
cache.retain_resources(&HashSet::new());
assert_eq!(cache.bytes, 0);
assert!(cache.images.is_empty());
}
@@ -94,15 +105,22 @@ mod tests {
for key in 0..2 {
cache.insert((key, false), Image::filled(4, 4, Kind::Gray, [0.5; 4]));
}
assert!(cache.get(&(0, false)).is_some());
cache.insert((2, false), Image::filled(4, 4, Kind::Gray, [0.5; 4]));
assert!(cache.get(&(1, false)).is_none());
let large = Image::filled(16, 16, Kind::Gray, [0.5; 4]);
let weak = Arc::downgrade(&large.data);
cache.insert((3, false), large);
assert!(weak.upgrade().is_none());
assert!(cache.get(&(0, false)).is_some());
cache.clear();
assert_eq!(cache.bytes, 0);
}
}
+12
View File
@@ -2,6 +2,7 @@ pub(crate) fn normalize(path: &str) -> String {
if path.contains("://") {
return path.to_string();
}
#[cfg(not(target_arch = "wasm32"))]
{
use std::path::{Component, PathBuf};
@@ -9,13 +10,16 @@ pub(crate) fn normalize(path: &str) -> String {
if let Ok(path) = path.canonicalize() {
return path.to_string_lossy().into_owned();
}
let absolute = if path.is_absolute() { path } else { std::env::current_dir().unwrap_or_default().join(path) };
// New files need a canonical parent so their identity stays the same after saving
if let (Some(parent), Some(filename)) = (absolute.parent(), absolute.file_name())
&& let Ok(parent) = parent.canonicalize()
{
return parent.join(filename).to_string_lossy().into_owned();
}
let mut normalized = PathBuf::new();
for part in absolute.components() {
match part {
@@ -26,6 +30,7 @@ pub(crate) fn normalize(path: &str) -> String {
part => normalized.push(part.as_os_str()),
}
}
normalized.to_string_lossy().into_owned()
}
#[cfg(target_arch = "wasm32")]
@@ -38,13 +43,16 @@ pub(crate) fn resolve(owner: &str, reference: &str) -> String {
if reference.contains("://") {
return reference.to_string();
}
if owner.starts_with("sbs://") {
return materialcraft_formats::sbs::library::resolve(owner, reference).unwrap_or_else(|_| format!("sbs://{reference}"));
}
let path = std::path::Path::new(reference);
if path.is_absolute() {
return normalize(reference);
}
normalize(&std::path::Path::new(owner).parent().unwrap_or(std::path::Path::new("")).join(path).to_string_lossy())
}
@@ -56,9 +64,11 @@ pub(crate) fn relative(owner: &str, source: &str) -> String {
{
return self::relative(&normalize(owner), &materialcraft_formats::sbs::library::root().join(relative).to_string_lossy());
}
if source.contains("://") {
return source.to_string();
}
let folder = std::path::Path::new(owner).parent().unwrap_or(std::path::Path::new(""));
let from: Vec<_> = folder.components().collect();
let to: Vec<_> = std::path::Path::new(source).components().collect();
@@ -66,6 +76,7 @@ pub(crate) fn relative(owner: &str, source: &str) -> String {
if common == 0 {
return source.replace('\\', "/");
}
let mut path = std::path::PathBuf::new();
for _ in common..from.len() {
path.push("..");
@@ -73,5 +84,6 @@ pub(crate) fn relative(owner: &str, source: &str) -> String {
for part in to.iter().skip(common) {
path.push(part.as_os_str());
}
path.to_string_lossy().replace('\\', "/")
}
+4
View File
@@ -185,6 +185,7 @@ pub(crate) fn read_package_bytes(bytes: &[u8], name: &str, linked: &dyn Fn(&str)
if !name.is_empty() {
crate::linked_files::normalize_references(&mut read.pkg, name);
}
Ok(read)
}
@@ -229,6 +230,7 @@ pub fn save_files_at(session: &Session, path: &str) -> Result<materialcraft_form
if !archive && materialcraft_formats::sbs::library::contains_file(std::path::Path::new(path)) {
return Ok(materialcraft_formats::project::ProjectFiles::build_linked(&package, filename)?);
}
Ok(materialcraft_formats::project::ProjectFiles::build(&package, filename, archive)?)
}
@@ -340,6 +342,7 @@ pub fn export_outputs(s: &Session, request: &ExportRequest) -> Result<Vec<Export
let reason = if request.outputs.is_some() { "the selected outputs are not connected" } else { "the graph has no connected output nodes" };
return Err(bad("file.export", reason));
}
Ok(files)
}
@@ -393,6 +396,7 @@ pub fn add_resource_into(s: &mut Session, file: &str, bytes: Vec<u8>, name: Opti
if let Some(resource) = s.pkg.resources.iter_mut().find(|resource| resource.name == name) {
resource.folder = folder.into();
}
Ok(name)
})
}
+5 -1
View File
@@ -39,6 +39,7 @@ pub fn validate(document: &Graph, target: &FunctionGraphTarget, function: &Funct
if !materialcraft_nodes::function::can_be_output(&node.op) {
return Err("Dot nodes cannot be function outputs".into());
}
let Some(expected) = expected_type(document, target) else { return Ok(()) };
let types = materialcraft_nodes::function::node_types(function)?;
@@ -47,12 +48,15 @@ pub fn validate(document: &Graph, target: &FunctionGraphTarget, function: &Funct
FunctionGraphTarget::Dynamic { node, parameter } => {
document.node(*node).is_ok_and(|node| materialcraft_nodes::accepts_scalar_function(&node.def, parameter))
}
// Quadrant luminosity functions may return one channel or an RGBA value.
// Quadrant luminosity functions may return one channel or an RGBA value
FunctionGraphTarget::Pattern { parameter, .. } => parameter == "opacity",
_ => false,
};
if actual != expected && !(scalar_channel && expected == ValueType::Float4 && actual == ValueType::Float) {
return Err(format!("Output node returns {}; expected {}", actual.label(), expected.label()));
}
Ok(())
}
+2 -1
View File
@@ -59,9 +59,10 @@ mod graph_property_commands;
mod graph_target;
mod history_prefs;
mod library_cache_commands;
mod library_cache_prefs;
pub use graph_target::GraphTarget;
pub use history_prefs::HistoryPrefs;
pub use library_cache_commands::LibraryCachePrefs;
pub use library_cache_prefs::LibraryCachePrefs;
pub mod graph_edit;
pub mod graph_grid;
pub mod inspect;
+8 -25
View File
@@ -2,33 +2,9 @@ use crate::{
Session, cmd,
commands::{CommandSpec, always, bad},
};
use serde::{Deserialize, Serialize};
use serde_json::{Value, json};
/// Disposable library thumbnails; independent of package contents and undo history.
#[derive(Clone, Copy, Debug, PartialEq, Eq, Serialize, Deserialize)]
#[serde(default, rename_all = "camelCase")]
pub struct LibraryCachePrefs {
/// Zero disables disk persistence.
pub disk_mib: u32,
pub max_previews: u32,
}
impl Default for LibraryCachePrefs {
fn default() -> Self {
Self { disk_mib: 64, max_previews: 4096 }
}
}
impl LibraryCachePrefs {
pub const MAX_DISK_MIB: u32 = 65_536;
// Bound directory scans and metadata allocations even with externally supplied settings.
pub const MAX_PREVIEWS: u32 = 65_536;
pub fn sanitized(self) -> Self {
Self { disk_mib: self.disk_mib.min(Self::MAX_DISK_MIB), max_previews: self.max_previews.clamp(1, Self::MAX_PREVIEWS) }
}
}
use crate::library_cache_prefs::LibraryCachePrefs;
fn limit(parameters: &Value, key: &str, default: u32, min: u32, max: u32) -> crate::Result<u32> {
match parameters.get(key) {
@@ -84,7 +60,9 @@ mod tests {
let modified = session.modified;
crate::execute(&mut session, "library.cache.configure", &json!({"diskMib": 512, "maxPreviews": 12000})).unwrap();
let expected = LibraryCachePrefs { disk_mib: 512, max_previews: 12000 };
assert_eq!(session.library_cache_prefs, expected);
for parameters in [
json!({"diskMib": -1}),
json!({"diskMib": 65537}),
@@ -96,12 +74,17 @@ mod tests {
assert!(crate::execute(&mut session, "library.cache.configure", &parameters).is_err());
assert_eq!(session.library_cache_prefs, expected);
}
assert_eq!(crate::execute(&mut session, "library.cache.configure", &json!({})).unwrap(), json!(expected));
crate::execute(&mut session, "library.cache.clear", &json!({})).unwrap();
assert_eq!(session.library_cache_clear_epoch, 1);
assert_eq!(session.history(), history);
assert_eq!(session.modified, modified);
crate::execute(&mut session, "library.cache.configure", &json!({"diskMib": 0})).unwrap();
assert_eq!(session.library_cache_prefs.disk_mib, 0);
}
}
+27
View File
@@ -0,0 +1,27 @@
use serde::{Deserialize, Serialize};
/// Disposable library thumbnails; independent of package contents and undo history.
#[derive(Clone, Copy, Debug, PartialEq, Eq, Serialize, Deserialize)]
#[serde(default, rename_all = "camelCase")]
pub struct LibraryCachePrefs {
/// Zero disables disk persistence.
pub disk_mib: u32,
pub max_previews: u32,
}
impl Default for LibraryCachePrefs {
fn default() -> Self {
Self { disk_mib: 64, max_previews: 4096 }
}
}
impl LibraryCachePrefs {
pub const MAX_DISK_MIB: u32 = 65_536;
// Bound directory scans and metadata allocations even with externally supplied settings
pub const MAX_PREVIEWS: u32 = 65_536;
pub fn sanitized(self) -> Self {
Self { disk_mib: self.disk_mib.min(Self::MAX_DISK_MIB), max_previews: self.max_previews.clamp(1, Self::MAX_PREVIEWS) }
}
}
+24 -11
View File
@@ -27,6 +27,7 @@ impl NodeLibrary {
if cancel.load(Ordering::Relaxed) {
return None;
}
let source = library.packages.get(package)?;
let index = source.graphs.iter().position(|candidate| candidate.function.is_none() && candidate.sbs_path() == graph)?;
let size = size.clamp(1, COOKED_SIZE);
@@ -37,16 +38,20 @@ impl NodeLibrary {
if cancel.load(Ordering::Relaxed) {
return None;
}
let graph = session.pkg.graphs.get(index)?;
let decode = session.resource_decoder();
let mut options = materialcraft_eval::Options::new(&decode);
// Thin lines and fine grain vanish when cooked at thumbnail size, so the graph cooks larger and is scaled down
let cooked_size = size.max(COOKED_SIZE);
options.max_size = Some(cooked_size);
options.cancel = Some(cancel);
// Unconnected experiments and intermediate previews do not contribute to a library tile.
// Unconnected experiments and intermediate previews do not contribute to a library tile
options.only = Some(graph.nodes.iter().filter(|node| node.def == "output").map(|node| node.id).collect());
options.inputs = sample_inputs(graph, cooked_size);
let cooked = materialcraft_eval::cook(&session.pkg, index, &options, &session.cache);
if cooked.cancelled || cancel.load(Ordering::Relaxed) {
return None;
@@ -59,27 +64,34 @@ impl NodeLibrary {
}
}
/// An image for every input node of `graph`, by identifier: normals for normal inputs, colors for color inputs, the height for the rest
/// An image for every image input of `graph`, by identifier: normals for normal inputs, colors for color inputs, the height for the rest
fn sample_inputs(graph: &materialcraft_core::Graph, size: u32) -> BTreeMap<String, Image> {
// Strong relief keeps normal inputs legible in small previews
const NORMAL_INTENSITY: f32 = 4.0;
let mut input_nodes = graph.nodes.iter().filter(|node| matches!(node.def.as_str(), "input_color" | "input_grayscale")).peekable();
if input_nodes.peek().is_none() {
return BTreeMap::new();
}
let height = Image::generate_uv(size, size, Kind::Gray, |u, v| {
[0.5 + 0.25 * (u * std::f32::consts::TAU).sin() + 0.25 * (v * std::f32::consts::TAU).cos(), 0.0, 0.0, 0.0]
});
let color = height.map(Kind::Color, |pixel| [0.15 + 0.7 * pixel[0], 0.35 + 0.3 * pixel[0], 0.8 - 0.5 * pixel[0], 1.0]);
let normal = materialcraft_nodes::atomic::height_to_normal(&height, 4.0, false, true, false);
let mut color = None;
let mut normal = None;
graph
.nodes
.iter()
.filter(|node| matches!(node.def.as_str(), "input_color" | "input_grayscale"))
input_nodes
.map(|node| {
let identifier = materialcraft_eval::identifier(node);
let image = if identifier.to_ascii_lowercase().contains("normal") {
&normal
let image: &Image = if identifier.to_ascii_lowercase().contains("normal") {
normal.get_or_insert_with(|| materialcraft_nodes::atomic::height_to_normal(&height, NORMAL_INTENSITY, false, true, false))
} else if node.def == "input_color" {
&color
// A blue-to-sand palette distinguishes color inputs from the grayscale height
color.get_or_insert_with(|| height.map(Kind::Color, |pixel| [0.15 + 0.7 * pixel[0], 0.35 + 0.3 * pixel[0], 0.8 - 0.5 * pixel[0], 1.0]))
} else {
&height
};
(identifier, image.clone())
})
.collect()
@@ -98,5 +110,6 @@ fn is_flat(image: &Image) -> bool {
}
}
}
(0..3).all(|channel| high[channel] - low[channel] < FLAT_SPREAD)
}
+6
View File
@@ -22,17 +22,21 @@ impl crate::Session {
for document in self.documents.values_mut() {
document.pkg.dependencies.remove(path);
}
#[cfg(not(target_arch = "wasm32"))]
{
let mut package = self.pkg.clone();
// Include inactive dependents so closing a source also refreshes their references
package.graphs.extend(self.documents.values().flat_map(|document| document.pkg.graphs.iter().cloned()));
package.dependencies = self.linked_sources.clone();
for warning in load(&mut package) {
self.warn(warning);
}
self.linked_sources.extend(package.dependencies);
}
self.refresh_links_from(std::collections::BTreeSet::from([path.to_string()]));
}
}
@@ -68,6 +72,7 @@ pub(crate) fn load_references(package: &mut Package, mut pending: Vec<String>) -
warnings.extend(read.warnings);
return Ok(read.pkg);
}
#[cfg(target_arch = "wasm32")]
return Err("external file is unavailable in the browser".into());
#[cfg(not(target_arch = "wasm32"))]
@@ -87,5 +92,6 @@ pub(crate) fn load_references(package: &mut Package, mut pending: Vec<String>) -
Err(error) => warnings.push(format!("Could not resolve {path}: {error}")),
}
}
warnings
}
+14 -3
View File
@@ -16,13 +16,16 @@ fn reaches(package: &Package, changed: &BTreeSet<String>, sources: &BTreeMap<Str
if !visited.insert(file) {
continue;
}
if changed.contains(file) {
return true;
}
if let Some(source) = sources.get(file) {
pending.extend(references(source).filter(|instance| !instance.package.is_empty()).map(|instance| instance.package.as_str()));
}
}
false
}
@@ -129,10 +132,11 @@ impl Session {
changed.insert(path.to_string());
}
}
let previous_sources = sources.clone();
// The full library is available for lookup, but only sources reachable from open work
// need interface refreshes. Cloning every installed graph on each edit scales with the
// entire library and retains compiled function bodies nobody is using.
// Only sources reachable from open work need interface refreshes
// Cloning every installed graph retains unused compiled functions
let mut active = open_paths.clone();
let mut pending: Vec<_> = self
.open_packages()
@@ -146,30 +150,36 @@ impl Session {
pending.extend(references(source).filter(|instance| !instance.package.is_empty()).map(|instance| instance.package.clone()));
}
}
let mut updated = false;
for (path, source) in &previous_sources {
if !active.contains(path) {
continue;
}
let mut package = source.as_ref().clone();
if interfaces(&mut package, &previous_sources) {
sources.insert(path.clone(), Arc::new(package));
updated = true;
}
}
updated |= interfaces(&mut self.pkg, &sources);
for document in self.documents.values_mut() {
updated |= interfaces(&mut document.pkg, &sources);
}
if !updated {
break;
}
}
if reaches(&self.pkg, &changed, &sources) {
self.modified = true;
self.dirty = true;
self.cook_epoch = self.cook_epoch.wrapping_add(1);
}
self.pkg.dependencies = sources.clone();
for document in self.documents.values_mut() {
if reaches(&document.pkg, &changed, &sources) {
@@ -184,6 +194,7 @@ impl Session {
*owned = source.clone();
}
}
self.linked_sources = sources;
}
}
+42 -3
View File
@@ -1,3 +1,10 @@
#[cfg(not(target_arch = "wasm32"))]
use std::io::Read as _;
/// Largest linked resource read from beside an SBS; matches the archive member cap.
#[cfg(not(target_arch = "wasm32"))]
const MAX_LINKED_RESOURCE_BYTES: u64 = 256 << 20;
/// Read resource paths relative to an SBS package, including portable library aliases.
pub fn reader(path: &str) -> impl Fn(&str) -> Option<Vec<u8>> + use<> {
let owner = path.to_string();
@@ -38,14 +45,46 @@ pub fn reader(path: &str) -> impl Fn(&str) -> Option<Vec<u8>> + use<> {
let folder = folder.as_ref()?;
let file = std::fs::canonicalize(folder.join(&relative)).ok()?;
let metadata = std::fs::metadata(&file).ok()?;
// Parent references are legal inside the configured library, never beyond its root.
if !file.starts_with(allowed_root.as_ref()?) || !metadata.is_file() {
// Parent references are legal inside the configured library, never beyond its root
if !file.starts_with(allowed_root.as_ref()?) || !metadata.is_file() || metadata.len() > MAX_LINKED_RESOURCE_BYTES {
return None;
}
std::fs::read(file).ok()
let mut bytes = Vec::new();
std::fs::File::open(&file).ok()?.take(MAX_LINKED_RESOURCE_BYTES + 1).read_to_end(&mut bytes).ok()?;
(bytes.len() as u64 <= MAX_LINKED_RESOURCE_BYTES).then_some(bytes)
}
#[cfg(target_arch = "wasm32")]
None
}
}
#[cfg(test)]
mod tests {
use super::*;
use std::path::PathBuf;
fn temp(name: &str) -> PathBuf {
std::env::temp_dir().join(format!("mc-linked-{}-{name}", std::process::id()))
}
#[test]
fn linked_resources_over_the_cap_are_ignored() {
let dir = temp("cap");
let _ = std::fs::remove_dir_all(&dir);
std::fs::create_dir_all(&dir).unwrap();
let sbs = dir.join("p.sbs");
std::fs::write(&sbs, b"x").unwrap();
let big = dir.join("big.bin");
std::fs::File::create(&big).unwrap().set_len(MAX_LINKED_RESOURCE_BYTES + 1).unwrap();
let read = reader(&sbs.to_string_lossy());
assert!(read("big.bin").is_none(), "a resource over the cap is not read");
std::fs::write(dir.join("small.bin"), b"ok").unwrap();
assert_eq!(read("small.bin").as_deref(), Some(&b"ok"[..]));
let _ = std::fs::remove_dir_all(&dir);
}
}
+35 -3
View File
@@ -140,6 +140,7 @@ impl NodeLibrary {
if !seen.insert(uri) {
continue;
}
let bytes = match library::read(uri) {
Ok(bytes) => bytes,
Err(error) => {
@@ -148,8 +149,16 @@ impl NodeLibrary {
}
};
// A layout's frames name library categories; anything else is a package of graphs
let placements = match materialcraft_formats::sbs::read_layout(&bytes, uri) {
let document = match materialcraft_formats::sbs::SbsDocument::parse(&bytes) {
Ok(document) => document,
Err(error) => {
out.warnings.push(format!("Library package {uri}: {error}"));
continue;
}
};
// Reuse the XML document when the frames do not classify this package as a layout
let placements = match document.layout(uri) {
Ok(frames) => placements(&frames),
Err(error) => {
out.warnings.push(format!("Library package {uri}: {error}"));
@@ -162,7 +171,7 @@ impl NodeLibrary {
continue;
}
match materialcraft_formats::sbs::read_sbs(&bytes, uri, &crate::file_cmds::linked_files(uri)) {
match document.read(uri, &crate::file_cmds::linked_files(uri)) {
Ok(mut read) => {
crate::linked_files::normalize_references(&mut read.pkg, uri);
out.warnings.extend(read.warnings);
@@ -171,11 +180,26 @@ impl NodeLibrary {
Err(error) => out.warnings.push(format!("Library package {uri}: {error}")),
}
}
let mut registry = Package::new("Library");
registry.dependencies = std::mem::take(&mut out.packages);
let pending = registry.dependencies.keys().cloned().collect();
out.warnings.extend(crate::linked_files::load_references(&mut registry, pending));
out.packages = registry.dependencies;
// Package identities stay distinct while equal immutable resource payloads share storage
let mut buffers = BTreeMap::<u64, materialcraft_core::ResourceBytes>::new();
for package in out.packages.values_mut().filter_map(Arc::get_mut) {
for resource in &mut package.resources {
match buffers.get(&resource.bytes.content_hash()) {
Some(bytes) if bytes == &resource.bytes => resource.bytes = bytes.clone(),
_ => {
buffers.insert(resource.bytes.content_hash(), resource.bytes.clone());
}
}
}
}
out
}
@@ -194,6 +218,7 @@ impl NodeLibrary {
{
return;
}
// The same node may sit in two categories; the id keeps each placement apart
if !ids.insert(entry.id.clone()) {
entry.id = format!("{}@{}", entry.id, entry.category);
@@ -208,6 +233,7 @@ impl NodeLibrary {
let id = format!("item:{}", category.to_ascii_lowercase());
push(&mut out, entry(id, category, "Graph items", "", item, None));
}
push(&mut out, entry("node:dot".into(), "Dot node", "Graph items", "A reroute point for wires.", LibraryItem::Node { def: "dot".into() }, None));
for placement in &self.placements {
@@ -233,6 +259,7 @@ impl NodeLibrary {
if matches!(definition.id, "input_color" | "input_grayscale" | "input_value" | "output") { "Atomic nodes" } else { definition.category };
push(&mut out, entry(id, definition.label, category, definition.description, item, None));
}
for operation in materialcraft_nodes::function::FUNCTIONS {
let id = format!("function:{}", operation.id);
if operation.id == "passthrough" || listed.contains(&id) {
@@ -242,6 +269,7 @@ impl NodeLibrary {
let category = category_path("Function nodes", Some(operation.category));
push(&mut out, entry(id, operation.label, &category, "", LibraryItem::Operation { operation: operation.id.into() }, None));
}
for kind in PatternNodeKind::ALL.into_iter().filter(|kind| *kind != PatternNodeKind::Dot) {
let id = format!("pattern:{}", kind.native_id());
push(&mut out, entry(id, kind.label(), "FxMap nodes", "", LibraryItem::Pattern { kind }, None));
@@ -278,6 +306,7 @@ impl NodeLibrary {
}
}
}
out
}
}
@@ -401,6 +430,7 @@ fn library_category(graph: &materialcraft_core::Graph) -> Option<String> {
if let Some(category) = named(GRAPH_CATEGORIES, graph.sbs_path().rsplit('/').next().unwrap_or_default()) {
return Some(category);
}
if let Some(category) = tags().find_map(|tag| named(TAG_CATEGORIES, tag)) {
return Some(category);
}
@@ -433,6 +463,7 @@ fn function_categories(graph: &materialcraft_core::Graph, helpers_hidden: bool)
}
}
}
if categories.is_empty() {
let path = graph.sbs_path();
match library_category(graph) {
@@ -441,6 +472,7 @@ fn function_categories(graph: &materialcraft_core::Graph, helpers_hidden: bool)
None => categories.push(function_category(&path)),
}
}
categories
}
+24
View File
@@ -4,6 +4,21 @@ use materialcraft_core::Graph;
#[path = "../tests/support/node_library.rs"]
mod fixture;
#[test]
fn loaded_packages_share_equal_resource_buffers_without_merging_resources() {
let files = ["sbs://rt_ao.sbs".to_string(), "sbs://rt_ao_v2.sbs".to_string()];
let library = NodeLibrary::load(&files);
let resources: Vec<_> =
files.iter().map(|file| library.packages[file].resources.iter().find(|resource| resource.source.ends_with("bluenoise_256.png")).unwrap()).collect();
assert!(!resources[0].bytes.is_empty());
assert_eq!(resources[0].bytes, resources[1].bytes);
assert!(resources[0].bytes.ptr_eq(&resources[1].bytes), "equal resource bytes should use one immutable buffer");
assert!(library.packages[&files[0]].graph_by_name("rt_ao").is_some());
assert!(library.packages[&files[1]].graph_by_name("rt_ao").is_some());
}
/// Installing a library must not compile and clone graphs that no open document references.
#[test]
fn unused_library_packages_remain_shared_without_interface_refresh() {
@@ -17,6 +32,7 @@ fn unused_library_packages_remain_shared_without_interface_refresh() {
let mut library = NodeLibrary::default();
library.packages.insert("sbs://unused.sbs".into(), source.clone());
let session = crate::Session::with_library(Package::new("Empty"), Arc::new(library));
assert!(Arc::ptr_eq(&source, &session.pkg.dependencies["sbs://unused.sbs"]));
}
@@ -92,6 +108,7 @@ fn hidden_graphs_and_helper_functions_are_not_listed() {
graph.function = Some(materialcraft_core::FunctionGraph::default());
graph
};
let mut package = Package::new("Tools");
let mut shown = attributed("shown", "Filter", "Blur");
shown.add_node("output", [0.0; 2]);
@@ -113,6 +130,7 @@ fn hidden_graphs_and_helper_functions_are_not_listed() {
("sbs://tools.sbs#shown", "Filters/Blurs"),
("sbs://user.sbs#loose", "Functions"),
];
assert_eq!(listed, BTreeMap::from(expected), "a package of functions alone lists its loose ones");
}
@@ -129,7 +147,9 @@ fn the_whole_bundled_library_is_categorized() {
};
let unknown: Vec<_> = packaged.iter().filter(|entry| !known(&entry.category)).map(|entry| (&entry.id, &entry.category)).collect();
assert!(unknown.is_empty(), "{unknown:?}");
let category = |id: &str| packaged.iter().find(|entry| entry.id == id).map(|entry| entry.category.as_str());
for (id, expected) in [
("sbs://noise_perlin_noise_v2.sbs#perlin_noise", "Texture generators/Noises"),
@@ -324,6 +344,7 @@ fn library_graphs_cook_previews_from_sample_inputs() {
let preview = |package: &str, graph: &str| NodeLibrary::preview(&library, package, graph, 32);
let filter = preview("sbs://blur_hq.sbs", "blur_hq").unwrap();
assert_eq!((filter.w, filter.h), (32, 32));
assert!(preview("sbs://materials/pbr/bricks_001.sbs", "bricks_001").is_some());
assert!(preview("sbs://noise_bnw_spots_1.sbs", "bnw_spots_1").is_some(), "an FX-Map noise reads the variables its size function sets");
@@ -343,10 +364,12 @@ fn repeated_library_previews_release_images_and_source_references() {
let image = NodeLibrary::preview(&library, package, graph, 72).unwrap_or_else(|| panic!("missing preview {package}#{graph}"));
let buffer = Arc::downgrade(&image.data);
drop(image);
assert!(buffer.upgrade().is_none(), "a completed thumbnail must release its image after upload");
assert_eq!(Arc::strong_count(&library), 1);
assert_eq!(library.packages.values().map(Arc::strong_count).collect::<Vec<_>>(), references);
}
println!("completed preview cycle {cycle}");
#[cfg(target_os = "linux")]
if let Ok(status) = std::fs::read_to_string("/proc/self/status") {
@@ -360,6 +383,7 @@ fn repeated_library_previews_release_images_and_source_references() {
fn cancelled_library_previews_stop_before_preparing_the_package() {
let library = Arc::new(NodeLibrary::default());
let cancelled = std::sync::atomic::AtomicBool::new(true);
assert!(NodeLibrary::preview_cancellable(&library, "unused", "unused", 72, &cancelled).is_none());
assert_eq!(Arc::strong_count(&library), 1);
}
+13 -5
View File
@@ -7,6 +7,9 @@ use sha2::{Digest, Sha256};
use crate::NodeLibrary;
const MAX_DEPENDENCY_PACKAGES: usize = 4096;
const RESOURCE_HASH_CHUNK_BYTES: usize = 64 * 1024;
impl NodeLibrary {
/// Hash the source packages and resource bytes used by a library preview. The caller must
/// also key its rendering implementation and display size. Missing dependencies or an
@@ -18,34 +21,39 @@ impl NodeLibrary {
let mut pending = BTreeSet::from([package]);
let mut visited = BTreeSet::new();
while let Some(path) = pending.pop_first() {
if cancel.load(Ordering::Relaxed) || visited.len() >= 4096 {
if cancel.load(Ordering::Relaxed) || visited.len() >= MAX_DEPENDENCY_PACKAGES {
return None;
}
if !visited.insert(path) {
continue;
}
let source = self.packages.get(path)?;
// Package serialization excludes dependency pointers and resource bytes. Stream the
// model, then hash every byte explicitly, without allocating a second package copy.
// Stream the model to include bytes omitted from package serialization without a copy
serde_json::to_writer(&mut hash, &(path, source.as_ref())).ok()?;
for resource in &source.resources {
hash.update((resource.bytes.len() as u64).to_le_bytes());
for chunk in resource.bytes.as_slice().chunks(64 * 1024) {
for chunk in resource.bytes.as_slice().chunks(RESOURCE_HASH_CHUNK_BYTES) {
if cancel.load(Ordering::Relaxed) {
return None;
}
hash.update(chunk);
}
}
for instance in source.graphs.iter().flat_map(|graph| graph.instances()) {
if !instance.package.is_empty() && !visited.contains(instance.package.as_str()) {
pending.insert(&instance.package);
if pending.len().saturating_add(visited.len()) > 4096 {
if pending.len().saturating_add(visited.len()) > MAX_DEPENDENCY_PACKAGES {
return None;
}
}
}
}
Some(hash.finalize().into())
}
}
@@ -31,21 +31,31 @@ fn key(library: &NodeLibrary) -> Option<[u8; 32]> {
#[test]
fn keys_survive_reloading_and_track_nested_content_not_unrelated_packages() {
let original = library();
assert_eq!(key(&original), key(&library()));
let mut changed = library();
changed.packages.insert("unrelated".into(), Arc::new(Package::new("unused")));
assert_eq!(key(&original), key(&changed));
Arc::make_mut(changed.packages.get_mut("child").unwrap()).graphs[0].seed += 1;
assert_ne!(key(&original), key(&changed));
let mut changed = library();
Arc::make_mut(changed.packages.get_mut("child").unwrap()).resources[0].bytes = ResourceBytes::new(vec![1, 2, 4]);
assert_ne!(key(&original), key(&changed), "same-size replacement must invalidate the preview");
}
#[test]
fn incomplete_and_cancelled_keys_are_not_persisted() {
let mut library = library();
assert!(library.preview_fingerprint("root", "main", &AtomicBool::new(true)).is_none());
library.packages.remove("child");
assert!(key(&library).is_none());
}
+10
View File
@@ -184,6 +184,7 @@ impl Session {
library_sources.insert(path.clone(), package.clone());
}
}
let mut session = Session {
environment: Default::default(),
document_id: 0,
@@ -224,13 +225,16 @@ impl Session {
group: None,
decoded: Arc::new(Mutex::new(crate::decoded_images::DecodedImages::default())),
};
session.refresh_links();
for warning in &library.warnings {
session.warn(warning.clone());
}
if !library.packages.contains_key(crate::function_library::STANDARD_PACKAGE) {
session.warn(format!("Missing function library: {} did not load", crate::function_library::STANDARD_PACKAGE));
}
session.library = library;
session
}
@@ -308,6 +312,7 @@ impl Session {
}
return result;
}
self.begin_group(label);
let r = f(self);
match r {
@@ -318,6 +323,7 @@ impl Session {
}
}
}
r
}
@@ -362,6 +368,7 @@ impl Session {
pub fn changed(&mut self) {
self.modified = true;
self.dirty = true;
// Publish a grouped edit only after it succeeds, so rollback cannot dirty other packages
if self.group.is_none() {
self.refresh_links();
@@ -427,6 +434,7 @@ impl Session {
.flat_map(|(file, package)| package.resources.iter().map(move |resource| (format!("{file}\0{}", resource.name), resource.bytes.clone()))),
);
let memo = self.decoded.clone();
// Release decoded images of deleted or replaced resources
let hashes = res.iter().map(|(_, bytes)| bytes.content_hash()).collect();
memo.lock().unwrap_or_else(PoisonError::into_inner).retain_resources(&hashes);
@@ -437,6 +445,7 @@ impl Session {
if let Some(i) = memo.lock().unwrap_or_else(PoisonError::into_inner).get(&key) {
return Some(i);
}
let img = materialcraft_formats::decode(bytes.as_slice(), linear).ok()?;
memo.lock().unwrap_or_else(PoisonError::into_inner).insert(key, img.clone());
Some(img)
@@ -472,6 +481,7 @@ impl Session {
if result.graph != self.graph || result.cancelled() {
return;
}
self.cooked_graph = Some(result.graph);
self.cooked = result.cooked;
self.material_cooked = result.materials;
+2
View File
@@ -649,9 +649,11 @@ fn library_samplers_read_image_inputs_in_both_processors() {
if definition == "value_processor" {
let expected = if sampler == "samplecol" { Value::Float4(color) } else { Value::Float(color[0]) };
assert_eq!(session.cooked.value(owner, 0), Some(&expected));
} else {
let expected = if sampler == "samplecol" { color } else { [color[0], color[0], color[0], 1.0] };
assert_eq!(session.cooked.image(owner, 0).unwrap().get(0, 0, false), expected);
}
}
+20 -2
View File
@@ -83,17 +83,23 @@ fn function_expansion_allows_several_bounded_bodies_but_still_caps_total_work()
graph.output = output;
graph
};
let mut allowed = caller(3);
function::resolve_instances(&mut allowed, &package, &BTreeMap::new());
assert_eq!(evaluate(&allowed), Value::Float(3.0));
let mut oversized = caller(64);
function::resolve_instances(&mut oversized, &package, &BTreeMap::new());
assert!(function::compile(&oversized, 0).err().unwrap().contains("limit"));
// Compiling a pre-resolved tree must enforce its own budget too.
// Compiling a pre-resolved tree must enforce its own budget too
let source = allowed.nodes.iter().find_map(|node| node.source.clone()).unwrap();
for node in oversized.nodes.iter_mut().filter(|node| node.instance.is_some()) {
node.source = Some(source.clone());
}
assert!(function::compile(&oversized, 0).err().unwrap().contains("limit"));
}
@@ -200,22 +206,28 @@ fn processor_calls_recook_when_source_changes_and_round_trip_without_copying_sou
}
session.preview_size = Some(16);
session.cook();
assert_eq!(session.cooked.image(processors[0], 0).unwrap().get(0, 0, false), [0.2, 0.4, 0.6, 1.0]);
assert_eq!(session.cooked.value(processors[1], 0), Some(&Value::Float4([0.2, 0.4, 0.6, 1.0])));
execute(&mut session, "package.activate", &json!({"document": 0})).unwrap();
execute(&mut session, "graph.open", &json!({"graph": 1})).unwrap();
edit(&mut session, &json!({"kind": "graph"}), json!({"action": "value", "id": 1, "value": Value::Float4([0.7,0.3,0.1,1.0])}));
execute(&mut session, "file.save", &json!({})).unwrap();
execute(&mut session, "package.activate", &json!({"document": document})).unwrap();
session.cook();
assert_eq!(session.cooked.image(processors[0], 0).unwrap().get(0, 0, false), [0.7, 0.3, 0.1, 1.0]);
assert_eq!(session.cooked.value(processors[1], 0), Some(&Value::Float4([0.7, 0.3, 0.1, 1.0])));
execute(&mut session, "file.saveAs", &json!({"path": target_path})).unwrap();
let saved = std::fs::read(&target_path).unwrap();
let xml = String::from_utf8(saved.clone()).unwrap();
assert!(xml.contains("source.sbs") && !xml.contains(source_path.to_str().unwrap()));
assert!(xml.contains("pkg:///Tint?dependency="));
assert!(!xml.contains("0.7 0.3 0.1"));
let mut reopened = Session::new(Package::new("Empty"));
execute(&mut reopened, "file.open", &json!({"path": target_path})).unwrap();
reopened.preview_size = Some(16);
@@ -223,11 +235,14 @@ fn processor_calls_recook_when_source_changes_and_round_trip_without_copying_sou
for result in reopened.cooked.nodes.values() {
assert!(result.error.is_none(), "{:?}", result.error);
}
std::fs::remove_file(&source_path).unwrap();
let mut missing = Session::new(Package::new("Empty"));
file_cmds::open_bytes(&mut missing, &saved, Some(target_path.to_string_lossy().into())).unwrap();
assert!(missing.graph().unwrap().functions().all(|function| function.nodes.iter().any(|node| node.instance.is_some())));
assert!(missing.graph().unwrap().functions().all(|function| !function::warnings(function).is_empty()));
std::fs::remove_dir_all(directory).unwrap();
}
@@ -280,6 +295,7 @@ fn calls_evaluate_arguments_once_and_share_the_variables_they_set() {
let total = caller.add("add", &[call, get], None);
caller.output = Some(caller.add("sequence", &[init, total], None));
function::resolve_instances(&mut caller, &package, &BTreeMap::new());
// The Get node ran once, before the argument's Set: 1; the call doubles its argument 2
assert_eq!(evaluate(&caller), Value::Float(5.0));
@@ -287,11 +303,13 @@ fn calls_evaluate_arguments_once_and_share_the_variables_they_set() {
let after = caller.add("get_float1", &[], Some(Value::Text("private".into())));
let read_after = caller.add("sequence", &[call, after], None);
caller.output = Some(caller.add("sequence", &[init, read_after], None));
assert_eq!(evaluate(&caller), Value::Float(99.0));
// A separate Get after the call sees the assignment while the call result stays cached.
// A separate Get after the call sees the assignment while the call result stays cached
let total = caller.add("add", &[call, after], None);
caller.output = Some(caller.add("sequence", &[init, total], None));
assert_eq!(evaluate(&caller), Value::Float(103.0));
}
@@ -58,6 +58,7 @@ fn every_standard_function_has_a_resolved_executable_body() {
assert_eq!(entries.len(), 171);
assert!(missing.is_empty(), "missing bundled functions: {missing:?}");
for entry in entries {
session.graph_mut().unwrap().function = Some(FunctionGraph::default());
session.reset_history();
@@ -66,6 +67,7 @@ fn every_standard_function_has_a_resolved_executable_body() {
let function = session.graph().unwrap().function.as_ref().unwrap();
assert!(matches!(function.nodes[0].source, Some(Ok(_))), "{}", entry["graph"]);
function::compile(function, 0).unwrap_or_else(|error| panic!("{}: {error}", entry["graph"]));
}
}
@@ -193,6 +195,7 @@ fn supplied_functions_evaluate_before_and_after_save() {
for node in function.nodes.iter().filter(|node| node.instance.is_some()) {
assert!(matches!(node.source, Some(Ok(_))), "{}: unresolved instance {}", graph.sbs_path(), node.id);
}
let source = function::compile(function, 0).unwrap();
let saved = after.pkg.graph_by_name(&graph.sbs_path()).unwrap().1;
let saved = function::compile(saved.function.as_ref().unwrap(), 0).unwrap();
@@ -225,6 +228,7 @@ fn supplied_functions_evaluate_before_and_after_save() {
(parameter.id.clone(), value)
})
.collect();
// This hidden helper requires a render region supplied by its caller
if graph.name == "compute_total_number" {
variables.insert("render_region".into(), Value::Float4([-0.5, 0.5, -0.5, 0.5]));
@@ -235,6 +239,7 @@ fn supplied_functions_evaluate_before_and_after_save() {
Ok(expected) => {
assert_eq!(saved.evaluate(&context, &mut EvaluationState::default()).unwrap(), expected, "{}", graph.sbs_path());
assert_eq!(converted.evaluate(&context, &mut EvaluationState::default()).unwrap(), expected, "converted {}", graph.sbs_path());
defaults += usize::from(profile == 0);
errors.clear();
break;
@@ -242,6 +247,7 @@ fn supplied_functions_evaluate_before_and_after_save() {
Err(error) => errors.push(error),
}
}
if !errors.is_empty() {
failures.push(format!("{}: {}", graph.sbs_path(), errors.join("; ")));
}
@@ -252,5 +258,6 @@ fn supplied_functions_evaluate_before_and_after_save() {
before.pkg.graphs.len() - failures.len(),
before.pkg.graphs.len()
);
assert!(failures.is_empty(), "{}", failures.join("\n"));
}
@@ -55,6 +55,7 @@ fn pattern_luminosity_accepts_scalar_and_float4_but_rejects_other_types() {
let Some(Value::PatternGraph(pattern)) = session.graph_mut().unwrap().node_mut(owner).unwrap().params.get_mut("pattern") else {
panic!("expected pattern")
};
pattern.nodes[0].parameters.clear();
pattern.nodes[0].parameters.insert("opacity".into(), Value::Function(Box::default()));
for color in [false, true] {
@@ -64,12 +65,15 @@ fn pattern_luminosity_accepts_scalar_and_float4_but_rejects_other_types() {
let output = function.output.unwrap();
execute(&mut session, "graph.function.set", &json!({"target": target, "function": function})).unwrap();
execute(&mut session, "graph.function.edit", &json!({"target": target, "edit": {"action": "output", "id": output}})).unwrap();
assert!(materialcraft_engine::function_inspection::warnings(session.graph().unwrap()).is_empty());
}
for value in [Value::Float2([0.5; 2]), Value::Int(1)] {
let function = FunctionGraph::constant(value);
let output = function.output.unwrap();
execute(&mut session, "graph.function.set", &json!({"target": target, "function": function})).unwrap();
assert!(execute(&mut session, "graph.function.edit", &json!({"target": target, "edit": {"action": "output", "id": output}})).is_err());
}
}
@@ -134,6 +134,7 @@ fn fx_map_starts_with_a_quadrant_and_grows_image_inputs_as_connected() {
execute(&mut session, "graph.node.param", &json!({"id": source.0, "param": "color_mode", "value": false})).unwrap();
let owner = add(&mut session, "fx_map");
let Value::PatternGraph(pattern) = &session.graph().unwrap().node(owner).unwrap().params["pattern"] else { panic!("missing default pattern") };
assert_eq!(pattern.nodes.len(), 1);
assert_eq!(pattern.nodes[0].kind, PatternNodeKind::Quadrant);
assert_eq!(pattern.nodes[0].position, [0.0, 0.0]);
@@ -141,17 +142,22 @@ fn fx_map_starts_with_a_quadrant_and_grows_image_inputs_as_connected() {
assert_eq!(session.input_ports(owner), vec!["background", "pattern0"]);
let fresh = read_sbs(&write_sbs(&session.pkg).unwrap(), "fresh_pattern.sbs", &|_| None).unwrap();
assert!(fresh.warnings.is_empty(), "{:?}", fresh.warnings);
let loaded_pattern = fresh.pkg.graphs[0].nodes.iter().find(|node| node.def == "fx_map").unwrap();
let Value::PatternGraph(loaded_pattern) = &loaded_pattern.params["pattern"] else { panic!("missing saved root") };
assert_eq!(loaded_pattern.nodes.len(), 1);
assert_eq!(loaded_pattern.nodes[0].kind, PatternNodeKind::Quadrant);
assert_eq!(loaded_pattern.root, Some(loaded_pattern.nodes[0].id));
for index in 0..6 {
execute(&mut session, "graph.connect", &json!({"from": source.0, "to": owner.0, "toPort": format!("pattern{index}")})).unwrap();
assert_eq!(session.input_ports(owner).last(), Some(&format!("pattern{}", index + 1)));
}
let Value::PatternGraph(pattern) = &session.graph().unwrap().node(owner).unwrap().params["pattern"] else { panic!("missing pattern") };
let mut pattern = (**pattern).clone();
@@ -159,25 +165,36 @@ fn fx_map_starts_with_a_quadrant_and_grows_image_inputs_as_connected() {
pattern.nodes[0].parameters.insert("imageindex".into(), Value::Int(6));
execute(&mut session, "graph.pattern.set", &json!({"id": owner.0, "pattern": pattern})).unwrap();
session.cook();
assert!(session.cooked.nodes[&owner].error.is_none(), "{:?}", session.cooked.nodes[&owner].error);
assert!((session.cooked.image(owner, 0).unwrap().mean()[0] - 0.5).abs() < 1e-5);
let bytes = write_sbs(&session.pkg).unwrap();
assert!(std::str::from_utf8(&bytes).unwrap().contains("<identifier v=\"inputpattern:5\"/>"));
let read = read_sbs(&bytes, "image_inputs.sbs", &|_| None).unwrap();
assert!(read.warnings.is_empty(), "{:?}", read.warnings);
let mut loaded = Session::new(read.pkg);
loaded.preview_size = Some(16);
let loaded_owner = loaded.graph().unwrap().nodes.iter().find(|node| node.def == "fx_map").unwrap().id;
assert_eq!(loaded.input_ports(loaded_owner), session.input_ports(owner));
loaded.cook();
assert!((loaded.cooked.image(loaded_owner, 0).unwrap().mean()[0] - 0.5).abs() < 1e-5);
execute(&mut session, "graph.disconnect", &json!({"to": owner.0, "toPort": "pattern5"})).unwrap();
assert_eq!(session.input_ports(owner).last().map(String::as_str), Some("pattern5"));
assert!(session.undo().is_some());
assert_eq!(session.input_ports(owner).last().map(String::as_str), Some("pattern6"));
let before = session.pkg.clone();
assert!(execute(&mut session, "graph.connect", &json!({"from": source.0, "to": owner.0, "toPort": "pattern999999999"})).is_err());
assert_eq!(session.pkg, before);
}
@@ -220,6 +237,7 @@ fn processors_sample_dynamic_image_inputs_after_sbs_round_trip() {
let processor = add(&mut session, definition);
for index in 0..6 {
execute(&mut session, "graph.connect", &json!({"from": source.0, "to": processor.0, "toPort": format!("input{index}")})).unwrap();
assert_eq!(session.input_ports(processor).last(), Some(&format!("input{}", index + 1)));
}
@@ -228,15 +246,21 @@ fn processors_sample_dynamic_image_inputs_after_sbs_round_trip() {
function.output = Some(function.add("samplelum", &[position], Some(Value::Int2([5, 1]))));
execute(&mut session, "graph.node.param", &json!({"id": processor.0, "param": "function", "value": Value::Function(Box::new(function))})).unwrap();
let bytes = write_sbs(&session.pkg).unwrap();
assert!(std::str::from_utf8(&bytes).unwrap().contains("<identifier v=\"input:5\"/>"));
let read = read_sbs(&bytes, "processor_images.sbs", &|_| None).unwrap();
assert!(read.warnings.is_empty(), "{:?}", read.warnings);
let mut loaded = Session::new(read.pkg);
loaded.preview_size = Some(16);
loaded.cook();
let processor = loaded.graph().unwrap().nodes.iter().find(|node| node.def == definition).unwrap().id;
assert_eq!(loaded.input_ports(processor).last().map(String::as_str), Some("input6"));
assert!(loaded.cooked.nodes[&processor].error.is_none(), "{:?}", loaded.cooked.nodes[&processor].error);
if definition == "value_processor" {
assert_eq!(loaded.cooked.value(processor, 0), Some(&Value::Float(0.5)));
} else {
@@ -42,23 +42,34 @@ fn source_edits_update_instances_and_dirty_dependents_before_saving() {
let instance = NodeId(execute(&mut session, "graph.instance.add", &json!({"document": 0, "graph": "Source"})).unwrap()["id"].as_u64().unwrap());
session.preview_size = Some(16);
session.cook();
assert_eq!(session.output_ports(instance), ["color"]);
assert!(session.cooked.nodes[&instance].error.is_none(), "{:?}", session.cooked.nodes[&instance].error);
assert_eq!(session.cooked.image(instance, 0).unwrap().get(0, 0, false), [0.25, 0.5, 0.75, 1.0]);
let history = session.history();
session.modified = false;
execute(&mut session, "package.activate", &json!({"document": 0})).unwrap();
execute(&mut session, "graph.select", &json!({"ids": [value.0]})).unwrap();
execute(&mut session, "graph.node.param", &json!({"id": value.0, "param": "color", "value": [0.8, 0.2, 0.1, 1.0]})).unwrap();
assert!(session.open_packages().iter().find(|entry| entry.0 == target_id).unwrap().3);
execute(&mut session, "package.activate", &json!({"document": target_id})).unwrap();
assert_eq!(session.history(), history);
session.cook();
assert_eq!(session.cooked.image(instance, 0).unwrap().get(0, 0, false), [0.8, 0.2, 0.1, 1.0]);
execute(&mut session, "edit.undo", &json!({})).unwrap();
assert!(session.graph().unwrap().nodes.is_empty());
execute(&mut session, "edit.redo", &json!({})).unwrap();
session.cook();
assert_eq!(session.cooked.image(instance, 0).unwrap().get(0, 0, false), [0.8, 0.2, 0.1, 1.0]);
}
@@ -93,18 +104,25 @@ fn failed_edit_groups_do_not_publish_source_changes() {
execute(session, "graph.node.param", &json!({"id": value.0, "param": "color", "value": [1.0, 0.0, 0.0, 1.0]})).unwrap();
execute(session, "graph.node.param", &json!({"id": 99999, "param": "color", "value": [0.0, 0.0, 0.0, 0.0]}))
});
assert!(result.is_err());
assert!(!session.open_packages().iter().find(|entry| entry.0 == target_id).unwrap().3);
execute(&mut session, "package.activate", &json!({"document": target_id})).unwrap();
session.preview_size = Some(16);
session.cook();
assert_eq!(session.cooked.image(instance, 0).unwrap().get(0, 0, false), [0.25, 0.5, 0.75, 1.0]);
execute(&mut session, "package.activate", &json!({"document": 0})).unwrap();
session.begin_group("Complete source edit on package switch");
execute(&mut session, "graph.node.param", &json!({"id": value.0, "param": "color", "value": [1.0, 0.0, 0.0, 1.0]})).unwrap();
execute(&mut session, "package.activate", &json!({"document": target_id})).unwrap();
assert!(session.modified);
session.cook();
assert_eq!(session.cooked.image(instance, 0).unwrap().get(0, 0, false), [1.0, 0.0, 0.0, 1.0]);
}
@@ -143,13 +161,20 @@ fn updates_propagate_through_three_packages_and_cycles_are_rejected() {
session.modified = false;
session.preview_size = Some(16);
session.cook();
assert_eq!(session.cooked.image(instance, 0).unwrap().get(0, 0, false), [0.25, 0.5, 0.75, 1.0]);
execute(&mut session, "package.activate", &json!({"document": 0})).unwrap();
assert!(execute(&mut session, "graph.instance.add", &json!({"document": outer, "graph": "Target"})).is_err());
execute(&mut session, "graph.node.param", &json!({"id": value.0, "param": "color", "value": [0.1, 0.2, 0.3, 1.0]})).unwrap();
assert!(session.open_packages().iter().filter(|entry| entry.0 != 0).all(|entry| entry.3));
execute(&mut session, "package.activate", &json!({"document": outer})).unwrap();
session.cook();
assert_eq!(session.cooked.image(instance, 0).unwrap().get(0, 0, false), [0.1, 0.2, 0.3, 1.0]);
}
@@ -168,13 +193,19 @@ fn closing_a_modified_source_discards_its_unsaved_changes_in_dependents() {
let instance = NodeId(execute(&mut session, "graph.instance.add", &json!({"document": 0, "graph": "Source"})).unwrap()["id"].as_u64().unwrap());
execute(&mut session, "package.activate", &json!({"document": 0})).unwrap();
execute(&mut session, "graph.node.param", &json!({"id": value.0, "param": "color", "value": [1.0, 0.0, 0.0, 1.0]})).unwrap();
assert!(execute(&mut session, "package.close", &json!({})).is_err());
execute(&mut session, "package.close", &json!({"discard": true})).unwrap();
assert_eq!(session.document_id, target_id);
session.preview_size = Some(16);
session.cook();
let color = session.cooked.image(instance, 0).unwrap().get(0, 0, false);
assert!(color.into_iter().zip([0.25, 0.5, 0.75, 1.0]).all(|(actual, expected)| (actual - expected).abs() < 1e-6));
std::fs::remove_dir_all(directory).unwrap();
}
@@ -199,10 +230,12 @@ fn named_value_inputs_and_outputs_keep_their_type_across_files() {
session.preview_size = Some(16);
session.cook();
let cooked = &session.cooked.nodes[&instance];
assert!(cooked.error.is_none(), "{:?}", cooked.error);
assert_eq!(cooked.outputs[0].as_value(), Some(&Value::Float(0.75)));
assert_eq!(session.output_ports(instance), ["result"]);
assert!(!materialcraft_engine::MaterialSource { graph: session.graph, node: instance, port: "result".into() }.is_image(&session.pkg));
let mut local = session.open_packages().into_iter().find(|entry| entry.0 == 0).unwrap().1.clone();
let mut caller = session.graph().unwrap().clone();
caller.node_mut(instance).unwrap().instance.as_mut().unwrap().package.clear();
@@ -215,6 +248,7 @@ fn named_value_inputs_and_outputs_keep_their_type_across_files() {
reopened.cook();
let instance = reopened.graph().unwrap().nodes.iter().find(|node| node.def == "graph_instance").unwrap().id;
let cooked = &reopened.cooked.nodes[&instance];
assert!(cooked.error.is_none(), "{:?}", cooked.error);
assert_eq!(cooked.outputs[0].as_value(), Some(&Value::Float(0.75)));
}
@@ -237,9 +271,12 @@ fn removing_a_source_output_does_not_erase_connected_instance_wires_on_save() {
execute(&mut session, "package.activate", &json!({"document": target_id})).unwrap();
let bytes = file_cmds::save_bytes(&session).unwrap();
let read = formats::sbs::read_sbs(&bytes, "target.sbs", &|_| None).unwrap();
assert_eq!(read.pkg.graphs[0].connections.len(), 1);
session.preview_size = Some(16);
session.cook();
assert!(session.cooked.nodes[&instance].error.is_some());
}
@@ -285,28 +322,38 @@ fn saved_dependencies_reopen_without_source_files_in_explorer_and_missing_source
execute(&mut session, "graph.connect", &json!({"from": instance, "fromPort": "color", "to": output})).unwrap();
execute(&mut session, "file.save", &json!({"path": target_path})).unwrap();
let saved = std::fs::read_to_string(&target_path).unwrap();
assert!(saved.contains("filename v=\"source.sbs\""));
assert!(!saved.contains(&directory.to_string_lossy().to_string()));
let mut reopened = Session::new(Package::new("Initial"));
file_cmds::open_file(&mut reopened, target_path.to_str().unwrap()).unwrap();
assert!(reopened.load_warnings.is_empty(), "{:?}", reopened.load_warnings);
assert_eq!(reopened.open_packages().len(), 2);
execute(&mut reopened, "graph.open", &json!({"graph": "Target"})).unwrap();
reopened.preview_size = Some(16);
reopened.cook();
let graph = reopened.graph().unwrap();
let output = graph.nodes.iter().find(|node| node.def == "output").unwrap().id;
let color = reopened.cooked.image(output, 0).unwrap().get(0, 0, false);
assert!(color.into_iter().zip([0.25, 0.5, 0.75, 1.0]).all(|(actual, expected)| (actual - expected).abs() < 1e-6));
std::fs::remove_file(&source_path).unwrap();
let mut missing = Session::new(Package::new("Initial"));
file_cmds::open_file(&mut missing, target_path.to_str().unwrap()).unwrap();
execute(&mut missing, "graph.open", &json!({"graph": "Target"})).unwrap();
assert_eq!(missing.graph().unwrap().connections.len(), 1);
let resaved = file_cmds::save_bytes(&missing).unwrap();
let resaved = formats::sbs::read_sbs(&resaved, "target.sbs", &|_| None).unwrap();
assert_eq!(resaved.pkg.graph_by_name("Target").unwrap().1.connections.len(), 1);
assert!(missing.path.is_some());
std::fs::remove_dir_all(directory).unwrap();
}
@@ -352,6 +399,7 @@ fn instance_defaults_follow_current_source_inputs_before_and_after_save() {
assert_eq!(cooked.nodes[&instance].error, None);
assert_eq!(cooked.nodes[&instance].parameters["tint"], Value::Color(expected));
assert_instance_color(&mut session, instance, expected);
}
@@ -360,7 +408,9 @@ fn instance_defaults_follow_current_source_inputs_before_and_after_save() {
std::fs::write(&target_path, file_cmds::save_bytes(&session).unwrap()).unwrap();
let mut reopened = Session::new(Package::new("Initial"));
file_cmds::open_file(&mut reopened, target_path.to_str().unwrap()).unwrap();
assert!(reopened.load_warnings.is_empty(), "{:?}", reopened.load_warnings);
reopened.graph = target_graph;
reopened.preview_size = Some(16);
let instance = reopened.graph().unwrap().nodes.iter().find(|node| node.def == "graph_instance").unwrap().id;
+12 -2
View File
@@ -13,17 +13,23 @@ fn bundled_voronoi_bitmaps_load_the_shared_resource_and_survive_portable_saving(
let path = "sbs://noise_voronoi.sbs";
let bytes = sbs::library::read(path).unwrap();
let read = sbs::read_sbs(&bytes, path, &file_cmds::linked_files(path)).unwrap();
assert!(!read.warnings.iter().any(|warning| warning.contains("resource")), "{:?}", read.warnings);
let bitmaps: Vec<_> = read.pkg.graphs.iter().flat_map(|graph| &graph.nodes).filter(|node| node.def == "bitmap").cloned().collect();
assert_eq!(bitmaps.len(), 4);
let expected = sbs::library::read("sbs://resources/perm.png").unwrap();
for bitmap in &bitmaps {
let Value::Text(name) = &bitmap.params["resource"] else { panic!("missing resource") };
let resource = read.pkg.resource(name).unwrap();
assert_eq!(resource.bytes.as_slice(), expected);
assert_eq!(resource.source, "resources/perm.png");
}
// Exercise the repaired bitmaps independently of remaining Voronoi function limitations.
// Exercise the repaired bitmaps independently of remaining Voronoi function limitations
let mut package = Package::new("Resource regression");
package.resources = read.pkg.resources;
let mut graph = Graph::new("Bitmaps");
@@ -37,13 +43,17 @@ fn bundled_voronoi_bitmaps_load_the_shared_resource_and_survive_portable_saving(
let cooked = session.cook().clone();
for node in &session.pkg.graphs[0].nodes {
assert!(cooked.nodes[&node.id].error.is_none());
let image = cooked.image(node.id, 0).unwrap();
let low = image.data.iter().copied().fold(f32::INFINITY, f32::min);
let high = image.data.iter().copied().fold(f32::NEG_INFINITY, f32::max);
assert!(low.is_finite() && high.is_finite() && high > low);
}
let saved = file_cmds::save_bytes(&session).unwrap();
let reopened = sbs::read_sbs(&saved, "relocated.sbs", &|_| None).unwrap();
assert!(reopened.warnings.is_empty(), "{:?}", reopened.warnings);
assert_eq!(reopened.pkg.resources[0].bytes.as_slice(), expected);
}
@@ -121,7 +131,7 @@ fn verify_library_resources(root: &std::path::Path) {
assert!(read("../../private.svg").is_none());
assert!(read("resources/absent.svg").is_none());
// Native filesystem opens use the same shared root as alias opens, including nested packages.
// Native filesystem opens use the same shared root as alias opens, including nested packages
let native = root.join("assets/sbs/nested/source.sbs");
let native_read = file_cmds::linked_files(native.to_str().unwrap());
assert_eq!(native_read("../resources/tile.svg").unwrap(), TILE);
@@ -192,12 +192,17 @@ fn multiple_graph_channels_and_multi_output_instances_use_the_correct_ports() {
execute(&mut session, "graph.open", &json!({"graph": 1})).unwrap();
let instance = instance(&mut session, "Colors");
execute(&mut session, "view.3d.assign", &json!({"node": instance, "port": "blue", "channel": "baseColor"})).unwrap();
assert_eq!(color(&mut session, "baseColor"), [0.1, 0.1, 0.8, 1.0]);
let key = session.material_key();
execute(&mut session, "view.3d.assign", &json!({"node": instance, "port": 0, "channel": "baseColor"})).unwrap();
assert_eq!(color(&mut session, "baseColor"), [0.8, 0.1, 0.1, 1.0]);
assert_ne!(session.material_key(), key);
execute(&mut session, "view.3d.assign", &json!({"graph": "Colors", "node": 1, "channel": "emissive"})).unwrap();
assert_eq!(color(&mut session, "emissive"), [0.8, 0.1, 0.1, 1.0]);
assert_eq!(session.graph, 1);
}
+16 -1
View File
@@ -12,15 +12,20 @@ fn bundled_cloud_and_white_noise_parameter_scopes_cook() {
for file in ["clouds_3.sbs", "noise_clouds_3_v2.sbs", "white_noise_v2.sbs"] {
materialcraft_engine::file_cmds::open_file(&mut session, &format!("sbs://{file}")).unwrap();
let warnings = materialcraft_engine::function_inspection::warnings(session.graph().unwrap());
assert!(warnings.is_empty(), "{file}: {warnings:?}");
let nodes = session.graph().unwrap().nodes.clone();
let cooked = session.cook();
let failures: Vec<_> = cooked.nodes.iter().filter_map(|(id, result)| result.error.as_ref().map(|error| (*id, error))).collect();
assert!(failures.is_empty(), "{file}: {failures:?}");
for node in nodes.iter().filter(|node| matches!(node.def.as_str(), "output" | "fx_map")) {
let image = cooked.image(node.id, 0).unwrap();
let low = image.data.iter().copied().fold(f32::INFINITY, f32::min);
let high = image.data.iter().copied().fold(f32::NEG_INFINITY, f32::max);
assert!(high - low > 0.05, "{file} lost variation: {low}..{high}");
}
}
@@ -35,25 +40,35 @@ fn bundled_brick_material_cooks_without_missing_functions_or_budget_errors() {
session.preview_size = Some(64);
materialcraft_engine::file_cmds::open_file(&mut session, "sbs://materials/pbr/bricks_001.sbs").unwrap();
let outputs: Vec<_> = session.graph().unwrap().nodes.iter().filter(|node| node.def == "output").map(|node| node.id).collect();
assert!(!outputs.is_empty());
let cooked = session.cook();
let failures: Vec<_> = cooked.nodes.iter().filter_map(|(id, result)| result.error.as_ref().map(|error| (*id, error))).collect();
assert!(failures.is_empty(), "{failures:?}");
for output in outputs {
let image = cooked.image(output, 0).unwrap();
assert!(image.data.iter().all(|value| value.is_finite()));
}
// Finite output alone missed flat noise, identical tile values and a half-turn hue error.
// Finite output alone missed flat noise, identical tile values and a half-turn hue error
for id in [24, 34, 48, 91] {
let image = cooked.image(materialcraft_engine::core::NodeId(id), 0).unwrap();
let low = image.data.iter().copied().fold(f32::INFINITY, f32::min);
let high = image.data.iter().copied().fold(f32::NEG_INFINITY, f32::max);
assert!(high - low > 0.05, "node {id} lost its variation: {low}..{high}");
if id == 24 {
assert!((0.15..0.85).contains(&image.mean()[0]), "fractal noise has lost its neutral range");
}
}
let color = cooked.image(materialcraft_engine::core::NodeId(99), 0).unwrap().mean();
assert!(color[0] > color[2], "the default brick color shifted to blue: {color:?}");
}
@@ -53,6 +53,7 @@ fn optimization_baseline() {
resource: &|_| None,
vars: &variables,
};
let ms = median_ms(|| {
black_box((definition.cook)(&context).unwrap());
});
@@ -81,11 +82,15 @@ fn optimization_baseline() {
let options = Options::new(&|_| None);
let cache = Mutex::new(Cache::default());
let initial = cook(&package, 0, &options, &cache);
assert_eq!(initial.nodes.len(), count);
assert!(initial.nodes.values().all(|result| result.error.is_none()));
let ms = median_ms(|| {
let cooked = cook(&package, 0, &options, &cache);
assert!(cooked.nodes.values().all(|result| result.cached));
black_box(cooked);
});
println!("warm_cook,{count},{branching},{ms:.3}");
@@ -274,6 +274,7 @@ fn instance_parameter_functions_work_without_stored_literals_and_use_instance_si
let loaded = formats::sbs::read_sbs(&bytes, "instance.sbs", &|_| None).unwrap();
let mut reopened = Session::new(loaded.pkg);
let owner = reopened.graph().unwrap().nodes[0].id;
assert!((reopened.cook().image(owner, 0).unwrap().get_gray(0, 0, false) - 0.75).abs() < 1e-6);
}
@@ -291,22 +292,30 @@ fn color_boolean_enum_and_text_parameters_use_typed_function_outputs() {
command(&mut session, "graph.node.function", json!({"id": owner, "param": parameter, "function": FunctionGraph::constant(value.clone())}));
let result = &session.cook().nodes[&owner];
assert_eq!(result.error, None, "{parameter}");
let expected = materialcraft_engine::nodes::find(definition).unwrap().param(parameter).unwrap().default.coerce_json(&json!(value)).unwrap();
assert_eq!(result.parameters[parameter], expected);
}
let owner = node(&mut session, "uniform");
command(&mut session, "graph.node.function", json!({"id": owner, "param": "color", "function": FunctionGraph::constant(Value::Float(0.5))}));
// A gray level fills the color channels, as the library's uniform colors rely on
let result = &session.cook().nodes[&owner];
assert_eq!(result.error, None);
assert_eq!(result.parameters["color"].as_vec4(), Some([0.5, 0.5, 0.5, 1.0]));
let owner = node(&mut session, "uniform");
command(&mut session, "graph.node.function", json!({"id": owner, "param": "color", "function": FunctionGraph::constant(Value::Int(1))}));
assert!(session.cook().nodes[&owner].error.as_ref().unwrap().contains("Float4"));
let bitmap = node(&mut session, "bitmap");
assert!(execute(&mut session, "graph.node.function", &json!({"id": bitmap, "param": "resource"})).is_err());
}
@@ -320,22 +329,32 @@ fn native_channel_parameters_accept_scalar_functions_without_changing_their_bodi
let function = FunctionGraph::constant(Value::Float(0.25));
command(&mut session, "graph.node.function", json!({"id": owner, "param": parameter, "function": function}));
let target = materialcraft_engine::FunctionGraphTarget::Dynamic { node: owner, parameter: (*parameter).into() };
assert!(materialcraft_engine::function_inspection::warnings_for(session.graph().unwrap(), &target, &function).is_empty());
let result = &session.cook().nodes[&owner];
assert_eq!(result.error, None, "{definition}.{parameter}");
let expected = if definition == "fx_map" { Value::Color([0.25, 0.25, 0.25, 1.0]) } else { Value::Float4([0.25; 4]) };
assert_eq!(result.parameters[*parameter], expected);
let bytes = formats::sbs::write_sbs(&session.pkg).unwrap();
let loaded = formats::sbs::read_sbs(&bytes, "scalar-channel.sbs", &|_| None).unwrap();
assert!(loaded.warnings.is_empty(), "{:?}", loaded.warnings);
let reopened_node = loaded.pkg.graphs[0].nodes.last().unwrap();
let reopened_id = reopened_node.id;
let saved_function = &reopened_node.functions[*parameter];
assert_eq!(saved_function.nodes.len(), 1);
assert_eq!(saved_function.nodes[0].value, Some(Value::Float(0.25)));
let mut reopened = Session::new(loaded.pkg);
reopened.preview_size = Some(16);
assert_eq!(reopened.cook().nodes[&reopened_id].parameters[*parameter], expected);
}
}
+7
View File
@@ -63,9 +63,13 @@ fn dynamic_roughness_survives_native_saving_and_invalidates_the_cook_cache() {
for _ in 0..2 {
let saved = write_sbs(&package).unwrap();
assert!(std::str::from_utf8(&saved).unwrap().contains("<name v=\"hurst\"/>"));
let loaded = read_sbs(&saved, "roughness.sbs", &|_| None).unwrap();
assert!(loaded.warnings.is_empty(), "{:?}", loaded.warnings);
package = loaded.pkg;
let owner = package.graphs[0].nodes[0].id;
let cache = std::sync::Mutex::new(materialcraft_engine::eval::Cache::default());
@@ -73,10 +77,13 @@ fn dynamic_roughness_survives_native_saving_and_invalidates_the_cook_cache() {
for (time, expected) in [(0.0, 1.0), (0.5, 1.0 / 3.0), (1.0, 0.5), (2.0, 2.0 / 3.0)] {
options.environment.time_seconds = time;
let result = materialcraft_engine::eval::cook(&package, 0, &options, &cache);
assert_eq!(result.nodes[&owner].error, None);
assert!(!result.nodes[&owner].cached);
assert!((result.image(owner, 0).unwrap().get(0, 0, false)[0] - expected).abs() < 1e-6);
let cached = materialcraft_engine::eval::cook(&package, 0, &options, &cache);
assert!(cached.nodes[&owner].cached);
assert_eq!(cached.image(owner, 0).unwrap().data, result.image(owner, 0).unwrap().data);
}
+14 -1
View File
@@ -4,7 +4,7 @@ use serde_json::json;
#[test]
fn imported_expression_cooks_follows_inputs_and_survives_undo_and_save() {
// Original structural XML: in_low = amount - 0.25, with no embedded native package.
// Original structural XML: in_low = amount - 0.25, with no embedded native package
let xml = r#"<package><content><graph><identifier v="Math"/>
<paraminputs><paraminput><identifier v="amount"/><uid v="90"/><type v="256"/>
<defaultValue><constantValueFloat1 v="0.75"/></defaultValue></paraminput></paraminputs>
@@ -22,7 +22,9 @@ fn imported_expression_cooks_follows_inputs_and_survives_undo_and_save() {
<paramNode><uid v="4"/><function v="swizzle4"/><connections><connection><identifier v="vector"/><connRef v="3"/></connection></connections></paramNode></paramNodes></dynamicValue></paramValue></parameter></parameters>
</compFilter></compImplementation></compNode></compNodes></graph></content></package>"#;
let imported = sbs::read_sbs(xml.as_bytes(), "math.sbs", &|_| None).unwrap();
assert!(imported.warnings.is_empty(), "{:?}", imported.warnings);
let mut s = new_session();
s.preview_size = Some(64);
s.pkg = imported.pkg;
@@ -33,19 +35,30 @@ fn imported_expression_cooks_follows_inputs_and_survives_undo_and_save() {
let mean = |s: &mut materialcraft_engine::Session| {
let levels = s.graph().unwrap().nodes.iter().find(|node| node.def == "levels").unwrap().id;
s.cook();
assert!(s.cooked.nodes[&levels].error.is_none());
s.cooked.image(levels, 0).unwrap().mean()[0]
};
assert!(mean(&mut s).abs() < 1e-5);
execute(&mut s, "graph.exposed.set", &json!({"id": "amount", "value": 0.5})).unwrap();
assert!((mean(&mut s) - 1.0 / 3.0).abs() < 1e-5);
execute(&mut s, "edit.undo", &json!({})).unwrap();
assert!(mean(&mut s).abs() < 1e-5);
execute(&mut s, "edit.redo", &json!({})).unwrap();
assert!((mean(&mut s) - 1.0 / 3.0).abs() < 1e-5);
let saved = sbs::write_sbs(&s.pkg).unwrap();
s.pkg = sbs::read_sbs(&saved, "saved.sbs", &|_| None).unwrap().pkg;
s.cache.lock().unwrap().clear();
assert!((mean(&mut s) - 1.0 / 3.0).abs() < 1e-5);
assert!(s.graph().unwrap().nodes.iter().find(|node| node.def == "levels").unwrap().functions.contains_key("in_low"));
}
@@ -139,17 +139,21 @@ fn pixel_ratio_and_tiling_inherit_and_preserve_native_relative_modes() {
execute(&mut session, "graph.node.base", &json!({"pixelRatio": 0, "tiling": false, "pixelRatioMode": mode, "tilingMode": mode})).unwrap();
let bytes = write_sbs(&session.pkg).unwrap();
let read = read_sbs(&bytes, "inheritance.sbs", &|_| None).unwrap();
assert!(read.warnings.is_empty(), "{:?}", read.warnings);
// SBS represents Inherit as a format value plus a relative mode
let mut expected_base = session.pkg.graphs[0].node(pixel).unwrap().base.clone();
expected_base.format = OutputFormat::U16;
expected_base.format_mode = ParameterInheritance::RelativeToInput;
assert_eq!(read.pkg.graphs[0].node(pixel).unwrap().base, expected_base);
let mut reopened = Session::new(read.pkg);
for session in [&mut session, &mut reopened] {
let cooked = session.cook();
assert_eq!(cooked.nodes[&pixel].error, None);
assert!((cooked.image(pixel, 0).unwrap().mean()[0] - expected).abs() < 1e-6, "{mode}");
}
@@ -180,13 +184,16 @@ fn pixel_processor_default_format_preserves_input_precision_on_sbs_round_trip()
package.graphs.push(graph);
let mut original = Session::new(package);
let before = original.cook();
assert_eq!(before.nodes[&pixel].error, None);
assert_eq!(before.nodes[&pixel].base.format, expected_format);
assert_eq!(before.image(pixel, 0).unwrap().get(0, 0, false)[0], expected_value);
let bytes = write_sbs(&original.pkg).unwrap();
let read = read_sbs(&bytes, "format-inheritance.sbs", &|_| None).unwrap();
assert!(read.warnings.is_empty(), "{:?}", read.warnings);
let mut reopened = Session::new(read.pkg);
let after = reopened.cook();
@@ -273,7 +280,9 @@ fn nested_function_instances_use_the_processing_callers_variable_scope() {
assert_eq!(cooked.nodes[&pixel].error, None);
assert!(cooked.nodes[&value].error.as_deref().unwrap().contains("undefined variable `$pos`"));
let image = cooked.image(pixel, 0).unwrap();
assert_eq!(&image.get(0, 0, false)[..2], &[0.03125; 2]);
assert_eq!(&image.get(15, 15, false)[..2], &[0.96875; 2]);
}
+3
View File
@@ -21,10 +21,13 @@ fn byte_only_sbs_import_resolves_library_resource_substitutes() {
let font_file = if fallback.is_ok() { "LiberationMono-Regular.ttf" } else { "cour.ttf" };
for (name, filename) in [("noise", "bluenoise_256_c.exr"), ("font", font_file)] {
let resource = session.pkg.resource(name).unwrap();
assert_eq!(resource.file, filename);
assert_eq!(resource.bytes.as_slice(), materialcraft_engine::formats::sbs::library::read(&format!("sbs://resources/{filename}")).unwrap_or_default());
}
let saved = String::from_utf8(file_cmds::save_bytes(&session).unwrap()).unwrap();
assert!(!saved.contains(".surface"));
assert_eq!(saved.contains("old/cour.ttf"), fallback.is_err());
}
+21 -15
View File
@@ -2,6 +2,9 @@ use crate::NodeOutput;
use std::collections::HashMap;
use std::sync::Arc;
const DEFAULT_LIMIT_BYTES: usize = 256 << 20;
const MAX_ENTRIES: usize = 8192;
/// LRU cache of node outputs, bounded by bytes
pub struct Cache {
entries: HashMap<u64, (Arc<Vec<NodeOutput>>, usize, u64)>,
@@ -17,8 +20,8 @@ mod tests;
impl Default for Cache {
fn default() -> Self {
// Active graph results are retained separately; history must not reserve a gigabyte.
Cache::new(256 << 20)
// Active graph results are retained separately; history must not reserve a gigabyte
Cache::new(DEFAULT_LIMIT_BYTES)
}
}
@@ -47,31 +50,34 @@ impl Cache {
pub(crate) fn get(&mut self, key: u64) -> Option<Arc<Vec<NodeOutput>>> {
self.tick = self.tick.saturating_add(1);
let t = self.tick;
self.entries.get_mut(&key).map(|e| {
e.2 = t;
e.0.clone()
let tick = self.tick;
self.entries.get_mut(&key).map(|entry| {
entry.2 = tick;
entry.0.clone()
})
}
pub(crate) fn put(&mut self, key: u64, v: Arc<Vec<NodeOutput>>) {
let size = v.iter().fold(0_usize, |total, output| total.saturating_add(output.memory_bytes()));
pub(crate) fn put(&mut self, key: u64, outputs: Arc<Vec<NodeOutput>>) {
let size = outputs.iter().fold(0_usize, |total, output| total.saturating_add(output.memory_bytes()));
if let Some(old) = self.entries.remove(&key) {
self.bytes = self.bytes.saturating_sub(old.1);
}
// An oversized result remains owned by its cook, never by this cache. Empty results
// are not worth retaining and otherwise permit unbounded zero-byte metadata.
// Oversized results remain owned by their cooks
// Empty results would permit unbounded zero-byte cache metadata
if size == 0 || size > self.cap {
return;
}
while self.bytes > self.cap - size || self.entries.len() >= 8192 {
let Some((&k, _)) = self.entries.iter().min_by_key(|(_, e)| e.2) else { break };
if let Some(e) = self.entries.remove(&k) {
self.bytes = self.bytes.saturating_sub(e.1);
while self.bytes > self.cap - size || self.entries.len() >= MAX_ENTRIES {
let Some((&oldest_key, _)) = self.entries.iter().min_by_key(|(_, entry)| entry.2) else { break };
if let Some(entry) = self.entries.remove(&oldest_key) {
self.bytes = self.bytes.saturating_sub(entry.1);
}
}
self.tick = self.tick.saturating_add(1);
self.entries.insert(key, (v, size, self.tick));
self.entries.insert(key, (outputs, size, self.tick));
self.bytes += size;
}
}
+12
View File
@@ -12,6 +12,7 @@ fn oversized_results_are_not_retained_or_allowed_to_flush_useful_entries() {
let large = output(8);
let weak = Arc::downgrade(&large);
cache.put(2, large);
assert!(weak.upgrade().is_none());
assert!(cache.get(1).is_some());
assert!(cache.get(2).is_none());
@@ -22,10 +23,13 @@ fn oversized_results_are_not_retained_or_allowed_to_flush_useful_entries() {
fn disabled_and_empty_caches_do_not_retain_outputs_or_metadata() {
let mut cache = Cache::new(0);
cache.put(1, output(4));
assert!(cache.is_empty());
for key in 0..10_000 {
cache.put(key, Arc::new(Vec::new()));
}
assert!(cache.is_empty());
}
@@ -36,17 +40,25 @@ fn repeated_edits_release_evicted_buffers_and_keep_recent_hits() {
let first_weak = Arc::downgrade(&first);
cache.put(0, first);
cache.put(1, output(4));
assert!(cache.get(0).is_some());
cache.put(2, output(4));
assert!(cache.get(1).is_none());
assert!(first_weak.upgrade().is_some());
for key in 3..1_000 {
cache.put(key, output(4));
assert!(cache.bytes() <= 128);
assert!(cache.len() <= 2);
}
assert!(first_weak.upgrade().is_none());
cache.clear();
assert_eq!(cache.bytes(), 0);
assert!(cache.is_empty());
}
+49 -40
View File
@@ -5,83 +5,92 @@ use std::collections::{BTreeMap, HashSet};
use std::hash::{DefaultHasher, Hash, Hasher};
use std::sync::{Arc, Mutex};
pub(crate) fn image_key(i: &Image, cache: &Mutex<Cache>) -> u64 {
let ptr = Arc::as_ptr(&i.data) as usize;
let shape = (i.w, i.h, i.kind.channels());
const MAX_INPUT_FINGERPRINTS: usize = 128;
pub(crate) fn image_key(image: &Image, cache: &Mutex<Cache>) -> u64 {
let buffer_id = Arc::as_ptr(&image.data) as usize;
let shape = (image.w, image.h, image.kind.channels());
if let Some(key) = cache
.lock()
.unwrap_or_else(std::sync::PoisonError::into_inner)
.input_hashes
.get(&ptr)
.and_then(|saved| saved.buffer.upgrade().filter(|data| Arc::ptr_eq(data, &i.data) && saved.shape == shape).map(|_| saved.key))
.get(&buffer_id)
.and_then(|saved| saved.buffer.upgrade().filter(|data| Arc::ptr_eq(data, &image.data) && saved.shape == shape).map(|_| saved.key))
{
return key ^ hash_of(&i.integer_sampling);
return key ^ hash_of(&image.integer_sampling);
}
let mut h = DefaultHasher::new();
(shape, i.data.len()).hash(&mut h);
for value in i.data.iter() {
value.to_bits().hash(&mut h);
let mut hasher = DefaultHasher::new();
(shape, image.data.len()).hash(&mut hasher);
for value in image.data.iter() {
value.to_bits().hash(&mut hasher);
}
let key = h.finish();
let key = hasher.finish();
let mut cache = cache.lock().unwrap_or_else(std::sync::PoisonError::into_inner);
if cache.input_hashes.len() >= 128 {
if cache.input_hashes.len() >= MAX_INPUT_FINGERPRINTS {
cache.input_hashes.retain(|_, saved| saved.buffer.strong_count() > 0);
if cache.input_hashes.len() >= 128 {
if cache.input_hashes.len() >= MAX_INPUT_FINGERPRINTS {
cache.input_hashes.clear();
}
}
cache.input_hashes.insert(ptr, crate::input_fingerprint::InputFingerprint { buffer: Arc::downgrade(&i.data), shape, key });
key ^ hash_of(&i.integer_sampling)
cache.input_hashes.insert(buffer_id, crate::input_fingerprint::InputFingerprint { buffer: Arc::downgrade(&image.data), shape, key });
key ^ hash_of(&image.integer_sampling)
}
pub(crate) fn hash_of(v: &impl Hash) -> u64 {
let mut h = DefaultHasher::new();
v.hash(&mut h);
h.finish()
pub(crate) fn hash_of(value: &impl Hash) -> u64 {
let mut hasher = DefaultHasher::new();
value.hash(&mut hasher);
hasher.finish()
}
#[cfg(test)]
pub(crate) fn graph_tree_hash(pkg: &Package, gi: usize) -> u64 {
graph_tree_hash_with_registry(pkg, pkg, gi)
pub(crate) fn graph_tree_hash(package: &Package, graph_index: usize) -> u64 {
graph_tree_hash_with_registry(package, package, graph_index)
}
pub(crate) fn graph_tree_hash_with_registry(registry: &Package, pkg: &Package, gi: usize) -> u64 {
let mut h = DefaultHasher::new();
let mut pending = vec![(pkg, gi)];
pub(crate) fn graph_tree_hash_with_registry(registry: &Package, package: &Package, graph_index: usize) -> u64 {
let mut hasher = DefaultHasher::new();
let mut pending = vec![(package, graph_index)];
let mut seen = HashSet::new();
while let Some((pkg, index)) = pending.pop() {
if !seen.insert((std::ptr::from_ref(pkg) as usize, index)) {
while let Some((package, index)) = pending.pop() {
if !seen.insert((std::ptr::from_ref(package) as usize, index)) {
continue;
}
let Some(graph) = pkg.graphs.get(index) else { continue };
index.hash(&mut h);
serde_json::to_string(graph).unwrap_or_default().hash(&mut h);
for (id, res) in &graph.input_images {
pkg.resource(res).map(|r| r.bytes.content_hash()).hash(&mut h);
id.hash(&mut h);
graph.input_images_linear.contains(id).hash(&mut h);
let Some(graph) = package.graphs.get(index) else { continue };
index.hash(&mut hasher);
serde_json::to_string(graph).unwrap_or_default().hash(&mut hasher);
for (id, resource_name) in &graph.input_images {
package.resource(resource_name).map(|resource| resource.bytes.content_hash()).hash(&mut hasher);
id.hash(&mut hasher);
graph.input_images_linear.contains(id).hash(&mut hasher);
}
for instance in graph.functions().flat_map(|function| &function.nodes).filter_map(|node| node.instance.as_ref()) {
let source = if instance.package.is_empty() { Some(pkg) } else { registry.dependencies.get(&instance.package).map(AsRef::as_ref) };
let source = if instance.package.is_empty() { Some(package) } else { registry.dependencies.get(&instance.package).map(AsRef::as_ref) };
if let Some(source) = source
&& let Some((index, _)) = source.graph_by_name(&instance.graph)
{
pending.push((source, index));
}
}
for n in &graph.nodes {
if let Some(Value::Text(name)) = n.params.get("resource") {
pkg.resource(name).map(|r| r.bytes.content_hash()).hash(&mut h);
for node in &graph.nodes {
if let Some(Value::Text(name)) = node.params.get("resource") {
package.resource(name).map(|resource| resource.bytes.content_hash()).hash(&mut hasher);
}
if n.def == "graph_instance"
&& let Some((source, index, _)) = crate::graph_instance::resolve(registry, pkg, n)
if node.def == "graph_instance"
&& let Some((source, index, _)) = crate::graph_instance::resolve(registry, package, node)
{
pending.push((source, index));
}
}
}
h.finish()
hasher.finish()
}
pub(crate) fn variable_key(node: &Node, parameters: &BTreeMap<String, Value>, variables: &BTreeMap<String, Value>) -> u64 {
+12 -7
View File
@@ -75,6 +75,7 @@ pub(crate) fn cook_node(
if output.as_ref().is_some_and(|output| output.as_value().is_some()) && !matches!(node.def.as_str(), "output" | "dot") {
input_error = Some(format!("input `{}` requires an image", port.id));
}
inputs.push(output.and_then(NodeOutput::as_image).cloned());
}
@@ -86,6 +87,7 @@ pub(crate) fn cook_node(
Ok((base, values)) => (base, values, None),
Err(error) => (crate::ResolvedBaseParameters::resolve(&node.base, input_base, prepared.parent_base), BTreeMap::new(), Some(error)),
};
if node.disabled
&& let Some(input) = input_base
{
@@ -103,6 +105,7 @@ pub(crate) fn cook_node(
key,
cached: false,
};
if let Some(error) = base_error {
return fail(error, 0);
}
@@ -173,8 +176,7 @@ pub(crate) fn cook_node(
incoming.first().copied().flatten().cloned().map(|output| vec![output]).ok_or("missing value input".into())
} else if let Some(image) = input_image {
let mut image = if node.def == "input_grayscale" && image.kind == Kind::Color {
// Grayscale input bridges use the default equal RGB weights, not
// display luminance. Preserve the fused order for float resources.
// Grayscale bridges use equal RGB weights and preserve fused float order
let weight = 1.0 / 3.0;
image.map(Kind::Gray, |pixel| [pixel[2].mul_add(weight, pixel[1].mul_add(weight, pixel[0] * weight)), 0.0, 0.0, 0.0])
} else if node.def == "input_color" {
@@ -182,16 +184,19 @@ pub(crate) fn cook_node(
} else {
image.clone()
};
// Bound resources clamp at their edges and resize through stored factor-two levels.
// A single resize changes the edge filter; skipping storage conversion leaks HDR
// values through integer input bridges and changes subsequent function samples.
// Bound resources clamp edges through stored factor-two levels
// Preserve resampling and storage conversion for HDR bridges
let resized = image.w != w || image.h != h;
while image.w != w || image.h != h {
let next = |current: u32, target: u32| {
if current < target { current.saturating_mul(2).min(target) } else { current.div_ceil(2).max(target) }
};
image = crate::output_precision::apply(image.resize(next(image.w, w), next(image.h, h), false), base.format);
}
let image = crate::output_precision::apply(image, base.format);
// Resized inputs already store the final level at the declared precision
let image = if resized { image } else { crate::output_precision::apply(image, base.format) };
Ok(vec![NodeOutput::Image(image)])
} else if node.def == materialcraft_core::MISSING_NODE {
let name = if node.label.is_empty() { "this node" } else { node.label.as_str() };
@@ -205,7 +210,7 @@ pub(crate) fn cook_node(
images
.into_iter()
.map(|image| {
// Bridges retain their source pixels; filters store their declared precision.
// Bridges retain their source pixels; filters store their declared precision
let image = if matches!(node.def.as_str(), "output" | "dot") { image } else { crate::output_precision::apply(image, base.format) };
NodeOutput::Image(image)
})
+15 -8
View File
@@ -3,31 +3,38 @@ use materialcraft_image::Image;
use std::sync::Arc;
pub(crate) fn apply(mut image: Image, format: OutputFormat) -> Image {
const HALF_FLOAT_MAX: f32 = 65504.0;
const U8_FIXED_SCALE: f64 = 4096.0;
const U16_FIXED_SCALE: f64 = 1_048_576.0;
image.integer_sampling = integer_sampling(format);
if format == OutputFormat::F16 {
for pixel in Arc::make_mut(&mut image.data) {
// Half-float storage saturates finite overflow and truncates toward zero.
// Leaving the f32 unconverted changes every downstream read and export.
let value = pixel.clamp(-65504.0, 65504.0);
// Half-float storage saturates finite overflow and truncates toward zero
// Leaving the f32 unconverted changes every downstream read and export
let value = pixel.clamp(-HALF_FLOAT_MAX, HALF_FLOAT_MAX);
let rounded = half::f16::from_f32(value);
*pixel = if rounded.to_f32().abs() > value.abs() { half::f16::from_bits(rounded.to_bits().saturating_sub(1)).to_f32() } else { rounded.to_f32() };
}
return image;
}
let (fraction, maximum) = match format {
OutputFormat::U8 => (4096.0_f64, 255.0_f64),
OutputFormat::Inherit | OutputFormat::U16 => (1_048_576.0, 65535.0),
OutputFormat::U8 => (U8_FIXED_SCALE, f64::from(u8::MAX)),
OutputFormat::Inherit | OutputFormat::U16 => (U16_FIXED_SCALE, f64::from(u16::MAX)),
OutputFormat::F16 | OutputFormat::F32 => return image,
};
for pixel in Arc::make_mut(&mut image.data) {
// Integer outputs retain four fractional guard bits before conversion.
// This truncation and halfway-down rounding are observable even for constants;
// rounding directly from the float changes noise and every downstream blend.
// Integer outputs retain four fractional guard bits before conversion
// Halfway-down rounding affects constants and downstream blends
let value = if pixel.is_nan() { 0.0 } else { f64::from(*pixel).clamp(0.0, 1.0) };
let fixed = (value * fraction).floor() / fraction;
let integer = (fixed * maximum - 0.5).ceil() as u32;
*pixel = integer as f32 / maximum as f32;
}
image
}
+1
View File
@@ -18,6 +18,7 @@ pub(crate) fn resolve_parameters(
parameters
}
};
for parameter in node.bindings.keys().chain(node.functions.keys()) {
if !parameters.contains_key(parameter)
&& let Some(descriptor) = ParameterDescriptor::for_node(package, node, parameter)
+3 -1
View File
@@ -12,7 +12,8 @@ pub(crate) fn evaluate(
) -> Result<Value, String> {
let program = compile(function, seed)?;
let scalar_channel = allow_scalar && expected == ValueType::Float4 && program.output_type() == ValueType::Float as u32;
// A color parameter also takes a gray level or an opaque RGB triple.
// A color parameter also takes a gray level or an opaque RGB triple
let widened = expected == ValueType::Float4 && [ValueType::Float as u32, ValueType::Float3 as u32].contains(&program.output_type());
if program.output_type() != expected as u32 && !scalar_channel && !widened {
return Err(format!("output must be {}", expected.label()));
@@ -28,6 +29,7 @@ pub(crate) fn evaluate(
Value::Float4(values) | Value::Color(values) => values.iter().all(|value| value.is_finite()),
_ => true,
};
if !finite {
return Err("function returned a non-finite value".into());
}
+6 -5
View File
@@ -27,12 +27,12 @@ impl ResolvedBaseParameters {
pub(crate) fn resolve(base: &BaseParameters, input: Option<Self>, parent: Self) -> Self {
let size_base = BaseParameter::OutputSize.inheritance(base).resolve([1; 2], input.map(|base| base.size), parent.size);
// Internal reductions and lookup images can reach one texel. Applying
// the graph editor's 16-pixel minimum here breaks their size functions.
// Internal images may be one texel; the editor minimum would break size functions
let size = std::array::from_fn(|axis| (f64::from(size_base[axis]) * 2f64.powi(base.size[axis].clamp(-12, 13))).round().clamp(1.0, 8192.0) as u32);
// Images carry size and format, but do not carry a random stream.
// Both relative seed modes use the containing graph's seed; chaining
// image filters must not accumulate the seeds of upstream nodes.
// Images carry size and format, but do not carry a random stream
// Relative seed modes use the graph seed without accumulating upstream seeds
let inherited_seed = base.seed_mode.resolve(0, None, parent.seed);
let inherited_pixel_size = base.pixel_size_mode.resolve([1.0; 2], input.map(|base| base.pixel_size), parent.pixel_size);
Self {
@@ -96,6 +96,7 @@ pub(crate) fn evaluate(
} else {
resolved.size
};
let value = crate::parameter_function::evaluate(function, parameter.value_type(), variables.to_mut(), resolved.seed, size, false)
.map_err(|error| format!("function of `{}`: {error}", parameter.identifier()))?;
parameter.set_value(&mut evaluated, &value)?;
+47 -9
View File
@@ -30,14 +30,19 @@ fn cooks_and_caches() {
let cache = Mutex::new(Cache::default());
let opts = Options::new(&no_res);
let a = cook(&p, 0, &opts, &cache);
assert!(a.nodes.values().all(|r| r.error.is_none() && !r.cached));
assert_eq!(a.image(out, 0).map(|i| (i.w, i.h)), Some((64, 64)));
// Same graph: everything from the cache
let b = cook(&p, 0, &opts, &cache);
assert!(b.nodes.values().all(|r| r.cached));
// Editing the blur re-cooks the blur and the output, not the noise
p.graphs[0].node_mut(blur).unwrap().params.insert("intensity".into(), Value::Float(5.0));
let c = cook(&p, 0, &opts, &cache);
assert!(c.nodes[&noise].cached && !c.nodes[&blur].cached && !c.nodes[&out].cached);
assert_eq!(outputs(&p.graphs[0], &c).len(), 1);
}
@@ -70,13 +75,15 @@ fn processor_tiling_variable_tracks_both_axes_and_invalidates_cached_values() {
#[test]
fn base_parameter_assignments_reach_pixel_functions_without_leaking_to_other_nodes() {
use materialcraft_core::{BaseParameter, FunctionGraph};
let mut init = FunctionGraph::default();
let value = init.add_constant(Value::Float(0.375));
let assign = init.add("set", &[value], Some(Value::Text("prepared_value".into())));
let size = init.add_constant(Value::Int2([0; 2]));
init.output = Some(init.add("sequence", &[assign, size], None));
let mut initialization = FunctionGraph::default();
let value = initialization.add_constant(Value::Float(0.375));
let assignment = initialization.add("set", &[value], Some(Value::Text("prepared_value".into())));
let output_size = initialization.add_constant(Value::Int2([0; 2]));
initialization.output = Some(initialization.add("sequence", &[assignment, output_size], None));
let mut function = FunctionGraph::default();
function.output = Some(function.add("get_float1", &[], Some(Value::Text("prepared_value".into()))));
let mut graph = Graph::new("Parameter scope");
graph.size = [4; 2];
graph.format = materialcraft_core::OutputFormat::F32;
@@ -86,18 +93,23 @@ fn base_parameter_assignments_reach_pixel_functions_without_leaking_to_other_nod
graph.node_mut(id).unwrap().params.insert("function".into(), Value::Function(Box::new(function.clone())));
graph.node_mut(id).unwrap().params.insert("color_mode".into(), Value::Bool(false));
}
graph.node_mut(initialized).unwrap().base.functions.insert(BaseParameter::OutputSize, init);
graph.node_mut(initialized).unwrap().base.functions.insert(BaseParameter::OutputSize, initialization);
let mut package = Package::new("Parameter scope");
package.graphs.push(graph);
let cache = Mutex::new(Cache::default());
for expected in [0.375, 0.625] {
package.graphs[0].node_mut(initialized).unwrap().base.functions.get_mut(&BaseParameter::OutputSize).unwrap().nodes[0].value =
Some(Value::Float(expected));
let cooked = cook(&package, 0, &Options::new(&no_res), &cache);
assert_eq!(cooked.nodes[&initialized].error, None);
assert!(!cooked.nodes[&initialized].cached);
assert!((cooked.image(initialized, 0).unwrap().mean()[0] - expected).abs() < 1e-6);
assert!(cooked.nodes[&isolated].error.as_deref().unwrap().contains("undefined variable `prepared_value`"));
// Unassigned user variables start at typed zero in each node's own scope
assert_eq!(cooked.nodes[&isolated].error, None);
assert_eq!(cooked.image(isolated, 0).unwrap().mean()[0], 0.0);
}
}
@@ -129,6 +141,7 @@ fn preview_cap_and_unknown_nodes() {
#[test]
fn graph_instances_cook_their_subgraph() {
let (mut p, ..) = sample();
// A second graph: input → invert → output, instanced from the first
let mut sub = Graph::new("inv");
sub.size = [6, 6];
@@ -147,12 +160,14 @@ fn graph_instances_cook_their_subgraph() {
let inst = g.add_node("graph_instance", [100.0, 100.0]);
g.node_mut(inst).unwrap().params.insert("graph".into(), Value::Text("inv".into()));
g.connect(u, "output", inst, "input").unwrap();
// A graph instancing itself is refused, not a stack overflow
let selfi = g.add_node("graph_instance", [0.0, 200.0]);
g.node_mut(selfi).unwrap().params.insert("graph".into(), Value::Text("g".into()));
let cache = Mutex::new(Cache::default());
let c = cook(&p, 0, &Options::new(&no_res), &cache);
let img = c.image(inst, 0).unwrap();
assert!((img.data[0] - (1.0 - 0.0722 * 0.25 - 0.2126 * 0.25 - 0.7152 * 0.25)).abs() < 1e-4, "{}", img.data[0]);
assert!(c.nodes[&selfi].error.as_deref().is_some_and(|e| e.contains("uses itself")), "{:?}", c.nodes[&selfi].error);
}
@@ -200,6 +215,7 @@ fn deep_shared_instance_walks_stay_bounded() {
let mut pkg = Package::new("deep");
for index in 0..2000 {
let mut graph = Graph::new(&format!("g{index}"));
// Repeated references must visit the next graph only once
for _ in 0..4 {
let id = graph.add_node("graph_instance", [0.0; 2]);
@@ -207,13 +223,18 @@ fn deep_shared_instance_walks_stay_bounded() {
}
pkg.graphs.push(graph);
}
assert!(instances_reach(&pkg, 0, 1999));
assert!(!instances_reach(&pkg, 0, 2000));
let before = graph_tree_hash(&pkg, 0);
pkg.graphs[1999].seed = 42;
assert_ne!(before, graph_tree_hash(&pkg, 0));
// A back edge must terminate both traversals too
pkg.graphs[1999].nodes[0].params.insert("graph".into(), Value::Text("g0".into()));
assert!(instances_reach(&pkg, 1, 0));
assert!(!instances_reach(&pkg, 0, 2000));
assert_ne!(before, graph_tree_hash(&pkg, 0));
@@ -245,6 +266,7 @@ fn same_size_resources_key_differently() {
let mut p = Package::new("p");
p.graphs.push(g);
p.resources.push(Resource::new("tex", "tex.bin", ResourceBytes::new(vec![1; 100])));
// Stands in for decoding: the first byte picks the shade
let shade = AtomicU8::new(1);
let res = |_: &str| Some(Image::filled(4, 4, Kind::Gray, [f32::from(shade.load(Ordering::Relaxed)) / 10.0; 4]));
@@ -255,6 +277,7 @@ fn same_size_resources_key_differently() {
p.resources[0].bytes = ResourceBytes::new(vec![2; 100]);
shade.store(2, Ordering::Relaxed);
let second = cook(&p, 0, &opts, &cache);
assert!(!second.nodes[&bm].cached);
assert!((second.image(bm, 0).unwrap().mean()[0] - mean).abs() > 0.05);
}
@@ -274,11 +297,14 @@ fn input_nodes_read_bound_preview_images() {
let cache = Mutex::new(Cache::default());
let c = cook(&p, 0, &Options::new(&res), &cache);
let img = c.image(o, 0).unwrap();
assert!(img.is_gray() && (img.data[0] - 0.8).abs() < 1e-4, "bound image, as gray");
// The caller's image wins over the bound one
let mut opts = Options::new(&res);
opts.inputs.insert("curvature".into(), Image::filled(4, 4, Kind::Gray, [0.1; 4]));
let c = cook(&p, 0, &opts, &cache);
assert!((c.image(o, 0).unwrap().data[0] - 0.1).abs() < 1e-4);
}
@@ -414,11 +440,12 @@ fn changed_unsampled_input_recooks() {
}
#[test]
fn linear_input_images_and_bitmap_nodes_request_data_reads() {
// The decoder returns 0.5 for a linear request and 0.2 for an sRGB one.
// The decoder returns 0.5 for a linear request and 0.2 for an sRGB one
let res = |k: &str| {
let (name, linear) = materialcraft_image::split_resource_key(k);
(name == "m").then(|| Image::filled(4, 4, Kind::Color, [if linear { 0.5 } else { 0.2 }; 4]))
};
let mut p = Package::new("p");
let mut g = Graph::new("g");
g.size = [4, 4];
@@ -434,9 +461,12 @@ fn linear_input_images_and_bitmap_nodes_request_data_reads() {
let c = cook(p, 0, &Options::new(&res), &cache);
(c.image(i, 0).unwrap().data[0], c.image(b, 0).unwrap().data[0])
};
assert_eq!(first(&p), (0.2, 0.2), "default is sRGB, as before");
p.graphs[0].input_images_linear.insert("n".into());
p.graphs[0].node_mut(b).unwrap().params.insert("colour_space".into(), Value::Enum(1));
assert_eq!(first(&p), (0.5, 0.5));
}
@@ -455,19 +485,25 @@ fn bound_input_image_resource_change_recooks_instance() {
let mut p = Package::new("p");
p.graphs = vec![top, sub];
p.resources.push(Resource::new("curv", "curv.bin", ResourceBytes::new(vec![1; 100])));
// Stands in for decoding: the first byte picks the shade.
// Stands in for decoding: the first byte picks the shade
let shade = AtomicU8::new(1);
let res = |_: &str| Some(Image::filled(4, 4, Kind::Gray, [f32::from(shade.load(Ordering::Relaxed)) / 10.0; 4]));
let cache = Mutex::new(Cache::default());
let opts = Options::new(&res);
let first = cook(&p, 0, &opts, &cache);
assert!(first.nodes[&inst].error.is_none(), "{:?}", first.nodes[&inst].error);
assert!(!first.nodes[&inst].cached);
let unchanged = cook(&p, 0, &opts, &cache);
assert!(unchanged.nodes[&inst].cached, "an unchanged instance should be served from the cache");
p.resources[0].bytes = ResourceBytes::new(vec![2; 100]);
shade.store(2, Ordering::Relaxed);
let second = cook(&p, 0, &opts, &cache);
assert!(!second.nodes[&inst].cached, "changing the bound input image resource must recook the instance");
}
@@ -590,6 +626,7 @@ fn enabled_generators_use_parent_size_and_seed_with_input_image_properties() {
assert_eq!(result.base.tiling, [false, true]);
assert_eq!(result.base.pixel_ratio, 1);
assert_eq!(result.base.pixel_size, [0.25, 0.5]);
if !dynamic {
assert_eq!(
node_size(&package.graphs[0], materialcraft_nodes::find(definition), package.graphs[0].node(generator).unwrap(), Some((32, 64)), Some(16)),
@@ -631,6 +668,7 @@ fn variables_set_by_a_base_parameter_function_reach_the_nodes_other_functions()
let cooked = cook(&package, 0, &Options::new(&no_res), &Mutex::new(Cache::default()));
let result = &cooked.nodes[&generator];
assert_eq!(result.error, None);
assert_eq!(result.parameters.get("roughness"), Some(&Value::Float(2.0)));
}
+13 -4
View File
@@ -8,13 +8,18 @@ use std::{
};
type Result<T> = std::result::Result<T, Box<dyn std::error::Error>>;
const MAX_DIRECTORY_DEPTH: usize = 6;
const MAX_BUNDLED_FILES: usize = 4096;
const MAX_ASSET_BYTES: usize = 256 << 20;
#[path = "build/sbs_font.rs"]
mod sbs_font;
fn scan(root: &Path, folder: &Path, files: &mut Vec<PathBuf>) -> Result<()> {
if folder.components().count() > 6 || files.len() > 4096 {
if folder.components().count() > MAX_DIRECTORY_DEPTH || files.len() > MAX_BUNDLED_FILES {
return Err("bundled library exceeds its scan limits".into());
}
let path = root.join(folder);
println!("cargo:rerun-if-changed={}", path.display());
for entry in std::fs::read_dir(path)? {
@@ -24,19 +29,22 @@ fn scan(root: &Path, folder: &Path, files: &mut Vec<PathBuf>) -> Result<()> {
if name.starts_with('.') {
continue;
}
let kind = entry.file_type()?;
let relative = folder.join(name);
if kind.is_dir() {
scan(root, &relative, files)?;
} else if kind.is_file() && (name.ends_with(".sbs") || relative.starts_with("resources")) {
if files.len() >= 4096 {
if files.len() >= MAX_BUNDLED_FILES {
return Err("bundled library exceeds its file limit".into());
}
files.push(relative);
} else if kind.is_symlink() {
return Err("symlinks are not bundled library assets".into());
}
}
Ok(())
}
@@ -54,10 +62,11 @@ fn main() -> Result<()> {
let mut packages = String::from("pub const BUNDLED: &[&str] = &[\n");
for (index, (relative, file)) in files.iter().enumerate() {
let mut bytes = Vec::new();
std::fs::File::open(file)?.take((256 << 20) + 1).read_to_end(&mut bytes)?;
if bytes.len() > 256 << 20 {
std::fs::File::open(file)?.take(MAX_ASSET_BYTES as u64 + 1).read_to_end(&mut bytes)?;
if bytes.len() > MAX_ASSET_BYTES {
return Err("bundled asset exceeds 256 MiB".into());
}
let mut encoder = flate2::write::DeflateEncoder::new(Vec::new(), flate2::Compression::best());
encoder.write_all(&bytes)?;
std::fs::write(compressed.join(format!("{index}.deflate")), encoder.finish()?)?;
+14 -4
View File
@@ -7,24 +7,31 @@ use std::{
type Result<T> = std::result::Result<T, Box<dyn std::error::Error>>;
const MAX_MANIFEST_BYTES: u64 = 1 << 20;
const MAX_FONT_BYTES: u64 = 16 << 20;
const MAX_LICENSE_BYTES: u64 = 64 << 10;
fn read(path: &Path, limit: u64) -> Result<Vec<u8>> {
let mut bytes = Vec::new();
std::fs::File::open(path)?.take(limit + 1).read_to_end(&mut bytes)?;
if bytes.is_empty() || bytes.len() as u64 > limit {
return Err(format!("{} is empty or exceeds its size limit", path.display()).into());
}
Ok(bytes)
}
fn checkout_file(root: &Path, relative: &str) -> Result<PathBuf> {
// A manifest must not embed arbitrary files from the build host, including through symlinks.
// A manifest must not embed arbitrary files from the build host, including through symlinks
if relative.is_empty() || relative.contains(['\\', ':']) || !Path::new(relative).components().all(|part| matches!(part, Component::Normal(_))) {
return Err("font manifest path is not checkout-relative".into());
}
let path = root.join(relative).canonicalize()?;
if !path.starts_with(root) {
return Err("font manifest path escapes the checkout".into());
}
println!("cargo:rerun-if-changed={}", path.display());
Ok(path)
}
@@ -32,7 +39,7 @@ fn checkout_file(root: &Path, relative: &str) -> Result<PathBuf> {
fn select(directory: &Path) -> Result<Vec<(PathBuf, PathBuf)>> {
let root = directory.canonicalize()?;
let manifest = checkout_file(&root, "fonts/latin-manifest.txt")?;
let bytes = read(&manifest, 1 << 20)?;
let bytes = read(&manifest, MAX_MANIFEST_BYTES)?;
let entries = craft_fonts_build::parse_manifest(std::str::from_utf8(&bytes)?)?;
let face = entries
.iter()
@@ -41,13 +48,15 @@ fn select(directory: &Path) -> Result<Vec<(PathBuf, PathBuf)>> {
if face.licence != "OFL-1.1" || face.index != 0 {
return Err("the SBS fallback must be the OFL-1.1 Liberation Mono Regular single font".into());
}
let font = checkout_file(&root, &face.file)?;
let bytes = read(&font, 16 << 20)?;
let bytes = read(&font, MAX_FONT_BYTES)?;
if !bytes.starts_with(b"\0\x01\0\0") || format!("{:x}", Sha256::digest(&bytes)) != face.sha256 {
return Err("Liberation Mono Regular does not match its catalog checksum or TrueType format".into());
}
let license = checkout_file(&root, &face.licence_file)?;
read(&license, 64 << 10)?;
read(&license, MAX_LICENSE_BYTES)?;
Ok(vec![("resources/LiberationMono-Regular.ttf".into(), font), ("resources/OFL-LiberationMono.txt".into(), license)])
}
@@ -61,6 +70,7 @@ pub fn assets() -> Result<Vec<(PathBuf, PathBuf)>> {
None if required => Err("CRAFT_FONTS_REQUIRED=1 requires CRAFT_FONTS_DIR for the SBS fallback font".into()),
None => return Ok(Vec::new()),
};
match selected {
Ok(assets) => Ok(assets),
Err(error) if required => Err(error),
@@ -1,16 +1,7 @@
use super::{
Result, layout, resources,
xml::{self, Edit},
};
use super::{Result, converted_package::ConvertedPackage, layout, resources, xml, xml_edit::XmlEdit};
use roxmltree::Node;
use std::collections::{BTreeMap, BTreeSet};
pub struct Converted {
pub xml: String,
pub resources: Vec<&'static str>,
pub report: serde_json::Value,
}
fn presentation(node: Node<'_, '_>) -> bool {
matches!(node.tag_name().name(), "GUILayout" | "GUIObjects" | "GUIName" | "desc")
|| (node.parent().is_some_and(|p| p.has_tag_name("attributes"))
@@ -29,12 +20,14 @@ pub fn semantics(document: &roxmltree::Document<'_>) -> String {
if node.ancestors().any(|n| presentation(n) || replaced_resource_field(n)) {
continue;
}
if node.has_tag_name("fileUID")
|| (matches!(node.tag_name().name(), "formatVersion" | "updaterVersion") && node.parent().is_some_and(|p| p.has_tag_name("package")))
|| (node.has_tag_name("filename") && node.parent().is_some_and(|p| p.has_tag_name("dependency")))
{
continue;
}
if node.is_element() && !node.has_tag_name("attributes") {
{
result.push_str(&node.ancestors().filter(|n| !n.has_tag_name("attributes")).count().to_string());
@@ -53,6 +46,7 @@ pub fn semantics(document: &roxmltree::Document<'_>) -> String {
result.push_str(node.text().unwrap_or(""));
}
}
result
}
@@ -95,20 +89,22 @@ fn attributes(owner: Node<'_, '_>, text: &str, description: &str) -> Result<Stri
Ok(output)
}
pub fn package(text: &str, name: &str, identities: &BTreeMap<String, String>) -> Result<Converted> {
pub fn package(text: &str, name: &str, identities: &BTreeMap<String, String>) -> Result<ConvertedPackage> {
let document = xml::parse(text)?;
let root = document.root_element();
if !root.has_tag_name("package") {
return Err("expected a native SBS package".into());
}
let before = semantics(&document);
let mut edits = Vec::new();
for tag in ["formatVersion", "updaterVersion"] {
if xml::child(root, tag).is_none() {
let offset = root.first_element_child().ok_or("package has no content")?.range().start;
edits.push(Edit { range: offset..offset, replacement: xml::leaf(tag, env!("CARGO_PKG_VERSION")) });
edits.push(XmlEdit { range: offset..offset, replacement: xml::leaf(tag, env!("CARGO_PKG_VERSION")) });
}
}
let mut resource_keys = Vec::new();
let mut graphs = 0;
let mut functions = 0;
@@ -119,6 +115,7 @@ pub fn package(text: &str, name: &str, identities: &BTreeMap<String, String>) ->
if node.ancestors().skip(1).any(|n| n.has_tag_name("GUIObjects") || n.has_tag_name("resource") || n.has_tag_name("attributes")) {
continue;
}
match node.tag_name().name() {
"graph" => graphs += 1,
"function" if node.parent().is_some_and(|p| p.has_tag_name("content")) => functions += 1,
@@ -128,9 +125,9 @@ pub fn package(text: &str, name: &str, identities: &BTreeMap<String, String>) ->
}
"attributes" => {
let owner = node.parent().ok_or("attribute owner is missing")?;
edits.push(Edit { range: node.range(), replacement: attributes(owner, text, &description(owner))? });
edits.push(XmlEdit { range: node.range(), replacement: attributes(owner, text, &description(owner))? });
}
"GUIObjects" => edits.push(Edit { range: node.range(), replacement: String::new() }),
"GUIObjects" => edits.push(XmlEdit { range: node.range(), replacement: String::new() }),
"GUIName" => xml::set(&mut edits, node, ""),
"desc" => {
let owner = node.parent().ok_or("description owner is missing")?;
@@ -169,13 +166,15 @@ pub fn package(text: &str, name: &str, identities: &BTreeMap<String, String>) ->
if !font_replacement && std::path::Path::new(original).file_stem() != output.file_stem() {
return Err(format!("{name}: resource `{key}` would change its filename stem").into());
}
let extension = std::path::Path::new(original).extension();
// Surface inputs are replaced with a standard image container;
// the requested font substitute is the only filename-stem exception.
// Only the requested font substitute may change the filename stem
if !font_replacement && extension != Some(std::ffi::OsStr::new("surface")) && extension != output.extension() {
return Err(format!("{name}: resource `{key}` would change its extension").into());
}
}
let mut replacement = String::from("<resource>");
for child in node.children().filter(Node::is_element) {
if !matches!(child.tag_name().name(), "source" | "filepath" | "format" | "attributes") {
@@ -186,21 +185,24 @@ pub fn package(text: &str, name: &str, identities: &BTreeMap<String, String>) ->
replacement.push_str(&xml::leaf("filepath", &xml::relative(name, recipe.path)?));
replacement.push_str(&attributes(node, text, recipe.description)?);
replacement.push_str("</resource>");
edits.push(Edit { range: node.range(), replacement });
edits.push(XmlEdit { range: node.range(), replacement });
resource_keys.push(recipe.key);
}
_ => {}
}
}
let output = xml::apply(text, edits)?;
let after = xml::parse(&output)?;
if semantics(&after) != before {
return Err(format!("{name}: conversion changed native computational data").into());
}
if after.descendants().any(|n| n.has_tag_name("binembedded") || n.has_tag_name("icon")) {
return Err(format!("{name}: source asset payload survived conversion").into());
}
Ok(Converted {
Ok(ConvertedPackage {
xml: output,
resources: resource_keys,
report: serde_json::json!({
@@ -218,21 +220,26 @@ pub fn verify_references(text: &str, owner: &str, available: &BTreeSet<String>)
if !is_dependency && !is_resource {
continue;
}
let reference = node.attribute("v").ok_or("missing reference value")?;
if reference == "?himself" {
continue;
}
if reference.contains("://") {
return Err("converted references must be package-relative".into());
}
let resolved = materialcraft_formats::sbs::library::resolve(&format!("sbs://{owner}"), reference)?;
let relative = materialcraft_formats::sbs::library::relative_path(&resolved)?;
if !available.contains(relative) {
return Err(format!("{owner}: reference does not resolve: {reference}").into());
}
if is_resource && !relative.starts_with("resources/") {
return Err("resource is outside the shared resources folder".into());
}
}
Ok(())
}
@@ -0,0 +1,5 @@
pub struct ConvertedPackage {
pub xml: String,
pub resources: Vec<&'static str>,
pub report: serde_json::Value,
}
@@ -1,15 +1,13 @@
use super::{
Result,
xml::{self, Edit},
};
use super::{Result, xml, xml_edit::XmlEdit};
use roxmltree::Node;
use std::collections::{BTreeMap, BTreeSet};
pub fn reposition(list: Node<'_, '_>, edits: &mut Vec<Edit>) -> Result<usize> {
pub fn reposition(list: Node<'_, '_>, edits: &mut Vec<XmlEdit>) -> Result<usize> {
let nodes: Vec<_> = list.children().filter(Node::is_element).collect();
if nodes.len() > 100_000 {
return Err("canvas exceeds the node limit".into());
}
let mut ids = BTreeMap::new();
for node in &nodes {
let id = xml::value(*node, "uid").ok_or("canvas node has no uid")?;
@@ -17,6 +15,7 @@ pub fn reposition(list: Node<'_, '_>, edits: &mut Vec<Edit>) -> Result<usize> {
return Err("duplicate canvas node uid".into());
}
}
let mut pending = BTreeMap::new();
let mut successors = BTreeMap::<&str, Vec<&str>>::new();
let mut columns = BTreeMap::<&str, usize>::new();
@@ -31,6 +30,7 @@ pub fn reposition(list: Node<'_, '_>, edits: &mut Vec<Edit>) -> Result<usize> {
if sources.iter().any(|source| !ids.contains_key(source)) {
return Err("canvas connection references an absent node".into());
}
pending.insert(*id, sources.len());
if sources.is_empty() {
ready.insert(*id);
@@ -39,6 +39,7 @@ pub fn reposition(list: Node<'_, '_>, edits: &mut Vec<Edit>) -> Result<usize> {
successors.entry(source).or_default().push(id);
}
}
let mut visited = 0;
while let Some(id) = ready.pop_first() {
visited += 1;
@@ -53,9 +54,11 @@ pub fn reposition(list: Node<'_, '_>, edits: &mut Vec<Edit>) -> Result<usize> {
}
}
}
if visited != nodes.len() {
return Err("canvas contains a connection cycle".into());
}
let mut rows = BTreeMap::<usize, usize>::new();
for (id, node) in ids {
let column = columns.get(id).copied().unwrap_or(0);
@@ -69,7 +72,8 @@ pub fn reposition(list: Node<'_, '_>, edits: &mut Vec<Edit>) -> Result<usize> {
let end = node.range().end.checked_sub(node.tag_name().name().len() + 3).ok_or("invalid node end")?;
end..end
};
edits.push(Edit { range, replacement });
edits.push(XmlEdit { range, replacement });
}
Ok(nodes.len())
}
@@ -4,48 +4,64 @@
#![deny(clippy::unwrap_used, clippy::expect_used, clippy::panic, clippy::unimplemented, clippy::todo, clippy::unreachable)]
mod convert;
mod converted_package;
mod layout;
mod references;
mod resource_random;
mod resource_recipe;
mod resources;
#[cfg(test)]
mod tests;
mod xml;
mod xml_edit;
use std::{collections::BTreeMap, io::Read, path::Path};
type Result<T> = std::result::Result<T, Box<dyn std::error::Error>>;
const MAX_FILE_BYTES: u64 = 64 << 20;
const MAX_PACKAGE_FILES: usize = 2048;
const MAX_DIRECTORY_DEPTH: usize = 6;
const MAX_CONVERTED_BYTES: usize = 512 << 20;
fn read(path: &Path) -> Result<Vec<u8>> {
let metadata = std::fs::symlink_metadata(path)?;
if !metadata.is_file() || metadata.len() > MAX_FILE_BYTES {
return Err(format!("{} is not a regular file within the 64 MiB limit", path.display()).into());
}
let mut bytes = Vec::new();
std::fs::File::open(path)?.take(MAX_FILE_BYTES + 1).read_to_end(&mut bytes)?;
if bytes.len() as u64 > MAX_FILE_BYTES {
return Err("file grew beyond the read limit".into());
}
Ok(bytes)
}
fn inventory(root: &Path, relative: &Path, files: &mut Vec<String>) -> Result<()> {
if relative.components().count() > 6 || files.len() > 2048 {
if relative.components().count() > MAX_DIRECTORY_DEPTH {
return Err("package inventory exceeds its depth or file limit".into());
}
let mut entries = std::fs::read_dir(root.join(relative))?.collect::<std::io::Result<Vec<_>>>()?;
entries.sort_by_key(|entry| entry.file_name());
for entry in entries {
// The caller sorts package names once after traversal; directory entries need not be retained
for entry in std::fs::read_dir(root.join(relative))? {
let entry = entry?;
let name = entry.file_name().into_string().map_err(|_| "non-UTF-8 package filename")?;
let kind = entry.file_type()?;
// Resource directories are deliberately never opened: replacements have independent recipes.
// Resource directories are deliberately never opened: replacements have independent recipes
if name.eq_ignore_ascii_case("resources") || name.ends_with(".resources") || name.starts_with('.') {
continue;
}
let path = relative.join(&name);
if kind.is_dir() {
inventory(root, &path, files)?;
} else if kind.is_file() && name.ends_with(".sbs") {
if files.len() >= MAX_PACKAGE_FILES {
return Err("package inventory exceeds its depth or file limit".into());
}
let path = path.to_string_lossy().replace('\\', "/");
materialcraft_formats::sbs::library::relative_path(&format!("sbs://{path}"))?;
files.push(path);
@@ -53,6 +69,7 @@ fn inventory(root: &Path, relative: &Path, files: &mut Vec<String>) -> Result<()
return Err(format!("symlink refused in package inventory: {}", path.display()).into());
}
}
Ok(())
}
@@ -61,11 +78,13 @@ fn main() -> Result<()> {
let [source, destination, report] = args.as_slice() else {
return Err("usage: sbs_library_build <source-directory> <destination-directory> <report.json>".into());
};
let source = Path::new(source).canonicalize()?;
let destination = Path::new(destination).canonicalize()?;
if source == destination || destination.starts_with(&source) || source.starts_with(&destination) {
return Err("source and destination trees must be separate".into());
}
let baseline = read(&destination.join("functions.sbs"))?;
let baseline_text = std::str::from_utf8(&baseline)?;
let baseline_doc = xml::parse(baseline_text)?;
@@ -86,13 +105,15 @@ fn main() -> Result<()> {
let mut converted = convert::package(&source, name, &identities)?;
converted.report["qualified_standard_function_references"] = serde_json::json!(qualified);
total = total.checked_add(converted.xml.len()).ok_or("library size overflow")?;
if total > 512 << 20 {
if total > MAX_CONVERTED_BYTES {
return Err("converted library exceeds 512 MiB".into());
}
needed.extend(converted.resources.iter().copied());
reports.push(converted.report);
outputs.insert(name.clone(), converted.xml.into_bytes());
}
let mut assets = Vec::new();
for key in needed {
let recipe = resources::recipe(key)?;
@@ -105,27 +126,32 @@ fn main() -> Result<()> {
outputs.insert(path, bytes);
}
}
// Validate the complete dependency closure before writing any package or asset.
// Validate the complete dependency closure before writing any package or asset
let available = outputs.keys().cloned().chain(std::iter::once("functions.sbs".into())).collect();
for (name, bytes) in outputs.iter().filter(|(name, _)| name.ends_with(".sbs")) {
convert::verify_references(std::str::from_utf8(bytes)?, name, &available)?;
}
for (name, bytes) in &outputs {
let target = destination.join(name);
if target.exists() && read(&target)? != *bytes {
return Err(format!("refusing to overwrite different existing content: {}", target.display()).into());
}
if let Some(parent) = target.parent() {
// Refuse existing symlink parents before creating output directories.
// Refuse existing symlink parents before creating output directories
let mut ancestor = parent;
while ancestor != destination {
if ancestor.exists() && !ancestor.canonicalize()?.starts_with(&destination) {
return Err("output directory escapes the destination".into());
}
ancestor = ancestor.parent().ok_or("invalid output directory")?;
}
}
}
for (name, bytes) in &outputs {
let target = destination.join(name);
if let Some(parent) = target.parent() {
@@ -133,9 +159,11 @@ fn main() -> Result<()> {
}
materialcraft_formats::write_atomic(&target, bytes)?;
}
if read(&destination.join("functions.sbs"))? != baseline {
return Err("the existing function library changed during conversion".into());
}
let report_data = serde_json::json!({
"version": 1, "converted_packages": reports.len(), "preserved_packages": ["functions.sbs"],
"graph_implementations": "retained from the supplied packages; presentation changes do not establish redistribution rights",
@@ -40,10 +40,12 @@ pub fn qualify(text: &str, standard: &BTreeSet<String>) -> Result<(String, usize
if local.contains(path) || !standard.contains(path) {
continue;
}
let [uid] = dependencies.as_slice() else { return Err("bare standard function requires one declared functions.sbs dependency".into()) };
xml::set(&mut edits, value, &format!("pkg:///{path}?dependency={uid}"));
}
}
let count = edits.len();
Ok((xml::apply(text, edits)?, count))
}
@@ -59,19 +61,28 @@ mod tests {
<function><identifier v="Sample_[A,B["/></function></content></group></content></group>"#;
let library = format!("<package><content>{body}</content></package>");
let standard = function_paths(&xml::parse(&library).unwrap());
assert_eq!(standard, BTreeSet::from(["Functions/Random/Sample_[A,B[".into()]));
let source = r#"<package><dependencies><dependency><filename v="sbs://functions.sbs"/><uid v="9"/></dependency></dependencies>
<content><graph><identifier v="Example"/><parameters><parameter><paramValue><dynamicValue><paramNodes><paramNode>
<function v="instance"/><funcDatas><funcData><name v="instance"/><constantValue><constantValueString v="pkg:///Functions/Random/Sample_[A,B["/>
</constantValue></funcData></funcDatas></paramNode></paramNodes></dynamicValue></paramValue></parameter></parameters></graph></content></package>"#;
let (qualified, count) = qualify(source, &standard).unwrap();
assert_eq!(count, 1);
assert_eq!(qualified, source.replace("pkg:///Functions/Random/Sample_[A,B[", "pkg:///Functions/Random/Sample_[A,B[?dependency=9"));
let explicit = source.replace("Sample_[A,B[", "Sample_[A,B[?dependency=7");
assert_eq!(qualify(&explicit, &standard).unwrap(), (explicit, 0));
let unknown = source.replace("Sample_[A,B[", "Unknown");
assert_eq!(qualify(&unknown, &standard).unwrap(), (unknown, 0));
let local = source.replacen("<content>", &format!("<content>{body}"), 1);
assert_eq!(qualify(&local, &standard).unwrap(), (local, 0));
assert!(qualify(&source.replace("sbs://functions.sbs", "sbs://other.sbs"), &standard).is_err());
}
@@ -0,0 +1,11 @@
pub(super) struct ResourceRandom(pub(super) u64);
impl ResourceRandom {
pub(super) fn next(&mut self) -> u64 {
// Xorshift64 uses the standard 13, 7, 17 shift sequence
self.0 ^= self.0 << 13;
self.0 ^= self.0 >> 7;
self.0 ^= self.0 << 17;
self.0
}
}
@@ -0,0 +1,6 @@
pub struct ResourceRecipe {
pub key: &'static str,
pub path: &'static str,
pub extension: &'static str,
pub description: &'static str,
}
@@ -1,47 +1,55 @@
//! Original procedural recipes and attributed library resources; never reads source assets.
use super::Result;
use super::{Result, resource_random::ResourceRandom, resource_recipe::ResourceRecipe};
use image::{DynamicImage, ImageBuffer, ImageFormat};
use std::io::Cursor;
pub struct Recipe {
pub key: &'static str,
pub path: &'static str,
pub extension: &'static str,
pub description: &'static str,
}
const RECIPES: &[Recipe] = &[
Recipe {
const RECIPES: &[ResourceRecipe] = &[
ResourceRecipe {
key: "perm",
path: "resources/perm.png",
extension: "png",
description: "Sixteen independently shuffled 1024-entry permutation rows. Original seeded lookup substitute; external sampling semantics require validation.",
},
Recipe {
ResourceRecipe {
key: "bluenoise_256",
path: "resources/bluenoise_256.png",
extension: "png",
description: "256 by 256 grayscale blue noise for dithering and sampling.",
},
Recipe {
ResourceRecipe {
key: "bluenoise_256_c",
path: "resources/bluenoise_256_c.exr",
extension: "exr",
description: "256 by 256 color blue noise for multichannel sampling.",
},
Recipe { key: "scratched_metal", path: "resources/scratched_metal.png", extension: "png", description: "Scratched metal texture for surface wear." },
Recipe { key: "leather", path: "resources/leather.png", extension: "png", description: "Leather grain texture for surface detail." },
Recipe { key: "scratch_damage", path: "resources/scratch_damage.png", extension: "png", description: "Scratch damage texture for worn surfaces." },
Recipe { key: "shuf4_64_32", path: "resources/shuf4_64_32.exr", extension: "exr", description: "Four-channel 64 by 32 floating-point sampling lookup." },
Recipe { key: "sun", path: "resources/sun.exr", extension: "exr", description: "512 by 16 solar coefficient lookup. All RGBA channels are numeric data." },
Recipe { key: "moss01_bmp", path: "resources/moss01_bmp.bmp", extension: "bmp", description: "Moss texture for coverage and surface detail." },
Recipe {
ResourceRecipe {
key: "scratched_metal",
path: "resources/scratched_metal.png",
extension: "png",
description: "Scratched metal texture for surface wear.",
},
ResourceRecipe { key: "leather", path: "resources/leather.png", extension: "png", description: "Leather grain texture for surface detail." },
ResourceRecipe { key: "scratch_damage", path: "resources/scratch_damage.png", extension: "png", description: "Scratch damage texture for worn surfaces." },
ResourceRecipe {
key: "shuf4_64_32",
path: "resources/shuf4_64_32.exr",
extension: "exr",
description: "Four-channel 64 by 32 floating-point sampling lookup.",
},
ResourceRecipe {
key: "sun",
path: "resources/sun.exr",
extension: "exr",
description: "512 by 16 solar coefficient lookup. All RGBA channels are numeric data.",
},
ResourceRecipe { key: "moss01_bmp", path: "resources/moss01_bmp.bmp", extension: "bmp", description: "Moss texture for coverage and surface detail." },
ResourceRecipe {
key: "courier_new",
path: "resources/LiberationMono-Regular.ttf",
extension: "ttf",
description: "Liberation Mono Regular fallback from craft-fonts for missing point-preview fonts.",
},
Recipe {
ResourceRecipe {
key: "DoorFrame",
path: "resources/DoorFrame.svg",
extension: "svg",
@@ -49,27 +57,18 @@ const RECIPES: &[Recipe] = &[
},
];
pub fn recipe(key: &str) -> Result<&'static Recipe> {
pub fn recipe(key: &str) -> Result<&'static ResourceRecipe> {
RECIPES.iter().find(|recipe| recipe.key == key).ok_or_else(|| format!("no resource recipe for `{key}`").into())
}
struct Random(u64);
impl Random {
fn next(&mut self) -> u64 {
self.0 ^= self.0 << 13;
self.0 ^= self.0 >> 7;
self.0 ^= self.0 << 17;
self.0
}
}
fn permutation(length: usize, seed: u64) -> Vec<usize> {
let mut values: Vec<_> = (0..length).collect();
let mut random = Random(seed);
let mut random = ResourceRandom(seed);
for index in (1..length).rev() {
let other = (random.next() % (index as u64 + 1)) as usize;
values.swap(index, other);
}
values
}
@@ -78,13 +77,16 @@ pub fn load(key: &str) -> Result<Vec<u8>> {
if !notices(key).is_empty() {
return Ok(materialcraft_formats::sbs::library::read(&format!("sbs://{}", recipe.path))?);
}
if key == "DoorFrame" {
return Ok(br#"<svg xmlns="http://www.w3.org/2000/svg" width="512" height="512" viewBox="0 0 512 512"><path fill="white" fill-rule="evenodd" d="M88 480V220C88 38 424 38 424 220V480H370V220C370 109 142 109 142 220V480Z"/></svg>"#.to_vec());
}
// Every other recipe must explicitly select its source rather than invent replacement art.
// Every other recipe must explicitly select its source rather than invent replacement art
if key != "perm" {
return Err(format!("no procedural generator for `{key}`").into());
}
let values: Vec<u16> = (0..16)
.flat_map(|row| permutation(1024, 0x6d6174657269616c ^ (row * 313 + 1)).into_iter().map(|v| ((v as f32 / 1023.0) * 65535.0).round() as u16))
.collect();
@@ -99,7 +101,7 @@ pub fn notices(key: &str) -> &'static [&'static str] {
match key {
"scratch_damage" | "leather" | "scratched_metal" | "moss01_bmp" => &["LICENSE_NICK_ROSE.txt"],
"bluenoise_256" | "bluenoise_256_c" => &["LICENSE_BLUE_NOISE.txt"],
"shuf4_64_32" | "sun" => &["LICENSE_HOSEK_WILKIE.txt", "sun_lookup_readme.txt"],
"shuf4_64_32" | "sun" => &["LICENSE_HOSEK_WILKIE.txt"],
"courier_new" => &["OFL-LiberationMono.txt"],
_ => &[],
}
@@ -20,6 +20,24 @@ fn identities(name: &str) -> BTreeMap<String, String> {
BTreeMap::from([(name.into(), xml::identity(name)), ("functions.sbs".into(), "0".into())])
}
#[test]
fn flat_package_inventory_enforces_the_total_file_limit() {
let directory = (0..1_000)
.map(|suffix| std::env::temp_dir().join(format!("materialcraft-inventory-{}-{suffix}", std::process::id())))
.find(|path| std::fs::create_dir(path).is_ok())
.unwrap();
for index in 0..=2048 {
std::fs::write(directory.join(format!("package-{index}.sbs")), b"").unwrap();
}
let mut files = Vec::new();
let result = inventory(&directory, Path::new(""), &mut files);
std::fs::remove_dir_all(directory).unwrap();
assert!(result.is_err(), "a flat directory must not bypass the total package limit");
assert!(files.len() <= 2048);
}
#[test]
fn conversion_starts_both_versions_at_the_workspace_release() {
let saved = materialcraft_formats::sbs::write_sbs(&materialcraft_core::Package::new("Version check")).unwrap();
@@ -28,6 +46,7 @@ fn conversion_starts_both_versions_at_the_workspace_release() {
for source in [PACKAGE.to_string(), PACKAGE.replacen("<content>", "<updaterVersion v=\"1.1.0.202502\"/><content>", 1)] {
let converted = convert::package(&source, "example.sbs", &identities("example.sbs")).unwrap();
let converted = xml::parse(&converted.xml).unwrap();
assert_eq!(xml::value(converted.root_element(), "updaterVersion"), Some(expected));
assert_eq!(xml::value(converted.root_element(), "formatVersion"), Some(expected));
assert_eq!(converted.root_element().children().filter(|n| n.has_tag_name("updaterVersion")).count(), 1);
@@ -38,18 +57,24 @@ fn conversion_starts_both_versions_at_the_workspace_release() {
fn native_conversion_keeps_unknown_computations_and_replaces_embedded_art_without_decoding_it() {
let owner = "materials/pbr/example.sbs";
let converted = convert::package(PACKAGE, owner, &identities(owner)).unwrap();
assert_eq!(convert::semantics(&xml::parse(PACKAGE).unwrap()), convert::semantics(&xml::parse(&converted.xml).unwrap()));
for retained in ["futureOperation", "future_function", "retain verbatim", "physicalSize v=\"1 2 3\""] {
assert!(converted.xml.contains(retained));
}
for removed in ["Old prose", "Old author", "payload", "Old annotation", "binembedded", "sbs://", "docked"] {
assert!(!converted.xml.contains(removed), "{removed}");
}
assert!(converted.xml.contains("../../resources/DoorFrame.svg"));
assert!(converted.xml.contains("../../functions.sbs"));
let document = xml::parse(&converted.xml).unwrap();
assert_eq!(document.descendants().filter(|n| n.has_tag_name("gpos")).count(), 3);
assert_eq!(converted.xml, convert::package(PACKAGE, owner, &identities(owner)).unwrap().xml);
let available = BTreeSet::from([owner.into(), "functions.sbs".into(), "resources/DoorFrame.svg".into()]);
convert::verify_references(&converted.xml, owner, &available).unwrap();
}
@@ -57,10 +82,13 @@ fn native_conversion_keeps_unknown_computations_and_replaces_embedded_art_withou
#[test]
fn unknown_assets_missing_dependencies_and_broken_graphs_are_errors() {
let ids = identities("example.sbs");
assert!(convert::package(&PACKAGE.replace("DoorFrame", "unknown_asset"), "example.sbs", &ids).is_err());
assert!(convert::package(&PACKAGE.replace("sbs://functions.sbs", "sbs://absent.sbs"), "example.sbs", &ids).is_err());
assert!(convert::package(&PACKAGE.replace("connRef v=\"10\"", "connRef v=\"99\""), "example.sbs", &ids).is_err());
let cycle = PACKAGE.replace("<compImplementation>", "<connections><connection><connRef v=\"20\"/></connection></connections><compImplementation>");
assert!(convert::package(&cycle, "example.sbs", &ids).is_err());
}
@@ -69,9 +97,13 @@ fn references_are_relative_portable_and_confined_to_the_library() {
assert_eq!(xml::relative("materials/pbr/a.sbs", "resources/a.png").unwrap(), "../../resources/a.png");
assert_eq!(xml::relative("materials/pbr/a.sbs", "materials/pbr/b.sbs").unwrap(), "b.sbs");
assert!(xml::relative("a.sbs", "../private.png").is_err());
let converted = convert::package(PACKAGE, "example.sbs", &identities("example.sbs")).unwrap();
assert!(convert::verify_references(&converted.xml, "example.sbs", &BTreeSet::new()).is_err());
let escaped = converted.xml.replace("resources/DoorFrame.svg", "../outside.svg");
assert!(convert::verify_references(&escaped, "example.sbs", &BTreeSet::new()).is_err());
}
@@ -86,6 +118,7 @@ fn resource_names_survive_and_surface_replacements_declare_their_actual_formats(
] {
let xml = format!("<package><content><resource><identifier v=\"{key}\"/><filepath v=\"sbs://resources/{original}\"/></resource></content></package>");
let output = convert::package(&xml, "example.sbs", &identities("example.sbs")).unwrap();
assert!(output.xml.contains(&format!("<filepath v=\"resources/{expected}\"/>")));
assert!(output.xml.contains(&format!("<format v=\"{format}\"/>")));
assert!(convert::package(&xml.replace(original, "different.png"), "example.sbs", &identities("example.sbs")).is_err());
@@ -95,15 +128,21 @@ fn resource_names_survive_and_surface_replacements_declare_their_actual_formats(
#[test]
fn generated_permutation_has_uniform_ranks_and_blue_noise_has_the_expected_shape() {
let permutation = image::load_from_memory(&resources::load("perm").unwrap()).unwrap().into_luma16();
assert_eq!(permutation.dimensions(), (1024, 16));
for row in permutation.as_raw().as_chunks::<1024>().0.iter() {
assert_eq!(row.iter().collect::<BTreeSet<_>>().len(), 1024);
assert_eq!(row.iter().copied().min(), Some(0));
assert_eq!(row.iter().copied().max(), Some(u16::MAX));
}
let noise = resources::load("bluenoise_256").unwrap();
assert_eq!(noise, resources::load("bluenoise_256").unwrap());
let noise = image::load_from_memory(&noise).unwrap().into_luma16();
assert_eq!(noise.dimensions(), (256, 256));
}
@@ -112,10 +151,13 @@ fn library_art_is_decodable_without_reading_source_assets() {
for key in ["DoorFrame", "scratched_metal", "leather", "scratch_damage", "moss01_bmp", "sun", "bluenoise_256_c"] {
let bytes = resources::load(key).unwrap();
let image = materialcraft_formats::decode(&bytes, true).unwrap();
assert!(image.w > 0 && image.h > 0);
assert!(image.data.iter().all(|value| value.is_finite()));
let min = image.data.iter().copied().fold(f32::INFINITY, f32::min);
let max = image.data.iter().copied().fold(f32::NEG_INFINITY, f32::max);
assert!(max > min, "constant replacement {key}");
}
}
@@ -126,20 +168,27 @@ fn curated_resources_keep_the_supplied_bytes_and_required_notices() {
let recipe = resources::recipe(key).unwrap();
if key == "courier_new" && materialcraft_formats::sbs::library::read(&format!("sbs://{}", recipe.path)).is_err() {
assert!(resources::load(key).is_err(), "a build without craft-fonts cannot export its font");
continue;
}
let bytes = resources::load(key).unwrap();
let supplied = materialcraft_formats::sbs::library::read(&format!("sbs://{}", recipe.path)).unwrap();
assert!(bytes == supplied, "converter replaced the supplied bytes for {key}");
assert!(!resources::notices(key).is_empty(), "missing license notice for {key}");
for notice in resources::notices(key) {
assert!(!materialcraft_formats::sbs::library::read(&format!("sbs://resources/{notice}")).unwrap().is_empty());
}
}
if let Ok(font) = resources::load("courier_new") {
assert!(font.starts_with(b"\0\x01\0\0"));
}
let shuffled = image::load_from_memory(&resources::load("shuf4_64_32").unwrap()).unwrap().into_rgba32f();
assert_eq!(shuffled.dimensions(), (64, 32));
assert!(shuffled.as_raw().iter().all(|v| v.is_finite()));
}
@@ -1,9 +1,10 @@
use super::Result;
use super::{Result, xml_edit::XmlEdit};
use roxmltree::{Document, Node};
use std::ops::Range;
pub fn parse(text: &str) -> Result<Document<'_>> {
Ok(Document::parse_with_options(text, roxmltree::ParsingOptions { allow_dtd: false, nodes_limit: 8_000_000 })?)
const MAX_XML_NODES: u32 = 8_000_000;
Ok(Document::parse_with_options(text, roxmltree::ParsingOptions { allow_dtd: false, nodes_limit: MAX_XML_NODES })?)
}
pub fn child<'a, 'b>(node: Node<'a, 'b>, tag: &str) -> Option<Node<'a, 'b>> {
@@ -34,22 +35,18 @@ pub fn title(identifier: &str) -> String {
}
pub fn identity(path: &str) -> String {
// Two FNV-style streams fill the UUID; masks set its version and variant bits
let hash = |seed: u64| path.bytes().fold(seed, |hash, byte| (hash ^ u64::from(byte)).wrapping_mul(0x100000001b3));
let bits = (u128::from(hash(0xcbf29ce484222325)) << 64) | u128::from(hash(0x9e3779b97f4a7c15));
let id = format!("{:032x}", (bits & !(0xf000_u128 << 64) & !(0xc000_u128 << 48)) | (0x4000_u128 << 64) | (0x8000_u128 << 48));
format!("{{{}-{}-{}-{}-{}}}", &id[..8], &id[8..12], &id[12..16], &id[16..20], &id[20..])
}
pub struct Edit {
pub range: Range<usize>,
pub replacement: String,
pub fn set(edits: &mut Vec<XmlEdit>, node: Node<'_, '_>, value: &str) {
edits.push(XmlEdit { range: node.range(), replacement: leaf(node.tag_name().name(), value) });
}
pub fn set(edits: &mut Vec<Edit>, node: Node<'_, '_>, value: &str) {
edits.push(Edit { range: node.range(), replacement: leaf(node.tag_name().name(), value) });
}
pub fn apply(text: &str, mut edits: Vec<Edit>) -> Result<String> {
pub fn apply(text: &str, mut edits: Vec<XmlEdit>) -> Result<String> {
edits.sort_by_key(|edit| (edit.range.start, edit.range.end));
let mut output = String::with_capacity(text.len());
let mut cursor = 0;
@@ -57,6 +54,7 @@ pub fn apply(text: &str, mut edits: Vec<Edit>) -> Result<String> {
if edit.range.start < cursor || edit.range.end < edit.range.start {
return Err("overlapping XML edits".into());
}
output.push_str(text.get(cursor..edit.range.start).ok_or("invalid XML edit boundary")?);
output.push_str(&edit.replacement);
cursor = edit.range.end;
@@ -69,6 +67,7 @@ pub fn relative(owner: &str, target: &str) -> Result<String> {
for path in [owner, target] {
materialcraft_formats::sbs::library::relative_path(&format!("sbs://{path}"))?;
}
let mut from: Vec<_> = owner.split('/').collect();
from.pop();
let to: Vec<_> = target.split('/').collect();
@@ -0,0 +1,6 @@
use std::ops::Range;
pub struct XmlEdit {
pub range: Range<usize>,
pub replacement: String,
}
+2 -1
View File
@@ -14,11 +14,12 @@ pub(crate) fn decode(bytes: &[u8], linear: bool) -> Result<Image> {
dds::ImageViewMut::new(bytemuck::cast_slice_mut(&mut data), size, dds::ColorFormat::RGBA_F32).ok_or_else(|| error("Invalid DDS dimensions".into()))?;
decoder.read_surface(view).map_err(|e| error(e.to_string()))?;
if !linear && decoder.format().precision() != dds::Precision::F32 {
for pixel in data.chunks_exact_mut(4) {
for pixel in data.as_chunks_mut::<4>().0 {
for value in pixel.iter_mut().take(3) {
*value = srgb_to_linear(*value);
}
}
}
Ok(Image { w: size.width, h: size.height, kind: Kind::Color, integer_sampling: false, data: std::sync::Arc::new(data) })
}
+3 -1
View File
@@ -30,6 +30,7 @@ pub fn decode(bytes: &[u8], linear: bool) -> Result<Image> {
image::ImageError::Limits(_) => FormatError::Image(format!("image is larger than {MAX_SIDE} pixels")),
e => FormatError::Image(e.to_string()),
};
let (w, h) = guard(|| open()?.into_dimensions().map_err(image_error), failed)?;
if w == 0 || h == 0 || w > MAX_SIDE || h > MAX_SIDE {
return Err(FormatError::Image(format!("{w}x{h} is outside 1..={MAX_SIDE} pixels")));
@@ -46,12 +47,13 @@ pub fn decode(bytes: &[u8], linear: bool) -> Result<Image> {
let mut data = img.into_rgba32f().into_raw();
if !(float || linear) {
for px in data.chunks_exact_mut(4) {
for px in data.as_chunks_mut::<4>().0 {
for c in px.iter_mut().take(3) {
*c = srgb_to_linear(*c);
}
}
}
Ok(Image { w, h, kind: Kind::Color, integer_sampling: false, data: Arc::new(data) })
}
+26 -2
View File
@@ -13,6 +13,13 @@ use std::io::{Cursor, Read};
pub const EXT: &str = "mcraft";
pub const SBS_ENTRY: &str = "package.sbs";
/// Most members one archive may hold; a header count is checked before any member is read.
const MAX_ENTRIES: usize = 100_000;
/// Largest member read.
const MAX_ENTRY_BYTES: u64 = 256 << 20;
/// Most bytes all members may expand to together. Without a total, many individually allowed
/// members could expand far past what one open should spend (audit: MaterialCraft #1).
const MAX_TOTAL_EXPANDED_BYTES: u64 = 1 << 30;
fn bad(message: impl Into<String>) -> FormatError {
FormatError::BadPackage(message.into())
@@ -76,10 +83,15 @@ fn resolve(owner: &str, reference: &str) -> Result<String> {
pub fn read(bytes: &[u8], identity: &str) -> Result<sbs::SbsRead> {
let mut zip = zip::ZipArchive::new(Cursor::new(bytes)).map_err(|error| bad(error.to_string()))?;
if zip.len() > MAX_ENTRIES {
return Err(bad(format!("archive has more than {MAX_ENTRIES} entries")));
}
let mut files = BTreeMap::new();
let mut seen = BTreeSet::new();
let mut total_expanded_bytes = 0u64;
for index in 0..zip.len() {
let mut entry = zip.by_index(index).map_err(|error| bad(error.to_string()))?;
let entry = zip.by_index(index).map_err(|error| bad(error.to_string()))?;
let name = entry.name().to_string();
let path = name.trim_end_matches('/');
if path.is_empty()
@@ -96,9 +108,19 @@ pub fn read(bytes: &[u8], identity: &str) -> Result<sbs::SbsRead> {
}
let mut content = Vec::new();
entry.read_to_end(&mut content).map_err(|error| bad(error.to_string()))?;
entry.take(MAX_ENTRY_BYTES + 1).read_to_end(&mut content).map_err(|error| bad(error.to_string()))?;
if content.len() as u64 > MAX_ENTRY_BYTES {
return Err(bad(format!("archive entry `{name}` is too large")));
}
total_expanded_bytes = total_expanded_bytes.saturating_add(content.len() as u64);
if total_expanded_bytes > MAX_TOTAL_EXPANDED_BYTES {
return Err(bad("archive expands past its total size limit"));
}
files.insert(name, content);
}
let roots: Vec<_> = files.keys().filter(|name| !name.contains('/') && name.to_ascii_lowercase().ends_with(".sbs")).cloned().collect();
if roots.len() != 1 {
return Err(bad("A MaterialCraft archive must contain exactly one SBS file at its root"));
@@ -139,9 +161,11 @@ pub fn read(bytes: &[u8], identity: &str) -> Result<sbs::SbsRead> {
names.insert(reference, format!("{prefix}{target}"));
pending.push(target);
}
remap(&mut read.pkg, &names);
packages.insert(file, read.pkg);
}
let mut root_package = packages.get(&root).cloned().ok_or_else(|| bad("Missing SBS package"))?;
root_package.dependencies = packages.into_iter().map(|(path, package)| (format!("{prefix}{path}"), std::sync::Arc::new(package))).collect();
Ok(sbs::SbsRead { pkg: root_package, warnings })
+2 -1
View File
@@ -13,7 +13,8 @@ pub(super) fn read_base_params(params: Option<XmlNode>, warnings: &mut Warnings)
let mut base = BaseParameters::default();
for element in params.into_iter().flat_map(|parameters| children_named(parameters, "parameter")) {
let Some(parameter) = child_value(element, "name").and_then(BaseParameter::named) else { continue };
// SBS stores parent as 1 and input as 2, independently of our enum order.
// SBS stores parent as 1 and input as 2, independently of our enum order
let inheritance = match child_value(element, "relativeTo").unwrap_or("2") {
"0" => ParameterInheritance::Absolute,
"1" => ParameterInheritance::RelativeToParent,
+6
View File
@@ -8,6 +8,12 @@ pub(super) const UPDATER_VERSION: &str = env!("CARGO_PKG_VERSION");
pub const SBS_EXT: &str = "sbs";
pub(super) const MAX_WARNINGS: usize = 600;
/// Largest SBS document read (audit: MaterialCraft #2).
pub(super) const MAX_SBS_BYTES: u64 = 256 << 20;
/// Most wires one SBS may carry, across all graphs (audit: MaterialCraft #2).
pub(super) const MAX_WIRES: usize = 1_000_000;
pub(super) const SNIFF_BYTES: usize = 4096;
pub(super) const SELF_DEPENDENCY: &str = "?himself";
pub(super) const FIRST_UID: u64 = 1_000_000_000;
+3
View File
@@ -13,6 +13,7 @@ pub(super) fn decode_embedded(strdata: &str) -> std::result::Result<Vec<u8>, Str
if length as u64 > crate::MAX_RESOURCE {
return Err("embedded resource exceeds the 256 MiB limit".into());
}
let mut out = Vec::new();
// Reading one extra byte detects data that contradicts the declared format length
@@ -49,7 +50,9 @@ mod tests {
assert!(decode_embedded(&base64_engine().encode(&raw)).unwrap_err().contains("declared"));
}
raw[..4].copy_from_slice(&u32::MAX.to_be_bytes());
assert!(decode_embedded(&base64_engine().encode(&raw)).unwrap_err().contains("256 MiB limit"));
}
}
+8
View File
@@ -18,10 +18,12 @@ pub(super) fn read(element: XmlNode<'_, '_>) -> Result<FunctionGraph, String> {
if graph.nodes.len() >= 4096 {
return Err("function exceeds the 4096-node limit".into());
}
let id = child_value(element, "uid").and_then(|value| value.parse().ok()).ok_or("function node has no valid uid")?;
if !ids.insert(id) {
return Err(format!("duplicate function uid {id}"));
}
let op = child_value(element, "function").ok_or("function node has no operation")?;
let value = child_element(element, "funcDatas")
.into_iter()
@@ -46,6 +48,7 @@ pub(super) fn read(element: XmlNode<'_, '_>) -> Result<FunctionGraph, String> {
if names.len() >= 4096 {
return Err("Function instance exceeds the 4096-input limit".into());
}
if let Some(instance) = &mut instance {
instance.inputs.push(materialcraft_core::GraphPort { identifier: port.into(), type_code: 0 });
} else {
@@ -62,6 +65,7 @@ pub(super) fn read(element: XmlNode<'_, '_>) -> Result<FunctionGraph, String> {
return Err(format!("{op}: duplicate input `{port}`"));
}
}
let position = child_element(element, "GUILayout").and_then(|layout| child_value(layout, "gpos")).map(parse_floats).unwrap_or_default();
graph.nodes.push(FunctionNode {
missing: missing.then_some(names),
@@ -84,13 +88,16 @@ pub(super) fn read(element: XmlNode<'_, '_>) -> Result<FunctionGraph, String> {
if graph.output.is_some_and(|id| !ids.contains(&id)) {
return Err("function result references a missing node".into());
}
if graph.nodes.iter().flat_map(|node| node.inputs.iter().flatten()).any(|id| !ids.contains(id)) {
return Err("function connection references a missing node".into());
}
// Earlier files may designate a Dot; keep the graph editable without promoting another node
if graph.output.and_then(|id| graph.node(id)).is_some_and(|node| node.missing.is_none() && !function::can_be_output(&node.op)) {
graph.output = None;
}
Ok(graph)
}
@@ -106,6 +113,7 @@ pub(super) fn write(graph: &FunctionGraph, writer: &mut SbsWriter) -> Result<Str
if graph.nodes.len() > 4096 {
return Err("function exceeds the 4096-node limit".into());
}
let types = function::node_types(graph)?;
let identifiers: BTreeMap<_, _> = graph.nodes.iter().map(|node| (node.id, writer.next_uid())).collect();
let mut output = String::from("<dynamicValue>");
+1 -1
View File
@@ -3,7 +3,7 @@ use materialcraft_nodes::{MAX_IMAGE_INPUTS, image_input_prefix};
pub(super) fn read(definition: &str, identifier: &str) -> Option<String> {
let prefix = image_input_prefix(definition)?;
// Native FX-Map sockets use inputpattern; accept the earlier writer's input spelling too.
// Native FX-Map sockets use inputpattern; accept the earlier writer's input spelling too
let native_identifier = if definition == "fx_map" { identifier.strip_prefix("inputpattern").map(|suffix| format!("input{suffix}")) } else { None };
let identifier = native_identifier.as_deref().unwrap_or(identifier);
let index = if identifier == "input" { 0 } else { identifier.strip_prefix("input:")?.parse::<usize>().ok()? };
+8 -1
View File
@@ -79,16 +79,19 @@ pub fn contains_file(path: &std::path::Path) -> bool {
if let Ok(file) = path.canonicalize() {
return file.starts_with(root);
}
path.parent().and_then(|parent| parent.canonicalize().ok()).is_some_and(|parent| parent.starts_with(root))
}
/// Read a validated library alias from the configured directory or the embedded fallback.
pub fn read(uri: &str) -> Result<Vec<u8>, String> {
let relative = relative_path(uri)?;
// These assets live in craft-fonts, so a source checkout's SBS directory cannot supply them.
// These assets live in craft-fonts, so a source checkout's SBS directory cannot supply them
if is_shared_font_asset(relative) {
return read_embedded(relative).map_err(|error| format!("{error}; build with CRAFT_FONTS_DIR containing Liberation Mono Regular"));
}
#[cfg(not(target_arch = "wasm32"))]
{
let root = root();
@@ -114,11 +117,13 @@ fn read_embedded(relative: &str) -> Result<Vec<u8>, String> {
if *size > 256 << 20 {
return Err("bundled asset exceeds 256 MiB".into());
}
let mut bytes = Vec::new();
flate2::read::DeflateDecoder::new(*compressed).take(*size as u64 + 1).read_to_end(&mut bytes).map_err(|error| error.to_string())?;
if bytes.len() != *size {
return Err("bundled asset has an invalid decompressed length".into());
}
Ok(bytes)
}
@@ -180,8 +185,10 @@ mod tests {
for (relative, _, _) in EMBEDDED_ASSETS {
if is_shared_font_asset(relative) {
assert_eq!(read(&format!("sbs://{relative}")).unwrap(), read_embedded(relative).unwrap());
continue;
}
assert_eq!(read_embedded(relative).unwrap(), std::fs::read(root.join(relative)).unwrap(), "{relative}");
}
}
+7 -8
View File
@@ -5,7 +5,6 @@
use super::{
LayoutFrame, LayoutItem,
constant::MAX_XML_NODES,
filter_map::FILTERS,
xml::{child_element, child_value, children_named, parse_floats},
};
@@ -19,14 +18,11 @@ const MAX_ITEMS: usize = 65_536;
/// The frames of every graph and parameter function in `bytes`, in document order
pub fn read_layout(bytes: &[u8], uri: &str) -> Result<Vec<LayoutFrame>> {
let text = std::str::from_utf8(bytes).map_err(|e| FormatError::BadPackage(format!("the SBS file is not UTF-8 text: {e}")))?;
let text = text.trim_start_matches('\u{feff}');
let options = roxmltree::ParsingOptions { allow_dtd: false, nodes_limit: MAX_XML_NODES };
let doc = roxmltree::Document::parse_with_options(text, options).map_err(|e| FormatError::BadPackage(format!("the SBS file is not valid XML: {e}")))?;
super::SbsDocument::parse(bytes)?.layout(uri)
}
pub(super) fn read_document(doc: &roxmltree::Document<'_>, uri: &str) -> Result<Vec<LayoutFrame>> {
let root = doc.root_element();
if root.tag_name().name() != "package" {
return Err(FormatError::BadPackage("not an SBS package (the root element is not <package>)".into()));
}
let mut dependencies = HashMap::new();
for dependency in child_element(root, "dependencies").into_iter().flat_map(|d| children_named(d, "dependency")) {
@@ -77,6 +73,7 @@ pub fn read_layout(bytes: &[u8], uri: &str) -> Result<Vec<LayoutFrame>> {
items += 1;
}
}
placed.sort_by(|a, b| a.0.cmp(&b.0).then(a.1[1].total_cmp(&b.1[1])).then(a.1[0].total_cmp(&b.1[0])));
let mut grouped: Vec<Vec<LayoutItem>> = vec![Vec::new(); scope_frames.len()];
for (frame, _, item) in placed {
@@ -84,6 +81,7 @@ pub fn read_layout(bytes: &[u8], uri: &str) -> Result<Vec<LayoutFrame>> {
list.push(item);
}
}
for ((title, _), items) in scope_frames.drain(..).zip(grouped) {
if frames.len() >= MAX_FRAMES {
break;
@@ -92,6 +90,7 @@ pub fn read_layout(bytes: &[u8], uri: &str) -> Result<Vec<LayoutFrame>> {
frames.push(LayoutFrame { title, function, items });
}
}
Ok(frames)
}
+17 -5
View File
@@ -13,6 +13,7 @@ fn relative_to(owner: &str, file: &str) -> String {
if file.contains(':') || file.starts_with('/') {
return file;
}
let owner = owner.replace('\\', "/");
owner.rsplit_once('/').map_or_else(|| file.clone(), |(folder, _)| format!("{folder}/{file}"))
}
@@ -25,6 +26,10 @@ fn resource_path(resource: Node<'_, '_>) -> String {
}
pub(super) fn read(reader: &mut SbsReader<'_, '_>, content: Node<'_, '_>, pkg: &mut Package, linked: &dyn Fn(&str) -> Option<Vec<u8>>) {
const MAX_SOURCE_BYTES: usize = 64 * 1024 * 1024;
const MAX_SOURCE_PACKAGES: usize = 64;
const MAX_IMPORTED_RESOURCES: usize = 1024;
let mut needed: BTreeMap<String, BTreeSet<String>> = BTreeMap::new();
for parameter in content
.descendants()
@@ -36,13 +41,15 @@ pub(super) fn read(reader: &mut SbsReader<'_, '_>, content: Node<'_, '_>, pkg: &
needed.entry(file).or_default().insert(path);
}
}
// Dependency resources cannot trigger recursive graph loads or unbounded XML work.
let mut remaining = 64 * 1024 * 1024_usize;
// Dependency resources cannot trigger recursive graph loads or unbounded XML work
let mut remaining = MAX_SOURCE_BYTES;
let mut remaining_data = crate::MAX_RESOURCE as usize;
if needed.len() > 64 {
if needed.len() > MAX_SOURCE_PACKAGES {
reader.warnings.add("Only the first 64 resource dependencies were read");
}
for (file, paths) in needed.into_iter().take(64) {
for (file, paths) in needed.into_iter().take(MAX_SOURCE_PACKAGES) {
let result = (|| -> Result<(), String> {
let bytes = linked(&file).ok_or("source package is unavailable")?;
remaining = remaining.checked_sub(bytes.len()).ok_or("linked resource XML exceeds 64 MiB")?;
@@ -52,14 +59,17 @@ pub(super) fn read(reader: &mut SbsReader<'_, '_>, content: Node<'_, '_>, pkg: &
if !document.root_element().has_tag_name("package") {
return Err("source is not an SBS package".into());
}
for resource in document.descendants().filter(|node| node.has_tag_name("resource")) {
let path = resource_path(resource);
if !paths.contains(&path) || reader.external_resource_names.contains_key(&(file.clone(), path.clone())) {
continue;
}
if reader.external_resource_names.len() >= 1024 {
if reader.external_resource_names.len() >= MAX_IMPORTED_RESOURCES {
return Err("linked resources exceed the 1024-resource limit".into());
}
let identifier = child_value(resource, "identifier").unwrap_or("resource");
let name = reader.read_resource(resource, identifier, &path, pkg, &|reference| {
linked(&relative_to(&file, reference)).filter(|bytes| bytes.len() <= remaining_data)
@@ -68,6 +78,7 @@ pub(super) fn read(reader: &mut SbsReader<'_, '_>, content: Node<'_, '_>, pkg: &
if !imported.source.is_empty() {
imported.source = relative_to(&file, &imported.source);
}
if let Some(left) = remaining_data.checked_sub(imported.bytes.len()) {
remaining_data = left;
} else {
@@ -77,6 +88,7 @@ pub(super) fn read(reader: &mut SbsReader<'_, '_>, content: Node<'_, '_>, pkg: &
}
reader.external_resource_names.insert((file.clone(), path), name);
}
Ok(())
})();
if let Err(error) = result {
+2
View File
@@ -24,6 +24,7 @@ mod parameter_widget;
mod pattern_graph;
mod placeholder;
mod read_node;
mod sbs_document;
mod sbs_read;
mod sbs_reader;
mod sbs_value;
@@ -38,6 +39,7 @@ pub use layout_frame::LayoutFrame;
pub use layout_item::LayoutItem;
pub use library_layout::read_layout;
pub use read::{is_sbs, read_sbs};
pub use sbs_document::SbsDocument;
pub use sbs_read::SbsRead;
pub use write::write_sbs;
+4 -2
View File
@@ -32,14 +32,16 @@ pub(super) fn read(node: &mut Node, implementation: XmlNode<'_, '_>) {
None => [0.0; 4],
};
let midpoint = parameters.get("midpoint").and_then(Value::as_f32).unwrap_or(0.5);
// Legacy gradient arrays use zero for their default linear segments.
// Treating it as a near-zero interpolation fraction clips successive ramps to white.
// Legacy gradient arrays use zero for their default linear segments
// Treating it as a near-zero interpolation fraction clips successive ramps to white
let midpoint = if midpoint == 0.0 { 0.5 } else { midpoint };
gradient.keys.push(GradientKey { pos: position, color, midpoint });
}
_ => {}
}
}
if !curve.points.is_empty() {
curve.points.sort_by(|left, right| left.x.total_cmp(&right.x));
node.params.insert("curve".into(), Value::Curve(curve));
+11 -9
View File
@@ -1,25 +1,23 @@
use super::{
constant::{MAX_GRAPHS, MAX_XML_NODES, SNIFF_BYTES},
constant::{MAX_GRAPHS, SNIFF_BYTES},
placeholder::report_placeholders,
sbs_read::SbsRead,
sbs_reader::SbsReader,
warnings::Warnings,
xml::{child_element, child_value, children_named},
};
use crate::{FormatError, Result};
use crate::Result;
use materialcraft_core::Package;
use roxmltree::Node as XmlNode;
use std::collections::HashMap;
pub fn read_sbs(bytes: &[u8], name: &str, linked: &dyn Fn(&str) -> Option<Vec<u8>>) -> Result<SbsRead> {
let text = std::str::from_utf8(bytes).map_err(|e| FormatError::BadPackage(format!("the SBS file is not UTF-8 text: {e}")))?;
let text = text.trim_start_matches('\u{feff}');
let options = roxmltree::ParsingOptions { allow_dtd: false, nodes_limit: MAX_XML_NODES };
let doc = roxmltree::Document::parse_with_options(text, options).map_err(|e| FormatError::BadPackage(format!("the SBS file is not valid XML: {e}")))?;
super::SbsDocument::parse(bytes)?.read(name, linked)
}
pub(super) fn read_document(doc: &roxmltree::Document<'_>, name: &str, linked: &dyn Fn(&str) -> Option<Vec<u8>>) -> Result<SbsRead> {
let text = doc.input_text();
let root = doc.root_element();
if root.tag_name().name() != "package" {
return Err(FormatError::BadPackage("not an SBS package (the root element is not <package>)".into()));
}
let mut warnings = Warnings::default();
let file_stem = name.rsplit(['/', '\\']).next().unwrap_or(name);
@@ -39,7 +37,9 @@ pub fn read_sbs(bytes: &[u8], name: &str, linked: &dyn Fn(&str) -> Option<Vec<u8
external_resource_names: HashMap::new(),
direct_resource_names: HashMap::new(),
node_count: 0,
wire_count: 0,
};
for dependency in child_element(root, "dependencies").into_iter().flat_map(|d| children_named(d, "dependency")) {
if let (Some(uid), Some(file)) = (child_value(dependency, "uid"), child_value(dependency, "filename")) {
reader.dependencies.insert(uid.to_string(), file.to_string());
@@ -65,6 +65,7 @@ pub fn read_sbs(bytes: &[u8], name: &str, linked: &dyn Fn(&str) -> Option<Vec<u8
reader.graph_names.insert(path.clone(), path.clone());
names.push(name);
}
if graph_elements.len() > MAX_GRAPHS {
reader.warnings.add(format!("only the first {MAX_GRAPHS} graphs were read"));
}
@@ -78,6 +79,7 @@ pub fn read_sbs(bytes: &[u8], name: &str, linked: &dyn Fn(&str) -> Option<Vec<u8
report_placeholders(&pkg, reader.warnings);
Ok(SbsRead { pkg, warnings: warnings.finish() })
}
pub fn is_sbs(bytes: &[u8]) -> bool {
let head = String::from_utf8_lossy(&bytes[..bytes.len().min(SNIFF_BYTES)]);
let mut rest = head.trim_start_matches('\u{feff}');
@@ -35,7 +35,7 @@ pub(super) fn load(
target: &str,
linked: &dyn Fn(&str) -> Option<Vec<u8>>,
) -> String {
// A missing replacement must never fall back to the old container or embedded payload.
// A missing replacement must never fall back to the old container or embedded payload
let bytes =
library::relative_path(target).ok().and_then(|_| linked(target)).filter(|bytes| bytes.len() as u64 <= crate::MAX_RESOURCE).unwrap_or_else(|| {
reader.warnings.add(format!("resource `{path}` could not be loaded: replacement `{target}` is invalid, missing or exceeds 256 MiB"));
@@ -60,15 +60,18 @@ pub(super) fn read_direct(reader: &mut SbsReader<'_, '_>, content: Node<'_, '_>,
if reference.trim().is_empty() || reader.direct_resource_names.contains_key(reference) {
continue;
}
let font = child_value(parameter, "name") == Some("fontdata");
let package_reference = reference.starts_with("pkg://");
if package_reference && (!font || reader.resource_name(reference).is_some()) {
continue;
}
if reader.direct_resource_names.len() >= 1024 {
reader.warnings.add("Only the first 1024 direct resource references were read");
break;
}
if font {
let original = if package_reference { reader.resolve_reference(reference).0 } else { reference.to_string() };
let filename = original.rsplit(['/', '\\']).next().unwrap_or("font");
@@ -81,11 +84,13 @@ pub(super) fn read_direct(reader: &mut SbsReader<'_, '_>, content: Node<'_, '_>,
name
} else {
reader.warnings.add(format!("font `{reference}` and fallback `{}` are unavailable", library::FALLBACK_FONT));
// Without a declaration there is no filename to invent: keep the unresolved SBS reference.
// Without a declaration there is no filename to invent: keep the unresolved SBS reference
if package_reference {
reader.direct_resource_names.insert(reference.into(), reference.into());
continue;
}
store(pkg, identifier, &path, reference, Vec::new())
};
if let Some(resource) = pkg.resources.last() {
@@ -94,6 +99,7 @@ pub(super) fn read_direct(reader: &mut SbsReader<'_, '_>, content: Node<'_, '_>,
reader.direct_resource_names.insert(reference.into(), name);
continue;
}
let Some(target) = target(reference) else { continue };
let name = if let Some(name) = targets.get(&target) {
name.clone()
+69
View File
@@ -0,0 +1,69 @@
use super::{
LayoutFrame, SbsRead,
constant::{MAX_SBS_BYTES, MAX_XML_NODES},
};
use crate::{FormatError, Result};
/// A validated SBS XML document that can supply layout and graph data without reparsing.
pub struct SbsDocument<'input> {
document: roxmltree::Document<'input>,
}
impl<'input> SbsDocument<'input> {
/// Parse borrowed input with the same byte, XML-node, and DTD limits as the SBS reader.
pub fn parse(bytes: &'input [u8]) -> Result<Self> {
if bytes.len() as u64 > MAX_SBS_BYTES {
return Err(FormatError::BadPackage(format!("the SBS file is {} bytes, over the {MAX_SBS_BYTES} byte limit", bytes.len())));
}
let text = std::str::from_utf8(bytes).map_err(|error| FormatError::BadPackage(format!("the SBS file is not UTF-8 text: {error}")))?;
let text = text.trim_start_matches('\u{feff}');
let options = roxmltree::ParsingOptions { allow_dtd: false, nodes_limit: MAX_XML_NODES };
let document = roxmltree::Document::parse_with_options(text, options)
.map_err(|error| FormatError::BadPackage(format!("the SBS file is not valid XML: {error}")))?;
if document.root_element().tag_name().name() != "package" {
return Err(FormatError::BadPackage("not an SBS package (the root element is not <package>)".into()));
}
Ok(Self { document })
}
/// Read the layout independently of whether its frames name library categories.
pub fn layout(&self, uri: &str) -> Result<Vec<LayoutFrame>> {
super::library_layout::read_document(&self.document, uri)
}
/// Read owned package data; the result does not retain the XML document or input bytes.
pub fn read(&self, name: &str, linked: &dyn Fn(&str) -> Option<Vec<u8>>) -> Result<SbsRead> {
super::read::read_document(&self.document, name, linked)
}
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn package_data_outlives_the_shared_document() {
let bytes = b"<package><content><graph><identifier v=\"Example\"/><compNodes/></graph></content></package>".to_vec();
let document = SbsDocument::parse(&bytes).unwrap();
let frames = document.layout("sbs://example.sbs").unwrap();
let read = document.read("example.sbs", &|_| None).unwrap();
drop(document);
drop(bytes);
assert!(frames.is_empty());
assert_eq!(read.pkg.graphs.len(), 1);
assert_eq!(read.pkg.graphs[0].name, "Example");
assert!(read.warnings.is_empty());
}
#[test]
fn malformed_documents_fail_before_either_reader_runs() {
for bytes in [b"<package>".as_slice(), b"<other/>", b"\xff", b"<!DOCTYPE package [<!ENTITY x 'x'>]><package/>"] {
assert!(SbsDocument::parse(bytes).is_err());
}
}
}
+48 -9
View File
@@ -1,7 +1,7 @@
use super::base_parameters::read_base_params;
use super::component_usage;
use super::connection::connect_all;
use super::constant::MAX_NODES;
use super::constant::{MAX_NODES, MAX_WIRES};
use super::constant::{NODE_HALF_SIZE, SELF_DEPENDENCY};
use super::embedded::decode_embedded;
use super::filter_map::{BASE_PARAMS, FilterMap, HANDLED_PARAMS, filter_by_sbs_name};
@@ -30,6 +30,7 @@ pub(super) struct SbsReader<'text, 'warnings> {
pub(super) external_resource_names: HashMap<(String, String), String>,
pub(super) direct_resource_names: HashMap<String, String>,
pub(super) node_count: usize,
pub(super) wire_count: usize,
}
impl<'text> SbsReader<'text, '_> {
@@ -67,6 +68,7 @@ impl<'text> SbsReader<'text, '_> {
if let Some(description) = child_value(item, "desc").filter(|text| !text.is_empty()) {
pkg.folder_descriptions.insert(path.clone(), description.into());
}
if let Some(group_content) = child_element(item, "content") {
pending.push((path, group_content));
}
@@ -93,6 +95,7 @@ impl<'text> SbsReader<'text, '_> {
{
return super::resource_replacement::load(self, pkg, identifier, path, &target, linked);
}
let external = source
.and_then(|source| child_element(source, "externalcopy"))
.and_then(|copy| child_value(copy, "filename"))
@@ -131,11 +134,12 @@ impl<'text> SbsReader<'text, '_> {
}
}
// A supplied font, including cour.ttf or embedded data, always wins. Only absence uses the shared fallback.
// A supplied font, including cour.ttf or embedded data, always wins. Only absence uses the shared fallback
if font && bytes.is_none() {
if let Some(name) = super::resource_replacement::load_font(pkg, identifier, path, linked) {
return name;
}
let why = if why_missing.is_empty() { "it has no font data" } else { &why_missing };
why_missing = format!("{why}; fallback `{}` is unavailable", super::library::FALLBACK_FONT);
}
@@ -147,7 +151,8 @@ impl<'text> SbsReader<'text, '_> {
let ext = if kind == "svg" { "svg".to_string() } else { format };
let mut resource = Resource::new(name.clone(), crate::safe_file_name(&format!("{name}.{ext}")), ResourceBytes::new(data));
// Keep the link as well as the loaded bytes so library saves can retain shared assets.
// Keep the link as well as the loaded bytes so library saves can retain shared assets
resource.source = loaded_source.into();
resource.folder = path.rsplit_once('/').map_or("", |(folder, _)| folder).into();
pkg.resources.push(resource);
@@ -168,6 +173,7 @@ impl<'text> SbsReader<'text, '_> {
self.warnings.add(format!("resource `{path}` could not be loaded: {why}"));
}
}
name
}
@@ -232,6 +238,7 @@ impl<'text> SbsReader<'text, '_> {
if let Some(dynamic) = dynamic {
super::canvas::read_canvas_items(dynamic, &mut graph, self.warnings);
}
return graph;
}
@@ -273,6 +280,7 @@ impl<'text> SbsReader<'text, '_> {
super::canvas::read_canvas_items(element, &mut graph, self.warnings);
graph
}
pub(super) fn read_graph_inputs(&mut self, element: XmlNode, graph: &mut Graph) -> HashMap<String, InputBridge> {
let mut image_inputs = HashMap::new();
for input in child_element(element, "paraminputs").into_iter().flat_map(|p| children_named(p, "paraminput")) {
@@ -289,6 +297,7 @@ impl<'text> SbsReader<'text, '_> {
);
continue;
}
if graph.exposed.iter().any(|e| e.id == id) {
continue;
}
@@ -301,6 +310,7 @@ impl<'text> SbsReader<'text, '_> {
);
continue;
}
let label = child_element(input, "attributes").and_then(|a| child_value(a, "label")).filter(|l| !l.is_empty()).unwrap_or(&id).to_string();
let (mut min, mut max) = (0.0, 1.0);
let widget_options = child_element(input, "defaultWidget").and_then(|w| child_element(w, "options"));
@@ -331,8 +341,10 @@ impl<'text> SbsReader<'text, '_> {
super::parameter_widget::read(input, &mut exposed);
graph.exposed.push(exposed);
}
image_inputs
}
pub(super) fn read_wires(&mut self, graph: &mut Graph, comp_nodes: &[XmlNode], read_nodes: &HashMap<String, ReadNode>) -> Vec<Connection> {
let name = graph.name.clone();
let mut wires = Vec::new();
@@ -368,6 +380,7 @@ impl<'text> SbsReader<'text, '_> {
add_placeholder_input(node, sbs_input);
}
}
sbs_input.to_string()
}
None => {
@@ -377,9 +390,16 @@ impl<'text> SbsReader<'text, '_> {
}
};
if self.wire_count >= MAX_WIRES {
self.warnings.add(format!("graph `{name}`: the {MAX_WIRES} wire limit was reached; later wires were dropped"));
return wires;
}
self.wire_count += 1;
wires.push(Connection { from: from.id, from_port, to: to.id, to_port });
}
}
wires
}
@@ -405,6 +425,7 @@ impl<'text> SbsReader<'text, '_> {
if let Some(node) = node {
fill_placeholder(node, "other", "empty node", "", None, &output_uids, None);
}
return ReadNode { id, index, inputs: HashMap::new(), outputs: Vec::new(), open_inputs: true };
};
@@ -428,8 +449,10 @@ impl<'text> SbsReader<'text, '_> {
if let Some(input) = input {
node.params.insert("default".into(), input.default.clone());
}
node.base = read_base_params(params, self.warnings);
}
ReadNode { id, index, inputs: HashMap::new(), outputs: all_outputs_to("output"), open_inputs: false }
}
"compOutputBridge" => {
@@ -443,6 +466,7 @@ impl<'text> SbsReader<'text, '_> {
node.params.insert("color_mode".into(), Value::Bool(channels == 1));
}
}
let inputs = HashMap::from([("inputNodeOutput".to_string(), "input".to_string())]);
ReadNode { id, index, inputs, outputs: Vec::new(), open_inputs: false }
}
@@ -454,8 +478,8 @@ impl<'text> SbsReader<'text, '_> {
let (id, index, node) = push_node(graph, filter.def, pos);
if let Some(node) = node {
self.read_filter_params(node, filter, params, &graph_name);
// Native files can omit the color switch while declaring a color
// output. Retain that compiled image type instead of the editor preset.
// Preserve a declared color output even when the color switch is omitted
if let [(_, gray)] = output_uids.as_slice()
&& find(filter.def).is_some_and(|definition| {
matches!(definition.param("color_mode").map(|parameter| &parameter.default), Some(Value::Bool(_)))
@@ -474,7 +498,7 @@ impl<'text> SbsReader<'text, '_> {
.flat_map(|parameters| children_named(parameters, "parameter"))
.any(|p| child_value(p, "name") == Some("format"))
{
// A processor defaults to its parent format, even with an image input.
// A processor defaults to its parent format, even with an image input
node.base.format = OutputFormat::U8;
node.base.format_mode = ParameterInheritance::RelativeToParent;
}
@@ -489,6 +513,7 @@ impl<'text> SbsReader<'text, '_> {
}
}
}
let inputs = filter.inputs.iter().map(|(sbs, port)| (sbs.to_string(), port.to_string())).collect();
ReadNode { id, index, inputs, outputs: all_outputs_to("output"), open_inputs: false }
}
@@ -497,6 +522,7 @@ impl<'text> SbsReader<'text, '_> {
if let Some(node) = node {
fill_placeholder(node, "filter", filter_name, "", params.map(|p| &text[p.range()]), &output_uids, None);
}
ReadNode { id, index, inputs: HashMap::new(), outputs: placeholder_outputs(&output_uids, None), open_inputs: true }
}
}
@@ -507,10 +533,12 @@ impl<'text> SbsReader<'text, '_> {
if let Some(node) = node {
fill_placeholder(node, "other", other, "", None, &output_uids, None);
}
ReadNode { id, index, inputs: HashMap::new(), outputs: placeholder_outputs(&output_uids, None), open_inputs: true }
}
}
}
pub(super) fn read_instance(&mut self, graph: &mut Graph, implementation: XmlNode<'_, 'text>, pos: [f32; 2], output_uids: &[(String, bool)]) -> ReadNode {
let graph_name = graph.name.clone();
let (path, dependency) = self.resolve_reference(child_value(implementation, "path").unwrap_or(""));
@@ -577,6 +605,7 @@ impl<'text> SbsReader<'text, '_> {
ReadNode { id, index, inputs: HashMap::new(), outputs: placeholder_outputs(output_uids, Some(&bridgings)), open_inputs: true }
}
pub(super) fn set_param(&mut self, node: &mut Node, id: &str, like: &Value, options: usize, param: XmlNode, graph_name: &str) {
let Some(param_value) = child_element(param, "paramValue") else { return };
@@ -607,11 +636,11 @@ impl<'text> SbsReader<'text, '_> {
let Some(def) = find(filter.def) else { return };
if filter.sbs == "uniform" {
// An omitted SBS output color is black; the editor's new-node preset is gray.
// An omitted SBS output color is black; the editor's new-node preset is gray
node.params.insert("color".into(), Value::Color([0.0, 0.0, 0.0, 1.0]));
}
if filter.sbs == "normal" {
// Native files omit these defaults; the editor's new-node preset differs.
// Native files omit these defaults; the editor's new-node preset differs
node.params.insert("opengl".into(), Value::Bool(false));
node.params.insert("height_alpha".into(), Value::Bool(true));
}
@@ -620,7 +649,8 @@ impl<'text> SbsReader<'text, '_> {
params.into_iter().flat_map(|p| children_named(p, "parameter")).filter(|p| !BASE_PARAMS.contains(&child_value(*p, "name").unwrap_or(""))).collect();
for param in &filter_params {
let param_name = child_value(*param, "name").unwrap_or("");
// Read the earlier writer's alias without emitting it alongside the native parameter.
// Read the earlier writer's alias without emitting it alongside the native parameter
let param_name = match (filter.sbs, param_name) {
("hsl", "luminance") => "luminosity",
("dyngradient", "position") => "coordinate",
@@ -667,10 +697,12 @@ impl<'text> SbsReader<'text, '_> {
if let Some(name) = self.direct_resource_names.get(url) {
return Some(name);
}
let dependency_uid = url.split_once('?').and_then(|(_, query)| query.split('&').find_map(|part| part.strip_prefix("dependency=")));
if dependency_uid.is_some_and(|uid| !self.dependencies.contains_key(uid)) {
return None;
}
let (path, dependency) = self.resolve_reference(url);
match dependency.filter(|file| file != SELF_DEPENDENCY) {
Some(file) => self.external_resource_names.get(&(file, path)),
@@ -682,6 +714,7 @@ impl<'text> SbsReader<'text, '_> {
if let Some(name) = self.resource_name(url) {
return Some(name.clone());
}
let dependency_uid = url.split_once('?').and_then(|(_, query)| query.split('&').find_map(|part| part.strip_prefix("dependency=")));
if let Some(uid) = dependency_uid.filter(|uid| !self.dependencies.contains_key(*uid)) {
self.warnings.add(format!("graph `{graph_name}`: resource reference has unknown dependency `{uid}`"));
@@ -689,21 +722,26 @@ impl<'text> SbsReader<'text, '_> {
let (path, _) = self.resolve_reference(url);
self.warnings.add(format!("graph `{graph_name}`: Node uses resource `{path}`, which is not in the package"));
}
None
}
}
pub(super) fn unique_name(taken: &[String], base: &str) -> String {
let free = |name: &str| !taken.iter().any(|t| t == name);
if free(base) {
return base.to_string();
}
(1..).map(|i| format!("{base}_{i}")).find(|name| free(name)).unwrap_or_else(|| base.to_string())
}
pub(super) fn push_node<'g>(graph: &'g mut Graph, def: &str, pos: [f32; 2]) -> (NodeId, usize, Option<&'g mut Node>) {
let id = graph.add_node(def, pos);
let index = graph.nodes.len() - 1;
(id, index, graph.nodes.last_mut())
}
pub(super) fn read_graph_outputs(element: XmlNode) -> HashMap<String, (String, u32, u32)> {
let mut outputs = HashMap::new();
for output in child_element(element, "graphOutputs").into_iter().flat_map(|o| children_named(o, "graphoutput")) {
@@ -712,5 +750,6 @@ pub(super) fn read_graph_outputs(element: XmlNode) -> HashMap<String, (String, u
let channels = child_value(output, "channels").and_then(|value| value.parse().ok()).unwrap_or(1);
outputs.insert(child_value(output, "uid").unwrap_or("").to_string(), (identifier, usage, channels));
}
outputs
}
+7 -2
View File
@@ -20,6 +20,7 @@ impl SbsValue {
SbsValue::Text(_) => 0.0,
}
}
pub(super) fn int(&self, i: usize) -> i32 {
match self {
SbsValue::Ints(v) => v.get(i).or(v.first()).copied().unwrap_or(0),
@@ -45,6 +46,7 @@ impl SbsValue {
}
}
}
pub(super) fn sbs_constant(holder: XmlNode) -> Option<SbsValue> {
sbs_constant_element(holder.children().find(|child| child.is_element())?)
}
@@ -65,6 +67,7 @@ pub(super) fn sbs_constant_element(element: XmlNode) -> Option<SbsValue> {
None
}
}
pub(super) fn to_param_value(like: &Value, value: &SbsValue, options: usize) -> Option<Value> {
let f = |i| value.float(i);
@@ -73,8 +76,8 @@ pub(super) fn to_param_value(like: &Value, value: &SbsValue, options: usize) ->
Value::Float2(_) => Value::Float2([f(0), f(1)]),
Value::Float3(_) => Value::Float3([f(0), f(1), f(2)]),
Value::Float4(_) => Value::Float4([f(0), f(1), f(2), f(3)]),
// These are graph components, including grayscale data and function inputs.
// Applying a display transfer here changes the arithmetic performed by nodes.
// These are graph components, including grayscale data and function inputs
// Applying a display transfer here changes the arithmetic performed by nodes
Value::Color(_) if value.component_count() >= 3 => {
let alpha = if value.component_count() >= 4 { f(3) } else { 1.0 };
@@ -100,6 +103,7 @@ pub(super) fn to_param_value(like: &Value, value: &SbsValue, options: usize) ->
Value::Curve(_) | Value::Gradient(_) | Value::Function(_) | Value::PatternGraph(_) => return None,
})
}
pub(super) fn to_natural_value(value: &SbsValue) -> Value {
let f = |i| value.float(i);
@@ -118,6 +122,7 @@ pub(super) fn to_natural_value(value: &SbsValue) -> Value {
},
}
}
pub(super) fn exposed_default(code: i64, value: Option<&SbsValue>) -> Value {
use type_code::*;
+6
View File
@@ -36,6 +36,7 @@ impl SbsWriter {
self.external_packages.insert(file.to_string(), uid);
uid
}
pub(super) fn output_ports(pkg: &Package, graph: &Graph, node: &Node) -> Vec<(String, u32)> {
match node.def.as_str() {
"output" => Vec::new(),
@@ -43,6 +44,7 @@ impl SbsWriter {
if let Some(instance) = &node.instance {
return instance.outputs.iter().map(|port| (port.identifier.clone(), port.type_code)).collect();
}
let sub_graph = match node.params.get("graph") {
Some(Value::Text(name)) => pkg.graph_by_name(name).map(|(_, g)| g),
_ => None,
@@ -82,6 +84,7 @@ impl SbsWriter {
for wire in &graph.connections {
wires_into.entry(wire.to).or_default().push(wire);
}
out.push_str("<compNodes>");
for node in &graph.nodes {
let uid = node_uids.get(&node.id).copied().unwrap_or(0);
@@ -141,6 +144,7 @@ impl SbsWriter {
self.write_implementation(out, node, outputs, ids, pkg)?;
out.push_str("</compImplementation></compNode>");
}
out.push_str("</compNodes><baseParameters>");
out.push_str(&param_xml("randomseed", &Value::Int(graph.seed)).unwrap_or_default());
let base = materialcraft_core::BaseParameters {
@@ -166,6 +170,7 @@ impl SbsWriter {
out.push_str("</rootOutputs></root></graph>");
Ok(())
}
pub(super) fn write_graph_inputs(&mut self, out: &mut String, graph: &Graph) -> HashMap<NodeId, u64> {
let mut entries = HashMap::new();
out.push_str("<paraminputs>");
@@ -212,6 +217,7 @@ impl SbsWriter {
out.push_str("</paraminputs>");
entries
}
pub(super) fn write_graph_outputs(&mut self, out: &mut String, graph: &Graph) -> HashMap<NodeId, u64> {
let mut output_uids = HashMap::new();
out.push_str("<graphOutputs>");
+5 -1
View File
@@ -81,6 +81,7 @@ fn first_sbs_save_assigns_the_workspace_version_to_both_fields() {
let unversioned = read_sbs(b"<package><content/></package>", "unversioned.sbs", &no_linked_files).unwrap().pkg;
for package in [Package::new("New file"), unversioned] {
assert!(package.format_version.is_none());
let text = String::from_utf8(write_sbs(&package).unwrap()).unwrap();
let document = roxmltree::Document::parse(&text).unwrap();
for field in ["formatVersion", "updaterVersion"] {
@@ -96,12 +97,15 @@ fn later_sbs_saves_keep_the_original_format_version_and_refresh_the_updater() {
let mut package = read_sbs(old.as_bytes(), "older.sbs", &no_linked_files).unwrap().pkg;
for _ in 0..3 {
assert_eq!(package.format_version.as_deref(), Some(baseline));
// Snapshot/restore and subsequent saves must retain the baseline too.
// Snapshot/restore and subsequent saves must retain the baseline too
package = serde_json::from_slice(&serde_json::to_vec(&package).unwrap()).unwrap();
let bytes = write_sbs(&package).unwrap();
let document = roxmltree::Document::parse(std::str::from_utf8(&bytes).unwrap()).unwrap();
assert_eq!(super::xml::child_value(document.root_element(), "formatVersion"), Some(baseline));
assert_eq!(super::xml::child_value(document.root_element(), "updaterVersion"), Some(env!("CARGO_PKG_VERSION")));
package = read_sbs(&bytes, "saved.sbs", &no_linked_files).unwrap().pkg;
}
}
+34 -5
View File
@@ -5,6 +5,8 @@ use std::collections::HashMap;
pub(super) struct Warnings {
pub(super) lines: Vec<(String, usize)>,
pub(super) line_index: HashMap<String, usize>,
/// Messages over the limit: they are counted but never stored (audit: MaterialCraft #2).
dropped: usize,
}
impl Warnings {
@@ -15,17 +17,44 @@ impl Warnings {
return;
}
if self.lines.len() >= MAX_WARNINGS {
self.dropped += 1;
return;
}
self.line_index.insert(message.clone(), self.lines.len());
self.lines.push((message, 1));
}
pub(super) fn finish(self) -> Vec<String> {
let total = self.lines.len();
let mut out: Vec<String> =
let total_messages = self.lines.len() + self.dropped;
let mut messages: Vec<String> =
self.lines.into_iter().take(MAX_WARNINGS).map(|(line, count)| if count > 1 { format!("{line} (×{count})") } else { line }).collect();
if total > MAX_WARNINGS {
out.push(format!("…and {} more warnings", total - MAX_WARNINGS));
if total_messages > MAX_WARNINGS {
messages.push(format!("…and {} more warnings", total_messages - MAX_WARNINGS));
}
out
messages
}
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn warning_messages_are_capped_as_they_arrive() {
let mut warnings = Warnings::default();
for i in 0..MAX_WARNINGS + 5 {
warnings.add(format!("warning {i}"));
}
assert_eq!(warnings.lines.len(), MAX_WARNINGS);
let messages = warnings.finish();
assert_eq!(messages.len(), MAX_WARNINGS + 1);
assert_eq!(messages.last().unwrap(), "…and 5 more warnings");
}
}
+21 -1
View File
@@ -31,15 +31,20 @@ fn existing_relative_fonts_including_cour_take_precedence_over_the_fallback() {
if path == "empty.ttf" {
return Some(Vec::new());
}
assert_eq!(path, file, "an available font must not read the fallback");
Some(vec![1, 2, 3])
})
.unwrap();
assert!(read.warnings.is_empty(), "{:?}", read.warnings);
assert_eq!(read.pkg.resources.len(), 1);
assert_eq!(read.pkg.resources[0].source, file);
assert_eq!(read.pkg.resources[0].bytes.as_slice(), &[1, 2, 3]);
let reopened = read_sbs(&write_sbs(&read.pkg).unwrap(), "moved.sbs", &|_| None).unwrap();
assert!(reopened.warnings.is_empty(), "{:?}", reopened.warnings);
assert_eq!(reopened.pkg.resources[0].bytes.as_slice(), &[1, 2, 3]);
}
@@ -51,29 +56,38 @@ fn every_missing_font_uses_liberation_mono_and_survives_a_portable_save() {
for file in ["cour.ttf", "fonts/Other.otf", "fonts/Collection.ttc", "sbs://resources/cursor.otf"] {
for xml in [resource(file), direct(file)] {
let read = read_sbs(xml.as_bytes(), "example.sbs", &|path| (path == FALLBACK).then(|| vec![4, 5, 6])).unwrap();
assert!(read.warnings.is_empty(), "{:?}", read.warnings);
assert_eq!(read.pkg.resources.len(), 1);
let font = &read.pkg.resources[0];
assert_eq!(font.source, FALLBACK);
assert_eq!(font.file, "LiberationMono-Regular.ttf");
assert_eq!(font.bytes.as_slice(), &[4, 5, 6]);
if let Some(graph) = read.pkg.graphs.first() {
assert_eq!(graph.nodes[0].params["fontdata"], Value::Text(font.name.clone()));
}
let reopened = read_sbs(&write_sbs(&read.pkg).unwrap(), "moved.sbs", &|_| None).unwrap();
assert!(reopened.warnings.is_empty(), "{:?}", reopened.warnings);
assert_eq!(reopened.pkg.resources[0].bytes.as_slice(), &[4, 5, 6]);
// An unavailable fallback must keep the SBS reference repairable after saving.
// An unavailable fallback must keep the SBS reference repairable after saving
for fallback in [None, Some(Vec::new())] {
let missing = read_sbs(xml.as_bytes(), "example.sbs", &|path| if path == FALLBACK { fallback.clone() } else { None }).unwrap();
let original = &missing.pkg.resources[0];
assert_eq!(original.source, file);
assert_eq!(original.file, file.rsplit(['/', '\\']).next().unwrap());
assert!(original.bytes.is_empty());
assert!(!missing.warnings.is_empty());
let saved = write_sbs(&missing.pkg).unwrap();
let repaired = read_sbs(&saved, "example.sbs", &|path| (path == file).then(|| vec![7, 8, 9])).unwrap();
assert!(repaired.warnings.is_empty(), "{:?}", repaired.warnings);
assert_eq!(repaired.pkg.resources[0].name, original.name);
assert_eq!(repaired.pkg.resources[0].bytes.as_slice(), &[7, 8, 9]);
@@ -91,16 +105,22 @@ fn every_missing_font_uses_liberation_mono_and_survives_a_portable_save() {
_ => None,
})
.unwrap();
assert_eq!(read.pkg.resources.len(), 1);
let font = &read.pkg.resources[0];
assert_eq!(font.name, "Original");
assert_eq!(font.bytes.as_slice(), &[4, 5, 6]);
assert_eq!(read.pkg.graphs[0].nodes[0].params["fontdata"], Value::Text(font.name.clone()));
let reopened = read_sbs(&write_sbs(&read.pkg).unwrap(), "moved.sbs", &|_| None).unwrap();
assert!(reopened.warnings.is_empty(), "{:?}", reopened.warnings);
assert_eq!(reopened.pkg.resources[0].name, "Original");
let missing = read_sbs(xml.as_bytes(), "example.sbs", &|_| None).unwrap();
assert!(missing.pkg.resources.is_empty());
assert_eq!(missing.pkg.graphs[0].nodes[0].params["fontdata"], Value::Text(reference.into()));
}
+10
View File
@@ -38,9 +38,11 @@ fn foreign_resource_uses_its_owner_without_shadowing_local_names_and_round_trips
}
})
.unwrap();
assert!(read.warnings.is_empty(), "{:?}", read.warnings);
assert_eq!(reads.borrow().iter().filter(|path| *path == "nested/source.sbs").count(), 1);
assert_eq!(read.pkg.resources.len(), 2);
let names: Vec<_> = read.pkg.graphs[0]
.nodes
.iter()
@@ -49,14 +51,19 @@ fn foreign_resource_uses_its_owner_without_shadowing_local_names_and_round_trips
_ => panic!("resource should be text"),
})
.collect();
assert_ne!(names[0], names[1]);
assert_eq!(names[1], names[2]);
assert_eq!(read.pkg.resource(names[0]).unwrap().bytes.as_slice(), &[1, 2, 3]);
let foreign = read.pkg.resource(names[1]).unwrap();
assert_eq!(foreign.bytes.as_slice(), &[4, 5, 6]);
assert_eq!(foreign.source, "nested/../resources/tile.png");
let saved = write_sbs(&read.pkg).unwrap();
let reopened = read_sbs(&saved, "relocated.sbs", &|_| None).unwrap();
assert!(reopened.warnings.is_empty(), "{:?}", reopened.warnings);
assert_eq!(reopened.pkg.resource(names[1]).unwrap().bytes.as_slice(), &[4, 5, 6]);
assert_eq!(reopened.pkg.graphs[0].nodes[1].params["resource"], Value::Text(names[1].into()));
@@ -75,12 +82,15 @@ fn missing_or_invalid_foreign_resource_never_falls_back_to_a_local_name() {
}
})
.unwrap();
assert!(!read.warnings.is_empty());
assert!(!read.pkg.graphs[0].nodes[1].params.contains_key("resource"));
assert_eq!(read.pkg.resources.len(), 1);
}
let unknown = package().replace("?dependency=2", "?dependency=99");
let read = read_sbs(unknown.as_bytes(), "caller.sbs", &|path| (path == "resources/local.png").then(|| vec![1])).unwrap();
assert!(read.warnings.iter().any(|warning| warning.contains("unknown dependency")));
assert!(!read.pkg.graphs[0].nodes[1].params.contains_key("resource"));
}
+9
View File
@@ -161,16 +161,21 @@ fn bundled_library_has_complete_bodies_and_current_export_metadata() {
let bytes = include_bytes!("../../../assets/sbs/functions.sbs");
let read = read_sbs(bytes, "functions.sbs", &|_| None).unwrap();
let xml = roxmltree::Document::parse(std::str::from_utf8(bytes).unwrap()).unwrap();
// The SBS schema version is independent of the MaterialCraft release version
for (field, expected) in [("formatVersion", "1.1.0.202401"), ("updaterVersion", env!("CARGO_PKG_VERSION"))] {
let version = xml.root_element().children().find(|element| element.has_tag_name(field)).unwrap().attribute("v").unwrap();
assert_eq!(version, expected, "bundled functions {field}");
}
assert!(read.warnings.is_empty(), "{:?}", read.warnings);
assert_eq!(read.pkg.graphs.len(), 171);
assert_eq!(read.pkg.graphs.iter().flat_map(|graph| graph.functions()).map(|function| function.nodes.len()).sum::<usize>(), 2334);
for graph in &read.pkg.graphs {
let function = graph.function.as_ref().unwrap();
assert!(function.output.is_some(), "{}", graph.sbs_path());
assert!(function.nodes.iter().all(|node| node.missing.is_none()), "{}", graph.sbs_path());
assert!(!graph.attributes.description.is_empty());
@@ -178,15 +183,19 @@ fn bundled_library_has_complete_bodies_and_current_export_metadata() {
assert!(graph.frames.is_empty() && graph.comments.is_empty() && graph.pins.is_empty());
}
let pi = &read.pkg.graph_by_name("Functions/Math/Pi").unwrap().1.attributes;
assert!(pi.category.is_empty());
assert_eq!(pi.tags, "Constant");
let saved = write_sbs(&read.pkg).unwrap();
let reopened = read_sbs(&saved, "functions.sbs", &|_| None).unwrap();
for graph in &read.pkg.graphs {
let reloaded = reopened.pkg.graph_by_name(&graph.sbs_path()).unwrap().1;
assert_eq!(reloaded.attributes.category, graph.attributes.category);
assert_eq!(reloaded.attributes.tags, graph.attributes.tags);
}
assert!(!xml.descendants().any(|element| element.has_tag_name("icon")));
}
+5
View File
@@ -19,19 +19,24 @@ fn native_hurst_constants_and_functions_survive_saving() {
let mut read = read_sbs(xml.as_bytes(), "roughness.sbs", &|_| None).unwrap();
for _ in 0..2 {
assert!(read.warnings.is_empty(), "{:?}", read.warnings);
let node = &read.pkg.graphs[0].nodes[0];
if value.contains("dynamicValue") {
let function = &node.functions["roughness"];
assert_eq!(function.nodes.len(), 1);
assert_eq!(function.output, Some(function.nodes[0].id));
assert_eq!(function.nodes[0].value, Some(Value::Float(0.75)));
} else {
assert_eq!(node.params["roughness"], Value::Float(0.75));
}
let bytes = write_sbs(&read.pkg).unwrap();
let text = std::str::from_utf8(&bytes).unwrap();
assert!(text.contains("<name v=\"hurst\"/>"));
assert!(!text.contains("<name v=\"roughness\"/>"));
read = read_sbs(&bytes, "resaved.sbs", &|_| None).unwrap();
}
}
+6
View File
@@ -7,9 +7,12 @@ fn an_omitted_native_uniform_color_is_black() {
<compImplementation><compFilter><filter v="uniform"/></compFilter></compImplementation>
</compNode></compNodes></graph></content></package>"#;
let read = read_sbs(xml, "noise.sbs", &|_| None).unwrap();
assert_eq!(read.pkg.graphs[0].nodes[0].params["color"], Value::Color([0.0, 0.0, 0.0, 1.0]));
let saved = write_sbs(&read.pkg).unwrap();
let reopened = read_sbs(&saved, "saved.sbs", &|_| None).unwrap();
assert_eq!(reopened.pkg.graphs[0].nodes[0].params["color"], read.pkg.graphs[0].nodes[0].params["color"]);
}
@@ -39,11 +42,14 @@ fn native_gradient_default_midpoints_are_linear_and_components_are_preserved() {
let mut read = read_sbs(xml.as_bytes(), "ramp.sbs", &|_| None).unwrap();
for _ in 0..3 {
assert!(read.warnings.is_empty(), "{:?}", read.warnings);
let Value::Gradient(gradient) = &read.pkg.graphs[0].nodes[0].params["gradient"] else { panic!("missing gradient") };
let end = 0.5;
assert!((gradient.eval(1.0)[0] - end).abs() < 1e-6);
assert!((gradient.eval(0.5)[0] - end * 0.5).abs() < 1e-6);
assert!((gradient.eval(0.5)[3] - if color { 0.4 } else { 1.0 }).abs() < 1e-6);
let saved = write_sbs(&read.pkg).unwrap();
read = read_sbs(&saved, "saved.sbs", &|_| None).unwrap();
}
+3
View File
@@ -18,12 +18,15 @@ fn native_graph_attributes_load_and_survive_sbs_save() {
"tags": "math utility", "hideInLibrary": true, "group": "Utilities", "userData": "pipeline=example"});
assert_eq!(serde_json::to_value(&imported.graphs[0].attributes).unwrap(), expected, "{kind} import");
let bytes = write_sbs(&imported).unwrap();
let saved = read_sbs(&bytes, "saved.sbs", &|_| None).unwrap().pkg;
assert_eq!(saved.graphs[0].name, "Stable_identifier");
assert_eq!(serde_json::to_value(&saved.graphs[0].attributes).unwrap(), expected, "{kind} save");
let xml = String::from_utf8(bytes).unwrap();
assert!(xml.contains("<authorURL v="));
assert!(xml.contains("<userdata v="));
assert!(xml.contains("<hideInLibrary v=\"1\"/>"));
@@ -13,12 +13,17 @@ fn fx_map_render_and_tiling_regions_keep_native_names_and_functions() {
let mut read = read_sbs(xml, "regions.sbs", &|_| None).unwrap();
for _ in 0..2 {
assert!(read.warnings.is_empty(), "{:?}", read.warnings);
let node = &read.pkg.graphs[0].nodes[0];
assert_eq!(node.params["tiling_region"], Value::Float4([0.0, 2.0, 0.0, 1.0]));
assert_eq!(node.functions["render_region"].nodes[0].value, Some(Value::Float4([0.0, 0.5, 0.0, 1.0])));
let bytes = write_sbs(&read.pkg).unwrap();
let saved = std::str::from_utf8(&bytes).unwrap();
assert!(saved.contains("<name v=\"renderroi\"/>") && saved.contains("<name v=\"tilingroi\"/>"));
read = read_sbs(&bytes, "saved.sbs", &|_| None).unwrap();
}
}
@@ -35,15 +40,22 @@ fn native_dynamic_gradient_coordinates_and_orientation_survive_saving() {
let mut read = read_sbs(xml, "gradient.sbs", &|_| None).unwrap();
for _ in 0..2 {
assert!(read.warnings.is_empty(), "{:?}", read.warnings);
let node = &read.pkg.graphs[0].nodes[0];
assert_eq!(node.params["orientation"], materialcraft_core::Value::Enum(1));
assert_eq!(node.functions["position"].nodes[0].value, Some(materialcraft_core::Value::Float(0.25)));
let bytes = write_sbs(&read.pkg).unwrap();
let saved = std::str::from_utf8(&bytes).unwrap();
assert!(saved.contains("<name v=\"coordinate\"/>") && saved.contains("<name v=\"uvselector\"/>"));
read = read_sbs(&bytes, "saved.sbs", &|_| None).unwrap();
}
let legacy = std::str::from_utf8(xml).unwrap().replace("coordinate", "position").replace("uvselector", "orientation");
assert!(read_sbs(legacy.as_bytes(), "legacy.sbs", &|_| None).unwrap().warnings.is_empty());
}
@@ -53,13 +65,18 @@ fn linked_resources_retain_the_successful_path_but_embedded_resources_do_not_lin
<filepath v="resources/example.ttf"/><source><externalcopy><filename v="absent.ttf"/></externalcopy></source>
</resource></content></package>"#;
let read = read_sbs(xml, "example.sbs", &|path| (path == "resources/example.ttf").then(|| vec![1, 2, 3])).unwrap();
assert!(read.warnings.is_empty(), "{:?}", read.warnings);
assert_eq!(read.pkg.resources[0].source, "resources/example.ttf");
let saved = write_sbs(&read.pkg).unwrap();
let saved = String::from_utf8(saved).unwrap();
assert!(saved.contains("<type v=\"font\"/>"));
let saved = saved.replace("<filepath v=\"\"/>", "<filepath v=\"fonts/cour.ttf\"/>");
let embedded = read_sbs(saved.as_bytes(), "saved.sbs", &|_| None).unwrap();
assert!(embedded.warnings.is_empty(), "{:?}", embedded.warnings);
assert!(embedded.pkg.resources[0].source.is_empty());
assert_eq!(embedded.pkg.resources[0].bytes.as_slice(), &[1, 2, 3]);
@@ -78,20 +95,29 @@ fn native_pattern_inputs_and_hsl_luminosity_round_trip() {
</parameters></compFilter></compImplementation></compNode>
</compNodes></graph></content></package>"#;
let read = read_sbs(xml.as_bytes(), "example.sbs", &|_| None).unwrap();
assert!(read.warnings.is_empty(), "{:?}", read.warnings);
let graph = &read.pkg.graphs[0];
assert_eq!(graph.connections.len(), 1);
assert_eq!(graph.connections[0].to_port, "pattern2");
assert_eq!(graph.nodes.iter().find(|node| node.def == "hsl").unwrap().params["lightness"], materialcraft_core::Value::Float(0.75));
let saved = write_sbs(&read.pkg).unwrap();
assert!(std::str::from_utf8(&saved).unwrap().contains("inputpattern:2"));
assert!(std::str::from_utf8(&saved).unwrap().contains("<name v=\"luminosity\"/>"));
assert!(!std::str::from_utf8(&saved).unwrap().contains("<name v=\"luminance\"/>"));
let reopened = read_sbs(&saved, "saved.sbs", &|_| None).unwrap();
assert!(reopened.warnings.is_empty(), "{:?}", reopened.warnings);
assert_eq!(reopened.pkg.graphs[0].connections.len(), 1);
let legacy = xml.replace("luminosity", "luminance").replace("inputpattern:2", "input:2");
let legacy = read_sbs(legacy.as_bytes(), "legacy.sbs", &|_| None).unwrap();
assert!(legacy.warnings.is_empty(), "{:?}", legacy.warnings);
assert_eq!(legacy.pkg.graphs[0].connections.len(), 1);
assert_eq!(legacy.pkg.graphs[0].nodes.iter().find(|node| node.def == "hsl").unwrap().params["lightness"], materialcraft_core::Value::Float(0.75));
+3
View File
@@ -12,10 +12,13 @@ fn archived_sbs_replaces_surface_references_from_the_shared_library() {
let xml = br#"<package><content><resource><identifier v="sun"/><format v="surface"/><filepath v="old/sun.surface"/></resource></content></package>"#;
let archive = zip(&[("package.sbs", xml), ("old/sun.surface", b"old container must never be read")]);
let read = project::read(&archive, "example.mcraft").unwrap();
assert!(read.warnings.is_empty(), "{:?}", read.warnings);
assert_eq!(read.pkg.resources[0].file, "sun.exr");
assert_eq!(read.pkg.resources[0].bytes.as_slice(), materialcraft_formats::sbs::library::read("sbs://resources/sun.exr").unwrap());
let files = ProjectFiles::build(&read.pkg, "saved.sbs", false).unwrap();
assert!(files.files.keys().any(|path| path.ends_with(".exr")));
assert!(!String::from_utf8_lossy(files.xml().unwrap()).contains(".surface"));
}
+17 -1
View File
@@ -25,7 +25,7 @@ fn surface_resources_use_library_replacements_before_embedded_or_external_data()
("sbs://resources/sun.surface", "surface", "sun.exr", "bitmap", "bitmapresourcepath", "resource"),
("../textures/arbitrary.SURFACE", "surface", "arbitrary.exr", "bitmap", "bitmapresourcepath", "resource"),
] {
// Valid old embedded data must not take precedence over the requested replacement.
// Valid old embedded data must not take precedence over the requested replacement
let mut original = materialcraft_core::Package::new("Old");
original.resources.push(materialcraft_core::Resource::new("asset", format!("asset.{format}"), materialcraft_core::ResourceBytes::new(vec![99])));
let embedded = String::from_utf8(write_sbs(&original).unwrap()).unwrap();
@@ -39,15 +39,20 @@ fn surface_resources_use_library_replacements_before_embedded_or_external_data()
(path == target).then(|| vec![1, 2, 3])
})
.unwrap();
assert!(read.warnings.is_empty(), "{:?}", read.warnings);
assert_eq!(reads.borrow().as_slice(), std::slice::from_ref(&target));
let imported = &read.pkg.resources[0];
assert_eq!(imported.file, expected);
assert_eq!(imported.source, target);
assert_eq!(imported.bytes.as_slice(), &[1, 2, 3]);
assert_eq!(read.pkg.graphs[0].nodes[0].params[key], Value::Text(imported.name.clone()));
let saved = write_sbs(&read.pkg).unwrap();
let reopened = read_sbs(&saved, "moved.sbs", &|_| None).unwrap();
assert!(reopened.warnings.is_empty(), "{:?}", reopened.warnings);
assert!(reopened.pkg.resources[0].file.ends_with(expected.rsplit_once('.').unwrap().1));
assert_eq!(reopened.pkg.resources[0].bytes.as_slice(), &[1, 2, 3]);
@@ -60,14 +65,18 @@ fn missing_replacement_keeps_repairable_target_and_never_reads_the_old_file() {
let xml = package(&resource("old/sun.surface", "surface", ""), "");
let read = read_sbs(xml.as_bytes(), "example.sbs", &|path| {
assert_eq!(path, "sbs://resources/sun.exr");
None
})
.unwrap();
assert_eq!(read.pkg.resources[0].file, "sun.exr");
assert_eq!(read.pkg.resources[0].source, "sbs://resources/sun.exr");
assert!(read.pkg.resources[0].bytes.is_empty());
assert!(read.warnings.iter().any(|w| w.contains("sbs://resources/sun.exr")));
let saved = String::from_utf8(write_sbs(&read.pkg).unwrap()).unwrap();
assert!(!saved.contains(".surface"));
assert!(saved.contains("sbs://resources/sun.exr"));
}
@@ -78,13 +87,17 @@ fn external_copy_and_direct_bitmap_references_are_replaced_and_invalid_targets_a
let xml = package(&(resource("", "surface", external) + &node("bitmap", "bitmapresourcepath", "old/shuf4_64_32.surface")), "");
let read = read_sbs(xml.as_bytes(), "example.sbs", &|path| {
assert_eq!(path, "sbs://resources/shuf4_64_32.exr");
Some(vec![8])
})
.unwrap();
assert!(read.warnings.is_empty(), "{:?}", read.warnings);
assert!(read.pkg.resources.iter().all(|r| r.file == "shuf4_64_32.exr" && r.bytes.as_slice() == [8]));
let xml = package(&resource("old/invalid%name.surface", "surface", ""), "");
let read = read_sbs(xml.as_bytes(), "example.sbs", &|_| panic!("invalid target must not be read")).unwrap();
assert!(read.pkg.resources[0].bytes.is_empty());
assert!(read.pkg.resources[0].file.ends_with(".exr"));
assert!(read.warnings.iter().any(|w| w.contains("invalid")));
@@ -101,11 +114,14 @@ fn foreign_font_binding_is_imported_once_and_round_trips() {
_ => None,
})
.unwrap();
assert!(read.warnings.is_empty(), "{:?}", read.warnings);
assert_eq!(read.pkg.resources.len(), 1);
assert_eq!(read.pkg.resources[0].file, "LiberationMono-Regular.ttf");
assert_eq!(read.pkg.graphs[0].nodes[0].params["fontdata"], Value::Text("asset".into()));
let reopened = read_sbs(&write_sbs(&read.pkg).unwrap(), "moved.sbs", &|_| None).unwrap();
assert!(reopened.warnings.is_empty(), "{:?}", reopened.warnings);
assert_eq!(reopened.pkg.graphs[0].nodes[0].params["fontdata"], Value::Text("asset".into()));
assert_eq!(reopened.pkg.resources[0].bytes.as_slice(), &[4, 5, 6]);
+55 -13
View File
@@ -59,6 +59,7 @@ impl Mesh {
if self.positions.is_empty() {
return (Vec3::ZERO, Vec3::ZERO);
}
self.positions.iter().fold((Vec3::splat(f32::MAX), Vec3::splat(f32::MIN)), |(lo, hi), p| (lo.min(*p), hi.max(*p)))
}
@@ -97,33 +98,40 @@ impl Mesh {
self.tangents.len()
)));
}
if !self.colors.is_empty() && self.colors.len() != n {
return Err(GeomError::Invalid(format!("{} vertex colors for {n} vertices", self.colors.len())));
}
if !self.indices.len().is_multiple_of(3) {
return Err(GeomError::Invalid("index count is not a multiple of 3".into()));
}
if self.triangle_count() > MAX_TRIANGLES {
return Err(GeomError::Invalid(format!("mesh has more than {MAX_TRIANGLES} triangles")));
}
if let Some(bad) = self.indices.iter().find(|i| **i as usize >= n) {
return Err(GeomError::Invalid(format!("index {bad} out of range ({n} vertices)")));
}
if self.positions.iter().any(|p| !p.is_finite()) {
return Err(GeomError::Invalid("mesh has non-finite positions".into()));
}
for s in &self.submeshes {
if s.start as usize + s.count as usize > self.indices.len() || s.material >= self.materials.len().max(1) {
return Err(GeomError::Invalid(format!("submesh {} is out of range", s.name)));
}
}
Ok(())
}
/// Area-weighted smooth normals over the index buffer.
pub fn compute_normals(&mut self) {
let mut n = vec![Vec3::ZERO; self.positions.len()];
for t in self.indices.chunks_exact(3) {
for t in self.indices.as_chunks::<3>().0 {
let (a, b, c) = (t[0] as usize, t[1] as usize, t[2] as usize);
let (Some(pa), Some(pb), Some(pc)) = (self.positions.get(a), self.positions.get(b), self.positions.get(c)) else { continue };
let face = (*pb - *pa).cross(*pc - *pa);
@@ -133,6 +141,7 @@ impl Mesh {
}
}
}
self.normals = n.into_iter().map(|v| v.try_normalize().unwrap_or(Vec3::Y)).collect();
}
@@ -145,14 +154,16 @@ impl Mesh {
if self.uvs.len() != self.positions.len() {
self.uvs = vec![Vec2::ZERO; self.positions.len()];
}
// mikktspace crashes (out-of-bounds) when every triangle is degenerate; skip it then.
// mikktspace crashes (out-of-bounds) when every triangle is degenerate; skip it then
let usable = !crate::unwrap::all_triangles_degenerate(self);
let mut g = Mikk { mesh: self };
if !usable || !mikktspace::generate_tangents(&mut g) {
self.simple_tangents();
}
for (t, n) in self.tangents.iter_mut().zip(&self.normals) {
// Orthonormalize against the normal; repair degenerate results.
// Orthonormalize against the normal; repair degenerate results
let v = t.truncate();
let o = (v - *n * n.dot(v)).try_normalize().unwrap_or_else(|| n.any_orthonormal_vector());
*t = o.extend(if t.w < 0.0 { -1.0 } else { 1.0 });
@@ -161,7 +172,7 @@ impl Mesh {
fn simple_tangents(&mut self) {
let mut acc = vec![Vec3::ZERO; self.positions.len()];
for t in self.indices.chunks_exact(3) {
for t in self.indices.as_chunks::<3>().0 {
let (a, b, c) = (t[0] as usize, t[1] as usize, t[2] as usize);
let (Some(pa), Some(pb), Some(pc)) = (self.positions.get(a), self.positions.get(b), self.positions.get(c)) else { continue };
let (Some(ua), Some(ub), Some(uc)) = (self.uvs.get(a), self.uvs.get(b), self.uvs.get(c)) else { continue };
@@ -171,6 +182,7 @@ impl Mesh {
if r.abs() < 1e-12 {
continue;
}
let tan = (e1 * d2.y - e2 * d1.y) / r;
for v in [a, b, c] {
if let Some(x) = acc.get_mut(v) {
@@ -178,6 +190,7 @@ impl Mesh {
}
}
}
self.tangents = acc.into_iter().map(|t| t.try_normalize().unwrap_or(Vec3::X).extend(1.0)).collect();
}
@@ -210,14 +223,16 @@ impl Mesh {
*x = true;
}
}
let mut shared = vec![false; self.positions.len()];
for (_, tri) in self.indices.chunks_exact(3).enumerate().filter(|(t, _)| !is_mine.get(*t).copied().unwrap_or(false)) {
for (_, tri) in self.indices.as_chunks::<3>().0.iter().enumerate().filter(|(t, _)| !is_mine.get(*t).copied().unwrap_or(false)) {
for v in tri {
if let Some(s) = shared.get_mut(*v as usize) {
*s = true;
}
}
}
let mut copies = std::collections::HashMap::new();
for k in mine.iter().flat_map(|t| t * 3..t * 3 + 3) {
let Some(&v) = self.indices.get(k) else { continue };
@@ -226,13 +241,15 @@ impl Mesh {
*i = v;
}
}
// (vertex, centroid of its first triangle), each vertex once.
// (vertex, centroid of its first triangle), each vertex once
let mut verts = Vec::with_capacity(mine.len() * 3);
for t in &mine {
let Some(tri) = self.tri(*t) else { continue };
let centroid = tri.iter().map(|v| self.uvs.get(*v).copied().unwrap_or_default()).sum::<Vec2>() / 3.0;
verts.extend(tri.map(|v| (v, centroid)));
}
verts.sort_by_key(|(v, _)| *v);
verts.dedup_by_key(|(v, _)| *v);
for (v, centroid) in verts {
@@ -266,7 +283,7 @@ impl Mesh {
self.tangents.extend_from_slice(&other.tangents);
self.uvs.extend_from_slice(&other.uvs);
if !(self.colors.is_empty() && other.colors.is_empty()) {
// Vertices of the side without colors become white.
// Vertices of the side without colors become white
self.colors.resize(base as usize, Vec4::ONE);
self.colors.extend(other.colors.iter().copied().chain(std::iter::repeat(Vec4::ONE)).take(other.positions.len()));
}
@@ -304,21 +321,26 @@ impl Mesh {
if has_c {
m.colors.push((self.colors[i] + self.colors[j]) * 0.5);
}
(m.positions.len() - 1) as u32
})
};
for t in self.indices.chunks_exact(3) {
for t in self.indices.as_chunks::<3>().0 {
if t.iter().any(|&i| i as usize >= self.positions.len()) {
continue;
}
let (a, b, c) = (t[0], t[1], t[2]);
let (ab, bc, ca) = (mid(&mut m, a, b), mid(&mut m, b, c), mid(&mut m, c, a));
m.indices.extend_from_slice(&[a, ab, ca, ab, b, bc, ca, bc, c, ab, bc, ca]);
}
for s in &mut m.submeshes {
s.start *= 4;
s.count *= 4;
}
m
}
@@ -368,21 +390,25 @@ impl mikktspace::Geometry for Mikk<'_> {
fn num_faces(&self) -> usize {
self.mesh.indices.len() / 3
}
fn num_vertices_of_face(&self, _face: usize) -> usize {
3
}
fn position(&self, face: usize, vert: usize) -> [f32; 3] {
self.mesh.positions.get(self.vi(face, vert)).copied().unwrap_or_default().to_array()
}
fn normal(&self, face: usize, vert: usize) -> [f32; 3] {
self.mesh.normals.get(self.vi(face, vert)).copied().unwrap_or(Vec3::Y).to_array()
}
fn tex_coord(&self, face: usize, vert: usize) -> [f32; 2] {
// MikkTSpace expects OpenGL texture coordinates (v up); our UVs are glTF (v down). With
// v flipped, the bitangent `cross(N, T) * w` points up the image, as glTF requires.
// Flip glTF V coordinates for MikkTSpace so the bitangent points up
let uv = self.mesh.uvs.get(self.vi(face, vert)).copied().unwrap_or_default();
[uv.x, 1.0 - uv.y]
}
fn set_tangent_encoded(&mut self, tangent: [f32; 4], face: usize, vert: usize) {
let i = self.vi(face, vert);
if let Some(t) = self.mesh.tangents.get_mut(i) {
@@ -422,10 +448,12 @@ mod tests {
#[test]
fn subdivision_shares_edge_midpoints() {
let m = quad().subdivided();
// 4 corners + 5 edge midpoints (the diagonal is shared), 8 triangles.
// 4 corners + 5 edge midpoints (the diagonal is shared), 8 triangles
assert_eq!((m.positions.len(), m.triangle_count()), (9, 8));
assert_eq!(m.uvs.len(), 9);
assert_eq!((m.submeshes[0].start, m.submeshes[0].count), (0, 24));
m.validate().unwrap();
}
@@ -444,40 +472,54 @@ mod tests {
let mut a = quad();
let b = quad();
a.append(&b);
assert_eq!(a.indices[6], 4);
assert_eq!(a.submeshes[1].material, 1);
assert_eq!(a.submeshes[1].start, 6);
a.validate().unwrap();
// Colors on one side only: the other side's vertices become white.
// Colors on one side only: the other side's vertices become white
let mut red = quad();
red.colors = vec![Vec4::new(1.0, 0.0, 0.0, 1.0); 4];
a.append(&red);
assert_eq!(a.colors.len(), 12);
assert_eq!((a.colors[0], a.colors[8]), (Vec4::ONE, Vec4::new(1.0, 0.0, 0.0, 1.0)));
a.append(&quad());
assert_eq!(a.colors[15], Vec4::ONE);
a.validate().unwrap();
a.colors.pop();
assert!(a.validate().is_err());
}
#[test]
fn material_uvs_remap_and_split_shared_vertices() {
// quad(): two triangles sharing an edge; put the second in its own material.
// quad(): two triangles sharing an edge; put the second in its own material
let mut m = quad();
m.materials = vec!["a".into(), "b".into()];
m.submeshes = vec![SubMesh { name: "a".into(), material: 0, start: 0, count: 3 }, SubMesh { name: "b".into(), material: 1, start: 3, count: 3 }];
let before = m.clone();
m.map_material_uvs(1, |uv| uv + Vec2::X);
m.validate().unwrap();
assert_eq!(m.vertex_count(), 6, "the two shared vertices are split");
assert_eq!(&m.indices[..3], &before.indices[..3]);
for k in 0..3 {
let (old, new) = (before.indices[k] as usize, m.indices[k] as usize);
assert_eq!(m.uvs[new], before.uvs[old], "material a keeps its UVs");
let (old, new) = (before.indices[3 + k] as usize, m.indices[3 + k] as usize);
assert_eq!((m.uvs[new], m.positions[new]), (before.uvs[old] + Vec2::X, before.positions[old]));
}
assert_eq!(m.uv_tiles(1).into_iter().collect::<Vec<_>>(), vec![(1, 0)]);
assert_eq!(m.uv_tiles(0).into_iter().collect::<Vec<_>>(), vec![(0, 0)]);
}
+16 -4
View File
@@ -76,6 +76,7 @@ impl Image {
Kind::Gray => vec![v[0]; n],
Kind::Color => v.iter().copied().cycle().take(n * 4).collect(),
};
Image { w, h, kind, integer_sampling: false, data: Arc::new(data) }
}
@@ -163,15 +164,17 @@ impl Image {
if !fx.is_finite() || !fy.is_finite() {
return [0.0, 0.0, 0.0, 1.0];
}
// Huge but finite coordinates would saturate the i64 cast and overflow at `+ 1`.
// Huge but finite coordinates would saturate the i64 cast and overflow at `+ 1`
let (fx, fy) = (fx.clamp(-1.0e9, 1.0e9), fy.clamp(-1.0e9, 1.0e9));
let (x0, y0) = (fx.floor(), fy.floor());
let (mut tx, mut ty) = (fx - x0, fy - y0);
if fixed_weights {
// Function texture reads use eight fractional bits for filter weights.
// Function texture reads use eight fractional bits for filter weights
tx = (tx * 256.0).round() / 256.0;
ty = (ty * 256.0).round() / 256.0;
}
let (x0, y0) = (x0 as i64, y0 as i64);
let a = self.get_axes(x0, y0, tiling);
let b = self.get_axes(x0 + 1, y0, tiling);
@@ -183,6 +186,7 @@ impl Image {
let bot = c[k] + (d[k] - c[k]) * tx;
o[k] = top + (bot - top) * ty;
}
o
}
@@ -200,7 +204,7 @@ impl Image {
pub fn sample_stored(&self, u: f32, v: f32, tile: bool) -> [f32; 4] {
let pixel = self.sample_filtered(u, v, [tile; 2], true);
if self.integer_sampling {
// Eight-bit sources also retain sixteen bits after interpolation.
// Eight-bit sources also retain sixteen bits after interpolation
pixel.map(|value| (value.clamp(0.0, 1.0) * 65535.0).round() / 65535.0)
} else {
pixel
@@ -221,6 +225,7 @@ impl Image {
if !x.is_finite() || !y.is_finite() {
return [0.0, 0.0, 0.0, 1.0];
}
self.get_axes(x as i64, y as i64, tiling)
}
@@ -258,6 +263,7 @@ impl Image {
if w == self.w && h == self.h {
return self.clone();
}
if w <= self.w && h <= self.h && self.w.is_multiple_of(w) && self.h.is_multiple_of(h) {
let (fx, fy) = ((self.w / w) as i64, (self.h / h) as i64);
let inv = 1.0 / (fx * fy) as f32;
@@ -271,9 +277,11 @@ impl Image {
}
}
}
s.map(|v| v * inv)
});
}
Image::generate_uv(w, h, self.kind, |u, v| self.sample(u, v, tile))
}
@@ -310,10 +318,12 @@ impl Image {
s[k] += *v as f64;
}
}
if c == 1 {
let m = (s[0] / n) as f32;
return [m, m, m, 1.0];
}
s.map(|v| (v / n) as f32)
}
@@ -398,9 +408,11 @@ mod tests {
#[test]
fn sample_wraps_and_clamps() {
let img = Image::gray(4, 1, |x, _| x as f32);
assert_eq!(img.get(-1, 0, true)[0], 3.0);
assert_eq!(img.get(-1, 0, false)[0], 0.0);
// Halfway between the last and the (wrapped) first pixel.
// Halfway between the last and the (wrapped) first pixel
assert!((img.sample(1.0, 0.5, true)[0] - 1.5).abs() < 1e-5);
}
+12 -9
View File
@@ -16,29 +16,31 @@ pub(super) fn definition() -> NodeDefinition {
fn gradient_map(context: &CookContext) -> CookResult {
let input = context.gray_or(0, 0.0);
let g = context.gradient("gradient");
let gradient = context.gradient("gradient");
if matches!(context.format, OutputFormat::Inherit | OutputFormat::U16) {
return Ok(vec![integer_map(&input, g, !context.b("color_mode"))]);
return Ok(vec![integer_map(&input, gradient, !context.b("color_mode"))]);
}
let lut: Vec<[f32; 4]> = (0..=512).map(|i| g.eval(i as f32 / 512.0)).collect();
const LOOKUP_INTERVALS: usize = 512;
let lookup_table: Vec<[f32; 4]> = (0..=LOOKUP_INTERVALS).map(|index| gradient.eval(index as f32 / LOOKUP_INTERVALS as f32)).collect();
let gray = !context.b("color_mode");
let look = |v: f32| lut.get((v.clamp(0.0, 1.0) * 512.0).round() as usize).copied().unwrap_or([0.0, 0.0, 0.0, 1.0]);
let look = |v: f32| lookup_table.get((v.clamp(0.0, 1.0) * LOOKUP_INTERVALS as f32).round() as usize).copied().unwrap_or([0.0, 0.0, 0.0, 1.0]);
Ok(vec![if gray { input.map(Kind::Gray, |p| [luminance(look(p[0])), 0.0, 0.0, 0.0]) } else { input.map(Kind::Color, |p| look(p[0])) }])
}
fn integer_map(input: &materialcraft_image::Image, mut gradient: Gradient, gray: bool) -> materialcraft_image::Image {
// The 16-bit path evaluates a 256-entry table in integer position space.
// Keeping normalized positions or sampling a coarse table by nearest value
// shifts narrow transitions and changes the subsequent noise octaves.
// The 16-bit path evaluates a 256-entry table in integer position space
// Coarse nearest sampling shifts transitions and downstream noise octaves
for key in &mut gradient.keys {
key.pos = (key.pos.clamp(0.0, 1.0) * 65535.0).round();
}
let table: Vec<[u32; 4]> =
(0..256).map(|index| gradient.eval(index as f32 * 257.0).map(|value| (value.clamp(0.0, 1.0) * 65535.0).round() as u32)).collect();
let lookup = |value: f32| {
let value = if value.is_finite() { value.clamp(0.0, 1.0) } else { 0.0 };
// The half-texel conversion precedes eight-bit interpolation weights;
// its float rounding is visible at the middle of a 16-bit ramp.
// Half-texel conversion before eight-bit weights preserves ramp rounding
let position = (value * 255.0 + 0.5) - 0.5;
let fixed = (position * 256.0).round() as usize;
let index = fixed / 256;
@@ -47,5 +49,6 @@ fn integer_map(input: &materialcraft_image::Image, mut gradient: Gradient, gray:
let right = table.get(index.saturating_add(1)).copied().unwrap_or(left);
std::array::from_fn(|channel| ((left[channel] * (256 - weight) + right[channel] * weight + 128) >> 8) as f32 / 65535.0)
};
if gray { input.map(Kind::Gray, |pixel| [luminance(lookup(pixel[0])), 0.0, 0.0, 0.0]) } else { input.map(Kind::Color, |pixel| lookup(pixel[0])) }
}
@@ -18,16 +18,16 @@ pub(super) fn definition() -> NodeDefinition {
}
fn grayscale_conversion(context: &CookContext) -> CookResult {
let img = context.any_or(0, Kind::Color, [0.0, 0.0, 0.0, 1.0]).to_color();
let w = [context.f("red"), context.f("green"), context.f("blue"), context.f("alpha")];
let image = context.any_or(0, Kind::Color, [0.0, 0.0, 0.0, 1.0]).to_color();
let weights = [context.f("red"), context.f("green"), context.f("blue"), context.f("alpha")];
let flatten = context.b("flatten_alpha");
let matte = context.f("matte");
Ok(vec![img.map(Kind::Gray, |p| {
// Preserve the fused, ordered dot product through float image storage.
let mut v = p[3].mul_add(w[3], p[2].mul_add(w[2], p[1].mul_add(w[1], p[0] * w[0])));
Ok(vec![image.map(Kind::Gray, |pixel| {
// Preserve the fused, ordered dot product through float image storage
let mut value = pixel[3].mul_add(weights[3], pixel[2].mul_add(weights[2], pixel[1].mul_add(weights[1], pixel[0] * weights[0])));
if flatten {
v = v * p[3] + matte * (1.0 - p[3]);
value = value * pixel[3] + matte * (1.0 - pixel[3]);
}
[v, 0.0, 0.0, 0.0]
[value, 0.0, 0.0, 0.0]
})])
}
+10 -9
View File
@@ -16,14 +16,15 @@ pub(super) fn definition() -> NodeDefinition {
}
fn hsl(context: &CookContext) -> CookResult {
let img = context.any_or(0, Kind::Color, [0.5, 0.5, 0.5, 1.0]).to_color();
// The normalized slider spans one full turn in either direction from its midpoint.
let (dh, sat, lig) = ((context.f("hue") - 0.5) * 2.0, context.f("saturation"), context.f("lightness"));
Ok(vec![img.map(Kind::Color, |p| {
let [h, s, l] = ops::rgb_to_hsl(p);
let s = if sat <= 0.5 { s * sat * 2.0 } else { s + (1.0 - s) * (sat - 0.5) * 2.0 };
let l = if lig <= 0.5 { l * lig * 2.0 } else { l + (1.0 - l) * (lig - 0.5) * 2.0 };
let c = ops::hsl_to_rgb(h + dh, s.clamp(0.0, 1.0), l.clamp(0.0, 1.0));
[c[0], c[1], c[2], p[3]]
let image = context.any_or(0, Kind::Color, [0.5, 0.5, 0.5, 1.0]).to_color();
// The normalized slider spans one full turn in either direction from its midpoint
let (hue_shift, saturation, lightness) = ((context.f("hue") - 0.5) * 2.0, context.f("saturation"), context.f("lightness"));
Ok(vec![image.map(Kind::Color, |pixel| {
let [h, s, l] = ops::rgb_to_hsl(pixel);
let s = if saturation <= 0.5 { s * saturation * 2.0 } else { s + (1.0 - s) * (saturation - 0.5) * 2.0 };
let l = if lightness <= 0.5 { l * lightness * 2.0 } else { l + (1.0 - l) * (lightness - 0.5) * 2.0 };
let color = ops::hsl_to_rgb(h + hue_shift, s.clamp(0.0, 1.0), l.clamp(0.0, 1.0));
[color[0], color[1], color[2], pixel[3]]
})])
}
+10 -3
View File
@@ -32,11 +32,13 @@ fn levels(context: &CookContext) -> CookResult {
pub fn levels_fn(v: f32, in_low: f32, in_mid: f32, in_high: f32, out_low: f32, out_high: f32) -> f32 {
let span = in_high - in_low;
let t = if span.abs() < 1e-6 { if v >= in_high { 1.0 } else { 0.0 } } else { ((v - in_low) / span).clamp(0.0, 1.0) };
// NaN (from an input or a hostile parameter) would otherwise spread through the graph
let t = if t.is_finite() { t } else { 0.0 };
let mid = if in_mid.is_finite() { in_mid.clamp(0.0, 1.0) } else { 0.5 };
// The midpoint slider controls reciprocal gamma below neutral and gamma above it.
// Treating it as an input value that maps to 0.5 changes authored levels curves.
// The midpoint slider controls reciprocal gamma below neutral and gamma above it
// Treating it as an input value that maps to 0.5 changes authored levels curves
let g = if mid <= 0.5 { 1.0 / (9.0 - 16.0 * mid) } else { 16.0 * mid - 7.0 };
out_low + t.powf(g) * (out_high - out_low)
}
@@ -48,20 +50,25 @@ mod levels_tests {
#[test]
fn levels_remaps_the_input_range_midtones_and_output_range() {
let close = |a: f32, b: f32| (a - b).abs() < 1e-5;
// Identity by default
for v in [0.0, 0.25, 0.5, 1.0] {
assert!(close(levels_fn(v, 0.0, 0.5, 1.0, 0.0, 1.0), v));
}
// The input range stretches and clips
assert!(close(levels_fn(0.4, 0.2, 0.5, 0.6, 0.0, 1.0), 0.5));
assert!(close(levels_fn(0.1, 0.2, 0.5, 0.6, 0.0, 1.0), 0.0));
assert!(close(levels_fn(0.9, 0.2, 0.5, 0.6, 0.0, 1.0), 1.0));
// The same gamma curve applies after normalizing the input range.
// The same gamma curve applies after normalizing the input range
assert!(close(levels_fn(0.25, 0.0, 0.25, 1.0, 0.0, 1.0), 0.7578583));
assert!(close(levels_fn(0.3, 0.2, 0.25, 0.6, 0.0, 1.0), 0.7578583));
// The output range, inverted when low > high
assert!(close(levels_fn(0.5, 0.0, 0.5, 1.0, 0.2, 0.6), 0.4));
assert!(close(levels_fn(0.0, 0.0, 0.5, 1.0, 1.0, 0.0), 1.0));
// A collapsed input range is a threshold; NaN never escapes
assert_eq!(levels_fn(0.49, 0.5, 0.5, 0.5, 0.0, 1.0), 0.0);
assert_eq!(levels_fn(0.5, 0.5, 0.5, 0.5, 0.0, 1.0), 1.0);
+13 -13
View File
@@ -15,21 +15,21 @@ pub(super) fn definition() -> NodeDefinition {
}
fn normal(context: &CookContext) -> CookResult {
let h = context.gray_or(0, 0.0);
let k = context.f("intensity") * context.w as f32 / 256.0;
Ok(vec![height_to_normal(&h, k, !context.b("opengl"), context.tile, context.b("height_alpha"))])
let height_map = context.gray_or(0, 0.0);
let intensity = context.f("intensity") * context.w as f32 / 256.0;
Ok(vec![height_to_normal(&height_map, intensity, !context.b("opengl"), context.tile, context.b("height_alpha"))])
}
pub fn height_to_normal(h: &Image, k: f32, directx: bool, tile: bool, alpha_height: bool) -> Image {
Image::generate(h.w, h.h, Kind::Color, |x, y| {
// Backward differences place a wrapped discontinuity on its owning pixel.
let height = h.get_gray(x as i64, y as i64, tile);
let dx = height - h.get_gray(x as i64 - 1, y as i64, tile);
let dy = height - h.get_gray(x as i64, y as i64 - 1, tile);
let (nx, ny, nz) = (-dx * k, dy * k, 1.0);
let l = (nx * nx + ny * ny + nz * nz).sqrt();
pub fn height_to_normal(height_map: &Image, intensity: f32, directx: bool, tile: bool, alpha_height: bool) -> Image {
Image::generate(height_map.w, height_map.h, Kind::Color, |x, y| {
// Backward differences place alpha wrapped discontinuity on its owning pixel
let height = height_map.get_gray(x as i64, y as i64, tile);
let dx = height - height_map.get_gray(x as i64 - 1, y as i64, tile);
let dy = height - height_map.get_gray(x as i64, y as i64 - 1, tile);
let (nx, ny, nz) = (-dx * intensity, dy * intensity, 1.0);
let length = (nx * nx + ny * ny + nz * nz).sqrt();
let ny = if directx { -ny } else { ny };
let a = if alpha_height { height } else { 1.0 };
[nx / l * 0.5 + 0.5, ny / l * 0.5 + 0.5, nz / l * 0.5 + 0.5, a]
let alpha = if alpha_height { height } else { 1.0 };
[nx / length * 0.5 + 0.5, ny / length * 0.5 + 0.5, nz / length * 0.5 + 0.5, alpha]
})
}
+1 -2
View File
@@ -13,8 +13,7 @@ pub(super) fn definition() -> NodeDefinition {
}
fn output(context: &CookContext) -> CookResult {
// Unconnected outputs cook as transparent RGBA, even when their saved
// graph interface declares grayscale. Preserve that declaration separately.
// Keep saved grayscale declarations even when an unconnected output cooks as RGBA
Ok(vec![context.sized(0).unwrap_or_else(|| Image::filled(context.w, context.h, Kind::Color, [0.0; 4]))])
}
+6 -1
View File
@@ -104,7 +104,7 @@ fn glyph(c: char) -> &'static str {
fn strokes(s: &str) -> impl Iterator<Item = Vec<[f32; 2]>> + '_ {
s.split(' ')
.filter(|t| !t.is_empty())
.map(|t| t.as_bytes().chunks_exact(2).map(|p| [p[0].saturating_sub(b'0') as f32, p[1].saturating_sub(b'0') as f32]).collect())
.map(|t| t.as_bytes().as_chunks::<2>().0.iter().map(|p| [p[0].saturating_sub(b'0') as f32, p[1].saturating_sub(b'0') as f32]).collect())
}
/// Longest text drawn (an allocation cap)
@@ -136,6 +136,7 @@ fn layout(context: &CookContext, text: &str) -> Vec<CapsuleSegment> {
let matrix = context.v4("matrix");
let anchor = [(0.5 + position[0]) * context.w as f32, (0.5 + position[1]) * context.h as f32];
let lines: Vec<&str> = text.split('\n').collect();
// The block is centered vertically on the anchor (cap height of the first line to baseline of the last)
let top = -((lines.len().max(1) - 1) as f32 * line_height + 6.0) * 0.5;
let mut segments = Vec::new();
@@ -166,6 +167,7 @@ fn layout(context: &CookContext, text: &str) -> Vec<CapsuleSegment> {
}
}
}
segments
}
@@ -217,9 +219,12 @@ mod tests {
resource: &no_res,
vars: &vars,
};
let img = &text(&context).unwrap()[0];
let lit = img.data.iter().filter(|value| **value > 0.5).count();
assert!(lit > 300, "{lit} pixels");
// Centered: nothing in the top and bottom quarters
assert!((0..256).all(|x| img.get(x, 20, false)[0] == 0.0 && img.get(x, 236, false)[0] == 0.0));
}
Loaded 100 of 158 files, more files were not shown because too many files have changed in this diff. Show more