web: ship the SBS library's bitmaps beside the wasm, not inside it (#263)

* web: ship the SBS library's bitmaps beside the wasm, not inside it

The SBS library import (#219) embeds every file under assets/sbs into
every build, the browser one included. Its resources/ bitmaps are already
compressed PNG/EXR/BMP (14.1 MB, deflate saves 1%), and with them the
release wasm grew from 14.3 MB to 33.3 MB, past the 24 MiB gate in
packaging/web/package.sh that guards Cloudflare Pages' 25 MiB file cap.
The nightly Release dry run has failed on Web (wasm) since.

For wasm32 the formats build script now leaves resources/ out of the
embedded library and lists it as EXTERNAL_ASSETS. index.html copies
assets/sbs/resources to sbs/resources in the dist, and the web app
downloads those files at startup (in parallel, before the session is
created) and hands them to library::install, so library::read serves
them as before. A failed download is logged and surfaces as a missing
asset on the nodes that use it; startup goes on. Native builds still
embed everything.

* web: bound the startup download of the library resources

A response that sends headers and then stalls would keep download_library,
and with it the loading screen, pending for good. One AbortController now
covers every request and body, aborted after two minutes; whatever is still
pending fails, is logged, and startup goes on without it.
This commit is contained in:
FredApps authored and GitHub committed 2026-11-16 22:12:24 +01:00
1 parent c382a005f0
commit e9b270eb94
8 files changed
+144 -7

No files matched your search

Generated
+1
View File
@@ -2983,6 +2983,7 @@ dependencies = [
"js-sys",
"log",
"materialcraft-engine",
"materialcraft-formats",
"materialcraft-ui-egui",
"wasm-bindgen",
"wasm-bindgen-futures",
+3 -1
View File
@@ -21,7 +21,9 @@ log = { workspace = true }
[target.'cfg(target_arch = "wasm32")'.dependencies]
# `wgpu` pulls egui-wgpu's defaults: WebGPU plus the WebGL2 fallback.
eframe = { workspace = true, features = ["default_fonts", "wgpu", "accesskit", "persistence"] }
# The SBS library's `resources/`, which the browser build downloads at startup.
materialcraft-formats = { workspace = true }
wasm-bindgen = "0.2"
wasm-bindgen-futures = "0.4"
js-sys = "0.3"
web-sys = { version = "0.3", features = ["Window", "Document", "Element", "HtmlCanvasElement", "Location", "KeyboardEvent", "EventTarget", "AddEventListenerOptions"] }
web-sys = { version = "0.3", features = ["Window", "Document", "Element", "HtmlCanvasElement", "Location", "KeyboardEvent", "EventTarget", "AddEventListenerOptions", "Response", "RequestInit", "AbortController", "AbortSignal"] }
+2
View File
@@ -6,6 +6,8 @@
<meta name="color-scheme" content="dark">
<title>MaterialCraft</title>
<link data-trunk rel="rust" data-bin="materialcraft-web" data-cargo-profile="wasm-release" data-wasm-opt="z" data-weak-refs />
<!-- The SBS library's bitmaps ship beside the wasm: embedded, they push it past Cloudflare Pages' 25 MiB file cap -->
<link data-trunk rel="copy-dir" href="../../assets/sbs/resources" data-target-path="sbs/resources" />
<style>
html, body { margin: 0; padding: 0; width: 100%; height: 100%; overflow: hidden; overscroll-behavior: none; background: #1c1c1c; }
/* Touch and pen strokes paint and orbit; the browser must not scroll, zoom or select. */
+8
View File
@@ -38,4 +38,12 @@ mod tests {
assert!(html.contains("id=\"materialcraft_canvas\"") && html.contains("data-cargo-profile=\"wasm-release\""));
assert!(!html.contains("<script"), "trunk injects the generated loader; the page has no script of its own");
}
/// The SBS library's bitmaps would push the wasm past the 25 MiB file cap of Cloudflare Pages,
/// so the page ships them beside it, where `web::LIBRARY_URL` downloads them from.
#[test]
fn loading_page_ships_the_library_resources_beside_the_wasm() {
let html = include_str!("../index.html");
assert!(html.contains(r#"<link data-trunk rel="copy-dir" href="../../assets/sbs/resources" data-target-path="sbs/resources" />"#), "{html}");
}
}
+56
View File
@@ -9,6 +9,11 @@ const CANVAS_ID: &str = "materialcraft_canvas";
const LOADING_ID: &str = "materialcraft_loading";
const STATUS_ID: &str = "materialcraft_status";
const PROGRESS_ID: &str = "materialcraft_progress";
/// Where `index.html` copies the SBS library's `resources/`, relative to the page
const LIBRARY_URL: &str = "sbs";
/// How long startup waits for the library files; a response that stalls would otherwise keep the
/// loading screen up for good. Two minutes fits the 14 MB of bitmaps on a slow connection
const LIBRARY_DOWNLOAD_TIMEOUT_MS: i32 = 120_000;
pub fn start() {
eframe::WebLogger::init(log::LevelFilter::Info).ok();
@@ -38,6 +43,8 @@ pub fn start() {
async fn run(canvas: web_sys::HtmlCanvasElement) -> Result<(), String> {
// The WebAssembly has downloaded and compiled when this runs.
step(0.3, "Downloading the material library…").await;
download_library().await;
step(0.5, "Preparing a new package…").await;
let session = materialcraft_engine::new_session();
step(0.8, "Starting the renderer…").await;
@@ -62,6 +69,55 @@ async fn run(canvas: web_sys::HtmlCanvasElement) -> Result<(), String> {
.map_err(|e| e.as_string().unwrap_or_else(|| format!("{e:?}")))
}
/// Fetch the SBS library files that ship beside the wasm (`index.html` copies them to
/// [`LIBRARY_URL`]) before the first cook needs one. A file that fails to download, or is still
/// downloading after [`LIBRARY_DOWNLOAD_TIMEOUT_MS`], is logged and reported by the nodes that use
/// it; it doesn't stop startup.
async fn download_library() {
use materialcraft_formats::sbs::library;
let Some(window) = web_sys::window() else { return };
// One deadline covers every request and body; aborting rejects whatever is still pending
let controller = web_sys::AbortController::new().ok();
let init = web_sys::RequestInit::new();
init.set_signal(controller.as_ref().map(|c| c.signal()).as_ref());
let abort = controller.map(|c| Closure::<dyn FnMut()>::new(move || c.abort()));
let timer = abort
.as_ref()
.and_then(|abort| window.set_timeout_with_callback_and_timeout_and_arguments_0(abort.as_ref().unchecked_ref(), LIBRARY_DOWNLOAD_TIMEOUT_MS).ok());
// Every request starts before the first is awaited, so they download in parallel
let requests: Vec<_> = library::external_assets()
.iter()
.map(|relative| {
let url = String::from(js_sys::encode_uri(&format!("{LIBRARY_URL}/{relative}")));
(*relative, wasm_bindgen_futures::JsFuture::from(window.fetch_with_str_and_init(&url, &init)))
})
.collect();
for (relative, request) in requests {
if let Err(e) = fetch_bytes(request).await.and_then(|bytes| library::install(relative, bytes)) {
log::warn!("couldn't download sbs://{relative}: {e}");
}
}
// The closure must outlive the timer, so cancel the timer before dropping it
if let Some(timer) = timer {
window.clear_timeout_with_handle(timer);
}
drop(abort);
}
async fn fetch_bytes(request: wasm_bindgen_futures::JsFuture) -> Result<Vec<u8>, String> {
let response: web_sys::Response = request.await.and_then(|r| r.dyn_into()).map_err(|e| format!("{e:?}"))?;
if !response.ok() {
return Err(format!("HTTP {}", response.status()));
}
let buffer = response.array_buffer().map_err(|e| format!("{e:?}"))?;
let buffer = wasm_bindgen_futures::JsFuture::from(buffer).await.map_err(|e| format!("{e:?}"))?;
Ok(js_sys::Uint8Array::new(&buffer).to_vec())
}
/// Show a startup step on the loading screen and let the browser paint it before the next
/// (blocking) step runs.
async fn step(progress: f32, status: &str) {
+18 -2
View File
@@ -1,4 +1,5 @@
//! Bundle the same SBS library and original resources used by native disk installations.
//! Bundle the same SBS library and original resources used by native disk installations. The
//! browser build leaves `resources/` out and lists it for download instead.
#![forbid(unsafe_code)]
#![deny(clippy::unwrap_used, clippy::expect_used, clippy::panic, clippy::unimplemented, clippy::todo, clippy::unreachable)]
@@ -55,6 +56,16 @@ fn main() -> Result<()> {
std::fs::create_dir_all(&compressed)?;
let mut files = Vec::new();
scan(&root, Path::new(""), &mut files)?;
// Web hosts cap single files (Cloudflare Pages at 25 MiB) and the bitmaps would push the wasm past it
// The browser build downloads `resources/` from beside the wasm instead
let browser = std::env::var("CARGO_CFG_TARGET_ARCH").is_ok_and(|arch| arch == "wasm32");
let (downloaded, files): (Vec<_>, Vec<_>) = files.into_iter().partition(|relative| browser && relative.starts_with("resources"));
let mut external = Vec::new();
for relative in &downloaded {
external.push(relative.to_str().ok_or("non-UTF-8 asset path")?.replace('\\', "/"));
}
external.sort();
let mut files: Vec<_> = files.into_iter().map(|relative| (relative.clone(), root.join(relative))).collect();
files.extend(sbs_font::assets()?);
files.sort_by_key(|(relative, _)| relative.to_string_lossy().replace('\\', "/"));
@@ -78,6 +89,11 @@ fn main() -> Result<()> {
}
assets.push_str("];\n");
packages.push_str("];\n");
std::fs::write(out.join("sbs_assets.rs"), format!("{assets}{packages}"))?;
let mut downloads = String::from("const EXTERNAL_ASSETS: &[&str] = &[\n");
for name in &external {
downloads.push_str(&format!("{name:?},\n"));
}
downloads.push_str("];\n");
std::fs::write(out.join("sbs_assets.rs"), format!("{assets}{packages}{downloads}"))?;
Ok(())
}
+51 -2
View File
@@ -109,12 +109,52 @@ pub fn read(uri: &str) -> Result<Vec<u8>, String> {
read_embedded(relative)
}
/// Library files that the browser build downloads from beside the wasm instead of embedding
/// them, as paths below the library root (`resources/…`). Empty on native builds.
pub fn external_assets() -> &'static [&'static str] {
EXTERNAL_ASSETS
}
/// The largest library asset read into memory, embedded or downloaded
const MAX_ASSET_BYTES: usize = 256 << 20;
/// The downloaded [`external_assets`], filled once by the web app's startup.
static DOWNLOADED: std::sync::RwLock<std::collections::BTreeMap<String, Vec<u8>>> = std::sync::RwLock::new(std::collections::BTreeMap::new());
/// Supply the bytes of one downloaded [`external_assets`] file.
pub fn install(relative: &str, bytes: Vec<u8>) -> Result<(), String> {
if !EXTERNAL_ASSETS.contains(&relative) {
return Err(format!("sbs://{relative} is not a downloadable library asset"));
}
if bytes.len() > MAX_ASSET_BYTES {
return Err("downloaded library asset exceeds 256 MiB".into());
}
DOWNLOADED.write().unwrap_or_else(std::sync::PoisonError::into_inner).insert(relative.into(), bytes);
Ok(())
}
fn read_downloaded(relative: &str) -> Result<Vec<u8>, String> {
if let Some(bytes) = DOWNLOADED.read().unwrap_or_else(std::sync::PoisonError::into_inner).get(relative) {
return Ok(bytes.clone());
}
if EXTERNAL_ASSETS.contains(&relative) {
return Err(format!("Library asset sbs://{relative} has not downloaded"));
}
Err(format!("Missing library asset sbs://{relative}"))
}
/// Compressed assets keep installed and browser builds independent of the source checkout.
fn read_embedded(relative: &str) -> Result<Vec<u8>, String> {
use std::io::Read;
let index = EMBEDDED_ASSETS.binary_search_by_key(&relative, |(name, _, _)| *name).map_err(|_| format!("Missing library asset sbs://{relative}"))?;
let Ok(index) = EMBEDDED_ASSETS.binary_search_by_key(&relative, |(name, _, _)| *name) else {
return read_downloaded(relative);
};
let (_, compressed, size) = EMBEDDED_ASSETS.get(index).ok_or("invalid bundled asset index")?;
if *size > 256 << 20 {
if *size > MAX_ASSET_BYTES {
return Err("bundled asset exceeds 256 MiB".into());
}
@@ -193,6 +233,15 @@ mod tests {
}
}
/// Only the listed browser downloads are accepted, and native builds embed everything instead.
#[test]
fn only_listed_library_assets_can_be_installed() {
assert!(install("functions.sbs", b"<package/>".to_vec()).is_err());
assert!(install("../outside.png", Vec::new()).is_err());
assert!(external_assets().is_empty(), "native builds embed every library asset");
assert!(read_embedded("resources/not-in-the-library.png").unwrap_err().contains("Missing library asset"));
}
/// Discovered packages have unique valid aliases and the embedded fallback is present.
#[test]
fn the_library_lists_every_package_once_and_bundles_a_fallback() {
+5 -2
View File
@@ -8,20 +8,23 @@ static site in `materialcraft-web-<version>/`:
| `index.html` | The page. It loads everything through relative URLs. |
| `materialcraft-web-<hash>.js` | wasm-bindgen glue (generated by trunk, ES module) |
| `materialcraft-web-<hash>_bg.wasm` | The app (built with the `wasm-release` Cargo profile) |
| `sbs/resources/` | The SBS library's bitmaps and their licences, downloaded when the app starts |
| `_headers`, `.htaccess` | Sample header rules for Netlify/Cloudflare Pages and Apache |
There is no server-side code and no hand-written JavaScript: trunk generates the glue from the
Rust crate `apps/materialcraft-web`. Upload the folder's contents anywhere that serves static files.
`packaging/web/package.sh` fails the build when a `.wasm` grows past 24 MiB, because Cloudflare
Pages and Workers reject any file over 25 MiB.
Pages and Workers reject any file over 25 MiB. That is why the SBS library's bitmaps ship in
`sbs/resources/` instead of inside the wasm.
## Any path works
All URLs in `index.html` are relative (`public_url = "./"` in `apps/materialcraft-web/Trunk.toml`),
so the site works at a domain root (`https://example.com/`), under a prefix
(`https://example.com/tools/materialcraft/`) and from a CDN bucket. The asset names carry a content
hash, so they can be cached forever. Only `index.html` needs revalidation.
hash, so they can be cached forever. `index.html` and the files in `sbs/` have no hash and need
revalidation.
## Required server settings