mirror of
https://github.com/storytold/wgpu.git
synced 2026-11-21 15:00:57 +01:00
Convert wiki and api-specs into documentation modules (#9779)
This commit is contained in:
1 parent
855ead0745
commit
fae070bf21
68 files changed
+3461
-273
No files matched your search
@@ -0,0 +1,3 @@
|
||||
# `wgpu` Contributor Documentation
|
||||
|
||||
This folder contains documentation for contributors of wgpu. If you're looking for user facing documentation, look at <https://wgpu.rs/doc/wgpu/documentation/index.html> for documentation on `trunk` and <https://docs.rs/wgpu/latest/wgpu/documentation/index.html> for the latest release.
|
||||
@@ -1,468 +0,0 @@
|
||||
# Cooperative Matrix Extensions
|
||||
|
||||
🧪Experimental🧪
|
||||
|
||||
`wgpu` supports an experimental cooperative matrix feature when `Features::EXPERIMENTAL_COOPERATIVE_MATRIX` is enabled.
|
||||
This exposes hardware-accelerated matrix multiply-accumulate (MMA) operations (for example, NVIDIA tensor cores,
|
||||
Metal SIMD-group matrices, and Vulkan `VK_KHR_cooperative_matrix`).
|
||||
|
||||
**Note**: The features documented here may have bugs and are subject to breaking changes. The API and shader
|
||||
semantics are expected to evolve. Please refer to the GitHub issue tracker for the latest status and discussions.
|
||||
|
||||
---
|
||||
|
||||
## Overview
|
||||
|
||||
Cooperative matrices allow a **workgroup** (or equivalent execution group) to collectively:
|
||||
|
||||
- load small matrix tiles from memory,
|
||||
- perform matrix multiply-accumulate operations on those tiles, and
|
||||
- store the results back to memory.
|
||||
|
||||
Conceptually, this is specialized hardware that evaluates:
|
||||
|
||||
> `C = A * B + C`
|
||||
|
||||
for relatively small tiles, but at very high throughput compared to composing the same operation from
|
||||
scalar/vector instructions.
|
||||
|
||||
Cooperative matrix operations are most useful in workloads such as:
|
||||
|
||||
- machine learning and inference,
|
||||
- dense linear algebra and scientific computing,
|
||||
- image processing, filtering, and transforms.
|
||||
|
||||
The cooperative nature means that all lanes in the cooperating execution group must participate in
|
||||
the operations; individual invocations cannot diverge.
|
||||
|
||||
Typical example:
|
||||
|
||||
- `A` is an M×K matrix.
|
||||
- `B` is a K×N matrix.
|
||||
- `C` is an M×N matrix, acting as the accumulator and result.
|
||||
|
||||
---
|
||||
|
||||
## Querying hardware support (host side)
|
||||
|
||||
Before using cooperative matrices in shaders, you must query what configurations your hardware and backend support.
|
||||
|
||||
On the `Adapter`, `wgpu` exposes:
|
||||
|
||||
- `Adapter::cooperative_matrix_properties() -> Vec<CooperativeMatrixProperties>`
|
||||
|
||||
Each `CooperativeMatrixProperties` describes a single supported configuration. Fields are:
|
||||
|
||||
- `m_size`: height of matrices A and C (type: `naga::CooperativeSize`)
|
||||
- `n_size`: width of matrices B and C (type: `naga::CooperativeSize`)
|
||||
- `k_size`: shared inner dimension of A and B (type: `naga::CooperativeSize`)
|
||||
- `ab_type`: scalar element type for A and B (type: `naga::Scalar`)
|
||||
- `cr_type`: scalar element type for C and the result (type: `naga::Scalar`)
|
||||
- `saturating_accumulation`: `bool` indicating whether overflow clamping on accumulation
|
||||
is supported for this configuration
|
||||
|
||||
Example usage:
|
||||
|
||||
```/dev/null/cooperative-matrix-host.rs#L1-40
|
||||
let coop_props = adapter.cooperative_matrix_properties();
|
||||
for prop in &coop_props {
|
||||
println!(
|
||||
"{:?}x{:?}x{:?} - AB: {:?}, CR: {:?}, saturating: {}",
|
||||
prop.m_size, prop.n_size, prop.k_size,
|
||||
prop.ab_type, prop.cr_type,
|
||||
prop.saturating_accumulation,
|
||||
);
|
||||
}
|
||||
```
|
||||
|
||||
You **must**:
|
||||
|
||||
1. Enable `Features::EXPERIMENTAL_COOPERATIVE_MATRIX` on the `Device`.
|
||||
2. Query `adapter.cooperative_matrix_properties()` and ensure that the configuration(s) you intend
|
||||
to use in WGSL are actually available on the running adapter/backend.
|
||||
3. Treat the sizes and types as a contract between your shaders and the underlying hardware implementation.
|
||||
Using unsupported configurations is an error.
|
||||
|
||||
---
|
||||
|
||||
## Feature and backend requirements
|
||||
|
||||
### `wgpu` feature
|
||||
|
||||
- Using cooperative matrices requires enabling:
|
||||
- `Features::EXPERIMENTAL_COOPERATIVE_MATRIX`
|
||||
|
||||
This feature may be restricted to certain backends and hardware.
|
||||
|
||||
### Hardware / backend notes
|
||||
|
||||
These are general guidelines, not a complete compatibility matrix:
|
||||
|
||||
- **Metal**:
|
||||
- Requires Apple7+ (A14) or Mac2+ (M1) GPU with MSL 2.3+.
|
||||
- Strong support for 8×8 `f32`, 8×8 `f16`, and mixed-precision modes (e.g. `f16` A/B and `f32` accumulator C).
|
||||
- Implementation is based on SIMD-group matrix operations.
|
||||
|
||||
- **Vulkan**:
|
||||
- Requires the `VK_KHR_cooperative_matrix` extension.
|
||||
- Many NVIDIA and AMD GPUs support `f16` at 16×16 tile sizes and similar.
|
||||
- 8×8 `f32` support is hardware-dependent.
|
||||
- Exact configurations are enumerated by `Adapter::cooperative_matrix_properties()`.
|
||||
|
||||
- **Other backends**:
|
||||
- May not support cooperative matrices at all. In that case the feature will not be exposed, and
|
||||
`adapter.cooperative_matrix_properties()` will return an empty list.
|
||||
|
||||
> Always treat the properties returned at runtime as the source of truth.
|
||||
|
||||
---
|
||||
|
||||
## `wgpu` API surface
|
||||
|
||||
This section summarizes the host-side API elements related to cooperative matrices.
|
||||
(For exact signatures and details, refer to the Rust documentation.)
|
||||
|
||||
### Adapter
|
||||
|
||||
- `Adapter::cooperative_matrix_properties() -> Vec<CooperativeMatrixProperties>`
|
||||
|
||||
Returns all cooperative matrix configurations supported by the adapter/backend.
|
||||
|
||||
### Structures
|
||||
|
||||
- `CooperativeMatrixProperties`
|
||||
- `m_size: naga::CooperativeSize`
|
||||
- `n_size: naga::CooperativeSize`
|
||||
- `k_size: naga::CooperativeSize`
|
||||
- `ab_type: naga::Scalar`
|
||||
- `cr_type: naga::Scalar`
|
||||
- `saturating_accumulation: bool`
|
||||
|
||||
The `naga` types (`CooperativeSize`, `Scalar`) are part of the shader translation layer and
|
||||
determine the legal WGSL/cooperative matrix combinations.
|
||||
|
||||
There are currently no dedicated `wgpu` buffer or texture types for cooperative matrices; they are
|
||||
expressed in WGSL as special value types accessed via pointers into ordinary `var<storage>` /
|
||||
`var<workgroup>` / `var<private>` / etc.
|
||||
|
||||
---
|
||||
|
||||
## WGSL extension specification
|
||||
|
||||
Cooperative matrices are enabled and accessed via WGSL extensions. The exact extension spelling
|
||||
may change; the details below describe the intended semantics.
|
||||
|
||||
### Enabling cooperative matrices in WGSL
|
||||
|
||||
Any WGSL program using cooperative matrices must declare an extension at the top of the shader, for example:
|
||||
|
||||
```/dev/null/example.wgsl#L1-3
|
||||
enable wgpu_cooperative_matrix;
|
||||
```
|
||||
|
||||
The shader is invalid if any cooperative matrix types or builtins are used without enabling this extension.
|
||||
|
||||
### Cooperative matrix types
|
||||
|
||||
A cooperative matrix is a value type parameterized by:
|
||||
|
||||
- tile size (M×N),
|
||||
- scalar element type `T`, and
|
||||
- role `R` indicating how the matrix participates in the multiply-accumulate:
|
||||
- `A`: left operand
|
||||
- `B`: right operand
|
||||
- `C`: accumulator / result
|
||||
|
||||
Conceptually:
|
||||
|
||||
```/dev/null/example.wgsl#L1-8
|
||||
// A: MxK, B: KxN, C: MxN
|
||||
type coop_matMxN<T, A>;
|
||||
type coop_matMxN<T, B>;
|
||||
type coop_matMxN<T, C>;
|
||||
```
|
||||
|
||||
Concrete examples (sizes and types must match a supported configuration from
|
||||
`Adapter::cooperative_matrix_properties`):
|
||||
|
||||
```/dev/null/example.wgsl#L10-20
|
||||
// 8x8 single-precision tiles
|
||||
alias CoopMatA = coop_mat8x8<f32, A>;
|
||||
alias CoopMatB = coop_mat8x8<f32, B>;
|
||||
alias CoopMatC = coop_mat8x8<f32, C>;
|
||||
|
||||
// 16x16 half-precision inputs, 16x16 f32 accumulator (mixed precision)
|
||||
alias CoopMat16x16A = coop_mat16x16<f16, A>;
|
||||
alias CoopMat16x16B = coop_mat16x16<f16, B>;
|
||||
alias CoopMat16x16C = coop_mat16x16<f32, C>;
|
||||
```
|
||||
|
||||
The actual set of legal `(M, N, T, R)` combinations is defined by the cooperative matrix
|
||||
properties returned at runtime; shaders must not use arbitrary combinations.
|
||||
|
||||
### Roles and semantics
|
||||
|
||||
- `A` role:
|
||||
- Treated as the left operand in the multiplication. Has shape M×K.
|
||||
- Participates as `A` in `A * B + C`.
|
||||
|
||||
- `B` role:
|
||||
- Treated as the right operand in the multiplication. Has shape K×N.
|
||||
- Participates as `B` in `A * B + C`.
|
||||
|
||||
- `C` role:
|
||||
- Treated as accumulator and result. Has shape M×N.
|
||||
- Participates as `C` in `A * B + C`.
|
||||
|
||||
These roles are part of the type; they are not interchangeable.
|
||||
|
||||
### Cooperative matrix operations
|
||||
|
||||
WGSL provides built-in functions for operating on cooperative matrices. The exact spelling may
|
||||
change; the semantics are:
|
||||
|
||||
#### `coopLoad` / `coopLoadT`
|
||||
|
||||
Collectively load a tile from memory into a cooperative matrix. Two variants
|
||||
select the memory layout:
|
||||
|
||||
- `coopLoad` — matrix is stored **column-major** in memory; `stride` is the
|
||||
number of elements between adjacent columns.
|
||||
- `coopLoadT` — matrix is stored **row-major** in memory (i.e. transposed
|
||||
relative to the canonical column-major layout used by `coopLoad`);
|
||||
`stride` is the number of elements between adjacent rows. This is the
|
||||
natural fit for C-style `ptr[i * num_cols + j]` storage.
|
||||
|
||||
```/dev/null/example.wgsl#L1-10
|
||||
fn coopLoad<T, R>(
|
||||
ptr: ptr<STORAGE_CLASS, T>, // base pointer to scalar or vector elements
|
||||
stride: u32 // elements between adjacent columns
|
||||
) -> coop_matMxN<T, R>;
|
||||
|
||||
fn coopLoadT<T, R>(
|
||||
ptr: ptr<STORAGE_CLASS, T>, // base pointer to scalar or vector elements
|
||||
stride: u32 // elements between adjacent rows
|
||||
) -> coop_matMxN<T, R>;
|
||||
```
|
||||
|
||||
- Loads an M×N tile (or M×K / K×N, depending on role and operation) from memory pointed to by `ptr`.
|
||||
- All invocations in the cooperative group must call the chosen variant in a converged fashion.
|
||||
- Memory address range must be valid and properly aligned for the scalar type.
|
||||
|
||||
> Implementation note: Each lane contributes to filling the tile based on an implementation-defined mapping from
|
||||
> invocation/lane ID to sub-fragment of the matrix.
|
||||
|
||||
#### `coopStore` / `coopStoreT`
|
||||
|
||||
Collectively store a cooperative matrix tile back to memory. Variant
|
||||
selection mirrors the load builtins:
|
||||
|
||||
- `coopStore` — writes **column-major**; `stride` between columns.
|
||||
- `coopStoreT` — writes **row-major**; `stride` between rows.
|
||||
|
||||
```/dev/null/example.wgsl#L12-23
|
||||
fn coopStore<T, R>(
|
||||
value: coop_matMxN<T, R>,
|
||||
ptr: ptr<STORAGE_CLASS, T>,
|
||||
stride: u32
|
||||
);
|
||||
|
||||
fn coopStoreT<T, R>(
|
||||
value: coop_matMxN<T, R>,
|
||||
ptr: ptr<STORAGE_CLASS, T>,
|
||||
stride: u32
|
||||
);
|
||||
```
|
||||
|
||||
- Stores `value` into the memory region addressed by `ptr` with given `stride`.
|
||||
- All invocations in the cooperative group must participate.
|
||||
- The store must not alias overlapping tiles in undefined ways.
|
||||
|
||||
#### `coopMultiplyAdd`
|
||||
|
||||
Perform a matrix multiply-accumulate operation on cooperative matrices:
|
||||
|
||||
```/dev/null/example.wgsl#L15-23
|
||||
fn coopMultiplyAdd<Tab, Tcr, MA, KA, KB, NB>(
|
||||
a: coop_matMAxKA<Tab, A>, // A: MAxKA tile
|
||||
b: coop_matKBxNB<Tab, B>, // B: KBxNB tile (KB == KA)
|
||||
c: coop_matMAxNB<Tcr, C> // C: MAxNB accumulator/result
|
||||
) -> coop_matMAxNB<Tcr, C>;
|
||||
```
|
||||
|
||||
Semantics:
|
||||
|
||||
- Computes `C' = A * B + C`.
|
||||
- Returns the resulting accumulator tile `C'`.
|
||||
- Implies:
|
||||
- `KA == KB` (inner dimension must match).
|
||||
- Types `(Tab, Tcr)` must be one of the supported AB/CR combinations given by
|
||||
`CooperativeMatrixProperties`.
|
||||
- Sizes `(MA, NB, KA)` must match a supported `(m_size, n_size, k_size)` triple.
|
||||
|
||||
For example, with a supported configuration:
|
||||
|
||||
```/dev/null/example.wgsl#L25-39
|
||||
enable wgpu_cooperative_matrix;
|
||||
|
||||
alias MatA = coop_mat8x8<f32, A>;
|
||||
alias MatB = coop_mat8x8<f32, B>;
|
||||
alias MatC = coop_mat8x8<f32, C>;
|
||||
|
||||
// Assumes each tile is stored column-major in memory (the plain `coopLoad`
|
||||
// / `coopStore` form); use `coopLoadT` / `coopStoreT` for row-major storage.
|
||||
fn matmul_tile(
|
||||
ptr_a: ptr<storage, f32>,
|
||||
ptr_b: ptr<storage, f32>,
|
||||
ptr_c: ptr<storage, f32>,
|
||||
stride: u32,
|
||||
) {
|
||||
let a: MatA = coopLoad<_, A>(ptr_a, stride);
|
||||
let b: MatB = coopLoad<_, B>(ptr_b, stride);
|
||||
let c: MatC = coopLoad<_, C>(ptr_c, stride);
|
||||
|
||||
let result: MatC = coopMultiplyAdd(a, b, c);
|
||||
coopStore(result, ptr_c, stride);
|
||||
}
|
||||
```
|
||||
|
||||
If `saturating_accumulation` is true for the chosen configuration, then overflow during accumulation
|
||||
is clamped (e.g. saturating arithmetic). If false, overflow behavior for the accumulator follows the
|
||||
underlying scalar type semantics (e.g. IEEE-754 for floats).
|
||||
|
||||
### Workgroup cooperation and execution model
|
||||
|
||||
Cooperative matrix operations are **collective**:
|
||||
|
||||
- All invocations in the relevant execution group must execute each cooperative operation in uniform control flow:
|
||||
- Using `coopLoad` / `coopLoadT`, `coopStore` / `coopStoreT`, or `coopMultiplyAdd` in divergent control flow
|
||||
(e.g. some lanes taking a branch, others not) is undefined behavior.
|
||||
- The exact execution group may be a workgroup, a SIMD-group / subgroup, or another backend-specific
|
||||
granularity; shaders must treat it abstractly.
|
||||
|
||||
- The workgroup (or cooperating group) size is constrained by both:
|
||||
- the cooperative matrix configuration, and
|
||||
- backend-specific implementation details.
|
||||
|
||||
For portable code:
|
||||
|
||||
- Choose a workgroup size that is known to be supported efficiently on your target backends, for example:
|
||||
- `@workgroup_size(8, 8, 1)` to operate on an 8×8 tile, or
|
||||
- a multiple of the tile size where each subgroup handles a tile.
|
||||
|
||||
- Avoid control-flow divergence around cooperative operations.
|
||||
|
||||
Example:
|
||||
|
||||
```/dev/null/example.wgsl#L1-42
|
||||
enable wgpu_cooperative_matrix;
|
||||
|
||||
struct Matrices {
|
||||
// Row-major tiles for A, B, C — use the `…T` load/store variants.
|
||||
data: array<f32>,
|
||||
};
|
||||
|
||||
@group(0) @binding(0)
|
||||
var<storage, read> buf_a: Matrices;
|
||||
@group(0) @binding(1)
|
||||
var<storage, read> buf_b: Matrices;
|
||||
@group(0) @binding(2)
|
||||
var<storage, read_write> buf_c: Matrices;
|
||||
|
||||
alias MatA = coop_mat8x8<f32, A>;
|
||||
alias MatB = coop_mat8x8<f32, B>;
|
||||
alias MatC = coop_mat8x8<f32, C>;
|
||||
|
||||
@compute @workgroup_size(8, 8, 1)
|
||||
fn main(
|
||||
@builtin(workgroup_id) wg_id: vec3<u32>,
|
||||
@builtin(local_invocation_id) lid: vec3<u32>,
|
||||
) {
|
||||
// Compute tile offset; this is one of many possible mappings.
|
||||
let tile_index = wg_id.x; // 1D tiling in this simple example
|
||||
let tile_offset = tile_index * 64u; // 8x8 tile has 64 elements
|
||||
|
||||
// Base pointers for tiles of A, B, C.
|
||||
let base_a = &buf_a.data[tile_offset];
|
||||
let base_b = &buf_b.data[tile_offset];
|
||||
let base_c = &buf_c.data[tile_offset];
|
||||
|
||||
let a: MatA = coopLoadT<f32, A>(base_a, 8u);
|
||||
let b: MatB = coopLoadT<f32, B>(base_b, 8u);
|
||||
let c: MatC = coopLoadT<f32, C>(base_c, 8u);
|
||||
|
||||
let result: MatC = coopMultiplyAdd(a, b, c);
|
||||
coopStoreT(result, base_c, 8u);
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Validation rules and undefined behavior
|
||||
|
||||
Implementations must validate the following where possible:
|
||||
|
||||
- The `wgpu_cooperative_matrix` WGSL extension is enabled if any cooperative matrix types
|
||||
or builtins are used.
|
||||
- Tile sizes `(M, N, K)` and scalar types `(ab_type, cr_type)` match at least one
|
||||
`CooperativeMatrixProperties` entry for the current adapter/backend.
|
||||
- Workgroup size, shader stage, and other pipeline configuration constraints required
|
||||
by the backend are satisfied.
|
||||
|
||||
The following are examples of **undefined behavior** (non-exhaustive):
|
||||
|
||||
- Using cooperative matrix operations without enabling the WGSL extension.
|
||||
- Using a cooperative matrix type `(M, N, T, R)` not supported by
|
||||
`Adapter::cooperative_matrix_properties()`.
|
||||
- Mismatching sizes or roles in `coopMultiplyAdd` (e.g. incompatible M/N/K, or incorrect roles).
|
||||
- Executing `coopLoad` / `coopLoadT`, `coopStore` / `coopStoreT`, or `coopMultiplyAdd` in divergent
|
||||
control flow within the cooperating execution group.
|
||||
- Providing invalid, misaligned, or out-of-bounds pointers to any of the load/store builtins.
|
||||
- Using a load/store variant (`coopLoad` vs `coopLoadT`, `coopStore` vs `coopStoreT`) whose memory
|
||||
layout does not match how the tile is actually stored.
|
||||
- Overlapping `coopStore` / `coopStoreT` targets in a way that creates data races or aliasing that
|
||||
the memory model does not allow.
|
||||
|
||||
---
|
||||
|
||||
## Example: 64×64 matrix multiply using 8×8 tiles
|
||||
|
||||
The example in `examples/features/src/cooperative_matrix` demonstrates using cooperative matrices to
|
||||
compute:
|
||||
|
||||
- `C = A * B + C` where:
|
||||
- `A` is 64×64,
|
||||
- `B` is 64×64,
|
||||
- `C` is 64×64.
|
||||
|
||||
A high-level tiling strategy:
|
||||
|
||||
1. Partition A, B, and C into 8×8 tiles.
|
||||
2. Launch one workgroup per output tile of C (i.e. 8×8 tiles for a 64×64 matrix = 8×8 = 64 tiles).
|
||||
3. Within each workgroup:
|
||||
- Loop over K-dimension tiles.
|
||||
- For each `k` tile:
|
||||
- Load an 8×8 tile of A (`MatA`).
|
||||
- Load an 8×8 tile of B (`MatB`).
|
||||
- Maintain an 8×8 accumulator tile (`MatC`) and repeatedly apply `coopMultiplyAdd`.
|
||||
4. After the K loop, store the final accumulator tile back to C.
|
||||
|
||||
Key points from the example:
|
||||
|
||||
- Workgroup size is chosen so that all cooperative operations are well-defined and efficient for 8×8 tiles.
|
||||
- Host-side code:
|
||||
- Enables `Features::EXPERIMENTAL_COOPERATIVE_MATRIX`.
|
||||
- Queries `cooperative_matrix_properties` and verifies that 8×8 `f32` or chosen configuration is supported.
|
||||
- Dispatches the compute pipeline with appropriate grid dimensions.
|
||||
|
||||
---
|
||||
|
||||
## Notes and best practices
|
||||
|
||||
- Always query `adapter.cooperative_matrix_properties()` and check that the configuration your shaders use exists.
|
||||
Do not hard-code assumptions about available tile sizes or element types.
|
||||
- Treat the cooperative execution group as an abstract concept; avoid making assumptions about how
|
||||
tiles are mapped to lanes beyond what is guaranteed by the spec.
|
||||
- Avoid divergent control flow around cooperative operations.
|
||||
- Consider providing a fallback non-cooperative implementation for devices that do not support the feature.
|
||||
- This is an experimental extension; API and semantics may change across versions of `wgpu` and `naga`.
|
||||
@@ -1,282 +0,0 @@
|
||||
# Mesh Shader Extensions
|
||||
|
||||
🧪Experimental🧪
|
||||
|
||||
`wgpu` supports an experimental version of mesh shading when `Features::EXPERIMENTAL_MESH_SHADER` is enabled.
|
||||
The status of the implementation is documented in [the mesh-shading issue](https://github.com/gfx-rs/wgpu/issues/7197).
|
||||
|
||||
**Note**: The features documented here may have major bugs in them and are expected to be subject
|
||||
to breaking changes. Suggestions for the API exposed by this should be posted on the issue above.
|
||||
|
||||
## Mesh shaders overview
|
||||
|
||||
### What are mesh shaders?
|
||||
|
||||
Mesh shaders are a new kind of rasterization pipeline intended to address some of the shortfalls with the vertex shader pipeline. The core idea of mesh shaders is that the GPU decides how to render the many small parts of a scene instead of the CPU issuing a draw call for every small part or issuing an inefficient monolithic draw call for a large part of the scene.
|
||||
|
||||
Mesh shaders are specifically designed to be used with **meshlet rendering**, a technique where every object is split into many subobjects called meshlets that are each rendered with their own parameters. With the standard vertex pipeline, each draw call specifies an exact number of primitives to render and the same parameters for all vertex shaders on an entire object (or even multiple objects). This doesn't leave room for different LODs for different parts of an object, for example a closer part having more detail, nor does it allow culling smaller sections (or primitives) of objects. With mesh shaders, each task workgroup might get assigned to a single object. It can then analyze the different meshlets(sections) of that object, determine which are visible and should actually be rendered, and for those meshlets determine what LOD to use based on the distance from the camera. It can then dispatch a mesh workgroup for each meshlet, with each mesh workgroup then reading the data for that specific LOD of its meshlet, determining which and how many vertices and primitives to output, determining which remaining primitives need to be culled, and passing the resulting primitives to the rasterizer.
|
||||
|
||||
Mesh shaders are most effective in scenes with many polygons. They can allow skipping processing of entire groups of primitives that are facing away from the camera or otherwise occluded, which reduces the number of primitives that need to be processed by more than half in most cases, and they can reduce the number of primitives that need to be processed for more distant objects. Scenes that are not bottlenecked by geometry (perhaps instead by fragment processing or post processing) will not see much benefit from using them.
|
||||
|
||||
Mesh shaders were first shown off in [NVIDIA's asteroids demo](https://www.youtube.com/watch?v=CRfZYJ_sk5E). Now, they form the basis for [Unreal Engine's Nanite](https://www.unrealengine.com/en-US/blog/unreal-engine-5-is-now-available-in-preview#Nanite).
|
||||
|
||||
### Mesh shader pipeline
|
||||
|
||||
With the current pipeline set to a mesh pipeline, a draw command like
|
||||
`render_pass.draw_mesh_tasks(x, y, z)` takes the following steps:
|
||||
|
||||
- If the pipeline has a task shader stage:
|
||||
- Dispatch a grid of task shader workgroups, where `x`, `y`, and `z` give
|
||||
the number of workgroups along each axis of the grid. Each task shader
|
||||
workgroup produces a mesh shader workgroup grid size `(mx, my, mz)` and a
|
||||
task payload value `mp`.
|
||||
|
||||
- For each task shader workgroup, dispatch a grid of mesh shader workgroups,
|
||||
where `mx`, `my`, and `mz` give the number of workgroups along each axis
|
||||
of the grid. Pass `mp` to each of these workgroup's mesh shader
|
||||
invocations.
|
||||
|
||||
- Alternatively, if the pipeline does not have a task shader stage:
|
||||
- Dispatch a single grid of mesh shader workgroups, where `x`, `y`, and `z`
|
||||
give the number of workgroups along each axis of the grid. These mesh
|
||||
shaders receive no task payload value.
|
||||
|
||||
- Each mesh shader workgroup produces a list of output vertices, and a list of
|
||||
primitives built from those vertices. The workgroup can supply per-primitive
|
||||
values as well, if needed. Each primitive selects its vertices by index, like
|
||||
an indexed draw call, from among the vertices generated by this workgroup.
|
||||
|
||||
Unlike a grid of ordinary compute shader workgroups collaborating to build
|
||||
vertex and index data in common storage buffers, the vertices and primitives
|
||||
produced by a mesh shader workgroup are entirely private to that workgroup,
|
||||
and are not accessible by other workgroups.
|
||||
|
||||
- Primitives produced by a mesh shader workgroup can have a culling flag. If a
|
||||
primitive's culling flag is false, it is skipped during rasterization.
|
||||
|
||||
- The primitives produced by all mesh shader workgroups are then rasterized in
|
||||
the usual way, with each fragment shader invocation handling one pixel.
|
||||
|
||||
Attributes from the vertices produced by the mesh shader workgroup are
|
||||
provided to the fragment shader with interpolation applied as appropriate.
|
||||
|
||||
If the mesh shader workgroup supplied per-primitive values, these are
|
||||
available to each primitive's fragment shader invocations. Per-primitive
|
||||
values are never interpolated; fragment shaders simply receive the values
|
||||
the mesh shader workgroup associated with their primitive.
|
||||
|
||||
## `wgpu` API
|
||||
|
||||
### New `wgpu` functions
|
||||
|
||||
`Device::create_mesh_pipeline` - Creates a mesh shader pipeline. This is very similar to creating a standard render pipeline, except that it takes a mesh shader state and optional task shader state instead of a vertex state. If the task state is omitted, during rendering the number of workgroups is passed directly from the draw call to the mesh shader state, with an empty payload.
|
||||
|
||||
`RenderPass::draw_mesh_tasks` - Dispatches the mesh shader pipeline. This ignores render pipeline specific information, such as vertex buffer bindings and index buffer bindings. The dispatch size must adhere to the limits described below.
|
||||
|
||||
`RenderPass::draw_mesh_tasks_indirect`, `RenderPass::multi_draw_mesh_tasks_indirect` and `RenderPass::multi_draw_mesh_tasks_indirect_count` - Dispatches the mesh shader pipeline with dispatch size taken from a buffer. This ignores render pipeline specific information, such as vertex buffer bindings and index buffer bindings. The dispatch size must adhere to the limits described below. Analogous to `draw_indirect`, `multi_draw_indirect` and `multi_draw_indirect_count`. Requires the corresponding indirect feature to be enabled.
|
||||
|
||||
An example of using mesh shaders to render a single triangle can be seen [here](../../examples/features/src/mesh_shader).
|
||||
|
||||
### Features
|
||||
|
||||
- Using mesh shaders requires enabling `Features::EXPERIMENTAL_MESH_SHADER`.
|
||||
- Using mesh shaders with multiview requires enabling `Features::EXPERIMENTAL_MESH_SHADER_MULTIVIEW`.
|
||||
- Using mesh shaders with point primitives requires enabling `Features::EXPERIMENTAL_MESH_SHADER_POINTS`.
|
||||
- Queries are unsupported
|
||||
- Primitive index support will be added once support lands in for them in general.
|
||||
|
||||
### Limits
|
||||
|
||||
> **NOTE**: More limits will be added when support is added to `naga`.
|
||||
|
||||
- `Limits::max_task_mesh_workgroup_total_count` - the maximum total number of workgroups from a `draw_mesh_tasks` command or similar. The dimensions passed must be less than or equal to this limit when multiplied together.
|
||||
- `Limits::max_task_mesh_workgroups_per_dimension` - the maximum for each of the 3 workgroup dimensions in a `draw_mesh_tasks` command. Each dimension passed must be less than or equal to this limit.
|
||||
- `max_task_invocations_per_workgroup` - The maximum total number of threads in a task shader workgroup, given by `workgroupSize.x * workgroupSize.y * workgroupSize.z`.
|
||||
- `max_task_invocations_per_dimension` the maximum value for each of `workgroupSize.x`, `workgroupSize.y` and `workgroupSize.z` in task shader workgroups.
|
||||
- `max_mesh_invocations_per_workgroup` - The maximum total number of threads in a mesh shader workgroup, given by `workgroupSize.x * workgroupSize.y * workgroupSize.z`.
|
||||
- `max_mesh_invocations_per_dimension` the maximum value for each of `workgroupSize.x`, `workgroupSize.y` and `workgroupSize.z` in mesh shader workgroups.
|
||||
- `max_task_payload_size` - the size of an `var<task_payload>` variable, in bytes.
|
||||
- `max_mesh_output_vertices` - the maximum number of vertices that a single mesh shader workgroup may output.
|
||||
- `max_mesh_output_primitives` - the maximum number of primitives that a single mesh shader workgroup may output.
|
||||
- `max_mesh_multiview_count` - the maximum number of views used when multiview rendering with a mesh shader pipeline.
|
||||
- `max_mesh_output_layers` - the maximum number of output layers for a mesh shader pipeline.
|
||||
|
||||
## Naga implementation
|
||||
|
||||
### Supported frontends
|
||||
|
||||
- 🛠️ WGSL
|
||||
- ❌ SPIR-V
|
||||
- 🚫 GLSL
|
||||
|
||||
### Supported backends
|
||||
|
||||
- 🛠️ SPIR-V
|
||||
- 🛠️ HLSL
|
||||
- ❌ MSL
|
||||
- 🚫 GLSL
|
||||
- 🚫 WGSL
|
||||
|
||||
✔️ = Complete
|
||||
🛠️ = In progress
|
||||
❌ = Planned
|
||||
🚫 = Unplanned/impossible
|
||||
|
||||
## `WGSL` extension specification
|
||||
|
||||
The majority of changes relating to mesh shaders will be in WGSL and `naga`.
|
||||
|
||||
Using any of these features in a `wgsl` program will require adding the `enable wgpu_mesh_shader;` directive to the top of a program.
|
||||
|
||||
Two new shader stages will be added to `WGSL`. Fragment shaders are also modified slightly. Both task shaders and mesh shaders are allowed to use any compute-available functionality, including subgroup operations.
|
||||
|
||||
### Task shader
|
||||
|
||||
A function with the `@task` attribute is a **task shader entry point**. A mesh shader pipeline may optionally specify a task shader entry point, and if it does, mesh draw commands using that pipeline dispatch a **task shader grid** of workgroups running the task shader entry point. Like compute shader dispatches, the three-component size passed to `draw_mesh_tasks`, or drawn from the indirect buffer for its indirect variants, specifies the size of the task shader grid as the number of workgroups along each of the grid's three axes.
|
||||
|
||||
A task shader entry point must have a `@workgroup_size` attribute, meeting the same requirements as one appearing on a compute shader entry point.
|
||||
|
||||
A task shader entry point must also have a `@payload(G)` property, where `G` is the name of a global variable in the `task_payload` address space. Each task shader workgroup has its own instance of this variable, visible to all invocations in the workgroup. Whatever value the workgroup collectively stores in that global variable becomes the **task payload**, and is provided to all invocations in the mesh shader grid dispatched for the workgroup. A task payload variable must be at least 4 bytes in size.
|
||||
|
||||
A task shader entry point must return a `vec3<u32>` value decorated with `@builtin(mesh_task_size)`. The return value of each workgroup's first invocation (that is, the one whose `local_invocation_index` is `0`) is taken as the size of a **mesh shader grid** to dispatch, measured in workgroups. (If the task shader entry point returns `vec3(0, 0, 0)`, then no mesh shaders are dispatched.) Mesh shader grids are described in the next section.
|
||||
|
||||
The output of a task shader is set to zero if it violates either of the limits `max_task_mesh_workgroup_total_count` or `max_task_mesh_workgroups_per_dimension`.
|
||||
|
||||
Each task shader workgroup dispatches an independent mesh shader grid: in mesh shader invocations, `@builtin` values like `workgroup_id` and `global_invocation_id` describe the position of the workgroup and invocation within that grid;
|
||||
and `@builtin(num_workgroups)` matches the task shader workgroup's return value. If this output violates any limits, it may be zeroed or cause undefined behavior, depending on the compilation options. Mesh shaders dispatched for other task shader workgroups are not included in the count. If it is necessary for a mesh shader to know which task shader workgroup dispatched it, the task shader can include its own workgroup id in the task payload.
|
||||
|
||||
Task shaders can use compute and subgroup builtin inputs, in addition to `view_index` and `draw_id`.
|
||||
|
||||
### Mesh shader
|
||||
|
||||
A function with the `@mesh` attribute is a **mesh shader entry point**. Mesh shaders must not return anything.
|
||||
|
||||
Like compute shaders, mesh shaders are invoked in a grid of workgroups, called a **mesh shader grid**. If the mesh shader pipeline has a task shader, then each task shader workgroup determines the size of a mesh shader grid to be dispatched, as described above. Otherwise, the three-component size passed to `draw_mesh_tasks`, or drawn from the indirect buffer for its indirect variants, specifies the size of the mesh shader grid directly, as the number of workgroups along each of the grid's three axes.
|
||||
|
||||
If the mesh shader pipeline has a task shader entry point, then the pipeline's mesh shader entry point must also have a `@payload(G)` attribute, and the sizes of the variables must match. Mesh shader invocations can read from, but not write to, this variable, which is initialized to whatever value was written to it by the task shader workgroup that dispatched this mesh shader grid.
|
||||
|
||||
If the mesh shader pipeline does not have a task shader entry point, then the mesh shader entry point must not have any `@payload` attribute.
|
||||
|
||||
A mesh shader entry point must have the following attributes:
|
||||
|
||||
- `@workgroup_size`: this has the same meaning as when it appears on a compute shader entry point.
|
||||
|
||||
- `@mesh(VAR)`: Here, `VAR` represents a workgroup variable storing the output information.
|
||||
|
||||
All mesh shader outputs are per-workgroup, and taken from the workgroup variable specified above. The type must have exactly 4 fields:
|
||||
|
||||
- A field decorated with `@builtin(vertex_count)`, with type `u32`: this field represents the number of vertices that will be drawn
|
||||
- A field decorated with `@builtin(primitive_count)`, with type `u32`: this field represents the number of primitives that will be drawn
|
||||
- A field decorated with `@builtin(vertices)`, typed as an array of `V`, where `V` is the vertex output type as specified below
|
||||
- A field decorated with `@builtin(primitives)`, typed as an array of `P`, where `P` is the primitive output type as specified below
|
||||
|
||||
For a vertex count `NV`, the first `NV` elements of the vertex array above are outputted. Therefore, `NV` must be less than or equal to the size of the vertex array. The same is true for primitives with `NP`.
|
||||
|
||||
The vertex output type `V` must meet the same requirements as a struct type returned by a `@vertex` entry point: all members must have either `@builtin` or `@location` attributes, there must be a `@builtin(position)`, and so on.
|
||||
|
||||
The primitive output type `P` must be a struct type, every member of which either has a `@location` or `@builtin` attribute. All members decorated with `@location` must also be decorated with `@per_primitive`, as must the corresponding fragment input. The `@per_primitive` decoration may only be applied to members decorated with `@location`. The following `@builtin` attributes are allowed:
|
||||
|
||||
- `triangle_indices`, `line_indices`, or `point_index`: The annotated member must be of type `vec3<u32>`, `vec2<u32>`, or `u32`.
|
||||
|
||||
The member's components are indices (or, its value is an index) into the list of vertices generated by this workgroup, identifying the vertices of the primitive to be drawn. These indices must be less than the value of `numVertices` passed to `setMeshOutputs`.
|
||||
|
||||
The type `P` must contain exactly one member with one of these attributes, determining what sort of primitives the mesh shader generates.
|
||||
|
||||
- `cull_primitive`: The annotated member must be of type `bool`. If it is true, then the primitive is skipped during rendering.
|
||||
|
||||
The `@location` attributes of `P` and `V` must not overlap, since they are merged to produce the user-defined inputs to the fragment shader.
|
||||
|
||||
Mesh shaders may write to the `primitive_index` builtin. This is treated just like a field decorated with `@location`, so if the mesh shader outputs `primitive_index` the fragment shader must input it, and if the fragment shader inputs it, the mesh shader must write it (unlike vertex shader pipelines).
|
||||
|
||||
Mesh shaders can use compute and mesh shader builtin inputs, in addition to `view_index`, and if no task shader is present, `draw_id`.
|
||||
|
||||
### Fragment shader
|
||||
|
||||
Fragment shaders can access vertex output data as if it is from a vertex shader. They can also access primitive output data, provided the input is decorated with `@per_primitive`. The `@per_primitive` decoration may only be applied to inputs or struct members decorated with `@location`.
|
||||
|
||||
The primitive state is part of the fragment input and must match the output of the mesh shader in the pipeline. Using `@per_primitive` also requires enabling the mesh shader extension. Additionally, the locations of vertex and primitive input cannot overlap.
|
||||
|
||||
### Full example
|
||||
|
||||
The following is a full example of WGSL shaders that could be used to create a mesh shader pipeline, showing off many of the features.
|
||||
|
||||
```wgsl
|
||||
enable wgpu_mesh_shader;
|
||||
|
||||
const positions = array(
|
||||
vec4(0., 1., 0., 1.),
|
||||
vec4(-1., -1., 0., 1.),
|
||||
vec4(1., -1., 0., 1.)
|
||||
);
|
||||
const colors = array(
|
||||
vec4(0., 1., 0., 1.),
|
||||
vec4(0., 0., 1., 1.),
|
||||
vec4(1., 0., 0., 1.)
|
||||
);
|
||||
|
||||
struct TaskPayload {
|
||||
colorMask: vec4<f32>,
|
||||
visible: bool,
|
||||
}
|
||||
struct VertexOutput {
|
||||
@builtin(position) position: vec4<f32>,
|
||||
@location(0) color: vec4<f32>,
|
||||
}
|
||||
struct PrimitiveOutput {
|
||||
@builtin(triangle_indices) indices: vec3<u32>,
|
||||
@builtin(cull_primitive) cull: bool,
|
||||
@per_primitive @location(1) colorMask: vec4<f32>,
|
||||
}
|
||||
struct PrimitiveInput {
|
||||
@per_primitive @location(1) colorMask: vec4<f32>,
|
||||
}
|
||||
|
||||
var<task_payload> taskPayload: TaskPayload;
|
||||
var<workgroup> workgroupData: f32;
|
||||
|
||||
@task
|
||||
@payload(taskPayload)
|
||||
@workgroup_size(1)
|
||||
fn ts_main() -> @builtin(mesh_task_size) vec3<u32> {
|
||||
workgroupData = 1.0;
|
||||
taskPayload.colorMask = vec4(1.0, 1.0, 0.0, 1.0);
|
||||
taskPayload.visible = true;
|
||||
return vec3(1, 1, 1);
|
||||
}
|
||||
|
||||
struct MeshOutput {
|
||||
@builtin(vertices) vertices: array<VertexOutput, 3>,
|
||||
@builtin(primitives) primitives: array<PrimitiveOutput, 1>,
|
||||
@builtin(vertex_count) vertex_count: u32,
|
||||
@builtin(primitive_count) primitive_count: u32,
|
||||
}
|
||||
|
||||
var<workgroup> mesh_output: MeshOutput;
|
||||
|
||||
@mesh(mesh_output)
|
||||
@payload(taskPayload)
|
||||
@workgroup_size(1)
|
||||
fn ms_main() {
|
||||
mesh_output.vertex_count = 3;
|
||||
mesh_output.primitive_count = 1;
|
||||
workgroupData = 2.0;
|
||||
|
||||
mesh_output.vertices[0].position = positions[0];
|
||||
mesh_output.vertices[0].color = colors[0] * taskPayload.colorMask;
|
||||
|
||||
mesh_output.vertices[1].position = positions[1];
|
||||
mesh_output.vertices[1].color = colors[1] * taskPayload.colorMask;
|
||||
|
||||
mesh_output.vertices[2].position = positions[2];
|
||||
mesh_output.vertices[2].color = colors[2] * taskPayload.colorMask;
|
||||
|
||||
mesh_output.primitives[0].indices = vec3<u32>(0, 1, 2);
|
||||
mesh_output.primitives[0].cull = !taskPayload.visible;
|
||||
mesh_output.primitives[0].colorMask = vec4<f32>(1.0, 0.0, 1.0, 1.0);
|
||||
}
|
||||
|
||||
@fragment
|
||||
fn fs_main(vertex: VertexOutput, primitive: PrimitiveInput) -> @location(0) vec4<f32> {
|
||||
return vertex.color * primitive.colorMask;
|
||||
}
|
||||
```
|
||||
@@ -1,411 +0,0 @@
|
||||
# Ray Tracing Extensions
|
||||
|
||||
🧪Experimental🧪
|
||||
|
||||
`wgpu` supports an experimental version of ray tracing which is subject to change. The extensions allow for acceleration structures to be created and built (with
|
||||
`Features::EXPERIMENTAL_RAY_QUERY` enabled) and interacted with in shaders. Currently `naga` only supports ray queries
|
||||
(accessible with `Features::EXPERIMENTAL_RAY_QUERY` enabled in wgpu).
|
||||
|
||||
**Note**: The features documented here may have major bugs in them and are expected to be subject
|
||||
to breaking changes, suggestions for the API exposed by this should be posted on [the ray-tracing issue](https://github.com/gfx-rs/wgpu/issues/1040).
|
||||
Large changes may mean that this documentation may be out of date.
|
||||
|
||||
**_This is not_** an introduction to raytracing, and assumes basic prior knowledge, to look at the fundamentals look at
|
||||
an [introduction](https://developer.nvidia.com/blog/introduction-nvidia-rtx-directx-ray-tracing/).
|
||||
|
||||
## `wgpu`'s raytracing API:
|
||||
|
||||
The documentation and specific details of the functions and structures provided
|
||||
can be found with their definitions.
|
||||
|
||||
Acceleration structures do not have a separate feature, instead they are enabled by `Features::EXPERIMENTAL_RAY_QUERY`, unlike vulkan.
|
||||
When ray tracing pipelines are added, that feature will also enable acceleration structures.
|
||||
|
||||
A [`Blas`] can be created with [`Device::create_blas`].
|
||||
A [`Tlas`] can be created with [`Device::create_tlas`].
|
||||
|
||||
The [`Tlas`] reference can be placed in a bind group to be used in a shader. A reference to a [`Blas`] can
|
||||
be used to create [`TlasInstance`] alongside a transformation matrix, custom data
|
||||
(this can be any data that should be given to the shader on a hit) which only the first 24
|
||||
bits may be set, and a mask to filter hits in the shader.
|
||||
|
||||
A [`Blas`] must be built in either the same build as any [`Tlas`] it is used to build or an earlier build call.
|
||||
Before a [`Tlas`] is used in a shader it must
|
||||
|
||||
- have been built
|
||||
- have all [`Blas`]es that it was last built with to have last been built in either the same build as
|
||||
this [`Tlas`] or an earlier build call.
|
||||
|
||||
### [`Blas`]es and [`Tlas`]es
|
||||
|
||||
Both acceleration structures are opaque objects. These objects typically (but are not guaranteed to)
|
||||
contain a tree-like structure of objects that are quick to be intersected with rays (as of 2026, these
|
||||
are usually axis-aligned bounding boxes) and contain two or more "branches" (these objects) and "leaves".
|
||||
For a [`Tlas`] these "leaves" are references to the [`Blas`] (not copies, hence a [`Tlas`] must be rebuild after
|
||||
any [`Blas`] it is storing is modified). For a [`Blas`] with triangle geometry the "leaves" are triangles. For
|
||||
a [`Blas`] with AABBs, the "leaves" might not exist or might contain only a small amount of information. This is
|
||||
because there is no requirement to store the AABBs as this might require a third intersection type. However,
|
||||
the "branches" will probably be fairly closely approximating the AABBs to maintain fairly good trace performance.
|
||||
|
||||
#### Building
|
||||
|
||||
Building an acceleration structure is a slow operation as it is likely to require building a tree-like structure.
|
||||
This happens on the GPU and can be encoded using [`CommandEncoder::build_acceleration_structures`].
|
||||
|
||||
For [`Blas`]es with triangle geometry, this will copy the triangles out of the vertex buffer (using indexing if
|
||||
provided), multiply them by the geometry transform matrix (if provided), and store a representation of this
|
||||
somewhere in the structure to be traced against (and computes surrounding objects for sets of triangles, repeating
|
||||
the process for objects of the surrounding objects etc. if the implementation uses a tree-like structure).
|
||||
|
||||
For [`Blas`]es with AABB geometry, this will construct object(s) in such a way that all possible rays will generate
|
||||
a candidate intersection that would have generated a candidate intersection with the AABBs. Note that this may be
|
||||
(though is very unlikely to be) a volume containing all space. (These may then have objects constructed around them
|
||||
if the implementation has a tree-like structure.)
|
||||
|
||||
For [`Tlas`]es, these will store references to the [`Blas`]es within the [`TlasInstance`]s as well as transforms,
|
||||
masks, and any other information required. (These [`Blas`]es then may have objects constructed around their transformed
|
||||
positions if the implementation uses a tree-like structure.) `wgpu` will store references to the [`Blas`]es to keep
|
||||
them within device memory and alive while the [`Tlas`] uses them. However, if you use functions to build it using
|
||||
the underlying functions (e.g. by using `as_hal` functions), you are responsible for keeping the [`Blas`]es alive.
|
||||
|
||||
Some memory is allocated when building to be "scratch" data (a temporary buffer used by the GPU to store data during
|
||||
the build) and instance staging memory (to copy the instances to the GPU). The some of this can be reused for between
|
||||
the [`Blas`] and [`Tlas`] builds so in general it is advisable to try and integrate the builds together. Because
|
||||
building is slow, it should be done as few times as possible. For moving geometry (players, particles, etc.), use
|
||||
[`PREFER_FAST_BUILD`](https://wgpu.rs/doc/wgpu_hal/struct.AccelerationStructureBuildFlags.html#associatedconstant.PREFER_FAST_BUILD)
|
||||
to speed up builds. Updating (performing a partial rebuild) is currently unsupported, but may be implemented in the
|
||||
future. You should compact any geometry which will not change (e.g. static level geometry) once it has been built.
|
||||
|
||||
### [`Blas`] compaction
|
||||
|
||||
Once a [`Blas`] has been built, it can be compacted. Acceleration structures are allocated conservatively, without
|
||||
knowing the exact data that is inside them. Once a [`Blas`] has been built, the driver can make data specific
|
||||
optimisations to make the [`BLAS`] smaller. To begin compaction call [`Blas::prepare_compaction_async`] on it. This
|
||||
method waits until all builds operating on the [`Blas`] are finished, prepares the [`Blas`] to be compacted, and runs
|
||||
the given callback. To check whether the [`Blas`] is ready, you can also call [`Blas::ready_for_compaction`] instead of
|
||||
waiting for the callback (useful if you are asynchronously compacting a large number of [`Blas`]es). Submitting a
|
||||
rebuild of a [`Blas`] terminates any [`Blas::prepare_compaction_async`], preventing the callback from being called, and
|
||||
making the [`Blas`] no longer ready to compact. Once a [`Blas`] is ready for compaction, it can be compacted using
|
||||
[`Queue::compact_blas`] this returns the new compacted [`Blas`], which is independent of the [`Blas`] passed in. The
|
||||
other [`Blas`] can be used for other things, including being rebuilt without affecting the new [`Blas`]. The returned
|
||||
[`Blas`] behaves largely like the [`Blas`] it was created from, except that it can be neither rebuilt, nor compacted
|
||||
again.
|
||||
|
||||
An example of compaction being run when [`Blas`]es are ready, this would be in a situation when memory was not a major
|
||||
problem, otherwise (e.g. if you get an out of memory error) you should compact immediately (and switching all
|
||||
non-compacted [`Blas`]es to compacted ones).
|
||||
|
||||
```rust
|
||||
use std::iter;
|
||||
use wgpu::Blas;
|
||||
|
||||
struct BlasToBeCompacted {
|
||||
blas: Blas,
|
||||
/// The index into the TlasInstance this BLAS is used in.
|
||||
tlas_index: usize,
|
||||
}
|
||||
|
||||
fn render(/*whatever args you need to render*/) {
|
||||
/* additional code to prepare the renderer */
|
||||
//An iterator of whatever BLASes you have called `prepare_compaction_async` on.
|
||||
let blas_s_pending_compaction: impl Iterator<Item = BlasToBeCompacted> = iter::empty();
|
||||
for blas_to_be_compacted in blas_s_pending_compaction {
|
||||
if blas_to_be_compacted.blas.ready_for_compaction() {
|
||||
let compacted_blas = queue.compact_blas(&blas_to_be_compacted.blas);
|
||||
tlas_instance[blas_to_be_compacted.tlas_index].set_blas(&compacted_blas);
|
||||
}
|
||||
}
|
||||
let mut encoder =
|
||||
device.create_command_encoder(&wgpu::CommandEncoderDescriptor { label: None });
|
||||
/* do other preparations on the TlasInstance.*/
|
||||
encoder.build_acceleration_structures(iter::empty(), iter::once(&tlas_package));
|
||||
/* more render code */
|
||||
queue.submit([encoder.finish()]);
|
||||
}
|
||||
```
|
||||
|
||||
[`Device::create_blas`]: https://wgpu.rs/doc/wgpu/struct.Device.html#method.create_blas
|
||||
[`CommandEncoder::build_acceleration_structures`]: https://wgpu.rs/doc/wgpu/struct.CommandEncoder.html#method.build_acceleration_structures
|
||||
[`Device::create_tlas`]: https://wgpu.rs/doc/wgpu/struct.Device.html#method.create_tlas
|
||||
[`Tlas`]: https://wgpu.rs/doc/wgpu/struct.Tlas.html
|
||||
[`Blas`]: https://wgpu.rs/doc/wgpu/struct.Blas.html
|
||||
[`TlasInstance`]: https://wgpu.rs/doc/wgpu/struct.TlasInstance.html
|
||||
[`Blas::prepare_compaction_async`]: https://wgpu.rs/doc/wgpu/struct.Blas.html#method.prepare_compaction_async
|
||||
[`Blas::ready_for_compaction`]: https://wgpu.rs/doc/wgpu/struct.Blas.html#method.ready_for_compaction
|
||||
[`Queue::compact_blas`]: https://wgpu.rs/doc/wgpu/struct.Queue.html#method.compact_blas
|
||||
|
||||
## `naga`'s raytracing API:
|
||||
|
||||
`naga` supports ray queries (also known as inline raytracing). To enable basic ray query functions you must add
|
||||
`enable wgpu_ray_query` to the shader, ray queries and acceleration structures also support tags which require extra
|
||||
`enable` extensions (see Acceleration structure tags for more info). Ray tracing pipelines are currently in
|
||||
development. Naming is mostly taken from vulkan.
|
||||
|
||||
### Ray Queries
|
||||
|
||||
```wgsl
|
||||
// - Initializes the `ray_query` to check where (if anywhere) the ray defined by `ray_desc` hits in `acceleration_structure`
|
||||
// - If `ray_desc` is "invalid" (see definition on struct) then either:
|
||||
// 1. the call is discarded (see behaviour of other calls if the ray query is uninitialized)
|
||||
// 2. the call behaves *as if* the minimum number of fields were changed to make `ray_desc` valid
|
||||
// - A ray query is "initialized" if this function has been called *and* was not discarded. If the call was discarded,
|
||||
// the ray query may either be treated as if it were uninitialized or left in its previous state.
|
||||
rayQueryInitialize(rq: ptr<function, ray_query>, acceleration_structure: acceleration_structure, ray_desc: RayDesc)
|
||||
// Overload.
|
||||
rayQueryInitialize(rq: ptr<function, ray_query<vertex_return>>, acceleration_structure: acceleration_structure<vertex_return>, ray_desc: RayDesc)
|
||||
|
||||
// - Traces the ray in the initialized ray_query (partially) through the scene.
|
||||
// - Returns true if a triangle that was hit by the ray was in a `Blas` that is not marked as opaque.
|
||||
// - Returns false if all triangles that were hit by the ray were in `Blas`es that were marked as opaque.
|
||||
// - The hit is considered `Candidate` if this function returns true, and the hit is considered `Committed` if
|
||||
// this function returns false.
|
||||
// - A `Candidate` intersection interrupts the ray traversal.
|
||||
// - A `Candidate` intersection may happen anywhere along the ray, it should not be relied on to give the closest hit. A
|
||||
// `Candidate` intersection is to allow the user themselves to decide if that intersection is valid*. If one wants to get
|
||||
// the closest hit a `Committed` intersection should be used.
|
||||
// - Calling this function multiple times will cause the ray traversal to continue if it was interrupted by a `Candidate`
|
||||
// intersection.
|
||||
// - If `rq` was not previously initialized, the call is discarded.
|
||||
// - If the previous proceed returned false (and no initialize was performed in between), the call is discarded.
|
||||
rayQueryProceed(rq: ptr<function, ray_query>) -> bool
|
||||
// Overload.
|
||||
rayQueryProceed(rq: ptr<function, ray_query<vertex_return>>) -> bool
|
||||
|
||||
// - Generates a hit from procedural geometry at a particular distance.
|
||||
// - If `rq` was not previously initialized, the call is discarded.
|
||||
// - If the previous proceed returned false (and no initialize was performed in between), the call is discarded.
|
||||
// - If `hit_t` is not between the `ray_desc.tmin` provided to the previous initialize and the `t` value of the
|
||||
// latest committed hit (or the last `ray_desc.tmax` if there are no committed hits), the call is discarded.
|
||||
rayQueryGenerateIntersection(rq: ptr<function, ray_query>, hit_t: f32)
|
||||
|
||||
// - Commits a hit from triangular non-opaque geometry.
|
||||
// - If `rq` was not previously initialized, the call is discarded.
|
||||
// - If the previous proceed returned false (and no initialize was performed in between), the call is discarded.
|
||||
rayQueryConfirmIntersection(rq: ptr<function, ray_query>)
|
||||
|
||||
// Aborts the query which is in progress, that is, the next `rayQueryProceed` is guaranteed to return `false`
|
||||
// and any call to `rayQueryGetCommittedIntersection` after the `rayQueryProceed` will return the closest
|
||||
// committed result so far. (`rayQueryProceed` must be called for `rayQueryGetCommittedIntersection` to return
|
||||
// a non zeroed value).
|
||||
//
|
||||
// - If `rq` was not previously initialized, the call is discarded.
|
||||
// - If the previous proceed returned false (and no initialize was performed in between), the call is discarded.
|
||||
rayQueryTerminate(rq: ptr<function, ray_query>)
|
||||
|
||||
// - Returns intersection details about a hit considered `Committed`.
|
||||
//
|
||||
// Depending on what type is hit, different fields will be populated. `RayIntersection::kind` is always populated
|
||||
// with the kind of hit.
|
||||
// - The following fields are populated if the closest hit was any object.
|
||||
// - t
|
||||
// - instance_custom_data
|
||||
// - instance_index
|
||||
// - sbt_record_offset
|
||||
// - geometry_index
|
||||
// - primitive_index
|
||||
// - object_to_world
|
||||
// - world_to_object
|
||||
// - The following fields are populated if the closest hit is was a triangle.
|
||||
// - barycentrics
|
||||
// - front_face
|
||||
// If `rq` was not previously initialized, the call is discarded.
|
||||
// If the previous proceed returned true or there were no previous proceed calls since the last initialize, a
|
||||
// zero-initialized `RayIntersection` is returned.
|
||||
rayQueryGetCommittedIntersection(rq: ptr<function, ray_query>) -> RayIntersection
|
||||
// Overload.
|
||||
rayQueryGetCommittedIntersection(rq: ptr<function, ray_query<vertex_return>>) -> RayIntersection
|
||||
|
||||
// - Returns intersection details about a hit considered `Candidate`.
|
||||
//
|
||||
// Depending on what type is hit, different fields will be populated. `RayIntersection::kind` is always populated
|
||||
// with the kind of hit.
|
||||
// - The following fields are populated if the closest hit was any object.
|
||||
// - instance_custom_data
|
||||
// - instance_index
|
||||
// - sbt_record_offset
|
||||
// - geometry_index
|
||||
// - primitive_index
|
||||
// - object_to_world
|
||||
// - world_to_object
|
||||
// - The following fields are populated if the closest hit is was a triangle.
|
||||
// - t
|
||||
// - barycentrics
|
||||
// - front_face
|
||||
//
|
||||
// Note that `t` is *only* returned for a candidate triangle intersection. This is because
|
||||
// `RAY_QUERY_INTERSECTION_AABB` is an AABB which has a volume.
|
||||
// If `rq` was not previously initialized, the call is discarded.
|
||||
// If the previous proceed returned false or there were no previous proceed calls since the last initialize, a
|
||||
// zero-initialized `RayIntersection` is returned.
|
||||
rayQueryGetCandidateIntersection(rq: ptr<function, ray_query>) -> RayIntersection
|
||||
// Overload.
|
||||
rayQueryGetCandidateIntersection(rq: ptr<function, ray_query<vertex_return>>) -> RayIntersection
|
||||
|
||||
// - Returns the vertices of the hit triangle considered `Committed`.
|
||||
// - If `rq` was not previously initialized, the call is discarded.
|
||||
// - If the previous proceed returned true or there were no previous proceed calls since the last initialize, a
|
||||
// zero-initialized `array` is returned.
|
||||
// - If the last hit was not a triangle, a zero-initialized `array` is returned.
|
||||
getCommittedHitVertexPositions(rq: ptr<function, ray_query<vertex_return>>) -> array<vec3<f32>, 3>
|
||||
|
||||
// - Returns the vertices of the hit triangle considered `Candidate`.
|
||||
// - If `rq` was not previously initialized, the call is discarded.
|
||||
// - If the previous proceed returned true or there were no previous proceed calls since the last initialize, a
|
||||
// zero-initialized `array` is returned.
|
||||
// - If the last hit was not a triangle, a zero-initialized `array` is returned.
|
||||
getCandidateHitVertexPositions(rq: ptr<function, ray_query<vertex_return>>) -> array<vec3<f32>, 3>
|
||||
|
||||
// A `RayDesc` is invalid if:
|
||||
// - `ray_desc.flags` contains more than one of `SKIP_TRIANGLES` and `SKIP_AABBS`
|
||||
// - `ray_desc.flags` contains more than one of `SKIP_TRIANGLES`, `CULL_BACK_FACING` and `CULL_FRONT_FACING`
|
||||
// - `ray_desc.flags` contains more than one of `FORCE_OPAQUE`, `FORCE_NO_OPAQUE`, `CULL_OPAQUE`, `CULL_NO_OPAQUE`
|
||||
// - `ray_desc.t_min` is less than `0.0`, or is not finite (`NaN` or `Inf`).
|
||||
// - `ray_desc.t_max` is less than `0.0`, less than `ray_desc.t_min` (the first rule is implied by the second).
|
||||
// - Any component of `ray_desc.origin` or `ray_desc.dir` is not finite (`NaN` or `Inf`)
|
||||
// - `ray_desc.dir`'s length is zero (this implies `ray_desc.dir`)
|
||||
struct RayDesc {
|
||||
// Contains flags to use for this ray (e.g. consider all `Blas`es opaque)
|
||||
flags: u32,
|
||||
// If the bitwise and of this and any `TlasInstance`'s `mask` is not zero then the object inside
|
||||
// the `Blas` contained within that `TlasInstance` may be hit.
|
||||
cull_mask: u32,
|
||||
// Only points on the ray whose t is greater than this may be hit.
|
||||
t_min: f32,
|
||||
// Only points on the ray whose t is less than this may be hit.
|
||||
t_max: f32,
|
||||
// The origin of the ray.
|
||||
origin: vec3<f32>,
|
||||
// The direction of the ray, t is calculated as the length down the ray divided by the length of `dir`.
|
||||
dir: vec3<f32>,
|
||||
}
|
||||
|
||||
struct RayIntersection {
|
||||
// the kind of the hit, no other member of this structure is useful if this is equal
|
||||
// to constant `RAY_QUERY_INTERSECTION_NONE`.
|
||||
kind: u32,
|
||||
// Distance from starting point, measured in units of `RayDesc::dir`.
|
||||
t: f32,
|
||||
// Corresponds to `instance.custom_data` where `instance` is the `TlasInstance`
|
||||
// that the intersected object was contained in.
|
||||
instance_custom_data: u32,
|
||||
// The index into the `TlasPackage` to get the `TlasInstance` that the hit object is in
|
||||
instance_index: u32,
|
||||
// The offset into the shader binding table. Currently, this value is always 0.
|
||||
sbt_record_offset: u32,
|
||||
// The index into the `Blas`'s build descriptor (e.g. if `BlasBuildEntry::geometry` is
|
||||
// `BlasGeometries::TriangleGeometries` then it is the index into that contained vector).
|
||||
geometry_index: u32,
|
||||
// The object hit's index into the provided buffer (e.g. if the object is a triangle
|
||||
// then this is the triangle index)
|
||||
primitive_index: u32,
|
||||
// Two of the barycentric coordinates, the third can be calculated (only useful if this is a triangle).
|
||||
barycentrics: vec2<f32>,
|
||||
// Whether the hit face is the front (only useful if this is a triangle).
|
||||
front_face: bool,
|
||||
// Matrix for converting from object-space to world-space.
|
||||
//
|
||||
// This matrix needs to be on the left side of the multiplication. Using it the other way round will not work.
|
||||
// Use it this way: `let transformed_vector = intersecion.object_to_world * vec4<f32>(x, y, z, transform_multiplier);
|
||||
object_to_world: mat4x3<f32>,
|
||||
// Matrix for converting from world-space to object-space
|
||||
//
|
||||
// This matrix needs to be on the left side of the multiplication. Using it the other way round will not work.
|
||||
// Use it this way: `let transformed_vector = intersecion.world_to_object * vec4<f32>(x, y, z, transform_multiplier);
|
||||
world_to_object: mat4x3<f32>,
|
||||
}
|
||||
|
||||
/// -- Flags for `RayDesc::flags` --
|
||||
|
||||
// All `Blas`es are marked as opaque.
|
||||
const FORCE_OPAQUE = 0x1;
|
||||
|
||||
// All `Blas`es are marked as non-opaque.
|
||||
const FORCE_NO_OPAQUE = 0x2;
|
||||
|
||||
// Instead of searching for the closest hit return the first hit.
|
||||
const TERMINATE_ON_FIRST_HIT = 0x4;
|
||||
|
||||
// Unused: implemented for raytracing pipelines.
|
||||
const SKIP_CLOSEST_HIT_SHADER = 0x8;
|
||||
|
||||
// If `RayIntersection::front_face` is false do not return a hit.
|
||||
const CULL_BACK_FACING = 0x10;
|
||||
|
||||
// If `RayIntersection::front_face` is true do not return a hit.
|
||||
const CULL_FRONT_FACING = 0x20;
|
||||
|
||||
// If the `Blas` a intersection is checking is marked as opaque do not return a hit.
|
||||
const CULL_OPAQUE = 0x40;
|
||||
|
||||
// If the `Blas` a intersection is checking is not marked as opaque do not return a hit.
|
||||
const CULL_NO_OPAQUE = 0x80;
|
||||
|
||||
// If the `Blas` a intersection is checking contains triangles do not return a hit.
|
||||
const SKIP_TRIANGLES = 0x100;
|
||||
|
||||
// If the `Blas` a intersection is checking contains AABBs do not return a hit.
|
||||
const SKIP_AABBS = 0x200;
|
||||
|
||||
/// -- Constants for `RayIntersection::kind` --
|
||||
|
||||
// The ray hit nothing.
|
||||
const RAY_QUERY_INTERSECTION_NONE = 0;
|
||||
|
||||
// The ray hit a triangle.
|
||||
const RAY_QUERY_INTERSECTION_TRIANGLE = 1;
|
||||
|
||||
// The ray hit a custom object, this will only happen in a committed intersection
|
||||
// if a ray which intersected a bounding box for a custom object which was then committed.
|
||||
const RAY_QUERY_INTERSECTION_GENERATED = 2;
|
||||
|
||||
// The ray hit a AABB, this will only happen in a candidate intersection
|
||||
// if the ray intersects the bounding box for a custom object.
|
||||
const RAY_QUERY_INTERSECTION_AABB = 3;
|
||||
```
|
||||
|
||||
### Ray Tracing Pipelines
|
||||
|
||||
Functions
|
||||
|
||||
```wgsl
|
||||
// Begins to check where (if anywhere) the ray defined by `ray_desc` hits in `acceleration_structure` calling through the `any_hit` shaders and `closest_hit` shader if something was hit or the `miss` shader if no hit was found
|
||||
traceRay<T>(acceleration_structure: acceleration_structure, ray_desc: RayDesc, payload: ptr<ray_payload, T>)
|
||||
```
|
||||
|
||||
> [!CAUTION]
|
||||
>
|
||||
> #### ⚠️Undefined behavior ⚠️:
|
||||
>
|
||||
> Calling `traceRay` inside another `traceRay` more than `max_recursion_depth` times
|
||||
>
|
||||
> \*this is only known undefined behaviour, and will be worked around in the future.
|
||||
|
||||
New shader stages
|
||||
|
||||
```wgsl
|
||||
// First stage to be called, allowed to call `traceRay`
|
||||
@ray_generation
|
||||
fn rg() {}
|
||||
|
||||
// Stage called on any hit that is not opaque, not allowed to call `traceRay`
|
||||
@any_hit
|
||||
fn ah() {}
|
||||
|
||||
// Stage called on the closest hit, allowed to call `traceRay`
|
||||
@closest_hit
|
||||
fn ch() {}
|
||||
|
||||
// Stage call if there was never a hit, allowed to call `traceRay`
|
||||
@miss
|
||||
fn miss() {}
|
||||
```
|
||||
|
||||
### Acceleration structure tags
|
||||
|
||||
These are tags that can be added to a acceleration structure (`acceleration_structure` ->
|
||||
`acceleration_structure<... insert tags here! ...>`) and to a ray query (`ray_query` ->
|
||||
`ray_query<... insert tags here! ...>`). These require more features.
|
||||
|
||||
| Tag | Requirements | Description |
|
||||
| --------------- | ------------------------------------- | ---------------------------------------------------------------------- |
|
||||
| `vertex_return` | `enable wgpu_ray_query_vertex_return` | Allows getting the vertices of the hit triangle when using ray queries |
|
||||
Binary file not shown.
|
Before Width: | Height: | Size: 146 KiB |
@@ -1 +0,0 @@
|
||||
<mxfile host="app.diagrams.net" modified="2023-12-04T19:38:59.956Z" agent="Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:120.0) Gecko/20100101 Firefox/120.0" etag="ZvsPE6AT2WA2xAhdKlfq" version="22.1.5" type="device"><diagram id="FLHi6BIjYAkfD6NTjxEH" name="Page-1">7V1be5s4E/41vrQfJHHyZU7tdrftl6dpm93c7IONbNMQSDFOnP76T9jIRgeDjMUh3XjbrREgsGbm1TujkTRAFw/r94n3uPgU+zgcQMNfD9DlAELgQJf8k5W8bEvGrrUtmCeBn1+0L7gJfuG80MhLV4GPl8yFaRyHafDIFk7jKMLTlCnzkiR+Zi+bxSH71EdvjoWCm6kXiqW3gZ8utqWuZezL/8DBfEGfDIz8zMSb3s+TeBXlzxtANNt8tqcfPFpXfv1y4fnxc+Gh6GqALpI4TrffHtYXOMzaljbb9r53B87u3jvBUapyQxgNJy9X7979+udrMPx4he9Wi7shgttqnrxwlTfIX/glf+H0hTbS5mfirCIwQOfPiyDFN4/eNDv7TNSClC3ShzA/HXoTHJ7vWuciDuOEnIriiFx/PoujNNcCYObH9BLShMbmQ8q9MJhHpGxKfh4mJ8+fcJIGRG5n+Yk0zp4rtkLeMNnleM03N1FjHD/gNCG/0cjPWmYuoVyDTSqx570+wLEzyvV6UdQGqiZeroXzXe17SZAvuTAOCObP2/n1nffzj6H/dPfpSwrMu4ehIwrmz5th5KXBEybFt3jy/vpb9uBHYie8uCoE5HvLxU6YsyAMC+1P9BdOpwebW5DKRpz5k7PqlmkS3+NChb49sS1bk6QotrxwzV+QFLANUU5OU2KyXUFMW9EMP1NR6ZYQBr6FHY0SGtsO8jRJyAQKErJalZAhk9Bw4i1JA/fYkHSYi80KA9oSYLMlsNaYMKyxIIwBtMM0a8vgiXydZ1+/rJbpTiLbs5OEnqQl5PmFW7RKzp3iEsk1ISi2A5L0P1KbAeOm5ERJXaFBsU9oU34YJ+kinseRF17tS8/3pR/jrKU2rfMDp+lL3t97qzRmxYDXQfo3+W7k3//JvpOOdnt0uS6cunyhBxH5hX/TCrKDwl3Z4f62zRG9b4t7lOOhMskt41UyxSXtQ4mql8xxWnadub0wa7xSRUhwuO0vGHqrW6zIaUmsVETGCFqMlI6SESxqyMixCkoCSlWkvmCRqmB1y3Vz61mSeC+FCx7jIEqXhZqvs4I9biBKP3LgQAbnAnDXm45Vdj35sn0D+d1wzMIUjz7bxs1v4jR11yRqyitzVURCnGz6CYlCf8w8EFYlK92Jh8D3N/qe4GXwy5ts6suUKRcCqdw6H1iXnAsDKx0eURlLbVPoNXaub/5Og7x/ONibDImtQGPM8rDt0XHqKGgAsEasAg2RMaIYR6uJZ7MlPlkDpE00bhm+uB7Gqepi9mCljlUi5NVHL6jcL/UCvhybBRTkOofhSJcSUXImMtDMrBntsn+uYnpiuNxoyxm5gMhmvT9JGejz/HE1nMYJLhDUbY0HGOpy4T1mX4mgvTDEYTxPvIcMb3ASkB+ZgRR77np/oioEMwvWmMbaJAyXOJCub8p8QRdOkK3LF+TiKsi0RF5rSnit3Zj7YbbNf46jqBQ/wECdDp+AFopggXoBFpDTJmBYpVxHuB6WcB1d4CJyFIl7e46fXhS92A4xYucFCxgxcS3TKlU9dYzgIhRjiecLJQiBNCCE9+3pc3p5++4q/gLCL//++1d082FIw1dtOb6spTuKnm8BSVS8KibYUR8vVH0j6pR07Ry5NgsAqMI54twb7vqTAeNq4fxIJpO7b5/MG/RjOZmMv34eIjE6eSHVwJZdmkOjMse6OqVGdrqrQwwBANbToRh8oqszdNAIuuPCh30KgGyNDXo9Kn2KPg57kL72vcNSILVFLddEcnex8pIeDLXKcTvtwZQ7MGBAlhuPx6iKHWdHBX05kf6ar4v+uojtnVyLU5XS0J3Fk+Ex+37bVtASuytp6wKGjUajPoGHAtvN7qBWlJuDfgIMHBkDlgzR6WDAcvxArwE/WOyAVY51fZSwXhdKcBE1u8JHtnmf2j0CVBDNUdmF71rFFEoi95jy2YuiePXKYEU/hshSLppyouVRkLaGGWuE2w/F9huEEFsRQvoRlLcRF7W1wTGYwDnRaMy9X8OgYAugcIOTp7hXmEATg5pNleP8klZDa3L1PhwcVfVagU281o0vyjmukTf3XsO4i4fdmVT09tTFk1kzLiqyWNfBlHFMU8IxG0tuNYEglVYHcpXHcY/gpvX7B6DqiQK7kR6i0ltENG2kItPjVLZKn6NKV7nrm4nQQjFC668plv0eQdpSG9WSjwJNmwvGb49ODdICwNTKOSDNxWTNtpzkUwEs57UUwzpHMKcXHBeO2R4RWscE02jW3A6GINdP6kuEkyISEEmuZETgaRXee5Fi2P63Ay86aKJhhMkwAJtlOdQEXpbDQiKnRc2hFxDDsbXGj2RMfJMGtfDC18DGjxowgpqCNQKJkWVFNTViJB2ObSursl1SbTXS00g6A1aagKPKB+IdRyczyB/TbKakGGElVXrhMBEnSvXbqdZgtnwuo9FiOEVqtT2eoXOC1bbk4fLmxDu4DUcpJQzORz6Ab5aVxZu7tixKxH4z02rG9aoyLepatWZadBrN3rS85aJPVjWzsv+kVrX5iL7R9tOIte2OOxsWMFvOVzJGABZzDgBx8JqZcJpVoi9dST3G0o/0W+iwET8TtRGsFa1fEhqZh3iJ/quREW25t0NjZFiIDY3oioxAplabraDBwIi4msc8jJ/71Ht0wslkK+K0nFIiTtt7j6f34sB+Zet2MBbvcAHj7sfibdksSB75Iv8sW19rDzGFdmRnrW7YDF3eytgebAwkDKL7sjas7KMKLSRbAYOWnTrgycrHUgzoV9WzM6O2kl+gSu/3v5tDXZ9wqZ+QNk2W/ekqtXSB9iEDrjE44LiIHUo6dfoJrYZ3m5rr9GyRM22C+HStsQN93wKvPcLAuV4vL931d/D4Dk9cGamTycucHwvcrjHbaSlphgVyBudPTrosrqGgMnPxoCAr3SWa49RR1mXl7ELED/dpGiiw+TW/zPJZj5ZZen0zIwtUOAXIucTR4SzNzrCmHX5oG9wkkDaxRupHiyM/ZfRwGnrLZTCVx1vY+R2wmRnO1UBYH0lUpz3T2RA94bQmv6hcXVJrcd4L4FcHbAi5gGFrRSKpostSaVpR9EqF7aDPVJ7gb/RK0xEXrVBcl0xcjoiPjfPwqknRHdtsX9GhGNp6xhOizvhhEmZvL+r96418lpq6lsAnMk7083YZrGM0Apax/3DuJCAPA4XT9Lla/UJpe4kKUxsaa/oKvaEA1EXW5kx0RAH4FLK6FEDIVdSEjLyTRBdXOPheoPT6ZpBUtoSxil0sSSOlhyOq/MSfgu2MYdF6Rlk+ap1Z0pXjlZUWtPkJ74KsvTod4ewX0+bnPJq8daiambBgHm+vupIJEZt5bI5bGJqljVKcmrnwfKI3whLf73GEEyKROBLPldKU7hZ5LcbPHr1pEM0HhxbzLAUVLeTEpq7oycSa4yLOyG4iLC3Xl+OGqKT8Q5Vf1s7wqM85XhvIWYDDDENtTmA1WjYEcsILW1Vcgr3eMtoARShRcmEYjsXJylG77wQGfQY+ux+4a9XTA4cisHXQ1DIdV4sFDbkRJXfkNpHYIm8RWV5UU2AqS2tVI6ztwa9GMqo60aQbMqo6ObGSjAqbwmjCaeiwuKt7aovcHmTBDQFIN7Nd/qsYqnM7AsPhlorSlCcIuezDJmZQyqOpIjnNNpL6KKjFb7/ODJ8p2PnQJZRNbm26q2shvnlCV+aKXVmpVve0KzONml0Z4MKXFp0erj8vT96ubU9xKDIvHROKTtMWvROKaLCqSnji+DJXj6nmwNbJeZFtX1RHB9htV1taZFNM+To6jMIFniv3zVHDNun+tSK0tbg8Z+UWWYCG0HSTZn5zC/PIzTDy99KW5VUim3KO/Z3wHiL2/i8v3tJSnlxqAGqRTkmFqBKGbVMq6ptLlyGVBklZlsNthmZKFl6VbUbMe+T6ZCVGE7/ItsgjvzAdSKZz8I1ZMIC8iLqzIZ6lZc6sTKj7zqzaSS3K0SiRY5yNtaRZ+2szQKgiVqdVsYoeTVYR+Xu++fMm3VOkK3FY25WuQq4t27gViNlnhJQ0drsIqRAHbzzpvMcC2iXUdCYgMTCb7wb+4eExuzuekf9drUnLR1ls1ji7/vCGgCchoEzmrSIgtcGOEJCTeSezzexx78gkFFPjt9tDvRmb+sRvu3dkUpIH3iNja2VxA4mxdc1L6HZMxYUnZmvZcmBv5naUuXXN7iX7O/bH3Fqa3SiYW+csE4lBrR/ek7ecJsHjW7jkNJPrnE4iWdyZE2l1gr50XLh8aMSs4631ZKxV4gryu3+ojrbaoLIqjeugSDVADJjdxsl9lpQNjWsvFZcDfDPxozxGyaKB7Zp43fnpTZq42i7bPbF3Sadc196dauho2t5FV/Vr7Mdvxq6lP+/c2EWP9esCkwLqIRnnwQbYg2m6SsQFi04Xd3XypkaBm2UC1yBh2eJzpgup1IsidRsTaZ0tB7QlwxyTrceAemnfUE9uyss2d7C1jqqILz5OZ+HPn9/v/MnqKU6uz5bW+bCtDZJqp7xx7SSRTuOZbMgc1U1ms6qrqp3PRg6TONvUZX85QbTFp9jPstuv/g8=</diagram></mxfile>
|
||||
@@ -0,0 +1,65 @@
|
||||
# Managing Cargo dependencies
|
||||
|
||||
In general, `wgpu` tries to stay up-to-date with the current Rust ecosystem; we are interested in keeping the version of dependency crates we use up-to-date. However, there are some considerations to keep in mind when upgrading or adding dependencies to `wgpu`.
|
||||
|
||||
## Adding or upgrading dependencies may cause dependency duplication downstream
|
||||
|
||||
Some crates in the Rust ecosystem are useful for both `wgpu` and its downstream consumers. Popular crates—like those in the `serde` ecosystem, for example—are often used in multiple places in a Rust crate's dependency tree. Normally, Cargo can unify dependencies behind a single SemVer-compatible version. When one or more, but not all, of these places consume a [SemVer-incompatible upgrade][semver-summary], Cargo uses both in the new dependency tree (see The Cargo Book's section titled [`Version-incompatibility hazards`](https://doc.rust-lang.org/cargo/reference/resolver.html#version-incompatibility-hazards)). These are often referred to as "duplicate dependencies"[^1].
|
||||
|
||||
[semver-summary]: https://semver.org/spec/v2.0.0.html#summary
|
||||
|
||||
[^1]: One can detect such duplicates with [`cargo tree --duplicates`](https://doc.rust-lang.org/cargo/commands/cargo-tree.html#option-cargo-tree---duplicates).
|
||||
|
||||
There exist many Rust projects that, for some reason or another, are sensitive to duplicate dependencies. The most common reason for this is to minimize the number of built dependencies, but there are other reasons for doing so, like projects that prefer to [vendor in their dependencies' source code](https://doc.rust-lang.org/cargo/commands/cargo-vendor.html) and keep diffs easy to inspect. **When a dependency of `wgpu` releases a SemVer-incompatible change and `wgpu` consumes it (even if it's new _to `wgpu`_), it may cause conflicts for projects that avoid duplicate dependencies.**
|
||||
|
||||
> [!TIP]
|
||||
> Downstreams like Firefox can and do work around Cargo's (well-motivated) refusal to unify SemVer-incompatible versions of crates, but this happens only on a case-by-case basis. In general, this requires source compatibility for the APIs actually consumed between both versions.
|
||||
|
||||
## Firefox needs to audit every dependency
|
||||
|
||||
In addition to the above, Firefox has unusual sensitivities to updates because of required auditing via [`cargo-vet`](https://github.com/mozilla/cargo-vet). **Firefox contributors _cannot_ bring in a Rust dependency—including new versions of dependencies already in use—without an audit indicating that the dependency meets [`safe-to-deploy`] criteria for _all_ of its feature combinations.** One can ask the following questions to determine whether an upgrade is feasible for Firefox; "yes" answers are significantly easier cases to deal with:
|
||||
|
||||
1. If the dependency had to be audited for [`safe-to-deploy`], would it take less than an hour to review?
|
||||
|
||||
<sup>One can use <https://diff.rs/> to quickly gauge the answer to this question for upgrades. Example: [`serde` 1.0.218 → 1.0.219](https://diff.rs/serde/1.0.218/1.0.219/src%2Flib.rs)</sup>
|
||||
|
||||
1. Does the same code work between the two dependency's versions without needing to be changed (viz., are they "source-compatible")?
|
||||
|
||||
<sup>NOTE: This is somewhat different from [SemVer compatibility][semver-summary], because it deals with code already written, rather than SemVer's dealing with code that _might_ be written.</sup>
|
||||
|
||||
1. Does the dependency already have an audit in Mozilla's [`supply-chain/audits.toml`](https://searchfox.org/mozilla-central/source/supply-chain/audits.toml) or in the other audit databases it `import`s in [`supply-chain/config.toml`](https://searchfox.org/mozilla-central/source/supply-chain/config.toml)?
|
||||
|
||||
[`safe-to-deploy`]: https://mozilla.github.io/cargo-vet/built-in-criteria.html#safe-to-deploy
|
||||
|
||||
> [!TIP]
|
||||
> [Searchfox](https://searchfox.org) can be used to answer question (3) and other simple questions with just a web browser. For example:
|
||||
>
|
||||
> - "Does Firefox use the `itertools` crate?" can be answered by a case-sensitive search scoped to files with the `rs` extension: https://searchfox.org/mozilla-central/search?q=itertools&path=*.rs&case=true®exp=false
|
||||
> - "Is `serde` 1.0.143 already audited?" can be answered by examining `supply-chain/audits.toml` ([permalink](https://searchfox.org/mozilla-central/rev/f3c8c63a097b61bb1f01e13629b9514e09395947/supply-chain/audits.toml#4730-4733)).
|
||||
|
||||
> [!TIP]
|
||||
> For more complicated questions like, "What features does Mozilla use in the `hashbrown` crate?" the best known way to satisfy the question is using Cargo's CLI, i.e., `cargo tree` or `cargo metadata`, in tandem with editing Cargo manifests directly. A Git mirror of Firefox's source code can be found at [`mozilla/gecko-dev`](https://github.com/mozilla/gecko-dev).
|
||||
>
|
||||
> A full checkout of `gecko-dev` fills multiple gigabytes, and is not recommended to make one locally unless you plan on doing work directly on Firefox. However, we can optimize to somewhat less than 1 GiB of storage, and speed up the checkout process by using `git sparse-checkout`:
|
||||
>
|
||||
> ```sh
|
||||
> git clone --sparse --depth=1 'https://github.com/mozilla/gecko-dev.git'
|
||||
> cd gecko-dev
|
||||
> git sparse-checkout set --no-cone
|
||||
> git sparse-checkout add \
|
||||
> '**/Cargo.toml' \
|
||||
> '**/lib.rs' \
|
||||
> '**/main.rs' \
|
||||
> '**/src/bin/*.rs' \
|
||||
> ;
|
||||
> ```
|
||||
>
|
||||
> From here, you can invoke Cargo with the `--locked` flag, i.e:
|
||||
>
|
||||
> ```sh
|
||||
> cargo --locked tree -i itertools
|
||||
> ```
|
||||
>
|
||||
> ```sh
|
||||
> cargo --locked metadata --format-version=1
|
||||
> ```
|
||||
Binary file not shown.
|
Before Width: | Height: | Size: 8.2 KiB |
@@ -0,0 +1,165 @@
|
||||
# Running the tests on Android and iOS
|
||||
|
||||
An approach of cross-compiling the binaries on a host, then using various tools such as `adb` and `ssh` to transfer the test binaries over to run them, can be used.
|
||||
|
||||
Below are example configuration and runner scripts that can be used to perform this yourself.
|
||||
|
||||
## Android
|
||||
|
||||
<details>
|
||||
<summary>.cargo/config.toml</summary>
|
||||
|
||||
```toml
|
||||
[target.aarch64-linux-android]
|
||||
# Runner script is written in powershell
|
||||
runner = ["pwsh", "-File", "<runner location>run-on-android.ps1"]
|
||||
rustflags = [
|
||||
"-C", "linker=clang",
|
||||
"-C", "link-arg=-fuse-ld=lld",
|
||||
"-C", "link-arg=--target=aarch64-linux-android",
|
||||
"-C", "link-arg=--sysroot=<ndk/sysroot location>",
|
||||
"-C", "link-arg=-B<ndk/sysroot location>/usr/lib/aarch64-linux-android/<version>",
|
||||
"-C", "link-arg=-L<ndk/sysroot location>/usr/lib/aarch64-linux-android/<version>",
|
||||
"-C", "link-arg=-L<ndk/sysroot location>/usr/lib/aarch64-linux-android",
|
||||
"-C", "link-arg=-L<ndk location>/toolchains/llvm/prebuilt/<host>/lib/clang/20/lib/linux/aarch64"
|
||||
]
|
||||
```
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary>run-on-android.ps1</summary>
|
||||
|
||||
```powershell
|
||||
param(
|
||||
[string]$BinaryPath
|
||||
)
|
||||
|
||||
$ErrorActionPreference = "Stop"
|
||||
|
||||
# /data/local/tmp is the most common directory where arbitrary binaries can be run
|
||||
$RemoteDir = "/data/local/tmp/runner"
|
||||
$BinaryName = Split-Path -Leaf $BinaryPath
|
||||
$RemotePath = "$RemoteDir/$BinaryName"
|
||||
|
||||
adb push --sync "$BinaryPath" "$RemotePath" *>$null
|
||||
|
||||
adb shell "chmod 755 $RemotePath"
|
||||
|
||||
$EscapedArgs = $args | ForEach-Object {
|
||||
if ($_ -match " ") {
|
||||
"`"$_`""
|
||||
} else {
|
||||
$_
|
||||
}
|
||||
}
|
||||
|
||||
$LinuxArgs = $EscapedArgs -join " "
|
||||
# Set LD_LIBRARY_PATH to the location of libc++_shared.so
|
||||
$DeviceCmd = "cd $RemoteDir && LD_LIBRARY_PATH=$RemoteDir ./$BinaryName $LinuxArgs"
|
||||
|
||||
adb shell "$DeviceCmd"
|
||||
exit $LASTEXITCODE
|
||||
```
|
||||
|
||||
</details>
|
||||
|
||||
All Android binaries link to `libc++_shared.so`. Remember to put a copy alongside the binaries.
|
||||
|
||||
## iOS
|
||||
|
||||
Apple does not normally provide shell access and `sshd` on their operating system, so jailbreaking the system is required.
|
||||
|
||||
Only a copy of the iOS sysroot is required to cross-compile, so Linux and Windows can also try this.
|
||||
|
||||
<details>
|
||||
<summary>.cargo/config.toml</summary>
|
||||
|
||||
```toml
|
||||
[target.aarch64-apple-ios]
|
||||
# Runner script is written in powershell
|
||||
runner = ["pwsh", "-File", "<runner location>run-on-ios.ps1"]
|
||||
rustflags = [
|
||||
"-C", "linker=clang",
|
||||
"-C", "link-arg=-fuse-ld=lld",
|
||||
"-C", "link-arg=--target=aarch64-apple-ios",
|
||||
"-C", "link-arg=--sysroot=<sysroot location>",
|
||||
"-C", "link-arg=-miphoneos-version-min=<minversion>",
|
||||
"-C", "link-arg=-rpath",
|
||||
"-C", "link-arg=@executable_path/Frameworks",
|
||||
"-Lnative=<sysroot location>/usr/lib",
|
||||
"-Lframework=<sysroot location>/System/Library/Frameworks",
|
||||
]
|
||||
```
|
||||
|
||||
</details>
|
||||
|
||||
If you're on a platform other than macOS, set an environment variable of `SDKROOT` to your sysroot location.
|
||||
|
||||
This avoids `cc-rs` trying to run `xcrun`.
|
||||
|
||||
<details>
|
||||
<summary>run-on-ios.ps1</summary>
|
||||
|
||||
```powershell
|
||||
param(
|
||||
[string]$BinaryPath
|
||||
)
|
||||
|
||||
$SshTarget = "<target user>@<target host>"
|
||||
$BinaryName = Split-Path -Leaf $BinaryPath
|
||||
# Select one of the following paths depending on rootless or rootful jailbreak
|
||||
$RemoteDir = "/var/jb/var/mobile/runner"
|
||||
#$RemoteDir = "/var/root/runner"
|
||||
$RemotePath = "$RemoteDir/$BinaryName"
|
||||
|
||||
$EscapedArgs = $args | ForEach-Object {
|
||||
if ($_ -match " ") {
|
||||
"`"$_`""
|
||||
} else {
|
||||
$_
|
||||
}
|
||||
}
|
||||
|
||||
$LinuxArgs = $EscapedArgs -join " "
|
||||
$DeviceCmd = "./$BinaryName $LinuxArgs"
|
||||
|
||||
ssh $SshTarget "test -f $RemotePath"
|
||||
|
||||
if ($LASTEXITCODE -ne 0) {
|
||||
scp -q "$BinaryPath" "$SshTarget`:$RemotePath"
|
||||
$DeviceCmd = "chmod 755 $BinaryName && ldid -Sent.xml $BinaryName && $DeviceCmd"
|
||||
}
|
||||
|
||||
ssh $SshTarget "cd $RemoteDir && $DeviceCmd"
|
||||
exit $LASTEXITCODE
|
||||
```
|
||||
|
||||
</details>
|
||||
|
||||
Due to iOS security measures, a binary needs to contain specific entitlements to run without a container and access IOKit, so it is required to have the `ldid` tool installed on your target device to set the entitlements with `ent.xml`. It should be placed in your runner directory.
|
||||
|
||||
<details>
|
||||
<summary>ent.xml</summary>
|
||||
|
||||
```xml
|
||||
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
|
||||
<plist version="1.0">
|
||||
<dict>
|
||||
<key>platform-application</key>
|
||||
<true/>
|
||||
<key>com.apple.private.security.container-required</key>
|
||||
<false/>
|
||||
<key>com.apple.security.iokit-user-client-class</key>
|
||||
<array>
|
||||
<string>AGXCommandQueue</string>
|
||||
<string>AGXDevice</string>
|
||||
<string>AGXDeviceUserClient</string>
|
||||
<string>AGXSharedUserClient</string>
|
||||
<string>IOSurfaceRootUserClient</string>
|
||||
</array>
|
||||
</dict>
|
||||
</plist>
|
||||
```
|
||||
|
||||
</details>
|
||||
Binary file not shown.
|
Before Width: | Height: | Size: 5.5 KiB |
Reference in new issue
Block a user