MIDI input devices: live play-through and MIDI recording (#493)

* MIDI input devices: live play-through and MIDI recording

Keyboards and pads could not be used at all: there was no MIDI device
layer (G-02).

- crates/playback/src/midi.rs opens every MIDI input port through midir
  (ALSA sequencer, CoreMIDI, WinMM), follows devices that are plugged in or
  removed, and passes channel messages to the audio thread over a lock-free
  ring, each with the time it arrived.
- The mix engine plays them on the instruments of record-enabled and
  input-monitored MIDI and instrument tracks (crates/mix/src/live_midi.rs):
  notes at their sample offset, the sustain pedal holding note-offs back.
- While recording, the events are kept with their timeline position and
  become MIDI clips on the record-enabled tracks
  (crates/engine/src/io/midi_take.rs), with controllers, pitch bend and
  aftertouch. A pass can be MIDI only: the audio input is opened for audio
  tracks alone.
- A MIDI or instrument track chooses a device and a channel at its input
  selector (kept in the track's input route as `MIDI:<channel>:<device>`).
- Setup > Playback Engine lists the input ports with a switch each and, on
  request, what arrives from one; Setup > MIDI > MIDI Input Devices opens
  it. The switches are app preferences.

Note lifecycle:

- A strip keeps the keys and the sustain pedal of every owner (input port
  and channel) apart. A pitch held by two devices or channels sounds until
  the last one releases it, and one owner's pedal does not hold another's
  notes. A strip follows 16 owners at a time; further ones share the last
  place (documented in live_midi.rs).
- A full queue is never silent. An event that does not reach the audio
  thread (the 1,024-entry input ring or the 256 events a callback takes)
  marks its port; the audio thread then drops what it still has queued from
  that port and the engine releases the notes and pedal the port holds. A
  port that closes, a replaced feed and events skipped while nothing
  listens take the same path.
- A take records where input was dropped: the notes that port held end
  there, and the status line says the take is incomplete. Recorded events
  the UI thread did not collect in time are counted and reported too.
  Recorded notes are told apart by device as well as channel and pitch.

On Linux, live play was checked with a hardware keyboard (Akai MPK mini 3)
and recording with a virtual one (crates/playback/examples/midi_keys.rs),
before the note-lifecycle work; that part is covered by tests only. macOS
and Windows are type-checked only; there is no MIDI input on FreeBSD or
the web.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>

* midi real device tests

* midi device drop test

* midi device fix pr review comments

* MIDI input: review fixes for note ownership, lost input and takes

Live notes (crates/mix/src/live_midi.rs):

- An owner without a place of its own joins the shared place while it is
  in use, even when another place has come free. It could get a second
  place before, and the release of its key in the shared place was then
  not found: the note stayed on. The same held for the pedal.
- A change of the track's input releases the shared place when the input
  no longer takes any one of the owners in it, not only the first.
- The strip remembers which port and channel last set each controller.
  When that source loses its input or the track stops listening to it,
  the instrument's controllers are set again from the clips and from the
  sources that are left, so a bend from an unplugged device does not stay.
- When playback runs out under monitoring, the clips' sustain pedal goes
  up with their notes.

Capture (crates/playback):

- A loss carries its time. What a port sends after it is played and comes
  after the mark in the take; before, it was dropped and recorded ahead
  of the mark.
- The age of an event is counted at the transport speed.
- Loop recording writes a mark where the loop goes round, and a key played
  with the playhead at the end of the loop stays inside the loop.
- The end of a record pass is acknowledged by its number, so the late
  answer to an earlier pass does not end the wait for a later one.
- A connection the backend closes marks its port, also when a device with
  the same name is opened in its place.

Takes (crates/engine/src/io/midi_take.rs, crates/ui-egui):

- A note held into the next round of a loop is split there, also when its
  release is later on the timeline than its press.
- A track taken out of record during a pass keeps what it recorded.
- A pass whose end the audio engine did not confirm is reported as
  incomplete in the status line.
- An inactive record-enabled audio track no longer turns a MIDI pass into
  an audio one that is then refused.

docs/midi-input.md: Windows drivers that allow one program per device, and
the new status text.

* MIDI input: review follow-ups on loss order, controllers and loop takes

- Each port has its own time of loss. One port's later loss no longer
  takes along what another port sent after its own: that is played, and
  comes after that port's mark in the take.
- A retired connection is closed before its port is marked as lost. Marked
  first, a note-on delivered while it closed carried a later time than the
  loss and could be played after the release.
- The strip keeps the last controller value of every source, not only the
  newest. When one of two sources that set the same controller goes away,
  the controller goes back to the value of the one that stays. 64 values
  are kept; past that the oldest is forgotten.
- A take can say where the pass stopped (MidiTakeEvent::END), and the UI
  says so for loop passes. A key still down then ends at the stop in the
  round it is in: held over a round or more it covers the loop, where it
  used to end at the loop end of its first round.
- A pass that records MIDI does not start while the audio engine has not
  confirmed the end of the previous one. It takes the MIDI since its last
  callback with the state of the pass it knows, so a pass started and
  stopped meanwhile had its notes written at the old position.

* MIDI input: loss marks in the queue, connection gate, stop position

- A loss from a full queue is also written into the queue, ahead of the
  first event that fits again. The audio thread deals with it at that
  place. Before, a loss flagged after the callback had read the flags was
  seen one callback late and ended a note played after it. The flag for a
  loss already met in the queue is not a second loss.
- A connection is shut off under the feed's lock when it is closed
  (MidiFeed::close_port), and what its device thread still delivers is
  discarded. Dropping a connection joins that thread on ALSA; CoreMIDI
  (MIDIPortDispose) and WinMM (midiInStop, midiInReset, midiInClose) do not
  promise to, so the order of drop and mark is no longer relied on.
- When the controller table is full and forgets a value, the strip keeps
  the port and channel it came from. The controllers are still reset when
  that source goes away, where a bend pushed out of the table used to stay.
- A loop pass ends where the audio engine is. The larger of the engine's
  and the UI's position was used, and just after the loop went round the
  UI's is still near the loop end: a short held note covered the loop.
- The end of a pass waits for the audio engine by what the pass set out to
  record, not by the devices connected at Stop. A pass whose last device
  was unplugged before Stop left nothing pending, and a new pass could
  start before the engine was done with it.

* MIDI input: stop at the loop end, and passes owed by the audio engine

- A key played while the playhead stands at the end of the loop is the
  last thing of that round. It was written as the first of the next round,
  with a mark that the loop had gone round; a pass stopped right there
  never went round, and a note held to the stop grew a second part.
- The feed knows whether an ended pass is still owed by the audio engine
  (MidiFeed::pass_owed), and the UI asks it before a pass that records
  MIDI starts. The UI kept that itself, where no test could stall the
  audio callback against it.

* MIDI input: stop boundary, loop rounds, loss marks and shared owners

- A pass ends at a time on the feed's clock (MidiFeed::end_pass). Only
  what arrived up to it is written to the take, however late the audio
  thread hears of the Stop, and the callback takes the queue in once more
  after the command before it confirms: a note that arrived between its
  reading the queue and the Stop was missing from a confirmed take.
- The confirmation carries where the playhead stood (MidiFeed::stopped_at).
  A loop pass ends there; the player's position read afterwards could be
  a round further on, which the take knows nothing about. Without a
  confirmation no stop position is used.
- The take builder is told whether the pass looped (MidiPass). In a pass
  that did not, a release stamped before its key press (a late callback)
  is one short note, where it was split around the ends of the clip. The
  stop position is a parameter, not an entry that the event cap could cut.
- A pass records the tracks that were in record when it started, and only
  those: the same list the end of the pass waits by. A track put in record
  during the pass got what was played before that, without a wait.
- A short loop goes round several times in a callback: each entry is put
  in the round it arrived in, so two entries a round apart have one mark
  between them.
- A mark in the queue is stamped with the time of its loss. A port that
  overflowed, played on and was then closed keeps a mark for the close
  (the new note was held to the end of the take), and a loss told by the
  port's flag is not written again when its mark arrives.
- Live events stay in order of arrival when they are played: a release
  taken by a shorter callback was due before its note-on.
- A callback reads the transport speed once, for rendering and for the
  ages of the next callback's events.
- The shared owner place is left alone by a source that never joined it:
  its note-off no longer ends another owner's note there, and its loss no
  longer releases the place.
- When a live source goes away, the instrument's controllers are restored
  before the block's clip events are sent, so a clip controller at the
  start of that block keeps its new value.
- Tests that assert MIDI positions use a clock the test sets.

* MIDI input: pass boundaries, loop punch-in and review follow-ups

- A pass begun while the transport runs has a start time on the feed
  (MidiFeed::begin_pass): what is played after Record is written down
  also when the audio thread takes it in before it reads the command.
- A pass goes on through a transport stop until the command that ends it.
  The UI sends the two one after the other; when they reached two
  callbacks, a note played just before the stop was left out of a
  confirmed take.
- The end of a pass is asked for every entry, so a Stop that comes while
  the queue is being read still cuts off what follows it.
- A mark in the queue carries the time its events were dropped, kept apart
  from a later close of the port. An overflow before Stop is in the take
  when the port closes after it.
- A pass punched in while a loop plays is a loop take over that loop. In a
  line from the punch-in to a stop in the next round nothing fell inside
  the range.
- In a loop take with marks, only the marks say that the loop went round:
  a release stamped before its press within one round is one short note.
- A key pressed in the last samples of the range starts at the last tick
  the clip plays, not at the tick of its end.
- When the clips' controllers are sent again while stopped, their sustain
  pedal goes up again: no pedal-up of theirs would follow.
- The render step is taken after the commands, so a session at another
  sample rate is rendered at that rate from the callback that installs it.
- The stop position and the incomplete-take warning go by one answer from
  the audio engine.

* MIDI input: test that a callback renders at the speed it keeps

A speed change that arrives while a callback runs takes effect from the
next one: the callback renders at the speed it read, and the next callback
counts the ages of its events at that same speed.

* MIDI input: volume and pan go back to MIDI's defaults with their device

The reset sent when a live source goes away is Reset All Controllers,
which leaves the channel volume (CC 7) and the pan (CC 10) alone. A
device that turned the volume down to nothing and was unplugged left the
track silent. When the source that went away had set one of the two, and
neither the clips nor a source that is still there gives it a value, the
instrument is sent the value MIDI names as the default (100, and 64).
The same when the track stops playing live input.

* MIDI input: third round of review follow-ups

- The stop cutoff is one value on the feed, in force from the store that
  ends the pass and gone when the audio thread confirms it.
- The start of a pass holds for the whole pass: what was played before
  Record stays out also when it is taken in after the start command.
- What is played just before a Stop is put inside the range that ends at
  the stop, not on its end.
- A port closed while the queue is read is met through the mark of its
  overflow; the flag of that close, a callback later, is not a second
  loss (it ended a key the new connection played).
- An overflow and the loss it is have one time. Read apart, they could
  differ by a microsecond and count as two losses.
- A volume or pan the full controller table forgot goes back to its
  default when its own device goes, not when another does.
- The chase of the clips' controllers uses the track's MIDI delay, as
  playing them does.
- Recorded controllers in the last samples of the range stay where the
  clip plays; in a take without marks the press as played, not the note's
  start, says whether the loop went round; a take cut at the event cap
  does not use the stop position.
- The loop the MIDI of a punched-in pass goes to is the loop the audio
  engine plays: not a range too short to loop, nor one that Play From
  dropped.
- The audio input stops taking in before the wait for the MIDI.

* MIDI input: fourth round of review follow-ups

- A record pass goes on through the natural end of playback until the UI
  ends it, up to the time playback ran out: what was played as it reached
  the end is in the take, what follows is not.
- Before the end of a pass is confirmed, the device threads are kept out
  of the queue while it is read once more, so a message stamped before
  the Stop is not put in after the answer. The audio thread does not wait
  for a device thread that is in there.
- A later loss of a port is noted as dealt with only at the end of the
  callback: with two overflow marks of one port in the queue, the second
  was skipped and the key played between them had no end.
- Whether a pass was begun is read before when: a pass seen as begun has
  its own start time.
- The volume and the pan go back to their defaults when the source that
  set them last goes, also when the controller table forgot its value,
  and not when an earlier setter goes.
- A clip's note-off that is held back because a live key holds the pitch
  is remembered for the block: the key going up earlier in the block no
  longer ends the note before the clip would have.
- In a take without marks, whether the loop went round between a press
  and its release is judged by their samples, not their ticks.
- The stop position of a loop pass is not used when recorded entries did
  not reach the take: one may be the mark of a round.
- Loop recording over a selection too short to loop is recorded as a
  punch, as the transport plays it.

* MIDI input: fifth round of review follow-ups

- Whether a pass was begun is asked for every event: a pass begun while
  the callback is reading the queue records from when it began.
- The ages of a pass's last events are counted up to when the transport
  stopped. Counted up to the callback that takes them in, the time the
  playhead stood still moved them back.
- An overflow the port's flag told of keeps its time when an older mark
  in the queue shows a later close: its own mark may never come.
- What becomes of the clips' held back note-offs is settled after the
  block's live events. A key that goes down again before the clip's note
  would have ended is not ended by it, and with two clip notes of a pitch
  in a block the one that sounds when the key goes up lasts to its end.
- When the source that set the volume or the pan last goes and another
  one's value is sent again, that one is the last setter from then on.

* MIDI input: sixth round of review follow-ups

- A loss is followed whenever the transport runs, and written to the take
  by when it happened, like an event: a port lost in a pass that began
  while the queue was being read ends its notes in the take.
- A pass started and ended by the commands of one callback is still that
  pass in the last read of the queue: what was played in it is kept.
- When a live key takes a pitch again, the clips' held back note-offs
  from since the last key went up are sent where they were, so the
  instrument lets go of the clip's note before the new press.

---------

Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
This commit is contained in:
Alexandru CarausandClaude Opus 5.5 authored and GitHub committed 2026-11-19 08:37:06 +07:00
1 parent 0089a99b92
commit a02d892f83
38 files changed
+7021 -90

No files matched your search

Generated
+231 -13
View File
@@ -193,7 +193,7 @@ dependencies = [
"android-properties",
"bitflags 2.13.2",
"cc",
"jni",
"jni 0.22.4",
"libc",
"log",
"ndk",
@@ -203,6 +203,15 @@ dependencies = [
"thiserror 2.0.21",
]
[[package]]
name = "android-build"
version = "0.1.4"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "f9fc9904ad2ad097c3c1cfe2eacaaf0fc24710936fa9ed941cb310b7c6ed2ab7"
dependencies = [
"windows-sys 0.52.0",
]
[[package]]
name = "android-properties"
version = "0.2.2"
@@ -826,6 +835,12 @@ dependencies = [
"shlex 2.0.1",
]
[[package]]
name = "cesu8"
version = "1.1.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "6d43a04d8753f35258c91f8ec639f792891f748a1edbd759cf1dcea3382ad83c"
[[package]]
name = "cexpr"
version = "0.6.0"
@@ -1101,6 +1116,27 @@ dependencies = [
"objc2-core-foundation",
]
[[package]]
name = "coremidi"
version = "0.9.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "a57ede822fdaf19280cf1320a5a5d3a522c75c910d01750af1e8122b6ad2595b"
dependencies = [
"block2 0.6.2",
"core-foundation 0.10.1",
"core-foundation-sys",
"coremidi-sys",
]
[[package]]
name = "coremidi-sys"
version = "3.2.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "1a2b8c2cefa9a8f712213c5a1383ffe428efc8f1a1fd1e2f757be94daf7e256a"
dependencies = [
"core-foundation-sys",
]
[[package]]
name = "cpal"
version = "0.18.2"
@@ -1114,7 +1150,7 @@ dependencies = [
"dasp_sample",
"futures-channel",
"futures-util",
"jni",
"jni 0.22.4",
"js-sys",
"libc",
"mach2",
@@ -1503,6 +1539,16 @@ dependencies = [
"libloading 0.8.9",
]
[[package]]
name = "dlopen2"
version = "0.9.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "60768c353d76ba6dd1b6332a5dbbdd3bc2dddadc2935f4cc6a9d0f17ca8ecb6a"
dependencies = [
"libc",
"windows-sys 0.61.2",
]
[[package]]
name = "document-features"
version = "0.2.12"
@@ -2643,6 +2689,33 @@ version = "1.0.18"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "8f42a60cbdf9a97f5d2305f08a87dc4e09308d1276d28c869c684d7777685682"
[[package]]
name = "java-locator"
version = "0.1.9"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "09c46c1fe465c59b1474e665e85e1256c3893dd00927b8d55f63b09044c1e64f"
dependencies = [
"glob",
]
[[package]]
name = "jni"
version = "0.21.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "1a87aa2bb7d2af34197c04845522473242e1aa17c12f4935d5856491a7fb8c97"
dependencies = [
"cesu8",
"cfg-if",
"combine",
"java-locator",
"jni-sys 0.3.1",
"libloading 0.7.4",
"log",
"thiserror 1.0.69",
"walkdir",
"windows-sys 0.45.0",
]
[[package]]
name = "jni"
version = "0.22.4"
@@ -2673,6 +2746,21 @@ dependencies = [
"syn 2.0.119",
]
[[package]]
name = "jni-min-helper"
version = "0.3.4"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "913370a00ba1851b0f3369b49d0f3e745da2d20d3f15ecc296a64234ccda45f7"
dependencies = [
"android-build",
"dlopen2",
"jni 0.21.1",
"libc",
"log",
"ndk-context",
"process_path",
]
[[package]]
name = "jni-sys"
version = "0.3.1"
@@ -2793,6 +2881,16 @@ dependencies = [
"cc",
]
[[package]]
name = "libloading"
version = "0.7.4"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "b67380fd3b2fbe7527a606e18729d21c6f3951633d0500574c4dc22d2d638b9f"
dependencies = [
"cfg-if",
"winapi",
]
[[package]]
name = "libloading"
version = "0.8.9"
@@ -2919,6 +3017,27 @@ dependencies = [
"autocfg",
]
[[package]]
name = "midir"
version = "0.11.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "fea58b9250a667b032ff61fb1bf77aed74873152563f5cd613016dce4c821832"
dependencies = [
"alsa",
"cc",
"coremidi",
"jni 0.21.1",
"jni-min-helper",
"js-sys",
"libc",
"log",
"ndk-context",
"parking_lot",
"wasm-bindgen",
"web-sys",
"windows",
]
[[package]]
name = "minimal-lexical"
version = "0.2.1"
@@ -3941,6 +4060,16 @@ dependencies = [
"unicode-ident",
]
[[package]]
name = "process_path"
version = "0.1.4"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "f676f11eb0b3e2ea0fbaee218fa6b806689e2297b8c8adc5bf73df465c4f6171"
dependencies = [
"libc",
"winapi",
]
[[package]]
name = "profiling"
version = "1.0.18"
@@ -4400,7 +4529,7 @@ checksum = "1167586491e2b18b8bfbb293e8180ec17c201c4f076d7cb3070ca964e7598f98"
dependencies = [
"core-foundation 0.10.1",
"core-foundation-sys",
"jni",
"jni 0.22.4",
"log",
"once_cell",
"rustls",
@@ -4926,6 +5055,7 @@ dependencies = [
"cpal",
"craft-audio",
"log",
"midir",
"serde",
"serde_json",
"soundcraft-dsp",
@@ -6043,7 +6173,7 @@ version = "1.2.5"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "d2c8ddfbbf66e501434657400268cefe51b95140a0266236415cda7627732df7"
dependencies = [
"jni",
"jni 0.22.4",
"log",
"ndk-context",
"objc2 0.6.5",
@@ -6251,6 +6381,22 @@ dependencies = [
"web-sys",
]
[[package]]
name = "winapi"
version = "0.3.9"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "5c839a674fcd7a98952e593242ea400abe93992746761e38641405d28b00f419"
dependencies = [
"winapi-i686-pc-windows-gnu",
"winapi-x86_64-pc-windows-gnu",
]
[[package]]
name = "winapi-i686-pc-windows-gnu"
version = "0.4.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "ac3b87c63620426dd9b991e5ce0329eff545bccbbb34f3be09ff6fb6ab51b7b6"
[[package]]
name = "winapi-util"
version = "0.1.11"
@@ -6260,6 +6406,12 @@ dependencies = [
"windows-sys 0.61.2",
]
[[package]]
name = "winapi-x86_64-pc-windows-gnu"
version = "0.4.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "712e227841d057c1ee1cd2fb22fa7e5a5461ae8e48fa2ca79ec42cfc1931183f"
[[package]]
name = "windows"
version = "0.62.2"
@@ -6372,13 +6524,22 @@ dependencies = [
"windows-link",
]
[[package]]
name = "windows-sys"
version = "0.45.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "75283be5efb2831d37ea142365f009c02ec203cd29a3ebecbc093d52315b66d0"
dependencies = [
"windows-targets 0.42.2",
]
[[package]]
name = "windows-sys"
version = "0.52.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "282be5f36a8ce781fad8c8ae18fa3f9beff57ec1b52cb3de0789201425d9a33d"
dependencies = [
"windows-targets",
"windows-targets 0.52.6",
]
[[package]]
@@ -6387,7 +6548,7 @@ version = "0.59.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "1e38bc4d79ed67fd075bcc251a1c39b32a1776bbe92e5bef1f0bf1f8c531853b"
dependencies = [
"windows-targets",
"windows-targets 0.52.6",
]
[[package]]
@@ -6399,20 +6560,35 @@ dependencies = [
"windows-link",
]
[[package]]
name = "windows-targets"
version = "0.42.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "8e5180c00cd44c9b1c88adb3693291f1cd93605ded80c250a75d472756b4d071"
dependencies = [
"windows_aarch64_gnullvm 0.42.2",
"windows_aarch64_msvc 0.42.2",
"windows_i686_gnu 0.42.2",
"windows_i686_msvc 0.42.2",
"windows_x86_64_gnu 0.42.2",
"windows_x86_64_gnullvm 0.42.2",
"windows_x86_64_msvc 0.42.2",
]
[[package]]
name = "windows-targets"
version = "0.52.6"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "9b724f72796e036ab90c1021d4780d4d3d648aca59e491e6b98e725b84e99973"
dependencies = [
"windows_aarch64_gnullvm",
"windows_aarch64_msvc",
"windows_i686_gnu",
"windows_aarch64_gnullvm 0.52.6",
"windows_aarch64_msvc 0.52.6",
"windows_i686_gnu 0.52.6",
"windows_i686_gnullvm",
"windows_i686_msvc",
"windows_x86_64_gnu",
"windows_x86_64_gnullvm",
"windows_x86_64_msvc",
"windows_i686_msvc 0.52.6",
"windows_x86_64_gnu 0.52.6",
"windows_x86_64_gnullvm 0.52.6",
"windows_x86_64_msvc 0.52.6",
]
[[package]]
@@ -6424,18 +6600,36 @@ dependencies = [
"windows-link",
]
[[package]]
name = "windows_aarch64_gnullvm"
version = "0.42.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "597a5118570b68bc08d8d59125332c54f1ba9d9adeedeef5b99b02ba2b0698f8"
[[package]]
name = "windows_aarch64_gnullvm"
version = "0.52.6"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "32a4622180e7a0ec044bb555404c800bc9fd9ec262ec147edd5989ccd0c02cd3"
[[package]]
name = "windows_aarch64_msvc"
version = "0.42.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "e08e8864a60f06ef0d0ff4ba04124db8b0fb3be5776a5cd47641e942e58c4d43"
[[package]]
name = "windows_aarch64_msvc"
version = "0.52.6"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "09ec2a7bb152e2252b53fa7803150007879548bc709c039df7627cabbd05d469"
[[package]]
name = "windows_i686_gnu"
version = "0.42.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "c61d927d8da41da96a81f029489353e68739737d3beca43145c8afec9a31a84f"
[[package]]
name = "windows_i686_gnu"
version = "0.52.6"
@@ -6448,24 +6642,48 @@ version = "0.52.6"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "0eee52d38c090b3caa76c563b86c3a4bd71ef1a819287c19d586d7334ae8ed66"
[[package]]
name = "windows_i686_msvc"
version = "0.42.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "44d840b6ec649f480a41c8d80f9c65108b92d89345dd94027bfe06ac444d1060"
[[package]]
name = "windows_i686_msvc"
version = "0.52.6"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "240948bc05c5e7c6dabba28bf89d89ffce3e303022809e73deaefe4f6ec56c66"
[[package]]
name = "windows_x86_64_gnu"
version = "0.42.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "8de912b8b8feb55c064867cf047dda097f92d51efad5b491dfb98f6bbb70cb36"
[[package]]
name = "windows_x86_64_gnu"
version = "0.52.6"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "147a5c80aabfbf0c7d901cb5895d1de30ef2907eb21fbbab29ca94c5b08b1a78"
[[package]]
name = "windows_x86_64_gnullvm"
version = "0.42.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "26d41b46a36d453748aedef1486d5c7a85db22e56aff34643984ea85514e94a3"
[[package]]
name = "windows_x86_64_gnullvm"
version = "0.52.6"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "24d5b23dc417412679681396f2b49f3de8c1473deb516bd34410872eff51ed0d"
[[package]]
name = "windows_x86_64_msvc"
version = "0.42.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "9aec5da331524158c6d1a4ac0ab1541149c0b9505fde06423b02f5ef0106b9f0"
[[package]]
name = "windows_x86_64_msvc"
version = "0.52.6"
+7
View File
@@ -49,6 +49,13 @@ rustfft = "6"
# Session bundles (one file holding a session and its audio, for the browser): stored entries only.
zip = { version = "9", default-features = false }
cpal = "0.18"
# MIDI input devices. Pinned: it talks to the system's MIDI service and runs on its threads.
# Only soundcraft-playback depends on it, and only on Linux, macOS and Windows; there it builds
# alsa (already used by cpal), coremidi + coremidi-sys, or the `windows` crate (already in the
# tree). Its Android and web dependencies (jni, android-build, java-locator, dlopen2,
# process_path, wasm-bindgen) are in Cargo.lock because the lock file covers every target, and
# are never compiled for SoundCraft.
midir = "=0.11.1"
# Which file paths stored in a session may be opened.
craft-docpath = { version = "0.1.0", git = "https://github.com/storytold/craft-libs", rev = "ea24422ae9859dc5da398d0462bc87f8e02e0d10" }
+13 -2
View File
@@ -130,7 +130,8 @@ where things are.
per plugin.
- **MIDI:** instrument and MIDI tracks, a piano-roll MIDI editor with a velocity lane, an event
list, a score view, step input, quantize, transpose, real-time properties, Standard MIDI Files,
and Audio-to-MIDI from a sung or played melody.
and Audio-to-MIDI from a sung or played melody. MIDI keyboards play the instruments live and
are recorded into MIDI clips ([how](docs/midi-input.md)).
- **Recording:** punch in on a selection or on the fly, pre/post-roll, loop recording with a
playlist per take, input monitoring through the full channel strip, and autosave with recovery.
- **Time and tempo:** Bars|Beats, Min:Secs, Timecode (every rate, drop-frame included),
@@ -178,6 +179,16 @@ and **⌥⌘↑/↓** its timebase. **⇧⌃⌘↓/↑** adds/removes the active
**⇧⌘↓/↑** works too. Windows/Linux use **Ctrl** for **⌘**. The active row follows the
field you click. A new row starts with its count focused and selected; Enter creates and Escape cancels.
### MIDI keyboards
Plug a keyboard in, record-enable an instrument track (its **R** button) and play: the keyboard
is heard through the track's instrument, and a record pass turns what you play into a MIDI clip.
**Setup › Playback Engine…** lists the MIDI devices with a switch each (**Setup › MIDI › MIDI
Input Devices…** opens the same window), and a track's input selector chooses a device and a
channel. Tried on Linux so far; on macOS and Windows it builds but has not been tried with a
device, and there is no MIDI input on FreeBSD or the web. The guide is
[docs/midi-input.md](docs/midi-input.md).
### Logs
The desktop app writes its `log` records to standard error and to `logs/soundcraft.log` in the
@@ -250,7 +261,7 @@ about egui, so the interface could be swapped for another one.
| `soundcraft-model` | Compatibility re-export of `craft-scraft-format`'s schema, validation and JSON API |
| `soundcraft-mix` | The mix engine (realtime and offline share it) |
| `soundcraft-engine` | Commands, undo, editing, import/export, bounce |
| `soundcraft-playback` | Audio devices (CoreAudio, WASAPI, ALSA, WebAudio) and recording |
| `soundcraft-playback` | Audio devices (CoreAudio, WASAPI, ALSA, WebAudio), MIDI input devices and recording |
| `soundcraft-automation` | MCP server and control-channel client |
| `soundcraft-ui-egui` | The user interface |
+7 -4
View File
@@ -2,7 +2,7 @@
**Stage: pre-alpha** · next: alpha, ~5% and ~110–180 h away (the alpha gate's three failing workflows)
> **Last reviewed:** 2026-10-24 · **Last updated:** 2026-11-09 · **Change:** trivial (Frequency Shifter plugin, Command-click target priority and cursor, Command-click automation points, plugin editors on Linux/X11, Cancel Zoom Toggle, side panels in dock columns, Auto Pan stereo balance, Alt-click returns a plugin parameter to its default, Clip List Auto Rename, Strip Silence rectangles, Strip Silence Rename, Strip Silence Extract and Separate, Clip Looping dialog, AudioSuite Vari-Fi speed ramps, Auto Pan plugin, Subharmonic plugin, Ring Modulator plugin, Float32 file headroom) · **Target:** Avid Pro Tools Ultimate 2026.4.1
> **Last reviewed:** 2026-10-24 · **Last updated:** 2026-11-18 · **Change:** minor (MIDI input in the progress log; previously trivial: Frequency Shifter plugin, Command-click target priority and cursor, Command-click automation points, plugin editors on Linux/X11, Cancel Zoom Toggle, side panels in dock columns, Auto Pan stereo balance, Alt-click returns a plugin parameter to its default, Clip List Auto Rename, Strip Silence rectangles, Strip Silence Rename, Strip Silence Extract and Separate, Clip Looping dialog, AudioSuite Vari-Fi speed ramps, Auto Pan plugin, Subharmonic plugin, Ring Modulator plugin, Float32 file headroom) · **Target:** Avid Pro Tools Ultimate 2026.4.1
SoundCraft is a clean-room, pure-Rust digital audio workstation that aims at full parity with
Avid Pro Tools, and then beyond it: faster, open, scriptable and agent-controllable. This page is
@@ -36,8 +36,10 @@ Detail: [target-app-parity.md](docs/target-app-parity.md#readiness-by-audience).
**Why pre-alpha:** the core-workflow gate in [docs/roadmap.md](docs/roadmap.md#alpha-gate) fails.
Editing, mixing, bouncing, and saving and reopening work end to end. But a Pro Tools user can't
record from a chosen audio interface at a chosen buffer size (only the default devices, a fixed
512-sample block, and takes held in RAM). They can't record MIDI from a keyboard (there is no
MIDI device layer). And they can't bring a session in or hand one back (no AAF/OMF/`.ptx`).
512-sample block, and takes held in RAM). Recording MIDI from a keyboard is built but checked on
Linux only: macOS and Windows are type-checked and have not been tried with a device, and there
is no MIDI input on FreeBSD or the web. And they can't bring a session in or hand one back (no
AAF/OMF/`.ptx`).
Readiness (~40 %) sits right at the alpha bar, but a gate failure keeps the app pre-alpha. The
2026-10-07 estimate ("~65 % overall, 110–150 h to 100 %") counted feature presence. This one
measures readiness and adds hardware, localization and field bug reports.
@@ -103,7 +105,7 @@ Other languages shipped: 0 complete. Brazilian Portuguese covers the fixed UI st
Ranked; the alpha gate, the alpha checklist and the beta plan are in [docs/roadmap.md](docs/roadmap.md).
1. **Alpha blocker:** audio interfaces: device and buffer-size choice, duplex stream, ASIO; record to disk (G-01, G-06). 35–55 h.
2. **Alpha blocker:** MIDI devices: record from keyboards, play instruments live, output (G-02). 20–35 h.
2. **Alpha blocker:** MIDI devices: keyboard input, live play and recording are in (checked on Linux only; macOS and Windows type-checked, none on FreeBSD or the web); output, clock and loop/merge recording remain (G-02). 12–22 h.
3. **Alpha blocker:** AAF/OMF import and export (G-03). 50–90 h.
4. macOS plugin entitlement (#153), then AU editors and embedded CLAP editors on macOS (G-05). 1 h for the entitlement (needed for the mix row), then 10–15 h.
5. Land the open bug-fix PRs and clear the correctness backlog (G-04). 15–25 h.
@@ -122,6 +124,7 @@ mutated and truncated movies. Field reports from the first public week are track
## Progress log
- **2026-11-18 (MIDI input):** MIDI input devices: keyboards play record-enabled and input-monitored MIDI and instrument tracks live and are recorded into MIDI clips, with a device and channel choice per track. The device switches are in the MIDI inputs section of Setup › Playback Engine (Setup › MIDI › MIDI Input Devices opens the same window) and are kept in the app preferences, per computer, not in the session. Controllers, pitch bend and aftertouch reach the instrument live; a pitch held on two devices or channels sounds until both release it; a full input queue releases the notes of its device and marks the take incomplete. On Linux, live play was checked with a hardware keyboard (Akai MPK mini 3) and recording with a virtual one; macOS and Windows are type-checked only, and there is no MIDI input on FreeBSD or the web. Guide: `docs/midi-input.md`. No readiness percentage change until that is measured.
- **2026-11-09:** A built-in Frequency Shifter (`freq_shifter`) moves every frequency in the signal by the same number of hertz, from 0.01 Hz to 5 kHz: up, down, both ways at once, or down on the left and up on the right of a stereo track. The unwanted sideband stays at least 100 dB down from 20 Hz to 20 Hz below Nyquist at 44.1 to 96 kHz. Feedback sends the output round again for a trail of further shifts, Mix blends in the dry signal, and every change glides without a click. No readiness percentage change.
- **2026-11-08:** Command-click near an existing automation breakpoint keeps that point as the click target with every active tool, and existing fade and clip-gain controls retain their normal priority. Empty automation space shows a crosshair and an insertion hint while Command is held. The active tool's drag gestures stay unchanged. No readiness percentage change.
- **2026-11-05:** Command-click in an automation lane adds a point at the clicked time and value with the current editing tool (#1394), using the existing Pencil mapping. Command-click preserves the existing selection and also works on the fallback volume lanes of Master, Aux and VCA tracks. Plain clicks and clip contents stay unchanged, and one Undo removes the new point. No readiness percentage change.
+67 -1
View File
@@ -39,11 +39,28 @@ impl EventQueue {
Self { ev: Vec::with_capacity(MAX_EVENTS) }
}
/// Queue an event for the next block. A full queue drops it, unless it ends notes (a note
/// off, all notes off, the sustain pedal going up): a release takes the place of the last
/// queued event that is not one, because a dropped release leaves a voice on for good while
/// a dropped note-on or controller is only missed. Should the queue hold nothing but
/// releases, the last one becomes All Notes Off, which covers the new one too.
pub(crate) fn push(&mut self, offset: usize, kind: EventKind) {
if self.ev.len() < MAX_EVENTS {
self.ev.push(Event { offset, kind });
} else {
return;
}
let releases = |k: &EventKind| matches!(k, EventKind::Off(_) | EventKind::AllOff | EventKind::Sustain(false));
if !releases(&kind) {
log::warn!("instrument event queue full; dropping event");
return;
}
match self.ev.iter_mut().rev().find(|e| !releases(&e.kind)) {
Some(e) => *e = Event { offset, kind },
None => {
if let Some(last) = self.ev.last_mut() {
last.kind = EventKind::AllOff;
}
}
}
}
@@ -527,3 +544,52 @@ impl Plugin for SubtractiveSynth {
self.events.push(0, EventKind::AllOff);
}
}
#[cfg(test)]
mod queue_tests {
use super::*;
#[test]
fn a_full_event_queue_still_takes_releases() {
let mut q = EventQueue::new();
for _ in 0..MAX_EVENTS {
q.push(7, EventKind::Bend(1.0));
}
// Not a release: dropped. Releases: each takes the place of a queued bend.
q.push(9, EventKind::On(60, 100));
q.push(11, EventKind::Off(60));
q.push(12, EventKind::Sustain(false));
q.push(13, EventKind::AllOff);
assert_eq!(q.ev.len(), MAX_EVENTS, "never grows on the audio thread");
assert!(!q.ev.iter().any(|e| matches!(e.kind, EventKind::On(..))));
let kept: Vec<usize> = q.ev.iter().filter(|e| !matches!(e.kind, EventKind::Bend(_))).map(|e| e.offset).collect();
assert_eq!(kept.len(), 3);
assert!([11, 12, 13].iter().all(|o| kept.contains(o)), "{kept:?}");
// Nothing but releases queued: the new one is covered by an All Notes Off.
let mut q = EventQueue::new();
for n in 0..MAX_EVENTS {
q.push(0, EventKind::Off((n % 100) as u8));
}
q.push(0, EventKind::Off(120));
assert!(matches!(q.ev.last().map(|e| e.kind), Some(EventKind::AllOff)));
}
#[test]
fn a_note_off_behind_a_flood_of_controllers_still_ends_the_note() {
let mut synth = crate::create("subtractive_synth").unwrap();
synth.prepare(48_000.0, 512, 2);
let mut io = vec![vec![0.0f32; 512]; 2];
synth.note_on(0, 60, 100);
synth.process(&mut io, 512);
assert!(io[0].iter().any(|x| x.abs() > 0.01));
for _ in 0..MAX_EVENTS + 100 {
synth.midi(0, [0xE0, 0, 64]);
}
synth.note_off(0, 60);
for _ in 0..200 {
io.iter_mut().for_each(|c| c.fill(0.0));
synth.process(&mut io, 512);
}
assert!(io[0].iter().all(|x| x.abs() < 1e-4), "the voice was left on");
}
}
+54 -1
View File
@@ -201,7 +201,15 @@ pub fn specs() -> Vec<CommandSpec> {
Ok(json!({"clips": ids}))
}
),
cmd!("track.input", "Track Input", [], None, "{tracks?, input: none|bus name|hardware name}", has_selection, |e, p| route(e, p, true)),
cmd!(
"track.input",
"Track Input",
[],
None,
"{tracks?, input: none|bus name|hardware name|MIDI:<channel 1-16 or *>:<device or *> (MIDI and instrument tracks)}",
has_selection,
|e, p| route(e, p, true)
),
cmd!("track.output", "Track Output", [], None, "{tracks?, output: main|none|bus name}", has_selection, |e, p| route(e, p, false)),
cmd!("track.timebase", "Track Timebase", [], None, "{tracks?, ticks: bool}", has_selection, |e, p| flag(
e,
@@ -1024,6 +1032,8 @@ fn route(e: &mut Engine, p: &Value, input: bool) -> Result<Value> {
if let Route::Hardware(name) = &r
&& create_bus
&& !name.to_ascii_lowercase().starts_with("in ")
// A MIDI input (device and channel) of a MIDI or instrument track, see `MidiInput`.
&& !name.starts_with("MIDI:")
{
let fmt = tracks
.first()
@@ -1046,6 +1056,49 @@ fn route(e: &mut Engine, p: &Value, input: bool) -> Result<Value> {
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn a_midi_input_is_kept_as_typed_and_never_becomes_a_bus() {
use soundcraft_model::MidiInput;
let mut e = Engine::default();
let id = e.execute("track.new", &json!({"name": "Keys", "kind": "instrument"})).unwrap()["tracks"][0].as_u64().unwrap();
let mut set = |input: &str| {
e.execute("track.input", &json!({"tracks": [id], "input": input})).unwrap();
assert_eq!(e.session().busses.len(), 0, "{input}: no bus is made for a MIDI input");
e.session().track(soundcraft_model::TrackId(id)).unwrap().mixer.input.clone()
};
// (what is typed, the device and channel the track listens to)
let cases: [(&str, Option<&str>, Option<u8>); 12] = [
("MIDI:*:*", None, None),
("MIDI:1:*", None, Some(0)),
("MIDI:16:Pad", Some("Pad"), Some(15)),
("MIDI:*:Keys", Some("Keys"), None),
// A device name may hold colons (ALSA names do): only the first one after the channel splits.
("MIDI:3:Launchkey MK3:MIDI 1", Some("Launchkey MK3:MIDI 1"), Some(2)),
// An empty device means every device.
("MIDI:5:", None, Some(4)),
("MIDI:*:", None, None),
// Channels outside 1–16 and other junk mean every channel; nothing fails.
("MIDI:0:Pad", Some("Pad"), None),
("MIDI:17:Pad", Some("Pad"), None),
("MIDI:-1:Pad", Some("Pad"), None),
("MIDI:abc:Pad", Some("Pad"), None),
// Without the second colon it is not a device choice: everything.
("MIDI:7", None, None),
];
for (typed, device, channel) in cases {
let route = set(typed);
assert_eq!(route, Route::Hardware(typed.to_string()), "kept as typed");
assert_eq!(MidiInput::of(&route), MidiInput { device, channel }, "{typed}");
}
// The route a choice is stored as reads back as the same choice.
let stored = MidiInput { device: Some("Launchkey MK3:MIDI 1"), channel: Some(9) }.route();
let Route::Hardware(text) = &stored else { panic!("not a hardware route") };
assert_eq!(MidiInput::of(&set(text)), MidiInput { device: Some("Launchkey MK3:MIDI 1"), channel: Some(9) });
// "none" takes the choice away: all devices, all channels.
assert_eq!(MidiInput::of(&set("none")), MidiInput::default());
}
#[test]
fn comp_from_alternate_playlist() {
let mut e = crate::demo::demo_engine();
+4
View File
@@ -14,6 +14,7 @@ use std::sync::atomic::{AtomicBool, Ordering};
mod bundle;
mod import;
mod midi;
mod midi_take;
mod record;
mod render;
mod session;
@@ -24,6 +25,9 @@ pub use bundle::{BUNDLE_EXTENSION, open_session_bytes, session_bundle};
pub use import::{DecodedImport, add_source, decode_import, import_audio_bytes, load_sampler_bytes, relink_videos};
pub(crate) use import::{MAX_IMPORT_SAMPLES, at_rate, place_decoded};
pub use midi::{export_midi, import_midi_bytes};
pub use midi_take::{
MAX_MIDI_TAKE_EVENTS, MidiPass, MidiTake, MidiTakeEvent, add_midi_recording, add_midi_recording_from, midi_record_inputs, records_midi,
};
pub use record::{Recording, Take, add_recording, live_span, place_recording};
pub use render::{commit_track, consolidate, refuse_longer, render_clip_gain, render_limit, sanitize_name};
pub(crate) use session::{DecodedFiles, decode_file, open_session_linked};
+742
View File
@@ -0,0 +1,742 @@
//! MIDI played during a record pass, turned into MIDI clips.
use super::{Clip, Engine, Range, Samples, Session, TrackId, TrackKind};
use soundcraft_midi::{CtrlEvent, CtrlKind, Note, Sequence};
use soundcraft_model::{ClipId, MidiInput, Route};
use std::collections::HashMap;
/// A MIDI message played during a record pass, at its timeline position; or, with the status
/// [`MidiTakeEvent::LOST`], the point where the input queues dropped events; or, with
/// [`MidiTakeEvent::WRAP`], the point where loop recording went back to the start of the loop.
/// Entries are in the order they were played.
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub struct MidiTakeEvent {
pub at: Samples,
/// Index into the take's device names.
pub device: u8,
pub status: u8,
pub data1: u8,
pub data2: u8,
}
impl MidiTakeEvent {
/// The `status` of an entry that marks dropped events: events of `device` (of every device
/// when it is `u8::MAX`) were lost at `at`. Their note-offs may be among them, so the notes
/// that device held end there.
pub const LOST: u8 = 0xFF;
/// The `status` of an entry that says loop recording went round: the entries after it were
/// played in the next round of the loop. A key held across it sounds to the end of the loop
/// and on from its start, also when its release is later on the timeline than its press.
pub const WRAP: u8 = 0xFE;
}
/// How a record pass went over its range.
#[derive(Debug, Clone, Copy, PartialEq, Eq, Default)]
pub enum MidiPass {
/// Once, from its start to its end. Positions only go forward: a release that is stamped
/// earlier than its key press (the audio callback ran late) ends the note at the press.
#[default]
Linear,
/// Loop recording: over the range again and again. `stopped` is where inside the loop the
/// pass stopped: the keys still down end there. When that is not known they count as held
/// to the end of the range. `marked`: the take has a [`MidiTakeEvent::WRAP`] entry at every
/// round, as the audio engine writes them, and only those say that the loop went round (a
/// release stamped before its press in one round is a late callback, as in a linear pass).
/// A take without those entries is not `marked`: there a release earlier on the timeline
/// than its key press is read as the loop having gone round.
Loop { stopped: Option<Samples>, marked: bool },
}
/// What a record pass made of the MIDI that was played.
#[derive(Debug, Clone, Default, PartialEq, Eq)]
pub struct MidiTake {
/// The clips made, one per track that received something.
pub clips: Vec<ClipId>,
/// Places within the pass where input was dropped ([`MidiTakeEvent::LOST`]): the take is
/// incomplete when this is not zero.
pub gaps: usize,
}
/// Whether a record pass would record MIDI: a record-enabled MIDI or instrument track exists.
pub fn records_midi(e: &Engine) -> bool {
e.session().tracks.iter().any(records_midi_on)
}
fn records_midi_on(t: &soundcraft_model::Track) -> bool {
t.mixer.record_arm && !t.inactive && matches!(t.kind, TrackKind::Instrument | TrackKind::Midi)
}
/// The MIDI kinds Setup › MIDI › Input Filter keeps out of recordings.
fn midi_filtered(s: &Session, status: u8) -> bool {
let kind = match status & 0xF0 {
0x80 | 0x90 => "notes",
0xA0 => "poly_aftertouch",
0xB0 => "cc",
0xC0 => "program",
0xD0 => "mono_aftertouch",
0xE0 => "pitch_bend",
_ => return true,
};
!s.edit.flags.is_empty() && s.edit.flag(&format!("midi.filter.{kind}"))
}
/// Whether a [`MidiTakeEvent::LOST`] entry for `lost` covers `device`.
fn lost_covers(lost: u8, device: u8) -> bool {
lost == u8::MAX || lost == device
}
/// What `track` recorded of `events` over `range`, in ticks from the start of the range. A key
/// already down when the range starts is left out; one still down at its end ends there, and one
/// whose device lost input ends where that happened. Keys are told apart by device, channel and
/// pitch: the same pitch held on two devices gives two notes. A key held while loop recording
/// goes round ([`MidiTakeEvent::WRAP`]) is two notes: to the end of the loop, and from its start;
/// that holds for a key still down when the pass stops too ([`MidiPass::Loop`]). In a pass that
/// did not loop ([`MidiPass::Linear`]) nothing is read as the loop going round.
fn midi_take_sequence(s: &Session, input: &MidiInput<'_>, range: Range, pass: MidiPass, events: &[MidiTakeEvent], devices: &[String]) -> Sequence {
let sr = s.sample_rate;
let base = s.tempo.samples_to_ticks(range.start, sr);
let tick = |at: Samples| s.tempo.samples_to_ticks(at, sr).saturating_sub(base).max(0);
let end_tick = tick(range.end).max(1);
let looped = matches!(pass, MidiPass::Loop { .. });
// Whether a release earlier on the timeline than its press says that the loop went round.
let inferred = matches!(pass, MidiPass::Loop { marked: false, .. });
// The last tick a note can start at and still be played: a key pressed within the last
// samples of the range may round to the tick of its end, where the clip no longer plays.
let last_start = end_tick.saturating_sub(1).max(0);
let mut seq = Sequence::default();
// The round of the loop being played: it goes up at every `WRAP`.
let mut round = 0u32;
// Keys down: (device, channel, pitch) -> (start tick, velocity, round, the sample the key was
// pressed at). Whether the loop went round between a press and its release is judged by
// their samples: two positions a few samples apart may share a tick, and a press in the
// last samples of the range starts its note a tick earlier so that it plays.
let mut down: HashMap<(u8, u8, u8), (i64, u8, u32, Samples)> = HashMap::new();
let close = |seq: &mut Sequence,
(_, channel, pitch): (u8, u8, u8),
(start, velocity, since, pressed): (i64, u8, u32, Samples),
end: i64,
release: u8,
round: u32,
end_at: Samples| {
let note = |start: i64, end: i64, release: u8| Note {
pitch,
velocity,
release_velocity: release,
channel,
start,
length: end.saturating_sub(start).max(1),
};
let rounds = round.saturating_sub(since);
// A pass in a line never goes round, and neither does a loop pass between two
// entries of one round when its rounds are marked: a release stamped before its
// press ends the note at the press (a note is at least a tick long).
if !looped || (rounds == 0 && (end_at >= pressed || !inferred)) {
seq.notes.push(note(start, end, release));
return;
}
// The pass went round between the key press and the release (loop recording goes back to
// the start of the loop): the capture said so, or the release is earlier on the timeline
// than the press. The note is held to the end of the pass and goes on from its start, as
// it was played. Held for a whole round or more, it covers the loop: the second note
// then ends where the first begins, so the two do not overlap.
seq.notes.push(note(start, end_tick, 64));
let end = if rounds > 1 || (rounds == 1 && end_at >= pressed) { start } else { end };
if end > 0 {
seq.notes.push(note(0, end, release));
}
};
for e in events.iter().take(MAX_MIDI_TAKE_EVENTS) {
// The mark sits at the end of the loop, which is the end of the range.
if e.status == MidiTakeEvent::WRAP {
round = round.saturating_add(1);
continue;
}
if e.at < range.start || e.at >= range.end {
continue;
}
let at = tick(e.at);
if e.status == MidiTakeEvent::LOST {
let ended: Vec<(u8, u8, u8)> = down.keys().copied().filter(|k| lost_covers(e.device, k.0)).collect();
for key in ended {
if let Some(open) = down.remove(&key) {
close(&mut seq, key, open, at, 64, round, e.at);
}
}
continue;
}
let (channel, d1, d2) = (e.status & 0x0F, e.data1 & 0x7F, e.data2 & 0x7F);
if midi_filtered(s, e.status) || !input.accepts(devices.get(usize::from(e.device)).map(String::as_str), channel) {
continue;
}
let key = (e.device, channel, d1);
// Like a note start, a controller in the last samples of the range stays where it plays.
let ctrl = |kind, value| CtrlEvent { tick: at.min(last_start), channel, kind, value };
match e.status & 0xF0 {
0x90 if d2 > 0 => {
// The same key again without a release: the first note ends here.
if let Some(open) = down.insert(key, (at.min(last_start), d2, round, e.at)) {
close(&mut seq, key, open, at, 64, round, e.at);
}
}
0x80 | 0x90 => {
if let Some(open) = down.remove(&key) {
close(&mut seq, key, open, at, if e.status & 0xF0 == 0x80 { d2 } else { 64 }, round, e.at);
}
}
0xA0 => seq.ctrls.push(ctrl(CtrlKind::PolyPressure(d1), i32::from(d2))),
0xB0 => seq.ctrls.push(ctrl(CtrlKind::Cc(d1), i32::from(d2))),
0xC0 => seq.ctrls.push(ctrl(CtrlKind::Program, i32::from(d1))),
0xD0 => seq.ctrls.push(ctrl(CtrlKind::ChannelPressure, i32::from(d1))),
0xE0 => seq.ctrls.push(ctrl(CtrlKind::PitchBend, (i32::from(d2) << 7 | i32::from(d1)) - 8192)),
_ => {}
}
}
// Still down when the pass stopped. A loop pass stops somewhere inside the loop: the keys end
// there, in the round the stop is in. Otherwise, or when nothing says where it stopped, they
// are held to the end of the range, in the round that was playing (after a round or more,
// that is the whole loop).
// A take cut at the event cap may have lost the marks of its last rounds: the stop position
// belongs to a round the entries that are left do not reach, and is not used.
let stop_at = match pass {
MidiPass::Loop { stopped: Some(at), .. } if events.len() <= MAX_MIDI_TAKE_EVENTS => Some(at.clamp(range.start, range.end)),
_ => None,
};
for (key, open) in down {
close(&mut seq, key, open, stop_at.map_or(end_tick, tick), 64, round, stop_at.unwrap_or(range.end));
}
seq.sort();
seq
}
/// Most events of one MIDI record pass that are kept.
pub const MAX_MIDI_TAKE_EVENTS: usize = 1 << 20;
/// Turn the MIDI played during a record pass that went once over `range` into a clip on every
/// MIDI and instrument track that is record-enabled now, each taking the events of its MIDI input
/// (device and channel). `devices` names the devices by [`MidiTakeEvent::device`]. A track that
/// received nothing gets no clip. One undo step ("Record"); with `same_pass` the clips join the
/// step the audio takes of this pass already made. The result says where input was dropped
/// during the pass.
pub fn add_midi_recording(e: &mut Engine, range: Range, events: &[MidiTakeEvent], devices: &[String], same_pass: bool) -> MidiTake {
let inputs = midi_record_inputs(e);
add_midi_recording_from(e, range, MidiPass::Linear, events, devices, &inputs, same_pass)
}
/// The MIDI inputs of the record-enabled MIDI and instrument tracks, to keep from the start of a
/// record pass and hand to [`add_midi_recording_from`] at its end.
pub fn midi_record_inputs(e: &Engine) -> Vec<(TrackId, Route)> {
e.session().tracks.iter().filter(|t| records_midi_on(t)).map(|t| (t.id, t.mixer.input.clone())).collect()
}
/// [`add_midi_recording`] for the tracks that were in record when the pass started, with the MIDI
/// inputs they had then (`inputs`, from [`midi_record_inputs`]), and for a pass that went over
/// `range` as `pass` says. What a pass records is settled when it starts: a track listed takes
/// the events of that input for the whole pass, so changing its device or channel while recording
/// does not throw away what it already recorded, and neither does taking it out of record. A
/// track that is not listed gets nothing, also when it was record-enabled during the pass: it
/// would get what was played before that, and the caller that waits for the end of the take
/// decides whether to wait by this same list.
pub fn add_midi_recording_from(
e: &mut Engine,
range: Range,
pass: MidiPass,
events: &[MidiTakeEvent],
devices: &[String],
inputs: &[(TrackId, Route)],
same_pass: bool,
) -> MidiTake {
let before = e.session_arc();
let mut out = MidiTake::default();
if range.len() <= 0 {
return out;
}
out.gaps =
events.iter().take(MAX_MIDI_TAKE_EVENTS).filter(|ev| ev.status == MidiTakeEvent::LOST && ev.at >= range.start && ev.at < range.end).count()
+ usize::from(events.len() > MAX_MIDI_TAKE_EVENTS);
let takes: Vec<(TrackId, String, Sequence)> = {
let s = e.session();
s.tracks
.iter()
.filter_map(|t| {
let (_, route) = inputs.iter().find(|(id, _)| *id == t.id)?;
Some((t.id, t.name.clone(), midi_take_sequence(s, &MidiInput::of(route), range, pass, events, devices)))
})
.filter(|(_, _, seq)| !seq.notes.is_empty() || !seq.ctrls.is_empty())
.collect()
};
for (tid, name, seq) in takes {
let s = e.session_mut();
let cid = s.new_clip_id();
crate::edit::place_clip(s, tid, Clip::midi(cid, name, range.start, range.len(), seq));
out.clips.push(cid);
}
if !out.clips.is_empty() && !same_pass {
e.push_undo("Record", before);
}
out
}
#[cfg(test)]
mod tests {
use super::*;
use serde_json::json;
use soundcraft_model::ClipContent;
fn armed(e: &mut Engine, name: &str, input: &str) -> TrackId {
let id = e.execute("track.new", &json!({"name": name, "kind": "instrument"})).unwrap()["tracks"][0].as_u64().unwrap();
e.execute("track.input", &json!({"tracks": [id], "input": input})).unwrap();
e.execute("mix.record_arm", &json!({"tracks": [id], "value": true})).unwrap();
TrackId(id)
}
fn ev(at: Samples, device: u8, status: u8, data1: u8, data2: u8) -> MidiTakeEvent {
MidiTakeEvent { at, device, status, data1, data2 }
}
fn sequence(e: &Engine, t: TrackId, range: Range) -> Sequence {
let clip = e.session().track(t).unwrap().clips().first().unwrap().clone();
assert_eq!(clip.range(), range);
let ClipContent::Midi { sequence } = clip.content else { panic!("not a MIDI clip") };
sequence
}
#[test]
fn midi_played_during_a_pass_becomes_a_clip_on_each_record_enabled_midi_track() {
let mut e = Engine::default();
let all = armed(&mut e, "Keys", "none");
let pad = armed(&mut e, "Pad", "MIDI:10:Pad");
let idle = armed(&mut e, "Idle", "MIDI:*:Nothing");
assert_eq!(e.session().busses.len(), 0, "a MIDI input is not a bus name");
assert!(records_midi(&e));
let events = [
ev(40_000, 0, 0x90, 59, 80), // before the pass: left out, and so is its release
ev(48_000, 0, 0x90, 60, 100), // one beat at 120 bpm, 48 kHz: 24 000 samples = 960 ticks
ev(50_000, 0, 0x80, 59, 0),
ev(60_000, 1, 0x99, 36, 127),
ev(72_000, 0, 0x80, 60, 30),
ev(72_000, 0, 0xE0, 0, 96),
ev(84_000, 1, 0x99, 36, 0),
ev(90_000, 0, 0xB0, 64, 127),
ev(95_000, 0, 0x90, 67, 90), // still down at the end
ev(96_000, 0, 0x90, 72, 90), // after the pass
];
let devices = ["Keys".to_string(), "Pad".to_string()];
let range = Range::new(48_000, 96_000);
let before = e.session().tracks.iter().map(|t| t.clips().len()).sum::<usize>();
let take = add_midi_recording(&mut e, range, &events, &devices, false);
assert_eq!((before, take.clips.len(), take.gaps), (0, 2, 0), "the track whose device was silent gets no clip");
let keys = sequence(&e, all, range);
let notes: Vec<(u8, u8, i64, i64, u8)> = keys.notes.iter().map(|n| (n.channel, n.pitch, n.start, n.length, n.release_velocity)).collect();
assert_eq!(notes, [(0, 60, 0, 960, 30), (9, 36, 480, 960, 64), (0, 67, 1880, 40, 64)]);
assert_eq!(
keys.ctrls.iter().map(|c| (c.tick, c.kind, c.value)).collect::<Vec<_>>(),
[(960, CtrlKind::PitchBend, 4096), (1680, CtrlKind::Cc(64), 127)]
);
let pads = sequence(&e, pad, range);
assert_eq!((pads.notes.len(), pads.notes[0].pitch, pads.ctrls.len()), (1, 36, 0), "only its device and channel");
assert!(e.session().track(idle).unwrap().clips().is_empty());
// One undo step for the pass; the input filter keeps kinds out.
e.run("edit.undo").unwrap();
assert_eq!(e.session().tracks.iter().map(|t| t.clips().len()).sum::<usize>(), 0);
e.execute("setup.midi_input_filter", &json!({"filter": ["pitch_bend", "cc"]})).unwrap();
add_midi_recording(&mut e, range, &events, &devices, false);
assert!(sequence(&e, all, range).ctrls.is_empty());
assert!(add_midi_recording(&mut e, Range::new(5, 5), &events, &devices, false).clips.is_empty());
}
#[test]
fn the_same_pitch_on_two_devices_is_two_notes_and_lost_input_ends_only_its_device() {
let mut e = Engine::default();
let all = armed(&mut e, "Keys", "none");
let range = Range::new(0, 96_000);
let events = [
ev(0, 0, 0x90, 60, 100),
ev(12_000, 1, 0x90, 60, 90), // the same pitch and channel from another device
ev(24_000, 0, 0x80, 60, 0), // ends the first device's note only
ev(24_000, 0, 0x90, 64, 100),
ev(36_000, 0, MidiTakeEvent::LOST, 0, 0), // device 0 lost input: its note-off never comes
ev(48_000, 1, 0x80, 60, 0),
ev(60_000, 1, 0x90, 67, 100),
ev(72_000, u8::MAX, MidiTakeEvent::LOST, 0, 0), // every device
ev(200_000, 0, MidiTakeEvent::LOST, 0, 0), // after the pass
];
let take = add_midi_recording(&mut e, range, &events, &["A".to_string(), "B".to_string()], false);
assert_eq!((take.clips.len(), take.gaps), (1, 2), "the take says it is incomplete");
let notes: Vec<(u8, i64, i64)> = sequence(&e, all, range).notes.iter().map(|n| (n.pitch, n.start, n.length)).collect();
assert_eq!(notes, [(60, 0, 960), (60, 480, 1440), (64, 960, 480), (67, 2400, 480)]);
}
/// `(channel, pitch, start, length, velocity, release velocity)` of every note.
fn notes_of(seq: &Sequence) -> Vec<(u8, u8, i64, i64, u8, u8)> {
seq.notes.iter().map(|n| (n.channel, n.pitch, n.start, n.length, n.velocity, n.release_velocity)).collect()
}
#[test]
fn note_edge_cases_make_the_notes_a_player_would_expect() {
// 120 bpm at 48 kHz: a quarter note is 24 000 samples and 960 ticks.
let s = Session::default();
let range = Range::new(0, 96_000);
let q = 24_000;
let cases: [(&str, Vec<MidiTakeEvent>, Vec<(u8, u8, i64, i64, u8, u8)>); 7] = [
(
"the same key again without a release: the first note ends where the second starts",
vec![ev(0, 0, 0x90, 60, 100), ev(q, 0, 0x90, 60, 90), ev(2 * q, 0, 0x80, 60, 0)],
vec![(0, 60, 0, 960, 100, 64), (0, 60, 960, 960, 90, 0)],
),
("a note-on with velocity 0 is a release", vec![ev(0, 0, 0x90, 60, 100), ev(q, 0, 0x90, 60, 0)], vec![(0, 60, 0, 960, 100, 64)]),
("a note-off keeps its release velocity", vec![ev(0, 0, 0x90, 60, 100), ev(q, 0, 0x80, 60, 30)], vec![(0, 60, 0, 960, 100, 30)]),
(
"data bytes above 127 are masked",
vec![ev(0, 0, 0x91, 0x80 | 60, 0x80 | 100), ev(q, 0, 0x81, 0x80 | 60, 0xFF)],
vec![(1, 60, 0, 960, 100, 127)],
),
("a key still down at the end of the pass ends there", vec![ev(q, 0, 0x90, 60, 100)], vec![(0, 60, 960, 2880, 100, 64)]),
("a release without a key down is nothing", vec![ev(q, 0, 0x80, 60, 0), ev(q, 0, 0x90, 61, 0)], vec![]),
(
"a note on and off at the same sample is one tick long, not empty",
vec![ev(q, 0, 0x90, 60, 100), ev(q, 0, 0x80, 60, 0)],
vec![(0, 60, 960, 1, 100, 0)],
),
];
for (what, events, want) in cases {
let seq = midi_take_sequence(&s, &MidiInput::default(), range, MidiPass::Linear, &events, &[]);
assert_eq!(notes_of(&seq), want, "{what}");
assert!(seq.notes.iter().all(|n| n.length > 0 && n.pitch < 128 && n.velocity < 128 && n.channel < 16), "{what}");
}
// Controller data is masked too: a pitch bend of two 0xFF bytes is full up, not more.
let seq =
midi_take_sequence(&s, &MidiInput::default(), range, MidiPass::Linear, &[ev(0, 0, 0xE0, 0xFF, 0xFF), ev(0, 0, 0xB0, 0x87, 0xFF)], &[]);
assert_eq!(seq.ctrls.iter().map(|c| (c.kind, c.value)).collect::<Vec<_>>(), [(CtrlKind::PitchBend, 8191), (CtrlKind::Cc(7), 127)]);
}
#[test]
fn a_note_held_across_the_loop_point_is_split_at_it() {
// A loop of four quarter notes (3840 ticks), recorded over several times: the timeline
// position goes back to the start, so a release can be earlier than its key press.
let s = Session::default();
let (q, range) = (24_000, Range::new(0, 96_000));
let events = [
ev(q / 2, 0, 0x90, 64, 100), // within one time round: an ordinary note
ev(q, 0, 0x80, 64, 0),
ev(3 * q + q / 2, 0, 0x90, 60, 100), // pressed half a quarter before the loop point…
ev(q / 2, 0, 0x80, 60, 40), // …released half a quarter after it
ev(3 * q, 0, 0x90, 67, 100), // pressed, then pressed again after the loop point
ev(q, 0, 0x90, 67, 90),
ev(2 * q, 0, 0x80, 67, 0),
ev(3 * q, 1, 0x90, 72, 100), // its device loses input after the loop point
ev(q, 1, MidiTakeEvent::LOST, 0, 0),
ev(3 * q, 0, 0x90, 48, 100), // released exactly at the loop start: nothing after it
ev(0, 0, 0x80, 48, 0),
];
let seq = midi_take_sequence(&s, &MidiInput::default(), range, MidiPass::Loop { stopped: None, marked: false }, &events, &[]);
let mut got = notes_of(&seq);
got.sort_by_key(|n| (n.1, n.2));
let want = [
(0, 48, 2880, 960, 100, 64),
(0, 60, 0, 480, 100, 40), // from the loop start to the release
(0, 60, 3360, 480, 100, 64), // from the key press to the loop point
(0, 64, 480, 480, 100, 0),
(0, 67, 0, 960, 100, 64), // the first press, carried over the loop point…
(0, 67, 960, 960, 90, 0), // …ends where the key is pressed again
(0, 67, 2880, 960, 100, 64),
(0, 72, 0, 960, 100, 64),
(0, 72, 2880, 960, 100, 64),
];
assert_eq!(got, want);
assert!(seq.notes.iter().all(|n| n.start >= 0 && n.start + n.length <= 3840), "every note is inside the loop");
}
#[test]
fn a_note_held_into_the_next_round_is_split_even_when_its_release_is_later_on_the_timeline() {
// The same loop. The capture says where it went round; positions alone cannot.
let s = Session::default();
let (q, range) = (24_000, Range::new(0, 96_000));
let wrap = ev(4 * q, u8::MAX, MidiTakeEvent::WRAP, 0, 0);
let events = [
ev(q / 2, 0, 0x90, 60, 100), // pressed at half a quarter in the first round…
ev(q, 0, 0x90, 64, 100), // (an ordinary note within the first round)
ev(2 * q, 0, 0x80, 64, 0),
ev(3 * q, 0, 0x90, 67, 100), // (held over two rounds)
wrap,
ev(q, 0, 0x80, 60, 40), // …released at one quarter in the second
ev(2 * q, 0, 0x90, 72, 100), // an ordinary note within the second round
ev(3 * q, 0, 0x80, 72, 0),
wrap,
ev(q, 0, 0x80, 67, 0),
ev(2 * q, 0, 0x90, 48, 100), // still down when the pass ends: to the end of the loop
];
let held = events;
let seq = midi_take_sequence(&s, &MidiInput::default(), range, MidiPass::Loop { stopped: None, marked: true }, &events, &[]);
let mut got = notes_of(&seq);
got.sort_by_key(|n| (n.1, n.2));
let want = [
(0, 48, 1920, 1920, 100, 64),
(0, 60, 0, 480, 100, 40), // from the loop start: up to its own start, so the two
(0, 60, 480, 3360, 100, 64), // parts cover what was held and do not overlap
(0, 64, 960, 960, 100, 0),
(0, 67, 0, 2880, 100, 0), // held for more than a round: the whole loop
(0, 67, 2880, 960, 100, 64),
(0, 72, 1920, 960, 100, 0),
];
assert_eq!(got, want);
assert!(seq.notes.iter().all(|n| n.start >= 0 && n.start + n.length <= 3840), "every note is inside the loop");
// Keys still down when the pass stops, after the loop went round. One was pressed at half
// a quarter in the first round, one at three quarters in the second.
let keys = [ev(q / 2, 0, 0x90, 60, 100), wrap, ev(3 * q, 0, 0x90, 64, 100), wrap];
let stopped = |at: i64| {
let mut got =
notes_of(&midi_take_sequence(&s, &MidiInput::default(), range, MidiPass::Loop { stopped: Some(at), marked: true }, &keys, &[]));
got.sort_by_key(|n| (n.1, n.2));
got.iter().map(|n| (n.1, n.2, n.3)).collect::<Vec<_>>()
};
// Stopped at one quarter in the third round: the first key was held for two rounds and
// covers the loop; the second went round once and is held from the loop start to the stop.
assert_eq!(stopped(q), [(60, 0, 480), (60, 480, 3360), (64, 0, 960), (64, 2880, 960)]);
// Stopped past both presses: both cover the loop, without overlapping themselves.
assert_eq!(stopped(3 * q + q / 2), [(60, 0, 480), (60, 480, 3360), (64, 0, 2880), (64, 2880, 960)]);
// Without the stop position a key still down counts as held to the end of the loop.
let got = notes_of(&midi_take_sequence(&s, &MidiInput::default(), range, MidiPass::Loop { stopped: None, marked: true }, &keys, &[]));
assert_eq!(got.len(), 4, "{got:?}");
// In one round a stop ends the key where it is, like the end of the range does.
let one_round = [ev(q, 0, 0x90, 60, 100)];
let got =
notes_of(&midi_take_sequence(&s, &MidiInput::default(), range, MidiPass::Loop { stopped: Some(2 * q), marked: true }, &one_round, &[]));
assert_eq!(got.iter().map(|n| (n.2, n.3)).collect::<Vec<_>>(), [(960, 960)]);
let events = held;
// The marks are not gaps: the take is complete.
let mut e = Engine::default();
armed(&mut e, "Keys", "none");
assert_eq!(add_midi_recording(&mut e, range, &events, &[], false).gaps, 0);
}
#[test]
fn a_release_stamped_before_its_press_in_a_pass_that_did_not_loop_is_one_short_note() {
// The audio callback ran late: the key press was written down at 1 024, and the release,
// taken by a later callback, at 672. Nothing went round.
let s = Session::default();
let range = Range::new(0, 96_000);
let events = [ev(24_000, 0, 0x90, 60, 100), ev(23_000, 0, 0x80, 60, 40), ev(48_000, 0, 0x90, 64, 100), ev(72_000, 0, 0x80, 64, 0)];
let got = notes_of(&midi_take_sequence(&s, &MidiInput::default(), range, MidiPass::Linear, &events, &[]));
assert_eq!(got, [(0, 60, 960, 1, 100, 40), (0, 64, 1920, 960, 100, 0)], "one note at the press, not two around the clip's ends");
// The same entries from a loop take without marks are a note held across the loop point.
let got = notes_of(&midi_take_sequence(&s, &MidiInput::default(), range, MidiPass::Loop { stopped: None, marked: false }, &events, &[]));
assert_eq!(got.iter().filter(|n| n.1 == 60).count(), 2);
// In a loop take with marks no mark lies between the two: a late callback again.
let got = notes_of(&midi_take_sequence(&s, &MidiInput::default(), range, MidiPass::Loop { stopped: None, marked: true }, &events, &[]));
assert_eq!(got, [(0, 60, 960, 1, 100, 40), (0, 64, 1920, 960, 100, 0)]);
// A key pressed in the last sample of the range starts at the last tick that plays.
let unmarked = MidiPass::Loop { stopped: None, marked: false };
// In a take without marks, a key pressed there and released a tick's worth earlier in
// the next round went round the loop: judged by where it was pressed, not where it starts.
let round = [ev(95_999, 0, 0x90, 60, 100), ev(95_975, 0, 0x80, 60, 0)];
let got = notes_of(&midi_take_sequence(&s, &MidiInput::default(), range, unmarked, &round, &[]));
assert_eq!(got.iter().map(|n| (n.2, n.3)).collect::<Vec<_>>(), [(0, 3839), (3839, 1)], "the tail and the round after it");
// Two positions within one tick: a release 22 samples before the press went round too,
// and a release 22 samples after it is one short note.
let near = [ev(24_011, 0, 0x90, 60, 100), ev(23_989, 0, 0x80, 60, 0)];
let got = notes_of(&midi_take_sequence(&s, &MidiInput::default(), range, unmarked, &near, &[]));
assert_eq!(got.iter().map(|n| (n.2, n.3)).collect::<Vec<_>>(), [(0, 960), (960, 2880)]);
let forward = [ev(23_989, 0, 0x90, 60, 100), ev(24_011, 0, 0x80, 60, 0)];
let got = notes_of(&midi_take_sequence(&s, &MidiInput::default(), range, unmarked, &forward, &[]));
assert_eq!(got.iter().map(|n| (n.2, n.3)).collect::<Vec<_>>(), [(960, 1)]);
// A controller there stays inside the clip too.
let cc = [ev(95_999, 0, 0xB0, 7, 90)];
let seq = midi_take_sequence(&s, &MidiInput::default(), range, MidiPass::Linear, &cc, &[]);
assert_eq!(seq.ctrls.iter().map(|c| c.tick).collect::<Vec<_>>(), [3839]);
let late = [ev(95_999, 0, 0x90, 60, 100)];
let got = notes_of(&midi_take_sequence(&s, &MidiInput::default(), range, MidiPass::Linear, &late, &[]));
assert_eq!(got.iter().map(|n| (n.2, n.3)).collect::<Vec<_>>(), [(3839, 1)], "not at the tick of the clip's end");
// A mark of the loop going round means nothing in a pass that did not loop.
let marked = [ev(24_000, 0, 0x90, 60, 100), ev(96_000, u8::MAX, MidiTakeEvent::WRAP, 0, 0), ev(48_000, 0, 0x80, 60, 0)];
let got = notes_of(&midi_take_sequence(&s, &MidiInput::default(), range, MidiPass::Linear, &marked, &[]));
assert_eq!(got.iter().map(|n| (n.2, n.3)).collect::<Vec<_>>(), [(960, 960)]);
}
#[test]
fn the_stop_position_of_a_loop_pass_is_not_one_of_the_capped_entries() {
// Exactly as many entries as a take keeps, the last a key that is still down at the stop.
let s = Session::default();
let range = Range::new(0, 96_000);
let mut events = vec![ev(-1, 0, 0xB0, 1, 0); MAX_MIDI_TAKE_EVENTS - 1];
events.push(ev(24_000, 0, 0x90, 60, 100));
let got =
notes_of(&midi_take_sequence(&s, &MidiInput::default(), range, MidiPass::Loop { stopped: Some(48_000), marked: true }, &events, &[]));
assert_eq!(got.iter().map(|n| (n.2, n.3)).collect::<Vec<_>>(), [(960, 960)], "the key ends at the stop, not at the loop end");
let mut e = Engine::default();
armed(&mut e, "Keys", "none");
let inputs = midi_record_inputs(&e);
let take = add_midi_recording_from(&mut e, range, MidiPass::Loop { stopped: Some(48_000), marked: true }, &events, &[], &inputs, false);
assert_eq!((take.clips.len(), take.gaps), (1, 0), "and a full take is not reported as cut");
// One entry more and the take is cut: the stop may be in a round whose mark was cut off,
// so the key is held to the end of the loop instead of ending early.
events.push(ev(96_000, u8::MAX, MidiTakeEvent::WRAP, 0, 0));
let got =
notes_of(&midi_take_sequence(&s, &MidiInput::default(), range, MidiPass::Loop { stopped: Some(48_000), marked: true }, &events, &[]));
assert_eq!(got.iter().map(|n| (n.2, n.3)).collect::<Vec<_>>(), [(960, 2880)]);
}
#[test]
fn a_track_taken_out_of_record_during_the_pass_keeps_what_it_recorded() {
let mut e = Engine::default();
let t = armed(&mut e, "Keys", "none");
let range = Range::new(0, 48_000);
let events = [ev(100, 0, 0x90, 60, 100), ev(200, 0, 0x80, 60, 0)];
let inputs = midi_record_inputs(&e);
e.execute("mix.record_arm", &json!({"tracks": [t.0], "value": false})).unwrap();
let take = add_midi_recording_from(&mut e, range, MidiPass::Linear, &events, &[], &inputs, false);
assert_eq!(take.clips.len(), 1, "it was record-enabled when the pass started");
assert_eq!(notes_of(&sequence(&e, t, range)).iter().map(|n| n.1).collect::<Vec<_>>(), [60]);
// A track that was not record-enabled at the start, and is not now, records nothing.
e.run("edit.undo").unwrap();
assert!(add_midi_recording_from(&mut e, range, MidiPass::Linear, &events, &[], &[], false).clips.is_empty());
// Nor does one that was put in record during the pass: it would get what was played
// before that, and nobody waited for the end of its take.
e.execute("mix.record_arm", &json!({"tracks": [t.0], "value": true})).unwrap();
assert!(add_midi_recording_from(&mut e, range, MidiPass::Linear, &events, &[], &[], false).clips.is_empty());
assert!(e.session().track(t).unwrap().clips().is_empty());
}
#[test]
fn a_track_records_the_input_it_had_when_the_pass_started() {
let mut e = Engine::default();
let t = armed(&mut e, "Keys", "MIDI:*:A");
let other = armed(&mut e, "Other", "MIDI:*:B");
let devices = ["A".to_string(), "B".to_string()];
let range = Range::new(0, 48_000);
let events = [ev(100, 0, 0x90, 60, 100), ev(200, 0, 0x80, 60, 0), ev(300, 1, 0x90, 72, 100), ev(400, 1, 0x80, 72, 0)];
let inputs = midi_record_inputs(&e);
assert_eq!(inputs.len(), 2);
// During the pass the first track is switched to device B.
e.execute("track.input", &json!({"tracks": [t.0], "input": "MIDI:*:B"})).unwrap();
let take = add_midi_recording_from(&mut e, range, MidiPass::Linear, &events, &devices, &inputs, false);
assert_eq!(take.clips.len(), 2);
assert_eq!(notes_of(&sequence(&e, t, range)).iter().map(|n| n.1).collect::<Vec<_>>(), [60], "what it recorded from A is kept");
assert_eq!(notes_of(&sequence(&e, other, range)).iter().map(|n| n.1).collect::<Vec<_>>(), [72]);
// The next pass uses the input as it is when that pass starts.
e.run("edit.undo").unwrap();
let inputs = midi_record_inputs(&e);
add_midi_recording_from(&mut e, range, MidiPass::Linear, &events, &devices, &inputs, false);
assert_eq!(notes_of(&sequence(&e, t, range)).iter().map(|n| n.1).collect::<Vec<_>>(), [72]);
}
#[test]
fn notes_keep_their_musical_place_at_any_tempo_and_sample_rate() {
for (hz, bpm) in [(44_100u32, 120.0), (48_000, 90.0), (96_000, 60.0), (192_000, 174.5)] {
let mut s = Session::new("t", soundcraft_time::SampleRate::new(hz).unwrap());
s.tempo.set_tempo(0, bpm).unwrap();
// The pass starts one quarter note in and lasts four.
let q = (f64::from(hz) * 60.0 / bpm).round() as i64;
let range = Range::new(q, 5 * q);
let events = [
ev(2 * q, 0, 0x90, 60, 100), // held to the end of the pass
ev(3 * q, 0, 0x90, 64, 100),
ev(4 * q, 0, 0x80, 64, 0),
];
let seq = midi_take_sequence(&s, &MidiInput::default(), range, MidiPass::Linear, &events, &[]);
let near = |got: i64, want: i64| (got - want).abs() <= 1;
let n = notes_of(&seq);
assert_eq!(n.len(), 2, "{hz} Hz, {bpm} bpm");
assert!(near(n[0].2, 960) && near(n[0].3, 2880), "{hz} Hz, {bpm} bpm: the held note ends with the pass: {n:?}");
assert!(near(n[1].2, 1920) && near(n[1].3, 960), "{hz} Hz, {bpm} bpm: {n:?}");
}
}
#[test]
fn each_input_filter_kind_keeps_its_own_messages_out_of_the_take() {
let events = [
ev(100, 0, 0x90, 60, 100),
ev(200, 0, 0x80, 60, 0),
ev(300, 0, 0xA0, 60, 50),
ev(400, 0, 0xB0, 1, 60),
ev(500, 0, 0xC0, 5, 0),
ev(600, 0, 0xD0, 70, 0),
ev(700, 0, 0xE0, 0, 80),
];
let range = Range::new(0, 48_000);
// What is left of (notes, poly pressure, controllers, programs, channel pressure, bends).
let kept = |filter: &[&str]| {
let mut e = Engine::default();
let t = armed(&mut e, "Keys", "none");
e.execute("setup.midi_input_filter", &json!({"filter": filter})).unwrap();
if add_midi_recording(&mut e, range, &events, &[], false).clips.is_empty() {
return [0; 6];
}
let seq = sequence(&e, t, range);
let ctrls = |f: &dyn Fn(&CtrlKind) -> bool| seq.ctrls.iter().filter(|c| f(&c.kind)).count();
[
seq.notes.len(),
ctrls(&|k| matches!(k, CtrlKind::PolyPressure(_))),
ctrls(&|k| matches!(k, CtrlKind::Cc(_))),
ctrls(&|k| matches!(k, CtrlKind::Program)),
ctrls(&|k| matches!(k, CtrlKind::ChannelPressure)),
ctrls(&|k| matches!(k, CtrlKind::PitchBend)),
]
};
assert_eq!(kept(&[]), [1; 6]);
for (i, kind) in ["notes", "poly_aftertouch", "cc", "program", "mono_aftertouch", "pitch_bend"].into_iter().enumerate() {
let mut want = [1; 6];
want[i] = 0;
assert_eq!(kept(&[kind]), want, "{kind}");
}
assert_eq!(kept(&["notes", "poly_aftertouch", "cc", "program", "mono_aftertouch", "pitch_bend"]), [0; 6], "nothing left: no clip");
assert_eq!(kept(&["sysex", "realtime"]), [1; 6], "kinds that never reach a take change nothing");
}
#[test]
fn midi_clips_join_the_undo_step_of_the_audio_takes_of_the_same_pass() {
let mut e = Engine::default();
let t = armed(&mut e, "Keys", "none");
let range = Range::new(0, 48_000);
let events = [ev(100, 0, 0x90, 60, 100), ev(200, 0, 0x80, 60, 0)];
let clips = |e: &Engine| e.session().track(t).unwrap().clips().len();
// On its own the take is one undo step called Record.
let before = e.undo_label().map(str::to_string);
assert_eq!(add_midi_recording(&mut e, range, &events, &[], false).clips.len(), 1);
assert_eq!(e.undo_label(), Some("Record"));
e.run("edit.undo").unwrap();
assert_eq!((clips(&e), e.undo_label().map(str::to_string)), (0, before.clone()));
// With audio takes of the same pass, it adds no step of its own: theirs covers it.
assert_eq!(add_midi_recording(&mut e, range, &events, &[], true).clips.len(), 1);
assert_eq!((clips(&e), e.undo_label().map(str::to_string)), (1, before));
// A take that makes no clip makes no step either.
let mut e = Engine::default();
armed(&mut e, "Keys", "MIDI:*:Nothing");
let before = e.undo_label().map(str::to_string);
assert!(add_midi_recording(&mut e, range, &events, &["Keys".to_string()], false).clips.is_empty());
assert_eq!(e.undo_label().map(str::to_string), before);
}
#[test]
fn a_device_index_without_a_name_reaches_only_tracks_that_take_every_device() {
let mut e = Engine::default();
let all = armed(&mut e, "All", "none");
let keys = armed(&mut e, "Keys", "MIDI:*:Keys");
let channel = armed(&mut e, "Ch 3", "MIDI:3:*");
let range = Range::new(0, 48_000);
// Device 200 is past the name list; `u8::MAX` is a port that never got an index.
let events = [ev(100, 200, 0x92, 60, 100), ev(200, 200, 0x82, 60, 0), ev(300, u8::MAX, 0x90, 62, 100), ev(400, u8::MAX, 0x80, 62, 0)];
let take = add_midi_recording(&mut e, range, &events, &["Keys".to_string()], false);
assert_eq!(take.clips.len(), 2);
assert_eq!(notes_of(&sequence(&e, all, range)).len(), 2);
assert_eq!(notes_of(&sequence(&e, channel, range)).iter().map(|n| n.1).collect::<Vec<_>>(), [60], "its channel, any device");
assert!(e.session().track(keys).unwrap().clips().is_empty(), "not the named device");
}
#[test]
fn events_past_the_limit_are_left_out_and_the_take_is_reported_incomplete() {
let mut e = Engine::default();
let t = armed(&mut e, "Keys", "none");
let range = Range::new(0, 48_000);
// The limit's worth of events from before the pass, then two notes: one inside the limit…
let mut events = vec![ev(-1, 0, 0xB0, 1, 0); MAX_MIDI_TAKE_EVENTS - 2];
events.extend([ev(100, 0, 0x90, 60, 100), ev(200, 0, 0x80, 60, 0)]);
let take = add_midi_recording(&mut e, range, &events, &[], false);
assert_eq!((take.clips.len(), take.gaps), (1, 0), "exactly the limit is complete");
e.run("edit.undo").unwrap();
// …and one past it.
events.extend([ev(300, 0, 0x90, 72, 100), ev(400, 0, 0x80, 72, 0)]);
let take = add_midi_recording(&mut e, range, &events, &[], false);
assert_eq!((take.clips.len(), take.gaps), (1, 1), "the cut is reported");
assert_eq!(notes_of(&sequence(&e, t, range)).iter().map(|n| n.1).collect::<Vec<_>>(), [60]);
}
}
+42 -10
View File
@@ -59,10 +59,12 @@ use soundcraft_model::{
use soundcraft_time::{Range, Samples, TICKS_PER_QUARTER};
use std::collections::HashMap;
use std::ops::Bound;
use std::sync::Arc;
mod automation;
mod clips;
mod instances;
mod live_midi;
mod meter;
mod midi;
mod offline;
@@ -73,6 +75,7 @@ mod threads;
use automation::{copy_with_gain, pan_at, pan_segment, send_level, transport_at, volume_level};
pub use instances::{INSTRUMENT_SLOT, InstanceSpec, NewInstance, PluginFailures, PreparedInstance, create_instance, instance_specs, is_third_party};
pub use live_midi::{LiveMidi, MAX_MIDI_PORTS, MAX_OWNERS, MidiPorts, midi_port_bit, midi_thru};
pub use meter::StripMeter;
use midi::sync_sample;
pub use offline::{
@@ -136,6 +139,14 @@ pub struct MixEngine {
aux_latency: usize,
/// Live input for the next block (planar), set by the audio host for input monitoring.
pub input: Vec<Vec<f32>>,
/// Live MIDI input for the next block, set by the audio host: record-enabled and
/// input-monitored MIDI and instrument tracks play it. Keep its capacity; `render` only reads.
pub midi: Vec<LiveMidi>,
/// Names of the ports the events in `midi` come from (for tracks that listen to one device).
pub midi_ports: Option<Arc<MidiPorts>>,
/// Ports ([`midi_port_bit`]) whose events were dropped before this block, set by the audio
/// host for one block: what they hold is released at its start, before `midi` is played.
pub midi_lost: u64,
/// Transport stopped: only monitored inputs sound (no clips).
pub monitor_only: bool,
/// Recording: record-armed tracks hear their input (auto input).
@@ -176,6 +187,9 @@ impl MixEngine {
aux_fed: Vec::new(),
aux_latency: 0,
input: Vec::new(),
midi: Vec::new(),
midi_ports: None,
midi_lost: 0,
monitor_only: false,
recording: false,
metronome: false,
@@ -258,6 +272,7 @@ impl MixEngine {
}
s.held.clear();
s.chase = true;
s.live.clear();
s.align.clear();
s.align_pre.clear();
s.out_extra.clear();
@@ -383,7 +398,7 @@ impl MixEngine {
}
}
let busses = std::mem::take(&mut self.busses);
let live_in = LiveInput { input: &self.input, monitor_only: self.monitor_only, recording: self.recording };
let live_in = self.live_input();
run_strips(&mut work, |(i, st)| {
if let Some(t) = s.tracks.get(*i) {
process_strip(s, t, st, &busses, &live_in, blk);
@@ -401,19 +416,36 @@ impl MixEngine {
self.write_output(s, blk, out);
}
/// The live audio and MIDI input of the block.
fn live_input(&self) -> LiveInput<'_> {
LiveInput {
input: &self.input,
midi: &self.midi,
ports: self.midi_ports.as_deref(),
midi_lost: self.midi_lost,
monitor_only: self.monitor_only,
recording: self.recording,
}
}
/// Silence what a seek leaves ringing, sync to the session and clear the block's accumulators.
fn begin_block(&mut self, s: &Session, pos: Samples, frames: usize) {
if pos != self.last_end {
// Seek: silence held notes and ringing plugins.
for st in self.strips.values_mut() {
if let Some(i) = &mut st.instrument {
i.plugin.all_notes_off();
// While stopped (monitoring only) the position stands still: that is not a seek, and
// notes played live keep sounding.
if !self.monitor_only {
if pos != self.last_end {
// Seek: silence held notes and ringing plugins.
for st in self.strips.values_mut() {
if let Some(i) = &mut st.instrument {
i.plugin.all_notes_off();
}
st.held.clear();
st.chase = true;
st.live.clear();
}
st.held.clear();
st.chase = true;
}
self.last_end = pos.saturating_add(frames as i64);
}
self.last_end = pos.saturating_add(frames as i64);
self.ensure_synced(s);
for b in self.busses.values_mut() {
for c in b.buf.iter_mut() {
@@ -503,7 +535,7 @@ impl MixEngine {
if st.scratch.len() < self.max_block {
st.scratch.resize(self.max_block, 0.0);
}
let live_in = LiveInput { input: &self.input, monitor_only: self.monitor_only, recording: self.recording };
let live_in = self.live_input();
process_strip(s, t, &mut st, &self.busses, &live_in, blk);
let ch = st.buf.len();
st.align.set(if pdc { lmax_a.saturating_sub(st.latency) } else { 0 }, ch);
File diff suppressed because it is too large. Load diff
+29 -5
View File
@@ -1,4 +1,5 @@
use super::clips::Key;
use super::live_midi::LiveNotes;
use super::strip::{PluginSlot, Strip};
use super::{ClipContent, CtrlKind, Range, SampleSource, Samples, Session, SourceAudio, Track};
use std::sync::Arc;
@@ -18,7 +19,12 @@ fn midi_offset(s: &Session, t: &Track) -> i64 {
}
/// Send note on/offs and controllers for MIDI clips overlapping the block.
pub(crate) fn schedule_notes(s: &Session, t: &Track, pos: Samples, frames: usize, inst: &mut PluginSlot, held: &mut Vec<u8>) {
///
/// `live` holds the notes played live on the same instrument. Instruments end notes by pitch, so
/// a clip note-off is not sent while a live key holds that pitch: the key's own release ends it.
/// A note-off held back that way is told to `live`: when the key goes up earlier in the same
/// block, the note still lasts until the clip's note-off.
pub(crate) fn schedule_notes(s: &Session, t: &Track, pos: Samples, frames: usize, inst: &mut PluginSlot, held: &mut Vec<u8>, live: &mut LiveNotes) {
let sr = s.sample_rate;
// MIDI Track Offsets: the track's notes sound `offset` samples later (earlier if negative), so
// the block looks that far back into the clips.
@@ -72,7 +78,12 @@ pub(crate) fn schedule_notes(s: &Session, t: &Track, pos: Samples, frames: usize
}
}
if off >= pos && off < end {
inst.plugin.note_off(usize::try_from(off - pos).unwrap_or(0), n.pitch);
let at = usize::try_from(off - pos).unwrap_or(0);
if live.holds(n.pitch) {
live.clip_off_held_back(n.pitch, at);
} else {
inst.plugin.note_off(at, n.pitch);
}
held.retain(|p| *p != n.pitch);
}
}
@@ -107,14 +118,23 @@ fn ctrl_message(kind: CtrlKind, value: i32, transpose: i64) -> [u8; 3] {
}
/// Before an instrument plays from `pos`: reset its controllers, then send each one the last value
/// it had on this track before `pos`.
pub(crate) fn chase_controllers(s: &Session, t: &Track, pos: Samples, inst: &mut PluginSlot) {
/// it had on this track before `pos`. Returns the control changes that were given a value, bit
/// `n` for CC `n`.
pub(crate) fn chase_controllers(s: &Session, t: &Track, pos: Samples, inst: &mut PluginSlot) -> u128 {
inst.plugin.midi(0, [0xB0, 121, 0]);
let pos = pos.saturating_sub(midi_offset(s, t));
const BEND: usize = 128;
const PRESSURE: usize = 129;
const PROGRAM: usize = 130;
// MIDI Real-Time Properties: the delay moves the track's controllers as it does when they
// are played (`schedule_notes`), so a value is in the past here when it was already sent.
let delay = if s.edit.values.is_empty() {
0i64
} else {
Key::new(format_args!("rtp.{}.delay", t.id.0)).map_or(0.0, |k| s.edit.value(k.as_str(), 0.0)).clamp(-1e7, 1e7) as i64
};
let mut last = [(i64::MIN, 0i32); 131];
let mut sent = 0u128;
let sr = s.sample_rate;
for clip in t.clips() {
let ClipContent::Midi { sequence } = &clip.content else { continue };
@@ -131,7 +151,7 @@ pub(crate) fn chase_controllers(s: &Session, t: &Track, pos: Samples, inst: &mut
CtrlKind::Program => PROGRAM,
CtrlKind::PolyPressure(_) => continue,
};
let at = s.tempo.tick_to_samples(base.saturating_add(c.tick), sr);
let at = s.tempo.tick_to_samples(base.saturating_add(c.tick.saturating_add(delay).max(0)), sr);
if at < until
&& let Some(l) = last.get_mut(slot)
&& at >= l.0
@@ -149,8 +169,12 @@ pub(crate) fn chase_controllers(s: &Session, t: &Track, pos: Samples, inst: &mut
};
if at != i64::MIN {
inst.plugin.midi(0, ctrl_message(kind, value, 0));
if slot < 128 {
sent |= 1u128 << slot;
}
}
}
sent
}
/// Session audio shared with a sample-playing instrument.
+64 -6
View File
@@ -1,6 +1,7 @@
use super::automation::{Level, apply_gain, transport_at, volume_level};
use super::clips::{ClipFx, clip_fx_params, render_clip};
use super::instances::{INSTRUMENT_SLOT, PluginFailures, PreparedInstance, create_instance_for, is_third_party};
use super::live_midi::{LiveMidi, LiveNotes, MidiPorts, midi_thru};
use super::midi::{chase_controllers, schedule_notes};
use super::pan::{PanSpec, apply_matrix, build_matrix, identity_matrix};
use super::routing::solo_muted;
@@ -41,8 +42,12 @@ pub(crate) struct Strip {
pub(crate) sample: Option<Arc<SourceAudio>>,
/// MIDI notes currently sounding on the instrument (pitch), for note-offs at stops/seek.
pub(crate) held: Vec<u8>,
/// Notes held by live MIDI input.
pub(crate) live: LiveNotes,
/// The instrument's next block starts somewhere new: send it the controllers in effect there.
pub(crate) chase: bool,
/// The last block played the clips: stopped monitoring has their sustain pedal to let go of.
pub(crate) clips_played: bool,
pub(crate) muted: bool,
pub(crate) gr: f32,
/// Per-clip effect chains (Clip Effects) and a scratch buffer to render clips into.
@@ -140,7 +145,9 @@ impl Strip {
instrument: None,
sample: None,
held: Vec::new(),
live: LiveNotes::default(),
chase: true,
clips_played: false,
muted: false,
gr: 0.0,
clip_fx: HashMap::new(),
@@ -205,6 +212,10 @@ impl SlotCtx<'_> {
/// Live input routed to monitored tracks.
pub(crate) struct LiveInput<'a> {
pub(crate) input: &'a [Vec<f32>],
pub(crate) midi: &'a [LiveMidi],
pub(crate) ports: Option<&'a MidiPorts>,
/// Ports whose events were dropped before this block.
pub(crate) midi_lost: u64,
pub(crate) monitor_only: bool,
pub(crate) recording: bool,
}
@@ -295,7 +306,7 @@ fn read_aux_bus(t: &Track, strip: &mut Strip, busses: &HashMap<BusId, BusBuf>, f
}
/// Render the track's instrument into the strip.
fn render_instrument(s: &Session, t: &Track, strip: &mut Strip, blk: Block, transport: &Transport) {
fn render_instrument(s: &Session, t: &Track, strip: &mut Strip, live: &LiveInput<'_>, blk: Block, transport: &Transport) {
let Block { pos, frames, .. } = blk;
if let Some(inst) = strip.instrument.as_mut().filter(|i| !i.failed) {
// #152: push the instrument's stored parameter values into the live plugin
@@ -307,10 +318,55 @@ fn render_instrument(s: &Session, t: &Track, strip: &mut Strip, blk: Block, tran
apply_params(inst, ins, t, INSTRUMENT_SLOT, pos);
}
inst.plugin.set_transport(transport);
if std::mem::take(&mut strip.chase) {
chase_controllers(s, t, pos, inst);
// Whether the clips' controller values were sent to the instrument for this block.
let mut chased = std::mem::take(&mut strip.chase);
if chased {
let from_clips = chase_controllers(s, t, pos, inst);
strip.live.restore_defaults(from_clips, inst.plugin.as_mut());
}
// Live sources that went away (a port that lost input, an input the track no longer
// listens to) let go first, before anything of this block reaches the instrument. The
// clip notes are the ones held when the block starts (told at the end of the last one).
// When such a source had moved the instrument's controllers and the track still plays
// live input, nothing else would set them back: the clips' values as they stand at the
// start of the block, then those of the devices that are still there. The clips' own
// events of this block come after that, so a controller a clip changes right at the
// start of the block keeps its new value.
strip.live.begin_block();
if midi_thru(t) && strip.live.reconcile(t, live.midi_lost, live.ports, inst.plugin.as_mut()) {
let from_clips = chase_controllers(s, t, pos, inst);
strip.live.restore_defaults(from_clips, inst.plugin.as_mut());
strip.live.resend_controllers(inst.plugin.as_mut());
chased = true;
}
if !live.monitor_only {
schedule_notes(s, t, pos, frames, inst, &mut strip.held, &mut strip.live);
strip.clips_played = true;
} else {
// Stopped: the clips are silent, only live MIDI input plays. Playback that ran to its
// end left the clips' notes without their note-offs: end them here, except a pitch a
// live key still holds.
for note in strip.held.drain(..) {
if !strip.live.holds(note) {
inst.plugin.note_off(0, note);
}
}
// The clips' sustain pedal goes up too (its own pedal-up will not be played), or the
// instrument would hold those notes for good. Live input keeps its pedal in
// `strip.live`, not on the instrument, so keys held live are not released by this.
// The same after the clips' controllers were sent again while stopped (a live device
// went away): a pedal the clips hold down there has no pedal-up coming either.
if std::mem::take(&mut strip.clips_played) || chased {
inst.plugin.midi(0, [0xB0, 64, 0]);
}
}
strip.live.set_clip_notes(&strip.held);
if midi_thru(t) {
strip.live.play(t, inst.plugin.as_mut(), live.midi, live.ports, frames);
} else if strip.live.touched() && strip.live.release(inst.plugin.as_mut()) {
// Live controllers moved the instrument: the next block sets them from the clips.
strip.chase = true;
}
schedule_notes(s, t, pos, frames, inst, &mut strip.held);
if strip.buf.len() >= 2 {
inst.plugin.process(&mut strip.buf, frames);
} else if let Some(c0) = strip.buf.first_mut() {
@@ -410,10 +466,12 @@ pub(crate) fn process_strip(s: &Session, t: &Track, strip: &mut Strip, busses: &
monitor_input(t, strip, live, frames);
}
TrackKind::Audio if live.monitor_only => {}
TrackKind::Instrument | TrackKind::Midi if live.monitor_only => {}
// Stopped: only a track that plays live MIDI input sounds (and one that just stopped
// doing so, to release its notes).
TrackKind::Instrument | TrackKind::Midi if live.monitor_only && !midi_thru(t) && !strip.live.touched() => {}
TrackKind::Audio => render_audio_clips(s, t, strip, blk),
TrackKind::Aux | TrackKind::Folder => read_aux_bus(t, strip, busses, frames),
TrackKind::Instrument | TrackKind::Midi => render_instrument(s, t, strip, blk, &transport),
TrackKind::Instrument | TrackKind::Midi => render_instrument(s, t, strip, live, blk, &transport),
_ => {}
}
let trim = db_to_gain(t.mixer.trim_db) * if t.mixer.phase_invert { -1.0 } else { 1.0 };
+529
View File
@@ -1606,3 +1606,532 @@ fn consolidate_many_tiny_effect_clips_stays_fast() {
assert!((out[0][1000] - 0.5 * db_to_gain(-6.0)).abs() < 1e-6);
assert_eq!(out[0][1001], 0.0);
}
fn peak(x: &[f32]) -> f32 {
x.iter().fold(0.0f32, |m, v| m.max(v.abs()))
}
/// Renders `blocks` stopped blocks, the events in the first one, and returns the last block's peak.
fn live_peak(eng: &mut MixEngine, s: &Session, events: &[(u8, u8, u8)], blocks: usize) -> f32 {
let mut out = vec![vec![0.0f32; 512]; 2];
eng.monitor_only = true;
eng.midi.clear();
eng.midi.extend(events.iter().map(|&(status, data1, data2)| LiveMidi { offset: 0, port: 0, status, data1, data2 }));
for _ in 0..blocks {
eng.render(s, 0, 512, &mut out);
eng.midi.clear();
eng.midi_lost = 0;
}
peak(&out[0])
}
#[test]
fn live_midi_plays_record_enabled_instruments_while_stopped() {
let mut s = synth_session(-6.0);
let t = s.tracks[0].id;
let mut eng = MixEngine::new(48_000.0, 512);
// Not record-enabled: the keyboard is not heard, and neither is the clip while stopped.
assert_eq!(live_peak(&mut eng, &s, &[(0x90, 60, 100)], 8), 0.0);
s.track_mut(t).unwrap().mixer.record_arm = true;
assert!(live_peak(&mut eng, &s, &[(0x90, 60, 100)], 8) > 0.01, "a held key sounds");
assert!(live_peak(&mut eng, &s, &[], 40) > 0.01, "and keeps sounding: a stopped transport is not a seek");
// The sustain pedal holds the note past its note-off (a note-on with velocity 0 here).
assert!(live_peak(&mut eng, &s, &[(0xB0, 64, 127), (0x90, 60, 0)], 40) > 0.01);
assert!(live_peak(&mut eng, &s, &[(0xB0, 64, 0)], 400) < 1e-4, "pedal up releases it");
// Taking the track out of record releases a held key.
assert!(live_peak(&mut eng, &s, &[(0x91, 64, 100)], 8) > 0.01);
s.track_mut(t).unwrap().mixer.record_arm = false;
assert!(live_peak(&mut eng, &s, &[], 400) < 1e-4);
}
#[test]
fn live_midi_honours_the_track_input_device_and_channel() {
let mut s = synth_session(-6.0);
let t = s.tracks[0].id;
let ports = Arc::new(MidiPorts::default());
ports.assign("Keys");
let mut eng = MixEngine::new(48_000.0, 512);
eng.midi_ports = Some(Arc::clone(&ports));
let tr = s.track_mut(t).unwrap();
tr.mixer.input_monitor = true;
tr.mixer.input = soundcraft_model::MidiInput { device: Some("Pad"), channel: None }.route();
assert_eq!(live_peak(&mut eng, &s, &[(0x90, 60, 100)], 8), 0.0, "another device");
s.track_mut(t).unwrap().mixer.input = soundcraft_model::MidiInput { device: Some("Keys"), channel: Some(1) }.route();
assert_eq!(live_peak(&mut eng, &s, &[(0x90, 60, 100)], 8), 0.0, "another channel");
assert!(live_peak(&mut eng, &s, &[(0x91, 60, 100)], 8) > 0.01);
}
#[test]
fn live_midi_sounds_on_top_of_the_clips_during_playback() {
let mut s = synth_session(-6.0);
s.tracks[0].mixer.record_arm = true;
let render = |events: &[LiveMidi]| {
let mut eng = MixEngine::new(48_000.0, 512);
let mut out = vec![vec![0.0f32; 512]; 2];
let mut all = Vec::new();
for b in 0..80 {
eng.midi.clear();
if b == 60 {
eng.midi.extend_from_slice(events);
}
eng.render(&s, b * 512, 512, &mut out);
all.extend_from_slice(&out[0]);
}
all
};
let (dry, played) = (render(&[]), render(&[LiveMidi { offset: 100, port: 0, status: 0x90, data1: 72, data2: 110 }]));
// The clip's note ends at 24 000 samples; block 60 starts at 30 720.
assert_eq!(dry[..30_720 + 100], played[..30_720 + 100], "nothing changes before the key goes down");
assert!(peak(&played[31_000..]) > peak(&dry[31_000..]) + 0.01);
}
#[test]
fn a_port_that_lost_events_lets_go_of_its_held_key_and_pedal() {
let mut s = synth_session(-6.0);
s.tracks[0].mixer.record_arm = true;
let mut eng = MixEngine::new(48_000.0, 512);
// A key and the pedal are down, then the queue drops the releases.
assert!(live_peak(&mut eng, &s, &[(0xB0, 64, 127), (0x90, 60, 100), (0x90, 64, 100), (0x80, 64, 0)], 8) > 0.01);
eng.midi_lost = midi_port_bit(7);
assert!(live_peak(&mut eng, &s, &[], 40) > 0.01, "another port's loss leaves them alone");
eng.midi_lost = midi_port_bit(0);
assert!(live_peak(&mut eng, &s, &[], 400) < 1e-4, "nothing is left sounding");
// The pedal went with them: a new note ends at its note-off.
assert!(live_peak(&mut eng, &s, &[(0x90, 60, 100)], 8) > 0.01);
assert!(live_peak(&mut eng, &s, &[(0x80, 60, 0)], 400) < 1e-4);
}
/// A record-enabled instrument track whose instrument writes down the MIDI messages it gets.
fn ears_session() -> (Session, TrackId, MixEngine, Arc<std::sync::Mutex<Vec<(usize, [u8; 3])>>>) {
let mut s = Session::default();
let t = s.add_track(TrackKind::Instrument, ChannelFormat::Stereo, None);
let tr = s.track_mut(t).unwrap();
tr.instrument = Some(Insert::new("clap:test.ears"));
tr.mixer.record_arm = true;
let heard = Arc::new(std::sync::Mutex::new(Vec::new()));
let ears = Ears { inner: soundcraft_dsp::create("gain").unwrap(), heard: Arc::clone(&heard) };
let mut eng = MixEngine::new(48_000.0, 512);
eng.set_external_instances(true);
eng.adopt(vec![PreparedInstance { track: t, slot: INSTRUMENT_SLOT, id: "clap:test.ears".into(), ch: 2, plugin: Box::new(ears) }]);
(s, t, eng, heard)
}
#[test]
fn live_controllers_bend_aftertouch_and_program_reach_the_instrument_at_their_offsets() {
let (mut s, t, mut eng, heard) = ears_session();
let ev = |offset, status, data1, data2| LiveMidi { offset, port: 0, status, data1, data2 };
eng.midi.extend([
ev(10, 0xB3, 1, 90), // mod wheel on channel 4: the channel is dropped
ev(200, 0xE0, 0x7F, 0x7F), // pitch bend, full up
ev(300, 0xD0, 55, 99), // channel pressure: one data byte
ev(301, 0xA0, 60, 70), // poly pressure
ev(400, 0xC1, 0xFF, 0xFF), // program change; a data byte above 127 is masked
ev(410, 0xB0, 64, 127), // sustain: handled by the host, not passed on
ev(420, 0xB0, 123, 0), // all notes off: handled by the host
ev(430, 0xB0, 126, 0), // another channel mode message: not passed on
ev(440, 0xF8, 0, 0), // not a channel message
ev(9_999, 0xB0, 7, 100), // an offset past the block lands on its last sample
]);
render_block(&mut eng, &s, 0);
let want = vec![
(0, [0xB0, 121, 0]), // the first block's controller reset
(10, [0xB0, 1, 90]),
(200, [0xE0, 0x7F, 0x7F]),
(300, [0xD0, 55, 0]),
(301, [0xA0, 60, 70]),
(400, [0xC0, 0x7F, 0]),
(511, [0xB0, 7, 100]),
];
assert_eq!(std::mem::take(&mut *heard.lock().unwrap()), want);
// Out of record: the controllers go back to what the clips say (a reset; there are none).
// The reset leaves the volume the device set, so that goes back to MIDI's default.
eng.midi.clear();
s.track_mut(t).unwrap().mixer.record_arm = false;
render_block(&mut eng, &s, 512);
assert!(heard.lock().unwrap().is_empty());
render_block(&mut eng, &s, 1024);
assert_eq!(std::mem::take(&mut *heard.lock().unwrap()), vec![(0, [0xB0, 121, 0]), (0, [0xB0, 7, 100])]);
render_block(&mut eng, &s, 1536);
assert!(heard.lock().unwrap().is_empty(), "once");
}
/// One playing block at `pos` with `events`; the left channel.
fn live_block(eng: &mut MixEngine, s: &Session, pos: i64, events: &[(u32, u8, u8, u8)]) -> Vec<f32> {
let mut out = vec![vec![0.0f32; 512]; 2];
eng.monitor_only = false;
eng.midi.clear();
eng.midi.extend(events.iter().map(|&(offset, status, data1, data2)| LiveMidi { offset, port: 0, status, data1, data2 }));
eng.render(s, pos, 512, &mut out);
eng.midi.clear();
out.swap_remove(0)
}
/// A record-enabled synth track with no clips.
fn live_synth_session() -> Session {
let mut s = synth_session(-6.0);
s.tracks[0].playlist_mut().unwrap().clips.clear();
s.tracks[0].mixer.record_arm = true;
s
}
#[test]
fn several_live_events_in_one_block_play_in_order_at_their_offsets() {
let s = live_synth_session();
let mut eng = MixEngine::new(48_000.0, 512);
// The note starts at its offset, not at the start of the block.
let out = live_block(&mut eng, &s, 0, &[(300, 0x90, 60, 100)]);
assert!(out[..300].iter().all(|x| *x == 0.0), "silent before the key goes down");
assert!(peak(&out[300..]) > 0.001);
// On and off in one block: the note ends. The tail is over well within a second.
let mut eng = MixEngine::new(48_000.0, 512);
live_block(&mut eng, &s, 0, &[(100, 0x90, 60, 100), (400, 0x80, 60, 0)]);
let tail = (1..100).map(|b| peak(&live_block(&mut eng, &s, b * 512, &[]))).last().unwrap();
assert!(tail < 1e-4, "{tail}");
// A release that comes before the key goes down does not end the note that follows.
let mut eng = MixEngine::new(48_000.0, 512);
live_block(&mut eng, &s, 0, &[(100, 0x80, 60, 0), (400, 0x90, 60, 100)]);
let held = (1..100).map(|b| peak(&live_block(&mut eng, &s, b * 512, &[]))).last().unwrap();
assert!(held > 0.01, "{held}");
// An offset past the block still plays, in this block.
let mut eng = MixEngine::new(48_000.0, 512);
live_block(&mut eng, &s, 0, &[(u32::MAX, 0x90, 60, 100)]);
assert!(peak(&live_block(&mut eng, &s, 512, &[])) > 0.001);
}
#[test]
fn a_seek_and_a_stop_silence_live_notes() {
let s = live_synth_session();
let quiet_after = |then: &dyn Fn(&mut MixEngine) -> i64| {
let mut eng = MixEngine::new(48_000.0, 512);
live_block(&mut eng, &s, 0, &[(0, 0xB0, 64, 127), (0, 0x90, 60, 100)]);
assert!(peak(&live_block(&mut eng, &s, 512, &[])) > 0.01);
let pos = then(&mut eng);
(0..100).map(|b| peak(&live_block(&mut eng, &s, pos + b * 512, &[]))).last().unwrap()
};
// Playing on: the key and the pedal keep the note.
assert!(quiet_after(&|_| 1024) > 0.01);
// A seek (the next block is somewhere else) and a transport reset end it, pedal or not.
assert!(quiet_after(&|_| 480_000) < 1e-4, "seek");
assert!(
quiet_after(&|eng| {
eng.reset();
1024
}) < 1e-4,
"stop"
);
}
#[test]
fn muted_inactive_and_solo_shaded_tracks_do_not_sound_live_input() {
let level = |change: &dyn Fn(&mut Session)| {
let mut s = live_synth_session();
change(&mut s);
let mut eng = MixEngine::new(48_000.0, 512);
live_block(&mut eng, &s, 0, &[(0, 0x90, 60, 100)]);
peak(&live_block(&mut eng, &s, 512, &[]))
};
assert!(level(&|_| {}) > 0.01);
assert_eq!(level(&|s| s.tracks[0].mixer.mute = true), 0.0, "muted");
assert_eq!(level(&|s| s.tracks[0].inactive = true), 0.0, "inactive");
assert_eq!(
level(&|s| {
let other = s.add_track(TrackKind::Audio, ChannelFormat::Mono, None);
s.track_mut(other).unwrap().mixer.solo = true;
}),
0.0,
"another track is soloed"
);
// A track taken out of record with a key down and put back has no note left over.
let mut s = live_synth_session();
let mut eng = MixEngine::new(48_000.0, 512);
live_block(&mut eng, &s, 0, &[(0, 0x90, 60, 100)]);
s.tracks[0].mixer.record_arm = false;
live_block(&mut eng, &s, 512, &[]);
s.tracks[0].mixer.record_arm = true;
let tail = (2..100).map(|b| peak(&live_block(&mut eng, &s, b * 512, &[]))).last().unwrap();
assert!(tail < 1e-4, "{tail}");
}
/// The peak of each of `blocks` playing blocks from `first`, with `events` in block `at`.
fn live_levels(eng: &mut MixEngine, s: &Session, first: i64, blocks: i64, at: i64, events: &[(u32, u8, u8, u8)]) -> Vec<f32> {
(first..first + blocks).map(|b| peak(&live_block(eng, s, b * 512, if b == at { events } else { &[] }))).collect()
}
#[test]
fn a_clip_note_and_a_live_key_on_one_pitch_do_not_cut_each_other() {
// The clip plays C4 (60) from 0 to 24 000 samples: blocks 0 to 46.
let mut s = synth_session(-6.0);
s.tracks[0].mixer.record_arm = true;
// The key goes down during the clip's note and stays down past its end: it keeps sounding.
let mut eng = MixEngine::new(48_000.0, 512);
let levels = live_levels(&mut eng, &s, 0, 200, 20, &[(0, 0x90, 60, 100)]);
assert!(levels[199] > 0.01, "the clip's note-off silenced the key that is still down: {}", levels[199]);
// The key's release then ends it.
let after = live_levels(&mut eng, &s, 200, 200, 200, &[(0, 0x80, 60, 0)]);
assert!(after[199] < 1e-4, "{}", after[199]);
// The other way round: a key pressed and released during the clip's note does not cut it.
let mut eng = MixEngine::new(48_000.0, 512);
live_levels(&mut eng, &s, 0, 10, 5, &[(0, 0x90, 60, 100)]);
let during = live_levels(&mut eng, &s, 10, 30, 10, &[(0, 0x80, 60, 0)]);
assert!(during[29] > 0.01, "the live release cut the clip's note: {}", during[29]);
// And the clip's own note-off ends it on time.
let end = live_levels(&mut eng, &s, 40, 160, -1, &[]);
assert!(end[159] < 1e-4, "{}", end[159]);
// Another pitch is not affected by any of this.
let mut eng = MixEngine::new(48_000.0, 512);
live_levels(&mut eng, &s, 0, 60, 5, &[(0, 0x90, 72, 100)]);
assert!(live_levels(&mut eng, &s, 60, 100, -1, &[])[99] > 0.01, "the other key is still down");
}
#[test]
fn clip_notes_end_when_playback_runs_out_while_the_track_is_still_monitored() {
let mut s = synth_session(-6.0);
s.tracks[0].mixer.record_arm = true;
let mut out = vec![vec![0.0f32; 512]; 2];
// Playback stops by itself in the middle of the clip's note (no reset, as at a range end),
// and the armed track goes on being rendered for live input.
let stopped = |eng: &mut MixEngine, out: &mut Vec<Vec<f32>>, live: &[LiveMidi]| {
eng.monitor_only = true;
eng.midi.clear();
eng.midi.extend_from_slice(live);
eng.render(&s, 5_120, 512, out);
eng.midi.clear();
peak(&out[0])
};
let mut eng = MixEngine::new(48_000.0, 512);
assert!(live_levels(&mut eng, &s, 0, 10, -1, &[])[9] > 0.01);
let tail = (0..200).map(|_| stopped(&mut eng, &mut out, &[])).last().unwrap();
assert!(tail < 1e-4, "the clip's note was left on: {tail}");
// A key held on the same pitch at that moment stays, and ends with its own release.
let mut eng = MixEngine::new(48_000.0, 512);
live_levels(&mut eng, &s, 0, 10, 5, &[(0, 0x90, 60, 100)]);
let held = (0..200).map(|_| stopped(&mut eng, &mut out, &[])).last().unwrap();
assert!(held > 0.01, "{held}");
stopped(&mut eng, &mut out, &[LiveMidi { offset: 0, port: 0, status: 0x80, data1: 60, data2: 0 }]);
let released = (0..200).map(|_| stopped(&mut eng, &mut out, &[])).last().unwrap();
assert!(released < 1e-4, "{released}");
}
#[test]
fn the_clips_sustain_pedal_goes_up_when_playback_runs_out_under_monitoring() {
use soundcraft_midi::{CtrlEvent, CtrlKind};
// The clip holds the pedal down from its start and never lifts it before playback ends.
let mut s = synth_session(-6.0);
s.tracks[0].mixer.record_arm = true;
let ClipContent::Midi { sequence } = &mut s.tracks[0].playlist_mut().unwrap().clips[0].content else { panic!("a MIDI clip") };
sequence.ctrls.push(CtrlEvent { tick: 0, channel: 0, kind: CtrlKind::Cc(64), value: 127 });
let mut out = vec![vec![0.0f32; 512]; 2];
let stopped = |eng: &mut MixEngine, out: &mut Vec<Vec<f32>>, live: &[LiveMidi]| {
eng.monitor_only = true;
eng.midi.clear();
eng.midi.extend_from_slice(live);
eng.render(&s, 5_120, 512, out);
eng.midi.clear();
peak(&out[0])
};
// Playback stops by itself in the middle of the note (no reset): the note-off alone would
// leave the instrument sustaining it for as long as the track is monitored.
let mut eng = MixEngine::new(48_000.0, 512);
assert!(live_levels(&mut eng, &s, 0, 10, -1, &[])[9] > 0.01);
let tail = (0..200).map(|_| stopped(&mut eng, &mut out, &[])).last().unwrap();
assert!(tail < 1e-4, "the clip's pedal kept its note on: {tail}");
// A key held live on the same pitch stays, and ends with its own release.
let mut eng = MixEngine::new(48_000.0, 512);
live_levels(&mut eng, &s, 0, 10, 5, &[(0, 0x90, 60, 100)]);
let held = (0..200).map(|_| stopped(&mut eng, &mut out, &[])).last().unwrap();
assert!(held > 0.01, "{held}");
stopped(&mut eng, &mut out, &[LiveMidi { offset: 0, port: 0, status: 0x80, data1: 60, data2: 0 }]);
let released = (0..200).map(|_| stopped(&mut eng, &mut out, &[])).last().unwrap();
assert!(released < 1e-4, "{released}");
}
#[test]
fn a_lost_ports_controllers_are_reset_and_the_other_ports_kept() {
let (s, _, mut eng, heard) = ears_session();
let ev = |port, status, data1, data2| LiveMidi { offset: 0, port, status, data1, data2 };
// Port 0 bends the pitch; port 1 sets the volume.
eng.midi.extend([ev(0, 0xE0, 0, 0x7F), ev(1, 0xB0, 7, 100)]);
render_block(&mut eng, &s, 0);
eng.midi.clear();
heard.lock().unwrap().clear();
// Port 0 is unplugged while the track goes on playing port 1: the bend must not stay. The
// controllers are reset (to the clips' values: there are none) and port 1's volume is sent
// again.
eng.midi_lost = midi_port_bit(0);
render_block(&mut eng, &s, 512);
eng.midi_lost = 0;
assert_eq!(std::mem::take(&mut *heard.lock().unwrap()), vec![(0, [0xB0, 121, 0]), (0, [0xB0, 7, 100])]);
// A port that set no controller changes nothing; neither does the same port again.
eng.midi_lost = midi_port_bit(5) | midi_port_bit(0);
render_block(&mut eng, &s, 1024);
assert!(heard.lock().unwrap().is_empty());
}
#[test]
fn a_clip_controller_at_the_start_of_the_block_a_port_is_lost_in_keeps_its_new_value() {
use soundcraft_midi::{CtrlEvent, CtrlKind, Sequence};
let (mut s, t, mut eng, heard) = ears_session();
// The clip sets the mod wheel at its start and again 24 ticks in: sample 600 at 120 bpm.
let mut seq = Sequence::default();
seq.ctrls.push(CtrlEvent { tick: 0, channel: 0, kind: CtrlKind::Cc(1), value: 10 });
seq.ctrls.push(CtrlEvent { tick: 24, channel: 0, kind: CtrlKind::Cc(1), value: 90 });
let cid = s.new_clip_id();
s.track_mut(t).unwrap().playlist_mut().unwrap().clips.push(Clip::midi(cid, "c", 0, 24_000, seq));
// A device bends the pitch in the block before the one that starts at sample 600.
eng.midi.push(LiveMidi { offset: 0, port: 0, status: 0xE0, data1: 0, data2: 0x7F });
render_block(&mut eng, &s, 88);
eng.midi.clear();
heard.lock().unwrap().clear();
// The device goes away. Its bend is taken back (a reset, then the clip's value as it stood
// before this block), and only then come the clip's events of this block: the mod wheel
// ends at the value the clip gives it here, not at the one from before.
eng.midi_lost = midi_port_bit(0);
render_block(&mut eng, &s, 600);
assert_eq!(std::mem::take(&mut *heard.lock().unwrap()), vec![(0, [0xB0, 121, 0]), (0, [0xB0, 1, 10]), (0, [0xB0, 1, 90])]);
}
#[test]
fn the_clips_sustain_pedal_is_not_put_down_again_when_a_device_goes_away_while_stopped() {
use soundcraft_midi::{CtrlEvent, CtrlKind};
// The clip holds the pedal down from its start; playback runs out in the middle of it.
let mut s = synth_session(-6.0);
s.tracks[0].mixer.record_arm = true;
let ClipContent::Midi { sequence } = &mut s.tracks[0].playlist_mut().unwrap().clips[0].content else { panic!("a MIDI clip") };
sequence.ctrls.push(CtrlEvent { tick: 0, channel: 0, kind: CtrlKind::Cc(64), value: 127 });
let mut out = vec![vec![0.0f32; 512]; 2];
let mut eng = MixEngine::new(48_000.0, 512);
live_levels(&mut eng, &s, 0, 10, -1, &[]);
let mut stopped = |eng: &mut MixEngine, lost: u64, live: &[LiveMidi]| {
eng.monitor_only = true;
eng.midi_lost = lost;
eng.midi.clear();
eng.midi.extend_from_slice(live);
eng.render(&s, 5_120, 512, &mut out);
eng.midi.clear();
eng.midi_lost = 0;
peak(&out[0])
};
let ev = |port, status, data1, data2| LiveMidi { offset: 0, port, status, data1, data2 };
// Stopped, two devices play: one bends the pitch and is then unplugged. The controllers are
// set again from the clips, whose pedal is down at this position and will never come up.
stopped(&mut eng, 0, &[ev(0, 0xE0, 0, 0x7F)]);
stopped(&mut eng, midi_port_bit(0), &[]);
// A key of the other device sounds while it is held, and ends with its release.
stopped(&mut eng, 0, &[ev(1, 0x90, 64, 100)]);
let held = (0..20).map(|_| stopped(&mut eng, 0, &[])).last().unwrap();
assert!(held > 0.01, "the live note plays: {held}");
stopped(&mut eng, 0, &[ev(1, 0x80, 64, 0)]);
let tail = (0..200).map(|_| stopped(&mut eng, 0, &[])).last().unwrap();
assert!(tail < 1e-4, "the clips' pedal held the live note: {tail}");
}
#[test]
fn a_volume_or_pan_left_by_a_device_that_went_away_goes_back_to_the_midi_default() {
use soundcraft_midi::{CtrlEvent, CtrlKind, Sequence};
let ev = |port, status, data1, data2| LiveMidi { offset: 0, port, status, data1, data2 };
let lost = |eng: &mut MixEngine, s: &Session, pos: i64, bits: u64| {
eng.midi.clear();
eng.midi_lost = bits;
render_block(eng, s, pos);
eng.midi_lost = 0;
};
// Port 0 turns the volume down to nothing and the pan hard left, and is unplugged. The
// reset of the controllers does not touch those two: they are set to MIDI's defaults.
let (s, _, mut eng, heard) = ears_session();
eng.midi.extend([ev(0, 0xB0, 7, 0), ev(0, 0xB0, 10, 0), ev(0, 0xB0, 1, 50)]);
render_block(&mut eng, &s, 0);
heard.lock().unwrap().clear();
lost(&mut eng, &s, 512, midi_port_bit(0));
assert_eq!(std::mem::take(&mut *heard.lock().unwrap()), vec![(0, [0xB0, 121, 0]), (0, [0xB0, 7, 100]), (0, [0xB0, 10, 64])]);
lost(&mut eng, &s, 1024, midi_port_bit(0));
assert!(heard.lock().unwrap().is_empty(), "once");
// A device that is still there has the say: its volume is sent again, no default.
let (s, _, mut eng, heard) = ears_session();
eng.midi.extend([ev(1, 0xB0, 7, 90), ev(0, 0xB0, 7, 0)]);
render_block(&mut eng, &s, 0);
heard.lock().unwrap().clear();
lost(&mut eng, &s, 512, midi_port_bit(0));
assert_eq!(std::mem::take(&mut *heard.lock().unwrap()), vec![(0, [0xB0, 121, 0]), (0, [0xB0, 7, 90])]);
// That device is the one whose volume the instrument has now: when it goes too, the default.
lost(&mut eng, &s, 1024, midi_port_bit(1));
assert_eq!(std::mem::take(&mut *heard.lock().unwrap()), vec![(0, [0xB0, 121, 0]), (0, [0xB0, 7, 100])]);
// So have the clips: their volume at this position, no default.
let (mut s, t, mut eng, heard) = ears_session();
let mut seq = Sequence::default();
seq.ctrls.push(CtrlEvent { tick: 0, channel: 0, kind: CtrlKind::Cc(7), value: 80 });
let cid = s.new_clip_id();
s.track_mut(t).unwrap().playlist_mut().unwrap().clips.push(Clip::midi(cid, "c", 0, 24_000, seq));
render_block(&mut eng, &s, 0);
eng.midi.push(ev(0, 0xB0, 7, 0));
render_block(&mut eng, &s, 512);
heard.lock().unwrap().clear();
lost(&mut eng, &s, 1024, midi_port_bit(0));
assert_eq!(std::mem::take(&mut *heard.lock().unwrap()), vec![(0, [0xB0, 121, 0]), (0, [0xB0, 7, 80])]);
// The track leaves record with the volume down: the next block's reset is followed by the
// default as well.
let (mut s, t, mut eng, heard) = ears_session();
eng.midi.push(ev(0, 0xB0, 7, 0));
render_block(&mut eng, &s, 0);
eng.midi.clear();
s.track_mut(t).unwrap().mixer.record_arm = false;
render_block(&mut eng, &s, 512);
heard.lock().unwrap().clear();
render_block(&mut eng, &s, 1024);
assert_eq!(std::mem::take(&mut *heard.lock().unwrap()), vec![(0, [0xB0, 121, 0]), (0, [0xB0, 7, 100])]);
}
#[test]
fn a_volume_the_full_controller_table_forgot_is_reset_only_when_its_own_device_goes() {
let ev = |port, status, data1, data2| LiveMidi { offset: 0, port, status, data1, data2 };
let (s, _, mut eng, heard) = ears_session();
// Keys (port 0) sets the volume. A third device then sets so many controllers that the
// table forgets that value, and Pad (port 1) bends the pitch.
eng.midi.push(ev(0, 0xB0, 7, 30));
render_block(&mut eng, &s, 0);
eng.midi.clear();
eng.midi.extend((0..crate::live_midi::MAX_LIVE_CTRLS as u8).map(|n| ev(2, 0xB0, 20 + n, 1)));
eng.midi.push(ev(1, 0xE0, 0, 0x7F));
render_block(&mut eng, &s, 512);
let lost = |eng: &mut MixEngine, pos: i64, port: u8| {
heard.lock().unwrap().clear();
eng.midi.clear();
eng.midi_lost = midi_port_bit(port);
render_block(eng, &s, pos);
eng.midi_lost = 0;
heard.lock().unwrap().iter().filter(|(_, m)| m[1] == 7 || m[1] == 121).map(|(_, m)| *m).collect::<Vec<_>>()
};
// Pad goes: the controllers are reset for its bend, but the volume is Keys', who is still
// there, and is left alone.
assert_eq!(lost(&mut eng, 1024, 1), [[0xB0, 121, 0]]);
// Keys goes: now the volume goes back to the default.
assert_eq!(lost(&mut eng, 1536, 0), [[0xB0, 121, 0], [0xB0, 7, 100]]);
// Two devices set the volume one after the other, and the table forgets both values. The
// instrument has the later one: when the earlier setter goes, it stays.
eng.midi.extend([ev(0, 0xB0, 7, 20), ev(3, 0xB0, 7, 30), ev(0, 0xE0, 0, 0x7F)]);
eng.midi.extend((0..crate::live_midi::MAX_LIVE_CTRLS as u8).map(|n| ev(2, 0xB0, 20 + n, 2)));
render_block(&mut eng, &s, 2048);
assert_eq!(lost(&mut eng, 2560, 0), [[0xB0, 121, 0]], "the later setter is still there");
assert_eq!(lost(&mut eng, 3072, 3), [[0xB0, 121, 0], [0xB0, 7, 100]]);
}
#[test]
fn a_clip_volume_moved_earlier_by_the_track_delay_counts_as_set_when_a_device_goes_away() {
use soundcraft_midi::{CtrlEvent, CtrlKind, Sequence};
let ev = |port, status, data1, data2| LiveMidi { offset: 0, port, status, data1, data2 };
// The clip sets the volume at tick 960; the track's MIDI delay of -960 ticks plays it at 0.
let (mut s, t, mut eng, heard) = ears_session();
let mut seq = Sequence::default();
seq.ctrls.push(CtrlEvent { tick: 960, channel: 0, kind: CtrlKind::Cc(7), value: 80 });
let cid = s.new_clip_id();
s.track_mut(t).unwrap().playlist_mut().unwrap().clips.push(Clip::midi(cid, "c", 0, 48_000, seq));
s.edit.values.insert(format!("rtp.{}.delay", t.0), -960.0);
render_block(&mut eng, &s, 0);
// A device turns the volume down and is unplugged, long before tick 960.
eng.midi.push(ev(0, 0xB0, 7, 0));
render_block(&mut eng, &s, 512);
heard.lock().unwrap().clear();
eng.midi.clear();
eng.midi_lost = midi_port_bit(0);
render_block(&mut eng, &s, 1024);
assert_eq!(std::mem::take(&mut *heard.lock().unwrap()), vec![(0, [0xB0, 121, 0]), (0, [0xB0, 7, 80])], "the clip's volume, not the default");
}
+4
View File
@@ -5,3 +5,7 @@
#![deny(clippy::unwrap_used, clippy::expect_used, clippy::panic, clippy::unimplemented, clippy::todo, clippy::unreachable)]
pub use craft_scraft_format::*;
mod midi_input;
pub use midi_input::MidiInput;
+74
View File
@@ -0,0 +1,74 @@
//! The MIDI input of a MIDI or instrument track, kept in its input route.
use crate::Route;
/// The MIDI input a MIDI or instrument track listens to. It lives in the track's input route as
/// `Route::Hardware("MIDI:<channel>:<device>")`, with `*` for any channel (else 1–16) and for any
/// device; every other route means all devices, all channels.
#[derive(Debug, Clone, Copy, PartialEq, Eq, Default)]
pub struct MidiInput<'a> {
/// Device (input port) name; `None`: all enabled devices.
pub device: Option<&'a str>,
/// MIDI channel 0..=15; `None`: all channels.
pub channel: Option<u8>,
}
impl<'a> MidiInput<'a> {
/// Reads a track's input route. Does not allocate (the audio thread calls it).
pub fn of(route: &'a Route) -> MidiInput<'a> {
let Route::Hardware(h) = route else { return MidiInput::default() };
let Some((ch, dev)) = h.strip_prefix("MIDI:").and_then(|r| r.split_once(':')) else { return MidiInput::default() };
MidiInput {
device: Some(dev).filter(|d| *d != "*" && !d.is_empty()),
channel: ch.parse::<u8>().ok().filter(|c| (1..=16).contains(c)).map(|c| c - 1),
}
}
/// The input route that stores this choice.
pub fn route(&self) -> Route {
if self.device.is_none() && self.channel.is_none() {
return Route::None;
}
let ch = self.channel.filter(|c| *c < 16).map_or_else(|| "*".to_string(), |c| (c + 1).to_string());
Route::Hardware(format!("MIDI:{ch}:{}", self.device.unwrap_or("*")))
}
/// Whether an event on `channel` (0..=15) from `device` reaches the track.
pub fn accepts(&self, device: Option<&str>, channel: u8) -> bool {
self.channel.is_none_or(|c| c == channel) && self.device.is_none_or(|d| device == Some(d))
}
/// Short text for the track's input selector.
pub fn label(&self) -> String {
match (self.device, self.channel) {
(None, None) => "All".to_string(),
(None, Some(c)) => format!("All ch {}", c + 1),
(Some(d), None) => d.to_string(),
(Some(d), Some(c)) => format!("{d} ch {}", c + 1),
}
}
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn midi_input_round_trips_through_the_input_route() {
for (device, channel) in [(None, None), (Some("Keys: with:colons"), None), (None, Some(9)), (Some("Pad"), Some(0))] {
let m = MidiInput { device, channel };
let r = m.route();
assert_eq!(MidiInput::of(&r), m, "{r:?}");
}
assert_eq!(MidiInput { device: None, channel: None }.route(), Route::None);
// Audio-style and broken routes mean everything.
for h in ["In 1-2", "MIDI:", "MIDI:17:*", "MIDI:0:*", "MIDI:x"] {
assert_eq!(MidiInput::of(&Route::Hardware(h.into())).channel, None, "{h}");
}
let m = MidiInput { device: Some("Pad"), channel: Some(2) };
assert!(m.accepts(Some("Pad"), 2));
assert!(!m.accepts(Some("Pad"), 3) && !m.accepts(Some("Keys"), 2) && !m.accepts(None, 2));
assert!(MidiInput::default().accepts(None, 7));
assert_eq!((m.label(), MidiInput::default().label()), ("Pad ch 3".to_string(), "All".to_string()));
}
}
+5 -1
View File
@@ -1,6 +1,6 @@
[package]
name = "soundcraft-playback"
description = "SoundCraft audio device I/O: realtime playback and recording through the mix engine"
description = "SoundCraft audio and MIDI device I/O: realtime playback and recording through the mix engine"
version.workspace = true
edition.workspace = true
license.workspace = true
@@ -22,6 +22,10 @@ cpal.workspace = true
[target.'cfg(target_arch = "wasm32")'.dependencies]
cpal = { workspace = true, features = ["wasm-bindgen"] }
# MIDI devices (ALSA sequencer, CoreMIDI, WinMM). Elsewhere there is no MIDI input (midi.rs).
[target.'cfg(any(target_os = "linux", target_os = "macos", target_os = "windows"))'.dependencies]
midir.workspace = true
[features]
# ASIO on Windows (cpal's ASIO host). Off by default: building it downloads Steinberg's ASIO SDK
# (its own licence) and needs LLVM (libclang) for the bindings. `cargo build -p soundcraft --features asio`.
+36
View File
@@ -0,0 +1,36 @@
//! A keyboard without the hardware: makes a virtual MIDI output port and plays an arpeggio on it,
//! so MIDI input can be tried (and tested by agents) on a machine with no MIDI device.
//!
//! `cargo run -p soundcraft-playback --example midi_keys -- [seconds] [channel 1-16]`
//!
//! SoundCraft lists the port as "SoundCraft test keys" in Setup › Playback Engine a moment after
//! this starts. Linux and macOS only (Windows has no virtual MIDI ports).
#[cfg(any(target_os = "linux", target_os = "macos"))]
fn main() -> Result<(), Box<dyn std::error::Error>> {
use midir::os::unix::VirtualOutput;
use std::time::{Duration, Instant};
let mut args = std::env::args().skip(1);
let seconds: f64 = args.next().and_then(|a| a.parse().ok()).unwrap_or(10.0);
let channel: u8 = args.next().and_then(|a| a.parse::<u8>().ok()).map_or(0, |c| c.clamp(1, 16) - 1);
let mut out = midir::MidiOutput::new("SoundCraft test keys")?.create_virtual("SoundCraft test keys").map_err(|e| e.to_string())?;
println!("virtual MIDI port \"SoundCraft test keys\" is up; playing on channel {} for {seconds} s", channel + 1);
// Time for a listener to notice the port and connect.
std::thread::sleep(Duration::from_secs(4));
let start = Instant::now();
for note in [60u8, 64, 67, 72].into_iter().cycle() {
if start.elapsed().as_secs_f64() >= seconds {
break;
}
out.send(&[0x90 | channel, note, 100])?;
std::thread::sleep(Duration::from_millis(200));
out.send(&[0x80 | channel, note, 0])?;
std::thread::sleep(Duration::from_millis(50));
}
Ok(())
}
#[cfg(not(any(target_os = "linux", target_os = "macos")))]
fn main() {
eprintln!("virtual MIDI ports need Linux or macOS");
}
+563 -8
View File
@@ -1,8 +1,10 @@
use super::{
Arc, AtomicBool, BLOCK, BTreeMap, Cmd, HashMap, HashSet, MAX_CHANNELS, MeterSnapshot, MixEngine, OLD_SESSIONS, Ordering, OutputTrouble, Plugin,
PluginEditor, PluginStateReader, PluginStates, PreparedInstance, Range, Receiver, Samples, Sender, Session, Shared, SyncSender, TrackId,
itu_stereo_fold, main_channels, record, sync_channel,
itu_stereo_fold, main_channels, midi, record, sync_channel,
};
use soundcraft_mix::{LiveMidi, midi_port_bit, midi_thru};
use soundcraft_model::TrackKind;
/// How the audio output is doing, for the UI.
#[derive(Default)]
@@ -80,6 +82,50 @@ pub(crate) struct AudioState {
pub(crate) device_rate: f64,
input: Option<Arc<record::InputRing>>,
recording: bool,
midi: Option<Arc<midi::MidiFeed>>,
/// Live MIDI events of this callback not played yet: frames (session rate) until each is due.
midi_pending: Vec<(i64, LiveMidi)>,
/// The delay every live event gets so that all keep their spacing: one callback (frames).
midi_delay: i64,
/// Ports ([`midi_port_bit`]) that lost events and still have to let go of their notes in
/// the engine: it is told with the next block it renders.
midi_lost: u64,
/// For each port (by the place of its bit in `midi_lost`), the time up to which its losses
/// were dealt with at their mark in the queue: the port's flag for the same loss, seen a
/// callback later, is not a second loss.
midi_loss_seen: [i64; 64],
/// For each port, the time of the last loss that was dealt with from the port's flag (and
/// written to the take from it): the mark of that same loss, which gets into the queue only
/// with the next event, is not a second loss.
midi_flag_seen: [i64; 64],
/// The transport speed of the last callback: the playhead moved at it while the events the
/// next callback takes were arriving.
midi_speed: f64,
/// Times loop recording went back to the start of the loop since the take was last written
/// to. A short loop goes round several times in one callback.
midi_wraps: u8,
/// The record pass a command of this callback ended, to be confirmed once the MIDI that
/// arrived up to that command is written down.
midi_ended: Option<u64>,
/// The transport stopped while a record pass ran. The pass goes on until the command that
/// ends it (the UI sends the two one after the other, and they may reach two callbacks):
/// what arrived before the stop is still written down. The value is the time up to which
/// that holds, which is when the transport stopped (after playback ran to its end by itself
/// the UI ends the pass only when it notices; for a Stop it gives the time itself, a little
/// earlier). The ages of the pass's last events are counted up to then.
midi_stopped: Option<i64>,
/// A command of this callback started a pass that the UI had begun while the transport was
/// running: should the pass also end in this callback, its last read of the queue still
/// counts from when it began.
midi_started: bool,
/// Called between taking the MIDI in and reading the commands, for tests of what arrives
/// right there.
#[cfg(test)]
pub(crate) before_commands: Option<Box<dyn FnMut() + Send>>,
/// Called after the callback has looked at the state of the pass and before it takes the
/// MIDI in, for tests of what happens right there.
#[cfg(test)]
pub(crate) before_capture: Option<Box<dyn FnMut() + Send>>,
/// Retired plugin instances go back to the UI thread to be dropped there.
garbage: Sender<Vec<Box<dyn Plugin>>>,
old_sessions: Option<SyncSender<Arc<Session>>>,
@@ -90,8 +136,13 @@ pub(crate) struct AudioState {
forget_shared: Vec<TrackId>,
}
/// Live MIDI events one device callback can take in. More than that in one callback counts as
/// lost input for the ports concerned (see `take_midi`).
pub(crate) const MIDI_EVENTS: usize = 256;
fn new_engine(sr: f32) -> MixEngine {
let mut m = MixEngine::new(sr, BLOCK);
m.midi = Vec::with_capacity(MIDI_EVENTS);
m.set_external_instances(true);
m.metronome = true;
m
@@ -125,6 +176,21 @@ impl AudioState {
device_rate,
input: None,
recording: false,
midi: None,
midi_pending: Vec::with_capacity(MIDI_EVENTS),
midi_delay: 0,
midi_lost: 0,
midi_loss_seen: [i64::MIN; 64],
midi_flag_seen: [i64::MIN; 64],
midi_speed: 1.0,
midi_wraps: 0,
midi_ended: None,
midi_stopped: None,
midi_started: false,
#[cfg(test)]
before_commands: None,
#[cfg(test)]
before_capture: None,
unsent: MeterSnapshot::default(),
forget_shared: Vec::new(),
}
@@ -135,6 +201,16 @@ impl AudioState {
while let Ok(c) = self.rx.try_recv() {
match c {
Cmd::Input(r) => self.input = Some(r),
Cmd::Midi(f) => {
self.mix.midi_ports = Some(Arc::clone(f.ports()));
self.midi = Some(f);
// Another feed: releases of what is held now would never come.
self.midi_pending.clear();
self.midi_lost = u64::MAX;
// Its times are on its own clock.
self.midi_loss_seen = [i64::MIN; 64];
self.midi_flag_seen = [i64::MIN; 64];
}
Cmd::Forget(gone) => {
self.unsent.forget(&gone);
if self.forget_shared.is_empty() {
@@ -144,12 +220,28 @@ impl AudioState {
}
self.forget_meters();
}
Cmd::Recording(r) => self.recording = r,
Cmd::Recording(r, pass) => {
self.recording = r;
self.midi_wraps = 0;
self.midi_stopped = None;
// The pass has begun here: its start is no longer waited for.
if r && let Some(feed) = &self.midi {
self.midi_started = self.playing && feed.begin_pending();
feed.pass_begun();
}
// The UI thread waits for this pass before it builds the clips. It is told
// once the callback has also taken what arrived since it last looked at the
// MIDI queue (see `fill`).
if !r && pass != 0 {
self.midi_ended = Some(self.midi_ended.map_or(pass, |p| p.max(pass)));
}
}
Cmd::Session(s, instances) => {
changed = true;
if (s.sample_rate.as_f64() - self.session.sample_rate.as_f64()).abs() > 0.5 {
let mut old = std::mem::replace(&mut self.mix, new_engine(s.sample_rate.as_f64() as f32));
let _ = self.garbage.send(old.drain_plugins());
self.mix.midi_ports = self.midi.as_ref().map(|f| Arc::clone(f.ports()));
}
self.mix.adopt(instances);
// The main format changed (a document edit, not steady state): resize.
@@ -169,19 +261,23 @@ impl AudioState {
Cmd::Play { from, end, looped } => {
self.pos = from;
self.end = end;
self.looped = looped.filter(|r| r.len() > 64);
self.looped = looped.filter(|r| super::loops(*r));
self.midi_wraps = 0;
self.midi_stopped = None;
self.playing = true;
self.pending_len = 0;
self.read = 0.0;
self.mix.reset();
}
Cmd::Stop => {
self.midi_stopped = (self.playing && self.recording).then(|| self.midi.as_ref().map_or(i64::MAX, |f| f.clock()));
self.playing = false;
self.end = None;
self.looped = None;
self.mix.reset();
}
Cmd::Locate(at) => {
self.midi_stopped = None;
self.pos = at;
self.pending_len = 0;
self.read = 0.0;
@@ -205,6 +301,249 @@ impl AudioState {
}
}
/// Events of the ports in `lost` ([`midi_port_bit`]) will not be played, and any of them may
/// have been a note-off or a pedal release. What is still queued from those ports goes too,
/// so that no note-on from before the loss is played after the release, and the engine lets
/// go of what the ports hold at its next block.
fn midi_lost(&mut self, lost: u64) {
if lost == 0 {
return;
}
self.midi_pending.retain(|(_, e)| lost & midi_port_bit(e.port) == 0);
self.midi_lost |= lost;
}
/// The transport speed now, as a factor.
fn speed(&self) -> f64 {
(f64::from(self.shared.speed.load(Ordering::Relaxed)) / 1000.0).clamp(0.1, 4.0)
}
/// Session frames to a device frame at the transport speed `speed`.
fn step(&self, speed: f64) -> f64 {
self.session.sample_rate.as_f64() / self.device_rate.max(1.0) * speed
}
/// Take the MIDI events that arrived since the callback last looked. `fill` does that before
/// this callback's commands are read, with the state of the pass as it was then (`pass`), so
/// that what was played up to a Stop still belongs to the pass; and once more after a command
/// that ended a pass, for what arrived in between. `callback` is the length of the callback
/// in session frames, `speed` the transport speed the events' ages are counted at.
///
/// Of a pass that was ended, only what arrived up to the Stop is written down
/// ([`midi::MidiFeed::record_until`], asked for every entry); the rest is played like any
/// live input. Of a pass the UI has begun during playback whose command has not been read
/// yet, what arrived since it began is written down already
/// ([`midi::MidiFeed::record_from`]).
///
/// Recording and playing are separate from here on. While a pass runs, each event is written
/// to the take now, at the position it arrived at (the position of this callback less the
/// event's age). To be heard, each is due `callback` session frames after it arrived, so that
/// events keep their spacing instead of bunching up at the start of a block; that countdown
/// is only for playing and never moves the recorded position.
///
/// An age is real time; the playhead covered it at the transport speed of the callback
/// before this one (half speed: half as many session frames). That speed is kept from then:
/// by now the UI may have set another (it puts the speed back before a Stop). A callback
/// reads the speed once, and renders at the very speed it keeps for the next one.
///
/// To be played, events stay in the order they arrived in: one is never due before an
/// earlier one that is still waiting (a callback shorter than the one before would
/// otherwise put a release ahead of its note-on).
///
/// Lost input has its place in that order. The events of a lost port that arrived before
/// the loss are written to the take but not played (their releases may be missing); the mark
/// follows them in the take; what arrived after the loss comes after the mark and is played,
/// after the engine let go of the port's notes at the start of its next block. Every port
/// has its own time of loss, so one port's loss does not take along what another sent.
///
/// A loss from a full queue also has a mark in the queue itself, ahead of the first event
/// that fitted again. That is its exact place even when the port's flag was raised after
/// this callback read the flags (the queue was drained meanwhile and a new note got in): the
/// mark is dealt with where it stands, and the flag, seen by the next callback, is known to
/// be the same loss (`midi_loss_seen`) instead of ending the new note. The other way round,
/// a loss told by the flag whose mark gets into the queue later is in the take once
/// (`midi_flag_seen`). A mark is stamped with the time of its loss: when the port's last loss
/// is later than that (the port overflowed, played on and was then closed), that later loss
/// keeps its own mark.
fn capture_midi(&mut self, callback: i64, speed: f64, pass: Pass) {
let Some(feed) = &self.midi else { return };
// The loss flags first, then the time they stand for: see `MidiFeed::take_lost`.
let mut input_lost = feed.take_lost();
let now = feed.clock();
// A flag for a loss that was already met at its mark in the queue is not a new loss.
// Losses are followed whenever a pass runs or may begin while the queue is read (the
// transport is running): whether one is written to the take is decided for each, like
// for an event, by when it happened (`TakeWriter::within`).
let record = pass.record || pass.playing;
// When each flagged port lost its input, by the place of its bit.
let mut lost_time = [i64::MIN; 64];
let flagged = input_lost;
for bit in (0..64u8).filter(|b| flagged & (1u64 << b) != 0) {
let seen = self.midi_loss_seen.get(usize::from(bit)).copied().unwrap_or(i64::MIN);
let at = feed.lost_at(bit_port(bit));
// Nor is one for a loss that was dealt with from an earlier flag: a port closed
// while the callback was reading the queue is met there through the mark of its
// overflow, and its own flag comes a callback later.
let told = seen.max(self.midi_flag_seen.get(usize::from(bit)).copied().unwrap_or(i64::MIN));
if at <= told {
input_lost &= !(1u64 << bit);
continue;
}
// The flag may stand for two losses: the queue overflowed, and the port was closed
// later. The overflow comes first when nothing has told of it yet (not its mark in
// the queue, not an earlier flag): it has its own time, and nothing of the port got
// into the queue between the two.
let overflow = feed.overflow_at(bit_port(bit));
let at = if overflow > told { overflow.min(at) } else { at };
if let Some(t) = lost_time.get_mut(usize::from(bit)) {
*t = at;
}
}
let per_us = self.session.sample_rate.as_f64() / 1e6 * speed;
let frames = |since: i64| (now.saturating_sub(since).max(0) as f64 * per_us) as i64;
// For the take, ages are counted up to when the transport stopped, if it has: the
// playhead stands still since then, and the time gone by since is not part of an age.
let take_now = now.min(pass.until);
let take_frames = |since: i64| (take_now.saturating_sub(since).max(0) as f64 * per_us) as i64;
self.midi_delay = callback;
if input_lost != 0 {
// What is already waiting from those ports came before the loss.
self.midi_pending.retain(|(_, e)| input_lost & midi_port_bit(e.port) == 0);
self.midi_lost |= input_lost;
}
let mut take = TakeWriter {
feed,
pos: pass.pos,
looped: pass.looped,
wraps: std::mem::take(&mut self.midi_wraps),
running: pass.running,
playing: pass.playing,
begun: pass.begun,
until: pass.until,
stopped: pass.stopped,
};
// The lost ports whose mark is not in the take yet. Each goes in where its loss was.
let mut marks = if record { input_lost } else { 0 };
let mut unplayed = 0u64;
while let Some((stamp, e)) = feed.next() {
let age = frames(stamp);
let take_age = take_frames(stamp);
if e.status == midi::RecordedMidi::LOST {
// The queue dropped events of this port here. What it sent before is above and
// may lack its releases: it is not played, and the engine lets go of the port's
// notes at its next block. What follows was sent after the loss and plays.
let bit = midi_port_bit(e.port);
let place = usize::from(e.port).min(63);
// The port's flag told of this very loss before the mark got into the queue.
let told = stamp <= self.midi_flag_seen.get(place).copied().unwrap_or(i64::MIN);
// The port lost its input again later (it was closed after it played on): the
// flag stands for that loss too, whose mark stays owed and goes in at its own
// time.
let again = feed.lost_at(e.port);
if again <= stamp {
marks &= !bit;
} else if input_lost & bit != 0
&& let Some(t) = lost_time.get_mut(place)
&& *t <= stamp
{
// Dealt with in this callback (what the port sent up to it is not played,
// see below), at its own mark in the queue if it has one, else at the end.
// A loss the flag told of that is later than this mark keeps its own time:
// an overflow whose mark never got into the queue comes before the close.
*t = again;
}
if record && !told {
while let Some((other, at)) = next_mark(&lost_time, marks, stamp) {
take.lost(at, take_frames(at).min(callback).max(take_age), other);
flag_seen(&mut self.midi_flag_seen, other, at);
marks &= !other;
}
take.lost(stamp, take_age, bit);
}
self.midi_pending.retain(|(_, p)| bit & midi_port_bit(p.port) == 0);
self.midi_lost |= bit;
if let Some(seen) = self.midi_loss_seen.get_mut(place) {
*seen = (*seen).max(stamp);
}
continue;
}
if record {
while let Some((bit, at)) = next_mark(&lost_time, marks, stamp) {
// Not older than this callback's own events, nor newer than this one.
take.lost(at, take_frames(at).min(callback).max(take_age), bit);
flag_seen(&mut self.midi_flag_seen, bit, at);
marks &= !bit;
}
}
// Asked for every event: a pass may begin while the queue is being read.
take.event(stamp, take_age, &e);
// From before its port's loss: its release may be among what was lost.
if input_lost & midi_port_bit(e.port) != 0 && stamp <= feed.lost_at(e.port) {
continue;
}
if self.midi_pending.len() < self.midi_pending.capacity() {
// Due a callback after it arrived, and not before what arrived ahead of it: the
// list is in order of arrival, so its last entry is the latest due.
let due = callback.saturating_sub(age).max(0);
let due = self.midi_pending.last().map_or(due, |(last, _)| due.max(*last));
self.midi_pending.push((due, e));
} else {
// More than one callback holds: this event is not played (it is recorded), and
// its port lets go of its notes.
unplayed |= midi_port_bit(e.port);
}
}
if record {
// Nothing arrived after these losses: their marks close the callback's events.
while let Some((bit, at)) = next_mark(&lost_time, marks, i64::MAX) {
take.lost(at, take_frames(at).min(callback), bit);
flag_seen(&mut self.midi_flag_seen, bit, at);
marks &= !bit;
}
take.flush();
}
// Every loss a flag told of was dealt with here, up to its time: the same loss told
// again (by its mark in the queue, or by a flag raised while the queue was read) is
// not news.
for bit in (0..64u8).filter(|b| input_lost & (1u64 << b) != 0) {
flag_seen(&mut self.midi_flag_seen, 1u64 << bit, lost_time.get(usize::from(bit)).copied().unwrap_or(i64::MIN));
}
self.midi_lost(unplayed);
}
/// Nothing plays live MIDI: let the queued events go, so that they do not sound late. Their
/// releases go with them, so every port lets go of its notes when the engine runs again.
/// (A take already has them: `capture_midi` wrote them down.)
fn drop_midi(&mut self) {
self.midi_pending.clear();
self.midi_lost = u64::MAX;
}
/// Hand the engine the live MIDI events due in the next `n` frames.
fn due_midi(&mut self, n: usize) {
self.mix.midi.clear();
// Ports that lost events let go of their notes at the start of this block.
self.mix.midi_lost = std::mem::take(&mut self.midi_lost);
let frames = i64::try_from(n).unwrap_or(i64::MAX);
let out = &mut self.mix.midi;
let mut lost = 0u64;
self.midi_pending.retain_mut(|(due, e)| {
if *due >= frames {
*due -= frames;
return true;
}
if out.len() >= out.capacity() {
// The engine's list is full (it holds as much as `midi_pending`, so this does
// not happen): treated like any other event that is not played.
lost |= midi_port_bit(e.port);
return false;
}
out.push(LiveMidi { offset: u32::try_from(*due).unwrap_or(0), ..*e });
false
});
self.midi_lost(lost);
}
/// Render the next block at the session rate into `pending`.
fn render_next(&mut self, quantum: usize) {
let mut n = quantum.clamp(1, BLOCK);
@@ -212,11 +551,19 @@ impl AudioState {
&& self.pos >= l.end
{
self.pos = l.start;
if self.playing && self.recording {
self.midi_wraps = self.midi_wraps.saturating_add(1);
}
}
if let Some(l) = self.looped {
n = n.min(usize::try_from((l.end - self.pos).max(1)).unwrap_or(BLOCK));
} else if let Some(e) = self.end {
if self.pos >= e {
// A record pass is not over yet: the UI ends it. What was played up to here
// still belongs to it.
if self.playing && self.recording {
self.midi_stopped = Some(self.midi.as_ref().map_or(i64::MAX, |f| f.clock()));
}
self.playing = false;
self.end = None;
self.shared.playing.store(false, Ordering::Relaxed);
@@ -239,6 +586,7 @@ impl AudioState {
}
ring.pop_into(&mut self.mix.input, n, self.session.sample_rate.hz());
}
self.due_midi(n);
self.mix.monitor_only = !self.playing;
self.mix.recording = self.recording;
self.mix.render(&self.session, self.pos, n, &mut self.pending);
@@ -274,10 +622,68 @@ impl AudioState {
/// Fill an interleaved device buffer.
pub(crate) fn fill(&mut self, out: &mut [f32], channels: usize) {
self.drain();
let channels = channels.max(1);
let monitoring = self.input.is_some() && self.session.tracks.iter().any(|t| t.mixer.input_monitor || t.mixer.record_arm);
let callback_frames = out.len() / channels;
// The speed is read once: the callback renders at the speed it keeps for the ages of
// the events the next one takes.
let speed = self.speed();
let step = self.step(speed);
let callback = ((callback_frames as f64 * step).ceil() as i64).max(0);
let age_speed = std::mem::replace(&mut self.midi_speed, speed);
// A pass runs while playing and recording, and on through a transport stop until the
// command that ends it. One the UI has begun during playback counts from when it began,
// though the command that starts it is only read below.
let running = self.recording && (self.playing || self.midi_stopped.is_some());
let begun = self.midi.as_ref().is_some_and(|f| f.begin_pending());
let pass = Pass {
record: running || (self.playing && begun),
running,
playing: self.playing,
begun,
until: self.midi_stopped.unwrap_or(i64::MAX),
stopped: self.midi_stopped.is_some(),
pos: self.pos,
looped: self.looped,
};
#[cfg(test)]
if let Some(hook) = &mut self.before_capture {
hook();
}
self.capture_midi(callback, age_speed, pass);
#[cfg(test)]
if let Some(hook) = &mut self.before_commands {
hook();
}
self.drain();
if let Some(ended) = self.midi_ended.take() {
// A command ended the pass. What arrived after the queue was read above and before
// that command belongs to the pass too: take it in as well, with the pass as it was,
// and only then say that everything up to the stop is written down.
// The device threads are kept out of the queue meanwhile, so that a message one
// of them stamped before the Stop is in the queue by now and not put in after the
// answer. (When one is in there this very moment and does not come out at once,
// the answer is given without that: the audio thread does not wait.)
let feed = self.midi.clone();
let quiet = feed.as_ref().and_then(|f| f.hold_writers());
// A pass that was started by a command of this same callback (while the transport
// was running) is still that pass here, though it no longer waits to be started.
let pass = Pass { begun: pass.begun || std::mem::take(&mut self.midi_started), ..pass };
self.capture_midi(callback, age_speed, pass);
if let Some(feed) = &feed {
feed.capture_done(ended, pass.pos);
}
drop(quiet);
}
self.midi_started = false;
// A command may have installed a session at another sample rate: the audio is rendered
// at the rate that is there now, at the same speed.
let step = self.step(speed);
// A keyboard plays through record-enabled and input-monitored MIDI and instrument tracks.
let midi_live = self.midi.as_ref().is_some_and(|f| f.connected() > 0)
&& self.session.tracks.iter().any(|t| matches!(t.kind, TrackKind::Instrument | TrackKind::Midi) && midi_thru(t));
let monitoring = midi_live || (self.input.is_some() && self.session.tracks.iter().any(|t| t.mixer.input_monitor || t.mixer.record_arm));
if !self.playing && !monitoring {
self.drop_midi();
out.iter_mut().for_each(|x| *x = 0.0);
self.unsent.silence();
self.forget_meters();
@@ -286,10 +692,10 @@ impl AudioState {
}
return;
}
let speed = f64::from(self.shared.speed.load(Ordering::Relaxed)) / 1000.0;
let step = self.session.sample_rate.as_f64() / self.device_rate.max(1.0) * speed.clamp(0.1, 4.0);
let callback_frames = out.len() / channels;
let quantum = ((callback_frames as f64 * step).ceil() as usize).clamp(1, BLOCK);
if !midi_live {
self.drop_midi();
}
for frame in out.chunks_mut(channels) {
while self.read >= self.pending_len as f64 {
self.read -= self.pending_len as f64;
@@ -318,6 +724,155 @@ impl AudioState {
}
}
/// A port that the bit at place `bit` of a lost-ports mask ([`midi_port_bit`]) stands for. The
/// last bit stands for the ports without an index, which share one time of loss.
fn bit_port(bit: u8) -> u8 {
if bit >= 63 { u8::MAX } else { bit }
}
/// Of the lost ports in `marks` ([`midi_port_bit`]), the one that lost its input first
/// (`lost_time`, by the place of each port's bit), if that was before `before` (or at any time,
/// for `i64::MAX`): its bit and the time.
fn next_mark(lost_time: &[i64; 64], marks: u64, before: i64) -> Option<(u64, i64)> {
(0..64u8)
.filter(|b| marks & (1u64 << b) != 0)
.map(|b| (1u64 << b, lost_time.get(usize::from(b)).copied().unwrap_or(i64::MIN)))
.filter(|(_, at)| *at < before || before == i64::MAX)
.min_by_key(|(_, at)| *at)
}
/// Note in `seen` (by the place of each port's bit) that the loss at `at` of the ports in `bits`
/// was dealt with from the ports' flags.
fn flag_seen(seen: &mut [i64; 64], bits: u64, at: i64) {
for (bit, seen) in seen.iter_mut().enumerate() {
if bits & (1u64 << bit) != 0 {
*seen = (*seen).max(at);
}
}
}
/// The record pass as a callback finds it, before it reads its commands: what the MIDI that
/// arrived until then is written down with.
#[derive(Clone, Copy)]
struct Pass {
/// A pass is running, or one was begun that the transport is running for.
record: bool,
/// A pass is running: recording, and playing or stopped with the pass still open.
running: bool,
/// The transport is running: a pass that is begun now (also while the queue is being read)
/// records from when it began.
playing: bool,
/// A pass was begun for the running transport when the callback started, or by one of its
/// commands.
begun: bool,
/// Entries stamped later than this are from after the transport stopped by itself.
until: i64,
/// The transport has stopped: the playhead stands where the pass ends.
stopped: bool,
/// The playhead at the start of the callback.
pos: Samples,
looped: Option<Range>,
}
/// Writes one callback's MIDI into the take: gives each entry its timeline position and says
/// where loop recording went round.
struct TakeWriter<'a> {
feed: &'a midi::MidiFeed,
/// The playhead at the start of this callback.
pos: Samples,
looped: Option<Range>,
/// Times the loop went round since the last callback, not yet written to the take.
wraps: u8,
/// A pass is running.
running: bool,
/// The transport is running: a pass begun meanwhile records from when it began.
playing: bool,
/// A pass was begun for the running transport, as far as the callback saw.
begun: bool,
/// Entries stamped later than this are not part of it either (the transport stopped then).
until: i64,
/// The playhead stands where the pass ends.
stopped: bool,
}
impl TakeWriter<'_> {
/// Whether an entry stamped `stamp` is part of the take: not from before the pass began, nor
/// from after it was ended. The end is asked for every entry: a Stop may come while the
/// queue is being read.
fn within(&self, stamp: i64) -> bool {
// Whether a pass was begun first, then when: the time is stored before the pass counts
// as begun, so a pass seen as begun here has its own start time.
let on = self.running || (self.playing && (self.begun || self.feed.begin_pending()));
on && stamp >= self.feed.record_from() && stamp <= self.until.min(self.feed.record_until())
}
/// Write down the event `e`, stamped `stamp`, that arrived `age` session frames ago.
fn event(&mut self, stamp: i64, age: i64, e: &LiveMidi) {
if self.within(stamp) {
let at = self.at(age);
self.feed.record(at, e);
}
}
/// Write down that the ports in `bits` lost input at the time `stamp`, `age` session frames
/// ago.
fn lost(&mut self, stamp: i64, age: i64, bits: u64) {
if self.within(stamp) {
let at = self.at(age);
self.feed.record_lost(at, bits);
}
}
/// The timeline position of what happened `age` session frames ago. Ages only get smaller
/// from one call to the next.
///
/// When the loop went round since the last callback (a short loop: perhaps several times),
/// the entry belongs to the round it arrived in: as many rounds back as its age reaches
/// behind the start of the loop. The rounds before that one are said to the take ahead of
/// the entry, the rest after it, so that two entries a round apart have one mark between
/// them however often the loop went round in this callback.
fn at(&mut self, age: i64) -> Samples {
let at = self.pos.saturating_sub(age.max(0));
let Some(l) = self.looped else {
// A pass that is being ended stops at the playhead, and its range ends there: what
// was played just before the Stop is the last thing inside it, not the first thing
// outside.
let ending = self.stopped || self.feed.record_until() != i64::MAX;
return if ending { at.min(self.pos.saturating_sub(1)) } else { at };
};
let len = l.len().max(1);
let back = if self.wraps == 0 || at >= l.start {
0
} else {
// Rounds between this entry and now. One from before every round counted here
// stays at the start of the oldest of them.
let rounds = l.start.saturating_sub(at).saturating_add(len - 1) / len;
u8::try_from(rounds).unwrap_or(u8::MAX).min(self.wraps)
};
self.wrapped(self.wraps - back);
self.wraps = back;
let at = if back > 0 { at.saturating_add(i64::from(back).saturating_mul(len)).max(l.start) } else { at };
// The playhead stands at the end of the loop until the next block takes it round (and
// stays there if the pass is stopped first): what is played now is the last thing of
// this round, not the first of a round that may never start.
if at >= l.end { l.end.saturating_sub(1).max(l.start) } else { at }
}
/// Say that the loop went round `rounds` times. Two marks stand for any more: a note held
/// that long covers the whole loop either way.
fn wrapped(&mut self, rounds: u8) {
for _ in 0..rounds.min(2) {
self.feed.record_wrap(self.looped.map_or(self.pos, |l| l.end));
}
}
/// Say that the loop went round, for the rounds no entry was written in.
fn flush(&mut self) {
let rounds = std::mem::take(&mut self.wraps);
self.wrapped(rounds);
}
}
/// Write one device frame from `nch` rendered channels: fold surround to stereo (a mono device
/// averages the two sides), or copy the channels straight through. `sample(c)` is channel `c`.
fn write_frame(frame: &mut [f32], channels: usize, nch: usize, sounding: bool, fold: &[(f32, f32)], sample: impl Fn(usize) -> f32) {
+24 -2
View File
@@ -22,7 +22,14 @@
//! through handles taken from each instance before it leaves ([`Player::open_editor`]).
#![deny(clippy::unwrap_used, clippy::expect_used, clippy::panic, clippy::unimplemented, clippy::todo, clippy::unreachable)]
/// Whether playback goes round `range` as a loop: a range of 64 frames or less is too short,
/// and is played once.
pub fn loops(range: Range) -> bool {
range.len() > 64
}
pub mod devices;
pub mod midi;
pub mod record;
pub use devices::AudioConfig;
@@ -118,7 +125,10 @@ enum Cmd {
/// Where the callback sends the snapshots it replaces (freeing one takes thousands of frees).
OldSessions(SyncSender<Arc<Session>>),
Input(Arc<record::InputRing>),
Recording(bool),
Midi(Arc<midi::MidiFeed>),
/// Recording on or off. Switching it off carries the number of the pass that ends
/// (`MidiFeed::end_pass`; 0: nobody waits for it), which the callback acknowledges.
Recording(bool, u64),
Play {
from: Samples,
end: Option<Samples>,
@@ -476,8 +486,20 @@ impl Player {
let _ = self.tx.send(Cmd::Input(ring));
}
/// Play MIDI devices through record-enabled and input-monitored MIDI and instrument tracks.
pub fn set_midi(&self, feed: Arc<midi::MidiFeed>) {
let _ = self.tx.send(Cmd::Midi(feed));
}
pub fn set_recording(&self, on: bool) {
let _ = self.tx.send(Cmd::Recording(on));
let _ = self.tx.send(Cmd::Recording(on, 0));
}
/// Switch recording off at the end of record pass `pass` ([`midi::MidiFeed::end_pass`]). The
/// audio thread answers on the feed once the MIDI played up to here is written down
/// ([`midi::MidiFeed::captured`]).
pub fn end_recording(&self, pass: u64) {
let _ = self.tx.send(Cmd::Recording(false, pass));
}
pub fn stop(&self) {
File diff suppressed because it is too large. Load diff
File diff suppressed because it is too large. Load diff
+21
View File
@@ -1527,6 +1527,27 @@ fn menu_translate(source: &str) -> Option<&'static str> {
"Autosave failed: {e}" => "Falha ao salvar automaticamente: {e}",
"Bounced {} (peak {:.1} dBFS, {:.1} LUFS)" => "Bounce concluído: {} (pico {:.1} dBFS, {:.1} LUFS)",
"Cannot record: {err}" => "Não foi possível gravar: {err}",
"Cannot record: no MIDI input device is connected (Setup > Playback Engine lists them)" => {
"Não foi possível gravar: nenhum dispositivo de entrada MIDI está conectado (Configuração > Mecanismo de reprodução mostra a lista)"
}
"Nothing was recorded: the record-enabled tracks received no input" => {
"Nada foi gravado: as faixas habilitadas para gravação não receberam entrada"
}
"MIDI input was dropped during the take: it is incomplete, and notes held at that point end there" => {
"Parte da entrada MIDI foi perdida durante o take: ele está incompleto, e as notas que estavam pressionadas terminam nesse ponto"
}
"The audio engine did not answer when the pass ended: the last MIDI notes played may be missing from the take" => {
"O mecanismo de áudio não respondeu quando a passagem terminou: as últimas notas MIDI tocadas podem estar faltando no take"
}
"Cannot record: the audio engine has not finished the previous record pass yet (is the audio device running?)" => {
"Não foi possível gravar: o mecanismo de áudio ainda não concluiu a passagem de gravação anterior (o dispositivo de áudio está funcionando?)"
}
"All MIDI" => "Todo MIDI",
"Channel" => "Canal",
"MIDI input" => "Entrada MIDI",
"All devices" => "Todos os dispositivos",
"No MIDI input device found" => "Nenhum dispositivo de entrada MIDI encontrado",
"{name} (not connected)" => "{name} (não conectado)",
"Click to add a MIDI clip to {name} at the insertion point." => "Clique para adicionar um clipe MIDI a {name} no ponto de inserção.",
"Could not open {path}: {e}" => "Não foi possível abrir {path}: {e}",
"Couldn't recover “{name}”: {e}" => "Não foi possível recuperar “{name}”: {e}",
+9 -4
View File
@@ -11,6 +11,7 @@ mod commands;
mod control;
mod files;
mod input;
mod midi;
mod playback;
mod plugins;
@@ -39,6 +40,12 @@ impl SoundApp {
record_start: 0,
punch: None,
recorder_failed: false,
midi_inputs: None,
midi_scan_at: 0.0,
midi_take: Vec::new(),
midi_take_lost: 0,
midi_take_inputs: Vec::new(),
play_loop: None,
lost_shown: 0,
take_plan: None,
ui: UiState::default(),
@@ -207,10 +214,8 @@ impl SoundApp {
self.autosave_changes();
self.write_automation();
self.autosave();
if self.recorder.is_none()
&& !self.recorder_failed
&& self.engine.session().tracks.iter().any(|t| t.mixer.input_monitor || t.mixer.record_arm)
{
self.service_midi(self.uptime);
if self.recorder.is_none() && !self.recorder_failed && audio::wants_audio_input(self.engine.session()) {
self.ensure_input();
}
// Autosave and the macOS menu bar only run on frames, and plugin editors are native windows egui never hears from.
+35 -1
View File
@@ -5,6 +5,7 @@ impl SoundApp {
pub(crate) fn meters_live(&self) -> bool {
self.engine.transport.playing
|| (self.recorder.is_some() && self.engine.session().tracks.iter().any(|t| t.mixer.input_monitor || t.mixer.record_arm))
|| (self.midi_connected() && self.engine.session().tracks.iter().any(|t| t.kind.is_midi() && soundcraft_mix::midi_thru(t)))
}
/// Reopens the audio devices with `cfg` (Setup › Playback Engine › Apply), keeping the
@@ -27,6 +28,7 @@ impl SoundApp {
self.recorder = None;
self.recorder_failed = false;
let player = soundcraft_playback::Player::with_config(self.engine.session_arc(), &self.ui.audio);
self.attach_midi(&player);
player.locate(at);
self.player = Some(player);
Ok(())
@@ -125,7 +127,9 @@ impl SoundApp {
}
self.recorder = None;
self.recorder_failed = false;
self.player = Some(Player::with_config(self.engine.session_arc(), &self.ui.audio));
let player = Player::with_config(self.engine.session_arc(), &self.ui.audio);
self.attach_midi(&player);
self.player = Some(player);
}
}
@@ -155,3 +159,33 @@ pub(super) fn feed_meter(d: &mut MeterDisplay, peak: [f32; 2], gr: f32, dt: f32)
}
d.gr = if gr > d.gr { gr } else { d.gr * fall };
}
/// Whether a track needs the audio input opened: an audio track (or an aux) that is
/// record-enabled or input-monitored. A MIDI or instrument track listens to MIDI devices, so
/// arming one does not open an audio device or complain that there is none.
pub(super) fn wants_audio_input(s: &soundcraft_model::Session) -> bool {
s.tracks.iter().any(|t| !t.kind.is_midi() && (t.mixer.input_monitor || t.mixer.record_arm))
}
#[cfg(test)]
mod audio_input_tests {
use super::wants_audio_input;
use soundcraft_model::{ChannelFormat, Session, TrackKind};
#[test]
fn only_audio_tracks_ask_for_the_audio_input() {
let mut s = Session::default();
let synth = s.add_track(TrackKind::Instrument, ChannelFormat::Stereo, None);
let midi = s.add_track(TrackKind::Midi, ChannelFormat::Stereo, None);
let vox = s.add_track(TrackKind::Audio, ChannelFormat::Mono, None);
assert!(!wants_audio_input(&s));
s.track_mut(synth).unwrap().mixer.record_arm = true;
s.track_mut(midi).unwrap().mixer.input_monitor = true;
assert!(!wants_audio_input(&s), "armed and monitored MIDI tracks listen to MIDI devices");
s.track_mut(vox).unwrap().mixer.input_monitor = true;
assert!(wants_audio_input(&s));
s.track_mut(vox).unwrap().mixer.input_monitor = false;
s.track_mut(vox).unwrap().mixer.record_arm = true;
assert!(wants_audio_input(&s));
}
}
+3
View File
@@ -138,6 +138,9 @@ impl SoundApp {
"dialog": self.dialogs.open_name(),
"frame_ms": self.frame_ms,
"audio_device": self.player.as_ref().map(|p| p.device_name.clone()),
"midi_inputs": self.midi_inputs.as_ref().map(|m| m.ports().iter().map(|p| json!({"name": p.name, "enabled": p.enabled, "open": p.open})).collect::<Vec<_>>()),
"midi_events": self.midi_inputs.as_ref().map(|m| m.feed().activity().0),
"midi_events_dropped": self.midi_inputs.as_ref().map(|m| m.feed().dropped()),
"status": self.ui.status,
})
}
+287
View File
@@ -0,0 +1,287 @@
//! MIDI input devices: opening them, following hotplug, and turning what is played during a
//! record pass into MIDI clips.
use super::{Range, Samples, SoundApp, json};
use soundcraft_engine::io::{MidiPass, MidiTakeEvent};
use soundcraft_playback::midi::MidiInputs;
impl SoundApp {
/// Whether a MIDI input port is open.
pub(crate) fn midi_connected(&self) -> bool {
self.midi_inputs.as_ref().is_some_and(|m| m.feed().connected() > 0)
}
/// Hand the MIDI inputs to a (new) audio engine.
pub(crate) fn attach_midi(&self, player: &soundcraft_playback::Player) {
if let Some(m) = &self.midi_inputs {
// A pass the old engine never confirmed is not this one's to answer: it starts idle.
m.feed().forgive_passes();
player.set_midi(std::sync::Arc::clone(m.feed()));
}
}
/// Whether the audio engine still owes the confirmation of an earlier record pass
/// ([`Self::await_midi_capture`] gave up on it): a pass that records MIDI waits for it
/// (`MidiFeed::pass_owed` says why).
pub(crate) fn midi_pass_pending(&self) -> bool {
self.midi_inputs.as_ref().is_some_and(|m| m.feed().pass_owed())
}
/// A record pass starts: whatever was played before is not part of it.
pub(crate) fn begin_midi_take(&mut self) {
if let Some(m) = &self.midi_inputs {
m.feed().take_recorded(&mut self.midi_take);
m.feed().take_recorded_lost();
}
self.midi_take.clear();
self.midi_take_lost = 0;
self.midi_take_inputs = soundcraft_engine::io::midi_record_inputs(&self.engine);
// From here on what is played belongs to the pass (punching in while playing: also what
// the audio engine takes in before it hears of the pass).
if let Some(m) = &self.midi_inputs {
m.feed().begin_pass();
}
}
/// A record pass is about to be stopped: its number on the MIDI feed, when there is MIDI to
/// wait for. The audio engine is told to stop recording with it
/// (`Player::end_recording`) and answers for that pass alone ([`Self::await_midi_capture`]).
///
/// That is decided by what the pass set out to record, not by the devices connected now: a
/// pass whose last device was unplugged before Stop still has its notes on the way to the
/// take, and the engine still has to say it is done with it before another pass starts.
pub(crate) fn end_midi_pass(&self) -> Option<u64> {
let feed = self.midi_inputs.as_ref()?.feed();
midi_pass_waits(self.player.is_some(), self.midi_take_inputs.len()).then(|| feed.end_pass())
}
/// After the audio engine was told to stop recording: wait until it has written down the MIDI
/// played up to the stop of `pass`, so that a note pressed just before Stop is in the clip.
/// That takes one audio callback (milliseconds). An audio device that has stopped calling
/// back never answers; the wait gives up after a tenth of a second and the take is built
/// from what is there. False when it gave up: the take may lack its last notes. The answer
/// to an earlier pass that comes in late does not count for this one.
pub(crate) fn await_midi_capture(&self, pass: Option<u64>) -> bool {
let (Some(pass), Some(m)) = (pass, &self.midi_inputs) else { return true };
#[cfg(not(target_arch = "wasm32"))]
{
let start = std::time::Instant::now();
while !m.feed().captured(pass) && start.elapsed() < std::time::Duration::from_millis(100) {
std::thread::sleep(std::time::Duration::from_millis(1));
}
m.feed().captured(pass)
}
#[cfg(target_arch = "wasm32")]
{
let _ = (pass, m);
true
}
}
/// Where the audio engine's playhead stood when it confirmed the end of `pass`
/// ([`Self::await_midi_capture`]): the stop position that goes with the take it wrote.
/// `None` when it did not confirm, or there was no MIDI to wait for.
pub(crate) fn midi_pass_stopped_at(&self, pass: Option<u64>) -> Option<Samples> {
self.midi_inputs.as_ref()?.feed().stopped_at(pass?)
}
/// Turn the MIDI played during the pass into clips on the MIDI and instrument tracks that
/// were record-enabled when it started, and on those only: the same tracks the end of the
/// pass waited for ([`Self::end_midi_pass`]). `with_audio`: audio takes of this pass already
/// made its undo step. `confirmed`: the audio engine said it had written down everything
/// played up to the stop ([`Self::await_midi_capture`]). `pass`: whether it was a loop pass,
/// and where that stopped inside the loop; keys still down end there. Returns how many
/// clips, and what to tell the user when the take is not complete.
pub(crate) fn finish_midi_take(&mut self, range: Range, with_audio: bool, confirmed: bool, pass: MidiPass) -> (usize, Option<String>) {
let Some(m) = &self.midi_inputs else { return (0, None) };
let feed = m.feed();
feed.take_recorded(&mut self.midi_take);
// Events the audio thread could not hand over (this thread fell behind).
let uncollected = self.midi_take_lost.saturating_add(u64::from(feed.take_recorded_lost()));
self.midi_take_lost = 0;
let events: Vec<MidiTakeEvent> = self
.midi_take
.drain(..)
.map(|e| MidiTakeEvent {
at: e.at,
device: e.port,
status: if e.is_lost() {
MidiTakeEvent::LOST
} else if e.is_wrap() {
MidiTakeEvent::WRAP
} else {
e.status
},
data1: e.data1,
data2: e.data2,
})
.collect();
let devices: Vec<String> = (0..soundcraft_mix::MAX_MIDI_PORTS)
.map(|i| u8::try_from(i).ok().and_then(|i| feed.ports().name(i)).unwrap_or_default().to_string())
.collect();
let pass = pass_with_known_stop(pass, uncollected);
let inputs = std::mem::take(&mut self.midi_take_inputs);
let take = soundcraft_engine::io::add_midi_recording_from(&mut self.engine, range, pass, &events, &devices, &inputs, with_audio);
(take.clips.len(), midi_take_problem(take.gaps, uncollected, confirmed))
}
/// Open the MIDI inputs once, hand them to the audio engine, and keep following devices that
/// are plugged in or removed. Collects what is played while recording.
pub(crate) fn service_midi(&mut self, now: f64) {
let Some(p) = &self.player else { return };
match &mut self.midi_inputs {
None => {
let m = MidiInputs::open(self.ui.midi_inputs_off.clone());
p.set_midi(std::sync::Arc::clone(m.feed()));
self.midi_inputs = Some(m);
self.midi_scan_at = now;
}
Some(m) => {
if (now - self.midi_scan_at).abs() >= 2.0 {
self.midi_scan_at = now;
m.rescan();
}
if self.engine.transport.recording {
m.feed().take_recorded(&mut self.midi_take);
self.midi_take_lost = self.midi_take_lost.saturating_add(u64::from(m.feed().take_recorded_lost()));
// A take longer than the engine keeps: say so instead of growing without end.
let most = soundcraft_engine::io::MAX_MIDI_TAKE_EVENTS;
if self.midi_take.len() > most {
self.midi_take_lost = self.midi_take_lost.saturating_add((self.midi_take.len() - most) as u64);
self.midi_take.truncate(most);
}
}
}
}
}
/// Use or ignore a MIDI input port. Remembered in the UI prefs.
pub fn set_midi_input_enabled(&mut self, name: &str, on: bool) {
if let Some(m) = &mut self.midi_inputs {
m.set_enabled(name, on);
self.ui.midi_inputs_off = m.disabled().to_vec();
}
// The session's own record of the devices in use follows the switch.
let _ = self.run("setup.midi_input_devices", json!({ if on { "enable" } else { "disable" }: [name] }));
}
}
/// Whether the end of a record pass waits for the audio engine's answer: there is an engine
/// (without one nothing takes MIDI in), and `tracks` MIDI or instrument tracks were
/// record-enabled when the pass started. How many devices are connected at Stop does not count.
fn midi_pass_waits(engine: bool, tracks: usize) -> bool {
engine && tracks > 0
}
/// `pass` without the stop position of a loop pass when `uncollected` recorded entries never
/// reached the take. One of them may be the mark of the loop going round: the stop position
/// would then be read in the wrong round, and a key held to the stop would end early. Without
/// it the keys still down are held to the end of the loop.
fn pass_with_known_stop(pass: MidiPass, uncollected: u64) -> MidiPass {
match pass {
MidiPass::Loop { marked, .. } if uncollected > 0 => MidiPass::Loop { stopped: None, marked },
pass => pass,
}
}
/// Where the MIDI of a record pass goes and how the pass ran over that range. `looped` is the
/// loop the pass recorded over: the loop of loop recording, or the loop that was playing when
/// recording was punched in (the playhead goes on round it, so the take is a loop take though
/// the audio is laid out in a line). `stopped` is where the audio engine said a loop pass
/// stopped. Otherwise the pass ran in a line, over the punch range or from where it started to
/// where it stopped.
pub(crate) fn midi_take_span(
looped: Option<Range>,
punch: Option<Range>,
started: Samples,
stopped_at: Samples,
stopped: Option<Samples>,
) -> (Range, MidiPass) {
match looped {
Some(l) => (l, MidiPass::Loop { stopped, marked: true }),
None => (punch.unwrap_or(Range::new(started, stopped_at)), MidiPass::Linear),
}
}
/// Where a record pass that plays on in a line stopped, from the audio engine's position
/// (`None`: no engine) and the UI's copy of it, which is a frame behind: the later of the two.
/// (A loop pass gets its stop position from the audio engine's confirmation instead:
/// `SoundApp::midi_pass_stopped_at`.)
pub(crate) fn pass_stop(engine: Option<Samples>, ui: Samples) -> Samples {
engine.map_or(ui, |at| at.max(ui))
}
/// What to tell the user about a MIDI take that is not complete: input was dropped at `gaps`
/// places, `uncollected` recorded events never reached this thread, or the audio engine did not
/// confirm the end of the pass (`confirmed` false). `None` for a complete take.
fn midi_take_problem(gaps: usize, uncollected: u64, confirmed: bool) -> Option<String> {
let mut problems = Vec::new();
if gaps > 0 || uncollected > 0 {
log::warn!("MIDI recording: input was dropped at {gaps} place(s); {uncollected} recorded event(s) were not collected");
problems.push(crate::localization::text("MIDI input was dropped during the take: it is incomplete, and notes held at that point end there"));
}
if !confirmed {
log::warn!("MIDI recording: the audio engine did not confirm the end of the pass; the last notes may be missing");
problems.push(crate::localization::text(
"The audio engine did not answer when the pass ended: the last MIDI notes played may be missing from the take",
));
}
(!problems.is_empty()).then(|| problems.join("; "))
}
#[cfg(test)]
mod tests {
use super::{MidiPass, Range, midi_pass_waits, midi_take_problem, midi_take_span, pass_stop, pass_with_known_stop};
#[test]
fn the_stop_of_a_loop_pass_is_not_used_when_entries_of_the_take_went_missing() {
let pass = MidiPass::Loop { stopped: Some(24_000), marked: true };
assert_eq!(pass_with_known_stop(pass, 0), pass, "a whole take keeps its stop");
// One entry never arrived: it may be the mark of a round, so the round of the stop is
// not known.
assert_eq!(pass_with_known_stop(pass, 1), MidiPass::Loop { stopped: None, marked: true });
assert_eq!(pass_with_known_stop(MidiPass::Linear, 3), MidiPass::Linear);
}
#[test]
fn a_pass_punched_in_while_a_loop_plays_is_a_loop_take_over_that_loop() {
// The loop 0..96 000 plays; Record at 72 000, Stop at 24 000 in the next round. In a
// line from 72 000 to 24 000 nothing played would be inside the range.
let l = Range::new(0, 96_000);
assert_eq!(midi_take_span(Some(l), None, 72_000, 24_000, Some(24_000)), (l, MidiPass::Loop { stopped: Some(24_000), marked: true }));
// The audio engine did not say where it stopped: still that loop.
assert_eq!(midi_take_span(Some(l), None, 72_000, 24_000, None), (l, MidiPass::Loop { stopped: None, marked: true }));
// No loop: the punch range, or from the start of the pass to its stop.
let punch = Range::new(10, 20);
assert_eq!(midi_take_span(None, Some(punch), 0, 50, None), (punch, MidiPass::Linear));
assert_eq!(midi_take_span(None, None, 5, 50, Some(7)), (Range::new(5, 50), MidiPass::Linear));
}
#[test]
fn a_pass_in_a_line_ends_at_the_later_of_the_engines_position_and_the_uis() {
assert_eq!(pass_stop(Some(1_000), 95_000), 95_000);
assert_eq!(pass_stop(Some(96_000), 95_000), 96_000);
assert_eq!(pass_stop(None, 7), 7, "without an engine the UI's is all there is");
}
#[test]
fn a_pass_waits_for_the_engine_whatever_is_connected_when_it_stops() {
// The last device was unplugged before Stop: the pass still recorded MIDI tracks.
assert!(midi_pass_waits(true, 1));
assert!(!midi_pass_waits(true, 0), "no MIDI track was recorded");
assert!(!midi_pass_waits(false, 2), "no engine takes MIDI in");
}
#[test]
fn a_take_whose_end_was_not_confirmed_is_reported_as_incomplete() {
assert_eq!(midi_take_problem(0, 0, true), None, "a complete take has nothing to report");
// The audio engine never answered: no loss was counted, but the last notes may be missing.
let unconfirmed = midi_take_problem(0, 0, false).unwrap();
assert!(unconfirmed.contains("did not answer") && unconfirmed.contains("may be missing"), "{unconfirmed}");
let dropped = midi_take_problem(1, 0, true).unwrap();
assert!(dropped.contains("was dropped") && !dropped.contains("did not answer"), "{dropped}");
assert!(midi_take_problem(0, 3, true).is_some());
let both = midi_take_problem(2, 0, false).unwrap();
assert!(both.contains("was dropped") && both.contains("did not answer"), "{both}");
}
}
+80 -14
View File
@@ -36,6 +36,8 @@ impl SoundApp {
} else {
self.sim = Some((from, end, looped));
}
// The loop as the audio engine takes it: a range too short to loop is played once.
self.play_loop = looped.filter(|l| soundcraft_playback::loops(*l));
self.engine.transport.playing = true;
self.engine.transport.position = from;
self.last_write_at = i64::MIN;
@@ -57,6 +59,7 @@ impl SoundApp {
self.finish_recording();
}
self.sim = None;
self.play_loop = None;
self.engine.transport.playing = false;
self.engine.transport.recording = false;
let stop_at = self.engine.transport.position;
@@ -100,15 +103,42 @@ impl SoundApp {
self.record_hint = Some((self.last_frame.unwrap_or_default(), why));
return;
}
self.ensure_input();
let takes = match self.arm_recorder() {
Ok(takes) => takes,
Err(err) => {
self.ui.status = crate::localized_format!("Cannot record: {err}", err = err);
return;
// A pass records audio, MIDI or both: it needs one of them ready. The audio input is
// only opened for audio tracks. An inactive track keeps its record button but takes no
// part (`plan_takes` leaves it out too), so it must not make the pass an audio one.
let audio_tracks = self.engine.session().tracks.iter().any(|t| t.mixer.record_arm && !t.inactive && !t.kind.is_midi());
let midi_tracks = soundcraft_engine::io::records_midi(&self.engine);
let midi_ready = midi_tracks && self.midi_connected();
if midi_tracks && self.midi_pass_pending() {
self.ui.status = crate::localization::text(
"Cannot record: the audio engine has not finished the previous record pass yet (is the audio device running?)",
)
.into();
return;
}
if audio_tracks {
self.ensure_input();
}
let takes = if audio_tracks {
// An audio track is record-enabled: its audio has to be recordable. A pass that would
// quietly record only the MIDI tracks is refused, with the reason.
match self.arm_recorder() {
Ok(takes) => takes,
Err(err) => {
self.ui.status = crate::localized_format!("Cannot record: {err}", err = err);
return;
}
}
} else if midi_ready {
// MIDI alone makes a pass.
Vec::new()
} else {
self.ui.status =
crate::localization::text("Cannot record: no MIDI input device is connected (Setup > Playback Engine lists them)").into();
return;
};
self.lost_shown = 0;
self.begin_midi_take();
if let Some(p) = &self.player {
p.set_recording(true);
}
@@ -123,7 +153,9 @@ impl SoundApp {
}
let e = self.engine.session().edit.clone();
let sel = e.selection;
if e.loop_record && !sel.is_empty() {
// Loop recording needs a selection the transport will loop: one too short for that is
// recorded as a punch, like any other selection.
if e.loop_record && !sel.is_empty() && soundcraft_playback::loops(sel) {
self.record_start = sel.start;
self.punch = None;
self.start_play(sel.start, None, Some(sel));
@@ -199,12 +231,23 @@ impl SoundApp {
fn finish_recording(&mut self) {
self.engine.transport.recording = false;
let midi_pass = self.end_midi_pass();
if let Some(p) = &self.player {
p.set_recording(false);
match midi_pass {
Some(pass) => p.end_recording(pass),
None => p.set_recording(false),
}
}
// The audio input stops taking in first: waiting for the MIDI below would otherwise add
// what the input hears meanwhile to the audio takes. No input device is open when the
// pass was MIDI only.
let (takes, problem) = self.recorder.as_mut().map_or_else(|| (Vec::new(), None), |r| r.finish());
// The audio thread writes down what was played up to the stop; the clips wait for that.
let midi_confirmed = self.await_midi_capture(midi_pass);
// Where the pass ended: the audio engine's own position is the freshest.
let engine_at = self.player.as_ref().map(|p| p.position());
let stopped_at = super::midi::pass_stop(engine_at, self.engine.transport.position);
let plan = self.take_plan.take();
let Some(r) = &mut self.recorder else { return };
let (takes, problem) = r.finish();
let tracks = takes.len().max(1);
let plan = plan.unwrap_or_else(|| soundcraft_engine::io::Recording {
start: self.record_start,
@@ -216,11 +259,33 @@ impl SoundApp {
self.punch = None;
self.autosaved_rev = 0;
let ids = soundcraft_engine::io::place_recording(&mut self.engine, &plan, takes);
self.ui.status = if plan.pass.is_some() {
crate::localized_format!("Loop-recorded {} take(s)", ids.len() / tracks)
} else {
crate::localized_format!("Recorded {} clip(s)", ids.len())
// The audio engine may have confirmed the pass while the audio takes were finished: the
// stop position and what the user is told go by one answer.
let midi_confirmed = midi_confirmed || self.midi_pass_stopped_at(midi_pass).is_some();
// MIDI goes where the audio of the pass goes: the loop, the punch range, or all of it. A
// pass punched in while a loop was playing went on round that loop.
let midi_loop = match plan.pass {
Some(len) => Some(Range::new(plan.start, plan.start.saturating_add(len))),
None => self.play_loop,
};
// A loop pass stopped somewhere inside the loop: where, the audio engine said with its
// confirmation, in step with the rounds it wrote into the take. Its position now may be
// a round further on; without the confirmation there is no position to trust, and the
// keys still down are held to the end of the loop (the take is reported as incomplete).
let (midi_range, midi_kind) =
super::midi::midi_take_span(midi_loop, plan.punch, self.record_start, stopped_at, self.midi_pass_stopped_at(midi_pass));
let (midi_clips, midi_problem) = self.finish_midi_take(midi_range, !ids.is_empty(), midi_confirmed, midi_kind);
self.ui.status = if plan.pass.is_some() && midi_clips == 0 {
crate::localized_format!("Loop-recorded {} take(s)", ids.len() / tracks)
} else if ids.is_empty() && midi_clips == 0 {
crate::localization::text("Nothing was recorded: the record-enabled tracks received no input").to_string()
} else {
crate::localized_format!("Recorded {} clip(s)", ids.len() + midi_clips)
};
// A take that lost MIDI input, or whose end was not confirmed, is not reported as complete.
if let Some(p) = midi_problem {
self.ui.status = format!("{} ({p})", self.ui.status);
}
if let Some(p) = problem {
log::warn!("recording: {p}");
self.ui.status = format!("{} ({p})", self.ui.status);
@@ -376,6 +441,7 @@ impl SoundApp {
if let Some(sim) = &mut self.sim {
*sim = (at, None, None);
}
self.play_loop = None;
self.last_write_at = i64::MIN;
self.engine.transport.position = at;
} else {
+16
View File
@@ -156,6 +156,8 @@ pub struct UiState {
pub window_size: [f32; 2],
pub window_maximized: bool,
pub ui_scale: f32,
/// MIDI input ports switched off in Setup, by name (all others are used).
pub midi_inputs_off: Vec<String>,
}
impl Default for UiState {
@@ -224,6 +226,7 @@ impl Default for UiState {
window_size: [1600.0, 1000.0],
window_maximized: false,
ui_scale: 1.0,
midi_inputs_off: Vec::new(),
}
}
}
@@ -288,6 +291,19 @@ pub struct SoundApp {
/// Punch range to trim the take to (pre/post-roll recordings).
punch: Option<Range>,
recorder_failed: bool,
/// MIDI device input, opened on the first frame of an app that has an audio engine.
pub midi_inputs: Option<soundcraft_playback::midi::MidiInputs>,
/// When the MIDI ports were last looked for (app uptime, seconds).
midi_scan_at: f64,
/// MIDI played during the current record pass.
midi_take: Vec<soundcraft_playback::midi::RecordedMidi>,
/// Recorded MIDI events of the current pass that were dropped before they got here.
midi_take_lost: u64,
/// The MIDI input each record-enabled track had when the current pass started: the pass
/// records from it even if the track is switched to another device meanwhile.
midi_take_inputs: Vec<(TrackId, soundcraft_model::Route)>,
/// The loop the transport is playing round, when it was started with one.
play_loop: Option<Range>,
/// Input this take lost to a slow disk, as last reported (ms).
lost_shown: u64,
/// Where the takes being recorded go, autosaved with the session so a crash can't lose them.
+1
View File
@@ -526,6 +526,7 @@ const MENU_WINDOWS: &[(&str, &str)] = &[
("Setup > Appearance", "window.ui_customization"),
("Setup > Hardware...", "window.playback_engine"),
("Setup > Playback Engine...", "window.playback_engine"),
("Setup > MIDI > MIDI Input Devices...", "window.playback_engine"),
("Setup > I/O...", "window.io_setup"),
("Setup > Keyboard Shortcuts...", "window.shortcuts"),
("Setup > Session", "setup.session"),
+91 -2
View File
@@ -10,8 +10,68 @@ pub(crate) fn route_name(app: &SoundApp, r: &Route) -> String {
}
}
/// The input selector's text on a MIDI or instrument track.
pub(crate) fn midi_input_name(track: &Track) -> String {
let input = soundcraft_model::MidiInput::of(&track.mixer.input);
if input.device.is_none() && input.channel.is_none() { crate::localization::text("All MIDI").to_string() } else { input.label() }
}
/// The MIDI input menu of a MIDI or instrument track: a device (or all) and a channel (or all).
pub(crate) fn midi_input_menu(app: &mut SoundApp, ui: &mut Ui, id: TrackId) {
let Some(route) = app.engine.session().track(id).map(|t| t.mixer.input.clone()) else { return };
let now = soundcraft_model::MidiInput::of(&route);
let (device, channel) = (now.device.map(str::to_string), now.channel);
let mut pick: Option<(Option<String>, Option<u8>)> = None;
let mut devices: Vec<(String, bool)> =
app.midi_inputs.as_ref().map(|m| m.ports().iter().filter(|p| p.enabled).map(|p| (p.name.clone(), p.open)).collect()).unwrap_or_default();
// A device the track listens to stays in the list while it is unplugged.
if let Some(d) = &device
&& !devices.iter().any(|(n, _)| n == d)
{
devices.push((d.clone(), false));
}
ui.strong(crate::localization::text("MIDI input"));
if ui.selectable_label(device.is_none(), crate::localization::text("All devices")).clicked() {
pick = Some((None, channel));
}
for (name, open) in &devices {
let text = if *open { name.clone() } else { crate::localized_format!("{name} (not connected)", name = name) };
if ui.selectable_label(device.as_deref() == Some(name.as_str()), text).clicked() {
pick = Some((Some(name.clone()), channel));
}
}
if devices.is_empty() {
ui.weak(crate::localization::text("No MIDI input device found"));
}
ui.separator();
ui.strong(crate::localization::text("Channel"));
egui::Grid::new(("midi-channel", id.0)).spacing(egui::vec2(2.0, 2.0)).show(ui, |ui| {
if ui.selectable_label(channel.is_none(), crate::localization::text("All")).clicked() {
pick = Some((device.clone(), None));
}
ui.end_row();
for ch in 0..16u8 {
if ui.selectable_label(channel == Some(ch), format!("{:>2}", ch + 1)).clicked() {
pick = Some((device.clone(), Some(ch)));
}
if ch % 8 == 7 {
ui.end_row();
}
}
});
if let Some((device, channel)) = pick {
let input = match (soundcraft_model::MidiInput { device: device.as_deref(), channel }).route() {
Route::Hardware(h) => h,
_ => "none".to_string(),
};
let _ = app.run("track.input", json!({"track": id.0, "input": input}));
}
}
pub(crate) fn input_name(app: &SoundApp, track: &Track) -> String {
if track.kind.has_playlist() && track.mixer.input == Route::None {
if track.kind.is_midi() {
midi_input_name(track)
} else if track.kind.has_playlist() && track.mixer.input == Route::None {
let channels = track.format.channels();
if channels == 1 { crate::localized_format!("In {}", 1) } else { crate::localized_format!("In 1-{}", channels) }
} else {
@@ -22,7 +82,11 @@ pub(crate) fn input_name(app: &SoundApp, track: &Track) -> String {
/// The input or output menu under `resp`. It closes only on a click outside, so its bus search
/// field can be typed in.
pub(crate) fn route_popup(app: &mut SoundApp, resp: &egui::Response, id: TrackId, input: bool) {
egui::Popup::menu(resp).close_behavior(egui::PopupCloseBehavior::CloseOnClickOutside).show(|ui| route_menu(app, ui, id, input));
// The input of a MIDI or instrument track is a MIDI device and channel.
let midi = input && app.engine.session().track(id).is_some_and(|t| t.kind.is_midi());
egui::Popup::menu(resp)
.close_behavior(egui::PopupCloseBehavior::CloseOnClickOutside)
.show(|ui| if midi { midi_input_menu(app, ui, id) } else { route_menu(app, ui, id, input) });
}
/// Input or output menu: no route and buses on both, plus the main output (outputs) or inputs 1-8
@@ -91,3 +155,28 @@ mod tests {
assert_eq!(input_name(&app, track), "In 1-2");
}
}
#[cfg(test)]
mod midi_input_tests {
use super::*;
#[test]
fn a_midi_track_names_its_device_and_channel_on_the_input_selector() {
let mut app = SoundApp::new(soundcraft_engine::Engine::default(), None, crate::Services::default());
let midi = app.run("track.new", json!({"name": "Synth", "kind": "instrument"})).unwrap()["tracks"][0].as_u64().unwrap();
let audio = app.run("track.new", json!({"name": "Vox", "format": "mono"})).unwrap()["tracks"][0].as_u64().unwrap();
let shown = |app: &SoundApp, id: u64| input_name(app, app.engine.session().track(TrackId(id)).unwrap());
assert_eq!(shown(&app, midi), "All MIDI", "a new MIDI track listens to every device and channel");
for (input, text) in [
("MIDI:10:Drum Pad", "Drum Pad ch 10"),
("MIDI:*:Keys", "Keys"),
("MIDI:2:*", "All ch 2"),
("MIDI:99:*", "All MIDI"),
("none", "All MIDI"),
] {
app.run("track.input", json!({"tracks": [midi], "input": input})).unwrap();
assert_eq!(shown(&app, midi), text, "{input}");
}
assert_eq!(shown(&app, audio), "In 1", "an audio track still shows its audio input");
}
}
+86
View File
@@ -508,6 +508,90 @@ fn number_combo(ui: &mut egui::Ui, id: &str, name: &str, value: &mut Option<u32>
combo.response.accessible_name(crate::localization::text(name));
}
/// What a MIDI message is, for the "last received" readout.
fn midi_event_text(e: &soundcraft_mix::LiveMidi) -> String {
let ch = e.channel() + 1;
match e.status & 0xF0 {
0x90 if e.data2 > 0 => format!("note {} on, velocity {}, channel {ch}", soundcraft_midi::note_name(e.data1), e.data2),
0x80 | 0x90 => format!("note {} off, channel {ch}", soundcraft_midi::note_name(e.data1)),
0xA0 => format!("aftertouch {} on note {}, channel {ch}", e.data2, soundcraft_midi::note_name(e.data1)),
0xB0 => format!("controller {} = {}, channel {ch}", e.data1, e.data2),
0xC0 => format!("program {}, channel {ch}", u16::from(e.data1) + 1),
0xD0 => format!("aftertouch {}, channel {ch}", e.data1),
0xE0 => format!("pitch bend {}, channel {ch}", (i32::from(e.data2) << 7 | i32::from(e.data1)) - 8192),
_ => format!("message {:#04x}", e.status),
}
}
/// The MIDI inputs of Setup › Playback Engine: every input port with a switch, and on request what
/// arrives from it, to see that a keyboard gets through.
fn midi_inputs(app: &mut SoundApp, ui: &mut egui::Ui) {
ui.strong("MIDI inputs");
let Some(m) = &app.midi_inputs else {
ui.weak("No audio engine, so MIDI devices are not opened.");
return;
};
if let Some(e) = m.error() {
ui.colored_label(ui.visuals().warn_fg_color, e);
return;
}
let ports = m.ports().to_vec();
if ports.is_empty() {
ui.weak("No MIDI input device found. Plug one in: it shows up here by itself.");
}
// Per port name: whether its debug readout is shown, and the last message seen from it.
let debug_id = egui::Id::unique("playback-engine-midi-debug");
let mut debug: std::collections::BTreeMap<String, (bool, Option<String>)> = ui.data(|d| d.get_temp(debug_id)).unwrap_or_default();
let (count, last) = m.feed().activity();
let dropped = m.feed().dropped();
if dropped > 0 {
ui.colored_label(
ui.visuals().warn_fg_color,
format!("{dropped} MIDI event(s) arrived faster than they could be played and were dropped; held notes of that device were released."),
);
}
if let Some(e) = last
&& let Some(from) = m.feed().ports().name(e.port)
{
debug.entry(from.to_string()).or_default().1 = Some(format!("{} (message {count} of all devices)", midi_event_text(&e)));
}
let mut toggle = None;
egui::Grid::new("pe-midi").num_columns(3).show(ui, |ui| {
for p in &ports {
let mut on = p.enabled;
if crate::widgets::checkbox(ui, &mut on, &p.name).on_hover_text("Play and record from this device").changed() {
toggle = Some((p.name.clone(), on));
}
match (p.enabled, p.open) {
(false, _) => ui.weak("off"),
(true, true) => ui.label("connected"),
(true, false) => ui.colored_label(ui.visuals().warn_fg_color, "could not be opened"),
};
let shown = &mut debug.entry(p.name.clone()).or_default().0;
if ui.selectable_label(*shown, "Test").on_hover_text("Show what arrives from this device").clicked() {
*shown = !*shown;
}
ui.end_row();
}
});
if let Some((name, on)) = toggle {
app.set_midi_input_enabled(&name, on);
}
for p in &ports {
let Some((true, seen)) = debug.get(&p.name) else { continue };
ui.separator();
ui.label(&p.name);
ui.weak(format!("Enabled: {}, open: {}", if p.enabled { "yes" } else { "no" }, if p.open { "yes" } else { "no" }));
ui.weak(match seen {
Some(text) => format!("Last received: {text}"),
None => "Last received: nothing yet".to_string(),
});
// Keep the readout live while it is shown.
ui.ctx().request_repaint_after_secs(0.1);
}
ui.data_mut(|d| d.insert_temp(debug_id, debug));
}
/// Setup › Playback Engine / Hardware: choose the audio system, devices, rate and buffer size.
pub fn playback_engine(app: &mut SoundApp, ctx: &egui::Context) {
let mut open = app.ui.show_playback_engine;
@@ -615,6 +699,8 @@ pub fn playback_engine(app: &mut SoundApp, ctx: &egui::Context) {
if ui.button(crate::localization::text("Reconnect audio device")).clicked() {
app.reconnect_audio();
}
ui.separator();
midi_inputs(app, ui);
});
if refresh {
audio_systems(ctx, true);
+1
View File
@@ -30,6 +30,7 @@ mod menu_search;
mod midi_double_click;
mod midi_editor_keys;
mod midi_front_visibility;
mod midi_inputs;
mod midi_panel;
mod midi_quantize;
mod mix_narrow_headers;
+230
View File
@@ -0,0 +1,230 @@
//! Setup › Playback Engine lists the MIDI input devices with a switch each: the switches are
//! labelled with the device's name, work from the keyboard, are kept in the app preferences, and
//! the list follows devices that are plugged in or removed. The monitor shows what arrives.
use egui::accesskit::Role;
use egui::{Event, Key, Modifiers, vec2};
use egui_kittest::{
Harness,
kittest::{NodeT, Queryable},
};
use soundcraft_playback::midi::MidiInputs;
use soundcraft_ui_egui::{Services, SoundApp, UiState};
fn names(list: &[&str]) -> Vec<String> {
list.iter().map(|n| (*n).to_string()).collect()
}
/// The app with the Playback Engine window open and two MIDI devices, none of them real.
fn harness(disabled: &[&str]) -> Harness<'static, SoundApp> {
let mut app = SoundApp::new(soundcraft_engine::Engine::default(), None, Services::default());
app.midi_inputs = Some(MidiInputs::with_port_names(names(disabled), names(&["Keys", "Drum Pad"])));
app.ui.midi_inputs_off = names(disabled);
app.ui.show_playback_engine = true;
let mut h = Harness::builder().with_size(vec2(1600.0, 1000.0)).build_ui_state(
|ui, app: &mut SoundApp| {
app.logic(&ui.ctx().clone());
app.ui(ui);
},
app,
);
h.run_steps(3);
h
}
fn press(h: &mut Harness<'_, SoundApp>, key: Key) {
h.event(Event::Key { key, physical_key: None, pressed: true, repeat: false, modifiers: Modifiers::NONE });
h.run_steps(1);
h.event(Event::Key { key, physical_key: None, pressed: false, repeat: false, modifiers: Modifiers::NONE });
h.run_steps(2);
}
fn port(h: &Harness<'_, SoundApp>, name: &str) -> Option<(bool, bool)> {
h.state().midi_inputs.as_ref().and_then(|m| m.ports().iter().find(|p| p.name == name).map(|p| (p.enabled, p.open)))
}
#[test]
fn every_midi_device_has_a_labelled_switch_that_shows_its_state() {
let h = harness(&["Drum Pad"]);
h.get_by_label("MIDI inputs");
// A checkbox per device, named after it, so a screen reader says which device it switches.
let keys = h.get_by_role_and_label(Role::CheckBox, "Keys");
let pad = h.get_by_role_and_label(Role::CheckBox, "Drum Pad");
assert_eq!(keys.accesskit_node().toggled(), Some(egui::accesskit::Toggled::True));
assert_eq!(pad.accesskit_node().toggled(), Some(egui::accesskit::Toggled::False));
// The state is also there as text, not only as a tick.
h.get_by_label("connected");
h.get_by_label("off");
assert_eq!(h.query_all_by_label("Test").count(), 2, "each device has its monitor");
}
#[test]
fn a_switch_works_from_the_keyboard_and_is_kept_in_the_preferences() {
let mut h = harness(&[]);
assert_eq!(port(&h, "Keys"), Some((true, true)));
h.get_by_role_and_label(Role::CheckBox, "Keys").focus();
h.run_steps(2);
press(&mut h, Key::Space);
assert_eq!(port(&h, "Keys"), Some((false, false)), "Space on the focused switch turns the device off");
assert_eq!(port(&h, "Drum Pad"), Some((true, true)), "the other device is untouched");
assert_eq!(h.state().ui.midi_inputs_off, ["Keys"]);
// The session's own list of devices follows the switch.
assert!(!h.state().engine.session().edit.flags.iter().any(|f| f.starts_with("midi.input.") && f.contains("Keys")));
// The preferences keep it: written out, read back, and used when the devices are opened again.
let saved = serde_json::to_string(&h.state().ui).unwrap();
let restored: UiState = serde_json::from_str(&saved).unwrap();
assert_eq!(restored.midi_inputs_off, ["Keys"]);
let reopened = MidiInputs::with_port_names(restored.midi_inputs_off.clone(), names(&["Keys", "Drum Pad"]));
assert_eq!(reopened.ports().iter().map(|p| (p.name.as_str(), p.enabled)).collect::<Vec<_>>(), [("Keys", false), ("Drum Pad", true)]);
// Preferences written before there were MIDI switches still load: every device is on.
let mut old: serde_json::Value = serde_json::from_str(&saved).unwrap();
old.as_object_mut().unwrap().remove("midi_inputs_off");
assert!(serde_json::from_value::<UiState>(old).unwrap().midi_inputs_off.is_empty());
// And on again, from the keyboard.
h.get_by_role_and_label(Role::CheckBox, "Keys").focus();
h.run_steps(2);
press(&mut h, Key::Space);
assert_eq!(port(&h, "Keys"), Some((true, true)));
assert!(h.state().ui.midi_inputs_off.is_empty());
}
#[test]
fn the_list_follows_devices_that_are_plugged_in_and_removed() {
let mut h = harness(&[]);
assert!(h.query_by_role_and_label(Role::CheckBox, "Launchkey").is_none());
let rescan = |h: &mut Harness<'_, SoundApp>, list: &[&str]| {
let m = h.state_mut().midi_inputs.as_mut().unwrap();
m.set_port_names(names(list));
let changed = m.rescan();
h.run_steps(2);
changed
};
assert!(rescan(&mut h, &["Keys", "Drum Pad", "Launchkey"]), "a new device is a change");
h.get_by_role_and_label(Role::CheckBox, "Launchkey");
assert!(!rescan(&mut h, &["Keys", "Drum Pad", "Launchkey"]), "nothing changed");
assert!(rescan(&mut h, &["Drum Pad", "Launchkey"]));
assert!(h.query_by_role_and_label(Role::CheckBox, "Keys").is_none(), "an unplugged device leaves the list");
assert!(rescan(&mut h, &[]));
h.get_by_label("No MIDI input device found. Plug one in: it shows up here by itself.");
}
#[test]
fn the_monitor_shows_what_arrives_from_a_device() {
let mut h = harness(&[]);
h.query_all_by_label("Test").next().unwrap().click();
h.run_steps(2);
h.get_by_label("Last received: nothing yet");
let feed = std::sync::Arc::clone(h.state().midi_inputs.as_ref().unwrap().feed());
feed.receive(feed.ports().assign("Keys"), &[0x92, 60, 100]);
h.run_steps(2);
h.get_by_label_contains("note C3 on, velocity 100, channel 3");
}
/// The app with a connected MIDI device, an instrument track and an audio track, none armed yet.
fn recording_harness() -> (Harness<'static, SoundApp>, u64, u64) {
let mut h = harness(&[]);
let new =
|h: &mut Harness<'_, SoundApp>, params: serde_json::Value| h.state_mut().run("track.new", params).unwrap()["tracks"][0].as_u64().unwrap();
let synth = new(&mut h, serde_json::json!({"name": "Synth", "kind": "instrument"}));
let vox = new(&mut h, serde_json::json!({"name": "Vox", "format": "mono"}));
h.state_mut().ui.show_playback_engine = false;
h.state_mut().ui.show_transport = true;
h.run_steps(3);
(h, synth, vox)
}
fn arm(h: &mut Harness<'_, SoundApp>, track: u64, on: bool) {
h.state_mut().run("mix.record_arm", serde_json::json!({"tracks": [track], "value": on})).unwrap();
h.run_steps(2);
}
#[test]
fn a_pass_with_only_midi_tracks_starts_without_an_audio_input() {
let (mut h, synth, _) = recording_harness();
arm(&mut h, synth, true);
assert!(h.state().recorder.is_none(), "arming an instrument track does not open the audio input");
h.get_by_label("transport.record").click();
h.run_steps(3);
assert!(h.state().engine.transport.recording, "{}", h.state().ui.status);
assert!(h.state().recorder.is_none());
}
#[test]
fn a_mixed_pass_whose_audio_cannot_be_recorded_is_refused_with_the_reason() {
let (mut h, synth, vox) = recording_harness();
arm(&mut h, synth, true);
arm(&mut h, vox, true);
// This app has no audio engine, so the audio track's take cannot be made. MIDI is ready,
// but recording only the instrument track would lose the audio without a word.
h.get_by_label("transport.record").click();
h.run_steps(3);
assert!(!h.state().engine.transport.recording);
let status = h.state().ui.status.clone();
assert!(status.starts_with("Cannot record: ") && status.contains("no audio input device"), "{status}");
// Without the audio track in record, the same pass starts.
arm(&mut h, vox, false);
h.get_by_label("transport.record").click();
h.run_steps(3);
assert!(h.state().engine.transport.recording, "{}", h.state().ui.status);
}
#[test]
fn an_inactive_record_enabled_audio_track_does_not_stop_a_midi_pass() {
let (mut h, synth, vox) = recording_harness();
arm(&mut h, synth, true);
arm(&mut h, vox, true);
// The audio track keeps its record button when it is made inactive, but records nothing:
// the pass is a MIDI one and needs no audio input.
h.state_mut().run("track.make_inactive", serde_json::json!({"tracks": [vox], "inactive": true})).unwrap();
h.run_steps(2);
let s = h.state().engine.session();
assert!(s.tracks.iter().any(|t| t.id.0 == vox && t.inactive && t.mixer.record_arm), "inactive and still record-enabled");
h.get_by_label("transport.record").click();
h.run_steps(3);
assert!(h.state().engine.transport.recording, "{}", h.state().ui.status);
assert!(h.state().recorder.is_none());
}
#[test]
fn a_midi_pass_does_not_start_while_the_previous_one_is_unconfirmed() {
let (mut h, synth, vox) = recording_harness();
arm(&mut h, synth, true);
// The audio engine never confirmed the end of the last pass (it stopped calling back). It
// would write the notes of a new pass down with the old pass's state.
let feed = std::sync::Arc::clone(h.state().midi_inputs.as_ref().unwrap().feed());
let pass = feed.end_pass();
h.get_by_label("transport.record").click();
h.run_steps(3);
assert!(!h.state().engine.transport.recording);
assert!(h.state().ui.status.contains("has not finished the previous record pass"), "{}", h.state().ui.status);
assert!(feed.pass_owed() && !feed.captured(pass), "still waiting");
// A pass without MIDI tracks does not wait for it (here it is refused for its own reason).
arm(&mut h, synth, false);
arm(&mut h, vox, true);
h.get_by_label("transport.record").click();
h.run_steps(3);
// It got past the MIDI guard to the audio setup, which refused it (this app has no audio
// engine), and the earlier pass is still owed.
let status = h.state().ui.status.clone();
assert!(!h.state().engine.transport.recording);
assert!(status.contains("no audio input device") && !status.contains("previous record pass"), "{status}");
assert!(feed.pass_owed() && !feed.captured(pass), "the attempt did not clear the pending pass");
// Once nothing is owed, the MIDI pass starts.
arm(&mut h, vox, false);
arm(&mut h, synth, true);
feed.forgive_passes();
h.get_by_label("transport.record").click();
h.run_steps(3);
assert!(h.state().engine.transport.recording, "{}", h.state().ui.status);
}
#[test]
fn a_midi_pass_without_a_connected_device_says_so() {
let (mut h, synth, _) = recording_harness();
arm(&mut h, synth, true);
h.state_mut().midi_inputs.as_mut().unwrap().set_port_names(Vec::new());
h.state_mut().midi_inputs.as_mut().unwrap().rescan();
h.get_by_label("transport.record").click();
h.run_steps(3);
assert!(!h.state().engine.transport.recording);
assert!(h.state().ui.status.contains("no MIDI input device is connected"), "{}", h.state().ui.status);
}
+1 -1
View File
@@ -43,7 +43,7 @@ Errors: `{"id": 1, "ok": false, "error": "bad parameters for `mix.volume`: no tr
| `engine.commands` | `{filter?}` | Every command: id, label, menu path, shortcut, params doc, enabled + reason. Includes UI-layer commands (`window.*`). |
| `engine.parity` | `{}` | Parity report against the incumbent's menu catalog. |
| `session.inspect` | `{detail?: "summary"|"full"}` | The document: tracks, clips (with times), inserts (+params when full), sends, routing, automation, markers, groups, busses, sources (`loaded` true when decoded audio is in the pool), selection, zoom (`samples_per_px`, `scroll`), transport, undo label. |
| `ui.inspect` | `{}` | UI state (`window`, panels, dialogs), window size, playing/position, audio device, and `edit_layout` (timeline rect and every track row's rect, for clicking). |
| `ui.inspect` | `{}` | UI state (`window`, panels, dialogs), window size, playing/position, audio device, MIDI inputs (`midi_inputs`: name, enabled, open; `midi_events`: messages received so far; `midi_events_dropped`: messages a full input queue dropped), and `edit_layout` (timeline rect and every track row's rect, for clicking). |
| `ui.set` | any `UiState` fields | e.g. `{"window": "Mix", "narrow_mix": true}`. |
| `ui.menu.list` | `{}` | Every catalog menu path and the command that implements it (or null). |
| `ui.menu.invoke` | `{path}` or `{id}` | Like clicking the menu item: may open its dialog (e.g. `"Track > New..."`). |
+19 -9
View File
@@ -1,6 +1,6 @@
# Where SoundCraft falls short of Pro Tools
> **Last reviewed:** 2026-10-25 · **Last updated:** 2026-11-13 · **Change:** minor (G-05: plugin editors on Linux/X11; previously trivial: G-09: the pt-BR catalog now covers helper-passed tooltips, the status bar, the remove-session prompt and engine error text; G-08: Frequency Shifter, 28 processors; G-04: volume automation read at every sample, #1354; G-01: `setup.playback_engine` applies the device buffer in the desktop app; G-04: Clip Looping crossfade and loop count) · **Target:** Avid Pro Tools Ultimate 2026.4.1
> **Last reviewed:** 2026-10-25 · **Last updated:** 2026-11-18 · **Change:** minor (G-02 shrunk: MIDI input devices work; previously minor: G-05: plugin editors on Linux/X11; previously trivial: G-09: the pt-BR catalog now covers helper-passed tooltips, the status bar, the remove-session prompt and engine error text; G-08: Frequency Shifter, 28 processors; G-04: volume automation read at every sample, #1354; G-01: `setup.playback_engine` applies the device buffer in the desktop app; G-04: Clip Looping crossfade and loop count) · **Target:** Avid Pro Tools Ultimate 2026.4.1
Every known shortfall, one entry each, ranked by how much it stops a Pro Tools user from doing
real work in SoundCraft. This is the work list: pick from the top. Rows marked **alpha blocker**
@@ -16,7 +16,7 @@ When you close a gap, delete its entry (or shrink it) in the same PR, and bump t
| # | Gap | Dimension | Doc | Est. (h) |
|---|---|---|---|---:|
| G-01 | **Alpha blocker.** No channel routing or duplex low-latency path (device, rate and buffer choice done; ASIO opt-in) | Hardware | [hardware-parity.md](hardware-parity.md) | 15–25 |
| G-02 | **Alpha blocker.** No MIDI device input or output | Hardware / MIDI | [hardware-parity.md](hardware-parity.md) | 20–35 |
| G-02 | **Alpha blocker.** MIDI devices: input works (checked on Linux only); no output, clock or loop/merge recording | Hardware / MIDI | [hardware-parity.md](hardware-parity.md) | 12–22 |
| G-03 | **Alpha blocker.** Cannot exchange sessions with Pro Tools (no AAF/OMF, no `.ptx`) | File formats | [file-format-parity.md](file-format-parity.md) | 50–90 |
| G-04 | Open correctness bugs in features counted as done (~25 issues) | Stability | this file | 25–40 |
| G-05 | (Entitlement part: alpha blocker for the mix row) Third-party VST3s blocked in the signed macOS build; AU editors missing | Ecosystem | [plugin-parity.md](plugin-parity.md) | 10–15 |
@@ -65,13 +65,22 @@ When you close a gap, delete its entry (or shrink it) in the same PR, and bump t
### G-02. MIDI devices
- **Missing:** all MIDI hardware I/O. No `midir`/CoreMIDI/WinMM/ALSA sequencer code exists;
`setup.midi_input_devices` and `setup.midi_studio` only store names. You cannot record from a
keyboard, play a hosted instrument live, drive external synths, or send MIDI clock/MTC.
`midi_editor.rs:288` notes audition "needs a live MIDI input path".
- **Impact:** MIDI production is limited to drawing notes and importing SMF.
- **Estimate:** ≈ 20–35 h (device layer, realtime input to armed tracks and instruments with
timestamps, MIDI record/merge/loop, thru, output tracks, clock out).
- **Done (2026-10-12):** MIDI input devices (`crates/playback/src/midi.rs`, `midir`): keyboards
play record-enabled and input-monitored MIDI and instrument tracks live and are recorded into
MIDI clips, with a device and channel choice per track. Controllers, pitch bend, aftertouch
and program changes reach the instrument live. Notes are kept per device and channel, and a
full input queue releases the notes of its device and marks the take incomplete. The device
switches (Setup › Playback Engine, also opened by Setup › MIDI › MIDI Input Devices) are app
preferences. User guide: [midi-input.md](midi-input.md). On Linux, live play was checked with
a hardware keyboard (Akai MPK mini 3) and recording with a virtual one;
**not yet on macOS or Windows** (type-checked only).
- **Missing:** MIDI output (external synths, MIDI tracks without an instrument), MIDI clock/MTC
out, loop and merge recording (a MIDI take replaces what it covers; a loop pass piles every
pass into one clip), audition from the MIDI editor (`midi_editor.rs:288`), MIDI
input on FreeBSD and the web. `setup.midi_input_devices` and `setup.midi_studio` still only
store names in the session; the real device switches are app preferences.
- **Impact:** you can play and record a keyboard, not drive hardware synths or sync other gear.
- **Estimate:** ≈ 12–22 h (output tracks and thru, clock out, merge/loop takes, controller path).
### G-03. Session interchange with Pro Tools
@@ -252,5 +261,6 @@ missing. Out of scope unless the owner decides otherwise.
| 2026-10-16 | trivial | G-04: Split into Mono channel preservation fixed and regression-tested; estimates unchanged |
| 2026-10-16 | minor | G-16: capture current hosted-plugin state before saving a track preset (#661); broader preset work remains (#639) |
| 2026-10-12 | minor | G-01 shrunk: audio device, rate and buffer choice; opt-in ASIO |
| 2026-10-12 | minor | G-02 shrunk: MIDI input devices, live play-through and MIDI recording landed; output, clock and loop/merge recording remain |
| 2026-10-10 | minor | Marked the alpha blockers (G-01, G-02, G-03; parts of G-05 and G-06) from the core-workflow gate |
| 2026-10-10 | major | Created from the full re-measure against Pro Tools Ultimate 2026.4.1: 22 ranked gaps with evidence and estimates |
+4 -3
View File
@@ -1,6 +1,6 @@
# Hardware parity
> **Last reviewed:** 2026-10-10 · **Last updated:** 2026-10-13 · **Change:** trivial (bounces of a bus fed by an aux compensate its latency (#246); previously minor: audio device, rate and buffer choice; ASIO feature) · **Target:** Avid Pro Tools Ultimate 2026.4.1
> **Last reviewed:** 2026-10-10 · **Last updated:** 2026-11-18 · **Change:** minor (MIDI input devices, live play-through with controllers, and MIDI recording) · **Target:** Avid Pro Tools Ultimate 2026.4.1
A DAW is judged by how it drives audio interfaces, MIDI gear, control surfaces and sync. This is
SoundCraft's weakest dimension. Ours is measured from `crates/playback` (cpal) and the absence of
@@ -31,9 +31,9 @@ Legend: ✅ works · 🟡 partial · ❌ missing.
| Feature | Pro Tools | SoundCraft (all platforms) |
|---|---|---|
| MIDI input devices (keyboards, pads) | ✅ | ❌ no MIDI device layer at all (`setup.midi_input_devices` stores names) |
| MIDI input devices (keyboards, pads) | ✅ | 🟡 Linux (ALSA sequencer), macOS (CoreMIDI) and Windows (WinMM) through `midir`: every input port is opened, hot-plugged devices are picked up, ports can be switched off in Setup › Playback Engine; each MIDI/instrument track chooses a device and channel at its input selector. Notes are recorded into MIDI clips with controllers, pitch bend and aftertouch. On Linux, live play was checked with a hardware keyboard (Akai MPK mini 3) and recording with a virtual one; not on macOS or Windows, and there is none on FreeBSD or the web |
| MIDI output to external gear | ✅ | ❌ |
| MIDI Thru, input filter | ✅ | ❌ (stored flags only) |
| MIDI Thru, input filter | ✅ | 🟡 record-enabled and input-monitored MIDI/instrument tracks play the keyboard through their instrument (notes, the sustain pedal, controllers, pitch bend, aftertouch and program changes); the input filter keeps the filtered kinds out of recordings. No thru to external gear |
| MIDI Beat Clock out, MTC generate/chase | ✅ | ❌ |
| MIDI learn / controller mapping | ✅ (via surfaces and plug-in maps) | ❌ |
| Virtual MIDI keyboard | ✅ (MIDI Keyboard window) | 🟡 window exists, no live audition path |
@@ -72,6 +72,7 @@ Legend: ✅ works · 🟡 partial · ❌ missing.
| Date | Change | Summary |
|---|---|---|
| 2026-11-18 | minor | MIDI input devices (`midir`): live play-through to record-enabled MIDI and instrument tracks (notes, pedal, controllers, pitch bend, aftertouch), MIDI recording into clips, per-track device and channel, device switches in Playback Engine; guide in `midi-input.md` |
| 2026-10-13 | trivial | bounces of a bus fed by an aux compensate its latency (#246) |
| 2026-10-12 | minor | Audio device settings: system, output, input, sample rate and buffer size; opt-in ASIO |
| 2026-10-10 | major | Created: audit of audio interface, MIDI, control-surface, sync and performance support against Pro Tools 2026.4.1 |
+100
View File
@@ -0,0 +1,100 @@
# MIDI input: playing and recording a keyboard
SoundCraft plays MIDI keyboards and pads through its instruments and records what is played into
MIDI clips.
**Where it works.** Linux (ALSA sequencer), macOS (CoreMIDI) and Windows (WinMM). It has been
played and recorded on Linux only; on macOS and Windows the code builds but has not been tried
with a device yet. There is no MIDI input on FreeBSD or in the web version. MIDI output, MIDI
clock and loop or merge recording are not there yet (see [gaps.md](gaps.md), G-02).
## 1. Switch a device on
Plug the device in. SoundCraft opens every MIDI input it finds, and looks for new ones every two
seconds, so there is usually nothing to do.
To see the devices, or to switch one off, open **Setup › Playback Engine…** and go to the
**MIDI inputs** section at the bottom. **Setup › MIDI › MIDI Input Devices…** opens the same
window.
- Each device has a checkbox with its name. Untick it and SoundCraft stops listening to that device.
- Next to it is the state: *connected*, *off*, or *could not be opened*.
- **Test** shows the last message that arrived from the device, to check that a keyboard gets
through. Press a key and it reads, for example, *note C3 on, velocity 100, channel 1*.
On Windows many MIDI drivers let only one program use a device at a time. While SoundCraft has
a device open, another program may not be able to use it: untick the device here to give it up.
The switches are app preferences: they are kept on this computer, for every session, and a
device that was switched off stays off the next time it is plugged in. They are not part of the
session file.
## 2. Choose what a track listens to
A MIDI or instrument track listens to **all devices on all channels** until told otherwise; its
input selector reads *All MIDI*.
Click the track's input selector (in the Mix window, or the I/O column of the Edit window) to
choose one device, one channel (1–16), or both. A device the track listens to stays in the list
while it is unplugged, marked *not connected*.
Scripts and agents set the same thing with the `track.input` command. The input is written
`MIDI:<channel>:<device>`, with `*` for all:
| Input | The track listens to |
|---|---|
| `none` or `MIDI:*:*` | every device, every channel |
| `MIDI:10:*` | channel 10 of every device |
| `MIDI:*:Launchkey MK3 MIDI 1` | every channel of that device |
| `MIDI:1:Launchkey MK3 MIDI 1` | channel 1 of that device |
```json
{"command": "track.input", "params": {"tracks": [3], "input": "MIDI:10:Drum Pad"}}
```
The device name is the one shown in Setup › Playback Engine. A channel outside 1–16 means every
channel.
## 3. Play
Record-enable the track (its **R** button) or switch its input monitor on. The keyboard now plays
the track's instrument, with the transport stopped or running.
- Notes, the sustain pedal, the other controllers, pitch bend, aftertouch and program changes
reach the instrument.
- Several devices or channels can play one track. A note held on two of them sounds until both
let go, and each has its own sustain pedal.
- Taking the track out of record, or switching its monitor off, ends the notes that are held.
## 4. Record
Record-enable one or more MIDI or instrument tracks and record as usual (**Record**, then
**Play**). No audio input is needed for a pass that records MIDI only.
When the pass stops, each track that was record-enabled and active when it started gets a MIDI
clip with what its input received: notes with their velocities, controllers, pitch bend, aftertouch and
program changes. What a pass records is settled when it starts: a track put in record while it
runs is recorded from the next pass, and one taken out of record keeps what it recorded. A track
that received nothing gets no clip. The clip covers the pass, the punch range, or the loop.
One **Undo** removes the take.
**Setup › MIDI › MIDI Input Filter…** keeps kinds of messages (for example aftertouch) out of
recordings.
## If something is lost
The path from a device to the engine has a fixed size. If a device sends more than it holds
(thousands of messages in a few milliseconds), SoundCraft drops the excess and then ends every
note that device is holding, so nothing is left sounding. Press the keys again to go on. The
same happens when a device is unplugged with keys down.
If that happens during a record pass, the status line says the take is incomplete, and the notes
that were held end at that point in the clip. The status line also says so when the audio engine
did not answer at the end of the pass (an audio device that stopped): the last notes played may
then be missing, and a new pass that records MIDI does not start until the audio engine answers. The MIDI inputs section of Setup › Playback Engine
shows how many messages were dropped.
## For agents
`ui.inspect` reports `midi_inputs` (name, enabled, open), `midi_events` (messages received so far)
and `midi_events_dropped`. See [control-protocol.md](control-protocol.md).
+4 -3
View File
@@ -1,6 +1,6 @@
# SoundCraft roadmap: milestones and what's next
> **Last reviewed:** 2026-11-05 · **Last updated:** 2026-11-09 · **Change:** trivial (28 built-in processors with the Frequency Shifter; previously trivial: G-05 estimate after Linux/X11 plugin editors; previously trivial: M15: partial pt-BR catalog in PR #1346) · **Target:** Avid Pro Tools Ultimate 2026.4.1
> **Last reviewed:** 2026-11-05 · **Last updated:** 2026-11-18 · **Change:** minor (MIDI input landed, the gate row needs checking on macOS and Windows; previously trivial: 28 built-in processors with the Frequency Shifter; previously trivial: G-05 estimate after Linux/X11 plugin editors; previously trivial: M15: partial pt-BR catalog in PR #1346) · **Target:** Avid Pro Tools Ultimate 2026.4.1
Forward-looking plan. Status and numbers: [`../ROADMAP.md`](../ROADMAP.md) and
[`target-app-parity.md`](target-app-parity.md); the ranked work list: [`gaps.md`](gaps.md).
@@ -11,7 +11,7 @@ Hours are Opus 5.5 agent-hours.
1. **Merge or close the open bug-fix PRs** (#123–#134, #146, #148, #75) and fix the rest of the
correctness backlog (G-04). ≈ 15–25 h.
2. **Audio interface basics** (G-01): device selection, buffer size, duplex stream, ASIO. ≈ 25–40 h.
3. **MIDI devices** (G-02): input to armed tracks and instruments, output, thru, clock. ≈ 20–35 h.
3. **MIDI devices** (G-02): input to armed tracks and instruments is in (checked on Linux); macOS and Windows checks, output, thru to gear, clock remain. ≈ 12–22 h.
4. **macOS plugin hosting** (G-05): library-validation entitlement now (#153), then AU editors and embedded CLAP editors on macOS. ≈ 10–15 h.
5. **A PR CI that runs `cargo xtask ci`** on Linux, macOS and Windows (G-14). ≈ 5–10 h.
@@ -23,7 +23,7 @@ main platform). Any **no** keeps SoundCraft **pre-alpha**, and those rows are th
| Workflow | Works end to end? | Evidence | Hours to pass |
|---|---|---|---:|
| Record audio from a chosen interface at a chosen buffer size, monitor, punch, keep the takes | **no** | Device, rate and buffer are chosen in the Playback Engine window and, in the desktop app, by `setup.playback_engine` (headless returns `applied: false`). The mix still runs in fixed 512-sample blocks. Separate input and output streams. Takes held in RAM until stop (PR #84). | 35–55 |
| Record MIDI from a controller and play it back through an instrument | **no** | Playback of drawn and imported MIDI works. No MIDI device layer at all, so nothing can be recorded and instruments can't be played live (G-02). | 20–35 |
| Record MIDI from a controller and play it back through an instrument | **unverified** | Built: keyboards play record-enabled instrument tracks live and are recorded into MIDI clips. On Linux, live play was checked with a hardware keyboard (Akai MPK mini 3) and recording with a virtual one; the row passes once it is checked on macOS and Windows too (G-02). | 2–4 |
| Edit: select, trim, fade, comp playlists, Slip/Shuffle/Grid | yes (rough) | Every Edit menu command; property-tested with undo; open bugs #3, #5, #140 don't block the workflow | — |
| Mix with built-in and third-party plugins, sends, busses, automation | partial (passes on CLAP/AU) | 28 built-in processors; CLAP and AU host on the signed macOS build, but its missing entitlement blocks third-party VST3 (#153). No AU plugin windows yet; VST3 windows on macOS, Windows and Linux/X11 | 1–2 (entitlement) |
| Bounce mix and stems; save and reopen the session | yes (rough) | WAV/BWF/AIFF/FLAC bounce, stems, `.scraft` save/reopen with plugin state; bugs #59, #103 in bus bounce and aux latency (PRs open) | — |
@@ -88,6 +88,7 @@ There is no `cargo xtask scorecard` in this repo; the measured number is `cargo
| Date | Change | Summary |
|---|---|---|
| 2026-11-18 | minor | MIDI input devices landed (G-02 shrunk); the MIDI gate row is unverified until checked on macOS and Windows |
| 2026-11-09 | trivial | 28 built-in processors with the Frequency Shifter |
| 2026-11-05 | trivial | M15: a pt-BR catalog covers menus and `text`/`localized_format!` strings; remains native review, CJK/RTL and the other languages. Estimates unchanged |
| 2026-10-30 | trivial | G-05 down to 10–15 h and the mix row's VST3 windows updated: VST3 and embedded CLAP editors are hosted on Linux/X11 (#527) |