Files
Alexandru CarausandClaude Opus 5.5 a02d892f83 MIDI input devices: live play-through and MIDI recording (#493)
* MIDI input devices: live play-through and MIDI recording

Keyboards and pads could not be used at all: there was no MIDI device
layer (G-02).

- crates/playback/src/midi.rs opens every MIDI input port through midir
  (ALSA sequencer, CoreMIDI, WinMM), follows devices that are plugged in or
  removed, and passes channel messages to the audio thread over a lock-free
  ring, each with the time it arrived.
- The mix engine plays them on the instruments of record-enabled and
  input-monitored MIDI and instrument tracks (crates/mix/src/live_midi.rs):
  notes at their sample offset, the sustain pedal holding note-offs back.
- While recording, the events are kept with their timeline position and
  become MIDI clips on the record-enabled tracks
  (crates/engine/src/io/midi_take.rs), with controllers, pitch bend and
  aftertouch. A pass can be MIDI only: the audio input is opened for audio
  tracks alone.
- A MIDI or instrument track chooses a device and a channel at its input
  selector (kept in the track's input route as `MIDI:<channel>:<device>`).
- Setup > Playback Engine lists the input ports with a switch each and, on
  request, what arrives from one; Setup > MIDI > MIDI Input Devices opens
  it. The switches are app preferences.

Note lifecycle:

- A strip keeps the keys and the sustain pedal of every owner (input port
  and channel) apart. A pitch held by two devices or channels sounds until
  the last one releases it, and one owner's pedal does not hold another's
  notes. A strip follows 16 owners at a time; further ones share the last
  place (documented in live_midi.rs).
- A full queue is never silent. An event that does not reach the audio
  thread (the 1,024-entry input ring or the 256 events a callback takes)
  marks its port; the audio thread then drops what it still has queued from
  that port and the engine releases the notes and pedal the port holds. A
  port that closes, a replaced feed and events skipped while nothing
  listens take the same path.
- A take records where input was dropped: the notes that port held end
  there, and the status line says the take is incomplete. Recorded events
  the UI thread did not collect in time are counted and reported too.
  Recorded notes are told apart by device as well as channel and pitch.

On Linux, live play was checked with a hardware keyboard (Akai MPK mini 3)
and recording with a virtual one (crates/playback/examples/midi_keys.rs),
before the note-lifecycle work; that part is covered by tests only. macOS
and Windows are type-checked only; there is no MIDI input on FreeBSD or
the web.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>

* midi real device tests

* midi device drop test

* midi device fix pr review comments

* MIDI input: review fixes for note ownership, lost input and takes

Live notes (crates/mix/src/live_midi.rs):

- An owner without a place of its own joins the shared place while it is
  in use, even when another place has come free. It could get a second
  place before, and the release of its key in the shared place was then
  not found: the note stayed on. The same held for the pedal.
- A change of the track's input releases the shared place when the input
  no longer takes any one of the owners in it, not only the first.
- The strip remembers which port and channel last set each controller.
  When that source loses its input or the track stops listening to it,
  the instrument's controllers are set again from the clips and from the
  sources that are left, so a bend from an unplugged device does not stay.
- When playback runs out under monitoring, the clips' sustain pedal goes
  up with their notes.

Capture (crates/playback):

- A loss carries its time. What a port sends after it is played and comes
  after the mark in the take; before, it was dropped and recorded ahead
  of the mark.
- The age of an event is counted at the transport speed.
- Loop recording writes a mark where the loop goes round, and a key played
  with the playhead at the end of the loop stays inside the loop.
- The end of a record pass is acknowledged by its number, so the late
  answer to an earlier pass does not end the wait for a later one.
- A connection the backend closes marks its port, also when a device with
  the same name is opened in its place.

Takes (crates/engine/src/io/midi_take.rs, crates/ui-egui):

- A note held into the next round of a loop is split there, also when its
  release is later on the timeline than its press.
- A track taken out of record during a pass keeps what it recorded.
- A pass whose end the audio engine did not confirm is reported as
  incomplete in the status line.
- An inactive record-enabled audio track no longer turns a MIDI pass into
  an audio one that is then refused.

docs/midi-input.md: Windows drivers that allow one program per device, and
the new status text.

* MIDI input: review follow-ups on loss order, controllers and loop takes

- Each port has its own time of loss. One port's later loss no longer
  takes along what another port sent after its own: that is played, and
  comes after that port's mark in the take.
- A retired connection is closed before its port is marked as lost. Marked
  first, a note-on delivered while it closed carried a later time than the
  loss and could be played after the release.
- The strip keeps the last controller value of every source, not only the
  newest. When one of two sources that set the same controller goes away,
  the controller goes back to the value of the one that stays. 64 values
  are kept; past that the oldest is forgotten.
- A take can say where the pass stopped (MidiTakeEvent::END), and the UI
  says so for loop passes. A key still down then ends at the stop in the
  round it is in: held over a round or more it covers the loop, where it
  used to end at the loop end of its first round.
- A pass that records MIDI does not start while the audio engine has not
  confirmed the end of the previous one. It takes the MIDI since its last
  callback with the state of the pass it knows, so a pass started and
  stopped meanwhile had its notes written at the old position.

* MIDI input: loss marks in the queue, connection gate, stop position

- A loss from a full queue is also written into the queue, ahead of the
  first event that fits again. The audio thread deals with it at that
  place. Before, a loss flagged after the callback had read the flags was
  seen one callback late and ended a note played after it. The flag for a
  loss already met in the queue is not a second loss.
- A connection is shut off under the feed's lock when it is closed
  (MidiFeed::close_port), and what its device thread still delivers is
  discarded. Dropping a connection joins that thread on ALSA; CoreMIDI
  (MIDIPortDispose) and WinMM (midiInStop, midiInReset, midiInClose) do not
  promise to, so the order of drop and mark is no longer relied on.
- When the controller table is full and forgets a value, the strip keeps
  the port and channel it came from. The controllers are still reset when
  that source goes away, where a bend pushed out of the table used to stay.
- A loop pass ends where the audio engine is. The larger of the engine's
  and the UI's position was used, and just after the loop went round the
  UI's is still near the loop end: a short held note covered the loop.
- The end of a pass waits for the audio engine by what the pass set out to
  record, not by the devices connected at Stop. A pass whose last device
  was unplugged before Stop left nothing pending, and a new pass could
  start before the engine was done with it.

* MIDI input: stop at the loop end, and passes owed by the audio engine

- A key played while the playhead stands at the end of the loop is the
  last thing of that round. It was written as the first of the next round,
  with a mark that the loop had gone round; a pass stopped right there
  never went round, and a note held to the stop grew a second part.
- The feed knows whether an ended pass is still owed by the audio engine
  (MidiFeed::pass_owed), and the UI asks it before a pass that records
  MIDI starts. The UI kept that itself, where no test could stall the
  audio callback against it.

* MIDI input: stop boundary, loop rounds, loss marks and shared owners

- A pass ends at a time on the feed's clock (MidiFeed::end_pass). Only
  what arrived up to it is written to the take, however late the audio
  thread hears of the Stop, and the callback takes the queue in once more
  after the command before it confirms: a note that arrived between its
  reading the queue and the Stop was missing from a confirmed take.
- The confirmation carries where the playhead stood (MidiFeed::stopped_at).
  A loop pass ends there; the player's position read afterwards could be
  a round further on, which the take knows nothing about. Without a
  confirmation no stop position is used.
- The take builder is told whether the pass looped (MidiPass). In a pass
  that did not, a release stamped before its key press (a late callback)
  is one short note, where it was split around the ends of the clip. The
  stop position is a parameter, not an entry that the event cap could cut.
- A pass records the tracks that were in record when it started, and only
  those: the same list the end of the pass waits by. A track put in record
  during the pass got what was played before that, without a wait.
- A short loop goes round several times in a callback: each entry is put
  in the round it arrived in, so two entries a round apart have one mark
  between them.
- A mark in the queue is stamped with the time of its loss. A port that
  overflowed, played on and was then closed keeps a mark for the close
  (the new note was held to the end of the take), and a loss told by the
  port's flag is not written again when its mark arrives.
- Live events stay in order of arrival when they are played: a release
  taken by a shorter callback was due before its note-on.
- A callback reads the transport speed once, for rendering and for the
  ages of the next callback's events.
- The shared owner place is left alone by a source that never joined it:
  its note-off no longer ends another owner's note there, and its loss no
  longer releases the place.
- When a live source goes away, the instrument's controllers are restored
  before the block's clip events are sent, so a clip controller at the
  start of that block keeps its new value.
- Tests that assert MIDI positions use a clock the test sets.

* MIDI input: pass boundaries, loop punch-in and review follow-ups

- A pass begun while the transport runs has a start time on the feed
  (MidiFeed::begin_pass): what is played after Record is written down
  also when the audio thread takes it in before it reads the command.
- A pass goes on through a transport stop until the command that ends it.
  The UI sends the two one after the other; when they reached two
  callbacks, a note played just before the stop was left out of a
  confirmed take.
- The end of a pass is asked for every entry, so a Stop that comes while
  the queue is being read still cuts off what follows it.
- A mark in the queue carries the time its events were dropped, kept apart
  from a later close of the port. An overflow before Stop is in the take
  when the port closes after it.
- A pass punched in while a loop plays is a loop take over that loop. In a
  line from the punch-in to a stop in the next round nothing fell inside
  the range.
- In a loop take with marks, only the marks say that the loop went round:
  a release stamped before its press within one round is one short note.
- A key pressed in the last samples of the range starts at the last tick
  the clip plays, not at the tick of its end.
- When the clips' controllers are sent again while stopped, their sustain
  pedal goes up again: no pedal-up of theirs would follow.
- The render step is taken after the commands, so a session at another
  sample rate is rendered at that rate from the callback that installs it.
- The stop position and the incomplete-take warning go by one answer from
  the audio engine.

* MIDI input: test that a callback renders at the speed it keeps

A speed change that arrives while a callback runs takes effect from the
next one: the callback renders at the speed it read, and the next callback
counts the ages of its events at that same speed.

* MIDI input: volume and pan go back to MIDI's defaults with their device

The reset sent when a live source goes away is Reset All Controllers,
which leaves the channel volume (CC 7) and the pan (CC 10) alone. A
device that turned the volume down to nothing and was unplugged left the
track silent. When the source that went away had set one of the two, and
neither the clips nor a source that is still there gives it a value, the
instrument is sent the value MIDI names as the default (100, and 64).
The same when the track stops playing live input.

* MIDI input: third round of review follow-ups

- The stop cutoff is one value on the feed, in force from the store that
  ends the pass and gone when the audio thread confirms it.
- The start of a pass holds for the whole pass: what was played before
  Record stays out also when it is taken in after the start command.
- What is played just before a Stop is put inside the range that ends at
  the stop, not on its end.
- A port closed while the queue is read is met through the mark of its
  overflow; the flag of that close, a callback later, is not a second
  loss (it ended a key the new connection played).
- An overflow and the loss it is have one time. Read apart, they could
  differ by a microsecond and count as two losses.
- A volume or pan the full controller table forgot goes back to its
  default when its own device goes, not when another does.
- The chase of the clips' controllers uses the track's MIDI delay, as
  playing them does.
- Recorded controllers in the last samples of the range stay where the
  clip plays; in a take without marks the press as played, not the note's
  start, says whether the loop went round; a take cut at the event cap
  does not use the stop position.
- The loop the MIDI of a punched-in pass goes to is the loop the audio
  engine plays: not a range too short to loop, nor one that Play From
  dropped.
- The audio input stops taking in before the wait for the MIDI.

* MIDI input: fourth round of review follow-ups

- A record pass goes on through the natural end of playback until the UI
  ends it, up to the time playback ran out: what was played as it reached
  the end is in the take, what follows is not.
- Before the end of a pass is confirmed, the device threads are kept out
  of the queue while it is read once more, so a message stamped before
  the Stop is not put in after the answer. The audio thread does not wait
  for a device thread that is in there.
- A later loss of a port is noted as dealt with only at the end of the
  callback: with two overflow marks of one port in the queue, the second
  was skipped and the key played between them had no end.
- Whether a pass was begun is read before when: a pass seen as begun has
  its own start time.
- The volume and the pan go back to their defaults when the source that
  set them last goes, also when the controller table forgot its value,
  and not when an earlier setter goes.
- A clip's note-off that is held back because a live key holds the pitch
  is remembered for the block: the key going up earlier in the block no
  longer ends the note before the clip would have.
- In a take without marks, whether the loop went round between a press
  and its release is judged by their samples, not their ticks.
- The stop position of a loop pass is not used when recorded entries did
  not reach the take: one may be the mark of a round.
- Loop recording over a selection too short to loop is recorded as a
  punch, as the transport plays it.

* MIDI input: fifth round of review follow-ups

- Whether a pass was begun is asked for every event: a pass begun while
  the callback is reading the queue records from when it began.
- The ages of a pass's last events are counted up to when the transport
  stopped. Counted up to the callback that takes them in, the time the
  playhead stood still moved them back.
- An overflow the port's flag told of keeps its time when an older mark
  in the queue shows a later close: its own mark may never come.
- What becomes of the clips' held back note-offs is settled after the
  block's live events. A key that goes down again before the clip's note
  would have ended is not ended by it, and with two clip notes of a pitch
  in a block the one that sounds when the key goes up lasts to its end.
- When the source that set the volume or the pan last goes and another
  one's value is sent again, that one is the last setter from then on.

* MIDI input: sixth round of review follow-ups

- A loss is followed whenever the transport runs, and written to the take
  by when it happened, like an event: a port lost in a pass that began
  while the queue was being read ends its notes in the take.
- A pass started and ended by the commands of one callback is still that
  pass in the last read of the queue: what was played in it is kept.
- When a live key takes a pitch again, the clips' held back note-offs
  from since the last key went up are sent where they were, so the
  instrument lets go of the clip's note before the new press.

---------

Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-11-19 08:37:06 +07:00
..
2026-11-18 17:17:26 +07:00