Turn audio into musical events

Let attacks in one audio signal trigger another instrument, connecting a recorded or live rhythm to new sounds.

Composed and inline triggers#

playOn accepts patternTrig, mergeTrig, filterTrig, quantizeTrig and onCount graphs, including named shared triggers. These emit one-sample pulses: coincident hits coalesce, adjacent-sample hits remain distinct. Each event fires at its reported sample offset; see Bring a sound to life.

onsets audioExpr [sensitivity] analyses an ordinary audio graph, so a filtered tapLive or input can feed the detector directly. It combines stereo energy in fixed 128-sample windows, reports on the window's final sample, and keeps the existing 50 ms refractory law. The source endpoint's routing delay is additional. Track/string onsets retains its legacy block detector contract. Sensitivity must be finite and within [0, 1], for both node forms and the legacy playOn detector. Source compilation and module loading share this check.

What it is#

The event arrow, reversed. Patterns normally make audio; onsets lets audio make events: it watches a track's bus (or the live mic) with a causal onset detector and emits a live Trigger stream — one trigger per transient. Two consumers ship:

• as a modulator — a trigger stream composes with the trigger-shaped control vocabulary (hold (onsets drums) m latches m on every hit, exactly like hold noteTrig m latches per note);
• as a track's rhythm — playOn synth trig pat fires the pattern's next event per trigger: the drums play the bassline.

It is the event-direction sibling of the Route audio between tracks family (tap/follower/vocode): the same previous-block bus read, one generalization further (signal → trigger stream).

Family note. The other reading of a source bus is pitch, not rhythm: Follow the pitch of live audio (pitchOf track or pitchOf "input") runs a per-block McLeod autocorrelation over the same previous-block bus and exposes the detected fundamental as a Sig Control Hz — the control-side sibling of these triggers (saw (pitchOf lead) shadows a melody the way playOn shadows a rhythm).

Surface#

drums = sound !"bd ~ ~ bd  sn ~ bd ~"

hits = onsets drums          -- a Trigger stream (default sensitivity 0.5)
hot  = onsets drums 0.8      -- fires on softer hits
live = onsets "input"          -- the live mic (marks the module non-deterministic)

-- each trigger fires the NEXT pattern event in order, cycling:
bassT = playOn bass hits !"e1 e1 g1 a1"

-- trigger streams are ordinary control inputs:
shimmer = macro 0.5
pad = patch {
  source = saw (freq pitch),
  ampEnv = adsr 200ms 400ms 0.8 600ms,
  cutoff = const 400Hz + hold (onsets drums) shimmer * 3000Hz,
  gain   = vel * 0.4
}

• onsets source [sensitivity] — source is the track's binding (onsets drums, or qualified through an import, onsets kit.drums), an aux-bus name as a string, or "input"; a track's name as a string (onsets "drums") is accepted legacy (Language reference §12). sensitivity is a compile-time constant in [0, 1] (default 0.5; 0 = only massive transients, 1 = hair trigger). Returns a Trigger (the noteTrig shape): the output is 1.0 across the render block containing an onset, else 0.0, so hold's rising-edge latch composes.
• playOn synth trig pitchPat — a track constructor, sibling of play. trig must be an onsets … stream (written inline or through a top-level binding). The pattern's events fire in order, cycling; their own onsets are ignored (the trigger supplies the rhythm) but each event's duration still sets the fired note's length, and velocity composes through the standard chain (velocity / event marks / gain pattern / gainTrack). Rests are skipped — a trigger always sounds — and an all-rest pattern is a compile error. Everything else is a normal track: |> effect chains, mixer strips, panTrack, scenes (quantized scene start/stop gates the firing; the detector keeps running so its adaptive state is warm).

Detection (as built)#

engine/src/voice/detect.rs::OnsetDetector — causal, block-rate (128 frames), fed by the source's previous-block post-fader RMS (track_rms, the follower's exact read; input.level for the mic). Per block:

1. flux = max(0, level − prev_level) (half-wave-rectified energy flux);
2. adaptive gate — flux must exceed a one-pole running mean of the flux (~0.5 s tc) by ratio (7.5 → 1.5 as sensitivity 0 → 1) and an absolute floor (0.0105 → 0.0005);
3. peak gate — the level must clear a decaying peak-hold of the recent level (~120 ms tc) by margin (1.9 → 1.1). This is what rejects beating/wobble in overlapping sampled tails (kick + snare ringing together) without waiting out a long refractory;
4. refractory — 50 ms; one hit fires once.

Timing is block-quantized: detection reads the previous block's RMS and fires at the next block's start, so a trigger lands within ~2 blocks (≈ 5.3 ms at 48 kHz) of the audio transient — the Route audio between tracks latency doctrine, inaudible as rhythm and feedback-safe by construction.

How it's implemented#

• Compiler: BuiltinKind::Onsets (onsets : Track → Trigger, optional sensitivity validated at lowering — compiler/src/lower/track.rs::onsets_args, shared by both surfaces) lowers to NodeKind::Onsets { source_track, sensitivity }. BuiltinKind::PlayOn lowers a normal Track plus Track.on_trigger: { source, sensitivity } (ir/src/ir.rs::OnsetTrigger). Trigger-typed top-level bindings (hits = onsets drums) ride the free-floating-modulator map so playOn and patch bodies resolve them.
• Diagnostics: refine.rs::check_causal treats onsets (node) and playOn (track) reads as causal edges like follower/tap — ok-level discharge naming the 1-block latency. Shared execution validation rejects missing track/bus sources before installation. onsets "input" in either position sets Module.nondeterministic (the input rule); an internal track source does not — detection on the engine's fixed 128-frame blocks is reproducible run-to-run at a given sample rate, though the trigger times are block-grid, not sample-exact, quantities.
• Engine: CompiledNode::Onsets (per-voice detector state, source resolved via resolve_bus — tracks, aux buses, "input", missing → silence) emits the trigger stream in the voice graph. TrackPlan.on_trigger (OnsetTriggerPlan, resolved at load) routes a running track to schedule_on_trigger (engine/src/lib.rs) instead of the clock scheduler: one detector per running track (kept across hot-swaps), on fire the next non-rest event triggers through the same event_dynamics/track_pan_at composition as scheduled notes, onto the track's own bus. onsets "input" counts toward CompiledModule::uses_live_input, so the desktop host lazily opens the mic.
• Browser: the same engine compiled to wasm (engine-wasm/) hosts the detector in an AudioWorklet, so onsets drums and onsets "input" both work. Live input arrives through WasmEngine::push_input, which the worklet calls with inputs[0] before process — the browser peer of the desktop's cpal capture thread (see Live audio input).

The trigger algebra (count / toggle / onCount / choose)#

Runtime Control Flow Stage 2 — Ty::Trigger's consumers, all shaped like hold (per-sample rising-edge state over any trigger-shaped control signal: noteTrig, onsets, a gate output, an onCount result):

kickT  = play kick !"c2 ~ c2 ~"
snareT = play tom !"~ c3 ~ c3"
-- a live bar-position-by-kicks: 0,1,2,3,0,…
beat = count (onsets kickT) 4
-- step a modulator through a table, one kick at a time:
cutoffMod = choose [0.0, 0.3, 0.7, 1.0] (count (onsets kickT) 4)
-- flip a state per snare:
flip = toggle (onsets snareT)
-- every 4th kick, as a trigger again (a clock divider):
accents = onCount 4 (onsets kickT)

• count trig [n] : Sig Control — rising-edge counter. With n, counts modulo n; without, saturates at 2^24 (f32's exact-integer ceiling, documented rather than wrapped). n is a fixed dimensionless integer from 1 through 16,777,216, finite and exact in engine precision. The result is dimensionless. Gate inputs follow the same contract as toggle.
• toggle trig : Sig Control Normalized — 0/1 flip per edge. Inputs may be explicit triggers, booleans, or dimensionless control gates; audio and physical quantities require an explicit conversion first.
• onCount n trig : Trigger — passes every n-th rising edge; the output mirrors a legacy gate’s high phase; pulse inputs produce one-sample outputs, so it composes with hold/count downstream. Its fixed dimensionless integer divisor must lie from 1 through 4,294,967,295. It accepts the same gate inputs as toggle.
• choose [v…] sel : Sig Control — stepwise table read, table[clamp(round( sel))] — the runtime nth that compile-time nth/% cannot be. The table contains fixed compatible numeric values, finite in engine precision. The control output retains their common unit, with representation conversion: choose [500ms, 1s] 1 yields one second; choose [1st, 300Cents] 1 yields three semitones. The dimensionless control selector is read each sample.

These contracts survive aliases, factories, imports and records. Both counter forms remain available through the same stored value: f = count accepts f trig and f trig n, including higher-order calls. f trig is a completed counter, so (f trig) n cannot add a modulus afterward. Table structure and divisor values prepare before graph emission; checking retains their obligations without evaluating producers. Required-argument functions also support sections. The source compiler and native/JSON preparation share counter, divisor and table value checks. Invalid values in any executable graph reject the replacement before installation; the running performance continues.

Semantics inherited and documented rather than hidden: onsets triggers are block-rate with a 50 ms refractory, so counters advance with ~2-block quantization; noteTrig is constant-1 for a voice's life, so a per-voice count noteTrig is degenerate (it reads 1 forever) — the useful counters run on onsets and on gate-derived triggers. Determinism: counter state is a pure function of the trigger history, which is seeded (patterns) or already non-determinism-flagged (mic) — no new flags. The refinement checker knows the ranges (count trig n ∈ [0, n−1], toggle/onCount ∈ [0, 1], choose spans its table).

Boundary latching, seeded randomness, hysteresis (the quick wins)#

Shipped 2026-08-20 as the first follow-on to Runtime Control Flow: five small builtins that round out the decision vocabulary without touching the scheduler.

-- commit a drifting modulator once per cycle (2 = every 2 cycles):
depth = latchCycle 2 ((lfoSine 0.3Hz) * 0.4 + 0.5)
-- one uniform draw per cycle / per note — replay-exact, seek-safe:
cutoffMod = choose [0.0, 0.4, 0.7, 1.0] (cycleRand "form" * 3)
pan       = noteRand "spread" * 2 - 1
-- threshold decisions that don't chatter:
voxT = play vox !"c4 ~ e4 ~"
open = holdFor 250ms (schmitt 0.25 0.6 (follower voxT 5ms 120ms))

• latchCycle cycles m : Sig Control u — cycle-boundary sample-and-hold; passes m's unit through (the hold rule). cycles is a required, fixed positive integer; use latchCycle 1 m for each cycle, or bind everyOther = latchCycle 2. The boundary grid derives from absolute transport time (the PatternMod clock convention — block-size independent, seek lands on the same grid; the same live-tempo caveat applies). One cycle is one bar on Volta's grid, so this is the bar latch too.
• cycleRand "id" : Sig Control Normalized — one uniform [0, 1) draw per cycle: hash(module seed, id, cycle index), a stateless counter-based RNG (rng::stream_bipolar). No history means no drift: replay is bit-exact, and every reader of the same id sees the same draw (a shared selector).
• noteRand "id" : Sig Control Normalized — one draw per note, constant for the voice's life, keyed on the deterministic voice coordinate (track, cycle, event) — the seeded-noise-stream convention.
• schmitt lo hi m : Sig Control Normalized — hysteresis comparator: on when m rises through hi, off when it falls through lo, held in between. Where gate re-decides per sample at one ADAA-smoothed threshold, schmitt latches — a hovering signal cannot flap the decision. Thresholds lift to the live-scalar path (macro-affine expressions ride live).
• holdFor dur g : Sig Control Normalized — release debounce: 1 while g ≥ 0.5 and for dur after it last was (the countdown refills on every high sample). dur accepts ms/s/cycle units.

Verification: engine/tests/control_quickwins.rs renders through the real engine and pins the contracts — latchCycle flat within a cycle and re-committing at boundaries, cycleRand bit-identical across re-renders and host block sizes, schmitt/holdFor producing clean 0/1 spans with the hold extending the open phase.

Notes & limits#

• Broadband detection. Energy flux hears the loud transients; a quiet hi-hat under a sustained pad on the same bus can be masked (the peak gate). Put the trigger material on its own track (the demo keeps hats off the watched bus), or wait for the spectral-flux stage (Turn audio into musical events §7.1).
• Sensitivity is compile-time. A macro-ridden sensitivity is a cheap follow-on (Turn audio into musical events §8).
• The cursor is purely sequential. Trigger count drift shifts the pitch cycle's phase against the bar — musically intended; a bar-locked variant is an open question (Turn audio into musical events §8).
• Stopped source → silence, no triggers. A stopped source track's bus reads 0, so the bass rests. A missing source is an invalid execution plan.

Verification#

engine/src/offline.rs renders the full engine offline and asserts: play_on_bass_fires_on_kick_onsets_and_cycles_pitches (a synthetic kick panned left drives a playOn sine bass panned right — every bass note starts within 4 blocks of its kick and the pitches cycle c3→e3→g3→c3, measured by zero-crossings); onsetjam_demo_song_bass_follows_the_sampled_drums (the shipped demo against the real bd/sn WAVs — per-hit sounding windows, first trigger ≤ 6 blocks after the first kick, inter-hit decay, first note ≈ e1). Detector unit tests live in engine/src/voice/detect.rs.

Example#

examples/songs/onsetjam.volta ("Onset Jam") — a sampled kick/snare groove drives a sub bass via playOn, with hold (onsets drums) shimmer stepping the bass filter to a new color per hit. It plays whole on both targets.