VOLTAreference

Reference / Connect and share

Follow the pitch of live audio

Connect and share

Open in the workbench →
On this page · 9 sections

Estimate the pitch of an incoming signal and use it to guide another sound or control.

What it is#

pitchOf reads another track's bus (or the live mic) and returns its detected fundamental as a control signal in Hz. It is the pitch reading of a source bus, alongside Make one sound follow another's RMS number, Route audio between tracks's tap signal, and Turn audio into musical events's onsets trigger stream — the same previous-block bus read, analysed for pitch. Because the output is Hz, it drives oscillator freq ports and arithmetic directly: a synth shadows another track's melody the way playOn shadows its rhythm.

Inline audio graphs#

pitchOf audioExpr [smooth] also accepts a local audio graph, such as a filtered tapLive "lead". Its prepared 2048-sample window advances every 128 samples; the reported frequency slews per sample (default 20 ms). Before a confident estimate it emits zero; unvoiced windows hold the previous estimate. It analyses (left + right) / 2, so antiphase stereo can cancel. Source routing delay is additional. The track/string overload retains its previous bus analysis law. Both forms require nonnegative smoothing that remains finite in engine precision. Zero is supported. Source compilation and module loading enforce the same values, output units and audio-input contract.

Surface#

lead = play "pluck" !"c4 e4 g4 c5"            -- the track being followed

-- a saw whose frequency IS the lead track's detected pitch
shadow = patch { source = saw (pitchOf lead), ampEnv = adsr 8ms 0ms 1 250ms, gain = vel * 0.4 }

octDown = patch { source = sine (pitchOf lead * 0.5), gain = vel * 0.4 }     -- octave-down shadow
glide   = patch { source = saw  (pitchOf lead 60ms), gain = vel * 0.4 }      -- 60 ms output slew
room    = patch { source = square (pitchOf "input"), gain = vel * 0.3 }      -- track the room
  • pitchOf source [smooth] — source is a track binding (preferred; a track-name string is the legacy spelling), an aux-bus name, or "input" (the live mic). smooth is an optional compile-time duration (default 20 ms) that slews the Hz output. Returns a Sig Control Hz: the detected fundamental, held at its last confident value on unvoiced / near-silent frames (0 before the first confident pitch).
  • Signature: Track → [Const Milliseconds] → Sig Control Hz.

Detection (as built)#

engine/src/voice/detect.rs::PitchTracker — causal, block-rate, one instance per CompiledNode::PitchOf occurrence (per voice). It reads the source's previous-block post-fader audio (the Tap's exact read; L/R downmixed to mono, input.l/r for the mic), ring-buffers it into a 2048-sample window, and per block runs the McLeod Pitch Method (Normalized Square Difference Function):

  1. NSDF n(τ) = 2·Σ x[j]x[j+τ] / (Σ x[j]² + Σ x[j+τ]²) over the lag range for [50 Hz, 2000 Hz]; the denominator is O(1) per lag via a prefix sum of squares, and the numerator comes for every lag at once from one FFT autocorrelation of the 2×-zero-padded window (4096 points).
  2. Peak-picking — the maximum of each positive NSDF lobe is a candidate; accept the first whose value clears 0.9 × global max and whose lag is in range. Picking the first (lowest-lag) peak locks the fundamental over its sub-octave — a rich saw at 110 Hz tracks 110, not 220/330.
  3. Parabolic interpolation around the chosen lag → sub-sample period → hz = sr / period.
  4. Confidence gate — the window RMS must clear a floor and the best NSDF peak must reach a clarity threshold (0.6). Below either → unvoiced → detect returns None and the tracker HOLDs the last confident pitch.
  5. Output slew — the Hz output ramps toward the held pitch with a one-pole lag k = exp(−1/(sr·smooth)); the first confident pitch snaps.

Timing is block-quantized (the previous-block read, ~2 blocks ≈ 5 ms at 48 kHz) — the Route audio between tracks latency doctrine. The 2048-sample window fills in ~16 blocks (~43 ms), then a 220 Hz sine locks to 220 ± 2 Hz within a couple of blocks.

How it's implemented#

  • Compiler: BuiltinKind::PitchOf, a control descriptor in compiler/src/builtins/controls.rs (ports track, smooth default 0.02 s), Track → [Const Milliseconds] → Sig Control Hz — the optional smooth arg validated at lowering by ir/src/reader_values.rs::pitch_smoothing (the onsets pattern). The named form lowers (compiler/src/lower/expr/controls.rs) to NodeKind::PitchOf { source_track, smooth } (smooth folded to seconds), rate Control / unit Hz; the inline-audio form lowers (compiler/src/lower/expr/events.rs::lower_inline_detector) to NodeKind::PitchOfAudio { input, smooth }.
  • Non-determinism: compiler/src/lower.rs flags the module when a PitchOf names "input" (the input / onsets "input" rule); a track source is reproducible and is not flagged.
  • Diagnostics: refine.rs::check_causal treats a pitchOf read as a causal cross-track edge — an ok-level Causal advisory naming the 1-block latency, while shared execution validation rejects missing audio sources. Range analysis reports the Hz output as [0, 2000].
  • Engine: CompiledNode::PitchOf { source, smooth } (per-voice NodeState::PitchTracker), source resolved via resolve_bus (tracks, aux buses and "input"; missing sources fail before preparation). It reads track_taps (the same buffers Tap reads) or the live-input hub, detects, HOLDs, and slews. pitchOf "input" counts toward CompiledModule::uses_live_input, so the desktop host lazily opens the mic.
  • Browser: the same engine in a wasm AudioWorklet, including pitchOf "input" — the page's capture hub feeds the engine's input ring (Live audio input). (The Web Audio path, which folded PitchOf to a constant 0, was deleted 2026-07-24.)

Notes & limits#

  • Monophonic fundamental only. One pitch per bus; a chord reads as its most prominent partial. Chord detection is chordOf (Follow the pitch of live audio §6 Stage 3).
  • Range [50 Hz, 2000 Hz]. Sources outside this don't lock (they hold / read 0). An expected-range hint is an open question.
  • Cost. One 4096-point FFT pair per detection per voice — about 1 % of a 128-frame block at 48 kHz, so polyphonic use is fine. (The direct O(window · lag-span) loop it replaced on 2026-09-15 cost 52 % of the block whenever its source was voiced.) tune shares the same tracker.
  • Confidence isn't exposed yet — pitchConf (Stage 2) would let a patch gate or crossfade on tracker certainty.
  • Missing/stopped source → silence → HOLD. A stopped source's bus reads 0, so the tracker freezes at its last confident pitch (plus the compile-time warn).

Causality#

A pitchOf reads the source track's previous-block post-fader bus, so its output lags the source by exactly one render block (~128 samples ≈ 2.7 ms at 48 kHz), plus the detector's ~43 ms window fill on first lock. This discharges the Spec §5.3 Causal obligation by construction — the source's last block is already settled, so no track ordering is needed and a cross-track cycle is feedback-safe. refine.rs emits the ok-level Causal advisory per pitchOf edge and (as a signal leaf) contributes no zero-delay voice-graph edges. Missing sources are rejected by shared execution validation.

Verification#

Engine unit tests for PitchTracker live in engine/src/voice/detect.rs: pitch_tracker_converges_on_a_sine (220 Hz sine → 220 ± 2 Hz within ~20 blocks), pitch_tracker_resists_octave_error_on_a_saw (a 110 Hz saw locks at 110, not 220/330), pitch_tracker_holds_on_silence_and_noise (unvoiced → frozen) and pitch_tracker_smooth_slews_a_step (a stepped target reached gradually); engine/src/voice/eval.rs has pitchof_node_tracks_the_source_bus_and_holds (end-to-end through the voice graph). Compiler tests (compiler/src/lower/tests.rs) cover the lowering shape and the "input" non-determinism flag.

Example#

examples/songs/shadow.volta ("Shadow") — a saw lead melody with two pitch shadows: a sine sub an octave below (sine (pitchOf lead * 0.5)) and a detuned saw doubling the lead (saw (pitchOf lead)). Uses a track source (not "input"), so it stays deterministic and CI-verifiable, and it plays whole on both targets.