Write a spectral effect

For advanced authors: write your own frequency-domain effect as a per-bin law — gates, tilts, freezes, blurs, robot voices, bin shifts and cross-synthesis in a few lines.

What it is#

The builtin STFT effects (Spectral effects: freeze, blur, spectralGate, tilt, …) each run one hand-written function over every bin of every frame. spectralMap and spectralPolar let you write that function: a kernel law (Write a custom audio processor) that the engine runs once per bin per frame of a streaming STFT, then resynthesises.

-- keep bins above 30 % of the frame peak (a soft per-bin gate)
effect bones (threshold: Normalized = 0.3) on track =
  \bus -> bus |> spectralMap (\m -> let c = threshold * framePeak in m * m * m / (m * m + c * c + 0.000001))

-- robot voice: every bin's phase reset each hop
effect robot () on track = \bus -> bus |> spectralPolar (\m ph -> [m, 0.0])

-- a per-bin one-pole across frames (one state cell PER BIN)
effect smear (amount: Normalized = 0.7) on track =
  \bus -> bus |> spectralMap (\m s -> let y = s + (1.0 - amount) * (m - s) in [y, y]) [0.0]

-- blur across ±4 neighbouring bins: one Repeat loop
effect spread () on track =
  \bus -> bus |> spectralMap (\m -> foldl (\acc d -> acc + magAt d) 0.0 (range (0 - 4) 5) / 9.0)

-- a finer frame, and a frame-level fold: the frame's energy, normalised away
effect level () on track =
  \bus -> bus |> spectralMap (\m -> m * 0.05 / (0.000001 + sqrt (frameFold 0))) [] { frame = 2048, fold = \m e -> e + m * m }

Surface#

spectralMap   : Sig -> (\m [ports…] [s1 … sN] -> m' | [m', next])        -> [init] -> [Sig…] -> {options} -> Sig Audio Bipolar
spectralPolar : Sig -> (\m ph [ports…] [s1 … sN] -> [m', ph'] | [m', ph', next]) -> [init] -> [Sig…] -> {options} -> Sig Audio Bipolar

• m is the bin's magnitude in amplitude units: the raw Hann-windowed FFT magnitude ÷ N/4 (256 at the default frame), so a sine of amplitude A centred on a bin reads ≈ A at its peak bin at every frame size. ph is its phase in radians. spectralMap keeps the input phase; spectralPolar returns both.
• State binders are cells per bin (per channel), up to 8, seeded from the init (a number, a list, or a record — named state, st.env) at the node's start and carried frame to frame. dline is refused — there are no per-bin delay lines.
• Ports: up to 4 signal arguments after the init. Each is STFT-analysed (its left channel, the kernel-port rule) and its binder reads the port's magnitude at the same bin — cross-synthesis in user space; magAt d side / phaseAt d side read any bin of its spectrum.
• Macros captured by the law (or its fold) are read once per frame, at the sample the frame fires on (the hop is the parameter rate). Ports and macros share the kernel parameter cap of 32.
• Options (a trailing record, after any ports):
  • frame — the STFT size: 256, 512, 1024 (default), 2048 or 4096, a compile-time number. The hop is a quarter of it (75 % overlap) and the latency one frame; a longer frame resolves frequency finer (bin width sampleRate / frame) and smears time more.
  • fold — a frame-level prologue: \m [ports…] a1 … aN -> a' | [a1', …, aN'], run over every bin of the frame BEFORE the law, its accumulators seeded from foldInit (a number or a list, default 0) each frame; the law reads the finished values as frameFold 0 … frameFold (N − 1). It visits bins in ascending order, the left channel then the right, so a stereo max is linked (like framePeak) and a stereo sum covers both channels. It sees the bin context and the ports but neither its own results (frameFold) nor the law's cells (stateAt).
• The expression envelope is the kernel's: arithmetic, comparisons, eager if, let, inlined scalar lambdas, abs min max tanh sin cos floor sqrt exp ln pow atan2, foldl over constant lists (a range is one loop), sampleRate. atan2 y x (the angle of a complex bin, −π … π) is a kernel op on every tier.

Bin context (spectral kernels only)#

Each is a compile error outside spectralMap / spectralPolar (and inside process). A law sees only the analysed input frame and the previous frame's cells, never another bin's output of this frame, so bin order is unobservable: the law is a true map. Port and cell binders keep their identity through an inlined helper lambda (blur3 s may call stateAt s 1).

Semantics#

• Frame: periodic Hann analysis, FFT N (1024 unless frame), hop *N*/4, Hann synthesis, overlap-add ÷ 1.5 — engine/src/spectral_fx.rs's layout. The node's latency is exactly one frame: \m -> m is a pure *N*-sample delay at every size (pinned). The latency is reported to graph alignment, so a parallel dry branch in the same graph (bus + (bus |> spectralMap f)) is aligned, and a track seat's PDC and dry path include it.
• Stereo: a stereo input runs both channels in lockstep (frames fire on the same sample; framePeak and the fold are linked); a mono input stays mono.
• Order within a frame: analyse the input (and every port — its phases only when a law reads them); snapshot the cells (only when a law uses stateAt); run the fold over every bin; run the law over every bin; resynthesise.
• Output policy: a non-finite m'/ph' flushes that bin to 0 and a non-finite next state (or fold accumulator) to 0; resynthesised samples are NaN-flushed and soft-limited to ±2 (identity on [−1, 1]) — the kernel policy. DC and Nyquist bins resynthesise with a zero imaginary part.
• Seats: in a patch (source = … |> spectralMap f — one STFT per voice) and on a track (effect … on track, audioFx, buses, master inserts). Every voice's units come from a pool grown at load and sized to the largest frame any template asks for (FFT plans are built on the control side too), so a note-on only resets one. Refused inside feedback regions (per-sample).
• Determinism: pure f32 arithmetic on the analysed frame; a law that wants noise hashes binIndex/frameIndex with + × floor only (hash01 in spectrallib), which is bit-identical on every platform. Default-frame nodes render byte-identically to the Stage 1–4 engine (the spectrallab stems, compared against the merge base).

The library (spectrallib)#

import "spectrallib" (specGateK, tiltK, freezeK, blurK, robotize, whisperize, binShift, specComp, sustainK, specImprint, pitchShiftPV, pitchShiftLocked, binShiftLocked) — examples/songs/spectrallib.volta, auditioned by examples/songs/spectrallab.volta. The first four re-implement builtins and are pinned by engine/tests/spectral_kernels.rs (measured offline on a stereo saw/square/noise bus, not listened to; the builtins stay the reference):

blurK cannot match blur sample for sample: the builtin's phase walk is one sequential xorshift per channel, stepped bin after bin, and a per-bin law draws its own hash instead — so the pin is on what blur does (its spectrum and its temporal smear), not on samples. freezeK and blurK take a bin's angle with the atan2 op (a library octant polynomial before Stage 5; the pins measured the same either way).

Effects no builtin does:

Phase-locked peak shifting#

pvShiftLaw / pvShiftLockedLaw ratio bins shift by moving PEAKS (Laroche & Dolson): each output bin reads the source bin its region's peak (the loudest within ±3) moved from, so a lobe is translated, not stretched; phases advance by ratio × the source's true advance (its previous phase is kept in a cell and read with stateAt) plus π·bins/2. The plain law integrates every bin's phase; the locked law only the peak's, giving each region bin the peak's phase plus its analysed offset from it. Measured offline, not listened to (+4 semitones, plain → locked): steady saw −29.5 → −35.4 dB of power off the shifted partials; vibrato saw log-spectral distance to the source played a third up 15.56 → 12.90 dB, 5 ms envelope distance 0.92 → 0.54 dB; staccato line 2.70 → 2.52 dB and 1.31 → 0.47 dB. At ratio 1 (binShiftLocked) the two are equal (pinned).

Seeing a law: the probe's flat-frame view#

The DSP probe (Write a custom audio processor) on a spectralMap / spectralPolar call (or an effect holding one) adds a flat-frame view (ProbeReport.spectral): spectral_kernel::flat_gain runs the law on flat frames — every bin and port at one level, phase 0, trueHz the bin centre — held 64 frames at −40, −20 and −6 dB, and plots each bin's output over its input. A filter-like law draws one curve; a gate one per level (pinned: tiltLaw at 0.6 is 3.6 dB/oct within 1e-4 dB).

Cost#

Two program runs per sample plus an FFT pair per hop (compiler/src/cost.rs). The law runs on the lane tier (spectral_kernel::run_lanes): a law is a true map over bins with uniform control flow, so each op runs over 64 bins at once, every bin seeing exactly the scalar reference's (run) f32 operations in order — bit-identical (random programs and every spectrallib law, pinned); portable Rust, no native or wasm codegen. The fold stays scalar. Measured on a loaded machine (Spectral Kernels S5 2026 09 28): the scalar interpreter was 70–95 % of a real law's node cost; lanes run the node 3.3–6.2× faster (specGateK 5.3×, now ≈ 3.9× its builtin); what remains is mostly per-bin libm.

How it's implemented#

• IR — NodeKind::SpectralKernel { …, frame, fold, fold_cells, fold_init } (defaults off the wire) and KernelOp::Bin { q } (q ≥ 8 a fold accumulator), BinAt { q } (q ≥ 2 a port) and StateAt { i }, refused by the time-domain kernel_values::check; validate_spectral_node checks frames, port and fold reads.
• Compiler — descriptors with frame/foldInit/fold in compiler/src/builtins/fn_operands.rs; lower_kernel and lower_spectral_fold in compiler/src/lower/expr.rs; context names in compiler/src/kernel.rs; skipped port slots before a named option are params::SKIPPABLE_ABSENT markers.
• Engine — engine/src/spectral_kernel.rs (SpectralUnit sized to a capacity frame with its plans at construction; run_lanes, run, flat_gain); a voice's pool is grown by graph::follow_capacity (count and largest frame); graph_quality.rs charges one frame of latency.
• Tests — compiler/tests/spectral_kernels.rs, engine/src/spectral_kernel.rs, engine/tests/spectral_kernels.rs, engine/tests/dsp_probe.rs, engine/tests/rt_safety.rs.