|Authors||Vincent Verfaille, Udo Zölzer, and Daniel Arfib|
|Publication Info||Retrieved here|
Discusses the use of adaptive controls (computed from sound features) for sound transformation (audio effects).
The features used for adaptive control of audio effect: Loudness (perceived intensity), time and rhythm (duration and repetition of sounds), pitch (frequencies and amplitudes of harmonics), timbre (non-pitch, non-loudness distinguishing characteristic of sounds), spatial hearing (auditory localization of sound).
Defines effect categories with the same categories as above: Loudness (tremolo, dynamic range), time (resampling and time-stretching), pitch (shifting, harmonization, autotune), timbre (time-varying delay line, spectral envelope, adaptive filtering), spatial effects (panning, delay/filter to simulate localization, delay/reverb, doppler effect), also “multidimensional” for effects that cover more than one of these categories.
Defined as “effects with a time-varying control derived from sound features transformed into valid control values using specific mapping functions”. Features can be derived from the to-be-effected input signal, another input signal, the output signal, etc. Argues that feature “accuracy” may or may not be needed, but perceptually motivated features may work better for adaptive control.
Proposes mappings from M features to N effect parameters by combining the features and “conditioning” the resulting signal. To combine features, they are normalized and a transfer function is applied to focus on the “interesting” part or to filter values, then they are linearly combined by some weighting and the combination is also warped/filtered/scaled.
Formulates possible adaptive effects in all of the categories above. Loudness: compression, “voiciness gate”, adaptive rate and depth tremolo. Time: adaptive time warping (eg for attack preservation or shortening gaps/changing signal envelope); also discusses preserving overall signal length with and without stretch amount boundaries. Pitch: discusses how to use PSOLA, phase vocoder and source-filter techniques with a specific pitch shift ratio; also describes adaptive intonation change. Timbre: Discusses an adaptive equalizer controlled by STFT output and adaptive spectral warping for changing the sound's harmonicity. Spatial: Discusses adjusting panning adaptively, eg based on timbre - also spectral panning (per-bin) and multi-channel panning. Multidimensional: Robotization which modifies both pitch and timbre and granular delay where grain size and delay time are adaptively controlled.