VisualHPS visualizes the harmonic product spectrum of the input signal. The harmonic product spectrum is found by multiplying downsampled versions of the frequency spectrum of the input signal together. The idea is that musical signals normally consist of a fundamental frequency component and any number (potentially 0 or infinite) of harmonics. Given an input spectrum, if we downsample it by N (that is, only look at every Nth bin) we are essentially shifting the peaks in the frequency spectrum. A signal with harmonics will have peaks at integer multiples of the fundamental frequency, so by downsampling by a sequence of integers we are shifting these harmonics to be in line with the fundamental. If we multiply together all of these downsampled spectra, the result will theoretically only have a peak at the fundamental frequency. This type of analysis is not particularly computationally intensive (aside from taking the FFT) because it only involves multiplications, and it allows for polyphonic pitch detection of the input signal.
VisualHPS visualizes this process to give the user a more concrete picture of what the harmonic product spectrum is. The VisualHPS screen consists of four views: waveform, frequency spectrum, harmonic product series, and harmonic product spectrum. The waveform display simply shows the waveform of the most recently received audio buffer. The frequency spectrum view shows the spectrum of this buffer, in addition to a waterfall-type display of previous spectra. The harmonic product series display shows a user-settable number of downsampled versions of the input signal's frequency spectrum. Below this, the harmonic product spectrum view shows these downsampled spectra multiplied together. Some rudimentary analysis is done to the final harmonic product spectrum to attempt to extract the fundamental frequencies; the "peak threshold" for this process is given above the harmonic product spectrum view and the resulting fundamental frequencies are displayed below. The fundamental frequencies found are then output via OSC, so that they can be synthesized or processed elsewhere.
The algorithm carried out by VisualHPS has some limitations. First, in order to get a high frequency accuracy, an FFT of a 16,384 sample buffer is taken in real time, which introduces a large CPU load. A short-time fourier transform should be used here instead. Second, the method for detecting peaks and outputting the fundamental pitches in the signal is not ideal. It involves testing whether every sample is greater than an integer times the average value of the HPS - a smart local maximum algorithm would be better.
The package contains all required files including RtAudio and oscpack. VisualHPS has only been tested on a few Mac systems; no guarantees for any other operating system, and it's highly unlikely that it will work on a Windows system because of the mutual exclusion type used. Some tweaking of the makefile and VisualHPS may be necessary. Some PureData packages are included that can be used to synthesize and convert to MIDI the osc data sent out by VisualHPS. The package also contains this text as README.
Download the package
Download a very unstable pre-compiled binary for Mac OS X ~10.5