Colin Raffel headshot

Colin Raffel

My research focuses on machine learning techniques for sequential data. I am currently a resident at Google Brain. I recently completed a PhD in Electrical Engineering at Columbia University In LabROSA, supervised by Dan Ellis. My thesis focused on learning-based methods for comparing sequences. The main product of my doctoral work is the Lakh MIDI Dataset, a large collection of music transcriptions which have been matched and aligned to corresponding audio recordings. In 2010, I received a Master's in Music, Science and Technology from Stanford University's CCRMA, supervised by Julius O. Smith III. I did my undergrad at Oberlin College, where I majored in Mathematics.

Publications

Extracting Ground Truth Information from MIDI Files: A MIDIfesto
Colin Raffel and Daniel P. W. Ellis
Proceedings of the 17th International Society for Music Information Retrieval Conference, 2016.

Learning-Based Methods for Comparing Sequences, with Applications to Audio-to-MIDI Alignment and Matching
Colin Raffel
PhD Thesis, 2016.

Feed-Forward Networks with Attention Can Solve Some Long-Term Memory Problems
Colin Raffel and Daniel P. W. Ellis
Proceedings of the 4th International Conference on Learning Representations (Workshop Track), 2016.

Pruning Subsequence Search with Attention-Based Embedding
Colin Raffel and Daniel P. W. Ellis
Proceedings of the 41st IEEE International Conference on Acoustics, Speech and Signal Processing, 2016.

Optimizing DTW-Based Audio-to-MIDI Alignment and Matching
Colin Raffel and Daniel P. W. Ellis
Proceedings of the 41st IEEE International Conference on Acoustics, Speech and Signal Processing, 2016.

Poker-CNN: A Pattern Learning Strategy for Making Draws and Bets in Poker Games
Nikolai Yakovenko, Liangliang Cao, Colin Raffel, and James Fan
Proceedings of the 30th AAAI Conference on Artificial Intelligence, 2016.

Accelerating Multimodal Sequence Retrieval with Convolutional Networks
Colin Raffel and Daniel P. W. Ellis
NIPS Multimodal Machine Learning Workshop, 2015.

Large-Scale Content-Based Matching of MIDI and Audio Files
Colin Raffel and Daniel P. W. Ellis
Proceedings of the 16th International Society for Music Information Retrieval Conference, 2015.
Best Student Paper Award

librosa: Audio and Music Signal Analysis in Python
Brian McFee, Colin Raffel, Dawen Liang, Daniel P. W. Ellis, Matt McVicar, and Oriol Nieto
Proceedings of the 14th Python in Science Conference, 2015.

mir_eval: A Transparent Implementation of Common MIR Metrics
Colin Raffel, Brian McFee, Eric J. Humphrey, Justin Salamon, Oriol Nieto, Dawen Liang, and Daniel P. W. Ellis
Proceedings of the 15th International Society for Music Information Retrieval Conference, 2014.
Best Poster Presentation Award

Estimating Timing and Channel Distortion Across Related Signals
Colin Raffel and Daniel P. W. Ellis
Proceedings of the 39th IEEE International Conference on Acoustics, Speech and Signal Processing, 2014.

Using Noise Substitution for Backwards-Compatible Audio Codec Improvement
Colin Raffel
Proceedings of the 129th Convention of the Audio Engineering Society, 2010.

Practical Modeling of Bucket-Brigade Device Circuits
Colin Raffel and Julius O. Smith
Proceedings of the 13th International Conference on Digital Audio Effects (DAFx-10), 2010.

The Lattice Harp: A New Hybrid Instrument and Controller
Colin Raffel, Nick Kruge, Diane Douglas, Edgar Berdahl, and Wendy Ju
Proceedings of the 2010 International Computer Music Conference, 2010.

Technical Reports, Whitepapers, etc.

This Is My Jam: Data Dump
Andreas Jansson, Colin Raffel, and Tillman Weyde
16th International Society for Music Information Retrieval Conference Late Breaking and Demo Papers, 2015.

Learning Efficient Representations for Sequence Retrieval
Colin Raffel
National Science Foundation Data Science Workshop, 2015.

Intuitive Analysis, Creation and Manipulation of MIDI Data with pretty_midi
Colin Raffel and Daniel P. W. Ellis
15th International Society for Music Information Retrieval Conference Late Breaking and Demo Papers, 2014.

Reproducing Pitch Experiments in "Measuring the Evolution of Contemporary Western Popular Music"
Colin Raffel and Daniel P. W. Ellis
Technical Report, 2013.
SoundSoftware.ac.uk Prize for Reproducible Research

Software Projects

mir_eval - a simple-to-use reference implementation of many music information retrieval metrics.

lasagne - a library for constructing neural networks in Theano.

pretty_midi - utility classes and functions for easily creating, manipulating, and analyzing MIDI data.

librosa - DSP routines for audio and music signal analysis.

Talks

The Lakh MIDI Dataset: How It Was Made, and How to Use It at BISH Bash Meetup, 2016.

Extracting Ground Truth Information from MIDI Files: A MIDIfesto at 17th International Society for Music Information Retrieval Conference, 2016.

Learning-Based Methods for Comparing Sequences, with Applications to Audio-to-MIDI Alignment and Matching at 2nd ICML Machine Learning for Music Discovery Workshop, 2016.

Optimizing DTW-Based Audio-to-MIDI Alignment and Matching at 41st IEEE International Conference on Acoustics, Speech, and Signal Processing, 2016.

Accelerating Large-Scale Sequence Retrieval with Convolutional Networks at IIT Bombay Electrical Engineering Seminar, 2015.

Large-Scale Content-Based Matching of MIDI and Audio Files at 16th International Society for Music Information Retrieval Conference, 2015.

Learning Efficient Representations for Sequence Retrieval at Boston Data Festival, 2015.

Using Convolutional Networks (with Attention) for Orders-of-Magnitude Speedup of DTW-Based Sequence Retrieval at Spotify Machine Learning Seminar, 2015.

Recurrent Networks in Lasagne at Mount Sinai Hammer Lab Seminar, 2015.

Lasagne Tutorial at Next.ml Boston, 2015.

Theano Tutorial at Next.ml Boston, 2015.

mir_eval at Objective Evaluation in Semantic Audio Analysis and Processing Panel at the 138th Convention of the Audio Engineering Society, 2015.

MIR at LabROSA in 2014 at NEMISIG 2015, 2015.

Large-Scale Content-Based Matching of Audio and MIDI Data at Stanford University DSP Seminar, 2015.

Advances and Challenges in Large-Scale Music Information Retrieval at Digital Music Research Network+8, 2013.

Quantifying Rhythmic Synchrony at Midwestern Music Cognition Symposium, 2013.

A Sequential Approach to Musical Event Detection at Carnegie Mellon University Music and Technology Seminar, 2011.

Using Noise Substitution for Backwards-Compatible Audio Codec Improvement at 129th Convention of the Audio Engineering Society, 2010.

ROW-mp3: An Enhanced MP3-Compatible Audio Codec at Stanford University DSP Seminar, 2010.

Practical Modeling of Bucket-Brigade Device Circuits at 13th International Conference on Digital Audio Effects, 2010.

The Lattice Harp: A New Hybrid Instrument and Controller at 2010 International Computer Music Conference, 2010.

An Effective Model of Bucket-Brigade Device-Based Audio Circuits at Stanford University DSP Seminar, 2010.

Voltage-Controlled Resistance: Modulate Anything at Circuitastrophe Circuit Bending Music Festival, 2008.

Poster Presentations

Feed-Forward Networks with Attention Can Solve Some Long-Term Memory Problems at 4th International Conference on Learning Representations, 2016.

Pruning Subsequence Search with Attention-Based Embedding at 41st IEEE International Conference on Acoustics, Speech, and Signal Processing, 2016.

Accelerating Multimodal Sequence Retrieval with Convolutional Networks at NIPS Multimodal Machine Learning Workshop, 2015.

Learning Efficient Representations for Sequence Retrieval at National Science Foundation Data Science Workshop, 2015.

Intuitive Analysis, Creation and Manipulation of MIDI Data with pretty_midi at 15th International Society for Music Information Retrieval Conference Late Breaking and Demo Papers, 2014.

mir_eval: A Transparent Implementation of Common MIR Metrics at 15th International Society for Music Information Retrieval Conference, 2014.

Estimating Timing and Channel Distortion Across Related Signals at 39th IEEE International Conference on Acoustics, Speech and Signal Processing, 2014.

Etc.

Experimentalists Anonymous is an online resource and community for people interested in do-it-yourself audio electronics which I have maintained since 2003.

eaced was my web-based business, through which I sold custom-built audio hardware from 2004-2009.

The Columbia Neural Network Reading Group and Seminar Series is a bi-weekly group I organize which hosts talks and paper discussions about neural networks and their applications.

Hacking Audio and Music Research (HAMR) is a series of research hackathons I have coordinated since 2013.

Synth-in-a-Month is an analog modular synthesizer I built in January of 2006.

I built a hollowbody electric guitar at the Totnes School of Guitarmaking during my final semester of high school.

The ooscc is an open-ended hardware controller I designed which communicates over Ethernet using Open Sound Control.