507-398-2646
San Francisco, CA 94107

Colin Raffel


Education

Google, Mountain View, CA

Resident in the Google Brain group, 2016–2017

Columbia University, New York, NY

PhD in Electrical Engineering at the Laboratory for the Recognition and Organization of Speech and Audio, advisor Dan Ellis, 4.1 cumulative GPA, 2012–2016

Stanford University, Stanford, CA

Master of Arts in Music, Science, and Technology at the Center for Computer Research in Music and Acoustics, 4.1 cumulative GPA, 2009–2010

Oberlin College, Oberlin, OH

Bachelor of Arts in Mathematics with Physics minor and independent studies in audio circuit design, 3.7 cumulative GPA, 2005–2009

Publications

Colin Raffel, Minh-Thang Luong, Peter J. Liu, Ron J. Weiss, and Douglas Eck, “Online and Linear-Time Attention by Enforcing Monotonic Alignments”, Proceedings of the 34th International Conference on Machine Learning, 2017.
Dieterich Lawson, George Tucker, Chung-Cheng Chiu, Colin Raffel, Kevin Swersky, and Navdeep Jaitly, “Learning Hard Alignments with Variational Inference”, arXiv preprint arXiv:1705.05524, 2017.
Colin Raffel and Dieterich Lawson, “Training a Subsampling Mechanism in Expectation”, Workshop Extended Abstracts of the 5th International Conference on Learning Representations, 2017.
Justin Gilmer, Colin Raffel, Samuel S. Schoenholz, Maithra Raghu, and Jascha Sohl-Dickstein, “Explaining the Learning Dynamics of Direct Feedback Alignment”, Workshop Extended Abstracts of the 5th International Conference on Learning Representations, 2017.
Colin Raffel and Daniel P. W. Ellis, “Extracting Ground Truth Information from MIDI Files: A MIDIfesto”, Proceedings of the 17th International Society for Music Information Retrieval Conference, 2016.
Colin Raffel, “Learning-Based Methods for Comparing Sequences, with Applications to Audio-to-MIDI Alignment and Matching”, PhD Thesis, 2016.
Colin Raffel and Daniel P. W. Ellis, “Feed-Forward Networks with Attention Can Solve Some Long-Term Memory Problems”, Workshop Extended Abstracts of the 4th International Conference on Learning Representations, 2016.
Colin Raffel and Daniel P. W. Ellis, “Pruning Subsequence Search with Attention-Based Embedding”, Proceedings of the 41st IEEE International Conference on Acoustics, Speech and Signal Processing, 2016.
Colin Raffel and Daniel P. W. Ellis, “Optimizing DTW-Based Audio-to-MIDI Alignment and Matching”, Proceedings of the 41st IEEE International Conference on Acoustics, Speech and Signal Processing, 2016.
Nikolai Yakovenko, Liangliang Cao, Colin Raffel, and James Fan, “Poker-CNN: A Pattern Learning Strategy for Making Draws and Bets in Poker Games”, Proceedings of the 30th AAAI Conference on Artificial Intelligence, 2016.
Colin Raffel and Daniel P. W. Ellis, “Accelerating Multimodal Sequence Retrieval with Convolutional Networks”, NIPS Multimodal Machine Learning Workshop, 2015.
Colin Raffel and Daniel P. W. Ellis, “Large-Scale Content-Based Matching of MIDI and Audio Files”, Proceedings of the 16th International Society for Music Information Retrieval Conference, 2015.
Brian McFee, Colin Raffel, Dawen Liang, Daniel P. W. Ellis, Matt McVicar, and Oriol Nieto, “librosa: Audio and Music Signal Analysis in Python”, Proceedings of the 14th Python in Science Conference, 2015.
Colin Raffel, Brian McFee, Eric J. Humphrey, Justin Salamon, Oriol Nieto, Dawen Liang, and Daniel P. W. Ellis, “mir_eval: A Transparent Implementation of Common MIR Metrics”, Proceedings of the 15th International Society for Music Information Retrieval Conference, 2014.
Colin Raffel and Daniel P. W. Ellis, “Estimating Timing and Channel Distortion Across Related Signals”, Proceedings of the 39th IEEE International Conference on Acoustics, Speech and Signal Processing, 2014.
Colin Raffel, “Using Noise Substitution for Backwards-Compatible Audio Codec Improvement”, Proceedings of the 129th Convention of the Audio Engineering Society, 2010.
Colin Raffel and Julius O. Smith, “Practical Modeling of Bucket-Brigade Device Circuits”, Proceedings of the 13th International Conference on Digital Audio Effects (DAFx-10), 2010.
Colin Raffel, Nick Kruge, Diane Douglas, Edgar Berdahl, and Wendy Ju, “The Lattice Harp: A New Hybrid Instrument and Controller”, Proceedings of the 2010 International Computer Music Conference, 2010.

Technical Reports, Whitepapers, etc.

Andreas Jansson, Colin Raffel, and Tillman Weyde, “This Is My Jam: Data Dump”, 16th International Society for Music Information Retrieval Conference Late Breaking and Demo Papers, 2015.
Colin Raffel, “Learning Efficient Representations for Sequence Retrieval”, National Science Foundation Data Science Workshop, 2015.
Colin Raffel and Daniel P. W. Ellis, “Intuitive Analysis, Creation and Manipulation of MIDI Data with pretty_midi”, 15th International Society for Music Information Retrieval Conference Late Breaking and Demo Papers, 2014.
Colin Raffel and Daniel P. W. Ellis, “Reproducing Pitch Experiments in "Measuring the Evolution of Contemporary Western Popular Music"”, Technical Report, 2013.

Honors and Awards

National Science Foundation Student Travel Grant, 41st IEEE International Conference on Acoustics, Speech, and Signal Processing, 2016
Best Student Paper, 16th International Society for Music Information Retrieval Conference, 2015
Travel Grant, National Science Foundation Data Science Workshop, 2015
Best Poster Presentation, 15th International Society for Music Information Retrival Conference, 2014
Student Travel Award, International Society for Music Information Retrieval, 2014
Prize for Reproducibility in Audio and Music Research, SoundSoftware.ac.uk, 2013
National Science Foundation Integrative Graduate Education and Research Traineeship Grant, 2012–2014
Ableton Best Hack Award, Music Hack Day San Francisco, 2012
Best Educational App: Rock Prodigy, Appy Awards, 2011
Echo Nest Best Hack Award, Music Hack Day Barcelona, 2010
7Digital Best Hack Award, Music Hack Day Barcelona, 2010

Business Ventures

Experimentalists Anonymous

Design, build, and sell custom electronic devices; host audio circuit schematic archive with over three million downloads served; maintain online message board with over fourteen thousand posts; provide site hosting for do-it-yourself audio hardware enthusiasts; help individuals start successful independent businesses selling electronic audio devices, 2003–present

Noisette Audio, LLC

Provided consulting services in the field of audio signal processing, analysis, and synthesis; develop software for electronic music composition, 2011–2012

Teaching

Teacher's Assistant, Deep Learning for Computer Vision and NLP

Advised students on software and final projects culminating in multiple conference submissions, 2015

Teacher's Assistant, Music Digital Signal Processing

Evaluated projects, provided assistance, and held office hours for a Master's-level course, 2013–2014

Electronics

Filled in for a professor on sabbatical and taught a recurring electronics course in the Oberlin College Physics Department, 2009

Circuit Bending

Taught a course on modifying electronic music circuits and basic electronics as part of the Oberlin Experimental College program; developed a curriculum, lab activities, and lectures, 2006

Students Supervised

Rafael Valle, visiting researcher at Columbia Univeristy, 2015
Hilary Mogul, undergraduate student at Columbia University, 2014
Dylan Kario, undergraduate student at Columbia University, 2014
Zhengshan Shi, visiting researcher at Columbia University, 2013

Talks

Online and Linear-Time Attention by Enforcing Monotonic Alignments at 34th International Conference on Machine Learning, 2017.
The Lakh MIDI Dataset: How It Was Made, and How to Use It at BISH Bash Meetup, 2016.
Extracting Ground Truth Information from MIDI Files: A MIDIfesto at 17th International Society for Music Information Retrieval Conference, 2016.
Learning-Based Methods for Comparing Sequences, with Applications to Audio-to-MIDI Alignment and Matching at 2nd ICML Machine Learning for Music Discovery Workshop, 2016.
Optimizing DTW-Based Audio-to-MIDI Alignment and Matching at 41st IEEE International Conference on Acoustics, Speech, and Signal Processing, 2016.
Accelerating Large-Scale Sequence Retrieval with Convolutional Networks at IIT Bombay Electrical Engineering Seminar, 2015.
Large-Scale Content-Based Matching of MIDI and Audio Files at 16th International Society for Music Information Retrieval Conference, 2015.
Learning Efficient Representations for Sequence Retrieval at Boston Data Festival, 2015.
Using Convolutional Networks (with Attention) for Orders-of-Magnitude Speedup of DTW-Based Sequence Retrieval at Spotify Machine Learning Seminar, 2015.
Recurrent Networks in Lasagne at Mount Sinai Hammer Lab Seminar, 2015.
Lasagne Tutorial at Next.ml Boston, 2015.
Theano Tutorial at Next.ml Boston, 2015.
mir_eval at Objective Evaluation in Semantic Audio Analysis and Processing Panel at the 138th Convention of the Audio Engineering Society, 2015.
MIR at LabROSA in 2014 at NEMISIG 2015, 2015.
Large-Scale Content-Based Matching of Audio and MIDI Data at Stanford University DSP Seminar, 2015.
Advances and Challenges in Large-Scale Music Information Retrieval at Digital Music Research Network+8, 2013.
Quantifying Rhythmic Synchrony at Midwestern Music Cognition Symposium, 2013.
A Sequential Approach to Musical Event Detection at Carnegie Mellon University Music and Technology Seminar, 2011.
Using Noise Substitution for Backwards-Compatible Audio Codec Improvement at 129th Convention of the Audio Engineering Society, 2010.
ROW-mp3: An Enhanced MP3-Compatible Audio Codec at Stanford University DSP Seminar, 2010.
Practical Modeling of Bucket-Brigade Device Circuits at 13th International Conference on Digital Audio Effects, 2010.
The Lattice Harp: A New Hybrid Instrument and Controller at 2010 International Computer Music Conference, 2010.
An Effective Model of Bucket-Brigade Device-Based Audio Circuits at Stanford University DSP Seminar, 2010.
Voltage-Controlled Resistance: Modulate Anything at Circuitastrophe Circuit Bending Music Festival, 2008.

Poster Presentations

Online and Linear-Time Attention by Enforcing Monotonic Alignments at 34th International Conference on Machine Learning, 2017.
Feed-Forward Networks with Attention Can Solve Some Long-Term Memory Problems at 4th International Conference on Learning Representations, 2016.
Pruning Subsequence Search with Attention-Based Embedding at 41st IEEE International Conference on Acoustics, Speech, and Signal Processing, 2016.
Accelerating Multimodal Sequence Retrieval with Convolutional Networks at NIPS Multimodal Machine Learning Workshop, 2015.
Learning Efficient Representations for Sequence Retrieval at National Science Foundation Data Science Workshop, 2015.
Intuitive Analysis, Creation and Manipulation of MIDI Data with pretty_midi at 15th International Society for Music Information Retrieval Conference Late Breaking and Demo Papers, 2014.
mir_eval: A Transparent Implementation of Common MIR Metrics at 15th International Society for Music Information Retrieval Conference, 2014.
Estimating Timing and Channel Distortion Across Related Signals at 39th IEEE International Conference on Acoustics, Speech and Signal Processing, 2014.

Employment

The Way of H

Developed pitch detection and audio timescale modification algorithms for Rock Prodigy, an educational iPhone application for guitarists, 2011–2013

Beat Kangz Electronics

Maintained and improved audio processing functionality for a hardware- and software-based music production system, 2011–2012

Imagine Research

Helped develop a machine learning-based sound classification and identification system, 2010–2012

SacredAgent

Created a musical event detection system which algorithmically demarcates beats and changes in instrumentation, dynamics, and melody, 2010

WOBC, Oberlin College Radio

Worked as Engineer, 2007–2008, Station Manager, 2008–2009, and Web Administrator, 2009

eBay

Employed as summer intern in user interface design department and Musical Instrument category management team, 2004

Academic Service

Late Breaking/Demo and Unconference Chair

Reviewed all submissions for the late breaking/demo session and coordinated the unconference at the 17th International Society for Music Information Retrieval Conference, 2016

Columbia Neural Network Research Group and Seminar Series

Organized and presented paper discussions, tutorials, and seminars on neural networks, 2014–2016

Crucial Python

Curated a seminar series on obscure but useful aspects of the Python programming language, 2014

Hacking Audio and Music Research (HAMR)

Started a series of hackathons where researchers rapidly prototype new ideas over the course of a weekend; past HAMRs have been held at Columbia University, Dartmouth College, Drexel University, and Academia Sinica, 2013–present

Intern Supervisor

Oversaw summer interns working on Music Information Retrieval research projects, 2013–2016

KZSU, Stanford University Radio

Developed web applications and a music fingerprinting system, 2009–2010

Mathematics Tutor

Assisted students studying algebra and linear algebra, 2008–2009

Engineering People of Oberlin Club

Served as the Electrical Engineering group coordinator and web administrator, 2008–2009

Oberlin Student Cooperative Association Website Coordinator

Maintained the website for Oberlin's eating and living cooperative organization, 2006–2007

Reviewing

International Society for Music Information Retrieval Conference, 2014–2017
Journal of New Music Research, 2016
EURASIP Journal on Audio, Speech, and Music Processing, 2015
IEEE International Symposium on Information Theory, 2014

Additional Coursework

Music Information Retrieval Workshop, Stanford, CA

Summer workshop covering machine learning and audio feature extraction, 2010

Rochester Community and Technical College, Rochester, MN

Courses in studio recording and electronic music composition, 2002–2004

Software Libraries

mir_eval, a simple-to-use reference implementation of many music information retrieval metrics.
lasagne, a library for constructing neural networks in Theano.
pretty_midi, utility classes and functions for easily creating, manipulating, and analyzing MIDI data.
librosa, DSP routines for audio and music signal analysis.