User Tools

Site Tools


a_neural_algorithm_of_artistic_style

A Neural Algorithm of Artistic Style

Presenter Leon Gatys
Context NIPS 2015 Deep Learning Symposium
Date 12/10/15

This work uses a pre-trained convolutional network (the convolutional part of the VGG network) for image processing (no learning involved). When an image is presented to a convolutional network, it produces a sequence of filtered versions of the input, with irrelevant information discarded although the content is ideally preserved. If the correlation between feature maps in different layers is preserved, the overall texture of the inputs are possible. Optimizing the input (from noise) such that the correlation is the same, texture synthesis is possible. To do style transfer, texture information from a painting is estimated and is combined with a representation of an input image far into the network: First, the correlations are computed for all layers for a painting, then the intermediate representations/activations for an input image are computed, then the activation correlations for a white noise input are compared to the correlations for the painting, then the highest-layer activation for the white noise is compared (via a Frobenius loss) to those of the query image, and finally backpropagation is used to optimize the pixels so that the correlations and activation loss are jointly minimized. The result is an image which matches the low-level features and colormap of the painting, and gradually gets the content right of the query image. This is, of course, not constrained to artistic style transfer - it can also make two images share some high-level characteristics.

a_neural_algorithm_of_artistic_style.txt ยท Last modified: 2015/12/17 21:59 (external edit)