The Generative Adversarial Network (GAN) is a relatively new Machine Learning architecture for neural networks, first introduced by Ian Goodfellow et al at the University of Montreal in 2014 [1]. It is an unsupervised, generative learning machine that has not the luxury of having labeled samples, but learns from its mistakes and tries not to make similar errors in the future. GANs are a new way of training a neural net.

Linear interpolation between Françoise Hardy, Jane Birkin and Brigitte Bardot.

GANs contain not one, but two independent nets that work separately and act as adversaries. The first neural net is called the Discriminator (D) and is the net that has to undergo training. The second network is called the Generator (G) and is tasked to generate random samples that resemble real samples with a twist rendering them as fake samples. However, GANs are notorious hard to train. To overcome this, we can introduce a third network, an Encoder (E). Instead of feeding the Generator with random noise, we connect the output of the encoder to the input of the Generator, as initial ‘educated guess’. The latent space Z is more meaningful now and I noticed an increase in training speed of about 10 times. Moreover, by adding a variational Bayes layer after the output of the Encoder, the results and stability of the training process improved even more. The network I programmed and trained is shown in figure 2.

Brigitte Bardot with and without makeup.

I trained the VAEGAN network with the aligned celeb dataset. And then the fun starts. You can encode every face to a vector Z and decode it with the Generator to an approximation. By searching the mean vector of a given feature, it is possible now to add and subtract facial features to the original image. Figure 3 shows Brigitte Bardot with make-up and without makeup, by searching the mean ‘with make-up’ vector and mean ‘without make-up’ vector. Figure 1 shows the linear interpolation between Françoise Hardy, Jane Birkin and Brigitte Bardot.


Let’s put a smile on your face…

[1] Ian J. Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, Yoshua Bengio, Generative Adversarial Networks, arXiv:1406.2661 [stat.ML]