Autoencoders are a data-compression model. They can be used to encode a given input into a representation of smaller dimension. A decoder can then be used to reconstruct the input back from the encoded version. In this blog post, I will share how I built an autoencoder in the library Lasagne. I will use the MNIST dataset for illustration.
Autoencoders are generally used in unsupervised data learning settings. When we have unlabeled data, we can use an autoencoder to learn an internal, low-dimensional representation of the input data. The model does not need output labels, rather the output is supposed to be the reconstruction of the input data itself. The errors are propagated back and slowly we can expect the model to learn a well encoded representation of the input data.
A simple autoencoder
Let's build a simple model with an input layer, a hidden (encoded) layer and an output layer.
First, let's get done with the imports and define model parameters.
Next, define a function to generate batches of data.
Build the network:
Now, we will use the above model. I have the MNIST dataset dumped in a file using pickle.
The data is normalized to be between 0 and 1, and reshaped to be of shape (784, ) from the original (1, 28, 28).
After loading the dataset into train and test variables, we proceed to train the model.
We train the network for 50 epochs. After that it seems to reach a reasonably good error of 0.023.
Now we will plot the images and visualize the results.
This is what we get.
The autoencoder does a rather good job at learning encodings and reconstructing the digits from them. The last row shows the 32 dimensional encoding learned for each of the ten test images (in a 4x8 sized image).
Find the entire working code for this autoencoder in Lasagne on my github.