Autoencoders are basically a network that maps input features to a smaller vector and then back to the original features. It is basically a form of compression and decompression.

Variations

Denoising: a random number of inputs are dropped or corrupted during encoding while the decoder aims to reconstruct the original (non-corrupt) value.

Sparse: encoder/decoder is penalized if too many neurons have non-zero weights. The network must learn to contextualize.

k-Sparse: encoder runs forward to compress features, then k-highest elements are retained and all others zeroed, and decoder network operates on this. Backpropagation does not travel through the zeroed elements of the encoding fingerprint.

Variational: encoder maps input to a distribution instead of vector, decoder samples the distribution to retrieve original inputs.

Vector Quantized Variational Autoencoder: encoder compresses as normal, but the result is looked up in the codebook, and the closest codebook value is used instead, and the decoder learns to reconstruct inputs from that. The codebook is also learned such that existing values are averaged (with exponentially weighted moving averages) with what they matched against, and the original input to the encoding layer recieves gradients.