Discrete Parameter Autoencoders for Semantic Hashing

Semantic hashing is a method for creating descriptive binary similarity hashes from feature vectors. The semantic hashing function is usually learned by a training an autoencoder neural network and applying a trick to binarise the middle layer. This study investigates the novel training algorithm Expectation Backpropagation (EBP) for training such a network. EBP is able to binarise the code layer without special additions to the algorithm and is capable of restricting the possible weights for the neural connections. The latter could be used efficiently implement the network in hardware or use it on embedded platforms as the weight matrix storage space is reduced. Using the EBP algorithm, a binary weight autoencoder network is trained on the MNIST dataset. Results indicate that 28-bit hashes created from the MNIST dataset are sufficiently distinctive for aiding in similarity search, either directly or as a way of pruning the search space.