Capturing latent 3D representations of parallax-sensitive landmarks

The problem of identifying 3D objects that have drastically different 2D representations in different views is challenging. This is because features that are important for matching are not always view-invariant and may not be visible from certain perspectives. This research looks to infer the 3D geometry of specific landmarks such that predictions of a viewpoint's orientation about the landmark can be made from 2D images. For our dataset we use Google Earth to visit four well-known landmark sites and capture 2D images from a range of perspectives about them. The landmarks are chosen to be sensitive to parallax in order to ensure wide variance in our training images. We implement a 5-layer autoencoder network that takes 224x224x3 sized images and encodes them into 3136 element-long vectors, then replicates the input image from the encoding vector. We use the bottleneck encodings to generate predictions of the camera's azimuth, elevation, and range relative to the landmark. We then compare input images with predicted parameters and replicated decoded images to measure the accuracy of our model. Our experimentation shows that a simple autoencoder network is capable of learning enough of the 3D geometry of a landmark to accurately predict viewpoint orientations from 2D images of landmarks.