Three-dimensional object tracking with occlusion using plenoptic image sequences

Object tracking is an important problem in computer vision research. Among the difficulties of object tracking, the problem of partial and full occlusion is one of the most serious and challenging problems. To address the problem, we proposed methods to object tracking using plenoptic image sequences. Our methods take advantage of the refocusing capability that plenoptic imaging provides. The proposed methods input the sequences of focal stacks constructed by applying the refocusing algorithm on the plenoptic image sequences. The proposed image selection algorithms select the sequence of optimal images that can maximize the tracking accuracy from the sequence of focal stacks. Focus measure approach and confidence measure approach were proposed as image selection methods and both approaches were validated by the experiments using three groups of plenoptic image sequences that include heavily occluded target objects. The experimental results showed that the proposed methods were promising comparing to the conventional 2-D object tracking algorithms. (C) 2018 Society of Photo-Optical Instrumentation Engineers (SPIE)