Simultaneous Exploration and Segmentation with Incomplete Data

We consider the problem of pan-tilt sensor control for active segmentation of incomplete multi-modal data. Since demanding optimal control does not allow for online replanning, we rather employ the optimal planner offline to provide guiding samples for learning a CNN-based control policy in a guided Q-learning framework. The proposed policy initialization and guided Q-learning avoids poor local optima and yields reasonable results already from hundreds of roll-outs. The results suggest that the proposed policy outperforms a baseline and is suitable for real-time control.