Interactive continual learning for robots: a neuromorphic approach

Intelligent robots need to recognize objects in their environment. This task is conceptually different from the typical image classification task in computer vision. Robots need to recognize particular object instances, not classes of objects, which makes these tasks simpler. However, these instances need to be recognized under different viewing angles, poses, and lighting conditions reliably. Moreover, for many application, robots need the capability to learn new objects quickly, e.g., in an interactive session with the user, and adapt object representations if conditions change and mistakes are made. This scenario creates a demand for object learning that (1) is continual, i.e. new objects can be added at any time without causing forgetting, (2) requires a small amount of data that can be acquired in a short interactive session with the user, and (3) stays open to later adaptation. Deep neural networks trained with slow gradient-based backpropagation, despite of their excellent performance on image processing tasks, are not well-suited for interactive robotic learning tasks. We thus explore smaller neural architectures and increase autonomy of the learning process by a neuronal state machine (NSM). The NSM regulates learning in the network and enables continual adaptation of the learned object prototypes. We implement this model as a spiking neural network in Intel's neuromorphic research chip Loihi and test it on a custom event-based camera dataset generated in a simulated 3D environment. Our spiking neuronal network uses simple feature extraction layers and a single plastic layer that stores visual patterns using online, on-chip local learning rules. This network reaches 96.55 +/- 2.02% of testing accuracy on sets of 8 3D objects with 8 different views per object in interactive, on-demand learning experiments; it demonstrates up to x300 energy efficiency and better or on par latency compared to other online learning methods. This work contributes to neuronal-network based machine learning for robots with a small power footprint and interactive learning capability.