Competitive Spiking And Indirect Entropy Minimization Of Rate Code: Efficient Search For Hidden Components
Journal of physiology, Paris(2004)
摘要
Our motivation, which originates from the psychological and physiological evidences of component-based representations ill the brain, is to find neural methods that call efficiently search for structures. Here, ail architecture made of coupled parallel working reconstruction subnetworks is presented. Each subnetwork utilizes non-negativity constraint oil the generative weights and oil the internal representation. 'Spikes' are generated within subnetworks via winner take all mechanism. Memory components are modified in order to directly minimize the reconstruction error and to indirectly minimize the entropy of the spike rate distribution, via a combination of a stochastic gradient search and a novel tuning method. This tuning dynamically changes the learning rate: the higher the entropy of the spike rate, the higher the learning rate of the gradient search ill the Subnetworks. This method effectively reduces the search space and increases the escape probability from high entropy local minima. We demonstrate that one subnetwork can develop localized and oriented components. Coupled networks call discover and sort components into the subnetworks, a problem subject to combinatorial explosion. Synergy between spike code and rate code is discussed. (C) 2005 Elsevier Ltd. All rights reserved.
更多查看译文
关键词
grouping components,reconstructive network,spike code,indirect entropy minimization,dynamic learning rate
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络