Performance comparison of different HTM-spatial pooler algorithms based on information-theoretic measures

Abstract This paper provided an information-theoretic framework for the performance comparison of different types of HTM-Spatial Pooler (SP) algorithms for the first time. There are two primary goals. The first goal is to measure SP's performance as a standalone component, and the second goal is to compute the SP performance on the whole performance of the HTM algorithm. For this purpose, four different SPs were introduced by making changes in each of the four parts of the SP algorithm. The SP algorithm was considered a black box. Three information-theoretic measures, i.e., Renyi mutual information, Renyi divergence, and Henze-Penrose divergence, were proposed to determine the similarities and differences between the input and output of the SP algorithm. So the accuracy of each method was computed by these information-theoretic measures. This paper was able to quantify the similarities and differences between the two SDRs in the SP algorithm, unlike previous papers that do experimentally. Then, the results were compared with the Overlap Score measure that was previously introduced and found that the results of the new measures were compatible with the previous one. Four different datasets: MNIST, Fashion-MNIST, Hotgym, and NYC-Taxi, were used to perform experiments. The performance of each SP algorithm in the whole HTM system was examined and analyzed using the accuracy and error percentage measures. Finally, it concluded that the best SP algorithm's efficiency leads to the best HTM algorithm.