Reinforcement learning for acoustic metamaterial design

This talk presents our findings and methodologies in applying reinforcement learning to the design of acoustic metamaterials [1]. Our reinforcement learning agents are capable of discovering configurations of cylindrical scatterers in water which minimize the scattering of an acoustic plane wave. These agents are successful in the optimization of two different parametric designs, i.e., radii and positions of each scatterer. The resultant designs produced by reinforcement learning algorithms such as double deep Q-learning network and deep deterministic policy gradient algorithms are comparable and in some cases superior to those produced by the gradient-based optimization solver such as fmincon. However, significant computational resources are required to train reinforcement learning models to completion. This poses a significant challenge when attempting to increase the complexity of a design; as training times increase to unfeasible levels. Methods to counteract this challenge are discussed in this talk. These methods include utilization of the Julia programming language as well as multiprocessing and multithreaded programming. Together they create a synergy which reduces training times by orders of magnitude. [1] T. Shah, L. Zhuo, P. Lai, A. Rosa-Moreno, F. Amirkulova, and P. Gerstoft, “Reinforcement learning applied to metamaterial design,” J. Acoust. Soc. Am. 150(1), 321–338 (2021).