Deep q-learning for simultaneous beam orientation and trajectory optimization for cyberknife

In this paper, we propose a Deep Reinforcement Learning algorithm to find the best beam orientations for radiosurgery treatment planning and particularly the Cyberknife system. We present a Deep Q-learning algorithm to find a subset of the beams and the order to traverse them. A reward function is defined to minimize the distance covered by the robotic arm while avoiding the selection of close beams. Individual beam scores are also generated based on their effect on the beam intensity and are incorporated in the reward function. The algorithm and the quality of the treatment plan are evaluated on three clinical lung case patients. Computational results show a reduction in the treatment time while maintaining the quality of the treatment in comparison with the plan using all of the beams. This results in a more comfortable treatment for the patients and creates the opportunity to treat a higher number of patients in the clinics.