Issues with Methods for Scoring Competitors in RoboCup Rescue

As part of the RoboCup yearly competition, participants in the RoboCup Rescue Simulation League compete to solve a simulated urban search and rescue problem. For the last couple of years there has been debate about what metrics are best for selecting the competition winners. We statistically evaluate a variety of the metrics used and propose new ones,with the objective of determining which has the most power in determining the winner and the final rankings of the competitors. We also suggest a variety of convenient changes to the competition that would help increase the accuracy of the rankings.