Discovering Feature Weights for Feature-based Indexing of Q-tables

In this paper we propose an approach to address the old problem of identifying the feature conditions under which a gaming strat- egy can be eective. For doing this, we will build on previous work on CBRetaliate, a system that combines case-based reasoning and reinforce- ment learning to play team-based First Person Shooter Games. In CBRe- taliate, cases are pairs (features, Q-table), where the Q-table associates a utility with each state-action pair, which is used to select an appropri- ate action in a given state. CBRetaliate learns cases as it plays against opponents. We propose to cluster cases in the case-base using a novel denition of similarity between their Q-tables; cases will be grouped in the same cluster if they have similar Q-tables. We propose to use stan- dard information gain formulas and use the clusters as the classication to assign feature weights. We expect that this approach would lead to identifying features that are crucial to select which Q-table to reuse in a given situation. In addition, we propose to use the same notions of Q- table similarity to nd substrategies that are common to every or nearly every case in the case base.