Statistical Model Selection

Statistical models are basic features of quantitative analysis and are commonly reported in academic articles and books to provide evidence of relationships and processes that are of importance in the population. Interpreting these models is, however, rarely straight forward as estimates for the model parameters and significance levels are dependent upon the methods used to build them, the so-called 'variable selection problem'. This chapter investigates how parameter estimates and significance values may be related to the methods used for their construction and uses simulated and real-world data to demonstrate that models may not always provide valid indications of 'actual' relationships and processes in the population. In particular, the use of subset-selection methods may enable variables that are not important in the population to enter into models simply due to chance. In addition to this, the validity of conclusions drawn from the research may be affected by the use of single-model selection, which forces analysts to report just one instance of what might be a multi-model solution. It is proposed that many published reports and papers (including undergraduate and postgraduate projects) interpret statistical models that are invalid as little or no consideration is taken as to how they were constructed or whether single-solution reporting is appropriate. This chapter aims to demonstrate these problems and proposes a solution in the form of a modelling procedure that uses a restricted-set of variables and a reporting method which enables multiple-models to be described.