Print this Page


A Review and Empirical Comparison of Approaches for Improving the Reliability of Objective Level Scores

This paper addresses the psychometric issues associated with estimating objective level scores, often referred to as subscores. The paper introduces the concepts of reliability and validity for subscores in the context of statewide achievement tests. Methods suggested in the literature for increasing the reliability of these subscore estimates are then reviewed and an empirical study comparing some of the more promising procedures is described. The study used test score data from four different statewide testing programs, representing three different subject areas and various testing conditions. The comparison of subscore augmentation approaches found that generally all methods were very successful in dramatically increasing the reliability of subscore estimates. However, this increase was accompanied by near-perfect correlations among the subscore estimates. This finding called into question the validity of the resultant subscores, and therefore the usefulness of the subscore augmentation process. Implications for practice are discussed.