Doctor of Philosophy (PhD), Ohio University, 2005, Educational Research and Evaluation (Education)

MCAT scores and undergraduate GPA are two screening devices that are widely used by medical schools in the U.S. to select aspiring medical students. Given their widespread use, it is important that the validities of MCAT scores and undergraduate GPA be investigated to ensure the accuracy of medical school admission decisions. This study was therefore designed to address three important aspects of validation, namely, predictive validity of MCAT subtest scores and undergraduate GPA, differential validity, and differential prediction of the MCAT subtest scores. First, predictive validity was evaluated as an index of the relationship between the predictors, MCAT subtest scores and undergraduate GPA, and the criterion, first-year medical school GPA. Second, differential validity was assessed by comparing the magnitude of validity coefficients obtained as the correlation between first-year medical school GPA and MCAT subtest scores for men and women and for White, Black, Asian, and Hispanic medical students. Third, differential prediction was examined by testing for differences in the regression systems obtained for the different subgroups of examinees. The sample used in the study consisted of 3,187 students drawn from 1992 and 1993 cohorts of 14 medical schools. Statistical procedures utilized in the research included regression analysis, Fisher's z transformations, F-ratio test of equality of standard errors of estimate, and ANCOVA tests of equality of regression slopes and intercepts. Results obtained showed moderately high correlations between the predictors and the criterion. MCAT subtest scores and previous grades were individually good predictors of medical school freshman grades. The combination of MCAT subtest scores and undergraduate GPA was, however, a more powerful indicator of performance in the first-year of medical school. Differential validity results showed that in most cases women had higher validity coefficients compared to men. With regards to differe (open full item for complete abstract)

Committee: George Johanson (Advisor) Subjects: Education, Tests and Measurements

Doctor of Philosophy (PhD), Ohio University, 2004, Educational Research and Evaluation (Education)

The focus of this study was to evaluate the effect of different approaches of anchor test construction on the accuracy of equating for test adaptation. The term “equating” in cross-lingual studies refers to a statistical procedure that adjusts test scores from the source language (SL) version of the test and the target language (TL) version of the test using a set of common translated items of the same examination so that scores can be interpreted interchangeably. In each test, the verbal section and the non-verbal section of the test were investigated. The Levine Linear equating method and Mean-Sigma equating method were utilized with an anchor item design and an equivalent group design, respectively. The double linking method and the standard errors of equating method were used to evaluate the accuracy of the equating for different anchor tests. The average difference between the two anchor tests for the verbal and non-verbal sections of the test over three target language groups reflected the degree of overall instability that existed in the cross-lingual equating process. These differences were associated with real and systematic variance that underlies the cross-lingual equating process. Scoring outcomes of an actual certification examination with a sample of nearly 9,000 examinees taking both SL and TL versions of the test data set were utilized for this research study. Findings indicated that the differences between the double linking chains for each anchor test were greater for the verbal section than the non-verbal section of the test. The results of the double linking method supported the notion that different choices for anchor items can result in different equatings and using items with the more stable parameters was a better choice than using items with less DIF. The results of MSEE did not show large differences between the parameter and the DIF methods of anchor item selection. However, the MSEE differences were in the same direction as the double lin (open full item for complete abstract)

Committee: George Johanson (Advisor) Subjects: