Doctor of Philosophy (PhD), Ohio University, 2019, Educational Research and Evaluation (Education)

Placement testing in intensive English programs (IEPs) involves methodological considerations that merit additional research. Neither quantitative nor qualitative methods alone have always proven successful to address common research problems in the field of IEP student placement, including questions regarding the selection of placement tests and the establishment of cut scores for those tests by means of standard-setting methods. These two key questions, alongside the methodology underlying them, comprise three necessary supporting elements for the achievement of appropriate student placement. This dissertation uses a three-article format to examine, in turn, each of these necessary supporting elements and its connections to the other two. The first manuscript, “The Application of Mixed Methods for Developing Student Placement Protocols in Intensive English Programs” serves as the methodological focus for this dissertation, exploring the potential of Mixed Methods Research to address the common challenges of inconsistent placement criteria and small sample sizes in IEP student placement and to yield recommendations regarding the selection of placement tests and the setting of cut scores. The second manuscript, “Placement Testing: One Test, Two Tests, Three Tests? How Many Tests are Sufficient?,” then provides a quantitative analysis to address the question of placement test selection so as to maximize placement accuracy and includes consideration of logistical issues as well. The final manuscript, “Setting Cut Scores for Student Success,” addresses the issue of establishing appropriate cut scores and presents an approach to synthesizing the potentially divergent cut score results that different standard-setting methods can yield. Taken together, these three manuscripts provide a close examination of multiple supporting elements that are necessary for the achievement of appropriate student placement; these manuscripts also challenge practitioners in the field (open full item for complete abstract)

Committee: Krisanna Machtmes PhD (Advisor); Yuchun Zhou PhD (Committee Member); Lijing Yang PhD (Committee Member); Sara Helfrich PhD (Committee Member) Subjects: Education; Education Policy; Educational Evaluation; Educational Tests and Measurements; English As A Second Language; Higher Education; Higher Education Administration; Language; Statistics

Doctor of Philosophy (PhD), Ohio University, 2018, Educational Research and Evaluation (Education)

The purpose of this dissertation research study was to evaluate an adaptation of the Bookmark Method in order to establish new reading growth standards in Guatemala for the early elementary grades. For this Bookmark Method adaptation, data from the longitudinal test results collected by the USAID Lifelong Learning Project in the Western Highlands of Guatemala was used to vertically-link items of three existing NAC (Pruebas Nacionales de Lectura) reading tests of the Ministry of Education of Guatemala for first, second, and third grades. After vertically linking two grade levels of the test scores (first and second; second and third), an Ordered Item Booklet (OIB) was created that contained the items of two adjacent grade-level tests. Immediately afterwards, a standard Bookmark workshop was carried out with selected elementary teachers that used the above-described OIB to establish three new cut scores or growth standards to apply to four new levels of growth in reading. To establish the new growth standards, the cut-off points set by participating teachers during such workshops were applied to student scores in the grade level they were enrolled in for 2016. Such scores corresponded to the ones that had been linked to the previous grade level. Results of the adaptation were evaluated by using different validity sources of information. The first validity source was a correlation of results of the adaptation with a different growth model. Results from applying the adapted Bookmark Method yielded a classification similar to the classification found in the normative model Student Growth Percentile (SGP). The second validity source was to understand the overall experience of teachers in the adapted Bookmark workshop through interviews. Results from interviews showed teachers had a comfortable and engaging experience during the workshop. The third validity source was to estimate agreement between teacher panelists. Agreement among teacher panelists varied widely on the it (open full item for complete abstract)

Committee: Gordon Brooks (Advisor) Subjects: Education Policy; Educational Evaluation