Skip to Main Content
 

Global Search Box

 
 
 
 

Files

File List


Dissertation_PSoto_062920.pdf (3.27 MB)

ETD Abstract Container

Abstract Header

Validity Evidence of Internal Structure and Subscores Use of the Portfolio in the Chilean Teachers’ Evaluation System

Soto Ramirez, Pamela
http://rave.ohiolink.edu/etdc/view?acc_num=osu159316412299089

Abstract Details

2020, Doctor of Philosophy, Ohio State University, Educational Studies.
There is consensus around the fact that quality of teaching is one of the most important school-level factors to influence student achievement at school. Evidence from research suggests that better-qualified teachers can be the determining factor for student achievement and development (Jordan et al., 1997; Sanders & Rivers, 1996; Wright et al., 1997). Therefore, policy makers advocate for ongoing improvements in teacher quality variables, in which the implementation of a well-designed teacher evaluation system has been found to be one of the most effective ways to improve teacher quality (Darling-Hammond, 2010; Looney, 2011; Rockoff & Speroni, 2011). The case of Chile is a particularly interesting example of a teacher evaluation system since its implementation, a validation process that has included not only the experience and documentation regarding the process, but also a comprehensive agenda regarding the validity and reliability of the instrument and evaluation consequences (Taut & Sun, 2014). The purpose of this dissertation is to contribute to the body of research on the evidence of the validity of Chilean teacher evaluation. Specifically, I focus on one of the teacher evaluation instruments: the portfolio. Through the portfolio, teachers provide evidence of their best practices in three modules: a set of pedagogical materials, video recording class, and collaborative work (not mandatory). In order to accomplish this goal, I use the data from the portfolio results of the 2017 Chilean National Teacher Evaluation (N = 21,982). I use descriptive statistics, exploratory confirmatory factor analysis (ECFA), and factorial invariance to assess the structure of the portfolio across different teacher subgroups: teachers with and without the collaborative work module score, rural/urban teachers, and six different teaching levels. I also compare the theoretical weight assigned to each one of the portfolio indicators with the empirical data. Finally, I evaluate if the portfolio subscores for the different modules have added value over the total portfolio score reported. For the analysis, Stata v.14 and Mplus 8 are used. Four main findings for portfolio validity were found. First, results from the ECFA indicate the portfolio’s theoretical structure fits the data well for the groups of teachers without collaborative work results. Conversely, for the teachers with the results from the collaborative work module, the data did not clearly support the theoretical dimensions proposed by the portfolio. The second finding was related to the portfolio structure invariance across teachers, depending on their school location and teaching level. The results showed a strong factorial invariance (invariant thresholds) across rural and urban teachers, and weak factorial invariance (factor loadings) across the six groups of teaching levels. The third finding showed differences between the theoretical weight assigned to each portfolio indicator and the empirical weight. Finally, the fourth finding showed that at least two of the three modules of the portfolio had added value over the total score. These findings provide evidence of portfolio validity, indicating that the instrument is invariant for teachers working in different contexts. However, a revision of the collaborative work module and the theoretical weights is suggested.
Jerome D’Agostino (Advisor)
Jessica Logan (Committee Member)
Shayne Piasta (Committee Member)
211 p.
Education; Education Policy; Educational Evaluation; Educational Tests and Measurements
Teaching Quality; Teacher Evaluation System; Validity; Exploratory Confirmatory Factor Analysis; Measurement Invariance

Recommended Citations

Citations

  • Soto Ramirez, P. (2020). Validity Evidence of Internal Structure and Subscores Use of the Portfolio in the Chilean Teachers’ Evaluation System [Doctoral dissertation, Ohio State University]. OhioLINK Electronic Theses and Dissertations Center. http://rave.ohiolink.edu/etdc/view?acc_num=osu159316412299089

    APA Style (7th edition)

  • Soto Ramirez, Pamela. Validity Evidence of Internal Structure and Subscores Use of the Portfolio in the Chilean Teachers’ Evaluation System. 2020. Ohio State University, Doctoral dissertation. OhioLINK Electronic Theses and Dissertations Center, http://rave.ohiolink.edu/etdc/view?acc_num=osu159316412299089.

    MLA Style (8th edition)

  • Soto Ramirez, Pamela. "Validity Evidence of Internal Structure and Subscores Use of the Portfolio in the Chilean Teachers’ Evaluation System." Doctoral dissertation, Ohio State University, 2020. http://rave.ohiolink.edu/etdc/view?acc_num=osu159316412299089

    Chicago Manual of Style (17th edition)

osu159316412299089
219
© 2020, all rights reserved.
This open access ETD is published by The Ohio State University and OhioLINK.