Exploring value-added models for American higher education institutions and Omani Post-Basic Education

Internationally there has been an increasing emphasis on performance, quality, accountability and transparency surrounding the learning outcomes in K-12 and higher education. Value added (VA) modelling has been used to assess such performances in these contexts. This thesis considers eleven VA models commonly used in K-12 settings, of which only one has been applied in the Higher Education context. Through simulations, this thesis assesses the Type I error rate, bias, root mean square error (RMSE), and reliability of these models for two scenarios to identify the relative strengths of these models. These two scenarios considered the aspects of all institutions being equal, and where some differed. Moreover, Hox’s (2010) strategy is applied to analyse data arising in the US Higher Education and Omani Post-Basic Education contexts, which has not before been conducted; this included the adaptation of all models for the Omani context. Simulations identify student size (i.e., number of students) and school size (i.e., number of schools) interact significantly in all eleven VA models in both scenarios considered. For Scenario 1, the hierarchical VA models produce the lowest Type I error rate, bias and RMSE, and higher reliability compared to the other VA models, whilst the mean residual model produces a Type I error rate closest to nominal rate. For Scenario 2, a hierarchical VA model Type X produces a Type I error rate closest to the nominal rate and higher reliability compared to the other VA models, whilst a hierarchical VA model Type A produces the lowest Type I error rate compared to the other VA models. The analysis of the US Higher Education data set shows that a 2-level contextualised value-added (CVA) model Type X for students’ progress towards the Collegiate Learning Assessment (CLA) is statistically significant compared to CVA model Type B. Significantly, the international organisation who currently performs VA analyses of the US Higher Education context, namely, the American Council for Aid to Education (CAE), doesn’t use the hierarchical VA model Type X, but rather Type B. The results of this thesis will be shared with the CAE at a later date. The analysis of the Omani Post-Basic Education data using the modified models for this context shows that a 4-level CVA model Type X for students’progress towards English or Mathematics avoids the masking of classroom and region effects found in the traditional 2-level model. One of the key contributions of this research is that it fills a gap in the Omani Education system and its assessment by introducing the concept of VA models to the Ministry of Education in Sultanate of Oman. VA models enable richer assessment of school effectiveness, estimating the size and extent of school effects in Oman, and should be an additional mechanism in the suite of tools for assessing and improving the education system. The findings of the implementation conducted in this thesis provide valuable information for directing improvements to current accountability and education practices in Oman, including the recommendation that a 4-level CVA model Type X be used.