Combining factor analysis with writing analytics for the formative assessment of written reflection
Computers in Human Behavior, (2021)DOI: https://doi.org/10.1016/j.chb.2021.106733
Abstract
The formative assessment of written reflection provides opportunities for students to improve their practice in an iterative manner using reflective writing. However, manual formative assessment of written reflection is time consuming and subjective. While progress has been made in deploying writing analytics tools to provide automated, formative feedback, few approaches to automated assessment are grounded in a validated, theory-based, formative assessment model. To address this, we propose a five-factor model of the Capability for Written Reflection (CWRef), grounded in the scholarship of reflective writing pedagogy. This paper uses Confirmatory Factor Analysis to validate the CWRef model by examining the relative contributions of textual features, derived from writing analytics, to each factor in the model, and their contributions to CWRef. The model was evaluated with two reflective writing corpora, showing which textual features, derived using Academic Writing Analytics and Linguistic Inquiry & Word Count, were significant indicators of factors in both corpora. In addition, it was found that the reflective writing context was an important factor influencing the validity of the CWRef model. Finally, we consider how this new analytical assessment model could enable improved tracking of progression in reflective writing, providing the basis for improved formative feedback.