Beyond Pre-Post Surveys: Exploring Validity Evidence For The Use Of Experience Sampling Methods To Measure Student Anxiety In Introductory Biology

Abstract

Classroom experiences elicit many emotions in students. Of these emotions, anxiety, or the feeling of uncertainty about a prospective event, has outsized impacts on students academic achievement and persistence. To understand the effects of student anxiety on student outcomes, education researchers often utilize single or dual-time point (e.g., pre-post) surveys. However, we posit that these methods have significant limitations, including the potential to miss variation in student emotions across time. One methodological solution to this problem is using experience sampling methods (ESM), or short, frequent surveys that capture student experiences as they occur. This study sought to provide evidence of validity for the use of ESM to measure student anxiety in Introductory Biology courses. We compared student anxiety data gathered via ESM and traditional survey methods (i.e., pre-post surveys) in terms of 1) differences in participant recruitment, 2) students experience of anxiety, and 3) the predictive ability on end-of-course outcomes. Our results revealed that: 1) Compared to non-ESM participants, the ESM group has significantly fewer men and had significantly higher perceptions of instructor support, 2) Average anxiety levels measured via ESM were lower and more variable than those measured via pre-post surveys, and 3) Post-survey measures of anxiety and average ESM measures of anxiety significantly predicted students final grade. We discuss the implications of these results for future applications of ESM and pre-post survey methods; specifically, we propose that ESM methods may be more advantageous than pre-post surveys in cases where researchers aim to understand intra-individual processes, like emotions.