Skip to main content Accessibility help

We use cookies to distinguish you from other users and to provide you with a better experience on our websites. Close this message to accept cookies or find out how to manage your cookie settings.

Close cookie message
×
Hostname: page-component-78c5997874-xbtfd Total loading time: 0 Render date: 2024-11-05T09:45:32.216Z Has data issue: false hasContentIssue false

27 - Addressing the Challenge of Measuring Student Engagement

from Part VI - Methods, Measures, and Perspective

Published online by Cambridge University Press:  15 February 2019

K. Ann Renninger  and
Suzanne E. Hidi
By
Jennifer A. Fredricks ,
Tara L. Hofkens  and
Ming-Te Wang
K. Ann Renninger
Affiliation:
Swarthmore College, Pennsylvania
Suzanne E. Hidi
Affiliation:
University of Toronto
Get access

Summary

Student engagement is a relatively new construct that describes concepts as varied as classroom behaviors, emotional reactions, motivational beliefs, self-regulatory processes, metacognitive strategies, school belonging and interactions with instructional materials. This chapter reviews a variety of methods to measure student engagement including self-report surveys, teacher ratings, interviews, administrative data, observations, experience sampling methods, and real-time measures. The authors outline the strengths and limitations of each method. Next, we present two examples from our own research on approaches to measuring engagement. The goal of these cases is to illustrate how we have addressed some of the challenges with measurement, as well as showing the importance of choosing a measurement technique that aligns with the research questions. First, we describe the results of a qualitative study to develop a new subject-specific measure of engagement. Next, information on the predictive validity of an observational measure to assess engagement at the class-level is presented. The chapter concludes with a discussion of measurement limitations, future directions, and implications for policy and practice.

Type
Chapter
Information
The Cambridge Handbook of Motivation and Learning , pp. 689 - 712
Publisher: Cambridge University Press
Print publication year: 2019

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

References

Antonenko, P., Paas, F., Grabner, R. & van Gog, T. (2010). Using electroencephalography to measure cognitive load. Educational Psychology Review, 22, 425–38. doi: 10.1007/s10648-010-9130-y.CrossRefGoogle Scholar
Appleton, J. J. (2012). Systems consultation: Developing the assessment-to-intervention link with the student engagement instrument. In Christenson, S., Reschly, A., & Wylie, C. (Eds.), Handbook of research on student engagement (pp. 725–42). New York, NY: Springer.Google Scholar
Appleton, J. J., Christenson, S. L., & Furlong, M. J. (2008). Student engagement with school: Critical conceptual and methodological issues of the construct. Psychology in the Schools, 45, 369–86. doi: 10.1002/pits.20303.Google Scholar
Arroyo, I., Cooper, D. G., Burleson, W., Woolf, B. P., Muldner, K., & Christopherson, R. (2009). Emotion sensors go to school. Conference on Artificial Intelligence in Education, 200, 17–24.Google Scholar
Azevedo, R. (2015). Defining and measuring engagement and learning in science: Conceptual, theoretical, methodological, and analytical issues. Educational Psychologist, 50, 84–94. doi: 10.1080/00461520.2015.1004069.Google Scholar
Azevedo, R., Moos, D., Johnson, A., & Chauncey, A. (2010). Measuring cognitive and metacognitive regulatory processes used during hypermedia learning: Issues and challenges. Educational Psychologist, 45, 210–23. doi: 10.1080/00461520.2010.515934.CrossRefGoogle Scholar
Balfanz, R., Herzog, L., & MacIver, P. J. (2007). Preventing student disengagement and keeping students on graduation path in urban middle grade schools: Early identification and effective interventions. Educational Psychologist, 42, 223–35. doi: 10.1080/00461520701621079.Google Scholar
Blumenfeld, P., Modell, J., Bartko, W. T., Secada, W., Fredricks, J., Friedel, J., & Paris, A. (2005). School engagement of inner city students during middle childhood. In Cooper, C. R., Garcia Coll, C., Bartko, W. T., Davis, H. M. & Chatman, C. (Eds.), Developmental pathways through middle childhood: Rethinking diversity and contexts as resources (pp. 145–70). Mahwah, NJ: Lawrence Erlbaum.Google Scholar
Boucheix, J. M., Lowe, R. K., Putri, D. K., & Groff, J. (2013). Cueing animations: Dynamic signaling aids information extraction and comprehension. Learning and Instruction, 25, 71–84. doi: 10.1016/j.learninstruc.2012.11.005.Google Scholar
Briesch, A. M., Hemphill, E. M., Volpe, R. J., & Daniels, B. (2015). An evaluation of observational methods for measuring response to class-wide intervention. School Psychology Quarterly, 30, 37–49. doi: 10.1037/spq0000065.CrossRefGoogle Scholar
Christenson, S., Reschly, A., & Wylie, C. (Eds.). (2012). Handbook of research on student engagement. New York, NY: Springer.Google Scholar
Conchas, G. Q. (2001). Structuring failure and success: Understanding the variability in Latino school engagement. Harvard Educational Review, 71, 475–504. doi: 10.17763/haer.71.3.280w814v1603473k.Google Scholar
Connell, J. P., Klem, A., Lacher, T., Leiderman, S., & Moore, W. (2009). First things first: Theory, research, and practice. Howell, NJ: Institute for Research and Reform in Education.Google Scholar
D'Mello, S. & Graesser, A. (2012). Dynamics of affective states during complex learning. Learning and Instruction, 22, 145–57. doi: 10.1016/j.learninstruc.2011.10.001.Google Scholar
Duchowski, A. (2007). Eye tracking methodology: Theory and practice (2nd ed.). New York, NY: Springer.Google Scholar
Eccles, J. S. & Wigfield, A. (2002). Motivational beliefs, values, and goals. Annual Review of Psychology, 53, 109–32. doi: 10.1146/annurev.psych.53.100901.135153.Google Scholar
Engle, R. A. & Conant, F. R. (2002). Guiding principles for fostering productive disciplinary engagement: Explaining an emergent argument in a community of learners’ classroom. Cognition and Instruction, 20, 399–483. doi: 10.1207/S1532690XCI2004_1.CrossRefGoogle Scholar
Ferguson, R. F. & Danielson, C. (2014). How framework for teaching and tripod 7Cs evidence distinguish key components of effective teaching. Designing Teacher Evaluation Systems, 98–143.Google Scholar
Finn, J. D. (1989). Withdrawing from school. Review of Educational Research, 59, 117–42. doi: 10.3102/00346543059002117.Google Scholar
Fredricks, J. A., Blumenfeld, P. C. & Paris, A. (2004). School engagement: Potential of the concept: State of the evidence. Review of Educational Research, 74, 59–119. doi: 10.3102/00346543074001059.Google Scholar
Fredricks, J. A. & McColskey, W. (2012). The measurement of student engagement: A comparative analysis of various methods and student self-report instruments. In Christenson, S., Reschy, A. L., & Wylie, C. (Eds.), Handbook of research on student engagement (pp. 763–83). New York, NY: Springer.Google Scholar
Fredricks, J., McColskey, W., Meli, J., Mordica, J., Montrosse, B., & Mooney, K. (2011). Measuring student engagement in upper elementary through high school: A description of 21 instruments. (Issues & Answers Report, REL 2010–No. 098). Washington, DC: US Department of Education, Institute of Education Sciences, National Center for Education. Available at http://ies.ed.gov/ncee/edlabs/projects/project.asp?projectID=268.Google Scholar
Fredricks, J. A., Wang, M. T., Schall, J., Hofkens, T., Snug, H., Parr, A., & Allerton, J. (2016). Using qualitative methods to develop a survey of math and science engagement. Learning and Instruction, 43, 5–15. doi: 10.1016/j.learninstruc.2016.01.009.Google Scholar
Gelhbach, H. & Brinkworth, M. E. (2011). Measure twice: cut down error: A process for enhancing the validity of survey scales. Review of General Psychology, 15, 380–7. doi: 10.1037/a0025704.Google Scholar
Gobert, J. D., Baker, R. S., & Wixon, M. B. (2015). Operationalizing and detecting disengagement within online science microworlds. Educational Psychologist, 50, 43–57. doi: 10.1080/00461520.2014.999919.CrossRefGoogle Scholar
Greene, B. (2015). Measuring cognitive engagement with self-report scales: Reflections from over 20 years of research. Educational Psychologist, 50, 13–40. doi: 10.1080/00461520.2014.989230.Google Scholar
Greene, J. A. & Azevedo, R. (2010). The measurement of learners’ self-regulated cognitive and metacognitive processes while using computer-based learning environments. Educational Psychologist, 45, 203–9. doi: 10.1080/ 00461520.2014.989230.Google Scholar
Gresalfi, M. S. (2009). Taking up opportunities to learn: Constructing dispositions in mathematics classrooms. Journal of the Learning Sciences, 18, 327–69. doi: 10.1080/10508400903013470.Google Scholar
Grossman, P., Loeb, S., Cohen, J. & Wyckoff, J. (2013). Measure for measure: The relationship between measures of instructional practice in middle school English language arts and teachers' value-added scores. American Journal of Education, 50, 4–36. doi: 10.1086/669901.Google Scholar
Hektner, J. M., Schmidt, J. A., & Csikzentmihalyi, M. (2007). Experience sampling method: Measuring the quality of everyday life. Thousand Oaks, CA: Sage Publications.CrossRefGoogle Scholar
Henrie, C. R., Halverson, L. R., & Graham, C. R. (2015). Measuring student engagement in technology-mediated learning: A review. Computers and Education, 90, 36–53. doi: 10.1016/j.compedu.2015.09.005.Google Scholar
Johnson, R. W. & Onwuegbuzie, A. J. (2004). Mixed methods research: A research paradigm whose time has come. Educational Researcher, 33, 14–26. doi: 10.3102/0013189X033007014.Google Scholar
Kane, T., Kerr, K., & Pianta, R. (2014). Designing teacher evaluation systems: New guidance from the measures of effective teaching project. New York, NY: John Wiley & Sons.Google Scholar
Kapoor, A., Burleson, W., & Picard, R. W. (2007). Automatic prediction of failure. International Journal of Human Computer Studies, 65, 724–6. doi: 10.1016/j.ijhcs.2007.02.003.Google Scholar
Lam, S., Wong, B. P. H., Yang, H. & Liu, Y. (2012). Understanding student engagement with a contextual model. In Christenson, S., Reschy, A. L., & Wylie, C. (Eds.), Handbook of research on student engagement (pp. 403–19). New York, NY: Springer.Google Scholar
Lane, K. L., Menzies, H. M, Oakes, W. P., & Kalberg, J. R. (2012). Systematic screenings of behavior to support instruction: From preschool to high school. New York, NY: Guilford Press.Google Scholar
Larson, R. W. (2000). Toward a psychology of positive youth development. American Psychologist, 55, 170–83. doi: 10.1037/0003-066X.55.1.170.Google Scholar
Larson, R. W. & Kleiber, D. (1993). Daily experiences of adolescents. In Tolan, P. H. & Cohler, B. J. (Eds.), Handbook of clinical research and practice with adolescents (pp. 125–45). Oxford: John Wiley.Google Scholar
Mandernach, J. (2015). Assessment of student engagement in higher education: A synthesis of literature and assessment tools. International Journal of Learning, Teaching, and Educational Research, 12, 1–14.Google Scholar
Mason, B., Gunersel, A. B., & Ney, E. (2014). Cultural and ethnic bias in teacher ratings of behavior: A criterion-focused review. Psychology in the Schools, 51, 1017–30. doi: 10.1002/pits.21800.Google Scholar
McNeal, K. S., Spry, J. M., Mitra, R., & Tipton, J. L. (2014). Measuring student engagement, knowledge, and perceptions of climate change in an introductory environment geology course. Journal of Geoscience Education, 62, 655–67. doi: 10.5408/13-111.1.CrossRefGoogle Scholar
Meece, J., Blumenfeld, P. C., & Hoyle, R. H. (1988). Students’ goal orientation and cognitive engagement in classroom activities. Journal of Educational Psychology, 80, 514–23. doi: 10.1037/0022-0663.80.4.514.Google Scholar
Miller, B. W. (2015). Using reading times and eye-movements to measure cognitive engagement. Educational Psychologist, 50, 31–42. doi: 10.1080/00461 520.2015.1004068.Google Scholar
Mostow, J., Chang, K. M., & Nelson, J. (2011, June). Toward exploiting EEG input in a reading tutor. In International conference on artificial intelligence in education (pp. 230–7). Berlin and Heidelberg: Springer.Google Scholar
Muthén, L. K. & Muthén, B. O. (1998–2014). Mplus user's guide. Seventh edition. Los Angeles, CA: Muthén & Muthén.Google Scholar
Nystrand, M. & Gamoran, A. (1991). Instructional discourse, student engagement, and literature achievement. Research in the Teaching of English, 25, 261–90.Google Scholar
Pianta, R. C., Hamre, B. K., Haynes, N. J., Mintz, S. L., & La Paro, K. M. (2007). Classroom Assessment Scoring System Manual, Middle/Secondary Version. Charlottesville, NC: University of Virginia Press.Google Scholar
Pianta, R. C., Hamre, B. K., & Mintz, S. L. (2012). Classroom Assessment Scoring System (CLASS): Secondary class manual. Charlottesville, VA: Teachstone.Google Scholar
Poh, M., Swenson, N. C., & Picard, R. W. (2010). A wearable sensor for unobstrusive, long-term assessment of electrodermal activity. IEE Transactions on Biomedical Engineering, 57, 1243–57.Google Scholar
Ponitz, C. C., Rimm-Kaufman, S. E., Grimm, K. J., & Curby, T. W. (2009). Kindergarten classroom quality, behavioral engagement, and reading achievement. School Psychology Review, 38, 102–20.Google Scholar
Reeve, J. M. & Tseng, C. (2011). Agency as a fourth aspect of students’ engagement with learning activities. Contemporary Educational Psychology, 36, 357–67. doi: 10.1016/j.cedpsych.2011.05.002.Google Scholar
Renninger, K. A. & Bachrach, J. E. (2015). Studying triggers for interest and engagement using observational methods. Educational Psychologist, 50, 58–69. doi: 10.1080/00461520.2014.999920.Google Scholar
Renninger, K. A. & Hidi, S. (2016). The power of interest for motivation and learning. New York, NY: Routledge.Google Scholar
Rimm-Kaufman, S. E., Baroody, A. E., Larsen, R. A., Curby, T. W., & Abruy, T. (2015). To what extent do teacher-student interaction quality and student gender contribution to fifth graders’ engagement in mathematics learning? Journal of Educational Psychology, 107, 17–185. doi: 10.1037/a0037252.Google Scholar
Rimm-Kaufman, S. E., Curby, T. W., Grimm, K. J., Nathanson, L., & Brock, L. (2009). The contribution of children's self-regulation and classroom quality to children's adaptive behaviors in the kindergarten classroom. Developmental Psychology, 45, 958–72. doi: 10.1037/a0015861.Google Scholar
Ryu, S. & Lombardi, D. (2015). Coding classroom interactions for collective and individual engagement. Educational Psychologist, 50, 70–83. doi: 10.1080/00461520.2014.1001891.Google Scholar
Shen, L., Wang, M., & Shen, R. (2009). Affective e-Learning: Using “emotional” data to improve learning in pervasive learning environment. Educational Technology & Society, 12(2), 176–89.Google Scholar
Shernoff, D. J. & Csikszentmihalyi, M. (2009). Flow in schools: Cultivating engaged learners and optimal learning environments. In Gilman, R., Huebner, E. S., & Furlong, M. (Eds.), Handbook of positive psychology in schools (pp. 131–45). New York, NY: Routledge.Google Scholar
Shernoff, D. J., Csikzentmihalyi, M., Schneider, B., & Shernoff, E. S. (2003). Student engagement in high schools from the perspective of flow theory. School Psychology Quarterly, 18, 158–76. doi: 10.1521/scpq.18.2.158.21860.Google Scholar
Sinatra, G., Heddy, B. C., & Lombard, D. (2015). The challenge of defining and measuring student engagement in science. Educational Psychologist, 1, 1–13. doi: 10.1080/00461520.2014.1002924.Google Scholar
Skiba, R. J., Michael, R. S., Nardo, A. C., & Peterson, R. L. (2002). The color of discipline: Sources of racial and gender disproportionality in school punishment. The Urban Review, 34, 317–42. doi: 10.1023/A:1021320817372.Google Scholar
Skinner, E. A., Kindermann, T. A., & Furrer, C. J. (2009). A motivational perspective on engagement and disaffection. Conceptualization and assessment of children's behavioral and emotional participation in academic activities in the classroom. Educational and Psychological Measurement, 69, 493–525. doi: 10.1177/0013164408323233.Google Scholar
Skinner, E. A. & Pitzer, J. R. (2012). Developmental dynamics of student engagement, coping, and everyday resilience. In Christenson, S., Reschy, A. L., & Wylie, C. (Eds.), Handbook of research on student engagement (pp. 21–45). New York, NY: Springer.Google Scholar
Stevens, R. H., Galloway, T. L., Berka, C., Johnson, R., & Sprang, M. (2008). Assessing student's mental representations of complex problem spaces with EEG technologies. In Proceedings of the Human Factors and Ergonomics Society Annual Meeting (Vol. 52, no. 3, pp. 167–71). Thousand Oaks, CA: SAGE Publications.Google Scholar
Tolan, P. H. & Deutsch, N. L. (2015). Mixed methods in developmental science. In Lerner, R. (Ed.), Handbook of child psychology and developmental science (Vol. 1, 7th ed., 1–45), Hoboken, NJ: Wiley.Google Scholar
Turner, J. C. & Meyer, D. K. (2000). Studying and understanding the instructional context of classroom: Using our past to forge our future. Educational Psychologist, 35, 69–85. doi: 10.1207/S15326985EP3502_2.Google Scholar
Uekawa, K., Borman, K., & Lee, R. (2007). Student engagement in the U.S. urban high school mathematics and science classrooms: Findings on social organization, race, and ethnicity. Urban Review, 39, 1–43. doi: 10.1007/s11256-006-0039-1.Google Scholar
Valentine, J. (2005). The instructional practices inventory: A process for profiling student engaged learning for school improvement. Columbia, MO: University of Missouri, Middle Level Leadership Center. Retrieved at http://mllc.missouri.edu/Upload%20Area-Docs/IPI%20Manuscript%2012-07.pdf.Google Scholar
Voelkl, K. E. (1997). Identification with school. American Journal of Education, 105, 204–319. doi: 10.1086/444158.CrossRefGoogle Scholar
Volpe, R. J., DiPerna, J. C., Hintze, J. M., & Shapiro, E. S. (2005). Observing students in classroom settings: A review of seven coding schemes. School Psychology Review, 34(4), 454–74.Google Scholar
Walker, H. & Severson, H. (1992). Systematic Screening for Behavioral Disorders (SSBD). (2nd ed.). Technical Manual. Longmont, CA: Sopris West.Google Scholar
Wang, M. T., Chow, A., Hofkens, T., & Salmela-Aro, K. (2015). The trajectories of student emotional engagement and school burnout with academic and psychological development: Findings from Finnish adolescents. Learning and Instruction, 36, 57–65. doi: 10.1016/j.learninstruc.2014.11.004.Google Scholar
Wang, M. T. & Degol, J. (2014). Staying engaged: Knowledge and research needs in student engagement. Child Development Perspectives, 8, 137–43. doi: 10.1111/cdep.12073.CrossRefGoogle ScholarPubMed
Wang, M. T. & Fredricks, J. A. (2014). The reciprocal links between school engagement and youth problem behavior during adolescence. Child Development, 85, 722–37. doi: 10.1111/cdev.12138.Google Scholar
Wang, M. T., Fredricks, J. A., Ye, F., Hofkens, T., & Schall, J. (2016). The math science engagement scale: Development, validation, and psychometric properties. Learning and Instruction, 43, 16–26. doi: 10.1016/j.learninstruc.2016.01.008.Google Scholar
Wang, M. T. & Peck, S. C. (2013). Adolescent educational success and mental health vary across school engagement profiles. Developmental Psychology, 49, 1266–76. doi: 10.1037/a0030028.Google Scholar
Waxman, H. C., Tharp, R. G., & Hilberg, R. S. (2004). Future directions for classroom observation research. In Waxman, H. C., Hilberg, R. S., & Tharp, R. G. (Eds.), Observational research in U.S. classrooms: New approaches for understanding cultural and linguistic diversity (pp. 266–77). Cambridge: Cambridge University Press.Google Scholar
Wigfield, A., Guthrie, J. T., Perencevich, K. C., Taboada, A., Klauda, S. L., McRae, A., & Barbosa, P. (2008). Role of reading engagement in mediating the effects of reading comprehension instruction on reading outcomes. Psychology in the Schools, 45, 432–45. doi: 10.1002/pits.20307.Google Scholar
Winne, P. H. & Perry, N. E. (2000). Measuring self-regulated learning. In Boekaerts, M., Pintrich, P., & Zeidner, M. (Eds.), Handbook of self-regulation (pp. 531–66). San Diego, CA: Academic Press.Google Scholar
Wood, B. K., Hojnoski, R. L., Laracy, S. D., & Olson, C. L. (2016). Comparison of observational methods and their relation to ratings of engagement in young children. Topics in Early Childhood Special Education, 35(4), 211. doi: 10.1177/0271121414565911.Google Scholar
Yair, G. (2000). Educational battlefields in America: The tug of war over students’ engagement with instruction. Sociology of Education, 73, 247–69. doi: 10.2307/2673233.Google Scholar

Save book to Kindle

To save this book to your Kindle, first ensure [email protected] is added to your Approved Personal Document E-mail List under your Personal Document Settings on the Manage Your Content and Devices page of your Amazon account. Then enter the ‘name’ part of your Kindle email address below. Find out more about saving to your Kindle.

Note you can select to save to either the @free.kindle.com or @kindle.com variations. ‘@free.kindle.com’ emails are free but can only be saved to your device when it is connected to wi-fi. ‘@kindle.com’ emails can be delivered even when you are not connected to wi-fi, but note that service fees apply.

Find out more about the Kindle Personal Document Service.

Available formats
×

Save book to Dropbox

To save content items to your account, please confirm that you agree to abide by our usage policies. If this is the first time you use this feature, you will be asked to authorise Cambridge Core to connect with your account. Find out more about saving content to Dropbox.

Available formats
×

Save book to Google Drive

To save content items to your account, please confirm that you agree to abide by our usage policies. If this is the first time you use this feature, you will be asked to authorise Cambridge Core to connect with your account. Find out more about saving content to Google Drive.

Available formats
×