I just finished teaching my last molecular cell biology class for the fall term and I am feeling out of sorts. Out of sorts because I don't know what to do with myself. For three months my energy has been focused on preparing in-class activities and reading quizzes for each and every class meeting. Team-Based Learning, a highly structured version of the flipped classroom, requires preparation for every class. Activities and questions that worked last year, don't necessarily work for the subsequent cohort of students. So for the last three months, I have been probing my students trying to get a sense of what they do and do not know so that I can provide learning experiences to address their knowledge and skill gap and trying to hold a mirror up to them so that students know what they do and do not know.
Now the class is over, the students are completing the end of term student rating of instruction. And I am at a loss over what to do with myself.
Yes, I have service committees that need my attention, I have papers that need writing and revising, I have reviews that need to be typed. But those were always completed in the background while teaching has been foremost in my mind. Now that is over for the term, I feel disoriented - I no longer have the lodestone of instructing my students to set the priorities for my day. Can I actually take the time to read that article, to browse that recent journal issue, to walk down the hall and say hello to my colleague?
Strangely, I feel guilty about even taking the time to consider going for a coffee with a colleague. This is how all-encompassing the teaching term is for me.
It didn't always use to be this way.
Certainly, when I first started teaching I was always running to stand still. But after a few years and being successfully tenured, the pace became reasonable. I was able to tweak and re-use previous lectures and received great student reviews for my efforts.
Then, about eight years ago, I became bored with the sound of my own voice and experienced the revelation that good lecturing doesn't necessarily equate with good learning. In 2012, I began experimenting with implementing active learning in my classes culminating with most of my classes being reworked with team-based learning. And then the realization inherent with learner-centred teaching that each cohort of students is different; that to be able to meet the needs of each student cohort, each student, required that I continually formatively assess my students in order to understand and meet their learning needs.
That is difficult exhausting work. But such rewarding work! After my last class today, a few students came up and thanked me for my efforts. One student expressed gratitude for my class structure that facilitated their ability to learn from the assigned readings. The reading guides are doing their work. My preparation efforts were well-received.
So now I am sitting in my office wondering what to do with myself. I have decided to sit quietly for a moment and enjoy the fruits of my labour: student learning and gratitude. The teaching and learning experience is fleeting. Just allow me a few minutes to enjoy it before it fades into the ether.
Wednesday 11 December 2019
Sunday 6 October 2019
student perceptions of active learning (3)
Before I discuss this paper, let me make it clear that I think the balance between lecture and active learning is completely dependent upon the class context. This is not an either/or situation. Teaching involves careful consideration of what our students need to learn and how to best support their learning efforts. Sometimes this will involve telling students something to clear up a misconception. Sometimes this will involve having our students apply or discuss their learning in order to make their learning stick by having them realize that they have not yet deeply learned a concept. I liken the situation to tuning the dial of a radio tuner.
Thus, whenever someone asks me how much active learning they should introduce into their classroom I answer: it depends. It depends upon the student cohort (their experiences and preparation), the year level of the course, the discipline of the course, whether the course is heavy on learning theory, or producing an assignment, what the learning culture is of the particular campus on which you find yourself teaching. All of these things will influence the right balance of lecturing and active learning that happens in your class. What I can say is that if only one is happening in class, then likely not enough of the other is occurring.
Having said that, I really liked this paper (Deslauriers et al 2019). It seems to me to be a well-controlled and thoughtful paper. The authors used a randomized, cross-over design to determine whether students' feeling of learning (FOL) correlate with their actual learning (as assessed with a test of learning - TOL) in an intro physics course taught with active learning vs lecture.
The results clearly indicate that students' FOL was greater when lecture was used whereas their TOL was greater in the active learning environment. This agrees with the Dunning-Kruger effect which suggests that novices have poor metacognition with which to assess their own learning. Their results also agree with other published results which show that students do not appreciate active learning (van Sickle 2016; Smith et al 2011) despite active learning producing better learning outcomes (Freeman et al 2014).
The FOL survey used in this study allowed the authors to also consider the impact of lecture fluency and they found that FOL was positively impacted by fluency. The literature suggests that fluency can inflate students' perceptions of learning as cited in Deslauriers et al (2019).
In a good mixed-methods design, the researchers followed up a subset of the participating students with interviews and they found that students' perceptions of learning were negatively impacted by the struggle required with active learning. When it was pointed out to them that studies show that cognitive effort positively impacts learning, students suggested that knowledge would influence their perception of learning in active learning courses.
So what is impacting the disconnect among novice students' perceptions of their own learning? As suggested above, one is that novices do not have well-developed metacognition. They have difficulties recognizing good judgement and, as a result, have difficulties judging their own learning. In addition, lecture fluency can mislead students into thinking they have learned something when in fact they have not. Finally, students unfamiliar with the cognitive effort required for active learning may resent the effort required to master a body of knowledge.
This resonates with my own teaching experience. Before I implemented active learning in my own classrooms a number of years ago, I was perplexed when students informed me of their frustration with their own learning because I explained things so well in class that they thought they understood what I was teaching but that perception did not translate into good exam performance. It was that student feedback that compelled me to look for other ways of teaching that would make it explicit to students what they did and did not know, what they had and had not learned. Team-based learning is the active learning structure I have implemented in my courses and it works well by hitting a number of the known factors by which active learning promotes student learning (Ambrose et al 2010; Brown et al 2014; Mazur 2009): pre-class preparation, attempting new problems in class, interacting with peers to practice their understanding of what they have learned. What these strategies do is make apparent to students what they do and do not know thereby enabling them to follow up and address the points of misunderstanding they have. In addition, being able to teach each other is a powerful way to cement their learning and create a robust knowledge structure and mental model of what they are learning that integrates with their existing understanding of their world.
So what are we to do to help students accept and embrace active learning despite the cognitive effort required with this teaching strategy? The authors conclude their paper by reporting on the results of an intervention they ran after receiving their study results. They spent time at the beginning of a subsequent term to show students the results of the research and explain how cognitive effort leads to increased learning. This was a 20-minute presentation of the results of the impact of active learning on learning gains and the influence of fluency on perceptions of learning. The researchers observed that students in the Q&A following the presentation were most interested in the idea that FOL and fluency can mislead their judgement about how their learning is progressing. A student survey showed that most students had a more favourable view of active learning as a result of the initial intervention.
These results agree with the earlier Finelli et al (2018) study that suggested that how instructors prepare students for active learning (explain why they are using a particular active learning strategy) goes a long way to mitigate students' resistance to active learning. More significantly in my mind is that the same Finelli study found that how instructors facilitated an active learning activity played a more important role in students response to active learning: being engaged with students during the activity promoted students' perceptions that active learning was enabling their learning.
Please notice, that as I stated above, in both the Finelli and Deslauriers papers it is not that lecturing is completely absent. Read carefully their papers and you will read that they note that mini-lectures were used as necessary. Similar to how I opened this blog post, it is not that lecturing is bad. It is that the injudicious use of lecture is bad. But of course, this can also be applied to active learning. Active learning will not fix bad teaching. Good teachers will be judicious about their use of lecture and active learning and implement either as the context dictates. Having students engaged in active learning for the sake of active learning is not the lesson here. The lesson here is that good instructors will implement lecturing and active learning as required by the particular context and that this will change from year to year, cohort to cohort, class to class, minute to minute.
This is what makes teaching such an interesting challenge as we support our students' learning efforts.
Brown, P. C., McDaniel, M. A., & Roediger, H. L. (2014). Make it stick: The science of successful learning. Cambridge, Massachusetts: Harvard University Press.
Deslauriers, L., McCarty, L. S., Miller, K., Callaghan, K., & Kestin, G. (2019). Measuring actual learning versus feeling of learning in response to being actively engaged in the classroom. Proceedings of the National Academy of Sciences, 201821936.
Finelli, B. C. J., Nguyen, K., Demonbrun, M., Borrego, M., Prince, M., Husman, J., … Waters, C. K. (2018). Reducing student resistance to active learning: Strategies for instructors. Journal of College Science Teaching, 47(5), 80–91. A PDF of this paper is available from Harvard here.
Freeman, S., Eddy, S. L., McDonough, M., Smith, M. K., Okoroafor, N., Jordt, H., & Wenderoth, M. P. (2014). Active learning increases student performance in science, engineering, and mathematics. Proceedings of the National Academy of Sciences of the United States of America, 111(23), 8410–8415.
Kruger, J., & Dunning, D. (1999). Unskilled and unaware of it: How difficulties in recognizing one’s own incompetence lead to inflated self-assessments. Journal of Personality & Social Psychology, 77(6), 1121–1134.
Mazur, E. (2009). Farewell, Lecture? Science, 323(5910), 50–51.
Smith, C. V, & Cardaciotto, L. (2011). Is active learning like broccoli? Student perceptions of active learning in large lecture classes. Journal of the Scholarship of Teaching & Learning, 11(1), 53–61.
Van Sickle, J. R. (2016). Discrepancies between student perception and achievement of learning outcomes in a flipped classroom. Journal of the Scholarship of Teaching and Learning, 16(2), 29–38.
Thus, whenever someone asks me how much active learning they should introduce into their classroom I answer: it depends. It depends upon the student cohort (their experiences and preparation), the year level of the course, the discipline of the course, whether the course is heavy on learning theory, or producing an assignment, what the learning culture is of the particular campus on which you find yourself teaching. All of these things will influence the right balance of lecturing and active learning that happens in your class. What I can say is that if only one is happening in class, then likely not enough of the other is occurring.
Having said that, I really liked this paper (Deslauriers et al 2019). It seems to me to be a well-controlled and thoughtful paper. The authors used a randomized, cross-over design to determine whether students' feeling of learning (FOL) correlate with their actual learning (as assessed with a test of learning - TOL) in an intro physics course taught with active learning vs lecture.
The results clearly indicate that students' FOL was greater when lecture was used whereas their TOL was greater in the active learning environment. This agrees with the Dunning-Kruger effect which suggests that novices have poor metacognition with which to assess their own learning. Their results also agree with other published results which show that students do not appreciate active learning (van Sickle 2016; Smith et al 2011) despite active learning producing better learning outcomes (Freeman et al 2014).
The FOL survey used in this study allowed the authors to also consider the impact of lecture fluency and they found that FOL was positively impacted by fluency. The literature suggests that fluency can inflate students' perceptions of learning as cited in Deslauriers et al (2019).
In a good mixed-methods design, the researchers followed up a subset of the participating students with interviews and they found that students' perceptions of learning were negatively impacted by the struggle required with active learning. When it was pointed out to them that studies show that cognitive effort positively impacts learning, students suggested that knowledge would influence their perception of learning in active learning courses.
So what is impacting the disconnect among novice students' perceptions of their own learning? As suggested above, one is that novices do not have well-developed metacognition. They have difficulties recognizing good judgement and, as a result, have difficulties judging their own learning. In addition, lecture fluency can mislead students into thinking they have learned something when in fact they have not. Finally, students unfamiliar with the cognitive effort required for active learning may resent the effort required to master a body of knowledge.
This resonates with my own teaching experience. Before I implemented active learning in my own classrooms a number of years ago, I was perplexed when students informed me of their frustration with their own learning because I explained things so well in class that they thought they understood what I was teaching but that perception did not translate into good exam performance. It was that student feedback that compelled me to look for other ways of teaching that would make it explicit to students what they did and did not know, what they had and had not learned. Team-based learning is the active learning structure I have implemented in my courses and it works well by hitting a number of the known factors by which active learning promotes student learning (Ambrose et al 2010; Brown et al 2014; Mazur 2009): pre-class preparation, attempting new problems in class, interacting with peers to practice their understanding of what they have learned. What these strategies do is make apparent to students what they do and do not know thereby enabling them to follow up and address the points of misunderstanding they have. In addition, being able to teach each other is a powerful way to cement their learning and create a robust knowledge structure and mental model of what they are learning that integrates with their existing understanding of their world.
So what are we to do to help students accept and embrace active learning despite the cognitive effort required with this teaching strategy? The authors conclude their paper by reporting on the results of an intervention they ran after receiving their study results. They spent time at the beginning of a subsequent term to show students the results of the research and explain how cognitive effort leads to increased learning. This was a 20-minute presentation of the results of the impact of active learning on learning gains and the influence of fluency on perceptions of learning. The researchers observed that students in the Q&A following the presentation were most interested in the idea that FOL and fluency can mislead their judgement about how their learning is progressing. A student survey showed that most students had a more favourable view of active learning as a result of the initial intervention.
These results agree with the earlier Finelli et al (2018) study that suggested that how instructors prepare students for active learning (explain why they are using a particular active learning strategy) goes a long way to mitigate students' resistance to active learning. More significantly in my mind is that the same Finelli study found that how instructors facilitated an active learning activity played a more important role in students response to active learning: being engaged with students during the activity promoted students' perceptions that active learning was enabling their learning.
Please notice, that as I stated above, in both the Finelli and Deslauriers papers it is not that lecturing is completely absent. Read carefully their papers and you will read that they note that mini-lectures were used as necessary. Similar to how I opened this blog post, it is not that lecturing is bad. It is that the injudicious use of lecture is bad. But of course, this can also be applied to active learning. Active learning will not fix bad teaching. Good teachers will be judicious about their use of lecture and active learning and implement either as the context dictates. Having students engaged in active learning for the sake of active learning is not the lesson here. The lesson here is that good instructors will implement lecturing and active learning as required by the particular context and that this will change from year to year, cohort to cohort, class to class, minute to minute.
This is what makes teaching such an interesting challenge as we support our students' learning efforts.
Resources
Ambrose, S. A., Bridges, M. W., DiPietro, M., Lovett, M. C., Norman, M. K., & Mayer, R. E. (2010). How learning works: Seven research-based principles for smart teaching (1st ed.). San Francisco, CA: Jossey-Bass.Brown, P. C., McDaniel, M. A., & Roediger, H. L. (2014). Make it stick: The science of successful learning. Cambridge, Massachusetts: Harvard University Press.
Deslauriers, L., McCarty, L. S., Miller, K., Callaghan, K., & Kestin, G. (2019). Measuring actual learning versus feeling of learning in response to being actively engaged in the classroom. Proceedings of the National Academy of Sciences, 201821936.
Finelli, B. C. J., Nguyen, K., Demonbrun, M., Borrego, M., Prince, M., Husman, J., … Waters, C. K. (2018). Reducing student resistance to active learning: Strategies for instructors. Journal of College Science Teaching, 47(5), 80–91. A PDF of this paper is available from Harvard here.
Freeman, S., Eddy, S. L., McDonough, M., Smith, M. K., Okoroafor, N., Jordt, H., & Wenderoth, M. P. (2014). Active learning increases student performance in science, engineering, and mathematics. Proceedings of the National Academy of Sciences of the United States of America, 111(23), 8410–8415.
Kruger, J., & Dunning, D. (1999). Unskilled and unaware of it: How difficulties in recognizing one’s own incompetence lead to inflated self-assessments. Journal of Personality & Social Psychology, 77(6), 1121–1134.
Mazur, E. (2009). Farewell, Lecture? Science, 323(5910), 50–51.
Smith, C. V, & Cardaciotto, L. (2011). Is active learning like broccoli? Student perceptions of active learning in large lecture classes. Journal of the Scholarship of Teaching & Learning, 11(1), 53–61.
Van Sickle, J. R. (2016). Discrepancies between student perception and achievement of learning outcomes in a flipped classroom. Journal of the Scholarship of Teaching and Learning, 16(2), 29–38.
Tuesday 1 October 2019
student perceptions of active learning (2)
This study (Smith & Cardaciotto, 2011) is similar to the more recent study published by Jenna Van Sickle. Both studies report positive student outcomes with active learning but that students rated the active experience as less positive than the more passive learning experience. So it is interesting that although there is ample evidence to suggest that active learning is good for students, students do not appreciate the experience. The authors liken this to telling children to eat their broccoli because it is good for them.
The Van Sickle study considered a math course whereas the Smith & Cardaciotto broccoli study considered introductory psychology. Smith & Cardaciotto state at the end that active learning activities need to be embedded in sound pedagogy and not simply involve students “doing” something. I am not sure why they tacked this on at the end of their paper as their study design did not address this at all. It is an important consideration, but their study design has nothing to say about this.
A limitation of this study in terms of it being integrated with other active learning studies is that the authors interpreted active learning as any activity that engaged the students cognitively. Thus, their active learning activities were done outside of class rather than the more typical understanding that active learning involves transforming what happens in the traditional didactic lecture - the activities happen inside of the classroom. They make the comparison that what they are doing is similar to the sciences which have a didactic lecture associated with an active lab or tutorial. What is interesting is that most of the published research suggests that active learning in science classes regardless of an active lab or tutorial will promote student learning outcomes. This broccoli study is odd this way thinking that active learning outside of class will do the trick. And even their two-course modules on brain and behaviour showed no differences between the content review and the active learning conditions which they suggest is because those course sections use more active learning during class meetings.
Another limitation is that they did not specifically consider student learning outcomes but rather only student perceptions of their own learning. They acknowledge this and request that subsequent studies specifically consider student learning outcomes as exam or grade results. They cite literature that indicates that students self-reports of learning correlates with actual learning outcomes and so can indicate an impact on student learning. But still, this study did not specifically study this. The reason they give is that the different instructors administered different exams and thus were not comparable.
What I find odd is that the authors make the assumption that students can accurately assess their learning and thus student perception surveys can indicate student learning outcomes to some extent though they do explain that this needs to be studied directly. The reason that I find this odd is that although the authors cite a study indicating the reliability of student perception reports of their learning, the Dunning-Kruger effect suggests that weak students over-estimate their learning whereas good students under-estimate their learning. So it does not suggest that student self-reports are reliable measures of their own learning.
More recent students (e.g. Finelli et al, 2018) suggest that the manner in which instructors explain and facilitate the learning activities in their classroom can go a long way to mitigate students' resistance to learning
Kruger, J., & Dunning, D. (1999). Unskilled and unaware of it: How difficulties in recognizing one’s own incompetence lead to inflated self-assessments. Journal of Personality & Social Psychology, 77(6), 1121–1134. https://doi.org/10.1037//0022-3514.77.6.1121
Smith, C. V, & Cardaciotto, L. (2011). Is active learning like broccoli? Student perceptions of active learning in large lecture classes. Journal of the Scholarship of Teaching & Learning, 11(1), 53–61. Retrieved from https://josotl.indiana.edu/article/view/1808/1805
Van Sickle, J. R. (2016). Discrepancies between student perception and achievement of learning outcomes in a flipped classroom. Journal of the Scholarship of Teaching and Learning, 16(2), 29–38. https://doi.org/10.14434/josotl.v16i2.19216
Wieman, C. E. (2014). Large-scale comparison of science teaching methods sends clear message. Proceedings of the National Academy of Sciences of the United States of America, 111(23), 8319–8320. https://doi.org/10.1073/pnas.1407304111
The Van Sickle study considered a math course whereas the Smith & Cardaciotto broccoli study considered introductory psychology. Smith & Cardaciotto state at the end that active learning activities need to be embedded in sound pedagogy and not simply involve students “doing” something. I am not sure why they tacked this on at the end of their paper as their study design did not address this at all. It is an important consideration, but their study design has nothing to say about this.
A limitation of this study in terms of it being integrated with other active learning studies is that the authors interpreted active learning as any activity that engaged the students cognitively. Thus, their active learning activities were done outside of class rather than the more typical understanding that active learning involves transforming what happens in the traditional didactic lecture - the activities happen inside of the classroom. They make the comparison that what they are doing is similar to the sciences which have a didactic lecture associated with an active lab or tutorial. What is interesting is that most of the published research suggests that active learning in science classes regardless of an active lab or tutorial will promote student learning outcomes. This broccoli study is odd this way thinking that active learning outside of class will do the trick. And even their two-course modules on brain and behaviour showed no differences between the content review and the active learning conditions which they suggest is because those course sections use more active learning during class meetings.
Another limitation is that they did not specifically consider student learning outcomes but rather only student perceptions of their own learning. They acknowledge this and request that subsequent studies specifically consider student learning outcomes as exam or grade results. They cite literature that indicates that students self-reports of learning correlates with actual learning outcomes and so can indicate an impact on student learning. But still, this study did not specifically study this. The reason they give is that the different instructors administered different exams and thus were not comparable.
What I find odd is that the authors make the assumption that students can accurately assess their learning and thus student perception surveys can indicate student learning outcomes to some extent though they do explain that this needs to be studied directly. The reason that I find this odd is that although the authors cite a study indicating the reliability of student perception reports of their learning, the Dunning-Kruger effect suggests that weak students over-estimate their learning whereas good students under-estimate their learning. So it does not suggest that student self-reports are reliable measures of their own learning.
More recent students (e.g. Finelli et al, 2018) suggest that the manner in which instructors explain and facilitate the learning activities in their classroom can go a long way to mitigate students' resistance to learning
Resources
Finelli, B. C. J., Nguyen, K., Demonbrun, M., Borrego, M., Prince, M., Husman, J., … Waters, C. K. (2018). Reducing student resistance to active learning: Strategies for instructors. Journal of College Science Teaching, 47(5), 80–91.Kruger, J., & Dunning, D. (1999). Unskilled and unaware of it: How difficulties in recognizing one’s own incompetence lead to inflated self-assessments. Journal of Personality & Social Psychology, 77(6), 1121–1134. https://doi.org/10.1037//0022-3514.77.6.1121
Smith, C. V, & Cardaciotto, L. (2011). Is active learning like broccoli? Student perceptions of active learning in large lecture classes. Journal of the Scholarship of Teaching & Learning, 11(1), 53–61. Retrieved from https://josotl.indiana.edu/article/view/1808/1805
Van Sickle, J. R. (2016). Discrepancies between student perception and achievement of learning outcomes in a flipped classroom. Journal of the Scholarship of Teaching and Learning, 16(2), 29–38. https://doi.org/10.14434/josotl.v16i2.19216
Wieman, C. E. (2014). Large-scale comparison of science teaching methods sends clear message. Proceedings of the National Academy of Sciences of the United States of America, 111(23), 8319–8320. https://doi.org/10.1073/pnas.1407304111
Friday 27 September 2019
student perceptions of active learning (1)
This paper (Van Sickle, 2016) articulates what I have been experiencing in my TBL courses: student engagement and learning has been raised, but students do not like the process. Learning is being enhanced, but students do not like how it has been achieved.
One of the implications of this study is that students’ self-reports are not accurate indicators of their learning: students are not able to accurately assess the quality of their own learning. In this study, students performed better in the flipped classroom (as indicated by exam scores) yet they rated the learning experience as less effective than a traditional lecture.
Jenna Van Sickle unpacks this discrepancy with five possibilities:
Note the difference in study design between this and those that study student perception-outcome dichotomy. This study looked at aggregate scores for perceptions and outcomes and compared those. In contrast, Dunning-Kruger analyses consider each students' perception of their own learning and compares that to their actual learning outcome. That is, a regression among the individual points between perception and outcome is assessed. This is a more accurate way to look at this dichotomy. So a follow-up study would be to do this same regression analysis among active learning and lecture classes to see if there is a difference in perception correlation with student learning outcomes.
Van Sickle, J. R. (2016). Discrepancies between student perception and achievement of learning outcomes in a flipped classroom. Journal of the Scholarship of Teaching and Learning, 16(2), 29–38. https://doi.org/10.14434/josotl.v16i2.19216
One of the implications of this study is that students’ self-reports are not accurate indicators of their learning: students are not able to accurately assess the quality of their own learning. In this study, students performed better in the flipped classroom (as indicated by exam scores) yet they rated the learning experience as less effective than a traditional lecture.
Jenna Van Sickle unpacks this discrepancy with five possibilities:
- Learning is messy and hard and students resent experiencing this reality.
- Less equitable student-instructor interaction in the flipped classroom. It increases markedly for some and none for others whereas in the traditional lecture everyone receives the same interaction. I don’t think I agree with this because most traditional lectures, I think, will involve some Q&A between instructor and student, but typically this will be with the same students willing to raise their hands. But note this in the context of Anna Risannen’s findings in their blended learning Science courses on the North Campus - it seems that the instructor has the greatest impact (on student learning or on student perceptions of learning?).
- First time experience with active learning. Students understanding of teaching may involve the teacher telling them what to learn whereas flipped requires students to take responsibility for this.
- Not all students will have done the pre-class assignment and thus feel ill-equipped to attend to the in-class assignments. Although this is their choice and responsibility when they assess the class with SETs they may remember that they felt uncomfortable in class and this is what influences their SET rating of the class.
- Finally, Jenna suggests that a class culture of being ok to be wrong may be uncomfortable for students. This kind of learning environment requires students to take risks in order to benefit from the active learning opportunities. Many students will, but some students may feel uncomfortable doing this. Instructors can mitigate this by giving feedback often in a manner that praises effort rather than ability (promoting a growth mindset). Teaching is cultivating the development of ability which requires focus, effort, and time on task. These are the attributes on which instructors need to be giving feedback because these are what produce learning. In contrast, in a lecture-based class, students will not have to risk a wrong answer. There is inevitably embarrassment with a public wrong answer and this will colour students’ perception of the class. Yet, this is exactly what I am trying to do with my use of TBL - I want students to realize when they do not know something. I want to prevent students from fooling themselves that they have learned something - students may resent my holding a mirror to their learning.
Note the difference in study design between this and those that study student perception-outcome dichotomy. This study looked at aggregate scores for perceptions and outcomes and compared those. In contrast, Dunning-Kruger analyses consider each students' perception of their own learning and compares that to their actual learning outcome. That is, a regression among the individual points between perception and outcome is assessed. This is a more accurate way to look at this dichotomy. So a follow-up study would be to do this same regression analysis among active learning and lecture classes to see if there is a difference in perception correlation with student learning outcomes.
Resources
Kruger, J., & Dunning, D. (1999). Unskilled and unaware of it: How difficulties in recognizing one’s own incompetence lead to inflated self-assessments. Journal of Personality & Social Psychology, 77(6), 1121–1134.Van Sickle, J. R. (2016). Discrepancies between student perception and achievement of learning outcomes in a flipped classroom. Journal of the Scholarship of Teaching and Learning, 16(2), 29–38. https://doi.org/10.14434/josotl.v16i2.19216
Tuesday 18 June 2019
the Goldilocks point between lecture and active learning: 20-60%
This paper by Henderson et al (2018) suggests that most students will not penalize instructors who use active learning with low student evaluations of teaching. Interestingly, those instructors who used lecture more than 60% of the time reported no change in their student ratings of instruction (SRIs) whereas those who reported using lecture less than 20% of class time reported decreased SRIs. Those instructors whose implementation of active learning was between 20 to 60% typically reported an increase in SRIs.
A limitation that the authors fully acknowledge is that these data were collected by surveying faculty who had completed their four-day new professor workshop (NPW) during the first couple of years of their academic appointment. Their NPW has the goal to develop in new professors the capacity to implement active learning strategies. However! These are self-reports. SRIs were not actually enumerated and statistically analyzed. This is an issue for this study. Are instructors remembering their SRIs differently? How do they interpret their SRIs? Is a median above three (assuming a 5-point Likert scale) considered good to them? Is a median below four considered bad? There are differences in institutional culture as exemplified among the departments at my university.
Despite this major limitation, this is the beginning of gathering real evidence for how students respond to active learning.
Note that the authors clearly explain their position on the use of SRIs - that they should be interpreted carefully and only be one aspect of triangulating teaching efficacy. They suggest, however, that SRIs probably don’t even assess teaching efficacy and that other aspects of multi-faceted evaluation of teaching are necessary to actually assess teaching efficacy.
In addition, I appreciate that the authors clearly explain that the correlations between the amount of lecturing and SRIs are not set in stone. There were some instructors in their study who lectured less than 20% of the time who reported improved SRIs while there were also those in the Goldilocks region of 20-60% that reported decreases in SRIs. They make the important point that how active learning is received by students is heavily influenced by how instructors set up and facilitate the active learning activity in addition to the particular instructional/department/program context/culture. These all influence student expectations for instruction and learning and if the educational experience does not match students’ expectations, that is when students will award poor SRIs.
Teaching and learning are context dependent. The answer to how much active learning should be implemented in any particular class is... it depends.
Henderson, C., Khan, R., & Dancy, M. (2018). Will my student evaluations decrease if I adopt an active learning instructional strategy? American Journal of Physics, 86(12), 934–942.
A limitation that the authors fully acknowledge is that these data were collected by surveying faculty who had completed their four-day new professor workshop (NPW) during the first couple of years of their academic appointment. Their NPW has the goal to develop in new professors the capacity to implement active learning strategies. However! These are self-reports. SRIs were not actually enumerated and statistically analyzed. This is an issue for this study. Are instructors remembering their SRIs differently? How do they interpret their SRIs? Is a median above three (assuming a 5-point Likert scale) considered good to them? Is a median below four considered bad? There are differences in institutional culture as exemplified among the departments at my university.
Despite this major limitation, this is the beginning of gathering real evidence for how students respond to active learning.
Note that the authors clearly explain their position on the use of SRIs - that they should be interpreted carefully and only be one aspect of triangulating teaching efficacy. They suggest, however, that SRIs probably don’t even assess teaching efficacy and that other aspects of multi-faceted evaluation of teaching are necessary to actually assess teaching efficacy.
In addition, I appreciate that the authors clearly explain that the correlations between the amount of lecturing and SRIs are not set in stone. There were some instructors in their study who lectured less than 20% of the time who reported improved SRIs while there were also those in the Goldilocks region of 20-60% that reported decreases in SRIs. They make the important point that how active learning is received by students is heavily influenced by how instructors set up and facilitate the active learning activity in addition to the particular instructional/department/program context/culture. These all influence student expectations for instruction and learning and if the educational experience does not match students’ expectations, that is when students will award poor SRIs.
Teaching and learning are context dependent. The answer to how much active learning should be implemented in any particular class is... it depends.
Resources
Benton, S. L., & Ryalls, K. R. (2016). Challenging misconceptions about student ratings of instruction. IDEA Paper, 58(April), 1–22.Henderson, C., Khan, R., & Dancy, M. (2018). Will my student evaluations decrease if I adopt an active learning instructional strategy? American Journal of Physics, 86(12), 934–942.
Thursday 13 June 2019
what is the thesis of your course?
Two online articles spoke to me today spurring me to articulate how I approach my courses. The one by Ryan Boyd is an essay review of Josh Eyler's recent book How Humans Learn and the other by Kevin Gannon considers how to manage survey courses. Both consider what interferes with student learning and suggest that part of the issue is how some university courses are taught: large passive lectures with too much content. I am finally reading Paolo Freire's book Pedagogy of the Oppressed, a book I have been meaning to read for many years, in which one of the sources of oppression he suggests is the volume of content that is found in some courses. This is what Gannon refers to as the fire hose approach to teaching and learning: students open their brains wide and teachers rapidly pour the content in. I think this is akin to Freire's articulation of the banking model of education in which teachers transmit the information and students receive it banking it for later use in their brains. What Freire articulates in his book is that this is a form of oppression because it prevents students from thinking, it represses their ability to develop their cognition.
A few of weeks ago at the University of Alberta's Festival of Teaching and Learning, Dr Jeanette Norden gave the keynote address in which she advocated educators to teach less, better. (Notice the key placement of the comma in that phrase: it reminds me of Eats, Shoots & Leaves.) She urged educators to teach less content but to support students' learning of that content to a deeper level.
All of these recent sources that I have been thinking about suggest the same thing: in our courses, give students the room to think about what they are learning. Otherwise, students will only learn what we are teaching on a superficial level. Content heavy courses will not nurture our students' cognitive abilities.
Now, granted, there is a continuum here that is dependent upon how advanced a course is, students' previous experiences, and the goal of the course. Some courses will be more content heavy than others depending upon this constellation of factors. So how do we design our courses to take this into account?
The way I do it is similar to how I orient my overall assessment of a students' essay: what is the thesis of this paper and is that thesis well supported and articulated? The same question can be applied to any course we teach: what is the thesis of my course and does its design support that thesis? When approached this way, we are encouraged to curate the course content as suggested by Gannon - no longer is a fire hose needed to deliver an abundance of course content if only a few examples will support the course's thesis. Similarly, if we well articulate our thesis, then, as suggested by Boyd, we can begin to ask the salient questions that our course seeks to answer in order to resolve or support the thesis.
Resources
Boyd, R. (2019). Beautiful questions: “How Humans Learn” and the future of education. Los Angeles Review of Books. May 27.Freire, P. (2018). Pedagogy of the oppressed (50th anniv). New York, NY: Bloomsbury Academic.
Gannon, K. (2019). How to Fix the Dreaded Survey Course. The Chronicle of Higher Education. May 7.
Monday 3 June 2019
2 hrs outside of class for every hr spent inside of class
This is the advice I learned as a student and the advice that many academic deans gave to our Augustana students since I started teaching in 1990. It has always generally been assumed that for students to be academically successful at college/university, they had to spend a minimum (yes, a minimum) of two hours studying outside of class for every hour spent inside of the class. If the typical academic program is 5 courses per term and each of those courses is meeting for three hours per week that sums to a minimum of 45 hrs per week (15 in class, 30 outside of class) learning academic skills and course material. I have always thought as a student and as a teacher, that a full-time undergraduate program of study is more than a full-time job.
Yes, this means that students in programs that require more than 15 hrs of class time per week (e.g. art studio, science courses with labs, engineering) expect/assume more time on task. This may not be that different from the Arts in which students have to read multiple novels or monographs for a given course. This is why I advocate for lab work during the 1st two years of students' undergraduate programs to be self-contained and not require any homework other than preparation for the lab. For more senior labs, these should be lab courses, not lec-lab courses. Otherwise, the expectations for students to spend time on coursework becomes unreasonable.
So why am I bringing this up now? A couple of years ago, I remember reading on my end of term student ratings of instruction a couple of comments responding to my start of term advice that to do well students should be spending a minimum of two hours studying outside of class for every hour spent inside of class. The comments amounted to shock and horror that an instructor, that an educational institution would have such expectations of their students. That such expectations were unreasonable because students had a life outside of school. When I discussed this with my colleagues, the more recent ones to the teaching profession shared the students' shock and horror - a 2:1 ratio of outside:inside course work was clearly unreasonable.
When did this change? When did the understanding change that learning does not require time on task? All of the evidence I have read suggests that academic success depends upon time on task and that one of the best things that instructors can do for their students is to encourage their time on task on educationally purposeful activities. This includes doing homework (reading the text, doing practice problems), discussing class material in a study group, preparing lab reports, conducting research for term papers, spending time in the studio painting or drawing, spending time rehearsing lines, doing the grunt work of memorizing vocabulary and grammatical rules for language acquisition, memorizing the chemical formulae for functional groups, practicing conversing in the language of the discipline.
Does anyone think that three hours per week in-class is sufficient to master their coursework? Where is this resistance to the sage advice of two hours outside of class for every hour inside of class coming from? Who thinks that students do not need to spend time doing the hard messy work of learning on their own time outside of class?
In the resources below it seems like the 2:1 adage is still prevalent. But I do like Lolita Paff's Faculty Focus article and the USU estimate study hours worksheet to be a much better-nuanced consideration of this issue. Bottom line from those two resources is that it depends upon the type and difficulty of the course. But it still looks to me that it ends up being approximately 2:1 but that the hours outside of class may be differentially allocated depending upon the constellation of courses in which a student is enrolled for a given term. Something that we can do to circumvent the student complaint that they did not perform as they thought they would on an exam given the amount of time spent studying for the exam is to discuss efficient vs weak study/learning strategies. Many students still use passive learning strategies (e.g. reading over their notes, using flashcards, reading the text with a highlighter in hand) rather than active learning strategies (e.g. rewriting/reorganizing their notes, engaging in retrieval practice, reading the text with pencil and paper in hand, spacing rather than massing their study, and mixing their study time by attending to different courses). I have found that having this conversation and having students think about their approach to studying often transforms a mediocre student into a high-performing student. Many students have found the book Make It Stick to be an invaluable resource.
Yes, this means that students in programs that require more than 15 hrs of class time per week (e.g. art studio, science courses with labs, engineering) expect/assume more time on task. This may not be that different from the Arts in which students have to read multiple novels or monographs for a given course. This is why I advocate for lab work during the 1st two years of students' undergraduate programs to be self-contained and not require any homework other than preparation for the lab. For more senior labs, these should be lab courses, not lec-lab courses. Otherwise, the expectations for students to spend time on coursework becomes unreasonable.
So why am I bringing this up now? A couple of years ago, I remember reading on my end of term student ratings of instruction a couple of comments responding to my start of term advice that to do well students should be spending a minimum of two hours studying outside of class for every hour spent inside of class. The comments amounted to shock and horror that an instructor, that an educational institution would have such expectations of their students. That such expectations were unreasonable because students had a life outside of school. When I discussed this with my colleagues, the more recent ones to the teaching profession shared the students' shock and horror - a 2:1 ratio of outside:inside course work was clearly unreasonable.
When did this change? When did the understanding change that learning does not require time on task? All of the evidence I have read suggests that academic success depends upon time on task and that one of the best things that instructors can do for their students is to encourage their time on task on educationally purposeful activities. This includes doing homework (reading the text, doing practice problems), discussing class material in a study group, preparing lab reports, conducting research for term papers, spending time in the studio painting or drawing, spending time rehearsing lines, doing the grunt work of memorizing vocabulary and grammatical rules for language acquisition, memorizing the chemical formulae for functional groups, practicing conversing in the language of the discipline.
Does anyone think that three hours per week in-class is sufficient to master their coursework? Where is this resistance to the sage advice of two hours outside of class for every hour inside of class coming from? Who thinks that students do not need to spend time doing the hard messy work of learning on their own time outside of class?
In the resources below it seems like the 2:1 adage is still prevalent. But I do like Lolita Paff's Faculty Focus article and the USU estimate study hours worksheet to be a much better-nuanced consideration of this issue. Bottom line from those two resources is that it depends upon the type and difficulty of the course. But it still looks to me that it ends up being approximately 2:1 but that the hours outside of class may be differentially allocated depending upon the constellation of courses in which a student is enrolled for a given term. Something that we can do to circumvent the student complaint that they did not perform as they thought they would on an exam given the amount of time spent studying for the exam is to discuss efficient vs weak study/learning strategies. Many students still use passive learning strategies (e.g. reading over their notes, using flashcards, reading the text with a highlighter in hand) rather than active learning strategies (e.g. rewriting/reorganizing their notes, engaging in retrieval practice, reading the text with pencil and paper in hand, spacing rather than massing their study, and mixing their study time by attending to different courses). I have found that having this conversation and having students think about their approach to studying often transforms a mediocre student into a high-performing student. Many students have found the book Make It Stick to be an invaluable resource.
Resources
University website resources
- https://www.umflint.edu/advising/surviving_college
- https://www.cliffsnotes.com/cliffsnotes/subjects/college/how-much-outside-class-study-time-is-recommended-for-every-hour-of-class-time-for-college-freshmen
- https://www.usu.edu/asc/assistance/pdf/estimate_study_hours.pdf
- https://wellness.ucsd.edu/CAPS/Documents/tx_forms/koch/college_success/college_success.pdf
- https://www.astrosociety.org/education/hints-on-how-to-succeed-in-college-classes/
- https://courses.lumenlearning.com/waymaker-collegesuccess/chapter/text-class-time-to-study-time-ratio/
- https://dus.psu.edu/academicsuccess/explanation.html
- https://www.maryville.edu/academicaffairs/credit-hourcontact-hour-guidelines/
- https://www.college.columbia.edu/coursepoints
- http://cte.rice.edu/blogarchive/2016/07/11/workload
- https://www.mcgill.ca/science/student/general/advising/skills
Wednesday 6 February 2019
responding to students' resistance to active learning
Our SoTL journal club at Augustana recently met to discuss an interesting article by Finelli, et al 2018 (Reducing student resistance to active learning: Strategies for instructors published in the J Coll Sci Teaching - see reference below). This paper reports the use of a student survey of instructional and facilitation practices and correlated these with students' self-reports of engagement and value of the active learning activities. Interestingly, the study also surveyed instructor’s self-reports of their teaching and facilitation efforts/activities and the study found no difference - teachers and students both perceived the educational setting similarly.
What was interesting to me is that the study found that student resistance to active learning was not high - most students appreciated and engaged in the active learning activities. The difficulty for many instructors working in an environment in which student evaluation of teaching (SET) is used to assess teaching ability, Likert ratings below 4 may be considered to be low. However, this paper reminds us that a SET rating above 3 means that most students appreciate the experience. But excellent teachers may be accustomed to the vast majority of students appreciating the learning experience, not just most students. So maybe a significant finding of this research is that teachers need to be satisfied - find solace - in the fact that active learning reaches most students.
The other interesting finding is that students noted that instructors typically use explanations to alleviate resistance to active learning. That is, that instructors explain to students what is expected of them for the activity and how it will benefit their learning. The statistical analysis in this paper, however, indicates that facilitation efforts may be more effective at reducing student resistance. Facilitation strategies include instructor demeanor toward the students and the activity, inviting students to ask questions about the activity, walking around the room to assist student teams, soliciting student feedback on the activity, and confronting students unengaged in the activity. The last two seemed to have the lowest impact on alleviating student resistance.
So, some good advice backed by evidence for responding to student resistance to active learning: facilitate student engagement with the activities by engaging with the students during the activity and being interested/happy/excited about the activity and how students are interacting with the activity.
There are limitations to the study and I appreciate the authors clearly indicating these. The participating classes were self-selected. The participating classes were not observed by a third party to corroborate the student and instructor ratings of the nature of the class environment. But the authors correctly indicate that some of the key aspects of the study would not have been caught by this triangulation as it is difficult to know/observe/measure students internal environment regarding how they value or emotionally respond to a learning activity. In addition, the low number of classes and student enrollment means that there is the concern of variability within the class being greater than the variability between classes. But this is also difficult to address because different students will perceive the course differently - not all students will observe how the instructor is facilitating their peers’ learning because they themselves are engaged with the learning activity. I am not sure what could have been done to address these limitations except to choose from a wider pool of volunteer classes to try and avoid participant bias. But even then, it is the ones that don’t want to participate that will solve this limitation but you can’t make someone participate in research if they don’t want to.
One set of questions raised by our journal club was should a SET rating of 3 be sufficient to persist with an active learning strategy? Should SETs dictate the learning strategy we implement? Should SETs dictate how faculty evaluation committees reward good teaching? These questions remind me of a large meta-analysis of SETs which concluded with a scathing indictment that I think deserves quoting in full:
In turn, our findings indicate that depending on their institutional focus, universities and colleges may need to give appropriate weight to SET ratings when evaluating their professors. Universities and colleges focused on student learning may need to give minimal or no weight to SET ratings. In contrast, universities and colleges focused on students' perceptions or satisfaction rather than learning may want to evaluate their faculty's teaching using primarily or exclusively SET ratings, emphasize to their faculty members the need to obtain as high SET ratings as possible (i.e., preferably the perfect ratings), and systematically terminate those faculty members who do not meet the standards. For example, they may need to terminate all faculty members who do not exceed the average SET ratings of the department or the university, the standard of satisfactory teaching used in some departments and universities today despite common sense objections that not every faculty member can be above the average. (Uttl, White & Gonzalez, 2017)
We also distinguished between how we, as instructors explain an activity to our students vs facilitating the activity and considered why facilitation might be more effective at reducing student resistance to active learning than explanation? I wonder if this is the difference between attending to the cognitive vs the affective domain of learning? Explaining why a particular activity is good for students gets at their rational side. But resistance, I think, is rarely rational. In contrast, facilitating an active learning experience allows students to directly interact with the instructor and may alleviate any tension or fear that students might have toward publically performing the activity. Facilitating the activity is a way for instructors to join students in the messy business of learning. And I think joining the students in learning rather than standing aloof while they carry out the activity may be critical to students feeling better about risking failure in front of their peers.
Our discussion ended with acknowledging that studies have shown that student engagement promotes student learning outcomes, but oftentimes students are internally engaged with the ideas/content. It is difficult (impossible?) to assess this level of engagement in contrast to the more easily assessed degree of classroom noise. One aspect that instructors who implement active learning need to be careful with is assuming that talking during the activity indicates student engagement and therefore that learning is occurring. Sometimes that noise is actually a discussion of who won the hockey game the night before or which Netflix show is currently being binge-watched. Instructors need to remember that sometimes the best engagement with learning occurs when the class is quiet as a result of students thinking about the implications of what was just discussed.
Like all teaching and learning, context matters.
Uttl, B., White, C. A., & Gonzalez, D. W. (2017). Meta-analysis of faculty’s teaching effectiveness: Student evaluation of teaching ratings and student learning are not related. Studies in Educational Evaluation, 54, 22–42. https://doi.org/10.1016/j.stueduc.2016.08.007
What was interesting to me is that the study found that student resistance to active learning was not high - most students appreciated and engaged in the active learning activities. The difficulty for many instructors working in an environment in which student evaluation of teaching (SET) is used to assess teaching ability, Likert ratings below 4 may be considered to be low. However, this paper reminds us that a SET rating above 3 means that most students appreciate the experience. But excellent teachers may be accustomed to the vast majority of students appreciating the learning experience, not just most students. So maybe a significant finding of this research is that teachers need to be satisfied - find solace - in the fact that active learning reaches most students.
The other interesting finding is that students noted that instructors typically use explanations to alleviate resistance to active learning. That is, that instructors explain to students what is expected of them for the activity and how it will benefit their learning. The statistical analysis in this paper, however, indicates that facilitation efforts may be more effective at reducing student resistance. Facilitation strategies include instructor demeanor toward the students and the activity, inviting students to ask questions about the activity, walking around the room to assist student teams, soliciting student feedback on the activity, and confronting students unengaged in the activity. The last two seemed to have the lowest impact on alleviating student resistance.
So, some good advice backed by evidence for responding to student resistance to active learning: facilitate student engagement with the activities by engaging with the students during the activity and being interested/happy/excited about the activity and how students are interacting with the activity.
There are limitations to the study and I appreciate the authors clearly indicating these. The participating classes were self-selected. The participating classes were not observed by a third party to corroborate the student and instructor ratings of the nature of the class environment. But the authors correctly indicate that some of the key aspects of the study would not have been caught by this triangulation as it is difficult to know/observe/measure students internal environment regarding how they value or emotionally respond to a learning activity. In addition, the low number of classes and student enrollment means that there is the concern of variability within the class being greater than the variability between classes. But this is also difficult to address because different students will perceive the course differently - not all students will observe how the instructor is facilitating their peers’ learning because they themselves are engaged with the learning activity. I am not sure what could have been done to address these limitations except to choose from a wider pool of volunteer classes to try and avoid participant bias. But even then, it is the ones that don’t want to participate that will solve this limitation but you can’t make someone participate in research if they don’t want to.
One set of questions raised by our journal club was should a SET rating of 3 be sufficient to persist with an active learning strategy? Should SETs dictate the learning strategy we implement? Should SETs dictate how faculty evaluation committees reward good teaching? These questions remind me of a large meta-analysis of SETs which concluded with a scathing indictment that I think deserves quoting in full:
In turn, our findings indicate that depending on their institutional focus, universities and colleges may need to give appropriate weight to SET ratings when evaluating their professors. Universities and colleges focused on student learning may need to give minimal or no weight to SET ratings. In contrast, universities and colleges focused on students' perceptions or satisfaction rather than learning may want to evaluate their faculty's teaching using primarily or exclusively SET ratings, emphasize to their faculty members the need to obtain as high SET ratings as possible (i.e., preferably the perfect ratings), and systematically terminate those faculty members who do not meet the standards. For example, they may need to terminate all faculty members who do not exceed the average SET ratings of the department or the university, the standard of satisfactory teaching used in some departments and universities today despite common sense objections that not every faculty member can be above the average. (Uttl, White & Gonzalez, 2017)
We also distinguished between how we, as instructors explain an activity to our students vs facilitating the activity and considered why facilitation might be more effective at reducing student resistance to active learning than explanation? I wonder if this is the difference between attending to the cognitive vs the affective domain of learning? Explaining why a particular activity is good for students gets at their rational side. But resistance, I think, is rarely rational. In contrast, facilitating an active learning experience allows students to directly interact with the instructor and may alleviate any tension or fear that students might have toward publically performing the activity. Facilitating the activity is a way for instructors to join students in the messy business of learning. And I think joining the students in learning rather than standing aloof while they carry out the activity may be critical to students feeling better about risking failure in front of their peers.
Our discussion ended with acknowledging that studies have shown that student engagement promotes student learning outcomes, but oftentimes students are internally engaged with the ideas/content. It is difficult (impossible?) to assess this level of engagement in contrast to the more easily assessed degree of classroom noise. One aspect that instructors who implement active learning need to be careful with is assuming that talking during the activity indicates student engagement and therefore that learning is occurring. Sometimes that noise is actually a discussion of who won the hockey game the night before or which Netflix show is currently being binge-watched. Instructors need to remember that sometimes the best engagement with learning occurs when the class is quiet as a result of students thinking about the implications of what was just discussed.
Like all teaching and learning, context matters.
Resources
Finelli, B. C. J., Nguyen, K., Demonbrun, M., Borrego, M., Prince, M., Husman, J., … Waters, C. K. (2018). Reducing student resistance to active learning: Strategies for instructors. Journal of College Science Teaching, 47(5), 80–91.Uttl, B., White, C. A., & Gonzalez, D. W. (2017). Meta-analysis of faculty’s teaching effectiveness: Student evaluation of teaching ratings and student learning are not related. Studies in Educational Evaluation, 54, 22–42. https://doi.org/10.1016/j.stueduc.2016.08.007
Subscribe to:
Posts (Atom)