faculty misunderstanding of Teaching Squares

Teaching squares are a faculty development tool that I have used in the past when I was Associate Dean (Teaching) at the Augustana Campus of the University of Alberta. I have published a short description of this tool as have others  (Haave 2014, Rhem2003, Berry 2011, Oberlies et al 2020). Teaching Squares was developed by Anne Wessely at St. Louis Community College and has been used by many colleges and universities. You can read the details of how to set it up in any of the articles listed in the resources below. 

The key to implementing Teaching Squares to develop the teaching praxis of post-secondary faculty is that it is a tool with which faculty reflect on their own teaching. Thus, the point is not to critique the teaching you are observing in a colleague's class. This includes critiquing both the negative and positive aspects of your colleague's teaching. The point is for Teaching Squares participants to reflect on their own teaching praxis. Thus, when participants see something different in a colleagues' class that should prompt them to think "hmmm... how would I do that in my class?" Or a participant might ask themselves "I wonder if how my colleague is engaging in their class would work in my class?" Or "I don't think I would feel comfortable doing that in my class. I wonder why?"

As you can see from the sample reflective questions above, it is not at all about critiquing (positive or negative) your colleagues' classes but rather Teaching Squares is a means to prompt critiquing your own class and your own approach to teaching. Thus, after participants in a Teaching Square have visited each other's classes at least once and they get together to debrief the experience, they discuss what they have started thinking about their own teaching, not about their colleagues' teaching. 

Unfortunately, many Teaching Squares participants make the mistake of using the after-class group meeting to discuss what they think their colleagues are doing well or correctly in their teaching. Hence, it is disappointing to read a recently published article that perpetuates this mistake stating that in the Teaching Squares debriefing discussion my article (Haave 2014) is cited as advocating that "Observations exclusively focus on positive features in the teaching sessions" (Lemus-Martinez et al 2021). I suspect that the authors misunderstood the statement in my article "The intention of the square is not to criticize each other’s teaching" to mean that only positive aspects of the observed teaching are discussed. A good critique involves a robust discussion of both the strong and weak aspects of whatever is being criticized. I wish that Lemus-Martinez et al (2021) had instead focused on the subsequent sentence in my Teaching Professor article that states "Rather, it’s an opportunity for faculty to reflect on their own teaching in light of colleagues’ teaching examples."

Hopefully, the authors read this blog and can correct their misunderstanding going forward as they continue to use the effective faculty development tool of Teaching Squares.

Resources

Berenson, C. (2017). Teaching squares: Observe and reflect on teaching and learning. Taylor Institute for Teaching and Learning Guide Series, Calgary, AB. https://taylorinstitute.ucalgary.ca/resources/teaching-squares-observe-and-reflect-teaching-and-learning

Berry, D. (2011). Learning by observing our peers. Collected Essays on Learning and Teaching, 1, 99. https://doi.org/10.22329/celt.v1i0.3186

CTLT. (ND). Teaching squares. Centre for Teaching, Learning & Technology, University of British Columbia. https://wiki.ubc.ca/images/c/c5/Teaching-squares.pdf

Haave, N. (2014). Teaching squares: A teaching development tool. The Teaching Professor, 28(10), 1. https://www.facultyfocus.com/articles/faculty-development/teaching-squares-cross-disciplinary-perspectives/

Lemus-Martinez, S. M., Weiler, T., Schneider, G. W., Moulik, S., & Athauda, G. (2021). “Teaching squares”: A grassroots approach to engaging medical educators in faculty development. Medical Teacher, 43(8), 910–911. https://doi.org/10.1080/0142159X.2021.1929903

Oberlies, M. K., Buxton, K., & Zeidman-Karpinski, A. (2020). Adapting evidence-based practices to improve library instruction: Using customized tools to support peer mentoring and observation. New Review of Academic Librarianship, 26(1), 6–30. https://doi.org/10.1080/13614533.2019.1628078

Rhem, J. (2003). Teaching squares. National Teaching and Learning Forum, 13(1), 1–3. https://tomprof.stanford.edu/posting/535

Berenson, C. (2017). Teaching squares: Observe and reflect on teaching and learning. Taylor Institute for Teaching and Learning Guide Series, Calgary, AB. https://taylorinstitute.ucalgary.ca/resources/teaching-squares-observe-and-reflect-teaching-and-learning

promoting educationally rich discussion

This paper by Leupen et al (2020) was of great interest to me because this is something that I struggle with in my own teaching: how to promote educationally-rich discussion among students? In this study, the authors recorded the conversation of a few teams a couple of times during the semester when student teams were discussing the solution to an in-class instructor assigned problem. The results are not much of a surprise to me. Students had deeper conceptual conversations when the questions assigned asked students to consider course material at a higher level of Bloom's taxonomy. 

If the results are unsurprising to me, why do I find the study interesting? I find it interesting because it validates my own suspicions that to promote students' critical thinking requires asking students to take course material and apply it to significant situations. 

The physiology course under investigation in this paper used the instructional strategy of team-based learning (TBL), a flipped-classroom approach to teaching and learning that I use in my own courses. What the authors identify, and I agree with them from my own experience, is that one of the reasons that students may have engaged at a deeper level with higher-order cognitive questions in their study may be because 1. student teams were stable throughout the term and thus students may feel more comfortable with each other in querying each other about their thinking; 2. the pre-class preparation inherent in TBL makes lower-order cognitive questions trivial because the students already know the rote-learning for which they were held accountable before the in-class applications of their learning; 3. that any student may be called upon during the inter-team discussion to explain the rationale for their answer.

I have difficulties doing this well. The difficulty for me is finding the right balance between posing a significant non-trivial problem to students that is difficult for them to solve on their own but is possible to solve as a team. This is Vygotsky's zone of proximal developing in which the potential for learning is rich. But finding that Goldilocks point for any given cohort of students or any given course year-level is difficult. Sometimes I get it right, but many times I make it too easy (trivial) or too difficult.

Regardless of the difficulty of teaching, it is good to hear in this article that the effort seems to be worth it.

Resouces

Leupen, S. M., Kephart, K. L., & Hodges, L. C. (2020). Factors influencing quality of team discussion Discourse analysis in an undergraduate team-based learning biology course. CBE—Life Sciences Education, 19(1), ar7.

Vygotsky, L. S. (1978). Interaction between learning and development. In M. Cole, V. John-Steiner, S. Scribner, & E. Souberman (Eds.), Mind in society: The development of higher psychological processes (pp. 79–91). Harvard University Press. 

i had a nightmare last night...

My family and I enter the hotel lobby after a day of sightseeing. It is a largish hotel with approximately six elevators. Four of the six are out of order. We wait for a while and one finally opens up but the elevator is jammed full - no room for us. So it leaves and we wait for another. A little while later the other working elevator opens its doors. It is full, but there is sufficient room for the three of us to squeeze in the front while the doors close behind us. I push the button for the sixth floor. 

The elevator goes down instead.

The doors open and we are pushed out with the other passengers in the elevator car into what appears to be the utility room. My family and I are pushed out as everyone else exits far enough away that we are unable to reach the elevator doors in time to ride back up.

We wait a while for the elevator to return.

It finally arrives and we are able to enter the elevator in time after everyone else exits. Where are all of these hotel patrons going to in the basement utility room? I don't know - its a dream.

We ride up to the lobby where more people board. I check to ensure that the floor six button has been pushed - it has - it is alight. 

The elevator goes up past the 6th floor. On floor 8 people exit. The floor 6 button is still glowing. Why didn't it stop on the 6th floor? The elevator continues up and stops periodically letting more people off on their floor. This continues until my spouse, daughter and I are the only three in the elevator car. The elevator continues to go up past the 18th floor. I thought there were only 18 floors in this hotel...

The elevator continues to rise and then from above, walls come down into the elevator car separating my spouse, daughter and I into separate cars. I am now alone in my own personal elevator.

As the elevator continues its ascent, the car starts to slowly tip on its side. Slowly enough that I am able to adjust where I am standing so that now I stand on the wall of the car. The car continues to tip until what was up is now down - I am standing on what was the ceiling of the car while the elevator gathers speed until I am riding down and doing loops as if I am on an amusement park roller-coaster. This continues (I do not up-chuck my lunch! how is that possible with this ride? I don't know - its a dream) until the elevator slows down and stops. The doors open, I step outside into the lobby where I started. 

I am alone. Now five of the six elevators are out of order. I look around but do not see my family. The one remaining working elevator opens its doors and I step inside and press floor six. I am the only occupant. The elevator again ascends past floor six. I continue to rise as the elevator gathers speed. The walls of the elevator start to close in around me. I am being squished thin like bread dough is kneaded into a long thin baguette. I am getting thinner - stretched out. I am being squished. I can't breathe. I can't...

I wake up.

I just finished a term in which I had to hastily switch from teaching my courses face-to-face using team-based learning to remotely teaching online. I continued to use team-based learning online using the breakout rooms in Zoom to facilitate my teams working on groups tests and applications of their learning which involved me simultaneously navigating between different breakout rooms, team quizzes on our LMS, and various Google Docs or Google Slides depending upon the in-class activity. On top of that, I acceded to student requests to record our synchronous mini-lectures and learned how to edit and post those online through Zoom. By the end of the term, I was getting nauseated at the thought of moderating another multi-modal synchronous class meeting. I was suffering from the stimulus and cognitive overload in terms of managing a class that was set up for F2F and now was trying to be replicated online. 

As soon as my course grades were submitted at the beginning of May and the term put to bed, I was required to jump in and consider revisions to our core curriculum and degree programs. Revisions that we had been planning for the last year or more but had finally come to the final push to get the details down on paper: which courses were revised? which new courses introduced? Which courses dropped? How does that affect the prerequisite structure of the program? Map our learning outcomes to the constellation of required courses. Think through those learning outcomes to ensure they make sense. Consider how our program changes affected the requirements of other programs. Go back and revise in response to the moving target that is campus-wide curriculum renewal.

Finally, we had our Faculty Council meeting and the necessary motions were passed.

It is now the end of May and I am faced with managing to re-think my courses so that they may be delivered well as an online course for Fall 2020. I am being charged to do this well this time rather than the triage of remote delivery (in contrast to online delivery I am told). I have two courses assigned to me in the fall that require restructuring for online delivery. How do I do that? How is that different from the triage that I just finished of remotely delivering my F2F courses? I have never done online teaching before. Here, I am told is an online course that will get me up to speed. Expect to take some time to complete this online course on online teaching. Becoming an online instructor does not happen overnight. I am told that normally it takes 12-18 months to produce one good online course. I have three months remaining (it is the end of May) to prepare two online courses.

In Alberta, our provincial government has made drastic cuts to education in response to the decline in oil revenues. All budgets, all departments at my university are being cut... drastically. My seconded part-time position as Assoc Director of our CTL is being closed at the end of June. As a result, I am asked to pick up a third course to teach online for the fall term. Ok, I say, I'll prepare one course per month: June, July, August. But don't forget to keep up with your research and by the way, would you also please help your colleagues think through how to transition to online learning (not remote delivery). Sure, I can do that.

So, first-year biology will be done in June, 2nd-year Molecular Cell Biology I can convert in July, and then I will try and squeeze in a revisioning of my 3rd yr biochemistry course in August. I can do that. 

But wait! It looks like there is a very good chance that we will still be online in the winter term. I am told to prepare for that contingency. So that is two more courses to prepare for the winter after the fall is completed. But wait! there is no time between the end of the fall term and the beginning of the winter term to restructure two more courses for online delivery. I'll have to do those courses also during the three remaining summer months. 

Ok, ok, 3 months times 4 weeks each equals 12 weeks. So five courses divided into 12 weeks equals a little more than 2 weeks time to prepare each of those 5 courses. Phew! can I do this? Oh sure you can, just use your graduate students to help you. My campus is an undergraduate campus - I don't have graduate students. Oh, that's ok, just employ your TAs to give you a hand. No, we don't have TAs, I just told you that we are an undergraduate campus. Oh, that's ok, we are certain that your CTL can give you a hand converting your F2F courses to online courses. No, I told you before that our CTL is experiencing drastic budget and personnel cuts just like everywhere else across the university in response to Jason Kenney's cuts to education. Oh, it appears you will just need to make the transition on your own. But you will be empowered to do that once you complete the online course that teaches you how to teach online.

Oh, but by the way, you will need to pick up a 6th course in the winter term. Remember that our campus has a teaching load of three courses per term and because you are no longer Assoc Director you need to make that up with an extra course in the fall (ok, got that one - first-year BIO), plus another one in the winter. Ok... if that is what I need to do. What are you going to assign me?

Biological Diversity is what we have left for you: first-year biodiversity. But I am a biochemist. Sorry, that's what we have available for you. Thanks for helping us to decrease our sessional instructor budget. But I haven't taken general biological diversity since 1978 when I was in grade 11. I took an invertebrate survey course once as an undergrad but that is all. No problem, you are an experienced teacher you can do this. But I have never taught this course before. I have never taught online before.

No problem, just take this online course that will teach you how to teach online and you will be fine.

But...

I wake up.

active learning as social justice

This is an interesting paper recently published in PNAS. Their meta-analysis of active learning studies that disaggregated under-represented from over-represented student groups suggests that active learning has a disproportionate impact on UR STEM students in terms of both passing a course and exam performance. I cannot specifically speak to the stats used in their analysis, but as far as I can tell it appears to me that the authors have tried to take a robust approach to their statistical analysis.

What I found interesting is that they suggest that implementation of active learning that is "high intensity" had greater benefits on UR students; there was less of an achievement gap between UR and OR students. What is high-intensity active learning? High-intensity active learning is simply courses in which a greater proportion of class time is devoted to active learning activities. The greater amount of class time during which students are engaged in the application of their learning, the narrower the achievement gap between UR and OR students in STEM courses. The question is, does this mean that 100% of the time in active learning activities is the best? I don't think so. There is certainly a Goldilocks balance between the instructor orienting students toward a new concept (lecturing) vs having students discover the skill and knowledge themselves through pre-class assignments and in-class application. And this balance will be different for different teaching and learning contexts: year-level, discipline, student cohort. The master teacher will know how to gauge their particular context and how to balance lecturing with active learning to best meet the learning needs of their students.

So, we know that active learning can improve students' learning overall. And now it appears that under-represented students may benefit disproportionally when active learning is implemented in their classroom. As with all social issues and culture changes, there will be push back from those students who will learn well no matter what type of instructional strategy is implemented in their classroom. In addition, active learning did not erase the achievement gap between UR and OR students it only decreased the gap. Active learning clearly does not address all of the issues influencing students ability to be successful. But implementing active learning appears to be one way of addressing EDI issues in our classrooms.


http://www.theinclusionsolution.me/equity-vs-equality-eliminating-opportunity-gaps-education/

Resources

Freeman, S., Eddy, S. L., McDonough, M., Smith, M. K., Okoroafor, N., Jordt, H., & Wenderoth, M. P. (2014). Active learning increases student performance in science, engineering, and mathematics. Proceedings of the National Academy of Sciences of the United States of America, 111(23), 8410–8415. https://doi.org/10.1073/pnas.1319030111

Theobald, E. J., Hill, M. J., Tran, E., Agrawal, S., Arroyo, E. N., Behling, S., … Freeman, S. (2020). Active learning narrows achievement gaps for underrepresented students in undergraduate science, technology, engineering, and math. Proceedings of the National Academy of Sciences, 117(12), 6476–6483. https://doi.org/10.1073/pnas.1916903117

last class of the term

I just finished teaching my last molecular cell biology class for the fall term and I am feeling out of sorts. Out of sorts because I don't know what to do with myself. For three months my energy has been focused on preparing in-class activities and reading quizzes for each and every class meeting. Team-Based Learning, a highly structured version of the flipped classroom, requires preparation for every class. Activities and questions that worked last year, don't necessarily work for the subsequent cohort of students. So for the last three months, I have been probing my students trying to get a sense of what they do and do not know so that I can provide learning experiences to address their knowledge and skill gap and trying to hold a mirror up to them so that students know what they do and do not know.

Now the class is over, the students are completing the end of term student rating of instruction. And I am at a loss over what to do with myself.

Yes, I have service committees that need my attention, I have papers that need writing and revising, I have reviews that need to be typed. But those were always completed in the background while teaching has been foremost in my mind. Now that is over for the term, I feel disoriented - I no longer have the lodestone of instructing my students to set the priorities for my day. Can I actually take the time to read that article, to browse that recent journal issue, to walk down the hall and say hello to my colleague?

Strangely, I feel guilty about even taking the time to consider going for a coffee with a colleague. This is how all-encompassing the teaching term is for me.

It didn't always use to be this way.

Certainly, when I first started teaching I was always running to stand still. But after a few years and being successfully tenured, the pace became reasonable. I was able to tweak and re-use previous lectures and received great student reviews for my efforts.

Then, about eight years ago, I became bored with the sound of my own voice and experienced the revelation that good lecturing doesn't necessarily equate with good learning. In 2012, I began experimenting with implementing active learning in my classes culminating with most of my classes being reworked with team-based learning. And then the realization inherent with learner-centred teaching that each cohort of students is different; that to be able to meet the needs of each student cohort, each student, required that I continually formatively assess my students in order to understand and meet their learning needs.

That is difficult exhausting work. But such rewarding work! After my last class today, a few students came up and thanked me for my efforts. One student expressed gratitude for my class structure that facilitated their ability to learn from the assigned readings. The reading guides are doing their work. My preparation efforts were well-received.

So now I am sitting in my office wondering what to do with myself. I have decided to sit quietly for a moment and enjoy the fruits of my labour: student learning and gratitude. The teaching and learning experience is fleeting. Just allow me a few minutes to enjoy it before it fades into the ether.

student perceptions of active learning (3)

Before I discuss this paper, let me make it clear that I think the balance between lecture and active learning is completely dependent upon the class context. This is not an either/or situation. Teaching involves careful consideration of what our students need to learn and how to best support their learning efforts. Sometimes this will involve telling students something to clear up a misconception. Sometimes this will involve having our students apply or discuss their learning in order to make their learning stick by having them realize that they have not yet deeply learned a concept. I liken the situation to tuning the dial of a radio tuner.

Thus, whenever someone asks me how much active learning they should introduce into their classroom I answer: it depends. It depends upon the student cohort (their experiences and preparation), the year level of the course, the discipline of the course, whether the course is heavy on learning theory, or producing an assignment, what the learning culture is of the particular campus on which you find yourself teaching. All of these things will influence the right balance of lecturing and active learning that happens in your class. What I can say is that if only one is happening in class, then likely not enough of the other is occurring.

Having said that, I really liked this paper (Deslauriers et al 2019). It seems to me to be a well-controlled and thoughtful paper. The authors used a randomized, cross-over design to determine whether students' feeling of learning (FOL) correlate with their actual learning (as assessed with a test of learning - TOL) in an intro physics course taught with active learning vs lecture.

The results clearly indicate that students' FOL was greater when lecture was used whereas their TOL was greater in the active learning environment. This agrees with the Dunning-Kruger effect which suggests that novices have poor metacognition with which to assess their own learning. Their results also agree with other published results which show that students do not appreciate active learning (van Sickle 2016; Smith et al 2011) despite active learning producing better learning outcomes (Freeman et al 2014).

The FOL survey used in this study allowed the authors to also consider the impact of lecture fluency and they found that FOL was positively impacted by fluency. The literature suggests that fluency can inflate students' perceptions of learning as cited in Deslauriers et al (2019).

In a good mixed-methods design, the researchers followed up a subset of the participating students with interviews and they found that students' perceptions of learning were negatively impacted by the struggle required with active learning. When it was pointed out to them that studies show that cognitive effort positively impacts learning, students suggested that knowledge would influence their perception of learning in active learning courses.

So what is impacting the disconnect among novice students' perceptions of their own learning? As suggested above, one is that novices do not have well-developed metacognition. They have difficulties recognizing good judgement and, as a result, have difficulties judging their own learning. In addition, lecture fluency can mislead students into thinking they have learned something when in fact they have not. Finally, students unfamiliar with the cognitive effort required for active learning may resent the effort required to master a body of knowledge.

This resonates with my own teaching experience. Before I implemented active learning in my own classrooms a number of years ago, I was perplexed when students informed me of their frustration with their own learning because I explained things so well in class that they thought they understood what I was teaching but that perception did not translate into good exam performance. It was that student feedback that compelled me to look for other ways of teaching that would make it explicit to students what they did and did not know, what they had and had not learned. Team-based learning is the active learning structure I have implemented in my courses and it works well by hitting a number of the known factors by which active learning promotes student learning (Ambrose et al 2010; Brown et al 2014; Mazur 2009): pre-class preparation, attempting new problems in class, interacting with peers to practice their understanding of what they have learned. What these strategies do is make apparent to students what they do and do not know thereby enabling them to follow up and address the points of misunderstanding they have. In addition, being able to teach each other is a powerful way to cement their learning and create a robust knowledge structure and mental model of what they are learning that integrates with their existing understanding of their world.

So what are we to do to help students accept and embrace active learning despite the cognitive effort required with this teaching strategy? The authors conclude their paper by reporting on the results of an intervention they ran after receiving their study results. They spent time at the beginning of a subsequent term to show students the results of the research and explain how cognitive effort leads to increased learning. This was a 20-minute presentation of the results of the impact of active learning on learning gains and the influence of fluency on perceptions of learning. The researchers observed that students in the Q&A following the presentation were most interested in the idea that FOL and fluency can mislead their judgement about how their learning is progressing. A student survey showed that most students had a more favourable view of active learning as a result of the initial intervention.

These results agree with the earlier Finelli et al (2018) study that suggested that how instructors prepare students for active learning (explain why they are using a particular active learning strategy) goes a long way to mitigate students' resistance to active learning. More significantly in my mind is that the same Finelli study found that how instructors facilitated an active learning activity played a more important role in students response to active learning: being engaged with students during the activity promoted students' perceptions that active learning was enabling their learning.

Please notice, that as I stated above, in both the Finelli and Deslauriers papers it is not that lecturing is completely absent. Read carefully their papers and you will read that they note that mini-lectures were used as necessary. Similar to how I opened this blog post, it is not that lecturing is bad. It is that the injudicious use of lecture is bad. But of course, this can also be applied to active learning. Active learning will not fix bad teaching. Good teachers will be judicious about their use of lecture and active learning and implement either as the context dictates. Having students engaged in active learning for the sake of active learning is not the lesson here. The lesson here is that good instructors will implement lecturing and active learning as required by the particular context and that this will change from year to year, cohort to cohort, class to class, minute to minute.

This is what makes teaching such an interesting challenge as we support our students' learning efforts.

Resources

Ambrose, S. A., Bridges, M. W., DiPietro, M., Lovett, M. C., Norman, M. K., & Mayer, R. E. (2010). How learning works: Seven research-based principles for smart teaching (1st ed.). San Francisco, CA: Jossey-Bass.

Brown, P. C., McDaniel, M. A., & Roediger, H. L. (2014). Make it stick: The science of successful learning. Cambridge, Massachusetts: Harvard University Press.

Deslauriers, L., McCarty, L. S., Miller, K., Callaghan, K., & Kestin, G. (2019). Measuring actual learning versus feeling of learning in response to being actively engaged in the classroom. Proceedings of the National Academy of Sciences, 201821936.

Finelli, B. C. J., Nguyen, K., Demonbrun, M., Borrego, M., Prince, M., Husman, J., … Waters, C. K. (2018). Reducing student resistance to active learning: Strategies for instructors. Journal of College Science Teaching, 47(5), 80–91. A PDF of this paper is available from Harvard here.

Freeman, S., Eddy, S. L., McDonough, M., Smith, M. K., Okoroafor, N., Jordt, H., & Wenderoth, M. P. (2014). Active learning increases student performance in science, engineering, and mathematics. Proceedings of the National Academy of Sciences of the United States of America, 111(23), 8410–8415.

Kruger, J., & Dunning, D. (1999). Unskilled and unaware of it: How difficulties in recognizing one’s own incompetence lead to inflated self-assessments. Journal of Personality & Social Psychology, 77(6), 1121–1134.

Mazur, E. (2009). Farewell, Lecture? Science, 323(5910), 50–51.

Smith, C. V, & Cardaciotto, L. (2011). Is active learning like broccoli? Student perceptions of active learning in large lecture classes. Journal of the Scholarship of Teaching & Learning, 11(1), 53–61.

Van Sickle, J. R. (2016). Discrepancies between student perception and achievement of learning outcomes in a flipped classroom. Journal of the Scholarship of Teaching and Learning, 16(2), 29–38.

student perceptions of active learning (2)

This study (Smith & Cardaciotto, 2011) is similar to the more recent study published by Jenna Van Sickle. Both studies report positive student outcomes with active learning but that students rated the active experience as less positive than the more passive learning experience. So it is interesting that although there is ample evidence to suggest that active learning is good for students, students do not appreciate the experience. The authors liken this to telling children to eat their broccoli because it is good for them.

The Van Sickle study considered a math course whereas the Smith & Cardaciotto broccoli study considered introductory psychology. Smith & Cardaciotto state at the end that active learning activities need to be embedded in sound pedagogy and not simply involve students “doing” something. I am not sure why they tacked this on at the end of their paper as their study design did not address this at all. It is an important consideration, but their study design has nothing to say about this.

A limitation of this study in terms of it being integrated with other active learning studies is that the authors interpreted active learning as any activity that engaged the students cognitively. Thus, their active learning activities were done outside of class rather than the more typical understanding that active learning involves transforming what happens in the traditional didactic lecture - the activities happen inside of the classroom. They make the comparison that what they are doing is similar to the sciences which have a didactic lecture associated with an active lab or tutorial. What is interesting is that most of the published research suggests that active learning in science classes regardless of an active lab or tutorial will promote student learning outcomes. This broccoli study is odd this way thinking that active learning outside of class will do the trick. And even their two-course modules on brain and behaviour showed no differences between the content review and the active learning conditions which they suggest is because those course sections use more active learning during class meetings.

Another limitation is that they did not specifically consider student learning outcomes but rather only student perceptions of their own learning. They acknowledge this and request that subsequent studies specifically consider student learning outcomes as exam or grade results. They cite literature that indicates that students self-reports of learning correlates with actual learning outcomes and so can indicate an impact on student learning. But still, this study did not specifically study this. The reason they give is that the different instructors administered different exams and thus were not comparable.

What I find odd is that the authors make the assumption that students can accurately assess their learning and thus student perception surveys can indicate student learning outcomes to some extent though they do explain that this needs to be studied directly. The reason that I find this odd is that although the authors cite a study indicating the reliability of student perception reports of their learning, the Dunning-Kruger effect suggests that weak students over-estimate their learning whereas good students under-estimate their learning. So it does not suggest that student self-reports are reliable measures of their own learning.

More recent students (e.g. Finelli et al, 2018) suggest that the manner in which instructors explain and facilitate the learning activities in their classroom can go a long way to mitigate students' resistance to learning

Resources

Finelli, B. C. J., Nguyen, K., Demonbrun, M., Borrego, M., Prince, M., Husman, J., … Waters, C. K. (2018). Reducing student resistance to active learning: Strategies for instructors. Journal of College Science Teaching, 47(5), 80–91.

Kruger, J., & Dunning, D. (1999). Unskilled and unaware of it: How difficulties in recognizing one’s own incompetence lead to inflated self-assessments. Journal of Personality & Social Psychology, 77(6), 1121–1134. https://doi.org/10.1037//0022-3514.77.6.1121

Smith, C. V, & Cardaciotto, L. (2011). Is active learning like broccoli? Student perceptions of active learning in large lecture classes. Journal of the Scholarship of Teaching & Learning, 11(1), 53–61. Retrieved from https://josotl.indiana.edu/article/view/1808/1805

Van Sickle, J. R. (2016). Discrepancies between student perception and achievement of learning outcomes in a flipped classroom. Journal of the Scholarship of Teaching and Learning, 16(2), 29–38. https://doi.org/10.14434/josotl.v16i2.19216

Wieman, C. E. (2014). Large-scale comparison of science teaching methods sends clear message. Proceedings of the National Academy of Sciences of the United States of America, 111(23), 8319–8320. https://doi.org/10.1073/pnas.1407304111