Tuesday, 18 June 2019

the Goldilocks point between lecture and active learning: 20-60%

This paper by Henderson et al (2018) suggests that most students will not penalize instructors who use active learning with low student evaluations of teaching. Interestingly, those instructors who used lecture more than 60% of the time reported no change in their student ratings of instruction (SRIs) whereas those who reported using lecture less than 20% of class time reported decreased SRIs. Those instructors whose implementation of active learning was between 20 to 60% typically reported an increase in SRIs.

A limitation that the authors fully acknowledge is that these data were collected by surveying faculty who had completed their four-day new professor workshop (NPW) during the first couple of years of their academic appointment. Their NPW has the goal to develop in new professors the capacity to implement active learning strategies. However! These are self-reports. SRIs were not actually enumerated and statistically analyzed. This is an issue for this study. Are instructors remembering their SRIs differently? How do they interpret their SRIs? Is a median above three (assuming a 5-point Likert scale) considered good to them? Is a median below four considered bad? There are differences in institutional culture as exemplified among the departments at my university.

Despite this major limitation, this is the beginning of gathering real evidence for how students respond to active learning.

Note that the authors clearly explain their position on the use of SRIs - that they should be interpreted carefully and only be one aspect of triangulating teaching efficacy. They suggest, however, that SRIs probably don’t even assess teaching efficacy and that other aspects of multi-faceted evaluation of teaching are necessary to actually assess teaching efficacy.

In addition, I appreciate that the authors clearly explain that the correlations between the amount of lecturing and SRIs are not set in stone. There were some instructors in their study who lectured less than 20% of the time who reported improved SRIs while there were also those in the Goldilocks region of 20-60% that reported decreases in SRIs. They make the important point that how active learning is received by students is heavily influenced by how instructors set up and facilitate the active learning activity in addition to the particular instructional/department/program context/culture. These all influence student expectations for instruction and learning and if the educational experience does not match students’ expectations, that is when students will award poor SRIs.

Teaching and learning are context dependent. The answer to how much active learning should be implemented in any particular class is... it depends.

Resources

Benton, S. L., & Ryalls, K. R. (2016). Challenging misconceptions about student ratings of instruction. IDEA Paper, 58(April), 1–22.

Henderson, C., Khan, R., & Dancy, M. (2018). Will my student evaluations decrease if I adopt an active learning instructional strategy? American Journal of Physics, 86(12), 934–942.

Thursday, 13 June 2019

what is the thesis of your course?


Two online articles spoke to me today spurring me to articulate how I approach my courses. The one by Ryan Boyd is an essay review of Josh Eyler's recent book How Humans Learn and the other by Kevin Gannon considers how to manage survey courses. Both consider what interferes with student learning and suggest that part of the issue is how some university courses are taught: large passive lectures with too much content. I am finally reading Paolo Freire's book Pedagogy of the Oppressed, a book I have been meaning to read for many years, in which one of the sources of oppression he suggests is the volume of content that is found in some courses. This is what Gannon refers to as the fire hose approach to teaching and learning: students open their brains wide and teachers rapidly pour the content in. I think this is akin to Freire's articulation of the banking model of education in which teachers transmit the information and students receive it banking it for later use in their brains. What Freire articulates in his book is that this is a form of oppression because it prevents students from thinking, it represses their ability to develop their cognition.

A few of weeks ago at the University of Alberta's Festival of Teaching and Learning, Dr Jeanette Norden gave the keynote address in which she advocated educators to teach less, better. (Notice the key placement of the comma in that phrase: it reminds me of Eats, Shoots & Leaves.) She urged educators to teach less content but to support students' learning of that content to a deeper level.

All of these recent sources that I have been thinking about suggest the same thing: in our courses, give students the room to think about what they are learning. Otherwise, students will only learn what we are teaching on a superficial level. Content heavy courses will not nurture our students' cognitive abilities.

Now, granted, there is a continuum here that is dependent upon how advanced a course is, students' previous experiences, and the goal of the course. Some courses will be more content heavy than others depending upon this constellation of factors. So how do we design our courses to take this into account?

The way I do it is similar to how I orient my overall assessment of a students' essay: what is the thesis of this paper and is that thesis well supported and articulated? The same question can be applied to any course we teach: what is the thesis of my course and does its design support that thesis? When approached this way, we are encouraged to curate the course content as suggested by Gannon - no longer is a fire hose needed to deliver an abundance of course content if only a few examples will support the course's thesis. Similarly, if we well articulate our thesis, then, as suggested by Boyd, we can begin to ask the salient questions that our course seeks to answer in order to resolve or support the thesis.

Resources

Boyd, R. (2019). Beautiful questions: “How Humans Learn” and the future of education. Los Angeles Review of Books. May 27.

Freire, P. (2018). Pedagogy of the oppressed (50th anniv). New York, NY: Bloomsbury Academic.

Gannon, K. (2019). How to Fix the Dreaded Survey Course. The Chronicle of Higher Education. May 7.

Monday, 3 June 2019

2 hrs outside of class for every hr spent inside of class

This is the advice I learned as a student and the advice that many academic deans gave to our Augustana students since I started teaching in 1990. It has always generally been assumed that for students to be academically successful at college/university, they had to spend a minimum (yes, a minimum) of two hours studying outside of class for every hour spent inside of the class. If the typical academic program is 5 courses per term and each of those courses is meeting for three hours per week that sums to a minimum of 45 hrs per week (15 in class, 30 outside of class) learning academic skills and course material. I have always thought as a student and as a teacher, that a full-time undergraduate program of study is more than a full-time job.

Yes, this means that students in programs that require more than 15 hrs of class time per week (e.g. art studio, science courses with labs, engineering) expect/assume more time on task. This may not be that different from the Arts in which students have to read multiple novels or monographs for a given course. This is why I advocate for lab work during the 1st two years of students' undergraduate programs to be self-contained and not require any homework other than preparation for the lab. For more senior labs, these should be lab courses, not lec-lab courses. Otherwise, the expectations for students to spend time on coursework becomes unreasonable.

So why am I bringing this up now?  A couple of years ago, I remember reading on my end of term student ratings of instruction a couple of comments responding to my start of term advice that to do well students should be spending a minimum of two hours studying outside of class for every hour spent inside of class. The comments amounted to shock and horror that an instructor, that an educational institution would have such expectations of their students. That such expectations were unreasonable because students had a life outside of school. When I discussed this with my colleagues, the more recent ones to the teaching profession shared the students' shock and horror - a 2:1 ratio of outside:inside course work was clearly unreasonable.

When did this change? When did the understanding change that learning does not require time on task? All of the evidence I have read suggests that academic success depends upon time on task and that one of the best things that instructors can do for their students is to encourage their time on task on educationally purposeful activities. This includes doing homework (reading the text, doing practice problems), discussing class material in a study group, preparing lab reports, conducting research for term papers, spending time in the studio painting or drawing, spending time rehearsing lines, doing the grunt work of memorizing vocabulary and grammatical rules for language acquisition, memorizing the chemical formulae for functional groups, practicing conversing in the language of the discipline.

Does anyone think that three hours per week in-class is sufficient to master their coursework? Where is this resistance to the sage advice of two hours outside of class for every hour inside of class coming from? Who thinks that students do not need to spend time doing the hard messy work of learning on their own time outside of class?

In the resources below it seems like the 2:1 adage is still prevalent. But I do like Lolita Paff's Faculty Focus article and the USU estimate study hours worksheet to be a much better-nuanced consideration of this issue. Bottom line from those two resources is that it depends upon the type and difficulty of the course. But it still looks to me that it ends up being approximately 2:1 but that the hours outside of class may be differentially allocated depending upon the constellation of courses in which a student is enrolled for a given term. Something that we can do to circumvent the student complaint that they did not perform as they thought they would on an exam given the amount of time spent studying for the exam is to discuss efficient vs weak study/learning strategies. Many students still use passive learning strategies (e.g. reading over their notes, using flashcards, reading the text with a highlighter in hand) rather than active learning strategies (e.g. rewriting/reorganizing their notes, engaging in retrieval practice, reading the text with pencil and paper in hand, spacing rather than massing their study, and mixing their study time by attending to different courses). I have found that having this conversation and having students think about their approach to studying often transforms a mediocre student into a high-performing student. Many students have found the book Make It Stick to be an invaluable resource.

Resources





University website resources

Wednesday, 6 February 2019

responding to students' resistance to active learning

Our SoTL journal club at Augustana recently met to discuss an interesting article by Finelli, et al 2018 (Reducing student resistance to active learning: Strategies for instructors published in the J Coll Sci Teaching - see reference below). This paper reports the use of a student survey of instructional and facilitation practices and correlated these with students' self-reports of engagement and value of the active learning activities. Interestingly, the study also surveyed instructor’s self-reports of their teaching and facilitation efforts/activities and the study found no difference - teachers and students both perceived the educational setting similarly.

What was interesting to me is that the study found that student resistance to active learning was not high - most students appreciated and engaged in the active learning activities. The difficulty for many instructors working in an environment in which student evaluation of teaching (SET) is used to assess teaching ability, Likert ratings below 4 may be considered to be low. However, this paper reminds us that a SET rating above 3 means that most students appreciate the experience. But excellent teachers may be accustomed to the vast majority of students appreciating the learning experience, not just most students. So maybe a significant finding of this research is that teachers need to be satisfied - find solace - in the fact that active learning reaches most students.

The other interesting finding is that students noted that instructors typically use explanations to alleviate resistance to active learning. That is, that instructors explain to students what is expected of them for the activity and how it will benefit their learning. The statistical analysis in this paper, however, indicates that facilitation efforts may be more effective at reducing student resistance. Facilitation strategies include instructor demeanor toward the students and the activity, inviting students to ask questions about the activity, walking around the room to assist student teams, soliciting student feedback on the activity, and confronting students unengaged in the activity. The last two seemed to have the lowest impact on alleviating student resistance.

So, some good advice backed by evidence for responding to student resistance to active learning: facilitate student engagement with the activities by engaging with the students during the activity and being interested/happy/excited about the activity and how students are interacting with the activity.

There are limitations to the study and I appreciate the authors clearly indicating these. The participating classes were self-selected. The participating classes were not observed by a third party to corroborate the student and instructor ratings of the nature of the class environment. But the authors correctly indicate that some of the key aspects of the study would not have been caught by this triangulation as it is difficult to know/observe/measure students internal environment regarding how they value or emotionally respond to a learning activity. In addition, the low number of classes and student enrollment means that there is the concern of variability within the class being greater than the variability between classes. But this is also difficult to address because different students will perceive the course differently - not all students will observe how the instructor is facilitating their peers’ learning because they themselves are engaged with the learning activity. I am not sure what could have been done to address these limitations except to choose from a wider pool of volunteer classes to try and avoid participant bias. But even then, it is the ones that don’t want to participate that will solve this limitation but you can’t make someone participate in research if they don’t want to.

One set of questions raised by our journal club was should a SET rating of 3 be sufficient to persist with an active learning strategy? Should SETs dictate the learning strategy we implement? Should SETs dictate how faculty evaluation committees reward good teaching? These questions remind me of a large meta-analysis of SETs which concluded with a scathing indictment that I think deserves quoting in full:

In turn, our findings indicate that depending on their institutional focus, universities and colleges may need to give appropriate weight to SET ratings when evaluating their professors. Universities and colleges focused on student learning may need to give minimal or no weight to SET ratings. In contrast, universities and colleges focused on students' perceptions or satisfaction rather than learning may want to evaluate their faculty's teaching using primarily or exclusively SET ratings, emphasize to their faculty members the need to obtain as high SET ratings as possible (i.e., preferably the perfect ratings), and systematically terminate those faculty members who do not meet the standards. For example, they may need to terminate all faculty members who do not exceed the average SET ratings of the department or the university, the standard of satisfactory teaching used in some departments and universities today despite common sense objections that not every faculty member can be above the average. (Uttl, White & Gonzalez, 2017)

We also distinguished between how we, as instructors explain an activity to our students vs facilitating the activity and considered why facilitation might be more effective at reducing student resistance to active learning than explanation? I wonder if this is the difference between attending to the cognitive vs the affective domain of learning? Explaining why a particular activity is good for students gets at their rational side. But resistance, I think, is rarely rational. In contrast, facilitating an active learning experience allows students to directly interact with the instructor and may alleviate any tension or fear that students might have toward publically performing the activity. Facilitating the activity is a way for instructors to join students in the messy business of learning. And I think joining the students in learning rather than standing aloof while they carry out the activity may be critical to students feeling better about risking failure in front of their peers.

Our discussion ended with acknowledging that studies have shown that student engagement promotes student learning outcomes, but oftentimes students are internally engaged with the ideas/content. It is difficult (impossible?) to assess this level of engagement in contrast to the more easily assessed degree of classroom noise. One aspect that instructors who implement active learning need to be careful with is assuming that talking during the activity indicates student engagement and therefore that learning is occurring. Sometimes that noise is actually a discussion of who won the hockey game the night before or which Netflix show is currently being binge-watched. Instructors need to remember that sometimes the best engagement with learning occurs when the class is quiet as a result of students thinking about the implications of what was just discussed.

Like all teaching and learning, context matters.

Resources

Finelli, B. C. J., Nguyen, K., Demonbrun, M., Borrego, M., Prince, M., Husman, J., … Waters, C. K. (2018). Reducing student resistance to active learning: Strategies for instructors. Journal of College Science Teaching, 47(5), 80–91.

Uttl, B., White, C. A., & Gonzalez, D. W. (2017). Meta-analysis of faculty’s teaching effectiveness: Student evaluation of teaching ratings and student learning are not related. Studies in Educational Evaluation, 54, 22–42. https://doi.org/10.1016/j.stueduc.2016.08.007



Tuesday, 18 September 2018

incorporating social media and back-channel communication

I am beginning to be less and less concerned with online learning. Before I used to consider it to be inferior to what was happening face-to-face. But now, I think it is just a different mode of the educational medium. As Brookfield suggests in this 11th chapter, the F2F classes rarely live up to the billing or potential. Classroom teaching can be executed well or poorly. Online teaching is the same. But I do wonder if the best online learning experience can be favourably compared to the best F2F learning environment? Perhaps the issue is that it is so context dependent being affected by the skills and inclinations of the teacher, the student cohort, and discipline being studied.

Regardless, Brookfield, similar to Bowen, suggests that we make use of whatever educational tools are at our disposal if they make sense in terms of facilitating student learning. For Brookfield, social media and back-channel communication is a teaching tool that can facilitate democratization of the classroom by enabling quieter students to provide input toward the educational environment. In addition, it provides a conduit to instructors for the lens of students’ voices when critically reflecting on our teaching. The advantage here is that it can happen in real time as we are teaching. Critical to its success in conveying students’ voices is that students be permitted to log in with an anonymous name/handle/identifier. I am not sure how I feel about that. I tire of the social irresponsibility that happens with anonymous posting to blogs and online articles. I think we need to take personal responsibility for our expressed thoughts. I understand that Brookfield is making the case that anonymity is crucial for enabling students to feel safe in voicing their concerns or confusion but I have witnessed so many examples of online conversations descending to ad hominem. On the other hand, the classroom environment may have sufficient social constraints that students will regulate themselves. There will be those who want to learn rather than read vulgarity or needless harassment of other students or the instructor. Brookfield does assert that it is necessary to lay the ground rules for using social media as a back-channel to the instructor for questions/issues. Some of the examples he cites I have seen used well in instructional workshops: TodaysMeet, Twitter, PollEverywhere, among others. I wonder if my own LMS, Moodle has anything similar? The Forum module in Moodle is simply too slow/cumbersome to act as a back-channel. Are there other social media equivalents in Moodle?

Brookfield also makes the case that social media is good for the lens of our colleagues. We can allow our colleagues access to our online record or to even observe the social media feeds to get a taste of our teaching without the necessity of being physically in our classrooms.

I do wonder, however, if these teaching strategies and educational technology tools are as significant for my smaller classes? I think they could work well in my first-yr biology courses for which student enrolment is typically between 70-90 students. But in my more senior courses, the enrolment smaller: 30-50 in 2nd yr and 10-20 for 3rd and 4th year. Maybe in the 2nd year, but I don’t think it makes sense in my smaller enrolment courses at the 3rd and 4th yr. Also, using social media assumes that my classroom does not have many moments in which I am circulating among students as they work and discuss a problem. I can see social media use working very well when I am lecturing - it still happens often enough - but not when students are working in their groups. I get around to them when they have a confusion that needs clarification.

Bottom line, I think, is that social media can be a useful teaching and learning tool but that it requires judicious use that is context-dependent. I like Jose Bowen’s suggestion - ask the students what they want/need to support their own learning. Many of my students have indicated that they do not particularly like learning online. But I wonder if what they are really saying is that they do not like the work of learning - online teaching, done well,  typically requires reading and deep processing. Or I wonder if what they are indicating is that they view their online world as their personal world and they desire to keep it separate from their more formal learning world?

Not sure…

Resources

Tuesday, 28 August 2018

learning from theory

I like using theory as part of my critical reflections. I seem to be doing already what Brookfield describes as scholarly personal narrative (SPN) in his tenth chapter of Becoming a Critically Reflective Teacher. But what I need to do to make it more critical is too actively search and incorporate/respond to theory that contrasts with my own understanding of my personal experience. Currently, I tend to use theory to justify my own understanding of events. Instead, Brookfield advocates to intentionally include alternative theories that help to problematize our own interpretation of our experiences. Doesn't mean that the alternatives will be correct and I will be wrong. Rather, what it does is help me critically assess my experiences and this is the point of being critically reflective. Being reflective is not useful if it simply continues to corroborate our own thinking. We need to use the literature to articulate what we understand but also to examine our experience from other points of view or other understandings of the world. This must be done courageously with an unflinching eye on what is truly happening in our classrooms.

Theory, for Brookfield, is not simply complicated and difficult text. Instead, it is simply another lens to consider our teaching and learning. For people without a critically reflective learning community, the literature can be used in their place. But also, in the context of a faculty learning community, it can help ensure that the community does not simply become mutually reinforcing of hegemony and status quo. Alternatively, it can help to nudge community members into new ways of understanding of what is occurring in our classrooms, with our students, and with ourselves in that particular teaching and learning context.

Scholarly personal narratives embed theory into the personal experience. This is what I have been trying to do with my blog: I reflect on an experience in the light of a particular paper or book I have read. The theory and the experience are woven together in an attempt to understand the experience and learn from it. We do not learn from experience… we learn from reflecting on experience - to paraphrase Dewey.

Brookfield's discussion of repressive tolerance is something that I have not heard of before but have certainly observed and experienced in retrospect. By giving equal voice to all opinions and ideas, the marginalized and radical remain at the periphery. As an educator, we need to give students a voice, but we do need to rebalance the voices so that the marginal finds space to be at the centre. By tolerating all voices equally, the marginalized remains at the margins simply because their voice gets drowned out by the hegemonic and voices of the status quo. As instructors, we do have a responsibility to bring what we understand to be true to the centre: The majority voice is not always the correct voice. Repressive tolerance is found on news talk shows where journalists bring in contrasting voices to try to produce a balanced discussion even if the alternative view is ridiculous and unfounded.

Where in my classes do I inadvertently practice repressive tolerance?

Something else that Brookfield raises in this chapter is an idea that I have understood for some time. By virtue of being raised in Western Civilization, I am effectively racist and sexist. My upbringing in this culture has embedded in my thinking racist ideas and sexist views of my place in society - examples of hegemonic thinking. All I can do is be constantly vigilant and exercise inclusive thinking with action.

Resources

Brookfield, S. D. (2017). Learning from theory. In Becoming a critically reflective teacher, 2nd ed, (pp. 171-187). San Francisco: CA, Jossey-Bass, A Wiley Brand.

Marcuse, H. (1969). Repressive Tolerance. In R. P. Wolff, B. J. Moore, & H. Marcuse (Eds.), A Critique of Pure Tolerance (pp. 95–137). Boston, MA: Beacon Press. 

Wednesday, 15 August 2018

using personal experience

In his 9th chapter of Becoming a Critically Reflective Teacher, Brookfield argues that our personal experience as learners can inform our own teaching praxis. The most significant influences on how we teach are those that we experienced as students. Our favourite teachers we want to emulate, our worst teachers we ensure that we never teach in like manner. He provides a number of examples that are readily available to us as learning experiences. Brookfield notes how interesting it is that graduate students who are working as TAs do not apply their own learning experiences in their graduate courses to their TAing of their undergraduate seminars and labs. What we appreciate as students we should consider implementing in our own teaching. The same can be applied to teaching development workshops and conference sessions we attend. In addition to learning content and theory, we can also reflect on how the workshop leader or session presenter is communicating their topic to you as a learner. Are they sharing something of themselves first before they ask that you share something of yourself with the class or with a nearby stranger? In addition, do they explain the rationale for using a particular activity rather than simply speaking their thinking? What is the connection to what they are presenting to what they are asking you to do? Do you do this with your own students? This is an example that Brookfield has experienced at workshops and conferences. The point he is making is to document the situation: Where and when did the experience occur? How did it make you feel? Why do you think you had that response? How can this impact your own teaching? What might your students be experiencing in a similar situation in your own classes?

Finally, a very powerful personal experience that can inform our own teaching is to learn a new skill. This reminds us of what it is like for disciplinary novices to learn our own field of expertise. Brookfield’s analysis is that teachers who have had to struggle to master the material they are teaching typically make the best teachers of that material or skill because they have had to articulate a strategy for learning it themselves. People who have found something easy to learn and master typically do not understand, cannot empathize with others for whom the skills and concepts are difficult to grasp. This gives them insight into their students' feelings and struggles and perhaps an ability to help bridge the novice-expert gap. But, as we age and become more familiar with the material, we can become distanced from what it was like to initially confront the course material. Somehow, we need to occasionally remake our courses, remake ourselves as teachers to bring us closer to where our students are when they first enter our class, enter our course and begin to engage with our material.

As a result of instructor familiarity with the material and thus distancing from novices, peers can become valuable resources for struggling students. I may be the expert in my subject area, but I may be less able to convey ideas about how to master the material. Students who have struggled but persevered may be good mediators of instruction between the teacher (the expert) and struggling student (the novice). This is a good rationale for implementing team-based learning which depends on collaborative teamwork. Brookfield suggests that we need to remember to not always cater to students' preferred modes of learning. We do need to stretch their learning abilities. But the converse is also true - if I am always challenging students with an unfamiliar way of learning (flipping the classroom) students may become fatigued with always being required to be an independent learner.

So is there anything I can do with my own implementation of TBL in my own courses? I could take aspects of TBL and implement them at different times during the term or during a class meeting. I think what many of my students have indicated on my end of term student evaluations of teaching for the last few years is that using TBL all of the time is simply too taxing - they need an educational rest while still participating in the course. This is certainly true for more junior courses. For the most part, students have the stamina and learning ability when they reach their 4th year of undergraduate studies. Even then, I do break up the active learning in our 4th-year capstone course with two or three videos throughout the term - a relatively passive learning experience but one that could be considered to be an educational resting point. I think I need to do more of this in my more junior courses.

Resources

Brookfield, S. D. (2017). Using personal experience. In Becoming a critically reflective teacher, 2nd ed, p 153-170. San Francisco: CA, Jossey-Bass, A Wiley Brand. pp xvi, 286.

Weimer, M. (2017, February 22). When the teacher becomes the student. The Teaching Professor Blog

Weimer, M. (2017, January 25). The benefits of peer learning. The Teaching Professor Blog.

Weimer, M. (2006). The lens of experience: Wisdom of practice. In Enhancing scholarly work on teaching and learning: Professional literature that makes a difference (pp. 53–90). San Francisco, CA: Jossey-Bass, a Wiley imprint.