Sunday 29 March 2015

grading homework for completion vs a portion for quality

The March 2015 issue of the Teaching Professor has an interesting commentary on a recent study that considers the impact on student learning of awarding completion marks for homework vs grading a random portion (10%) of student homework according to a quality control rubric (grading rubric). What they found was that the quality of the student work was higher when a random proportion of their homework was graded rather than simply receiving a mark for attempting (completing) the homework. In addition, they found that student learning outcomes (exam marks) were modestly improved with the randomized marking.

I am glad to read this because I have been using this approach for many years with students' writing dossiers in my biology capstone course. There, my students must prepare a two-page type-written response to the day's assigned reading for entry into the class. That is a lot of pages to mark! Many years ago I attended a critical thinking conference by Richard Paul in which he advocated that it was important to encourage students to write daily to help them articulate their thinking. To encourage attention to the assignment he advised that only a portion of their writing dossier need be marked.

My tweak to this approach has been to have students identify what they consider to be their weakest and strongest piece in their writing dossier explaining why they think each is so. To reward their critical analysis of their own writing I assure students that I will mark their best but only comment (not grade) their worst provided they articulate what they think makes each strong or weak. In addition, I'll randomly choose a couple more writing pieces to grade.

It's nice to have the evidence in support of this approach to marking selections of student work.

Resources


Galyon CE, Voils KL, Blondin DA, and Williams RL. 2015. The effects of randomized homework contingencies on college students’ daily homework and unit exam performance. Innovative Higher Education, 40(1): 63-77.

Weimer M. 2015. Designing homework that enhances learning. The Teaching Professor 29(3): 2-3 http://www.magnapubs.com/newsletter/the-teaching-professor/111/Designing-Homework-That-Enhances-Learning-13405-1.html


Tuesday 24 March 2015

McCalla Professorship and learning philosophies

This year I was awarded a McCalla Professorship by the University of Alberta. So, for the next year I am carrying out a study that will investigate whether or not developing students' learning philosophies have an impact on their intellectual development and the learning outcomes for the first year biology course I'll be teaching in Fall 2015. Many of you have requested more information about the study, so I have posted below the rationale for the study that I submitted to the awards committee.

*********************************************************************************

Developing Students’ Metacognition through Learning Philosophies:
Impact on Student Learning Outcomes
Neil Haave

"We do not learn from experience . . . we learn from reflecting on experience" - John Dewey

A McCalla Professorship will allow me to investigate whether developing students’ learning philosophy enhances their intellectual development and course learning outcomes.
While I was Divisional Registration Officer for Augustana in the 1990s, I met with students to discuss their programs and advise them which courses to take in the coming year. Their Program Planning Form nicely organized their courses into the different requirements for graduation. However, students seemed to view their education as a list of boxes to be checked. Students did not understand the overarching coherence in their major and general education requirements. Curricular coherence was present but Augustana never explicitly explained it to students. As a result students collected a pile of bricks without assembling them into a home1.
I subsequently served as Chair of Science for ten years and my accumulated curricular experience later enabled me to effectively chair the ad hoc core curriculum review committee which produced Augustana’s current core. We constructed a general education that met the breadth needs of our students, inculcated in our students the particular values held by the Augustana professoriate, and developed students’ communication, thinking, and research skills. In the late 2000s then Augustana Dean Roger Epp noticed that students displayed difficulties articulating their skills to potential employers. Similarly, I had noticed students having difficulty applying prior learning from pre-requisite classes. Students engaged in a learning cycle of memorize-regurgitate-purge as a result of not integrating their learning experiences into an interconnected and robust knowledge structure.
Dean Epp appointed me Associate Dean (Teaching) from 2010-13 to consider how to better engage students’ in their education so they could articulate their learning and skills to themselves and others. This resulted in the 2012-13 Augustana e-portfolio pilot during which I realized students were not asking questions about their own learning. I had inappropriately assumed that students were metacognitively engaged in their learning processes2.
During the e-portfolio pilot I began having my students consider the what, why, and how of their learning while they reflected on the artifacts they collected in their e-portfolio. Each of these questions metacognitively engaged students in their learning through development of their own learning philosophy3 enabling students to make connections among their different learning experiences in addition to their own personal lives.
Developing students’ metacognition has been suggested to improve students’ academic achievement2,4,5 possibly by developing students’ critical thinking6. A lack of metacognition has been correlated with students’ inability to assess their own knowledge and academic abilities7,8. Evidence exists that student learning outcomes may be improved by attending to metacognition through activities that use personal response systems9, in-class writing and discussion assignments10, and correlates with students’ mastery goals11. Long term benefits of using metacognitive prompts during students’ learning have been shown for primary and secondary school students12 but not for post-secondary students. Other studies have indicated that metacognitive prompts on their own, without cognitive prompts, have no impact on student learning outcomes13, and that their usefulness decreases as student skill increases14. In addition, one Australian study has shown that metacognitive and self-efficacy abilities are not correlated to student learning outcomes as measured by GPA15.
There are theories and practices that consider students’ intellectual development16, but the impact of developing students’ intellectual level has been assessed by considering general education learning outcomes (critical thinking, citizenship, intercultural competence, communication) not students’ understanding of their major subject, although this has been proposed to improve17. My proposal considers whether the development of students’ metacognitive ability through the creation of their own learning philosophy is able to positively impact students’ content mastery in a course. I will use AUBIO 111 – Integrative Biology I as my lab. My aim is to show that as students consider their own thinking and learning processes, they will also be engaged in the epistemology of science18. Thus, by performing the metacognitive task of asking themselves what, why and how they know something, they will also begin their intellectual development as scientists asking themselves how we know what we know about biology: what is our basis for the knowledge claims we make in biology? Thus metacognition about their own learning may aid their cognitive development as biologists and scientists.
My specific question is do first year biology students score higher on the Perry Scheme of intellectual development19,20 when they have produced a learning philosophy to develop their metacognition, and does this enhance their content mastery of the course as indicated by their course grade and exam results?
The research carried out will directly impact my own teaching and potentially the way that others teach. By guiding students in the development of their own learning philosophy my aim is to metacognitively engage students with course material at a deeper level and avoid the unengaged learning cycle of memorize-regurgitate-purge. Funds from the professorship will be used to hire undergraduate students who will aid in the organization and analysis of the collected data. The proposed research aligns with the University Academic Plan by attending to the student experience. By promoting students’ metacognition through the development of their own learning philosophy I hope to show that they more deeply engage in course material by moving them to a higher level of intellectual development as measured on the Perry Scheme as a result of a transformative learning experience21.

Proposed Project
In Fall 2015 I have the opportunity to teach two separate lecture sections of AUBIO 111 – Integrative Biology I. I will use these two classes to investigate the impact of developing students' learning philosophies on student learning outcomes as indicated by their general intellectual development and more specifically on their ability to understand the particular course content. The impact of the metacognitive exercises on student learning outcomes will be assessed by using a mixed-methods approach (a qualitative student self-assessment + quantitative effect on ability to answer final staged exam questions) comparing student understandings of their learning philosophy at the beginning and end of the course using an assessment index to determine students’ level of intellectual development22–24. One course section will encounter activities designed to explicitly develop their learning philosophy and metacognitive ability whereas the other will not. A staged question on the final exam and on a pre-test at beginning of term tiered according to Bloom’s taxonomy will determine if developing students’ metacognitive ability impacts their ability to perform at the higher levels of Bloom’s taxonomy of learning25. This will be compared to students’ metacognitive ability as mapped on to the Perry Scheme of learner developmen 19,20.
This study will thus answer two questions. Does developing students' metacognitive ability/learning philosophy impact their: 
  1. intellectual development as mapped by the Perry Scheme, 
  2. depth of learning the course content as mapped by Bloom’s taxonomy of learning.
References
1. Smith, B. L. (1998). Curricular structures for cumulative learning. In J. N. Gardner, G. Van der Veer, & Associates (Eds.), The Senior Year Experience: Facilitating Integration, Reflection, Closure, and Transition (pp. 81–94). San Francisco, CA: Jossey-Bass Inc., Publishers.
2. Ambrose, S. A., Bridges, M. W., DiPietro, M., Lovett, M. C., & Norman, M. K. (2010). How do students become self-directed learners? In How Learning Works: 7 Research-Based Principles for Smart Teaching (pp. 188–216). San Francisco, CA: John Wiley & Sons, Inc.
15. Zeegers, P. (2004). Student learning in higher education: A path analysis of academic achievement in science. Higher Education Research and Development, 23(1), 35–56.
20. Allen, R. D. (1981). Intellectual Development and the Understanding of Science: Applications of William Perry’s Theory to Science Teaching. Journal of College Science Teaching, 11(2), 94–97.
22. Moore, W. S. (1989). The learning environment preferences: Exploring the construct validity of an objective measure of the Perry Scheme of intellectual development. Journal of College Student Development, 30(6), 504–514.
23. Baxter-Magolda, M., & Porterfield, W. D. (1985). A new approach to assess intellectual development on the Perry Scheme. Journal of College Student Personnel, 26(4), 343–350.

Friday 13 March 2015

mid-course feedback from students

So many post-secondary institutions are now making mid-course evaluations routine. I had read of this practice in a Faculty Focus post a couple of years ago and have been wanting to try it for some time. What has held me back is that I also remember reading that faculty should only implement this with their students if they are willing to consider students' suggestions and to make time to discuss the results of the feedback with the students. This term I implemented some innovations in my senior capstone course such as an e-portfolio assignment and in-class team assignments and discussions. I wanted to see how these were being received by students. After searching for some resources for best practices in the types of questions to use, I came across a great resource from UBC and found that some of their sample questions would work well for me if tweaked and also sparked some creativity to come up with a couple of my own questions.

I used the survey tool available in Moodle (our university's LMS) and created an anonymous feedback form making it available to the students in my class during Spring Break. Ten out of 22 students completed the survey over the week and I received some very good feedback. This is what I found:

(A) The material covered in class is challenging and intellectually stimulating.
-  1. strongly disagree:  0
-  2. disagree:  0
-  3. neutral:  1 (10.00 %)
-  4. agree:  3 (30.00 %)
-  5. strongly agree:  6 (60.00 %)
 (B) I am able to make overarching connections between each week’s material.
-  1. strongly disagree:  0
-  2. disagree:  0
-  3. neutral:  1 (10.00 %)
-  4. agree: 8 (80.00 %)
-  5. strongly agree:  1 (10.00 %)
 (C) Sensitive material in this course is discussed by the instructor and my classmates in a way that…
-  a) always respects my feelings and the feelings of other students: 8 (80.00 %)
-  b) usually respects my feelings and the feelings of other students:  2 (20.00 %)
-  c) sometimes respects my feelings and the feelings of other students:  0
-  d) rarely respects my feelings and the feelings of other students:  0

 (D) What aspect of the course do you think is contributing the most to your intellectual development?

In this section I was pleased that students indicated they felt that the course content was challenging and that the course structure (team-based discussions) were facilitating their processing of the material helping them to think in ways they hadn't done before. Students also wrote that having a rotating group leader to facilitate discussion of the assigned readings was helping them to broaden their thinking. One identified the class structure as a flipped classroom (students are required to write a two page response to the assigned reading for entry into the classroom) while others indicated that the writing was helping them make connections between assigned readings and to their other courses and life outside the classroom.

(E) What aspect of the course do you think is contributing the least to your intellectual development? 

This was interesting because what students had indicated above was aiding their intellectual development others indicated here were not contributing: writing, discussion leaders, e-portfolio, reading. When I parse through the responses carefully, what students really seem to be saying is that the workload for the course is too high. That is the only way I can reconcile the seeming contradiction in their responses. The readings, discussion, writing are contributing to their intellectual development, yet the readings, discussion, writing are contributing the least to their intellectual development. I wonder if there is also a tension between having students actively engaged in the material (peer discussion and student leaders) with students desire to have an "expert" tell them what is correct. 

(F) If you were the instructor, what one change would you implement to make this course a better learning experience?

This question produced results indicating that students were feeling the workload for the course to be too high. They had different suggestions: one reading per week, shorter readings, break up the required readings with in-class discussion of videos, make the course into a 6 cr full year course.

(G) What do you know now about the course that you wish you knew at the start of term?

Some students commented that they wished they had known how much work the course would be and how much philosophy would be part of the course content. For some reason students equate philosophy with difficulty. Odd, because the philosophy we are doing is simply considering the structure of biological thinking and how it developed. But we do use philosophical approaches to investigate these (e.g. David Hull, Thomas Kuhn, Karl Popper, Andrew Pickering, Ernst Mayr). I think they may simply be finding thinking hard work.... which it is.


A couple of students noted that they would not change a thing and that the 'horror' stories they had heard about the course from previous students were largely exaggerated.  That was nice to hear. On the other hand a couple of students indicated that they were not finding the writing dossier or e-portfolio to be an educationally meaningful experience. I am not sure what to with those comments because a number of other students have indicated that these are facilitating their thinking. Again, I wonder if students dislike the writing and e-portfolio mostly because of the time they are taking to complete.

So how did I respond to their feedback? I determined that I could delete three of the 11 remaining assigned readings without compromising the integrity of the course and have replaced them with in-class videos with subsequent class discussion. Students have responded very positively to the changes and indicated they appreciated that I acted on their feedback.

I'll use these questions again for mid-course feedback in future terms. One thing I also learned from this is that I prefer the questions that I placed on my feedback form to some of the questions available in our USRI - Universal Student Ratings of Instruction.

Resources

Tunks KW. 2012. Transforming Teaching through Supplementary Evaluations. Faculty Focus, August 20. http://www.facultyfocus.com/articles/faculty-evaluation/transforming-teaching-through-supplementary-evaluations/

Office of the AMS VP Academic and University Affairs. (n.d.) Guide to Collecting Mid-Course Feedback at UBC. UBC Mid-Course Feedback. University of British Columbia. http://midterm.teacheval.ubc.ca/about/


Monday 9 March 2015

mindfulness in the classroom

Mindfulness is a practice that continues to pop up in different contexts for me for the last year or more. It seems to be a method to reprogram our brains to eject old habits that interfere with our goals and aspirations and replace them with habits that can create in us the person we wish to be. I first became aware of this ability to intentionally rewire our brains when I read The Shallows. It seems that there are similarities between rewiring our brains through distractions vs meditation and is not unlike the rewiring that people have to go through when engaged in physiotherapy to relearn how to walk or talk after a stroke or suffering other types of nervous damage. It seems to me that we should be able to take these same insights from physical therapy and observations about how computer and internet use affect our brain - affects our thinking - and apply them to improving our ability to learn and perhaps our ability to teach by creating learning environments that foster students' ability to stay on task and focus. This post on Faculty Focus suggests some approaches for promoting mindfulness in our classrooms and personal lives.

Resources

Carr N. 2011. The Shallows: What the Internet Is Doing to Our Brains. WW Norton & Co. http://www.theshallowsbook.com/nicholascarr/Nicholas_Carrs_The_Shallows.html

Roush KL. 2015. Moving from Multitasking to Mindfulness. Faculty Focus (March 9). http://www.facultyfocus.com/articles/teaching-and-learning/moving-multitasking-mindfulness/

Friday 6 March 2015

the role of narrative in teaching and learning

Marilla Svinicki writes the recurring column 'Ad Rem' in the National Teaching and Learning Forum. For the last couple of issues she has been writing about the 25 principles of teaching produced by the APA and APS and recently published. In her February 2015 column Marilla discusses the importance of teaching through stories and how the research shows that students learn deeper when the content is contextualized as a story rather than as a list of facts. Doesn't matter how well those facts are organized, they simply will not be remembered - learned - as well as they would be when learned within a narrative. Stories speak to humans. For whatever reason we crave stories in all their forms, TV shows, movies, novels, anecdotes, case studies. I am sure that someone out there knows the academic reason for this propensity. For me, it speaks to my need to make sense of my world and my life. Hearing others' stories helps me place my own personal story in context and helps me make sense of what is happening in my life.

I think this is what makes stories such a powerful teaching tool. This is what makes case studies valuable to student learning - it places a problem in a narrative that draws in the student and engages them in the issue that needs to be addressed. When I teach biochemistry, molecular cell biology, histology, history and theory of biology I try to structure the entire course with some sort of narrative to draw the students into the subject matter. In biochemistry it is following the flow of energy and matter and how organisms have evolved to release that energy in usable forms. In molecular cell biology it is considering the problem of how proteins in all of their myriad forms are able to get to the right place and time. In history and theory of biology it is about how we understand the world through the filter of our experience and theories. In each of these courses, to draw the students into the narrative, I need to provide students with different examples - stories - of how each concept has significance and consequence to how we live our lives.

Narrative plays a powerful role in teaching and learning.

Resource

Graesser, A.C. 2009. Inaugural Editorial for Journal of Educational Psychology. Journal of Educational Psychology, 101(2), 259–261. http://www.apa.org/pubs/journals/features/edu-101-2-259.pdf

Svinicki, M. (2015), One Story is Worth A Thousand Pictures?. The National Teaching & Learning Forum, 24: 12. http://dx.doi.org/10.1002/ntlf.30020

Tuesday 3 March 2015

keeping your teaching fresh

I just participated in an online presentation by Maryellen Weimer entitled "How to Keep Your Teaching Fresh" hosted by Magna Publications. She always has interesting advice to give. The online community participation through Twitter and the chat function was great. So many instructors that are enthusiastic and engaged in how to create the best learning environment for their students. Here are some of the things that I learned.

Maryellen began by discussing how to engage our classes beyond the handful of students who always respond to our questions. She suggested the three-hand technique to encourage diversity of response during class discussion. This simply involves choosing the 3rd or 4th hand that goes up to answer our question or provide a comment. Sometimes this takes some time - but that is ok. Patience is important in teaching and learning.

I often use reading quizzes in my courses particularly when I use Team-Based Learning as the instructional strategy. The point of the quizzes is formative not summative assessment. Thus to decrease student anxiety about being tested on material before being covered in class, Maryellen suggested that students be permitted to bring in their own notes they prepared while reading the article. If they bring in sheets of paper with notes - that is fine. The point is to encourage student learning, not to assess what they have learned for these sorts of formative assessments. I like that and think I will use it.

She also discussed the merits of following individual quizzes with group quizzes. I routinely use these in my courses, again particularly when I use TBL as the instructional strategy. I have found that IF AT cards are particularly good at stimulating conversation during these group activities and provide a means for student learning during the assessment: Something advocated by Eric Mazur. The online chat during Maryellen's presentation queried whether or not there is an optimum group size. My reading of the TBL literature suggests that the sweet spot is between 5 and 7 students. Less than five and there is the tendency for one or two students to dominate the conversation. More than seven and there is the propensity for social loafing. Bonnie Mullinix responded to my response suggesting that the optimium group size is dependent upon class size and complexity of activity. I knew that making groups within small classes becomes redundant (creating groups is one way of trying to create the conditions of the small class) but I hadn't considered that group size should decrease in response to the complexity of the assignment. I am hoping I find a reference that corroborates this inverse relationship.

Maryellen also discussed the efficacy of study groups citing a post from the Teaching Professor (Study Groups Pay Off, Aug/Sept 1991) in which Robinson suggested that students who form their own study groups register with their instructor and email updates to them regarding their meeting and what they accomplished. Further, because students do not always have experience with study groups, instructors can guide them with emailed suggestions of group study activities and techniques and reward study groups for their efforts by applying bonus points to members of the study group based on the average that study group members earn on assignments or tests. Interesting idea....

There was also some discussion on the propensity of instructors to try a new teaching and learning technique only to abandon them as soon as they receive negative feedback from students. Maryellen suggests that trying something new is never an unmitigated disaster. Some strategies work better for some students and instructors than others. It is important for us to remember that we as instructors are not experiencing the new technique from the point of learning - it is the students. Thus it is important for instructors to seek mid-course feedback from students and then to consider the number of students giving similar comments. We shouldn't toss out a new teaching strategy based on only one or two negative comments. Implementing a new teaching and learning technique requires student feedback but also critical reflection on that feedback by instructors to determine what teaching strategies will work and how they might be improved.

The issue of providing extra credit to students was raised. Maryellen cited a survey from the 1980s that suggested that most faculty were not in favour of giving students extra credit for example in the case of compensating for a less than desired mark on an exam or assignment. Maryellen suggested that this could be used as a learning incentive if instructors permitted the possibility of extra credit only if students achieve a certain level on an assignment or exam. I am not sure how I feel about that. In her book Learner-Centered Teaching Maryellen gives the example she used in one of her courses in which students could do as much or as little of the work that was available to be completed in the course. To earn a particular grade required that students earn a certain number of marks but that there were a number of different ways to do this - few assignments for which students earn a very high percentage mark or many more assignments/quizzes for which a lower percent mark is earned. It is a controversial way of approaching the awarding of student completed work but well worth considering if only to shake up your own thinking about marking and assigning grades.

Maryellen's last piece of advice was to not change everything at once. Rather, she advises instructors to make incremental change. This is the same advice I received from my friend Mark Lewis many many years ago when I first started my university teaching career. He advised to choose one aspect of a course to change and then do only that. To do any more only hastens our burnout as teachers. In addition, when implementing an innovation in our teaching we must be careful to field test it first before inflicting it on our students. We can manage less than desired results in our classes if it is only one innovation that needs attention. But if we have a number of new innovations that we are juggling at the same time.... we shouldn't be surprised when we drop more than one ball.