Monday 26 January 2015

cumulative exams promote long-lasting learning

This is something that I have long suspected but never had the evidence to back up my suspicions. Now I have the evidence: cumulative exams promote long-term learning. But what is interesting, that will likely affect my course design, is that what matters is that each subsequent exam/quiz, throughout the course be cumulative, not just the final exam. I have always made my MT exams and quizzes, non-cumulative, much to the satisfaction of my students. However, the study cited in this Faculty Focus blog post suggests that it is the cumulative nature of tests throughout the term that makes the difference, not just having a cumulative final exam.

My students may resent me for it, but my job as an educator is to design a learning environment that best promotes their learning. Looks like this is the end of non-cumulative tests in my courses.


Resource

Weimer M. 2014 Do Cumulative Exams Motivate Students? Faculty Focus, April 10. http://www.facultyfocus.com/articles/teaching-and-learning/cumulative-exams-motivate-students/

Sunday 18 January 2015

reading, writing, thinking - is there a difference?

I have been working on a rubric for marking students' written responses to reading assignments. The VALUE rubrics published by AAC&U can be quite useful but I found that the one they developed for written communication was not capturing what I was emphasizing when I mark. I do attend to writing structure and style, but what really interests me is what the students do with the reading in terms of integrating it with what they have learned and experienced  before. Here is how my initial draft of the marking rubric was structured:


A – Excellent
B – Very good
C – Fair
D – Poor
F – Fail
SCORE
Structure
Correct sentence structure is used throughout. Each paragraph develops a single idea. (15-12)
A couple of minor errors in sentence structure are apparent. Each paragraph develops a single idea. (11-10)
Minor errors in sentence structure are apparent throughout. Paragraphs do not always develop an idea. (9)
Sentences are not always clear. The point of paragraphs are not always apparent. (8)
Sentences lack logical structure. Paragraphs are incoherently developed. (7-0)
__/15
Style
Paragraphs are logically linked. The first paragraph accurately sets the tone for the response. The last paragraph correctly summarizes the response and ties up loose ends. Word choice is appropriate & writing is concise. Word usage is varied with appropriate phrasing. (15-12)
Paragraphs are linked. The first paragraph sets a tone for the response. The last paragraph summarizes the response and ties up loose ends. Writing is concise with appropriate phrasing. (11-10)
The first paragraph introduces the response. The last paragraph provides a summary. Writing is concise with appropriate phrasing. (9)
Paragraphs are poorly linked. The first paragraph weakly introduces the response. The last paragraph weakly summarizes the response. Word choice not always appropriate or varied. (8)
Paragraphs are not linked. The response is not introduced nor summarized. Word choice is inappropriate & writing is wordy. Word usage is not varied. Inappropriate phrasing is used.   (7-0)
__/15
Content
The thesis is clearly, accurately & concisely stated. The reading is well synthesized with other readings or experiences. The consequences & significance of the reading are carefully considered. (70-56)
The thesis is clearly identified and synthesized with other readings or experiences. The consequences & significance of the reading are considered. (55-48)
A thesis is identified and somewhat synthesized with other readings or experiences. The consequences & significance of the reading are somewhat considered. (47-40)                    
A thesis is identified and poorly synthesized with other readings or experiences. There is little consideration of the consequences or significance of the reading. (39-35)
A thesis is not identified. There is no synthesis of the reading with other experiences. There is no consideration of the consequences or significance of the reading. (34-0)
__/70
TOTAL





__/100

It did a good job of attending to writing structure and organization and predominantly weighted the content of students' writing. When I proofed it a couple of hours later the word 'reading' in my rubric reminded me that one of the VALUE rubrics is for reading. I never paid much attention to that because I always thought I was assessing students' writing not their reading. But when I re-read the AAC&U reading rubric I noticed that what the rubric was capturing is similar to the focus of my own rubric in terms of being able to explain, integrate, and reflect on what students have read. I wonder how many other instructors have not made this link that when we assess students' writing, we are attending to their thinking (writing is thinking) but that their thinking is framed by what they have read. To actively engage with a reading assignment requires cognitive effort - students are thinking.

When I started to think about that I downloaded the VALUE rubric for thinking and also integration and noticed that there were also aspects of those that I was trying to capture in my rubric. So, I edited my rubric again to try and emphasize and tease out those aspects of what I wanted to assess for students. I came up with something better but then realized that my rubric was beginning to look like a summative assessment rather than  formative assignment which I had originally intended for students. I want students to engage with the reading, critically thinking about how it integrated with the rest of the course and also with their degree program (this particular course is the capstone of Augustana's biology major).

After messing around with it some more this is what I have come up with:




Excellent
Very good
Fair
Poor
Unacceptable
SCORE
Syntax & Mechanics
Uses graceful language that skillfully communicates meaning to readers with clarity and fluency, and is virtually error free. (10-9)
Uses straightforward language that generally conveys meaning to readers. The language in the portfolio has few errors. (8-7)
Uses language that generally conveys meaning to readers with clarity, although writing may include some errors. (6-5)
Uses language that sometimes impedes meaning because of errors in usage. (4-1)
Sentences lack logical structure. Paragraphs are incoherently developed. (0)
__/10
Organization
Paragraphs are logically linked. The first paragraph accurately sets the tone for the response. The last paragraph correctly summarizes the response & ties up loose ends. (10-9)
Paragraphs are linked. The first paragraph sets a tone for the response. The last paragraph summarizes the response & ties up loose ends. (8-7)
The first paragraph introduces the response. The last paragraph provides a summary. (6-5)
Paragraphs are poorly linked. The first paragraph weakly introduces the response. The last paragraph weakly summarizes the response. (4-1)
Paragraphs are not linked. The response is not introduced nor summarized. (0)
__/10
Comprehension
The thesis of the assigned reading is clearly, accurately & concisely stated. The argument & evidence for the thesis are accurately assessed & evaluated. (10-9)
The thesis is clearly identified. The argument & evidence for the thesis are assessed & evaluated. (8-7)
The thesis is poorly identified. The argument & evidence for the thesis are somewhat assessed & evaluated. (6-5)
The thesis is misunderstood. The argument & evidence for the thesis are poorly assessed & evaluated. (4-1)
A thesis is not identified. The argument & evidence for the thesis are not assessed & evaluated. (0)
__/10
Connections
The consequences & significance of the reading are carefully considered. The assigned text is meaningfully synthesized with other class readings and experiences outside the classroom (other life and academic experiences) to deepen understanding of biology and to broaden own points of view. (10-9)
The consequences & significance of the reading are considered. The assigned text is synthesized with other class readings and experiences outside the classroom to deepen understanding of biology and to broaden own point of view. (8-7)
The consequences & significance of the reading are somewhat considered. The assigned text is somewhat synthesized with other class readings and experiences outside the classroom to deepen understanding of biology and to broaden own points of view. (6-5)
There is little consideration of the consequences or significance of the reading. The text is poorly synthesized with other readings or experiences. There is little deepening of biological understanding or broadening of point of view. (4-1)
There is no consideration of the consequences or significance of the reading. There is no synthesis of the text with other experiences. Biological understanding or point of view are neither deepened nor broadened. (0)
__/10
TOTAL





__/40



I am probably just slow (been teaching for almost 25 years now) or perhaps I have just never given myself the space to think critically what it is that students are doing when they respond to a reading assignment. It is interesting that I am trying to develop in students the ability to think, synthesize, read, and write. This is what I am generally doing with all of my assignments. Which makes it difficult to produce a focused rubric. I am probably wanting a general assessment rubric for writing assignments and I wonder if that is fair to students and to myself as their instructor? Is it just too complicated? In addition, how do I produce a rubric for assessing what is supposed to be a formative not a summative assessment. You may ask why I don't just make it a pass/fail or participation type assignment. My answer to that is that from my experience, unless I value the assignment by awarding marks to the assignment, students do not adequately attend to the assignment. I am torn between Kimberly Tanner's advice on how to produce low stakes assignments that engage students vs Cathy Small's (writing as Rebekah Nathan) observation that students will not attend to assignments that are not valued with marks.

I hope the second rubric above suffices for this weekend because I need to mark my students assignments. I need to find a balance between what I am attending to in developing students skills (thinking, writing, researching) and providing formative experiences that students take seriously.

Teaching is an iterative process.

Resource


Nathan, R. (2005). My Freshman Year: What a Professor Learned by Becoming a Student.
New York (NY):Penguin Group. http://books.google.ca/books/about/My_Freshman_Year.html?id=s2fGK_vAvPIC&redir_esc=y

Rhodes, T.L. 2010. Assessing Outcomes and Improving Achievement: Tips and Tools for Using Rubrics. Washington (DC): AAC&U. https://www.aacu.org/publications-research/publications/assessing-outcomes-and-improving-achievement-tips-and-tools-using

Schinske, J., and Tanner, K. (2014). Teaching more by grading less (or differently). CBE-Life Sci. Educ. 13, 159–166. http://www.lifescied.org/content/13/2/159.full