I just finished teaching my last molecular cell biology class for the fall term and I am feeling out of sorts. Out of sorts because I don't know what to do with myself. For three months my energy has been focused on preparing in-class activities and reading quizzes for each and every class meeting. Team-Based Learning, a highly structured version of the flipped classroom, requires preparation for every class. Activities and questions that worked last year, don't necessarily work for the subsequent cohort of students. So for the last three months, I have been probing my students trying to get a sense of what they do and do not know so that I can provide learning experiences to address their knowledge and skill gap and trying to hold a mirror up to them so that students know what they do and do not know.
Now the class is over, the students are completing the end of term student rating of instruction. And I am at a loss over what to do with myself.
Yes, I have service committees that need my attention, I have papers that need writing and revising, I have reviews that need to be typed. But those were always completed in the background while teaching has been foremost in my mind. Now that is over for the term, I feel disoriented - I no longer have the lodestone of instructing my students to set the priorities for my day. Can I actually take the time to read that article, to browse that recent journal issue, to walk down the hall and say hello to my colleague?
Strangely, I feel guilty about even taking the time to consider going for a coffee with a colleague. This is how all-encompassing the teaching term is for me.
It didn't always use to be this way.
Certainly, when I first started teaching I was always running to stand still. But after a few years and being successfully tenured, the pace became reasonable. I was able to tweak and re-use previous lectures and received great student reviews for my efforts.
Then, about eight years ago, I became bored with the sound of my own voice and experienced the revelation that good lecturing doesn't necessarily equate with good learning. In 2012, I began experimenting with implementing active learning in my classes culminating with most of my classes being reworked with team-based learning. And then the realization inherent with learner-centred teaching that each cohort of students is different; that to be able to meet the needs of each student cohort, each student, required that I continually formatively assess my students in order to understand and meet their learning needs.
That is difficult exhausting work. But such rewarding work! After my last class today, a few students came up and thanked me for my efforts. One student expressed gratitude for my class structure that facilitated their ability to learn from the assigned readings. The reading guides are doing their work. My preparation efforts were well-received.
So now I am sitting in my office wondering what to do with myself. I have decided to sit quietly for a moment and enjoy the fruits of my labour: student learning and gratitude. The teaching and learning experience is fleeting. Just allow me a few minutes to enjoy it before it fades into the ether.
Showing posts with label formative assessment. Show all posts
Showing posts with label formative assessment. Show all posts
Wednesday, 11 December 2019
Friday, 20 January 2017
pre-testing and expectations in team-based learning
Last Friday I gave my 2nd-year biochemistry class its first readiness assurance test or RAT in team-based learning (TBL) terminology. It was a mix of new material (amino acids) and material they should have learned from their pre-requisite chemistry courses on pH and buffers. Typically, TBL RATs aim to be reading quizzes to encourage students to prepare for practicing to use their newly learned knowledge in class in subsequent classes in what TBL terms Apps (for applications). The design of RATs aims to produce a class average of 70%. My average last week was 49%.
So, what happened? Interestingly, when I analyzed the marks, it appears to me that students did better on the new material: recognizing amino acid structure and calculating pI of amino acids. What students had the most difficulty with was their understanding of their prior learning on pH and buffers. This surprised me and so I checked with my chemistry colleagues and they suggested that my questions on pH and buffers were more conceptual and that first-year chemistry courses tend to focus on calculating pH. In addition, my chemistry colleagues reminded me that our students prefer to plug and play (calculate) rather than think. I don't think our campus is unusual in this regard - thinking is difficult work!
But it does raise an interesting issue for the implementation of TBL as a teaching and learning strategy. My understanding of TBL practice suggests that RATs should focus more on the conceptual than on calculation style questions. As a result, this should promote discussion of the questions during the team phase of the two-stage style of testing that is inherent in RATs. In readiness assurance tests, students first complete the test (10 MCQs in my classes) individually and then repeat the same test as a team using IF-AT cards so that students receive immediate feedback about their understanding. It is great at immediately revealing misconceptions. I have been using this technique since 2011 and can attest that it works well.
However, there does seem to be an apparent tension in the Readiness Assurance Process of Team-Based Learning. The RAP is what makes TBL a flipped classroom teaching strategy. In the RAP, students are assigned pages to read from the textbook (or article, or podcast, or whatever students need to initially prepare themselves to learn in class), and then during the first class of the course module/topic, students are administered a RAT in the two-stage testing style I described above. The RAP is intended to encourage students to do their pre-class preparation and hold them accountable for that preparation. It is not intended to be the end of teaching and learning for the particular course module, but rather is supposed to mark the beginning of teaching and learning. Thus, the RAT should be considered to be, in essence, a reading quiz. It is suggested in the TBL literature that a typical RAT could be constructed based on the topic headings and subheading in the assigned textbook chapter. However, the TBL literature also suggests that the questions should generate discussion and debate during the team portion of the RAT. What I have difficulty in implementing TBL in my classes, is that there seems to be a tension between producing a RAT that is a reading quiz vs producing a RAT that generates discussion and debate. Typically, a reading quiz is designed fairly low on Bloom's taxonomy of learning (mostly questions testing recall). In contrast, questions that foster debate and discussion need to move beyond simple right/wrong answers. Hence, it seems that there is a tension inherent in the design of RATs: they should be designed as reading quizzes that are able to foster debate.
I have a hard time constructing these sorts of tests and I believe that is what produced the poor class average on my first RAT of the term in my biochemistry class last week. What I thought were simple recall questions based upon what students had learned in prior courses, ended up exposing some fundamental misconception about their learning. I guess that is what RATs are supposed to do. I was just surprised how many misconceptions students had about pH and buffers given they have been learning this since high school. On the other hand, if you don't use it, you lose it. And I suspect that for many of my biochemistry students they have not had to consider pH and buffers for over a year or two.
The way I handled the situation is that I made the RAT out of 9 instead of 10 (there was one question that no one answered correctly - a couple of teams got it correct on their second attempt) and I have also informed students that I will not include their lowest RAT result when I calculate their final grade for the course. Hopefully, that is sufficient to press the reset button so that students feel like they are not stumbling right out of the gate in the first week of classes.
Haide, P., Kubitz, K., & McCormack, W. T. (2014). Analysis of the team-based learning literature: TBL comes of age. Journal on Excellence in College Teaching, 25(3&4), 303–333.
Metoyer, S. K., Miller, S. T., Mount, J., & Westmoreland, S. L. (2014). Examples from the trenches: Improving student learning in the sciences using team-based learning. Journal of College Science Teaching, 43(5), 40–47.
So, what happened? Interestingly, when I analyzed the marks, it appears to me that students did better on the new material: recognizing amino acid structure and calculating pI of amino acids. What students had the most difficulty with was their understanding of their prior learning on pH and buffers. This surprised me and so I checked with my chemistry colleagues and they suggested that my questions on pH and buffers were more conceptual and that first-year chemistry courses tend to focus on calculating pH. In addition, my chemistry colleagues reminded me that our students prefer to plug and play (calculate) rather than think. I don't think our campus is unusual in this regard - thinking is difficult work!
But it does raise an interesting issue for the implementation of TBL as a teaching and learning strategy. My understanding of TBL practice suggests that RATs should focus more on the conceptual than on calculation style questions. As a result, this should promote discussion of the questions during the team phase of the two-stage style of testing that is inherent in RATs. In readiness assurance tests, students first complete the test (10 MCQs in my classes) individually and then repeat the same test as a team using IF-AT cards so that students receive immediate feedback about their understanding. It is great at immediately revealing misconceptions. I have been using this technique since 2011 and can attest that it works well.
However, there does seem to be an apparent tension in the Readiness Assurance Process of Team-Based Learning. The RAP is what makes TBL a flipped classroom teaching strategy. In the RAP, students are assigned pages to read from the textbook (or article, or podcast, or whatever students need to initially prepare themselves to learn in class), and then during the first class of the course module/topic, students are administered a RAT in the two-stage testing style I described above. The RAP is intended to encourage students to do their pre-class preparation and hold them accountable for that preparation. It is not intended to be the end of teaching and learning for the particular course module, but rather is supposed to mark the beginning of teaching and learning. Thus, the RAT should be considered to be, in essence, a reading quiz. It is suggested in the TBL literature that a typical RAT could be constructed based on the topic headings and subheading in the assigned textbook chapter. However, the TBL literature also suggests that the questions should generate discussion and debate during the team portion of the RAT. What I have difficulty in implementing TBL in my classes, is that there seems to be a tension between producing a RAT that is a reading quiz vs producing a RAT that generates discussion and debate. Typically, a reading quiz is designed fairly low on Bloom's taxonomy of learning (mostly questions testing recall). In contrast, questions that foster debate and discussion need to move beyond simple right/wrong answers. Hence, it seems that there is a tension inherent in the design of RATs: they should be designed as reading quizzes that are able to foster debate.
I have a hard time constructing these sorts of tests and I believe that is what produced the poor class average on my first RAT of the term in my biochemistry class last week. What I thought were simple recall questions based upon what students had learned in prior courses, ended up exposing some fundamental misconception about their learning. I guess that is what RATs are supposed to do. I was just surprised how many misconceptions students had about pH and buffers given they have been learning this since high school. On the other hand, if you don't use it, you lose it. And I suspect that for many of my biochemistry students they have not had to consider pH and buffers for over a year or two.
The way I handled the situation is that I made the RAT out of 9 instead of 10 (there was one question that no one answered correctly - a couple of teams got it correct on their second attempt) and I have also informed students that I will not include their lowest RAT result when I calculate their final grade for the course. Hopefully, that is sufficient to press the reset button so that students feel like they are not stumbling right out of the gate in the first week of classes.
Resources
Dihoff, R., Brosvic, G. M., ML, M. L. E., & Cook, M. J. (2004). Provision of feedback during preparation for academic testing: learning is enhanced by immediate but not delayed feedback. The Psychological Record, 54(2), 207–231.Haide, P., Kubitz, K., & McCormack, W. T. (2014). Analysis of the team-based learning literature: TBL comes of age. Journal on Excellence in College Teaching, 25(3&4), 303–333.
Metoyer, S. K., Miller, S. T., Mount, J., & Westmoreland, S. L. (2014). Examples from the trenches: Improving student learning in the sciences using team-based learning. Journal of College Science Teaching, 43(5), 40–47.
Sunday, 18 January 2015
reading, writing, thinking - is there a difference?
I have been working on a rubric for marking students' written responses to reading assignments. The VALUE rubrics published by AAC&U can be quite useful but I found that the one they developed for written communication was not capturing what I was emphasizing when I mark. I do attend to writing structure and style, but what really interests me is what the students do with the reading in terms of integrating it with what they have learned and experienced before. Here is how my initial draft of the marking rubric was structured:
It did a good job of attending to writing structure and organization and predominantly weighted the content of students' writing. When I proofed it a couple of hours later the word 'reading' in my rubric reminded me that one of the VALUE rubrics is for reading. I never paid much attention to that because I always thought I was assessing students' writing not their reading. But when I re-read the AAC&U reading rubric I noticed that what the rubric was capturing is similar to the focus of my own rubric in terms of being able to explain, integrate, and reflect on what students have read. I wonder how many other instructors have not made this link that when we assess students' writing, we are attending to their thinking (writing is thinking) but that their thinking is framed by what they have read. To actively engage with a reading assignment requires cognitive effort - students are thinking.
When I started to think about that I downloaded the VALUE rubric for thinking and also integration and noticed that there were also aspects of those that I was trying to capture in my rubric. So, I edited my rubric again to try and emphasize and tease out those aspects of what I wanted to assess for students. I came up with something better but then realized that my rubric was beginning to look like a summative assessment rather than formative assignment which I had originally intended for students. I want students to engage with the reading, critically thinking about how it integrated with the rest of the course and also with their degree program (this particular course is the capstone of Augustana's biology major).
After messing around with it some more this is what I have come up with:
I am probably just slow (been teaching for almost 25 years now) or perhaps I have just never given myself the space to think critically what it is that students are doing when they respond to a reading assignment. It is interesting that I am trying to develop in students the ability to think, synthesize, read, and write. This is what I am generally doing with all of my assignments. Which makes it difficult to produce a focused rubric. I am probably wanting a general assessment rubric for writing assignments and I wonder if that is fair to students and to myself as their instructor? Is it just too complicated? In addition, how do I produce a rubric for assessing what is supposed to be a formative not a summative assessment. You may ask why I don't just make it a pass/fail or participation type assignment. My answer to that is that from my experience, unless I value the assignment by awarding marks to the assignment, students do not adequately attend to the assignment. I am torn between Kimberly Tanner's advice on how to produce low stakes assignments that engage students vs Cathy Small's (writing as Rebekah Nathan) observation that students will not attend to assignments that are not valued with marks.
I hope the second rubric above suffices for this weekend because I need to mark my students assignments. I need to find a balance between what I am attending to in developing students skills (thinking, writing, researching) and providing formative experiences that students take seriously.
Teaching is an iterative process.
Nathan, R. (2005). My Freshman Year: What a Professor Learned by Becoming a Student.
New York (NY):Penguin Group. http://books.google.ca/books/about/My_Freshman_Year.html?id=s2fGK_vAvPIC&redir_esc=y
Rhodes, T.L. 2010. Assessing Outcomes and Improving Achievement: Tips and Tools for Using Rubrics. Washington (DC): AAC&U. https://www.aacu.org/publications-research/publications/assessing-outcomes-and-improving-achievement-tips-and-tools-using
Schinske, J., and Tanner, K. (2014). Teaching more by grading less (or differently). CBE-Life Sci. Educ. 13, 159–166. http://www.lifescied.org/content/13/2/159.full
A – Excellent
|
B – Very good
|
C – Fair
|
D – Poor
|
F – Fail
|
SCORE
|
|
Structure
|
Correct sentence structure is used throughout. Each paragraph develops
a single idea. (15-12)
|
A couple of minor errors in sentence structure are apparent. Each
paragraph develops a single idea. (11-10)
|
Minor errors in sentence structure are apparent throughout. Paragraphs
do not always develop an idea. (9)
|
Sentences are not always clear. The point of paragraphs are not always
apparent. (8)
|
Sentences lack logical structure. Paragraphs are incoherently
developed. (7-0)
|
__/15
|
Style
|
Paragraphs are logically linked. The first paragraph accurately sets
the tone for the response. The last paragraph correctly summarizes the
response and ties up loose ends. Word choice is appropriate & writing is
concise. Word usage is varied with appropriate phrasing. (15-12)
|
Paragraphs are linked. The first paragraph sets a tone for the
response. The last paragraph summarizes the response and ties up loose ends.
Writing is concise with appropriate phrasing. (11-10)
|
The first paragraph introduces the response. The last paragraph provides
a summary. Writing is concise with appropriate phrasing. (9)
|
Paragraphs are poorly linked. The first paragraph weakly introduces the
response. The last paragraph weakly summarizes the response. Word choice not
always appropriate or varied. (8)
|
Paragraphs are not linked. The response is not introduced nor
summarized. Word choice is inappropriate & writing is wordy. Word usage
is not varied. Inappropriate phrasing is used. (7-0)
|
__/15
|
Content
|
The thesis is clearly, accurately & concisely stated. The reading
is well synthesized with other readings or experiences. The consequences
& significance of the reading are carefully considered. (70-56)
|
The thesis is clearly identified and synthesized with other readings
or experiences. The consequences & significance of the reading are
considered. (55-48)
|
A thesis is identified and
somewhat synthesized with other readings or experiences. The consequences
& significance of the reading are somewhat considered. (47-40)
|
A thesis is identified and poorly synthesized with other readings or
experiences. There is little consideration of the consequences or
significance of the reading. (39-35)
|
A thesis is not identified. There is no synthesis of the reading with
other experiences. There is no consideration of the consequences or
significance of the reading. (34-0)
|
__/70
|
TOTAL
|
__/100
|
It did a good job of attending to writing structure and organization and predominantly weighted the content of students' writing. When I proofed it a couple of hours later the word 'reading' in my rubric reminded me that one of the VALUE rubrics is for reading. I never paid much attention to that because I always thought I was assessing students' writing not their reading. But when I re-read the AAC&U reading rubric I noticed that what the rubric was capturing is similar to the focus of my own rubric in terms of being able to explain, integrate, and reflect on what students have read. I wonder how many other instructors have not made this link that when we assess students' writing, we are attending to their thinking (writing is thinking) but that their thinking is framed by what they have read. To actively engage with a reading assignment requires cognitive effort - students are thinking.
When I started to think about that I downloaded the VALUE rubric for thinking and also integration and noticed that there were also aspects of those that I was trying to capture in my rubric. So, I edited my rubric again to try and emphasize and tease out those aspects of what I wanted to assess for students. I came up with something better but then realized that my rubric was beginning to look like a summative assessment rather than formative assignment which I had originally intended for students. I want students to engage with the reading, critically thinking about how it integrated with the rest of the course and also with their degree program (this particular course is the capstone of Augustana's biology major).
After messing around with it some more this is what I have come up with:
Excellent
|
Very good
|
Fair
|
Poor
|
Unacceptable
|
SCORE
|
|
Syntax & Mechanics
|
Uses graceful language that
skillfully communicates meaning to readers with clarity and fluency, and is
virtually error free. (10-9)
|
Uses straightforward language
that generally conveys meaning to readers. The language in the portfolio has
few errors. (8-7)
|
Uses language that
generally conveys meaning to readers with clarity, although writing may
include some errors. (6-5)
|
Uses language that
sometimes impedes meaning because of errors in usage. (4-1)
|
Sentences lack logical structure. Paragraphs are incoherently
developed. (0)
|
__/10
|
Organization
|
Paragraphs are logically linked. The first paragraph accurately sets
the tone for the response. The last paragraph correctly summarizes the
response & ties up loose ends. (10-9)
|
Paragraphs are linked. The first paragraph sets a tone for the
response. The last paragraph summarizes the response & ties up loose ends.
(8-7)
|
The first paragraph introduces the response. The last paragraph provides
a summary. (6-5)
|
Paragraphs are poorly linked. The first paragraph weakly introduces the
response. The last paragraph weakly summarizes the response. (4-1)
|
Paragraphs are not linked. The response is not introduced nor
summarized. (0)
|
__/10
|
Comprehension
|
The thesis of the assigned reading is clearly, accurately &
concisely stated. The argument & evidence for the thesis are accurately
assessed & evaluated. (10-9)
|
The thesis is clearly identified. The argument & evidence for the
thesis are assessed & evaluated. (8-7)
|
The thesis is poorly identified.
The argument & evidence for the thesis are somewhat assessed &
evaluated. (6-5)
|
The thesis is misunderstood. The argument & evidence for the
thesis are poorly assessed & evaluated. (4-1)
|
A thesis is not identified. The argument & evidence for the thesis
are not assessed & evaluated. (0)
|
__/10
|
Connections
|
The consequences & significance of the reading are carefully considered.
The assigned text is meaningfully synthesized with other class readings and
experiences outside the classroom (other life and academic experiences) to
deepen understanding of biology and to broaden own points of view. (10-9)
|
The consequences & significance of the reading are considered. The
assigned text is synthesized with other class readings and experiences
outside the classroom to deepen understanding of biology and to broaden own
point of view. (8-7)
|
The consequences &
significance of the reading are somewhat considered. The assigned text is
somewhat synthesized with other class readings and experiences outside the
classroom to deepen understanding of biology and to broaden own points of
view. (6-5)
|
There is little consideration of the consequences or significance of the
reading. The text is poorly synthesized with other readings or experiences.
There is little deepening of biological understanding or broadening of point
of view. (4-1)
|
There is no consideration of the consequences or significance of the
reading. There is no synthesis of the text with other experiences. Biological
understanding or point of view are neither deepened nor broadened. (0)
|
__/10
|
TOTAL
|
__/40
|
I am probably just slow (been teaching for almost 25 years now) or perhaps I have just never given myself the space to think critically what it is that students are doing when they respond to a reading assignment. It is interesting that I am trying to develop in students the ability to think, synthesize, read, and write. This is what I am generally doing with all of my assignments. Which makes it difficult to produce a focused rubric. I am probably wanting a general assessment rubric for writing assignments and I wonder if that is fair to students and to myself as their instructor? Is it just too complicated? In addition, how do I produce a rubric for assessing what is supposed to be a formative not a summative assessment. You may ask why I don't just make it a pass/fail or participation type assignment. My answer to that is that from my experience, unless I value the assignment by awarding marks to the assignment, students do not adequately attend to the assignment. I am torn between Kimberly Tanner's advice on how to produce low stakes assignments that engage students vs Cathy Small's (writing as Rebekah Nathan) observation that students will not attend to assignments that are not valued with marks.
I hope the second rubric above suffices for this weekend because I need to mark my students assignments. I need to find a balance between what I am attending to in developing students skills (thinking, writing, researching) and providing formative experiences that students take seriously.
Teaching is an iterative process.
Resource
Nathan, R. (2005). My Freshman Year: What a Professor Learned by Becoming a Student.
New York (NY):Penguin Group. http://books.google.ca/books/about/My_Freshman_Year.html?id=s2fGK_vAvPIC&redir_esc=y
Rhodes, T.L. 2010. Assessing Outcomes and Improving Achievement: Tips and Tools for Using Rubrics. Washington (DC): AAC&U. https://www.aacu.org/publications-research/publications/assessing-outcomes-and-improving-achievement-tips-and-tools-using
Schinske, J., and Tanner, K. (2014). Teaching more by grading less (or differently). CBE-Life Sci. Educ. 13, 159–166. http://www.lifescied.org/content/13/2/159.full
Subscribe to:
Posts (Atom)