This semester, I’m trying to improve on last semester’s online quizzes that ensured students did the reading and tested their comprehension. For that process, I provided students with a set of reading comprehension questions and then wrote a new (though somewhat similar) set of questions for the quizzes, which had to be matching, true/false, or multiple choice so that the computer could grade them. This semester, I’m shifting the workload from writing questions to grading–I’m just asking students to answer the reading questions on our LMS’s discussion fora. Here’s my descriptions of the exercise from the syllabus:
For each regular class, there will be a discussion forum on the Isidore site with a series of questions related to the reading due for that class. Each student must provide a 1-3 sentence answer for at least 2 questions before class (and/or polite responses to other student’s posts). Each question should receive at least 3 answers from someone before class, so that you are provided with different explanations to help you better understand the readings—this will also serve as a partial study guide from the exams. These posts will be graded for completion—not for accuracy—although slight extra credit will be awarded for particularly good answers. These are open-book questions, so I encourage you to use the readings to help you answer these questions. They will also guide our discussion in class on the following day.
In class and in the syllabus, I used these arguments to convince my students of the merit of the exercise:
- It helps them understand the readings better because the questions suggest what is helpful to notice or consider.
- One explanation of a topic may not be clear for one person, but a slightly different one may help students understand the idea much better. Therefore, their classmates’ posts could help them understand a section of the reading or textbook that particularly confused them. Indeed, classmates are often better at bridging the gap of understanding among themselves than professors are, and students learn better through explaining something. Mike Caulfield first exposed me to this idea of choral explanations on his blog, and it’s the idea behind quora. Ever since reading about choral explanations, I’ve been trying to find a way to incorporate them into my classes, and I think this strategy comes close.
- The allure of extra credit for very good answers.
- It will be a good reference for them while studying for the exam. This argument partially helps me avoid the expectation that I provide a study guide.
- It also helps me prepare for class. In the 15-20 min. before class, I skim the discussion fora to see how well students are “getting it,” what they are understanding, what I need to emphasize or shift their perceptions on, and gauge how well that class will go based on their responses.
How it is working
In general, I think it’s working really well, but not quite in the idealistic way that the syllabus implies. To illustrate and guide my various comments, here are several screenshots from a reading about Old Kingdom Egypt:
- Students are not ensuring that the questions are answered in roughly equal numbers. Usually, the first few questions have more answers than later ones. This makes me wonder how often students are doing the reading (I called this class out on this yesterday and I am curious to see if that will change their habits). If we try to mentally adjust for this trend, the numbers of answers can indicate how well students are understanding the material. For example, Question 11 is clearly harder (and more confusing) for students than Question 5, so I can try to remember this while I teach and write future questions. This assessment is an unintended benefit.
- For this class, I frequently ask students to read from the textbook (D. Brendan Nagle’s The Ancient World: A Social and Cultural History) and to read a primary source. I abbreviated this as “Q1 (N)” and “Q7 (EE)” when we read the Enuma Elish, but this confused at least one student. Therefore, I adjusted to the notation shown here where the first question on a reading is “Q1 (Nagle)” and then abbreviated for later questions.
- The questions are of varying difficulty and require different levels of thought. When I asked students about why some questions receive more responses, they said that some questions are just easier and some are harder. The harder ones (with few responses) are often the ones based on primary sources because the primary sources are harder to read than the textbook. I told them that the “easier questions” are sometimes there to highlight things that I want them to notice/know for class or the exam, but I acknowledged this variability is a possible issue.
- The quality of answers is variable. The first two answers are very similar and repeat each other, which is not as helpful when there are 8 or 9 like this. In fact, during our conversation about the discussion fora yesterday, one student complained about this repetition. The third answer shows more effort and is actually more helpful, I think, for understanding what ma’at is.
- Despite this variety of responses, the narrative answers still give me more insight into students’ reading comprehension, engagement, and language abilities than the online quiz scores did. I appreciate this greater sense of students’ language abilities, so that I have more realistic expectations for papers and I know how often to remind students about the on-campus writing center–and that observation applies not just to international but to domestic students as well.
- Assuming I have a comfortable amount of time before the lecture, I find myself responding to their posts more often than I expected. In this instance, I use “Very good” as a flag both to students so that they pay attention to this while studying, and as a flag to me to give this student extra credit when I go back and grade for completion.
- I can reinforce good habits, such as providing quotations and explaining them. Depending on when/if the students read the comments, they get feedback before an exam or paper, which carry heavier weight in their final grades.
- I can guide how students think about the ancient world outside of the classroom and possibly before exams and papers.
- Unfortunately, only one of my comments (which are often phrased as questions) has received a response, so I am inclined to think that the students are probably not reading these comments.
- I can add to the choral explanations. Generally, if only one student answers to a question, I usually comment that it is a good answer, clarify part of it, or add my own perspective on the question. I am trying to avoid answering a question that no student has answered.
- Few students have actually responded to another student’s posts as suggested in the syllabus directions. While I am disappointed that this has not inspired much discussion of ideas outside of class (an idealistic hope!), I sometimes take these opportunities to arbitrate some of the disagreement or point out value in each student’s contribution.
Things I would change for future (lessons or semesters)
- I am trying to ask fewer, more complicated questions so that (1) each question receives more answers, (2) so that students need to put more thought into them rather than imitating each other, and (3) students’ skills develop as they move up Bloom’s Taxonomy and grow as historians.
- Next semester, I would grade for effort, not completion. Again, this is to discourage imitation and to encourage active learning, engagement with the sources, critical thinking, and actually practice being a historian.
- Similarly, I would ask that students answer questions in more than 1-3 sentences, perhaps a word limit would be better. I might also ask students to answer more questions (or ask fewer questions overall so that students are answering a greater proportion of the overall questions). Again, the goal would be to encourage deeper engagement.
- In general, I think the approach needs fine-tuning: how to grade it efficiently, how to make sure students are doing the entire assignment, and how to structure it so there is even more engagement that doesn’t ask students to spend too much time at home working on the assignment. I’m very satisfied with how this experiment is going and I would only consider going back to online quizzes for introductory level courses.