Tag: feedback

From the Inbox

Grading English Language Learners’ Writing

This semester, I’m teaching more international students than I have in the past. Since some of them do not always have the best English, it raises a question: how do I grade their writing? How much do I focus on their ideas and how much do I focus on their ability to communicate their ideas and arguments? While this problem is not unique to international students, it is particularly important when the students seem to only have been studying English for a year or two rather than a decade or two with native speakers.

Personally, I find it ethically problematic to significantly punish students because they cannot communicate their ideas because they do not know our language. But how do I know what aspects of their paper are unclear due to a language barrier and what is off due to confusion, bad ideas, or problematic interpretations?  This semester, most of my international students with poor English are from China, so I was particularly happy to stumble across this explanation of Chinese language while preparing a class about ancient China:

Chinese is almost the opposite [of Japanese]. The word order is closer to English, with the verb coming before the object (e.g., “I read [a] book”). The Chinese language is also completely uninflected, having no tenses or plurals or any grammatical modification of word endings whatsoever. Honorifics, for the most part, are absent: Chinese is not a respect language. Chinese is also monosyllabic, at least to the extent that every Chinese character (written symbol), without exception, is pronounced as a single syllable and is a discrete unit of meaning. (Chris Holcombe, A History of East Asia: From the Origins of Civilization to the Twenty-First Century, 1st edition, 2011, p. 16)

From this, I think that, when grading the writing of students whose first language is Chinese and who are still learning English, we can be a bit more lenient about tenses and the inflection of plurals. Spelling, too, may also be a problem while adjusting to the Latin alphabet.  Additionally, many sentences are likely going to be pretty short and simple–just as our Latin sentences rarely approach Cicero’s in length or complexity. With these cautious allowances, I think we could be fairer to our students as they struggle with learning a complex language. Of course, they may require us to do more work to decode our students’ papers–and in some ways our Latin learning and teaching experiences have helped prepare us very well for that task–and that provides other grading dilemmas regarding what is their idea and what is our [creative?] reading of their paper (It’s worth noting that we also face the problem of decoding some native English students’ papers).

I’m not saying we shouldn’t gently encourage them to improve their English–I think we should. I think we should also encourage them, along with all other students, to seek out the help of an on-campus writing center, learning center, or a peer editor. I just think we shouldn’t excessively punish them because they aren’t a skilled native speaker.  So how much is fair to deduct from their scores due to grammatical and syntactical problems? Perhaps a few points, just like you would with a native English speaker. After all, their papers will probably already suffer if a language barrier diminishes their ability to understand their sources, so avoiding double jeopardy seems fair.

Essentially, I suggest that we should not treat them worse than you would a native English speaker. Provide them with similar advice, resources, and guidance about writing. Deduct similar, reasonable amounts of points for grammar mistakes, but put more effort into trying to understand their writing and ideas by understanding how English and Chinese differ. That may help us see through some of the fog that clouds our vision of their ideas and help us arrive at a more authentic and appropriate grade.

What is in a name?

A paper by any other name would be as formal, right? Apparently not. This semester, I assigned several brief writing assignments in my mythology and Roman archaeology classes. I called them “Exercises” so that they would not seems as stressful and help communicate that they should be brief. Instead, I think the word “Exercise” communicated that the papers were less formal than I hoped. Many of the students frequently used the first person–“I think…,” “We know this because…,” “I chose to talk about…,” or something along those lines. It was not the rare, more formal “In this paper, I argue…” or “…, I argue,…” which you see in some scholarly settings. While reading these, I was impressed by how distracting the first person was for me and so how much I hated reading it. This is the most obvious example of how these “Exercises” seemed too informal.  Two were handwritten, many were a single paragraph of few hundred words, and many lacked a thesis statement.

Now, I am sure I seem like a crotchety old man here, and I am well aware that some of my complaints reflect other issues–students needing to improve as writers, changing language norms, students are often less invested in academics than their professors were/are–but there seems to have been a communication disconnect here. The students seem not to have understood what I meant, and this can have very bad pedagogical implications. For instance, when I taught my Roman archaeology class about the Colosseum, which was built on the artificial lake in Nero’s Domus Aurea, I said “The Flavians sought to give back the land from the Domus Aurea to the people because it was acquired after the Great Fire of 64.” Based on their answers on exams, students seem to have heard “The Flavians were giving back to the Roman people.” and understood it as a way to repay the people for something. These are two very different concepts of the Colosseum. One sees the Colosseum as a just return of land and an attempt to placate the people or buy their support, the other suggests the building was part of a social contract in which the emperor must thank the people for their support. The latter is anachronistic–the Romans did not have a Lockian social contract.

My pedantic point is that we must be careful about how we communicate with and phrase things for our students. We have been trained to think in certain terms and to understand our subject in a specific way. Our students most likely do not know what we mean by some of these terms (e.g. identity, reception, “draws on,” “in dialogue with,” power) or they have very different ideas about what we mean by them (e.g. paper, quiz, test, exam, short essay [vs. essay], monograph, blog post). In order to be good teachers, we must find a way to communicate most effectively: try to remember what it’s like to be a student, try to think in their terms, and teach our students what our/academic terminology means.

Next time I do these exercise assignments, I will call them “Papers” and let a word limit be the main indicator of how long it should be. Next time, I will say “The Flavians returned the land of the Domus Aurea to the people” or “The Flavians made the Domus Aurea‘s land more open, more public.” Next time, my exam sections will just be called “Essays” instead of “Short Essays.” Next semester, I will continue to improve.

Students’ papers as arguments

So far this semester, I have graded students’ brief writing assignments in two different classes. Both papers were grounded in describing an object.  One paper asked students to describe a building and then identify what type of building it was (i.e. Roman temple), and the other paper explicitly asked students to compare a vase painting to events described in the Iliad. Even though the object description genre may seem fairly objective, as I graded the papers and provided comments, I noticed that many of my comments were talking about the papers as if they were arguments.  For example, I often used the words “stronger” or “more convincing” in my comments.  This is good.  As the UCLA History Writing Center states:

2.  A history research paper makes a historical argument.

Your paper must take a position on the problem you have posed.  You are not simply making observations about the material you have read:  you are using your observations to craft an argument that teaches us something new about the past.  The argument you make might revise a claim made by other scholars, might illustrate a completely new way of looking at a topic, or might reveal a point that scholars have missed in previous research.

3.  A history research paper is grounded in evidence from primary sources.

Unlike research papers in other disciplines, a history paper relies on primary source material, meaning materials that were produced during the period your paper addresses.  They might be letters, diaries, census data, maps, speeches, treaties–any raw material from a historical moment.  Primary sources are the key pieces of evidence you will use to support your argument.

By describing papers as arguments (which they are), I am trying to communicate two things to my students: (1) you need to be kind to your reader, and (2) you need to support your argument with citations for your evidence.  This framework for the comments hints that, if you are not clear and not kind, you will not convince your reader.  So you need to clearly lay out your logic and indicate your evidence (through citations).  I think it also shows why these citations are helpful. Many students know (at least cognitively, if not in practice) that evidence strengthens arguments.  Citations of the evidence, as indications that you are using evidence, help persuade your reader that your argument is based on evidence and primary sources (and therefore is more believable).  So by connecting their papers to arguments (something they may be much more familiar with than classroom/academic paper writing), I remind them why evidence and citations are important.

A greater focus on citations and their persuasive force, I think, also encourages students to ground themselves in the evidence.  In other words, it encourages them to think, “If I say something, I need to cite something, so I need to find evidence to support this claim.”  This will, again, make stronger arguments and papers.

Hopefully this framework for my comments will have helped improve the next batch of students’ writing assignments that I get on Tuesday!

As a side note, by comparing the two assignments, I have noticed two other things that helped make the papers go well:

  • Students are well prepared for the exercise by in-class activities and discussions.
  • The expectations are clearly communicated and delineated, both in class and in the writing prompt.  This could be through providing an example of the assignment, a checklist for the paper‘s requirements and expectations, or a very detailed prompt.

How do we make students happier?

I keep coming back to this theme, partly because of my desire to help students’ emotional health, partly because students’ feelings about classes affect what they learn from those classes or how they apply their knowledge to their life, partly because students’ emotions affect the classroom environment, and partly because students’ emotions affect enrollment and retention rates. So wouldn’t it be great if someone could tell us exactly how to make students happier?

It turns out someone can. In the article, “The Relationship Between Teacher Management Communication Style and Affective Learning,” Rebecca M. ChoryRebecca M. Chory and James C. McCroskey sought to understand how Management Communication Styles (MCS) affected student satisfaction and affective learning.  The MCS idea characterizes the way that supervisors manage their subordinates. There is a continuum between supervisor-centered communication styles (in which the supervisor makes decisions and tells the subordinates) and subordinate-centered styles (in which the subordinates make decisions). This continuum can be illustrated by four verbs: tell, sell, consult, and join. Earlier studies of businesses show that employee satisfaction is greater when it is more employee-centered and involves more employees, and greatest when the “consult” style is used.

In order to understand how this idea worked in the classroom, 108 students (53 male, 55 female) in communications classes at West Virginia University completed a survey about the teacher in the class prior to the communications class, but 16 incomplete surveys were omitted. The survey asked students to numerically evaluate the teacher’s MCS, the teacher’s body language, and the student’s affective domain (which Chory and McCroskey understood to be related to positive feelings towards a subject, ranging from selective attention to behavioral commitment to internalization). The survey suggests that teachers’ MCS was positively correlated to students’ affective learning. The following findings are also noteworthy:

  • Class size, student age, and teacher type (i.e. professor, visiting professor, or TA) did not affect affective learning. I should note that the age range within a college class was most likely not that large.
  • The more students attend class, the more they liked the class. It is unclear which causes the other, but they both might cause each other.
  • As classes became more interactive between teacher and students, more student-centered, and offered students more control over the class, students were happier. This is in line with the ideas behind collaborative learning and group work. So be a facilitator of peer-to-peer learning and not a sage on stage.
  • A major factor in the students’ perceptions of a teacher’s MCS was nonverbal immediacy. According to the On the Cutting Edge website hosted by Carleton College, a nonverbally immediate teacher has a relaxed posture and does not use a monotone voice, gestures while talking, moves around the room while teaching, looks and smiles at the class, avoids looking at notes or the board/PowerPoint, appropriately touches students, removes barriers between students, and dresses professionally but casually (perhaps becoming more casually dressed throughout the term). Even though it is not discussed in this study, it is worth noting that verbal immediacy involves using pronouns like “we” instead of “you” and “I”, calling on students by name, allowing students to call the teacher by name, allowing for small talk, providing feedback, and asking how students are feeling.

Rubrics

In continuation of my series of posts about helping students develop better emotional health and providing students with useful feedback (see standards-based grading, SE2R feedback, and commenting on students’ writing), I wanted to explore rubrics.  Even though they may not seem like the most exciting topic, they are not completely straightforward either.  I list here several pros of rubrics, cons of them, and other things to consider about them.  They are based on my own thoughts and those from John Bean’s Engaging Ideas.

response-to-literature-rubric3.png

Pros

  • Help grade quickly. 
  • Communicate to students what criteria you will use to grade their papers and how important each criterion is.
  • Help adjust to grading a new assignment.  If you are trying something new and the assignment is unusual to you, it may help to use a rubric and establish grading guidelines for yourself. They can help you communicate with students and share what is important to you, and they will help you emphasize these criteria properly when you grade.
  • Present an “objective” grade. Teachers can be trained to look for the same things in papers and evaluate the same paper in the same way. One way to develop these norms is to discuss four papers that are similar but on a range of levels. Teachers compare the papers and justify what criteria make one paper better than another. During the discussion, a facilitator can list all the criteria that are mentioned. At the end, teachers vote on what criteria are important and these become the norms for a class or department. This process also encourages teachers to explain grades and grade with more subtlety.
  • Provide a numerical score with which to compare students’ papers or work.

Cons

  • Create the illusion that all readers will read the paper in the same way. Some teachers read for ideas, some read for clarity, others for organizations. By creating a quantifiable rubric, we suggest that all readers look for the same things. In reality, all of these factors, as well as the field in which we study and teach, influence how we perceive a paper.
  • Create the illusion of precision even though it may be hard to explain why a paper received an 8 or a 7 in a given category.

Things to consider

  • How detailed do you want to make the rubric? Do you want to provide “An A paper looks like… A B paper looks like…, etc.” (like the picture, above) or do you want to break the rubric down into smaller quantifiable chunks like “Paper clearly states a thesis.” and “Paper shows evidence of understanding opposing arguments.”? Depending on your approach, you will communicate what you value or read for.
  • Do you use a generic rubric for all fields, like this one for reading or this one for writing? Do you use a generic rubric for Latin or Classics? Or do you tailor a rubric to each assignment?
  • What will actually give students valuable, useful feedback? A generic rubric or a task-specific rubric? A circled number or narrative feedback?
  • You can use more than the rubric to grade. John Bean says he reads and comments on papers, determines a provisional letter grade and writes end comments on the paper, goes through a rubric and assigns number values, but does not let the number determine the final grade. Once he has graded all the papers, he makes sure the rubric’s total and the letter grades on all the papers fall in roughly the same range. When they disagree, he rereads papers and makes adjustments to the rubric grade and/or the letter grade on the anomalous papers. When he is done, he uses the rubrics to identify common problems among all students or to make conferences with students easier for him. He also tells students that the rubric does not necessarily dictate the paper grade.

Providing feedback on students’ writing

The best kind of commentary enhances the writer’s feeling of dignity. The worst kind can be experienced as dehumanizing and insulting–often to the bewilderment of the teacher, whose intentions were kindly but whose techniques ignored the personal dimension of writing. (Engaging Ideas, p. 317)

With this comment, John Bean explains the importance of providing strong, useful feedback that makes students want to improve their writing and ideas.  Unfortunately, many teachers’ comments are stifling, hurtful, and discouraging.  We want to build confidence so that students want to try again.  In order to do build this confidence and motivation, we need to provide feedback that draws attention to both the good and the bad in students’ work.

Drawing on a lot of research about writing, Bean offers a strategy for how to comment on a(n undergraduate) student’s writing.  His main goal is not to act as judge and jury but to act as coach, finding the potential within a draft and telling students how to improve their papers.  He provides several questions, in descending order of concern, that teachers should ask themselves while reading papers:

  • Does the draft follow the assignment? If no, return for rethought with few/no other comments or grades.
  • Does the draft address a problem/question? Does it have a thesis? Writers may have clarified their ideas as they wrote, so their ideas are clearer at the end than at the beginning.  This is usually problematic for readers who want the problem and thesis clearly stated at the beginning of the paper.  A nice mitigating comment would be to say the thesis was unclear at the beginning but it became clearer where the author wanted to go by the end.
  • What is the overall quality of the writer’s ideas/argument?  This is a good time to comment on the strength of the ideas, the complexity of their development, and compare students’ ideas to other sources or scholars’ work.
  • Is the draft effectively organized? This is a good time to explain where you as a reader got confused or what could make the writing easier to follow. Do the title, introduction, or a forecasting paragraph give the reader a sense of what’s coming? Are transitions and topic sentences effective, clear, and help clarify the path of the argument? Can the student summarize the purpose of a paragraph or section?
  • Does the paper follow the “old/new contract”?  The reader’s brain wants to make connections between something old/known at the beginning of the sentence and the end of the sentence with something new that advances the argument. If authors don’t move from old to new within each sentence or paragraph, confusion usually results. If you teach your students this idea, you can quickly explain your confusion with the comment “O/N” or something similar.
  • Is the draft free of grammar, punctuation, spelling errors? A good way to comment on these is not to spend all your time and anger fixing everything, but to use a checkmark to mark lines with grammatical problems.  Then, tell the student the common problems (e.g. improper apostrophe use) and/or that they will receive a higher grade when the problems are fixed and/or that they will not receive a grade until they problems are fixed. Alternatively, you can edit a sentence or paragraph as an example to show your students what they need to do for the rest of the paper.
  • Is the draft free of stylistic and rhetorical problems (e.g. wordiness, choppiness, weak verbs, excessive use of the passive voice)? Is the style and register appropriate for the genre of writing and audience?
  • Finally, provide end comments that both justify the grade and coach towards improvement–alone the former is usually negative.  Bean advises that you focus on the potential of the current draft and mention strengths, then a few areas for improvement, and then suggestions for improvement.  A similar strategy is the SE2R Narrative feedback method.

Practically speaking, Bean suggests follows this strategy at a late-stage of the writing process or as  “final draft” which students have the option of revising (cf. Test Corrections).  Indeed, the strategy’s potential for guiding revisions is great and could be applied during the writing process before papers are turned in.  You could encourage students to read through this (or a similar) list of questions before they turn in the paper, enlist peer-editors who are given this check-list or list of questions, or you could provide comments yourself.  The latter is clearly more work for you, but it can be made a little easier with the use of technology, especially Google Docs or similar apps based on Google Docs.

 

SE2R Narrative feedback method

Feedback is crucial to learning. Students are reassured by acknowledgment of their progress and they need to know what to improve upon. Regular feedback is one of the most effective ways of improving a student’s grade. Feedback can also be tricky. We need to motivate students to improve rather than burden them with a depressing, litany of “no”s and problems. We need to acknowledge what they did well as well as what they did wrong. We also need to provide clear, substantive feedback rather than just unclear lines on a page near good or bad sentences–how do students even know what is good and what is bad?

The most common form of feedback is a grade, perhaps with incorrect answers marked wrong and perhaps with a few comments. But if we include a grade, students often only look at the grade and not the comments. Indeed, as a student, I remember wanting to read and consider the comments but being very drawn to the grade. Standards-based grading eliminates the numerical or letter grade, so the importance of comments and narrative feedback increases dramatically. While researching standards-based grading, I found many teachers liked the SE2R system because it gave teachers a structure for how to provide narrative feedback. It can be summarized in this nice infographic:

SE2R-poster.png

Pros:

  • A structured, routine way to provide comments.
  • It ties the comments to the learning standards.
  • It helps you conceptualize what they have done when you summarize their work.
  • It shows the student how you interpreted their work. Depending on the subject, this offers the student the chance to understand if they communicated what they wanted to communicate. This is better than making them guess from what perspective you were commenting on their work.
  • It shows the student what they have done and where they need to improve.
  • It is a clearer way to communicate than “92%” or “Good job!” or “See me.” It should limit the number of conversations that start with “Wait, I don’t understand why I got this grade.”
  • It provides students with resources for how to improve.
  • It offers students the chance to improve their grade.

Cons:

  • Students may still be distracted by a letter grade and not focus on comments. Perhaps you could not give them a letter or number grade until after the work is resubmitted.

Finally, this method may or may not save time while grading students work. You might spend more time commenting at the end of a paper, but you may not need to provide as many detailed comments throughout the paper–although those would still be helpful for students. Some quizzes and tests might take longer to grade with this method, or this method could be used flexibly on only those assessments that could have received the “See me” comment. It all depends on the assessment and on the quality and amount of feedback you want (or need) to provide to help your students learn and succeed.

 

Stereotype threat

“Stereotype threat is being at risk of confirming, as self-characteristic, a negative stereotype about one’s group,” according to Steele and Aronson 1995 who first defined this phenomenon. This sad phenomenon causes people to think along these lines, for example: “I am a girl. Girls are stereotypically bad at math, so I am bad at math.” Then this belief plays out in real life and the student performs worse than the dominant group. Students suffering from the stereotype threat perform poorly in tests or in class, discount the assessment or goals of the assessment, and/or become disenchanted with a class or field of study. This low performance seems to be due to anxiety, stress, emotional regulation, and performance regulation that distract from the task at hand; and this can in turn lead to inequality and segregated fields–or different preferences in one’s life because this does not apply only to school. This is a very important concern for Classics, a field dominated by white people.

Who is vulnerable to stereotype threat?

Everyone is vulnerable to stereotype threat, whether the stereotype has positive or negative connotations. If someone identifies as part of a social group, and that group has a stereotype about their performance in a particular field, their performance in that field is more likely to be affected by the stereotype, especially if it is highlighted somehow, such as on a questionnaire before a standardized test. The more one identifies with that group or has been stigmatized because of their identity in that group, the more pronounced the effects of stereotype threat may become.  Similarly, people who care more about success in a particular field are more likely to be affected by the stereotypes related to that field–this is not to say we should encourage students to care less but be aware of this stereotype threat.

There are some other interesting  factors for who is vulnerable. Ironically, students who are more proactive about overcoming the stereotype may be more likely to suffer from stereotype threat. Students who use their sense of humor to cope with negative events and who have a positive self-image are more likely to resist stereotype threat.  People with a high education level are less susceptible to stereotype threat–so we should be most aware of this with students in lower grade levels and in introductory courses.

And of course, stereotype threat is only a factor if people are aware of stereotypes related to thheir group. Children tend to learn more of these stereotypes between ages 6 and 11.

What situations trigger stereotype threat?

  • Asking about membership in a stereotyped group before an evaluation
  • Making the test-taker aware of a stereotype through subtle means, such as having a white examiner or grader evaluate a black student’s language abilities (Black students perform as well as white students when the examiner is black).
  • When a student is the only member of a minority group in the room during an evaluation that the stereotype might claim that student would do poorly on. Is the one student of color in a college Classics course an example of this?
  • The description of the task recalls or alludes to the stereotype.
  • Students are being evaluated, especially when the test is said to evaluate their intelligence or the limits of their abilities.

How do we reduce the effects of stereotype threat?

  • Cast the task or test in a light that doesn’t recall stereotypes. A unit test or final exam is, by its nature, diagnostic but we can say that it is fair for all sexes and races.
  • Not ask about students’ sex and racial identities until after the test, if at all.
  • Encourage students to identify as a member of a not stereotyped group. For example, call a class of students “Latinists” or “historians.” This positive title may encourage them to do better on the test or tasks too–a positive stereotype threat.
  • Encourage students to think of themselves as more complex people than “Boy” or “Girl” or “White” or “Black” or “Hispanic” or “Asian,” etc.
  • Give students the ability to self-affirm and say what they are good at throughout the semester to counteract the negative effects of stereotype threat. This is similar to encouraging metacognition.
  • When giving feedback, try to avoid playing into the stereotype threat. Instead, offer positive feedback and encouragement. Say that while you hold students to high standards, you believe that students can meet them.
  • Provide role models that work against the stereotype. These role models may be the teacher, the person administering the test, or the subject of an essay (such as a famous female archaeologist).
  • Explain why students may feel anxiety and attribute it to an outside source, such as stereotypes that don’t reflect students’ abilities or the stressful time of middle school.
  • Emphasize how intelligence grows over time and with work, how the struggle is good for you.  This destroys the validity of a stereotype that assumes a students’ intelligence doesn’t ever change.  Similarly, downplaying “talent” and “genius” will also help students see that intelligence is about learning and the struggle.

The insights and ideas in this blog post are all summarized from this incredibly helpful website, Reducing Stereotype Threat, that has lots of links to studies on this phenomenon.

T. H. M. Gellar-Goad’s “How Learning Works in the Greek and Latin Classroom” Blog Posts

In June 2014, T. H. N. Gellar-Goad began a seven part series of blog posts for the Society of Classical Studies. These posts took the insights from How Learning Works by Susan Ambrose et al. and discussed how to apply these ideas to teaching Latin and Greek. The last post of the series was in July 2015. They are all excellent and helpful. I provide links to them below, especially because the SCS website has recently changed and old links may have become dead.

These are all very good posts, and I hope to return to many of the topics and ideas in these posts for my own classes and my future blog posts.