Dr. Terri Flateby, director of assessment at the University of South Florida, presents a UWC-sponsored workshop on assessing writing Feb. 21 during the assessment conference hosted by Texas A&M.

Instructors who give writing assignments are all too familiar with grading students’ written work. They’re typically less familiar, though, with assessing student writing. The latter is concerned not with evaluating the performance of individuals, but rather with identifying overall patterns of strengths and weaknesses in student work to find ways to teach both writing and course material more effectively.

For Dr. Terri Flateby, that’s an important distinction and one she hopes writing instructors will give more attention.

Flateby, director of assessment at the University of South Florida, came to Texas A&M in February to present a workshop on writing assessment. The workshop, sponsored by the University Writing Center, offered faculty a chance to think about how writing can enhance their pedagogical goals.

For Flateby, writing assessment means “looking at the aggregate and asking what is it students are doing well in general and what is it they’re not doing as well and what can we do to improve?” Those questions should lead back, Flateby says, to additional reflection on instructional methods, assignment design, and curriculum mapping.

To facilitate writing assessment, Flateby and her colleague Elizabeth Metzger have created an assessment tool known as the Cognitive Level and Quality Writing Assessment (CLAQWA.) The instrument is designed both to give specific feedback on writing and to help instructors determine the cognitive level achieved in a text.

The emphasis on cognitive development is one of the things that sets CLAQWA apart from most other assessment instruments. For Flateby, identifying the level of thinking students’ exhibit in their work is crucial and something she believes instructors should consider from the outset.

“I think it’s very important when an assignment is developed for the instructor to think about the cognitive level students need to reach in that assignment,” Flateby explains. “Faculty often think they’re asking students to analyze or synthesize, when in fact they’re just asking students to explain something. So, the faculty member is disappointed, believing students haven’t done what was requested.”

“Faculty sometimes hesitate to give writing assignments because the writing is so poor,” Flateby acknowledges. But she believes there are tools that can help make reading student papers a less onerous task.

One of the many offshoots of the CLAQWA tool, for instance, is a version designed for peer review. It’s very popular on Flateby’s home campus and Flateby herself has become a strong believer in the value of having students respond to the writing of their peers.

“We have faculty who will not give a writing assignment without peer review,” Flateby says. “They feel so strongly about it because they’ve seen how the writing improves.”

Students do have to be given some guidance in how to comment effectively, though. “An instructor can’t just say ‘Review each other’s papers,’” Flateby says. But with appropriate instruction, she feels the results can be impressive.

When asked about the upcoming program- level assessment of Texas A&M’s W course initiative, Flateby encourages faculty here to recognize that any kind of assessment is an iterative process.

“Assessment can be an evolution,” she observes. “It doesn’t have to be perfect.”

She herself would like to see universities integrate writing into many more courses: “I’m a strong believer in having writing not only in an actual W course, but having writing continue throughout the curriculum. If writing is isolated, students don’t understand the importance of it.”

She compares learning to write to learning to play the piano: if pianists become proficient, but then stop playing, what happens?

“If they go back to playing years later, the proficiency is not going to be the same, and they’re not going to develop and learn nuances,” says Flateby. “There’s an important analogy there. We have to continue to develop students’ writing throughout the curriculum and in various disciplines.”

How do your students react when they read your comments—if they read them? Do those comments help your students write better?

Research suggests that instructors’ written comments—at least the more typical kinds of comments—do little to help students improve their writing. Since making those observations can be time-consuming, it’s worth taking another look at how to make your comments matter. Perhaps the most helpful way to think about paper comments is to imagine sharing your own writing with a colleague. You’d expect a respectful tone, an honest (but kind) sense of the worth of your effort, and—most of all—some specific guidance on how to proceed. With that in mind, here are some additional suggestions.

Time your words.

Commenting on final papers is largely futile; once you put a grade on a paper, students have little interest in anything else you say. Reserve the bulk of your comments for earlier drafts, when students can consider your advice in their revisions.

Find your focus.

Ideally, when you give students an assignment, you have a specific pedagogical goal in mind. Perhaps you want students to summarize both sides of an argument or use research to defend their position. Keep that goal in mind as you read, so you can focus your comments on bigger issues. That’s more efficient for you and less overwhelming for students.

Ask questions.

By asking questions, you can encourage students to expand their assumptions, or push the limits of their knowledge. When you ask questions rather than make suggestions, you keep students engaged in the process and remind them that it’s ultimately their work, not yours.

Be specific.

When you’re working through a batch of papers, it’s tempting to rely on stock phrases: “Good introduction” or “Needs more analysis.” But such wholesale comments are of little use to students when they revise. The more specific your questions and comments, the more likely you are to see improved final drafts.

Be quiet.

Sometimes students don’t need an instructor to say anything about their writing, particularly on brief assignments or explorations leading to a longer project. Or take a few minutes of class time to give general impressions about what you read. (“From your journals, I see that many of you are confusing two key concepts.”)

Resist the urge to correct.

When most instructors see grammatical mistakes and spelling errors, they feel compelled to correct them. Unfortunately, correcting errors for students doesn’t teach them much. It also sends the message that you’re hunting down errors rather than considering what students have to say. If a draft has an overwhelming number of errors, you might mark up one paragraph or one page and write a note offering to discuss the problems in person or suggesting the student get help from he University Writing Center. You can also er models for students to revise as a class or in a group. Another option: teach students to correct the three or four errors that paricularly frustrate you as a reader, and limit yourself to identifying only those. But isn’t it a teacher’s job to point out errors? Not if pointng them out will do nothing to help students eliminate them in the future.

Resist the urge to rewrite.

Just as it’s tempting to fix errors, it’s also tempting to rewrite a wordy passage or awkward sentence. But making such editorial changes denies students the chance to learn to revise for themselves. Furthermore, there are countless ways to revise a sentence; by choosing one, you impose your preferences on your students. Besides, if students are going to undertake a major revision, they may cut or reword the problematic section anyway. If you’d like to teach students how to revise for style and clarity, consider having the class work together to rework a model—perhaps a less-than-stellar piece of “professional” writing from your field.

Be a reader.

Writing is ultimately about communicating ideas and information to a reader, a fact students often forget. Let them know you are, first and foremost, a reader, trying to make sense of their words. The reader’s comment “I can’t follow your meaning here” is less hostile than the teacher’s “Faulty syntax!” and more likely to encourage students to revise.

Make it a conversation.

Before students turn in a draft, ask them to write their own comments about it. Then you can address their concerns and answer their questions, which allows you to be teacher and advisor, rather than judge and jury. It also encourages them to take responsibility for their written efforts and reflect on what is and isn’t working for them in their writing process.

Writing courses are intensive for faculty, too

What’s it like to teach a W? The University Writing Center recently sent an informal survey to some W course instructors to get their impressions about teaching a writing-intensive class. Some, perhaps buried under the stacks of yet-to-be-graded student papers, never answered. But a few found time to share their insights. What follows is a sample of their responses. (To read more, click to www.writingcenter.tamu.edu/blog.)

What advice would you give to someone teaching a W course for the first time?

 

"Be sure to develop self-discipline with grading papers: get them back as early as possible, but try to spread out the grading so you don’t wear yourself out."
— Jim Aune, Communication

"Attend a UWC faculty workshop."
— Harvey Tucker, Political Science

"Many of the students have varying levels of trepidation about a W course, so allay their fears. Make expectations clear and let them know you don’t expect perfect grammar and organization on every assignment, that writing can have different purposes."
— Courtney Schumacher, Atmospheric Sciences

"Use the UWC: they can help! Also, try to demonstrate the relevancy of writing and/or writing assignments to students in their career preparation. If students can see how learning to write can help them in their careers, they tend to be more engaged in the assignment."
— Roel Lopez, Wildlife and Fisheries Sciences

Have you noticed changes in your students writing over the course of the semester? If so, what kind of changes?

"My students write much better at the end of the course than at the beginning. It even carries over into their problem working in that they write complete, proper sentences at the end instead of fragments. So grading their problems and proofs is easier at the end. Also, they read more critically, so they get more out of the textbook. I think they are amazed at the power of language, both written and oral, and become more precise in its usage."
— Sue Geller, Mathematics

"They get better. Not in all cases, but in enough to make it worthwhile. They are better organized, more conscious of writing in complete sentences, and more adept at transitions from idea to idea. They usually learn (re-learn?) about paragraphs, a concept many have forgotten and do not use in the beginning of the semester."
— Sarah Bednarz, Geography

Has more writing for the students meant more work for you? If so, how have you handled the increased demands on your time?

"Although the additional writing has its rewards . . . it has clearly meant more work for me in terms of assessment/grading. This semester our department has hired two graduate teaching assistants to assist
the [department’s] W instructors with the increased workload, and I believe this will be a tremendous help to all of us. In the past, I guess I just slept less."
— Pat Wiese, Teaching, Learning and Culture

"Converting my course into a W course did not mean more work. The course already was writing intensive due to the nature of the course material. Working with the UWC, I was able to develop grading rubrics and developed an on-line paper submission Web site that increased my efficiency in grading assignments in the course."
— Roel Lopez, Wildlife and Fisheries Sciences

"For too long, I graded papers on a schedule that seemed to be endless. I have learned to come up with assignments that are low-stakes enough that the students have to write a lot but I can just check quickly, more to see if they made an earnest attempt to learn the material than to nail them for grammar and style errors."
— Darla-Jean Weatherford, Petroleum Engineering

"Of course it means more work for me, but it is work I genuinely enjoy. I like reading the ways students express their ideas—it is a window into their lives and thought processes. Sometimes what I see is pretty appalling. There is no sense of logic or how to lay out an argument, or understanding of what it means to evaluate or synthesize. Understanding this has helped me to become better in my teaching of the thinking/research/writing process. I don’t know how I handle the increased demands on my time other than to think of it as contributing to my personal pedagogical development—the scholarship of teaching— and to realize it is all part of my job as a professor."
— Sarah Bednarz, Geography

How are you responding to your students’ writing? Do you use a rubric, hold conferences, or facilitate peer review?

"I provide detailed instructions for writing assignments and also a list of frequently seen problems and intolerable writing mistakes. I give detailed comments on each submission. Students who write multiple short papers improve their skills and grades quickly."
— Harvey Tucker, Political Science

"I use several tactics in providing feedback to students including grading rubrics, peer review, weekly class discussions, and use of Microsoft’s ‘track changes.’"
— Roel Lopez, Wildlife and Fisheries Sciences

"I go through seven stages: a) shock and disbelief, b) denial, c) bargaining, d) guilt, e) anger, f ) depression, g) acceptance and hope. Acceptance and hope usually come long after the semester is over, so I am generally in an earlier stage when the grades need to be given."
— Sam Cohn, Sociology

Has anything about teaching a W course surprised you?

"How much the students’ writing typically improves after the first assignment. I suspect many A&M students can actually write pretty well, but often they are just not challenged to write well. When they find out that good writing does matter in their grade, they respond and ‘kick it up a notch.’"
— Andrew Klein, Geography

"Our students are very, very poor writers. Students resent the extra work; hence student evaluations have dropped a little. The lack of meaningful support from the administration for instructors making the extra effort is discouraging. Just a quarter-time graduate assistantship to help with the grading for each W course would be reasonable. Instead, I use contracts and grants to subsidize my W course."
— T. H. Friend, Animal Science

"I was surprised at how much fun the students have with the term paper. Who’d have thought that a term paper in math would be fun?"
— Sue Geller, Mathematics

How has writing in the course affected student learning?

"Students are much more engaged and interested in the course. They easily see the relevancy of the writing assignments in their future careers. Working with real landowners really makes this connection."
— Roel Lopez, Wildlife and Fisheries Sciences

"Since they read more critically, they learn more from the textbook. Since they are aware of the need for precision in their language, they produce better work."
— Sue Geller, Mathematics

What’s the best thing about teaching a W course?

"The knowledge that you are testing people on real intellectual skills and their ability to deal with ideas—rather than rote gurgitation on a spitback exam. The opportunity to have extended written dialogue with students on sociological themes, rather than check whether they have read pages 80-220."
— Sam Cohn, Sociology

"Feeling like you’re dealing with students as individuals (something that, alas, doesn’t happen often in such a large university with such large classes)."
— Jim Aune, Communication

"By far the best thing about teaching a W course is getting to better learn about students. I especially enjoyed reading student book reviews as it provides insight into their philosophies and interests. This is something I don’t normally experience in my other classes."
— Andrew Klein, Geography

What’s the worst thing about teaching a W course?

"Dealing with my own shortcomings in writing and in English. Like many in my area, I was drawn to engineering because I enjoyed science and math and didn’t care so much for English class. If I’d have known then that I’d need a better handle on this, I’d have paid more attention."
— Ronald Lacey, Agricultural Engineering

"Tell my wife I love her. I will be home after I finish this stack."
— Sam Cohn, Sociology

UWC Executive Director Valerie Balester is beginning the process of assessing the effectiveness of the Wcourse program, an undertaking made more difficult by the organic nature of how students learn to write.

As we move into the second phase of implementing writing-intensive courses at Texas A&M, we naturally want to know how well we are doing. Fortunately, research data about writing instruction can help us design an assessment. We know, for instance, that requiring more writing will not alone improve writing ability. Writers need guidance, and, as David E. Harris and Robert Schaible point out in their review of research, writers must be evaluated not only on content, but also on organization, style, grammar, spelling, and presentation.

W courses have been designed with such research in mind. Now we need to evaluate if our efforts are paying off at Texas A&M. Are our students better writers as a result of taking W courses?

It seems a simple enough question with a simple enough means of assessment: test student writing at the beginning of a W course, re-test at the end, and measure improvement. Unfortunately, many factors complicate the process.

Let’s start with the obvious: the inherent difficulties of measuring writing ability. With careful definition of a writing task and training of raters, a reliable measure of a particular document can be achieved. However, even if students can write one type of document well in one instance, they may not be equally competent writing other types of documents. Measuring a single performance on a single assignment provides only a partial view of writing ability. One way to address this limitation is to examine portfolios of student work, written and revised over time.

Another, perhaps less obvious, complication of assessing writing is that writing skill develops over time and at different rates. Some students show only minimal improvement over the course of a semester, with more substantial results becoming apparent only later, assuming that the students continue to write. Further, other variables can affect performance, from time-on-task and prior knowledge of a subject to how seriously students view the writing requirement.

Our expectations, therefore, must be realistic, and our measurements should not be limited to a single semester or too narrowly focused on individuals.

By the same token, we should not seek results that define better writing merely by better written products. We could achieve better written products easily enough by editing our students’ work for them. But that wouldn’t make them better writ- ers. To help our students achieve that, we must:

1. 1. Increase their repertoire of rhetorical strategies, for example, by enhancing their ability to analyze and address various audiences and understand specific genres.
2. 2. Influence their writing habits and processes, for example, by encouraging them to plan, revise, edit, and proofread, and to seek out and value feedback.
3. 3. Improve their attitudes about writing, for example, by decreasing writing anxiety and raising awareness about the importance of writing to their future.

To measure changes in strategies, habits, and attitudes, our assessment should also include indirect measures of writing ability, such as reflective writing or surveys of students and of faculty who observe them.

Over the course of the next year, I will devise an assessment strategy for the W course program. It will be a realistic plan that will include student writing and surveys of students and faculty, as well as data from sources such as the National Survey of Student Engagement conducted on this campus. What we learn will help us decide how to invest wisely in a program we cannot afford to do without. In the end, I know we’ll discover new questions we’ll want to ask. After all, assessment, if it is to be of use, must be ongoing.