It may seem obvious that to measure improvement in writing or public speaking, we need only administer a pre-test, apply a treatment (something we teach), and then conduct a post-test. If we see a higher score at the end of the process, skills have improved.
But when it comes to measuring changes in communicative competence, pre- and post-test measures can be inadequate. The reasons are many—students learn in many ways, so attributing change to one variable (the “treatment”) can be difficult. Control groups are almost impossible to create because seldom are two classrooms identical. Changes in ability can take quite a bit of time to surface, too. One 15-week class may not produce significant or visible results. To make assessment more complicated, scores on writing are often rated in widely different ways by readers, so conducting a meaningful study that depends on reliable raters can be difficult. And validity is always an issue: does a test of grammar predict ability to write a proposal? Does a timed test show what a student can do if given the time to revise? Bottom Line? Communicative competence is a highly complex blend of social and cognitive skills, and it’s difficult to assess.
Still, we can, and we should measure student performance in writing and speaking.
One thing we have to measure is the quality of products—a speech or an essay. Such judgments, to be useful, don’t have to follow the pre/post-test paradigm. Instead of looking at the change in individual students in one semester, we might examine how well students perform over a longer time. For example, we can start with a set of papers produced in a given course over a few years and compare changes in quality. The change would not be measured by a single student but by the class performance. If teaching over those years pays more attention to communication instruction, the quality overall should improve. Similarly, we might do a longitudinal study of a cohort of students as they move through a specific curriculum, or follow some of our seniors as they enter the work world or graduate study.
Another important but sometimes neglected measure is changes in attitudes and behaviors. Let’s assume that for a capstone course students are doing some difficult and challenging writing and presentations in genres that are new to them, and at the same time they are learning new material. Because of the demands on their time and the cognitive overload, a significant change in their written or oral communication may not be evident by the end of the course. However, they may have learned other valuable lessons that will result in gains later. They may, for example, learn they must pre-write and do careful and thorough research; they may have discovered the importance of practicing a speech over and over or of multiple revisions. The new habits may not have been absorbed in enough time to result in immediate and dramatic improvement, but they will probably be adopted in the future. And if we ask about attitudes and habits in a survey, an interview, or even a written reflection, we might be able to capture those changes.
As you consider how to measure communication skills, check out the newest section of the University Writing Center’s faculty pages on assessment: http://writingcenter.tamu.edu/about-w-c-courses/assessing-w-and-c-courses/. Let me know how I can support your efforts to assess communicative competence. I’ll help you think it through and do my best to connect you to the resources you need.