Report to WASC Self Study Steering Committee
Edward J. Fink, Ph.D.
Department of Communications
11 June 1999
I teach UNIV 100, Introduction to University Studies, in the Fullerton First Year (FFY) program. This past semester, Spring 99, was the second semester of the class, UNIV 100B. I employed multiple assessment methods in this course to evaluate student learning. I found this combination of methods to be beneficial in generating fair grades for the students.
The students in UNIV 100B were required to perform service learning activities as part of their course work. Specifically, each student volunteered a minimum of 30 hours at a community organization. This service learning component constituted 60% of the course grade.
To assess the learning outcomes of the service projects for the students in my section of the course, I used three means of evaluation. First, 10% of the grade was based objectively on participation and completion of the minimum 30 required hours of volunteer work. Second, 20% was based on a written evaluation by the site supervisor (a standard evaluation for all class sections). Third, 30% was based on a reflective essay.
To evaluate the essays, I employed a primary trait scoring method with 10 traits. Each trait was scored as either 1 (poor), 2 (satisfactory) or 3 (good). This three-point scale eliminated the headache of five or more points, which often require shades of discrimination that are too subjective to be replicable. I summed the scores across the 10 items and assigned grades according to the following scale:
A = 27-30 (90-100%)
B = 24-26 (80-89%)
C = 21-23 (70-79%)
D = 18-20 (60-69%)
F = 17 or below (0-59%)
The ten primary traits I used for assessment were:
1_____ Clearly stated thesis (simple, declarative statement, preferably up-front)
2_____ Organization of thoughts (ideas flow smoothly between sentences and paragraphs)
3_____ Concise conclusion (argument summarized succinctly)
4_____ Clarity of writing (declarative sentences, complete thoughtsone per sentence, no obtuse language, paragraph ties, smooth flow of words)
5_____ Correct mechanics of English (spelling, punctuation, grammar, sentence construction)
6_____ Uniqueness of thought (a perspective all your own)
7_____ Creativity of ideas and writing (demonstrated in language usage)
8_____ Depth of analysis (evidence of reflection and critical thinking)
9_____ Logical argument (each point reasoned and supported well)
10____ Validity (essay makes sense)
By using primary trait scoring to evaluate the essays, I was able to offer my students assessment feedback for each item on the evaluation sheet. This allowed me to identify for them both strengths and weaknesses in their critical thinking and writing. By utilizing supervisor evaluations and objective participation scores along with reflective essays, I was able to assess the students learning outcomes by more than just one method. This enabled me to give them due credit for their service projects without holding them to one means of assessment. I believe this triangulation of evaluation methods resulted in grades that were as fair as could be achieved for assessing student learning outcomes through service learning activities.
An analysis of the grades from each of the three assessment methods reveals no significant correlation. All students completed at least 30 hours of community service, so all students earned an "A" for the participation component of the grade (10%). Additionally, all students received high marks from their site supervisors, resulting again in a grade of "A" for this component of the evaluation (20%). The grades for the essays, however, ranged from A to C. Therefore, those who received lower essay scores received higher scores from their site supervisors and for their participation. This explains the lack of correlation between scores across the three methods. High scores for participation or from site supervisors were not reliable predictors of essay scores, and vice versa.
I interpret this lack of correlation as a positive outcome for the students. For those who were not great essay writers, this triangulation of assessment was particularly helpful. Generally, the students performed outstanding volunteer service and learned much about their roles in their communities. For these efforts, they earned high scores for participation and from their supervisors, which constituted two of the assessment methods. The third method--a reflective essay--yielded a wider range of scores and was therefore useful in differentiating among students. These three methods of evaluation complemented each other to yield overall balanced grades for the students service learning activities.
Anecdotal evidence suggests that the reflective essay itself was a valuable learning experience. A large body of literature explores the utility of writing as a learning tool across the curriculum. The experience of the students in my section of UNIV 100B supports the value of such writing. The students were given a standard evaluation to complete after their service learning, and on that evaluation a number of them wrote that the reflective essay was a very useful tool for them to process what they had learned from their volunteer efforts.
These findings suggest that it is beneficial to employ multiple means of assessment when evaluating student learning, especially learning from non-traditional activities, such as community-based service learning. Just as students learn in different ways, so should different ways of assessing their learning be employed. In particular, I found participation, site supervisor evaluations and reflective essays to be three useful--and complementary--methods for assessing student learning from the community service activities in UNIV 100B. I believe that evaluating the students performance from this triangulated approach resulted in fairer, more balanced grades than a single method of assessment would have yielded.