Assessing Student Communication Skill in an Advanced Business Communications Course
Student Learning Outcome:
Upon completion of course BUAD 301 Advanced Business Communications, students will be able to communicate, influence, and inform using effective oral and written communication.
Methods:
A pre-test and post-test (final exam) required students to write a recommendation report using the case-analysis method. Each instructor developed the final-exam case for his/her class and scored the papers using prime trait analysis (rubric scoring across five criteria and process levels). Scores were then categorized as Below Average (0-59%), Good (60-79%), and Excellent (80-100%).
BUSINESS WRITING PROGRAM'S C-L-A-S-S EVALUATION CHART
| Criteria: | Very Poor (0) | Poor (1) | Average (2) | Good (3) | Excellent (4) | Score |
|---|---|---|---|---|---|---|
| C CONTENT/ CASE: completeness, case understanding |
Misinterprets case assignment; makes major factual errors; misses two or more elements of the case. | Misses one major element of the case; leaves out essential information; some minor factual errors | Handles case material competently; includes essential information; factually correct. | Handles all elements of the case with skill; develops and supports ideas in a better-than-average way. | Handles all elements of the case professionally; develops and supports ideas using well-chosen examples and creative details. | |
| L* LITERACY: grammar, spelling, punctuation |
Makes numerous, serious, grammatical or syntactical errors; frequently misspells homonyms; fails to run spell check; exhibits carelessness. | Makes disruptive grammatical/syntactical errors such as run-ons, fragments, unintelligible sentences. | Writes generally correct prose; occasionally fails to catch minor grammatical errors. | Proofreads well enough toeliminate most grammaticalerrors; may have minorproblems with punctuation or usage. | Makes virtually no grammaticalor syntactical errors. Establishes credibility with the audience. | |
| A AUDIENCE: "you" attitude; awareness of reader's needs |
Lacks audience awareness. Is rude, hostile, discourteous, or insulting to the reader. | Writer-focused; lacks you attitude, positive emphasis, awareness of reader’s information needs. | Is polite; does not slight the reader. Uses positive emphasis. | Is courteous; addresses readers' needs and/or concerns; makes no unreasonable demands. | Reader-focused; addresses readers' questions and/or objections; creates goodwill. | |
| S STRATEGY: purpose, effectiveness of approach, professionalism, means used.Writer solves the problem. |
Presents a disorganized, unprofessional document. Projects a negative imageof the writer and of the organization. | Is unclear about purpose; unclear topic sentences, arrangementof ideas, and transitions. | Is clear; correctly uses the "checklist" approach; makes no serious false step; gets the job done. | Employs good strategy; finds a fresh way of solving the problem; effective sequencing of ideas. | Adopts strategy to achieve desired outcome; clearly defines purpose and uses logical and/or emotional appeal effectively. | |
| S STYLE: a) tone, word choice, clarity, conciseness, fluidity _ _ _ _ _ _ b) document design |
a) Uses garbled style. Plagiarizes. _ _ _ _ _ _ _ b) Format interferes with readability. |
a) Writes in a notably awkward manner: misuses words and idioms; uses slang; wordy; uses some borrowed language. _ _ _ _ _ _ _ b) Imbalanced or cluttered design. |
a) Writes serviceable prose; uses active voice, strong, action verbs; rarely uses jargon or clichés. _ _ _ _ _ _ _ b) Readable format. |
concisely, and coherently; employs syntactical variety with general success. Creates a friendly, business-like, positive style. _ _ _ _ _ _ _ _ b) Design helps readers find the information they need. |
a) Demonstrates a sophisticated grasp of the language; writes in a fluid manner; varies syntax and vocabulary; uses original language. _ _ _ _ _ _ _ _ b) Design helps readers understand and remember information. |
|
| This is what your grade would mean at work. | Your position is in jeopardy. | The boss is scrutinizing your work for improvement. | The boss judges this document acceptable subject to minor revisions. | Your job promotion is probable. | Your job promotion is ensured. |
Comment: ________________________________________________________________________________________
*Literacy is a threshold category. You must earn a "C" in literacy to receive a passing grade on your assignment.
Update: 6/09/05; 12/04/2011
Results:
After examining and comparing the results between 2009 and 2012, it was found that students demonstrated significant improvement in all areas. Although the change in the rubric and possible instructor bias may have resulted in what may appear to be inflated scores, the trends are clear. The criterion that was specifically focused upon—the Strategy criterion—was exceptional: “Below Average” ratings fell from 30.8% in 2009 to 11.84% in 2012. Strategy and Style have demonstrated the greatest reduction in below average scores on a year-to-year basis.
| Evaluation | Above Average (%) | Average (%) | Below Average (%) | |||
|---|---|---|---|---|---|---|
| 2009 | 2012 | 2009 | 2012 | 2009 | 2012 | |
| Overall Score | 17.3 | 43.96 | 56.3 | 50.86 | 26.5 | 5.18 |
| Content | 21.4 | 50 | 54.5 | 37.5 | 24.1 | 12.5 |
| Literacy | 24.8 | 59.86 | 53.3 | 31.58 | 21.6 | 8.56 |
| Audience | 16.1 | 44.08 | 70.8 | 46.71 | 13.1 | 9.21 |
| Strategy | 18.9 | 44.74 | 50.3 | 43.42 | 30.8 | 11.84 |
| Style | 15.6 | 57.61 | 55.2 | 32.45 | 29.2 | 9.86 |
| 2009 N=288; 2012 N=152 | ||||||
Improvement Actions:
Significant improvements in student learning across all categories (content, literacy, audience, strategy, and style) occurred and demonstrate that good improvements were made, exceeding expectations for assessment of the goal.
Based on the results, the following actions were taken before the Spring 2014 assessment cycle:
1. Faculty development seminars were offered to explain formal rhetorical elements and strategies used in the textbook, and offer possible methods to teach this form of critical thinking and persuasion to students.
2. Faculty development seminars were offered that demonstrate how to incorporate writing tools (methods of analysis, development, and organization) into assignments.
3. Evaluation and standardization of the case-analysis method taught in BUAD 301 was continued.
4. In the five-step case-analysis method taught in BUAD 301, instead of requiring “Facts and Related Issues,” it was decided “Analysis of the Problem and the Assumptions” should be substituted instead.
