Effective Use of Rubric to Assess Critical Thinking in Educational Leadership Students
Student Learning Outcome:
Think critically, using analytical, qualitative, and quantitative reasoning, to apply previously learned concepts to new situations, complex challenges, and everyday problems.
Methods:
Student progress toward the learning outcome was assessed using a scoring rubric for related items on the qualifying exam (administered at the end of the first year of coursework). For this SLO, attention was paid to the following outcomes as scored on the rubric:
- Breadth and depth of knowledge
- Conceptual synthesis
- Application of theory to practice
- Use of evidence
Qualifying exams were scored using a three-point rubric (pass with commendation, pass, no pass) addressing 10 outcomes. Students were required to receive a “pass” on all 10 outcomes to consider this SLO met. Students who do not pass on a first attempt were allowed a second attempt.
Exams were administered and scored by the qualifying exam committee, which consists of 4-6 program faculty. The committee uses a double-blind scoring process.
| Criteria |
Pass With Commendation (Exceptional) |
Pass (Very Good/Good) |
No Pass (Unacceptable) |
Comments |
|---|---|---|---|---|
| 1. Breadth and depth of knowledge | The writer uses multiple perspectives, theories, and concepts from the courses to demonstrate a clear understanding of the question. | The writer uses a few perspectives, theories, and concepts from the courses to demonstrate an understanding of the question. | The writer does not refer to any perspectives, theories, or concepts from the courses to demonstrate an understanding of the question; or applies perspectives, theories, or concepts incorrectly to answer the question. | |
| 2. Conceptual synthesis | The writer makes multiple connections across the course concepts and identifies critical issues when answering the question. | The writer makes at least one connection across the course concepts and identifies critical issues when answering the question. | The writer does not connect the course concepts; or does not identify a critical issue when answering the question. | |
| 3. Application of theory to practice | The writer presents a robust, well-substantiated, coherent argument for which theories address the issues of practice raised by the question. | The writer presents an argument for which theories address the issues of practice raised by the question. | The writer either does not present an argument for which theories address the issues of practice raised by the question, or applies theories incorrectly to answer the question. | |
| 4. Critical analysis of the research literature | The writer correctly references and critically analyzes the perspectives of several authors and/or theorists whose writings are highly relevant to answering the question. | The writer references and critically analyzes one or two authors and/or theorists whose writings are relevant to answering the question. | The writer does not reference or critically analyze any authors and/or theorists whose writings are relevant to answering the question; or references authors and/or theorists whose writings are unrelated to answering the question. | |
|
5. Informed decision making |
The writer makes excellent recommendations for improving or changing practice by referencing theories in the professional literature, the concepts covered in the courses, and/or data directly pertaining to the question. | The writer makes good recommendations for improving or changing practice by referencing theories in the professional literature, the concepts covered in the courses, and/or data related to the question. | The writer cannot recommend how to improve practice; or does not reference the professional literature, concepts covered in the courses, and/or relevant data; or makes a poor or inappropriate judgment. | |
| 6. Point of View | The writer demonstrates a sophisticated understanding of the issues raised by the question and lays out a very clear line of reasoning that answers the question substantively. | The writer demonstrates an understanding of the issues raised by the question and lays out a line of reasoning that answers the question adequately. | The writer does not express a point of view; or misunderstands the issues raised by the question, or answers the question without providing justification. | |
| 7. Diversity | The writer thoughtfully addresses issues pertaining to educational equity, diversity, inclusion, and/or multiculturalism implied by the question. | The writer takes educational equity, diversity, inclusion, and/or multiculturalism implied by the question into account when answering the question. | The writer does not discuss issues of educational equity, diversity, inclusion, and/or multiculturalism implied by the question. | |
| 8. Self-awareness | The writer demonstrates self-awareness by discussing how her life history has shaped her views about education when answering the question; the writer is a reflective practitioner. | The writer demonstrates some self-awareness by discussing how her life history has shaped her views about education when answering the question; the writer is reflective. | The writer does not convey self-knowledge and does not discuss how her life history has shaped her views about education; the writer is not reflective. | |
| 9.Evidence | The writer presents a fully developed, relevant, and persuasive argument for all debatable assertions and draws evidence from multiple sources when answering the question. | The writer presents an argument for most debatable assertions and draws evidence from a few sources when answering the question. | The writer does not present evidence or support for her claims from any source; or relies solely on her own experiences, or refers to questionable sources. | |
| 10. Control of language | The writer communicates very effectively as | The writer communicates with few errors in the | The writer has intermittent control of language |
_____ Pass with Distinction
_____ Pass
_____ Pass with Reservation
_____ Fail
Results:
9 of the 14 students who completed the exam passed on the first attempt. Of the 5 students who did not pass on the first attempt, two passed on the second attempt, two took leaves of absence to further prepare for the exam, and one failed on the second attempt. The two students who took a leave of absence to further prepare for a re-take were both African women. No patterns were found to exist in success rate by gender or ethnicity.
Improvement Actions:
Final scores were compiled and analyzed by the program director and then reported to the Department, the Steering Committee, and the Executive Board for feedback and recommendations. Scores were slightly lower than in past years, which may reflect changes in a textbook and instructor for one of the core classes. Supplemental feedback from students suggested that this text did not provide sufficient depth in organization theory. The textbooks for that class are being adjusted for the coming year to provide greater exposure to primary source material.
Also, the fact that two African women did not pass and took leaves to further prepare for the exam may reflect language-related concerns in the classroom, especially as related to mastery of breadth and depth of content, an area in which both struggled. A faculty member is meeting with these students to review core content in preparation for the exam.
