The Coulter Faculty Center eLearning Faculty Fellows at Western Carolina University developed the “Online Course Assessment Tool and Peer Assessment Process” or “OCAT.” This is essentially a checklist style rubric designed to provide peer feedback on the design of their online courses.
What Problem Did This Solve?
According to the notes on the rubric, they developed this tool and confidential peer assessment process to provide faculty with constructive peer feedback on the design and instruction of online courses. Their expected benefits were:
- Constructive feedback regarding teaching effectiveness
- Instructional improvement
- Faculty development
- Opportunities for peer support
What Does The Rubric Assess?
The rubric has seven sections. The first five sections assess course design and teaching concerns. The last two sections provide spaces for summary narratives from a peer reviewer and an instructor response.
The first five sections assessing the course include:
- Course Overview & Organization
- Learner Objectives & Competencies
- Resources & Materials
- Learner Interaction
- Learner Assessment
There are some interesting ideas in this rubric that I think are important to student success in online courses that you usually do not see covered in a rubric. For instance, under “Resources & Materials” they include “provides opportunities for students to contribute to course resources.” This is a powerful learning opportunity for students in online courses. Of course, I would like to see something about open education resources or openly licensed materials here (open textbooks or Creative Commons licenses).
Weaknesses of the Rubric
The implied definition of an online course in this rubric relies too much on content delivery rather than knowledge creation or community building. To be fair, the rubric includes statements like “fosters interaction among constituencies inside and outside the course as appropriate (e.g. student-student, student-instructor, and with external persons or agencies)” but “fostering” is not the same as integrating that interaction into the curriculum. There is a recognition of different learning modalities (“learning styles”) in the rubric but no attention to accessibility. They must address this elsewhere which, given the expertise needed to really address accessibility, is understandable.
Strengths of the Rubric
This is one of those rubrics where the purpose and the process is as important as what it assesses. Some of the issues around using rubrics for assessing online courses include Academic Freedom and possible union issues around evaluating faculty. This rubric is focused on peer assessment. A note in the rubric says “the peer assessment instrument itself will also be available for faculty use as a self-assessment faculty development tool” which, I think, is the most valuable use of an assessment process. The process includes meeting with a trained, peer reviewer. The other strength of this rubric is that each section has a space for writing down “instructional item(s) emerging from peer discussion not included in the list above.” This helps address the individuality of the teacher and different teaching styles. Rubrics should not be used to create cookie-cutter courses. It is obvious that the team that put this rubric together did some research and brought in their local academic community to develop this process.