I have a lot of experience with this rubric as a teacher, instructional designer, and administrator. I have worked with the Quality Matters rubric in some form or another since the early 2000’s. I was certified as a reviewer through the California State University Chancellor’s office and have reviewed numerous courses using the rubric. While there are many things to like about the rubric, I am a big fan of the early organization. One of the most useful things accomplished by Quality Matters is their research library which gathers research on the standards of the rubric such as interaction.
History
The QM rubric has certainly been an influential rubric. Before fiscal year 2007 or their grant, the rubrics and research were freely available to the public (after that, they introduced a business model of fees and subscriptions to support the research and training). According to their site, Quality Matters began with a small group of colleagues in the MarylandOnline, Inc. (MOL) consortium who were trying to solve a common problem among institutions: how do we measure and guarantee the quality of a course? In 2003, they applied for a Fund for the Improvement of Postsecondary Education (FIPSE) grant from the U.S. Department of Education to create the Quality Matters program.
What does the rubric assess?
The rubric addresses eight research-based standards for successful, quality online courses. They include:
- Course Overview Introduction
- Learning Objectives (Competencies)
- Assessment and Measurement
- Instructional Materials
- Course Activities and Learner Interaction
- Course Technology
- Learner Support
- Accessibility and Usability
Strengths of QM
I have seen this rubric in action. It has its short-comings but one of the most important things it does is to get an instructor to think about what happens in an online course and how courses, online, hybrid, and face-to-face, can be successful. The rubric gets faculty to think about how they communicate the goals and expectations of the course to their students (standards 1 and 2). Sometimes the disconnect in a course can be that an objective of the course isn’t actually assessed. I can personally attest that faculty with little to no online experience were able to use the QM rubric to build a successful online course. The tool helps faculty to understand issues in online learning that they may not have considered before such as student interaction and accessibility of course materials.
I have worked with a few instructors who, after going through the Quality Matters training, not only created an excellent online course, but applied the principles to improve their face-to-face classes as well. It is a great opportunity to get instructors thinking about teaching and learning in general rather than their subject in particular.
This points out one of the things this rubric does well: it is not so much the rubric itself but the process. Some of the best work on a college campus comes when you bring faculty together to review one another’s courses and talk about what worked for them and what didn’t. The faculty are often very creative in meeting to outcomes of the standards in ways that I am sure the QM folks could not imagine.
I like the caveat included in the rubric about accessibility (standard 8). It says “Meeting QM’s accessibility Standards does not guarantee or imply that specific country/federal/state/local accessibility regulations are met. Consult with an accessibility specialist to ensure that accessibility regulations are met.” The idea that a course can be declared accessible by checking off five items in a rubric is a bit ridiculous.
Weaknesses of QM
I think that the financial model is an issue. I know a lot of faculty who are enthusiastic about it because QM pays faculty to review courses from part of the fees they charge the institutions. I understand and value the recognition of faculty expertise and time, yet we typically don’t pay faculty extra money to observe and review face-to-face courses. For small, cash-strapped institutions subscriptions and fees are a burden.
The QM rubric seems weighted towards objectives and assessment and, in my experience, does not allow for adequate assessment of teacher-student or student-student interaction, community building, fostering student agency, or opportunities for students to guide the learning experience. I have written before on this blog about my conflicted relationship with course objectives. With the rise over the last 20 years of online learning, we are learning a lot about how students learn (e.g. Connectivism) and as much as I appreciate all of the research that has gone into this rubric, it needs a refresh. Standards 2-6 all have “learning objectives and competencies” as the first concern. While learning objectives have their uses, they are often more important to the institution’s ability to supposedly measure them than with the student learning. Sally Stewart did some interesting research around these questions (see the The Trouble with Learning Outcomes). And I get it, students need to know what will be covered in their Psych 100 class, but the objectives don’t teach the class. Often, the most important things that happen in a classroom are the things that can’t be measured, but we will not see a standard for “opportunities for transformative learning experiences” or creativity anytime soon. I am not above negotiating course objectives when students express a need or desire, or when we need to change the direction of a class due to current events – that kind of flexibility and openness is valuable to learners. But how do we measure it? Should we? Doesn’t that kind of teaching style contribute to the success of the students?
Additionally, if rubrics are going to be used to educate faculty on how online learning works then these rubrics need to measure the openness of a course: does the instructor use OER or open textbooks? Accessibility to a course should also account for economically accessible.
QM Rubric as a Facilitation Tool
Overall, the rubric is worth using as a starting point: the opportunities for faculty to work together on courses and programs is invaluable. It is an useful tool to introduce face-to-face faculty to online learning and to change courses that are one-way information dumps to something more engaging. Personally, I think this can happen without the cost and the tools institutions use should be open (Creative Commons license at least).
