I have been commenting on “online homework systems” and testing in general as assessment via blog postings and Twitter from David Wiley and Clint Lalonde. Obviously, it is difficult to tease out any kind of real argument on Twitter, so I thought I would put down something here about my ideas around OER and assessment.
I will say up front that I am not a huge fan of testing. It can be a one dimensional assessment. Tests often measure what someone currently remembers, not whether they know how to use the new knowledge or information. One might argue that tests are practice except that the best practice is to apply the information in the context (i.e. a project that uses math vs. a quiz). With that said, my argument is not against tests per se: I have seen brilliantly scaffolded tests that require a focused application of critical thinking to work through the series of questions. I have seen tests that once you worked through the questions, you came away with a deeper understanding of what was being taught. And then I have seen what the online publishers produce. I have seen it all from publishers: wrong answers, questions not tied to any readings or lessons, misinformation, spelling and grammar issues that make the question unclear, and lots of rote memorization. Although it is not the only way to ensure higher order thinking skills, I can smell a test a mile away that was written without Bloom’s Taxonomy at least somewhere in the background. I also have had to bring tests with just plain old wrong information to the instructors who were surprised by the problem because they did not have time to review every question in the test bank for accuracy: right or wrong, they counted on the publisher for that.
I think from a teaching and learning perspective, from an instructional design perspective, there are better ways to assess what someone has learned than questions from a test bank. There are even better ways to practice. I am concerned about teachers who rely on tests as a sole measure of what their students have learned. It is an easy way to “teach,” we should be concerned. Promoting open textbooks along with test banks is not particularly a problem in itself, but what it does do is encourage the kind of thinking that conflates a textbook, “ancillary” materials, or tests, with a class. Instructors often use testing and test banks because they are “easier.” They are locked into that method of teaching because that is how they were instructed and that is what their departments expect.
There are alternatives, but they are hard to lock-down behind an LMS and hard to monetize. I read on Clint Lalonde’s blog about a report from Open Oregon called “Report on Lack of Homework Systems as a Barrier to the Adoption of Open Education Resources.” I understand this: if your project is to expand open textbook adoption, then creating a homework system might be a priority to do that. The other approach is to help faculty understand the need for authentic assessment in an open context along with open textbooks. I am not assuming, for instance, that Clint’s project is “drill and kill” or that it would be the only form of assessment in his courses – I appreciate the thoughtfulness that he has brought to this over the course of the three blog postings of his that I read. What I would hope from those in the open education field is that they ask themselves questions like: “How does this practice expand access to education?” “How is this work ‘open’?” It is okay if the answer is “It doesn’t, it will be locked into an LMS and available at an extra cost.” Great. No judgement. You then you are in business and not, at least in that moment, in education. Schools purchase all kinds of services that are not open: food services, janitorial, and building maintenance, for instance. The next question we should ask is: “Is there an open practice that does this better or as well?” If knowledge should be free, open and accessible, and there are practices that are, according to Wiley’s perspective, only possible given the so-called “5Rs,” then how does a testing system, locked in an LMS, reflect that practice? If I lack imagination, it is only because I can’t imagine how it does. If I am wrong- great. Let’s keep expanding access to education for more students.
These aren’t radical or new ideas: teachers and institutions have been looking at different kinds of assessment for years. For instance, I had really hoped that portfolio-based assessment would have been more wide-spread. (That was twenty years ago now!) There are some great programs out there that use portfolio-based assessment – especially in teacher education. One of the problems with portfolios is that as soon as people started the practice, the private sector produced a product. The practice is useful, requires self-reflection on the part of assessment, and provides a meaningful artifact for the student, but the products around eportfoilos are generally terrible. You can get a grant or investment for a product, and it is nearly impossible to get a grant to explore a practice.
And there are alternatives. There are a LOT of alternatives. David Wiley wrote a paper that addresses some aspects of alternative OER-based assessment in “Defining OER-Enabled Pedagogy.” This article gives great alternative assessments. Again, in 2009, Wiley and Mott wrote an article called “Open for Learning: the CMS and the Open Learning Network” that says that the “…course management system (CMS) reinforces the status quo and hinders substantial teaching and learning innovation in higher education.” So apparently, that substantial teaching lies somewhere outside of the LMS.