Online Learning Efficacy Research Database

Oregon State University LogoOne of the challenges with working with faculty new to online teaching and learning is helping them to understand that learning modality (online or off) is not a factor in student success. Oregon State University is meeting this challenge with its Online Learning Efficacy Research Database:

“This Ecampus Research Unit project is a searchable resource of academic studies of education efficacy across modalities. Filter by discipline or journal to find research in your subject area of interest.”

Researchers can search by modality, peer-reviewed, research sample size, education range, date range, and by journal.

I have relied for years on the No Significant Difference database but they have not been updated in four years and they seem to be pretty well tied to their hard copy book which was published in 2001. NSD includes studies from 1928 to 2013, but the OLERD only includes studies from 1998 through the present. I think this is more relevant to online learning and the current questions being asked about it.

I would recommend this database for instructional designers and elearning folks working with faculty and administrators who might be new to online learning or have to bring research-based answers to their departments:

Skepticism for online education remains a significant barrier to engaging faculty in online teaching and learning initiatives. We repeatedly receive questions from faculty about whether online environments are equivalent to the face-to-face environment in terms of learning outcomes. (from the OLERD FAQ)

This database is meant to provide faculty, administrators of distance education programs and other relevant stakeholder groups a way to quickly and easily search the literature on the topic of online learning efficacy.

I will definitely be following and promoting this project.

Posted in education | Leave a comment

Impacts of OER in Community College

I appreciate Stephen Downes take on OER – that there should be “no significant difference” between classes taught with OER or commercial text as far as outcomes go. I got into education to see that happen. The impact that I have measured in the past is grades, completion rates, and student satisfaction. I completely respect and value his definition of impact, and it would take a very deep longitudinal study to track that. In the community colleges I have worked at, we have had some more immediate problems to solve. There are five areas where OER and open textbooks are critical to the success of community college students:

Most of our students are working. Money is a huge issue. It should not be this way but it is. It is barbaric that the richest society on the planet can’t take care of health care or education, but that is where we are. The state of Washington has one of the highest homeless populations among its students. There are 40,000 homeless students in Washington. And a study came out that said that across the U.S., 14% of the college students are homeless. How are students supposed to learn, let alone have a transformative experience when they are worried about the next meal or where they are going to sleep? Eliminating the cost of textbooks is an essential step in solving this problem.

Financial Aide 
For various reasons, some that the students might be able to control and some they can’t, financial aide is often late. This means that students who are on financial aide are often two weeks behind in the reading. This just adds to the stress of being in college. Yes, we can put books on reserve in the library, but it is not the same experience as having access to a text when work schedules and transportation allows.

Relevance and Corrections
Students are often paying for books, resources, and at least chapters that are redundant with other classes that they will not use. Also, if there are errors in the books, we often have to wait two years, or what ever the publication cycle is, to get corrected texts. With OERs, faculty and students can customize the texts and make corrections instantly.

Student-Authored OER
Even better than making corrections, I have had the honor of working with a class where the students created the text. The assignments in the Communications class were designed to update the text and keep it relevant for the next class. The students had real ownership of the learning and often were available to help the next iteration of students work on the text (which was a wiki).

Student Satisfaction
Each semester, I would survey the students to have them talk about their experiences with the courses and learning materials. The open text book classes always had a greater student satisfaction because the students had immediate access to the materials (sometimes before they even enrolled in the course), the materials were always relevant, and they appreciated the lower cost.

Often the research around OERs is performed at colleges where the students do not have the financial concerns, are highly motivated, and are not under the stress that community college students find themselves. I would find it very easy to believe that for students at MIT or Stanford, there would be no significant difference, but for students in the community colleges, it is all the difference in the world.

Posted in OER | Tagged , , , , , | Leave a comment

Connectivism and Open Education

Ramon Llull's Tree of Science from L'arbre de ciènciaVahid Masrour sent out a link to our Open Education MOOC this morning to Mark Weller’s “Openness and Education: a beginner’s guide” and it got me to thinking a lot about Connectivism. Connectivism is the learning theory founded and championed by George Siemens and Stephen Downes. There are a number of components of that theory that I think are important to the discussion of open education. I list some of Siemens’ principles of Connectivism below (my comments in italics):

  • Learning and knowledge rest in diversity of opinions.
    This should be the very spirit of openness – seeking the diversity of opinion – Alexander von Humboldt once wrote that “truth rests in a diversity of opinions.”
  • Learning is a process of connecting specialized nodes or information sources.
    This is something that we must do – learning is an activity – not a passive reception of information – or the mere existence of information. Possession of a textbook, OER or not does not mean I will learn from it. 
  • Learning may reside in non-human appliances.
  • Capacity to know more is more critical than what is currently known.
    This is particularly important as we face unprecedented information over-load, a crisis in information management, and a deficit of critical thinking. 
  • Nurturing and maintaining connections is needed to facilitate continual learning.
    Notice that again, this is something that the student does. It might be modeled by the instructor, but the principle demonstrates great faith in student agency. 
  • Ability to see connections between fields, ideas, and concepts is a core skill.
  • Currency (accurate, up-to-date knowledge) is the intent of all connectivist learning activities.
    I see connections here to the benefits of OER – with OER we are able to crowd-source the creation and updating of knowledge without having to wait for the two-year publication cycle for corrections to textbooks. 

I am one of those who come to open education through the humanities. I still see it through that lens. I don’t see it through the technology lens even though George Siemens once called it “a learning theory for the digital age.” I think it is that, but I think what makes it a viable theory and a useful tool is that it answers questions and solves problems that go far beyond the “digital age.” I think that the conversations we are having around Open Education are important. For myself, Connectivism is the pedagogy of Open Education – it has all the requirements for a theory that not only addresses how learning occurs, but it accurately describes what happens in an open learning environment (facilitation, student agency, creating teaching and learning community, etc.). I don’t agree (or maybe understand) every part of this theory. I never liked “learning may reside in non-human appliances” when the very definition of “learning” requires a person to do the learning, but never mind that for now.

One of the things that Education Theory is really bad at is accounting for how people learned, taught, or organized information in the past. When read books about how people are supposed to learn, I always think back to Chaucer’s time and wonder how he became so educated in a day when pedagogues literally beat information into students. Despite the educational theories of the day, Chaucer managed to learn.

I read some criticism somewhere of Connectivism once that made the ludicrous claim that unless Connectivism could account for every sphere of human life, it couldn’t account for any of it. This is a standard that no theory is subjected to – certainly not a scientific one. No scientific theory would be foolish enough to make that claim. Theories are informed by facts, hypotheses, and experiment – not by claims to have answered the secret to Life, the Universe, and Everything. As soon as you have a brilliant mind like Whitehead come out with the Principia Mathematica who proves once and for all that there is an ultimate system of math, you have an equally brilliant mind like Gödel come along in the next generation and prove that any “mathematical system” has to be either incomplete or inconsistent. And despite this, my High School still insisted that I learn the stuff.

While working in the K-12 schools and studying Education for my masters degree, Constructivism was the main theory du jour. In a nut shell, Constructivism says that learning is socially constructed and that folks learn by applying past experience to new knowledge. I still have a lot of respect for it as it addressed many issues in education I was encountering at the time. It also helped shape early online learning experiences. But like most theories, there are some limitations. There are forms of learning and experience that learning theories can account for, but they usually fall short with the mechanisms or tools of learning. The invention of writing, universal literacy, and cheap paper were huge technical innovations that spread information and learning. Unfortunately, Stanford had not been invented yet so there was no one around to finally codify, once and for all, how learning took place in the human mind back then. There has to be a huge connection between the technologies of information and how we communicate as a species, but I have not found a learning theory that accounts for that. But that understanding, making those connections, is one of the things that the Humanities does really well. Lets take one tool or mechanism for learning; something as simple as concept maps.

Concept maps were an important part of my work as a developmental education instructor and are important in my work as an adult basic education instructor today. My students use them as a way to start their drafts of papers, to make connections between ideas and situations we are studying (history of Afghanistan, for instance), and I also use them as a way for students to build presentations. The concept map was also extensively used in the course “Connectivism, and Connective Knowledge” taught by George Siemens and Stephen Downes. They were used there to demonstrate knowledge of connectivism as both a formative and summative assessment. So are concept maps new? Are they “the digital age”? Of course not, artistic representations of branching and connected information are a human archetype – it is hard-wired into how we think. I used to give a presentation on concept maps that would annoy the hell out of traditional education researchers because the only evidence I gave was artistic evidence and proximity data (if two things or words are close together in context, they must share a similar meaning). The presentation began with a series of images to give the workshop participants the sense of the breadth and scope of the use of the image of the visual branching of information.

The point of the presentation is that through all time and across all cultures, there has been some kind of external, connected expression of what we know. Manuel Lima in his “The Book of Trees” does a great job of expressing this.

My point with all of this is that Connectivism is a viable learning theory because it addresses not only the Digital Age but the past as well. I think it fits in with the spirit and goals of Open Education and should be looked at seriously by any one interested in Open Education.

Posted in connectivism | Tagged , , , | 1 Comment

Thoughts on Open Pedagogy v. OER-Enabled Pedagogy

I think I understand David Wiley’s frustration with the semantics around “Open Education” – it is a very broad term. “Open” means many things to many people as does “education.” I am not sure if “OER-enabled Pedagogy” really addresses the issue. It sounds too much like content is what guides the teaching and learning. What is the opposite of OER-enabled Pedagogy? Copyright-enabled Pedagogy? Copyright-disabled Pedagogy? Commercially-enabled?

I understand what Open Pedagogy is – I know mostly because I suffered for 16 years under its opposite. Education is closed in the sense that it is expensive, classist, inaccessible, and strangled by corporations who define the curriculum with their “great” but expensive textbooks (or technologies). The closed classroom is hierarchical and led by an “expert” whose job is to transmit information to the empty vessels. There are social justice issues here, but that does not muddy the definition any more than the closure in traditional education muddies the definition of “traditional education.”

I am teaching a course right now that uses some materials from an open textbook but most of the materials are links to newspapers and journals. We are not interested, as a class, in textbooks per se because we are looking at social studies from what is happening in the world today at this moment. We are looking at the Trump policy in Afghanistan right now. The course is student-driven, networked, and dependent on student sharing (blogs and presentations). In my practice as a teacher, this is an open class, an open pedagogy – it is not particularly dependent on the 5 R’s to succeed as a class, but it is open. The students learn a way of using information, communicating, and building learning networks.

One of the things I do with the students is getting them away from the idea of debate: that only leaves us with right and wrong, winners and losers. It shuts down the discussion. We are working on deliberation: we survey ourselves on our values as individuals and as a group and try to evaluate the issues from a shared values/different values perspective. I think that “OER-enabled Pedagogy” is not “wrong” but it represents a different set of values than “Open Pedagogy.” Wittgenstein used to say that philosophy is not a theory but an activity, and like David, I am interested in what OER enables us to do but I am not sure it is time to stop the defining process.

Posted in Open education | Tagged , , , , , , | Leave a comment

Questions about Creative Common’s licenses

This has certainly been an issue for some faculty. I think part of the issue is trust. Open education is more than just the content – once it all becomes just about the “OER” then we lose that sense of trust and the spirit of collegiality. We have reduced faculty work and possibly the teaching and learning itself into a transaction. This faculty member I am talking about is sharing her materials to help students and faculty and is not interested in enabling corporations to monetize, however indirectly, her work.

I don’t see an issue with non-commercial licenses, but I am interested in hearing why “CC-by” is such the gold standard 🙂

Posted in OER | Tagged , , , , | 1 Comment

Promoting Open Textbooks

This is a presentation that I have used for the last ten years to promote open textbooks and OER at various campuses I have worked at. It is in desperate need of updating but I think it is a good model and one that I am going to pursue at Green River College as well. The data in this presentation starting at slide 40 comes from a survey I did of students in the Biology dept at Humboldt State University – it provided a striking profile of our students.

My typical strategy is as follows:

  1. Connect with faculty – survey them and find out who is doing what.
  2. Go to dept meetings and discuss the issues with faculty.
  3. Talk to students in student government and the newspaper.
  4. Find a faculty member in a high stakes, high cost course – such as Biology 100 and pilot.
  5. Survey those students afterwards and publish the results.

In addition to this, what I learned at Humboldt State University is to NOT FORGET THE LIBRARY. The library is an invaluable ally (given the right library dean). I had the good fortune of working with the Library Dean, Cyril Oberlander, at HSU (formerly of CUNY). He is a tireless and transformational advocate for open textbooks and open practices. A transformational advocate is not just alliterative hyperbole: he has had the librarians creating online OER research guides for faculty in all disciplines.


Posted in opentextbooks | Tagged , , | Leave a comment

North West eLearning Presentation

This is the presentation I am doing for the North West eLearning Conference (Thurs. 4:00 PM, Olympia Room). Great conference so far – as usual! All resources are linked. Email me at if you have any questions or would like to continue the conversation.

This presentation was based on a series of postings on quality assurance rubrics for online classes on this blog.

Posted in assessment | Tagged , , , , | Leave a comment

How to Close a Commons with an Open License

There are some interesting paradoxes that arise from alternatives to copyright. I have seen a big push to license articles and textbooks with a CC-by license (free to use if you attribute the author) and to avoid the CC-by-nc (free to use with attribution and for non-commercial purposes). The licenses available at Creative Commons have their use. They are all equally important because each one addresses a particular purpose. I think that there are times to use CC-by and a time to use CC-by-nc. But there are still problems with just CC-by, even though I am a big fan of that and CC-by-sa (Share alike).

The motivation to work with a textbook should be that it is the best one available with an open license, not that it is the one textbook you will be free to one day make a profit from. I would like to see the proliferation of good work because it is good work: not the proliferation of a work because it is licensed in such a way that a private corporation can profit from it. There are too many companies locking down content in ways that make them less accessible to the public.

One company started out by promoting open textbooks and the Creative Commons. Next, they created and charged for a platform to deliver that content. Then they needed to expand their business and make a profit for their shareholders. That is when the problems came in – “sustainabilty.” The open textbooks were Creative Commons licensed as CC-by (you only have to attribute the work to the author). In other words, they could not profit from those books unless they pulled that license and that is exactly what they did. Fortunately, managed to pull down and re-host the textbooks with the truly open license and that other company no longer calls itself an open textbook platform.

I voiced concerns about this when I was on an OER board in California about 7 or 8 years ago, and what I found was that it was very difficult to talk to members of the OER community about the problems with profit – I came across as “anti-business” and was actually called a “Socialist” <gasp!> in a public meeting. I am not anti-business or even anti-profit – but I am vehemently for open access to information and, unfortunately, money can be used to guide, promote and eventually kill a work. This is what happens to commercial textbooks! Have we not learned anything about the limitations of the commercial market?

I understand the importance of the different licenses but it is ironic that the most open license can lead to the shutting down of public access to supposedly open work. David Bollier wrote in his posting, “The Commons, Short and Sweet” that copyright “…privatizes and commodifies resources that belong to a community or to everyone, and dismantles a commons-based culture (egalitarian co-production and co-governance) with a market order (money-based producer/consumer relationships and hierarchies).” And yet this is exactly what can happen with the CC-by license.

There are arguments against the NC license that warn that if I use that license, corporations won’t be able to put my work on a DVD and distribute it with their magazine. Well so what? Maybe I don’t want my work distributed with their magazine. The irony of the argument is that my digital work (which I create for free) will not be available to prop up a dying industry! I can’t think of one magazine that bundles DVDs with their product that wouldn’t sue me if I just started posting their articles and materials on my blog. My argument is not against the magazines or capitalism, but it is against locking my content down in their business model.

Michael W. Carrol makes the point that “Granting readers full reuse rights unleashes the full range of human creativity to translate, combine, analyze, adapt, and preserve the scientific record, whereas traditional copyright arrangements in scientific publishing increasingly are inhibiting scholarly communication.” But the problem with that is that scientific publications are so expensive right now that many libraries can’t afford to subscribe to them. How does giving the work to a commercial corporation for free solve that problem?

Aaron Wolf has an argument against  the non commercial license that goes “I doubt a network TV show, for example, would use a song licensed as CC BY-SA (other than by paying the creator under a regular copyright license), because they wouldn’t want to license the whole show under Creative Commons.” Who says that they would have to license the whole show that way? Where has that been litigated? Are there no other interpretations of the license? Who gets to decide?

Saying that something is not truly open because it is not open to economic exploitation is ludicrous in all the Latinate meanings of the word. If we value education, we need to encourage our governments to support and invest OER in the same way that they support the military or industry. Openness includes access – if we focus our resources on working with corporations for our OER, then we risk seeing the materials that are not as “sustainable” (that is, profitable) withering away. We also risk having the openly licensed materials getting locked up behind “platforms” and “services.” And what happens when those corporations go away? Or get a new board not as hip to openness as the current one? I appreciate the efforts at the California State University system and the community colleges in creating and promoting OER and Open Textbooks but I would like to see more of a national/international effort that values access over profit.

Posted in OER | Tagged , , , , | Leave a comment

Wrapping Up The Rubrics

I started this series with a general overview of rubrics in general with the posting  Evaluating Online Courses: a prelude. I learned a lot reviewing the seven rubrics below for online courses:

  1. CA Community College’s Online Education Initiative Rubric
  2. Quality Matters
  3. Illinois Online Network’s Quality Online Course Initiative
  4. Online Course Evaluation Project
  5. CMU’s Quality Assurance Checklist
  6. OCAT and Peer Assessment Process
  7. CSU’s Quality Online Learning & Teaching Rubric

Looking at these rubrics was a nice walk through the recent history of online learning. Also, each rubric or checklist tells us how each institution defines online learning. I still have a lot of questions that I will need to answer like:

  • How do we define an online course?
  • How does that definition inform the review process?
  • Who should review or assess an online course? In house or out?

These are all questions that institutions should ask as they formulate an implementation process. It is a good time to look at strengths and challenges of existing courses and programs. It is also an opportunity to see what resources and challenges such a process would require or bring.

There is a temptation when creating online course rubrics to attempt to assess everything related to the online class, including things like ADA 508 compliance that should be referred to experts. But this also points out an opportunity: a rubric can be used to help faculty inventory the services and resources on a campus that are there to support teaching and learning.

Photo of hands at the keyboard.

The process should be updated every year or two because of the rapid changes in technology. Concerns that were prevalent ten years ago may not be a concern today. Also, just like choosing (or not choosing) a learning management system or a content management system, the process should reflect the needs of the whole community. If you have a population of students with special needs (i.e. developmental students or first generation college goers), then access to the resources to insure student success should be assessed in online courses.

Why Open Source?
Each institution or community is unique enough, has its own population of faculty, students, and staff; its own access to resources (as rich or limited as that may be); its own challenges to warrant taking the time to work out a process that addresses those needs. This is why it is so important that this work be open source – artifacts created by this process should be openly-licensed. Copyrighting a process is like saying that all institutions are alike and have the same academic and student culture. For a while, all of the Quality Matters training material, research, and copies of the rubric were either not available or harder to find after they chose the copywritten path after fiscal year 2007 marked the end of their grant. This closes door on future development for the sake of “sustainability.” I appreciate the open license on the CSU’s QOLT rubric because if I make changes to solve local problems, then the CSU can get a copy of it and when they encounter similar situations, we will have already done the hard work for them.

A Meta-Rubric Example
Much like the rubrics used to evaluate software – I have been thinking about a meta-rubric for reviewing online course assessment rubrics. My example includes criteria that are common in the education technology world: ease of use, cost, and validity of scoring. And then I have included criteria that would be useful for a few of the community colleges that I have worked for, as well as some of my own values: openness and creating community. My example rubric is meant to start a conversation about what is important for your institution.

This is my “meta-rubric” example. Please feel free to copy, share, or comment.


Posted in assessment | Tagged , , , , , , | Leave a comment

Why Open Matters

Picture of an open door.For myself, “open” covers open pedagogy, open education resources, and open textbooks. I think open education is one of the most significant things to hit education in 500 years – probably since the invention or the wide-spread use of cheap paper. Open education is all about increasing the access to education and overcoming the barriers of geography, culture, and economics.

I currently teach Adult Basic Education. Since the 90s, I have been involved in teaching and education support (tutoring, writing labs, etc.) and have taught developmental English.  Open education or open pedagogy allows me to facilitate my students learning in such a way that they become independent learners. The syllabus becomes a negotiated contract, and the students from the very beginning of class, are brought into the process of mapping out where they are going to go though out the quarter. ABE students are taking courses for a number of reasons: some need to pass the GED, some are looking for HS or college credit to move on to a 2 year degree, some take classes as a step in finding their path. An open pedagogy allows me to help students understand that they have choices and that they are in charge of their learning. They are not just in a “class” but in a network of learners, creators, and professionals who will help them learn far beyond what we are doing in the physical building. Many of these students come from backgrounds and institutions where their own agency has not been respected or fostered. An open pedagogy allows us to teach students how to learn – how to make the critical connections that will allow them to be successful in college, their personal and professional lives.

Open education resources (OER) include open textbooks. ABE students are often the most vulnerable of the student population. Using peer-reviewed, open education resources or open textbooks allows me to lower the cost of education for students who are often at the mercy of their financial aide situation or lack there of. Open textbooks allow me to reuse and remix materials according to the needs of the students rather than hoping that we have found a textbook will actually work for them. I also am using open tools like WordPress.

The Pacific Northwest is growing fast. There are a 1000 people a month moving to Seattle each month. This growth is increasing rents and the cost of living, and so I have a moral obligation to use an open textbook (high quality, peer-reviewed that meets the course outcomes).

For a few years I was not teaching but working in the admin side of education as an instructional designer and director of academic technology. My previous teaching (and instructional design) was heavily influenced by George Siemens and Stephen Downes’ MOOC, Connectivism and Connective Knowledge, and Cormier’s Rhizomatic Education: Curriculum as Community.

Posted in education | Tagged , , | 2 Comments