All Hail MOOCs! Just Don’t Ask if They Actually Work

Despite booming enrollment and enthusiastic administrators, scant research offers little evidence that online courses are effective

  • Share
  • Read Later
Matt Rourke / AP

University of Pennsylvania professor Peter Struck, accompanied by teaching assistant Cat Gillespie, teaches a mythology class during a live recording of a massive, open, online classes (MOOC), in Philadelphia, May 30, 2013.

Dozens of top universities and colleges are scrambling to get in on the latest trend in higher education, massive open online courses known as MOOCs. Enrollment is ballooning by the hundreds of thousands each semester. A third of administrators say they think residential campuses will eventually be obsolete. Google just announced it’s teaming up with Harvard and MIT to create “a YouTube for MOOCs.” And The Economist asked this summer if the courses portend “the fall of the ivory tower.”

There’s only one hitch: No one really knows if students learn anything in a MOOC. Scant existing research suggests that the success rate of online education, in general, is poor. And even the people behind MOOCs are becoming concerned about sky-high expectations, which they say represent a misunderstanding of their purpose.

“At this point, there’s just no way to really know whether they’re effective or not,” said Shanna Jaggars, assistant director of the Community College Research Center at Columbia University’s Teachers College, which has produced some of the most recent scholarship about online education.

(MORE:  The Hottest Seats in Class)

“Everyone in the research field agrees that, for the particular purpose of replacing on-campus education, the evidence [for MOOCs] is ambiguous at best,” said Andrew Ho, a professor at the Harvard Graduate School of Education and research director for HarvardX. “Far more research is needed. And we’re conducting some of it. But we’re way out over our skis when it comes to that particular purpose of MOOCs.”

Enrollment in online college courses of all kinds increased by 29 percent to more than 6.7 million between 2010 and last year, the latest period for which the fast-changing figures are available, according to the Babson Survey Research Group. And this explosion is happening at a time when the number of students in conventional universities and colleges has started to decline.

MOOCs alone—as opposed to other kinds of online classes, including those with limited enrollment and for which tuition is charged—are growing so quickly, it’s impossible to know how many people take them. Barely a year and a half after its debut, Coursera, a startup launched by Stanford faculty, reports that about 4.4 million students have signed up. The MIT-Harvard MOOC collaboration called edX, which premiered just four months later, boasts more than a million.

But those numbers probably don’t augur a new wave of learning. About 90 percent of people who register for MOOCs fail to complete them, most providers acknowledge. Advocates say that’s because there are no admissions requirements and the courses are free; they compare it to borrowing a book from the library and browsing it casually or returning it unread.

(MORE: MOOC Brigade: What I Learned From Learning Online)

 A survey of students by the market-research company Qualtrics and the education technology provider Instructure seems to confirm that trend. Seventy-five percent said the main reason they signed up for a MOOC was that it didn’t cost them anything, while 29 percent of those who dropped out said they got too busy to continue, and 20 percent said they lost interest.

Two-thirds of those students said they would be more likely to complete a MOOC if they could get college credit or a certificate of completion for it, something that’s still not widely available. Until it is, said Jaggars, it will be hard to measure the effectiveness of MOOCs—a Catch-22, since without knowing their effectiveness, it’s unlikely colleges will give academic credit for them.

To study what happens when students get credit for online courses, Teachers College looked at online courses at community colleges in Virginia and Washington State that were not MOOCs—since tuition was charged and credit given—but were like them in other ways. The results were not encouraging. Thirty-two percent of the students in online courses in Virginia quit before finishing, compared with 19 percent of classmates in conventional classrooms. The equivalent numbers in Washington State were 18 percent versus 10 percent. Online students were also less likely to get at least a C, less likely to return for the subsequent semester, and ultimately less likely to graduate.

In July, San Jose State University suspended its experiment with offering MOOCs for credit after only half of credit-seeking students who took the online courses passed, compared to three-quarters of those who took the traditional versions. In one of the three pilot classes, which were offered during the spring, fewer than 30 percent of the online students passed. And while the university and its partners hailed an apparently dramatic improvement in results in the summer semester, a closer look showed that more than half of the summer students already had at least a bachelor’s degree, compared to none of the students who took online courses in the spring—and even then, more of the summer registrants dropped out.

(MORE: MOOC Brigade: Can Online Courses Keep Students from Cheating?)

While their scale is unprecedented, the underlying way that MOOCs are taught isn’t really new, Jaggars and others said. It’s very, very old—a system in which professors lecture to huge numbers of students with whom they seldom, if ever, interact. “In general, students don’t do as well in online courses as they do in conventional courses,” said Jaggars. “A lot of that has to do with the engagement. There’s just less of it in online courses.”

None of the evidence has slowed the MOOC craze. Seventy-seven percent of academic leaders already think that online education is as good or better than face-to-face classes, and 69 percent say that it’s essential to their long-term strategy, the Babson group found (though the administrators also conceded that only 30 percent of their faculty agreed). Four in 10 said their schools plan to offer MOOCs within three years, according to a survey released in the spring by the IT company Enterasys. And legislators in several states are pushing to speed up the shift to MOOCs for college credit, which they see as a way to expand access to higher education while reducing costs.

John McCardell, vice chancellor of Sewanee: The University of the South, is one of the skeptics. He points out that the American Council on Education has recommended only 10 MOOCs for credit, and even those recommendations are merely advisory. Quoting poetry he said he studied in an old-fashioned brick-and-mortar classroom years ago, McCardell invoked Alexander Pope’s advice:  “Be not the first by whom the new are tried, nor yet the last to lay the old aside.”

“That might be a useful thought to keep in mind,” McCardell said, “as the world seems to be rushing headlong to embrace this latest pedagogical fad.”

This story was produced by The Hechinger Report, a nonprofit, nonpartisan education-news outlet based at Teachers College, Columbia University.

SEE ALSO:  The Big Surprise of Martin Luther King’s Speech 
 
17 comments
MaryFloyd
MaryFloyd

Between the Sewanee: University of the South guy in the article misquoting the poem he failed to learn in an old-fashioned schoolroom (should be "by whom the new IS tried") and the commenter who thinks "learner" can be either singular or plural, I'm almost afraid here. Still, I'm a Ph.D. student with an A in graduate statistics and I couldn't keep up with Stanford's freshman MOOC in thinking like a mathematician. I'm repeating it this semester, trying to pay more attention and relying on Wikipedia to explain mathematical terms that are unfamiliar to me.




mrbomb13
mrbomb13

With articles like this, over time, TIME Magazine will increasingly alienate itself from its younger demographic (ages 18-35).

To qualify my comments, I have 1) completed an online MBA program (in addition to a classroom one) from a major university, and 2) now teach in that same online program at the same university.  With both experiences under my belt, I can assure readers that the author (Jon Marcus) and The Hechinger Report have no idea what they are talking about whatsoever.

As a student in the online program, I was able to pace my learning, and effectively complete the curriculum, communicate with other students and the instructor, and still come away learned and ready to apply that knowledge.  In fact, it was because of my experience in such a program that I landed my current job as a Senior Analyst at a major Fortune 500 company.

As both an analyst and an online instructor in the program, I can now use my knowledge and the available tools to enhance the material for students.  As instructors, we are held to account for our content and pedagogy (teaching methods).  As a result, we receive overwhelmingly positive feedback from students and the university administration. 

From this, I can promise readers that online courses are not going away anytime soon.  They will only increase in popularity for being 1) more cost-effective (no classroom/commuting involved), 2) more convenient, 3) more scalable to the individual student, and 4) more adaptable to current technologies (as opposed to an 80-year old podium lecturer).  Online education will play an increasingly major part of higher education as time progresses, and studies like the stand-alone Hechinger Report will be drowned out by the over-whelming positive reviews of said courses.

room5liveson
room5liveson

No one really knows if anyone is learning anything in traditional classrooms either. I have been teaching English to middle- and high-school students for ten years in Los Angeles. Accurate assessments, standardized tests that have severe biases, over-reliance on "snapshot" assessing -- it's all quite complicated, even for the most advanced teachers. And then try addressing retention. If I teach an intensive course on the history of apples for a month, I can be sure to show data proving my students improved their knowledge of apples. But check back in another month or a year. MOOCs are just another delivery system that relies on all the factors that impact a traditional classroom: motivation level, teacher expertise, etc.

Beckermann
Beckermann

The issue with 'success rates', often chalked up here to the completion rate metric is that we are quantifying something in a very rigid and circumscribed manner without due consideration for the education's effect or impact. Coursera and other MOOCs have average completion rates of c. 10% or slightly higher and that's with the top 1% of motivated students, many of whom already have degrees according to the study published in Inside Higher Ed earlier this year. Massive online courses that serve vocational offerings have an even bigger challenge with diverse learning communities facing more motivational and pragmatic challenges to learn in underprivileged countries and locales without requisite infrastructure, unlike the USA.


Yet, some of those offering free online courses seek more than just a taster to the already privileged, they seek to educate the 99% and to do that on the massive global scale; a company in Ireland called Alison.com comes to mind. They recently reported completion rates nigh on 20% and even higher in some courses, yet their user base is spread globally from the US to UK and a large cohort in India, Pakistan and West and South Africa. They seem to have achieved this by so-called 'self paced' learning. The learner can engage with the course material precisely when it suits them to do so, particularly useful for professional people or those with distinct commitments whose time is limited. This flexibility for the learner gives greater incentive to persevere knowing that he/she can control their time more adaptively. The real result here then is greater completion rates but also greater impact that goes far beyond the percentage points. The education of these people has much greater traction and personal benefit for the recipients than casual browsers of MOOCs who already have a Master's for example.


rebeccapistiner
rebeccapistiner

I have a post-graduate degree. And I have taken several courses on Coursera and completed them. I don't get credit from the University. I get a certificate of completion. I don't get anything that anyone else might measure as important. But, I will keep taking these courses, as long as I keep learning. I hope they continue. I have expanded my own library of knowledge without spending a dime. And isn't that what they are for?  It sure is a better way to spend time online than FaceBook or You Tube. 


linusjf
linusjf

MOOCs have to be treated much as classroom courses. What you get out of it, is what you put in. 

You don't get to be an expert by skimming....

HenryFaley
HenryFaley

A couple of points.  I took a Statistics course offered by  Princeton.  A lot of people dropped out.  Clearly, it was very difficult and required a lot of work to complete it.  I received no certificates or credits, but the bottom line is that for no cost I learned a lot about the subject.  By the way, I took this course in a regular University many years ago, this course was much more difficult..

formerlyjames
formerlyjames

So many people don't complete the program...so what?  An analogy is made with libraries.  Should libraries be eliminated based on quality of use?  This seems to me to be a false concern. 

Lkirk
Lkirk

You should take a course.  About the 90%...  look at it as a measure of demand/hunger.  Look at success more broadly too -- knowledge/advantage gained from a single lecture/module or a highly motivated high school student looking for deeper understanding and mastery or a worker improving her productivity.  In the end his may be the "Massive" part of this that has the greater value.  And who cares if someone isn't qualified or ready yet.  That will sort itself out over time...


RoccoJohnson
RoccoJohnson

I think that a lot of the dropout issue is that people are allowed to enroll without the qualifications to get in. Also, when students have vested no financial resources to get in, they have nothing to lose by dropping out. 

You could compare these dropout statistics to those of the number of Americans who start diets only never to finish. People want the benefits of an online education without having to do the work necessary to get that education—just like diets.

NaveedXVO
NaveedXVO

Colleges won't want to give credit for them. They could already, obviously, easily. Just provide a proctored testing system (charge for it), that's essentially most of what universities do now.  But then how would they accomplish their dream to extract tens of thousands of dollars from every human in the western world?

AlexMcHardy
AlexMcHardy

@mrbomb13 I believe you missed the aim of this article. The author was simply arguing that there is scant knowledge regarding the effectiveness of these MOOCs when it comes to student learning. The popularity is not in question, especially because of the points you made above. Unfortunately, inexpensive, convenient, more scalable, and technologically current education does not equate to quality. 

linusjf
linusjf

You should have a clear purpose for doing a MOOC.

If the course or a series of courses meets that purpose, go for it!1

NaveedXVO
NaveedXVO

Think of all the jobs that would be destroyed? Those people won't go down without a fight.

mrbomb13
mrbomb13

@AlexMcHardy @mrbomb13 

First, thank you for your reply.  I recognized the author's main point about the lack of knowledge, but do not agree with that assertion based upon  reasonable inference.

If the quality you mentioned was absent, I can guarantee that free market economics would have driven the modern MOOCs out of business soon after their debut over 10 years ago.  Students would have soon realized that a computer screen was a clearly inferior substitute for the interactions/dynamics of a live classroom.  With that kind of bad feedback, it would have been 'back to the drawing board' for the new MOOCs of that time.

However, that has not proved to be the case.  Younger generations of students find such MOOCs to be more effective for them, because they have grown up with the internet, and find the self-pacing of an online course preferable to a live lecture that may run too fast/slow for them.  Instead of feeling rushed/unchallenged in a live classroom, they can absorb the material at their own pace on their own time.  The online setting respects their time, and empowers them to 1) guide their own learning, and 2) force them to be responsible for that learning.  Those are not necessarily detrimental to their learning. 

With the option of real-time classrooms still existing, MOOCs can serve as a separate, but equal option to such interested students.

JamesFrancis
JamesFrancis

@mrbomb13 @AlexMcHardy Working out at home is possible too but its not easy to do or keep up.


Sometimes you have to join a gym to stay motivated and in the mindset.  And I suspect the same happens with school.  Not to mention there can be hard-to-replicate human qualities tied to physical being in a setting with other people that motivates learning. 


 That being said there could be two different groups here too.  The ones that can go to the classroom could be a different demographic then those who have to/opt-to do online courses.   The ones who go online could be 'busier' and hence more likely to drop out but they would never have done the inclass options.    


How much one cannibalizes the other would be interesting to know.


Aside: gyms make a lot of their money on the people who have memberships but dont show up.  Schools might not see low attendance/completion as a bad thing once they start charging to take the course. 

AlexMcHardy
AlexMcHardy

I respect your perspective on this issue because of your first hand accounts. Unfortunately, anecdotal evidence and "reasonable inference" provide no real knowledge to the question of whether or not these courses or this mode of education is effective. Being popular with students in no way translates to MOOCs being effective in a measured sense. We must be careful before we declare them a useful tool in education.

Follow

Get every new post delivered to your Inbox.

Join 2,105 other followers