Monday, January 14, 2013

What was it like to teach a MOOC?

Since I get asked this question a lot, including by the media, I thought it’d be useful to blog it.

What MOOCs have I taught?

I split our 15-week on-campus  into two pieces, an “introduction” and “advanced” MOOC.  The Intro MOOC was offered three times on Coursera and is now in its second offering on EdX, with whom UC Berkeley has an institutional partnership.  The Advanced MOOC has been offered just once so far on EdX.  (The course number is CS 169.1x for the Intro and Cs 169.2x for Advanced.)

What was it like to adapt the campus course?

It has been a lot of work, but it has greatly improved the on-campus course as well as making it available worldwide.

Because of high enrollment demand in the on-campus course, we had already started thinking about automatic grading of programming assignments.  We and our on-campus TAs spent hundreds of engineer-hours creating sophisticated automatic graders for programming assignments since we believed neither MOOC nor on-campus students would be well served by a software engineering course whose assessments were based solely on multiple choice or short-answer questions.

While this was grueling, the autograders are reusable and customizable, and have graded hundreds of thousands of assignments so far, which would require an army of TAs to do manually.  Thus it helped us expand access to the course on-campus as well as online.

I also reorganized my 90-minute lectures into 8-12 minute “lecturelets”, each covering a specific topic and accompanied by self-check questions.  Our colleagues, the founders of Coursera, suggested this format worked well for MOOCs, but we found that it also made the on-campus lectures livelier and better-attended.

We live-captured our lectures and postprocessed them afterward.  In general, we were unwilling to spend time on the MOOC that would not also improve the on-campus course.  My teaching ratings and the on-campus course’s ratings (given anonymously by the students) both went up when the MOOC technology was integrated.

What worked well?

The automatic grading allowed vastly more students to practice the material, and modulo some glitches, was widely praised.

The online students took the same quizzes and did the same programming assignments as the Berkeley students, on comparable deadlines, and the best of them did just as well.

Intelligence is uniformly distributed worldwide: students from 130 countries enrolled, and thanks to lecture subtitling provided by EdX, were able to follow the lectures despite not being native English speakers.

More important than the number of students reached was the smaller number (hundreds) of truly motivated students to whom we were able to give an opportunity otherwise unavailable to them, such as a student in the Gaza Strip who only received 6 hours of electricity each day and used part of it to complete our course.

What challenges did you encounter?

This is an enormous amount of work.  We did it on our own time and without extra compensation, but not all instructors will have that luxury.  We also had discretionary money we used to pay TAs to help support the online course; there will have to be more systematic sources for those funds eventually.

Online students don’t get face-to-face time with instructors and TAs, as on-campus students do.  The discussion forums work pretty well, but despite being monitored by TAs, they’re basically self-service.

The on-campus students get to do team projects in small groups working with external customers.  In their evaluations they indicated this was the most valuable aspect of the course.  Yet the online students don’t get to do this part, as we haven’t figured out how to scale up the process of forming project teams and matching them up with external customers.

Discussion-oriented learning doesn’t work well at large scale.  Some parts of our course dont’ require it, but for the parts that do, such as office hours and TA design reviews of team projects, there was no way to give the online students this experience.

Nonetheless, our goal was not to replicate every aspect of the on-campus course, but rather to identify which aspects, if converted to MOOC format, would work well for both online and on-campus students and focus on making those things great.

There is lots of cheating in MOOCs.  We have hard as well as anecdotal evidence from our courses and colleagues’ courses.  But since no credit is offered, we’ve chosen to focus on improving the course for the vast majority of honest students rather than try to catch the cheaters.

A very small fraction of MOOC students are vocal jerks or have a disproportionate sense of entitlement about everything being free.

Would you do it again?

Absolutely, I’m planning to.  And as I develop new on-campus courses, I’ll be blending in MOOC technology, such as shorter lectures with self-check questions and extensive autograding.  That allows the TAs and me to shift our scarce on-campus student contact time from a lower-value activity (repeating the same lectures) to a higher-value activity (face time with students who’ve watched lectures and/or done the prep work using MOOC resources)

Book summary: A Thread Across the Ocean

A Thread Across the Ocean: The Heroic Story of the Transatlantic Cable by John Steele Gordon The most wonderful thing about the writ...