Friday, December 20, 2013

A demoralizing semester for instructors

DISCLAIMER: I believe most students don't cheat, most students don't try to make life miserable for their instructors, and most students really do want to learn and take responsibility for their own education.

That said, the fraction of students to whom this doesn't apply—those who do cheat, or have a sense of entitlement about the grade they "deserve", and when something goes wrong (including getting caught cheating or not getting the grade they "deserve"), it's everyone's fault but their own—seems to have hit a high point this semester.

I've been teaching as a faculty member for 13 years and as a GSI or assistant instructor for several years before that during my MS and PhD. I've never felt as down on teaching as I do at the end of this semester.

Colleague Dave Patterson and I co-taught our this semester.  Modulo a few bumps, it generally went smoothly, although there is the inevitable flurry of grade complaints at the end—some based on a simple misunderstanding of grading policy, but distressingly, some displaying a truly stunning sense of entitlement and lack of self-accountability.  One student was upset that their grade was affected because other students in their project team complained that s/he was a slacker.  This student was convinced that in other teams, "buddies" colluded to give each other high grades, whereas in her/his team things were different, resulting in an unfair disadvantage.  (The data didn't show this by any means, but the student hadn't seen the data, so apparently s/he was free to make assumptions about it.)  Another was wondering why we didn't move them up to a higher grade given that they were near a borderline (e.g. A-/B+).  I asked on what grounds they thought we should do that, and the answer was essentially that they were near the borderline.  If we assume that fairness means applying the same policies and considerations to all students, which I believe we have scrupulously done, the transitive closure of a policy that accommodates this student's request would result in most people getting an A.  And while that may be the case at Harvard, Berkeley is at least trying to hold the line on grade inflation.

But on balance, we got off light: our colleagues Randy Katz and Anthony Joseph discovered possibly-widespread cheating in their courses, and a Reddit thread allegedly containing (pseudonymous) remarks from students in his class about the incident is disturbing in that students justify cheating by blaming it on someone else:

  • It's the GSIs' (teaching assistants') fault for not policing us when they know cheating is happening.

  • It's the instructor's fault for not changing the projects enough since last semester.

  • It's the instructor's fault for making this course so much more difficult and time-consuming than its prerequisites.

  • It's the instructor's fault for not making the boundary between "collaboration" and "cheating" crystal-clear.

  • It's the Department's fault for enforcing a high GPA threshold to stay in the major.

  • (Variation:) It's the Department's fault for creating circumstances that cause people to cheat.

Are we running out of other people to blame yet?  Hell no:

  • It's the Internet's fault because why do the project if you can search for the answers online in just a few minutes?  (To the student who made this comment: your parents will be delighted to know that their tuition money and your four years at college are being so well spent.)

  • (Variation:) It's the fault of students who took the class last semester for making their code publicly available online.

No one has yet blamed Obama, Fox News, or global warming, but I suspect it won't be long before those are all implicated.

(And for those who are truly unable to determine if they themselves are merely collaborating or cheating, here is a good method: Does it feel like you're cheating?  If yes, then you are.)

Of course, everyone makes bad judgments at some point in their lives, so in the interest of making it a "teachable moment", Randy opened a voluntary amnesty period during which those who came forward voluntarily and admitted to cheating would less severely penalized than those who did not.  Amazingly, even this was considered controversial, with one student even reporting that a TA from another course had advised him, in a prisoner's-dilemma sort of way, to stay quiet and deny everything if confronted.  That sounds like the advice a lawyer might have given to Al Capone, and I'd be distressed if it was really true that one of our own TA's said this.

Never mind the fact that cheating is known to eventually screw the cheaters; the simple truth is that cheating is a choice that shows a lapse of personal integrity and a lack of self-accountability. Lack of integrity brought us Michael Milken, Enron, Bernie Madoff, and the malfeasance related to securities analyst fraud and mortgage-backed securities fraud that brought down Lehman Brothers, laid low JP Morgan, and plunged the country into its worst economic recession since the 1930s.  So, to the students who cheat and then try to justify it, when you carry your lack of integrity into the business world and end up on the witness stand, please don't reveal that you graduated from Berkeley—just plead the Fifth and spare us the shame.

On the other hand, I applaud the anonymous student who made this comment on the Reddit thread, and who appears to get it:

My grades in those classes could've been raised by getting other people's code, but I knew that cheating would only hurt my understanding of the concepts the projects were trying to teach.  All these people who are cheating are the type of people I absolutely hate working with at my job. They always do the bare minimum and then hope that someone else covers their asses. Cheating in college only sets you up for a slippery slope that's hard to avoid when you get into the real world.

Incidentally, I've known Randy since I was a graduate student in 1994—he was on my dissertation committee and we even co-taught a freshman seminar together—and I know firsthand that his skill and dedication as an instructor are as laudable as his numerous teaching awards would suggest.  Among other things, he's received the highest teaching honor the University can bestow and the highest distinction as an educator that the Institute of Electrical and Electronic Engineers can bestow.  When one of the Department's best instructors feels burned out and unenthusiastic about teaching because some students are unable to commit to a code of ethics that basically boils down to "don't lie", something's really wrong.

As always, my opinions are my own and don't necessarily represent those of UC Berkeley or the EECS Department or anyone else.

Monday, December 2, 2013

Rethinking the format of a software engineering textbook

Why did Dave Patterson and I decide in April 2011 to write a textbook for our “SaaS-flavored” CS169?  Aren’t there a ton of software engineering textbooks out there already?  And aren’t there a ton of practitioner-targeted books for teaching Rails, Agile, Cucumber, etc.?

Among the textbooks, precious few focus on Agile, mostly treating it as a side topic, but we think Agile is a great fit for the classroom—1 or 2 week iterations doing complete “mini-lifecycles” of a part of a software system.  The ones that teach Agile use Java (presumably because so many universities are ), whose viscosity outweighs the “lightweight process” advantage Agile is supposed to confer.  Our view is that learning a new language and tools is worthwhile  if it furthers the educational goal of teaching how to build long-lasting maintainable software with maximum productivity.   (I’m one of the contrarians who is still conflicted about the elimination of the Scheme version of the Abelson & Sussman SICP course, since Scheme is arguably the best language for doing what SICP tries to do.)

Also, despite the fact that the #1 request from our industrial colleagues was “teach students how to work with  legacy code,” most software engineering textbooks we looked at barely mention this topic, with a couple of notable exceptions that focus on using existing open-source software as a teaching vehicle.  (For the record, I think that’s a great idea, but not for our course: while OSS affords opportunities for teaching about legacy and refactoring, it often lives in ecosystems whose testing environments are less than delightful to use, e.g. C or C++ code.)

There’s hundreds of practitioner books on Agile, Rails, RSpec, Cucumber, Git, refactoring in Ruby, design patterns in Ruby & Rails, ad nauseam—and that’s the problem.  We probably perused over 10,000 pages of text in those books in preparing the material for our book.  There is no single narrative that weaves the topics together in a way that makes sense to engineers new to SaaS+Agile+Rails, puts them in the context of software engineering history and best practices, and does it in an amount of text that is realistic for undergrads to consume during a 10-15 week course.

Once we decided to write the book, the next easy decision was to focus on ebooks.  Ebooks aren’t the future of textbooks, they’re the present.  So we knew from day one that we needed an authoring environment that would allow us to derive multiple ebook formats plus a dead-tree version from the same source files.  (I’ll probably release this environment as open source once it’s cleaned up; it’s LaTeX, tex4ht, and a bunch of Ruby scripts.)

But since today’s ebooks are generally inferior versions of their printed counterparts, we came up with a whole list of enhancements for what we thought an ebook could be, inspired in part by Al Gore’s Our Choice ebook for the Apple iPad.  Many (but not all) of the features we want are likely to be present in the upcoming iBooks edition of our textbook.  And as I described in another post, ebooks can be updated frequently and errata corrections pushed out to students practically on-demand, and delivering an ebook to an international student is a lot less daunting than delivering physical books.

Ebook aside, though, it’s worth describing a few of the features that even transfer over to the print version, kind of, because I think we’re learning how books of all formats can be better integrated with online materials.

  • Every code example in the textbook, and many additional ones that are part of the lectures or the homeworks, are up on Pastebin.  So even students who doggedly refuse to learn an editor with syntax highlighting can see what they’re missing, plus Pastebin provides 1-click copy-and-paste so they can try the code themselves.  We created automation to keep the Pastebin links in sync with the URLs that appear in the book and ebook whenever the examples are updated and/or the book is revved.

  • Many book chapters include screencasts to show how to use specific tools.  Anyone can watch these for free on Vimeo(albeit out of context), and we’ll be doing an iBooks version of the book in which the screencasts are embedded right into the ebook.

  • There’s no glossary.  With Wikipedia, who needs one?  Instead, important terms in the book are linked to the corresponding Wikipedia entries.  (In the print book, they appear as URIs in the endnotes of each chapter, but in the Kindle book, the links are live as long as you’re connected to the Internet).

  • At the moment there’s no index.  This may be a hardship if you only own the print version, but the electronic version is searchable.  We anticipate it will be common for print book owners to also own the electronic version (though sadly there’s no way for us to bundle the two purchases, since the print book is created and distributed by the CreateSpace print-on-demand shop and the Kindle ebook is distributed by Amazon, which ironically owns CreateSpace).

  • Separately from term definitions via Wikipedia, the book includes various links to interesting news articles, YouTube videos, and other online materials related to the text.  If you’re reading the ebook on a device like the iPad or Kindle Fire, you can just click on the videos and watch them.  The print book has URIs that you have to type in manually.

So that’s what we’ve been doing and why.  The alpha edition, with some missing chapters, is now available (January 2012) at  By March there will be some minor revisions to it and by Fall we hope to have a beta edition that is content-complete.

Visiting UC Berkeley

Visiting UC Berkeley

Here’s some useful travel info if you’re a colleague visiting UC Berkeley.

Hotels within walking distance of campus: Avoids traffic and parking-pass hassles if walking is an option for you.  Here’s a Google Map of suggested hotels within walking distance, including reservations links and phone numbers, also showing the campus itself, the BART (rapid transit) station, and the locations of Soda Hall (Computer Science) and South Hall (School of Information or iSchool).

Hotels you can’t walk to: Next best is to stay within a short cab ride, bus ride, or BART (train) ride, rather than driving.  There are some big-box hotels in Emeryville (Crowne, Doubletree, etc.)

If Armando is hosting you, you can also email for additional hotel advice, and (important) for parking passes if you will be driving to campus (not recommended if you can avoid it).

Driving/GPS: aim for “corner of Hearst and LeRoy, Berkeley, CA”, which is the entrance to Soda Hall.  Pickup your parking pass from Room 565 Soda Hall from Roxana Infante or Tamille Johnson, who will direct you to the parking structures.

Getting here without a car from Oakland International Airport (OAK):

  • Taxi: $50-60; 30 minutes + up to 15 minutes extra if heavy traffic

  • Public transit:  $6;  45 minutes + some walking time.  From the terminal take the “AirBART” shuttle bus ($3) to the BART (rail) station.  Take a Richmond train and get off at Downtown Berkeley (20 minutes).  The Google map linked above shows the location of the Downtown Berkeley BART station in relation to the campus and hotels. Trains run every 15-20 minutes.

Getting here without a car from San Francisco International airport (SFO):

  • Taxi: $85-95; 40 minutes + up to 30 minutes extra if heavy traffic

  • Public transit: $8.80;  55 minutes + some walking time.  Inside the airport’s International terminal is the BART (rail) station.  Trains leave every 15-20 minutes.  Take train marked “San Francisco/Pittsburg/Bay Point” and transfer at 19th Street/Oakland for Richmond train to Downtown Berkeley.  These transfers are timed and should add zero latency to your trip.

Getting here without a car from downtown San Francisco (financial district, Union Square, etc.):

  • Public transit, $3.85: Have your concierge direct you to the nearest BART (rail) station and take a train for Richmond or Pittsburg/Bay Point, whichever arrives first. (After 7pm and on Sundays, the only choice is Pittsburg/Bay Point.)  The Richmond train goes directly to Downtown Berkeley station (on Google map above).  If you took a Bay Point train, transfer at 19th St. Oakland to a Richmond train.  Trains come every 7 minutes during working hours and every 15-20 minutes outside those hours.

  • Public transit, $4.00: If you’re staying very close to the Transbay Bus Terminal (Howard St. between Beale and Main), the “F” line leaves at :10 and :40 after the hour and drops you off right at the corner of Hearst and LeRoy across from Soda Hall.  In light traffic, this is comparable to BART travel time; in heavy traffic, BART is faster but you have to walk from the BART station to Soda or South Hall (~15 minutes).

Getting here from San Jose Intl. Airport (SJC):

SJC is far away, and public transit is possible but takes a really long time. If you’re flying to SJC, rent a car.  Be advised traffic is quite bad at nearly all hours of the day on I-880, the main freeway connecting Berkeley and San Jose.  Travel time is about 1 hour 10 minutes, plus up to 30 minutes extra if heavy traffic.

Sunday, September 22, 2013

edX, Google, and

(This entry is being cross-posted to the Berkeley Teaching Blog.)

A few days ago Google and edX announced a new collaboration:  Google and edX will jointly develop, a site owned and operated by edX at which instructors at non-edX-affiliated universities can develop and publish courses online using the Open edX learning platform.

While they haven’t announced pricing, if history is any guide, courses with small enrollments may be free to publish and consume, whereas courses with larger enrollments may require payment.

And Google agreed that any improvements it makes to the Open edX software in the process will be donated back to the open source code base so that ALL edX users can benefit.

I think this is a big deal for many reasons.

First, it means greatly expanded access for instructors.  While everyone has been excited about “open education”, until now the only way to publish a high-visibility MOOC on edX or Coursera was via institutional agreement with your university. eliminates that barrier; in Anant Agarwal’s words (the president of edX), it’s like “YouTube for MOOCs.”  (Though I think they should have gone with “YouMOOC”.)  Just like YouTube, even if much of what gets published turns out to be mediocre, there will also likely be some real gems, since can give voice to amazing teachers who might otherwise have gone undiscovered outside their home institutions.  Of course, institutions will need to come to agreement with their faculty on the relationship between faculty-authored content for their campus courses and the availability of that content in a MOOC or mini-MOOC, whether they have institutional agreements in place with edX or not.

Second, it means expanded mindshare for Open edX.  Everyone knows MOOC tools are still relatively immature, but having so many more individuals use Open edX will greatly expand the “community of expertise” around it, which benefits both instructors at edX-affiliated schools and anyone else  It’ll be like a community-based helpdesk.

Third, Google is highly regarded for their software engineering prowess (deservedly so, in my opinion), and they have agreed that any improvements they make to Open edX in the process of running will be donated back to the open source codebase that is also used by the edX universities, so that everyone can benefit. While no specific intentions have been announced, it’s exciting to speculate.  For example, many instructors already use shared documents (Google Drive) or discussion groups (Google Groups) in their own courses; Google would be in a unique position to better integrate those services with the edX learning platform.  Such improvements would also benefit universities not affiliated with edX but who are using edX internally, such as Stanford University.

Fourth, Google’s education folks have close ties to academia, and the company has a long history of supporting university education and research through both cash grants and collaborations.  Maggie Johnson, Google’s Director of Education and University Relations, was a senior lecturer in computer science at Stanford for many years before moving to Google.  Peter Norvig, their Director of Research, is a co-author of widely-used textbooks on Artificial Intelligence and was the co-instructor of the MOOC on that topic that “started the MOOC movement”.  These folks  understand how universities work, which I believe makes it more likely that Google can find ways to support universities’ use of MOOC technology as part of an overall instructional mission.

In short, Google’s commitment to collaborate with edX gives a boost of credibility to not-for-profit open education generally and to edX in particular that I believe is a win for everyone involved —not just the folks at Google and edX, but the soon-to-be-thousands of instructors and millions of students who will be the primary beneficiaries of this technology.

Tuesday, July 16, 2013

What to expect when teaching your first MOOC

Now that the media seems to have exhausted the “MOOC honeymoon” stories, they’re looking for “MOOC disaster” stories. Two recent high-profile stories involve the truncation of a Georgia Tech MOOC due to a technology mismatch and a professor abandoning his MOOC in mid-course over disagreements on how to teach it.  How can instructors new to MOOCs prevent these problems from happening to you?  What should you expect if you’re getting into this area?  Here’s some thoughts based on our experience offering our  as a MOOC several times on edX and Coursera.

An incremental-refinement plan is better than being perfect

Especially if you’re recording your lectures in a studio-like setting, remember that you can always revise them later.  This Fall we will revise our lectures for the third time.  Leonardo da Vinci said “Art is never finished, only abandoned,” and you will always find ways to improve your material, but balance this with the need to juggle all the other commitments most faculty must manage.  Instead, focus on sustainability: once you’ve invested the enormous amount of work required to do a quality MOOC, what resources will you need to re-offer the MOOC between refreshes of the material?  We’ve managed to offer our MOOC two to three additional times between refreshes using community TAs (see “CONSIDER GIVING UP SOME CONTROL” below).

On the Internet, nobody knows you’re a dog

The New Yorker magazine famously printed this caption in the early nineties to draw attention to the anonymity availble on the Internet. Unfortunately, a very small fraction of MOOC students take advantage of this anonymity to engage in antisocial or antaognistic behavior on the forums, towards either their fellow students or the course staff.  We found that the few perpetrators were cowards hiding behind an anomyous throwaway email address.  Up to a certain point you can instruct your community TAs to shut down destructive threads, but if the behavior persists, see if you can have the students expelled from the course. Don’t let their behavior get you down and don’t let it sour the experience for the vast majority of students who are diligent and appreciative of your work!

Consider giving up some control

Most Berkeley campus courses use student discussion forums, and as conscientious instructors, we’re used to checking the forums and posting answers to questions there frequently.  But on-campus course forums tend to follow a regular rhythm as students work during the day, go to sleep (eventually), prepare for exams, or enjoy a short break following an exam or during a holiday. The cross-cultural, cross-time-zone reach of MOOCs obliterates this rhythm, and you may find it a lot more time-consuming to keep up with the forums.  The challenge is exacerbated by the fact that most MOOCs don’t have formal office hours or other means for students to get direct help, so the forums are even more critical to the student experience.

In our case, the first time we offered the course we recruited some of the strongest undergraduates from the previous campus offering of the course to serve as forum monitors.

Cultivate your Community TAs

On subsequent offerings, we recruited volunteer “World TAs” from among the highest-scoring MOOC students, and retained an undergraduate working about 20 hours a week to organize the volunteers’ efforts as well as serving as “Head TA.”  This system has worked well: the world TAs get some recognition, the course gets forum coverage by multilingual students spanning all the time zones (in our most recent offering, there was coverage nearly 24×7), and we get to have a life.  We still check in every week or two with our TAs to see how things are going, and often do 5-minute impromptu videos (Prof. Jennifer Widom at Stanford called them ’screenside chats’) on topics in the news relevant to that week’s course content.  We use a Google Group as a mailinglist to organize the world TAs and to coordinate the schedule of when each of them will monitor the forums.

Guest instructor Prof. Sam Joseph has taken this practice to a new level, posting short video interviews with community TAs who took the course and are now successfully getting professional work as a result of their skills!

Dry-run the technology

WIth hundreds of thousands of students, course technology has to work perfectly.  We extended the EdX platform with sophisticated autograders for our programming assignments.  Critical to our success was “dry running” new autograders and new assignments in our classroom (about 165 students last time around) to fix both logic bugs in the autograders and problems with the grading rubrics for new homeworks.  Dry runs will save you a world of pain.

This is only a start, but as MOOC instructors, we’re rolling up our sleeves this summer and do the work to make our course even better. All in all, it’s way more work than “just” owning an on-campus course, but it’s also tremendously rewarding.

Thursday, July 11, 2013

How I got my MIT 6.004 nerd kit past airport security

As an undergrad at MIT in the late 80s, in my Digital Design and Computer Architecture courses we actually built stuff using discrete TTL parts on high-end protoboards with integrated power supplies.  At MIT these lab kits, ubiquitous on campus because of the large size of those intro courses, went by the moniker nerd kits.

My vintage ~1987 nerd kit had been sitting in storage in northern New Jersey pretty much since graduation, so during my 4th of July weekend visit, I decided to repatriate it to my home in San Francisco, where I have a small collection vintage computer equipment.

You can see why I was apprehensive about getting through airport security, worrying about what a TSA security screener might think this is:

My vintage (c.1987) MIT 6.004 nerd kit

I left plenty of extra time to check in at Logan Airport.  As expected, after it went through the conveyor X-ray, it was pulled aside to the separate station where they swab things to make sure they’re not bombs.  I assured him that I was a computer science instructor transporting a piece of vintage teaching equipment, there was nothing dangerous or sharp in the case, and he was free to open, examine, and do whatever else he wished for as long as he needed to, and that I’d be happy to answer any of his questions.

When the TSA security tech opened the cover, a gentleman next to me who had been observing the proceedings said: “I see you have a nerd kit.”

I was stunned, since this immediately meant he not only had an MIT connection, but had probably had one for a long time, since kits like these haven’t been used at MIT since at least the 90s.  He looked vaguely familiar but I couldn’t place him.   I asked how he knew what it was.

“I taught using those back in the 80’s,” he said.  ”I assume you took 6.004.”

Me: “Yes, and I won the design contest, which is why I got to keep this.  But you weren’t teaching that class when I took it, in 1987.”

Him: “Ah, yes.  I was teaching 6.001 around that time [a rigorous entry-level software course].”

Finally it clicked who he was, and I lit up.  ”You’re Rodney Brooks!  I didn’t recognize you with short hair, but when I took 6.001, you taught my recitation section and you were really great!  Of course, you taught thousands of students so I wouldn’t expect you to remember me.”

Brooks: “Hmmm…and your name is…?”  I told him.  ”Oh yes!  Sure I remember you!  You’re faculty at Berkeley now, right?”  Then he turned to the security guy swabbing my nerd kit: “It’s OK.  I taught this guy when he was just a boy.”

The security guy closed up the nerd kit and sent me on my way.

Stranger than fiction.

Sunday, May 5, 2013

Meeting some MOOC students in Paris

Last week I traveled to Paris for CHI 2013 to speak on a panel about online education.

I’m getting to do that a lot these days, and while the travel can be tiring, it’s certainly more fun when I get the chance to meet up with MOOC students face to face!

In a previous post I described my experience meeting up with two students in Singapore and Indonesia.  This time around I got to meet Julien Coupez (left) and Daniel Kindler (back, second from left) the evening I arrived in Paris.  We had a round of drinks near the Place de Clichy with my colleague Prof. Marti Hearst (right), an accomplished researcher in human-computer interaction and information sciences at Berkeley with whom I’m co-advising graduate research on learning in MOOCs.

It was fun to hear about how Julien’s “social bookmarking” startup NexBoo already uses some of the Agile techniques covered in CS 169.1x, while Daniel wanted to learn about software practices that are not widely used by his current employer (a large financial services company).

As always, connecting with colleagues around the world is a way of reminding myself that at some level we’re all playing on the same team.  Thanks to Daniel and Julien for making the time to get together, and for making this visit to one of my favorite cities even more enjoyable!

À bientôt, folks!

Sunday, March 10, 2013

The traveling MOOC professor

I just returned from a short but much-needed vacation to Thailand and Indonesia, with a brief stopover in Singapore.  (My wife and I are scuba divers, so we were diving in the Raja Ampat area of Indonesia, which was spectacular; and since we were going to be on the other side of the world, we decided to also check out Chiang Mai, since we’d never been to Thailand, and Singapore, since everyone has said it’s a great place to visit.)

Since I periodically receive nice emails from students in our , in a fit of inspiration while traveling,  I wondered if any of those students might live in Jakarta or Singapore, two unfamiliar cities where we had stopovers.  With EdX’s help, I sent an email to the MOOC students, and I was pleased to receive numerous replies!

In Singapore, I met Tony Luong (left) and Hung Mai (right), Vietnamese nationals currently living and working for IBM in Singapore.  We had a nice sushi dinner with them (at Sushi Tei in the City Square shopping center near Little India, if you know Singapore), during which we talked about the differences between living and working in IT in the USA vs. Asia, professional plans for the future (Hung is coming to the US to work soon!), software engineering, and life.

In Jakarta, I was greeted at the airport by Reza Anwar and his wife Tanti Ruwani.  Both are Indonesian nationals who had had part of their higher education in the West, so they spoke perfect English and were lovely hosts, giving us a brief driving tour of downtown Jakarta and a wonderful traditional Indonesian dinner at a very nice restaurant housed in a former Dutch-colonial mansion.  Trading stories about life, business, and career aspirations with students from other countries was inspiring.

I’m keenly aware of my role as an ambassador not only of Berkeley, and not only of the USA, but of higher education in general when doing visits like this.  I was very fortunate to get in touch with students who had enjoyed taking my course (and had very kind things to say about it) and I’m now thinking I’ll try to find students to connect with whenever I travel abroad!  It’s one more way to realize that despite whatever other differences people may have, at some level, we are all indeed in this together.

Thanks to Tony, Hung, Reza, and Tanti for turning what could have been unremarkable stopovers into a rewarding social and professional experience!

Monday, January 14, 2013

What was it like to teach a MOOC?

Since I get asked this question a lot, including by the media, I thought it’d be useful to blog it.

What MOOCs have I taught?

I split our 15-week on-campus  into two pieces, an “introduction” and “advanced” MOOC.  The Intro MOOC was offered three times on Coursera and is now in its second offering on EdX, with whom UC Berkeley has an institutional partnership.  The Advanced MOOC has been offered just once so far on EdX.  (The course number is CS 169.1x for the Intro and Cs 169.2x for Advanced.)

What was it like to adapt the campus course?

It has been a lot of work, but it has greatly improved the on-campus course as well as making it available worldwide.

Because of high enrollment demand in the on-campus course, we had already started thinking about automatic grading of programming assignments.  We and our on-campus TAs spent hundreds of engineer-hours creating sophisticated automatic graders for programming assignments since we believed neither MOOC nor on-campus students would be well served by a software engineering course whose assessments were based solely on multiple choice or short-answer questions.

While this was grueling, the autograders are reusable and customizable, and have graded hundreds of thousands of assignments so far, which would require an army of TAs to do manually.  Thus it helped us expand access to the course on-campus as well as online.

I also reorganized my 90-minute lectures into 8-12 minute “lecturelets”, each covering a specific topic and accompanied by self-check questions.  Our colleagues, the founders of Coursera, suggested this format worked well for MOOCs, but we found that it also made the on-campus lectures livelier and better-attended.

We live-captured our lectures and postprocessed them afterward.  In general, we were unwilling to spend time on the MOOC that would not also improve the on-campus course.  My teaching ratings and the on-campus course’s ratings (given anonymously by the students) both went up when the MOOC technology was integrated.

What worked well?

The automatic grading allowed vastly more students to practice the material, and modulo some glitches, was widely praised.

The online students took the same quizzes and did the same programming assignments as the Berkeley students, on comparable deadlines, and the best of them did just as well.

Intelligence is uniformly distributed worldwide: students from 130 countries enrolled, and thanks to lecture subtitling provided by EdX, were able to follow the lectures despite not being native English speakers.

More important than the number of students reached was the smaller number (hundreds) of truly motivated students to whom we were able to give an opportunity otherwise unavailable to them, such as a student in the Gaza Strip who only received 6 hours of electricity each day and used part of it to complete our course.

What challenges did you encounter?

This is an enormous amount of work.  We did it on our own time and without extra compensation, but not all instructors will have that luxury.  We also had discretionary money we used to pay TAs to help support the online course; there will have to be more systematic sources for those funds eventually.

Online students don’t get face-to-face time with instructors and TAs, as on-campus students do.  The discussion forums work pretty well, but despite being monitored by TAs, they’re basically self-service.

The on-campus students get to do team projects in small groups working with external customers.  In their evaluations they indicated this was the most valuable aspect of the course.  Yet the online students don’t get to do this part, as we haven’t figured out how to scale up the process of forming project teams and matching them up with external customers.

Discussion-oriented learning doesn’t work well at large scale.  Some parts of our course dont’ require it, but for the parts that do, such as office hours and TA design reviews of team projects, there was no way to give the online students this experience.

Nonetheless, our goal was not to replicate every aspect of the on-campus course, but rather to identify which aspects, if converted to MOOC format, would work well for both online and on-campus students and focus on making those things great.

There is lots of cheating in MOOCs.  We have hard as well as anecdotal evidence from our courses and colleagues’ courses.  But since no credit is offered, we’ve chosen to focus on improving the course for the vast majority of honest students rather than try to catch the cheaters.

A very small fraction of MOOC students are vocal jerks or have a disproportionate sense of entitlement about everything being free.

Would you do it again?

Absolutely, I’m planning to.  And as I develop new on-campus courses, I’ll be blending in MOOC technology, such as shorter lectures with self-check questions and extensive autograding.  That allows the TAs and me to shift our scarce on-campus student contact time from a lower-value activity (repeating the same lectures) to a higher-value activity (face time with students who’ve watched lectures and/or done the prep work using MOOC resources)

Book summary: A Thread Across the Ocean

A Thread Across the Ocean: The Heroic Story of the Transatlantic Cable by John Steele Gordon The most wonderful thing about the writ...