Instructors at Harvey Mudd College, a school known for its science and engineering experiments, are studying a new group of subjects: their students.
Over the last couple years, three professors have split some of their classes in half. One group takes part of the course online while the other has only classroom instruction. As part of a federally funded study, the professors have compared the groups to see which performed better.
So far, there hasn't been much difference. "It wasn't necessarily the outcome we were imagining," said Jeff Groves, the school's dean of faculty.
If the study's preliminary findings hold up for another two years, the results could have bigger ramifications at Harvey Mudd, which has offered only a handful of courses with online elements. As the authors noted in an academic paper: "Students (like us faculty) hate change."
Groves and others said the results of the study could lead to a different teaching approach for some instructors. If students can perform equally well by watching the lecture at home, professors have more time for discussion and student interaction in class.
"We tend to hire people who have the potential to be innovative educators, so I do think it [the study] will be talked about a lot and may very well have an effect," Groves said.
The approach represents what some experts say is a more effective way of teaching online. Beginning several years ago, many colleges, administrators and politicians, including Gov. Jerry Brown, touted online classes as a way to provide greater college access, hold down tuition and get more students to graduate.
But after several high-profile failures in large online efforts, some educators have begun pulling back from Internet-only courses and are offering "flipped" classes, where students can watch lectures online and meet in person for discussions and group exercises.
In 2013, San Jose State University offered five online-only courses in partnership with Udacity, a Silicon Valley company. But San Jose students had high failure rates, especially in remedial courses, and the school suspended its partnership with Udacity.
"I think there was a sense of loss of ownership" with online courses because professors had little direct contact with their students, said Dean Florez, president of the Michelson 20 Million Minds Foundation, a nonprofit that focuses on higher education policy.
Flipped classes give "faculty a way of teaching online in their own way," Florez said. "It's a natural evolution."
In addition to Harvey Mudd, schools including Villanova University, the University of South Florida and North Carolina State also have received federal grants to study flipped courses, which are becoming popular in some middle and high schools.
Harvey Mudd professors Darryl Yong, Rachel Levy, Karl Haushalter and Nancy Lape received the nearly $200,000 federal grant even though only one, Yong, had experience teaching any kind of online class. The others leaned on him for tips as they prepared their videos.
If undergraduates in the flipped class performed significantly worse than the control group, the professors would've stopped the experiment.
"We would certainly not have felt comfortable continuing to teach using a method we had proven was significantly less effective," said Lape, an engineering professor.
Students who take the courses aren't told which section they will be in until the first day. The students in the traditional group attend lectures, while the others watch them online before class. In this semester's flipped chemical engineering class, students take a quiz on the lecture at the beginning of each class.
They can email the professor before the class and ask questions before the test. Sometimes, discussions will closely match what's on the test so "everyone gets the right answer, which is fine," Lape said. "It shows they're thinking the right way."
Undergraduates in the two sections are then compared with each other on a variety of factors, including tests they take before and after the course designed to measure how much they learned, and homework.
Even though students in both sections perform equally well, some admit it's sometimes frustrating to follow the online lecture. "I like to ask questions in the moment," said Emily Beese, a sophomore engineering major.
Others said they liked the flexibility of the format because they could watch the lecture as many times as they wanted. "I don't normally ask questions during the lecture anyway," said Yeahmoon Hong, a senior engineering major. Still, Hong said he sometimes forgets to watch the videos until the last minute because he's not used to doing so. "I sometimes push it too far," he said.
A few students have requested to switch classes because they didn't want to take the flipped version, Lape said.
"We're not asking one group to sacrifice their learning for the sake of the study," Lape said.
Lape said preparing to teach the courses was a challenge since each is different. In flipped sections, Lape has to spend more time talking with groups of students to make sure they've understood the online lecture, and in the traditional course she has to remind herself to create an environment where students are comfortable asking questions.
Each "requires different kinds of skills," she said.
Other schools have found that students are performing surprisingly well in flipped courses.
At Villanova, a group of professors found that students in flipped engineering courses scored three percentage points higher than their counterparts in traditional courses.
Undergraduates in the bottom third did even better, outscoring their counterparts in regular classes by seven percentage points, said Randy Weinstein, a Villanova chemical engineering professor.
Weinstein said professors have been able to track when students rewind videos to see how well they do on tests after longer online lectures. "There's just a lot more data for us to track and use to make decisions," he said.
The only thing Weinstein said they haven't been able to do is to run a controlled, Harvey Mudd-style experiment. One professor proposed a similar study at Villanova, but administrators decided it would violate the school's policy against experimenting on humans, Weinstein said.
"It would be great to have a controlled experiment," Weinstein said. "It's the gold standard."