Skip to content
In-school trials of software can be easily influenced
Lacking good research about education software, school districts often do their own evaluations. But their trials are easily influenced by software vendors -- or by administrators already inclined toward buying the product.
A test of Plato Learning algebra software in Prince George's County this spring offers a prime example.
The county was using the software in some after-school and summer programs when a Plato vice president approached district CEO Andre J. Hornsby in January and suggested the county hold a free trial of the software for its regular classes.
School officials rushed to set up trials in ninth-grade math classes at two low-performing high schools, Fairmont Heights and Potomac. At both schools, two classes, selected at random, were to use the software every other day, alternated with instruction by a teacher; a third class was to use the software every day; and the other classes did not use the software.
Students were tested early in the semester and again at the end of the year to measure their progress. The result? District officials say that the software improved performance, and they are using it again this fall at the two high schools, while still deciding whether to purchase it for other schools.
But teachers involved in the trial question its value. The software was not ready until March, they said at the end of the school year, so students used it for only three months. And there were technical problems at Potomac High, in Oxon Hill, so that classes in the experimental groups went days at a time without using it.
"I'm not sure the data is that scientific as far as control groups and all those things, because [the trial] was ... so rushed," said George Wake, the math teacher coordinator at Fairmont Heights.
The trial's disorganization was visible during a reporter's visit in the final week of the school year at Fairmont Heights, which sits among small bungalows in low-income Capitol Heights, near the Washington line.
Some students spent their time in the computer lab furtively going online to write e-mail or, in one case, to look for information on HIV. Several of those who did work on the Plato software asked their classmates for help on the "mastery tests" at the end of each section, which monitor student progress and are supposed to be done alone.
As it turned out, though, the trial might have been moot. Teachers said administrators had told them in early May, well before the end of the trial, that the district had agreed with Plato to continue using the software in the fall and eventually buy it for countywide use. Administrators declined to say how much the program would cost, but Plato purchases at other high schools have cost about $60,000 per school, not including annual upkeep.
District officials acknowledged problems with the trial but denied that any decisions about Plato were made in advance. They declined to release their 70-page evaluation of the pilot, saying it was still in draft form.
"I'm doing an exhaustive evaluation of this program to determine its effects, using the most powerful research design that I have at my disposal," said Leroy J. Tompkins, the district's chief accountability officer, earlier this month. "We are fairly optimistic that there's some positive benefit, enough to warrant a broader look at the program."
The district's eagerness to consider software programs has attracted many vendors. As a big district with 137,000 students, almost half of them low-income, Prince George's receives large amounts of federal aid, which is easier to spend on technology than are local dollars. With 43 of its 197 schools deemed "in need of improvement" under No Child Left Behind, its administrators are looking for quick fixes.
Add in the fact that, thanks to its location just outside the nation's capital, the district is watched closely by other big metropolitan districts, and you have the makings of a software company's dream territory.
"It's a good area to get our foot in the door," said Jill Stine, a mid-Atlantic sales representative for Plato.
The county's spending on education software has well outpaced other large Maryland districts. This month, it is launching a $3.9 million "instruction management" system from the Grow Network Inc., which will compile student test scores and suggest Internet resources to address weaknesses. Meanwhile, the county is also requesting proposals for a new "student information system" to track student data that could cost more than $10 million.
This fall, the county is the site of a trial of reading software from Scholastic Inc. The county is also debuting early-literacy software from LeapFrog SchoolHouse that will be used in all kindergarten classes with large numbers of poor students -- at a cost of $1 million. In many schools, the software will be used for the same purpose as the Waterford reading software, produced by Pearson Digital Learning, that the district bought years ago.
Judith Finch, the district's director of technology support, said the heavy spending on software is a sign of Hornsby's belief in technology. "I don't think we're a target, and I don't think we're easy. What we're doing, in the long run, is investing in the very best for our youngsters," she said.
But the district's buying spree has been met with reservations from some teachers.
At Fairmont Heights, several math teachers questioned the decision to spend thousands on Plato. They said student gains in math might be more enduring if the district invested in reducing class size -- about 30 students now and slated to grow.
"If classes were smaller, say, 20 or 15, they would be very controllable," said Wake. "It's the money [for Plato], that's what I don't understand. They could reduce class size with more teachers -- but that's not my decision."
Prince George's schools
Chief executive officer: Andre J. Hornsby Enrollment: 137,000 students Racial/ethnic makeup: 77 percent African-American, 10 percent Hispanic, 9 percent white, 3 percent Asian-American Number of schools: 197 Schools receiving Title I funding, 2003: 73 Federal Title I funding (2003): $26.6 million Schools "in need of improvement": 43 Average SAT scores (2003): 443 (verbal), 436 (math) Average teacher salary(2003): $50,734 SOURCE: Prince George's County, Maryland Department of Education, U.S. Department of Education