Q&A: What teachers and parents should make of The Times’ rankings

With the recent unveiling of The Times’ teacher and school “effectiveness” database, teachers and parents have asked us what to make of this information. Here’s our advice.

Teachers:

I try my best to be an excellent teacher, and I’m always trying to improve. I really thought I was doing a good job. But The Times gave me an “average” value-added rating. Should I change how I teach now that I know I’m just “average”?

No. You might actually be an excellent teacher — even just based on this limited measure of improving students’ math and reading California Standards Test (CST) scores. A recent study by the U.S. Department of Education indicates that value-added scores using three years of data (the typical amount used in practice) can be so unreliable that there may be about a 25% chance that your score put you in the wrong category. (Teachers end up in The Times database when they’ve taught at least 60 students, which at the elementary level represents about three years of data.) In other words, if The Times rated you as average, you could be great or you could be terrible.

My principal assigns me to teach some of the most challenging kids at my school. I think that’s why I got rated as less effective than other teachers. But The Times says that my value-added score generally won’t be affected “by low-performing students, English-language students or other students with challenges.” Who’s right?

You’re right. The kinds of students in your class do affect your score. See one relevant study here. The Times’ own technical report shows it too — just look at the estimates of the effects of “averaged lagged test score” (a measure of how well the students in your class were doing before they got to you) in Table 6 and the proportion of gifted, English-language learners or African American students in each classroom in Table 7. Each effect looks small, but each could affect your score. And as you know, none of these variables completely captures the unique mix of students (and potential obstacles to good teaching) that you get each year.

I teach in a school in a well-off neighborhood where parents send their kids to expensive summer camps and hire tutors. The Times labeled me as “most effective,” but I suspect that some of that effectiveness results from kids’ outside activities. Am I wrong?

Not necessarily. One problem with The Times’ value-added scores is that the time between the tests includes summer vacation. It’s well known that kids from disadvantaged backgrounds fall behind their more advantaged counterparts in reading during the summer. So poor kids who had the same scores as rich kids in the spring will probably be further behind when they go back to school in the fall. The poor kids’ teachers will need to produce larger gains than the rich kids’ teachers to earn the same value-added score. And, yes, anything that parents do during the school year to improve kids’ reading and math skills (hiring tutors, helping with homework and so on) gives teachers more value-added credit than they deserve.

Students in my classroom go to other teachers for their math instruction. How do I know that The Times is calculating my math value-added score based on the students I actually teach? What if my scores are based on misinformation?

The data set undoubtedly contains some errors. The Times analyst also made specific decisions about which teachers to exclude and how to attribute kids’ learning gains to teachers when students switch classes for particular subjects. Teachers have a right to know more about those decisions and how they affect the scores. Now that The Times has publicly labeled teachers with these statistical constructions in the interest of transparency, it should release the underlying data and programming code so teachers and researchers can explore the sensitivity of the results to different data management and analytical decisions. Both fairness and science demand that this research be replicated. You should too.

Parents:

I always try to do what’s best for my children. I see from The Times’ analysis that a couple of the teachers in my child’s grade are “most effective.” Should I try to get my child into one of those classes?

Not necessarily; see our answers above. Based on what’s known from other value-added studies, The Times’ value-added scores have labeled some teachers as effective when, in fact, they’re just average. And remember, some teachers who raise scores on average may not be the best teacher for your child. Your best bet when choosing a teacher, just as it was before The Times published these scores, is to ask other parents about their experiences with different teachers, insist that the principal let you observe classrooms and try to choose a good match between your child’s personality and a teacher’s style. And of course, no matter what the teacher’s value-added score is, be vigilant during the school year: Pay attention to the work your children bring home and the skills they’re learning, and talk with the teacher if you don’t think their work or progress is appropriate for your child.

My child has been assigned to a “least effective” teacher. What should I do?

As detailed above, that teacher may be misclassified. But as parents, we understand your wariness. So we recommend talking to the principal to find out what might be unique about the kids that teacher usually teaches. We also recommend meeting the teacher, paying close attention to the quality of homework assigned and the feedback the teacher provides, asking your child about his or her experiences in the class, and observing or volunteering in the class. Do the kids pay attention or are they out of control? Does the teacher try to help students understand the material? Does the teacher try to make class interesting?

My school is rated as not particularly effective? Should we change schools?

Don’t move your child just because of The Times’ report. Even though The Times put schools into categories, the paper’s own technical report says that school-to-school value-added differences are small. Also, changing schools may hurt your child’s achievement, so make sure your child’s experience in his or her current school is bad before making a change.

We staunchly believe that all children should master the skills tested on the CSTs. We also believe that no child should have to spend a school year, especially in the crucial elementary years, with a bad teacher. But just as you wouldn’t buy a home or a car based on just one piece of information, take care not to judge your schools or teachers based on just The Times’ score. Students, parents and school communities often have a good sense of how effective a teacher is. That information should still count for a lot in your decisions about your child’s schooling.

Tiffani Chin is executive director of EdBoost Learning Center and the author of “School Sense: How to Help Your Child Succeed in Elementary School.” Meredith Phillips is an associate professor of public policy and sociology at UCLA and coeditor of “The Black-White Test Score Gap.”