Testing the system

Last weekend, the Times launched its “Grading the Teachers” series, which presented the concept of a value-added approach to evaluating teachers. Individual test scores were scrutinized over several years as students moved from one teacher to another. The scores were analyzed to see which teachers consistently raised the test scores of their students and which lowered them. Later this month, the paper will post the data on its website. Times editorial writer Marjorie Miller spoke to a variety of education leaders and professionals about the pluses and minuses of this method of evaluating teachers. Their remarks — or writings — have been edited for clarity and length. In coming days, we’ll feature the voices of classroom teachers.

Stephen Krashen
Professor emeritus, USC’s Rossier School of Education

Value-added evaluations of teachers assume that higher test scores are always the result of teaching. Not so. Test scores are influenced by other factors.

We can generate higher scores by teaching “test preparation” strategies for getting higher scores without students learning anything. We can generate higher scores by testing selectively, making sure that low scorers are not in school the day of the test. And of course we can generate higher scores by direct cheating, sharing information about specific test questions with students.

Teachers who prepare students for higher scores on tests of specific procedures and facts are not teaching; they are simply drilling students with information that is often soon forgotten. Moreover, research shows that value-added evaluations are not stable year to year for individual teachers, and that different reading tests will give you different value-added scores for the same teacher.

If The Times is serious about helping children, don’t bash teachers, address poverty. American children from high-income families do very well on international tests, but our children of poverty do much worse, and nearly 75% of LAUSD students are poor enough to qualify for free lunches. Our overall scores are mediocre because the U.S. has such a high percentage of children living in poverty (25%, compared with Denmark’s 2%). We need to protect children against the negative effects of poverty with better nutrition, better healthcare and more access to books. They do not need more standardized testing.

Ted Mitchell
President and chief executive, New Schools Venture Fund

One part of the crisis in education is our inability to use data to make decisions. We are driving blind, making decisions on the basis of hunch and anecdote. The data we need are data about teacher effectiveness. Unlike a static measure such as how well kids did on a single exam, value-added analyses put the emphasis on taking kids from wherever they are and doing all you can to help them improve. Such analysis gives a good sense of how a teacher is doing over time. It’s a way to give credit to teachers who are moving the needle.

Value-added analysis is not perfect, of course, but we should embrace the data we have while trying to make improvements in our data system. I think this kind of analysis should be a part of a teacher’s evaluation, and as we improve the methodology, it should grow in importance. What’s key is to get started using it now.

As for making value-added data public, I’m all for it. One of the big problems in public education is the lack of transparency and public trust. This is the first step in rebuilding that trust. It is about true transparency. Now the system will have to respond. In the last five days, we have been having more spirited conversations about teacher effectiveness than we have in the last 10 years combined.

Steve Zimmer
Los Angeles Unified School District board member

If you are to give validity to the value-added approach to measuring a teacher’s performance, the prerequisite is that the standardized test is a valid measure of a student’s learning and knowledge, and that in itself is controversial. To say it is the most important, or the sole measure, is without any validation in the mainstream academic conversation on teacher effectiveness. It invalidates every other piece of teaching and has consequences that extend far beyond an individual teacher’s accountability. If test score are determinant, and if you want to keep teaching, then you have to alter the place standardized testing assumes in your instructional priority. Not because the school board says so, or the principal says so, or because you think so, but because The Times says it’s the most valid measure and it’s what is published.

There’s a legitimate discourse about the accountability of public servants. I favor families having access to information about a school’s performance and even to aspects of a teacher’s performance, but I have reservations about whether this should be played out in public. We don’t publish a database of every infraction a police officer has, or the attendance records of our firefighters.

I would say teacher evaluations are part of the compact between a teacher, a family and the school leadership. There are ways and means by which a family can have access to a school’s performance and to the comprehensive landscape of an individual teacher’s approach to the classroom without completely violating the privacy rights of the teacher.

Marco Petruzzi
Chief executive, Green Dot Public Schools

At Green Dot, we are very much in favor of using student test scores as one of the key elements to measure teacher effectiveness, and we’re very much in favor of transparency, if used appropriately. Value-added measures work very well for elementary schools but are more complicated to use in high school, where students sometimes take only one course of one subject (chemistry, for example). You can and should still use the methodology, but it should be complemented with other measures of teacher effectiveness, like feedback from students and parents, evaluation of student portfolios, classroom observations, attitude, etc. to get a fuller picture.

The primary goal of value-added analysis should not be just evaluation; it should form the basis of clear conversations with teachers about what’s working and not working in their classrooms and how to become more effective. At Green Dot, our teachers sit down after every benchmark test, compare test results and talk about how their students are progressing. One teacher might say, “60% of your students are proficient on a topic and mine are only at 20%; I’ve got to come see what you’re doing.” This is a healthy discussion and the cornerstone of any growing profession. Our teachers embrace this system because it supports their career development and is not purely evaluative.

I’m open to publicly grading schools, and I’m also in favor of transparency within a school community, where you can set the data within the right context. I’m uncomfortable, however, with publishing a narrow data point with a person’s name attached to it for public consumption without the proper context. Teachers, as professionals, should be judged on a broad set of data. Student performance should absolutely be at the core of this effort.

John Rogers
Associate professor, UCLA Graduate School of Education and Information Studies and director of the Institute for Democracy, Education and Access

There are serious consequences to the Los Angeles Times publicizing data about 6,000 Los Angeles third- to fifth-grade teachers and grading those teachers from “least effective” to “most effective.” The Times says that the data will give parents a powerful tool for action. But it is not clear how parents can put information about individual teachers to good use. As offered by The Times, this flawed tool will not improve learning. And it likely will cause harm.

One possible immediate effect could be that capable prospective teachers will decide not to enter a profession in which they risk public embarrassment based solely on an undeveloped statistical method. Consider the well-documented estimates that 25% of valued-added assessments are likely to be in error. Even the best teachers will want to avoid grades three, four and five.

What of the parents who are told their child’s teacher is “ineffective,” or even marginally less effective than a teacher across the hall? Many parents are likely to pressure principals for a “high scoring” teacher. This is not a recipe for school improvement.

Value-added methods are a limited and underdeveloped tool. By focusing narrowly on standardized tests, these analyses ignore much learning that matters to students, parents and teachers and cannot stand alone as a measure of “effectiveness.” The National Academy of Sciences has identified several of the problems posed by value-added methods.

First, the National Academy of Science notes that student assignments to schools and classrooms are rarely random. It’s not possible to definitively determine whether higher or lower student test scores result from teacher effectiveness or are an artifact of how students are distributed.

Second, you can’t compare the growth of struggling students with the growth of high performers. In technical terms, standardized tests do not form equal interval scales. Enabling students to move from the 20th percentile to the 30th is not the same as helping students move from the 80th to the 90th percentile.

Third, estimates of teacher effectiveness can range widely from year to year. In recent studies, 10% to 15% of teachers in the lowest category of effectiveness one year moved to the highest category the following year ,while 10% to 15% of teachers in the highest category fell to the lowest tier.

The National Academy of Sciences concluded that value-added methods “should not be used as the sole or primary basis for making operational decisions because the extent to which the measures reflect the contribution of teachers themselves, rather than other factors, is not understood.”

The Times should heed this caution, lest its efforts to cure the ills of LAUSD lead to even greater harm.

Yolie Flores
LAUSD board member

Whether or not you agree with the Los Angeles Times’ article on teacher effectiveness, it has been undeniably successful in spurring a long-overdue public dialogue on this issue. Teacher effectiveness is finally at the center of the public eye, and what we decide to do with this moment will affect an entire generation of students — for better or for worse. Let us not squander this opportunity.

The Times’ articles could not have come at a more critical moment. This past April, the LAUSD Teacher Effectiveness Task Force, created in 2009, issued recommendations calling for the development and implementation of a robust evaluation system for teachers and administrators. Such a system will employ multiple measures for addressing effectiveness, including standardized test scores. Peer-to-peer review, principal assessments, parent and student input, and other measures were identified as equally critical evaluation components. The task force, composed of teachers, principals, parents, union and community leaders, as well as district and education experts, was the first of its kind in LAUSD and established a clear and provocative blueprint for reform in five key areas: tenure, compensation, evaluation, support mechanisms and legislative action.

The task force itself is not enough, however. It cannot, nor was it ever intended, to work in a vacuum. Every stakeholder must step up to the plate if we are to transform our system to one that reflects, first and foremost, the needs of children rather than the adults it employs. Our union leaders must stop fighting every effort that signals positive change for our students simply because it threatens the archaic status quo. Further, our state legislators must find the political courage to draft and pass legislation that allows school districts to move from a seniority-based system to one that includes measures of effectiveness when making employment decisions. And parents must be fully engaged and be equal partners in the education of their children, demanding great teachers but also ensuring that learning does not cease when the school day is over.

All of our children deserve great teachers, and we must no longer entertain cheap compromises or stall tactics that fall short of ensuring this fundamental educational necessity.

Spike Dolomite Ward
Founder and director, Arts and Education Aid Council

I greatly admire the many passionate teachers in this school district. It may seem contradictory, but I also support releasing the names and scores of teachers, because I feel like UTLA has protected poor teachers for far too long and this may make it easier to get rid of dead weight. Not much else can be done, at this point, but to try to expose poor teachers publicly and hope the truly inferior teachers will leave. They have hidden for far too long behind their union and have hurt the children of Los Angeles. Perhaps public humiliation is the last and only resort to get rid of them.

At the same time, I would say to parents: “Don’t rush to conclusions.” If you know a teacher, have a good relationship and are involved, these scores are not going to tell the whole story. Personally, I wouldn’t be too concerned if my child’s teacher received a “bad score,” because I see the big picture.

A.J. Duffy
President, United Teachers Los Angeles

There is a saying: “When all you have is a hammer, everything looks like a nail.” Since the L.A. Times only has a set of data on the California Standards Test from around 6,000 teachers, that is what it hammers on.

The Times is right to bring attention to the lack of effective data analysis in the LAUSD. It makes little sense for the district to collect all of this data on the test score trends if it does not share this data with the teachers themselves. However, value-added models are notorious for their lack of precision. That is why even many advocates of this kind of data acknowledge the importance of multiple measures, including improved administrator observations.

The Times is missing many critical tools in its analysis: screwdrivers to tighten up administrators’ observations and give teachers clearer feedback; saws to cut away the mindless paperwork that keeps administrators tied to their desks instead of in the classroom supporting teachers; a mirror to hold up so the governor and legislators can see the true face of our underfunded schools. (California is the eighth-largest economy in the world, but our state is 47th in per-pupil spending in the U.S.)

Value-added analysis tells teachers nothing about how to improve their practice. To truly build a strong school system, teacher evaluations should be overhauled to create a meaningful, comprehensive definition of effective teaching. This includes creating a definition of student learning, and it is so much more than a standardized test score, which measures only 15% of what is taught in class. Test scores must be reinforced and corroborated by multiple indicators, such as teacher judgment, teacher-generated assessments and observations, grades, performance-based tasks and portfolios of student work. We need to use the data — all the data — to enhance learning through changes in instruction and to target appropriate services to students.

If we accept the blueprint for education expounded by the Times, it will lead to the further narrowing of the curriculum, teaching to the test, and the exclusion of critical-thinking skills, the arts and any other area not measured by standardized tests. Public education will become about building a perfect test score, not about creating a well-rounded, engaged, intellectually curious child.

Bonnie Reiss
California secretary of education

Bravo to the intrepid reporters at the Los Angeles Times for doing what school districts throughout California and our nation should be doing — providing parents with open and transparent data on teacher effectiveness.

Rather than spending precious time organizing a boycott of this paper and threatening other media outlets, union leaders should work with their district leaders to create fair, multiple-measure evaluation plans that support effective teachers. Our students live in a competitive world where they are evaluated by assessments; they take tests throughout their schooling, gain admission to college based on these grades and other test scores, and then compete in difficult job markets.

It is time for those union leaders who have dozens of excuses for why teacher evaluations might not be fair to teachers to put their considerable abilities toward figuring out the best method that both protects students’ interest and is fair to teachers. A value-added assessment method that takes into consideration student growth is a good starting point. This method fairly recognizes that if a sixth-grade teacher gets a class of students all reading at second-grade level, they might not be expected to get their students to sixth-grade level by the end of the term. However, teachers are still expected to help their students improve academically.

Our public education system must be allowed to consider the effectiveness of the adults entrusted to educate our students. All students deserve a high-quality education that prepares them for college and career, and the needs of students must always come first.

Ben Austin
Executive director, Parent Revolution

When I was 6 years old, my teacher noticed I wrote in mirror writing, and I was diagnosed with dyslexia. I eventually needed to take the SAT and LSAT with extended time, and to this day I still have trouble reading. So I am sympathetic to the critique that standardized tests are not the holy grail in public education. They alone can’t tell us who’s an effective teacher: We also need principal, peer and parent evaluations.

But we get caught up in a lot of arguments that have nothing to do with kids. Certainly, as a parent, I want to know how my child is doing on tests. LAUSD has signed on to Race to the Top, and in its application, it committed to using value-added analysis as 30% of teacher evaluation. I endorsed that. I believe tests are an important part of evaluating whether a teacher is effective or not, and it’s a no-brainer that they should be included in our analysis.

The Times was able to acquire this information because it was not considered part of the personnel files of individual teachers. So long as the results are dumped into a file cabinet and are not being used to evaluate the teachers, I — as a parent — want access to the information, and I am comfortable with allowing it to be made public. I am also sympathetic to privacy concerns, so if a district is going to use these scores to evaluate teachers, hold them accountable and reward them, then I am comfortable not allowing them to be made public.

Tamar Galatzan
LAUSD board member

What you didn’t read in The Times articles on value-added analysis are the far more stringent actions the district has undertaken over the past two years to either dismiss bad teachers or create the conditions where they will decide to retire. This effort, which has the enthusiastic support of the superintendent and the board, and the tacit support of United Teachers Los Angeles, has already produced a substantial increase in the number of poor performers who have left LAUSD classrooms, never to return.

The most dramatic rise is in the number of teachers who were asked to leave during the two-year probationary period before tenure is granted. A total of 89 of these teachers will not be returning to LAUSD this fall, which is more than twice as many as in typical years.

The reason is simple: increased vigilance. Teacher performance is being observed, monitored and assessed much more closely than before. Administrators are now being held directly accountable for providing accurate and critical evaluations of all teachers at their respective schools. The district is also redrafting the evaluations that principals use to assess teacher performance.

Of course, there are also poorly-performing teachers who do manage to receive tenure — or develop bad habits later. To catch these underperformers, the district has tightened the Peer Assistance Review program. Created by the state Legislature in 1999, Peer Assistance Review was originally intended to provide assistance to teachers who were faltering in the classroom. At LAUSD, the program has evolved into a method for weeding out poor performers who, in many cases, are well along in their careers.

The board has also mandated that every teacher have his or her full English-learner certification. The belief is that teachers who have this training are better equipped to reach students whose limited language abilities not only put them at a disadvantage but can impede the ability of the rest of the class to learn. Failure to meet the generous deadlines for obtaining the certificate is grounds for dismissal — as five teachers learned the hard way this past year.