Few educational reforms have been as lionized or demonized as charter schools. A new study that shows the schools performing well behind traditional schools has triggered more discussion and rhetoric, but done little to answer the most lingering questions: Precisely what role do charter schools play in American public education, and how will we know if they are playing it well?

The study shows that, based on performance on the National Assessment of Educational Progress, students in charter schools scored significantly lower in reading and math than students in traditional public schools. When performance was broken down by race, ethnicity and income, most students in the subgroups still did better in traditional public schools than in charter schools. In Ohio, 43 percent of the state's 148 charter schools rated academic emergency or watch - the two lowest rankings - on the recently released state report cards. Another 36 schools were not rated. Last year some schools simply did not report data.

Critics of charter schools say these findings show that charter schools are failing at their primary task, which is raising academic achievement. It's true that for years, this country has placed heavy weight on the NAEP findings, using them annually to rate - and often berate - our public schools. But there is still room to discuss whether academic progress is the only measure of charter schools' success. Could a student simply find a better, more satisfying fit in a non-traditional school? Or is it hypocritical to measure traditional public schools only on academic performance while allowing charter schools to be measured in other ways? These fundamental questions about charter schools have gone unanswered too long.

It's also time to clean up their reporting practices and sharpen their accountability. The gaps between what we suppose and what we know about charter schools show up all the time. A case in point: Defenders of charter schools have rushed to say the results of the NAEP study are skewed because charter schools serve the "toughest" students. Yes, charter schools open in struggling districts. But there is no data to show that the most challenging students in those districts enroll in charter schools. A more likely case can be made that children whose parents shop for alternative schools have more home support than other students.

The absence of objective data to show who attends charter schools, why they enroll and how they're performing when they enter is an inexcusable flaw in this reform effort. Without it, no one can measure the progress of individual students or of charter schools as a whole. Meanwhile, children are being put through transitions - almost always detrimental to learning - and tax dollars are flowing while no one can say definitively if this experiment is working.

There is an urgent need to define purpose, set expectations and then measure them with integrity. The charter school experiment is too important - and too costly - to be hung on conjecture.