Education and Statistics

Recently The Notebook published a piece by Dale Mezzacappa that illustrates the misuse of educational statistics.

The data show that "about a quarter of students initially referred for expulsion by their schools since August 2009 were not ultimately expelled by the School Reform Commission," and that since April "more than one-third of the cases brought to the SRC did not result in expulsion."

That is all the data show. The 1/4 number comes from a 13 month period. the 1/3 number comes from April to current. Over the 13 months, the exact number is 76 out of 324 students not expelled. The author presents no n for the 1/3 number.

Using these numbers, the author goes on to assert that, "still, the reversals point to the possibility that schools are recommending students for expulsion without learning enough about the circumstances of the incident, or that the District’s zero tolerance discipline policy leaves too little discretion to school administrators and hearing officers in the earliest stages of the process."

Unfortunately, the numbers do not point to that at all. In fact, the numbers really do not show anything.

1) There is no baseline measurement for a previous 13-month period. How do we know if 1/4 is high, low, or the same?
2) There is no baseline measurement for April-October (the 1/3 number). How do we know if 1/3 is high, low, or the same?
3) It is impossible to compare April, 2010-October, 2010 to August, 2009-October, 2010 (the 1/4 to 1/3 "increase"). How do we know that this "increase" is not due to external effects, such as the end of school and the ensuing beginning of the school year?
4) There is no n presented for the April-current number. How do we know if there were 3 or 100 expulsion cases?

When a commenter at the bottom pointed this out, Paul Socolar explained that no baseline exists and then asserted that "the suggestion that expulsion data need to be compared to expulsions from the same month or time of year in a previous year might make sense if expulsion cases were handled in a timely manner..." That makes little sense, and the commenter responded yet again with the point that, if the study was done on a longer term, periodic or cyclical effects could be assumed in both set and comparisons could be made.

However, no one responded to that. Why? Probably because it is right. Furthermore, it is right and essentially undoes the supposedly statistical support that launched the premise of the article. Unfortunately, that premise was valid and important, but once again, sloppy misuse of data undermine a legitimate point.