January 2005

Sassy stats can beguile, so tackle numbers with strong context

By Matt Baron

Of the thousands upon thousands of stories I’ve written, and the tens of thousands more that I’ve read, a troubling number have been disturbing. And an even more daunting number have included at least one statistic.

But only one has ever begun with the words “a disturbing statistic,” let alone had the audacity to use the phrase in connection with a college football game. Brace yourself for the lead in its entirety, as reported by the Associated Press in late October 2004:

“A disturbing statistic emerged from Nebraska’s film study yesterday of its 45-21 loss at Kansas State: 33 missed tackles. Defensive coordinator Kevin Cosgrove says if those tackles had been made, Kansas State would have had only about half of the 418 yards it amassed against the Huskers.”

Beyond the obvious point—that Nebraska Cornhusker football just ain’t what it used to be—there are at least five “Go Figure” lessons lurking in that paragraph. To make sure we don’t repeat the Nebraska D’s glaring deficiency, in a journalistic sense, let’s tackle those lessons one at a time.

1. If you’re going to start a story with the words “a disturbing statistic,” be ready to back up the hackneyed phrase with something truly disturbing. Whether the 33 missed tackles qualifies as “truly disturbing” is debatable, and will be addressed further in a later point.

2. Rather than using “disturbing” to modify “statistic,” come up with something a little fresher—as in something that hasn’t been used about a half-million times already. And, please, refrain from “daunting” and “troubling”—those references in the lead were most decidedly tongue-in-cheek.

3. Be willing to take a chance with a new slant. Rather than trot out the same old, second-day story statistics that you might find in the wake of any football game, the AP writer heard the missed-tackles figure from the team’s defensive coordinator and ventured into new ground with the off-the-beaten path angle.

So kudos to the writer for trying something new.

4. If you have the guts to apply Lesson 3, have the presence of mind to supply some context, too. In the football story, that was lacking on a few fronts.

For one thing, there is no clarification of what constitutes a missed tackle. How close would a defender need to be in order to be considered a failed would-be tackler? Does the stat count only those defenders who get at least a hand on the ball carrier?

And on another contextual note, let’s circle back to the use of “disturbing statistic” in the first place. Are 33 missed tackles really “disturbing”? Or are they just about average? Maybe it’s an “encouraging statistic” because the team had been missing 47 tackles a game and the conference average is 39 misses. Without any point of reference, we are adrift.

5. When you see a statistic, ask it: Are you objective or subjective?

Within the football story’s context, the 33 missed tackles—and therefore the assertion that “about half” of the yards gained came after those misses—are clearly subjective measurements.

A memory device for such data is to call them Subjectively Arrived-at Statistics (SAS). Extend the acronym to “sassy,” which means improperly forward or bold, and you get an even more vivid picture of the hazards they pose.

By contrast, the “418 yards amassed” by Kansas State is an example of an objective statistic—it’s a figure that multiple sources can agree upon based on observation.

Of course, the ideal story incorporates rock-solid objective data with well-reported, context-filled figures that nobody else has even thought to provide.

Give Yourself A Break

Any journalist interested in gaining a firmer grip on using numbers ought to read “Give Me A Break,” by John Stossel, co-host of ABC’s 20/20 program. The book provides a fascinating account into Stossel’s journalistic odyssey.

Of particular interest is his uphill battle against skeptical producers to report numbers in a responsible way. In a special that ranked various dangers, called “Are We Scaring Ourselves to Death?”, Stossel reports the top danger isn’t any chemical or multisyllabic disease. What do you think he found to be the No. 1 danger? (Answer at end of column.)

Killing a Myth

It is an oft-told tale: a loved one fights off death longer than anyone predicted, and observers credit the perseverance to a yearning to reach that next birthday or to see family members for one last Christmas.

In Sports Illustrated’s wonderful lead story picking the Boston Red Sox as Sportsmen of the Year, loyal fan George Sumner, 79, fights cancer long enough to see his team win the World Series.

So do people, in general, have the ability to stiff-arm death long enough to get to that next milestone? Heart-warming, anecdotal stories notwithstanding, the answer appears to be “no.”

In a recent study published in the Journal of the American Medical Association, researchers who dug through more than 300,000 cancer deaths in Ohio between 1989 and 2000 found no trend toward such a phenomenon.

Kudos to study co-author Donn Young, a statistician and research scientist at Ohio State University’s Comprehensive Cancer Center. He pursued the analysis after finding he was the only naysayer in a story touting patients’ ability to delay their deaths.

Young said “wishful thinking and selective memory”—those enemies of objective statistics—are partly to blame for the widespread, but errant, notion that people have the ability to put off their deaths as special occasions approach.

Answer: Poverty.