The tyranny of averages

I have never taken a statistics course. In chemistry, one of my professors suggested that if you really needed statistics, the experiment didn’t work. We analytical chemists love our numbers, but these numbers are primarily viewed in the realm of precision and accuracy, and not with respect to what the numbers otherwise mean. For us, ‘p’ is something we do and not something we report.  

In biology and politics, I find numbers to be more annoying each day, especially numbers that are not useful for anything other than supporting a point of view. Many of them seem to be taken out of context, if not totally imagined. Others are based on experiments that can’t possibly be repeated. Almost all of them beg further inquiry. Examples include the percent of university students who complete their degree in four years; how about a bit more granularity? What subjects were studied? What class of university? Did students catch fire with a new major and actually want to learn more? The claim that a college education is associated with higher lifetime earnings is clearly specious for elementary school teachers. There are many professions where earnings are not the primary motivator or metric for success. We’ve all seen quoted figures on declining unemployment, with the counter issues of those who took on part-time work or dropped out of the workforce. The dogma over the last year has been that 20 percent of college coeds have been sexually assaulted. That’s amazing. Who knew? Does anyone?

Press releases tell us about average differences in progression-free survival in oncology trials without even suggesting a range for each arm in the trial. Even if there is no difference on average, something very significant may have happened. Using a metaphor, two streams may average a meter deep, but for only one of them would you drown while crossing with lead boots. At my age, we are expected to be interested in the average lifespan, but that’s not my N=1 concern. More recently we’ve seen some movement toward looking at the outliers with a good or bad average trial result. There are nice examples in cancer trials, where a few subjects were off the curve on the positive side. Some of us see opportunity there with respect to personalized medicine. In a related example, we ask not only why some subsets of the population pass away at below-average ages, but now also consider the exceptional longevity of the 'welderly' (well elderly). Is this a result of genotype, phenotype or (more likely) a complex mix? We finally have good tools and have just begun using them..

Each year in the spring semester I ask my drug development class to confidentially estimate the percent of healthcare costs related to prescription drugs. The population is too small to be representative, but a third of these students are premeds. This year their answers ranged from 20 percent to 85 percent of a $3-trillion healthcare spend in 2014. For decades, the actual number has continued to hover around 10 percent, as was again the case last year. I suspect (but do not know) that the extraordinarily high cost of the newest targeted therapeutics is balanced by the commoditized generic drug pricing. The rest of healthcare expenses have grown even faster than drugs to help steady the 10-percent metric. So here again we may have selected a most favorable relative number vs. the absolute truth; that’s called selection bias. It’s not to place blame, but to think more deeply. What is very costly today and provides a necessary return on innovation will be generic tomorrow. That’s how it must work to keep innovation a sustainable activity. In life sciences we’ve not found the equivalent of Moore’s Law for semiconductors. That’s a challenge we’ve tried to meet with genomics, proteomics, high-throughput screening, combichem and adaptive trials and so much more. Those haven’t worked as well as had been hoped. Let’s keep trying. Unlike for semiconductors, biology fights back.

Perhaps second only to lawyers, statisticians have been the butt of many jokes. The comedian Steven Wright noted that “42.7 percent of all statistics are made up on the spot.” Statistician George Box noted that “essentially, all [statistical] models are wrong, but some are useful.” Mark Twain was less kind, but made the same point when suggesting that “all generalizations are false, including this one.”  

I don’t like going to the doctor for a “physical exam” when what I get is more accurately a “physical chemistry exam” with my numbers compared with means and ranges for others I’ve never met. We will get better and better at comparing data to my own baseline and noting trends over time. In the State of the Union address a month ago, the president suggested a “Precision Medicine Initiative” that would “give all of us access to the personalized information we need to keep ourselves and our families healthier.” That’s a vote against the tyranny of averages and for the value of individual genotypic and phenotypic data, which are increasingly affordable, recordable and accessible. Bring on the initiative and let the chemistry talk. Make sure the geneticists don’t stay in charge, because phenotype provides more meat for the N=1 decision to select pharmacology that works.

Peter T. Kissinger is professor of chemistry at Purdue University, chairman emeritus of BASi and a director of Chembio Diagnostics, Phlebotics and Prosolia.