Meditations of an oncology geek

Survivorship Bias

without comments

12 July 2013

Today I’m writing in a slightly less formal style as I share a few thoughts floating through my head, dear reader. I have opted to write them down in hopes that a soul or two might find them enlightening entertaining.

As a young scientist I have had the privilege of learning the fine art of banging my head against a wall in more ways than I could have possibly imagined, all the while not getting smarter, per se, but usually just a little less ignorant of my own ignorance.

In my quest to become less unaware of what I am unaware about, I’ve become fascinated by biases and logical fallacies. After all, we are a group of animals that evolved to hunt, farm, run, or otherwise outfox aging, disease, and each other to reproduce in time to see our genes survive the next generation or two. We did not evolve to intuitively understand permutations compounded by time, like evolution, statistics, or the cosmos. Such things were little use to our ancestors that were more likely to be preoccupied with their next meal and securing mating rights.

When it comes to understanding things in a fourth dimension (time) we are not always so bright. While a dingy might a boat be well-suited for fishing on a small lake, it would be hopefully difficult to cross an ocean in one. That’s not to say it couldn’t be done, but the tool was not developed for the job. Similarly, our minds contain many evolutionary vestiges that helped us get to where we are, but might not help get us where we want to be.

An example of this is Survivorship Bias, or “…concentrating on the people or things that ‘survived’ some process and inadvertently overlooking those that didn’t because of their lack of visibility.” (source)

This morning I came across a fun read by David McRaney in his blog You Are Not So Smart

“Survivorship bias also flash-freezes your brain into a state of ignorance from which you believe success is more common than it truly is and therefore you leap to the conclusion that it also must be easier to obtain. You develop a completely inaccurate assessment of reality thanks to a prejudice that grants the tiny number of survivors the privilege of representing the much larger group to which they originally belonged.”

I could easily draw analogies to the number of graduate students and postdoctoral fellows that truly believe that they will make it to the level of professor. Rarely do we focus on the postdocs that don’t make it and meekly fade away. Or better yet, for the reasons they don’t make it. Perhaps by recognizing our real chances of success, we can change our goals, or change our definition of “success” and avoid more years of grad students pounding their heads into oblivion? But I digress…

McRaney also touched upon an alternate idea of “luck.”

“Wiseman speculated that what we call luck is actually a pattern of behaviors that coincide with a style of understanding and interacting with the events and people you encounter throughout life. Unlucky people are narrowly focused, he observed. They crave security and tend to be more anxious, and instead of wading into the sea of random chance open to what may come, they remain fixated on controlling the situation, on seeking a specific goal. As a result, they miss out on the thousands of opportunities that may float by. Lucky people tend to constantly change routines and seek out new experiences. Wiseman saw that the people who considered themselves lucky, and who then did actually demonstrate luck was on their side over the course of a decade, tended to place themselves into situations where anything could happen more often and thus exposed themselves to more random chance than did unlucky people. The lucky try more things, and fail more often, but when they fail they shrug it off and try something else. Occasionally, things work out.”

In contrast to M.D., Pharm.D., R.N., and *shudder* N.M.D. students, a Ph.D. student does not have the degree of certainty in a defined career path, or security offered by it. But perhaps one could see this as an opportunity for creativity and maneuver oneself into a position to become lucky… To be open to other career paths and other directions… again, I digress…

Something that IS very important to science today is a subset of survival bias known as “publication bias” also touched upon by McRaney:

“The people who use the diet, or the product, or the pill, and fail to lose weight don’t get trotted out for photo shoots – only the successes do. That same phenomenon has become a problem in science publications, especially among the younger sciences like psychology, but it is now under repair. For far too long, studies that fizzled out or showed insignificant results have not been submitted for publication at the same level as studies that end up with positive results, or even worse, they’ve been rejected by prominent journals. Left unchecked, over time you end up with science journals that only present the survivors of the journal process – studies showing significance. Psychologists are calling it the File Drawer Effect. The studies that disprove or weaken the hypotheses of high-profile studies seem to get stuffed in the file drawer, so to speak. Many scientists are pushing for the widespread publication of replication, failure, and insignificance. Only then, they argue, will the science journals and the journalism that reports on them accurately describe the world being explored.”

When I stop to think about the publication process and the biases and innate conflict of interest to produce positive and not negative results, I have to wonder about all of the science that I’ve missed. Or rather, all of the science that was never published because it didn’t work. What else are we missing or not seeing? It’s nearly impossible to know, and for this Homo Sapien it’s not always easy to embrace uncertainty, to consider an idea without accepting it, or to realize that the best lab or study or data might be an illusion casting a disproportionate shadow on our visions or reality, buoyed in our own minds by our survivorship bias.

Ok, time for a second cup of coffee and back to things that don’t make my head hurt as much, like peering down a microscope or finding optimal conditions for frozen tissue antibody conjugation.

Happy Friday!

Written by Ryon

July 12th, 2013 at 11:04 am

Posted in Science Blog

Leave a Reply