042: 🧠 How Near Misses and Failures Lead to Success

Oh hello there,

This is the 42th edition of Making Connections, where we take a random (illustrated) walk down tech, fitness, product thinking, org design, nerd culture, persuasion, and behavior change.

Like what you see? Why not bring someone to the party?

Your email address

Please check your inbox and click the link to confirm your subscription.

Please enter a valid email address!

An error occurred, please try again later.

🖼 Morning Larks vs Night Owls

Do you have a morning lark / night owl face off in your household? I’m not a morning lark but my wife is definitely a night owl without a set schedule (no morning meeting to wake up for) so this pain is very real.

🧠 How Near Misses and Failures Lead to Success

In Silicon Valley, there exists the ideas that failure is badge of honor or that it’s better to back second-time founders who won’t make the same mistakes of first-time ones. These ideas are predicated on the notion that we learn from our mistakes and failure leads to reflection, insight, and future success. But where’s the proof?

I’ve enjoyed the work of Dashun Wang, a professor at Northwestern’s Kellogg School of Business, who studies these kinds of questions and has two papers that look at this - one on learning from failure, and the other from early near-misses in one’s career.

In a new paper published in the 150th anniversary issue of Nature, Wang and colleagues developed a mathematical model to pinpoint what separates those who succeed from those who merely try, try again. Wang found that success comes down to learning from one’s prior mistakes—for instance, continuing to improve the parts of an invention that aren’t working rather than scrapping them, or recognizing which sections of a denied application to keep and which to rewrite.

In this paper, they studied 3 different data sets: 776k NIH grants, 58k startups who received venture funding, and 170k terrorist attacks. I find the choice of studying terrorist attacks a bit morbid but also genius in that it’s a completely different field and one that’s probably less well understood by most college-educated people compared to the other two.

So the researchers looked at the first attempt and the penultimate attempt (the one right before a win) for each aspiring scientist, entrepreneur, and terrorist in their dataset. To measure improvement (or lack thereof) over time, the researchers looked at changes in how the scientists’ grant applications were rated, the amount of venture funding the startups received, and the number of individuals wounded in terrorists’ attacks.

Analysis revealed that the chance theory doesn’t hold up. In all three datasets, an individual’s second-to-last attempt did tend have a higher probability of success than their very first effort.

The first they they disproved was the idea that success was purely luck. They found that people tended to do better in later attempts (if they were ultimately successful) than in their first ones. This suggest they learned or somehow got better at what they were doing. I don’t think all of it can be chalked up to “learning” per se - you might get access to more resources. E.g. a local warlord funds your 3rd terrorist attack after you gain notoriety, etc.

This model assumes that every attempt has several components—like the introduction and budget sections of a grant proposal, for instance, or the location and tactics used in a terrorist attack. Importantly, even if an attempt fails overall, some of its components may still have been good. When mounting a new attempt, an individual has to choose, for each component, whether to go back to the drawing board or to improve upon a version from a prior (failed) attempt.

The researchers found that the learning ability of a given entrepreneur, scientist, or terrorist can be discerned by simply measuring how much time passes between their first few attempts. As a result, their model was able to accurately predict which entrepreneurs, scientists, and terrorists would eventually succeed long before any outward signs of success appear.

In this case, they basically use the idea of “reusing the good parts” of previous attempts as a proxy of learning and measure that by seeing how quickly they try again. Presumably if you try again soon after, you have reused components. Starting over would take longer and therefore meant you didn’t really learn anything from the previous attempt.

Again, not 100% sold on this idea of learning being the only reason you can move faster, but the idea that you build momentum and speed make sense. If you can move faster with each following attempt, however that’s happening, it probably means you’re on the right track. And on the other hand, the “throw things at the wall and see what sticks” strategy probably won’t work after the first few tries.

—

NIH’s evaluation process also made these a good type of grant to study. When a researcher submits a grant application to the NIH, it is reviewed by a panel of experts and assigned a numerical score. Then, depending on how much funding is available, the NIH determines a cutoff point—say, applications that scored in the top 15th percentile are funded, and the rest are not.

For the authors, this meant it was easy to determine which grants fell just short of receiving funding (they called these “near miss” grants) and which managed to squeak past the cutoff point (they called these “narrow wins”).

In this second paper they focused on biomedical researchers getting a specific type of grant from the NIH known as an R01. They’re one of the oldest grants (going back to 1990) and are important in one’s early career, so those who get it presumably have a real leg up over those who don’t.

To figure out just how much of a difference these early successes or setbacks made to a scientific career, the researchers traced the careers of 623 near-miss and 561 narrow-win scientists.

Notably, it turned out that the two groups published at similar rates over the next 10 years—not what you’d expect, given that narrow winners got an early leg up from their NIH grant funding. Even more surprising, scientists in the near-miss group were actually more likely to have “hit” papers (that is, papers that cracked the top-five percent of citations in a particular field and year). In the five years after they applied for NIH funding, 16.1 percent of papers produced by scientists in the near-miss group were hits, compared to 13.3 percent for the narrow-win group.

So in the long run, the near-miss researchers actually beat the near-win people! But why?

They tested a variety of hypotheses, which included that 12% of the near-miss people would leave the field, so those who remained were stronger. But even removing the weakest near-win researchers wasn’t enough to account for the difference.

They also looked at whether the near-miss people pivoted to “hotter topics”, which there was some evidence for, but not enough to explain the difference.

With all of these alternative explanations ruled out, the team was left to conclude that failure itself might be the cause of the performance gap between the near-miss and narrow-win groups.

In other words, with no clear external factor that can explain the disappointed scientists’ success, it’s reasonable to think that the experience of adversity made them better in the end—confirming the conventional wisdom that “what doesn’t kill you makes you stronger.”

I find this final explanation a bit flimsy and inconclusive, especially since they didn’t do any kind of qualitative interviewing, but I think even the fact that a near-miss can lead to greater success in the long run is inspiring.

To be honest, sometimes I feel my own entrepreneurial journey of getting into Y Combinator on my first try with a kind of absurd idea put me in the “near-win” category to my detriment. But that’s a story for another day.

👉 Self-Compassion Journaling

Continuing with theme from last week, where I shared a practice from the Search Inside Yourself program I’m currently doing at work, here’s another one. Research has found that self-compassion, treating yourself with kindness and understanding can make you more motivated to study for a tough test after failure, more willing to make amends and change after doing something morally wrong, and more willing to change a personal weakness.

We did a quick journaling exercise where we responded to the following prompt for 5 minutes:

Write a letter to yourself from the perspective of a close friend or loved one expressing compassion and understanding about your current challenges

This might seem self-indulgent but I’m willing to bet you’re going through some real stress and hardship in your life right now (we’re still in a pandemic!) and this exercise can help you see your challenges with greater perspective.

Alright that’s it for this week. Talk to you soon!

-J

👨🏻‍💻 About Me

Jason Shen is an entrepreneur and business leader passionate about technology and human resilience. His past startups have reimagined transportation, recruiting, and gaming; backed by notable investors at Y Combinator, Techstars, and Amazon. As an operator, he’s built products and led teams at companies like Facebook, Etsy, and the Smithsonian.

Jason has written about productivity, resilience as well as the future of work in publications like Fast Company, VOX, TechCrunch and has spoken at events at TED, Google and The White House where his ideas have reached millions. He lives in Brooklyn with his wife, two kettlebells, and many piles of books.