Welcome to the stochastic age

In 1990, Kleiner Perkins rejected 99.4% of the proposals it received, while investing in 12 new companies a year. Those investees made Kleiner Perkins “the most successful financial institution in the history of the world,” boasting “returns of about 40 percent per year, compounded, for coming up on thirty years.”

Nowadays, the Valley’s VC poster child is Y Combinator, who invest in more like 250 companies annually. They’re famously selective, accepting something like 1.5% of applicants, but still noticeably less selective than Kleiner Perkins in its heyday. They invest less money (though not necessarily that much less; KP bought 25% of Netscape for a mere $5 million back in 1994) in more companies.

In 1995, three networks controlled essentially all American television, and made only enough to fill the week; nowadays there is so much TV that you could bingewatch a new scripted series every day of the year. In 1995, the top ten movies of the year were responsible for 14% of the total box office. So far in 2018, the top ten have claimed a full 25% of the total gross. Something similar happened in publishing; the so-called “midlist” was largely replaced by a “bestseller or bust” attitude.

In 1995, if you were a journalist, your readership was dictated almost entirely by who published you. No matter how compelling your piece in the Halifax Daily News may have been, the same number of people would glance at your headline as at the others in that issue, and that number would be drastically smaller than that of any article, no matter how buried, in the New York Times. Now, the relative readership of any article, both between and within publications, is determined mostly by social media sharing, and inevitably follows a power-law curve, such that a surprisingly small number of pieces attract the lion’s share of readers.

What do these fields have in common? The number of “hits” has remained relatively constant, while their value has grown, and the number of “swings” has grown to the point where it is difficult for any person, or even any group, to pay close attention to them all. And the outcomes inevitably follow a power law. So it doesn’t make sense to focus on individual outcomes any more; instead you focus on cohorts, and you think stochastically.

“Stochastic” means “randomly determined,” and your initial inclination may be to recoil — of course producers and investors and publishers aren’t acting randomly! They put enormous amounts of analysis, effort, and intelligence into what they do! Which is true. But I put it to you that as gatekeepers’ power has diminished, and the number of would-be directors, CEOs, and pundits has skyrocketed, while the costs of trying have shrunk — randomness has become a more and more important factor.

It’s easy to cite anecdotes. What if Excite had bought Google when it was offered to them for $1 million? How far were we, really, from a world in which Picplz succeeded and Instagram failed? Any honest success story will include elements of luck, which is, in this context, another word for randomness. My contention is that the world’s larger trends — greater interconnectedness, faster speed, democratized access to technology — make randomness an ever-more-important factor.

This is not automatically a good thing. People talk about “stochastic terrorism,” a.k.a. “The use of mass public communication, usually against a particular individual or group, which incites or inspires acts of terrorism which are statistically probable but happen seemingly at random.” Think of killers who dedicate their attack to ISIS after the fact despite no previous communication with them, or, more generally and contentiously, political violence promoted by broadcasting hatred and extremism.

And it seems that climate change is increasingly a stochastic disaster. Warmer weather means more energy in the atmosphere, which means more volatile behavior, which means more catastrophes like droughts, wildfires, hurricanes. Does climate change cause those things? Not directly. It increases the probability of them happening. It means both more and bigger hits, if you will.

This doesn’t apply to every field of human endeavor. But it seems to apply to essentially every field driven by unusual, extreme successes or failures — to Extremistan, to use Nassim Taleb’s term. Extremistan seems to everywhere be growing more extreme, and there’s no end in sight.