Following the publication of Jill Lepore’s “The Disruption Machine” and Clayton Christensen’s vigorous response in an interview there has been a healthy debate around the merits and even existence of disruption in many posts and tweets. I had been busy preparing for and then teaching a computer bootcamp for our children and some friends so I had mostly ignored this debate. It is of course highly relevant to my claim that we are at the beginning of a massive transition from industrial to information society.

I was particularly intrigued by Nassim Taleb’s tweeted claim that

“Disruption” has to be BS as it is mathematically incompatible with the Lindy Effect. The 2 cannot coexist.

The Lindy Effect states that for many types of objects but especially for technology and ideas the best predictor of their expected future lifespan is how long they have already been around. I believe he pinpoints the crux of the debate, although he is wrong on a critical detail which is the power of non-linear change (something he certainly appreciates as it permeates his books including the fantastic Antifragile).

The reason a lot of technology persists is because in many areas change has only been incremental. We continue to use glasses, tables, silverware, etc. because these are technologies for which change has been linear/incremental (at best). But we drive cars to work instead of riding horses because cars represent a non-linear change over horses. In fact, horses turn out to have been eclipsed by non-linear technology changes in agriculture, warfare and transportation which is why we have gone from having 26 million working horses in the US in 1916 to a few thousand now.

Nowhere have we seen more exponential (ie highly non-linear) change than in information technology. We no longer use an Abacus, or a slide rule, or a mechanical tabulator, or a room filling mainframe, or a million dollar workstation because the phone in our pockets has more compute power than all of these! These technologies were disrupted by the exponentially better ones that came after them. And the Internet has been an exponential change for the size and scale of networks, which is why there are now networks that hundreds of millions and even billions of people participating in them. That too has and will continue to disrupt existing smaller networks and hierarchies.

Yet even in the world of information technology and networks the word disruption is used all too frequently and for things that aren’t in fact disruptive. Craigslist represented a non-linear change for the classified ad business. It took ads that cost tens or even hundreds of dollars and reached thousands of people and replaced it with ads that were free and reached millions. That’s a highly non-linear change and it resulted in a disruption of the newspaper industry. New web sites that are offering prettier listings, more functionality, etc. are not disrupting Craigslist as much as they tend to be closer to incremental improvements. Hence the persistence (much to many technologist’s consternation) of Craigslist.

So yes. The Lindy Effect is real. Disruption is rare. But it does exist and it is caused by non-linear changes in technology. We happen to be in a period of such non-linear change because we have figured out how to use computers for lots and lots of things, including machine learning and robotics, DNA analysis and synthesis, scanning and additive manufacturing of objects. As a result the exponential improvements in computers powered by Moore’s and Metcalfe’s "laws" are invading many other industries.