Serendipity and strategy in rapid innovation

Abstract: Innovation is to organizations what evolution is to organisms: it is how organisations adapt to changes in the environment and improve. Governments, institutions and firms that innovate are more likely to prosper and stand the test of time; those that fail to do so fall behind their competitors and succumb to market and environmental change. Yet despite steady advances in our understanding of evolution, what drives innovation remains elusive. On the one hand, organizations invest heavily in systematic strategies to drive innovation. On the other, historical analysis and individual experience suggest that serendipity plays a significant role in the discovery process. To unify these two perspectives, we analyzed the mathematics of innovation as a search process for viable designs across a universe of building blocks. We then tested our insights using historical data from language, gastronomy and technology. By measuring the number of makeable designs as we acquire more components, we observed that the relative usefulness of different components is not fixed, but cross each other over time. When these crossovers are unanticipated, they appear to be the result of serendipity. But when we can predict crossovers ahead of time, they offer an opportunity to strategically increase the growth of our product space. Thus we find that the serendipitous and strategic visions of innovation can be viewed as different manifestations of the same thing: the changing importance of component building blocks over time.