The insights from this blog have been drawn from the book Complexity by M. Mitchell Waldrop. This book outlines how a group of diverse Scientific luminaries including several Nobel Laureates developed the Santa Fe Institute to study complexity – how single elements organize themselves into complex structures. This think tank made several revolutionary discoveries that impacted many sciences from biology to economics. Sounds heavy. There were a few key points my small IQ could understand.
Increasing Returns
Why have high tech companies scrambled to locate to Silicon Valley? Why did the VHS system take the market from Beta despite it being technically inferior?
‘increasing returns’ is a mental model that is well known in Biology and Engineering also referred to as positive feedback – like mild tropical winds growing into a hurricane, seeds and embryos growing into fully developed creatures. The neoclassical economics theory assumes the complete opposite that the economy is dominated by negative feedback – the theory of diminishing returns being that twice the fertilizer doesn’t produce twice the crop yield and that the economy finds an equilibrium and harmony.
In some cases, this is very true take for example a business enjoying a large margin with no competitive advantage a competitor is incentivized to enter and share the margin with the incumbent hence finding an equilibrium rather than the incumbent continuing to grow and taking the entire market. Technology however is different. Take for example the QWERTY keyboard, this was invented by Christopher Scholes in 1873 specifically to slow typists down to prevent jamming in typewriters, this technology was then mass produced by the Remington Sewing Machine Company which meant the entire market was trained using this layout, therefore locking this in as the standard forever. Consider the VHS videotape format that by luck gained a slightly larger market share initially which was compounded by the big incentive of video stores wanting to stock only one format and users only wanting one video player which led to VHS dominating the market despite it being inferior to Beta.
Interestingly, in any system there are patterns that are the result of a rich mixture of positive feedback and negative feedback. For example, spilling some water on a polished tray would result in a complex pattern of beads that forms as the result of 3 things:
Negative feedback system of gravity forcing the water flat,
Positive feedback system of surface tension attracting water molecules together, and
LUCK through tiny accidents of history small dust motes and invisible irregularities of the tray
Its not a linear process from adoption to lock in and many feedback systems contribute to it, some of which are unpredictable.
Technology Lock In
An intriguing example of technology lock in is the internal combustion engine. Gasoline powered engines were considered dangerous, less efficient, noisy and gas was hard to obtain in the right grade. Despite all these shortcomings it did eventually win out as the locked in technology arguably due to several lucky events:
In 1895 A gasoline powered Duryea won a Horseless-carriage competition that led to Ransom Old’s 1986 patent and mass production of gasoline vehicles.
In 1914 there was an outbreak of hoof and mouth disease in North America that led to the withdrawal of horse troughs which were the only places where steam cars could fill up with water. At the point in time where gas vehicles were being mass produced.
Despite improvements in technology by the Stanley Brothers condenser and boiler system that didn’t need to be refilled the steam engine never recovered as gasoline power quickly became locked in by gas car owners, gas stations, refineries etc.
Increasing returns isn’t an isolated phenomenon at all: the principle applies to everything in high technology. Take for example Microsoft Windows, The company spent close to $50M in Research and Development to get the first copy out the door. The second copy costs ~$10 in materials, it’s the same story in electronics, computers, pharmaceuticals even aerospace (The cost of the first B2 Bomber was $21 billion, cost of the next copy $500 million). High Technology could almost be defined as “Congealed Knowledge” whereby the marginal cost is virtually zero, this makes every copy that is produced cheaper and cheaper. Furthermore, every additional copy that is produced offers the chance for scale efficiencies and improvements. From a customer perspective there is an equally large reward to flock to a standard. For example, an airline would prefer to own a fleet of the same jet, so pilots and engineers don’t have to be retrained and switch. Another customer example is an office choosing one type of software which ensures support and training is more efficient.
When you compare all the above to standard bulk commodities industries such as grain, coal, or cement. The know how of this production was acquired many generations ago. Today the direct costs are labor, land and raw materials, areas where diminishing returns can set in easily. (farmers producing more grain resulting in less productive land being utilized). These industries tend be described and understood better by standard neoclassical economics of equilibrium vs increasing returns.
Linear vs Non-Linear Systems
The name linear refers to the fact that if you plot a relationship on a graph the plot would be a straight line. A lot of study in science has been focused on linear systems in which the whole is precisely equal to the sum of its parts this implies that each part is free to do its own thing regardless of what else is happening. An awful lot of nature works in a linear fashion such. Sound for example can act in a linear way which is why we can hear and recognize two different instruments. Light is a linear system which is why we can recognize and identify a walk/don’t walk sign in daylight. In some ways an economy is linear in that small economic agents can act independently when someone buys a newspaper at a corner store it has no effect on your decision to buy a tube of toothpaste. However, our brain is certainly non-linear which is why the combined sound of multiple musical instruments which becomes music in your brain is worth more than all of them individually. The economy is also non-linear in that millions of buyers can reinforce each other and create booms and busts.
As outlined earlier many systems share both properties in differing conditions, and quantities. This is the theory of emergence. The point where one system transitions from linear to non-linear relationships. Take water for example there is nothing very complicated about a water molecule its behavior is governed by well understood equations of atomic physics. But now put a few zillion of these molecules together and you suddenly have a substance that shimmers, sloshes and gurgles they have collectively taken on a new property that none of them possessed alone – liquidity. The non-linear systems thinking is perhaps why stock market behavior is so unexplainable when zillions of rules and expectations are introduced, they create a new property as a collective.
Technology networks and change
Conventional economic theory of technological development was that technologies came at random and were made possible by inventors that were exogenous to the economy. Another economic model viewed technology development as purely a commodity of Research and Development, if you spent X you delivered Y as an outcome over time. When you look through economic history as opposed to economic theory technology is not a commodity at all, innovations are like an evolving ecosystem and rarely developed in a vacuum. They are made possible by other innovations already in place for example a laser printer is basically a Xerox machine with a little computer circuitry inside, the former enabled by the two latter technologies, also only possible at scale because of the need for high-speed printing. The technological webs are highly dynamic and unstable and can grow in an organic fashion. For example, laser printers give rise to desktop publishing software and desktop publishing opens the need for graphics programs and more powerful hardware and better peripheral devices and on it goes.
These technological webs undergo bursts of evolutionary growth and mass extinction events just like biological ecosystems. Say a new technology comes along - the automobile and replaces the old technology - the horse, along with the horse goes the blacksmith, watering troughs, stables and so on. The whole subnetwork collapses below but the automobile brings with it paved roads, gas stations, fast food restaurants, motels and new network is developed. Another example of increasing returns, once a technology is embedded the networks are heavily incentivized to help it grow and prosper.
To dislodge an incumbent technology the hurdles are greater than simply the functionality improvement, each technology is different but as an example see below some key elements that are required to be surpassed by sheer new technology functionality:
Adaptive Systems
Complex systems contain clusters of subs systems that interact and as they continue to interact they reinforce one another. Once a cluster becomes stable it moves on to become a building block for a larger cluster and so on. This hierarchy of building blocks transforms the entire systems ability to learn, evolve and adapt. Take the concepts of red, car and road once these building blocks are understood and refined through experience it can be adapted and recombined into many new concepts such as a “red Saab by the side of the road” this is much more efficient than creating something new and starting over from scratch. It is akin to pattern recognition and identifying similar business models to assign to valuations. Instead of moving slowly through the immense space of possibilities step by step an adaptive system reshuffles its building blocks to take giant steps quickly. A great illustration of this concept is the way police artists used to work in the days before computers when constructing a drawing of a suspect to match a witness description the idea was to divide the face up into 10 different sections (Building blocks) each with 10 strips of paper with different options of noses, ears, eyes, forehead etc. These building block decisions were easy to dissect but together they combine into a complex possibility of over 10 Billion different faces, to identify a face from 10 Billion is almost impossible. Using the building block method can describe a great many complicated things with relatively few building blocks.
These building blocks can be informally organized, In the cognitive realm anything we call a “Skill” or “expertise” is an implicit model – or more precisely a huge interlocking set of standard operating procedures (building blocks) that have been inscribed on the nervous system and refined by years of experience. The best example of this is medieval architects who created gothic cathedrals, they didn’t have the tools of modern physics or structural engineering they learned standard operating procedures from rules of thumb passed down from master to apprentice many of these structures are still standing thousands of years later. This system of learning receives environmental feedback, this was Darwin’s great insight. If a model survives in the environment, then it improves through natural selection and this steady improvement is called evolution.
Learning Classifier Systems
A learning classifier system is a rule-based machine learning method that combines a discovery component with a learning component. Classifier systems seek to identify a set of context dependent rules that collectively store and apply knowledge in a piecewise manner to make predictions.
In 1978 an early classifier system was tested to learn how to run a simulated maze and ended up being 10 times faster than the incumbent method. This classifier system also exhibited transfer which is that it could apply learnings from previous mazes to the current one. From this development further applications were built in Poker and Chess to test their effectiveness they worked with great success. The most impressive application of the Classifier system was applied in 1982 to solve the pipeline problem. Running a Gas pipeline is a complex problem that involves hundreds of compressors pumping gas through thousands of miles of large diameter pipe all while customers demands change hourly and pipes and compressors spring leaks continually. Safety constraints demand that pressure remain at certain levels and everything effects everything else in the system. Pipeline operators learn their craft through long apprenticeships and then drive their system with “feel” and “instinct”. The classifier system learned to operate a simulated pipeline beautifully, starting from a set of totally random classifiers it achieved expert-level performance in about 1000 days of experience. The most insightful outcome of the classifier system in the pipeline problem was the way the system organized knowledge about leaks it developed rules and then adapted the rules when the facts changed. The three key principles that underpin the classifier system are:
Knowledge can be expressed in terms of mental structures that behave very much like rules;
Rules are always in competition so that experience causes useful rules to grow stronger and unhelpful rules to grow weaker
Plausible new rules are generated from combining old rules
These three rules create a hierarchy of the structure of human knowledge.
Artificial Life
Imagine a machine that floats around a pond filled with lots of parts, this machine is a constructor and given a description of itself it could construct another machine once it locates the proper parts. That sounds like re-production; however, it isn’t. The newly created copy of the machine won’t have a description of itself which means it won’t be able to make any further copies. Therefore, to achieve true re-production the original machine also requires a description copier, a device that will take the original description, duplicate it, and then attach the duplicate description to the offspring machine. Once that happens the offspring will have everything required to re-produce. To restate this in a more formal way, the genetic material of any self-reproducing system natural or artificial must play two fundamentally different roles:
A program/algorithm that can be executed during the construction of the offspring, and
Serve as a passive data, a description that can be duplicated and given to the offspring to pass on.
This theory was espoused by John Von Neumann in the late 1940’s before Watson and Crick discovered the double helix in 1953. This turned out to be an incredible prediction considering that what we went on discover about DNA and cell division process.
Corporate culture is the DNA of a business and the ability for a company to truly replicate its business model to grow requires a) the algorithm of the business model and b) the passive data (Culture) to pass on to the employees that will become the next generation within the business. Without firstly the corporate culture of beliefs, myths, rules and ways of doing things nothing can be passed on at all, secondly that DNA map needs to be implanted in the expanded operations strongly.
The Artificial Stock Market
One of the most difficult chestnuts to crack in economic theory is stock market behavior. Considering that neoclassical theory considers that all economic agents are perfectly rational then all investors must be perfectly rational. Moreover, since everyone shares the same information, they will always agree about what every stock is worth – simply the net present value of its future cash flows discounted by the interest rate. Therefore, this perfectly rational market will never get caught up in bubbles or busts. Therefore the New York Stock Exchange trading floor would be a quiet place, in reality of course the NYSE is a barely controlled riot, with multiple bubbles and crashes, fear, uncertainty, euphoria and every combination. A Martian that read the Wall Street journal might think that the “market” was a living organism that had moods of being jittery, depressed, upbeat, and confident.
If you replace the above assumptions of perfectly rational agents and replace them with Artificial intelligence agents that can learn, adapt, and classify rules they will act more like a human does. The Santa Fe institute modeled this change with one stock that the agents could buy and sell and as they learned rules for trading you could observe what rules they developed and which ones were reinforced and prioritized by different participants. The model began with a stock that had fundamental value of $30. The agents were then introduced to the market with total stupidity, random rules were developed from experience, as expected they learned energetically. On the second run of this model, it was observed that some the agents had developed a primitive form of technical analysis reinforced by positive outcomes from random rules which further reinforced more positive outcomes for new agents further reinforcing these rules even further which ran the price up to $34. In different models the exact opposite happened, and the price fell to $25 as agents tried to continue selling reinforcing further selling.
The fascinating reality is that the market is combination of participants that have rules developed and reinforced by individual experiences that drive different expectations of outcomes. Monetary reward is a powerful incentive to overcome especially when reinforced with consecutive positive outcomes, hence as markets rise they can continue to rise for a long time.
I hope you enjoyed the read.
Thanks,
Shaun Trewin