Predictin’ Ain’t Easy
January 20th, 2006

Ray Kurzweil made some suggestive meta-predictive statements in a recent interview:

How are we able to make reliable predictions about the overall future of these technologies when each individual project is very unpredictable? But we see the same thing in other areas of science. Take thermodynamics. It’s impossible to predict the path of a single molecule in the air, because it follows a random unpredictable path, and that’s true of all of the particles. Yet, the overall properties of the gas, made up of all of these unpredictable particles is very predictable according to the laws of thermodynamics.

How this metaphor pertains to prediction markets and collective intelligence is best considered on a case-by-case basis. In some instances such as Galton’s ox weight guessing contest, we can see how individual errors and biases tend to cancel each other out as the sought-after information is distilled in some aggregate measure of belief. Likewise, when crowds are asked to predict the action of large groups, as with election markets, they have a reasonably good record. Now, it is somewhat well-known that prediction markets are less accurate when they are asked about the actions of a small group of agents, as with SCOTUS nominations, jury verdicts and other conclave-like decisions. The thermodynamics metaphor works here too but in reverse — the small group of decision-makers are difficult to predict. Also, when crowd members have similar biases or lose their independence, this causes a problem, as biases may wildly exaggerate themselves.

Leaving the metaphor aside, in most cases it is more useful to think of all of this as a result of information and the degree of its dispersion, but as Nassim Taleb points out in a preview of his planned book The Scandal of Prediction, there is little practical difference in human affairs between ontological and epistemic unknowns (between truly random outcomes and those for which we simply lack the information to make a correct prediction). At some point over at my blog, I will have more on Taleb’s critique of prediction as it pertains to markets that go by that name. One of Taleb’s main targets is overconfidence, especially as it relates to expert prediction. The Kurzweil interview clearly had a defensive tone towards this sort of view, and it is easy to understand why.

Fatal error: Call to undefined function sociable_html() in /home/permutype/ on line 36