EconTech

Ok, they get PAM wrong and that is disappointing, to say the least. However, I have to agree with the substance of some of the criticism of Hanson, though NOT the claims of hyperbole (again, SA are ad hominem attacks really necessary. Let your arguments speak for themselves please).

First, some criticism against SA. The idea of a futarchy is fascinating. Suggesting such an idea be discussed and further developed is far from insane. It is still a sub-type of democracy; so, it isn’t even particularly anti-democratic. It helps to clarify the line between value and fact based judgments, something which many, e.g. Rodrik, indicate as being one of the major pitfalls of modern policy analysis. Of course, I agree. This is not utopian thinking. It is confronting the fact that we don’t really know how to coordinate the various experts and relevant public opinion to make an optimal policy decision. These decisions are getting rather large and important rather fast and figuring out a way of achieving this end is a most pressing question in meta-policy analysis.

Now, you may know that I am not a huge proponent of prediction markets. There are two major reasons for this. One is that I agree with Hanson that social scientists know a lot. There is strong evidence that that knowledge can be put to making quite accurate predictions in a number of arenas. Thus, my position is that if all you have to resort to is some type of polling, then prediction markets will, on net, give you better predictions. However, if the models are giving us better predictions than the prediction markets, then there is some predictive capacity being lost in the market. Of course, there are lots of models and which one is the best is more often than not an open question. The expected response is to let the prediction market decide. But this gets us back to the original point, that predictive capacity is being lost.

The second, and more fundamental, reason to not be a fan of prediction markets is that, usually, we don’t just want a prediction, but we want a reason behind that prediction. Markets can only get us the former. The causes can be guessed at and analyzed and traders polled, but that just brings us to more model building and guesswork.

That prediction markets are “more accurate” begs the question: relative to what? Polls are not known for their predictive capacity in the first place; so being more accurate to them is a bit of a straw man. Polling a group of experts tends to produce the same kind of problem. These polls are usually “what do you think off the top of your head” type of polls. Experts are not experts because they immediately know the answer. They are experts at breaking down the problem into analyzable chunks and presenting their thoughts on that. If those carefully considered analyses are polled, I suspect the results will be quite different. This is not the same thing as polling every study in existence. Besides the obvious biases in quite a few, some exist to push the barriers of research and to test hypotheses. In some fields, up to a third of these are found to have flawed conclusions. But the peer review process and further modeling is what reveals these mistakes. Could prediction markets do as well?

Colorful Character Again

I just learned of a new Scientific American article on prediction markets, which is pretty positive:

A paper … compares the performance of the IEM as a predictor of presidential elections from 1988 to 2004 with 964 polls over that same period and shows that the market was closer to the outcome of an election 74 percent of the time. … Attracted by the markets’ apparent soothsaying powers, companies such as Hewlett-Packard (HP), Google and Microsoft have established internal markets that allow employees to trade on the prospect of meeting a quarterly sales goal or a deadline for release of a new software product. As in other types of prediction markets, traders frequently seem to do better than the internal forecasts do. … Prediction markets may truly hark back to the future. “My long-run prediction is that newspapers in 2020 will look like newspapers in 1920,” Wharton School’s Wolfers says. If that happens, the wisdom of crowds will have arrived at a juncture that truly rivals the musings of the most seasoned pundits.

But I am personally singled out as the colorful character who is way too positive:

The ardor for market-based answers can at times border on the hyperbolic. Robin Hanson, a professor of economics at George Mason University, has advocated that if trading patterns on prediction markets suggest that implementation of a particular policy will cause the economy to grow and unemployment to shrink, then policy officials should, by fiat, adopt that policy - an interest rate cut or a public works project, perhaps. Hanson reasons that the collective information held by traders is superior to the analyses that can be marshaled by a panel of economists or other experts. Hanson has even proposed a form of government called futarchy, based on policy-making markets.

So even if markets are consistently more accurate it is “hyperbolic ardor” to suggest we actually follow their advice? The article then repeats old errors about DARPA’s Policy Analysis Market (PAM):

Such utopian leanings have sometimes led advocates to push too far too fast. Several years ago the Defense Advanced Research Projects Agency (DARPA) began planning for a project called the Policy Analysis Market, which would have allowed investors to trade on geopolitical events, not unlike the Intrade Iran contract, including assassinations, wars and the next al-Qaeda attack. If the market - for which Hanson was an adviser - bid up a contract that would pay off if a terrorist attack occurred, the Department of Homeland Security might then decide to raise the threat condition status from yellow to red. Or so went the rationale.

No, PAM was not intended to warn us about individual terrorist attacks! PAM was intended to forecast geopolitical trends - two Senators claimed otherwise based on a small miscellaneous section of a sample web page, but out of hundreds of articles on PAM it has been years since a journalist repeated this error. As someone who grew up reading Scientific American, it is sad to see their standards sink so low.