Emergic: Rajesh Jain's Blog

Emergic: Rajesh Jain's Blog header image 2

TECH TALK: Black Swan: Mediocristan and Extremistan

June 7th, 2007 · 1 Comment

The ideas behind Mediocristan and Extremistan in Nassim Talebs The BlackSwan are worth exploring in more depth. Chetan Parikh has reproduced a table from the book which explains the differences between Mediocristan and Extremistan.

The Portfolio wrote:

N.N.T., who lives in New York and has taught at the University of Massachusetts at Amherst, previously traded derivatives on Wall Street. The academics who drive him to tears are the ones who have explainedor misexplainedhis old profession. They think that markets are from Mediocristan when in fact they inhabit Extremistan.

Say what? Mediocristan is the terrain of the ordinary, the part of the world that conforms to the bell curve. It answers to statistics and knowable probabilities. Height resides in Mediocristan. You may find one 7-footer on your block, almost certainly not two. Experience (and biology) enable us to frame the odds. Weight is also from Mediocristan. Pick any 1,000 people and their average weight will be close to that of the general population (even if you include the worlds fattest person). Personal wealth, however, is from Extremistan. For instance, the average wealth of 1,000 people will be very different if one of those people is Bill Gates.

This distinction is potent. In Extremistan, past events are a faulty guide to projecting the future. Gates may be the worlds richest person, but it isnt unthinkable that someday, someone (at Google, perhaps?) will be twice as rich. Wars also reside in Extremistan. Prior to World War II, the planet had never experienced a conflict as terrible. Then we did. Suppose you frequent a pond. Day after day you see swansalways white. Naturally (but incorrectly) you presume that all swans are white. World War II was a black swanhorrific and unpredictable.

The Financial Times added:

Taleb claims that there are too many extreme events in securities markets for such markets to be located in Mediocristan. The black swan of October 1987, when the Dow Jones index fell by about 20 per cent, was the first trigger for his personal reassessment. The event was simply outside the realms of possibility in classical statistics. Taleb would first substitute power laws and the mathematics of extreme statistics for the reassurance of normal distributions. But this still gives more credence to economists and financial analysts than he allows. Probabilities can be defined and predictions made only if the events that are the subject of the probabilities and predictions can be described. Donald Rumsfeld distinguished known unknowns and unknown unknowns. Statistics, old and new, deal with known unknowns. Talebs world is determined by unknown unknowns – black swans.

No one, he says, could have predicted the invention of the wheel or measured the probability that the wheel would be invented, because if you could do either of these things you would already have invented the wheel. The invention of the wheel was a black swan.

Arlene Goldbard went further:

Taleb argues convincingly that we treat far too much of our reality as if it were Mediocristan when in fact much of it often behaves like Extremistan, where there are occasional black swans (his name for the unexpected event and the title of his most recent book) among the white. So, for example, out of the many thousands of books, films and recordings released each year, a small number will account for the largest part of sales, and it is not possible to predict with certainty which of the many works released will find black swan-style success (or failure). Indeed, in any endeavor susceptible to notable, unpredictable exceptions, no amount of examining the past will enable us to foretell the future.

Whats going on here? Taleb discusses many factors contributing to our tendency to see our world as Mediocristan. There is the fact that our brains evolved long ago to deal with a world with many fewer variables, much less organized information, and a vastly smaller number of theories to explain them. The more complex any given situation, the larger number of examples you need to understand what is happening there. For instance, sampling the sales of a few dozen published books each year wont tell you much about the prospects of the thousands of others not sampled. Its just as likely as not that your sample would include one or more black swansunexpectedly huge winners or losersso anything you might conclude based on it would not be generalizable to the rest.

Tomorrow: More Reviews


TECH TALK Black Swan+T

Tags: Tech Talk

1 response so far ↓

  • 1 Peter Cotton // Feb 11, 2012 at 8:19 pm

    The Extremistan/Mediocristan distinction was laid before us in The Black Swan and can only be interpreted as meta-guidance for applied mathematicians. Actually it was meta-nonsense for people who had never applied mathematics, never intended to apply mathematics, but wanted to think deep thoughts about other people who applied mathematics … but let’s set that aside and examine it on it’s merits.

    Taleb presented a perfectly clear instruction manual. The world has linear and non-linear phenomena. But mathematicians have to be careful, he argued, not to use the wrong kind of mathematics. You wouldn’t want to use linear mathematics to model a non-linear problem best treated with fractals, power laws and so forth now, would you? And I suppose you wouldn’t want to use fractals to model a linear system either – though Taleb is highly skeptical that any exist so that is a decidedly less important topic.

    Take web page popularity. It follows a Zipf distribution, landing it squarely in the groovy world of power laws. So it would be a massive mistake to try to apply linear mathematics to it, right? To me more precise I’d say it would be a massive mistake, a violation of a universal principal no less, to apply Linear Algebra to it, much less a singular value decomposition which is about as close to the epicenter of “linear mathematics” (to humor that ridiculous phrase) as one could surely get. Strange then, that Larry Page sought fit to do so when inventing Page Rank, the algorithm distinguishing Google from Yahoo that powered the greatest commercial success in recent history.

Leave a Comment