Book Summary: Fooled by Randomness - Nassim Nicholas Taleb
Back to BookshelfNote: These are notes I’ve highlighted in the book, that I can go back to for re-reading. This is not a complete summary of the book.
Randomness, (chance, luck) influences our lives and work more than we realize. Mild success can be explained by skills and labor. Wild success is attributable to variance. Because of hindsight bias, survivorship bias – we forget people who failed, remember only the few who succeed. And, make up reasons and patterns for their success even though it was largely random.
Prologue
At the cost of appearing biased, I have to say that the literary mind can be intentionally prone to the confusion between noise and meaning, that is, between a randomly constructed arrangement and a precisely intended message.
Our mind is not equipped with the adequate machinery to handle probabilities; such infirmity even strikes the expert, sometimes just the expert.
They believe in reason and rationality—that we should overcome cultural impediments on our way to becoming a better human race—thinking we can control our nature at will and transform it by mere edict in order to attain, among other things, happiness and rationality. Basically this category would include those who think that the cure for obesity is to inform people that they should be healthy.
On the other hand there is the Tragic Vision of humankind that believes in the existence of inherent limitations and flaws in the way we think and act and requires an acknowledgment of this fact as a basis for any individual and collective action. This category of people includes Karl Popper (falsificationism and distrust of intellectual “answers,” actually of anyone who is confident that he knows anything with certainty), Friedrich Hayek and Milton Friedman (suspicion of governments), Adam Smith (intention of man), Herbert Simon (bounded rationality), Amos Tversky and Daniel Kahneman (heuristics and biases), the speculator George Soros, etc.
I am convinced of that after spending almost all my adult and professional years in a fierce fight between my brain (not Fooled by Randomness ) and my emotions (completely Fooled by Randomness ) in which the only success I’ve had is in going around my emotions rather than rationalizing them. Perhaps ridding ourselves of our humanity is not in the works; we need wily tricks, not some grandiose moralizing help.
PART I
What came with the help of luck could be taken away by luck (and often rapidly and unexpectedly at that). The flipside, which deserves to be considered as well (in fact it is even more of our concern), is that things that come with little help from luck are more resistant to randomness.
Clearly, the quality of a decision cannot be solely judged based on its outcome, but such a point seems to be voiced only by people who fail (those who succeed attribute their success to the quality of their decision)
Realism can be punishing. Probabilistic skepticism is worse. It is difficult to go about life wearing probabilistic glasses, as one starts seeing fools of randomness all around, in a variety of situations—obdurate in their perceptional illusion.
did not judge his heroes by the result: Heroes won and lost battles in a manner that was totally independent of their own valor; their fate depended upon totally external forces, generally the explicit agency of the scheming gods (not devoid of nepotism).
Heroes are heroes because they are heroic in behavior, not because they won or lost.
researchers of the brain believe that mathematical truths make little sense to our mind, particularly when it comes to the examination of random outcomes.
I have anecdotal evidence in my business that MBAs tend to blow up in financial markets, as they are trained to simplify matters a couple of steps beyond their requirement.
As a derivatives trader I noticed that people do not like to insure against something abstract; the risk that merits their attention is always something vivid.
It is a fact that our brain tends to go for superficial clues when it comes to risk and probability, these clues being largely determined by what emotions they elicit or the ease with which they come to mind
both risk detection and risk avoidance are not mediated in the “thinking” part of the brain but largely in the emotional one (the “risk as feelings” theory). The consequences are not trivial: It means that rational thinking has little, very little, to do with risk avoidance. Much of what rational thinking seems to do is rationalize one’s actions by fitting some logic to them.
This is one of the many reasons that journalism may be the greatest plague we face today—as the world becomes more and more complicated and our minds are trained for more and more simplification.
The invisible histories have a scientific name, alternative sample paths, a name borrowed from the field of mathematics of probability called stochastic processes.
Several branches of research have been examining our inability to learn from our own reactions to past events: Every man believes himself to be quite different, a matter that amplifies the “why me?” shock upon a diagnosis.
A mistake is not something to be determined after the fact, but in the light of the information until that point.
Unlike many “hard” sciences, history cannot lend itself to experimentation
We will examine the dubious value of the highly frequent news with a more technical discussion of signal filtering and observation frequency farther down. If there is anything better than noise in the mass of “urgent” news pounding us, it would be like a needle in a haystack.
Robert Shiller. Not just in financial markets; but overall his 1981 paper may be the first mathematically formulated introspection on the manner in which society in general handles information. Shiller made his mark with his 1981 paper on the volatility of markets, where he determined that if a stock price is the estimated value of “something” (say the discounted cash flows from a corporation), then market prices are way too volatile in relation to tangible manifestations of that “something” (he used dividends as proxy). Prices swing more than the fundamentals they are supposed to reflect, they visibly overreact by being too high at times (when their price overshoots the good news or when they go up without any marked reason) or too low at others. The volatility differential between prices and information meant that something about “rational expectation” did not work. (Prices did not rationally reflect the long-term value of securities and were overshooting in either direction.) Markets had to be wrong. Shiller then pronounced markets to be not as efficient as established by financial theory (efficient markets meant, in a nutshell, that prices should adapt to all available information in such a way as to be totally unpredictable to us humans and prevent people from deriving profits). This conclusion set off calls by the religious orders of high finance for the destruction of the infidel who committed such apostasy.
I would suggest London’s Anatole Kaletsky and New York’s Jim Grant and Alan Abelson as the underrated representatives of such a class among financial journalists; Gary Stix among scientific journalists)
The same methodology can explain why the news (the high scale) is full of noise and why history (the low scale) is largely stripped of it (though fraught with interpretation problems).
why people who look too closely at randomness burn out, their emotions drained by the series of pangs they experience
Regardless of what people claim, a negative pang is not offset by a positive one (some psychologists estimate the negative effect for an average loss to be up to 2.5 the magnitude of a positive one); it will lead to an emotional deficit.
We do not need to be rational and scientific when it comes to the details of our daily life—only in those that can harm us and threaten our survival. Modern life seems to invite us to do the exact opposite; become extremely realistic and intellectual when it comes to such matters as religion and personal behavior, yet as irrational as possible when it comes to matters ruled by randomness (say, portfolio or real estate investments).
This does not happen too often with dentists or pianists—because these professions are more immune to randomness.
Darwinian ideas are about reproductive fitness, not about survival
Darwinian fitness applies to species developing over a very long time, not observed over a short term—time aggregation eliminates much of the effects of randomness; things (I read noise ) balance out over the long run, as people say.
Owing to the abrupt rare events, we do not live in a world where things “converge” continuously toward betterment. Nor do things in life move continuously at all.
We discovered that, in the very small, particles jump (discretely) between states; they do not slide between them.
Whenever there is asymmetry in outcomes, the average survival has nothing to do with the median survival.
Why do they confuse probability and expectation, that is, probability and probability times the payoff? Mainly because much of people’s schooling comes from examples in symmetric environments, like a coin toss, where such a difference does not matter. In fact, the so-called bell curve that seems to have found universal use in society is entirely symmetric.
The best description of my lifelong business in the market is “skewed bets,” that is, I try to benefit from rare events, events that do not tend to repeat themselves frequently, but, accordingly, present a large payoff when they occur. I try to make money infrequently, as infrequently as possible, simply because I believe that rare events are not fairly valued, and that the rarer the event, the more undervalued it will be in price.
I associate rare events with any misunderstanding of the risks derived from a narrow interpretation of past time series.
We are human and act according to our knowledge, which integrates past data. If rational traders detect a pattern of stocks rising on Mondays, then, immediately such a pattern becomes detectable, it would be ironed out by people buying on Friday in anticipation of such an effect. There is no point searching for patterns that are available to everyone with a brokerage account; once detected, they would be self-canceling.
Pseudoscience came with a collection of idealistic nerds who tried to create a tailor-made society, the epitome of which is the central planner.
No amount of observations of white swans can allow the inference that all swans are white, but the observation of a single black swan is sufficient to refute that conclusion
For instance, a conventional-wisdom, empirical style statement like automobile accidents happen closer to home can be tested by taking the average distance between the accident and the domicile of the driver (if, say, about 20% of accidents happen within a twelve-mile radius). However, one needs to be careful in the interpretation; a naive reader of the result would tell you that you are more likely to have an accident if you drive in your neighborhood than if you did so in remote places, which is a typical example of naive empiricism. Why? Because accidents may happen closer to home simply because people spend their time driving close to home (if people spend 20% of their time driving in a twelve-mile radius).*
I can use data to disprove a proposition, never to prove one. I can use history to refute a conjecture, never to affirm it. For instance, the statement The market never goes down 20% in a given three-month period can be tested but is completely meaningless.
You can more safely use the data to reject than to confirm hypotheses. Why? Consider the following statements: Statement A: No swan is black, because I looked at four thousand swans and found none. Statement B: Not all swans are white.
The reality does not have the same closed and symmetric laws and regulations as games
To conclude, extreme empiricism, competitiveness, and an absence of logical structure to one’s inference can be a quite explosive combination.
Science is mere speculation, mere formulation of conjecture
we like to emit logical and rational ideas but we do not necessarily enjoy this execution.
PART II
The major problem with inference in general is that those whose profession is to derive conclusions from data often fall into the trap faster and more confidently than others.
A result is that in real life, the larger the deviation from the norm, the larger the probability of it coming from luck rather than skills:
Remember that nobody accepts randomness in his own success, only his failure. His ego was pumped up as he was heading up a department of “great traders” who were then temporarily making a fortune in the markets and attributing the idea to the soundness of their business, their insights, or their intelligence.
An outstanding recent paper by Sullivan, Timmerman, and White goes further and considers that the rules that may be in use successfully today may be the result of a survivorship bias.
Accordingly, rules have their value. We just follow them not because they are the best but because they are useful and they save time and effort.
Since the Kahneman and Tversky results, an entire discipline called behavioral finance and economics has flourished.
A normative science (clearly a self-contradictory concept) offers prescriptive teachings; it studies how things should be. Some economists, for example those of the efficient-market religion, believe that our studies should be based on the hypothesis that humans are rational and act rationally because it is the best thing for them to do (it is mathematically “optimal”). The opposite is a positive science, which is based on how people actually are observed to behave. In spite of economists’ envy of physicists, physics is an inherently positive science while economics, particularly microeconomics and financial economics, is predominantly a normative one. Normative economics is like religion without the aesthetics
System 1 is effortless, automatic, associative, rapid, parallel process, opaque (i.e., we are not aware of using it), emotional, concrete, specific, social, and personalized. System 2 is effortful, controlled, deductive, slow, serial, self-aware, neutral, abstract, sets, asocial, and depersonalized.
The mathematician and cognitive scientist David Marr, who pioneered the field of object recognition, made the apt remark that one does not learn how birds fly by studying feathers but rather by studying aerodynamics. I will present the theses of two watershed works presented in readable books, Damasio’s Descartes’ Error and LeDoux’s Emotional Brain.
The implication is that we feel emotions (limbic brain) then find an explanation (neocortex). As we saw with Claparède’s discovery, much of the opinions and assessments that we have concerning risks may be the simple result of emotions.
Deborah Bennett’s Randomness
A test of a disease presents a rate of 5% false positives. The disease strikes 1/1,000 of the population. People are tested at random, regardless of whether they are suspected of having the disease. A patient’s test is positive. What is the probability of the patient being stricken with the disease? Most doctors answered 95%, simply taking into account the fact that the test has a 95% accuracy rate. The answer is the conditional probability that the patient is sick and the test shows it—close to 2%. Less than one in five professionals got it right. I will simplify the answer (using the frequency approach). Assume no false negatives. Consider that out of 1,000 patients who are administered the test, one will be expected to be afflicted with the disease. Out of a population of the remaining 999 healthy patients, the test will identify about 50 with the disease (it is 95% accurate). The correct answer should be that the probability of being afflicted with the disease for someone selected at random who presented a positive test is the following ratio: Number of afflicted persons / Number of true and false positives here 1 in 51.
I noticed that very few option traders can maintain what I call a “long volatility” position, namely a position that will most likely lose a small quantity of money at expiration, but is expected to make money in the long run because of occasional spurts.
They most commonly get mixed up between absence of evidence and evidence of absence, Say hospital A delivered 52% boys and hospital B delivered the same year only 48%; would you try to give the explanation that you had a boy because it was delivered in hospital A?
PART III
The epiphany I had in my career in randomness came when I understood that I was not intelligent enough, nor strong enough, to even try to fight my emotions. Besides, I believe that I need my emotions to formulate my ideas and get the energy to execute them.
I am fully aware that such anger is self-destructive and offers no benefit, and that if I were to develop anger for every idiot around me doing something of the sort, I would be long dead
Without tricks I would not escape the toxicity of the information age.
I had this idea of stripping people of language while. The idea came to me that perhaps I could use a built-in bias, here prejudice, to offset another built-in bias, our predisposition to take information seriously. It seems to work.
Our bias is immediately to establish a causal link between two independent events.
It is very hard for us to just shut up. We are not cut out for it. Popper or not, we take things too seriously.
Most of us know pretty much how we should behave. It is the execution that is the problem, not the absence of knowledge. I am tired of the moralizing slow-thinkers who pound me with platitudes like I should floss daily, eat my regular apple, and visit the gym outside of the New Year’s resolution. In the markets the recommendation would be to ignore the noise component in the performance. We need tricks to get us there but before that we need to accept the fact that we are mere animals in need of lower forms of tricks, not lectures
Self-contradiction is made culturally to be shameful, a matter that can prove disastrous in science.
What characterizes real speculators like Soros from the rest is that their activities are devoid of path dependence. They are totally free from their past actions. Every day is a clean slate.
There is no rational reason to keep a painting you would not buy at its current market rate—only an emotional investment.
People confuse science and scientists. Science is great, but individual scientists are dangerous. They are human; they are marred by the biases humans have. Perhaps even more.
Having control over randomness can be expressed in the manner in which one acts in the small and the large. Recall that epic heroes were judged by their actions, not by the results.
Postscript
A mild degree of unpredictability in your behavior can help you to protect yourself in situations of conflict. Say you always have the same threshold of reactions. You take a set level of abuse, say seventeen insulting remarks per week, before getting into a rage and punching the eighteenth offender in the nose.