D. Kahneman, O. Sibony, C. R. Sunstein: Noise: A Flaw in Human Judgment

For those familiar with some other works of Daniel Kahneman, Olivier Sibony or Cass R. Sunstein, the new book Noise: A Flaw in Human Judgment certainly does not come as a surprise. It is rather a continuation of their previous articles and books on judgment and decision making, only that this time in the spotlight is what they call „noise“, an „undesirable variability in judgments of the same problem“ (p. 36).
Whether „noise“ will reach the same prominence as other concepts popularized by the authors such as „nudge“, different types of cognitive biases, etc. only time can say, but what they try to make clear is that noise is ubiquitous („wherever there is judgment, there is noise“ – p. 12, italics in original) and highly consequential.


Structure

Just as bias was underpinned in Thinking, Fast and Slow by countless examples from different disciplines, the same approach is used to indicate the presence of noise in human judgment: From medicine, to forensic science, to hiring decisions, to sentencing decisions, etc. the authors make sure to offer compelling evidence throughout the book.

The book is divided in 6 parts, plus Introduction: Two Kinds of Error, Review and Conclusion: Taking Noise Seriously, Epilogue: A Less Noisy World and 3 appendixes of practical importance: How to Conduct a Noise Audit, A Checklist for a Decision Observer and Correcting Predictions. Before briefly summarizing the content of the book, it should be clear how the authors define and use the word judgment, namely „the mental activity of making a judgment and […] its product“ (p. 40).

Summary

Part I: Finding Noise

The first part, Finding Noise, looks at noise in the American criminal justice system and tells the story of Marvin Frankel who in the 1970s raised awareness on the huge disparities in sentencing people though the offenses committed were similar. It is the first example of costly „system noise“, as the decisions of judges of similar cases are expected to be identical (at least in an ideal world).

Professionals in the private sector can also take inconsistent decisions and the example provided here is that of an insurance company, where the authors conducted a so-called „noise audit“ („In a noise audit, multiple individuals judge the same problems. Noise is the variability of these judgments.“ – p. 221). A company would expect its employees (with similar experience and training) to deal with the same case in a similar way, but what the noise audit revealed, were again considerable differences.

Part II: Your Mind Is a Measuring Instrument

The main message of part two, Your Mind Is a Measuring Instrument, is that both bias and noise contribute to judgmental errors and reducing them is a worthwhile goal, if one aims for accurate judgments. The authors introduce in this part a classification of system noise, which contains level noise („variability in the average level of judgments by different judges“ – p. 78) and pattern noise („variability in judges’ responses to particular cases“ – p. 78).

While the examples illustrating level and pattern noise are about errors in the judicial system, one can easily replace the word „judge“ with any other decision-making person. Level noise and pattern noise are elements impacting the evaluation of a situation, due to some characteristics of the evaluators: some are more optimistic, some are more pessimistic, some are more severe, etc. (level noise), while some feel strongly about certain subjects and this again influences their evaluation (pattern noise).

In addition, there is also a more subtle type of noise, occasion noise, which also affects judgment. For instance, the moment of the day, events which occurred lately (ones favorite soccer team just lost a match), the weather, stress, etc. all can affect our judgments even without knowing it. Occasion noise can be amplified within groups due to social pressure and social influence (p. 102-103). Group polarization is also an interesting phenomenon: „when people speak with one another, they often end up at a more extreme point in line with their original inclinations“ (p. 103).

Part III: Noise in Predictive Judgments

Part three, Noise in Predictive Judgments, deals with the (lack of) accuracy of human judgment compared with the accuracy reached by formulas and artificial intelligence, and the authors come to the same conclusion as in Thinking, Fast and Slow: „In predictive judgments, human experts are easily outperformed by simple formulas – models of reality, models of a judge, or even randomly generated models“ (p. 122). Supported by many examples, this conclusion should be worrisome, but people nevertheless turn to their intuition, gut feeling and the like. One explanation for this and for the aversion towards algorithms is that judgment completion in people is accompanied by an „internal signal“ (p. 138), „a pleasing sense of coherence, in which the evidence considered and the judgment reached feel right“ (p. 138) and this feeling seems hard to replace, in spite of the improvements of judgment brought by algorithmic prediction methods (p. 146).

The idea of narrative fallacy which was explored also in Thinking, Fast and Slow is taken up again in order to address the differences between causal and statistical thinking. Moreover, human inclination towards causal thinking is seen as one of the reasons why noise is overlooked as a source of error (p. 158).

Part IV: How Noise Happens

Part four, How Noise Happens, revisits some operations of the fast System 1 thinking (for those who read Thinking, Fast and Slow this certainly rings a bell) such as anchoring, substitution, availability, jumping to conclusions, matching, etc. as they produce both systematic errors and noise. The authors underline that „blaming every undesirable outcome on biases is a worthless explanation“ (p. 164) and therefore, we should no longer neglect noise’s weight in erroneous judgments.

Part V: Improving Judgments

The longest part of the book, Improving Judgments, contains advice and strategies for noise reduction under the label „decision hygiene“ (p. 243). Statistical literacy (p. 238) is praised throughout the book, but considering that it is difficult to achieve, other tools and techniques needing less effort are recommended. Introducing a decision observer (p. 240), being open-minded – defined as „the humility of those who are constantly aware that their judgment is a work in progress and who yearn to be corrected“ (p. 234) –, introducing (simple) guidelines and rules, such as the Apgar score, sequencing information and keeping in mind that „[m]ore information is not always better“ (p. 256), breaking up a complicate question / decision / evaluation into smaller parts, as illustrated in the extensive chapter on the „mediating assessments protocol“ (p. 312), etc. are some of the suggestions for reducing noise. The need for introducing and maintaining „decision hygiene“ is illustrated by spectacular studies and stories from forensic science, medicine and human resource management (hiring and performance evaluation of employees).

Part VI: Optimal Noise

The last part of the book, Optimal Noise, looks at noise as a component which sometimes can and sometimes cannot be eliminated. It discusses seven arguments for why reducing or eliminating noise might not be a good idea. A cost-benefit analysis should be conducted, the strategies for noise reduction should be kept under observation, evaluated and revisited, etc. but noise is often described as a characteristic of us being humans and not machines or as the authors put it „Noise is mostly a by-product of our uniqueness […]“ (p. 216).

Review and Conclusion: Taking Noise Seriously

The section Review and Conclusion: Taking Noise Seriously brings together some of the most important ideas of the book in a concise way. Among them are the six principles of decision hygiene (the following bullet points cite the titles of the subsections describing the principles – pp. 371-374, italicized and bold in original):

  • „The goal of judgment is accuracy, not individual expression.“
  • „Think statistically, and take the outside view of the case.“
  • „Structure judgments into several independent tasks.“
  • „Resist premature intuitions.“
  • „Obtain independent judgments from multiple judges, then consider aggregating those judgments.“
  • „Favor relative judgments and relative scales.“

A last point worth mentioning here is the comparison of decision hygiene measures with handwashing: „Handwashing does not prevent all diseases. Likewise, decision hygiene will not prevent all mistakes. It will not make every decision brilliant. But like handwashing, it addresses an invisible yet pervasive and damaging problem“ (p. 324).

Discussion

For all those interested in how the human mind works, how we evaluate, forecast, form opinions and judgments, Noise: A Flaw in Human Judgment is a book worth reading. Written in a smooth style and full of captivating studies and stories from various domains, Noise completes the picture of how not only bias, but also noise lead to erroneous judgments.

However, the reader should not expect a groundbreaking book, as it reiterates many of the ideas of Kahneman’s previous book, as well as some ideas which can be found in some earlier articles such as Before You Make That Big Decision (from 2011, by Daniel Kahneman, Dan Lovallo, and Olivier Sibony) or Noise: How to Overcome the High, Hidden Cost of Inconsistent Decision Making (from 2016, by Daniel Kahneman, Andrew M. Rosenfield, Linnea Gandhi, and Tom Blaser).

The book can be read also as an extensive literature review covering topics from many domains (see the „Notes“ at the end of the book containing some of the sources used by the authors). Those interested in getting indications and tools for how they can fight noise will find the appendixes at the end of the book most certainly useful. The authors themselves recommend readers „interested in practical applications of noise reduction“ (p. 9) to skip some of the parts of the book and to read part 5 where the idea of „decision hygiene“ or the „mediating assessments protocol“ are presented.

When reading Noise I often had the feeling that the book follows the same successful „recipe“ as Thinking, Fast and Slow, yet, I couldn’t read it with the same enthusiasm: I perceived it sometimes as being too ambitious, sometimes as being repetitive and sometimes as superficial.


Bibliographical Information

Daniel Kahneman, Olivier Sibony, Cass R. Sunstein
Noise: A Flaw in Human Judgment

2021; ix, 454 pages; Paperback
Little, Brown Spark
ISBN: 978-0-316-26665-9


About the Authors

Daniel Kahneman

Daniel Kahneman was born in Tel Aviv in 1934. He obtained his first degree in psychology and mathematics from the Hebrew University in Jerusalem. He wrote his dissertation in 1961 at the University of California in Berkeley.

In 2002 Kahneman was awarded the Nobel Prize in Economic Sciences

for having integrated insights from psychological research into economic science, especially concerning human judgment and decision-making under uncertainty

https://www.nobelprize.org/prizes/economic-sciences/2002/summary/

Source: The information about the author is taken from Kahneman’s biography on the Nobel Prize site (Access date 13.04.2020).

Olivier Sibony

Olivier Sibony holds a PhD from Université Paris-Dauphine obtained in 2017 and since 2021 he is a professor (education track) at HEC Paris. He worked 25 years as a consultant at McKinsey and Company and he is a specialist in Consumer Packaged Goods and Luxury Goods among others. He is also a successful author and co-author bringing on the market You’re About to Make a Terrible Mistake (2020) and Cracked It! How to Solve Big Problems and Sell Solutions Like Top Strategy Consultants (2018, with Bernard Garrette and Corey Phelps).

Sources: Information taken from the presentation on his own website and the website of HEC Paris (Access date: 11.12.2021).

Cass R. Sunstein

Cass R. Sunstein obtained his Juris Doctor in 1978 from Harvard University and is a renowned law professor and author of a wealth of books and articles. He is currently the Robert Walmsley University Professor at Harvard University, where he is also the director of the Program on Behavioral Economics and Public Policy. Between 2009 and 2012 he worked as an Administrator of the White House Office of Information and Regulatory Affairs. He is a member of the American Academy of Arts and Sciences and a receiver of many awards: Goldsmith Book Award (in 1994) for his book Democracy and the Problem of Free Speech, Regulatory Innovation Award (in 2012), the Holberg Prize (in 2018), etc.

Sources: Information taken from his presentation page at the Harward Kennedy School and his presentation page at the Harward Law School, where Sunstein’s CV is also available (Access date: 14.12.21).


Folgende Rezensionen könnten Sie ebenfalls interessieren:

Iuliana Ancuţa Ilie

Iuliana Ancuţa Ilie is a Research Assistant in the areas Intercultural Management and Management Accounting at Pforzheim University. With an interdisciplinary background in literary studies and international business administration she publishes in academic journals and books on subjects such as cross-cultural management, international human research management and diversity management.

Schreibe einen Kommentar