The Cult Of Statistical Significance

How the most important statistical method used in many of the sciences doesn't pass the test for basic common sense

“McCloskey and Ziliak have been pushing this very elementary, very correct, very important argument through several articles over several years and for reasons I cannot fathom it is still resisted. If it takes a book to get it across, I hope this book will do it. It ought to.” —Thomas Schelling, Distinguished University Professor, School of Public Policy, University of Maryland, and 2005 Nobel Prize Laureate in Economics “With humor, insight, piercing logic and a nod to history, Ziliak and McCloskey show how economists—and other scientists—suffer from a mass delusion about statistical analysis. The quest for statistical significance that pervades science today is a deeply flawed substitute for thoughtful analysis. . . . Yet few participants in the scientific bureaucracy have been willing to admit what Ziliak and McCloskey make clear: the emperor has no clothes.” —Kenneth Rothman, Professor of Epidemiology, Boston University School of Health The Cult of Statistical Significance shows, field by field, how “statistical significance,” a technique that dominates many sciences, has been a huge mistake. The authors find that researchers in a broad spectrum of fields, from agronomy to zoology, employ “testing” that doesn’t test and “estimating” that doesn’t estimate. The facts will startle the outside reader: how could a group of brilliant scientists wander so far from scientific magnitudes? This study will encourage scientists who want to know how to get the statistical sciences back on track and fulfill their quantitative promise. The book shows for the first time how wide the disaster is, and how bad for science, and it traces the problem to its historical, sociological, and philosophical roots. Stephen T. Ziliak is the author or editor of many articles and two books. He currently lives in Chicago, where he is Professor of Economics at Roosevelt University. Deirdre N. McCloskey, Distinguished Professor of Economics, History, English, and Communication at the University of Illinois at Chicago, is the author of twenty books and three hundred scholarly articles. She has held Guggenheim and National Humanities Fellowships. She is best known for How to Be Human* Though an Economist (University of Michigan Press, 2000) and her most recent book, The Bourgeois Virtues: Ethics for an Age of Commerce (2006).

This book presents a philosophical approach to probability and probabilistic thinking, considering the underpinnings of probabilistic reasoning and modeling, which effectively underlie everything in data science. The ultimate goal is to call into question many standard tenets and lay the philosophical and probabilistic groundwork and infrastructure for statistical modeling. It is the first book devoted to the philosophy of data aimed at working scientists and calls for a new consideration in the practice of probability and statistics to eliminate what has been referred to as the "Cult of Statistical Significance." The book explains the philosophy of these ideas and not the mathematics, though there are a handful of mathematical examples. The topics are logically laid out, starting with basic philosophy as related to probability, statistics, and science, and stepping through the key probabilistic ideas and concepts, and ending with statistical models. Its jargon-free approach asserts that standard methods, such as out-of-the-box regression, cannot help in discovering cause. This new way of looking at uncertainty ties together disparate fields — probability, physics, biology, the “soft” sciences, computer science — because each aims at discovering cause (of effects). It broadens the understanding beyond frequentist and Bayesian methods to propose a Third Way of modeling.

A witty and thoughtful romp through the profession and practice of economics

“McCloskey and Ziliak have been pushing this very elementary, very correct, very important argument through several articles over several years and for reasons I cannot fathom it is still resisted. If it takes a book to get it across, I hope this book will do it. It ought to.” —Thomas Schelling, Distinguished University Professor, School of Public Policy, University of Maryland, and 2005 Nobel Prize Laureate in Economics “With humor, insight, piercing logic and a nod to history, Ziliak and McCloskey show how economists—and other scientists—suffer from a mass delusion about statistical analysis. The quest for statistical significance that pervades science today is a deeply flawed substitute for thoughtful analysis. . . . Yet few participants in the scientific bureaucracy have been willing to admit what Ziliak and McCloskey make clear: the emperor has no clothes.” —Kenneth Rothman, Professor of Epidemiology, Boston University School of Health The Cult of Statistical Significance shows, field by field, how “statistical significance,” a technique that dominates many sciences, has been a huge mistake. The authors find that researchers in a broad spectrum of fields, from agronomy to zoology, employ “testing” that doesn’t test and “estimating” that doesn’t estimate. The facts will startle the outside reader: how could a group of brilliant scientists wander so far from scientific magnitudes? This study will encourage scientists who want to know how to get the statistical sciences back on track and fulfill their quantitative promise. The book shows for the first time how wide the disaster is, and how bad for science, and it traces the problem to its historical, sociological, and philosophical roots. Stephen T. Ziliak is the author or editor of many articles and two books. He currently lives in Chicago, where he is Professor of Economics at Roosevelt University. Deirdre N. McCloskey, Distinguished Professor of Economics, History, English, and Communication at the University of Illinois at Chicago, is the author of twenty books and three hundred scholarly articles. She has held Guggenheim and National Humanities Fellowships. She is best known for How to Be Human* Though an Economist (University of Michigan Press, 2000) and her most recent book, The Bourgeois Virtues: Ethics for an Age of Commerce (2006).

This book describes the classical axiomatic theories of decision under uncertainty, as well as critiques thereof and alternative theories. It focuses on the meaning of probability, discussing some definitions and surveying their scope of applicability. The behavioral definition of subjective probability serves as a way to present the classical theories, culminating in Savage's theorem. The limitations of this result as a definition of probability lead to two directions - first, similar behavioral definitions of more general theories, such as non-additive probabilities and multiple priors, and second, cognitive derivations based on case-based techniques.

There are a number of important questions associated with statistical experiments: when does one given experiment yield more information than another; how can we measure the difference in information; how fast does information accumulate by repeating the experiment? The means of answering such questions has emerged from the work of Wald, Blackwell, LeCam and others and is based on the ideas of risk and deficiency. The present work which is devoted to the various methods of comparing statistical experiments, is essentially self-contained, requiring only some background in measure theory and functional analysis. Chapters introducing statistical experiments and the necessary convex analysis begin the book and are followed by others on game theory, decision theory and vector lattices. The notion of deficiency, which measures the difference in information between two experiments, is then introduced. The relation between it and other concepts, such as sufficiency, randomisation, distance, ordering, equivalence, completeness and convergence are explored. This is a comprehensive treatment of the subject and will be an essential reference for mathematical statisticians.

In this witty, accessible, and revealing book, Deirdre McCloskey demystifies economic theory and practice to show that behind the economists claim to certainty is the ancient art of storytelling. If You're So Smart will engage, enlighten, and empower anyone trying to evaluate the experts who stand ready to engineer our lives. "Writing with delicious wit and great seriousness."—Publishers Weekly. " "McCloskey is more interesting on an uninspired day than most of her peers can manage at their very best."—Peter Passell, New York Times

Tomas Sedlacek has shaken the study of economics as few ever have. Named one of the "Young Guns" and one of the "five hot minds in economics" by the Yale Economic Review, he serves on the National Economic Council in Prague, where his provocative writing has achieved bestseller status. How has he done it? By arguing a simple, almost heretical proposition: economics is ultimately about good and evil. In The Economics of Good and Evil, Sedlacek radically rethinks his field, challenging our assumptions about the world. Economics is touted as a science, a value-free mathematical inquiry, he writes, but it's actually a cultural phenomenon, a product of our civilization. It began within philosophy--Adam Smith himself not only wrote The Wealth of Nations, but also The Theory of Moral Sentiments--and economics, as Sedlacek shows, is woven out of history, myth, religion, and ethics. "Even the most sophisticated mathematical model," Sedlacek writes, "is, de facto, a story, a parable, our effort to (rationally) grasp the world around us." Economics not only describes the world, but establishes normative standards, identifying ideal conditions. Science, he claims, is a system of beliefs to which we are committed. To grasp the beliefs underlying economics, he breaks out of the field's confines with a tour de force exploration of economic thinking, broadly defined, over the millennia. He ranges from the epic of Gilgamesh and the Old Testament to the emergence of Christianity, from Descartes and Adam Smith to the consumerism in Fight Club. Throughout, he asks searching meta-economic questions: What is the meaning and the point of economics? Can we do ethically all that we can do technically? Does it pay to be good? Placing the wisdom of philosophers and poets over strict mathematical models of human behavior, Sedlacek's groundbreaking work promises to change the way we calculate economic value.

This empirical research methods course enables informed implementation of statistical procedures, giving rise to trustworthy evidence.