In Carl Sagan's novel Contact, heroine Ellie Arroway manages to travel through worm-holes into uncharted regions of the universe, rendezvous with intelligent extraterrestrial life, and return safely to earth to tell her colleagues about her amazing journey s1Sagan, C.: 1985. Contact. NewYork: Simon & Schuster.. There's only one problem: no one believes her. Ellie's entire adventure occurred within a span of a few moments, and observers did not even see her spacecraft leave its launch site. Unable to document her reported experiences, Ellie's colleagues conclude that there is simply no compelling evidence that her adventure actually happened.
Of course, as readers of the novel, we know that Ellie is right and her colleagues are wrong. Why then don't they
believe her? They doubt her story because, as good scientists, Ellie's peers examine all hypotheses using die honored
principle of Ockham's Razor. That is, with all other things being equal, the
simplest hypothesis is most likely to be correct
. They scrutinize Ellie's elaborate claim, involving
mind-boggling excursions far beyond the range of established science, and find the story to be
fantastically improbable. Even Ellie must admit that the finely honed blade of Ockham's
Razor seems to slice her tale to pieces.
Although Ellie and her plight are fictitious, her harrowing brush with Ockham's Razor raises interesting questions for skeptics. Like Ellie's scientific colleagues, we have learned to accept Ockham's Razor as a powerful tool for weeding out bogus theories. But how do we know that die maxim "simpler is better" will always lead us down the royal road to truth? How can we say, prior to further investigations, that simple theories are automatically more likely to be true than complex theories? Might not Ockham's Razor, thought by many skeptics to be the surest weapon against pseudoscience, simply be an unproven philosophical assumption? If so, perhaps we will share the fate of Ellie's peers and reject the correct answers simply because they don't conform to prior expectations.
This article presents a brief historical sketch of the principle, and cites examples of the misuse of Ockham's Razor to support dubious theories and how that misuse led to the rejection of good science. This discussion leads to an attempt to identify the limitations and qualifications needed to apply the principle properly when choosing among various theories. I then try to determine what justification, if any, we can have for using Ockham's Razor, and argue that skeptics should learn to use the principle carefully in conjunction with other criteria of theory selection. I hope to show that Ockham's Razor is a dangerous weapon if mishandled, but that those who follow the proper safety precautions will find it to be a very helpful tool for evaluating theories.
The principle of Ockham's Razor is named after William of Ockham (1285-1349), a distinguished medieval philosopher and
theologian. Contrary to popular assumptions, Ockham did not invent the principle that has become associated with him.
The idea that simplicity and efficiency are important advantages of a theory dates back at least to Aristotle, who stated that the more perfect a nature is, the fewer means it
requires for its operation
s2Ross, W. D. 1930. The Works of Aristotle, Translated into English, vol. II. Oxford: Clarendon..
Centuries after Ockham's time, Isaac
Newton would also cite the principle of simplicity in his Principia Mathematica: We are to admit no more
causes of natural things than such as are both true and sufficient to explain their appearances
s3Newton, I.: 1999. The Principia: Mathematical Principles of Natural Philosophy, Berkeley: University of California Press..
Ockham emphasized the principle of parsimony as an antidote to the various
unwarranted assumptions he perceived in the philosophy of his time. He used a number of different formulations of the
principle in his writings. He stated, for example, that it is futile to do with more what can be done with
fewer
, and perhaps most famously, that plurality should not be assumed without necessity
s4Boehner, P.(ed.) 1957. Ockham: Philosophical Writings. Edinburgh: Nelson.. A common term for
this concept is parsimony. Ockham's formulations match up quite well with modern definitions of parsimony, which state
that the most parsimonious models are those requiring the fewest assumptions s5Schick Jr, Theodore & Vaughn, Lewis. 1999. How to Think About Weird Things. 2nd Ed. Mountain View, California: Mayfleld Publishing. For an excellent overview of the methodology used to evaluate new scientific theories, consult Friedlander, Michael W. 1995. At the Fringes of Science. Boulder, CO: Westview Press..
Ockham's use of the principle was specifically targeted at the champions of the philosophical school known as realism, who argued for the reality of traits called universals. Universals are concepts pertaining to the characteristics of individuals or groups of individuals. For example, suppose we were talking about the wisdom of Aristotle, or the heroism of Socrates. A realist would claim that these ideas of wisdom and heroism are not just concepts created by our mind, but eternal truths about reality as well. Nominalists such as Ockham and his mentor Duns Scotus rejected the concept of universals as an unnecessary assumption that does little to improve our understanding, and perhaps even confuses us s6McGrath, Allster: 2001. Christian Theology: An Introduction. Oxford: Blackwell.. Universals may be useful for visualizing and talking about our perceptions of the world, but they don't necessarily exist in objective reality. And if there really is a shadowy realm of unchanging universals out there somewhere, we don't need to assume the existence of this realm in order to fully explain and understand the world. Why not just avoid assumptions that aren't needed?
As skeptics, we may ask how Ockham reconciled his support of philosophical parsimony with his theological beliefs. After all, isn't belief in God the ultimate example of the kind of universal Ockham was rejecting? We can certainly argue that the concept of God simply represents the synthesis of all of the concepts about life we care most deeply about-love, justice, and mercy, for example. We can describe and discuss these concepts without having to propose the existence of a deity. Perhaps we also know that many cosmologists believe that the universe in some way has always existed, and we can see that there are no scientifically or logically necessary reasons to believe in God, Ockham, however, did not use the principle of parsimony to question the existence of a deity. While he always maintained that plurality should not be assumed without necessity, his life and personal habits of thinking convinced him that God's reality was indeed necessary. Likewise, he found revealed religion as recorded in the Bible to be absolutely essential for understanding the nature of this God.
Of course, none of this is meant to belittle Ockham, or to imply that proper use of Ockam's Razor compels us to reject belief in God. However, the preceding example does show that reliance on parsimony will not lead all of us to the same conclusions. Some of us may regard a certain idea as essential, while others will think differently. In fact, both the supporters and enemies of a certain theory often use the principle of Ockham's Razor with equal passion. Since both sides cannot be right, it should go without saying that Ockam's Razor doesn't always lead to the best possible theory.
Although some of us may wish to believe otherwise, too much reliance on "simplicity" leads to error more often than it leads to truth. By endorsing any theories that subjectively seem simpler than rival theories, we risk paying too little attention to other important criteria for evaluating theories. And when perceived simplicity of a theory commands too much of our attention, the door is opened to reject any and all theories we personally find too complex to understand.
Examples of abuse of Ockham's Razor in support of worthless theories are abundant. Pseudoscientists often invoke Ockham's Razor to defend apparently simple theories discredited by mainstream science. They regale us with stories about ghosts, aliens, forest monsters and other "unexplained" phenomenona, and then shake their heads impatiently when scientists deny the validity of their "evidence." Aren't scientists just grasping at straws when they refute the stories of the thousands of people who have experienced these strange happenings? The simplest explanation of paranormal events, at least to the paranormalists, is that the events are exactly what they seem. Why do we need complex scientific explanations of ghost photos, for example, when we can just assume that the unidentified patch of light in our photo from Gettysburg is the specter of a Civil War soldier?
Creationists are especially good at using apparent simplicity to attack established science. Who needs this far-fetched theory of evolution, they ask? We don't need to review all of this research from geology, genetics, developmental biology, and anatomy purporting to show that all life on earth slowly evolved by descent from common ancestors. It's much simpler to just say "God did it" and end it there. By allegedly adopting the principle of Ockham's Razor, creationists give their theories the veneer of intellectual rigor. They claim to be more scientific than scientists, who just want to overcomplicate everything with their convoluted theories and denials of simple faith in biblical authority.
Sometimes, abuses of simplicity strengthen stereotypes about certain groups. For example, some social scientists argue that genetic inferiority is the simplest and therefore most likely explanation for the lower measured abilities of nonwhites on standardized intelligence tests s7Gould, Stephen Jay. 1996. The Mismeasure of Man. New York: W. W. Norton.. After all, it's much "simpler" to just assume that nonwhites are inferior instead of accounting for the complex effects of culture, history, economics and racism on their test performances. On the other hand, behavioral geneticists and evolutionary psychologists argue that their explanation for group differences are more complex than social and cultural explanations, and thus the charge that Ockham's Razor applies to sociologists and not sociobiologists.
Unfortunately, even skeptics are known to enlist Ockham's Razor in the defense of dubious theories. Allegedly following the scientific method, these skeptics maintain that Ockham's Razor is simply an example of the principle of reductionism, which attempts to explain complex phenomena in terms of simpler models. These skeptics claim that reductionism is the very basis of scientific inquiry. To an extent, they are right, since a theory must be simpler than the phenomenon it seeks to explain if it is to be of any use. However, skeptics sometimes take reductionism a bit too far.
Let's consider the examples of evolutionary psychology and its closely related cousin, memetics. While many evolutionary psychologists are modest and responsible when discussing the reach of their theory, some of their peers are not so cautious. Overzealous evolutionary psychologists seek to explain all of human nature, including consciousness, sexual attitudes, religious beliefs and moral sentiments in terms of evolutionary advantages conferred by those traits. At some point in our ancestral environment, the traits we associate with human nature developed and caused those people (or proto-people) who possessed these traits to reproduce more successfully than those who lacked these traits. Therefore, evolutionary psychologists reason, every aspect of human nature exists because it served an important function in our evolutionary past, and contributed to the selective fitness of the species. Even something as apparently trivial as our sweet tooth may have evolved for good reasons, since it may have conferred an advantage on our ancestors when supplies of sugar were relatively scarce s8Wright, Robert. 1995. The Moral Animal: Why We are the Way We Are. New York: Vintage Books.. According to some evolutionary psychologists, natural selection acts as a "universal algorithm," determining nearly everything about human nature s9Dennett, Daniel. 1995. Darwin's Dangerous Idea: Evolution and the Meanings of Life. New York: Touchstone Books..
Memetics takes the idea of natural selection as a universal algorithm even further by postulating the existence of entities called memes, defined as hypothetical units of information transferred from person to person. A meme can be anything from a guitar solo, a stanza of poetry, a religious dogma, or political slogan s10Blackmore, Susan. 2000. The Meme Machine. Oxford: Oxford University Press.. Memes either survive and reproduce or disappear, in a manner analogous to the natural selection of genes. To memetecists, this theory explains why some ideas spread and establish themselves more successfully than others. Religious ideas, for instance, allegedly reproduce the way viruses reproduce in the body of a host organism: they infect an otherwise healthy and rational person and fill his head with pleasant absurdities about the existence of a God and an afterlife. Indeed, one of the main appeals of both evolutionary psychology and memetics to some skeptics is the simplistic materialism of both theories, and their usefulness in dispensing with religious claims.
All of these abusers of Ockham's Razor fail to understand that simplicity is not a criterion to use in isolation from other important evaluative factors. The principle of Ockham's Razor doesn't tell us that simpler is always better, it merely says that the simpler theory is better if all other factors are equal. Thus, simplicity must be considered as one factor among several. Other factors used to assess the value of a particular theory include the following s11Schick & Vaughn, 1999. Op. cit.:
There are no rigid rules for applying or weighing these criteria, but good theories tend to satisfy one or more of them. Inadequate theories, on the contrary, consistently fail to meet most of these criteria. For example, let's consider the proposition that the hazy patch of light in my Gettysburg photo is a ghost. We can say that this theory is certainly not testable, since there is no way to confirm the presence of the ghost. The ghost hypothesis also is not fruitful, because it is just an ad hoc hypothesis invented to explain a single anomaly in our photograph. The theory doesn't add anything to our understanding or knowledge, so it has very poor scope. It isn't conservative, either, because it contradicts a great deal of background knowledge. This background knowledge tells us, among other things, that a hypothetical noncorporeal entity such as a ghost couldn't possibly show up in a photograph, because it would have to possess matter in order to reflect light toward our camera lens. Finally, the notion of a ghost is far from simple, since it proposes a being with alleged characteristics and powers that not even paranormalists seem able to consistently describe.
Now, let's consider the alternate hypothesis that the image in our photo is merely an aberration caused by technical problems with our camera. This theory is testable, because we can experiment with various settings on our camera to reproduce images similar to those in our ghostly photo s12Nickell, Joe. 2001. "Ghostly Photos." In Real-Life X-Files: lnvestigating the Paranormal. Lexington, KY: University Press of Kentucky, 128-132.. The theory is also fruitful, because it allows us to predict that certain camera settings will cause specific kinds of "spectral images," even if ghost hunters haven't reported these kinds of images yet. The theory also has good scope and conservatism, because it allows us to explain a wide variety of "mysterious" photographic images in a manner fully consistent with our established knowledge about the world. And yes, it's a simpler theory because it doesn't require us to make complicated or dubious assumptions. No wonder that polls of scientists belonging to the National Academy of Science (NAS), an elite organization consisting of the best scientists in their field, consistently show that almost no NAS members believe in ghosts. Familiarity with the criteria of good science breeds contempt for ad hoc explanations.
Let's not be too smug, however. We've seen that even scientists and skeptics can overemphasize simplicity in their efforts to evaluate theories, as in the cases of overambitious evolutionary psychologists and memeticists. Each of these groups of alleged skeptics would do well to review the criteria of testability, fruitfulness, scope and conservatism a bit more. In their extreme emphasis on natural selection, these groups ignore the fact that much evolutionary change is not completely adaptive, but results from a plurality of factors such as genetic drift and contingency s13Ridley, Mark. 1996. Evolution. Oxford: Blackwell. s14Gould, Stephen Jay. 1997. "Evolution: The Pleasures of Pluralism." The New York Review of Books, June 26.. Both theories are unconservative, since they contradict well-established knowledge, and they're not fruitful because they don't predict or explain phenomena better than competing theories. Memetics, for instance, distorts and sometimes completely contradicts the complex model of cultural transmission of ideas presented by mainstream social sciences s15Polichak, James W. 1998. "Memes: What are They Good For? A Critique of Memetic Approaches to Information Processing." Skeptic, Vol.6, No, 3, 45-53.. When scientists forget to use parsimony in careful conjunction with other criteria, they don't perform much better than pseudoscientists in separating the good science from the bad.
Therefore, we see that parsimony is one criterion among several for evaluating theories, and none of these criteria has clear priority over the others. But we still haven't determined why simplicity should be a criterion for assessing the merits of a theory. Why is simplicity a virtue when it comes to scientific theories?
Before we can determine why parsimony tends to be an attribute of good scientific theories, we have to determine our ultimate goal in selecting a theory. That is, what do we hope to accomplish? Clearly, we must know what we want before we can justify the best ways to accomplish it. The answer that immediately comes to mind is that we wish to determine which theories are true. After all, the pursuit of truth seems to be the whole point of doing science. However, there are times when the true theory, at least in the strictest sense, may not be the best choice.
Let's review an example adapted from an important paper by philosopher of science Elliot Sober s16Sober, Elliot. "Instrumentalism, Parsimony and the Akaike Framework." 2000. Proceedings of the Philosophy of Science Association. Available online at www.philosophy.wisc.edu/sober /papers.htm. Suppose we are interested in measuring the effects of a new fertilizer on the growth of corn. We measure the heights of corn in two large populations, consisting of one population grown with the new fertilizer and one grown with the old fertilizer. Then, we compare the mean heights of the populations. If u(f) and u(o) are the mean heights of the corn populations with and without the new fertilizer, respectively, then the two hypotheses under consideration are
The null hypothesis says that there is no difference between the mean heights of the two populations. We know that there must be some difference in the mean heights of two populations containing, say, thousands of ears of corn apiece. Yet, scientists justifiably do not reject the null hypothesis unless the difference between the two means is statistically significant. That is, scientists provisionally accept a hypothesis they know to be technically false.
Why would scientists do such a thing? They refuse to reject null hypotheses under these circumstances because a major goal of science is predictive accuracy. As we have previously seen, scientific theories should make testable and fruitful predictions to be of use to us. In order to maximize the predictive power of their theories, scientists are willing to settle for a lower degree of truth. In many cases, including the two corn populations in our current example, null hypotheses do a better job at predicting new data than hypotheses meeting a more rigid defnition of truth.
Of course, it's worth stressing that pursuit of predictive power doesn't mean abandoning the quest for truth. Theories that predict more accurately must possess some degree of truth. As science philosopher Ernest Nagel pointed out, there is little or no difference between actively seeking theories with predictive accuracy and those with claims to realism s17Nagel, Ernest: 1979. The Structure of Science. Indianapolis: Hackett.. If true theories make the most accurate predictions, then maximizing prediction will lead to some degree of truth. It's just that the best theory probably will not perfectly match our empirical observations. That is, there will probably not be a perfect "goodness-of-fit" between our observations and the values predicted by our theory. Models with perfect fit may simply be a reproduction of our empirical observations, and may fail to predict new data sets.
As a further example, suppose we are interested in determining the relationship between an independent variable (x) and a dependent variable (y). We perform a series of measurements of x and y, and plot the results on a coordinate graph. The result of our plot is a cluster of points roughly following a straight line, as shown in figure 1. The most accurate model of this data would be a complex function connecting all of the points. However, most scientists would say that we are justified in modeling the points as a straight line with slope m and y-intercept c as shown in the figure.
There are at least two reasons for doing so. First, even in the best of all situations, some deviation between a parameter's actual value and measured value will occur. If the actual relationship between x and y is represented by the straight line, then a complex, jagged curve connecting the points will be wrong, even though it exactly matches the empirical data we collected! Second, as we've seen, a model should be selected partly based on its ability to predict new data. The process of data selection is twofold: we evaluate empirical data to choose the most likely models, and then subject those models to further tests based on their predictions s18Forster, Malcom. "The New Science of Simplicity." In A. Zellner, H.A. Keuzenkamp, and M. McAleer (eds.) 2001. Simplicity, Inference and Modeling. Cambridge: Cambridge University Press, 83-119.. Simpler models may help us make better predictions with greater ease than more complex models.
Thus, we want theories that approximate what we commonly think of as truth, but also enable us to make useful and testable predictions. And it aims out that the simplicity of a theory has some bearing on its predictive abilities. The simplicity of a theory, measured in terms of the number of variables or adjustable parameters it contains, affects the accuracy and scope of its predictions. For instance, hypothesis (1) above is a simpler theory than (2), since there is only one model showing that the mean heights of the populations are the same, and many possible models showing that they are different. But in our case, the simpler null hypothesis does a better job at predicting new data than the more complex hypothesis (2), Under the range of data sets under investigation, the simpler hypothesis is superior, because it trades a statistically insignificant loss of "truth" with a greater increase in predictive power.
A relationship between simplicity and prediction is demonstrated in FIGURE 2 below. Suppose we have two models that fit our empirical data about equally well: a simple hypothesis represented by H1, and a complex hypothesis represented by H2. The simple hypothesis H1 makes a limited range of predictions, but it predicts the data in region C1 more accurately than the more complex hypothesis H2. Thus, we prefer H1 if our data set falls within the range represented by C1, but we may prefer H2 if the data falls outside that range.
These examples show that model selection is a complex process involving the consideration and reconciliation of several evaluative criteria. How we weigh and apply these criteria depends, in part, on our ultimate goals, for example, do we wish to maximize goodness-of-fit, predictive accuracy, or scope? Our choice will determine the relative importance we assign to simplicity in choosing our theory. Luckily, mathematical models exist to help us quantify exactly what we gain and lose in our trade-offs among different criteria. FIGURE 2 above is a model derived using a statistical theory known as Bayesian theoiy that rewards simpler models for their sharper predictions in certain data ranges. As we've already seen, this model allows us to determine the relationship among predictive power, simplicity and scope. A mathematical tool known as the Aikake method also allows us to quantify the degree with which simplicity and goodness-of-fit both contribute to the expected predictive accuracy of a model s19Forster, Malcolm and Sober, Elliot. 1994. "How to Tell When Simpler, More Unified, or Less Ad Hoc Theories Will Provide More Accurate Predictions." British Journal for the Philosophy of Science, Volume 45, 1-35. s20Aikake, H. 1973. "Information Theory as an Extension of the Maximum Likelihood Principle." In B. Petrov and F.Csaki (eds.) Second International Symposium on Information Theory. BudapestAkademia Kiado, 267-281.. We have choices in selecting a theory, but not arbitrary choices. The use of mathematical models such as the Bayesian and Aikake methods gives us reliable estimates of the relative roles of simplicity and other factors in determining the value of a given theory.
These models also allow us to finally answer the question of whether Ockham's Razor rests on unproven philosophical assumptions. The answer is a resounding "no." Simplicity is not an arbitrary yardstick for assessing theories, or a whim of skeptics and scientists. We apply Ockham's Razor to theories because of empirical evidence that it works, and we have the mathematical models to show how and why it works. Simplicity has a definite and demonstrable relevance to the value of a theory, and interacts in important ways with other evaluative criteria. And contrary to some critics of Ockham's Razor, our use of simplicity as a criterion does not imply a belief that the world itself is simple. Rather, we have learned that better theories tend to be no more complicated than necessary to explain the world around us, in all its wondrous complexity. Even chaos theory, with all of its inherent unpredictability, is expressible in comparatively simple mathematical equations s21Stewart, lan. 1991. "Portrait of Chaos." In Nina Hall (ed.), Exploring Chaos: A Guide to the New Science of Disorder. London: W. W. Norton..
This means, of course, that we must take great care in application of Ockham's Razor. We should not be heedless reductionists, clinging to simple theories regardless of their adequacy. Instead, we should use mathematical models such as Bayes theorem and the Aikake network to model our models, and determine how much simplicity contributes to their utility. Then we will be well equipped to measure the strengths and weaknesses of scientific theories, and can use Ockham's Razor without injuring the reliability of our knowledge.
I began this article with a fictional example of the failure of Ockham's Razor. How can we be sure that our own use of Ockham's Razor won't compel us to reject valid ideas, or to embrace dubious theories?
The short answer is that we can't be sure. However, we can use Ockham's Razor more carefully, using mathematical models to determine the costs and benefits of simplicity in our theories, always mindful of our ultimate goals in choosing theories. We can also do our best to acquire higher quality data with which to form and evaluate these theories.
We should also remember that science is always provisional. Theories currently meeting all of our criteria of selection may be unsatisfactory for future applications. Still, science progresses through the careful amending and acquisition of knowledge and the use of empirically validated methods. Ockham's Razor is an important method of improving this knowledge acquisition. Even safe use of Ockham's Razor will not eliminate error altogether, but it will at least minimize our errors relative to other, less reliable methods of evaluating theories.
s22MacKay, D.J.C. (1992): "Bayesian Interpolation", Neural Computation, 4(3), 415447 s23Jeffreys, William H. and Berger, James O. 1991. "Sharpening Ockham's Razor on a Bayesian Strop." Technical Report #91-44C, Purdue University Department of Physics s24Rasmussen, Carl Edward, & Zoubin, Ghahramani. 2001. "Occam's Razor", Advances in Neural Information Processing Systems n° 13, MIT Press - More discussion of both the advantages and limitations of simple theories using the Bayesian paradigm s25Huffman, Roald, Minkin, Vladimir I., and Carpenter, Barry K. 1997. "Ockham's Razor and Chemistry." HYLE-An International Journal for the Philosophy of Chemistry, Volume 3, 3-28 - Useful examples applying Ockham's Razor and Bayesian statistics to chemical reaction mechanisms