The following letter was sent to me in response to my column in Scientific American (which generated hundreds of letters in response, so I penned the following response) in which I discussed the now-infamous (and infuriatingly counter-intuitive) probability problem called the Monty Hall Problem, or the Three Door Problem, in which a contestant chooses one of three doors, behind one of which is a car and the other two goats. Monty then reveals what’s behind one of the other doors (only ever showing a goat and never showing you your own door pick), which is always a goat, then asks if you want to change doors. Most people say it doesn’t matter because now it’s 50/50, but the correct answer is that you should always switch, which will give you a two-thirds chance of winning. There are simulations of the game online, but my correspondent took it upon himself to test the game with his own computer program. Here are his very interesting results, which also nicely show the scientific method at work:
I am writing to thank you for your articles in Scientific American, specifically the one in the October 2008 issue discussing the ‘Monty Hall Problem’. Thanks to your essay, I think I finally understand the scientific method.
After reading about the ‘Monty Hall Problem’, I couldn’t shake the idea that switching doors shouldn’t make a difference. I knew that I must be wrong, but couldn’t get my head around the problem; I couldn’t get to sleep for a couple of hours that night either. So, instead of just believing that I was right or wrong and leaving it at that, I decided to see if I could find any objective data that would support one view or the other.
I wrote a little Visual Basic application within an Access database and ran 100,000 sessions where the contestant switched doors every time. The contestant was successful a little over 62% of the time. This seemed to lean to the conclusion that switching leads to a two-thirds success rate, but 62.2% seemed odd. I ran 1,000,000 sessions to see if the numbers be more definitive; they weren’t, still 62.2%. So, I looked through the database tables where I recorded the results to see what was going on. It was then that the true meaning of the scientific method became apparent to me. Looking through the data, I developed a new theory of the ‘Monty Hall Problem’ and why the strategy of switching doors should be successful two-thirds of the time. The new theory was elegant, the logic seemed clear, even obvious, and it seemed to agree with the data. The remaining problem was what was happening with the missing four-and-a-half percent. My suspicion was that this was caused by the random number generator I was using to pick the door with the car behind it and the door the contestant chose during each trial not being random enough. I rewrote the function choosing these doors, attempting to make them more random and ran 100,000 new trials and ended up with a success rate of 66.43%, close enough to satisfy me that the switching strategy is indeed the way to go.
As I mentioned, this little exercise opened my eyes to the true meaning and power of the scientific method. I was confronted with two competing and mutually exclusive theories explaining how something works. Instead of stubbornly standing by my own gut feeling, or believing another theory simply on faith, I ran an experiment to see if either theory would be supported or disproved. Examining the data led me to support the switching strategy and to develop a new theory explaining why this is so. I also developed a new theory to explain the remaining discrepancies in the data, ran a second, refined experiment, and gained further support for the theory behind the switching strategy.
I’ve read most of Stephen Jay Gould and Carl Sagan, I even subscribe to Scientific American. I always thought that I believed in the scientific method. However, it took your article, and its inspiring me to use the scientific method for myself to finally truly understand it.
Thank you Douglas Millar!