Holmesian Deduction
Sherlock Holmes is perhaps the most iconic detective in literature. His character continues to enthrall – there is a new BBC series with a modern Sherlock Holmes, and other popular TV characters, such as House, are significantly based on Holmes. What I think is endlessly compelling about Holmes is the seemingly preternatural skill with which he “deduces” specific facts about people and situations, based upon careful observation and a rigorous thought process. But then he makes it all seem so easy in retrospect when he reveals his method.
Because Holmes is such a fascinating character and Doyle wrote prolifically about this character, Holmes is also a useful and frequently used example of logic and the process of detective work. I took a course on Holmes in medical school, using Sherlock Holmes short stories as examples of diagnostic principles (Arthur Conan Doyle was a physician, and clearly drew upon this experience in writing Holmes). A recent Scientific American article, for example, used Holmesian logic as an example of how not to make several common fallacies of thinking – falling for the conjunction fallacy, the representativeness heuristic, and failure to consider the base-rate.
Briefly, we tend to assume that someone belongs to a category if they have features we find representative of that category, or if we can readily bring to mind similar examples. We tend not to consider the base rate – what percentage of the relevant population belongs to that category. Sherlock Holmes encapsulated part of this idea with his famous quip to Watson that if you hear clopping on a cobblestone street in London, think horse, not zebra. This principle is common in medical diagnosis, and in fact we call rare diseases (those with a low base-rate) “zebras” after Holmes’ example. In other words – even if a person has signs and symptoms that resemble a specific disease, the probability of that diagnosis is still low if the base rate is very low – it’s a rare disease. In fact, an atypical presentation of a very common disease may be more likely than a typical presentation of a rare disease.
But we tend to be more compelled by the representativeness of a person or situation than the base rate. Our evolved innate calculations of probability are flawed in this way.
The article also discusses the conjunction fallacy – the tendency to think that two likely conclusions are more likely that either alone. Mathematically, this cannot be true – the probability of A+B must be less than the probability of either A or B. This is easy enough to understand, but we tend to ignore this when confronted with a representative example. For example, if I describe for you a typical nerd and ask about the probability that they work in IT and play video games, your naive logic tells you that the probability of both is greater than the probability of either by itself, because they are both representative of the typical nerd.
Holmes is able to arrive at such accurate statements about those he observes partly because he does not fall for all the typical fallacies and heuristics that hamper our thinking. When they are explained, they make perfect logical sense. The average person, however, is typically not aware of all the flaws in logic that plague our thinking day to day.
However, I have to point out that the process that Sherlock Holmes engages in is usually not deduction, even though that is the term the character Holmes uses to describe his process. The new BBC Holmes series, for example, has a website called The Science of Deduction (which exists also in the world of the series as Holmes’ website).
Deduction is the logical process of going from the general to the specific. If one or more premises – general statements about the world – are true, then a specific conclusion must also be true. This is deduction. The textbook example of this is: Premise 1: All men are mortal; Premise 2: Socrates is a man; Conclusion: Socrates is mortal.
There are times when Holmes does indeed use deduction, but not when he is coming to conclusions based upon his careful observations. Some logicians have referred to what Sherlock Holmes does as “Holmesian deduction” to distinguish it from deduction itself. What Holmes is largely doing is well-informed inference. He begins with careful observation of details. He then uses base rate knowledge (rather than superficial representativeness thinking) to make specific high probability guesses. Watson, he observes, has a tan. He did not get the tan in London, so he must have recently arrived from a tropical location. He puts this together with other bits of information to infer that Watson probably served in Afghanistan.
Holmes also uses a great deal of inductive knowledge. Induction is the process of going from the specific to the general – which is mostly what scientific investigation involves. We might note that every swan observed is white, and induce the general principle that all swans are white. While deductive conclusions must be true, inductive conclusions are tentative – they are extrapolations from limited data. They are also falsifiable (at least in principle) as was demonstrated by the discovery of black swans in Australia. Holmes develops general principles from inductive processes (science) and then applies them to his inferences. He spends much of his time between cases doing detailed analyses of cigar ash and other such things, providing knowledge he can then use in his investigations.
In many ways what Holmes does is very similar to the cold reading of fake psychics and real mentalists. A skilled cold reader will be armed with knowledge of the base rate of many things, such as male and female names. They will use observation and feedback in order to feed the process with information, and then make inferences about what is likely to be true. Just as with Holmes, they have the ability to amaze their target, seemingly pulling very specific information out of thin air. But just like Holmes, they are really just using a process of observation and inference.
Regardless of the purpose (medical diagnosis, entertainment, investigation, or fraud) what the character Holmes does aptly demonstrate is the power of substituting rigorous logical methodology for the naive reasoning that humans evolved.
And surely we may be entitled to conclude this? After all, it’s up to us what we choose to call things. The existence of the ‘black swan’ falsifies the hypothesis only because we allowed it to, because we chose to call this black thing a swan. We could equally as easily have chosen the name ‘blazzon’ or something.
It’s no use appealing to the idea of ‘species’, saying because that is then special pleading – species didn’t matter before, why should it now?
The black swan is a different species from white swans.
Even better then
That reminds me of an old Sid Harris cartoon. The setting: a biological research lab. “Rosenblatt didn’t discover it, we named it Rosenblatt because it *looks* like Rosenblatt.”
That is not a defensible definition of deduction, if you are trying to define the word “deduction” as it is used in logic. (Holmes’s use of the word is plainly indifferent to the distinction between induction and deduction.) An argument is deductively valid if and only if it is impossible for the premises to be true and the conclusion false. This has nothing to do with going from general to specific (a fatally vague phrasing in any case). For instance, an argument of the form “If A, then B; if B, then C; therefore, if A, then C” is a valid deductive argument regardless of whether it goes from the general to the specific.
MKR – as I wrote, deduction is the process of starting with premises and leading to a conclusion that must be true if the premises are true. This usually involves premises that are general statements, but you are correct that they do not have to be.
Mr. Novella, I am glad to learn that that is what you had in mind, but it is not what you wrote in the passage that I quoted. You wrote that “Deduction is the logical process of going from the general to the specific,” which is not true.
It’s interesting that Holmes is the icon for deductive logic — because most of the actual stories by Doyle don’t hinge on deduction. Holmes solves many of his cases by acute observation, or special knowledge, code-breaking, or plain old legwork. In several cases he doesn’t actually do much but watch as some domestic tragedy plays itself out.
There are some genuinely impressive feats of deduction in the stories: “The Adventure of the Naval Treaty” is solved by pure logic without any cheating.
About “The Creeping Man” or “The Speckled Band” the less said, the better. SNAKES DON’T WORK THAT WAY, Mr. Doyle!
The Holmes quotation about hoofbeats and zebras is a false attribution. At least, I have been unable to find a direct quote in Doyle matching this. According to wikipedia the correct attribution is http://en.wikipedia.org/wiki/Theodore_Woodward
“In many ways what Holmes does is very similar to the cold reading of fake psychics and real mentalists.”
Which must’ve inspired the series Psych and The Mentalist.
It’s ironic that Arthur Conan Doyle was taken in by psychics and spiritualism.
All scientific skeptics should know the formula for Bayesian inference:
prior odds * Bayes factor = posterior odds
Where:
H0 = null hypothesis
H1 = alternative hypothesis
E = evidence
P(H1)/P(H0) = prior odds
P(E given H1)/P(E given H0) = Bayes factor
P(H1 given E)/P(H0 given E) = posterior odds
How does one use Bayesian inference correctly? The only example I’ve seen is where someone decided that chance and probability were opposites, and concluded that there’s a 50:50 chance of any side of a die coming up. Do you simply compare the prior odds to the posterior odds, and if the posterior ones are greater, it infers that the alternative hypothesis is correct?
No, the alternative hypothesis is more likely to be correct if the odds are greater than 1. Prior just means before seeing evidence. Posterior means after seeing evidence.
Here’s a textbook example that could save your life. Suppose that given your age, sex, and other background info, you have a 1% chance of having cancer, so your prior odds of having cancer are 1:99 or 1/99.
You do a blood test that catches 80% of cancers but has a 10% false positive rate, so its Bayes factor is 80/10=8.
The test is positive, so your posterior odds of having cancer are 1/99 * 8 = 8/99, and your new chance of cancer is 8/(8+99) = 7.5%.
Now here’s the kicker. When physicians were given this problem, 95% of them said the new chance of cancer is around 75%, because they thought it would be close to the test’s 80% sensitivity.
http://en.wikipedia.org/wiki/Confusion_of_the_inverse
Juries are instructed that “Neither the indictment itself nor the fact that an indictment has been filed constitutes evidence,” but it does increase the probability that the defendant is guilty. It’s not like random people are charged with crimes for no reason.
The ultimate question for a jury is never whether the defendant committed the crime charged, only whether the government has proven beyond a reasonable doubt by evidence adduced at trial that the defendant committed the crime charged. An indicment is not evidence; thus, the instruction you noted. Juries can and have acquitted defendants they knew perfectly well committed the crime because the government didn’t – or couldn’t – present evidence to get over the high standard required. They are right to do so.
But what are the prior odds? If the jury truly assumes nothing about the defendant, then the odds of guilt are 50:50 to begin with. If the jury assumes that the defendant is as likely to be guilty as any person, i.e. the base rate, then the prior odds of guilt can be extremely low. Now, I think (and seriously hope I’m right) that the probability of guilt among those charged with a crime is higher than the base rate. And as far as I’m concerned, anything that increases the probability of guilt is evidence.
By the way, I found the jury instruction not to use special expertise.
http://www.skepticblog.org/2011/10/31/lie-detection/#comment-67757
I disagree. If the jury TRULY assumes nothing about the defendant, than the odds of guilt are 1:7 billion.
Also, the Prosecutor’s Fallacy may be germane to this conversation: http://en.wikipedia.org/wiki/Prosecutor%27s_fallacy
You assume that one person in the world is guilty. If you TRULY assume nothing, then nobody or everybody could be guilty.
We are to suppose innocence until guilt is proven.That is the way that I would look at it (at least in the US system). So, knowing nothing about the accused,a juror should assume that they are 100% innocent,not 50% chance of innocence.Of course,in reality,people often assume that someone is probably guilty,or else they wouldn’t be in that position.
Many people in Texas (and elsewhere) have had to be freed due to DNA evidence proving their innocence,but you can bet,that at the time of their trial,most were being presumed guilty even before the trial began.
Also,in regards to the 1 in a billion,it is also always a possibility that no crime was actually committed by anyone ,in a trial situation.
If you assume 100% probability of innocence, then by definition there’s no evidence that would change your mind.
What I meant by assuming nothing is that you have no idea who the defendant is or why he was charged. It could be a random person off the street with almost zero chance of guilt, or it could be someone like the Fort Hood shooter with a high chance of guilt. Without any prior knowledge, the prior odds are 50:50.
Max,that doesn’t make any sense. Without any knowledge of a crime, you cannot assign any probability at all as to whether someone is guilty or not with any degree of rationality. Even if you had a statistical evaluation of how often a person is found guilty of a given crime, it tells you nothing about a specific situation before you hear the case. Human activity is not like flipping a coin,where the probability can be approximated prior to a toss.
First, I’m doing reductio ad absurdum here. My point is that you always make assumptions and have some prior probability in mind. Making no assumptions would be absurd.
The coin analogy is not that a fair coin is 50/50, but that an unknown coin is 50/50, because for every unfair coin, there’s another unfair coin that does the opposite, and since there’s no reason to prefer one over the other, it all averages out to 50/50.
If you assume that only one person is guilty, then the prior odds are 1 in 7 billion, but if multiple people can be guilty, then it’s the same problem as the unknown coin. For every division of people into guilty and innocent parties, there’s an opposite division.
tmac57,
“Even if you had a statistical evaluation of how often a person is found guilty of a given crime, it tells you nothing about a specific situation before you hear the case.”
It tells me the base rate that Steven was talking about. Same idea as a statistical evaluation of disease prevalence before evaluating a specific patient. That’s why there are different guidelines for high-risk patients.
I once served on a jury that acquitted an armed bank robber who we all thought was probably guilty, but there was a strong reasonable doubt. We found out after the trial that he was indeed guilty, but we weren’t allowed to hear some of the evidence. (The ONLY evidence they had was eyewitness testimony).
Had we known “perfectly well”, he had committed the crime we would have perfectly well convicted him.
Indeed the jury probably did the correct thing within the bounds set out for them,because ‘probably guilty’ people sometimes turn out to be innocent:
http://www.sdbar.org/pamphlets/guilty_client.shtm
Some people prefer the expedient of going with their gut impression of guilt,but they wouldn’t want to find themselves on the opposite end of that type of ‘justice’.
“It’s not like random people are charged with crimes for no reason.”
Oh, yes, they are. It’s called a frameup. Sometimes cops wanting credit for an arrest will pick out some likely-looking person, maybe one of the “usual suspects’ and have him charged on perjured evidence, forced confessions or other bogus criteria. Prosecutors charge and try people because they have been pressured into going forward with a weak case or no case or because the DA has a grudge or a bias. People are charged because they have on the same shirt color as the witness claimed to see, or the same model car, or because they happened to be found in the vicinity, or because of their race; too often they are convicted for the same reasons. Sometimes the reasoning is “we couldn’t get him for the other job so let’s pin this one on him.”
All this is less common than it used to be, but you can bet it still goes on.
I meant every time, not sometimes. If an indictment doesn’t tell you anything about a person’s guilt, that implies that an innocent person is as likely to be indicted as a guilty person, which implies that the process no better than charging random people every time.
“People are charged because they have on the same shirt color as the witness claimed to see, or the same model car, or because they happened to be found in the vicinity, or because of their race; too often they are convicted for the same reasons.”
It may be weak evidence, but it’s still better than a truly random person off the street.
Per the US Constitution, all defendants are presumed innocent unless and until the government can prove their guilt by proof beyond a reasonable doubt. Jurors are commonly asked during jury selection if they agree with this. If they say no, they will be removed from the jury for cause. From the jury’s point of view, it has nothing to do with percentages or base rate.
As to the issue of how many defendants are actually convicted either by a plea or a jury verdict, if it’s not quite high, the charging authority needs to do reassess how they charge crimes and/or how they prosecute them because they shouldn’t be charging crimes for which they’re unlikely to get convictions. 50-50 would be shameful.
For me it has everything to do with percentages, and I actually have a pretty high standard for proof beyond a reasonable doubt, since even 95% would mean five wrong convictions out of a hundred.
Oh, and another instruction is that the defendant’s refusal to take the stand isn’t evidence of guilt. Again, ANYTHING that a guilty person is more likely to do than an innocent person increases the chance of guilt according to the little Bayesian formula above.
http://www.skepticblog.org/2011/11/07/holmesian-deduction/#comment-67751
I suppose one can figure percentages for anything and if you’re just talking about the number of convictions per charge filed, that is done routinely by court systems. It has nothing to do with how jurors are supposed to approach a case since they must presume the defendant not guilty.
Of course there’s an instruction that the defendant’s refusal to take the stand isn’t evidence of guilt. While you’re looking up all these instructions, please read the US Constitution – particularly the Bill of Rights. Also, please note that each jurisdiction – each state and the federal government – have a slightly different set of instructions, some based on the US Constitution, some based on a state constiution, and some based on case law. However, the one you mentioned applies everywhere because it’s a right guaranteed by the US Constitution.
The Constitution guarantees the right not to take the stand, but it can’t tell me what to believe.
I had to spend 10 hrs explaining to some idiot on my jury that the guy is INNOCENT until proven guilty. Made me almost miss my flight for a holiday because some dumbass thought there was a 50% chance the guy did it. Luckily the judge had had enough after a day and a half of deliberations and let us have 11/12 to pass judgement rather than unanimous.
I hope you never serve on a jury Max.
If you are innocent it is almost always better to NOT take the stand. This is because in follow up appeals the written transcript can be read back to you and you can be challenged and made to look like an idiot for telling something slightly differently from a trial a year ago and the lawyer can put on whatever voice they like while explaining it.
The juror wanted to convict someone with a 50% chance of guilt? Did you explain that this threshold would make half of convictions wrongful?
If guilty defendants are as likely to testify as innocent defendants, then refusal to testify doesn’t tell you anything. If guilty defendants are less likely to testify than innocent defendants, then refusal to testify increases the chance of guilt. Do the math.
Craig,
“Luckily the judge had had enough after a day and a half of deliberations and let us have 11/12 to pass judgement rather than unanimous.”
Is that even legal? Shouldn’t there be a hung jury in this case?
Max-“If guilty defendants are less likely to testify than innocent defendants, then refusal to testify increases the chance of guilt. Do the math.”
So,put yourself in the position of a juror.The defendant chooses not to take the stand.Do you take this as a sign of guilt,or do you consider that the defendant’s attorney might well have advised the client that it is a risky move for pragmatic and statistical reasons not withstanding their guilt or innocence?
If 60% of innocent defendants and 90% of guilty defendants do some particular thing, then doing that particular thing increases the odds of guilt by a factor of 90/60=1.5.
“In fact, an atypical presentation of a very common disease may be more likely than a typical presentation of a rare disease.”
I’ve seen it cited as a pitfall of Occam’s razor, looking for one disease that explains all symptoms, and overlooking the possibility of multiple more common diseases.
Just a quick note to say that *induction*–contrary to popular usage–is a deductively valid tool in the context of mathematics.
So perhaps we should split hairs here, and refer to things that are inductive (mathematically) vs things that are inductive (commonly). Truths established by the former can be fully trusted, while truths established by the latter cannot be fully trusted. Mathematical induction produces results that are 100% true and cannot be falsified.
No scientific work is truly deductive. Mathematics and logic are the only fields that enjoy exclusive rights to deduction.
The conjunction fallacy is closely related to the confusion of the inverse.
http://en.wikipedia.org/wiki/Confusion_of_the_inverse
Experimental subjects said that P(A and B | C) > P(A | C), when they were really thinking P(C | A and B) > P(C | A).
For example, is a heavy smoker more likely to have bad breath, or to have bad breath and lung cancer? Bad breath.
But here’s the inverse: Is a person more likely to be a heavy smoker if he has bad breath, or if he has bad breath and lung cancer? Bad breath and lung cancer.
“… the probability of A+B must be less than the probability of either A or B”
That’s *equal* to or less than, with the added qualifier that A, B, and A+B must also be equal to or greater than 0.
I’d just like to add that Holmes bored me by the time I was 11 because (a) his logic is mostly faulty and (b) he trumps everyone else not because of fantastic logic but because of facts concealed by the author. If a victim were found with a hatchet in his/her head, that fact would be conveniently secreted by Doyle through most of the narrative. Watson has a reputation for being a buffoon due to his ability to miss the obvious. However, it is interesting to see that people selectively remember Holmes’ good days and quotable quotes.
And there’s the “when you have eliminated the impossible …” mistake. Whatever remains, however improbable, must be the truth? Yeah, but not terribly useful if you haven’t thought of one of those (indeterminately many) (improbable) explanations.
I think that’s again due to the induction situation: once you have eliminated the likely outocmes, the less likely ones become more valid. But the quote is misleading.
I believe you are thinking of the Basil Rathbone Sherlock Holmes. In the short stories, Dr. Watson is the purported author, documenting their adventures, and I don’t recall him portraying himself as a buffoon. He was the war veteran who carried the gun. Re-read one of the stories, and I think you will see they are quite different than any of the movie portrayals.
I’m just nit-picking. Take no offense.
And the really sad thing is that Conan-Doyle applied none of Holmes’ methods to his own investigations of the “spirit world.” If he had, he wouldn’t have believed in spiritualism or fairies.
Isn’t Holmes using inductive reasoning not deductive?
He makes inferences, which is more like deductive reasoning.
Premise 1: It’s hard to get a tan in London.
Premise 2: Watson has a tan.
Conclusion: Watson must have arrived from some place where he could get a tan.
Premise 1: All men are mortal;
Premise 2: Socrates is a man;
Conclusion: Socrates is mortal.
syllogism
Like many others Doyle found the real post war world something to get away from. Spiritualism or fairies made him and others feel better. And all he was doing was making money to live on. He did jump start the use of scientific crime detection. One time he was pushed into investigating a real crime. He had inducted the size the criminal and that he was very poor when the local cop showed up with the tramp thief. Who was like Dole said he was.
“He did jump start the use of scientific crime detection.”
Why would you say that? Poe’s Dupin came long before and a contemporary of Holmes, Pudd’nhead Wilson was a far better scientist than Holmes ever was.
http://www.socialresearchmethods.net/kb/dedind.php
Here’s what I was taught in school. Holmes makes observations then forms hypotheses, thus his form of reasoning is inductive.
I have a friend who often displays a distressing lack of critical thinking skills, particularly with regards to Big Government conspiracy theories. I’ve modified the hoofbeats quote to apply to him: When he doesn’t hear hoofbeats, he thinks pegasus.
I’m glad someone remembered Pudd’nhead Wilson. And what the book showed to all who could see. It may have been Mark Twain’s most far reaching book, but it was so disturbing it’s still mostly forgotten. Dupin’s time was not right. Dole did spread use of scientific crime detection. But the French were first for real. Few want to think. They want to find things they think are already true. ME TOO!
Forget Holmes. Jonathan Creek is the best british tv sleuth. If you combined James Randi with the early 70’s character Banacek and put them in a young nerdy brit you get Jonathan Creek.
I apologize if this has already been mentioned, but I believe there is a small mistake in the statement that “mathematically, this cannot be true – the probability of A+B must be less than the probability of either A or B.” I think the correct statement would be that “the probability of A+B must be less than or equal to the probability of either A or B.”
It may be a little late for a corrective comment, but I just thought that a fact-checking observation is always adequate at a website determined to scientific rigor like this one.
Otherwise, it is a lovely article.