- The following is an excerpt from an article by Kevin Boone. Bayesian statistics is so simple, yet fundamental a concept that I really believe everyone should have some basic understanding of it. Please, take your time and read carefully.
‘Bayesian statistics’ is a big deal at the moment. It has been put forward as a solution to a number of important problems in, among other disciplines, law and medicine. These problems are concerned with such matters as determining the likelihood that a particular suspect committed a murder if his fingerprints are found on the weapon, or the likelihood that a person who tests positive for HIV really has an HIV infection. These are, clearly, important matters. However, most people can’t see that there are any difficulties here at all. If the likelihood of two people having the same fingerprints is, say, one in 500,000, and fingerprints matching person X are found on the murder weapon, then surely it is a near-certainty that X is the murderer? And if I tell you that 99.5% of people with a confirmed HIV infection test positive for HIV, then surely a person who tests positive is 99.5% likely to have the virus? In fact, both of these conclusions of wrong. Dead wrong, if you’ll pardon the expression. The importance of Bayes’ theorem is that it will tell you the true likelihood of a person having an HIV infection if he tests positive, and the true likelihood of person X being the murderer if his fingerprints turn up on the weapon. Or, at least, it will do so if you can feed in the right data, which isn’t always easy.
In a celebrated court case (R v Adams  1 Cr App R 377, for any lawyers that are interested) Lord Bingham, one of the UK’s most senior judges, refused to allow the defence to present an argument to the jury based on Bayes’ theorem. He conceded that it was a methodologically sound approach, but that it would ‘confuse the jury’. In fact, Bayes’ theorem is extremely straightforward, and need not confuse anyone who can add, multiply, and divide. The result of this judge’s decision could well have been that an innocent person was convicted, because a likelihood based on a Bayesian calculation is not merely better than the intuitive result, it is the only right answer. Any different answer is simply wrong. Not wrong in the sense of ‘stealing is wrong’, but wrong in the sense that ‘2+2=5′ is wrong.
In this article I will explain how Bayes’ theorem works, from first principles. I assume of the reader no knowledge of mathematics beyond elementary arithemetic. At the end of the article I will attempt to show how Bayes’ theorem can be derived from the common-sense example I present, and to follow this does require a knowledge of basic algebra. But you don’t need to be able to follow the derivation to appreciate how Bayes’ method works. I will start by considering, as the Reverend Mr Bayes himself did, the best way to bet on a horse race.