1% of women at age 40 who participate in routine screening have breast cancer. 80% of women with breast cancer will get positive mammographies. 9.6% of women without breast cancer will also get positive mammographies. A woman in this age group had a positive mammography in a routine screening. What is the probability that she actually has breast cancer?
There’s a long history of studies that ask physicians this question and repeatedly find that only about 15% of doctors get it right, with most guessing in the 75% - 85% range. The correct answer is 7.8% and requires the use of Bayes’ theorem.
7.8%! No wonder doctors don’t want to do unnecessary tests, right? The vast majority of people with positive results are false positives!
The question is interesting because it highlights one of the common and more valid reasons to resist greater levels of routine testing: every test has some non-zero false positive and false negative rates, meaning that all else equal you’re going to end up doing more unnecessary potentially invasive confirmatory testing in absolute terms that you would otherwise.
However, not all else has to be equal. Physics ran into this problem long ago when particle accelerators started producing tons of data and they almost had to abandon p-values as tests of significance because the statistics were getting all slanted.
What’s the solution? Layering. If you run the above diagnostic on one patient twice, the probability of a false result decreasing dramatically. But we can actually do better: almost never is an important result only accessible from one line of measurement. There are usually independent paths to information about the putative condition of interest. For example, a suspicious MRI can often be followed up by a PET, or an immunoassay, neither of which involve surgery.
As an aside, science in general has a variation of this problem in that it’s positively biased. Negative results, as a rule, aren’t published. If ten labs run the same experiment and one gets a positive result, we’re going to end up with a paper describing that the existence of the effect in question and zero papers disagreeing with it. Further, there’s no incentive for anyone in science to reproduce each others’ work for the sake of verifying it; in fact, the incentives usually run the opposite way. This is how we end up in a world where only 6 out of 53 “landmark” cancer studies end up later being validated by a major drug company. (To be clear, I think this result is an interesting anecdote, but I do believe that the aggregate knowledge of the field is generally good and has advanced enormously.)
Most cancers are treatable if caught sufficiently early; in fact, stage of detection is usually the single biggest deciding factor for long-term survival. As I’ve said multiple times here before, there is a huge opportunity in primary care. If you are working on this and making significant progress, I would love to meet you and hear more about what you’re doing.