The medical test paradox, and redesigning Bayes' rule

I agree that this is definitely a trend, so I feel like highlighting a great example of educational content that is very dense, concise, and with just a touch of whimsy: 3Blue1Brown on Youtube[0]. He has been doing math education videos on YouTube for years. I just watched his latest video on the medical test paradox and redesigning the Bayes rule to address it[1]. All his videos are absolutely phenomenal - the best educational content I've seen on YouTube. I cannot recommend them highly enough. In my experience it is more akin to reading a (very engaging) textbook than watching a video. I often find myself pausing to think things through and can _feel_ myself grasping at new ideas and learning while watching. [0] [LINK] [1] [VIDEO]

Almost no tests in existence can confirm or deny particular infection. Think of them (and symptoms) more as odds updater. If you have 20 minutes to spare this video [0] by 3 Blue 1 Brown is a brilliant introduction to the concept. [0] [VIDEO]

3Blue1Brown explains it well. [VIDEO] If the population who do _not_ actually have serious cancer is orders of magnitudes larger than those who do then there will be far more people in the false positive group than the true positive group and thus the vast majority of operations will be unnecessary.

I really like this video [1] from 3blue1brown, where he proposes to think about significance as a way to update the probability. One positive test (or in this analog a study) updates the probability by X % and thus you nearly always need more tests (or studies) for a 'meaningful' judgment. [1] [VIDEO]

For me, taking the logarithm makes it even more intuitive somehow - the test becomes something like a modifier in an RPG-style game. In your example, the test gives +2 to the possibility of having a disease, a more powerful test could be +3, etc. Adding all the different +/- modifiers works just as well, and all this math is easy to do in one’s head. Edit: also, obligatory 3blue1brown video [VIDEO]

The advantage of the odds formulation is that you can compute posterior odds in your head, whereas computing the posterior probabilities via the standard form of the theorem (that you wrote down here) involves an unpleasant normalization factor. E.g. for the example in [LINK] : prior odds of infection = 2 : 98 × likelihood ratio = 95 : 5 = posterior odds ≈ 200 : 500 and thus the probabilities (if you actually need them at this point) are ≈ (2/7, 5/7). See also 3Blue1Brown’s exposition: [VIDEO] .

Usually the detectors are only called in once a basic "smell test" has failed. Those tests are imperfect, yes, but Bayesian probability tells us how to work out the rest. I have 0 trouble believing that the prior probability of an unscrupulous individual offloading an unpleasant and perceived-as-just-ceremonial duty to the "thinking machine" is around 20%. See: [VIDEO]