Anthony's Science Blog

I woke up and suddenly understood Bayesian statistics…

March 2, 2008 – 1:44 pm

It sounds crazy, but that’s pretty much what happened. I woke up too early (about 6) and as I lay in bed, my mind drifted to a Bayesian statistics example that I had read on Friday afternoon. I had read all of the parts of it and understood the sentences and concepts presented (for about the 10th time; I’ve been curious about this for years) but it wasn’t until this morning that I felt like I got it. I’ve always known it has something to do with prior information and it’s a computationally intensive technique, but this morning I was finally able to answer the question: why do people bother doing something this abstruse?

For me, a couple things made it click. First, the example, which was in Introduction to Applied Bayesian Statistics and Estimation for Social Scientists (pp. 47-49; kudos to author Scott Lynch). It had to do with a young woman who is pregnant and takes a pregnancy test that comes out positive. But the test has a known error rate for false positives and negatives. Ok, great, so we do some math and find out the chances that she is pregnant. But here’s the kicker! Let’s say we also know that she only had sex once and we know the likelihood of pregnancy from a single encounter (which is relatively low). That is relevant prior information and the Bayesian methods have a way of incorporating that, which is like, pretty cool. When we incorporate that information, it turns out that our expectation of pregnancy should be a lot lower.

Another important part of the puzzle that helped me put this simple pregnancy example in perspective is something I found out while learning a lot of other multivariate statistics recently: a lot of the hoopla about Bayesian techniques is not necessarily about the inclusion of prior information, but instead about the computational techniques that are used to estimate model parameters (computationally intense resampling techniques like Markov chain Monte Carlo methods). (Apparently a lot of models don’t even have much prior information.) And to put that in perspective, model parameters are the core of statistics: every measure of central tendency, estimation of uncertainty and model coefficient has a model parameter and the estimation of those is pretty much what statistics is.

So, perhaps I’m just slow and dense, or perhaps the concepts and philosophy behind Bayesian statistics are actually challenging and subtle! I still have no idea how to implement this stuff or most of the details, but I feel like when I read about it now, I will understand why people bother. Either way, it’s about time, and I’m looking forward to exploring more.