The Theology of Type 1 & Type 2 Errors: Deciding Who is Going to Hell

A large part of my day job is teaching statistics. I teach both undergraduate and graduate statistics classes. Theology--this blog--is really just a side hobby of mine. Little of what I talk about here on this blog makes it into my lectures on multiple regression, factor analysis, and Analysis of Variance.

But it is hard, at times, not to see theological issues emerging in my statistical lectures. Worlds collide was it were. For example, consider how Type 1 and Type 2 errors can help us think about who is going to heaven or hell.

What are Type 1 and Type 2 errors? When researchers look at trends in data sets they have to make a decision about if the trend they are looking at is real or illusory. By "real" I mean that the trend is due to some underlying causal mechanism. However, trends can emerge in data by mere chance. Think of the constellations in the sky. We see patterns up there--the figures of the Zodiac--but the patterns are the product of random forces.

In short, given our knack for seeing order in chaos researchers need tools to determine if a given trend is real or illusory. In the social sciences this tool is called Null Hypothesis testing. The Null Hypothesis is the assumption that the trend you are seeing, despite appearances, is illusory. It is "due to chance." Generally, an Alternative Hypothesis is pitted against the Null Hypothesis. The Alternative Hypothesis is the assumption that the trend is real, due to some systematic relationship between the two variables.

In Null Hypothesis Testing you assess the viability of the Null (by using probabilities) to see if the Null or the Alternative Hypothesis is the best explanation for what you are seeing. Starting with the Null the researcher faces one of two choices: Reject or Fail to Reject the Null. If you reject the Null you think the Alternative hypothesis is the best explanation for the data: You think the trend you are seeing is real. If you Fail to Reject the Null you stick with the Null and conclude that the trend you are seeing is illusory, likely due to chance.

Now what is interesting about all this is that the whole process is error-prone. You could reject the Null (think the trend is real) when, in fact, the trend was illusory. Or, you could think the trend is illusory (fail to reject the null) when, in fact, the trend is real.

In short, there are two kinds of mistakes you can make. The first is called a Type 1 error when you reject the Null (consider the trend to be real) when the Null is true (trend is actually illusory). The second is a Type 2 error when you fail to reject the Null (consider the trend illusory) when the Null is false (the trend is actually real).

Personally, I find this language confusing. An easier way to think of it is to note that Type 1 errors are "false positives": You make a positive claim but are wrong (i.e., you claim the trend is real but it is not). A Type 1 error is like crying wolf. You reject the Null and cry "Eureka!" But you are wrong. You didn't make the discovery you thought you made. Conversely, Type 2 error is a "false negative." Instead of crying wolf a Type 2 error is a Trojan Horse kind of mistake. You didn't raise the alarm, but you should have. You missed the threat (or that trend in the data set).

To illustrate all this I draw the following table on the board a couple of times a year:
Again, that table isn't very easy to understand so I use the following illustration to help my students get the logic of Type 1 and Type 2 errors. Imagine, I say, a guy who has just started dating a girl. He remembers that on their first date she mentioned that her birthday was coming up this week. But the guy can't remember the exact day. It might be today. Or maybe not. Embarrassed to admit to her that he didn't remember he decides to make a guess. He has two choices. When he sees her today he can say "Happy Birthday!" Or he can say nothing, hoping that today isn't her birthday. The reality behind the situation is pretty simple: Either today is her birthday or it isn't.

The four possible outcomes--guesses plotted against reality--are given below:
Saying "Happy Birthday!" when it is not her birthday is like a Type 1 error. It is a false positive: I'm saying it is your birthday when, in fact, it isn't.

Conversely, staying quiet when today is her birthday is like a Type 2 error. It is a false negative: Today is my birthday and you said nothing to me, you missed it.

Now at this point you are probably wondering, what does any of this have to do with theology? Well, the interesting thing in all this is that Type 1 and Type 2 errors are pretty much everywhere. And they often occur when you have to make decisions about people. Think about hiring practices or college admissions. On the front end you have to make a choice: Will they thrive or fail? After the selection (the hire or admission) the reality unfolds. They either do a good job or they don't. They either graduate or they don't. In short, whenever we make decisions about people we often make Type 1 and Type 2 errors. We hire people who flake out on us. Or admit people who can't make the grade. These are Type 1 errors, "false positives." But we also pass on people who would have made great hires. Or we deny admission to students who would have graduated. These are Type 2 errors, the "false negatives."

And here is where the theological application emerges. Christians often divide the world into two groups, the saints and the sinners. The saved and the lost. The church and the world. Think of this as a kind of "admissions decision." Are you "accepted" into the "church" or not?

And these are not trivial considerations. How does the church define where its borders will be? Where does the right hand of fellowship begin and end? How inclusive or exclusive should we be? Who needs to be evangelized? Who is going to heaven and who is going to hell?

Importantly, Type 1 and Type 2 errors will be involved in this process. There will be people we fellowship who, in the eyes of God, stand under judgment. These are Type 1 errors. For example, many of my conservative readers likely think that my recent posts about same-sex marriage are courting a Type 1 error: Extending grace into an area that, in reality, stands under God's judgment. However, we can also make Type 2 errors. We might condemn or disfellowship people when, in fact, God's grace has welcomed these people into the church. In short, while a conservative reader might think I'm making a Type 1 error, I think, given how I see things, that he is making a Type 2 error.

The problem is, this side of heaven no one knows who is right or who is wrong. Which is frustrating for all parties. But I think something more can be said about the matter. Specifically, although we don't know in a given case if we are making a Type 1 or Type 2 error we can choose the kinds of errors we will generally make. And this raises a very interesting question: Knowing we are going to make errors, what kind of errors should we make? What should be our theology of error?

Let me explain what this looks like.

As I said above, Christians, while looking at individuals and groups, make judgments regarding the saintliness of other people. Are these other people "Christians" or not? Likely, these judgments are a mix of doctrinal and moral observations. What do these people believe and what do they do with their lives? Plot these judgments on a horizontal axis, from sinners to saints.

At the same time Christians are making these judgments, God is making his own judgments. Plot God's judgments on a vertical axis, also going from sinners to saints.

With both axes plotted we have something like the following, with the red dots representing people or communities:
A couple of observations about this diagram. What I'm trying to show here is that, while there is some agreement between God and the judging Christian, there is also some discrepancy between human and divine judgments. That is, when humans see someone as virtuous or wicked God, for the most part, agrees. However, the association isn't perfect. We might see someone a very virtuous while God knows him to be hypocrite. Or, we might label someone a "sinner" where God sees this person as a saint. That is, generally vice is vice and virtue is virtue, in heaven and on earth. But there are locations of disagreement. Our perceptions of morality don't always align with God's perspective.

Now, in light of these judgments Christians have to make a choice, grouping people into the categories of Saved or Lost. In statistical language, the point at which this judgment is made is called the selection ratio:
You can think of the selection ratio as a kind of "cut off" score where a decision is made. Like a minimum SAT or ACT score for a college admission. Scores above the cut off are "selected," admitted to the college. Scores below the selection ratio are denied admission.

But the selection ratio isn't the only thing in play. In heaven God has his own opinion about who is Lost or Saved. And God's decision is going to define the ultimate outcome. In statistical language God's decision is called the base rate. God's base rate is represented below:
With these ideas in hand we can now see how the church can make Type 1 and Type 2 errors. Specifically, using some sort of selection ratio the church carves up the world into the Saved and the Lost. The former are called "the church" and the latter are objects of judgment and evangelism. But behind all this is God's base rate, his own judgments about who he will or will not save. And, given that human and divine opinions are not in 100% agreement (due to our human limitations), we have the possibility for error. That is, we might see someone as Lost who, in the end, God will Save. Or, we can see someone as Saved who, in the the end, God will damn. These errors can be seen when we plot the selection ratio and the base rate together:
These errors are just like the Type 1 and Type 2 errors encountered above:
Okay, we are nearing the theological payoff of these observations. Here's the important point: By adjusting the selection ratio we can affect the kinds of errors we will make. We have two choices.

One thing we can do is adjust the selection ratio upward, moving it rightward. This is what we call "raising standards." As the selection ratio moves toward the right fewer and fewer people are selected because the standards are going up. Think again of college admissions. If we raise the SAT/ACT scores we move the selection ratio to the right. This reduces our overall admissions as we become more "selective."

The advantages and disadvantages of increasing the selection ratio are obvious. On the downside we are reducing quantity. But on the upside we are improving quality. And by improving quality we are reducing a kind of error. Specifically, we are reducing our Type 1 error rate. We are making fewer "false positives." By increasing quality standards we aren't admitting people who can't make the cut.

However, and this is the key insight, in decreasing our Type 1 errors we have increased our Type 2 errors. By becoming more selective we've eliminated our false positives but have increased our false negatives. That is, we are turning away more and more students from the school when many of these students would have been successful if we had admitted them.

This is, incidentally, a model for how conservative churches are handling the judgments of Saved and Lost. Conservative churches have increased the selection ratio, they raise holiness and/or doctrinal standards. And they do this to be "conservative," to reduce false positives. They want to be sure to fellowship only the true Christians. They don't want to waste fellowship on the Lost. This strategy is shown below:
By increasing their standards of fellowship the conservative church has become more choosy.

The trouble, as we see in the figure above, is that when you move the selection ratio rightward you are, indeed, decreasing false positives. However, while you are reducing false positives you are increasing your number of false negative (Type 2) errors. That is, by raising standards you are disfellowshiping many Christians whom God will save. You think a lot of people on earth are going to hell when, in fact, they are going to be with you in heaven. In short, heaven is going to be a lot more crowded than your selection ratio led you to believe.

So what happens when you lower standards, becoming more inclusive as we see in many liberal churches? This is what happens when you move the selection ratio leftward:
As you can see, when you move the selection ratio to the left, fellowshiping more and more people, you systematically reduce your false negative errors. By casting such a broad net of fellowship you "catch" all the Saved. But your inclusiveness also catches a lot of the Lost. That is, in reducing your false negatives you've increased your false positives. Just the opposite situation we saw in the conservative church.

Finally we can see the point I made above. That is, although we can never know when and where we are making errors (i.e., we don't know God's base rate) we can, generally speaking, choose the kinds of errors we will make. Specifically, we can increase or decrease our selection ratio. We can become more exclusive or inclusive. More sectarian or more ecumenical. So which direction should we go? Well, one way to make that decision is to think theologically about the kinds of errors related to those decisions. If we become more exclusive we make Type 2--false negative--errors. That is, we disfellowship actual saints. We've become stingy with God's grace. Conversely, if we become more inclusive we make more Type 1--false positive--errors. We extend fellowship to sinners. We've become too profligate with God's grace.

I think these patterns of errors can be evaluated on theological grounds. We can ask, should we be a Type 1 error (inclusive) or Type 2 error (exclusive) church? We have some control over this.

In my personal opinion, I think God wants me to be a Type 1 error person. I should err on the side of grace. True, my liberalism will cause me to extend grace where I shouldn't. I know I'm running a high Type 1 error rate. But I prefer this to the alternative: Running a high Type 2 error rate, being judgmental of people among whom God is at work. And I make these choices for a host of theological reasons. Call this my Nazareth Strategy. When people looked at Jesus and said "Can anything good come out of Nazareth?" they were making Type 2 error. I don't want to make that kind of error. Because, yes, something good can come out of Nazareth and I want to spot it.

But of course, others might see this all differently. Regardless, I think this analysis highlights an important issue:

You are going to make errors about God's grace. You can't control that. So in light of that fact, what kinds of errors are you going to make?

Because you do have some control over that.

On to Part 2: Divine Base Rates and the Great Drama of Salvation