How to Define the Types of Statistical Errors

You will now learn about the errors that can be made in hypothesis testing.


In general, we can have two types of errors – type I error and type II error.
Sounds a bit boring, but this will be a fun lecture, I promise!


First we will define the problems, and then we will see some interesting examples.


Type I error is when you reject a true null hypothesis and is the more serious error. It is also called ‘a false positive’. The probability of making this error is alpha – the level of significance. Since you, the researcher, choose the alpha, the responsibility for making this error lies solely on you.


Type II error is when you accept a false null hypothesis. The probability of making this error is denoted by beta. Beta depends mainly on sample size and population variance. So, if your topic is difficult to test due to hard sampling or has high variability, it is more likely to make this type of error. As you can imagine, if the data set is hard to test, it is not your fault, so Type II error is considered a smaller problem.


We should also mention that the probability of rejecting a false null hypothesis is equal to 1 minus beta. This is the researcher’s goal – to reject a false null hypothesis. Therefore, 1 minus beta is called the power of the test. Generally, researchers increase the power of a test by increasing the sample size.


This is a common table statisticians use to summarize the types of errors.


Now, let’s see an example that I heard from my professor back when I was studying statistics in university.


You are in love with this girl from the other class, but are unsure if she likes you.


There are two errors you can make.


First, if she likes you back and you don’t invite her out, you are making the type I error.


The null hypothesis in this situation is: she likes you back. It turns out that she really did like you back. Unfortunately, you did not invite her out, because after testing the situation, you wrongly thought the null hypothesis was false. In other words, you made a type I error – you rejected a true null hypothesis and lost your chance. It is a very serious problem, because you could have been made for each other, but you didn’t even try.


Now imagine another situation. She doesn’t like you back, but you go and invite her out. The null hypothesis is still: she likes you back, but this time it is false. In reality she doesn’t really like you back, that is. However, after testing, you accept the null hypothesis and wrongly go and invite her out. She tells you she has a boyfriend that is much older, smarter and better at statistics than you and turns her back.


You made a type II error – accepted a false null hypothesis. However, it is no big deal, as you go back to your normal life without her and soon forget about this awkward situation.