Example can be taken of court verdict:

Null Hypothesis: The accused person is questioned in the court proceedings assuming that there are chances of him/her being guilty

The lawyer of the accused person tries to produce evidence and arguments in order to prove the person not guilty.

The job of the prosecutor is to prove him guilty beyond a reasonable doubt.

Now, if the prosecutor is not able to provide sufficient evidence that the person is guilty then the judge/jury declares the person as 'Not guilty', because evidence doesn't support that the person is guilty. But the judge doesn't declare as the person is innocent, it's because, even the judge / jury isn't sure that the person is innocent. So, not being guilty based on evidence doesn't prove that the person is innocent.

Statistical Reasoning

My goal is to statistically examine whether the coin is biased or unbiased. As you might have guessed, it’s easier to reject the hypothesis that the coin is unbiased that try to prove it is unbiased. Hence, we start perform an experiment and perform z-test with the below setup —

Null Hypothesis (H0): p = 0.5 ( p = true probability of heads)

Alternate Hypothesis: p not = 0.5

Experiment: We tossed the coin 5 times and we got 4 heads.

We can perform z-test where we assume Null hypothesis is true.

z = (observed probability of heads — true probability of heads) / standard_error

z = (0.8–0.5) / (sqrt(0.5*(1–0.5) /5 ) ~ 1.34

From the normal distribution charts, we can get that —p-value ~ 18% ( since this is > 5%, we should conclude that we fail to reject the null hypothesis.)

Instead, if I were to say that I accept the null hypothesis ? What’s wrong with that ?

To illustrate the problem with this, let’s repeat above z-test with a twist. This time, we will be using a different pair of null and alternate hypothesis.

Null Hypothesis (H0): p = 0.55 ( p = true probability of heads)

Alternate Hypothesis (H1): p != 0.55

Again, on the basis of same results, we can compute z-value and p-value.

z = ( 0.8–0.55) / (sqrt(0.55*(1–0.55) /5 ) ~ 1.12