3) Select a probability of error level
TYPE I AND TYPE II ERRORS
Even in the best research project, there is always a possibility (hopefully a small one) that the researcher will make a mistake regarding the relationship between the two variables.
There are two possible mistakes or errors.
The first is called a Type I error.
This occurs when the researcher assumes that a relationship exists when in fact the evidence is that it does not.
In a Type I error, the researcher should accept the null hypothesis and reject the research hypothesis, but the opposite occurs.
The probability of committing a Type I error is called alpha.
The second is called a Type II error.
This occurs when the researcher assumes that a relationship does not exist when in fact the evidence is that it does.
In a Type II error, the researcher should reject the null hypothesis and accept the research hypothesis, but the opposite occurs.
The probability of committing a Type II error is called beta.
Generally, reducing the possibility of committing a Type I error increases the possibility of committing a Type II error and vice versa, reducing the possibility of committing a Type II error increases the possibility of committing a Type I error.
Researchers generally try to minimize Type I errors, because when a researcher assumes a relationship exists when one really does not, things may be worse off than before. In Type II errors, the researcher misses an opportunity to confirm that a relationship exists, but is no worse off than before.
In this example, which type of error would you prefer to commit?
Research Hypothesis: El Nino has reduced crop yields in County X, making it eligible for government disaster relief.
Null Hypothesis: El Nino has not reduced crop yields in County X, making it ineligible for government disaster relief.
If a Type I error is committed, then the County is assumed to be eligible for disaster relief, when it really is not (the null hypothesis should be accepted, but it is rejected). The government may be spending disaster relief funds when it should not, and taxes may be raised.
If a Type II error is committed, then the County is assumed to be ineligible for disaster relief, when it really is eligible (the null hypothesis should be accepted, but it is rejected). The government may not spend disaster relief funds when it should, and farmers may go into bankruptcy.
In this example, which type of error would you prefer to commit?
Research Hypothesis: The new drug is better at treating heart attacks than the old drug
Null Hypothesis: The new drug is no better at treating heart attacks than the old drug
If a Type I error is committed, then the new drug is assumed to be better when it really is not (the null hypothesis should be accepted, but it is rejected). People may be treated with the new drug, when they would have been better off with the old one.
If a Type II error is committed, then the new drug is assumed to be no better when it really is better (the null hypothesis should be rejected, but it is accepted). People may not be treated with the new drug, although they would be better off than with the old one.
SELECT A PROBABILITY OF ERROR LEVEL (ALPHA LEVEL)
Researchers generally specify the probability of committing a Type I error that they are willing to accept, i.e., the value of alpha.
In the social sciences, most researchers select an alpha=.05. This means that they are willing to accept a probability of 5% of making a Type I error, of assuming a relationship between two variables exists when it really does not.
In research involving public health, however, an alpha of .01 is not unusual. Researchers do not want to have a probability of being wrong more than 0.1% of the time, or one time in a thousand.
If the relationship between the two variables is strong (as assessed by a Measure of Association), and the level chosen for alpha is .05, then moderate or small sample sizes will detect it. As relationships get weaker, however, and/or as the level of alpha gets smaller, larger sample sizes will be needed for the research to reach statistical significance.