Review exercises

Question 1

Suppose that you want to compare the means of \(2\) groups, in which situation is Student’s t-test a good approach:

We need the following: (1) it is reasonable to assume that there are no outliers and (2) for moderate and small sample sizes the sample distribution should be at least approximately normal. We need the following: (1) it is reasonable to assume that there are no outliers and (2) for moderate and small sample sizes the sample distribution should be at least approximately normal. If these criteria are met then we should test if the variances are the same between groups. If we can't reject the null, then we can use Student's t-test. We should do the following tests: (1) test if there are no outliers, (2) test if the data are normally distributed (one test per sample) and (3) test if the variances are the same. If we can't reject the null for all tests, then we should use Student's t-test. We need the following: (1) it is reasonable to assume that there are no outliers, (2) the variances of the two groups are known to be the same (or at least it is reasonable to assume that this is the case) and (3) for moderate and small sample sizes the sample distribution should be at least approximately normal.

Question 2

Suppose that you want to compare the means of \(2\) groups, in which situation is Welsh’s t-test a good approach:

When it is reasonable to assume that there are no outliers. When it is reasonable to assume that the sample distribution is at least approximately normal with not many outliers. When it is reasonable to assume that there are no outliers and the sample size is sufficiently large We need the following: (1) it is reasonable to assume that there are no outliers, (2) the variances of the two groups are known to be the same (or at least it is reasonable to assume that this is the case).

Question 3

Suppose that you want to compare the means of \(2\) groups, in which situation is Wilcoxon rank sum test inadequate:

When there are no outliers. When the distribution is not symmetric and the sample is large. When it is reasonable to assume that there are no outliers and the sample distribution should be at least approximately normal. When it is reasonable to assume that there are no outliers and the variances are the same.

Question 4

A researcher administers many hypothesis tests as part of a research project. He finds that, of \(250\) tests \(11\) were significant at the \(5\%\) level. The researcher feels very proud of this fact and thinks that he makes a big discovery. What do you think of this situation?

It is very interesting that \(11\) tests are significant. It is interesting that \(11\) tests are significant but since we did many tests we should find a theoretical argument for our analysis. One would expect \(12.5\) (\(5\%\) of \(250\)) significant tests even in the purely null case, merely by chance, so we should find at least \(13\) significant tests for the result to be interesting. One would expect \(12.5\) (\(5\%\) of \(250\)) significant tests even in the purely null case, merely by chance. So, finding only \(11\) significant results is actually somewhat disappointing.