An account on server A is more expensive than an account on server B. However, server A is faster. To see whether it's optimal to go with the faster but more expensive server, a manager needs to know how much faster it is . A certain computer algorithm is executed 20 times on server A and 30 times on server B with the following results
Nico Patterson
Answered question
2022-11-10
Statistics and confidence - intervals An account on server A is more expensive than an account on server B. However, server A is faster. To see whether it's optimal to go with the faster but more expensive server, a manager needs to know how much faster it is . A certain computer algorithm is executed 20 times on server A and 30 times on server B with the following results,
A 95% confidence interval for the difference between the mean execution times on server A and server B is [-1.4,-0.2] . Is there a significant difference between the two servers? (a) Use the confidence interval above to conduct a two-sided test at the 1% level of significance. (b) Compute a p-value of the two-sided test in (a). (c) Is server A really faster? How strong is the evidence? Formulate the suitable hypothesis and alternative and compute the corresponding p-value.
is not in the acceptance range of -2.686 and 2.686 so we reject the null hypothesis b) therefore the results show that there is a highly significant difference between the two servers. c)
, ,
therefore there is highly significant evidence and we reject Conclusions: From a), b) and c) we can say that there is highly significant evidence that there is a difference between the two servers and that server A is faster than server B.
Demarion Ortega
Beginner2022-11-12Added 2 answers
Step 1 Setup: Assume that both samples
are i.i.d. observations coming from a normal distribution with unknown mean and variance. Here and . This can in short be formulated as are realizations from independent random variables and
Let and denote the sample means, i.e.
Then is estimating the true mean , and the difference of means is of course estimated by . Step 2 Assumption of Equal Variances: We furthermore assume that the variance in the two groups are equal, i.e. and we denote the common variance by just . If denotes the sample variance of the ith sample, i.e. then is the estimator of the common variance . Now one can show that
where . Knowing that T follows a t-distribution with f degrees of freedom is exactly what we need in order to find confidence intervals and perform tests. 99% confidence interval: A 99% confidence interval is obtained by manipulating (1) and we obtain that
where is the 99.5% fractile of a t-distribution with f degrees of freedom and
Step 3 Hypothesis testing: We wish to test the statistical hypothesis that the two means are equal meaning that Server A and Server B are equally fast. The hypothesis is written as
and is of course equivalent to testing that . Under the hypothesis, (1) becomes
and as numerically large values are critical for the hypothesis we obtain the p-value
where denotes the CDF of a t-distribution with f degress of freedom. We accept H if and reject it if . Checking whether actually corresponds to checking whether is in the 99% confidence interval or not. Model Checking: This answer is based on some crucial assumption which, of course, have to be checked. That this is indeed two i.i.d. samples from normal distributions should of course be checked. Futhermore, the assumption of equal variances should be checked. This can be checked by performing an F-test.