odcinaknr

2022-10-02

Test of confidence intervals?
In one of my assignments I have to "test" if the confidence intervals for a set of parameters in a mixed effect model is accurate. I'm asked to simulate from fittet parameters and there after refit them using the same model many times, and lastly take 2.5% and 97.5% quantiles of them and compare with the original CIs. My question is, how does this procedure in anyway measure how accurate my original confidence intervals are?

Danielle Gilbert

Step 1
Each time you simulate data from fitted parameters you find another estimate of the parameter. If the original 95% CI is valid, about 95% of these new estimates ought to lie in the original CI.
You must be studying, or about to study, parametric bootstrapping. There are so many different formulations of this idea that I hesitate to get into a theoretical discussion, without knowing the particulars of your course and text, for fear of causing additional confusion.
Take a very simple case. I have a sample of $n=36$ observations from $Norm\left(\mu ,\sigma =10\right).$. Suppose $\overline{X}=105.9,$, so that a 95% z-interval for $\mu$ is $106.9±1.96\left(10/6\right)$ or (103.6,110.2)Now I take a large number, say $B=100000,$, of samples of size 36 from Norm(106.9,10) and use R to carry out the procedure you describe.

Step 2
The result is not far from the original CI (103.6,110,2). In this trivial case, the agreement is not surprising because we are just re-establishing by simulation that $\overline{X}\sim Norm\left(\mu ,\sigma /\sqrt{n}\right).$
In more complex cases, modifications must be made in the procedure, especially when dealing with distributions and estimators that have heavily skewed distributions.

Do you have a similar question?