Test of confidence intervals?In one of my assignments I have to "test" if the confidence intervals for a set of parameters in a mixed effect model is accurate. I'm asked to simulate from fittet parameters and there after refit them using the same model many times, and lastly take 2.5% and 97.5% quantiles of them and compare with the original CIs. My question is, how does this procedure in anyway measure how accurate my original confidence intervals are?

Question

Test of confidence intervals?In one of my assignments I have to &quot;test&quot; if the confidence intervals for a set of parameters in a mixed effect model is accurate. I&#039;m asked to simulate from fittet parameters and there after refit them using the same model many times, and lastly take 2.5% and 97.5% quantiles of them and compare with the original CIs. My question is, how does this procedure in anyway measure how accurate my original confidence intervals are?

Danielle Gilbert · Accepted Answer

Step 1Each time you simulate data from fitted parameters you find another estimate of the parameter. If the original 95% CI is valid, about 95% of these new estimates ought to lie in the original CI.You must be studying, or about to study, parametric bootstrapping. There are so many different formulations of this idea that I hesitate to get into a theoretical discussion, without knowing the particulars of your course and text, for fear of causing additional confusion.Take a very simple case. I have a sample of   n  =  36 observations from   N  o  r  m  (  μ  ,  σ  =  10  )  .. Suppose             X      ¯        =  105.9  ,, so that a 95% z-interval for   μ is   106.9  ±  1.96  (  10      /    6  ) or (103.6,110.2)Now I take a large number, say   B  =  100000  ,, of samples of size 36 from Norm(106.9,10) and use R to carry out the procedure you describe.  B  =      10    5    ;  m  u  .  e  s  t  =  106.9  ;  s  g  =  10  ;  n  =  36    R  D  T  A  =  m  a  t  r  i  x  (  r  n  o  r  m  (  B  ∗  n  ,  m  u  .  e  s  t  ,  s  g  )  ,  n  r  o  w  =  B  )    x  .  b  a  r  =  r  o  w  M  e  a  n  s  (  R  D  T  A  )  each row of B x n matrix is a sample    quantile  (  x  .  b  a  r  ,  c  (  .025  ,  .975  )  )    2.5  %     97.5  %    103.6111     110.1773Step 2The result is not far from the original CI (103.6,110,2). In this trivial case, the agreement is not surprising because we are just re-establishing by simulation that             X      ¯        ∼  N  o  r  m  (  μ  ,  σ      /        n    )  .In more complex cases, modifications must be made in the procedure, especially when dealing with distributions and estimators that have heavily skewed distributions.

Answered question

Answer & Explanation

New Questions in College Statistics