Why are errors independent but residuals dependent?Sum of the residuals within a random sample is necessarily zero, and thus the residuals are necessarily not independent. But also we assume that E(ϵ)=0. Why doesn't it imply errors are also not independent?

Question

Why are errors independent but residuals dependent?Sum of the residuals within a random sample is necessarily zero, and thus the residuals are necessarily not independent. But also we assume that E(ϵ)=0. Why doesn&#039;t it imply errors are also not independent?

optativaspv · Accepted Answer

In regression, residuals are calculated based on a fitted model for which the underlying parameters are estimated from the data we observed, because those underlying parameters are unknown to us. For this reason, residuals are not independent: a constraint is imposed on the model fit to make the estimated parameters uniquely determined (as in the case of ordinary least squares fitting in linear regression).This speaks to a subtle but important property of residuals: they are in a sense estimates or realizations of error conditional on the assumption that the true error is faithfully represented by the data you observed. Error in a model is intended to capture natural random variation of the response (dependent) variable not explained by the predictors (independent variables). But a residual could be calculated from any model fit and it need not be true to this underlying error.

joyoshibb · Accepted Answer

The simplest case is this:      X    1    ,  …  ,      X    n    are uncorrelated and have expected value μ and variance       σ    2    .      X    ¯    =  (      X    1    +  ⋯  +      X    n    )  /  n therefore has expected value   μ and variance       σ    2    /  n  .The errors are       ε    i    =      X    i    −  μ  . The sum of the errors has variance       σ    2    /  n  ,, so the sum is not zero.The residuals are                       ε                    ^        i    =      X    i    −      X    ¯    .. The sum of these is necessarily zero, so these are negatively correlated.

Why are errors independent but residuals dependent? As far i know the sum of the residuals within a random sample is necessarily zero, and thus the residuals are necessarily not independent. But also we assume that E(epsilon)=0. Why doesn't it imply errors are also not independent?

Answered question

Answer & Explanation

New Questions in High school statistics