2
$\begingroup$

The question titled “How are the standard errors of coefficients calculated in a regression?” is asking how the standard errors of regression coefficient estimates are computed (for example, the output you see from the lm() function in R).

The answer is assuming :

\begin{array}{l} \mathbf{\epsilon} \sim N(0, \sigma^2 \mathbf{I}) \end{array}

I guess standard errors can be calculated without assuming normality of errors (as is true for least squares). It is only when one wants to give confidence intervals that normality may be needed.

Improve this question
$\endgroup$
7
  • $\begingroup$ Your guess is correct. A standard error is calculated without any distributional assumptions at all. Those assumptions are needed only for interpreting the SE probabilistically. What, then, is your question? $\endgroup$ Commented Oct 25 at 12:56
  • $\begingroup$ Well my question was : why then is it assumed that it is normal ? It was more a comment. The assumptions in stats.stackexchange.com/questions/44838/… are then somehow misleading. The linear model does NOT need normality and the demonstration could have been made without writing that the error are normally distributed as the answer does not discuss Confidence Interval but standard errors. $\endgroup$ Commented Oct 25 at 13:41
  • 1
    $\begingroup$ I cannot find any reference to a Normal distribution assumption in that thread. As I wrote initially, standard errors are neither defined nor computed using any distributional assumptions, so it's hard to see what might be misleading about an answer that discusses SEs. $\endgroup$ Commented Oct 25 at 15:58
  • $\begingroup$ @whuber unless I'm completely misreading it, the accepted answer begins: the linear model can be written: \begin{array}{l} \mathbf{y} = \mathbf{X} \mathbf{\beta} + \mathbf{\epsilon} \\ \mathbf{\epsilon} \sim N(0, \sigma^2 \mathbf{I}), \end{array} $\endgroup$ Commented Oct 25 at 16:58
  • 2
    $\begingroup$ Thanks, @Rick. I failed to notice that mainly because (a) neither the words "Normal" or "Gaussian" appear on that page and--more to the point--nothing in the accepted answer ever appeals to a Normality assumption to draw any inferences or conclusions. It's a calculation of variances and standard errors only. Some of the comments do refer to a Bayesian analysis that requires a distributional assumption. $\endgroup$ Commented Oct 25 at 19:19

1 Answer 1

Reset to default
8
$\begingroup$

To answer your question directly, the accepted answer in the thread you reference, does indeed implicitly make the assumption of normally distributed errors:

\begin{array}{l} \mathbf{y} = \mathbf{X} \mathbf{\beta} + \mathbf{\epsilon} \\ \mathbf{\epsilon} \sim N(0, \sigma^2 \mathbf{I}), \end{array}

This is not necessary. For example, note, that the Gauss-Markov theorem only assumes that the $\epsilon$'s:

  • Have zero mean: $\mathbf{E}(\epsilon_{i}) = 0$
  • Have constant variance: $\text{Var}(\epsilon_{i}) = \sigma^{2} < \infty $
  • Are uncorrelated with each other

Yet, proof of the theorem involves derivation of the variance of $\hat{\beta}$. Standard error could be defined from there.

The assumption of normality of the errors makes the least squares solution also the maximum likelihood estimate. From there, the distributions of the parameters can be derived and confidence intervals defined. See this nice explanation or one from this site. Furthermore, since for the least squares case, we arrive at a $t$ pivot, the intervals are “exact” while for the generalized linear model, we’d rely on the asymptotic covariance matrix of $\hat{\beta}$.

Improve this answer
$\endgroup$
2
  • $\begingroup$ Thanks. I was pretty sure my guess was correct and I agree with your answer. It was just strange to see 190 up votes or so when the assumptions are basically misleading. $\endgroup$ Commented Oct 25 at 17:23
  • $\begingroup$ Yeah I think we're just used to going right to normal errors since that can often be part of the model-diagnostic procedure $\endgroup$ Commented Oct 25 at 17:24

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.