The fitted model, and t
y = b0 + b1*x1 + b2*x2 + ... + bp*xp
t = Estimate / Std. Error
H0: the coefficient is zero; p < 0.05 => significant
Reading the coefficients block. Every slope is a partial effect, so always add the holding-constant clause.
- b0
- Intercept; the prediction when every predictor is zero, often not meaningful
- bj
- Change in y per one unit of xj, holding the other predictors constant
- Std. Error
- How much the estimate would vary on resampling
Fit and precision
R^2 = 1 - SS_residual / SS_total, SS_total = sum (y - ybar)^2
Adj R^2 = 1 - (1 - R^2) x (n - 1) / (n - p - 1)
RSE = sqrt( SS_residual / (n - p - 1) )
df = n - p - 1
Judging how much of the variation is explained and how large a typical error is. Quote the adjusted figure in a multiple regression.
- n
- Number of observations
- p
- Number of predictors, excluding the intercept
- RSE
- Average prediction error, in the units of y; read it against y's own scale
The F test
F = MS_regression / MS_residual
= (SS_reg / p) / (SS_res / (n - p - 1))
H0: every slope coefficient is zero
Judging the model as a whole. A significant F with an insignificant t on one predictor is normal.
- MS_regression
- SS explained divided by p
- MS_residual
- SS unexplained divided by n - p - 1; its square root is the RSE
VIF, for multicollinearity
VIF(xj) = 1 / ( 1 - R^2 of xj regressed on the other predictors )
VIF > 10 high multicollinearity | VIF < 5 comfortable in business
standard error inflates by sqrt(VIF)
Before trusting any individual coefficient, and always when a predictor is insignificant despite a strong model.
- R^2 of xj
- How well the other predictors already explain this one
- sqrt(VIF)
- The factor by which the coefficient's standard error is widened