Skip to content
Sem 5 Prep
← BI & Data ScienceMultiple linear regression and reading summary(lm)

Formulas for this chapter

The fitted model, and t

y = b0 + b1*x1 + b2*x2 + ... + bp*xp t = Estimate / Std. Error H0: the coefficient is zero; p < 0.05 => significant

Reading the coefficients block. Every slope is a partial effect, so always add the holding-constant clause.

b0
Intercept; the prediction when every predictor is zero, often not meaningful
bj
Change in y per one unit of xj, holding the other predictors constant
Std. Error
How much the estimate would vary on resampling

Fit and precision

R^2 = 1 - SS_residual / SS_total, SS_total = sum (y - ybar)^2 Adj R^2 = 1 - (1 - R^2) x (n - 1) / (n - p - 1) RSE = sqrt( SS_residual / (n - p - 1) ) df = n - p - 1

Judging how much of the variation is explained and how large a typical error is. Quote the adjusted figure in a multiple regression.

n
Number of observations
p
Number of predictors, excluding the intercept
RSE
Average prediction error, in the units of y; read it against y's own scale

The F test

F = MS_regression / MS_residual = (SS_reg / p) / (SS_res / (n - p - 1)) H0: every slope coefficient is zero

Judging the model as a whole. A significant F with an insignificant t on one predictor is normal.

MS_regression
SS explained divided by p
MS_residual
SS unexplained divided by n - p - 1; its square root is the RSE

VIF, for multicollinearity

VIF(xj) = 1 / ( 1 - R^2 of xj regressed on the other predictors ) VIF > 10 high multicollinearity | VIF < 5 comfortable in business standard error inflates by sqrt(VIF)

Before trusting any individual coefficient, and always when a predictor is insignificant despite a strong model.

R^2 of xj
How well the other predictors already explain this one
sqrt(VIF)
The factor by which the coefficient's standard error is widened
Step 4 of 25
The real wordsTheory

Interpreting one coefficient

The template sentence, and it is worth memorising word for word.

"Each extra 1,000 lb of weight reduces mileage by 3.88 mpg, holding horsepower constant."

Three parts: the unit of the predictor, the unit of the outcome, and the holding-constant clause. Drop the third and the sentence is wrong, because in a multiple regression the coefficient is a partial effect.

The intercept is the predicted outcome when every predictor is zero, and is often meaningless on its own: a car of zero weight and zero horsepower does not exist.

← BackContinue →