Skip to content
Sem 5 Prep
← BI & Data ScienceMultiple linear regression and reading summary(lm)

Formulas for this chapter

The fitted model, and t

y = b0 + b1*x1 + b2*x2 + ... + bp*xp t = Estimate / Std. Error H0: the coefficient is zero; p < 0.05 => significant

Reading the coefficients block. Every slope is a partial effect, so always add the holding-constant clause.

b0
Intercept; the prediction when every predictor is zero, often not meaningful
bj
Change in y per one unit of xj, holding the other predictors constant
Std. Error
How much the estimate would vary on resampling

Fit and precision

R^2 = 1 - SS_residual / SS_total, SS_total = sum (y - ybar)^2 Adj R^2 = 1 - (1 - R^2) x (n - 1) / (n - p - 1) RSE = sqrt( SS_residual / (n - p - 1) ) df = n - p - 1

Judging how much of the variation is explained and how large a typical error is. Quote the adjusted figure in a multiple regression.

n
Number of observations
p
Number of predictors, excluding the intercept
RSE
Average prediction error, in the units of y; read it against y's own scale

The F test

F = MS_regression / MS_residual = (SS_reg / p) / (SS_res / (n - p - 1)) H0: every slope coefficient is zero

Judging the model as a whole. A significant F with an insignificant t on one predictor is normal.

MS_regression
SS explained divided by p
MS_residual
SS unexplained divided by n - p - 1; its square root is the RSE

VIF, for multicollinearity

VIF(xj) = 1 / ( 1 - R^2 of xj regressed on the other predictors ) VIF > 10 high multicollinearity | VIF < 5 comfortable in business standard error inflates by sqrt(VIF)

Before trusting any individual coefficient, and always when a predictor is insignificant despite a strong model.

R^2 of xj
How well the other predictors already explain this one
sqrt(VIF)
The factor by which the coefficient's standard error is widened
Step 1 of 25
The ideaTheory

Splitting the credit between causes

Sales went up. The advertising budget went up, the shop got bigger, and two more staff were hired. Which of the three did it?

Multiple regression is the machine that divides the credit. It gives each predictor a number saying how much the outcome moves when that one changes and the others stay still.

Everything else in this chapter is how to tell whether those numbers are trustworthy, which is what the rest of the output is for.

← BackContinue →