Skip to content
Sem 5 Prep
← BI & Data ScienceMultiple linear regression and reading summary(lm)

Formulas for this chapter

The fitted model, and t

y = b0 + b1*x1 + b2*x2 + ... + bp*xp t = Estimate / Std. Error H0: the coefficient is zero; p < 0.05 => significant

Reading the coefficients block. Every slope is a partial effect, so always add the holding-constant clause.

b0
Intercept; the prediction when every predictor is zero, often not meaningful
bj
Change in y per one unit of xj, holding the other predictors constant
Std. Error
How much the estimate would vary on resampling

Fit and precision

R^2 = 1 - SS_residual / SS_total, SS_total = sum (y - ybar)^2 Adj R^2 = 1 - (1 - R^2) x (n - 1) / (n - p - 1) RSE = sqrt( SS_residual / (n - p - 1) ) df = n - p - 1

Judging how much of the variation is explained and how large a typical error is. Quote the adjusted figure in a multiple regression.

n
Number of observations
p
Number of predictors, excluding the intercept
RSE
Average prediction error, in the units of y; read it against y's own scale

The F test

F = MS_regression / MS_residual = (SS_reg / p) / (SS_res / (n - p - 1)) H0: every slope coefficient is zero

Judging the model as a whole. A significant F with an insignificant t on one predictor is normal.

MS_regression
SS explained divided by p
MS_residual
SS unexplained divided by n - p - 1; its square root is the RSE

VIF, for multicollinearity

VIF(xj) = 1 / ( 1 - R^2 of xj regressed on the other predictors ) VIF > 10 high multicollinearity | VIF < 5 comfortable in business standard error inflates by sqrt(VIF)

Before trusting any individual coefficient, and always when a predictor is insignificant despite a strong model.

R^2 of xj
How well the other predictors already explain this one
sqrt(VIF)
The factor by which the coefficient's standard error is widened
Step 2 of 25
The real wordsTheory

Simple against multiple

Simple linear regressionOne predictor. y = b0 + b1x. The line of best fit on a scatter plot.
Multiple linear regressionTwo or more predictors. y = b0 + b1x1 + b2x2 + ... The coefficient on each is its effect holding the others constant.
R: model <- lm(mpg ~ wt + hp, data = mtcars) summary(model)

Read the tilde as "explained by". Predictors are joined with a plus.

← BackContinue →