Skip to content
Sem 5 Prep
← BI & Data SciencePrincipal component analysis

Formulas for this chapter

Principal component score

PCk score = w1 x z1 + w2 x z2 + ... + wp x zp where zj is the standardised value of variable j

Placing one observation on a component. The values must be standardised with the same means and standard deviations the loadings were computed from.

wj
Loading of variable j on this component; sign and size say how it contributes
zj
Standardised value: (raw value minus mean) over standard deviation
score
Centred at zero, so positive means above average for this dataset

Variance explained

eigenvalue(k) = ( standard deviation of PCk )^2 % variance = eigenvalue(k) / sum of all eigenvalues = eigenvalue(k) / p on standardised data cumulative % = running total

Reading a summary(prcomp) or an eigenvalue table. Check the eigenvalues sum to p before computing anything.

eigenvalue
Variance captured by that component
p
Number of variables; the total variance on standardised data

How many components to keep

Method 1: keep components until cumulative variance reaches 90 % Method 2: Kaiser, keep every component with eigenvalue >= 1 Scree plot: keep the components before the elbow (fviz_eig)

After the eigenvalue table. Use both, and if they disagree say which you took and what variance was lost.

90 %
Section B's stated threshold; other thresholds are used, so state yours
eigenvalue >= 1
A standardised variable has variance 1, so a lesser component explains less than one raw column

PCA in R

pca_result <- prcomp(data, scale = TRUE, center = TRUE) summary(pca_result) # sd, proportion and cumulative per PC pca_result$x # the scores, one row per observation fviz_eig(pca_result) # scree plot fviz_pca_biplot(pca_result, repel = TRUE)

Always with scale and center TRUE when the variables are in different units, which is nearly always.

scale = TRUE
Standardises spread, so a large-unit variable cannot dominate
center = TRUE
Subtracts each mean
$x
Scores: where each observation sits on each component
Step 2 of 21
The real wordsTheory

Dimension reduction, defined

Dimension reductionThe set of mathematical techniques that reduce the number of variables by replacing them with new meta-variables, each a weighted combination of the originals, while keeping the patterns the originals carried.

Section B's deck states the goal plainly: describe a dataset with fewer, better-constructed variables and still capture the insight.

PCA is the standard procedure for doing it. It is unsupervised: there is no target variable, and nothing is being predicted.

← BackContinue →