Principal component score
PCk score = w1 x z1 + w2 x z2 + ... + wp x zp
where zj is the standardised value of variable j
Placing one observation on a component. The values must be standardised with the same means and standard deviations the loadings were computed from.
- wj
- Loading of variable j on this component; sign and size say how it contributes
- zj
- Standardised value: (raw value minus mean) over standard deviation
- score
- Centred at zero, so positive means above average for this dataset
Variance explained
eigenvalue(k) = ( standard deviation of PCk )^2
% variance = eigenvalue(k) / sum of all eigenvalues
= eigenvalue(k) / p on standardised data
cumulative % = running total
Reading a summary(prcomp) or an eigenvalue table. Check the eigenvalues sum to p before computing anything.
- eigenvalue
- Variance captured by that component
- p
- Number of variables; the total variance on standardised data
How many components to keep
Method 1: keep components until cumulative variance reaches 90 %
Method 2: Kaiser, keep every component with eigenvalue >= 1
Scree plot: keep the components before the elbow (fviz_eig)
After the eigenvalue table. Use both, and if they disagree say which you took and what variance was lost.
- 90 %
- Section B's stated threshold; other thresholds are used, so state yours
- eigenvalue >= 1
- A standardised variable has variance 1, so a lesser component explains less than one raw column
PCA in R
pca_result <- prcomp(data, scale = TRUE, center = TRUE)
summary(pca_result) # sd, proportion and cumulative per PC
pca_result$x # the scores, one row per observation
fviz_eig(pca_result) # scree plot
fviz_pca_biplot(pca_result, repel = TRUE)
Always with scale and center TRUE when the variables are in different units, which is nearly always.
- scale = TRUE
- Standardises spread, so a large-unit variable cannot dominate
- center = TRUE
- Subtracts each mean
- $x
- Scores: where each observation sits on each component