Euclidean distance (KNN)
d = sqrt( (x1 - x2)^2 + (y1 - y2)^2 )
Excel: =SQRT(($B$6-B2)^2 + ($C$6-C2)^2)
Step 1 of KNN, once per training row. Normalise every feature first, or the widest column decides the neighbours.
- (x1, y1)
- The test point being classified
- (x2, y2)
- A training point with a known label
Min-max normalisation
x' = (x - min) / (max - min)
R: preProcess(data, method = c("range"))
Before every distance-based method. The min and max are taken down each column of the training data, and the same figures must be reused on the test rows.
- min, max
- Smallest and largest value of that feature in the training data
- x'
- The rescaled value, between 0 and 1
KNN prediction rule
classification: predicted class = mode of the k nearest labels
regression: predicted value = mean of the k nearest targets
k odd for two classes
Step 3, after the distances are ranked. The same neighbour set serves both tasks; only this line changes.
- k
- Number of neighbours consulted, chosen on held-back data
- mode
- The most common label; ties broken by the nearer neighbour