Skip to content
Sem 5 Prep
← BI & Data Sciencek-Nearest Neighbours

Formulas for this chapter

Euclidean distance (KNN)

d = sqrt( (x1 - x2)^2 + (y1 - y2)^2 ) Excel: =SQRT(($B$6-B2)^2 + ($C$6-C2)^2)

Step 1 of KNN, once per training row. Normalise every feature first, or the widest column decides the neighbours.

(x1, y1)
The test point being classified
(x2, y2)
A training point with a known label

Min-max normalisation

x' = (x - min) / (max - min) R: preProcess(data, method = c("range"))

Before every distance-based method. The min and max are taken down each column of the training data, and the same figures must be reused on the test rows.

min, max
Smallest and largest value of that feature in the training data
x'
The rescaled value, between 0 and 1

KNN prediction rule

classification: predicted class = mode of the k nearest labels regression: predicted value = mean of the k nearest targets k odd for two classes

Step 3, after the distances are ranked. The same neighbour set serves both tasks; only this line changes.

k
Number of neighbours consulted, chosen on held-back data
mode
The most common label; ties broken by the nearer neighbour
Step 2 of 22
The real wordsTheory

KNN, defined

k-nearest neighboursA supervised learning algorithm used for classification (mostly) and regression. Its principle is that similar data points tend to exist in close proximity to each other in the feature space, so if a sample's nearest neighbours share a characteristic, the sample probably does too.

It is supervised: the neighbours have labels, and without them there is nothing to vote with.

kThe count of nearest neighbours consulted. Chosen by the analyst before predicting.
← BackContinue →