Skip to content
Sem 5 Prep
← BI & Data Sciencek-Nearest Neighbours

Formulas for this chapter

Euclidean distance (KNN)

d = sqrt( (x1 - x2)^2 + (y1 - y2)^2 ) Excel: =SQRT(($B$6-B2)^2 + ($C$6-C2)^2)

Step 1 of KNN, once per training row. Normalise every feature first, or the widest column decides the neighbours.

(x1, y1)
The test point being classified
(x2, y2)
A training point with a known label

Min-max normalisation

x' = (x - min) / (max - min) R: preProcess(data, method = c("range"))

Before every distance-based method. The min and max are taken down each column of the training data, and the same figures must be reused on the test rows.

min, max
Smallest and largest value of that feature in the training data
x'
The rescaled value, between 0 and 1

KNN prediction rule

classification: predicted class = mode of the k nearest labels regression: predicted value = mean of the k nearest targets k odd for two classes

Step 3, after the distances are ranked. The same neighbour set serves both tasks; only this line changes.

k
Number of neighbours consulted, chosen on held-back data
mode
The most common label; ties broken by the nearer neighbour
Step 1 of 22
The ideaTheory

Birds of a feather flock together

That is the professor's own phrase for it, and it is the whole algorithm.

You are handed an unlabelled point. Look at the handful of labelled points nearest to it. Whatever most of them are, that is your answer.

No equation is fitted, no coefficients are estimated. The training data is the model.

← BackContinue →