Skip to content
Sem 5 Prep
← BI & Data Sciencek-Nearest Neighbours

Formulas for this chapter

Euclidean distance (KNN)

d = sqrt( (x1 - x2)^2 + (y1 - y2)^2 ) Excel: =SQRT(($B$6-B2)^2 + ($C$6-C2)^2)

Step 1 of KNN, once per training row. Normalise every feature first, or the widest column decides the neighbours.

(x1, y1)
The test point being classified
(x2, y2)
A training point with a known label

Min-max normalisation

x' = (x - min) / (max - min) R: preProcess(data, method = c("range"))

Before every distance-based method. The min and max are taken down each column of the training data, and the same figures must be reused on the test rows.

min, max
Smallest and largest value of that feature in the training data
x'
The rescaled value, between 0 and 1

KNN prediction rule

classification: predicted class = mode of the k nearest labels regression: predicted value = mean of the k nearest targets k odd for two classes

Step 3, after the distances are ranked. The same neighbour set serves both tasks; only this line changes.

k
Number of neighbours consulted, chosen on held-back data
mode
The most common label; ties broken by the nearer neighbour
Step 4 of 22
Quick checkTheory

The class's grey point

The slide's own worked case. A grey point must be classified into lime green, green or orange. The distances to the four labelled points come out as:

to lime green 21 to lime green 24 to orange 31 to green 45

With K = 3, what is the answer, and what are the vote counts?

← BackContinue →