Skip to content
Sem 5 Prep
← BI & Data Sciencek-Nearest Neighbours

Formulas for this chapter

Euclidean distance (KNN)

d = sqrt( (x1 - x2)^2 + (y1 - y2)^2 ) Excel: =SQRT(($B$6-B2)^2 + ($C$6-C2)^2)

Step 1 of KNN, once per training row. Normalise every feature first, or the widest column decides the neighbours.

(x1, y1)
The test point being classified
(x2, y2)
A training point with a known label

Min-max normalisation

x' = (x - min) / (max - min) R: preProcess(data, method = c("range"))

Before every distance-based method. The min and max are taken down each column of the training data, and the same figures must be reused on the test rows.

min, max
Smallest and largest value of that feature in the training data
x'
The rescaled value, between 0 and 1

KNN prediction rule

classification: predicted class = mode of the k nearest labels regression: predicted value = mean of the k nearest targets k odd for two classes

Step 3, after the distances are ranked. The same neighbour set serves both tasks; only this line changes.

k
Number of neighbours consulted, chosen on held-back data
mode
The most common label; ties broken by the nearer neighbour
Step 3 of 22
The real wordsTheory

The three steps

  1. Calculate distances. From the test point to every point in the training data.
  2. Find the neighbours. Rank the points by increasing distance and take the K closest.
  3. Vote. Assign the test point to the most common class among those K. For regression, take their average instead.
d = sqrt( (x1 - x2)^2 + (y1 - y2)^2 ) the class Excel formula, =SQRT((x-xi)^2+(y-yi)^2)
← BackContinue →