Skip to content
Sem 5 Prep
← Visual AnalyticsPreparing data for visual analytics

Formulas for this chapter

Bin mean, bin median, bin boundary

sort, cut into bins of equal count, then replace every value by the bin's mean, or its median, or its nearer extreme

Whenever a question says 'smooth by binning' or gives a bin size. Bins hold an equal count, not an equal width. Ties on the boundary rule are broken downwards in the class's worked slides.

bin size
Number of values per bin, given in the question
bin mean
Average of the values in that bin; replaces all of them
bin median
Middle value, or the average of the two middle values for an even count
bin boundary
The bin's smallest and largest values; interior values move to whichever is nearer

Min-max normalisation

x' = (x - min) / (max - min) maps into [0, 1]

When several series of different magnitudes must share one axis, or when a visual encoding needs a bounded input such as colour intensity or marker size. Sensitive to the extremes, so one new record can rescale everything.

x
The original value
min, max
Smallest and largest values in that column
x'
The normalised value; the minimum becomes 0 and the maximum becomes 1

Standardisation (z-score)

z = (x - mean) / SD gives mean 0 and SD 1

When the question is whether several series move together, or when outliers make a min-max range unrepresentative. Produces negative values, so it cannot feed a visual that needs non-negative numbers.

x
The original value
mean
Average of the column
SD
Standard deviation of the column
z
How many standard deviations above (positive) or below (negative) the mean the value lies
Step 1 of 31
The ideaTheory

Four spellings of one city

A customer list holds Mumbai, mumbai, MUMBAI and Mumabi. Count customers by city and you get four cities instead of one.

Nothing is missing and nothing is wrong in any single row. The dataset is still unusable, because the computer matches text exactly and you do not.

Data cleaning is finding those four and turning them into one. Everything in this chapter is a version of that job.

← BackContinue →