Skip to content
Sem 5 Prep
← Visual AnalyticsPreparing data for visual analytics

Formulas for this chapter

Bin mean, bin median, bin boundary

sort, cut into bins of equal count, then replace every value by the bin's mean, or its median, or its nearer extreme

Whenever a question says 'smooth by binning' or gives a bin size. Bins hold an equal count, not an equal width. Ties on the boundary rule are broken downwards in the class's worked slides.

bin size
Number of values per bin, given in the question
bin mean
Average of the values in that bin; replaces all of them
bin median
Middle value, or the average of the two middle values for an even count
bin boundary
The bin's smallest and largest values; interior values move to whichever is nearer

Min-max normalisation

x' = (x - min) / (max - min) maps into [0, 1]

When several series of different magnitudes must share one axis, or when a visual encoding needs a bounded input such as colour intensity or marker size. Sensitive to the extremes, so one new record can rescale everything.

x
The original value
min, max
Smallest and largest values in that column
x'
The normalised value; the minimum becomes 0 and the maximum becomes 1

Standardisation (z-score)

z = (x - mean) / SD gives mean 0 and SD 1

When the question is whether several series move together, or when outliers make a min-max range unrepresentative. Produces negative values, so it cannot feed a visual that needs non-negative numbers.

x
The original value
mean
Average of the column
SD
Standard deviation of the column
z
How many standard deviations above (positive) or below (negative) the mean the value lies
Step 4 of 31
The real wordsTheory

Load

LoadThe last ETL stage. Writes the extracted and transformed output to a data warehouse.

Two forms, in the deck's words: either the physical insertion of a record as a new row in a database table, or link processes for each record from the main source.

So ETL is one sentence: read from many places, fix and reshape, write once to the warehouse the dashboards then read.

← BackContinue →