Skip to content
Sem 5 Prep
← Visual AnalyticsPreparing data for visual analytics

Formulas for this chapter

Bin mean, bin median, bin boundary

sort, cut into bins of equal count, then replace every value by the bin's mean, or its median, or its nearer extreme

Whenever a question says 'smooth by binning' or gives a bin size. Bins hold an equal count, not an equal width. Ties on the boundary rule are broken downwards in the class's worked slides.

bin size
Number of values per bin, given in the question
bin mean
Average of the values in that bin; replaces all of them
bin median
Middle value, or the average of the two middle values for an even count
bin boundary
The bin's smallest and largest values; interior values move to whichever is nearer

Min-max normalisation

x' = (x - min) / (max - min) maps into [0, 1]

When several series of different magnitudes must share one axis, or when a visual encoding needs a bounded input such as colour intensity or marker size. Sensitive to the extremes, so one new record can rescale everything.

x
The original value
min, max
Smallest and largest values in that column
x'
The normalised value; the minimum becomes 0 and the maximum becomes 1

Standardisation (z-score)

z = (x - mean) / SD gives mean 0 and SD 1

When the question is whether several series move together, or when outliers make a min-max range unrepresentative. Produces negative values, so it cannot feed a visual that needs non-negative numbers.

x
The original value
mean
Average of the column
SD
Standard deviation of the column
z
How many standard deviations above (positive) or below (negative) the mean the value lies
Step 3 of 31
The real wordsTheory

Transform

TransformThe middle ETL stage: filtration, cleansing and preparation of the extracted data, with lookup tables.

The deck's list of what happens here: authentication of records, refutation, and integration of data; and the data being sorted, filtered, cleared, standardised, translated or verified for consistency.

A lookup table is a small reference sheet: state code to state name, product code to product name. Transform uses it to expand codes into readable values.

← BackContinue →