Skip to content
Sem 5 Prep
← Visual AnalyticsPreparing data for visual analytics

Formulas for this chapter

Bin mean, bin median, bin boundary

sort, cut into bins of equal count, then replace every value by the bin's mean, or its median, or its nearer extreme

Whenever a question says 'smooth by binning' or gives a bin size. Bins hold an equal count, not an equal width. Ties on the boundary rule are broken downwards in the class's worked slides.

bin size
Number of values per bin, given in the question
bin mean
Average of the values in that bin; replaces all of them
bin median
Middle value, or the average of the two middle values for an even count
bin boundary
The bin's smallest and largest values; interior values move to whichever is nearer

Min-max normalisation

x' = (x - min) / (max - min) maps into [0, 1]

When several series of different magnitudes must share one axis, or when a visual encoding needs a bounded input such as colour intensity or marker size. Sensitive to the extremes, so one new record can rescale everything.

x
The original value
min, max
Smallest and largest values in that column
x'
The normalised value; the minimum becomes 0 and the maximum becomes 1

Standardisation (z-score)

z = (x - mean) / SD gives mean 0 and SD 1

When the question is whether several series move together, or when outliers make a min-max range unrepresentative. Produces negative values, so it cannot feed a visual that needs non-negative numbers.

x
The original value
mean
Average of the column
SD
Standard deviation of the column
z
How many standard deviations above (positive) or below (negative) the mean the value lies
Step 2 of 31
The real wordsTheory

Extract

ExtractThe first ETL stage. Reads data from multiple data sources and pulls out the required set of data.

The deck attaches two conditions to it, and both are markable.

  • It recovers the necessary data with optimum usage of resources, so you take the columns you need and no more
  • It should not disturb the data sources, their performance or their functioning, so a heavy extract runs overnight and never locks the live system
← BackContinue →