Missing values
mean(x, na.rm = TRUE)
cor(x, y, use = "complete.obs")
Any calculation over a column that may have gaps. Never substitute zero for a missing value.
- NA
- Value exists in the world but is absent from the data
- na.rm = TRUE
- Drop NAs from both the sum and the count
Group summaries
aggregate(y ~ g, data = df, FUN)
df[which.max(df$y), ]
Totals, averages or counts within each level of a category, and pulling the whole record of the largest value.
- y ~ g
- The number to summarise, broken down by the grouping column
- FUN
- sum, mean, length, max: the function applied within each group
- which.max
- Position of the maximum, so it belongs in the row slot
The four base charts
hist(x, main =, xlab =, col =) one variable's spread
barplot(h, names.arg =, col =) categories compared
plot(x, y, col =, pch = 19) two variables together
boxplot(y ~ group, data =, col =) spread within groups
Choose from the question being asked. Aggregate before barplot, and scatter before correlating.
- main / xlab / ylab
- Title and axis labels
- pch
- Point shape; 19 is a solid dot
- names.arg
- Bar labels, taken from the aggregated group names
Correlation and its heatmap
r = sum((x - xbar)(y - ybar)) / sqrt( sum(x - xbar)^2 x sum(y - ybar)^2 )
cor(x, y, method = "pearson")
cor(num_data) # the matrix
corrplot(cor(num_data), method = "color") # the heatmap
distinct pairs = n(n - 1) / 2
After the scatter plot, and before any regression. A block of strong correlation among predictors is a multicollinearity warning.
- r
- -1 to +1, no units, measures the straight-line relationship only
- diagonal
- Always 1: a variable against itself
- n
- Number of numeric columns in the matrix