Conditional probability
P(A | B) = P(A and B) / P(B)
P(A and B) = P(A | B) x P(B)
Whenever the question says "given that". Read the numerator off the joint cell and the denominator off the B total.
- P(A and B)
- Probability both happen, the joint cell over the grand total
- P(B)
- Probability of the condition, the new denominator
Bayes' theorem
P(A | B) = P(B | A) x P(A) / P(B)
P(B) = P(B | A) x P(A) + P(B | not A) x P(not A)
When you are given the likelihood the wrong way round: a test's sensitivity, a machine's defect rate, a per-class table. Build the denominator over every cause first.
- P(A)
- Prior, belief before the evidence
- P(B | A)
- Likelihood, how well A explains the evidence
- P(A | B)
- Posterior, belief after the evidence
- P(B)
- Evidence, the total probability of what was observed
Naive Bayes score
score(class) = P(class) x P(x1|class) x P(x2|class) x ... x P(xn|class)
P(class | x) = score(class) / sum of scores over all classes
Classifying a row with several categorical predictors. The scores are not probabilities until you divide by their total.
- P(class)
- Prior, that class's share of the training rows
- P(xi|class)
- Count of that level within the class, over the class size
- n
- Number of predictors, each contributing one factor
Conditional probability table in Excel
P(level | class) = COUNTIFS(target, class, predictor, level) / COUNTIF(target, class)
Laplace smoothing: add 1 to every count before dividing
Building the tables from raw rows. Each predictor's entries must add to 1 within a class.
- target
- The class column, here Subscriber
- predictor
- One categorical column, here Age, Gender, Income or Location