Agenda

• Who am I?
• Different Approaches to Attribution
  • Rule-Based: Canned Approaches
  • Ad-hoc Pre-Post Testing
  • A/B Testing & Experimental Methods
  • Markov Chain Models in R

• https://github.com/ksasso/onlineattrib_inR_talk

Some Examples:

• Last Interaction
• First Interaction
• Last Non-Direct Click
• Linear Attribution
• Time Decay
• Position Based

Google Analytics

An example:

• We track the same customers' sales over a two 2-week period in which our Facebook ad was and was not live
  • We then compare sales or conversion between these two time periods for the same customer (i.e., repeated measures)
• We can also treat ad_exposure as a predictor along with some other likely important factors (i.e., purchase hx, buying segment, days of week)
  • Use more advanced modeling techniques to predict conversion or revenue

Statistical Approaches:

• Similar to what we will see for A/B test in basic form, but more limited inferences (not causal)
• Utilize more complicated machine learning models (i.e., glm, random forest, or xgBoost model) more on this soon !

Some Modeling Options

The 3rd Variable Problem:

• Cities with more churches have higher crime rates. But what about population?

Central Limit Theorem & Random Assignment

• Randomly assign customers to groups based on chance procedure
• Minimizes differences between groups such that differences can be attributed to experiment
• Central Limit Theorem: When random variables (independent) are added to a set their distribution tends toward a normal distribution (informally a “bell curve”) even if the original variables themselves are not normally distributed
  • Helps justify the approximation of large-sample statistics to the normal distribution in controlled experiments

Example A/B test diagram

Statistical Approaches:

• Statistical methods same as ad-hoc pre-post, but now campaign exposure is randomly assigned!
• Randomization in R

library(dplyr)
head(mtcars)
mtcars <- mtcars %>% 
  mutate(row_id  = row_number())
test <- mtcars %>% 
  sample_frac(.5)
control <- mtcars %>% 
  filter(!row_id %in% test$row_id)

Incorporating additional Predictors: Linear Method

counts <- c(18,17,15,20,10,20,25,13,12)
outcome <- gl(3,1,9)
treatment <- gl(3,3)
print(d.AD <- data.frame(treatment, outcome, counts))
glm.D93 <- glm(counts ~ outcome + treatment, family = poisson())
summary(glm.D93)

Incorporating additional Predictors: xgBoost