[ad_1]
This article is part of my 2019 personal challenge — reviewing 50 Marketing Science research papers. It focuses on ads measurement which is the center of my current work. I will organize it into several sections. Survey/State-of-art has articles that can give some “big picture” in ads measurement research. Methodology section has articles introducing ads measurement approaches that fall into two major streams — modeling with observational data, and experiment. And the last section covers some special topics such as measuring the synergy effect between different media.
P.K. Kannan, Werner Reinartzb, Peter C. Verhoef, The path to purchase and attribution modeling: Introduction to special section. In International Journal of Research in Marketing, 1 September 2016, Vol.33(3), pp.449-456: a survey in attribution modeling. It introduced the state-of-the-art, reviewed prior work in the domain, and proposed research agenda for the future including both tactical and methodological issues such as viewability, cookie deletion, purchase funnel, integrating online and offline, real-time analytics etc. This article can serve as a map of attribution researches in the academic.
Research priorities 2016-2018 from Marketing Science Institute: 5 research priorities summarized by MSI from surveys of academic and member company trustees. “Quantitative models to understand causality, levers, and influence in a complex world” is the No.1 priority. Specifically, MTA and MMM was mentioned in this priority in words such as “Improving multi-touch attribution, marketing mix, and ROI models — across all media, digital and non-digital”. Some key words such as “all media”, “omni-channel”, “omni-screen”, “impact of creative”, “synergy effect” were mentioned indicating some key areas of improvement. Research priorities 2018-2020 was also released recently.
Methodology – Modeling with Observational Data
Xuhui Shao, Lexin Li. Data-driven Multi-touch Attribution Models. In Proceedings of the 17th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, 2011: introduced bagged logistic regression which reduces model variance, and a simple version of a probabilistic model.
Vibhanshu Abhisheky, Peter S. Faderz, and Kartik Hosanagar, Media Exposure through the Funnel: A Model of Multi-Stage Attribution. In SSRN Electronic Journal2012: introduced a dynamic hidden Markov model (HMM) that capture users movement down the conversion funnel as a result of ads exposures.
Daniel & Feit, Elea & Bradlow, Eric. (2016). Measuring Multi-Channel Advertising Effectiveness Using Consumer-Level Advertising Response Data. Introduced a Bayesian Tobit model to model multi-channel attribution. Unlike Shao, Li (2011), this approach: 1) has an individual-level time series model built through MCMC Bayesian framework; 2) introduced interaction terms between every two advertising channels to estimate marketing synergy effect; 3) use CIR(cumulative impulse response) for scoring and targeting.
Methodology – Experiment
The authors challenged several industry widely used methods to evaluate advertising effectiveness such as using intermediate metrics (Click-through-rate), comparing exposed and unexposed users. The authors also think measuring advertising effectiveness using regression models built on observational data is subject to bias because individual-level sales are volatile leading to high signal-to-noise ratio (pages 196-197).
On the other hand, the authors think large-scale randomized experiments is the golden standard to measure the true causal effect of advertising, and briefly introduced the evolution of ads experiments (from charity ads, to “ghost ads”/control ads, to methods which increase statistical power). While statistical power is still a challenge experimentation method needs to tackle due to the volatile, dilute, and even long-run effect of advertising.
The authors introduced two approaches to alleviate the power problem in large-scale online ads experiments.
- Covariate approach: add covariates (such as user demographics, past purchase history, ad-exposure intensity) into the experimental linear regression to reduce the residual variance of the outcome.
- Control-ad approach: Using control ads in the control group in order to distinguish users who would not be treated and exclude them in treatment effect calculation.
In the specific case of an apparel retailer from the authors, 1) improved the precision of the treatment effect estimation by 5%, while 2) improved the precision by 31%.
Several other approaches to increase power reviewed by the authors include:
- Chose a less sparse or noisy outcome (e.g. binary transaction instead of revenue sales)
- Use meta-study to analyze a group of individual experiments collectively
- Form pairs of users by minimizing some objective function that defines the distance between two nodes in the graph of users. Then randomly assign two users to each group for each pair.
- ensure 50%, 50% split between control and treatment groups
The authors identified a type of bias in measuring online ads effectiveness using observational data and labeled it as “activity bias”. The authors explained that the “activity bias” happens because of the positive correlation between users online activities so that the exposed users usually have more other online activities including online purchase/search/web visit which is the outcome of the ad of interest.
The authors evaluated several observational methods (regression etc.) by comparing them to controlled experiments. In their specific comparison, a simple exposed and unexposed comparison gave a lift of 1198% while the true lift from the experiment was 5.4%, and after adding covariates to control for past online activities in a regression approach, the lift overstates the impact by roughly +-10%.
Special Topic – Media Synergy/Spillover
Ghose, A., & Todri, V. (2015). Towards a digital attribution model: Measuring the impact of display advertising on online consumer behavior. MIS Quarterly Vol. 40 No. 4, pp. 889-910/December 2016: simulated a quasi-experiment using observational data to analysis the causal impact of display exposure on consumer behavior (search, website visit, purchase). They found exposure to display can significantly increase consumers’ propensity to search for the brand, visit the brand website, and make purchases even when they don’t engage with the display directly.
Kireyev, Pavel & Pauwels, Koen & Gupta, Sunil. (2015). Do Display Ads Influence Search? Attribution and Dynamics in Online Advertising. International Journal of Research in Marketing. 33. 10.1016/j.ijresmar.2015.09.007: Provided a review of academic prior work about media synergy/spillover effect (display & search, generic search & branded search, online & offline etc.) especially display & search. Used persistence modeling techniques to capture the complex dynamic synergy/spillover effect in online advertising (Dekimpe and Hanssens 1999) on aggregated data. The research found display can increase search click and conversion. (MMM)
[ad_2]
Source link
