Interference Among First-Price Pacing Equilibria: A Bias and Variance Analysis
CoRR(2024)
摘要
Online A/B testing is widely used in the internet industry to inform
decisions on new feature roll-outs. For online marketplaces (such as
advertising markets), standard approaches to A/B testing may lead to biased
results when buyers operate under a budget constraint, as budget consumption in
one arm of the experiment impacts performance of the other arm. To counteract
this interference, one can use a budget-split design where the budget
constraint operates on a per-arm basis and each arm receives an equal fraction
of the budget, leading to “budget-controlled A/B testing.” Despite clear
advantages of budget-controlled A/B testing, performance degrades when budget
are split too small, limiting the overall throughput of such systems. In this
paper, we propose a parallel budget-controlled A/B testing design where we use
market segmentation to identify submarkets in the larger market, and we run
parallel experiments on each submarket.
Our contributions are as follows: First, we introduce and demonstrate the
effectiveness of the parallel budget-controlled A/B test design with submarkets
in a large online marketplace environment. Second, we formally define market
interference in first-price auction markets using the first price pacing
equilibrium (FPPE) framework. Third, we propose a debiased surrogate that
eliminates the first-order bias of FPPE, drawing upon the principles of
sensitivity analysis in mathematical programs. Fourth, we derive a plug-in
estimator for the surrogate and establish its asymptotic normality. Fifth, we
provide an estimation procedure for submarket parallel budget-controlled A/B
tests. Finally, we present numerical examples on semi-synthetic data,
confirming that the debiasing technique achieves the desired coverage
properties.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要