A/B are increasingly popular! However, they do come at a cost. When talking to companies, I hear several concerns that reduce their appetite for extensive A/B testing. First, the randomized assignment forces companies to assign the wrong treatment (e.g. marketing offer) to a bunch of customers in order to learn from their response. Second, it blocks the possibility for any other marketing intervention while the test is running.

In a recently published paper, my co-authors and I have developed a batch-mode “smart” experimentation algorithm that substantially reduces the downsides of A/B tests. In particular, it decreases experimentation time by 6-30% in our simulations. Interested to know more? You can find a link to our paper: https://www.aurelielemmens.com/publications/batch-mode-active-learning-for-individual-treatment-effect-estimation/.

Looking forward to your reactions!