Analyzing A/B Testing: Case Study From Production Experiment

A/B experiments are an integral part of any research and development setup, including algorithmic trading, where there is an endless choice of determining which set of configurations is suitable under which circumstances. We run A/B experiments on production order flow to determine if a given set of configurations successfully helped improve performance. Proper attention is paid to the design of the experiment to have homogeneity in order sizes and timing. However, substantial differences still need to be accommodated to draw meaningful inferences. Several of our clients face a similar challenge of analyzing different brokers’ algorithms for execution as they execute through different market states and regimes, which impact the execution cost. 

Request the full whitepaper by filling out the form below.