Best Practices for A/B Testing


#1

Are there recommendations for A/B Testing in Production of data analysis changes (new machine learning model, changes in business practice/rules, etc.)?

My guess right now would be to use scipy.stats modules and/or statsmodel in Python DS Notebook; R has more packages on multivariate A/B Testing.

mlxtend package from Python has in-sample hypothesis testing for classifier/regression accuracy http://rasbt.github.io/mlxtend/user_guide/evaluate/paired_ttest_5x2cv/ ; Is there a same package in C3?