The Number One Educational Resource for the Game Industry
-
Overview:
-
A/B testing is an important tool for modern game designers and product managers. It provides a robust framework to facilitate experimentation with many aspects of the game in a controlled way and to confidently make decisions about positive changes to the game's design. The Bayesian approach to A/B testing, in comparison to the more traditional null hypothesis testing method, permits more dynamic analysis of the information as it arrives (i.e. differences in "behavior" of users subjected to the different variants in the test) and allows for real-time A/B testing and optimization of games. Using Bayes' rule, we can incorporate any prior information we have about the test and then constantly update our beliefs based on evidence we gain throughout the test. This allows us to make value judgments as the test is ongoing, weighing up the risk of a particular option to the potential gain, in support of or in contrast to our own intuition.
Recommended
