You've been logged out of GDC Vault since the maximum users allowed for this account has been reached. To access Members Only content on GDC Vault, please log out of GDC Vault from the computer which last accessed this account.

Click here to find out about GDC Vault Membership options for more users.


The Number One Educational Resource for the Game Industry

Session Name:

A/B Testing for Game Design Iteration: A Bayesian Approach


A/B testing is an important tool for modern game designers and product managers. It provides a robust framework to facilitate experimentation with many aspects of the game in a controlled way and to confidently make decisions about positive changes to the game's design. The Bayesian approach to A/B testing, in comparison to the more traditional null hypothesis testing method, permits more dynamic analysis of the information as it arrives (i.e. differences in "behavior" of users subjected to the different variants in the test) and allows for real-time A/B testing and optimization of games. Using Bayes' rule, we can incorporate any prior information we have about the test and then constantly update our beliefs based on evidence we gain throughout the test. This allows us to make value judgments as the test is ongoing, weighing up the risk of a particular option to the potential gain, in support of or in contrast to our own intuition.

Did you know free users get access to 30% of content from the last 2 years?

Get your team full access to the most up to date GDC content

  • GDC 2014
  • Steven Collins
  • Swrve
  • free content
  • Programming
  • Programming
  • Business, Marketing & Management
  • Business & Marketing