[Literature] Game Balance ch8 - Statistics and metrics

My notes from course 8 of the Game Balance class of Summer 2010, by Ian Schreiber.

Roll	2d6	1d11+1
Mean	7	7
Median	7	7
Std Dev	2.4	3.3
68% of rolls will be in	[5; 9]	[4; 10]

Statistics tell us that something is happening, but they do not tell why. The why comes when designing appropriate metrics. Causality can not be established from correlations! For instance, if there are less Troll players than Elf players, what can it mean?

• Trolls are less aesthetically appealing.
• Trolls are less fun to use.
• Trolls have a higher learning curve.
• Trolls are underpowered.
• ...

Remember that top score players are outliers.
Playtesters should be as similar as possible to the target market. Playtesting with metrics is an iterative process that needs to be planned ahead of time. Moreover, playtesting needs to be done on novice players (they try the system once or twice) and on expert players (they try it regularly for a couple of months). Experts are needed to check for emerging/unexpected strategies.

Metrics

Developers are mixed between using intuition and/or metrics:

Zynga's Mark Pincus: metrics-driven design,

Chris Hecker: intuition-driven design,

and Playfish's Jeferson Valadares: both.

One can measure how much impact a change in game design has on the customer base. A-B testing allows more control on external factors (eg players of a particular region may stop playing for a while if there's a natural disaster).

MAU versus DAU: a purely viral game will have MAU = 30*DAU (there are 30 days in a month, and people only play for one day, then leave) whereas an old and sticky game will have DAU and MAU close together.
ARPU differs from ARPPU. Plotting ARPU or ARPPU against time is a good way to check if the start-, mid- or end-game generate revenue or not.