Showing posts with label statistics. Show all posts
Showing posts with label statistics. Show all posts

27 January 2016

Powerball lottery - Boosting jackpots to boost sales

The January 2016 jackpot reached $1.5B because the carried-over jackpot increased faster than ever before in the streak of 19 drawings without winner. To show this, the table below compares the $1.5B jackpot to the $564M jackpot of February 2015, which concluded a streak of 20 drawings without winner.

StreakJackpot amount ($M)
Jan'16Feb'15
15255208
16300230
17334261
18529289
19948317
201,500395
21-564
StreakTickets sold (M)
Jan'16Feb'15
152624
164128
175429
1817637
1944048
2063573
21-190

The February 2015 jackpot reached half a billion dollars after 20 consecutive drawings without winner. The January 2016 jackpot reached three times that amount in one drawing less. In fact, it is surprising that the January 2016 jackpot could triple from $529M to $1.5B in two drawings. What happened?

Funding Powerball's jackpot

To understand how jackpots build up, we have to understand some of the inner workings of the Powerball lottery. The 2015 Powerball group rules published by the Multi-State Lottery Association (MUSL) managing Powerball are sometimes ambiguous, so this section may not be perfectly accurate. To keep it simple, the Power Play option is ignored.

Although MUSL runs Powerball, each state lottery is in charge of advertising, selling tickets, and giving non-jackpot prizes to their respective winners. Each state lottery keeps half of ticket sales, which probably directly goes into the state's budget. The other half funds MUSL's four accounts: the prize pool, jackpot pool, bootstrap pool, and reserve.

The prize pool (aka Powerball Set Prize Pool) is filled weekly by each state lottery to pay the small prizes. Ignoring Power Play, it should receive at least $4 / 38.32 + ... + $1M / 11.7M = 32 cents per ticket to be able to pay out the prizes. In-between two drawings, its balance is close to zero.

The jackpot pool (aka Grand Prize Pool) holds the jackpot money. When the other pools (usually the bootstrap pool) need to be topped-up, up to 5% of sales is transferred from it to them. By the time the jackpot reaches $110M annuity/$70M cash, ticket sales have filled all other pools, and the jackpot pool receives the full 68 cents per ticket.

The bootstrap pool (aka Set-Aside Account) backs up the jackpot pool when the jackpot is won at the beginning of a streak. For example, every streak starts with a jackpot of $40M annuity/$25M cash. These drawings usually sell 10 million tickets, ie $20M in sales. The state takes $10M, small prize winners $3.2M (32 cents per ticket), and so there is only $6.8M left to pay the $25M jackpot. The extra $18.2M comes from this pool. Fortunately, 10M tickets have only 4% chance to win the jackpot, so the bootstrap pool is rarely emptied. It is filled by "taxing" the jackpot pool 5% of sales, ie 10 cents per ticket, until it has reached its $20M cap.

The reserve (aka PRA and SPRA) exists so that MUSL does not get bad press for not paying winners their full prize. This could happen because of a system error, miscalculation, or because all other pools are depleted. When the reserve itself is depleted, prizes stop being fixed (eg $1M or $7) and become parimutuel to prevent breaking the bank. Until the reserve reaches its $40M cap, filling it takes precedence over the other pools.

January 2016 vs February 2015

To illustrate how these pools work, we can use the streak of 20 drawings leading to the jackpot of $1.5B annuity / $930M cash as an example. The streak starts on November 7, when 374k 2-dollar tickets and 101k 3-dollar tickets won small prizes. Ticket sales (estimated) are 24.87 * (374k * $2 + 101k * $3) = $26M. The state takes $13M, and small-prize winners $4M. Let's imagine that the reserve is full. The jackpot had reached $90M cash on November 4, so the bootstrap pool should be full, but to illustrate its workings, let's assume it is empty. Since the jackpot is not won, the bootstrap pool receives 5% of sales = $1.3M. The jackpot pool receives the remaining $7.7M. This process goes on until December 19, when the bootstrap pool reaches its $20M cap. All these numbers are listed in the first table below.

On closure day, January 13, when the $930M cash jackpot is won, the ten-week streak has raised a total of $3.4B. Around $623M has been paid to small-prize winners, $930M shared between the jackpot's three winners, and $1.7B kept by the states. As for the February 2015 jackpot, the states only received $679M in roughly the same amount of time (ten weeks and a half). Judging from these two jackpots only, the 2015 tweaks seem to have tripled Powerball's profits.

Pool amounts (in $M) leading to the $1.5B annuity jackpot
date jackpot
(cash)
ticket
sales
prize
pool
jackpot
pool
bootstrap
"tax"
bootstrap
pool
11/725264.27.61.31.3
11/1132223.0151.12.4
11/1438284.1231.43.8
11/1844247.7261.25.0
11/2150276.4321.36.3
11/2557283.1411.47.7
11/2863284.8491.49.1
12/269323.7591.610.7
12/580388.2681.912.6
12/991335.5781.714.3
12/12103387.4881.916.2
12/16113396.2992.018.2
12/19127519.61131.820
12/23143589.9132full20
12/26161546.3153full20
12/301898414181full20
1/221011821219full20
1/633337566341full20
1/9597963159663full20
1/139301,3742741,077full20
closure9303,4406230NA20
Pool amounts (in $M) leading to the $564M annuity jackpot
date jackpot
(cash)
ticket
sales
prize
pool
jackpot
pool
bootstrap
"tax"
bootstrap
pool
12/326222.27.71.11.1
12/633264.4151.32.4
12/1039254.1221.23.6
12/1346274.8291.35.0
12/1752254.4361.26.2
12/20593210.2401.67.8
12/2465359.2471.89.6
12/27723210.3511.611.1
12/3178388.4591.913.1
1/385354.6701.714.8
1/795355.0811.816.6
1/101054110.2902.118.6
1/14114387.01001.420
1/171264816108full20
1/211355212122full20
1/241506218135full20
1/281706011154full20
1/311887415176full20
2/420610018209full20
2/725715130255full20
2/1138140174381full20
closure3811,3582780NA20

The Powerball tweaks have only been implemented for three months now. With little data to rely on, it is not guaranteed that profits will stay triple what they were before. Profits may have soared this time because of the buzz surrounding the record-breaking jackpot. The next billion-dollar jackpot may see mediocre sales because people got tired of Powerball or found other games to play. Will the next billion-dollar jackpot generate a billion dollars in profits? Will players adapt to the new odds? We will probably find out within a year.

26 January 2016

Powerball lottery - Tweaks

The jackpot fatigue theory

The Powerball mechanics have been tweaked several times since it started in 1992. Starting in January 2012, the game had 59 white balls and 35 red balls so that a billion-dollar jackpot would happen every 10 years. No such jackpot happened until the rules changed again in 2015, but as the table below shows, the jackpot reached half a billion several times.

Jackpots above $300M, 2012-2015
Date Jackpot ($M) Tickets (M)
2/11/2012 336 89
8/15/2012 337 86
11/28/2012 588 286
3/23/2013 338 80
5/18/2013 591 243
9/18/2013 399 93
2/19/2014 425 86
2/11/2015 564 191
9/30/2015 310 51

Powerball sales dropped 19% nationally in 2014. Lottery officials suggested two explanations: the lack of a huge jackpot in 2014, and jackpot fatigue: lotteries need increasingly bigger jackpots to attract the casual players who only buy tickets when the jackpot is huge.

The table above confirms that there was only one jackpot above $300M in 2014, but it rejects the fatigue theory. For jackpots between $300M and $350M, the number of tickets sold decreased from 89M in 2012 to 51M in 2015. And for the three jackpots between $550M and $600M, the number of tickets sold went from 286M in 2012 to 191M in 2015. Sales from the biggest jackpots lost 35% in three years.

Yet, the fatigue theory made New York state lottery officials shift their focus from jackpot-driven games, where jackpots get very big too rarely, to instant scratch-off games with more frequent prizes. New York state is a major actor in the Powerball lottery: it ranks third in ticket sales, after California and Florida. So it's likely that the tweaks of October 2015 were an attempt to address jackpot fatigue.

October 2015 tweaks

In October 2015, white balls increased from 59 to 69, and red balls decreased from 35 to 26. Thus the odds of winning a prize increased from 1:32 to 1:25, but the odds of winning the jackpot decreased from 1:175M to 1:292M. Lower jackpot odds means longer streaks until the jackpot is won, ie bigger jackpots. Projections made in 2012 suggested that a billion-dollar jackpot would happen every 10 years. Data from November 2015 to January 2016 suggests that billion-dollar jackpots should now happen every year or so, and there is a 63% chance for one to show up within 5 years. This tweak is similar to the British National Lottery tweak of June 2015: 5 balls used to be picked among 49, which was raised to 59, resulting in the £58M jackpot of January 2016, the largest in the National Lottery's history.

Decreasing the odds of winning the jackpot decreases the expected value of a ticket. This expected value is plotted in the graph below, against the jackpot value. Before the rule change, the jackpot had to reach $200M for a ticket to be worth $1. Based on drawings data from 2015, $200M jackpots were expected to occur every 24 weeks. Now, after the rule change, a ticket is worth $1 when the jackpot reaches $450M, which is expected to happen every 34 weeks.

Long story short, the October 2015 tweaks increased the chance of winning a consolation prize, but decreased the chance of winning the jackpot and the expected value of a ticket. Since the expected value of a ticket estimates how much each lottery ticket costs to the organizers, their profits must have increased!

25 January 2016

Powerball lottery - Odds, expected value, and billion dollar jackpots

Basic odds and expected values

Wikipedia has a good introductory page about lottery mathematics. In short, the number of possibilities for drawing 5 white balls among 69 and one red ball among 26 is C(69,5) * C(26,1) = 292M. Despite these very low odds, 14 people won the jackpot in 2015, and 72 from 2011 to 2015. The chances of winning prizes are listed in the table below. The expected value column is the product of a prize by its odds.

Match Prize Value Odds Expected value
5 whites + red Jackpot $40M to $1.5B 1 in 292M = .00000034% $0.14 to $5.14
5 whites Match-5 $1M 1 in 11.7M = .0000085% $0.09
1-4 whites + red
or 3-4 whites
Consolation $4 to $50k 1 in 24.7 = 4.04% $0.24

The Powerplay mutliplier applies only to consolation prizes. Taking into account the probability of each multiplier, their expected value becomes $0.65. Power Play also doubles the Match-5 prize by 2, so the Match-5 expected value becomes $0.17. While Power Play does increase values, it does not increase them by enough to cover the $1 cost of the Power Play option. Power Play is not worth it.

Huge jackpots do not justify buying a ticket

Intuitively, the January 16 2015 jackpot of $1.5B is so huge that it may seem statistically worth it to buy a ticket: you spend $2 to receive $1.5B / 292M = $5.14. Each ticket would net $3.14. Put another way, one could earn $1.5B after investing 292M * $2 = $584M. The theory is promising, but it has flaws.

First, 635M tickets were sold. An alternative way to find this number is to mutliply the number of winners by the odds of winning: 26.11M * 24.87 = 650M tickets. Based on the method described here, there is an 89% chance that there is another winning ticket. More specifically, the chances of having exactly 0, 1, 2, 3, 4, and 5 winners among the 650M tickets already sold are 11, 25, 27, 19, 11, and 5%. Taking these probabilities into account, $5.14 becomes $5.14 * (11% / 1 + 25% / 2 + 27% / 3 + ... ) = $2.09. So, buying a Powerball ticket is still (barely) worth it. Second, there is a 40% federal income tax on lottery winnings. That brings it down to $1.26 per $2 ticket. Buying a ticket for the $1.5B jackpot is not statistically worth it anymore.

Last, all tickets have to be bought in person. Since Powerball drawings happen twice a week, one would have to buy 70M tickets per day, or 845 per second. CNN reports that in February 1992, an Australian consortium tried to corner a $24 million Virginia Lotto jackpot. But the group was only able to purchase 2.4 million of the 7 million combinations before time ran out.

Billion-dollar jackpots every year

Could the jackpot get big enough that buying a ticket will be statistically worth it? There is no data point after $1.5B, so we can't say. But we can compute the expected time until such a jackpot appears again.

First, For the jackpot to reach $1.5B after 20 drawings, all 957 million tickets from the previous 19 drawings had to be losers. That is a 4% chance. Then, we can adapt the coin-toss method, using ticket sales and jackpot values for the 20 drawings leading to the $1.5B jackpot, to compute the expected time until the jackpot reaches $1.5B. Unless my math is wrong, I found that the expected number of drawings to reach 19 consecutive drawings without winner is 421. With two drawings a week, that means a $1.5B jackpot should appear every 4 years. A billion-dollar jackpot requires 18 consecutive drawings without winner, which can be expected every year.

24 January 2016

Powerball lottery - Rules and patterns of play

On January 13 2016, the Powerball lottery made the news by reaching a jackpot of $1.5B, the largest in US history. Lotteries are interesting because they are a gambling game used by governments to raise funds without raising taxes. Let's have a closer look at the rules, odds, and profits made by lotteries, using Powerball as an example since it's one of the lotteries with highest sales.

Basics

The official simplified rules stipulate that five white balls are drawn among 69, and one red ball among 26. The jackpot is won by matching all five white balls in any order and the red Powerball. The jackpot starts at $40M, and increases every drawing by at least $10M until won. Combined with the minuscule odds of being won, this mechanic makes it possible for the jackpot to become huge. The match-5 prize, worth $1 million, is won by matching five white balls in any order. Consolation prizes, worth $4 to $50k, are won by matching at least three white balls or at least the red Powerball. Tickets cost $2. Players can buy as many tickets as they want.

Drawings happen twice a week, on Wednesdays and Saturdays. The boxplot below plots the median number of tickets sold per drawing. Data consists of drawings with jackpots below $200M, from April 13 2013 (right after California joined the Powerball group) to January 20 2016. Drawings sell an average of 14.0 million tickets on Wednesdays, versus 15.4 millions on Saturdays (p=.0001).

Jackpots predict sales

Ticket sales are very predictable from the jackpot amount. This is shown in the graph below. For jackpots under $300M, the number of tickets sold increases quadratically with the jackpot. Above $300M, we do not have enough data to build a model, but the trend seems logarithmic, probably because ticket sales grow much faster than the jackpot.

Optional features

Power Play is an optional feature that launched in 2001. As of 2015, it multiplies the value of non-jackpot winnings by 2 to 10 by paying an extra $1 per ticket. A wheel determines the multiplier for each drawing (24/43 chance for 2x down to 1/43 for 10x). Interestingly, Power Play used to have a 1x multiplier, but it was removed within a year. Why have a bonus that is sometimes not a bonus?

Random picks: Players can choose their 6 numbers themselves, or have the computer pick them randomly. About 70% to 80% of purchases are computer picks.

Annuity vs cash

Jackpot winners can choose to receive their share of the jackpot immediately through a 30-year annuity, or immediately in cash. When winners choose the cash, they immediately receive the advertised cash amount, which is usually 60-70% the annuity amount. That is why the lottery organizers advertise the jackpot's annuity, and not its cash.

When winners choose the annuity, they immediately receive 1/30th of the cash amount. The organizers invest the rest into government bonds or securities. Every year for 29 years, they sell 1/29th of the bonds and give the proceeds to the winner, with a 5%-interest per year. Choosing the annuity prevents people from blowing through all their money quickly. Yet only one jackpot winner out of 72 chose the 30-year annuity from 2011 to 2015.

27 April 2014

Assist bundles in Forza 4 and 5

Forza Motorsport 4 is a realistic simulation racing game. If they want to, players can handle the clutch, tire-wear, and other minute racing details. For beginners, the game provides assists that take care of these details automatically. For example, the Braking and Steering assists make turning easier, the Line assist shows the optimal path to follow, and the Transmission assist takes care of switching the gears. For a description of the assists, check these websites.

While I was at Microsoft Research during the Summer of 2013, my mentors and I looked at patterns of progression in FM4. Among the things we published, we found that players rarely used the assist bundles shipped in FM4. FM5 came out in November 2013. This post describes how the FM5 designers used our findings on FM4 to improve the assist bundles.

Assists in FM4 and FM5

Here are the assist bundles shipped in FM4.

Easy Medium Hard Advanced Expert
Steering Assisted Normal Simulation
Stability ON OFF
Traction ON OFF
Brakes Assisted + ABS ABS only OFF
Transmission Automatic Manual Manual + clutch
Trajectory Line Full Braking only OFF
Damage Cosmetic Limited Simulation

Below are the FM4 bundles we suggested, based on player data in Career mode.

Easy Medium Hard Advanced Expert Comments
Steering Assisted Normal Simulation Unchanged
Stability ON OFF Unchanged
Traction ON OFF Unchanged
Brakes Assisted + ABS ABS only OFF ABS-only from Medium onwards
Transmission Automatic Manual Manual + clutch Manual from Hard onwards
Trajectory Line Full Braking only OFF Bumped down: shifted all cells to the left.
Damage Cosmetic Limited Simulation Simulation from Hard onwards

And below are the bundles shipped in FM5.

Easy Medium Hard Pro Veteran Comments
Steering Assisted Normal Simpler: now binary. In Custom bundle, Assisted steering forces Assisted brakes.
Stability ON OFF Bumped up
Traction ON OFF Bumped up
Brakes Assisted + ABS ABS only OFF available in Custom bundle.
Transmission Automatic Manual Bumped up. Clutch available in Custom bundle.
Trajectory Line Full Braking only OFF available in Custom bundle.
Damage Cosmetic Simulation Simpler: now binary.

In short, the bundles in FM5 are easier than their FM4 equivalent because the Stability, Traction, and Transmission assists have been bumped up. Clutch transmission, ABS-OFF, and no trajectory line are only available in a Custom bundle. The FM5 designers did not exactly follow the bundles we suggested. Why is that? See below.

Assist progression

For each race number, we look at how much of the player base still has an assist enabled. The goal is to see how fast players disable an assist. The graph below shows exactly this.

Let's take the Transmission assist as an example. In the first race, the Transmission is automatic for 80% of players (the orange line starts at 0.8), manual without clutch for 15% (the space between the orange and brown lines), and manual with clutch for 5% (the brown line starts at 0.95). Even among the players who reach 100 races (17% of the total player base, cf gray line), only 15% use a manual transmission with clutch. That's why in FM5, manual transmission with clutch is only available in a Custom bundle: very few players use it. Same story for the Trajectory Line-OFF and ABS-OFF assists. In the end, the FM5 designers actually used our findings in their product!

Of course, we need another study to assess whether FM5 players are actually using the default FM5 bundles or not.

A note on Rewind

When you miss a turn, you can press Y to rewind the game and try the turn again. In FM4, the ability to rewind is an assist. Turning the assist OFF grants a 20% credit bonus at the end of a race. In FM5, each rewind costs 1% of the race's credit. I like this change for several reasons:

  • In FM4, I enabled rewind in case I needed it during a race. But I only rewinded every 3-4 races. 20% was a lot of credits missed, though, so it felt a bit frustrating.
  • A fixed price encourages players to rewind as much as they want. So, to get more bang for the buck, they increase the difficulty of the AI opponents, and rewind every time they don't make a perfect turn. This is the same lame trick as saving-and-rerolling in RPGs. Racing this way is excruciating. It also takes longer to complete a race, so the amount of credits gained per hour may actually be lower with more difficult AI.
  • FM5's rewind cost is proportional. This is advantageous for players who rewind rarely, and disadvantageous for those who rewind dozens of times per race.
  • I prefer paying the cost automatically, in the stats screen at the end of a race, rather than manually, through an assist menu before the race.

26 June 2013

Influence of gameplay on skill in Halo: Reach

Huang et al. Influence of Gameplay on Skill in Halo Reach, 2013

Data: 3m player data (entire population) for the first 7 months of Halo Reach, and 70 players in a survey. Mixed methods: use survey data to help explain the big dataset.

Most played mode is Team Slayer: from 3v3 to 5v5, one point per kill, first team to reach 50 kills. Match ends after 15 min.

Player skill metric: TrueSkill's mean μ. More frequent players have bigger μ drop at week 1, but their μ increases faster over the weeks.
The longer the break between two games, the bigger drop in skill. It takes 10 games (3h of gameplay) to regain the skill lost after a break of 1 month.
Most of the top 100 players use the DMR (same range as sniper rifle) and sniper rifle.

08 April 2012

Gold buying patterns

Picks from a paper I wrote about gold buying patterns for FDG 2012. The data comes from an online questionnaire completed from March to May 2010 by 2800+ WoW players from around the world. Unless mentioned, all results are significant with a p-value below 0.01.

  • Overall, 14% of people have ever bought gold.
  • Men are twice more likely to buy gold than women (17% vs 8%), but there is no difference between Asians and Westerners.
  • Achievement increases the likelihood to buy gold, while immersion decreases it. The effect of achievement is stronger on men.
  • 12% of people who only play with people they know IRL have bought gold. This ratio increases to 15% for people who play with both RL relatives and friends made IG, and to 21% for people who play only with friends they made IG.
  • Overall, people who have taken longer breaks from the game are more likely to buy gold.
  • But really, it's a big mess to know which variables influence gold buying: in the correlation graph below, vertices represent variables, and edges bearing positive/negative values indicate positive/negative correlations between two variables. Values closer to 1 in absolute value indicate higher correlations.
  • That's where GLM come in handy: they account for interactions between variables in regressions from multiple variables.
  • Controlling for all other variables, the odds of buying gold increase when playing on a private server, being a man, having frozen one's subscription, having made friends IG, playing for achievement, and having played the game for a long time. On the other hand, the odds
  • Controlling for all other variables, the odds of buying gold decrease when having had a college education, playing for immersion or socializing, and playing with cousins, siblings, or spouse.

07 April 2012

MapReduce for MMOs

MapReduce is a powerful tool to parallelize batches of computations. MMOs may sometimes have to run batches, but from what local game companies tell me, nobody in the game industry is currently using MapReduce. I guess, this is mostly due to studios not knowing what to do with it. Here are some examples.

Business intelligence

Basic metrics such as weekly play time or stop rate can give a rough perspective of the retention of an MMO. These metrics can be estimated with a couple SQL queries on dumps of the production database(s). It starts taking more time and effort to distinguish accross server shard, faction, race, or class. Still, a SQL script running for a few hours can do the job. Fancier analyses such as machine learning or social network graphs explorations take even more time and effort. MapReduce can be used to tune machine learning algorithms through Mahout, and even to process graphs (Google's Pregel also seems interesting for parallel processing of graphs: the Pregel version of PageRank takes 15 lines of code).

Detecting bots, hacks, or gold farmers is not as straightforward, but I think it is doable. First, the typical deviant behaviors have to be determined and made explicit by humans. For instance, speed-hackers send too many messages per second to the server, while gold farmers interact with less players, but more intensely, than normal players. Then, detecting deviant behaviors can be a machine learning classification or a graph parsing problem. In both cases, MapReduce can help.

Game-specific

Matchmaking and ladder: Some pre-calculations or updates to parameters of the ladder and match-making algorithms could be done offline by a small MapReduce cluster. A player's skill is unlikely to change much in 12 hours, so a cron task could run the job twice a day. According to Josh Menke from Blizzard, matchmaking involves gradient descent or Gaussian Density Filtering. Not sure whether Mahout supports GDF, but gradient descent is supported.

Tuning and balancing can take days for system designers. MapReduce could do that automatically: each mapper job is given a particular set of system parameters: player 1 has skill A (cost x SP and inflicts y damage) and skill B (cost z SP and heals w HP), player 2 has skill C (...) and skill D (...). Mappers run a few hundred Monte-Carlo simulations of a player 1 versus player 2 match with a fixed set of parameters (player1:A,B; skillA:x,y; skillB:z,w; ...). When done, mappers pass average statistics (win/loss ratio, average amount of gold at the end of the match, ...) of the 100 matches to reducers who sort them. The interesting configurations for balance are those with a win/loss ratio close to 50%. Naturally, this brute-force way of balancing assumes a proficient AI, and designers will still have to tweak the configurations returned by MapReduce so that they feel fun.

Practical concerns

Engineering detail: MMOs have hundreds of shards, but really only one MapReduce cluster should be needed. Each shard could send its jobs to the MapReduce cluster when it needs them done, and wait asynchronously for the MapReduce answer on a particular port. If the MapReduce job uses data from the production database, producing a daily dump may induce a temporary extra load on the shard's database machines, but this should be fine during empty hours.

MapReduce can be a double-edged sword if overused. Exploring the parameter space of learning algorithms too aggressively may lead to less accurate models.


Edit: Some people have been using MapReduce for analytics: mogade's platform and keighl have been using it through mongodb, but it's more of an engineering constraint (scatter-gather queries in a nosql DB to build a ladder board) than an analytics or machine-learning endeavor.

01 November 2011

21st Century Game Design - Part I

21st Century Game Design, by Chris Bateman and Richard Boon, 2005.

Part I - Games exist primarily to satisfy the needs of an audience

ch1 - Zen game design

Zen Buddhism can not be learned, it can only be experienced. There is no objective perspective on anything. Hence zen game design's tenets: game design reflects needs + there's no single method to design + there exist methods to game design. These methods are:

  • first principles: what you want to do -> game world abstraction -> design -> implementation
  • clone and tweak: most common method. existing design -> tweak -> implementation
  • meta-rules: goal = provoking debate. meta-rules -> design -> implementation
  • expressing technology: in teams without actual game designers. technology -> game implementation
  • Frankenstein: art or technical materials -> design -> implementation
  • story-driven: narrative -> design -> implementation

Participants in the game project: audience, publisher, producer, programmers, artists, marketing/PR, license holder. Example: saving for causal audience is vital; for hardcore audience, it should not break gameplay; for programmers, it's a technical detail; for producer, it's looking at how other games do it.

ch2 - Designing for the market

The commercial success for a medium clears the way for artistic expression, not the way around

A game design is successful when the target audience is satisfied. This justifies the need for an audience model. Existing models: simple distinction hardcore/casual, distinction by genre (but genres are too vague), EA's model, and ihobo's model.

Simple hardcore/casual distinction
hardcore casual
plays lots of games plays few games
game literate game illiterate
plays for the challenge plays to relax, kill time, and just for fun
segment can be polarized: many can buy the same title hard to polarize, diverse and disparate

EA's model:

EA's model take-away: do not ignore hardcores because they are the ones pushing a game to broader segments. Corollary: no TV ads are needed if the game is not made for casuals.

iHobo's model:

Evangelist clusters = gaming press, mainstream press, and the 3 million of hardcores in the world. Target clusters = Testosterone (9M players worldwide), lifestyle (30M), and family (90M) gamers.

Design tools for market penetration (aka demographic game design):

  • Looking for good gameplay (ie the game being performance-oriented, with stats, clear goals and victory conditions) vs good toyplay (unorganized). Hardcores are driven by gameplay, but lifestyle and family gamers are driven by both.
  • Controls should remain accessible for casuals.
  • The minimum play session length is usually expressed in terms of the duration of a level or the time between two save points. For casuals, it should be below 15 minutes, but hardcores do not mind core activities of a game taking at least an hour or two. Ex: a typical DotA match takes 45 to 60 minutes, whereas a (small size) Mine Sweeper can take less than a minute. Nintendo games are also famous for allowing the player to quit at any time and provide core activities of at most a few minutes.
  • The average play session length is also lower for casuals: they may complete one level at a time, whereas hardcores can aim at 10 levels per play session.
  • Play window: total time spent playing the game. The longer the play window, the longer hardcores will spend evangelizing the game. Therefore, despite most of the players not completing the game, content is crucial! The play window can also be extended by introducing hidden features, higher difficulty levels, variety in characters to play with (to increase replayability), and online PVP (although that only works for Testosterone and hardcore gamers).

Phases of penetration: taking the example of The Sims.

  1. Hardcore penetration: the game needs challenge, progress, and depth.
  2. Hardcore evangelism: the game needs to appeal to the Lifestyle gamer, easy to reach fun, strong marketing, and a strong license.
  3. Casual penetration: the game needs fun, toys, short minimum play session.
  4. Casual evangelism: the game needs to get the attention of the mainstream press.

ch3 - Myers-Briggs typology of gamers

Assumption: nature of games people enjoy and frequency of play vary with player personality and reaction to situations. The Myers-Briggs model was developed in the 1940s and indicates how an individual would prefer to react to situations in general. See the Myers-Briggs type frequencies in the US. Four pairs of traits:

Type Opposite type Game design
Introversion (50% of pop)
think then act, needs private time, 1-to-1 communication and relationships
Extroversion (50% of pop)
act then think, likes people, deprived when alone
Most games are played by introverts. Extraverts can take long breaks from the game, so provide a todo list for them when they come back to play, otherwise they'll forget what they had to do in their previous play session. Extraverts like DDR because of its performance aspect.
Sensing (70% of pop)
live in the present, apply common sense, based on prior experience, likes clear and concrete info
iNtuition (30% of pop)
live in near future, new and imaginative approaches, based on theory, comfortable with fuzzy information, seek for patterns)
Learning and problem solving are frequent gameplay elements in many genres. Learning: in tutorials, S will accept linear series of lessons, but N would rather guess by themselves. Problem solving: S will use trial and error, while N will like to use their lateral thinking skills. Therefore, make lateral thinking puzzles (at most) secondary objectives, or allow the player to progress without having completed all of them. Ex: Super Mario 64 only requires 30 stars to unlock new levels. S want simple and usual mechanics, while N won't mind having to guess the rules and a steep learning curve.
Thinking (30% of women, 60% of men)
decide from facts and logic, objective, focus on task, think that conflicts are sometimes unavoidable
Feeling (70% of women, 40% of men)
decide from emotion, subjective, focus on consequences to people, wish to avoid conflicts
Clear goals for T. Personal encouragement for F, but T may feel patronized. Solution: useful AND aesthetic/fun items are rewards that will satisfy both T and F. Gathering collectibles give goals to T, but should not be a grind. F are motivated and rewarded when they see their actions have impact on the world or other characters. T enjoy receiving critical feedback (a game over with tips), but F will take it personally. Ex: Zelda gives clear goals (good for T), falling or getting hit results in losing half a heart (and not instant death) and Link has an impact on the game world (good for F).
Judging (55% of pop)
plan then move, single task at a time, ahead of deadlines, targets and routines to manage life
Perceiving (45% of pop)
plan as you go, multitask, work better before deadline, avoid routine and commitment
J want to beat the game (get all the secret bonuses) and complete objectives. P want to improve their abilities, and enjoy the process. For P, goals completed = feedback that they're on track. Non-linear structure is good for P because if they don't like a level, they can try another and keep progressing. J needs to know what to do to progress. Ex: in Tony Hawk or GTA, players need to collect points (good for J) but they can collect them the way they want (various kinds of skate figures or driving/killing missions or sandbox play, good for P).

TJ vs FP: TJ want challenges to overcome (what most current games provide), FP want easy fun (cf Sims or casual games).

Study hypothesis: hardcore player is a 14-28 year old tech savvy male who plays up to 8 games per month. Supposedly, he plays on his own (hence I), is methodological, goal-oriented enjoys conflicts (T), plays games until completion and looks for perfect score/overachiever (J). Previous quantitative work from the Bartle test by Andreasen showed the average hardcore MMO player is IST. Therefore, let's suppose hardcores are IT. Overall, 15% of women and 35% of men are of type IT.

ch4 - DGD1

DGD1 is intended as a tool to aid in market-oriented game design.

Methods: between 2002 and 2004, ask 408 participants (incl 122 women) to answer a 32-question Myers-Briggs personality test, as well as questions on purchasing and playing habits, and do you consider yourself hardcore, casual, or no idea?. Only look at people who play at least one game per year. Survey advertised on hardcore and casual websites/game portals + university students.

Results: clustering gave a sketchy and incomplete result, and FE and SI dimensions did not help to cluster, but 4 clusters appeared anyway: conqueror (TJ), manager (TP), Wanderer (FP), and participant (FJ). Hypothesis rejected: hardcores are found in E and S (and not only I and T). Still, I and N are higher for hardcores and MMO players than casuals. For each of the four types, twice more respondents reported they were casuals than hardcores.

The DGD1 demographic model
Type Hardcores Desc Casuals Desc Progress Story Social
Conqueror ITJ. Want meaningful challenges, strategies and puzzles, want to complete the game. Want lots of content, try to beat themselves. The game is too easy if they don't die at least a few times. Anger, frustration, boredom, and fiero. ISTJ. FPS and racing games, they play to compete and win. Rely on genre conventions and do not like deviations from the genre. Fiero (although it's oblivious to them) and schadenfreude in PVP, or in GTA for rampages Rapid advancement: stats in RPG, better gear in FPS Focus on plot twists/events, not on characters Online: vocal hardcores from forums and blogs. They also like to win discussions
Manager ITP. Strategy and tactics. Winning is less important than mastering the game systems: process-oriented, not goal oriented. Conquerors consider them rivals and targets. Patient. Look for challenging but not impossible. Don't look for hidden features but rather refine their current knowledge. Fiero. Civ series. ISTP. Want familiar settings and realism. Like construction and management games like SimCity. Hate being stuck even if they suck. Hate interruptions and like smooth difficulty curves. Steady. Give up if no reliable strategy is found quickly. Plot, not characters. None?
Wanderer INFP. Easy fun and toyplay, not challenges. Variety keeps the fun going. Complete levels in aesthetically pleasing ways. Cf Puzzle Bobble/Bust-a-Move: simple controls, bright colors, and actions with direct and satisfying changes to the environment. See also Mario Party and Super Monkey Ball. Need to be able to give up the current task for another different task. May turn to Conqueror or Manager relatives for help. Emotions: finesse, aesthetics, wonder, awe and mystery, but no fiero. ENFP. Want to accomplish something in the game world without the need for challenges. Games = way to relax. Feeling of progression or else boredom. Lack of market vectors to reach them [although nowadays there's Facebook] New toys, colorful and imaginative environments Emotions. Empathy to characters or investment in world/immersion. Talk about what they like but avoid arguments
Participant FJ. Games as social entertainment. Cf DDR, The Sims. Little survey data about this group. Narrative of group of players Characters and emotions, but in control of them, not just spectator. Multiplayer, but must face other players in person, not just online (no MMO)

ch5 - Player abilities

Flow = subjects believe they can complete their activity. Subjects have clear goals and direct and clear feedback. Effortless involvement. Goals should be short-term for participant and conqueror, but long-term for Wanderer and manager because they like to figure out the short-term goals themselves.

Caillois' table of the four categories of play helps understand how flow is related to toyplay. In the table, there really is a continuum between Paidia and Ludus.

The relation between the four play styles of DGD1 and Caillois' categories of games
Conqueror
Agon
Manager
Agon (Alea tolerated)
Participant
Mimicry
Wanderer
Mimicry (Alea tolerated)
Caillois' table of the four categories of play
- Agon
(competition)
Alea
(chance)
Mimicry
(simulation)
Ilinx
(vertigo)
Paidia
(spontaneous play)
Spontaneous races Counting out rhymes, coin flipping Masks and disguisement Children whirling, swinging
Ludus
(structured play)
Sports Betting, lotteries Theatre Skiing, mountain climbing

People with high Myers-Briggs Feeling scores prefer avoiding conflicts, therefore they don't like Agon. They're also more likely to like Mimicry since they focus on people. For example, Wanderers appreciate finesse, which is a component of Mimicry. Ilinx resembles immersion, it appeals to everyone.

Temperament theory gives patterns of behaviors, while Myers-Briggs gives patterns of perception or judgement.

Temperament theory
Temperament Core needs Myers-Briggs traits Skills % of pop
Rational Knowledge, competence NT Strategic: Think and plan ahead, identify the means to achieve a goal, coordinate actions strategically 10%
Idealist Unique identity, search for meaning and significance NF Diplomatic: Resolve conflicts while recognizing individuality, empathy, find similarities through abstraction 15%
Artisan Freedom to act and ability to impact SP Tactical: Read the current content and manage the situation, work out the next step and take action, improvise to overcome problems 25%
Guardian Belonging and sense of responsibility/duty SJ Logistical: Organizing and meeting needs, optimizing and standardizing, protect and ensure safety 50%

Temperament, Myers-Briggs and DGD1
Type Myers-Briggs
traits
Hardcore
temperament
trait
Casual
temperament
trait
Flow provenance Examples
Conqueror TJ strategic logistical Capacity to see in advance how to address problems (strategic) and iterate/repeat to improve/optimize the solution (logistical). Willingness to fail and repeat Production of units in RTS, monsters or bosses with patterns (cf Doom monsters)
Manager TP strategic tactical Planning ahead (strategic) and reacting to rapidly changing situations (tactical). Hardcores like to get lost in their thoughts, ideally without time limitations. Casuals have flow in the action, and need short-term goals. RTS have both spontaneous maneuvers and long-term strategies. Civ, Chess or puzzles for hardcores.
Wanderer FP diplomatic tactical Immersion, explicit short-term goals (tactical). Completion of goals is not a big thing, it happens almost as a side-effect of exploration. Give them time to explore. Platformers (goal is obvious and challenges relatively easy)
Participant FJ diplomatic logistical Feeling of belonging, toyplay, optimize relationships (logistical) with other characters or players, immerse themselves in social situation The Sims, Animal Crossing

Casual audience is best approached with familiar settings and content, and with gameplay that revolves around optimization or thinking on your feet (tactical). Hardcores prefer original games that give them a sense of identity (diplomatic), and problems to solve (strategic), e.g. Final Fantasy focuses on story and strategic battles.

07 October 2011

MMO player research methods

Types of data of interest in MMO player studies:

  • demographic data: country, gender, age, job, psychological traits, tech-saviness, happiness, revenue
  • marketing data: how much spent on games per month, how many games bought,
  • play data (applicable to all games and game genres): weekly play time, average play session duration, game and genre literacy,
  • genre-specific and game-specific data: for MMOs and WoW in particular: who you play with, guild position, achievement/immersion/social motivation scores

Challenges of player studies: using tools and methods to convert data into useful information, and avoiding erroneous conclusions by crossing results obtained from various methods.

List of qualitative tools.
Methods Qualitative Quantitative
Data collection tools Note taking or recording during open-ended interviews, lab studies, think aloud, or participant observation/ethnographic play. All methods gather all types of data - you just need to ask. Snowball sampling is useful to collect more people concerned by the same phenomenon (eg people from the same guild). Questionnaires can gather multiple types of data together, but beware: what people say they do differs from what they actually do. Non-obtrusive logging of game data has the advantage of being objective, and can be done using programmable game add-ons that players need to install. It's even possible to cross game and marketing data together from the developer/publisher side (cf the EverQuest dataset from SOE).
Data analysis tools Note tagging and affinity diagrams are methods used during the note parsing phase. Stop interviewing when respondents do not say/show anything new anymore. Preferably, do not wait to start parsing the data; parsing as notes are taken gives an immediate feedback loop useful to detect useless interview questions, and to know when to stop accumulating data. The most popular tools to use for stat analysis are SPSS, R, and Excel. Obtaining causal relationships is usually quite hard. On the other hand, simple comparisons and correlations are often successful. Regressions can work too.
Data mining/machine learning using Weka or Matlab. Clustering (PCA, LDA, and even KPCA if small dataset) can identify patterns during exploratory phases. Feature selection or decision trees to identify the most important features. Poisson process or Markov chains for temporal evolutions. Bayes, neural network, SVM, random forests, and others for classification.
Pros and cons Deep, and useful to hint at the reasons explaining a particular phenomena, especially in MMOs, where the metagame (forums, blogs, chats) has a huge influence on the actual game.
Snowball sampling in WoW brings lots of players from the same guild(s), or with similar opinions. Therefore (and also because of small sample size): poor ability to generalize.
Qualitative methods can also be used as exploratory studies to help build a quantitative questionnaire with relevant questions.
Broad and useful to detect surface trends.
Sample bias/representativity: it's very hard to select a representative sample of the player population. For example, selecting players from wow.com brings a lot of hardcore players (since hardcore players read forums while the most casual players don't).
Very hard to explain completely a phenomenon because there's always important features missing from the dataset. Machine learning is also difficult for that same reason.
It's very easy to get lost in post-hoc effects, or to simply not be able to explain a particular number because you've never played the game. Generally, demographic data can not be caused by game-specific data: it's not because people are hardcore that they're young, but rather the opposite.
Researchers Bardzell, Bartle, Kow, Nardi, Pace, Pearce, TL Taylor Andreasen (= quantitative Bartle test), Bateman, Ducheneaut, Seay (not working on that domain anymore?), Williams, Yee,