[Literature] Surveys in Virtual Worlds

Quick introduction ...

Currently, there are very many researchers in sociology focusing on Virtual World populations. Actually, there is also some psychology, economy, law and even politics research led in MMOs. Various analysis of the MMO worlds can be done :

• Analysis of virtual life : basically how and why do people play, the amount of money produced in an MMOG ...
• Analysis of virtual world possibilities : online/immersive education through an MMOG, Taxing virtual worlds ...
• Comparison to real life : avertising, politics, ...
• But also analysis of real life thanks to virtual life : comparison of LA gangs to WoW guilds, ...

Researchers need data. To collect data from a huge amount of players, they need fieldwork tools. And the best tool for collecting people's thoughts about a very precise topic is the survey.
Basically, surveys can be conducted in the street. That is sometimes expensive and long, and brings few results. Maybe that's why street pollsters are voluntary. Thanks to Graham Bell in 1876, larger scale surveys were made possible. The rain does not impact anymore on the amount of people surveyed in the day. But pollsters still have to survey people themselves. In 1990 started an era of online surveys thanks to HTML pages. Now, pollsters only have to think about their survey, put it online and wait for their <form> to be filled/submitted by millions of people.
What about MMO player surveying? Players have to logout from the game, connect to a website on which they could take the survey. They are no more in a "player" stance : no more blade, gun, fun or handsomeness. They are simply answering a survey addressed to any WWW user, that is to say everyone (actually, not everyone ...). The ideal is an IG survey.

VDCI : the State of the Art

Chronology

In April 2008 Mark Bell, Edward Castronova and Gert Wagner published a paper concerning VASI and VDCI. Topher Zwiers, a SL educator describes a VDCI presentation from Mark Bell in a post on his blog. Castronova published on Terra Nova a very short description of the tool.
But in June 2009 (actually the survey ran for 30 days in early 2009 (02/03 to 03/05).), these same people published another paper discussing how they led a survey in SL thanks to the VDCI.

VASI and VDCI

If you already know about VASI and VDCI, then you can skip this part (or tell me if you see points where I could be wrong or inaccurate).
Researchers looked for a tool that preserves immersion because players have feelings and perceptions that are particular to that environment. The method they propose to solve this immersion break is called VASI, and its implementation VDCI. They used a mailing list, a classified ad system where respondents select themselves and then are teleported somewhere, and a random location protocol where their research avatar was teleported if it was possible. When it was possible, a sort of NPC kiosk appeared. No participation rate can be estimated because this NPC waited for people to talk to it.
The VDCI is a HUD (official HUD description), that is to say a control panel allowing the user to perform actions he is the only one able to see. Particularly filling a form to answer a survey. The player gets this HUD by an NPC, wears it and then can answer the questions. When he has finished, the research avatar gives the player some virtual money.

Implementation

If you already know technically about VASI and VDCI, then you can skip this part (or tell me if you see points where I could be wrong or inaccurate).
The VDCI uses LSL [...] which formatted HTTP calls that use PHP to write the respondent's answers to a MySQL database. Since Second Life is Open Source, and their wiki is quite well documented, it was not very hard to understand : see LSL, HTTPRequest in LSL and the Server Architecture (see RPC server). I think the RPC server must handle a proxy somewhere. It could be worth spending time in SL sources some day ...
Good remarkable thing, when the database did not record whether this was a person responding due to a classified ad, an email or encounter with a random kiosk, it was refered to as <no record>. I presume in the PHP scripts an argument was given to tell the provenance of the answer, and if this argument was missing or incorrect, an error was detected. Anticipating errors and making them appear in the final figure not only show the impact that could have had these errors in the final results, but also show the reliability of the use of the VDCI system in SL. I think that the percentage of <no record> may vary depending on the server(s) bandwidth and processing capacities.

Limits

• identity in SL is fluid, so ensuring respondent identity is difficult. Actually, this is the same for any MMOG : players exchange passwords when they play in teams, guilds, with friends, etc. Also, VDCI captures the avatars name [...] to ensure that the same avatar does not take the survey multiple times. As they write, this does not prohibit a user with multiple avatars to take the survey multiple times. There might be a very easy-to-say solution for this : store the fact that an account took the survey in an account variable. This solution means either UGC API (or language) can set client-side account variables or a server owner (Linden Labs for SL) has scripted the item for the survey (thus the account variable is stored server-side).
• Researchers were only able top place kiosks at 10% of the randomly chosen places, concentrated on the eatsern side [...] much more populated than the western side, so this is not a bias. I honestly dont know if this makes a bias. IRL, voluntary pollsters stand at very populated locations. In any MMOG, there are strategical spots where many people meet : capital city, current expansion zone where every high level is, PVP zones, etc. but people can also meet in a small unfrequented street in any town of the world or during a quest in a very particular place. These "common" zones hosting 3 players a day are part of the world and should not be forgotten.
  Anyway, because there were 1543 respondents based on classified ad and 75 respondents based on the quasi-random protocol for 2094 valid responses, and because for the overall sample the quasi response rate is 2094/1100000 = 0.2%, I consider many active people selected themselves to participate in this survey. These active players are the usual 10% of the population producing 90% of the server content/life. To my mind, passive players have not at all been surveyed.
• This then paid the avatar 250 linden dollars. : a survey should be led, consisting of only one question : how much do you want to receive for this survey? And the answer can range from 0 to NaN. The fact that 90% of the classified ad respondents are willing to get re-contacted does not definitely mean that those people want to be paid for doing it again. I do agree on the fact that quasi-random sample and [...] classified ad sample are the samebut in the way the quasi-random survey was led : only active people were surveyed. So concluding that classified-ad sampling obtains a representative sample of the SL population is not exactly true. Quasi-random is simply more expensive, but the same sort of people are surveyed.
• While conducting a natural experiment about the fieldwork strategy in SL, they found an increeased number of respondents after the new placement of the class ad (in a day : more than 100 compared to the usual 30). Many people may answer the survey because it is something new : a special research avatar, a kiosk, a new ad, a mail. After some days, people are used to the kiosk or the ad, it has become part of the everyday scenery of the place. And this happens faster if the NPC/element is passive : a walking, yelling and bursting NPC with fireworks might take a bit longer to be forgotten/ignored.

What else? What now?

In other MMOGs

SL, as a MMOSG, has much to do with UGC : items, places ... But for typical MMORPGs like WoW, UGC has not the same place. SL VDCI needs the player to wear a HUD which is definitely something available IG thanks to UGC. The only way to implement such HUD from a UGC point of view could be WoW add-ons. I honestly doubt that current WoW LUA API permit transmitting data out from the game. I have not seen such things on wowprogramming or on wowwiki. Other recent MMORPGs dont seem to put forward add-ons, and I think WoW was the first MMO to implement such a programming tool.

MMO firms

Exctracts from the previously quoted article from Mark Bell, Edward Castronova and Gert Wagner :

• About the register of SL users (inhabitants) which can be used for drawing a random sample : Linden Lab does not provide this kind of information for commercial or for research purpose
• According to Linden Lab, from 02/03/09 to 03/05/09 about 1.1 million avatars were active in SL.

If firms were to participate in such surveys or share the use of such tools, it would be easier and safer for IG pollsters to implement their survey. Also, much could be done to ensure the fact that a real person (and not avatar) can answer the survey once and only once.
I contacted Mark W Bell to know if they asked Linden Labs about their survey, what support they could have asked, etc.

Quote of the day

Found on a French forum : je cherche un add-on qui permet d'xp tout en étant AFK pour faire plus de RP (translation : I'm looking for an add-on that could make me xp while being AFK so that I can RP more)