On days where I’m a bit less-than-motivated to write, I usually surf around MMO-related blogs and sites in hopes of coming across something that catches my interest. Many times these things get my mind working. This entry is the result of one such sojourn.
MMORPG.com has an article in September of 2006 about Richard Vogel talking about beta testing these days.  I think the thought that most stood out was that the MMO beta tester population is not being utilized to its potential.
Richard mentioned that the MMO beta tester population today is different than it was in the past. Currently, many people join the beta test simply either to play the game for free or to try it before they buy it. Those participants who are truly testing the game and submitting reports are fewer than they were in the past. Having been in the opposite extremes of this “beta test motivation” continuum, I hope to throw out a few ideas on how to further improve upon the existing beta test programs.
One poster on the forum thread commenting on the article captured a few techniques that could vastly improve the beta test process:
You want opinions of the non-vocal majority? Implement in-game polls. Ask. Randomily send tells and request feedback.
In my beta test experiences, I’ve always been “one of the masses” with the test assignment of “just play the game and report any bugs you see”. As a result I felt a bit aimless, and knowing that anything I do in-game will eventually be wiped, I don’t particularly want to grind xp. I wanted to have some kind of assignment, even if that assignment is only “do whatever it takes to buy a house and decorate it somehow”. Also, I wanted some instruction as to how I could submit the most useful bug reports.
So, by simply sending out in-game messages to beta testers requesting the “testing” of certain types of activities as well as certain areas to especially pay attention to, a few things could happen: an increase in the “testing” aspect of the beta, intense testing of particular sub-systems, as well as a likely increased depth of reports. Imagine an occasional tip on how to best use the screenshot or video capture utilities to create bug reports.
Some forum commenters mentioned what amounts to more enhanced in-game bug reporting tools. I think it would be quite beneficial to include a small package of relevant utilities with the beta client download and updates that give the typical non-techie-type the ability to easily make screen shots and videos. For me it took more time than I really liked just locating programs to take screen shots and videos, much less figure out how to use them and then figure out how to create useable reports.
I recognize the beta test period can be simply a stress test opportunity, meaning general player comments and even bug reports are not particularly going to be helpful, especially if you already have a great QA team. However, even if it’s only a stress test, tell everyone. Send in-game messages and tells requesting people to try to do as many inventory open-close-move-delete operations as they can over the next minute. This way you can get focused use, and I bet this focused use could allow for some highly efficient stress tests on top of the regular ad-hoc stress testing.