Category: Quality Assurance

Posted on

Bugs or No Bugs

We have a choice between adding new content and features or consolidating what we have and adding new features later. I’ve written about this before, but I worry it’s a decision that too often is decided in favor of new features.

I’ve taken a stab at this topic before in “Frustration of Reasonable Desires“, bince since the topic has had some time to stew, I wanted to revisit it.

A lack of features is only frustrating when the features that are missing are pretty essential to the game experience. A feature that does not work cannot be truly considered a feature. If the core features of a game do not work well, and since to some degree every other feature depends upon those core features, one could conclude the game does not in fact consist of features, and to opine that more features need to be added at the expense of bug-fixing is to ignore the fact that it is not “features” that will be added, but rather ever more elaoborate bugs.

I’m trying to make the point that there comes a point where one must cosolidate that which one has created. Create, Consolidate, then create again, and then consolidate once more.

I read recently that it is merely a matter of opinion and preference whether or not one fixes bugs in existing features. If the existing bugs are not utter game-breakers then they can (even should) be left as-is in favor of new features and content. I’m reacting to that. I believe bug-free systems, elegant code, efficient production tools, happy customers, and a happy bottom line all go together.

Some good food for thought on the matter has been posted recently at Zen of Design: “Triage and Extrapolation“.

Posted on

MMO Beta Testing

On days where I’m a bit less-than-motivated to write, I usually surf around MMO-related blogs and sites in hopes of coming across something that catches my interest.  Many times these things get my mind working.  This entry is the result of one such sojourn.

MMORPG.com has an article in September of 2006 about Richard Vogel talking about beta testing these days.  I think the thought that most stood out was that the MMO beta tester population is not being utilized to its potential.

Richard mentioned that the MMO beta tester population today is different than it was in the past.  Currently, many people join the beta test simply either to play the game for free or to try it before they buy it.  Those participants who are truly testing the game and submitting reports are fewer than they were in the past.  Having been in the opposite extremes of this “beta test motivation” continuum, I hope to throw out a few ideas on how to further improve upon the existing beta test programs.

One poster on the forum thread commenting on the article captured a few techniques that could vastly improve the beta test process:

You want opinions of the non-vocal majority?  Implement in-game polls.  Ask.  Randomily send tells and request feedback.

In my beta test experiences, I’ve always been “one of the masses” with the test assignment of “just play the game and report any bugs you see”.  As a result I felt a bit aimless, and knowing that anything I do in-game will eventually be wiped, I don’t particularly want to grind xp.  I wanted to have some kind of assignment, even if that assignment is only “do whatever it takes to buy a house and decorate it somehow”.  Also, I wanted some instruction as to how I could submit the most useful bug reports.

So, by simply sending out in-game messages to beta testers requesting the “testing” of certain types of activities as well as certain areas to especially pay attention to, a few things could happen: an increase in the “testing” aspect of the beta, intense testing of particular sub-systems, as well as a likely increased depth of reports.  Imagine an occasional tip on how to best use the screenshot or video capture utilities to create bug reports.

Some forum commenters mentioned what amounts to more enhanced in-game bug reporting tools.  I think it would be quite beneficial to include a small package of relevant utilities with the beta client download and updates that give the typical non-techie-type the ability to easily make screen shots and videos.  For me it took more time than I really liked just locating programs to take screen shots and videos, much less figure out how to use them and then figure out how to create useable reports.

I recognize the beta test period can be simply a stress test opportunity, meaning general player comments and even bug reports are not particularly going to be helpful, especially if you already have a great QA team.  However, even if it’s only a stress test, tell everyone.  Send in-game messages and tells requesting people to try to do as many inventory open-close-move-delete operations as they can over the next minute.  This way you can get focused use, and I bet this focused use could allow for some highly efficient stress tests on top of the regular ad-hoc stress testing.