PING:Past Beta Testers of sports text sims

[miami_fan]

Another game has been released and once again the debate begins about "How could (insert bug here) could have made it past the beta testers?" Now, I have never been a beta tester and have no intentions to apply to be one. My attention to detail is not that great when it comes to computers;) To the extent that you can (without breaking any legal agreements), can you give me an idea what you are looking for during the prcoess? How comfortable are you in telling the developers that the game is not ready for release in its current form despite an announced release date? Do you find developers receptive to your findings? I don't want anyone to slam a developer or anything like that. I am just trying to get a better idea of the process. An "under the hood" look at beta testing. Thanks in advance

[Mizzou B-ball fan]

Although I can't cite specifics, I've been involved with the beta testing of OOTP 2007 over the past month or two. You honestly wouldn't believe the sheer amount of scenarios and data that is run through numerous times to produce some of the bugs that programmers eventually fix. There are also a lot of very dedicated fans who run their favorite program through the ringer because they want it to be good.

I know some people complain about a bug here or there, but after being involved in a couple of these beta tests, I really don't have much of a complaint regarding bugs that make it through. There's a cut off point as far as new features go in the process. After that, people just test the same things refining the game. I think there are some games in smaller companies where bugs get through that should not get through, but I think that speaks more to the man-power available to both find and fix problems in a game than any actual oversight.

Probably the only thing that might be improved would be to add more full-time testers for a program, but that's going to increase your price point with the additional costs. I think a much better option is the beta testing. It may let a few problems through, but the vast majority of bugs (90-95%) are squashed in the testing and you don't have to pay $50 for your game that way.

[JonInMiddleGA]

How comfortable are you in telling the developers that the game is not ready for release in its current form despite an announced release date?

Utterly and completely.

Do you find developers receptive to your findings?

Not in the slightest. Easily the single most frustrating experience of my life that involved a computer.

YMMV (and almost certainly would depending upon the developer) but my one & only experience was more than enough to last a lifetime.

[Tim Tellean]

Beta testing is a tricky business. I've done 9 of them so far and every developer is different. Many times stuff is added, subtracted or changed literally to the last minute. With small developers these guys work with fewer testers (Usually 8-10 or so) and unfortunately most of those people do a poor job of testing. You see them on day 1 when they get a copy and then sporadically they will d/l updates and post little to no information. Its tough for a developer because they can't really play the game and code and test it at the same time. Beta testing is a very time consuming effort and truly only a few people do it well and those guys get picked on games from many companies.
Its very tough for a small developer to make the game perfect also because so many machine configurations are different and people don't stay real current on updating there drivers.
I've not found a developer yet that doesn't want to hear it in fact the opposite is true they want to hear it all. Which is also an issue because in testing the issues are numerous and some get lost in the mix.
Some people sign up for beta thinking they get a free game and can do little to no work for it what they fail to realize is the reliance the developer has on them for helping the game the tester sees a post and thinks oh thats been mentioned no need for me to add but in reality there error is different even very subtle difference. Mostly though they don't post and d/l updates looking for a free game.

[miami_fan]

Thanks everyone for the insight. I am sure I am going to think of other questions but now I have a basis to bitch and complain!;) Just kidding.:D

[JeffR]

How comfortable are you in telling the developers that the game is not ready for release in its current form despite an announced release date?

Very. What's the point of being a tester if you're not going to speak up about something like that?

Do you find developers receptive to your findings?

Depends. If you're brand-new to the project and they don't know you, I wouldn't expect any release dates to get held over your concerns. But if you've been around for a while, demonstrated that you know what you're talking about, and have some common sense about evaluating whether or not something is a game-killing bug, then yeah, I'd expect them to be receptive. If you're in that position and they still won't listen, I'd take a walk.

It also helps if you're willing to put in the extra testing time to help fix the problem(s) you're complaining about. If you think you've found a critical bug, a developer builds a new .exe specifically to fix that problem, and you then say you're too busy to test it, I wouldn't blame the dev for taking you less seriously the next time around.

[BYU 14]

Utterly and completely.



Not in the slightest. Easily the single most frustrating experience of my life that involved a computer.

YMMV (and almost certainly would depending upon the developer) but my one & only experience was more than enough to last a lifetime.

I had a fairly good experience testing OOTP on versions 2-4. I think the problem on those early Beta tests was lack of an overall plan and assigning people to check certain parts of the game. I found Markus and co very receptive, but in some cases just hamstrung by the "growing pains" that come as you start a franchise.

I had an experience with one other game (shall remain anonymous) and agree with Jon in that it can be a highly frustrating experience!!

[cartman]

I think the biggest difference between single dev/smaller development shops and the big production studios is the QA process. The big studios have full on regression testing, scripted scenarios, etc. The smaller shops are usually relying on fans of the game, who, for the most part, don't have a clue as to what is involved in a real software QA process.