RE: Building the MWiF Test Plan

Matrix Games Forums

Forums Register Login Photo Gallery Member List Search Calendars FAQ

My Profile Inbox Address Book My Subscription My Forums Log Out

RE: Building the MWiF Test Plan

View related threads: (in this forum | in all forums)

Logged in as: Guest

Users viewing this topic: none

Printable Version

All Forums >> [New Releases from Matrix Games] >> World in Flames >> RE: Building the MWiF Test Plan

Page: << < prev 1 [2]

Message

<< Older Topic Newer Topic >>

RE: Building the MWiF Test Plan - 9/7/2007 2:07:21 AM

Shannon V. OKeets

Posts: 22095
Joined: 5/19/2005
From: Honolulu, Hawaii
Status: offline

quote:

ORIGINAL: NeBert
Great discussion!
Until now I was more a consumer of the forum but with some experience in testing and creating testplans I would like to bring in some additional points.

1. Patrice mentioned the problem to just jump into a specific situation of the game (I call it Test Setup): This will be one of the most time consuming tasks for testers without (or with only little) output to the programmers!

Possible solution could be
- to have some kind of Testsetup-Editor (yes I know, this means effort) but it will pay back more than this considering some 10 - 20 testers.
- a pool of saved games within the testgroup to have lots of starting points for different tasks (e.g. Naval Landings, ...)

2. Bug-Reporting
My experience is that once I find a bug the description for the programmer takes quite a time. Some Configuration-Export function (Unit-Setup etc.) will help progammers understanding the problem and reduce the non-testing time for testers.

3. Bug-tracing
A tracing tool is essential! Even if it�s just an Excel-Sheet (better would be a small Database).

4. What about different Hardware- and Software-Configurations among the testers? (RAM, Harddisk, Operating System, ....) Is it already in test or is it part of the coming integration tests?

5. Test-Tracing
A database for all testcases and their performance is as important as for bug-tracing (again, Excel could be sufficient with the problem of multi-user access)

Hope I could help a bit

Thanks.

1 - There is a debugging function (Place) that lets players place units on the map. It was part of CWIF and I have fixed dozens of bugs replated to it but it still is such a powerful tool that it is easy to crash the game by placing units in unreasonable locations ("Look! There's a Chinese militia in Berlin! - And Germany is in the middle of the Production Phase!"). A pool of saved games corresponding to test cases is an excellent idea.

2 - MadExcept was doing a fine job automatically reporting errors to me via email with the internal program call sequence. We were using it for version 3.xx. But I lost it when I reinstalled my compiler in the spring. I should have it reinstalled for 5.00.

3 - Greyshaft's area.

4 - MadExcept provides that. Also, even without MadExcept, most beta testers use one machine so the information only needs to be gathered once.

5 - Greyshaft's area.

_____________________________

Steve

Perfection is an elusive goal.

(in reply to NeBert)

Post #: 31

RE: Building the MWiF Test Plan - 9/7/2007 10:17:49 PM

NeBert

Posts: 53
Joined: 2/11/2007
Status: offline

quote:

Nebert,
Thanks for your input. I have to go and feed two-year old Greyshaft Junior but we'll talk more later.

Seems we have some similar "constraints" for spare time activities like testing complex software. We just sent our two little Generals to bed...

I fully agree to you in dividing testers and test planners - different people have differnt ideas and my experience at work is that 1+1 is normally more than 2. I would expect better output, most probably even quicker that acting only with lone fighters. Avoiding double work will certainly increase testers morale as well.

So, due to my time-constraints (family, job is demanding in addition) I didn�t apply for a tester in the recent call, but with some additional informations about the expectations (or in other words: a clear outcome out of the discussion about the testing-philosophy) and if a rough timeframe of about 5 hours a week (peak workload might be higher) is enough I could try to help pushing the development forward.

Regards

(in reply to Greyshaft)

Post #: 32

RE: Building the MWiF Test Plan - 9/7/2007 11:38:15 PM

Greyshaft

Posts: 2252
Joined: 10/27/2003
From: Sydney, Australia
Status: offline

quote:

ORIGINAL: NeBert
So, due to my time-constraints (family, job is demanding in addition) I didn�t apply for a tester in the recent call, but with some additional informations about the expectations (or in other words: a clear outcome out of the discussion about the testing-philosophy) and if a rough timeframe of about 5 hours a week (peak workload might be higher) is enough I could try to help pushing the development forward.

Constraints noted.
Resume noted.
You're in.
Now raise your right hand and repeat after me... "I, NeBert, do hearby solemnly swear etc etc"

Welcome to the Test Planner Team. Remember that this does not give you access to the MWiF software or the Development Forums. If either of us is killed or captured then Matrix will disavow all knowledge of our actions.

* Please visit the download page of www.a-d-g.com.au and download the WiF rules.
* Please send me your email by PM

_____________________________

/Greyshaft

(in reply to NeBert)

Post #: 33

RE: Building the MWiF Test Plan - 9/8/2007 3:44:34 AM

siRkid

Posts: 6650
Joined: 1/29/2002
From: Orland FL
Status: offline

I do software testing for a living and have been a beta tester on a lot of games, I was even the coordinator for War in the Pacific. I am not applying for a position because I know nothing of the game but I would like to offer one suggestion. Take the game manual and turn all the statements into questions and build a checklist. For example lets say the Rules state that a unit can only move one hex when it is out of supply. You check list would have a question that reads, "Can you only move a unit that is out of supply one hex?" Then you could divide the list up and assign them to the testers. Just my two cents.

< Message edited by Kid -- 9/8/2007 3:45:35 AM >

_____________________________

Former War in the Pacific Test Team Manager and Beta Tester for War in the East.

(in reply to Greyshaft)

Post #: 34

RE: Building the MWiF Test Plan - 9/8/2007 4:30:42 AM

Froonp

Posts: 7995
Joined: 10/21/2003
From: Marseilles, France
Status: offline

quote:

ORIGINAL: Kid

I do software testing for a living and have been a beta tester on a lot of games, I was even the coordinator for War in the Pacific. I am not applying for a position because I know nothing of the game but I would like to offer one suggestion. Take the game manual and turn all the statements into questions and build a checklist. For example lets say the Rules state that a unit can only move one hex when it is out of supply. You check list would have a question that reads, "Can you only move a unit that is out of supply one hex?" Then you could divide the list up and assign them to the testers. Just my two cents.

Hopefully, something I can understand in this thread !!! Thanks Kid !!!

So, Greyshaft, understood ? Turn all into questions as Kid said !!!

(in reply to siRkid)

Post #: 35

RE: Building the MWiF Test Plan - 9/8/2007 4:46:58 AM

jchastain

Posts: 2164
Joined: 8/8/2003
From: Marietta, GA
Status: offline

I would actually say turn everything into 2 questions. You should test the positive and negative of every rule. So if the rule is "To load a ground unit on amphibeous transport it must be in supply" then you should test:

Can I load a land unit onto transport when it is in supply?
Am I prevented from loading a land unit onto transport when it is out of supply?

(in reply to Froonp)

Post #: 36

RE: Building the MWiF Test Plan - 9/8/2007 4:56:56 AM

Greyshaft

Posts: 2252
Joined: 10/27/2003
From: Sydney, Australia
Status: offline

quote:

ORIGINAL: Kid
I do software testing for a living and have been a beta tester on a lot of games, I was even the coordinator for War in the Pacific. I am not applying for a position because I know nothing of the game but I would like to offer one suggestion. Take the game manual and turn all the statements into questions and build a checklist. For example lets say the Rules state that a unit can only move one hex when it is out of supply. You check list would have a question that reads, "Can you only move a unit that is out of supply one hex?" Then you could divide the list up and assign them to the testers. Just my two cents.

Thanks for the input Kid. I have taken discrete topics as defined in the original published WiF rules and copied each of them onto a separate tab of an Excel sheet (see screenshot below - note how the tabs correlate to the index). I will track how MWiF modifies the rules and then have converted the rules into questions (see screenshot in post @17 in this thread on previous page). Sounds like I'm in line with your suggestion but please feel free to offer ideas on fine tuning the model.