RE: New Update v1.1.0 Now Available!

Matrix Games Forums

Forums Register Login Photo Gallery Member List Search Calendars FAQ

My Profile Inbox Address Book My Subscription My Forums Log Out

RE: New Update v1.1.0 Now Available!

View related threads: (in this forum | in all forums)

Logged in as: Guest

Users viewing this topic: none

Printable Version

All Forums >> [New Releases from Matrix Games] >> World in Flames >> RE: New Update v1.1.0 Now Available!

Page: << < prev 1 2 [3]

Message

<< Older Topic Newer Topic >>

RE: New Update v1.1.0 Now Available! - 1/27/2014 10:25:47 PM

Shannon V. OKeets

Posts: 22095
Joined: 5/19/2005
From: Honolulu, Hawaii
Status: offline

quote:

ORIGINAL: Dabrion

quote:

ORIGINAL: paulderynck

The most frustrating thing is that many bug fixes seem to generate other bugs - often in game situations (FREX Multiple States of War ) or option sets that no one has time to try. Add to that if you are doing an update a week, the beta testers (all of whom actually have real lives to lead) have about 48 hours to test things.

Trying to code a game with so many complicated rules, exceptions and options - and stay as true as possible to the rules - makes MWiF "A Subway Named Mobius".

That's what makes it so different from strategic games designed for a computer where the designer controls everything he wants to simulate and can simply toss out some especially complex-to-code feature or option he'd thought of putting in originally - because nobody knows it would violate rule XYZ.

Complexity and coupling of code passages is not intrinsic to wargames. There are means to address this. I find the "it's so incredibly hard, it can never be done" tune to be a sad excuse for not using modern software engineering and agile development techniques.

The issue here is not with the problem domain or the solution itself. If you had unit tests, you would immediately know if that new code change impacts code written earlier. You also could automate 80% of what is called "beta testing" in MWiF terms. I can see how this all is a very frustrating experience, monkey patching code all the time.. doesn't have to be that way..

"Unit Test", what a great phrase! Could you give one example of it for MWIF? That shouldn't be difficult since you feel sure that 80% of the code could be automated for testing using "unit tests".

By the way, I have no interest in a theoretical definition - I live and work in a world of practical applications.

_____________________________

Steve

Perfection is an elusive goal.

(in reply to Dabrion)

Post #: 61

RE: New Update v1.1.0 Now Available! - 1/27/2014 11:37:40 PM

paulderynck

Posts: 8201
Joined: 3/24/2007
From: Canada
Status: offline

quote:

ORIGINAL: Dabrion

Complexity and coupling of code passages is not intrinsic to wargames. There are means to address this. I find the "it's so incredibly hard, it can never be done" tune to be a sad excuse for not using modern software engineering and agile development techniques.

Never said it couldn't be done. Just said it's complex and extremely ambitious.

Yes the methods you describe are the proper ones... for a development studio with employee beta testers.

< Message edited by paulderynck -- 1/28/2014 12:39:50 AM >

_____________________________

Paul

(in reply to Dabrion)

Post #: 62

RE: New Update v1.1.0 Now Available! - 1/28/2014 12:00:15 AM

Dabrion

Posts: 733
Joined: 11/5/2013
From: Northpole
Status: offline

quote:

ORIGINAL: Shannon V. OKeets

quote:

ORIGINAL: Dabrion

quote:

ORIGINAL: paulderynck

The most frustrating thing is that many bug fixes seem to generate other bugs - often in game situations (FREX Multiple States of War ) or option sets that no one has time to try. Add to that if you are doing an update a week, the beta testers (all of whom actually have real lives to lead) have about 48 hours to test things.

Trying to code a game with so many complicated rules, exceptions and options - and stay as true as possible to the rules - makes MWiF "A Subway Named Mobius".

That's what makes it so different from strategic games designed for a computer where the designer controls everything he wants to simulate and can simply toss out some especially complex-to-code feature or option he'd thought of putting in originally - because nobody knows it would violate rule XYZ.

Complexity and coupling of code passages is not intrinsic to wargames. There are means to address this. I find the "it's so incredibly hard, it can never be done" tune to be a sad excuse for not using modern software engineering and agile development techniques.

The issue here is not with the problem domain or the solution itself. If you had unit tests, you would immediately know if that new code change impacts code written earlier. You also could automate 80% of what is called "beta testing" in MWiF terms. I can see how this all is a very frustrating experience, monkey patching code all the time.. doesn't have to be that way..

"Unit Test", what a great phrase! Could you give one example of it for MWIF? That shouldn't be difficult since you feel sure that 80% of the code could be automated for testing using "unit tests".

By the way, I have no interest in a theoretical definition - I live and work in a world of practical applications.

If your current workflow looks something like this:

1) implement feature
2) generate new build
3) pass on to testing crowd
4) crowd reports back
5) terminate or goto 1)

Then there is no reason why 3-5 cannot be done by a machine, if you can express the protocol and design constraints for a machine.

0) write UT for feature
1) implement feature
2) run feature against UT and all other UT
3) *all* UT pass or fail
5) if all pass, feature is positively implemented, else goto 1)
6) collect UT for future use

This is the general setting for test-driven development, providing a growing set of UT for regression testing. If new code conflicts an old UT, red flags will come up. The better the tests cover the lines of code, the better the criterion will be. The counter intuitive part might be that you will want to write UT before writing actual code. New builds should automatically run against the complete suite (aka continuous integration).

That was probably the theoretic part you are not interested in. Simple truth is, I cannot provide a relevant example without knowing your code base. 80% is a lower bound, one that I feel comfortable stating without knowing the code base.

Yet here is how I would approach: start to dissect RAW on a paragraph basis, write a UT for paragraphs (that's about the right granularity to start out). There are also many examples in RAW (and the FAQ) that make usable UT. After all these are the basic tools to arbitrate rule disputes over the table, so they are just as good here. Some of the tests will look very stupid, one liners that assert simple conditions. Others will have you build or mock an environment to run the test in (Form class in a headless environment). Tests can have more code than the feature. When you fix a bug, along with the fix goes a new UT, explicitly asserting the fix.

It probably helps to structure reusable parts into services/libraries, in order to have one test for the service (stacking, supply comes to mind). That way you have all logic in callable (and independently testable) tasklets that can be called from your frontend. The major hindrance for coming up with a "practical example" is that I don't know the structure of your code. But since past bugfixes, had the "fixes the bug in one place but not in another" disease, I conclude you could do some refactoring there. If there is any logic in the frontend code classes (I am guessing most of it is), I would move that to dedicated libraries, that do not depend on the frontend classes.

If you are interested in something more practical, I suggest we take it offline (email or pm me).

_____________________________

�WiF is like sex: sure, it may give some practical results, but that's not why we do it.�
- Richard P. Feynman, 'WiF, Sex, and the Dual Slit Experiment'.

(in reply to Shannon V. OKeets)

Post #: 63

RE: New Update v1.1.0 Now Available! - 1/28/2014 5:49:56 PM

Shannon V. OKeets

Posts: 22095
Joined: 5/19/2005
From: Honolulu, Hawaii
Status: offline

You mentioned stacking limits. Okay, ...

The program currently checks stacking during: land movement, rail movement, naval movement, undoing moves, conquest, surrender, carpet bombing, and air transport. The other major occasion is when it determines whether units are selectable, which gets called innumerable times during game play. Overstacked conditions are processed on a hex by hex basis as a digression which is in a do while loop (destroying units until the hex is no longer overstacked).

The actual test for whether a hex is overstacked depends on the hex terrain, presence of cities, ports, whether the hex is a coastal hex, weather, which country controls the hex, the units in the hex, which countries the units belong to, the major powers that control the units, and the relationships between all those various countries.

Relationships between countries are quite varied. It includes such things as conquered countries where there can be units belonging to the original country that are still fighting against the conqueror, partisans which are fighting against the conqueror, territorial units which are fighting on the side of the conqueror, and other odds ducks like notional units, the AOI unit, warlords, captured naval units, and lend leased air units. For instance, having a partisan and a territorial belonging to a conquered minor country in the same hex means the hex is overstacked. There are also the rules about the two Chinese minor countries and that the USSR is the controlling major power for the Communist Chinese but that minor country isn't aligned to the USSR. Oh, and invading units (from the air or the sea) are not overstacked in a hex while they are invading, even though their presence means that units from both sides are in the hex. Of course air units flying missions in a hex are not subject to stacking limits, so whether the air unit is flying or sitting on the ground matters too. Therefore, phase and subphase of the game need to be considered.

This list of things the unit test for stacking limits would need to cover is off the top of my head. I see I left out damaged ports and whether the city is a capital (for oil and build points).

The program does an examination of all hexes on the map when it checks for stacking limits, so the unit test could do that in several passes, loading different saved games for optional rules, having the weather conditions variable across all the terrain types, including coastal hexes, and which countries control which hexes. It would also need to have all the different relationships between countries covered, both major powers and minor countries. For instance, when there are German units in Finland while the USSR and Finland are at war but the USSR and Germany are not. Then someone would need to set up all the different overstacked conditions using the debug Place tool (which is sensitive to permitting overstacking, so that code would need to be modified). To be thorough, (and that's our goal with the unit test isn't it?), all the possible combinations and permutations of overstacking would need to be present in the saved game, just to make sure the unit test identifies all overstacked hexes while simultaneously giving a passing grade on weird combinations of units in a hex which do not violate stacking.

That shouldn't be too difficult to do. Someone who knows what they are doing and understands all the WIF rules well could probably set up those saved games in a month or two. So, one unit test knocked off the to-do list. What other unit tests should be created? How about one of the types of combat?

---
My point is that creating Units Tests for MWIF would be another project unto itself, of comparable size to writing and debugging the code for MWIF.

_____________________________

Steve

Perfection is an elusive goal.

(in reply to Dabrion)

Post #: 64

RE: New Update v1.1.0 Now Available! - 1/28/2014 6:16:10 PM

AxelNL

Posts: 2386
Joined: 9/24/2011
From: The Netherlands
Status: offline

I have looked professionally (albeit from a managerial point of view) towards automated testing. It is viable for regression testing if you are willing to invest a bit more time than a test cycle to create e.g. an automated script. As soon as content changes which need to be tested the business case gets more difficult. We have had some success in automating SAP regression testing for billing purposes, but when I worked at a communications company we had a nasty ' time to market' push for complete new products - and investing in automation was not feasible.

For MWiF I can image to create a script based on a ' mouse-tamer'. A little program which learnes mouse movements and clicks in order to recreate that later as a sort of real-time macro. When you start such a macro your see the mouse moving and clicking exactly as learned. In that case one can regression test e.g. the set-up of a Global War scenario, or do a complicated naval battle based on a save-game. As Betatesters are volunteers, this is not something which will add to their motivation - which is probably a factor which can be important. When things go wrong the mouse tamer does not know that, so either you still need to invest the time to oversee the process - or analyse log files (if that would be feasible) to find out where things went wrong.

(in reply to Shannon V. OKeets)

Post #: 65

RE: New Update v1.1.0 Now Available! - 1/28/2014 8:11:46 PM

paulderynck

Posts: 8201
Joined: 3/24/2007
From: Canada
Status: offline

quote:

ORIGINAL: Shannon V. OKeets
This list of things the unit test for stacking limits would need to cover is off the top of my head. I see I left out damaged ports and whether the city is a capital (for oil and build points).

Let's not forget we need saved games for all the different interaction of options that effect stacking.
No partisans, no Territorials, no Guns & Divisions. Now turn each of those on or off in all combinations. That's 8 saved games already.

Let's not forget about the "what-if" stacking that needs to be analyzed to determine if a retreat path is available or not, since this is even before the destroy overstacked units digression.

< Message edited by paulderynck -- 1/28/2014 9:13:07 PM >

_____________________________

Paul

(in reply to Shannon V. OKeets)

Post #: 66

RE: New Update v1.1.0 Now Available! - 2/1/2014 8:17:43 AM

yvesp

Posts: 2083
Joined: 9/12/2008
Status: offline

quote:

ORIGINAL: Shannon V. OKeets

quote:

ORIGINAL: Dabrion

quote:

ORIGINAL: paulderynck

The most frustrating thing is that many bug fixes seem to generate other bugs - often in game situations (FREX Multiple States of War ) or option sets that no one has time to try. Add to that if you are doing an update a week, the beta testers (all of whom actually have real lives to lead) have about 48 hours to test things.

Trying to code a game with so many complicated rules, exceptions and options - and stay as true as possible to the rules - makes MWiF "A Subway Named Mobius".

That's what makes it so different from strategic games designed for a computer where the designer controls everything he wants to simulate and can simply toss out some especially complex-to-code feature or option he'd thought of putting in originally - because nobody knows it would violate rule XYZ.

Complexity and coupling of code passages is not intrinsic to wargames. There are means to address this. I find the "it's so incredibly hard, it can never be done" tune to be a sad excuse for not using modern software engineering and agile development techniques.

The issue here is not with the problem domain or the solution itself. If you had unit tests, you would immediately know if that new code change impacts code written earlier. You also could automate 80% of what is called "beta testing" in MWiF terms. I can see how this all is a very frustrating experience, monkey patching code all the time.. doesn't have to be that way..

"Unit Test", what a great phrase! Could you give one example of it for MWIF? That shouldn't be difficult since you feel sure that 80% of the code could be automated for testing using "unit tests".

By the way, I have no interest in a theoretical definition - I live and work in a world of practical applications.

As a professional expert Software Architect and programmer, my opinion is that unit test are only good in a handful of cases. Theoretical books about it are usually writen by people who don't do practical, industrial programming.

Unit testing logically requires testing each and every function against all possible input. That's just not possible considering that there are far more different input combinations than their are stars in the sky... So the next step is to reduce the number of inputs so that at least each branch in the tested code is run. That's what is considered the minimal unit test and it is usually already difficult enough to implement correctly. For exemple, you are supposed to test code that catches errors that are not even supposed even to happen... So you have to bring up some ways to generate these errors.

All this to say that writing a correct set of unit tests takes far more than twice the time it took to write the original code.

Now, in addition, the methods you test don't live in limbo : they are in the program environment, which itself may change the code behaviour. The most obvious case for MWIF are optional rules. A good unit test should at least minimally check each method against all possible combinations of optional rules. I did not count, but I believe that's more than 60. Even if you can reduce the number of relevant options that could possibly affect the function,you'll likely get more than ten in most cases. That just about a thousand cases...

Last, but not least, unit testing has its own limits: every time you change a method, you have to update the tests. Every time you split a method, you have to discard the old tests and write new tests for two methods...

The net result of relying solely on extensive unit testing is that very soon, everytime you code, you feel like youare glued: for even a simple change in a method, which you logically know is correct, you may have to adapt hundreds of lines of test results,if not the tests themselves.

Well well...

All this to say that unit testing is far from being the alpha and omega of writing correct and resilient code. Relying on a huge batch of unit test is a poor way for discarding one's own errors to the next programmer. It doesn't warrant solid software architecture and actually hinders any late change to the architecture, considering that heavy change means dropping thousand of lines of test code.

Another drawback to entranched unit testing is that, for easily understood reasons, programmers then tend to write huge methods, which goes against simplicity and reuse.
On my part, my methods usually rarely exceed five line of code. Often,they are just one line or two. Unit testing for that ? Come on!

A last word: what I say must not even be read as a is. Suppose I am writing sofware for running a plane or a nuclear plant. In that case I would insist on extensive unit tests, well chosen (not all methods, but critical methods such as that one that recovers the input from sensor X or Y, which may in turn use utility methods) ; it would carry a hefty cost of course ; but that cost would be tied with the security side of the project.

Just my two cents...
Yves

< Message edited by yvesp -- 2/1/2014 9:21:00 AM >

(in reply to Shannon V. OKeets)

Post #: 67

RE: New Update v1.1.0 Now Available! - 2/1/2014 9:02:10 AM

AxelNL

Posts: 2386
Joined: 9/24/2011
From: The Netherlands
Status: offline

quote:

ORIGINAL: yvesp

quote:

ORIGINAL: Shannon V. OKeets

quote:

ORIGINAL: Dabrion

quote:

ORIGINAL: paulderynck

The most frustrating thing is that many bug fixes seem to generate other bugs - often in game situations (FREX Multiple States of War ) or option sets that no one has time to try. Add to that if you are doing an update a week, the beta testers (all of whom actually have real lives to lead) have about 48 hours to test things.

Trying to code a game with so many complicated rules, exceptions and options - and stay as true as possible to the rules - makes MWiF "A Subway Named Mobius".

That's what makes it so different from strategic games designed for a computer where the designer controls everything he wants to simulate and can simply toss out some especially complex-to-code feature or option he'd thought of putting in originally - because nobody knows it would violate rule XYZ.

Complexity and coupling of code passages is not intrinsic to wargames. There are means to address this. I find the "it's so incredibly hard, it can never be done" tune to be a sad excuse for not using modern software engineering and agile development techniques.

The issue here is not with the problem domain or the solution itself. If you had unit tests, you would immediately know if that new code change impacts code written earlier. You also could automate 80% of what is called "beta testing" in MWiF terms. I can see how this all is a very frustrating experience, monkey patching code all the time.. doesn't have to be that way..

"Unit Test", what a great phrase! Could you give one example of it for MWIF? That shouldn't be difficult since you feel sure that 80% of the code could be automated for testing using "unit tests".

By the way, I have no interest in a theoretical definition - I live and work in a world of practical applications.

As a professional expert Software Architect and programmer, my opinion is that unit test are only good in a handful of cases. Theoretical books about it are usually writen by people who don't do practical, industrial programming.

Unit testing logically requires testing each and every function against all possible input. That's just not possible considering that there are far more different input combinations than their are stars in the sky... So the next step is to reduce the number of inputs so that at least each branch in the tested code is run. That's what is considered the minimal unit test and it is usually already difficult enough to implement correctly. For exemple, you are supposed to test code that catches errors that are not even supposed even to happen... So you have to bring up some ways to generate these errors.

All this to say that writing a correct set of unit tests takes far more than twice the time it took to write the original code.

Now, in addition, the methods you test don't live in limbo : they are in the program environment, which itself may change the code behaviour. The most obvious case for MWIF are optional rules. A good unit test should at least minimally check each method against all possible combinations of optional rules. I did not count, but I believe that's more than 60. Even if you can reduce the number of relevant options that could possibly affect the function,you'll likely get more than ten in most cases. That just about a thousand cases...

Last, but not least, unit testing has its own limits: every time you change a method, you have to update the tests. Every time you split a method, you have to discard the old tests and write new tests for two methods...

The net result of relying solely on extensive unit testing is that very soon, everytime you code, you feel like youare glued: for even a simple change in a method, which you logically know is correct, you may have to adapt hundreds of lines of test results,if not the tests themselves.

Well well...

All this to say that unit testing is far from being the alpha and omega of writing correct and resilient code. Relying on a huge batch of unit test is a poor way for discarding one's own errors to the next programmer. It doesn't warrant solid software architecture and actually hinders any late change to the architecture, considering that heavy change means dropping thousand of lines of test code.

Another drawback to entranched unit testing is that, for easily understood reasons, programmers then tend to write huge methods, which goes against simplicity and reuse.
On my part, my methods usually rarely exceed five line of code. Often,they are just one line or two. Unit testing for that ? Come on!

A last word: what I say must not even be read as a is. Suppose I am writing sofware for running a plane or a nuclear plant. In that case I would insist on extensive unit tests, well chosen (not all methods, but critical methods such as that one that recovers the input from sensor X or Y, which may in turn use utility methods) ; it would carry a hefty cost of course ; but that cost would be tied with the security side of the project.

Just my two cents...
Yves

+1

You also could be a good writer I think. My sincere compliments for your eloquent English! (I lived a year in Paris so I feel I have enough experience to compliment you here

)

< Message edited by AxelNL -- 2/1/2014 10:03:27 AM >

(in reply to yvesp)

Post #: 68

RE: New Update v1.1.0 Now Available! - 2/1/2014 9:19:26 AM

yvesp

Posts: 2083
Joined: 9/12/2008
Status: offline

quote:

ORIGINAL: AxelNL

You also could be a good writer I think. My sincere compliments for your eloquent English! (I lived a year in Paris so I feel I have enough experience to compliment you here )

Thanks

That's very nice from you. I do my best to write correct English, and I usually read in English books to improve myself. From what you tell, this has not been in vain...

Thanks again,

Yves

(in reply to AxelNL)

Post #: 69

RE: New Update v1.1.0 Now Available! - 2/1/2014 9:28:13 AM

AxelNL

Posts: 2386
Joined: 9/24/2011
From: The Netherlands
Status: offline

quote:

ORIGINAL: yvesp

quote:

ORIGINAL: AxelNL

You also could be a good writer I think. My sincere compliments for your eloquent English! (I lived a year in Paris so I feel I have enough experience to compliment you here )

Thanks

That's very nice from you. I do my best to write correct English, and I usually read in English books to improve myself. From what you tell, this has not been in vain...

Thanks again,

Yves

Not in vain at all! My pleasure! I have fond memories of living in France - but not of the French ability to converse in English. I paid them back by being as good in French as they were in English

. Might have become much better in the mean time, however. I lived in Paris in 1991.

To be fair to Dabrion: I think his use of English is also very very good. But for a German that is a bit less special, I dare to say. (sorry if I abuse in one sentence the 2 great nationalities around me!)

(in reply to yvesp)

Post #: 70

RE: New Update v1.1.0 Now Available! - 2/1/2014 11:10:23 AM

Dabrion

Posts: 733
Joined: 11/5/2013
From: Northpole
Status: offline

Am I supposed to answer this display of ignorance and denial? I will not!

_____________________________

�WiF is like sex: sure, it may give some practical results, but that's not why we do it.�
- Richard P. Feynman, 'WiF, Sex, and the Dual Slit Experiment'.

(in reply to AxelNL)

Post #: 71

RE: New Update v1.1.0 Now Available! - 2/1/2014 11:31:24 AM

yvesp

Posts: 2083
Joined: 9/12/2008
Status: offline

quote:

ORIGINAL: AxelNL

Not in vain at all! My pleasure! I have fond memories of living in France - but not of the French ability to converse in English. I paid them back by being as good in French as they were in English . Might have become much better in the mean time, however. I lived in Paris in 1991.

We might have met...
Well, you're right, French are notoriously bad at foreign languages. My accent in English is quite thick (well, books don't really help there), and American people tell me that I have a "cute French accent". I know what that means

I think the next generation will be better at languages ; the French education system is slowly addressing the problem.

Yves

(in reply to AxelNL)

Post #: 72

RE: New Update v1.1.0 Now Available! - 2/1/2014 2:06:35 PM

Centuur

Posts: 8802
Joined: 6/3/2011
From: Hoorn (NED).
Status: offline

quote:

ORIGINAL: Dabrion

Am I supposed to answer this display of ignorance and denial? I will not!

Why? Are you saying your English isn't good at all? Ignorance? How so? I don't know about the others here, but you English is as good as mine, I think...

If you think you're English isn't good at all, than you are totally wrong. Than you are in denial...

Sorry, but I couldn't resist a little teasing here. I hope you don't mind...

_____________________________

Peter

(in reply to Dabrion)

Post #: 73

RE: New Update v1.1.0 Now Available! - 2/3/2014 8:02:43 AM

TykeAbroad

Posts: 8
Joined: 1/8/2014
Status: offline

My 2 cents. Although I would naturally love to be playing NetPlay, with no errors, 24 hrs a day (I was seriously hooked on the board game), and have my 'argghhh' moments when something goes wrong, I've waited about 15+ years (since first discussions I remember about a computer version) to be this close and am loving being even this close. So keep up the good work on the patches, we'll get there, and I for one will wait with a happy smile on my face.

(in reply to Centuur)

Post #: 74

RE: New Update v1.1.0 Now Available! - 2/3/2014 12:21:53 PM

pzgndr

Posts: 3170
Joined: 3/18/2004
From: Maryland
Status: offline

quote:

ORIGINAL: yvesp

Unit testing logically requires testing each and every function against all possible input. That's just not possible considering that there are far more different input combinations than their are stars in the sky...

All this to say that writing a correct set of unit tests takes far more than twice the time it took to write the original code.

The Barbarossa and Guadalcanal scenarios already provide pretty good test cases for most functions. Once the AIO is running and Steve can run quick AIO vs AIO games, that could help troubleshoot issues and verify fixes. More incentive to get the AIO up and running...

(in reply to yvesp)

Post #: 75