Accuracy of combat reports - An analysis (Full Version)

All Forums >> [New Releases from Matrix Games] >> War in the Pacific: Admiral's Edition



Message

Yakface -> Accuracy of combat reports - An analysis (1/31/2015 8:34:46 PM)

I had a concern over sync in my game. Which lead me to do some analysis on the reliability of the ground combat portion of the combat reports.

Method

Modified a scn to put 7 Japanese divisions onto Luzon. They made their way to Clark field, as did most of the Allied army on the island.

Once set-up I ran a series of combats. Each involving 2 days of Japanese attacks, one directly after the other. Notes were made of:

1) The actual AV in the hex
2) The raw AV reported in the combat report*
3) the casualties*

* see image

Then the same with the second day's attack and finally the actual AV in the hex on day 3



[image]local://upfiles/21862/8E9EB351A64D4DDCADC60CFCD4B41D37.jpg[/image]



Yakface -> RE: Accuracy of combat reports - Analysis (1/31/2015 8:35:42 PM)

Here are the results + analysis





Editted table to be more explanatory and smaller and a minor error



[image]local://upfiles/21862/8525EF8B21C54879B8DFFAF18BD15E9F.jpg[/image]



Yakface -> RE: Accuracy of combat reports - Analysis (1/31/2015 8:37:08 PM)

Explanation of table:

the first 10 lines are the test. The red figures are the actual observations with some number processing in black

Column E: what the expected AV would be if the casualties in the report had been 100% accurate

Column F: the expected AV in column E minus the actual AV in column D. - figure means the report over estimated losses, a positive figure means more casualties inflicted than were reported.

Column G: actual AV divided by reported loss to get a percentage difference.

Column H: how far the results deviated from the reported as a factor of 2. This is treated differently for under reporting or over reporting. As an under-report of 50% is the same in statistical terms as an over report of 100%. Negative and positive figures have been expressed in a way that can be directly compared:

If the actual result was under-reported in the combat report this calcualtion was applied:

=(figure in column G)*100/100-1

If the report exaggerated the casualties:

=100/(figure in column G*100)-1



Yakface -> RE: Accuracy of combat reports - Analysis (1/31/2015 9:00:05 PM)

Conclusions:

The reports are moderately accurate.

Specifically:

1) the Raw Av in the hex is equal to the actual AV in the Hex +1 for each combat unit in the hex. There is no randomness - it is spot on (with the +1/unit unusual feature).

NB - this may not be true when defending - the test was just on Japanese attacks.

2) There was over reporting and under-reporting at times. More frequently over reporting. However the results didn't deviate by a factor of greater that .54 in either direction. Ie +54% actual over reported (under-reporting casualties in the combat report) or -35% actual over reported (exaggeration of casualties in the report)

There are some uncertainties in the method. It is a little rough and ready (for example a destroyed squad could have already been disabled, or healthy. depending upon which it may have had an AV value or not so counting all destroyed squads as a loss of AV may be innaccurate. However since the vast majority of casualties were diablements, this introduces only minor uncertainty.)



Yakface -> RE: Accuracy of combat reports - Analysis (1/31/2015 9:08:25 PM)

Forgot to say - the result at the bottom is the result from the turn with suspected sync problems.

I don't know if I have enough datapoints (effectively I've got 20: 10 tests, 2 days each) to be able to say with certainty I have a problem, however it looks anomolous (out by a factor of 3.6 compared to a high of .54 in the test). Is there anyone out there with more statistical expertise who is able to say that it is mathematically significant given the size of the sample?



wdolson -> RE: Accuracy of combat reports - Analysis (2/1/2015 2:44:26 AM)

There are known problems with the reporting of specific casualties on the ground combat report. We made some improvements during development, but the way it was designed, it was impossible to make it perfect without a very deep rewrite which we didn't have time for.

Bill



witpqs -> RE: Accuracy of combat reports - Analysis (2/1/2015 3:26:35 AM)

The way I look at it, the staff is restricted to 1940's Morse Code technology for sending combat reports back to me, not to mentioning encryption/decryption. I figure I'm lucky to get reports so quickly!

[:D]



Yakface -> RE: Accuracy of combat reports - Analysis (2/1/2015 7:41:51 AM)

There are a few unusual features - for example, motorized support casualties are added to both the total of vehicle and non-combat numbers, so there is a built in exaggeration there which will affect every combat where a motorized support is damaged/destroyed



btd64 -> RE: Accuracy of combat reports - Analysis (2/1/2015 5:30:42 PM)

Fog of war.[:D]
I had a combat report, several times, different battles, different games, all AI though, say a unit was in a battle when in fact it was either set to defend or in the next hex.
Air battles, I had 12 betty's bombing a base with 14 damaged and 1 killed by flak.
It's not perfect. I like the inaccuracy in the reports. It keeps the game fun and interesting. And coming back for more.[:)]....GP



witpqs -> RE: Accuracy of combat reports - Analysis (2/1/2015 5:37:42 PM)


quote:

ORIGINAL: General Patton

Fog of war.[:D]
I had a combat report, several times, different battles, different games, all AI though, say a unit was in a battle when in fact it was either set to defend or in the next hex.
Air battles, I had 12 betty's bombing a base with 14 damaged and 1 killed by flak.
It's not perfect. I like the inaccuracy in the reports. It keeps the game fun and interesting. And coming back for more.[:)]....GP


Set to defend, yes they include all units there. In a different hex, wow - I haven't hit that one yet!



dr.hal -> RE: Accuracy of combat reports - Analysis (2/1/2015 6:38:30 PM)

Nicely done Yakface, I for one appreciate the effort. Certainly if you want to get "scientific" about it, you would need to increase your data points. But these data that you do provide are enough to give us a feel for what is going on. And in truth, I think it is absolutely "accurate" to be inaccurate, if you get my meaning. As is said above, "fog of war." Additionally it is historically the norm to over exaggerate the damage you do to an enemy. During Vietnam I certainly remember the numbers of dead NVR and Viet Cong being reported and if one were to total those numbers up for the 10 years of conflict, I believe the statisticians stated the result would be more than the male military aged population of North Vietnam. So a "beefed up" number is to be expected. I've always take the reports with a large bucket (not grain) of salt. Thanks for your work. Hal



Yakface -> RE: Accuracy of combat reports - Analysis (2/1/2015 9:45:23 PM)

Yeah - I have no problem with the fog of war. As people have said, uncertainty keeps things interesting. It's useful to know what the limits are to that uncertainty. It seems to be that you can expect to have done somewhere between two thirds and 150% of reported casualties



Page: [1]

Valid CSS!




Forum Software © ASPPlayground.NET Advanced Edition 2.4.5 ANSI
0.9375