Matrix Games Forums

 Forums  Register  Login  Photo Gallery  Member List  Search  Calendars  FAQ 

My Profile  Inbox  Address Book  My Subscription  My Forums  Log Out

Accuracy of combat reports - An analysis

 
View related threads: (in this forum | in all forums)

Logged in as: Guest
Users viewing this topic: none
  Printable Version
All Forums >> [New Releases from Matrix Games] >> War in the Pacific: Admiral's Edition >> Accuracy of combat reports - An analysis Page: [1]
Login
Message << Older Topic   Newer Topic >>
Accuracy of combat reports - An analysis - 1/31/2015 8:34:46 PM   
Yakface


Posts: 846
Joined: 8/5/2006
Status: offline 		I had a concern over sync in my game. Which lead me to do some analysis on the reliability of the ground combat portion of the combat reports.

Method

Modified a scn to put 7 Japanese divisions onto Luzon. They made their way to Clark field, as did most of the Allied army on the island.

Once set-up I ran a series of combats. Each involving 2 days of Japanese attacks, one directly after the other. Notes were made of:

1) The actual AV in the hex
2) The raw AV reported in the combat report*
3) the casualties*

* see image

Then the same with the second day's attack and finally the actual AV in the hex on day 3






Attachment (1)

< Message edited by Yakface -- 1/31/2015 10:47:32 PM >
Post #: 1
RE: Accuracy of combat reports - Analysis - 1/31/2015 8:35:42 PM   
Yakface


Posts: 846
Joined: 8/5/2006
Status: offline 		Here are the results + analysis





Editted table to be more explanatory and smaller and a minor error






Attachment (1)

< Message edited by Yakface -- 1/31/2015 10:38:58 PM >

(in reply to Yakface)
Post #: 2
RE: Accuracy of combat reports - Analysis - 1/31/2015 8:37:08 PM   
Yakface


Posts: 846
Joined: 8/5/2006
Status: offline 		Explanation of table:

the first 10 lines are the test. The red figures are the actual observations with some number processing in black

Column E: what the expected AV would be if the casualties in the report had been 100% accurate

Column F: the expected AV in column E minus the actual AV in column D. - figure means the report over estimated losses, a positive figure means more casualties inflicted than were reported.

Column G: actual AV divided by reported loss to get a percentage difference.

Column H: how far the results deviated from the reported as a factor of 2. This is treated differently for under reporting or over reporting. As an under-report of 50% is the same in statistical terms as an over report of 100%. Negative and positive figures have been expressed in a way that can be directly compared:

If the actual result was under-reported in the combat report this calcualtion was applied:

=(figure in column G)*100/100-1

If the report exaggerated the casualties:

=100/(figure in column G*100)-1


< Message edited by Yakface -- 1/31/2015 9:48:50 PM >

(in reply to Yakface)
Post #: 3
RE: Accuracy of combat reports - Analysis - 1/31/2015 9:00:05 PM   
Yakface


Posts: 846
Joined: 8/5/2006
Status: offline 		Conclusions:

The reports are moderately accurate.

Specifically:

1) the Raw Av in the hex is equal to the actual AV in the Hex +1 for each combat unit in the hex. There is no randomness - it is spot on (with the +1/unit unusual feature).

NB - this may not be true when defending - the test was just on Japanese attacks.

2) There was over reporting and under-reporting at times. More frequently over reporting. However the results didn't deviate by a factor of greater that .54 in either direction. Ie +54% actual over reported (under-reporting casualties in the combat report) or -35% actual over reported (exaggeration of casualties in the report)

There are some uncertainties in the method. It is a little rough and ready (for example a destroyed squad could have already been disabled, or healthy. depending upon which it may have had an AV value or not so counting all destroyed squads as a loss of AV may be innaccurate. However since the vast majority of casualties were diablements, this introduces only minor uncertainty.)

< Message edited by Yakface -- 1/31/2015 10:39:39 PM >

(in reply to Yakface)
Post #: 4
RE: Accuracy of combat reports - Analysis - 1/31/2015 9:08:25 PM   
Yakface


Posts: 846
Joined: 8/5/2006
Status: offline 		Forgot to say - the result at the bottom is the result from the turn with suspected sync problems.

I don't know if I have enough datapoints (effectively I've got 20: 10 tests, 2 days each) to be able to say with certainty I have a problem, however it looks anomolous (out by a factor of 3.6 compared to a high of .54 in the test). Is there anyone out there with more statistical expertise who is able to say that it is mathematically significant given the size of the sample?

< Message edited by Yakface -- 1/31/2015 10:40:16 PM >

(in reply to Yakface)
Post #: 5
RE: Accuracy of combat reports - Analysis - 2/1/2015 2:44:26 AM   
wdolson

 

Posts: 10398
Joined: 6/28/2006
From: Near Portland, OR
Status: offline 		There are known problems with the reporting of specific casualties on the ground combat report. We made some improvements during development, but the way it was designed, it was impossible to make it perfect without a very deep rewrite which we didn't have time for.

Bill

_____________________________

WitP AE - Test team lead, programmer

(in reply to Yakface)
Post #: 6
RE: Accuracy of combat reports - Analysis - 2/1/2015 3:26:35 AM   
witpqs


Posts: 26087
Joined: 10/4/2004
From: Argleton
Status: offline 		The way I look at it, the staff is restricted to 1940's Morse Code technology for sending combat reports back to me, not to mentioning encryption/decryption. I figure I'm lucky to get reports so quickly!



_____________________________

Intel Monkey: https://sites.google.com/view/staffmonkeys/home

(in reply to wdolson)
Post #: 7
RE: Accuracy of combat reports - Analysis - 2/1/2015 7:41:51 AM   
Yakface


Posts: 846
Joined: 8/5/2006
Status: offline 		There are a few unusual features - for example, motorized support casualties are added to both the total of vehicle and non-combat numbers, so there is a built in exaggeration there which will affect every combat where a motorized support is damaged/destroyed

(in reply to witpqs)
Post #: 8
RE: Accuracy of combat reports - Analysis - 2/1/2015 5:30:42 PM   
btd64


Posts: 9973
Joined: 1/23/2010
From: Mass. USA. now in Lancaster, OHIO
Status: offline 		Fog of war.
I had a combat report, several times, different battles, different games, all AI though, say a unit was in a battle when in fact it was either set to defend or in the next hex.
Air battles, I had 12 betty's bombing a base with 14 damaged and 1 killed by flak.
It's not perfect. I like the inaccuracy in the reports. It keeps the game fun and interesting. And coming back for more.....GP


_____________________________

Intel i7 4.3GHz 10th Gen,16GB Ram,Nvidia GeForce MX330

AKA General Patton

WPO,WITP,WITPAE-Mod Designer/Tester
DWU-Beta Tester
TOAW4-Alpha/Beta Tester

"Do everything you ask of those you command"....Gen. George S. Patton

(in reply to Yakface)
Post #: 9
RE: Accuracy of combat reports - Analysis - 2/1/2015 5:37:42 PM   
witpqs


Posts: 26087
Joined: 10/4/2004
From: Argleton
Status: offline
quote:

ORIGINAL: General Patton

Fog of war.
I had a combat report, several times, different battles, different games, all AI though, say a unit was in a battle when in fact it was either set to defend or in the next hex.
Air battles, I had 12 betty's bombing a base with 14 damaged and 1 killed by flak.
It's not perfect. I like the inaccuracy in the reports. It keeps the game fun and interesting. And coming back for more.....GP


Set to defend, yes they include all units there. In a different hex, wow - I haven't hit that one yet!

_____________________________

Intel Monkey: https://sites.google.com/view/staffmonkeys/home

(in reply to btd64)
Post #: 10
RE: Accuracy of combat reports - Analysis - 2/1/2015 6:38:30 PM   
dr.hal


Posts: 3335
Joined: 6/3/2006
From: Covington LA via Montreal!
Status: offline 		Nicely done Yakface, I for one appreciate the effort. Certainly if you want to get "scientific" about it, you would need to increase your data points. But these data that you do provide are enough to give us a feel for what is going on. And in truth, I think it is absolutely "accurate" to be inaccurate, if you get my meaning. As is said above, "fog of war." Additionally it is historically the norm to over exaggerate the damage you do to an enemy. During Vietnam I certainly remember the numbers of dead NVR and Viet Cong being reported and if one were to total those numbers up for the 10 years of conflict, I believe the statisticians stated the result would be more than the male military aged population of North Vietnam. So a "beefed up" number is to be expected. I've always take the reports with a large bucket (not grain) of salt. Thanks for your work. Hal

_____________________________


(in reply to witpqs)
Post #: 11
RE: Accuracy of combat reports - Analysis - 2/1/2015 9:45:23 PM   
Yakface


Posts: 846
Joined: 8/5/2006
Status: offline 		Yeah - I have no problem with the fog of war. As people have said, uncertainty keeps things interesting. It's useful to know what the limits are to that uncertainty. It seems to be that you can expect to have done somewhere between two thirds and 150% of reported casualties

< Message edited by Yakface -- 2/1/2015 10:45:47 PM >

(in reply to dr.hal)
Post #: 12
Page:   [1]
All Forums >> [New Releases from Matrix Games] >> War in the Pacific: Admiral's Edition >> Accuracy of combat reports - An analysis Page: [1]
Jump to:





New Messages No New Messages
Hot Topic w/ New Messages Hot Topic w/o New Messages
Locked w/ New Messages Locked w/o New Messages
 Post New Thread
 Reply to Message
 Post New Poll
 Submit Vote
 Delete My Own Post
 Delete My Own Thread
 Rate Posts


Forum Software © ASPPlayground.NET Advanced Edition 2.4.5 ANSI

0.703