More Picky Data!

Matrix Games Forums

Forums Register Login Photo Gallery Member List Search Calendars FAQ

My Profile Inbox Address Book My Subscription My Forums Log Out

More Picky Data!

View related threads: (in this forum | in all forums)

Logged in as: Guest

Users viewing this topic: none

Printable Version

All Forums >> [New Releases from Matrix Games] >> War in the Pacific: Admiral's Edition >> More Picky Data!

Page: [1]

Message

<< Older Topic Newer Topic >>

More Picky Data! - 10/1/2017 2:20:47 AM

witpqs

Posts: 26087
Joined: 10/4/2004
From: Argleton
Status: offline

I'm currently working my way through parsing the various naval actions in the combat reports.

Here is my approach with the many-headed set of naval actions. I've found that some have introductory messages, such as "31 Coastal gun shots fired in self defense." or "TF 15 troops unloading over beach at Makin, 135,135." Others have no introductory messages at all (after the title line of course). I can pick my way past those because they are predictable based on the type of naval action.

There is a large central section which, varying greatly, might include sections for:
Japanese aircraft
Allied aircraft
Japanese aircraft losses
Allied aircraft losses
Japanese Ships
Allied Ships
Japanese ground losses
Allied ground losses
base/city damages done

After that central section some - not all - actions have a series of lines chronicling the action, such as "CL Capetown firing at 144th Infantry Regiment", "CL Ceres firing at Ambon" and others pertaining to surface combat, etc. Most I will ignore but some will identify LCUs present and I will parse them. What might be in the lower section depends on the type of action, so that is where the code will treat different naval actions differently.

The various blocks of the central section (listed above) are not reliable insofar as which ones are present. I need to key on something to know when the central section has ended and the lower section is beginning. To that end I am going to look for certain key phrases that should never appear in the central section but that do appear in the first line of the lower section. So far I have this list.

" firing at "
" fire at "
" fired at "
" lost overboard "
" lost in surf "
"Maximum visibility "
" launches "
" is sighted "
" eludes ASW attack "
" cannot establish contact "

I have made this a list in the config.ini file so it can be updated quickly and easily because I have a suspicion that AE might hold some rare messages that I haven't seen.

If you know of any other strings that appear in first lines of that lower section please let me know!

_____________________________

Intel Monkey: https://sites.google.com/view/staffmonkeys/home

Post #: 1

RE: More Picky Data! - 10/1/2017 5:08:20 AM

Lokasenna

Posts: 9297
Joined: 3/3/2012
From: Iowan in MD/DC
Status: offline

Don't forget "No activity spotted on shore"! Or whatever it is.

(in reply to witpqs)

Post #: 2

RE: More Picky Data! - 10/1/2017 5:45:23 AM

witpqs

Posts: 26087
Joined: 10/4/2004
From: Argleton
Status: offline

quote:

ORIGINAL: Lokasenna

Don't forget "No activity spotted on shore"! Or whatever it is.

Got it - thanks!

quote:

No activity noticed on shore.

_____________________________

Intel Monkey: https://sites.google.com/view/staffmonkeys/home

(in reply to Lokasenna)

Post #: 3

RE: More Picky Data! - 10/1/2017 6:26:11 AM

BBfanboy

Posts: 18046
Joined: 8/4/2010
From: Winnipeg, MB
Status: offline

I had several naval actions last turn. The descriptive section after the action results always starts with the visual range and weather info. See pic below:

Attachment (1)

_____________________________

No matter how bad a situation is, you can always make it worse. - Chris Hadfield : An Astronaut's Guide To Life On Earth

(in reply to witpqs)

Post #: 4

RE: More Picky Data! - 10/1/2017 2:42:16 PM

witpqs

Posts: 26087
Joined: 10/4/2004
From: Argleton
Status: offline

quote:

ORIGINAL: BBfanboy

I had several naval actions last turn. The descriptive section after the action results always starts with the visual range and weather info. See pic below:

Yes, I am catching surface combat central section with "Maximum visibility ", thanks. I should have been more clear that I am talking about the various action types of naval combats.

quote:

"Pre-Invasion action"
"Invasion Support action"
"Amphibious Assault"
"Naval bombardment"
"Night Naval bombardment"
"Day Time Surface Combat"
"Night Time Surface Combat"
"encounters mine field"
"ASW attack"
"Submarine attack"
"Midget Sub attack"
"Sub attack"
"Sub vs Sub:"

_____________________________

Intel Monkey: https://sites.google.com/view/staffmonkeys/home

(in reply to BBfanboy)

Post #: 5

RE: More Picky Data! - 10/1/2017 4:42:38 PM

BBfanboy

Posts: 18046
Joined: 8/4/2010
From: Winnipeg, MB
Status: offline

So, things like "**MAGAZINE EXPLODES** on [ship]" or "**MASSIVE EXPLOSION** on [ship]"?

[ship] hits rocks while unloading troops at [hex] (not sure the syntax is exactly correct)

_____________________________

No matter how bad a situation is, you can always make it worse. - Chris Hadfield : An Astronaut's Guide To Life On Earth

(in reply to witpqs)

Post #: 6

RE: More Picky Data! - 10/1/2017 4:56:05 PM

witpqs

Posts: 26087
Joined: 10/4/2004
From: Argleton
Status: offline

quote:

ORIGINAL: BBfanboy

So, things like "**MAGAZINE EXPLODES** on [ship]" or "**MASSIVE EXPLOSION** on [ship]"?
Those are already extracted by the air attacks code, where they form the very last group of messages in any battle. They will also be extracted in the naval battles code, where they are mixed into the lower section. Like so:

quote:

Reduced visibility due to Rain with 7% moonlight
Maximum visibility in Rain and 7% moonlight: 1,000 yards
Range closes to 21,000 yards...
Range closes to 19,000 yards...
Range closes to 17,000 yards...
Range closes to 15,000 yards...
Range closes to 13,000 yards...
Range closes to 11,000 yards...
Range closes to 10,000 yards...
Range closes to 9,000 yards...
Range closes to 8,000 yards...
Range closes to 7,000 yards...
Range closes to 6,000 yards...
Range closes to 5,000 yards...
Range closes to 4,000 yards...
Range closes to 3,000 yards...
Range closes to 2,000 yards...
Range closes to 1,000 yards...
Magazine explodes on DD Yukikaze
DD Yukikaze sunk by BB Prince of Wales at 1,000 yards
BB Prince of Wales engages xAP Koan Maru at 1,000 yards
DD Bulmer engages xAP Koan Maru at 1,000 yards
Range increases to 2,000 yards
BB Prince of Wales engages DD Karii at 2,000 yards
BB Prince of Wales engages xAP Koan Maru at 2,000 yards
xAP Koan Maru sunk by DD Express at 2,000 yards
Range increases to 4,000 yards
DD Karii engages DD Express at 4,000 yards
DD Vampire engages DD Karii at 4,000 yards
Japanese Task Force Manages to Escape
Task forces break off...

Capturing those messages is one of the reasons I need to detect where the lower section begins.

quote:

[ship] hits rocks while unloading troops at [hex] (not sure the syntax is exactly correct)

I thought that was in a different report? I'll search the sample combat reports I'm using.

_____________________________

Intel Monkey: https://sites.google.com/view/staffmonkeys/home

(in reply to BBfanboy)

Post #: 7

RE: More Picky Data! - 10/1/2017 4:57:21 PM

witpqs

Posts: 26087
Joined: 10/4/2004
From: Argleton
Status: offline

In the combat reports I am finding that ' rock ' only appears in ship names.

_____________________________

Intel Monkey: https://sites.google.com/view/staffmonkeys/home

(in reply to witpqs)

Post #: 8

RE: More Picky Data! - 10/1/2017 4:58:50 PM

witpqs

Posts: 26087
Joined: 10/4/2004
From: Argleton
Status: offline

This is the current triggers list:

" firing at "
" fire at "
" fired at "
" lost overboard "
" lost in surf "
"Maximum visibility "
"Reduced visibility "
" launches "
" is sighted "
" eludes ASW attack "
" cannot establish contact "
"No activity noticed on shore."

_____________________________

Intel Monkey: https://sites.google.com/view/staffmonkeys/home

(in reply to witpqs)

Post #: 9

RE: More Picky Data! - 10/1/2017 5:59:20 PM

witpqs

Posts: 26087
Joined: 10/4/2004
From: Argleton
Status: offline

Updated list:

" firing at "
" fire at "
" fired at "
" lost overboard "
" lost in surf "
" accidentally lost "
" damaged beyond repair "
" lost from landing craft "
"Maximum visibility "
"Reduced visibility "
" launches "
" is sighted "
" eludes ASW attack "
" cannot establish contact "
"No activity noticed on shore."
" mines cleared"

_____________________________

Intel Monkey: https://sites.google.com/view/staffmonkeys/home

(in reply to witpqs)

Post #: 10

RE: More Picky Data! - 10/1/2017 6:01:26 PM

Zorch

Posts: 7087
Joined: 3/7/2010
Status: offline

quote:

ORIGINAL: witpqs

Updated list:

" firing at "
" fire at "
" fired at "
" lost overboard "
" lost in surf "
" accidentally lost "
" damaged beyond repair "
" lost from landing craft "
"Maximum visibility "
"Reduced visibility "
" launches "
" is sighted "
" eludes ASW attack "
" cannot establish contact "
"No activity noticed on shore."
" mines cleared"

" rescues 7 castaways on 3 hour cruise ".

(in reply to witpqs)

Post #: 11

RE: More Picky Data! - 10/1/2017 6:39:50 PM

witpqs

Posts: 26087
Joined: 10/4/2004
From: Argleton
Status: offline

quote:

ORIGINAL: Zorch

quote:

ORIGINAL: witpqs

Updated list:

" firing at "
" fire at "
" fired at "
" lost overboard "
" lost in surf "
" accidentally lost "
" damaged beyond repair "
" lost from landing craft "
"Maximum visibility "
"Reduced visibility "
" launches "
" is sighted "
" eludes ASW attack "
" cannot establish contact "
"No activity noticed on shore."
" mines cleared"

" rescues 7 castaways on 3 hour cruise ".

That messages only occurs on the last turn of the game...

BTW, did they get rescued? I never saw the last episode.

_____________________________

Intel Monkey: https://sites.google.com/view/staffmonkeys/home

(in reply to Zorch)

Post #: 12

RE: More Picky Data! - 10/1/2017 6:44:28 PM

Lowpe

Posts: 22133
Joined: 2/25/2013
Status: offline

Lots of great midget sub messages.

(in reply to witpqs)

Post #: 13

RE: More Picky Data! - 10/1/2017 7:20:55 PM

witpqs

Posts: 26087
Joined: 10/4/2004
From: Argleton
Status: offline

Another update:

" firing at "
" fire at "
" fired at "
" lost overboard "
" lost in surf "
" accidentally lost "
" damaged beyond repair "
" lost from landing craft "
"Maximum visibility "
"Reduced visibility "
" launches "
" is sighted "
" eludes ASW attack "
" cannot establish contact "
"No activity noticed on shore."
" mines cleared"
"Poor visibility "
"Low visibility "

_____________________________

Intel Monkey: https://sites.google.com/view/staffmonkeys/home

(in reply to Lowpe)

Post #: 14

RE: More Picky Data! - 10/1/2017 7:22:48 PM

witpqs

Posts: 26087
Joined: 10/4/2004
From: Argleton
Status: offline

quote:

ORIGINAL: Lowpe

Lots of great midget sub messages.

Ya know, it might be a genetic predisposition or something but midget subs almost always kick my butt at Pearl Harbor!

_____________________________

Intel Monkey: https://sites.google.com/view/staffmonkeys/home

(in reply to Lowpe)

Post #: 15

RE: More Picky Data! - 10/1/2017 8:40:03 PM

witpqs

Posts: 26087
Joined: 10/4/2004
From: Argleton
Status: offline

And yet a newer list:

" Battery engaging "
" Ships Reported to be Approaching!"
" accidentally lost "
" cannot establish contact "
" damaged beyond repair "
" elects not to launch "
" eludes ASW attack "
" fire at "
" fired at "
" firing at "
" firing to "
" is located "
" is sighted "
" launches "
" lost from landing craft "
" lost in surf "
" lost overboard "
" mine cleared"
" mines cleared"
"Low visibility "
"Maximum visibility "
"No activity noticed on shore."
"Poor visibility "
"Reduced sighting "
"Reduced visibility "

_____________________________

Intel Monkey: https://sites.google.com/view/staffmonkeys/home

(in reply to witpqs)

Post #: 16

RE: More Picky Data! - 10/1/2017 9:32:39 PM

Zorch

Posts: 7087
Joined: 3/7/2010
Status: offline

quote:

ORIGINAL: witpqs

And yet a newer list:

" Battery engaging "
" Ships Reported to be Approaching!"
" accidentally lost "
" cannot establish contact "
" damaged beyond repair "
" elects not to launch "
" eludes ASW attack "
" fire at "
" fired at "
" firing at "
" firing to "
" is located "
" is sighted "
" launches "
" lost from landing craft "
" lost in surf "
" lost overboard "
" mine cleared"
" mines cleared"
"Low visibility "
"Maximum visibility "
"No activity noticed on shore."
"Poor visibility "
"Reduced sighting "
"Reduced visibility "

" Raises shields and fires photon torpedo ".
" Finds Amelia Earhart ".
" Raises skull & crossbones flag ".

Yes, they did rescued in a TV movie after the series was canceled.

(in reply to witpqs)

Post #: 17

RE: More Picky Data! - 10/1/2017 10:28:20 PM

witpqs

Posts: 26087
Joined: 10/4/2004
From: Argleton
Status: offline

quote:

ORIGINAL: Zorch

quote:

ORIGINAL: witpqs

And yet a newer list:

" Battery engaging "
" Ships Reported to be Approaching!"
" accidentally lost "
" cannot establish contact "
" damaged beyond repair "
" elects not to launch "
" eludes ASW attack "
" fire at "
" fired at "
" firing at "
" firing to "
" is located "
" is sighted "
" launches "
" lost from landing craft "
" lost in surf "
" lost overboard "
" mine cleared"
" mines cleared"
"Low visibility "
"Maximum visibility "
"No activity noticed on shore."
"Poor visibility "
"Reduced sighting "
"Reduced visibility "

" Raises shields and fires photon torpedo ".
" Finds Amelia Earhart ".
" Raises skull & crossbones flag ".

Yes, they did rescued in a TV movie after the series was canceled.

The other two I'll put in but this one yields an error:

quote:

Amelia Earhart not found!

_____________________________

Intel Monkey: https://sites.google.com/view/staffmonkeys/home

(in reply to Zorch)

Post #: 18

RE: More Picky Data! - 10/2/2017 11:06:57 PM

MakeeLearn

Posts: 4278
Joined: 9/11/2016
Status: offline

"I'm afraid I can't do that. This mission is too important for me to allow you to jeopardize it."

(in reply to witpqs)

Post #: 19

RE: More Picky Data! - 10/3/2017 1:17:18 AM

witpqs

Posts: 26087
Joined: 10/4/2004
From: Argleton
Status: offline

quote:

ORIGINAL: MakeeLearn

"I'm afraid I can't do that. This mission is too important for me to allow you to jeopardize it."

The mission is proceeding well, HAL.

In the last stages of parsing the combat reports for useful info, which was way more involved than I had anticipated. Think the 80/20 rule.

_____________________________

Intel Monkey: https://sites.google.com/view/staffmonkeys/home

(in reply to MakeeLearn)

Post #: 20

RE: More Picky Data! - 10/3/2017 1:36:27 AM

witpqs

Posts: 26087
Joined: 10/4/2004
From: Argleton
Status: offline

BTW here is an update of the list that started this thread.

" Battery engaging "
" Ships Reported to be Approaching!"
" accidentally lost "
" cannot establish contact "
" Defenses firing "
" damaged beyond repair "
" elects not to launch "
" eludes ASW attack "
" fire at "
" fired at "
" firing at "
" firing to "
" is located "
" is sighted "
" launches "
" lost from landing craft "
" lost in surf "
" lost overboard "
" mine cleared"
" mines cleared"
"Low visibility "
"Maximum visibility "
"No activity noticed on shore."
"Poor visibility "
"Reduced sighting "
"Reduced visibility "

_____________________________

Intel Monkey: https://sites.google.com/view/staffmonkeys/home

(in reply to witpqs)

Post #: 21

RE: More Picky Data! - 10/3/2017 1:55:27 AM

Zorch

Posts: 7087
Joined: 3/7/2010
Status: offline

" ramming speed "
" cloaking device activated "

Attachment (1)

(in reply to witpqs)

Post #: 22

RE: More Picky Data! - 10/4/2017 8:20:36 PM

Will_L

Posts: 245
Joined: 9/28/2000
From: NYC-Queens
Status: offline

"You can have this oar when you pry it from my cold, dead fingers."

quote:

ORIGINAL: Zorch

" ramming speed "
" cloaking device activated "

(in reply to Zorch)

Post #: 23

RE: More Picky Data! - 10/8/2017 5:09:41 PM

witpqs

Posts: 26087
Joined: 10/4/2004
From: Argleton
Status: offline

Here is the current list. I suspect there might be more which need including but this list gets me through 1,328 days of combat reports from my latest PBM.

" Battery engaging "
" Defenses firing "
" Ships Reported to be Approaching!"
" accidentally lost "
" acting as spotter "
" cannot establish contact "
" damaged beyond repair "
" dead in the water ..."
" dropped into water "
" elects not to launch "
" eludes ASW attack "
" fire at "
" fired at "
" firing at "
" firing to "
" is located "
" is sighted "
" launches "
" lost from landing craft "
" lost in surf "
" lost overboard "
" mine cleared"
" mines cleared"
"Ammo storage explosion on "
"Fuel storage explosion on "
"Heavy smoke from fires obscuring "
"Improved night sighting "
"Low visibility "
"Magazine explodes on "
"Massive explosion on "
"Maximum visibility "
"No activity noticed on shore."
"Poor visibility "
"Reduced sighting "
"Reduced visibility "

On a slightly different topic, I have a question about ship damage. In the section of a battle where the ships of each side are all listed there is also shown whatever hits each ships is thought to have sustained and certain damage levels. (The lists below use my variable names for each item.)

For hits we have:
"shlhits" - Shell hits.
"trphits" - Torpedo hits.
"bmbhits" - Bomb hits.
"hits" - This is how hits on submarines underwater are reported.
"minhits" - Mine hits.

For damage levels we have:
"dmgonfire" - On fire.
"dmghvfire" - Heavy fires.
"dmghvdmg" - Heavy damage.
"dmgsunk" - Sunk.

There are also additional damage messages that appear in the lower section of the battle report. These are the ones I know about.
"dmgmage" - Magazine explosion.
"dmgmase" - Massive explosion.
"dmgflse" - Fuel storage explosion.
"dmgamse" - Ammunition storage explosion.
"dmgosmk" - Obscured by heavy smoke.
"dmgdead" - Dead in the water.

My question is: are there more of those additional damage messages? Do you have samples?

< Message edited by witpqs -- 10/8/2017 5:11:07 PM >

_____________________________

Intel Monkey: https://sites.google.com/view/staffmonkeys/home

(in reply to Will_L)

Post #: 24

RE: More Picky Data! - 10/8/2017 6:15:25 PM

BBfanboy

Posts: 18046
Joined: 8/4/2010
From: Winnipeg, MB
Status: offline

Are you only interested in ship-ship and amphib combats right now?
Here is an old CR from the Marianas scenario that I dug up. Mostly sub and carrier aircraft action ...

Attachment (1)

_____________________________

No matter how bad a situation is, you can always make it worse. - Chris Hadfield : An Astronaut's Guide To Life On Earth

(in reply to witpqs)

Post #: 25

RE: More Picky Data! - 10/8/2017 6:51:53 PM

Zorch

Posts: 7087
Joined: 3/7/2010
Status: offline

"warpcoredamage" - call Scotty asap

Attachment (1)

(in reply to BBfanboy)

Post #: 26

RE: More Picky Data! - 10/8/2017 8:53:58 PM

inqistor

Posts: 1813
Joined: 5/12/2010
Status: offline

Kobayashi Maru?

(in reply to Zorch)

Post #: 27

RE: More Picky Data! - 10/9/2017 12:23:19 AM

witpqs

Posts: 26087
Joined: 10/4/2004
From: Argleton
Status: offline

quote:

ORIGINAL: Zorch

"warpcoredamage" - call Scotty asap

An oldie but a goody!

_____________________________

Intel Monkey: https://sites.google.com/view/staffmonkeys/home

(in reply to Zorch)

Post #: 28

RE: More Picky Data! - 10/9/2017 12:35:01 AM

witpqs

Posts: 26087
Joined: 10/4/2004
From: Argleton
Status: offline

quote:

ORIGINAL: BBfanboy

Are you only interested in ship-ship and amphib combats right now?
Here is an old CR from the Marianas scenario that I dug up. Mostly sub and carrier aircraft action ...

You blink your eyes you miss a lot!

I have combat report parsing fully functional.

The thing is, to catch something I have to know about it. Even though I certainly "knew" about all of those additional dmaage messages having seen them at one time or another, I did not recall them all when I put the list together. Out of the 6 there now I had 3 up until two days ago, and 4 up until last night/early this morning. I just let it run on my test data until it found something that did not compute and made barfing sounds. Then I added that damage type. I expect someone to report a damage message "Captain's strawberries missing, crew abandons ship!". When it comes up, I'll add it.

In a similar vein, that long list of trigger messages is how I am alerting to a transition from the upper section of certain battle reports to the lower section (naval battles in this case). So as I ran through the test data I heard "chug, chug, BARF!" whereupon I would add the newly identified trigger message. Then as they say "lather, rinse, repeat...".

It successfully goes all the way through my (I think it is) 1,328 days long test data set, so this is a call for possible missing items. AE is known for having rare messages that might not be in my test set.

_____________________________

Intel Monkey: https://sites.google.com/view/staffmonkeys/home

(in reply to BBfanboy)

Post #: 29

RE: More Picky Data! - 10/13/2017 10:01:27 PM

witpqs

Posts: 26087
Joined: 10/4/2004
From: Argleton
Status: offline

Current list:

" Battery engaging "
" Defenses firing "
" Ships Reported to be Approaching!"
" accidentally lost "
" acting as spotter "
" cannot acquire firing solution "
" cannot establish contact "
" damaged beyond repair "
" dead in the water ..."
" decides to submerge "
" dropped into water "
" elects not to launch "
" eludes ASW attack "
" fire at "
" fired at "
" firing at "
" firing to "
" is located "
" is sighted "
" launches "
" lost from landing craft "
" lost in surf "
" lost overboard "
" low on gun ammo "
" mine cleared"
" mines cleared"
" on the surface"
"Ammo storage explosion on "
"Fuel storage explosion on "
"Heavy smoke from fires obscuring "
"Improved night sighting "
"Low visibility "
"Magazine explodes on "
"Massive explosion on "
"Maximum visibility "
"No activity noticed on shore."
"Poor visibility "
"Reduced sighting "
"Reduced visibility "

_____________________________

Intel Monkey: https://sites.google.com/view/staffmonkeys/home

(in reply to witpqs)

Post #: 30