RE: New tool: WitpDecoder; No more spreadsheets!

Matrix Games Forums

Forums Register Login Photo Gallery Member List Search Calendars FAQ

My Profile Inbox Address Book My Subscription My Forums Log Out

RE: New tool: WitpDecoder; No more spreadsheets!

View related threads: (in this forum | in all forums)

Logged in as: Guest

Users viewing this topic: none

Printable Version

All Forums >> [Current Games From Matrix.] >> [World War II] >> War In The Pacific - Struggle Against Japan 1941 - 1945 >> RE: New tool: WitpDecoder; No more spreadsheets!

Page: << < prev 2 3 [4] 5 6 next > >>

Message

<< Older Topic Newer Topic >>

RE: New tool: WitpDecoder; No more spreadsheets! - 11/19/2006 4:23:18 AM

Reg

Posts: 2787
Joined: 5/26/2000
From: NSW, Australia
Status: offline

quote:

ORIGINAL: Helpless

I have no problem to run witpdecoder on CHS. If later on it got screwed by the game code itself , nothing could be done by scenario developers.

After a bit more testing I will agree on this.

As an allied player the saved games I tried to load were on the Allied orders phase. I ran a turn so I could access the Japanese task forces and the error changed to Task Force destination base (SYS_FK_137). I also got errors on LCU prep destination on other test runs. Perhaps the AI is having issues with the changes to the bases in CHS. (Further info: I was running all areas under computer control).

Woos, maybe this is just something to be aware of if players use computer control.

For your info, check out TF62 below and in the next post. The TF destination is open ocean but Palau is displayed as destination in the TF list. Maybe this is what is upsetting the load process.

Attachment (1)

< Message edited by Reg -- 11/19/2006 4:29:42 AM >

_____________________________

Cheers,
Reg.

(One day I will learn to spell - or check before posting....)
Uh oh, Firefox has a spell checker!! What excuse can I use now!!!

(in reply to Helpless)

Post #: 91

RE: New tool: WitpDecoder; No more spreadsheets! - 11/19/2006 4:26:46 AM

Woos

Posts: 683
Joined: 6/5/2005
From: Germany
Status: offline

quote:

ORIGINAL: Reg

Couldn't write things to the DB due to java.sql.SQLException: Integrity constraint
violation - no parent SYS_FK_136 table: BaseStatus
<<list of java error references>>

==== Extracts from witp.script (which creates WitpDecoder database) ====

CREATE CACHED TABLE "BaseStatus"("baseID" SMALLINT DEFAULT 0 NOT NULL PRIMARY KEY, ...

CREATE CACHED TABLE "TFStatus"(... ,"homeHarborID" SMALLINT DEFAULT 0 NOT NULL, ... ,CONSTRAINT SYS_FK_136 FOREIGN KEY("homeHarborID") REFERENCES "BaseStatus"("baseID") ON DELETE CASCADE,

==== Conclusion ====

My save game file contains a Task Force with a non-existent home base!! (plus the usual leader bugs)

The conclusion was correct if ..... WitP had not the interesting idea of saving Dot bases in several different ways (as I just found out; I'm beginning to wonder if the Matrixgames guys still understand their save game format). And the version you have only recognizes two of those ways and drops bases saved in the third one. So you might want to look for a TF with a dot base as home base (or send me your save file + .csv files).

Saving error reports into a file is something I will probably do in a future version (I want to get rid of the console).

(in reply to Reg)

Post #: 92

RE: New tool: WitpDecoder; No more spreadsheets! - 11/19/2006 4:28:39 AM

Reg

Posts: 2787
Joined: 5/26/2000
From: NSW, Australia
Status: offline

Note TF62 as above.

Attachment (1)

_____________________________

Cheers,
Reg.

(One day I will learn to spell - or check before posting....)
Uh oh, Firefox has a spell checker!! What excuse can I use now!!!

(in reply to Reg)

Post #: 93

RE: New tool: WitpDecoder; No more spreadsheets! - 11/19/2006 4:34:19 AM

scout1

Posts: 2899
Joined: 8/24/2004
From: South Bend, In
Status: offline

Got one of my scenario's to work (in some manner), and all I can say is

OH MY GOD, this tool is truely KICK ASS. I think I died and went to heaven ....

(in reply to Helpless)

Post #: 94

RE: New tool: WitpDecoder; No more spreadsheets! - 11/19/2006 5:00:16 AM

scout1

Posts: 2899
Joined: 8/24/2004
From: South Bend, In
Status: offline

If the boys haven't used this tool (otr something like it), it's a crime. Just ran it on one of my games with GoodBoyladdie and found that I have

12 occurences where 1 leader commands more than one unit
15 occurences where I have no leader or a foreign one

I'm really looking forward to the potential of this toolset ... God, I might even get to think strategically now, instead of just as the supply sargenr who really doesn't have a good picture of what is really going on .....

(in reply to scout1)

Post #: 95

RE: New tool: WitpDecoder; No more spreadsheets! - 11/19/2006 11:00:54 PM

Woos

Posts: 683
Joined: 6/5/2005
From: Germany
Status: offline

V0.2 is available as zip-File from the old location http://extweb.retsiemuab.de/witp/witpdecoder.zip

Should be better in handling dot bases. Has a useful documentation PDF file and finally the industry Tab is working so knowing where to send the Oil and Res transports is easier (the last spreadsheet I wanted to replace). See below

Attachment (1)

< Message edited by Woos -- 11/19/2006 11:05:20 PM >

(in reply to scout1)

Post #: 96

RE: New tool: WitpDecoder; No more spreadsheets! - 11/19/2006 11:17:02 PM

Helpless

Posts: 15793
Joined: 8/27/2004
Status: offline

Spoilage stats kicks ass..

_____________________________

Pavel Zagzin
WITE/WITW/WITE-2 Development

(in reply to Woos)

Post #: 97

RE: New tool: WitpDecoder; No more spreadsheets! - 11/20/2006 12:34:24 AM

BaitBoy

Posts: 227
Joined: 8/6/2004
Status: offline

_____________________________

"You go over there and attract their attention while I . . . "

Member Henchmen and Sidekicks Local 272

(in reply to Helpless)

Post #: 98

RE: New tool: WitpDecoder; No more spreadsheets! - 11/20/2006 1:43:37 AM

jrcar

Posts: 3613
Joined: 4/19/2002
From: Seymour, Australia
Status: offline

Amazing, thanks you. Looking froward to using it. The Poilage stats will be very handy.

Cheers

Rob

(in reply to BaitBoy)

Post #: 99

RE: New tool: WitpDecoder; No more spreadsheets! - 11/20/2006 1:50:08 AM

scout1

Posts: 2899
Joined: 8/24/2004
From: South Bend, In
Status: offline

Looking forward to this update ....

Now, just so that I'm clear. If I have 4 games, slots 005, 006, 007 and 010, all of which are the same exact scenario, I still need an independent subdirectory for each fully populated to run the info on each. True ? Not complaining, just want to make sure I follow directions ....

And since I haven't gotten that far yet, is the info provided available to be printed for offline evaluation ? IE is it limited to screen viewing only or are there options ?

Grand looking tool thus far AND I haven't found a way to break it yet ..

(in reply to Woos)

Post #: 100

RE: New tool: WitpDecoder; No more spreadsheets! - 11/20/2006 2:27:49 AM

Woos

Posts: 683
Joined: 6/5/2005
From: Germany
Status: offline

quote:

ORIGINAL: scout1
If I have 4 games, slots 005, 006, 007 and 010, all of which are the same exact scenario, I still need an independent subdirectory for each fully populated to run the info on each. True ? Not complaining, just want to make sure I follow directions ....

Why do you continue to ask questions which I have answered before, here in the forum and in the tool documentation (which is now even seperately downloadable) ....

quote:

ORIGINAL: scout1
And since I haven't gotten that far yet, is the info provided available to be printed for offline evaluation ? IE is it limited to screen viewing only or are there options ?

.... and others which can be easily found out by just using the tool? Answering these questions costs time, which I can better use for improving the tool or using and thereby testing it.

(in reply to scout1)

Post #: 101

RE: New tool: WitpDecoder; No more spreadsheets! - 11/20/2006 12:33:06 PM

TheDudelyLlama

Posts: 14
Joined: 11/8/2006
From: Directly under the earths sun..... now!!
Status: offline

From witpDecoder.log

Couldn't write things to the DB due to
java.sql.SQLException: Unique constraint violation: SYS_CT_71
	at org.hsqldb.jdbc.Util.sqlException(Unknown Source)
	at org.hsqldb.jdbc.jdbcStatement.fetchResult(Unknown Source)
	at org.hsqldb.jdbc.jdbcStatement.executeUpdate(Unknown Source)
	at de.retsiemuab.witpDecoder.ai.a(Unknown Source)
	at de.retsiemuab.witpDecoder.ag.a(Unknown Source)
	at de.retsiemuab.witpDecoder.b.a(Unknown Source)
	at de.retsiemuab.witpDecoder.as.run(Unknown Source)
	at org.eclipse.swt.custom.BusyIndicator.showWhile(BusyIndicator.java:67)
	at de.retsiemuab.witpDecoder.B.a(Unknown Source)
	at de.retsiemuab.witpDecoder.h.widgetSelected(Unknown Source)
	at org.eclipse.swt.widgets.TypedListener.handleEvent(TypedListener.java:90)
	at org.eclipse.swt.widgets.EventTable.sendEvent(EventTable.java:66)
	at org.eclipse.swt.widgets.Widget.sendEvent(Widget.java:928)
	at org.eclipse.swt.widgets.Display.runDeferredEvents(Display.java:3348)
	at org.eclipse.swt.widgets.Display.readAndDispatch(Display.java:2968)
	at de.retsiemuab.witpDecoder.Y.a(Unknown Source)
	at de.retsiemuab.witpDecoder.Main.a(Unknown Source)
	at de.retsiemuab.witpDecoder.Main.main(Unknown Source)

(in reply to Woos)

Post #: 102

RE: New tool: WitpDecoder; No more spreadsheets! - 11/20/2006 8:05:38 PM

scout1

Posts: 2899
Joined: 8/24/2004
From: South Bend, In
Status: offline

Sorry, but couldn't load it from the machine I was on. Will dive in and "attempt" to keep my questions more focused.....

(in reply to Woos)

Post #: 103

RE: New tool: WitpDecoder; No more spreadsheets! - 11/20/2006 8:40:43 PM

Mynok

Posts: 12108
Joined: 11/30/2002
Status: offline

Excuse me while I <DROOL> over that industry screen..........

(in reply to scout1)

Post #: 104

RE: New tool: WitpDecoder; No more spreadsheets! - 11/20/2006 8:43:11 PM

Mike Solli

Posts: 15792
Joined: 10/18/2000
From: the flight deck of the Zuikaku
Status: offline

OMG, what am I going to do with all my spreadsheets??!! I might just have enough time to start another PBEM.

(in reply to scout1)

Post #: 105

RE: New tool: WitpDecoder; No more spreadsheets! - 11/20/2006 8:50:26 PM

Mynok

Posts: 12108
Joined: 11/30/2002
Status: offline

quote:

ORIGINAL: Mike Solli

OMG, what am I going to do with all my spreadsheets??!! I might just have enough time to start another PBEM.

Better find a good lawyer, Woos! I predict there is a future in divorce court as an "expert witness" for you.

< Message edited by Mynok -- 11/20/2006 8:53:24 PM >

(in reply to Mike Solli)

Post #: 106

RE: New tool: WitpDecoder; No more spreadsheets! - 11/20/2006 9:07:07 PM

Woos

Posts: 683
Joined: 6/5/2005
From: Germany
Status: offline

quote:

ORIGINAL: Mynok
Excuse me while I <DROOL> over that industry screen..........

Be aware that it has a few problems.
A) As stated in the doc you have to create a list of bases per cluster (Japan, Taiwan, ..) by hand for each new map (and possibly per scenario if that changes bases).
B) It has no idea if all the bases in the cluster have an uninterrupted connection and can thus exchange their resources and oil. E.g. if you would attack the Soviets and take Okha in the first turn, its oil storage and production would be counted in the "East China, Korea, Russia" cluster even though its oil could not be used automatically by any factory in that cluster until the Russian lands opposite of it are taken also (on Andrew Brown's map their is an oil pipeline). So use with care!

(in reply to Mynok)

Post #: 107

RE: New tool: WitpDecoder; No more spreadsheets! - 11/20/2006 9:29:45 PM

Mynok

Posts: 12108
Joined: 11/30/2002
Status: offline

Knowing what bases are connected is the easy part. Adding up all the industry, resources, oil and making calculating what the needs are.....that's the time consuming drudgery.

(in reply to Woos)

Post #: 108

RE: New tool: WitpDecoder; No more spreadsheets! - 11/20/2006 11:20:17 PM

mc3744

Posts: 1957
Joined: 3/9/2004
From: Italy
Status: offline

WOW!!

I just found this!!

Amazing!!

Thanks Woos,

I'm going to try it out. It looks just great!

_____________________________

Nec recisa recedit

(in reply to Mynok)

Post #: 109

RE: New tool: WitpDecoder; No more spreadsheets! - 11/21/2006 11:12:52 AM

Arkady

Posts: 1262
Joined: 5/31/2002
From: 27th Penal Battalion
Status: offline

So we can assume that security of PBEM is breached...

Great tool anyway...
I would prefer those functions included in game though

_____________________________

(in reply to mc3744)

Post #: 110

RE: New tool: WitpDecoder; No more spreadsheets! - 11/21/2006 12:18:39 PM

Sneer

Posts: 2654
Joined: 10/29/2003
Status: offline

i can't download new .jar file
object not found error

_____________________________

(in reply to Arkady)

Post #: 111

Why using a DB with integrity constraints is usefull - 11/21/2006 9:17:31 PM

Woos

Posts: 683
Joined: 6/5/2005
From: Germany
Status: offline

Now that I learned about the next complication in the save file format for bases it seems about time to explain why using a DB with integrity constraints is a good thing and dumping data into a file in various ways as WitP does is bad. Maybe some people at Matrixgames will learn.

WitP save files contain information about bases. Even information about bases not used. Normally a base is not used if its type is set to 0. That is except for dot bases, which use a secondary type indicator, which, if set to "base" even if the primary one is 0, indicates a dot base. That is unless it is a special kind of dot base which even has the secondary indicator set to 0, too. I don't know what is special about them, so the only way to distinguish them from non-existant bases is to check whether they have a position defined. So, to get you all back on track, it is a base
a) if its type is set or
b) if the type is 0 but the secondary type is set or
c) if both types are 0 but the position is set.

Can you still follow? Well, now the interesting question, when is an entry in the save file not a base? You think you know it? Well, forget it. In addition to everything you might think because of the above statements a base is also not a base if
a) its type is set but
b) its position set to -1,-1.
Completely logical, isn't it? I mean just setting the type to 0 wouldn't have been an alternative.

Not imagineing such a complication is the reason for the error "Some Dude" reported above and will currently prevent the use of witpDecoder on any restricted scenario (e.g. the Coral sea scenario). * Edit * Coral Sea scenario will never be supported by witpDecoder. After solving the above problems it turned out that also the Truk base is used as Headquarter for the "Truk Base Force". Now that is an interesting hack given the WitP save file format but can not be supported in witpDecoder which - for good reasons - dinstinguishes between bases and LCUs.

Now, how did I find it (and why are integrity constraints so great)? Well integrity constraints let you formulate your understanding of the data you intend to store. witpDecoder for example tells the DB that it assumes that all bases will have different positions (quite obvious if you look at the map). Storing the second base with a -1,-1 position into it the DB complains with the exception seen in Some Dude's posting. Result: Report is investigated and error found and fixed instead of lots of headaches later on why other functions don't work as expected.
Using an DB and formulating into it that the leader part of the ship-leader relationship needs to be unique would have found the leader bug long ago. Now it is still in and I can't put that constraint into the witpDecoder DB unless I want to prevent lots of people from using the tool.

OK enough ranting. Answers to the questions:

quote:

ORIGINAL:Arkady
So we can assume that security of PBEM is breached.

Please read the section on "PBEM security" in the documentation (available seperately via a link in the first post).

quote:

ORIGINAL:Sneer
i can't download new .jar file
object not found error

witpDecoder2.jar was an intermediate version between 0.1 and 0.2. Now that 0.2 is out I removed it since you don't need it anymore (and it would break things).

< Message edited by Woos -- 11/21/2006 11:43:18 PM >

(in reply to Sneer)

Post #: 112

RE: Why using a DB with integrity constraints is usefull - 11/21/2006 11:16:57 PM

Mynok

Posts: 12108
Joined: 11/30/2002
Status: offline

quote:

Now that I learned about the next complication in the save file format for bases it seems about time to explain why using a DB with integrity constraints is a good thing and dumping data into a file in various ways as WitP does is bad. Maybe some people at Matrixgames will learn.

A-freaking-men!!!!

With all the *free* database engines out there now with full referential integrity functionality, there is zero excuse to not use one for a game of this scope. ZERO. Oh....and they were available when UV was being designed too.

One of my favorite slides ever is from Tom Kyte, an Oracle developer. It is part of his worst practices seminar and it reads:

Probably you should reinvent as many database features as possible

(in reply to Woos)

Post #: 113

RE: New tool: WitpDecoder; No more spreadsheets! - 11/22/2006 2:30:08 AM

jcjordan

Posts: 1900
Joined: 6/27/2001
Status: offline

Excellent tool & great work Woos, can't wait for the Allied side to come out even though it won't help much in the industry side for most but in my custom CHS I've made some a/c factories for certain a/c instead of a flat rate so interested in seeing how they come out.

(in reply to Reg)

Post #: 114

RE: Why using a DB with integrity constraints is usefull - 11/22/2006 4:01:40 AM

wdolson

Posts: 10398
Joined: 6/28/2006
From: Near Portland, OR
Status: offline

quote:

ORIGINAL: Mynok
A-freaking-men!!!! With all the *free* database engines out there now with full referential integrity functionality, there is zero excuse to not use one for a game of this scope. ZERO. Oh....and they were available when UV was being designed too.

WitP suffers from some of the same problems many major programs like MS Word suffer from, their roots are very long. UV added a graphical interface and some improvements to the old DOS game Pacific War. I would not be surprised at all if the databases in WitP were essentially the same format as the original game. Some fields may have been added over time, but I would bet it's essentially the same.

Back when Pacific War was developed, it was rare to have a game with a database big enough to have to worry about things like referrential integrity. The programmers of those early games probably had little or no experience with any kind of database.

I wouldn't be surprised if the data is the way it is simply because it's based on a very old game engine.

Bill

(in reply to Mynok)

Post #: 115

RE: Why using a DB with integrity constraints is usefull - 11/22/2006 5:55:24 PM

witpqs

Posts: 26087
Joined: 10/4/2004
From: Argleton
Status: offline

I think you nailed it, wdolsen. We are dealing with software paleontology.

(in reply to wdolson)

Post #: 116

RE: Why using a DB with integrity constraints is usefull - 11/23/2006 6:28:07 AM

Knavey

Posts: 3052
Joined: 9/12/2002
From: Valrico, Florida
Status: offline

Watching this thread with interest. I want to see how it turns out when you finally get the Allies up and running. Looks very promising so far.

_____________________________

x-Nuc twidget
CVN-71
USN 87-93
"Going slow in the fast direction"

(in reply to witpqs)

Post #: 117

RE: Why using a DB with integrity constraints is usefull - 11/23/2006 7:45:20 PM

saj42

Posts: 1125
Joined: 4/19/2005
From: Somerset, England
Status: offline

Just installed the tool - BRILLIANT

Now all I must do is remember to upload the savegames before they get overwritten

_____________________________

Banner by rogueusmc

(in reply to Knavey)

Post #: 118

RE: Why using a DB with integrity constraints is usefull - 11/23/2006 7:58:30 PM

Dino

Posts: 1032
Joined: 11/14/2005
From: Serbia
Status: offline

I beleive you only need the latest savegame, anyway.

(in reply to saj42)

Post #: 119

RE: Why using a DB with integrity constraints is usefull - 11/23/2006 9:53:06 PM

Woos

Posts: 683
Joined: 6/5/2005
From: Germany
Status: offline

quote:

ORIGINAL: Knavey
Watching this thread with interest. I want to see how it turns out when you finally get the Allies up and running. Looks very promising so far.

Currently I'm adding support for "automatically appearing" resources at bases (like e.g. United States and lots of Chinese bases). The Japanese don't have that so their is no support in V0.2, the Allies rely heavily on that so not including it would give wrong values on the Storage and Industry tabs.

Then one of the next thing will be to include a "Nationality" object, which will abstract the now hard-coded knowledge of how to get only certain things into/out of the database. While I'm at it I'll probably also add "IJN","IJA" as well as "Chinese", "Russian", "CW", ... as different nationalities to support people playing m vs n games to only get vague information about the other "friendly" nationalities. Once that is done, some GUI changes are needed, the up-front DB initialization needs to be shifted around and then its ready.
I think I predicted 2 weeks of Japanese-only happyness about half a week ago. That's still about right. I also want to play the game after all.

And Dino is correct, currently most of the history functions are not implemented, so loading lots of old savefile yields no advantage (after the first 5 the "5dayAvgGain" column in the storage tab should fill). Additionally the next version will change the database format (so you will have to read in all old savegames again). And before at least V1.0 is reached the database format will continue to change from time to time. So currently no need to have a lot of save games read into the tool as long as you have the latest 5.

< Message edited by Woos -- 11/23/2006 9:57:38 PM >

(in reply to Knavey)

Post #: 120

Page: << < prev 2 3 [4] 5 6 next > >>

All Forums >> [Current Games From Matrix.] >> [World War II] >> War In The Pacific - Struggle Against Japan 1941 - 1945 >> RE: New tool: WitpDecoder; No more spreadsheets!