RE: New tool: WitpDecoder; No more spreadsheets! (Full Version)

All Forums >> [Current Games From Matrix.] >> [World War II] >> War In The Pacific - Struggle Against Japan 1941 - 1945



Message


Reg -> RE: New tool: WitpDecoder; No more spreadsheets! (11/19/2006 4:23:18 AM)

quote:

ORIGINAL: Helpless

I have no problem to run witpdecoder on CHS. If later on it got screwed by the game code itself , nothing could be done by scenario developers.


After a bit more testing I will agree on this.

As an allied player the saved games I tried to load were on the Allied orders phase. I ran a turn so I could access the Japanese task forces and the error changed to Task Force destination base (SYS_FK_137). I also got errors on LCU prep destination on other test runs. Perhaps the AI is having issues with the changes to the bases in CHS. (Further info: I was running all areas under computer control).

Woos, maybe this is just something to be aware of if players use computer control.

For your info, check out TF62 below and in the next post. The TF destination is open ocean but Palau is displayed as destination in the TF list. Maybe this is what is upsetting the load process.

[image]local://upfiles/446/D1D2B86BC90741C29A537B308408C6CE.gif[/image]




Woos -> RE: New tool: WitpDecoder; No more spreadsheets! (11/19/2006 4:26:46 AM)

quote:

ORIGINAL: Reg

Couldn't write things to the DB due to java.sql.SQLException: Integrity constraint
violation - no parent SYS_FK_136 table: BaseStatus
<<list of java error references>>

==== Extracts from witp.script (which creates WitpDecoder database) ====

CREATE CACHED TABLE "BaseStatus"("baseID" SMALLINT DEFAULT 0 NOT NULL PRIMARY KEY, ...

CREATE CACHED TABLE "TFStatus"(... ,"homeHarborID" SMALLINT DEFAULT 0 NOT NULL, ... ,CONSTRAINT SYS_FK_136 FOREIGN KEY("homeHarborID") REFERENCES "BaseStatus"("baseID") ON DELETE CASCADE,

==== Conclusion ====

My save game file contains a Task Force with a non-existent home base!! (plus the usual leader bugs)

The conclusion was correct if ..... WitP had not the interesting idea of saving Dot bases in several different ways (as I just found out; I'm beginning to wonder if the Matrixgames guys still understand their save game format). And the version you have only recognizes two of those ways and drops bases saved in the third one. So you might want to look for a TF with a dot base as home base (or send me your save file + .csv files).

Saving error reports into a file is something I will probably do in a future version (I want to get rid of the console).




Reg -> RE: New tool: WitpDecoder; No more spreadsheets! (11/19/2006 4:28:39 AM)

Note TF62 as above.

[image]local://upfiles/446/FF83843DCDE54B179F1EA05B96CEAD2C.gif[/image]




scout1 -> RE: New tool: WitpDecoder; No more spreadsheets! (11/19/2006 4:34:19 AM)

Got one of my scenario's to work (in some manner), and all I can say is

OH MY GOD, this tool is truely KICK ASS. I think I died and went to heaven ....




scout1 -> RE: New tool: WitpDecoder; No more spreadsheets! (11/19/2006 5:00:16 AM)

If the boys haven't used this tool (otr something like it), it's a crime. Just ran it on one of my games with GoodBoyladdie and found that I have

12 occurences where 1 leader commands more than one unit
15 occurences where I have no leader or a foreign one

I'm really looking forward to the potential of this toolset ... God, I might even get to think strategically now, instead of just as the supply sargenr who really doesn't have a good picture of what is really going on .....




Woos -> RE: New tool: WitpDecoder; No more spreadsheets! (11/19/2006 11:00:54 PM)

V0.2 is available as zip-File from the old location http://extweb.retsiemuab.de/witp/witpdecoder.zip

Should be better in handling dot bases. Has a useful documentation PDF file and finally the industry Tab is working so knowing where to send the Oil and Res transports is easier (the last spreadsheet I wanted to replace). See below

[image]local://upfiles/16906/F4C0D3D014CA4CF38BF954D44725C5F3.jpg[/image]




Helpless -> RE: New tool: WitpDecoder; No more spreadsheets! (11/19/2006 11:17:02 PM)

Spoilage stats kicks ass.. [&o] [:)]




BaitBoy -> RE: New tool: WitpDecoder; No more spreadsheets! (11/20/2006 12:34:24 AM)

[&o][&o][&o]




jrcar -> RE: New tool: WitpDecoder; No more spreadsheets! (11/20/2006 1:43:37 AM)

Amazing, thanks you. Looking froward to using it. The Poilage stats will be very handy.

Cheers

Rob




scout1 -> RE: New tool: WitpDecoder; No more spreadsheets! (11/20/2006 1:50:08 AM)

Looking forward to this update ....

Now, just so that I'm clear. If I have 4 games, slots 005, 006, 007 and 010, all of which are the same exact scenario, I still need an independent subdirectory for each fully populated to run the info on each. True ? Not complaining, just want to make sure I follow directions ....

And since I haven't gotten that far yet, is the info provided available to be printed for offline evaluation ? IE is it limited to screen viewing only or are there options ?

Grand looking tool thus far AND I haven't found a way to break it yet ..[;)]




Woos -> RE: New tool: WitpDecoder; No more spreadsheets! (11/20/2006 2:27:49 AM)

quote:

ORIGINAL: scout1
If I have 4 games, slots 005, 006, 007 and 010, all of which are the same exact scenario, I still need an independent subdirectory for each fully populated to run the info on each. True ? Not complaining, just want to make sure I follow directions ....

Why do you continue to ask questions which I have answered before, here in the forum and in the tool documentation (which is now even seperately downloadable) ....

quote:

ORIGINAL: scout1
And since I haven't gotten that far yet, is the info provided available to be printed for offline evaluation ? IE is it limited to screen viewing only or are there options ?

.... and others which can be easily found out by just using the tool? Answering these questions costs time, which I can better use for improving the tool or using and thereby testing it.




TheDudelyLlama -> RE: New tool: WitpDecoder; No more spreadsheets! (11/20/2006 12:33:06 PM)

From witpDecoder.log

Couldn't write things to the DB due to
java.sql.SQLException: Unique constraint violation: SYS_CT_71
	at org.hsqldb.jdbc.Util.sqlException(Unknown Source)
	at org.hsqldb.jdbc.jdbcStatement.fetchResult(Unknown Source)
	at org.hsqldb.jdbc.jdbcStatement.executeUpdate(Unknown Source)
	at de.retsiemuab.witpDecoder.ai.a(Unknown Source)
	at de.retsiemuab.witpDecoder.ag.a(Unknown Source)
	at de.retsiemuab.witpDecoder.b.a(Unknown Source)
	at de.retsiemuab.witpDecoder.as.run(Unknown Source)
	at org.eclipse.swt.custom.BusyIndicator.showWhile(BusyIndicator.java:67)
	at de.retsiemuab.witpDecoder.B.a(Unknown Source)
	at de.retsiemuab.witpDecoder.h.widgetSelected(Unknown Source)
	at org.eclipse.swt.widgets.TypedListener.handleEvent(TypedListener.java:90)
	at org.eclipse.swt.widgets.EventTable.sendEvent(EventTable.java:66)
	at org.eclipse.swt.widgets.Widget.sendEvent(Widget.java:928)
	at org.eclipse.swt.widgets.Display.runDeferredEvents(Display.java:3348)
	at org.eclipse.swt.widgets.Display.readAndDispatch(Display.java:2968)
	at de.retsiemuab.witpDecoder.Y.a(Unknown Source)
	at de.retsiemuab.witpDecoder.Main.a(Unknown Source)
	at de.retsiemuab.witpDecoder.Main.main(Unknown Source)




scout1 -> RE: New tool: WitpDecoder; No more spreadsheets! (11/20/2006 8:05:38 PM)

Sorry, but couldn't load it from the machine I was on. Will dive in and "attempt" to keep my questions more focused.....




Mynok -> RE: New tool: WitpDecoder; No more spreadsheets! (11/20/2006 8:40:43 PM)


[X(] [X(] [X(] [X(] [&o] [&o] [&o] [&o] [&o] [&o] [&o]

Excuse me while I <DROOL> over that industry screen..........





Mike Solli -> RE: New tool: WitpDecoder; No more spreadsheets! (11/20/2006 8:43:11 PM)

OMG, what am I going to do with all my spreadsheets??!! I might just have enough time to start another PBEM. [:D]




Mynok -> RE: New tool: WitpDecoder; No more spreadsheets! (11/20/2006 8:50:26 PM)

quote:

ORIGINAL: Mike Solli

OMG, what am I going to do with all my spreadsheets??!! I might just have enough time to start another PBEM. [:D]


[:D] [:D] [:D] Better find a good lawyer, Woos! I predict there is a future in divorce court as an "expert witness" for you. [:D] [:D] [:D]




Woos -> RE: New tool: WitpDecoder; No more spreadsheets! (11/20/2006 9:07:07 PM)

quote:

ORIGINAL: Mynok
Excuse me while I <DROOL> over that industry screen..........

Be aware that it has a few problems.
A) As stated in the doc you have to create a list of bases per cluster (Japan, Taiwan, ..) by hand for each new map (and possibly per scenario if that changes bases).
B) It has no idea if all the bases in the cluster have an uninterrupted connection and can thus exchange their resources and oil. E.g. if you would attack the Soviets and take Okha in the first turn, its oil storage and production would be counted in the "East China, Korea, Russia" cluster even though its oil could not be used automatically by any factory in that cluster until the Russian lands opposite of it are taken also (on Andrew Brown's map their is an oil pipeline). So use with care!




Mynok -> RE: New tool: WitpDecoder; No more spreadsheets! (11/20/2006 9:29:45 PM)


Knowing what bases are connected is the easy part. Adding up all the industry, resources, oil and making calculating what the needs are.....that's the time consuming drudgery.





mc3744 -> RE: New tool: WitpDecoder; No more spreadsheets! (11/20/2006 11:20:17 PM)

WOW!! [X(][X(][X(]

I just found this!!

Amazing!! [8D]

Thanks Woos, [&o] I'm going to try it out. It looks just great!




Arkady -> RE: New tool: WitpDecoder; No more spreadsheets! (11/21/2006 11:12:52 AM)

So we can assume that security of PBEM is breached...[&:]

Great tool anyway...
I would prefer those functions included in game though




Sneer -> RE: New tool: WitpDecoder; No more spreadsheets! (11/21/2006 12:18:39 PM)

i can't download new .jar file
object not found error




Woos -> Why using a DB with integrity constraints is usefull (11/21/2006 9:17:31 PM)

Now that I learned about the next complication in the save file format for bases it seems about time to explain why using a DB with integrity constraints is a good thing and dumping data into a file in various ways as WitP does is bad. Maybe some people at Matrixgames will learn.

WitP save files contain information about bases. Even information about bases not used. Normally a base is not used if its type is set to 0. That is except for dot bases, which use a secondary type indicator, which, if set to "base" even if the primary one is 0, indicates a dot base. That is unless it is a special kind of dot base which even has the secondary indicator set to 0, too. I don't know what is special about them, so the only way to distinguish them from non-existant bases is to check whether they have a position defined. So, to get you all back on track, it is a base
a) if its type is set or
b) if the type is 0 but the secondary type is set or
c) if both types are 0 but the position is set.

Can you still follow? Well, now the interesting question, when is an entry in the save file not a base? You think you know it? Well, forget it. In addition to everything you might think because of the above statements a base is also not a base if
a) its type is set but
b) its position set to -1,-1.
Completely logical, isn't it? I mean just setting the type to 0 wouldn't have been an alternative.[:@]

Not imagineing such a complication is the reason for the error "Some Dude" reported above and will currently prevent the use of witpDecoder on any restricted scenario (e.g. the Coral sea scenario). * Edit * Coral Sea scenario will never be supported by witpDecoder. After solving the above problems it turned out that also the Truk base is used as Headquarter for the "Truk Base Force". Now that is an interesting hack given the WitP save file format but can not be supported in witpDecoder which - for good reasons - dinstinguishes between bases and LCUs.

Now, how did I find it (and why are integrity constraints so great)? Well integrity constraints let you formulate your understanding of the data you intend to store. witpDecoder for example tells the DB that it assumes that all bases will have different positions (quite obvious if you look at the map). Storing the second base with a -1,-1 position into it the DB complains with the exception seen in Some Dude's posting. Result: Report is investigated and error found and fixed instead of lots of headaches later on why other functions don't work as expected.
Using an DB and formulating into it that the leader part of the ship-leader relationship needs to be unique would have found the leader bug long ago. Now it is still in and I can't put that constraint into the witpDecoder DB unless I want to prevent lots of people from using the tool.

OK enough ranting. Answers to the questions:
quote:

ORIGINAL:Arkady
So we can assume that security of PBEM is breached.

Please read the section on "PBEM security" in the documentation (available seperately via a link in the first post).

quote:

ORIGINAL:Sneer
i can't download new .jar file
object not found error

witpDecoder2.jar was an intermediate version between 0.1 and 0.2. Now that 0.2 is out I removed it since you don't need it anymore (and it would break things).




Mynok -> RE: Why using a DB with integrity constraints is usefull (11/21/2006 11:16:57 PM)


quote:

Now that I learned about the next complication in the save file format for bases it seems about time to explain why using a DB with integrity constraints is a good thing and dumping data into a file in various ways as WitP does is bad. Maybe some people at Matrixgames will learn.


A-freaking-men!!!! [&o] With all the *free* database engines out there now with full referential integrity functionality, there is zero excuse to not use one for a game of this scope. ZERO. Oh....and they were available when UV was being designed too.

One of my favorite slides ever is from Tom Kyte, an Oracle developer. It is part of his worst practices seminar and it reads:

Probably you should reinvent as many database features as possible

[:D]





jcjordan -> RE: New tool: WitpDecoder; No more spreadsheets! (11/22/2006 2:30:08 AM)

Excellent tool & great work Woos, can't wait for the Allied side to come out even though it won't help much in the industry side for most but in my custom CHS I've made some a/c factories for certain a/c instead of a flat rate so interested in seeing how they come out.




wdolson -> RE: Why using a DB with integrity constraints is usefull (11/22/2006 4:01:40 AM)

quote:

ORIGINAL: Mynok
A-freaking-men!!!! [&o] With all the *free* database engines out there now with full referential integrity functionality, there is zero excuse to not use one for a game of this scope. ZERO. Oh....and they were available when UV was being designed too.


WitP suffers from some of the same problems many major programs like MS Word suffer from, their roots are very long. UV added a graphical interface and some improvements to the old DOS game Pacific War. I would not be surprised at all if the databases in WitP were essentially the same format as the original game. Some fields may have been added over time, but I would bet it's essentially the same.

Back when Pacific War was developed, it was rare to have a game with a database big enough to have to worry about things like referrential integrity. The programmers of those early games probably had little or no experience with any kind of database.

I wouldn't be surprised if the data is the way it is simply because it's based on a very old game engine.

Bill




witpqs -> RE: Why using a DB with integrity constraints is usefull (11/22/2006 5:55:24 PM)

I think you nailed it, wdolsen. We are dealing with software paleontology.




Knavey -> RE: Why using a DB with integrity constraints is usefull (11/23/2006 6:28:07 AM)

Watching this thread with interest.  I want to see how it turns out when you finally get the Allies up and running.  Looks very promising so far.




saj42 -> RE: Why using a DB with integrity constraints is usefull (11/23/2006 7:45:20 PM)

Just installed the tool - BRILLIANT [&o]
Now all I must do is remember to upload the savegames before they get overwritten




Dino -> RE: Why using a DB with integrity constraints is usefull (11/23/2006 7:58:30 PM)

I beleive you only need the latest savegame, anyway.




Woos -> RE: Why using a DB with integrity constraints is usefull (11/23/2006 9:53:06 PM)

quote:

ORIGINAL: Knavey
Watching this thread with interest. I want to see how it turns out when you finally get the Allies up and running. Looks very promising so far.

Currently I'm adding support for "automatically appearing" resources at bases (like e.g. United States and lots of Chinese bases). The Japanese don't have that so their is no support in V0.2, the Allies rely heavily on that so not including it would give wrong values on the Storage and Industry tabs.

Then one of the next thing will be to include a "Nationality" object, which will abstract the now hard-coded knowledge of how to get only certain things into/out of the database. While I'm at it I'll probably also add "IJN","IJA" as well as "Chinese", "Russian", "CW", ... as different nationalities to support people playing m vs n games to only get vague information about the other "friendly" nationalities. Once that is done, some GUI changes are needed, the up-front DB initialization needs to be shifted around and then its ready.
I think I predicted 2 weeks of Japanese-only happyness about half a week ago. That's still about right. I also want to play the game after all.


And Dino is correct, currently most of the history functions are not implemented, so loading lots of old savefile yields no advantage (after the first 5 the "5dayAvgGain" column in the storage tab should fill). Additionally the next version will change the database format (so you will have to read in all old savegames again). And before at least V1.0 is reached the database format will continue to change from time to time. So currently no need to have a lot of save games read into the tool as long as you have the latest 5.




Page: <<   < prev  2 3 [4] 5 6   next >   >>

Valid CSS!




Forum Software © ASPPlayground.NET Advanced Edition 2.4.5 ANSI
3.796875