RE: AGElint: refined error messaging (Full Version)

All Forums >> [Current Games From Matrix.] >> [World War I] >> Revolution Under Siege Gold >> Mods and Scenarios



Message

berto -> RE: AGElint: refined error messaging (9/28/2012 10:21:52 AM)


The following problem is solved!

quote:

ORIGINAL: berto

Here are some ways we have refined the agelint error messaging to clarify, reduce confusion, and dispel the appearance of false positives.

One example:


Aliases/NATO Families Images pre Steam.ini, Fri Dec 10 2010 15:06:48

ERROR: in Aliases/NATO Families Images pre Steam.ini, at (or near) line 42: line-specific or other contextual syntax error (else file doesn't end with CR/LF): #s# (0x73), #symbol_admiral_2#

33 $Imgfam76 = symbol_navalengineer.png^M
34 $Imgfam91 = symbol_warship.png^M
35 $Imgfam92 = symbol_warship.png^M
36 $Imgfam93 = symbol_smallsteam.png^M
37 $Imgfam94 = symbol_privateer.png^M
38 $Imgfam95 = symbol_transport.png^M
39 $Imgfam96 = symbol_ironclad.png^M
40 $Imgfam97 = symbol_privateer.png^M
41 $Imgfam112 = symbol_balloons.png^M
42> $Imgfam198 = symbol_admiral_2.png


The problem? The ACW "Aliases/NATO Families Images pre Steam.ini" file fails to end "properly", with a proper CR/LF sequence. We can see this by cat'ing the file:


[root@berto agelint]# cat "/media/KINGSTON/Games/AGEOD/AGEod's American Civil War/ACW/Aliases/NATO Families Images pre Steam.ini"
// Pre-Steam

$Imgfam0 = symbol_general_2.png
$Imgfam11 = symbol_armyHQ.png
$Imgfam12 = symbol_hq1.png
$Imgfam21 = symbol_inf.png
...
$Imgfam96 = symbol_ironclad.png
$Imgfam97 = symbol_privateer.png
$Imgfam112 = symbol_balloons.png
$Imgfam198 = symbol_admiral_2.png[root@berto agelint]#


Do you see where the command prompt, '[root@berto agelint]#' immediately follows the 'symbol_admiral_2.png'?

This in itself presents no problem to the AGE engine. The data file will process just fine. But it creates insuperable difficulties for the agelint lexer! Since we really can't solve the lexer problem generally (without adding excessive code complications, or suppressing other possibly legitimate error messages), we mention parenthetically that possible explanation beyond the more generic "line-specific or other contextual syntax error." Note that, in the error output, you can infer the end-of-file, no CR/LF problem because you can see lines of context before the indicated error line but not afterwards.

The "line-specific or other contextual syntax error" -- that in itself is an improvement over the previous, simply stated "syntax error". In conjunction with the 'at (or near) line ...', it reminds you to look not just at the indicated line but also at the context. Again, the parsing technology is not always capable of pinpointing the exact point of syntax error, so you sometimes have to look around (usually before). Again, this is not a failure unique to agelint. The C and Perl (and other) compilers also share this limitation. (See earlier posts.)

...


So, for example, using the above example, we now have:


[root@berto agelint]# ./agelint +e -g acw "Aliases/NATO Families Images pre Steam.ini"
[nil]


That is, the above agelint command gives no output -- no error message(s) -- hence the file is correctly reported as being error-free.

How did I solve this very annoying false positive problem? Through trickery!

In agelint.c, when opening the lexer (txt.l) input stream, instead of 


                txtin = fopen(filecurr, "r");

we now have 


        static char catcmd[MAXPATH+12];

...

                /* a straightforward file read, as in
                txtin = fopen(filecurr, "r");
                   doesn't always work, because some files don't end with
                   CR/LF, and give the lexer fits (the lexer is line-oriented,
                   and assumes each line ends with CR/LF); so instead we
                   open a process that does a 'cat <file>; echo', where
                   the echo supplies a (sometimes needed, sometimes extra)
                   file-ending CR/LF, then read the process output -- thereby
                   tricking the lexer into "thinking" that the final line of
                   each file ends "properly"
                */
                sprintf(catcmd, "cat %c%s%c; echo", '\"', filecurr, '\"');
                txtin = popen(catcmd, "r");

So, this annoying category of false positive is gone!

A side benefit of this is that, in ERROR statements (all of them!), we can get rid of '(else file doesn't end with CR/LF)' as in


ERROR: in /media/KINGSTON/Games/AGEOD/Revolution under Siege FY/RUS/GameData/Abilities/70-Patriot.abi, at (or near) line 14: line-specific or other contextual syntax error (else file doesn't end with CR/LF): #$# (0x24), #$#


and instead have the shorter


ERROR: in /media/KINGSTON/Games/AGEOD/Revolution under Siege FY/RUS/GameData/Abilities/70-Patriot.abi, at (or near) line 14: line-specific or other contextual syntax error: #$# (0x24), #$#


Aside from code improvements like this one, also new additions (chkareas.pl, chkregions.pl), I am now in the process of adapting AGElint to the newly released Alea Jacta Est (AJE).

We might expect to see a new AGElint release in the not too distant future. [:)]



berto -> RE: AGElint: refined error messaging (10/5/2012 11:10:59 PM)


I have further refined the above technique. I can now deal more effectively than ever before with otherwise troublesome end-of-file situations. More accurate error reports, fewer false positives.

I am nearly finished with adapting AGElint to AJE. Are there data bugs in AJE? Yes, but not nearly as many as in AGEOD's previous two releases. When I have thoroughly vetted the error reports, and the AGElint code (at which point I release), I will be opening an AJE Data Bugs thread in the Matrix AJE subforum.

I'll be revisiting bug reports for RUS and most of the other recently updated AGEOD titles too.

But I'm in no special rush. Haste makes mistakes.



berto -> AGElint progress report (10/19/2012 10:50:41 PM)


Work on the next AGElint release continues.

For the first time, I am making a serious, concerted attempt to code and QA chkaliases.pl & chklocals.pl right, vetting the results as carefully as I now, standard practice, vet all chklint (agelint) results.

I am also, to the best of my ability and patience, attempting to fully adapt the entire AGElint toolkit to AJE and the latest additions and amendments to the AGE engine as documented at the AGE Wiki.

It is possible I might be releasing AGElint 1.2 by the end of next week. If not, not too long after that. Again, there is no special rush. I think it will be well worth the wait.



berto -> RE: AGElint progress report (10/26/2012 10:47:27 PM)


By now, I have thoroughly vetted chklint & chkaliases results across seven supported AGEOD games -- ACW, AJE, NCP, PON, ROP, RUS & WIA. I am taking extraordinary care this time to reduce false positives to a bare minimum. Still...

Are there bugs in each game? You betcha! Dozens and even hundreds of them in every game (including AGEOD's newest release, Alea Jacta Est), many of them indisputable and of greater or lesser seriousness.

(And are there bugs in AGElint? You betcha! But I try my darndest to find them and fix them! [;)])

Across all the games, I am observing in total easily well more than a thousand game data file bugs.

Besides undermining the perceived quality of the games, I strongly believe that not fixing the bugs (especially when provided with tools and techniques to discover them efficiently and easily -- if one could just get over the, IMO, minor Cygwin Windows install speed bump) is a colossal waste of player and developer time. I will write and post an essay about that sooner or later.

I will be releasing the next AGElint v1.2 soon, most likely by late next week.

But before that, I still need to vet results for chklocals, chkfiles, chkimages, chkareas, and others.

I will also likely set up shop at the WarGamer.com Forum, giving me an alternative venue for proselytizing for AGElint and hard-core QA.

Please stay tuned.



Chilperic -> RE: AGElint progress report (10/27/2012 11:27:05 PM)

As an user of agelint, I may just confirm this piece of software is a must for modding the AGE engine. [:)]



berto -> AGElint -- v1.2 official release (10/31/2012 9:39:14 AM)


Here is the fourth "official" AGElint version, 1.2 (20121031):

AGElint version 1.2 features adaptation to Alea Jacta Est, updates reflecting new AGE scripting commands, and significantly fewer (if any) false positives.

Here is a more detailed list of changes in AGElint 1.2:


Changes from AGElint_1.1.0 (20120117) to AGElint_1.2 (20121031):

  • Adapted all code and data to AGEOD's newest game, Alea Jacta Est.
  • Updated agelint to reflect new AGE scripting commands, and revised command syntaxes.
  • Fully reviewed, and thoroughly vetted, chklint.pl (agelint) results for: ACW, AJE, NCP, PON, ROP, RUS & WIA.
  • Generally solved, in agelint & the .pl scripts, problems where data files don't end with CR/LF (EOL, end-of-line).
  • Greatly reduced the number of false positives output by agelint and the various chk*.pl scripts.
  • Added automated setting of agelintroot & gameroot in all scripts, not just the chk*.pl ones (by, for example, universal reference to agelint.conf).
  • Squelched some problematic chk.pl cases (for several games).
  • Fixed an issue where dochk might not function for Windows users.
  • Bug fixes.
  • Other minor code & doc fixes/changes.


In previous AGElint releases, and for Cygwin Windows users only, dochk may not have worked for you (and if not, my apologies). It's due to the different ways Linux and Windows implement EOL (end-of-line). This has been fixed! (But note: If you attempt to edit dochk using NotePad or some other Windows editor, all dochk code lines will run together in one continuous stream. No, if you wish to edit dochk, you will need to do it, using vi or emacs or whatever, entirely within the Cygwin Windows environment. Linux users should have no difficulties whatsoever.)

I have painstakingly reviewed and vetted all chklint.pl (agelint) results, also results for chkaliases.pl, chklocals.pl, etc. There should be few if any false positives. Any remaining false positives would stem more from ambiguity in the AgeWiki docs, else honest differences of opinion. If you approach AGElint with an open mind, and generous spirit, you should find much value in it, and little to quibble over.

For this release, I had hoped to include two new data checking scripts: chkregions.pl, to error check the regions specifications; and chkareas.pl, to error check GameData/Areas.ini. These are still works-in-progress, Not Ready for Prime Time. Expect to see them in some future release.

I am quite happy with this release. Some problems remain, but they are hidden, behind the scenes.

The "hidden" problems? They include:

  • A loosely specified event syntax sequence. Due to some technical quirks with the bison (yacc) parser (the "lookahead" mechanism), I am yet unable to report erroneous action statements in Conditions, condition statements in Actions, and mistakenly omitted Conditions and Actions statements. This will be addressed in future versions.
  • In some situations, errors/warnings will mask subsequent errors/warnings. I need to improve parser 'error' recovery, for detecting *all* errors etc. in a given data file.

There are other things I still want to implement -- see the distribution TODO file -- but in general I am pleased with AGElint 1.2. I hope you will be, too.

Enjoy!



berto -> RE: AGElint -- v1.2 official release (10/31/2012 10:15:07 AM)


In Linux, I can run a full series of dochks, on six AGEOD games -- ACW, AJE, NCP, ROP, RUS & WIA -- in about one hour. On average, about ten minutes for each game. PON, with its much larger data set, requires up to a half hour.

In Cygwin Windows, on the same hardware, dochk'ing a single game (say, AJE) requires several hours or more. Compared to Linux, that's more than 25X slower!

I couldn't say why that is so; whether it's due to inefficient Cygwin Windows code, or inherent slowness of Windows (maybe in how Windows implements its file system). (I run Windows 7.)

On Cygwin Windows (also of course on Linux), running agelint against a single data file is quite fast. It's only when you process thousands of data files en masse (when doing a full dochk, or run chkaliases.pl etc.) that it takes considerable time.

Not to gloat or anything, but still one more example of why Linux (and Unix) are superior to Windows for raw data processing. (I am not here asserting the superiority of one OS over another for ordinary consumer use, such as game play.)



Chilperic -> RE: AGElint -- v1.2 official release (10/31/2012 10:31:42 AM)

Downloading now [:)]

Thanks




Chilperic -> RE: AGElint -- v1.2 official release (11/1/2012 12:19:54 PM)

Works very well.

Just an advice for installation under Windows: don't modify the agelint config file with Notepad. Use the alternative method explained by Berto or use Word or OpenOffice. Notepad adds unknown characters to the file.



Chilperic -> RE: AGElint -- v1.2 official release (11/1/2012 2:16:13 PM)

A false positive:

ERROR: in /cygdrive/c/Revolution under Siege/RUS/Events/NavalAI.sct, at (or near) line 30: illegal value: 10000

27
28 Actions
29 AI.SetLocalInterest = $Kazan Shore;10000;$Kazanka;10000;$Simbirsk Shore;10000;$Samara Confluent;10000;$Tzarytsine Shore;10000;$Volga Delta;10000;$Upper Kama;10000;
30>
31 EndEvent
Syntax: AI.SetLocalInterest = RegionUID1|Coeff1|RegionUID2|Coeff2 ..

The max value isn't anymore 999. Any value as far I know may be applied.



berto -> RE: AGElint -- v1.2 official release (11/1/2012 2:38:32 PM)


quote:

ORIGINAL: Chliperic

A false positive:

ERROR: in /cygdrive/c/Revolution under Siege/RUS/Events/NavalAI.sct, at (or near) line 30: illegal value: 10000

27
28 Actions
29 AI.SetLocalInterest = $Kazan Shore;10000;$Kazanka;10000;$Simbirsk Shore;10000;$Samara Confluent;10000;$Tzarytsine Shore;10000;$Volga Delta;10000;$Upper Kama;10000;
30>
31 EndEvent
Syntax: AI.SetLocalInterest = RegionUID1|Coeff1|RegionUID2|Coeff2 ..

The max value isn't anymore 999. Any value as far I know may be applied.


You're right. At the AGEWiki

http://www.ageod.net/agewiki/AI.SetLocalInterest

it says:

quote:

Coeff(n)

Positive Integer value (1 to 999 or higher) to set the interest of the AI

So I should shift that "bug" from ERROR instead to the WARNING (or even NOTICE) category.

BTW, one of the ways I doublecheck, vet, second guess the AGEWiki is to reference my KWD database of actual usage, detailing all uses for every script command and data file parameter across all supported AGEOD games.

So when I see the AGEWiki assert something, and see it verified in actual usage, I am highly confident about my bug assertions.

On the other hand, occasionally the AGEWiki docs and the KWD database examples are in conflict. That's when I have to make a judgment call.

It's not so black-and-white as some might think. Lots of gray area here.

Thanks for the feedback.



Chilperic -> RE: AGElint -- v1.2 official release (11/1/2012 2:43:26 PM)


quote:

ORIGINAL: berto


quote:

ORIGINAL: Chliperic

A false positive:

ERROR: in /cygdrive/c/Revolution under Siege/RUS/Events/NavalAI.sct, at (or near) line 30: illegal value: 10000

27
28 Actions
29 AI.SetLocalInterest = $Kazan Shore;10000;$Kazanka;10000;$Simbirsk Shore;10000;$Samara Confluent;10000;$Tzarytsine Shore;10000;$Volga Delta;10000;$Upper Kama;10000;
30>
31 EndEvent
Syntax: AI.SetLocalInterest = RegionUID1|Coeff1|RegionUID2|Coeff2 ..

The max value isn't anymore 999. Any value as far I know may be applied.


You're right. At the AGEWiki

http://www.ageod.net/agewiki/AI.SetLocalInterest

it says:

quote:

Coeff(n)

Positive Integer value (1 to 999 or higher) to set the interest of the AI

So I should shift that "bug" from ERROR instead to the WARNING (or even NOTICE) category.

BTW, one of the ways I doublecheck, vet, second guess the AGEWiki is to reference my KWD database of actual usage, detailing all uses for every script command and data file parameter across all supported AGEOD games.

So when I see the AGEWiki assert something, and see it verified in actual usage, I am highly confident about my bug assertions.

On the other hand, occasionally the AGEWiki docs and the KWD database examples are in conflict. That's when I have to make a judgment call.

It's not so black-and-white as some might think. Lots of gray area here.

Thanks for the feedback.



Don't worry too much. I'm using a lot of event commands and that's one of the 3 possible false positive I've found. [:)]



berto -> AGElint -- performance specs (11/1/2012 8:18:44 PM)


quote:

ORIGINAL: berto

In Linux, I can run a full series of dochks, on six AGEOD games -- ACW, AJE, NCP, ROP, RUS & WIA -- in about one hour. On average, about ten minutes for each game. PON, with its much larger data set, requires up to a half hour.

In Cygwin Windows, on the same hardware, dochk'ing a single game (say, AJE) requires several hours or more. Compared to Linux, that's more than 25X slower!

...

On Cygwin Windows (also of course on Linux), running agelint against a single data file is quite fast. It's only when you process thousands of data files en masse (when doing a full dochk, or run chkaliases.pl etc.) that it takes considerable time.

On my main home Linux system (telemann), I have a directory, /home/berto/games/AGEOD, containing the latest text data files for all seven AGElint-supported AGEOD games. /home/berto/games/AGEOD has been stripped of all binary data files -- such as image files, sound files -- also stripped of miscellaneous other files, such as the main game executable (.exe) file.

Inventorying:


telemann:/home/berto/games/AGEOD # find . -type f | wc -l
66526

telemann:/home/berto/games/AGEOD # du -skh *
48M AGEod's American Civil War
70M Alea Jacta Est
95M Napoleon's Campaigns
246M Pride of Nations
58M Revolution under Siege
47M Rise of Prussia
58M Wars in America

telemann:/home/berto/games/AGEOD # du -skh * | awk '{t+=$1}; END {print t}'
622


So, in native Linux -- on a near six-year-old, dual-core, 2.6GHz AMD Athlon 64 X2, 4 GB RAM system -- AGElint can chunk through 622 MB of data in 66,526 files multiple times (running chklint.pl, chkaliases.pl, chklocals.pl, chkfiles.pl, chkimages.pl, ...) -- all of that in less than 2 hours (across all seven supported AGEOD games).

Even on older hardware, that's none too shabby, blazingly fast performance even!

That's for Linux. In Cygwin for Windows, and on similar hardware, not quite so fast. [:(]

But there's possible help. See my next post.



berto -> RE: AGElint -- performance specs (11/1/2012 8:32:21 PM)


So, what are the alternatives?

  • Run AGElint on faster hardware. I imagine that on more recent quad-core (or better), hyper fast systems, AGElint will run much more speedily.
  • Using Ubuntu, or some other Linux alternative (I prefer OpenSUSE Linux, or Fedora Core Linux), install Linux side-by-side Windows in a dual boot configuration. When you boot your system into Linux, you still have access to your Windows files. (I have configured dual boot Windows/Linux systems dozens of times. It's not that difficult, especially not with Ubuntu.)
  • On your Windows system, run Linux in a virtual machine. (At home, I have configured two Windows 7 systems to run Linux web servers in Oracle VirtualBox VMs.) In this configuration, too, your Linux VM still has access to your Windows files.

    So, in addition to Cygwin for Windows, there are several other ways to run AGElint on your Windows PC.

    Of course, running AGElint in native Linux on bare metal is still best. [:)]

    The point is, you can have your cake(s) and eat it (them) too. If you don't have a spare system to run Linux natively, you can still run AGElint, more or less fast, on your one Windows system.

    You just have to summon the will. Where there's a will, there's a way.



    • berto -> AGElint is poppycock? No way! (11/8/2012 8:38:19 AM)


    In the AJE Game Data Bugs thread

    http://www.matrixgames.com/forums/tm.asp?m=3206201

    I reported and documented hundreds of bugs, glitches, and data anomalies in the latest available Alea Jacta Est release. Of those, about 200 were acknowledged as significant (worth fixing) by a principle AJE developer.

    Not a few false positives, a considerable number of minor errors (not worth the trouble of fixing), but a substantial number (~200) of genuine, significant errors (worth fixing) too.

    Based on his feedback, I will further refine the AGElint code and rule sets, and issue an updated AGElint release soon. The war on false positives continues!

    In late October, as one of the final steps in the AGElint 1.2 vetting process, I did a complete error check across all seven supported AGEOD games: ACW, AJE, NCP, PON, ROP, RUS, WIA.

    This is what I found:


    [root@berto chk_QA_20121029]# for g in acw aje ncp pon rop rus wia; do echo $g; egrep -c "^ERROR:" chklint_$g\_QA_20121029_error_rpt.txt; egrep -c "^WARNING:" chklint_$g\_QA_20121029_warning_rpt.txt; awk '{t+=$1}; END {print t}' chkaliases_$g\_QA_20121029_lst.txt; awk '{t+=$1}; END {print t}' chklocals_$g\_QA_20121029_lst.txt; echo; done
    acw
    140
    280
    191
    137

    aje
    86
    37
    493
    532

    ncp
    22
    12
    238
    2325

    pon
    586
    673
    3647
    12255

    rop
    122
    26
    123
    236

    rus
    159
    199
    422
    174

    wia
    59
    71
    300
    387


    So, in AJE for example, I discovered


    aje
    86
    37
    493
    532


    That is, 86 script/data file ERRORs, 37 script/data file WARNINGs, 493 bad alias refs, and 532 missing localizations.

    Of the nearly 1,000 reported bad alias refs and missing localizations, by far most of them were deemed unimportant (and will be filtered out of future AGElint releases); but well over a hundred of the alias and localization errors were considered significant enough to require fixing.

    Looking at the numbers across all games above, you can see that there are (very) roughly similar number of errors in each of the supported games. Although PON, with its substantially larger data file set, has a substantially larger number of reported errors too.

    Earlier in this thread (see above), I wrote:

    quote:

    ORIGINAL: berto

    By now, I have thoroughly vetted chklint & chkaliases results across seven supported AGEOD games -- ACW, AJE, NCP, PON, ROP, RUS & WIA. I am taking extraordinary care this time to reduce false positives to a bare minimum. Still...

    Are there bugs in each game? You betcha! Dozens and even hundreds of them in every game (including AGEOD's newest release, Alea Jacta Est), many of them indisputable and of greater or lesser seriousness.

    (And are there bugs in AGElint? You betcha! But I try my darndest to find them and fix them! [;)])

    Based on a careful review of reported bugs together with a principle AJE developer; weeding out false positives; and disregarding real but insignificant errors -- we are still left with roughly 200 significant errors in the latest AJE (Public Beta Patch 1.01).

    I remain confident in my earlier assertion:

    quote:

    Across all the games, I am observing in total easily well more than a thousand game data file bugs.

    In fact, taking into account the rather larger numbers for PON, the number of significant errors is more like a few thousand.

    My standard disclaimer:

    [Disclaimer: I make no claim about the significance or insignificance of any discovered bug, problem, glitch, or anomaly. Whether or not it impacts game play, or goes entirely unnoticed. Whether in the larger scheme of things it's important, or unimportant. I just call them as I see them.]

    But I will without hesitation positively proclaim:

    AGElint has proved its worth as a useful, even indispensable, bug checking and all-around QA toolkit.

    (Remember too: "AGElint is more than a beginning, but far from an ending ... There's still much more untapped potential, much more to be done ...")

    Quoting the AGElint README:

    quote:

    What is AGElint? Akin to the well-known C programming language utility lint. ("Lint: A Unix C language processor which carries out more thorough checks on the code than is usual with C compilers. Lint is named after the bits of fluff it supposedly picks from programs." See here.). Similarly, AGElint is a toolkit of programs designed to discover and report bugs, problems, glitches, anomalies in the AGEOD game data files.

    If anybody thinks that AGElint is poppycock, that I'm just making this up, please read this:

    http://en.wikipedia.org/wiki/Static_code_analysis

    http://en.wikipedia.org/wiki/Automated_code_review

    Proactive, pre-execution debugging (AGElint) vs. reactive, post-execution debugging (the traditional way).

    Static analysis (AGElint) vs. dynamic analysis (playtest the game, inspect the log files).

    Each approach has its strengths and weaknesses, but together they've got the bugs surrounded -- no escape!

    AGElint is poppycock? No way!



    berto -> RE: AGElint is poppycock? No way! (11/9/2012 10:04:36 AM)


    So, by one means or another, you have debugged an AGEOD game data set to your satisfaction, to your considered QA standards. Then what?

    I wager this: Every AGEOD developer, maintainer, or modder has at one time or another written a new event script, or modified a model or unit file, or whatever. And omitted a semi-colon. Or misspelled an alias. Or did one of dozens of other stupid, "insignificant" mistakes that we fallible, distracted, brain-farting humans are prone to make.

    Then spent many minutes, hours even: running the CSV splitter, loading the game, playing the game, sooner or later somehow or other seeing the results, inspecting the log files. Returning to the code source. Staring and staring at the code or data. Then after a while, maybe much later -- after how many rounds of CSV splitter ops, game loads, log inspections, etc.? -- finding that stupid, "insignificant" missing semi-colon, or misspelling, or whatever.

    Some of these stupid little typos are quite hard for human beings to spot. By contrast, computer programs are very good, unfailingly perfect even (if coded right), at finding these hard-to-spot typos.

    I suspect this: In just a single bad bug hunt, you might waste more time with your inefficient bug hunting techniques than the time it would take you to download and install AGElint (and maybe also Cygwin Windows), then learn to use it.

    Which is faster?

  • Code a new event. (Run the CSV splitter...) Restart the game. Run the game to the point in question. See the results. Check the log files for errors.
  • Code a new event. Run AGElint against the one edited file. See your errors (if any) immediately.

    Unless I'm mistaken (quite possible!), I would think the latter is faster. AGElint can process individual files in the blink of an eye, as fast as you can enter the agelint command and hit the Enter key. . Does your typical AGEOD game load in less than a couple of minutes? And then how long does it take to execute the code in question, see the results, then review the pertinent log file(s)?

    The proof is in the pudding. An AGEOD code/modding wizard says that by using AGElint his development and debugging time has been cut by a factor of 10 -- i.e., he's achieving the same results in 10% of the time. This replaces his earlier modus operandi of coding/editing, launching and running the game, reviewing the log files, discovering bugs, ..., wash/rinse/repeat.

    Sure, it takes time to install and learn to use AGElint, then time to fix the many discovered bugs. But over the long run, the initial effort more than pays for itself in time savings. Not to mention better code, more peace of mind, fewer forum bug reports, greater player satisfaction, ...

    Do you want to spend your time chasing after and reading about bugs? Or doing other good stuff, such as actually playing the games?

    Your choice.



    • Chilperic -> RE: AGElint is poppycock? No way! (11/9/2012 10:30:01 PM)

    Hi, I'm the wizard [:D]

    In fact I'm not. I would like, but I'm not so skilled.

    However, since I'm modding on the AGE engine since 2007, I've learnt a few things. Among these, there's one:

    agelint is free to use, openly available. Agelint reports an incredible number of errors in AGE game data.

    Since 2007, the AGE engine has become very complex to the point to be almost impossible to use without a real debugging tool if ever your modding effort goes beyond changing a few values or write ine or two events.

    Modding in AGE without a debugging tool is like taking command of a jet without having leant to flight.

    There is an official tool for AGEOD game, hidden in the hands of "official" team and those having receiving some "official" stamp of AGEOD. I will not evaluate this official tool, as I have not the last version. However, this tool has never been released for anyone.

    What Berto has done with agelint is just amazing, by its quality AND by the open release on it. In one sentence, he has done for encouraging modding than anything AGEOD has ever done.

    I will add once the main line commands are learnt, agelint is easy to use, CTD safe, and easy to install on Windows thanks to Berto's titorial. For those playing my mods, a part of their strenght comes from Agelint.




    berto -> AGElint -- v1.2.1 official release (11/10/2012 5:51:28 PM)


    Here is the fifth "official" AGElint version, 1.2.1 (20121110):

    No major changes in AGElint version 1.2.1, basically just fewer false positives.

    Here is a more detailed list of changes in AGElint 1.2.1:


    Changes from AGElint_1.2 (20121031) to AGElint_1.2.1 (20121110):

    • Eliminated false positives based on a review with the AJE lead designer.
    • Squelched some problematic chk*.pl cases (for AJE).
    • Other minor code & doc fixes/changes.


    I have painstakingly reviewed and vetted all chklint.pl (agelint) results, also results for chkaliases.pl, chklocals.pl, etc. There should be fewer than ever false positives. Any remaining false positives would stem more from ambiguity in the AgeWiki docs, else honest differences of opinion. If you approach AGElint with an open mind, and generous spirit, you should find much value in it, and little to quibble over.

    Enjoy!



    berto -> RE: AGElint -- v1.2 official release (11/11/2012 4:09:24 PM)


    quote:

    ORIGINAL: Chliperic

    A false positive:

    ERROR: in /cygdrive/c/Revolution under Siege/RUS/Events/NavalAI.sct, at (or near) line 30: illegal value: 10000

    27
    28 Actions
    29 AI.SetLocalInterest = $Kazan Shore;10000;$Kazanka;10000;$Simbirsk Shore;10000;$Samara Confluent;10000;$Tzarytsine Shore;10000;$Volga Delta;10000;$Upper Kama;10000;
    30>
    31 EndEvent
    Syntax: AI.SetLocalInterest = RegionUID1|Coeff1|RegionUID2|Coeff2 ..

    The max value isn't anymore 999. Any value as far I know may be applied.


    I fixed this one case in AGElint 1.2.1, but here's a more general fix, for AI.SetLocalInterest and elsewhere (all additions, or edits, in boldface).

    In agelint.h:


    #define FOREVER 999
    #define NEVER -999
    #define ERRONEOUS -999
    #define _NULLMAX 99999
    #define _NULLMIN -99999


    In txt.l (indentation in the original code fails to display here):


    /* check integer value
    */
    void
    chkint (const int val, const int min, const int low, const int high, const int max)
    {
    if (((val < min) && (min != _NULLMIN)) || ((val > max) && (max != _NULLMAX))) {
    txterrmsg(_ERROR, TRUE, linenorhs, "illegal value: %d", val);
    } else if (isshow_warn && ((val < low) || (val > high))) {
    txterrmsg(_WARNING, TRUE, linenorhs, "suspicious value: %d", val);
    }
    return;
    }


    In txt.y (indentation in the original code fails to display here):


    aichgarealocalinterest: _AICHGAREALOCALINTEREST eq alsval vb int
    { chkint($5, _NULLMIN, -9999, 9999, _NULLMAX);
    /*
    http://www.ageod.net/agewiki/AI.ChgAreaLocalInterest
    Syntax: AI.ChgAreaLocalInterest = <Area_Alias>|<Interest_Coeff>
    */
    }
    ;

    ...

    aisetarealocalinterest: _AISETAREALOCALINTEREST eq alsval vb intpos
    { chkint($5, 0, 1, 9999, _NULLMAX);
    /*
    http://www.ageod.net/agewiki/AI.SetAreaLocalInterest
    Syntax: AI.SetAreaLocalInterest = <Area_Alias>|<Interest_Coeff>
    */
    }
    ;

    ...

    aisetlocalinterestparam: alsval sc intpos {
    chkint($3, 0, 1, 9999, _NULLMAX);
    /* check that alsval is a region alias */
    }
    | alsval { txterrmsg(_ERROR, TRUE, linenorhs, "missing Coeff in AI.SetLocalInterest"); }
    | intpos { txterrmsg(_ERROR, TRUE, linenorhs, "missing RegionUID in AI.SetLocalInterest"); }
    ;


    You can either make these edits yourself (then recompile agelint, of course). Or just live with, ignore the false positives until the next AGElint release.



    berto -> AGElint -- dochk 'set -x' trace mode (11/11/2012 4:24:16 PM)


    In the AGElint USAGE file (included in the release .zip file), you will find examples of agelint usage, also sample uses of chklint.pl, chkaliases.pl, etc.

    In the dochk script, the one that does a complete error check of a game's entire data set (e.g., './dochk rus QA'), I have


    #!/bin/bash

    set -x

    ...


    where 'set -x' turns on trace mode. So, in the dochk output, you will see


    ...

    rus

    + '[' rus = +G ']'
    + '[' rus = -z ']'
    + GAME=rus
    + VRSN=QA
    + DATE=
    + '[' xQA = x ']'
    + '[' x = x ']'
    ++ date +%Y%m%d
    + DATE=20121111
    + RMGAMEROOT=./rmgameroot.pl
    + GRPRPT=./grprpt.pl
    ++ pwd
    + AGELINTROOT=/home/berto/games/ageod/agelint
    ++ egrep '^gameroot' agelint.conf
    ++ awk '-F"' '{print $2}'
    + GAMEROOT=/media/KINGSTON/Games/AGEOD
    + ACWDIR='AGEod'\''s American Civil War/ACW'
    + AJEDIR='Alea Jacta Est/AJE'
    + NCPDIR='Napoleon'\''s Campaigns/NCP'
    + PONDIR='Pride of Nations/VGN'
    + ROPDIR='Rise of Prussia/ROP'
    + RUSDIR='Revolution under Siege/RUS'
    + WIADIR='Wars in America/WIA'
    + '[' rus = acw ']'
    + '[' rus = aje ']'
    + '[' rus = ncp ']'
    + '[' rus = pon ']'
    + '[' rus = rop ']'
    + '[' rus = rus ']'
    ++ awk '-F"' '{print $2}'
    ++ egrep '^rusdir' agelint.conf
    + RUSDIR='Revolution under Siege/RUS'
    + GAMEDIR='/media/KINGSTON/Games/AGEOD/Revolution under Siege/RUS'
    + '[' ']'
    + echo 'doing chklint...'
    doing chklint...
    + ./chklint.pl +t -n +w -e +v -s +c9 +u +x +E -g rus
    + cat chklint.nosquelch_warning.rus.out
    + ./rmgameroot.pl -g rus
    + ./chklint.pl +t -n -w +e +v -s +c9 +u +x +E -g rus
    + cat chklint.nosquelch_error.rus.out
    + ./rmgameroot.pl -g rus
    + echo 'doing chkaliases...'
    doing chkaliases...
    + ./chkaliases.pl +i +E -g rus
    + cat chkaliases_rus.out
    + ./rmgameroot.pl -g rus
    + ./grprpt.pl
    + ./grprpt.pl
    + ./rmgameroot.pl -g rus
    + sort -t: -k3 chkaliases_rus.out
    + cat chkaliases_rus.out
    + sort
    + uniq -c
    + sed '1,$ s/ not found//'
    + sort -nr
    + awk -F: '{print $NF}'
    + cd '/media/KINGSTON/Games/AGEOD/Revolution under Siege/RUS/Aliases'
    + cd /home/berto/games/ageod/agelint

    ...


    For the highlighted (boldfaced) text, you will see more example usages of chklint.pl, chkaliases,pl, etc. that I use in the course of a dochk run. Refer to these -- or cut-and-paste to rerun outside of dochk -- as needed.

    If you want to turn off trace mode in dochk, you can either edit 'set -x' to 'set +x' or comment it out via '#set -x'.



    berto -> diff -- a short tutorial (11/13/2012 9:09:48 AM)


    Here are some useful tips.

    To determine globally the differences from one AGEOD release to the next, try for example:


    Robert@roberto /home/Robert/Games/AGEOD/agelint
    $ diff -r "/cygdrive/c/Games/AGEOD/Revolution under Siege 1.06" "/cygdrive/c/Games/AGEOD/Revolution under Siege" | tee RUS_1.06_1.06a.diff


    The '| tee RUS_1.06_1.06a.diff' "pipes" the diff command output to the tee command, which sends the output to both screen and the indicated file, RUS_1.06_1.06a.diff.

    If you prefer, you could use


    Robert@roberto /home/Robert/Games/AGEOD/agelint
    $ diff -r "/cygdrive/c/Games/AGEOD/Revolution under Siege 1.06" "/cygdrive/c/Games/AGEOD/Revolution under Siege" > RUS_1.06_1.06a.diff


    which outputs just to the file.

    To determine the differences between two folders, try for example:


    Robert@roberto /home/Robert/Games/AGEOD/agelint
    $ diff -r "/cygdrive/c/Games/AGEOD/Revolution under Siege 1.06/RUS/Events" "/cygdrive/c/Games/AGEOD/Revolution under Siege/RUS/Events" | tee RUS_Events.diff


    To determine the differences between two files, try for example (note the absence of the '-r' option):


    Robert@roberto /home/Robert/Games/AGEOD/agelint
    $ diff "/cygdrive/c/Games/AGEOD/Revolution under Siege 1.06/RUS/Events/GRNAI.sct" "/cygdrive/c/Games/AGEOD/Revolution under Siege/RUS/Events/GRNAI.sct" | tee GRNAI.sct.diff


    A really useful option to try is -c #, as in


    Robert@roberto /home/Robert/Games/AGEOD/agelint
    $ diff -c "/cygdrive/c/Games/AGEOD/Revolution under Siege 1.06/RUS/Events/GRNAI.sct" "/cygdrive/c/Games/AGEOD/Revolution under Siege/RUS/Events/GRNAI.sct" | tee GRNAI.sct.diff


    which shows context around the diffs of, by default, 3 extra lines. You can control the amount of context, as in


    Robert@roberto /home/Robert/Games/AGEOD/agelint
    $ diff -C 9 "/cygdrive/c/Games/AGEOD/Revolution under Siege 1.06/RUS/Events/GRNAI.sct" "/cygdrive/c/Games/AGEOD/Revolution under Siege/RUS/Events/GRNAI.sct" | tee GRNAI.sct.diff


    which would extend the context to 9 surrounding lines, before and after.

    And so on. Like most Unix/Linux commands, there are a ton of options available. If you have the so-called Cygwin "man[ual] pages" installed in your setup (in Linux, the man pages would be installed by default), you might try the command


    $ man diff


    It pays to learn some of this Unix/Linux stuff. You might be surprised at what you find. [;)]



    berto -> RE: AGElint: excludes.???.dat (11/16/2012 9:03:55 AM)


    In the early stages of AGElint's development, honest differences of opinion (as I saw it) arose over which game data files to bug check, or not to check. Rather than impose one view vs. another, I implemented the file Exclude feature, thereby leaving it for each AGElint user to decide.

    For each AGElint-supported AGEOD game, there is a file -- for example, excludes.wia.dat -- where you specify which files AGElint should exclude, or ignore. You can specify files by pattern match, else by listing the exact file names that should be skipped.

    The AGElint file Exclude feature is detailed here:

  • AGElint: excludes.???.dat [post 122]

    At the AGEOD Forum, WIA Patch 1.10e was announced several days ago. In the patch announcement, it says:

    quote:

    Removed the setups for the ''original'' 1755 Campaign, 1756 Montcalm scenario, and 1759 Annus Mirablis scenario, leaving the ''COL'' faction variants intact ... the game will still include the Events files for these scenarios to allow ongoing games to be completed.

    This is a perfect opportunity to demonstrate the AGElint file Excludes feature.

    In excludes.wia.dat we add (added files in boldface):


    [root@berto agelint]# cat excludes.wia.dat
    # wia excludes
    (Test|Parse|Samples)
    Scripts/Script.ini
    Scens/1755 Campaign.scn
    Scens/1759 Annus Mirabilis.scn
    Scens/1756 Montcalm.scn
    Scens/1756 Montcalm_RoiLouis.scn
    Events/1755 Campaign.sct
    Events/1759 Annus Mirabilis.sct
    Events/1756 Montcalm.sct
    Events/1756 Montcalm_RoiLouis.sct


    AGElint will now ignore the listed files.

    (Even though the Scens files above were removed, there's no harm in leaving them in excludes.wia.dat. AGElint will not complain if any excludes.???.dat references non-existent files.)

    On 20121111, before the obsolete scenario setups were removed from WIA, also on 20121116, after removal of the scenario setups, I ran AGElint against the WIA game data files (using the command 'dochk wia QA'). diffing the report files, I see for example:


    [root@berto agelint]# diff chklint_wia_QA_20121111_error_rpt.txt chklint_wia_QA_20121116_error_rpt.txt | less

    ...

    < Events/1755 Campaign.sct, Fri Sep 7 2012 06:41:46
    <
    < ERROR: in Events/1755 Campaign.sct, at (or near) line 149: line-specific or other contextual syntax error: #-# (0x2d), #-1#
    <
    < in event evt_nam_FortEdward70:
    <
    < 140 StartEvent = evt_nam_FortEdward70|1|2|evt_txt_FortEdward70|Event-img2_Fort1.bmp|$Tajacook|NULL
    < 141
    < 142 Conditions
    < 143 NotEnemy = $Tajacook
    < 144
    < 145 EvalRgnOwned = $Tajacook
    < 146 MinDate = 1755/09/01
    < 147 EvalRgnStruc = $Tajacook;FACALL;STRUCDEF $Village;NOT
    < 148 EvalRgnStruc = $Tajacook;FACALL;STRUCDEF $Fort;NOT
    < 149> CountUnits = $Tajacook;-1;$GBR;-1;>=;4
    < 150
    < 151 Actions
    < 152 SelectFaction = $GBR
    < 153
    < 154 CreateStruc
    < 155 SetType = $Fort
    < 156 SetLevel = 1
    < 157 SetName = Fort Edward
    < 158 Apply
    <
    < http://www.ageod.net/agewiki/CountUnits
    <
    < Syntax: CountUnits = <area alias>; <region or area>; <faction>; <unit type>; <operator>; <number of units>

    ...


    Interpretation: lines marked '<' are in the command's first listed file, chklint_wia_QA_20121111_error_rpt.txt, but not in the second, chklint_wia_QA_20121116_error_rpt.txt. The ERROR in Events/1755 Campaign.sct was reported before. After, it is not reported -- because the Events/1755 Campaign.sct file was excluded from the later (20121116, post excludes.wia.dat edit) AGElint error check.

    (Note: This is not the place to quibble about whether the above is an actual ERROR or not. The point here is to demonstrate the AGElint Exclude feature, nothing more.)

    You activate the Exclude mechanism via the '+E' command-line switch, as in:


    ./chklocals.pl +i +E +M -g wia


    You de-activate excludes -- i.e., error check all files, including the ones specified in excludes.wia.dat -- via the '-E' command-line switch, as in:


    ./chklocals.pl +i -E +M -g wia


    Just in case you want to error check all files for some reason.

    If AGElint reports ERRORs (and WARNINGs ...) that you think are false positives because reported in obsoleted game data files (or test files or ...) -- the solution is pretty simple: Just exclude the files in the excludes.???.dat!



    • berto -> AGEpp -- an AGE code beautifier (11/20/2012 7:58:01 PM)


    I have just announced, and released, a companion utility to the AGElint toolkit: AGEpp -- an AGE code beautifier

    AGEpp is a tool to reformat poorly laid out, confusing, hard to read AGE event scripting code.

    In future, I will include the AGEpp programs (only; not the docs) -- agepp, agepp.pl, etc. -- in the AGElint distribution .zip file. That way, as an AGElint user, you will have the AGEpp utilities without having to do a separate AGEpp download and install.

    Enjoy!



    Chilperic -> RE: AGEpp -- an AGE code beautifier (11/20/2012 10:28:08 PM)

    Works very well indeed.



    Page: <<   < prev  3 4 5 6 [7]

    Valid CSS!




    Forum Software © ASPPlayground.NET Advanced Edition 2.4.5 ANSI
    1