berto
Posts: 20708
Joined: 3/13/2002 From: metro Chicago, Illinois, USA Status: offline
|
Quoting from the (private, closed off) Dev Forum, three days in the life... quote:
ORIGINAL: berto I will look into this [newly reported issue]. But before then, I need to finish csluachk.pl. I am also working on some crash bugs happening in WinXP auto tests, but not on other Windows versions. quote:
ORIGINAL: berto Latest auto crash/bug/freeze test results: Win7 -- 71 scenarios tested, no problems. Win 8.1 -- 87 scenarios tested, one crash in Latrun_1_1948.scn, close to game exit (I think, haven't investigated it yet). WinXP -- Focusing narrowly on several AIW scenarios with a crash history, in my latest tests AIW59.scn & AIW60.scn crash predictably, test after test. Something to do with the org mechanism. By means of code TRACEing, I have narrowed my search down to a single function. Getting closer, but not there yet. The debugging continues... quote:
ORIGINAL: berto quote:
ORIGINAL: berto Win 8.1 -- 87 scenarios tested, one crash in Latrun_1_1948.scn, close to game exit (I think, haven't investigated it yet). Confirmed. Game play had concluded. That test game crashed as it was finishing up and freeing graphics memory. So not that big a deal. Still, an embarrassment, shouldn't happen, and I will fix this if I can. quote:
ORIGINAL: berto quote:
ORIGINAL: berto Latest auto crash/bug/freeze test results: Win7 -- 71 scenarios tested, no problems. Famous Last Words. There were problems; I was just looking in the wrong place. Been sweating bullets the last two days hunting down these crash bugs. I believe I have squashed them. Testing will tell... quote:
ORIGINAL: berto At the moment, I have four different automated crash/bug/freeze test sequences running concurrently, on Win10, Win8.1, Win7 & WinXP, with two of them at the L3 logging level. quote:
ORIGINAL: berto That whooshing sound you hear is me heaving a big sigh of relief. On all four test systems, 13 to 14 tested scenarios in, no crashes, segfaults, or other signs of trouble. Everything appears to be WAD again. A classic case of Fix One Thing, Break Another. Or in this case, fix one thing, break several others. Recall this recent fix, applied and announced about a week ago: quote:
Fixed a bug where, under unusual circumstances, the correct scenario map might not show in the Scenario Information dialog. In the fix ... <censored> ... Eventually I cleaned up the code. Eventually I fixed all affected places. Eventually, the automated tests ... showed no more crashes. Problem(s) solved! ... Now up to the 17th scenario in one of the test sequences with it, and the others, all apparently rock solid stable and evidently WAD, I can confidently expect that this crash bug episode is now behind me. I can return to more meaningful pursuits, get back to the fun stuff. One lesson learned: During the entire code development cycle, I need to run automated code regression tests on a near daily basis, not just in the final few weeks before release. Better to discover, and to fix, crash and other bugs when they first surface, not weeks and months later when they are naturally much harder to find. A larger takeaway here. This underscores the notions: If it ain't truly broke, don't fix it. Because in fixing one thing, we might break another, or others. At this point, so near to release, only fix the truly significant bugs. Don't risk fixing the low priority ones. Even seemingly simple and straightforward new features and wish list fulfillments -- Don't succumb to the temptation. Don't do it! The implicit soft code freeze is hardening. We are approaching the time where we need to freeze this thing solid. Sometimes the question is asked: Do you guys ever test this thing? Why, yes we do. We test and test and test like crazy. But despite our good intentions, despite our best efforts, bugs will slip in and through. This is a hard business. You try your best, but The Beast is so complicated that you just have to accept the fact: There Will Be Bugs. <shrug>
_____________________________
|