Shannon V. OKeets
Posts: 22095
Joined: 5/19/2005
From: Honolulu, Hawaii
Status: offline
|
February 1, 2010 Status Report for Matrix Games’ MWIF Forum
Accomplishments of January 2010
Project Management
I monitored all the threads in the MWIF World in Flames forum daily.
Hardware and Software
I spent virtually all of January trying to get Delphi 2010 and Theme Engine 9.10 working correctly. I have succeeded in getting Delphi to compile and link MWIF but still can not get the IDE/debugger (Interactive Development Environment) to execute MWIF in debug mode. What this means is that I am unable to examine internal values of variables while the program is running. Now I can still debug MWIF by executing it the same way the beta testers do, which provides me with reports on what the program was doing when it halts/crashes. But that doesn’t let me control its execution to check on the status of variables along the way to the fatal error. As an analogy, the debugger should let me see what the engine is doing prior to the car running off the road. Right now I just get to sift through the mangled wreckage.
In my descriptions of Theme Engine 9.10 and Delphi 2010 below, getting everything to work correctly is a lot like opening a safe which has a combination lock requiring you to know ten 2 digit numbers. If any of the settings is wrong, then the program fails to run. The task of finding the correct settings requires the programmer (me!) fault isolating each problem and rigorously exploring all possible values to see what works.
Beta Testing
I released versions 4.00.01, 4.00.02, 4.00.03, 4.00.04, and 4.00.05 of MWIF to the beta testers in January. That is a lot less progress than there would be normally in 5 new versions, since almost all the changes related to misbehavior by Theme Engine. There is more about Theme Engine 9.10 below.
I made a few corrections to bugs as part of new versions but they do not have enough heft to warrant mentioning. Hence, the sections below report Nothing New.
Yesterday I began to refocus my attention on bugs. For right now I am choosing which ones to fix according to those that are easiest to solve given a non-functioning debugger.
Saved Games
Nothing new.
Map and Units
Nothing new other than some more naval unit writeups from Rob/Warspite.
Scenarios and Optional Rules
Nothing new.
MWIF Game Engine and CWIF Conversion
Nothing new.
Player Interface
Nothing new.
Internet - NetPlay
Nothing new.
PBEM
Nothing new.
Artificial Intelligence (AI)
Nothing new.
Player’s Manual
Nothing new.
Tutorials, Training Videos, and Context Sensitive Help
Nothing new.
Historical Video, Music, and Sound Effects
Nothing new.
Marketing
Nothing new.
Communications
Nothing new.
===================================
Here is a write-up on Theme Engine that I will post to their forum as soon as I am officially ‘registered’ as a forum member there. This mostly coherent summary of knowledge gained here was tortuously acquired.
===================================
I bought a new computer in December and decided to upgrade from Delphi 2007 to Delphi 2010 and from Theme Engine 6.06 to 9.10 at the same time. Here are some lessons learned related to the Theme Engine upgrade, which I hope will make a similar transition easier for others.
As background, my old system had 3 GB of RAM, 40GB of unused disk, a 3.8 GHz CPU, and ran under Windows XP. My new system has 12 GB of RAM, 770GB of unused disk, a quad core 2.67 GHz CPU, and runs under Win7 Home Edition.
I encountered 3 major problems in converting my main application (320,000 source lines) from Theme Engine 6.06 to 9.10.
1 . The lists of themes in TTeThemeList must have unique internal names. I had a couple of ‘placeholder’ themes in my list which were more or less duplicates of other themes in the list. Note that changing the file names of the ‘mskn’ files makes no difference. It is the “theme names” assigned when the themes are created that must be unique. From what I can gather, although TTeThemeList assigns a unique index to each theme, and although you always change themes using that index, internally the TE code uses the index to determine the theme’s name and then searches an internal list of themes by name instead of by index. If it finds the name in its internal list, it then deletes an internal reference using the original index. So, if you have themes A1 and A2 with the same name, and reference A1 (which has already been deleted from the internal list), TE finds A2 when it does its search. It then believes it has to delete that internal entry, tries to delete A1 using A1's index, and crashes with an Access Violation. This problem can be avoided by making sure all the themes in TTeThemeList are unique.
2. TTeButton now contains the property FXEnabled. This has the effect of giving every TTeButton a TBitmap, regardless of whether FXEnabled is set to True or False. Indeed, the bitmap is referenced whenever the cursor enters or exits the TTeButton component. Again, that occurs even when FXEnabled is set to False. Now my game/application comes razor thin close to tapping out the available Windows graphics resources. The map is stored using 5000+ bitmaps for coastlines and the combat units use another 2000+. In addition, the map is rendered on the fly since it is a hexagon tessellation of 70,2000 hexagons. Basically, I can’t afford a bitmap for each TTeButton, especially since some of the forms contain 2 or 3 dozen buttons. The extraneous bitmaps created for TTeButton caused my application to crash by running out of Windows resources.
I feel there is no need for TTeButton to contain the FXEnabled property. If programmers need that capability, then they can use TTeEffectButton or TTefxButton. TTeButton should have been left as it was without needlessly burdening it with additional features and resource requirements.
Anyway, my solution to this was to edit the Theme Engine source code, commenting out the creation and deletion of the TBitmap for TTeButton and similarly commenting out the references to that bitmap in the OnEnter and OnExit routines for TTeButton. The FXEnabled property is still present, but after my edits, it simply never creates or uses the TBitmap.
3. I have no solution for the third problem I encountered. It is up to the Theme Engine authors to fix this bug. What happens is that if you use TTeOpenDialog it is incapable of switching the View of the files in the directory. Whenever you use the drop down View menu and click on an alternate view (e.g., Detail or Icons), the program crashes - generating a Theme Engine error message: “Error setting path”. I guess I could have read more of the code and figured out what was going wrong but my joy with rewriting TE internals had waned by the time I discovered this bug.
===================================
Here is a write-up on Delphi 2010 that I posted to that forum earlier today. I’ve already received one suggestion which might help me figure out the cause of my problem with the IDE/debugger. As with Theme Engine, this mostly coherent summary of knowledge gained here was tortuously acquired.
===================================
I bought a new computer in December and decided to upgrade from Delphi 2007 to Delphi 2010. Here are some lessons learned related to the Delphi upgrade, which I hope will make a similar transition easier for others. I still have an insurmountable problem with the speed of the IDE (#13 below).
I encountered many major problems in converting my main application (320,000 source lines) from Delphi 2007 to Delphi 2010. Smaller projects and utility programs converted without any trouble. Please note that I am not a Delphi ‘guru’ (though I have been programming for 40+ years and in Delphi for 5 years) and there are apt to be inaccuracies in some of the statements below, since most of them are based on my personal experience and not a definitive knowledge of how the Delphi 2010 internals work.
1. The very first thing to do is to create a definitive list all the libraries your source code uses: where the package (dpk), source modules, (pas), compiled modules (dcu), design time library (bpl), and run time compiled library (dcp) files are stored. You need all this information to provide search paths and folder names so Delphi can find the libraries when it needs them. Oh, and you should do that for your application code too, of course. You did get the latest versions of all those libraries so they support Delphi 2010, didn’t you?
2 . The 3 chapters by Mario Cantu on converting from ASCII to Unicode should be read and Mario’s advice followed so that the conversion from single byte strings to double byte strings doesn’t cause problems. The compiler will identify most of these for you and between what Mario has written and the compiler you shouldn’t have too much trouble making this change. It may take some time to perform all the edits, depending on your existing code. I found I needed to also examine older (third party) free libraries I was using in some detail to make sure they executed Unicode correctly. Actually, by eliminating unused routines, I cut 90% of the modules from these libraries, reducing the number from 443 to 43. By so doing, I had far less code to worry about for the conversion to Unicode.
3. The ‘automatic’ conversion of old project files (dproj) to new is not necessarily performed well by Delphi 2010. Because I was changing both computers and operating systems, the old settings were sometimes complete nonsense, especially for search paths. You need to go through all the general settings for your projects in Tools - Options - Environment Options. Especially important is Delphi Options - Library - Win32 if that is what you are using (see below).
4. I strongly recommend making a copy (in a simple text file) of all the settings you discover when you first open: (1) Tools - Options and (2) Project - Options. Then you should keep a record of what you try and any success/failure you experience. Yes, this is tedious to do. But believe me, when lost in the labyrinth of the project settings, it is a great comfort and time saver to be able to return to your starting point (or last most successful parameter settings) and review what you have already tried. And, no, you will not be able to retain all that information in your head. I have an excellent memory, able to play chess blindfolded, and this stuff was way beyond my ability to keep track of it.
5. Tools - Options - Environment Options - Delphi Options - Library - Win32 had a lot of values defined by the automatic conversion that I had to edit (through trial and error). The Library path is for source code and appears to be used by the compiler (obviously) and also the debugger (for identifying individual lines in the source code for reporting in the call stack). You should have an entry for Package output directory and DCP output directory. For Browsing path, Namespace prefixes, and Debug DCU path, I have blank entries. One of the Delphi forum members reported that the presence of extraneous entries in the Directories settings caused the IDE to run slowly. I am under the impression that cleaning these up did increase the IDE’s speed of execution for me - though I have no definite proof of that. Regardless, neatness counts in programming.
6. Tools - Options - Environment Options - Tool Palette - Environment Variables also had garbage carried over from the Delphi 2007 settings. Note that these apply to all projects and they can be modified for individual projects in Projects - Options - Debugger - Environment Block. I found this quite confusing since Tools - Options contains User overrides, and then Project - Options also contain User overrides. My sense of this is that it is a layering of defaults and overrides. Figuring out what is going on internally exceeded my patience. I just made sure the setting for the project was the best I could get it. In practice, I eliminated all the overrides except for Path. Note that this is one place where my point #4 comes in very handy. The User overrides are a concatenated string of path names and infuriating to work with as such. For example, some of my paths are strings of 500+ characters, each of which have to be absolutely correct. Adding and removing path names, even simply reading the current path names, is a real pain.
7. Project - Options - Delphi Compiler contains 3 build configurations: Base, Debug, and Release. The IDE always assumes you want to use Base which is very annoying. I have to believe that programmers spend 99% of their time using Debug, which means that the first thing you should do when you fire up the IDE is go into this drop down menu and set the Build Configuration to Debug. I got the Base settings the way I wanted them and then let the IDE insert its default changes for Debug and Release. The Search path is the most difficult setting to get correct here. This is where you need to know where your library code is stored. Now the Search path inherits from the Environment options, so a lot of this was taken care of when you set #6 above.
However, I removed:
• $(BDS)\Lib,
• $(BDS)\Lib\Indy10, and
• $(BDSCOMMONDIR)\Dcp
from my Environment path and added them to the Search path for the Project. My logic was to keep the Environment path as clean as possible (this may be a misguided notion). What is important here is to have the folders that contain the Dcps in this list. Intensely aggravating is that if you are missing a folder for a Dcp, the compiler reports that the Bpl cannot be found. That is completely wrong; it is the Dcp that it can’t find. Many an hour can be irrecoverably lost trying to figure out why the compiler can’t find a Bpl. May the programmer that wrote that code have a fiery post life experience.
8. Project - Options - Resource Compiler - Directories and Conditionals inherits its settings from #7. I added $(BDS)\Dcp to this path. Oh, by the way, I delete invalid paths whenever I encountered them. The only exception is when it is clearly a typo on my part.
9. Project - Options - Debugger - Source path contains only one addition for my project (for a third party library), but this doesn’t appear to do anything since the debugger still can’t find that library’s source code (for inclusion in the call stack).
10. Okay, here is the biggest change I had to make to my code. None of this was true for Delphi 2007! The ‘uses’ clauses in each of your modules should be pristine, completely devoid of references to modules that are not needed. The compile time for my complete project (i.e., after deleting all the DCUs) ran to over 5 minutes. Once I had cleaned up all the uses clauses (360 modules, 2 uses clauses per module) the compile time for the entire project was 16 seconds.
Now these edits were extraordinarily time consuming to do and mercifully a friend told me about the freeware program Icarus from Peganza.com. That lovely little program analyzes all your files when you give it the name of your project file. Icarus’ very long report identifies unused references for each module and also identifies references that should be in the Implementation section instead of the Interface section. Now it isn’t perfect, with some false positives (i.e., it says the reference isn’t needed when it really is). But it found 99.9% of the unused references correctly for me and I figure it saved me 40-50 hours of work. Even with this automated help, it took me a long time to make all the edits in the 360 modules.
The payoff is clearly visible in the time require to compile the full project, or even just a couple of modules that you have changed. I have to believe it also improves the response time of the IDE when debugging, though I still have major problems with that.
11. I still get a fatal error whenever I try to do a Build. This crashes the IDE and requires me to use the Task Manager to terminate Delphi 2010. Though I find this bug to be astonishing, it isn’t an impediment to me getting work done. I just do a Compile of the project and Delphi creates the EXE without complaints by it or me.
12. Somewhat annoying is the 15 seconds the IDE takes the first time I click on Search from the main menu bar (or equivalently use the shortcut Control-Shift-F). After the first global search the response time is under a second, but the first one always takes ~15 seconds. I assume it is doing some preparatory work and searches through all the files in my project. This doesn’t happen with small projects. Before I made the #10 changes above, this use to take 5 minutes. There is no way to Esc from this command if you enter it by accident. You either wait for it to complete or use the Task manager to kill Delphi 2010. I have encountered this at other times too for other IDE commands. I am not sure what provokes this bizarre behavior by the IDE but whichever programmer/analyst/manager made this decision about the IDE interface deserves a job building igloos in Death Valley in the midday sun - when he completes 100 of them he can take a 5 minute break.
13. Completely unacceptable is the response time of the debugger. As you can tell by the above, I have made massive attempts to get this to function correctly. As it is, the debugger can disappear (i.e., busy cursor) for minutes at a time when executing my application. When the same program is run stand alone (outside of the debugger in XP emulation mode), the response time is instantaneous, as it was when I used the Delphi 2007 debugger. But if I execute the program from within the IDE, the program can come to an abrupt halt at virtually any time - usually very shortly after I start its execution. If I am lucky, the debugger returns from its introspection in 2-3 minutes. I have seen it take 10+ minutes and I now use the rule of thumb that if it takes 2+ minutes, I kill Delphi 2010 using the Task Manager.
14. I have implemented the small fix to the IDE by Andreas (http://andy.jgknet.de/).
I’ve spent the last 6 weeks working on this every day. Any ideas or help would be appreciated.
_____________________________
Steve
Perfection is an elusive goal.
|
Printable Version
RE: When? - 
I like the irony 
, Gee it wasnt Bo it really was Numdydar saying these things but come on Shannon just for a moment hmmmm. And you thought I was borderline wow this guy makes me look like an angel
The irony of it.
RE: When? - 
New Messages
No New Messages
Hot Topic w/ New Messages
Hot Topic w/o New Messages
Locked w/ New Messages
Locked w/o New Messages
Post New Thread