[Resolved] Concurrent Event Problem (Full Version)

All Forums >> [New Releases from Matrix Games] >> Command: Modern Operations series >> Tech Support



Message

jkgarner -> [Resolved] Concurrent Event Problem (6/6/2021 11:22:43 AM)

When I use Lua to generate multiple Time triggered events with the same execution time, ONLY the first presented to the simulation executes. All others for that time are dropped by the simulation and not executed.

I first encountered this problem several months ago and posted a thread:
Forum Thread: Unit Scripting and Execution Woes

An acquaintance suggested that the issue can be avoided by using RantomTime triggered Events instead. I said that perhaps if RandomTime trigger events use a different mechanism, but if all they do is randomly generate a Time trigger for execution, then it will only hide the issue, and collisions in randomly selected times will cause dropped events.

Yesterday in testing using RandomTime events, I had collisions in the times and dropped events. To quantify this: I had a small test scenario with only 8 units, 4 of whom needed to execute an event at the same time. All events were generated using a Lua script. In generating the RandomTime event, I allowed for random selection of execution over a 60 second range for all events. In 10 runs, 3 runs dropped a single event. 3 dropped events was too few to establish a pattern for which events were dropped. So I ran the test scenario another 40 times. After additional 40 runs, I had 13 dropped events. What I noticed was that units appearing later in the script had a greater chance of having their events dropped. The first unit in the script was never dropped. The last unit (4th unit inserted) was dropped 8 times. The 3rd: 4 times, the 2nd: 1 time. I believe this distribution is an artifact of the random generation on RandomTime events, in connection with the simulation executing the first event presented and dropping all subsequent events. My conclusion is that using RandomTime events is NOT a solution to the problem of not being able to execute concurrent Time events within the simulation.

Now I am back at the issue I brought up several months ago: How do you get the simulation to execute multiple concurrent events?

I consider the simulation's inability to process concurrent events to be a bug. It may be a design decision and intended behavior, but in my mind such a decision is limiting for a simulation. I speak now as a programmer who has worked with discrete event simulations professionally for aver 20 years. All simulations of scale must be able to handle concurrent events. Some opt to drop subsequent events, and some choose to establish a queue of events and process them in FIFO or FILO order. In my experience, choosing to drop or ignore concurrent events causes problems when scaling up the complexity or size of the data-sets executed by the simulation. Thus when presented with the problem of executing concurrent events, my advice is to queue them up. I prefer FIFO over FILO as being more defensible.

When I consider the issues with dropping Lua scheduled time triggered events, I invariably wonder how deep this issue goes into the simulation. Is this a relic of the external event insertion mechanism provided by Lua? Or does the core simulation prevent concurrent events? If the core simulation prevents concurrent events, what is the resolution of the timing mechanism? Is it seconds or milliseconds? The resolution presented by Lua and the GUI is seconds. Presenting the highest fidelity available to the user when concurrent events is not permitted gives the user the greatest amount of control to try and avoid concurrent events, though in my mind, it is not a real or valid solution.

To side-step the issue of concurrent events being dropped, I am considering tracking the number of events generated thus far during the generation process and offsetting the time of execution of all initial events by this ordinal number multiplied by times the smallest time step I am permitted to use. This means that I intend to offset the first event generated by 0 seconds, the second event by 1 second, the third by 2 and so forth. This will allow insertion and execution of all the initial events.

The real issue is that many of these initial events generate additional events during their execution and there is no mechanism to prevent the collision of the times of these events. I anticipate that collision will happen and that I will have scripted events dropped. The results are unpredictable, except to say that the more complex the scenario and the more units being automated through the scripting mechanism, the greater the chance of collisions and dropped events. Thus event dropping of concurrent events, from my perspective, is a bug.








michaelm75au -> RE: Concurrent Event Problem (6/6/2021 12:04:20 PM)

Can you supply a simple scenario showing this?



jkgarner -> RE: Concurrent Event Problem (6/7/2021 5:14:09 AM)

Mike,

Thanks for being willing to look into this. I have done some extensive testing and tracked down the issue. Multiple Time events do execute in the simulation, but ONLY if the code triggered by the event does not attempt to delete the event that triggered the code in the first place.

I have attached a file to this post that contains some test Lua code. Sorry for the size of the file, but it was necessary. To play around with what I was doing, copy the contents of the file into the lua console and execute it.

I started by writing code to simply generate concurrent time events, and executing them. This worked. The code has functions to do this-- the RunTest() function executes this portion.

I then replicated our more complex system that generated the error, and sure, enough it happened EVERY time. To see the error, edit the Start function and add in one of the event deleting mechanisms. The ClearScriptEvent is more complete in its deletion than simply calling ScenEdit_SetEvent with the remove flag. Having either call in this function causes the error to occur. Remove both and the code works like the other code.

Is this behavior intended? You all provide an isActive flag, which I could use to disable an event, but it is still in the system. It is also visible in the GUI. We generate many events, and I am concerned that the volume of events will slow down the system. For starters, it makes examining and debugging events nearly impossible.

Most simulations that I have worked with in the past allow for event deletion once the event code has executed. Some require deletion as part of the execution (often at the end, but can be at the beginning) This is done to keep resources available for processing. If they are not deleted, then they can build up and take significant resources.

Our code was structured with this behavior in mind. While we can easily remove the delete code easily enough, but I would really rather find a way to delete the events after they are executed.

Do you have any thoughts?






KnightHawk75 -> RE: Concurrent Event Problem (6/7/2021 7:24:51 AM)

quote:

but ONLY if the code triggered by the event does not attempt to delete the event that triggered the code in the first place.

quote:

but I would really rather find a way to delete the events after they are executed.

Pretty sure we discussed that in the past (http://www.matrixgames.com/forums/fb.asp?m=4973321), and this question in the past. In that thread eventually what ever you were trying to do worked out after correcting other code without my suggestion, though it sounds very similar to what your doing here. I'll suggested it again that while perhaps not ideal just mark it inactive and throw the guid of the event to delete into a table, and then run through that table at an interval of your choosing that removes all those with entries in the table. Unless you're generating\removing hundreds, or thousands per second I would think it would suffice for cleaning up resources pretty quickly if you run the clean up cycle every second.

I've used that in the past seemingly without issue, though in my case the cleanup table was more like
gKH.EventCleaner.cleaupTable = {[eventguid]={triggers=true|false,conditions=true|false,actions=true|false,locked=true|false}}, where besides removing the event I could optionally tag the associated T|C|A's to also be deleted by the cleanup cycle since at times those were specific to said event, you might need something little more involved if you have more than one T|C|A associated to the event though and want some but not other TCA entries deleted. I wasn't creating or deleting more than dozen or so a second, but I'd see if it works for you.

If during concurrent execution code in one event needs to know if itself is already flagged for deletion, it can check for cleanupTable[eventguid] being non-nil, or if needing to check existence of the event, if actually exists one could then double check with the cleanup table the same way as an added factor in that determination.



jkgarner -> RE: Concurrent Event Problem (6/7/2021 9:42:05 AM)

Thanks knighthawk75, I woke up this morning with the exact same solution in mind. Perhaps it was the subconscious remembering your words.[;)]

I have another question for you.

I currently have 2 functions to delete events, one that I wrote based upon your code: 
CleanupEvent = 
function( eventName )		
    local e,p = pcall(ScenEdit_GetEvent,eventName);
    if(e == true and p ~=nil)  then
		--remove all the actions connected to this event!
		for k,v in pairs(p.actions) do
            local aName = ''
            if (v.LuaScript ~= nil) then aName = v.LuaScript.Description 
            elseif (v.ChangeMissionStatus ~= nil) then aName = v.ChangeMissionStatus.Description 
            elseif (v.NewStatus ~= nil) then aName = v.NewStatus.Description 
            elseif (v.EndScenario ~= nil) then aName = v.EndScenario.Description 
            elseif (v.Message ~= nil) then aName = v.Message.Description 
            elseif (v.Text ~= nil) then aName = v.Text.Description 
            elseif (v.Points ~= nil) then aName = v.Points.Description 
            elseif (v.SideID ~= nil) then aName = v.SideID.Description 
            elseif (v.TeleportInArea ~= nil) then aName = v.TeleportInArea.Description 
            else aName = nil end
            if (aName ~= nil) then
				--print('disconnecting action: '..aName..'from event: '..eventName)
                ScenEdit_SetEventAction(eventName,{mode='remove',name=aName})
				--print('destroying action: '..aName)
                ScenEdit_SetAction({mode='remove',name=aName})
            end
		end
        
		--remove all the triggers connected to this event!
		for k,v in pairs(p.triggers) do
            local tName = ''
            if (v.Points ~= nil) then tName = v.Points.Description 
            elseif (v.RandomTime) then tName = v.RandomTime.Description 
            elseif (v.RegularTime) then tName = v.RegularTime.Description 
            elseif (v.ScenEnded) then tName = v.ScenEnded.Description 
            elseif (v.ScenLoaded) then tName = v.ScenLoaded.Description 
            elseif (v.Time) then tName = v.Time.Description 
            elseif (v.UnitDamaged) then tName = v.UnitDamaged.Description 
            elseif (v.UnitDestroyed) then tName = v.UnitDestroyed.Description 
            elseif (v.UnitDetected) then tName = v.UnitDetected.Description 
            elseif (v.UnitEmissions) then tName = v.UnitEmissions.Description 
            elseif (v.UnitEntersArea) then tName = v.UnitEntersArea.Description 
            elseif (v.UnitRemainsInArea) then tName = v.UnitRemainsInArea.Description 
            elseif (v.UnitBaseStatus) then tName = v.UnitBaseStatus.Description 
            elseif (v.UnitCargoMoved) then tName = v.UnitCargoMoved.Description 
            else tName = nil end
            if (tName ~= nil) then
				--print('disconnecting tigger: '..tName..'from event: '..eventName)
                ScenEdit_SetEventTrigger(eventName,{mode='remove',name=tName})
				--print('destroying tiggger: '..tName)
                ScenEdit_SetTrigger({mode='remove',name=tName})
            end
		end
	
		--remove all the conditions connected to this event!
		for k,v in pairs(p.conditions) do
            local cName = ''
            if (v.LuaScript~= nil) then cName = v.LuaScript.Description
            elseif (v.ScenHasStarted) then cName = v.ScenHasStarted.Description
            elseif (v.SidePosture) then cName = v.SidePosture.Description
			else cName = nil end
            if (cName ~= nil) then
				--print('disconnecting condition: '..cName..'from event: '..eventName)
                ScenEdit_SetEventCondition(eventName,{mode='remove',name=cName})
				--print('destroying condition: '..cName)
                ScenEdit_SetCondition({mode='remove',name=cName})
            end
		end

		--remove the event
		ScenEdit_SetEvent(eventName, {mode='remove'})
	end
end


and two written by a co-worker 
RemoveEvent = 
function(eventName)
    local event = ScenEdit_GetEvent(eventName)
    ScenEdit_SetEvent(event.guid, {mode = 'remove'})
end
and
LeMay.Event.RemoveEventAndTCAs = 
function(eventName)
    local event = ScenEdit_GetEvent(eventName)
    local eventTriggers = event.triggers
    local eventConditions = event.conditions
    local eventActions = event.actions

    ScenEdit_SetEvent(event.guid, {mode = 'remove'})

    --The first loop gets all the TCAs, the second loop gets the TCA details which is where the id is.
    for k, v in pairs(eventTriggers) do
        for key, value in pairs(v) do
            if type(value) == "table" then 
                local id = value.ID
                ScenEdit_SetTrigger({description = id, mode = 'remove'})
            end
        end
    end

    for k, v in pairs(eventConditions) do
        for key, value in pairs(v) do
            if type(value) == "table" then
                local id = value.ID
                ScenEdit_SetCondition({description = id, mode = 'remove'})
            end
        end
    end

    for k, v in pairs(eventActions) do
        for key, value in pairs(v) do
            if type(value) == "table" then 
                local id = value.ID
                ScenEdit_SetAction({description = id, mode = 'remove'})
            end
        end
    end
end


The co-worker obviously prefers his function, and with implementing your suggestion(above) I will be consolidating/removing some delete code (most likely remap all the functions to a single function that implements your suggestion.)

Talk to me about what is wrong with the implementations.

My understanding is as follows:

The RemoveEvent only deletes the event and can leave orphaned triggers, conditions, and actions. I got that. This is a bad thing.

The RemoveEventAndTCAs functions will delete the event and all TCAs connected to it in any way. This will include TCAs that may be shared with other events, which can cause them to error and abort your scripts. Clearly a bad thing.

The CleanupEvent function deletes the event and any TCAs that are connected to them AND share the same name. If the TCA that has the same name as the event happens to be used by another event, then that event will error out if called/triggered.


It seems to me that the whole area of events, trigers, conditions, and actions,is fairly complex and while it provides a great deal of flexibility, great care must be taken to keep these all straight and make sure that deletion happens correctly.

Aside from your suggestion above, which involves a recurring EventCleanup event and a global list of events for deletion, what is the best way to allow easy and safe cleanup of events (and their associared TCAs) allowing for the user to share TCAs?

I am thinking of startign with the CleanupEvent function, alter the parameter to a table and allow the user to specify the level and type of delete to perform.

{ name='eventName', delete=[eventOnly|allActions|namedActions|allTriggers|namedTriggers|allConditions|namedConditions|allTCAs|namedTCAs] }

Then based upon the delete parameter place the guid of the element on perform the appropriate delete list (event, trigger, condition, action), then every yea-often (perhaps 15 minutes) delete all the items on the delete lists..

Please let me know what you think of the plan. Thanks.



michaelm75au -> RE: Concurrent Event Problem (6/7/2021 10:15:02 AM)

The basic event handling each cycle looks at all the triggers to determine which ones are activated.
It then goes thru the event table and executes each event that was triggered from above.
If the action of one of the triggered events is to remove another triggered event, it might not be present in the event table when it continues checking the table (as it may have been removed).
Although I would have thought the 'for' loop would have got an exception due to the number of items in the table being changed.

With the example I may spent some time on seeing what actually happened.[&:]



jkgarner -> RE: Concurrent Event Problem (6/7/2021 10:32:21 AM)

Thanks Micheal.

In my code, each event had its own trigger defined (with its own name!) Unless the system consolidates triggers in some way.. (e.g. I have a new time trigger that I am trying to define that happens to have the same time as another trigger, so for efficiency sake, the system puts the action of the second on the action list of the first and with the intent of then using the first and dropping the second, then deleting the first then also delete the action list including the second action) I do not see how that would be the case. But your are the expert with visibility into the code-base, so if you have time, please take a look at this. The code absolutely generates the condition on demand.

My expectation is that events with different defined triggers (regardless of whether or not they are the same time as another trigger) should remain independent. Deleting the first should NOT impact the second in any way. I coded my Lua with this thought.

I suspect some unintended behavior is involved.

FYI: KnightHawk75 suggested a Lua work-around, which I will probably implement because any change you make will not be in place until sometime in the Fall or later. I need this to work tomorrow.

I look forward to your analysis.





michaelm75au -> RE: Concurrent Event Problem (6/7/2021 10:56:59 AM)

Weird. I must be doing something wrong as this is working for me.
I took your script and uncommented line
--ScenEdit_SetEvent(event.guid, {mode = 'remove'})
And ran the script.
There are 4 events in editor and all were triggered. The units are moving.


[image]local://upfiles/3086/77F23DCD016C463CB2E8CAC85919DD9E.jpg[/image]



KnightHawk75 -> RE: Concurrent Event Problem (6/7/2021 10:56:38 PM)

quote:

The RemoveEvent only deletes the event and can leave orphaned triggers, conditions, and actions. I got that. This is a bad thing.

Yup.

quote:

The RemoveEventAndTCAs functions will delete the event and all TCAs connected to it in any way. This will include TCAs that may be shared with other events, which can cause them to error and abort your scripts. Clearly a bad thing.

yup.

quote:

The CleanupEvent function deletes the event and any TCAs that are connected to them AND share the same name. If the TCA that has the same name as the event happens to be used by another event, then that event will error out if called/triggered.

yup, which is why where possible I'd suggest using event's guid, and TCA guids, for debugging you may want to include the name's in your table but where you can use guids.

quote:

It seems to me that the whole area of events, trigers, conditions, and actions,is fairly complex and while it provides a great deal of flexibility, great care must be taken to keep these all straight and make sure that deletion happens correctly.

Nodders... yup need them to happen and in right order (like best to disassociate TCA's from mainevent before deleting them and then at the end wipe the main event), and agree even though it's added bloat from delete-management perspective having everything have it's own instance of a T|C|A is often a good thing. That said TCA's that will never get deleted and only removed by all means share where you can.

quote:

Aside from your suggestion above, which involves a recurring EventCleanup event and a global list of events for deletion, what is the best way to allow easy and safe cleanup of events (and their associared TCAs) allowing for the user to share TCAs?

Got me. If there is better way that is generally applicable idk, but that doesn't mean there isn't one. ;)

quote:

I am thinking of startign with the CleanupEvent function, alter the parameter to a table and allow the user to specify the level and type of delete to perform.

{ name='eventName', delete=[eventOnly|allActions|namedActions|allTriggers|namedTriggers|allConditions|namedConditions|allTCAs|namedTCAs] }

Then based upon the delete parameter place the guid of the element on perform the appropriate delete list (event, trigger, condition, action), then every yea-often (perhaps 15 minutes) delete all the items on the delete lists..


That sounds like a good plan.
So I was thinking more like each entry in the CleanupTable looked like:
[eventguid] ={name="Event name here for debug purposes",Ts={TCAEntries} Cs={TCAEntries} As={TCAEntries}}
Each TCAEntry looked like:
TCAEntry = {[1]={guid='guidhere',name='dbg purposes'},[2]=...thenextone...}
In the clean up routine it can then just for-through em disassociate, and then delete only what's specified. 
--just pseudo code this ain't real and shows no error checking or mundane tasks.
function ProcessCleanUpCycle()
for g,entry in pairs(cleanupTable) do
   print(stringformat("Cleaning up %s with name %s",));
   curEvent = ScenEdit_GetEvent(g); 
   t = GetAllEventTriggers(curEvent)
   for _,t do RemoveTriggersFromEvent(curEvent,t); DeleteTriggers(entry.Ts); end
   c = GetAllEventConditions(curEvent)
   for _,c do RemoveConditionsFromEvent(curEvent,c); DeleteConditions(entry.Cs); end
   a = GetAllEventActions(curEvent)
   for _, a do RemoveActionsFromEvent(curEvent,a); DeleteActions(entry.As); end
   ScenEdit_SetEvent(guid,{Mode='remove'});
   print(string.format("Event  %s with name %s has been removed",tostring());
end

--just pseudo code this ain't real and shows no error checking or mundane tasks.
function GetAllEventTriggers(theEvent)
  local tbl = {};
  for every trigger listed in the event do
    table.insert(tbl,{guid=trigger.guid,name=trigger.description});
  end
  return tbl;
end
--pseudo sample
RemoveTriggersFromEvent(theEvent,theTriggers)
  for _,t in ipairs(theTriggers) do
     print(stringformat('Removing Trigger %s from event %s',tostring(t.name),tostring(theEvent.description))); --dbg
     ScenEdit_SetEventTrigger(theEvent.guid,{description=t.guid,Mode='remove'}
  end
end
 --pseudo sample
function DeleteTriggers(theEvent,TCAEntries)
  for _,tca in ipairs(TCAEntries) do
     print('Deleting Trigger: ' .. tostring(tca.name) ) --dbg
     ScenEdit_SetTrigger({description=tca.guid,Mode='remove'});
  end
end
--etc...

In the above though when you want to delete something you specify up front whats to be deleted, anything not specified is retained though everything is disassociated with the event to be deleted. I only include action= incase there is some other thing you might want to do besides delete otherwise it's not needed.

But I don't see anything wrong with your proposed way too, whatever is most comfortable and works best for you, and I could be wrong but it's probably a ~wash in either way's overhead and amount of code (mostly not shown) that has to be written.



KnightHawk75 -> RE: Concurrent Event Problem (6/7/2021 11:03:02 PM)

quote:

ORIGINAL: michaelm75au

Weird. I must be doing something wrong as this is working for me.
I took your script and uncommented line
--ScenEdit_SetEvent(event.guid, {mode = 'remove'})
And ran the script.
There are 4 events in editor and all were triggered. The units are moving.


Yes I got that too... with only uncommenting that line.

BUT if you leave that commented and uncomment the line a few above it CleanupScriptEvent(sName) that basically does the same thing, you should get only the first event firing, along with the following in the exception log
for modifying the list during enumeration.
quote:


6/7/2021 6:53:26 PM -- B1147.25 -- Collection was modified; enumeration operation may not execute.
Exception: Collection was modified; enumeration operation may not execute.
Stack Trace: at System.ThrowHelper.ThrowInvalidOperationException(ExceptionResource resource)
at System.Collections.Generic.List`1.Enumerator.MoveNextRare()
at System.Collections.Generic.List`1.Enumerator.MoveNext()
at Command_Core.Scenario.(List`1 )
Call Stack & Error details:
Error at 101028,

It is weird that a remove in the 'Start' function doesn't cause the exception, but does if outsourced to another, something I'm (or we're all missing) about this. hmm



KnightHawk75 -> RE: Concurrent Event Problem (6/7/2021 11:12:30 PM)

@JKgarner - unrealted to this issue, in the script there are a couple of lines that have isActive=active instead of isActive=true. Just throwing it out there as fyi, not that correcting it made any difference for the topic at hand.




michaelm75au -> RE: Concurrent Event Problem (6/7/2021 11:31:51 PM)


quote:

ORIGINAL: KnightHawk75

quote:

ORIGINAL: michaelm75au

Weird. I must be doing something wrong as this is working for me.
I took your script and uncommented line
--ScenEdit_SetEvent(event.guid, {mode = 'remove'})
And ran the script.
There are 4 events in editor and all were triggered. The units are moving.


Yes I got that too... with only uncommenting that line.

BUT if you leave that commented and uncomment the line a few above it CleanupScriptEvent(sName) that basically does the same thing, you should get only the first event firing, along with the following in the exception log
for modifying the list during enumeration.
quote:


6/7/2021 6:53:26 PM -- B1147.25 -- Collection was modified; enumeration operation may not execute.
Exception: Collection was modified; enumeration operation may not execute.
Stack Trace: at System.ThrowHelper.ThrowInvalidOperationException(ExceptionResource resource)
at System.Collections.Generic.List`1.Enumerator.MoveNextRare()
at System.Collections.Generic.List`1.Enumerator.MoveNext()
at Command_Core.Scenario.(List`1 )
Call Stack & Error details:
Error at 101028,

It is weird that a remove in the 'Start' function doesn't cause the exception, but does if outsourced to another, something I'm (or we're all missing) about this. hmm


That exception is what I would have expected if the event table was modified during execution.
I'll have a closer look at what that script is doing, but using the 'remove' itself seems to be okay. I thought that was suppose to give the same error



jkgarner -> RE: Concurrent Event Problem (6/8/2021 2:11:46 AM)

Micheal,

Last night I was running on PE 1.15.5. The code errors as described.
It also errors on 1.15.3.

Today, I had a chance to run the code on PE 2. It did not error at all.

Just throwing this out there for you to consider.





michaelm75au -> RE: Concurrent Event Problem (6/8/2021 2:18:19 AM)

I didn't realize you were using PE 1. There was some code creep between CMO and PE 1. It should be better now between CMO and PE2.



jkgarner -> RE: Concurrent Event Problem (6/8/2021 2:25:39 AM)

I think the best solution is to have every event have its own unique trigger, its own unique condition, and its own unique action. A true one-to-one relationship between events and triggers and beteween events and actions. And a zero or one-to-one relationship betweeen conditions and events. Then go the next step and make all share the same name. This will allow for name checking upon deletion, and when deleting, delete the linked TCAs gor extreme safety (as in the CleanupScriptEvent function), should the rule be vioolated. But in truth simply deleting all connected TCAs ( as in the RemoveEventAndRTCAs function would be sufficient under this strict regime. as in the RemoveEventAnd RCAs method.

Some may say that this regime is too restrictive and I need to reuse the script code... TO this I say, the script code should be kept extremely simple, as in a single function call. Move all the complexity into a Lua Code library where it can be properly managed under source control, and simply call the appropriate function in the event action. When the complex functions are part of a library (instead of being embedded in an event, then you can create unit test functions to exercise the function and prove its correctness in automated tests.

The goal is code safety. No errors. If errors crop up, then they are handled so the system does not abort.

Thank you both for your insights.



jkgarner -> RE: Concurrent Event Problem (6/8/2021 2:36:20 AM)

FYI: we have not migrate to PE 2, yet. But will shortly.

The question I have is how many events can exist in the system before we start seeing degredation i performance.

To be certain, there are other factors. As in the complexity of events, and the number and complexity of units and their interactions as well as the desired run speed.

Say I have a scenario with 4000 units (this includes all type, including complicated facilities and groups, such as air bases) and the possibility of easily 1000 events being generated over the course of each day for several days: how many days until I see performance hits in this setup? 3? 5? 14?





KnightHawk75 -> RE: Concurrent Event Problem (6/8/2021 12:31:22 PM)

quote:


That exception is what I would have expected if the event table was modified during execution.
I'll have a closer look at what that script is doing, but using the 'remove' itself seems to be okay. I thought that was suppose to give the same error


So I spent the night digging deeper.
Ok from stepping though some of the code the problem is not deleting the event from the observable dictionary.
The problem isn't even with removing and deleting the T and C's because they get copies or shallow copies as part of the process that work allow deletes on those to work - either by design or maybe luck of implementation.
The problem is with the Action removals and deletes because the lists far as I can tell and are not operated on as copies, thus the originals are being ref'd and enumerated using a .net foreach while things get turned over to LuaScript code.

So the processing appears to go something like this in pseudo (.net) 
function ObfuscatedToHell_u001e (list<EventTrigger> Arg_1){
 try{
  --Arg_1 it would appear at a glance is a separately made copy\list.
  try{foreach(EventTrigger in Arg_1) {
    list<SimEvent> copyOfSimEventsAsList = ObservableDicSimEvents.Values.ToList() 
     --^this is a shallow copy but allows event deletion as list itself isn't changed just underlying ref'd content?
    try{ foreach(simevent in copyOfSimEventsAsList){
      -- if simEvent active proceed.. set reset Lua EventX var to this event.
      -- if simEvent.triggers?.contains(EventTrigger) ...proceed...
      -- processrandomnness setting if set... if set and grab random > set pct ...proceed... otherwise continue to next iteration.
      -- simEvent.ProcessConditions?(ScenarioContext) if end result is true proceed....  otherwise continue to next iteration
      --boatload of code to setting up some var and what not depending on the 'type' of trigger, ie UnitX\Y\C info. 

        try{ foreach(eventAction eventAction in simEvent.Actions){ 
           eventAction.RunAction?(scenariocontext,simEvent)
          -- YourCodeExecutingHere to delete this trigger and action it calls ScenEdit_SetEventXTCAX(remove) and ScenEdit_SetXTCAX(delete). 
          -- Those calls best I can tell operate on the original observabledics and they technically succeed.
          -- the action enumeration here is really tied to the original even though it looks like a copy.
          <--exception thrown here ( or at the top if you prefer, during next attempted iteration) 
           -- as the original Dictionary internal version 1 has changed to versionstamp 2 now.
          }
          continue; 
        }
        finally{dispose of related foreach enumerator actions} <--executes
      }
    }
    finally{dispose of related foreach enumerator simevents} <--executes
  } 
  finally{dispose of related foreach enumerator eventTriggers};<--executes 
 }
 catch(){ master catch, go log the exception} <--executes
}

Result:
It seems the incoming eventtriggers is a copy, which doesn't prevent removing or deleting triggers themselves.
The copyOfSimEventsAsList copy that is made is shallow (I think?) but still ok because the copy's item count isn't changing, just the underlying by ref'd object are changing so it doesn't prevent removing Events themselves.
But when it comes to actions enumeration it's indirectly (via shadowcopy ref) directly enumerating the original while the actions get called to make a delete on the same copy which changes the version stamp when calling SetEventAction inside the action, and movenext() sees that version change and balks.

Way's to correct such that one could delete "itself" while running that come to mind that could be tried:
1. Devs change that action foreach to operate on more indirect\byval'ish so to speak clone\copy of the list of actions taken just before the foreach. In theory this would make it act like the others. no? Also would allow the user to add and associate new actions to the exiting action during it's own execution that would only be seen in a future processing cycle. Downside...i mean if someone has a dozens+ actions tied to this single event, and each action is some large say 25k+ of lua code...probably lots of memory overhead to make a copy of all that. But how then again how often is that gonna happen?
2. Don't use a foreach() for the enumeration of the list, use a for() based in theory, this might present other challenges though. hmm

3. User never removes or deletes an 'action' from with-in any event already tied to this same action. Uses aforementioned delayed cleanup system via another dedicated event.

4. User records the TCA guids they want to delete first. Removes and Deletes the T's and C's from inside the action, then the Event itself, but defers just the deletion of the A's to the cleanup system. See updated attached working sample.

5. ?? some tweak or other magic ;)

Attached is an update version of JK's sample with my changes, updated to work with-in the system atm (in retail 1147.25 anyway).




jkgarner -> RE: Concurrent Event Problem (6/9/2021 10:42:41 AM)

Thanks KnightHawk75.
[&o]

I will most likely lift this solution (with credit and somewhat modified) to use while I am compelled to use version 1 and change.




Dimitris -> RE: Concurrent Event Problem (6/10/2021 8:41:51 AM)

Moving this to Tech Support as it describes a potential bug.



michaelm75au -> RE: Concurrent Event Problem (6/10/2021 9:35:20 AM)

A fix to cater for 'deleting an action with an action' has been made for the next build.



KnightHawk75 -> RE: Concurrent Event Problem (6/12/2021 12:38:14 AM)

quote:

ORIGINAL: michaelm75au

A fix to cater for 'deleting an action with an action' has been made for the next build.


Massive THANK YOU!! Dimitris,Michael, and JKgarner for finding it, fixing it and extensive examples that highlighted it and made it easier to track down.


quote:


Thanks KnightHawk75.
I will most likely lift this solution (with credit and somewhat modified) to use while I am compelled to use version 1 and change.

No problemo, no need for "credit", and by all means adapt as needed as it was just a sample.



jkgarner -> RE: Concurrent Event Problem (6/14/2021 11:48:20 AM)

@KnightHawk75:

I decided that implementing a delayed event deletion rather than the delayed action delete was a cleaner solution in that it results in less code, and is more easily understood.

Thanks for your help in identifying the actual problem.




Page: [1]

Valid CSS!




Forum Software © ASPPlayground.NET Advanced Edition 2.4.5 ANSI
1.140625