Matrix Games Forums

Forums  Register  Login  Photo Gallery  Member List  Search  Calendars  FAQ 

My Profile  Inbox  Address Book  My Subscription  My Forums  Log Out

SANDMAN - a fatigue/effectiveness modeling framework

 
View related threads: (in this forum | in all forums)

Logged in as: Guest
Users viewing this topic: none
  Printable Version
All Forums >> [New Releases from Matrix Games] >> Command: Modern Operations series >> Mods and Scenarios >> SANDMAN - a fatigue/effectiveness modeling framework Page: [1]
Login
Message << Older Topic   Newer Topic >>
SANDMAN - a fatigue/effectiveness modeling framework - 2/9/2022 11:54:51 PM   
musurca

 

Posts: 128
Joined: 7/16/2020
Status: offline
Hi everyone. After reading comments on the CMO Discord about the lack of pilot fatigue (@KJohnston: "I just think to myself that the E-3 operators have a mini fridge full of red bull or something"), I went down a bit of a rabbit hole wondering what it might look like to model fatigue in a plausible, exhaustive way—and ended up creating this framework for adding fatigue to your scenarios, which I call SANDMAN.

DOWNLOAD SANDMAN v0.1.1

SANDMAN simulates the current fatigue level of your pilots using a realistic approach derived from the SAFTE model (see Hursh, et al. in the bibliography) based on their sustained activity, sleep debt, and circadian rhythms, then degrades their proficiency accordingly. You may also see exhausted pilots taking short, dangerous "micronaps" in which their plane goes out of comms, and becomes vulnerable to enemy action. Extremely exhausted pilots may even begin to bolter/go-around on landings, or even (under very, very rare conditions) crash.

You can monitor the effectiveness of your pilots, and attempt to stand down particularly exhausted pilots for rest, using the Fatigue Avoidance Scheduling Tool (available from the Special Actions menu). Note that leaving planes "parked" will be the most effective way to rest your pilots. The pilots of "readying" planes will still rest somewhat, but very ineffectively.

While major penalties to pilot effectiveness will mostly occur in longer scenarios (24hrs+), you will always see effectiveness vary +/- 7-16% based on circadian rhythms during the daily cycle, making the time of day at which you launch operations a significant factor in planning. (In this circadian model, alertness peaks at 2000hrs, and reaches a minimum at 0300hrs.)

How to install into your scenarios:

1) Download the latest release.
2) Open your scenario in the Scenario Editor.
3) Go to Editor -> Lua Script Console
4) Paste the contents of the file sandman_min.lua into the white box, then click Run.
5) Complete the SANDMAN wizard and voila! You now have sleepy pilots.

Hope you find it useful! I've also put the source code up on GitHub if anyone wants to contribute/make a pull request to the framework. In particular, the risk functions I wrote for crashes and micronaps are more speculative; if anyone has better insight or hard data on those subjects it would be very helpful.

Thanks to @stww2, @Fido81, and @Tempest for contributing to the discussion as I was developing this, as well as @rbsj, whose thread on fatigue-related crashes during carrier landings was an excellent starting point.



Bibliography:

BRANFORD et al. Biomathematical Fatigue Models Guidance Document. Civil Aviation Safety Authority (CASA) Australia, March 2014.

MILLER & MELFI. Causes and Effects of Fatigue in Experienced Military Aircrew. Air Force Research Laboratory, Jan 2006.

HURSH et al. Fatigue models for applied research in warfighting. Aviat Space Environ Med 2004; 00(3, Suppl.):A000 – 00.

ROMA et al. Flight Attendant Work/Rest Patterns, Alertness, and Performance Assessment: Field Validation of Biomathematical Fatigue Modeling. Federal Aviation Administration/Office of Aerospace Medicine, Sept 2012.

< Message edited by musurca -- 2/11/2022 4:22:03 AM >
Post #: 1
RE: SANDMAN - a fatigue/effectiveness modeling framework - 2/10/2022 12:42:29 AM   
stww2

 

Posts: 219
Joined: 5/23/2017
Status: offline
quote:

what it might look like to model fatigue in a plausible, exhaustive way
I see what you did there!


(in reply to musurca)
Post #: 2
RE: SANDMAN - a fatigue/effectiveness modeling framework - 2/10/2022 9:26:50 AM   
RockPaperScissors


Posts: 27
Joined: 10/2/2016
Status: offline
Thank you ... this is a similar approach to fatigue modelling as in the (excellent) SimEx scenario(s) ... looking forward to enriching scenarios with this.

KR, RPS

(in reply to stww2)
Post #: 3
RE: SANDMAN - a fatigue/effectiveness modeling framework - 2/10/2022 3:51:02 PM   
BDukes

 

Posts: 1695
Joined: 12/27/2017
Status: offline
Wow nice add. Another step toward longer game models.

Mike

_____________________________

Don't call it a comeback...

(in reply to RockPaperScissors)
Post #: 4
RE: SANDMAN - a fatigue/effectiveness modeling framework - 2/10/2022 6:19:23 PM   
Gunner98

 

Posts: 5508
Joined: 4/29/2005
From: The Great White North!
Status: offline
This looks interesting. I haven't downloaded it or played with it yet but have a couple questions:

-Is this meant for the scenario designer to setup as default in the scenario or for a player to add if desired?
-I suppose a designer could just auto run the script at scenario start
-If for the designer, a Special Action might be nice so a player can turn the feature on/off at the start of the game
-Can a designer set a fatigue level for the scenario to start at? Is this customizable? I.e. Sqn XXX has been in the fight for 3 days and is bagged but Sqn YYY is just arriving and is fresh.
-Can/are spare pilots factored in? I know that many nations have about 1.2 to 1.4 pilots (bit of a guess but I think its right) per airframe in active Sqns

this looks like a great addition - says a guy who fields regular complaints of overly complex scenarios already ...

Thank you

_____________________________

Check out our novel, Northern Fury: H-Hour!: http://northernfury.us/
And our blog: http://northernfury.us/blog/post2/
Twitter: @NorthernFury94 or Facebook https://www.facebook.com/northernfury/

(in reply to BDukes)
Post #: 5
RE: SANDMAN - a fatigue/effectiveness modeling framework - 2/11/2022 5:00:30 AM   
musurca

 

Posts: 128
Joined: 7/16/2020
Status: offline
Just put out v0.1.1, which includes some small fixes and adds the following Lua API for scenario designers:

-- disable/enable SANDMAN and all fatigue tracking
Sandman_Disable()
Sandman_Enable()

-- set the sleep deficit for a particular unit to a random value between min_hrs and max_hrs
Sandman_SetRandomSleepDeficit(guid, min_hrs, max_hrs)

-- get the current effectiveness for a particular unit as a number [0-1]
Sandman_GetEffectiveness(guid)

-- get the current hourly risk level for a particular unit as a number [0-1]
Sandman_GetCrashRisk(guid)
Sandman_GetMicroNapRisk(guid)


quote:

-Is this meant for the scenario designer to setup as default in the scenario or for a player to add if desired?

Both. When you run that script in the editor, it creates all of the necessary Events and Actions, then injects its own code into the scenario so that it remains persistent after save. So if you’re a designer, you just install it once toward the end of the scenario creation process and that’s it. (In other words, no need to run the script every time on load. IKE users will be familiar with this process already.)

If you’re a player, you can just do the same thing with any existing scenario; install the script, save the scenario, and you’ll be good to go.

Note that the framework only knows about the planes that were in the scenario when it was first run, so if you add or remove planes, you'll want to install it again.

quote:

-If for the designer, a Special Action might be nice so a player can turn the feature on/off at the start of the game

That's a good idea. In this release, I added a Lua function for use by scenario designers to disable SANDMAN (see above, Sandman_Disable() ), so that you can create your own Special Action to give players the option to do so.

quote:

-Can a designer set a fatigue level for the scenario to start at? Is this customizable? I.e. Sqn XXX has been in the fight for 3 days and is bagged but Sqn YYY is just arriving and is fresh.

Yes and yes, although it's not yet as customizable as I'd like. When you install the script, it runs a wizard asking you if you want to customize the default values. You can do so, and set both the MINIMUM and MAXIMUM number of hours that pilots have been awake when the scenario starts.

Right now these values are applied to all pilots from all sides. In the future, the wizard will ask you to customize this by side. I added another Lua function for scenario designers so that you can manually set starting effectiveness levels for individual pilots (see above, Sandman_SetRandomSleepDeficit(guid, min_hrs, max_hrs) ), if you want to make one squadron fresher than another.

quote:

-Can/are spare pilots factored in? I know that many nations have about 1.2 to 1.4 pilots (bit of a guess but I think its right) per airframe in active Sqns

Not yet, but I'd like to add this—perhaps by creating a pool of reserve pilots that can be swapped in to replace exhausted pilots once they're on the ground, via a Special Action. If you or anyone else has more information about how this is done, which types of units tend to have spare pilots, etc., it would be very useful.

quote:

says a guy who fields regular complaints of overly complex scenarios already

Ha -- yep, this framework is definitely meant for a very particular sort of person!

(in reply to Gunner98)
Post #: 6
RE: SANDMAN - a fatigue/effectiveness modeling framework - 2/11/2022 11:28:28 AM   
Gunner98

 

Posts: 5508
Joined: 4/29/2005
From: The Great White North!
Status: offline
OK this looks great, I need to give it a run this weekend.

B

_____________________________

Check out our novel, Northern Fury: H-Hour!: http://northernfury.us/
And our blog: http://northernfury.us/blog/post2/
Twitter: @NorthernFury94 or Facebook https://www.facebook.com/northernfury/

(in reply to musurca)
Post #: 7
RE: SANDMAN - a fatigue/effectiveness modeling framework - 2/11/2022 9:13:25 PM   
AndrewJ

 

Posts: 2318
Joined: 1/5/2014
Status: offline
This looks like a great addition to long multi-day scenarios.

The circadian rhythm alertness variation sounds interesting.

Do the high and low points of alertness, in Zulu time, vary depending on the base location of the aircraft? For example, an aircraft based in one hemisphere being alert when one on the opposite hemisphere is not?

If an aircraft re-bases a significant distance, such as intercontinental squadron deployments, will the planes (gradually?) adopt the alertness times of their new base?



(in reply to Gunner98)
Post #: 8
RE: SANDMAN - a fatigue/effectiveness modeling framework - 2/11/2022 9:54:21 PM   
musurca

 

Posts: 128
Joined: 7/16/2020
Status: offline
quote:

Do the high and low points of alertness, in Zulu time, vary depending on the base location of the aircraft? For example, an aircraft based in one hemisphere being alert when one on the opposite hemisphere is not?

If an aircraft re-bases a significant distance, such as intercontinental squadron deployments, will the planes (gradually?) adopt the alertness times of their new base?

Not yet. Right now, when installing SANDMAN, the user/scenario designer is asked to calibrate the local time zone by positioning the camera. That local time is then used for the circadian rhythms of all pilots. For a scenario in which you're using a spread of time zones, you could pick one in the middle. You could also pick a timezone on the other side of the world if you're dealing with rapidly redeployed pilots who haven't adjusted yet.

In the next release, I'm planning to do what you're suggesting: get the timezone by base location, track the pilot's starting internal circadian clock, and slowly shift it over time. Keep in mind that you probably won't see significant shifts in a pilot's circadian rhythm over the course of a CMO scenario, even if it's on the longer side. With regard to shifting circadian rhythms, "a phase advance (eastward time change) takes about 1.5 d per hour of shift" (Hursh)—which explains why jetlag is so difficult to beat.

< Message edited by musurca -- 2/12/2022 5:37:38 PM >

(in reply to AndrewJ)
Post #: 9
RE: SANDMAN - a fatigue/effectiveness modeling framework - 2/11/2022 10:46:52 PM   
AndrewJ

 

Posts: 2318
Joined: 1/5/2014
Status: offline
quote:

With regard to shifting circadian rhythms, "a phase advance (eastward time change) takes about 1.5 d per hour of shift" (Hursh)—which explains why jetlag is so difficult to beat.


That slow? My (limited) experience with five hour trans-Atlantic time shifts was that, provided I stayed active in the day, got as much sunlight as possible, and slept once it was dark, I could reset and operate normally within a couple of days. (Although maybe my body temp cycle hadn't truly caught up, but honestly, I wasn't checking that. )

Regardless, a very interesting addition, which would reduce the temptation to send just one more tanker to those picket fighters...



Edit: Would you actually need to know the specific time zones? Or would it be more biologically representative to pick up the effective "solar time" from the lat/long of the base?

< Message edited by AndrewJ -- 2/11/2022 10:52:20 PM >

(in reply to musurca)
Post #: 10
RE: SANDMAN - a fatigue/effectiveness modeling framework - 2/12/2022 8:22:50 AM   
musurca

 

Posts: 128
Joined: 7/16/2020
Status: offline
quote:

That slow? My (limited) experience with five hour trans-Atlantic time shifts was that, provided I stayed active in the day, got as much sunlight as possible, and slept once it was dark, I could reset and operate normally within a couple of days.

I envy you! That figure in the literature is prefaced with "in general," so there is an amount of individual variation; also, people like me—for whom jetlag is like, as William Gibson put it, "leaving the soul behind"—likely drag down the average. Beyond that, apparently light exposure, as well as “sleep, activity, and social cues” accelerate the process of adjustment, so you’re doing all the right things. The model likely accounts for a general case where the subject is not so conscientious about the adjustment.

quote:

Edit: Would you actually need to know the specific time zones? Or would it be more biologically representative to pick up the effective "solar time" from the lat/long of the base?

This is an interesting question, and from the literature, I gather that there’s no simple answer. Ultimately, the “the peak of the circadian rhythm has a reliable relationship to the timing of the period of wakefulness” (Hursh), which would tend to converge on time zone (sometimes called the "social clock") and/or shift-work schedule as opposed to the "sun clock". However, phase shifts are “induced primarily by light exposure rather than by the displacement of sleep or activity” (Czeisler). Switching to a night shift without light exposure by just forcing yourself to wake up in the dark, for example, "may take 2.6 times as long to complete as a comparable transmeridian shift in phase” (Hursh).

The reality is that I’ll probably have to account for both time zone (for initial setting) and length of day by latitude & time of year (to calculate rate of phase shift) as this model becomes more sophisticated. However, after playing around a bit with the CMO map tonight, I think the existing time zone model as reflected in the "local time" is simpler than I thought it was, and is just a function of longitude, without any of the complicated artificial divisions—which is easy enough to replicate.

(in reply to AndrewJ)
Post #: 11
RE: SANDMAN - a fatigue/effectiveness modeling framework - 2/12/2022 12:01:56 PM   
Parel803

 

Posts: 579
Joined: 10/10/2019
From: Netherlands
Status: offline
Impresive
regards GJ

(in reply to musurca)
Post #: 12
RE: SANDMAN - a fatigue/effectiveness modeling framework - 2/12/2022 3:43:44 PM   
boogabooga

 

Posts: 457
Joined: 7/18/2018
Status: offline
I messed around with version v0.1.0. A few thoughts:

-Why code on one line?

-There was a run through where every single plane that I sent back to base crashed, regardless of the airborne time. Perhaps you have a flag somewhere that needs reset?

-I'm fairly confided that usually, air units have more flight crew than airframes. Therefore, I don't think that you need to permanently track the fatigue with the airframe. Rather, when a plane returns to base, I would assume that the flight crew is replaced by a "fresh" one with the same fatigue level as the other parked A/C.

-I recommend that you take into account the aircraft size and number of crew. Very large aircraft who are going on long-range or long-loiter missions may even have a backup crew with sleeping quarters. Or, they might have the pilot and copilot take turns on rest cycles. But, it will have a lot more resources available for fatigue mitigation than say, a lone single-seat fighter.

-Perhaps the penalties on single seat fighters that have been airborne for say, 14+ hours should be even worse somehow. A "seriously, no human can function this long" level of fatigue.

-Are you being careful to exclude UAVs?


(in reply to Parel803)
Post #: 13
RE: SANDMAN - a fatigue/effectiveness modeling framework - 2/12/2022 6:52:51 PM   
musurca

 

Posts: 128
Joined: 7/16/2020
Status: offline
quote:

-Why code on one line?

This code is "compiled" (i.e. minified) both to reduce the scenario file size, and to discourage trying to modify it in place, as this will not work due to the way the code is duplicated and converted to an escaped string for re-injection back into the scenario via the wizard.

The best way to play around with the source is to clone the GitHub repository (where the source is nicely laid out in separate files with comments) and set up a build environment as laid out in the README.

(And that way, if you fix a bug or contribute to the model, you can easily submit a "pull request" and I can merge your changes back into the repo for everyone. It would be great for this to be a community-authored fatigue model!)

quote:

-There was a run through where every single plane that I sent back to base crashed, regardless of the airborne time. Perhaps you have a flag somewhere that needs reset?

I think this is fixed in v0.1.1 but let me know if you see it happen again. Did you see a special message about it (to the effect of "crashed while attempting to land") or did they just run out of fuel?

The chance of a crash, even with extremely exhausted pilots, should be so minuscule that you should almost never see it happen. Under certain conditions the built-in math.random() may return wacky results, and in the PBEM mod I actually replaced it with a Lua reimplementation to make it perform more reliably. I was trying to avoid cluttering this module with such things, but will import that code as well if this pops up again.

quote:

-I'm fairly confided that usually, air units have more flight crew than airframes. Therefore, I don't think that you need to permanently track the fatigue with the airframe. Rather, when a plane returns to base, I would assume that the flight crew is replaced by a "fresh" one with the same fatigue level as the other parked A/C.

It depends on the situation, I think. Sometimes this is true and sometimes not. As Gunner wrote above, "many nations have about 1.2 to 1.4 pilots (bit of a guess but I think its right) per airframe in active Sqns." The main thing is that you almost never have as much manpower as you'd really like.

In a future release, I'm working on a way for the scenario designer to specify the numbers of reserve pilots, and either swap them out automatically upon return or else manually choose which pilots should be refreshed and which shouldn't. Any data you can find on this would be extremely helpful!

quote:

-I recommend that you take into account the aircraft size and number of crew. Very large aircraft who are going on long-range or long-loiter missions may even have a backup crew with sleeping quarters. Or, they might have the pilot and copilot take turns on rest cycles. But, it will have a lot more resources available for fatigue mitigation than say, a lone single-seat fighter.

True. Unfortunately as far as I know, ScenEdit_QueryDB() doesn't yet support the 'Aircraft' part of the DB, so there's no easy way to automatically pull the number of crew members for an aircraft type out of CMO. I can think of some awkward ways to pull this out of the DB files and maintain them in the code, but it would have to be done again for every DB update. (Possibly data for only the most common 'large aircraft', which would be resistant to DB updates, could be stored.) For the moment the best way to do this is probably to give the scenario designer more functions to flag multicrewed units.

You're right that it makes sense that a multicrew airframe should be more stable in terms of circadian effectiveness swings, and somewhat more resistant to micronaps. For large aircraft with a backup aircrew on board, it might even make sense to simulate it with a shift-based model (see below). I'm not entirely sure, however, that any rest those crews would be getting in the air would be significantly restorative. The literature seems to suggest that the quality of sleep really matters in this regard. But again, any data that can be scraped here is helpful.

quote:

-Perhaps the penalties on single seat fighters that have been airborne for say, 14+ hours should be even worse somehow. A "seriously, no human can function this long" level of fatigue.

This is a good point, but since humans CAN stay in the air this long (and have—see records of solo flights, or the Black Buck bombing runs), I think the way to handle this is to find some data on the additional effectiveness degradation due to considerations of 'discomfort' of being trapped in the same position for hours—the awkwardness of using the bathroom, cramps, etc.—and see if it's significant in comparison to the effects of fatigue.

quote:

-Are you being careful to exclude UAVs?

No, as even "unmanned" craft are still manned by the people who monitor them, who are subject to the effects of fatigue. Again, I think this depends on the situation—and someone please jump in here if you have some data on this—but UAVs tend to be manned in shifts with multi-person crews. In ideal circumstances, where manning levels run close to 100%, these shifts would be short and crews would be able to rest—but the literature I've read on this suggests that manning levels rarely reach ideal levels, so fatigue would become a significant factor in a crisis.

The solution here is probably to develop an additional shift-based fatigue system that can be enabled for particular aircraft. I was already thinking of moving in this direction for future updates, by accounting for watches if fatigue were to be extended to naval units.

(in reply to boogabooga)
Post #: 14
RE: SANDMAN - a fatigue/effectiveness modeling framework - 2/12/2022 6:56:30 PM   
RockPaperScissors


Posts: 27
Joined: 10/2/2016
Status: offline
Just another thought for further enrichment; in SIMEX 2018, Apache85 modelled fatigue by progessivly lowering the crew's proficiency with prolonged airborne time.

Excerpt from the game notes: "In C:MANO with air-to-air refuelling it is possible for an aircraft to stay aloft indefinitely. In real life it is uncommon and unsafe for the crew of an aircraft to remain aloft for extended periods. In SIMEX 2018 this is modelled by the Aircrew Fatigue mechanic, which will reduce the proficiency of aircraft that remain aloft for extended periods. If the aircraft remains aloft for an extreme period, there is a chance the pilot may crash due to fatigue."

KR, RPS

(in reply to boogabooga)
Post #: 15
RE: SANDMAN - a fatigue/effectiveness modeling framework - 2/12/2022 7:37:01 PM   
musurca

 

Posts: 128
Joined: 7/16/2020
Status: offline
quote:

Just another thought for further enrichment; in SIMEX 2018, Apache85 modelled fatigue by progessivly lowering the crew's proficiency with prolonged airborne time.

Excerpt from the game notes: "In C:MANO with air-to-air refuelling it is possible for an aircraft to stay aloft indefinitely. In real life it is uncommon and unsafe for the crew of an aircraft to remain aloft for extended periods. In SIMEX 2018 this is modelled by the Aircrew Fatigue mechanic, which will reduce the proficiency of aircraft that remain aloft for extended periods. If the aircraft remains aloft for an extreme period, there is a chance the pilot may crash due to fatigue."

Thank you for the recommendation! This is essentially what SANDMAN does, but I poked into the code for SIMEX 2018 to look at Apache's approach. The difference is that his exhaustion metric is based on airborne time for a particular aircraft, which is then compared an 'exhaustion threshold' that's hand-tuned per class (14hrs for an EA-18G vs. 20hrs for a Poseidon, for example). If SIMEX is modeling an exercise, it makes sense that these numbers should be hand-tuned if they represent not a human limitation, but rather an upper regulatory limit based on managing liability during training. (In that case, a ‘crash,’ rare in the real world, is intended to punish you for breaking a rule.) It also assumes that tired pilots are replaced by fresh ones throughout the entire exercise every time a plane lands. This is all a good approach if you just want to keep the player from running 60-hr CAPs.

What SANDMAN considers, in contrast, are more basic human limitations, for modeling a crisis in non-ideal circumstances where regulatory limits cannot be maintained. It also considers pilots as persistent entities over time, who accumulate sleep debt during extended periods of activity without sufficient rest, as well as human beings with circadian clocks that are sticky, shift slowly, and affect their performance depending on when they’re deployed. It also features a tool for monitoring exactly how fatigued your pilots are.

For a ultra-long scenario like SIMEX (8 days), it would be really interesting to swap out the existing fatigue model with SANDMAN, as you would see crews on both sides start to seriously degrade within a few days without the player actively attempting to stand them down.

It would also be interesting to combine these two approaches: use SANDMAN to apply physical effects of fatigue, but at the same time compare actual flying time to regulatory limits and apply penalties accordingly (by subtracting points or otherwise).

< Message edited by musurca -- 2/12/2022 7:43:08 PM >

(in reply to RockPaperScissors)
Post #: 16
RE: SANDMAN - a fatigue/effectiveness modeling framework - 2/12/2022 9:58:40 PM   
KnightHawk75

 

Posts: 1450
Joined: 11/15/2018
Status: offline
quote:

so there's no easy way to automatically pull the number of crew members for an aircraft type out of CMO.

While you're right in general about QueryDB(), in build ~1147.3x+ (I forget which, november'ish) there is UnitWrapper.crew which should now provide this number, I've not used it much but it seemed to work without issues when I've tried it.

ps. Nice work as usual.



< Message edited by KnightHawk75 -- 2/12/2022 10:01:04 PM >

(in reply to musurca)
Post #: 17
RE: SANDMAN - a fatigue/effectiveness modeling framework - 2/13/2022 1:14:50 PM   
boogabooga

 

Posts: 457
Joined: 7/18/2018
Status: offline
I'm not wanting to install all the prerequisites to build from source. I've just downloaded the source as a zip to look at in notepad++. I hope that is okay :).

BTW, where can one find SIMEX 2018? I'm not finding it...

musurca, could you please elaborate on what "effectiveness" is actually? How is- say- 86% different than 100%? What value corresponds to a drop in proficiency level?

< Message edited by boogabooga -- 2/13/2022 3:54:22 PM >

(in reply to KnightHawk75)
Post #: 18
RE: SANDMAN - a fatigue/effectiveness modeling framework - 2/13/2022 7:48:14 PM   
musurca

 

Posts: 128
Joined: 7/16/2020
Status: offline
quote:

While you're right in general about QueryDB(), in build ~1147.3x+ (I forget which, november'ish) there is UnitWrapper.crew which should now provide this number, I've not used it much but it seemed to work without issues when I've tried it.

KnightHawk to the rescue as usual -- thank you! I wonder if that number is 0 for UAVs? Will have to do some experimentation.

quote:

I'm not wanting to install all the prerequisites to build from source. I've just downloaded the source as a zip to look at in notepad++. I hope that is okay :).

BTW, where can one find SIMEX 2018? I'm not finding it...

musurca, could you please elaborate on what "effectiveness" is actually? How is- say- 86% different than 100%? What value corresponds to a drop in proficiency level?

Ha, of course. I've been meaning to rewrite the build script in PowerShell to eliminate the requirement for installing the WSL, but haven't gotten around to it in the extra time I have for "fun" tasks (and I really don't enjoy PS). If anyone wanted to do that I'd happily merge that into the repo.

Here's where I found SIMEX: https://www.matrixgames.com/forums/tm.asp?m=5120464

"Effectiveness" is a metric derived from cognitive capacity and performance in sleep studies, and is related to how people perform a complex task from a baseline measurement. In some studies the subjects were given a difficult task to perform on a computer; in others crewmen attempted to deliver artillery rounds on target.

Currently SANDMAN quantizes that effectiveness percentage and degrades proficiency from the pilot's starting baseline, with a 25% penalty to slightly narrow the window at which the pilot is at absolute tip-top performance. (I may remove the penalty in the future.) While we are working within the limited room of CMO's tiered proficiency system, it's also not so crazy, as the graphs from sleep studies indicate that measurable performance does tend to degrade in tiered stages rather than as a linear function.

Here's what it looks like for an Ace pilot—and I do recommend that you start your pilots at Ace or Veteran, to maximize the levels for quantization:


Beyond that, the raw effectiveness percentage is also the input to the functions that determine current risk of micronaps and crashing. The numbers there are speculative (but, I think, plausible and reasonable) until I find more data.

EDIT: And here's a graph showing how "effectiveness" degrades over time using the model (assuming no sleep whatsoever, and a start at peak awareness of a local time of 1800hrs), which is why I think the tiered approach is actually pretty decent:


< Message edited by musurca -- 2/13/2022 9:05:24 PM >

(in reply to KnightHawk75)
Post #: 19
RE: SANDMAN - a fatigue/effectiveness modeling framework - 2/14/2022 12:35:46 AM   
boogabooga

 

Posts: 457
Joined: 7/18/2018
Status: offline
How many "micronaps" and what is the crash probability over that time period?

(in reply to musurca)
Post #: 20
RE: SANDMAN - a fatigue/effectiveness modeling framework - 2/14/2022 7:11:58 PM   
musurca

 

Posts: 128
Joined: 7/16/2020
Status: offline
This is a graph of micronap risk per minute vs time without sleep for a pilot. In the next release I'll halve those 3AM peaks at 56+ hours (although keep in mind we're talking about people who have not slept at all for 3 days). Once a nap starts, it's twice as likely to continue as stop in the next minute. This model is based on Miller & Melfi, as well as scattered accounts of pilots talking about nodding off who gave some estimate of how long they'd been awake at that point. The general idea is that this phenomenon is incredibly common, but data from which one might build up a model is scarce.



This the graph for crash risk per minute while landing at an airbase, based on the default 'normal' chance of crashes set to 3.5 per 100K flight hours. (We're talking about hundredths of a percent here.) Landing on a ship will multiply this risk by anywhere from 5-20x, depending on weather and time of day. Boltering & go-arounds are currently 100x more likely. This model is based on the FMCSA's study of transportation safety which suggests a polynomial relationship between crash likelihood and hours without sleep, as well as a SAFTE data point which noted that pilots at effectiveness ~50% were 65x more likely to be in an accident. Increased risks based on time of day, weather, land vs. sea, etc. are purely speculative on my part.



EDIT: Note that currently crashes can ONLY happen during approach for landing.

< Message edited by musurca -- 2/14/2022 9:35:39 PM >

(in reply to boogabooga)
Post #: 21
RE: SANDMAN - a fatigue/effectiveness modeling framework - 2/15/2022 12:42:49 AM   
boogabooga

 

Posts: 457
Joined: 7/18/2018
Status: offline
Okay, every time I launch this scenario, one of the F-16s begins a "nap" almost immediately after takeoff, and the "nap" seems permanent.



Attachment (1)

(in reply to musurca)
Post #: 22
RE: SANDMAN - a fatigue/effectiveness modeling framework - 2/15/2022 6:17:48 AM   
musurca

 

Posts: 128
Joined: 7/16/2020
Status: offline
Thanks for finding this. The culprit, as feared, is the stock math.random(), which is returning the exact same small improbable value over and over for that plane, so that the nap never ends. (If you change the random seed, notice that the bug goes away.) In the next release, I'll replace it with the Xorshift32 implementation I use in IKE.

I haven't had time to report this in the Tech Support forum and create a save that demonstrates the issue, but I think what's happening is that math.random() resets its seed value every frame.

(in reply to boogabooga)
Post #: 23
RE: SANDMAN - a fatigue/effectiveness modeling framework - 2/16/2022 3:15:41 AM   
boogabooga

 

Posts: 457
Joined: 7/18/2018
Status: offline
I mean, if math.random() is bugged, is that a CMO problem or a lua problem?

BTW, I recommend that you put those graphs in the readme. Perhaps try to document the model itself as well as possible.

< Message edited by boogabooga -- 2/16/2022 3:17:36 AM >

(in reply to musurca)
Post #: 24
RE: SANDMAN - a fatigue/effectiveness modeling framework - 2/16/2022 4:13:29 AM   
musurca

 

Posts: 128
Joined: 7/16/2020
Status: offline
The problem relates to the implementation of math.random() in the embedded Lua library and how it interacts with the game. Beyond that I can't speculate without doing a deep dive to produce a save file that reproduces the conditions under which it happens. (I'll probably use the one you sent as a starting point.)

However, it's basically the same issue I noticed while working on the PBEM mod, which stores and resets the current random seed 'state' to prevent save-scumming random events. math.random() started returning the same sequence of values over and over, so that under certain conditions, e.g., the downed pilots in "North Pacific Shootout" all had the same name. You wouldn't notice this being a problem in most scenarios, as different calls to math.random() don't necessarily line up with the same context in the code. However, because the SANDMAN update loop just iterates through the same list of units every minute, entries in the pseudo-random sequence tend to line up with the same conditions, which is why that one F-16 kept napping over and over. I verified that this is indeed happening by printing out the values in the save file you posted.

What I suspect is happening is either that, under certain conditions...
a) math.random() is resetting its own state, either every frame or else within a period less than the 60 seconds between SANDMAN updates; or,
b) math.random()'s underlying PRNG is rolling over MUCH faster than it should.

I wouldn't blame the CMO team, as you can find Stack Overflow questions about this same problem happening in other embedded Lua contexts. The advice is always some black magic suggestion like "depending on your platform, read a passage from the Necronomicon and call math.random() three or four times to prime it before use," but that of course is total nonsense, as that is not how pseudo-random number generators should work in production. The Lua users wiki provides a series of increasingly insane suggestions before recommending that you just import the third-party lrandom library.

My solution to minimize time spent debugging arcane issues, both here and in IKE, is to reimplement a proper PRNG in pure Lua, and avoid math.random() entirely.

EDIT:
quote:

BTW, I recommend that you put those graphs in the readme. Perhaps try to document the model itself as well as possible.

Sure, that's a good idea. Since this is all still in flux as I make changes based on the excellent suggestions so far, the graphs may fall behind as the model changes.

< Message edited by musurca -- 2/16/2022 8:06:00 AM >

(in reply to boogabooga)
Post #: 25
RE: SANDMAN - a fatigue/effectiveness modeling framework - 2/16/2022 3:44:55 PM   
boogabooga

 

Posts: 457
Joined: 7/18/2018
Status: offline
Sorry, can't resist...

https://xkcd.com/1277/

(in reply to musurca)
Post #: 26
RE: SANDMAN - a fatigue/effectiveness modeling framework - 2/16/2022 5:07:14 PM   
musurca

 

Posts: 128
Joined: 7/16/2020
Status: offline
LOL. Nothing turns me into a ranting Galtian lunatic faster than the mere mention of math.random!!

(in reply to boogabooga)
Post #: 27
RE: SANDMAN - a fatigue/effectiveness modeling framework - 2/17/2022 7:13:56 AM   
nukkxx5058


Posts: 2932
Joined: 2/3/2005
From: France
Status: offline
Wow ! Just wow !!!

(in reply to musurca)
Post #: 28
RE: SANDMAN - a fatigue/effectiveness modeling framework - 2/21/2022 2:40:22 PM   
boogabooga

 

Posts: 457
Joined: 7/18/2018
Status: offline
If I change the plane set after an initial SANDMAN install, do I just reinstall, or do I need to delete all of the events / special actions first? What about the global key/value register?

(in reply to nukkxx5058)
Post #: 29
Page:   [1]
All Forums >> [New Releases from Matrix Games] >> Command: Modern Operations series >> Mods and Scenarios >> SANDMAN - a fatigue/effectiveness modeling framework Page: [1]
Jump to:





New Messages No New Messages
Hot Topic w/ New Messages Hot Topic w/o New Messages
Locked w/ New Messages Locked w/o New Messages
 Post New Thread
 Reply to Message
 Post New Poll
 Submit Vote
 Delete My Own Post
 Delete My Own Thread
 Rate Posts


Forum Software © ASPPlayground.NET Advanced Edition 2.4.5 ANSI

0.719