herwin
Posts: 6059
Joined: 5/28/2004 From: Sunderland, UK Status: offline
|
I'm interested in your comments about the AI. I work on biologically-inspired robotics, so I have to deal with the scripting problem. Our current direction involves a action value system that incorporates three elements: reflexes, habits, and goal-oriented behaviour. The decision-making involves selection from available actions by summing the expected values of each (in terms of each element), weighted by 1/(variance). (Thus high risk scripts would have low weights.) Then select actions in terms of these estimated values, starting at the top. Goal-oriented behaviour involves having a forward model of each possible operation. We are just beginning to understand how forward modelling takes place in the brain. It seems to involve the posterior part of the cerebellum, which doesn't work with the motor system, but instead seems to play a more general role. The Purkinge cells of the cerebellum are the biggest and most computationally sophisticated neurones in the CNS. They linearize actions and learn the partials of the payoffs as a function of adjustable parameters so they can report back the change of payoffs as a function of changes to parameter settings. What this suggests is that your AI might take the approach of integrating reflex behaviour, habitual behaviour (using an actor-critic approach that caches payoffs), and goal oriented behaviour. All three categories involve scripts. The goal-oriented behaviour scripts have adjustable parameters. You then need a model for each goal-oriented script that predicts payoffs and converts parameter settings into deltas to the payoffs. Keep track of script performance in an engine database, and optimise over time for each installation.
_____________________________
Harry Erwin "For a number to make sense in the game, someone has to calibrate it and program code. There are too many significant numbers that behave non-linearly to expect that. It's just a game. Enjoy it." herwin@btinternet.com
|