[MUD-Dev] FEAR Animat style AI for Quake 2

[Sasha Hart]

[Dave Rickey]

> This is the kind of problem that neural networks and genetic
> algorithms are good at, after the guy with the rifle turns and
> runs away a few times, the system will come to associate that with
> "5 seconds from now that guy will be standing behind that
> barricade shooting at me," or whatever tactic is being excercised
> that would trick simpler AI, and have a grenade already primed
> when the player gets there.

First, the example given by Ted about the rifle pointing immediately
made me think of habituation. If the animal reduced threat estimates
when threatened repeatedly without fallout, it would just start
ignoring the dope. There is a lot of good work on habituation,
particularly in Aplysia (sea slug).

If the purpose is to take predictive action (the grenade being a
perfect example) do note that this is PRECISELY what classical
conditioning is about. This is also an area with a lot of good work,
from the Rescorla-Wagner model to its very successful cousin,
Temporal Difference (TD) learning.

Possible advantages being cheaper computation and faster learning
(since they can learn online and don't need a bunch of training
runs).

Reinforcement learning methods might be interesting - I'm very fond
of the text by Sutton & Barto, which has an online version:

  http://www-anw.cs.umass.edu/~rich/book/the-book.html

> How good they can get at it is mostly a matter of how much
> processing power you can dedicate to GA selection or NN back-
> propagation, and how many inputs you can wire up and process in
> real time.

Also your expertise in working with and picking parameters for the
techniques, of course. Backprop is a bit of a black art and gets
misapplied all the time.

I don't know nearly enough about GAs, but I would have to convince
myself that they weren't too slow to learn - because of the premise
that you operate on the entire 'genome' rather than piecewise, which
at least intuitively suggests to me that you get much slower
sampling than you would with an online method. Then again, I
generally play with problems that probably don't play to the
strengths of GAs.

> We're not far from being able to run this grade of AI for entire
> worlds simulataneously (several thousand players interacting with
> a like number of AI agents at any one time).  Maybe 5 years before
> we see a game using this approach.

Depends on the algorithms, of course - GAs and NNs are not cheap,
but for some problems, even quite general ones, there are pretty
cheap alternatives.

Sasha


_______________________________________________
MUD-Dev mailing list
MUD-Dev at kanga.nu
https://www.kanga.nu/lists/listinfo/mud-dev