[MUD-Dev] GDC 2002: Beyond Psychological Theory: Getting Data that Improves Games

[Sasha Hart]

[from paper by Bill Fulton]

> The insufficiency of formal theories of psychology. 

[...]

> But academic theories of psychology don't get granular enough to
> tell us whether gamers find the handling of the Ferrari a bit too
> sensitive.

To provide answers which they aren't built to have: no one should be
surprised that Skinner's work (much of it in the areas of apparatus
design, writing of popular & controversial books with relatively
small amounts of academic content, development of terminology which
is either odd or entirely necessary, depending on your POV) has
simply nothing to say about whether the Ferrari's controls feel
right. (This is probably not very well-mannered, as the author was
probably making this nose-on-the-face point, albeit in slightly
inflammatory terms. :) ).

> But while theories of psychology from academia can be quite useful
> as a lens to examine your game, their limitation is that they are
> typically too abstract to provide concrete action items at the
> level designers need.

As in:

  A) The source has boiled it down for you in some pat formula
  (useful for when you just want a little question answered roughly,
  not to crank up SAS for a couple hours). A description I'd apply
  to John Hopson's article.

  B) The theory isn't "abstract," it's insubstantial! It barely has
  contact with the data, let alone modeling it in any comprehensive
  sense! All too common...  Just like everywhere else, there are
  hacks in psychology. To be fair, there are also incredibly hard
  questions and things which no one has really gotten to. Psychology
  is hardly comprehensive, hardly unified and hardly mature.  (Oof,
  I've passed my quota of awful generalizations for the month).

> I think very few people had light bulbs go on when they learned
> that Skinner's theory of conditioning stipulates that people will
> do stuff for rewards.

Certainly Thorndike's light didn't go on, as he had put the Law of
Effect decades before Skinner really got rolling. What Skinner did
is formulate a neat argument that behavior could be studied in terms
of functional units (the lever press isn't just an arm muscle
twitch), not just S-R units (e.g. the "knee-jerk" reflex). If you
want to beat your brain out on it, read _The generic nature of
stimulus and response_.  Skinner also made some gadgets for working
with animals and wrote a lot, including a ton of popularizations and
vaguely philosophical-political stuff that tend to obscure the fact
that there is real science and substantial theory out there, not
just a statement that people work for rewards (duh) or that all
behavior is for rewards (not exactly, no).

Most of it ISN'T by Skinner, some of it precedes him, and some of it
calls itself cognitive science, and ...

Take away the experimental work, the other theory AND the formalism
and you do have a "duh" idea that people like good stuff. (No
surprises there.)

> One of Hopson's examples is about how players in an RPG behave
> differently depending upon how close they are to reinforcement
> (e.g., going up a level, getting a new item, etc.). He talks about
> how if reinforcers are too infrequent, the player may lose
> motivation to get that next level. However, how often is not often
> enough? Or too often? (Who wants to level up every five seconds?) 
> Both "too often" and "not often enough" will de-motivate the
> player.

 I complained about this article too a while ago, but it's sad to
 jump on it given how little ANYONE has written on the
 subject. There are a lot of controversies and cans of worms to
 avoid here and I think the article does a good job of that.

That said, I do agree that this isn't enough to do precise work
with.  Unfortunately, figuring out any more requires people to do
research for free.

> Designers need to find a 'sweet spot' between too often and not
> often enough that provides the optimal (or at least a sufficient)
> level of motivation for the player to keep trying to level
> up. Theory may help designers begin to ask the more pertinent
> questions, but no theory will tell you exactly how often a player
> should level up in three hours of play in a particular RPG games.

This is in my view the top question for anyone working with
leveling, or for that matter anyone trying to get away from
leveling. Why should we care about levels at all? What makes us care
more or less?

This is just the kind of data that (IMO) would be immensely helpful
across games. Unfortunately the article goes on to recommend the
same old stuff (ask players how fun this is - useful, but not at all
general). Meanwhile I'm sure that many of us have tweaked this
parameter and have seen something about it. I'm curious if anyone
has observations on this point.

For my part, I have played games which leveled up to 1/15 min. and
enjoyed them; that's probably about as short a leveling rate as I've
seen. The problem really seems to be not that fast leveling is
particularly demotivating, but that it burns through areas
quickly. I also imagine that once the rate dropped much below once
every 5 minutes that one might just ignore it (but I don't
know). For my part, in games with levels that are more than nominal,
I find that I tend to settle in games that are about 1 level a day,
as it pretty adequately fits the casualness of my interest in those
games. I also often quit within a month, in my perception at roughly
the same reduction in level rate that generally puts me off of a
game. Of course, this isn't particularly good data, just idle
yarn-spinning.

  (If anyone wants to write/contribute programs or playtest to such
  ivory-tower ends, I'm going to try rounding something up this
  fall. Data publicly available, etc. Long-time personal interest of
  mine. Perhaps a post will follow. Write me if this sounds at all
  interesting.)

>  1.  The feedback should accurately represent the opinions of the
>  target gamers.

Opinion usually entails what they say - but we just spent a
paragraph reading about how everyone is a wannabe designer, so how
should I read gamers' opinions? Why not consider looking at their
actual choices of one game over another, one skill over another,
etc. as an index of their preferences?

Moreover: is my only interest in what they like and don't like? What
about how easy it is to learn something, how lonely the game is? 
There are a ton of kinds of feedback which aren't opinions.

> The level of feedback in the reports is extremely granular,
> because the tests are designed to yield granular, actionable
> findings.

In fact, it sounds so fine-grained that it all boils down to
questions like the one where some weapons' sounds specifically sound
dumb. Easy to act on.

Doesn't really allow for much flexibility; obviously if a suggestion
requires a redesign then it's not as 'actionable.' Pretty much
ruling out (and with good common sense) gathering useless
"theoretical" data like how much leveling is enough. Which is why we
aren't really getting it until someone just does the experiments and
shares the findings. Given the actually quite low requirements for
effectively asking meaningful questions (no 2 years graduate
research necessary), I don't see any reason why anyone who wants to
shouldn't simply build these into their games, in-progress or
ongoing.

ETS (the company that does SATs/GREs/etc.) routinely puts
experimental sections in its tests. Good information is
outstandingly cheap to us in much the same way (computers being
great measurement devices, and players already practiced with
feeding them). Really it's just taking the interest.

Sasha
_______________________________________________
MUD-Dev mailing list
MUD-Dev at kanga.nu
https://www.kanga.nu/lists/listinfo/mud-dev