[MUD-Dev] Metrics for assessing game design

[ceo]

Paul Schwanz wrote:
> Matt Mihaly wrote:
>> On Wed, 16 Jul 2003, ceo wrote:

>>>   P.S. About 50% of those who've dismissed this out of hand have
>>>   been professional games developers who felt that "of course
>>>   it's not possible to rate fun, and never will be; it's
>>>   something you feel in your gut, not something you can
>>>   analyse...[subtext: how dare you imply that my job as a games
>>>   designer can be pared down to a few simple rules; it's much
>>>   more complex than that]". I only really mean this as the
>>>   combination of a metric, plus that the act of evaluating
>>>   against it generates guidelines for how you can improve a
>>>   game's score.

>> Well, I don't think it's a matter of not being able to analyse. I
>> think it's a matter of there being no such thing as an
>> "ultimately fun" game.  The very idea of paradise misses the
>> point. There can be no paradise, no ultimate game, without
>> specifying who the paradise is for or who will be playing the
>> game. The ultimately fun game for me is almost surely not the
>> same as for you or for Nancy Reagan. In the end, I don't think it
>> can be measured simply because there is no common measuring
>> stick.

Hmm. That's quite close to what some of the dismissers said, which
suggests I've still not made things clear enough.I'm not aiming for
paradise. I'd probably be delighted if the number of games published
each year that sucked so abominably you can't believe the authors
bothered were reduced to only a few hundred (it's currently
something like 2000, IIRC).

If everyone were making great games, or if most games were on time
and on budget (or even within 10%) there'd be little or no need for
something like this. But, frankly, the industry as a whole is
grossly inefficent in ways that are definitely NOT conducive to
either quality or success. And I haven't met anyone who doesn't want
it to change; I've met plenty who are so jaded they don't believe it
can change, but that doesn't mean they're right :).

I agree quite strongly with one of this week's Gamasutra articles
which points out that the infamous "crunch time" on games dev
projects probably is the single greatest drain on creativity within
the industry - and it has no plus side.

To diverge slightly :), I used to be a Fine Arts artist (drawing,
painting, etc). The people I worked with were a spectrum from those
who were mediocre with the occasional exceptional piece, through to
people who consistently produced excellent works. The consistently
good people mostly agreed that great art was largely accidental,
some going as far to say that the artist's skill merely increased
the chances of a felicitous accident. Others argued that the skill
of the artist was making lots of mistakes, and deciding which were
the good ones, and which ones to paint over.

I don't remember anyone seriously claiming that you couldn't very
effectively (and accurately) rate pieces of art, often in a very
precise manner. If it were impossible to construct metrics, people
would not agree on "good" or "great" art. One thing that muddies the
waters is that the metrics used are often highly sensitive to the
culture of the viewer - 16th century tastes are very different from
20th century tastes. However, this can easily be factored in, and
this is one of the reasons for studying the history of art - it
allows you to see artworks as they would have been seen by people of
the time.

What I'm trying to show with the fun-metrics is that a huge bulk of
what is fun in a computer game can be measured with just one
technique. If I were to attempt to pigeonhole it, I'd say the type
of fun measured is "what makes classic games where gameplay triumphs
despite poor graphics, sound, etc". For instance, Warcraft is
extremely dull without the graphics. I know this because I used to
play Freecraft (the open source clone) many years ago, way before
they'd drawn any artwork. The cartoony graphics are a major part of
the enjoyment, and are not covered by these metrics. OTOH, if you've
ever wondered why you still go back and play Tetris every now and
then, or why simcity and civilization stick in your mind as being so
much better than anything that followed for many years...

> And thakfully so for those such as yourself or Daniel James who
> are courting the niche markets with original gameplay.  (Beware
> thread convergence.)  However, I suspect that those who dismiss
> Adam's proposed metrics out of hand on such a basis might simpy
> deny themselves the use of a tool that could be quite effective
> and valuable once its limitations are understood.  Perhaps it
> might be more accurate to state that this set of metrics helps
> rate a certain kind of fun, but this is reason to employ it
> wisely, not to dismiss it out of hand.

I tried writing a brand-new game from scratch, with no clear design
document or "great idea", on the grounds that if this metric works
as well as I think, I should be able to iteratively produce a fun
game, using the metrics to show what I should change and how
(although I still have to decide what to change TO). It's working
quite well, but I need a month more of spare time to implement the
latest iteration of changes - which I'm pretty convinced will make
it into a good game - and at the moment my free time has evaporated
:(. Sadly, the current iteration was only half-implemented, so at
the moment I have nothing to show :) (current commitments caught me
a bit by surprise!)

FYI I'll also point out that I came across this completely by
accident :).

I was actually trying to evaluate games to use whilst experimenting
with evolutionary-AI (i.e. automatically evolving better and better
AI's). At one point, I sat down with a long list of pre-existing
computer games and ranked them according to how well they were
likely to work with EP. The games were just about anything I or any
of my friends or colleagues could think of, with duplicates ommitted
(e.g., Quake appeared once, instead of Quake, Doom, Unreal, etc).

What surprised me was that despite eventually coming up with
hundreds of games, all the games that were at the top of the
rankings were the most consistently popular and addictive ones. I've
made several attempts over the last few years to try to explain this
correlation - is it something to do with humans evolving over time
to enjoy the kinds of processes that are easy to improve at by
evolutionary methods? - but have come up with nothing convincing. My
best explanations are so flaky I've given up, and concentrate now on
just refining and using the results :)

Adam M
_______________________________________________
MUD-Dev mailing list
MUD-Dev at kanga.nu
https://www.kanga.nu/lists/listinfo/mud-dev