[MUD-Dev] Statistics (was: Girl appeal (was: Boys and Girls))

[Sasha Hart]

[Caliban]

> For suitable values of scientific. While the data may be
> interesting and valid, it's still coming from a series of
> anecdotes, and that's just "anecdotal evidence". But there's a
> point beyond which you transcend "anecdotal" and enter the realm
> of "statistical".

That point is not defined in sample-size-space, but
observation-validity-space :) As I think you are pointing out.

I think there is usually very little problem with self-report,
insofar as the questions are ones the "respondent" is qualified to
answer. Start with things like age and sex; they could be lying but
if you haven't tipped them off to anything, why should they? In many
cases this will just add noise unless you are studying something
about which people lie or forget in interesting ways. The
reliability of the measure (for example, the correlation between
test and retest answers) is a pretty good starting point.

I usually feel okay about correlations involving things like "How
much time do you play EQ," just because for the correlation the
absolute value doesn't matter so much as whether the value is up or
down, and there's little reason to assume respondents are reacting
specially to the question.

The only non-heuristic way to deal with this is by having controls,
counterbalancing for the order questions are presented, being
omnipotent, etc. But having the suggestive information is better
than having to pull it out of, er, thin air.

Things like "(why)Are you addicted to the game," on the other
hand...! This is another problem with taking free responses and
other popular qualitative data: not only are they impossible or
horribly laborious to analyze, but they are terribly sensitive to
other things and will often influence how subsequent questions get
answered. and figuring out what they really mean is hard in the best
of conditions. At best they can give you a starting point or an
intuitive idea of what's going on, but if your conclusions are
riding on them, then I, for one, will probably not accept them on
that basis.

> Well, there's the rub. I tend to draw a line between statistics
> and science, because they seem almost mutually exclusive; the
> average statistician doesn't understand science, and the average
> scientist doesn't understand statistics.

Heh. I was confused about this until I realized that "the average
scientist" was probably a theoretical physicist or maybe a cell
biologist or something. Unfortunately, things like behavior and
population biology often need both. What a pain in the ass!
_______________________________________________
MUD-Dev mailing list
MUD-Dev at kanga.nu
https://www.kanga.nu/lists/listinfo/mud-dev