Event handling (was: request for comments)

[s001gmu at nova.wright.edu]

On Fri, 9 Jan 1998, Vadim Tkachenko wrote:

> s001gmu at nova.wright.edu wrote:
> > 
> > On Thu, 8 Jan 1998, Vadim Tkachenko wrote:
> > 

<original question dropped>

> 
> [skipped]
> 
> > > - One thread per connection listener (it's possible to install more than
> > > one protocol adaptor, and therefore, connection listener)
> > > - One thread per incoming connection
> > 
> > so... at least 2 threads per user?
> 
> Of course not - connection listener thread just accept()s the incoming
> connections and
> spawns the controlling thread

ahhh... gotcha.   I believe I am heading towards a model like this, though
if I understand what you mean by protocol adapter, I am only planning on
using one. 

> > The decision to go with ticks and a limited number of threads to handle
> > the events for a tick stems directly from concerns of system load.  I
> > don't want to design a game that _requires_ it be the only significant
> > process running on a server. 
> 
> Well, I look at it from slightly different point of view - 
> 
> - my MUD project is for fun, but this is a guinea pig for every new
> idea/concept/technique
> - If I would write that seriously, the commercial application is a
> target, then why wouldn't I dedicate the computer to run it?

I am working from the perspective of possibly finding a benevolent
sysadmin, willing to share a server with me.  Different goals, different
tactics.  :)

<humor snipped... ;) > 

<...>
 
> > As I said, it's very much a result of a lot of factors, including, but not
> > limited to cpu speed, threads implementation, compiler, OS, the DB in
> > use... etc.
> 
> Aha, and I also forgot to think that I was not the only user on the site
> at that time, so effectively fail-threshold lies above 50.

well, under those circumstances it did.  Other circumstances might drive
the number in various driections.
 
> > I am unfamiliar with the minSpare/maxSpare management strategy... the
> > O'reily PThreads book didn't discuss it, and as that's the only book I
> > really have avail... :)  Care to give a brief tutorial, or point me to a
> > web page?  I am a tinkerer at heart, and would prefer an algorithm/design
> > strategy to source code, as I prefer to code my own stuff.  :)
> 
> And I've never heard about PThreads book... See, all my knowledge is 99%
> experience-based - you know, when O'Reilly book eats out 50% of your
> monthly income (which was true until recently for me), guess what...

It's a pretty good book, though it only deals with somewhat cursory
examples, and is (obviously) about only the Posix Threads Standard.  It
does have some very nice, if somewhat general, discussions of optimization
and how to evaluate performance of a threaded program.
 
> OK, so the minSpare/maxSpare strategy is what I saw in Apache HTTPD
> (http://www.apache.org).

thanky much... I'll take a peek at that right-quick.
 
> - You keep at least minSpare spare request handlers, just in case, so
> first minSpare incoming
> requests arriving simultaneously will be served really fast.
> - After some requests are finished, you keep at most maxSpare spare
> request handlers, terminating anything extra (variation: after they
> expire by themselves).

I figgured it was something like that.  True, that does seem to localize
most of the thread-creation cost to boot-time.  Especially if you pick a
decent range and initial number of threads.
 
> That's basically it. Also, the implied consideration is you have to have
> maxRunning request handlers and never exceed the limit or you're likely
> to hit the system limits - just couple of examples:

of course... :)
 
> - Artificial limit on evaluation versions of JDBC implementations
> - MAX_THREAD
> 
> Also, you may consider dynamically change minSpare, maxSpare and
> maxRunning depending on other conditions.
> 
> Also, there are two types of requests worth mentioning:
> 
> - packet request
> - channel request

have to read up a bit, but I think I gather the gist of the points.. :)
 
> > mmm.. even in an asynchronous event handler, you can get two events
> > targetting the same data executing at 'the same time'.  This requires
> > either a lockless/rollback method or a data locking method.
> 
> Well, there are known methods to resolve this - either mutex semaphores
> or synchronized methods in Java (btw, the absense of semaphores in Java
> was really annoying, so I recreated them :-)

uh... exactly.  which was the point of my mentioning the discussion on
lockless v. locked model.  both are well tested and accepted models for
handling data integrity in MT environs, synchronous or otherwise...
 
> > Well, when you come down to it, even a so-called asynchronous model is
> > still time-slice or tick-based.  the system's clock just has a very fine
> > grain.  I suppose it's a throw-back to paper RPGs.  I played a lot of them
> > for many years (really haven't played much the past couple... *sigh*), and
> > it's hard to break out of the mold they establish.
> 
> Yeah, it was different for me - with my computer experience with games
> which didn't engage turns (at least, obviously), like Eye of Beholder,
> Ultima Underworld, I had no prejudice

*nod*  our end goal is to not make it obvious to the player that the game
is turn based, but the underlying mechanics will be.  The more that I
think about it, the less I like the 'tick' idea... Why artificially impose
a less granular clock on top of the system clock?  Why not just let the
system clock determine the timing?  Situations where the character's speed
should be far faster than the players typing speed (IE: combat, etc), can
(and should?) be handled by the computer (IE: the computer is generating
the events, not player commands).  All other commands don't need to be
turn-based, as speed has already been deemd uniportant by the designer
(me), by that system not being coded as an auto-generated event.

mmm... ahh well.  another redesign spawned by the list.. *grumble*  ;)
 
> > but when I tell my
> > character to "cast fireball at bubba", it becomes trivial to implement the
> > delay inherent to the action by scheduling the event to go off 3 ticks
> > down the road.
> 
> By the way, this is exactly the point where extra threads appear in my
> model. Every time-based action is asynchronously handled by the separate
> thread, which allows to handle effects like spell backfires,
> concentration, partial force release, accumulation rules, side effects
> with extreme ease.

*nod*  I think I see where you are going.. the event, "cast spell" is
plunked on the queue, and handled as quickly as possible.  Handling means
calling a method which may "sleep" for a while, to take care of the delay.
If it's interrupted while sleeping (someone interrupts bubba casting the
spell), you have some sort of 'interuppted' method which handles the
results of someone busting up bubba's concentration.  Right?

That being the case, I'd still prefer to let events spend their delay time
on the queue, instead of in a thread.  Each sleeping thread is a thread
that could be used by something else.  Why allocate a scarse resource
before it's 100% needed?  Again, this harks back to my initial goals, of
building a non-cpu intensive game... well, at least, less so than other
games... ;)
 
> > yes, you could just as easily let bubba's character wait for however long
> > and then generate a 'fireball is cast at bubba from boffo' event, to be
> > handled immediately.  Either way works.  It's just a matter of where you
> > choose to do your time and potential interrupt handeling.
> 
> See, I do not consider 'cast fireball' as EVENT. I consider it as a
> PROCESS.

*nod*  see above... 
 
> > > Let me point out one possible common target: if you use the concept of
> > > context, or environment, and it's not a fixed value, then ...
> > >
> > > Like, time, weather in the area, gravity, etc.
> > 
> > mmm.. definately something to consider.  Thanks.  Anything else I missed,
> > anyone?  ;)
> 
> Just an idea: lycanthrops, whose properties change with a phase of the
> moon[s]

mmm.. I dunno that that is really a valid concern... I am leaning more and
more towards JC's Spoof/Watcher model, which allows for such things
nicely (definately planning on spoofs.. still thinking about watchers..
dunno if my design goals necessarily coincide with the ones you chose that
lead to spoofs/watchers, JC).  W/O using watchers (to watch the phase of
the moon), I would schedule an event to go off at the beginning of the
time the lycanthrop should morph, to trigger the morph, and then one to go
off at the end of the time, to morph the poor bastich back.  No real
worries about multiple events targeting the same data at the same time, as
the events would occure once a month!  :)

an example of what I am concerned about would be something like:
bubba and biffy both blast boffo with a ball of blazing fire, in
rapid succession.  Both fireballs go off at close enough to the same time
that the handle_event methods are both trying to access boffo's soon to be
much depleted HP stat at the same time.  Reducing the number of such ...
collisions (?) is a primary goal... less collisions, the less time you
spend with threads waiting on eachother to release locks (IE: executing
serialy, not in parallel)

<handling events in list at termination/reboot>

  <two options, handle at termination or save and handle at startup>

 <my response of "mm.. maybe best to just let them die at termination>

> 
> Bad thing for me, because I'll have a lot of asynchronous events for
> each object - for example, I'm going to implement the algorithm of
> applying the temporary changes, illustrated by example:
> 
> - You get hit by the poisoned sword.
> - Consequently, there are two state changes: your limb gets a wound
> (let's make it a light one), and you get poisoned.
> - Wound is going to heal by itself, slowly (asynchronous process, which,
> in according to the abovementioned flexibility, may have variable rate
> depending on your state, weather, nutrition etc.)
> - Poison is going to continue to affect you (asynchronous process), once
> again, with variable rate as above.

mmm.. so you handle all state changes, and resulting moves back towards
the norm as an event-process?  (IE: each wound is treated as an event?)
I think I'd treat the situation as a whole, rather than each individual
wound.  IE: there is one event logged (for each <thing> that is not at
it's normal state) which takes care of moving that thing back towards it's
normal state.  If I get a wound, it schedules the normalizer to go off
whenever, to up my hp a bit and reschedule another normalizer event.  If I
get another wound, the wound event wouldn't schedule a new normalizer,
'cause one is already pending for the previous wound.  how the normalizer
is scheduled can be as complicated as you like.

I'm thinking the best solution might be to fall back on DB backups...
every <so often> the program dumps a 'current state' of the pc to the DB.
if a crash occurs, the last 'current state' is loaded.  Current states
would include enough info to reconstruct the events for the character in
question.  Some events would have to be handled seperately... combat type
events, etc.  mmm.. definately requires some thought.  Mayhaps I shall go
dig up my concurant software design notes, and DB notes on rollback
techniques.

The main issue I see is:

Bubba and Boffo are fighting.  Bubba and Boffo are on different clocks for
their 'db backup' events.  Combat is interrupted by a crash / termination
/ reboot / whatever.  If all events are stored in the db backup, Bubba and
Boffo will have different pending combat events, based on the different
times at which their last backup was made.

Easy solution, put everyone on the same clock.
problem: large db writes, all at once.  slams system pretty good with a
lot of ppl on.

definately a DB rollback issue.
 
Any comments from the peanut gallery?  :)

-Greg