[MUD-Dev] Microthreads for Python

[J C Lawrence]

While Python already supports threads, its not really a fully
scalable model.  The following addresses some of that:

  http://world.std.com/%7ewware/uthread.html

--<cut>--
It would be nice to use Python for programs involving huge numbers
of very light-weight threads. This would make it easy to simulate
large swarms of interacting agents, which sounds like fun.

Python functions are compiled to a bytecode, which runs on a virtual
machine, defined in Python/ceval.c in the eval_code2
function. Python uses frame objects to administer information
involved in nested function calls. In ceval.c, the frame stack is
implicitly embedded in the C stack, because eval_code2 calls itself
recursively when one Python function calls another.

Microthreads are implemented with a modified version of the code in
Python/ceval.c, which is Python's bytecode interpreter. Each thread
maintains an explicit frame stack, whereas normally frames are kept
on the C stack via recursive calls to eval_code2. My version of
eval_code2 can be stepped for a number of steps, unlike the normal
version which runs in an infinite loop until the outermost function
returns.

I've defined several objects in C: a "working" object (which is a
placeholder like the normal "None" object), a "uthread" object, a
"semaphore" object (also known as a mutex), and a "queue"
object. The semaphore and queue are microthread-safe. There is also
a microthread-safe version of the whrandom random number generator.

uthread.thread() creates a microthread. As arguments, it takes a
function, and any arguments that the function expects. For instance
to create a thread that computes f(x,y,z), you would write something
like 'mythread = uthread.thread(f,x,y,z)'.

Every thread has a step method (mythread.step()) which will step the
microthread for a finite number of opcodes. It returns the thread's
return value, if the thread has finished, or if the thread has not
yet finished, it returns uthread.working (the "working" placeholder
object). If you attempt to step a thread which has already finished,
an error will result.

uthread.semaphore() creates a semaphore object. It takes an optional
integer argument (default value is 1) telling how many instances of
the semaphore may be outstanding at any moment. A semaphore has the
following methods: claim() will claim one instance of a semaphore,
or block until an instance becomes available. qclaim() is a
non-blocking version of claim, which returns a boolean telling
whether the claim was successful. release() will release an instance
of the semaphore.

uthread.queue() creates a queue object, and takes an integer
argument, specifying a number of stages.  Each stage in the queue is
a normal list. A queue has the following methods: put(x) will append
x to the list at the rear of the queue, get() will remove and return
the first object from the list at the front of the queue (None if
the front queue is empty), and step() will advance the lists one
step toward the front of the queue, concatenating the front two
lists to become the new front list.

There is much similarity between this project and Stackless Python
by Christian Tismer, as several people on the Python newsgroup have
pointed out. Eventually it will probably make sense to merge
microthreads into Stackless Python, but it's not my most immediate
priority.

Some of the applications that interest me for microthreads are
amorphous computing and computational economics. Microthreads could
also be used to write agents in a game, such as robot tanks or
PacMan monsters
--<cut>--

The Stackless Python referenced above can be found at:

  http://www.pns.cc/stackless/stackless.htm

but currently that site is note accepting connections.

--
J C Lawrence                                 Home: claw at kanga.nu
----------(*)                              Other: coder at kanga.nu
--=| A man is as sane as he is dangerous to his environment |=--


_______________________________________________
MUD-Dev maillist  -  MUD-Dev at kanga.nu
http://www.kanga.nu/lists/listinfo/mud-dev