[DGD] Hydra

[bart at wotf.org]

As an aside, the Linux amd64 archive on the hydra download site has a .zip
extension, but it is a tar.gz file.

Bart.

On Mon, 20 Aug 2018 00:18:53 +0200, Felix A. Croes wrote
> Today I released Hydra version 1.3.12.
> 
> I have been working on DGD since 1992, and on Hydra since 2002.  Although
> DGD muds peaked around 2005, I think there is still a case for LPC 
> as a programming language, for error handling with atomic functions, 
> for the event-driven model (it has been fun to watch the node.js 
> folks rediscover old LPmud wisdom), and for object-oriented database 
> management systems. So I continued developing DGD as a hobby after 
> it stopped being my day job.
> 
> Hydra was born from the realisation that DGD's event-driven tasks
> could be parallelized: let multiple tasks run concurrently on copies
> of objects, and when they finish, commit the changes when the objects
> have not been modified by any other task in the meanwhile; otherwise,
> lob off a Hydra's head (discard the task's changes) and let it grow
> back (reschedule the task to try again).
> 
> What makes this especially interesting is that to the LPC developer, 
> the concurrent execution of tasks is invisible, and it appears that tasks
> are still run in sequence because the commits are serial.  The 
> complex machinery of parallelization is hidden from view.
> 
> Of course, the machinery of parallelization was not hidden from my view.
> Developing Hydra has been a very time-consuming hobby.  For the past 
> two years, I have been working on optimizing the locking 
> architecture, culminating in today's release.  The multiple CPU, 
> multi-core future that I expected in 2002 has definitely arrived.  
> Hydra is well suited to take advantage of the modern multi-core CPU
architecture.
> 
> Not on all CPUs, however.  As far as Hydra is concerned, hyper-threading
> is a mis-feature.  Any type of feature sharing between hardware threads
> or cores is likely detrimental.  Other features, such as "turbo boost"
> (increasing single-core performance when no other cores are in use)
>  complicate measuring the benefits of multi-core execution 
> accurately. Furthermore, Hydra has been hard hit by this year's 
> speculative execution bugs.  The mitigations for Meltdown, Spectre 
> et al slow it down considerably.
> 
> Additionally, Hydra does not perform equally well on all operating systems.
> I have been using macOS as my main development system; unfortunately,
> Hydra scales abysmally on macOS.  I have run some tests on a 2008-
> era dual CPU system with a total of 8 cores, which thankfully lacks 
> hyper-threading and a host of other features that modern CPUs are 
> often blessed with.  I see that Hydra scales well on Windows 10 and 
> Solaris 10, but when using 4 or more threads dedicated to running 
> LPC tasks, Linux is slightly faster. Interestingly, with fewer than 
> 4 threads dedicated to LPC tasks, Linux is much faster than other 
> operating systems I tested.  Overall, I see best performance on 
> Linux kernels without speculative execution bug mitigations.
> 
> Since testing on a 10 year old system is hardly definitive, I want 
> to ask for volunteers to test Hydra on such modern systems as they 
> have available. The test should be run on bare hardware.  Running it 
> virtualized slows down Hydra a lot, and is also likely to have a 
> large impact on other VMs running on the same hardware.
> 
> The test itself is simple.  Download Hydra from
> 
>     http://download.dworkin.nl/hydra/
> 
> and the test from
> 
>     http://download.dworkin.nl/hydra/benchmarks/bm002.zip
> 
> Edit test.dgd from bm002 to let the base directory point to wherever 
> you put bm002/src, and run Hydra specifying the number of threads dedicated
> to LPC tasks, for example:
> 
>     hydra-1.3.12 -1 test.dgd > test-1
>     hydra-1.3.12 -2 test.dgd > test-2
>     hydra-1.3.12 -4 test.dgd > test-4
> 
> Then post the results, plus CPU information (/proc/cpuinfo on Linux) 
> to the mailing list.
> 
> Test results for more than 4 cores are especially interesting. 
>  Hydra runs more threads internally than are used for LPC, and you 
> should leave some room for the other OS tasks, so tests up to the 
> number of cores minus two are meaningful.  If your system has 
> hyperthreading, divide the number of "cores" reported in 
> /proc/cpuinfo by Linux by 2.
> 
> The benchmark is synthetic and does not test Hydra's true 
> performance; rather, it attempts to measure the performance of 
> Hydra's locking architecture.
> 
> Regards,
> Felix Croes
> ____________________________________________
> https://mail.dworkin.nl/mailman/listinfo/dgd


--
https://www.bartsplace.net/
https://wotf.org/
https://www.flickr.com/photos/mrobjective/