You mentioned DrQueue, that batch processing app fulfills many of your
requirements. I wrote an ebuild for it too which is on bugs.gentoo

Hanni

On 11/01/2008, Jos Houtman <jos@hyves.nl> wrote:
>
> Thanx for the replies,
>
> I will certainly try the beowulf mailinglist next week,
> The suggestions are great gearman seems to fit except for being perl
> oriented (atleast that's what I read sofar).
> Torque/pbs, don't know about that yet.
>
> Thanx again,
>
> Jos Houtman
> System administrator Hyves.nl
> email: jos@hyves.nl
>
> -----Original Message-----
> From: Robin H. Johnson [mailto:robbat2@gentoo.org]
> Sent: donderdag 10 januari 2008 15:30
> To: gentoo-cluster@lists.gentoo.org
> Subject: Re: [gentoo-cluster] cluster or distributed queue, general
> question
>
> On Thu, Jan 10, 2008 at 02:59:27PM +0100, Jos Houtman wrote:
> > For my master thesis I took up a project that requires mapping of a
> number of statically defined parallel jobs into a more dynamic
> environment that allows better scaling.
> > The situation as described below let me to believe a cluster or
> distributed queue (DrQueue?) solution is necessary. For the situation
> see [situation] at the end of this email.
> Off the top of my head, many of your requirements are available in two
> totally different apps:
> - Gearman, written by Brad Fitzpatrick @ LiveJournal. Perl mainly, I
>   think there are other interfaces as well to it.
> - Torque/PBS - somewhat less of a fit, I'm not certain about running
>   perpetual jobs.
>
> You may also need some degree of STONITH for the job running only once
> during node failure case. (Say the job manager crashes, the job is still
> running, but you have no control of it. You need to zap it hard).
>
> --
> Robin Hugh Johnson
> Gentoo Linux Developer & Infra Guy
> E-Mail     : robbat2@gentoo.org
> GnuPG FP   : 11AC BA4F 4778 E3F6 E4ED  F38E B27B 944E 3488 4E85
> --
> gentoo-cluster@lists.gentoo.org mailing list
>
>


-- 
E-mail: hanni.ali@gmail.com
Mobile: +44 (0) 7985580147
My Blog: http://ainkaboot.co.uk/blogs/hanni/
Website: http://ainkaboot.co.uk http://drqueue.org