From: "Robin H. Johnson" <robbat2@gentoo.org>
To: gentoo-cluster@lists.gentoo.org
Subject: Re: [gentoo-cluster] cluster or distributed queue, general question
Date: Thu, 10 Jan 2008 06:30:20 -0800 [thread overview]
Message-ID: <20080110143020.GB13680@curie-int.orbis-terrarum.net> (raw)
In-Reply-To: <AD5924F6A589EF419765D39834C6BAFE699FBA@hyves1.exchange.cysonet.com>
[-- Attachment #1: Type: text/plain, Size: 1100 bytes --]
On Thu, Jan 10, 2008 at 02:59:27PM +0100, Jos Houtman wrote:
> For my master thesis I took up a project that requires mapping of a number of statically defined parallel jobs into a more dynamic environment that allows better scaling.
> The situation as described below let me to believe a cluster or distributed queue (DrQueue?) solution is necessary. For the situation see [situation] at the end of this email.
Off the top of my head, many of your requirements are available in two
totally different apps:
- Gearman, written by Brad Fitzpatrick @ LiveJournal. Perl mainly, I
think there are other interfaces as well to it.
- Torque/PBS - somewhat less of a fit, I'm not certain about running
perpetual jobs.
You may also need some degree of STONITH for the job running only once
during node failure case. (Say the job manager crashes, the job is still
running, but you have no control of it. You need to zap it hard).
--
Robin Hugh Johnson
Gentoo Linux Developer & Infra Guy
E-Mail : robbat2@gentoo.org
GnuPG FP : 11AC BA4F 4778 E3F6 E4ED F38E B27B 944E 3488 4E85
[-- Attachment #2: Type: application/pgp-signature, Size: 329 bytes --]
next prev parent reply other threads:[~2008-01-10 14:30 UTC|newest]
Thread overview: 5+ messages / expand[flat|nested] mbox.gz Atom feed top
2008-01-10 13:59 [gentoo-cluster] cluster or distributed queue, general question Jos Houtman
2008-01-10 14:12 ` Panagiotis Christopoulos
2008-01-10 14:30 ` Robin H. Johnson [this message]
2008-01-11 10:26 ` Jos Houtman
2008-01-11 12:02 ` Hanni Ali
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20080110143020.GB13680@curie-int.orbis-terrarum.net \
--to=robbat2@gentoo.org \
--cc=gentoo-cluster@lists.gentoo.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox