[gentoo-user] maintaining clones

[gentoo-user] maintaining clones

[Helmut Jarausch]

Hi,

I have 4 identical machines, they only differ in the 2 files
/etc/conf.d/hostname
/etc/conf.d/net

I'd like to maintain only one of them (updating
GenToo upto several times a week)
and 'rsync' the other ones.

Now, rsync'ing a life root filesystem is risky.
I don't see any problems for the FS holding /usr.

The question is, which directories (files)
are effected by an 'emerge' operation
and which of these do need rsyncing?

E.g. /var/tmp doesn't need to be synchronized
while (probably) /var/lib does.


Many thanks for your help,
Helmut.

-- 
Helmut Jarausch

Lehrstuhl fuer Numerische Mathematik
RWTH - Aachen University
D 52056 Aachen, Germany

Re: [gentoo-user] maintaining clones

[Neil Bothwick]

[-- Attachment #1: Type: text/plain, Size: 955 bytes --]

On Fri, 31 Jul 2009 09:07:14 +0200 (CEST), Helmut Jarausch wrote:

> I have 4 identical machines, they only differ in the 2 files
> /etc/conf.d/hostname
> /etc/conf.d/net
> 
> I'd like to maintain only one of them (updating
> GenToo upto several times a week)
> and 'rsync' the other ones.
> 
> Now, rsync'ing a life root filesystem is risky.
> I don't see any problems for the FS holding /usr.

The whole idea sounds a little risky. I'd use binary packages to keep the
other machines up to date. Set FEATURES="buildpkg" in make.conf on each
computer and set PKGDIR to a directory accessible by all over NFS. Run
your normal emerge -u --whatever world on the first then run the same
with -k on the others. That way they all get the same updates but only
the first has to compile them.

I'd also set up distcc to reduce compile times, but that's a separate
step.


-- 
Neil Bothwick

Very funny Scotty.. now beam down my pants!

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: [gentoo-user] maintaining clones

[Helmut Jarausch]

On 31 Jul, Neil Bothwick wrote:
> On Fri, 31 Jul 2009 09:07:14 +0200 (CEST), Helmut Jarausch wrote:
> 
>> I have 4 identical machines, they only differ in the 2 files
>> /etc/conf.d/hostname
>> /etc/conf.d/net
>> 
>> I'd like to maintain only one of them (updating
>> GenToo upto several times a week)
>> and 'rsync' the other ones.
>> 
>> Now, rsync'ing a life root filesystem is risky.
>> I don't see any problems for the FS holding /usr.
> 
> The whole idea sounds a little risky. I'd use binary packages to keep the
> other machines up to date. Set FEATURES="buildpkg" in make.conf on each
> computer and set PKGDIR to a directory accessible by all over NFS. Run
> your normal emerge -u --whatever world on the first then run the same
> with -k on the others. That way they all get the same updates but only
> the first has to compile them.
> 
> I'd also set up distcc to reduce compile times, but that's a separate
> step.
> 

Thanks for your help!

I've done so in the past but I've made bad experience.
Unfortunately portage isn't so clever, yet.

Many times (on the 'clones', as well) I had to block packages before
emerge and then unblock again. I even had to unmerge some packages
temporarily and emerge them later on again.

Sometime I have to patch an ebuild file (temporarily) and so on.

And let alone updating baselayout.

So, it would be much easier if I could simulate cloning the
machines each day.

Helmut.

-- 
Helmut Jarausch

Lehrstuhl fuer Numerische Mathematik
RWTH - Aachen University
D 52056 Aachen, Germany

Re: [gentoo-user] maintaining clones

[Neil Bothwick]

[-- Attachment #1: Type: text/plain, Size: 701 bytes --]

On Fri, 31 Jul 2009 09:59:02 +0200 (CEST), Helmut Jarausch wrote:

> I've done so in the past but I've made bad experience.
> Unfortunately portage isn't so clever, yet.

portage is a lot cleverer than it used to be, especially with regard to
blocks.

> Many times (on the 'clones', as well) I had to block packages before
> emerge and then unblock again. I even had to unmerge some packages
> temporarily and emerge them later on again.

It sounds like you were using -K rather than -k or trying to build binary
packages with --buildpkgonly, which is doomed to failure when trying to
build dependent packages.


-- 
Neil Bothwick

Maybe... How much are you bribing me this time?

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: [gentoo-user] maintaining clones

[Helmut Jarausch]

On 31 Jul, Neil Bothwick wrote:
> On Fri, 31 Jul 2009 09:59:02 +0200 (CEST), Helmut Jarausch wrote:
> 
>> I've done so in the past but I've made bad experience.
>> Unfortunately portage isn't so clever, yet.
> 
> portage is a lot cleverer than it used to be, especially with regard to
> blocks.

Yes, I'm using portage-2.2_rc33 but still ... 
> 
>> Many times (on the 'clones', as well) I had to block packages before
>> emerge and then unblock again. I even had to unmerge some packages
>> temporarily and emerge them later on again.
> 
> It sounds like you were using -K rather than -k or trying to build binary
> packages with --buildpkgonly, which is doomed to failure when trying to
> build dependent packages.

No, on the 'master' machine I have in /etc/make.conf
FEATURES="buildpkg -stricter"

and on the clones I use '-k'.

But still, portage checks dependencies on the clone.
And, if it sees a problem (like those when upgrading to kde-4
or qt-4.5.x) it refused to merge the binary package.

Helmut.


-- 
Helmut Jarausch

Lehrstuhl fuer Numerische Mathematik
RWTH - Aachen University
D 52056 Aachen, Germany

Re: [gentoo-user] maintaining clones

[Neil Bothwick]

[-- Attachment #1: Type: text/plain, Size: 669 bytes --]

On Fri, 31 Jul 2009 10:46:17 +0200 (CEST), Helmut Jarausch wrote:

> No, on the 'master' machine I have in /etc/make.conf
> FEATURES="buildpkg -stricter"
> 
> and on the clones I use '-k'.
> 
> But still, portage checks dependencies on the clone.
> And, if it sees a problem (like those when upgrading to kde-4
> or qt-4.5.x) it refused to merge the binary package.

Those sort of blocks, such as when switching from monolithic to split
ebuilds, are rare and easily dealt with. Dealing with unexpected side
effects of syncing in-use files could be a lot more problematic.


-- 
Neil Bothwick

All mail what i send is thoughly proof-red, definately!

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: [gentoo-user] maintaining clones

[Dan Farrell]

On Fri, 31 Jul 2009 09:07:14 +0200 (CEST)
Helmut Jarausch <jarausch@igpm.rwth-aachen.de> wrote:

> Hi,
> 
> I have 4 identical machines, they only differ in the 2 files
> /etc/conf.d/hostname
> /etc/conf.d/net
> 
> I'd like to maintain only one of them (updating
> GenToo upto several times a week)
> and 'rsync' the other ones.

hmm... network booting?  network mounting?  install packages once on
one system, share them with everyone.  Share passwd/shadow files and
the like manually, or symlink them to skeletal versions symlinked to
somewhere that can be obscured and replaced by a network boot.  you
could even boot them from thumb drives or cds.  

of course, it would be a good bit of work to configure initially,
and might not go whithout a hitch.  

Re: [gentoo-user] maintaining clones

[Dan Farrell]

On Fri, 31 Jul 2009 09:57:48 +0100
Neil Bothwick <neil@digimed.co.uk> wrote:

> Dealing with unexpected side
> effects of syncing in-use files could be a lot more problematic.

perhaps a digest of some kind?  md5 the files, write up a little script
to keep the rest of the nodes synced up?  

Re: [gentoo-user] maintaining clones

[Ajai Khattri]

On Fri, 31 Jul 2009, Dan Farrell wrote:

> hmm... network booting?  network mounting?  install packages once on
> one system, share them with everyone.  Share passwd/shadow files and
> the like manually, or symlink them to skeletal versions symlinked to
> somewhere that can be obscured and replaced by a network boot.  you
> could even boot them from thumb drives or cds.
>
> of course, it would be a good bit of work to configure initially,
> and might not go whithout a hitch.

For configuration, you may want to look at something like puppet to manage 
that. Your build machine would the puppetmaster and keep the other 
machines' configs up-to-date.



-- 
A

[gentoo-user] Re: maintaining clones

[Nicolas Sebrecht]

On Fri, Jul 31, 2009 at 09:07:14AM +0200, Helmut Jarausch wrote:
> 
> Hi,
> 
> I have 4 identical machines, they only differ in the 2 files
> /etc/conf.d/hostname
> /etc/conf.d/net
> 
> I'd like to maintain only one of them (updating
> GenToo upto several times a week)
> and 'rsync' the other ones.

Why not use a DCVS?

-- 
Nicolas Sebrecht