[gentoo-user] Hoping someone can help explain distcc to me

[gentoo-user] Hoping someone can help explain distcc to me

[Paul Hartman]

Hi,

I have a fast desktop computer and a slow laptop. Both are ~amd64
Gentoo. After some of the recent discussions about Gentoo on slow
devices, I thought I'd dust off the laptop and try to bring it up to
date.

I'd like to use distcc to make the desktop do all the compiling during
emerges. I've never been able to get distcc working properly, or, at
least, I've never been able to get it working to the point where using
distcc is any faster than not using it at all.

Desktop: 192.168.0.100
Laptop: 192.168.0.107

My goal is to just don't use the laptop at all. Do all compiling on
the desktop. So far, while it appears to be working (some things are
being compiled on the desktop, a very insignificant amount -- CPU
usage on the desktop is basically 0% during emerges on the laptop),
compile times are no better than just compiling everything on the
laptop. The laptop still has 100% CPU activity during emerges. I
realize the laptop is still going to do work during emerges, but
thought at least distcc might be able to help offload most of the
heavy part (the compiling).

Both machines contain "distcc" in FEATURES. It's not using
-march=native. I've tried various -jN values with no real difference
in performance.

On the desktop, /etc/conf.d/distcc contains (among other things):
DISTCCD_OPTS="${DISTCCD_OPTS} --allow 192.168.0.0/24"
DISTCCD_OPTS="${DISTCCD_OPTS} --listen 192.168.0.100"

And /etc/distccd/hosts contains:
192.168.0.0/24

On the laptop, /etc/conf.d/distcc contains (among other things):
DISTCCD_OPTS="${DISTCCD_OPTS} --allow 192.168.0.0/24"
DISTCCD_OPTS="${DISTCCD_OPTS} --listen 192.168.0.107"

And /etc/distccd/hosts contains:
192.168.0.100

Does anyone know any tricks or can tell me if I'm doing something
wrong? How much of a speed-up should I realistically expect? Should
distcc be able to exploit the fast machine to the fullest of its
abilities?

Thanks

Re: [gentoo-user] Hoping someone can help explain distcc to me

[Peter Humphrey]

On Saturday 20 August 2011 23:56:08 Paul Hartman wrote:

> I have a fast desktop computer and a slow laptop. Both are ~amd64
> Gentoo. After some of the recent discussions about Gentoo on slow
> devices, I thought I'd dust off the laptop and try to bring it up to
> date.
> 
> I'd like to use distcc to make the desktop do all the compiling during
> emerges. I've never been able to get distcc working properly, or, at
> least, I've never been able to get it working to the point where using
> distcc is any faster than not using it at all.

Your laptop isn't sending its compilation jobs to your desktop. Have you 
used gkrellm or similar to check for network traffic?

I think I'd take a different approach, one that I dare say you can 
anticipate. By all means take the advice you're about to be offered to get 
distcc working, but why not create a chroot on your desktop, NFS export the 
packages directory from the laptop to it and do the whole job there? Then 
return to the laptop and emerge -k. It does need more keystrokes but it uses 
far less CPU. You just need to copy /var/lib/portage/world, make.conf and 
the /etc/portage tree to the chroot before you start.

I did once get distcc working, but as Neil has observed re Atoms, a lot of 
work is still done before compilation is farmed out to the distcc server, 
rather diluting the benefit of distributed compilation.

-- 
Rgds
Peter		Linux Counter 5290, 1994-04-23

Re: [gentoo-user] Hoping someone can help explain distcc to me

[Paul Hartman]

On Sat, Aug 20, 2011 at 7:35 PM, Peter Humphrey
<peter@humphrey.ukfsn.org> wrote:
> On Saturday 20 August 2011 23:56:08 Paul Hartman wrote:
>
>> I have a fast desktop computer and a slow laptop. Both are ~amd64
>> Gentoo. After some of the recent discussions about Gentoo on slow
>> devices, I thought I'd dust off the laptop and try to bring it up to
>> date.
>>
>> I'd like to use distcc to make the desktop do all the compiling during
>> emerges. I've never been able to get distcc working properly, or, at
>> least, I've never been able to get it working to the point where using
>> distcc is any faster than not using it at all.
>
> Your laptop isn't sending its compilation jobs to your desktop. Have you
> used gkrellm or similar to check for network traffic?

Well, there are definitely some distcc jobs going to the desktop, I
see them, it's just that they finish so fast and there's a "long" time
between them. The logs indicate it is successfully compiling things.
But, it is in no way saturating the resources of the desktop, which is
what I was (perhaps naively) hoping to see.

> I think I'd take a different approach, one that I dare say you can
> anticipate. By all means take the advice you're about to be offered to get
> distcc working, but why not create a chroot on your desktop, NFS export the
> packages directory from the laptop to it and do the whole job there? Then
> return to the laptop and emerge -k. It does need more keystrokes but it uses
> far less CPU. You just need to copy /var/lib/portage/world, make.conf and
> the /etc/portage tree to the chroot before you start.
>
> I did once get distcc working, but as Neil has observed re Atoms, a lot of
> work is still done before compilation is farmed out to the distcc server,
> rather diluting the benefit of distributed compilation.

That was going to be my next approach if distcc just doesn't do enough. :)

Could I just export the entire laptop - everything from the root
directory and below - and chroot into that over the network? Then I
wouldn't even need to emerge -k...

Re: [gentoo-user] Hoping someone can help explain distcc to me

[victor romanchuk]

> Both machines contain "distcc" in FEATURES. It's not using
> -march=native. I've tried various -jN values with no real difference
> in performance.

only client (your laptop) machine should be distcc featured. for server
(desktop) that feature is useless

> On the desktop, /etc/conf.d/distcc contains (among other things):
> DISTCCD_OPTS="${DISTCCD_OPTS} --allow 192.168.0.0/24"
> DISTCCD_OPTS="${DISTCCD_OPTS} --listen 192.168.0.100"

this is server distcc daemon configuration, one just instructs daemon on what
network interface to listen for incoming connections (interface with ip
192.168.0.100 in your case) and filter incoming distcc connections by source
address: allow only those coming from local network machines with ip addresses
192.168.0.1 to 192.168.0.254

then distccd have to be started: /etc/init.d/distccd start

> And /etc/distccd/hosts contains:
> 192.168.0.0/24

this file configures distcc client behavior (actually the configuration can be
complex, see distcc man page for details), but in trivial case (for home
computing) it might look like:

192.168.0.100/6 127.0.0.1/1

e.g the client is able to send up to 6 distcc jobs to 192.168.0.100 and limit to
one job at local machine. client's /etc/make.conf has to have distcc feature
enabled (FEATURES="distcc"). surely you can play with job distribution rules
around the network. `distcc --show-hosts` command displays what you configured.
the number of cuncurrently running jobs (-j flag) has to be not less than sum of
local and remote jobs

i had noticed that distcc is peevish about CFLAGS: these should be compatible on
both client and server. in my case i made these similar on both machines (laptop
is core2duo and desktop is core2quad; both are running  amd64 arch)

yet another way to install packages on weak notebook running it on the same arch
as desktop runs, - is to create binaries at powerful machine (while emerging or
with quickpkg utility) and share $PKGDIR with laptop

hth

Re: [gentoo-user] Hoping someone can help explain distcc to me

[Peter Humphrey]

On Sunday 21 August 2011 02:08:51 Paul Hartman wrote:

> Could I just export the entire laptop - everything from the root
> directory and below - and chroot into that over the network? Then I
> wouldn't even need to emerge -k...

No, I tried that and got myself tied in knots - well, actually it was the 
whole portage tree that I exported, not the entire system. I forget what 
went wrong now, but it's definitely cleaner to tell the server to build the 
packages and the client to install from them. The emerge -k step is quick 
too, and you have the advantage that you can see whether the packages are 
actually there, unless you've switched colours off or not specified -v. (I 
once found that they weren't there, which prompted me to go looking for the 
config problem. Like Dale, I'm quite a good tester!)

You just have to make sure that the chroot is identical to the client.

-- 
Rgds
Peter		Linux Counter 5290, 1994-04-23

Re: [gentoo-user] Hoping someone can help explain distcc to me

[Dale]

Peter Humphrey wrote:
> On Sunday 21 August 2011 02:08:51 Paul Hartman wrote:
>
>    
>> Could I just export the entire laptop - everything from the root
>> directory and below - and chroot into that over the network? Then I
>> wouldn't even need to emerge -k...
>>      
> No, I tried that and got myself tied in knots - well, actually it was the
> whole portage tree that I exported, not the entire system. I forget what
> went wrong now, but it's definitely cleaner to tell the server to build the
> packages and the client to install from them. The emerge -k step is quick
> too, and you have the advantage that you can see whether the packages are
> actually there, unless you've switched colours off or not specified -v. (I
> once found that they weren't there, which prompted me to go looking for the
> config problem. Like Dale, I'm quite a good tester!)
>
> You just have to make sure that the chroot is identical to the client.
>
>    

Since you mentioned me.  I wish I could set up a quicky from my 4 core 
64 bit machine to compile 32 bit packages for a older 2GHz machine that 
belongs to a friend.  I was going to put Mandriva on it but the CD won;t 
boot up properly.  It stops at starting udev.  Grrrrr.

How hard is it to set up a 64 bit machine to compile programs for a 32 
bit system?

Dale

:-)  :-)

Re: [gentoo-user] Hoping someone can help explain distcc to me

[Matthew Finkel]

[-- Attachment #1: Type: text/plain, Size: 1640 bytes --]

On Sat, Aug 20, 2011 at 10:46 PM, Dale <rdalek1967@gmail.com> wrote:

> Peter Humphrey wrote:
>
>> On Sunday 21 August 2011 02:08:51 Paul Hartman wrote:
>>
>>
>>
>>> Could I just export the entire laptop - everything from the root
>>> directory and below - and chroot into that over the network? Then I
>>> wouldn't even need to emerge -k...
>>>
>>>
>> No, I tried that and got myself tied in knots - well, actually it was the
>> whole portage tree that I exported, not the entire system. I forget what
>> went wrong now, but it's definitely cleaner to tell the server to build
>> the
>> packages and the client to install from them. The emerge -k step is quick
>> too, and you have the advantage that you can see whether the packages are
>> actually there, unless you've switched colours off or not specified -v. (I
>> once found that they weren't there, which prompted me to go looking for
>> the
>> config problem. Like Dale, I'm quite a good tester!)
>>
>> You just have to make sure that the chroot is identical to the client.
>>
>>
>>
>
> Since you mentioned me.  I wish I could set up a quicky from my 4 core 64
> bit machine to compile 32 bit packages for a older 2GHz machine that belongs
> to a friend.  I was going to put Mandriva on it but the CD won;t boot up
> properly.  It stops at starting udev.  Grrrrr.
>
> How hard is it to set up a 64 bit machine to compile programs for a 32 bit
> system?
>
> Dale
>
> :-)  :-)
>
>
It's actually quite easy. IIRC, when I did it last, the only difference is
that when you chroot into the subsystem you need prefix the command with
linux32, e.g. linux32 chroot /path/to/chroot /bin/bash

[-- Attachment #2: Type: text/html, Size: 2262 bytes --]

Re: [gentoo-user] Hoping someone can help explain distcc to me

[Peter Humphrey]

On Sunday 21 August 2011 04:04:05 Matthew Finkel wrote:

> On Sat, Aug 20, 2011 at 10:46 PM, Dale <rdalek1967@gmail.com> wrote:
> > How hard is it to set up a 64 bit machine to compile programs for a 32
> > bit system?
> > 
> > Dale
> > 
> > :-)  :-)
> 
> It's actually quite easy. IIRC, when I did it last, the only difference
> is that when you chroot into the subsystem you need prefix the command
> with linux32, e.g. linux32 chroot /path/to/chroot /bin/bash

Yes, just follow this guide: 
http://www.gentoo.org/proj/en/base/amd64/howtos/chroot.xml

I did that and it was straightforward as far as I remember. I did spend some 
time thinking at a few stages, to get an understanding of what I was doing 
rather than just blindly following somebody else's prescription.

Then it's a matter of writing some simple scripts to mount the packages 
directory on the big host. Here's mine, most of which I scrounged from 
somewhere:

$ cat /etc/init.d/atom
#!/sbin/runscript

depend() {
   need localmount
   need bootmisc
}

start() {
    ebegin "Mounting 32-bit chroot dirs"
    mount -o bind /dev /mnt/atom/dev >/dev/null
    mount -o bind /dev/pts /mnt/atom/dev/pts >/dev/null
    mount -o bind /dev/shm /mnt/atom/dev/shm >/dev/null
    mount -t proc /proc /mnt/atom/proc >/dev/null
    mount -o bind /sys /mnt/atom/sys >/dev/null
    mount -o bind /tmp /mnt/atom/tmp >/dev/null
    mount -t nfs -o vers=3 192.168.2.2:/usr/portage/packages 
/mnt/atom/usr/portage/packages
    eend $? "An error occurred while attempting to mount 32-bit chroot 
directories"
    ebegin "Copying 32-bit chroot files"
    cp -pf /etc/resolv.conf /mnt/atom/etc/ >/dev/null
    cp -pf /etc/passwd /mnt/atom/etc/ >/dev/null
    cp -pf /etc/shadow /mnt/atom/etc/ >/dev/null
    cp -pf /etc/group /mnt/atom/etc/ >/dev/null
    cp -pf /etc/hosts /mnt/atom/etc/ > /dev/null
    cp -Ppf /etc/localtime /mnt/atom/etc/ >/dev/null
    eend $? "An error occurred while attempting to copy 32-bit chroot files."
}

stop() {
    ebegin "Unmounting 32-bit chroot dirs"
    umount -f /mnt/atom/dev/pts >/dev/null
    umount -f /mnt/atom/dev/shm >/dev/null
    umount -f /mnt/atom/dev >/dev/null
    umount -f /mnt/atom/proc >/dev/null
    umount -f /mnt/atom/sys >/dev/null
    umount -f /mnt/atom/tmp >/dev/null
    umount -f /mnt/atom/usr/portage/packages >/dev/null
    eend $? "An error occurred while attempting to unmount 32-bit chroot 
directories"
}

I could list the steps of my daily routine to upgrade both the client and 
the chroot if that would help.

-- 
Rgds
Peter		Linux Counter 5290, 1994-04-23

Re: [gentoo-user] Hoping someone can help explain distcc to me

[Alex Schuster]

victor romanchuk writes:

> > Both machines contain "distcc" in FEATURES. It's not using
> > -march=native. I've tried various -jN values with no real difference
> > in performance.

-jN in make.conf's MAPEOPTS variable I assume, not as argument to emerge, 
which does something different. It also hides the normal compile messages, 
in case of problems you should see something like 'failed to distribute to 
<host>'.
And syslog on the desktop should list every distributed job.

Distcc helps a lot here when I use Gentoo on slow systems. Although they are 
still slow, because of time spent for configuring, or linking.

And maybe you are also using ccache, and your tests were recompiles of stuff 
already cached, so the slow machine is fast at this?


> i had noticed that distcc is peevish about CFLAGS: these should be
> compatible on both client and server. in my case i made these similar on
> both machines (laptop is core2duo and desktop is core2quad; both are
> running  amd64 arch)

I don't think this is true - as long as the CHOST is identical, there should 
be no problem.

> yet another way to install packages on weak notebook running it on the
> same arch as desktop runs, - is to create binaries at powerful machine
> (while emerging or with quickpkg utility) and share $PKGDIR with laptop

This means some extra work, and also use flags need to be compatible, but 
the speedup would be much bigger than with just distcc.

What about exporting the whole root file system and mounting it on the fast 
desktop, chrooting and emerging?

And then there's Sabayon, based on Gentoo, but with binary packages. And 
when you want to customize things (use different use flags), you can still 
compile stuff manually. I will try this for a slow desktop PC now, all this 
compiling is so annoying.

	Wonko

Re: [gentoo-user] Hoping someone can help explain distcc to me

[Joost Roeleveld]

On Sunday, August 21, 2011 10:41:56 AM Peter Humphrey wrote:
> On Sunday 21 August 2011 04:04:05 Matthew Finkel wrote:
> > On Sat, Aug 20, 2011 at 10:46 PM, Dale <rdalek1967@gmail.com> wrote:
> > > How hard is it to set up a 64 bit machine to compile programs for a
> > > 32
> > > bit system?
> > > 
> > > Dale
> > > 
> > > :-)  :-)
> > 
> > It's actually quite easy. IIRC, when I did it last, the only difference
> > is that when you chroot into the subsystem you need prefix the command
> > with linux32, e.g. linux32 chroot /path/to/chroot /bin/bash
> 
> Yes, just follow this guide:
> http://www.gentoo.org/proj/en/base/amd64/howtos/chroot.xml
> 
> I did that and it was straightforward as far as I remember. I did spend some
> time thinking at a few stages, to get an understanding of what I was doing
> rather than just blindly following somebody else's prescription.
> 
> Then it's a matter of writing some simple scripts to mount the packages
> directory on the big host. Here's mine, most of which I scrounged from
> somewhere:
> 
> $ cat /etc/init.d/atom
> #!/sbin/runscript
> 
> depend() {
>    need localmount
>    need bootmisc
> }
> 
> start() {
>     ebegin "Mounting 32-bit chroot dirs"
>     mount -o bind /dev /mnt/atom/dev >/dev/null
>     mount -o bind /dev/pts /mnt/atom/dev/pts >/dev/null
>     mount -o bind /dev/shm /mnt/atom/dev/shm >/dev/null
>     mount -t proc /proc /mnt/atom/proc >/dev/null
>     mount -o bind /sys /mnt/atom/sys >/dev/null
>     mount -o bind /tmp /mnt/atom/tmp >/dev/null
>     mount -t nfs -o vers=3 192.168.2.2:/usr/portage/packages
> /mnt/atom/usr/portage/packages
>     eend $? "An error occurred while attempting to mount 32-bit chroot
> directories"
>     ebegin "Copying 32-bit chroot files"
>     cp -pf /etc/resolv.conf /mnt/atom/etc/ >/dev/null
>     cp -pf /etc/passwd /mnt/atom/etc/ >/dev/null
>     cp -pf /etc/shadow /mnt/atom/etc/ >/dev/null
>     cp -pf /etc/group /mnt/atom/etc/ >/dev/null
>     cp -pf /etc/hosts /mnt/atom/etc/ > /dev/null
>     cp -Ppf /etc/localtime /mnt/atom/etc/ >/dev/null
>     eend $? "An error occurred while attempting to copy 32-bit chroot
> files." }
> 
> stop() {
>     ebegin "Unmounting 32-bit chroot dirs"
>     umount -f /mnt/atom/dev/pts >/dev/null
>     umount -f /mnt/atom/dev/shm >/dev/null
>     umount -f /mnt/atom/dev >/dev/null
>     umount -f /mnt/atom/proc >/dev/null
>     umount -f /mnt/atom/sys >/dev/null
>     umount -f /mnt/atom/tmp >/dev/null
>     umount -f /mnt/atom/usr/portage/packages >/dev/null
>     eend $? "An error occurred while attempting to unmount 32-bit chroot
> directories"
> }
> 
> I could list the steps of my daily routine to upgrade both the client and
> the chroot if that would help.

That would help as I'm planning on setting this up myself as well for my 
netbook.

Is there a way to automate the steps inside the chroot without having to have 
a script inside the chroot?

--
Joost

Re: [gentoo-user] Hoping someone can help explain distcc to me

[victor romanchuk]

>
>> i had noticed that distcc is peevish about CFLAGS: these should be
>> compatible on both client and server. in my case i made these similar on
>> both machines (laptop is core2duo and desktop is core2quad; both are
>> running  amd64 arch)
> I don't think this is true - as long as the CHOST is identical, there should 
> be no problem.

CHOST defines the arch (i686, amd64, arm ..) whilst CFLAGS control gcc behavior
and the binary code generation produced by compiler. in my case core2quad
(q8300) i'm using in the desktop supports sse4.1 instruction set and notebook
powered with core2duo (t7600) does not have that cpu feature. having option
`-msse4.1' set in CFLAGS at desktop side will causes frequent compilation
failures (initiated by distcc) or, in worst case - arbitrary crashes at notebook
when running binaries compiled in distributed distcc environment

>> yet another way to install packages on weak notebook running it on the
>> same arch as desktop runs, - is to create binaries at powerful machine
>> (while emerging or with quickpkg utility) and share $PKGDIR with laptop
> This means some extra work, and also use flags need to be compatible, but 
> the speedup would be much bigger than with just distcc.
>
> What about exporting the whole root file system and mounting it on the fast 
> desktop, chrooting and emerging?
>

i do not insist the distcc is the only or most efficient way to maintain gentoo
installation on a slow machine (having something more powerful nearby).  just
tried to explain how does distcc work

victor

Re: [gentoo-user] Hoping someone can help explain distcc to me

[Peter Humphrey]

On Sunday 21 August 2011 14:53:15 Joost Roeleveld wrote:

> That would help as I'm planning on setting this up myself as well for my
> netbook.

Right. I have two Konsoles open on my workstation, which is the compilation host. In one I "su -" and in the 
other I "ssh serv" (this is the client Atom box, which among other things runs http-replicator to serve the 
portage tree to the LAN). Naming is going to get confusing if I'm not careful, so I'll refer to the ssh session as 
"Atom" and the compilation host as "Host". Then my steps are:

Host:	# /etc/init.d/atom start	(this is the script I showed yesterday)
		# linux32 chroot /mnt/atom /bin/bash
		# env-update && . /etc/profile

Atom:	$ sudo emerge --sync && sudo eix-update

Host:	# emerge --sync && emerge -auvD -j 5 --changed-use --keep-going world

Atom:	$ sudo emerge -auDkv --jobs=3 --changed-use --with-bdeps y --keep-going world

Host:	(various clean-up operations such as depclean, eclean and localepurge)
		# exit
		# /etc/init.d/atom stop

Atom:	(similar cleaning up)
		$ exit

That's it as far as I remember.

> Is there a way to automate the steps inside the chroot without having to
> have a script inside the chroot?

I'd be reluctant to try to automate it any more than this. It's about as simple to use as can be and as I want it. 
I've set up aliases for most of those long commands to save my wrists, and of course command-line recall is 
wonderful. The task that takes longest is portage on the Atom calculating what packages it needs to emerge 
from.

I try not to forget to copy any USE-flag changes etc between the Atom and the chroot, but of course I'm no 
more than human.

-- 
Rgds
Peter		Linux Counter 5290, 1994-04-23

Re: [gentoo-user] Hoping someone can help explain distcc to me

[Neil Bothwick]

[-- Attachment #1.1: Type: text/plain, Size: 515 bytes --]

On Sun, 21 Aug 2011 15:53:15 +0200, Joost Roeleveld wrote:

> Is there a way to automate the steps inside the chroot without having
> to have a script inside the chroot?

This is the script I use. You can call it with "ch hostname" or symlink
it to chhostname. That way I can use the same script for multiple
computers. The -w option causes it to enter the chroot, emerge world and
exit.


-- 
Neil Bothwick

Printer: (n.) a small box attached to a computer and used to start
fires in cold weather.

[-- Attachment #1.2: ch --]
[-- Type: application/octet-stream, Size: 3378 bytes --]

#!/usr/bin/python2.7
# -*- coding: iso-8859-1 -*-

import argparse, os, socket, subprocess, sys

# read arguments
parser = argparse.ArgumentParser(description='Enter chroot build environment for a host')
parser.add_argument('-H', '--host')
parser.add_argument('-k', '--kernel', action='store_true')
parser.add_argument('-c' , '--command')
parser.add_argument('-e', '--exit', action='store_true')
parser.add_argument('-w', '--world', action='store_true')
parser.add_argument('-v', '--verbose', action='store_true')
args = parser.parse_args()

# must be run as root
if os.getuid() != 0:
	print '\n%smust be run as root!\n' % os.path.basename(sys.argv[0])
	parser.print_help()
	sys.exit(1)

# get host from name or command line
if os.path.basename(sys.argv[0]) != 'ch':
	args.host = os.path.basename(sys.argv[0])[2:]

if not args.host:
	parser.print_help()
	sys.exit(1)

if args.verbose:
	print 'Creating autorun file'

# create .autorun file if asked
if args.world:
	args.command = 'world-update'
	args.exit = True

if args.command:
	arun = open('/mnt/%s/.autorun' % args.host, 'w')
	arun.write('%s\n' % args.command)
	if args.exit:
		arun.write('exit\n')
	arun.close()
else:
	try:
		os.remove('/mnt/%s/.autorun' % args.host)
	except OSError:
		pass

if args.verbose:
	print 'Pinging host'

# check if host is available for syncing
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.settimeout(10)
try:
	s.connect((args.host, 22))
	s.close()
	sync_paths = ['etc/portage/', 'var/lib/portage/world', 'var/lib/portage/world_sets', 'usr/local/bin/', 'home/nelz/bin/']
	sync_cmd = ['rsync', '--archive', '--delete-before']

	if args.verbose:
		print 'Syncing host'
		sync_cmd.append('--verbose')

	if args.kernel:
		print 'Syncing kernel sources... be patient'
		sync_paths.extend(['usr/src/', 'lib/modules/', 'boot/'])

	for path in sync_paths:
		if os.path.exists(os.path.join('/mnt/', args.host, path)):
			src_dest = ['%s:/%s' % (args.host, path), os.path.join('/mnt/', args.host, path)]
			if args.verbose:
				print 'syncing', ' to '.join(src_dest)
			subprocess.call(sync_cmd + src_dest)

except socket.error:
	print args.host, 'not available, syncing skipped'

if args.verbose:
	print 'Setting up chroot mounts'

# set up mounts in the chroot
try:
	os.mkdir('/var/tmp/%s' % args.host, 1777)
except OSError:
	pass

subprocess.call(['mount', '-t', 'proc', 'none', '/mnt/%s/proc' % args.host])
subprocess.call(['mount', '--bind', '/var/tmp/%s' % args.host, '/mnt/%s/var/tmp' % args.host])
for dir in ('dev', 'var/portage', 'mnt/portage'):
	subprocess.call(['mount', '--bind', '/%s' % dir, '/mnt/%s/%s' % (args.host, dir)])
subprocess.call(['cp', '-f', '/etc/resolv.conf', '/mnt/%s/etc/' % args.host])

if args.verbose:
	print 'Entering chroot'

# enter chroot
if os.path.exists('/mnt/%s/bin/zsh' % args.host):
	#subprocess.call(['chroot', '/mnt/%s' % args.host, '/bin/zsh'])
	subprocess.call(['su' , '-', '-c', 'chroot /mnt/%s /bin/zsh' % args.host])
else:
	#subprocess.call(['chroot', '/mnt/%s' % args.host, '/bin/bash'])
	subprocess.call(['su' , '-', '-c', 'chroot /mnt/%s /bin/bash' % args.host])

if args.verbose:
	print 'Cleaning up'

# clean up after exiting chroot
try:
	os.remove('/mnt/%s/.autorun' % args.host)
except OSError:
	pass

for dir in ('var/tmp', 'mnt/portage', 'var/portage', 'proc', 'dev'):
	subprocess.call(['umount' , '/mnt/%s/%s' % (args.host, dir)])

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

[gentoo-user] Re: Hoping someone can help explain distcc to me

[Paul Hartman]

On Sat, Aug 20, 2011 at 5:56 PM, Paul Hartman
<paul.hartman+gentoo@gmail.com> wrote:
>
> I'd like to use distcc to make the desktop do all the compiling during
> emerges. I've never been able to get distcc working properly, or, at
> least, I've never been able to get it working to the point where using
> distcc is any faster than not using it at all.

Thanks to all who replied for your suggestions and info! After
experimenting with various combinations of settings it seems to be
working now. I think I was too aggressive with the number of parallel
make jobs, the laptop just couldn't coordinate that many. I also
stopped distccd on the laptop, in case it was picking up jobs
unintentionally.

Here's the results, a real-life test with ImageMagick 6.7.1.0. The
machines were idle, files were already downloaded, and I don't use
ccache, so it should be actual compile time comparison only:

Normal emerge on laptop, no parallel jobs in MAKEOPTS, no distcc:
merge time: 8 minutes and 27 seconds.

With -j3 in MAKEOPTS, using distcc:
merge time: 4 minutes and 40 seconds.

Great!

Of course, on the desktop it only takes 1 minutes 17 seconds, but an
almost 50% reduction in emerge time on the laptop is still great to
see.

Re: [gentoo-user] Hoping someone can help explain distcc to me

[Alex Schuster]

victor romanchuk writes:

> >> i had noticed that distcc is peevish about CFLAGS: these should be
> >> compatible on both client and server. in my case i made these
> >> similar on both machines (laptop is core2duo and desktop is
> >> core2quad; both are running  amd64 arch)
> > 
> > I don't think this is true - as long as the CHOST is identical, there
> > should be no problem.
> 
> CHOST defines the arch (i686, amd64, arm ..) whilst CFLAGS control gcc
> behavior and the binary code generation produced by compiler. in my case
> core2quad (q8300) i'm using in the desktop supports sse4.1 instruction
> set and notebook powered with core2duo (t7600) does not have that cpu
> feature. having option `-msse4.1' set in CFLAGS at desktop side will
> causes frequent compilation failures (initiated by distcc) or, in worst
> case - arbitrary crashes at notebook when running binaries compiled in
> distributed distcc environment

But how could the gcc on the desktop, called via distcc, even see the CFLAGS 
in the desktops make.conf? This just cannot be.
I have been using march=athlon-xp on my desktop, and yet used it as distcc 
server for my Pentium-III notebook for years. Worked fine.

	Wonko

Re: [gentoo-user] Hoping someone can help explain distcc to me

[Bill Longman]

On 08/22/2011 01:41 PM, Alex Schuster wrote:
>>> > > I don't think this is true - as long as the CHOST is identical, there
>>> > > should be no problem.
>> > 
>> > CHOST defines the arch (i686, amd64, arm ..) whilst CFLAGS control gcc
>> > behavior and the binary code generation produced by compiler. in my case
>> > core2quad (q8300) i'm using in the desktop supports sse4.1 instruction
>> > set and notebook powered with core2duo (t7600) does not have that cpu
>> > feature. having option `-msse4.1' set in CFLAGS at desktop side will
>> > causes frequent compilation failures (initiated by distcc) or, in worst
>> > case - arbitrary crashes at notebook when running binaries compiled in
>> > distributed distcc environment
> But how could the gcc on the desktop, called via distcc, even see the CFLAGS 
> in the desktops make.conf? This just cannot be.
> I have been using march=athlon-xp on my desktop, and yet used it as distcc 
> server for my Pentium-III notebook for years. Worked fine.

Re: [gentoo-user] Hoping someone can help explain distcc to me

[Bill Longman]

On 08/22/2011 01:41 PM, Alex Schuster wrote:
>>> > > I don't think this is true - as long as the CHOST is identical, there
>>> > > should be no problem.
>> > 
>> > CHOST defines the arch (i686, amd64, arm ..) whilst CFLAGS control gcc
>> > behavior and the binary code generation produced by compiler. in my case
>> > core2quad (q8300) i'm using in the desktop supports sse4.1 instruction
>> > set and notebook powered with core2duo (t7600) does not have that cpu
>> > feature. having option `-msse4.1' set in CFLAGS at desktop side will
>> > causes frequent compilation failures (initiated by distcc) or, in worst
>> > case - arbitrary crashes at notebook when running binaries compiled in
>> > distributed distcc environment
> But how could the gcc on the desktop, called via distcc, even see the CFLAGS 
> in the desktops make.conf? This just cannot be.
> I have been using march=athlon-xp on my desktop, and yet used it as distcc 
> server for my Pentium-III notebook for years. Worked fine.

The arguments sent to gcc tell it what kind of object gcc is supposed to
create. If gcc cannot create the object you request because it's not the
right "kind" of gcc, implies that, within the same gcc version, it is
unable to reproduce object files.

Re: [gentoo-user] Hoping someone can help explain distcc to me

[Dale]

Peter Humphrey wrote:
> On Sunday 21 August 2011 04:04:05 Matthew Finkel wrote:
>
>    
>> On Sat, Aug 20, 2011 at 10:46 PM, Dale<rdalek1967@gmail.com>  wrote:
>>      
>>> How hard is it to set up a 64 bit machine to compile programs for a 32
>>> bit system?
>>>
>>> Dale
>>>
>>> :-)  :-)
>>>        
>> It's actually quite easy. IIRC, when I did it last, the only difference
>> is that when you chroot into the subsystem you need prefix the command
>> with linux32, e.g. linux32 chroot /path/to/chroot /bin/bash
>>      
> Yes, just follow this guide:
> http://www.gentoo.org/proj/en/base/amd64/howtos/chroot.xml
>
> I did that and it was straightforward as far as I remember. I did spend some
> time thinking at a few stages, to get an understanding of what I was doing
> rather than just blindly following somebody else's prescription.
>
> Then it's a matter of writing some simple scripts to mount the packages
> directory on the big host. Here's mine, most of which I scrounged from
> somewhere:
>
> $ cat /etc/init.d/atom
> #!/sbin/runscript
>
> depend() {
>     need localmount
>     need bootmisc
> }
>
> start() {
>      ebegin "Mounting 32-bit chroot dirs"
>      mount -o bind /dev /mnt/atom/dev>/dev/null
>      mount -o bind /dev/pts /mnt/atom/dev/pts>/dev/null
>      mount -o bind /dev/shm /mnt/atom/dev/shm>/dev/null
>      mount -t proc /proc /mnt/atom/proc>/dev/null
>      mount -o bind /sys /mnt/atom/sys>/dev/null
>      mount -o bind /tmp /mnt/atom/tmp>/dev/null
>      mount -t nfs -o vers=3 192.168.2.2:/usr/portage/packages
> /mnt/atom/usr/portage/packages
>      eend $? "An error occurred while attempting to mount 32-bit chroot
> directories"
>      ebegin "Copying 32-bit chroot files"
>      cp -pf /etc/resolv.conf /mnt/atom/etc/>/dev/null
>      cp -pf /etc/passwd /mnt/atom/etc/>/dev/null
>      cp -pf /etc/shadow /mnt/atom/etc/>/dev/null
>      cp -pf /etc/group /mnt/atom/etc/>/dev/null
>      cp -pf /etc/hosts /mnt/atom/etc/>  /dev/null
>      cp -Ppf /etc/localtime /mnt/atom/etc/>/dev/null
>      eend $? "An error occurred while attempting to copy 32-bit chroot files."
> }
>
> stop() {
>      ebegin "Unmounting 32-bit chroot dirs"
>      umount -f /mnt/atom/dev/pts>/dev/null
>      umount -f /mnt/atom/dev/shm>/dev/null
>      umount -f /mnt/atom/dev>/dev/null
>      umount -f /mnt/atom/proc>/dev/null
>      umount -f /mnt/atom/sys>/dev/null
>      umount -f /mnt/atom/tmp>/dev/null
>      umount -f /mnt/atom/usr/portage/packages>/dev/null
>      eend $? "An error occurred while attempting to unmount 32-bit chroot
> directories"
> }
>
> I could list the steps of my daily routine to upgrade both the client and
> the chroot if that would help.
>
>    

I followed the guide and some info here and it seems to have worked 
great.  I'm glad.  The cooler on the heatsink is VERY whimpy.  It's 
aluminum and has a little 60mm fan on it.  I didn't measure it but its 
certainly not as big as my 80mm.  I can barely feel the air it moves.  
Compiling on htat would likely be a problem pretty quick.

Now to get the install moved over to the other rig.  DVD and a tarball 
maybe?  Hmmmm, I got to think on this one.  It's almost 5Gbs worth.  
Maybe man scp will help.  ;-)

Thanks for the link.  This was awesome!!

Dale

:-)  :-)

P. S.  Now I can build binaries for my old rig too.  :-D