[gentoo-user] distcc - amd64 and x86

[gentoo-user] distcc - amd64 and x86

[Samuraiii]

[-- Attachment #1: Type: text/plain, Size: 964 bytes --]

Hello,

I have (right now) 3 computers runing Gentoo and as two of them are with
only 1GB of ram - which for libreoffice compiling is not enough.

So my questions are:

1) Is it possible with distcc overcome this memory limitation? -call
compile on "weak" machine and leave memory load on "strong" one

2) second question is about arch of each computer
      one is Core2Duo second Athlon 64 X2 and last is Pentium M
      What steps I need to take to exploit distcc for all given machines
      Do I need specific toolchains for each arch in question?

3) How is distcc prone to network failures?

Thank you in advance

S



-- 
Samuraiii
e-mail: samuraiii@volny.cz <mailto:samuraiii@volny.cz>
GnuPG key ID: 0x80C752EA
<http://pgp.mit.edu:11371/pks/lookup?search=0x80C752EA&op=vindex&fingerprint=on&exact=on>
(obtainable on http://pgp.mit.edu)
Full copy of public timestamp block <http://publictimestamp.org>
signatures id- (from ) is included in header of html.

[-- Attachment #2.1: Type: text/html, Size: 1956 bytes --]

[-- Attachment #2.2: bg-linky.gif --]
[-- Type: image/gif, Size: 55 bytes --]

Re: [gentoo-user] distcc - amd64 and x86

[Michael Mol]

On Sun, Feb 5, 2012 at 12:04 PM, Samuraiii <samuraiii@volny.cz> wrote:
> Hello,
>
> I have (right now) 3 computers runing Gentoo and as two of them are with
> only 1GB of ram - which for libreoffice compiling is not enough.
>
> So my questions are:
>
> 1) Is it possible with distcc overcome this memory limitation? -call compile
> on "weak" machine and leave memory load on "strong" one

To a limited extent, yes. You could configure things such that
compiles happen remotely, but links always have to happen locally. And
anything not done with a C or C++ compiler happens locally.

AFAICT, any C or C++ app's most memory-consumptive act is linking.

Your better bet is probably going to be to add swap.

>
> 2) second question is about arch of each computer
>       one is Core2Duo second Athlon 64 X2 and last is Pentium M
>       What steps I need to take to exploit distcc for all given machines
>       Do I need specific toolchains for each arch in question?

Yes. But I don't know the routine for this. I'll be watching this
thread myself, as I'd love to get some distcc crosscompile magic going
at home.

>
> 3) How is distcc prone to network failures?

If a dispatched compile fails (either because of a failure on the
remote end, or because of some kind of network glitch), the compile is
tried again locally.

If a remote distccd can't be reached, the local dispatcher won't try
again for a configurable amount of time. By default, that's one
minute.

-- 
:wq

[gentoo-user] «-»: [gentoo-user] distcc - amd64 and x86

[Samuraiii]

[-- Attachment #1: Type: text/plain, Size: 1359 bytes --]





On 2012-02-05 18:38, Michael Mol wrote:
> On Sun, Feb 5, 2012 at 12:04 PM, Samuraiii <samuraiii@volny.cz> wrote:
>> Hello,
>>
>> I have (right now) 3 computers runing Gentoo and as two of them are with
>> only 1GB of ram - which for libreoffice compiling is not enough.
>>
>> So my questions are:
>>
>> 1) Is it possible with distcc overcome this memory limitation? -call compile
>> on "weak" machine and leave memory load on "strong" one
> To a limited extent, yes. You could configure things such that
> compiles happen remotely, but links always have to happen locally. And
> anything not done with a C or C++ compiler happens locally.
>
> AFAICT, any C or C++ app's most memory-consumptive act is linking.
>
> Your better bet is probably going to be to add swap.
The swap thing is good idea but wont work because (libreoffice is one of
examples) ebuild checks for available ram not swap so when there is eg.
962MB of ram it fails right in begining of merge

I myself have really huge (4GB+) swap for some reasons
-- 
Samuraiii
e-mail: samuraiii@volny.cz <mailto:samuraiii@volny.cz>
GnuPG key ID: 0x80C752EA
<http://pgp.mit.edu:11371/pks/lookup?search=0x80C752EA&op=vindex&fingerprint=on&exact=on>
(obtainable on http://pgp.mit.edu)
Full copy of public timestamp block <http://publictimestamp.org>
signatures id- (from ) is included in header of html.

[-- Attachment #2.1: Type: text/html, Size: 2533 bytes --]

[-- Attachment #2.2: bg-linky.gif --]
[-- Type: image/gif, Size: 55 bytes --]

Re: [gentoo-user] «-»: [gentoo-user] distcc - amd64 and x86

[Dale]

Samuraiii wrote:
> 
> 
> 
> 
> On 2012-02-05 18:38, Michael Mol wrote:
>> On Sun, Feb 5, 2012 at 12:04 PM, Samuraiii <samuraiii@volny.cz> wrote:
>>> Hello,
>>>
>>> I have (right now) 3 computers runing Gentoo and as two of them are with
>>> only 1GB of ram - which for libreoffice compiling is not enough.
>>>
>>> So my questions are:
>>>
>>> 1) Is it possible with distcc overcome this memory limitation? -call compile
>>> on "weak" machine and leave memory load on "strong" one
>> To a limited extent, yes. You could configure things such that
>> compiles happen remotely, but links always have to happen locally. And
>> anything not done with a C or C++ compiler happens locally.
>>
>> AFAICT, any C or C++ app's most memory-consumptive act is linking.
>>
>> Your better bet is probably going to be to add swap.
> The swap thing is good idea but wont work because (libreoffice is one of
> examples) ebuild checks for available ram not swap so when there is eg.
> 962MB of ram it fails right in begining of merge
> 
> I myself have really huge (4GB+) swap for some reasons
> -- 
> Samuraiii
> e-mail: samuraiii@volny.cz <mailto:samuraiii@volny.cz>
> GnuPG key ID: 0x80C752EA
> <http://pgp.mit.edu:11371/pks/lookup?search=0x80C752EA&op=vindex&fingerprint=on&exact=on>
> (obtainable on http://pgp.mit.edu)
> Full copy of public timestamp block <http://publictimestamp.org>
> signatures id- (from ) is included in header of html.


It does the same for portages work space too.  I have portages work
directory on tmpfs and I always have to mount with the size=12g option
so that LOo will even start.

Thing is, it rarely uses more than 4Gbs or so.  Is there a way to
disable this mess?  I got the space for my compile.  I really don't need
the checks.  Another reason for the question, the OP could add a ungodly
amount of swap and just listen the hard drive heads sing.

Dale

:-)  :-)

-- 
I am only responsible for what I said ... Not for what you understood or
how you interpreted my words!

Miss the compile output?  Hint:
EMERGE_DEFAULT_OPTS="--quiet-build=n"

Re: [gentoo-user] «-»: [gentoo-user] distcc - amd64 and x86

[Michael Orlitzky]

On 02/05/2012 01:46 PM, Dale wrote:
>
> It does the same for portages work space too.  I have portages work
> directory on tmpfs and I always have to mount with the size=12g option
> so that LOo will even start.
>
> Thing is, it rarely uses more than 4Gbs or so.  Is there a way to
> disable this mess?  I got the space for my compile.  I really don't need
> the checks.  Another reason for the question, the OP could add a ungodly
> amount of swap and just listen the hard drive heads sing.
>
> Dale
>
> :-)  :-)
>

I_KNOW_WHAT_I_AM_DOING="yes" will disable it. The prerequisite check is 
done in check-reqs.eclass, so setting it in make.conf should disable 
prereq checking for all packages if you're OK with the voided warranty.

[gentoo-user] distcc - amd64 and x86

[Dale]

Michael Orlitzky wrote:
> On 02/05/2012 01:46 PM, Dale wrote:
>>
>> It does the same for portages work space too.  I have portages work
>> directory on tmpfs and I always have to mount with the size=12g option
>> so that LOo will even start.
>>
>> Thing is, it rarely uses more than 4Gbs or so.  Is there a way to
>> disable this mess?  I got the space for my compile.  I really don't need
>> the checks.  Another reason for the question, the OP could add a ungodly
>> amount of swap and just listen the hard drive heads sing.
>>
>> Dale
>>
>> :-)  :-)
>>
> 
> I_KNOW_WHAT_I_AM_DOING="yes" will disable it. The prerequisite check is
> done in check-reqs.eclass, so setting it in make.conf should disable
> prereq checking for all packages if you're OK with the voided warranty.
> 
> 


I always thought that was people making a joke.  So, I put it in there.
 Sure enough, it's there.

root@fireball / # emerge --info | grep -i doing
I_KNOW_WHAT_I_AM_DOING="yes"
root@fireball / #

Kewl.  Now I need to add that to one of those other files just to apply
that to LOo.  I don't want to break EVERYTHING.  lol

Thanks MUCH!

Dale

:-)  :-)

-- 
I am only responsible for what I said ... Not for what you understood or
how you interpreted my words!

Miss the compile output?  Hint:
EMERGE_DEFAULT_OPTS="--quiet-build=n"

Re: [gentoo-user] distcc - amd64 and x86

[Canek Peláez Valdés]

On Sun, Feb 5, 2012 at 11:04 AM, Samuraiii <samuraiii@volny.cz> wrote:
> Hello,
>
> I have (right now) 3 computers runing Gentoo and as two of them are with
> only 1GB of ram - which for libreoffice compiling is not enough.

Mmmh. LibreOffice 3.4.x requires 1 GB of RAM, all other versions in
portage requires only 512 MB (including 3.5.x). Are you sure it is
your RAM the problem, or maybe you have /var/tmp (and therefore
/var/tmp/portage) in tmpfs? If that's the case, what I do is:

1. add the following line to /etc/portage/package.env:

app-office/libreoffice		notmpfs.conf

2. have the file /etc/portage/env/notmpfs.conf with the following:

# cat /etc/portage/env/notmpfs.conf
PORTAGE_TMPDIR="/var/notmpfs"

3. have the directory /var/notmpfs:

drwxrwxrwt  3 root root  4096 Feb  1 20:47 notmpfs

This way, all my packages get compiled in the tmpfs (and therefore
much faster, since it is memory), and libreoffice (which is to big to
fit in there) gets compiled on disk.

Hope it helps.

Regards.
-- 
Canek Peláez Valdés
Posgrado en Ciencia e Ingeniería de la Computación
Universidad Nacional Autónoma de México