[gentoo-user] Problems with MAKEOPTS -j

[gentoo-user] Problems with MAKEOPTS -j

[Pandu Poluan]

Have anyone experienced problems with MAKEOPTS?

I have a quad-core system, so I put in "-j5". But that resulted in
emerge fails for binutils and ppl. I have to change it to "-j3" before
ppl can be emerged, even "-j1" before binutils can be emerged.

Anyone experiencing the same?

Rgds,


-- 
--
Pandu E Poluan - IT Optimizer
My website: http://pandu.poluan.info/

Re: [gentoo-user] Problems with MAKEOPTS -j

[Joost Roeleveld]

On Thursday, August 25, 2011 02:37:21 PM Pandu Poluan wrote:
> Have anyone experienced problems with MAKEOPTS?
> 
> I have a quad-core system, so I put in "-j5". But that resulted in
> emerge fails for binutils and ppl. I have to change it to "-j3" before
> ppl can be emerged, even "-j1" before binutils can be emerged.
> 
> Anyone experiencing the same?

Yes, I used to have emerge -j and make -j set up limited to a load-level.

I've changed it to make -j5 myself with emerge without "-j"

It isn't just the ones you listed, I always have a lot of failed emerges that 
work ok when limiting the -j.

--
Joost

Re: [gentoo-user] Problems with MAKEOPTS -j

[Dale]

Joost Roeleveld wrote:
> On Thursday, August 25, 2011 02:37:21 PM Pandu Poluan wrote:
>    
>> Have anyone experienced problems with MAKEOPTS?
>>
>> I have a quad-core system, so I put in "-j5". But that resulted in
>> emerge fails for binutils and ppl. I have to change it to "-j3" before
>> ppl can be emerged, even "-j1" before binutils can be emerged.
>>
>> Anyone experiencing the same?
>>      
> Yes, I used to have emerge -j and make -j set up limited to a load-level.
>
> I've changed it to make -j5 myself with emerge without "-j"
>
> It isn't just the ones you listed, I always have a lot of failed emerges that
> work ok when limiting the -j.
>
> --
> Joost
>
>
>    

This is one of those times when things work for me.  I have mine set to 
-j5 for my quad core and it seems to work fine.  I have read where 
others have the problems to tho.  Is my luck changing perhaps?

Just wanted to chime in with one that works.

Dale

:-)  :-)

Re: [gentoo-user] Problems with MAKEOPTS -j

[Michael Mol]

On Thu, Aug 25, 2011 at 3:37 AM, Pandu Poluan <pandu@poluan.info> wrote:
> Have anyone experienced problems with MAKEOPTS?
>
> I have a quad-core system, so I put in "-j5". But that resulted in
> emerge fails for binutils and ppl. I have to change it to "-j3" before
> ppl can be emerged, even "-j1" before binutils can be emerged.
>
> Anyone experiencing the same?

Sometimes. My understanding is that it comes from packages which are
badly constructed, and can't reliably handle parallel builds. I'm told
that these cases are bugs and should be reported. Sometimes, if I
watch build output fly by, I'll even see something like

make -j10 -j1 (some target name)

where a maintainer decided to put an overriding -j1 after MAKEOPTS.


About two years ago, I found that, on my system (quad-core AMD Phenom
9650), -j8 resulted in the fastest build time, as measured by building
ffmpeg.[1] Currently, I'm running -j10, and that's because I've been
using distcc to pass a couple compile tasks off to other systems.
(Though with the box I was deferring to scrapped for parts, I'll drop
this down to -j8 again)

[1] Tested by building in tmpfs. You can find my data here:
http://multimedia.cx/eggs/ffmpeg-and-multiple-build-threads/#comment-150325

-- 
:wq

Re: [gentoo-user] Problems with MAKEOPTS -j

[Dale]

Michael Mol wrote:
> Sometimes. My understanding is that it comes from packages which are
> badly constructed, and can't reliably handle parallel builds. I'm told
> that these cases are bugs and should be reported. Sometimes, if I
> watch build output fly by, I'll even see something like
>
> make -j10 -j1 (some target name)
>
> where a maintainer decided to put an overriding -j1 after MAKEOPTS.
>
>
> About two years ago, I found that, on my system (quad-core AMD Phenom
> 9650), -j8 resulted in the fastest build time, as measured by building
> ffmpeg.[1] Currently, I'm running -j10, and that's because I've been
> using distcc to pass a couple compile tasks off to other systems.
> (Though with the box I was deferring to scrapped for parts, I'll drop
> this down to -j8 again)
>
> [1] Tested by building in tmpfs. You can find my data here:
> http://multimedia.cx/eggs/ffmpeg-and-multiple-build-threads/#comment-150325
>
>    

This is interesting.  I changed mine to j8 too.  See how this works.  
CPUs are so fast nowadays, they can do a lot pretty fast.  I'm trying to 
imagine 10 years from now.  O_O

Dale

:-)  :-)

Re: [gentoo-user] Problems with MAKEOPTS -j

[Michael Mol]

On Thu, Aug 25, 2011 at 9:16 AM, Dale <rdalek1967@gmail.com> wrote:
> Michael Mol wrote:
>>
>> Sometimes. My understanding is that it comes from packages which are
>> badly constructed, and can't reliably handle parallel builds. I'm told
>> that these cases are bugs and should be reported. Sometimes, if I
>> watch build output fly by, I'll even see something like
>>
>> make -j10 -j1 (some target name)
>>
>> where a maintainer decided to put an overriding -j1 after MAKEOPTS.
>>
>>
>> About two years ago, I found that, on my system (quad-core AMD Phenom
>> 9650), -j8 resulted in the fastest build time, as measured by building
>> ffmpeg.[1] Currently, I'm running -j10, and that's because I've been
>> using distcc to pass a couple compile tasks off to other systems.
>> (Though with the box I was deferring to scrapped for parts, I'll drop
>> this down to -j8 again)
>>
>> [1] Tested by building in tmpfs. You can find my data here:
>>
>> http://multimedia.cx/eggs/ffmpeg-and-multiple-build-threads/#comment-150325
>>
>>
>
> This is interesting.  I changed mine to j8 too.  See how this works.  CPUs
> are so fast nowadays, they can do a lot pretty fast.  I'm trying to imagine
> 10 years from now.  O_O

I imagine there's a *lot* of money to be made researching AST
optimization and simplification transform searches as delegated to
GPUs. I don't imagine it'll speed up compiling much, but I do imagine
the resulting optimized programs will run far faster.  Especially if
there's a trend toward declarative programming languages. (I wonder if
there's a Prolog implementation that dispatches work to OpenCL. I'm
surprised the Mozart/Oz folks haven't dug in that direction yet.)

-- 
:wq

Re: [gentoo-user] Problems with MAKEOPTS -j

[Peter Humphrey]

On Thursday 25 August 2011 14:16:45 Dale wrote:

> CPUs are so fast nowadays, they can do a lot pretty fast.  I'm trying to
> imagine 10 years from now.  O_O

You'll be back to HAL by then - the one you speak to, that is.

-- 
Rgds
Peter		Linux Counter 5290, 1994-04-23

Re: [gentoo-user] Problems with MAKEOPTS -j

[Paul Hartman]

On Thu, Aug 25, 2011 at 2:37 AM, Pandu Poluan <pandu@poluan.info> wrote:
> Have anyone experienced problems with MAKEOPTS?
>
> I have a quad-core system, so I put in "-j5". But that resulted in
> emerge fails for binutils and ppl. I have to change it to "-j3" before
> ppl can be emerged, even "-j1" before binutils can be emerged.
>
> Anyone experiencing the same?

I use -j12 and don't have any issues. I also use "emerge -j" along
with it which sometimes results in hundreds of gcc running at once
(emerging all of kde simultaneously), and it survives.

Certain rare packages don't support parallel make, but typically the
ebuild will override your -j setting with -j1 instead. binutils for
certain works on my system with -j12.

Re: [gentoo-user] Problems with MAKEOPTS -j

[Dale]

Peter Humphrey wrote:
> On Thursday 25 August 2011 14:16:45 Dale wrote:
>
>    
>> CPUs are so fast nowadays, they can do a lot pretty fast.  I'm trying to
>> imagine 10 years from now.  O_O
>>      
> You'll be back to HAL by then - the one you speak to, that is.
>
>    

<  Dale orders more silver bullets. >

They do say history repeats itself.  I hope they don't make as big a 
mess next time.  lol

Dale

:-)  :-)