[gentoo-dev] parallel fun in src_install - going beyond the serial monotony of 'make install'

[gentoo-dev] parallel fun in src_install - going beyond the serial monotony of 'make install'

[Robin H. Johnson]

[-- Attachment #1: Type: text/plain, Size: 1781 bytes --]

In the present devmanual, for src_install, it notes that 
	make install DESTDIR="${D}"
is the preferred way to fire off the install, and to not use emake, for
fear of parallel issues.

This has four nasty side effects:
- Global assumption that make is GNU Make (Hi flameeyes).
- Doesn't pass MAKEOPTS for options other than -jX.
- Removes ability to use EXTRA_EMAKE.
- Large package installs are make very slow.

I did a bit of checking of a random selection of packages (looking at
the Makefile and internal dependancy trees, as well as a test with -j7
on my 4-way box), and found very few problems with the concept of
parallel installs.

For a start, practically all Makefiles generated by a recent version
(newer than 2001 for definite) are parallel safe. There are a few minor
cases I saw where this wasn't true, but those packages also had other
parallel build problems.

For packages that are definitely aren't parallel-install safe, we should
use 'emake -j1', just like the src_compile phase.

I'm not saying we should change the entire existing tree, but for new
packages, please consider emake for your src_install phase.

A nice use for EXTRA_EMAKE while debugging is being able to pass in
--debug=basic to make, and have it tell you why it rebuilt a target in
src_install, since src_install should not be compiling things ;-).

To forestall one question I expect, I'd like to point a basic fact about
how Makefiles work:
Multiple commands in a single target will always be executed in series,
and not parallel - so this target will work fine:
install:
   mkdir -p $(DESTDIR)/bin
   cp -f bar $(DESTDIR)/bin

-- 
Robin Hugh Johnson
E-Mail     : robbat2@gentoo.org
GnuPG FP   : 11AC BA4F 4778 E3F6 E4ED  F38E B27B 944E 3488 4E85

[-- Attachment #2: Type: application/pgp-signature, Size: 241 bytes --]

Re: [gentoo-dev] parallel fun in src_install - going beyond the serial monotony of 'make install'

[Robin H. Johnson]

[-- Attachment #1: Type: text/plain, Size: 760 bytes --]

On Thu, Jun 08, 2006 at 01:58:07AM -0700, Robin H. Johnson wrote:
> For a start, practically all Makefiles generated by a recent version
> (newer than 2001 for definite) are parallel safe. There are a few minor
> cases I saw where this wasn't true, but those packages also had other
> parallel build problems.
Typo here, I left out the word 'autotools'.

Corrected version:
For a start, practically all Makefiles generated by a recent version of
autotools (newer than 2001 for definite) are parallel safe. There are a
few minor cases I saw where this wasn't true, but those packages also
had other parallel build problems.

-- 
Robin Hugh Johnson
E-Mail     : robbat2@gentoo.org
GnuPG FP   : 11AC BA4F 4778 E3F6 E4ED  F38E B27B 944E 3488 4E85

[-- Attachment #2: Type: application/pgp-signature, Size: 241 bytes --]

Re: [gentoo-dev] parallel fun in src_install - going beyond the serial monotony of 'make install'

[Harald van Dijk]

On Thu, Jun 08, 2006 at 01:58:07AM -0700, Robin H. Johnson wrote:
> In the present devmanual, for src_install, it notes that 
> 	make install DESTDIR="${D}"
> is the preferred way to fire off the install, and to not use emake, for
> fear of parallel issues.

Actually, it uses `make DESTDIR="${D}" install`. Is there a reason you
changed it around, or does it simply not matter at all?

I completely agree with the rest of what you wrote.
-- 
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] parallel fun in src_install - going beyond the serial monotony of 'make install'

[Mike Frysinger]

[-- Attachment #1: Type: text/plain, Size: 617 bytes --]

On Thursday 08 June 2006 06:08, Harald van Dijk wrote:
> On Thu, Jun 08, 2006 at 01:58:07AM -0700, Robin H. Johnson wrote:
> > In the present devmanual, for src_install, it notes that
> > 	make install DESTDIR="${D}"
> > is the preferred way to fire off the install, and to not use emake, for
> > fear of parallel issues.
>
> Actually, it uses `make DESTDIR="${D}" install`. Is there a reason you
> changed it around, or does it simply not matter at all?

it doesnt matter

> I completely agree with the rest of what you wrote.

some people have started using `emake -j1 DESTDIR="${D}" install`
-mike

[-- Attachment #2: Type: application/pgp-signature, Size: 827 bytes --]