[gentoo-dev] RFC: KDE 'child' packages (separate ebuilds for konqueror kmail etc.)

[gentoo-dev] RFC: KDE 'child' packages (separate ebuilds for konqueror kmail etc.)

[Dan Armak]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Hello everyone,

During the last week I've been working on separating the kde-base/* packages 
into 'child' packages, a feature requested by some people.  For example, 
instead of kdenetwork we'd have separate kmail, knode, kppp etc. ebuilds. The 
actual eclass framework to do this is ready.

However, there is a problem. A 'parent' package (e.g. kde-base/kdenetwork) 
will still exist since most people will want to merge all of it. There are 
two ways to go:

1. The parent and child packages will not be related. They will overwrite each 
other's files. This is unacceptable.

2. The parent package will depend on all of its children and install nothing 
itself. Like kde-base/kde depends on kdebase kdenetwork etc., so will 
kde-base/kdenetwork depend on net-www/konqueror, net-mail/kmail etc. This is 
the approach I've taken so far.

The problem is that while every 'child' package only compiles and installs the 
relevant files, it still needs to run the complete configure script of its 
parent package. In the admittedly worst-case example of kdebase, the script 
takes ~1.5 minutes to execute on my P3-900. kdebase has 39 child packages, so 
emerging kdebase with the new setup (i.e. emerging all child packages) will 
take about an hour longer than emerging a monolithic kdebase takes now - 
worse for slower computers.

Most users won't accept this tradeoff of compile time for flexibility. So if 
you have an alternative solution, please suggest it. Otherwise there won't be 
any kde child packages. Speak now or remain forever silent :-)

- -- 
Dan Armak
Gentoo Linux developer (KDE)
Matan, Israel
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.0.7 (GNU/Linux)

iD8DBQE8348IUI2RQ41fiVERAqSWAJ92lM8cVCcH0bhefNNHbO+ww3VuHQCfVApF
gMSpHtMGOR7pgMKhdm8nJ/c=
=R1ij
-----END PGP SIGNATURE-----

Re: [gentoo-dev] RFC: KDE 'child' packages (separate ebuilds for konqueror kmail etc.)

[Mikko Moilanen]

On Monday 13 May 2002 10:01, Dan Armak wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> The problem is that while every 'child' package only compiles and installs
> the relevant files, it still needs to run the complete configure script of
> its parent package. In the admittedly worst-case example of kdebase, the
> script takes ~1.5 minutes to execute on my P3-900. kdebase has 39 child
> packages, so emerging kdebase with the new setup (i.e. emerging all child
> packages) will take about an hour longer than emerging a monolithic kdebase
> takes now - worse for slower computers.
>
> Most users won't accept this tradeoff of compile time for flexibility. So
> if you have an alternative solution, please suggest it. Otherwise there
> won't be any kde child packages. Speak now or remain forever silent :-)

I think that flexibility is more important than short compile time. For me it 
took more than 12 hours to compile KDE and X (PII@400, 128Mt, 1,7Gt). So if 
it takes an hour more, I dont care. Also I think that it would be nice if 
people can choose to take only something from mighty KDE project, not all.

Oh, and thankyou for so very much of doing extreamly great job! :D
-- 


Mikko

Re: [gentoo-dev] RFC: KDE 'child' packages (separate ebuilds for konqueror kmail etc.)

[Eric Noack]

> Hello everyone,
>
> During the last week I've been working on separating the > kde-base/* packages into 'child' packages, a feature requested by some > people.  For example, instead of kdenetwork we'd have separate kmail, > knode, kppp etc. ebuilds. The actual eclass framework to do this is > ready.
> ...

As a Gentoo user my humble opinion is that being able to install single parts of things like KDE is much more important than compile-time.
Specially huge packages like KDE which takes real long time to compile anyway are things one would probably compile "over-night" if one wants the whole package, so additional 1 or 2 hours wont count. And if one doesnt want to install the whole thing, one would be glad to be able
to choose what to install and what not.

Make it so, and keep up the good work. :-)

Gentoo Rules


Eric

Re: [gentoo-dev] RFC: KDE 'child' packages (separate ebuilds for konqueror kmail etc.)

[Alexander Gretencord]

On Monday 13 May 2002 12:01, Dan Armak wrote:
> Hello everyone,
>
> During the last week I've been working on separating the kde-base/*
> packages into 'child' packages, a feature requested by some people.  For
> example, instead of kdenetwork we'd have separate kmail, knode, kppp etc.
> ebuilds. The actual eclass framework to do this is ready.
>
> However, there is a problem. A 'parent' package (e.g. kde-base/kdenetwork)
> will still exist since most people will want to merge all of it. There are
> two ways to go:
>
> 1. The parent and child packages will not be related. They will overwrite
> each other's files. This is unacceptable.
>
> 2. The parent package will depend on all of its children and install
> nothing itself. Like kde-base/kde depends on kdebase kdenetwork etc., so
> will kde-base/kdenetwork depend on net-www/konqueror, net-mail/kmail etc.
> This is the approach I've taken so far.
>
> The problem is that while every 'child' package only compiles and installs
> the relevant files, it still needs to run the complete configure script of
> its parent package. In the admittedly worst-case example of kdebase, the
> script takes ~1.5 minutes to execute on my P3-900. kdebase has 39 child
> packages, so emerging kdebase with the new setup (i.e. emerging all child
> packages) will take about an hour longer than emerging a monolithic kdebase
> takes now - worse for slower computers.
>
> Most users won't accept this tradeoff of compile time for flexibility. So
> if you have an alternative solution, please suggest it. Otherwise there
> won't be any kde child packages. Speak now or remain forever silent :-)

Contrary to the other two mails, I'd not like this tradeoff.

There is a third option: If someone wants say kmail and knode (which is what 
I'd like from kdenetwork myself) then run the configure script for kdenetwork 
once and then compile kmail and knode (and its dependencies like 
libkdenetwork) and install them. Of course you couldn't just have child 
packages in that case and have the kdenetwork script run the children one 
after another.

Alex

-- 
"They that can give up essential liberty to obtain a little temporary safety 
deserve neither liberty nor safety."
Benjamin Franklin

Re: [gentoo-dev] Re: [gentoo-user] RFC: KDE 'child' packages - new direction

[Bob Phan]

On Mon, 13 May 2002, Dan Armak wrote:

<snip>
> Description:
> Portage-related side:
> 1a. An ebuild sets $SUBPACKAGELIST to a list of the subpackages it has. (Not 
> to confuse with $SUBPACKAGES below. Someone suggest a better pair of names 
> please!)

$SUBPACKAGE_AVAIL
$SUBPACKAGE_REQ

The underscore draws the eye to the part that is different.  Just my
2 cents.

-- 
/*
 * Bob Phan <bob@endlessrecursion.net,rphan@nrgn.com>
 * Computational Chemistry Informatics
 * Neurogen Corporation
 * (203)488-8201 x4645
 *
 * To understand recursion, you must first understand recursion.
 */

Re: [gentoo-dev] RFC: KDE 'child' packages (separate ebuilds for konqueror kmail etc.)

[Dan Armak]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Monday 13 May 2002 15:46, Alexander Gretencord wrote:
> Contrary to the other two mails, I'd not like this tradeoff.
>
> There is a third option: If someone wants say kmail and knode (which is
> what I'd like from kdenetwork myself) then run the configure script for
> kdenetwork once and then compile kmail and knode (and its dependencies like
> libkdenetwork) and install them. Of course you couldn't just have child
> packages in that case and have the kdenetwork script run the children one
> after another.

The problem is, there's no clean way to imlpement this with ebuilds (and even 
eclasses). How can I make the kdenetwork ebuild compile a selective list of 
'children' while only running configure once? Granted, there's emerge 
- --inject now, but an ebuild is _not_ supposed to be using that. Do you have 
some specific idea for an implementation in mind?

- -- 
Dan Armak
Gentoo Linux developer (KDE)
Matan, Israel
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.0.7 (GNU/Linux)

iD8DBQE83/k3UI2RQ41fiVERAljvAJsGR2xCrzQhpEtDrNVnbOWAPBYQ7gCfdqxA
KjfcXtv/RfuLgmpr/T6FWkc=
=uYnd
-----END PGP SIGNATURE-----

Re: [gentoo-dev] RFC: KDE 'child' packages (separate ebuilds for konqueror kmail etc.)

[Dan Armak]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Monday 13 May 2002 14:18, Eric Noack wrote:
> > Hello everyone,
> >
> > During the last week I've been working on separating the > kde-base/*
> > packages into 'child' packages, a feature requested by some > people. 
> > For example, instead of kdenetwork we'd have separate kmail, > knode,
> > kppp etc. ebuilds. The actual eclass framework to do this is > ready. ...
>
> As a Gentoo user my humble opinion is that being able to install single
> parts of things like KDE is much more important than compile-time.
> Specially huge packages like KDE which takes real long time to compile
> anyway are things one would probably compile "over-night" if one wants the
> whole package, so additional 1 or 2 hours wont count. And if one doesnt
> want to install the whole thing, one would be glad to be able to choose
> what to install and what not.
>
> Make it so, and keep up the good work. :-)

It wouldn't be another hour or two. More like another four. Which is a lot, 
and not everyone compiles kde by night.

This seems like a case where the benefits and problems of a new feature are 
more or less balanced. Hmmm.....

- -- 
Dan Armak
Gentoo Linux developer (KDE)
Matan, Israel
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.0.7 (GNU/Linux)

iD8DBQE83/mwUI2RQ41fiVERAsS+AJsErRZfal3gMFg37/LtuhFKpkFKrACfZvMo
U+Ar3OZLGdAWzDbx32CZ8RY=
=CZfB
-----END PGP SIGNATURE-----

[gentoo-dev] Re: [gentoo-user] RFC: KDE 'child' packages - new direction

[Dan Armak]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Hello everyone,

My RFC turned up many proponents as wellas opponents of the change. After an 
irc discussion with the other gentoo developers, we've decided to introduce 
generic "subpackage" support into portage. (Or try).

It will take some time, and so won't be available with kde-3.0.1 (which should 
be released Wednesday or Thursday, according to the original plans). But, it 
will solve everything in the best way possible, enabling flexible subpackages 
(in an even better way than my original idea) while eliminating the multiple 
configure runs problem.

Here's the porposal I'm posting on gentoo-core (a private developer list):

To bring everyone up to speed:
KDE child packages as implemented so far have one big disadvantage for people 
who want to emerge all of kde: the configure script has to be run for every 
child packages, resulting in a big overhead (1-2 minutes to run the script, 
20-40 child packages per parent --> extra hours of compilnig time).

I asked on -dev and -user how good/bad such a tradeoff would be, and responses 
have been quite mixed so far. Not quite balanced but nearly so (more people 
don't care about compile time and/or want modular packages).

New proposal: "subpackages", much as described in the portage.py header 
comments. Note that I'm not talking about "dev,run,..." subpackages mentioned 
there, but about separating e.g. kdenetwork into kmail, kppp, knode... But of 
course any ebuild could create any subpackages it wanted.

Description:
Portage-related side:
1a. An ebuild sets $SUBPACKAGELIST to a list of the subpackages it has. (Not 
to confuse with $SUBPACKAGES below. Someone suggest a better pair of names 
please!)

1b. We'll probably want subpackages to have revisions independant of their 
master ebuild, so that they can be updated separately. This should be 
specified here as well. Like this:
SUBPACKAGELIST="foo1 foo2-r1 foo3-r4" (foo1 is at r0)
Or any other way that'll be more comfortable for the python side to process. 
Maybe with a char that can't be a part of an ebuild name, like so:
SUBPACKAGELIST="foo1:0 foo2:1 foo3:4"
Or similar.

2. A user can call "emerge kdenetwork:kmail,knode..." (maybe even "ebuild 
kdenetwork:this,that,... action"?). A dep can also contain such a string.

3. The ebuild is processed in the standard fashion; a variable $SUBPACKAGES is 
made available to it to indicate the subpackages requested. The ebuild is 
then responsible for building and installing the packages indicated. (My 
kde-child -based ebuilds can do it already.)

4a. In src_install, each subpackage 'foo''s files go into ${D}/foo/.
Or:
4b. As proposed in the portage.py comments, there's an src_subpkg() function 
that does:
subpkg foo # set grab's subpackage to foo
grap files/*.so # register these files as belonging to subpackage foo
grap foodir # grab a directory recursively

5. In /var/db/pkg/category, a record "package:subpackage" is made for every 
subpackage appropriately. If I "emerge kdenetwork", records for all 
subpackages will be created.

Summary: this is excellent AFAICS. It will be easier to implement ebuild-wise 
than child ebuilds (since no new separate ebuilds need be created), it won't 
force people to run configure more than once when not necessary, etc.

Summary of portage-side changes required:
- - master:foo1,foo2... support in emerge command lines and dependency strings, 
with $SUBPACKAGES provided to the ebuild.
- - master:foo subpackage registration in /var/db/pkg with $SUBPACKAGELIST from 
the ebuild taken into account (e.g. if all subpackages have been merged, a 
dependency on the master ebuild is satisifed; if only some have been merged, 
the master ebuild is called with the complementing $SUBPACKAGES).
- - support of src_subpkg() and subpkg() and grab() functions.

Ebuild-wise, I'll be ready a day or two after the release of such a portage. 
Most of the work has already been done.

Comments welcome!

- -- 
Dan Armak
Gentoo Linux developer (KDE)
Matan, Israel
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.0.7 (GNU/Linux)

iD8DBQE84AaFUI2RQ41fiVERAkNFAJ9AZeNpdlq0B2Tii768ak/tqk4xJQCfdzeY
+6WAFDe8UD9Xh2ouIuCDKvs=
=iZXH
-----END PGP SIGNATURE-----

Re: [gentoo-dev] RFC: KDE 'child' packages (separate ebuilds for konqueror kmail etc.)

[Paul de Vrieze]

On Monday 13 May 2002 19:34, Dan Armak wrote:
>
> The problem is, there's no clean way to imlpement this with ebuilds (and
> even eclasses). How can I make the kdenetwork ebuild compile a selective
> list of 'children' while only running configure once? Granted, there's
> emerge --inject now, but an ebuild is _not_ supposed to be using that. Do
> you have some specific idea for an implementation in mind?

The easiest way that I see is package specific use flags. This is where an 
optional use flag system can come into the view

Paul

-- 
Paul de Vrieze
Junior Researcher
Mail: pauldv@cs.kun.nl
Homepage: http://www.devrieze.net

Re: [gentoo-dev] RFC: KDE 'child' packages (separate ebuilds for konqueror kmail etc.)

[Miguel S. Filipe]

Dan Armak wrote:

>-----BEGIN PGP SIGNED MESSAGE-----
>Hash: SHA1
>
>Hello everyone,
>
>During the last week I've been working on separating the kde-base/* packages 
>into 'child' packages, a feature requested by some people.  For example, 
>instead of kdenetwork we'd have separate kmail, knode, kppp etc. ebuilds. The 
>actual eclass framework to do this is ready.
>
>However, there is a problem. A 'parent' package (e.g. kde-base/kdenetwork) 
>will still exist since most people will want to merge all of it. There are 
>two ways to go:
>
>1. The parent and child packages will not be related. They will overwrite each 
>other's files. This is unacceptable.
>
>2. The parent package will depend on all of its children and install nothing 
>itself. Like kde-base/kde depends on kdebase kdenetwork etc., so will 
>kde-base/kdenetwork depend on net-www/konqueror, net-mail/kmail etc. This is 
>the approach I've taken so far.
>
>The problem is that while every 'child' package only compiles and installs the 
>relevant files, it still needs to run the complete configure script of its 
>parent package. In the admittedly worst-case example of kdebase, the script 
>takes ~1.5 minutes to execute on my P3-900. kdebase has 39 child packages, so 
>emerging kdebase with the new setup (i.e. emerging all child packages) will 
>take about an hour longer than emerging a monolithic kdebase takes now - 
>worse for slower computers.
>
>Most users won't accept this tradeoff of compile time for flexibility. So if 
>you have an alternative solution, please suggest it. Otherwise there won't be 
>any kde child packages. Speak now or remain forever silent :-)
>
>- -- 
>Dan Armak
>Gentoo Linux developer (KDE)
>Matan, Israel
>-----BEGIN PGP SIGNATURE-----
>Version: GnuPG v1.0.7 (GNU/Linux)
>
>iD8DBQE8348IUI2RQ41fiVERAqSWAJ92lM8cVCcH0bhefNNHbO+ww3VuHQCfVApF
>gMSpHtMGOR7pgMKhdm8nJ/c=
>=R1ij
>-----END PGP SIGNATURE-----
>
>_______________________________________________
>gentoo-dev mailing list
>gentoo-dev@gentoo.org
>http://lists.gentoo.org/mailman/listinfo/gentoo-dev
>
>  
>
And couldn't those small intependent ebuilds being run in parallel? 
Would that shorten the compile-time?

But has it's being said, it's ok to take longer compile times for more 
flexibility, after all if we're woried about the time it takes, we 
wouldn't be compiling things like kde, X and other BIG packages at all.

Re: [gentoo-dev] RFC: KDE 'child' packages (separate ebuilds for konqueror kmail etc.)

[Justin Lambert]

I would love to have seperate ebuilds for the parts of KDE, many of the 
packages I compile are for only one program.

On Monday 13 May 2002 12:36 pm, Dan Armak wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> On Monday 13 May 2002 14:18, Eric Noack wrote:
> > > Hello everyone,
> > >
> > > During the last week I've been working on separating the > kde-base/*
> > > packages into 'child' packages, a feature requested by some > people.
> > > For example, instead of kdenetwork we'd have separate kmail, > knode,
> > > kppp etc. ebuilds. The actual eclass framework to do this is > ready.
> > > ...
> >
> > As a Gentoo user my humble opinion is that being able to install single
> > parts of things like KDE is much more important than compile-time.
> > Specially huge packages like KDE which takes real long time to compile
> > anyway are things one would probably compile "over-night" if one wants
> > the whole package, so additional 1 or 2 hours wont count. And if one
> > doesnt want to install the whole thing, one would be glad to be able to
> > choose what to install and what not.
> >
> > Make it so, and keep up the good work. :-)
>
> It wouldn't be another hour or two. More like another four. Which is a lot,
> and not everyone compiles kde by night.
>
> This seems like a case where the benefits and problems of a new feature are
> more or less balanced. Hmmm.....
>
> - --
> Dan Armak
> Gentoo Linux developer (KDE)
> Matan, Israel
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v1.0.7 (GNU/Linux)
>
> iD8DBQE83/mwUI2RQ41fiVERAsS+AJsErRZfal3gMFg37/LtuhFKpkFKrACfZvMo
> U+Ar3OZLGdAWzDbx32CZ8RY=
> =CZfB
> -----END PGP SIGNATURE-----
>
> _______________________________________________
> gentoo-dev mailing list
> gentoo-dev@gentoo.org
> http://lists.gentoo.org/mailman/listinfo/gentoo-dev