[gentoo-dev] GLEP ??: Metapackages

[gentoo-dev] GLEP ??: Metapackages

[Stephen Bennett]

As those who hang around in the mysterious realms of Portage development
may know, there's some feeling around that the current system of virtual
packages has some serious limitations. The currently-proposed
alternative (as discussed previously, notably in
http://thread.gmane.org/gmane.linux.gentoo.devel/18922), involves a
system of metapackages. These would essentially consist of a 
non-installable ebuild that consists entirely of a set of dependency
information. Once the dependencies for the metapackage are satisfied,
it's considered to be installed, and packages depending on it can go
ahead and be built.

This approach brings several advantages over the current system,
particularly: 
- Allowing one version of a package to provide a different version of a
virtual, where these are necessary.
- Fixing the screwup with .51's virtual handling whereby gcc-2.95.x has
PROVIDE="sys-apps/texinfo", a package depends on >=texinfo-4.6, so
portage tries to install >=gcc-4.6.
- Provides, in one easily accessible place, a list of package that could
be used to satisfy the dependency. This has advantages for speed (no
searching the tree for PROVIDEs) and for user-friendliness.

Anyway, with portage development as it is now, this got brought up
again, and the current state of the GLEP can be found at
http://dev.gentoo.org/~spb/metapkg-glep.txt. Comments/suggestions/flames
welcome.

--
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] GLEP ??: Metapackages

[Paul de Vrieze]

[-- Attachment #1: Type: text/plain, Size: 1033 bytes --]

On Tuesday 01 March 2005 23:15, Stephen Bennett wrote:
> As those who hang around in the mysterious realms of Portage
> development may know, there's some feeling around that the current
> system of virtual packages has some serious limitations. The
> currently-proposed
> alternative (as discussed previously, notably in
> http://thread.gmane.org/gmane.linux.gentoo.devel/18922), involves a
> system of metapackages. These would essentially consist of a
> non-installable ebuild that consists entirely of a set of dependency
> information. Once the dependencies for the metapackage are satisfied,
> it's considered to be installed, and packages depending on it can go
> ahead and be built.

I see one feature that is really made more necessary for this. You want to 
be able to select ranges of package versions that can not directly be 
gotten by using wildcards. That way you can do real version mapping.

Paul

-- 
Paul de Vrieze
Gentoo Developer
Mail: pauldv@gentoo.org
Homepage: http://www.devrieze.net

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-dev] GLEP ??: Metapackages

[Alec Warner]

Stephen Bennett wrote:

>As those who hang around in the mysterious realms of Portage development
>may know, there's some feeling around that the current system of virtual
>packages has some serious limitations. The currently-proposed
>alternative (as discussed previously, notably in
>http://thread.gmane.org/gmane.linux.gentoo.devel/18922), involves a
>system of metapackages. These would essentially consist of a 
>non-installable ebuild that consists entirely of a set of dependency
>information. Once the dependencies for the metapackage are satisfied,
>it's considered to be installed, and packages depending on it can go
>ahead and be built.
>
>This approach brings several advantages over the current system,
>particularly: 
>- Allowing one version of a package to provide a different version of a
>virtual, where these are necessary.
>- Fixing the screwup with .51's virtual handling whereby gcc-2.95.x has
>PROVIDE="sys-apps/texinfo", a package depends on >=texinfo-4.6, so
>portage tries to install >=gcc-4.6.
>- Provides, in one easily accessible place, a list of package that could
>be used to satisfy the dependency. This has advantages for speed (no
>searching the tree for PROVIDEs) and for user-friendliness.
>
>Anyway, with portage development as it is now, this got brought up
>again, and the current state of the GLEP can be found at
>http://dev.gentoo.org/~spb/metapkg-glep.txt. Comments/suggestions/flames
>welcome.
>
>  
>
As long as you keep the PROVIDES line in the ebuild.  Assuming repoman 
does full tree check  when commiting it can make sure that all packages 
that PROVIDE virtual also exist in the metapackage.  That would be my 
only concern, people forgetting to add lines to the correct files :)

>--
>gentoo-dev@gentoo.org mailing list
>  
>
--
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] GLEP ??: Metapackages

[James Northrup]

[-- Attachment #1: Type: text/plain, Size: 606 bytes --]

Alec Warner wrote:

> Stephen Bennett wrote:
>
>> Anyway, with portage development as it is now, this got brought up
>> again, and the current state of the GLEP can be found at
>> http://dev.gentoo.org/~spb/metapkg-glep.txt. Comments/suggestions/flames
>> welcome.
>
> As long as you keep the PROVIDES line in the ebuild.  Assuming repoman 
> does full tree check  when commiting it can make sure that all 
> packages that PROVIDE virtual also exist in the metapackage.  That 
> would be my only concern, people forgetting to add lines to the 
> correct files :)
>
Metapackages: good. 
120 meg rsync: bad


[-- Attachment #2: glamdring-inc.vcf --]
[-- Type: text/x-vcard, Size: 297 bytes --]

begin:vcard
fn:james northrup
n:northrup;james
org:Glamdring Incorporated Enterprises
adr:;;1406B Peralta Ave;Berkeley;CA;94702;US
email;internet:james_northrup@comcast.net
title:CTO
tel;fax:(510) 725-4587
tel;home:(510) 725-4587
tel;cell:(408) 896-6239
x-mozilla-html:TRUE
version:2.1
end:vcard

Re: [gentoo-dev] GLEP ??: Metapackages

[Dan Armak]

[-- Attachment #1: Type: text/plain, Size: 2332 bytes --]

On Wednesday 02 March 2005 00:15, Stephen Bennett wrote:
> As those who hang around in the mysterious realms of Portage development
> may know, there's some feeling around that the current system of virtual
> packages has some serious limitations. The currently-proposed
> alternative (as discussed previously, notably in
> http://thread.gmane.org/gmane.linux.gentoo.devel/18922), involves a
> system of metapackages. These would essentially consist of a
> non-installable ebuild that consists entirely of a set of dependency
> information. Once the dependencies for the metapackage are satisfied,
> it's considered to be installed, and packages depending on it can go
> ahead and be built.
It's a cool idea. What's missing in the proto-GLEP is an explanation of why 
you can't do this with a normal ebuild (that doesn't install any files), and 
need the new concept of metapackages.

The only reason I've found (IIRC it's described in the linked thread too) is 
this scenario:
1. virtual/foo has DEPEND="|| ( pkgA pkgB )".
2. User has pkgB-1.5 emerged.
3. User emerges package depending on >=virtual/foo-2.
4. pkgA-2.0 is pulled in, instead of upgrading pkgB to 2.0.

There's also the question of portage not checking RDEPENDs when unmerging, so 
you can unmerge a dep (and things will break) but you can't unmerge a package 
providing a virtual (portage will catch that). But the correct solution here, 
if we're going to modify portage, is to add generic RDEPEND checking support 
to emerge unmerge...

Am I missing another reason? Regardless, the reason for this should be in the 
GLEP.

Also, the GLEP says: "On a side note, this system of metapackages would 
provide an implementation of 'package sets' as proposed in GLEP 21 [2]_."

I don't see how that would happen. A package set exists to install all of a 
list of packages, while a virtual/metapackage exists to install one of a list 
of (often mutually exclusive) packages. These are very different goals. How 
would metapackages help with sets any more than ordinary ebuilds already do?

Please enlighten me if I've totally missed your point.

-- 
Dan Armak
Gentoo Linux developer (KDE)
Public GPG key: http://dev.gentoo.org/~danarmak/danarmak-gpg-public.key
Fingerprint: DD70 DBF9 E3D4 6CB9 2FDD  0069 508D 9143 8D5F 8951

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-dev] GLEP ??: Metapackages

[Stephen Bennett]

On Sun, 2005-03-06 at 19:40 +0200, Dan Armak wrote:
> It's a cool idea. What's missing in the proto-GLEP is an explanation of why 
> you can't do this with a normal ebuild (that doesn't install any files), and 
> need the new concept of metapackages.

The idea is that a metapackage, unlike a normal ebuild, doesn't exist in
the installed package db, and its deps are always inspected when it
turns up in the depgraph. That means that you avoid the situation where
you merge package A, which depends on virtual/x11, and then pulls in
xorg-x11. Then, for whatever reason, you unmerge xorg, and virtual/x11
is still in the vdb, so the next app you merge that deps on it will
break. That was explained in the -dev thread I linked; I probably should
add an explanation into the GLEP.

> There's also the question of portage not checking RDEPENDs when unmerging, so 
> you can unmerge a dep (and things will break) but you can't unmerge a package 
> providing a virtual (portage will catch that). But the correct solution here, 
> if we're going to modify portage, is to add generic RDEPEND checking support 
> to emerge unmerge...

Allegedly the new dep resolver that's being worked on should handle
that... not sure if it'll get into the next portage release though. But
agreed, it is needed, and from what I've been given to understand should
handle this situation properly without too much extra effort.

> Also, the GLEP says: "On a side note, this system of metapackages would 
> provide an implementation of 'package sets' as proposed in GLEP 21 [2]_."
> 
> I don't see how that would happen. A package set exists to install all of a 
> list of packages, while a virtual/metapackage exists to install one of a list 
> of (often mutually exclusive) packages. These are very different goals. How 
> would metapackages help with sets any more than ordinary ebuilds already do?

Since the metapackage has some arbitrary DEPEND string that has to be
met, there's no reason why this couldn't require all packages reckoned
to be part of a set, rather than one of the packages reckoned to provide
a virtual. 

--
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] GLEP ??: Metapackages

[Dan Armak]

[-- Attachment #1: Type: text/plain, Size: 2240 bytes --]

On Sunday 06 March 2005 20:12, Stephen Bennett wrote:
> On Sun, 2005-03-06 at 19:40 +0200, Dan Armak wrote:
> > It's a cool idea. What's missing in the proto-GLEP is an explanation of
> > why you can't do this with a normal ebuild (that doesn't install any
> > files), and need the new concept of metapackages.
>
> The idea is that a metapackage, unlike a normal ebuild, doesn't exist in
> the installed package db, and its deps are always inspected when it
> turns up in the depgraph. That means that you avoid the situation where
> you merge package A, which depends on virtual/x11, and then pulls in
> xorg-x11. Then, for whatever reason, you unmerge xorg, and virtual/x11
> is still in the vdb, so the next app you merge that deps on it will
> break. That was explained in the -dev thread I linked; I probably should
> add an explanation into the GLEP.
OK. That would be fixed (for all ebuilds) by RDEPEND support on unmerge.

> > Also, the GLEP says: "On a side note, this system of metapackages would
> > provide an implementation of 'package sets' as proposed in GLEP 21 [2]_."
> >
> > I don't see how that would happen. A package set exists to install all of
> > a list of packages, while a virtual/metapackage exists to install one of
> > a list of (often mutually exclusive) packages. These are very different
> > goals. How would metapackages help with sets any more than ordinary
> > ebuilds already do?
>
> Since the metapackage has some arbitrary DEPEND string that has to be
> met, there's no reason why this couldn't require all packages reckoned
> to be part of a set, rather than one of the packages reckoned to provide
> a virtual.
Yes, but you can already do this with an ebuild. The only difference is again 
the RDEPEND issues...

Well, this should be mentioned in the GLEP IMO.

BTW, the new functionality I think we really need for sets (which isn't 
specified in GLEP 21) is for the unmerge command to act on an entire set. 
E.g. to unmerge all of the (slotted) KDE 3.4 after installing 3.5.

-- 
Dan Armak
Gentoo Linux developer (KDE)
Public GPG key: http://dev.gentoo.org/~danarmak/danarmak-gpg-public.key
Fingerprint: DD70 DBF9 E3D4 6CB9 2FDD  0069 508D 9143 8D5F 8951

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-dev] GLEP ??: Metapackages

[Thomas de Grenier de Latour]

On Tue, 01 Mar 2005 22:15:51 +0000
Stephen Bennett <spb@gentoo.org> wrote:

> http://dev.gentoo.org/~spb/metapkg-glep.txt.
> Comments/suggestions/flames welcome.

A first minor drawback i see is that it complicates things for
people who use overlay ebuilds. For instance, making your own
kernel sources ebuilds right now is has simple as inheriting an
eclass, which will give them the right PROVIDES line. But with
this glep, you will also have to overlay the metapkg and add
declare your sources in it, and then manually maintain it to keep
in sync with the one in the tree (cause you don't want to break
other official kernel sources packages). In that respect, the
current virtuals system is much more confortable, because ebuilds
are really standalone, not inter-dependent. 


An imho more serious drawback is that there is, in this proposal,
no way for a user to explicitly declare what implementation of a
virtual he prefers. As i've already said in the previous thread, i
think this /etc/portage/profiles/virtuals was a very good thing,
and there is no such user prefs file with this glep. I'll try to
explain why it bothers me with an example:

 - metaM depends on "|| ( pkgA pkgB pkgC )"
 - pkgD depends on metaM
 - pkgE depends on pkgC

If user "emerge pkgD" and then "emerge pkgE", he will have both
pkgA and pkgC installed as deps.
If at the contrary he "emerge pkgE" before "emerge pkgD", he will
only have pkgC installed to satisfy both deps.
If he then wants to clone his system on an other station by doing
an "emerge $(< world.orig)", he may have only pkgC or both pkgA
and pkgC, depending on how the world file is ordered.

And if his prefered implementation of metaM is pkgB anyway, he
would better emerge it explicitly soon enough, before anything
else that depends on metaM. Actually, a picky user should dig
through all the virtuals right after his installation and
explicitly emerge all the packages for which he has a preference,
whereas with the current virtuals implementation he can state that
in a config file let emerge deal with it when the virtuals are
actually depended on.

So, in short, dropping the "virtuals" file is a regression imho,
because it makes portage behavior less deterministic; it will
depend more on the history of user actions (calls to emerge), and
less on user prefs, making it harder to understand and control.

Oh, and btw, this lack of "virtuals" prefs file may also be a
problem for official profiles. For instance, I see that "base"
currently has "virtual/jdk  dev-java/blackdown-jdk", whereas ppc
profiles override that with "virtual/jdk  dev-java/ibm-jdk-bin".
And i guess they have good reasons to do so... How will that be
handled with metapkgs? By having numerous "arch? ( ... )" in the
DEPEND string? 

-- 
TGL.
--
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] GLEP ??: Metapackages

[Stephen Bennett]

On Mon, 2005-03-07 at 11:26 +0100, Thomas de Grenier de Latour wrote:
> So, in short, dropping the "virtuals" file is a regression imho,
> because it makes portage behavior less deterministic; it will
> depend more on the history of user actions (calls to emerge), and
> less on user prefs, making it harder to understand and control.

What's been discussed, but isn't in the GLEP draft, is a /etc/portage
file to let you override metapackage dependencies. It's not in there
because the syntax and implementation hasn't been worked out fully, but
it should work akin to /etc/portage/virtuals.

> Oh, and btw, this lack of "virtuals" prefs file may also be a
> problem for official profiles. For instance, I see that "base"
> currently has "virtual/jdk  dev-java/blackdown-jdk", whereas ppc
> profiles override that with "virtual/jdk  dev-java/ibm-jdk-bin".
> And i guess they have good reasons to do so... How will that be
> handled with metapkgs? By having numerous "arch? ( ... )" in the
> DEPEND string? 

It could be done with arch? ( ) in the DEPEND string, or just by not
keywording packages on a given arch, or masking them in the profile. The
dependency resolver is already fully able to cope with that sort of
situation, whereas the current virtuals mechanism isn't.

--
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] GLEP ??: Metapackages

[Thomas de Grenier de Latour]

Two other very minor drawbacks i've just thought of:

 - no easy replacement for the use of virtuals in "use.defaults"
files (currently, virtual/jre, virtual/opengl and virtual/x11 are
used in that files)

 - no easy replacement for "has_version virtual/something" in
ebuilds (currently, such statements exists with virtual/emacs,
virtual/ghc, virtual/php, virtual/os-headers, virtual/jdk, but
that's in less than 30 ebuilds in the whole tree)

-- 
TGL.
--
gentoo-dev@gentoo.org mailing list