[gentoo-dev] Suggestion for new dependency syntax

[gentoo-dev] Suggestion for new dependency syntax

[Karl Trygve Kalleberg]

Hi gang.

I have discussed the need for this with drobbins last week. Sorry it took
so long before my suggestion materialized.

Woodchip: Warning, long and complicated mail (but this is justified by 
sticking to one topic through the entire mail, I should hope) ;p



We have sometimes discussed the need for a more precise syntax (and adhering
implementation) for specifying dependencies between packages.

Why do we want a more powerful dependency system ?

- If for no other reason, it's because the current syntax cannot gracefully
  handle ! (not)
- We really want to be able to have a set-theoretic-based syntax, since that
  is what is employed (logically) inside Portage.

Since the syntax is so closely tied to the functionality it represents, I 
will spare you the linguistic babble about semantics and rules, and rather
let the syntax speak for itself.


New syntax for complex dependencies:

 dev-libs/glib[ >=1.2 ]

 - This can of course also be written a tad simpler as

 >=dev-libs/glib-1.2


 dev-libs/glib[ >=1.2 <=1.2.14 ]

 - This defines the interval from 1.2.0 to 1.2.14, inclusive.

 dev-libs/glib[ >1.2 <1.2.14 =1.2.18]

 - This defines the interval from 1.2.1 to 1.2.13, inclusive (yes,
   read twice if you disagree), plus 1.2.18

 dev-libs/glib[1.2*]

 - This defines any 1.2 version, and is equivalent to

 dev-libs/glib-1.2*

 
 dev-libs/glib[1.2* !1.2.12* 1.2.12.3]

 - All 1.2 versions are okay, _except_ any 1.2.12 versions , 
   but 1.2.12.3 is okay again. (Call this set A).

 dev-libs/glib[ ((1.2* !1.2.12* 1.2.12.3) | 1.3*) & >=1.4.12 ]

 - Any of the versions in set A are okay, or any 1.3 version. In addition,
   we also have to have 1.4.12 or newer installed.

   This is where the new system allows us to package things that are really
   weird and require multiple versions of various things, without resorting 
   to calling the base package libxml2 or ssh2 or qt3, or whatever.

The same syntactical constructs can also be applied at inter-package
level:

DEPEND="[ dev-libs/foo[1.2* !1.2.12*] | >=dev-libs/bar-1.3 ]
        =dev-libs/baz-1.0"

DEPEND="foo? ([ dev-libs/foo[1.2* !1.2.12*] & 
                >=dev-libs/baz-1.0 |
                =dev-libs/baar-1.0] )"


In general, [] enclose the new syntactical constructs, and thus act as
an 'escape mechanism' from the old syntax.

One thing about the '|' (or) construct: foo | bar is equivalent to:
if not foo, then bar, it does NOT mean foo & !bar.


The main problem of exporting such as powerful/possibly complex mechanism 
as this, is that ebuilds can become unmaintainable. I really do not think 
that will be a problem, since for the majority of ebuilds, the "simplified"
(ie, old) syntax is more than enough. 

Only for fairly complex dependendency cases will the (full) power of the 
new syntax be applied, and that will probably always be done by the hardcore
Gentoo packagers anyway. 

I have elected not to comment on some of the more subtle points at various
points above. If anybody wants to get really dirty, I suggest we do it on
IRC.

Comments are required!



Kind regards,

Karl T

Re: [gentoo-dev] Suggestion for new dependency syntax

[Dan Armak]

Hi,

This looks really great :-). 

One note: what about conflicting/incompatible packages? Will/should we have a 
syntax for saying, this package must not be installed?
Is that what happens when I say !foo[...] (as distinct from foo[!...])?

-- 

Dan Armak
Gentoo Linux Developer, Desktop Team
Matan, Israel

Re: [gentoo-dev] Suggestion for new dependency syntax

[Karl Trygve Kalleberg]

On Thu, Oct 25, 2001 at 07:12:20AM +0200, Dan Armak wrote:
> Hi,
> 
> This looks really great :-). 
> 
> One note: what about conflicting/incompatible packages? Will/should we have a 
> syntax for saying, this package must not be installed?
> Is that what happens when I say !foo[...] (as distinct from foo[!...])?

I am working on that problem just now. I do not really want the ebuilds to
be the (only?) place to specify that. Reason is as follows:
1) We add package foo-1.0 on Monday
2) We add package bar-1.0 on Tuesday, which we know conflicts with foo.
   At this point, we will have to update all foo ebuilds to tell Portage
   that foo cannot coexist with bar.

I think we should have packages.conflict or somesuch alongside
packages.mask, and that all specification of conflicts should be handled
there. Otherwise, Portage would have to parse *all* ebuilds whenever it
tried to calculate dependencies (or all conflicts would have to be updated
in all mutually conflicting ebuilds whenever a new ebuild enters the tree,
and that is butt ugly).


Kind regards,

Karl T

Re: [gentoo-dev] Suggestion for new dependency syntax

[Jon Nelson]

On Fri, 26 Oct 2001 13:52:21 +0200
Karl Trygve Kalleberg <karltk@prosalg.no> wrote:

> On Thu, Oct 25, 2001 at 07:12:20AM +0200, Dan Armak wrote:
> > Hi,
> > 
> > This looks really great :-). 
> > 
> > One note: what about conflicting/incompatible packages? Will/should we
> have a 
> > syntax for saying, this package must not be installed?
> > Is that what happens when I say !foo[...] (as distinct from
> foo[!...])?

Instead of using ! why not add a CONFLICTS = " foo bar baz " line.
Debian has Conflicts, Requires, Suggests and a few others.
RedHat has Conflicts, Requires, PreRequires etc..
("it" below refers to the dependency)
Requires means it is required to *run*
PreRequires means it must be installed and config'd for the package 
  to install


-- 
Jon Nelson
jnelson@jamponi.net
C and Python Programmer
Motorcycle Enthusiast

[gentoo-dev] "Conflicts" [was -Re: Suggestion for new dependency syntax-]

[Tom von Schwerdtner]

> Instead of using ! why not add a CONFLICTS = " foo bar baz " line.
> Debian has Conflicts, Requires, Suggests and a few others.
> RedHat has Conflicts, Requires, PreRequires etc..
> ("it" below refers to the dependency)
> Requires means it is required to *run*
> PreRequires means it must be installed and config'd for the package 
>   to install

Debian's "conflicts" are only conflicts in the sense that the Debian 
package management system (or practices) make it so.  There is no reason 
why you cant have 4 httpd servers and every DBMS on the planet installed 
on your system.  Granted, in most cases you only need one httpd server or 
one DBMS, but you shouldnt need to work around the package mangement 
system to get what you want.

If there is a conflict system for Gentoo (and if there is I dont think it 
should be called "conflicts" when its not), there should be a simple way 
to turn it off (emerge --noconflicts).

-Tom  

[gentoo-dev] Re: "Conflicts" [was -Re: Suggestion for new dependency syntax-]

[Jon Nelson]

On Fri, 26 Oct 2001 18:10:10 +0200 (CEST)
"Tom von Schwerdtner" <tvon@etria.org> wrote:

> > Instead of using ! why not add a CONFLICTS = " foo bar baz " line.
> > Debian has Conflicts, Requires, Suggests and a few others.
> > RedHat has Conflicts, Requires, PreRequires etc..
> > ("it" below refers to the dependency)
> > Requires means it is required to *run*
> > PreRequires means it must be installed and config'd for the package 
> >   to install
> 
> Debian's "conflicts" are only conflicts in the sense that the Debian 
> package management system (or practices) make it so.  There is no reason
> why you cant have 4 httpd servers and every DBMS on the planet installed
> on your system.  Granted, in most cases you only need one httpd server
> or 
> one DBMS, but you shouldnt need to work around the package mangement 
> system to get what you want.

Debian's "conflicts" don't prevent more than 1 httpd from being
installed -- they prevent a case like this:

Package: foo
Conflicts: bar

Package: bar
<notice, no Conflicts line>

You install foo, then want to install bar.
it would complain, saying, "hey, you can't do that without
damaging the install of foo!"

You install bar, and want to install foo.
...

Conflicts are more useful when two packages really do conflict,
not at the purpose level, but at the filesystem level.

Which reminds me.  On my Debian box, I can install
gcc-2.95.X *and* gcc-3.0 and both are available and both
are seperate.  I choose which I want to use by calling
either gcc (gcc-2.95.X) or gcc-3.0 (gcc-3.0), something that
can often be set in configure scripts like this:

CC=gcc-3.0 ./configure


-- 
Jon Nelson
jnelson@jamponi.net
C and Python Programmer
Motorcycle Enthusiast

Re: [gentoo-dev] Suggestion for new dependency syntax

[Grant Goodyear]

Conflicts are pretty rare, and most commonly of the ! virtual/mta
type (where we only want one system mailer installed), so I would
think that keeping the conflicts part of the ebuilds shouldn't be
all that bad.  

-g2boojum-
-- 
___________________________________________________________________
|     Grant Goodyear                  |  The Secrets of Physics:   |
|     Dept. of Chemistry - Clemson U  |1. Add zero.                |
|     Clemson, SC  29634              |2. Multiply by one.         |
|-------------------------------------|3. Expand in a Taylor series|
|e-mail: goodyea@clemson.edu          |4. Integrate by parts.      |
|www:bernacchi.chem.uh.edu/~grant     |5. Fourier transform.       |
|                                     |6. Add auxiliary variables  |
|_____________________________________|____________________________|
 

Re: [gentoo-dev] Suggestion for new dependency syntax

[Jon Nelson]

On Fri, 26 Oct 2001 11:10:25 -0400
Grant Goodyear <grant@g2.ces.clemson.edu> wrote:

> Conflicts are pretty rare, and most commonly of the ! virtual/mta
> type (where we only want one system mailer installed), so I would
> think that keeping the conflicts part of the ebuilds shouldn't be
> all that bad.  

What data are you using for the "Conflicts are rare" statement?
Using 
http://http.us.debian.org/debian/dists/testing/main/binary-i386/Packages
I get:

[jnelson@agamemnon jnelson]$ grep "Package: " Packages | wc -l
   6669
[jnelson@agamemnon jnelson]$ grep "Conflicts: " Packages | wc -l
   1637
[jnelson@agamemnon jnelson]$ grep "Replaces: " Packages | wc -l
   1265
[jnelson@agamemnon jnelson]$ grep "Provides: " Packages | wc -l
    923

[jnelson@agamemnon jnelson]$ python
Python 1.5.2 (#0, Aug  2 2001, 11:14:43)  [GCC egcs-2.91.66 19990314/Linux
(egcs-1.1.2 release)] on linux-i386
Copyright 1991-1995 Stichting Mathematisch Centrum, Amsterdam
>>> 1637.0 * 100 / 6669.0
24.5464087569
>>> 1265.0 * 100 / 6669.0
18.9683610736
>>> 923.0 * 100 / 6669.0
13.8401559454


So roughly 25% of the packages in Debian have a Conflicts line,
and ~20 percent have a Replaces line (afaik, replaces is for
when a package changes names).  Provides is for meta packages,
and is present in roughly 14% of the packages.

25% is certainly not "pretty rare".

An example from Debian:

Replaces: cdgrab
Provides: cdgrab
Depends: cd-discid, wget, cdparanoia | cdda2wav, vorbis-tools (>=
1.0beta4-1)
Suggests: eject, distmp3, id3 (>= 0.12), id3v2
Conflicts: cdgrab

Yikes!  All of that is useful, however.

As I understand it, gentoo has 'meta' packages as well.  Debian
uses these to great effect, but I've always been bothered
by the naming scheme -- perhaps 'meta-c-compiler' vs. 'gcc', and
'meta-httpd' vs. 'httpd' or some specific httpd would be better.


-- 
Jon Nelson
jnelson@jamponi.net
C and Python Programmer
Motorcycle Enthusiast

Re: [gentoo-dev] Suggestion for new dependency syntax

[Grant Goodyear]

> What data are you using for the "Conflicts are rare" statement?
> Using 
> http://http.us.debian.org/debian/dists/testing/main/binary-i386/Packages
> I get:
> 
> [jnelson@agamemnon jnelson]$ grep "Package: " Packages | wc -l
>    6669
> [jnelson@agamemnon jnelson]$ grep "Conflicts: " Packages | wc -l
>    1637
> [jnelson@agamemnon jnelson]$ grep "Replaces: " Packages | wc -l
>    1265
> [jnelson@agamemnon jnelson]$ grep "Provides: " Packages | wc -l
>     923

Admittedly, my data was apocryphal.  I've seen very few complaints
about conflicting packages on either #gentoo or gentoo-dev.  The
example you gave was a package conflicting with a previous version of
itself, which strikes me as being a bit artificial, but you may well
have a good point.  *Shrug*  I'll go back to being agnostic on the
issue.

-g2boojum-
-- 
___________________________________________________________________
|     Grant Goodyear                  |  The Secrets of Physics:   |
|     Dept. of Chemistry - Clemson U  |1. Add zero.                |
|     Clemson, SC  29634              |2. Multiply by one.         |
|-------------------------------------|3. Expand in a Taylor series|
|e-mail: goodyea@clemson.edu          |4. Integrate by parts.      |
|www:bernacchi.chem.uh.edu/~grant     |5. Fourier transform.       |
|                                     |6. Add auxiliary variables  |
|_____________________________________|____________________________|
 

Re: [gentoo-dev] Suggestion for new dependency syntax

[Tom von Schwerdtner]

> What data are you using for the "Conflicts are rare" statement?
> Using 
>
[snip]
> 
> So roughly 25% of the packages in Debian have a Conflicts line,
> and ~20 percent have a Replaces line (afaik, replaces is for
> when a package changes names).  Provides is for meta packages,
> and is present in roughly 14% of the packages.
> 
> 25% is certainly not "pretty rare".

Not if you take Debian conflicts seriously, but you really shouldnt.  
Debian packages control /etc to a large extent, and it is very hard to 
have scripts that properly setup /etc files (esp on Debian where you have 
the 'alternatives' setup) when a system can potentially have multiple 
apps/servers that do the same abstract job (MTA/DBMS/HTTPD/SSHD...etc..), 
so Debian takes care of this by saying "OK, you can only have 1 httpd, 1 
DBMS, 1 MTA and 1 sshd".  This way, the scipts just have to deal with 
whatever package they are contained in and dont have to deal with the 
situation where there is already a package installed that does the same 
abstract job.

Gentoo leaves /etc to the admin, which is good, so there is no reason why 
Gentoo should have the same "conflicts" that Debian has.

Gentoo also installs from sources instead of pre-built binaries, which 
leaves alot more flexability to the configuration of the system.

Basically, Debian conflicts are just that, _Debian_ conflicts, and they 
should not be taken as anything more than that.

-Tom

BTW: I am not some seasoned Debian developer, these views come from using 
Debian for a while and having to work around the various Debian conflicts 
that I ran into......yes, Debian conflicts irritate the hell out of me :)

Re: [gentoo-dev] Suggestion for new dependency syntax

[Jon Nelson]

On Fri, 26 Oct 2001 18:55:44 +0200 (CEST)
"Tom von Schwerdtner" <tvon@etria.org> wrote:

> Not if you take Debian conflicts seriously, but you really shouldnt.  
> Debian packages control /etc to a large extent, and it is very hard to 
> have scripts that properly setup /etc files (esp on Debian where you
> have 
> the 'alternatives' setup) when a system can potentially have multiple 
> apps/servers that do the same abstract job
> (MTA/DBMS/HTTPD/SSHD...etc..), 
> so Debian takes care of this by saying "OK, you can only have 1 httpd, 1
> DBMS, 1 MTA and 1 sshd".  This way, the scipts just have to deal with 
> whatever package they are contained in and dont have to deal with the 
> situation where there is already a package installed that does the same 
> abstract job.

I'm sorry, but it's clear you aren't very familiar with Debian.
Nowhere does it say, either in Policy or in Conflicts, etc... that you
can't have multiple httpds installed. I currently have boa, thttpd,
and apache installed an I'm running Debian 'frozen'.  It was as simple
as 'apt-get install boa thttpd apache'

I'm also very confused by what you mean by "scripts" when you say
"the scripts just have to deal with whatever package they are 
contained in and..."   Do you mean init scripts?? Debian's Apache
properly (IMHO) ships /etc/init.d/apache, not /etc/init.d/httpd.
Boa ships /etc/init.d/boa and so on.
Apache, Boa,and others do not conflict, although most "provide"
httpd.  httpd is a "virtual" (meta) package for other packages that
want to depend on one but don't care specifically which.

> Gentoo leaves /etc to the admin, which is good, so there is no reason
> why 
> Gentoo should have the same "conflicts" that Debian has.

Again, I think you misunderstand what 'conflicts' are in Debian.
I have no idea what conflicts has to do with /etc, specifically.

> BTW: I am not some seasoned Debian developer, these views come from
> using 
> Debian for a while and having to work around the various Debian
> conflicts 
> that I ran into......yes, Debian conflicts irritate the hell out of me
> :)

I've been a Debian developer and user for many years now, and
Debian "conflicts" are, IMHO, extraordinarily useful -- they 
prevent me from accidentally killing other programs when I install.

Perhaps a quick read of:
http://www.debian.org/doc/debian-policy/ch-relationships.html
will forestall any more "I heard this so it must be true.".




-- 
Jon Nelson
jnelson@jamponi.net
C and Python Programmer
Motorcycle Enthusiast