[gentoo-dev] Destroying dependancies...

[gentoo-dev] Destroying dependancies...

[Per Wigren]

Yesterday, for about 30 hours all my newly recieved mail went to /dev/null 
because of a portage limitation...
I have an old computes which acts as a firewall/router, ftpserver webserver 
and mailserver with IMAP and spamassassin...  I have "mysql" in my USE-flags 
on said server..  Yesterday portage upgraded MySQL from 3.23.56 to 4.0.12..  
postfix and maildrop both linked to /usr/lib/libmysqlclient.so.10 which went 
away when MySQL was upgraded..  When the still running postfix delivered the 
mail using dropmail, dropmail just died with "error while loading shared 
libraries: no such file or directory" and for some reason (I'd call it a bug 
in postfix) it didn't defer them, they just went to /dev/null...
Anyway, I won't whine about it.. Instead I'm looking into finding a solution 
for the REAL problem... 

A solution I've been thinking about is this:

When a package is installed portage will run ldd on all binaries and libs and 
put its library-depends in a file in its /var/db/pkg-directory.

Later when a package is upgraded, it will scan the LIBDEPS-files for broken 
dependancies and tell the user something like:
* WARNING: The following packages needs to be reinstalled because of broken 
dependancies:
  * net-www/blabla
  * app-games/blablabla
  * media-video/blabla

And when a package is uninstalled with -C or --clean, it will scan the 
LIBDEPS-files for the libs it will remove and refuse to uninstall it if it 
breaks anything. Maybe with a warning like this:
* WARNING: dev-libs/libblablabla can't be uninstalled because the following 
packages depends on it:
  * net-www/blabla
  * app-games/blablabla
  * media-video/blabla
* use --force to uninstall it anyway.



Anyone have a better solution?

Regards,
Per Wigren

--
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] Destroying dependancies...

[Michael Taylor]

Would there be any way to do this with the dependancy information already stored in the "/var/db/pkg" directory.  It will probably be less accurate than the ldd sollution since it wouldn't necessarily mean that a binary or library was linked against a package, but it should at least give a "worst case senario" of what you might have to reemerge.

--Michael

On Wed, 26 Mar 2003 01:39:31 +0100
Per Wigren <wigren@home.se> wrote:

> Yesterday, for about 30 hours all my newly recieved mail went to /dev/null 
> because of a portage limitation...
> I have an old computes which acts as a firewall/router, ftpserver webserver 
> and mailserver with IMAP and spamassassin...  I have "mysql" in my USE-flags 
> on said server..  Yesterday portage upgraded MySQL from 3.23.56 to 4.0.12..  
> postfix and maildrop both linked to /usr/lib/libmysqlclient.so.10 which went 
> away when MySQL was upgraded..  When the still running postfix delivered the 
> mail using dropmail, dropmail just died with "error while loading shared 
> libraries: no such file or directory" and for some reason (I'd call it a bug 
> in postfix) it didn't defer them, they just went to /dev/null...
> Anyway, I won't whine about it.. Instead I'm looking into finding a solution 
> for the REAL problem... 
> 
> A solution I've been thinking about is this:
> 
> When a package is installed portage will run ldd on all binaries and libs and 
> put its library-depends in a file in its /var/db/pkg-directory.
> 
> Later when a package is upgraded, it will scan the LIBDEPS-files for broken 
> dependancies and tell the user something like:
> * WARNING: The following packages needs to be reinstalled because of broken 
> dependancies:
>   * net-www/blabla
>   * app-games/blablabla
>   * media-video/blabla
> 
> And when a package is uninstalled with -C or --clean, it will scan the 
> LIBDEPS-files for the libs it will remove and refuse to uninstall it if it 
> breaks anything. Maybe with a warning like this:
> * WARNING: dev-libs/libblablabla can't be uninstalled because the following 
> packages depends on it:
>   * net-www/blabla
>   * app-games/blablabla
>   * media-video/blabla
> * use --force to uninstall it anyway.
> 
> 
> 
> Anyone have a better solution?
> 
> Regards,
> Per Wigren
> 
> --
> gentoo-dev@gentoo.org mailing list
> 
> 


-----------------------------------------------------------
Michael Taylor
mtaylor@member.fsf.org

--
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] Destroying dependancies...

[Paul de Vrieze]

[-- Attachment #1: signed data --]
[-- Type: text/plain, Size: 660 bytes --]

On Wednesday 26 March 2003 02:04, Michael Taylor wrote:
> Would there be any way to do this with the dependancy information already
> stored in the "/var/db/pkg" directory.  It will probably be less accurate
> than the ldd sollution since it wouldn't necessarily mean that a binary or
> library was linked against a package, but it should at least give a "worst
> case senario" of what you might have to reemerge.
>

All the information is there (has been for at least a year, probably longer), 
just the code isn't there. I'd say, go for it!

Paul

-- 
Paul de Vrieze
Researcher
Mail: pauldv@cs.kun.nl
Homepage: http://www.cs.kun.nl/~pauldv

[-- Attachment #2: signature --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-dev] Destroying dependancies...

[Per Wigren]

Hi!
No, I didn't see that, because it was upgraded as part of a -u world, which I 
didn't watch all the time... It should have said it at the end of the -u 
world instead of in the middle.. Then I had probably seen it..

Anyway, I said I wasn't going to whine about it.. Instead I want to find a 
better solution for portage to handle stuff like this in the future..

// Per

On Wednesday 26 March 2003 02:22, you wrote:
> On Wed, Mar 26, 2003 at 01:39:31AM +0100, Per Wigren wrote:
> > Anyone have a better solution?
>
> I was involved in the discussion on MySQL v4 on the bugzilla tracking
> system.
>
> Did you not read the warning that the MySQL v4 build spat out when you
> merged it in?
>
> "If you're upgrading from MySQL-3.x, you must recompile the other
> packages on your system that link with libmysqlclient after the upgrade
> completes.  To obtain such a list of packages for your system, you may
> use the: /usr/portage/dev-db/mysql/files/rebuilder.sh script."


--
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] (Was: Destroying dependancies...) ewarn/einfo logging

[Robin H. Johnson]

[-- Attachment #1: Type: text/plain, Size: 2051 bytes --]

On Wed, Mar 26, 2003 at 02:43:10AM +0100, Per Wigren wrote:
> No, I didn't see that, because it was upgraded as part of a -u world,
> which I didn't watch all the time... It should have said it at the end
> of the -u world instead of in the middle.. Then I had probably seen
> it..
Hmm, that's a good point, it would have been much more visible if all
the notes were at the end of a large emerge, and perhaps also
automatically logged to a file somewhere for later review? I know this
would have personally also saved me many headaches if I had noticed the
output from vpopmail when I was trying to get it warning.

I don't know how one would go about spitting all the stuff out at the
end, as it is presently done with the ewarn/einfo functions in ebuilds.
However putting a copy of the messages in a directory somewhere
shouldn't be too hard, and this would go a long way to what we want,
while being a useful feature in itself.

Somebody did once submit this to the bugzilla tree, but it looks like it
never went very far:
http://bugs.gentoo.org/show_bug.cgi?id=11359

I've copied the original submitter on this email, so that he hopefully
sees it and checks if those patches still apply to portage.

> Anyway, I said I wasn't going to whine about it.. Instead I want to
> find a better solution for portage to handle stuff like this in the
> future..
Agreed. I was merely inquiring if the warning had been seen or not. The
developers have noted the need for a way to cause a set of packages to
be rebuilt after a merge, as this issue has happened on two? packages
now (libpng and mysql) I believe. The problem is that to effectively
find out what packages need to be rebuilt, large parts of the system
must be scanned (see the script in
/usr/portage/dev-db/mysql/files/rebuilder.sh)

-- 
Robin Hugh Johnson
E-Mail     : robbat2@orbis-terrarum.net
Home Page  : http://www.orbis-terrarum.net/?l=people.robbat2
ICQ#       : 30269588 or 41961639
GnuPG FP   : 11AC BA4F 4778 E3F6 E4ED  F38E B27B 944E 3488 4E85

[-- Attachment #2: Type: application/pgp-signature, Size: 232 bytes --]

Re: [gentoo-dev] (Was: Destroying dependancies...) ewarn/einfo logging

[Nick Jones]

> Hmm, that's a good point, it would have been much more visible if all
> the notes were at the end of a large emerge, and perhaps also
> automatically logged to a file somewhere for later review? I know this
> would have personally also saved me many headaches if I had noticed the
> output from vpopmail when I was trying to get it warning.

PORT_LOGDIR=   <--- Look for that in make.conf

--
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] (Was: Destroying dependancies...) ewarn/einfo logging

[Robin H. Johnson]

[-- Attachment #1: Type: text/plain, Size: 1091 bytes --]

On Wed, Mar 26, 2003 at 03:12:48AM -0600, Nick Jones wrote:
> > Hmm, that's a good point, it would have been much more visible if all
> > the notes were at the end of a large emerge, and perhaps also
> > automatically logged to a file somewhere for later review? I know this
> > would have personally also saved me many headaches if I had noticed the
> > output from vpopmail when I was trying to get it warning.
> PORT_LOGDIR=   <--- Look for that in make.conf
Umm, from what I can see, something is not consistant...
Looking at that because I had it before, some files contain just the
ewarn/einfo (which is what I wanted), but others contain the entire
merge output, which isn't wanted, and is wasteful unless you want it for
debugging purposes (15mb file for glibc).

Is PORT_LOGDIR supposed to contain just ewarn/einfo or the entire
output?

-- 
Robin Hugh Johnson
E-Mail     : robbat2@orbis-terrarum.net
Home Page  : http://www.orbis-terrarum.net/?l=people.robbat2
ICQ#       : 30269588 or 41961639
GnuPG FP   : 11AC BA4F 4778 E3F6 E4ED  F38E B27B 944E 3488 4E85

[-- Attachment #2: Type: application/pgp-signature, Size: 232 bytes --]

Re: [gentoo-dev] (Was: Destroying dependancies...) ewarn/einfo logging

[Peter Ruskin]

On Wednesday 26 Mar 2003 09:12, Nick Jones wrote:
> > Hmm, that's a good point, it would have been much more visible if all
> > the notes were at the end of a large emerge, and perhaps also
> > automatically logged to a file somewhere for later review? I know
> > this would have personally also saved me many headaches if I had
> > noticed the output from vpopmail when I was trying to get it warning.
>
> PORT_LOGDIR=   <--- Look for that in make.conf
>
I have that set to /var/log/portage but /var/log/portage contains only 
zero-length files such as 4959-dia-0.91.log, whether or not they pass or 
fail

Peter
-- 
Gentoo-1.4.3.4 Unstable. KDE: 3.1.1 Qt: 3.1.2
AMD Athlon(tm) XP 1900+ 768MB.	Kernel: 2.4.20-xfs-r2.	GCC 3.2.2


--
gentoo-dev@gentoo.org mailing list

Re[2]: [gentoo-dev] (Was: Destroying dependancies...) ewarn/einfo logging

[Renat Golubchyk]

Hi!

>> PORT_LOGDIR=   <--- Look for that in make.conf
>>
PR> I have that set to /var/log/portage but /var/log/portage contains only 
PR> zero-length files such as 4959-dia-0.91.log, whether or not they pass or 
PR> fail
Same problem here.

Cheers,
Renat


--
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] Destroying dependancies...

[Alain Penders]

On Wed, Mar 26, 2003 at 01:39:31AM +0100, Per Wigren wrote:
> Yesterday, for about 30 hours all my newly recieved mail went to /dev/null 
> because of a portage limitation...
> I have an old computes which acts as a firewall/router, ftpserver webserver 
> and mailserver with IMAP and spamassassin...  I have "mysql" in my USE-flags 
> on said server..  Yesterday portage upgraded MySQL from 3.23.56 to 4.0.12..  
> postfix and maildrop both linked to /usr/lib/libmysqlclient.so.10 which went 
> away when MySQL was upgraded..  When the still running postfix delivered the 
> mail using dropmail, dropmail just died with "error while loading shared 
> libraries: no such file or directory" and for some reason (I'd call it a bug 
> in postfix) it didn't defer them, they just went to /dev/null...
> Anyway, I won't whine about it.. Instead I'm looking into finding a solution 
> for the REAL problem... 

The problem is that portage doesn't do reverse dependancy checking.  It needs 
to check that a package doesn't break another package that depends on it 
before doing an upgrade... and if it does break something, propose upgrading 
those packages as well.

Alain

--
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] Destroying dependancies...

[Per Wigren]

On Friday 28 March 2003 16:14, Alain Penders wrote:
> On Wed, Mar 26, 2003 at 01:39:31AM +0100, Per Wigren wrote:
> > Yesterday, for about 30 hours all my newly recieved mail went to
> > /dev/null because of a portage limitation...
> > I have an old computes which acts as a firewall/router, ftpserver
> > webserver and mailserver with IMAP and spamassassin...  I have "mysql" in
> > my USE-flags on said server..  Yesterday portage upgraded MySQL from
> > 3.23.56 to 4.0.12.. postfix and maildrop both linked to
> > /usr/lib/libmysqlclient.so.10 which went away when MySQL was upgraded.. 
> > When the still running postfix delivered the mail using dropmail,
> > dropmail just died with "error while loading shared libraries: no such
> > file or directory" and for some reason (I'd call it a bug in postfix) it
> > didn't defer them, they just went to /dev/null... Anyway, I won't whine
> > about it.. Instead I'm looking into finding a solution for the REAL
> > problem...
>
> The problem is that portage doesn't do reverse dependancy checking.  It
> needs to check that a package doesn't break another package that depends on
> it before doing an upgrade... and if it does break something, propose
> upgrading those packages as well.

But how do we check that with the current information?
To use the postfix-case above as an example..
Postfix works fine with both MySQL v3.x and MySQL v4.x.
If it is compiled against v3 it has to be recompiled if MySQL is upgraded from 
v3 to v4, but not if it is upgraded from v4.0.11 to v4.0.12..

To use MPlayer as another example..
MPlayer has to be recompiled EVERY TIME that DirectFB is upgraded, even minor 
version, because it links to .so.<complete versionnumber> ..  But it doesn't 
need to be recompiled if for example Vorbis is upgraded from v1.0.1 to v1.0.2 
or even v1.8.5 because it links to libvorbis.so.0 ...


--
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] Destroying dependancies...

[Matt Tucker]

-- Per Wigren <wigren@home.se> spake thusly:

> But how do we check that with the current information?
> To use the postfix-case above as an example..
> Postfix works fine with both MySQL v3.x and MySQL v4.x.
> If it is compiled against v3 it has to be recompiled if MySQL is
> upgraded from  v3 to v4, but not if it is upgraded from v4.0.11 to
> v4.0.12..
> 
> To use MPlayer as another example..
> MPlayer has to be recompiled EVERY TIME that DirectFB is upgraded,
> even minor  version, because it links to .so.<complete versionnumber>
> ..  But it doesn't  need to be recompiled if for example Vorbis is
> upgraded from v1.0.1 to v1.0.2  or even v1.8.5 because it links to
> libvorbis.so.0 ...

Those may be two different answers. In the case of MySQL, perhaps 4.0
should've been a different slot? In the case of MPlayer, clearly
portage needs a new feature which allows either MPlayer or DirectFB to
specify that a recompile needs to happen when DirectFB is upgraded.

Perhaps it would make more sense for it to be specified in the
dependancy, because that would allow MySQL 4.0 to specify the same
thing. Of course, you'd need to base it on version:

    RECOMPILE_DEPENDS="<DirectFB-${PV}"     or
    RECOMPILE_DEPENDS="*"

    RECOMPILE_DEPENDS="<MySQL-4.0"


--
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] Destroying dependancies...

[Per Wigren]

On Friday 28 March 2003 19:12, Matt Tucker wrote:
> -- Per Wigren <wigren@home.se> spake thusly:
> > But how do we check that with the current information?
> > To use the postfix-case above as an example..
> > Postfix works fine with both MySQL v3.x and MySQL v4.x.
> > If it is compiled against v3 it has to be recompiled if MySQL is
> > upgraded from  v3 to v4, but not if it is upgraded from v4.0.11 to
> > v4.0.12..
> >
> > To use MPlayer as another example..
> > MPlayer has to be recompiled EVERY TIME that DirectFB is upgraded,
> > even minor  version, because it links to .so.<complete versionnumber>
> > ..  But it doesn't  need to be recompiled if for example Vorbis is
> > upgraded from v1.0.1 to v1.0.2  or even v1.8.5 because it links to
> > libvorbis.so.0 ...
>
> Those may be two different answers. In the case of MySQL, perhaps 4.0
> should've been a different slot? In the case of MPlayer, clearly
> portage needs a new feature which allows either MPlayer or DirectFB to
> specify that a recompile needs to happen when DirectFB is upgraded.
>
> Perhaps it would make more sense for it to be specified in the
> dependancy, because that would allow MySQL 4.0 to specify the same
> thing. Of course, you'd need to base it on version:
>
>     RECOMPILE_DEPENDS="<DirectFB-${PV}"     or
>     RECOMPILE_DEPENDS="*"
>
>     RECOMPILE_DEPENDS="<MySQL-4.0"

I still don't get how that would work.. I also think that would make room for 
a lot of human mistakes..

The easiest way must be to run "ldd" on all executables and "*.so*" on 
install-time and put the dependancies in a LIBDEPS-file..
When uninstalling/upgrading, check for "*.so*" that will be removed and "grep 
libfile.so.0 /var/db/pkg/*/*/LIBDEPS" and mark them for recompilation...

// Per Wigren

--
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] Destroying dependancies...

[Robin H. Johnson]

[-- Attachment #1: Type: text/plain, Size: 1289 bytes --]

On Fri, Mar 28, 2003 at 05:54:54PM +0100, Per Wigren wrote:
> But how do we check that with the current information?
> To use the postfix-case above as an example..
> Postfix works fine with both MySQL v3.x and MySQL v4.x.
> If it is compiled against v3 it has to be recompiled if MySQL is upgraded from 
> v3 to v4, but not if it is upgraded from v4.0.11 to v4.0.12..
> 
> To use MPlayer as another example..
> MPlayer has to be recompiled EVERY TIME that DirectFB is upgraded, even minor 
> version, because it links to .so.<complete versionnumber> ..  But it doesn't 
> need to be recompiled if for example Vorbis is upgraded from v1.0.1 to v1.0.2 
> or even v1.8.5 because it links to libvorbis.so.0 ...
The MPlayer and MySQL examples are different. For MPlayer it's the link
to .so.<complete versionnumber> that means you need to recompile.

But for MySQL v4, even if you linked just to the plain .so, you had to
recompile, because the API and structures changed. If you just left it
there and tried to run things, they would segfault.

-- 
Robin Hugh Johnson
E-Mail     : robbat2@orbis-terrarum.net
Home Page  : http://www.orbis-terrarum.net/?l=people.robbat2
ICQ#       : 30269588 or 41961639
GnuPG FP   : 11AC BA4F 4778 E3F6 E4ED  F38E B27B 944E 3488 4E85

[-- Attachment #2: Type: application/pgp-signature, Size: 232 bytes --]

Re: [gentoo-dev] Destroying dependancies...

[Per Wigren]

On Friday 28 March 2003 23:55, Robin H. Johnson wrote:
> On Fri, Mar 28, 2003 at 05:54:54PM +0100, Per Wigren wrote:
> > But how do we check that with the current information?
> > To use the postfix-case above as an example..
> > Postfix works fine with both MySQL v3.x and MySQL v4.x.
> > If it is compiled against v3 it has to be recompiled if MySQL is upgraded
> > from v3 to v4, but not if it is upgraded from v4.0.11 to v4.0.12..
> >
> > To use MPlayer as another example..
> > MPlayer has to be recompiled EVERY TIME that DirectFB is upgraded, even
> > minor version, because it links to .so.<complete versionnumber> ..  But
> > it doesn't need to be recompiled if for example Vorbis is upgraded from
> > v1.0.1 to v1.0.2 or even v1.8.5 because it links to libvorbis.so.0 ...
>
> The MPlayer and MySQL examples are different. For MPlayer it's the link
> to .so.<complete versionnumber> that means you need to recompile.
>
> But for MySQL v4, even if you linked just to the plain .so, you had to
> recompile, because the API and structures changed. If you just left it
> there and tried to run things, they would segfault.

Yes, "of course".. It links to the major versionnumber for a reason...
The MySQL-libraries are ABI-compatible withing the major version so it doesn't 
need to link to .so.x.y.z or .so.x.y, just .so.x.
DirectFB is still under development (beta/alpha) and is not ABI-stable, so 
programs are linking to .so.x.y.z..
IMHO the only good way to track what packages should be recompiled is to check 
what it links to with ldd.  If a program links to the .so instead of .so.x.y 
of a library that is not considered ABI-stable, it should be seen as a bug in 
the program.

// Per Wigren

--
gentoo-dev@gentoo.org mailing list