* [gentoo-dev] making revdep-rebuild (partially) obsolete
@ 2010-12-31 20:42 Enrico Weigelt
2010-12-31 20:54 ` Mike Gilbert
` (3 more replies)
0 siblings, 4 replies; 12+ messages in thread
From: Enrico Weigelt @ 2010-12-31 20:42 UTC (permalink / raw
To: gentoo developers
Hi folks,
just a little braindump, how revdep-rebuild could be made
(partially) obsolete in future:
Today it might happen that on an library update an old .so file
gets unmerged while still used by somebody - that's what revdep-
rebuild scans for. While it should catch those cases, we still
have some downtime for certain packages (in bad cases, when it
broke somewhere deep in the dependency chain, rebuild might take
quite a lot of time).
The main problem IMHO is that portage doesn't record which libraries
some package links in, so it doesn't know which ones have to be
protected from unmerge (unless explicitly stated somewhere).
So I'd propose to add record that information. On next merge,
this information can be used for an automatic library-protect.
This would also record which libraries have been protected from
removal and for whom. Subsequent merges will update this that,
and once all importers have been unmerged, depclean can clean
up the leftover dirt.
What do you think about this idea ?
cu
--
----------------------------------------------------------------------
Enrico Weigelt, metux IT service -- http://www.metux.de/
phone: +49 36207 519931 email: weigelt@metux.de
mobile: +49 151 27565287 icq: 210169427 skype: nekrad666
----------------------------------------------------------------------
Embedded-Linux / Portierung / Opensource-QM / Verteilte Systeme
----------------------------------------------------------------------
^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: [gentoo-dev] making revdep-rebuild (partially) obsolete
2010-12-31 20:42 [gentoo-dev] making revdep-rebuild (partially) obsolete Enrico Weigelt
@ 2010-12-31 20:54 ` Mike Gilbert
2010-12-31 20:56 ` Michał Górny
` (2 subsequent siblings)
3 siblings, 0 replies; 12+ messages in thread
From: Mike Gilbert @ 2010-12-31 20:54 UTC (permalink / raw
To: gentoo-dev
On 12/31/2010 03:42 PM, Enrico Weigelt wrote:
> What do you think about this idea ?
I think you should check out the preserve-libs feature in portage-2.2.
^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: [gentoo-dev] making revdep-rebuild (partially) obsolete
2010-12-31 20:42 [gentoo-dev] making revdep-rebuild (partially) obsolete Enrico Weigelt
2010-12-31 20:54 ` Mike Gilbert
@ 2010-12-31 20:56 ` Michał Górny
2011-01-02 4:09 ` Enrico Weigelt
2010-12-31 21:06 ` dev-random
2010-12-31 21:41 ` Zac Medico
3 siblings, 1 reply; 12+ messages in thread
From: Michał Górny @ 2010-12-31 20:56 UTC (permalink / raw
To: gentoo-dev; +Cc: weigelt
[-- Attachment #1: Type: text/plain, Size: 785 bytes --]
On Fri, 31 Dec 2010 21:42:32 +0100
Enrico Weigelt <weigelt@metux.de> wrote:
> The main problem IMHO is that portage doesn't record which libraries
> some package links in, so it doesn't know which ones have to be
> protected from unmerge (unless explicitly stated somewhere).
> So I'd propose to add record that information. On next merge,
> this information can be used for an automatic library-protect.
> This would also record which libraries have been protected from
> removal and for whom. Subsequent merges will update this that,
> and once all importers have been unmerged, depclean can clean
> up the leftover dirt.
>
> What do you think about this idea ?
You mean what do we think about portage-2.2 and preserved-libs?
--
Best regards,
Michał Górny
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 198 bytes --]
^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: [gentoo-dev] making revdep-rebuild (partially) obsolete
2010-12-31 20:42 [gentoo-dev] making revdep-rebuild (partially) obsolete Enrico Weigelt
2010-12-31 20:54 ` Mike Gilbert
2010-12-31 20:56 ` Michał Górny
@ 2010-12-31 21:06 ` dev-random
2010-12-31 21:41 ` Zac Medico
3 siblings, 0 replies; 12+ messages in thread
From: dev-random @ 2010-12-31 21:06 UTC (permalink / raw
To: gentoo-dev
> ...
> Subsequent merges will update this that,
> ...
Subsequent merges may happen after a long while. Old, possibly
vulunerable library will still be used, and most likely unseen by
admin.
^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: [gentoo-dev] making revdep-rebuild (partially) obsolete
2010-12-31 20:42 [gentoo-dev] making revdep-rebuild (partially) obsolete Enrico Weigelt
` (2 preceding siblings ...)
2010-12-31 21:06 ` dev-random
@ 2010-12-31 21:41 ` Zac Medico
3 siblings, 0 replies; 12+ messages in thread
From: Zac Medico @ 2010-12-31 21:41 UTC (permalink / raw
To: gentoo-dev
On 12/31/2010 12:42 PM, Enrico Weigelt wrote:
> The main problem IMHO is that portage doesn't record which libraries
> some package links in, so it doesn't know which ones have to be
> protected from unmerge (unless explicitly stated somewhere).
> So I'd propose to add record that information. On next merge,
> this information can be used for an automatic library-protect.
> This would also record which libraries have been protected from
> removal and for whom. Subsequent merges will update this that,
> and once all importers have been unmerged, depclean can clean
> up the leftover dirt.
As other's have mentioned, this is already implemented in portage-2.2
with FEATURES=preserve-libs and @preserved-rebuild.
However, before this feature is unmasked, I think that it's critical to
implement bug 192319 [1] (abi-slot-deps) in order to ensure that reverse
dependencies of preserved libraries are rebuilt at the earliest possible
opportunity since this will minimize the possibility of symbol
collisions [2].
[1] http://bugs.gentoo.org/show_bug.cgi?id=192319
[2]
http://blog.flameeyes.eu/2008/06/30/a-few-risks-i-see-related-to-the-new-portage-2-2-preserve-libs-behaviour
--
Thanks,
Zac
^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: [gentoo-dev] making revdep-rebuild (partially) obsolete
2010-12-31 20:56 ` Michał Górny
@ 2011-01-02 4:09 ` Enrico Weigelt
2011-01-02 5:08 ` Mike Frysinger
2011-01-02 5:17 ` [gentoo-dev] " Zac Medico
0 siblings, 2 replies; 12+ messages in thread
From: Enrico Weigelt @ 2011-01-02 4:09 UTC (permalink / raw
To: gentoo-dev
* Micha?? Górny <mgorny@gentoo.org> schrieb:
> > What do you think about this idea ?
>
> You mean what do we think about portage-2.2 and preserved-libs?
Well, I'm still using portage-2.1, so I wans't aware of whats going
on there. For now it seems the preservation is still done explicitly
(preserve_old_lib calls in certain ebuilds ?). My proposal is to
record the necessary information (eg. which so some executable/so
is linked against) automatically - does portage-2.2 do that ?
BTW: several blog/maillist postings talked about the problem that
even on recompile, older library versions could be linked in even
on recompile. Somebody suggested to move away preserved libs to
another directly (which is then added to ld.so.conf). What do you
think about that ?
Another approach could be building everything in an separate,
minimal sysroot or chroot. (I admit, I have no idea how complex
it would be to implement that in portage - my Briegel buildsystem
does always does this)
cu
--
----------------------------------------------------------------------
Enrico Weigelt, metux IT service -- http://www.metux.de/
phone: +49 36207 519931 email: weigelt@metux.de
mobile: +49 151 27565287 icq: 210169427 skype: nekrad666
----------------------------------------------------------------------
Embedded-Linux / Portierung / Opensource-QM / Verteilte Systeme
----------------------------------------------------------------------
^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: [gentoo-dev] making revdep-rebuild (partially) obsolete
2011-01-02 4:09 ` Enrico Weigelt
@ 2011-01-02 5:08 ` Mike Frysinger
2011-01-02 7:31 ` [gentoo-dev] " Ryan Hill
2011-01-02 5:17 ` [gentoo-dev] " Zac Medico
1 sibling, 1 reply; 12+ messages in thread
From: Mike Frysinger @ 2011-01-02 5:08 UTC (permalink / raw
To: gentoo-dev; +Cc: Enrico Weigelt
[-- Attachment #1: Type: Text/Plain, Size: 286 bytes --]
On Saturday, January 01, 2011 23:09:11 Enrico Weigelt wrote:
> BTW: several blog/maillist postings talked about the problem that
> even on recompile, older library versions could be linked in even
> on recompile.
you'll need to provide an actual example. i have yet to see one.
-mike
[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 836 bytes --]
^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: [gentoo-dev] making revdep-rebuild (partially) obsolete
2011-01-02 4:09 ` Enrico Weigelt
2011-01-02 5:08 ` Mike Frysinger
@ 2011-01-02 5:17 ` Zac Medico
2011-01-02 5:46 ` Mike Frysinger
1 sibling, 1 reply; 12+ messages in thread
From: Zac Medico @ 2011-01-02 5:17 UTC (permalink / raw
To: gentoo-dev
On 01/01/2011 08:09 PM, Enrico Weigelt wrote:
> Well, I'm still using portage-2.1, so I wans't aware of whats going
> on there. For now it seems the preservation is still done explicitly
> (preserve_old_lib calls in certain ebuilds ?). My proposal is to
> record the necessary information (eg. which so some executable/so
> is linked against) automatically - does portage-2.2 do that ?
Yes, portage-2.2 uses /var/db/pkg/*/*/NEEDED.ELF.3 for that.
--
Thanks,
Zac
^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: [gentoo-dev] making revdep-rebuild (partially) obsolete
2011-01-02 5:17 ` [gentoo-dev] " Zac Medico
@ 2011-01-02 5:46 ` Mike Frysinger
2011-01-02 8:45 ` Enrico Weigelt
0 siblings, 1 reply; 12+ messages in thread
From: Mike Frysinger @ 2011-01-02 5:46 UTC (permalink / raw
To: gentoo-dev
[-- Attachment #1: Type: Text/Plain, Size: 651 bytes --]
On Sunday, January 02, 2011 00:17:03 Zac Medico wrote:
> On 01/01/2011 08:09 PM, Enrico Weigelt wrote:
> > Well, I'm still using portage-2.1, so I wans't aware of whats going
> > on there. For now it seems the preservation is still done explicitly
> > (preserve_old_lib calls in certain ebuilds ?). My proposal is to
> > record the necessary information (eg. which so some executable/so
> > is linked against) automatically - does portage-2.2 do that ?
>
> Yes, portage-2.2 uses /var/db/pkg/*/*/NEEDED.ELF.3 for that.
although portage has long been generating the NEEDED files in vdb. even
stable portage generates these files.
-mike
[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 836 bytes --]
^ permalink raw reply [flat|nested] 12+ messages in thread
* [gentoo-dev] Re: making revdep-rebuild (partially) obsolete
2011-01-02 5:08 ` Mike Frysinger
@ 2011-01-02 7:31 ` Ryan Hill
2011-01-02 8:00 ` Mike Frysinger
0 siblings, 1 reply; 12+ messages in thread
From: Ryan Hill @ 2011-01-02 7:31 UTC (permalink / raw
To: gentoo-dev
[-- Attachment #1: Type: text/plain, Size: 1222 bytes --]
On Sun, 2 Jan 2011 00:08:34 -0500
Mike Frysinger <vapier@gentoo.org> wrote:
> On Saturday, January 01, 2011 23:09:11 Enrico Weigelt wrote:
> > BTW: several blog/maillist postings talked about the problem that
> > even on recompile, older library versions could be linked in even
> > on recompile.
>
> you'll need to provide an actual example. i have yet to see one.
Not exactly the same thing, but the recent spidermonkey bump changed the lib
from libjs.so to libmozjs.so. mediatomb kept finding the old libjs on
rebuild. I had to uninstall mediatomb to dump the library and reinstall it.
There's not much we can do about stuff like that, but it's not an uncommon
occurrence. I usually run into a sticky preserved lib or two every couple
months.
I still think it's much better than the current portage 2.1 behavior which
results in a broken mediatomb. I'll take the rare possibility of a symbol
collision over a guaranteed broken package any day.
--
fonts, gcc-porting, it makes no sense how it makes no sense
toolchain, wxwidgets but i'll take it free anytime
@ gentoo.org EFFD 380E 047A 4B51 D2BD C64F 8AA8 8346 F9A4 0662
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 198 bytes --]
^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: [gentoo-dev] Re: making revdep-rebuild (partially) obsolete
2011-01-02 7:31 ` [gentoo-dev] " Ryan Hill
@ 2011-01-02 8:00 ` Mike Frysinger
0 siblings, 0 replies; 12+ messages in thread
From: Mike Frysinger @ 2011-01-02 8:00 UTC (permalink / raw
To: gentoo-dev
[-- Attachment #1: Type: Text/Plain, Size: 1404 bytes --]
On Sunday, January 02, 2011 02:31:06 Ryan Hill wrote:
> On Sun, 2 Jan 2011 00:08:34 -0500 Mike Frysinger wrote:
> > On Saturday, January 01, 2011 23:09:11 Enrico Weigelt wrote:
> > > BTW: several blog/maillist postings talked about the problem that
> > > even on recompile, older library versions could be linked in even
> > > on recompile.
> >
> > you'll need to provide an actual example. i have yet to see one.
>
> Not exactly the same thing, but the recent spidermonkey bump changed the
> lib from libjs.so to libmozjs.so. mediatomb kept finding the old libjs on
> rebuild. I had to uninstall mediatomb to dump the library and reinstall
> it.
>
> There's not much we can do about stuff like that, but it's not an uncommon
> occurrence. I usually run into a sticky preserved lib or two every couple
> months.
>
> I still think it's much better than the current portage 2.1 behavior which
> results in a broken mediatomb. I'll take the rare possibility of a symbol
> collision over a guaranteed broken package any day.
yes, this is a case where linking would pick up the old library. but that is
*only* because the SONAME is stupid and is the same thing as the link name.
imo, these packages are pretty much broken and should get their SONAME fixed.
a simple change as making the SONAME "libjs.so.0" instead of "libjs.so" would
make this a non-issue.
-mike
[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 836 bytes --]
^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: [gentoo-dev] making revdep-rebuild (partially) obsolete
2011-01-02 5:46 ` Mike Frysinger
@ 2011-01-02 8:45 ` Enrico Weigelt
0 siblings, 0 replies; 12+ messages in thread
From: Enrico Weigelt @ 2011-01-02 8:45 UTC (permalink / raw
To: gentoo-dev
* Mike Frysinger <vapier@gentoo.org> schrieb:
> although portage has long been generating the NEEDED files in vdb. even
> stable portage generates these files.
Ah, okay, I wasn't aware of that.
What's the difference between NEEEDED and NEEDED.2 ? Multiarch ?
cu
--
----------------------------------------------------------------------
Enrico Weigelt, metux IT service -- http://www.metux.de/
phone: +49 36207 519931 email: weigelt@metux.de
mobile: +49 151 27565287 icq: 210169427 skype: nekrad666
----------------------------------------------------------------------
Embedded-Linux / Portierung / Opensource-QM / Verteilte Systeme
----------------------------------------------------------------------
^ permalink raw reply [flat|nested] 12+ messages in thread
end of thread, other threads:[~2011-01-02 8:51 UTC | newest]
Thread overview: 12+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2010-12-31 20:42 [gentoo-dev] making revdep-rebuild (partially) obsolete Enrico Weigelt
2010-12-31 20:54 ` Mike Gilbert
2010-12-31 20:56 ` Michał Górny
2011-01-02 4:09 ` Enrico Weigelt
2011-01-02 5:08 ` Mike Frysinger
2011-01-02 7:31 ` [gentoo-dev] " Ryan Hill
2011-01-02 8:00 ` Mike Frysinger
2011-01-02 5:17 ` [gentoo-dev] " Zac Medico
2011-01-02 5:46 ` Mike Frysinger
2011-01-02 8:45 ` Enrico Weigelt
2010-12-31 21:06 ` dev-random
2010-12-31 21:41 ` Zac Medico
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox