[gentoo-amd64] emerge --depclean question

[gentoo-amd64] emerge --depclean question

[Mark Knecht]

Hi,
   I haven't done this in quite awhile so this evening I started
looking at dependencies a bit. I thought I might take a shot at
cleaning things up old stuff left around from updates over the last
year or so. I ran emerge -p --depclean and was told that there were
about 45 things to remove so I decided I'd try removing a few
one-at-a-time and see how it went. First I sync'ed and did an emerge
-DuN world and made sure the machine was up to date. I then removed
what seemed like a pretty random entry late on the list - imlib - and
then a revdep-rebuild -p to see what it thought. I was then told I
needed to emerge revisions of two new things - gnome-lib and
gtk-pixbuf - neither of which existed according to both emerge and
eix. I then re-emerged imlib, reran revdep-rebuild and now I see the
machine is consistent again.

   What's up with that?

   I then tried again with something called 'jasper'. Rerunning
revdep-rebuild is now telling me it needs to reemerge netpbm and this
will pick up jasper again. I did that emerge and reran revdep-rebuild
again and the machine is once again clean.

   What's up with that?

   Is this a question that certain ebuilds are not written correctly
and don't somehow call for the right dependencies, or is it something
else?

   Also, is there any better tool these days for cleaning up stale
packages left around than emerge --depclean?

Thanks,
Mark
-- 
gentoo-amd64@gentoo.org mailing list

[gentoo-amd64] Re: emerge --depclean question

[Duncan]

"Mark Knecht" <markknecht@gmail.com> posted
5bdc1c8b0612301751k5f65c464nf8b9bd99c3b75091@mail.gmail.com, excerpted
below, on  Sat, 30 Dec 2006 17:51:29 -0800:

> I haven't done this in quite awhile so this evening I started
> looking at dependencies a bit. I thought I might take a shot at
> cleaning things up old stuff left around from updates over the last
> year or so. I ran emerge -p --depclean and was told that there were
> about 45 things to remove so I decided I'd try removing a few
> one-at-a-time and see how it went. First I sync'ed and did an emerge
> -DuN world and made sure the machine was up to date. I then removed
> what seemed like a pretty random entry late on the list - imlib - and
> then a revdep-rebuild -p to see what it thought. I was then told I
> needed to emerge revisions of two new things - gnome-lib and
> gtk-pixbuf - neither of which existed according to both emerge and
> eix. I then re-emerged imlib, reran revdep-rebuild and now I see the
> machine is consistent again.

gnome-lib and gtk-pixbuf are GNOME-1.  Somewhere, you still have something
wanting GNOME-1 stuff, which has been removed from the tree as nothing
current depends on it.

You can use equery -d <package> to get a list of merged packages depending
on <package>.  Try that with gnome-lib and see what comes up.  Maybe you
don't need it any more and can unmerge it.  Or, if it's still in the tree
the dependencies must now be fixed, so remerging it should kill the
dependencies on gnome-lib.  (equery is part of gentoolkit.)

It's also possible you have "orphaned" libraries around that somehow got
left behind after an unmerge.  Portage won't know about these so
--depclean won't consider them.  However, revdep-rebuild does a scan of
the actual libraries and applications you have, including orphans and
binary-only packages (which remerging won't fix since you just remerge the
same binary libraries with the same compiled in dependencies), and thus
catches things portage wouldn't.

>    Is this a question that certain ebuilds are not written correctly
> and don't somehow call for the right dependencies, or is it something
> else?

It's probably either packages that have changed since you merged them, or
orphans, as mentioned above.  It could however also be just as you said,
dependencies that aren't properly noted in the ebuild.  This can be either
a missed dependency that needs to be there, or something the package
configure script detected on its own, that wasn't part of the ebuild. 
Gentoo devs try not to have any of those "autodetected" things, because it
causes problems exactly like what you are seeing.  Instead, they try to
specify a hard dependency if the package really needs it, make it
conditional on a USE flag if it's optional, or unconditionally tell the
config script NOT to use it if Gentoo has a different way of handling it
or something, as sometimes happens.  However, sometimes upstream changes
the dependencies or optional dependencies and the Gentoo devs don't catch
the changed config it until someone runs into problems and files a bug.

>    Also, is there any better tool these days for cleaning up stale
> packages left around than emerge --depclean?

Not really... that I know of anyway.  There are a few other tools such as
dep, from the udept package, but it's worse, not better.  (From what I can
tell, it checks the dependencies in the ebuilds themselves but not the
ones brought in by the eclasses, so it misses some.  It's quite a bit
faster, but it's also not anywhere near as dependable.)

Basically, it's a process of using --depclean -p, and getting a list, then
using common sense and trial and error to figure out what's safe to remove
and what's not.  It's not an easy or simple process, which is why there
are such big warnings on it.  However, it's worthwhile to do once in
awhile, both because it helps keep the troubles away, and because stuff
that it says needs removed is stuff portage won't be regularly updating,
and that can quickly become a security issue if there's a vulnerability in
one of those packages.

One thing that makes things MUCH easier, however, and you may already be
doing this, is using FEATURES=buildpkg.  If you are, testing unmerges is
vastly easier, since if something needs the package, you can simply emerge
-k the binary package that was created during the initial merge -- that's
what the buildpkg feature DOES.

If you haven't been using the buildpkg feature, it's also possible to
quickpkg the package already on your system before you remove it.  Again,
if it turns out you actually need it, you don't have to spend all that
time recompiling it, all you have to do is emerge -k it, to remerge the
binary package you created using quickpkg.

-- 
Duncan - List replies preferred.   No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master."  Richard Stallman

-- 
gentoo-amd64@gentoo.org mailing list

Re: [gentoo-amd64] Re: emerge --depclean question

[Mark Knecht]

Duncan,
   Makes perfect sense. Thanks.

   I'll spend some time today grinding through all this stuff and get
it cleaned up.

Thanks,
Mark

On 12/31/06, Duncan <1i5t5.duncan@cox.net> wrote:
> "Mark Knecht" <markknecht@gmail.com> posted
> 5bdc1c8b0612301751k5f65c464nf8b9bd99c3b75091@mail.gmail.com, excerpted
> below, on  Sat, 30 Dec 2006 17:51:29 -0800:
>
> > I haven't done this in quite awhile so this evening I started
> > looking at dependencies a bit. I thought I might take a shot at
> > cleaning things up old stuff left around from updates over the last
> > year or so. I ran emerge -p --depclean and was told that there were
> > about 45 things to remove so I decided I'd try removing a few
> > one-at-a-time and see how it went. First I sync'ed and did an emerge
> > -DuN world and made sure the machine was up to date. I then removed
> > what seemed like a pretty random entry late on the list - imlib - and
> > then a revdep-rebuild -p to see what it thought. I was then told I
> > needed to emerge revisions of two new things - gnome-lib and
> > gtk-pixbuf - neither of which existed according to both emerge and
> > eix. I then re-emerged imlib, reran revdep-rebuild and now I see the
> > machine is consistent again.
>
> gnome-lib and gtk-pixbuf are GNOME-1.  Somewhere, you still have something
> wanting GNOME-1 stuff, which has been removed from the tree as nothing
> current depends on it.
>
> You can use equery -d <package> to get a list of merged packages depending
> on <package>.  Try that with gnome-lib and see what comes up.  Maybe you
> don't need it any more and can unmerge it.  Or, if it's still in the tree
> the dependencies must now be fixed, so remerging it should kill the
> dependencies on gnome-lib.  (equery is part of gentoolkit.)
>
> It's also possible you have "orphaned" libraries around that somehow got
> left behind after an unmerge.  Portage won't know about these so
> --depclean won't consider them.  However, revdep-rebuild does a scan of
> the actual libraries and applications you have, including orphans and
> binary-only packages (which remerging won't fix since you just remerge the
> same binary libraries with the same compiled in dependencies), and thus
> catches things portage wouldn't.
>
> >    Is this a question that certain ebuilds are not written correctly
> > and don't somehow call for the right dependencies, or is it something
> > else?
>
> It's probably either packages that have changed since you merged them, or
> orphans, as mentioned above.  It could however also be just as you said,
> dependencies that aren't properly noted in the ebuild.  This can be either
> a missed dependency that needs to be there, or something the package
> configure script detected on its own, that wasn't part of the ebuild.
> Gentoo devs try not to have any of those "autodetected" things, because it
> causes problems exactly like what you are seeing.  Instead, they try to
> specify a hard dependency if the package really needs it, make it
> conditional on a USE flag if it's optional, or unconditionally tell the
> config script NOT to use it if Gentoo has a different way of handling it
> or something, as sometimes happens.  However, sometimes upstream changes
> the dependencies or optional dependencies and the Gentoo devs don't catch
> the changed config it until someone runs into problems and files a bug.
>
> >    Also, is there any better tool these days for cleaning up stale
> > packages left around than emerge --depclean?
>
> Not really... that I know of anyway.  There are a few other tools such as
> dep, from the udept package, but it's worse, not better.  (From what I can
> tell, it checks the dependencies in the ebuilds themselves but not the
> ones brought in by the eclasses, so it misses some.  It's quite a bit
> faster, but it's also not anywhere near as dependable.)
>
> Basically, it's a process of using --depclean -p, and getting a list, then
> using common sense and trial and error to figure out what's safe to remove
> and what's not.  It's not an easy or simple process, which is why there
> are such big warnings on it.  However, it's worthwhile to do once in
> awhile, both because it helps keep the troubles away, and because stuff
> that it says needs removed is stuff portage won't be regularly updating,
> and that can quickly become a security issue if there's a vulnerability in
> one of those packages.
>
> One thing that makes things MUCH easier, however, and you may already be
> doing this, is using FEATURES=buildpkg.  If you are, testing unmerges is
> vastly easier, since if something needs the package, you can simply emerge
> -k the binary package that was created during the initial merge -- that's
> what the buildpkg feature DOES.
>
> If you haven't been using the buildpkg feature, it's also possible to
> quickpkg the package already on your system before you remove it.  Again,
> if it turns out you actually need it, you don't have to spend all that
> time recompiling it, all you have to do is emerge -k it, to remerge the
> binary package you created using quickpkg.
>
> --
> Duncan - List replies preferred.   No HTML msgs.
> "Every nonfree program has a lord, a master --
> and if you use the program, he is your master."  Richard Stallman
>
> --
> gentoo-amd64@gentoo.org mailing list
>
>
-- 
gentoo-amd64@gentoo.org mailing list

Re: [gentoo-amd64] Re: emerge --depclean question

[Mark Knecht]

On 12/31/06, Duncan <1i5t5.duncan@cox.net> wrote:
<SNIP
>
> Basically, it's a process of using --depclean -p, and getting a list, then
> using common sense and trial and error to figure out what's safe to remove
> and what's not.  It's not an easy or simple process, which is why there
> are such big warnings on it.  However, it's worthwhile to do once in
> awhile, both because it helps keep the troubles away, and because stuff
> that it says needs removed is stuff portage won't be regularly updating,
> and that can quickly become a security issue if there's a vulnerability in
> one of those packages.
>

Hi Duncan,
   I'm proceeding along getting a number of packages removed which is
good. However here's an example one I'm not sure how to handle or why
it's showing up. I've run --depclean -p and see what appears to be an
unneeded library called libwmf. However when I try tracing my way back
to find what depending on it I eventually get to xine-ui which I use
all the time:

lightning ~ # equery d libwmf
[ Searching for packages depending on libwmf... ]
media-gfx/imagemagick-6.3.0.5
lightning ~ # equery d imagemagick
[ Searching for packages depending on imagemagick... ]
gnome-extra/libgsf-1.14.2
media-libs/xine-lib-1.1.2-r3
media-video/transcode-1.0.2-r3
lightning ~ # equery d xine-lib
[ Searching for packages depending on xine-lib... ]
media-sound/amarok-1.4.3-r1
media-video/xine-ui-0.99.5_pre20060716
media-video/totem-2.16.4
lightning ~ #

   I'm pretty sure that if I removed libwfm I'd find I had to reinstall it.

   Am I missing something or incorrect in how I'm tracing back through things?

Thanks in advance,
Mark
-- 
gentoo-amd64@gentoo.org mailing list

Re: [gentoo-amd64] Re: emerge --depclean question

[Marek Wróbel]

Mark Knecht wrote:
> <SNIP
>   I'm proceeding along getting a number of packages removed which is
> good. However here's an example one I'm not sure how to handle or why
> it's showing up. I've run --depclean -p and see what appears to be an
> unneeded library called libwmf. However when I try tracing my way back
> to find what depending on it I eventually get to xine-ui which I use
> all the time:
> 
> lightning ~ # equery d libwmf
> [ Searching for packages depending on libwmf... ]
> media-gfx/imagemagick-6.3.0.5
> lightning ~ # equery d imagemagick
> [ Searching for packages depending on imagemagick... ]
> gnome-extra/libgsf-1.14.2
> media-libs/xine-lib-1.1.2-r3
> media-video/transcode-1.0.2-r3
> lightning ~ # equery d xine-lib
> [ Searching for packages depending on xine-lib... ]
> media-sound/amarok-1.4.3-r1
> media-video/xine-ui-0.99.5_pre20060716
> media-video/totem-2.16.4
> lightning ~ #

This issue can be related to changes in USE flags. xine-lib depends on
imagemagick only when "imagemagick" USE flag is set. And imagemagck
depends on libwmf only when "wmf" USE flag is set. So these packages
were merged probably when you had these flags set. Later USE flags may
have changed (new portage profile?) and currently they most likely are
not set.

Moreover, I have noticed that equery takes conditional dependencies into
consideration regardless of USE flags state. Thus it shows that
imagemagick is required by xine-lib even when it actually isn't (because
you don't have "imagemagick" flag set).

You have run emerge -uDN, so none of your packages (except those listed
by -p --depclean) should be linked with any of libraries to be removed.
I think that running --depclean will not break your system or when
something is broken, it will be remerged by revdep-rebuild without
pulling in removed packages.

Running revdep-rebuild after removing only some packages pending removal
isn't a good thing, because remaining ones are likely to be broken. I
recommend running revdep-rebuild after depclean is completed.

Of course, you should earlier revise your USE flags to check whether
some features needed by you weren't excluded. I cannot also guarantee
anything, so if it is a critical machine, you should undertake some
safety steps like quickpkg'ing packages before removal.

Hope it helps,
Marek Wróbel
-- 
gentoo-amd64@gentoo.org mailing list

Re: [gentoo-amd64] Re: emerge --depclean question

[Mark Knecht]

On 12/31/06, Marek Wróbel <smbmarek@poczta.onet.pl> wrote:
> Mark Knecht wrote:
> > <SNIP
> >   I'm proceeding along getting a number of packages removed which is
> > good. However here's an example one I'm not sure how to handle or why
> > it's showing up. I've run --depclean -p and see what appears to be an
> > unneeded library called libwmf. However when I try tracing my way back
> > to find what depending on it I eventually get to xine-ui which I use
> > all the time:
> >
> > lightning ~ # equery d libwmf
> > [ Searching for packages depending on libwmf... ]
> > media-gfx/imagemagick-6.3.0.5
> > lightning ~ # equery d imagemagick
> > [ Searching for packages depending on imagemagick... ]
> > gnome-extra/libgsf-1.14.2
> > media-libs/xine-lib-1.1.2-r3
> > media-video/transcode-1.0.2-r3
> > lightning ~ # equery d xine-lib
> > [ Searching for packages depending on xine-lib... ]
> > media-sound/amarok-1.4.3-r1
> > media-video/xine-ui-0.99.5_pre20060716
> > media-video/totem-2.16.4
> > lightning ~ #
>
> This issue can be related to changes in USE flags. xine-lib depends on
> imagemagick only when "imagemagick" USE flag is set. And imagemagck
> depends on libwmf only when "wmf" USE flag is set. So these packages
> were merged probably when you had these flags set. Later USE flags may
> have changed (new portage profile?) and currently they most likely are
> not set.
>
> Moreover, I have noticed that equery takes conditional dependencies into
> consideration regardless of USE flags state. Thus it shows that
> imagemagick is required by xine-lib even when it actually isn't (because
> you don't have "imagemagick" flag set).
>

This is pretty much what happened. I've finished removing all the
packages that depclean wanted me to and things look good now.
revdep-rebuild is happy. emerge -DuN world is happy. --depclean says
there is nothing to clean out.

It seems that maybe over time some of the packages higher up have had
USE flags added to them. Maybe the old package didn't make a function
prunable with a flag so the library got installed and later the flag
was added to prune it out but the new emerge only upgraded the package
and didn't remove the libraries that were no longer needed.

At least hat's the best I can figure out.

Everything is clean now. thanks for the help!

Cheers,
Mark

Happy 2007. Off to have a beer.

-- 
gentoo-amd64@gentoo.org mailing list