[gentoo-dev] Clarify log message?

[gentoo-dev] Clarify log message?

[Dirkjan Ochtman]

I've seen this pop up a lot recently:

 * One or more symlinks to directories have been preserved in order to
 * ensure that files installed via these symlinks remain accessible. This
 * indicates that the mentioned symlink(s) may be obsolete remnants of an
 * old install, and it may be appropriate to replace a given symlink with
 * the directory that it points to.
 *
 *      /var/run
 *

This might just be me being dense, but this doesn't seem very
actionable to me. Who should replace the given symlink with the
directory that it points to, the user or the package maintainer? And
where should it be replaced? In ebuild code?

Cheers,

Dirkjan

Re: [gentoo-dev] Clarify log message?

[Zac Medico]

On 01/14/2013 01:33 AM, Dirkjan Ochtman wrote:
> I've seen this pop up a lot recently:
> 
>  * One or more symlinks to directories have been preserved in order to
>  * ensure that files installed via these symlinks remain accessible. This
>  * indicates that the mentioned symlink(s) may be obsolete remnants of an
>  * old install, and it may be appropriate to replace a given symlink with
>  * the directory that it points to.
>  *
>  *      /var/run
>  *
> 
> This might just be me being dense, but this doesn't seem very
> actionable to me. Who should replace the given symlink with the
> directory that it points to, the user or the package maintainer?

It depends on who created the symlink in the first place, and whether or
not the symlink is still desirable. Unfortunately, there are a number of
possible scenarios.

You probably want to keep that /var/run symlink, at least until all of
your installed packages have been fixed to use /run directly. You can
suppress the warning by putting a setting like this in make.conf:

  UNINSTALL_IGNORE="${UNINSTALL_IGNORE} /var/run"

That will prevent portage from trying to uninstall that symlink.

> And where should it be replaced? In ebuild code?

It's possible for ebuild code to do it, if appropriate for the given
scenario. For example, the skype ebuild removes an obsolete
"${EROOT}"/usr/share/${PN} symlink in pkg_preinst.
-- 
Thanks,
Zac

Re: [gentoo-dev] Clarify log message?

[Ian Stakenvicius]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

On 14/01/13 04:44 AM, Zac Medico wrote:
> On 01/14/2013 01:33 AM, Dirkjan Ochtman wrote:
>> I've seen this pop up a lot recently:
>> 
>> * One or more symlinks to directories have been preserved in
>> order to * ensure that files installed via these symlinks remain
>> accessible. This * indicates that the mentioned symlink(s) may be
>> obsolete remnants of an * old install, and it may be appropriate
>> to replace a given symlink with * the directory that it points
>> to. * *      /var/run *
>> 
>> This might just be me being dense, but this doesn't seem very 
>> actionable to me. Who should replace the given symlink with the 
>> directory that it points to, the user or the package maintainer?
> 
> It depends on who created the symlink in the first place, and
> whether or not the symlink is still desirable. Unfortunately, there
> are a number of possible scenarios.
> 
> You probably want to keep that /var/run symlink, at least until all
> of your installed packages have been fixed to use /run directly.
> You can suppress the warning by putting a setting like this in
> make.conf:
> 
> UNINSTALL_IGNORE="${UNINSTALL_IGNORE} /var/run"
> 
> That will prevent portage from trying to uninstall that symlink.
> 
>> And where should it be replaced? In ebuild code?
> 
> It's possible for ebuild code to do it, if appropriate for the
> given scenario. For example, the skype ebuild removes an obsolete 
> "${EROOT}"/usr/share/${PN} symlink in pkg_preinst.


This particular symlink was put there by openrc though, wasn't it?  So
I'd expect that on the whole this should be left for openrc to deal
with or otherwise left to the user...?

[tangent]
it's a bit late for /var/run , but i wonder if, for the next path
migration, there might be some way to account for which packages use
the old path, say, make a list somewhere, and have the ebuilds remove
their atom from that list as they migrate to the new path..  Then once
the list is empty the compatibility symlink could be cleaned up
automatically or the user notified..  Probably this would need to be
handled via an eclass and specific function calls in all relevant
ebuilds, as i doubt there would be a way to do this generically in
portage itself.
[/tangent]
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.19 (GNU/Linux)

iF4EAREIAAYFAlD0GjkACgkQ2ugaI38ACPD5rwD/f4MkzFrlSShRWt/9WB12kybX
uY0uzf5SFmfihzNthOABALr6bT3bB7PAr4eh4tmQH+jH9Z635OD5/BcdJBYw7tLl
=S5ML
-----END PGP SIGNATURE-----

Re: [gentoo-dev] Clarify log message?

[Zac Medico]

On 01/14/2013 06:46 AM, Ian Stakenvicius wrote:
> This particular symlink was put there by openrc though, wasn't it?

Yeah, openrc uses the migrate_to_run function from /etc/init.d/bootmisc.

> So
> I'd expect that on the whole this should be left for openrc to deal
> with or otherwise left to the user...?

As things are now, the symlink is an orphan, and emerge will
automatically remove the symlink when the last package that installed
something under /var/run/ is uninstalled.

> [tangent]
> it's a bit late for /var/run , but i wonder if, for the next path
> migration, there might be some way to account for which packages use
> the old path, say, make a list somewhere, and have the ebuilds remove
> their atom from that list as they migrate to the new path..  Then once
> the list is empty the compatibility symlink could be cleaned up
> automatically or the user notified..  Probably this would need to be
> handled via an eclass and specific function calls in all relevant
> ebuilds, as i doubt there would be a way to do this generically in
> portage itself.
> [/tangent]
> 

That sounds a lot like the existing behavior (automatic symlink removal
by emerge).
-- 
Thanks,
Zac

Re: [gentoo-dev] Clarify log message?

[Ian Stakenvicius]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

On 14/01/13 09:57 AM, Zac Medico wrote:
> On 01/14/2013 06:46 AM, Ian Stakenvicius wrote:
>> This particular symlink was put there by openrc though, wasn't
>> it?
> 
> Yeah, openrc uses the migrate_to_run function from
> /etc/init.d/bootmisc.
> 
>> So I'd expect that on the whole this should be left for openrc to
>> deal with or otherwise left to the user...?
> 
> As things are now, the symlink is an orphan, and emerge will 
> automatically remove the symlink when the last package that
> installed something under /var/run/ is uninstalled.

...that doesn't sound good ; /var/run traditionally isn't a path used
solely via src_install() but rather a path used by packages at
runtime, no?  If that's the case, that symlink probably should've been
set up to remain until user intervention removes it..


> 
>> [tangent] it's a bit late for /var/run , but i wonder if, for the
>> next path migration, there might be some way to account for which
>> packages use the old path, say, make a list somewhere, and have
>> the ebuilds remove their atom from that list as they migrate to
>> the new path..  Then once the list is empty the compatibility
>> symlink could be cleaned up automatically or the user notified..
>> Probably this would need to be handled via an eclass and specific
>> function calls in all relevant ebuilds, as i doubt there would be
>> a way to do this generically in portage itself. [/tangent]
>> 
> 
> That sounds a lot like the existing behavior (automatic symlink
> removal by emerge).

OK i'm a little confused.  Putting my earlier note aside, if the
symlink will be auto-cleaned after no packages use it, what's the
point/need for the original message from portage then??  Is it just QA
for the ebuild maintainer?
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.19 (GNU/Linux)

iF4EAREIAAYFAlD0H68ACgkQ2ugaI38ACPC+vAEAhGHZMfN2/dY+SXHFBStbckkH
9geiYEJAOEa6kcbOS28A/2hUq67WlVRiRP0vqW1WdeTod7aHhvc//q2TH994BWaI
=pWiP
-----END PGP SIGNATURE-----

Re: [gentoo-dev] Clarify log message?

[Zac Medico]

On 01/14/2013 07:09 AM, Ian Stakenvicius wrote:
> On 14/01/13 09:57 AM, Zac Medico wrote:
>> On 01/14/2013 06:46 AM, Ian Stakenvicius wrote:
>>> This particular symlink was put there by openrc though, wasn't
>>> it?
> 
>> Yeah, openrc uses the migrate_to_run function from
>> /etc/init.d/bootmisc.
> 
>>> So I'd expect that on the whole this should be left for openrc to
>>> deal with or otherwise left to the user...?
> 
>> As things are now, the symlink is an orphan, and emerge will 
>> automatically remove the symlink when the last package that
>> installed something under /var/run/ is uninstalled.
> 
> ...that doesn't sound good ; /var/run traditionally isn't a path used
> solely via src_install() but rather a path used by packages at
> runtime, no?  If that's the case, that symlink probably should've been
> set up to remain until user intervention removes it..

It would be possible would be possible to protect the symlink by having
openrc or baselayout install the symlink, so that it's not an orphan.

Alternatively, users could manually protect it, by adding this setting
to make.conf:

 UNINSTALL_IGNORE="${UNINSTALL_IGNORE} /var/run"

>>> [tangent] it's a bit late for /var/run , but i wonder if, for the
>>> next path migration, there might be some way to account for which
>>> packages use the old path, say, make a list somewhere, and have
>>> the ebuilds remove their atom from that list as they migrate to
>>> the new path..  Then once the list is empty the compatibility
>>> symlink could be cleaned up automatically or the user notified..
>>> Probably this would need to be handled via an eclass and specific
>>> function calls in all relevant ebuilds, as i doubt there would be
>>> a way to do this generically in portage itself. [/tangent]
>>>
> 
>> That sounds a lot like the existing behavior (automatic symlink
>> removal by emerge).
> 
> OK i'm a little confused.  Putting my earlier note aside, if the
> symlink will be auto-cleaned after no packages use it, what's the
> point/need for the original message from portage then??  Is it just QA
> for the ebuild maintainer?

Unfortunately, there are a number of different possible scenarios. It
may serve as QA for the ebuild maintainer. It may be triggered by a
symlink that the sysadmin has manually created. In any case, there's a
performance penalty, since portage has to search for other packages that
installed files underneath the symlink. The performance penalty can be
avoided for a given symlink by adding it to UNINSTALL_IGNORE (which
makes the message useful, regardless of where the symlink originated from).
-- 
Thanks,
Zac

Re: [gentoo-dev] Clarify log message?

[Zac Medico]

On 01/14/2013 07:44 AM, Zac Medico wrote:
> On 01/14/2013 07:09 AM, Ian Stakenvicius wrote:
>> OK i'm a little confused.  Putting my earlier note aside, if the
>> symlink will be auto-cleaned after no packages use it, what's the
>> point/need for the original message from portage then??  Is it just QA
>> for the ebuild maintainer?
> 
> Unfortunately, there are a number of different possible scenarios. It
> may serve as QA for the ebuild maintainer. It may be triggered by a
> symlink that the sysadmin has manually created. In any case, there's a
> performance penalty, since portage has to search for other packages that
> installed files underneath the symlink. The performance penalty can be
> avoided for a given symlink by adding it to UNINSTALL_IGNORE (which
> makes the message useful, regardless of where the symlink originated from).

You can measure the performance penalty for the /var/run symlink by
running this command:

  time portageq owners / /var/run

-- 
Thanks,
Zac

Re: [gentoo-dev] Clarify log message?

[Ian Stakenvicius]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

On 14/01/13 10:53 AM, Zac Medico wrote:
> On 01/14/2013 07:44 AM, Zac Medico wrote:
>> On 01/14/2013 07:09 AM, Ian Stakenvicius wrote:
>>> OK i'm a little confused.  Putting my earlier note aside, if
>>> the symlink will be auto-cleaned after no packages use it,
>>> what's the point/need for the original message from portage
>>> then??  Is it just QA for the ebuild maintainer?
>> 
>> Unfortunately, there are a number of different possible
>> scenarios. It may serve as QA for the ebuild maintainer. It may
>> be triggered by a symlink that the sysadmin has manually created.
>> In any case, there's a performance penalty, since portage has to
>> search for other packages that installed files underneath the
>> symlink. The performance penalty can be avoided for a given
>> symlink by adding it to UNINSTALL_IGNORE (which makes the message
>> useful, regardless of where the symlink originated from).
> 
> You can measure the performance penalty for the /var/run symlink
> by running this command:
> 
> time portageq owners / /var/run
> 

Based on the performance penalty, would it make sense then for
system-managed symlinks like /var/run that it would be automatically
added to
UNINSTALL_IGNORE and its removal managed separately by whatever put it
there in the first place??

(and additionally, that the warning wouldn't be presented to end-users
because of it being a system-managed migration symlink instead of a
end-user-managed one)?

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.19 (GNU/Linux)

iF4EAREIAAYFAlD0McYACgkQ2ugaI38ACPADjgD8D1nRkHqYNQLwtfb43X5QwAFI
V9EM0yqdOWX9zI98+tABAJS5KTiZkejCVwK9Ord0VRtkyGVkGgZ+HDBvuOJHsIUA
=3+S4
-----END PGP SIGNATURE-----

Re: [gentoo-dev] Clarify log message?

[Zac Medico]

On 01/14/2013 08:26 AM, Ian Stakenvicius wrote:
> On 14/01/13 10:53 AM, Zac Medico wrote:
>> On 01/14/2013 07:44 AM, Zac Medico wrote:
>>> On 01/14/2013 07:09 AM, Ian Stakenvicius wrote:
>>>> OK i'm a little confused.  Putting my earlier note aside, if
>>>> the symlink will be auto-cleaned after no packages use it,
>>>> what's the point/need for the original message from portage
>>>> then??  Is it just QA for the ebuild maintainer?
>>>
>>> Unfortunately, there are a number of different possible
>>> scenarios. It may serve as QA for the ebuild maintainer. It may
>>> be triggered by a symlink that the sysadmin has manually created.
>>> In any case, there's a performance penalty, since portage has to
>>> search for other packages that installed files underneath the
>>> symlink. The performance penalty can be avoided for a given
>>> symlink by adding it to UNINSTALL_IGNORE (which makes the message
>>> useful, regardless of where the symlink originated from).
> 
>> You can measure the performance penalty for the /var/run symlink
>> by running this command:
> 
>> time portageq owners / /var/run
> 
> 
> Based on the performance penalty, would it make sense then for
> system-managed symlinks like /var/run that it would be automatically
> added to
> UNINSTALL_IGNORE and its removal managed separately by whatever put it
> there in the first place??

There's currently no way for ebuilds to add things to UNINSTALL_IGNORE
though (aside from having them edit make.conf). And how is openrc going
to know when to remove it?

> (and additionally, that the warning wouldn't be presented to end-users
> because of it being a system-managed migration symlink instead of a
> end-user-managed one)?

Do you want to introduce a new variable for this purpose, and allow
ebuilds to modify it via /etc/env.d or something?
-- 
Thanks,
Zac

Re: [gentoo-dev] Clarify log message?

[Ian Stakenvicius]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

On 14/01/13 11:36 AM, Zac Medico wrote:
> On 01/14/2013 08:26 AM, Ian Stakenvicius wrote:
>> On 14/01/13 10:53 AM, Zac Medico wrote:
>>> On 01/14/2013 07:44 AM, Zac Medico wrote:
>>>> On 01/14/2013 07:09 AM, Ian Stakenvicius wrote:
>>>>> OK i'm a little confused.  Putting my earlier note aside,
>>>>> if the symlink will be auto-cleaned after no packages use
>>>>> it, what's the point/need for the original message from
>>>>> portage then??  Is it just QA for the ebuild maintainer?
>>>> 
>>>> Unfortunately, there are a number of different possible 
>>>> scenarios. It may serve as QA for the ebuild maintainer. It
>>>> may be triggered by a symlink that the sysadmin has manually
>>>> created. In any case, there's a performance penalty, since
>>>> portage has to search for other packages that installed files
>>>> underneath the symlink. The performance penalty can be
>>>> avoided for a given symlink by adding it to UNINSTALL_IGNORE
>>>> (which makes the message useful, regardless of where the
>>>> symlink originated from).
>> 
>>> You can measure the performance penalty for the /var/run
>>> symlink by running this command:
>> 
>>> time portageq owners / /var/run
>> 
>> 
>> Based on the performance penalty, would it make sense then for 
>> system-managed symlinks like /var/run that it would be
>> automatically added to UNINSTALL_IGNORE and its removal managed
>> separately by whatever put it there in the first place??
> 
> There's currently no way for ebuilds to add things to
> UNINSTALL_IGNORE though (aside from having them edit make.conf).
> And how is openrc going to know when to remove it?

That would be where some sort of external tracking would need to
occur, which could be triggered via pkg_postinst and/or
pkg_{pre,post}rm in the eclass


> 
>> (and additionally, that the warning wouldn't be presented to
>> end-users because of it being a system-managed migration symlink
>> instead of a end-user-managed one)?
> 
> Do you want to introduce a new variable for this purpose, and
> allow ebuilds to modify it via /etc/env.d or something?

Possibly...

For the proof-of-concept phase I'm thinking of doing it via an eclass,
a file that'd be managed by that eclass, an external tool if required
to manage said file and/or process removal of the symlink (for cases
when the symlink isn't put in place by an ebuild during pkg_postinst,
ie if an init script should clean it up), and either source  that file
or a separate file in make.conf.

At least then we can see if there really is a benefit to changing how
it is now.  I can probably hack an install so that the /var/run
migration can be redone, for testing..

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.19 (GNU/Linux)

iF4EAREIAAYFAlD0OcAACgkQ2ugaI38ACPDkEgD/XYba0NejxqOPv41EsNa8U7tr
rm4vanPJubJCBX+rsrwA+wUKW3w9nOEkXw87AYJmq0/QFx0vFVfKJLUsrcF168gk
=imL1
-----END PGP SIGNATURE-----

Re: [gentoo-dev] Clarify log message?

[Zac Medico]

On 01/14/2013 09:00 AM, Ian Stakenvicius wrote:
> At least then we can see if there really is a benefit to changing how
> it is now.  I can probably hack an install so that the /var/run
> migration can be redone, for testing..

It might be a lot simpler to just go and patch all the ebuilds that
installed stuff in /var/run with a one-liner like this at the end of
src_install:

  rm -rf "${ED}"/var/run

Once all the ebuilds are fixed, emerge will automatically remove the
/var/run symlink, and it's history.
-- 
Thanks,
Zac

Re: [gentoo-dev] Clarify log message?

[Diego Elio Pettenò]

On 14/01/2013 18:14, Zac Medico wrote:
> It might be a lot simpler to just go and patch all the ebuilds that
> installed stuff in /var/run with a one-liner like this at the end of
> src_install:

It's a good idea to do so to avoid the check coming from portage to
happena nd waste time, but ...

> Once all the ebuilds are fixed, emerge will automatically remove the
> /var/run symlink, and it's history.

That should not happen. Ever. Because if it did we'd be in deep shit.
The fact that /var/run is a symlink is what saves us from having to
convert everything to actually use /run, and instead we still refer to
/var/run in most places _but_ we use checkpath for it.

If /var/run symlink was killed, we'd end up with a ton of services
creating a (now permanent) /var/run directory structure, with even more
trouble.

-- 
Diego Elio Pettenò — Flameeyes
flameeyes@flameeyes.eu — http://blog.flameeyes.eu/

Re: [gentoo-dev] Clarify log message?

[Zac Medico]

On 01/14/2013 09:23 AM, Diego Elio Pettenò wrote:
> On 14/01/2013 18:14, Zac Medico wrote:
>> It might be a lot simpler to just go and patch all the ebuilds that
>> installed stuff in /var/run with a one-liner like this at the end of
>> src_install:
> 
> It's a good idea to do so to avoid the check coming from portage to
> happena nd waste time, but ...
> 
>> Once all the ebuilds are fixed, emerge will automatically remove the
>> /var/run symlink, and it's history.
> 
> That should not happen. Ever. Because if it did we'd be in deep shit.
> The fact that /var/run is a symlink is what saves us from having to
> convert everything to actually use /run, and instead we still refer to
> /var/run in most places _but_ we use checkpath for it.
> 
> If /var/run symlink was killed, we'd end up with a ton of services
> creating a (now permanent) /var/run directory structure, with even more
> trouble.

Ah, right! So, those packages also need to be patched to use /run
directly, instead of relying on the symlink.
-- 
Thanks,
Zac

Re: [gentoo-dev] Clarify log message?

[Diego Elio Pettenò]

On 14/01/2013 18:32, Zac Medico wrote:
> Ah, right! So, those packages also need to be patched to use /run
> directly, instead of relying on the symlink.

Or the symlink should just stay for however long that's going to take,
which is probably a good idea, as it does not bother anyone.

I don't think that's going to be a problem for running systems though:
the symlink will never be entirely empty.

-- 
Diego Elio Pettenò — Flameeyes
flameeyes@flameeyes.eu — http://blog.flameeyes.eu/