[gentoo-portage-dev] [PATCHv3 1/2] MEDIUM: misc-functions: Be more quiet when removing non existing INSTALL_MASK

[gentoo-portage-dev] [PATCHv3 1/2] MEDIUM: misc-functions: Be more quiet when removing non existing INSTALL_MASK

[Bertrand Jaquin]

From: Bertrand Jacquin <bertrand@jacquin.bzh>

When FEATURES="nodoc noinfo noman" is used, you will get the following
output for every packages, even virtual that contain no file:

  # FEATURES="nodoc noinfo noman" emerge -va1t virtual/cron
  ..
  >>> Installing (1 of 1) virtual/cron-0::gentoo
   * Removing /usr/share/man
   * Removing /usr/share/info
   * Removing /usr/share/doc
   * checking 0 files for package collisions
  >>> Merging virtual/cron-0 to /

The following patch makes the output as:

  # FEATURES="nodoc noinfo noman" emerge -va1t virtual/cron
  ..
  >>> Installing (1 of 1) virtual/cron-0::gentoo
   * checking 0 files for package collisions
  >>> Merging virtual/cron-0 to /

Signed-off-by: Bertrand Jacquin <bertrand@jacquin.bzh>
---
 bin/misc-functions.sh | 7 +++++--
 1 file changed, 5 insertions(+), 2 deletions(-)

diff --git a/bin/misc-functions.sh b/bin/misc-functions.sh
index e08c228..36a3bb8 100755
--- a/bin/misc-functions.sh
+++ b/bin/misc-functions.sh
@@ -267,9 +267,12 @@ install_mask() {
 	local no_inst
 	for no_inst in ${install_mask}; do
 		set +o noglob
-		__quiet_mode || einfo "Removing ${no_inst}"
+
 		# normal stuff
-		rm -Rf "${root}"/${no_inst} >&/dev/null
+		if [[ -e "${root}"/${no_inst} ]] ; then
+			__quiet_mode || einfo "Removing ${no_inst}"
+			rm -Rf "${root}"/${no_inst} >&/dev/null
+		fi
 
 		# we also need to handle globs (*.a, *.h, etc)
 		find "${root}" \( -path "${no_inst}" -or -name "${no_inst}" \) \

[gentoo-portage-dev] [PATCHv3 2/2] MEDIUM: misc-functions: Be more verbose when removing INSTALL_MASK glob

[Bertrand Jaquin]

From: Bertrand Jacquin <bertrand@jacquin.bzh>

When glob are defined in INSTALL_MASK, no output is given on what file
has been deleted.

The following patch provide more information to user about what is
actually removed.

Example:

  # INSTALL_MASK='*.h' emerge -va1t x11-proto/xproto
  ..
  >>> Installing (1 of 2) x11-proto/xproto-7.0.27::gentoo
   * Removing /usr/include/X11/DECkeysym.h
   * Removing /usr/include/X11/HPkeysym.h
   * Removing /usr/include/X11/Sunkeysym.h
   * Removing /usr/include/X11/X.h
   * Removing /usr/include/X11/XF86keysym.h
   * Removing /usr/include/X11/XWDFile.h
   * Removing /usr/include/X11/Xalloca.h
   * Removing /usr/include/X11/Xarch.h
   * Removing /usr/include/X11/Xatom.h
   * Removing /usr/include/X11/Xdefs.h
   * Removing /usr/include/X11/Xfuncproto.h
   * Removing /usr/include/X11/Xfuncs.h
   * Removing /usr/include/X11/Xmd.h
   * Removing /usr/include/X11/Xos.h
   * Removing /usr/include/X11/Xos_r.h
   * Removing /usr/include/X11/Xosdefs.h
   * Removing /usr/include/X11/Xpoll.h
   * Removing /usr/include/X11/Xproto.h
   * Removing /usr/include/X11/Xprotostr.h
   * Removing /usr/include/X11/Xthreads.h
   * Removing /usr/include/X11/Xw32defs.h
   * Removing /usr/include/X11/Xwindows.h
   * Removing /usr/include/X11/Xwinsock.h
   * Removing /usr/include/X11/ap_keysym.h
   * Removing /usr/include/X11/keysym.h
   * Removing /usr/include/X11/keysymdef.h
   * checking 1 files for package collisions
  >>> Merging x11-proto/xproto-7.0.27 to /

Signed-off-by: Bertrand Jacquin <bertrand@jacquin.bzh>
---
 bin/misc-functions.sh | 8 +++++++-
 1 file changed, 7 insertions(+), 1 deletion(-)

diff --git a/bin/misc-functions.sh b/bin/misc-functions.sh
index 36a3bb8..68c0c76 100755
--- a/bin/misc-functions.sh
+++ b/bin/misc-functions.sh
@@ -276,7 +276,13 @@ install_mask() {
 
 		# we also need to handle globs (*.a, *.h, etc)
 		find "${root}" \( -path "${no_inst}" -or -name "${no_inst}" \) \
-			-exec rm -fR {} \; >/dev/null 2>&1
+			-print0 2> /dev/null \
+		| LC_ALL=C sort -z \
+		| while read -r -d ''; do
+			__quiet_mode || einfo "Removing /${REPLY#${root}}"
+			rm -Rf "${REPLY}" >&/dev/null
+		done
+
 	done
 	# set everything back the way we found it
 	set +o noglob

Re: [gentoo-portage-dev] [PATCHv3 1/2] MEDIUM: misc-functions: Be more quiet when removing non existing INSTALL_MASK

[Zac Medico]

These are in the master branch now:

https://gitweb.gentoo.org/proj/portage.git/commit/?id=5a1b870fe54ac06f864a648c3ea5cc118f6ce911

https://gitweb.gentoo.org/proj/portage.git/commit/?id=e74e2670e3f043608fced9847e54bdbb19f35169


On 04/20/2015 12:45 PM, Bertrand Jaquin wrote:
> 
>  		# normal stuff
> -		rm -Rf "${root}"/${no_inst} >&/dev/null
> +		if [[ -e "${root}"/${no_inst} ]] ; then
> +			__quiet_mode || einfo "Removing ${no_inst}"
> +			rm -Rf "${root}"/${no_inst} >&/dev/null
> +		fi

I modified the test as follows in order to ensure that it still works
with shell globs:

if [[ -e "${root}"/${no_inst} || "${root}"/${no_inst} != $(echo
"${root}"/${no_inst}) ]] ; then

-- 
Thanks,
Zac

Re: [gentoo-portage-dev] [PATCHv3 1/2] MEDIUM: misc-functions: Be more quiet when removing non existing INSTALL_MASK

[Bertrand Jacquin]

On 21/04/2015 00:39, Zac Medico wrote:
> These are in the master branch now:
> 
> https://gitweb.gentoo.org/proj/portage.git/commit/?id=5a1b870fe54ac06f864a648c3ea5cc118f6ce911
> https://gitweb.gentoo.org/proj/portage.git/commit/?id=e74e2670e3f043608fced9847e54bdbb19f35169

Thanks for that :)

> I modified the test as follows in order to ensure that it still works
> with shell globs:
> 
> if [[ -e "${root}"/${no_inst} || "${root}"/${no_inst} != $(echo
> "${root}"/${no_inst}) ]] ; then

I do not really understand the use case here, do you have an example ?

Cheers,

-- 
Bertrand

Re: [gentoo-portage-dev] [PATCHv3 1/2] MEDIUM: misc-functions: Be more quiet when removing non existing INSTALL_MASK

[Zac Medico]

On 04/20/2015 05:31 PM, Bertrand Jacquin wrote:
> On 21/04/2015 00:39, Zac Medico wrote:
>> These are in the master branch now:
>>
>> https://gitweb.gentoo.org/proj/portage.git/commit/?id=5a1b870fe54ac06f864a648c3ea5cc118f6ce911
>>
>> https://gitweb.gentoo.org/proj/portage.git/commit/?id=e74e2670e3f043608fced9847e54bdbb19f35169
>>
> 
> Thanks for that :)
> 
>> I modified the test as follows in order to ensure that it still works
>> with shell globs:
>>
>> if [[ -e "${root}"/${no_inst} || "${root}"/${no_inst} != $(echo
>> "${root}"/${no_inst}) ]] ; then
> 
> I do not really understand the use case here, do you have an example ?
> 
> Cheers,
> 

Well, I don't use INSTALL_MASK myself, so I don't have a real-world
use-case for you. However, it's clear that the code will expand shell
globs, so I preserved that behavior for compatibility.
-- 
Thanks,
Zac

[gentoo-portage-dev] Re: [PATCHv3 1/2] MEDIUM: misc-functions: Be more quiet when removing non existing INSTALL_MASK

[Duncan]

Zac Medico posted on Mon, 20 Apr 2015 17:37:15 -0700 as excerpted:

> On 04/20/2015 05:31 PM, Bertrand Jacquin wrote:
>> On 21/04/2015 00:39, Zac Medico wrote:
>>> These are in the master branch now:
>>>
>>> https://gitweb.gentoo.org/proj/portage.git/commit/?
id=5a1b870fe54ac06f864a648c3ea5cc118f6ce911
>>>
>>> https://gitweb.gentoo.org/proj/portage.git/commit/?
id=e74e2670e3f043608fced9847e54bdbb19f35169
>>>
>> 
>> Thanks for that :)
>> 
>>> I modified the test as follows in order to ensure that it still works
>>> with shell globs:
>>>
>>> if [[ -e "${root}"/${no_inst} || "${root}"/${no_inst} != $(echo
>>> "${root}"/${no_inst}) ]] ; then
>> 
>> I do not really understand the use case here, do you have an example ?
>> 
>> Cheers,
>> 
> 
> Well, I don't use INSTALL_MASK myself, so I don't have a real-world
> use-case for you. However, it's clear that the code will expand shell
> globs, so I preserved that behavior for compatibility.

I do, with shell globs, tho I didn't bother checking the above to see if 
they'd have been affected.

The two install-masks with globs I use here are:

*.la

(Unmasked on libtool itself, since it has a *.la file that other 
package's .configure scripts test for.)

/etc/cron.*/

(I use systemd's timers in place of cron and thus crontablets.)

-- 
Duncan - List replies preferred.   No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master."  Richard Stallman

Re: [gentoo-portage-dev] Re: [PATCHv3 1/2] MEDIUM: misc-functions: Be more quiet when removing non existing INSTALL_MASK

[Michael Orlitzky]

On 04/21/2015 05:48 AM, Duncan wrote:
>>
>> Well, I don't use INSTALL_MASK myself, so I don't have a real-world
>> use-case for you. However, it's clear that the code will expand shell
>> globs, so I preserved that behavior for compatibility.
> 
> I do, with shell globs, tho I didn't bother checking the above to see if 
> they'd have been affected.
> 

The docs for INSTALL_MASK (man 5 make.conf) don't mention that globs
will work. It's expecting a "space delimited list of file names." Does
it really take a space-delimited list of globs instead? If so, how does
that reconcile with the fact that * could match spaces?

Re: [gentoo-portage-dev] Re: [PATCHv3 1/2] MEDIUM: misc-functions: Be more quiet when removing non existing INSTALL_MASK

[Zac Medico]

On 04/21/2015 02:48 AM, Duncan wrote:
> Zac Medico posted on Mon, 20 Apr 2015 17:37:15 -0700 as excerpted:
> 
>> On 04/20/2015 05:31 PM, Bertrand Jacquin wrote:
>>> On 21/04/2015 00:39, Zac Medico wrote:
>>>> These are in the master branch now:
>>>>
>>>> https://gitweb.gentoo.org/proj/portage.git/commit/?
> id=5a1b870fe54ac06f864a648c3ea5cc118f6ce911
>>>>
>>>> https://gitweb.gentoo.org/proj/portage.git/commit/?
> id=e74e2670e3f043608fced9847e54bdbb19f35169
>>>>
>>>
>>> Thanks for that :)
>>>
>>>> I modified the test as follows in order to ensure that it still works
>>>> with shell globs:
>>>>
>>>> if [[ -e "${root}"/${no_inst} || "${root}"/${no_inst} != $(echo
>>>> "${root}"/${no_inst}) ]] ; then
>>>
>>> I do not really understand the use case here, do you have an example ?
>>>
>>> Cheers,
>>>
>>
>> Well, I don't use INSTALL_MASK myself, so I don't have a real-world
>> use-case for you. However, it's clear that the code will expand shell
>> globs, so I preserved that behavior for compatibility.
> 
> I do, with shell globs, tho I didn't bother checking the above to see if 
> they'd have been affected.
> 
> The two install-masks with globs I use here are:
> 
> *.la

Since it doesn't have an absolute path, this glob is handled by the
find/rm code which comes just after the shell glob code that I was
talking about.

> (Unmasked on libtool itself, since it has a *.la file that other 
> package's .configure scripts test for.)
> 
> /etc/cron.*/
> 
> (I use systemd's timers in place of cron and thus crontablets.)

This is absolute, so it is handled by the shell glob code in question.
-- 
Thanks,
Zac

Re: [gentoo-portage-dev] Re: [PATCHv3 1/2] MEDIUM: misc-functions: Be more quiet when removing non existing INSTALL_MASK

[Zac Medico]

On 04/21/2015 04:21 AM, Michael Orlitzky wrote:
> On 04/21/2015 05:48 AM, Duncan wrote:
>>>
>>> Well, I don't use INSTALL_MASK myself, so I don't have a real-world
>>> use-case for you. However, it's clear that the code will expand shell
>>> globs, so I preserved that behavior for compatibility.
>>
>> I do, with shell globs, tho I didn't bother checking the above to see if 
>> they'd have been affected.
>>
> 
> The docs for INSTALL_MASK (man 5 make.conf) don't mention that globs
> will work. It's expecting a "space delimited list of file names." Does
> it really take a space-delimited list of globs instead? If so, how does
> that reconcile with the fact that * could match spaces?

How does it conflict?
-- 
Thanks,
Zac

Re: [gentoo-portage-dev] Re: [PATCHv3 1/2] MEDIUM: misc-functions: Be more quiet when removing non existing INSTALL_MASK

[Zac Medico]

On 04/21/2015 10:27 AM, Zac Medico wrote:
> On 04/21/2015 02:48 AM, Duncan wrote:
>> Zac Medico posted on Mon, 20 Apr 2015 17:37:15 -0700 as excerpted:
>>
>>> On 04/20/2015 05:31 PM, Bertrand Jacquin wrote:
>>>> On 21/04/2015 00:39, Zac Medico wrote:
>>>>> These are in the master branch now:
>>>>>
>>>>> https://gitweb.gentoo.org/proj/portage.git/commit/?
>> id=5a1b870fe54ac06f864a648c3ea5cc118f6ce911
>>>>>
>>>>> https://gitweb.gentoo.org/proj/portage.git/commit/?
>> id=e74e2670e3f043608fced9847e54bdbb19f35169
>>>>>
>>>>
>>>> Thanks for that :)
>>>>
>>>>> I modified the test as follows in order to ensure that it still works
>>>>> with shell globs:
>>>>>
>>>>> if [[ -e "${root}"/${no_inst} || "${root}"/${no_inst} != $(echo
>>>>> "${root}"/${no_inst}) ]] ; then
>>>>
>>>> I do not really understand the use case here, do you have an example ?
>>>>
>>>> Cheers,
>>>>
>>>
>>> Well, I don't use INSTALL_MASK myself, so I don't have a real-world
>>> use-case for you. However, it's clear that the code will expand shell
>>> globs, so I preserved that behavior for compatibility.
>>
>> I do, with shell globs, tho I didn't bother checking the above to see if 
>> they'd have been affected.
>>
>> The two install-masks with globs I use here are:
>>
>> *.la
> 
> Since it doesn't have an absolute path, this glob is handled by the
> find/rm code which comes just after the shell glob code that I was
> talking about.
> 
>> (Unmasked on libtool itself, since it has a *.la file that other 
>> package's .configure scripts test for.)
>>
>> /etc/cron.*/
>>
>> (I use systemd's timers in place of cron and thus crontablets.)
> 
> This is absolute, so it is handled by the shell glob code in question.
> 

Oh, I see what you mean now. It's reconciled by a temporary 'set -o
noglob', followed by 'set +o noglob'.

-- 
Thanks,
Zac

Re: [gentoo-portage-dev] Re: [PATCHv3 1/2] MEDIUM: misc-functions: Be more quiet when removing non existing INSTALL_MASK

[Michael Orlitzky]

On 04/21/2015 01:28 PM, Zac Medico wrote:
>>
>> The docs for INSTALL_MASK (man 5 make.conf) don't mention that globs
>> will work. It's expecting a "space delimited list of file names." Does
>> it really take a space-delimited list of globs instead? If so, how does
>> that reconcile with the fact that * could match spaces?
> 
> How does it conflict?
> 

I guess it's more of a filenames-with-spaces question. Would this work?

  INSTALL_MASK="Boyd\ -\ Convex Optimization.pdf"

If you can escape the spaces, then the space-separated globs aren't
ambiguous. I was thinking of something like this:

  $ /bin/ls B*\ Convex*
  Boyd - Convex Optimization.pdf

Normally if you stick something like that in a quoted variable, its
spaces can be unescaped:

  INSTALL_MASK="B* Convex*"

But now it's two globs instead of one. The whole thing makes sense if
you can leave the space escaped though.

Re: [gentoo-portage-dev] Re: [PATCHv3 1/2] MEDIUM: misc-functions: Be more quiet when removing non existing INSTALL_MASK

[Zac Medico]

On 04/21/2015 12:08 PM, Michael Orlitzky wrote:
> On 04/21/2015 01:28 PM, Zac Medico wrote:
>>>
>>> The docs for INSTALL_MASK (man 5 make.conf) don't mention that globs
>>> will work. It's expecting a "space delimited list of file names." Does
>>> it really take a space-delimited list of globs instead? If so, how does
>>> that reconcile with the fact that * could match spaces?
>>
>> How does it conflict?
>>
> 
> I guess it's more of a filenames-with-spaces question. Would this work?
> 
>   INSTALL_MASK="Boyd\ -\ Convex Optimization.pdf"

No, it doesn't seem to work given the current code. For example:

$ INSTALL_MASK="Boyd\ -\ Convex Optimization.pdf"
$ for x in $INSTALL_MASK; do echo "$x" ; done
Boyd\
-\
Convex
Optimization.pdf

> If you can escape the spaces, then the space-separated globs aren't
> ambiguous. I was thinking of something like this:
> 
>   $ /bin/ls B*\ Convex*
>   Boyd - Convex Optimization.pdf
> 
> Normally if you stick something like that in a quoted variable, its
> spaces can be unescaped:
> 
>   INSTALL_MASK="B* Convex*"
> 
> But now it's two globs instead of one. The whole thing makes sense if
> you can leave the space escaped though.

I don't think the inventor of INSTALL_MASK thought about these kinds of
cases.
-- 
Thanks,
Zac

Re: [gentoo-portage-dev] [PATCHv3 1/2] MEDIUM: misc-functions: Be more quiet when removing non existing INSTALL_MASK

[Bertrand Jacquin]

On 21/04/2015 01:37, Zac Medico wrote:
> On 04/20/2015 05:31 PM, Bertrand Jacquin wrote:
>> On 21/04/2015 00:39, Zac Medico wrote:
>>> These are in the master branch now:
>>> 
>>> https://gitweb.gentoo.org/proj/portage.git/commit/?id=5a1b870fe54ac06f864a648c3ea5cc118f6ce911
>>> 
>>> https://gitweb.gentoo.org/proj/portage.git/commit/?id=e74e2670e3f043608fced9847e54bdbb19f35169
>>> 
>> 
>> Thanks for that :)
>> 
>>> I modified the test as follows in order to ensure that it still works
>>> with shell globs:
>>> 
>>> if [[ -e "${root}"/${no_inst} || "${root}"/${no_inst} != $(echo
>>> "${root}"/${no_inst}) ]] ; then
>> 
>> I do not really understand the use case here, do you have an example ?
>> 
>> Cheers,
>> 
> 
> Well, I don't use INSTALL_MASK myself, so I don't have a real-world
> use-case for you. However, it's clear that the code will expand shell
> globs, so I preserved that behavior for compatibility.

Sorry Zac, I was speaking about the $(echo ..), what does it bring to 
use it here ?

Cheers,

-- 
Bertrand

Re: [gentoo-portage-dev] [PATCHv3 1/2] MEDIUM: misc-functions: Be more quiet when removing non existing INSTALL_MASK

[Zac Medico]

On 04/22/2015 04:44 PM, Bertrand Jacquin wrote:
> On 21/04/2015 01:37, Zac Medico wrote:
>> On 04/20/2015 05:31 PM, Bertrand Jacquin wrote:
>>> On 21/04/2015 00:39, Zac Medico wrote:
>>>> These are in the master branch now:
>>>>
>>>> https://gitweb.gentoo.org/proj/portage.git/commit/?id=5a1b870fe54ac06f864a648c3ea5cc118f6ce911
>>>>
>>>>
>>>> https://gitweb.gentoo.org/proj/portage.git/commit/?id=e74e2670e3f043608fced9847e54bdbb19f35169
>>>>
>>>>
>>>
>>> Thanks for that :)
>>>
>>>> I modified the test as follows in order to ensure that it still works
>>>> with shell globs:
>>>>
>>>> if [[ -e "${root}"/${no_inst} || "${root}"/${no_inst} != $(echo
>>>> "${root}"/${no_inst}) ]] ; then
>>>
>>> I do not really understand the use case here, do you have an example ?
>>>
>>> Cheers,
>>>
>>
>> Well, I don't use INSTALL_MASK myself, so I don't have a real-world
>> use-case for you. However, it's clear that the code will expand shell
>> globs, so I preserved that behavior for compatibility.
> 
> Sorry Zac, I was speaking about the $(echo ..), what does it bring to
> use it here ?
> 
> Cheers,
> 

It's a way to check the result of check the result of shell glob
expansion. The rm call that follows will be subject to identical expansion.
-- 
Thanks,
Zac