[gentoo-dev] [PATCH 1/2] eapi8-dosym.eclass: New eclass.

[gentoo-dev] [PATCH 1/2] eapi8-dosym.eclass: New eclass.

[Ulrich Müller]

This implements the dosym command proposed for EAPI 8 (called dosym8
because we cannot use the same name as the package-manager builtin).

"dosym -r <target> <link>" will expand the (apparent) path of <target>
relative to the (apparent) path of the directory containing <link>.
The main aim of this is to allow for an absolute path to be specified
as the link target, and the function will count path components and
convert it into a relative path.

Since we're inside ED at this point but the image will finally be
installed in EROOT, we don't try to resolve any pre-existing symlinks
in <target> or <link>. In other words, path expansion only looks at
the specified apparent paths, without touching any actual files in ED
or EROOT.

Signed-off-by: Ulrich Müller <ulm@gentoo.org>
---
 eclass/eapi8-dosym.eclass | 108 ++++++++++++++++++++++++++++++++++++++
 1 file changed, 108 insertions(+)
 create mode 100644 eclass/eapi8-dosym.eclass

diff --git a/eclass/eapi8-dosym.eclass b/eclass/eapi8-dosym.eclass
new file mode 100644
index 000000000000..52f0ffe3e62b
--- /dev/null
+++ b/eclass/eapi8-dosym.eclass
@@ -0,0 +1,108 @@
+# Copyright 2020 Gentoo Authors
+# Distributed under the terms of the GNU General Public License v2
+
+# @ECLASS: eapi8-dosym.eclass
+# @MAINTAINER:
+# PMS team <pms@gentoo.org>
+# @AUTHOR:
+# Ulrich Müller <ulm@gentoo.org>
+# @SUPPORTED_EAPIS: 5 6 7
+# @BLURB: Testing implementation of EAPI 8 dosym -r option
+# @DESCRIPTION:
+# A stand-alone implementation of the dosym command aimed for EAPI 8.
+# Intended to be used for wider testing of the proposed option and to
+# allow ebuilds to switch to the new model early, with minimal change
+# needed for actual EAPI 8.
+#
+# https://bugs.gentoo.org/708360
+
+case ${EAPI} in
+	5|6|7) ;;
+	*) die "${ECLASS}: EAPI=${EAPI:-0} not supported" ;;
+esac
+
+# @FUNCTION: _dosym8_canonicalize
+# @USAGE: <path>
+# @INTERNAL
+# @DESCRIPTION:
+# Transparent bash-only replacement for GNU "realpath -m -s".
+# Resolve references to "/./", "/../" and remove extra "/" characters
+# from <path>, without touching any actual file.
+_dosym8_canonicalize() {
+	local path slash i prev out IFS=/
+
+	path=( $1 )
+	[[ $1 == /* ]] && slash=/
+
+	while true; do
+		# Find first instance of non-".." path component followed by "..",
+		# or as a special case, "/.." at the beginning of the path.
+		# Also drop empty and "." path components as we go along.
+		prev=
+		for i in ${!path[@]}; do
+			if [[ -z ${path[i]} || ${path[i]} == . ]]; then
+				unset "path[i]"
+			elif [[ ${path[i]} != .. ]]; then
+				prev=${i}
+			elif [[ ${prev} || ${slash} ]]; then
+				# Found, remove path components and reiterate
+				[[ ${prev} ]] && unset "path[prev]"
+				unset "path[i]"
+				continue 2
+			fi
+		done
+		# No (further) instance found, so we're done
+		break
+	done
+
+	out="${slash}${path[*]}"
+	echo "${out:-.}"
+}
+
+# @FUNCTION: dosym8
+# @USAGE: [-r] <target> <link>
+# @DESCRIPTION:
+# Create a symbolic link <link>, pointing to <target>.  If the
+# directory containing the new link does not exist, create it.
+#
+# If called with option -r, expand <target> relative to the apparent
+# path of the directory containing <link>.  For example, "dosym8 -r
+# /bin/foo /usr/bin/foo" will create a link named "../../bin/foo".
+dosym8() {
+	local option_r
+
+	case $1 in
+		-r) option_r=t; shift ;;
+	esac
+
+	[[ $# -eq 2 ]] || die "${FUNCNAME}: bad number of arguments"
+
+	local target=$1 link=$2
+
+	if [[ ${option_r} ]]; then
+		local linkdir comp
+
+		# Expansion makes sense only for an absolute target path
+		[[ ${target} == /* ]] \
+			|| die "${FUNCNAME}: -r specified but no absolute target path"
+
+		target=$(_dosym8_canonicalize "${target}")
+		linkdir=$(_dosym8_canonicalize "/${link#/}")
+		linkdir=${linkdir%/*}	# poor man's dirname(1)
+		linkdir=${linkdir:-/}	# always keep the initial "/"
+
+		local ifs_save=${IFS-$' \t\n'} IFS=/
+		for comp in ${linkdir}; do
+			if [[ ${target%%/*} == "${comp}" ]]; then
+				target=${target#"${comp}"}
+				target=${target#/}
+			else
+				target=..${target:+/}${target}
+			fi
+		done
+		IFS=${ifs_save}
+		target=${target:-.}
+	fi
+
+	dosym "${target}" "${link}"
+}
-- 
2.29.2

[gentoo-dev] [PATCH 2/2] eclass/tests: Initial test cases for eapi8-dosym.eclass.

[Ulrich Müller]

Signed-off-by: Ulrich Müller <ulm@gentoo.org>
---
 eclass/tests/eapi8-dosym.sh | 78 +++++++++++++++++++++++++++++++++++++
 1 file changed, 78 insertions(+)
 create mode 100755 eclass/tests/eapi8-dosym.sh

diff --git a/eclass/tests/eapi8-dosym.sh b/eclass/tests/eapi8-dosym.sh
new file mode 100755
index 000000000000..ac1defc6c5f1
--- /dev/null
+++ b/eclass/tests/eapi8-dosym.sh
@@ -0,0 +1,78 @@
+#!/bin/bash
+# Copyright 2020 Gentoo Authors
+# Distributed under the terms of the GNU General Public License v2
+
+EAPI=7
+
+source tests-common.sh
+
+inherit eapi8-dosym
+
+dosym() {
+	echo "$1"
+}
+
+# reference implementation using GNU realpath
+ref_canonicalize() {
+	realpath -m -s "$1"
+}
+
+ref_dosym_r() {
+	local link=$(realpath -m -s "/${2#/}")
+	realpath -m -s --relative-to="$(dirname "${link}")" "$1"
+}
+
+randompath() {
+	dd if=/dev/urandom bs=128 count=1 2>/dev/null | LC_ALL=C sed \
+		-e 's/[^a-zA-M]//g;s/[A-E]/\/.\//g;s/[F-J]/\/..\//g;s/[K-M]/\//g' \
+		-e 's/^/\//;q'
+}
+
+teq() {
+	local expected=$1; shift
+	tbegin "$* -> ${expected}"
+	local got=$("$@")
+	[[ ${got} == "${expected}" ]]
+	tend $? "returned: ${got}"
+}
+
+for f in ref_canonicalize "_dosym8_canonicalize"; do
+	# canonicalize absolute paths
+	teq / ${f} /
+	teq /foo/baz/quux ${f} /foo/bar/../baz/quux
+	teq /foo ${f} /../../../foo
+	teq /bar ${f} /foo//./..///bar
+	teq /baz ${f} /foo/bar/../../../baz
+	teq /a/d/f/g ${f} /a/b/c/../../d/e/../f/g
+
+	# canonicalize relative paths (not actually used)
+	teq . _dosym8_canonicalize .
+	teq foo _dosym8_canonicalize foo
+	teq foo _dosym8_canonicalize ./foo
+	teq ../foo _dosym8_canonicalize ../foo
+	teq ../baz _dosym8_canonicalize foo/bar/../../../baz
+done
+
+for f in ref_dosym_r "dosym8 -r"; do
+	teq ../../bin/foo ${f} /bin/foo /usr/bin/foo
+	teq ../../../doc/foo-1 \
+		${f} /usr/share/doc/foo-1 /usr/share/texmf-site/doc/fonts/foo
+	teq ../../opt/bar/foo ${f} /opt/bar/foo /usr/bin/foo
+	teq ../c/d/e ${f} /a/b/c/d/e a/b/f/g
+	teq b/f ${f} /a/b///./c/d/../e/..//../f /a/././///g/../h
+	teq ../h ${f} /a/././///g/../h /a/b///./c/d/../e/..//../f
+	teq . ${f} /foo /foo/bar
+	teq .. ${f} /foo /foo/bar/baz
+	teq '../../fo . o/b ar' ${f} '/fo . o/b ar' '/baz / qu .. ux/qu x'
+	teq '../../f"o\o/b$a[]r' ${f} '/f"o\o/b$a[]r' '/ba\z/qu$u"x/qux'
+done
+
+# set RANDOMTESTS to a positive number to enable random tests
+for (( i = 0; i < RANDOMTESTS; i++ )); do
+	targ=$(randompath)
+	link=$(randompath)
+	out=$(ref_dosym_r "${targ}" "${link}")
+	teq "${out}" dosym8 -r "${targ}" "${link}"
+done
+
+texit
-- 
2.29.2

Re: [gentoo-dev] [PATCH 1/2] eapi8-dosym.eclass: New eclass.

[Alec Warner]

[-- Attachment #1: Type: text/plain, Size: 5742 bytes --]

On Thu, Nov 19, 2020 at 2:32 AM Ulrich Müller <ulm@gentoo.org> wrote:

> This implements the dosym command proposed for EAPI 8 (called dosym8
> because we cannot use the same name as the package-manager builtin).
>
> "dosym -r <target> <link>" will expand the (apparent) path of <target>
> relative to the (apparent) path of the directory containing <link>.
> The main aim of this is to allow for an absolute path to be specified
> as the link target, and the function will count path components and
> convert it into a relative path.
>
> Since we're inside ED at this point but the image will finally be
> installed in EROOT, we don't try to resolve any pre-existing symlinks
> in <target> or <link>. In other words, path expansion only looks at
> the specified apparent paths, without touching any actual files in ED
> or EROOT.
>
> Signed-off-by: Ulrich Müller <ulm@gentoo.org>
> ---
>  eclass/eapi8-dosym.eclass | 108 ++++++++++++++++++++++++++++++++++++++
>  1 file changed, 108 insertions(+)
>  create mode 100644 eclass/eapi8-dosym.eclass
>
> diff --git a/eclass/eapi8-dosym.eclass b/eclass/eapi8-dosym.eclass
> new file mode 100644
> index 000000000000..52f0ffe3e62b
> --- /dev/null
> +++ b/eclass/eapi8-dosym.eclass
> @@ -0,0 +1,108 @@
> +# Copyright 2020 Gentoo Authors
> +# Distributed under the terms of the GNU General Public License v2
> +
> +# @ECLASS: eapi8-dosym.eclass
> +# @MAINTAINER:
> +# PMS team <pms@gentoo.org>
> +# @AUTHOR:
> +# Ulrich Müller <ulm@gentoo.org>
> +# @SUPPORTED_EAPIS: 5 6 7
> +# @BLURB: Testing implementation of EAPI 8 dosym -r option
> +# @DESCRIPTION:
> +# A stand-alone implementation of the dosym command aimed for EAPI 8.
> +# Intended to be used for wider testing of the proposed option and to
> +# allow ebuilds to switch to the new model early, with minimal change
> +# needed for actual EAPI 8.
> +#
> +# https://bugs.gentoo.org/708360
> +
> +case ${EAPI} in
> +       5|6|7) ;;
> +       *) die "${ECLASS}: EAPI=${EAPI:-0} not supported" ;;
> +esac
> +
> +# @FUNCTION: _dosym8_canonicalize
> +# @USAGE: <path>
> +# @INTERNAL
> +# @DESCRIPTION:
> +# Transparent bash-only replacement for GNU "realpath -m -s".
> +# Resolve references to "/./", "/../" and remove extra "/" characters
> +# from <path>, without touching any actual file.
>

Take this as a nit, but do we have any way to test this function
(particularly around edge cases.)
I see eclass/tests exists, could we add a couple for this?


> +_dosym8_canonicalize() {
>

in dosym8() you save and restore IFS, but you don't here, is there a reason
for that?


> +       local path slash i prev out IFS=/
> +
> +       path=( $1 )
> +       [[ $1 == /* ]] && slash=/
> +
> +       while true; do
> +               # Find first instance of non-".." path component followed
> by "..",
> +               # or as a special case, "/.." at the beginning of the path.
> +               # Also drop empty and "." path components as we go along.
> +               prev=
> +               for i in ${!path[@]}; do
> +                       if [[ -z ${path[i]} || ${path[i]} == . ]]; then
> +                               unset "path[i]"
> +                       elif [[ ${path[i]} != .. ]]; then
> +                               prev=${i}
> +                       elif [[ ${prev} || ${slash} ]]; then
> +                               # Found, remove path components and
> reiterate
> +                               [[ ${prev} ]] && unset "path[prev]"
> +                               unset "path[i]"
> +                               continue 2
> +                       fi
> +               done
> +               # No (further) instance found, so we're done
> +               break
> +       done
> +
> +       out="${slash}${path[*]}"
> +       echo "${out:-.}"
> +}
> +
> +# @FUNCTION: dosym8
> +# @USAGE: [-r] <target> <link>
> +# @DESCRIPTION:
> +# Create a symbolic link <link>, pointing to <target>.  If the
> +# directory containing the new link does not exist, create it.
> +#
> +# If called with option -r, expand <target> relative to the apparent
> +# path of the directory containing <link>.  For example, "dosym8 -r
> +# /bin/foo /usr/bin/foo" will create a link named "../../bin/foo".
> +dosym8() {
> +       local option_r
> +
> +       case $1 in
> +               -r) option_r=t; shift ;;
> +       esac
> +
> +       [[ $# -eq 2 ]] || die "${FUNCNAME}: bad number of arguments"
> +
> +       local target=$1 link=$2
> +
> +       if [[ ${option_r} ]]; then
> +               local linkdir comp
> +
> +               # Expansion makes sense only for an absolute target path
> +               [[ ${target} == /* ]] \
> +                       || die "${FUNCNAME}: -r specified but no absolute
> target path"
> +
> +               target=$(_dosym8_canonicalize "${target}")
> +               linkdir=$(_dosym8_canonicalize "/${link#/}")
> +               linkdir=${linkdir%/*}   # poor man's dirname(1)
> +               linkdir=${linkdir:-/}   # always keep the initial "/"
> +
> +               local ifs_save=${IFS-$' \t\n'} IFS=/
> +               for comp in ${linkdir}; do
> +                       if [[ ${target%%/*} == "${comp}" ]]; then
> +                               target=${target#"${comp}"}
> +                               target=${target#/}
> +                       else
> +                               target=..${target:+/}${target}
> +                       fi
> +               done
> +               IFS=${ifs_save}
> +               target=${target:-.}
> +       fi
> +
> +       dosym "${target}" "${link}"
> +}
> --
> 2.29.2
>
>
>

[-- Attachment #2: Type: text/html, Size: 7770 bytes --]

Re: [gentoo-dev] [PATCH 1/2] eapi8-dosym.eclass: New eclass.

[Alec Warner]

[-- Attachment #1: Type: text/plain, Size: 6082 bytes --]

On Thu, Nov 19, 2020 at 7:28 AM Alec Warner <antarus@gentoo.org> wrote:

>
>
> On Thu, Nov 19, 2020 at 2:32 AM Ulrich Müller <ulm@gentoo.org> wrote:
>
>> This implements the dosym command proposed for EAPI 8 (called dosym8
>> because we cannot use the same name as the package-manager builtin).
>>
>> "dosym -r <target> <link>" will expand the (apparent) path of <target>
>> relative to the (apparent) path of the directory containing <link>.
>> The main aim of this is to allow for an absolute path to be specified
>> as the link target, and the function will count path components and
>> convert it into a relative path.
>>
>> Since we're inside ED at this point but the image will finally be
>> installed in EROOT, we don't try to resolve any pre-existing symlinks
>> in <target> or <link>. In other words, path expansion only looks at
>> the specified apparent paths, without touching any actual files in ED
>> or EROOT.
>>
>> Signed-off-by: Ulrich Müller <ulm@gentoo.org>
>> ---
>>  eclass/eapi8-dosym.eclass | 108 ++++++++++++++++++++++++++++++++++++++
>>  1 file changed, 108 insertions(+)
>>  create mode 100644 eclass/eapi8-dosym.eclass
>>
>> diff --git a/eclass/eapi8-dosym.eclass b/eclass/eapi8-dosym.eclass
>> new file mode 100644
>> index 000000000000..52f0ffe3e62b
>> --- /dev/null
>> +++ b/eclass/eapi8-dosym.eclass
>> @@ -0,0 +1,108 @@
>> +# Copyright 2020 Gentoo Authors
>> +# Distributed under the terms of the GNU General Public License v2
>> +
>> +# @ECLASS: eapi8-dosym.eclass
>> +# @MAINTAINER:
>> +# PMS team <pms@gentoo.org>
>> +# @AUTHOR:
>> +# Ulrich Müller <ulm@gentoo.org>
>> +# @SUPPORTED_EAPIS: 5 6 7
>> +# @BLURB: Testing implementation of EAPI 8 dosym -r option
>> +# @DESCRIPTION:
>> +# A stand-alone implementation of the dosym command aimed for EAPI 8.
>> +# Intended to be used for wider testing of the proposed option and to
>> +# allow ebuilds to switch to the new model early, with minimal change
>> +# needed for actual EAPI 8.
>> +#
>> +# https://bugs.gentoo.org/708360
>> +
>> +case ${EAPI} in
>> +       5|6|7) ;;
>> +       *) die "${ECLASS}: EAPI=${EAPI:-0} not supported" ;;
>> +esac
>> +
>> +# @FUNCTION: _dosym8_canonicalize
>> +# @USAGE: <path>
>> +# @INTERNAL
>> +# @DESCRIPTION:
>> +# Transparent bash-only replacement for GNU "realpath -m -s".
>> +# Resolve references to "/./", "/../" and remove extra "/" characters
>> +# from <path>, without touching any actual file.
>>
>
> Take this as a nit, but do we have any way to test this function
> (particularly around edge cases.)
> I see eclass/tests exists, could we add a couple for this?
>

hah, and you added them in the second patch, sorry I didn't read ahead! :)

-A


>
>
>> +_dosym8_canonicalize() {
>>
>
> in dosym8() you save and restore IFS, but you don't here, is there a
> reason for that?
>
>
>> +       local path slash i prev out IFS=/
>> +
>> +       path=( $1 )
>> +       [[ $1 == /* ]] && slash=/
>> +
>> +       while true; do
>> +               # Find first instance of non-".." path component followed
>> by "..",
>> +               # or as a special case, "/.." at the beginning of the
>> path.
>> +               # Also drop empty and "." path components as we go along.
>> +               prev=
>> +               for i in ${!path[@]}; do
>> +                       if [[ -z ${path[i]} || ${path[i]} == . ]]; then
>> +                               unset "path[i]"
>> +                       elif [[ ${path[i]} != .. ]]; then
>> +                               prev=${i}
>> +                       elif [[ ${prev} || ${slash} ]]; then
>> +                               # Found, remove path components and
>> reiterate
>> +                               [[ ${prev} ]] && unset "path[prev]"
>> +                               unset "path[i]"
>> +                               continue 2
>> +                       fi
>> +               done
>> +               # No (further) instance found, so we're done
>> +               break
>> +       done
>> +
>> +       out="${slash}${path[*]}"
>> +       echo "${out:-.}"
>> +}
>> +
>> +# @FUNCTION: dosym8
>> +# @USAGE: [-r] <target> <link>
>> +# @DESCRIPTION:
>> +# Create a symbolic link <link>, pointing to <target>.  If the
>> +# directory containing the new link does not exist, create it.
>> +#
>> +# If called with option -r, expand <target> relative to the apparent
>> +# path of the directory containing <link>.  For example, "dosym8 -r
>> +# /bin/foo /usr/bin/foo" will create a link named "../../bin/foo".
>> +dosym8() {
>> +       local option_r
>> +
>> +       case $1 in
>> +               -r) option_r=t; shift ;;
>> +       esac
>> +
>> +       [[ $# -eq 2 ]] || die "${FUNCNAME}: bad number of arguments"
>> +
>> +       local target=$1 link=$2
>> +
>> +       if [[ ${option_r} ]]; then
>> +               local linkdir comp
>> +
>> +               # Expansion makes sense only for an absolute target path
>> +               [[ ${target} == /* ]] \
>> +                       || die "${FUNCNAME}: -r specified but no absolute
>> target path"
>> +
>> +               target=$(_dosym8_canonicalize "${target}")
>> +               linkdir=$(_dosym8_canonicalize "/${link#/}")
>> +               linkdir=${linkdir%/*}   # poor man's dirname(1)
>> +               linkdir=${linkdir:-/}   # always keep the initial "/"
>> +
>> +               local ifs_save=${IFS-$' \t\n'} IFS=/
>> +               for comp in ${linkdir}; do
>> +                       if [[ ${target%%/*} == "${comp}" ]]; then
>> +                               target=${target#"${comp}"}
>> +                               target=${target#/}
>> +                       else
>> +                               target=..${target:+/}${target}
>> +                       fi
>> +               done
>> +               IFS=${ifs_save}
>> +               target=${target:-.}
>> +       fi
>> +
>> +       dosym "${target}" "${link}"
>> +}
>> --
>> 2.29.2
>>
>>
>>

[-- Attachment #2: Type: text/html, Size: 8495 bytes --]

Re: [gentoo-dev] [PATCH 1/2] eapi8-dosym.eclass: New eclass.

[Ulrich Mueller]

[-- Attachment #1: Type: text/plain, Size: 380 bytes --]

>>>>> On Thu, 19 Nov 2020, Alec Warner wrote:

>> +_dosym8_canonicalize() {

> in dosym8() you save and restore IFS, but you don't here, is there a reason
> for that?

>> +       local path slash i prev out IFS=/

IFS is a local variable, so its scope ends when the function returns.
We also don't call any other function from it, so we can get away
without save/restore.

Ulrich

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 507 bytes --]

Re: [gentoo-dev] [PATCH 1/2] eapi8-dosym.eclass: New eclass.

[Ulrich Mueller]

[-- Attachment #1: Type: text/plain, Size: 1148 bytes --]

>>>>> On Thu, 19 Nov 2020, Ulrich Müller wrote:

> This implements the dosym command proposed for EAPI 8 (called dosym8
> because we cannot use the same name as the package-manager builtin).

> "dosym -r <target> <link>" will expand the (apparent) path of <target>
> relative to the (apparent) path of the directory containing <link>.
> The main aim of this is to allow for an absolute path to be specified
> as the link target, and the function will count path components and
> convert it into a relative path.

> Since we're inside ED at this point but the image will finally be
> installed in EROOT, we don't try to resolve any pre-existing symlinks
> in <target> or <link>. In other words, path expansion only looks at
> the specified apparent paths, without touching any actual files in ED
> or EROOT.

Pushed to master.

Some simple examples to demonstrate usage:

dosym8 -r /bin/foo /usr/bin/foo
- creates a relative symlink named ../../bin/foo

dosym8 -r /usr/share/doc/foo-1 "${TEXMF}"/doc/latex/foo
- creates a relative symlink named ../../../doc/foo-1
  (with TEXMF="/usr/share/texmf-site")

Enjoy!
Ulrich

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 507 bytes --]