[gentoo-dev] [RFC] [PATCH] appimage.eclass: new eclass

[gentoo-dev] [RFC] [PATCH] appimage.eclass: new eclass

[Mykyta Holubakha]

Signed-off-by: Mykyta Holubakha <hilobakho@gmail.com>

I'm proposing to add a new eclass: appimage.eclass, to facilitate
extraction off AppImage bundles. The rationale is that some upstreams
have migrated to distributing their proprietary software exclusively as
AppImage bundles. (for instance dev-util/staruml-bin).

An example ebuild can be seen at https://git.io/fx3Mg

I'd like to ask the following questions:

1. Can I put myself and proxy-maint under @MAINTAINER (or do I need to
find a gentoo dev)?

2. Are we OK with executing AppImage bundles downloaded from the
Internet (an alternative would be to implement a proper extractor
program, which would unpack the images without executing them, and add
it to DEPENDs).

Thanks

---
 eclass/appimage.eclass | 69 ++++++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 69 insertions(+)
 create mode 100644 eclass/appimage.eclass

diff --git a/eclass/appimage.eclass b/eclass/appimage.eclass
new file mode 100644
index 00000000000..454bdedc07b
--- /dev/null
+++ b/eclass/appimage.eclass
@@ -0,0 +1,69 @@
+# Copyright 2018 Gentoo Foundation
+# Distributed under the terms of the GNU General Public License v2
+
+# @ECLASS: appimage.eclass
+# @MAINTAINER:
+# maintainer-wanted@gentoo.org
+# @AUTHOR:
+# Mykyta Holubakha <hilobakho@gmail.com>
+# @BLURB: convenience eclass for extracting AppImage bundles
+# @DESCRIPTION:
+# This eclass provides a src_unpack function to extract AppImage bundles
+
+case "${EAPI:-0}" in
+	6|7)
+		;;
+	*)
+		die "EAPI ${EAPI:-0} is not supported"
+		;;
+esac
+
+EXPORT_FUNCTIONS src_unpack
+
+# @VARIABLE: APPIMAGE_EXTRACT_DIR
+# @DEFAULT_UNSET: squashfs_root
+# @DESCRIPTION:
+# This variable specifies the directory, in which the AppImage bundle
+# is expected to be extracted.
+
+# @VARIABLE: APPIMAGE_EXTRACT_DEST
+# @DEFAULT_UNSET: ${P}
+# @DESCRIPTION:
+# This variable specifies the directory, to which the extracted image
+# will be moved.
+
+# @FUNCTION: appimage_src_unpack
+# @DESCRIPTION:
+# Unpack all the AppImage bundles from ${A} (with .appimage extension).
+# Other files are passed to regular unpack.
+appimage_src_unpack() {
+	debug-print-function ${FUNCNAME} "${@}"
+
+	local extract_dir="${APPIMAGE_EXTRACT_DIR:-squashfs-root}"
+	local extract_dest="${APPIMAGE_EXTRACT_DEST:-${P}}"
+
+	local f
+	for f in ${A}
+	do
+		case "${f}" in
+			*.appimage|*.AppImage)
+				cp "${DISTDIR}/${f}" "${WORKDIR}"
+				debug-print "${FUNCNAME}: unpacking bundle ${f} to ${extract_dest}"
+				chmod +x "${f}" \
+					|| die "Failed to add execute permissions to bundle"
+				"${WORKDIR}/${f}" --appimage-help >/dev/null 2>/dev/null \
+					|| die "Invalid AppImage bundle"
+				"${WORKDIR}/${f}" --appimage-extract >/dev/null 2>/dev/null \
+					|| die "Failed to extract AppImage bundle"
+				rm -f "${f}" || die "Failed to remove bundle copy"
+				mv "${extract_dir}" "${extract_dest}" \
+					|| die "Failed to move AppImage bundle to destination"
+				;;
+			*)
+				debug-print "${FUNCNAME}: falling back to unpack for ${f}"
+				unpack "${f}"
+				;;
+		esac
+	done
+}
+
-- 
2.15.1

Re: [gentoo-dev] [RFC] [PATCH] appimage.eclass: new eclass

[Andrew Savchenko]

[-- Attachment #1: Type: text/plain, Size: 1173 bytes --]

Hi!

On Sat,  6 Oct 2018 14:17:50 +0300 Mykyta Holubakha wrote:
> I'm proposing to add a new eclass: appimage.eclass, to facilitate
> extraction off AppImage bundles. The rationale is that some upstreams
> have migrated to distributing their proprietary software exclusively as
> AppImage bundles. (for instance dev-util/staruml-bin).
> 
> An example ebuild can be seen at https://git.io/fx3Mg
>
> I'd like to ask the following questions:
>
> 1. Can I put myself and proxy-maint under @MAINTAINER (or do I need to
> find a gentoo dev)?

Likely no. We have no such eclasses right now. Eclasses have more
strict requirements than ebuilds, e.g. they should not be changed
without a prior ML discussion except for project-specific eclasses.

> 2. Are we OK with executing AppImage bundles downloaded from the
> Internet (an alternative would be to implement a proper extractor
> program, which would unpack the images without executing them, and add
> it to DEPENDs).

This would be a considerable security risk, so no. You should use
some extractor. Looks like appimage carries filesystem inside with
some offset.

Best regards,
Andrew Savchenko

[-- Attachment #2: Type: application/pgp-signature, Size: 833 bytes --]

Re: [gentoo-dev] [RFC] [PATCH] appimage.eclass: new eclass

[Ulrich Mueller]

[-- Attachment #1: Type: text/plain, Size: 4304 bytes --]

>>>>> On Sat, 06 Oct 2018, Mykyta Holubakha wrote:

> Signed-off-by: Mykyta Holubakha <hilobakho@gmail.com>

> I'm proposing to add a new eclass: appimage.eclass, to facilitate
> extraction off AppImage bundles. The rationale is that some upstreams
> have migrated to distributing their proprietary software exclusively as
> AppImage bundles. (for instance dev-util/staruml-bin).

Ask upstream to use a friendlier package format?

> An example ebuild can be seen at https://git.io/fx3Mg

> I'd like to ask the following questions:

> 1. Can I put myself and proxy-maint under @MAINTAINER (or do I need to
> find a gentoo dev)?

You should contact proxy-maintainers in any case, because somebody needs
to commit it for you.

> 2. Are we OK with executing AppImage bundles downloaded from the
> Internet (an alternative would be to implement a proper extractor
> program, which would unpack the images without executing them, and add
> it to DEPENDs).

These are ELF binaries unpacking an included ISO 9660 image, right?
Not sure if it's a good idea to execute those in src_unpack. Won't they
require external dependencies to run?

I have also looked at staruml-bin-3.0.2.ebuild (your example above) and
I believe that it is highly problematic from a license point of view.
The distfile includes bundled LGPL libraries without including their
source, which means that it cannot be distributed.

> ---
>  eclass/appimage.eclass | 69 ++++++++++++++++++++++++++++++++++++++++++++++++++
>  1 file changed, 69 insertions(+)
>  create mode 100644 eclass/appimage.eclass

> diff --git a/eclass/appimage.eclass b/eclass/appimage.eclass
> new file mode 100644
> index 00000000000..454bdedc07b
> --- /dev/null
> +++ b/eclass/appimage.eclass
> @@ -0,0 +1,69 @@
> +# Copyright 2018 Gentoo Foundation

This should be "Gentoo Authors" by the new copyright policy.

> +# Distributed under the terms of the GNU General Public License v2
> +
> +# @ECLASS: appimage.eclass
> +# @MAINTAINER:
> +# maintainer-wanted@gentoo.org
> +# @AUTHOR:
> +# Mykyta Holubakha <hilobakho@gmail.com>
> +# @BLURB: convenience eclass for extracting AppImage bundles
> +# @DESCRIPTION:
> +# This eclass provides a src_unpack function to extract AppImage bundles
> +
> +case "${EAPI:-0}" in

The quotes are not needed.

> +	6|7)
> +		;;
> +	*)
> +		die "EAPI ${EAPI:-0} is not supported"
> +		;;
> +esac
> +
> +EXPORT_FUNCTIONS src_unpack
> +
> +# @VARIABLE: APPIMAGE_EXTRACT_DIR
> +# @DEFAULT_UNSET: squashfs_root

@DEFAULT_UNSET doesn't take any parameters.

> +# @DESCRIPTION:
> +# This variable specifies the directory, in which the AppImage bundle
> +# is expected to be extracted.
> +
> +# @VARIABLE: APPIMAGE_EXTRACT_DEST
> +# @DEFAULT_UNSET: ${P}
> +# @DESCRIPTION:
> +# This variable specifies the directory, to which the extracted image
> +# will be moved.
> +
> +# @FUNCTION: appimage_src_unpack
> +# @DESCRIPTION:
> +# Unpack all the AppImage bundles from ${A} (with .appimage extension).
> +# Other files are passed to regular unpack.
> +appimage_src_unpack() {
> +	debug-print-function ${FUNCNAME} "${@}"
> +
> +	local extract_dir="${APPIMAGE_EXTRACT_DIR:-squashfs-root}"
> +	local extract_dest="${APPIMAGE_EXTRACT_DEST:-${P}}"
> +
> +	local f
> +	for f in ${A}
> +	do
> +		case "${f}" in

No quotes here.

> +			*.appimage|*.AppImage)
> +				cp "${DISTDIR}/${f}" "${WORKDIR}"

Add "|| die".

> +				debug-print "${FUNCNAME}: unpacking bundle ${f} to ${extract_dest}"
> +				chmod +x "${f}" \
> +					|| die "Failed to add execute permissions to bundle"
> +				"${WORKDIR}/${f}" --appimage-help >/dev/null 2>/dev/null \
> +					|| die "Invalid AppImage bundle"
> +				"${WORKDIR}/${f}" --appimage-extract >/dev/null 2>/dev/null \
> +					|| die "Failed to extract AppImage bundle"

See above, presumably it would be better to use an external tool for
extraction.

> +				rm -f "${f}" || die "Failed to remove bundle copy"
> +				mv "${extract_dir}" "${extract_dest}" \
> +					|| die "Failed to move AppImage bundle to destination"
> +				;;
> +			*)
> +				debug-print "${FUNCNAME}: falling back to unpack for ${f}"
> +				unpack "${f}"
> +				;;
> +		esac
> +	done
> +}
> +
> -- 
> 2.15.1



[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 487 bytes --]

Re: [gentoo-dev] [RFC] [PATCH] appimage.eclass: new eclass

[Mikle Kolyada]

[-- Attachment #1.1: Type: text/plain, Size: 418 bytes --]



On 06.10.2018 14:17, Mykyta Holubakha wrote:
> Signed-off-by: Mykyta Holubakha <hilobakho@gmail.com>
>
> I'm proposing to add a new eclass: appimage.eclass, to facilitate
> extraction off AppImage bundles. The rationale is that some upstreams
> have migrated to distributing their proprietary software exclusively as
> AppImage bundles. 

Why making the separate eclass? Merge it with the unpacker one.


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

Re: [gentoo-dev] [RFC] [PATCH] appimage.eclass: new eclass

[Bernd Waibel]

[-- Attachment #1: Type: text/plain, Size: 3213 bytes --]

Mykyta Holubakha <hilobakho@gmail.com> schrieb am Sa., 6. Okt. 2018 um
13:18 Uhr:

> Signed-off-by: Mykyta Holubakha <hilobakho@gmail.com>
>
> I'm proposing to add a new eclass: appimage.eclass, to facilitate
> extraction off AppImage bundles. The rationale is that some upstreams
> have migrated to distributing their proprietary software exclusively as
> AppImage bundles. (for instance dev-util/staruml-bin).
>
> An example ebuild can be seen at https://git.io/fx3Mg
>
>
I was also thinking of something, too. This direction of development
happens for a couple of years already, with more and more vendors /
upstream jumping on it. It was also highly recommended by Linus Torvalds as
a way of distributing packages easily along different linux flavors. I
don't think we should easily deny the direction where this is going.

Most appimages are distributed through central well-known servers. The
files have a sha1 BuildID attached to them (can be seen via file command),
which can possibly be used to verify them, so decreasing the possible
vulnerability to such packages somewhat. There might also be additional
ways to verify the integrity of them through the appimage API. They are
statically linked packages with no external dependencies, and contain a
squashfs filesystem in it.

From a users point of view, I think it'd be a good idea, to give users the
possibility of installing appimages with the package manager, so they can
be handled accordingly, instead of just let them download the files and
move them into a separate folder in ${HOME} for execution. This would be
especially useful for propietary bin-packages, as well as for some big
packages, which are actually hard to maintain for proxy-maintainers, and
therefore get removed from the tree (freecad comes to my mind as an
example).

2. Are we OK with executing AppImage bundles downloaded from the
> Internet (an alternative would be to implement a proper extractor
> program, which would unpack the images without executing them, and add
> it to DEPENDs).
>

It think, this has mostly the same attack vectors which apply for
bin-packages. So there's not much new from my point of view.


> +# @FUNCTION: appimage_src_unpack
> +# @DESCRIPTION:
> +# Unpack all the AppImage bundles from ${A} (with .appimage extension).
> +# Other files are passed to regular unpack.
>

I'm not sure, if it makes sense to unpack them. Most of them are not built
on gentoo systems, so the extracted binaries and libraries might not easily
run without having the ebuild maintainer check every single one of them for
run-time dependencies.

For example, the freecad appimage contains a bin/mkdir program linked
against libselinux.so.1 which is not available on non-selinux profiles by
default, or its usr/lib/freecad/lib/DraftUtils.so, a freecad internal
library, links against libboost_system.so.1.55.0 while we have
libboost_system.so.1.65.0.

I would propose to keep them packed and install them in /opt/AppImages or
something like that, thus providing only a src_install function with the
eclass. Ebuild maintainers could then just add a .desktop file or extract
one from the image if it's present and add it to the files/ directory for
desktop integration.

[-- Attachment #2: Type: text/html, Size: 4095 bytes --]