From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from lists.gentoo.org (pigeon.gentoo.org [208.92.234.80]) by finch.gentoo.org (Postfix) with ESMTP id 86F84138010 for ; Mon, 25 Mar 2013 22:42:37 +0000 (UTC) Received: from pigeon.gentoo.org (localhost [127.0.0.1]) by pigeon.gentoo.org (Postfix) with SMTP id A5F3EE0795; Mon, 25 Mar 2013 22:42:32 +0000 (UTC) Received: from smtp.gentoo.org (smtp.gentoo.org [140.211.166.183]) (using TLSv1 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by pigeon.gentoo.org (Postfix) with ESMTPS id B3684E0761 for ; Mon, 25 Mar 2013 22:42:31 +0000 (UTC) Received: from pomiocik.lan (87-205-59-1.adsl.inetia.pl [87.205.59.1]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) (Authenticated sender: mgorny) by smtp.gentoo.org (Postfix) with ESMTPSA id 658EE33BE2C; Mon, 25 Mar 2013 22:42:29 +0000 (UTC) From: =?UTF-8?q?Micha=C5=82=20G=C3=B3rny?= To: gentoo-dev@lists.gentoo.org Cc: ulm@gentoo.org, zmedico@gentoo.org, =?UTF-8?q?Micha=C5=82=20G=C3=B3rny?= Subject: [gentoo-dev] [PATCH] Introduce multibuild_merge_root() to merge interim installs. Date: Mon, 25 Mar 2013 23:42:50 +0100 Message-Id: <1364251370-1579-1-git-send-email-mgorny@gentoo.org> X-Mailer: git-send-email 1.8.1.5 In-Reply-To: <20813.57273.945749.854552@a1i15.kph.uni-mainz.de> References: <20813.57273.945749.854552@a1i15.kph.uni-mainz.de> Precedence: bulk List-Post: List-Help: List-Unsubscribe: List-Subscribe: List-Id: Gentoo Linux mail X-BeenThere: gentoo-dev@lists.gentoo.org Reply-to: gentoo-dev@lists.gentoo.org X-Archives-Salt: 48b40a59-9db5-40d3-b223-ad2edf3feee4 X-Archives-Hash: cccf263c6da1998ee3d7e2b4c739cbe7 This is mostly a copy from distutils-r1. The function does copy all the files from one location onto another, preserving whatever possible. It can be run in parallel too without the risk of race conditions. As suggested by Ulrich and Zac, I've modified the code to use checks based on userland rather than tool checking. The code supports BSD and GNU userlands as defined in the profiles. With any other userland, it explicitly dies. --- gx86/eclass/distutils-r1.eclass | 41 +------------------------------- gx86/eclass/multibuild.eclass | 52 +++++++++++++++++++++++++++++++++++++++++ 2 files changed, 53 insertions(+), 40 deletions(-) diff --git a/gx86/eclass/distutils-r1.eclass b/gx86/eclass/distutils-r1.eclass index 0982e6c..3c21741 100644 --- a/gx86/eclass/distutils-r1.eclass +++ b/gx86/eclass/distutils-r1.eclass @@ -449,49 +449,10 @@ distutils-r1_python_install() { if [[ ! ${DISTUTILS_SINGLE_IMPL} ]]; then _distutils-r1_rename_scripts "${root}" - _distutils-r1_merge_root "${root}" "${D}" + multibuild_merge_root "${root}" "${D}" fi } -# @FUNCTION: distutils-r1_merge_root -# @USAGE: -# @INTERNAL -# @DESCRIPTION: -# Merge the directory tree from to , removing -# the in the process. -_distutils-r1_merge_root() { - local src=${1} - local dest=${2} - - local lockfile=${T}/distutils-r1-merge-lock - - if type -P lockf &>/dev/null; then - # On BSD, we have 'lockf' wrapper. - tar -C "${src}" -f - -c . \ - | lockf "${lockfile}" tar -x -f - -C "${dest}" - else - local lock_fd - if type -P flock &>/dev/null; then - # On Linux, we have 'flock' which can lock fd. - redirect_alloc_fd lock_fd "${lockfile}" '>>' - flock ${lock_fd} - else - ewarn "distutils-r1: no locking service found, please report." - fi - - cp -a -l -n "${src}"/. "${dest}"/ - - if [[ ${lock_fd} ]]; then - # Close the lock file when we are done with it. - # Prevents deadlock if we aren't in a subshell. - eval "exec ${lock_fd}>&-" - fi - fi - [[ ${?} == 0 ]] || die "Merging ${EPYTHON} image failed." - - rm -rf "${src}" -} - # @FUNCTION: distutils-r1_python_install_all # @DESCRIPTION: # The default python_install_all(). It installs the documentation. diff --git a/gx86/eclass/multibuild.eclass b/gx86/eclass/multibuild.eclass index a3f1402..1152245 100644 --- a/gx86/eclass/multibuild.eclass +++ b/gx86/eclass/multibuild.eclass @@ -238,5 +238,57 @@ run_in_build_dir() { return ${ret} } +# @FUNCTION: multibuild_merge_root +# @USAGE: +# @DESCRIPTION: +# Merge the directory tree (fake root) from to +# (the real root). Both directories have to be real, absolute paths +# (i.e. including ${D}). Source root will be removed. +# +# This functions uses locking to support merging during parallel +# installs. +multibuild_merge_root() { + local src=${1} + local dest=${2} + + local lockfile=${T}/multibuild_merge_lock + local ret + + if use userland_BSD; then + # Locking is done by 'lockf' which can wrap a command. + # 'cp -a -n' is broken: + # http://www.freebsd.org/cgi/query-pr.cgi?pr=174489 + # using tar instead which is universal but terribly slow. + + tar -C "${src}" -f - -c . \ + | lockf "${lockfile}" tar -x -f - -C "${dest}" + [[ ${PIPESTATUS[*]} == '0 0' ]] + ret=${?} + elif use userland_GNU; then + # GNU has 'flock' which can't wrap commands but can lock + # a fd which is good enough for us. + # and cp works with '-a -n'. + + local lock_fd + redirect_alloc_fd lock_fd "${lockfile}" '>>' + flock ${lock_fd} + + cp -a -l -n "${src}"/. "${dest}"/ + ret=${?} + + # Close the lock file when we are done with it. + # Prevents deadlock if we aren't in a subshell. + eval "exec ${lock_fd}>&-" + else + die "Unsupported userland (${USERLAND}), please report." + fi + + if [[ ${ret} -ne 0 ]]; then + die "${MULTIBUILD_VARIANT:-(unknown)}: merging image failed." + fi + + rm -rf "${src}" +} + _MULTIBUILD=1 fi -- 1.8.1.5