From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from lists.gentoo.org (pigeon.gentoo.org [208.92.234.80]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by finch.gentoo.org (Postfix) with ESMTPS id C5CE11382C5 for ; Sun, 11 Mar 2018 19:20:32 +0000 (UTC) Received: from pigeon.gentoo.org (localhost [127.0.0.1]) by pigeon.gentoo.org (Postfix) with SMTP id D1CEFE0954; Sun, 11 Mar 2018 19:20:31 +0000 (UTC) Received: from smtp.gentoo.org (smtp.gentoo.org [140.211.166.183]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by pigeon.gentoo.org (Postfix) with ESMTPS id A030FE0946 for ; Sun, 11 Mar 2018 19:20:31 +0000 (UTC) Received: from oystercatcher.gentoo.org (unknown [IPv6:2a01:4f8:202:4333:225:90ff:fed9:fc84]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.gentoo.org (Postfix) with ESMTPS id 2CBCA335C31 for ; Sun, 11 Mar 2018 19:20:30 +0000 (UTC) Received: from localhost.localdomain (localhost [IPv6:::1]) by oystercatcher.gentoo.org (Postfix) with ESMTP id B1A2F247 for ; Sun, 11 Mar 2018 19:20:28 +0000 (UTC) From: "Michał Górny" To: gentoo-commits@lists.gentoo.org Content-Transfer-Encoding: 8bit Content-type: text/plain; charset=UTF-8 Reply-To: gentoo-dev@lists.gentoo.org, "Michał Górny" Message-ID: <1520796017.88dc67703f960d50fc43f2d22474c6ed742094b7.mgorny@gentoo> Subject: [gentoo-commits] data/glep:master commit in: / X-VCS-Repository: data/glep X-VCS-Files: glep-0074.rst X-VCS-Directories: / X-VCS-Committer: mgorny X-VCS-Committer-Name: Michał Górny X-VCS-Revision: 88dc67703f960d50fc43f2d22474c6ed742094b7 X-VCS-Branch: master Date: Sun, 11 Mar 2018 19:20:28 +0000 (UTC) Precedence: bulk List-Post: List-Help: List-Unsubscribe: List-Subscribe: List-Id: Gentoo Linux mail X-BeenThere: gentoo-commits@lists.gentoo.org X-Archives-Salt: 3053bcad-75cf-4442-b082-5b2637b4f6e3 X-Archives-Hash: eaca67c2e133d5c82f2e46629705e239 commit: 88dc67703f960d50fc43f2d22474c6ed742094b7 Author: Michał Górny gentoo org> AuthorDate: Thu Feb 8 17:05:35 2018 +0000 Commit: Michał Górny gentoo org> CommitDate: Sun Mar 11 19:20:17 2018 +0000 URL: https://gitweb.gentoo.org/data/glep.git/commit/?id=88dc6770 glep-0074: Remove single filesystem limitation Remove the limitation that all files covered by the Manifest must reside on a single filesystem. This breaks valid uses of overlayfs without providing any real advantage. The removal is justified further in the updated rationale section. Approved on 2018-03-11 Council meeting. Closes: https://bugs.gentoo.org/648638 glep-0074.rst | 66 +++++++++++++++++++++++++++++++++++------------------------ 1 file changed, 39 insertions(+), 27 deletions(-) diff --git a/glep-0074.rst b/glep-0074.rst index 3835247..06cf6e0 100644 --- a/glep-0074.rst +++ b/glep-0074.rst @@ -6,10 +6,10 @@ Author: Michał Górny , Ulrich Müller Type: Standards Track Status: Final -Version: 1 +Version: 1.1 Created: 2017-10-21 -Last-Modified: 2017-12-16 -Post-History: 2017-10-26, 2017-11-16 +Last-Modified: 2018-03-11 +Post-History: 2017-10-26, 2017-11-16, 2018-02-08 Content-Type: text/x-rst Requires: 59, 61 Replaces: 44, 58, 60 @@ -126,13 +126,6 @@ a different file type. If the tree contain files of other types that are not otherwise ignored, they need to be covered by an explicit ``IGNORE``. -All the local (non-``DIST``) files covered by a Manifest tree must -reside on the same filesystem. It is an error to specify entries -applying to files on another filesystem. If files or directories that -are not otherwise ignored reside on a different filesystem, or symbolic -links point to targets on a different filesystem, they must -be explicitly excluded via ``IGNORE``. - Path and filename encoding -------------------------- @@ -325,22 +318,18 @@ Algorithm for finding parent Manifests In order to find the top-level Manifest from the current directory the following algorithm can be used: -1. Store the current directory as *original* and the device ID - of the containing filesystem (``st_dev``) as *startdev*, - -2. If the device ID of the containing filesystem (``st_dev``) - of the current directory is different than *startdev*, stop. +1. Store the current directory as *original*, -3. If the current directory contains a ``Manifest`` file: +2. If the current directory contains a ``Manifest`` file: a. If an ``IGNORE`` entry in the ``Manifest`` file covers the *original* directory (or one of the parent directories), stop. b. Otherwise, store the current directory as *last_found*. -4. If the current directory is the root system directory (``/``), stop. +3. If the current directory is the root system directory (``/``), stop. -5. Otherwise, enter the parent directory and jump to step 2. +4. Otherwise, enter the parent directory and jump to step 2. Once the algorithm stops, *last_found* will contain the relevant top-level Manifest. If *last_found* is null, then the directory tree @@ -594,16 +583,39 @@ additional files in it, skipping some of the categories from syncing. This configuration can extend beyond the limits of this GLEP, e.g. by allowing wildcards or regular expressions. -The algorithm is restricted to work on a single filesystem. This is -mostly relevant when scanning for top-level Manifest -- we do not want -to cross filesystem boundaries then. However, to ensure consistent -bidirectional behavior we need to also ban them when operating downwards -the tree. -The directories and files on different filesystems need to be ignored -explicitly as implicitly skipping them would cause confusion. -In particular, tools might then claim that a file does not exist when -it clearly does because it was skipped due to filesystem boundaries. +Cross-filesystem Manifests +-------------------------- + +The first version of this specification had an additional requirement +that all files covered by the Manifest tree must reside on a single +filesystem. This requirement has been removed in version 1.1 for +the reasons outlined in this section. + +The original rationale stated that this restriction aims to prevent +crossing filesystem boundaries in the top-level Manifest lookup +algorithm. While that seemed a good idea at the time, there is no real +reason to prevent that and this particular method worked correctly only +if the files were placed in a dedicated filesystem. + +Worse than that, the original rationale did not anticipate the use +of overlayfs which combines multiple filesystems while preserving their +original metadata, including device and inode numbers. As a result, +if the repository was checked out to an overlayfs, it was quite possible +that different files had different device numbers, and the Manifest +checks failed due to crossing filesystem boundaries. + +Given no clear solution to that and no good reason to reject use +of overlayfs, the restriction was lifted. + +The only potential drawback of this is that the implementation may now +follow maliciously placed symbolic links pointing outside the tree. +If a regular file was replaced by such a symlink, the user could +be tricked into reporting the verification failure with the report +containing the checksums of the target file. However, for this to happen +the client would have to use rsync with ``--links`` option but without +``--safe-links`` which is neither the default behavior of rsync nor +the default configuration used by Portage. Filename character set restriction