* [gentoo-portage-dev] [PATCH] Check for and report read-only filesystems
@ 2014-01-11 3:07 Chris Reffett
2014-01-11 5:09 ` Brian Dolbec
2014-01-13 23:24 ` Mike Frysinger
0 siblings, 2 replies; 17+ messages in thread
From: Chris Reffett @ 2014-01-11 3:07 UTC (permalink / raw
To: gentoo-portage-dev
[-- Attachment #1: Type: text/plain, Size: 950 bytes --]
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
Hi all,
Attached is a patch to test if Portage is going to write to a
read-only filesystem and print out the list of filesystems that need
to be remounted RW. This leaves ${D} intact rather than having some
files moved before hitting the RO filesystem. Fixes bug 378869. Since
git.overlays.gentoo.org is down, I haven't had the chance to rebase
this against latest, but I can resubmit if it doesn't cleanly apply.
This is my first patch to the list, so I apologize if I didn't submit
correctly.
Chris Reffett
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.22 (GNU/Linux)
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/
iKYEARECAGYFAlLQtYhfFIAAAAAALgAoaXNzdWVyLWZwckBub3RhdGlvbnMub3Bl
bnBncC5maWZ0aGhvcnNlbWFuLm5ldEM2NzU5RjUyMDczREJDQkVDQTBDRkE1NERC
Nzk1QThBNDI2MTgzNTQACgkQ23laikJhg1SCCgCfZIo57KiijmACnDTa2vC9UMAb
9YwAoIhnc/nHDcIXBbzF9Tie3eTJyWpH
=j/1A
-----END PGP SIGNATURE-----
[-- Attachment #2: 0001-Test-for-read-only-filesystems-bail-out-during-prein.patch --]
[-- Type: text/x-patch, Size: 4573 bytes --]
From c464ac5e0007a087def9a96efea9653e508e57c1 Mon Sep 17 00:00:00 2001
From: Chris Reffett <creffett@gentoo.org>
Date: Fri, 10 Jan 2014 09:03:26 -0500
Subject: [PATCH] Test for read-only filesystems, bail out during preinst if
there are any which will be written to and display a useful error message.
Fixes bug 378869.
---
pym/portage/dbapi/vartree.py | 31 ++++++++++++++++++++++++
pym/portage/util/checkwriteable.py | 49 ++++++++++++++++++++++++++++++++++++++
2 files changed, 80 insertions(+)
create mode 100644 pym/portage/util/checkwriteable.py
diff --git a/pym/portage/dbapi/vartree.py b/pym/portage/dbapi/vartree.py
index ed62323..8ae7014 100644
--- a/pym/portage/dbapi/vartree.py
+++ b/pym/portage/dbapi/vartree.py
@@ -28,6 +28,7 @@ portage.proxy.lazyimport.lazyimport(globals(),
'portage.util:apply_secpass_permissions,ConfigProtect,ensure_dirs,' + \
'writemsg,writemsg_level,write_atomic,atomic_ofstream,writedict,' + \
'grabdict,normalize_path,new_protect_filename',
+ 'portage.util.checkwriteable:check_dirs_writeable',
'portage.util.digraph:digraph',
'portage.util.env_update:env_update',
'portage.util.listdir:dircache,listdir',
@@ -3508,6 +3509,8 @@ class dblink(object):
This function does the following:
+ calls check_dirs_writeable to verify that no files will be
+ written to read-only filesystems
calls self._preserve_libs if FEATURES=preserve-libs
calls self._collision_protect if FEATURES=collision-protect
calls doebuild(mydo=pkg_preinst)
@@ -3685,6 +3688,7 @@ class dblink(object):
eagain_error = False
myfilelist = []
+ mydirlist = []
mylinklist = []
paths_with_newlines = []
def onerror(e):
@@ -3717,6 +3721,9 @@ class dblink(object):
unicode_errors.append(new_parent[ed_len:])
break
+ relative_path = parent[srcroot_len:]
+ mydirlist.append("/" + relative_path)
+
for fname in files:
try:
fname = _unicode_decode(fname,
@@ -3829,6 +3836,30 @@ class dblink(object):
for other in others_in_slot])
prepare_build_dirs(settings=self.settings, cleanup=cleanup)
+ # Check for read-only filesystems
+ rofilesystems = check_dirs_writeable(mydirlist)
+
+ if rofilesystems:
+ msg = _("One or more files installed to this package are "
+ "set to be installed to read-only filesystems. "
+ "Please mount the filesystems below read-write "
+ "and retry.")
+ msg = textwrap.wrap(msg, 70)
+ msg.append("")
+ for f in rofilesystems:
+ msg.append("\t%s" % os.path.join(destroot,
+ f.lstrip(os.path.sep)))
+ msg.append("")
+ self._elog("eerror", "preinst", msg)
+
+ msg = _("Package '%s' NOT merged due to read-only file systems.") % \
+ self.settings.mycpv
+ msg += _(" If necessary, refer to your elog "
+ "messages for the whole content of the above message.")
+ msg = textwrap.wrap(msg, 70)
+ eerror(msg)
+ return 1
+
# check for package collisions
blockers = self._blockers
if blockers is None:
diff --git a/pym/portage/util/checkwriteable.py b/pym/portage/util/checkwriteable.py
new file mode 100644
index 0000000..9bd9056
--- /dev/null
+++ b/pym/portage/util/checkwriteable.py
@@ -0,0 +1,49 @@
+#-*- coding:utf-8 -*-
+# Copyright 2014 Gentoo Foundation
+# Distributed under the terms of the GNU General Public License v2
+
+from __future__ import unicode_literals
+
+import re
+
+from portage.util import writemsg
+from portage.localization import _
+
+
+def check_dirs_writeable(dir_list):
+ """
+ Use /proc/mounts to check that no directories installed by the ebuild are set to
+ be installed to a read-only filesystem
+
+ @param dir_list: Directories installed by the ebuild
+ @type dir_list: List
+ @return:
+ 1. A list of filesystems which are both set to be written to and are mounted
+ read-only, may be empty.
+ """
+ ro_filesystems = set()
+ ro_filesystems_written = set()
+
+ try:
+ with open("/proc/mounts") as procmounts:
+ roregex = re.compile(r'(\A|,)ro(\Z|,)')
+ for line in procmounts:
+ if roregex.search(line.split(" ")[3].strip()) is not None:
+ romount = line.split(" ")[1].strip()
+ ro_filesystems.add(romount)
+
+ # If /proc/mounts can't be read, assume that there are no RO
+ # filesystems and return
+ except EnvironmentError:
+ writemsg(_("!!! /proc/mounts cannot be read"))
+ return []
+
+ if not ro_filesystems:
+ return ro_filesystems
+
+ for directory in dir_list:
+ for filesystem in ro_filesystems:
+ if re.match(filesystem, directory):
+ ro_filesystems_written.add(filesystem)
+
+ return ro_filesystems_written
--
1.8.5.1
^ permalink raw reply related [flat|nested] 17+ messages in thread
* Re: [gentoo-portage-dev] [PATCH] Check for and report read-only filesystems
2014-01-11 3:07 [gentoo-portage-dev] [PATCH] Check for and report read-only filesystems Chris Reffett
@ 2014-01-11 5:09 ` Brian Dolbec
2014-01-11 14:22 ` Chris Reffett
2014-01-13 23:24 ` Mike Frysinger
1 sibling, 1 reply; 17+ messages in thread
From: Brian Dolbec @ 2014-01-11 5:09 UTC (permalink / raw
To: gentoo-portage-dev
[-- Attachment #1: Type: text/plain, Size: 867 bytes --]
On Fri, 2014-01-10 at 22:07 -0500, Chris Reffett wrote:
> Hi all,
> Attached is a patch to test if Portage is going to write to a
> read-only filesystem and print out the list of filesystems that need
> to be remounted RW. This leaves ${D} intact rather than having some
> files moved before hitting the RO filesystem. Fixes bug 378869. Since
> git.overlays.gentoo.org is down, I haven't had the chance to rebase
> this against latest, but I can resubmit if it doesn't cleanly apply.
> This is my first patch to the list, so I apologize if I didn't submit
> correctly.
>
> Chris Reffett
yeah, patch looks good.
Only thing I didn't like is the return 1 IS that suppose to be True or
sys.exit() value?
If that is what the module was using, then it's ok. Personally I'm not
a fan of using 0, 1 for False, True.
But that will come later...
[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 620 bytes --]
^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: [gentoo-portage-dev] [PATCH] Check for and report read-only filesystems
2014-01-11 5:09 ` Brian Dolbec
@ 2014-01-11 14:22 ` Chris Reffett
0 siblings, 0 replies; 17+ messages in thread
From: Chris Reffett @ 2014-01-11 14:22 UTC (permalink / raw
To: gentoo-portage-dev
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
On 01/11/2014 12:09 AM, Brian Dolbec wrote:
> On Fri, 2014-01-10 at 22:07 -0500, Chris Reffett wrote:
>> Hi all, Attached is a patch to test if Portage is going to write
>> to a read-only filesystem and print out the list of filesystems
>> that need to be remounted RW. This leaves ${D} intact rather than
>> having some files moved before hitting the RO filesystem. Fixes
>> bug 378869. Since git.overlays.gentoo.org is down, I haven't had
>> the chance to rebase this against latest, but I can resubmit if
>> it doesn't cleanly apply. This is my first patch to the list, so
>> I apologize if I didn't submit correctly.
>>
>> Chris Reffett
>
>
> yeah, patch looks good.
>
> Only thing I didn't like is the return 1 IS that suppose to be
> True or sys.exit() value?
>
> If that is what the module was using, then it's ok. Personally I'm
> not a fan of using 0, 1 for False, True.
>
> But that will come later...
>
That was just following the style of the rest of the module, for
example a collision will return 1. This can be added to the "stuff to
be fixed up in future patches" list.
Chris Reffett
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.22 (GNU/Linux)
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/
iKUEARECAGYFAlLRU7VfFIAAAAAALgAoaXNzdWVyLWZwckBub3RhdGlvbnMub3Bl
bnBncC5maWZ0aGhvcnNlbWFuLm5ldEM2NzU5RjUyMDczREJDQkVDQTBDRkE1NERC
Nzk1QThBNDI2MTgzNTQACgkQ23laikJhg1QLjQCfSJSpacHoI/IQPS/o+NFJvP6q
d8YAmP+RmhoWwa3J1eRNk0BAxX1TtDg=
=a7If
-----END PGP SIGNATURE-----
^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: [gentoo-portage-dev] [PATCH] Check for and report read-only filesystems
2014-01-11 3:07 [gentoo-portage-dev] [PATCH] Check for and report read-only filesystems Chris Reffett
2014-01-11 5:09 ` Brian Dolbec
@ 2014-01-13 23:24 ` Mike Frysinger
2014-01-17 0:39 ` [gentoo-portage-dev] [PATCH v2] Test for read-only filesystems, bail out during preinst if there are any which will be written to and display a useful error message. Fixes bug 378869 Chris Reffett
1 sibling, 1 reply; 17+ messages in thread
From: Mike Frysinger @ 2014-01-13 23:24 UTC (permalink / raw
To: gentoo-portage-dev; +Cc: Chris Reffett
[-- Attachment #1: Type: Text/Plain, Size: 1718 bytes --]
On Friday 10 January 2014 22:07:52 Chris Reffett wrote:
> Attached is a patch to test if Portage is going to write to a
> read-only filesystem and print out the list of filesystems that need
> to be remounted RW. This leaves ${D} intact rather than having some
> files moved before hitting the RO filesystem. Fixes bug 378869. Since
> git.overlays.gentoo.org is down, I haven't had the chance to rebase
> this against latest, but I can resubmit if it doesn't cleanly apply.
> This is my first patch to the list, so I apologize if I didn't submit
> correctly.
please use `git send-email` to post patches. attaching them makes things
harder to review.
> --- /dev/null
> +++ b/pym/portage/util/checkwriteable.py
>
> + with open("/proc/mounts") as procmounts:
this won't work on non-Linux systems
also, as a general style thing, unless there's a real need for the var to have
a "full" name, just use "f" or "fp"
> + roregex = re.compile(r'(\A|,)ro(\Z|,)')
> + for line in procmounts:
> + if roregex.search(line.split(" ")[3].strip()) is not None:
> + romount = line.split(" ")[1].strip()
> + ro_filesystems.add(romount)
ad hoc parsing of /proc/mounts isn't a good thing. at the risk of over
engineering, there should be a func that would take care of expanding the
mount paths into a list of namedcollections (one per mount). then you simply
walk it looking at its options.
> + for directory in dir_list:
> + for filesystem in ro_filesystems:
> + if re.match(filesystem, directory):
> + ro_filesystems_written.add(filesystem)
there's no need to use re and in fact you don't want to. you're looking at
wrong strings here, not regexes.
-mike
[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 836 bytes --]
^ permalink raw reply [flat|nested] 17+ messages in thread
* [gentoo-portage-dev] [PATCH v2] Test for read-only filesystems, bail out during preinst if there are any which will be written to and display a useful error message. Fixes bug 378869.
2014-01-13 23:24 ` Mike Frysinger
@ 2014-01-17 0:39 ` Chris Reffett
2014-01-19 2:00 ` [gentoo-portage-dev] [PATCH v3] " Chris Reffett
0 siblings, 1 reply; 17+ messages in thread
From: Chris Reffett @ 2014-01-17 0:39 UTC (permalink / raw
To: gentoo-portage-dev
v2: Reformat, add a function to return an appropriate read-only checker
for the operating system, so that this can be extended to other OSes.
---
pym/portage/dbapi/vartree.py | 32 +++++++++++++++++
pym/portage/util/rochecker.py | 81 +++++++++++++++++++++++++++++++++++++++++++
2 files changed, 113 insertions(+)
create mode 100644 pym/portage/util/rochecker.py
diff --git a/pym/portage/dbapi/vartree.py b/pym/portage/dbapi/vartree.py
index ed62323..c77cf07 100644
--- a/pym/portage/dbapi/vartree.py
+++ b/pym/portage/dbapi/vartree.py
@@ -32,6 +32,7 @@ portage.proxy.lazyimport.lazyimport(globals(),
'portage.util.env_update:env_update',
'portage.util.listdir:dircache,listdir',
'portage.util.movefile:movefile',
+ 'portage.util.rochecker:get_ro_checker',
'portage.util._dyn_libs.PreservedLibsRegistry:PreservedLibsRegistry',
'portage.util._dyn_libs.LinkageMapELF:LinkageMapELF@LinkageMap',
'portage.util._async.SchedulerInterface:SchedulerInterface',
@@ -3508,6 +3509,8 @@ class dblink(object):
This function does the following:
+ calls get_ro_checker to retrieve a function for checking whether Portage
+ will write to a read-only filesystem, then runs it against the directory list
calls self._preserve_libs if FEATURES=preserve-libs
calls self._collision_protect if FEATURES=collision-protect
calls doebuild(mydo=pkg_preinst)
@@ -3685,6 +3688,7 @@ class dblink(object):
eagain_error = False
myfilelist = []
+ mydirlist = []
mylinklist = []
paths_with_newlines = []
def onerror(e):
@@ -3717,6 +3721,9 @@ class dblink(object):
unicode_errors.append(new_parent[ed_len:])
break
+ relative_path = parent[srcroot_len:]
+ mydirlist.append("/" + relative_path)
+
for fname in files:
try:
fname = _unicode_decode(fname,
@@ -3829,6 +3836,31 @@ class dblink(object):
for other in others_in_slot])
prepare_build_dirs(settings=self.settings, cleanup=cleanup)
+ # Check for read-only filesystems
+ ro_checker_func = get_ro_checker()
+ rofilesystems = ro_checker_func(mydirlist)
+
+ if rofilesystems:
+ msg = _("One or more files installed to this package are "
+ "set to be installed to read-only filesystems. "
+ "Please mount the filesystems below read-write "
+ "and retry.")
+ msg = textwrap.wrap(msg, 70)
+ msg.append("")
+ for f in rofilesystems:
+ msg.append("\t%s" % os.path.join(destroot,
+ f.lstrip(os.path.sep)))
+ msg.append("")
+ self._elog("eerror", "preinst", msg)
+
+ msg = _("Package '%s' NOT merged due to read-only file systems.") % \
+ self.settings.mycpv
+ msg += _(" If necessary, refer to your elog "
+ "messages for the whole content of the above message.")
+ msg = textwrap.wrap(msg, 70)
+ eerror(msg)
+ return 1
+
# check for package collisions
blockers = self._blockers
if blockers is None:
diff --git a/pym/portage/util/rochecker.py b/pym/portage/util/rochecker.py
new file mode 100644
index 0000000..67e8131
--- /dev/null
+++ b/pym/portage/util/rochecker.py
@@ -0,0 +1,81 @@
+#-*- coding:utf-8 -*-
+# Copyright 2014 Gentoo Foundation
+# Distributed under the terms of the GNU General Public License v2
+"""
+Methods to check whether Portage is going to write to read-only filesystems.
+Since the methods are not portable across different OSes, each OS needs its
+own method. To expand RO checking for different OSes, add a method which
+accepts a list of directories and returns a list of mounts which need to be
+remounted RW, then add "elif ostype == (the ostype value for your OS)" to
+get_ro_checker()
+"""
+from __future__ import unicode_literals
+
+import logging
+import re
+
+from portage.util import writemsg_level
+from portage.localization import _
+from portage.data import ostype
+
+
+def get_ro_checker():
+ """
+ Uses the system type to find an appropriate method for testing whether Portage
+ is going to write to any read-only filesystems
+
+ @return:
+ 1. A method for testing for RO filesystems appropriate to the current system
+ """
+ if ostype == "Linux":
+ return linux_ro_checker
+ else:
+ return empty_ro_checker
+
+
+def linux_ro_checker(dir_list):
+ """
+ Use /proc/mounts to check that no directories installed by the ebuild are set
+ to be installed to a read-only filesystem
+
+ @param dir_list: Directories installed by the ebuild
+ @type dir_list: List
+ @return:
+ 1. A list of filesystems which are both set to be written to and are mounted
+ read-only, may be empty.
+ """
+ ro_filesystems = set()
+ ro_filesystems_written = set()
+
+ try:
+ with open("/proc/mounts") as f:
+ roregex = re.compile(r'(\A|,)ro(\Z|,)')
+ for line in f:
+ if roregex.search(line.split(" ")[3].strip()) is not None:
+ romount = line.split(" ")[1].strip()
+ ro_filesystems.add(romount)
+
+ # If /proc/mounts can't be read, assume that there are no RO
+ # filesystems and return
+ except EnvironmentError:
+ writemsg_level(_("!!! /proc/mounts cannot be read"),
+ level=logging.WARNING, noiselevel=-1)
+ return []
+
+ if not ro_filesystems:
+ return ro_filesystems
+
+ for directory in dir_list:
+ for filesystem in ro_filesystems:
+ if filesystem == directory:
+ ro_filesystems_written.add(filesystem)
+
+ return ro_filesystems_written
+
+
+def empty_ro_checker(dir_list):
+ """
+ Always returns [], this is the fallback function if the system does not have
+ an ro_checker method defined
+ """
+ return []
--
1.8.5.1
^ permalink raw reply related [flat|nested] 17+ messages in thread
* [gentoo-portage-dev] [PATCH v3] Test for read-only filesystems, bail out during preinst if there are any which will be written to and display a useful error message. Fixes bug 378869.
2014-01-17 0:39 ` [gentoo-portage-dev] [PATCH v2] Test for read-only filesystems, bail out during preinst if there are any which will be written to and display a useful error message. Fixes bug 378869 Chris Reffett
@ 2014-01-19 2:00 ` Chris Reffett
2014-01-19 9:17 ` Mike Frysinger
0 siblings, 1 reply; 17+ messages in thread
From: Chris Reffett @ 2014-01-19 2:00 UTC (permalink / raw
To: gentoo-portage-dev
v2: Reformat, add a function to return an appropriate read-only checker
for the operating system, so that this can be extended to other OSes.
v3: minor formatting tweaks from bernalex, including use os.path.join()
instead of string concatenation
---
pym/portage/dbapi/vartree.py | 32 +++++++++++++++++
pym/portage/util/rochecker.py | 81 +++++++++++++++++++++++++++++++++++++++++++
2 files changed, 113 insertions(+)
create mode 100644 pym/portage/util/rochecker.py
diff --git a/pym/portage/dbapi/vartree.py b/pym/portage/dbapi/vartree.py
index ed62323..917634f 100644
--- a/pym/portage/dbapi/vartree.py
+++ b/pym/portage/dbapi/vartree.py
@@ -32,6 +32,7 @@ portage.proxy.lazyimport.lazyimport(globals(),
'portage.util.env_update:env_update',
'portage.util.listdir:dircache,listdir',
'portage.util.movefile:movefile',
+ 'portage.util.rochecker:get_ro_checker',
'portage.util._dyn_libs.PreservedLibsRegistry:PreservedLibsRegistry',
'portage.util._dyn_libs.LinkageMapELF:LinkageMapELF@LinkageMap',
'portage.util._async.SchedulerInterface:SchedulerInterface',
@@ -3508,6 +3509,8 @@ class dblink(object):
This function does the following:
+ calls get_ro_checker to retrieve a function for checking whether Portage
+ will write to a read-only filesystem, then runs it against the directory list
calls self._preserve_libs if FEATURES=preserve-libs
calls self._collision_protect if FEATURES=collision-protect
calls doebuild(mydo=pkg_preinst)
@@ -3685,6 +3688,7 @@ class dblink(object):
eagain_error = False
myfilelist = []
+ mydirlist = []
mylinklist = []
paths_with_newlines = []
def onerror(e):
@@ -3717,6 +3721,9 @@ class dblink(object):
unicode_errors.append(new_parent[ed_len:])
break
+ relative_path = parent[srcroot_len:]
+ mydirlist.append(os.path.join("/", relative_path))
+
for fname in files:
try:
fname = _unicode_decode(fname,
@@ -3829,6 +3836,31 @@ class dblink(object):
for other in others_in_slot])
prepare_build_dirs(settings=self.settings, cleanup=cleanup)
+ # Check for read-only filesystems
+ ro_checker = get_ro_checker()
+ rofilesystems = ro_checker(mydirlist)
+
+ if rofilesystems:
+ msg = _("One or more files installed to this package are "
+ "set to be installed to read-only filesystems. "
+ "Please mount the following filesystems as read-write "
+ "and retry.")
+ msg = textwrap.wrap(msg, 70)
+ msg.append("")
+ for f in rofilesystems:
+ msg.append("\t%s" % os.path.join(destroot,
+ f.lstrip(os.path.sep)))
+ msg.append("")
+ self._elog("eerror", "preinst", msg)
+
+ msg = _("Package '%s' NOT merged due to read-only file systems.") % \
+ self.settings.mycpv
+ msg += _(" If necessary, refer to your elog "
+ "messages for the whole content of the above message.")
+ msg = textwrap.wrap(msg, 70)
+ eerror(msg)
+ return 1
+
# check for package collisions
blockers = self._blockers
if blockers is None:
diff --git a/pym/portage/util/rochecker.py b/pym/portage/util/rochecker.py
new file mode 100644
index 0000000..67e8131
--- /dev/null
+++ b/pym/portage/util/rochecker.py
@@ -0,0 +1,81 @@
+#-*- coding:utf-8 -*-
+# Copyright 2014 Gentoo Foundation
+# Distributed under the terms of the GNU General Public License v2
+"""
+Methods to check whether Portage is going to write to read-only filesystems.
+Since the methods are not portable across different OSes, each OS needs its
+own method. To expand RO checking for different OSes, add a method which
+accepts a list of directories and returns a list of mounts which need to be
+remounted RW, then add "elif ostype == (the ostype value for your OS)" to
+get_ro_checker()
+"""
+from __future__ import unicode_literals
+
+import logging
+import re
+
+from portage.util import writemsg_level
+from portage.localization import _
+from portage.data import ostype
+
+
+def get_ro_checker():
+ """
+ Uses the system type to find an appropriate method for testing whether Portage
+ is going to write to any read-only filesystems
+
+ @return:
+ 1. A method for testing for RO filesystems appropriate to the current system
+ """
+ if ostype == "Linux":
+ return linux_ro_checker
+ else:
+ return empty_ro_checker
+
+
+def linux_ro_checker(dir_list):
+ """
+ Use /proc/mounts to check that no directories installed by the ebuild are set
+ to be installed to a read-only filesystem
+
+ @param dir_list: Directories installed by the ebuild
+ @type dir_list: List
+ @return:
+ 1. A list of filesystems which are both set to be written to and are mounted
+ read-only, may be empty.
+ """
+ ro_filesystems = set()
+ ro_filesystems_written = set()
+
+ try:
+ with open("/proc/mounts") as f:
+ roregex = re.compile(r'(\A|,)ro(\Z|,)')
+ for line in f:
+ if roregex.search(line.split(" ")[3].strip()) is not None:
+ romount = line.split(" ")[1].strip()
+ ro_filesystems.add(romount)
+
+ # If /proc/mounts can't be read, assume that there are no RO
+ # filesystems and return
+ except EnvironmentError:
+ writemsg_level(_("!!! /proc/mounts cannot be read"),
+ level=logging.WARNING, noiselevel=-1)
+ return []
+
+ if not ro_filesystems:
+ return ro_filesystems
+
+ for directory in dir_list:
+ for filesystem in ro_filesystems:
+ if filesystem == directory:
+ ro_filesystems_written.add(filesystem)
+
+ return ro_filesystems_written
+
+
+def empty_ro_checker(dir_list):
+ """
+ Always returns [], this is the fallback function if the system does not have
+ an ro_checker method defined
+ """
+ return []
--
1.8.5.1
^ permalink raw reply related [flat|nested] 17+ messages in thread
* Re: [gentoo-portage-dev] [PATCH v3] Test for read-only filesystems, bail out during preinst if there are any which will be written to and display a useful error message. Fixes bug 378869.
2014-01-19 2:00 ` [gentoo-portage-dev] [PATCH v3] " Chris Reffett
@ 2014-01-19 9:17 ` Mike Frysinger
2014-01-19 10:39 ` Alexander Berntsen
2014-01-21 0:50 ` [gentoo-portage-dev] [PATCH v4] " Chris Reffett
0 siblings, 2 replies; 17+ messages in thread
From: Mike Frysinger @ 2014-01-19 9:17 UTC (permalink / raw
To: gentoo-portage-dev; +Cc: Chris Reffett
[-- Attachment #1: Type: Text/Plain, Size: 1442 bytes --]
On Saturday 18 January 2014 21:00:48 Chris Reffett wrote:
> --- /dev/null
> +++ b/pym/portage/util/rochecker.py
> @@ -0,0 +1,81 @@
> +#-*- coding:utf-8 -*-
> +# Copyright 2014 Gentoo Foundation
> +# Distributed under the terms of the GNU General Public License v2
> +"""
> +Methods to check whether Portage is going to write to read-only
> filesystems. +Since the methods are not portable across different OSes,
> each OS needs its +own method. To expand RO checking for different OSes,
> add a method which +accepts a list of directories and returns a list of
> mounts which need to be +remounted RW, then add "elif ostype == (the
> ostype value for your OS)" to
prefer OSes -> OS's
> +get_ro_checker()
> +"""
period at the end
> +def get_ro_checker():
> + """
> + Uses the system type to find an appropriate method for testing whether
> Portage
> + is going to write to any read-only filesystems
> +
> + @return:
> + 1. A method for testing for RO filesystems appropriate to the current
> system
> + """
not a new issue, but we really need to get away from this style and use
PEP257. someone feel like fixing that in the code base ? :)
> + if ostype == "Linux":
> + return linux_ro_checker
> + else:
> + return empty_ro_checker
isn't this a glorified dict ?
_CHECKERS = {
'Linux': linux_ro_checker,
}
def get_ro_checker()
return _CHECKERS.get(ostype, empty_ro_checker)
-mike
[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 836 bytes --]
^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: [gentoo-portage-dev] [PATCH v3] Test for read-only filesystems, bail out during preinst if there are any which will be written to and display a useful error message. Fixes bug 378869.
2014-01-19 9:17 ` Mike Frysinger
@ 2014-01-19 10:39 ` Alexander Berntsen
2014-01-19 21:43 ` Alec Warner
2014-01-21 3:28 ` Mike Frysinger
2014-01-21 0:50 ` [gentoo-portage-dev] [PATCH v4] " Chris Reffett
1 sibling, 2 replies; 17+ messages in thread
From: Alexander Berntsen @ 2014-01-19 10:39 UTC (permalink / raw
To: gentoo-portage-dev
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256
On 19/01/14 10:17, Mike Frysinger wrote:
> prefer OSes -> OS's
That's just not proper English. It makes no sense. Please don't prefer
that. If for nothing else, then to prevent me from sighing whenever I
have to read it.
> not a new issue, but we really need to get away from this style and
> use PEP257. someone feel like fixing that in the code base ? :)
I agree. If you give me at least a week (won't have time until next
weekend), I can look into it.
- --
Alexander
alexander@plaimi.net
http://plaimi.net/~alexander
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.22 (GNU/Linux)
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/
iF4EAREIAAYFAlLbq10ACgkQRtClrXBQc7U7MAD/Yl2pd0ood3UKmvNMg9gdWMsw
hQqnReHPRwLpDNRccOcA/jUODDanycVmNLpD/rjkYnKyllnbUSIxduYuMcJFqXme
=vaDs
-----END PGP SIGNATURE-----
^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: [gentoo-portage-dev] [PATCH v3] Test for read-only filesystems, bail out during preinst if there are any which will be written to and display a useful error message. Fixes bug 378869.
2014-01-19 10:39 ` Alexander Berntsen
@ 2014-01-19 21:43 ` Alec Warner
2014-01-21 3:28 ` Mike Frysinger
1 sibling, 0 replies; 17+ messages in thread
From: Alec Warner @ 2014-01-19 21:43 UTC (permalink / raw
To: gentoo-portage-dev
[-- Attachment #1: Type: text/plain, Size: 1568 bytes --]
On Sun, Jan 19, 2014 at 2:39 AM, Alexander Berntsen <alexander@plaimi.net>wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA256
>
> On 19/01/14 10:17, Mike Frysinger wrote:
> > prefer OSes -> OS's
> That's just not proper English. It makes no sense. Please don't prefer
> that. If for nothing else, then to prevent me from sighing whenever I
> have to read it.
>
The bikeshed should be red! Be happy in the fact that this bit has no
significant bearing on the code and just pick one.
> > not a new issue, but we really need to get away from this style and
> > use PEP257. someone feel like fixing that in the code base ? :)
> I agree. If you give me at least a week (won't have time until next
> weekend), I can look into it.
>
Sorry, my fault. I added docstrings in 2006 (looks like the first one was
vercmp in
http://git.overlays.gentoo.org/gitweb/?p=proj/portage.git;a=blobdiff;f=pym/portage_versions.py;h=8e58c2f47766c0ed6c007c0cba1fccebca7d4eb4;hp=ec01773ad199a87d7906eb3862a14b9c8102fc29;hb=cfc497009e6054a5403461be676bb94deebbbf50;hpb=53f14db70339391fb13ca0adab353645f84be3f2)
I'm happy for these to change. ;p
-A
> - --
Alexander
> alexander@plaimi.net
> http://plaimi.net/~alexander
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v2.0.22 (GNU/Linux)
> Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/
>
> iF4EAREIAAYFAlLbq10ACgkQRtClrXBQc7U7MAD/Yl2pd0ood3UKmvNMg9gdWMsw
> hQqnReHPRwLpDNRccOcA/jUODDanycVmNLpD/rjkYnKyllnbUSIxduYuMcJFqXme
> =vaDs
> -----END PGP SIGNATURE-----
>
>
[-- Attachment #2: Type: text/html, Size: 3117 bytes --]
^ permalink raw reply [flat|nested] 17+ messages in thread
* [gentoo-portage-dev] [PATCH v4] Test for read-only filesystems, bail out during preinst if there are any which will be written to and display a useful error message. Fixes bug 378869.
2014-01-19 9:17 ` Mike Frysinger
2014-01-19 10:39 ` Alexander Berntsen
@ 2014-01-21 0:50 ` Chris Reffett
2014-01-21 3:32 ` Mike Frysinger
1 sibling, 1 reply; 17+ messages in thread
From: Chris Reffett @ 2014-01-21 0:50 UTC (permalink / raw
To: gentoo-portage-dev
v2: Reformat, add a function to return an appropriate read-only checker
for the operating system, so that this can be extended to other OSes.
v3: minor formatting tweaks from bernalex, including use os.path.join()
instead of string concatenation
v4: Update copyright header, change the code in rochecker to open with
io.open in order to not break on non-ascii characters as reported by
Arfrever. Change get_ro_checker to use a dict as suggested by vapier.
---
pym/portage/dbapi/vartree.py | 34 ++++++++++++++++-
pym/portage/util/rochecker.py | 87 +++++++++++++++++++++++++++++++++++++++++++
2 files changed, 120 insertions(+), 1 deletion(-)
create mode 100644 pym/portage/util/rochecker.py
diff --git a/pym/portage/dbapi/vartree.py b/pym/portage/dbapi/vartree.py
index ed62323..22cf222 100644
--- a/pym/portage/dbapi/vartree.py
+++ b/pym/portage/dbapi/vartree.py
@@ -1,4 +1,4 @@
-# Copyright 1998-2013 Gentoo Foundation
+# Copyright 1998-2014 Gentoo Foundation
# Distributed under the terms of the GNU General Public License v2
from __future__ import unicode_literals
@@ -32,6 +32,7 @@ portage.proxy.lazyimport.lazyimport(globals(),
'portage.util.env_update:env_update',
'portage.util.listdir:dircache,listdir',
'portage.util.movefile:movefile',
+ 'portage.util.rochecker:get_ro_checker',
'portage.util._dyn_libs.PreservedLibsRegistry:PreservedLibsRegistry',
'portage.util._dyn_libs.LinkageMapELF:LinkageMapELF@LinkageMap',
'portage.util._async.SchedulerInterface:SchedulerInterface',
@@ -3508,6 +3509,8 @@ class dblink(object):
This function does the following:
+ calls get_ro_checker to retrieve a function for checking whether Portage
+ will write to a read-only filesystem, then runs it against the directory list
calls self._preserve_libs if FEATURES=preserve-libs
calls self._collision_protect if FEATURES=collision-protect
calls doebuild(mydo=pkg_preinst)
@@ -3685,6 +3688,7 @@ class dblink(object):
eagain_error = False
myfilelist = []
+ mydirlist = []
mylinklist = []
paths_with_newlines = []
def onerror(e):
@@ -3717,6 +3721,9 @@ class dblink(object):
unicode_errors.append(new_parent[ed_len:])
break
+ relative_path = parent[srcroot_len:]
+ mydirlist.append(os.path.join("/", relative_path))
+
for fname in files:
try:
fname = _unicode_decode(fname,
@@ -3829,6 +3836,31 @@ class dblink(object):
for other in others_in_slot])
prepare_build_dirs(settings=self.settings, cleanup=cleanup)
+ # Check for read-only filesystems
+ ro_checker = get_ro_checker()
+ rofilesystems = ro_checker(mydirlist)
+
+ if rofilesystems:
+ msg = _("One or more files installed to this package are "
+ "set to be installed to read-only filesystems. "
+ "Please mount the following filesystems as read-write "
+ "and retry.")
+ msg = textwrap.wrap(msg, 70)
+ msg.append("")
+ for f in rofilesystems:
+ msg.append("\t%s" % os.path.join(destroot,
+ f.lstrip(os.path.sep)))
+ msg.append("")
+ self._elog("eerror", "preinst", msg)
+
+ msg = _("Package '%s' NOT merged due to read-only file systems.") % \
+ self.settings.mycpv
+ msg += _(" If necessary, refer to your elog "
+ "messages for the whole content of the above message.")
+ msg = textwrap.wrap(msg, 70)
+ eerror(msg)
+ return 1
+
# check for package collisions
blockers = self._blockers
if blockers is None:
diff --git a/pym/portage/util/rochecker.py b/pym/portage/util/rochecker.py
new file mode 100644
index 0000000..2bbfe9d
--- /dev/null
+++ b/pym/portage/util/rochecker.py
@@ -0,0 +1,87 @@
+#-*- coding:utf-8 -*-
+# Copyright 2014 Gentoo Foundation
+# Distributed under the terms of the GNU General Public License v2
+"""
+Methods to check whether Portage is going to write to read-only filesystems.
+Since the methods are not portable across different OSes, each OS needs its
+own method. To expand RO checking for different OSes, add a method which
+accepts a list of directories and returns a list of mounts which need to be
+remounted RW, then add "elif ostype == (the ostype value for your OS)" to
+get_ro_checker().
+"""
+from __future__ import unicode_literals
+
+import io
+import logging
+import re
+
+from portage import _encodings
+from portage.util import writemsg_level
+from portage.localization import _
+from portage.data import ostype
+
+
+def get_ro_checker():
+ """
+ Uses the system type to find an appropriate method for testing whether Portage
+ is going to write to any read-only filesystems
+
+ @return:
+ 1. A method for testing for RO filesystems appropriate to the current system
+ """
+ return _CHECKERS.get(ostype, empty_ro_checker)
+
+
+def linux_ro_checker(dir_list):
+ """
+ Use /proc/mounts to check that no directories installed by the ebuild are set
+ to be installed to a read-only filesystem
+
+ @param dir_list: Directories installed by the ebuild
+ @type dir_list: List
+ @return:
+ 1. A list of filesystems which are both set to be written to and are mounted
+ read-only, may be empty.
+ """
+ ro_filesystems = set()
+ ro_filesystems_written = set()
+
+ try:
+ with io.open("/proc/mounts", mode='r', encoding=_encodings['content'],
+ errors='replace') as f:
+ roregex = re.compile(r'(\A|,)ro(\Z|,)')
+ for line in f:
+ if roregex.search(line.split(" ")[3].strip()) is not None:
+ romount = line.split(" ")[1].strip()
+ ro_filesystems.add(romount)
+
+ # If /proc/mounts can't be read, assume that there are no RO
+ # filesystems and return
+ except EnvironmentError:
+ writemsg_level(_("!!! /proc/mounts cannot be read"),
+ level=logging.WARNING, noiselevel=-1)
+ return []
+
+ if not ro_filesystems:
+ return ro_filesystems
+
+ for directory in dir_list:
+ for filesystem in ro_filesystems:
+ if filesystem == directory:
+ ro_filesystems_written.add(filesystem)
+
+ return ro_filesystems_written
+
+
+def empty_ro_checker(dir_list):
+ """
+ Always returns [], this is the fallback function if the system does not have
+ an ro_checker method defined
+ """
+ return []
+
+
+# A map from ostype output to the appropriate function to return in get_ro_checker
+_CHECKERS = {
+ "Linux": linux_ro_checker,
+}
--
1.8.5.1
^ permalink raw reply related [flat|nested] 17+ messages in thread
* Re: [gentoo-portage-dev] [PATCH v3] Test for read-only filesystems, bail out during preinst if there are any which will be written to and display a useful error message. Fixes bug 378869.
2014-01-19 10:39 ` Alexander Berntsen
2014-01-19 21:43 ` Alec Warner
@ 2014-01-21 3:28 ` Mike Frysinger
2014-01-21 3:53 ` Gordon Pettey
1 sibling, 1 reply; 17+ messages in thread
From: Mike Frysinger @ 2014-01-21 3:28 UTC (permalink / raw
To: gentoo-portage-dev; +Cc: Alexander Berntsen
[-- Attachment #1: Type: Text/Plain, Size: 574 bytes --]
On Sunday 19 January 2014 05:39:25 Alexander Berntsen wrote:
> On 19/01/14 10:17, Mike Frysinger wrote:
> > prefer OSes -> OS's
>
> That's just not proper English.
i don't think that phrase means what you think it means. if you're wishing
for English to be a standard, then you're in for a rude surprise.
> It makes no sense.
of course it does.
> Please don't prefer
> that. If for nothing else, then to prevent me from sighing whenever I
> have to read it.
https://en.wikipedia.org/wiki/English_plurals#Plurals_of_letters_and_abbreviations
-mike
[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 836 bytes --]
^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: [gentoo-portage-dev] [PATCH v4] Test for read-only filesystems, bail out during preinst if there are any which will be written to and display a useful error message. Fixes bug 378869.
2014-01-21 0:50 ` [gentoo-portage-dev] [PATCH v4] " Chris Reffett
@ 2014-01-21 3:32 ` Mike Frysinger
2014-01-21 4:00 ` [gentoo-portage-dev] [PATCH v5] " Chris Reffett
0 siblings, 1 reply; 17+ messages in thread
From: Mike Frysinger @ 2014-01-21 3:32 UTC (permalink / raw
To: gentoo-portage-dev
[-- Attachment #1: Type: Text/Plain, Size: 646 bytes --]
On Monday 20 January 2014 19:50:54 Chris Reffett wrote:
> + # Check for read-only filesystems
please use full sentences. that means putting a period at the end. this
applies to many additions in this patch.
> + try:
> + with io.open("/proc/mounts", mode='r',
encoding=_encodings['content'],
> + errors='replace') as f:
> + roregex = re.compile(r'(\A|,)ro(\Z|,)')
> + for line in f:
> + if roregex.search(line.split(" ")[3].strip()) is not
None:
> + romount = line.split(" ")[1].strip()
> + ro_filesystems.add(romount)
pretty sure the body of this with block is indented one too many times
-mike
[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 836 bytes --]
^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: [gentoo-portage-dev] [PATCH v3] Test for read-only filesystems, bail out during preinst if there are any which will be written to and display a useful error message. Fixes bug 378869.
2014-01-21 3:28 ` Mike Frysinger
@ 2014-01-21 3:53 ` Gordon Pettey
2014-01-21 22:44 ` Mike Frysinger
0 siblings, 1 reply; 17+ messages in thread
From: Gordon Pettey @ 2014-01-21 3:53 UTC (permalink / raw
To: gentoo-portage-dev
[-- Attachment #1: Type: text/plain, Size: 843 bytes --]
If your going to make that argumint, ewe mite a's well right the
documentation in LOL-1337. Encouraging bad grammar in documentation just
make's thing's harder for everybody.
On Mon, Jan 20, 2014 at 9:28 PM, Mike Frysinger <vapier@gentoo.org> wrote:
> On Sunday 19 January 2014 05:39:25 Alexander Berntsen wrote:
> > On 19/01/14 10:17, Mike Frysinger wrote:
> > > prefer OSes -> OS's
> >
> > That's just not proper English.
>
> i don't think that phrase means what you think it means. if you're wishing
> for English to be a standard, then you're in for a rude surprise.
>
> > It makes no sense.
>
> of course it does.
>
> > Please don't prefer
> > that. If for nothing else, then to prevent me from sighing whenever I
> > have to read it.
>
>
> https://en.wikipedia.org/wiki/English_plurals#Plurals_of_letters_and_abbreviations
> -mike
>
[-- Attachment #2: Type: text/html, Size: 1485 bytes --]
^ permalink raw reply [flat|nested] 17+ messages in thread
* [gentoo-portage-dev] [PATCH v5] Test for read-only filesystems, bail out during preinst if there are any which will be written to and display a useful error message. Fixes bug 378869.
2014-01-21 3:32 ` Mike Frysinger
@ 2014-01-21 4:00 ` Chris Reffett
2014-01-22 5:24 ` Mike Frysinger
0 siblings, 1 reply; 17+ messages in thread
From: Chris Reffett @ 2014-01-21 4:00 UTC (permalink / raw
To: gentoo-portage-dev
v2: Reformat, add a function to return an appropriate read-only checker
for the operating system, so that this can be extended to other OSes.
v3: minor formatting tweaks from bernalex, including use os.path.join()
instead of string concatenation
v4: Update copyright header, change the code in rochecker to open with
io.open in order to not break on non-ascii characters as reported by
Arfrever. Change get_ro_checker to use a dict as suggested by vapier.
v5: Fix comment format as requested by vapier.
---
pym/portage/dbapi/vartree.py | 34 ++++++++++++++++-
pym/portage/util/rochecker.py | 88 +++++++++++++++++++++++++++++++++++++++++++
2 files changed, 121 insertions(+), 1 deletion(-)
create mode 100644 pym/portage/util/rochecker.py
diff --git a/pym/portage/dbapi/vartree.py b/pym/portage/dbapi/vartree.py
index ed62323..cf6781b 100644
--- a/pym/portage/dbapi/vartree.py
+++ b/pym/portage/dbapi/vartree.py
@@ -1,4 +1,4 @@
-# Copyright 1998-2013 Gentoo Foundation
+# Copyright 1998-2014 Gentoo Foundation
# Distributed under the terms of the GNU General Public License v2
from __future__ import unicode_literals
@@ -32,6 +32,7 @@ portage.proxy.lazyimport.lazyimport(globals(),
'portage.util.env_update:env_update',
'portage.util.listdir:dircache,listdir',
'portage.util.movefile:movefile',
+ 'portage.util.rochecker:get_ro_checker',
'portage.util._dyn_libs.PreservedLibsRegistry:PreservedLibsRegistry',
'portage.util._dyn_libs.LinkageMapELF:LinkageMapELF@LinkageMap',
'portage.util._async.SchedulerInterface:SchedulerInterface',
@@ -3508,6 +3509,8 @@ class dblink(object):
This function does the following:
+ calls get_ro_checker to retrieve a function for checking whether Portage
+ will write to a read-only filesystem, then runs it against the directory list
calls self._preserve_libs if FEATURES=preserve-libs
calls self._collision_protect if FEATURES=collision-protect
calls doebuild(mydo=pkg_preinst)
@@ -3685,6 +3688,7 @@ class dblink(object):
eagain_error = False
myfilelist = []
+ mydirlist = []
mylinklist = []
paths_with_newlines = []
def onerror(e):
@@ -3717,6 +3721,9 @@ class dblink(object):
unicode_errors.append(new_parent[ed_len:])
break
+ relative_path = parent[srcroot_len:]
+ mydirlist.append(os.path.join("/", relative_path))
+
for fname in files:
try:
fname = _unicode_decode(fname,
@@ -3829,6 +3836,31 @@ class dblink(object):
for other in others_in_slot])
prepare_build_dirs(settings=self.settings, cleanup=cleanup)
+ # Check for read-only filesystems.
+ ro_checker = get_ro_checker()
+ rofilesystems = ro_checker(mydirlist)
+
+ if rofilesystems:
+ msg = _("One or more files installed to this package are "
+ "set to be installed to read-only filesystems. "
+ "Please mount the following filesystems as read-write "
+ "and retry.")
+ msg = textwrap.wrap(msg, 70)
+ msg.append("")
+ for f in rofilesystems:
+ msg.append("\t%s" % os.path.join(destroot,
+ f.lstrip(os.path.sep)))
+ msg.append("")
+ self._elog("eerror", "preinst", msg)
+
+ msg = _("Package '%s' NOT merged due to read-only file systems.") % \
+ self.settings.mycpv
+ msg += _(" If necessary, refer to your elog "
+ "messages for the whole content of the above message.")
+ msg = textwrap.wrap(msg, 70)
+ eerror(msg)
+ return 1
+
# check for package collisions
blockers = self._blockers
if blockers is None:
diff --git a/pym/portage/util/rochecker.py b/pym/portage/util/rochecker.py
new file mode 100644
index 0000000..2d3c89f
--- /dev/null
+++ b/pym/portage/util/rochecker.py
@@ -0,0 +1,88 @@
+#-*- coding:utf-8 -*-
+# Copyright 2014 Gentoo Foundation
+# Distributed under the terms of the GNU General Public License v2
+"""
+Methods to check whether Portage is going to write to read-only filesystems.
+Since the methods are not portable across different OSes, each OS needs its
+own method. To expand RO checking for different OSes, add a method which
+accepts a list of directories and returns a list of mounts which need to be
+remounted RW, then add "elif ostype == (the ostype value for your OS)" to
+get_ro_checker().
+"""
+from __future__ import unicode_literals
+
+import io
+import logging
+import re
+
+from portage import _encodings
+from portage.util import writemsg_level
+from portage.localization import _
+from portage.data import ostype
+
+
+def get_ro_checker():
+ """
+ Uses the system type to find an appropriate method for testing whether Portage
+ is going to write to any read-only filesystems.
+
+ @return:
+ 1. A method for testing for RO filesystems appropriate to the current system.
+ """
+ return _CHECKERS.get(ostype, empty_ro_checker)
+
+
+def linux_ro_checker(dir_list):
+ """
+ Use /proc/mounts to check that no directories installed by the ebuild are set
+ to be installed to a read-only filesystem.
+
+ @param dir_list: A list of directories installed by the ebuild.
+ @type dir_list: List
+ @return:
+ 1. A list of filesystems which are both set to be written to and are mounted
+ read-only, may be empty.
+ """
+ ro_filesystems = set()
+ ro_filesystems_written = set()
+
+ try:
+ with io.open("/proc/mounts", mode='r', encoding=_encodings['content'],
+ errors='replace') as f:
+ roregex = re.compile(r'(\A|,)ro(\Z|,)')
+ for line in f:
+ if roregex.search(line.split(" ")[3].strip()) is not None:
+ romount = line.split(" ")[1].strip()
+ ro_filesystems.add(romount)
+
+ # If /proc/mounts can't be read, assume that there are no RO
+ # filesystems and return.
+ except EnvironmentError:
+ writemsg_level(_("!!! /proc/mounts cannot be read"),
+ level=logging.WARNING, noiselevel=-1)
+ return []
+
+ if not ro_filesystems:
+ return ro_filesystems
+
+ for directory in dir_list:
+ for filesystem in ro_filesystems:
+ if filesystem == directory:
+ ro_filesystems_written.add(filesystem)
+
+ return ro_filesystems_written
+
+
+def empty_ro_checker(dir_list):
+ """
+ Always returns [], this is the fallback function if the system does not have
+ an ro_checker method defined.
+ """
+ return []
+
+
+# _CHECKERS is a map from ostype output to the appropriate function to return
+# in get_ro_checker.
+_CHECKERS = {
+ "Linux": linux_ro_checker,
+}
--
1.8.5.3
^ permalink raw reply related [flat|nested] 17+ messages in thread
* Re: [gentoo-portage-dev] [PATCH v3] Test for read-only filesystems, bail out during preinst if there are any which will be written to and display a useful error message. Fixes bug 378869.
2014-01-21 3:53 ` Gordon Pettey
@ 2014-01-21 22:44 ` Mike Frysinger
0 siblings, 0 replies; 17+ messages in thread
From: Mike Frysinger @ 2014-01-21 22:44 UTC (permalink / raw
To: gentoo-portage-dev
[-- Attachment #1: Type: Text/Plain, Size: 515 bytes --]
On Monday 20 January 2014 22:53:25 Gordon Pettey wrote:
> If your going to make that argumint, ewe mite a's well right the
> documentation in LOL-1337. Encouraging bad grammar in documentation just
> make's thing's harder for everybody.
(1) don't top post
(2) you posted nothing to support your assertion that using 's to pluralize
acronyms is "bad grammar"
(3) i posted information that shows it's an accepted form
so if you don't have anything useful to post, then just refrain from doing so
-mike
[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 836 bytes --]
^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: [gentoo-portage-dev] [PATCH v5] Test for read-only filesystems, bail out during preinst if there are any which will be written to and display a useful error message. Fixes bug 378869.
2014-01-21 4:00 ` [gentoo-portage-dev] [PATCH v5] " Chris Reffett
@ 2014-01-22 5:24 ` Mike Frysinger
2014-01-22 18:33 ` [gentoo-portage-dev] [PATCH v6] " Chris Reffett
0 siblings, 1 reply; 17+ messages in thread
From: Mike Frysinger @ 2014-01-22 5:24 UTC (permalink / raw
To: gentoo-portage-dev; +Cc: Chris Reffett
[-- Attachment #1: Type: Text/Plain, Size: 421 bytes --]
On Monday 20 January 2014 23:00:31 Chris Reffett wrote:
> + if not ro_filesystems:
> + return ro_filesystems
> +
> + for directory in dir_list:
> + for filesystem in ro_filesystems:
> + if filesystem == directory:
> + ro_filesystems_written.add(filesystem)
> +
> + return ro_filesystems_written
is this just the intersection of two sets ? so all this logic could be:
return ro_filesystems ^ set(dir_list)
-mike
[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 836 bytes --]
^ permalink raw reply [flat|nested] 17+ messages in thread
* [gentoo-portage-dev] [PATCH v6] Test for read-only filesystems, bail out during preinst if there are any which will be written to and display a useful error message. Fixes bug 378869.
2014-01-22 5:24 ` Mike Frysinger
@ 2014-01-22 18:33 ` Chris Reffett
0 siblings, 0 replies; 17+ messages in thread
From: Chris Reffett @ 2014-01-22 18:33 UTC (permalink / raw
To: gentoo-portage-dev
v2: Reformat, add a function to return an appropriate read-only checker
for the operating system, so that this can be extended to other OSes.
v3: minor formatting tweaks from bernalex, including use os.path.join()
instead of string concatenation
v4: Update copyright header, change the code in rochecker to open with
io.open in order to not break on non-ascii characters as reported by
Arfrever. Change get_ro_checker to use a dict as suggested by vapier.
v5: Fix comment format as requested by vapier.
v6: Change linux_ro_checker to use set.intersection instead of the
longer check-and-append system as suggested by vapier.
---
pym/portage/dbapi/vartree.py | 34 +++++++++++++++++-
pym/portage/util/rochecker.py | 80 +++++++++++++++++++++++++++++++++++++++++++
2 files changed, 113 insertions(+), 1 deletion(-)
create mode 100644 pym/portage/util/rochecker.py
diff --git a/pym/portage/dbapi/vartree.py b/pym/portage/dbapi/vartree.py
index ed62323..cf6781b 100644
--- a/pym/portage/dbapi/vartree.py
+++ b/pym/portage/dbapi/vartree.py
@@ -1,4 +1,4 @@
-# Copyright 1998-2013 Gentoo Foundation
+# Copyright 1998-2014 Gentoo Foundation
# Distributed under the terms of the GNU General Public License v2
from __future__ import unicode_literals
@@ -32,6 +32,7 @@ portage.proxy.lazyimport.lazyimport(globals(),
'portage.util.env_update:env_update',
'portage.util.listdir:dircache,listdir',
'portage.util.movefile:movefile',
+ 'portage.util.rochecker:get_ro_checker',
'portage.util._dyn_libs.PreservedLibsRegistry:PreservedLibsRegistry',
'portage.util._dyn_libs.LinkageMapELF:LinkageMapELF@LinkageMap',
'portage.util._async.SchedulerInterface:SchedulerInterface',
@@ -3508,6 +3509,8 @@ class dblink(object):
This function does the following:
+ calls get_ro_checker to retrieve a function for checking whether Portage
+ will write to a read-only filesystem, then runs it against the directory list
calls self._preserve_libs if FEATURES=preserve-libs
calls self._collision_protect if FEATURES=collision-protect
calls doebuild(mydo=pkg_preinst)
@@ -3685,6 +3688,7 @@ class dblink(object):
eagain_error = False
myfilelist = []
+ mydirlist = []
mylinklist = []
paths_with_newlines = []
def onerror(e):
@@ -3717,6 +3721,9 @@ class dblink(object):
unicode_errors.append(new_parent[ed_len:])
break
+ relative_path = parent[srcroot_len:]
+ mydirlist.append(os.path.join("/", relative_path))
+
for fname in files:
try:
fname = _unicode_decode(fname,
@@ -3829,6 +3836,31 @@ class dblink(object):
for other in others_in_slot])
prepare_build_dirs(settings=self.settings, cleanup=cleanup)
+ # Check for read-only filesystems.
+ ro_checker = get_ro_checker()
+ rofilesystems = ro_checker(mydirlist)
+
+ if rofilesystems:
+ msg = _("One or more files installed to this package are "
+ "set to be installed to read-only filesystems. "
+ "Please mount the following filesystems as read-write "
+ "and retry.")
+ msg = textwrap.wrap(msg, 70)
+ msg.append("")
+ for f in rofilesystems:
+ msg.append("\t%s" % os.path.join(destroot,
+ f.lstrip(os.path.sep)))
+ msg.append("")
+ self._elog("eerror", "preinst", msg)
+
+ msg = _("Package '%s' NOT merged due to read-only file systems.") % \
+ self.settings.mycpv
+ msg += _(" If necessary, refer to your elog "
+ "messages for the whole content of the above message.")
+ msg = textwrap.wrap(msg, 70)
+ eerror(msg)
+ return 1
+
# check for package collisions
blockers = self._blockers
if blockers is None:
diff --git a/pym/portage/util/rochecker.py b/pym/portage/util/rochecker.py
new file mode 100644
index 0000000..c13bdfc
--- /dev/null
+++ b/pym/portage/util/rochecker.py
@@ -0,0 +1,80 @@
+#-*- coding:utf-8 -*-
+# Copyright 2014 Gentoo Foundation
+# Distributed under the terms of the GNU General Public License v2
+"""
+Methods to check whether Portage is going to write to read-only filesystems.
+Since the methods are not portable across different OSes, each OS needs its
+own method. To expand RO checking for different OSes, add a method which
+accepts a list of directories and returns a list of mounts which need to be
+remounted RW, then add "elif ostype == (the ostype value for your OS)" to
+get_ro_checker().
+"""
+from __future__ import unicode_literals
+
+import io
+import logging
+import re
+
+from portage import _encodings
+from portage.util import writemsg_level
+from portage.localization import _
+from portage.data import ostype
+
+
+def get_ro_checker():
+ """
+ Uses the system type to find an appropriate method for testing whether Portage
+ is going to write to any read-only filesystems.
+
+ @return:
+ 1. A method for testing for RO filesystems appropriate to the current system.
+ """
+ return _CHECKERS.get(ostype, empty_ro_checker)
+
+
+def linux_ro_checker(dir_list):
+ """
+ Use /proc/mounts to check that no directories installed by the ebuild are set
+ to be installed to a read-only filesystem.
+
+ @param dir_list: A list of directories installed by the ebuild.
+ @type dir_list: List
+ @return:
+ 1. A list of filesystems which are both set to be written to and are mounted
+ read-only, may be empty.
+ """
+ ro_filesystems = set()
+ ro_filesystems_written = set()
+
+ try:
+ with io.open("/proc/mounts", mode='r', encoding=_encodings['content'],
+ errors='replace') as f:
+ roregex = re.compile(r'(\A|,)ro(\Z|,)')
+ for line in f:
+ if roregex.search(line.split(" ")[3].strip()) is not None:
+ romount = line.split(" ")[1].strip()
+ ro_filesystems.add(romount)
+
+ # If /proc/mounts can't be read, assume that there are no RO
+ # filesystems and return.
+ except EnvironmentError:
+ writemsg_level(_("!!! /proc/mounts cannot be read"),
+ level=logging.WARNING, noiselevel=-1)
+ return []
+
+ return set.intersection(ro_filesystems, set(dir_list))
+
+
+def empty_ro_checker(dir_list):
+ """
+ Always returns [], this is the fallback function if the system does not have
+ an ro_checker method defined.
+ """
+ return []
+
+
+# _CHECKERS is a map from ostype output to the appropriate function to return
+# in get_ro_checker.
+_CHECKERS = {
+ "Linux": linux_ro_checker,
+}
--
1.8.5.3
^ permalink raw reply related [flat|nested] 17+ messages in thread
end of thread, other threads:[~2014-01-22 18:34 UTC | newest]
Thread overview: 17+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2014-01-11 3:07 [gentoo-portage-dev] [PATCH] Check for and report read-only filesystems Chris Reffett
2014-01-11 5:09 ` Brian Dolbec
2014-01-11 14:22 ` Chris Reffett
2014-01-13 23:24 ` Mike Frysinger
2014-01-17 0:39 ` [gentoo-portage-dev] [PATCH v2] Test for read-only filesystems, bail out during preinst if there are any which will be written to and display a useful error message. Fixes bug 378869 Chris Reffett
2014-01-19 2:00 ` [gentoo-portage-dev] [PATCH v3] " Chris Reffett
2014-01-19 9:17 ` Mike Frysinger
2014-01-19 10:39 ` Alexander Berntsen
2014-01-19 21:43 ` Alec Warner
2014-01-21 3:28 ` Mike Frysinger
2014-01-21 3:53 ` Gordon Pettey
2014-01-21 22:44 ` Mike Frysinger
2014-01-21 0:50 ` [gentoo-portage-dev] [PATCH v4] " Chris Reffett
2014-01-21 3:32 ` Mike Frysinger
2014-01-21 4:00 ` [gentoo-portage-dev] [PATCH v5] " Chris Reffett
2014-01-22 5:24 ` Mike Frysinger
2014-01-22 18:33 ` [gentoo-portage-dev] [PATCH v6] " Chris Reffett
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox