From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from pigeon.gentoo.org ([208.92.234.80] helo=lists.gentoo.org) by finch.gentoo.org with esmtp (Exim 4.60) (envelope-from ) id 1RxHWT-0006mc-LD for garchives@archives.gentoo.org; Tue, 14 Feb 2012 12:30:18 +0000 Received: from pigeon.gentoo.org (localhost [127.0.0.1]) by pigeon.gentoo.org (Postfix) with SMTP id 9BCFBE086B; Tue, 14 Feb 2012 12:30:10 +0000 (UTC) Received: from smtp.gentoo.org (smtp.gentoo.org [140.211.166.183]) by pigeon.gentoo.org (Postfix) with ESMTP id 565DEE086B for ; Tue, 14 Feb 2012 12:30:10 +0000 (UTC) Received: from hornbill.gentoo.org (hornbill.gentoo.org [94.100.119.163]) (using TLSv1 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by smtp.gentoo.org (Postfix) with ESMTPS id 93993644A6 for ; Tue, 14 Feb 2012 12:30:09 +0000 (UTC) Received: from localhost.localdomain (localhost [127.0.0.1]) by hornbill.gentoo.org (Postfix) with ESMTP id 7F3A5E5404 for ; Tue, 14 Feb 2012 12:30:06 +0000 (UTC) From: "Zac Medico" To: gentoo-commits@lists.gentoo.org Content-type: text/plain; charset=UTF-8 Reply-To: gentoo-dev@lists.gentoo.org, "Zac Medico" Message-ID: <1329222542.a75341bf3a66f75edd68d1a8bc5efdb51c0c0740.zmedico@gentoo> Subject: [gentoo-commits] proj/portage:master commit in: pym/portage/dbapi/, pym/portage/, pym/_emerge/ X-VCS-Repository: proj/portage X-VCS-Files: pym/_emerge/EbuildFetcher.py pym/portage/dbapi/_MergeProcess.py pym/portage/process.py X-VCS-Directories: pym/portage/dbapi/ pym/portage/ pym/_emerge/ X-VCS-Committer: zmedico X-VCS-Committer-Name: Zac Medico X-VCS-Revision: a75341bf3a66f75edd68d1a8bc5efdb51c0c0740 X-VCS-Branch: master Date: Tue, 14 Feb 2012 12:30:06 +0000 (UTC) Precedence: bulk List-Post: List-Help: List-Unsubscribe: List-Subscribe: List-Id: Gentoo Linux mail X-BeenThere: gentoo-commits@lists.gentoo.org Content-Transfer-Encoding: quoted-printable X-Archives-Salt: 3f72f446-c485-4052-8aec-48102bf41470 X-Archives-Hash: 67749f7b423609cd55767803ff54e4e6 commit: a75341bf3a66f75edd68d1a8bc5efdb51c0c0740 Author: Zac Medico gentoo org> AuthorDate: Tue Feb 14 12:29:02 2012 +0000 Commit: Zac Medico gentoo org> CommitDate: Tue Feb 14 12:29:02 2012 +0000 URL: http://git.overlays.gentoo.org/gitweb/?p=3Dproj/portage.git;a= =3Dcommit;h=3Da75341bf After python fork, don't close fds for PyPy 1.8. If we close all open file descriptors after a fork, with PyPy 1.8 it triggers "[Errno 9] Bad file descriptor" later in the subprocess. Apparently it is holding references to file descriptors and closing them after they've already been closed and re-opened for other purposes. As a workaround, we don't close the file descriptors, so that they won't be re-used and therefore we won't be vulnerable to this kind of interference. The obvious caveat of not closing the fds is that the subprocess can hold locks that belonged to the parent process, even after the parent process has released the locks. Hopefully this won't be a major problem though, since the subprocess has to exit at release the lock eventually, when the EbuildFetcher or _MergeProcess task is complete. --- pym/_emerge/EbuildFetcher.py | 6 +++++- pym/portage/dbapi/_MergeProcess.py | 7 ++++++- pym/portage/process.py | 20 +++++++++++--------- 3 files changed, 22 insertions(+), 11 deletions(-) diff --git a/pym/_emerge/EbuildFetcher.py b/pym/_emerge/EbuildFetcher.py index 61c7848..6ad4341 100644 --- a/pym/_emerge/EbuildFetcher.py +++ b/pym/_emerge/EbuildFetcher.py @@ -6,6 +6,7 @@ import traceback from _emerge.SpawnProcess import SpawnProcess import copy import io +import platform import signal import sys import portage @@ -166,7 +167,10 @@ class EbuildFetcher(SpawnProcess): portage.process.spawned_pids.append(pid) return [pid] =20 - portage.process._setup_pipes(fd_pipes) + # TODO: Find out why PyPy 1.8 with close_fds=3DTrue triggers + # "[Errno 9] Bad file descriptor" in subprocesses. + close_fds =3D platform.python_implementation() !=3D 'PyPy' + portage.process._setup_pipes(fd_pipes, close_fds=3Dclose_fds) =20 # Use default signal handlers in order to avoid problems # killing subprocesses as reported in bug #353239. diff --git a/pym/portage/dbapi/_MergeProcess.py b/pym/portage/dbapi/_Merg= eProcess.py index 9bb67c9..cf59265 100644 --- a/pym/portage/dbapi/_MergeProcess.py +++ b/pym/portage/dbapi/_MergeProcess.py @@ -2,6 +2,7 @@ # Distributed under the terms of the GNU General Public License v2 =20 import io +import platform import signal import traceback =20 @@ -143,7 +144,11 @@ class MergeProcess(SpawnProcess): return [pid] =20 os.close(elog_reader_fd) - portage.process._setup_pipes(fd_pipes) + + # TODO: Find out why PyPy 1.8 with close_fds=3DTrue triggers + # "[Errno 9] Bad file descriptor" in subprocesses. + close_fds =3D platform.python_implementation() !=3D 'PyPy' + portage.process._setup_pipes(fd_pipes, close_fds=3Dclose_fds) =20 # Use default signal handlers since the ones inherited # from the parent process are irrelevant here. diff --git a/pym/portage/process.py b/pym/portage/process.py index 47b0a21..e7313ab 100644 --- a/pym/portage/process.py +++ b/pym/portage/process.py @@ -386,7 +386,7 @@ def _exec(binary, mycommand, opt_name, fd_pipes, env,= gid, groups, uid, umask, # And switch to the new process. os.execve(binary, myargs, env) =20 -def _setup_pipes(fd_pipes): +def _setup_pipes(fd_pipes, close_fds=3DTrue): """Setup pipes for a forked process.""" my_fds =3D {} # To protect from cases where direct assignment could @@ -397,14 +397,16 @@ def _setup_pipes(fd_pipes): # Then assign them to what they should be. for fd in my_fds: os.dup2(my_fds[fd], fd) - # Then close _all_ fds that haven't been explicitly - # requested to be kept open. - for fd in get_open_fds(): - if fd not in my_fds: - try: - os.close(fd) - except OSError: - pass + + if close_fds: + # Then close _all_ fds that haven't been explicitly + # requested to be kept open. + for fd in get_open_fds(): + if fd not in my_fds: + try: + os.close(fd) + except OSError: + pass =20 def find_binary(binary): """