public inbox for gentoo-portage-dev@lists.gentoo.org
 help / color / mirror / Atom feed
* [gentoo-portage-dev] [PATCH] locks: use fcntl.flock if fcntl.lockf is broken (bug 595146)
@ 2016-09-26  6:44 Zac Medico
  2016-10-02  0:08 ` [gentoo-portage-dev] " Zac Medico
  0 siblings, 1 reply; 4+ messages in thread
From: Zac Medico @ 2016-09-26  6:44 UTC (permalink / raw
  To: gentoo-portage-dev; +Cc: Zac Medico

This is needed for Windows Subsystem for Linux (WSL), as well as
older versions of PyPy.

X-Gentoo-bug: 595146
X-Gentoo-bug-url: https://bugs.gentoo.org/show_bug.cgi?id=595146
---
 pym/portage/locks.py | 59 +++++++++++++++++++++++++++++++++++++++++++++-------
 1 file changed, 52 insertions(+), 7 deletions(-)

diff --git a/pym/portage/locks.py b/pym/portage/locks.py
index 42ff1e3..a4564cf 100644
--- a/pym/portage/locks.py
+++ b/pym/portage/locks.py
@@ -8,8 +8,9 @@ __all__ = ["lockdir", "unlockdir", "lockfile", "unlockfile", \
 
 import errno
 import fcntl
-import platform
+import multiprocessing
 import sys
+import tempfile
 import time
 import warnings
 
@@ -27,17 +28,61 @@ if sys.hexversion >= 0x3000000:
 
 HARDLINK_FD = -2
 _HARDLINK_POLL_LATENCY = 3 # seconds
-_default_lock_fn = fcntl.lockf
-
-if platform.python_implementation() == 'PyPy':
-	# workaround for https://bugs.pypy.org/issue747
-	_default_lock_fn = fcntl.flock
 
 # Used by emerge in order to disable the "waiting for lock" message
 # so that it doesn't interfere with the status display.
 _quiet = False
 
 
+_lock_fn = None
+
+
+def _get_lock_fn():
+	"""
+	Returns fcntl.lockf if proven to work, and otherwise returns fcntl.flock.
+	On some platforms fcntl.lockf is known to be broken.
+	"""
+	global _lock_fn
+	if _lock_fn is not None:
+		return _lock_fn
+
+	def _test_lock(fd, lock_path):
+		os.close(fd)
+		try:
+			with open(lock_path, 'a') as f:
+				fcntl.lockf(f.fileno(), fcntl.LOCK_EX|fcntl.LOCK_NB)
+		except EnvironmentError as e:
+			if e.errno == errno.EAGAIN:
+				# Parent process holds lock, as expected.
+				sys.exit(0)
+
+		# Something went wrong.
+		sys.exit(1)
+
+	fd, lock_path = tempfile.mkstemp()
+	try:
+		try:
+			fcntl.lockf(fd, fcntl.LOCK_EX)
+		except EnvironmentError:
+			pass
+		else:
+			proc = multiprocessing.Process(target=_test_lock,
+				args=(fd, lock_path))
+			proc.start()
+			proc.join()
+			if proc.exitcode == os.EX_OK:
+				# Use fcntl.lockf becase the test passed.
+				_lock_fn = fcntl.lockf
+				return _lock_fn
+	finally:
+		os.close(fd)
+		os.unlink(lock_path)
+
+	# Fall back to fcntl.flock.
+	_lock_fn = fcntl.flock
+	return _lock_fn
+
+
 _open_fds = set()
 
 def _close_fds():
@@ -146,7 +191,7 @@ def lockfile(mypath, wantnewlockfile=0, unlinkfile=0,
 
 	# try for a non-blocking lock, if it's held, throw a message
 	# we're waiting on lockfile and use a blocking attempt.
-	locking_method = portage._eintr_func_wrapper(_default_lock_fn)
+	locking_method = portage._eintr_func_wrapper(_get_lock_fn())
 	try:
 		if "__PORTAGE_TEST_HARDLINK_LOCKS" in os.environ:
 			raise IOError(errno.ENOSYS, "Function not implemented")
-- 
2.7.4



^ permalink raw reply related	[flat|nested] 4+ messages in thread

* [gentoo-portage-dev] Re: [PATCH] locks: use fcntl.flock if fcntl.lockf is broken (bug 595146)
  2016-09-26  6:44 [gentoo-portage-dev] [PATCH] locks: use fcntl.flock if fcntl.lockf is broken (bug 595146) Zac Medico
@ 2016-10-02  0:08 ` Zac Medico
  2016-10-02  4:30   ` Brian Dolbec
  0 siblings, 1 reply; 4+ messages in thread
From: Zac Medico @ 2016-10-02  0:08 UTC (permalink / raw
  To: Zac Medico, gentoo-portage-dev

On 09/25/2016 11:44 PM, Zac Medico wrote:
> This is needed for Windows Subsystem for Linux (WSL), as well as
> older versions of PyPy.
> 
> X-Gentoo-bug: 595146
> X-Gentoo-bug-url: https://bugs.gentoo.org/show_bug.cgi?id=595146
> ---
>  pym/portage/locks.py | 59 +++++++++++++++++++++++++++++++++++++++++++++-------
>  1 file changed, 52 insertions(+), 7 deletions(-)

Any feedback on this? I think we should merge it, for inclusion in
portage-2.3.2.
-- 
Thanks,
Zac


^ permalink raw reply	[flat|nested] 4+ messages in thread

* Re: [gentoo-portage-dev] Re: [PATCH] locks: use fcntl.flock if fcntl.lockf is broken (bug 595146)
  2016-10-02  0:08 ` [gentoo-portage-dev] " Zac Medico
@ 2016-10-02  4:30   ` Brian Dolbec
  2016-10-02  4:48     ` Zac Medico
  0 siblings, 1 reply; 4+ messages in thread
From: Brian Dolbec @ 2016-10-02  4:30 UTC (permalink / raw
  To: gentoo-portage-dev

On Sat, 1 Oct 2016 17:08:46 -0700
Zac Medico <zmedico@gentoo.org> wrote:

> On 09/25/2016 11:44 PM, Zac Medico wrote:
> > This is needed for Windows Subsystem for Linux (WSL), as well as
> > older versions of PyPy.
> > 
> > X-Gentoo-bug: 595146
> > X-Gentoo-bug-url: https://bugs.gentoo.org/show_bug.cgi?id=595146
> > ---
> >  pym/portage/locks.py | 59
> > +++++++++++++++++++++++++++++++++++++++++++++------- 1 file
> > changed, 52 insertions(+), 7 deletions(-)  
> 
> Any feedback on this? I think we should merge it, for inclusion in
> portage-2.3.2.


Sorry, been out all day, and again tomorrow...

Yeah, looks fine
-- 
Brian Dolbec <dolsen>



^ permalink raw reply	[flat|nested] 4+ messages in thread

* Re: [gentoo-portage-dev] Re: [PATCH] locks: use fcntl.flock if fcntl.lockf is broken (bug 595146)
  2016-10-02  4:30   ` Brian Dolbec
@ 2016-10-02  4:48     ` Zac Medico
  0 siblings, 0 replies; 4+ messages in thread
From: Zac Medico @ 2016-10-02  4:48 UTC (permalink / raw
  To: gentoo-portage-dev

On 10/01/2016 09:30 PM, Brian Dolbec wrote:
> On Sat, 1 Oct 2016 17:08:46 -0700
> Zac Medico <zmedico@gentoo.org> wrote:
> 
>> On 09/25/2016 11:44 PM, Zac Medico wrote:
>>> This is needed for Windows Subsystem for Linux (WSL), as well as
>>> older versions of PyPy.
>>>
>>> X-Gentoo-bug: 595146
>>> X-Gentoo-bug-url: https://bugs.gentoo.org/show_bug.cgi?id=595146
>>> ---
>>>  pym/portage/locks.py | 59
>>> +++++++++++++++++++++++++++++++++++++++++++++------- 1 file
>>> changed, 52 insertions(+), 7 deletions(-)  
>>
>> Any feedback on this? I think we should merge it, for inclusion in
>> portage-2.3.2.
> 
> 
> Sorry, been out all day, and again tomorrow...
> 
> Yeah, looks fine

Thanks, merged:

https://gitweb.gentoo.org/proj/portage.git/commit/?id=5ef5fbaab88de47d4dbab333661d3525261d7633
-- 
Thanks,
Zac


^ permalink raw reply	[flat|nested] 4+ messages in thread

end of thread, other threads:[~2016-10-02  4:48 UTC | newest]

Thread overview: 4+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2016-09-26  6:44 [gentoo-portage-dev] [PATCH] locks: use fcntl.flock if fcntl.lockf is broken (bug 595146) Zac Medico
2016-10-02  0:08 ` [gentoo-portage-dev] " Zac Medico
2016-10-02  4:30   ` Brian Dolbec
2016-10-02  4:48     ` Zac Medico

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox