[gentoo-portage-dev] [PATCH] SyncManager.async: initialize attributes before fork (bug 561234)

[gentoo-portage-dev] [PATCH] SyncManager.async: initialize attributes before fork (bug 561234)

[Zac Medico]

Since commit 496ff326dc18890889d1ea5d2aec590394635960, the pre_sync
method executes in a forked child process. Therefore, initialize
attributes before the fork (in the async method).

Fixes: 496ff326dc18 ("sync repositories in parallel (bug 557426)")
X-Gentoo-Bug: 561234
X-Gentoo-Bug-url: https://bugs.gentoo.org/show_bug.cgi?id=561234
---
 pym/portage/sync/controller.py | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/pym/portage/sync/controller.py b/pym/portage/sync/controller.py
index e992cc4..5471841 100644
--- a/pym/portage/sync/controller.py
+++ b/pym/portage/sync/controller.py
@@ -115,6 +115,9 @@ class SyncManager(object):
 		return []
 
 	def async(self, emerge_config=None, repo=None):
+		self.settings, self.trees, self.mtimedb = emerge_config
+		self.xterm_titles = "notitles" not in self.settings.features
+		self.portdb = self.trees[self.settings['EROOT']]['porttree'].dbapi
 		proc = AsyncFunction(target=self.sync,
 			kwargs=dict(emerge_config=emerge_config, repo=repo))
 		proc.addExitListener(self._sync_callback)
@@ -199,13 +202,10 @@ class SyncManager(object):
 
 
 	def pre_sync(self, repo):
-		self.settings, self.trees, self.mtimedb = self.emerge_config
-		self.xterm_titles = "notitles" not in self.settings.features
 		msg = ">>> Syncing repository '%s' into '%s'..." \
 			% (repo.name, repo.location)
 		self.logger(self.xterm_titles, msg)
 		writemsg_level(msg + "\n")
-		self.portdb = self.trees[self.settings['EROOT']]['porttree'].dbapi
 		try:
 			st = os.stat(repo.location)
 		except OSError:
-- 
2.4.6

[gentoo-portage-dev] [PATCH v2] SyncManager.async: initialize attributes before fork (bug 561234)

[Zac Medico]

Since commit 496ff326dc18890889d1ea5d2aec590394635960, the pre_sync
method executes in a forked child process. Therefore, initialize
attributes before the fork (in the async method).

Fixes: 496ff326dc18 ("sync repositories in parallel (bug 557426)")
X-Gentoo-Bug: 561234
X-Gentoo-Bug-url: https://bugs.gentoo.org/show_bug.cgi?id=561234
---
[PATCH v2] adds self.emerge_config to the initialized attributes

 pym/portage/sync/controller.py | 8 ++++----
 1 file changed, 4 insertions(+), 4 deletions(-)

diff --git a/pym/portage/sync/controller.py b/pym/portage/sync/controller.py
index e992cc4..28dbc57 100644
--- a/pym/portage/sync/controller.py
+++ b/pym/portage/sync/controller.py
@@ -115,13 +115,16 @@ class SyncManager(object):
 		return []
 
 	def async(self, emerge_config=None, repo=None):
+		self.emerge_config = emerge_config
+		self.settings, self.trees, self.mtimedb = emerge_config
+		self.xterm_titles = "notitles" not in self.settings.features
+		self.portdb = self.trees[self.settings['EROOT']]['porttree'].dbapi
 		proc = AsyncFunction(target=self.sync,
 			kwargs=dict(emerge_config=emerge_config, repo=repo))
 		proc.addExitListener(self._sync_callback)
 		return proc
 
 	def sync(self, emerge_config=None, repo=None):
-		self.emerge_config = emerge_config
 		self.callback = None
 		self.repo = repo
 		self.exitcode = 1
@@ -199,13 +202,10 @@ class SyncManager(object):
 
 
 	def pre_sync(self, repo):
-		self.settings, self.trees, self.mtimedb = self.emerge_config
-		self.xterm_titles = "notitles" not in self.settings.features
 		msg = ">>> Syncing repository '%s' into '%s'..." \
 			% (repo.name, repo.location)
 		self.logger(self.xterm_titles, msg)
 		writemsg_level(msg + "\n")
-		self.portdb = self.trees[self.settings['EROOT']]['porttree'].dbapi
 		try:
 			st = os.stat(repo.location)
 		except OSError:
-- 
2.4.6

Re: [gentoo-portage-dev] [PATCH v2] SyncManager.async: initialize attributes before fork (bug 561234)

[Brian Dolbec]

On Wed, 23 Sep 2015 12:14:05 -0700
Zac Medico <zmedico@gentoo.org> wrote:

> Since commit 496ff326dc18890889d1ea5d2aec590394635960, the pre_sync
> method executes in a forked child process. Therefore, initialize
> attributes before the fork (in the async method).
> 
> Fixes: 496ff326dc18 ("sync repositories in parallel (bug 557426)")
> X-Gentoo-Bug: 561234
> X-Gentoo-Bug-url: https://bugs.gentoo.org/show_bug.cgi?id=561234
> ---
> [PATCH v2] adds self.emerge_config to the initialized attributes
> 
>  pym/portage/sync/controller.py | 8 ++++----
>  1 file changed, 4 insertions(+), 4 deletions(-)
> 
> diff --git a/pym/portage/sync/controller.py
> b/pym/portage/sync/controller.py index e992cc4..28dbc57 100644
> --- a/pym/portage/sync/controller.py
> +++ b/pym/portage/sync/controller.py
> @@ -115,13 +115,16 @@ class SyncManager(object):
>  		return []
>  
>  	def async(self, emerge_config=None, repo=None):
> +		self.emerge_config = emerge_config
> +		self.settings, self.trees, self.mtimedb =
> emerge_config
> +		self.xterm_titles = "notitles" not in
> self.settings.features
> +		self.portdb =
> self.trees[self.settings['EROOT']]['porttree'].dbapi proc =
> AsyncFunction(target=self.sync,
> kwargs=dict(emerge_config=emerge_config, repo=repo))
> proc.addExitListener(self._sync_callback) return proc
>  
>  	def sync(self, emerge_config=None, repo=None):
> -		self.emerge_config = emerge_config
>  		self.callback = None
>  		self.repo = repo
>  		self.exitcode = 1
> @@ -199,13 +202,10 @@ class SyncManager(object):
>  
>  
>  	def pre_sync(self, repo):
> -		self.settings, self.trees, self.mtimedb =
> self.emerge_config
> -		self.xterm_titles = "notitles" not in
> self.settings.features msg = ">>> Syncing repository '%s' into
> '%s'..." \ % (repo.name, repo.location)
>  		self.logger(self.xterm_titles, msg)
>  		writemsg_level(msg + "\n")
> -		self.portdb =
> self.trees[self.settings['EROOT']]['porttree'].dbapi try:
>  			st = os.stat(repo.location)
>  		except OSError:

OK, looks good

I suppose we'll need to do another release soon, but how long should we
wait...(new repoman code) or should we just patch 2.2.21

-- 
Brian Dolbec <dolsen>

Re: [gentoo-portage-dev] [PATCH v2] SyncManager.async: initialize attributes before fork (bug 561234)

[Zac Medico]

On 09/23/2015 02:58 PM, Brian Dolbec wrote:
> OK, looks good

Thanks.

> 
> I suppose we'll need to do another release soon, but how long should we
> wait...(new repoman code) or should we just patch 2.2.21

Well, you ran repoman on the whole tree and compared the output to the
old version, right? Did you find any problems there? If not, and long as
you've tested it with a few commits, I'd say that's enough to testing to
release it.

So, I think we're ready to cut a new release as soon as we have the
"portage-2.2.21 ignores PORTAGE_RSYNC_EXTRA_OPTS" bug fixed:

https://bugs.gentoo.org/show_bug.cgi?id=561240
-- 
Thanks,
Zac

Re: [gentoo-portage-dev] [PATCH v2] SyncManager.async: initialize attributes before fork (bug 561234)

[Brian Dolbec]

On Wed, 23 Sep 2015 15:48:42 -0700
Zac Medico <zmedico@gentoo.org> wrote:

> On 09/23/2015 02:58 PM, Brian Dolbec wrote:
> > OK, looks good
> 
> Thanks.
> 
> > 
> > I suppose we'll need to do another release soon, but how long
> > should we wait...(new repoman code) or should we just patch 2.2.21
> 
> Well, you ran repoman on the whole tree and compared the output to the
> old version, right? Did you find any problems there? If not, and long
> as you've tested it with a few commits, I'd say that's enough to
> testing to release it.
> 
> So, I think we're ready to cut a new release as soon as we have the
> "portage-2.2.21 ignores PORTAGE_RSYNC_EXTRA_OPTS" bug fixed:
> 
> https://bugs.gentoo.org/show_bug.cgi?id=561240

well, I had the terminal set for 40,000 lines of scrollback but that
wasn't enough.  I know, I should have redirected the output to a file
It also took about 4 hours...  It did complete with out any tracebacks.

I'll set it up to do simultaneous runs tonight. It certainly didn't tax
this system for cpu power or memory... I suppose it'll put the zfs
caching to the test too :)   And compare the results tomorrow.
-- 
Brian Dolbec <dolsen>