[gentoo-catalyst] [PATCH 2/5] generic_stage_target.py: Fix an intermittent snapshot_cache_path keyerror

[gentoo-catalyst] [PATCH 2/5] generic_stage_target.py: Fix an intermittent snapshot_cache_path keyerror

[Brian Dolbec]

---
 catalyst/targets/generic_stage_target.py | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/catalyst/targets/generic_stage_target.py b/catalyst/targets/generic_stage_target.py
index 095327a..7fd583e 100644
--- a/catalyst/targets/generic_stage_target.py
+++ b/catalyst/targets/generic_stage_target.py
@@ -470,10 +470,10 @@ class generic_stage_target(generic_target):
 					hash_function=self.settings["hash_function"],verbose=False)
 
 	def set_snapcache_path(self):
+		self.settings["snapshot_cache_path"] = \
+			normpath(self.settings["snapshot_cache"] + "/" +
+				self.settings["snapshot"])
 		if "SNAPCACHE" in self.settings:
-			self.settings["snapshot_cache_path"] = \
-				normpath(self.settings["snapshot_cache"] + "/" +
-					self.settings["snapshot"])
 			self.snapcache_lock=\
 				LockDir(self.settings["snapshot_cache_path"])
 			print "Caching snapshot to "+self.settings["snapshot_cache_path"]
-- 
1.8.5.3

[gentoo-catalyst] [PATCH 3/5] Add more working files, directories to .gitignore

[Brian Dolbec]

---
 .gitignore | 5 +++++
 1 file changed, 5 insertions(+)

diff --git a/.gitignore b/.gitignore
index d52b297..9dc5a73 100644
--- a/.gitignore
+++ b/.gitignore
@@ -3,3 +3,8 @@ dist
 build
 files
 MANIFEST
+test*
+*.geany
+outgoing/*
+Scratch/*
+*.patch
-- 
1.8.5.3

[gentoo-catalyst] [PATCH 4/5] catalyst/targets/generic_target.py: Pass TERM through to the chroot

[Brian Dolbec]

From: "W. Trevor King" <wking@tremily.us>

Avoid:

  Running stage1-chroot.sh in chroot /var/tmp/catalyst/tmp/default/...
  tput: No value for $TERM and no -T specified

by passing the caller's TERM environment variable [1] through to the
chroot.  If the caller does not supply TERM, default to 'dumb' which
disables color etc., but should be the most portable.  On Gentoo, the
dumb terminfo (/usr/share/terminfo/d/dumb) is distributed as part of
ncurses [2].  You can list supported terminals with toe, which is also
distributed with ncurses [2]:

  $ toe
  ansi            ansi/pc-term compatible with color
  dumb            80-column dumb tty
  linux           linux console
  ...

[1]: http://pubs.opengroup.org/onlinepubs/9699919799/basedefs/V1_chap08.html#tag_08_03
[2]: http://www.gnu.org/software/ncurses/
---
 catalyst/targets/generic_target.py | 8 ++++++--
 1 file changed, 6 insertions(+), 2 deletions(-)

diff --git a/catalyst/targets/generic_target.py b/catalyst/targets/generic_target.py
index de51994..382f1c7 100644
--- a/catalyst/targets/generic_target.py
+++ b/catalyst/targets/generic_target.py
@@ -1,3 +1,5 @@
+import os
+
 from catalyst.support import *
 
 class generic_target:
@@ -7,5 +9,7 @@ class generic_target:
 	def __init__(self,myspec,addlargs):
 		addl_arg_parse(myspec,addlargs,self.required_values,self.valid_values)
 		self.settings=myspec
-		self.env={}
-		self.env["PATH"]="/bin:/sbin:/usr/bin:/usr/sbin"
+		self.env = {
+			'PATH': '/bin:/sbin:/usr/bin:/usr/sbin',
+			'TERM': os.getenv('TERM', 'dumb'),
+			}
-- 
1.8.5.3

[gentoo-catalyst] [PATCH 5/5] Add arm64 support

[Brian Dolbec]

From: Mike Frysinger <vapier@gentoo.org>

Brian Dolbec: Modify patch for the new directory structure and apply.
              Cleanup the imports.

dgdg
---
 catalyst/arch/arm64.py | 15 +++++++++++++++
 1 file changed, 15 insertions(+)
 create mode 100644 catalyst/arch/arm64.py

diff --git a/catalyst/arch/arm64.py b/catalyst/arch/arm64.py
new file mode 100644
index 0000000..00e7527
--- /dev/null
+++ b/catalyst/arch/arm64.py
@@ -0,0 +1,15 @@
+
+from catalyst import builder
+
+class arch_arm64(builder.generic):
+	"builder class for arm64"
+	def __init__(self,myspec):
+		builder.generic.__init__(self,myspec)
+		self.settings["CHROOT"]="chroot"
+		self.settings["CFLAGS"]="-O2 -pipe"
+		self.settings["CFLAGS"]="-O2 -pipe"
+		self.settings["CHOST"]="aarch64-unknown-linux-gnu"
+
+def register():
+	"Inform main catalyst program of the contents of this plugin."
+	return ({ "arm64":arch_arm64 }, ("aarch64","arm64", ))
-- 
1.8.5.3

Re: [gentoo-catalyst] [PATCH 5/5] Add arm64 support

[Brian Dolbec]

On Sun,  2 Mar 2014 15:01:02 -0800
Brian Dolbec <dolsen@gentoo.org> wrote:

> From: Mike Frysinger <vapier@gentoo.org>
> 
> Brian Dolbec: Modify patch for the new directory structure and apply.
>               Cleanup the imports.
> 
> dgdg
> ---

In git, I cleaned up the errant "dgdg" from the commit message.
-- 
Brian Dolbec <dolsen>

Re: [gentoo-catalyst] [PATCH 3/5] Add more working files, directories to .gitignore

[Brian Dolbec]

On Sun,  2 Mar 2014 15:01:00 -0800
Brian Dolbec <dolsen@gentoo.org> wrote:

> ---
>  .gitignore | 5 +++++
>  1 file changed, 5 insertions(+)
> 
> diff --git a/.gitignore b/.gitignore
> index d52b297..9dc5a73 100644
> --- a/.gitignore
> +++ b/.gitignore
> @@ -3,3 +3,8 @@ dist
>  build
>  files
>  MANIFEST
> +test*
> +*.geany
> +outgoing/*
> +Scratch/*
> +*.patch

I've added one more entry, it's in git:

+patches/*

-- 
Brian Dolbec <dolsen>

[gentoo-catalyst] Re: [PATCH 2/5] generic_stage_target.py: Fix an intermittent snapshot_cache_path keyerror

[W. Trevor King]

[-- Attachment #1: Type: text/plain, Size: 1515 bytes --]

On Sun, Mar 02, 2014 at 03:00:59PM -0800, Brian Dolbec wrote:
>  	def set_snapcache_path(self):
> +		self.settings["snapshot_cache_path"] = \
> +			normpath(self.settings["snapshot_cache"] + "/" +
> +				self.settings["snapshot"])
>  		if "SNAPCACHE" in self.settings:
> -			self.settings["snapshot_cache_path"] = \
> -				normpath(self.settings["snapshot_cache"] + "/" +
> -					self.settings["snapshot"])

If we're getting snapshot_cache_path key errors, I think the solution
is to protect those call sites with:

  if 'SNAPCACHE' in self.settings:

blocks, not to define a snapcache-only setting for folks who are not
using snapcaches.  There are not particularly many snapshot_cache_path
references in catalyst.  Grepping through them, the following looks
suspicious:

catalyst/targets/generic_stage_target.py-215-           self.mountmap["portdir"] = normpath("/".join([
catalyst/targets/generic_stage_target.py:216:                   self.settings["snapshot_cache_path"],
catalyst/targets/generic_stage_target.py-217-                   self.settings["repo_name"],
catalyst/targets/generic_stage_target.py-218-                   ]))
catalyst/targets/generic_stage_target.py-219-           if "SNAPCACHE" not in self.settings:

Everything else looks fine.  So what should portdir be if SNAPCACHE
isn't set?

Cheers,
Trevor

-- 
This email may be signed or encrypted with GnuPG (http://www.gnupg.org).
For more information, see http://en.wikipedia.org/wiki/Pretty_Good_Privacy

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 836 bytes --]

[gentoo-catalyst] Re: [PATCH 3/5] Add more working files, directories to .gitignore

[W. Trevor King]

[-- Attachment #1: Type: text/plain, Size: 735 bytes --]

On Sun, Mar 02, 2014 at 03:01:00PM -0800, Brian Dolbec wrote:
> ---
>  .gitignore | 5 +++++
>  1 file changed, 5 insertions(+)
> 
> diff --git a/.gitignore b/.gitignore
> index d52b297..9dc5a73 100644
> --- a/.gitignore
> +++ b/.gitignore
> @@ -3,3 +3,8 @@ dist
>  build
>  files
>  MANIFEST
> +test*
> +*.geany
> +outgoing/*
> +Scratch/*
> +*.patch

These all look like entries for a local $GIT_DIR/info/exclude, instead
of the shared, in-tree .gitignore, since they're specific to your
workflow and not catalyst or it's build process.

Cheers,
Trevor

-- 
This email may be signed or encrypted with GnuPG (http://www.gnupg.org).
For more information, see http://en.wikipedia.org/wiki/Pretty_Good_Privacy

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 836 bytes --]

Re: [gentoo-catalyst] Re: [PATCH 3/5] Add more working files, directories to .gitignore

[Brian Dolbec]

On Tue, 4 Mar 2014 20:50:38 -0800
"W. Trevor King" <wking@tremily.us> wrote:

> On Sun, Mar 02, 2014 at 03:01:00PM -0800, Brian Dolbec wrote:
> > ---
> >  .gitignore | 5 +++++
> >  1 file changed, 5 insertions(+)
> > 
> > diff --git a/.gitignore b/.gitignore
> > index d52b297..9dc5a73 100644
> > --- a/.gitignore
> > +++ b/.gitignore
> > @@ -3,3 +3,8 @@ dist
> >  build
> >  files
> >  MANIFEST
> > +test*
> > +*.geany
> > +outgoing/*
> > +Scratch/*
> > +*.patch
> 
> These all look like entries for a local $GIT_DIR/info/exclude, instead
> of the shared, in-tree .gitignore, since they're specific to your
> workflow and not catalyst or it's build process.
> 
> Cheers,
> Trevor
> 

Yes, they are.  I've never professed to be a git "expert"
Please enlighten me on that.  It's annoying trying to wade through
all those files making commits.


-- 
Brian Dolbec <dolsen>

Re: [gentoo-catalyst] Re: [PATCH 2/5] generic_stage_target.py: Fix an intermittent snapshot_cache_path keyerror

[Brian Dolbec]

On Tue, 4 Mar 2014 20:47:47 -0800
"W. Trevor King" <wking@tremily.us> wrote:

> On Sun, Mar 02, 2014 at 03:00:59PM -0800, Brian Dolbec wrote:
> >  	def set_snapcache_path(self):
> > +		self.settings["snapshot_cache_path"] = \
> > +			normpath(self.settings["snapshot_cache"] + "/" +
> > +				self.settings["snapshot"])
> >  		if "SNAPCACHE" in self.settings:
> > -			self.settings["snapshot_cache_path"] = \
> > -				normpath(self.settings["snapshot_cache"] + "/" +
> > -					self.settings["snapshot"])
> 
> If we're getting snapshot_cache_path key errors, I think the solution
> is to protect those call sites with:
> 
>   if 'SNAPCACHE' in self.settings:
> 
> blocks, not to define a snapcache-only setting for folks who are not
> using snapcaches.  There are not particularly many snapshot_cache_path
> references in catalyst.  Grepping through them, the following looks
> suspicious:
> 
> catalyst/targets/generic_stage_target.py-215-           self.mountmap["portdir"] = normpath("/".join([
> catalyst/targets/generic_stage_target.py:216:                   self.settings["snapshot_cache_path"],
> catalyst/targets/generic_stage_target.py-217-                   self.settings["repo_name"],
> catalyst/targets/generic_stage_target.py-218-                   ]))
> catalyst/targets/generic_stage_target.py-219-           if "SNAPCACHE" not in self.settings:
> 
> Everything else looks fine.  So what should portdir be if SNAPCACHE
> isn't set?
> 
> Cheers,
> Trevor
> 

Yeah, that's where it was failing.  I don't think it matters to be
honest. But I admit I haven't analyzed it thoroughly.  I ran into this
one while trying to track down other problems.  So I quick fixed it.
It's amazing the errors you find turning things off that are normally
on.

I'm open to a better alternative.  I think the potential is there for
more of these types of errors which depend on options selected.

-- 
Brian Dolbec <dolsen>

Re: [gentoo-catalyst] Re: [PATCH 3/5] Add more working files, directories to .gitignore

[W. Trevor King]

[-- Attachment #1: Type: text/plain, Size: 1310 bytes --]

On Tue, Mar 04, 2014 at 09:17:42PM -0800, Brian Dolbec wrote:
> On Tue, 4 Mar 2014 20:50:38 -0800, W. Trevor King wrote:
> > On Sun, Mar 02, 2014 at 03:01:00PM -0800, Brian Dolbec wrote:
> > These all look like entries for a local $GIT_DIR/info/exclude, instead
> > of the shared, in-tree .gitignore, since they're specific to your
> > workflow and not catalyst or it's build process.
>
> Yes, they are.  I've never professed to be a git "expert"
> Please enlighten me on that.  It's annoying trying to wade through
> all those files making commits.

The “DESCRIPTION” section of gitignore(5) goes through it pretty
clearly [1]. $GIT_DIR/info/exclude (likely
~/src/catalyst/.git/info/exclude or similar for you) has the same
syntax as .gitignore, but it's not versioned.  gitignore(5) recommends
it for [1]:

  Patterns which are specific to a particular repository but which do
  not need to be shared with other related repositories (e.g.,
  auxiliary files that live inside the repository but are specific to
  one user’s workflow)

Cheers,
Trevor

[1]: https://www.kernel.org/pub/software/scm/git/docs/gitignore.html

-- 
This email may be signed or encrypted with GnuPG (http://www.gnupg.org).
For more information, see http://en.wikipedia.org/wiki/Pretty_Good_Privacy

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 836 bytes --]

[gentoo-catalyst] [PATCH] generic_stage_target: Don't set mountmap['portdir'] without SNAPCACHE

[W. Trevor King]

The default portdir path uses the snapshot_cache_path setting, but we
only set a default for that (in
generic_stage_target.set_snapcache_path) if SNAPCACHE is enabled.  By
shifting the portdir calculation into the SNAPCACHE-only block here,
we avoid running into key-errors when the user has SNAPCACHE disabled
and (unsurprisingly) doesn't specify an explicit snapshot_cache_path.
---
On Tue, Mar 04, 2014 at 09:25:08PM -0800, Brian Dolbec wrote:
> On Tue, 4 Mar 2014 20:47:47 -0800, W. Trevor King wrote:
> > There are not particularly many snapshot_cache_path references in
> > catalyst.  Grepping through them, the following looks suspicious:
> >
> > catalyst/targets/generic_stage_target.py-215- self.mountmap["portdir"] = normpath("/".join([
> > catalyst/targets/generic_stage_target.py:216:  self.settings["snapshot_cache_path"],
> > catalyst/targets/generic_stage_target.py-217-  self.settings["repo_name"],
> > catalyst/targets/generic_stage_target.py-218-  ]))
> > catalyst/targets/generic_stage_target.py-219- if "SNAPCACHE" not in self.settings:
> >
> > Everything else looks fine.  So what should portdir be if
> > SNAPCACHE isn't set?
>
> Yeah, that's where it was failing.  …
>
> I'm open to a better alternative.  I think the potential is there
> for more of these types of errors which depend on options selected.

This patch should fix it.  Analysis in the commit message itself.

Cheers,
Trevor

 catalyst/targets/generic_stage_target.py | 8 ++++----
 1 file changed, 4 insertions(+), 4 deletions(-)

diff --git a/catalyst/targets/generic_stage_target.py b/catalyst/targets/generic_stage_target.py
index 2b3d7ce..1412ed5 100644
--- a/catalyst/targets/generic_stage_target.py
+++ b/catalyst/targets/generic_stage_target.py
@@ -212,11 +212,11 @@ class generic_stage_target(generic_target):
 		self.mountmap = SOURCE_MOUNTS_DEFAULTS.copy()
 		# update them from settings
 		self.mountmap["distdir"] = self.settings["distdir"]
-		self.mountmap["portdir"] = normpath("/".join([
-			self.settings["snapshot_cache_path"],
-			self.settings["repo_name"],
-			]))
 		if "SNAPCACHE" not in self.settings:
+			self.mountmap["portdir"] = normpath("/".join([
+				self.settings["snapshot_cache_path"],
+				self.settings["repo_name"],
+				]))
 			self.mounts.remove("portdir")
 			#self.mountmap["portdir"] = None
 		if os.uname()[0] == "Linux":
-- 
1.8.5.2.8.g0f6c0d1

Re: [gentoo-catalyst] [PATCH 4/5] catalyst/targets/generic_target.py: Pass TERM through to the chroot

[W. Trevor King]

[-- Attachment #1: Type: text/plain, Size: 247 bytes --]

Looks good to me ;).  Thanks for updating the commit message summary.

Cheers,
Trevor

-- 
This email may be signed or encrypted with GnuPG (http://www.gnupg.org).
For more information, see http://en.wikipedia.org/wiki/Pretty_Good_Privacy

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 836 bytes --]

Re: [gentoo-catalyst] Re: [PATCH 3/5] Add more working files, directories to .gitignore

[Brian Dolbec]

On Wed, 5 Mar 2014 07:32:46 -0800
"W. Trevor King" <wking@tremily.us> wrote:

> On Tue, Mar 04, 2014 at 09:17:42PM -0800, Brian Dolbec wrote:
> > On Tue, 4 Mar 2014 20:50:38 -0800, W. Trevor King wrote:
> > > On Sun, Mar 02, 2014 at 03:01:00PM -0800, Brian Dolbec wrote:
> > > These all look like entries for a local $GIT_DIR/info/exclude,
> > > instead of the shared, in-tree .gitignore, since they're specific
> > > to your workflow and not catalyst or it's build process.
> >
> > Yes, they are.  I've never professed to be a git "expert"
> > Please enlighten me on that.  It's annoying trying to wade through
> > all those files making commits.
> 
> The “DESCRIPTION” section of gitignore(5) goes through it pretty
> clearly [1]. $GIT_DIR/info/exclude (likely
> ~/src/catalyst/.git/info/exclude or similar for you) has the same
> syntax as .gitignore, but it's not versioned.  gitignore(5) recommends
> it for [1]:
> 
>   Patterns which are specific to a particular repository but which do
>   not need to be shared with other related repositories (e.g.,
>   auxiliary files that live inside the repository but are specific to
>   one user’s workflow)
> 
> Cheers,
> Trevor
> 
> [1]: https://www.kernel.org/pub/software/scm/git/docs/gitignore.html
> 

Done, that you for that info  This commit is removed from pending, will
push once some other feedback changes are finished testing.

-- 
Brian Dolbec <dolsen>

Re: [gentoo-catalyst] [PATCH] generic_stage_target: Don't set mountmap['portdir'] without SNAPCACHE

[Brian Dolbec]

On Sat,  8 Mar 2014 10:25:09 -0800
"W. Trevor King" <wking@tremily.us> wrote:

> This patch should fix it.  Analysis in the commit message itself.
> 
> Cheers,
> Trevor
> 
>  catalyst/targets/generic_stage_target.py | 8 ++++----
>  1 file changed, 4 insertions(+), 4 deletions(-)
> 
> diff --git a/catalyst/targets/generic_stage_target.py
> b/catalyst/targets/generic_stage_target.py index 2b3d7ce..1412ed5
> 100644 --- a/catalyst/targets/generic_stage_target.py
> +++ b/catalyst/targets/generic_stage_target.py
> @@ -212,11 +212,11 @@ class generic_stage_target(generic_target):
>  		self.mountmap = SOURCE_MOUNTS_DEFAULTS.copy()
>  		# update them from settings
>  		self.mountmap["distdir"] = self.settings["distdir"]
> -		self.mountmap["portdir"] = normpath("/".join([
> -			self.settings["snapshot_cache_path"],
> -			self.settings["repo_name"],
> -			]))
>  		if "SNAPCACHE" not in self.settings:
> +			self.mountmap["portdir"] =
> normpath("/".join([
> +				self.settings["snapshot_cache_path"],
> +				self.settings["repo_name"],
> +				]))
>  			self.mounts.remove("portdir")
>  			#self.mountmap["portdir"] = None
>  		if os.uname()[0] == "Linux":


Not quite correct.  Note the not in that if...

It turns out it was a regression for the conversion in previous commits.
They original code had the "mountmap['usr/portage'] = ..." inside
 if "SNAPCACHE"...

 Anyway, here is the final fixed and rebased patch pushed to the
 pending branch:

diff --git a/catalyst/targets/generic_stage_target.py b/catalyst/targets/generic_stage_target.py
index 9c39d00..eaf2c1f 100644
@@ -215,13 +215,14 @@ class generic_stage_target(generic_target):
 		self.mountmap = SOURCE_MOUNTS_DEFAULTS.copy()
 		# update them from settings
 		self.mountmap["distdir"] = self.settings["distdir"]
-		self.mountmap["portdir"] = normpath("/".join([
-			self.settings["snapshot_cache_path"],
-			self.settings["repo_name"],
-			]))
 		if "SNAPCACHE" not in self.settings:
 			self.mounts.remove("portdir")
-			#self.mountmap["portdir"] = None
+			self.mountmap["portdir"] = None
+		else:
+			self.mountmap["portdir"] = normpath("/".join([
+				self.settings["snapshot_cache_path"],
+				self.settings["repo_name"],
+				]))
 		if os.uname()[0] == "Linux":
 			self.mounts.append("devpts")
 			self.mounts.append("shm")

-- 
Brian Dolbec <dolsen>

Re: [gentoo-catalyst] [PATCH] generic_stage_target: Don't set mountmap['portdir'] without SNAPCACHE

[W. Trevor King]

[-- Attachment #1: Type: text/plain, Size: 1538 bytes --]

On Sat, Mar 22, 2014 at 03:39:34PM -0700, Brian Dolbec wrote:
> On Sat,  8 Mar 2014 10:25:09 -0800 "W. Trevor King" wrote:
> >  		if "SNAPCACHE" not in self.settings:
> > +			self.mountmap["portdir"] =
> > normpath("/".join([
> > +				self.settings["snapshot_cache_path"],
> > +				self.settings["repo_name"],
> > +				]))
> >  			self.mounts.remove("portdir")
> >  			#self.mountmap["portdir"] = None
> >  		if os.uname()[0] == "Linux":
> 
> Not quite correct.  Note the not in that if...

Ah, good point ;).

> diff --git a/catalyst/targets/generic_stage_target.py b/catalyst/targets/generic_stage_target.py
> index 9c39d00..eaf2c1f 100644
> @@ -215,13 +215,14 @@ class generic_stage_target(generic_target):
>  		self.mountmap = SOURCE_MOUNTS_DEFAULTS.copy()
>  		# update them from settings
>  		self.mountmap["distdir"] = self.settings["distdir"]
> -		self.mountmap["portdir"] = normpath("/".join([
> -			self.settings["snapshot_cache_path"],
> -			self.settings["repo_name"],
> -			]))
>  		if "SNAPCACHE" not in self.settings:
>  			self.mounts.remove("portdir")
> -			#self.mountmap["portdir"] = None
> +			self.mountmap["portdir"] = None
> +		else:
> +			self.mountmap["portdir"] = normpath("/".join([
> +				self.settings["snapshot_cache_path"],
> +				self.settings["repo_name"],
> +				]))

This looks good to me.

Cheers,
Trevor

-- 
This email may be signed or encrypted with GnuPG (http://www.gnupg.org).
For more information, see http://en.wikipedia.org/wiki/Pretty_Good_Privacy

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 836 bytes --]