[gentoo-catalyst] [PATCH 1/2] snapshot: do not preserve owner/group

[gentoo-catalyst] [PATCH 1/2] snapshot: do not preserve owner/group

[Mike Frysinger]

There's no need to preserve ownership on the files we sync into the
snapshot, so just let rsync create them as it will (current uid/gid).
This slightly speeds things up when the files are owned by a non-root
user too.
---
 catalyst/targets/snapshot.py | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/catalyst/targets/snapshot.py b/catalyst/targets/snapshot.py
index 6007aaa..e1ca7b7 100644
--- a/catalyst/targets/snapshot.py
+++ b/catalyst/targets/snapshot.py
@@ -56,7 +56,7 @@ class snapshot(TargetBase, GenBase):
 		ensure_dirs(mytmp)
 
 		target_snapshot = self.settings["portdir"] + "/ " + mytmp + "/%s/" % self.settings["repo_name"]
-		cmd("rsync -a --delete --exclude /packages/ --exclude /distfiles/ " +
+		cmd("rsync -a --no-o --no-g --delete --exclude /packages/ --exclude /distfiles/ " +
 			"--exclude /local/ --exclude CVS/ --exclude .svn --filter=H_**/files/digest-* " +
 			target_snapshot,
 			"Snapshot failure", env=self.env)
-- 
2.5.2

[gentoo-catalyst] [PATCH 2/2] catalyst: create namespaces for building

[Mike Frysinger]

This helps isolate the catalyst builds from the rest of the system
and allows us to build as non-root user in more cases.  This might
not work everywhere, but it's a start (snapshot generation works).
---
 catalyst/main.py | 9 +++++++++
 1 file changed, 9 insertions(+)

diff --git a/catalyst/main.py b/catalyst/main.py
index 65e1431..f984653 100644
--- a/catalyst/main.py
+++ b/catalyst/main.py
@@ -10,6 +10,8 @@ import argparse
 import os
 import sys
 
+from snakeoil.process import namespaces
+
 __selfpath__ = os.path.abspath(os.path.dirname(__file__))
 
 from DeComp.definitions import (COMPRESS_DEFINITIONS, DECOMPRESS_DEFINITIONS,
@@ -336,6 +338,13 @@ def main():
 	if "target" not in addlargs:
 		raise CatalystError("Required value \"target\" not specified.")
 
+	# Start off by creating unique namespaces to run in.
+	namespaces.simple_unshare(
+		mount=True, uts=True, ipc=True, pid=True, net=False, user=True,
+		hostname='catalyst')
+
+	# We should be root now, either a real root, or in a userns as root.
+	# If we aren't, then we've failed, and need to abort.
 	if os.getuid() != 0:
 		# catalyst cannot be run as a normal user due to chroots, mounts, etc
 		print "!!! catalyst: This script requires root privileges to operate"
-- 
2.5.2

Re: [gentoo-catalyst] [PATCH 2/2] catalyst: create namespaces for building

[Brian Dolbec]

On Tue,  6 Oct 2015 16:32:53 -0400
Mike Frysinger <vapier@gentoo.org> wrote:

> This helps isolate the catalyst builds from the rest of the system
> and allows us to build as non-root user in more cases.  This might
> not work everywhere, but it's a start (snapshot generation works).
> ---
>  catalyst/main.py | 9 +++++++++
>  1 file changed, 9 insertions(+)
> 
> diff --git a/catalyst/main.py b/catalyst/main.py
> index 65e1431..f984653 100644
> --- a/catalyst/main.py
> +++ b/catalyst/main.py
> @@ -10,6 +10,8 @@ import argparse
>  import os
>  import sys
>  
> +from snakeoil.process import namespaces
> +
>  __selfpath__ = os.path.abspath(os.path.dirname(__file__))
>  
>  from DeComp.definitions import (COMPRESS_DEFINITIONS,
> DECOMPRESS_DEFINITIONS, @@ -336,6 +338,13 @@ def main():
>  	if "target" not in addlargs:
>  		raise CatalystError("Required value \"target\" not
> specified.") 
> +	# Start off by creating unique namespaces to run in.
> +	namespaces.simple_unshare(
> +		mount=True, uts=True, ipc=True, pid=True, net=False,
> user=True,
> +		hostname='catalyst')
> +
> +	# We should be root now, either a real root, or in a userns
> as root.
> +	# If we aren't, then we've failed, and need to abort.
>  	if os.getuid() != 0:
>  		# catalyst cannot be run as a normal user due to
> chroots, mounts, etc print "!!! catalyst: This script requires root
> privileges to operate"

I was hoping to get a release out before merging much more, but, I
doubt this will affect stability.

looks good 

-- 
Brian Dolbec <dolsen>

Re: [gentoo-catalyst] [PATCH 2/2] catalyst: create namespaces for building

[Mike Frysinger]

[-- Attachment #1: Type: text/plain, Size: 1809 bytes --]

On 09 Nov 2015 22:45, Brian Dolbec wrote:
> On Tue,  6 Oct 2015 16:32:53 -0400 Mike Frysinger wrote:
> > This helps isolate the catalyst builds from the rest of the system
> > and allows us to build as non-root user in more cases.  This might
> > not work everywhere, but it's a start (snapshot generation works).
> > ---
> >  catalyst/main.py | 9 +++++++++
> >  1 file changed, 9 insertions(+)
> > 
> > diff --git a/catalyst/main.py b/catalyst/main.py
> > index 65e1431..f984653 100644
> > --- a/catalyst/main.py
> > +++ b/catalyst/main.py
> > @@ -10,6 +10,8 @@ import argparse
> >  import os
> >  import sys
> >  
> > +from snakeoil.process import namespaces
> > +
> >  __selfpath__ = os.path.abspath(os.path.dirname(__file__))
> >  
> >  from DeComp.definitions import (COMPRESS_DEFINITIONS,
> > DECOMPRESS_DEFINITIONS, @@ -336,6 +338,13 @@ def main():
> >  	if "target" not in addlargs:
> >  		raise CatalystError("Required value \"target\" not
> > specified.") 
> > +	# Start off by creating unique namespaces to run in.
> > +	namespaces.simple_unshare(
> > +		mount=True, uts=True, ipc=True, pid=True, net=False,
> > user=True,
> > +		hostname='catalyst')
> > +
> > +	# We should be root now, either a real root, or in a userns
> > as root.
> > +	# If we aren't, then we've failed, and need to abort.
> >  	if os.getuid() != 0:
> >  		# catalyst cannot be run as a normal user due to
> > chroots, mounts, etc print "!!! catalyst: This script requires root
> > privileges to operate"
> 
> I was hoping to get a release out before merging much more, but, I
> doubt this will affect stability.

it does ;).  i haven't fully debugged it yet, but i think the use of pid
namespaces messes up signal handling.  i'll follow up when i'm happy with
its state.
-mike

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 819 bytes --]

Re: [gentoo-catalyst] [PATCH 2/2] catalyst: create namespaces for building

[Brian Dolbec]

On Tue, 10 Nov 2015 01:49:07 -0500
Mike Frysinger <vapier@gentoo.org> wrote:

> On 09 Nov 2015 22:45, Brian Dolbec wrote:
> > On Tue,  6 Oct 2015 16:32:53 -0400 Mike Frysinger wrote:  
> > > This helps isolate the catalyst builds from the rest of the system
> > > and allows us to build as non-root user in more cases.  This might
> > > not work everywhere, but it's a start (snapshot generation works).
> > > ---
> > >  catalyst/main.py | 9 +++++++++
> > >  1 file changed, 9 insertions(+)
> > > 
> > > diff --git a/catalyst/main.py b/catalyst/main.py
> > > index 65e1431..f984653 100644
> > > --- a/catalyst/main.py
> > > +++ b/catalyst/main.py
> > > @@ -10,6 +10,8 @@ import argparse
> > >  import os
> > >  import sys
> > >  
> > > +from snakeoil.process import namespaces
> > > +
> > >  __selfpath__ = os.path.abspath(os.path.dirname(__file__))
> > >  
> > >  from DeComp.definitions import (COMPRESS_DEFINITIONS,
> > > DECOMPRESS_DEFINITIONS, @@ -336,6 +338,13 @@ def main():
> > >  	if "target" not in addlargs:
> > >  		raise CatalystError("Required value \"target\"
> > > not specified.") 
> > > +	# Start off by creating unique namespaces to run in.
> > > +	namespaces.simple_unshare(
> > > +		mount=True, uts=True, ipc=True, pid=True,
> > > net=False, user=True,
> > > +		hostname='catalyst')
> > > +
> > > +	# We should be root now, either a real root, or in a
> > > userns as root.
> > > +	# If we aren't, then we've failed, and need to abort.
> > >  	if os.getuid() != 0:
> > >  		# catalyst cannot be run as a normal user due to
> > > chroots, mounts, etc print "!!! catalyst: This script requires
> > > root privileges to operate"  
> > 
> > I was hoping to get a release out before merging much more, but, I
> > doubt this will affect stability.  
> 
> it does ;).  i haven't fully debugged it yet, but i think the use of
> pid namespaces messes up signal handling.  i'll follow up when i'm
> happy with its state.
> -mike

sounds good :)  Thanks

-- 
Brian Dolbec <dolsen>

Re: [gentoo-catalyst] [PATCH 1/2] snapshot: do not preserve owner/group

[Mike Frysinger]

[-- Attachment #1: Type: text/plain, Size: 104 bytes --]

i've pushed just the first one here.  the namespaces one i'd like more
feedback/testing on first.
-mike

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 819 bytes --]