[gentoo-portage-dev] [PATCH 0/4] sync: sync-umask and sync-user support

[gentoo-portage-dev] [PATCH 0/4] sync: sync-umask and sync-user support

[Michał Górny]

Hello, friends.

A quick batch adding sync-umask and sync-user overrides for syncing.
Should work both for new clones and repo updates -- assuming the change
you do is meaningful :).

[gentoo-portage-dev] [PATCH 1/4] sync: allow overriding sync-umask for the repository

[Michał Górny]

---
 pym/portage/repository/config.py | 16 ++++++++++++----
 pym/portage/sync/controller.py   |  3 +++
 2 files changed, 15 insertions(+), 4 deletions(-)

diff --git a/pym/portage/repository/config.py b/pym/portage/repository/config.py
index d37ce6a..678cc68 100644
--- a/pym/portage/repository/config.py
+++ b/pym/portage/repository/config.py
@@ -85,8 +85,9 @@ class RepoConfig(object):
 		'main_repo', 'manifest_hashes', 'masters', 'missing_repo_name',
 		'name', 'portage1_profiles', 'portage1_profiles_compat', 'priority',
 		'profile_formats', 'sign_commit', 'sign_manifest', 'sync_cvs_repo',
-		'sync_type', 'sync_uri', 'thin_manifest', 'update_changelog',
-		'user_location', '_eapis_banned', '_eapis_deprecated', '_masters_orig')
+		'sync_type', 'sync_umask', 'sync_uri', 'thin_manifest',
+		'update_changelog', 'user_location', '_eapis_banned',
+		'_eapis_deprecated', '_masters_orig')
 
 	def __init__(self, name, repo_opts, local_config=True):
 		"""Build a RepoConfig with options in repo_opts
@@ -154,6 +155,11 @@ class RepoConfig(object):
 			sync_type = sync_type.strip()
 		self.sync_type = sync_type or None
 
+		sync_umask = repo_opts.get('sync-umask')
+		if sync_umask is not None:
+			sync_umask = sync_umask.strip()
+		self.sync_umask = sync_umask or None
+
 		sync_uri = repo_opts.get('sync-uri')
 		if sync_uri is not None:
 			sync_uri = sync_uri.strip()
@@ -375,6 +381,8 @@ class RepoConfig(object):
 			repo_msg.append(indent + "sync-cvs-repo: " + self.sync_cvs_repo)
 		if self.sync_type:
 			repo_msg.append(indent + "sync-type: " + self.sync_type)
+		if self.sync_umask:
+			repo_msg.append(indent + "sync-umask: " + self.sync_umask)
 		if self.sync_uri:
 			repo_msg.append(indent + "sync-uri: " + self.sync_uri)
 		if self.masters:
@@ -464,7 +472,7 @@ class RepoConfigLoader(object):
 						# repos.conf is allowed to override.
 						for k in ('aliases', 'auto_sync', 'eclass_overrides',
 							'force', 'masters', 'priority', 'sync_cvs_repo',
-							'sync_type', 'sync_uri',
+							'sync_type', 'sync_umask', 'sync_uri',
 							):
 							v = getattr(repos_conf_opts, k, None)
 							if v is not None:
@@ -915,7 +923,7 @@ class RepoConfigLoader(object):
 	def config_string(self):
 		str_or_int_keys = ("auto_sync", "format", "location",
 			"main_repo", "priority", "sync_cvs_repo",
-			"sync_type", "sync_uri")
+			"sync_type", "sync_umask", "sync_uri")
 		str_tuple_keys = ("aliases", "eclass_overrides", "force")
 		repo_config_tuple_keys = ("masters",)
 		keys = str_or_int_keys + str_tuple_keys + repo_config_tuple_keys
diff --git a/pym/portage/sync/controller.py b/pym/portage/sync/controller.py
index 21aa7a7..0e5efb6 100644
--- a/pym/portage/sync/controller.py
+++ b/pym/portage/sync/controller.py
@@ -213,6 +213,9 @@ class SyncManager(object):
 				if not st.st_mode & 0o020:
 					umask = umask | 0o020
 				spawn_kwargs["umask"] = umask
+		# override the defaults when sync_umask is set
+		if repo.sync_umask is not None:
+			spawn_kwargs["umask"] = int(repo.sync_umask, 8)
 		self.spawn_kwargs = spawn_kwargs
 
 		if self.usersync_uid is not None:
-- 
2.2.0

Re: [gentoo-portage-dev] [PATCH 1/4] sync: allow overriding sync-umask for the repository

[Zac Medico]

On 12/05/2014 04:03 PM, Michał Górny wrote:
> ---
>  pym/portage/repository/config.py | 16 ++++++++++++----
>  pym/portage/sync/controller.py   |  3 +++
>  2 files changed, 15 insertions(+), 4 deletions(-)

LGTM.
-- 
Thanks,
Zac

[gentoo-portage-dev] [PATCH] sync: allow overriding sync-umask for the repository

[Michał Górny]

---
Changes:
* added a entry in the manpage

 man/portage.5                    |  5 +++++
 pym/portage/repository/config.py | 16 ++++++++++++----
 pym/portage/sync/controller.py   |  3 +++
 3 files changed, 20 insertions(+), 4 deletions(-)

diff --git a/man/portage.5 b/man/portage.5
index 150294b..8c3d389 100644
--- a/man/portage.5
+++ b/man/portage.5
@@ -901,6 +901,11 @@ Valid non\-empty values: cvs, git, rsync
 This attribute can be set to empty value to disable synchronization of given
 repository. Empty value is default.
 .TP
+.B sync\-umask
+Specifies umask used to synchronize the repository.
+.br
+Takes an octal permission mask, e.g. 022.
+.TP
 .B sync\-uri
 Specifies URI of repository used for synchronization performed by `emerge
 \-\-sync`.
diff --git a/pym/portage/repository/config.py b/pym/portage/repository/config.py
index d37ce6a..678cc68 100644
--- a/pym/portage/repository/config.py
+++ b/pym/portage/repository/config.py
@@ -85,8 +85,9 @@ class RepoConfig(object):
 		'main_repo', 'manifest_hashes', 'masters', 'missing_repo_name',
 		'name', 'portage1_profiles', 'portage1_profiles_compat', 'priority',
 		'profile_formats', 'sign_commit', 'sign_manifest', 'sync_cvs_repo',
-		'sync_type', 'sync_uri', 'thin_manifest', 'update_changelog',
-		'user_location', '_eapis_banned', '_eapis_deprecated', '_masters_orig')
+		'sync_type', 'sync_umask', 'sync_uri', 'thin_manifest',
+		'update_changelog', 'user_location', '_eapis_banned',
+		'_eapis_deprecated', '_masters_orig')
 
 	def __init__(self, name, repo_opts, local_config=True):
 		"""Build a RepoConfig with options in repo_opts
@@ -154,6 +155,11 @@ class RepoConfig(object):
 			sync_type = sync_type.strip()
 		self.sync_type = sync_type or None
 
+		sync_umask = repo_opts.get('sync-umask')
+		if sync_umask is not None:
+			sync_umask = sync_umask.strip()
+		self.sync_umask = sync_umask or None
+
 		sync_uri = repo_opts.get('sync-uri')
 		if sync_uri is not None:
 			sync_uri = sync_uri.strip()
@@ -375,6 +381,8 @@ class RepoConfig(object):
 			repo_msg.append(indent + "sync-cvs-repo: " + self.sync_cvs_repo)
 		if self.sync_type:
 			repo_msg.append(indent + "sync-type: " + self.sync_type)
+		if self.sync_umask:
+			repo_msg.append(indent + "sync-umask: " + self.sync_umask)
 		if self.sync_uri:
 			repo_msg.append(indent + "sync-uri: " + self.sync_uri)
 		if self.masters:
@@ -464,7 +472,7 @@ class RepoConfigLoader(object):
 						# repos.conf is allowed to override.
 						for k in ('aliases', 'auto_sync', 'eclass_overrides',
 							'force', 'masters', 'priority', 'sync_cvs_repo',
-							'sync_type', 'sync_uri',
+							'sync_type', 'sync_umask', 'sync_uri',
 							):
 							v = getattr(repos_conf_opts, k, None)
 							if v is not None:
@@ -915,7 +923,7 @@ class RepoConfigLoader(object):
 	def config_string(self):
 		str_or_int_keys = ("auto_sync", "format", "location",
 			"main_repo", "priority", "sync_cvs_repo",
-			"sync_type", "sync_uri")
+			"sync_type", "sync_umask", "sync_uri")
 		str_tuple_keys = ("aliases", "eclass_overrides", "force")
 		repo_config_tuple_keys = ("masters",)
 		keys = str_or_int_keys + str_tuple_keys + repo_config_tuple_keys
diff --git a/pym/portage/sync/controller.py b/pym/portage/sync/controller.py
index 41f3830..1c8c756 100644
--- a/pym/portage/sync/controller.py
+++ b/pym/portage/sync/controller.py
@@ -221,6 +221,9 @@ class SyncManager(object):
 				if not st.st_mode & 0o020:
 					umask = umask | 0o020
 				spawn_kwargs["umask"] = umask
+		# override the defaults when sync_umask is set
+		if repo.sync_umask is not None:
+			spawn_kwargs["umask"] = int(repo.sync_umask, 8)
 		self.spawn_kwargs = spawn_kwargs
 
 		if self.usersync_uid is not None:
-- 
2.2.0

[gentoo-portage-dev] [PATCH 2/4] sync: allow overriding sync-user for the repository

[Michał Górny]

---
 pym/portage/repository/config.py | 13 +++++++---
 pym/portage/sync/controller.py   | 52 +++++++++++++++++++++++++++++++++++++++-
 2 files changed, 61 insertions(+), 4 deletions(-)

diff --git a/pym/portage/repository/config.py b/pym/portage/repository/config.py
index 678cc68..f45684b 100644
--- a/pym/portage/repository/config.py
+++ b/pym/portage/repository/config.py
@@ -85,7 +85,7 @@ class RepoConfig(object):
 		'main_repo', 'manifest_hashes', 'masters', 'missing_repo_name',
 		'name', 'portage1_profiles', 'portage1_profiles_compat', 'priority',
 		'profile_formats', 'sign_commit', 'sign_manifest', 'sync_cvs_repo',
-		'sync_type', 'sync_umask', 'sync_uri', 'thin_manifest',
+		'sync_type', 'sync_umask', 'sync_uri', 'sync_user', 'thin_manifest',
 		'update_changelog', 'user_location', '_eapis_banned',
 		'_eapis_deprecated', '_masters_orig')
 
@@ -165,6 +165,11 @@ class RepoConfig(object):
 			sync_uri = sync_uri.strip()
 		self.sync_uri = sync_uri or None
 
+		sync_user = repo_opts.get('sync-user')
+		if sync_user is not None:
+			sync_user = sync_user.strip()
+		self.sync_user = sync_user or None
+
 		auto_sync = repo_opts.get('auto-sync')
 		if auto_sync is not None:
 			auto_sync = auto_sync.strip().lower()
@@ -385,6 +390,8 @@ class RepoConfig(object):
 			repo_msg.append(indent + "sync-umask: " + self.sync_umask)
 		if self.sync_uri:
 			repo_msg.append(indent + "sync-uri: " + self.sync_uri)
+		if self.sync_user:
+			repo_msg.append(indent + "sync-user: " + self.sync_user)
 		if self.masters:
 			repo_msg.append(indent + "masters: " + " ".join(master.name for master in self.masters))
 		if self.priority is not None:
@@ -472,7 +479,7 @@ class RepoConfigLoader(object):
 						# repos.conf is allowed to override.
 						for k in ('aliases', 'auto_sync', 'eclass_overrides',
 							'force', 'masters', 'priority', 'sync_cvs_repo',
-							'sync_type', 'sync_umask', 'sync_uri',
+							'sync_type', 'sync_umask', 'sync_uri', 'sync_user',
 							):
 							v = getattr(repos_conf_opts, k, None)
 							if v is not None:
@@ -923,7 +930,7 @@ class RepoConfigLoader(object):
 	def config_string(self):
 		str_or_int_keys = ("auto_sync", "format", "location",
 			"main_repo", "priority", "sync_cvs_repo",
-			"sync_type", "sync_umask", "sync_uri")
+			"sync_type", "sync_umask", "sync_uri", 'sync_user')
 		str_tuple_keys = ("aliases", "eclass_overrides", "force")
 		repo_config_tuple_keys = ("masters",)
 		keys = str_or_int_keys + str_tuple_keys + repo_config_tuple_keys
diff --git a/pym/portage/sync/controller.py b/pym/portage/sync/controller.py
index 0e5efb6..3104524 100644
--- a/pym/portage/sync/controller.py
+++ b/pym/portage/sync/controller.py
@@ -6,6 +6,7 @@ from __future__ import print_function
 
 import sys
 import logging
+import grp
 import pwd
 
 import portage
@@ -193,7 +194,56 @@ class SyncManager(object):
 		self.usersync_uid = None
 		spawn_kwargs = {}
 		spawn_kwargs["env"] = self.settings.environ()
-		if ('usersync' in self.settings.features and
+		if repo.sync_user is not None:
+			def get_sync_user_data(sync_user):
+				spl = sync_user.split(':', 1)
+				if not spl[0]:
+					writemsg("!!! Invalid sync-user = %s\n" % sync_user,
+							noiselevel=-1)
+					return ()
+
+				user = spl[0]
+				try:
+					try:
+						# maybe it's a uid?
+						uid = int(user)
+					except ValueError:
+						pw = pwd.getpwnam(user)
+					else:
+						pw = pwd.getpwuid(uid)
+				except KeyError:
+					writemsg("!!! User '%s' invalid or does not exist\n"
+							% user, noiselevel=-1)
+					return ()
+
+				if len(spl) > 1:
+					group = spl[1]
+					try:
+						try:
+							# maybe it's a gid?
+							gid = int(group)
+						except ValueError:
+							gp = grp.getgrnam(group)
+						else:
+							pw = grp.getgrgid(gid)
+					except KeyError:
+						writemsg("!!! Group '%s' invalid or does not exist\n"
+								% group, noiselevel=-1)
+						return ()
+
+					gr = gp.gr_gid
+				else:
+					gr = pw.pw_gid
+
+				return (pw.pw_uid, gr, pw.pw_dir)
+
+			# user or user:group
+			(uid, gid, home) = get_sync_user_data(repo.sync_user)
+			spawn_kwargs["uid"] = uid
+			spawn_kwargs["gid"] = gid
+			spawn_kwargs["groups"] = [gid]
+			spawn_kwargs["env"]["HOME"] = home
+		elif ('usersync' in self.settings.features and
 			portage.data.secpass >= 2 and
 			(st.st_uid != os.getuid() and st.st_mode & 0o700 or
 			st.st_gid != os.getgid() and st.st_mode & 0o070)):
-- 
2.2.0

Re: [gentoo-portage-dev] [PATCH 2/4] sync: allow overriding sync-user for the repository

[Zac Medico]

On 12/05/2014 04:03 PM, Michał Górny wrote:
> +				if len(spl) > 1:
> +					group = spl[1]
> +					try:
> +						try:
> +							# maybe it's a gid?
> +							gid = int(group)
> +						except ValueError:
> +							gp = grp.getgrnam(group)
> +						else:
> +							pw = grp.getgrgid(gid)

Does this behave the same as other tools (like chown) for strings like "12345" ? I vaguely recall
someone saying that chown tries a string lookup first, and if that fails then it tries an integer
lookup.

> +					except KeyError:
> +						writemsg("!!! Group '%s' invalid or does not exist\n"
> +								% group, noiselevel=-1)
> +						return ()
> +
> +					gr = gp.gr_gid
> +				else:
> +					gr = pw.pw_gid
> +
> +				return (pw.pw_uid, gr, pw.pw_dir)
> +
> +			# user or user:group
> +			(uid, gid, home) = get_sync_user_data(repo.sync_user)

If it returns () above, this will raise a ValueError, right?

>>> (x, y, z) = ()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ValueError: need more than 0 values to unpack

Otherwise, LGTM.
-- 
Thanks,
Zac

[gentoo-portage-dev] [PATCH] sync: allow overriding sync-user for the repository

[Michał Górny]

---
Changes:
* manpage entry
* perform username lookup before UID lookup (POSIX agrees)
* support specifying ':group' only
* fix ValueError on errors

 man/portage.5                    | 12 ++++++++++
 pym/portage/repository/config.py | 13 +++++++---
 pym/portage/sync/controller.py   | 51 +++++++++++++++++++++++++++++++++++++++-
 3 files changed, 72 insertions(+), 4 deletions(-)

diff --git a/man/portage.5 b/man/portage.5
index 8c3d389..4a02c64 100644
--- a/man/portage.5
+++ b/man/portage.5
@@ -934,6 +934,18 @@ ssh://ssh\-user@192.168.0.1:22/\\${HOME}/portage\-storage
 .TP
 Note: For the ssh:// scheme, key\-based authentication might be of interest.
 .RE
+.TP
+.B sync\-user
+Specifies the credentials used to perform the synchronization.
+.br
+Syntax: [user][:group]
+.br
+If only user is provided, the primary group of the user will be used.
+If only group is provided, the current user will be preserved and only
+group id will be changed.
+.br
+This key takes precedence over FEATURES=userpriv. If user or group id
+is provided, Portage no longer uses owner of the directory.
 .RE
 
 .I Example:
diff --git a/pym/portage/repository/config.py b/pym/portage/repository/config.py
index 678cc68..f45684b 100644
--- a/pym/portage/repository/config.py
+++ b/pym/portage/repository/config.py
@@ -85,7 +85,7 @@ class RepoConfig(object):
 		'main_repo', 'manifest_hashes', 'masters', 'missing_repo_name',
 		'name', 'portage1_profiles', 'portage1_profiles_compat', 'priority',
 		'profile_formats', 'sign_commit', 'sign_manifest', 'sync_cvs_repo',
-		'sync_type', 'sync_umask', 'sync_uri', 'thin_manifest',
+		'sync_type', 'sync_umask', 'sync_uri', 'sync_user', 'thin_manifest',
 		'update_changelog', 'user_location', '_eapis_banned',
 		'_eapis_deprecated', '_masters_orig')
 
@@ -165,6 +165,11 @@ class RepoConfig(object):
 			sync_uri = sync_uri.strip()
 		self.sync_uri = sync_uri or None
 
+		sync_user = repo_opts.get('sync-user')
+		if sync_user is not None:
+			sync_user = sync_user.strip()
+		self.sync_user = sync_user or None
+
 		auto_sync = repo_opts.get('auto-sync')
 		if auto_sync is not None:
 			auto_sync = auto_sync.strip().lower()
@@ -385,6 +390,8 @@ class RepoConfig(object):
 			repo_msg.append(indent + "sync-umask: " + self.sync_umask)
 		if self.sync_uri:
 			repo_msg.append(indent + "sync-uri: " + self.sync_uri)
+		if self.sync_user:
+			repo_msg.append(indent + "sync-user: " + self.sync_user)
 		if self.masters:
 			repo_msg.append(indent + "masters: " + " ".join(master.name for master in self.masters))
 		if self.priority is not None:
@@ -472,7 +479,7 @@ class RepoConfigLoader(object):
 						# repos.conf is allowed to override.
 						for k in ('aliases', 'auto_sync', 'eclass_overrides',
 							'force', 'masters', 'priority', 'sync_cvs_repo',
-							'sync_type', 'sync_umask', 'sync_uri',
+							'sync_type', 'sync_umask', 'sync_uri', 'sync_user',
 							):
 							v = getattr(repos_conf_opts, k, None)
 							if v is not None:
@@ -923,7 +930,7 @@ class RepoConfigLoader(object):
 	def config_string(self):
 		str_or_int_keys = ("auto_sync", "format", "location",
 			"main_repo", "priority", "sync_cvs_repo",
-			"sync_type", "sync_umask", "sync_uri")
+			"sync_type", "sync_umask", "sync_uri", 'sync_user')
 		str_tuple_keys = ("aliases", "eclass_overrides", "force")
 		repo_config_tuple_keys = ("masters",)
 		keys = str_or_int_keys + str_tuple_keys + repo_config_tuple_keys
diff --git a/pym/portage/sync/controller.py b/pym/portage/sync/controller.py
index 1c8c756..de803b7 100644
--- a/pym/portage/sync/controller.py
+++ b/pym/portage/sync/controller.py
@@ -6,6 +6,7 @@ from __future__ import print_function
 
 import sys
 import logging
+import grp
 import pwd
 
 import portage
@@ -201,7 +202,55 @@ class SyncManager(object):
 		self.usersync_uid = None
 		spawn_kwargs = {}
 		spawn_kwargs["env"] = self.settings.environ()
-		if ('usersync' in self.settings.features and
+		if repo.sync_user is not None:
+			def get_sync_user_data(sync_user):
+				user = None
+				group = None
+				home = None
+
+				spl = sync_user.split(':', 1)
+				if spl[0]:
+					username = spl[0]
+					try:
+						try:
+							pw = pwd.getpwnam(username)
+						except KeyError:
+							pw = pwd.getpwuid(int(username))
+					except (ValueError, KeyError):
+						writemsg("!!! User '%s' invalid or does not exist\n"
+								% username, noiselevel=-1)
+						return (user, group, home)
+					user = pw.pw_uid
+					group = pw.pw_gid
+					home = pw.pw_dir
+
+				if len(spl) > 1:
+					groupname = spl[1]
+					try:
+						try:
+							gp = grp.getgrnam(groupname)
+						except KeyError:
+							pw = grp.getgrgid(int(groupname))
+					except (ValueError, KeyError):
+						writemsg("!!! Group '%s' invalid or does not exist\n"
+								% groupname, noiselevel=-1)
+						return (user, group, home)
+
+					group = gp.gr_gid
+
+				return (user, group, home)
+
+			# user or user:group
+			(uid, gid, home) = get_sync_user_data(repo.sync_user)
+			if uid is not None
+				spawn_kwargs["uid"] = uid
+				self.usersync_uid = uid
+			if gid is not None:
+				spawn_kwargs["gid"] = gid
+				spawn_kwargs["groups"] = [gid]
+			if home is not None:
+				spawn_kwargs["env"]["HOME"] = home
+		elif ('usersync' in self.settings.features and
 			portage.data.secpass >= 2 and
 			(st.st_uid != os.getuid() and st.st_mode & 0o700 or
 			st.st_gid != os.getgid() and st.st_mode & 0o070)):
-- 
2.2.0

Re: [gentoo-portage-dev] [PATCH] sync: allow overriding sync-user for the repository

[Michał Górny]

[-- Attachment #1: Type: text/plain, Size: 657 bytes --]

Dnia 2014-12-07, o godz. 10:01:23
Michał Górny <mgorny@gentoo.org> napisał(a):

> +			if uid is not None

Missing ':' here, I won't resend the patch for it :P.

> +				spawn_kwargs["uid"] = uid
> +				self.usersync_uid = uid
> +			if gid is not None:
> +				spawn_kwargs["gid"] = gid
> +				spawn_kwargs["groups"] = [gid]
> +			if home is not None:
> +				spawn_kwargs["env"]["HOME"] = home
> +		elif ('usersync' in self.settings.features and
>  			portage.data.secpass >= 2 and
>  			(st.st_uid != os.getuid() and st.st_mode & 0o700 or
>  			st.st_gid != os.getgid() and st.st_mode & 0o070)):



-- 
Best regards,
Michał Górny

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 949 bytes --]

Re: [gentoo-portage-dev] [PATCH] sync: allow overriding sync-user for the repository

[Zac Medico]

On 12/07/2014 01:07 AM, Michał Górny wrote:
> Dnia 2014-12-07, o godz. 10:01:23
> Michał Górny <mgorny@gentoo.org> napisał(a):
> 
>> +			if uid is not None
> 
> Missing ':' here, I won't resend the patch for it :P.
> 
>> +				spawn_kwargs["uid"] = uid
>> +				self.usersync_uid = uid
>> +			if gid is not None:
>> +				spawn_kwargs["gid"] = gid
>> +				spawn_kwargs["groups"] = [gid]
>> +			if home is not None:
>> +				spawn_kwargs["env"]["HOME"] = home
>> +		elif ('usersync' in self.settings.features and
>>  			portage.data.secpass >= 2 and
>>  			(st.st_uid != os.getuid() and st.st_mode & 0o700 or
>>  			st.st_gid != os.getgid() and st.st_mode & 0o070)):
> 
> 
> 

LGTM.
-- 
Thanks,
Zac

Re: [gentoo-portage-dev] [PATCH] sync: allow overriding sync-user for the repository

[Arfrever Frehtes Taifersar Arahesis]

[-- Attachment #1: Type: Text/Plain, Size: 105 bytes --]

I would suggest to have a separate sync-group attribute and to support only user in sync-user attribute.

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 819 bytes --]

[gentoo-portage-dev] [PATCH 3/4] sync: ensure sync_{umask,user} is respected when creating repo

[Michał Górny]

---
 pym/portage/sync/controller.py | 24 ++++++++++++++++++------
 1 file changed, 18 insertions(+), 6 deletions(-)

diff --git a/pym/portage/sync/controller.py b/pym/portage/sync/controller.py
index 3104524..4fa1e5a 100644
--- a/pym/portage/sync/controller.py
+++ b/pym/portage/sync/controller.py
@@ -185,11 +185,6 @@ class SyncManager(object):
 			st = os.stat(repo.location)
 		except OSError:
 			st = None
-		if st is None:
-			writemsg_level(">>> '%s' not found, creating it."
-				% _unicode_decode(repo.location))
-			portage.util.ensure_dirs(repo.location, mode=0o755)
-			st = os.stat(repo.location)
 
 		self.usersync_uid = None
 		spawn_kwargs = {}
@@ -243,7 +238,24 @@ class SyncManager(object):
 			spawn_kwargs["gid"] = gid
 			spawn_kwargs["groups"] = [gid]
 			spawn_kwargs["env"]["HOME"] = home
-		elif ('usersync' in self.settings.features and
+
+		if st is None:
+			perms = {'mode': 0o755}
+			# respect sync-user if set
+			if 'umask' in spawn_kwargs:
+				perms['mode'] &= ~spawn_kwargs['umask']
+			if 'uid' in spawn_kwargs:
+				perms['uid'] = spawn_kwargs['uid']
+			if 'gid' in spawn_kwargs:
+				perms['gid'] = spawn_kwargs['gid']
+
+			writemsg_level(">>> '%s' not found, creating it."
+				% _unicode_decode(repo.location))
+			portage.util.ensure_dirs(repo.location, **perms)
+			st = os.stat(repo.location)
+
+		if (repo.sync_user is None and
+			'usersync' in self.settings.features and
 			portage.data.secpass >= 2 and
 			(st.st_uid != os.getuid() and st.st_mode & 0o700 or
 			st.st_gid != os.getgid() and st.st_mode & 0o070)):
-- 
2.2.0

Re: [gentoo-portage-dev] [PATCH 3/4] sync: ensure sync_{umask,user} is respected when creating repo

[Zac Medico]

On 12/05/2014 04:03 PM, Michał Górny wrote:
> ---
>  pym/portage/sync/controller.py | 24 ++++++++++++++++++------
>  1 file changed, 18 insertions(+), 6 deletions(-)

LGTM.
-- 
Thanks,
Zac

[gentoo-portage-dev] [PATCH 4/4] sync: Add backwards compat with SYNC_{UMASK,USER} variables

[Michał Górny]

Support SYNC_UMASK and SYNC_USER variables that were used in Funtoo
Portage, as fallbacks to sync-umask and sync-user repo keys.
---
 pym/portage/package/ebuild/_config/special_env_vars.py |  2 +-
 pym/portage/package/ebuild/config.py                   | 12 ++++++++++++
 pym/portage/repository/config.py                       | 13 +++++++++++++
 3 files changed, 26 insertions(+), 1 deletion(-)

diff --git a/pym/portage/package/ebuild/_config/special_env_vars.py b/pym/portage/package/ebuild/_config/special_env_vars.py
index 387f4ae..5c13e5b 100644
--- a/pym/portage/package/ebuild/_config/special_env_vars.py
+++ b/pym/portage/package/ebuild/_config/special_env_vars.py
@@ -180,7 +180,7 @@ environ_filter += [
 	"RESUMECOMMAND_HTTP", "RESUMECOMMAND_HTTPS",
 	"RESUMECOMMAND_RSYNC", "RESUMECOMMAND_SFTP",
 	"UNINSTALL_IGNORE", "USE_EXPAND_HIDDEN", "USE_ORDER",
-	"__PORTAGE_HELPER"
+	"__PORTAGE_HELPER", "SYNC_UMASK", "SYNC_USER",
 ]
 
 # No longer supported variables
diff --git a/pym/portage/package/ebuild/config.py b/pym/portage/package/ebuild/config.py
index 59e239b..0ca2bfd 100644
--- a/pym/portage/package/ebuild/config.py
+++ b/pym/portage/package/ebuild/config.py
@@ -499,6 +499,8 @@ class config(object):
 			portdir = ""
 			portdir_overlay = ""
 			portdir_sync = None
+			sync_umask = None
+			sync_user = None
 			for confs in [make_globals, make_conf, self.configdict["env"]]:
 				v = confs.get("PORTDIR")
 				if v is not None:
@@ -511,12 +513,22 @@ class config(object):
 				v = confs.get("SYNC")
 				if v is not None:
 					portdir_sync = v
+				v = confs.get("SYNC_UMASK")
+				if v is not None:
+					sync_umask = v
+				v = confs.get("SYNC_USER")
+				if v is not None:
+					sync_user = v
 
 			known_repos = frozenset(known_repos)
 			self["PORTDIR"] = portdir
 			self["PORTDIR_OVERLAY"] = portdir_overlay
 			if portdir_sync:
 				self["SYNC"] = portdir_sync
+			if sync_umask:
+				self["SYNC_UMASK"] = sync_umask
+			if sync_user:
+				self["SYNC_USER"] = sync_user
 			self.lookuplist = [self.configdict["env"]]
 			if repositories is None:
 				self.repositories = load_repository_config(self)
diff --git a/pym/portage/repository/config.py b/pym/portage/repository/config.py
index f45684b..9b7bfa9 100644
--- a/pym/portage/repository/config.py
+++ b/pym/portage/repository/config.py
@@ -827,6 +827,19 @@ class RepoConfigLoader(object):
 					"!!! %s\n" % _("Set 'masters = %s' in this file for future compatibility") %
 					self.mainRepo().name, level=logging.WARNING, noiselevel=-1)
 
+		# respect SYNC_UMASK and SYNC_USER whenever not overriden
+		fallback_umask = settings.get('SYNC_UMASK')
+		fallback_user = settings.get('SYNC_USER')
+		if fallback_umask is not None or fallback_user is not None:
+			for repo_name, repo in prepos.items():
+				if repo_name == "DEFAULT":
+					continue
+
+				if repo.sync_umask is None and fallback_umask is not None:
+					repo.sync_umask = fallback_umask
+				if repo.sync_user is None and fallback_user is not None:
+					repo.sync_user = fallback_user
+
 		self._prepos_changed = True
 		self._repo_location_list = []
 
-- 
2.2.0

Re: [gentoo-portage-dev] [PATCH 4/4] sync: Add backwards compat with SYNC_{UMASK,USER} variables

[Zac Medico]

On 12/05/2014 04:03 PM, Michał Górny wrote:
> Support SYNC_UMASK and SYNC_USER variables that were used in Funtoo
> Portage, as fallbacks to sync-umask and sync-user repo keys.
> ---
>  pym/portage/package/ebuild/_config/special_env_vars.py |  2 +-
>  pym/portage/package/ebuild/config.py                   | 12 ++++++++++++
>  pym/portage/repository/config.py                       | 13 +++++++++++++
>  3 files changed, 26 insertions(+), 1 deletion(-)

Why should we add backward compatibility code for something that was
never supported in the master branch?

Wouldn't it make more sense to have the Funtoo ebuild run some code that
generates a config update for repos.conf?
-- 
Thanks,
Zac

Re: [gentoo-portage-dev] [PATCH 4/4] sync: Add backwards compat with SYNC_{UMASK,USER} variables

[Michał Górny]

[-- Attachment #1: Type: text/plain, Size: 784 bytes --]

Dnia 2014-12-06, o godz. 23:15:09
Zac Medico <zmedico@gentoo.org> napisał(a):

> On 12/05/2014 04:03 PM, Michał Górny wrote:
> > Support SYNC_UMASK and SYNC_USER variables that were used in Funtoo
> > Portage, as fallbacks to sync-umask and sync-user repo keys.
> > ---
> >  pym/portage/package/ebuild/_config/special_env_vars.py |  2 +-
> >  pym/portage/package/ebuild/config.py                   | 12 ++++++++++++
> >  pym/portage/repository/config.py                       | 13 +++++++++++++
> >  3 files changed, 26 insertions(+), 1 deletion(-)
> 
> Why should we add backward compatibility code for something that was
> never supported in the master branch?

I just thought it wouldn't hurt if we're backporting features.

-- 
Best regards,
Michał Górny

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 949 bytes --]

Re: [gentoo-portage-dev] [PATCH 4/4] sync: Add backwards compat with SYNC_{UMASK,USER} variables

[Zac Medico]

On 12/07/2014 12:22 AM, Michał Górny wrote:
> Dnia 2014-12-06, o godz. 23:15:09
> Zac Medico <zmedico@gentoo.org> napisał(a):
> 
>> On 12/05/2014 04:03 PM, Michał Górny wrote:
>>> Support SYNC_UMASK and SYNC_USER variables that were used in Funtoo
>>> Portage, as fallbacks to sync-umask and sync-user repo keys.
>>> ---
>>>  pym/portage/package/ebuild/_config/special_env_vars.py |  2 +-
>>>  pym/portage/package/ebuild/config.py                   | 12 ++++++++++++
>>>  pym/portage/repository/config.py                       | 13 +++++++++++++
>>>  3 files changed, 26 insertions(+), 1 deletion(-)
>>
>> Why should we add backward compatibility code for something that was
>> never supported in the master branch?
> 
> I just thought it wouldn't hurt if we're backporting features.

Adding extra code for new redundant variables seems undesirable. Are
they somehow better than using equivalent repos.conf [DEFAULT] settings
for sync-user and sync-mask?
-- 
Thanks,
Zac

Re: [gentoo-portage-dev] [PATCH 4/4] sync: Add backwards compat with SYNC_{UMASK,USER} variables

[Michał Górny]

[-- Attachment #1: Type: text/plain, Size: 1213 bytes --]

Dnia 2014-12-07, o godz. 01:04:19
Zac Medico <zmedico@gentoo.org> napisał(a):

> On 12/07/2014 12:22 AM, Michał Górny wrote:
> > Dnia 2014-12-06, o godz. 23:15:09
> > Zac Medico <zmedico@gentoo.org> napisał(a):
> > 
> >> On 12/05/2014 04:03 PM, Michał Górny wrote:
> >>> Support SYNC_UMASK and SYNC_USER variables that were used in Funtoo
> >>> Portage, as fallbacks to sync-umask and sync-user repo keys.
> >>> ---
> >>>  pym/portage/package/ebuild/_config/special_env_vars.py |  2 +-
> >>>  pym/portage/package/ebuild/config.py                   | 12 ++++++++++++
> >>>  pym/portage/repository/config.py                       | 13 +++++++++++++
> >>>  3 files changed, 26 insertions(+), 1 deletion(-)
> >>
> >> Why should we add backward compatibility code for something that was
> >> never supported in the master branch?
> > 
> > I just thought it wouldn't hurt if we're backporting features.
> 
> Adding extra code for new redundant variables seems undesirable. Are
> they somehow better than using equivalent repos.conf [DEFAULT] settings
> for sync-user and sync-mask?

They're easier to set in env than the fancy PORTAGE_REPOSITORIES.

-- 
Best regards,
Michał Górny

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 949 bytes --]

Re: [gentoo-portage-dev] [PATCH 4/4] sync: Add backwards compat with SYNC_{UMASK,USER} variables

[Zac Medico]

On 12/07/2014 01:06 AM, Michał Górny wrote:
> Dnia 2014-12-07, o godz. 01:04:19
> Zac Medico <zmedico@gentoo.org> napisał(a):
> 
>> On 12/07/2014 12:22 AM, Michał Górny wrote:
>>> Dnia 2014-12-06, o godz. 23:15:09
>>> Zac Medico <zmedico@gentoo.org> napisał(a):
>>>
>>>> On 12/05/2014 04:03 PM, Michał Górny wrote:
>>>>> Support SYNC_UMASK and SYNC_USER variables that were used in Funtoo
>>>>> Portage, as fallbacks to sync-umask and sync-user repo keys.
>>>>> ---
>>>>>  pym/portage/package/ebuild/_config/special_env_vars.py |  2 +-
>>>>>  pym/portage/package/ebuild/config.py                   | 12 ++++++++++++
>>>>>  pym/portage/repository/config.py                       | 13 +++++++++++++
>>>>>  3 files changed, 26 insertions(+), 1 deletion(-)
>>>>
>>>> Why should we add backward compatibility code for something that was
>>>> never supported in the master branch?
>>>
>>> I just thought it wouldn't hurt if we're backporting features.
>>
>> Adding extra code for new redundant variables seems undesirable. Are
>> they somehow better than using equivalent repos.conf [DEFAULT] settings
>> for sync-user and sync-mask?
> 
> They're easier to set in env than the fancy PORTAGE_REPOSITORIES.

Do these settings commonly need to be overridden in the env?
-- 
Thanks,
Zac

Re: [gentoo-portage-dev] [PATCH 4/4] sync: Add backwards compat with SYNC_{UMASK,USER} variables

[Michał Górny]

[-- Attachment #1: Type: text/plain, Size: 1669 bytes --]

Dnia 2014-12-07, o godz. 10:33:44
Zac Medico <zmedico@gentoo.org> napisał(a):

> On 12/07/2014 01:06 AM, Michał Górny wrote:
> > Dnia 2014-12-07, o godz. 01:04:19
> > Zac Medico <zmedico@gentoo.org> napisał(a):
> > 
> >> On 12/07/2014 12:22 AM, Michał Górny wrote:
> >>> Dnia 2014-12-06, o godz. 23:15:09
> >>> Zac Medico <zmedico@gentoo.org> napisał(a):
> >>>
> >>>> On 12/05/2014 04:03 PM, Michał Górny wrote:
> >>>>> Support SYNC_UMASK and SYNC_USER variables that were used in Funtoo
> >>>>> Portage, as fallbacks to sync-umask and sync-user repo keys.
> >>>>> ---
> >>>>>  pym/portage/package/ebuild/_config/special_env_vars.py |  2 +-
> >>>>>  pym/portage/package/ebuild/config.py                   | 12 ++++++++++++
> >>>>>  pym/portage/repository/config.py                       | 13 +++++++++++++
> >>>>>  3 files changed, 26 insertions(+), 1 deletion(-)
> >>>>
> >>>> Why should we add backward compatibility code for something that was
> >>>> never supported in the master branch?
> >>>
> >>> I just thought it wouldn't hurt if we're backporting features.
> >>
> >> Adding extra code for new redundant variables seems undesirable. Are
> >> they somehow better than using equivalent repos.conf [DEFAULT] settings
> >> for sync-user and sync-mask?
> > 
> > They're easier to set in env than the fancy PORTAGE_REPOSITORIES.
> 
> Do these settings commonly need to be overridden in the env?

I have no clue. I just don't want users to lose functionality when it
doesn't require much effort (anymore, consider I've lost over an hour
to figure out why settings are incomplete...).

-- 
Best regards,
Michał Górny

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 949 bytes --]