[gentoo-portage-dev] emerge: add support for --pkg-format

[gentoo-portage-dev] emerge: add support for --pkg-format

[Tomáš Čech]

[-- Attachment #1: Type: text/plain, Size: 684 bytes --]

Hi,

I made a tiny patch adding support for binary package format choice as
it was discussed on IRC channel.

1] please review
2] please, tell me if the chosen names will fit in your world

I tested this patch on top of origin/master and works for me, besides
this:

emerge: incomplete set configuration, missing set(s): "selected", "system", and "world"
         This usually means that '/usr/share/portage/config/sets/portage.conf'
         is missing or corrupt.
         Falling back to default world and system set configuration!!!


Since I've installed 2.1.12.2, but used master, I take it as expected
and default configuration is fine.

Best regards,

Tomas Cech
Sleep_Walker

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

[gentoo-portage-dev] [PATCH] emerge: accept --pkg-format option

[Tomáš Čech]

Accept --pkg-format option which will override settings of
PORTAGE_BINPKG_FORMAT. Currently takes only one choice of
'tar' (original gentoo binary package) and 'rpm'.

Signed-off-by: Tomáš Čech <sleep_walker@suse.cz>
---
 man/emerge.1                |  4 ++++
 man/make.conf.5             |  5 +++++
 pym/_emerge/EbuildBinpkg.py | 17 +++++++++++------
 pym/_emerge/actions.py      |  4 ++++
 pym/_emerge/main.py         |  7 +++++++
 5 files changed, 31 insertions(+), 6 deletions(-)

diff --git a/man/emerge.1 b/man/emerge.1
index da6bcba..23e7aed 100644
--- a/man/emerge.1
+++ b/man/emerge.1
@@ -633,6 +633,10 @@ exhaustively apply the entire history of package moves,
 regardless of whether or not any of the package moves have
 been previously applied.
 .TP
+.BR \-\-pkg-format
+Specify which binary package format will be created as target.
+Possible choices now are tar and rpm.
+.TP
 .BR \-\-prefix=DIR
 Set the \fBEPREFIX\fR environment variable.
 .TP
diff --git a/man/make.conf.5 b/man/make.conf.5
index adf32bd..fff4c90 100644
--- a/man/make.conf.5
+++ b/man/make.conf.5
@@ -1,3 +1,4 @@
+
 .TH "MAKE.CONF" "5" "May 2013" "Portage VERSION" "Portage"
 .SH "NAME"
 make.conf \- custom settings for Portage
@@ -707,6 +708,10 @@ setting as the base URI.
 This variable contains options to be passed to the tar command for creation
 of binary packages.
 .TP
+.B PORTAGE_BINPKG_FORMAT
+This variable sets default format used for binary packages. Possible values
+are tar and rpm.
+.TP
 \fBPORTAGE_BUNZIP2_COMMAND\fR = \fI[bunzip2 command string]\fR
 This variable should contain a command that is suitable for portage to call
 for bunzip2 extraction operations.
diff --git a/pym/_emerge/EbuildBinpkg.py b/pym/_emerge/EbuildBinpkg.py
index 34a6aef..c636a1e 100644
--- a/pym/_emerge/EbuildBinpkg.py
+++ b/pym/_emerge/EbuildBinpkg.py
@@ -17,15 +17,20 @@ class EbuildBinpkg(CompositeTask):
 		root_config = pkg.root_config
 		bintree = root_config.trees["bintree"]
 		bintree.prevent_collision(pkg.cpv)
-		binpkg_tmpfile = os.path.join(bintree.pkgdir,
-			pkg.cpv + ".tbz2." + str(os.getpid()))
-		bintree._ensure_dir(os.path.dirname(binpkg_tmpfile))
 
-		self._binpkg_tmpfile = binpkg_tmpfile
-		self.settings["PORTAGE_BINPKG_TMPFILE"] = self._binpkg_tmpfile
+		if self.settings["PORTAGE_BINPKG_FORMAT"] == "rpm":
+			requested_phase = "rpm"
+		else:
+			requested_phase = "package"
+			binpkg_tmpfile = os.path.join(bintree.pkgdir,
+						      pkg.cpv + ".tbz2." + str(os.getpid()))
+			bintree._ensure_dir(os.path.dirname(binpkg_tmpfile))
+
+			self._binpkg_tmpfile = binpkg_tmpfile
+			self.settings["PORTAGE_BINPKG_TMPFILE"] = self._binpkg_tmpfile
 
 		package_phase = EbuildPhase(background=self.background,
-			phase='package', scheduler=self.scheduler,
+			phase=requested_phase, scheduler=self.scheduler,
 			settings=self.settings)
 
 		self._start_task(package_phase, self._package_phase_exit)
diff --git a/pym/_emerge/actions.py b/pym/_emerge/actions.py
index 6d5d535..30a9746 100644
--- a/pym/_emerge/actions.py
+++ b/pym/_emerge/actions.py
@@ -2909,6 +2909,10 @@ def adjust_config(myopts, settings):
 		settings["NOCOLOR"] = "true"
 		settings.backup_changes("NOCOLOR")
 
+	if "--pkg-format" in myopts:
+		settings["PORTAGE_BINPKG_FORMAT"] = myopts["--pkg-format"]
+		settings.backup_changes("PORTAGE_BINPKG_FORMAT")
+
 def display_missing_pkg_set(root_config, set_name):
 
 	msg = []
diff --git a/pym/_emerge/main.py b/pym/_emerge/main.py
index fe9fb29..df7a41f 100644
--- a/pym/_emerge/main.py
+++ b/pym/_emerge/main.py
@@ -138,6 +138,7 @@ def insert_optional_args(args):
 		'--keep-going'           : y_or_n,
 		'--load-average'         : valid_floats,
 		'--package-moves'        : y_or_n,
+		'--pkg-format'           : ('tar','rpm'),
 		'--quiet'                : y_or_n,
 		'--quiet-build'          : y_or_n,
 		'--quiet-fail'           : y_or_n,
@@ -546,6 +547,12 @@ def parse_opts(tmpcmdline, silent=False):
 			"action"   : "store"
 		},
 
+		"--pkg-format": {
+			"help"     : "format of result binary package",
+			"type"     : "choice",
+			"choices"  : ("tar", "rpm")
+		},
+
 		"--quiet": {
 			"shortopt" : "-q",
 			"help"     : "reduced or condensed output",
-- 
1.8.3.1

Re: [gentoo-portage-dev] [PATCH] emerge: accept --pkg-format option

[Zac Medico]

On 07/23/2013 11:33 PM, Tomáš Čech wrote:
> Accept --pkg-format option which will override settings of
> PORTAGE_BINPKG_FORMAT. Currently takes only one choice of
> 'tar' (original gentoo binary package) and 'rpm'.

The patch looks good. However, for flexibility, I think we should
support multiple formats simultaneously. The EbuildBuild class could use
an instance of TaskSequence to execute multiple EbuildBinpkg instances
sequentially. Example pseudocode:

binpkg_tasks = TaskSequence()
for pkg_fmt in settings.get("PORTAGE_BINPKG_FORMAT", "tar").split():
    binpkg_tasks.add(EbuildBinpkg(pkg_fmt))

-- 
Thanks,
Zac

[gentoo-portage-dev] [PATCH] emerge: accept --pkg-format option

[Tomáš Čech]

Accept --pkg-format option which will override settings of
PORTAGE_BINPKG_FORMAT. Currently takes choices of 'tar' (original gentoo
binary package), 'rpm' or its combinations.

Signed-off-by: Tomáš Čech <sleep_walker@suse.cz>
---
 man/emerge.1                |  4 ++++
 man/make.conf.5             |  4 ++++
 pym/_emerge/EbuildBinpkg.py | 18 +++++++++++-------
 pym/_emerge/EbuildBuild.py  | 12 +++++++++---
 pym/_emerge/actions.py      | 17 +++++++++++++++++
 pym/_emerge/main.py         |  5 +++++
 pym/portage/const.py        |  1 +
 7 files changed, 51 insertions(+), 10 deletions(-)

diff --git a/man/emerge.1 b/man/emerge.1
index da6bcba..1571e8a 100644
--- a/man/emerge.1
+++ b/man/emerge.1
@@ -633,6 +633,10 @@ exhaustively apply the entire history of package moves,
 regardless of whether or not any of the package moves have
 been previously applied.
 .TP
+.BR \-\-pkg-format
+Specify which binary package format will be created as target.
+Possible choices now are tar and rpm or their combinations.
+.TP
 .BR \-\-prefix=DIR
 Set the \fBEPREFIX\fR environment variable.
 .TP
diff --git a/man/make.conf.5 b/man/make.conf.5
index adf32bd..8811513 100644
--- a/man/make.conf.5
+++ b/man/make.conf.5
@@ -707,6 +707,10 @@ setting as the base URI.
 This variable contains options to be passed to the tar command for creation
 of binary packages.
 .TP
+.B PORTAGE_BINPKG_FORMAT
+This variable sets default format used for binary packages. Possible values
+are tar and rpm or both.
+.TP
 \fBPORTAGE_BUNZIP2_COMMAND\fR = \fI[bunzip2 command string]\fR
 This variable should contain a command that is suitable for portage to call
 for bunzip2 extraction operations.
diff --git a/pym/_emerge/EbuildBinpkg.py b/pym/_emerge/EbuildBinpkg.py
index 34a6aef..145cd31 100644
--- a/pym/_emerge/EbuildBinpkg.py
+++ b/pym/_emerge/EbuildBinpkg.py
@@ -10,22 +10,26 @@ class EbuildBinpkg(CompositeTask):
 	This assumes that src_install() has successfully completed.
 	"""
 	__slots__ = ('pkg', 'settings') + \
-		('_binpkg_tmpfile',)
+		('_binpkg_tmpfile', 'pkg_format')
 
 	def _start(self):
 		pkg = self.pkg
 		root_config = pkg.root_config
 		bintree = root_config.trees["bintree"]
 		bintree.prevent_collision(pkg.cpv)
-		binpkg_tmpfile = os.path.join(bintree.pkgdir,
-			pkg.cpv + ".tbz2." + str(os.getpid()))
-		bintree._ensure_dir(os.path.dirname(binpkg_tmpfile))
+		if self.pkg_format == "rpm":
+			requested_phase = "rpm"
+		else:
+			requested_phase = "package"
+			binpkg_tmpfile = os.path.join(bintree.pkgdir,
+				pkg.cpv + ".tbz2." + str(os.getpid()))
+			bintree._ensure_dir(os.path.dirname(binpkg_tmpfile))
 
-		self._binpkg_tmpfile = binpkg_tmpfile
-		self.settings["PORTAGE_BINPKG_TMPFILE"] = self._binpkg_tmpfile
+			self._binpkg_tmpfile = binpkg_tmpfile
+			self.settings["PORTAGE_BINPKG_TMPFILE"] = self._binpkg_tmpfile
 
 		package_phase = EbuildPhase(background=self.background,
-			phase='package', scheduler=self.scheduler,
+			phase=requested_phase, scheduler=self.scheduler,
 			settings=self.settings)
 
 		self._start_task(package_phase, self._package_phase_exit)
diff --git a/pym/_emerge/EbuildBuild.py b/pym/_emerge/EbuildBuild.py
index 75d906f..8755815 100644
--- a/pym/_emerge/EbuildBuild.py
+++ b/pym/_emerge/EbuildBuild.py
@@ -10,6 +10,8 @@ from _emerge.EbuildMerge import EbuildMerge
 from _emerge.EbuildFetchonly import EbuildFetchonly
 from _emerge.EbuildBuildDir import EbuildBuildDir
 from _emerge.MiscFunctionsProcess import MiscFunctionsProcess
+from _emerge.TaskSequence import TaskSequence
+
 from portage.util import writemsg
 import portage
 from portage import os
@@ -306,10 +308,14 @@ class EbuildBuild(CompositeTask):
 			self.scheduler.output(msg,
 				log_path=self.settings.get("PORTAGE_LOG_FILE"))
 
-		packager = EbuildBinpkg(background=self.background, pkg=self.pkg,
-			scheduler=self.scheduler, settings=self.settings)
+		binpkg_tasks = TaskSequence()
+		requested_binpkg_formats = self.settings.get("PORTAGE_BINPKG_FORMAT", "tar").split()
+		for pkg_fmt in portage.const.SUPPORTED_BINPKG_FORMATS:
+			if pkg_fmt in requested_binpkg_formats:
+				binpkg_tasks.add(EbuildBinpkg(background=self.background, pkg=self.pkg,
+					pkg_format=pkg_fmt, scheduler=self.scheduler, settings=self.settings))
 
-		self._start_task(packager, self._buildpkg_exit)
+		self._start_task(binpkg_tasks, self._buildpkg_exit)
 
 	def _buildpkg_exit(self, packager):
 		"""
diff --git a/pym/_emerge/actions.py b/pym/_emerge/actions.py
index 6d5d535..9ba50a0 100644
--- a/pym/_emerge/actions.py
+++ b/pym/_emerge/actions.py
@@ -38,6 +38,7 @@ from portage import shutil
 from portage import eapi_is_supported, _encodings, _unicode_decode
 from portage.cache.cache_errors import CacheError
 from portage.const import GLOBAL_CONFIG_PATH, VCS_DIRS, _DEPCLEAN_LIB_CHECK_DEFAULT
+from portage.const import SUPPORTED_BINPKG_FORMATS
 from portage.dbapi.dep_expand import dep_expand
 from portage.dbapi._expand_new_virt import expand_new_virt
 from portage.dep import Atom
@@ -2909,6 +2910,10 @@ def adjust_config(myopts, settings):
 		settings["NOCOLOR"] = "true"
 		settings.backup_changes("NOCOLOR")
 
+	if "--pkg-format" in myopts:
+		settings["PORTAGE_BINPKG_FORMAT"] = myopts["--pkg-format"]
+		settings.backup_changes("PORTAGE_BINPKG_FORMAT")
+
 def display_missing_pkg_set(root_config, set_name):
 
 	msg = []
@@ -3591,6 +3596,18 @@ def run_action(emerge_config):
 	adjust_configs(emerge_config.opts, emerge_config.trees)
 	apply_priorities(emerge_config.target_config.settings)
 
+	for fmt in emerge_config.target_config.settings["PORTAGE_BINPKG_FORMAT"].split():
+		if not fmt in portage.const.SUPPORTED_BINPKG_FORMATS:
+			if "--pkg-format" in emerge_config.opts:
+				problematic="--pkg-format"
+			else:
+				problematic="PORTAGE_BINPKG_FORMAT"
+
+			writemsg_level(("emerge: %s is not set correctly. Format " + \
+				"'%s' is not supported.\n") % (problematic, fmt),
+				level=logging.ERROR, noiselevel=-1)
+			return 1
+
 	if emerge_config.action == 'version':
 		writemsg_stdout(getportageversion(
 			emerge_config.target_config.settings["PORTDIR"],
diff --git a/pym/_emerge/main.py b/pym/_emerge/main.py
index fe9fb29..edf40a5 100644
--- a/pym/_emerge/main.py
+++ b/pym/_emerge/main.py
@@ -546,6 +546,11 @@ def parse_opts(tmpcmdline, silent=False):
 			"action"   : "store"
 		},
 
+		"--pkg-format": {
+			"help"     : "format of result binary package",
+			"action"   : "store",
+		},
+
 		"--quiet": {
 			"shortopt" : "-q",
 			"help"     : "reduced or condensed output",
diff --git a/pym/portage/const.py b/pym/portage/const.py
index 087c0e7..bf22977 100644
--- a/pym/portage/const.py
+++ b/pym/portage/const.py
@@ -169,6 +169,7 @@ if "PORTAGE_OVERRIDE_EPREFIX" in os.environ:
 
 VCS_DIRS = ("CVS", "RCS", "SCCS", ".bzr", ".git", ".hg", ".svn")
 
+SUPPORTED_BINPKG_FORMATS = ("tar", "rpm")
 # ===========================================================================
 # END OF CONSTANTS -- END OF CONSTANTS -- END OF CONSTANTS -- END OF CONSTANT
 # ===========================================================================
-- 
1.8.3.1

Re: [gentoo-portage-dev] [PATCH] emerge: accept --pkg-format option

[Zac Medico]

On 07/27/2013 04:09 PM, Tomáš Čech wrote:
> Accept --pkg-format option which will override settings of
> PORTAGE_BINPKG_FORMAT. Currently takes choices of 'tar' (original gentoo
> binary package), 'rpm' or its combinations.
> 
> Signed-off-by: Tomáš Čech <sleep_walker@suse.cz>

Thanks, I've applied your patch:

http://git.overlays.gentoo.org/gitweb/?p=proj/portage.git;a=commit;h=73a972e0cac5d7e5f59a58c11e998793fe2b1a2d
-- 
Thanks,
Zac