[gentoo-commits] proj/catalyst:wip/mattst88 commit in: catalyst/

[gentoo-commits] proj/catalyst:wip/mattst88 commit in: catalyst/

[Matt Turner]

commit:     9a1b915db1b6ba0f8616eab7d9a36f17d3cc6648
Author:     Matt Turner <mattst88 <AT> gentoo <DOT> org>
AuthorDate: Mon May 18 23:36:41 2020 +0000
Commit:     Matt Turner <mattst88 <AT> gentoo <DOT> org>
CommitDate: Tue May 19 22:54:21 2020 +0000
URL:        https://gitweb.gentoo.org/proj/catalyst.git/commit/?id=9a1b915d

catalyst: Delete unused {required,valid}_build_targets

Fixes: 11423a21603e ([2 of 3] Update module loading for the new python structure)
Signed-off-by: Matt Turner <mattst88 <AT> gentoo.org>

 catalyst/defaults.py | 19 -------------------
 1 file changed, 19 deletions(-)

diff --git a/catalyst/defaults.py b/catalyst/defaults.py
index f6bc1e14..0da717f1 100644
--- a/catalyst/defaults.py
+++ b/catalyst/defaults.py
@@ -8,25 +8,6 @@ from DeComp.definitions import COMPRESSOR_PROGRAM_OPTIONS, XATTRS_OPTIONS
 from DeComp.definitions import DECOMPRESSOR_PROGRAM_OPTIONS, LIST_XATTRS_OPTIONS
 
 
-# these should never be touched
-required_build_targets = [
-    "generic_stage_target",
-    "targetbase",
-]
-
-# new build types should be added here
-valid_build_targets = [
-    "embedded_target",
-    "livecd_stage1_target",
-    "livecd_stage2_target",
-    "netboot_target",
-    "snapshot_target",
-    "stage1_target",
-    "stage2_target",
-    "stage3_target",
-    "stage4_target",
-]
-
 required_config_file_values = [
     "distdir",
     "portdir",

[gentoo-commits] proj/catalyst:wip/mattst88 commit in: catalyst/

[Matt Turner]

commit:     ed05b6e13d0eea2f6be2a631bea870978da150d0
Author:     Matt Turner <mattst88 <AT> gentoo <DOT> org>
AuthorDate: Mon May 18 23:49:54 2020 +0000
Commit:     Matt Turner <mattst88 <AT> gentoo <DOT> org>
CommitDate: Tue May 19 22:54:21 2020 +0000
URL:        https://gitweb.gentoo.org/proj/catalyst.git/commit/?id=ed05b6e1

catalyst: Add a missing options to valid_config_file_values

And remove required_config_file_values since we want to support running
catalyst without a config file.

Signed-off-by: Matt Turner <mattst88 <AT> gentoo.org>

 catalyst/defaults.py | 20 +++++++++++---------
 1 file changed, 11 insertions(+), 9 deletions(-)

diff --git a/catalyst/defaults.py b/catalyst/defaults.py
index 2f2c907f..412cb956 100644
--- a/catalyst/defaults.py
+++ b/catalyst/defaults.py
@@ -8,18 +8,20 @@ from DeComp.definitions import COMPRESSOR_PROGRAM_OPTIONS, XATTRS_OPTIONS
 from DeComp.definitions import DECOMPRESSOR_PROGRAM_OPTIONS, LIST_XATTRS_OPTIONS
 
 
-required_config_file_values = [
-    "distdir",
-    "portdir",
-    "sharedir",
-    "storedir",
-]
-
-valid_config_file_values = required_config_file_values[:]
-valid_config_file_values.extend([
+valid_config_file_values = frozenset([
     "digests",
+    "distdir",
     "envscript",
     "options",
+    "port_logdir",
+    "repo_basedir",
+    "repo_name",
+    "repos",
+    "sharedir",
+    "storedir",
+    "target_distdir",
+    "target_pkgdir",
+    "var_tmpfs_portage",
 ])
 
 confdefaults = {

[gentoo-commits] proj/catalyst:wip/mattst88 commit in: catalyst/

[Matt Turner]

commit:     ab674651496be8d6a784d8fa0b978da76958d666
Author:     Matt Turner <mattst88 <AT> gentoo <DOT> org>
AuthorDate: Mon May 18 23:48:22 2020 +0000
Commit:     Matt Turner <mattst88 <AT> gentoo <DOT> org>
CommitDate: Tue May 19 22:54:21 2020 +0000
URL:        https://gitweb.gentoo.org/proj/catalyst.git/commit/?id=ab674651

catalyst: Remove spec file options from valid_config_file_values

Signed-off-by: Matt Turner <mattst88 <AT> gentoo.org>

 catalyst/defaults.py | 5 -----
 1 file changed, 5 deletions(-)

diff --git a/catalyst/defaults.py b/catalyst/defaults.py
index ccabd88d..2f2c907f 100644
--- a/catalyst/defaults.py
+++ b/catalyst/defaults.py
@@ -17,12 +17,7 @@ required_config_file_values = [
 
 valid_config_file_values = required_config_file_values[:]
 valid_config_file_values.extend([
-    "compression_mode",
-    "compressor_arch",
-    "compressor_options",
-    "decompressor_search_order",
     "digests",
-    "distcc",
     "envscript",
     "options",
 ])

[gentoo-commits] proj/catalyst:wip/mattst88 commit in: catalyst/

[Matt Turner]

commit:     efed4b7a1bd5e199920af2919e874cfdbc125e6d
Author:     Matt Turner <mattst88 <AT> gentoo <DOT> org>
AuthorDate: Mon May 18 22:52:36 2020 +0000
Commit:     Matt Turner <mattst88 <AT> gentoo <DOT> org>
CommitDate: Tue May 19 22:54:21 2020 +0000
URL:        https://gitweb.gentoo.org/proj/catalyst.git/commit/?id=efed4b7a

catalyst: Remove PythonDir setting

Was used to find the arch directory containing, e.g., sparc.py, but all
that code is gone now after the modules were converted to TOML.

Signed-off-by: Matt Turner <mattst88 <AT> gentoo.org>

 catalyst/defaults.py | 1 -
 catalyst/main.py     | 3 ---
 2 files changed, 4 deletions(-)

diff --git a/catalyst/defaults.py b/catalyst/defaults.py
index 404f4892..f6bc1e14 100644
--- a/catalyst/defaults.py
+++ b/catalyst/defaults.py
@@ -64,7 +64,6 @@ confdefaults = {
     "options": set(),
     "pkgdir": "/var/cache/binpkgs",
     "port_tmpdir": "/var/tmp/portage",
-    "PythonDir": "./catalyst",
     "repo_basedir": "/var/db/repos",
     "repo_name": "gentoo",
     "repos": "%(storedir)s/repos",

diff --git a/catalyst/main.py b/catalyst/main.py
index bad712fa..b01d7a6a 100644
--- a/catalyst/main.py
+++ b/catalyst/main.py
@@ -55,9 +55,6 @@ def parse_config(config_files):
         else:
             conf_values[x] = confdefaults[x]
 
-    # add our python base directory to use for loading target arch's
-    conf_values["PythonDir"] = os.path.dirname(os.path.realpath(__file__))
-
     # print out any options messages
     for opt in conf_values['options']:
         if opt in option_messages:

[gentoo-commits] proj/catalyst:wip/mattst88 commit in: catalyst/

[Matt Turner]

commit:     40368d89e416725e50d1809bda7b0942aaca20b6
Author:     Matt Turner <mattst88 <AT> gentoo <DOT> org>
AuthorDate: Mon May 18 23:43:52 2020 +0000
Commit:     Matt Turner <mattst88 <AT> gentoo <DOT> org>
CommitDate: Tue May 19 22:54:21 2020 +0000
URL:        https://gitweb.gentoo.org/proj/catalyst.git/commit/?id=40368d89

catalyst: Disallow DEBUG and VERBOSE in spec files

Catalyst has --debug/-d and --verbose/-v options for this.

Signed-off-by: Matt Turner <mattst88 <AT> gentoo.org>

 catalyst/defaults.py | 2 --
 1 file changed, 2 deletions(-)

diff --git a/catalyst/defaults.py b/catalyst/defaults.py
index 0da717f1..ccabd88d 100644
--- a/catalyst/defaults.py
+++ b/catalyst/defaults.py
@@ -20,13 +20,11 @@ valid_config_file_values.extend([
     "compression_mode",
     "compressor_arch",
     "compressor_options",
-    "DEBUG",
     "decompressor_search_order",
     "digests",
     "distcc",
     "envscript",
     "options",
-    "VERBOSE",
 ])
 
 confdefaults = {

[gentoo-commits] proj/catalyst:wip/mattst88 commit in: catalyst/

[Matt Turner]

commit:     17622798705f1c8e6d24594c8aa99ccbbcf7e323
Author:     Matt Turner <mattst88 <AT> gentoo <DOT> org>
AuthorDate: Mon May 18 22:41:54 2020 +0000
Commit:     Matt Turner <mattst88 <AT> gentoo <DOT> org>
CommitDate: Tue May 19 22:54:21 2020 +0000
URL:        https://gitweb.gentoo.org/proj/catalyst.git/commit/?id=17622798

catalyst: Switch internal snapshot option parsing to SpecParser

The --snapshot/-s option internally creates a .spec file but uses the
ConfigParser (nominally used for parsing catalyst.conf) rather than
SpecParser (used for parsing .spec files) and as a result has to use
'=' rather than ':' as the key/value delimiter.

Signed-off-by: Matt Turner <mattst88 <AT> gentoo.org>

 catalyst/main.py | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/catalyst/main.py b/catalyst/main.py
index b01d7a6a..be06ccd7 100644
--- a/catalyst/main.py
+++ b/catalyst/main.py
@@ -291,8 +291,8 @@ def _main(parser, opts):
 
     mycmdline = list()
     if opts.snapshot:
-        mycmdline.append('target=snapshot')
-        mycmdline.append('snapshot_treeish=' + opts.snapshot)
+        mycmdline.append('target: snapshot')
+        mycmdline.append('snapshot_treeish: ' + opts.snapshot)
 
     conf_values['DEBUG'] = opts.debug
     conf_values['VERBOSE'] = opts.debug or opts.verbose
@@ -354,7 +354,7 @@ def _main(parser, opts):
 
     if mycmdline:
         try:
-            cmdline = catalyst.config.ConfigParser()
+            cmdline = catalyst.config.SpecParser()
             cmdline.parse_lines(mycmdline)
             addlargs.update(cmdline.get_values())
         except CatalystError:

[gentoo-commits] proj/catalyst:wip/mattst88 commit in: catalyst/

[Matt Turner]

commit:     c1f795bd90abdd5e8852212558f8d4d229f7f3b8
Author:     Matt Turner <mattst88 <AT> gentoo <DOT> org>
AuthorDate: Tue May 19 00:13:14 2020 +0000
Commit:     Matt Turner <mattst88 <AT> gentoo <DOT> org>
CommitDate: Tue May 19 22:54:21 2020 +0000
URL:        https://gitweb.gentoo.org/proj/catalyst.git/commit/?id=c1f795bd

catalyst: Verify config options against valid_config_file_values

Signed-off-by: Matt Turner <mattst88 <AT> gentoo.org>

 catalyst/main.py | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/catalyst/main.py b/catalyst/main.py
index a2444eeb..c44afbfe 100644
--- a/catalyst/main.py
+++ b/catalyst/main.py
@@ -15,7 +15,8 @@ from DeComp.contents import ContentsMap
 
 from catalyst import log
 import catalyst.config
-from catalyst.defaults import confdefaults, option_messages, DEFAULT_CONFIG_FILE
+from catalyst.defaults import (confdefaults, option_messages,
+                               DEFAULT_CONFIG_FILE valid_config_file_values)
 from catalyst.support import CatalystError
 from catalyst.version import get_version
 
@@ -47,8 +48,7 @@ def parse_config(config_files):
         if opt in option_messages:
             log.info(option_messages[opt])
 
-    for key in ["digests", "envscript", "var_tmpfs_portage", "port_logdir",
-                "local_overlay", "repos"]:
+    for key in valid_config_file_values:
         if key in config:
             conf_values[key] = config[key]
 

[gentoo-commits] proj/catalyst:wip/mattst88 commit in: catalyst/

[Matt Turner]

commit:     2f37a2c19afb8e89c0139257a9765c969a34115e
Author:     Matt Turner <mattst88 <AT> gentoo <DOT> org>
AuthorDate: Tue May 19 19:14:29 2020 +0000
Commit:     Matt Turner <mattst88 <AT> gentoo <DOT> org>
CommitDate: Tue May 19 22:54:21 2020 +0000
URL:        https://gitweb.gentoo.org/proj/catalyst.git/commit/?id=2f37a2c1

catalyst: Drop ConfigParser

Signed-off-by: Matt Turner <mattst88 <AT> gentoo.org>

 catalyst/config.py | 11 -----------
 1 file changed, 11 deletions(-)

diff --git a/catalyst/config.py b/catalyst/config.py
index b527ada0..e1963f71 100644
--- a/catalyst/config.py
+++ b/catalyst/config.py
@@ -118,14 +118,3 @@ class SpecParser(ParserBase):
     def __init__(self, filename=""):
         if filename:
             self.parse_file(filename)
-
-
-class ConfigParser(ParserBase):
-
-    key_value_separator = '='
-    multiple_values = False
-    empty_values = True
-
-    def __init__(self, filename=""):
-        if filename:
-            self.parse_file(filename)

[gentoo-commits] proj/catalyst:wip/mattst88 commit in: catalyst/

[Matt Turner]

commit:     3db73f91f7b1d654bbed0463ec8f0a1703b7e2aa
Author:     Matt Turner <mattst88 <AT> gentoo <DOT> org>
AuthorDate: Mon May 18 23:34:13 2020 +0000
Commit:     Matt Turner <mattst88 <AT> gentoo <DOT> org>
CommitDate: Tue May 19 22:54:21 2020 +0000
URL:        https://gitweb.gentoo.org/proj/catalyst.git/commit/?id=3db73f91

catalyst: Disallow config file options in spec files

Signed-off-by: Matt Turner <mattst88 <AT> gentoo.org>

 catalyst/support.py | 3 +--
 1 file changed, 1 insertion(+), 2 deletions(-)

diff --git a/catalyst/support.py b/catalyst/support.py
index 0925af47..c4a5c797 100644
--- a/catalyst/support.py
+++ b/catalyst/support.py
@@ -8,7 +8,6 @@ import time
 from subprocess import Popen
 
 from catalyst import log
-from catalyst.defaults import valid_config_file_values
 
 BASH_BINARY = "/bin/bash"
 
@@ -211,7 +210,7 @@ def addl_arg_parse(myspec, addlargs, requiredspec, validspec):
     "helper function to help targets parse additional arguments"
     messages = []
     for x in addlargs.keys():
-        if x not in validspec and x not in valid_config_file_values and x not in requiredspec:
+        if x not in validspec and x not in requiredspec:
             messages.append("Argument \""+x+"\" not recognized.")
         else:
             myspec[x] = addlargs[x]

[gentoo-commits] proj/catalyst:wip/mattst88 commit in: catalyst/

[Matt Turner]

commit:     9e6b6f653090e0708db55f89a965cfa38b16ce5e
Author:     Matt Turner <mattst88 <AT> gentoo <DOT> org>
AuthorDate: Tue May 19 00:13:14 2020 +0000
Commit:     Matt Turner <mattst88 <AT> gentoo <DOT> org>
CommitDate: Tue May 19 23:04:19 2020 +0000
URL:        https://gitweb.gentoo.org/proj/catalyst.git/commit/?id=9e6b6f65

catalyst: Verify config options against valid_config_file_values

Signed-off-by: Matt Turner <mattst88 <AT> gentoo.org>

 catalyst/main.py | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/catalyst/main.py b/catalyst/main.py
index 4a55ebe2..853e77d8 100644
--- a/catalyst/main.py
+++ b/catalyst/main.py
@@ -15,7 +15,8 @@ from DeComp.contents import ContentsMap
 
 from catalyst import log
 import catalyst.config
-from catalyst.defaults import confdefaults, option_messages, DEFAULT_CONFIG_FILE
+from catalyst.defaults import (confdefaults, option_messages,
+                               DEFAULT_CONFIG_FILE, valid_config_file_values)
 from catalyst.support import CatalystError
 from catalyst.version import get_version
 
@@ -47,8 +48,7 @@ def parse_config(config_files):
         if opt in option_messages:
             log.info(option_messages[opt])
 
-    for key in ["digests", "envscript", "var_tmpfs_portage", "port_logdir",
-                "local_overlay", "repos"]:
+    for key in valid_config_file_values:
         if key in config:
             conf_values[key] = config[key]
 

[gentoo-commits] proj/catalyst:wip/mattst88 commit in: catalyst/

[Matt Turner]

commit:     213af690f2d657ab8c1ebcb9045a6230f7c12cf8
Author:     Matt Turner <mattst88 <AT> gentoo <DOT> org>
AuthorDate: Tue May 19 19:14:29 2020 +0000
Commit:     Matt Turner <mattst88 <AT> gentoo <DOT> org>
CommitDate: Tue May 19 23:04:19 2020 +0000
URL:        https://gitweb.gentoo.org/proj/catalyst.git/commit/?id=213af690

catalyst: Drop ConfigParser

Signed-off-by: Matt Turner <mattst88 <AT> gentoo.org>

 catalyst/config.py | 11 -----------
 1 file changed, 11 deletions(-)

diff --git a/catalyst/config.py b/catalyst/config.py
index b527ada0..e1963f71 100644
--- a/catalyst/config.py
+++ b/catalyst/config.py
@@ -118,14 +118,3 @@ class SpecParser(ParserBase):
     def __init__(self, filename=""):
         if filename:
             self.parse_file(filename)
-
-
-class ConfigParser(ParserBase):
-
-    key_value_separator = '='
-    multiple_values = False
-    empty_values = True
-
-    def __init__(self, filename=""):
-        if filename:
-            self.parse_file(filename)

[gentoo-commits] proj/catalyst:wip/mattst88 commit in: catalyst/

[Matt Turner]

commit:     956f1ff062328d912174e0bbcfb5c2efb20bac14
Author:     Matt Turner <mattst88 <AT> gentoo <DOT> org>
AuthorDate: Tue May 19 19:14:29 2020 +0000
Commit:     Matt Turner <mattst88 <AT> gentoo <DOT> org>
CommitDate: Tue May 19 23:30:23 2020 +0000
URL:        https://gitweb.gentoo.org/proj/catalyst.git/commit/?id=956f1ff0

catalyst: Drop ConfigParser

Signed-off-by: Matt Turner <mattst88 <AT> gentoo.org>

 catalyst/config.py | 11 -----------
 1 file changed, 11 deletions(-)

diff --git a/catalyst/config.py b/catalyst/config.py
index b527ada0..e1963f71 100644
--- a/catalyst/config.py
+++ b/catalyst/config.py
@@ -118,14 +118,3 @@ class SpecParser(ParserBase):
     def __init__(self, filename=""):
         if filename:
             self.parse_file(filename)
-
-
-class ConfigParser(ParserBase):
-
-    key_value_separator = '='
-    multiple_values = False
-    empty_values = True
-
-    def __init__(self, filename=""):
-        if filename:
-            self.parse_file(filename)

[gentoo-commits] proj/catalyst:wip/mattst88 commit in: catalyst/

[Matt Turner]

commit:     b81a3c3634953a3cdb8cf7b1961c2635244573ff
Author:     Matt Turner <mattst88 <AT> gentoo <DOT> org>
AuthorDate: Tue May 19 00:13:14 2020 +0000
Commit:     Matt Turner <mattst88 <AT> gentoo <DOT> org>
CommitDate: Tue May 19 23:30:23 2020 +0000
URL:        https://gitweb.gentoo.org/proj/catalyst.git/commit/?id=b81a3c36

catalyst: Verify config options against valid_config_file_values

Signed-off-by: Matt Turner <mattst88 <AT> gentoo.org>

 catalyst/main.py | 7 ++++++-
 1 file changed, 6 insertions(+), 1 deletion(-)

diff --git a/catalyst/main.py b/catalyst/main.py
index a20fb0d7..220de372 100644
--- a/catalyst/main.py
+++ b/catalyst/main.py
@@ -15,7 +15,8 @@ from DeComp.contents import ContentsMap
 
 from catalyst import log
 import catalyst.config
-from catalyst.defaults import confdefaults, option_messages, DEFAULT_CONFIG_FILE
+from catalyst.defaults import (confdefaults, option_messages,
+                               DEFAULT_CONFIG_FILE, valid_config_file_values)
 from catalyst.support import CatalystError
 from catalyst.version import get_version
 
@@ -44,6 +45,10 @@ def parse_config(config_files):
         if opt in option_messages:
             log.info(option_messages[opt])
 
+    for key in conf_values:
+        if key not in valid_config_file_values:
+            log.critical(f"Unknown option '{key} in config file")
+
     if "envscript" in conf_values:
         log.info('Envscript support enabled.')
 

[gentoo-commits] proj/catalyst:wip/mattst88 commit in: catalyst/

[Matt Turner]

commit:     ffb4ef0ab01be58730e8072008ad6864b0e50b3f
Author:     Matt Turner <mattst88 <AT> gentoo <DOT> org>
AuthorDate: Tue May 19 00:13:14 2020 +0000
Commit:     Matt Turner <mattst88 <AT> gentoo <DOT> org>
CommitDate: Tue May 19 23:39:37 2020 +0000
URL:        https://gitweb.gentoo.org/proj/catalyst.git/commit/?id=ffb4ef0a

catalyst: Verify config options against valid_config_file_values

Signed-off-by: Matt Turner <mattst88 <AT> gentoo.org>

 catalyst/main.py | 7 ++++++-
 1 file changed, 6 insertions(+), 1 deletion(-)

diff --git a/catalyst/main.py b/catalyst/main.py
index 159fe454..94900b00 100644
--- a/catalyst/main.py
+++ b/catalyst/main.py
@@ -15,7 +15,8 @@ from DeComp.contents import ContentsMap
 
 from catalyst import log
 import catalyst.config
-from catalyst.defaults import confdefaults, option_messages, DEFAULT_CONFIG_FILE
+from catalyst.defaults import (confdefaults, option_messages,
+                               DEFAULT_CONFIG_FILE, valid_config_file_values)
 from catalyst.support import CatalystError
 from catalyst.version import get_version
 
@@ -44,6 +45,10 @@ def parse_config(config_files):
         if opt in option_messages:
             log.info(option_messages[opt])
 
+    for key in conf_values:
+        if key not in valid_config_file_values:
+            log.critical(f"Unknown option '{key} in config file")
+
     if "envscript" in conf_values:
         log.info('Envscript support enabled.')
 

[gentoo-commits] proj/catalyst:wip/mattst88 commit in: catalyst/

[Matt Turner]

commit:     637e8f750a889c62d625ca12f60b0e48ab554e40
Author:     Matt Turner <mattst88 <AT> gentoo <DOT> org>
AuthorDate: Tue May 19 19:14:29 2020 +0000
Commit:     Matt Turner <mattst88 <AT> gentoo <DOT> org>
CommitDate: Tue May 19 23:39:37 2020 +0000
URL:        https://gitweb.gentoo.org/proj/catalyst.git/commit/?id=637e8f75

catalyst: Drop ConfigParser

Signed-off-by: Matt Turner <mattst88 <AT> gentoo.org>

 catalyst/config.py | 11 -----------
 1 file changed, 11 deletions(-)

diff --git a/catalyst/config.py b/catalyst/config.py
index b527ada0..e1963f71 100644
--- a/catalyst/config.py
+++ b/catalyst/config.py
@@ -118,14 +118,3 @@ class SpecParser(ParserBase):
     def __init__(self, filename=""):
         if filename:
             self.parse_file(filename)
-
-
-class ConfigParser(ParserBase):
-
-    key_value_separator = '='
-    multiple_values = False
-    empty_values = True
-
-    def __init__(self, filename=""):
-        if filename:
-            self.parse_file(filename)

[gentoo-commits] proj/catalyst:wip/mattst88 commit in: catalyst/

[Matt Turner]

commit:     2751ef36ebb4d2d81a847e43578292794e16da19
Author:     Matt Turner <mattst88 <AT> gentoo <DOT> org>
AuthorDate: Tue May 19 00:13:14 2020 +0000
Commit:     Matt Turner <mattst88 <AT> gentoo <DOT> org>
CommitDate: Tue May 19 23:45:26 2020 +0000
URL:        https://gitweb.gentoo.org/proj/catalyst.git/commit/?id=2751ef36

catalyst: Verify config options against valid_config_file_values

Signed-off-by: Matt Turner <mattst88 <AT> gentoo.org>

 catalyst/main.py | 10 ++++++++--
 1 file changed, 8 insertions(+), 2 deletions(-)

diff --git a/catalyst/main.py b/catalyst/main.py
index 159fe454..543895c6 100644
--- a/catalyst/main.py
+++ b/catalyst/main.py
@@ -15,7 +15,8 @@ from DeComp.contents import ContentsMap
 
 from catalyst import log
 import catalyst.config
-from catalyst.defaults import confdefaults, option_messages, DEFAULT_CONFIG_FILE
+from catalyst.defaults import (confdefaults, option_messages,
+                               DEFAULT_CONFIG_FILE, valid_config_file_values)
 from catalyst.support import CatalystError
 from catalyst.version import get_version
 
@@ -34,7 +35,12 @@ def parse_config(config_files):
     for config_file in config_files:
         log.notice('Loading configuration file: %s', config_file)
         try:
-            conf_values.update(toml.load(config_file))
+            config = toml.load(config_file)
+            for key in config:
+                if key not in valid_config_file_values:
+                    log.critical("Unknown option '%s' in config file %s",
+                                 key, config_file)
+            conf_values.update(config)
         except Exception as e:
             log.critical('Could not find parse configuration file: %s: %s',
                          config_file, e)

[gentoo-commits] proj/catalyst:wip/mattst88 commit in: catalyst/

[Matt Turner]

commit:     5505b112619bc4ef4fff1421b88d130701ecc22c
Author:     Matt Turner <mattst88 <AT> gentoo <DOT> org>
AuthorDate: Tue May 19 19:14:29 2020 +0000
Commit:     Matt Turner <mattst88 <AT> gentoo <DOT> org>
CommitDate: Tue May 19 23:45:26 2020 +0000
URL:        https://gitweb.gentoo.org/proj/catalyst.git/commit/?id=5505b112

catalyst: Drop ConfigParser

Signed-off-by: Matt Turner <mattst88 <AT> gentoo.org>

 catalyst/config.py | 11 -----------
 1 file changed, 11 deletions(-)

diff --git a/catalyst/config.py b/catalyst/config.py
index b527ada0..e1963f71 100644
--- a/catalyst/config.py
+++ b/catalyst/config.py
@@ -118,14 +118,3 @@ class SpecParser(ParserBase):
     def __init__(self, filename=""):
         if filename:
             self.parse_file(filename)
-
-
-class ConfigParser(ParserBase):
-
-    key_value_separator = '='
-    multiple_values = False
-    empty_values = True
-
-    def __init__(self, filename=""):
-        if filename:
-            self.parse_file(filename)

[gentoo-commits] proj/catalyst:wip/mattst88 commit in: catalyst/

[Matt Turner]

commit:     0b200aa972b668a7a8e896995874904c47107377
Author:     Matt Turner <mattst88 <AT> gentoo <DOT> org>
AuthorDate: Mon May 18 23:36:41 2020 +0000
Commit:     Matt Turner <mattst88 <AT> gentoo <DOT> org>
CommitDate: Wed May 20 00:14:19 2020 +0000
URL:        https://gitweb.gentoo.org/proj/catalyst.git/commit/?id=0b200aa9

catalyst: Delete unused {required,valid}_build_targets

Fixes: 11423a21603e ([2 of 3] Update module loading for the new python structure)
Signed-off-by: Matt Turner <mattst88 <AT> gentoo.org>

 catalyst/defaults.py | 19 -------------------
 1 file changed, 19 deletions(-)

diff --git a/catalyst/defaults.py b/catalyst/defaults.py
index f6bc1e14..0da717f1 100644
--- a/catalyst/defaults.py
+++ b/catalyst/defaults.py
@@ -8,25 +8,6 @@ from DeComp.definitions import COMPRESSOR_PROGRAM_OPTIONS, XATTRS_OPTIONS
 from DeComp.definitions import DECOMPRESSOR_PROGRAM_OPTIONS, LIST_XATTRS_OPTIONS
 
 
-# these should never be touched
-required_build_targets = [
-    "generic_stage_target",
-    "targetbase",
-]
-
-# new build types should be added here
-valid_build_targets = [
-    "embedded_target",
-    "livecd_stage1_target",
-    "livecd_stage2_target",
-    "netboot_target",
-    "snapshot_target",
-    "stage1_target",
-    "stage2_target",
-    "stage3_target",
-    "stage4_target",
-]
-
 required_config_file_values = [
     "distdir",
     "portdir",

[gentoo-commits] proj/catalyst:wip/mattst88 commit in: catalyst/

[Matt Turner]

commit:     a366ca0ce932edbd073c569e5f0d551cbc52adae
Author:     Matt Turner <mattst88 <AT> gentoo <DOT> org>
AuthorDate: Mon May 18 23:48:22 2020 +0000
Commit:     Matt Turner <mattst88 <AT> gentoo <DOT> org>
CommitDate: Wed May 20 00:24:54 2020 +0000
URL:        https://gitweb.gentoo.org/proj/catalyst.git/commit/?id=a366ca0c

catalyst: Remove spec file options from valid_config_file_values

Signed-off-by: Matt Turner <mattst88 <AT> gentoo.org>

 catalyst/defaults.py | 5 -----
 1 file changed, 5 deletions(-)

diff --git a/catalyst/defaults.py b/catalyst/defaults.py
index ccabd88d..2f2c907f 100644
--- a/catalyst/defaults.py
+++ b/catalyst/defaults.py
@@ -17,12 +17,7 @@ required_config_file_values = [
 
 valid_config_file_values = required_config_file_values[:]
 valid_config_file_values.extend([
-    "compression_mode",
-    "compressor_arch",
-    "compressor_options",
-    "decompressor_search_order",
     "digests",
-    "distcc",
     "envscript",
     "options",
 ])

[gentoo-commits] proj/catalyst:wip/mattst88 commit in: catalyst/

[Matt Turner]

commit:     7d3c8c9ca59d6c934ebd2c50269124ab74c99346
Author:     Matt Turner <mattst88 <AT> gentoo <DOT> org>
AuthorDate: Mon May 18 23:43:52 2020 +0000
Commit:     Matt Turner <mattst88 <AT> gentoo <DOT> org>
CommitDate: Wed May 20 00:20:25 2020 +0000
URL:        https://gitweb.gentoo.org/proj/catalyst.git/commit/?id=7d3c8c9c

catalyst: Disallow DEBUG and VERBOSE in spec files

Catalyst has --debug/-d and --verbose/-v options for this.

Since conf_values is assigned in a very confusing manner in main.py, I
suspect these values were added to the list due to a misunderstanding.

Signed-off-by: Matt Turner <mattst88 <AT> gentoo.org>

 catalyst/defaults.py | 2 --
 1 file changed, 2 deletions(-)

diff --git a/catalyst/defaults.py b/catalyst/defaults.py
index 0da717f1..ccabd88d 100644
--- a/catalyst/defaults.py
+++ b/catalyst/defaults.py
@@ -20,13 +20,11 @@ valid_config_file_values.extend([
     "compression_mode",
     "compressor_arch",
     "compressor_options",
-    "DEBUG",
     "decompressor_search_order",
     "digests",
     "distcc",
     "envscript",
     "options",
-    "VERBOSE",
 ])
 
 confdefaults = {

[gentoo-commits] proj/catalyst:wip/mattst88 commit in: catalyst/

[Matt Turner]

commit:     09c9a23de7abc124442e7954db08f198dca62d34
Author:     Matt Turner <mattst88 <AT> gentoo <DOT> org>
AuthorDate: Mon May 18 23:49:54 2020 +0000
Commit:     Matt Turner <mattst88 <AT> gentoo <DOT> org>
CommitDate: Wed May 20 00:24:54 2020 +0000
URL:        https://gitweb.gentoo.org/proj/catalyst.git/commit/?id=09c9a23d

catalyst: Add a missing options to valid_config_file_values

And remove required_config_file_values since we want to support running
catalyst without a config file.

Signed-off-by: Matt Turner <mattst88 <AT> gentoo.org>

 catalyst/defaults.py | 20 +++++++++++---------
 1 file changed, 11 insertions(+), 9 deletions(-)

diff --git a/catalyst/defaults.py b/catalyst/defaults.py
index 2f2c907f..412cb956 100644
--- a/catalyst/defaults.py
+++ b/catalyst/defaults.py
@@ -8,18 +8,20 @@ from DeComp.definitions import COMPRESSOR_PROGRAM_OPTIONS, XATTRS_OPTIONS
 from DeComp.definitions import DECOMPRESSOR_PROGRAM_OPTIONS, LIST_XATTRS_OPTIONS
 
 
-required_config_file_values = [
-    "distdir",
-    "portdir",
-    "sharedir",
-    "storedir",
-]
-
-valid_config_file_values = required_config_file_values[:]
-valid_config_file_values.extend([
+valid_config_file_values = frozenset([
     "digests",
+    "distdir",
     "envscript",
     "options",
+    "port_logdir",
+    "repo_basedir",
+    "repo_name",
+    "repos",
+    "sharedir",
+    "storedir",
+    "target_distdir",
+    "target_pkgdir",
+    "var_tmpfs_portage",
 ])
 
 confdefaults = {

[gentoo-commits] proj/catalyst:wip/mattst88 commit in: catalyst/

[Matt Turner]

commit:     49055f32f5ad518501990ab320ae8be39f8d9f3b
Author:     Matt Turner <mattst88 <AT> gentoo <DOT> org>
AuthorDate: Mon May 18 23:34:13 2020 +0000
Commit:     Matt Turner <mattst88 <AT> gentoo <DOT> org>
CommitDate: Wed May 20 00:24:54 2020 +0000
URL:        https://gitweb.gentoo.org/proj/catalyst.git/commit/?id=49055f32

catalyst: Disallow config file options in spec files

Signed-off-by: Matt Turner <mattst88 <AT> gentoo.org>

 catalyst/support.py | 3 +--
 1 file changed, 1 insertion(+), 2 deletions(-)

diff --git a/catalyst/support.py b/catalyst/support.py
index 0925af47..c4a5c797 100644
--- a/catalyst/support.py
+++ b/catalyst/support.py
@@ -8,7 +8,6 @@ import time
 from subprocess import Popen
 
 from catalyst import log
-from catalyst.defaults import valid_config_file_values
 
 BASH_BINARY = "/bin/bash"
 
@@ -211,7 +210,7 @@ def addl_arg_parse(myspec, addlargs, requiredspec, validspec):
     "helper function to help targets parse additional arguments"
     messages = []
     for x in addlargs.keys():
-        if x not in validspec and x not in valid_config_file_values and x not in requiredspec:
+        if x not in validspec and x not in requiredspec:
             messages.append("Argument \""+x+"\" not recognized.")
         else:
             myspec[x] = addlargs[x]

[gentoo-commits] proj/catalyst:wip/mattst88 commit in: catalyst/

[Matt Turner]

commit:     313b193aead3c5ab2e95902f7ab5f999828e9c23
Author:     Matt Turner <mattst88 <AT> gentoo <DOT> org>
AuthorDate: Tue May 19 00:13:14 2020 +0000
Commit:     Matt Turner <mattst88 <AT> gentoo <DOT> org>
CommitDate: Wed May 20 00:24:54 2020 +0000
URL:        https://gitweb.gentoo.org/proj/catalyst.git/commit/?id=313b193a

catalyst: Verify config options against valid_config_file_values

Signed-off-by: Matt Turner <mattst88 <AT> gentoo.org>

 catalyst/main.py | 10 ++++++++--
 1 file changed, 8 insertions(+), 2 deletions(-)

diff --git a/catalyst/main.py b/catalyst/main.py
index 159fe454..543895c6 100644
--- a/catalyst/main.py
+++ b/catalyst/main.py
@@ -15,7 +15,8 @@ from DeComp.contents import ContentsMap
 
 from catalyst import log
 import catalyst.config
-from catalyst.defaults import confdefaults, option_messages, DEFAULT_CONFIG_FILE
+from catalyst.defaults import (confdefaults, option_messages,
+                               DEFAULT_CONFIG_FILE, valid_config_file_values)
 from catalyst.support import CatalystError
 from catalyst.version import get_version
 
@@ -34,7 +35,12 @@ def parse_config(config_files):
     for config_file in config_files:
         log.notice('Loading configuration file: %s', config_file)
         try:
-            conf_values.update(toml.load(config_file))
+            config = toml.load(config_file)
+            for key in config:
+                if key not in valid_config_file_values:
+                    log.critical("Unknown option '%s' in config file %s",
+                                 key, config_file)
+            conf_values.update(config)
         except Exception as e:
             log.critical('Could not find parse configuration file: %s: %s',
                          config_file, e)

[gentoo-commits] proj/catalyst:wip/mattst88 commit in: catalyst/

[Matt Turner]

commit:     474584e2f8822c1b878952e07b1d28a925c0b5e2
Author:     Matt Turner <mattst88 <AT> gentoo <DOT> org>
AuthorDate: Tue May 19 19:14:29 2020 +0000
Commit:     Matt Turner <mattst88 <AT> gentoo <DOT> org>
CommitDate: Wed May 20 00:25:11 2020 +0000
URL:        https://gitweb.gentoo.org/proj/catalyst.git/commit/?id=474584e2

catalyst: Drop ConfigParser

Signed-off-by: Matt Turner <mattst88 <AT> gentoo.org>

 catalyst/config.py | 11 -----------
 1 file changed, 11 deletions(-)

diff --git a/catalyst/config.py b/catalyst/config.py
index b527ada0..e1963f71 100644
--- a/catalyst/config.py
+++ b/catalyst/config.py
@@ -118,14 +118,3 @@ class SpecParser(ParserBase):
     def __init__(self, filename=""):
         if filename:
             self.parse_file(filename)
-
-
-class ConfigParser(ParserBase):
-
-    key_value_separator = '='
-    multiple_values = False
-    empty_values = True
-
-    def __init__(self, filename=""):
-        if filename:
-            self.parse_file(filename)

[gentoo-commits] proj/catalyst:wip/mattst88 commit in: catalyst/

[Matt Turner]

commit:     3b721ac38a6d86c952e8d65e0bba942aa49a3628
Author:     Matt Turner <mattst88 <AT> gentoo <DOT> org>
AuthorDate: Mon May 18 22:41:54 2020 +0000
Commit:     Matt Turner <mattst88 <AT> gentoo <DOT> org>
CommitDate: Wed May 20 01:49:37 2020 +0000
URL:        https://gitweb.gentoo.org/proj/catalyst.git/commit/?id=3b721ac3

catalyst: Switch internal snapshot option parsing to SpecParser

The --snapshot/-s option internally creates a .spec file but uses the
ConfigParser (nominally used for parsing catalyst.conf) rather than
SpecParser (used for parsing .spec files) and as a result has to use
'=' rather than ':' as the key/value delimiter.

Signed-off-by: Matt Turner <mattst88 <AT> gentoo.org>

 catalyst/main.py | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/catalyst/main.py b/catalyst/main.py
index b01d7a6a..be06ccd7 100644
--- a/catalyst/main.py
+++ b/catalyst/main.py
@@ -291,8 +291,8 @@ def _main(parser, opts):
 
     mycmdline = list()
     if opts.snapshot:
-        mycmdline.append('target=snapshot')
-        mycmdline.append('snapshot_treeish=' + opts.snapshot)
+        mycmdline.append('target: snapshot')
+        mycmdline.append('snapshot_treeish: ' + opts.snapshot)
 
     conf_values['DEBUG'] = opts.debug
     conf_values['VERBOSE'] = opts.debug or opts.verbose
@@ -354,7 +354,7 @@ def _main(parser, opts):
 
     if mycmdline:
         try:
-            cmdline = catalyst.config.ConfigParser()
+            cmdline = catalyst.config.SpecParser()
             cmdline.parse_lines(mycmdline)
             addlargs.update(cmdline.get_values())
         except CatalystError:

[gentoo-commits] proj/catalyst:wip/mattst88 commit in: catalyst/

[Matt Turner]

commit:     f40238e0c5644db5dcaaab9f0305cc9d62b144a1
Author:     Matt Turner <mattst88 <AT> gentoo <DOT> org>
AuthorDate: Mon May 18 22:52:36 2020 +0000
Commit:     Matt Turner <mattst88 <AT> gentoo <DOT> org>
CommitDate: Wed May 20 01:49:37 2020 +0000
URL:        https://gitweb.gentoo.org/proj/catalyst.git/commit/?id=f40238e0

catalyst: Remove PythonDir setting

Was used to find the arch directory containing, e.g., sparc.py, but all
that code is gone now after the modules were converted to TOML.

Signed-off-by: Matt Turner <mattst88 <AT> gentoo.org>

 catalyst/defaults.py | 1 -
 catalyst/main.py     | 3 ---
 2 files changed, 4 deletions(-)

diff --git a/catalyst/defaults.py b/catalyst/defaults.py
index 404f4892..f6bc1e14 100644
--- a/catalyst/defaults.py
+++ b/catalyst/defaults.py
@@ -64,7 +64,6 @@ confdefaults = {
     "options": set(),
     "pkgdir": "/var/cache/binpkgs",
     "port_tmpdir": "/var/tmp/portage",
-    "PythonDir": "./catalyst",
     "repo_basedir": "/var/db/repos",
     "repo_name": "gentoo",
     "repos": "%(storedir)s/repos",

diff --git a/catalyst/main.py b/catalyst/main.py
index bad712fa..b01d7a6a 100644
--- a/catalyst/main.py
+++ b/catalyst/main.py
@@ -55,9 +55,6 @@ def parse_config(config_files):
         else:
             conf_values[x] = confdefaults[x]
 
-    # add our python base directory to use for loading target arch's
-    conf_values["PythonDir"] = os.path.dirname(os.path.realpath(__file__))
-
     # print out any options messages
     for opt in conf_values['options']:
         if opt in option_messages:

[gentoo-commits] proj/catalyst:wip/mattst88 commit in: catalyst/

[Matt Turner]

commit:     4c97ff75f71b5fc86b0a3b89bc960966fd0959a6
Author:     Matt Turner <mattst88 <AT> gentoo <DOT> org>
AuthorDate: Mon May 18 23:43:52 2020 +0000
Commit:     Matt Turner <mattst88 <AT> gentoo <DOT> org>
CommitDate: Wed May 20 01:49:37 2020 +0000
URL:        https://gitweb.gentoo.org/proj/catalyst.git/commit/?id=4c97ff75

catalyst: Disallow DEBUG and VERBOSE in spec files

Catalyst has --debug/-d and --verbose/-v options for this.

Since conf_values is assigned in a very confusing manner in main.py, I
suspect these values were added to the list due to a misunderstanding.

Signed-off-by: Matt Turner <mattst88 <AT> gentoo.org>

 catalyst/defaults.py | 2 --
 1 file changed, 2 deletions(-)

diff --git a/catalyst/defaults.py b/catalyst/defaults.py
index 0da717f1..ccabd88d 100644
--- a/catalyst/defaults.py
+++ b/catalyst/defaults.py
@@ -20,13 +20,11 @@ valid_config_file_values.extend([
     "compression_mode",
     "compressor_arch",
     "compressor_options",
-    "DEBUG",
     "decompressor_search_order",
     "digests",
     "distcc",
     "envscript",
     "options",
-    "VERBOSE",
 ])
 
 confdefaults = {

[gentoo-commits] proj/catalyst:wip/mattst88 commit in: catalyst/

[Matt Turner]

commit:     59a3b592d0ede9f0bb6e323f58443acb6ef3b297
Author:     Matt Turner <mattst88 <AT> gentoo <DOT> org>
AuthorDate: Mon May 18 23:49:54 2020 +0000
Commit:     Matt Turner <mattst88 <AT> gentoo <DOT> org>
CommitDate: Wed May 20 01:49:37 2020 +0000
URL:        https://gitweb.gentoo.org/proj/catalyst.git/commit/?id=59a3b592

catalyst: Add a missing options to valid_config_file_values

And remove required_config_file_values since we want to support running
catalyst without a config file.

Signed-off-by: Matt Turner <mattst88 <AT> gentoo.org>

 catalyst/defaults.py | 20 +++++++++++---------
 1 file changed, 11 insertions(+), 9 deletions(-)

diff --git a/catalyst/defaults.py b/catalyst/defaults.py
index 2f2c907f..412cb956 100644
--- a/catalyst/defaults.py
+++ b/catalyst/defaults.py
@@ -8,18 +8,20 @@ from DeComp.definitions import COMPRESSOR_PROGRAM_OPTIONS, XATTRS_OPTIONS
 from DeComp.definitions import DECOMPRESSOR_PROGRAM_OPTIONS, LIST_XATTRS_OPTIONS
 
 
-required_config_file_values = [
-    "distdir",
-    "portdir",
-    "sharedir",
-    "storedir",
-]
-
-valid_config_file_values = required_config_file_values[:]
-valid_config_file_values.extend([
+valid_config_file_values = frozenset([
     "digests",
+    "distdir",
     "envscript",
     "options",
+    "port_logdir",
+    "repo_basedir",
+    "repo_name",
+    "repos",
+    "sharedir",
+    "storedir",
+    "target_distdir",
+    "target_pkgdir",
+    "var_tmpfs_portage",
 ])
 
 confdefaults = {

[gentoo-commits] proj/catalyst:wip/mattst88 commit in: catalyst/

[Matt Turner]

commit:     8443f89f3c5dfbc011a333e90bfef772a95e47cd
Author:     Matt Turner <mattst88 <AT> gentoo <DOT> org>
AuthorDate: Tue May 19 00:13:14 2020 +0000
Commit:     Matt Turner <mattst88 <AT> gentoo <DOT> org>
CommitDate: Wed May 20 01:49:37 2020 +0000
URL:        https://gitweb.gentoo.org/proj/catalyst.git/commit/?id=8443f89f

catalyst: Verify config options against valid_config_file_values

Signed-off-by: Matt Turner <mattst88 <AT> gentoo.org>

 catalyst/main.py | 10 ++++++++--
 1 file changed, 8 insertions(+), 2 deletions(-)

diff --git a/catalyst/main.py b/catalyst/main.py
index 159fe454..543895c6 100644
--- a/catalyst/main.py
+++ b/catalyst/main.py
@@ -15,7 +15,8 @@ from DeComp.contents import ContentsMap
 
 from catalyst import log
 import catalyst.config
-from catalyst.defaults import confdefaults, option_messages, DEFAULT_CONFIG_FILE
+from catalyst.defaults import (confdefaults, option_messages,
+                               DEFAULT_CONFIG_FILE, valid_config_file_values)
 from catalyst.support import CatalystError
 from catalyst.version import get_version
 
@@ -34,7 +35,12 @@ def parse_config(config_files):
     for config_file in config_files:
         log.notice('Loading configuration file: %s', config_file)
         try:
-            conf_values.update(toml.load(config_file))
+            config = toml.load(config_file)
+            for key in config:
+                if key not in valid_config_file_values:
+                    log.critical("Unknown option '%s' in config file %s",
+                                 key, config_file)
+            conf_values.update(config)
         except Exception as e:
             log.critical('Could not find parse configuration file: %s: %s',
                          config_file, e)

[gentoo-commits] proj/catalyst:wip/mattst88 commit in: catalyst/

[Matt Turner]

commit:     f300760bd89aef89810c3ebf6570df1ccd3379c2
Author:     Matt Turner <mattst88 <AT> gentoo <DOT> org>
AuthorDate: Tue May 19 19:14:29 2020 +0000
Commit:     Matt Turner <mattst88 <AT> gentoo <DOT> org>
CommitDate: Wed May 20 01:52:18 2020 +0000
URL:        https://gitweb.gentoo.org/proj/catalyst.git/commit/?id=f300760b

catalyst: Drop ConfigParser

Signed-off-by: Matt Turner <mattst88 <AT> gentoo.org>

 catalyst/config.py | 11 -----------
 1 file changed, 11 deletions(-)

diff --git a/catalyst/config.py b/catalyst/config.py
index b527ada0..e1963f71 100644
--- a/catalyst/config.py
+++ b/catalyst/config.py
@@ -118,14 +118,3 @@ class SpecParser(ParserBase):
     def __init__(self, filename=""):
         if filename:
             self.parse_file(filename)
-
-
-class ConfigParser(ParserBase):
-
-    key_value_separator = '='
-    multiple_values = False
-    empty_values = True
-
-    def __init__(self, filename=""):
-        if filename:
-            self.parse_file(filename)

[gentoo-commits] proj/catalyst:wip/mattst88 commit in: catalyst/

[Matt Turner]

commit:     5b1793bc3d71b81c70a83b8fd92c0069c490f64d
Author:     Matt Turner <mattst88 <AT> gentoo <DOT> org>
AuthorDate: Mon May 18 23:36:41 2020 +0000
Commit:     Matt Turner <mattst88 <AT> gentoo <DOT> org>
CommitDate: Wed May 20 01:49:37 2020 +0000
URL:        https://gitweb.gentoo.org/proj/catalyst.git/commit/?id=5b1793bc

catalyst: Delete unused {required,valid}_build_targets

Fixes: 11423a21603e ([2 of 3] Update module loading for the new python structure)
Signed-off-by: Matt Turner <mattst88 <AT> gentoo.org>

 catalyst/defaults.py | 19 -------------------
 1 file changed, 19 deletions(-)

diff --git a/catalyst/defaults.py b/catalyst/defaults.py
index f6bc1e14..0da717f1 100644
--- a/catalyst/defaults.py
+++ b/catalyst/defaults.py
@@ -8,25 +8,6 @@ from DeComp.definitions import COMPRESSOR_PROGRAM_OPTIONS, XATTRS_OPTIONS
 from DeComp.definitions import DECOMPRESSOR_PROGRAM_OPTIONS, LIST_XATTRS_OPTIONS
 
 
-# these should never be touched
-required_build_targets = [
-    "generic_stage_target",
-    "targetbase",
-]
-
-# new build types should be added here
-valid_build_targets = [
-    "embedded_target",
-    "livecd_stage1_target",
-    "livecd_stage2_target",
-    "netboot_target",
-    "snapshot_target",
-    "stage1_target",
-    "stage2_target",
-    "stage3_target",
-    "stage4_target",
-]
-
 required_config_file_values = [
     "distdir",
     "portdir",

[gentoo-commits] proj/catalyst:wip/mattst88 commit in: catalyst/

[Matt Turner]

commit:     fd731821ae4c7b15c7fedfad794bd3b0d05ebb7a
Author:     Matt Turner <mattst88 <AT> gentoo <DOT> org>
AuthorDate: Mon May 18 23:34:13 2020 +0000
Commit:     Matt Turner <mattst88 <AT> gentoo <DOT> org>
CommitDate: Wed May 20 01:49:37 2020 +0000
URL:        https://gitweb.gentoo.org/proj/catalyst.git/commit/?id=fd731821

catalyst: Disallow config file options in spec files

Signed-off-by: Matt Turner <mattst88 <AT> gentoo.org>

 catalyst/support.py | 3 +--
 1 file changed, 1 insertion(+), 2 deletions(-)

diff --git a/catalyst/support.py b/catalyst/support.py
index 0925af47..c4a5c797 100644
--- a/catalyst/support.py
+++ b/catalyst/support.py
@@ -8,7 +8,6 @@ import time
 from subprocess import Popen
 
 from catalyst import log
-from catalyst.defaults import valid_config_file_values
 
 BASH_BINARY = "/bin/bash"
 
@@ -211,7 +210,7 @@ def addl_arg_parse(myspec, addlargs, requiredspec, validspec):
     "helper function to help targets parse additional arguments"
     messages = []
     for x in addlargs.keys():
-        if x not in validspec and x not in valid_config_file_values and x not in requiredspec:
+        if x not in validspec and x not in requiredspec:
             messages.append("Argument \""+x+"\" not recognized.")
         else:
             myspec[x] = addlargs[x]

[gentoo-commits] proj/catalyst:wip/mattst88 commit in: catalyst/

[Matt Turner]

commit:     6ed67c24fe3bb8e3baa8c936249e4507c45f86a2
Author:     Matt Turner <mattst88 <AT> gentoo <DOT> org>
AuthorDate: Mon May 18 23:48:22 2020 +0000
Commit:     Matt Turner <mattst88 <AT> gentoo <DOT> org>
CommitDate: Wed May 20 01:49:37 2020 +0000
URL:        https://gitweb.gentoo.org/proj/catalyst.git/commit/?id=6ed67c24

catalyst: Remove spec file options from valid_config_file_values

Signed-off-by: Matt Turner <mattst88 <AT> gentoo.org>

 catalyst/defaults.py | 5 -----
 1 file changed, 5 deletions(-)

diff --git a/catalyst/defaults.py b/catalyst/defaults.py
index ccabd88d..2f2c907f 100644
--- a/catalyst/defaults.py
+++ b/catalyst/defaults.py
@@ -17,12 +17,7 @@ required_config_file_values = [
 
 valid_config_file_values = required_config_file_values[:]
 valid_config_file_values.extend([
-    "compression_mode",
-    "compressor_arch",
-    "compressor_options",
-    "decompressor_search_order",
     "digests",
-    "distcc",
     "envscript",
     "options",
 ])

[gentoo-commits] proj/catalyst:wip/mattst88 commit in: catalyst/

[Matt Turner]

commit:     200c12f64d3ed28747e8498ede15cbbacf9af6bf
Author:     Matt Turner <mattst88 <AT> gentoo <DOT> org>
AuthorDate: Tue May 19 19:14:29 2020 +0000
Commit:     Matt Turner <mattst88 <AT> gentoo <DOT> org>
CommitDate: Wed May 20 02:29:23 2020 +0000
URL:        https://gitweb.gentoo.org/proj/catalyst.git/commit/?id=200c12f6

catalyst: Drop ConfigParser

Signed-off-by: Matt Turner <mattst88 <AT> gentoo.org>

 catalyst/config.py | 11 -----------
 1 file changed, 11 deletions(-)

diff --git a/catalyst/config.py b/catalyst/config.py
index b527ada0..e1963f71 100644
--- a/catalyst/config.py
+++ b/catalyst/config.py
@@ -118,14 +118,3 @@ class SpecParser(ParserBase):
     def __init__(self, filename=""):
         if filename:
             self.parse_file(filename)
-
-
-class ConfigParser(ParserBase):
-
-    key_value_separator = '='
-    multiple_values = False
-    empty_values = True
-
-    def __init__(self, filename=""):
-        if filename:
-            self.parse_file(filename)

[gentoo-commits] proj/catalyst:wip/mattst88 commit in: catalyst/

[Matt Turner]

commit:     3d3a1b1b23adff8d97ca746c10d97bb4a02f88bb
Author:     Matt Turner <mattst88 <AT> gentoo <DOT> org>
AuthorDate: Tue May 19 19:14:29 2020 +0000
Commit:     Matt Turner <mattst88 <AT> gentoo <DOT> org>
CommitDate: Wed May 20 02:33:46 2020 +0000
URL:        https://gitweb.gentoo.org/proj/catalyst.git/commit/?id=3d3a1b1b

catalyst: Drop ConfigParser

Signed-off-by: Matt Turner <mattst88 <AT> gentoo.org>

 catalyst/config.py | 11 -----------
 1 file changed, 11 deletions(-)

diff --git a/catalyst/config.py b/catalyst/config.py
index b527ada0..e1963f71 100644
--- a/catalyst/config.py
+++ b/catalyst/config.py
@@ -118,14 +118,3 @@ class SpecParser(ParserBase):
     def __init__(self, filename=""):
         if filename:
             self.parse_file(filename)
-
-
-class ConfigParser(ParserBase):
-
-    key_value_separator = '='
-    multiple_values = False
-    empty_values = True
-
-    def __init__(self, filename=""):
-        if filename:
-            self.parse_file(filename)

[gentoo-commits] proj/catalyst:wip/mattst88 commit in: catalyst/

[Matt Turner]

commit:     103ae1aaa8a363b0cf2abb667ed13c624b3c431a
Author:     Matt Turner <mattst88 <AT> gentoo <DOT> org>
AuthorDate: Tue May 19 19:14:29 2020 +0000
Commit:     Matt Turner <mattst88 <AT> gentoo <DOT> org>
CommitDate: Wed May 20 03:39:21 2020 +0000
URL:        https://gitweb.gentoo.org/proj/catalyst.git/commit/?id=103ae1aa

catalyst: Drop ConfigParser

Signed-off-by: Matt Turner <mattst88 <AT> gentoo.org>

 catalyst/config.py | 11 -----------
 1 file changed, 11 deletions(-)

diff --git a/catalyst/config.py b/catalyst/config.py
index b527ada0..e1963f71 100644
--- a/catalyst/config.py
+++ b/catalyst/config.py
@@ -118,14 +118,3 @@ class SpecParser(ParserBase):
     def __init__(self, filename=""):
         if filename:
             self.parse_file(filename)
-
-
-class ConfigParser(ParserBase):
-
-    key_value_separator = '='
-    multiple_values = False
-    empty_values = True
-
-    def __init__(self, filename=""):
-        if filename:
-            self.parse_file(filename)

[gentoo-commits] proj/catalyst:wip/mattst88 commit in: catalyst/

[Matt Turner]

commit:     016c8460c88c89e55f85e34664fe7928b0f1c6a6
Author:     Matt Turner <mattst88 <AT> gentoo <DOT> org>
AuthorDate: Tue May 19 19:14:29 2020 +0000
Commit:     Matt Turner <mattst88 <AT> gentoo <DOT> org>
CommitDate: Thu May 21 20:26:15 2020 +0000
URL:        https://gitweb.gentoo.org/proj/catalyst.git/commit/?id=016c8460

catalyst: Drop ConfigParser

Signed-off-by: Matt Turner <mattst88 <AT> gentoo.org>

 catalyst/config.py | 11 -----------
 1 file changed, 11 deletions(-)

diff --git a/catalyst/config.py b/catalyst/config.py
index b527ada0..e1963f71 100644
--- a/catalyst/config.py
+++ b/catalyst/config.py
@@ -118,14 +118,3 @@ class SpecParser(ParserBase):
     def __init__(self, filename=""):
         if filename:
             self.parse_file(filename)
-
-
-class ConfigParser(ParserBase):
-
-    key_value_separator = '='
-    multiple_values = False
-    empty_values = True
-
-    def __init__(self, filename=""):
-        if filename:
-            self.parse_file(filename)

[gentoo-commits] proj/catalyst:wip/mattst88 commit in: catalyst/

[Matt Turner]

commit:     4bcc64dcf2b27ab3d0effed0fa2516e3513a9171
Author:     Matt Turner <mattst88 <AT> gentoo <DOT> org>
AuthorDate: Wed Oct 28 21:59:17 2020 +0000
Commit:     Matt Turner <mattst88 <AT> gentoo <DOT> org>
CommitDate: Fri Oct 30 22:40:52 2020 +0000
URL:        https://gitweb.gentoo.org/proj/catalyst.git/commit/?id=4bcc64dc

catalyst: Add and use namespace context manager

Wraps snakeoil's simple_unshare; returns to the previous namespaces on
context exit. Will be used by the next commit.

Signed-off-by: Matt Turner <mattst88 <AT> gentoo.org>

 catalyst/context.py | 32 ++++++++++++++++++++++++++++++++
 catalyst/main.py    | 17 +++++++----------
 2 files changed, 39 insertions(+), 10 deletions(-)

diff --git a/catalyst/context.py b/catalyst/context.py
new file mode 100644
index 00000000..8a58f33d
--- /dev/null
+++ b/catalyst/context.py
@@ -0,0 +1,32 @@
+
+import contextlib
+import os
+
+from snakeoil.process.namespaces import setns, simple_unshare
+
+@contextlib.contextmanager
+def namespace(mount=False, uts=False, ipc=False, net=False, pid=False,
+              user=False, hostname=None):
+    namespaces = {
+        (mount, "mnt"):  None,
+        (uts,   "uts"):  None,
+        (ipc,   "ipc"):  None,
+        (net,   "net"):  None,
+        (pid,   "pid"):  None,
+        (user,  "user"): None,
+    }
+
+    # Save fds of current namespaces
+    for ns in [ns for ns in namespaces if ns[0]]:
+        fp = open(f"/proc/self/ns/{ns[1]}")
+        namespaces[ns] = fp
+
+    simple_unshare(mount=mount, uts=uts, ipc=ipc, net=net, pid=pid, user=user,
+                   hostname=hostname)
+    try:
+        yield
+    finally:
+        for ns in [ns for ns in namespaces if ns[0]]:
+            fp = namespaces[ns]
+            setns(fp.fileno(), 0)
+            fp.close()

diff --git a/catalyst/main.py b/catalyst/main.py
index 543895c6..93a4a0d3 100644
--- a/catalyst/main.py
+++ b/catalyst/main.py
@@ -7,14 +7,13 @@ import textwrap
 
 import toml
 
-from snakeoil.process import namespaces
-
 from DeComp.definitions import (COMPRESS_DEFINITIONS, DECOMPRESS_DEFINITIONS,
                                 CONTENTS_DEFINITIONS)
 from DeComp.contents import ContentsMap
 
 from catalyst import log
 import catalyst.config
+from catalyst.context import namespace
 from catalyst.defaults import (confdefaults, option_messages,
                                DEFAULT_CONFIG_FILE, valid_config_file_values)
 from catalyst.support import CatalystError
@@ -356,15 +355,13 @@ def _main(parser, opts):
     # use pid & user namespaces, but snakeoil's namespace module has signal
     # transfer issues (CTRL+C doesn't propagate), and user namespaces need
     # more work due to Gentoo build process (uses sudo/root/portage).
-    namespaces.simple_unshare(
-        mount=True, uts=True, ipc=True, pid=False, net=False, user=False,
-        hostname='catalyst')
+    with namespace(mount=True, uts=True, ipc=True, hostname='catalyst'):
+        # everything is setup, so the build is a go
+        try:
+            success = build_target(addlargs)
+        except KeyboardInterrupt:
+            log.critical('Catalyst build aborted due to user interrupt (Ctrl-C)')
 
-    # everything is setup, so the build is a go
-    try:
-        success = build_target(addlargs)
-    except KeyboardInterrupt:
-        log.critical('Catalyst build aborted due to user interrupt (Ctrl-C)')
     if not success:
         sys.exit(2)
     sys.exit(0)

[gentoo-commits] proj/catalyst:wip/mattst88 commit in: catalyst/

[Matt Turner]

commit:     dccfafea4e8b4622a7527a764d581ac0123b107e
Author:     Felix Bier <Felix.Bier <AT> rohde-schwarz <DOT> com>
AuthorDate: Mon Nov  9 19:36:33 2020 +0000
Commit:     Matt Turner <mattst88 <AT> gentoo <DOT> org>
CommitDate: Sat Nov 14 16:34:57 2020 +0000
URL:        https://gitweb.gentoo.org/proj/catalyst.git/commit/?id=dccfafea

catalyst: Restore root and cwd after exiting mount namespace

This commit saves the file descriptor of /proc/self/{root,cwd}
before entering into the new mount namespace. When restoring the
previous mount namespace, it restores /proc/self/{root,cwd}
based on the saved file descriptors.

Without this change, catalyst cannot be run in a chroot when
using the recent changes regarding mount namespaces: After the
mount namespace has been exited, /proc/self/root points to the "/"
of the host system, not the "/" of the chroot. Therefore, the
cleanup phase of catalyst runs outside of the chroot.

The code is similar to how nsenter(1) sets root and cwd:
https://git.kernel.org/pub/scm/utils/util-linux/util-linux.git/tree/sys-utils/nsenter.c#n452

Tested in a Gentoo chroot and in Gentoo VM (non-chroot).

Signed-off-by: Felix Bier <felix.bier <AT> rohde-schwarz.com>
Signed-off-by: Matt Turner <mattst88 <AT> gentoo.org>

 catalyst/context.py | 22 ++++++++++++++++++++++
 1 file changed, 22 insertions(+)

diff --git a/catalyst/context.py b/catalyst/context.py
index 8a58f33d..01a6d930 100644
--- a/catalyst/context.py
+++ b/catalyst/context.py
@@ -16,11 +16,21 @@ def namespace(mount=False, uts=False, ipc=False, net=False, pid=False,
         (user,  "user"): None,
     }
 
+    dirs = {
+        "root": None,
+        "cwd":  None,
+    }
+
     # Save fds of current namespaces
     for ns in [ns for ns in namespaces if ns[0]]:
         fp = open(f"/proc/self/ns/{ns[1]}")
         namespaces[ns] = fp
 
+    # Save fds of current directories
+    if mount:
+        for d in dirs:
+            dirs[d] = os.open(f"/proc/self/{d}", os.O_RDONLY)
+
     simple_unshare(mount=mount, uts=uts, ipc=ipc, net=net, pid=pid, user=user,
                    hostname=hostname)
     try:
@@ -30,3 +40,15 @@ def namespace(mount=False, uts=False, ipc=False, net=False, pid=False,
             fp = namespaces[ns]
             setns(fp.fileno(), 0)
             fp.close()
+
+        if mount:
+            # Restore original root and cwd. Since we cannot directly chroot to
+            # a fd, first change the current directory to the fd of the
+            # original root, then chroot to "."
+
+            os.fchdir(dirs["root"])
+            os.chroot(".")
+            os.fchdir(dirs["cwd"])
+
+            for fd in dirs.values():
+                os.close(fd)

[gentoo-commits] proj/catalyst:wip/mattst88 commit in: catalyst/

[Matt Turner]

commit:     b8b6de7bc84677d9ae13cf8bfff2c0e6a281ff75
Author:     Matt Turner <mattst88 <AT> gentoo <DOT> org>
AuthorDate: Sun Jan 24 02:34:06 2021 +0000
Commit:     Matt Turner <mattst88 <AT> gentoo <DOT> org>
CommitDate: Thu Jan 28 02:06:46 2021 +0000
URL:        https://gitweb.gentoo.org/proj/catalyst.git/commit/?id=b8b6de7b

catalyst: Remove fallback make.conf parsing code

Signed-off-by: Matt Turner <mattst88 <AT> gentoo.org>

 catalyst/support.py | 42 +++---------------------------------------
 1 file changed, 3 insertions(+), 39 deletions(-)

diff --git a/catalyst/support.py b/catalyst/support.py
index f3a865a7..fa652987 100644
--- a/catalyst/support.py
+++ b/catalyst/support.py
@@ -2,7 +2,6 @@
 import glob
 import sys
 import os
-import re
 import shutil
 import time
 from pathlib import Path
@@ -12,6 +11,8 @@ import libmount
 
 from portage.repository.config import RepoConfig
 
+from snakeoil.bash import read_bash_dict
+
 from catalyst import log
 
 BASH_BINARY = "/bin/bash"
@@ -135,47 +136,10 @@ defined are not preserved. In other words, "foo", "bar", "oni" ordering is prese
                                     print_traceback=True)
 
 
-def parse_makeconf(mylines):
-    mymakeconf = {}
-    pos = 0
-    pat = re.compile("([0-9a-zA-Z_]*)=(.*)")
-    while pos < len(mylines):
-        if len(mylines[pos]) <= 1:
-            # skip blanks
-            pos += 1
-            continue
-        if mylines[pos][0] in ["#", " ", "\t"]:
-            # skip indented lines, comments
-            pos += 1
-            continue
-        else:
-            myline = mylines[pos]
-            mobj = pat.match(myline)
-            pos += 1
-            if mobj.group(2):
-                clean_string = re.sub(r"\"", r"", mobj.group(2))
-                mymakeconf[mobj.group(1)] = clean_string
-    return mymakeconf
-
-
 def read_makeconf(mymakeconffile):
     if os.path.exists(mymakeconffile):
         try:
-            try:
-                import snakeoil.bash  # import snakeoil.fileutils
-                return snakeoil.bash.read_bash_dict(mymakeconffile, sourcing_command="source")
-            except ImportError:
-                try:
-                    import portage.util
-                    return portage.util.getconfig(mymakeconffile, tolerant=1, allow_sourcing=True)
-                except Exception:
-                    try:
-                        import portage_util
-                        return portage_util.getconfig(mymakeconffile, tolerant=1, allow_sourcing=True)
-                    except ImportError:
-                        with open(mymakeconffile, "r") as myf:
-                            mylines = myf.readlines()
-                        return parse_makeconf(mylines)
+            return read_bash_dict(mymakeconffile, sourcing_command="source")
         except Exception:
             raise CatalystError("Could not parse make.conf file " +
                                 mymakeconffile, print_traceback=True)

[gentoo-commits] proj/catalyst:wip/mattst88 commit in: catalyst/

[Matt Turner]

commit:     e40f31c10a79499a822596948b68fcf6ea3f647f
Author:     Matt Turner <mattst88 <AT> gentoo <DOT> org>
AuthorDate: Mon Nov  2 17:43:34 2020 +0000
Commit:     Matt Turner <mattst88 <AT> gentoo <DOT> org>
CommitDate: Fri Jan 29 23:48:02 2021 +0000
URL:        https://gitweb.gentoo.org/proj/catalyst.git/commit/?id=e40f31c1

catalyst: Switch spec files to TOML

Signed-off-by: Matt Turner <mattst88 <AT> gentoo.org>

 catalyst/config.py | 120 -----------------------------------------------------
 catalyst/main.py   |  26 ++++--------
 2 files changed, 9 insertions(+), 137 deletions(-)

diff --git a/catalyst/config.py b/catalyst/config.py
deleted file mode 100644
index e1963f71..00000000
--- a/catalyst/config.py
+++ /dev/null
@@ -1,120 +0,0 @@
-
-import re
-
-from catalyst import log
-from catalyst.support import CatalystError
-
-
-class ParserBase():
-
-    filename = ""
-    lines = None
-    values = None
-    key_value_separator = "="
-    multiple_values = False
-    empty_values = True
-    eval_none = False
-
-    def __getitem__(self, key):
-        return self.values[key]
-
-    def get_values(self):
-        return self.values
-
-    def dump(self):
-        dump = ""
-        for x in self.values:
-            dump += x + " = " + repr(self.values[x]) + "\n"
-        return dump
-
-    def parse_file(self, filename):
-        try:
-            with open(filename, "r") as myf:
-                self.lines = myf.readlines()
-        except:
-            raise CatalystError("Could not open file " + filename,
-                                print_traceback=True)
-        self.filename = filename
-        self.parse()
-
-    def parse_lines(self, lines):
-        self.lines = lines
-        self.parse()
-
-    def parse(self):
-        values = {}
-        cur_array = []
-
-        trailing_comment = re.compile(r'\s*#.*$')
-
-        for x, myline in enumerate(self.lines):
-            myline = myline.strip()
-
-            # Force the line to be clean
-            # Remove Comments ( anything following # )
-            myline = trailing_comment.sub("", myline)
-
-            # Skip any blank lines
-            if not myline:
-                continue
-
-            if self.key_value_separator in myline:
-                # Split on the first occurence of the separator creating two strings in the array mobjs
-                mobjs = myline.split(self.key_value_separator, 1)
-                mobjs[1] = mobjs[1].strip().strip('"')
-
-                # Start a new array using the first element of mobjs
-                cur_array = [mobjs[0]]
-                if mobjs[1]:
-                    # do any variable substitiution embeded in it with
-                    # the values already obtained
-                    mobjs[1] = mobjs[1] % values
-                    if self.multiple_values:
-                        # split on white space creating additional array elements
-                        subarray = mobjs[1].split()
-                        cur_array += subarray
-                    else:
-                        cur_array += [mobjs[1]]
-
-            # Else add on to the last key we were working on
-            else:
-                if self.multiple_values:
-                    cur_array += myline.split()
-                else:
-                    raise CatalystError("Syntax error: %s" %
-                                        x, print_traceback=True)
-
-            # XXX: Do we really still need this "single value is a string" behavior?
-            if len(cur_array) == 2:
-                values[cur_array[0]] = cur_array[1]
-            else:
-                values[cur_array[0]] = cur_array[1:]
-
-        if not self.empty_values:
-            # Make sure the list of keys is static since we modify inside the loop.
-            for x in list(values.keys()):
-                # Delete empty key pairs
-                if not values[x]:
-                    log.warning('No value set for key "%s"; deleting', x)
-                    del values[x]
-
-        if self.eval_none:
-            # Make sure the list of keys is static since we modify inside the loop.
-            for x in list(values.keys()):
-                # reset None values
-                if isinstance(values[x], str) and values[x].lower() in ['none']:
-                    log.info('None value found for key "%s"; reseting', x)
-                    values[x] = None
-        self.values = values
-
-
-class SpecParser(ParserBase):
-
-    key_value_separator = ':'
-    multiple_values = True
-    empty_values = False
-    eval_none = True
-
-    def __init__(self, filename=""):
-        if filename:
-            self.parse_file(filename)

diff --git a/catalyst/main.py b/catalyst/main.py
index 0de1040f..81495c62 100644
--- a/catalyst/main.py
+++ b/catalyst/main.py
@@ -13,7 +13,6 @@ from DeComp.definitions import (COMPRESS_DEFINITIONS, DECOMPRESS_DEFINITIONS,
 from DeComp.contents import ContentsMap
 
 from catalyst import log
-import catalyst.config
 from catalyst.context import namespace
 from catalyst.defaults import (confdefaults, option_messages,
                                DEFAULT_CONFIG_FILE, valid_config_file_values)
@@ -276,11 +275,6 @@ def _main(parser, opts):
         myconfigs = [DEFAULT_CONFIG_FILE]
     myspecfile = opts.file
 
-    mycmdline = list()
-    if opts.snapshot:
-        mycmdline.append('target: snapshot')
-        mycmdline.append('snapshot_treeish: ' + opts.snapshot)
-
     conf_values['DEBUG'] = opts.debug
     conf_values['VERBOSE'] = opts.debug or opts.verbose
 
@@ -299,7 +293,7 @@ def _main(parser, opts):
         options.append('enter-chroot')
 
     # Make sure we have some work before moving further.
-    if not myspecfile and not mycmdline:
+    if not myspecfile and not opts.snapshot:
         parser.error('please specify one of either -f or -C or -s')
 
     # made it this far so start by outputting our version info
@@ -320,7 +314,6 @@ def _main(parser, opts):
     # initialize our (de)compression definitions
     conf_values['decompress_definitions'] = DECOMPRESS_DEFINITIONS
     conf_values['compress_definitions'] = COMPRESS_DEFINITIONS
-    # TODO add capability to config/spec new definitions
 
     if "digests" in conf_values:
         valid_digests = hashlib.algorithms_available
@@ -338,16 +331,15 @@ def _main(parser, opts):
 
     if myspecfile:
         log.notice("Processing spec file: %s", myspecfile)
-        spec = catalyst.config.SpecParser(myspecfile)
-        addlargs.update(spec.get_values())
-
-    if mycmdline:
         try:
-            cmdline = catalyst.config.SpecParser()
-            cmdline.parse_lines(mycmdline)
-            addlargs.update(cmdline.get_values())
-        except CatalystError:
-            log.critical('Could not parse commandline')
+            addlargs.update(toml.load(myspecfile))
+        except Exception as e:
+            log.critical('Could not find parse spec file: %s: %s',
+                         myspecfile, e)
+
+    if opts.snapshot:
+        addlargs['target'] = 'snapshot'
+        addlargs['snapshot_treeish'] = opts.snapshot
 
     if "target" not in addlargs:
         raise CatalystError("Required value \"target\" not specified.")

[gentoo-commits] proj/catalyst:wip/mattst88 commit in: catalyst/

[Matt Turner]

commit:     caf55a942580e02e66ed846f5c3ab4ad5ab8846f
Author:     Felix Bier <Felix.Bier <AT> rohde-schwarz <DOT> com>
AuthorDate: Thu Feb  4 00:37:06 2021 +0000
Commit:     Matt Turner <mattst88 <AT> gentoo <DOT> org>
CommitDate: Sat Feb 20 21:27:29 2021 +0000
URL:        https://gitweb.gentoo.org/proj/catalyst.git/commit/?id=caf55a94

Extend get_repo_name to handle squashed repos

This commit extends the method get_repo_name to also handle
squashed repos. This is done by mounting the squash file to
a temporary directory and then extracting the repository from
that directory with the already existing code.

This is motivated by wanting to mount each repo
to e.g. /var/db/repos/<repo-name> in a later commit.
For squashed repos, we don't know <repo-name> without
mounting the repo first. For this reason, it is mounted to
a temporary directory first to extract <repo-name>.

Signed-off-by: Felix Bier <felix.bier <AT> rohde-schwarz.com>
Signed-off-by: Matt Turner <mattst88 <AT> gentoo.org>

 catalyst/support.py | 36 +++++++++++++++++++++++++++++++++++-
 1 file changed, 35 insertions(+), 1 deletion(-)

diff --git a/catalyst/support.py b/catalyst/support.py
index fc50fa34..37d53bc4 100644
--- a/catalyst/support.py
+++ b/catalyst/support.py
@@ -10,10 +10,12 @@ from subprocess import Popen
 import libmount
 
 from portage.repository.config import RepoConfig
+from tempfile import TemporaryDirectory
 
 from snakeoil.bash import read_bash_dict
 
 from catalyst import log
+from catalyst.context import namespace
 
 BASH_BINARY = "/bin/bash"
 
@@ -148,7 +150,7 @@ def read_makeconf(mymakeconffile):
         return makeconf
 
 
-def get_repo_name(repo_path):
+def get_repo_name_from_dir(repo_path):
     """ Get the name of the repo at the given repo_path.
 
          References:
@@ -164,6 +166,38 @@ def get_repo_name(repo_path):
     return repo_config.name
 
 
+def get_repo_name_from_squash(repo_squash_path):
+    """ Get the name of the repo at the given repo_squash_path.
+        To obtain the name, the squash file is mounted to a temporary directory.
+    """
+
+    repo_name = None
+
+    # Mount squash file to temp directory in separate mount namespace
+    with TemporaryDirectory() as temp, namespace(mount=True):
+        try:
+            source = str(repo_squash_path)
+            target = str(temp)
+            fstype = 'squashfs'
+            options = 'ro,loop'
+            cxt = libmount.Context(source=source, target=target,
+                                   fstype=fstype, options=options)
+            cxt.mount()
+            repo_name = get_repo_name_from_dir(target)
+
+        except Exception as e:
+            raise CatalystError(f"Couldn't mount: {source}, {e}") from e
+
+    return repo_name
+
+
+def get_repo_name(repo_path):
+    if not Path(repo_path).is_dir():
+        return get_repo_name_from_squash(repo_path)
+
+    return get_repo_name_from_dir(repo_path)
+
+
 def ismount(path):
     """Like os.path.ismount, but also support bind mounts"""
     path = Path(path)

[gentoo-commits] proj/catalyst:wip/mattst88 commit in: catalyst/

[Matt Turner]

commit:     4fd2ac23250ab2ac1f6a506ee433f466a4f9e026
Author:     Felix Bier <Felix.Bier <AT> rohde-schwarz <DOT> com>
AuthorDate: Sat Feb 13 23:18:17 2021 +0000
Commit:     Matt Turner <mattst88 <AT> gentoo <DOT> org>
CommitDate: Sat Feb 20 21:27:29 2021 +0000
URL:        https://gitweb.gentoo.org/proj/catalyst.git/commit/?id=4fd2ac23

Enable recursive globbing for clear_path

This commit enables recursive globbing in clear_path, allowing the
usage of '**' to match an arbitrary number of sub-directories.

Before this commit, clear_path used only non-recursive globbing. This
allowed to use '*' to expand names within one directory, e.g. '/a/*/c'
can expand to '/a/b/c', but not '/a/b/b/c'. With this commit, '/a/**/c'
can be used to expand to '/a/b/c', '/a/b/b/c', '/a/b/b/b/c' etc.

This is motivated by wanting to recursively delete all occurences of a
filename with the 'stage4/rm' entry of a spec file. The '/rm' entries
are processed with 'clear_path' in the existing code.

Additionally, 'glob.glob' is replaced with 'glob.iglob',
which returns the same files as 'glob.glob', but as an iterator
instead of as a list (so it no longer necessary to hold
all matches in memory at once).

Recursive globbing has been added in Python 3.5.

References:
https://docs.python.org/3/library/glob.html#glob.glob
https://docs.python.org/3/library/glob.html#glob.iglob

Signed-off-by: Felix Bier <felix.bier <AT> rohde-schwarz.com>
Signed-off-by: Matt Turner <mattst88 <AT> gentoo.org>

 catalyst/fileops.py | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/catalyst/fileops.py b/catalyst/fileops.py
index 5c6f5cd8..59525420 100644
--- a/catalyst/fileops.py
+++ b/catalyst/fileops.py
@@ -99,7 +99,7 @@ def clear_dir(target, mode=0o755, remove=False,
 
 def clear_path(target_path):
     """Nuke |target_path| regardless of it being a dir, file or glob."""
-    targets = glob.glob(target_path)
+    targets = glob.iglob(target_path, recursive=True)
     for path in targets:
         clear_dir(path, remove=True)
 

[gentoo-commits] proj/catalyst:wip/mattst88 commit in: catalyst/

[Matt Turner]

commit:     fbe56d75989fabf29e7b642515df69737cf62cc1
Author:     Matt Turner <mattst88 <AT> gentoo <DOT> org>
AuthorDate: Wed Jun  9 04:32:08 2021 +0000
Commit:     Matt Turner <mattst88 <AT> gentoo <DOT> org>
CommitDate: Wed Jun  9 04:32:08 2021 +0000
URL:        https://gitweb.gentoo.org/proj/catalyst.git/commit/?id=fbe56d75

catalyst: Remove clear_dir's never used clear_nondir parameter

Signed-off-by: Matt Turner <mattst88 <AT> gentoo.org>

 catalyst/fileops.py | 11 +++--------
 1 file changed, 3 insertions(+), 8 deletions(-)

diff --git a/catalyst/fileops.py b/catalyst/fileops.py
index 59525420..4252285e 100644
--- a/catalyst/fileops.py
+++ b/catalyst/fileops.py
@@ -46,8 +46,7 @@ def ensure_dirs(path, gid=-1, uid=-1, mode=0o755, minimal=True,
     return succeeded
 
 
-def clear_dir(target, mode=0o755, remove=False,
-              clear_nondir=True):
+def clear_dir(target, mode=0o755, remove=False):
     '''Universal directory clearing function
 
     @target: string, path to be cleared or removed
@@ -74,12 +73,8 @@ def clear_dir(target, mode=0o755, remove=False,
             log.error('clear_dir failed', exc_info=True)
             return False
     elif os.path.exists(target):
-        if clear_nondir:
-            log.debug("Clearing (unlinking) non-directory: %s", target)
-            os.unlink(target)
-        else:
-            log.info('clear_dir failed: %s: is not a directory', target)
-            return False
+        log.debug("Clearing (unlinking) non-directory: %s", target)
+        os.unlink(target)
     else:
         log.debug("Conditions not met to clear: %s", target)
         log.debug("                      isdir: %s", os.path.isdir(target))

[gentoo-commits] proj/catalyst:wip/mattst88 commit in: catalyst/

[Matt Turner]

commit:     f365eb43a81989ad24f75995df97552e08d3e7ad
Author:     Matt Turner <mattst88 <AT> gentoo <DOT> org>
AuthorDate: Thu Nov 17 23:41:33 2022 +0000
Commit:     Matt Turner <mattst88 <AT> gentoo <DOT> org>
CommitDate: Thu Nov 17 23:42:24 2022 +0000
URL:        https://gitweb.gentoo.org/proj/catalyst.git/commit/?id=f365eb43

catalyst: Remove stray space

Signed-off-by: Matt Turner <mattst88 <AT> gentoo.org>

 catalyst/support.py | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/catalyst/support.py b/catalyst/support.py
index a69566d2..6945d053 100644
--- a/catalyst/support.py
+++ b/catalyst/support.py
@@ -78,7 +78,7 @@ def file_check(filepath, extensions=None):
     files = glob.glob("%s.*" % filepath)
     # remove any false positive files
     files = [x for x in files if
-             not x.endswith( ".CONTENTS") and
+             not x.endswith(".CONTENTS") and
              not x.endswith(".CONTENTS.gz") and
              not x.endswith(".DIGESTS") and
              not x.endswith(".sha256")]