From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from lists.gentoo.org (pigeon.gentoo.org [208.92.234.80]) by finch.gentoo.org (Postfix) with ESMTP id 099CD1389E2 for ; Tue, 16 Dec 2014 05:50:20 +0000 (UTC) Received: from pigeon.gentoo.org (localhost [127.0.0.1]) by pigeon.gentoo.org (Postfix) with SMTP id DFAC2E08C4; Tue, 16 Dec 2014 05:50:14 +0000 (UTC) Received: from smtp.gentoo.org (smtp.gentoo.org [140.211.166.183]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by pigeon.gentoo.org (Postfix) with ESMTPS id 4CD8EE08C2 for ; Tue, 16 Dec 2014 05:50:14 +0000 (UTC) Received: from x200ma.gaikai.int (ip70-181-96-121.oc.oc.cox.net [70.181.96.121]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) (Authenticated sender: zmedico) by smtp.gentoo.org (Postfix) with ESMTPSA id F388434029D; Tue, 16 Dec 2014 05:50:11 +0000 (UTC) From: Zac Medico To: gentoo-portage-dev@lists.gentoo.org Cc: Zac Medico Subject: [gentoo-portage-dev] [PATCH v2] Support eapi-x in layout.conf profile-formats (532670) Date: Mon, 15 Dec 2014 21:49:47 -0800 Message-Id: <1418708987-25674-1-git-send-email-zmedico@gentoo.org> X-Mailer: git-send-email 2.0.4 In-Reply-To: <1418695192-2471-1-git-send-email-zmedico@gentoo.org> References: <1418695192-2471-1-git-send-email-zmedico@gentoo.org> Precedence: bulk List-Post: List-Help: List-Unsubscribe: List-Subscribe: List-Id: Gentoo Linux mail X-BeenThere: gentoo-portage-dev@lists.gentoo.org Reply-to: gentoo-portage-dev@lists.gentoo.org X-Archives-Salt: c85441c7-7bd1-4588-b957-7b9ca2a4f7d9 X-Archives-Hash: 31b01e499a7ab319d185f5d1497b5602 Add support for an eapi-x flag in the metadata/layout.conf profile-formats field, where x must be a supported EAPI. This may be used to set the default EAPI for profile directories that do not contain an eapi file. For example, setting "profile-formats = eapi-5" will cause all profiles that do not contain an eapi file to default to EAPI 5 instead of EAPI 0. This is convenient for organizations that maintain their own profiles in separate repositories from Gentoo, since they typically want to use EAPI 5 for all of their profiles, and this allows them to avoid having separate eapi files in each directory of their profiles. The default EAPI setting is stored in the RepoConfig.eapi attribute, which is used as the "default" parameter for all read_corresponding_eapi_file calls. Each profile node in LocationsManager.profiles_complex now has an eapi attribute which serves to cache the computed EAPI for that node, allowing redundant read_corresponding_eapi_file calls to be eliminated. As a result, functions such as grabfile_package now have eapi and eapi_default parameters, where eapi is used to pass in a cached EAPI, and eapi_default is used to pass in a default for read_corresponding_eapi_file to use when a cached EAPI is not available. For /etc/portage/profile, the EAPI is considered to have a default value of None, which means that atoms from any supported EAPI are allowed. X-Gentoo-Bug: 532670 X-Gentoo-Bug-URL: https://bugs.gentoo.org/show_bug.cgi?id=532670 --- PATCH v2 fixes some eapi_default parameters for correct behavior with /etc/portage/profile where eapi_default should be None. man/portage.5 | 9 +- pym/portage/_sets/ProfilePackageSet.py | 3 +- pym/portage/_sets/profiles.py | 27 +++-- .../package/ebuild/_config/KeywordsManager.py | 6 +- .../package/ebuild/_config/LocationsManager.py | 31 ++++-- pym/portage/package/ebuild/_config/MaskManager.py | 12 +- pym/portage/package/ebuild/_config/UseManager.py | 88 ++++++++++++--- pym/portage/package/ebuild/config.py | 9 +- pym/portage/repository/config.py | 47 ++++++-- .../tests/resolver/test_profile_default_eapi.py | 123 +++++++++++++++++++++ pym/portage/util/__init__.py | 11 +- 11 files changed, 305 insertions(+), 61 deletions(-) create mode 100644 pym/portage/tests/resolver/test_profile_default_eapi.py diff --git a/man/portage.5 b/man/portage.5 index 88cf3bb..2ed4501 100644 --- a/man/portage.5 +++ b/man/portage.5 @@ -254,6 +254,10 @@ use.stable.force, package.use.stable.mask and package.use.stable.force. These files behave similarly to previously supported USE configuration files, except that they only influence packages that are merged due to a stable keyword. + +If the eapi file does not exist, then the \fBEAPI\fR defaults to +\fI0\fR unless the default has been overridden by a profile-formats +setting inside \fImetadata/layout.conf\fR of the containing repository. .TP .BR make.defaults The profile default settings for Portage. The general format is described @@ -1114,7 +1118,7 @@ The default setting for repoman's --echangelog option. The cache formats supported in the metadata tree. There is the old "pms" format and the newer/faster "md5-dict" format. Default is to detect dirs. .TP -.BR profile\-formats " = [pms|portage-1|portage-2|profile-bashrcs|profile-set]" +.BR profile\-formats " = [pms|portage-1|portage-2|profile-bashrcs|profile-set|eapi-x]" Control functionality available to profiles in this repo such as which files may be dirs, or the syntax available in parent files. Use "portage-2" if you're unsure. The default is "portage-1-compat" mode which is meant to be compatible @@ -1123,6 +1127,9 @@ Setting profile-bashrcs will enable the per-profile bashrc mechanism \fBpackage.bashrc\fR. Setting profile-set enables support for using the profile \fBpackages\fR file to add atoms to the @profile package set. See the profile \fBpackages\fR section for more information. +Setting eapi-x, where \fIx\fR must be a specific supported \fBEAPI\fR, +causes a profile to be interpreted with the specified \fBEAPI\fR unless +the profile has an \fIeapi\fR file specifying a different \fBEAPI\fR. .RE .RE diff --git a/pym/portage/_sets/ProfilePackageSet.py b/pym/portage/_sets/ProfilePackageSet.py index c2f5fee..2fcafb6 100644 --- a/pym/portage/_sets/ProfilePackageSet.py +++ b/pym/portage/_sets/ProfilePackageSet.py @@ -23,7 +23,8 @@ class ProfilePackageSet(PackageSet): def load(self): self._setAtoms(x for x in stack_lists( [grabfile_package(os.path.join(y.location, "packages"), - verify_eapi=True) for y in self._profiles + verify_eapi=True, eapi=y.eapi, eapi_default=None) + for y in self._profiles if "profile-set" in y.profile_formats], incremental=1) if x[:1] != "*") diff --git a/pym/portage/_sets/profiles.py b/pym/portage/_sets/profiles.py index 39a2968..ccb3432 100644 --- a/pym/portage/_sets/profiles.py +++ b/pym/portage/_sets/profiles.py @@ -1,4 +1,4 @@ -# Copyright 2007 Gentoo Foundation +# Copyright 2007-2014 Gentoo Foundation # Distributed under the terms of the GNU General Public License v2 import logging @@ -14,15 +14,15 @@ __all__ = ["PackagesSystemSet"] class PackagesSystemSet(PackageSet): _operations = ["merge"] - def __init__(self, profile_paths, debug=False): + def __init__(self, profiles, debug=False): super(PackagesSystemSet, self).__init__() - self._profile_paths = profile_paths + self._profiles = profiles self._debug = debug - if profile_paths: - description = self._profile_paths[-1] - if description == "/etc/portage/profile" and \ - len(self._profile_paths) > 1: - description = self._profile_paths[-2] + if profiles: + desc_profile = profiles[-1] + if desc_profile.user_config and len(profiles) > 1: + desc_profile = profiles[-2] + description = desc_profile.location else: description = None self.description = "System packages for profile %s" % description @@ -30,10 +30,12 @@ class PackagesSystemSet(PackageSet): def load(self): debug = self._debug if debug: - writemsg_level("\nPackagesSystemSet: profile paths: %s\n" % \ - (self._profile_paths,), level=logging.DEBUG, noiselevel=-1) + writemsg_level("\nPackagesSystemSet: profiles: %s\n" % + (self._profiles,), level=logging.DEBUG, noiselevel=-1) - mylist = [grabfile_package(os.path.join(x, "packages"), verify_eapi=True) for x in self._profile_paths] + mylist = [grabfile_package(os.path.join(x.location, "packages"), + verify_eapi=True, eapi=x.eapi, eapi_default=None) + for x in self._profiles] if debug: writemsg_level("\nPackagesSystemSet: raw packages: %s\n" % \ @@ -49,5 +51,6 @@ class PackagesSystemSet(PackageSet): def singleBuilder(self, options, settings, trees): debug = get_boolean(options, "debug", False) - return PackagesSystemSet(settings.profiles, debug=debug) + return PackagesSystemSet( + settings._locations_manager.profiles_complex, debug=debug) singleBuilder = classmethod(singleBuilder) diff --git a/pym/portage/package/ebuild/_config/KeywordsManager.py b/pym/portage/package/ebuild/_config/KeywordsManager.py index af606f1..e1a8e2b 100644 --- a/pym/portage/package/ebuild/_config/KeywordsManager.py +++ b/pym/portage/package/ebuild/_config/KeywordsManager.py @@ -1,4 +1,4 @@ -# Copyright 2010-2012 Gentoo Foundation +# Copyright 2010-2014 Gentoo Foundation # Distributed under the terms of the GNU General Public License v2 __all__ = ( @@ -22,7 +22,7 @@ class KeywordsManager(object): rawpkeywords = [grabdict_package( os.path.join(x.location, "package.keywords"), recursive=x.portage1_directories, - verify_eapi=True) \ + verify_eapi=True, eapi=x.eapi, eapi_default=None) for x in profiles] for pkeyworddict in rawpkeywords: if not pkeyworddict: @@ -38,7 +38,7 @@ class KeywordsManager(object): raw_p_accept_keywords = [grabdict_package( os.path.join(x.location, "package.accept_keywords"), recursive=x.portage1_directories, - verify_eapi=True) \ + verify_eapi=True, eapi=x.eapi, eapi_default=None) for x in profiles] for d in raw_p_accept_keywords: if not d: diff --git a/pym/portage/package/ebuild/_config/LocationsManager.py b/pym/portage/package/ebuild/_config/LocationsManager.py index 6641092..1fc7e45 100644 --- a/pym/portage/package/ebuild/_config/LocationsManager.py +++ b/pym/portage/package/ebuild/_config/LocationsManager.py @@ -19,7 +19,7 @@ from portage.eapi import eapi_allows_directories_on_profile_level_and_repository from portage.exception import DirectoryNotFound, ParseError from portage.localization import _ from portage.util import ensure_dirs, grabfile, \ - normalize_path, shlex_split, writemsg + normalize_path, read_corresponding_eapi_file, shlex_split, writemsg from portage.util._path import exists_raise_eaccess, isdir_raise_eaccess from portage.repository.config import parse_layout_conf, \ _portage1_profiles_allow_directories @@ -31,7 +31,7 @@ _PORTAGE1_DIRECTORIES = frozenset([ 'use.mask', 'use.force']) _profile_node = collections.namedtuple('_profile_node', - 'location portage1_directories user_config profile_formats') + 'location portage1_directories user_config profile_formats eapi') _allow_parent_colon = frozenset( ["portage-2"]) @@ -129,11 +129,16 @@ class LocationsManager(object): custom_prof = os.path.join( self.config_root, CUSTOM_PROFILE_PATH) if os.path.exists(custom_prof): + # For read_corresponding_eapi_file, specify default=None + # in order to allow things like wildcard atoms when + # is no explicit EAPI setting. self.user_profile_dir = custom_prof self.profiles.append(custom_prof) self.profiles_complex.append( _profile_node(custom_prof, True, True, - ('profile-bashrcs', 'profile-set'))) + ('profile-bashrcs', 'profile-set'), + read_corresponding_eapi_file( + custom_prof + os.sep, default=None))) del custom_prof self.profiles = tuple(self.profiles) @@ -153,9 +158,21 @@ class LocationsManager(object): repo_loc = None compat_mode = False current_formats = () + eapi = None + + intersecting_repos = [x for x in known_repos + if current_abs_path.startswith(x[0])] + if intersecting_repos: + # Handle nested repositories. The longest path + # will be the correct one. + repo_loc, layout_data = max(intersecting_repos, + key=lambda x:len(x[0])) + eapi = [x for x in layout_data['profile-formats'] + if x.startswith("eapi-")] + eapi = eapi and eapi[0][len("eapi-"):] eapi_file = os.path.join(currentPath, "eapi") - eapi = "0" + eapi = eapi or "0" f = None try: f = io.open(_unicode_encode(eapi_file, @@ -174,11 +191,7 @@ class LocationsManager(object): if f is not None: f.close() - intersecting_repos = [x for x in known_repos if current_abs_path.startswith(x[0])] if intersecting_repos: - # protect against nested repositories. Insane configuration, but the longest - # path will be the correct one. - repo_loc, layout_data = max(intersecting_repos, key=lambda x:len(x[0])) allow_directories = eapi_allows_directories_on_profile_level_and_repository_level(eapi) or \ any(x in _portage1_profiles_allow_directories for x in layout_data['profile-formats']) compat_mode = not eapi_allows_directories_on_profile_level_and_repository_level(eapi) and \ @@ -238,7 +251,7 @@ class LocationsManager(object): self.profiles.append(currentPath) self.profiles_complex.append( _profile_node(currentPath, allow_directories, False, - current_formats)) + current_formats, eapi)) def _expand_parent_colon(self, parentsFile, parentPath, repo_loc, repositories): diff --git a/pym/portage/package/ebuild/_config/MaskManager.py b/pym/portage/package/ebuild/_config/MaskManager.py index 0f060c9..55c8c7a 100644 --- a/pym/portage/package/ebuild/_config/MaskManager.py +++ b/pym/portage/package/ebuild/_config/MaskManager.py @@ -39,7 +39,8 @@ class MaskManager(object): path = os.path.join(loc, 'profiles', 'package.mask') pmask_cache[loc] = grabfile_package(path, recursive=repo_config.portage1_profiles, - remember_source_file=True, verify_eapi=True) + remember_source_file=True, verify_eapi=True, + eapi_default=repo_config.eapi) if repo_config.portage1_profiles_compat and os.path.isdir(path): warnings.warn(_("Repository '%(repo_name)s' is implicitly using " "'portage-1' profile format in its profiles/package.mask, but " @@ -105,7 +106,8 @@ class MaskManager(object): if not repo.portage1_profiles: continue repo_lines = grabfile_package(os.path.join(repo.location, "profiles", "package.unmask"), \ - recursive=1, remember_source_file=True, verify_eapi=True) + recursive=1, remember_source_file=True, + verify_eapi=True, eapi_default=repo.eapi) lines = stack_lists([repo_lines], incremental=1, \ remember_source_file=True, warn_for_unmatched_removal=True, strict_warn_for_unmatched_removal=strict_umatched_removal) @@ -119,12 +121,14 @@ class MaskManager(object): profile_pkgmasklines.append(grabfile_package( os.path.join(x.location, "package.mask"), recursive=x.portage1_directories, - remember_source_file=True, verify_eapi=True)) + remember_source_file=True, verify_eapi=True, + eapi=x.eapi, eapi_default=None)) if x.portage1_directories: profile_pkgunmasklines.append(grabfile_package( os.path.join(x.location, "package.unmask"), recursive=x.portage1_directories, - remember_source_file=True, verify_eapi=True)) + remember_source_file=True, verify_eapi=True, + eapi=x.eapi, eapi_default=None)) profile_pkgmasklines = stack_lists(profile_pkgmasklines, incremental=1, \ remember_source_file=True, warn_for_unmatched_removal=True, strict_warn_for_unmatched_removal=strict_umatched_removal) diff --git a/pym/portage/package/ebuild/_config/UseManager.py b/pym/portage/package/ebuild/_config/UseManager.py index 1c8c60e..f869908 100644 --- a/pym/portage/package/ebuild/_config/UseManager.py +++ b/pym/portage/package/ebuild/_config/UseManager.py @@ -107,10 +107,32 @@ class UseManager(object): self.repositories = repositories - def _parse_file_to_tuple(self, file_name, recursive=True, eapi_filter=None): + def _parse_file_to_tuple(self, file_name, recursive=True, + eapi_filter=None, eapi=None, eapi_default="0"): + """ + @param file_name: input file name + @type file_name: str + @param recursive: triggers recursion if the input file is a + directory + @type recursive: bool + @param eapi_filter: a function that accepts a single eapi + argument, and returns true if the the current file type + is supported by the given EAPI + @type eapi_filter: callable + @param eapi: the EAPI of the current profile node, which allows + a call to read_corresponding_eapi_file to be skipped + @type eapi: str + @param eapi_default: the default EAPI which applies if the + current profile node does not define a local EAPI + @type eapi_default: str + @rtype: tuple + @return: collection of use flags + """ ret = [] lines = grabfile(file_name, recursive=recursive) - eapi = read_corresponding_eapi_file(file_name) + if eapi is None: + eapi = read_corresponding_eapi_file( + file_name, default=eapi_default) if eapi_filter is not None and not eapi_filter(eapi): if lines: writemsg(_("--- EAPI '%s' does not support '%s': '%s'\n") % @@ -131,19 +153,46 @@ class UseManager(object): return tuple(ret) def _parse_file_to_dict(self, file_name, juststrings=False, recursive=True, - eapi_filter=None, user_config=False): + eapi_filter=None, user_config=False, eapi=None, eapi_default="0"): + """ + @param file_name: input file name + @type file_name: str + @param juststrings: store dict values as space-delimited strings + instead of tuples + @type juststrings: bool + @param recursive: triggers recursion if the input file is a + directory + @type recursive: bool + @param eapi_filter: a function that accepts a single eapi + argument, and returns true if the the current file type + is supported by the given EAPI + @type eapi_filter: callable + @param user_config: current file is part of the local + configuration (not repository content) + @type user_config: bool + @param eapi: the EAPI of the current profile node, which allows + a call to read_corresponding_eapi_file to be skipped + @type eapi: str + @param eapi_default: the default EAPI which applies if the + current profile node does not define a local EAPI + @type eapi_default: str + @rtype: tuple + @return: collection of use flags + """ ret = {} location_dict = {} - eapi = read_corresponding_eapi_file(file_name, default=None) - if eapi is None and not user_config: - eapi = "0" if eapi is None: + eapi = read_corresponding_eapi_file(file_name, + default=eapi_default) + extended_syntax = eapi is None and user_config + if extended_syntax: ret = ExtendedAtomDict(dict) else: ret = {} file_dict = grabdict_package(file_name, recursive=recursive, - allow_wildcard=(eapi is None), allow_repo=(eapi is None), - verify_eapi=(eapi is not None)) + allow_wildcard=extended_syntax, allow_repo=extended_syntax, + verify_eapi=(not extended_syntax), eapi=eapi, + eapi_default=eapi_default) if eapi is not None and eapi_filter is not None and not eapi_filter(eapi): if file_dict: writemsg(_("--- EAPI '%s' does not support '%s': '%s'\n") % @@ -185,35 +234,41 @@ class UseManager(object): def _parse_repository_files_to_dict_of_tuples(self, file_name, repositories, eapi_filter=None): ret = {} for repo in repositories.repos_with_profiles(): - ret[repo.name] = self._parse_file_to_tuple(os.path.join(repo.location, "profiles", file_name), eapi_filter=eapi_filter) + ret[repo.name] = self._parse_file_to_tuple( + os.path.join(repo.location, "profiles", file_name), + eapi_filter=eapi_filter, eapi_default=repo.eapi) return ret def _parse_repository_files_to_dict_of_dicts(self, file_name, repositories, eapi_filter=None): ret = {} for repo in repositories.repos_with_profiles(): - ret[repo.name] = self._parse_file_to_dict(os.path.join(repo.location, "profiles", file_name), eapi_filter=eapi_filter) + ret[repo.name] = self._parse_file_to_dict( + os.path.join(repo.location, "profiles", file_name), + eapi_filter=eapi_filter, eapi_default=repo.eapi) return ret def _parse_profile_files_to_tuple_of_tuples(self, file_name, locations, eapi_filter=None): return tuple(self._parse_file_to_tuple( os.path.join(profile.location, file_name), - recursive=profile.portage1_directories, eapi_filter=eapi_filter) - for profile in locations) + recursive=profile.portage1_directories, + eapi_filter=eapi_filter, eapi=profile.eapi, + eapi_default=None) for profile in locations) def _parse_profile_files_to_tuple_of_dicts(self, file_name, locations, juststrings=False, eapi_filter=None): return tuple(self._parse_file_to_dict( os.path.join(profile.location, file_name), juststrings, recursive=profile.portage1_directories, eapi_filter=eapi_filter, - user_config=profile.user_config) - for profile in locations) + user_config=profile.user_config, eapi=profile.eapi, + eapi_default=None) for profile in locations) def _parse_repository_usealiases(self, repositories): ret = {} for repo in repositories.repos_with_profiles(): file_name = os.path.join(repo.location, "profiles", "use.aliases") - eapi = read_corresponding_eapi_file(file_name) + eapi = read_corresponding_eapi_file( + file_name, default=repo.eapi) useflag_re = _get_useflag_re(eapi) raw_file_dict = grabdict(file_name, recursive=True) file_dict = {} @@ -238,7 +293,8 @@ class UseManager(object): ret = {} for repo in repositories.repos_with_profiles(): file_name = os.path.join(repo.location, "profiles", "package.use.aliases") - eapi = read_corresponding_eapi_file(file_name) + eapi = read_corresponding_eapi_file( + file_name, default=repo.eapi) useflag_re = _get_useflag_re(eapi) lines = grabfile(file_name, recursive=True) file_dict = {} diff --git a/pym/portage/package/ebuild/config.py b/pym/portage/package/ebuild/config.py index 65de93e..b7dd9ea 100644 --- a/pym/portage/package/ebuild/config.py +++ b/pym/portage/package/ebuild/config.py @@ -564,8 +564,10 @@ class config(object): self.user_profile_dir = locations_manager.user_profile_dir try: - packages_list = [grabfile_package(os.path.join(x, "packages"), - verify_eapi=True) for x in self.profiles] + packages_list = [grabfile_package( + os.path.join(x.location, "packages"), + verify_eapi=True, eapi=x.eapi, eapi_default=None) + for x in profiles_complex] except IOError as e: if e.errno == IsADirectory.errno: raise IsADirectory(os.path.join(self.profile_path, @@ -758,7 +760,8 @@ class config(object): portage.dep.ExtendedAtomDict(dict) bashrc = grabdict_package(os.path.join(profile.location, "package.bashrc"), recursive=1, allow_wildcard=True, - allow_repo=True, verify_eapi=False) + allow_repo=True, verify_eapi=True, + eapi=profile.eapi, eapi_default=None) if not bashrc: continue diff --git a/pym/portage/repository/config.py b/pym/portage/repository/config.py index 9096d73..abc8756 100644 --- a/pym/portage/repository/config.py +++ b/pym/portage/repository/config.py @@ -190,11 +190,9 @@ class RepoConfig(object): location = None self.location = location - eapi = None missing = True self.name = name if self.location is not None: - eapi = read_corresponding_eapi_file(os.path.join(self.location, REPO_NAME_LOC)) self.name, missing = self._read_valid_repo_name(self.location) if missing: # The name from repos.conf has to be used here for @@ -208,7 +206,7 @@ class RepoConfig(object): elif name == "DEFAULT": missing = False - self.eapi = eapi + self.eapi = None self.missing_repo_name = missing # sign_commit is disabled by default, since it requires Git >=1.7.9, # and key_id configured by `git config user.signingkey key_id` @@ -258,10 +256,26 @@ class RepoConfig(object): 'sign-commit', 'sign-manifest', 'thin-manifest', 'update-changelog'): setattr(self, value.lower().replace("-", "_"), layout_data[value]) - self.portage1_profiles = eapi_allows_directories_on_profile_level_and_repository_level(eapi) or \ - any(x in _portage1_profiles_allow_directories for x in layout_data['profile-formats']) - self.portage1_profiles_compat = not eapi_allows_directories_on_profile_level_and_repository_level(eapi) and \ - layout_data['profile-formats'] == ('portage-1-compat',) + # If profile-formats specifies a default EAPI, then set + # self.eapi to that, otherwise set it to "0" as specified + # by PMS. + eapi = [x for x in layout_data['profile-formats'] + if x.startswith("eapi-")] + eapi = eapi and eapi[0][len("eapi-"):] + self.eapi = eapi or "0" + + profile_root_eapi = read_corresponding_eapi_file( + os.path.join(self.location, REPO_NAME_LOC)) + + self.portage1_profiles = ((eapi is not None and + eapi_allows_directories_on_profile_level_and_repository_level(eapi)) or + eapi_allows_directories_on_profile_level_and_repository_level(profile_root_eapi) or + any(x in _portage1_profiles_allow_directories for x in layout_data['profile-formats'])) + self.portage1_profiles_compat = ( + layout_data['profile-formats'] == ('portage-1-compat',) and + not (eapi is not None and + eapi_allows_directories_on_profile_level_and_repository_level(eapi)) and + not eapi_allows_directories_on_profile_level_and_repository_level(profile_root_eapi)) self._eapis_banned = frozenset(layout_data['eapis-banned']) self._eapis_deprecated = frozenset(layout_data['eapis-deprecated']) @@ -1073,6 +1087,14 @@ def parse_layout_conf(repo_location, repo_name=None): else: raw_formats = set(raw_formats.split()) unknown = raw_formats.difference(_valid_profile_formats) + eapi_formats = tuple(x for x in unknown + if x.startswith("eapi-") and + portage.eapi_is_supported(x[len("eapi-"):])) + raw_formats.intersection_update(_valid_profile_formats) + raw_formats.update(eapi_formats) + raw_formats = tuple(raw_formats) + unknown.difference_update(eapi_formats) + if unknown: repo_name = _get_repo_name(repo_location, cached=repo_name) warnings.warn((_("Repository named '%(repo_name)s' has unsupported " @@ -1082,7 +1104,16 @@ def parse_layout_conf(repo_location, repo_name=None): layout_filename=layout_filename, unknown_fmts=" ".join(unknown))), DeprecationWarning) - raw_formats = tuple(raw_formats.intersection(_valid_profile_formats)) + + if len(eapi_formats) > 1: + warnings.warn((_("Repository named '%(repo_name)s' has " + "multiple default eapi settings ('profile-formats = " + "%(eapi_fmts)s' setting in '%(layout_filename)s") % + dict(repo_name=repo_name or 'unspecified', + layout_filename=layout_filename, + eapi_fmts=" ".join(eapi_formats))), + SyntaxWarning) + data['profile-formats'] = raw_formats return data, layout_errors diff --git a/pym/portage/tests/resolver/test_profile_default_eapi.py b/pym/portage/tests/resolver/test_profile_default_eapi.py new file mode 100644 index 0000000..4b4b9d9 --- /dev/null +++ b/pym/portage/tests/resolver/test_profile_default_eapi.py @@ -0,0 +1,123 @@ +# Copyright 2014 Gentoo Foundation +# Distributed under the terms of the GNU General Public License v2 + +from __future__ import unicode_literals + +import io + +from portage import os, _encodings +from portage.const import USER_CONFIG_PATH +from portage.tests import TestCase +from portage.tests.resolver.ResolverPlayground import ResolverPlayground +from portage.dep import ExtendedAtomDict +from portage.util import ensure_dirs + +class ProfileDefaultEAPITestCase(TestCase): + + def testProfileDefaultEAPI(self): + + repo_configs = { + "test_repo": { + "layout.conf": ("profile-formats = eapi-5",), + } + } + + profiles = ( + ( + "", + { + "package.mask": ("sys-libs/A:1",), + "package.use": ("sys-libs/A:1 flag",) + } + ), + ( + "default/linux", + { + "package.mask": ("sys-libs/B:1",), + "package.use": ("sys-libs/B:1 flag",), + "package.keywords": ("sys-libs/B:1 x86",) + } + ), + ( + "default/linux/x86", + { + "package.mask": ("sys-libs/C:1",), + "package.use": ("sys-libs/C:1 flag",), + "package.keywords": ("sys-libs/C:1 x86",), + "parent": ("..",) + } + ), + ) + + user_profile = { + "package.mask": ("sys-libs/D:1",), + "package.use": ("sys-libs/D:1 flag",), + "package.keywords": ("sys-libs/D:1 x86",), + } + + test_cases = ( + (lambda x: x._mask_manager._pmaskdict, { + "sys-libs/A": ("sys-libs/A:1::test_repo",), + "sys-libs/B": ("sys-libs/B:1",), + "sys-libs/C": ("sys-libs/C:1",), + "sys-libs/D": ("sys-libs/D:1",), + }), + (lambda x: x._use_manager._repo_puse_dict, { + "test_repo": { + "sys-libs/A": { + "sys-libs/A:1": ("flag",) + } + } + }), + (lambda x: x._use_manager._pkgprofileuse, ( + {"sys-libs/B": {"sys-libs/B:1": "flag"}}, + {"sys-libs/C": {"sys-libs/C:1": "flag"}}, + {}, + {"sys-libs/D": {"sys-libs/D:1": "flag"}}, + )), + (lambda x: x._keywords_manager._pkeywords_list, ( + {"sys-libs/B": {"sys-libs/B:1": ["x86"]}}, + {"sys-libs/C": {"sys-libs/C:1": ["x86"]}}, + {"sys-libs/D": {"sys-libs/D:1": ["x86"]}}, + ) + ) + ) + + playground = ResolverPlayground(debug=False, + repo_configs=repo_configs) + try: + repo_dir = (playground.settings.repositories. + get_location_for_name("test_repo")) + profile_root = os.path.join(repo_dir, "profiles") + profile_info = [(os.path.join(profile_root, p), data) + for p, data in profiles] + profile_info.append((os.path.join(playground.eroot, + USER_CONFIG_PATH, "profile"), user_profile)) + + for prof_path, data in profile_info: + ensure_dirs(prof_path) + for k, v in data.items(): + with io.open(os.path.join(prof_path, k), mode="w", + encoding=_encodings["repo.content"]) as f: + for line in v: + f.write("%s\n" % line) + + # The config must be reloaded in order to account + # for the above profile customizations. + playground.reload_config() + + for fn, expected in test_cases: + result = self._translate_result(fn(playground.settings)) + self.assertEqual(result, expected) + + finally: + playground.cleanup() + + + @staticmethod + def _translate_result(result): + if isinstance(result, ExtendedAtomDict): + result = dict(result.items()) + elif isinstance(result, tuple): + result = tuple(dict(x.items()) for x in result) + return result diff --git a/pym/portage/util/__init__.py b/pym/portage/util/__init__.py index d0cca5b..61fe787 100644 --- a/pym/portage/util/__init__.py +++ b/pym/portage/util/__init__.py @@ -425,7 +425,7 @@ def read_corresponding_eapi_file(filename, default="0"): return eapi def grabdict_package(myfilename, juststrings=0, recursive=0, allow_wildcard=False, allow_repo=False, - verify_eapi=False, eapi=None): + verify_eapi=False, eapi=None, eapi_default="0"): """ Does the same thing as grabdict except it validates keys with isvalidatom()""" @@ -441,7 +441,8 @@ def grabdict_package(myfilename, juststrings=0, recursive=0, allow_wildcard=Fals if not d: continue if verify_eapi and eapi is None: - eapi = read_corresponding_eapi_file(myfilename) + eapi = read_corresponding_eapi_file( + myfilename, default=eapi_default) for k, v in d.items(): try: @@ -460,13 +461,15 @@ def grabdict_package(myfilename, juststrings=0, recursive=0, allow_wildcard=Fals return atoms def grabfile_package(myfilename, compatlevel=0, recursive=0, allow_wildcard=False, allow_repo=False, - remember_source_file=False, verify_eapi=False, eapi=None): + remember_source_file=False, verify_eapi=False, eapi=None, + eapi_default="0"): pkgs=grabfile(myfilename, compatlevel, recursive=recursive, remember_source_file=True) if not pkgs: return pkgs if verify_eapi and eapi is None: - eapi = read_corresponding_eapi_file(myfilename) + eapi = read_corresponding_eapi_file( + myfilename, default=eapi_default) mybasename = os.path.basename(myfilename) atoms = [] for pkg, source_file in pkgs: -- 2.0.4