From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from lists.gentoo.org (pigeon.gentoo.org [208.92.234.80]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by finch.gentoo.org (Postfix) with ESMTPS id 8A37D138337 for ; Sun, 23 Sep 2018 01:31:06 +0000 (UTC) Received: from pigeon.gentoo.org (localhost [127.0.0.1]) by pigeon.gentoo.org (Postfix) with SMTP id 40A19E083D; Sun, 23 Sep 2018 01:31:05 +0000 (UTC) Received: from smtp.gentoo.org (dev.gentoo.org [IPv6:2001:470:ea4a:1:5054:ff:fec7:86e4]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by pigeon.gentoo.org (Postfix) with ESMTPS id 03A37E083D for ; Sun, 23 Sep 2018 01:31:05 +0000 (UTC) Received: from oystercatcher.gentoo.org (oystercatcher.gentoo.org [148.251.78.52]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.gentoo.org (Postfix) with ESMTPS id 5F049335CEE for ; Sun, 23 Sep 2018 01:31:03 +0000 (UTC) Received: from localhost.localdomain (localhost [IPv6:::1]) by oystercatcher.gentoo.org (Postfix) with ESMTP id 9F4F13E2 for ; Sun, 23 Sep 2018 01:31:01 +0000 (UTC) From: "Thomas Deutschmann" To: gentoo-commits@lists.gentoo.org Content-Transfer-Encoding: 8bit Content-type: text/plain; charset=UTF-8 Reply-To: gentoo-dev@lists.gentoo.org, "Thomas Deutschmann" Message-ID: <1537666235.59c9e92b639712d4c37ece540582db83afdc8608.whissi@gentoo> Subject: [gentoo-commits] repo/gentoo:master commit in: www-client/firefox/files/, www-client/firefox/ X-VCS-Repository: repo/gentoo X-VCS-Files: www-client/firefox/files/firefox-60.0-do-not-force-lld.patch www-client/firefox/files/firefox-60.0-sandbox-lto.patch www-client/firefox/firefox-62.0.2.ebuild www-client/firefox/metadata.xml X-VCS-Directories: www-client/firefox/ www-client/firefox/files/ X-VCS-Committer: whissi X-VCS-Committer-Name: Thomas Deutschmann X-VCS-Revision: 59c9e92b639712d4c37ece540582db83afdc8608 X-VCS-Branch: master Date: Sun, 23 Sep 2018 01:31:01 +0000 (UTC) Precedence: bulk List-Post: List-Help: List-Unsubscribe: List-Subscribe: List-Id: Gentoo Linux mail X-BeenThere: gentoo-commits@lists.gentoo.org X-Archives-Salt: ceab3f7e-d73c-4cb5-aaef-c5a847840a62 X-Archives-Hash: bd05376bcb5c85eccc9aa883ef518086 commit: 59c9e92b639712d4c37ece540582db83afdc8608 Author: Thomas Deutschmann gentoo org> AuthorDate: Sun Sep 23 00:51:42 2018 +0000 Commit: Thomas Deutschmann gentoo org> CommitDate: Sun Sep 23 01:30:35 2018 +0000 URL: https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=59c9e92b www-client/firefox: add explicit Clang and LTO support - Set USE=lto to enable Link Time Optimization (LTO). It works with either GCC or Clang. - When you want to use Clang to build Firefox you now have to set USE=clang. If not set, GCC will be used. - Using Clang will require lld linker provided by sys-devel/lld. Closes: https://bugs.gentoo.org/666580 Closes: https://bugs.gentoo.org/663846 Package-Manager: Portage-2.3.49, Repoman-2.3.10 Signed-off-by: Thomas Deutschmann gentoo.org> .../files/firefox-60.0-do-not-force-lld.patch | 262 +++++++++++++++++++++ .../firefox/files/firefox-60.0-sandbox-lto.patch | 13 + www-client/firefox/firefox-62.0.2.ebuild | 53 ++++- www-client/firefox/metadata.xml | 3 + 4 files changed, 323 insertions(+), 8 deletions(-) diff --git a/www-client/firefox/files/firefox-60.0-do-not-force-lld.patch b/www-client/firefox/files/firefox-60.0-do-not-force-lld.patch new file mode 100644 index 00000000000..a8325bc745b --- /dev/null +++ b/www-client/firefox/files/firefox-60.0-do-not-force-lld.patch @@ -0,0 +1,262 @@ +https://hg.mozilla.org/mozilla-central/rev/e8c173a632a4 +https://hg.mozilla.org/mozilla-central/rev/dbeb248015cc +https://hg.mozilla.org/mozilla-central/rev/de7abe2c2b3e + +--- a/build/moz.configure/toolchain.configure ++++ b/build/moz.configure/toolchain.configure +@@ -1331,10 +1331,6 @@ def lto(value, c_compiler): + + # clang and clang-cl + if c_compiler.type in ('clang', 'clang-cl'): +- # Until Bug 1457168 is fixed, we have to hardcode -fuse-ld=lld here +- if c_compiler.type == 'clang': +- flags.append("-fuse-ld=lld") +- + if len(value) and value[0].lower() == 'full': + flags.append("-flto") + ldflags.append("-flto") +@@ -1362,8 +1358,6 @@ add_old_configure_assignment('MOZ_LTO', lto.enabled) + add_old_configure_assignment('MOZ_LTO_CFLAGS', lto.flags) + add_old_configure_assignment('MOZ_LTO_LDFLAGS', lto.ldflags) + +-imply_option('--enable-linker', 'lld', when='--enable-lto') +- + # ASAN + # ============================================================== + +@@ -1566,7 +1560,7 @@ set_config('CARGO_INCREMENTAL', cargo_incremental) + + @depends(target) + def is_linker_option_enabled(target): +- if target.kernel not in ('Darwin', 'WINNT', 'SunOS'): ++ if target.kernel not in ('WINNT', 'SunOS'): + return True + + +@@ -1578,21 +1572,40 @@ option('--enable-gold', + imply_option('--enable-linker', 'gold', when='--enable-gold') + + js_option('--enable-linker', nargs=1, +- help='Select the linker {bfd, gold, lld, lld-*}', ++ help='Select the linker {bfd, gold, ld64, lld, lld-*}', + when=is_linker_option_enabled) + + + @depends('--enable-linker', c_compiler, developer_options, '--enable-gold', +- extra_toolchain_flags, when=is_linker_option_enabled) ++ extra_toolchain_flags, target, lto.enabled, ++ when=is_linker_option_enabled) + @checking('for linker', lambda x: x.KIND) + @imports('os') + @imports('shutil') + def select_linker(linker, c_compiler, developer_options, enable_gold, +- toolchain_flags): +- +- linker = linker[0] if linker else None ++ toolchain_flags, target, lto): ++ ++ if linker: ++ linker = linker[0] ++ elif lto and c_compiler.type == 'clang' and target.kernel != 'Darwin': ++ # If no linker was explicitly given, and building with clang for non-macOS, ++ # prefer lld. For macOS, we prefer ld64, or whatever the default linker is. ++ linker = 'lld' ++ else: ++ linker = None + +- if linker not in ('bfd', 'gold', 'lld', None) and not linker.startswith("lld-"): ++ def is_valid_linker(linker): ++ if target.kernel == 'Darwin': ++ valid_linkers = ('ld64', 'lld') ++ else: ++ valid_linkers = ('bfd', 'gold', 'lld') ++ if linker in valid_linkers: ++ return True ++ if 'lld' in valid_linkers and linker.startswith('lld-'): ++ return True ++ return False ++ ++ if linker and not is_valid_linker(linker): + # Check that we are trying to use a supported linker + die('Unsupported linker ' + linker) + +@@ -1602,13 +1615,34 @@ def select_linker(linker, c_compiler, developer_options, enable_gold, + + def try_linker(linker): + # Generate the compiler flag +- linker_flag = ["-fuse-ld=" + linker] if linker else [] ++ if linker == 'ld64': ++ linker_flag = ['-fuse-ld=ld'] ++ elif linker: ++ linker_flag = ["-fuse-ld=" + linker] ++ else: ++ linker_flag = [] + cmd = cmd_base + linker_flag + version_check + if toolchain_flags: + cmd += toolchain_flags + +- cmd_output = check_cmd_output(*cmd).decode('utf-8') +- if 'GNU ld' in cmd_output: ++ # ld64 doesn't have anything to print out a version. It does print out ++ # "ld64: For information on command line options please use 'man ld'." ++ # but that would require doing two attempts, one with --version, that ++ # would fail, and another with --help. ++ # Instead, abuse its LD_PRINT_OPTIONS feature to detect a message ++ # specific to it on stderr when it fails to process --version. ++ env = dict(os.environ) ++ env['LD_PRINT_OPTIONS'] = '1' ++ retcode, stdout, stderr = get_cmd_output(*cmd, env=env) ++ cmd_output = stdout.decode('utf-8') ++ stderr = stderr.decode('utf-8') ++ if retcode == 1 and 'Logging ld64 options' in stderr: ++ kind = 'ld64' ++ ++ elif retcode != 0: ++ return None ++ ++ elif 'GNU ld' in cmd_output: + # We are using the normal linker + kind = 'bfd' + +@@ -1627,13 +1661,21 @@ def select_linker(linker, c_compiler, developer_options, enable_gold, + ) + + result = try_linker(linker) ++ if result is None: ++ if linker: ++ die("Could not use {} as linker".format(linker)) ++ die("Failed to find a linker") + + if (linker is None and enable_gold.origin == 'default' and + developer_options and result.KIND == 'bfd'): +- gold = try_linker('gold') +- +- if gold.KIND == 'gold': +- result = gold ++ # try and use lld if available. ++ tried = try_linker('lld') ++ if tried is None or tried.KIND != 'lld': ++ tried = try_linker('gold') ++ if tried is None or tried.KIND != 'gold': ++ tried = None ++ if tried: ++ result = tried + + # If an explicit linker was given, error out if what we found is different. + if linker and not linker.startswith(result.KIND): +@@ -1644,7 +1686,7 @@ def select_linker(linker, c_compiler, developer_options, enable_gold, + + set_config('LD_IS_BFD', depends(select_linker.KIND) + (lambda x: x == 'bfd' or None)) +-set_config('LINKER_LDFLAGS', select_linker.LINKER_FLAG) ++add_old_configure_assignment('LINKER_LDFLAGS', select_linker.LINKER_FLAG) + + + js_option('--enable-clang-plugin', env='ENABLE_CLANG_PLUGIN', +--- a/build/moz.configure/util.configure ++++ b/build/moz.configure/util.configure +@@ -19,20 +19,13 @@ def configure_error(message): + their inputs from moz.configure usage.''' + raise ConfigureError(message) + +-# A wrapper to obtain a process' output that returns the output generated +-# by running the given command if it exits normally, and streams that +-# output to log.debug and calls die or the given error callback if it +-# does not. +- + ++# A wrapper to obtain a process' output and return code. ++# Returns a tuple (retcode, stdout, stderr). + @imports(_from='__builtin__', _import='unicode') + @imports('subprocess') +-@imports('sys') +-@imports(_from='mozbuild.configure.util', _import='LineIO') + @imports(_from='mozbuild.shellutil', _import='quote') +-def check_cmd_output(*args, **kwargs): +- onerror = kwargs.pop('onerror', None) +- ++def get_cmd_output(*args, **kwargs): + # subprocess on older Pythons can't handle unicode keys or values in + # environment dicts. Normalize automagically so callers don't have to + # deal with this. +@@ -49,12 +42,24 @@ def check_cmd_output(*args, **kwargs): + + kwargs['env'] = normalized_env + ++ log.debug('Executing: `%s`', quote(*args)) ++ proc = subprocess.Popen(args, stdout=subprocess.PIPE, ++ stderr=subprocess.PIPE, **kwargs) ++ stdout, stderr = proc.communicate() ++ return proc.wait(), stdout, stderr ++ ++ ++# A wrapper to obtain a process' output that returns the output generated ++# by running the given command if it exits normally, and streams that ++# output to log.debug and calls die or the given error callback if it ++# does not. ++@imports(_from='mozbuild.configure.util', _import='LineIO') ++@imports(_from='mozbuild.shellutil', _import='quote') ++def check_cmd_output(*args, **kwargs): ++ onerror = kwargs.pop('onerror', None) ++ + with log.queue_debug(): +- log.debug('Executing: `%s`', quote(*args)) +- proc = subprocess.Popen(args, stdout=subprocess.PIPE, +- stderr=subprocess.PIPE, **kwargs) +- stdout, stderr = proc.communicate() +- retcode = proc.wait() ++ retcode, stdout, stderr = get_cmd_output(*args, **kwargs) + if retcode == 0: + return stdout + +--- a/js/src/old-configure.in ++++ b/js/src/old-configure.in +@@ -63,6 +63,8 @@ dnl ======================================================== + USE_PTHREADS= + _PTHREAD_LDFLAGS="" + ++LDFLAGS="$LDFLAGS $LINKER_LDFLAGS" ++ + MOZ_DEFAULT_COMPILER + + if test -z "$JS_STANDALONE"; then +--- a/old-configure.in ++++ b/old-configure.in +@@ -78,6 +78,8 @@ dnl ======================================================== + MOZ_USE_PTHREADS= + _PTHREAD_LDFLAGS="" + ++LDFLAGS="$LDFLAGS $LINKER_LDFLAGS" ++ + MOZ_DEFAULT_COMPILER + + if test "$COMPILE_ENVIRONMENT"; then +--- a/python/mozbuild/mozbuild/frontend/context.py ++++ b/python/mozbuild/mozbuild/frontend/context.py +@@ -384,8 +384,6 @@ class LinkFlags(BaseCompileFlags): + + self.flag_variables = ( + ('OS', self._os_ldflags(), ('LDFLAGS',)), +- ('LINKER', context.config.substs.get('LINKER_LDFLAGS'), +- ('LDFLAGS',)), + ('DEFFILE', None, ('LDFLAGS',)), + ('MOZBUILD', None, ('LDFLAGS',)), + ('FIX_LINK_PATHS', context.config.substs.get('MOZ_FIX_LINK_PATHS'), +--- a/python/mozbuild/mozbuild/test/frontend/test_emitter.py ++++ b/python/mozbuild/mozbuild/test/frontend/test_emitter.py +@@ -242,7 +242,6 @@ class TestEmitterBasic(unittest.TestCase): + def test_link_flags(self): + reader = self.reader('link-flags', extra_substs={ + 'OS_LDFLAGS': ['-Wl,rpath-link=/usr/lib'], +- 'LINKER_LDFLAGS': ['-fuse-ld=gold'], + 'MOZ_OPTIMIZE': '', + 'MOZ_OPTIMIZE_LDFLAGS': ['-Wl,-dead_strip'], + 'MOZ_DEBUG_LDFLAGS': ['-framework ExceptionHandling'], +@@ -250,7 +249,6 @@ class TestEmitterBasic(unittest.TestCase): + sources, ldflags, lib, compile_flags = self.read_topsrcdir(reader) + self.assertIsInstance(ldflags, ComputedFlags) + self.assertEqual(ldflags.flags['OS'], reader.config.substs['OS_LDFLAGS']) +- self.assertEqual(ldflags.flags['LINKER'], reader.config.substs['LINKER_LDFLAGS']) + self.assertEqual(ldflags.flags['MOZBUILD'], ['-Wl,-U_foo', '-framework Foo', '-x']) + self.assertEqual(ldflags.flags['OPTIMIZE'], []) + diff --git a/www-client/firefox/files/firefox-60.0-sandbox-lto.patch b/www-client/firefox/files/firefox-60.0-sandbox-lto.patch new file mode 100644 index 00000000000..990cb4fbee7 --- /dev/null +++ b/www-client/firefox/files/firefox-60.0-sandbox-lto.patch @@ -0,0 +1,13 @@ +https://bugs.gentoo.org/666580 + +--- a/security/sandbox/linux/moz.build ++++ b/security/sandbox/linux/moz.build +@@ -101,7 +101,7 @@ if CONFIG['CC_TYPE'] in ('clang', 'gcc'): + # forcing there to be only one partition. + for f in CONFIG['OS_CXXFLAGS']: + if f.startswith('-flto') and CONFIG['CC_TYPE'] != 'clang': +- LDFLAGS += ['--param lto-partitions=1'] ++ LDFLAGS += ['--param=lto-partitions=1'] + + DEFINES['NS_NO_XPCOM'] = True + DisableStlWrapping() diff --git a/www-client/firefox/firefox-62.0.2.ebuild b/www-client/firefox/firefox-62.0.2.ebuild index 7e8629aee9c..fa0ad02bc27 100644 --- a/www-client/firefox/firefox-62.0.2.ebuild +++ b/www-client/firefox/firefox-62.0.2.ebuild @@ -40,7 +40,7 @@ KEYWORDS="~amd64 ~x86" SLOT="0" LICENSE="MPL-2.0 GPL-2 LGPL-2.1" -IUSE="bindist dbus debug eme-free +gmp-autoupdate hardened hwaccel jack neon +IUSE="bindist clang dbus debug eme-free +gmp-autoupdate hardened hwaccel jack lto neon pulseaudio +screenshot selinux startup-notification system-harfbuzz system-icu system-jpeg system-libevent system-sqlite system-libvpx test wifi" RESTRICT="!bindist? ( bindist )" @@ -103,8 +103,13 @@ RDEPEND="${CDEPEND} DEPEND="${CDEPEND} app-arch/zip app-arch/unzip - >=sys-devel/binutils-2.16.1 + >=sys-devel/binutils-2.30 sys-apps/findutils + >=sys-devel/llvm-4.0.1 + >=sys-devel/clang-4.0.1 + clang? ( + >=sys-devel/lld-4.0.1 + ) pulseaudio? ( media-sound/pulseaudio ) elibc_glibc? ( virtual/cargo @@ -114,8 +119,6 @@ DEPEND="${CDEPEND} virtual/cargo virtual/rust ) - >=sys-devel/llvm-4.0.1 - >=sys-devel/clang-4.0.1 amd64? ( >=dev-lang/yasm-1.1 virtual/opengl ) x86? ( >=dev-lang/yasm-1.1 virtual/opengl )" @@ -180,6 +183,8 @@ src_prepare() { eapply "${WORKDIR}/firefox" eapply "${FILESDIR}"/${PN}-60.0-blessings-TERM.patch # 654316 + eapply "${FILESDIR}"/${PN}-60.0-do-not-force-lld.patch + eapply "${FILESDIR}"/${PN}-60.0-sandbox-lto.patch # 666580 # Enable gnomebreakpad if use debug ; then @@ -242,6 +247,20 @@ src_configure() { # get your own set of keys. _google_api_key=AIzaSyDEAOvatFo0eTgsV_ZlEzx0ObmepsMzfAc + if use clang && ! tc-is-clang ; then + # Force clang + einfo "Enforcing the use of clang due to USE=clang ..." + CC=${CHOST}-clang + CXX=${CHOST}-clang++ + strip-unsupported-flags + elif ! use clang && ! tc-is-gcc ; then + # Force gcc + einfo "Enforcing the use of gcc due to USE=-clang ..." + CC=${CHOST}-gcc + CXX=${CHOST}-gcc++ + strip-unsupported-flags + fi + #################################### # # mozconfig, CFLAGS and CXXFLAGS setup @@ -260,11 +279,29 @@ src_configure() { # Must pass release in order to properly select linker mozconfig_annotate 'Enable by Gentoo' --enable-release - # Must pass --enable-gold if using ld.gold - if tc-ld-is-gold ; then - mozconfig_annotate 'tc-ld-is-gold=true' --enable-gold + # Don't let user's LTO flags clash with upstream's flags + filter-flags -flto* + + if use lto ; then + if use clang ; then + # Upstream only supports lld when using clang + mozconfig_annotate "forcing ld=lld due to USE=clang and USE=lto" --enable-linker=lld + else + # Linking only works when using ld.gold when LTO is enabled + mozconfig_annotate "forcing ld=gold due to USE=lto" --enable-linker=gold + fi + + mozconfig_annotate '+lto' --enable-lto=full else - mozconfig_annotate 'tc-ld-is-gold=false' --disable-gold + # Avoid auto-magic on linker + if use clang ; then + # This is upstream's default + mozconfig_annotate "forcing ld=lld due to USE=clang" --enable-linker=lld + elif tc-ld-is-gold ; then + mozconfig_annotate "linker is set to gold" --enable-linker=gold + else + mozconfig_annotate "linker is set to bfd" --enable-linker=bfd + fi fi # It doesn't compile on alpha without this LDFLAGS diff --git a/www-client/firefox/metadata.xml b/www-client/firefox/metadata.xml index b0018ba7325..35cdb0ef1ea 100644 --- a/www-client/firefox/metadata.xml +++ b/www-client/firefox/metadata.xml @@ -8,6 +8,7 @@ Disable official Firefox branding (icons, name) which are not binary-redistributable according to upstream. + Use Clang compiler instead of GCC Build with user-specified compiler optimizations (-Os, -O0, -O1, -O2, -O3) from CFLAGS (unsupported) Disable EME (DRM plugin) cabability at build time @@ -16,6 +17,8 @@ downloaded and kept up-to-date in user profiles Force-enable hardware-accelerated rendering (Mozilla bug 594876) Enable or disable jemalloc + Enable Link Time Optimization (LTO). Requires Gold linker when using GCC + or LDD linker when using Clang Add support for profile-guided optimization using gcc-4.5, for faster binaries. This option will double the compile time. Enable support for using rust compiler (experimental)