From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from lists.gentoo.org (pigeon.gentoo.org [208.92.234.80]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by finch.gentoo.org (Postfix) with ESMTPS id B979D1382C5 for ; Sun, 9 May 2021 15:22:38 +0000 (UTC) Received: from pigeon.gentoo.org (localhost [127.0.0.1]) by pigeon.gentoo.org (Postfix) with SMTP id D923BE0391; Sun, 9 May 2021 15:22:35 +0000 (UTC) Received: from smtp.gentoo.org (smtp.gentoo.org [140.211.166.183]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by pigeon.gentoo.org (Postfix) with ESMTPS id 4BF28E01E4 for ; Sun, 9 May 2021 15:22:35 +0000 (UTC) Message-ID: <0dd707925e0947632b430de6bd30e9ad700fc84c.camel@gentoo.org> Subject: Re: [gentoo-project] call for agenda items -- council meeting 2021-05-09 From: David Seifert To: gentoo-project@lists.gentoo.org Date: Sun, 09 May 2021 17:22:29 +0200 In-Reply-To: References: <5517143.1IzOArtZ34@pinacolada> Content-Type: text/plain; charset="UTF-8" User-Agent: Evolution 3.40.1 Precedence: bulk List-Post: List-Help: List-Unsubscribe: List-Subscribe: List-Id: Gentoo Project discussion list X-BeenThere: gentoo-project@lists.gentoo.org Reply-To: gentoo-project@lists.gentoo.org X-Auto-Response-Suppress: DR, RN, NRN, OOF, AutoReply MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Archives-Salt: 867567e9-cb22-4242-91af-e52b07f5c307 X-Archives-Hash: 0c881f7a9aa4101e3f2edf672dbd8cd8 On Wed, 2021-04-28 at 01:56 +0100, Sam James wrote: > > > > On 27 Apr 2021, at 19:56, Andreas K. Huettel > > wrote: > > > > I'd like to kick off a discussion whether LTO should be considered > > "supported". With that I essentially mean that bugs involving LTO > > should be considered valid, and fixes (be it only stripping -flto > > from flags, or similar solutions) should be committed to the tree. > > > > Forgive me for giving a tiny bit of unstructured opinion about > USE=lto, before I dive into the actual proposal: > > 1) I’m really happy to use LTO whenever it is supported upstream (just > like -O3, etc) but I don't use it out of thin air. > > 2) For that reason, I personally like it when USE=lto exists even when > no specific build system hacks are required (because it tells me > “upstream will help with bugs” and so on) but I completely understand > this is >   a bit at odds with what we usually do, and therefore is something I > just need to get used to not having. > > Of course, this problem goes away if we’re going to generally > encourage tinderboxes and general LTO usage, just like we did with as- > needed. > > > I would like to clarify this before possibly suggesting an > > initiative to make the Gentoo repository LTO-safe (similar to what > > we did years ago with --as-needed). > > I’d be interested in if slyfox or Soap had any input on heuristics to > help determine if something is likely to be unsafe. LTO is really good > at “provoking” undefined behaviour. Build completing means very little > in terms of success here. > > I don’t really want to go around running UBSAN on everything to know > it’s safe to use it. Polynomial-C mentioned data corruption at runtime > with some packages in #gentoo-dev the other day too (this kind of > experience is very real and we need to mitigate it). > > Obviously that would be a good candidate for stripping out LTO, but > how are we supposed to notice this stuff if it only happens under > certain circumstances? > > My rough plan would be: > - Coordinate via e.g. wiki pages (and IRC as usual) > - *Strong* focus on packages with test suites so that we can get some > idea of whether it’s working correctly with LTO. Let’s ignore those > without tests in the first round(s). > - Provide some rough documentation for developers on how to build with > UBSAN which we can use at least for critical applications > - For codebases which are known to be “rough” (and we would include > feedback from the LTO overlay [0] here), we’d possibly filter LTO > flags proactively (at least if they’re critical packages). > > > > > Background is, just about every binary distribution out there builds > > with LTO by default now. It's not so great if we then keep telling > > people "LTO is dangerous". > > Right. Fedora are doing this and Clear Linux has been doing this for a > very long time too. What I find interesting is that I’ve never > actually come > across any patches in either to fix LTO issues, which either means I’m > (un)lucky or they’re not hitting issues so often? > > Obviously, we end up hitting more than other people because of often > exotic configurations on the user side, but it is what it is. > > This is one of those situations where reaching out to some folks we > know in other distros for some (unstructured) thoughts might not be a > bad idea - just > to find out some e.g. heads up on problematic codebases. > > Best, > sam > > [0] https://github.com/InBetweenNames/gentooLTO > > > > > Cheers, > > Andreas > > > > (Yes I'm aware of the LTO overlay. It may be a great source.) > > > > -- > > Andreas K. Hüttel > > dilfridge@gentoo.org > > Gentoo Linux developer > > (council, toolchain, base-system, perl, libreoffice) > My corollary from this discussion is: if we decide to support LTO, why not also -O3? Fundamentally, the issue with LTO vs as-needed is that the former triggers failures at runtime, whereas the latter always manifests at link-time. Fixing LTO in general is much more involved, requires more knowledge of C/C++, and requires knowledge of sanitizer instrumentation and what not. Finally, it might expose a number of insidious security vulnerabilities. The activation energy for getting this working in most of the tree will be an order of magnitude greater than for as-needed, and people should keep this in mind. David