From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from lists.gentoo.org (pigeon.gentoo.org [208.92.234.80]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits)) (No client certificate requested) by finch.gentoo.org (Postfix) with ESMTPS id 8C55C158232 for ; Sun, 8 Dec 2024 05:06:00 +0000 (UTC) Received: from pigeon.gentoo.org (localhost [127.0.0.1]) by pigeon.gentoo.org (Postfix) with SMTP id DB724E1247; Sun, 8 Dec 2024 05:05:56 +0000 (UTC) Received: from smtp.gentoo.org (dev.gentoo.org [IPv6:2001:470:ea4a:1:5054:ff:fec7:86e4]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by pigeon.gentoo.org (Postfix) with ESMTPS id 4DEDAE1239 for ; Sun, 8 Dec 2024 05:05:56 +0000 (UTC) From: Sam James To: =?utf-8?B?TWljaGHFgiBHw7Nybnk=?= Cc: gentoo-dev@lists.gentoo.org Subject: [gentoo-dev] Re: LLVM build strategy In-Reply-To: <87a5d6wza1.fsf@gentoo.org> (Sam James's message of "Sun, 08 Dec 2024 04:53:58 +0000") Organization: Gentoo References: <87f27044c599b4168d27d79367fd4b47575502c9.camel@gentoo.org> <87a5d6wza1.fsf@gentoo.org> User-Agent: mu4e 1.12.7; emacs 31.0.50 Date: Sun, 08 Dec 2024 05:05:52 +0000 Message-ID: <871pyiwyq7.fsf@gentoo.org> Precedence: bulk List-Post: List-Help: List-Unsubscribe: List-Subscribe: List-Id: Gentoo Linux mail X-BeenThere: gentoo-dev@lists.gentoo.org Reply-to: gentoo-dev@lists.gentoo.org X-Auto-Response-Suppress: DR, RN, NRN, OOF, AutoReply MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Archives-Salt: b3c57472-30b0-4360-9d9a-c3c32eba43dd X-Archives-Hash: 314608946e5fffa30df80c5d20a94604 Sam James writes: > Micha=C5=82 G=C3=B3rny writes: > >> Hello, >> >> Given that the number of LLVM packages is growing, and probably will >> grow again (I'm introducing "offload" right now, expect at least MLIR >> soon, there are open requests for flang, polly...), I'd like to propose >> creating dedicated categories for these packages and moving them there. >> >> [...] >> >> WDYT? > > I fear this sort of assumes we won't switch to monobuild any time soon. (I'm not even sure I'm advocating a move, it's more that I'd like to flesh out the arguments for it so I have it clear in my mind.) > > I keep thinking [0] about how sustainable our current setup is: > * Fedora moved away from it for >=3D18 [1]. > * As we saw with offload, it broke a few times in just a week. So it's > only Gentoo who cares about this configuration AFAIK. > * It's not working great if we're already not able to easily package > mlir, flang, or polly. > > Violet attempted working on a merge before [2]. > > The trade-offs I see are: > * Increased disk space requirement for building LLVM > > I think that's a legitimate concern although perhaps not that big of a > deal given it is, after all, a whole toolchain. GCC takes quite a bit > to build too. > > * Build time > > Build time for mono LLVM would increase as we're building more > components at least for some users. > > But the time added by > building more components (see below) should be balanced out by ccache if > doing development and also, importantly, not needing to force on all > targets anymore (they keep growing). > > The cumulative time should be the same (although maybe a bit less > given the targets change) given that most people need the > same set of components because of mesa, firefox, or other things which > need libclang. > > * Moving to an upstream supported-configuration > > We may have a bit of pain in moving to it and getting used to it, but > we're no longer swimming upstream and being the only people caring > about our choice of build configuration (and regularly having to > explain and justify it to upstream). > > Upstream also recommend doing a bootstrap build. We don't and can't do > that right now. > > Folks upstream state at every opportunity that they don't care about > standalone builds, > e.g. https://discourse.llvm.org/t/rfc-do-something-with-the-subproject-= tarballs-in-the-release-page/75024/2. > > * Better support for LLVM as a system toolchain > > The current setup doesn't work well for people using LLVM as a system > toolchain (because some of the components *must* be upgraded together), > it doesn't work well for people who want to use mlir/flang/polly, and it > doesn't work well for users on constrained hardware because we have to > force on all targets. It also prohibits more optimisation, PGO, and > bootstrapping it to test reliability. > > (This is why I'm not too sympathetic to claims that the monobuild is > mostly for binary distributions, because we're actually *more* > vulnerable to issues as a result of it being split when building from > source if using the LLVM toolchain.) > > * Maintaining older LLVMs > > It's easier for older LLVMs to be either maintained by somebody else > (for e.g. Rust's purposes) or in an overlay if it's a single package > rather than many. > > This is also true for e.g. testing old snapshots or keeping binary > packages to downgrade to once they got cleaned up. Another issue with monobuild would be handling the case where USE-depends-on-other-USE and you either have a lot of REQUIRED_USE, or USE that do nothing. This is more of an issue for the runtimes, I think. We would also have an issue where we either have to default-enable Clang, or require many, many users to enable it manually/via autounmask (as opposed to now where it gets dragged in as a dep). > [...]