From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from smtp.gentoo.org (woodpecker.gentoo.org [140.211.166.183]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by finch.gentoo.org (Postfix) with ESMTPS id BF7221584AD for ; Fri, 02 May 2025 21:16:44 +0000 (UTC) Received: from lists.gentoo.org (bobolink.gentoo.org [140.211.166.189]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits)) (No client certificate requested) (Authenticated sender: relay-lists.gentoo.org@gentoo.org) by smtp.gentoo.org (Postfix) with ESMTPSA id AB3E93430CF for ; Fri, 02 May 2025 21:16:44 +0000 (UTC) Received: from bobolink.gentoo.org (localhost [127.0.0.1]) by bobolink.gentoo.org (Postfix) with ESMTP id 5A7AC110497; Fri, 02 May 2025 21:15:42 +0000 (UTC) Received: from mail.muc.de (mail.muc.de [193.149.48.3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by bobolink.gentoo.org (Postfix) with ESMTPS id BF3A6110287 for ; Fri, 02 May 2025 21:15:39 +0000 (UTC) Received: (qmail 15355 invoked by uid 3782); 2 May 2025 23:15:38 +0200 Received: from muc.de (p4fe155b4.dip0.t-ipconnect.de [79.225.85.180]) (using STARTTLS) by colin.muc.de (tmda-ofmipd) with ESMTP; Fri, 02 May 2025 23:15:37 +0200 Received: (qmail 30238 invoked by uid 1000); 2 May 2025 21:15:37 -0000 Date: Fri, 2 May 2025 21:15:37 +0000 To: gentoo-user@lists.gentoo.org Subject: Re: [gentoo-user] Fragile python update is wedged [Was: Fragile rust update is wedged]. Message-ID: References: Precedence: bulk List-Post: List-Help: List-Unsubscribe: List-Subscribe: List-Id: Gentoo Linux mail X-BeenThere: gentoo-user@lists.gentoo.org Reply-to: gentoo-user@lists.gentoo.org X-Auto-Response-Suppress: DR, RN, NRN, OOF, AutoReply MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: X-Submission-Agent: TMDA/1.3.x (Ph3nix) From: Alan Mackenzie X-Primary-Address: acm@muc.de X-Archives-Salt: 6848109c-54fc-4f75-b03e-bcd9105f8f93 X-Archives-Hash: 07dea8a7938f9ad84f4e0bbf9953ab9a Hello, Eli. On Fri, May 02, 2025 at 16:07:17 -0400, Eli Schwartz wrote: > On 5/2/25 3:07 PM, Alan Mackenzie wrote: > > On Fri, May 02, 2025 at 13:47:02 -0400, Eli Schwartz wrote: > >> On 5/2/25 1:07 PM, Alan Mackenzie wrote: > >>> I've just been trying the update for python 3.13. It went well on my > >>> new machine (well, after unmerging app-portage/unsymlink-lib, which was > >>> debris from some 2019 update). > >>> On my old machine, however, there was a seg fault while merging rust. > >>> This is a known problem in certain first generation Ryzen processors. > >>> However, this left the build wedged: emerge says that there's a circular > >>> dependency from rust to itself. More precisely, the error message looks > >>> like: > >> This is clearly not a python update, then. > > Well, it was. ;-) > Your saying that it was, doesn't make it the case. So you can repeat a > false claim if you want -- or explain why you think it is true. It was an update. It was for the purpose of replacing python 3.12 with python 3.13. Thus it was a python update. > >>> This would appear to be a bug in the rust ebuild - it has set itself as a > >>> builtime dependency before it had been built - or something like that. > >>> How do I best recover from this? > >> It is not a bug in the rust ebuild -- the rust ebuild has always, > >> always, always depended on itself. In the past, the rust ebuild silently > >> downloaded a temporary copy of dev-lang/rust-bin to work around this. > > I disagree. By attempting this emerging of rust, and failing, some > > persistent information indicating this dependency has been created. This > > persistent information, wherever it is, should not have been created, or > > it should have been removed on the emerge failure. > Since you do not know what this "persistent information" is, I assert in > response that it doesn't exist, therefore it hasn't been created. Well, thanks for being so generous and understanding. At the start of this python update, I was able to start emerge, which carried on until the seg fault part way through emerging rust. On attempting to restart the python update, portage reported a "circular dependency" in rust and failed to build any packages. Some persistent information thus clearly differed between these two attempts. I don't know what was changed, and I shouldn't have to find out, being a user of emerge, not a maintainer. > If you are nonetheless certain that it exists, you are welcome to > explain what "it" is and where I can find it. See above. > >> This was changed in November 2024. > > Hmmm... OK. > >> The rust compiler is written in the rust programming language, using > >> bleeding-edge features of the most recent version of rust. They are > >> aware that this is a bootstrap problem and don't consider it a problem, > >> because rust is officially installed by downloading prebuilt binaries > >> into $HOME. > > This sounds bad, but how is it different from C and C++ in GCC? > Because: > - GCC 15 can be compiled with GCC 5.4 > - GCC 11 - 14 can be compiled with GCC 4.8.3 > - GCC 4.8 - 10 can be compiled with a C++98 compiler > - GCC 4.7 can be compiled with a C compiler > The bootstrappable.org project uses TinyCC to build GCC 2.95.0, which > then builds GCC 4.7.4, which gives you a very nearly C++11 compiler. > It takes you 4 hops from TinyCC to GCC 15, which is a quarter of a > decade of technology. ?Quarter of a century, perhaps? OK, thanks. That was more or less what I suspected. GCC was at the heart of the GNU project right from the beginning, and having people able to build it easily was always important. I'm glad that's still the case. > For rust, the best bootstrap we have is unofficial -- mrustc -- and > relies on first bootstrapping GCC with 4 hops, then compiling 13 > versions of Rust released between December 2023 and today. And that's > our hack to avoid building 80 versions released over the course of 10 > years (rust 1.0 was released on May 15, 2015). Somehow that feels like it's pushing the definition of free software in a direction it was never intended to go. > >> Compiling rust officially requires building the pre 1.0 version of rust, > >> itself written in ocaml, and then building *every* version of rust since > >> then, one by one, in turn, without skipping a single one. That is over > >> 80 different versions of rust, you need to compile all of them in order > >> to get to the current one. > > No thanks! There's only so much time left before the Sun becomes a red > > giant. But this attitude must limit the portability of rust to different > > processors, such as are found in embedded systems. > >> On amd64, you can use the unofficial dev-lang/mrustc (third-party > >> reimplementation of rust written in C++) which allows you to skip all > >> the way to rust 1.74, and from there it is "only" 13 or so versions to > >> build. > > Doesn't sound much like a solution. > You're entirely correct. The upstream rust developers imposed this > constraint. If that's the way they see things, it would seem that having a project dependent on rust should figure high on that project's risk list. > >> Or, you may use rust-bin. Alternatively, you may use: > >> ``` > >> emerge --getbinpkg dev-lang/rust > >> ``` > > I think I'll be going for rust-bin. This is quite new in Gentoo (in the > > last few years, at any rate), but I couldn't find the news item about it > > I vaguely remember reading. > > Hopefully there's nothing tricky about moving from package foo to > > foo-bin. > > And then I can, hopefully, finish the update of the python stuff. Though > > I would classify it as a design bug that all python programs need to be > > rebuilt just because there's a minor version update in python itself. We > > don't need to do this when a new version of GCC comes out. > It's not a minor version update in python itself. Python doesn't really > use semver. Use what? > GCC, and in general, C/C++, tend to take this sort of compatibility a > lot more serious than other languages. e.g. rust and golang simply > statically compile, and that's why @rust-rebuild and @golang-rebuild > exist. Ruby has to be rebuilt on version updates just like python. Zig > software only ever supports a single version of the compiler, and has to > be rewritten in tandem with the compiler upgrade. Thank goodness for C! Anyhow, my old machine is now in an unusable state. When I repeat the emerge incantation from the news item, emerge fails with obscure messages about conflicting package versions. These messages take several hours to parse and understand, being basically dumps of the internal state of emerge. One way or another, I'll get that machine back in working order. But I can't help thinking it shouldn't be this difficult, even in Gentoo. I think my first summary of my problem was fair: The python update scheme is fragile. In particular if the emerge is interrupted for any reason, it cannot be restarted, and cannot be reverted to a workable state, without a great deal of time and expert knowledge. > -- > Eli Schwartz -- Alan Mackenzie (Nuremberg, Germany).