From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from lists.gentoo.org (pigeon.gentoo.org [208.92.234.80]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by finch.gentoo.org (Postfix) with ESMTPS id 9583A158041 for ; Sun, 7 Apr 2024 11:24:34 +0000 (UTC) Received: from pigeon.gentoo.org (localhost [127.0.0.1]) by pigeon.gentoo.org (Postfix) with SMTP id D8E29E2A54; Sun, 7 Apr 2024 11:24:29 +0000 (UTC) Received: from james.steelbluetech.co.uk (james.steelbluetech.co.uk [78.40.151.100]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by pigeon.gentoo.org (Postfix) with ESMTPS id 63DE2E2A43 for ; Sun, 7 Apr 2024 11:24:28 +0000 (UTC) Received: from ukinbox.ecrypt.net (hq2.ehuk.net [10.0.10.2]) by james.steelbluetech.co.uk (Postfix) with ESMTP id 16CBFBFC19 for ; Sun, 7 Apr 2024 12:24:27 +0100 (BST) DKIM-Filter: OpenDKIM Filter v2.10.3 james.steelbluetech.co.uk 16CBFBFC19 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ehuk.net; s=default; t=1712489067; bh=kYLogxfm9+2VfSOkLF3w03aiLr+bLtgP6ptxtTbpi/8=; h=Date:Subject:From:To:References:In-Reply-To:From; b=UZUq//m8LORFhYJYYig8OXzLBe+zmbriSmVhtU4bFki+XVz9FnPPzvIZICcx48r++ ICcL/KDXbc6ZJ+Jg/iE7hd6PtaGI4gdpqLyzTfEIiuNittF1tXRdSRAGFbRi/GHG3h wOyqtwluxvrEU2IxuNNDH4uYJJu9Xtmpo3wHIw5mOrGviSMOwRHk8KCn5DoFBke8ya RitZ7yw4p8SoQ/Cl8+Xfus7kLzC/PZ4O4Ulji5HN3+6uHaPo48JyXiCS9BNoqW3R/8 iVBc8Lo86Ji3Tc2YTiUcakqq/VkehOoFUi4iCPqHDBhJRrJiGU0xlggUZLI8WZNqDu PF3KzBEUFSSgw== Message-ID: Date: Sun, 7 Apr 2024 12:24:27 +0100 Subject: Re: [gentoo-dev] Current unavoidable use of xz utils in Gentoo From: "Eddie Chapman" To: gentoo-dev@lists.gentoo.org User-Agent: SquirrelMail/1.5.2 [SVN] Precedence: bulk List-Post: List-Help: List-Unsubscribe: List-Subscribe: List-Id: Gentoo Linux mail X-BeenThere: gentoo-dev@lists.gentoo.org Reply-to: gentoo-dev@lists.gentoo.org X-Auto-Response-Suppress: DR, RN, NRN, OOF, AutoReply MIME-Version: 1.0 Content-Type: text/plain;charset=utf-8 Content-Transfer-Encoding: 8bit References: <875xwy8wxo.fsf@gentoo.org> <963ef0b6-7c2a-4730-b09d-5a829c3ff4c0@gmail.com> <92ef54a0-7a49-49f3-b3cc-d38a2b9adebd@ehuk.net> <877cha31sm.fsf@gentoo.org> In-Reply-To: <877cha31sm.fsf@gentoo.org> X-Scanned-By: MIMEDefang X-Archives-Salt: 7d0bf4cb-78c0-44e7-a79e-17c0001ba713 X-Archives-Hash: bee343390f1044dd655953d9cb3a518a Sam James wrote: > Eddie Chapman writes: >> Below is a guide I've written to removing app-arch/xz-utils in case >> anyone else wants to do so. Attached is the current version of the Bash >> wrapper script I now use in place of /usr/bin/xz >> >> Comments, corrections on anything technical in the guide or script are >> welcome, apart from flames about how this is ridiculous and unnecessary >> :-). > > For an experiment I'm doing (distinct from trying to purge xz-utils, > just verification work), I've packaged the following: * app-arch/gxz (pure > Go impl.) > * app-arch/7zip (7zip upstream are supporting Linux now, app-arch/p7zip > was an unofficial port) > > You might find those useful too. That's fantastic. I wrote about p7zip vs. upstream 7zip in another mail in this thread and was intending to create a local ebuild for 7zip soon but won't have to know it's in tree :-) > At a glance, it appears https://github.com/fpgaminer/rust-lzma and > https://github.com/gendx/lzma-rs don't provide executables - just a > library - so I didn't bother looking further. >> >> ==== Guide to removing xz utils on a Gentoo system ==== >> >> [...] >> There is one significant thing that breaks, which is Gemato >> (app-portage/gemato). Gemato requires lzma support in core python in >> order to do GPG signature verification. This means you will have to say >> goodbye (for now) to verifying upstream GPG signatures on distfiles, and >> verification of Portage metadata after doing an emerge --sync. These >> features have been added to Portage relatively recently (2022?) so are >> "nice to have", without them your system is just less > > No.. much older. It was introduced around the time of the github mirror > being hacked. It's not just theatre! > > Like, this is very much NOT hypothetical. Thanks, couldn't remember when it was. > It's not just about metadata, it's about the ebuilds if using rsync, or > the whole git checkout if using git. Completely agree with you that this was a great feature to be added from a security point of view. Without it there was still a level of trust, however small, that could be placed in the choice of mirror. But there's no doubt gpg sigs of repo data are order of magnitudes better, so yes it was a little unjust to describe it as only "nice to have". But in the current situation I personally consider it so critically important to get rid of xz utils from my systems that a short, temporary period of not having this while switching to another method of verification I consider an acceptable tradeoff (side note to anyone reading: yes I know how at odds I am with the rest of the world on this, it has now been argued to death in this thread so for anyone thinking about replying about that, maybe lets do everyone a favour, agree to disagree, and move on :-) ) >> hardened, but still with the very high level of security that Gentoo >> systems have has always had prior to these features, in my opinion. >> Personally I can live without them for now. Verifying hashes >> in Manifest files still works fine and that's the main thing. You may >> disagree in which case, well, don't do this then. I'm going to figure >> out an alternative way I can verify Portage metadata soon, as there are >> other ways if you are creative. > > See grobian's reply which should help. > > >> [...] >> - Portage binary packages: You cannot use xz compression if you create >> Portage binary packages. You will need to use one of bzip2, gzip, >> lz4, lzip, lzop, or zstd in BINPKG_COMPRESS in make.conf instead of xz >> (if that is what you were using, or is it the default?). I have >> > > zstd is the default for "new" installs (since a few years ago), yeah. > >> [...] >> - sys-apps/fwupd might stop working properly (though it will still >> build fine) due to what you have to change with dev-libs/libxmlb below. >> I'm not sure as I haven't checked yet, I just suspect it >> will. So bear that in mind if you need to rely on sys-apps/fwupd at the >> moment. But this "might" is temporary, upstream has now decided to make >> lzma optional, so this will trickle down to Gentoo soon. > > Just for completeness, this is > https://blogs.gnome.org/hughsie/2024/04/03/fwupd-and-xz-metadata/. > Thanks for all the useful additions of info :-) cheers, Eddie