Re: [gentoo-dev] Current unavoidable use of xz utils in Gentoo

["Eddie Chapman" <eddie@ehuk.net>]

Sam James wrote:
> Eddie Chapman <eddie@ehuk.net> writes:
>> Below is a guide I've written to removing app-arch/xz-utils in case
>> anyone else wants to do so.  Attached is the current version of the Bash
>> wrapper script I now use in place of /usr/bin/xz
>>
>> Comments, corrections on anything technical in the guide or script are
>> welcome, apart from flames about how this is ridiculous and unnecessary
>> :-).
>
> For an experiment I'm doing (distinct from trying to purge xz-utils,
> just verification work), I've packaged the following: * app-arch/gxz (pure
> Go impl.)
> * app-arch/7zip (7zip upstream are supporting Linux now, app-arch/p7zip
> was an unofficial port)
>
> You might find those useful too.

That's fantastic. I wrote about p7zip vs. upstream 7zip in another mail in
this thread and was intending to create a local ebuild for 7zip soon but
won't have to know it's in tree :-)

> At a glance, it appears https://github.com/fpgaminer/rust-lzma and
> https://github.com/gendx/lzma-rs don't provide executables - just a
> library - so I didn't bother looking further.
>>
>> ==== Guide to removing xz utils on a Gentoo system ====
>>
>> [...]
>> There is one significant thing that breaks, which is Gemato
>> (app-portage/gemato). Gemato requires lzma support in core python in
>> order to do GPG signature verification. This means you will have to say
>> goodbye (for now) to verifying upstream GPG signatures on distfiles, and
>> verification of Portage metadata after doing an emerge --sync. These
>> features have been added to Portage relatively recently (2022?) so are
>> "nice to have", without them your system is just less
>
> No.. much older. It was introduced around the time of the github mirror
> being hacked. It's not just theatre!
>
> Like, this is very much NOT hypothetical.

Thanks, couldn't remember when it was.

> It's not just about metadata, it's about the ebuilds if using rsync, or
> the whole git checkout if using git.

Completely agree with you that this was a great feature to be added from a
security point of view. Without it there was still a level of trust,
however small, that could be placed in the choice of mirror. But there's
no doubt gpg sigs of repo data are order of magnitudes better, so yes it
was a little unjust to describe it as only "nice to have".

But in the current situation I personally consider it so critically
important to get rid of xz utils from my systems that a short, temporary
period of not having this while switching to another method of
verification I consider an acceptable tradeoff (side note to anyone
reading: yes I know how at odds I am with the rest of the world on this,
it has now been argued to death in this thread so for anyone thinking
about replying about that, maybe lets do everyone a favour, agree to
disagree, and move on :-) )

>> hardened, but still with the very high level of security that Gentoo
>> systems have has always had prior to these features, in my opinion.
>> Personally I can live without them for now. Verifying hashes
>> in Manifest files still works fine and that's the main thing. You may
>> disagree in which case, well, don't do this then. I'm going to figure
>> out an alternative way I can verify Portage metadata soon, as there are
>> other ways if you are creative.
>
> See grobian's reply which should help.
>
>
>> [...]
>> - Portage binary packages: You cannot use xz compression if you create
>> Portage binary packages. You will need to use one of bzip2, gzip,
>> lz4, lzip, lzop, or zstd in BINPKG_COMPRESS in make.conf instead of xz
>> (if that is what you were using, or is it the default?). I have
>>
>
> zstd is the default for "new" installs (since a few years ago), yeah.
>
>> [...]
>> - sys-apps/fwupd might stop working properly (though it will still
>> build fine) due to what you have to change with dev-libs/libxmlb below.
>> I'm not sure as I haven't checked yet, I just suspect it
>> will. So bear that in mind if you need to rely on sys-apps/fwupd at the
>> moment. But this "might" is temporary, upstream has now decided to make
>> lzma optional, so this will trickle down to Gentoo soon.
>
> Just for completeness, this is
> https://blogs.gnome.org/hughsie/2024/04/03/fwupd-and-xz-metadata/.
>

Thanks for all the useful additions of info :-)

cheers,
Eddie