[gentoo-dev] Profile 23.0 testing with stages and binhost (part 1 of 2)

[gentoo-dev] Profile 23.0 testing with stages and binhost (part 1 of 2)

[Andreas K. Huettel]

[-- Attachment #1: Type: text/plain, Size: 2197 bytes --]

Hi all, 

for the following arches (and only these)

** GLIBC:
** alpha, arm, ia64, loong, m68k, ppc, ppc64, riscv, s390, sparc, x86

** MUSL:
** riscv

the 23.0 profiles are ready for testing, including stage downloads,
binary packages, and update instructions for existing installations.

IMPORTANT Draft update instructions IMPORTANT
https://wiki.gentoo.org/wiki/Project:Toolchain/23.0_update_instructions

Stage downloads (temporarily, for all listed arches):
[preferably] https://distfiles.gentoo.org/experimental/x86/23.0_stages/
[direct/osu] https://gentoo.osuosl.org/experimental/x86/23.0_stages/

The changes can be seen here
https://wiki.gentoo.org/wiki/Project:Toolchain/23.0_profile_transition

and the timeline so far here
https://wiki.gentoo.org/wiki/Project:Toolchain/23.0_profile_timeline

Please try things out, and file bugs if you encounter problems.

In particular, also double-check the documentation - the update
path table was a lot of manual work, and it's not impossible that 
C-C / C-V went wrong somwhere.

The update instructions also double as the news item that should be
published 2 weeks from now on these arches (unless there are 
unexpected problems). I'll reply to this e-mail with a separate 
mail containing the full text, for easier discussion.

Note 1: The next steps are, in 2 weeks:
* make 23.0 profiles the same stability level as 17.x profiles,
* degrade 17.x profiles all to exp (so the CI doesn't explode)
* publish news item
* replace stage downloads with 23.0 version (in situ)

Note 2: Musl is still missing out on *stable* arches, since this
requires musl-1.2.4 stable.

Note 3: The remaining arches follow when they are ready.
Why are they not ready?
 * amd64: comes last :)
 * arm64: bug #916381 still needs to be implemented
 * hppa: slow hardware (one gcc build takes ~1 week)
 * mips: slow qemu emulation (one gcc build takes ~1 day),
     many variants and complex profile structure
 * musl on stable arches: needs musl-1.2.4 stable

Cheers & have fun,
Andreas

-- 
Andreas K. Hüttel
dilfridge@gentoo.org
Gentoo Linux developer
(council, toolchain, base-system, perl, libreoffice)

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 981 bytes --]

[gentoo-dev] Profile 23.0 news item (was: Re: Profile 23.0 testing with stages and binhost (part 1 of 2))

[Andreas K. Huettel]

[-- Attachment #1: Type: text/plain, Size: 5776 bytes --]

News item / update instructions draft:


Title: Profile upgrade to version 23.0 available
Author: Andreas K. Huettel <dilfridge@gentoo.org>
Posted: yyyy-mm-dd
Revision: 1
News-Item-Format: 2.0
Display-If-Keyword: alpha
Display-If-Keyword: arm
Display-If-Keyword: ia64
Display-If-Keyword: loong
Display-If-Keyword: m68k
Display-If-Keyword: ppc
Display-If-Keyword: ppc64
Display-If-Keyword: riscv
Display-If-Keyword: s390
Display-If-Keyword: sparc
Display-If-Keyword: x86

[*** Ignore this message for now if you are using musl. 
     musl profiles and stages are not ready yet.         ***]

A profile upgrade to version 23.0 is available for your architecture. 
The new 23.0 profiles enable some toolchain hardening features and 
performance enhancements by default, and standardize settings.
You can find the list of changes on the wiki tracking page [1].

We strongly advise to precisely follow the upgrade instructions found
below. The 17.0, 17.1, 20.0, and 22.0 profiles will be marked deprecated 
in 2 months and removed a year later. The exact dates depend on the 
architecture, see [2].

Upgrade instructions

Note 1: The use of binary packages is completely optional, and also not 
as much tested as the source-based upgrade path yet. If you prefer to 
only use the traditional source-based installation, omit the "--getbinpkg" 
parameter in all emerge invocations.

Note 2: If you have manually changed your CHOST to a value different from 
what the stages and profiles set, you may have to do that in the future too. 
In that case you should know what you are doing, hopefully; please read the 
instructions with a critical eye then.

1. Ensure your system backups are up to date. Please also update
   your system fully and depclean before proceeding.
   glibc older than 2.36 and musl older than 1.2.4 is not supported anymore.

2. If you are still using one of the long-deprecated amd64 17.0 profiles 
   (other than x32 or musl), then first complete the migration to the 
   corresponding 17.1 profile. Instructions can be found at [3].
   
3. If you are currently using systemd in a split-usr configuration, then first 
   complete the migration to the corresponding merged-usr profile of the 
   same profile version. Details on how to do this can be found in the news 
   item [4].

4. Run "emerge --info" and note down the value of the CHOST variable.

5. Edit /etc/portage/make.conf; if there is a line defining the CHOST variable,
   remove it. Also delete all lines defining CHOST_... variables.

6. Select the 23.0 profile corresponding to your current profile, either using
   "eselect profile" or by manually setting the profile symlink.
   Note that old profiles are by default split-usr and the 23.0 profiles by
   default merged-usr, i.e. example upgrades are
      default/linux/amd64/17.1                    ==> default/linux/amd64/23.0/split-usr
      default/linux/amd64/17.1/systemd/merged-usr ==> default/linux/amd64/23.0/systemd
   A detailed table can be found at [5].
   In rare cases (hppa, x86) the table will tell you to pick between two choices. 
   What you need should be obvious from your *OLD* CHOST value (from step 4).

7. Delete the contents of your binary package cache at ${PKGDIR}
     rm -r /var/cache/binpkgs/*

8. In the file or directory /etc/portage/binrepos.conf (if existing), update
   the URI in all configuration such that they point to 23.0 profile binhost 
   directories. The exact paths can be found in the table at [5], too.

9. Rebuild or reinstall from binary (if available) the following packages in
   this order, with the same version as already active:
     emerge --ask --oneshot --getbinpkg sys-devel/binutils
   (you may have to run binutils-config and re-select your binutils now)
     emerge --ask --oneshot --getbinpkg sys-devel/gcc
   (If this command fails because of mismatched "openmp" useflag requirements, 
    make sure you have FEATURES=preserved-libs enabled, ignore the advice given
    by emerge, and try again with only --nodeps added to the command line.)
   (you may have to run gcc-config and re-select your gcc now)
   and the C library, i.e. for glibc-based systems
     emerge --ask --oneshot --getbinpkg sys-libs/glibc
   or for musl-based systems
     emerge --ask --oneshot --getbinpkg sys-libs/musl

10. Re-run "emerge --info" and check if CHOST has changed compared to step 3.

If the CHOST has NOT changed, skip to step 13 (env-update). Otherwise, 

11. Recheck with binutils-config and gcc-config that valid installed versions
   of binutils and gcc are selected.

12. Check /etc/env.d, /etc/env.d/binutils, and /etc/env.d/gcc for files that
   refer to the *OLD* CHOST value, and remove them. 
   Examples how to do this can be found in the similar procedure at [6].

13. Run env-update && source /etc/profile

14. Re-emerge libtool:
   emerge --ask --oneshot --getbinpkg libtool

15. Just for safety, delete the contents of your binary package cache at 
    ${PKGDIR} again:
     rm -r /var/cache/binpkgs/*

16. Rebuild world:
   emerge --ask --emptytree --getbinpkg @world


[1] https://wiki.g.o/wiki/Project:Toolchain/23.0_profile_transition
[2] https://wiki.g.o/wiki/Project:Toolchain/23.0_profile_timeline
[3] https://www.gentoo.org/support/news-items/2019-06-05-amd64-17-1-profiles-are-now-stable.html
[4] https://www.gentoo.org/support/news-items/2022-12-01-systemd-usrmerge.html
[5] https://wiki.g.o/wiki/Project:Toolchain/23.0_update_table
[6] https://wiki.g.o/wiki/Changing_the_CHOST_variable#Verifying_things_work

-- 
Andreas K. Hüttel
dilfridge@gentoo.org
Gentoo Linux developer
(council, toolchain, base-system, perl, libreoffice)

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 981 bytes --]