From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from lists.gentoo.org (pigeon.gentoo.org [208.92.234.80]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits)) (No client certificate requested) by finch.gentoo.org (Postfix) with ESMTPS id 592AE1581D3 for ; Tue, 14 May 2024 12:02:43 +0000 (UTC) Received: from pigeon.gentoo.org (localhost [127.0.0.1]) by pigeon.gentoo.org (Postfix) with SMTP id 5F10CE2A27; Tue, 14 May 2024 12:02:40 +0000 (UTC) Received: from smtp.gentoo.org (woodpecker.gentoo.org [140.211.166.183]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by pigeon.gentoo.org (Postfix) with ESMTPS id 18EC8E2A1C for ; Tue, 14 May 2024 12:02:40 +0000 (UTC) Message-ID: <2861c2f2-a305-454a-99c8-0c6f2c9db745@gentoo.org> Date: Tue, 14 May 2024 14:02:35 +0200 Precedence: bulk List-Post: List-Help: List-Unsubscribe: List-Subscribe: List-Id: Gentoo Linux mail X-BeenThere: gentoo-dev@lists.gentoo.org Reply-to: gentoo-dev@lists.gentoo.org X-Auto-Response-Suppress: DR, RN, NRN, OOF, AutoReply MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: [gentoo-dev] [PATCH 0/7] Changes to dracut kernel module/microcode handling To: gentoo-dev@lists.gentoo.org, pr@gentoo.org References: <20240514115924.29167-1-andrewammerlaan@gentoo.org> <20240514115924.29167-7-andrewammerlaan@gentoo.org> Content-Language: en-US, nl-NL From: Andrew Nowa Ammerlaan Autocrypt: addr=andrewammerlaan@gentoo.org; keydata= xsBNBF3n3cUBCAC6uoDZ0XzaO29l8AzUblXQ5rxZI7nbGEnfFqjEQCK3oEXxsDa9Ez1myx3M ir53Vyx64Iz1Bq/TOS/PttgguPpiLggCpTTD2vavp5SwFmg272+P8bUJVJF2mMRm0OR/YPiA B5dNfcoLqKIj+ZMOtrZ72B7agkUn+iDt8lB2fZ7XhfZMyQBXICYSe+EiJJmTuvIhHhOn7GCT VjpwGYCCSw3F/j2VPmJPUftz6Nb4oWaiaJ6ZwroS2ECYqZKeo+dXCsmB/LZWYqIFSSPILTLZ f1Hh/TklnQqkNVO+nY/B/o9RVYAhWJbl/F4VaKlRXemE+pDZIALlK8kt0IFU6liUOHHlABEB AAHNLUFuZHJldyBBbW1lcmxhYW4gPGFuZHJld2FtbWVybGFhbkBnZW50b28ub3JnPsLAlwQT AQgAQQIbAwULCQgHAgYVCgkICwIEFgIDAQIeAQIXgAIZARYhBAb/U0G9gF2wvH0HpqGfY2zU 7bzRBQJlNiYEBQkJL3u/AAoJEKGfY2zU7bzRUeYH/33eX3sOyo3++xcqR/KrTNodkgWAknPe Jl8BiYdIn7zEgif5Fz6Uu8IzjfDpPd8uR82sbV2uQWarrpNmnPrAACKuAuYN9vnuLZ+9UWz6 ybGqMm545+qsFtUTTzdveMPEWr2nr+payfxthK6OdgZU5ZseLxDS9KYmBeAC7RVnIWMVDn9n opmuFK5iGxIUvIbYIl/xrk2HPAIsh1ScLBy4z7r8PFmWT1XGC0Na6PJyEG2KiQXwjKxwsljQ 6mKEAkKOkbifD0CSO8eg56ccf8WYo0s/+SiYjBjI9SEhbgZbiUbpTSw3eT/g4V2SKX1CYs1z 717XjlMKzqBNaw+AzWgrk0TOwE0EXefdxQEIAJtT7965MCxOTic3mISWSI6Z3mFFYmUkxQt8 gBVsTAezOrkd6xEt/HnFPZqeGnbSiV8gMFPKv4RkaXxWfQYKm+9/12qJNEFdVop1rpe77lU2 h0elVXuWiWsNmwqEhQcs1mq/awzO81Lyob9Miai2qNQ9MBikmFAp9c4n8C42kPLVrTKPmemI 95gZ1Y830W+udYg1jNqLF2ucMDUX1M1U2EfazWI0pNCwPoKnOqAJS+VQbyxtJ1IlE3+9sk+6 hjlTTF+RDYGv5hUoWkmcXDM2X/Cl0XB4XYOWr17Wa6+WXC+80/iLxxolMqM4KfuIR5OizbqK 2CRAJY7la7TSv1lTD1cAEQEAAcLAfAQYAQgAJgIbDBYhBAb/U0G9gF2wvH0HpqGfY2zU7bzR BQJlNiHABQkJL3d7AAoJEKGfY2zU7bzRjDwH/1fp/87km2YYVgrfP1aWLjAA/TwcEVycRJQQ S9Q6xuzgD5AYhjzBSONoN46cwf+gla6xndY0lCawsZN7whtJ/DhqSZEfL0HgHkJ6T8FCXexf n1s6XmIAxqIrMmfsuOkAPLJIHzAAGzQX8DXcRSj1cIDUpa1Uy7ncVvI4EzJBRtJVJXIbl+53 NGauXU8ZuprPYkMSPuW3eHATFc0F5DhmlFUXh+HYYK+2QTO73TENMhngkrYcw63je5bRp/+f 72XFKlf1gXHK1ivg8nYueyUfrxZTBGKagusOiQeOao2I1uYcHoFhPYJrQWePMyZiYyB6PR0K DR4B/Ulo3v0eBXaaYzo= Organization: Gentoo Linux In-Reply-To: <20240514115924.29167-7-andrewammerlaan@gentoo.org> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-Archives-Salt: 3ddffd9e-ebb9-4821-88c3-6cc4215da44c X-Archives-Hash: abe0f4e64173e70ca229bc4526120ee9 Hi all, This patch series essential moves the dist-kernel initramfs/UKI rebuilding out off zfs-kmod.ebuild and into linux-mod-r1.eclass. In the process we fix several issues: - zfs-kmod.ebuild binpkgs no longer rebuild currently running kernel, but uses instead the kernel version that matches the version the modules were built for (Bug 931213, commit 1,2,3,6,7). - zfs.ebuild binpkgs no longer rebuild currently running kernel, but use the eselected kernel instead (commit 6,7). In most "normal" cases the eselected kernel version will match the kernel version zfs-kmod modules were built for due to the slot operator on the virtual/dist-kernel dependency in both ebuilds, and the automatic adjustments of the /usr/src/linux symlink when dist-kernels are used. (TODO: consider merging the zfs and zfs-kmod ebuilds to make this 100% robust). - Which modules are and which ones are not included in the Dracut generated initramfs is now properly defined and won't change after random dist-kernel reinstallations (commit 1,2). - As a result of the above, Dracut's "--reproducible" setting actually makes something that is reproducible (apart from package upgrades and configuration changes of course). - It is now super easy for kernel module package maintainers to define that a module can be included in the initramfs, USE flags and (initramfs/UKI) rebuilds are taken care off by the eclass (commits 1,2) - We mirror the changes to the kernel modules in the packages that install CPU microcode (linux-firmware and intel-microcode) by introducing the "dist-kernel" flag there which alters the behaviour of the "initramfs" flag to instead configure Dracut to include the microcode and then rebuild the Dracut initramfs/UKI. This ensures that microcode updates are always propagated into the initramfs. (commits 4,5) Most users will not have to do anything, we are only setting defaults in /usr/lib/dracut/dracut.conf.d, the (existing) user configuration in /etc/dracut.conf.d takes precedence. Nonetheless, I wrote a news item to accompany these changes, users should ensure that they do not unintentionally overwrite USE=initramfs, and they may also want to get rid of custom configuration/workarounds they have set in /etc/dracut.conf. Best regards, Andrew Title: Changes to dracut kernel module/microcode handling Author: Andrew Ammerlaan Posted: 2024-05-10 Revision: 1 News-Item-Format: 2.0 Display-If-Installed: sys-kernel/dracut Display-If-Installed: virtual/dist-kernel Display-If-Installed: sys-kernel/linux-firmware Display-If-Installed: sys-firmware/intel-microcode Impact ==================== Several changes were made regarding out-of-tree kernel modules, CPU microcode, and how these are handled in initial RAM file systems (initramfs) generated by sys-kernel/dracut for distribution kernels. Depending on the local Dracut and USE flag configuration, some configuration adjustments may be required as a result of these changes. Background (the problem) ==================== Previously Dracut implicitly included all out-of-tree kernel modules it could find. This leads to several problems: - It unnecessarily increases the size of the initramfs - It creates a bit of a mess when using distribution kernels, consider the following: 1) Distribution kernel is upgraded 2) Initramfs for the new kernel is generated, it does not include any out-of-tree kernel modules. 3) Portage triggers rebuild of the out-of-tree kernel modules 4) If zfs is installed, its rebuild will trigger an initramfs re-installation. Otherwise no rebuild is triggered. Problem: What is and is not included in the initramfs is now ambiguous. It depends on the emerge order of the kernel modules when zfs is used. And will completely change if at some later stage regeneration of the initramfs is triggered manually via e.g.: emerge --config sys-kernel/gentoo-kernel As a result, Dracut's "--reproducible" setting is not working. And the functionality of the initramfs may change (seemingly) at random. Background (the fix) ==================== Several things have been changed: - Out-of-tree kernel modules installed by portage are explicitly omitted from the initramfs generated by Dracut by default. - Packages that install a kernel module for which it might make sense to have it in the initramfs, have gained the "initramfs" USE flag. When this flag is enabled, Dracut is instructed to include the installed kernel modules. Packages for which it is essential that its kernel modules are included in the initramfs have this new flag enabled by default. - When distribution kernels are used (USE=dist-kernel), and a module that should be in the initramfs is installed (USE=initramfs) the initramfs is always re-generated. - The packages installing CPU microcode (sys-kernel/linux-firmware and sys-firmware/intel-microcode) have been adjusted to mirror the above changes for out-of-tree kernel modules. Both packages have gained the "dist-kernel" USE flag, and the "initramfs" flag is now enabled by default. When both flags are enabled, Dracut is configured to include the installed microcode in the initramfs, and then the initramfs is regenerated. When the "dist-kernel" flag is disabled, the "initramfs" flag behaves as it previously did. User Action Required ==================== Users should double check two things: 1) Please ensure that you are *not* globally enabling or disabling the "initramfs" USE flag. Enabling it globally might result in an unnecessarily large initramfs. Disabling it globally might result in missing functionality in the initramfs. Which could lead to boot failure if, for example, the zfs module is missing while the root partition is a zfs. 2) Any add_drivers, or omit_drivers lines in /etc/dracut.conf or /etc/dracut.conf.d/* may override the Dracut configuration snippets installed by the kernel module packages in /usr/lib/dracut/dracut.conf.d. Please review your Dracut configuration files to ensure that you are not unintentionally overriding the settings set by Portage. Frequently Asked Questions ==================== A package installing a kernel module I would like in my initramfs has not gained the "initramfs" USE flag. How do I proceed? Please report a new bug on bugs.gentoo.org, requesting that the package maintainer consider adding support to the package for including the modules in the initramfs. In the meantime you can locally override the configuration provided by the package (see below). Note though that when distribution kernels are used, regeneration of the initramfs must be triggered manually via e.g.: emerge --config sys-kernel/gentoo-kernel How do I override the provided Dracut configuration snippets to include/exclude a custom list of modules? To override the provided configuration snippet, create a new file /etc/dracut.conf.d/10-PACKAGENAME.conf, replacing PACKAGENAME with the name of the package providing the module. Add to this file: omit_drivers+=" my list of drivers to omit " and/or add_drivers+=" my list of drivers to include "