From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from lists.gentoo.org (pigeon.gentoo.org [208.92.234.80]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by finch.gentoo.org (Postfix) with ESMTPS id 67CA5138334 for ; Wed, 12 Jun 2019 22:05:47 +0000 (UTC) Received: from pigeon.gentoo.org (localhost [127.0.0.1]) by pigeon.gentoo.org (Postfix) with SMTP id 573F3E07C7; Wed, 12 Jun 2019 22:05:45 +0000 (UTC) Received: from smtp.gentoo.org (smtp.gentoo.org [140.211.166.183]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by pigeon.gentoo.org (Postfix) with ESMTPS id BD374E07C7 for ; Wed, 12 Jun 2019 22:05:44 +0000 (UTC) Received: from symphony.aura-online.co.uk (154.189.187.81.in-addr.arpa [81.187.189.154]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) (Authenticated sender: chewi) by smtp.gentoo.org (Postfix) with ESMTPSA id B1D70345FA4 for ; Wed, 12 Jun 2019 22:05:42 +0000 (UTC) Date: Wed, 12 Jun 2019 23:05:33 +0100 From: James Le Cuirot To: gentoo-pms@lists.gentoo.org Subject: Re: [gentoo-pms] [PATCH] Correct the definition of ESYSROOT as EPREFIX isn't always applicable Message-ID: <20190612230533.5481f211@symphony.aura-online.co.uk> In-Reply-To: <86e060cfba8bd04fba34365e9805852a1ccce472.camel@gentoo.org> References: <20190601222921.12072-1-chewi@gentoo.org> <86e060cfba8bd04fba34365e9805852a1ccce472.camel@gentoo.org> X-Mailer: Claws Mail 3.17.3 (GTK+ 2.24.32; x86_64-pc-linux-gnu) Precedence: bulk List-Post: List-Help: List-Unsubscribe: List-Subscribe: List-Id: Gentoo Package Manager Specification discussions X-BeenThere: gentoo-pms@gentoo.org X-BeenThere: gentoo-pms@lists.gentoo.org Reply-To: gentoo-pms@lists.gentoo.org X-Auto-Response-Suppress: DR, RN, NRN, OOF, AutoReply MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha256; boundary="Sig_//jBKsKiSuTS9s8WQFFl4XSz"; protocol="application/pgp-signature" X-Archives-Salt: 05e418aa-f2de-4238-901f-f8991963c728 X-Archives-Hash: af816767e98dc6d1e2463970e5a71d79 --Sig_//jBKsKiSuTS9s8WQFFl4XSz Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable On Sun, 02 Jun 2019 08:10:34 +0200 Micha=C5=82 G=C3=B3rny wrote: > On Sat, 2019-06-01 at 23:29 +0100, James Le Cuirot wrote: > > Unfortunately my conception of ESYSROOT was a little short-sighted. It > > was previously defined as SYSROOT + EPREFIX but if SYSROOT does not > > equal ROOT then EPREFIX does not make sense in this context. Consider > > a more concrete example: > > =20 >=20 > Once again, you are missing the point that e.g. pkg-config files will > have EPREFIX hardcoded. I had remembered that you objected to this but I had forgotten why. I said at the time that I believed your concerns were misplaced and that hasn't changed but I wanted to actually try it out to prove it to myself as much as you. If EPREFIX is set in the environment by the user, it only applies to ROOT. If you're emerging a bunch of packages in a single run with some going to / and some going to ROOT, Portage will adjust EPREFIX for each destination and hardcode the correct prefix value into the .pc files accordingly. Up till now, we have said that you can set SYSROOT to either match / or ROOT. The former is typically used for native while the latter is typically used for cross. Whether you're building against / or ROOT, the .pc files picked up from each will have the prefix hardcoded for each. This is exactly what we want! It was surprisingly hard to find a simple package that worked under prefix and located a header via pkg-config that otherwise would not be found without an -I flag. After a long search, the package I eventually settled on was your very own x11-libs/libtinynotify, which uses pkg-config to query dbus-1. My setup is as follows: BROOT=3D/home/chewi/prefix ROOT=3D/home/chewi/myroot EPREFIX=3D/myprefix Therefore, Portage sets: EROOT=3D/home/chewi/myroot/myprefix I wanted to try this with both SYSROOT=3D/ and SYSROOT=3D${ROOT}. Without my associated changes to Portage, the former actually blows up. Calculating dependencies... done! Traceback (most recent call last): File "/home/chewi/prefix/usr/lib/python-exec/python3.6/emerge", line 53= , in retval =3D emerge_main() File "/home/chewi/prefix/usr/lib64/python3.6/site-packages/_emerge/main= .py", line 1289, in emerge_main return run_action(emerge_config) File "/home/chewi/prefix/usr/lib64/python3.6/site-packages/_emerge/acti= ons.py", line 3325, in run_action retval =3D action_build(emerge_config, spinner=3Dspinner) ... File "/home/chewi/prefix/usr/lib64/python3.6/site-packages/_emerge/depg= raph.py", line 4722, in _select_atoms_highest_available pkgsettings =3D self._frozen_config.pkgsettings[root] KeyError: '/myprefix/' This error precisely highlights the problem with PMS as it stands now. If SYSROOT=3D/ and EPREFIX=3D/myprefix then SYSROOT + EPREFIX is also /myprefix, which Portage hasn't been told about because it doesn't even exist. It only exists under /home/chewi/myroot. With my changes in place, things are resolved as follows. dbus is installed in both locations because libtinynotify is only EAPI 6. [ebuild N ] sys-apps/dbus-1.12.14 to /home/chewi/myroot/myprefix/ USE= =3D"-X -debug -doc -elogind (-selinux) -static-libs -systemd -test -user-se= ssion"=20 [ebuild N ] sys-apps/dbus-1.12.14 USE=3D"-X -debug -doc -elogind (-se= linux) -static-libs -systemd -test -user-session"=20 [ebuild N ] x11-libs/libtinynotify-0.2.1 to /home/chewi/myroot/myprefi= x/ USE=3D"-debug -doc -static-libs"=20 Updating libtinynotify to EAPI 7 while building with SYSROOT=3D${ROOT} changes the output to this. [ebuild N ] sys-apps/dbus-1.12.14 to /home/chewi/myroot/myprefix/ USE= =3D"-X -debug -doc -elogind (-selinux) -static-libs -systemd -test -user-se= ssion"=20 [ebuild N ] x11-libs/libtinynotify-0.2.1 to /home/chewi/myroot/myprefi= x/ USE=3D"-debug -doc -static-libs"=20 In all cases, both packages build successfully but pkg-config adds -I/home/chewi/prefix/usr/include/dbus-1.0 regardless of SYSROOT. This is because native builds like this have no special SYSROOT handling for pkg-config. This handling exists in crossdev's cross-pkg-config for cross builds but even that currently has no prefix support whatsoever. I therefore have some heavy modifications pending for cross-pkg-config. It is debatable whether we should apply SYSROOT handling to pkg-config all the time. While we do technically allow SYSROOT!=3D/ for native builds, this is so problematic in general that we probably shouldn't go out of our way to support it. We don't need to do anything for the SYSROOT=3D/ case because the default behaviour is what we want. In my example, I'm not cross-compiling so I could force the use of cross-pkg-config to make it work but to keep things simple, I have written a short bashrc that does the equivalent. This file is placed in /home/chewi/myroot/myprefix/etc/portage because we only want it to apply when building packages for ROOT. We therefore need to set PORTAGE_CONFIGROOT so that it actually gets used. if [[ -n ${SYSROOT%/} ]]; then myprefix=3D${EPREFIX%/} else myprefix=3D${PORTAGE_OVERRIDE_EPREFIX%/} fi unset PKG_CONFIG_PATH export PKG_CONFIG_SYSTEM_LIBRARY_PATH=3D"${myprefix}/usr/lib64:${myprefix= }/lib64" export PKG_CONFIG_SYSROOT_DIR=3D"${SYSROOT}" export PKG_CONFIG_LIBDIR=3D"${SYSROOT}${myprefix}/usr/lib64/pkgconfig:${S= YSROOT}${myprefix}/usr/share/pkgconfig" In cross-pkg-config, I check ESYSROOT first and determine the prefix by chopping off SYSROOT. This allows us to support a distinct SYSROOT and is necessary because we didn't define a variable for SYSROOT's prefix. For EAPIs before 7, I fall back to something along the lines of the above. Using PORTAGE_OVERRIDE_EPREFIX works but it's not ideal so I may just bake the value into cross-pkg-config when crossdev is installed. And the result? It works as expected. Regardless of whether libtinynotify uses EAPI 6 or 7, the build output includes the following: With SYSROOT=3D/ : libtool: compile: x86_64-pc-linux-gnu-gcc -DHAVE_CONFIG_H -I. -I/home/ch= ewi/prefix/usr/include/dbus-1.0 -I/home/chewi/prefix/usr/lib64/dbus-1.0/inc= lude -O2 -pipe -O2 -pipe -c lib/common.c -fPIC -DPIC -o .libs/libtinynotif= y_la-common.o With SYSROOT=3D${ROOT} : libtool: compile: x86_64-pc-linux-gnu-gcc -DHAVE_CONFIG_H -I. -I/home/ch= ewi/myroot/myprefix/usr/include/dbus-1.0 -I/home/chewi/myroot/myprefix/usr/= lib64/dbus-1.0/include -O2 -pipe -O2 -pipe -c lib/common.c -fPIC -DPIC -o = .libs/libtinynotify_la-common.o For completeness, I decided to try the distinct SYSROOT case as well with SYSROOT=3D/home/chewi/mysysroot. The short bashrc above does not support this but the revised cross-pkg-config does. You would typically need the libc and kernel headers installed to this SYSROOT first but I haven't defined CC=3D"x86_64-pc-linux-gnu-gcc --sysroot=3D${ESYSROOT}" like I normally do under cross-boss. Only pkg-config is being affected here, which is sufficient for this demonstration. And it works! libtool: compile: x86_64-pc-linux-gnu-gcc -DHAVE_CONFIG_H -I. -I/home/ch= ewi/mysysroot/usr/include/dbus-1.0 -I/home/chewi/mysysroot/usr/lib64/dbus-1= .0/include -O2 -pipe -O2 -pipe -c lib/common.c -fPIC -DPIC -o .libs/libtin= ynotify_la-common.o Distinct SYSROOTs are not something I expect to be used frequently but they are initially needed when cross-building a brand new system into an empty directory, which is exactly what cross-boss does. I hope this makes sense now. --=20 James Le Cuirot (chewi) Gentoo Linux Developer --Sig_//jBKsKiSuTS9s8WQFFl4XSz Content-Type: application/pgp-signature Content-Description: OpenPGP digital signature -----BEGIN PGP SIGNATURE----- iQIzBAEBCAAdFiEEPxcZ3tkwcedKm2a8EiZBXQDdMTcFAl0Bdy0ACgkQEiZBXQDd MTfw7g//UkDIvLhWexrj6NANfatBi0I48yscNRL93lQ04712tu4uAZdhKp05X2Ps 1uDOkr3KiXaymD2QvvgEcCUDDlLnzC0GkeGkCKfc4FumPz1D3L/PZPN9/Upjo8du QO46IOQ9qlSVX1HLazIOPWi1ATH4lnez57m66+tNMP9fXZigQl5hm+Zr6jimTo/u ZdMhBxIwO5/tWUp62yu2Zi98lZCgbnIi4vnPFdp/lnihi5AONJad6WoSUGyCMhr/ QlcYAq3YHhgUwRYgUuXhc+DazsPtDM4Pp7yDaDmE9SdM3XvOMXHlVed3XRem0Lpo bxzEUHBIcAo7OTTKNmEFfdGXjy99bnJ3p85t2rva8H5kLJdr0YZXI9gkjMZXwZ7/ oZshTIKZmOHYFceFv/aYaMtx7njO1AW2dNi7+yS8ZdFxiWQ7/fZe8ODIJP5pcVji 311h5USPJ2LE1+o/u7sZViDkZSb19KfKUBi9Q11C8RelXIjebviwqWygGtEKNfcx YRIlV04avn8flpKTEvgkrLOW9PpPV5UxSM+1D8NAGL6hDDn96I2Z5NQhpT277MZu 9NBqEs7xOKBP+uMKxoWmvkFmdODkKpplKCDxT+Z1udb3f1nvCRl9BdDHbCxUKjSG Ic4Q5wd8LpwHP9zXRQy1M7F6fjtmBmX3rSNbYkvD+jNZnwrWIkQ= =FUEu -----END PGP SIGNATURE----- --Sig_//jBKsKiSuTS9s8WQFFl4XSz--