* [gentoo-dev] Multilib dependencies need to have >= on least version having EAPI=5 or multilib support
@ 2014-06-18 22:13 Michał Górny
2014-06-19 15:26 ` Sven Vermeulen
2014-06-19 19:21 ` Greg Turner
0 siblings, 2 replies; 5+ messages in thread
From: Michał Górny @ 2014-06-18 22:13 UTC (permalink / raw
To: gentoo-dev
[-- Attachment #1: Type: text/plain, Size: 5395 bytes --]
Hi, developers and users.
TLDR: if you do [${MULTILIB_USEDEP}] or [abi_x86_32], always use >= dep
on the oldest package version that is EAPI=5 or multilib. In other
words, the >= dependency must not match a single non-multilib, EAPI<5
ebuild. And please do not commit new EAPI<5 ebuilds following EAPI=5
ebuilds.
Long version:
First of all, I'd like to thank all supporters of our little multilib
effort. We're getting pretty close to finishing stage 1 of our roadmap
[1], with most of important packages being multilib-ready already [2].
Most of our effort right now is focused on fixing 32-bit package
dependencies [3].
However, it was lately brought to my attention that the very complex
issue regarding portage behavior wrt USE_EXPAND flags and EAPI<5. Since
PMS is awfully incomprehensible in this area, I won't get into the fine
details. The same issue was discussed wrt Python in bug #446720 [4].
The important part is that a dependency of the following form:
dev-foo/bar[abi_x86_64(-),abi_x86_32(-)...]
can be satisfied not only by multilib ebuilds. It is also satisfied by
EAPI<5 ebuilds that don't have abi_* flags in IUSE if the user has
enabled respective abi_* flags globally. In other words, if the user
has:
ABI_X86="32 64"
in his make.conf, portage thinks that all EAPI<5 packages (both ebuilds
and installed ones) instantly started supporting multilib.
This has a few important implications:
1. when trying to install (upgrade) a multilib package, portage may not
upgrade its dependencies to proper multilib versions, therefore
triggering seemingly random failures or limitations,
2. when trying to stabilize a multilib package without stabilizing
multilib versions of its dependencies, repoman may not complain if
there is at least a single EAPI<5 version available,
3. when such a stabilization happens, the upgrade of this package on
stable arch will cause emerge to downgrade the dependency to EAPI<5
version.
While we were roughly aware of the first implication, we were not of
the latter two. This was specifically brought to us by _AxS_ who
keyworded multilib ffmpeg on stable amd64 (without keywording libsdl)
and noticed that this causes portage to downgrade EAPI=5, non-multilib
libsdl-1.2.15-r4 to EAPI=2 libsdl-1.2.15-r2. A similar issue was also
reported on the Forums [5].
We've discussed this with the PMS team and the Portage team, and it
seems that the only safe and portable way of fixing this is through
using proper >= dependencies. More specifically, making sure that all
dependencies on multilib packages require proper versions so that no
EAPI<5, non-multilib package can satisfy it.
We've decided to satisfy this requirement through using dependencies of
the following form:
>=dev-foo/bar-${MIN_PV}[${MULTILIB_USEDEP}]
where ${MIN_PV} corresponds to a version that is at least the lower of
the following two versions:
a) the lowest version of the package that is EAPI=5 and
is not followed by any EAPI<5 ebuild,
b) the lowest version of the package that supports multilib and is not
followed by any non-multilib ebuild.
While a) may seem unnecessary considering b), it decreases the number
of stabilizations necessary. It also makes it possible to backport
the multilib support without adjusting deps. However, it relies on
developers not committing new EAPI<5 ebuilds following EAPI=5 ebuilds,
or new non-multilib ebuilds following multilib ebuilds.
Since adding proper versions for every single dependency would be
tedious, we provide multilib-dep-fixor [6] to help with that. The usage
is pretty simple:
multilib-dep-fixor *.ebuild
and it tries to rewrite all multilib dependencies into proper >=-deps.
However, it has a few limitations. More specifically:
a) it doesn't handle variable substitution (${PV}, ${SLOT} in versions),
b) it doesn't handle <, <=, = and ~ deps,
c) it doesn't handle the cases when the newest ebuild is non-multilib
and EAPI<5 (for example, rotting masked or live ebuild),
d) it doesn't look into the past and takes the safe side. So if
the oldest available ebuild is EAPI=5 or multilib, it adds >= dep
on that version.
I've committed dependency corrections to most of the gentoo-x86 ebuilds
today. Please note that this may cause some of the open stablereqs to
require new packages being stabilized. The issues related to the semi-
automatic fixing can be reported in bug #513718 [7].
I will be committing the remaining dependency corrections as soon as it
becomes possible -- that is, as soon as required dependencies are
stabilized. The stable packages with missing stable dependencies are
tracked in bug #507148 [8].
Thank you for your cooperation. If you have any questions, please do
not hesitate to ask.
[1]:https://wiki.gentoo.org/wiki/Project:Multilib/RoadMap
[2]:https://wiki.gentoo.org/wiki/Multilib_porting_status#Emul-.2A_Package_Porting_Overview
[3]:https://wiki.gentoo.org/wiki/Multilib_porting_status#Dependency_update_status
[4]:https://bugs.gentoo.org/show_bug.cgi?id=488776
[5]:https://forums.gentoo.org/viewtopic-t-993228-highlight-.html
[6]:https://bitbucket.org/mgorny/multilib-dep-fixor/src
[7]:https://bugs.gentoo.org/show_bug.cgi?id=513718
[8]:https://bugs.gentoo.org/show_bug.cgi?id=507148
--
Best regards,
Michał Górny
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 949 bytes --]
^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: [gentoo-dev] Multilib dependencies need to have >= on least version having EAPI=5 or multilib support
2014-06-18 22:13 [gentoo-dev] Multilib dependencies need to have >= on least version having EAPI=5 or multilib support Michał Górny
@ 2014-06-19 15:26 ` Sven Vermeulen
2014-06-19 19:21 ` Greg Turner
1 sibling, 0 replies; 5+ messages in thread
From: Sven Vermeulen @ 2014-06-19 15:26 UTC (permalink / raw
To: gentoo-dev
On Thu, Jun 19, 2014 at 12:13:12AM +0200, Michał Górny wrote:
> Hi, developers and users.
[...]
> Thank you for your cooperation. If you have any questions, please do
> not hesitate to ask.
Hi Michał
Thank you for your endless effort to get Gentoo to this stage (and further).
Wkr,
Sven Vermeulen
PS I'm in a positive mood
^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: [gentoo-dev] Multilib dependencies need to have >= on least version having EAPI=5 or multilib support
2014-06-18 22:13 [gentoo-dev] Multilib dependencies need to have >= on least version having EAPI=5 or multilib support Michał Górny
2014-06-19 15:26 ` Sven Vermeulen
@ 2014-06-19 19:21 ` Greg Turner
2014-06-19 20:02 ` Ciaran McCreesh
1 sibling, 1 reply; 5+ messages in thread
From: Greg Turner @ 2014-06-19 19:21 UTC (permalink / raw
To: gentoo-dev
Ouch!
I suspect a nice side-effect of this policy is that portage will be
faster as it will be less prone to backtracking.
One thing people should be aware of is how this policy will interact
with slotted ebuilds. >=foo/bar-3.5[${MULTILIB_USEDEP}] could be met
by the ancient >=foo/bar-4.0 EAPI0 ebuild, but a dep like:
"|| (
>=foo/bar-3.5:3[${MULTILIB_USEDEP}]
>=foo/bar-4.5:4[${MULTILIB_USEDEP}]
>=foo/bar-5.2:5[${MULTILIB_USEDEP}]
)"
would solve such a problem correctly.
Also, what Sven said! Thanks for your Herculean stick-to-it-iveness,
Michał, and thanks to the others on the multilib team, and the many
who have contributed to the multilib-utilization effort via bugzilla
and IRC! It seems that we are nearing the day when the emul-linux-x86
ebuilds will be empty shells or even a thing of the past.
On Wed, Jun 18, 2014 at 3:13 PM, Michał Górny <mgorny@gentoo.org> wrote:
> Hi, developers and users.
>
> TLDR: if you do [${MULTILIB_USEDEP}] or [abi_x86_32], always use >= dep
> on the oldest package version that is EAPI=5 or multilib. In other
> words, the >= dependency must not match a single non-multilib, EAPI<5
> ebuild. And please do not commit new EAPI<5 ebuilds following EAPI=5
> ebuilds.
>
> Long version:
>
> First of all, I'd like to thank all supporters of our little multilib
> effort. We're getting pretty close to finishing stage 1 of our roadmap
> [1], with most of important packages being multilib-ready already [2].
> Most of our effort right now is focused on fixing 32-bit package
> dependencies [3].
>
> However, it was lately brought to my attention that the very complex
> issue regarding portage behavior wrt USE_EXPAND flags and EAPI<5. Since
> PMS is awfully incomprehensible in this area, I won't get into the fine
> details. The same issue was discussed wrt Python in bug #446720 [4].
>
> The important part is that a dependency of the following form:
>
> dev-foo/bar[abi_x86_64(-),abi_x86_32(-)...]
>
> can be satisfied not only by multilib ebuilds. It is also satisfied by
> EAPI<5 ebuilds that don't have abi_* flags in IUSE if the user has
> enabled respective abi_* flags globally. In other words, if the user
> has:
>
> ABI_X86="32 64"
>
> in his make.conf, portage thinks that all EAPI<5 packages (both ebuilds
> and installed ones) instantly started supporting multilib.
>
>
> This has a few important implications:
>
> 1. when trying to install (upgrade) a multilib package, portage may not
> upgrade its dependencies to proper multilib versions, therefore
> triggering seemingly random failures or limitations,
>
> 2. when trying to stabilize a multilib package without stabilizing
> multilib versions of its dependencies, repoman may not complain if
> there is at least a single EAPI<5 version available,
>
> 3. when such a stabilization happens, the upgrade of this package on
> stable arch will cause emerge to downgrade the dependency to EAPI<5
> version.
>
> While we were roughly aware of the first implication, we were not of
> the latter two. This was specifically brought to us by _AxS_ who
> keyworded multilib ffmpeg on stable amd64 (without keywording libsdl)
> and noticed that this causes portage to downgrade EAPI=5, non-multilib
> libsdl-1.2.15-r4 to EAPI=2 libsdl-1.2.15-r2. A similar issue was also
> reported on the Forums [5].
>
>
> We've discussed this with the PMS team and the Portage team, and it
> seems that the only safe and portable way of fixing this is through
> using proper >= dependencies. More specifically, making sure that all
> dependencies on multilib packages require proper versions so that no
> EAPI<5, non-multilib package can satisfy it.
>
> We've decided to satisfy this requirement through using dependencies of
> the following form:
>
> >=dev-foo/bar-${MIN_PV}[${MULTILIB_USEDEP}]
>
> where ${MIN_PV} corresponds to a version that is at least the lower of
> the following two versions:
>
> a) the lowest version of the package that is EAPI=5 and
> is not followed by any EAPI<5 ebuild,
>
> b) the lowest version of the package that supports multilib and is not
> followed by any non-multilib ebuild.
>
> While a) may seem unnecessary considering b), it decreases the number
> of stabilizations necessary. It also makes it possible to backport
> the multilib support without adjusting deps. However, it relies on
> developers not committing new EAPI<5 ebuilds following EAPI=5 ebuilds,
> or new non-multilib ebuilds following multilib ebuilds.
>
>
> Since adding proper versions for every single dependency would be
> tedious, we provide multilib-dep-fixor [6] to help with that. The usage
> is pretty simple:
>
> multilib-dep-fixor *.ebuild
>
> and it tries to rewrite all multilib dependencies into proper >=-deps.
> However, it has a few limitations. More specifically:
>
> a) it doesn't handle variable substitution (${PV}, ${SLOT} in versions),
>
> b) it doesn't handle <, <=, = and ~ deps,
>
> c) it doesn't handle the cases when the newest ebuild is non-multilib
> and EAPI<5 (for example, rotting masked or live ebuild),
>
> d) it doesn't look into the past and takes the safe side. So if
> the oldest available ebuild is EAPI=5 or multilib, it adds >= dep
> on that version.
>
>
> I've committed dependency corrections to most of the gentoo-x86 ebuilds
> today. Please note that this may cause some of the open stablereqs to
> require new packages being stabilized. The issues related to the semi-
> automatic fixing can be reported in bug #513718 [7].
>
> I will be committing the remaining dependency corrections as soon as it
> becomes possible -- that is, as soon as required dependencies are
> stabilized. The stable packages with missing stable dependencies are
> tracked in bug #507148 [8].
>
>
> Thank you for your cooperation. If you have any questions, please do
> not hesitate to ask.
>
> [1]:https://wiki.gentoo.org/wiki/Project:Multilib/RoadMap
> [2]:https://wiki.gentoo.org/wiki/Multilib_porting_status#Emul-.2A_Package_Porting_Overview
> [3]:https://wiki.gentoo.org/wiki/Multilib_porting_status#Dependency_update_status
> [4]:https://bugs.gentoo.org/show_bug.cgi?id=488776
> [5]:https://forums.gentoo.org/viewtopic-t-993228-highlight-.html
> [6]:https://bitbucket.org/mgorny/multilib-dep-fixor/src
> [7]:https://bugs.gentoo.org/show_bug.cgi?id=513718
> [8]:https://bugs.gentoo.org/show_bug.cgi?id=507148
>
> --
> Best regards,
> Michał Górny
^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: [gentoo-dev] Multilib dependencies need to have >= on least version having EAPI=5 or multilib support
2014-06-19 19:21 ` Greg Turner
@ 2014-06-19 20:02 ` Ciaran McCreesh
2014-06-19 20:29 ` Greg Turner
0 siblings, 1 reply; 5+ messages in thread
From: Ciaran McCreesh @ 2014-06-19 20:02 UTC (permalink / raw
To: gentoo-dev
[-- Attachment #1: Type: text/plain, Size: 550 bytes --]
On Thu, 19 Jun 2014 12:21:35 -0700
Greg Turner <gmt@malth.us> wrote:
> One thing people should be aware of is how this policy will interact
> with slotted ebuilds. >=foo/bar-3.5[${MULTILIB_USEDEP}] could be met
> by the ancient >=foo/bar-4.0 EAPI0 ebuild, but a dep like:
>
> "|| (
> >=foo/bar-3.5:3[${MULTILIB_USEDEP}]
> >=foo/bar-4.5:4[${MULTILIB_USEDEP}]
> >=foo/bar-5.2:5[${MULTILIB_USEDEP}]
> )"
>
> would solve such a problem correctly.
No it wouldn't... Best version first in a || block.
--
Ciaran McCreesh
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 198 bytes --]
^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: [gentoo-dev] Multilib dependencies need to have >= on least version having EAPI=5 or multilib support
2014-06-19 20:02 ` Ciaran McCreesh
@ 2014-06-19 20:29 ` Greg Turner
0 siblings, 0 replies; 5+ messages in thread
From: Greg Turner @ 2014-06-19 20:29 UTC (permalink / raw
To: gentoo-dev
On Thu, Jun 19, 2014 at 1:02 PM, Ciaran McCreesh
<ciaran.mccreesh@googlemail.com> wrote:
>> "|| (
>> >=foo/bar-3.5:3[${MULTILIB_USEDEP}]
>> >=foo/bar-4.5:4[${MULTILIB_USEDEP}]
>> >=foo/bar-5.2:5[${MULTILIB_USEDEP}]
>> )"
>>
>> would solve such a problem correctly.
>
> No it wouldn't... Best version first in a || block.
Yes, good point, didn't even think of it. Man, that is some pretty
subtle business. Especially if "best" might change over time...
perhaps in such circumstances it would be better to create a virtual?
^ permalink raw reply [flat|nested] 5+ messages in thread
end of thread, other threads:[~2014-06-19 20:29 UTC | newest]
Thread overview: 5+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2014-06-18 22:13 [gentoo-dev] Multilib dependencies need to have >= on least version having EAPI=5 or multilib support Michał Górny
2014-06-19 15:26 ` Sven Vermeulen
2014-06-19 19:21 ` Greg Turner
2014-06-19 20:02 ` Ciaran McCreesh
2014-06-19 20:29 ` Greg Turner
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox