[gentoo-dev] version/slot locked dependencies in eclasses like autotools.eclass and vala.eclass

[gentoo-dev] version/slot locked dependencies in eclasses like autotools.eclass and vala.eclass

[Zac Medico]

[-- Attachment #1.1: Type: text/plain, Size: 817 bytes --]

Hi,

In sys-app/portage-2.3.20, emerge now defaults to --dynamic-deps=n. This
means that unless people explicitly set
EMERGE_DEFAULT_OPTS="--dynamic-deps=y" they're going to have to rebuild
packages any time that the runtime dependencies of those packages need
to be updated. It's possible to trigger these rebuilds using the emerge
--changed-deps=y option.

Some eclasses like autotools.eclass and vala.eclass generate
version/slot locked dependencies that cause the dependencies of
inheriting ebuilds to change when the versions in the eclasses are
updated. If possible, it would be nice to avoid this version/slot
locking. If not possible, then what should be do?

Should we tell users to use the emerge --changed-deps=y option? Maybe
make --changed-deps=y a default setting?
-- 
Thanks,
Zac


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 224 bytes --]

Re: [gentoo-dev] version/slot locked dependencies in eclasses like autotools.eclass and vala.eclass

[Michael Orlitzky]

On 01/21/2018 11:24 PM, Zac Medico wrote:
> 
> Some eclasses like autotools.eclass and vala.eclass generate
> version/slot locked dependencies that cause the dependencies of
> inheriting ebuilds to change when the versions in the eclasses are
> updated. If possible, it would be nice to avoid this version/slot
> locking. If not possible, then what should be do?
> 

This changes the deps in stable ebuilds, and was already a no-no.

If the dependencies are to remain in the eclasses, then the eclasses
should get a new revision when those dependencies change. Afterwards,
the consumers can be revbumped and stabilized normally to utilize the
new eclass.



> Should we tell users to use the emerge --changed-deps=y option? Maybe
> make --changed-deps=y a default setting?
> 

Our tree shouldn't require a portage-only option to work. Besides, it's
better engineering to have the one person who makes the change alert the
rest of us. Having a million people poll "Did somebody change this? How
about now?" constantly is silly.

Re: [gentoo-dev] version/slot locked dependencies in eclasses like autotools.eclass and vala.eclass

[Zac Medico]

[-- Attachment #1.1: Type: text/plain, Size: 1305 bytes --]

On 01/21/2018 08:57 PM, Michael Orlitzky wrote:
> On 01/21/2018 11:24 PM, Zac Medico wrote:
>>
>> Some eclasses like autotools.eclass and vala.eclass generate
>> version/slot locked dependencies that cause the dependencies of
>> inheriting ebuilds to change when the versions in the eclasses are
>> updated. If possible, it would be nice to avoid this version/slot
>> locking. If not possible, then what should be do?
>>
> 
> This changes the deps in stable ebuilds, and was already a no-no.
> 
> If the dependencies are to remain in the eclasses, then the eclasses
> should get a new revision when those dependencies change. Afterwards,
> the consumers can be revbumped and stabilized normally to utilize the
> new eclass.

Sounds good!

>> Should we tell users to use the emerge --changed-deps=y option? Maybe
>> make --changed-deps=y a default setting?
>>
> 
> Our tree shouldn't require a portage-only option to work. Besides, it's
> better engineering to have the one person who makes the change alert the
> rest of us. Having a million people poll "Did somebody change this? How
> about now?" constantly is silly.

Okay, think I'll prepare a news item suggesting to use --changed-deps=y
if necessary for the initial transition to --dynamic-deps=n.
-- 
Thanks,
Zac


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 224 bytes --]

[gentoo-dev] Re: version/slot locked dependencies in eclasses like autotools.eclass and vala.eclass

[Duncan]

Zac Medico posted on Sun, 21 Jan 2018 22:20:21 -0800 as excerpted:

> On 01/21/2018 08:57 PM, Michael Orlitzky wrote:
>> On 01/21/2018 11:24 PM, Zac Medico wrote:
>>>
>>> Some eclasses like autotools.eclass and vala.eclass generate
>>> version/slot locked dependencies that cause the dependencies of
>>> inheriting ebuilds to change when the versions in the eclasses are
>>> updated. If possible, it would be nice to avoid this version/slot
>>> locking. If not possible, then what should be do?
>>>
>>>
>> This changes the deps in stable ebuilds, and was already a no-no.
>> 
>> If the dependencies are to remain in the eclasses, then the eclasses
>> should get a new revision when those dependencies change. Afterwards,
>> the consumers can be revbumped and stabilized normally to utilize the
>> new eclass.
> 
> Sounds good!

How does that work with live-vcs ebuilds, aka -9999 ebuilds?

To date I've seen very few if any -9999-rX ebuilds, I've assumed because 
they'll be rebuilt if the sources change anyway, and with @smart-live-
rebuild upstream changes are detected and trigger rebuilds automatically.

But now we're talking gentoo level dependency changes that either don't 
match a specific upstream change or that occur *after* the upstream 
change, so there may /be/ no timely upstream update to trigger a rebuild.

Does this then mean we should be getting -9999-rX revision bumps now, for 
gentoo level dependency changes?

If so, gentoo/kde's overlay... and I since I run live-git of nearly 
everything kde here... are in for quite some hundreds-of-packages-at-once 
fun when those eclass dep-bumps occur...

(Tho it can't be /that/ bad, since I've been running with --dynamic-deps=n 
for years now, and without --changed-deps for I'd guess a year or so 
now.  And upstream does mass version and dep bumps regularly already, 
triggering the mass rebuilds even if that's the only commit, so I suppose 
another triggered rebuild when gentoo/kde revbumps after dep-bumping to 
reflect what upstream already did, won't be /that/ much different or 
/that/ much more work.  Only now I guess I'll be seeing it in -9999-rX 
revbumps, too.)

-- 
Duncan - List replies preferred.   No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master."  Richard Stallman

Re: [gentoo-dev] version/slot locked dependencies in eclasses like autotools.eclass and vala.eclass

[Mart Raudsepp]

On Sun, 2018-01-21 at 20:24 -0800, Zac Medico wrote:
> Hi,
> 
> In sys-app/portage-2.3.20, emerge now defaults to --dynamic-deps=n.
> This
> means that unless people explicitly set
> EMERGE_DEFAULT_OPTS="--dynamic-deps=y" they're going to have to
> rebuild
> packages any time that the runtime dependencies of those packages
> need
> to be updated. It's possible to trigger these rebuilds using the
> emerge
> --changed-deps=y option.
> 
> Some eclasses like autotools.eclass and vala.eclass generate
> version/slot locked dependencies that cause the dependencies of
> inheriting ebuilds to change when the versions in the eclasses are
> updated. If possible, it would be nice to avoid this version/slot
> locking. If not possible, then what should be do?

These are mostly build time only depends, why should the user now all
of a sudden care before an unrelated rebuild or upgrade, which would
actually matter only then, not before?

Re: [gentoo-dev] Re: version/slot locked dependencies in eclasses like autotools.eclass and vala.eclass

[Michael Orlitzky]

On 01/22/2018 05:10 AM, Duncan wrote:
>>>
>>> If the dependencies are to remain in the eclasses, then the eclasses
>>> should get a new revision when those dependencies change. Afterwards,
>>> the consumers can be revbumped and stabilized normally to utilize the
>>> new eclass.
>>
>> Sounds good!
> 
> How does that work with live-vcs ebuilds, aka -9999 ebuilds?
> 

It doesn't. As with upstream code changes, you have to figure out
yourself when it's time to re-emerge a live ebuild.

Re: [gentoo-dev] version/slot locked dependencies in eclasses like autotools.eclass and vala.eclass

[Mike Gilbert]

On Sun, Jan 21, 2018 at 11:57 PM, Michael Orlitzky <mjo@gentoo.org> wrote:
> On 01/21/2018 11:24 PM, Zac Medico wrote:
>>
>> Some eclasses like autotools.eclass and vala.eclass generate
>> version/slot locked dependencies that cause the dependencies of
>> inheriting ebuilds to change when the versions in the eclasses are
>> updated. If possible, it would be nice to avoid this version/slot
>> locking. If not possible, then what should be do?
>>
>
> This changes the deps in stable ebuilds, and was already a no-no.
>
> If the dependencies are to remain in the eclasses, then the eclasses
> should get a new revision when those dependencies change. Afterwards,
> the consumers can be revbumped and stabilized normally to utilize the
> new eclass.

While that sounds like the right thing to do in theory, updating all
consumers of autotools.eclass whenever a new version of automake is
stabilized is really a lot of work for very little benefit.

Re: [gentoo-dev] version/slot locked dependencies in eclasses like autotools.eclass and vala.eclass

[Michał Górny]

W dniu nie, 21.01.2018 o godzinie 20∶24 -0800, użytkownik Zac Medico
napisał:
> Hi,
> 
> In sys-app/portage-2.3.20, emerge now defaults to --dynamic-deps=n. This
> means that unless people explicitly set
> EMERGE_DEFAULT_OPTS="--dynamic-deps=y" they're going to have to rebuild
> packages any time that the runtime dependencies of those packages need
> to be updated. It's possible to trigger these rebuilds using the emerge
> --changed-deps=y option.
> 
> Some eclasses like autotools.eclass and vala.eclass generate
> version/slot locked dependencies that cause the dependencies of
> inheriting ebuilds to change when the versions in the eclasses are
> updated. If possible, it would be nice to avoid this version/slot
> locking. If not possible, then what should be do?

[Disclaimer: this is what I recall from memory, it might not be correct]

I think this was discussed at the time when all the things were
discussed, and the conclusion was pretty much that there is no valid
reason an for eclass to have to retroactively update dependencies
in installed packages.

We can discuss this in greater detail once someone has a good use case
for this. However, FWICS the eclasses mentioned here were already
dismissed as build-time only dependencies.

> Should we tell users to use the emerge --changed-deps=y option? Maybe
> make --changed-deps=y a default setting?

No. The idea is that not all dependency changes need to be explicitly
propagated. The developer needs to weigh the pros and cons
of propagating the change, and choose wisely. There is really no need to
enforce a lot of unnecessarily frequent rebuilds because of minor
dependency correction that doesn't really apply to the user.

-- 
Best regards,
Michał Górny

Re: [gentoo-dev] version/slot locked dependencies in eclasses like autotools.eclass and vala.eclass

[Rich Freeman]

On Mon, Jan 22, 2018 at 11:57 AM, Michał Górny <mgorny@gentoo.org> wrote:
> W dniu nie, 21.01.2018 o godzinie 20∶24 -0800, użytkownik Zac Medico
>
>> Should we tell users to use the emerge --changed-deps=y option? Maybe
>> make --changed-deps=y a default setting?
>
> No. The idea is that not all dependency changes need to be explicitly
> propagated. The developer needs to weigh the pros and cons
> of propagating the change, and choose wisely. There is really no need to
> enforce a lot of unnecessarily frequent rebuilds because of minor
> dependency correction that doesn't really apply to the user.
>

I tend to agree, but one of the complications here is the break in
time between the error and the consequences.

If a dev commits a change and in six hours users start reporting
breakage, then the error is easily identified and fixed, and the dev
tends to learn not to do that again.

If a dev commits a change, and in 12 months users get weird breakage
when building or using other packages, then everybody runs around in
circles, the error is at least somewhat painful to identify, and the
dev has probably made the same error 5 other times since, if they're
even still maintaining that package.

A repoman warning would definitely help here.

Forcing unnecessary rebuilds isn't really the ideal solution, though
I'll admit I've been doing this for a while now since things were
getting painful a while back.

-- 
Rich

Re: [gentoo-dev] version/slot locked dependencies in eclasses like autotools.eclass and vala.eclass

[Michael Orlitzky]

On 01/22/2018 11:37 AM, Mike Gilbert wrote:
>>
>> If the dependencies are to remain in the eclasses, then the eclasses
>> should get a new revision when those dependencies change. Afterwards,
>> the consumers can be revbumped and stabilized normally to utilize the
>> new eclass.
> 
> While that sounds like the right thing to do in theory, updating all
> consumers of autotools.eclass whenever a new version of automake is
> stabilized is really a lot of work for very little benefit.

Is now a good time to reconsider why we're manually listing stable
versions of automake in autotools.eclass?

Re: [gentoo-dev] version/slot locked dependencies in eclasses like autotools.eclass and vala.eclass

[Zac Medico]

On 01/22/2018 05:14 AM, Mart Raudsepp wrote:
> On Sun, 2018-01-21 at 20:24 -0800, Zac Medico wrote:
>> Hi,
>>
>> In sys-app/portage-2.3.20, emerge now defaults to --dynamic-deps=n.
>> This
>> means that unless people explicitly set
>> EMERGE_DEFAULT_OPTS="--dynamic-deps=y" they're going to have to
>> rebuild
>> packages any time that the runtime dependencies of those packages
>> need
>> to be updated. It's possible to trigger these rebuilds using the
>> emerge
>> --changed-deps=y option.
>>
>> Some eclasses like autotools.eclass and vala.eclass generate
>> version/slot locked dependencies that cause the dependencies of
>> inheriting ebuilds to change when the versions in the eclasses are
>> updated. If possible, it would be nice to avoid this version/slot
>> locking. If not possible, then what should be do?
> 
> These are mostly build time only depends, why should the user now all
> of a sudden care before an unrelated rebuild or upgrade, which would
> actually matter only then, not before?

For various reasons, current versions of portage enable the
--with-bdeps=y option by default [1]. Basically, failing to update
installed packages and possibly leaving them with broken dependencies is
not really a sane default behavior.

[1] https://bugs.gentoo.org/598444
-- 
Thanks,
Zac

[gentoo-dev] Re: version/slot locked dependencies in eclasses like autotools.eclass and vala.eclass

[Duncan]

Michael Orlitzky posted on Mon, 22 Jan 2018 10:04:30 -0500 as excerpted:

> On 01/22/2018 05:10 AM, Duncan wrote:
>>>>
>>>> If the dependencies are to remain in the eclasses, then the eclasses
>>>> should get a new revision when those dependencies change. Afterwards,
>>>> the consumers can be revbumped and stabilized normally to utilize the
>>>> new eclass.
>>>
>>> Sounds good!
>> 
>> How does that work with live-vcs ebuilds, aka -9999 ebuilds?
>> 
>> 
> It doesn't. As with upstream code changes, you have to figure out
> yourself when it's time to re-emerge a live ebuild.

Thanks for the confirmation.

Hmm... I wonder if @smart-live-rebuild could (without /too/ much trouble) 
be made to detect upgraded deps, comparing the live repo version against 
what's in /var/db/pkg/, as it already does for upstream changes?

If it already has the feature I've not seen any indication of it from the 
output.  All the updates I've seen appear to be due to upstream repo 
updates.

-- 
Duncan - List replies preferred.   No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master."  Richard Stallman

Re: [gentoo-dev] version/slot locked dependencies in eclasses like autotools.eclass and vala.eclass

[Mike Gilbert]

On Mon, Jan 22, 2018 at 12:34 PM, Michael Orlitzky <mjo@gentoo.org> wrote:
> On 01/22/2018 11:37 AM, Mike Gilbert wrote:
>>>
>>> If the dependencies are to remain in the eclasses, then the eclasses
>>> should get a new revision when those dependencies change. Afterwards,
>>> the consumers can be revbumped and stabilized normally to utilize the
>>> new eclass.
>>
>> While that sounds like the right thing to do in theory, updating all
>> consumers of autotools.eclass whenever a new version of automake is
>> stabilized is really a lot of work for very little benefit.
>
> Is now a good time to reconsider why we're manually listing stable
> versions of automake in autotools.eclass?

Yeah, it seems like we might be able to replace it with an unbounded dependency.

For the purpose of automake-wrapper, we might set an environment
variable using the output of "best_version dev-utils/automake".

Re: [gentoo-dev] version/slot locked dependencies in eclasses like autotools.eclass and vala.eclass

[Mart Raudsepp]

On Mon, 2018-01-22 at 12:07 -0800, Zac Medico wrote:
> On 01/22/2018 05:14 AM, Mart Raudsepp wrote:
> > On Sun, 2018-01-21 at 20:24 -0800, Zac Medico wrote:
> > > Hi,
> > > 
> > > In sys-app/portage-2.3.20, emerge now defaults to --dynamic-
> > > deps=n.
> > > This
> > > means that unless people explicitly set
> > > EMERGE_DEFAULT_OPTS="--dynamic-deps=y" they're going to have to
> > > rebuild
> > > packages any time that the runtime dependencies of those packages
> > > need
> > > to be updated. It's possible to trigger these rebuilds using the
> > > emerge
> > > --changed-deps=y option.
> > > 
> > > Some eclasses like autotools.eclass and vala.eclass generate
> > > version/slot locked dependencies that cause the dependencies of
> > > inheriting ebuilds to change when the versions in the eclasses
> > > are
> > > updated. If possible, it would be nice to avoid this version/slot
> > > locking. If not possible, then what should be do?
> > 
> > These are mostly build time only depends, why should the user now
> > all
> > of a sudden care before an unrelated rebuild or upgrade, which
> > would
> > actually matter only then, not before?
> 
> For various reasons, current versions of portage enable the
> --with-bdeps=y option by default [1]. Basically, failing to update
> installed packages and possibly leaving them with broken dependencies
> is
> not really a sane default behavior.
> 
> [1] https://bugs.gentoo.org/598444

Sure, and now in combination with --with-bdeps=y and --dynamic-deps=n,
thing are bad for these cases. I'm saying that users shouldn't have to
care about these cases.

That said, I'm not sure why the slot deps have to be there for the
cases $vala_depend is put into DEPEND; those might just be able to be
an unbounded dev-lang/vala. However where they are truly needed in
RDEPEND, they will need something here, just not sure := is going to
cut it. Maybe the interface is stable enough that anything will be fine
with the newest version by now, but not sure. Bottom-line is, we aren't
going to be revbumping all the vala.eclass users as a new GNOME version
with new vala appears. The idea is to get everyone to use the new
versions in stable ASAP, not rebuild all of them in stable first.
In any case, this will need investigation, and it is not a good time to
have such an investigation forced upon us with our current limited
manpower.

Re: [gentoo-dev] version/slot locked dependencies in eclasses like autotools.eclass and vala.eclass

[Zac Medico]

[-- Attachment #1.1: Type: text/plain, Size: 3278 bytes --]

On 01/23/2018 01:06 AM, Mart Raudsepp wrote:
> On Mon, 2018-01-22 at 12:07 -0800, Zac Medico wrote:
>> On 01/22/2018 05:14 AM, Mart Raudsepp wrote:
>>> On Sun, 2018-01-21 at 20:24 -0800, Zac Medico wrote:
>>>> Hi,
>>>>
>>>> In sys-app/portage-2.3.20, emerge now defaults to --dynamic-
>>>> deps=n.
>>>> This
>>>> means that unless people explicitly set
>>>> EMERGE_DEFAULT_OPTS="--dynamic-deps=y" they're going to have to
>>>> rebuild
>>>> packages any time that the runtime dependencies of those packages
>>>> need
>>>> to be updated. It's possible to trigger these rebuilds using the
>>>> emerge
>>>> --changed-deps=y option.
>>>>
>>>> Some eclasses like autotools.eclass and vala.eclass generate
>>>> version/slot locked dependencies that cause the dependencies of
>>>> inheriting ebuilds to change when the versions in the eclasses
>>>> are
>>>> updated. If possible, it would be nice to avoid this version/slot
>>>> locking. If not possible, then what should be do?
>>>
>>> These are mostly build time only depends, why should the user now
>>> all
>>> of a sudden care before an unrelated rebuild or upgrade, which
>>> would
>>> actually matter only then, not before?
>>
>> For various reasons, current versions of portage enable the
>> --with-bdeps=y option by default [1]. Basically, failing to update
>> installed packages and possibly leaving them with broken dependencies
>> is
>> not really a sane default behavior.
>>
>> [1] https://bugs.gentoo.org/598444
> 
> Sure, and now in combination with --with-bdeps=y and --dynamic-deps=n,
> thing are bad for these cases. I'm saying that users shouldn't have to
> care about these cases.
> 
> That said, I'm not sure why the slot deps have to be there for the
> cases $vala_depend is put into DEPEND; those might just be able to be
> an unbounded dev-lang/vala. However where they are truly needed in
> RDEPEND, they will need something here, just not sure := is going to
> cut it. Maybe the interface is stable enough that anything will be fine
> with the newest version by now, but not sure. Bottom-line is, we aren't
> going to be revbumping all the vala.eclass users as a new GNOME version
> with new vala appears. The idea is to get everyone to use the new
> versions in stable ASAP, not rebuild all of them in stable first.
> In any case, this will need investigation, and it is not a good time to
> have such an investigation forced upon us with our current limited
> manpower.

Is it so bad if users have to become accustomed to using
--changed-deps=y for some time, until we get things sorted out? I feel
like there is never going to be a time when everyone is ready for this
all at once, and defaulting --dynamic-deps=n serves as a way to prevent
these issues from being entirely forgotten about.

For vala_depend, maybe something like this works:

VALA_MIN_API_VERSION="0.32" translates to >=dev-lang/vala-0.32
VALA_MAX_API_VERSION="0.36" translates to <dev-lang/vala-0.37

Both VALA_MIN_API_VERSION and VALA_MAX_API_VERSION unset translates to
simply to dev-lang/vala.

For vala_best_api_version, maybe use best_version, with atoms generated
using VALA_MIN_API_VERSION and VALA_MAX_API_VERSION if the ebuild set them.
-- 
Thanks,
Zac


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 224 bytes --]