[gentoo-dev] News Item: Portage Dynamic Deps

[gentoo-dev] News Item: Portage Dynamic Deps

[Zac Medico]

[-- Attachment #1.1.1: Type: text/plain, Size: 1148 bytes --]

Please review.

Title: Portage Dynamic Deps
Author: Zac Medico <zmedico@gentoo.org>
Posted: 2018-01-28
Revision: 1
News-Item-Format: 2.0
Display-If-Installed: <sys-apps/portage-2.3.20

Beginning with Portage 2.3.20, the previous default --dynamic-deps=y
setting has changed to --dynamic-deps=n. Due to this change, some users
may experience emerge dependency calculation failures triggered by
installed packages that have outdated dependencies. In order to avoid
problems of this nature, use the emerge --changed-deps=y option with
your next deep @world update.

Here is an example command that will trigger rebuilds of any installed
packages that have outdated dependencies:

# emerge --update --deep --changed-deps=y @world

According to Gentoo policy, future ebuild dependency changes need to be
accompanied by a revision bump in order to trigger rebuilds for users.
Therefore, you should only need to use --changed-deps=y for a single
deep @world update. After that, if you encounter installed packages with
outdated dependencies in a future deep @world update, then you should
report it as a bug.
-- 
Thanks,
Zac

[-- Attachment #1.1.2: 2018-01-28-portage-dynamic-deps.en.txt --]
[-- Type: text/plain, Size: 1086 bytes --]

Title: Portage Dynamic Deps
Author: Zac Medico <zmedico@gentoo.org>
Posted: 2018-01-28
Revision: 1
News-Item-Format: 2.0
Display-If-Installed: <sys-apps/portage-2.3.20

Beginning with Portage 2.3.20, the previous default --dynamic-deps=y
setting has changed to --dynamic-deps=n. Due to this change, some users
may experience emerge dependency calculation failures triggered by
installed packages that have outdated dependencies. In order to avoid
problems of this nature, use the emerge --changed-deps=y option with
your next deep @world update.

Here is an example command that will trigger rebuilds of any installed
packages that have outdated dependencies:

# emerge --update --deep --changed-deps=y @world

According to Gentoo policy, future ebuild dependency changes need to be
accompanied by a revision bump in order to trigger rebuilds for users.
Therefore, you should only need to use --changed-deps=y for a single
deep @world update. After that, if you encounter installed packages with
outdated dependencies in a future deep @world update, then you should
report it as a bug.

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 224 bytes --]

Re: [gentoo-dev] News Item: Portage Dynamic Deps

[Alexis Ballier]

On Sun, 21 Jan 2018 23:01:08 -0800
Zac Medico <zmedico@gentoo.org> wrote:

> Please review.
> 
> Title: Portage Dynamic Deps
> Author: Zac Medico <zmedico@gentoo.org>
> Posted: 2018-01-28
> Revision: 1
> News-Item-Format: 2.0
> Display-If-Installed: <sys-apps/portage-2.3.20
> 
> Beginning with Portage 2.3.20, the previous default --dynamic-deps=y
> setting has changed to --dynamic-deps=n. Due to this change, some
> users may experience emerge dependency calculation failures triggered
> by installed packages that have outdated dependencies. In order to
> avoid problems of this nature, use the emerge --changed-deps=y option
> with your next deep @world update.

What's the rationale behind this ?

What I mean is: while '--dynamic-deps=n --changed-deps=n' is the
technically correct behavior, this just seems like throwing unbearable
dep calculation failure messages at users' faces while we could default
to '--dynamic-deps=n --changed-deps=y' and get the already
policy-mandated behavior of 'force a rebuild when you change deps'.

Re: [gentoo-dev] News Item: Portage Dynamic Deps

[Zac Medico]

[-- Attachment #1.1: Type: text/plain, Size: 1327 bytes --]

On 01/22/2018 12:24 AM, Alexis Ballier wrote:
> On Sun, 21 Jan 2018 23:01:08 -0800
> Zac Medico <zmedico@gentoo.org> wrote:
> 
>> Please review.
>>
>> Title: Portage Dynamic Deps
>> Author: Zac Medico <zmedico@gentoo.org>
>> Posted: 2018-01-28
>> Revision: 1
>> News-Item-Format: 2.0
>> Display-If-Installed: <sys-apps/portage-2.3.20
>>
>> Beginning with Portage 2.3.20, the previous default --dynamic-deps=y
>> setting has changed to --dynamic-deps=n. Due to this change, some
>> users may experience emerge dependency calculation failures triggered
>> by installed packages that have outdated dependencies. In order to
>> avoid problems of this nature, use the emerge --changed-deps=y option
>> with your next deep @world update.
> 
> What's the rationale behind this ?
> 
> What I mean is: while '--dynamic-deps=n --changed-deps=n' is the
> technically correct behavior, this just seems like throwing unbearable
> dep calculation failure messages at users' faces while we could default
> to '--dynamic-deps=n --changed-deps=y' and get the already
> policy-mandated behavior of 'force a rebuild when you change deps'.

It depends on how difficult it is to enforce the revision bumps. If it's
really difficult, then it does make sense to enable --changed-deps=y by
default.
-- 
Thanks,
Zac


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 224 bytes --]

Re: [gentoo-dev] News Item: Portage Dynamic Deps

[Dirkjan Ochtman]

[-- Attachment #1: Type: text/plain, Size: 553 bytes --]

On Mon, Jan 22, 2018 at 8:01 AM, Zac Medico <zmedico@gentoo.org> wrote:

> According to Gentoo policy, future ebuild dependency changes need to be
> accompanied by a revision bump in order to trigger rebuilds for users.
> Therefore, you should only need to use --changed-deps=y for a single
> deep @world update. After that, if you encounter installed packages with
> outdated dependencies in a future deep @world update, then you should
> report it as a bug.


Can we get repoman to warn about changing dependencies without revbumps?

Cheers,

Dirkjan

[-- Attachment #2: Type: text/html, Size: 955 bytes --]

Re: [gentoo-dev] News Item: Portage Dynamic Deps

[Zac Medico]

[-- Attachment #1.1: Type: text/plain, Size: 974 bytes --]

On 01/22/2018 01:38 AM, Dirkjan Ochtman wrote:
> On Mon, Jan 22, 2018 at 8:01 AM, Zac Medico <zmedico@gentoo.org
> <mailto:zmedico@gentoo.org>> wrote:
> 
>     According to Gentoo policy, future ebuild dependency changes need to be
>     accompanied by a revision bump in order to trigger rebuilds for users.
>     Therefore, you should only need to use --changed-deps=y for a single
>     deep @world update. After that, if you encounter installed packages with
>     outdated dependencies in a future deep @world update, then you should
>     report it as a bug.
> 
> 
> Can we get repoman to warn about changing dependencies without revbumps?

That will require a bit of work since it's going to have to call bash to
generate metadata for the previous state and compare that to the current
state, but it can be done.

That won't help for eclass changes, though, unless we generate metadata
for all consumers and that's expensive.
-- 
Thanks,
Zac


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 224 bytes --]

Re: [gentoo-dev] News Item: Portage Dynamic Deps

[Andreas K. Huettel]

[-- Attachment #1: Type: text/plain, Size: 771 bytes --]

Am Montag, 22. Januar 2018, 08:01:08 CET schrieb Zac Medico:
>
> According to Gentoo policy, future ebuild dependency changes need to be
> accompanied by a revision bump in order to trigger rebuilds for users.
> Therefore, you should only need to use --changed-deps=y for a single
> deep @world update. After that, if you encounter installed packages with
> outdated dependencies in a future deep @world update, then you should
> report it as a bug.

Did you come up with a solution how to handle eclass-generated dependency 
changes then?

I'd loathe to have to do identical revision bumps for, say, all perl-
module.eclass consumers...

-- 
Andreas K. Hüttel
dilfridge@gentoo.org
Gentoo Linux developer
(council, toolchain, perl, libreoffice, comrel)

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 981 bytes --]

Re: [gentoo-dev] News Item: Portage Dynamic Deps

[James Le Cuirot]

On Mon, 22 Jan 2018 11:28:21 +0100
"Andreas K. Huettel" <dilfridge@gentoo.org> wrote:

> Am Montag, 22. Januar 2018, 08:01:08 CET schrieb Zac Medico:
> >
> > According to Gentoo policy, future ebuild dependency changes need
> > to be accompanied by a revision bump in order to trigger rebuilds
> > for users. Therefore, you should only need to use --changed-deps=y
> > for a single deep @world update. After that, if you encounter
> > installed packages with outdated dependencies in a future deep
> > @world update, then you should report it as a bug.  
> 
> Did you come up with a solution how to handle eclass-generated
> dependency changes then?
> 
> I'd loathe to have to do identical revision bumps for, say, all perl-
> module.eclass consumers...

Isn't there a rule about allowing existing dependency version bumps? I
can't remember exactly how it went. Something about subsets. ;)

-- 
James Le Cuirot (chewi)
Gentoo Linux Developer

Re: [gentoo-dev] News Item: Portage Dynamic Deps

[Ian Stakenvicius]

[-- Attachment #1.1: Type: text/plain, Size: 1005 bytes --]

On 2018-01-22 05:28 AM, Andreas K. Huettel wrote:
> Am Montag, 22. Januar 2018, 08:01:08 CET schrieb Zac Medico:
>>
>> According to Gentoo policy, future ebuild dependency changes need to be
>> accompanied by a revision bump in order to trigger rebuilds for users.
>> Therefore, you should only need to use --changed-deps=y for a single
>> deep @world update. After that, if you encounter installed packages with
>> outdated dependencies in a future deep @world update, then you should
>> report it as a bug.
> 
> Did you come up with a solution how to handle eclass-generated dependency 
> changes then?
> 
> I'd loathe to have to do identical revision bumps for, say, all perl-
> module.eclass consumers...
> 

Based on what I'm seeing in perl-module.eclass now I don't think this
is an issue -- first, perl isn't slotted; second, perl doesn't specify
${PV} information in the eclass.  If it started to do either of these,
then yes revbumps are going to start being necessary..



[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 248 bytes --]

Re: [gentoo-dev] News Item: Portage Dynamic Deps

[Ciaran McCreesh]

On Mon, 22 Jan 2018 11:28:21 +0100
"Andreas K. Huettel" <dilfridge@gentoo.org> wrote:
> Am Montag, 22. Januar 2018, 08:01:08 CET schrieb Zac Medico:
> > According to Gentoo policy, future ebuild dependency changes need
> > to be accompanied by a revision bump in order to trigger rebuilds
> > for users. Therefore, you should only need to use --changed-deps=y
> > for a single deep @world update. After that, if you encounter
> > installed packages with outdated dependencies in a future deep
> > @world update, then you should report it as a bug.  
> 
> Did you come up with a solution how to handle eclass-generated
> dependency changes then?
> 
> I'd loathe to have to do identical revision bumps for, say, all perl-
> module.eclass consumers...

perl-cleaner is rather strong evidence that the Perl situation needs a
major change anyway...

-- 
Ciaran McCreesh

Re: [gentoo-dev] News Item: Portage Dynamic Deps

[Kent Fredric]

[-- Attachment #1: Type: text/plain, Size: 661 bytes --]

On Mon, 22 Jan 2018 16:45:13 +0000
Ciaran McCreesh <ciaran.mccreesh@googlemail.com> wrote:

> perl-cleaner is rather strong evidence that the Perl situation needs a
> major change anyway...

perl-cleaner is now *mostly* redundant. Its only residual use is
catching circumstances where slot dependencies weren't adequately
provided early enough for them to work.( ~1% of cases )

dev-lang/perl also has additional papering over now that aims to
increase the number of situations where subgraphs of the dependency
graph that don't require C based extensions may continue to work while
portage is still "rebuild pending" on those parts of the graph.

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [gentoo-dev] News Item: Portage Dynamic Deps

[Michał Górny]

W dniu pon, 22.01.2018 o godzinie 11∶28 +0100, użytkownik Andreas K.
Huettel napisał:
> Am Montag, 22. Januar 2018, 08:01:08 CET schrieb Zac Medico:
> > 
> > According to Gentoo policy, future ebuild dependency changes need to be
> > accompanied by a revision bump in order to trigger rebuilds for users.
> > Therefore, you should only need to use --changed-deps=y for a single
> > deep @world update. After that, if you encounter installed packages with
> > outdated dependencies in a future deep @world update, then you should
> > report it as a bug.
> 
> Did you come up with a solution how to handle eclass-generated dependency 
> changes then?
> 
> I'd loathe to have to do identical revision bumps for, say, all perl-
> module.eclass consumers...
> 

I think one of the reasons for the policy back then was to actually stop
people from creating horribly broken monstrosities of eclasses.

-- 
Best regards,
Michał Górny

[gentoo-dev] Re: News Item: Portage Dynamic Deps

[Michael Palimaka]

On 01/22/2018 09:28 PM, Andreas K. Huettel wrote:
> Am Montag, 22. Januar 2018, 08:01:08 CET schrieb Zac Medico:
>>
>> According to Gentoo policy, future ebuild dependency changes need to be
>> accompanied by a revision bump in order to trigger rebuilds for users.
>> Therefore, you should only need to use --changed-deps=y for a single
>> deep @world update. After that, if you encounter installed packages with
>> outdated dependencies in a future deep @world update, then you should
>> report it as a bug.
> 
> Did you come up with a solution how to handle eclass-generated dependency 
> changes then?

No.

Bug #641346 was filed for clarification about this, but it just got
closed without answering the question or consulting anyone.

Now, every time we want to make a minor change we need to revbump half
the tree due to a change that has been forced mostly by people not
actually involved in any actual ebuild maintenance.

Re: [gentoo-dev] Re: News Item: Portage Dynamic Deps

[Rich Freeman]

On Tue, Jan 23, 2018 at 7:40 AM, Michael Palimaka <kensington@gentoo.org> wrote:
> On 01/22/2018 09:28 PM, Andreas K. Huettel wrote:
>> Am Montag, 22. Januar 2018, 08:01:08 CET schrieb Zac Medico:
>>>
>>> According to Gentoo policy, future ebuild dependency changes need to be
>>> accompanied by a revision bump in order to trigger rebuilds for users.
>>> Therefore, you should only need to use --changed-deps=y for a single
>>> deep @world update. After that, if you encounter installed packages with
>>> outdated dependencies in a future deep @world update, then you should
>>> report it as a bug.
>>
>> Did you come up with a solution how to handle eclass-generated dependency
>> changes then?
>
> No.
>
> Bug #641346 was filed for clarification about this, but it just got
> closed without answering the question or consulting anyone.

From the bug: "I don't see the need for anything further before the
default behavior can be changed in portage, I'm all for it matching
PMS behavior."  (More details in the comment.)

The question was "I would like to ask Council to state more precisely
what needs to be specifically documented before we can stop enabling
dynamic-deps in Portage by default."

So, the answer to the question appears to be "nothing."

That might not be an answer that you like, but it is an answer.  I
can't vouch for who was or wasn't consulted before it was given.

-- 
Rich

Re: [gentoo-dev] Re: News Item: Portage Dynamic Deps

[Michael Orlitzky]

On 01/23/2018 07:40 AM, Michael Palimaka wrote:
>>
>> Did you come up with a solution how to handle eclass-generated dependency 
>> changes then?
> 
> No.
> 
> Bug #641346 was filed for clarification about this, but it just got
> closed without answering the question or consulting anyone.
> 
> Now, every time we want to make a minor change we need to revbump half
> the tree due to a change that has been forced mostly by people not
> actually involved in any actual ebuild maintenance.

You could always set "--dynamic-deps y" on your machine, and ignore the
breakage caused to end users (i.e. the situation last week).

[gentoo-dev] Re: News Item: Portage Dynamic Deps

[Michael Palimaka]

On 01/24/2018 12:15 AM, Michael Orlitzky wrote:
> On 01/23/2018 07:40 AM, Michael Palimaka wrote:
>>>
>>> Did you come up with a solution how to handle eclass-generated dependency 
>>> changes then?
>>
>> No.
>>
>> Bug #641346 was filed for clarification about this, but it just got
>> closed without answering the question or consulting anyone.
>>
>> Now, every time we want to make a minor change we need to revbump half
>> the tree due to a change that has been forced mostly by people not
>> actually involved in any actual ebuild maintenance.
> 
> You could always set "--dynamic-deps y" on your machine, and ignore the
> breakage caused to end users (i.e. the situation last week).

You mean the breakage caused by changing default options without any
consultation or notification?

Re: [gentoo-dev] Re: News Item: Portage Dynamic Deps

[Rich Freeman]

On Tue, Jan 23, 2018 at 8:40 AM, Michael Palimaka <kensington@gentoo.org> wrote:
> On 01/24/2018 12:15 AM, Michael Orlitzky wrote:
>> On 01/23/2018 07:40 AM, Michael Palimaka wrote:
>>>>
>>>> Did you come up with a solution how to handle eclass-generated dependency
>>>> changes then?
>>>
>>> No.
>>>
>>> Bug #641346 was filed for clarification about this, but it just got
>>> closed without answering the question or consulting anyone.
>>>
>>> Now, every time we want to make a minor change we need to revbump half
>>> the tree due to a change that has been forced mostly by people not
>>> actually involved in any actual ebuild maintenance.
>>
>> You could always set "--dynamic-deps y" on your machine, and ignore the
>> breakage caused to end users (i.e. the situation last week).
>
> You mean the breakage caused by changing default options without any
> consultation or notification?
>

It would already be broken on any PMS-compliant package manager I
imagine.  The goal is to make the repo and PMS align so that we're not
depending on non-PMS behavior.  Either our ebuild policies ought to
change, or PMS ought to change.  It is dumb to publish a specification
and then deliberately do things that break software that follows that
specification.

-- 
Rich

[gentoo-dev] Re: News Item: Portage Dynamic Deps

[Martin Vaeth]

Rich Freeman <rich0@gentoo.org> wrote:
>
> It would already be broken on any PMS-compliant package manager

No, it is solely the interpretation of the package manager.
PMS does not specify what information is stored in /var/db
or how that information is used.

[gentoo-dev] Re: News Item: Portage Dynamic Deps

[Ulrich Mueller]

[-- Attachment #1: Type: text/plain, Size: 634 bytes --]

>>>>> On Wed, 24 Jan 2018, Martin Vaeth wrote:

> Rich Freeman <rich0@gentoo.org> wrote:
>> 
>> It would already be broken on any PMS-compliant package manager

> No, it is solely the interpretation of the package manager.
> PMS does not specify what information is stored in /var/db
> or how that information is used.

"Runtime dependencies (RDEPEND). These must be installed and usable
before the results of an ebuild merging are treated as usable."
https://projects.gentoo.org/pms/6/pms.html#x1-770008.1

IMHO this implies that the dependencies at merge time are the relevant
ones, but not any later changes to the ebuild.

Ulrich

[-- Attachment #2: Type: application/pgp-signature, Size: 490 bytes --]

[gentoo-dev] Re: News Item: Portage Dynamic Deps

[Martin Vaeth]

Ulrich Mueller <ulm@gentoo.org> wrote:
> This is an OpenPGP/MIME signed message (RFC 2440 and 3156)
> --pgp+signed++Zg8D+I6sgRUw0D
> Content-Type: text/plain; charset=us-ascii
> Content-Transfer-Encoding: 7bit
>
>>>>>> On Wed, 24 Jan 2018, Martin Vaeth wrote:
>
>> Rich Freeman <rich0@gentoo.org> wrote:
>>> 
>>> It would already be broken on any PMS-compliant package manager
>
>> No, it is solely the interpretation of the package manager.
>> PMS does not specify what information is stored in /var/db
>> or how that information is used.
>
> "Runtime dependencies (RDEPEND). These must be installed and usable
> before the results of an ebuild merging are treated as usable."
> https://projects.gentoo.org/pms/6/pms.html#x1-770008.1
>
> IMHO this implies that the dependencies at merge time are the relevant
> ones

IMHO this specifies what is relevant when an emerge merging happens.
Nothing more, nothing less.
_If_ one would be willing to interpret it to have a meaning also _after_
the emerge, then of course the RDEPEND in PMS can refer to the only value
which is specified in PMS, i.e. that stored in the ebuild (and not in
any database which is explicitly not specified by PMS).
In other words: _If_ one puts any unsaid interpretation into that
sentence, then this can only be dynamic deps.

[gentoo-dev] Re: News Item: Portage Dynamic Deps

[Ulrich Mueller]

[-- Attachment #1: Type: text/plain, Size: 1484 bytes --]

>>>>> On Wed, 24 Jan 2018, Martin Vaeth wrote:

> Ulrich Mueller <ulm@gentoo.org> wrote:
>> "Runtime dependencies (RDEPEND). These must be installed and usable
>> before the results of an ebuild merging are treated as usable."
>> https://projects.gentoo.org/pms/6/pms.html#x1-770008.1
>> 
>> IMHO this implies that the dependencies at merge time are the
>> relevant ones

> IMHO this specifies what is relevant when an emerge merging happens.
> Nothing more, nothing less.

Exactly. RDEPEND is to be evaluated at the time before the package is
merged. For PDEPEND it is even clearer: "These must be installed at
some point before the package manager finishes the batch of installs."

> _If_ one would be willing to interpret it to have a meaning also
> _after_ the emerge, then of course the RDEPEND in PMS can refer to
> the only value which is specified in PMS, i.e. that stored in the
> ebuild

... at the time when the package was merged. You cannot rely on
anything else, because the ebuild may be gone in the meantime.

> (and not in any database which is explicitly not specified by PMS).
> In other words: _If_ one puts any unsaid interpretation into that
> sentence, then this can only be dynamic deps.

No. The only thing that PMS doesn't specify is the special format of
the VDB. However, the spec says that variables must keep their values
between phase functions, which includes pkg_prerm() and pkg_postrm().
By this, *some* sort of storage mechanism must exist.

Ulrich

[-- Attachment #2: Type: application/pgp-signature, Size: 490 bytes --]

Re: [gentoo-dev] News Item: Portage Dynamic Deps

[Zac Medico]

[-- Attachment #1.1: Type: text/plain, Size: 977 bytes --]

On 01/22/2018 02:28 AM, Andreas K. Huettel wrote:
> Am Montag, 22. Januar 2018, 08:01:08 CET schrieb Zac Medico:
>>
>> According to Gentoo policy, future ebuild dependency changes need to be
>> accompanied by a revision bump in order to trigger rebuilds for users.
>> Therefore, you should only need to use --changed-deps=y for a single
>> deep @world update. After that, if you encounter installed packages with
>> outdated dependencies in a future deep @world update, then you should
>> report it as a bug.
> 
> Did you come up with a solution how to handle eclass-generated dependency 
> changes then?
> 
> I'd loathe to have to do identical revision bumps for, say, all perl-
> module.eclass consumers...

Would it be possible to isolate the dependency changes in virtual
ebuilds? If the eclass depends on a virtual, the maybe updating to the
latest version of the virtual can serve as the means apply the needed
dependency change?
-- 
Thanks,
Zac


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 224 bytes --]