[gentoo-dev] don't rely on dynamic deps

[gentoo-dev] don't rely on dynamic deps

[hasufell]

afaiu dynamic deps are broken and not defined in PMS

still... people seem to fix deps without revbumping, causing users who
either don't use dynamic deps (it's optional for portage through
--dynamic-deps=y, although it's on by default) or who use a different PM
to not get the fix, at worst resulting in broken dependency calculation

suggestion:
* stop fixing dependencies without revbumping
* add an appropriate question to the dev quiz
* remove dynamic deps from portage (afair that is already considered by
the portage team)

Re: [gentoo-dev] don't rely on dynamic deps

[Samuli Suominen]

On 21/07/14 22:37, hasufell wrote:
> afaiu dynamic deps are broken and not defined in PMS
>
> still... people seem to fix deps without revbumping, causing users who
> either don't use dynamic deps (it's optional for portage through
> --dynamic-deps=y, although it's on by default) or who use a different PM
> to not get the fix, at worst resulting in broken dependency calculation
>
> suggestion:
> * stop fixing dependencies without revbumping
> * add an appropriate question to the dev quiz
> * remove dynamic deps from portage (afair that is already considered by
> the portage team)
>

Revision bumping for dependency change that doesn't cause the package's
file content
to change doesn't make sense; triggers useless rebuilds for users.

Portage is the official package manager, and has dynamic deps enabled by
default.

And it's already in ebuild-quiz.txt to ensure knowing when to, or not to,
revbump:

*** Ebuild technical/policy questions

1.	You change a package's ebuild to install an init script. Previously,
	the package had no init script at all.
	Is a revision bump necessary? Why? What about when adding a patch?


So, -1, useless rebuilds is one of the biggest problems lately, it's an
relatively new problem,
people are revbumping packages for the simplest things like EAPI4->5

- Samuli

Re: [gentoo-dev] don't rely on dynamic deps

[Ciaran McCreesh]

[-- Attachment #1: Type: text/plain, Size: 307 bytes --]

On Mon, 21 Jul 2014 22:42:23 +0300
Samuli Suominen <ssuominen@gentoo.org> wrote:
> people are revbumping packages for the simplest things like EAPI4->5

EAPI changing to 5 should always get a revbump, since it causes
confusion if anyone has a USE dependency upon your package.

-- 
Ciaran McCreesh

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 181 bytes --]

Re: [gentoo-dev] don't rely on dynamic deps

[Samuli Suominen]

On 21/07/14 22:50, Ciaran McCreesh wrote:
> On Mon, 21 Jul 2014 22:42:23 +0300
> Samuli Suominen <ssuominen@gentoo.org> wrote:
>> people are revbumping packages for the simplest things like EAPI4->5
> EAPI changing to 5 should always get a revbump, since it causes
> confusion if anyone has a USE dependency upon your package.
>

What kind of confusion?   In my experience, Portage handles it well

Re: [gentoo-dev] don't rely on dynamic deps

[Ian Stakenvicius]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

On 21/07/14 04:06 PM, Samuli Suominen wrote:
> 
> On 21/07/14 22:50, Ciaran McCreesh wrote:
>> On Mon, 21 Jul 2014 22:42:23 +0300 Samuli Suominen
>> <ssuominen@gentoo.org> wrote:
>>> people are revbumping packages for the simplest things like
>>> EAPI4->5
>> EAPI changing to 5 should always get a revbump, since it causes 
>> confusion if anyone has a USE dependency upon your package.
>> 
> 
> What kind of confusion?   In my experience, Portage handles it
> well
> 

Says the guy that's emerging things with --nodeps most of the time... :D


-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.22 (GNU/Linux)

iF4EAREIAAYFAlPNdFUACgkQ2ugaI38ACPAtHgD/YIrBc6LpzTRXm2lxWWSEXkUo
Wp6IM5mIthE1+0DepsYA/jTG85TGSHF7R326e8eaAlr02FT2g7M47wMjOLzzsvB/
=nDEq
-----END PGP SIGNATURE-----

Re: [gentoo-dev] don't rely on dynamic deps

[Samuli Suominen]

On 21/07/14 23:13, Ian Stakenvicius wrote:
> On 21/07/14 04:06 PM, Samuli Suominen wrote:
>
> > On 21/07/14 22:50, Ciaran McCreesh wrote:
> >> On Mon, 21 Jul 2014 22:42:23 +0300 Samuli Suominen
> >> <ssuominen@gentoo.org> wrote:
> >>> people are revbumping packages for the simplest things like
> >>> EAPI4->5
> >> EAPI changing to 5 should always get a revbump, since it causes
> >> confusion if anyone has a USE dependency upon your package.
> >>
>
> > What kind of confusion?   In my experience, Portage handles it
> > well
>
>
> Says the guy that's emerging things with --nodeps most of the time... :D
>
>
>

Portage's dependency calculation takes long time, but if you add useless
rebuilds of packages on top of that, that makes things even worse

(Yeah, I saw the ":D")

Re: [gentoo-dev] don't rely on dynamic deps

[Ciaran McCreesh]

[-- Attachment #1: Type: text/plain, Size: 1012 bytes --]

On Mon, 21 Jul 2014 23:06:22 +0300
Samuli Suominen <ssuominen@gentoo.org> wrote:
> On 21/07/14 22:50, Ciaran McCreesh wrote:
> > On Mon, 21 Jul 2014 22:42:23 +0300
> > Samuli Suominen <ssuominen@gentoo.org> wrote:
> >> people are revbumping packages for the simplest things like
> >> EAPI4->5
> > EAPI changing to 5 should always get a revbump, since it causes
> > confusion if anyone has a USE dependency upon your package.
> 
> What kind of confusion?   In my experience, Portage handles it well

The way (+) and (-) work depends upon the EAPI of the things they're
being matched against (not the EAPI of the package with the
dependencies). When developers are adding in >= dependencies to
restrict to matching against EAPI 5 things (as they have to do for
multilib, for example), they would need to check the CVS log to see if
any ebuild has *ever* been EAPI < 5. It's less work and less error-prone
for developers to just always do a bump when switching to EAPI 5.

-- 
Ciaran McCreesh

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 181 bytes --]

Re: [gentoo-dev] don't rely on dynamic deps

[hasufell]

Samuli Suominen:
> So, -1, useless rebuilds is one of the biggest problems lately

I am not sure if that is a joke.

We have:
* a broken PM which does incomplete dep calculation, gives wrong
suggestions to the user, has totally useless error/debug output,
randomly fails to remove files, allows to break your system in numerous
ways and whatnot... and I'm not going through bugzilla now to prove it
* overcomplex eclasses, because people try to avoid getting stuff into
the PM, resulting in more confusion for the PM
* repeatedly broken stable packages
* people coding against a PM instead of PMS and thus relying on
undocumented behavior and breaking the "meta-distribution" part of gentoo
* a PM codebase no one wants to be involved in

and you tell me the biggest problems are useless rebuilds?

Reality check, please. (btw... I didn't come up with the subslot idea,
so maybe check with those guys about useless rebuilds)


Removing dynamic deps is an easy way to improve the strictness of
portage, adhere better to PMS and improve compatibility with other PMs.

After that, we can discuss if there is a _sane_ way to avoid such rebuilds.

Re: [gentoo-dev] don't rely on dynamic deps

[Ian Stakenvicius]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

On 21/07/14 04:28 PM, hasufell wrote:
> 
> Reality check, please. (btw... I didn't come up with the subslot
> idea, so maybe check with those guys about useless rebuilds)
> 
> 
> Removing dynamic deps is an easy way to improve the strictness of 
> portage, adhere better to PMS and improve compatibility with other
> PMs.
> 
> After that, we can discuss if there is a _sane_ way to avoid such
> rebuilds.
> 


subslot rebuilds aren't supposed to be useless; however if the subslot
is changed unnecessarily then yes, it can trigger those rebuilds.

I wonder if there may be some form of extension we could add to
portage, such that it could do a VDB-only "re-emerge" somehow, when
the in-tree ebuild doesn't match the in-VDB one.  If that could be
implemented properly (and i'm not sure that it could, tbh), maybe that
would help reduce issues with dynamic deps, too...


-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.22 (GNU/Linux)

iF4EAREIAAYFAlPNewMACgkQ2ugaI38ACPB67gEAnK/FOF+6xQjXg3R3in3B/WgG
loDxg1XOpMDR6NQPE0QA/jeDo3Vxt5qawbohvpnoWVwPwxbpHSfWkQ0UIwnQcDRw
=EiHA
-----END PGP SIGNATURE-----

Re: [gentoo-dev] don't rely on dynamic deps

[Tom Wijsman]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Mon, 21 Jul 2014 16:41:39 -0400
Ian Stakenvicius <axs@gentoo.org> wrote:

> I wonder if there may be some form of extension we could add to
> portage, such that it could do a VDB-only "re-emerge" somehow, when
> the in-tree ebuild doesn't match the in-VDB one.  If that could be
> implemented properly (and i'm not sure that it could, tbh), maybe that
> would help reduce issues with dynamic deps, too...

Hmm, dynamic dependencies ... dynamic rebuilds ... dynamic compilation;
one day we will have hot code pushes where upstream changes immediately
reflects itself upon one's system. Does such a gimmick succeed? Meteor!

- -- 
With kind regards,

Tom Wijsman (TomWij)
Gentoo Developer

E-mail address  : TomWij@gentoo.org
GPG Public Key  : 6D34E57D
GPG Fingerprint : C165 AF18 AB4C 400B C3D2  ABF0 95B2 1FCD 6D34 E57D
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2

iQEcBAEBAgAGBQJTzuAsAAoJEPWZc8roOL/QNYsH/1uJrazScukDYnGj3lsPkl7P
5l7l7caugq5CwFfJ+VLZR5byV6MePBff+rJsO6Ch8v4yM+h+IFnOn7pLS27Oqm71
LRaGHeTH/pge3jq9mm53b7ABi2IjuNSKIr69/loYMJaNgHQLZCtR/wFVxKR3XFrA
dhJN7WKhHAG0+1HRlNWCdYpHllG+cmayLARlwfWGbZii/OZWh0eAVEUV0pFdZwlP
WaeOcnLebn+TFWPyKEkGKkmz7yI7fFMaoKBueEAZ9PwEUmhdSK5PEeY2U/OpLOA7
7wOYDe9J1k04q5DwLzZe9LvqjwjYtGIP4sL1/b+9TCw59+akKC+DmnDv67vuTHs=
=wV/h
-----END PGP SIGNATURE-----

Re: [gentoo-dev] don't rely on dynamic deps

[Jan Matejka]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA512

On Wed, 23 Jul 2014 00:05:32 +0200
Tom Wijsman <TomWij@gentoo.org> wrote:

> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
> 
> On Mon, 21 Jul 2014 16:41:39 -0400
> Ian Stakenvicius <axs@gentoo.org> wrote:
> 
> > I wonder if there may be some form of extension we could add to
> > portage, such that it could do a VDB-only "re-emerge" somehow, when
> > the in-tree ebuild doesn't match the in-VDB one.  If that could be
> > implemented properly (and i'm not sure that it could, tbh), maybe
> > that would help reduce issues with dynamic deps, too...
> 
> Hmm, dynamic dependencies ... dynamic rebuilds ... dynamic
> compilation; one day we will have hot code pushes where upstream
> changes immediately reflects itself upon one's system. Does such a
> gimmick succeed? Meteor!

https://en.wikipedia.org/wiki/KGraft

- --
Jan Matějka        | Developer
https://gentoo.org | Gentoo Linux
GPG: A33E F5BC A9F6 DAFD 2021  6FB6 3EBF D45B EEB6 CA8B
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.22 (GNU/Linux)

iQEcBAEBCgAGBQJT/gT1AAoJEIN+7RD5ejahISgIAKbVgqLmuJADmJ37TddtmPPP
7WPq5UYdGT4wq9rElrU3GCu10u+pxQqOOKzVjnXiSlcCF1xqkG07caaYO/Xw/ccS
VclMPDngRU7YUSl9F6UeDkDK9l+48qE/1/ohrSL9f8giA5H5SZ3G3fanWN/oFp6W
FsGx7J6ddRmM/LaSHrso4ngbMU7XIMFZgUeR5X3t5bWef/Si9adxJzfbMoCQFUDN
7qXT8EkY9U9Po5/aAqtsriyGWZttuIvjGbmiNdrdJI/IT/Rshlqzqy/viqcK2aOX
bCO1dn5wRLxfV7JBgmg14UJqcDK6HgkSkl4rrl574M4arq7kZtbMEKbSvDcpurw=
=Wphr
-----END PGP SIGNATURE-----

Re: [gentoo-dev] don't rely on dynamic deps

[Michał Górny]

[-- Attachment #1: Type: text/plain, Size: 548 bytes --]

Dnia 2014-07-21, o godz. 20:28:24
hasufell <hasufell@gentoo.org> napisał(a):

> * a broken PM which does incomplete dep calculation, gives wrong
> suggestions to the user, has totally useless error/debug output,
> randomly fails to remove files, allows to break your system in numerous
> ways and whatnot... and I'm not going through bugzilla now to prove it

Just to add a single point, portage also randomly silently refuses to
update stuff because of subslot operator (instead of causing rebuilds).

-- 
Best regards,
Michał Górny

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 949 bytes --]

Re: [gentoo-dev] don't rely on dynamic deps

[Jeroen Roovers]

On Mon, 21 Jul 2014 20:28:24 +0000
hasufell <hasufell@gentoo.org> wrote:

> We have:
> * a broken PM which does incomplete dep calculation, gives wrong
> suggestions to the user, has totally useless error/debug output,
> randomly fails to remove files, allows to break your system in
> numerous ways and whatnot... and I'm not going through bugzilla now
> to prove it
> * overcomplex eclasses, because people try to avoid getting stuff into
> the PM, resulting in more confusion for the PM
> * repeatedly broken stable packages
> * people coding against a PM instead of PMS and thus relying on
> undocumented behavior and breaking the "meta-distribution" part of
> gentoo
> * a PM codebase no one wants to be involved in

So you suggest we work around a bug in the PM which would be a single
fix. Everywhere.


    jer

Re: [gentoo-dev] don't rely on dynamic deps

[Ciaran McCreesh]

[-- Attachment #1: Type: text/plain, Size: 271 bytes --]

On Mon, 21 Jul 2014 23:01:58 +0200
Jeroen Roovers <jer@gentoo.org> wrote:
> So you suggest we work around a bug in the PM which would be a single
> fix. Everywhere.

Dynamic dependencies is not fixable. It's an irredeemably broken
concept.

-- 
Ciaran McCreesh

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 181 bytes --]

Re: [gentoo-dev] don't rely on dynamic deps

[Jeroen Roovers]

On Mon, 21 Jul 2014 22:06:08 +0100
Ciaran McCreesh <ciaran.mccreesh@googlemail.com> wrote:

> On Mon, 21 Jul 2014 23:01:58 +0200
> Jeroen Roovers <jer@gentoo.org> wrote:
> > So you suggest we work around a bug in the PM which would be a
> > single fix. Everywhere.
> 
> Dynamic dependencies is not fixable. It's an irredeemably broken
> concept.

It works fine for me.


     jer

Re: [gentoo-dev] don't rely on dynamic deps

[Ciaran McCreesh]

[-- Attachment #1: Type: text/plain, Size: 760 bytes --]

On Mon, 21 Jul 2014 23:13:06 +0200
Jeroen Roovers <jer@gentoo.org> wrote:
> On Mon, 21 Jul 2014 22:06:08 +0100
> Ciaran McCreesh <ciaran.mccreesh@googlemail.com> wrote:
> > On Mon, 21 Jul 2014 23:01:58 +0200
> > Jeroen Roovers <jer@gentoo.org> wrote:
> > > So you suggest we work around a bug in the PM which would be a
> > > single fix. Everywhere.
> > 
> > Dynamic dependencies is not fixable. It's an irredeemably broken
> > concept.
> 
> It works fine for me.

You should read the relevant bugs before making such a bold claim...
You're illustrating why we have the problem to begin with: you're
confusing "I've not noticed it go wrong, or realised that breakages
I've seen are because of this" with "works fine".

-- 
Ciaran McCreesh

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 181 bytes --]

Re: [gentoo-dev] don't rely on dynamic deps

[Tom Wijsman]

[-- Attachment #1: Type: text/plain, Size: 452 bytes --]

On Mon, 21 Jul 2014 23:01:58 +0200
Jeroen Roovers <jer@gentoo.org> wrote:

> So you suggest we work around a bug in the PM which would be a single
> fix. Everywhere.

Which bugs? Which fixes? Where? ... Did this thread spawn from nothing?

-- 
With kind regards,

Tom Wijsman (TomWij)
Gentoo Developer

E-mail address  : TomWij@gentoo.org
GPG Public Key  : 6D34E57D
GPG Fingerprint : C165 AF18 AB4C 400B C3D2  ABF0 95B2 1FCD 6D34 E57D

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 473 bytes --]

Re: [gentoo-dev] don't rely on dynamic deps

[Michał Górny]

[-- Attachment #1: Type: text/plain, Size: 901 bytes --]

Dnia 2014-07-21, o godz. 22:42:23
Samuli Suominen <ssuominen@gentoo.org> napisał(a):

> So, -1, useless rebuilds is one of the biggest problems lately,

[citation needed].

In other words, please support such claims with evidence. Because
honestly I didn't see very much people complaining about unnecessary
rebuilds, except in this specific thread.

But I guess they're indeed a larger issue than, for example, portage
forcing wrong branches of || dependencies or other dependency
calculation errors that result in people being unable to update their
systems. But I don't really visit Forums often.

> it's an
> relatively new problem,
> people are revbumping packages for the simplest things like EAPI4->5

If you ever happen to change EAPI in a stable ebuild without revbump,
please be kind enough to revoke your own commit rights. Thanks.

-- 
Best regards,
Michał Górny

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 949 bytes --]

Re: [gentoo-dev] don't rely on dynamic deps

[Tom Wijsman]

[-- Attachment #1: Type: text/plain, Size: 1033 bytes --]

On Tue, 22 Jul 2014 23:11:37 +0200
Michał Górny <mgorny@gentoo.org> wrote:

> [citation needed].
> 
> In other words, please support such claims with evidence. Because
> honestly I didn't see very much people complaining about unnecessary
> rebuilds, except in this specific thread.
> 
> But I guess they're indeed a larger issue than, for example, portage
> forcing wrong branches of || dependencies or other dependency
> calculation errors that result in people being unable to update their
> systems. But I don't really visit Forums often.

[citation needed].

Actually, please try to go a step further and reproduce it; surprise!

> If you ever happen to change EAPI in a stable ebuild without revbump,
> please be kind enough to revoke your own commit rights. Thanks.

So much revocations. Thanks.

-- 
With kind regards,

Tom Wijsman (TomWij)
Gentoo Developer

E-mail address  : TomWij@gentoo.org
GPG Public Key  : 6D34E57D
GPG Fingerprint : C165 AF18 AB4C 400B C3D2  ABF0 95B2 1FCD 6D34 E57D

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 473 bytes --]

Re: [gentoo-dev] don't rely on dynamic deps

[Ciaran McCreesh]

[-- Attachment #1: Type: text/plain, Size: 552 bytes --]

On Tue, 22 Jul 2014 23:11:37 +0200
Michał Górny <mgorny@gentoo.org> wrote:
> But I guess they're indeed a larger issue than, for example, portage
> forcing wrong branches of || dependencies or other dependency
> calculation errors that result in people being unable to update their
> systems. But I don't really visit Forums often.

Users are so used to broken dependency resolution that they don't
complain about it very much, and even when they do, they're rarely able
to identify the actual cause of the problem.

-- 
Ciaran McCreesh

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 181 bytes --]

Re: [gentoo-dev] don't rely on dynamic deps

[Tom Wijsman]

[-- Attachment #1: Type: text/plain, Size: 1504 bytes --]

On Mon, 21 Jul 2014 22:42:23 +0300
Samuli Suominen <ssuominen@gentoo.org> wrote:

> Revision bumping for dependency change that doesn't cause the
> package's file content
> to change doesn't make sense; triggers useless rebuilds for users.

A merged ebuild that misses a dependency needs an useless extra emerge.
Think about the triggers instead of the extra rebuilds or extra emerges.
 
> Portage is the official package manager, and has dynamic deps enabled
> by default.

Is it a feature or is it a hack?

> And it's already in ebuild-quiz.txt to ensure knowing when to, or not
> to, revbump:
> 
> *** Ebuild technical/policy questions
> 
> 1.	You change a package's ebuild to install an init script.
> Previously, the package had no init script at all.
> 	Is a revision bump necessary? Why? What about when adding a
> patch?

That's not about dynamic dependencies; but yes, too much rev bumps.

> So, -1, useless rebuilds is one of the biggest problems lately, it's
> an relatively new problem,
> people are revbumping packages for the simplest things like EAPI4->5

Useless triggers are the problem; why are the rev bumps needed, why are
dependencies forgotten, ...? Sounds like a developer work flow issue...

https://bugs.gentoo.org/show_bug.cgi?id=499852

-- 
With kind regards,

Tom Wijsman (TomWij)
Gentoo Developer

E-mail address  : TomWij@gentoo.org
GPG Public Key  : 6D34E57D
GPG Fingerprint : C165 AF18 AB4C 400B C3D2  ABF0 95B2 1FCD 6D34 E57D

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 473 bytes --]

Re: [gentoo-dev] don't rely on dynamic deps

[Pacho Ramos]

El mar, 22-07-2014 a las 23:56 +0200, Tom Wijsman escribió:
[...]
> Useless triggers are the problem; why are the rev bumps needed, why are
> dependencies forgotten, ...? Sounds like a developer work flow issue...
> 
> https://bugs.gentoo.org/show_bug.cgi?id=499852
> 

There are lots of cases of upstream forgetting to update configure
checks and changing without noticing the real requirements. For example,
moving from gtk+-2.90.0 to gtk+-3.3. Since we don't such old setups, we
don't notice a newer gtk+ is needed. Sometimes, a user with a really old
stable setup tries to update, find the problem and we need to update the
dependency

[gentoo-dev] Re: don't rely on dynamic deps

[Martin Vaeth]

Tom Wijsman <TomWij@gentoo.org> wrote:
>
> Useless triggers are the problem; why are the rev bumps needed, why are
> dependencies forgotten, ...?

It is not only about *forgotten* dependencies:
It is whenever something is restructured, a new project appears, etc.

Some examples:

1. Package foo/bar splits into foo/bar-base and foo/bar-gui.
Should all the packages depending on foo/bar be recompiled,
just because the dependency must be relaxed?

2. Package foo/bar-base and foo/bar-gui merge to foo/bar.
In this case the dependency needs to be strengthened,
but technically a recompile should not be necessary
(perhaps there are exceptions where it is necessary for
newer versions, but these should be rare; in any case,
the developer should know what is "correct").

3. Package foo/bar was forked to foo/baz, both providing
the same ABI. Maybe a new virtual is introduced in gentoo,
or maybe the alternative dependency should be directly
added in all ebuilds.
Absolutely no reason to force a rebuild, but with static
dependencies and without a bump, the user is not able
to change to the new foo/baz.

4. Package foo/bar was forked to foo/baz, both providing
the same API but not the same ABI.
This is a problematic situation, but it is so with and
without dynamic/static dependencies: Even if a user once
reemerges a package depending on it, he may nevertheless later
replace foo/bar by foo/baz and gets a broken system
despite all dependencies are satisfied all of the time.

Actually, 4. is an example why subslotting can *never*
fully replace revdep-rebuild.

Although 4. touches also a different problem, actually
the triggering of a rebuild in case 4., just because
a new alternative is available, is completely useless.
(*If* one wants to fix 4., one must introduce an
extended dependency syntax, and this would have problems
with virtuals etc.)

Probably there are many more examples than 1.-4, but I hope
that the point becomes clear: Whenever packages split, merge,
or can substitute each other, dependency changes are necessary,
and rebuilds caused by these are unnecessary.

Unfortunately, such things happen *very* frequently...

Nobody is to blame for this; the PM just should be ready
to deal with such situations without unnecessary rebuilds,
be it by dynamic deps or by a mechanism to avoid
recompilation.

Re: [gentoo-dev] Re: don't rely on dynamic deps

[Pacho Ramos]

El sáb, 26-07-2014 a las 12:00 +0000, Martin Vaeth escribió:
[...]
> Probably there are many more examples than 1.-4, but I hope
> that the point becomes clear: Whenever packages split, merge,
> or can substitute each other, dependency changes are necessary,
> and rebuilds caused by these are unnecessary.
> 
> Unfortunately, such things happen *very* frequently...
> 
> Nobody is to blame for this; the PM just should be ready
> to deal with such situations without unnecessary rebuilds,
> be it by dynamic deps or by a mechanism to avoid
> recompilation.
> 

I have also seen the need of add slots to DEPEND/RDEPEND due the
inclusion of an updated package like gstreamer:1.0/gtk+:3/libpng... or
needing to add forgotten USE deps (like package[introspection?] or
package[X]). 

Re: [gentoo-dev] don't rely on dynamic deps

[Andreas K. Huettel]

[-- Attachment #1: Type: Text/Plain, Size: 1195 bytes --]

Am Montag, 21. Juli 2014, 21:37:17 schrieb hasufell:
> afaiu dynamic deps are broken and not defined in PMS
> 
> still... people seem to fix deps without revbumping, causing users who
> either don't use dynamic deps (it's optional for portage through
> --dynamic-deps=y, although it's on by default) or who use a different PM
> to not get the fix, at worst resulting in broken dependency calculation
> 
> suggestion:
> * stop fixing dependencies without revbumping
> * add an appropriate question to the dev quiz
> * remove dynamic deps from portage (afair that is already considered by
> the portage team)

Actually the quizzes are pretty much clear on that. 

Revision must be bumped when the on-disk files installed by the ebuild are 
changed. 
Nothing about dependencies.

This has been policy for a LONG time, and we're not going to change it 
overnight just because you protest.

Now... whether dynamic deps are technically the right thing to do is another 
question. It merits discussion, but we need to be really sure about the 
consequences of any change.

-- 

Andreas K. Huettel
Gentoo Linux developer 
dilfridge@gentoo.org
http://www.akhuettel.de/


[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 951 bytes --]

Re: [gentoo-dev] don't rely on dynamic deps

[Ciaran McCreesh]

[-- Attachment #1: Type: text/plain, Size: 646 bytes --]

On Mon, 21 Jul 2014 21:53:04 +0200
"Andreas K. Huettel" <dilfridge@gentoo.org> wrote:
> Revision must be bumped when the on-disk files installed by the
> ebuild are changed. 
> Nothing about dependencies.
> 
> This has been policy for a LONG time, and we're not going to change
> it overnight just because you protest.

Policy used to be that you'd do a revbump when you wanted users to
reinstall stuff, and you wouldn't otherwise. The only complication is
that sometimes you want users to reinstall stuff so that there's
accurate dependency information available, rather than because
something has changed.

-- 
Ciaran McCreesh

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 181 bytes --]

Re: [gentoo-dev] don't rely on dynamic deps

[Pacho Ramos]

El lun, 21-07-2014 a las 20:55 +0100, Ciaran McCreesh escribió:
> On Mon, 21 Jul 2014 21:53:04 +0200
> "Andreas K. Huettel" <dilfridge@gentoo.org> wrote:
> > Revision must be bumped when the on-disk files installed by the
> > ebuild are changed. 
> > Nothing about dependencies.
> > 
> > This has been policy for a LONG time, and we're not going to change
> > it overnight just because you protest.
> 
> Policy used to be that you'd do a revbump when you wanted users to
> reinstall stuff, and you wouldn't otherwise. The only complication is
> that sometimes you want users to reinstall stuff so that there's
> accurate dependency information available, rather than because
> something has changed.
> 

Maybe this could be solved by having two kinds of revisions:
- One would rebuild all as usually (for example, -r1...)
- The other one would only regenerate VDB and wouldn't change the
installed files (for example, -r1.1)

But I am not sure if it could be viable from a "technical" point of
view :(

Re: [gentoo-dev] don't rely on dynamic deps

[Jeroen Roovers]

On Mon, 21 Jul 2014 23:06:07 +0200
Pacho Ramos <pacho@gentoo.org> wrote:

> Maybe this could be solved by having two kinds of revisions:
> - One would rebuild all as usually (for example, -r1...)
> - The other one would only regenerate VDB and wouldn't change the
> installed files (for example, -r1.1)

Or the package manager looks at changed in *DEPEND between the repo
and vdb and resolves those. Oh wait, portage already does. And we've
been talking about this for something like six years now? Probably
longer.


     jer

Re: [gentoo-dev] don't rely on dynamic deps

[Ciaran McCreesh]

[-- Attachment #1: Type: text/plain, Size: 848 bytes --]

On Mon, 21 Jul 2014 23:15:41 +0200
Jeroen Roovers <jer@gentoo.org> wrote:
> On Mon, 21 Jul 2014 23:06:07 +0200
> Pacho Ramos <pacho@gentoo.org> wrote:
> > Maybe this could be solved by having two kinds of revisions:
> > - One would rebuild all as usually (for example, -r1...)
> > - The other one would only regenerate VDB and wouldn't change the
> > installed files (for example, -r1.1)
> 
> Or the package manager looks at changed in *DEPEND between the repo
> and vdb and resolves those.

...assuming that the ebuild hasn't been removed, and that it can be
associated correctly when overlays are involved, and that the change
wasn't a change where a saved pkg_prerm uses the old dependency, not
the new one, or all the other ways this breaks.

You need to think your cunning plan the whole way through.

-- 
Ciaran McCreesh

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 181 bytes --]

Re: [gentoo-dev] don't rely on dynamic deps

[Michał Górny]

[-- Attachment #1: Type: text/plain, Size: 1108 bytes --]

Dnia 2014-07-21, o godz. 22:21:42
Ciaran McCreesh <ciaran.mccreesh@googlemail.com> napisał(a):

> On Mon, 21 Jul 2014 23:15:41 +0200
> Jeroen Roovers <jer@gentoo.org> wrote:
> > On Mon, 21 Jul 2014 23:06:07 +0200
> > Pacho Ramos <pacho@gentoo.org> wrote:
> > > Maybe this could be solved by having two kinds of revisions:
> > > - One would rebuild all as usually (for example, -r1...)
> > > - The other one would only regenerate VDB and wouldn't change the
> > > installed files (for example, -r1.1)
> > 
> > Or the package manager looks at changed in *DEPEND between the repo
> > and vdb and resolves those.
> 
> ...assuming that the ebuild hasn't been removed, and that it can be
> associated correctly when overlays are involved, and that the change
> wasn't a change where a saved pkg_prerm uses the old dependency, not
> the new one, or all the other ways this breaks.

You forgot about slot operators.

The funny thing is, almost none of the Gentoo developers even know that
slot operators disable dynamic dependencies completely in portage.

-- 
Best regards,
Michał Górny

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 949 bytes --]

Re: [gentoo-dev] don't rely on dynamic deps

[Maxim Kammerer]

On Tue, Jul 22, 2014 at 12:25 AM, Michał Górny <mgorny@gentoo.org> wrote:
> The funny thing is, almost none of the Gentoo developers even know that
> slot operators disable dynamic dependencies completely in portage.

So *that's* why I now have to change RDEPENDs in both the source
ebuild and in VDB in order to augment package's dependencies before
depclean...

-- 
Maxim Kammerer
Liberté Linux: http://dee.su/liberte

[gentoo-dev] Re: don't rely on dynamic deps

[Martin Vaeth]

Maxim Kammerer <mk@dee.su> wrote:
> On Tue, Jul 22, 2014 at 12:25 AM, Michał Górny <mgorny@gentoo.org> wrote:
>> The funny thing is, almost none of the Gentoo developers even know that
>> slot operators disable dynamic dependencies completely in portage.
>
> So *that's* why I now have to change RDEPENDs in both the source
> ebuild and in VDB in order to augment package's dependencies before
> depclean...

Yes, static deps are a completely broken idea if not every change
causes a revbump: There does not exist something like a "minor"
change in dependencies.

[gentoo-dev] Re: don't rely on dynamic deps

[Martin Vaeth]

Ciaran McCreesh <ciaran.mccreesh@googlemail.com> wrote:
> Jeroen Roovers <jer@gentoo.org> wrote:
>> On Mon, 21 Jul 2014 23:06:07 +0200
>> Pacho Ramos <pacho@gentoo.org> wrote:
>> > Maybe this could be solved by having two kinds of revisions:
>> > - One would rebuild all as usually (for example, -r1...)
>> > - The other one would only regenerate VDB and wouldn't change the
>> > installed files (for example, -r1.1)
>> Or the package manager looks at changed in *DEPEND between the repo
>> and vdb and resolves those.
>
> ...assuming that the ebuild hasn't been removed, and that it can be
> associated correctly when overlays are involved, and that the change
> wasn't a change where a saved pkg_prerm uses the old dependency, not
> the new one, or all the other ways this breaks.
>
> You need to think your cunning plan the whole way through.

It works, since it is completely equivalent to a revbump,
only that the unnecesary recompilation is avoided:
All of your problems exist (or don't exist) for usual revbumps
as well.

Re: [gentoo-dev] Re: don't rely on dynamic deps

[Ciaran McCreesh]

[-- Attachment #1: Type: text/plain, Size: 1761 bytes --]

On Sat, 26 Jul 2014 12:54:08 +0000 (UTC)
Martin Vaeth <martin@mvath.de> wrote:
> Ciaran McCreesh <ciaran.mccreesh@googlemail.com> wrote:
> > Jeroen Roovers <jer@gentoo.org> wrote:
> >> On Mon, 21 Jul 2014 23:06:07 +0200
> >> Pacho Ramos <pacho@gentoo.org> wrote:
> >> > Maybe this could be solved by having two kinds of revisions:
> >> > - One would rebuild all as usually (for example, -r1...)
> >> > - The other one would only regenerate VDB and wouldn't change the
> >> > installed files (for example, -r1.1)
> >> Or the package manager looks at changed in *DEPEND between the repo
> >> and vdb and resolves those.
> >
> > ...assuming that the ebuild hasn't been removed, and that it can be
> > associated correctly when overlays are involved, and that the change
> > wasn't a change where a saved pkg_prerm uses the old dependency, not
> > the new one, or all the other ways this breaks.
> >
> > You need to think your cunning plan the whole way through.
> 
> It works, since it is completely equivalent to a revbump,
> only that the unnecesary recompilation is avoided:
> All of your problems exist (or don't exist) for usual revbumps
> as well.

At this point, I think it would be most helpful towards us reaching a
conclusion if you agreed to refrain from commenting further until
you've understood the problem at hand.

You see, the rest of us are using "broken" to mean "broken" in a
technical sense, based upon our understanding of how ebuilds, the VDB
and metadata work. You seem to be using it to mean "does something you
superficially or ideologically don't like".

This is a technical discussion, and you need to read up on how things
work before you can make a meaningful contribution.

-- 
Ciaran McCreesh

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 181 bytes --]

[gentoo-dev] Re: don't rely on dynamic deps

[Martin Vaeth]

Ciaran McCreesh <ciaran.mccreesh@googlemail.com> wrote:
>
> At this point, I think it would be most helpful towards us reaching a
> conclusion if you agreed to refrain from commenting further until
> you've understood the problem at hand.

In other words: After I disproved all your wrong arguments,
you try repeatedly to ignore my technical points and instead
prefer to attack me personally of not understanding what I am saying.
Fortunately, this is a developer's mailing list which will
not get fooled by your strategy.

> You see, the rest of us are using "broken" to mean "broken" in a
> technical sense, based upon our understanding of how ebuilds, the VDB
> and metadata work.

It seems by "the rest of use" you mean me:
That's why I pointed out repeatedly *what* is broken
and why (namely the concept of having orphaned packages,
and I wlil not repeat the example).

> You seem to be using it to mean "does something you
> superficially or ideologically don't like".

You seem to be using it this way: That's what you call
dynamic deps broken but static not, although both face
the same problems.

> This is a technical discussion

Exactly. So instead of writing such pointless personal attacks,
you should give technical arguments.

Re: [gentoo-dev] Re: don't rely on dynamic deps

[Ciaran McCreesh]

[-- Attachment #1: Type: text/plain, Size: 460 bytes --]

On Sat, 26 Jul 2014 14:57:20 +0000 (UTC)
Martin Vaeth <martin@mvath.de> wrote:
> > This is a technical discussion
> 
> Exactly. So instead of writing such pointless personal attacks,
> you should give technical arguments.

The technical reasons that dynamic dependencies can never work have
already been explained. However, you continue to ignore them, and you
continue to refuse to read what PMS actually says on the matter.

-- 
Ciaran McCreesh

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 181 bytes --]

[gentoo-dev] Re: don't rely on dynamic deps

[Martin Vaeth]

Pacho Ramos <pacho@gentoo.org> wrote:
>
> Maybe this could be solved by having two kinds of revisions:
> - One would rebuild all as usually (for example, -r1...)
> - The other one would only regenerate VDB and wouldn't change the
> installed files (for example, -r1.1)

I made the same suggestion already on the corresponding bug
	https://bugs.gentoo.org/show_bug.cgi?id=516612#c33
without any response.

It seems to me that this could avoid the problem of useless
recompilation and would allow fine-graining of the issue by the
ebuild maintainer (if not the maintainer of the ebuild, who else
should be able to decide whether recompilation might be
necessary to handle certain exceptions?)
and simultaneously allow to revbump even on presumably
tiny dependency changes.

I still have not seen an argument against this idea.

Of course, this would need an EAPI bump and could only be used
for packages which are (or switch to(?)) this new EAPI, so a few
(core) packages which should stay EAPI=0 for a long time
are excluded from this for still quite a while.
But apart from that few exceptions...?

Re: [gentoo-dev] Re: don't rely on dynamic deps

[Pacho Ramos]

El mar, 22-07-2014 a las 07:39 +0000, Martin Vaeth escribió:
> Pacho Ramos <pacho@gentoo.org> wrote:
> >
> > Maybe this could be solved by having two kinds of revisions:
> > - One would rebuild all as usually (for example, -r1...)
> > - The other one would only regenerate VDB and wouldn't change the
> > installed files (for example, -r1.1)
> 
> I made the same suggestion already on the corresponding bug
> 	https://bugs.gentoo.org/show_bug.cgi?id=516612#c33
> without any response.

Just CCed :)

> 
> It seems to me that this could avoid the problem of useless
> recompilation and would allow fine-graining of the issue by the
> ebuild maintainer (if not the maintainer of the ebuild, who else
> should be able to decide whether recompilation might be
> necessary to handle certain exceptions?)
> and simultaneously allow to revbump even on presumably
> tiny dependency changes.
> 
> I still have not seen an argument against this idea.
> 
> Of course, this would need an EAPI bump and could only be used
> for packages which are (or switch to(?)) this new EAPI, so a few
> (core) packages which should stay EAPI=0 for a long time
> are excluded from this for still quite a while.
> But apart from that few exceptions...?
> 
> 

Also, this could be a benefit in the long term if we need to spread any
changes to VDB in the future.

Re: [gentoo-dev] Re: don't rely on dynamic deps

[Alexander Berntsen]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

On 22/07/14 09:39, Martin Vaeth wrote:
> Pacho Ramos <pacho@gentoo.org> wrote:
>> 
>> Maybe this could be solved by having two kinds of revisions: -
>> One would rebuild all as usually (for example, -r1...) - The
>> other one would only regenerate VDB and wouldn't change the 
>> installed files (for example, -r1.1)
> 
> I made the same suggestion already on the corresponding bug 
> https://bugs.gentoo.org/show_bug.cgi?id=516612#c33 without any
> response.
Martin,

I have no arguments against your idea to tell the PM that "this bump
only changes dependencies". My initial reaction is that this is a Good
Thing. Would you like to try to implement it?
- -- 
Alexander
bernalex@gentoo.org
https://secure.plaimi.net/~alexander
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/

iF4EAREIAAYFAlPOM9wACgkQRtClrXBQc7WV1AD+LbojEp7TVY51WobwTflCPzQK
wZbmGWiyyBhylG7AgWIBAJRKURylzKxjPWcmjGwllf2CXcublXZCmndzbHeoTm0B
=doak
-----END PGP SIGNATURE-----

Re: [gentoo-dev] don't rely on dynamic deps

[Ian Stakenvicius]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

On 21/07/14 05:06 PM, Pacho Ramos wrote:
> El lun, 21-07-2014 a las 20:55 +0100, Ciaran McCreesh escribió:
>> On Mon, 21 Jul 2014 21:53:04 +0200 "Andreas K. Huettel"
>> <dilfridge@gentoo.org> wrote:
>>> Revision must be bumped when the on-disk files installed by
>>> the ebuild are changed. Nothing about dependencies.
>>> 
>>> This has been policy for a LONG time, and we're not going to
>>> change it overnight just because you protest.
>> 
>> Policy used to be that you'd do a revbump when you wanted users
>> to reinstall stuff, and you wouldn't otherwise. The only
>> complication is that sometimes you want users to reinstall stuff
>> so that there's accurate dependency information available, rather
>> than because something has changed.
>> 
> 
> Maybe this could be solved by having two kinds of revisions: - One
> would rebuild all as usually (for example, -r1...) - The other one
> would only regenerate VDB and wouldn't change the installed files
> (for example, -r1.1)
> 
> But I am not sure if it could be viable from a "technical" point
> of view :(
> 
> 

eww, no.  Using ${PVR} to detect how portage should update things
would be asking for trouble, imo.  Besides, I don't think detection of
when to just update VDB is the issue.  The main issue that I see is
- -how- VDB should be adjusted based on what changes are made to the
ebuilds.  For instance, if minimum versions of deps are adjusted
in-place, should vdb be updated to match?  what happens if the minimum
version of the currently-installed dep is below the new one?  etc. etc.

Also, in theory an EAPI bump with nothing else changing should be
re-generatable in the VDB, but i have a gut feeling (no evidence, just
a feeling) that going from say, EAPI2 to EAPI5 without doing some of
the phase functions again (ie 'merge', maybe there are others that can
affect VDB?) will result in a different VDB from a regular rebuild.


-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.22 (GNU/Linux)

iF4EAREIAAYFAlPObO0ACgkQ2ugaI38ACPCerQEAgTgQOvCDl0dbB5sOOZ4diBNs
cheQR18XFo7MsVBX3uUA/0zP1cAiWy1zAF+crrfCC42krtvGmSSiU4JG0dFo4452
=iNmo
-----END PGP SIGNATURE-----

[gentoo-dev] Re: don't rely on dynamic deps

[Martin Vaeth]

Ian Stakenvicius <axs@gentoo.org> wrote:
>
> The main issue that I see is
> - -how- VDB should be adjusted based on what changes are made to the
> ebuilds.  For instance, if minimum versions of deps are adjusted
> in-place, should vdb be updated to match?  what happens if the minimum
> version of the currently-installed dep is below the new one?  etc. etc.

All these problems disappear with minor revisions:
You have to "install" the minor revisions just like any major revision,
just that some phases will be shortcut.
In particular, if the new dependencies are not satisfied, you get
conflicts as usual if you would want to upgrade to a new version
with dependencies not being satisfied.

> Also, in theory an EAPI bump with nothing else changing should be
> re-generatable in the VDB, but i have a gut feeling (no evidence, just
> a feeling) that going from say, EAPI2 to EAPI5 without doing some of
> the phase functions again (ie 'merge', maybe there are others that can
> affect VDB?) will result in a different VDB from a regular rebuild.

As far as I can see, the phase functions which can be skipped
are those from "EbuildExecuter".
If a package is special and needs to execute these functions,
then this package must not use minor revisions and needs to
be recompiled. Howeer, these packages should be rather rare;
I cannot even imagine a reason why this should be necessary.

For "merge" only the regeneration of CONTENTS in /var/db/pkg
should be skipped.

Concerning the confusion with "minor" revisions, I think that
just the version comparison function in portage needs to be tuned to
ignore minor revisions unless an extra parameter "minor" is passed:

This extra parameter is essentially only needed in the "best"
function and in the check whether phases need to be run.
The version comparison function should just return
-2, 2, 0 (if minor is not passed or False)
-2, 2, -1, 1, 0 (if minor True; the values +-1 mean that
*only* the minor revision differs).

For some parts (some eapi support, parts of EbuildExecuter and
version comparison), I have already some patches ready
for portage.  However, I have almost no free time, and I am
not familiar enough with python and portage to do the rest
in a reasonable time
(e.g. I cannot put some writable attribute to EbuildExecuter
since apparently some python hack is used in portage which I
do not understand).
If there is interest, I can post my patches so far. Where?

What is not included in these patches, and I will probably
not find the time to include it:

1. The actual skipping of phases (since all my attempts to
   set some flag led to the python error that all
   attributes of EbuildExecuter are readonly :( )
2. The modification of the merge phase to not regenerate
   CONTENTS
3. Reporting an error if minor versions are used with
   non-matching EAPI. Surprisingly, this needs a bigger
   rewrite, since the code generating the regular expression
   for the version handling is currently not appropriate to
   such an extension.
4. Updating of .tbz2 files - this is certainly a bigger work.
5. In particular, the patches could not be tested yet...

I would suggest to postpone 3. and 4. until the decision
is made...

Re: [gentoo-dev] Re: don't rely on dynamic deps

[Alexander Berntsen]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

On 22/07/14 20:40, Martin Vaeth wrote:
> If there is interest, I can post my patches so far. Where?
If you think these patches are useful for Portage, please send them to
dev-portage@gentoo.org.
- -- 
Alexander
bernalex@gentoo.org
https://secure.plaimi.net/~alexander
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/

iF4EAREIAAYFAlPOslgACgkQRtClrXBQc7X58QD7BCSHxg3yiSynXe4EKpYk8R/n
pZKQPCoJqQXFtkFyU/0A/2BTRMm8T60AzHFNlaeKudRPQ4EC8ACbkP8+v4C6XVoW
=0GbF
-----END PGP SIGNATURE-----

[gentoo-dev] Re: don't rely on dynamic deps

[Martin Vaeth]

Alexander Berntsen <bernalex@gentoo.org> wrote:
> If you think these patches are useful for Portage, please send them to
> dev-portage@gentoo.org.

I submitted them to the gentoo.portage.devel mailing list as was
recommended to me in a pm.  I am sorry that due to lack of time
and experience with python, I am currently unable to finish the
work properly as it should be.
However, I think it should become visible that the whole thing
can be done rather unintrusive, even if I should have
missed some places and one or two patches still are needed.

[gentoo-dev] Re: don't rely on dynamic deps

[Ulrich Mueller]

[-- Attachment #1: Type: text/plain, Size: 1151 bytes --]

>>>>> On Tue, 22 Jul 2014, Martin Vaeth wrote:

> Ian Stakenvicius <axs@gentoo.org> wrote:
>> 
>> The main issue that I see is
>> - -how- VDB should be adjusted based on what changes are made to the
>> ebuilds.  For instance, if minimum versions of deps are adjusted
>> in-place, should vdb be updated to match?  what happens if the minimum
>> version of the currently-installed dep is below the new one?  etc. etc.

> All these problems disappear with minor revisions:
> You have to "install" the minor revisions just like any major revision,
> just that some phases will be shortcut.
> In particular, if the new dependencies are not satisfied, you get
> conflicts as usual if you would want to upgrade to a new version
> with dependencies not being satisfied.

Other problems appear, though. Documentation is installed in a ${PF}
subdir, so install locations actually do change when updating the
minor revision. Also some method for updating binary packages would be
needed.

All in all, I'm not convinced if the cure wouldn't be worse than the
disease here. It would introduce another level of complexity, in order
to avoid a few rebuilds.

Ulrich

[-- Attachment #2: Type: application/pgp-signature, Size: 490 bytes --]

[gentoo-dev] Re: don't rely on dynamic deps

[Martin Vaeth]

Ulrich Mueller <ulm@gentoo.org> wrote:
>
> Other problems appear, though. Documentation is installed in a ${PF}
> subdir, so install locations actually do change when updating the
> minor revision.

Yes, the minor revisions should not be exported into the variables
of ebuild.sh.  I had forgotten this.

> Also some method for updating binary packages would be needed.

I already mentioned that. But even if this is not implemented,
this is only a minor issue.

> All in all, I'm not convinced if the cure wouldn't be worse than the
> disease here.

The disease is making the distribution almost useless
or having broken dependencies.

> It would introduce another level of complexity, in order
> to avoid a few rebuilds.

It seems you never counted the number of silent modifications
to the tree: Just compare the number of changed packages in
metadata/ with the number of packages shown by eix-sync...
I would guess it means roughly that you have to recompile your
whole installation once a week, 95% of the rebuilds being due to
not fixing the package manager to work properly with dynamic deps.

Actually, I still think that implementing dynamic deps correctly
would be better, but minor revisions do not exclude this
and are probably simpler to implement.

[gentoo-dev] Re: don't rely on dynamic deps

[Ulrich Mueller]

[-- Attachment #1: Type: text/plain, Size: 947 bytes --]

>>>>> On Tue, 22 Jul 2014, Martin Vaeth wrote:

> Ulrich Mueller <ulm@gentoo.org> wrote:
>> Other problems appear, though. Documentation is installed in a
>> ${PF} subdir, so install locations actually do change when updating
>> the minor revision.

> Yes, the minor revisions should not be exported into the variables
> of ebuild.sh.  I had forgotten this.

That doesn't help. Imagine there is cat/foo-1 and the minor revision
is bumped. The new ebuild is cat/foo-1-r0.1 then, and PF changes even
if the minor revision is ignored (namely, from foo-1 to foo-1-r0).

> Actually, I still think that implementing dynamic deps correctly
> would be better, but minor revisions do not exclude this
> and are probably simpler to implement.

The PM could just update the VDB whenever it detects that the metadata
of the ebuild has changed (relative to the VDB). You can get that
without introducing the additional complication of minor revisions.

Ulrich

[-- Attachment #2: Type: application/pgp-signature, Size: 490 bytes --]

[gentoo-dev] Re: don't rely on dynamic deps

[Martin Vaeth]

Ulrich Mueller <ulm@gentoo.org> wrote:
> The new ebuild is cat/foo-1-r0.1 then, and PF changes even
> if the minor revision is ignored (namely, from foo-1 to foo-1-r0).

PF has to be filled correctly, of course:
The versions foo-1 and foo-1-r0 are identical according to PMS
and should thus lead to the same $PF.

If this is currently not happening in portage, this is a bug
which should be fixed, independently of whether minor revisions
are introduced or not.

Note that PR=0 in both cases.

BTW: Even if PF would have the wrong value, this would not do
any harm for minor revisions: If the minor revision has just
changed, nothing is merged to the filesystem and /var/db/*/*/CONTENTS
is unchanged, either. So nothing breaks in this case.
If the user re-emerges the minor revision (and if PF would have
the wrong value), then the minor revision would be visible to the
user in the sense that the name of the Doc-Directory is influenced
(and after a further minor revision bump, this revision number
would be outdated). An arguable cosmetical issue,
but nothing breaks, either.

>> Actually, I still think that implementing dynamic deps correctly
>> would be better, but minor revisions do not exclude this
>> and are probably simpler to implement.
>
> The PM could just update the VDB whenever it detects that the metadata
> of the ebuild has changed (relative to the VDB). You can get that
> without introducing the additional complication of minor revisions.

...but by introducing all the additional complications Ian
has mentioned. More precisely: What happens if new dependencies
are introduced which are not satisfied?
One has to face it: Portage must not just silently "update" the
database and thus silently produce a /var/db which does not
satisfy its own dependencies...

[gentoo-dev] Re: don't rely on dynamic deps

[Ulrich Mueller]

[-- Attachment #1: Type: text/plain, Size: 758 bytes --]

>>>>> On Tue, 22 Jul 2014, Martin Vaeth wrote:

> PF has to be filled correctly, of course:
> The versions foo-1 and foo-1-r0 are identical according to PMS
> and should thus lead to the same $PF.

This is not so. These versions are equal in comparision, so they
cannot be in the tree at the same time. However, PF will be different
for them.

Note that this also occurs elsewhere. For example, versions 1.0.2 and
1.000.2 cound as equal, but again PF will be different.

> If this is currently not happening in portage, this is a bug
> which should be fixed, independently of whether minor revisions
> are introduced or not.

Portage behaves as PMS specifies. PF is derived from the ebuild's
name, without any "normalisation" to a canonical version.

Ulrich

[-- Attachment #2: Type: application/pgp-signature, Size: 490 bytes --]

Re: [gentoo-dev] Re: don't rely on dynamic deps

[Andreas K. Huettel]

Am Dienstag 22 Juli 2014, 22:40:03 schrieb Ulrich Mueller:
> >>>>> On Tue, 22 Jul 2014, Martin Vaeth wrote:
> > PF has to be filled correctly, of course:
> > The versions foo-1 and foo-1-r0 are identical according to PMS
> > and should thus lead to the same $PF.
> 
> This is not so. These versions are equal in comparision, so they
> cannot be in the tree at the same time. However, PF will be different
> for them.

Well we'd need a new EAPI for this anyway. So we might as well redefine -r0 
there. 

-- 
Andreas K. Huettel
Gentoo Linux developer
kde, council

Re: [gentoo-dev] Re: don't rely on dynamic deps

[Ian Stakenvicius]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

On 22/07/14 04:51 PM, Andreas K. Huettel wrote:
> Am Dienstag 22 Juli 2014, 22:40:03 schrieb Ulrich Mueller:
>>>>>>> On Tue, 22 Jul 2014, Martin Vaeth wrote:
>>> PF has to be filled correctly, of course: The versions foo-1
>>> and foo-1-r0 are identical according to PMS and should thus
>>> lead to the same $PF.
>> 
>> This is not so. These versions are equal in comparision, so they 
>> cannot be in the tree at the same time. However, PF will be
>> different for them.
> 
> Well we'd need a new EAPI for this anyway. So we might as well
> redefine -r0 there.
> 

I still don't follow why we need new EAPI for this, as presented.
What we are talking about here is optional PM behaviour only, and a
convention that developers will need to adopt.  It doesn't much matter
if a PM doesn't implement minor-revision-vdbonly-merging because that
PM would just do a full re-emerge same as any other revbump.

The only need for EAPI change that I can see is to allow non-integer
revision values, but that wasn't on mva's list of changes from what I
remember.  Am I missing something else, here?

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.22 (GNU/Linux)

iF4EAREIAAYFAlPPtWIACgkQ2ugaI38ACPCjBQD+K0aQW3lJqVUJTo1nO9nnFlsY
NfrgaIuu6eescdN6FDkBALwizKGBI4I0iSmj2ywis/4OTNsvFBQm9sxywXq7HFz1
=3Ajb
-----END PGP SIGNATURE-----

[gentoo-dev] Re: don't rely on dynamic deps

[Martin Vaeth]

Ian Stakenvicius <axs@gentoo.org> wrote:
>
> The only need for EAPI change that I can see is to allow non-integer
> revision values

A change of the version/-revision format certainly requires
an EAPI change; one must also define rules how to compare
these versions, etc. (despite it is obvious, here).

Please be aware that on the corresponding bug there is
already the request to keep minor revisions for sub-distributions
of gentoo; so maybe instead of minor revisions one should
think about anpther suffix (besides -r...) or another format:
This would simulateneously avoid the problem of $PF
discussed earlier - by definition, this additional suffix
would not appear in $PF.

Re: [gentoo-dev] Re: don't rely on dynamic deps

[Rich Freeman]

On Tue, Jul 22, 2014 at 4:06 PM, Martin Vaeth <martin@mvath.de> wrote:
> ...but by introducing all the additional complications Ian
> has mentioned. More precisely: What happens if new dependencies
> are introduced which are not satisfied?
> One has to face it: Portage must not just silently "update" the
> database and thus silently produce a /var/db which does not
> satisfy its own dependencies...

While this is problematic, I think portage actually can handle this
(but I haven't tested this recently).  Portage already allows you to
clean a package without its reverse deps leading to a system in
exactly the state you describe.  I believe portage will just try to
bring the package back at the next emerge @world (or any other set
containing the reverse dep).

If there is a problem with the dependency version then the system is
already broken anyway - portage just doesn't realize it.

I think that allowing devs to instruct portage to update VDB with
USE/dep/etc changes is potentially less problematic than having
portage trying to guess what the right thing to do is.  We could then
either use that feature or revbump as appropriate under the specific
circumstances.

Ultimately I think the most important thing we need is for PMs to
follow some kind of defined behavior.  In our efforts to get portage
to do the "right thing" we sometimes end up with a portage that does
things that nobody really understands.  Things like that tend to lead
to convenience 95% of the time and head-banging the other 5%.

I'm all for workarounds, but I'd advocate simple ones that lead to
easily predicted behavior (and failure modes).  I'd rather safely
eliminate 70% of useless rebuilds than unsafely try to eliminate 90%
of them.  However, I do agree that we need to be sensitive to
rebuilds.

Rich

[gentoo-dev] Re: don't rely on dynamic deps

[Duncan]

Rich Freeman posted on Tue, 22 Jul 2014 21:15:01 -0400 as excerpted:

> On Tue, Jul 22, 2014 at 4:06 PM, Martin Vaeth <martin@mvath.de> wrote:
>> ...but by introducing all the additional complications Ian has
>> mentioned. More precisely: What happens if new dependencies are
>> introduced which are not satisfied?
>> One has to face it: Portage must not just silently "update" the
>> database and thus silently produce a /var/db which does not satisfy its
>> own dependencies...
> 
> While this is problematic, I think portage actually can handle this (but
> I haven't tested this recently).  Portage already allows you to clean a
> package without its reverse deps leading to a system in exactly the
> state you describe.  I believe portage will just try to bring the
> package back at the next emerge @world (or any other set containing the
> reverse dep).

FWIW I tested this with a number of packages just the other day while 
doing an initial test-build of qt:5 and kde:5 from the relevant 
overlays.  Various kde-workspace5/plasma5 packages can't coexist with the 
kde:4 versions, but I didn't want to remove the kde:4 revdeps or set-
elements from my installed sets until I was sure the kde:5 versions 
worked fine.

Apparently the kwin:5 version I was testing doesn't like my radeon turks 
(hd6670 IIRC) hardware their current v3.16-pre kernel drm drivers as once 
I had the whole setup installed and tried to startx with it, kwin ended 
up in a segfault/respawn cycle, so my surgical unmerge of specific kde:4 
packages was just as well, making it relatively easy to simply emerge -k 
@world and get back to a working system from the binpkgs, automatically 
unmerging the kde:5 blockers on the way since I hadn't actually let 
portage put them in @world just yet.

Yes, a simple emerge --deep @world pulls everything missing but needed 
back in, so that bit of portage at least seems to be working just fine.

-- 
Duncan - List replies preferred.   No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master."  Richard Stallman

[gentoo-dev] Re: don't rely on dynamic deps

[Martin Vaeth]

Rich Freeman <rich0@gentoo.org> wrote:
> On Tue, Jul 22, 2014 at 4:06 PM, Martin Vaeth <martin@mvath.de> wrote:
>> ...but by introducing all the additional complications Ian
>> has mentioned. More precisely: What happens if new dependencies
>> are introduced which are not satisfied?
>> One has to face it: Portage must not just silently "update" the
>> database and thus silently produce a /var/db which does not
>> satisfy its own dependencies...
>
> While this is problematic, I think portage actually can handle this
> (but I haven't tested this recently).

The problem here arises if new dependencies with automatic
subslots (foo/bar:=) are added which are not yet installed:
Portage cannot fill these correctly.

Solving all these difficulties appears harder to me than
implementing dynamic deps correctly.

> I think that allowing devs to instruct portage to update VDB with
> USE/dep/etc changes is potentially less problematic than having
> portage trying to guess what the right thing to do is.

I completely agree. The idea of minor revisions is just one of
several possibilities to achieve this; there are several others
(e.g. another metadata variable describing which versions
can be updated by skipping the prepare/config/compile/merge
phases). The implementation, however, would be rather similar:
Usually you need a new EAPI (or in case of noninvasive change
might discuss whether to change EAPI's retroactively), and
some "reemerging without the time-consuming phases"
must be implemented.

Re: [gentoo-dev] don't rely on dynamic deps

[Tom Wijsman]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Tue, 22 Jul 2014 09:53:49 -0400
Ian Stakenvicius <axs@gentoo.org> wrote:

> Using ${PVR} to detect how portage should update things
> would be asking for trouble, imo.

This entire sub thread reads like a dynamic dependencies alternative in
disguise, the difference lies in an increase of the level of control
and in the place where this then gets reimplemented.

The increase of control comes from the maintainer being able to decide
whether the dependencies in the vdb are updated or not; however, this
gives rise to a mindset where you consider this level of control for
other variables as well (which syntax we'll [ab]use for that?) as well
as end up with more ebuilds for the sake of updating vdb dependencies.

Using an extension like -rX.Y seems odd; at the very least, I think an
incremental variable or something along that line in the ebuild would
work better. This allows for array usage like VERSION[dependencies]=1,
thus allowing other variables to be dynamic as well; you compare that
number against the one in the vdb, bingo...

Or is it just a figment?

Please think a design through and don't take a cheap shot with -rX.Y.

- -- 
With kind regards,

Tom Wijsman (TomWij)
Gentoo Developer

E-mail address  : TomWij@gentoo.org
GPG Public Key  : 6D34E57D
GPG Fingerprint : C165 AF18 AB4C 400B C3D2  ABF0 95B2 1FCD 6D34 E57D
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2

iQEcBAEBAgAGBQJTzulZAAoJEPWZc8roOL/QzSMH/0wrGF+6joDtUlmUiNuTZBHB
Y0aYkr+Je7R4jj+NQxMY08j+odyImgnT+IrNrQcs7VEboXkrKS0s7ZEmQqCpNvmp
vVLvGUeAlzPgGz31aKIzMBhe5TuyCTwOvArU+DVcxDEktcbHP+sDBPTojQAr932e
qJtf6fdXDaUklu+MPlNofroREd2hjUrkS34ll6W5E+KizNMfRO7n4SAN38pkkE+C
4t3elp1I2Eei7HQT/cNUY78ab8Sgiy6N5CryN73zx6jyw9EwShLFV/8VN3M9SJ1B
jvBmD01EjsW6FLz6fvUPy2dz4GBKaC4YY0qhK5XocBwROUu2yCGV/w/es1JROB0=
=xuKt
-----END PGP SIGNATURE-----

Re: [gentoo-dev] don't rely on dynamic deps

[Ian Stakenvicius]

Sent from an iPhone, sorry for the HTML...

> On Jul 22, 2014, at 6:44 PM, Tom Wijsman <TomWij@gentoo.org> wrote:
> 
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
> 
> On Tue, 22 Jul 2014 09:53:49 -0400
> Ian Stakenvicius <axs@gentoo.org> wrote:
> 
>> Using ${PVR} to detect how portage should update things
>> would be asking for trouble, imo.
> 
> This entire sub thread reads like a dynamic dependencies alternative in
> disguise, the difference lies in an increase of the level of control
> and in the place where this then gets reimplemented.
> 
> The increase of control comes from the maintainer being able to decide
> whether the dependencies in the vdb are updated or not; however, this
> gives rise to a mindset where you consider this level of control for
> other variables as well (which syntax we'll [ab]use for that?) as well
> as end up with more ebuilds for the sake of updating vdb dependencies.
> 
> Using an extension like -rX.Y seems odd; at the very least, I think an
> incremental variable or something along that line in the ebuild would
> work better. This allows for array usage like VERSION[dependencies]=1,
> thus allowing other variables to be dynamic as well; you compare that
> number against the one in the vdb, bingo...
> 
> Or is it just a figment?
> 
> Please think a design through and don't take a cheap shot with -rX.Y.
> 

The thing about -rX.Y is that it allows this new-dynamic-deps thing to act like a regular rev bump to any PM that doesn't bother to implement it (or dynamic deps for that matter).  Instant backwards-compatibility is a handy feature.

Re: [gentoo-dev] don't rely on dynamic deps

[Ciaran McCreesh]

[-- Attachment #1: Type: text/plain, Size: 396 bytes --]

On Tue, 22 Jul 2014 20:01:55 -0400
Ian Stakenvicius <axs@gentoo.org> wrote:
> The thing about -rX.Y is that it allows this new-dynamic-deps thing
> to act like a regular rev bump to any PM that doesn't bother to
> implement it (or dynamic deps for that matter).  Instant
> backwards-compatibility is a handy feature.

...but it doesn't actually solve the problem.

-- 
Ciaran McCreesh

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 181 bytes --]

[gentoo-dev] Re: don't rely on dynamic deps

[Martin Vaeth]

Ciaran McCreesh <ciaran.mccreesh@googlemail.com> wrote:
> Ian Stakenvicius <axs@gentoo.org> wrote:
>> The thing about -rX.Y is that it allows this new-dynamic-deps thing
>> to act like a regular rev bump to any PM that doesn't bother to
>> implement it (or dynamic deps for that matter).  Instant
>> backwards-compatibility is a handy feature.
>
> ...but it doesn't actually solve the problem.

Neither do revbumps.
Both, dynamic and static deps are broken.
They are broken in different ways, but both are broken.

So the only reason which might justify changing the
policy is that current portage *implementation* of
dynamic deps is broken.

However, if it should actually be decided to have
some hundreds reemerges every week, at least this
should be implemented in a way that it is not so
time-consuming.

Re: [gentoo-dev] Re: don't rely on dynamic deps

[Ciaran McCreesh]

[-- Attachment #1: Type: text/plain, Size: 288 bytes --]

On Sat, 26 Jul 2014 13:00:31 +0000 (UTC)
Martin Vaeth <martin@mvath.de> wrote:
> Both, dynamic and static deps are broken.
> They are broken in different ways, but both are broken.

You keep using that word. I do not think it means what you think it
means.

-- 
Ciaran McCreesh

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 181 bytes --]

Re: [gentoo-dev] don't rely on dynamic deps

[Ian Stakenvicius]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

On 22/07/14 06:44 PM, Tom Wijsman wrote:
> On Tue, 22 Jul 2014 09:53:49 -0400 Ian Stakenvicius
> <axs@gentoo.org> wrote:
> 
>> Using ${PVR} to detect how portage should update things would be
>> asking for trouble, imo.
> 
> This entire sub thread reads like a dynamic dependencies
> alternative in disguise, the difference lies in an increase of the
> level of control and in the place where this then gets
> reimplemented.


It is.

Here's the situation as I see it -- the portage tree needs to be
consistent at snapshot time.  But things can change all over the
place, deps are moved, virtuals replace single or groups of atoms,
packages get split, etc. etc. etc.

Dynamic deps are the best solution outside of the (rather limited)
profiles/updates functions we have right now to allow us to make
whatever non-files-on-${ROOT} changes we need to make to the vdb.  So
realistically what we should be doing is either trying to work out a
better solution to dynamic deps (something that will failover nicely
for PMs that don't support dynamic deps) or perhaps adding more
functions to support VDB updating via profiles/updates/

Am I off-base here?  Thoughts?

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2

iF4EAREIAAYFAlPSbVgACgkQ2ugaI38ACPBDpAEAnqx8hBGkmmiVGE6Pz7Rh+BE9
ed5KuWwihJdjPGjXdjoA/ifwGD8oUO8epWIq4rahW+egUFhklKtPu57jIYSjY90y
=cZb0
-----END PGP SIGNATURE-----

Re: [gentoo-dev] don't rely on dynamic deps

[Ian Stakenvicius]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

On 25/07/14 10:44 AM, Ian Stakenvicius wrote:
> On 22/07/14 06:44 PM, Tom Wijsman wrote:
>> On Tue, 22 Jul 2014 09:53:49 -0400 Ian Stakenvicius 
>> <axs@gentoo.org> wrote:
> 
>>> Using ${PVR} to detect how portage should update things would
>>> be asking for trouble, imo.
> 
>> This entire sub thread reads like a dynamic dependencies 
>> alternative in disguise, the difference lies in an increase of
>> the level of control and in the place where this then gets 
>> reimplemented.
> 
> 
> It is.
> 
> Here's the situation as I see it -- the portage tree needs to be 
> consistent at snapshot time.  But things can change all over the 
> place, deps are moved, virtuals replace single or groups of atoms, 
> packages get split, etc. etc. etc.
> 
> Dynamic deps are the best solution outside of the (rather limited) 
> profiles/updates functions we have right now to allow us to make 
> whatever non-files-on-${ROOT} changes we need to make to the vdb.
> So realistically what we should be doing is either trying to work
> out a better solution to dynamic deps (something that will failover
> nicely for PMs that don't support dynamic deps) or perhaps adding
> more functions to support VDB updating via profiles/updates/
> 
> Am I off-base here?  Thoughts?
> 

Ignore this, i should've read the rest of the thread first before posting.

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2

iF4EAREIAAYFAlPScN4ACgkQ2ugaI38ACPBS5gD+MXU3VUvwhp1u/0wIDHeXEQdX
TmJXhvDhuhuE+7ehee0A/1HGASXipYsejfJxPesQFO4Egs1Yzj20PXlVmil9H8FY
=WwNJ
-----END PGP SIGNATURE-----

Re: [gentoo-dev] don't rely on dynamic deps

[hasufell]

Ian Stakenvicius:
> Dynamic deps are the best solution outside of the (rather limited)
> profiles/updates functions we have right now to allow us to make
> whatever non-files-on-${ROOT} changes we need to make to the vdb.  So
> realistically what we should be doing is either trying to work out a
> better solution to dynamic deps (something that will failover nicely
> for PMs that don't support dynamic deps) or perhaps adding more
> functions to support VDB updating via profiles/updates/
> 
> Am I off-base here?  Thoughts?
> 
> 

Yes, as was already explained. Those are currently just dreams or
abstract thoughts.

Dynamics deps are already broken, not consistently enabled (e.g. when
subslots are in use), optional and not defined in PMS.

People really don't seem to understand what is going on here. We rely on
behavior that depends not only on a portage specific feature, but also
on the context and can pretty much be considered undefined.

I guess I have to repost
https://wiki.gentoo.org/wiki/Project:Portage/Dynamic_dependencies

Dynamic deps don't work for you, even if you think that. Coming up with
an alternative approach will probably take a lot of effort and shouldn't
be considered a blocker to fix a fundamental bug in dependency
calculation, which is already broken in portage in many ways.

If we already bikeshed about one of the simplest ways to improve
dependency calculation, I wonder what will happen if someone wants to
actually fix it.

Everyone else who thinks got an idea on how to fix dynamic deps support
(or similar) should:
* write a PMS patch and get it merged
* join the portage team and volunteer to implement it instead of yelling
at them

Re: [gentoo-dev] don't rely on dynamic deps

[Ciaran McCreesh]

[-- Attachment #1: Type: text/plain, Size: 557 bytes --]

On Fri, 25 Jul 2014 15:09:55 +0000
hasufell <hasufell@gentoo.org> wrote:
> Everyone else who thinks got an idea on how to fix dynamic deps
> support (or similar) should:
> * write a PMS patch and get it merged
> * join the portage team and volunteer to implement it instead of
> yelling at them

That's not really helpful advice: dynamic dependencies can't be fixed.
Instead, you should say that anyone who thinks they have an idea on how
to fix dynamic deps should think about it until they understand why
it's wrong...

-- 
Ciaran McCreesh

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 181 bytes --]

Re: [gentoo-dev] don't rely on dynamic deps

[hasufell]

Ciaran McCreesh:
> On Fri, 25 Jul 2014 15:09:55 +0000
> hasufell <hasufell@gentoo.org> wrote:
>> Everyone else who thinks got an idea on how to fix dynamic deps
>> support (or similar) should:
>> * write a PMS patch and get it merged
>> * join the portage team and volunteer to implement it instead of
>> yelling at them
> 
> That's not really helpful advice: dynamic dependencies can't be fixed.
> Instead, you should say that anyone who thinks they have an idea on how
> to fix dynamic deps should think about it until they understand why
> it's wrong...
> 

I was rather talking about the "fix useless rebuilds" issue. It's a
valid point.

What would you suggest? Can the VDB be fixed in another way to avoid
such rebuilds?

Re: [gentoo-dev] don't rely on dynamic deps

[Ciaran McCreesh]

[-- Attachment #1: Type: text/plain, Size: 891 bytes --]

On Fri, 25 Jul 2014 15:23:58 +0000
hasufell <hasufell@gentoo.org> wrote:
> > That's not really helpful advice: dynamic dependencies can't be
> > fixed. Instead, you should say that anyone who thinks they have an
> > idea on how to fix dynamic deps should think about it until they
> > understand why it's wrong...
> 
> I was rather talking about the "fix useless rebuilds" issue. It's a
> valid point.

It's like saying "OK, the house is on fire, but I really like the
wallpaper, so we should just stay in the building"...

> What would you suggest? Can the VDB be fixed in another way to avoid
> such rebuilds?

There are ways of doing it, if you're prepared to make ebuild authors
put in an awful lot of work for very little gain. But it shouldn't be a
priority, and we need to fix existing breakages before doing something
ambitious like that.

-- 
Ciaran McCreesh

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 181 bytes --]

Re: [gentoo-dev] don't rely on dynamic deps

[hasufell]

Ciaran McCreesh:
> On Fri, 25 Jul 2014 15:23:58 +0000
> hasufell <hasufell@gentoo.org> wrote:
>>> That's not really helpful advice: dynamic dependencies can't be
>>> fixed. Instead, you should say that anyone who thinks they have an
>>> idea on how to fix dynamic deps should think about it until they
>>> understand why it's wrong...
>>
>> I was rather talking about the "fix useless rebuilds" issue. It's a
>> valid point.
> 
> It's like saying "OK, the house is on fire, but I really like the
> wallpaper, so we should just stay in the building"...
> 
>> What would you suggest? Can the VDB be fixed in another way to avoid
>> such rebuilds?
> 
> There are ways of doing it, if you're prepared to make ebuild authors
> put in an awful lot of work for very little gain. But it shouldn't be a
> priority, and we need to fix existing breakages before doing something
> ambitious like that.
> 

I agree. After all, we have *-bin packages for stuff like libreoffice,
pypy, firefox etc anyway if rebuilding is really a problem for someone.

That said, it might maybe even make more sense to provide an official
binhost, although that is probably not less ambitious as fixing VDB on
the fly.

Sounds like that would be an interesting gentoo project. But afais PMS
doesn't really specify how binary packages should look like, so we will
hit incompatibility problems there as well.

[gentoo-dev] Re: don't rely on dynamic deps

[Duncan]

hasufell posted on Fri, 25 Jul 2014 15:44:47 +0000 as excerpted:

> Sounds like that would be an interesting gentoo project. But afais PMS
> doesn't really specify how binary packages should look like, so we will
> hit incompatibility problems there as well.

AFAIK binpkgs are purely an individual PM feature, considered outside the 
domain of PMS.

Unless things have changed, paludis doesn't support them at least in 
portage form and there's active antipathy toward adding that (altho I 
believe there was discussion of adding rpm support at one point).

I don't know whether pkgcore supports them or not, tho if it does, I 
suspect its support is close to the portage-native form.  But while I 
believe someone's working on pkgcore again now, until it gets EAPI-5 
support it's pretty much out of the picture anyway.

Gentoo did at one point do binpkg ISO-images, but I've not seen or heard 
anything of that in years, and of course while they did form a convenient 
quick-install foundation, they were quickly outdated, and there was no 
gentoo-only mechanism to continue with binpkgs -- you did your quick-
install from binpkg, optionally changed any USE flags you wanted and 
rebuilt using --newuse, and continued with conventional gentoo build-from-
source after that.


Meanwhile, it's worth noting that a mostly from-sources distro such as 
gentoo has the luxury of bypassing many of the legal issues involved with 
binary distribution.  Among other things there's the patent issues that 
don't normally apply (at least in the US) to source distribution due to 
freedom-of-speech-overrides, and the gpl source provision (including our 
patches) obligations as well.

Partially for that reason, gentoo as a distribution has in the past 
chosen to deemphasize binaries and leave most of the binary distribution 
angle to the gentoo-based distros.  That lets us continue focusing on 
what we do best, while leaving the gentoo-based distros a bit more space 
to work on what they can do better.  While there'd certainly be some 
convenience to a binaries server, is it really going to be worth the 
cost, in legal hassle, in blurring our sources focus, and in killing that 
exclusive niche for our downstream distros?

Meanwhile, a question for the infra and foundation folks:  A quick look 
at the the download links and mirrors says that we're still distributing 
10.1 images for at least x86 and amd64.  Based on the file-dates on the 
mirrors, that was 2009, and I don't see corresponding links to sources, 
which means at least for the gplv2 binaries on those images, we're 
obligated to provide sources, including our patches, until 2017 (three 
years from now) and counting.  While the ebuilds and tarballs aren't 
likely to be a huge issue, are we sure we have those patches archived and 
will until three years after we quit distributing those binaries, such 
that we can provide them on-demand?  If not, those images need to come 
down.

-- 
Duncan - List replies preferred.   No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master."  Richard Stallman

[gentoo-dev] Re: don't rely on dynamic deps

[Martin Vaeth]

hasufell <hasufell@gentoo.org> wrote:
>
> Dynamics deps are already broken, not consistently enabled (e.g. when
> subslots are in use)

Just to make it clear: No, dynamic deps are not broken.

What is broken is that portage does not use them consistently.

It would be a rather bad idea to change policy just because of this
portage bug and force users to permanently do unnecessary
recompilations. At least, for me, it would mean that I have
to change distribution, since I cannot afford this.

> optional and not defined in PMS.

Static deps are also optional and not defined in PMS.

In fact, PMS makes no claim *where* to read the DEP strings from;
it only specified how they are to be stored in the tree.

Quite the opposite, PMS claims that one cannot rely on
anything stored in /var/db

Re: [gentoo-dev] Re: don't rely on dynamic deps

[Ciaran McCreesh]

[-- Attachment #1: Type: text/plain, Size: 1090 bytes --]

On Sat, 26 Jul 2014 12:41:16 +0000 (UTC)
Martin Vaeth <martin@mvath.de> wrote:
> hasufell <hasufell@gentoo.org> wrote:
> > Dynamics deps are already broken, not consistently enabled (e.g.
> > when subslots are in use)
> 
> Just to make it clear: No, dynamic deps are not broken.

Yes they are.

> What is broken is that portage does not use them consistently.

Because using them consistently is impossible by design.

> It would be a rather bad idea to change policy just because of this
> portage bug and force users to permanently do unnecessary
> recompilations. At least, for me, it would mean that I have
> to change distribution, since I cannot afford this.

This is not a Portage bug.
 
> > optional and not defined in PMS.
> 
> Static deps are also optional and not defined in PMS.
> 
> In fact, PMS makes no claim *where* to read the DEP strings from;
> it only specified how they are to be stored in the tree.

Incorrect.

> Quite the opposite, PMS claims that one cannot rely on
> anything stored in /var/db

Incorrect.

-- 
Ciaran McCreesh

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 181 bytes --]

[gentoo-dev] Re: don't rely on dynamic deps

[Martin Vaeth]

Ciaran McCreesh <ciaran.mccreesh@googlemail.com> wrote:
> Martin Vaeth <martin@mvath.de> wrote:
>> hasufell <hasufell@gentoo.org> wrote:
>> > Dynamics deps are already broken, not consistently enabled (e.g.
>> > when subslots are in use)
>> Just to make it clear: No, dynamic deps are not broken.
>
> Yes they are.

Could you please stop your childish behaviour?

>> What is broken is that portage does not use them consistently.
>
> Because using them consistently is impossible by design.

It seems that *you* should take some reading before you
continue with discussion.

Re: [gentoo-dev] Re: don't rely on dynamic deps

[hasufell]

Martin Vaeth:
> Ciaran McCreesh <ciaran.mccreesh@googlemail.com> wrote:
>> Martin Vaeth <martin@mvath.de> wrote:
>>> hasufell <hasufell@gentoo.org> wrote:
>>>> Dynamics deps are already broken, not consistently enabled (e.g.
>>>> when subslots are in use)
>>> Just to make it clear: No, dynamic deps are not broken.
>>
>> Yes they are.
> 
> Could you please stop your childish behaviour?
> 
>>> What is broken is that portage does not use them consistently.
>>
>> Because using them consistently is impossible by design.
> 
> It seems that *you* should take some reading before you
> continue with discussion.
> 
> 

I think no one cares about this part of the thread anymore. Can you take
it elsewhere?

Re: [gentoo-dev] Re: don't rely on dynamic deps

[Ciaran McCreesh]

[-- Attachment #1: Type: text/plain, Size: 347 bytes --]

On Sat, 26 Jul 2014 15:04:31 +0000 (UTC)
Martin Vaeth <martin@mvath.de> wrote:
> It seems that *you* should take some reading before you
> continue with discussion.

I wrote PMS, the dev manual and a package manager... I understand the
issues involved. If you want to contribute, you should at least read
the spec.

-- 
Ciaran McCreesh

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 181 bytes --]

Re: [gentoo-dev] Re: don't rely on dynamic deps

[Samuli Suominen]

On 26/07/14 15:49, Ciaran McCreesh wrote:
> On Sat, 26 Jul 2014 12:41:16 +0000 (UTC)
> Martin Vaeth <martin@mvath.de> wrote:
>> hasufell <hasufell@gentoo.org> wrote:
>>> Dynamics deps are already broken, not consistently enabled (e.g.
>>> when subslots are in use)
>> Just to make it clear: No, dynamic deps are not broken.
> Yes they are.

We just succesfully converted ~300 ebuilds in tree without revision
bumps from virtual/udev[gudev,introspection,static-libs]
to virtual/libudev and virtual/libgudev
Tested it on multiple boxes, went fine. Nobody has filed bugs at
http://bugs.gentoo.org/, nobody has filed a single forums post,
nobody has said anything at #gentoo, Freenode
Only one person said he had to manually build 2 GNOME related packages,
simple-scan and something else

So, broken? Far from it. More like essential feature.

People have just listed some known races dynamic deps have, and I take
those races anyday over an regression that causes
endless rebuilding...

- Samuli

Re: [gentoo-dev] Re: don't rely on dynamic deps

[hasufell]

Samuli Suominen:
> 
> On 26/07/14 15:49, Ciaran McCreesh wrote:
>> On Sat, 26 Jul 2014 12:41:16 +0000 (UTC)
>> Martin Vaeth <martin@mvath.de> wrote:
>>> hasufell <hasufell@gentoo.org> wrote:
>>>> Dynamics deps are already broken, not consistently enabled (e.g.
>>>> when subslots are in use)
>>> Just to make it clear: No, dynamic deps are not broken.
>> Yes they are.
> 
> We just succesfully converted ~300 ebuilds in tree without revision
> bumps from virtual/udev[gudev,introspection,static-libs]
> to virtual/libudev and virtual/libgudev
> Tested it on multiple boxes, went fine. Nobody has filed bugs at
> http://bugs.gentoo.org/, nobody has filed a single forums post,
> nobody has said anything at #gentoo, Freenode
> Only one person said he had to manually build 2 GNOME related packages,
> simple-scan and something else
> 
> So, broken? Far from it. More like essential feature.
> 
> People have just listed some known races dynamic deps have, and I take
> those races anyday over an regression that causes
> endless rebuilding...
> 

I'm not sure if you realize that you just disabled dynamic deps support
on most of those converted ebuilds.

Re: [gentoo-dev] Behaving productively on the list

[Andreas K. Huettel]

Am Sonntag, 27. Juli 2014, 13:50:43 schrieb hasufell:

> > So, broken? Far from it. More like essential feature.

> 
> I'm not sure if you realize that you just disabled dynamic deps support
> on most of those converted ebuilds.

PLEASE, everyone, don't just make statements like the the one above ^ but 
instead explain what has happened and best bring up a real-world example. 

"... of those converted ebuilds, because ....!
For example, see ... which you just changed as follows: ... This has the 
consequence that ..."

This will hopefully also cut down the pointless chatter "Yes you do. No you 
dont."


-- 

Andreas K. Huettel
Gentoo Linux developer 
dilfridge@gentoo.org
http://www.akhuettel.de/

Re: [gentoo-dev] Behaving productively on the list

[hasufell]

Andreas K. Huettel:
> Am Sonntag, 27. Juli 2014, 13:50:43 schrieb hasufell:
> 
>>> So, broken? Far from it. More like essential feature.
> 
>>
>> I'm not sure if you realize that you just disabled dynamic deps support
>> on most of those converted ebuilds.
> 
> PLEASE, everyone, don't just make statements like the the one above ^ but 
> instead explain what has happened and best bring up a real-world example. 
> 
> "... of those converted ebuilds, because ....!
> For example, see ... which you just changed as follows: ... This has the 
> consequence that ..."
> 
> This will hopefully also cut down the pointless chatter "Yes you do. No you 
> dont."
> 
> 

This is getting really silly. It was explained ~5 times in this thread
and the explanation was linked 2+ times here. Yet no one cares to read
up on the issue.

People who randomly post here without knowing what they talk about DONT
behave productively on the list.

You are blaming the wrong people.

Re: [gentoo-dev] Behaving productively on the list

[Andreas K. Huettel]

[-- Attachment #1: Type: Text/Plain, Size: 999 bytes --]

Am Sonntag, 27. Juli 2014, 14:04:00 schrieb hasufell:
> >> I'm not sure if you realize that you just disabled dynamic deps support
> >> on most of those converted ebuilds.
> > 
> > PLEASE, everyone, don't just make statements like the the one above ^ but
> > instead explain what has happened and best bring up a real-world example.
> > 
> > "... of those converted ebuilds, because ....!
> > For example, see ... which you just changed as follows: ... This has the
> > consequence that ..."
> > 
> > This will hopefully also cut down the pointless chatter "Yes you do. No
> > you dont."
> 
> This is getting really silly. It was explained ~5 times in this thread
> and the explanation was linked 2+ times here. Yet no one cares to read
> up on the issue.

Link to it again. It costs you 10sec at most. You make your point much more 
clearly. 

Also, the examples do help.

-- 

Andreas K. Huettel
Gentoo Linux developer 
dilfridge@gentoo.org
http://www.akhuettel.de/


[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 951 bytes --]

Re: [gentoo-dev] Behaving productively on the list

[hasufell]

Andreas K. Huettel:
> Am Sonntag, 27. Juli 2014, 14:04:00 schrieb hasufell:
>>>> I'm not sure if you realize that you just disabled dynamic deps support
>>>> on most of those converted ebuilds.
>>>
>>> PLEASE, everyone, don't just make statements like the the one above ^ but
>>> instead explain what has happened and best bring up a real-world example.
>>>
>>> "... of those converted ebuilds, because ....!
>>> For example, see ... which you just changed as follows: ... This has the
>>> consequence that ..."
>>>
>>> This will hopefully also cut down the pointless chatter "Yes you do. No
>>> you dont."
>>
>> This is getting really silly. It was explained ~5 times in this thread
>> and the explanation was linked 2+ times here. Yet no one cares to read
>> up on the issue.
> 
> Link to it again. It costs you 10sec at most. You make your point much more 
> clearly. 
> 
> Also, the examples do help.
> 

http://article.gmane.org/gmane.linux.gentoo.devel/92053
http://article.gmane.org/gmane.linux.gentoo.devel/92055
http://article.gmane.org/gmane.linux.gentoo.devel/92043
http://article.gmane.org/gmane.linux.gentoo.devel/92138
http://article.gmane.org/gmane.linux.gentoo.devel/92259

I think that's enough for now. And it cost me a few minutes to go
through the thread which is what people should do when they respond here.

Re: [gentoo-dev] Re: don't rely on dynamic deps

[Samuli Suominen]

On 27/07/14 14:50, hasufell wrote:
> Samuli Suominen:
>> On 26/07/14 15:49, Ciaran McCreesh wrote:
>>> On Sat, 26 Jul 2014 12:41:16 +0000 (UTC)
>>> Martin Vaeth <martin@mvath.de> wrote:
>>>> hasufell <hasufell@gentoo.org> wrote:
>>>>> Dynamics deps are already broken, not consistently enabled (e.g.
>>>>> when subslots are in use)
>>>> Just to make it clear: No, dynamic deps are not broken.
>>> Yes they are.
>> We just succesfully converted ~300 ebuilds in tree without revision
>> bumps from virtual/udev[gudev,introspection,static-libs]
>> to virtual/libudev and virtual/libgudev
>> Tested it on multiple boxes, went fine. Nobody has filed bugs at
>> http://bugs.gentoo.org/, nobody has filed a single forums post,
>> nobody has said anything at #gentoo, Freenode
>> Only one person said he had to manually build 2 GNOME related packages,
>> simple-scan and something else
>>
>> So, broken? Far from it. More like essential feature.
>>
>> People have just listed some known races dynamic deps have, and I take
>> those races anyday over an regression that causes
>> endless rebuilding...
>>
> I'm not sure if you realize that you just disabled dynamic deps support
> on most of those converted ebuilds.
>

There is a bug in the package manager, you mean.

Re: [gentoo-dev] Re: don't rely on dynamic deps

[hasufell]

Samuli Suominen:
> 
> On 27/07/14 14:50, hasufell wrote:
>> Samuli Suominen:
>>> On 26/07/14 15:49, Ciaran McCreesh wrote:
>>>> On Sat, 26 Jul 2014 12:41:16 +0000 (UTC)
>>>> Martin Vaeth <martin@mvath.de> wrote:
>>>>> hasufell <hasufell@gentoo.org> wrote:
>>>>>> Dynamics deps are already broken, not consistently enabled (e.g.
>>>>>> when subslots are in use)
>>>>> Just to make it clear: No, dynamic deps are not broken.
>>>> Yes they are.
>>> We just succesfully converted ~300 ebuilds in tree without revision
>>> bumps from virtual/udev[gudev,introspection,static-libs]
>>> to virtual/libudev and virtual/libgudev
>>> Tested it on multiple boxes, went fine. Nobody has filed bugs at
>>> http://bugs.gentoo.org/, nobody has filed a single forums post,
>>> nobody has said anything at #gentoo, Freenode
>>> Only one person said he had to manually build 2 GNOME related packages,
>>> simple-scan and something else
>>>
>>> So, broken? Far from it. More like essential feature.
>>>
>>> People have just listed some known races dynamic deps have, and I take
>>> those races anyday over an regression that causes
>>> endless rebuilding...
>>>
>> I'm not sure if you realize that you just disabled dynamic deps support
>> on most of those converted ebuilds.
>>
> 
> There is a bug in the package manager, you mean.
> 

I'm eager to hear how you want to make subslots work with dynamic deps.

:= gets converted to :${SLOT}/${SUBSLOT} in vardb and this is used to
trigger the rebuilds.

How do you record the subslot a package was built against in the live tree?

Re: [gentoo-dev] Re: don't rely on dynamic deps

[Rich Freeman]

On Sun, Jul 27, 2014 at 8:31 AM, hasufell <hasufell@gentoo.org> wrote:
>
> I'm eager to hear how you want to make subslots work with dynamic deps.
>
> := gets converted to :${SLOT}/${SUBSLOT} in vardb and this is used to
> trigger the rebuilds.
>
> How do you record the subslot a package was built against in the live tree?
>

Well, suppose the dependency is removed because it never was a true
dependency to begin with.  Portage can handle this by deleting the
corresponding entry from vardb.

That is a dynamic dependency change, and offhand I don't see how it
breaks with subslots.

This is why we have to be careful about tossing around phrases like
"dynamic deps don't work" - they don't work in particular
circumstances, and it is helpful to the discussion if we try to
characterize when they do/don't work rather than painting with broad
strokes.

I do think that this needs some attention so that we can make portage
more predictable, but I think the argument has been made that we have
a LOT of changes in the tree today which don't involve revbumps, and
turning them into revbumps could cause a lot of turmoil for users.
So, I'm interested in seeing if there is a better compromise to be
found.

Rich

Re: [gentoo-dev] Re: don't rely on dynamic deps

["Paweł Hajdan, Jr."]

[-- Attachment #1: Type: text/plain, Size: 775 bytes --]

On 7/27/14, 1:42 PM, Samuli Suominen wrote:
> Only one person said he had to manually build 2 GNOME related packages,
> simple-scan and something else
> 
> So, broken? Far from it. More like essential feature.
> 
> People have just listed some known races dynamic deps have, and I take
> those races anyday over an regression that causes
> endless rebuilding...

I think this really boils down to deciding on the tradeoff between
correctness and speed.

Even the example above shows that with dynamic rebuilds some manual
rebuilds were needed, which I interpret as dynamic deps not being correct.

Static deps go the other way around: they do lead to more rebuilds, and
I think no one is denying that. However, they seem more predictable to me.

Paweł


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 841 bytes --]

Re: [gentoo-dev] Re: don't rely on dynamic deps

[hasufell]

"Paweł Hajdan, Jr.":
> On 7/27/14, 1:42 PM, Samuli Suominen wrote:
>> Only one person said he had to manually build 2 GNOME related packages,
>> simple-scan and something else
>>
>> So, broken? Far from it. More like essential feature.
>>
>> People have just listed some known races dynamic deps have, and I take
>> those races anyday over an regression that causes
>> endless rebuilding...
> 
> I think this really boils down to deciding on the tradeoff between
> correctness and speed.
> 
> Even the example above shows that with dynamic rebuilds some manual
> rebuilds were needed, which I interpret as dynamic deps not being correct.
> 
> Static deps go the other way around: they do lead to more rebuilds, and
> I think no one is denying that. However, they seem more predictable to me.
> 
> Paweł
> 

his argument is pointless, because he got static deps on all those
ebuilds now anyway

Re: [gentoo-dev] Re: don't rely on dynamic deps

[Ciaran McCreesh]

[-- Attachment #1: Type: text/plain, Size: 808 bytes --]

On Sun, 27 Jul 2014 14:42:24 +0300
Samuli Suominen <ssuominen@gentoo.org> wrote:
> We just succesfully converted ~300 ebuilds in tree without revision
> bumps from virtual/udev[gudev,introspection,static-libs]
> to virtual/libudev and virtual/libgudev
> Tested it on multiple boxes, went fine.

Testing can't prove that it's correct, only that it's incorrect... All
Aside from all the way you've disabled dynamic dependencies for a whole
bunch of packages without realising it, your other problem is that some
of the breakage won't show up until later, when people start bumping
and removing ebuilds.

And this is the problem: you need to think carefully about dynamic
dependencies and fully understand the problem. Superficial testing won't
give you the whole story.

-- 
Ciaran McCreesh

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 181 bytes --]

Re: [gentoo-dev] Re: don't rely on dynamic deps

[Michał Górny]

[-- Attachment #1: Type: text/plain, Size: 827 bytes --]

Dnia 2014-07-27, o godz. 14:42:24
Samuli Suominen <ssuominen@gentoo.org> napisał(a):

> 
> On 26/07/14 15:49, Ciaran McCreesh wrote:
> > On Sat, 26 Jul 2014 12:41:16 +0000 (UTC)
> > Martin Vaeth <martin@mvath.de> wrote:
> >> hasufell <hasufell@gentoo.org> wrote:
> >>> Dynamics deps are already broken, not consistently enabled (e.g.
> >>> when subslots are in use)
> >> Just to make it clear: No, dynamic deps are not broken.
> > Yes they are.
> 
> We just succesfully converted ~300 ebuilds in tree without revision
> bumps from virtual/udev[gudev,introspection,static-libs]
> to virtual/libudev and virtual/libgudev
> Tested it on multiple boxes, went fine.

How did you exactly test them? Did you at least bother checking if
portage actually uses the deps you added?

-- 
Best regards,
Michał Górny

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 949 bytes --]

Re: [gentoo-dev] Re: don't rely on dynamic deps

[Samuli Suominen]

On 27/07/14 16:47, Michał Górny wrote:
> Dnia 2014-07-27, o godz. 14:42:24
> Samuli Suominen <ssuominen@gentoo.org> napisał(a):
>
>> On 26/07/14 15:49, Ciaran McCreesh wrote:
>>> On Sat, 26 Jul 2014 12:41:16 +0000 (UTC)
>>> Martin Vaeth <martin@mvath.de> wrote:
>>>> hasufell <hasufell@gentoo.org> wrote:
>>>>> Dynamics deps are already broken, not consistently enabled (e.g.
>>>>> when subslots are in use)
>>>> Just to make it clear: No, dynamic deps are not broken.
>>> Yes they are.
>> We just succesfully converted ~300 ebuilds in tree without revision
>> bumps from virtual/udev[gudev,introspection,static-libs]
>> to virtual/libudev and virtual/libgudev
>> Tested it on multiple boxes, went fine.
> How did you exactly test them? 

That you could still emerge packages, even world, without Portage
complaining about unsatisfied
deps

> Did you at least bother checking if
> portage actually uses the deps you added?
>

When you `cd /var/db/pkg` and run `grep virtual/udev.*gudev
*/*/*.ebuild`, you can indeed still see some:

app-cdr/xfburn-0.5.2/xfburn-0.5.2.ebuild:    udev? ( virtual/udev[gudev] )"
gnome-base/gvfs-1.20.2/gvfs-1.20.2.ebuild:        virtual/udev[gudev] )
media-gfx/gimp-2.8.10-r1/gimp-2.8.10-r1.ebuild:    udev? (
virtual/udev[gudev] )"
sys-fs/udisks-1.0.5-r1/udisks-1.0.5-r1.ebuild:    >=virtual/udev-208[gudev]
sys-fs/udisks-2.1.3/udisks-2.1.3.ebuild:   
>=virtual/udev-${UDEV_VERSION}[gudev]
virtual/libgudev-208/libgudev-208.ebuild:# $Header:
/var/cvsroot/gentoo-x86/virtual/libgudev/libgudev-208.ebuild,v 1.7
2014/06/18 20:55:21 mgorny Exp $
xfce-extra/thunar-volman-0.8.0/thunar-volman-0.8.0.ebuild:   
virtual/udev[gudev]

But if you try to emerge these packages, like, for example:

$ sudo emerge -pv xfburn thunar-volman

These are the packages that would be merged, in reverse order:

Calculating dependencies... done!
[ebuild   R    ] xfce-extra/thunar-volman-0.8.0  USE="libnotify -debug" 0 kB
[ebuild   R    ] app-cdr/xfburn-0.5.2  USE="udev -debug -gstreamer" 0 kB

Portage is using the fresh copies from gentoo-x86

I'm _not_ a Portage, the package manager, developer, so I'd really
appericiate some *examples* where this would become a problem, binary
packages is known one, we have lived with that problem since forever, so
something else, please.
For everything I do with Portage, this is fine with me, and I expect
it's fine for the vast majority of users as well.
And this majority of users won't appericiate the suggested solution of
"lets always revision bump, despite of triggering rebuild for everyone"

To clarify, I know dynamic deps is not perfect, but it's the best
solution we have implemented to avoid the rebuilding problem, and long
as that's true... And seems like you, yourself, have thought about the
same issue:
http://bugs.gentoo.org/show_bug.cgi?id=486778
As in, you can't skip that bug, and go directly to
http://bugs.gentoo.org/show_bug.cgi?id=516612

- Samuli

(Excuse my grammar, woke up five minutes ago, to make some coffee now...)

Re: [gentoo-dev] Re: don't rely on dynamic deps

[Alexander Tsoy]

В Sun, 27 Jul 2014 14:42:24 +0300
Samuli Suominen <ssuominen@gentoo.org> пишет:

> 
> On 26/07/14 15:49, Ciaran McCreesh wrote:
> > On Sat, 26 Jul 2014 12:41:16 +0000 (UTC)
> > Martin Vaeth <martin@mvath.de> wrote:
> >> hasufell <hasufell@gentoo.org> wrote:
> >>> Dynamics deps are already broken, not consistently enabled (e.g.
> >>> when subslots are in use)
> >> Just to make it clear: No, dynamic deps are not broken.
> > Yes they are.
> 
> We just succesfully converted ~300 ebuilds in tree without revision
> bumps from virtual/udev[gudev,introspection,static-libs]
> to virtual/libudev and virtual/libgudev
> Tested it on multiple boxes, went fine. Nobody has filed bugs at
> http://bugs.gentoo.org/, nobody has filed a single forums post,
> nobody has said anything at #gentoo, Freenode
> Only one person said he had to manually build 2 GNOME related
> packages, simple-scan and something else

As Michał already noted in this thread, dynamic deps does not play nice
with slot operators. So I had the same problem with "2 GNOME related
packages":

!!! Multiple package instances within a single package slot have been pulled
!!! into the dependency graph, resulting in a slot conflict:

virtual/udev:0

  (virtual/udev-208-r2::gentoo, installed) pulled in by
    >=virtual/udev-171:0/0=[gudev] required by (media-video/cheese-3.12.2::gentoo, installed)
    virtual/udev:0/0=[gudev] required by (x11-misc/colord-1.2.1::gentoo, installed)

  (virtual/udev-215::gentoo, ebuild scheduled for merge) pulled in by
    =virtual/udev-215 required by (games-util/xboxdrv-0.8.5-r1::gentoo, installed)
    (and 22 more with the same problem)

> 
> So, broken? Far from it. More like essential feature.
> 
> People have just listed some known races dynamic deps have, and I take
> those races anyday over an regression that causes
> endless rebuilding...
> 
> - Samuli
> 

-- 
Alexander Tsoy

Re: [gentoo-dev] Re: don't rely on dynamic deps

[Samuli Suominen]

On 30/07/14 07:45, Alexander Tsoy wrote:
> В Sun, 27 Jul 2014 14:42:24 +0300
> Samuli Suominen <ssuominen@gentoo.org> пишет:
>
>> On 26/07/14 15:49, Ciaran McCreesh wrote:
>>> On Sat, 26 Jul 2014 12:41:16 +0000 (UTC)
>>> Martin Vaeth <martin@mvath.de> wrote:
>>>> hasufell <hasufell@gentoo.org> wrote:
>>>>> Dynamics deps are already broken, not consistently enabled (e.g.
>>>>> when subslots are in use)
>>>> Just to make it clear: No, dynamic deps are not broken.
>>> Yes they are.
>> We just succesfully converted ~300 ebuilds in tree without revision
>> bumps from virtual/udev[gudev,introspection,static-libs]
>> to virtual/libudev and virtual/libgudev
>> Tested it on multiple boxes, went fine. Nobody has filed bugs at
>> http://bugs.gentoo.org/, nobody has filed a single forums post,
>> nobody has said anything at #gentoo, Freenode
>> Only one person said he had to manually build 2 GNOME related
>> packages, simple-scan and something else
> As Michał already noted in this thread, dynamic deps does not play nice
> with slot operators. So I had the same problem with "2 GNOME related
> packages":

I've revision bumped x11-misc/colord and media-gfx/simple-scan
because of this reason, I'll do media-video/cheese as well

If it's 2-3 packages out of ~300, I'd rather pick them out than
revision bump all ~300 for the 2-3. Or not pick them out at all
and let users do the rebuild (which is the obvious answer
to the output you posted)

>
> !!! Multiple package instances within a single package slot have been pulled
> !!! into the dependency graph, resulting in a slot conflict:
>
> virtual/udev:0
>
>   (virtual/udev-208-r2::gentoo, installed) pulled in by
>     >=virtual/udev-171:0/0=[gudev] required by (media-video/cheese-3.12.2::gentoo, installed)
>     virtual/udev:0/0=[gudev] required by (x11-misc/colord-1.2.1::gentoo, installed)
>
>   (virtual/udev-215::gentoo, ebuild scheduled for merge) pulled in by
>     =virtual/udev-215 required by (games-util/xboxdrv-0.8.5-r1::gentoo, installed)
>     (and 22 more with the same problem)
>

Re: [gentoo-dev] Re: don't rely on dynamic deps

[Peter Stuge]

Samuli Suominen wrote:
> let users do the rebuild (which is the obvious answer
> to the output you posted)

Reality check time, Samuli.

Unless emerge says "Your dependencies are b0rk, please rebuild $P to fix it."
that answer is nowhere near obvious.

Watch out with the tunnel vision.


//Peter

Re: [gentoo-dev] Re: don't rely on dynamic deps

["Paweł Hajdan, Jr."]

[-- Attachment #1: Type: text/plain, Size: 1060 bytes --]

On 7/30/14, 7:36 AM, Samuli Suominen wrote:
> If it's 2-3 packages out of ~300, I'd rather pick them out than
> revision bump all ~300 for the 2-3. Or not pick them out at all
> and let users do the rebuild (which is the obvious answer
> to the output you posted)

Peter Stuge pointed it out already, but I also wanted to say rebuilding
the affected packages is not obvious to me either.

Paweł

>>
>> !!! Multiple package instances within a single package slot have been pulled
>> !!! into the dependency graph, resulting in a slot conflict:
>>
>> virtual/udev:0
>>
>>   (virtual/udev-208-r2::gentoo, installed) pulled in by
>>     >=virtual/udev-171:0/0=[gudev] required by (media-video/cheese-3.12.2::gentoo, installed)
>>     virtual/udev:0/0=[gudev] required by (x11-misc/colord-1.2.1::gentoo, installed)
>>
>>   (virtual/udev-215::gentoo, ebuild scheduled for merge) pulled in by
>>     =virtual/udev-215 required by (games-util/xboxdrv-0.8.5-r1::gentoo, installed)
>>     (and 22 more with the same problem)
>>
> 
> 
> 



[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 841 bytes --]

Re: [gentoo-dev] Re: don't rely on dynamic deps

[Rich Freeman]

On Wed, Jul 30, 2014 at 3:38 AM, "Paweł Hajdan, Jr."
<phajdan.jr@gentoo.org> wrote:
> On 7/30/14, 7:36 AM, Samuli Suominen wrote:
>> If it's 2-3 packages out of ~300, I'd rather pick them out than
>> revision bump all ~300 for the 2-3. Or not pick them out at all
>> and let users do the rebuild (which is the obvious answer
>> to the output you posted)
>
> Peter Stuge pointed it out already, but I also wanted to say rebuilding
> the affected packages is not obvious to me either.

Sure, but this seems more like a portage bug (or at least a portage
output bug) rather than a fundamental issue.

After all, there was no true block - just a need for a rebuild.

I heard prerm as a reason why dynamic deps can break (especially with
slot operator deps, though obviously it also breaks for
non-slot-operator deps that should be expressed as such), though as
has been pointed out those will break unless we unmerge and remerge
all reverse-deps on every upgrade.  Are there other issues.

To be honest I was expecting a plethora of issues that can go wrong
with dynamic deps, but so far I'm hearing something like 2-3, and if
that really is all that there is then this may be a solvable issue.

Rich

Re: [gentoo-dev] Re: don't rely on dynamic deps

[Ciaran McCreesh]

[-- Attachment #1: Type: text/plain, Size: 463 bytes --]

On Wed, 30 Jul 2014 07:18:22 -0400
Rich Freeman <rich0@gentoo.org> wrote:
> Sure, but this seems more like a portage bug (or at least a portage
> output bug) rather than a fundamental issue.
> 
> After all, there was no true block - just a need for a rebuild.

It's often not possible to produce a decent error message, given the
limited information that ebuilds supply and the unnecessary
pseudocleverness they employ to do it.

-- 
Ciaran McCreesh

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 181 bytes --]

Re: [gentoo-dev] Re: don't rely on dynamic deps

[Samuli Suominen]

On 30/07/14 14:18, Rich Freeman wrote:
> On Wed, Jul 30, 2014 at 3:38 AM, "Paweł Hajdan, Jr."
> <phajdan.jr@gentoo.org> wrote:
>> On 7/30/14, 7:36 AM, Samuli Suominen wrote:
>>> If it's 2-3 packages out of ~300, I'd rather pick them out than
>>> revision bump all ~300 for the 2-3. Or not pick them out at all
>>> and let users do the rebuild (which is the obvious answer
>>> to the output you posted)
>> Peter Stuge pointed it out already, but I also wanted to say rebuilding
>> the affected packages is not obvious to me either.
> Sure, but this seems more like a portage bug (or at least a portage
> output bug) rather than a fundamental issue.
>
> After all, there was no true block - just a need for a rebuild.
>
> I heard prerm as a reason why dynamic deps can break (especially with
> slot operator deps, though obviously it also breaks for
> non-slot-operator deps that should be expressed as such), though as
> has been pointed out those will break unless we unmerge and remerge
> all reverse-deps on every upgrade.  Are there other issues.
>
> To be honest I was expecting a plethora of issues that can go wrong
> with dynamic deps, but so far I'm hearing something like 2-3, and if
> that really is all that there is then this may be a solvable issue.
>
>

That's what I've been trying to point out, people are seriously suggesting
disabling dynamic deps for race conditions
It's like fixing one audio driver in the kernel by deleting whole ALSA block

:-(

- Samuli

Re: [gentoo-dev] Re: don't rely on dynamic deps

[Tom Wijsman]

[-- Attachment #1: Type: text/plain, Size: 645 bytes --]

On Wed, 30 Jul 2014 15:38:43 +0300
Samuli Suominen <ssuominen@gentoo.org> wrote:

> That's what I've been trying to point out, people are seriously
> suggesting disabling dynamic deps for race conditions
> It's like fixing one audio driver in the kernel by deleting whole
> ALSA block

It is more like fixing multiple broken audio drivers; if there is only
a single problem with dynamic deps, it wouldn't receive much attention.

-- 
With kind regards,

Tom Wijsman (TomWij)
Gentoo Developer

E-mail address  : TomWij@gentoo.org
GPG Public Key  : 6D34E57D
GPG Fingerprint : C165 AF18 AB4C 400B C3D2  ABF0 95B2 1FCD 6D34 E57D

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 473 bytes --]

[gentoo-dev] Re: don't rely on dynamic deps

[Ulrich Mueller]

[-- Attachment #1: Type: text/plain, Size: 165 bytes --]

>>>>> On Sat, 26 Jul 2014, Martin Vaeth wrote:

> Quite the opposite, PMS claims that one cannot rely on
> anything stored in /var/db

Where does it say so?

Ulrich

[-- Attachment #2: Type: application/pgp-signature, Size: 490 bytes --]

[gentoo-dev] PMS (was: don't rely on dynamic deps)

[Martin Vaeth]

Ulrich Mueller <ulm@gentoo.org> wrote:
>>>>>> On Sat, 26 Jul 2014, Martin Vaeth wrote:
>
>> Quite the opposite, PMS claims that one cannot rely on
>> anything stored in /var/db
>
> Where does it say so?

"Appendix B: Unspecified Items

The following items are not specified by this document,
and must not be relied upon by ebuilds.
[...]
The VDB (/var/db/pkg)
[...]"

This is bugging me for quite a while already, because I
had to try and to rely on undocumented behaviour for all
features of eix which access /var/db/pkg.

It also is one of the reasons why it might be dangerous
if you use another PM even only for testing purposes:
In theory, it might even be possible that a package installed
by one PM is not recognized by the other.
I guess that this is not a big problem in practice, but
in fine points like the discussed prerm perhaps some
problems might arise through mixing.

Re: [gentoo-dev] PMS

[Ulrich Mueller]

[-- Attachment #1: Type: text/plain, Size: 545 bytes --]

>>>>> On Fri, 1 Aug 2014, Martin Vaeth wrote:

> Ulrich Mueller <ulm@gentoo.org> wrote:
>>>>>>> On Sat, 26 Jul 2014, Martin Vaeth wrote:
>> 
>>> Quite the opposite, PMS claims that one cannot rely on
>>> anything stored in /var/db
>> 
>> Where does it say so?

> "Appendix B: Unspecified Items

> The following items are not specified by this document,
> and must not be relied upon by ebuilds.
> [...]
> The VDB (/var/db/pkg)
> [...]"

This says that ebuilds must not access the VDB. It says nothing about
the package manager doing so.

Ulrich

[-- Attachment #2: Type: application/pgp-signature, Size: 490 bytes --]

[gentoo-dev] Re: PMS

[Martin Vaeth]

Ulrich Mueller <ulm@gentoo.org> wrote:
>> Ulrich Mueller <ulm@gentoo.org> wrote:
>>>>>>>> On Sat, 26 Jul 2014, Martin Vaeth wrote:
>>>
>>>> Quite the opposite, PMS claims that one cannot rely on
>>>> anything stored in /var/db
>>>
>>> Where does it say so?
>
>> "Appendix B: Unspecified Items
>
>> The following items are not specified by this document,
>> and must not be relied upon by ebuilds.
>> [...]
>> The VDB (/var/db/pkg)
>> [...]"
>
> This says that ebuilds must not access the VDB. It says nothing about
> the package manager doing so.

Nobody claimed the contrary. PM != PMS. The behaviour of PM
concerning /var/db is "just" undocumented (as concerns PMS).

Re: [gentoo-dev] don't rely on dynamic deps

[Ulrich Mueller]

[-- Attachment #1: Type: text/plain, Size: 674 bytes --]

>>>>> On Wed, 23 Jul 2014, Tom Wijsman wrote:

> Using an extension like -rX.Y seems odd; at the very least, I think
> an incremental variable or something along that line in the ebuild
> would work better.

It would also account for changes in eclasses, which any scheme bound
to the ebuild's filename cannot do.

> This allows for array usage like VERSION[dependencies]=1, thus
> allowing other variables to be dynamic as well; you compare that
> number against the one in the vdb, bingo...

Taking this one step further, I wonder if one couldn't directly
compare *DEPEND in the ebuild against the one in the VDB. Why would
one need to introduce another variable?

Ulrich

[-- Attachment #2: Type: application/pgp-signature, Size: 490 bytes --]

Re: [gentoo-dev] don't rely on dynamic deps

[Ciaran McCreesh]

[-- Attachment #1: Type: text/plain, Size: 603 bytes --]

On Mon, 21 Jul 2014 23:06:07 +0200
Pacho Ramos <pacho@gentoo.org> wrote:
> Maybe this could be solved by having two kinds of revisions:
> - One would rebuild all as usually (for example, -r1...)
> - The other one would only regenerate VDB and wouldn't change the
> installed files (for example, -r1.1)
> 
> But I am not sure if it could be viable from a "technical" point of
> view :(

This in no way solves the problem. Consider the following course of
events:

User installs foo-1.1-r1
Developer makes foo-1.1-r1.1
foo-1.1* is removed from the tree
User syncs

-- 
Ciaran McCreesh

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 181 bytes --]

Re: [gentoo-dev] don't rely on dynamic deps

[Rich Freeman]

On Wed, Jul 23, 2014 at 9:33 AM, Ciaran McCreesh
<ciaran.mccreesh@googlemail.com> wrote:
> On Mon, 21 Jul 2014 23:06:07 +0200
> Pacho Ramos <pacho@gentoo.org> wrote:
>> Maybe this could be solved by having two kinds of revisions:
>> - One would rebuild all as usually (for example, -r1...)
>> - The other one would only regenerate VDB and wouldn't change the
>> installed files (for example, -r1.1)
>>
>> But I am not sure if it could be viable from a "technical" point of
>> view :(
>
> This in no way solves the problem. Consider the following course of
> events:
>
> User installs foo-1.1-r1
> Developer makes foo-1.1-r1.1
> foo-1.1* is removed from the tree
> User syncs

An updates-like mechanism would help here, since the updates could
persist longer.

Also, the user is probably going to end up uninstalling foo anyway or
updating it to a newer revision, which means that whatever was broken
with -r1 will tend to become a bit of a moot issue.  Portage doesn't
really support hanging onto PVs that aren't in the tree all that well
to begin with.

Just a general comment not aimed at this particular part of the thread
- a solution doesn't have to be perfect to be useful.  If we come up
with a good clean solution that avoids rebuilds in a half-dozen
specific circumstances and we agree to only use it in those
circumstances, there is no reason we can't use it, even if there is
some other circumstance that will still require a revbump.  I'm
sensing in this thread that we're forcing ourselves to choose between
a hack that can be applied 100% of the time but which can break
randomly, or a hypothetical perfect solution that never breaks but
which will probably never exist either.  A solution that works 80% of
the time and never breaks as long as it is properly applied is an
acceptable compromise.

Rich

Re: [gentoo-dev] don't rely on dynamic deps

[Ciaran McCreesh]

[-- Attachment #1: Type: text/plain, Size: 636 bytes --]

On Thu, 24 Jul 2014 21:45:58 -0400
Rich Freeman <rich0@gentoo.org> wrote:
> Just a general comment not aimed at this particular part of the thread
> - a solution doesn't have to be perfect to be useful.

Wrong. The reason everything is such a mess at the moment is precisely
because we've accumulated so much "good enough" and "not thinking your
cunning plan all the way through" that nothing is actually correct any
more. It's one thing to get away with this occasionally, but quite
another to build an entire system upon it. We can't afford more
mistakes, and we have to actively fix existing ones.

-- 
Ciaran McCreesh

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 181 bytes --]

Re: [gentoo-dev] don't rely on dynamic deps

[Alexander Berntsen]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

On 25/07/14 17:01, Ciaran McCreesh wrote:
> The reason everything is such a mess at the moment is precisely 
> because we've accumulated so much "good enough" and "not thinking
> your cunning plan all the way through" that nothing is actually
> correct any more. It's one thing to get away with this
> occasionally, but quite another to build an entire system upon it.
> We can't afford more mistakes, and we have to actively fix existing
> ones.
This is my position as well.
- -- 
Alexander
bernalex@gentoo.org
https://secure.plaimi.net/~alexander
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/

iF4EAREIAAYFAlPSfWkACgkQRtClrXBQc7XOcwD/Vmzd/B4IysGbczDAxI/X6CLw
miOuIePVOSwIWRSMvx4A/0Kdy6SCxx2JCko9xMTcnAagqaoleWJeoPIwb1bUkE74
=2rq+
-----END PGP SIGNATURE-----

Re: [gentoo-dev] don't rely on dynamic deps

[Rich Freeman]

On Fri, Jul 25, 2014 at 11:01 AM, Ciaran McCreesh
<ciaran.mccreesh@googlemail.com> wrote:
> On Thu, 24 Jul 2014 21:45:58 -0400
> Rich Freeman <rich0@gentoo.org> wrote:
>> Just a general comment not aimed at this particular part of the thread
>> - a solution doesn't have to be perfect to be useful.
>
> Wrong. The reason everything is such a mess at the moment is precisely
> because we've accumulated so much "good enough" and "not thinking your
> cunning plan all the way through" that nothing is actually correct any
> more.

I think we might be saying the same thing in different ways.  I wasn't
suggesting that we should implement solutions that fail in random
ways, but rather that if necessary we should focus more on simpler
solutions that we can get right, but which perhaps don't cover all of
our problems.

That is, I'm more for a perfect solution for a small problem rather
than a good-enough solution (which isn't) for a big problem.

For example, perhaps there is a way to safely add an unconditional
dependency to an installed package.  That won't solve every dependency
problem, but it could be helpful.

Another way to go about things would be to try to find ways to reduce
the chance of commiting a package that has an incorrect dependency in
the first place, so that we don't have to fix so many mistakes.  Then
perhaps the extra rebuilds when there is a rare mistake might be more
forgivable.

Rich

[gentoo-dev] Re: don't rely on dynamic deps

[Martin Vaeth]

Ciaran McCreesh <ciaran.mccreesh@googlemail.com> wrote:
>
> Wrong. The reason everything is such a mess at the moment is precisely
> because we've accumulated so much "good enough" and "not thinking your
> cunning plan all the way through"

There *is* no perfect solution.
And the reason why everything is such a mess at the moment is
that portage's implementation of subslot dependencies was
broken (because it used a mechanism which has broken dynamic deps).

*This* is what needs to be fixed but which is perhaps not
easy to fix.

However, changing the policy in a manner causing so many
unnecessary ebuilds (and fixing not all problems either)
is throwing the baby out with the bathwater.

When having the choice between only non-perfect solutions,
I vote for the one which keeps the distribution usable.

[gentoo-dev] Re: don't rely on dynamic deps

[Martin Vaeth]

Rich Freeman <rich0@gentoo.org> wrote:
>>
>> User installs foo-1.1-r1
>> Developer makes foo-1.1-r1.1
>> foo-1.1* is removed from the tree
>> User syncs
>
> An updates-like mechanism would help here, since the updates could
> persist longer.

Unfortunately, this is not an option:
I assure you that you do not want to keep the updates
in dependencies forever; their number is simply too large.
This would mean the portage tree will keep growing in an
insane manner: It "almost" means to force every user to
keep the *full* CVS (or in future maybe git) repository.

Re: [gentoo-dev] don't rely on dynamic deps

[Pacho Ramos]

El mié, 23-07-2014 a las 14:33 +0100, Ciaran McCreesh escribió:
> On Mon, 21 Jul 2014 23:06:07 +0200
> Pacho Ramos <pacho@gentoo.org> wrote:
> > Maybe this could be solved by having two kinds of revisions:
> > - One would rebuild all as usually (for example, -r1...)
> > - The other one would only regenerate VDB and wouldn't change the
> > installed files (for example, -r1.1)
> > 
> > But I am not sure if it could be viable from a "technical" point of
> > view :(
> 
> This in no way solves the problem. Consider the following course of
> events:
> 
> User installs foo-1.1-r1
> Developer makes foo-1.1-r1.1
> foo-1.1* is removed from the tree
> User syncs
> 

Isn't there a way for PMs to know that they need to not rebuild full if
user has 1.1-r1 *installed* in his system and pretends to update to
-r1.1? 

[gentoo-dev] Re: don't rely on dynamic deps

[Martin Vaeth]

Pacho Ramos <pacho@gentoo.org> wrote:
>
> Isn't there a way for PMs to know that they need to not rebuild full if
> user has 1.1-r1 *installed* in his system and pretends to update to
> -r1.1?

This is exactly the idea of -r1.1, and this (at least the check)
already works in the code snippet I had sent to the portage list:
From 1.1-r1 to 1.1-r1.1 no recompilation should happen, while e.g.
from 1.1 to 1.1-r1.1 the bump is as usual.

Package managers which do not want to implement minor revisions
could just treat -r1.1 as a "usual" revbump and recompile
unconditionally, that is, except for a slight extension of
the allowed versions (and corresponding comparison rules)
nothing would change for them (although their users would
then not benefit from the possible avoidance of some
recompilations offered perhaps by other package managers).

[gentoo-dev] Re: don't rely on dynamic deps

[Martin Vaeth]

Ciaran McCreesh <ciaran.mccreesh@googlemail.com> wrote:
>
> User installs foo-1.1-r1
> Developer makes foo-1.1-r1.1
> foo-1.1* is removed from the tree
> User syncs

How is this different from your suggestion
(which you *claim* to be non-broken):

User installs foo-1.1-r1
Developer makes foo-1.1-r2
foo-1.1* is removed from the tree
User syncs

In fact, the result is completely the same,
no matter whether you have minor revisions or not,
and no matter whether you have static or dynamic deps.

What is *actually* broken here is that the user
has installed a package which is not maintained
anymore: *This* is what needs to be fixed.
This issue is completely independent of static
vs. dynamic deps.
You misuse this problem as a strawman, only.

Re: [gentoo-dev] Re: don't rely on dynamic deps

[Ciaran McCreesh]

[-- Attachment #1: Type: text/plain, Size: 1255 bytes --]

On Sat, 26 Jul 2014 12:32:20 +0000 (UTC)
Martin Vaeth <martin@mvath.de> wrote:
> Ciaran McCreesh <ciaran.mccreesh@googlemail.com> wrote:
> > User installs foo-1.1-r1
> > Developer makes foo-1.1-r1.1
> > foo-1.1* is removed from the tree
> > User syncs
> 
> How is this different from your suggestion
> (which you *claim* to be non-broken):
> 
> User installs foo-1.1-r1
> Developer makes foo-1.1-r2
> foo-1.1* is removed from the tree
> User syncs
> 
> In fact, the result is completely the same,
> no matter whether you have minor revisions or not,
> and no matter whether you have static or dynamic deps.
> 
> What is *actually* broken here is that the user
> has installed a package which is not maintained
> anymore: *This* is what needs to be fixed.
> This issue is completely independent of static
> vs. dynamic deps.
> You misuse this problem as a strawman, only.

Uhm. That works just fine... I don't think you understand how this
works: we can always use the metadata that's in VDB for dealing with the
installed package. The issue is that sometimes Portage tries to guess
that it's better to use the metadata from an ebuild instead of what's
in VDB when dealing with an installed package.

-- 
Ciaran McCreesh

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 181 bytes --]

[gentoo-dev] Re: don't rely on dynamic deps

[Martin Vaeth]

Ciaran McCreesh <ciaran.mccreesh@googlemail.com> wrote:
>> User installs foo-1.1-r1
>> Developer makes foo-1.1-r2
>> foo-1.1* is removed from the tree
>> User syncs
>>
>> In fact, the result is completely the same

You completely ignored this essential point:
The result is completely the same, and you are
just arguing with a strawman.

But, OK, so I will use your strawman to prove
how static deps are broken:

>> What is *actually* broken here is that the user
>> has installed a package which is not maintained
>> anymore: *This* is what needs to be fixed.
>
> Uhm. That works just fine...

Not at all:

1. Some package depending on foo/bar is removed,
but the user keeps it, since deps are stored in /var/db.

2. foo/bar is split into foo/bar-A foo/bar-B
for whatever reasons. Of course, all maintained ebuilds
fix the dependency, and let us assume they are even
revbumped.

3. The orphaned package of course still depends on
installed foo/bar, causing all sort of blockers.

4. The user has no way for fixing the issue than
in modifying /var/db manually. He cannot even put
an ebuild into his overlay and only modify this
ebuild and the metadata, because the PM dumbly
insists on using only the (broken and outdated
since ages) information in /var/db

> I don't think you understand how this works:

Quite the opposite: I see that it fixes many issues.
It has also some issues with orphaned packages,
but *every* approach will have issues with orphaned
packages.

Re: [gentoo-dev] Re: don't rely on dynamic deps

[Ciaran McCreesh]

[-- Attachment #1: Type: text/plain, Size: 339 bytes --]

On Sat, 26 Jul 2014 13:16:13 +0000 (UTC)
Martin Vaeth <martin@mvath.de> wrote:
> But, OK, so I will use your strawman to prove
> how static deps are broken:

This is not broken. This is exactly what is supposed to happen, and it
is exactly what *does* happen some of the time with dynamic
dependencies too.

-- 
Ciaran McCreesh

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 181 bytes --]

[gentoo-dev] Re: don't rely on dynamic deps

[Martin Vaeth]

Ciaran McCreesh <ciaran.mccreesh@googlemail.com> wrote:
>> But, OK, so I will use your strawman to prove
>> how static deps are broken:
>
> This is not broken. This is exactly what is supposed to happen

"It's not a bug it's a feature"
Of course, one can always close the eyes when faced
with problems.

> and it
> is exactly what *does* happen some of the time with dynamic
> dependencies too.

Yes, both concepts have problems.
Since neither solution is perfect, why choose the one
with unnecessary rebuilds?

Re: [gentoo-dev] Re: don't rely on dynamic deps

[hasufell]

Martin Vaeth:
> Ciaran McCreesh <ciaran.mccreesh@googlemail.com> wrote:
>>> But, OK, so I will use your strawman to prove
>>> how static deps are broken:
>>
>> This is not broken. This is exactly what is supposed to happen
> 
> "It's not a bug it's a feature"
> Of course, one can always close the eyes when faced
> with problems.
> 
>> and it
>> is exactly what *does* happen some of the time with dynamic
>> dependencies too.
> 
> Yes, both concepts have problems.
> Since neither solution is perfect, why choose the one
> with unnecessary rebuilds?
> 
> 

You are not contributing anything useful to the thread currently.

Read the whole thread. Read up on dynamic deps. Read up on PMS.

Re: [gentoo-dev] Re: don't rely on dynamic deps

[Ciaran McCreesh]

[-- Attachment #1: Type: text/plain, Size: 543 bytes --]

On Sat, 26 Jul 2014 14:46:42 +0000 (UTC)
Martin Vaeth <martin@mvath.de> wrote:
> Yes, both concepts have problems.

The problems are of a different kind. Static dependencies don't do
something that you want them to do. Dynamic dependencies are outright
broken.

> Since neither solution is perfect, why choose the one
> with unnecessary rebuilds?

We are picking the *correct* solution, not the one that sometimes hides
an occasional inconvenience (but unreliably) at the expense of being
utterly broken.

-- 
Ciaran McCreesh

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 181 bytes --]

[gentoo-dev] Re: don't rely on dynamic deps

[Martin Vaeth]

Ciaran McCreesh <ciaran.mccreesh@googlemail.com> wrote:
> Martin Vaeth <martin@mvath.de> wrote:
>
> The problems are of a different kind. Static dependencies don't do
> something that you want them to do. Dynamic dependencies are outright
> broken.

Please, stop your childish behaviour.
You prove nothing be repeating claims which had just been disproved.

Re: [gentoo-dev] Re: don't rely on dynamic deps

[Ciaran McCreesh]

[-- Attachment #1: Type: text/plain, Size: 715 bytes --]

On Sat, 26 Jul 2014 15:11:36 +0000 (UTC)
Martin Vaeth <martin@mvath.de> wrote:
> Ciaran McCreesh <ciaran.mccreesh@googlemail.com> wrote:
> > Martin Vaeth <martin@mvath.de> wrote:
> > The problems are of a different kind. Static dependencies don't do
> > something that you want them to do. Dynamic dependencies are
> > outright broken.
> 
> Please, stop your childish behaviour.
> You prove nothing be repeating claims which had just been disproved.

Let's start with the easiest issue: please point us all to the place
where you "proved" how dynamic dependencies still work in the face of
ebuild removals. Your solution to this problem will be of great
benefit to all of us.

-- 
Ciaran McCreesh

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 181 bytes --]

[gentoo-dev] Re: don't rely on dynamic deps

[Martin Vaeth]

Ciaran McCreesh <ciaran.mccreesh@googlemail.com> wrote:
> Martin Vaeth <martin@mvath.de> wrote:
>> Ciaran McCreesh <ciaran.mccreesh@googlemail.com> wrote:
>> > Martin Vaeth <martin@mvath.de> wrote:
>> > The problems are of a different kind. Static dependencies don't do
>> > something that you want them to do. Dynamic dependencies are
>> > outright broken.
>> Please, stop your childish behaviour.
>> You prove nothing be repeating claims which had just been disproved.
>
> Let's start with the easiest issue: please point us all to the place
> where you "proved" how dynamic dependencies still work in the face of
> ebuild removals.

*Neither* dynamic deps nor static deps solve this problem satisfactory
(How often did I repeat this now?).
Probably there does not exist *any* satisfactory solution to orphaned
packages at all. So this case is not a valid argument to prefer one
method over another.

Re: [gentoo-dev] Re: don't rely on dynamic deps

[Ciaran McCreesh]

[-- Attachment #1: Type: text/plain, Size: 1462 bytes --]

On Sat, 26 Jul 2014 15:40:40 +0000 (UTC)
Martin Vaeth <martin@mvath.de> wrote:
> > Let's start with the easiest issue: please point us all to the place
> > where you "proved" how dynamic dependencies still work in the face
> > of ebuild removals.
> 
> *Neither* dynamic deps nor static deps solve this problem satisfactory
> (How often did I repeat this now?).

With static dependencies, you have correct dependency information, and
the worst that can happen is occasionally you might have to rebuild a
package where nothing substantial has changed. However, this is a
general issue with bumps (recompiling the whole thing for an init
script or language file change, recompiling the whole thing for a change
to only one of the binaries provided by a package, and so on), so it is
not a "static dependencies" problem.

With dynamic dependencies, you have incorrect dependency information,
your system randomly breaks on a sync, you sometimes can't uninstall
packages due to pkg_* breakage, uninstalling a package sometimes looks
safe but isn't, overlays don't work, subslots don't work, binaries
don't work, and dependencies can appear to be met when they aren't.

So in summary, dynamic dependencies are broken, and static dependencies
are correct, and the only issue you think you have with static
dependencies isn't a problem specific to static dependencies and isn't
reliably solved by dynamic dependencies.

-- 
Ciaran McCreesh

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 181 bytes --]

Re: [gentoo-dev] Re: don't rely on dynamic deps

[Ian Stakenvicius]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

On 26/07/14 11:22 AM, Ciaran McCreesh wrote:
> 
> Let's start with the easiest issue: please point us all to the
> place where you "proved" how dynamic dependencies still work in the
> face of ebuild removals. Your solution to this problem will be of
> great benefit to all of us.
> 

This is something I personally don't understand.  If an ebuild for a
package installed on the system has been removed from the tree, but
newer and/or older ebuilds exist in the tree, then the installed
package can #1 only be trusted in accordance with the ebuild copy
enbedded in VDB (that i get), BUT, #2 should be forced to either
upgrade or downgrade so that it matches what *is* in the tree anyhow,
and that's done via a standard ${PV} comparison that should happen
regardless of whether static or dynamic deps methods are in place.

IMO, if currently-installed versions of packages are satisfying
dependencies after those packages have been removed from the tree, I
don't see this as being particularly valid anyhow.  Sure, end-users
should be able to force this using masks or whatnot in the particular
cases they need to do this, but i don't think this should be in any
way a default behaviour, should it??  Ebuilds are removed for a reason...


-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2

iF4EAREIAAYFAlPWXncACgkQ2ugaI38ACPBWLQEAp3sB8lfdZ8FYmXRsxNy6SlHE
AR40+p+/x6J5+m4D618BAK4XKG64W92WFWne2rn3cDtdKuoQ+wwN2RBw066MoPJQ
=TyGx
-----END PGP SIGNATURE-----

Re: [gentoo-dev] Re: don't rely on dynamic deps

[Ciaran McCreesh]

[-- Attachment #1: Type: text/plain, Size: 1212 bytes --]

On Mon, 28 Jul 2014 10:30:15 -0400
Ian Stakenvicius <axs@gentoo.org> wrote:
> On 26/07/14 11:22 AM, Ciaran McCreesh wrote:
> > 
> > Let's start with the easiest issue: please point us all to the
> > place where you "proved" how dynamic dependencies still work in the
> > face of ebuild removals. Your solution to this problem will be of
> > great benefit to all of us.
> > 
> 
> This is something I personally don't understand.  If an ebuild for a
> package installed on the system has been removed from the tree, but
> newer and/or older ebuilds exist in the tree, then the installed
> package can #1 only be trusted in accordance with the ebuild copy
> enbedded in VDB (that i get), BUT, #2 should be forced to either
> upgrade or downgrade so that it matches what *is* in the tree anyhow,
> and that's done via a standard ${PV} comparison that should happen
> regardless of whether static or dynamic deps methods are in place.

But you can't run pkg_prerm unless a package's dependencies are
satisfied. How do you know what those dependencies are, if you don't
use VDB and if the ebuild isn't there?

(This is a real issue: see the botched ruby-config switch.)

-- 
Ciaran McCreesh

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 181 bytes --]

Re: [gentoo-dev] Re: don't rely on dynamic deps

[Ian Stakenvicius]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

On 28/07/14 10:43 AM, Ciaran McCreesh wrote:
> On Mon, 28 Jul 2014 10:30:15 -0400 Ian Stakenvicius
> <axs@gentoo.org> wrote:
>> On 26/07/14 11:22 AM, Ciaran McCreesh wrote:
>>> 
>>> Let's start with the easiest issue: please point us all to the 
>>> place where you "proved" how dynamic dependencies still work in
>>> the face of ebuild removals. Your solution to this problem will
>>> be of great benefit to all of us.
>>> 
>> 
>> This is something I personally don't understand.  If an ebuild
>> for a package installed on the system has been removed from the
>> tree, but newer and/or older ebuilds exist in the tree, then the
>> installed package can #1 only be trusted in accordance with the
>> ebuild copy enbedded in VDB (that i get), BUT, #2 should be
>> forced to either upgrade or downgrade so that it matches what
>> *is* in the tree anyhow, and that's done via a standard ${PV}
>> comparison that should happen regardless of whether static or
>> dynamic deps methods are in place.
> 
> But you can't run pkg_prerm unless a package's dependencies are 
> satisfied. How do you know what those dependencies are, if you
> don't use VDB and if the ebuild isn't there?
> 
> (This is a real issue: see the botched ruby-config switch.)
> 

Yes, that's an issue -- I thought the pkg-*rm case had to do with the
ebuild's phase definition itself being (or not being) updated, I
haddn't thought about it in terms of dependencies being unsatisfied.

Keeping every single dependency around and valid just so that pkg_*rm
can completely cleanly seems like so much overkill, though..
Unfortunately the only way I see around that would be to have some
form of reduced subset, which would mean yet-another-*DEPEND, and we
all know that's not going to happen..

I wonder if there would be a way to somehow add restrictions to
pkg_*rm phases s.t. either REPLACED_BY_VERSION's dependencies must be
satisfied at the time the phases are run, or REPLACED_BY_VERSION is
empty and the in-VDB ones must be satisfied.  It'll be a pain for
dev's to make sure their stuff works within these restrictions, and
the likeliness of repoman being able to enforce any sort of QA on this
probably so close to zero it doesn't matter, but i'm not seeing
another way.

(outside of forcing the in-vdb deps to always remain satisfied, of
course; however i think the dependencies-get-upgraded-first approach
which is generally necessary for all but PDEPEND can still cause
breakage here too, no??)



-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2

iF4EAREIAAYFAlPWbpMACgkQ2ugaI38ACPBkdwEAsPg3Gu4I3LuXfBuvrxfGJ3D6
sv/ZOROo0a0xPzQ8IZgA/icJDURR+a0mrvT2fwSzXNfd+azvaaKxjNOcRPHOcSYE
=YElO
-----END PGP SIGNATURE-----

[gentoo-dev] Re: don't rely on dynamic deps

[Martin Vaeth]

Ian Stakenvicius <axs@gentoo.org> wrote:
>
> Keeping every single dependency around and valid just so that pkg_*rm
> can completely cleanly seems like so much overkill, though..

It is not only overkill, it would require a merging strategy which
AFAIK portage currently does not use and which would lead to blockers:

If you really want to guarantee that all := dependencies are
satisfied in the moment of unmerging, you need to do the following.

Assume foo depends on bar:=. Now if bar gets an upgrade, first
foo must be unmerged, then bar must be upgraded, and then
foo must be reemerged.

If you have a circular dependency of := chains, an upgrade
is impossible forever: You would manually have to break this
chain by setting some useflags exactly in the same way as
you did when you manages to install the chain for the first time.
Otherwise, you will just miss the possibly avilable updates
forever.

The problems if you really want to guarantee that all
dependencies are still installed when unmerging a package
are hardly reasonably solvable.

One should better make a rule that if an ebuild has such a
special requirement that it needs a certain version for
being unmerged, it must block all other versions
(one could make a syntax for the := case or even disallow
this case for such ebuilds) instead of letting the PM jump
through hoops for the "normal" cases.

Re: [gentoo-dev] don't rely on dynamic deps

[Michał Górny]

[-- Attachment #1: Type: text/plain, Size: 1394 bytes --]

Dnia 2014-07-21, o godz. 23:06:07
Pacho Ramos <pacho@gentoo.org> napisał(a):

> El lun, 21-07-2014 a las 20:55 +0100, Ciaran McCreesh escribió:
> > On Mon, 21 Jul 2014 21:53:04 +0200
> > "Andreas K. Huettel" <dilfridge@gentoo.org> wrote:
> > > Revision must be bumped when the on-disk files installed by the
> > > ebuild are changed. 
> > > Nothing about dependencies.
> > > 
> > > This has been policy for a LONG time, and we're not going to change
> > > it overnight just because you protest.
> > 
> > Policy used to be that you'd do a revbump when you wanted users to
> > reinstall stuff, and you wouldn't otherwise. The only complication is
> > that sometimes you want users to reinstall stuff so that there's
> > accurate dependency information available, rather than because
> > something has changed.
> > 
> 
> Maybe this could be solved by having two kinds of revisions:
> - One would rebuild all as usually (for example, -r1...)
> - The other one would only regenerate VDB and wouldn't change the
> installed files (for example, -r1.1)
> 
> But I am not sure if it could be viable from a "technical" point of
> view :(

I'm afraid it couldn't. The major problem is not knowing *when* to
migrate metadata, portage usually gets that right. The problem is in
getting the correct output which is often near to impossible.

-- 
Best regards,
Michał Górny

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 949 bytes --]

Re: [gentoo-dev] don't rely on dynamic deps

[Pacho Ramos]

El vie, 25-07-2014 a las 00:06 +0200, Michał Górny escribió:
[...]
> > Maybe this could be solved by having two kinds of revisions:
> > - One would rebuild all as usually (for example, -r1...)
> > - The other one would only regenerate VDB and wouldn't change the
> > installed files (for example, -r1.1)
> > 
> > But I am not sure if it could be viable from a "technical" point of
> > view :(
> 
> I'm afraid it couldn't. The major problem is not knowing *when* to
> migrate metadata, portage usually gets that right. The problem is in
> getting the correct output which is often near to impossible.
> 

But, isn't it something up to developer in that case? I mean, we should
have a list of things that deserve that VDB regeneration and, then, make
the -r1.1 revbump instead of the major one.

Re: Re: [gentoo-dev] don't rely on dynamic deps

[Andreas K. Huettel]

> > Maybe this could be solved by having two kinds of revisions:
> > - One would rebuild all as usually (for example, -r1...)
> > - The other one would only regenerate VDB and wouldn't change the
> > installed files (for example, -r1.1)
> > 
> > But I am not sure if it could be viable from a "technical" point of
> > view :(
> 
> I'm afraid it couldn't. The major problem is not knowing *when* to
> migrate metadata, portage usually gets that right. The problem is in
> getting the correct output which is often near to impossible.

I think we'd appreciate some more information here:

* What exactly is the metadata that we're talking about?
* How much of it can be generated by sourcing the ebuild, without running 
phases?
* When does that exactly break? Example?

Thanks!!!

-- 
Andreas K. Huettel
Gentoo Linux developer
kde, council

Re: [gentoo-dev] don't rely on dynamic deps

[Ian Stakenvicius]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

On 25/07/14 01:15 PM, Andreas K. Huettel wrote:
> 
>>> Maybe this could be solved by having two kinds of revisions: -
>>> One would rebuild all as usually (for example, -r1...) - The
>>> other one would only regenerate VDB and wouldn't change the 
>>> installed files (for example, -r1.1)
>>> 
>>> But I am not sure if it could be viable from a "technical"
>>> point of view :(
>> 
>> I'm afraid it couldn't. The major problem is not knowing *when*
>> to migrate metadata, portage usually gets that right. The problem
>> is in getting the correct output which is often near to
>> impossible.
> 
> I think we'd appreciate some more information here:
> 
> * What exactly is the metadata that we're talking about? * How much
> of it can be generated by sourcing the ebuild, without running 
> phases? * When does that exactly break? Example?
> 

It may help this overall discussion to step back even further here:

* What do we want to accomplish, exactly?

 - allow portage to emerge updated packages without rebuilding them
(ie running src_* and pkg_*inst phases), when it can be determined
from metadata changes (or for -rX.Y method perhaps it is assumed
- -just- on filename changes) that this is not necessary.

* What are all of the cases that would validly trigger this
dont-bother-to-run-phases shortcut?

- - addition of missing dependency [1]
- - new useflag not enabled by default
- - removed useflag that was not previously enabled
- - changing of dependency atom(s) -- ie swapping a specific atom to
virtual one
- - EAPI bump (maybe??)
- - ....  <-- add more please!


* For each of the above, in what cases is a full rebuild probably
necessary anyhow??

- - dependency atom minimum version is increased, and the in-VDB version
of this dep is too old (??)
- - new dependency atom is missing from VDB (??)
- - EAPI bump adds new entries to VDB that need to be calculated from
say, merge phase
- - ....



..i think from there we can perhaps determine what metadata could be
updated and/or needs to be updated to maintain consistency in the
dont-rebuild cases we want.
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2

iF4EAREIAAYFAlPSlY0ACgkQ2ugaI38ACPBDHAD+POgCJlbPv9HHwhK4wxd8b2ir
ywM71Aemu6SPfCTE2MIBAKATag94jCHLlqwkYN2jrW23tYqTi5ajOwLzoZ/EqHx0
=qUpY
-----END PGP SIGNATURE-----

Re: [gentoo-dev] don't rely on dynamic deps

[Ian Stakenvicius]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

On 25/07/14 01:36 PM, Ian Stakenvicius wrote:
> On 25/07/14 01:15 PM, Andreas K. Huettel wrote:
> 
>>>> Maybe this could be solved by having two kinds of revisions:
>>>> - One would rebuild all as usually (for example, -r1...) -
>>>> The other one would only regenerate VDB and wouldn't change
>>>> the installed files (for example, -r1.1)
>>>> 
>>>> But I am not sure if it could be viable from a "technical" 
>>>> point of view :(
>>> 
>>> I'm afraid it couldn't. The major problem is not knowing
>>> *when* to migrate metadata, portage usually gets that right.
>>> The problem is in getting the correct output which is often
>>> near to impossible.
> 
>> I think we'd appreciate some more information here:
> 
>> * What exactly is the metadata that we're talking about? * How
>> much of it can be generated by sourcing the ebuild, without
>> running phases? * When does that exactly break? Example?
> 
> 
> It may help this overall discussion to step back even further
> here:
> 
> * What do we want to accomplish, exactly?
> 
> - allow portage to emerge updated packages without rebuilding them 
> (ie running src_* and pkg_*inst phases), when it can be determined 
> from metadata changes (or for -rX.Y method perhaps it is assumed 
> -just- on filename changes) that this is not necessary.
> 
> * What are all of the cases that would validly trigger this 
> dont-bother-to-run-phases shortcut?
> 
> - addition of missing dependency [1] - new useflag not enabled by
> default - removed useflag that was not previously enabled -
> changing of dependency atom(s) -- ie swapping a specific atom to 
> virtual one - EAPI bump (maybe??) - ....  <-- add more please!
> 
> 
> * For each of the above, in what cases is a full rebuild probably 
> necessary anyhow??
> 
> - dependency atom minimum version is increased, and the in-VDB
> version of this dep is too old (??) - new dependency atom is
> missing from VDB (??) - EAPI bump adds new entries to VDB that need
> to be calculated from say, merge phase - ....
> 
> 
> 
> ..i think from there we can perhaps determine what metadata could
> be updated and/or needs to be updated to maintain consistency in
> the dont-rebuild cases we want.
> 

I forgot: [1] since the package has merged prior to this anyhow,
whatever dependency it was must have been installed already when this
package was emerged.  So if it can still be found in vdb then we
should be safe to add the dependency to vdb of this package, otherwise
we may need to trigger a rebuild.
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2

iF4EAREIAAYFAlPSllcACgkQ2ugaI38ACPCL3QD+ITe3bzJrJIORniniY9rzTEd5
W5nmL5zB+HiZuMZKdZQA/jrn88lx8bfn/f6DetPdrYiFjjShcXoeMbh5nCNmtcz9
=LEAi
-----END PGP SIGNATURE-----

Re: [gentoo-dev] don't rely on dynamic deps

[Taahir Ahmed]

[-- Attachment #1: Type: text/plain, Size: 2430 bytes --]

It seems like a simple before/after comparison of active useflags and the text 
of the src_* functions (skipping build and install if they are completely 
identical) should catch the majority of unnecessary rebuilds.

On 25 July 2014 13:36 Ian Stakenvicius wrote:
> On 25/07/14 01:15 PM, Andreas K. Huettel wrote:
> >
> >>> Maybe this could be solved by having two kinds of revisions: -
> >>> One would rebuild all as usually (for example, -r1...) - The
> >>> other one would only regenerate VDB and wouldn't change the
> >>> installed files (for example, -r1.1)
> >>>
> >>> But I am not sure if it could be viable from a "technical"
> >>> point of view :(
> >>
> >> I'm afraid it couldn't. The major problem is not knowing *when*
> >> to migrate metadata, portage usually gets that right. The problem
> >> is in getting the correct output which is often near to
> >> impossible.
> >
> > I think we'd appreciate some more information here:
> >
> > * What exactly is the metadata that we're talking about? * How much
> > of it can be generated by sourcing the ebuild, without running
> > phases? * When does that exactly break? Example?
> >
> 
> It may help this overall discussion to step back even further here:
> 
> * What do we want to accomplish, exactly?
> 
>  - allow portage to emerge updated packages without rebuilding them
> (ie running src_* and pkg_*inst phases), when it can be determined
> from metadata changes (or for -rX.Y method perhaps it is assumed
> -just- on filename changes) that this is not necessary.
> 
> * What are all of the cases that would validly trigger this
> dont-bother-to-run-phases shortcut?
> 
> - addition of missing dependency [1]
> - new useflag not enabled by default
> - removed useflag that was not previously enabled
> - changing of dependency atom(s) -- ie swapping a specific atom to
> virtual one
> - EAPI bump (maybe??)
> - ....  <-- add more please!
> 
> 
> * For each of the above, in what cases is a full rebuild probably
> necessary anyhow??
> 
> - dependency atom minimum version is increased, and the in-VDB version
> of this dep is too old (??)
> - new dependency atom is missing from VDB (??)
> - EAPI bump adds new entries to VDB that need to be calculated from
> say, merge phase
> - ....
> 
> 
> 
> ..i think from there we can perhaps determine what metadata could be
> updated and/or needs to be updated to maintain consistency in the
> dont-rebuild cases we want.
> 

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 473 bytes --]

Re: [gentoo-dev] don't rely on dynamic deps

[Taahir Ahmed]

[-- Attachment #1: Type: text/plain, Size: 31 bytes --]

My apologies for the top-reply.

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 473 bytes --]

[gentoo-dev] Re: don't rely on dynamic deps

[Martin Vaeth]

Michał Górny <mgorny@gentoo.org> wrote:
>> Maybe this could be solved by having two kinds of revisions:
>> - One would rebuild all as usually (for example, -r1...)
>> - The other one would only regenerate VDB and wouldn't change the
>> installed files (for example, -r1.1)
>
> I'm afraid it couldn't. The major problem is not knowing *when* to
> migrate metadata, portage usually gets that right. The problem is in
> getting the correct output which is often near to impossible.

Could you explain where you see here a problem with -r1.1
which is not caused as well with -r2?

The only difference should be that when revbumping -r1 to -r1.1
there is actually no recompilation done (and perhaps the
PF and PR variables are treated differently) - everything else
should be exactly the same as for current revbumps.

And once more, this is only one of the several possibilities
how to tell portage that actually no compilation is necessary:
Some other metadata/variable/whatever might be used as well.

The idea is to act "as usual", just to skip unnecessary phases...

Re: [gentoo-dev] Re: don't rely on dynamic deps

[Ciaran McCreesh]

[-- Attachment #1: Type: text/plain, Size: 421 bytes --]

On Sat, 26 Jul 2014 13:41:34 +0000 (UTC)
Martin Vaeth <martin@mvath.de> wrote:
> The idea is to act "as usual", just to skip unnecessary phases...

So someone adds optional selinux support to a package, and then you end
up with selinux being "on", despite not having it, and then another
package depends upon your package with [selinux], and the dependency is
mistakenly treated as met...

-- 
Ciaran McCreesh

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 181 bytes --]

[gentoo-dev] Re: don't rely on dynamic deps

[Martin Vaeth]

Ciaran McCreesh <ciaran.mccreesh@googlemail.com> wrote:
> Martin Vaeth <martin@mvath.de> wrote:
>> The idea is to act "as usual", just to skip unnecessary phases...
>
> So someone adds optional selinux support to a package, and then you end
> up with selinux being "on", despite not having it, and then another
> package depends upon your package with [selinux], and the dependency is
> mistakenly treated as met...

If the developer has added IUSE=selinux and bumps from -r1 to -r1.1,
he has of course verified that this USE-change does not require
recompilation either way, since otherwise he would not have been
allowed to explicitly say the package manager that recompilation
is unnecessary.
So the dependency is *correctly* treated as met.

Re: [gentoo-dev] Re: don't rely on dynamic deps

[Michał Górny]

[-- Attachment #1: Type: text/plain, Size: 971 bytes --]

Dnia 2014-07-26, o godz. 15:01:46
Martin Vaeth <martin@mvath.de> napisał(a):

> Ciaran McCreesh <ciaran.mccreesh@googlemail.com> wrote:
> > Martin Vaeth <martin@mvath.de> wrote:
> >> The idea is to act "as usual", just to skip unnecessary phases...
> >
> > So someone adds optional selinux support to a package, and then you end
> > up with selinux being "on", despite not having it, and then another
> > package depends upon your package with [selinux], and the dependency is
> > mistakenly treated as met...
> 
> If the developer has added IUSE=selinux and bumps from -r1 to -r1.1,
> he has of course verified that this USE-change does not require
> recompilation either way, since otherwise he would not have been
> allowed to explicitly say the package manager that recompilation
> is unnecessary.
> So the dependency is *correctly* treated as met.

Excuse me but are we talking about updating *DEPEND or IUSE?

-- 
Best regards,
Michał Górny

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 949 bytes --]

[gentoo-dev] Re: don't rely on dynamic deps

[Martin Vaeth]

Michał Górny <mgorny@gentoo.org> wrote:
>
>> Ciaran McCreesh <ciaran.mccreesh@googlemail.com> wrote:
>> > Martin Vaeth <martin@mvath.de> wrote:
>> >> The idea is to act "as usual", just to skip unnecessary phases...
>> >
>> > So someone adds optional selinux support to a package, and then you end
>> > up with selinux being "on", despite not having it, and then another
>> > package depends upon your package with [selinux], and the dependency is
>> > mistakenly treated as met...
>> If the developer has added IUSE=3Dselinux and bumps from -r1 to -r1.1,
>> he has of course verified that this USE-change does not require
>> recompilation either way, since otherwise he would not have been
>> allowed to explicitly say the package manager that recompilation
>> is unnecessary.
>> So the dependency is *correctly* treated as met.
>
> Excuse me but are we talking about updating *DEPEND or IUSE?

You are right, according to my original suggestion, the developer
should not even have been allowed to bump only a minor revision.
This is already discussing some possible extension/misuse
of the feature.

Re: [gentoo-dev] don't rely on dynamic deps

[Michał Górny]

[-- Attachment #1: Type: text/plain, Size: 2120 bytes --]

Dnia 2014-07-21, o godz. 21:53:04
"Andreas K. Huettel" <dilfridge@gentoo.org> napisał(a):

> Am Montag, 21. Juli 2014, 21:37:17 schrieb hasufell:
> > afaiu dynamic deps are broken and not defined in PMS
> > 
> > still... people seem to fix deps without revbumping, causing users who
> > either don't use dynamic deps (it's optional for portage through
> > --dynamic-deps=y, although it's on by default) or who use a different PM
> > to not get the fix, at worst resulting in broken dependency calculation
> > 
> > suggestion:
> > * stop fixing dependencies without revbumping
> > * add an appropriate question to the dev quiz
> > * remove dynamic deps from portage (afair that is already considered by
> > the portage team)
> 
> Actually the quizzes are pretty much clear on that. 
> 
> Revision must be bumped when the on-disk files installed by the ebuild are 
> changed. 
> Nothing about dependencies.
> 
> This has been policy for a LONG time, and we're not going to change it 
> overnight just because you protest.

This is not an argument. Just because we're doing things wrong for
a very long time doesn't mean we have to keep doing that. Especially
now that the breakage is getting much more visible with spread of
EAPI=5 and subslots.

> Now... whether dynamic deps are technically the right thing to do is another 
> question. It merits discussion, but we need to be really sure about the 
> consequences of any change.

Yes, it does. I'm not sure if it leads anywhere, though. Dynamic deps
are a pipe dream. You can't implement them properly, so we're using
half-working implementation as an excuse to be lazy.

Of course, the problem is that many developers just assume how dynamic
deps work. They don't know the details, they don't test it. They just
say 'I need not to revbump because dynamic-deps!' Then stuff breaks
because dynamic deps don't work. Or sometimes because they actually
work and the developer didn't think of them. Or sometimes because they
randomly start and stop working depending on the phase of the moon.

-- 
Best regards,
Michał Górny

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 949 bytes --]

Re: [gentoo-dev] don't rely on dynamic deps

[Samuli Suominen]

On 21/07/14 23:56, Michał Górny wrote:
> Now... whether dynamic deps are technically the right thing to do is another 
> question. It merits discussion, but we need to be really sure about the 
> consequences of any change.
> Yes, it does. I'm not sure if it leads anywhere, though. Dynamic deps
> are a pipe dream. You can't implement them properly, so we're using
> half-working implementation as an excuse to be lazy.

What's lazy is maintainer doing revision bump without thinking
if it's really required, spreading his laziness upon every users
machine (by triggering revision bump driven rebuild)

Re: [gentoo-dev] don't rely on dynamic deps

[Michał Górny]

[-- Attachment #1: Type: text/plain, Size: 872 bytes --]

Dnia 2014-07-22, o godz. 00:13:13
Samuli Suominen <ssuominen@gentoo.org> napisał(a):

> 
> On 21/07/14 23:56, Michał Górny wrote:
> > Now... whether dynamic deps are technically the right thing to do is another 
> > question. It merits discussion, but we need to be really sure about the 
> > consequences of any change.
> > Yes, it does. I'm not sure if it leads anywhere, though. Dynamic deps
> > are a pipe dream. You can't implement them properly, so we're using
> > half-working implementation as an excuse to be lazy.
> 
> What's lazy is maintainer doing revision bump without thinking
> if it's really required, spreading his laziness upon every users
> machine (by triggering revision bump driven rebuild)

Yes, users much more prefer random breakage over time. And debugging
the issues save us a lot of time!

-- 
Best regards,
Michał Górny

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 949 bytes --]

Re: [gentoo-dev] don't rely on dynamic deps

[William Hubbs]

[-- Attachment #1: Type: text/plain, Size: 379 bytes --]

I'm picking a random msg to reply to.

My concern about doing a revbump just because the deps change is that
the new revision has to be committed in ~arch, so we then have to hit
the arch teams, which we know are overworked anyway, with stable
requests just because we changed the dependencies. Isn't that causing a
lot of possibly unnecessary work for our arch teams?

William


[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 181 bytes --]

Re: [gentoo-dev] don't rely on dynamic deps

[hasufell]

William Hubbs:
> I'm picking a random msg to reply to.
> 
> My concern about doing a revbump just because the deps change is that
> the new revision has to be committed in ~arch, so we then have to hit
> the arch teams, which we know are overworked anyway, with stable
> requests just because we changed the dependencies. Isn't that causing a
> lot of possibly unnecessary work for our arch teams?
> 

Procedure over logic?

Just commit it straight to arch if repoman doesn't complain.

Re: [gentoo-dev] don't rely on dynamic deps

[Alexander Berntsen]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

On 22/07/14 02:36, hasufell wrote:
> William Hubbs:
>> My concern about doing a revbump just because the deps change is
>> that the new revision has to be committed in ~arch, so we then
>> have to hit the arch teams, which we know are overworked anyway,
>> with stable requests just because we changed the dependencies.
>> Isn't that causing a lot of possibly unnecessary work for our
>> arch teams?
> Procedure over logic?
> 
> Just commit it straight to arch if repoman doesn't complain.
William,

this is, as Julian pointed out, a problem you can solve by changing
your policies. This is not a problem related to the Portage software,
in which dynamic-deps are broken.
- -- 
Alexander
bernalex@gentoo.org
https://secure.plaimi.net/~alexander
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/

iF4EAREIAAYFAlPOMgYACgkQRtClrXBQc7VuuwD/UNQzX5aHSBbfXhyYRxH4oYzK
N9aEf88WLVJK2JVKJBkA/iDF6ozQ9I0WKWpi/jvZa/W7yxKeZsXu5ACa5mbgM88+
=9/RG
-----END PGP SIGNATURE-----

Re: [gentoo-dev] don't rely on dynamic deps

[William Hubbs]

[-- Attachment #1: Type: text/plain, Size: 1124 bytes --]

On Tue, Jul 22, 2014 at 11:42:30AM +0200, Alexander Berntsen wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA256
> 
> On 22/07/14 02:36, hasufell wrote:
> > William Hubbs:
> >> My concern about doing a revbump just because the deps change is
> >> that the new revision has to be committed in ~arch, so we then
> >> have to hit the arch teams, which we know are overworked anyway,
> >> with stable requests just because we changed the dependencies.
> >> Isn't that causing a lot of possibly unnecessary work for our
> >> arch teams?
> > Procedure over logic?
> > 
> > Just commit it straight to arch if repoman doesn't complain.
> William,
> 
> this is, as Julian pointed out, a problem you can solve by changing
> your policies. This is not a problem related to the Portage software,
> in which dynamic-deps are broken.

s/your/our/

Repoman refuses to commit if you try to go directly to stable without
using --force.

I'm just being cautious; I'm not sure whether this qualifies as the type
of emergency situation where commiting directly to stable is a good
thing or not.

William


[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 181 bytes --]

Re: [gentoo-dev] don't rely on dynamic deps

[Rick "Zero_Chaos" Farina]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On 07/21/2014 06:52 PM, William Hubbs wrote:
> I'm picking a random msg to reply to.
> 
> My concern about doing a revbump just because the deps change is that
> the new revision has to be committed in ~arch, so we then have to hit
> the arch teams, which we know are overworked anyway, with stable
> requests just because we changed the dependencies. Isn't that causing a
> lot of possibly unnecessary work for our arch teams?

If you edit an ebuild in place, and that ebuild has stable keywords, how
is that any different than revbumping it and keeping it with stable
keywords? The two are functionally identical, people get a changed
ebuild, and the arch teams do nothing. The only difference is that 1000
different ways dynamic deps are broken.

And just for fun, since no one has mentioned it yet, dynamic deps don't
work at all on binpkgs since the Packages file contains the deps (like
vardb) and it doesn't get updated (just like vardb).

Revbump or make dynamic deps actually work (ha).

- -Zero
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/

iQIcBAEBAgAGBQJTzbjHAAoJEKXdFCfdEflKVL4P/2Sb88YYcbHJZ1pfnVukxvvi
zYD00knmGiQOcmSxZMPnhVXjV+Z1SzyOPtmZa5KXLd2xwmVvNwJdUV0ssjfDi6Gg
fxnAEWWuZ6xEvpIlm7WIXq5VFGL/AO4HUso6mHqSVZbbgYLPiFCooSAn291Epmjn
vGIkUf5pNtRp6gLQFTqM2ksDzJWaOF9bmuapcCSumv3uhKK2Empy+6kHTKYVd/m5
Mvo10p7W8amozV+pl7Si7tWGOf/U+Xo3N0gt9VJzz8oxIpooXZ+nSB3p9bXSbKZN
sspz9khHLKFyhOAfoBbtLB9cqwPfPqgMlt6h4n9l2uJ+E0nMz0vRIlQnMarKD2FE
/3DG6zrvWU0eRuYM7c5iqOrlL1WYgAA3p6CEBo5U9+h+8eF2bgxAJt/c7txJG0Ws
EpSlwOIFyfVOWaJeb1WpBt72sMH+URF/YRl6K0ZKmp7+X5Ga/39ZznRfhRblRp9h
nBMyIer9Ai7wNbc7A1qMvjedqOknOwSJjKEDkaUAIn5RTFa5VmLAGURSeuQnIvjE
ogel9ENjdoZJW0i8bEO8ArCS9X5vgdWiwZHcjZL62KNYoqc6ySrsTUm0Eo3vDevo
wr782AzzLPVftPXNHMnEthBs5ztbLr6YlXv/V5jfCc1YRQGUeEMjBkny/Nxm1VLi
/l1lbtaTcm5FcqSjen4Q
=asgW
-----END PGP SIGNATURE-----

Re: [gentoo-dev] don't rely on dynamic deps

[Samuli Suominen]

On 22/07/14 04:05, Rick "Zero_Chaos" Farina wrote:
>
> And just for fun, since no one has mentioned it yet, dynamic deps don't
> work at all on binpkgs since the Packages file contains the deps (like
> vardb) and it doesn't get updated (just like vardb).

Known long standing pitfall. It's managable.

>
> Revbump or make dynamic deps actually work (ha).
>

If they are really as broken people claim, why is the feature enabled by
default in the
official package manager?
As in, why I'm not seeing a bug with title "sys-apps/portage: disable
dynamic deps by default"
with a Comment #0 listing all the culprits why it should be disabled by
default?
Or do people think it works well enough, so that it's pros overweight
the cons? I do,
and many others seem to think so as well.

Re: [gentoo-dev] don't rely on dynamic deps

[Tom Wijsman]

[-- Attachment #1: Type: text/plain, Size: 1690 bytes --]

On Tue, 22 Jul 2014 08:10:20 +0300
Samuli Suominen <ssuominen@gentoo.org> wrote:

> On 22/07/14 04:05, Rick "Zero_Chaos" Farina wrote:
> >
> > And just for fun, since no one has mentioned it yet, dynamic deps
> > don't work at all on binpkgs since the Packages file contains the
> > deps (like vardb) and it doesn't get updated (just like vardb).
> 
> Known long standing pitfall. It's managable.

It is one of the reasons I see binpkgs as breaking progress instead of
being part of the progress; it is manageable, but it could be better.

> > Revbump or make dynamic deps actually work (ha).
> 
> If they are really as broken people claim, why is the feature enabled
> by default in the
> official package manager?

Why are these discussions resounding more strongly with each occurrence?

> As in, why I'm not seeing a bug with title "sys-apps/portage: disable
> dynamic deps by default"

https://bugs.gentoo.org/show_bug.cgi?id=516612

> with a Comment #0 listing all the culprits why it should be disabled
> by default?

They've been gathered throughout this thread; yes, they could use a
summary, but that doesn't change that all the culprits exist out there.

> Or do people think it works well enough, so that it's pros overweight
> the cons? I do, 
> and many others seem to think so as well.

Depends on what the pros and cons are; it makes the difference between
merely thoughts and an actual reality, to construct towards a decision.

-- 
With kind regards,

Tom Wijsman (TomWij)
Gentoo Developer

E-mail address  : TomWij@gentoo.org
GPG Public Key  : 6D34E57D
GPG Fingerprint : C165 AF18 AB4C 400B C3D2  ABF0 95B2 1FCD 6D34 E57D

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 473 bytes --]

Re: [gentoo-dev] don't rely on dynamic deps

[Patrick Lauer]

On Wednesday 23 July 2014 01:06:15 Tom Wijsman wrote:
> On Tue, 22 Jul 2014 08:10:20 +0300
> 
> Samuli Suominen <ssuominen@gentoo.org> wrote:
> > On 22/07/14 04:05, Rick "Zero_Chaos" Farina wrote:
> > > And just for fun, since no one has mentioned it yet, dynamic deps
> > > don't work at all on binpkgs since the Packages file contains the
> > > deps (like vardb) and it doesn't get updated (just like vardb).
> > 
> > Known long standing pitfall. It's managable.
> 
> It is one of the reasons I see binpkgs as breaking progress instead of
> being part of the progress; it is manageable, but it could be better.
> 

So you'd rather have people not using Gentoo because you can't agilely pivot 
your strategy mixin?

Without binpkg support I'd feel the need to hack it up, just to get things 
fast enough. It's one of the features that haven't been made popular enough 
(eh, we could easily provide binpkg-updates for @system for all profiles and 
the most common arches (amd64, x86, maybe arm) - I've been running a cronjob 
doing that for x86+amd64 for about a year now)

This perspective of "I don't need it, thus it shouldn't exist" is quite ... 
amusing.

[gentoo-dev] binpkg (was: don't rely on dynamic deps)

[Martin Vaeth]

Patrick Lauer <patrick@gentoo.org> wrote:
>
> Without binpkg support I'd feel the need to hack it up, just to get things
> fast enough.

binpkg support has some severe problems currently, anyway.
I already described it in the bug, but since perhaps the
description was not clear, I repeat it here:

I regularly build binpackages for some client.
This client is updated only very infrequently
(once every few months).

Of course, before I emerge the binpkgs, I update the
portage tree on that client, and also copy my
/etc/portage/package* directories.

Then it turns out that I can use *less* than half of
my binpkgs on the client: The binpkgs are just ignored.

One reason I found: I had temporarily set ~ARCH for some
package in /etc/portage/package.accept_keywords
when building the package, and removed the entry again
(and thus also on the client) once the package has become
stable. I hope that you agree that my expectation is
sane that the binpkg should be used anyway - but it isn't.

For others I do not know the reason, but probably it is
very similar that some variables/settings at the time
of building the package were different from the current
ones in the tree.

So, I do not know all the details, but currently
binpkg support is somewhat broken already.
What we would need is *more* pushing of information of
the current tree data to binpkgs, not *less*.
So dynamic deps (or another update mechanism)
appear to be a first step in *fixing* support for binpkgs...

Re: [gentoo-dev] don't rely on dynamic deps

[Tom Wijsman]

[-- Attachment #1: Type: text/plain, Size: 1885 bytes --]

On Sat, 26 Jul 2014 12:36:31 +0800
Patrick Lauer <patrick@gentoo.org> wrote:

> On Wednesday 23 July 2014 01:06:15 Tom Wijsman wrote:
> > On Tue, 22 Jul 2014 08:10:20 +0300
> > 
> > Samuli Suominen <ssuominen@gentoo.org> wrote:
> > > On 22/07/14 04:05, Rick "Zero_Chaos" Farina wrote:
> > > > And just for fun, since no one has mentioned it yet, dynamic
> > > > deps don't work at all on binpkgs since the Packages file
> > > > contains the deps (like vardb) and it doesn't get updated (just
> > > > like vardb).
> > > 
> > > Known long standing pitfall. It's managable.
> > 
> > It is one of the reasons I see binpkgs as breaking progress instead
> > of being part of the progress; it is manageable, but it could be
> > better.
> 
> So you'd rather have people not using Gentoo because you can't
> agilely pivot your strategy mixin?

No, that's not what I've said; the paragraph you quote is based on an
observation, not a statement. In that observation; if people want a
mixin, they would go for a full mixin and not for a half broken mixin.

> Without binpkg support I'd feel the need to hack it up, just to get
> things fast enough. It's one of the features that haven't been made
> popular enough (eh, we could easily provide binpkg-updates for
> @system for all profiles and the most common arches (amd64, x86,
> maybe arm) - I've been running a cronjob doing that for x86+amd64 for
> about a year now)
> 
> This perspective of "I don't need it, thus it shouldn't exist" is
> quite ... amusing.

+1; "It doesn't work, it shouldn't exist" is less amusing, more ideal
it should be "it doesn't work, let's fix or replace it" if we can...

-- 
With kind regards,

Tom Wijsman (TomWij)
Gentoo Developer

E-mail address  : TomWij@gentoo.org
GPG Public Key  : 6D34E57D
GPG Fingerprint : C165 AF18 AB4C 400B C3D2  ABF0 95B2 1FCD 6D34 E57D

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 473 bytes --]

Re: [gentoo-dev] don't rely on dynamic deps

[Tom Wijsman]

[-- Attachment #1: Type: text/plain, Size: 709 bytes --]

On Mon, 21 Jul 2014 17:52:51 -0500
William Hubbs <williamh@gentoo.org> wrote:

> My concern about doing a revbump just because the deps change is that
> the new revision has to be committed in ~arch, so we then have to hit
> the arch teams, which we know are overworked anyway, with stable
> requests just because we changed the dependencies. Isn't that causing
> a lot of possibly unnecessary work for our arch teams?

Rev bumps that only add dependencies shouldn't need separate STABLEREQs.

-- 
With kind regards,

Tom Wijsman (TomWij)
Gentoo Developer

E-mail address  : TomWij@gentoo.org
GPG Public Key  : 6D34E57D
GPG Fingerprint : C165 AF18 AB4C 400B C3D2  ABF0 95B2 1FCD 6D34 E57D

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 473 bytes --]

Re: [gentoo-dev] don't rely on dynamic deps

[Alexandre Rostovtsev]

On Mon, 2014-07-21 at 22:56 +0200, Michał Górny wrote:
> Yes, it does. I'm not sure if it leads anywhere, though. Dynamic deps
> are a pipe dream. You can't implement them properly, so we're using
> half-working implementation as an excuse to be lazy.

Why not adapt the updates mechanism for modifying rdepends? Perhaps
something like

rdepends-add foo-bar/blah-3.14 "wombat? ( >=dev-libs/wombat-1.0 )"

This would give the package manager all the benefits of static dep
resolution while allowing us to dynamically make simple changes (like
adding a missing dependency to an ebuild) without forcing users to
rebuild the package.

Re: [gentoo-dev] don't rely on dynamic deps

[Tom Wijsman]

[-- Attachment #1: Type: text/plain, Size: 1576 bytes --]

On Mon, 21 Jul 2014 21:34:10 -0400
Alexandre Rostovtsev <tetromino@gentoo.org> wrote:

> On Mon, 2014-07-21 at 22:56 +0200, Michał Górny wrote:
> > Yes, it does. I'm not sure if it leads anywhere, though. Dynamic
> > deps are a pipe dream. You can't implement them properly, so we're
> > using half-working implementation as an excuse to be lazy.
> 
> Why not adapt the updates mechanism for modifying rdepends? Perhaps
> something like
> 
> rdepends-add foo-bar/blah-3.14 "wombat? ( >=dev-libs/wombat-1.0 )"
> 
> This would give the package manager all the benefits of static dep
> resolution while allowing us to dynamically make simple changes (like
> adding a missing dependency to an ebuild) without forcing users to
> rebuild the package.

Thinking this through:

 1) What about rdepends-change and rdepends-del? If you only support
 addition; you get the same problem as with things like pkgmove,
 changing and/or removing it could become somewhat problematic.

 2) This needs two commits every time you want to do this; one commit
 for the updates/, another to keep the ebuild recent for (rev)bumps.

 3) It'll be a lot of fun to attempt to support this in Repoman.

 4) How do we clean up these entries? Doesn't this info grow fast?

 5) The first paramater: Should that point to a single ebuild? Should
 that support ranges?

-- 
With kind regards,

Tom Wijsman (TomWij)
Gentoo Developer

E-mail address  : TomWij@gentoo.org
GPG Public Key  : 6D34E57D
GPG Fingerprint : C165 AF18 AB4C 400B C3D2  ABF0 95B2 1FCD 6D34 E57D

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 473 bytes --]

Re: [gentoo-dev] don't rely on dynamic deps

[Alexandre Rostovtsev]

[-- Attachment #1: Type: text/plain, Size: 2432 bytes --]

On Wed, 2014-07-23 at 01:13 +0200, Tom Wijsman wrote:
> On Mon, 21 Jul 2014 21:34:10 -0400
> Alexandre Rostovtsev <tetromino@gentoo.org> wrote:
> > Why not adapt the updates mechanism for modifying rdepends? Perhaps
> > something like
> > 
> > rdepends-add foo-bar/blah-3.14 "wombat? ( >=dev-libs/wombat-1.0 )"
> > 
> > This would give the package manager all the benefits of static dep
> > resolution while allowing us to dynamically make simple changes (like
> > adding a missing dependency to an ebuild) without forcing users to
> > rebuild the package.
> 
> Thinking this through:
> 
>  1) What about rdepends-change and rdepends-del? If you only support
>  addition; you get the same problem as with things like pkgmove,
>  changing and/or removing it could become somewhat problematic.

rdepends-add is easy to implement: just append a string to RDEPENDS and
reparse it. Deletion is trickier. It's clear what it means to delete an
atom from a flat list of atoms. But specifying what it means to delete a
part of a complex expression (conditionals, disjunctions) requires some
careful thought.

>  2) This needs two commits every time you want to do this; one commit
>  for the updates/, another to keep the ebuild recent for (rev)bumps.

Yes, but that's just more incentive to migrate to git :)

>  3) It'll be a lot of fun to attempt to support this in Repoman.

Ideally, repoman would suggest what you should add to updates/ when it
detects that you are committing a change to an existing ebuild.

>  4) How do we clean up these entries? Doesn't this info grow fast?

The point is to *not* clean up these entries for months/years. I want to
address the situation where a users installs from an ebuild with wrong
dependencies, the next day the ebuild gets its dependencies fixed in
cvs, two weeks later the ebuild gets deleted from cvs, and only then the
user resyncs. With our current dynamic depends, as well as with the
proposed "minor revisions" mechanism, the user will still have a broken
vdb. I want to make sure the user's vdb gets updated even when the
ebuild in question is gone from portage.

>  5) The first paramater: Should that point to a single ebuild? Should
>  that support ranges?

Just like everything else in profiles/, so for now, that means a single
dependency specification. If you want version ranges, they should be
allowed in package.mask too :)

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 819 bytes --]

[gentoo-dev] Re: don't rely on dynamic deps

[Martin Vaeth]

Alexandre Rostovtsev <tetromino@gentoo.org> wrote:
>
> rdepends-add is easy to implement [...] Deletion is trickier [...]
>
> The point is to *not* clean up these entries for months/years.

So, essentially, you want the developer to do part of CVS/git's job,
namely keeping a history of changes in a compressed format,
keeping the history forever (or almost forever).
As mentioned in another post, you highly understimate the
amount of data which would have to be treated this way:
For every python release and many other eclass changes,
almost all packages in the tree are involved, usually
several times a months.

Re: [gentoo-dev] Re: don't rely on dynamic deps

[Michał Górny]

[-- Attachment #1: Type: text/plain, Size: 1402 bytes --]

Dnia 2014-07-26, o godz. 14:02:29
Martin Vaeth <martin@mvath.de> napisał(a):

> Alexandre Rostovtsev <tetromino@gentoo.org> wrote:
> >
> > rdepends-add is easy to implement [...] Deletion is trickier [...]
> >
> > The point is to *not* clean up these entries for months/years.
> 
> So, essentially, you want the developer to do part of CVS/git's job,
> namely keeping a history of changes in a compressed format,
> keeping the history forever (or almost forever).
> As mentioned in another post, you highly understimate the
> amount of data which would have to be treated this way:
> For every python release and many other eclass changes,
> almost all packages in the tree are involved, usually
> several times a months.

Python is irrelevant here. Our dependencies are USE-conditional, so
dependencies are added and removed along with USE flags.

If we add new implementation, you need to rebuild the package anyway to
use it, and there is no point populating extra dependencies. Revbump
isn't necessary either since --changed-use will pick it up if necessary.

If we remove an implementation, PM isn't supposed to remove
the dependencies until the package is rebuilt with flag disabled. If it
was enabled, --changed-use is supposed to clean it up. If it was not,
the extra dependencies do not matter (and are not even stored in vdb).

-- 
Best regards,
Michał Górny

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 949 bytes --]

[gentoo-dev] Re: don't rely on dynamic deps

[Martin Vaeth]

Michał Górny <mgorny@gentoo.org> wrote:
>
> Python is irrelevant here. Our dependencies are USE-conditional

You are right, I overlooked this.

Re: [gentoo-dev] don't rely on dynamic deps

[Michał Górny]

[-- Attachment #1: Type: text/plain, Size: 1562 bytes --]

Dnia 2014-07-21, o godz. 21:34:10
Alexandre Rostovtsev <tetromino@gentoo.org> napisał(a):

> On Mon, 2014-07-21 at 22:56 +0200, Michał Górny wrote:
> > Yes, it does. I'm not sure if it leads anywhere, though. Dynamic deps
> > are a pipe dream. You can't implement them properly, so we're using
> > half-working implementation as an excuse to be lazy.
> 
> Why not adapt the updates mechanism for modifying rdepends? Perhaps
> something like
> 
> rdepends-add foo-bar/blah-3.14 "wombat? ( >=dev-libs/wombat-1.0 )"
> 
> This would give the package manager all the benefits of static dep
> resolution while allowing us to dynamically make simple changes (like
> adding a missing dependency to an ebuild) without forcing users to
> rebuild the package.

At the moment updates are stateless. In other words, PM has no idea if
the update has been applied or not, and the update should be defined so
that it can't be applied multiple times.

IOW, if you do a pkgmove via updates, the originating package ebuild
must no longer exist so that the update can be only applied once. If
you do a slotmove, the originating slot must no longer be used
in affected ebuilds.

Now, with dependency updates you lack this guarantee. Package manager
can add the dependency... and then add it again... and again...
and again. It could supposedly try to match dependencies and find out
whether a particular dependency has been added already but that sounds
like something that could easily cause issues in the future.

-- 
Best regards,
Michał Górny

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 949 bytes --]

Re: [gentoo-dev] don't rely on dynamic deps

[Tom Wijsman]

[-- Attachment #1: Type: text/plain, Size: 1212 bytes --]

On Mon, 21 Jul 2014 21:53:04 +0200
"Andreas K. Huettel" <dilfridge@gentoo.org> wrote:

> Actually the quizzes are pretty much clear on that. 
> 
> Revision must be bumped when the on-disk files installed by the
> ebuild are changed. 
> Nothing about dependencies.
> 
> This has been policy for a LONG time, and we're not going to change
> it overnight just because you protest.

As suggested, later PMS features (eg. (+) / (-), ...) and PM features
(eg. binpkgs, ...) can change that; so, a policy adaption or a clearly
described workflow may at some point be necessary to stay actual. 

> Now... whether dynamic deps are technically the right thing to do is
> another question. It merits discussion, but we need to be really sure
> about the consequences of any change.

To deal with that one needs a clear workflow; given that it replaces an
automatic feature or hack without much rebuilds, by something that
needs you to decide whether or not to rev bump and cause more rebuilds.

-- 
With kind regards,

Tom Wijsman (TomWij)
Gentoo Developer

E-mail address  : TomWij@gentoo.org
GPG Public Key  : 6D34E57D
GPG Fingerprint : C165 AF18 AB4C 400B C3D2  ABF0 95B2 1FCD 6D34 E57D

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 473 bytes --]

Re: [gentoo-dev] don't rely on dynamic deps

[Alexander Berntsen]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

Friends,

Michał has documented the shortcomings of dynamic deps in our wiki[0].
(Thank you!) This documentation also includes two of our possible
solutions.

1. Improve dynamic-deps. This is, as Michał pointed out earlier in
this thread a pipe dream.

2. Remove dynamic-deps. This is what I think currently makes sense.

3. This is undocumented in the Wiki, but it is certainly an option: do
nothing. It is of course the worst option; but it is perhaps the most
probably course of action as well...


dynamic-deps do not work, and nobody is stepping up to fix them. PMS
defines a static dependency model, and this is what other package
managers use as far as I know.


Julian,

would you like to share your experiences with Paludis? My guess is
that Paludis is more predictable in this respect. I.e., instead of
breaking stuff, I expect Paludis to simply give up.


Sebastian,

I CC'd you because I would love to hear your opinion on this.


Trofi,

please share your opinion too!



To sum up: My vote is disable dynamic-deps. And I would be happy to
apply a patch that does this with the information I have today.


[0]  <https://wiki.gentoo.org/wiki/Project:Portage/Dynamic_dependencies>
- -- 
Alexander
bernalex@gentoo.org
https://secure.plaimi.net/~alexander
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/

iF4EAREIAAYFAlPNi7oACgkQRtClrXBQc7UP6gEAnnWR7L7hDqvaL64ygDwqaBV/
4upsbo6z2FJK4BehajgA/30wolmft/L9vTR/RzH9Wlsu6+NoUBTBMeJGNuIdBCIl
=++4v
-----END PGP SIGNATURE-----

Re: [gentoo-dev] don't rely on dynamic deps

["Paweł Hajdan, Jr."]

[-- Attachment #1: Type: text/plain, Size: 799 bytes --]

On 7/21/14, 11:52 PM, Alexander Berntsen wrote:
> Michał has documented the shortcomings of dynamic deps in our wiki[0].
> (Thank you!) This documentation also includes two of our possible
> solutions.
>
> [0]  <https://wiki.gentoo.org/wiki/Project:Portage/Dynamic_dependencies>

Thank you, this is very useful. I didn't understand the issue much
before reading that page.

One question: why for "Removal of a USE flag along with the relevant
dependencies" dynamic deps say "revbump + unnecessary rebuild"? What
would happen without the revbump?

> 1. Improve dynamic-deps. This is, as Michał pointed out earlier in
> this thread a pipe dream.

Agreed.

> 2. Remove dynamic-deps. This is what I think currently makes sense.

+1 I also think it's the best option.

Paweł


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 841 bytes --]

Re: [gentoo-dev] don't rely on dynamic deps

[Kent Fredric]

[-- Attachment #1: Type: text/plain, Size: 1686 bytes --]

On 22 July 2014 19:25, "Paweł Hajdan, Jr." <phajdan.jr@gentoo.org> wrote:

> On 7/21/14, 11:52 PM, Alexander Berntsen wrote:
> > Michał has documented the shortcomings of dynamic deps in our wiki[0].
> > (Thank you!) This documentation also includes two of our possible
> > solutions.
> >
> > [0]  <https://wiki.gentoo.org/wiki/Project:Portage/Dynamic_dependencies>
>
> Thank you, this is very useful. I didn't understand the issue much
> before reading that page.
>
> One question: why for "Removal of a USE flag along with the relevant
> dependencies" dynamic deps say "revbump + unnecessary rebuild"? What
> would happen without the revbump?
>
> > 1. Improve dynamic-deps. This is, as Michał pointed out earlier in
> > this thread a pipe dream.
>
> Agreed.
>
> > 2. Remove dynamic-deps. This is what I think currently makes sense.
>
> +1 I also think it's the best option.
>
> Paweł
>
>
Ok, we can side step this discussion partially:

Lets pretend for a moment dynamic deps get banned.

People will still unconsciously make that mistake and things will still
break when they do.

So we'll probably need a repoman check that is smart enough to know "X is
modified" and compare the DEPEND fields with the previous incarnation prior
to commit, and then at very least, warn people doing `repoman full` that
they've modified the dependencies, and that a -r1 bump is thus highly
recommended.

And that check can be added *now* prior to banning/disabling dynamic deps.

And people who want to pay attention to that warning can start doing it
before policy dictates they must.


-- 
Kent

*KENTNL* - https://metacpan.org/author/KENTNL

[-- Attachment #2: Type: text/html, Size: 2860 bytes --]

Re: [gentoo-dev] don't rely on dynamic deps

[Alexander Berntsen]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

On 22/07/14 20:44, Kent Fredric wrote:
> So we'll probably need a repoman check that is smart enough to know
>  "X is modified" and compare the DEPEND fields with the previous 
> incarnation prior to commit, and then at very least, warn people 
> doing `repoman full` that they've modified the dependencies, and 
> that a -r1 bump is thus highly recommended.
> 
> And that check can be added *now* prior to banning/disabling
> dynamic deps.
> 
> And people who want to pay attention to that warning can start
> doing it before policy dictates they must.
This would be a good thing to do. Would you like to try implementing it?
- -- 
Alexander
bernalex@gentoo.org
https://secure.plaimi.net/~alexander
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/

iF4EAREIAAYFAlPOso8ACgkQRtClrXBQc7V7BgEAgElHG5RcxpJWS2eTQ7YVFdDX
NZ55itqGeMngocAoitUA+QF4N0w07NrQSpPecPaF5AeuLOspGI7xjfaU/2BNFLGO
=1H0R
-----END PGP SIGNATURE-----

Re: [gentoo-dev] don't rely on dynamic deps

[Tom Wijsman]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Tue, 22 Jul 2014 20:50:55 +0200
Alexander Berntsen <bernalex@gentoo.org> wrote:

> On 22/07/14 20:44, Kent Fredric wrote:
> > So we'll probably need a repoman check that is smart enough to know
> >  "X is modified" and compare the DEPEND fields with the previous 
> > incarnation prior to commit, and then at very least, warn people 
> > doing `repoman full` that they've modified the dependencies, and 
> > that a -r1 bump is thus highly recommended.
> > 
> > And that check can be added *now* prior to banning/disabling
> > dynamic deps.
> > 
> > And people who want to pay attention to that warning can start
> > doing it before policy dictates they must.
> This would be a good thing to do. Would you like to try implementing
> it?

Just a side note, this would need VCS diff support in Repoman; there
are some other feature requests on b.g.o that are also pending this
VCS diff support, but it takes some understanding of the Repoman code.

This becomes easier to implement after the refactor of Repoman
completes; checks are organized and consolidated into several files
due to the refactor, rather than mixed together in a giant single file.

Having repoman help with the developer workflow sounds like a neat idea.

- -- 
With kind regards,

Tom Wijsman (TomWij)
Gentoo Developer

E-mail address  : TomWij@gentoo.org
GPG Public Key  : 6D34E57D
GPG Fingerprint : C165 AF18 AB4C 400B C3D2  ABF0 95B2 1FCD 6D34 E57D
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2

iQEcBAEBAgAGBQJTzvOZAAoJEPWZc8roOL/QVcoH/0JhGPYQg5yPYDafriSIxpD+
ydS0jwCZudHiX7YZk/L5XiVLKTiwEPNLUzEmmYtkCnPXtJAZTFUYlTmn9sY9GUEh
dV9Gr6gG5LpwjGDF2E9F5ivkSdUqGbx8ztqKibSAm4POy+uLm7EDEBtRJe095VVo
k/kKukGSpa9hlnLB43hP9u1rs0vuwx0YB6FwK+dRVxGJAyPipgxg1jt6uRqOO9+a
sBOzc910js8rpO1bkL4vGbrRViVUoFFOylWrXDxEuuyoaAWROZbItqe4xK2XagI3
wYJa/aLzGO9b3oTDQPVSEdIpgqfuJCI39BH4UrlAUZgT4GIuB8VroOx9MAOtHZc=
=jVQd
-----END PGP SIGNATURE-----

[gentoo-dev] Re: don't rely on dynamic deps

[Duncan]

Kent Fredric posted on Wed, 23 Jul 2014 06:44:57 +1200 as excerpted:

> Ok, we can side step this discussion partially:
> 
> Lets pretend for a moment dynamic deps get banned.
> 
> People will still unconsciously make that mistake and things will still
> break when they do.
> 
> So we'll probably need a repoman check that is smart enough to know "X
> is modified" and compare the DEPEND fields with the previous incarnation
> prior to commit, and then at very least, warn people doing `repoman
> full` that they've modified the dependencies, and that a -r1 bump is
> thus highly recommended.
> 
> And that check can be added *now* prior to banning/disabling dynamic
> deps.
> 
> And people who want to pay attention to that warning can start doing it
> before policy dictates they must.

Be aware that with eclasses generated deps taken into account, this 
/could/ trigger a LOT of arguably unnecessary revision bumps.

Right now I'm trying emerge --dynamic-deps n --update --deep @world, 
and...

Due to dynamic-deps and the (default) WANT_AUTOMAKE=LATEST (or whatever 
it is) stuff in the eclasses, I only have automake slots 1.11 and 1.14 
installed here, but I still have well over 100 packages originally built 
with automake:1.13, that if I turn off dynamic-deps either need it pulled 
back in, or need to be rebuilt to pull in the automake:1.14 dep.

I guess that's a variant of category 2 in the wiki list.  It's a minor 
build-only dep change that doesn't need to be propagated immediately as 
the package is already installed, with dynamic-deps applying it 
immediately, but static-deps wouldn't apply it until a rebuild.  As long 
as the older automake slot remains around to fill the dep, not a problem.

Of course for users the fix is simple and what portage recommends, simply 
pull automake:1.13 back in (or don't let it be unmerged in the first 
place) and don't worry about it.

I decided to go the rebuild route instead, here, and once I decided the 
number of rebuilds wasn't going to be easy to do with --tree giving me 
one at a time, I concocted a pipe involving bzgrep environment.bz2 to 
list them all so I could rebuild them in parallel, then did a wc -l on 
the output to get a count.

But a repoman dynamic-deps checker might have flagged all those 125-ish 
package for revision-bump when automake:1.14 was introduced and thus 
became a dependency candidate.

And I'm on ~arch, doing --deep --newuse @world updates routinely, with a 
not insignificant portion of my packages being live-ebuilds, so many of 
the package originally built with automake:1.13 have certainly been 
rebuilt by now, here.  But I see a distinct possibility of an automake 
slot bump triggering thousands of rev-bumps, often to all available 
versions of a package at once, tree-wide.


Meanwhile, one personal benefit to this discussion is that at least I 
understand now why merging a binpkg the other day pulled in some automake 
slot that depclean immediately wanted to remove.  Binpkgs are static-deps 
and that one must have been built with that automake slot, but of course 
depclean was using dynamic-deps, so...  Now I have a proper explanation 
for the behavior. =:^)

-- 
Duncan - List replies preferred.   No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master."  Richard Stallman

[gentoo-dev] Re: don't rely on dynamic deps

[Martin Vaeth]

Kent Fredric <kentfredric@gmail.com> wrote:
>
> So we'll probably need a repoman check that is smart enough to know "X is
> modified" and compare the DEPEND fields with the previous incarnation prior
> to commit, and then at very least, warn people doing `repoman full` that
> they've modified the dependencies, and that a -r1 bump is thus highly
> recommended.

Do not forget modification of eclasses which then require mass bumps!

Re: [gentoo-dev] Re: don't rely on dynamic deps

[Kent Fredric]

[-- Attachment #1: Type: text/plain, Size: 469 bytes --]

On 27 July 2014 02:12, Martin Vaeth <martin@mvath.de> wrote:

>
> Do not forget modification of eclasses which then require mass bumps!


I'm curious what the -r1.1 technique would do here.

I mean, wouldn't that mean you have 2 ebuilds that are identical except for
the '.1' simply due to the eclass change?

That's going to be confusing.


-r1.1 weirdness feels like it may cause more problems than it solves.

-- 
Kent

*KENTNL* - https://metacpan.org/author/KENTNL

[-- Attachment #2: Type: text/html, Size: 1189 bytes --]

[gentoo-dev] Re: don't rely on dynamic deps

[Martin Vaeth]

Kent Fredric <kentfredric@gmail.com> wrote:
> On 27 July 2014 02:12, Martin Vaeth <martin@mvath.de> wrote:
>>
>> Do not forget modification of eclasses which then require mass bumps!
>
> I'm curious what the -r1.1 technique would do here.
>
> I mean, wouldn't that mean you have 2 ebuilds that are identical except for
> the '.1' simply due to the eclass change?
>
> That's going to be confusing.

Not at all, it is completely identical to a revision bump:
If you would use -r2 instead of -r1.1, you also would end up
in -r1 and -r2 being identical.
Actually, in both cases, you would *remove* -r1, since -r1 is incorrect
since it should have been bumped.

> -r1.1 weirdness feels like it may cause more problems than it solves.

So far, nobody pointed out any problem which would be caused by -r1.1.
Which is not surprising, since the idea is that -r1.1 cannot be
distinguished from -r2:
It is only a hint to the PM that he *may* shortcut certain phases when
updating from -r1.

[gentoo-dev] Avoiding rebuilds (was: don't rely on dynamic deps)

[Ulrich Mueller]

[-- Attachment #1: Type: text/plain, Size: 952 bytes --]

>>>>> On Sun, 27 Jul 2014, Martin Vaeth wrote:

> Kent Fredric <kentfredric@gmail.com> wrote:
>> -r1.1 weirdness feels like it may cause more problems than it solves.

> So far, nobody pointed out any problem which would be caused by -r1.1.
> Which is not surprising, since the idea is that -r1.1 cannot be
> distinguished from -r2:
> It is only a hint to the PM that he *may* shortcut certain phases when
> updating from -r1.

I wonder if it wouldn't be saner to leave our revision syntax
untouched.

Instead, one could introduce a variable INSTALL_VERSION that would
default to ${PVR} but could be set to the version of a previous ebuild
instead. The PM could compare it against INSTALL_VERSION in the VDB
and skip build and installation if versions match.

Advantages:
- Support for the variable could be optional. PMs not supporting it
  would do a regular revbump instead.
- One could even imagine USE-conditional syntax for the variable.

Ulrich

[-- Attachment #2: Type: application/pgp-signature, Size: 490 bytes --]

Re: [gentoo-dev] Avoiding rebuilds (was: don't rely on dynamic deps)

[Rich Freeman]

On Sun, Jul 27, 2014 at 6:16 AM, Ulrich Mueller <ulm@gentoo.org> wrote:
> I wonder if it wouldn't be saner to leave our revision syntax
> untouched.
>
> Instead, one could introduce a variable INSTALL_VERSION that would
> default to ${PVR} but could be set to the version of a previous ebuild
> instead. The PM could compare it against INSTALL_VERSION in the VDB
> and skip build and installation if versions match.
>
> Advantages:
> - Support for the variable could be optional. PMs not supporting it
>   would do a regular revbump instead.
> - One could even imagine USE-conditional syntax for the variable.

This isn't all that different from the concept I put out around having
the PM cache hashes of last-seen ebuilds, except that the hash is
editable so that developers can mess with it this way, allowing
revbumps to avoid rebuilds.

If an eclass/virtual/etc change impacted 200 packages, would you then
revbump all of them but setting INSTALL_VERSION on all the bumps?
That could involve a lot of ebuild munging.  Also, if you add
hard-coded INSTALL_VERSIONs to ebuilds that lacked them previously,
maintainers would have to notice that and remove those the next time
they did a bump or even a non-revbump might not trigger a rebuild
since it is still pointing to the old minor revision.

I take it that we would also have the package manager not do dynamic
deps if the revision doesn't change if we used the INSTALL_VERSION
approach?

This scenario still beg's Kent's question, "why do we have to point
out to portage that the deps need reparsing?"  If we're going to have
dynamic deps, then shouldn't reparsing deps be the default when they
change, and rebuilds should be the fallback when a big change happens
that dynamic deps can't handle?  If so, then we already have a
mechanism for that - you change the revision when you want a rebuild,
and then this just comes down to devs revbumping when they need to,
and having the PM able to detect other changes and properly handle
them.

And yes, I know the argument is that some changes can never be
properly handled, but my point is that we only use dyanmic deps for
changes which can be handled properly, so we're just arguing over
whether that is the empty set or not.

Rich

Re: [gentoo-dev] Avoiding rebuilds

[hasufell]

Ulrich Mueller:
>>>>>> On Sun, 27 Jul 2014, Martin Vaeth wrote:
> 
>> Kent Fredric <kentfredric@gmail.com> wrote:
>>> -r1.1 weirdness feels like it may cause more problems than it solves.
> 
>> So far, nobody pointed out any problem which would be caused by -r1.1.
>> Which is not surprising, since the idea is that -r1.1 cannot be
>> distinguished from -r2:
>> It is only a hint to the PM that he *may* shortcut certain phases when
>> updating from -r1.
> 
> I wonder if it wouldn't be saner to leave our revision syntax
> untouched.
> 
> Instead, one could introduce a variable INSTALL_VERSION that would
> default to ${PVR} but could be set to the version of a previous ebuild
> instead. The PM could compare it against INSTALL_VERSION in the VDB
> and skip build and installation if versions match.
> 
> Advantages:
> - Support for the variable could be optional. PMs not supporting it
>   would do a regular revbump instead.
> - One could even imagine USE-conditional syntax for the variable.
> 

Aren't we putting a lot of trust in ebuild writers here? If any, I'd say
this should definitely be an optional feature for portage as well and
default to off.

The only time I would really consider using this would be for the very
few time-intensive packages I maintain like paraview, rstudio or
blender. For the rest, it's too much effort.

It will probably also cause confusion for comaintainers and
collaborators, especially when INSTALL_VERSION points to a version that
has already been removed.

[gentoo-dev] Re: Avoiding rebuilds

[Martin Vaeth]

hasufell <hasufell@gentoo.org> wrote:
> Ulrich Mueller:
>>
>> I wonder if it wouldn't be saner to leave our revision syntax
>> untouched.

As already mentioned, -r1.1 is only one of several possible ways
how to achieve the same aim; I am not speaking in favour for a
particular method.
The -r1.1 method has the advantage of being simple and transparent
to the user and developer.  Other approaches have other advantages:

>> Instead, one could introduce a variable INSTALL_VERSION that would

(It would have to be a variable stored in the metadata/ cache
and thus also would only work with a new API, but these are only
technical details.)

>> default to ${PVR} but could be set to the version of a previous ebuild
>> instead. The PM could compare it against INSTALL_VERSION in the VDB
>> and skip build and installation if versions match.

It should be a list and have empty default (*never* including the
version itself), but these are also technical details.
This solution would have the advantage that you could specify
*full* versions and thus have even more fine-grained control when
recompilations are necessary. One could also allow specify version
ranges, slots, overlays, etc., perhaps even make the behaviour
dependent of USE-flags, as you already mentioned, all
similarl to current DEPEND syntax.

The disadvantage is that it is slightly more work than -r1.1,
less transparent, and easily overlooked to remove for a version bump,
causing issues like these:

> It will probably also cause confusion for comaintainers and
> collaborators, especially when INSTALL_VERSION points to a version that
> has already been removed.

Re: [gentoo-dev] Re: Avoiding rebuilds

[Brian Dolbec]

On Mon, 28 Jul 2014 05:49:07 +0000 (UTC)
Martin Vaeth <martin@mvath.de> wrote:

> hasufell <hasufell@gentoo.org> wrote:
> > Ulrich Mueller:
> >>
> >> I wonder if it wouldn't be saner to leave our revision syntax
> >> untouched.
> 
> As already mentioned, -r1.1 is only one of several possible ways
> how to achieve the same aim; I am not speaking in favour for a
> particular method.
> The -r1.1 method has the advantage of being simple and transparent
> to the user and developer.  Other approaches have other advantages:
> 
> >> Instead, one could introduce a variable INSTALL_VERSION that would
> 
> (It would have to be a variable stored in the metadata/ cache
> and thus also would only work with a new API, but these are only
> technical details.)
> 
> >> default to ${PVR} but could be set to the version of a previous
> >> ebuild instead. The PM could compare it against INSTALL_VERSION in
> >> the VDB and skip build and installation if versions match.
> 
> It should be a list and have empty default (*never* including the
> version itself), but these are also technical details.
> This solution would have the advantage that you could specify
> *full* versions and thus have even more fine-grained control when
> recompilations are necessary. One could also allow specify version
> ranges, slots, overlays, etc., perhaps even make the behaviour
> dependent of USE-flags, as you already mentioned, all
> similarl to current DEPEND syntax.
> 
> The disadvantage is that it is slightly more work than -r1.1,
> less transparent, and easily overlooked to remove for a version bump,
> causing issues like these:
> 
> > It will probably also cause confusion for comaintainers and
> > collaborators, especially when INSTALL_VERSION points to a version
> > that has already been removed.
> 
> 

I haven't had the energy to read all the mails over all the dynamic
deps thread...

the -r1.1 syntax has been in use by the prefix since early in it's
existence.  I haven't kept track, but they may have finally done away
with it.

There are many other problems with using that syntax, namely most other
tools are not compatible with it, so more than just portage needs to be
modified.  Adding that syntax to ebuilds will cause disruptions and bugs
for years to come.

So, please, forget about this syntax as a viable solution.

-- 
Brian Dolbec <dolsen>

[gentoo-dev] Re: Avoiding rebuilds

[Steven J. Long]

On Mon, Jul 28, 2014 at 05:49:07AM +0000, Martin Vaeth wrote:
> hasufell <hasufell@gentoo.org> wrote:
> > Ulrich Mueller:
> >>
> >> I wonder if it wouldn't be saner to leave our revision syntax
> >> untouched.
> 
> As already mentioned, -r1.1 is only one of several possible ways
> how to achieve the same aim; I am not speaking in favour for a
> particular method.

Sure. As dolsen prefix is using it, even if this weren't better done
as metadata.

> The -r1.1 method has the advantage of being simple and transparent
> to the user and developer.  Other approaches have other advantages:
> 
> >> Instead, one could introduce a variable INSTALL_VERSION that would
> 
> (It would have to be a variable stored in the metadata/ cache
> and thus also would only work with a new API, but these are only
> technical details.)

Agreed again, there's far too much conflabation of EAPI vs impl round here.
Not helped by the obfuscatory troll you've had the mispleasure to encounter.
Think of it as an initiation.. ;-)

> >> default to ${PVR} but could be set to the version of a previous ebuild
> >> instead. The PM could compare it against INSTALL_VERSION in the VDB
> >> and skip build and installation if versions match.
> 
> It should be a list and have empty default (*never* including the
> version itself), but these are also technical details.
> This solution would have the advantage that you could specify
> *full* versions and thus have even more fine-grained control when
> recompilations are necessary. One could also allow specify version
> ranges, slots, overlays, etc., perhaps even make the behaviour
> dependent of USE-flags, as you already mentioned, all
> similarl to current DEPEND syntax.
> 
> The disadvantage is that it is slightly more work than -r1.1,
> less transparent, and easily overlooked to remove for a version bump,
> causing issues like these:
> 
> > It will probably also cause confusion for comaintainers and
> > collaborators, especially when INSTALL_VERSION points to a version that
> > has already been removed.

So use another name that can't be confused. Perhaps REPLACES_VERSION, or
w/e the primadonna will allow, since we're still feeding him goats..
Perhaps doubt he'll want to pluralise it, in that tedious nu-skool way
of his. More likely he'll just use anything we discuss as an excuse
for more FUD.

Regards,
steveL

PS: Now you know just why..
-- 
#friendly-coders -- We're friendly, but we're not /that/ friendly ;-)

[gentoo-dev] Re: Avoiding rebuilds

[Martin Vaeth]

Steven J. Long <slong@rathaus.eclipse.co.uk> wrote:
>>
>> > It will probably also cause confusion for comaintainers and
>> > collaborators, especially when INSTALL_VERSION points to a version
>> > that has already been removed.
>
> So use another name that can't be confused.

Perhaps there is a misunderstanding: I did not understand that the
confusion is caused by the name, but by the lack of information
about its entries:

For instance, if you bump a version, you must never forget to
check whether this variable needs to be updated.
Moreover, if you want to update that variable, you should
understand precisely why which version is listed here
in order to decide whether a recompile from that version
might be needed with the current bump or not.
This decision is not necessarily easy if the corresponding
referred ebuilds are already in the CVS attic.

Of course, if in doubt, it is a safe strategy to always
remove that variable (it can only cause redundant compilation,
while it can be fatal if you leave a version falsely).

Therefore, an automatism to "forget" this variable automatically
if not changed would be preferrable, but one would need a mechanism
for this (I have only some strange ideas for such a mechanisms:
One could encode the current ebuild version into the name of
that variable; or one could require that the current version
is the first entry in this variable [although, semanatically,
it is ignored and just serves as a "proof" that the ebuild
maintainer checked that variable]).

[gentoo-dev] Re: Avoiding rebuilds

[Steven J. Long]

Martin Vaeth wrote:
> Steven J. Long wrote:
Please set your client not to include email addresses (for publically
web-archived newsgroups.)

> >> > It will probably also cause confusion for comaintainers and
> >> > collaborators, especially when INSTALL_VERSION points to a version
> >> > that has already been removed.
> >
> > So use another name that can't be confused.
> 
> Perhaps there is a misunderstanding: I did not understand that the
> confusion is caused by the name, but by the lack of information
> about its entries:

Yeah, perhaps that's a language thing then. "install version" in English
implies you're installing it currently, and the removal conflicts in
semantic terms. It just doesn't feel right.

> For instance, if you bump a version, you must never forget to
> check whether this variable needs to be updated.
> Moreover, if you want to update that variable, you should
> understand precisely why which version is listed here
> in order to decide whether a recompile from that version
> might be needed with the current bump or not.
> This decision is not necessarily easy if the corresponding
> referred ebuilds are already in the CVS attic.

My instant thought there is that this is a maintainer decision. I agree
the consequences of getting it wrong aren't good. What about giving it
a working-name of CHANGE_VDB?

Regardless of how it's implemented the PM has to have an installed pkg
db; for instance istr portage can share a sqlite db with eix.
Irrespective of the impl or its configuration, the concept of a pkg db
is hardly contentious; it's central to the domain, and implicit in
REPLACING_VERSIONS and REPLACED_BY_VERSION. Based on the long prior
thread, I'd say there's consensus for the need in very specific
circumstances to change vdb entries, ideally at the granularity of the
ebuild; and it's better if this isn't part of the normal dep-calc.

Calling it that directly is simpler, and it stands out as something
both unusual and changing the vdb, which any Gentoo admin is familiar
with. The imperative form is in line with what is going to happen:
we're telling the mangler to change the vdb, for matching atoms, if
it installs this package. It could end up CHANGE_PKG_DB instead;
sticking an E on the front, or making it into some obfuscated C++
style name, that can be bikeshedded after it's actually specified.

However as you've seen, it's a lot harder to have a focussed discussion
on the dev ML than it is on the forums. Having waded through that thread
on the web[1] I have no wish ever to do it again, nor would I inflict it
on someone trying to catch up with the thinking behind changes in the
future. Indeed it would be quite embarrassing in the context of
attracting new people, as has happened before.

At this stage though, I cannot say that I have any sort of overall grasp
on the various constraints you've outlined, based on the requirements
you and others have specified. Could I ask therefore that you collect
your thoughts into a forum post, where we can collaborate without the
flak-storm of agenda-driven political FUD poisoning the discussion?
It's much easier since the OP is always at the top of the thread, and
we can push the content block to a wiki page once we're done, and go
for a GLEP from there, after wider discussion.

While I could go back over the thread and try to pull out your thoughts,
it would take me a long time, be painfully tedious (ie I ain't going to,
come what may;) and not consistent, nor as comprehensive as you simply
grabbing it from your mailbox, and tidying up what you're thinking.

If you want some help making it more fluent English, feel free to email
me off-list with a first draft, assuming this is okay with you.

> Of course, if in doubt, it is a safe strategy to always
> remove that variable (it can only cause redundant compilation,
> while it can be fatal if you leave a version falsely).
> 
> Therefore, an automatism to "forget" this variable automatically
> if not changed would be preferrable, but one would need a mechanism
> for this (I have only some strange ideas for such a mechanisms:
> One could encode the current ebuild version into the name of
> that variable; or one could require that the current version
> is the first entry in this variable [although, semanatically,
> it is ignored and just serves as a "proof" that the ebuild
> maintainer checked that variable]).

Sounds like something that repoman could check, once a GLEP and impl
are in place. By then it should be much easier to add, and maintain,
a specific check as the repoman code is currently being modularised.

Anyhow, good to have a man of your experience and approach contributing
to the dev discussion at last. Plenty of devs use eix as you may have
seen in various posts; I can tell you from IRC that knowing its
switches is almost seen as black-magic sometimes ;p

I don't ofc: I just tell them to post on the forums and get the info
from you, if they can't work out the manpages, which as you know is
exactly what I do quite frequently.. so thanks for the support as
well as the excellent util.

Regards,
steveL.

[1] http://marc.info/?t=140597147200005&r=9&w=2
-- 
#friendly-coders -- We're friendly, but we're not /that/ friendly ;-)

[gentoo-dev] Re: Avoiding rebuilds

[Martin Vaeth]

Steven J. Long <slong@rathaus.eclipse.co.uk> wrote:
>
> collect your thoughts into a forum post

You are right: Not everybody on this list is interested in all
technical details, so it is perhaps better to shift this discussion
to the forums. I have opened the topic

https://forums.gentoo.org/viewtopic-p-7593700.html#7593700

with a summary of the various suggestions (and a very rough sketch
of their advantages/disadvantages).

> Sounds like something that repoman could check, once a GLEP and impl
> are in place.

The problem I meant is that repoman is no able to check it:
It can be completely reasonable that the value of that metadata
variable is unchanged, that is, repoman should not even spit a
warning in this case.
Unless one "adds" an artificial mechanism (like encoding the revision
into the variable name or variable content) which *forces* the
maintainer to change/check that variable.
But any such mechanism I can think of is inconvenient and
possibly confusing.

Re: [gentoo-dev] Re: don't rely on dynamic deps

[Kent Fredric]

[-- Attachment #1: Type: text/plain, Size: 2253 bytes --]

On 27 July 2014 19:16, Martin Vaeth <martin@mvath.de> wrote:

> Not at all, it is completely identical to a revision bump:
> If you would use -r2 instead of -r1.1, you also would end up
> in -r1 and -r2 being identical.
> Actually, in both cases, you would *remove* -r1, since -r1 is incorrect
> since it should have been bumped.
>
>
It just seems strange to me to go round a large tree and bump every ebuild
affected by an eclass change, simply not modifiying the code, buy changing
the -r value on the end of it.

In a "no dynamic deps, period" scenario, this just strikes me as 2 flavours
of the same weirdness, -r2 and -r1.1 are just equally weird choices to make
if the ebuild itself doesn't change at all.

For some things it would be a nightmare of impossibility for even a trivial
change if some eclass changed to have one new dependency/one less
dependency. You'd need some system in place to go around the tree -r
bumping things, and the system would involve humans who are prone to making
mistakes, and create commit churn to make it happen.

The same problem would still persist with people forgetting to -r bump, or
missing one ebuild in the ~200 affected, but with dynamic deps off, those
bugs would lie in wait and confuse portage until somebody filed a bug and
it got responded to.

Sure, having an approved system to ship dependency-only changes to the end
users tree is something we do need, I'll grant that, but the point of
failure is already significantly weighing on developer time, and this would
see a growth in developer time to manage this ( I could be over estimating
how much, but -r bumping everything that used a specific eclass is
something I very much do not envy ).

However, if there was a system in place such that developers didn't have to
manually do any -r bumping , but "some system" acknowledge the changes and
scheduled some kind of VDB update in response to some kind of data that was
transmitted as part of the sync, that would be nice.

-r bumping is of course *a* way to transmit the data.

Just I feel strongly inclined that we shouldn't be making developers expend
*more* effort to solve this problem if there's a way we can make them spend
*less*.

-- 
Kent

*KENTNL* - https://metacpan.org/author/KENTNL

[-- Attachment #2: Type: text/html, Size: 3167 bytes --]

Re: [gentoo-dev] Re: don't rely on dynamic deps

[Rich Freeman]

On Sun, Jul 27, 2014 at 6:43 AM, Kent Fredric <kentfredric@gmail.com> wrote:
>
> In a "no dynamic deps, period" scenario, this just strikes me as 2 flavours
> of the same weirdness, -r2 and -r1.1 are just equally weird choices to make
> if the ebuild itself doesn't change at all.

You have a good point here.

With this proposal we have three kinds of ebuild changes:
1.  Those that do not involve any revbump.
2.  Those with a "minor" revbump (ie -r1 -> -r1.1).
3.  Those with a normal revbump (ie -r1 -> -r2).

What is the purpose of allowing the first?  Presumably that should be
used in even more limited circumstances than a minor revbump, so why
allow it at all?  Why not just make an in-place change equivalent to a
minor revbump and ditch the minor revbump entirely?  Devs would still
need to distinguish between what requires a revbump and what does not.

Doing this would require having portage cache a hash of whatever
ebuild it last parsed, and perhaps its eclasses as well if we permit
revbump-less eclass changes.  Then it would have to check for when
these change, perhaps this might be stored in the metadata to speed
things up.

You could view such an approach as being equivalent to just sticking a
".hash" on every ebuild in the tree as a minor revision tag, so that
any change triggers a minor revision.  The only difference between
that and the proposal that I can see is that the minor revisions would
be unordered.  However, if we go with the theory that defective
ebuilds get removed immediately then there is no point in ordering
minor revisions because there will only be one in the tree at any time
for a given major revision, so the package manager couldn't take
advantage of any information conveyed by ordering.

This probably isn't too different from what portage is doing already, except:
1.  Portage is only looking for dependency changes when it has another
reason to look closely at the ebuild - it has no mechanism to detect
that an ebuild changes, and this would add one.
2.  We don't have any clear policies today on when ebuilds should be
revbumped or not as the result of things like dependency changes, and
this would cause us to make some.

The fact that this isn't a big change does concern me that it may not
fix all of the underlying problems.  However, some of those problems
might not actually have clean solutions other than never committing
mistakes in the first place.  For example, if a prerm dependency was
missing then removing the ebuild can potentially fail whether you use
dynamic deps or not until it is satisfied.

Rich

Re: [gentoo-dev] Re: don't rely on dynamic deps

[Rich Freeman]

On Sun, Jul 27, 2014 at 8:04 AM, Rich Freeman <rich0@gentoo.org> wrote:
>
> Doing this would require having portage cache a hash of whatever
> ebuild it last parsed, and perhaps its eclasses as well if we permit
> revbump-less eclass changes.  Then it would have to check for when
> these change, perhaps this might be stored in the metadata to speed
> things up.

I was thinking about this a little more, and the way I'd do it is ID
whatever ebuild variables we want to allow to be dynamic.  Then the
ebuild would be sourced, and then those variables would be extracted
and hashed, and that would be treated as the ebuilds dependency state.
That would be stored in the metadata, and portage would store it in
vdb when a package is installed.

Then portage could look for any change in state and that would trigger
a build-less re-merge, which would update vdb with the new state
(including the new hash).

With this approach you don't need minor revision numbers - any change
to an ebuild would be considered a minor revision, and when that is
inappropriate a full revbump should be used instead.

Rich

Re: [gentoo-dev] Re: don't rely on dynamic deps

[Kent Fredric]

[-- Attachment #1: Type: text/plain, Size: 720 bytes --]

On 28 July 2014 09:46, Rich Freeman <rich0@gentoo.org> wrote:

> Then portage could look for any change in state and that would trigger
> a build-less re-merge, which would update vdb with the new state
> (including the new hash).
>

If we're scared about this being worse than what we have, I notice there's
a bunch of --rebuild-if-whaetever variables.

For some reason I was under the impression there was already a
--rebuild-if-deps-changed flag, but I seem to have been wrong about that.

--rebuild-if-deps-changed=fast # VDB only updates
--rebuild-if-deps-changed=full # complete reinstall if vdb != tree
--rebuild-if-deps-changed=n    # current behaviour.


-- 
Kent

*KENTNL* - https://metacpan.org/author/KENTNL

[-- Attachment #2: Type: text/html, Size: 1477 bytes --]

Re: [gentoo-dev] Re: don't rely on dynamic deps

[Ian Stakenvicius]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

On 27/07/14 08:04 AM, Rich Freeman wrote:
> On Sun, Jul 27, 2014 at 6:43 AM, Kent Fredric
> <kentfredric@gmail.com> wrote:
>> 
>> In a "no dynamic deps, period" scenario, this just strikes me as
>> 2 flavours of the same weirdness, -r2 and -r1.1 are just equally
>> weird choices to make if the ebuild itself doesn't change at
>> all.
> 
> You have a good point here.
> 
> With this proposal we have three kinds of ebuild changes: 1.  Those
> that do not involve any revbump. 2.  Those with a "minor" revbump
> (ie -r1 -> -r1.1). 3.  Those with a normal revbump (ie -r1 ->
> -r2).
> 
> What is the purpose of allowing the first?  Presumably that should
> be used in even more limited circumstances than a minor revbump, so
> why allow it at all?  Why not just make an in-place change
> equivalent to a minor revbump and ditch the minor revbump entirely?
> Devs would still need to distinguish between what requires a
> revbump and what does not.
> 
> Doing this would require having portage cache a hash of whatever 
> ebuild it last parsed, and perhaps its eclasses as well if we
> permit revbump-less eclass changes.  Then it would have to check
> for when these change, perhaps this might be stored in the metadata
> to speed things up.
> 
> You could view such an approach as being equivalent to just
> sticking a ".hash" on every ebuild in the tree as a minor revision
> tag, so that any change triggers a minor revision.  The only
> difference between that and the proposal that I can see is that the
> minor revisions would be unordered.  However, if we go with the
> theory that defective ebuilds get removed immediately then there is
> no point in ordering minor revisions because there will only be one
> in the tree at any time for a given major revision, so the package
> manager couldn't take advantage of any information conveyed by
> ordering.
> 
> This probably isn't too different from what portage is doing
> already, except: 1.  Portage is only looking for dependency changes
> when it has another reason to look closely at the ebuild - it has
> no mechanism to detect that an ebuild changes, and this would add
> one. 2.  We don't have any clear policies today on when ebuilds
> should be revbumped or not as the result of things like dependency
> changes, and this would cause us to make some.
> 
> The fact that this isn't a big change does concern me that it may
> not fix all of the underlying problems.  However, some of those
> problems might not actually have clean solutions other than never
> committing mistakes in the first place.  For example, if a prerm
> dependency was missing then removing the ebuild can potentially
> fail whether you use dynamic deps or not until it is satisfied.
> 

The primary underlying problem I see about this is that it doesn't
force devs to start doing something to the tree that will suddenly
help make all of the static-deps-only PMs (ie, those that aren't going
to implement this new hash-changed-so-re-evaluate-ebuild method)
suddenly work in a more consistent fashion.  IIRC, the very first post
of this thread was a reminder to dev's to revbump so that static-deps
behaviour is more correct/consistent.


However, if we put something into the next EAPI about this and make it
a requirement for all PMs (although I have no idea how we would roll
this out; maybe make it a profile-level requirement instead of an
ebuild-level one, if there is such a thing??) then it would become
much less of an issue..  Mind you, we need to solve all of the
problems first and make sure PMS documents all of the requirements in
an appropriately complete and ambiguity-preventing manner that
everyone agrees with..  Easy, right? :)





-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2

iF4EAREIAAYFAlPWlfYACgkQ2ugaI38ACPApTwD9H+PX4f1XGtauwbjfXczPqAYf
yBqDW9MOwIlWPCqeu6IA/ipySyWxA/J12RSuLTNyj98li9Qmeq0GLR37KSZ2Cc/p
=05lZ
-----END PGP SIGNATURE-----

Re: [gentoo-dev] Re: don't rely on dynamic deps

[Rich Freeman]

On Mon, Jul 28, 2014 at 2:27 PM, Ian Stakenvicius <axs@gentoo.org> wrote:
>
> The primary underlying problem I see about this is that it doesn't
> force devs to start doing something to the tree that will suddenly
> help make all of the static-deps-only PMs (ie, those that aren't going
> to implement this new hash-changed-so-re-evaluate-ebuild method)
> suddenly work in a more consistent fashion.  IIRC, the very first post
> of this thread was a reminder to dev's to revbump so that static-deps
> behaviour is more correct/consistent.

I think the intent here is to define how we want the PM to behave, and
what kinds of changes should require revbumps (ie those the PM can't
handle otherwise).

Obvious a side-effect of this will be that PMs that don't behave as we
intend them to may have issues.

>
> However, if we put something into the next EAPI about this and make it
> a requirement for all PMs (although I have no idea how we would roll
> this out; maybe make it a profile-level requirement instead of an
> ebuild-level one, if there is such a thing??)

It may make sense to do this via a new EAPI, though I think figuring
out what we want to do comes first.  That is, I want to ask the
question "if no PMs existed and we were writing our first one, how
would we want it to behave?"  Getting from here to there is the next
problem.

Really the issue comes down to how we maintain ebuilds.  If we aren't
revbumping for dependency errors, then PMs that don't handle dynamic
deps wouldn't update their dependencies.  That certainly has
consequences, but whether they're considered "bugs/problems/etc" is a
bit up for debate.

I'm not convinced that it makes sense to do "micro-revbumps" just to
force PMs that don't have any concept of dynamic dependencies to treat
them as full revbumps.  Devs can still forget to do them, and it
results in churn that doesn't seem necessary to me.  On the other hand
I don't want to make life even more difficult on those using
alternative PMs (though it sounds like we're doing this already).

It seems like we aren't getting many more new options here.

Rich

Re: [gentoo-dev] don't rely on dynamic deps

[Samuli Suominen]

On 22/07/14 10:25, "Paweł Hajdan, Jr." wrote:
> On 7/21/14, 11:52 PM, Alexander Berntsen wrote:
>> 2. Remove dynamic-deps. This is what I think currently makes sense.
> +1 I also think it's the best option.
>
>

Not before someone has implemented an alternative way to avoid useless
rebuilding.
The quality of the distribution doesn't improve by killing one of the
most important
features the package manager has.
The quality of the distribution improves by providing an alternative
with less problems.

Sounds like to me, that the people who want to remove the feature so
badly, are the
ones volunteering for the job as well.

- Samuli

Re: [gentoo-dev] don't rely on dynamic deps

[Tom Wijsman]

[-- Attachment #1: Type: text/plain, Size: 893 bytes --]

On Tue, 22 Jul 2014 22:05:54 +0300
Samuli Suominen <ssuominen@gentoo.org> wrote:

> Not before someone has implemented an alternative way to avoid useless
> rebuilding.
> The quality of the distribution doesn't improve by killing one of the
> most important
> features the package manager has.
> The quality of the distribution improves by providing an alternative
> with less problems.

Those are apples with eggs; what you claim to be most important in
itself also has problems as stated in this thread, it doesn't stop there
as even beyond that it is a hack that deals with triggered problems. So,
I don't think it is right to discuss replacing problems with problems.

-- 
With kind regards,

Tom Wijsman (TomWij)
Gentoo Developer

E-mail address  : TomWij@gentoo.org
GPG Public Key  : 6D34E57D
GPG Fingerprint : C165 AF18 AB4C 400B C3D2  ABF0 95B2 1FCD 6D34 E57D

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 473 bytes --]

Re: [gentoo-dev] don't rely on dynamic deps

[hasufell]

Samuli Suominen:
> 
> On 22/07/14 10:25, "Paweł Hajdan, Jr." wrote:
>> On 7/21/14, 11:52 PM, Alexander Berntsen wrote:
>>> 2. Remove dynamic-deps. This is what I think currently makes sense.
>> +1 I also think it's the best option.
>>
>>
> 
> Not before someone has implemented an alternative way to avoid useless
> rebuilding.
> The quality of the distribution doesn't improve by killing one of the
> most important
> features the package manager has.
> The quality of the distribution improves by providing an alternative
> with less problems.
> 
> Sounds like to me, that the people who want to remove the feature so
> badly, are the
> ones volunteering for the job as well.
> 


There seems to be a misunderstanding.

The feature is already _optional_ and not even active in all
circumstances (did you read the wiki entry?). If your ebuilds
assume that random portage features are enabled, then that's pretty much
undefined behavior.

We can debate whether there are dependency changes not worth a revbump.
We can debate how to reinstate dynamic deps support or how to update the
VDB.

But considering any of that as a blocker to fix a fundamental bug in
dependency calculation, handling of VDB and PMS compatibility is close
to being silly, I'm sorry.

Re: [gentoo-dev] don't rely on dynamic deps

[Ciaran McCreesh]

[-- Attachment #1: Type: text/plain, Size: 339 bytes --]

On Tue, 22 Jul 2014 22:05:54 +0300
Samuli Suominen <ssuominen@gentoo.org> wrote:
> The quality of the distribution doesn't improve by killing one of the
> most important features the package manager has.

Uh, that's a bit of an odd claim, given that dynamic deps often doesn't
do what you're after anyway...

-- 
Ciaran McCreesh

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 181 bytes --]

Re: [gentoo-dev] don't rely on dynamic deps

[Michał Górny]

[-- Attachment #1: Type: text/plain, Size: 2357 bytes --]

Dnia 2014-07-22, o godz. 09:25:45
""Paweł Hajdan, Jr."" <phajdan.jr@gentoo.org> napisał(a):

> On 7/21/14, 11:52 PM, Alexander Berntsen wrote:
> > Michał has documented the shortcomings of dynamic deps in our wiki[0].
> > (Thank you!) This documentation also includes two of our possible
> > solutions.
> >
> > [0]  <https://wiki.gentoo.org/wiki/Project:Portage/Dynamic_dependencies>  
> 
> Thank you, this is very useful. I didn't understand the issue much
> before reading that page.
> 
> One question: why for "Removal of a USE flag along with the relevant
> dependencies" dynamic deps say "revbump + unnecessary rebuild"? What
> would happen without the revbump?

Well, for completeness I'll start by listing what would happen with
static deps. Though nothing will actually happen. Already-built
packages may have the flag enabled along with deps. When people rebuild
-- through --newuse, --changed-use or otherwise -- they will get
the functionality and dependencies removed along with the flag.

How dynamic-deps would handle that are pretty much a matter of
implementation choice. I can think of two major possibilities:

1. dynamic-deps are applied nevertheless. Since the relevant
dependencies are removed along with the flag, dynamic-deps causes
portage to no longer pull in needed libraries even though package was
built with the flag enabled and still links to them. Long story short,
linkage is broken.

2. PM notices IUSE no longer matches and doesn't apply dynamic-deps.
While this prevents the breakage from happening, it's one additional
point to the list of cases when dynamic-deps are disabled. Which means
that no further dependency changes to the ebuild have dynamic effect
and -- with current portage implementation -- the dependencies are
'reverted' to the state when the package was installed, even though
just before the change portage used ebuild dependencies.

Of course, you could try to invent some smart logic that would handle
this specific case better. But it makes the dependency resolution even
more complex and hard to understand, plus someone needs to actually do
it. And then, you're going to hit even more corner cases and implement
additional workarounds for each one. And then each of those workarounds
will create new corner cases...

-- 
Best regards,
Michał Górny

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 949 bytes --]

Re: [gentoo-dev] don't rely on dynamic deps

[Tom Wijsman]

[-- Attachment #1: Type: text/plain, Size: 839 bytes --]

On Tue, 22 Jul 2014 09:25:45 +0200
""Paweł Hajdan, Jr."" <phajdan.jr@gentoo.org> wrote:

> One question: why for "Removal of a USE flag along with the relevant
> dependencies" dynamic deps say "revbump + unnecessary rebuild"? What
> would happen without the revbump?

Assuming dynamic dependencies don't exist, another package would still
depend on the USE flag in the vdb; the only way to force that USE flag
dependency to go away is with an unnecessarily rebuilding rev bump, as
without it it would complain that the USE flag doesn't exist.

(We're talking about RDEPEND="some/pkg[useflag]" --> RDEPEND="some/pkg")


-- 
With kind regards,

Tom Wijsman (TomWij)
Gentoo Developer

E-mail address  : TomWij@gentoo.org
GPG Public Key  : 6D34E57D
GPG Fingerprint : C165 AF18 AB4C 400B C3D2  ABF0 95B2 1FCD 6D34 E57D

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 473 bytes --]

[gentoo-dev] Re: don't rely on dynamic deps

[Martin Vaeth]

Tom Wijsman <TomWij@gentoo.org> wrote:
>
> Assuming dynamic dependencies don't exist, another package would still
> depend on the USE flag in the vdb; the only way to force that USE flag
> dependency to go away is with an unnecessarily rebuilding rev bump, as
> without it it would complain that the USE flag doesn't exist.

Yes, this is a typical example:
After every adding or removing of a python version
(I am not speaking about *installed* versions)
with the new policy every package depending on some python
needs to be bumped.
This demonstrates very well what I said:
With the new policy you would have to recompile
your whole system about once a week.

I rather prefer to have a system which perhaps very rarely
forces me to use revdep-rebuild (which cannot be omitted
anyway, as shown in my previous posting) than to have a
permanently outdated system, without having any possibility
to know what *really* needs to be updated or is only update
because of the broken "static deps" idea.

[gentoo-dev] Re: don't rely on dynamic deps

[Michael Palimaka]

On 07/22/2014 07:52 AM, Alexander Berntsen wrote:
> 
> To sum up: My vote is disable dynamic-deps. And I would be happy to
> apply a patch that does this with the information I have today.

What a great way to kill the distro.

I can already heat my house with the number of unnecessary rebuilds - I
can't imagine how many people will be left once we have to needlessly
rebuild libreoffice and half the tree every time someone makes some
minor change. If developers can't revbump correctly to address the
shortcomings of dynamic deps, why do we expect they will be able to do
so for static deps?

When can we expect this issue to be brought before the Council? I look
forward to seeing the specific examples of unavoidable breakage that
would be required to make such a decision.

Re: [gentoo-dev] Re: don't rely on dynamic deps

[Kristian Fiskerstrand]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA512

On 07/22/2014 10:21 AM, Michael Palimaka wrote:
> On 07/22/2014 07:52 AM, Alexander Berntsen wrote:
>> 
>> To sum up: My vote is disable dynamic-deps. And I would be happy
>> to apply a patch that does this with the information I have
>> today.
> 
> What a great way to kill the distro.
> 
> I can already heat my house with the number of unnecessary rebuilds
> - I can't imagine how many people will be left once we have to
> needlessly rebuild libreoffice and half the tree every time someone
> makes some minor change. If developers can't revbump correctly to
> address the shortcomings of dynamic deps, why do we expect they
> will be able to do so for static deps?
> 

I find it somewhat curious that the difference between ~arch and
stable hasn't been brought up in this discussion yet. IMHO a user on
~arch should expect a higher number of rebuilds, it _is_ after all
testing, whereby at the point it reaches stable, the deps are
hopefully more likely to be correct to begin with.

Does anyone have any insight into where these changes most often occur?

- -- 
- ----------------------------
Kristian Fiskerstrand
Blog: http://blog.sumptuouscapital.com
Twitter: @krifisk
- ----------------------------
Public PGP key 0xE3EDFAE3 at hkp://pool.sks-keyservers.net
fpr:94CB AFDD 3034 5109 5618 35AA 0B7F 8B60 E3ED FAE3
- ----------------------------
Ad astra per aspera
To the stars through thorns
-----BEGIN PGP SIGNATURE-----

iQIcBAEBCgAGBQJTziGbAAoJEPw7F94F4Tag61QP/iznpmFoc2jKO1Ex8fHEFURA
8D6mgzmkBW2OYWux8JnSfmS7WBrXvs0869Ey/dTgQeMWsdaauhFSqGUTL8k2s3pD
2NZUiNM9fBDGrVR589r0EX5jpPoylsq+ViYc/GtsWfUx8QZGRQOs75ecIB3G5dfS
eg15r/UI7Q5/vQv93s49Wl3EKWR8peqf46nsluXLMLZff80I6S2T0wGTZP9lMHd7
62Lwo4MxQEm+0XBqVNiY438qaF9eZBR8W14BPUwn2VWD0tL8CtNLiHZwLAAnYZrs
FE4mgAFUQu+cmJow4DAPkOxMqiqFHLlyh2wVKkxNVTp4fwYdLLeQZWLVLqF3Z7BR
cx75ocKCZmxcKqPA6gYj0hcl1W8uj7WyAwIOcYTzBBFf0eSaBzErtZ1WS7GVM/7z
1cauEo7DsDjiW2THZDkSy/zLn/O9XxRwMddqT7rT7IxDiy+V0n6AW4WD7mA/sAkd
YfEnQmZARF0hK7msiy88ScBK73NpmAmU04kVoIV1IUaKNjaahWi4sk8MDKbV/V7z
qnXvbQEejH9O9FbsY0ugWB6TL1imyfE3Va+Z63/nF3GFtO+XwJ3fqpT0VbDR3YBL
sYXNQ9CHRBoemIOaCLLGPJbkwAALS2/gt9aVsxHLE75efvuGAqj2Qa9g8Paj5WBH
Pq8eqnjDYOX02BBjSzSr
=sYA4
-----END PGP SIGNATURE-----

Re: [gentoo-dev] Re: don't rely on dynamic deps

[Pacho Ramos]

El mar, 22-07-2014 a las 10:32 +0200, Kristian Fiskerstrand escribió:
[...]
> I find it somewhat curious that the difference between ~arch and
> stable hasn't been brought up in this discussion yet. IMHO a user on
> ~arch should expect a higher number of rebuilds, it _is_ after all
> testing, whereby at the point it reaches stable, the deps are
> hopefully more likely to be correct to begin with.
> 
> Does anyone have any insight into where these changes most often occur?
> 

Well, I have seen multiple times of this kind of fixes being noticed by
people running really old stable boxes. They notice them when they
update to latest stable and, then, we need to fix depends raising the
versions usually :/

Maybe this discussion should be focused on trying to think about how to
standardize a way for distinguish between revision bumps needing full
rebuild or only VDB updates :|

Re: [gentoo-dev] Re: don't rely on dynamic deps

[Sven Vermeulen]

On July 22, 2014 11:25:05 AM CEST, Pacho Ramos <pacho@gentoo.org> wrote:
>El mar, 22-07-2014 a las 10:32 +0200, Kristian Fiskerstrand escribió:
>[...]
>> I find it somewhat curious that the difference between ~arch and
>> stable hasn't been brought up in this discussion yet. IMHO a user on
>> ~arch should expect a higher number of rebuilds, it _is_ after all
>> testing, whereby at the point it reaches stable, the deps are
>> hopefully more likely to be correct to begin with.
>> 
>> Does anyone have any insight into where these changes most often
>occur?
>> 
>
>Well, I have seen multiple times of this kind of fixes being noticed by
>people running really old stable boxes. They notice them when they
>update to latest stable and, then, we need to fix depends raising the
>versions usually :/
>
>Maybe this discussion should be focused on trying to think about how to
>standardize a way for distinguish between revision bumps needing full
>rebuild or only VDB updates :|

As someone who regularly adds in dependencies without bumping (adding USE=selinux dependencies to the proper SELinux policy) because that would trigger lots of totally unnecessary rebuilds: 

+1

Wkr,
  Sven
-- 
Sent from my Android device with K-9 Mail. Please excuse my brevity.

Re: [gentoo-dev] Re: don't rely on dynamic deps

[Ciaran McCreesh]

[-- Attachment #1: Type: text/plain, Size: 522 bytes --]

On Tue, 22 Jul 2014 19:03:16 +0200
Sven Vermeulen <swift@gentoo.org> wrote:
> As someone who regularly adds in dependencies without bumping (adding
> USE=selinux dependencies to the proper SELinux policy) because that
> would trigger lots of totally unnecessary rebuilds: 

Er... You do realise that doing that with dynamic dependencies leads to
packages depending upon selinux that don't actually use selinux, right?
Thus triggering lots of totally unnecessary rebuilds on an ABI change.

-- 
Ciaran McCreesh

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 181 bytes --]

Re: [gentoo-dev] Re: don't rely on dynamic deps

[Samuli Suominen]

On 22/07/14 20:11, Ciaran McCreesh wrote:
> On Tue, 22 Jul 2014 19:03:16 +0200
> Sven Vermeulen <swift@gentoo.org> wrote:
>> As someone who regularly adds in dependencies without bumping (adding
>> USE=selinux dependencies to the proper SELinux policy) because that
>> would trigger lots of totally unnecessary rebuilds: 
> Er... You do realise that doing that with dynamic dependencies leads to
> packages depending upon selinux that don't actually use selinux, right?
> Thus triggering lots of totally unnecessary rebuilds on an ABI change.
>

Err, I believe he meant adding RDEPEND -only USE="selinux" to pull in
eg. sec-policy/selinux-dante
from net-proxy/dante
So, how does that exactly make "packages depending upon selinux that
don't actually use selinux"
Doesn't make any sense

- Samuli

Re: [gentoo-dev] Re: don't rely on dynamic deps

[Samuli Suominen]

On 22/07/14 11:21, Michael Palimaka wrote:
> On 07/22/2014 07:52 AM, Alexander Berntsen wrote:
>> To sum up: My vote is disable dynamic-deps. And I would be happy to
>> apply a patch that does this with the information I have today.
> What a great way to kill the distro.
>
> I can already heat my house with the number of unnecessary rebuilds - I
> can't imagine how many people will be left once we have to needlessly
> rebuild libreoffice and half the tree every time someone makes some
> minor change. If developers can't revbump correctly to address the
> shortcomings of dynamic deps, why do we expect they will be able to do
> so for static deps?
>
> When can we expect this issue to be brought before the Council? I look
> forward to seeing the specific examples of unavoidable breakage that
> would be required to make such a decision.
>
>

+1

Re: [gentoo-dev] Re: don't rely on dynamic deps

[Tom Wijsman]

[-- Attachment #1: Type: text/plain, Size: 557 bytes --]

On Tue, 22 Jul 2014 18:21:00 +1000
Michael Palimaka <kensington@gentoo.org> wrote:

> What a great way to kill the distro.
> 
> I can already heat my house with the number of unnecessary rebuilds

Do you upgrade @world every hour and thus have it cause excessive heat?

If I upgrade every X weeks they become much more cool and necessary...

-- 
With kind regards,

Tom Wijsman (TomWij)
Gentoo Developer

E-mail address  : TomWij@gentoo.org
GPG Public Key  : 6D34E57D
GPG Fingerprint : C165 AF18 AB4C 400B C3D2  ABF0 95B2 1FCD 6D34 E57D

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 473 bytes --]

Re: [gentoo-dev] Re: don't rely on dynamic deps

[Michael Palimaka]

On 07/23/2014 09:36 AM, Tom Wijsman wrote:
> On Tue, 22 Jul 2014 18:21:00 +1000
> Michael Palimaka <kensington@gentoo.org> wrote:
> 
>> What a great way to kill the distro.
>>
>> I can already heat my house with the number of unnecessary rebuilds
> 
> Do you upgrade @world every hour and thus have it cause excessive heat?
> 
> If I upgrade every X weeks they become much more cool and necessary...
> 

Shouldn't we strive to avoid the unnecessary rebuilds in the first
place? Doing updates on your schedule only avoids the symptom, not the
problem.

Re: [gentoo-dev] Re: don't rely on dynamic deps

[Tom Wijsman]

[-- Attachment #1: Type: text/plain, Size: 1469 bytes --]

On Wed, 23 Jul 2014 22:14:41 +1000
Michael Palimaka <kensington@gentoo.org> wrote:

> On 07/23/2014 09:36 AM, Tom Wijsman wrote:
> > On Tue, 22 Jul 2014 18:21:00 +1000
> > Michael Palimaka <kensington@gentoo.org> wrote:
> > 
> >> What a great way to kill the distro.
> >>
> >> I can already heat my house with the number of unnecessary rebuilds
> > 
> > Do you upgrade @world every hour and thus have it cause excessive
> > heat?
> > 
> > If I upgrade every X weeks they become much more cool and
> > necessary...
> > 
> 
> Shouldn't we strive to avoid the unnecessary rebuilds in the first
> place? Doing updates on your schedule only avoids the symptom, not the
> problem.

We should strive to do both; cause less rebuilds, update less often.

It is comparable to flooding on IRC channels; if you send much more
messages, you are much more likely to experience a kick and/or a ban.

It is easier not to flood than to convince people there is no problem
with you flooding the channel; out of all the IRC channels I know of,
I've only come across one where they don't mind pasted long code blocks
but that's mostly because of the lack of active moderation and people.

(With "flooding" as "updating" and "kick/ban" as "rebuilds")

-- 
With kind regards,

Tom Wijsman (TomWij)
Gentoo Developer

E-mail address  : TomWij@gentoo.org
GPG Public Key  : 6D34E57D
GPG Fingerprint : C165 AF18 AB4C 400B C3D2  ABF0 95B2 1FCD 6D34 E57D

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 473 bytes --]

[gentoo-dev] Re: don't rely on dynamic deps

[Michael Palimaka]

On 07/26/2014 07:59 AM, Tom Wijsman wrote:
> On Wed, 23 Jul 2014 22:14:41 +1000
> Michael Palimaka <kensington@gentoo.org> wrote:
> 
>> On 07/23/2014 09:36 AM, Tom Wijsman wrote:
>>> On Tue, 22 Jul 2014 18:21:00 +1000
>>> Michael Palimaka <kensington@gentoo.org> wrote:
>>>
>>>> What a great way to kill the distro.
>>>>
>>>> I can already heat my house with the number of unnecessary rebuilds
>>>
>>> Do you upgrade @world every hour and thus have it cause excessive
>>> heat?
>>>
>>> If I upgrade every X weeks they become much more cool and
>>> necessary...
>>>
>>
>> Shouldn't we strive to avoid the unnecessary rebuilds in the first
>> place? Doing updates on your schedule only avoids the symptom, not the
>> problem.
> 
> We should strive to do both; cause less rebuilds, update less often.
> 
> It is comparable to flooding on IRC channels; if you send much more
> messages, you are much more likely to experience a kick and/or a ban.
> 
> It is easier not to flood than to convince people there is no problem
> with you flooding the channel; out of all the IRC channels I know of,
> I've only come across one where they don't mind pasted long code blocks
> but that's mostly because of the lack of active moderation and people.
> 
> (With "flooding" as "updating" and "kick/ban" as "rebuilds")
> 
Each person should update at a frequency that suits them. Recommending
to update every $period is not a valid solution to unnecessary rebuilds.

Re: [gentoo-dev] Re: don't rely on dynamic deps

[Tom Wijsman]

[-- Attachment #1: Type: text/plain, Size: 1949 bytes --]

On Sun, 27 Jul 2014 03:12:07 +1000
Michael Palimaka <kensington@gentoo.org> wrote:

> On 07/26/2014 07:59 AM, Tom Wijsman wrote:
> > On Wed, 23 Jul 2014 22:14:41 +1000
> > Michael Palimaka <kensington@gentoo.org> wrote:
> > 
> >> On 07/23/2014 09:36 AM, Tom Wijsman wrote:
> >>> On Tue, 22 Jul 2014 18:21:00 +1000
> >>> Michael Palimaka <kensington@gentoo.org> wrote:
> >>>
> >>>> What a great way to kill the distro.
> >>>>
> >>>> I can already heat my house with the number of unnecessary
> >>>> rebuilds
> >>>
> >>> Do you upgrade @world every hour and thus have it cause excessive
> >>> heat?
> >>>
> >>> If I upgrade every X weeks they become much more cool and
> >>> necessary...
> >>>
> >>
> >> Shouldn't we strive to avoid the unnecessary rebuilds in the first
> >> place? Doing updates on your schedule only avoids the symptom, not
> >> the problem.
> > 
> > We should strive to do both; cause less rebuilds, update less often.
> > 
> > It is comparable to flooding on IRC channels; if you send much more
> > messages, you are much more likely to experience a kick and/or a
> > ban.
> > 
> > It is easier not to flood than to convince people there is no
> > problem with you flooding the channel; out of all the IRC channels
> > I know of, I've only come across one where they don't mind pasted
> > long code blocks but that's mostly because of the lack of active
> > moderation and people.
> > 
> > (With "flooding" as "updating" and "kick/ban" as "rebuilds")
> > 
> Each person should update at a frequency that suits them. Recommending
> to update every $period is not a valid solution to unnecessary
> rebuilds.

The more one floods, the more one accepts kicks and/or bans; expected.

-- 
With kind regards,

Tom Wijsman (TomWij)
Gentoo Developer

E-mail address  : TomWij@gentoo.org
GPG Public Key  : 6D34E57D
GPG Fingerprint : C165 AF18 AB4C 400B C3D2  ABF0 95B2 1FCD 6D34 E57D

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 473 bytes --]

[gentoo-dev] Re: don't rely on dynamic deps

[Michael Palimaka]

On 07/27/2014 05:21 AM, Tom Wijsman wrote:
> On Sun, 27 Jul 2014 03:12:07 +1000
> Michael Palimaka <kensington@gentoo.org> wrote:
> 
>> On 07/26/2014 07:59 AM, Tom Wijsman wrote:
>>> On Wed, 23 Jul 2014 22:14:41 +1000
>>> Michael Palimaka <kensington@gentoo.org> wrote:
>>>
>>>> On 07/23/2014 09:36 AM, Tom Wijsman wrote:
>>>>> On Tue, 22 Jul 2014 18:21:00 +1000
>>>>> Michael Palimaka <kensington@gentoo.org> wrote:
>>>>>
>>>>>> What a great way to kill the distro.
>>>>>>
>>>>>> I can already heat my house with the number of unnecessary
>>>>>> rebuilds
>>>>>
>>>>> Do you upgrade @world every hour and thus have it cause excessive
>>>>> heat?
>>>>>
>>>>> If I upgrade every X weeks they become much more cool and
>>>>> necessary...
>>>>>
>>>>
>>>> Shouldn't we strive to avoid the unnecessary rebuilds in the first
>>>> place? Doing updates on your schedule only avoids the symptom, not
>>>> the problem.
>>>
>>> We should strive to do both; cause less rebuilds, update less often.
>>>
>>> It is comparable to flooding on IRC channels; if you send much more
>>> messages, you are much more likely to experience a kick and/or a
>>> ban.
>>>
>>> It is easier not to flood than to convince people there is no
>>> problem with you flooding the channel; out of all the IRC channels
>>> I know of, I've only come across one where they don't mind pasted
>>> long code blocks but that's mostly because of the lack of active
>>> moderation and people.
>>>
>>> (With "flooding" as "updating" and "kick/ban" as "rebuilds")
>>>
>> Each person should update at a frequency that suits them. Recommending
>> to update every $period is not a valid solution to unnecessary
>> rebuilds.
> 
> The more one floods, the more one accepts kicks and/or bans; expected.
> 

How about just not causing the problem in the first place? :-)

Re: [gentoo-dev] Re: don't rely on dynamic deps

[Tom Wijsman]

[-- Attachment #1: Type: text/plain, Size: 2233 bytes --]

On Sun, 27 Jul 2014 05:30:26 +1000
Michael Palimaka <kensington@gentoo.org> wrote:

> On 07/27/2014 05:21 AM, Tom Wijsman wrote:
> > On Sun, 27 Jul 2014 03:12:07 +1000
> > Michael Palimaka <kensington@gentoo.org> wrote:
> > 
> >> On 07/26/2014 07:59 AM, Tom Wijsman wrote:
> >>> On Wed, 23 Jul 2014 22:14:41 +1000
> >>> Michael Palimaka <kensington@gentoo.org> wrote:
> >>>
> >>>> Shouldn't we strive to avoid the unnecessary rebuilds in the
> >>>> first place? Doing updates on your schedule only avoids the
> >>>> symptom, not the problem.
> >>>
> >>> We should strive to do both; cause less rebuilds, update less
> >>> often.
> >>>
> >>> It is comparable to flooding on IRC channels; if you send much
> >>> more messages, you are much more likely to experience a kick
> >>> and/or a ban.
> >>>
> >>> It is easier not to flood than to convince people there is no
> >>> problem with you flooding the channel; out of all the IRC channels
> >>> I know of, I've only come across one where they don't mind pasted
> >>> long code blocks but that's mostly because of the lack of active
> >>> moderation and people.
> >>>
> >>> (With "flooding" as "updating" and "kick/ban" as "rebuilds")
> >>>
> >> Each person should update at a frequency that suits them.
> >> Recommending to update every $period is not a valid solution to
> >> unnecessary rebuilds.
> > 
> > The more one floods, the more one accepts kicks and/or bans;
> > expected.
> > 
> 
> How about just not causing the problem in the first place? :-)

That's the ideal, no revision bumps needed at all; though, the lack of
resources doesn't make that possible. Attempts to do it stall the
introduction of the ebuild; so, that's why we release and revbump it.

This story goes further upstream; if they would list deps right, we
wouldn't need to revbump. So; if we want to fix the cause, we would
need to fix it upstream although they experience a lack of resources.

TL;DR: With the water tap wide open, we'll keep mopping.

-- 
With kind regards,

Tom Wijsman (TomWij)
Gentoo Developer

E-mail address  : TomWij@gentoo.org
GPG Public Key  : 6D34E57D
GPG Fingerprint : C165 AF18 AB4C 400B C3D2  ABF0 95B2 1FCD 6D34 E57D

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 473 bytes --]

[gentoo-dev] Re: don't rely on dynamic deps

[Martin Vaeth]

Tom Wijsman <TomWij@gentoo.org> wrote:
> Michael Palimaka <kensington@gentoo.org> wrote:
>
>> What a great way to kill the distro.
>>
>> I can already heat my house with the number of unnecessary rebuilds
>
> Do you upgrade @world every hour and thus have it cause excessive heat?
>
> If I upgrade every X weeks they become much more cool and necessary...

One of the main advantages of gentoo is the flowing upgrade,
especially since this can only be very poorly emulated by
a binary distro.

If you really suggest that the user waits one month and
then recompiles the whole installation, you give up
this advantage of gentoo: The user is not up-to-date
for a long time, and moreover, then needs practically
a full reinstall; both are things which he wants to avoid
and why perhaps he has chosen gentoo in the first place.

At least, for me it is the case: if I have to reinstall
all packages every months - and even have delay in security
updates for a month - I will certainly switch the
distribution. I guess many others think similarly.

Re: [gentoo-dev] Re: don't rely on dynamic deps

[Tom Wijsman]

[-- Attachment #1: Type: text/plain, Size: 2543 bytes --]

On Sat, 26 Jul 2014 14:25:23 +0000 (UTC)
Martin Vaeth <martin@mvath.de> wrote:

> Tom Wijsman <TomWij@gentoo.org> wrote:
> > Michael Palimaka <kensington@gentoo.org> wrote:
> >
> >> What a great way to kill the distro.
> >>
> >> I can already heat my house with the number of unnecessary rebuilds
> >
> > Do you upgrade @world every hour and thus have it cause excessive
> > heat?
> >
> > If I upgrade every X weeks they become much more cool and
> > necessary...
> 
> One of the main advantages of gentoo is the flowing upgrade,
> especially since this can only be very poorly emulated by
> a binary distro.
> 
> If you really suggest that the user waits one month and
> then recompiles the whole installation, you give up
> this advantage of gentoo: The user is not up-to-date
> for a long time, and moreover, then needs practically
> a full reinstall; both are things which he wants to avoid
> and why perhaps he has chosen gentoo in the first place.
> 
> At least, for me it is the case: if I have to reinstall
> all packages every months - and even have delay in security
> updates for a month - I will certainly switch the
> distribution. I guess many others think similarly.

Simple equation: The more frequent the user updates, the more frequent
the user will experience the minor inconveniences by upstream and
distribution maintainers. Otherwise we'd be using a 9999-only system.

Dynamic deps, as well as rev bumps, alter this equation; the problem
with that is that such alterations don't come free and without flaws,
which is essentially where you get to reconsider how you alter it.

In a similar way the user has to reconsider whether updating less is
acceptable compared to compiling an occasional inconvenience. Choosing
between a stable and non-stable tree is a big gap of difference in
convenience, choosing how often you update is fine tuning.

To get the idea: "Upstream released W.X.Y.Z+1; it was only yesterday
I've compiled W.X.Y.Z, turns out the difference is not so important."
Agreed that this can very well be an important security update; but
if you go back to the equation, that still is a minor inconvenience.

PS: Not suggesting 1 month; but rather that updating not enough, or too
much, can make one experience serious effects that such choices imply.

-- 
With kind regards,

Tom Wijsman (TomWij)
Gentoo Developer

E-mail address  : TomWij@gentoo.org
GPG Public Key  : 6D34E57D
GPG Fingerprint : C165 AF18 AB4C 400B C3D2  ABF0 95B2 1FCD 6D34 E57D

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 473 bytes --]

Re: [gentoo-dev] don't rely on dynamic deps

[hasufell]

Alexander Berntsen:
> 
> Julian,
> 
> would you like to share your experiences with Paludis? My guess is
> that Paludis is more predictable in this respect. I.e., instead of
> breaking stuff, I expect Paludis to simply give up.
> 

Relying on dynamic deps as they are currently implemented simply causes
the vdb to enter a broken state after some time.

If you switch from portage to paludis then, you will encounter a lot of
"unsuitable candidates" messages during dependency resolving and will
have to figure out that the broken vdb is the reason. Afais paludis does
not allow you to randomly break the vdb (or I haven't found the --nodeps
option yet).

[gentoo-dev] Re: don't rely on dynamic deps

[Martin Vaeth]

Alexander Berntsen <bernalex@gentoo.org> wrote:
>
> 1. Improve dynamic-deps. This is, as Michał pointed out earlier in
> this thread a pipe dream.

Not necessarily. Just somebody with enough knowledge in
portage and python has to fix it.
The problem is that in gentoo there seems to be currently
nobody with these skills and free time...

> PMS defines a static dependency model

No. PMS does not specify which dependency information has
to be taken. IIRC it does not even specify that dependency
information has to be stored in /var/db at all.

Re: [gentoo-dev] Re: don't rely on dynamic deps

[Ciaran McCreesh]

[-- Attachment #1: Type: text/plain, Size: 299 bytes --]

On Sat, 26 Jul 2014 14:09:44 +0000 (UTC)
Martin Vaeth <martin@mvath.de> wrote:
> > PMS defines a static dependency model
> 
> No. PMS does not specify which dependency information has
> to be taken.

Yes it does. Please read PMS, and do not guess as to what it says.

-- 
Ciaran McCreesh

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 181 bytes --]

[gentoo-dev] Re: don't rely on dynamic deps

[Martin Vaeth]

Ciaran McCreesh <ciaran.mccreesh@googlemail.com> wrote:
>> No. PMS does not specify which dependency information has
>> to be taken.
>
> Yes it does. Please read PMS, and do not guess as to what it says.

Looking for /var/db/pkg gave exactly one hit:
The assertion that this is *not* part of PMS.

The section on dependencies is only about the ebuild format.

So either it is very hidden or not in there.

Re: [gentoo-dev] Re: don't rely on dynamic deps

[Ciaran McCreesh]

[-- Attachment #1: Type: text/plain, Size: 683 bytes --]

On Sat, 26 Jul 2014 14:33:38 +0000 (UTC)
Martin Vaeth <martin@mvath.de> wrote:
> Ciaran McCreesh <ciaran.mccreesh@googlemail.com> wrote:
> >> No. PMS does not specify which dependency information has
> >> to be taken.
> >
> > Yes it does. Please read PMS, and do not guess as to what it says.
> 
> Looking for /var/db/pkg gave exactly one hit:
> The assertion that this is *not* part of PMS.
> 
> The section on dependencies is only about the ebuild format.
> 
> So either it is very hidden or not in there.

I said read PMS, not grep PMS. You're making all kinds of wild claims
based upon what you guess PMS says, not what it actually says.

-- 
Ciaran McCreesh

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 181 bytes --]

Re: [gentoo-dev] Re: don't rely on dynamic deps

[hasufell]

Ciaran McCreesh:
> On Sat, 26 Jul 2014 14:33:38 +0000 (UTC)
> Martin Vaeth <martin@mvath.de> wrote:
>> Ciaran McCreesh <ciaran.mccreesh@googlemail.com> wrote:
>>>> No. PMS does not specify which dependency information has
>>>> to be taken.
>>>
>>> Yes it does. Please read PMS, and do not guess as to what it says.
>>
>> Looking for /var/db/pkg gave exactly one hit:
>> The assertion that this is *not* part of PMS.
>>
>> The section on dependencies is only about the ebuild format.
>>
>> So either it is very hidden or not in there.
> 
> I said read PMS, not grep PMS. You're making all kinds of wild claims
> based upon what you guess PMS says, not what it actually says.
> 

This looks like a private lesson in PMS reading. Can you guys do that on
IRC?

[gentoo-dev] Re: don't rely on dynamic deps

[Martin Vaeth]

hasufell <hasufell@gentoo.org> wrote:
>>> Ciaran McCreesh <ciaran.mccreesh@googlemail.com> wrote:
>
> This looks like a private lesson

*If* there is a corresponding passage in PMS, it seems to be
somewhat hidden (as I pointed out) and certainly many others
haven't found this, either.
(As the eix maintainer, I know PMS probably better than many
in this list, which of course does not exclude that I might have
missed some passage anyway, since nobody reads PMS like a book.)

It would help everybody if the corresponding passage is cited
(if there is such a passage).

Instead, Ciaran posts only pointless personal attacks which
have nothing lost in a technical discussion.
That's why I do not reply to his posts again.
If I was even correct and PMS does not contain such
a passage, his behaviour is beyond any words.

Re: [gentoo-dev] Re: don't rely on dynamic deps

[Michał Górny]

[-- Attachment #1: Type: text/plain, Size: 656 bytes --]

Dnia 2014-07-26, o godz. 14:09:44
Martin Vaeth <martin@mvath.de> napisał(a):

> Alexander Berntsen <bernalex@gentoo.org> wrote:
> >
> > 1. Improve dynamic-deps. This is, as Michał pointed out earlier in
> > this thread a pipe dream.  
> 
> Not necessarily. Just somebody with enough knowledge in
> portage and python has to fix it.
> The problem is that in gentoo there seems to be currently
> nobody with these skills and free time...

All people with enough knowledge already know that this is technically
impossible. This is not theoretical physics, ignoring the impossible
won't breed anything here.

-- 
Best regards,
Michał Górny

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 949 bytes --]

[gentoo-dev] Re: don't rely on dynamic deps

[Martin Vaeth]

Michał Górny <mgorny@gentoo.org> wrote:
>
> All people with enough knowledge already know that this is technically
> impossible.

We already discussed in the bug how it *would* be possible,
just nobody implements it:

Portage would have to use dynamic deps throughout,
using the data stored in /var/db only to find out
the correct information for := dependencies.

This would fix the behaviour except for some
corner cases concerning orphaned packages which
can lead to broken situations with any approach.

> This is not theoretical physics

Indeed, it just would just need a little programming.

Re: [gentoo-dev] Re: don't rely on dynamic deps

[hasufell]

Martin Vaeth:
> 
> Indeed, it just would just need a little programming.
> 

would you like to implement it?

[gentoo-dev] Re: don't rely on dynamic deps

[Martin Vaeth]

hasufell <hasufell@gentoo.org> wrote:
> Martin Vaeth:
>
>> Indeed, it just would just need a little programming.
>
> would you like to implement it?

I thought about it, but unfortunately, I am currently
(and for a long period) so busy with my job
that this is not realistic - I did not even find the
time to implement minor revisions completely (also,
since it would cost me too much time to get sufficient
familiar with python and the portage implementation).

Re: [gentoo-dev] Re: don't rely on dynamic deps

[Michał Górny]

[-- Attachment #1: Type: text/plain, Size: 747 bytes --]

Dnia 2014-07-26, o godz. 15:27:51
Martin Vaeth <martin@mvath.de> napisał(a):

> Michał Górny <mgorny@gentoo.org> wrote:
> >
> > All people with enough knowledge already know that this is technically
> > impossible.
> 
> We already discussed in the bug how it *would* be possible,
> just nobody implements it:
> 
> Portage would have to use dynamic deps throughout,
> using the data stored in /var/db only to find out
> the correct information for := dependencies.

This is only one of the issue. And what if the match for := is
incompatible with new dependency atom? Like when you replace
'dev-foo/bar:=' with '>=dev-foo/bar-2:=' but bar-1 is installed.
What if a new := dependency is added?

-- 
Best regards,
Michał Górny

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 949 bytes --]

[gentoo-dev] Re: don't rely on dynamic deps

[Martin Vaeth]

Michał Górny <mgorny@gentoo.org> wrote:
>
> This is only one of the issue.

Not all cases need to be solved: If a developer decides
that no revbump is not necessary, he should not have a
too problematic change...

> And what if the match for :=3D is
> incompatible with new dependency atom? Like when you replace
> 'dev-foo/bar:=3D' with '>=3Ddev-foo/bar-2:=3D' but bar-1 is installed.

This is simple: The dependency is not satisfied.

> What if a new :=3D dependency is added?

If it is *added* in an || ( ... ), it can safely be ignored.
Adding such a dependency unconditionally should probably
require a revbump.

Re: [gentoo-dev] Re: don't rely on dynamic deps

[Ciaran McCreesh]

[-- Attachment #1: Type: text/plain, Size: 434 bytes --]

On Sat, 26 Jul 2014 15:59:58 +0000 (UTC)
Martin Vaeth <martin@mvath.de> wrote:
> > And what if the match for :=3D is
> > incompatible with new dependency atom? Like when you replace
> > 'dev-foo/bar:=3D' with '>=3Ddev-foo/bar-2:=3D' but bar-1 is
> > installed.
> 
> This is simple: The dependency is not satisfied.

That isn't simple at all... It means you can't uninstall or upgrade the
package...

-- 
Ciaran McCreesh

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 181 bytes --]

Re: [gentoo-dev] Re: don't rely on dynamic deps

[Rich Freeman]

On Sat, Jul 26, 2014 at 12:02 PM, Ciaran McCreesh
<ciaran.mccreesh@googlemail.com> wrote:
> On Sat, 26 Jul 2014 15:59:58 +0000 (UTC)
> Martin Vaeth <martin@mvath.de> wrote:
>> > And what if the match for :=3D is
>> > incompatible with new dependency atom? Like when you replace
>> > 'dev-foo/bar:=3D' with '>=3Ddev-foo/bar-2:=3D' but bar-1 is
>> > installed.
>>
>> This is simple: The dependency is not satisfied.
>
> That isn't simple at all... It means you can't uninstall or upgrade the
> package...

Why not?

How is this any different from unmerging bar-1 back when bar-1
satisfied the dependency (using --unmerge which breaks reverse
dependencies)?

If you want to upgrade or re-install the package I would expect
portage to pull in the missing dependency.  I'd expect the next emerge
-u world to do that as well, which it already does if you --unmerge a
dependency).

If there would be some unintended side-effect from doing things this
way I'm all ears, but as long as you don't get into @system circular
dependencies issues I'd expect portage to be able to install any
packages that are missing after such a dependency change.

Sure, until the missing dep is installed I'd expect a risk of
breakage, but that is no different than what I'd expect if the package
were modified without a bump and the package manager didn't attempt to
support dynamic dependencies.

Rich

Re: [gentoo-dev] Re: don't rely on dynamic deps

[Ciaran McCreesh]

[-- Attachment #1: Type: text/plain, Size: 722 bytes --]

On Sat, 26 Jul 2014 12:36:45 -0400
Rich Freeman <rich0@gentoo.org> wrote:
> On Sat, Jul 26, 2014 at 12:02 PM, Ciaran McCreesh
> <ciaran.mccreesh@googlemail.com> wrote:
> > On Sat, 26 Jul 2014 15:59:58 +0000 (UTC)
> > Martin Vaeth <martin@mvath.de> wrote:
> >> > And what if the match for :=3D is
> >> > incompatible with new dependency atom? Like when you replace
> >> > 'dev-foo/bar:=3D' with '>=3Ddev-foo/bar-2:=3D' but bar-1 is
> >> > installed.
> >>
> >> This is simple: The dependency is not satisfied.
> >
> > That isn't simple at all... It means you can't uninstall or upgrade
> > the package...
> 
> Why not?

pkg_prerm. See the discussion about the ruby-config problem.

-- 
Ciaran McCreesh

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 181 bytes --]

Re: [gentoo-dev] Re: don't rely on dynamic deps

[Ciaran McCreesh]

[-- Attachment #1: Type: text/plain, Size: 923 bytes --]

On Sat, 26 Jul 2014 15:27:51 +0000 (UTC)
Martin Vaeth <martin@mvath.de> wrote:
> Michał Górny <mgorny@gentoo.org> wrote:
> > All people with enough knowledge already know that this is
> > technically impossible.
> 
> We already discussed in the bug how it *would* be possible,
> just nobody implements it:
> 
> Portage would have to use dynamic deps throughout,
> using the data stored in /var/db only to find out
> the correct information for := dependencies.
> 
> This would fix the behaviour except for some
> corner cases concerning orphaned packages which
> can lead to broken situations with any approach.

Your solution fails spectacularly in the following ways:

* Ebuild removal

* Overlays

* Introduction of := dependencies

* pkg_*rm

Which brings us back to the "all people with enough knowledge
already know that this is technically impossible" thing...

-- 
Ciaran McCreesh

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 181 bytes --]

[gentoo-dev] Re: don't rely on dynamic deps

[Martin Vaeth]

Ciaran McCreesh <ciaran.mccreesh@googlemail.com> wrote:
> Your solution fails spectacularly in the following ways:
>
> * Ebuild removal

Already discussed as to fail with static deps, too.

> * Overlays

Not an issue: Exactly the information of that ebuild
which *would* be used if you reemerge contains
the relevant data.

> * Introduction of :=3D dependencies

This is not a "minor update" in dependencies
and thus requires a revbump.

> * pkg_*rm

Not related.

> Which brings us back to the

... previous childish behaviour

Re: [gentoo-dev] Re: don't rely on dynamic deps

[Ciaran McCreesh]

[-- Attachment #1: Type: text/plain, Size: 1153 bytes --]

On Sat, 26 Jul 2014 16:05:58 +0000 (UTC)
Martin Vaeth <martin@mvath.de> wrote:
> Ciaran McCreesh <ciaran.mccreesh@googlemail.com> wrote:
> > Your solution fails spectacularly in the following ways:
> >
> > * Ebuild removal
> 
> Already discussed as to fail with static deps, too.

Uh, static dependencies don't behave any differently when an ebuild is
removed. I don't think you understand how that works.

> > * Overlays
> 
> Not an issue: Exactly the information of that ebuild
> which *would* be used if you reemerge contains
> the relevant data.

The association between an installed package and "the ebuild it came
from" doesn't work correctly when overlays around. Again, you don't
understand the issue.

> > * Introduction of :=3D dependencies
> 
> This is not a "minor update" in dependencies
> and thus requires a revbump.

So what is a "minor update", and what are you planning to do to prevent
what you call "useless rebuilds" when := dependencies are introduced?

> > * pkg_*rm
> 
> Not related.

Yes it is. Read and understand the previous discussion about the
ruby-config issue.

-- 
Ciaran McCreesh

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 181 bytes --]

Re: [gentoo-dev] Re: don't rely on dynamic deps

[Rich Freeman]

On Sat, Jul 26, 2014 at 12:14 PM, Ciaran McCreesh
<ciaran.mccreesh@googlemail.com> wrote:
> On Sat, 26 Jul 2014 16:05:58 +0000 (UTC)
> Martin Vaeth <martin@mvath.de> wrote:
>> Ciaran McCreesh <ciaran.mccreesh@googlemail.com> wrote:
>> > Your solution fails spectacularly in the following ways:
>
>> > * Introduction of :=3D dependencies
>>
>> This is not a "minor update" in dependencies
>> and thus requires a revbump.
>
> So what is a "minor update", and what are you planning to do to prevent
> what you call "useless rebuilds" when := dependencies are introduced?
>

Picking this to illustrate my point, not to endorse a particular
implementation here...

This is what I'm getting at when I argue that we don't need a solution
to every possible problem.  If we only accept solutions that handle
conditional dependencies, IUSE changes, new eclass inherits, dep
additions, dep removals, etc, then it does seem likely to me that
we'll never find a good solution.

On the other hand, if we can come up with something that RELIABLY
fixes things for 3/4ths of these, then that is probably good enough.
I'm not certain at this point that even such a partial solution
doesn't exist, but the fact that any particular solution doesn't
handle every possible case isn't automatically a reason to reject it.

Preventing unnecessary rebuilds is a worthwhile goal, even if we can't
get 100% of the way there.  If you don't care whatsoever about
unnecessary rebuilds then we can simplify things tremendously - just
have the package manager default to --emptytree on all operations.
Sure, it might cause a "few" unnecessary ebuilds but whether your
package manager attempts to support dynamic deps or not you'll
certainly have an up-to-date dependency cache.

Rich

Re: [gentoo-dev] Re: don't rely on dynamic deps

[Ciaran McCreesh]

[-- Attachment #1: Type: text/plain, Size: 327 bytes --]

On Sat, 26 Jul 2014 12:28:27 -0400
Rich Freeman <rich0@gentoo.org> wrote:
> Sure, it might cause a "few" unnecessary ebuilds but whether your
> package manager attempts to support dynamic deps or not you'll
> certainly have an up-to-date dependency cache.

VDB is not a cache. This is important.

-- 
Ciaran McCreesh

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 181 bytes --]

[gentoo-dev] Re: don't rely on dynamic deps

[Martin Vaeth]

Ciaran McCreesh <ciaran.mccreesh@googlemail.com> wrote:
>>
>> > * Overlays
>> Not an issue: Exactly the information of that ebuild
>> which *would* be used if you reemerge contains
>> the relevant data.
>
> The association between an installed package and "the ebuild it came
> from" doesn't work correctly when overlays around.

It doesn't have to: That ebuild which would be installed
*has* to carry the up-to-date information for that package.
Otherwise, the overlay is broken. An overlay is *not* a
different slot for a package.

> Again, you don't understand the issue.

So, that's it. This was my last reply to your trolling.

I hope the remaining developers will be smart enough
to not let you destroy the distribution again.

EOD with you.

Re: [gentoo-dev] Re: don't rely on dynamic deps

[Ciaran McCreesh]

[-- Attachment #1: Type: text/plain, Size: 893 bytes --]

On Sat, 26 Jul 2014 18:36:27 +0000 (UTC)
Martin Vaeth <martin@mvath.de> wrote:
> Ciaran McCreesh <ciaran.mccreesh@googlemail.com> wrote:
> >> > * Overlays
> >> Not an issue: Exactly the information of that ebuild
> >> which *would* be used if you reemerge contains
> >> the relevant data.
> >
> > The association between an installed package and "the ebuild it came
> > from" doesn't work correctly when overlays around.
> 
> It doesn't have to: That ebuild which would be installed
> *has* to carry the up-to-date information for that package.
> Otherwise, the overlay is broken. An overlay is *not* a
> different slot for a package.

Uh huh, so you add an overlay, and suddenly the dependencies for a
random subset of your installed packages change in ways that don't in
any way reflect what you have installed. How is this the desired
behaviour?

-- 
Ciaran McCreesh

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 181 bytes --]

Re: [gentoo-dev] Re: don't rely on dynamic deps

[Peter Stuge]

[-- Attachment #1: Type: text/plain, Size: 1957 bytes --]

Ciaran McCreesh wrote:
> Uh huh, so you add an overlay, and suddenly the dependencies for a
> random subset of your installed packages change in ways that don't in
> any way reflect what you have installed. How is this the desired
> behaviour?

There are several different cases of dependency data which apply in
different ways for different operations.

For a model to be accurate it needs to consider all cases, and treat
them differently where neccessary.

Adding an overlay doesn't change anything about what is installed.

Installing one version of an ebuild from an overlay, with another
version of the ebuild (from same overlay, or not) already installed,
clearly needs the package manager to consider what the overlay ebuild
says. That's the purpose of the overlay.

Overlay maintainers already have the responsibility to create
well-formed ebuilds, and there are even tools like overlint to help
with the task. If they don't, the applicability of their overlay
decreases. Sometimes this is on purpose, and it can be completely
unproblematic. It's a little like rewriting public git history.
Many people cry bloody murder about that, but in lots of cases it's
perfectly fine.

So are overlay dependencies which aren't perfectly well-formed.
That's probably why they're in an overlay in the first place.

It's not the responsibility of the package manager to magically
resolve dependencies without sufficient information.

It's the responsibility of ebuild authors to provide such
information, so that dependencies can be resolved accurately for all
package manager operations.

It seems that this may be a case of trying to do way too much, and
as a result doing a bad job at everything. I agree firmly with Rich
that it is much better to do a good job at some (well-known, perhaps
even documented) things, than a bad job at all things.

It's fine to simply error out with a message saying that the
encountered situation is not supported.


//Peter

[-- Attachment #2: Type: application/pgp-signature, Size: 190 bytes --]

Re: [gentoo-dev] don't rely on dynamic deps

["Paweł Hajdan, Jr."]

[-- Attachment #1: Type: text/plain, Size: 738 bytes --]

On 7/21/14, 11:52 PM, Alexander Berntsen wrote:
> Michał has documented the shortcomings of dynamic deps in our wiki[0].
> (Thank you!) [...]
> [0]  <https://wiki.gentoo.org/wiki/Project:Portage/Dynamic_dependencies>

There's one more thing I'd like to ask about:

For "Minor linking change w/ dependency change (unnecessary linking
removed)" the "dynamic deps" cell is red with "revbump + mostly
unnecessary rebuild", and "static deps" says "applied after rebuild".

Arguably with dynamic deps one could also skip the revbump, and the
update would similarly be applied after rebuild.

Is my logic correct? I'm just trying to understand it better, and don't
intend an argument in favor of any of the options.

Paweł


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 841 bytes --]

Re: [gentoo-dev] don't rely on dynamic deps

[Rich Freeman]

On Sun, Jul 27, 2014 at 10:05 AM, "Paweł Hajdan, Jr."
<phajdan.jr@gentoo.org> wrote:
> On 7/21/14, 11:52 PM, Alexander Berntsen wrote:
>> Michał has documented the shortcomings of dynamic deps in our wiki[0].
>> (Thank you!) [...]
>> [0]  <https://wiki.gentoo.org/wiki/Project:Portage/Dynamic_dependencies>
>
> There's one more thing I'd like to ask about:
>
> For "Minor linking change w/ dependency change (unnecessary linking
> removed)" the "dynamic deps" cell is red with "revbump + mostly
> unnecessary rebuild", and "static deps" says "applied after rebuild".
>
> Arguably with dynamic deps one could also skip the revbump, and the
> update would similarly be applied after rebuild.

With dynamic deps you'd need to revbump if there is a linking change.
Otherwise portage would just allow the dependency to be removed, and
then linking will break, since the executable is unnecessarily linked
to the dependency (in that scenario).

If you wanted to apply the change after rebuild then you'd need to
have a way to communicate to portage whether a dependency removal can
be applied immediately (extraneous unlinked dependency) or only after
rebuild (extraneous linked dependency).  The table speaks mostly to
portage as-is without additions like those posted in this thread.

I think that USE changes basically involve potentially-unnecessary
rebuilds no mater what in most real-life cases.  Most people use the
--newuse option when updating their system, so even if static
dependencies don't require rebuilds they're going to get them anyway.
Not using --newuse can cause different issues.

One thing I would question in that table is "applied immediately (but
can break hard when dynamic-deps stop working))."  How can dynamically
removing an "unused dependency" cause something to break, setting
aside bugs in the package manager?  If removing a dependency causes
something to break, how can it be "unused?"

Rich

Re: [gentoo-dev] don't rely on dynamic deps

["Paweł Hajdan, Jr."]

[-- Attachment #1: Type: text/plain, Size: 924 bytes --]

On 7/27/14, 4:42 PM, Rich Freeman wrote:
> With dynamic deps you'd need to revbump if there is a linking change.
> Otherwise portage would just allow the dependency to be removed, and
> then linking will break, since the executable is unnecessarily linked
> to the dependency (in that scenario).

Right, I see - I think I got that right when first reading this, but got
confused after reading so many messages in this thread. Thanks for
patient explanation.

It seems really tricky to correctly reason about dependency resolution.

> One thing I would question in that table is "applied immediately (but
> can break hard when dynamic-deps stop working))."  How can dynamically
> removing an "unused dependency" cause something to break, setting
> aside bugs in the package manager?  If removing a dependency causes
> something to break, how can it be "unused?"

Yeah, I was also wondering about this.



[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 841 bytes --]

Re: [gentoo-dev] don't rely on dynamic deps

[Ciaran McCreesh]

[-- Attachment #1: Type: text/plain, Size: 353 bytes --]

On Sun, 27 Jul 2014 16:56:17 +0200
""Paweł Hajdan, Jr."" <phajdan.jr@gentoo.org> wrote:
> It seems really tricky to correctly reason about dependency
> resolution.

It's actually very easy if you do away with all the things that are
making it unnecessarily complicated... Nearly all of the complexity is
self-inflicted.

-- 
Ciaran McCreesh

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 181 bytes --]

Re: [gentoo-dev] don't rely on dynamic deps

[Rich Freeman]

On Sun, Jul 27, 2014 at 10:59 AM, Ciaran McCreesh
<ciaran.mccreesh@googlemail.com> wrote:
> On Sun, 27 Jul 2014 16:56:17 +0200
> ""Paweł Hajdan, Jr."" <phajdan.jr@gentoo.org> wrote:
>> It seems really tricky to correctly reason about dependency
>> resolution.
>
> It's actually very easy if you do away with all the things that are
> making it unnecessarily complicated... Nearly all of the complexity is
> self-inflicted.

What would you do away with?  Being able to virtualize packages
without recompiling everything that depends on them?

I do appreciate your argument, but at the same time for every complex
problem there is an answer that is clear, simple, and wrong.

There are a lot of things in Gentoo that could be done in a simpler
fashion, and 10 years ago Gentoo was a lot simpler than it is today.
The thing is, all that complexity was added for a reason.  I'm all for
refactoring, but we need to be careful to not toss the baby with the
bathwater if it is at all possible.

It might be an interesting exercise if we could take something like
kde-meta+firefox+openoffice on the desktop/kde profile (or gnome if
you wish) and determine just how many rebuilds would have resulted in
the last six months if all the changes we're talking about actually
involved revbumps, and how much cpu time that would take on an
"average" system (I don't care what but distinguishing
firefox/openoffice/qt/kdelibs from some package with 1k lines of code
is useful).

Rich

Re: [gentoo-dev] don't rely on dynamic deps

[Ciaran McCreesh]

[-- Attachment #1: Type: text/plain, Size: 1537 bytes --]

On Sun, 27 Jul 2014 11:09:05 -0400
Rich Freeman <rich0@gentoo.org> wrote:
> On Sun, Jul 27, 2014 at 10:59 AM, Ciaran McCreesh
> <ciaran.mccreesh@googlemail.com> wrote:
> > On Sun, 27 Jul 2014 16:56:17 +0200
> > ""Paweł Hajdan, Jr."" <phajdan.jr@gentoo.org> wrote:
> >> It seems really tricky to correctly reason about dependency
> >> resolution.
> >
> > It's actually very easy if you do away with all the things that are
> > making it unnecessarily complicated... Nearly all of the complexity
> > is self-inflicted.
> 
> What would you do away with?  Being able to virtualize packages
> without recompiling everything that depends on them?

Well that's never worked properly or consistently to begin with, so all
we'd be doing away with is the pretence that we can get away with it.

> I do appreciate your argument, but at the same time for every complex
> problem there is an answer that is clear, simple, and wrong.

Unfortunately that's the answer Gentoo usually goes with.

> There are a lot of things in Gentoo that could be done in a simpler
> fashion, and 10 years ago Gentoo was a lot simpler than it is today.
> The thing is, all that complexity was added for a reason.

Most of that complexity was added due to "not thinking things through
fully" and "adding in a short-term hack with long-term consequences".
The reason was rarely "we need this complexity", and usually "we need
something, and on the surface of it this looks like it solves exactly
that something".

-- 
Ciaran McCreesh

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 181 bytes --]

Re: [gentoo-dev] don't rely on dynamic deps

[Peter Stuge]

[-- Attachment #1: Type: text/plain, Size: 308 bytes --]

Ciaran McCreesh wrote:
> Rich Freeman <rich0@gentoo.org> wrote:
> > What would you do away with?  Being able to virtualize packages
> > without recompiling everything that depends on them?
> 
> Well that's never worked properly or consistently to begin with

Please answer the question?


//Peter

[-- Attachment #2: Type: application/pgp-signature, Size: 190 bytes --]

Re: [gentoo-dev] don't rely on dynamic deps

[Ciaran McCreesh]

[-- Attachment #1: Type: text/plain, Size: 712 bytes --]

On Sun, 27 Jul 2014 19:02:05 +0200
Peter Stuge <peter@stuge.se> wrote:
> Ciaran McCreesh wrote:
> > Rich Freeman <rich0@gentoo.org> wrote:
> > > What would you do away with?  Being able to virtualize packages
> > > without recompiling everything that depends on them?
> > 
> > Well that's never worked properly or consistently to begin with
> 
> Please answer the question?

That does answer the question: as can be seen from the udev virtual
example earlier in this thread, we do not have a way of reliably
avoiding recompilations. Thus it is wrong to pretend that we have this
feature now, or that fully static dependencies would somehow be to blame
for recompilations.

-- 
Ciaran McCreesh

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 181 bytes --]

Re: [gentoo-dev] don't rely on dynamic deps

[Peter Stuge]

[-- Attachment #1: Type: text/plain, Size: 589 bytes --]

Ciaran McCreesh wrote:
> Peter Stuge <peter@stuge.se> wrote:
> > Ciaran McCreesh wrote:
> > > Rich Freeman <rich0@gentoo.org> wrote:
> > > > What would you do away with?  Being able to virtualize packages
> > > > without recompiling everything that depends on them?
> > > 
> > > Well that's never worked properly or consistently to begin with
> > 
> > Please answer the question?
> 
> That does answer the question

It doesn't. The question is "What would you do away with?" and you
didn't list the things you would do away with.

Please answer the question?


//Peter

[-- Attachment #2: Type: application/pgp-signature, Size: 190 bytes --]

Re: [gentoo-dev] don't rely on dynamic deps

[Ciaran McCreesh]

[-- Attachment #1: Type: text/plain, Size: 858 bytes --]

On Sun, 27 Jul 2014 19:16:58 +0200
Peter Stuge <peter@stuge.se> wrote:
> Ciaran McCreesh wrote:
> > Peter Stuge <peter@stuge.se> wrote:
> > > Ciaran McCreesh wrote:
> > > > Rich Freeman <rich0@gentoo.org> wrote:
> > > > > What would you do away with?  Being able to virtualize
> > > > > packages without recompiling everything that depends on them?
> > > > 
> > > > Well that's never worked properly or consistently to begin with
> > > 
> > > Please answer the question?
> > 
> > That does answer the question
> 
> It doesn't. The question is "What would you do away with?" and you
> didn't list the things you would do away with.
> 
> Please answer the question?

Dynamic dependencies, || dependencies (in favour of a heavily
restricted, non-abusable alternative) and REQUIRED_USE would be an
excellent start.

-- 
Ciaran McCreesh

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 181 bytes --]

[gentoo-dev] Re: don't rely on dynamic deps

[Duncan]

Paweł Hajdan, Jr. posted on Sun, 27 Jul 2014 16:56:17 +0200 as excerpted:

>> One thing I would question in that table is "applied immediately (but
>> can break hard when dynamic-deps stop working))."  How can dynamically
>> removing an "unused dependency" cause something to break, setting aside
>> bugs in the package manager?  If removing a dependency causes something
>> to break, how can it be "unused?"
> 
> Yeah, I was also wondering about this.


I believe what is being discussed there is the following (as I understand 
it):

1) Foo incorrectly deps on bar, and doesn't actually use that dep for 
anything.  Say it was a historical dep that upstream forgot to remove in 
the docs when they removed the dep, and the (gentoo) maintainer didn't 
detect it initially.

2) User installs foo with the incorrect dep, thus pulling in bar.

At this point use has an unnecessary package installed but is otherwise 
fine, deps are wrongly pulling it in.

3) Foo maintainer detects the error (upstream removed the dep in the 
unreleased version and maintainer realizes it wasn't needed in the 
current version either) and removes the dep on bar, but since it was 
unused anyway and thus doesn't affect the package as installed (except in 
vardb), he takes advantage of dynamic-deps and no revbump is done.

At this point, user has an unnecessary dep that portage has just been 
told about due to dynamic deps, and will unmerge it at the next depclean, 
but the user's dep-tree is still self-consistent and everything's fine.

4) Due to dynamic-deps, portage depcleans bar.

Due to dynamic-deps, the user's dep-tree is still consistent and 
everything's fine.

5) Since nothing in the tree needs bar, it is removed.

Due to dynamic-deps, the user's dep-tree is still consistent, because the 
in-/usr/portage foo.ebuild doesn't say anything about bar since #3.

6) Finally, foo is removed from tree.

** User isn't ready to get rid of foo yet, but user's dep-tree just blew 
up, because now portage falls back to the static vardb from original 
install, and that still says foo deps on bar, which is nowhere to be 
found!

What's worse, the usual trick of digging the ebuild out of vardb and 
putting it in the user's overlay to shut portage up and let the user keep 
foo doesn't work, because the vardb ebuild has an unsatisfied dep.

With the repeated caveat that this is as I understand it, if the above is 
clear along with the implications, you can stop reading here.  The below 
simply expands on the above and its implications...

The "breaks hard" bit can be applied for two reasons.  First, exactly as 
I said above, the user's now SOL -- they had no warning things were going 
to break and now it's too late to do anything about it (unless they're 
advanced enough to figure out how to dig the last, corrected ebuild 
version out of a snapshot or the cvs attic, or to figure out that dep 
isn't actually needed on their own, and edit the ebuild they pulled out 
of vardb accordingly).

Second, "breaks hard" can apply when a whole set of related packages were 
removed at once, all with the same bad dep.  Consider for instance 
someone with kde-4.11 installed and how deeply dependent typical kde 
ebuilds tend to be on the kde eclasses.  If the bad dep was in an eclass 
and applied to say 200+ kde packages the user may well have installed, 
then got corrected such that dynamic-deps killed the bad dep, then all of 
kde-4.11 was removed from the tree at once...  A user may well find out 
they have 200+ broken deps at once!  "Breaks hard", indeed!


Now we look at the same set of triggering events with static deps.  There 
are two cases, depending on whether the foo maintainer accounted for 
static deps and did a revision bump at step 3 or not:

With a revision bump at step #3, at step #4, before the user can do the 
depclean, they'll have to do the revision upgrade.  The aftermath of step 
#6 will then be hugely different as the vardb copy will have the bad dep 
removed as well.

If the foo maintainer didn't do a revision bump, with static deps the 
user will have never depcleaned bar in step #4 as the vardb deps still 
say it's needed.  So when foo is removed in step #6, again, no big deal.  
Sure the user had a useless dep installed the whole time, but when foo is 
removed from tree, fine, the user's dep-tree remains self-consistent and 
no blowup.  And the user remains free to dig that ebuild out of vardb and 
install it in the overlay again, at least temporarily, to keep portage 
from yelling about it until the user has time to find an alternative to 
the package, or to upgrade to the new version (kde 4.12 or 4.13 in my 
mass-breakage example).  Reinstalling may or may not be possible 
depending on how much the ebuild depended on eclasses and what had 
happened to them, but keeping the same thing installed at least 
temporarily should be fine.

And the static-deps case of the user not upgrading in a timely manner and 
thus skipping step #4 entirely, resolves exactly as if the maintainer 
hadn't done a bump.  Useless dep installed the whole time but no breakage.

To complete the alternatives-tree, the no-window-update scenario with 
dynamic-deps would fall back to static/installed when the user updated 
after both packages were removed, and if an eclass wasn't involved, 
pulling the ebuilds out of vardb to stick in the overlay temporarily 
should still shut up portage, and the user could even rebuild from his 
overlay copy.  But if an eclass was involved with that dep, however, 
pulling those ebuilds out of vardb and into the user overlay would 
trigger dynamic deps again, and what would happen then would depend on 
exactly how the eclass had changed.  If the eclass had simply removed the 
bad dep then the effect would be extending the correction window, since 
with dynamic-deps that new information would now be used, allowing bar to 
be unmerged even with foo still merged.  But if the eclasses had been 
updated to remove the old logic entirely (in the kde case, if the eclasses 
had their 4.11 logic removed already), then while portage would fall-back 
to the static deps as long as the ebuild wasn't available, as soon as it 
became available again, all hell would break loose as portage would see 
the ebuild in the overlay and try to use dynamic deps, breaking due to 
the eclasses no longer having the required support at all!

Just in case someone missed the disclaimer above, that's as I understand 
it.  I /think/ I get it, but I don't claim to be a PM/PMS expert and it's 
possible I have it wrong.

-- 
Duncan - List replies preferred.   No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master."  Richard Stallman

[gentoo-dev] Re: don't rely on dynamic deps

[Martin Vaeth]

Duncan <1i5t5.duncan@cox.net> wrote:
>
> 1) Foo incorrectly deps on bar
> 2) User installs foo with the incorrect dep
> 3) Foo maintainer detects the error
> 4) Due to dynamic-deps, portage depcleans bar.
> 5) Since nothing in the tree needs bar, it is removed.
> 6) Finally, foo is removed from tree.
>
> ** User isn't ready to get rid of foo yet, but user's dep-tree just blew
> up

This is the problem of orphaned (without maintained ebuilds) packages:
Both, dynamic and static deps break on the similar situation for
different reasons.

The above describes one problem which you get in this case with
dynamic deps; to get a similar problem with static deps, you need
a slightly different situation:

1-3 as above, 4 is skipped (due to static deps).
5) bar is split into bar-base and bar-gui.
6) as above (foo is removed from the tree).

Now the static deps will keep in bar forever, omitting that the
packages in the tree can install bar-base or bar-gui.
The user's dep-tree blows up through blockers or merge conflict
which cannot be resolved.

I repeat once more: For packages no longer maintained in the tree,
there is no secure way to treat them always correctly;
"no longer maintained" means exactly this.

Either the user should remove them, or he should take full
responsibility by maintaining them by himself in his overlay.
Choosing a strategy (dynamic or static deps) should not concentrate
on users who refuse to choose this only reasonable solution -
as excpected, they get a solution which works sometimes, but they
have to keep the pieces when it breaks, no matter which strategy
is decided.

> What's worse, the usual trick of digging the ebuild out of vardb and
> putting it in the user's overlay to shut portage up and let the user keep
> foo doesn't work, because the vardb ebuild has an unsatisfied dep.

This is not so bad: The user has to put a corrected ebuild into his
overlay and must reemerge the package (currently, the latter could
be skipped with dynamic deps).
In fact, no matter whether you have static or dynamic deps, this is
the only way to cleanly avoid the problems if you want to keep a
package installed which is not maintained anymore:
*You* must maintain it. There simply is no magic which can avoid this.

Re: [gentoo-dev] Re: don't rely on dynamic deps

[Peter Stuge]

Martin Vaeth wrote:
> The user has to put a corrected ebuild into his overlay and must
> reemerge the package (currently, the latter could be skipped with
> dynamic deps).
> In fact, no matter whether you have static or dynamic deps, this is
> the only way to cleanly avoid the problems if you want to keep a
> package installed which is not maintained anymore:
> *You* must maintain it. There simply is no magic which can avoid this.

It could be avoided if portage tree sync applied each tree change
locally rather than jump from old to new. There was a delta idea
discussed a year or so ago in connection with some git discussions.

The user's vardb could then automatically receive the last state of
the ebuild, before it was removed.

That's no small change though.


//Peter

[gentoo-dev] Re: don't rely on dynamic deps

[Martin Vaeth]

Peter Stuge <peter@stuge.se> wrote:
> Martin Vaeth wrote:
>> In fact, no matter whether you have static or dynamic deps, this is
>> the only way to cleanly avoid the problems if you want to keep a
>> package installed which is not maintained anymore:
>> *You* must maintain it. There simply is no magic which can avoid this.
>
> It could be avoided if portage tree sync applied each tree change
> locally rather than jump from old to new. [...]
> The user's vardb could then automatically receive the last state of
> the ebuild, before it was removed.

It doesn't help reliably, either, since the last state of the ebuild,
before it was removed, will be outdated at some point, too.
It *is* necessary to adapt the dependencies to the current tree,
and if no maintainer is there to do this correctly for that package,
all empirics to "push" such changes will fail in some situations.

Re: [gentoo-dev] Re: don't rely on dynamic deps

[Peter Stuge]

Martin Vaeth wrote:
> > The user's vardb could then automatically receive the last state of
> > the ebuild, before it was removed.
> 
> It doesn't help reliably, either, since the last state of the ebuild,
> before it was removed, will be outdated at some point, too.

Sorry, I don't see how. Can you give an example? Thanks!


//Peter

[gentoo-dev] Re: don't rely on dynamic deps

[Martin Vaeth]

Peter Stuge <peter@stuge.se> wrote:
> Martin Vaeth wrote:
>> > The user's vardb could then automatically receive the last state of
>> > the ebuild, before it was removed.
>>
>> It doesn't help reliably, either, since the last state of the ebuild,
>> before it was removed, will be outdated at some point, too.
>
> Sorry, I don't see how. Can you give an example? Thanks!

1. foo depends on bar
2. user installs foo
3. foo is treecleaned
4. bar ebuild is replaced by the pair bar-base and bar-gui to
   allow for finer dependency. All ebuilds in the tree take
   care about this new structure (possibly with revbumps).
   Of course, the dependency for an already removed package
   is not fixed.
5. bar{-base,gui} gets several upgrades.
6. foo on user's system still blocks bar from being removed
   and thus blocks the installation of bar-base and bar-gui
   forever.
   Alternatively (if no conflict arises), foo depends forever on an
   ancient (hence possibly full of security bugs) version of bar
   which should have been upgraded ages ago.

In both cases of 6., the user is not even aware that he uses
long obsolete packages unless portage prints a big fat warning
for orphaned packages (which currently is not the case.
Well, at least eix -t will be print a message.)

Note that 4. cannot be replaced by any "pushing" mechanism,
since only the maintainer of the ebuild can know which is
the "right" dependency for the new tree structure. Such a
maintainer does not exist for treecleaned packages (which
is the purpose of treecleaning, after all...)

Re: [gentoo-dev] Re: don't rely on dynamic deps

[Rich Freeman]

On Mon, Jul 28, 2014 at 2:50 PM, Martin Vaeth <martin@mvath.de> wrote:
>
> In both cases of 6., the user is not even aware that he uses
> long obsolete packages unless portage prints a big fat warning
> for orphaned packages (which currently is not the case.
> Well, at least eix -t will be print a message.)
>

This is really the crux of these sorts of issues.  It doesn't matter
if dependencies are static or dynamic - if you hang onto orphans then
you're going to have cruft in your vdb which is going to lead to
blockers of some kind eventually.

Portage should probably generate a warning when there are orphan packages.

The same is true if you keep cruft in a local overlay or such.  We can
have all the pretty virtuals/etc we want, but if users stick
hard-coded obsolete package names in their overlays or have them in
their vdb, then they're going to get blockers.  Though, we could do a
better job with the error messages even when that happens...

Rich

Re: [gentoo-dev] Re: don't rely on dynamic deps

[Peter Stuge]

Martin Vaeth wrote:
> >> > The user's vardb could then automatically receive the last state of
> >> > the ebuild, before it was removed.
> >>
> >> It doesn't help reliably, either, since the last state of the ebuild,
> >> before it was removed, will be outdated at some point, too.
> >
> > Sorry, I don't see how. Can you give an example? Thanks!
> 
> 1. foo depends on bar
> 2. user installs foo
> 3. foo is treecleaned
> 4. bar ebuild is replaced by the pair bar-base and bar-gui to
>    allow for finer dependency. All ebuilds in the tree take
>    care about this new structure (possibly with revbumps).
>    Of course, the dependency for an already removed package
>    is not fixed.
> 5. bar{-base,gui} gets several upgrades.
> 6. foo on user's system still blocks bar from being removed
>    and thus blocks the installation of bar-base and bar-gui
>    forever.

Thanks for spelling it out!

This suggests another data model bug to me: that dependencies belong
to the dependent packages, rather than to dependency providers.

What I mean is that in the above example, bar "knows" that bar has
turned into bar-{base,gui}, but foo doesn't.


> Note that 4. cannot be replaced by any "pushing" mechanism,
> since only the maintainer of the ebuild can know which is
> the "right" dependency for the new tree structure. Such a
> maintainer does not exist for treecleaned packages (which
> is the purpose of treecleaning, after all...)

If the user updates their package database things would still work
if 4 modifies the effective dependencies for foo to reflect the new
reality of bar -> bar-{base,gui}.

foo is not being maintained, but bar is. It might be incorrect to say
that foo depends on both bar-base and -gui (foo might just need -base)
but it is perfectly safe to automatically make the most pessimistic
assumption when upgrading the outdated dependency on bar in all
installed-but-outdated-ebuilds.

The code required for this would even allow to partially automate
dependency changes like bar -> bar-{base,gui} across the tree.
Maintainers could get notified when a package they depend on change,
and the safe default is to just roll along. Less dev work! \o/


The more I think about dependencies the more convinced I am that
dependency information must be versioned, ie. dependencies only
matter in the context of the particular package database snapshot
they came from, and that installed dependencies must be kept up to
date as the package database evolves.

Otherwise there's just a pile of heuristics, which throw people,
which I guess is why dynamic-deps cause problems and ire.


Rich Freeman wrote:
> This is really the crux of these sorts of issues.  It doesn't matter
> if dependencies are static or dynamic - if you hang onto orphans then
> you're going to have cruft in your vdb which is going to lead to
> blockers of some kind eventually.

I think the vdb can and should be updated according to portage changes.

Someone just needs to code it. ;)


//Peter

Re: [gentoo-dev] Re: don't rely on dynamic deps

[Kent Fredric]

[-- Attachment #1: Type: text/plain, Size: 353 bytes --]

On 29 July 2014 19:33, Peter Stuge <peter@stuge.se> wrote:

> I think the vdb can and should be updated according to portage changes.
>
> Someone just needs to code it. ;)
>

And an appropriate method for doing this must be decided upon.

And that part entails >70% of the discussion dispute :)


-- 
Kent

*KENTNL* - https://metacpan.org/author/KENTNL

[-- Attachment #2: Type: text/html, Size: 1069 bytes --]

[gentoo-dev] Re: don't rely on dynamic deps

[Martin Vaeth]

Peter Stuge <peter@stuge.se> wrote:
>
> This suggests another data model bug to me: that dependencies belong
> to the dependent packages, rather than to dependency providers.

Sometimes it is this way, sometimes the opposite.
It would not be natural to associate to a huge library like openssl
the packages which use it (or can make use of it if compiled with
certain ./configure options).

> What I mean is that in the above example, bar "knows" that bar has
> turned into bar-{base,gui}, but foo doesn't.

But this is not always so.
I could also have given the example that bar is replaced by
another library which perhaps is (mainly) API but not ABI compatible
but nevertheless provides the same command line tools.
(Perhaps openssl and some of the planned successors are such an
example?)
So the installations of bar and bar-successor must block each other,
but nevertheless package foo would need to be recompiled if you want
to use the modern (presumably "safer") bar-successor - which cannot
be done for foo.
Again, in this situation the user relies on the outdated bar and
does not know it. Or - if something else needs bar-successor -
the user gets blockers.

> If the user updates their package database things would still work
> if 4 modifies the effective dependencies for foo to reflect the new
> reality of bar -> bar-{base,gui}.

I would be very careful with such magic modifications:
It is always dangerous if you rely on a heuristic which needs to
be "smarter" than the developer; no matter how many cases you
cover in the heuristic, sooner or later there will be some
split/replacement/alternative/merge of packages which cannot be treated
by the cases covered through the heuristics and which needs
some form of manual treatment in some cases.

> The more I think about dependencies the more convinced I am that
> dependency information must be versioned, ie. dependencies only
> matter in the context of the particular package database snapshot
> they came from, and that installed dependencies must be kept up to
> date as the package database evolves.

Yes, this is exactly the problem with static dependencies.
Regular revbumps *might* be a solution to this, but there is a
certain danger that due to certain (e.g. circular) static dependencies
blocking updates on certain user's systems the revbumps remain
"invisible" to these users.
This is probably not very likely, but I think I could construct
realistic such cases (with a "bug" in a dependency, such cases
are very easy to construct: Some =... instead of ~... is sufficient).

> Otherwise there's just a pile of heuristics, which throw people,
> which I guess is why dynamic-deps cause problems and ire.

This is exactly the main problem with dynamic dependencies.

To summarize a bit differently:

The issue is caused by gentoo being a rolling release distribution.
This causes permanent tree restructuring, and the latter is
incompatible with the idea that a package, once installed,
should remain working, unconditionally on the tree changes.

Users may expect and wish for the latter, but fact is that these
two requirements are in some situations incompatible with each other.

If you want a completely safe solution which in no special cases
can ever cause any problems, after every sync you would have to
unmerge every package and then remerge it again (resolving circular
deps by temporary removing of USE-flags and recompiling).

The other extreme are dynamic deps without any revbumps ever
which take only the rolling release character into account.
You have immediately the most current tree information,
but in some cases this contradicts the actual installed packages.

Now static-deps with many revbump and dynamic-deps with only a
revbumps for essential changes are two compromises between these
two extremes.
Nothing is safe *and* up-to-date in all cases, but both work
up to a reasonable number of exceptions:
The static-deps are on the "safe" solution side with more recompilations
but still possibly outdated information, the dynamic-deps are on the
rolling release side, requiring the user to take more responsibility
if he wants to avoid a broken system.

The highly emotional reactions on the topic are probably due to the
fact that corresponding users have systems which for some reason or
another should emphasize more the static or "rolling release" type
of gentoo: Shifting this emphasis is a politicial issue and
fundamental decision, of course.

Re: [gentoo-dev] Re: don't rely on dynamic deps

[Rich Freeman]

On Tue, Jul 29, 2014 at 3:33 AM, Peter Stuge <peter@stuge.se> wrote:
>
> Rich Freeman wrote:
>> This is really the crux of these sorts of issues.  It doesn't matter
>> if dependencies are static or dynamic - if you hang onto orphans then
>> you're going to have cruft in your vdb which is going to lead to
>> blockers of some kind eventually.
>
> I think the vdb can and should be updated according to portage changes.
>
> Someone just needs to code it. ;)

So, I'll agree that vdb should change when portage changes (and we
should manage portage changes so that this can be done reliably), but
we're talking about orphans here.  Portage is only going to get one
side of the story when dealing with an orphan.

In your example of a package split the original package went away, and
perhaps with some mechanism we could get portage to update all former
dependencies to use both sides of the split.

But, how about virtualization of a package?  Your orphan depends on
non-virtual udev, but now you want to install systemd which of course
blocks udev.  Maybe your package really does depend on the "real" udev
(probably not a good example here - think ffmpeg instead perhaps), or
maybe it can use the virtual.  Just telling portage that the virtual
replacement has been made is one problem, but figuring out whether to
use it is going to be a wild guess for a PM.

And there are likely other variations as well.

Rich

Re: [gentoo-dev] don't rely on dynamic deps

[Kent Fredric]

[-- Attachment #1: Type: text/plain, Size: 1845 bytes --]

On 28 July 2014 02:42, Rich Freeman <rich0@gentoo.org> wrote:

> One thing I would question in that table is "applied immediately (but
> can break hard when dynamic-deps stop working))."  How can dynamically
> removing an "unused dependency" cause something to break, setting
> aside bugs in the package manager?  If removing a dependency causes
> something to break, how can it be "unused?"
>

My apologies if this scenario has been explained before, I saw things along
these lines, but must may have missed their point:

I get the impression that this happens:

1. User installs Foo
2. Gentoo needs to change what Foo depends on
3. Gentoo simply tweaks the ebuild and doesn't bump [A]
4. User syncs.
5. Subsequent emerges see dependencies from [A] which are fixed and works
in the interim
6. A gets removed.
7. User syncs.
8. Shadowing effect of [A] is removed, and Foo is now back depending on the
wrong thing.

=~ So although this problem will be resolved by also removing Foo, Foo
still exists on the system with bad dependencies which could lead to some
kind of problem, preventing emerge resolution from making sense.

Sure, Foo will get reaped by a depclean if it is no longer in world and
nothing depends on it, .... but depclean tends to require world to be
deeply resolved first.



And more questions for the -r1.1 style "Ship a new ebuild" style proposal.

::gentoo ships Foo-1.0
::whatever *also* ships Foo-1.0 ( with different dependencies )

User installs from ::gentoo

::Whatever publishes a -r0.1 style ( or equivalent) dependency fix

What happens for the user, and why.

I'm of the mind ::Whatever should not tamper with ::gentoo here, though it
could be useful, it could also prove problematic.

Or maybe the idea is just too weird and quirky to implement this way.

-- 
Kent

*KENTNL* - https://metacpan.org/author/KENTNL

[-- Attachment #2: Type: text/html, Size: 3354 bytes --]

Re: [gentoo-dev] don't rely on dynamic deps

[Rich Freeman]

On Sun, Jul 27, 2014 at 11:44 AM, Kent Fredric <kentfredric@gmail.com> wrote:
>
> On 28 July 2014 02:42, Rich Freeman <rich0@gentoo.org> wrote:
>>
>> One thing I would question in that table is "applied immediately (but
>> can break hard when dynamic-deps stop working))."  How can dynamically
>> removing an "unused dependency" cause something to break, setting
>> aside bugs in the package manager?  If removing a dependency causes
>> something to break, how can it be "unused?"
>
>
> My apologies if this scenario has been explained before, I saw things along
> these lines, but must may have missed their point:
>
> I get the impression that this happens:
>
> 1. User installs Foo
> 2. Gentoo needs to change what Foo depends on

Why?  Is this about removing an unused dependency?

> 3. Gentoo simply tweaks the ebuild and doesn't bump [A]

What is "[A]?"  What ebuild was tweaked, and how was it tweaked?

> 8. Shadowing effect of [A] is removed, and Foo is now back depending on the
> wrong thing.

What do you mean by "shadowing effect?"

You need to be a bit more clear on your scenario here.  I'm not really
getting much out of your example.  My question was how can removing an
unused dependency break things.  I'm not sure if your example includes
an unused dependency, whether it was removed, and whether it breaks
anything.

Rich

Re: [gentoo-dev] don't rely on dynamic deps

[Kent Fredric]

[-- Attachment #1: Type: text/plain, Size: 1706 bytes --]

On 28 July 2014 03:52, Rich Freeman <rich0@gentoo.org> wrote:

> Why?  Is this about removing an unused dependency?
>
> > 3. Gentoo simply tweaks the ebuild and doesn't bump [A]
>
> What is "[A]?"  What ebuild was tweaked, and how was it tweaked?
>

Here, "A" is the derived version of the ebuild of "Foo" the user installed,
in the present dynamic deps situation.

That is, Although user and tree have the same $PVR value, one has different
DEPENDS, and my impression based on past discussions is in that scenario,
the DEPENDS from /usr/portage/  take preceidence over VDB.

Though perhaps it depends on your definition of "used". If your definition
of "used" is "other things in portage needs it", then this case is
satisfied by it being "used" by the world file.

Users will likely know in such a scenario that they are "using" a thing
that is no longer in tree, but if it no longer being "in tree" changes how
its dependencies resolve for the worse.

*Most* examples of this you will see will be resolved by pkgmoves munging
the installed dependencies in VDB to cease to be broken.

I guess, in a way, pkgmove and dynamic deps are the same problem from
different perspectives.

pkgmove is an instruction that "set( X ) that requires Y needs to change"

and dynamic deps are "X  requires set ( Y ) and needs to change."

Just in the case of the former, the effect is made permanent as part of the
sync, and in the case of the latter ( based on my understanding ) the
effect is temporary ( bounded by the lifetime of the ebuild in /usr/portage
) and resolved at runtime for each and every merge ....

( Do correct me if I'm wrong about this )





-- 
Kent

*KENTNL* - https://metacpan.org/author/KENTNL

[-- Attachment #2: Type: text/html, Size: 2955 bytes --]

[gentoo-dev] Re: don't rely on dynamic deps

[Martin Vaeth]

Kent Fredric <kentfredric@gmail.com> wrote:
> 1. User installs Foo
> 2. Gentoo needs to change what Foo depends on
> 3. Gentoo simply tweaks the ebuild and doesn't bump [A]
> 4. User syncs.
> 5. Subsequent emerges see dependencies from [A] which are fixed and works
>    in the interim
> 6. A gets removed.
> 7. User syncs.
> 8. Shadowing effect of [A] is removed, and Foo is now back
>    depending on the wrong thing.

If you have 6., you have the same szenario even if Gentoo bumps [A],
if 4. is omitted (one must not rely on the user syncing very regularly).
Even much worse:

> I guess, in a way, pkgmove and dynamic deps are the same problem from
> different perspectives.

s/dynamic deps/changes in dependencies due to tree restructuring/

In a sense, it is. But even if you would use the space-consuming
variant of using some history:
If the restructuring of the tree happens *after* the package is removed
(that is, if 6. happens before the *cause* for 3.) you end up in any
case with a situation which can be broken in many aspects.

Orphant installs (without a corresponding ebuild) is a situation
which is impossible to handle "correctly".
The user must *strongly* be adviced ("forced" if gentoo would not
always leave choice) to remove the package unless he maintains the
corresponding ebuild in his own overlay.

Since there is no satisfactory solution for such orphant installs,
they should be kept out of the discussion.

> pkgmove is an instruction that "set( X ) that requires Y needs to change"
>
> and dynamic deps are "X  requires set ( Y ) and needs to change."

s/dynamic deps/changes in dependencies due to tree restructuring/

The difference is that pkgmove can be done generically for all
packages while more complicated tree restructuring is impossible
to do generically. For instance, for a package split the maintainer
of the ebuild has to know which are the correct dependencies;
a package merge must usually be combined with appropriate USE-deps, etc.

Re: [gentoo-dev] don't rely on dynamic deps

[Michał Górny]

[-- Attachment #1: Type: text/plain, Size: 1611 bytes --]

Dnia 2014-07-27, o godz. 10:42:19
Rich Freeman <rich0@gentoo.org> napisał(a):

> On Sun, Jul 27, 2014 at 10:05 AM, "Paweł Hajdan, Jr."
> <phajdan.jr@gentoo.org> wrote:
> > On 7/21/14, 11:52 PM, Alexander Berntsen wrote:
> >> Michał has documented the shortcomings of dynamic deps in our wiki[0].
> >> (Thank you!) [...]
> >> [0]  <https://wiki.gentoo.org/wiki/Project:Portage/Dynamic_dependencies>
> >
> > There's one more thing I'd like to ask about:
> >
> > For "Minor linking change w/ dependency change (unnecessary linking
> > removed)" the "dynamic deps" cell is red with "revbump + mostly
> > unnecessary rebuild", and "static deps" says "applied after rebuild".
> >
> > Arguably with dynamic deps one could also skip the revbump, and the
> > update would similarly be applied after rebuild.
> 
> One thing I would question in that table is "applied immediately (but
> can break hard when dynamic-deps stop working))."  How can dynamically
> removing an "unused dependency" cause something to break, setting
> aside bugs in the package manager?  If removing a dependency causes
> something to break, how can it be "unused?"

Consider the following:

1. A depends on B, both are installed,

2. dependency on B is removed, emerge --depclean uninstalls B thanks
to dynamic-deps,

3. B is treecleaned (nothing depends on it),

4. old version of A is removed (user doesn't update it yet), therefore
dependency on B is restored from vdb.

So, now user has package A installed which has unsatisfied dependency
on non-available package.

-- 
Best regards,
Michał Górny

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 949 bytes --]

Re: [gentoo-dev] don't rely on dynamic deps

[Peter Stuge]

[-- Attachment #1: Type: text/plain, Size: 985 bytes --]

Michał Górny wrote:
> Consider the following:
> 
> 1. A depends on B, both are installed,
> 
> 2. dependency on B is removed, emerge --depclean uninstalls B thanks
> to dynamic-deps,
> 
> 3. B is treecleaned (nothing depends on it),

So far I follow.


> 4. old version of A is removed (user doesn't update it yet),

ebuild removed from tree.


> therefore dependency on B is restored from vdb.

Why would removing the ebuild from the tree *add* a dependency to a
system where the ebuild is installed? That doesn't seem desirable.

To me it seems like a simple data model bug that vdb does not get
updated to reflect the new situation after step 2 above.


> So, now user has package A installed which has unsatisfied
> dependency on non-available package.

What is the purpose of keeping only dependencies as-they-were when
the package was installed, if the package manager does not somehow
benefit from that information in the future?


//Peter

[-- Attachment #2: Type: application/pgp-signature, Size: 190 bytes --]

Re: [gentoo-dev] don't rely on dynamic deps

[Ciaran McCreesh]

[-- Attachment #1: Type: text/plain, Size: 298 bytes --]

On Sun, 27 Jul 2014 22:51:13 +0200
Peter Stuge <peter@stuge.se> wrote:
> To me it seems like a simple data model bug that vdb does not get
> updated to reflect the new situation after step 2 above.

Rewriting VDB won't help if the user doesn't sync at the right time.

-- 
Ciaran McCreesh

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 181 bytes --]

Re: [gentoo-dev] don't rely on dynamic deps

[Kent Fredric]

[-- Attachment #1: Type: text/plain, Size: 1183 bytes --]

On 28 July 2014 08:56, Ciaran McCreesh <ciaran.mccreesh@googlemail.com>
wrote:

> > To me it seems like a simple data model bug that vdb does not get
> > updated to reflect the new situation after step 2 above.
>
> Rewriting VDB won't help if the user doesn't sync at the right time.
>

Indeed. pkgmove has this problem solved with a mass of quarterly move
files, but I'm not sure I'd want to have

a)  However many `depchange` entries required to make it happen linger on
for all eternity in some cruft file just in case people don't sync more
than once every 2 years:  ( Yes, we still have an updates/1Q-2009 file for
people stuck in a time warp who need change updates )

b) The burden of maintainers having to manually update that index. ( That's
effectively what the -r1.1 and INSTALL_FROM proposals amount to )

The only saving grace here if we applied this strategy, is we could
conceivably generate the index in an automated fashion due to ebuild edits
usually being more obvious to an SCM than a move is.  ( And you could
conceivably generate them by simply comparing snapshot diffs without any
SCM involvement )





-- 
Kent

*KENTNL* - https://metacpan.org/author/KENTNL

[-- Attachment #2: Type: text/html, Size: 2074 bytes --]

Re: [gentoo-dev] don't rely on dynamic deps

[Rich Freeman]

On Sun, Jul 27, 2014 at 5:27 PM, Kent Fredric <kentfredric@gmail.com> wrote:
>
> On 28 July 2014 08:56, Ciaran McCreesh <ciaran.mccreesh@googlemail.com>
> wrote:
>>
>> > To me it seems like a simple data model bug that vdb does not get
>> > updated to reflect the new situation after step 2 above.
>>
>> Rewriting VDB won't help if the user doesn't sync at the right time.
>
> Indeed. pkgmove has this problem solved with a mass of quarterly move files,
> but I'm not sure I'd want to have

Does it matter?  If a user never syncs again, they'll have a broken
package (though portage won't realize it).  If they wait until the
package is removed from the tree before syncing then they'll still
have a broken package.  If the broken package happens to work for
them, then the user won't care, and if it doesn't work for them,
they'll sync in the updates one way or another (using an overlay if
necessary).

As far as I can tell, the only thing that is needed for this to work
is for portage to be able to detect when an installed package
dependency changes in the repository, and this could be facilitated by
metadata.  Maintainers would then revbump when a change can't be
correctly handled, and they won't revbump when it can be.

Do we have to guarantee that users who don't sync before a PV is
removed get all updates to that PV if they have it installed when they
eventually do sync?  If I have some package installed that was
treecleaned a year ago and it depends on udev then obviously it won't
know anything about the new virtual - we live with that already
without too many problems.  Bottom line is that bad things happen if
you hang onto packages that aren't maintained in some repository
(possibly your overlay maintained by yourself).

Rich

Re: [gentoo-dev] don't rely on dynamic deps

[Kent Fredric]

[-- Attachment #1: Type: text/plain, Size: 627 bytes --]

On 28 July 2014 09:34, Rich Freeman <rich0@gentoo.org> wrote:

> and if it doesn't work for them,
> they'll sync in the updates one way or another (using an overlay if
> necessary).
>

However, in the case the package gets removed from tree, an updates based
approach would allow the dependencies to be cleaned up long after the
package itself is gone.

And this proves useful, because one of the questions people tend to ask (
or should ) when experiencing problems is "did you sync lately". If they
did sync lately, our best attempts at fixing their bug is employed.


-- 
Kent

*KENTNL* - https://metacpan.org/author/KENTNL

[-- Attachment #2: Type: text/html, Size: 1362 bytes --]

Re: [gentoo-dev] don't rely on dynamic deps

[Rich Freeman]

On Sun, Jul 27, 2014 at 5:50 PM, Kent Fredric <kentfredric@gmail.com> wrote:
>
> On 28 July 2014 09:34, Rich Freeman <rich0@gentoo.org> wrote:
>>
>> and if it doesn't work for them,
>> they'll sync in the updates one way or another (using an overlay if
>> necessary).
>
>
> However, in the case the package gets removed from tree, an updates based
> approach would allow the dependencies to be cleaned up long after the
> package itself is gone.

Maybe, but is it really our goal to fix broken packages that aren't
even maintained any longer?  The latest version of the package will
always be in cvs/etc and users can always go fetch it, but do we need
a special updates mechanism simply for the purpose of fixing packages
that we've already decided are unsustainable?

If an updates-like approach is the best approach for active packages,
then I'd consider the side-benefit to treecleaned ones as being
beneficial.  However, I wouldn't really view this as a primary
concern.  At least, that is my sense of it right now.  The primary
focus needs to be on making dynamic deps work in a sensible way for
active packages, which we're apparently having problems with already.

Rich

Re: [gentoo-dev] don't rely on dynamic deps

[Michał Górny]

[-- Attachment #1: Type: text/plain, Size: 748 bytes --]

Dnia 2014-07-27, o godz. 22:51:13
Peter Stuge <peter@stuge.se> napisał(a):

> What is the purpose of keeping only dependencies as-they-were when
> the package was installed, if the package manager does not somehow
> benefit from that information in the future?

You have to ask the one who implemented that. Maybe it was for
performance reasons (dynamic-deps cause a lot of overhead
themselves...), maybe just because it was hard to implement it
properly.

Feel free to provide a patch. Just please don't forget that you
shouldn't write the updates before emerge finishes updates or starts
depclean on relevant packages, so that the installed package dependency
tree doesn't end up being broken.

-- 
Best regards,
Michał Górny

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 949 bytes --]

Re: [gentoo-dev] don't rely on dynamic deps

[Rich Freeman]

On Sun, Jul 27, 2014 at 4:24 PM, Michał Górny <mgorny@gentoo.org> wrote:
> Dnia 2014-07-27, o godz. 10:42:19
>
> Consider the following:
>
> 1. A depends on B, both are installed,
>
> 2. dependency on B is removed, emerge --depclean uninstalls B thanks
> to dynamic-deps,
>
> 3. B is treecleaned (nothing depends on it),
>
> 4. old version of A is removed (user doesn't update it yet), therefore
> dependency on B is restored from vdb.
>
> So, now user has package A installed which has unsatisfied dependency
> on non-available package.

I'd think that portage should update vdb as soon as it detects the
dependency change.  Then B would no longer depend on A in vdb.  It
shouldn't hold onto outdated information.  Basically a dependency
change should trigger a no-rebuild merge if it is safe to do so, and
if not there should be a revbump anyway.

If A is removed before there is a sync, then A is still installed on
the user's system and portage still thinks that B depends on A, so it
remains consistent.

I acknowldege that this isn't how portage behaves today.

Rich

Re: [gentoo-dev] don't rely on dynamic deps

[Michał Górny]

[-- Attachment #1: Type: text/plain, Size: 1229 bytes --]

Dnia 2014-07-27, o godz. 17:08:27
Rich Freeman <rich0@gentoo.org> napisał(a):

> On Sun, Jul 27, 2014 at 4:24 PM, Michał Górny <mgorny@gentoo.org> wrote:
> > Dnia 2014-07-27, o godz. 10:42:19
> >
> > Consider the following:
> >
> > 1. A depends on B, both are installed,
> >
> > 2. dependency on B is removed, emerge --depclean uninstalls B thanks
> > to dynamic-deps,
> >
> > 3. B is treecleaned (nothing depends on it),
> >
> > 4. old version of A is removed (user doesn't update it yet), therefore
> > dependency on B is restored from vdb.
> >
> > So, now user has package A installed which has unsatisfied dependency
> > on non-available package.  
> 
> I'd think that portage should update vdb as soon as it detects the
> dependency change.  Then B would no longer depend on A in vdb.  It
> shouldn't hold onto outdated information.

You just introduced the opposite breakage -- when a dependency on C was
added, it ends up in vdb before C is installed. Now if C and current
version of A are removed before C gets installed, you end up having
broken dependency in vdb...

Plus, 'as soon' means you're making --pretend actually write something.
That's bad.

-- 
Best regards,
Michał Górny

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 949 bytes --]

Re: [gentoo-dev] don't rely on dynamic deps

[Rich Freeman]

On Sun, Jul 27, 2014 at 5:17 PM, Michał Górny <mgorny@gentoo.org> wrote:
> Dnia 2014-07-27, o godz. 17:08:27
> Rich Freeman <rich0@gentoo.org> napisał(a):
>>
>> I'd think that portage should update vdb as soon as it detects the
>> dependency change.  Then B would no longer depend on A in vdb.  It
>> shouldn't hold onto outdated information.
>
> You just introduced the opposite breakage -- when a dependency on C was
> added, it ends up in vdb before C is installed. Now if C and current
> version of A are removed before C gets installed, you end up having
> broken dependency in vdb.
>

There is a possibility that you had the broken dependency before you
even synced - you just didn't realize it yet.  After all, if the dep
was added only as the result of an actual on-disk change, there should
have been a revbump.  So, having a missing dep in vdb just makes vdb
reflect reality.  You can get a missing dep in vdb by unmerging a
package without using depclean.

An unmet dependency is still a dependency.

> Plus, 'as soon' means you're making --pretend actually write something.
> That's bad.

This isn't all that different from package moves.  I believe that
modifies vdb as soon as it detects updates.

There is the issue of syncing and then not updating and then having
all the packages removed from the tree  - you're then missing a
package and have no easy way to install it.  But, in that case you can
put the necessary ebuilds into your overlay and then portage can make
everything right.

Rich

Re: [gentoo-dev] don't rely on dynamic deps

[Ciaran McCreesh]

[-- Attachment #1: Type: text/plain, Size: 482 bytes --]

On Sun, 27 Jul 2014 17:26:27 -0400
Rich Freeman <rich0@gentoo.org> wrote:
> But, in that case you can put the necessary ebuilds into your overlay
> and then portage can make everything right.

Oh? Please explain to us a) how the overlay interaction *actually* works
with dynamic dependencies currently, and b) how it can work both in the
case you describe, and in the case where an overlay has a substantially
different ebuild for the same package.

-- 
Ciaran McCreesh

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 181 bytes --]

Re: [gentoo-dev] don't rely on dynamic deps

[Rich Freeman]

On Sun, Jul 27, 2014 at 5:33 PM, Ciaran McCreesh
<ciaran.mccreesh@googlemail.com> wrote:
> On Sun, 27 Jul 2014 17:26:27 -0400
> Rich Freeman <rich0@gentoo.org> wrote:
>> But, in that case you can put the necessary ebuilds into your overlay
>> and then portage can make everything right.
>
> Oh? Please explain to us a) how the overlay interaction *actually* works
> with dynamic dependencies currently,

No idea.  I doubt it is specified behavior.  Certainly we should learn
whatever lessons we can from what has been done already, but we aren't
constrained by it.

> and b) how it can work both in the
> case you describe, and in the case where an overlay has a substantially
> different ebuild for the same package.

I'd think that a change in repository should probably be treated like
a revbump.  There is no way to know that foo-1-r1 in one overlay is
the same as foo-1-r1 in another.  The package manager has to figure
out which overlay to use already, and if the same PV shows up in a
higher-priority overlay then it is treated as a revbump of whatever is
in the lower-priority overlay and it triggers a rebuild.  Maybe you
could get clever about checking for identical ebuilds/eclasses/etc and
avoid a rebuild if it literally is the same file, but I wouldn't go
further than that.

Rich

[gentoo-dev] Re: don't rely on dynamic deps

[Martin Vaeth]

Rich Freeman <rich0@gentoo.org> wrote:
>
> I'd think that a change in repository should probably be treated like
> a revbump.

I agree. At least, currently eix already shows [U] in such a situation,
and IMHO it would be wise if portage would suggest to upgrade/downgrade
then, too. But this is a topic which is rather unrelated to the
topic of dynamic deps.

Re: [gentoo-dev] don't rely on dynamic deps

[Arfrever Frehtes Taifersar Arahesis]

[-- Attachment #1: Type: Text/Plain, Size: 920 bytes --]

2014-07-27 23:33 Ciaran McCreesh napisał(a):
> On Sun, 27 Jul 2014 17:26:27 -0400
> Rich Freeman <rich0@gentoo.org> wrote:
> > But, in that case you can put the necessary ebuilds into your overlay
> > and then portage can make everything right.
> 
> Oh? Please explain to us a) how the overlay interaction *actually* works
> with dynamic dependencies currently

Portage uses dynamic dependencies from ebuild in original repository of given installed ebuild.
Name of repository is stored in /var/db/pkg/${CATEGORY}/${PF}/repository file.

> and b) how it can work both in the case you describe

A user would have to do:
# echo ${new_repository} > /var/db/pkg/${CATEGORY}/${PF}/repository
# touch /var/db/pkg/${CATEGORY}/${PF}
# touch /var/db/pkg/${CATEGORY}
# touch /var/db/pkg

(It is possible that updating of timestamps is not yet strictly required...)

--
Arfrever Frehtes Taifersar Arahesis

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 819 bytes --]

Re: [gentoo-dev] don't rely on dynamic deps

[Ian Stakenvicius]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

On 27/07/14 05:08 PM, Rich Freeman wrote:
> On Sun, Jul 27, 2014 at 4:24 PM, Michał Górny <mgorny@gentoo.org>
> wrote:
>> Dnia 2014-07-27, o godz. 10:42:19
>> 
>> Consider the following:
>> 
>> 1. A depends on B, both are installed,
>> 
>> 2. dependency on B is removed, emerge --depclean uninstalls B
>> thanks to dynamic-deps,
>> 
>> 3. B is treecleaned (nothing depends on it),
>> 
>> 4. old version of A is removed (user doesn't update it yet),
>> therefore dependency on B is restored from vdb.
>> 
>> So, now user has package A installed which has unsatisfied
>> dependency on non-available package.
> 
> I'd think that portage should update vdb as soon as it detects the 
> dependency change.  Then B would no longer depend on A in vdb.  It 
> shouldn't hold onto outdated information.  Basically a dependency 
> change should trigger a no-rebuild merge if it is safe to do so,
> and if not there should be a revbump anyway.

As has been mentioned or alluded to before, this is fine as long as
end-users --sync when the dependency change is still in the tree.
However, if that doesn't happen then we still end up with the issue.

Of course, if that is the case, then #2 shouldn't happen either
(because the end-user system wouldn't see B as having been removed and
therefore --depclean won't remove it).


...why do i feel like i'm getting the same headache i had in my 2nd
year databases course, when i was trying to wrap my head around ACID
compliance and transaction visibility....
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2

iF4EAREIAAYFAlPWelAACgkQ2ugaI38ACPCokAEAvDNcn+kJ6WTpL+hMAjexRuJX
mbHoj9pGsuFQ2kqoL7YA/1n9mZ2zDpVBurXLflU2KpqNgGx3E/ujozBOveHzoII+
=0Zgq
-----END PGP SIGNATURE-----

Re: [gentoo-dev] don't rely on dynamic deps

[Rich Freeman]

On Mon, Jul 28, 2014 at 12:29 PM, Ian Stakenvicius <axs@gentoo.org> wrote:
> As has been mentioned or alluded to before, this is fine as long as
> end-users --sync when the dependency change is still in the tree.
> However, if that doesn't happen then we still end up with the issue.
>
> Of course, if that is the case, then #2 shouldn't happen either
> (because the end-user system wouldn't see B as having been removed and
> therefore --depclean won't remove it).

Agree.  Things stay consistent either way if everything is done properly.

Bottom line is that if you keep PVs installed which aren't in portage,
you're on your own.  They could contain known bugs that we fixed in a
revbump that you missed, or they could contain security issues that we
don't bother to check for since we don't have the PV in our
repository, etc.  If you keep such a PV installed then you're the
maintainer, so good luck!

If a more recent version is in portage, then your old ebuild will
probably uninstall cleanly (or at least as cleanly as it would with
static deps), and the upgrade will hopefully be in better shape.

Rich

[gentoo-dev] Re: don't rely on dynamic deps

[Martin Vaeth]

Michał Górny <mgorny@gentoo.org> wrote:
>
> Consider the following:
>
> 1. A depends on B, both are installed,
>
> 2. dependency on B is removed, emerge --depclean uninstalls B thanks
> to dynamic-deps,
>
> 3. B is treecleaned (nothing depends on it),
>
> 4. old version of A is removed (user doesn't update it yet), therefore
> dependency on B is restored from vdb.
>
> So, now user has package A installed which has unsatisfied dependency
> on non-available package.

And this is per se not a bad situation:

The version of A which the user has installed is obsolete and needs
to be upgraded. This happens with emerge -NDu @world which should
be the first (portage-related) action the user does after syncing
(certainly before removing packages with emerge depclean, as he is
already instructed clearly).

When he does not, it is a user error.

When he insists on not updating A, he should have a very valid
reason for this and take responsibility for the situation by
maintaining A in his overlay: As already mentioned several times,
it is impossible to handle unmaintained packages/versions correctly
in all situations.

Whenever a package is removed or all available versions of a
package[:slot] are masked and no update happens, portage *must*
print a big fat warning, since this is a situation which can
always lead to problematic situations - completely independent
of whether dynamic deps or static deps are used.

Re: [gentoo-dev] don't rely on dynamic deps

[Michał Górny]

[-- Attachment #1: Type: text/plain, Size: 981 bytes --]

Dnia 2014-07-27, o godz. 16:05:34
""Paweł Hajdan, Jr."" <phajdan.jr@gentoo.org> napisał(a):

> On 7/21/14, 11:52 PM, Alexander Berntsen wrote:
> > Michał has documented the shortcomings of dynamic deps in our wiki[0].
> > (Thank you!) [...]
> > [0]  <https://wiki.gentoo.org/wiki/Project:Portage/Dynamic_dependencies>
> 
> There's one more thing I'd like to ask about:
> 
> For "Minor linking change w/ dependency change (unnecessary linking
> removed)" the "dynamic deps" cell is red with "revbump + mostly
> unnecessary rebuild", and "static deps" says "applied after rebuild".
> 
> Arguably with dynamic deps one could also skip the revbump, and the
> update would similarly be applied after rebuild.

No, the update would be applied immediately. In other words, emerge
will remove the dependency and allow it to be depcleaned even though
the package still links to it. You need to revbump to avoid the broken
linkage.


-- 
Best regards,
Michał Górny

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 949 bytes --]

Re: [gentoo-dev] don't rely on dynamic deps

[Tom Wijsman]

[-- Attachment #1: Type: text/plain, Size: 1326 bytes --]

On Mon, 21 Jul 2014 19:37:17 +0000
hasufell <hasufell@gentoo.org> wrote:

> afaiu dynamic deps are broken and not defined in PMS

It goes a step further than that!

The PMS imposes certain limits on dependencies; it states that DEPEND
must be present before executing src_* phases, that RDEPEND must be
present before treating the results as usable and that PDEPEND must be
done before finishing the batch of installs.

http://dev.gentoo.org/~ulm/pms/5/pms.html#x1-750008.1

If you attempt to fit in dynamic dependencies in that specification;
the src_* phases could never run, the results can never be considered
usable and the batch of installs can never finish.

> still... people seem to fix deps without revbumping, causing users who
> either don't use dynamic deps (it's optional for portage through
> --dynamic-deps=y, although it's on by default) or who use a different
> PM to not get the fix, at worst resulting in broken dependency
> calculation

Why do we have a PMS if we don't take into account other PMs?

Is Gentoo still a meta distribution? How does the Portage tree portage?

-- 
With kind regards,

Tom Wijsman (TomWij)
Gentoo Developer

E-mail address  : TomWij@gentoo.org
GPG Public Key  : 6D34E57D
GPG Fingerprint : C165 AF18 AB4C 400B C3D2  ABF0 95B2 1FCD 6D34 E57D

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 473 bytes --]

[gentoo-dev] Re: don't rely on dynamic deps

[Duncan]

Tom Wijsman posted on Tue, 22 Jul 2014 23:47:48 +0200 as excerpted:

> On Mon, 21 Jul 2014 19:37:17 +0000 hasufell <hasufell@gentoo.org> wrote:
> 
>> afaiu dynamic deps are broken and not defined in PMS
> 
> It goes a step further than that!
> 
> The PMS imposes certain limits on dependencies; it states that DEPEND
> must be present before executing src_* phases, that RDEPEND must be
> present before treating the results as usable and that PDEPEND must be
> done before finishing the batch of installs.
> 
> http://dev.gentoo.org/~ulm/pms/5/pms.html#x1-750008.1
> 
> If you attempt to fit in dynamic dependencies in that specification; the
> src_* phases could never run, the results can never be considered usable
> and the batch of installs can never finish.

How long have dynamic-deps been around?  Since EAPI-0?  Because if so, 
that interpretation must be incorrect, since EAPI-0 was defined as 
portage behavior at the time, and AFAIK, no EAPI since then has been 
approved without a working portage implementation.

In the context of dynamic-deps I'd interpret the above to mean within a 
single portage session.  What happens some sessions later when an 
ebuild's deps are dynamic-updated after a tree sync is an entirely new 
session, and unless I'm missing something, the above PMS requirements can 
be met within a single session with dynamic-deps.

>> still... people seem to fix deps without revbumping, causing users who
>> either don't use dynamic deps (it's optional for portage through
>> --dynamic-deps=y, although it's on by default) or who use a different
>> PM to not get the fix, at worst resulting in broken dependency
>> calculation
> 
> Why do we have a PMS if we don't take into account other PMs?
> 
> Is Gentoo still a meta distribution? How does the Portage tree portage?

This remains a valid question.

(FWIW, after some cleanup I set dynamic-deps=n in defaultopts, and did my 
last update, about nine days after my last sync and update, without 
dynamic-deps.  We'll see how it goes longer term.  Tho I guess people 
like me who have been running --newuse --deep for ages are half-way-there 
already.)

-- 
Duncan - List replies preferred.   No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master."  Richard Stallman

Re: [gentoo-dev] Re: don't rely on dynamic deps

[Tom Wijsman]

[-- Attachment #1: Type: text/plain, Size: 1657 bytes --]

On Fri, 25 Jul 2014 05:44:34 +0000 (UTC)
Duncan <1i5t5.duncan@cox.net> wrote:

> How long have dynamic-deps been around?  Since EAPI-0?  Because if
> so, that interpretation must be incorrect, since EAPI-0 was defined
> as portage behavior at the time, and AFAIK, no EAPI since then has
> been approved without a working portage implementation.

Good question, probably needs a dig in the old Portage history; it is
something long the search terms of dynamic dependencies in
FakeVarTree, given that the parameter[1] has been added later on.

EAPI specifies what PMs need to conform to, not the other way around;
EAPI-0 may very well be derived from Portage, that doesn't make such
side features that have not been specified in EAPI-0 a part of EAPI-0.

 [1]: Add emerge --dynamic-deps <y|n> option.
 http://git.overlays.gentoo.org/gitweb/?p=proj/portage.git;a=commit;h=f8e0c75e
 
> In the context of dynamic-deps I'd interpret the above to mean within
> a single portage session.  What happens some sessions later when an 
> ebuild's deps are dynamic-updated after a tree sync is an entirely
> new session, and unless I'm missing something, the above PMS
> requirements can be met within a single session with dynamic-deps.

Apart from the words "merge" and "batch", which are in the piece of
text that I've quoted; I'm not sure how for the rest of the piece a
session can be deduced or interpreted from what is specified.

-- 
With kind regards,

Tom Wijsman (TomWij)
Gentoo Developer

E-mail address  : TomWij@gentoo.org
GPG Public Key  : 6D34E57D
GPG Fingerprint : C165 AF18 AB4C 400B C3D2  ABF0 95B2 1FCD 6D34 E57D

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 473 bytes --]

[gentoo-dev] Re: don't rely on dynamic deps

[Duncan]

Tom Wijsman posted on Sat, 26 Jul 2014 00:09:58 +0200 as excerpted:

> EAPI specifies what PMs need to conform to, not the other way around;
> EAPI-0 may very well be derived from Portage, that doesn't make such
> side features that have not been specified in EAPI-0 a part of EAPI-0.

Not being around at the time, you may not know some of the history, but 
feel free to ask Ciaranm if you need a more authoritative source.

The thing is, EAPI-0 was not originally completely specified, and to my 
knowledge, remains that way, because that would have been real-world 
essentially impossible to do.  Instead, a convenient shortcut was taken.  
EAPI-0 was defined as what portage did at the time, with EAPI-1 for sure 
and I believe EAPI-2 at least, being defined as the the previous EAPI, 
with specifically defined changes, but with the base EAPI still only 
fuzzily defined as, basically, what portage did at the time.

And since the beginning, while there have been other unapproved EAPIs not 
allowed in the main gentoo tree, because portage was and remains the 
official default PM, no EAPI has been approved for main-tree deployment 
until portage had a working implementation.

So while portage can and certainly does have bugs where it doesn't meet 
EAPI requirements, particularly for behavior there since EAPI-0 and not 
specifically defined to be different in a specific EAPI since then, to 
the extent that PMS applies at all, the interpretation of PMS must still 
be read in the context of what portage did all those years ago with the 
original EAPI-0 spec, since EAPI-0 was /defined/ based on portage 
behavior at the time.

Which then begs the question[1] I asked, how old /is/ this dynamic-deps 
behavior?  Does it extend back to EAPI-0?  My gut sense from memory as a 
user back then and now is that it does, but that's simply a gut sense I'm 
ill equipped to go back and verify.

---
[1] Begs the question:  Yes, I'm aware of the legal and philosophical 
"circular logic" usage, now generally legacy in terms of real-world use 
except in the philosophic and legal areas.  I deliberately choose to use 
the phrase in the newer and now much more common sense, rhetorically 
personifying the question such that it can "beg to be asked".

-- 
Duncan - List replies preferred.   No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master."  Richard Stallman