[gentoo-project] Call for agenda items - Council meeting 2014-08-12

[gentoo-project] Call for agenda items - Council meeting 2014-08-12

[Ulrich Mueller]

[-- Attachment #1: Type: text/plain, Size: 456 bytes --]

In two weeks from now, the council will meet again. This is the time
to raise and prepare items that the council should put on the agenda
to discuss or vote on.

Please respond to this message with agenda items. Do not hesitate to
repeat your agenda item here with a pointer if you previously
suggested one (since the last meeting).

The agenda for the next meeting will be sent out on Tuesday 2014-08-05.

Please reply to the gentoo-project list.

Ulrich

[-- Attachment #2: Type: application/pgp-signature, Size: 490 bytes --]

Re: [gentoo-project] Call for agenda items - Council meeting 2014-08-12

[Pacho Ramos]

El mar, 29-07-2014 a las 11:18 +0200, Ulrich Mueller escribió:
> In two weeks from now, the council will meet again. This is the time
> to raise and prepare items that the council should put on the agenda
> to discuss or vote on.
> 
> Please respond to this message with agenda items. Do not hesitate to
> repeat your agenda item here with a pointer if you previously
> suggested one (since the last meeting).
> 
> The agenda for the next meeting will be sent out on Tuesday 2014-08-05.
> 
> Please reply to the gentoo-project list.
> 
> Ulrich

Would like to ask for action about how to finally handle bash-completion
on Gentoo. Looks like we are using a different approach as upstream to
handle completions now, there was a try (one year ago) to try to switch
to upstream style but seems that the revision implementing that was
later dropped due some arguments.

Currently, we are blocked with old stable version as testing one neither
works:

https://bugs.gentoo.org/show_bug.cgi?id=472938 -> It explains the
upstream way of handling completions. The version that implemented it
was 2.1-r1, it was later removed by Samuli (that did most of the work)
as looks like others complained.

https://bugs.gentoo.org/show_bug.cgi?id=486306 -> this shows how current
2.1 version is the tree has some problems blocking its stabilization
until a decision is made finally

Re: [gentoo-project] Call for agenda items - Council meeting 2014-08-12

[Michał Górny]

[-- Attachment #1: Type: text/plain, Size: 932 bytes --]

Dnia 2014-07-29, o godz. 14:06:18
Pacho Ramos <pacho@gentoo.org> napisał(a):

> Would like to ask for action about how to finally handle bash-completion
> on Gentoo. Looks like we are using a different approach as upstream to
> handle completions now, there was a try (one year ago) to try to switch
> to upstream style but seems that the revision implementing that was
> later dropped due some arguments.

Just to be clear, the arguments were not due to new bash-completion
itself but due to lack of completeness and documentation. If I recall
correctly, it was reverted simply because nobody was willing to do
the remaining work and we had no real documentation for the new system.
In other words, users upgraded, completions stopped working and didn't
know why or how to proceed.

I don't know if the Council needs to say anything here. It's just
a matter of doing the work.

-- 
Best regards,
Michał Górny

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 949 bytes --]

Re: [gentoo-project] Call for agenda items - Council meeting 2014-08-12

[Pacho Ramos]

El mar, 29-07-2014 a las 21:22 +0200, Michał Górny escribió:
> Dnia 2014-07-29, o godz. 14:06:18
> Pacho Ramos <pacho@gentoo.org> napisał(a):
> 
> > Would like to ask for action about how to finally handle bash-completion
> > on Gentoo. Looks like we are using a different approach as upstream to
> > handle completions now, there was a try (one year ago) to try to switch
> > to upstream style but seems that the revision implementing that was
> > later dropped due some arguments.
> 
> Just to be clear, the arguments were not due to new bash-completion
> itself but due to lack of completeness and documentation. If I recall
> correctly, it was reverted simply because nobody was willing to do
> the remaining work and we had no real documentation for the new system.
> In other words, users upgraded, completions stopped working and didn't
> know why or how to proceed.
> 
> I don't know if the Council needs to say anything here. It's just
> a matter of doing the work.
> 

Yeah, I have seen new comments in relevant bug report and looks like
that was the case. Sorry for the misunderstanding. There is no need to
include it in agenda then

Re: [gentoo-project] Call for agenda items - Council meeting 2014-08-12

[Samuli Suominen]

On 02/08/14 12:23, Pacho Ramos wrote:
> El mar, 29-07-2014 a las 21:22 +0200, Michał Górny escribió:
>> Dnia 2014-07-29, o godz. 14:06:18
>> Pacho Ramos <pacho@gentoo.org> napisał(a):
>>
>>> Would like to ask for action about how to finally handle bash-completion
>>> on Gentoo. Looks like we are using a different approach as upstream to
>>> handle completions now, there was a try (one year ago) to try to switch
>>> to upstream style but seems that the revision implementing that was
>>> later dropped due some arguments.
>> Just to be clear, the arguments were not due to new bash-completion
>> itself but due to lack of completeness and documentation. If I recall
>> correctly, it was reverted simply because nobody was willing to do
>> the remaining work and we had no real documentation for the new system.
>> In other words, users upgraded, completions stopped working and didn't
>> know why or how to proceed.
>>
>> I don't know if the Council needs to say anything here. It's just
>> a matter of doing the work.
>>
> Yeah, I have seen new comments in relevant bug report and looks like
> that was the case. Sorry for the misunderstanding. There is no need to
> include it in agenda then
>
>

Notably, there is no distribution specific documentation for the
upstream handling
of bash-completion files in any distribution I've seen, it's just a
upstream package,
with it's own documentation
I wouldn't even know what to put into the documentation, that wouldn't
be straight
copy'n'paste from man pages, files in /usr/share/doc, ...

So I guess it's more about the eselect module, and allowing it to move
specific
bash-completion files to temporary directory, making them out-of-scope for
the bash-completion autoloader (yes, there is an autoloader) that mimics
the outcome of putting specific bash-completion files to INSTALL_MASK

So, yeah, I don't know how council would be able to help here... Other than
request people to write the new eselect module...

- Samuli

Re: [gentoo-project] Call for agenda items - Council meeting 2014-08-12

[Michał Górny]

[-- Attachment #1: Type: text/plain, Size: 1781 bytes --]

Dnia 2014-08-03, o godz. 07:18:42
Samuli Suominen <ssuominen@gentoo.org> napisał(a):

> 
> On 02/08/14 12:23, Pacho Ramos wrote:
> > El mar, 29-07-2014 a las 21:22 +0200, Michał Górny escribió:
> >> Dnia 2014-07-29, o godz. 14:06:18
> >> Pacho Ramos <pacho@gentoo.org> napisał(a):
> >>
> >>> Would like to ask for action about how to finally handle bash-completion
> >>> on Gentoo. Looks like we are using a different approach as upstream to
> >>> handle completions now, there was a try (one year ago) to try to switch
> >>> to upstream style but seems that the revision implementing that was
> >>> later dropped due some arguments.
> >> Just to be clear, the arguments were not due to new bash-completion
> >> itself but due to lack of completeness and documentation. If I recall
> >> correctly, it was reverted simply because nobody was willing to do
> >> the remaining work and we had no real documentation for the new system.
> >> In other words, users upgraded, completions stopped working and didn't
> >> know why or how to proceed.
> >>
> >> I don't know if the Council needs to say anything here. It's just
> >> a matter of doing the work.
> >>
> > Yeah, I have seen new comments in relevant bug report and looks like
> > that was the case. Sorry for the misunderstanding. There is no need to
> > include it in agenda then
> 
> So I guess it's more about the eselect module, and allowing it to move
> specific
> bash-completion files to temporary directory, making them out-of-scope for
> the bash-completion autoloader (yes, there is an autoloader) that mimics
> the outcome of putting specific bash-completion files to INSTALL_MASK

Wouldn't it be better to generate exclude commands in bashrc?



-- 
Best regards,
Michał Górny

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 949 bytes --]

Re: [gentoo-project] Call for agenda items - Council meeting 2014-08-12

[Ulrich Mueller]

[-- Attachment #1: Type: text/plain, Size: 506 bytes --]

>>>>> On Sun, 03 Aug 2014, Samuli Suominen wrote:

> So I guess it's more about the eselect module, and allowing it to
> move specific bash-completion files to temporary directory, making
> them out-of-scope for the bash-completion autoloader (yes, there is
> an autoloader) that mimics the outcome of putting specific
> bash-completion files to INSTALL_MASK

Do I get this right, you want the eselect module move files installed
by a package to another directory, effectively making them orphans?

Ulrich

[-- Attachment #2: Type: application/pgp-signature, Size: 490 bytes --]

Re: [gentoo-project] Call for agenda items - Council meeting 2014-08-12

[Samuli Suominen]

On 03/08/14 11:55, Ulrich Mueller wrote:
>>>>>> On Sun, 03 Aug 2014, Samuli Suominen wrote:
>> So I guess it's more about the eselect module, and allowing it to
>> move specific bash-completion files to temporary directory, making
>> them out-of-scope for the bash-completion autoloader (yes, there is
>> an autoloader) that mimics the outcome of putting specific
>> bash-completion files to INSTALL_MASK
> Do I get this right, you want the eselect module move files installed
> by a package to another directory, effectively making them orphans?
>
> Ulrich

Of course it would require a pkg_postrm() phase that cleans up possible
orphans
But mgorny pointed out another solution in this thread, "Wouldn't it be
better to generate exclude commands in bashrc?"
And the answer to that would be "yes, of course"
As in, the specifics of how this would be best to achieve, are yet to be
determined...

- Samuli

Re: [gentoo-project] Call for agenda items - Council meeting 2014-08-12

[Ulrich Mueller]

[-- Attachment #1: Type: text/plain, Size: 631 bytes --]

>>>>> On Sun, 03 Aug 2014, Samuli Suominen wrote:

> On 03/08/14 11:55, Ulrich Mueller wrote:
>> Do I get this right, you want the eselect module move files
>> installed by a package to another directory, effectively making
>> them orphans?

> Of course it would require a pkg_postrm() phase that cleans up
> possible orphans

Still, this would be very bad design.

> But mgorny pointed out another solution in this thread, "Wouldn't it
> be better to generate exclude commands in bashrc?"
> And the answer to that would be "yes, of course"

Can you remind me what was wrong with the current method, namely using
symlinks?

Ulrich

[-- Attachment #2: Type: application/pgp-signature, Size: 490 bytes --]

Re: [gentoo-project] Call for agenda items - Council meeting 2014-08-12

[Samuli Suominen]

On 03/08/14 13:11, Ulrich Mueller wrote:
>>>>>> On Sun, 03 Aug 2014, Samuli Suominen wrote:
>> On 03/08/14 11:55, Ulrich Mueller wrote:
>>> Do I get this right, you want the eselect module move files
>>> installed by a package to another directory, effectively making
>>> them orphans?
>> Of course it would require a pkg_postrm() phase that cleans up
>> possible orphans
> Still, this would be very bad design.

I agree. I'm trying to distance myself from the whole issue, still gets
my blood pressure raising how some people behaved... so don't
expect too specific answers from me regarding the implementation
specifics.

>
>> But mgorny pointed out another solution in this thread, "Wouldn't it
>> be better to generate exclude commands in bashrc?"
>> And the answer to that would be "yes, of course"
> Can you remind me what was wrong with the current method, namely using
> symlinks?

Other than upstream packages checking for existance of directory
/usr/share/bash-completion/completions
and detemining if they will install the completion files or not, not
much else
It's just tedious work to need to hack every upstream package to the
Gentoo quirks, sort of love the
"stick close to upstream as possible" mantra
We could keep the old symlink method and just start using
/usr/share/bash-completion/completions as
a compromise, if that's the conclusion people draw, I have nothing
against that
As in, the main point was to start using the upstream directories, to be
compatible with reverse dependencies
out-of-box

- Samuli

Re: [gentoo-project] Call for agenda items - Council meeting 2014-08-12

[William Hubbs]

[-- Attachment #1: Type: text/plain, Size: 643 bytes --]

On Sun, Aug 03, 2014 at 01:35:13PM +0300, Samuli Suominen wrote:
> It's just tedious work to need to hack every upstream package to the
> Gentoo quirks, sort of love the
> "stick close to upstream as possible" mantra

As a long-time Gentoo user and developer, one of the things I've always
liked about Gentoo is this as well. There was very little custom work on
maintaining packages in Gentoo, because we have always stayed as close
to upstream as we possibly could.
I haven't really looked at the bash completion system, but I think we
should do things the way upstream does where possible. I do not like
divergance from upstream.

William


[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 181 bytes --]

Re: [gentoo-project] Call for agenda items - Council meeting 2014-08-12

[Michał Górny]

[-- Attachment #1: Type: text/plain, Size: 1271 bytes --]

Dnia 2014-08-03, o godz. 12:11:48
Ulrich Mueller <ulm@gentoo.org> napisał(a):

> >>>>> On Sun, 03 Aug 2014, Samuli Suominen wrote:
> 
> > On 03/08/14 11:55, Ulrich Mueller wrote:
> >> Do I get this right, you want the eselect module move files
> >> installed by a package to another directory, effectively making
> >> them orphans?
> 
> > Of course it would require a pkg_postrm() phase that cleans up
> > possible orphans
> 
> Still, this would be very bad design.
> 
> > But mgorny pointed out another solution in this thread, "Wouldn't it
> > be better to generate exclude commands in bashrc?"
> > And the answer to that would be "yes, of course"
> 
> Can you remind me what was wrong with the current method, namely using
> symlinks?

The old method was pretty bad, and we can do it much better nowadays.
IMO the best thing we could do is enabling by default and letting
eselect disable stuff via adding (and enable via removing):

  complete -r foo

to some well-defined file that gets sourced in bashrc.

Then there's a matter of sourcing the file you need to source to enable
completions at all. We could enable that unconditionally in bashrc, or
use it for global completion switch in eselect.

-- 
Best regards,
Michał Górny

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 949 bytes --]

[gentoo-project] Re: [gentoo-dev-announce] Call for agenda items - Council meeting 2014-08-12

[Patrick McLean]

[-- Attachment #1: Type: text/plain, Size: 499 bytes --]

On Tue, 29 Jul 2014 11:18:18 +0200
Ulrich Mueller <ulm@gentoo.org> wrote:

> Please respond to this message with agenda items. Do not hesitate to
> repeat your agenda item here with a pointer if you previously
> suggested one (since the last meeting).

I would like the council to discuss/vote on running phase functions
from all eclasses by default in a future EAPI (preferably EPAI=6),
instead of the last inherited one.

Here is the bug for reference:
https://bugs.gentoo.org/516014

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 819 bytes --]

[gentoo-project] Re: [gentoo-dev-announce] Call for agenda items - Council meeting 2014-08-12

[Ulrich Mueller]

[-- Attachment #1: Type: text/plain, Size: 521 bytes --]

>>>>> On Tue, 29 Jul 2014, Patrick McLean wrote:

> I would like the council to discuss/vote on running phase functions
> from all eclasses by default in a future EAPI (preferably EPAI=6),
> instead of the last inherited one.

> Here is the bug for reference:
> https://bugs.gentoo.org/516014

This is an intrusive change. Has it been discussed in the gentoo-dev
mailing list? If yes, please provide a pointer to the discussion.

If not, it should be discussed there first, before being submitted to
the council.

Ulrich

[-- Attachment #2: Type: application/pgp-signature, Size: 490 bytes --]

Re: [gentoo-project] Re: [gentoo-dev-announce] Call for agenda items - Council meeting 2014-08-12

[hasufell]

Ulrich Mueller:
>>>>>> On Tue, 29 Jul 2014, Patrick McLean wrote:
> 
>> I would like the council to discuss/vote on running phase functions
>> from all eclasses by default in a future EAPI (preferably EPAI=6),
>> instead of the last inherited one.
> 
>> Here is the bug for reference:
>> https://bugs.gentoo.org/516014
> 
> This is an intrusive change. Has it been discussed in the gentoo-dev
> mailing list? If yes, please provide a pointer to the discussion.
> 

http://article.gmane.org/gmane.linux.gentoo.devel/91839

Re: [gentoo-project] Re: [gentoo-dev-announce] Call for agenda items - Council meeting 2014-08-12

[hasufell]

hasufell:
> Ulrich Mueller:
>>>>>>> On Tue, 29 Jul 2014, Patrick McLean wrote:
>>
>>> I would like the council to discuss/vote on running phase functions
>>> from all eclasses by default in a future EAPI (preferably EPAI=6),
>>> instead of the last inherited one.
>>
>>> Here is the bug for reference:
>>> https://bugs.gentoo.org/516014
>>
>> This is an intrusive change. Has it been discussed in the gentoo-dev
>> mailing list? If yes, please provide a pointer to the discussion.
>>
> 
> http://article.gmane.org/gmane.linux.gentoo.devel/91839
> 
> 

sorry, I was mixing up posts

Re: [gentoo-project] Call for agenda items - Council meeting 2014-08-12

[Michał Górny]

[-- Attachment #1: Type: text/plain, Size: 4486 bytes --]

Dnia 2014-07-29, o godz. 11:18:18
Ulrich Mueller <ulm@gentoo.org> napisał(a):

> In two weeks from now, the council will meet again. This is the time
> to raise and prepare items that the council should put on the agenda
> to discuss or vote on.
> 
> Please respond to this message with agenda items. Do not hesitate to
> repeat your agenda item here with a pointer if you previously
> suggested one (since the last meeting).

Following my earlier mail to gentoo-dev [1], I would like the Council to
either abolish the specific policies enforced by games team, or confirm
that they are non-binding.

The games team is currently (against the structure given by GLEP 39
[2]) claiming itself to have sole and complete authority over every
game package in Gentoo. Additionally, it tries to fit them in 
a Gentoo-specific model that causes many games to diverge from upstream
and gain a lot of unnecessary complexity, sometimes even causing
security issues.

I have tried to convince the games team to consider changing
the policies multiple times, sometimes getting rude responses or no
reply at all. At this point, I believe that the only solution is to
ask the Council to clarify the policies which can be enforced
by a single team.

More specifically, I would like the Council to appropriately confirm or
establish that:

1. every developer is allowed to commit and maintain game ebuilds,
without the need to ask for permission or review from games team,

2. games team does not have authority to override maintainer decisions
on game packages,

3. the use of group 'games' to control access to games can be
deprecated and needs not to be enforced,

4. the /usr/games hierarchy (esp. Gentoo-specific /usr/games/lib*)
and other locations listed in games.eclass are entirely optional. Using
upstream install locations is recommended,

5. the use of games.eclass is entirely optional. Preferably, the eclass
would be deprecated since it mostly serves the purpose of enforcing
the rules objected in (3) and (4).


Rationale
---------

1. Game packages do not include core system packages or packages
otherwise critical to Gentoo. They also do not have any common pitfalls
specific only to games that could justify the requirement of review.
Being an active Gentoo developer should be the only necessary quality
required to commit game ebuilds.

2. Since no special treatment is justified, the games team should not
claim the right to override maintainer's decisions, remove maintainers
or remove packages stating 'non-games team commit' as a reason.

3. The attempt of limiting access to games through use of 'games' group
is unjustified and unprofessional. It caused multiple issues
in the past, including repeatedly ignored security issue from 2006 [3]
and multiple uses of RESTRICT=userpriv [4].

I recall hearing that game access ought to be restricted because
the games are more likely to be of worse quality and the sysadmin wants
limit the access to them to the trusted users. Yet at the same time
the restriction is causing game ebuilds to run game tools with root
privileges since otherwise the portage user is unable to access them.

4. This specific hierarchy is specified by FHS as optional and is
not beneficial to users (esp. considering the objection in (3)).
In some cases, it forces a lot of patching on packages using
non-autoconf build systems that would otherwise be installed in /usr
hierarchy, with no visible benefit. Additionally, this causes Gentoo to
diverge from upstream and therefore from other distributions.

5. Most of the code in the games.eclass is meant to redefine phase
functions and provide wrappers over standard PMS helpers. The sole
purpose of all that is to force the ownership and permissions (objected
in (3)), and /usr/games hierarchy (objected in (4)). If Council agrees
on abolishing those requirements, the eclass becomes mostly a no-op
(or equivalent to base.eclass which it is inheriting).

Moreover, the games team currently requires the eclass to be always
inherited last. Since it redefines multiple phase functions, this
sometimes results in ebuilds needing to restore all 'original' phase
functions.

[1]:http://thread.gmane.org/gmane.linux.gentoo.devel/91839/
[2]:https://wiki.gentoo.org/wiki/GLEP:39
[3]:https://bugs.gentoo.org/show_bug.cgi?id=125902
[4]:https://bugs.gentoo.org/show_bug.cgi?id=516576

-- 
Best regards,
Michał Górny

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 949 bytes --]

Re: [gentoo-project] Call for agenda items - Council meeting 2014-08-12

[Alexander Berntsen]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

On 30/07/14 09:26, Michał Górny wrote:
> 3. the use of group 'games' to control access to games can be 
> deprecated and needs not to be enforced,
I  would like the council to consider removing this group altogether,
and fixing all ebuilds to not use it. Maybe we could finally get rid
of this[0] 8 year old bug in the process.

[0]  <https://bugs.gentoo.org/show_bug.cgi?id=125902>
- -- 
Alexander
bernalex@gentoo.org
https://secure.plaimi.net/~alexander
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/

iF4EAREIAAYFAlPYyNAACgkQRtClrXBQc7WSVAD+LkIcUSLMNx0AILeEwO1TqeHr
lnFxAxfHCBo+Ez7I7TYBAIRfhFO3oj6vdjx2HPztvg5QZW985ZjFr6duFmVAWEQf
=M+NX
-----END PGP SIGNATURE-----

Re: [gentoo-project] Call for agenda items - Council meeting 2014-08-12

[Andrew Savchenko]

[-- Attachment #1: Type: text/plain, Size: 918 bytes --]

Hello,

On Wed, 30 Jul 2014 12:28:32 +0200 Alexander Berntsen wrote:
> On 30/07/14 09:26, Michał Górny wrote:
> > 3. the use of group 'games' to control access to games can be 
> > deprecated and needs not to be enforced,
> I  would like the council to consider removing this group altogether,
> and fixing all ebuilds to not use it.

Please carefully consider this matter. Having a dedicated group is
quite convenient to limit users from using games on workstations
and is also handy as a parental control feature. Of course,
a dedicated group is not an ultimate solution and can be evaded by
technically skilled users, but it nevertheless helps.

> Maybe we could finally get rid
> of this[0] 8 year old bug in the process.
> 
> [0]  <https://bugs.gentoo.org/show_bug.cgi?id=125902>

If application doesn't validate its input, this is an application's
bug.

Best regards,
Andrew Savchenko

[-- Attachment #2: Type: application/pgp-signature, Size: 836 bytes --]

Re: [gentoo-project] Call for agenda items - Council meeting 2014-08-12

[Michał Górny]

[-- Attachment #1: Type: text/plain, Size: 1916 bytes --]

Dnia 2014-07-30, o godz. 15:44:28
Andrew Savchenko <bircoph@gmail.com> napisał(a):

> On Wed, 30 Jul 2014 12:28:32 +0200 Alexander Berntsen wrote:
> > On 30/07/14 09:26, Michał Górny wrote:
> > > 3. the use of group 'games' to control access to games can be 
> > > deprecated and needs not to be enforced,
> > I  would like the council to consider removing this group altogether,
> > and fixing all ebuilds to not use it.
> 
> Please carefully consider this matter. Having a dedicated group is
> quite convenient to limit users from using games on workstations
> and is also handy as a parental control feature.

Please tell me, how many uses of games on workstations were actually
prevented thanks to it? How often do you happen to have a station that
has multiple users, games installed and you want to limit *all*
portage-installed games to subset of those users?

This a misguided attempt of fixing a social issue via technical means,
and it backfires a lot. In some cases it's an overkill, in some cases
it's incomplete. Following this logic, we ought to also limit access to
all ECMAScript-capable web browsers and scripting languages, including
the shell... oh wait, we just bricked the machine.

The only correct reason to limit access to games is when those involve
proprietary games for which only one of the users has license. But
then, it's a very big sledgehammer and a very small nut -- think of
the casualties.

> > Maybe we could finally get rid
> > of this[0] 8 year old bug in the process.
> > 
> > [0]  <https://bugs.gentoo.org/show_bug.cgi?id=125902>
> 
> If application doesn't validate its input, this is an application's
> bug.

If users are allowed to edit files they were not supposed to edit, then
it's a distribution bug. Failure to validate input is orthogonal to
this, and should be fixed independently of it.

-- 
Best regards,
Michał Górny

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 949 bytes --]

Re: [gentoo-project] Call for agenda items - Council meeting 2014-08-12

[Alexander Berntsen]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

On 30/07/14 13:44, Andrew Savchenko wrote:
> Please carefully consider this matter. Having a dedicated group is
>  quite convenient to limit users from using games on workstations
> and is also handy as a parental control feature. Of course, a
> dedicated group is not an ultimate solution and can be evaded by
> technically skilled users, but it nevertheless helps.
If you need to limit users from using games on a workstation, something
is wrong somewhere else. As for parental control -- it makes no sense.
They can install games locally for their own user, and they can still
look at whatever you don't want them to look at using the WWW; and they
can also watch films you do not want them to watch, listen to music you
object to, or read books you disagree with.

Restricting access to games, or indeed films, music, books or other
things, is not a problem I think we should be concerned about. It's
much too difficult to get right, and in my opinion completely
pointless and stupid. If you want to censor your children's access to
creative outlets -- do it yourself on your own computer, instead of
tasking us with the job. We are not nannies.
- -- 
Alexander
bernalex@gentoo.org
https://secure.plaimi.net/~alexander
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/

iF4EAREIAAYFAlPY98UACgkQRtClrXBQc7XoYgD+JS08CopLa8d2Mx6iuhnsdoyh
W1AbLfxaPEio0yoDXtMBAI05aLH6ZGA31My70U4vSkJGFeX+Sf53nEnMqAcA0xPA
=rd2P
-----END PGP SIGNATURE-----

Re: [gentoo-project] Call for agenda items - Council meeting 2014-08-12

[Andrew Savchenko]

[-- Attachment #1: Type: text/plain, Size: 1996 bytes --]

Hello,

On Wed, 30 Jul 2014 15:48:53 +0200 Alexander Berntsen wrote:
> On 30/07/14 13:44, Andrew Savchenko wrote:
> > Please carefully consider this matter. Having a dedicated group is
> >  quite convenient to limit users from using games on workstations
> > and is also handy as a parental control feature. Of course, a
> > dedicated group is not an ultimate solution and can be evaded by
> > technically skilled users, but it nevertheless helps.
> If you need to limit users from using games on a workstation, something
> is wrong somewhere else. As for parental control -- it makes no sense.
> They can install games locally for their own user,

On noexec partinion for /home and user temp dirs it will be not
possible to run them.

> and they can still
> look at whatever you don't want them to look at using the WWW; and they
> can also watch films you do not want them to watch, listen to music you
> object to, or read books you disagree with.
> 
> Restricting access to games, or indeed films, music, books or other
> things, is not a problem I think we should be concerned about. It's
> much too difficult to get right, and in my opinion completely
> pointless and stupid. If you want to censor your children's access to
> creative outlets -- do it yourself on your own computer, instead of
> tasking us with the job. We are not nannies.

It looks like my reasoning wrong was misunderstood a bit. ATM I'm
not personally interested in these features, but I find them useful
in some practical use cases. And looks like I'm not alone here,
because these features were introduced long time ago and for a
reason. I just want to point out that current games policy have not
only cons, but also pros. And code to support this policy is
already here (while it may be buggy, portage itself is quite buggy).
And please do not refer to "other distros", because Gentoo is very
different by its nature from majority of distros.

Best regards,
Andrew Savchenko

[-- Attachment #2: Type: application/pgp-signature, Size: 819 bytes --]

Re: [gentoo-project] Call for agenda items - Council meeting 2014-08-12

[Michał Górny]

[-- Attachment #1: Type: text/plain, Size: 2979 bytes --]

Dnia 2014-07-31, o godz. 11:36:51
Andrew Savchenko <bircoph@gmail.com> napisał(a):

> Hello,
> 
> On Wed, 30 Jul 2014 15:48:53 +0200 Alexander Berntsen wrote:
> > On 30/07/14 13:44, Andrew Savchenko wrote:
> > > Please carefully consider this matter. Having a dedicated group is
> > >  quite convenient to limit users from using games on workstations
> > > and is also handy as a parental control feature. Of course, a
> > > dedicated group is not an ultimate solution and can be evaded by
> > > technically skilled users, but it nevertheless helps.
> > If you need to limit users from using games on a workstation, something
> > is wrong somewhere else. As for parental control -- it makes no sense.
> > They can install games locally for their own user,
> 
> On noexec partinion for /home and user temp dirs it will be not
> possible to run them.

...which doesn't help with scripting languages like Python or bash.

> > and they can still
> > look at whatever you don't want them to look at using the WWW; and they
> > can also watch films you do not want them to watch, listen to music you
> > object to, or read books you disagree with.
> > 
> > Restricting access to games, or indeed films, music, books or other
> > things, is not a problem I think we should be concerned about. It's
> > much too difficult to get right, and in my opinion completely
> > pointless and stupid. If you want to censor your children's access to
> > creative outlets -- do it yourself on your own computer, instead of
> > tasking us with the job. We are not nannies.
> 
> It looks like my reasoning wrong was misunderstood a bit. ATM I'm
> not personally interested in these features, but I find them useful
> in some practical use cases. And looks like I'm not alone here,
> because these features were introduced long time ago and for a
> reason.

This is not an argument. Many things were introduced long time ago for
a reason, sometimes because someone made something up. And then they are
kept for a long time for another reason called stubbornness.

> I just want to point out that current games policy have not
> only cons, but also pros. And code to support this policy is
> already here (while it may be buggy, portage itself is quite buggy).

The problem is that the cons outweight the pros, and the pros may be
used by a few users while the cons affect everyone.

> And please do not refer to "other distros", because Gentoo is very
> different by its nature from majority of distros.

Without specifics, this is meaningless. There's no well-defined
'nature' of Gentoo, nor I have any idea how the differences apply to
this specific case.

That said, cross-distribution compatibility is important. If Gentoo
diverges from other distributions, it breeds packages that don't work
properly with other distributions and this is exactly the opposite of
what programmers using Gentoo expect.

-- 
Best regards,
Michał Górny

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 949 bytes --]

Re: [gentoo-project] Call for agenda items - Council meeting 2014-08-12

[hasufell]

Michał Górny:
> 
> That said, cross-distribution compatibility is important.

To be honest, this is a minor case and you shouldn't expect the council
to do anything about it.

Last time it really made a difference was about the pkgconfig discussion
and neither QA nor the council cared enough to enforce consistent rules
about it.

No one cares about cross-distro compatibility.

The more important argument is simplicity imo and maintenance burden. We
are not just talking about games supposed to install into
/usr/games/bin, but literally in any location, since those games
variables are modifiable and advertised as such (so this wasn't really
about FHS compliance in the first place). Please look at the downstream
patches, sed hacks, symlink workarounds, wrapper scripts and eclasses
(even gnome-games.eclass) that were introduced because of this.

Sure, it's doable, but still a lot of work to support a corner case.

In paludis, if you want to modify permissions for all games globally,
you could easily do so via an ebuild phase hook, e.g.

# cat /etc/paludis/hooks/ebuild_install_post/games_perms.bash
if [[ ${CATEGORY} =~ "games" ]] ; then
  find /var/tmp/paludis/${CATEGORY}-${PF}/image/usr/bin -type f ...
fi

I'm not sure why this case has to be supported via an eclass.

Re: Re: [gentoo-project] Call for agenda items - Council meeting 2014-08-12

[Andreas K. Huettel]

Am Mittwoch 30 Juli 2014, 15:44:28 schrieb Andrew Savchenko:
> Hello,
> 
> On Wed, 30 Jul 2014 12:28:32 +0200 Alexander Berntsen wrote:
> > On 30/07/14 09:26, Michał Górny wrote:
> > > 3. the use of group 'games' to control access to games can be
> > > deprecated and needs not to be enforced,
> > 
> > I  would like the council to consider removing this group altogether,
> > and fixing all ebuilds to not use it.
> 
> Please carefully consider this matter. Having a dedicated group is
> quite convenient to limit users from using games on workstations
> and is also handy as a parental control feature. Of course,
> a dedicated group is not an ultimate solution and can be evaded by
> technically skilled users, but it nevertheless helps.
> 

https://www.google.com/search?q=browser+games
nuff said

-- 
Andreas K. Huettel
Gentoo Linux developer
kde, council

Re: [gentoo-project] Call for agenda items - Council meeting 2014-08-12

[Andrew Savchenko]

[-- Attachment #1: Type: text/plain, Size: 996 bytes --]

On Wed, 30 Jul 2014 18:23:45 +0200 Andreas K. Huettel wrote:
> Am Mittwoch 30 Juli 2014, 15:44:28 schrieb Andrew Savchenko:
> > Hello,
> > 
> > On Wed, 30 Jul 2014 12:28:32 +0200 Alexander Berntsen wrote:
> > > On 30/07/14 09:26, Michał Górny wrote:
> > > > 3. the use of group 'games' to control access to games can be
> > > > deprecated and needs not to be enforced,
> > > 
> > > I  would like the council to consider removing this group altogether,
> > > and fixing all ebuilds to not use it.
> > 
> > Please carefully consider this matter. Having a dedicated group is
> > quite convenient to limit users from using games on workstations
> > and is also handy as a parental control feature. Of course,
> > a dedicated group is not an ultimate solution and can be evaded by
> > technically skilled users, but it nevertheless helps.
> > 
> 
> https://www.google.com/search?q=browser+games
> nuff said

This may be filtered by a proxy.

Best regards,
Andrew Savchenko

[-- Attachment #2: Type: application/pgp-signature, Size: 819 bytes --]

Re: [gentoo-project] Call for agenda items - Council meeting 2014-08-12

[Patrick Lauer]

On Thursday 31 July 2014 11:21:19 Andrew Savchenko wrote:
> On Wed, 30 Jul 2014 18:23:45 +0200 Andreas K. Huettel wrote:
> > Am Mittwoch 30 Juli 2014, 15:44:28 schrieb Andrew Savchenko:
> > > Hello,
> > > 
> > > On Wed, 30 Jul 2014 12:28:32 +0200 Alexander Berntsen wrote:
> > > > On 30/07/14 09:26, Michał Górny wrote:
> > > > > 3. the use of group 'games' to control access to games can be
> > > > > deprecated and needs not to be enforced,
> > > > 
> > > > I  would like the council to consider removing this group altogether,
> > > > and fixing all ebuilds to not use it.
> > > 
> > > Please carefully consider this matter. Having a dedicated group is
> > > quite convenient to limit users from using games on workstations
> > > and is also handy as a parental control feature. Of course,
> > > a dedicated group is not an ultimate solution and can be evaded by
> > > technically skilled users, but it nevertheless helps.
> > 
> > https://www.google.com/search?q=browser+games
> > nuff said
> 
> This may be filtered by a proxy.
> 

If you're playing those thought games - users could even run qemu to boot a 
whole OS! Or write a Tetris clone in Python!11

There's no way to fix a social issue like this with technical means properly, 
so I'd suggest not spending so much time on trying to figure out a corner case 
that is relatively irrelevant.

Re: Re: [gentoo-project] Call for agenda items - Council meeting 2014-08-12

[Andreas K. Huettel]

Am Mittwoch 30 Juli 2014, 09:26:38 schrieb Michał Górny:
> Dnia 2014-07-29, o godz. 11:18:18
> 
> Ulrich Mueller <ulm@gentoo.org> napisał(a):
> > In two weeks from now, the council will meet again. This is the time
> > to raise and prepare items that the council should put on the agenda
> > to discuss or vote on.
> > 
> > Please respond to this message with agenda items. Do not hesitate to
> > repeat your agenda item here with a pointer if you previously
> > suggested one (since the last meeting).
> 
> Following my earlier mail to gentoo-dev [1], I would like the Council to
> either abolish the specific policies enforced by games team, or confirm
> that they are non-binding.
> 
[snip]

I'd seriously appreciate comments from someone in the games team (or maybe 
even the games team lead) now.  (Avoiding "default judgment"... :)

Last time I checked, that was:
Mr_Bones_, Tupone, radhermit, tristan, vapier

-- 
Andreas K. Huettel
Gentoo Linux developer
kde, council

Re: Re: Re: [gentoo-project] Call for agenda items - Council meeting 2014-08-12

[Andreas K. Huettel]

[-- Attachment #1: Type: text/plain, Size: 561 bytes --]

> I didn't bother replying before because I think my assumption was that
> the council was clever enough to recognize silly when they saw it.
> There never was a compelling argument for dispelling policy set by
> groups in the general case and I'm not sure why the games group would
> be considered to have lesser status in that regard.

That actually doesn't address any of the issues brought up.

The main question is, why should the games team have more status than other 
projects?

-- 
Andreas K. Huettel
Gentoo Linux developer
kde, council

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 951 bytes --]

Re: Re: Re: [gentoo-project] Call for agenda items - Council meeting 2014-08-12

[Rich Freeman]

On Wed, Jul 30, 2014 at 2:15 PM, Andreas K. Huettel
<dilfridge@gentoo.org> wrote:
>> I didn't bother replying before because I think my assumption was that
>> the council was clever enough to recognize silly when they saw it.
>> There never was a compelling argument for dispelling policy set by
>> groups in the general case and I'm not sure why the games group would
>> be considered to have lesser status in that regard.
>
> That actually doesn't address any of the issues brought up.
>
> The main question is, why should the games team have more status than other
> projects?

++

Generally I am in favor of giving projects that adhere to GLEP39 more
of a say in things than individual maintainers, but setting aside
QA/Comrel/Infra there aren't really any projects out there which claim
the same kind of scope as games.  Amd64 might have a say over your
KEYWORDS, or systemd might want to install a text file you don't have
to look at, but none of them are going to basically rewrite your
ebuild, rename it, or tell you to get it out of the tree.  The
Gnome/KDE projects don't care if you install an application that uses
libkde/etc, though you'd do well to coordinate if you don't want
random breakage on the next big change.

If this were just an issue about games not accepting new members I
think that would be pretty trivial to fix, but I haven't gotten any
replies to my solicitation.  So, this is more than whether the lead
responds to member requests.

Some options open to the council are:
1.  Let the games project keep its policy, and anybody who wants to
change this has to join the project and call for elections (the
council can shoe-horn members onto the project if necessary).
2.  Directly tweak games policy but preserve the project and its
scope.  So, games would still have to adhere to games project policy,
but the Council might change specific policies (use of eclass, group,
etc).
3.  Restrict the games project scope, such as giving it authority if
the package maintainer elects to put it in the games herd.
4.  Do nothing.

I think #1 is the least intrusive (besides #4), but since nobody
responded to my call for new members I don't know that it will
accomplish anything.  I don't really care for #2 - it seems the most
meddlesome and I'm not sure what would be left for the project to
actually do if we basically ripped out all their policies and told
them they can't change them.

Do we really have a sense for how much demand there is for change?  #4
(or something equivalent like nicely asking games to deal with this)
might make sense if we think this is really just 2-3 devs with an
opinion, and that there isn't a larger demand out there.

However, right now #3 is where I'm somewhat leaning personally.  I
think it would be helpful if more members of the community with a
strong opinion speak up.  Apologies to the games project if it seems
like we're demanding a performance, but in the end the Council is as
much bottom-up as top-down and if all we hear is people demanding a
change, it is a bit hard to just ignore it, and really, listening to
the community is everybody's job anyway.

Rich

[gentoo-project] Re: Call for agenda items - Council meeting 2014-08-12

[Michael Palimaka]

On 07/31/2014 08:53 PM, Rich Freeman wrote:
> Some options open to the council are:
> 1.  Let the games project keep its policy, and anybody who wants to
> change this has to join the project and call for elections (the
> council can shoe-horn members onto the project if necessary).
> 2.  Directly tweak games policy but preserve the project and its
> scope.  So, games would still have to adhere to games project policy,
> but the Council might change specific policies (use of eclass, group,
> etc).
> 3.  Restrict the games project scope, such as giving it authority if
> the package maintainer elects to put it in the games herd.
> 4.  Do nothing.

Do options 1 and 2 mean endorsing the Games project as "special" (eg.
like QA or ComRel)? This is concerning, and not in the spirit of GLEP 39.

Options 3 and 4 are the same. Nobody has yet been able to provide any
evidence that the Games team has any elevated authority since GLEP 39
was implemented.

Re: [gentoo-project] Call for agenda items - Council meeting 2014-08-12

[hasufell]

Rich Freeman:
> 
> Some options open to the council are:
> 1.  Let the games project keep its policy, and anybody who wants to
> change this has to join the project and call for elections (the
> council can shoe-horn members onto the project if necessary).

You want to force-push members into a team while "preserving" the old
structure? That calls for trouble.

> 2.  Directly tweak games policy but preserve the project and its
> scope.  So, games would still have to adhere to games project policy,
> but the Council might change specific policies (use of eclass, group,
> etc).

That's contradictory, IMO. You are practically telling the games team
"your policies suck, so we just changed them". That doesn't really mean
they preserve their scope.

> 3.  Restrict the games project scope, such as giving it authority if
> the package maintainer elects to put it in the games herd.

This calls for trouble as well and will cause massive inconsistency.

> 4.  Do nothing.
> 

Pretty common these days, but what about

5. Have undertakers check for slacking project leads and accept related
complaints. If a project has a slacking lead, then the project should be
given a (long) deadline to fix it. If they don't fix it, then they
should be disbanded. A project cannot function properly without an
active lead (unless the project structure defines that there is no lead
anyway).

Re: [gentoo-project] Call for agenda items - Council meeting 2014-08-12

[Rich Freeman]

On Thu, Jul 31, 2014 at 7:49 AM, hasufell <hasufell@gentoo.org> wrote:
> Rich Freeman:
>>
>> Some options open to the council are:
>> 1.  Let the games project keep its policy, and anybody who wants to
>> change this has to join the project and call for elections (the
>> council can shoe-horn members onto the project if necessary).
>
> You want to force-push members into a team while "preserving" the old
> structure? That calls for trouble.

I'm just saying it is one possible option, so don't get carried away
with the "you."  It was actually my first preference though, and it
looks like others would prefer this as well.

It isn't about force-pushing.  Gentoo projects are generally supposed
to be open-access (with the exception of QA, Comrel, and Infra), so if
people want to join they should be able to do so unless there is a
good reason to exclude them.

Likewise, teams are supposed to elect leads annually.

So, this is just about removing bureaucratic roadblocks for people who
want to join the games project.  The majority can then set policy as
they see fit, which is basically the process in GLEP 39.  Project
leads not responding to requests to join is an abnormality.

However, I don't exactly see people lining up to join the games
project, so this is a bit of a moot point.

>
>> 2.  Directly tweak games policy but preserve the project and its
>> scope.  So, games would still have to adhere to games project policy,
>> but the Council might change specific policies (use of eclass, group,
>> etc).
>
> That's contradictory, IMO. You are practically telling the games team
> "your policies suck, so we just changed them". That doesn't really mean
> they preserve their scope.

To clarify - by scope I meant preserving their claim on any game in
the tree.  So, with this option all games have to still follow games
project policy, but those policies would be directly tweaked by the
Council.

I don't like this option either - I was just trying to spark
discussion and include potential options.

>> 3.  Restrict the games project scope, such as giving it authority if
>> the package maintainer elects to put it in the games herd.
>
> This calls for trouble as well and will cause massive inconsistency.

It certainly has that potential.  However, part of GLEP 39 is that
projects are allowed to compete, and inconsistent treatment of
/usr/games or the games group isn't half as confusing as multiple udev
implementations, sysvinit, package managers, and so on.

I think it is an option worth considering.  I wouldn't call it my
favorite option though.

>
> 5. Have undertakers check for slacking project leads and accept related
> complaints. If a project has a slacking lead, then the project should be
> given a (long) deadline to fix it. If they don't fix it, then they
> should be disbanded. A project cannot function properly without an
> active lead (unless the project structure defines that there is no lead
> anyway).
>

So, this is basically a variation on #1, but I'd rather see people
joining a project and breathing life into it rather than projects
simply being disbanded entirely.

That said, if NOBODY was on the games project and its policies were in
the way then disbanding it would be an option.  That really isn't the
case - there are active devs on the games project - they're just not
following GLEP 39 by holding elections and generally being accepting
of members.

I realize this isn't really adding anything much - I really just
wanted to clarify my meaning with the various options and generally
spark feedback.

Rich

Re: Re: Re: [gentoo-project] Call for agenda items - Council meeting 2014-08-12

[Denis Dupeyron]

On Thu, Jul 31, 2014 at 4:53 AM, Rich Freeman <rich0@gentoo.org> wrote:
> 1.  Let the games project keep its policy, and anybody who wants to
> change this has to join the project and call for elections

This is the obvious answer to the question. I wonder why we're even
discussing it. Unless actions by $team conflict with our rules and/or
actions with other teams, issues within $team are to be solved from
within $team.

On the matter of calling for elections, since every project must hold
elections every year per GLEP 39, it's legitimate for the council to
call for an election when it doesn't happen. No need for anybody to
join the project and call for it. When I was a council member I would
regularly follow-up with lead status in TLPs, and have actually forced
elections in the KDE project in an amiable way once. No need to get
the big hammer of justice from the closet: just gather the people, get
them to vote, and problem solved. If the current and previous councils
had been doing their job as they should have, instead of flooding
mailing lists, we wouldn't be in that situation today.

Denis.

Re: Re: Re: [gentoo-project] Call for agenda items - Council meeting 2014-08-12

[Seemant Kulleen]

[-- Attachment #1: Type: text/plain, Size: 917 bytes --]

On 31 July 2014 11:03, Denis Dupeyron <calchan@gentoo.org> wrote:

>
> This is the obvious answer to the question. I wonder why we're even
> discussing it. Unless actions by $team conflict with our rules and/or
> actions with other teams, issues within $team are to be solved from
> within $team.
>
> On the matter of calling for elections, since every project must hold
> elections every year per GLEP 39, it's legitimate for the council to
> call for an election when it doesn't happen. No need for anybody to
> join the project and call for it.



+1 to this.  Makes sense.


> If the current and previous councils
> had been doing their job as they should have, instead of flooding
> mailing lists, we wouldn't be in that situation today.


"Jigga-what" to this. The current council just started.  This is their
first go-around.  Let's hold off on the criticizing and let them work
things out.


Thanks,

Seemant

[-- Attachment #2: Type: text/html, Size: 1513 bytes --]

Re: Re: Re: [gentoo-project] Call for agenda items - Council meeting 2014-08-12

[Denis Dupeyron]

On Thu, Jul 31, 2014 at 12:17 PM, Seemant Kulleen <seemantk@gmail.com> wrote:
> "Jigga-what" to this. The current council just started.  This is their first
> go-around.  Let's hold off on the criticizing and let them work things out.

It's not a new problem, which is why I was including previous councils
in my compliments. And the current council is almost identical to the
last one. How many years is it going to take them to warm up?

This is an easy problem to solve, dammit. Council, let me make you an
offer. Since there is currently no lead in the games team (last
elections were held more than 12 months ago), nominate me and give me
a 2-week mandate. Just the time to find suitable candidates and run
elections. If that can fix a chunk of the spamming and whining on this
list, hell I'll do it. This offer won't be valid forever. I can't
guarantee that if you pull your collective head out of your collective
ass in a month that I'll still have the time to do it at that time.

Denis.

[gentoo-project] Re: Call for agenda items - Council meeting 2014-08-12

[Michael Palimaka]

On 08/01/2014 04:17 AM, Seemant Kulleen wrote:
> 
> On 31 July 2014 11:03, Denis Dupeyron <calchan@gentoo.org
> <mailto:calchan@gentoo.org>> wrote:
> 
> 
>     This is the obvious answer to the question. I wonder why we're even
>     discussing it. Unless actions by $team conflict with our rules and/or
>     actions with other teams, issues within $team are to be solved from
>     within $team.
> 
>     On the matter of calling for elections, since every project must hold
>     elections every year per GLEP 39, it's legitimate for the council to
>     call for an election when it doesn't happen. No need for anybody to
>     join the project and call for it. 
> 
> 
> 
> +1 to this.  Makes sense.
The issues described extend beyond the confines of the games team. If I
add a new game package, they will forcibly change the ebuild to be how
they want. GLEP 39 does not provide for this sort of authority.

Re: [gentoo-project] Call for agenda items - Council meeting 2014-08-12

[hasufell]

Denis Dupeyron:
> On Thu, Jul 31, 2014 at 4:53 AM, Rich Freeman <rich0@gentoo.org> wrote:
>> 1.  Let the games project keep its policy, and anybody who wants to
>> change this has to join the project and call for elections
> 
> This is the obvious answer to the question. I wonder why we're even
> discussing it. Unless actions by $team conflict with our rules and/or
> actions with other teams, issues within $team are to be solved from
> within $team.

a) There was an example of a conflict, you seem to have missed it.
b) Issues are also about getting in touch with the team and it hasn't
been solved for a long time, so... "are to be solved from within $team"
is a no-op.
c) The initial idea of mgorny was to discuss the games team policy. As
already pointed out in b) no one from the team answered on the thread.
So the next logical step is asking the council.

> 
> On the matter of calling for elections, since every project must hold
> elections every year per GLEP 39, it's legitimate for the council to
> call for an election when it doesn't happen. No need for anybody to
> join the project and call for it. When I was a council member I would
> regularly follow-up with lead status in TLPs, and have actually forced
> elections in the KDE project in an amiable way once. No need to get
> the big hammer of justice from the closet: just gather the people, get
> them to vote, and problem solved.

Pretty sure this will not solve any of the problems.

> If the current and previous councils
> had been doing their job as they should have, instead of flooding
> mailing lists, we wouldn't be in that situation today.
> 

What is ComRel doing these days? Why do we have to discuss communication
related issues with the council?

Even after this was all over dev ML, I have a feeling you didn't
actually try to mediate and contact the team/project lead, since you
don't seem to realize that there is a problem.
If you did, then I'm sorry and would like to hear the results.

So, let's not make assumptions about why we are in this situation...

Re: [gentoo-project] Call for agenda items - Council meeting 2014-08-12

[Denis Dupeyron]

On Thu, Jul 31, 2014 at 12:51 PM, hasufell <hasufell@gentoo.org> wrote:
> What is ComRel doing these days?

Why are you suddenly pulling comrel into an issue with elections in a
project? In case you didn't know it, comrel has no authority over the
council or metastructure.

Denis.

Re: [gentoo-project] Call for agenda items - Council meeting 2014-08-12

[hasufell]

Denis Dupeyron:
> On Thu, Jul 31, 2014 at 12:51 PM, hasufell <hasufell@gentoo.org> wrote:
>> What is ComRel doing these days?
> 
> Why are you suddenly pulling comrel into an issue with elections in a
> project? In case you didn't know it, comrel has no authority over the
> council or metastructure.
> 

Hum. The problem is not just elections, but communication. There was a
lengthy discussion on dev ML where you ended up replying with an inside
joke to an argument, so maybe you missed some parts of it.

I'm not going to repeat all the stuff that has already been said, but I
was under the impression that communication problems with a project is
very well within the scope of ComRel.

Re: [gentoo-project] Call for agenda items - Council meeting 2014-08-12

[Michał Górny]

[-- Attachment #1: Type: text/plain, Size: 4454 bytes --]

Dnia 2014-07-31, o godz. 06:53:50
Rich Freeman <rich0@gentoo.org> napisał(a):

> On Wed, Jul 30, 2014 at 2:15 PM, Andreas K. Huettel
> <dilfridge@gentoo.org> wrote:
> >> I didn't bother replying before because I think my assumption was that
> >> the council was clever enough to recognize silly when they saw it.
> >> There never was a compelling argument for dispelling policy set by
> >> groups in the general case and I'm not sure why the games group would
> >> be considered to have lesser status in that regard.
> >
> > That actually doesn't address any of the issues brought up.
> >
> > The main question is, why should the games team have more status than other
> > projects?
> 
> ++
> 
> Generally I am in favor of giving projects that adhere to GLEP39 more
> of a say in things than individual maintainers, but setting aside
> QA/Comrel/Infra there aren't really any projects out there which claim
> the same kind of scope as games.  Amd64 might have a say over your
> KEYWORDS, or systemd might want to install a text file you don't have
> to look at, but none of them are going to basically rewrite your
> ebuild, rename it, or tell you to get it out of the tree.  The
> Gnome/KDE projects don't care if you install an application that uses
> libkde/etc, though you'd do well to coordinate if you don't want
> random breakage on the next big change.

Well, just to be clear, I don't mind GNOME/KDE claiming maintainership
over core components of both DEs. Much like I don't even mind games
team maintaining core components necessary for gaming like SDL.

> If this were just an issue about games not accepting new members I
> think that would be pretty trivial to fix, but I haven't gotten any
> replies to my solicitation.  So, this is more than whether the lead
> responds to member requests.

I'm afraid that games team is a bit like upside-down compared to other
teams, and that is a social issue. While pretty much every other team
is happy to accept contributors, games team feels -- lightly said --
unwelcome. I don't think you can really resolve it via jamming it new
members by force.

While I can't speak for the specific people, I think they are pretty
much afraid that such an attempt would result in they feeling
unwelcome, and possibly having no real status.

> Some options open to the council are:
> 1.  Let the games project keep its policy, and anybody who wants to
> change this has to join the project and call for elections (the
> council can shoe-horn members onto the project if necessary).

As I see it, this can have two results:

a. games team elects new lead from current members, nothing changes,

b. you shoe-horn enough new members to force a new lead amongst them,
and existing members likely leave the project because of this.

> 2.  Directly tweak games policy but preserve the project and its
> scope.  So, games would still have to adhere to games project policy,
> but the Council might change specific policies (use of eclass, group,
> etc).

This doesn't feel correct to have team policies set by Council,
and again, the games team is likely to either ignore the decision or
disband itself because of it.

> 3.  Restrict the games project scope, such as giving it authority if
> the package maintainer elects to put it in the games herd.

I'd dare say this is not something the Council needs to do but only to
confirm. I'd be really happy to drop <herd>games</herd> from my game
packages as soon as Council confirms that the games team isn't allowed
to use that as an excuse to remove them or take over the maintainership.

> Do we really have a sense for how much demand there is for change?  #4
> (or something equivalent like nicely asking games to deal with this)
> might make sense if we think this is really just 2-3 devs with an
> opinion, and that there isn't a larger demand out there.

It's not as simple as some people disliking how things are. I believe
that the policies and behavior of the games team is the reason why
people are getting discouraged from contributing.

As a result, we have games team which can't handle the workload, few
contributors which have patience to work with games team and a lot of
people who would be happy to improve gaming experience in Gentoo but
simply lost the will to or otherwise lost the ability to improve their
ebuild skills.

-- 
Best regards,
Michał Górny

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 949 bytes --]

[gentoo-project] Re: Call for agenda items - Council meeting 2014-08-12

[Michael Palimaka]

On 07/29/2014 07:18 PM, Ulrich Mueller wrote:
> In two weeks from now, the council will meet again. This is the time
> to raise and prepare items that the council should put on the agenda
> to discuss or vote on.
> 
> Please respond to this message with agenda items. Do not hesitate to
> repeat your agenda item here with a pointer if you previously
> suggested one (since the last meeting).
> 
> The agenda for the next meeting will be sent out on Tuesday 2014-08-05.
> 
> Please reply to the gentoo-project list.
> 
> Ulrich
> 

I am disappointed that the Portage team declined to bring the issue of
disabling dynamic dependencies to the Council's attention themselves.
Thus, I must request the Council to consider the Portage team's recent
decision[1] in this matter.

If we are to change our default dependency model, we need to do it
properly - we need wider consensus, more open discussion of what's
happening, and a proper plan of how to handle the pitfalls of the new
model. Otherwise, we're just trading one set of problems for another.

Specifically, I request the Council block the removal of dynamic
dependencies until the following issues are resolved:

1. Although there has been considerable discussion[2] regarding dynamic
dependencies in general, there has been no specific discussion regarding
their actual removal.

2. The Portage team had made no announcement of their decision, nor do
they apparently intend to until 30 days prior to a new Portage release.
This is not adequate time for such a substantial change.

3. Few of the Portage team members were present[3] for the meeting, and
no vote was held to reach the decision.

4. While there is a good description of the theoretical issues affecting
both dependency models[4], multiple requests for specific examples of
in-tree breakage have been ignored. This makes it difficult to assess
the actual breakage that removing dynamic dependencies is supposed to
address.

5. The removal plan doesn't consider the impact from increased number of
rebuilds due to revbumps containing only dependency changes. Without
replacement functionality, widespread virtual or slot changes can cause
hundreds of packages to be rebuilt.

[1]: http://article.gmane.org/gmane.linux.gentoo.portage.devel/4351
[2]: http://thread.gmane.org/gmane.linux.gentoo.devel/92030
[3]: https://wiki.gentoo.org/wiki/Project:Portage/Meetings#Past_Meetings
[4]: https://wiki.gentoo.org/wiki/Project:Portage/Dynamic_dependencies

Re: [gentoo-project] Re: Call for agenda items - Council meeting 2014-08-12

[Samuli Suominen]

On 31/07/14 17:40, Michael Palimaka wrote:
> On 07/29/2014 07:18 PM, Ulrich Mueller wrote:
>> In two weeks from now, the council will meet again. This is the time
>> to raise and prepare items that the council should put on the agenda
>> to discuss or vote on.
>>
>> Please respond to this message with agenda items. Do not hesitate to
>> repeat your agenda item here with a pointer if you previously
>> suggested one (since the last meeting).
>>
>> The agenda for the next meeting will be sent out on Tuesday 2014-08-05.
>>
>> Please reply to the gentoo-project list.
>>
>> Ulrich
>>
> I am disappointed that the Portage team declined to bring the issue of
> disabling dynamic dependencies to the Council's attention themselves.
> Thus, I must request the Council to consider the Portage team's recent
> decision[1] in this matter.

+1

It seems like the Portage team is ignoring multiple developers
who have said they either rely upon dynamic deps, or have otherwise
raised conserns about the feature removal without replacement in
place.
The last hope seems to be that the council stops them for deleting
the working code (with some races, races we can live with)

Re: [gentoo-project] Re: Call for agenda items - Council meeting 2014-08-12

[Ciaran McCreesh]

[-- Attachment #1: Type: text/plain, Size: 438 bytes --]

On Thu, 31 Jul 2014 17:59:17 +0300
Samuli Suominen <ssuominen@gentoo.org> wrote:
> It seems like the Portage team is ignoring multiple developers
> who have said they either rely upon dynamic deps, or have otherwise
> raised conserns about the feature removal without replacement in
> place.

Well dynamic dependencies aren't guaranteed to work regardless, so
anyone relying upon them has broken ebuilds.

-- 
Ciaran McCreesh

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 181 bytes --]

Re: [gentoo-project] Re: Call for agenda items - Council meeting 2014-08-12

[hasufell]

Samuli Suominen:
> 
> It seems like the Portage team is ignoring multiple developers
> who have said they either rely upon dynamic deps, or have otherwise
> raised conserns about the feature removal without replacement in
> place.
> 

You will have no one left working on portage then if the council
randomly overwrites technical decisions (that are both PMS compatible
and fix non-trivial bugs) of package manager maintainers.

Good luck with that.

Re: [gentoo-project] Re: Call for agenda items - Council meeting 2014-08-12

[Ciaran McCreesh]

[-- Attachment #1: Type: text/plain, Size: 312 bytes --]

On Fri, 01 Aug 2014 00:40:18 +1000
Michael Palimaka <kensington@gentoo.org> wrote:
> 3. Few of the Portage team members were present[3] for the meeting,
> and no vote was held to reach the decision.

Why vote? It's a simple factual matter. There is a right answer and a
wrong one.

-- 
Ciaran McCreesh

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 181 bytes --]

Re: [gentoo-project] Re: Call for agenda items - Council meeting 2014-08-12

[Alexander Berntsen]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

Friends,

dynamic dependencies is a bug. We decided to fix this regression. I
don't think this is a council issue.

We will document how to prepare for the transition, to make sure that
it causes as few problems as possible. We will announce the release
that finally does disable dynamic dependencies as early as possible,
so that tree hackers are well prepared for it. The reason we have not
announced anything yet is because we want to have everything in place,
both in terms of code and in terms of documentation. In other words,
we want to have something substantial to show for ourselves, rather
than start a -dev bikeshed over nothing.

Please note that I am releasing a new version of Portage today. This
release will *not* turn off dynamic dependencies. Turning off dynamic
dependencies by default is not a change that will suddenly appear
without anyone knowing. We have every intention of preparing tree
hackers for it.
- -- 
Alexander
bernalex@gentoo.org
https://secure.plaimi.net/~alexander
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/

iF4EAREIAAYFAlPaadoACgkQRtClrXBQc7WgGAEAgxTGpKV5oec4Tl1aqM+yswnx
XgkFZQJ73nux+Hc6Dm4BALLXlGc3/PXOX0e35lSxxeAmSXp612cY6maymLnMr7aO
=SgDZ
-----END PGP SIGNATURE-----

Re: [gentoo-project] Re: Call for agenda items - Council meeting 2014-08-12

[Rich Freeman]

On Thu, Jul 31, 2014 at 12:07 PM, Alexander Berntsen
<bernalex@gentoo.org> wrote:
>
> dynamic dependencies is a bug. We decided to fix this regression. I
> don't think this is a council issue.
>

So, I realize there is a bit of a fine line in the
telling-contributors-how-to-contribute department here.  To some
extent how portage is developed is up to the portage project (though
anybody who wants to could always fork it and add yet another package
manager to the tree).

What really does fall into the Council's domain strongly is PMS and
tree policy.  If we have the tree target a package manger that does
not support dynamic dependencies, then we would want to do revbumps
anytime dependencies change (new virtuals, eclass upgrades, etc).  If
we target a package manager that does do dynamic dependencies then we
probably would want to forbid revbumps on such changes, which of
course would tend to break things for anybody using a package manager
that didn't support dynamic dependencies.

So, this is more than just a portage design question, and I think it
is fair for the Council to take up.  Obviously the feelings of the
portage maintainers should be carefully considered.

So, not trying to take a position pro/con in this email.  I just
wanted to state that I think this is something with wide-ranging
impact and more than just a portage issue.

Rich

Re: [gentoo-project] Re: Call for agenda items - Council meeting 2014-08-12

[Alexander Berntsen]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

Rich,

I'm going to reply in a slightly reversed order -- bear with me!

On 01/08/14 02:34, Rich Freeman wrote:
> So, this is more than just a portage design question, and I think
> it is fair for the Council to take up.  Obviously the feelings of
> the portage maintainers should be carefully considered.
Dynamic dependencies are not specified. As such, this is in fact a bug
rather than a question of design. If you want to talk design, I invite
you to do this in #gentoo-portage or on the gentoo-portage-dev mailing
list.

Personally, I am slightly surprised by the reactions and uproar this bug
fix has caused. At my day job, we commend each other for fixing bugs,
and express gratefulness for the effort. On that note I would like to
express my esteem for Michał, and the work he has put in here. I know he
has worked many hours with finding the least intrusive possible fix for
this bug. Thanks, Michał!

> So, I realize there is a bit of a fine line in the 
> telling-contributors-how-to-contribute department here.  To some 
> extent how portage is developed is up to the portage project
> (though anybody who wants to could always fork it and add yet
> another package manager to the tree).
> 
> What really does fall into the Council's domain strongly is PMS and
>  tree policy.  If we have the tree target a package manger that
> does not support dynamic dependencies, then we would want to do
> revbumps anytime dependencies change (new virtuals, eclass
> upgrades, etc). If we target a package manager that does do dynamic
> dependencies then we probably would want to forbid revbumps on such
> changes, which of course would tend to break things for anybody
> using a package manager that didn't support dynamic dependencies.
I appreciate that PMS and tree policy is important.

PMS does not specify dynamic dependencies. This means that if Portage
uses dynamic dependencies, and tree hackers rely on this behaviour, we
are needlessly making life difficult for users of other package
managers. Consequently, Portage should strive to follow the PMS's
intention as closely as possible. If you want to do some work on
formulating how dependencies are handled, please use the gentoo-pms
mailing list.

Tree policy, I'm afraid, has to adapt to Portage; not the other way
around. The Portage team is too small to be "bossed around" or "take
orders". That's just the way things are right now. But we follow the PMS
as closely as we are able to. Contributions are of course always welcome.

> So, not trying to take a position pro/con in this email.  I just 
> wanted to state that I think this is something with wide-ranging 
> impact and more than just a portage issue.
It has impact. We, the Portage team, appreciate this and will do our
best to be cooperative in the transitional phase.
- -- 
Alexander
bernalex@gentoo.org
https://secure.plaimi.net/~alexander
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/

iF4EAREIAAYFAlPbf0IACgkQRtClrXBQc7XKPAEAqVPLmVWpekj8qhEeSMSUTM3F
mgKlGHa3Ph+ZuWmWzxcA/1Nj7fT+FHG+ieCE9r6pKJuL7tmcNN5LZpkdlrjVHf+j
=p4+U
-----END PGP SIGNATURE-----

Re: [gentoo-project] Re: Call for agenda items - Council meeting 2014-08-12

[Rich Freeman]

On Fri, Aug 1, 2014 at 7:51 AM, Alexander Berntsen <bernalex@gentoo.org> wrote:
> On 01/08/14 02:34, Rich Freeman wrote:
>> So, this is more than just a portage design question, and I think
>> it is fair for the Council to take up.  Obviously the feelings of
>> the portage maintainers should be carefully considered.
> Dynamic dependencies are not specified.

None of this stuff is specified, dynamic or static.  The only thing in
PMS is how to declare a dependency.  It says nothing about how a
package manager should rely on this.

As was pointed out, portage prerm is broken with both dynamic and
static deps (and in my last email I limited this to slot operator
deps, but in reality it is broken anytime it depends on linkage
whether the slot operator dep is expressed or not).  I don't think
that is a huge issue in practice, but I've yet to hear an example of
anything which is.

> As such, this is in fact a bug
> rather than a question of design.

So, at work I have the luxury of systems that have broken requirements
instead of just broken implementations, but for as good as PMS is it
falls very short of being a full functional requirement specification
for portage.  The lack of a specified function in PMS does not mean
that implementing this in portage is a bug.

>
> Personally, I am slightly surprised by the reactions and uproar this bug
> fix has caused. At my day job, we commend each other for fixing bugs,
> and express gratefulness for the effort.

The problem is that not all agree that dynamic dependencies are a bug.
It isn't obvious that static deps are specified behavior, and even if
it were specifications can be changed before software is released when
there is agreement that they're incorrect.

I am certainly grateful to the few that are continuing to contribute
to portage development, however!

Also, I apologize if this email comes across as antagonistic.  I'm a
bit on the fence regarding dynamic deps and mostly for practical
reasons.  I've yet to be convinced that they aren't workable (and this
may simply be because I haven't noticed the argument against them).
On the other hand, Michał's post has gone a long way towards
convincing me that removing them isn't that big of a deal, which
leaves open the option of taking them out for now and then trying to
come up with a cleaner implementation later.

>
> PMS does not specify dynamic dependencies. This means that if Portage
> uses dynamic dependencies, and tree hackers rely on this behaviour, we
> are needlessly making life difficult for users of other package
> managers. Consequently, Portage should strive to follow the PMS's
> intention as closely as possible. If you want to do some work on
> formulating how dependencies are handled, please use the gentoo-pms
> mailing list.

I haven't spotted anything in PMS that specifies or necessitates
static dependencies (citations are welcome).  The content of VDB is
not within PMS's scope, and that is where most of the impact of
dynamic dependencies resides.

>
> Tree policy, I'm afraid, has to adapt to Portage; not the other way
> around.

The reality is that both portage and the tree policy need to adapt to
the needs of the community, otherwise there won't be anybody around
maintaining either.  Nobody can change that - Portage team, PMS team,
or even the Council.  That is why we're better off focusing on
communicating the need for change and understanding its impact, and
not arguing about who gets to make the call.  It also isn't enough to
just tell people what they have to do to comply - they need to be
convinced that they're doing it for a reasonably good reason, or at
least that it was a debatable decision and the Council had to make a
call one way or the other.

I think that Michał has already taken some steps to make the impact of
changing more clear.  If we can just get a quick argument as to the
need for change I think that would be helpful.  It need not be
exhaustive - if there are 500 things that break with dynamic deps a
top 5-10 list should be more than sufficient.

Rich

Re: [gentoo-project] Re: Call for agenda items - Council meeting 2014-08-12

[Ciaran McCreesh]

[-- Attachment #1: Type: text/plain, Size: 1775 bytes --]

On Fri, 1 Aug 2014 08:44:25 -0400
Rich Freeman <rich0@gentoo.org> wrote:
> As was pointed out, portage prerm is broken with both dynamic and
> static deps

No: prerm is only broken with static dependencies if either a developer
screws up, or there's a bug in the package mangler. But with dynamic
dependencies, prerm is broken by design.

> I don't think that is a huge issue in practice, but I've yet to hear
> an example of anything which is.

The ruby-config issue was real. But the bigger issue is: Portage's
dependency resolver simply doesn't work, and most of the time when it
goes wrong you don't realise what the root cause is. A proper fix is
needed for this, and the way to do that is to remove all the
unnecessary complexity. Dynamic dependencies are one example of these:
they're *only* necessary if developers are in the habit of screwing up.

> The problem is that not all agree that dynamic dependencies are a bug.

It's a simple matter of fact... You can disagree about what kind of
cheese the moon is made of, but that doesn't change the fact that it's
made of Cheddar.

> > Tree policy, I'm afraid, has to adapt to Portage; not the other way
> > around.
> 
> The reality is that both portage and the tree policy need to adapt to
> the needs of the community, otherwise there won't be anybody around
> maintaining either. 

This is about looking at the long term needs of the community, not the
short term needs. The current situation is a mess: Portage gives
incorrect resolutions, incomprehensible error messages, and sometimes
randomly and non-reproducibly uninstalls bash for unknown reasons, and
fully fixing this requires improvements to the quality of the data
provided by ebuild writers.

-- 
Ciaran McCreesh

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 181 bytes --]

Re: [gentoo-project] Re: Call for agenda items - Council meeting 2014-08-12

[hasufell]

Rich Freeman:
>>
>> Tree policy, I'm afraid, has to adapt to Portage; not the other way
>> around.
> 
> The reality is that both portage and the tree policy need to adapt to
> the needs of the community, otherwise there won't be anybody around
> maintaining either.

Rich, we are almost at the point where portage is unmaintained. Can't
say I feel sorry about that. I'd rather hack on paludis than portage
(although it isn't exactly well documented and would probably take me 2+
months just to understand some basics).

Anyway, I don't think the recent reactions help in any way to boost
portage development. People should really think about this and how this
is perceived by the remaining portage devs. If you guys want to actually
help portage, you should come up with code fixes that reduce the number
of rebuilds instead of voting on unimplemented solutions (which I think
is highly contradictory).

Re: [gentoo-project] Re: Call for agenda items - Council meeting 2014-08-12

[Rich Freeman]

On Fri, Aug 1, 2014 at 9:03 AM, hasufell <hasufell@gentoo.org> wrote:
> Rich Freeman:
>>>
>>> Tree policy, I'm afraid, has to adapt to Portage; not the other way
>>> around.
>>
>> The reality is that both portage and the tree policy need to adapt to
>> the needs of the community, otherwise there won't be anybody around
>> maintaining either.
>
> Rich, we are almost at the point where portage is unmaintained. Can't
> say I feel sorry about that. I'd rather hack on paludis than portage
> (although it isn't exactly well documented and would probably take me 2+
> months just to understand some basics).

So, if you want an example of why I think most people prefer portage
to paludis, I'd bring up @preserved-rebuild.  It is the biggest hack
that anybody could have come up with, and I can easily list 47 ways of
why the way paludis does things is more
elegant/accurate/well-behaved/etc.

The thing is, with @preserved-rebuild I don't have to run
revdep-rebuild for the packages that either can't be or simply aren't
migrated to slot operator deps.  That is a huge win.  Also, random
things aren't broken during the time that I'm rebuilding, so I don't
end up chrooting into my system from a rescue CD when I forget to run
revdep-rebuild.  I'll be happy when the day comes when we can get rid
of it, but that day is not yet here.

Generally speaking portage has favored usability over beauty of
design.  That has made it harder to maintain, but far more popular.

And don't get me wrong - I ran paludis for years.  I didn't migrate
back to portage until it began to do more than just resolve
dependencies.

>
> Anyway, I don't think the recent reactions help in any way to boost
> portage development. People should really think about this and how this
> is perceived by the remaining portage devs. If you guys want to actually
> help portage, you should come up with code fixes that reduce the number
> of rebuilds instead of voting on unimplemented solutions (which I think
> is highly contradictory).
>

If we're just going to turn portage into paludis, why would we bother?
 I think that paludis largely exemplifies this kind of design already,
or it least it did so back when I was running it.

And what I'm really asking for here is for somebody to actually
explain what is actually wrong with dynamic dependencies.  I have seen
47 almost-certainly-sincere claims that they're broken, but little in
the way of examples, and the one that has been given (prerm) seems
likely to break with static deps the way it is implemented today (we
don't unmerge reverse-deps before upgrading the dep, which breaks
linking that might be required to unmerge the package in the first
place - though it probably only breaks 0.01% of the time and the cure
is likely worse than the disease).

The last thing I want to do here is frustrate somebody who is doing
the right thing.  I'd just like to understand their thinking, and I
think many others would like to understand as well.

Rich

Re: [gentoo-project] Re: Call for agenda items - Council meeting 2014-08-12

[Seemant Kulleen]

[-- Attachment #1: Type: text/plain, Size: 883 bytes --]

Hi Rich,

And what I'm really asking for here is for somebody to actually
> explain what is actually wrong with dynamic dependencies.  I have seen
> 47 almost-certainly-sincere claims that they're broken, but little in
> the way of examples, and the one that has been given (prerm) seems
>

Are you saying you agree that the prerm example is a valid one, except for:


> likely to break with static deps the way it is implemented today (we
> don't unmerge reverse-deps before upgrading the dep, which breaks
> linking that might be required to unmerge the package in the first
> place - though it probably only breaks 0.01% of the time and the cure
> is likely worse than the disease).
>

I got lost here.  Are you invalidating the example or is this a more meta
invalidating your invalidation?

Surely a 99.9% valid example is pretty valid, or did I misinterpret?

Cheers,

Seemant

[-- Attachment #2: Type: text/html, Size: 1386 bytes --]

Re: [gentoo-project] Re: Call for agenda items - Council meeting 2014-08-12

[Rich Freeman]

On Fri, Aug 1, 2014 at 9:33 AM, Seemant Kulleen <seemantk@gmail.com> wrote:
>
> Are you saying you agree that the prerm example is a valid one, except for:
>
>>
>> likely to break with static deps the way it is implemented today (we
>> don't unmerge reverse-deps before upgrading the dep, which breaks
>> linking that might be required to unmerge the package in the first
>> place - though it probably only breaks 0.01% of the time and the cure
>> is likely worse than the disease).
>
>
> I got lost here.  Are you invalidating the example or is this a more meta
> invalidating your invalidation?
>
> Surely a 99.9% valid example is pretty valid, or did I misinterpret?
>

I'm saying that the objection that was raised with prerm seems to be
equally broken with static dependencies, and can be fixed for static
and dynamic dependencies in the same way, as far as I can tell.

There is always the issue that if you add a new dependency it can
start out unmet, but the same is true with static dependencies, except
that the package manager doesn't even realize that it is unmet.  That
is, with static dependencies the package manager gets a perfectly
consistent view of the universe, which is wrong and breaks in all the
cases where it would break with a dynamic dependency.

Rich

Re: [gentoo-project] Re: Call for agenda items - Council meeting 2014-08-12

[Ciaran McCreesh]

[-- Attachment #1: Type: text/plain, Size: 3221 bytes --]

On Fri, 1 Aug 2014 09:24:46 -0400
Rich Freeman <rich0@gentoo.org> wrote:
> The thing is, with @preserved-rebuild I don't have to run
> revdep-rebuild for the packages that either can't be or simply aren't
> migrated to slot operator deps.  That is a huge win.  Also, random
> things aren't broken during the time that I'm rebuilding, so I don't
> end up chrooting into my system from a rescue CD when I forget to run
> revdep-rebuild.  I'll be happy when the day comes when we can get rid
> of it, but that day is not yet here.

Unfortunately, like dynamic dependencies, there's a vicious feedback
cycle of increasingly ugly hacks with preserved-rebuild. People start
to use it, and it sometimes doesn't work, so another hack is added in
to work around one thing at the expense of three others, so people
carry on using it, so another hack is added in, and so on. It's not a
sustainable development model, and it's not something that will be
fixed by letting users and developers continue with a short-term view.

> Generally speaking portage has favored usability over beauty of
> design.  That has made it harder to maintain, but far more popular.

Portage favours usability in the case that nothing goes wrong, and it
does it by making it ever more likely that something will go horribly
wrong to the point that you have to give up and reinstall everything.
Paludis tries hard to make sure everything is correct, at the expense
that you have to invest smaller amounts of time as you go along fixing
errors. The key point is, this investment would be much smaller if the
quality of inputs was higher. This would be good for users, but also
for developers: wouldn't you like to replace all your horrible
complicated eclasses that generate perverse dependency strings with
something much simpler?

> And what I'm really asking for here is for somebody to actually
> explain what is actually wrong with dynamic dependencies.

The big issues are:

* They suddenly stop working if an ebuild is removed.

* They can make a 'sync' break a user's system.

* They don't work with binary packages.

* They don't work with overlays.

* They don't work with "resurrecting" packages in overlays.

* They're utterly incompatible with subslot deps.

* Someone adds selinux support to foo. Then a new bar starts requiring
  foo[selinux]. The user hasn't rebuilt their foo to get selinux
  support, but the dependency is still met, thanks to dynamic
  dependencies.

* The ruby-config example (details from memory, probably inaccurate,
  but the idea is right): Ruby ebuilds used to dep upon something like
  ruby-config, and they used it in pkg_prerm. The ebuilds were changed
  to use eselect-ruby instead. Users would replace ruby-config with
  eselect-ruby, and then be unable to uninstall or upgrade old ebuilds,
  because "dynamic" dependencies incorrectly changed the dependency
  without changing the pkg_prerm function.

But most fundamentally, the idea that a thing in VDB is somehow always
associated with exactly one ebuild in the tree, and that changes to
that ebuild are reflected in VDB, just doesn't work except in trivial
cases.

-- 
Ciaran McCreesh

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 181 bytes --]

Re: [gentoo-project] Re: Call for agenda items - Council meeting 2014-08-12

[Rich Freeman]

On Fri, Aug 1, 2014 at 9:37 AM, Ciaran McCreesh
<ciaran.mccreesh@googlemail.com> wrote:
> On Fri, 1 Aug 2014 09:24:46 -0400
> Rich Freeman <rich0@gentoo.org> wrote:
>> The thing is, with @preserved-rebuild I don't have to run
>> revdep-rebuild for the packages that either can't be or simply aren't
>> migrated to slot operator deps.  That is a huge win.  Also, random
>> things aren't broken during the time that I'm rebuilding, so I don't
>> end up chrooting into my system from a rescue CD when I forget to run
>> revdep-rebuild.  I'll be happy when the day comes when we can get rid
>> of it, but that day is not yet here.
>
> Unfortunately, like dynamic dependencies, there's a vicious feedback
> cycle of increasingly ugly hacks with preserved-rebuild.

No argument there at all.  Hence my statement that I'll be happy when
the day comes when I can be rid of it.

>
>> Generally speaking portage has favored usability over beauty of
>> design.  That has made it harder to maintain, but far more popular.
>
> Portage favours usability in the case that nothing goes wrong, and it
> does it by making it ever more likely that something will go horribly
> wrong to the point that you have to give up and reinstall everything.

So, I've never had to reinstall everything, and this is on a system
that has basically been continuously updated since around 2002-3 that
has used multiple package managers, desktop environments, etc.

In my experience things go wrong with portage only VERY rarely.  I
agree that it is undesirable when they do.

> Paludis tries hard to make sure everything is correct, at the expense
> that you have to invest smaller amounts of time as you go along fixing
> errors. The key point is, this investment would be much smaller if the
> quality of inputs was higher. This would be good for users, but also
> for developers: wouldn't you like to replace all your horrible
> complicated eclasses that generate perverse dependency strings with
> something much simpler?

I absolutely would love to see this.

The thing is, giving up things like @preserved-rebuild seems a bit
like threatening to kill kittens until people wake up and clean their
code.  We're talking about ripping out 80% solutions in favor of 20%
solutions in order to motivate ourselves to build 100% solutions.
Burning bridges has the advantage of ideological purity, but it isn't
always the most practical option.

>
>> And what I'm really asking for here is for somebody to actually
>> explain what is actually wrong with dynamic dependencies.
>
> The big issues are:

Thanks.

>
> * They suddenly stop working if an ebuild is removed.
>

This is only the result of the current implementation, which begins
taking into account dynamic dependencies but then doesn't update VDB.
I think a better approach is that once a dynamic dependency is used,
the VDB should be updated to reflect it.  Then there is no breakage
when ebuilds are removed.  For PMs that don't implement dynamic deps
this just reduces to current behavior (they never use a dynamic dep,
so they don't update VDB).

> * They can make a 'sync' break a user's system.

I think we need to be careful with terms like "break" here.  A sync
can result in a package suddenly having missing dependencies.

The thing is, there is a good chance they were missing them before the
sync, and the package manager was just blissfully unaware of this
because the dependency string was wrong.

That is my concern with this kind of approach.  It results in a much
cleaner data model, but it doesn't actually fix the reality that
things break when they have wrong dependencies.  With dynamic
dependencies the solution is that the PM just resolves the new
dependency.  When static deps the solution is to revbump the package
forcing a complete rebuild just to get the new dependency to resolve.
Either way the package is broken until the dep is added (perhaps in a
subtle way).

>
> * They don't work with binary packages.
>

Sort-of.  This is really just the VDB updating problem again, though
complicated by the fact that your binary package contains its own VDB.
It might be fixable at time of merging it, and if the original ebuild
isn't around at that time then you're back to the static dep model
which either breaks or not in the same way it would have before.

> * They don't work with overlays.
>
> * They don't work with "resurrecting" packages in overlays.

A lot of things don't work with overlays when it comes to changing
dependencies.  The only reason we can do a lot of things in portage is
that we modify reverse deps along with the deps themselves.

I haven't thought overlays through, and I'm willing to believe that
things break in odd ways with overlays.  We may never be able to
handle dynamic deps properly with them.

>
> * They're utterly incompatible with subslot deps.

I get that they aren't implemented with subslot deps.  I'm not quite
sure why they can't be.  When a new dep shows up, record the subslot
used to satisfy it when it is first satisifed.

>
> * Someone adds selinux support to foo. Then a new bar starts requiring
>   foo[selinux]. The user hasn't rebuilt their foo to get selinux
>   support, but the dependency is still met, thanks to dynamic
>   dependencies.
>

I don't see this as having anything to do with dynamic dependencies.
If foo was built without selinux, then it wouldn't meet the
dependency.  USE changes can't be assumed as not impacting the
installed files - I doubt this would be true in even a small minority
of cases.

> * The ruby-config example (details from memory, probably inaccurate,
>   but the idea is right): Ruby ebuilds used to dep upon something like
>   ruby-config, and they used it in pkg_prerm. The ebuilds were changed
>   to use eselect-ruby instead. Users would replace ruby-config with
>   eselect-ruby, and then be unable to uninstall or upgrade old ebuilds,
>   because "dynamic" dependencies incorrectly changed the dependency
>   without changing the pkg_prerm function.

Thanks.  This is helpful.

I'm not entirely sure if this is just an implementation issue.

>
> But most fundamentally, the idea that a thing in VDB is somehow always
> associated with exactly one ebuild in the tree, and that changes to
> that ebuild are reflected in VDB, just doesn't work except in trivial
> cases.

I'll ponder this a bit.

The specifics are helpful.  Even if many of these debatably have
solutions I do agree that we aren't there today.

Rich

Re: [gentoo-project] Re: Call for agenda items - Council meeting 2014-08-12

[hasufell]

Rich Freeman:
> On Fri, Aug 1, 2014 at 9:37 AM, Ciaran McCreesh
> <ciaran.mccreesh@googlemail.com> wrote:
>> On Fri, 1 Aug 2014 09:24:46 -0400
>> Rich Freeman <rich0@gentoo.org> wrote:
>>> The thing is, with @preserved-rebuild I don't have to run
>>> revdep-rebuild for the packages that either can't be or simply aren't
>>> migrated to slot operator deps.  That is a huge win.  Also, random
>>> things aren't broken during the time that I'm rebuilding, so I don't
>>> end up chrooting into my system from a rescue CD when I forget to run
>>> revdep-rebuild.  I'll be happy when the day comes when we can get rid
>>> of it, but that day is not yet here.
>>
>> Unfortunately, like dynamic dependencies, there's a vicious feedback
>> cycle of increasingly ugly hacks with preserved-rebuild.
> 
> No argument there at all.  Hence my statement that I'll be happy when
> the day comes when I can be rid of it.
> 

Rich, this is a _fundamental_ problem we have in gentoo. It's the lack
of a development model.

>>
>> * They suddenly stop working if an ebuild is removed.
>>
> 
> This is only the result of the current implementation, which begins
> taking into account dynamic dependencies but then doesn't update VDB.
> I think a better approach is that once a dynamic dependency is used,
> the VDB should be updated to reflect it.  Then there is no breakage
> when ebuilds are removed.  For PMs that don't implement dynamic deps
> this just reduces to current behavior (they never use a dynamic dep,
> so they don't update VDB).
> 

We only have the current implementation. So what you are saying is again
just an idea without code. Voting on it now without a reference
implementation will not result in anything useful.

And that "update VDB" approach is not trivial and has to be thought
through a lot (problems were already pointed out by mgorny), maybe even
by fixing PMS before the PM. Otherwise we have not gained anything and
still randomly break 3rd party PMs by relying on portage specific
behavior. We are practically nullifying the effort PMS was started for.

It almost sounds to me like people want to keep the old development
model "let's fix this real quick".
I think what the portage team does is the only sane approach: fix the
dependency resolution regression (that's the _most_ important part of
the PM). And then... think about how to minimize rebuilds (mgorny is
apparently already working on that). If that leads to another dynamic
deps solution (although I doubt it), so be it.


> 
>> * They don't work with overlays.
>>
>> * They don't work with "resurrecting" packages in overlays.
> 
> A lot of things don't work with overlays when it comes to changing
> dependencies.  The only reason we can do a lot of things in portage is
> that we modify reverse deps along with the deps themselves.
> 
> I haven't thought overlays through, and I'm willing to believe that
> things break in odd ways with overlays.  We may never be able to
> handle dynamic deps properly with them.
> 

Gentoo is becoming anti-modular. That is contradictory to our
meta-distribution model.

Not only, but also because of that we will lose a lot of people to NixOS
in the next few years, I am pretty sure of that.

Re: [gentoo-project] Re: Call for agenda items - Council meeting 2014-08-12

[Rich Freeman]

On Fri, Aug 1, 2014 at 10:35 AM, hasufell <hasufell@gentoo.org> wrote:
> Rich Freeman:
> We only have the current implementation. So what you are saying is again
> just an idea without code. Voting on it now without a reference
> implementation will not result in anything useful.

Oh, I agree completely.  I was trying to get at that in the last line
of my email.

>>> * They don't work with overlays.
>>>
>>> * They don't work with "resurrecting" packages in overlays.
>>
>> A lot of things don't work with overlays when it comes to changing
>> dependencies.  The only reason we can do a lot of things in portage is
>> that we modify reverse deps along with the deps themselves.
>>
>> I haven't thought overlays through, and I'm willing to believe that
>> things break in odd ways with overlays.  We may never be able to
>> handle dynamic deps properly with them.
>>
>
> Gentoo is becoming anti-modular. That is contradictory to our
> meta-distribution model.

Most of the overlay issues are with namespace.  Overlays work great if
they are self-contained.  If they depend on packages in the main
repository and those packages change, they will break.

>
> Not only, but also because of that we will lose a lot of people to NixOS
> in the next few years, I am pretty sure of that.
>

I don't see how they would solve the issues we have with overlays.  I
don't know much about them but their approach to isolated builds
should at least help prevent dependency errors in the first place.  If
package foo splits into two packages with different names, I don't
know how NixOS would prevent the need to update reverse dependencies
(unless it expresses things more granularly).

On the topic of dependency verification: one thing that probably
wouldn't be terribly hard to do is to configure our build jails based
on the declared DEPENDs.  The jail already has both inclusion and
exclusion functionality - it is just set to include everything that
isn't explicitly excluded.  If you fed the jail a list of everything
in DEPEND and @system then you should be able to reliably detect
missing DEPENDs.

But, that is more non-existent code.

What might be another approach would be to actually containerize or
use SELinux to enforce RDEPENDs when a package runs.  I'm not sure if
NixOS purports to do that.

Rich

Re: [gentoo-project] Re: Call for agenda items - Council meeting 2014-08-12

[hasufell]

Rich Freeman:
>>
> 
> I don't see how they would solve the issues we have with overlays.

just FYI, see http://article.gmane.org/gmane.linux.distributions.nixos/13676

there are also some further links in that thread

[gentoo-project] Re: Call for agenda items - Council meeting 2014-08-12

[Michael Palimaka]

On 08/02/2014 12:35 AM, hasufell wrote:
> Rich Freeman:
>> On Fri, Aug 1, 2014 at 9:37 AM, Ciaran McCreesh
>> <ciaran.mccreesh@googlemail.com> wrote:
>>> On Fri, 1 Aug 2014 09:24:46 -0400
>>> Rich Freeman <rich0@gentoo.org> wrote:
>>>> The thing is, with @preserved-rebuild I don't have to run
>>>> revdep-rebuild for the packages that either can't be or simply aren't
>>>> migrated to slot operator deps.  That is a huge win.  Also, random
>>>> things aren't broken during the time that I'm rebuilding, so I don't
>>>> end up chrooting into my system from a rescue CD when I forget to run
>>>> revdep-rebuild.  I'll be happy when the day comes when we can get rid
>>>> of it, but that day is not yet here.
>>>
>>> Unfortunately, like dynamic dependencies, there's a vicious feedback
>>> cycle of increasingly ugly hacks with preserved-rebuild.
>>
>> No argument there at all.  Hence my statement that I'll be happy when
>> the day comes when I can be rid of it.
>>
> 
> Rich, this is a _fundamental_ problem we have in gentoo. It's the lack
> of a development model.

It's also one of Gentoo's strengths. Please draft a document and present
it to the Council if you would like to change it.

Re: [gentoo-project] Re: Call for agenda items - Council meeting 2014-08-12

[hasufell]

Michael Palimaka:
> On 08/02/2014 12:35 AM, hasufell wrote:
>> Rich Freeman:
>>> On Fri, Aug 1, 2014 at 9:37 AM, Ciaran McCreesh
>>> <ciaran.mccreesh@googlemail.com> wrote:
>>>> On Fri, 1 Aug 2014 09:24:46 -0400
>>>> Rich Freeman <rich0@gentoo.org> wrote:
>>>>> The thing is, with @preserved-rebuild I don't have to run
>>>>> revdep-rebuild for the packages that either can't be or simply aren't
>>>>> migrated to slot operator deps.  That is a huge win.  Also, random
>>>>> things aren't broken during the time that I'm rebuilding, so I don't
>>>>> end up chrooting into my system from a rescue CD when I forget to run
>>>>> revdep-rebuild.  I'll be happy when the day comes when we can get rid
>>>>> of it, but that day is not yet here.
>>>>
>>>> Unfortunately, like dynamic dependencies, there's a vicious feedback
>>>> cycle of increasingly ugly hacks with preserved-rebuild.
>>>
>>> No argument there at all.  Hence my statement that I'll be happy when
>>> the day comes when I can be rid of it.
>>>
>>
>> Rich, this is a _fundamental_ problem we have in gentoo. It's the lack
>> of a development model.
> 
> It's also one of Gentoo's strengths. Please draft a document and present
> it to the Council if you would like to change it.
> 
> 

Absolutely not.

The development model is within the scope of a specific project, so the
portage team can decide what model they want to follow.

Re: [gentoo-project] Re: Call for agenda items - Council meeting 2014-08-12

[Ciaran McCreesh]

[-- Attachment #1: Type: text/plain, Size: 2876 bytes --]

On Fri, 1 Aug 2014 10:00:34 -0400
Rich Freeman <rich0@gentoo.org> wrote:
> The thing is, giving up things like @preserved-rebuild seems a bit
> like threatening to kill kittens until people wake up and clean their
> code.  We're talking about ripping out 80% solutions in favor of 20%
> solutions in order to motivate ourselves to build 100% solutions.

preserved-rebuild isn't an 80% solution. It's a 90% problem.

> > * They suddenly stop working if an ebuild is removed.
> 
> This is only the result of the current implementation, which begins
> taking into account dynamic dependencies but then doesn't update VDB.
> I think a better approach is that once a dynamic dependency is used,
> the VDB should be updated to reflect it.

This only fixes the special case where the change is "seen".

> That is my concern with this kind of approach.  It results in a much
> cleaner data model, but it doesn't actually fix the reality that
> things break when they have wrong dependencies.

Things are already broken if developers are specifying wrong
dependencies. We have bigger problems if your assertion is that
developers getting dependencies wrong is so common that revbumping to
fix them would affect users.

> > * They don't work with binary packages.
> >
> 
> Sort-of.  This is really just the VDB updating problem again, though
> complicated by the fact that your binary package contains its own VDB.
> It might be fixable at time of merging it, and if the original ebuild
> isn't around at that time then you're back to the static dep model
> which either breaks or not in the same way it would have before.

Static dependencies only break if a developer screws up, which should
be rare. Dynamic dependencies break under normal operation.

> > * They're utterly incompatible with subslot deps.
> 
> I get that they aren't implemented with subslot deps.  I'm not quite
> sure why they can't be.  When a new dep shows up, record the subslot
> used to satisfy it when it is first satisifed.

There's no simple mapping in the "complex mess of || dependencies" case.

> > * Someone adds selinux support to foo. Then a new bar starts
> > requiring foo[selinux]. The user hasn't rebuilt their foo to get
> > selinux support, but the dependency is still met, thanks to dynamic
> >   dependencies.
> 
> I don't see this as having anything to do with dynamic dependencies.
> If foo was built without selinux, then it wouldn't meet the
> dependency.  USE changes can't be assumed as not impacting the
> installed files - I doubt this would be true in even a small minority
> of cases.

I included this one because it was presented as a "feature" of dynamic
dependencies earlier in the discussion.

> I'm not entirely sure if this is just an implementation issue.

It isn't. It's a design flaw.

-- 
Ciaran McCreesh

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 181 bytes --]

Re: [gentoo-project] Re: Call for agenda items - Council meeting 2014-08-12

[Michał Górny]

[-- Attachment #1: Type: text/plain, Size: 4143 bytes --]

Dnia 2014-08-01, o godz. 00:40:18
Michael Palimaka <kensington@gentoo.org> napisał(a):

> I am disappointed that the Portage team declined to bring the issue of
> disabling dynamic dependencies to the Council's attention themselves.

I don't understand your disappointment. If you disagree with portage
team decision, it is your duty to bring it to the Council.

> If we are to change our default dependency model, we need to do it
> properly - we need wider consensus, more open discussion of what's
> happening, and a proper plan of how to handle the pitfalls of the new
> model. Otherwise, we're just trading one set of problems for another.

Yes, exactly. We need to get dynamic-deps right if they are ever
supposed to become the default. That's one of the reasons that we want
to revert the problematic changes and make Portage use the default
model once again.

> Specifically, I request the Council block the removal of dynamic
> dependencies until the following issues are resolved:
> 
> 1. Although there has been considerable discussion[2] regarding dynamic
> dependencies in general, there has been no specific discussion regarding
> their actual removal.

The thread pretty quickly turned into a bikeshed about that, so I don't
understand what you're talking about.

> 2. The Portage team had made no announcement of their decision, nor do
> they apparently intend to until 30 days prior to a new Portage release.
> This is not adequate time for such a substantial change.

I don't know what you mean by 'they apparently intend' but that sounds
offensive. I'd really prefer if you sticked to the facts.

The Portage team is going to announce the decision when it's final
and the code necessary for non-painful migration is in place.
Otherwise, the announcement would only bring barren discussion that
couldn't be supported by facts or numbers.

> 3. Few of the Portage team members were present[3] for the meeting, and
> no vote was held to reach the decision.

I don't understand how this is relevant.

> 4. While there is a good description of the theoretical issues affecting
> both dependency models[4], multiple requests for specific examples of
> in-tree breakage have been ignored. This makes it difficult to assess
> the actual breakage that removing dynamic dependencies is supposed to
> address.

The listing of theoretical issues is wrong and doesn't correspond to
Portage code, and yes, that's my fault. However, it only proves that
nobody on the thread bothered to look at the relevant Portage code,
even the people who started providing patches...

> 5. The removal plan doesn't consider the impact from increased number of
> rebuilds due to revbumps containing only dependency changes. Without
> replacement functionality, widespread virtual or slot changes can cause
> hundreds of packages to be rebuilt.

Please support this claim with specific examples or numbers. Otherwise,
it's just a 'theoretical issue' that you can't support.

If you are really curious, I am working hard on providing tools to fix
the vdb inconsistencies caused by dynamic-deps. There were no specific
data because it wasn't available until today.

My regularly updated desktop system (2-3 days between @world updates)
after disabling dynamic-deps has 77 packages needing rebuild. That
number includes a few virtuals, Perl packages and other low-effort
cases. And this is after the big, scary virtual/*udev changes.

Over the next days I will obviously have more numbers. More
specifically, the number of packages needing rebuild after dependency
changes made by developers. It should be noted that the above number
includes one-time rebuild of packages that are simply ancient.

There is a lot of FUD about unnecessary rebuilds. Sadly, most people
seem to fight a holy war against them without realizing the real
impact. In fact, more unnecessary rebuilds are caused by unnecessary
USE flags than by dependency changes. Yet the same people believe in
adding more flags to contain even more minor aspects of packages...

-- 
Best regards,
Michał Górny

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 949 bytes --]

Re: [gentoo-project] Re: Call for agenda items - Council meeting 2014-08-12

[Samuli Suominen]

On 31/07/14 22:12, Michał Górny wrote:
> We need to get dynamic-deps right if they are ever supposed to become
the default.

Obviously, dynamic-deps _is_ the default.  Don't distract the
non-developer audience by spewing
non-facts.

- Samuli

Re: [gentoo-project] Re: Call for agenda items - Council meeting 2014-08-12

[Ciaran McCreesh]

[-- Attachment #1: Type: text/plain, Size: 459 bytes --]

On Thu, 31 Jul 2014 22:32:49 +0300
Samuli Suominen <ssuominen@gentoo.org> wrote:
> On 31/07/14 22:12, Michał Górny wrote:
> > We need to get dynamic-deps right if they are ever supposed to
> > become
> the default.
> 
> Obviously, dynamic-deps _is_ the default.  Don't distract the
> non-developer audience by spewing
> non-facts.

Please re-read all the times it has been pointed out to you that it is
not that simple.

-- 
Ciaran McCreesh

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 181 bytes --]

Re: [gentoo-project] Re: Call for agenda items - Council meeting 2014-08-12

[Rich Freeman]

On Thu, Jul 31, 2014 at 3:12 PM, Michał Górny <mgorny@gentoo.org> wrote:
>
> Yes, exactly. We need to get dynamic-deps right if they are ever
> supposed to become the default. That's one of the reasons that we want
> to revert the problematic changes and make Portage use the default
> model once again.

Do we actually have some kind of list of issues with dynamic deps?
The only specific one that I think I've seen is with prerm and subslot
deps, but as was pointed out that issue actually is as much of a
problem with static deps unless you unmerge all the reverse-deps
before upgrading anything, followed by a re-merge.

I have no problem with accepting that it is broken, but it would be
nice to deal with more specifics and not with it in general.

>
> If you are really curious, I am working hard on providing tools to fix
> the vdb inconsistencies caused by dynamic-deps. There were no specific
> data because it wasn't available until today.
>
> My regularly updated desktop system (2-3 days between @world updates)
> after disabling dynamic-deps has 77 packages needing rebuild. That
> number includes a few virtuals, Perl packages and other low-effort
> cases. And this is after the big, scary virtual/*udev changes.
>
> Over the next days I will obviously have more numbers. More
> specifically, the number of packages needing rebuild after dependency
> changes made by developers. It should be noted that the above number
> includes one-time rebuild of packages that are simply ancient.
>
> There is a lot of FUD about unnecessary rebuilds. Sadly, most people
> seem to fight a holy war against them without realizing the real
> impact. In fact, more unnecessary rebuilds are caused by unnecessary
> USE flags than by dependency changes. Yet the same people believe in
> adding more flags to contain even more minor aspects of packages...

Thank you for this.  It is very helpful in gauging the likely impact
of having more revbumps.

One thing I don't want to do is create a barrier to anybody who wants
to upgrade an eclass or do work on virtuals.  I can just imagine
endless debates about whether splitting a virtual is worth it since it
will cause up to 250 rebuilds, etc.

Is there any easy way to compare tree vs installed deps using the API?
 I have a script that is part of the way there, but I'm struggling to
compare the vartree and porttree depdendencies (the portree
dependencies need to be correctly reduced - if somebody has the list
of function calls needed to reduce an RDEPEND from porttree to account
for USE/etc in /etc/portage and substitute in subslots that would be
helpful.

code snippet:
    depstr = vartree.dbapi.aux_get(cpv, ["RDEPEND"])[0]
    depstr2 = porttree.dbapi.aux_get(cpv, ["RDEPEND"] )[0]
    flatdeps=portage.dep.flatten(portage.dep.paren_reduce(depstr))
    flatdeps2=portage.dep.flatten(portage.dep.use_reduce(portage.dep.paren_reduce(depstr2)))

Ideally those should be the same if static deps = dynamic deps, but I
suspect it isn't appying the right USE flags (I'm not passing any),
and it isn't substituting actual subslot values either.  I'm also not
sure if flatten is going to properly handle operators/etc.

Rich

Re: [gentoo-project] Re: Call for agenda items - Council meeting 2014-08-12

[Michał Górny]

[-- Attachment #1: Type: text/plain, Size: 3087 bytes --]

Dnia 2014-07-31, o godz. 22:17:59
Rich Freeman <rich0@gentoo.org> napisał(a):

> On Thu, Jul 31, 2014 at 3:12 PM, Michał Górny <mgorny@gentoo.org> wrote:
> >
> > Yes, exactly. We need to get dynamic-deps right if they are ever
> > supposed to become the default. That's one of the reasons that we want
> > to revert the problematic changes and make Portage use the default
> > model once again.
> 
> Do we actually have some kind of list of issues with dynamic deps?
> The only specific one that I think I've seen is with prerm and subslot
> deps, but as was pointed out that issue actually is as much of a
> problem with static deps unless you unmerge all the reverse-deps
> before upgrading anything, followed by a re-merge.

I already listed the major issues in my second reply to Michael. And I
forgot about prerm() again, thanks for adding it :).

> > If you are really curious, I am working hard on providing tools to fix
> > the vdb inconsistencies caused by dynamic-deps. There were no specific
> > data because it wasn't available until today.
> >
> > My regularly updated desktop system (2-3 days between @world updates)
> > after disabling dynamic-deps has 77 packages needing rebuild. That
> > number includes a few virtuals, Perl packages and other low-effort
> > cases. And this is after the big, scary virtual/*udev changes.
> >
> > Over the next days I will obviously have more numbers. More
> > specifically, the number of packages needing rebuild after dependency
> > changes made by developers. It should be noted that the above number
> > includes one-time rebuild of packages that are simply ancient.
> >
> > There is a lot of FUD about unnecessary rebuilds. Sadly, most people
> > seem to fight a holy war against them without realizing the real
> > impact. In fact, more unnecessary rebuilds are caused by unnecessary
> > USE flags than by dependency changes. Yet the same people believe in
> > adding more flags to contain even more minor aspects of packages...
> 
> Thank you for this.  It is very helpful in gauging the likely impact
> of having more revbumps.
> 
> One thing I don't want to do is create a barrier to anybody who wants
> to upgrade an eclass or do work on virtuals.  I can just imagine
> endless debates about whether splitting a virtual is worth it since it
> will cause up to 250 rebuilds, etc.
> 
> Is there any easy way to compare tree vs installed deps using the API?

Not an easy way. However, if you take the two patches I posted on
gentoo-portage-dev [1,2] you can play a bit with @changed-deps. You can
add a few pprint()s to the '!=' clause to see what diffs it is seeing
after preprocessing.

However, it will see some 'extra' changes from || ( foo bar:= )
to || ( foo:= bar:= ) due to weird portage behavior. This vdb records
will be fixed after rebuilding the relevant packages thanks to patch
[2].

[1]:http://article.gmane.org/gmane.linux.gentoo.portage.devel/4357
[2]:http://article.gmane.org/gmane.linux.gentoo.portage.devel/4358

-- 
Best regards,
Michał Górny

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 949 bytes --]

Re: [gentoo-project] Re: Call for agenda items - Council meeting 2014-08-12

[Rich Freeman]

On Fri, Aug 1, 2014 at 2:51 AM, Michał Górny <mgorny@gentoo.org> wrote:
> Dnia 2014-07-31, o godz. 22:17:59
> Rich Freeman <rich0@gentoo.org> napisał(a):
>
>> On Thu, Jul 31, 2014 at 3:12 PM, Michał Górny <mgorny@gentoo.org> wrote:
>> >
>> > Yes, exactly. We need to get dynamic-deps right if they are ever
>> > supposed to become the default. That's one of the reasons that we want
>> > to revert the problematic changes and make Portage use the default
>> > model once again.
>>
>> Do we actually have some kind of list of issues with dynamic deps?
>> The only specific one that I think I've seen is with prerm and subslot
>> deps, but as was pointed out that issue actually is as much of a
>> problem with static deps unless you unmerge all the reverse-deps
>> before upgrading anything, followed by a re-merge.
>
> I already listed the major issues in my second reply to Michael. And I
> forgot about prerm() again, thanks for adding it :).

Any chance of a link?  I'm honestly not finding this list in my list
history, and a search on gmane for kensington with your from address
only comes up with one email in the last few weeks - the one I replied
to.

Is it possible you're referring to a private email, or some list not on gmane?

Rich

Re: [gentoo-project] Re: Call for agenda items - Council meeting 2014-08-12

[Michał Górny]

[-- Attachment #1: Type: text/plain, Size: 1573 bytes --]

Dnia 2014-08-01, o godz. 05:31:40
Rich Freeman <rich0@gentoo.org> napisał(a):

> On Fri, Aug 1, 2014 at 2:51 AM, Michał Górny <mgorny@gentoo.org> wrote:
> > Dnia 2014-07-31, o godz. 22:17:59
> > Rich Freeman <rich0@gentoo.org> napisał(a):
> >
> >> On Thu, Jul 31, 2014 at 3:12 PM, Michał Górny <mgorny@gentoo.org> wrote:
> >> >
> >> > Yes, exactly. We need to get dynamic-deps right if they are ever
> >> > supposed to become the default. That's one of the reasons that we want
> >> > to revert the problematic changes and make Portage use the default
> >> > model once again.
> >>
> >> Do we actually have some kind of list of issues with dynamic deps?
> >> The only specific one that I think I've seen is with prerm and subslot
> >> deps, but as was pointed out that issue actually is as much of a
> >> problem with static deps unless you unmerge all the reverse-deps
> >> before upgrading anything, followed by a re-merge.
> >
> > I already listed the major issues in my second reply to Michael. And I
> > forgot about prerm() again, thanks for adding it :).
> 
> Any chance of a link?  I'm honestly not finding this list in my list
> history, and a search on gmane for kensington with your from address
> only comes up with one email in the last few weeks - the one I replied
> to.
> 
> Is it possible you're referring to a private email, or some list not on gmane?

We're sorry about it, it seems that neither of us have noticed that
the mails started going off the list. They have been re-sent now.

-- 
Best regards,
Michał Górny

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 949 bytes --]

[gentoo-project] Re: Call for agenda items - Council meeting 2014-08-12

[Michael Palimaka]

On 08/01/2014 12:17 PM, Rich Freeman wrote:
> One thing I don't want to do is create a barrier to anybody who wants
> to upgrade an eclass or do work on virtuals.  I can just imagine
> endless debates about whether splitting a virtual is worth it since it
> will cause up to 250 rebuilds, etc.

This is what concerns me. I have no objection to static dependencies per
se, but we need something (whether it be dynamic dependencies or some
new functionality to work in conjunction with static dependencies) to
avoid unnecessary rebuilds.

A quick grep of the tree shows we have approximately 1948 packages that
were affected by the mass dev-util/pkgconfig -> virtual/pkgconfig migration.

Re: [gentoo-project] Re: Call for agenda items - Council meeting 2014-08-12

[hasufell]

Michael Palimaka:
> 
> but we need something [...] to avoid unnecessary rebuilds.
> 

People are already working on it. Join them if you find that it's an
important issue instead of forcing council votes that rather discourage
further efforts.

[gentoo-project] Re: Call for agenda items - Council meeting 2014-08-12

[Michael Palimaka]

On 08/02/2014 03:03 AM, hasufell wrote:
> Michael Palimaka:
>>
>> but we need something [...] to avoid unnecessary rebuilds.
>>
> 
> People are already working on it. Join them if you find that it's an
> important issue instead of forcing council votes that rather discourage
> further efforts.
> 
> 
Where are they working on it? We wouldn't have half this issue in the
first place if things were happening out in the open instead of behind
closed doors.

Re: [gentoo-project] Re: Call for agenda items - Council meeting 2014-08-12

[hasufell]

Michael Palimaka:
> On 08/02/2014 03:03 AM, hasufell wrote:
>> Michael Palimaka:
>>>
>>> but we need something [...] to avoid unnecessary rebuilds.
>>>
>>
>> People are already working on it. Join them if you find that it's an
>> important issue instead of forcing council votes that rather discourage
>> further efforts.
>>
>>
> Where are they working on it? We wouldn't have half this issue in the
> first place if things were happening out in the open instead of behind
> closed doors.
> 
> 

Last I checked both the #gentoo-portage channel where portage meetings
are held and the portage-devel ML are open.

Check those channels out.

[gentoo-project] Re: Call for agenda items - Council meeting 2014-08-12

[Michael Palimaka]

On 08/02/2014 03:37 AM, hasufell wrote:
> Michael Palimaka:
>> On 08/02/2014 03:03 AM, hasufell wrote:
>>> Michael Palimaka:
>>>>
>>>> but we need something [...] to avoid unnecessary rebuilds.
>>>>
>>>
>>> People are already working on it. Join them if you find that it's an
>>> important issue instead of forcing council votes that rather discourage
>>> further efforts.
>>>
>>>
>> Where are they working on it? We wouldn't have half this issue in the
>> first place if things were happening out in the open instead of behind
>> closed doors.
>>
>>
> 
> Last I checked both the #gentoo-portage channel where portage meetings
> are held and the portage-devel ML are open.
> 
> Check those channels out.

I monitor both those channels and have yet to see anything about work
towards avoiding unnecessary rebuilds. Please be more specific.

Re: [gentoo-project] Re: Call for agenda items - Council meeting 2014-08-12

[Samuli Suominen]

On 01/08/14 20:23, Michael Palimaka wrote:
> On 08/02/2014 03:03 AM, hasufell wrote:
>> Michael Palimaka:
>>> but we need something [...] to avoid unnecessary rebuilds.
>>>
>> People are already working on it. Join them if you find that it's an
>> important issue instead of forcing council votes that rather discourage
>> further efforts.
>>
>>
> Where are they working on it? We wouldn't have half this issue in the
> first place if things were happening out in the open instead of behind
> closed doors.
>
>

The workflow seems to have been...

1. Declare dynamic deps a /bug/
2. Tell people it will be disabled by force, without getting council
involved, and be quite rude about it...
3. Work on some replacement for the feature, development done mostly
silently,
    in a way most people didn't even know about it

When it should have been...

1. Work on some replacement for the feature, announce some design specifics
    of it in the ML, and explain it will be the replacement for the
dynamic deps
   which will be disabled as redudant. Get people involved with good
spirits.

(The above message is written as approx. and is not to be taken
literally or as an offense of anykind. Just saying
there was no need to get people up in arms if the plan was to provide
replacing feature all along.)

- Samuli

Re: [gentoo-project] Re: Call for agenda items - Council meeting 2014-08-12

[Tom Wijsman]

[-- Attachment #1: Type: text/plain, Size: 1422 bytes --]

On Fri, 01 Aug 2014 21:27:04 +0300
Samuli Suominen <ssuominen@gentoo.org> wrote:

> The workflow seems to have been...
> 
> 1. Declare dynamic deps a /bug/
> 2. Tell people it will be disabled by force, without getting council
> involved, and be quite rude about it...
> 3. Work on some replacement for the feature, development done mostly
> silently,
>     in a way most people didn't even know about it
> 
> When it should have been...
> 
> 1. Work on some replacement for the feature, announce some design
> specifics of it in the ML, and explain it will be the replacement for
> the dynamic deps
>    which will be disabled as redudant. Get people involved with good
> spirits.
> 
> (The above message is written as approx. and is not to be taken
> literally or as an offense of anykind. Just saying
> there was no need to get people up in arms if the plan was to provide
> replacing feature all along.)

These workflows declare that we need a replacement, but that is not
necessarily the plan; thus this workflow is a misrepresentation and
just one possible scenario. Not sure about "up in arms" but most of the
scenario has so far been discussion giving rise to ideas and motions.

-- 
With kind regards,

Tom Wijsman (TomWij)
Gentoo Developer

E-mail address  : TomWij@gentoo.org
GPG Public Key  : 6D34E57D
GPG Fingerprint : C165 AF18 AB4C 400B C3D2  ABF0 95B2 1FCD 6D34 E57D

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 473 bytes --]

[gentoo-project] Re: Call for agenda items - Council meeting 2014-08-12

[Michael Palimaka]

Thanks for taking the time to compose a reply that actually has some
substance - you're the only person involved with Portage that has
bothered so far.

On 08/01/2014 05:12 AM, Michał Górny wrote:

>> If we are to change our default dependency model, we need to do it
>> properly - we need wider consensus, more open discussion of what's
>> happening, and a proper plan of how to handle the pitfalls of the new
>> model. Otherwise, we're just trading one set of problems for another.
>
> Yes, exactly. We need to get dynamic-deps right if they are ever
> supposed to become the default. That's one of the reasons that we want
> to revert the problematic changes and make Portage use the default
> model once again.
What do you mean? Dynamic dependencies are the current default by
definition.

>> Specifically, I request the Council block the removal of dynamic
>> dependencies until the following issues are resolved:
>>
>> 1. Although there has been considerable discussion[2] regarding dynamic
>> dependencies in general, there has been no specific discussion regarding
>> their actual removal.
>
> The thread pretty quickly turned into a bikeshed about that, so I don't
> understand what you're talking about.
Perhaps I missed it due to the size of the thread, but the discussion
appeared to be more abstract rather than concerning any definite change.
I see relatively little comment from Portage team members in any case.

>> 2. The Portage team had made no announcement of their decision, nor do
>> they apparently intend to until 30 days prior to a new Portage release.
>> This is not adequate time for such a substantial change.
>
> I don't know what you mean by 'they apparently intend' but that sounds
> offensive. I'd really prefer if you sticked to the facts.
How is that offensive? It is the apparent intention I've been able to
discern from the limited information the Portage team is providing.

> The Portage team is going to announce the decision when it's final
> and the code necessary for non-painful migration is in place.
> Otherwise, the announcement would only bring barren discussion that
> couldn't be supported by facts or numbers.
This is not acceptable for such a significant change.

>> 3. Few of the Portage team members were present[3] for the meeting, and
>> no vote was held to reach the decision.
>
> I don't understand how this is relevant.
It's not acceptable for one or two people to make a decision that
affects the entire distribution, let alone if they don't have the
consensus of their own team.

>> 4. While there is a good description of the theoretical issues affecting
>> both dependency models[4], multiple requests for specific examples of
>> in-tree breakage have been ignored. This makes it difficult to assess
>> the actual breakage that removing dynamic dependencies is supposed to
>> address.
>
> The listing of theoretical issues is wrong and doesn't correspond to
> Portage code, and yes, that's my fault. However, it only proves that
> nobody on the thread bothered to look at the relevant Portage code,
> even the people who started providing patches...
The burden of proof is on the person wanting to make the change, so it
would be nice if the Portage team would provide better information.
Despite repeated requests, we have little information to substantiate
claims like "it's fundamentally broken" and "dynamic dependencies is a
bug. We decided to fix this regression."

>> 5. The removal plan doesn't consider the impact from increased number of
>> rebuilds due to revbumps containing only dependency changes. Without
>> replacement functionality, widespread virtual or slot changes can cause
>> hundreds of packages to be rebuilt.
>
> Please support this claim with specific examples or numbers. Otherwise,
> it's just a 'theoretical issue' that you can't support.
I'm happy to provide more examples beyond the 77
useless-but-apparently-acceptable rebuilds you mentioned below if
necessary. It's not hard to check yourself, though. Almost 9000 ebuilds
depend on virtual/pkgconfig. Even if we assume that any future virtual
introduction will only affect an absolute maximum of 10% of that number
of packages, that's still a ridiculous number of unnecessary rebuilds.

> There is a lot of FUD about unnecessary rebuilds. Sadly, most people
> seem to fight a holy war against them without realizing the real
> impact. In fact, more unnecessary rebuilds are caused by unnecessary
> USE flags than by dependency changes.
Do you have some numbers to back up that statement? I can think of very
few instances when I have had to rebuild against my will due to USE flag
changes.

> Yet the same people believe in
> adding more flags to contain even more minor aspects of packages...
Are you referring to flags for things like logrotate or systemd units?
When they were around, I either had them enabled or I didn't - I was
never forced to rebuild because of them.

Re: [gentoo-project] Re: Call for agenda items - Council meeting 2014-08-12

[Michał Górny]

[-- Attachment #1: Type: text/plain, Size: 10501 bytes --]

Dnia 2014-08-01, o godz. 05:52:40
Michael Palimaka <kensington@gentoo.org> napisał(a):

> On 08/01/2014 05:12 AM, Michał Górny wrote:
>   
> >> If we are to change our default dependency model, we need to do it
> >> properly - we need wider consensus, more open discussion of what's
> >> happening, and a proper plan of how to handle the pitfalls of the new
> >> model. Otherwise, we're just trading one set of problems for another.  
> > 
> > Yes, exactly. We need to get dynamic-deps right if they are ever
> > supposed to become the default. That's one of the reasons that we want
> > to revert the problematic changes and make Portage use the default
> > model once again.  
> What do you mean? Dynamic dependencies are the current default by
> definition.  

Unless I've missed something big, dynamic dependencies were enabled
in Portage without prior discussion or much testing. The developers
started to rely on it though that was never desired or agreed on. As
far as I'm aware, the Council never voted on the matter but it rather
spread from developer to developer. This also implies that it was never
properly described, and most of the developers do not know the details
or the pitfalls.

That said, I'd dare say that the Gentoo contract still obliges us to
follow PMS, and therefore the dependency model implied by PMS
is the default one.

> >> Specifically, I request the Council block the removal of dynamic
> >> dependencies until the following issues are resolved:
> >>
> >> 1. Although there has been considerable discussion[2] regarding dynamic
> >> dependencies in general, there has been no specific discussion regarding
> >> their actual removal.  
> > 
> > The thread pretty quickly turned into a bikeshed about that, so I don't
> > understand what you're talking about.  
> Perhaps I missed it due to the size of the thread, but the discussion
> appeared to be more abstract rather than concerning any definite change.
> I see relatively little comment from Portage team members in any case.
>   
> >> 2. The Portage team had made no announcement of their decision, nor do
> >> they apparently intend to until 30 days prior to a new Portage release.
> >> This is not adequate time for such a substantial change.  
> > 
> > I don't know what you mean by 'they apparently intend' but that sounds
> > offensive. I'd really prefer if you sticked to the facts.  
> How is that offensive? It is the apparent intention I've been able to
> discern from the limited information the Portage team is providing.
>   
> > The Portage team is going to announce the decision when it's final
> > and the code necessary for non-painful migration is in place.
> > Otherwise, the announcement would only bring barren discussion that
> > couldn't be supported by facts or numbers.  
> This is not acceptable for such a significant change.  

I think we misunderstand each other.

The Portage team is quite small, and has a lot of work to do. As I
mentioned, we are still collecting data and focusing on having the code
necessary to collect it. In particular, today I've submitted the last
patch necessary for @changed-deps set to work properly.

By using this set, we can check how many packages had dependencies
updated without revision bump and therefore estimate the scale
of the problem. Furthermore, if dynamic dependencies are disabled
in Portage, it will help users fixing the inconsistencies in their vdb
which may prevent emerge from working properly afterwards.

I simply meant that we haven't announced anything yet because we're
nowhere close to doing this. We don't want to bother people too much
before we know the scale of the problem, and before we can provide real
data and suggestions.

> >> 3. Few of the Portage team members were present[3] for the meeting, and
> >> no vote was held to reach the decision.  
> > 
> > I don't understand how this is relevant.  
> It's not acceptable for one or two people to make a decision that
> affects the entire distribution, let alone if they don't have the
> consensus of their own team.  

While I don't want to diminish the role of the remaining team members,
I would like to point out that those two were the team leads
and the most active members. And those members didn't reach a strong
conclusion -- as you pointed out, they never even voted. We just agreed
on a goal we're going to work on reaching.

> >> 4. While there is a good description of the theoretical issues affecting
> >> both dependency models[4], multiple requests for specific examples of
> >> in-tree breakage have been ignored. This makes it difficult to assess
> >> the actual breakage that removing dynamic dependencies is supposed to
> >> address.  
> > 
> > The listing of theoretical issues is wrong and doesn't correspond to
> > Portage code, and yes, that's my fault. However, it only proves that
> > nobody on the thread bothered to look at the relevant Portage code,
> > even the people who started providing patches...  
> The burden of proof is on the person wanting to make the change, so it
> would be nice if the Portage team would provide better information.
> Despite repeated requests, we have little information to substantiate
> claims like "it's fundamentally broken" and "dynamic dependencies is a
> bug. We decided to fix this regression."  

There are basically two sides to the issue: the issues with dynamic
dependencies themselves and the issues caused by developers relying
on them. An elaborate listing would likely require much more skill than
I have.

However, a short listing would include:

1. the dependency tree for installed packages is non-deterministic. It
can suffer rapid changes due to seemingly irrelevant events, most
importantly -- inability to find the original ebuild. This involves not
only removal of ebuilds but also temporary changes to PORTDIR_OVERLAY,
for example. Users don't expect that and the issues are very hard to
debug -- why is A pulling in non-existing package B? it's nowhere in
A's deps.

2. dynamic dependencies allow for dependencies stored in vdb to become
inconsistent. I mean, since portage usually doesn't use them
and the developers change dependencies in-place, the vdb dependency
tree may contain completely impossible combinations of packages
(depending on the time a particular package was installed). The problem
is that only emerge does this, and the vdb is simply broken to any
other package manager or tool -- including portage-utils (AFAICS
'qdepends -Q' gives meaningless output nowadays), eix, gentoolkit.

3. most of the developers don't understand when the revbump becomes
necessary because of dynamic-deps (i.e. when you don't want them to
apply). This occurs pretty rarely yet it's something I doubt we will
ever be able to change. It is simply too confusing.

4. getting := and dynamic-deps working together is pretty hard,
and the code doing that in Portage is not very good. I suspect that
the resulting dependency trees may be the cause of some issues people
are reporting. If backtracking is involved, dynamic-deps are also
likely to cause much longer dependency resolution. However, it's all
very complex and hard to debug, so I have no definitive proof.

I believe that fixing many of the issues in Portage will require severe
changes to the code. Disabling the dynamic dependency support will
likely make debugging easier for us, and therefore get us closer to
having properly working dependency resolver.

Additionally, disabled dynamic dependencies give users the ability to
choose any package manager for Gentoo. With dynamic dependencies
enabled, emerge breaks vdb pretty soon (as pointed out in (2)) which in
turn breaks other package managers, and leaves user with no choice but
to deepen the breakage.

> >> 5. The removal plan doesn't consider the impact from increased number of
> >> rebuilds due to revbumps containing only dependency changes. Without
> >> replacement functionality, widespread virtual or slot changes can cause
> >> hundreds of packages to be rebuilt.  
> > 
> > Please support this claim with specific examples or numbers. Otherwise,
> > it's just a 'theoretical issue' that you can't support.  
> I'm happy to provide more examples beyond the 77
> useless-but-apparently-acceptable rebuilds you mentioned below if
> necessary. It's not hard to check yourself, though. Almost 9000 ebuilds
> depend on virtual/pkgconfig. Even if we assume that any future virtual
> introduction will only affect an absolute maximum of 10% of that number
> of packages, that's still a ridiculous number of unnecessary rebuilds.  

If you assume that the immediate rebuild is actually a necessity.
Please note that many of the changes can actually be carried out within
a reasonable timeframe as part of package upgrade.

Given your example, the change was necessary to support a different
implementation of pkg-config. We may argue that choosing a non-default
implementation is pretty rare, and requesting those people to rebuild
a number of packages shouldn't really be a problem.

> > There is a lot of FUD about unnecessary rebuilds. Sadly, most people
> > seem to fight a holy war against them without realizing the real
> > impact. In fact, more unnecessary rebuilds are caused by unnecessary
> > USE flags than by dependency changes.  
> Do you have some numbers to back up that statement? I can think of very
> few instances when I have had to rebuild against my will due to USE flag
> changes.  

This is hard to count since it differs on per-case basis. I'm talking
about the two common cases here: when user doesn't know/notice that he
has to enable/disable a particular flag, and when user installs a new
package that requires USE changes on other packages.

> > Yet the same people believe in
> > adding more flags to contain even more minor aspects of packages...  
> Are you referring to flags for things like logrotate or systemd units?
> When they were around, I either had them enabled or I didn't - I was
> never forced to rebuild because of them.  

Rather flags for various package install aspects -- switching very
commonly used aspects, changing library install details. Think of
USE=tinfo on ncurses or attempt to add a flag to control library
install to pypy. But that's really irrelevant to the topic.

-- 
Best regards,
Michał Górny

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 949 bytes --]

Re: [gentoo-project] Call for agenda items - Council meeting 2014-08-12

[Michał Górny]

[-- Attachment #1: Type: text/plain, Size: 1915 bytes --]

Dnia 2014-07-29, o godz. 11:18:18
Ulrich Mueller <ulm@gentoo.org> napisał(a):

> In two weeks from now, the council will meet again. This is the time
> to raise and prepare items that the council should put on the agenda
> to discuss or vote on.
> 
> Please respond to this message with agenda items. Do not hesitate to
> repeat your agenda item here with a pointer if you previously
> suggested one (since the last meeting).

I would additionally like to ask Council to consider a three more items
for EAPI 6.

Specifically:

1. passing additional configure options (using the usual --help magic):

  a. --docdir: bug #173592 [1],

  b. --htmldir: bug #468202 [2],

  those pretty much follow the previously accepted doc
  install improvements. Ciaran had some doubts about passing --docdir
  but it seems that they were purely because of lack of --help check
  in Exherbo. Anyway, we'll do proper testing in Portage before the EAPI
  is finalized and we can revert those items if proven necessary.

2. additional default suffixes for dohtml: bug #423245 [3].

  I don't have a strong opinion on this one, just bringing it in line
  with other doc changes -- it would be easier do this now when we're
  changing a lot.

3. build-time switching variant of || (): bug #489458 [4].

  We need this for a long time and since we're going to need to rework
  || () logic within Portage anyway, EAPI 6 seems a perfect opportunity
  to finally fix this. The idea is that ||= () says that the package
  uses first package on the list that was installed during build-time,
  and needs rebuild every time you need to change the provider.

[1]:https://bugs.gentoo.org/show_bug.cgi?id=173592
[2]:https://bugs.gentoo.org/show_bug.cgi?id=468202
[3]:https://bugs.gentoo.org/show_bug.cgi?id=423245
[4]:https://bugs.gentoo.org/show_bug.cgi?id=489458

-- 
Best regards,
Michał Górny

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 949 bytes --]

Re: [gentoo-project] Call for agenda items - Council meeting 2014-08-12

[Ulrich Mueller]

[-- Attachment #1: Type: text/plain, Size: 1089 bytes --]

>>>>> On Tue, 5 Aug 2014, Michał Górny wrote:

> I would additionally like to ask Council to consider a three more
> items for EAPI 6.

> Specifically:

> 1. passing additional configure options (using the usual --help
> magic):
> [...]

> 2. additional default suffixes for dohtml: bug #423245 [3].
> [...]

> 3. build-time switching variant of || (): bug #489458 [4].

I'd rather not add further features to EAPI 6, otherwise it'll never
be finished. I don't see items 1 and 2 as essential features. In
addition, comments on the bugs for item 1 seem to indicate that it
will break some packages.

Concerning item 2, dohtml already has a -A option that allows adding
further extensions to the list. It doesn't have any option for their
removal, so I've mixed feelings about adding further extensions as
default.

Item 3 would be important enough to add it. OTOH, it still is in the
flow; your comment #14 was posted to the bug only today (!). There are
at least two previous examples where we have had bad experience with
such last-minute changes.

Ulrich

[-- Attachment #2: Type: application/pgp-signature, Size: 490 bytes --]

Re: [gentoo-project] Call for agenda items - Council meeting 2014-08-12

[Michał Górny]

Dnia 2014-08-05, o godz. 12:25:17
Ulrich Mueller <ulm@gentoo.org> napisał(a):

> >>>>> On Tue, 5 Aug 2014, Michał Górny wrote:
> 
> > I would additionally like to ask Council to consider a three more
> > items for EAPI 6.
> 
> > Specifically:
> 
> > 1. passing additional configure options (using the usual --help
> > magic):
> > [...]
> 
> > 2. additional default suffixes for dohtml: bug #423245 [3].
> > [...]
> 
> > 3. build-time switching variant of || (): bug #489458 [4].
> 
> I'd rather not add further features to EAPI 6, otherwise it'll never
> be finished. I don't see items 1 and 2 as essential features. In
> addition, comments on the bugs for item 1 seem to indicate that it
> will break some packages.

Comments indicate only that Exherbo done it the way that broke some
packages. However, we already have a good way of avoiding that
(--help). And since it is trivial to implement in-place in Portage, we
should be able to do a tinderbox run before the meeting.

> Concerning item 2, dohtml already has a -A option that allows adding
> further extensions to the list. It doesn't have any option for their
> removal, so I've mixed feelings about adding further extensions as
> default.

Me too but I want the Council to have the final say. IMHO dohtml is 
a big mistake that most developers see as wrapper for 'insinto; doins'
while instead it is a big pack of magic features.

> Item 3 would be important enough to add it. OTOH, it still is in the
> flow; your comment #14 was posted to the bug only today (!). There are
> at least two previous examples where we have had bad experience with
> such last-minute changes.

I doubt it can go anywhere far from it. My comment only summarizes most
of the stuff, and there's still time before the meeting. Considering
the agenda, I suspect that the meeting may even need to be continued
next week -- which brings us even more time.

Of course, there's matter of testing it. However, I am going to work
on || deps in Portage anyway. In the worst case, we can decide to drop
it from final EAPI 6 set. However, I see it as more likely to be
implemented than runtime USE.

--
Michał Górny