[gentoo-dev] Implicit system dependency

[gentoo-dev] Implicit system dependency

[Michael Orlitzky]

When I was taking my ebuild quizzes, I asked for someone to clarify the
implicit system dependency that we have enshrined in the devmanual:

  https://bugs.gentoo.org/show_bug.cgi?id=485356

There is... some agreement, but also special cases and special-special
cases that are folklore-only at this point. To me it seems like a fine
thing to ask the council to sort out, so I'm asking here for discussion.

Can we come up with an idiot-proof list (or FLOWCHART, even!) of what
should and should not be excluded from *DEPEND?

Re: [gentoo-dev] Implicit system dependency

[Rick "Zero_Chaos" Farina]

[-- Attachment #1: Type: text/plain, Size: 1025 bytes --]

On 11/04/2014 08:16 PM, Michael Orlitzky wrote:
> When I was taking my ebuild quizzes, I asked for someone to clarify the
> implicit system dependency that we have enshrined in the devmanual:
> 
>   https://bugs.gentoo.org/show_bug.cgi?id=485356
> 
> There is... some agreement, but also special cases and special-special
> cases that are folklore-only at this point. To me it seems like a fine
> thing to ask the council to sort out, so I'm asking here for discussion.
> 
> Can we come up with an idiot-proof list (or FLOWCHART, even!) of what
> should and should not be excluded from *DEPEND?
> 
> 
In my opinion, it's safe to ignore deps on glibc and gcc at this time, I
personally don't ignore any other deps.  On of my long term goals is to
remove as much as humanly possible from the system set and replace it
with the packages.default concept.  I can't say we are far enough to
actually do this yet, but that's why it's a long term goal.

https://bugs.gentoo.org/show_bug.cgi?id=393445

-Zero


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 884 bytes --]

Re: [gentoo-dev] Implicit system dependency

[Luca Barbato]

On 05/11/14 02:16, Michael Orlitzky wrote:
> When I was taking my ebuild quizzes, I asked for someone to clarify the
> implicit system dependency that we have enshrined in the devmanual:
>
>    https://bugs.gentoo.org/show_bug.cgi?id=485356
>
> There is... some agreement, but also special cases and special-special
> cases that are folklore-only at this point. To me it seems like a fine
> thing to ask the council to sort out, so I'm asking here for discussion.
>
> Can we come up with an idiot-proof list (or FLOWCHART, even!) of what
> should and should not be excluded from *DEPEND?
>

Assume a C runtime and a C compiler do exist.

Re: [gentoo-dev] Implicit system dependency

[Mike Gilbert]

On Wed, Nov 5, 2014 at 12:30 PM, Luca Barbato <lu_zero@gentoo.org> wrote:
> On 05/11/14 02:16, Michael Orlitzky wrote:
>>
>> When I was taking my ebuild quizzes, I asked for someone to clarify the
>> implicit system dependency that we have enshrined in the devmanual:
>>
>>    https://bugs.gentoo.org/show_bug.cgi?id=485356
>>
>> There is... some agreement, but also special cases and special-special
>> cases that are folklore-only at this point. To me it seems like a fine
>> thing to ask the council to sort out, so I'm asking here for discussion.
>>
>> Can we come up with an idiot-proof list (or FLOWCHART, even!) of what
>> should and should not be excluded from *DEPEND?
>>
>
> Assume a C runtime and a C compiler do exist.
>

I would extend that to a C++ compiler and library as well.

Re: [gentoo-dev] Implicit system dependency

[Luca Barbato]

On 05/11/14 18:49, Mike Gilbert wrote:
> On Wed, Nov 5, 2014 at 12:30 PM, Luca Barbato <lu_zero@gentoo.org> wrote:
>> On 05/11/14 02:16, Michael Orlitzky wrote:
>>>
>>> When I was taking my ebuild quizzes, I asked for someone to clarify the
>>> implicit system dependency that we have enshrined in the devmanual:
>>>
>>>     https://bugs.gentoo.org/show_bug.cgi?id=485356
>>>
>>> There is... some agreement, but also special cases and special-special
>>> cases that are folklore-only at this point. To me it seems like a fine
>>> thing to ask the council to sort out, so I'm asking here for discussion.
>>>
>>> Can we come up with an idiot-proof list (or FLOWCHART, even!) of what
>>> should and should not be excluded from *DEPEND?
>>>
>>
>> Assume a C runtime and a C compiler do exist.
>>
>
> I would extend that to a C++ compiler and library as well.

We are having yet another C++-moment (libstdc++ as usual) so it might 
change, please beware.

lu

[gentoo-dev] Re: Implicit system dependency

[Michael Palimaka]

On 05/11/14 12:16, Michael Orlitzky wrote:
> When I was taking my ebuild quizzes, I asked for someone to clarify the
> implicit system dependency that we have enshrined in the devmanual:
> 
>   https://bugs.gentoo.org/show_bug.cgi?id=485356
> 
> There is... some agreement, but also special cases and special-special
> cases that are folklore-only at this point. To me it seems like a fine
> thing to ask the council to sort out, so I'm asking here for discussion.
> 
> Can we come up with an idiot-proof list (or FLOWCHART, even!) of what
> should and should not be excluded from *DEPEND?
> 
> 

Suggested policy to get the ball rolling:

In general, a package must explicitly depend upon what it directly uses.
However, to avoid ebuild complexity and developer burden there are some
exceptions. Packages that appear in the base system set may be omitted
from an ebuild's dependency list in the following circumstances:

* C compiler and runtime

* C++ compiler and runtime

* A POSIX-compliant shell

* bash, baselayout, binutils, coreutils, findutils, grep, make

* Any archive tools (eg. tar, bzip2, xz-utils) where used for unpacking
SRC_URI only

* Any command guaranteed by PMS at build-time only (eg. sed, patch)

Note that in some cases it might be necessary to explicitly specify a
dependency that's listed above, such as when a specific implementation
is required (eg. a binary package requiring glibc).

Re: [gentoo-dev] Re: Implicit system dependency

[Michael Orlitzky]

On 11/13/2014 05:30 AM, Michael Palimaka wrote:
> 
> Suggested policy to get the ball rolling:
> 
> In general, a package must explicitly depend upon what it directly uses.
> However, to avoid ebuild complexity and developer burden there are some
> exceptions. Packages that appear in the base system set may be omitted
> from an ebuild's dependency list in the following circumstances:
> 
> * C compiler and runtime

Specifically sys-devel/gcc and sys-libs/glibc (i.e. what's in @system),
or just anything?

> * C++ compiler and runtime

Isn't it possible to disable C++ in GCC with USE="-cxx"?

Re: [gentoo-dev] Re: Implicit system dependency

[Ian Stakenvicius]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

On 13/11/14 09:05 AM, Michael Orlitzky wrote:
> On 11/13/2014 05:30 AM, Michael Palimaka wrote:
>> 
>> Suggested policy to get the ball rolling:
>> 
>> In general, a package must explicitly depend upon what it
>> directly uses. However, to avoid ebuild complexity and developer
>> burden there are some exceptions. Packages that appear in the
>> base system set may be omitted from an ebuild's dependency list
>> in the following circumstances:
>> 
>> * C compiler and runtime
> 
> Specifically sys-devel/gcc and sys-libs/glibc (i.e. what's in
> @system), or just anything?
> 

I would sincerely hope that nothing in the tree explicitly requires
gcc as a C compiler.

Glibc is a bit different, it may be necessary to explicitly depend on
it (or use the elibc_glibc flag) if the package can't work with the
libc alternatives, but ideally

>> * C++ compiler and runtime
> 
> Isn't it possible to disable C++ in GCC with USE="-cxx"?

It is..  but unfortunately there's no way in DEPEND to ensure it's
satisfied, as you can have a gcc installed with that flag enabled but
have a second one (that's actually selected in gcc-config) with it
disabled.  A pkg_pretend check or a pkg_setup check (if you don't want
it to just fail in src_configure) is probably the best way to enforce
that one at this time.  Unless there are other ways I'm not aware of??

There's also the whole c++98 vs c++11 issue that's sort-of part of
this -- minimum clang version, minimum gcc version might relate.
Again though, afaik there's no easy way to deal with this in DEPEND.
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2

iF4EAREIAAYFAlRky4AACgkQ2ugaI38ACPD1twD/da6rptWk1/vl2iDPSBWLmox2
5rXr7aEci8yCBoyDsk8A/0ZAGBtxlBWqoTGKzkJdm32pow4cOtFBEBO+YoVJkEyx
=xXHM
-----END PGP SIGNATURE-----

Re: [gentoo-dev] Re: Implicit system dependency

[Ian Stakenvicius]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

On 13/11/14 10:17 AM, Ian Stakenvicius wrote:
> On 13/11/14 09:05 AM, Michael Orlitzky wrote:
>> On 11/13/2014 05:30 AM, Michael Palimaka wrote:
>>> 
>>> Suggested policy to get the ball rolling:
>>> 
>>> In general, a package must explicitly depend upon what it 
>>> directly uses. However, to avoid ebuild complexity and
>>> developer burden there are some exceptions. Packages that
>>> appear in the base system set may be omitted from an ebuild's
>>> dependency list in the following circumstances:
>>> 
>>> * C compiler and runtime
> 
>> Specifically sys-devel/gcc and sys-libs/glibc (i.e. what's in 
>> @system), or just anything?
> 
> 
> I would sincerely hope that nothing in the tree explicitly
> requires gcc as a C compiler.
> 
> Glibc is a bit different, it may be necessary to explicitly depend
> on it (or use the elibc_glibc flag) if the package can't work with
> the libc alternatives, but ideally [...]

... we shouldn't be depending on the specific libc implementation
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2

iF4EAREIAAYFAlRky8wACgkQ2ugaI38ACPBEEwD+JmErQK2aUPcYsZY6e55lWYfO
oenrhAK3S4bKX8CdOWoA/1NKBesQnsv6e8KEwPEQrHlQO3DcCA/DVVWPWjUSVCjo
=+Web
-----END PGP SIGNATURE-----

Re: [gentoo-dev] Re: Implicit system dependency

[Matt Turner]

On Thu, Nov 13, 2014 at 7:17 AM, Ian Stakenvicius <axs@gentoo.org> wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA256
>
> On 13/11/14 09:05 AM, Michael Orlitzky wrote:
>> On 11/13/2014 05:30 AM, Michael Palimaka wrote:
>>>
>>> Suggested policy to get the ball rolling:
>>>
>>> In general, a package must explicitly depend upon what it
>>> directly uses. However, to avoid ebuild complexity and developer
>>> burden there are some exceptions. Packages that appear in the
>>> base system set may be omitted from an ebuild's dependency list
>>> in the following circumstances:
>>>
>>> * C compiler and runtime
>>
>> Specifically sys-devel/gcc and sys-libs/glibc (i.e. what's in
>> @system), or just anything?
>>
>
> I would sincerely hope that nothing in the tree explicitly requires
> gcc as a C compiler.

You say this, and then mention glibc in the next sentence. Glibc can
only be built with gcc. :)

> Glibc is a bit different, it may be necessary to explicitly depend on
> it (or use the elibc_glibc flag) if the package can't work with the
> libc alternatives, but ideally

Re: [gentoo-dev] Re: Implicit system dependency

[Ian Stakenvicius]

> On Nov 15, 2014, at 3:57 PM, Matt Turner <mattst88@gentoo.org> wrote:
> 
>> On Thu, Nov 13, 2014 at 7:17 AM, Ian Stakenvicius <axs@gentoo.org> wrote:
>> -----BEGIN PGP SIGNED MESSAGE-----
>> Hash: SHA256
>> 
>>> On 13/11/14 09:05 AM, Michael Orlitzky wrote:
>>>> On 11/13/2014 05:30 AM, Michael Palimaka wrote:
>>>> 
>>>> Suggested policy to get the ball rolling:
>>>> 
>>>> In general, a package must explicitly depend upon what it
>>>> directly uses. However, to avoid ebuild complexity and developer
>>>> burden there are some exceptions. Packages that appear in the
>>>> base system set may be omitted from an ebuild's dependency list
>>>> in the following circumstances:
>>>> 
>>>> * C compiler and runtime
>>> 
>>> Specifically sys-devel/gcc and sys-libs/glibc (i.e. what's in
>>> @system), or just anything?
>> 
>> I would sincerely hope that nothing in the tree explicitly requires
>> gcc as a C compiler.
> 
> You say this, and then mention glibc in the next sentence. Glibc can
> only be built with gcc. :)
> 

Sorry, I meant to say nothing other than toolchain-related packages :). Thanks for clarifying for me 


>> Glibc is a bit different, it may be necessary to explicitly depend on
>> it (or use the elibc_glibc flag) if the package can't work with the
>> libc alternatives, but ideally
> 

[gentoo-dev] Re: Implicit system dependency

[Michael Palimaka]

On 14/11/14 01:05, Michael Orlitzky wrote:
> On 11/13/2014 05:30 AM, Michael Palimaka wrote:
>>
>> Suggested policy to get the ball rolling:
>>
>> In general, a package must explicitly depend upon what it directly uses.
>> However, to avoid ebuild complexity and developer burden there are some
>> exceptions. Packages that appear in the base system set may be omitted
>> from an ebuild's dependency list in the following circumstances:
>>
>> * C compiler and runtime
> 
> Specifically sys-devel/gcc and sys-libs/glibc (i.e. what's in @system),
> or just anything?

Since they can be (in theory) replaced with some other implementation,
anything unless there's a reason (like a binary package explicitly
linking to glib).

> 
>> * C++ compiler and runtime
> 
> Isn't it possible to disable C++ in GCC with USE="-cxx"?

It is, but I think if that's disabled you're on your own. :-)

Re: [gentoo-dev] Re: Implicit system dependency

[hasufell]

On 11/13/2014 04:27 PM, Michael Palimaka wrote:
>>> * C++ compiler and runtime
>>
>> Isn't it possible to disable C++ in GCC with USE="-cxx"?
> 
> It is, but I think if that's disabled you're on your own. :-)
> 

I keep hearing this sentence, but it still doesn't make much sense to
me. Invalid configurations should be reported as invalid to the user.

Since I run my own server with USE="-* foo bar" I have come across a lot
of "expect users to not disable it, but don't actually disallow it or
even warn them".

I've seen a lot of compiler checks in ebuilds in pkg_pretend that look
quite similar. Maybe it's time to enhance toolchain-funcs?

[gentoo-dev] Re: Implicit system dependency

[Michael Palimaka]

On 14/11/14 03:57, hasufell wrote:
> On 11/13/2014 04:27 PM, Michael Palimaka wrote:
>>>> * C++ compiler and runtime
>>>
>>> Isn't it possible to disable C++ in GCC with USE="-cxx"?
>>
>> It is, but I think if that's disabled you're on your own. :-)
>>
> 
> I keep hearing this sentence, but it still doesn't make much sense to
> me. Invalid configurations should be reported as invalid to the user.
> 
> Since I run my own server with USE="-* foo bar" I have come across a lot
> of "expect users to not disable it, but don't actually disallow it or
> even warn them".
> 
> I've seen a lot of compiler checks in ebuilds in pkg_pretend that look
> quite similar. Maybe it's time to enhance toolchain-funcs?

It definitely would be nice to have some better mechanism for compiler
checking in general. Already version checking is quite ugly with
boilerplate code everywhere with wrong dependencies like >=gcc-4.7.

Re: [gentoo-dev] Re: Implicit system dependency

[Mike Gilbert]

On Thu, Nov 13, 2014 at 10:27 AM, Michael Palimaka
<kensington@gentoo.org> wrote:
> On 14/11/14 01:05, Michael Orlitzky wrote:
>> Isn't it possible to disable C++ in GCC with USE="-cxx"?
>
> It is, but I think if that's disabled you're on your own. :-)

Perhaps we should add a package.use.force entry for this. Is there any
reason not to?

Re: [gentoo-dev] Re: Implicit system dependency

[Alexander Hof]

Mike Gilbert wrote:
> On Thu, Nov 13, 2014 at 10:27 AM, Michael Palimaka
> <kensington@gentoo.org> wrote:
>> On 14/11/14 01:05, Michael Orlitzky wrote:
>>> Isn't it possible to disable C++ in GCC with USE="-cxx"?
>>
>> It is, but I think if that's disabled you're on your own. :-)
> 
> Perhaps we should add a package.use.force entry for this. Is there any
> reason not to?
> 

There are people that don't want c++ and gcc:4.7 can still bootstrap
without.

Re: [gentoo-dev] Re: Implicit system dependency

[Mike Gilbert]

On Thu, Nov 13, 2014 at 5:10 PM, Alexander Hof <gentoodev@cosmofox.net> wrote:
> Mike Gilbert wrote:
>> On Thu, Nov 13, 2014 at 10:27 AM, Michael Palimaka
>> <kensington@gentoo.org> wrote:
>>> On 14/11/14 01:05, Michael Orlitzky wrote:
>>>> Isn't it possible to disable C++ in GCC with USE="-cxx"?
>>>
>>> It is, but I think if that's disabled you're on your own. :-)
>>
>> Perhaps we should add a package.use.force entry for this. Is there any
>> reason not to?
>>
>
> There are people that don't want c++ and gcc:4.7 can still bootstrap
> without.
>

Those people "know what they are doing" and could un-force the use
flag. That would prevent people from accidentally disabling it via
USE="-*".

I'm not normally one to prevent people from shooting themselves, but
in this case the safety would be simple to toggle.

Re: [gentoo-dev] Re: Implicit system dependency

[Alexander Hof]

Mike Gilbert wrote:
>> There are people that don't want c++ and gcc:4.7 can still bootstrap
>> without.
>>
> 
> Those people "know what they are doing" and could un-force the use
> flag. That would prevent people from accidentally disabling it via
> USE="-*".

Are we talking about forcing +cxx globally or for gcc (+toolchain)?

Has this been a major problem in the past? Shouldn't people who set
USE="-*" also "know what they are doing"?

> I'm not normally one to prevent people from shooting themselves, but
> in this case the safety would be simple to toggle.

What if there was some other reason someone had to set a +cxx for some
package in package.use.force, and I'm missing out that reason with my
-cxx in profiles/package.use.force that I need to circumvent your
safety-measure?

Re: [gentoo-dev] Re: Implicit system dependency

[hasufell]

On 11/14/2014 11:42 PM, Alexander Hof wrote:
> Mike Gilbert wrote:
>>> There are people that don't want c++ and gcc:4.7 can still bootstrap
>>> without.
>>>
>>
>> Those people "know what they are doing" and could un-force the use
>> flag. That would prevent people from accidentally disabling it via
>> USE="-*".
> 
> Are we talking about forcing +cxx globally or for gcc (+toolchain)?
> 
> Has this been a major problem in the past? Shouldn't people who set
> USE="-*" also "know what they are doing"?
> 

* don't ever assume that the user knows what he is doing
* still allow him to break things if he really chooses to
* don't make excuses for gentoo bugs like not properly checking the
toolchain for validity, including necessary package
dependencies/requirements

Re: [gentoo-dev] Re: Implicit system dependency

[Alexander Hof]

hasufell wrote:

>> Are we talking about forcing +cxx globally or for gcc (+toolchain)?
>>
>> Has this been a major problem in the past? Shouldn't people who set
>> USE="-*" also "know what they are doing"?
>>
> 
> * don't ever assume that the user knows what he is doing
> * still allow him to break things if he really chooses to
> * don't make excuses for gentoo bugs like not properly checking the
> toolchain for validity, including necessary package
> dependencies/requirements

+1, that's what I wanted to express.

Re: [gentoo-dev] Re: Implicit system dependency

[William Hubbs]

[-- Attachment #1: Type: text/plain, Size: 668 bytes --]

On Fri, Nov 14, 2014 at 11:42:57PM +0100, Alexander Hof wrote:
> Mike Gilbert wrote:
> >> There are people that don't want c++ and gcc:4.7 can still bootstrap
> >> without.
> >>
> > 
> > Those people "know what they are doing" and could un-force the use
> > flag. That would prevent people from accidentally disabling it via
> > USE="-*".
> 
> Are we talking about forcing +cxx globally or for gcc (+toolchain)?
> 
> Has this been a major problem in the past? Shouldn't people who set
> USE="-*" also "know what they are doing"?

This is my feeling as well.

If someone using Gentoo uses USE="-* foo bar ..." they get to keep the
pieces.

William

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 181 bytes --]

Re: [gentoo-dev] Re: Implicit system dependency

[hasufell]

On 11/17/2014 09:40 PM, William Hubbs wrote:
> On Fri, Nov 14, 2014 at 11:42:57PM +0100, Alexander Hof wrote:
>> Mike Gilbert wrote:
>>>> There are people that don't want c++ and gcc:4.7 can still bootstrap
>>>> without.
>>>>
>>>
>>> Those people "know what they are doing" and could un-force the use
>>> flag. That would prevent people from accidentally disabling it via
>>> USE="-*".
>>
>> Are we talking about forcing +cxx globally or for gcc (+toolchain)?
>>
>> Has this been a major problem in the past? Shouldn't people who set
>> USE="-*" also "know what they are doing"?
> 
> This is my feeling as well.
> 
> If someone using Gentoo uses USE="-* foo bar ..." they get to keep the
> pieces.
> 
> William
> 

Using USE="-*" reveals so many random assumptions and untested ebuild
configurations that we should definitely rethink that sentiment.

And arch testers partly do exactly that.
So I think it's an excuse for bad ebuild USE flags and dependencies.
If your ebuild does not compile with USE="-*" (except because of
REQUIRED_USE or pkg_setup bailing out), then you did something wrong.

People already use this configuration and all related bug reports are valid.

Re: [gentoo-dev] Re: Implicit system dependency

[Andreas K. Huettel]

[-- Attachment #1: Type: Text/Plain, Size: 1361 bytes --]

Am Montag, 17. November 2014, 22:36:10 schrieb hasufell:
> > 
> > If someone using Gentoo uses USE="-* foo bar ..." they get to keep the
> > pieces.
> > 
> > William
> 
> Using USE="-*" reveals so many random assumptions and untested ebuild
> configurations that we should definitely rethink that sentiment.
> 
> And arch testers partly do exactly that.
> So I think it's an excuse for bad ebuild USE flags and dependencies.
> If your ebuild does not compile with USE="-*" (except because of
> REQUIRED_USE or pkg_setup bailing out), then you did something wrong.
> 
> People already use this configuration and all related bug reports are
> valid.

That's at most an argument that USE="-*" should be a theoretically valid 
configuration. It does not mean that the setting makes sense for anyone.

USE="-*" was maybe a reasonable idea before we had use defaults. 

Now, by setting USE="-*", you deviate from upstream defaults at random places 
and pointlessly mess up the dependency calculations of python / ruby / 
multilib / ... packages.

Message to users- if you want a minimum set of useflags, start from the main 
default profile of your arch. That's what it is for. Everything else, and you 
sure get to keep the pieces.

-- 

Andreas K. Huettel
Gentoo Linux developer 
dilfridge@gentoo.org
http://www.akhuettel.de/


[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 951 bytes --]

Re: [gentoo-dev] Re: Implicit system dependency

[hasufell]

On 11/18/2014 12:05 AM, Andreas K. Huettel wrote:
> USE="-*" was maybe a reasonable idea before we had use defaults. 
> 
> Now, by setting USE="-*", you deviate from upstream defaults at random places 
> and pointlessly mess up the dependency calculations of python / ruby / 
> multilib / ... packages.
> 

Those necessary USE expands are set of course. And the python USE deps
and USE flag deps are pretty much correct, so there is no surprise on
that front. You get tons of correct warnings about unmet stuff and it's
quite trivial to fix these.

I didn't know that gentoo is about "defaults". If anything... that's
what most gentoo users don't care about in my experience. They want
control and correct dependencies without random assumptions.

I'm not really sure if we still disagree, except that you think it's
"dangerous". Sure it is, especially when ebuild deps are wrong.

That said... if disabling a USE flag breaks 300+ packages, then there
are a few possibilities:
* fix all deps in those 300+ packages (seems like a waste of time here,
but is still correct)
* make it difficult to disable the USE flag and spit lots of warnings
(which is rather a hack)
* remove the flag (maybe provide unsupported hackery in the toolchain
overlay)

I personally don't have a strong opinion on any of those solutions. But
I'm quite tired of people telling me how to use gentoo and what to
expect about correctness of dependencies.

Re: [gentoo-dev] Re: Implicit system dependency

[Andreas K. Huettel]

[-- Attachment #1: Type: Text/Plain, Size: 744 bytes --]

Am Dienstag, 18. November 2014, 00:38:36 schrieb hasufell:

> I personally don't have a strong opinion on any of those solutions. But
> I'm quite tired of people telling me how to use gentoo and what to
> expect about correctness of dependencies.

Earth to hasufell. Please stop confusing people. We all know you can handle 
the resulting mess quite well. We just don't want to answer a thousand 
questions when things break for others. That is the whole point of sane 
defaults.

(Besides one of the ideas of Gentoo *is* to stick to upstream settings as much 
as possible. Which you deliberately break when setting USE="-*".)

-- 

Andreas K. Huettel
Gentoo Linux developer 
dilfridge@gentoo.org
http://www.akhuettel.de/


[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 951 bytes --]

Re: [gentoo-dev] Re: Implicit system dependency

[hasufell]

On 11/18/2014 12:47 AM, Andreas K. Huettel wrote:
> Am Dienstag, 18. November 2014, 00:38:36 schrieb hasufell:
> 
> We just don't want to answer a thousand 
> questions when things break for others. That is the whole point of sane 
> defaults.
> 

Except that sane defaults are not a substitute for correct dependencies
(like people omitting USE flag deps on libsdl, because they assume users
won't disable them).

Also, you don't have to answer questions if it's clear that certain
settings break stuff and what they break. There are ways to communicate
this (even in USE flag descriptions).
If you don't communicate it, then you will have to answers questions...

Re: [gentoo-dev] Re: Implicit system dependency

[Ian Stakenvicius]

> On Nov 17, 2014, at 7:03 PM, hasufell <hasufell@gentoo.org> wrote:
> 
>> On 11/18/2014 12:47 AM, Andreas K. Huettel wrote:
>> Am Dienstag, 18. November 2014, 00:38:36 schrieb hasufell:
>> 
>> We just don't want to answer a thousand 
>> questions when things break for others. That is the whole point of sane 
>> defaults.
> 
> Except that sane defaults are not a substitute for correct dependencies
> (like people omitting USE flag deps on libsdl, because they assume users
> won't disable them).
> 
> Also, you don't have to answer questions if it's clear that certain
> settings break stuff and what they break. There are ways to communicate
> this (even in USE flag descriptions).
> If you don't communicate it, then you will have to answers questions...
> 

Can we all agree that dependencies should be correct regardless of the use flag settings?  And leave the rest of this discussion to the bikeshed it belongs in ? :)

Re: [gentoo-dev] Re: Implicit system dependency

[Mike Gilbert]

On Mon, Nov 17, 2014 at 7:08 PM, Ian Stakenvicius <axs@gentoo.org> wrote:
>
>
>> On Nov 17, 2014, at 7:03 PM, hasufell <hasufell@gentoo.org> wrote:
>>
>>> On 11/18/2014 12:47 AM, Andreas K. Huettel wrote:
>>> Am Dienstag, 18. November 2014, 00:38:36 schrieb hasufell:
>>>
>>> We just don't want to answer a thousand
>>> questions when things break for others. That is the whole point of sane
>>> defaults.
>>
>> Except that sane defaults are not a substitute for correct dependencies
>> (like people omitting USE flag deps on libsdl, because they assume users
>> won't disable them).
>>
>> Also, you don't have to answer questions if it's clear that certain
>> settings break stuff and what they break. There are ways to communicate
>> this (even in USE flag descriptions).
>> If you don't communicate it, then you will have to answers questions...
>>
>
> Can we all agree that dependencies should be correct regardless of the use flag settings?  And leave the rest of this discussion to the bikeshed it belongs in ? :)

Indeed.

Back to the original topic: as I understand it, toolchain deps are
just really hard to do correctly and would increase the complexity of
the average ebuild quite a lot, which is why we don't try. Especially
when you introduce the possibility of cross-compilation.

[gentoo-dev] Re: Implicit system dependency

[Duncan]

Andreas K. Huettel posted on Tue, 18 Nov 2014 00:05:03 +0100 as excerpted:


> Message to users- if you want a minimum set of useflags, start from the
> main default profile of your arch. That's what it is for. Everything
> else, and you sure get to keep the pieces.

But for no-multilib, there's still both far too many USE flags enabled by 
default, and far too much stuff in @system, once dependencies are 
included.

So here I'm on about the only amd64-no-multilib profile available, but 
with all @system packages negated and with USE="-* ...".

And you know what?  It actually works very well! =:^)

Tho obviously I was a reasonably mature gentoo user before I tried it, I 
already had my main system setup, and when I set USE=-* I did an emerge
--verbose --pretend and looked at what flags on what packages would 
change, before deciding whether that was sane and what I wanted, or not, 
and setting specific flags either globally or per-package accordingly.

Similarly with the @system package negation.  Negate the list a few 
packages at a time and do an emerge --pretend --depclean and see what 
comes up.  If it's sane, unmerge it.  Else add it to the appropriate 
custom set that's covered by the world-sets file.

Nothing really dangerous or insane about that, as long as you take it a 
step at a time using --pretend first, and don't let it /do/ anything 
insane -- make the necessary set and USE flag changes to prevent emerge 
trying to remove something critical, and /then/ let it have at it with 
the rest.

Like I said, as could be expected with gentoo, which routinely takes into 
account that users /might/ have a good reason for not wanting the 
defaults, it actually works really well. =:^)


Tho I actually appreciate the "you get to keep the pieces" aspect as 
well.  Unlike many distros, gentoo actually respects the user and their 
right to decide enough to give them the /power/ to break the system, if 
they "drink and emerge", or similar foolish things.  The guard rails are 
there and that's appreciated, but there's also unlocked gates (with clear 
warnings on them) thru those guard rails, because that's what gentoo is 
/about/.  Sure, people can and do go thru those gates from time to time, 
but it's their responsibility to be appropriately roped up if they do, 
and if they can't do that and end up falling off the gentoo cliff and 
landing on arch or fedora (or even osx or ms windows!) instead, well, it 
was probably for the best.

So let's keep the warnings there as they do warn the unwary, but also the 
unlocked gates thru those default-use railings.  =:^)

-- 
Duncan - List replies preferred.   No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master."  Richard Stallman

Re: [gentoo-dev] Re: Implicit system dependency

[Vadim A. Misbakh-Soloviov]

[-- Attachment #1: Type: text/plain, Size: 1173 bytes --]

В письме от Вт, 18 ноября 2014 03:28:08 пользователь Duncan написал:
> Tho I actually appreciate the "you get to keep the pieces" aspect as
> Unlike many distros, gentoo actually respects the user and their
> right to decide enough to give them the /power/ to break the system, if
> they "drink and emerge", or similar foolish things.  The guard rails are
> there and that's appreciated, but there's also unlocked gates (with clear
> warnings on them) thru those guard rails, because that's what gentoo is
> /about/.  Sure, people can and do go thru those gates from time to time,
> but it's their responsibility to be appropriately roped up if they do,
> and if they can't do that and end up falling off the gentoo cliff and
> landing on arch or fedora (or even osx or ms windows!) instead, well, it
> was probably for the best.
> 
> So let's keep the warnings there as they do warn the unwary, but also the
> unlocked gates thru those default-use railings.  =:^)

That message should be sent as auto-reply to every 1.5 messages in this thread 
(and maybe in entire gentoo-dev, I think)! :)

-- 
Best regards,
mva

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 819 bytes --]

Re: [gentoo-dev] Re: Implicit system dependency

[William Hubbs]

[-- Attachment #1: Type: text/plain, Size: 889 bytes --]

On Tue, Nov 18, 2014 at 12:05:03AM +0100, Andreas K. Huettel wrote:
> That's at most an argument that USE="-*" should be a theoretically valid 
> configuration. It does not mean that the setting makes sense for anyone.
> 
> USE="-*" was maybe a reasonable idea before we had use defaults. 
> 
> Now, by setting USE="-*", you deviate from upstream defaults at random places 
> and pointlessly mess up the dependency calculations of python / ruby / 
> multilib / ... packages.
> 
> Message to users- if you want a minimum set of useflags, start from the main 
> default profile of your arch. That's what it is for. Everything else, and you 
> sure get to keep the pieces.

Agreed. If you want to turn things off, I would recommend starting your
use with something like:

USE="-foo -bar -bas ..."

so that you turn off the specific things you want to turn off.

William

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 181 bytes --]

Re: [gentoo-dev] Re: Implicit system dependency

[Gordon Pettey]

[-- Attachment #1: Type: text/plain, Size: 1443 bytes --]

On Sat, Nov 22, 2014 at 1:14 PM, William Hubbs <williamh@gentoo.org> wrote:

> On Tue, Nov 18, 2014 at 12:05:03AM +0100, Andreas K. Huettel wrote:
> > That's at most an argument that USE="-*" should be a theoretically valid
> > configuration. It does not mean that the setting makes sense for anyone.
> >
> > USE="-*" was maybe a reasonable idea before we had use defaults.
> >
> > Now, by setting USE="-*", you deviate from upstream defaults at random
> places
> > and pointlessly mess up the dependency calculations of python / ruby /
> > multilib / ... packages.
> >
> > Message to users- if you want a minimum set of useflags, start from the
> main
> > default profile of your arch. That's what it is for. Everything else,
> and you
> > sure get to keep the pieces.
>
> Agreed. If you want to turn things off, I would recommend starting your
> use with something like:
>
> USE="-foo -bar -bas ..."
>
> so that you turn off the specific things you want to turn off.


That's quite infeasible given the number of package-level defaults. It is
far easier to parse conflicts when I know anything that has been enabled
was explicitly enabled by myself,and not through random-maintainer-X's
preference. 3743 package-level defaults of 1474 USEs is just a few too
many. Starting with USE="-*" provides sanity. As has been said so many
times in this and related threads, if users wanted upstream's defaults, we
wouldn't be using a distro with USE.

[-- Attachment #2: Type: text/html, Size: 1927 bytes --]

[gentoo-dev] Re: Implicit system dependency

[Duncan]

Mike Gilbert posted on Fri, 14 Nov 2014 15:55:10 -0500 as excerpted:

> On Thu, Nov 13, 2014 at 5:10 PM, Alexander Hof <gentoodev@cosmofox.net>
> wrote:
>> Mike Gilbert wrote:
>>> On Thu, Nov 13, 2014 at 10:27 AM, Michael Palimaka
>>> <kensington@gentoo.org> wrote:
>>>> On 14/11/14 01:05, Michael Orlitzky wrote:
>>>>> Isn't it possible to disable C++ in GCC with USE="-cxx"?
>>>>
>>>> It is, but I think if that's disabled you're on your own. :-)
>>>
>>> Perhaps we should add a package.use.force entry for this. Is there any
>>> reason not to?
>>>
>>>
>> There are people that don't want c++ and gcc:4.7 can still bootstrap
>> without.
>>
>>
> Those people "know what they are doing" and could un-force the use flag.
> That would prevent people from accidentally disabling it via USE="-*".
> 
> I'm not normally one to prevent people from shooting themselves, but in
> this case the safety would be simple to toggle.

The problem is the precedent that sets.  (Package.)use.force is the 
equivalent of "going nuke" and IMO it should be kept that way.

^^ TL;DR ^^

Overriding use.force (package or otherwise) should be clearly no-man's-
land, something done only for stuff like cross-compiling, etc, so far out 
of gentoo support that normally nobody sane would ask for it under those 
conditions, and if they did, they'd clearly mention the conditions and 
explain why they thought the override was necessary and didn't affect 
whatever bug they were seeing.

Use.forcing something isn't even /close/ to default-use, which can be and 
is routinely overridden by people (including me) who "know what they are 
doing" with USE=-*, and as soon as use.forcing something, and then 
overriding it, becomes normal and routine, then we'll need a "really 
force" option to override use.force, much like use.force was the "really 
force" option to override use-defaults.

So I'd suggest (package.)use.forcing isn't appropriate with c++, because 
there /are/ legitimate reasons to unset it, as already discussed, and as 
soon as people start having to do that with the one flag, nobody user or 
dev in gentoo is going to be able to assume (package.)use.force isn't 
routinely overridden ever again.   Again, then we'll need a 
(package.)use.force.really option, and the race will be on to 
(package.)use.force.yes.I.really.really.really.really.mean.it!

-- 
Duncan - List replies preferred.   No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master."  Richard Stallman

Re: [gentoo-dev] Re: Implicit system dependency

[Michał Górny]

[-- Attachment #1: Type: text/plain, Size: 580 bytes --]

Dnia 2014-11-13, o godz. 13:13:01
Mike Gilbert <floppym@gentoo.org> napisał(a):

> On Thu, Nov 13, 2014 at 10:27 AM, Michael Palimaka
> <kensington@gentoo.org> wrote:
> > On 14/11/14 01:05, Michael Orlitzky wrote:
> >> Isn't it possible to disable C++ in GCC with USE="-cxx"?
> >
> > It is, but I think if that's disabled you're on your own. :-)
> 
> Perhaps we should add a package.use.force entry for this. Is there any
> reason not to?

You may want to install a minimalistic version of an old gcc for some
(non-C++) testing.

-- 
Best regards,
Michał Górny

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 949 bytes --]

Re: [gentoo-dev] Re: Implicit system dependency

[Michael Orlitzky]

On 11/13/2014 01:13 PM, Mike Gilbert wrote:
> On Thu, Nov 13, 2014 at 10:27 AM, Michael Palimaka
> <kensington@gentoo.org> wrote:
>> On 14/11/14 01:05, Michael Orlitzky wrote:
>>> Isn't it possible to disable C++ in GCC with USE="-cxx"?
>>
>> It is, but I think if that's disabled you're on your own. :-)
> 
> Perhaps we should add a package.use.force entry for this. Is there any
> reason not to?
> 

Under the assumption that gcc[cxx] is needed for the package manager,
you still only need *one* GCC with C++ support. You could have another
slot without C++ for testing or whatever.

Re: [gentoo-dev] Re: Implicit system dependency

[Andrew Savchenko]

[-- Attachment #1: Type: text/plain, Size: 965 bytes --]

On Fri, 14 Nov 2014 09:10:43 -0500 Michael Orlitzky wrote:
> On 11/13/2014 01:13 PM, Mike Gilbert wrote:
> > On Thu, Nov 13, 2014 at 10:27 AM, Michael Palimaka
> > <kensington@gentoo.org> wrote:
> >> On 14/11/14 01:05, Michael Orlitzky wrote:
> >>> Isn't it possible to disable C++ in GCC with USE="-cxx"?
> >>
> >> It is, but I think if that's disabled you're on your own. :-)
> > 
> > Perhaps we should add a package.use.force entry for this. Is there any
> > reason not to?
> > 
> 
> Under the assumption that gcc[cxx] is needed for the package manager,
> you still only need *one* GCC with C++ support. You could have another
> slot without C++ for testing or whatever.

Seconded. Simple practical example (aside from testing) from my
system: I need libg2c.so for old crappy fortran software I have to
use. That's why I'm keeping sys-devel:3.4[fortran nls] around. I
definitely do not need cxx flag here.

Best regards,
Andrew Savchenko

[-- Attachment #2: Type: application/pgp-signature, Size: 819 bytes --]

Re: [gentoo-dev] Re: Implicit system dependency

[Rich Freeman]

On Thu, Nov 13, 2014 at 5:30 AM, Michael Palimaka <kensington@gentoo.org> wrote:
>
> In general, a package must explicitly depend upon what it directly uses.
> However, to avoid ebuild complexity and developer burden there are some
> exceptions. Packages that appear in the base system set may be omitted
> from an ebuild's dependency list in the following circumstances:
>

So, I'm not a big fan of implicit dependencies in general.  What does
the new policy buy us that the existing one does not?  Why not try to
find a way to ditch implicit dependencies entirely?  If the issue is
that nobody wants to have a laundry list of dependencies in every
package, why not use something like a virtual to pull in all the
commonly-used stuff.  Then for the 0.1% of the tree where it matters
we could list things explicitly so that we don't have a big pile of
packages that portage can't parallelize.

It seems like the problems with the current approach are:
1.  @system can vary by profile, which allows bugs to creep in since
maintainers can't stay on top of what is and isn't always in @system).
2.  PMs can't build @system packages in parallel, since it doesn't
know what the real deps are.
3.  The way we use @system makes it hard to tell if it is safe to
remove some package which is otherwise heavily-used.  You never really
know if you can safely do without bash, and so on.  (Note, this bit
wouldn't be helped at all by simply turning system into a big
virtual.)

I'm not entirely sure what problem you're trying to solve, so the
above may or may not be helpful.  I'm fine with incremental
improvements if they actually improve something.  Otherwise, the
status quo does have the benefit of simplicity - you don't have to
list @system packages as dependencies ever, but you can do so if you
wish or it brings some benefit (deps on specific versions/slots,
slot-operator deps, etc).

--
Rich

[gentoo-dev] Re: Implicit system dependency

[Michael Palimaka]

On 14/11/14 01:17, Rich Freeman wrote:
> On Thu, Nov 13, 2014 at 5:30 AM, Michael Palimaka <kensington@gentoo.org> wrote:
>>
>> In general, a package must explicitly depend upon what it directly uses.
>> However, to avoid ebuild complexity and developer burden there are some
>> exceptions. Packages that appear in the base system set may be omitted
>> from an ebuild's dependency list in the following circumstances:
>>
> 
> So, I'm not a big fan of implicit dependencies in general.  What does
> the new policy buy us that the existing one does not?  Why not try to
> find a way to ditch implicit dependencies entirely?  If the issue is
> that nobody wants to have a laundry list of dependencies in every
> package, why not use something like a virtual to pull in all the
> commonly-used stuff.  Then for the 0.1% of the tree where it matters
> we could list things explicitly so that we don't have a big pile of
> packages that portage can't parallelize.

The problem is the current policy is not particularly clear - the issue
of when to explicitly specify a system dependency keeps coming up. The
first two sentences say all system dependencies are implicit and should
not be specified, while contradicted by the third in some (but not all
cases).

Ditching implicit dependencies is an interesting idea but not practical.
Nobody wants to the laundry list, and there's little benefit in
maintaining a virtual/system clone of @system.

> It seems like the problems with the current approach are:
> 1.  @system can vary by profile, which allows bugs to creep in since
> maintainers can't stay on top of what is and isn't always in @system).
> 2.  PMs can't build @system packages in parallel, since it doesn't
> know what the real deps are.
> 3.  The way we use @system makes it hard to tell if it is safe to
> remove some package which is otherwise heavily-used.  You never really
> know if you can safely do without bash, and so on.  (Note, this bit
> wouldn't be helped at all by simply turning system into a big
> virtual.)
> 
> I'm not entirely sure what problem you're trying to solve, so the
> above may or may not be helpful.  I'm fine with incremental
> improvements if they actually improve something.  Otherwise, the
> status quo does have the benefit of simplicity - you don't have to
> list @system packages as dependencies ever, but you can do so if you
> wish or it brings some benefit (deps on specific versions/slots,
> slot-operator deps, etc).

This is just about clarifying things. The wording of the current policy
is vague ("don't depend on system dependencies, but consider sometimes
doing it") and I think something more explicit would be better. An
update would help better reflect reality too - most people depend on
app-arch/bzip2 for libbzip2 even though it's in @system.

Re: [gentoo-dev] Re: Implicit system dependency

[Rich Freeman]

On Thu, Nov 13, 2014 at 10:07 AM, Michael Palimaka
<kensington@gentoo.org> wrote:
>
> Ditching implicit dependencies is an interesting idea but not practical.
> Nobody wants to the laundry list, and there's little benefit in
> maintaining a virtual/system clone of @system.
>

Well, the idea would be to maintain the virtual INSTEAD of @system, or
have @system just pull in the virtual and make some arch-specific
additions.

As far as benefits go, they include:
1.  No need to have multiple ways of grouping packages.
2.  You can more than one virtual, so that you could just pull in the
super-lazy equivalent to @system, or maybe you just pull in POSIX+bash
and C++ or something like that.
3.  You can split up that virtual so that convenience packages like
ssh aren't in the same virtual as widespread dependencies like
bash/zlib/glibc/gcc/etc.  There is no reason that you can't build
openssh in parallel, but right now you can't because we lump it in
with glibc.
4.  You can choose when to use the virtual at all, versus explicitly
naming all dependencies.

For 99% of packages it would be the same.  We could even have that
dependency added automatically if something isn't done in the ebuild
to disable it, which would make ebuilds work the same as they do now.
However, for the packages that are actually in @system we could list
explicit dependencies and then portage would actually be able to
handle some things automatically.  Also, by using virtuals that are
the same across all archs, we have a bit more consistency.

Policy-wise, though, the status quo isn't that bad.  You never have to
list dependencies that are in @system, full stop.  You can list a
dependency that is in @system anytime you want to, full stop.

That is, it is never right or wrong to list an unversioned dependency
that is in @system.  Sometimes doing one or the other is advantageous
(such as when you have a versioned dependency, or a virtual is in
@system but you need a specific implementation, or you want to use a
slot-op dep).  I'm fine with examples, but they shouldn't be firm
rules, just helpful guidelines.

--
Rich

[gentoo-dev] Re: Implicit system dependency

[Michael Palimaka]

On 14/11/14 11:06, Rich Freeman wrote:
> On Thu, Nov 13, 2014 at 10:07 AM, Michael Palimaka
> <kensington@gentoo.org> wrote:
>>
>> Ditching implicit dependencies is an interesting idea but not practical.
>> Nobody wants to the laundry list, and there's little benefit in
>> maintaining a virtual/system clone of @system.
>>
> 
> Well, the idea would be to maintain the virtual INSTEAD of @system, or
> have @system just pull in the virtual and make some arch-specific
> additions.

Will that work? Some profiles remove packages from the base @system and
replace it with their own implementations (eg. BSD).

> 
> As far as benefits go, they include:
> 1.  No need to have multiple ways of grouping packages.
> 2.  You can more than one virtual, so that you could just pull in the
> super-lazy equivalent to @system, or maybe you just pull in POSIX+bash
> and C++ or something like that.
> 3.  You can split up that virtual so that convenience packages like
> ssh aren't in the same virtual as widespread dependencies like
> bash/zlib/glibc/gcc/etc.  There is no reason that you can't build
> openssh in parallel, but right now you can't because we lump it in
> with glibc.
> 4.  You can choose when to use the virtual at all, versus explicitly
> naming all dependencies.
> 
> For 99% of packages it would be the same.  We could even have that
> dependency added automatically if something isn't done in the ebuild
> to disable it, which would make ebuilds work the same as they do now.
> However, for the packages that are actually in @system we could list
> explicit dependencies and then portage would actually be able to
> handle some things automatically.  Also, by using virtuals that are
> the same across all archs, we have a bit more consistency.

Definitely interesting ideas, but I think it's beyond the scope of
what's trying to be addressed here. Solving bug #393445 would also go a
long way to sorting out core-system vs. want-to-have packages.

> Policy-wise, though, the status quo isn't that bad.  You never have to
> list dependencies that are in @system, full stop.  You can list a
> dependency that is in @system anytime you want to, full stop.
> 
> That is, it is never right or wrong to list an unversioned dependency
> that is in @system.  Sometimes doing one or the other is advantageous
> (such as when you have a versioned dependency, or a virtual is in
> @system but you need a specific implementation, or you want to use a
> slot-op dep).  I'm fine with examples, but they shouldn't be firm
> rules, just helpful guidelines.

Maybe some dependencies (within reason) should always be listed, for
example when there's linkage eg. to libbzip2 or liblzma. That would make
it easy to find consumers eg. if an alternate implementation appeared,
and already appears to be a common practice.

Perhaps instead of creating a whitelist, we could just update the text a
bit:

All packages have an implicit compile-time and runtime dependency upon
the entire system target. For toolchain packages part of the system
target (such as gcc, libc, binutils and so on) it is not necessary, nor
advisable, to specify dependencies, except where specific versions or
packages (for example, glibc over uclibc) are required.

For other non-toolchain packages part of the system target (such as
bzip2 or wget) it is optional to specify a dependency. Consideration
should be given to packages that don't appear in system targets in other
profiles or might be removed in the future. Where package links to
package in the system target (such as libbz2) it is recommended to
specify a dependency.

Re: [gentoo-dev] Re: Implicit system dependency

[Rich Freeman]

On Thu, Nov 13, 2014 at 9:38 PM, Michael Palimaka <kensington@gentoo.org> wrote:
> On 14/11/14 11:06, Rich Freeman wrote:
>>
>> Well, the idea would be to maintain the virtual INSTEAD of @system, or
>> have @system just pull in the virtual and make some arch-specific
>> additions.
>
> Will that work? Some profiles remove packages from the base @system and
> replace it with their own implementations (eg. BSD).

Maybe.  The thing is that a package either depends on something or it
doesn't.  If it really does depend on something, then the fact that it
isn't available on BSD/etc isn't going to magically make the package
work.  We just loosely define system dependencies in a way that makes
them work 98% of the time, basically accepting that things are going
to break and we get away with it because few of our users actually run
on BSD/etc.

If it is just a matter of preference then a profile could install an
alternative package that is in a virtual.  However, this won't work if
everybody still uses some convenience virtual that pulls in bash/etc
and the BSD folks don't want to install bash unnecessarily.

The only perfect solution is explicit dependencies across the board.
If we want to be lazy and not have to list 50 deps for hello world,
then the package manager isn't going to know whether hello world
actually works on every arch/profile/etc.

Maybe the better solution is some kind of automation around (R)DEPEND.
In any case, I agree that it is a bit beyond the original scope of
this.

> Maybe some dependencies (within reason) should always be listed, for
> example when there's linkage eg. to libbzip2 or liblzma. That would make
> it easy to find consumers eg. if an alternate implementation appeared,
> and already appears to be a common practice.

The problem is that if it isn't 100% then it isn't all that great a
solution.  It also isn't really necessary unless you want to remove a
package from the system set.  If an alternative implementation
appears, then you create a virtual, place that virtual in the system
set, and all the packages in the tree remain happy to use whatever
implementation the user has installed without the risk of it getting
depcleaned.

In the past when we've considered removing a package from @system
there is usually a thread on -dev, and if there is a general sense
that we want to move forward then the announcement goes out for
everybody to check their packages and add an explicit dependency if
needed (often with tinderbox runs and the like).  Then after a delay
the package is removed from @system and maintainers get to deal with
anything they missed.

I don't have a problem with devs listing dependencies anytime they're
real and they feel it makes sense to do so.  I wouldn't make a push to
have them go out of their way to do it for any particular package
unless we are actively looking to remove it from @system, or there is
some other driver.  Slot operator deps would be the one case where I
would try to push on maintainers to list deps.

And I do apologize for piling on a bit - trying to get rid of @system
has been one of my soap box issues for a while.  It really seems like
an ugly, if practical, solution.

--
Rich

Re: [gentoo-dev] Re: Implicit system dependency

[Zac Medico]

On 11/13/2014 08:01 PM, Rich Freeman wrote:
> On Thu, Nov 13, 2014 at 9:38 PM, Michael Palimaka <kensington@gentoo.org> wrote:
>> On 14/11/14 11:06, Rich Freeman wrote:
>>>
>>> Well, the idea would be to maintain the virtual INSTEAD of @system, or
>>> have @system just pull in the virtual and make some arch-specific
>>> additions.
>>
>> Will that work? Some profiles remove packages from the base @system and
>> replace it with their own implementations (eg. BSD).
> 
> Maybe.  The thing is that a package either depends on something or it
> doesn't.  If it really does depend on something, then the fact that it
> isn't available on BSD/etc isn't going to magically make the package
> work.  We just loosely define system dependencies in a way that makes
> them work 98% of the time, basically accepting that things are going
> to break and we get away with it because few of our users actually run
> on BSD/etc.
> 
> If it is just a matter of preference then a profile could install an
> alternative package that is in a virtual.  However, this won't work if
> everybody still uses some convenience virtual that pulls in bash/etc
> and the BSD folks don't want to install bash unnecessarily.

Maybe I'm missing something, but if you are using virtuals, then you can
make the deps conditional on profile forced/masked flags like
userland_BSD and userland_GNU if necessary. These behave like normal USE
flags, aside from the fact the they are forced or masked by profiles.
-- 
Thanks,
Zac

Re: [gentoo-dev] Re: Implicit system dependency

[Anthony G. Basile]

On 11/13/14 23:15, Zac Medico wrote:
> On 11/13/2014 08:01 PM, Rich Freeman wrote:
>> On Thu, Nov 13, 2014 at 9:38 PM, Michael Palimaka <kensington@gentoo.org> wrote:
>>> On 14/11/14 11:06, Rich Freeman wrote:
>>>> Well, the idea would be to maintain the virtual INSTEAD of @system, or
>>>> have @system just pull in the virtual and make some arch-specific
>>>> additions.
>>> Will that work? Some profiles remove packages from the base @system and
>>> replace it with their own implementations (eg. BSD).
>> Maybe.  The thing is that a package either depends on something or it
>> doesn't.  If it really does depend on something, then the fact that it
>> isn't available on BSD/etc isn't going to magically make the package
>> work.  We just loosely define system dependencies in a way that makes
>> them work 98% of the time, basically accepting that things are going
>> to break and we get away with it because few of our users actually run
>> on BSD/etc.
>>
>> If it is just a matter of preference then a profile could install an
>> alternative package that is in a virtual.  However, this won't work if
>> everybody still uses some convenience virtual that pulls in bash/etc
>> and the BSD folks don't want to install bash unnecessarily.
> Maybe I'm missing something, but if you are using virtuals, then you can
> make the deps conditional on profile forced/masked flags like
> userland_BSD and userland_GNU if necessary. These behave like normal USE
> flags, aside from the fact the they are forced or masked by profiles.

Sorry Zac, I posted my reply before I read this.  This is essentially 
the point I was making.  However, I think this will be cumbersome.  With 
the current way we do things, its easy to delete packages from @system 
by just doing '-*sys-apps/man-pages' (for example) in a profile's 
packages file.  It is not so easy to delete from a DEPEND string, so I 
foresee some tricky if logic here.

-- 
Anthony G. Basile, Ph.D.
Gentoo Linux Developer [Hardened]
E-Mail    : blueness@gentoo.org
GnuPG FP  : 1FED FAD9 D82C 52A5 3BAB  DC79 9384 FA6E F52D 4BBA
GnuPG ID  : F52D4BBA

Re: [gentoo-dev] Re: Implicit system dependency

[Rich Freeman]

On Fri, Nov 14, 2014 at 7:20 AM, Anthony G. Basile <blueness@gentoo.org> wrote:
>
> Sorry Zac, I posted my reply before I read this.  This is essentially the
> point I was making.  However, I think this will be cumbersome.  With the
> current way we do things, its easy to delete packages from @system by just
> doing '-*sys-apps/man-pages' (for example) in a profile's packages file.  It
> is not so easy to delete from a DEPEND string, so I foresee some tricky if
> logic here.
>

Appreciating that we're on a slight tangent, why are these packages
typically removed?  Were they ever truly dependencies?  I can't
imagine too many packages breaking if man-pages isn't present.

If we went the virtual route I would suggest that we end up having a
few meta-virtuals.  There would be the lazy dependency virtual that
pulls in stuff like gcc/libc/posix/etc.  There would be a virtual that
pulls in useful-to-user stuff like ssh/man-pages/etc.  Then there
would be a virtual that pulls in both of the other virtuals.  Ebuilds
would only pull in the dependency virtual.

I agree that this would involve a fair bit of work, though I imagine
it could be done without any changes to portage insofar as testing
things out goes (just create a virtual, stick it in @system in a
profile, and test away).

--
Rich

Re: [gentoo-dev] Re: Implicit system dependency

[Andrew Savchenko]

[-- Attachment #1: Type: text/plain, Size: 3148 bytes --]

Hi,

On Fri, 14 Nov 2014 07:20:50 -0500 Anthony G. Basile wrote:
> On 11/13/14 23:15, Zac Medico wrote:
> > On 11/13/2014 08:01 PM, Rich Freeman wrote:
> >> On Thu, Nov 13, 2014 at 9:38 PM, Michael Palimaka <kensington@gentoo.org> wrote:
> >>> On 14/11/14 11:06, Rich Freeman wrote:
> >>>> Well, the idea would be to maintain the virtual INSTEAD of @system, or
> >>>> have @system just pull in the virtual and make some arch-specific
> >>>> additions.
> >>> Will that work? Some profiles remove packages from the base @system and
> >>> replace it with their own implementations (eg. BSD).
> >> Maybe.  The thing is that a package either depends on something or it
> >> doesn't.  If it really does depend on something, then the fact that it
> >> isn't available on BSD/etc isn't going to magically make the package
> >> work.  We just loosely define system dependencies in a way that makes
> >> them work 98% of the time, basically accepting that things are going
> >> to break and we get away with it because few of our users actually run
> >> on BSD/etc.
> >>
> >> If it is just a matter of preference then a profile could install an
> >> alternative package that is in a virtual.  However, this won't work if
> >> everybody still uses some convenience virtual that pulls in bash/etc
> >> and the BSD folks don't want to install bash unnecessarily.
> > Maybe I'm missing something, but if you are using virtuals, then you can
> > make the deps conditional on profile forced/masked flags like
> > userland_BSD and userland_GNU if necessary. These behave like normal USE
> > flags, aside from the fact the they are forced or masked by profiles.
> 
> Sorry Zac, I posted my reply before I read this.  This is essentially 
> the point I was making.  However, I think this will be cumbersome.  With 
> the current way we do things, its easy to delete packages from @system 
> by just doing '-*sys-apps/man-pages' (for example) in a profile's 
> packages file.  It is not so easy to delete from a DEPEND string, so I 
> foresee some tricky if logic here.

There is another drawback of using virtuals instead of @system set.
For old systems (in practical terms they are systems not updated
@world for more than several month) it is wise to update kernel,
@system and only afterwards whole @world.

Virtuals will not catch updates in underlying packages if --deep is
not used and it can't be used, because some packages from @system
may indirectly depend on packages from @world (e.g. cairo, qt or
xorg) which will trigger half of the @world update with -D @system
which makes it impossible and impractical to use -D @system before
full @world update on old setups.

We already have this problem with virtua/libc: it is not updated at
all, so when I run emerge -uav @system for the purposes described
above I have to manually add sys-devel/glibc to the list.

If @system will be removed at all, it will be much harder to
perform deep and large @world updates. Practically users willing to
update old systems will have to write and manage @system set on
their own.

Best regards,
Andrew Savchenko

[-- Attachment #2: Type: application/pgp-signature, Size: 819 bytes --]

Re: [gentoo-dev] Re: Implicit system dependency

[Zac Medico]

On 11/14/2014 06:14 AM, Andrew Savchenko wrote:
> Hi,
> 
> On Fri, 14 Nov 2014 07:20:50 -0500 Anthony G. Basile wrote:
>> On 11/13/14 23:15, Zac Medico wrote:
>>> On 11/13/2014 08:01 PM, Rich Freeman wrote:
>>>> On Thu, Nov 13, 2014 at 9:38 PM, Michael Palimaka <kensington@gentoo.org> wrote:
>>>>> On 14/11/14 11:06, Rich Freeman wrote:
>>>>>> Well, the idea would be to maintain the virtual INSTEAD of @system, or
>>>>>> have @system just pull in the virtual and make some arch-specific
>>>>>> additions.
>>>>> Will that work? Some profiles remove packages from the base @system and
>>>>> replace it with their own implementations (eg. BSD).
>>>> Maybe.  The thing is that a package either depends on something or it
>>>> doesn't.  If it really does depend on something, then the fact that it
>>>> isn't available on BSD/etc isn't going to magically make the package
>>>> work.  We just loosely define system dependencies in a way that makes
>>>> them work 98% of the time, basically accepting that things are going
>>>> to break and we get away with it because few of our users actually run
>>>> on BSD/etc.
>>>>
>>>> If it is just a matter of preference then a profile could install an
>>>> alternative package that is in a virtual.  However, this won't work if
>>>> everybody still uses some convenience virtual that pulls in bash/etc
>>>> and the BSD folks don't want to install bash unnecessarily.
>>> Maybe I'm missing something, but if you are using virtuals, then you can
>>> make the deps conditional on profile forced/masked flags like
>>> userland_BSD and userland_GNU if necessary. These behave like normal USE
>>> flags, aside from the fact the they are forced or masked by profiles.
>>
>> Sorry Zac, I posted my reply before I read this.  This is essentially 
>> the point I was making.  However, I think this will be cumbersome.  With 
>> the current way we do things, its easy to delete packages from @system 
>> by just doing '-*sys-apps/man-pages' (for example) in a profile's 
>> packages file.  It is not so easy to delete from a DEPEND string, so I 
>> foresee some tricky if logic here.
> 
> There is another drawback of using virtuals instead of @system set.
> For old systems (in practical terms they are systems not updated
> @world for more than several month) it is wise to update kernel,
> @system and only afterwards whole @world.
> 
> Virtuals will not catch updates in underlying packages if --deep is
> not used and it can't be used, because some packages from @system
> may indirectly depend on packages from @world (e.g. cairo, qt or
> xorg) which will trigger half of the @world update with -D @system
> which makes it impossible and impractical to use -D @system before
> full @world update on old setups.
> 
> We already have this problem with virtua/libc: it is not updated at
> all, so when I run emerge -uav @system for the purposes described
> above I have to manually add sys-devel/glibc to the list.

There was a time long ago when portage actually behaved the way you want
here. I implemented the behavior myself, but then I changed it to the
way it is now because others complained that "virtuals should behave
just like normal packages." We could certainly add an emerge option
which would trigger the behavior that you want.
-- 
Thanks,
Zac

Re: [gentoo-dev] Re: Implicit system dependency

[Andrew Savchenko]

[-- Attachment #1: Type: text/plain, Size: 1991 bytes --]

On Fri, 14 Nov 2014 10:03:27 -0800 Zac Medico wrote:
[...]
> >> Sorry Zac, I posted my reply before I read this.  This is essentially 
> >> the point I was making.  However, I think this will be cumbersome.  With 
> >> the current way we do things, its easy to delete packages from @system 
> >> by just doing '-*sys-apps/man-pages' (for example) in a profile's 
> >> packages file.  It is not so easy to delete from a DEPEND string, so I 
> >> foresee some tricky if logic here.
> > 
> > There is another drawback of using virtuals instead of @system set.
> > For old systems (in practical terms they are systems not updated
> > @world for more than several month) it is wise to update kernel,
> > @system and only afterwards whole @world.
> > 
> > Virtuals will not catch updates in underlying packages if --deep is
> > not used and it can't be used, because some packages from @system
> > may indirectly depend on packages from @world (e.g. cairo, qt or
> > xorg) which will trigger half of the @world update with -D @system
> > which makes it impossible and impractical to use -D @system before
> > full @world update on old setups.
> > 
> > We already have this problem with virtua/libc: it is not updated at
> > all, so when I run emerge -uav @system for the purposes described
> > above I have to manually add sys-devel/glibc to the list.
> 
> There was a time long ago when portage actually behaved the way you want
> here. I implemented the behavior myself, but then I changed it to the
> way it is now because others complained that "virtuals should behave
> just like normal packages." We could certainly add an emerge option
> which would trigger the behavior that you want.

If this switch will be available, it would be great! And in such
case I don't mind for system->virtual switch for my profiles.
Moreover it would be great to have this switch even now to avoid
"glibc not updated with @system" issues.

Best regards,
Andrew Savchenko

[-- Attachment #2: Type: application/pgp-signature, Size: 819 bytes --]

[gentoo-dev] Re: Implicit system dependency

[Michael Palimaka]

On 14/11/14 15:01, Rich Freeman wrote:
> On Thu, Nov 13, 2014 at 9:38 PM, Michael Palimaka <kensington@gentoo.org> wrote:
>> On 14/11/14 11:06, Rich Freeman wrote:
>>>
>>> Well, the idea would be to maintain the virtual INSTEAD of @system, or
>>> have @system just pull in the virtual and make some arch-specific
>>> additions.
>>
>> Will that work? Some profiles remove packages from the base @system and
>> replace it with their own implementations (eg. BSD).
> 
> Maybe.  The thing is that a package either depends on something or it
> doesn't.  If it really does depend on something, then the fact that it
> isn't available on BSD/etc isn't going to magically make the package
> work.  We just loosely define system dependencies in a way that makes
> them work 98% of the time, basically accepting that things are going
> to break and we get away with it because few of our users actually run
> on BSD/etc.
> 
> If it is just a matter of preference then a profile could install an
> alternative package that is in a virtual.  However, this won't work if
> everybody still uses some convenience virtual that pulls in bash/etc
> and the BSD folks don't want to install bash unnecessarily.
> 
> The only perfect solution is explicit dependencies across the board.
> If we want to be lazy and not have to list 50 deps for hello world,
> then the package manager isn't going to know whether hello world
> actually works on every arch/profile/etc.
> 
> Maybe the better solution is some kind of automation around (R)DEPEND.
> In any case, I agree that it is a bit beyond the original scope of
> this.
> 
>> Maybe some dependencies (within reason) should always be listed, for
>> example when there's linkage eg. to libbzip2 or liblzma. That would make
>> it easy to find consumers eg. if an alternate implementation appeared,
>> and already appears to be a common practice.
> 
> The problem is that if it isn't 100% then it isn't all that great a
> solution.  It also isn't really necessary unless you want to remove a
> package from the system set.  If an alternative implementation
> appears, then you create a virtual, place that virtual in the system
> set, and all the packages in the tree remain happy to use whatever
> implementation the user has installed without the risk of it getting
> depcleaned.
> 
> In the past when we've considered removing a package from @system
> there is usually a thread on -dev, and if there is a general sense
> that we want to move forward then the announcement goes out for
> everybody to check their packages and add an explicit dependency if
> needed (often with tinderbox runs and the like).  Then after a delay
> the package is removed from @system and maintainers get to deal with
> anything they missed.
> 
> I don't have a problem with devs listing dependencies anytime they're
> real and they feel it makes sense to do so.  I wouldn't make a push to
> have them go out of their way to do it for any particular package
> unless we are actively looking to remove it from @system, or there is
> some other driver.  Slot operator deps would be the one case where I
> would try to push on maintainers to list deps.
> 
> And I do apologize for piling on a bit - trying to get rid of @system
> has been one of my soap box issues for a while.  It really seems like
> an ugly, if practical, solution.

I think the proposed text in my previous reply is an improvement on the
current text. It certainly doesn't attempt to solve the problem of
implicit dependencies 100%, but I don't think that's a good reason to
avoid improvement.

The new text doesn't change the status quo at all. It just attempts to
clarify that:

* It's up to the developer if they wish to include a system dependency
or not (ie. it's not frowned upon, which was the issue raised in the
original bug)

* It's recommended (but not required) to depend upon a system dependency
if it's linked against (this is already common practice and there seems
to be support for requiring it, but I don't want to get too
controversial :-)

Re: [gentoo-dev] Re: Implicit system dependency

[Rich Freeman]

On Fri, Nov 14, 2014 at 9:49 PM, Michael Palimaka <kensington@gentoo.org> wrote:
> On 14/11/14 15:01, Rich Freeman wrote:
>>
>> And I do apologize for piling on a bit - trying to get rid of @system
>> has been one of my soap box issues for a while.  It really seems like
>> an ugly, if practical, solution.
>
> I think the proposed text in my previous reply is an improvement on the
> current text. It certainly doesn't attempt to solve the problem of
> implicit dependencies 100%, but I don't think that's a good reason to
> avoid improvement.

Hence my last comment.  :)

I don't have a problem with the change.

--
Rich

Re: [gentoo-dev] Re: Implicit system dependency

[Anthony G. Basile]

On 11/13/14 21:38, Michael Palimaka wrote:
> On 14/11/14 11:06, Rich Freeman wrote:
>> On Thu, Nov 13, 2014 at 10:07 AM, Michael Palimaka
>> <kensington@gentoo.org> wrote:
>>> Ditching implicit dependencies is an interesting idea but not practical.
>>> Nobody wants to the laundry list, and there's little benefit in
>>> maintaining a virtual/system clone of @system.
>>>
>> Well, the idea would be to maintain the virtual INSTEAD of @system, or
>> have @system just pull in the virtual and make some arch-specific
>> additions.
> Will that work? Some profiles remove packages from the base @system and
> replace it with their own implementations (eg. BSD).

Naively implemented, this would break my uclibc and musl builds which do 
exactly that.  However, you could build in logic, like `if use 
elibc_uclibc; then`, or DEPEND="elibc_uclibc? ( ... )" etc.

I get the benefits of this approach, but I'm happy enough with the 
status quo that I'm not in favor of the extra work.

-- 
Anthony G. Basile, Ph.D.
Gentoo Linux Developer [Hardened]
E-Mail    : blueness@gentoo.org
GnuPG FP  : 1FED FAD9 D82C 52A5 3BAB  DC79 9384 FA6E F52D 4BBA
GnuPG ID  : F52D4BBA

[gentoo-dev] Re: Implicit system dependency

[Ulrich Mueller]

[-- Attachment #1: Type: text/plain, Size: 936 bytes --]

>>>>> On Thu, 13 Nov 2014, Michael Palimaka wrote:

> Suggested policy to get the ball rolling:

> In general, a package must explicitly depend upon what it directly
> uses. However, to avoid ebuild complexity and developer burden there
> are some exceptions. Packages that appear in the base system set may
> be omitted from an ebuild's dependency list in the following
> circumstances:

> * C compiler and runtime

> * C++ compiler and runtime

> * A POSIX-compliant shell

> * bash, baselayout, binutils, coreutils, findutils, grep, make

awk? diffutils? texinfo?

> * Any archive tools (eg. tar, bzip2, xz-utils) where used for
> unpacking SRC_URI only

> * Any command guaranteed by PMS at build-time only (eg. sed, patch)

> Note that in some cases it might be necessary to explicitly specify
> a dependency that's listed above, such as when a specific
> implementation is required (eg. a binary package requiring glibc).

Ulrich

[-- Attachment #2: Type: application/pgp-signature, Size: 490 bytes --]

[gentoo-dev] Re: Implicit system dependency

[Michael Palimaka]

On 14/11/14 01:36, Ulrich Mueller wrote:
>>>>>> On Thu, 13 Nov 2014, Michael Palimaka wrote:
> 
>> Suggested policy to get the ball rolling:
> 
>> In general, a package must explicitly depend upon what it directly
>> uses. However, to avoid ebuild complexity and developer burden there
>> are some exceptions. Packages that appear in the base system set may
>> be omitted from an ebuild's dependency list in the following
>> circumstances:
> 
>> * C compiler and runtime
> 
>> * C++ compiler and runtime
> 
>> * A POSIX-compliant shell
> 
>> * bash, baselayout, binutils, coreutils, findutils, grep, make
> 
> awk? diffutils? texinfo?

I have no particular opinion on what should be included (or even if we
should have a whitelist - I am mainly interested in capturing library
usage). If we do have a whitelist, it would be useful to figure out what
in @system is intended to be actually be part of the base system and
what is there for convenience (eg. virtual/ssh).