[gentoo-dev] [RFC] Reducing the size of the system package set

[gentoo-dev] [RFC] Reducing the size of the system package set

[Diego 'Flameeyes' Pettenò]

[-- Attachment #1: Type: text/plain, Size: 4646 bytes --]


Here comes the official proposal, copy and paste from my blog with an
extra post scriptum at the end.

I already ranted about the fact that the dependency tree of our ebuilds
is vastly incomplete, as many lack dependency on zlib; trying to get
this fixed was impossible, as Donnie and other insisted that as zlib was
in system, we shouldn’t depend on it at all. I disagree, and I would
like to know why we can’t depend on a system package, but whatever.

Anyway, as having a complete dependency tree is almost impossible
because of that, I have an alternative proposal: reducing the size of
the system package set.  Right now system contains stuff like ncurses,
readline, zlib, autoconf, automake and m4, perl, gnuconfig, and so
on. Those are packages that certainly would be part of any base Gentoo
system, but are those actual part of the system set of packages? I
sincerely doubt it.

The reason of the existence of the system package set is related first
and foremost to breaking circular dependencies: for instance if any
package that used the C compiler would depend on gcc, then the packages
that gcc depends upon would create a circular dependency between gcc and
itself. Also, specifying libc in almost any ebuild would be quite
pointless, as well as coreutils (or freebsd-bin/ubin) for cp, mv,
install, …

But why autoconf and automake? Well the easy answer is that those are
often used without making sure they are depended upon explicitly… or at
least this was the case till me and Martin added autotools.eclass to the
tree; nowadays all the ebuilds using autotools should have proper
autoconf/automake dependency already, and if they don’t they are broken
anyway. So why leaving them in system? And what about m4? m4 is not part
of a common Unix system, it’s just an autoconf dependency, why isn’t it
just an autoconf dependency?

For what concern the three main libraries, there aren’t that many
packages using zlib directly nowadays, this is especially easy to spot
on a system built with --as-needed, as without that you actually do see
zlib used in every other binary, for indirect dependencies. Nor there
aren’t tons and tons of packages using readline, or ncurses. Actually in
my own vserver’s chroot I only found four packages using readline, none
of them part of system: ruby with the readline extension (uhm I wonder
if I should ask for this to become an USE flag, I certainly don’t need
it and I’d rather get rid of it), psql from postgresql (which maybe it’s
still good to have with readline compiled in, but I could easily get rid
of), bc (which is just an e2fsprogs build-dep, and I could build without
readline just fine), and mysql.

A little bit different the status of ncurses, which is used by screen,
gettext (only a build-dep, not needed for runtime on Linux anyway),
procps, psmisc and util-linux (and I wonder why we don’t have a switch
to turn it off), texinfo (wonder why we can’t remove it entirely
actually) and yet again ruby. Still, I wonder why ncurses is in system
rather than being properly on the dependencies list of those packages.

As for perl, that’s probably a bit more justified, there are tons of
packages using perl directly or indirectly, especially in build
systems. But I would like those to depend on perl properly rather than
having perl in system, as there are cases where perl is not really
needed at runtime at least.

And the only users of gnuconfig are the packages making use of the old
and deprecated gnuconfig.eclass, or portage’s econf. Why can’t it be a
dependency of portage then?

There are probably other packages that should, in my opinion, be removed
From system, but these are certainly some of the most
common. Unfortunately there’s a recursive problem here: to remove the
packages from system without breaking stuff you’d need a proper deptree,
and to get a proper deptree you need to remove the packages from
system. This is what actually stops me from proposing this right away…

P.S.:

So there are more things that were brought to my attention, like ssh,
flex, bison, e2fsprogs, and so on. We should probably look into what to
keep, rather than what to remove.

Also, rocket proposed me to try building a stage with a slimmed down
system. I could try, but it would be a waste of time if we then decide
not to go this route, and that's _a lot_ of time that would go to waste,
so I'd rather first see what the other devs think, before going to do
the actual tests.

-- 
Diego "Flameeyes" Pettenò
http://farragut.flameeyes.is-a-geek.org/

[-- Attachment #2: Type: application/pgp-signature, Size: 196 bytes --]

Re: [gentoo-dev] [RFC] Reducing the size of the system package set

[Petteri Räty]

[-- Attachment #1: Type: text/plain, Size: 1837 bytes --]

Diego 'Flameeyes' Pettenò kirjoitti:
> Here comes the official proposal, copy and paste from my blog with an
> extra post scriptum at the end.
> 
> I already ranted about the fact that the dependency tree of our ebuilds
> is vastly incomplete, as many lack dependency on zlib; trying to get
> this fixed was impossible, as Donnie and other insisted that as zlib was
> in system, we shouldn’t depend on it at all. I disagree, and I would
> like to know why we can’t depend on a system package, but whatever.

At least one reason is that otherwise lots of ebuild submissions would 
have coreutils/gcc/libc/whatever in DEPEND/RDEPEND.

> 
> But why autoconf and automake? Well the easy answer is that those are
> often used without making sure they are depended upon explicitly… or at
> least this was the case till me and Martin added autotools.eclass to the
> tree; nowadays all the ebuilds using autotools should have proper
> autoconf/automake dependency already, and if they don’t they are broken
> anyway. So why leaving them in system? And what about m4? m4 is not part
> of a common Unix system, it’s just an autoconf dependency, why isn’t it
> just an autoconf dependency?
> 

IMHO these could be removed from the system set. But likely the packages 
left in there would pull it to the stages any way.

> 
> P.S.:
> 
> So there are more things that were brought to my attention, like ssh,
> flex, bison, e2fsprogs, and so on. We should probably look into what to
> keep, rather than what to remove.
 >

Well e2fsprogs provides fsck so it's a dep of baselayout. But it 
wouldn't hurt to cleanup the list even if it doesn't get them removed 
from stages. I don't know about other system packages but at least 
baselayout is not really depending on much atm.

Regards,
Petteri


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 260 bytes --]

Re: [gentoo-dev] [RFC] Reducing the size of the system package set

[Piotr Jaroszyński]

On Tuesday 08 of January 2008 02:02:56 Petteri Räty wrote:
> Diego 'Flameeyes' Pettenò kirjoitti:
> > Here comes the official proposal, copy and paste from my blog with an
> > extra post scriptum at the end.
> >
> > I already ranted about the fact that the dependency tree of our ebuilds
> > is vastly incomplete, as many lack dependency on zlib; trying to get
> > this fixed was impossible, as Donnie and other insisted that as zlib was
> > in system, we shouldn’t depend on it at all. I disagree, and I would
> > like to know why we can’t depend on a system package, but whatever.
>
> At least one reason is that otherwise lots of ebuild submissions would
> have coreutils/gcc/libc/whatever in DEPEND/RDEPEND.

We could probably introduce some fancy virtuals like c-toolchain to reduce the 
bloat, but the transition would be somehow tricky... would probably need a 
list of already converted packages.

-- 
Best Regards,
Piotr Jaroszyński
--
gentoo-dev@lists.gentoo.org mailing list

Re: [gentoo-dev] [RFC] Reducing the size of the system package set

[Donnie Berkholz]

On 00:42 Tue 08 Jan     , Diego 'Flameeyes' Pettenò wrote:
> Anyway, as having a complete dependency tree is almost impossible
> because of that, I have an alternative proposal: reducing the size of
> the system package set.  Right now system contains stuff like ncurses,
> readline, zlib, autoconf, automake and m4, perl, gnuconfig, and so
> on. Those are packages that certainly would be part of any base Gentoo
> system, but are those actual part of the system set of packages? I
> sincerely doubt it.

What is your goal? Is there something you're trying to accomplish that's 
impossible? It's clear that changing this would be a fair amount of 
work, and I don't understand the benefits.

Thanks,
Donnie
-- 
gentoo-dev@lists.gentoo.org mailing list

[gentoo-dev] Re: [RFC] Reducing the size of the system package set

[Diego 'Flameeyes' Pettenò]

[-- Attachment #1: Type: text/plain, Size: 1270 bytes --]

Donnie Berkholz <dberkholz@gentoo.org> writes:

> What is your goal? Is there something you're trying to accomplish that's 
> impossible? It's clear that changing this would be a fair amount of 
> work, and I don't understand the benefits.

With the current size of system packages set, having a complete deptree
is impossible. You're one of the main followers of the idea that if a
package is in system you don't have to depend on it, and I already
talked to you about the problems there are with emerge -e world when for
instance zlib is broken.

So my goals are:
 - have a deptree as complete as possible;
 - being able to have an emerge -e world that actually builds first the
 stuff that's going to be needed (zlib before packages using zlib);
 - avoid overriding the system package set in profiles like embedded
 where stuff like autoconf and perl might not be wanted on the resulting
 filesystem.

The first goal is a prerequisite if we want to move to other stuff like
a true multilib-handling package manager (we don't want to force down
the users' throats multiple copies of autoconf, considering it's a
script, do we?) or proper cross-building environments.

-- 
Diego "Flameeyes" Pettenò
http://farragut.flameeyes.is-a-geek.org/

[-- Attachment #2: Type: application/pgp-signature, Size: 196 bytes --]

Re: [gentoo-dev] Re: [RFC] Reducing the size of the system package set

[Santiago M. Mola]

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1: Type: text/plain; charset=UTF-8, Size: 739 bytes --]

On Jan 8, 2008 12:34 PM, Diego 'Flameeyes' Pettenò <flameeyes@gmail.com> wrote:
>
> So my goals are:
>  - have a deptree as complete as possible;

Your goals make sense but reducing the system package set seems like
workaround to get you indirectly where you want. Having a complete
deptree and having a small system package set should be independent
goals, maybe your proposal should focus on making possible to have a
deptree as complete as possible independently of what packages are on
system set.

Note that I'm not stating that we can't discuss about removing package
X or Y from system, but I think we should discuss that separately.

Regards,
Santiago

-- 
Santiago M. Mola
Jabber ID: cooldwind@gmail.com
éí¢‡^¾X¬¶È\x1ežÚ(¢¸&j)bž	b²

[gentoo-dev] Re: [RFC] Reducing the size of the system package set

[Diego 'Flameeyes' Pettenò]

"Santiago M. Mola" <coldwind@gentoo.org> writes:

> Having a complete
> deptree and having a small system package set should be independent
> goals, maybe your proposal should focus on making possible to have a
> deptree as complete as possible independently of what packages are on
> system set.

QA team already rejected such a proposal.

-- 
Diego "Flameeyes" Pettenò
http://farragut.flameeyes.is-a-geek.org/

-- 
gentoo-dev@lists.gentoo.org mailing list

Re: [gentoo-dev] [RFC] Reducing the size of the system package set

[Stefan Hellermann]

Donnie Berkholz schrieb:
> On 00:42 Tue 08 Jan     , Diego 'Flameeyes' Pettenò wrote:
>> Anyway, as having a complete dependency tree is almost impossible
>> because of that, I have an alternative proposal: reducing the size of
>> the system package set.  Right now system contains stuff like ncurses,
>> readline, zlib, autoconf, automake and m4, perl, gnuconfig, and so
>> on. Those are packages that certainly would be part of any base Gentoo
>> system, but are those actual part of the system set of packages? I
>> sincerely doubt it.
> 
> What is your goal? Is there something you're trying to accomplish that's 
> impossible? It's clear that changing this would be a fair amount of 
> work, and I don't understand the benefits.

There are many people that don't use the development part of the system-set, they use 
prebuild-packages or emerge with the ROOT= option. I have a set of slow pentium2 machines 
here which boot over nfs. They will never execute gcc nor emerge, I'm using emerge with 
ROOT= on the buildhost to update their trees. Also I'm using a server divided into many 
vservers (linux-vserver.org), every vserver has a stage3 and only one additional package 
to serve a purpose. If it is possible to reduce the size of the stage3 it would save 
really much space.
I've tried to not use the system-set and set up a virtual called virtual/minimal-system 
which depends on all the packages I need (no gcc or perl, only coreutils, glibc, 
baselayout and some packages that are really needed for booting up). This is what I think 
  should be the system-set. And there would be a development-set which most people would 
use, but don't have to. And maybe a set with packages like man, texinfo ...

My own minimal-system virtual was a fair amount of work, but not really usable due to the 
missing dependencies in the system-set. Also I had huge problems while I tried updating 
this set with "ROOT=/path/to/tree emerge -uDaN virtual/minimal-system" so I'm not using it 
anymore.

I'm using gentoo for a long time now, but if there's something that can be done to reduce 
its minimal footprint, then it would get even better :)

Cheers
Stefan

> 
> Thanks,
> Donnie

-- 
gentoo-dev@lists.gentoo.org mailing list

Re: [gentoo-dev] [RFC] Reducing the size of the system package set

[Petteri Räty]

[-- Attachment #1: Type: text/plain, Size: 639 bytes --]

Stefan Hellermann kirjoitti:
> I've tried to not use the system-set and set up a virtual called 
> virtual/minimal-system which depends on all the packages I need (no gcc 
> or perl, only coreutils, glibc, baselayout and some packages that are 
> really needed for booting up). This is what I think  should be the 
> system-set. And there would be a development-set which most people would 
> use, but don't have to. And maybe a set with packages like man, texinfo ...
> 

Probably would be useful to separate DEPEND and RDEPEND base systems. So 
things like gcc would only be needed for the DEPEND part.

Regards,
Petteri


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 260 bytes --]

Re: [gentoo-dev] Re: [RFC] Reducing the size of the system package set

[Brian Harring]

[-- Attachment #1: Type: text/plain, Size: 820 bytes --]

On Tue, Jan 08, 2008 at 01:37:28PM +0100, Diego 'Flameeyes' Petten?? wrote:
> "Santiago M. Mola" <coldwind@gentoo.org> writes:
> 
> > Having a complete
> > deptree and having a small system package set should be independent
> > goals, maybe your proposal should focus on making possible to have a
> > deptree as complete as possible independently of what packages are on
> > system set.
> 
> QA team already rejected such a proposal.

Don't suppose you have a url for their reasoning?  Among other things, 
system set can vary, thus resulting in horked dependencies.  
Additionally, a more accurate dep details is useful for 
parallelization hints- breaking the reliance on system set as much as 
possible, and treating it more like a users world set is something 
I've wanted for a long while.
~brian

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

[gentoo-dev] Re: [RFC] Reducing the size of the system package set

[Diego 'Flameeyes' Pettenò]

Brian Harring <ferringb@gmail.com> writes:

> Don't suppose you have a url for their reasoning? 

https://bugs.gentoo.org/show_bug.cgi?id=151758

-- 
Diego "Flameeyes" Pettenò
http://farragut.flameeyes.is-a-geek.org/

-- 
gentoo-dev@lists.gentoo.org mailing list

Re: [gentoo-dev] [RFC] Reducing the size of the system package set

[Chris Gianelloni]

[-- Attachment #1: Type: text/plain, Size: 1293 bytes --]

On Tue, 2008-01-08 at 00:30 -0800, Donnie Berkholz wrote:
> On 00:42 Tue 08 Jan     , Diego 'Flameeyes' Pettenò wrote:
> > Anyway, as having a complete dependency tree is almost impossible
> > because of that, I have an alternative proposal: reducing the size of
> > the system package set.  Right now system contains stuff like ncurses,
> > readline, zlib, autoconf, automake and m4, perl, gnuconfig, and so
> > on. Those are packages that certainly would be part of any base Gentoo
> > system, but are those actual part of the system set of packages? I
> > sincerely doubt it.
> 
> What is your goal? Is there something you're trying to accomplish that's 
> impossible? It's clear that changing this would be a fair amount of 
> work, and I don't understand the benefits.

Come work on a Gentoo release some time.  Implicit dependencies waste
more time than pretty much anything else.  Almost all circular
dependency issues we currently have are due to implicit dependencies.
Also, several things have been added either to system or (more commonly)
packages.build, to try to work around these deficiencies in the tree.

-- 
Chris Gianelloni
Release Engineering Strategic Lead
Alpha/AMD64/x86 Architecture Teams
Games Developer/Foundation Trustee
Gentoo Foundation

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-dev] [RFC] Reducing the size of the system package set

[Chris Gianelloni]

[-- Attachment #1: Type: text/plain, Size: 939 bytes --]

On Tue, 2008-01-08 at 21:49 +0200, Petteri Räty wrote:
> Stefan Hellermann kirjoitti:
> > I've tried to not use the system-set and set up a virtual called 
> > virtual/minimal-system which depends on all the packages I need (no gcc 
> > or perl, only coreutils, glibc, baselayout and some packages that are 
> > really needed for booting up). This is what I think  should be the 
> > system-set. And there would be a development-set which most people would 
> > use, but don't have to. And maybe a set with packages like man, texinfo ...
> > 
> 
> Probably would be useful to separate DEPEND and RDEPEND base systems. So 
> things like gcc would only be needed for the DEPEND part.

Be careful with this one.  Lots of things end up linking with libstdc
++.so.* from GCC.

-- 
Chris Gianelloni
Release Engineering Strategic Lead
Alpha/AMD64/x86 Architecture Teams
Games Developer/Foundation Trustee
Gentoo Foundation

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-dev] [RFC] Reducing the size of the system package set

[Petteri Räty]

[-- Attachment #1: Type: text/plain, Size: 949 bytes --]

Chris Gianelloni kirjoitti:
> On Tue, 2008-01-08 at 21:49 +0200, Petteri Räty wrote:
>> Stefan Hellermann kirjoitti:
>>> I've tried to not use the system-set and set up a virtual called 
>>> virtual/minimal-system which depends on all the packages I need (no gcc 
>>> or perl, only coreutils, glibc, baselayout and some packages that are 
>>> really needed for booting up). This is what I think  should be the 
>>> system-set. And there would be a development-set which most people would 
>>> use, but don't have to. And maybe a set with packages like man, texinfo ...
>>>
>> Probably would be useful to separate DEPEND and RDEPEND base systems. So 
>> things like gcc would only be needed for the DEPEND part.
> 
> Be careful with this one.  Lots of things end up linking with libstdc
> ++.so.* from GCC.
> 

virtual/libstdc++ similar virtuals could exist for other gcc parts that 
are needed at runtime.

Regards,
Petteri


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 260 bytes --]

Re: [gentoo-dev] [RFC] Reducing the size of the system package set

[Mike Frysinger]

[-- Attachment #1: Type: text/plain, Size: 5896 bytes --]

On Monday 07 January 2008, Diego 'Flameeyes' Pettenò wrote:
> I already ranted about the fact that the dependency tree of our ebuilds
> is vastly incomplete, as many lack dependency on zlib; trying to get
> this fixed was impossible, as Donnie and other insisted that as zlib was
> in system, we shouldn’t depend on it at all. I disagree, and I would
> like to know why we can’t depend on a system package, but whatever.

the system dep rule isnt hard as in "if it's system, you cannot depend on it".  
that's silly.  it applies to packages which, if they do not exist, the system 
would not be usable.  things like grep/sed/tail/ps/etc... do not belong in 
the depend strings.  you cannot have a usable system without such utilities 
on your system.  that means packages like grep/sed/coreutils/procps/etc... do 
not belong in depend strings.  there is also the issue that these packages 
tend to be swappable based on the system (embedded/bsd/whatever), so listing 
them only causes problems.

> Anyway, as having a complete dependency tree is almost impossible
> because of that, I have an alternative proposal: reducing the size of
> the system package set.  Right now system contains stuff like ncurses,
> readline, zlib, autoconf, automake and m4, perl, gnuconfig, and so
> on. Those are packages that certainly would be part of any base Gentoo
> system, but are those actual part of the system set of packages? I
> sincerely doubt it.

for ncurses/readline, they certainly are part of the system set.  that doesnt 
mean they should not show up in depend strings, it just means they are system 
packages that every Gentoo system should have installed.

i have no problem with shunting the rest.  the only thing you need to keep in 
mind is that if we do move all of these things to build-only depend (which 
they are logically), then people who like to depclean their system would 
constantly be removing/adding them.

> The reason of the existence of the system package set is related first
> and foremost to breaking circular dependencies: for instance if any
> package that used the C compiler would depend on gcc, then the packages
> that gcc depends upon would create a circular dependency between gcc and
> itself. Also, specifying libc in almost any ebuild would be quite
> pointless, as well as coreutils (or freebsd-bin/ubin) for cp, mv,
> install, …

not really.  the system package set is what went into releases and we wanted 
all of these things in our stage tarballs.  it is nigh impossible to emerge 
anything on a Gentoo system without any of these packages, so forcing them 
all by default didnt cause any problems.

> For what concern the three main libraries, there aren’t that many
> packages using zlib directly nowadays, this is especially easy to spot
> on a system built with --as-needed, as without that you actually do see
> zlib used in every other binary, for indirect dependencies. Nor there
> aren’t tons and tons of packages using readline, or ncurses. Actually in
> my own vserver’s chroot I only found four packages using readline, none
> of them part of system: ruby with the readline extension (uhm I wonder
> if I should ask for this to become an USE flag, I certainly don’t need
> it and I’d rather get rid of it), psql from postgresql (which maybe it’s
> still good to have with readline compiled in, but I could easily get rid
> of), bc (which is just an e2fsprogs build-dep, and I could build without
> readline just fine), and mysql.

bash uses readline as well ... but currently statically links it in ... i 
could (should?) change that ...

> A little bit different the status of ncurses, which is used by screen,
> gettext (only a build-dep, not needed for runtime on Linux anyway),
> procps, psmisc and util-linux (and I wonder why we don’t have a switch
> to turn it off), texinfo (wonder why we can’t remove it entirely
> actually) and yet again ruby. Still, I wonder why ncurses is in system
> rather than being properly on the dependencies list of those packages.

not sure how you missed the fact that *bash* needs it.  that seems pretty 
critical.

> As for perl, that’s probably a bit more justified, there are tons of
> packages using perl directly or indirectly, especially in build
> systems. But I would like those to depend on perl properly rather than
> having perl in system, as there are cases where perl is not really
> needed at runtime at least.

i'd say quite the opposite ... requiring perl in system blows.  it's there for 
two reasons: autotools and openssl.  but both are build time requirements 
only.

> And the only users of gnuconfig are the packages making use of the old
> and deprecated gnuconfig.eclass, or portage’s econf. Why can’t it be a
> dependency of portage then?

you've missed all of the autotool reliance on gnuconfig.  they symlink their 
config files to gnuconfig.  the fact that it is tied into econf is irrelevant 
i think ... any autotool based package has a dep on gnuconfig as bundled 
config files should always update to this.  how it gets updated doesnt 
matter.  if your package manager isnt auto-updating things, it sucks.  as for 
how to express that depend, it's a crap shoot.

> So there are more things that were brought to my attention, like ssh,
> flex, bison, e2fsprogs, and so on. We should probably look into what to
> keep, rather than what to remove.

flex/bison are in the exact same boat as autotools.  dont know why you 
separated them out.  openssh and e2fsprogs are part of the system set because 
every Gentoo system out there should have them installed.  if you dont like 
it, feel free to tweak your files locally, but to attempt to account for a 
few people at the detriment of 99.9% of the people out there makes no sense 
at all.
-mike

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 835 bytes --]

Re: [gentoo-dev] [RFC] Reducing the size of the system package set

[Chris Gianelloni]

[-- Attachment #1: Type: text/plain, Size: 3656 bytes --]

On Wed, 2008-01-09 at 15:51 -0500, Mike Frysinger wrote:
> > Anyway, as having a complete dependency tree is almost impossible
> > because of that, I have an alternative proposal: reducing the size of
> > the system package set.  Right now system contains stuff like ncurses,
> > readline, zlib, autoconf, automake and m4, perl, gnuconfig, and so
> > on. Those are packages that certainly would be part of any base Gentoo
> > system, but are those actual part of the system set of packages? I
> > sincerely doubt it.
> 
> for ncurses/readline, they certainly are part of the system set.  that doesnt 
> mean they should not show up in depend strings, it just means they are system 
> packages that every Gentoo system should have installed.

Well, one solution for some of this would be to move said things to a
"higher" level profile.  Rather than have them all in base/packages,
move some to default-linux/packages (or even further down, if
appropriate).  When the stages get built against these profiles, we
would end up with the complete "system" set, but other profiles
inheriting from the lower profiles like base won't have to negate
things.

> i have no problem with shunting the rest.  the only thing you need to keep in 
> mind is that if we do move all of these things to build-only depend (which 
> they are logically), then people who like to depclean their system would 
> constantly be removing/adding them.

Well, unless they were added to another profile.  I think the main
reason for Diego's request is for non-Linux non-default profiles that
inherit from base.

> not really.  the system package set is what went into releases and we wanted 
> all of these things in our stage tarballs.  it is nigh impossible to emerge 
> anything on a Gentoo system without any of these packages, so forcing them 
> all by default didnt cause any problems.

Exactly.  I just think that we can still accomplish what Diego is asking
by shifting where things get added to "system" in the profiles.  The
end-result would be the same (for default Linux installs) but everybody
else would have a cleaner common base from which to start.

> i'd say quite the opposite ... requiring perl in system blows.  it's there for 
> two reasons: autotools and openssl.  but both are build time requirements 
> only.

Indeed.  We ended up having to get perl into the stage1 because of
exactly these problems.  It sucks.  I'd love to be able to remove perl
(and anything else not necessarily required) out of the base system set.
If they're required in other profiles, they should be added in said
profiles.

> > So there are more things that were brought to my attention, like ssh,
> > flex, bison, e2fsprogs, and so on. We should probably look into what to
> > keep, rather than what to remove.
> 
> flex/bison are in the exact same boat as autotools.  dont know why you 
> separated them out.  openssh and e2fsprogs are part of the system set because 
> every Gentoo system out there should have them installed.  if you dont like 
> it, feel free to tweak your files locally, but to attempt to account for a 
> few people at the detriment of 99.9% of the people out there makes no sense 
> at all.

Well, openssh has always been questionable.  Sure, *I* think it should
be on any Gentoo system I'd want to touch, but it really isn't necessary
for a lot of people.  Moving this to, say, the "server" profiles only
would be acceptable to me, but then again, so is leaving it how it is
now.

-- 
Chris Gianelloni
Release Engineering Strategic Lead
Alpha/AMD64/x86 Architecture Teams
Games Developer

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-dev] [RFC] Reducing the size of the system package set

[Mike Frysinger]

[-- Attachment #1: Type: text/plain, Size: 5139 bytes --]

On Wednesday 09 January 2008, Chris Gianelloni wrote:
> On Wed, 2008-01-09 at 15:51 -0500, Mike Frysinger wrote:
> > > Anyway, as having a complete dependency tree is almost impossible
> > > because of that, I have an alternative proposal: reducing the size of
> > > the system package set.  Right now system contains stuff like ncurses,
> > > readline, zlib, autoconf, automake and m4, perl, gnuconfig, and so
> > > on. Those are packages that certainly would be part of any base Gentoo
> > > system, but are those actual part of the system set of packages? I
> > > sincerely doubt it.
> >
> > for ncurses/readline, they certainly are part of the system set.  that
> > doesnt mean they should not show up in depend strings, it just means they
> > are system packages that every Gentoo system should have installed.
>
> Well, one solution for some of this would be to move said things to a
> "higher" level profile.  Rather than have them all in base/packages,
> move some to default-linux/packages (or even further down, if
> appropriate).  When the stages get built against these profiles, we
> would end up with the complete "system" set, but other profiles
> inheriting from the lower profiles like base won't have to negate
> things.

i dont think Diego's goal is completely BSD driven ... so moving them from 
say "base" to "default-linux" doesnt quite help.  they're stuck in this gray 
middle ground.

> > i have no problem with shunting the rest.  the only thing you need to
> > keep in mind is that if we do move all of these things to build-only
> > depend (which they are logically), then people who like to depclean their
> > system would constantly be removing/adding them.
>
> Well, unless they were added to another profile.  I think the main
> reason for Diego's request is for non-Linux non-default profiles that
> inherit from base.

for these build-only deps, it isnt a Linux problem.  it's a "any system that 
compiles thing from source" problem.  so non-Linux systems would be just as 
affected.

> > not really.  the system package set is what went into releases and we
> > wanted all of these things in our stage tarballs.  it is nigh impossible
> > to emerge anything on a Gentoo system without any of these packages, so
> > forcing them all by default didnt cause any problems.
>
> Exactly.  I just think that we can still accomplish what Diego is asking
> by shifting where things get added to "system" in the profiles.  The
> end-result would be the same (for default Linux installs) but everybody
> else would have a cleaner common base from which to start.

sure, i'm not arguing for the status quo.  but i'm not also not arguing for 
stripping them all (which would make Diego happy i imagine).  i wonder if the 
profiles frags we talked about a while ago is the only real solution and 
shuffling packages around in the meantime is only a temporary (and fragile) 
workaround.

> > i'd say quite the opposite ... requiring perl in system blows.  it's
> > there for two reasons: autotools and openssl.  but both are build time
> > requirements only.
>
> Indeed.  We ended up having to get perl into the stage1 because of
> exactly these problems.  It sucks.  I'd love to be able to remove perl
> (and anything else not necessarily required) out of the base system set.
> If they're required in other profiles, they should be added in said
> profiles.

i often debate simply chucking the perl build requirements of openssl in favor 
of autoconf ... maybe i'll set up a git tree on Gentoo's git for this ... i 
think that'd solve the "perl required in stages" issue ?

> > > So there are more things that were brought to my attention, like ssh,
> > > flex, bison, e2fsprogs, and so on. We should probably look into what to
> > > keep, rather than what to remove.
> >
> > flex/bison are in the exact same boat as autotools.  dont know why you
> > separated them out.  openssh and e2fsprogs are part of the system set
> > because every Gentoo system out there should have them installed.  if you
> > dont like it, feel free to tweak your files locally, but to attempt to
> > account for a few people at the detriment of 99.9% of the people out
> > there makes no sense at all.
>
> Well, openssh has always been questionable.  Sure, *I* think it should
> be on any Gentoo system I'd want to touch, but it really isn't necessary
> for a lot of people.  Moving this to, say, the "server" profiles only
> would be acceptable to me, but then again, so is leaving it how it is
> now.

i'd argue pretty vehemently against removing openssh from any default official 
Gentoo install.  ssh is defacto standard for loginning into any other 
machines.  it should be on all Gentoo desktops/severs/etc...  
specialized/embedded/whatever are certainly free to cull openssh (and doing 
so is actually beyond trivial).  whether we express this requirement in base/ 
or frags or something is certainly open for discussion, but i believe 
removing it from a stage3 in any of our standard releases is a huge 
disservice to everyone.
-mike

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 835 bytes --]

Re: [gentoo-dev] [RFC] Reducing the size of the system package set

[Josh Saddler]

[-- Attachment #1: Type: text/plain, Size: 2272 bytes --]

Mike Frysinger wrote:
>> Well, openssh has always been questionable.  Sure, *I* think it should
>> be on any Gentoo system I'd want to touch, but it really isn't necessary
>> for a lot of people.  Moving this to, say, the "server" profiles only
>> would be acceptable to me, but then again, so is leaving it how it is
>> now.
> 
> i'd argue pretty vehemently against removing openssh from any default official 
> Gentoo install.  ssh is defacto standard for loginning into any other 
> machines.  it should be on all Gentoo desktops/severs/etc...  
> specialized/embedded/whatever are certainly free to cull openssh (and doing 
> so is actually beyond trivial).  whether we express this requirement in base/ 
> or frags or something is certainly open for discussion, but i believe 
> removing it from a stage3 in any of our standard releases is a huge 
> disservice to everyone.

We all know that ssh is good for sysadmins and netadmins and Gentoo
developers, etc. However, desktop users -- i.e., those not in those
categories, which is most everyone else, likely do not find it as
useful. I'd say that most of our userbase doesn't need to remotely
connect to other machines. Our development/admin experiences are *not*
typical of our installed userbase, judging by our huge forums. Their
machines aren't used for such purposes. openssh may be a de facto
standard for the kind of application it is, but that doesn't mean we
need to force it on end-users. Our philosophy has been more of "opt-in",
rather than "opt-out", and quite simply, ssh isn't a critical system
package; a base installed system won't be broken if it's  not there. If
you *need* it to do work (admins, I'm looking at you), then you can
*emerge* it. Just like vim, cvs, dev_tool_foo etc.

I *am* a desktop user. And . . . aside from Gentoo development, I have
no need for ssh. It could easily be removed from the system profile --
the only place it might be kept is in the liveCD environment, for remote
installations. Other than that, we don't need to ship it in our stages;
just on the media.


(Whoa, sudden feeling of deja vu. I'm 80% positive this was previously
brought up on the MLs within the last two to three years...and I stated
the same view then!)


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-dev] [RFC] Reducing the size of the system package set

[Chris Gianelloni]

[-- Attachment #1: Type: text/plain, Size: 2127 bytes --]

On Wed, 2008-01-09 at 16:42 -0500, Mike Frysinger wrote:
> > Indeed.  We ended up having to get perl into the stage1 because of
> > exactly these problems.  It sucks.  I'd love to be able to remove perl
> > (and anything else not necessarily required) out of the base system set.
> > If they're required in other profiles, they should be added in said
> > profiles.
> 
> i often debate simply chucking the perl build requirements of openssl in favor 
> of autoconf ... maybe i'll set up a git tree on Gentoo's git for this ... i 
> think that'd solve the "perl required in stages" issue ?

As far as I know, only openssl would really be affected.  If that were
fixed, we very likely could drop perl from system.  I'd have to check.
I *know* that we could drop it from stage1/stage2, though.

> > Well, openssh has always been questionable.  Sure, *I* think it should
> > be on any Gentoo system I'd want to touch, but it really isn't necessary
> > for a lot of people.  Moving this to, say, the "server" profiles only
> > would be acceptable to me, but then again, so is leaving it how it is
> > now.
> 
> i'd argue pretty vehemently against removing openssh from any default official 
> Gentoo install.  ssh is defacto standard for loginning into any other 
> machines.  it should be on all Gentoo desktops/severs/etc...  
> specialized/embedded/whatever are certainly free to cull openssh (and doing 
> so is actually beyond trivial).  whether we express this requirement in base/ 
> or frags or something is certainly open for discussion, but i believe 
> removing it from a stage3 in any of our standard releases is a huge 
> disservice to everyone.

Well, I wasn't really saying that it wouldn't end up in the stages.
More that by moving it into the sub-profiles, it allows people to choose
a profile where it isn't in "system" anymore.  Of course, most of this
is rather moot considering you can override parts of the profile and
someone could quite easily just remove this one package, if they so
desired.

-- 
Chris Gianelloni
Release Engineering Strategic Lead
Games Developer

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-dev] [RFC] Reducing the size of the system package set

[Petteri Räty]

[-- Attachment #1: Type: text/plain, Size: 685 bytes --]

Mike Frysinger kirjoitti:
> 
> i'd argue pretty vehemently against removing openssh from any default official 
> Gentoo install.  ssh is defacto standard for loginning into any other 
> machines.  it should be on all Gentoo desktops/severs/etc...  
> specialized/embedded/whatever are certainly free to cull openssh (and doing 
> so is actually beyond trivial).  whether we express this requirement in base/ 
> or frags or something is certainly open for discussion, but i believe 
> removing it from a stage3 in any of our standard releases is a huge 
> disservice to everyone.
> -mike

I wouldn't say ssh is any more important than a dhcp client.

Regards,
Petteri


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 260 bytes --]

Re: [gentoo-dev] [RFC] Reducing the size of the system package set

[Rémi Cardona]

Petteri Räty a écrit :

> I wouldn't say ssh is any more important than a dhcp client.

+1 from me.

With a quick glance at portage, I found at least those ssh implementations :
  - openssh
  - ssh
  - ossh
  - dropbear
  - ... (probably a few more whose name I couldn't find)

This might even be worth a virtual/ssh.

Cheers,

Rémi
-- 
gentoo-dev@lists.gentoo.org mailing list

Re: [gentoo-dev] [RFC] Reducing the size of the system package set

[Mike Frysinger]

[-- Attachment #1: Type: text/plain, Size: 665 bytes --]

On Thursday 10 January 2008, Rémi Cardona wrote:
> With a quick glance at portage, I found at least those ssh implementations
> : - openssh

none of those are suitable replacements for openssh.

>   - ssh

non-commercial only, not tested on all arches

>   - ossh

only tested on x86, no idea on quality

>   - dropbear

only good for embedded systems

>   - ... (probably a few more whose name I couldn't find)

probably all are just as unworthy

> This might even be worth a virtual/ssh.

that would only make sense if the packages were drop-in replacements (which 
they arent) and other packages actually needed to depend on it
-mike

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 835 bytes --]

[gentoo-dev] Re: [RFC] Reducing the size of the system package set

[Duncan]

Josh Saddler <nightmorph@gentoo.org> posted 478546B6.2080009@gentoo.org,
excerpted below, on  Wed, 09 Jan 2008 14:12:06 -0800:

> We all know that ssh is good for sysadmins and netadmins and Gentoo
> developers, etc. However, desktop users -- i.e., those not in those
> categories, which is most everyone else, likely do not find it as
> useful. [reluctant snip] If you *need* it to do work (admins, I'm
> looking at you), then you can *emerge* it. Just like vim, cvs,
> dev_tool_foo etc.
> 
> I *am* a desktop user. And . . . aside from Gentoo development, I have
> no need for ssh. It could easily be removed from the system profile --
> the only place it might be kept is in the liveCD environment, for remote
> installations. Other than that, we don't need to ship it in our stages;
> just on the media.

As another desktop user (who FWIW has his virtual/ssh entry pointed at 
sys-apps/baselayout, one more reason he's glad Gentoo's flexible like 
that =8^) ...

++

-- 
Duncan - List replies preferred.   No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master."  Richard Stallman

-- 
gentoo-dev@lists.gentoo.org mailing list

Re: [gentoo-dev] [RFC] Reducing the size of the system package set

[Michael Haubenwallner]

On Tue, 2008-01-08 at 14:37 -0800, Chris Gianelloni wrote:
> Implicit dependencies waste more time than pretty much anything else.  Almost all circular
> dependency issues we currently have are due to implicit dependencies.

Maybe OT (an idea in a very early state):

For the implicit dependencies one could think of path-sandbox to help:

Inform libsandbox which files are provided by packages both in *DEPEND
and the system package set, and let it completely deny access to
not-listed files.

Or let it report any access to not-listed files to help finding the
implicit dependencies.

Thoughts?

/haubi/
-- 
Michael Haubenwallner
Gentoo on a different level

-- 
gentoo-dev@lists.gentoo.org mailing list

Re: [gentoo-dev] [RFC] Reducing the size of the system package set

[Marius Mauch]

On Tue, 08 Jan 2008 00:42:57 +0100
flameeyes@gmail.com (Diego 'Flameeyes' Pettenò) wrote:

> I already ranted about the fact that the dependency tree of our
> ebuilds is vastly incomplete, as many lack dependency on zlib; trying
> to get this fixed was impossible, as Donnie and other insisted that
> as zlib was in system, we shouldn’t depend on it at all. I disagree,
> and I would like to know why we can’t depend on a system package, but
> whatever.

Somewhat related:
http://thread.gmane.org/gmane.linux.gentoo.portage.devel/2575

Marius
--
gentoo-dev@lists.gentoo.org mailing list

Re: [gentoo-dev] [RFC] Reducing the size of the system package set

[Santiago M. Mola]

On Jan 10, 2008 4:42 PM, Michael Haubenwallner <haubi@gentoo.org> wrote:
>
> For the implicit dependencies one could think of path-sandbox to help:
>
> Inform libsandbox which files are provided by packages both in *DEPEND
> and the system package set, and let it completely deny access to
> not-listed files.
>

That would rock, also to prevent automagic dependencies. Of course,
that should be a configurable option disabled by default.

-- 
Santiago M. Mola
Jabber ID: cooldwind@gmail.com
-- 
gentoo-dev@lists.gentoo.org mailing list

Re: [gentoo-dev] [RFC] Reducing the size of the system package set

[Mike Frysinger]

[-- Attachment #1: Type: text/plain, Size: 840 bytes --]

On Thursday 10 January 2008, Michael Haubenwallner wrote:
> On Tue, 2008-01-08 at 14:37 -0800, Chris Gianelloni wrote:
> > Implicit dependencies waste more time than pretty much anything else. 
> > Almost all circular dependency issues we currently have are due to
> > implicit dependencies.
>
> Maybe OT (an idea in a very early state):
>
> For the implicit dependencies one could think of path-sandbox to help:
>
> Inform libsandbox which files are provided by packages both in *DEPEND
> and the system package set, and let it completely deny access to
> not-listed files.
>
> Or let it report any access to not-listed files to help finding the
> implicit dependencies.

it'd be quite some time before this would see implementation, but can you 
summarize the ideas and open a feature request in bugzilla please
-mike

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 835 bytes --]

Re: [gentoo-dev] [RFC] Reducing the size of the system package set

[Michael Haubenwallner]

On Thu, 2008-01-10 at 14:32 -0500, Mike Frysinger wrote:
> On Thursday 10 January 2008, Michael Haubenwallner wrote:
> > On Tue, 2008-01-08 at 14:37 -0800, Chris Gianelloni wrote:
> > > Implicit dependencies waste more time than pretty much anything else. 
> > > Almost all circular dependency issues we currently have are due to
> > > implicit dependencies.
> >
> > Maybe OT (an idea in a very early state):
> >
> > For the implicit dependencies one could think of path-sandbox to help:
> >
> > Inform libsandbox which files are provided by packages both in *DEPEND
> > and the system package set, and let it completely deny access to
> > not-listed files.
> >
> > Or let it report any access to not-listed files to help finding the
> > implicit dependencies.
> 
> it'd be quite some time before this would see implementation, but can you 
> summarize the ideas and open a feature request in bugzilla please

done: http://bugs.gentoo.org/show_bug.cgi?id=205312

/haubi/
-- 
Michael Haubenwallner
Gentoo on a different level

-- 
gentoo-dev@lists.gentoo.org mailing list