[gentoo-dev] sys-libs/zlib: punt from system in profiles

[gentoo-dev] sys-libs/zlib: punt from system in profiles

[Mike Frysinger]

[-- Attachment #1: Type: text/plain, Size: 239 bytes --]

we have USE=zlib already which should cover automatically pulling in zlib when 
necessary, and we have that by default in make.conf.  so there's no need to 
explicitly list zlib as part of the system target.  so time to drop it.
-mike

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 836 bytes --]

Re: [gentoo-dev] sys-libs/zlib: punt from system in profiles

[Matt Turner]

On Tue, Nov 29, 2011 at 3:09 PM, Mike Frysinger <vapier@gentoo.org> wrote:
> we have USE=zlib already which should cover automatically pulling in zlib when
> necessary, and we have that by default in make.conf.  so there's no need to
> explicitly list zlib as part of the system target.  so time to drop it.
> -mike

I don't object to this, but just to satisfy my curiosity, what are the
effects of removing, say, zlib or readline from the system profile?
It'd be very difficult if not impossible to construct a system without
these. The only effect I've noticed from changes like these are that
sometimes the packages will be depcleaned from stage3 tarballs if
nothing has them in RDEPEND.

Are there any other cases?

Matt

Re: [gentoo-dev] sys-libs/zlib: punt from system in profiles

[Zac Medico]

On 11/29/2011 06:14 PM, Matt Turner wrote:
> On Tue, Nov 29, 2011 at 3:09 PM, Mike Frysinger <vapier@gentoo.org> wrote:
>> we have USE=zlib already which should cover automatically pulling in zlib when
>> necessary, and we have that by default in make.conf.  so there's no need to
>> explicitly list zlib as part of the system target.  so time to drop it.
>> -mike
> 
> I don't object to this, but just to satisfy my curiosity, what are the
> effects of removing, say, zlib or readline from the system profile?
> It'd be very difficult if not impossible to construct a system without
> these. The only effect I've noticed from changes like these are that
> sometimes the packages will be depcleaned from stage3 tarballs if
> nothing has them in RDEPEND.
> 
> Are there any other cases?

One nice thing about removing them from the system profile is that it
allows for greater parallelization with emerge --jobs, since packages
that's aren't in the system set don't trigger the code for bug 256616
[1] which limits parallelization as a safety precaution.

[1] https://bugs.gentoo.org/show_bug.cgi?id=256616
-- 
Thanks,
Zac

Re: [gentoo-dev] sys-libs/zlib: punt from system in profiles

[Mike Frysinger]

[-- Attachment #1: Type: Text/Plain, Size: 1175 bytes --]

On Tuesday 29 November 2011 21:14:49 Matt Turner wrote:
> On Tue, Nov 29, 2011 at 3:09 PM, Mike Frysinger <vapier@gentoo.org> wrote:
> > we have USE=zlib already which should cover automatically pulling in zlib
> > when necessary, and we have that by default in make.conf.  so there's no
> > need to explicitly list zlib as part of the system target.  so time to
> > drop it.
> 
> I don't object to this, but just to satisfy my curiosity, what are the
> effects of removing, say, zlib or readline from the system profile?
> It'd be very difficult if not impossible to construct a system without
> these. The only effect I've noticed from changes like these are that
> sometimes the packages will be depcleaned from stage3 tarballs if
> nothing has them in RDEPEND.

you should be able to USE='-readline -zlib' and, for the most part, not have 
them get used.  certainly for readline, almost nothing in the core (or much of 
the tree) uses it -- if we ignore bash because it bundles a copy of readline.

i also want to keep it simple for slimmed down systems, and to keep the "core" 
set as small as possible for older systems during upgrade paths.
-mike

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 836 bytes --]

Re: [gentoo-dev] sys-libs/zlib: punt from system in profiles

[Mike Frysinger]

[-- Attachment #1: Type: Text/Plain, Size: 1062 bytes --]

On Tuesday 29 November 2011 21:14:49 Matt Turner wrote:
> On Tue, Nov 29, 2011 at 3:09 PM, Mike Frysinger <vapier@gentoo.org> wrote:
> > we have USE=zlib already which should cover automatically pulling in zlib
> > when necessary, and we have that by default in make.conf.  so there's no
> > need to explicitly list zlib as part of the system target.  so time to
> > drop it.
> 
> I don't object to this, but just to satisfy my curiosity, what are the
> effects of removing, say, zlib or readline from the system profile?
> It'd be very difficult if not impossible to construct a system without
> these. The only effect I've noticed from changes like these are that
> sometimes the packages will be depcleaned from stage3 tarballs if
> nothing has them in RDEPEND.

another advantage: manual searches of packages linking against readline shows 
a bunch of packages lacking USE=readline control (which i've fixed up).  
hopefully people who dep clean this thing will notice and file bugs (we've had 
a few along those lines in the pass).
-mike

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 836 bytes --]

[gentoo-dev] Re: sys-libs/zlib: punt from system in profiles

[Duncan]

Zac Medico posted on Tue, 29 Nov 2011 18:29:20 -0800 as excerpted:

> One nice thing about removing them from the system profile is that it
> allows for greater parallelization with emerge --jobs, since packages
> that's aren't in the system set don't trigger the code for bug 256616
> [1] which limits parallelization as a safety precaution.
> 
> [1] https://bugs.gentoo.org/show_bug.cgi?id=256616

And a VERY nice benefit that is, too. =:^)

Except, don't dependencies of system profile packages get the same safety 
precaution applied, and at the level we're talking here, zlib, etc, 
aren't such packages going to almost certainly be dependencies of @system 
on a normal system, even if they're not in @system itself?

If so, that's a relatively limited benefit in most cases.

Meanwhile, the better flexibility in terms of embedded is the benefit 
that immediately comes to mind here.  From what I've read, gentoo is 
quite popular with embedded, and a lot of embedded folks go to a lot of 
work to remove "unnecessary" packages from the system set.  If it's 
possible to do that at the general gentoo level without too much trouble 
and without affecting general system target functionality, it's surely 
going to save a lot of embedded folks a lot of duplicated effort, over 
time.

-- 
Duncan - List replies preferred.   No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master."  Richard Stallman

Re: [gentoo-dev] Re: sys-libs/zlib: punt from system in profiles

[Zac Medico]

On 11/29/2011 08:51 PM, Duncan wrote:
> Zac Medico posted on Tue, 29 Nov 2011 18:29:20 -0800 as excerpted:
> 
>> One nice thing about removing them from the system profile is that it
>> allows for greater parallelization with emerge --jobs, since packages
>> that's aren't in the system set don't trigger the code for bug 256616
>> [1] which limits parallelization as a safety precaution.
>>
>> [1] https://bugs.gentoo.org/show_bug.cgi?id=256616
> 
> And a VERY nice benefit that is, too. =:^)
> 
> Except, don't dependencies of system profile packages get the same safety 
> precaution applied, and at the level we're talking here, zlib, etc, 
> aren't such packages going to almost certainly be dependencies of @system 
> on a normal system, even if they're not in @system itself?
> 
> If so, that's a relatively limited benefit in most cases.

Well, maybe there's not much parallelization benefit for sys-libs/zlib.
If it wasn't for implicit system dependencies, the system set and its
dependencies wouldn't need this kind of special treatment.
-- 
Thanks,
Zac

[gentoo-dev] Re: sys-libs/zlib: punt from system in profiles

[Duncan]

Zac Medico posted on Tue, 29 Nov 2011 22:23:59 -0800 as excerpted:

> On 11/29/2011 08:51 PM, Duncan wrote:
>> Zac Medico posted on Tue, 29 Nov 2011 18:29:20 -0800 as excerpted:
>> 
>>> One nice thing about removing them from the system profile is that it
>>> allows for greater parallelization with emerge --jobs[.]
>>> https://bugs.gentoo.org/show_bug.cgi?id=256616

>> Except, don't dependencies of system profile packages get the same
>> safety precaution applied [and such is zlib]?

> Well, maybe there's not much parallelization benefit for sys-libs/zlib.
> If it wasn't for implicit system dependencies, the system set and its
> dependencies wouldn't need this kind of special treatment.

Hmm... very good point, from a man that "oughta" know. =:^)

So while this one removal from @system isn't much, reducing @system over 
time both reduces the effective footprint of the problem, and encourages 
explicit dependencies in a way that both cuts down on the need for 
special @system treatment in the first place, and encourages further 
reductions to the system set.

The end result of that process sounds like something I can live with. =:^)

Thanks.

-- 
Duncan - List replies preferred.   No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master."  Richard Stallman

Re: [gentoo-dev] Re: sys-libs/zlib: punt from system in profiles

[Mike Frysinger]

[-- Attachment #1: Type: Text/Plain, Size: 632 bytes --]

On Wednesday 30 November 2011 01:23:59 Zac Medico wrote:
> If it wasn't for implicit system dependencies, the system set and its
> dependencies wouldn't need this kind of special treatment.

if it wasn't for implicit system dependencies, we'd have significant bloat in 
the tree and circular deps out the wazoo

there's absolutely no reason every package should depend on sh, find, gawk, ls, 
sed, grep, make, gcc, libc, as/ld

i wouldn't mind seeing gzip/bzip2/xz being auto calculated from SRC_URI ... 
though it would make a system without any of those tricky to bootstrap, so 
we'd have to pick at least one
-mike

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 836 bytes --]

Re: [gentoo-dev] Re: sys-libs/zlib: punt from system in profiles

[Zac Medico]

On 11/30/2011 08:09 AM, Mike Frysinger wrote:
> On Wednesday 30 November 2011 01:23:59 Zac Medico wrote:
>> If it wasn't for implicit system dependencies, the system set and its
>> dependencies wouldn't need this kind of special treatment.
> 
> if it wasn't for implicit system dependencies, we'd have significant bloat in 
> the tree

I'm not suggesting that we do away with implicit dependencies entirely.
However, I think it's good to minimize them, as removing packages like
sys-libs/zlib from the system set tends to do.

> and circular deps out the wazoo

Ignoring circular dependencies doesn't make them go away. Ignoring
dependencies can lead to build failures that could have been avoided if
they were expressed in a way that the dependency resolver could properly
account for them.

> there's absolutely no reason every package should depend on sh, find, gawk, ls, 
> sed, grep, make, gcc, libc, as/ld

Agreed, but as said, it's good to minimize these implicit deps.

> i wouldn't mind seeing gzip/bzip2/xz being auto calculated from SRC_URI ... 
> though it would make a system without any of those tricky to bootstrap, so 
> we'd have to pick at least one

Yeah, that would be a nice EAPI extension.
-- 
Thanks,
Zac

Re: [gentoo-dev] Re: sys-libs/zlib: punt from system in profiles

[Rich Freeman]

On Wed, Nov 30, 2011 at 11:34 AM, Zac Medico <zmedico@gentoo.org> wrote:
> Ignoring circular dependencies doesn't make them go away. Ignoring
> dependencies can lead to build failures that could have been avoided if
> they were expressed in a way that the dependency resolver could properly
> account for them.

++

One man's gawk is another man's KDE.  A solution that expresses all
dependencies and handles them well is much more elegant than one that
requires hard-coding a list of core dependencies that we just think
are too complicated to work out.

Of course, to really get to the point where we'd have no system set at
all we'd need to somehow automate the dependency generation, since
otherwise ebuild maintenance would be very painful.  Considering that
we can't even tell if a program will halt it is clearly impossible to
guarantee a perfect set of runtime dependencies.  Of course, you might
be able to come up with something that is good enough - especially
when combined with the ability to add them in manually.

None of this completely solves the fundamental bootstrapping problem.
However, a full set of dependency specifications would let you at
least determine what the minimal bootstrap actually is.

I see all of this as more of an aspirational goal - one that we
shouldn't fret about not being able to achieve, or subject ourselves
to tremendous pain to get a little closer to.  However, we also
shouldn't hold back when opportunities to take a step closer arise.

Rich

Re: [gentoo-dev] Re: sys-libs/zlib: punt from system in profiles

[Mike Frysinger]

[-- Attachment #1: Type: Text/Plain, Size: 1277 bytes --]

On Wednesday 30 November 2011 11:34:05 Zac Medico wrote:
> On 11/30/2011 08:09 AM, Mike Frysinger wrote:
> > On Wednesday 30 November 2011 01:23:59 Zac Medico wrote:
> >> If it wasn't for implicit system dependencies, the system set and its
> >> dependencies wouldn't need this kind of special treatment.
> > 
> > if it wasn't for implicit system dependencies, we'd have significant
> > bloat in the tree
> 
> I'm not suggesting that we do away with implicit dependencies entirely.
> However, I think it's good to minimize them, as removing packages like
> sys-libs/zlib from the system set tends to do.

i'm good with this as well which is why i've been culling the simpler stuff

> > and circular deps out the wazoo
> 
> Ignoring circular dependencies doesn't make them go away. Ignoring
> dependencies can lead to build failures that could have been avoided if
> they were expressed in a way that the dependency resolver could properly
> account for them.

that doesn't address the previously mentioned issue.  there's absolutely no 
reason why the majority of packages in the tree should have to list 
"coreutils" or "sed" or "grep" in their DEPEND, nor is there any reason why 
the system should have been lacking them in the first place.
-mike

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 836 bytes --]

Re: [gentoo-dev] sys-libs/zlib: punt from system in profiles

[Mike Frysinger]

[-- Attachment #1: Type: Text/Plain, Size: 320 bytes --]

On Tuesday 29 November 2011 15:09:33 Mike Frysinger wrote:
> we have USE=zlib already which should cover automatically pulling in zlib
> when necessary, and we have that by default in make.conf.  so there's no
> need to explicitly list zlib as part of the system target.  so time to
> drop it.

this is now punted
-mike

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 836 bytes --]