* Re: [gentoo-dev] Packages that explicitly DEPEND on sys-apps/sed
2011-06-14 23:27 ` Brian Harring
@ 2011-06-17 0:51 ` Mike Frysinger
2011-06-17 2:13 ` [gentoo-dev] " Duncan
2011-06-17 18:50 ` [gentoo-dev] " Mike Frysinger
2011-06-17 2:53 ` [gentoo-dev] " Steven J Long
` (2 subsequent siblings)
3 siblings, 2 replies; 18+ messages in thread
From: Mike Frysinger @ 2011-06-17 0:51 UTC (permalink / raw
To: gentoo-dev
[-- Attachment #1: Type: Text/Plain, Size: 2572 bytes --]
On Tuesday, June 14, 2011 19:27:47 Brian Harring wrote:
> On Tue, Jun 14, 2011 at 10:08:54AM -0400, Rich Freeman wrote:
> > On Tue, Jun 14, 2011 at 8:54 AM, Brian Harring <ferringb@gmail.com> wrote:
> > > The implicit system set dependency thing really, really needs to die;
> > > at the time of the rule, portage couldn't handle resolving graphs of
> > > that sort. ?PM resolvers for gentoo are generally a fair bit saner
> > > now thus doing what you're suggesting isn't really beneficial (frankly
> > > it causes some issues for stages, as zac noted).
> >
> > ++
> >
> > It seems to me that the best policy would be for every package to just
> > list all its dependencies, and then users are free to run the default
> > experience that includes everything in @system, or a more trimmed-down
> > experience.
>
> An annoying, but valid complaint agains this is that the deps start
> getting heavy to maintain for developers, and aren't always viable to
> represent. Unpackers for example, are a pain in the ass for current
> EAPIs- that could be reduced in pain via addition of basic implicit
> deps to EAPI5 (if a src_uri ends in .bz2, then dep on virtual/bzip2).
i think you vastly underestimate how bad this is. what you're proposing is
the majority list:
sys-apps/baselayout (after all, it provides / /usr /usr/share /etc)
app-shells/bash
sys-apps/coreutils
sys-apps/gawk
sys-apps/grep
sys-apps/sed (every autotooled package uses it)
sys-apps/which (g'luck tracking down all the consumers)
sys-devel/gcc
sys-devel/binutils
virtual/libc
sys-apps/make
requiring people to remember to use these deps if their ebuild happens to
execute them is silly:
sys-apps/diffutils (provides `patch` after all)
sys-apps/findutils
archiving/compression deps absolutely need to be in the PM. otherwise you're
talking about:
app-arch/tar (a quick count shows 22k ebuilds need it out of all 27k)
app-arch/gzip (14k ebuilds)
app-arch/bzip2 (8k ebuilds)
these are mostly done already via the autotools eclass, so these should get
dropped from system:
sys-devel/autoconf
sys-devel/automake
sys-devel/libtool
sys-devel/m4
> The trickier point is gcc, but in my view, that's where we get the
> most gain- if the toolchain is represented in the deps it makes
> integrated cross compilation easier (keyword is integrated; crossdev
> already makes it reasonably straightforward I realize).
i dont understand this at all. sounds to me like this complicates cross-
compilation unnecessarily.
-mike
[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 836 bytes --]
^ permalink raw reply [flat|nested] 18+ messages in thread
* [gentoo-dev] Re: Packages that explicitly DEPEND on sys-apps/sed
2011-06-17 0:51 ` Mike Frysinger
@ 2011-06-17 2:13 ` Duncan
2011-06-17 2:46 ` Mike Frysinger
2011-06-17 18:50 ` [gentoo-dev] " Mike Frysinger
1 sibling, 1 reply; 18+ messages in thread
From: Duncan @ 2011-06-17 2:13 UTC (permalink / raw
To: gentoo-dev
Mike Frysinger posted on Thu, 16 Jun 2011 20:51:36 -0400 as excerpted:
>> An annoying, but valid complaint agains this is that the deps start
>> getting heavy to maintain for developers, and aren't always viable to
>> represent. Unpackers for example, are a pain in the ass for current
>> EAPIs- that could be reduced in pain via addition of basic implicit
>> deps to EAPI5 (if a src_uri ends in .bz2, then dep on virtual/bzip2).
>
> i think you vastly underestimate how bad this is. what you're proposing
> is the majority list:
> sys-apps/baselayout (after all, it provides / /usr /usr/share /
etc)
> app-shells/bash sys-apps/coreutils sys-apps/gawk sys-apps/grep
> sys-apps/sed (every autotooled package uses it)
> sys-apps/which (g'luck tracking down all the consumers) sys-devel/
gcc
> sys-devel/binutils virtual/libc sys-apps/make
>
> requiring people to remember to use these deps if their ebuild happens
> to execute them is silly:
> sys-apps/diffutils (provides `patch` after all)
> sys-apps/findutils
>
> archiving/compression deps absolutely need to be in the PM. otherwise
> you're talking about:
> app-arch/tar (a quick count shows 22k ebuilds need it out of all
27k)
> app-arch/gzip (14k ebuilds)
> app-arch/bzip2 (8k ebuilds)
>
> these are mostly done already via the autotools eclass, so these should
> get dropped from system:
> sys-devel/autoconf sys-devel/automake sys-devel/libtool sys-devel/
m4
That last comment, on the autotools eclass, suggests a solution, a
basedeps eclass, which most ebuilds could simply inherit.
The eclass could handle most archiving dependencies automatically, using
source URL extensions to determine type. Similarly with stuff like make
by scanning for calls to it or emake. Bash and baselayout would be
mandatory, and variables could be used for quick-list adds and overrides.
It'd take a LOT of work and testing and may even then not be workable
unless implemented as an arbitrary list that pretty much simply moves the
@system list from its current location into an eclass, but the idea's
interesting. Possible/practical or wishful thinking?
--
Duncan - List replies preferred. No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master." Richard Stallman
^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [gentoo-dev] Re: Packages that explicitly DEPEND on sys-apps/sed
2011-06-17 2:13 ` [gentoo-dev] " Duncan
@ 2011-06-17 2:46 ` Mike Frysinger
0 siblings, 0 replies; 18+ messages in thread
From: Mike Frysinger @ 2011-06-17 2:46 UTC (permalink / raw
To: gentoo-dev
[-- Attachment #1: Type: Text/Plain, Size: 2285 bytes --]
On Thursday, June 16, 2011 22:13:48 Duncan wrote:
> Mike Frysinger posted on Thu, 16 Jun 2011 20:51:36 -0400 as excerpted:
> > these are mostly done already via the autotools eclass, so these should
> > get dropped from system:
> > sys-devel/autoconf sys-devel/automake sys-devel/libtool sys-devel/m4
>
> That last comment, on the autotools eclass, suggests a solution, a
> basedeps eclass, which most ebuilds could simply inherit.
that's not what i was suggesting at all, nor do i think it feasible
> The eclass could handle most archiving dependencies automatically, using
> source URL extensions to determine type. Similarly with stuff like make
> by scanning for calls to it or emake.
the automatic code scanning wouldnt work as it would need to consider the
default implementation of functions (which really only the PM knows), and even
then it cant handle code which is dynamic (like the default PM funcs).
if [ -e Makefile ] ; then emake ; fi
how do you automatically detect utilities that only the package executes ?
> Bash and baselayout would be mandatory, and variables could be used for
> quick-list adds and overrides.
gee, that sounds exactly like the system set we already have, except this
solution generates a lot more overhead (all the execution/caching required to
process this new eclass), and more pains. the system set is in the profile
which other profiles can easily override (by design). much harder to pull
that off with eclasses.
> It'd take a LOT of work and testing and may even then not be workable
> unless implemented as an arbitrary list that pretty much simply moves the
> @system list from its current location into an eclass, but the idea's
> interesting. Possible/practical or wishful thinking?
impractical wishful thinking
further, many of these deps themselves depend on their own existence.
coreutils needs expr/touch/tail/head/...... which coreutils provides. bash
needs a shell which bash provides. sed needs sed which sed provides. gawk
needs gawk which gawk provides. make needs make which make provides.
friends also need their friends. make/bash/sed/grep/coreutils pretty much all
need each other. which is another reason why they're in the system set.
-mike
[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 836 bytes --]
^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [gentoo-dev] Packages that explicitly DEPEND on sys-apps/sed
2011-06-17 0:51 ` Mike Frysinger
2011-06-17 2:13 ` [gentoo-dev] " Duncan
@ 2011-06-17 18:50 ` Mike Frysinger
1 sibling, 0 replies; 18+ messages in thread
From: Mike Frysinger @ 2011-06-17 18:50 UTC (permalink / raw
To: gentoo-dev
[-- Attachment #1: Type: Text/Plain, Size: 1113 bytes --]
more thoughts as to why this is a bad idea ... how do you deal with runtime
library requirements which only the compiler knows about ? sys-devel/gcc
provides many runtime libraries such as libgcc_s.so. but whether the package
actually needs that at runtime may depend purely on the arch/abi, or what code
*just happens* to be generated. embedded cpus tend to need it more often than
desktop cpus because libgcc_s.so provides fun things like 64bit multiplication
and division routines when the hardware lacks support. so if your target
happens to do 64bit multiplication, you now have RDEPEND on sys-devel/gcc.
glibc itself will load up libgcc_s.so via dlopen when unwinding in threaded
situations. but this only necessary when the package uses threads, and needs
unwind support (iirc), and glibc is using NPTL. you could say "ah just have
glibc itself always require gcc", but if we're not being explicit in our deps,
i dont see any difference from the implicit system set (except in the "worse"
direction).
these dependencies cannot be expressed via ebuilds/eclasses.
-mike
[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 836 bytes --]
^ permalink raw reply [flat|nested] 18+ messages in thread
* [gentoo-dev] Re: Packages that explicitly DEPEND on sys-apps/sed
2011-06-14 23:27 ` Brian Harring
2011-06-17 0:51 ` Mike Frysinger
@ 2011-06-17 2:53 ` Steven J Long
2011-06-17 3:11 ` Mike Frysinger
2011-06-17 19:28 ` [gentoo-dev] " Bruno
2011-06-22 18:11 ` Donnie Berkholz
3 siblings, 1 reply; 18+ messages in thread
From: Steven J Long @ 2011-06-17 2:53 UTC (permalink / raw
To: gentoo-dev
Brian Harring wrote:
>> And no, I don't think that Gentoo should fully support reduced-@system
>> builds, but there is no harm in making them more of a viable option.
>
> Personally... I think gentoo should aim for it actually. Question is
> how close we can get to it w/out overly burdening developers.
>
I always thought of @system as providing a POSIX-compatible userland,
exactly so that deps wouldn't need to be listed. eg every POSIX compatible
system has to have find, sed etc[1] and of course ed which is missing,
annoyingly enough for scripting.
With regard to having a slimmed-down @system it might appear to make sense
to look at moving to only allowing POSIX options, testing perhaps with
sys-apps/minised[2] (assuming we'd have a virtual in @system) which I
haven't tried. AIUI however there's far too many sed GNUisms to go down
that road.
Even on FreeBSD they're still using gsed; the fact that people will not stop
using GNU-style regexes-- and that they need it for their own base system--
is only driving them to work on including the same extensions in their own
software.[3]
Given that we're not about to clear the GNUisms from the tree, it's been "a
system package since at least 2004" with dependencies on versions "as high
as 4.0.5, which went stable in 2003" I'd concur that the existing ebuild/
eclass dependencies should be trimmed as and when, if it can't be
automated.
Hopefully at some point there'll be another implementation with the same or
similar capabilities, which might even make it into POSIX. Until then it
would appear Gentoo needs gsed just like it needs bash, afaict. An embedded
system might well have a different profile; presumably applications would
not be built on the target tho.
[1] http://pubs.opengroup.org/onlinepubs/9699919799/utilities/contents.html
[2] http://www.exactcode.de/oss/minised/
[3] See point 3 at:
http://freebsd.1045724.n5.nabble.com/RFC-Replacing-our-regex-implementation-td4380832.html
--
#friendly-coders -- We're friendly but we're not /that/ friendly ;-)
^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [gentoo-dev] Re: Packages that explicitly DEPEND on sys-apps/sed
2011-06-17 2:53 ` [gentoo-dev] " Steven J Long
@ 2011-06-17 3:11 ` Mike Frysinger
2011-06-18 6:26 ` Jeroen Roovers
0 siblings, 1 reply; 18+ messages in thread
From: Mike Frysinger @ 2011-06-17 3:11 UTC (permalink / raw
To: gentoo-dev
[-- Attachment #1: Type: Text/Plain, Size: 777 bytes --]
On Thursday, June 16, 2011 22:53:23 Steven J Long wrote:
> With regard to having a slimmed-down @system it might appear to make sense
> to look at moving to only allowing POSIX options, testing perhaps with
> sys-apps/minised[2] (assuming we'd have a virtual in @system) which I
> haven't tried. AIUI however there's far too many sed GNUisms to go down
> that road.
we've already discussed POSIX strictness in the tree multiple times, and it's
been shot down multiple times for good reason. GNU extensions make life a
hell of a lot easier, the majority of people are developing/testing on such
systems (Linux with GNU userland), and the burden on other ports is trivial if
not negligible. let's stop kicking this horse as we're at the glue stage now.
-mike
[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 836 bytes --]
^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [gentoo-dev] Re: Packages that explicitly DEPEND on sys-apps/sed
2011-06-17 3:11 ` Mike Frysinger
@ 2011-06-18 6:26 ` Jeroen Roovers
2011-06-18 17:40 ` Mike Frysinger
0 siblings, 1 reply; 18+ messages in thread
From: Jeroen Roovers @ 2011-06-18 6:26 UTC (permalink / raw
To: gentoo-dev
On Thu, 16 Jun 2011 23:11:36 -0400
Mike Frysinger <vapier@gentoo.org> wrote:
> we've already discussed POSIX strictness in the tree multiple times,
> and it's been shot down multiple times for good reason. GNU
> extensions make life a hell of a lot easier, the majority of people
> are developing/testing on such systems (Linux with GNU userland), and
> the burden on other ports is trivial if not negligible. let's stop
> kicking this horse as we're at the glue stage now. -mike
OK, right, so where are we heading with this? sys-apps/sed is in system
and it's bound to be >=sed-4 and we're sticking with it. So are we
going to get rid of it in all the ebuilds that still set that dep?
jer
^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [gentoo-dev] Re: Packages that explicitly DEPEND on sys-apps/sed
2011-06-18 6:26 ` Jeroen Roovers
@ 2011-06-18 17:40 ` Mike Frysinger
0 siblings, 0 replies; 18+ messages in thread
From: Mike Frysinger @ 2011-06-18 17:40 UTC (permalink / raw
To: gentoo-dev
[-- Attachment #1: Type: Text/Plain, Size: 666 bytes --]
On Saturday, June 18, 2011 02:26:56 Jeroen Roovers wrote:
> OK, right, so where are we heading with this? sys-apps/sed is in system
> and it's bound to be >=sed-4 and we're sticking with it. So are we
> going to get rid of it in all the ebuilds that still set that dep?
for the packages that just depend on raw sed, it should most likely be
dropped. for those depending on a newer version, they're probably doing it
because they dont work with an older one, and those cant be dropped until our
system set forces a new enough sed. the base profile only requires "sys-
apps/sed" atm, so that'd probably need updating to read ">=sys-apps/sed-4".
-mike
[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 836 bytes --]
^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [gentoo-dev] Packages that explicitly DEPEND on sys-apps/sed
2011-06-14 23:27 ` Brian Harring
2011-06-17 0:51 ` Mike Frysinger
2011-06-17 2:53 ` [gentoo-dev] " Steven J Long
@ 2011-06-17 19:28 ` Bruno
2011-06-22 18:11 ` Donnie Berkholz
3 siblings, 0 replies; 18+ messages in thread
From: Bruno @ 2011-06-17 19:28 UTC (permalink / raw
To: gentoo-dev
On Tue, 14 June 2011 Brian Harring <ferringb@gmail.com> wrote:
> On Tue, Jun 14, 2011 at 10:08:54AM -0400, Rich Freeman wrote:
> > On Tue, Jun 14, 2011 at 8:54 AM, Brian Harring <ferringb@gmail.com> wrote:
> > > The implicit system set dependency thing really, really needs to die;
> > > at the time of the rule, portage couldn't handle resolving graphs of
> > > that sort. ?PM resolvers for gentoo are generally a fair bit saner
> > > now thus doing what you're suggesting isn't really beneficial (frankly
> > > it causes some issues for stages, as zac noted).
> >
> > ++
> >
> > It seems to me that the best policy would be for every package to just
> > list all its dependencies, and then users are free to run the default
> > experience that includes everything in @system, or a more trimmed-down
> > experience.
>
> An annoying, but valid complaint agains this is that the deps start
> getting heavy to maintain for developers, and aren't always viable to
> represent. Unpackers for example, are a pain in the ass for current
> EAPIs- that could be reduced in pain via addition of basic implicit
> deps to EAPI5 (if a src_uri ends in .bz2, then dep on virtual/bzip2).
>
> Or devs could just be nudged into adding the appropriate DEPEND.
> repoman checking for it either way wouldn't be hard.
IMHO a big distinction between DEPEND and RDEPEND should be done.
For RDEPEND all dependencies should be listed (including those packages
provided by @system)
This would allow easy installing into chroots with package manager's
help, especially in combination with binary packages.
For DEPEND it could be sufficient to assume @system is present or at
least limit to those dependencies needed by the package/ebuild itself,
but not those coming implicitly with features of package manager used
(e.g. default unpacking, emake, einstall, do*)
Eclasses "extending" package manager features should include their extra
DEPEND needs.
On the other hand, it would be nice if package manager could auto-detect
at least part of the runtime dependencies (library linking should be easy
as package manager already looks for them -- what's completely missing
is shebang/interpreter dependencies).
This way only version constraints would need to get added to RDEPEND
inside ebuilds as needed (and the few cases where dlopen makes it
impossible for package manager to see the linking).
A nice benefit of this would be that it can adapt to changes caused by
INSTALL_MASK, e.g. reduces dependencies that are not needed anymore
because some files were not installed.
Bruno
^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [gentoo-dev] Packages that explicitly DEPEND on sys-apps/sed
2011-06-14 23:27 ` Brian Harring
` (2 preceding siblings ...)
2011-06-17 19:28 ` [gentoo-dev] " Bruno
@ 2011-06-22 18:11 ` Donnie Berkholz
3 siblings, 0 replies; 18+ messages in thread
From: Donnie Berkholz @ 2011-06-22 18:11 UTC (permalink / raw
To: gentoo-dev
[-- Attachment #1: Type: text/plain, Size: 1177 bytes --]
On 16:27 Tue 14 Jun , Brian Harring wrote:
> On Tue, Jun 14, 2011 at 10:08:54AM -0400, Rich Freeman wrote:
> > And no, I don't think that Gentoo should fully support reduced-@system
> > builds, but there is no harm in making them more of a viable option.
>
> Personally... I think gentoo should aim for it actually. Question is
> how close we can get to it w/out overly burdening developers.
Where this has been most useful for me is when I'm building out a
minimal system (e.g., a diskless terminal or cluster node) using
ROOT=/somewhere/else. It's nice to just start emerging stuff there
instead of having to unpack a stage1 or something first.
I wonder if we need another set that's really @base (truly minimal, like
what Mike posted elsewhere), so @system would then serve as what we
think is necessary for a running Gentoo installation.
On a related note that would accomplish similar purposes, it would also
be nice if we could somehow discriminate between DEPEND and RDEPEND for
@system packages so build-only deps could be removed.
--
Thanks,
Donnie
Donnie Berkholz
Sr. Developer, Gentoo Linux
Blog: http://dberkholz.com
[-- Attachment #2: Type: application/pgp-signature, Size: 198 bytes --]
^ permalink raw reply [flat|nested] 18+ messages in thread