[gentoo-user] Skipping static libraries

[gentoo-user] Skipping static libraries

[Enrico Weigelt]

Hi folks,

is there a way for telling emerge that it should NOT install
static (.a) libraries ?

thx
-- 
---------------------------------------------------------------------
 Enrico Weigelt    ==   metux IT service - http://www.metux.de/
---------------------------------------------------------------------
 Please visit the OpenSource QM Taskforce:
 	http://wiki.metux.de/public/OpenSource_QM_Taskforce
 Patches / Fixes for a lot dozens of packages in dozens of versions:
	http://patches.metux.de/
---------------------------------------------------------------------
-- 
gentoo-user@lists.gentoo.org mailing list

Re: [gentoo-user] Skipping static libraries

[Alan McKinnon]

On Friday 14 March 2008, Enrico Weigelt wrote:
> Hi folks,
>
> is there a way for telling emerge that it should NOT install
> static (.a) libraries ?

I shan't answer your question as I ca never find the answer myself, but 
I keep wanting to do this myself, then I remember why portage 
installs .a by default

A long time ago I read a HUGE thread on b.g.o. about this, the eventual 
conclusion is that after installation of a lib, a user might well want 
to link those libs statically and if they are not there, portage will 
barf big time and has no way to recover or even know what went wrong. 
Then the poor user sits in mystery wondering which package to remerge 
to get the .a

In short, the devs at the time reckoned it was a huge risk at the 
expense of not much disk space (does seem reasonable). Ubuntu can get 
away with this, the user gets what the packager feels like giving them, 
our users have *much* more freedom.

What's your reasoning for wanting to do this?
-- 
Alan McKinnon
alan dot mckinnon at gmail dot com

-- 
gentoo-user@lists.gentoo.org mailing list

Re: [gentoo-user] Skipping static libraries

[Enrico Weigelt]

* Alan McKinnon <alan.mckinnon@gmail.com> wrote:

> A long time ago I read a HUGE thread on b.g.o. about this, the eventual 
> conclusion is that after installation of a lib, a user might well want 
> to link those libs statically and if they are not there, portage will 
> barf big time and has no way to recover or even know what went wrong. 
> Then the poor user sits in mystery wondering which package to remerge 
> to get the .a

Well, if there was an optional useflag (ie. "staticlib") which is 
turned on by default (or maybe "nostaticlib"), I don't see any
problem. Most people will leave them enabled, only who are sure
about this will disable them.

BTW: this leads me to another issue, which goes aroud in my mind for 
a longer time: packages should be split off into several "stripes"
in the binary output. Other packages then could depend on single
stripes at different levels. For example: if some package foo requires
libbar, it would only depend on the buildtime stripe(s) for building
(eg. so headers get pulled in), while at runtime it will only depend
on the runtime stripe(s) of "bar". There could also be separate stripes
for certain locales, docs, examples, etc.

Such an approach could be really useful for binary-only targets (which
don't build themselves) and network installations (eg. arch-dependent
and -independent stripes could be installed separately). 

> Ubuntu can get away with this, the user gets what the packager feels 
> like giving them, our users have *much* more freedom.

I don't have much experiences w/ Ubuntu, but AFAIK this is an binary-
distro. Those distros usually split the (binary) output into two 
separate packages, eg. "libfoo" is only the binary library, while 
"libfoo-devel" contains all the buildtime stuff (*.h, *.pc, ...).
While this approach is quite simple, it's not very flexible and does't
allow finer granulation (eg. separating locales, etc), but requires
great care for filing dependencies.

> What's your reasoning for wanting to do this?

a) saving space / skipping unused stuff
b) preventing accidential static linking. 

While working on PHP and IMAP c-client, I discovered a broken 
libc-client.so* installation (turned out that just a symlink was missing).
PHP always tried to build in the static library (since .so wasnt found),
which leads to missing deps (eg. crypt,pam), after I removed the explicit
deps within PHP. The whole issue began with PHP breaking at this point
after I rebuild evrything w/o pam. PHP explicitly links in pam and crypt
for IMAP c-client if it finds them. Obviously an dirty hack around 
libc-client's extremly poor engineering.


cu
-- 
---------------------------------------------------------------------
 Enrico Weigelt    ==   metux IT service - http://www.metux.de/
---------------------------------------------------------------------
 Please visit the OpenSource QM Taskforce:
 	http://wiki.metux.de/public/OpenSource_QM_Taskforce
 Patches / Fixes for a lot dozens of packages in dozens of versions:
	http://patches.metux.de/
---------------------------------------------------------------------
-- 
gentoo-user@lists.gentoo.org mailing list

Re: [gentoo-user] Skipping static libraries

[Alan McKinnon]

On Saturday 15 March 2008, Enrico Weigelt wrote:
> * Alan McKinnon <alan.mckinnon@gmail.com> wrote:
> > A long time ago I read a HUGE thread on b.g.o. about this, the
> > eventual conclusion is that after installation of a lib, a user
> > might well want to link those libs statically and if they are not
> > there, portage will barf big time and has no way to recover or even
> > know what went wrong. Then the poor user sits in mystery wondering
> > which package to remerge to get the .a
>
> Well, if there was an optional useflag (ie. "staticlib") which is
> turned on by default (or maybe "nostaticlib"), I don't see any
> problem. Most people will leave them enabled, only who are sure
> about this will disable them.

OK, I see your reasoning. It fits in the same general category 
where "USE=build" belongs - highly specialized.

However, I can predict with confidence that your chances of getting this 
accepted into portage are zero, for the following excellent reasons:

1. regardless of how many warning and how many FAQs state to not do it, 
a sizeable portion of idiots out there are going to enable it 
anyway "just to see what happens". We usually call these people ricers, 
it's the same twits who set CFLAGS=-O9999 and expect devs to help them 
out of the hole they dig for themselves. Most devs will take one look 
at this and decide why should they have their inbox filled with support 
mails from twits who do YADT (yet another dumb thing) at no real 
benefit to the dev himself? Heck, I wouldn't.

2. It's likely to break an unknown number of things in the tree. There 
is no easy way to know which ebuilds require a .a to be present, short 
of actually building everything in that state and seeing what breaks. 
Most devs would compare this to #1 and point blank refuse to support it 
for the same reason

3. Considering the current state of portage, good luck on finding a dev 
willing to build such a low-level feature into it :-) Getting it into 
paludis would be much easier IMHO

In summary, you have an excellent chance of breaking stuff that isn't 
broken for an arguably zero or near-zero benefit

> BTW: this leads me to another issue, which goes aroud in my mind for
> a longer time: packages should be split off into several "stripes"
> in the binary output. Other packages then could depend on single
> stripes at different levels. For example: if some package foo
> requires libbar, it would only depend on the buildtime stripe(s) for
> building (eg. so headers get pulled in), while at runtime it will
> only depend on the runtime stripe(s) of "bar". There could also be
> separate stripes for certain locales, docs, examples, etc.
>
> Such an approach could be really useful for binary-only targets
> (which don't build themselves) and network installations (eg.
> arch-dependent and -independent stripes could be installed
> separately).

Well, you do only need the headers to build, but the whole point of 
building the app is to run it, which requires the binaries anyway.

Frankly, it's easier to just manually delete the headers when the build 
is done

> > Ubuntu can get away with this, the user gets what the packager
> > feels like giving them, our users have *much* more freedom.
>
> I don't have much experiences w/ Ubuntu, but AFAIK this is an binary-
> distro. Those distros usually split the (binary) output into two
> separate packages, eg. "libfoo" is only the binary library, while
> "libfoo-devel" contains all the buildtime stuff (*.h, *.pc, ...).
> While this approach is quite simple, it's not very flexible and
> does't allow finer granulation (eg. separating locales, etc), but
> requires great care for filing dependencies.

Yes, Ubuntu is binary. The packagers operate under policies that say 
things like "You will always use dynamic linking", so they could delete 
all static libs from the shipped packages, as any such package that 
requires a .a is by definition broken.

The foo and foo-devel split is mostly a historical thing, dating from 
way back in the days of Red Hat 5 or even earlier. Back then, most 
users were on dial up (and binaries were much smaller than today) 
so -devel packages were introduced mostly to conserve bandwidth and 
resulting disk space. It's very much a hack actually.

You could successfully argue that this is no longer the case and we 
don't need -devel packages anymore as bandwidth and disk space are 
cheap. Add the fact that -devel packages cause lots of confusion to 
users with mysterious failures when they build something.

> > What's your reasoning for wanting to do this?
>
> a) saving space / skipping unused stuff
> b) preventing accidential static linking.
>
> While working on PHP and IMAP c-client, I discovered a broken
> libc-client.so* installation (turned out that just a symlink was
> missing). PHP always tried to build in the static library (since .so
> wasnt found), which leads to missing deps (eg. crypt,pam), after I
> removed the explicit deps within PHP. The whole issue began with PHP
> breaking at this point after I rebuild evrything w/o pam. PHP
> explicitly links in pam and crypt for IMAP c-client if it finds them.
> Obviously an dirty hack around libc-client's extremly poor
> engineering.

Hmmm. There are many thousand ebuilds in the tree. Many more in 3rd 
party overlays. Your idea fixes 1 problem in 1 ebuild.

My vote goes to the solution that tells the php packager to find a work 
around for the problem that they can check in their ebuild without 
disrupting the result of the ecosystem

-- 
Alan McKinnon
alan dot mckinnon at gmail dot com

-- 
gentoo-user@lists.gentoo.org mailing list

Re: [gentoo-user] Skipping static libraries

[Chris Brennan]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1


| Hmmm. There are many thousand ebuilds in the tree. Many more in 3rd
| party overlays. Your idea fixes 1 problem in 1 ebuild.


Just for the sake of amusment and to give a sense of perspective. By my
count (ls -lshaR /usr/portage | grep ebuild | wc -l) I get 24,708
ebuilds (and that's from a sync at 0000-5)
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.7 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFH2++e8hUIAnGfls4RAvywAKCJyKnB2oQv5UDmIwZRVLDttEWALACeL2s3
2CvwrYQaNhLgJiVg36kH2+c=
=9lc3
-----END PGP SIGNATURE-----
-- 
gentoo-user@lists.gentoo.org mailing list

Re: [gentoo-user] Skipping static libraries

[Etaoin Shrdlu]

On Saturday 15 March 2008, Chris Brennan wrote:

> | Hmmm. There are many thousand ebuilds in the tree. Many more in 3rd
> | party overlays. Your idea fixes 1 problem in 1 ebuild.
>
> Just for the sake of amusment and to give a sense of perspective. By
> my count (ls -lshaR /usr/portage | grep ebuild | wc -l) I get 24,708
> ebuilds (and that's from a sync at 0000-5)

Cool. Since some packages have multiple ebuilds, to get an approximate 
number of unique packages in portage, I did something like this

$ find /usr/portage | grep 'metadata\.xml' | wc -l
12618

Still an excellent number...and this is only for the official tree!
-- 
gentoo-user@lists.gentoo.org mailing list

Re: [gentoo-user] Skipping static libraries

[Alan McKinnon]

On Saturday 15 March 2008, Etaoin Shrdlu wrote:
> On Saturday 15 March 2008, Chris Brennan wrote:
> > | Hmmm. There are many thousand ebuilds in the tree. Many more in
> > | 3rd party overlays. Your idea fixes 1 problem in 1 ebuild.
> >
> > Just for the sake of amusment and to give a sense of perspective.
> > By my count (ls -lshaR /usr/portage | grep ebuild | wc -l) I get
> > 24,708 ebuilds (and that's from a sync at 0000-5)
>
> Cool. Since some packages have multiple ebuilds, to get an
> approximate number of unique packages in portage, I did something
> like this
>
> $ find /usr/portage | grep 'metadata\.xml' | wc -l
> 12618
>
> Still an excellent number...and this is only for the official tree!

It's been a while since I ran that one myself.

12618 packages! Didnt realise it was so many. And just the sunrise 
overlay has 943.

That's probably more packages than even Debian has ...

-- 
Alan McKinnon
alan dot mckinnon at gmail dot com

-- 
gentoo-user@lists.gentoo.org mailing list

Re: [gentoo-user] Skipping static libraries

[Etaoin Shrdlu]

On Sunday 16 March 2008, Alan McKinnon wrote:

> 12618 packages! Didnt realise it was so many. And just the sunrise
> overlay has 943.
>
> That's probably more packages than even Debian has ...

Well, while I love Gentoo and would never change it with anything else, 
Debian is (still) the undiscussed champion in terms of number of 
available packages.

$ wget -nv http://packages.debian.org/stable/allpackages
12:32:32 URL:http://packages.debian.org/stable/allpackages 
[3585331/3585331] -> "allpackages" [1]
$ grep '^<dt>' allpackages | wc -l
22753

For unstable, the number is even higher:

$ rm allpackages
$ wget -nv http://packages.debian.org/unstable/allpackages
12:34:12 URL:http://packages.debian.org/unstable/allpackages 
[4924226/4924226] -> "allpackages" [1]
$ grep '^<dt>' allpackages | wc -l
27602


But we'll get there as well! :-)
-- 
gentoo-user@lists.gentoo.org mailing list

Re: [gentoo-user] Skipping static libraries

[Emil Beinroth]

[-- Attachment #1: Type: text/plain, Size: 1176 bytes --]

On Sun, Mar 16, 2008 at 12:11:21PM +0100, Etaoin Shrdlu wrote:
> Well, while I love Gentoo and would never change it with anything else, 
> Debian is (still) the undiscussed champion in terms of number of 
> available packages.
> 
> $ wget -nv http://packages.debian.org/stable/allpackages
> 12:32:32 URL:http://packages.debian.org/stable/allpackages 
> [3585331/3585331] -> "allpackages" [1]
> $ grep '^<dt>' allpackages | wc -l
> 22753
> 
> For unstable, the number is even higher:
> 
> $ rm allpackages
> $ wget -nv http://packages.debian.org/unstable/allpackages
> 12:34:12 URL:http://packages.debian.org/unstable/allpackages 
> [4924226/4924226] -> "allpackages" [1]
> $ grep '^<dt>' allpackages | wc -l
> 27602

That is not really fair, we do not have foo-qt3, foo-qt4,
foo-mysql, foo-psql .. We have "foo" with useflags.

So either remove the -dev, -qt3, -qt4, virtual packages, [..] from that
list, or calculate the number of use-flag permutations possible for our
packages.

I guess that gentoo beats debian :)

Cheers, Emil

-- 
Emil Beinroth
83059 Kolbermoor | Germany
 
BUFFERS=20 FILES=15 2nd down, 4th quarter, 5 yards to go!

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-user] Skipping static libraries

[Etaoin Shrdlu]

On Sunday 16 March 2008, 13:22, Emil Beinroth wrote:

> > 27602
>
> That is not really fair, we do not have foo-qt3, foo-qt4,
> foo-mysql, foo-psql .. We have "foo" with useflags.
>
> So either remove the -dev, -qt3, -qt4, virtual packages, [..] from
> that list, or calculate the number of use-flag permutations possible
> for our packages.

You are damn right. I overlooked the fact that a single gentoo ebuild, 
depending on USE flags, can provide and install the equivalent of many 
debian packages.
So, calculating a number using automated tools becomes quite difficult, 
since each package needs to be examined and compared with the 
corresponding offering in the other distro.
A (very) rough estimate could be probably done by unifying each foo-* 
Debian packages into a single entry, which results in something like

$ grep '^<dt>' allpackages | \
sed 's/.*<a href[^>]*>\([^<]*\)<\/a>.*/\1/g' | \
cut -d '-' -f 1 | \
uniq | wc -l
12225

Comments (also about the methodology used to calculate this) are left as 
an exercise for the reader :-)

> I guess that gentoo beats debian :)

I hope so too!
-- 
gentoo-user@lists.gentoo.org mailing list

Re: [gentoo-user] Skipping static libraries

[Gustavo Campos]

>  $ grep '^<dt>' allpackages | \
>  sed 's/.*<a href[^>]*>\([^<]*\)<\/a>.*/\1/g' | \
>  cut -d '-' -f 1 | \
>  uniq | wc -l
>  12225

No matter how much I live and code, regular expressions always scare
me like hell


>
>  Comments (also about the methodology used to calculate this) are left as
>  an exercise for the reader :-)
>
>
>  > I guess that gentoo beats debian :)
>
>  I hope so too!
>
>
> --
>  gentoo-user@lists.gentoo.org mailing list
>
>



-- 
Gustavo Campos

Ciência da Computação / Computer Science - UFMG
--
gentoo-user@lists.gentoo.org mailing list