[gentoo-dev] RFC: Installation of static libraries, USE=static-libs proposal

[gentoo-dev] RFC: Installation of static libraries, USE=static-libs proposal

[Mart Raudsepp]

[-- Attachment #1: Type: text/plain, Size: 3306 bytes --]

Hello again,


Over a year or two ago, it was communicated that it supposedly a policy
that USE=static should only control if a package installs static
libraries INSTEAD of shared libraries, and never to be used to control
if static libraries are installed in _addition_ to shared ones or not.

Packages were coerced to stop using USE=static for controlling that, and
most of them ended up unconditionally installing both static and shared
libraries. What's worse - they were told that if a package can provide
both shared libraries and static libraries at once, it just MUST (or
SHOULD) install them both instead of choosing to not ship the static
libraries.
End result that affects me: GNOME does not fit on LiveCD installation
media anymore.


So I'm proposing a USE=static-libs or similar to get out of this
problem, and a lifting of the supposed (I wasn't around as a dev that
long ago to know for sure) policy of having to install both instead of
choosing to never install static libraries.

I am quite sure that absolutely nothing whatsoever uses about 97% of the
static GNOME libraries we are now installing as an end result. How can I
be sure? Because everything worked just fine when static libraries
weren't installed in the past thanks to that not happening from
USE=static missing on most systems for years, and you'd have to be quite
crazy if you required static version of desktop libraries with well
established and followed ABI guarantees.

Those that aren't absolutely sure that there is no use case for static
libraries, could then use a USE=static-libs or similar to not
unnecessarily install static libraries that are not going to ever be
used. And LiveCD media could avoid all these static libraries, that it
currently has to ship just because the packages by default install that
cruft for no real technical reason (and it has to follow that due to
GRPs).
I would use USE=static-libs on packages like gtkmm, that have good
reasons to provide a static version - it allows Gentoo users that would
like to do commercial or universal C++ based development against Gentoo
system packages, as it avoids feared libstdc++ ABI breaks (there's even
a request for it in bugzilla).



There are packages in the tree that are required to install static
libraries, or something else in the system breaks. So INSTALL_MASK=*.a
is not a solution in my eyes.


Useful comments, thoughts, agreements?



I have had some additional ideas for handling static libraries better at
a package manager level, but those still need to be fleshed out and put
into writing.
Things like possibility to rebuild all packages that link to a static
library that was now upgraded, so the higher level package could use a
relinking to benefit from the bug fixes from the new library; optional
ability to install only the type of library currently needed - shared or
static, etc. Much of it goes to blue-sky ideas though with minimal
benefit for normal desktop/server systems and potentially high
maintenance effort, so I haven't put much effort into putting it into
writing. But maybe someone interested wants to chat on IRC on the topic.


-- 
Mart Raudsepp
Gentoo Developer
Mail: leio@gentoo.org
Weblog: http://planet.gentoo.org/developers/leio

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 197 bytes --]

[gentoo-dev] Re: RFC: Installation of static libraries, USE=static-libs proposal

[Duncan]

Mart Raudsepp <leio@gentoo.org> posted
1214877951.29224.54.camel@localhost, excerpted below, on  Tue, 01 Jul 2008
05:05:51 +0300:

> Over a year or two ago, it was communicated that it supposedly a policy
> that USE=static should only control if a package installs static
> libraries INSTEAD of shared libraries, and never to be used to control
> if static libraries are installed in _addition_ to shared ones or not.

That's as I understand it too.

> Packages were coerced to stop using USE=static for controlling that, and
> most of them ended up unconditionally installing both static and shared
> libraries. What's worse - they were told that if a package can provide
> both shared libraries and static libraries at once, it just MUST (or
> SHOULD) install them both instead of choosing to not ship the static
> libraries.

OK, but see below.

> End result that affects me: GNOME does not fit on LiveCD installation
> media anymore.

Ouch!

> So I'm proposing a USE=static-libs or similar to get out of this
> problem, and a lifting of the supposed (I wasn't around as a dev that
> long ago to know for sure) policy of having to install both instead of
> choosing to never install static libraries.

I'm not sure this is warranted.  See below.

> I am quite sure that absolutely nothing whatsoever uses about 97% of the
> static GNOME libraries we are now installing as an end result. [...]

Probably others than GNOME, too.

> There are packages in the tree that are required to install static
> libraries, or something else in the system breaks. So INSTALL_MASK=*.a
> is not a solution in my eyes.

This is the ticklish bit, but there's still a way around it for users 
(such as those trying to fit GNOME on a liveCD) that need it.  Useing 
portage's bashrc, setup a conditional that excepts packages that need 
static libs and set INSTALL_MASK='*.a' for everything else.

If you've not yet seen Ed Catmur's bashrc script setup, "I+'5 d4 60mb!"  
I don't personally find many of its capabilities useful, but his auto-
patching setup has sure come in handy, and the entire thing is just 
incredibly extensible on a nicely solid base. =8^)  It should make 
setting this up a breeze, and be handily expansible for 'most anything 
else you might come up with as well.  If it were me, I'd use that as a 
base and go from there utilizing the idea I suggested above.

http://catmur.co.uk/gentoo/

Additionally, you don't mention whether you checked with them already or 
not, but releng and subprojects may have some suggestions in this area.

-- 
Duncan - List replies preferred.   No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master."  Richard Stallman

-- 
gentoo-dev@lists.gentoo.org mailing list

Re: [gentoo-dev] Re: RFC: Installation of static libraries, USE=static-libs proposal

[Rémi Cardona]

Duncan a écrit :
> Probably others than GNOME, too.

Thus Mart's effort to bring it to gentoo-dev :)

> This is the ticklish bit, but there's still a way around it for users 
> (such as those trying to fit GNOME on a liveCD) that need it.  Useing 
> portage's bashrc, setup a conditional that excepts packages that need 
> static libs and set INSTALL_MASK='*.a' for everything else.

No, it was pointed out that this cannot be done for LiveCD material, as 
the packages would have a different content as a regular install. So 
this is just out of the question.

For those wondering : "find /usr/lib64 -name "*.a" | xargs du -ch" will 
tell you how much disk space is wasted by static libraries.

On my Gnome box, this is 246M. I know we won't be able to bring this to 
0, but having it closer to 10~20M is a very worthy goal imho.

Cheers,

Rémi
-- 
gentoo-dev@lists.gentoo.org mailing list

Re: [gentoo-dev] Re: RFC: Installation of static libraries, USE=static-libs proposal

[Mart Raudsepp]

[-- Attachment #1: Type: text/plain, Size: 1541 bytes --]

On T, 2008-07-01 at 11:33 +0200, Rémi Cardona wrote:
> Duncan a écrit :
> > Probably others than GNOME, too.
> 
> Thus Mart's effort to bring it to gentoo-dev :)

And for constructive discussing of it, including with releng and other
teams.

> > This is the ticklish bit, but there's still a way around it for users 
> > (such as those trying to fit GNOME on a liveCD) that need it.  Useing 
> > portage's bashrc, setup a conditional that excepts packages that need 
> > static libs and set INSTALL_MASK='*.a' for everything else.
> 
> No, it was pointed out that this cannot be done for LiveCD material, as 
> the packages would have a different content as a regular install. So 
> this is just out of the question.
> 
> For those wondering : "find /usr/lib64 -name "*.a" | xargs du -ch" will 
> tell you how much disk space is wasted by static libraries.
> 
> On my Gnome box, this is 246M. I know we won't be able to bring this to 
> 0, but having it closer to 10~20M is a very worthy goal imho.

In addition I'm looking for a clean solution, not every Gentoo user
having to have a INSTALL_MASK set with a few exceptions that they don't
know; and if they don't know what are the exceptions, they'll have
trivial problems like bash not working, iirc.

Btw, just to be clear, I'm not claiming this is the sole reason GNOME
doesn't fit on LiveCD's anymore, but it is a big contributor for that.


-- 
Mart Raudsepp
Gentoo Developer
Mail: leio@gentoo.org
Weblog: http://planet.gentoo.org/developers/leio

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 197 bytes --]

Re: [gentoo-dev] Re: RFC: Installation of static libraries, USE=static-libs proposal

[Enrico Weigelt]

Hi,

<snip>

I'm really curious to know why a new (global) useflag couldn't
do the trick.

Let's say, we introduce a new useflag called "static-libs" and
enable it by defaulin all profiles. Then we can have a look at
the lib packages step by step and add support when it seems 
useful there and test out carefully. There's no presure to this
for the whole distro at once.


cu
-- 
---------------------------------------------------------------------
 Enrico Weigelt    ==   metux IT service - http://www.metux.de/
---------------------------------------------------------------------
 Please visit the OpenSource QM Taskforce:
 	http://wiki.metux.de/public/OpenSource_QM_Taskforce
 Patches / Fixes for a lot dozens of packages in dozens of versions:
	http://patches.metux.de/
---------------------------------------------------------------------
-- 
gentoo-dev@lists.gentoo.org mailing list

Re: [gentoo-dev] Re: RFC: Installation of static libraries, USE=static-libs proposal

[Rémi Cardona]

Enrico Weigelt wrote:
> Hi,
>
> <snip>
>
> I'm really curious to know why a new (global) useflag couldn't
> do the trick.

Read Mart's mail again, that's exactly what he's proposing.

And if no-one objects, I'll be working this inside the gnome2 eclass 
(with review on this list before the final commit of course) to see how 
things should be handled.

Cheers

Rémi
-- 
gentoo-dev@lists.gentoo.org mailing list

Re: [gentoo-dev] RFC: Installation of static libraries, USE=static-libs proposal

[Peter Volkov]

[-- Attachment #1: Type: text/plain, Size: 1478 bytes --]

В Втр, 01/07/2008 в 05:05 +0300, Mart Raudsepp пишет:
> Over a year or two ago, it was communicated that it supposedly a policy
> that USE=static

Well, I don't have web-reference at hand now, but there was a thread in
gentoo-dev with the subject: "Say no to static libraries!". Summarizing
some ideas from there:

1. Some packages will break if you build their deps with USE=static.
This can be fixed when we start to use USE-deps in the tree.

2. We already have mechanism to make what you want. Just drop
EXTRA_ECONF="--disable-static" into your make.conf and to workaround
problem stated in point 1 use

EXTRA_ECONF="${EXTRA_ECONF/--disable-static}" 

in /etc/portage/env/cat/pkg. (For those who interested  list of packages
for which I have to filter --disable-static is in attachment).


Well, I'm using EXTRA_ECONF for more then year now and I'd like to say
that it's not perfect solution. Not all packages are autotools based and
ignore --disable-static and now I have 103M of static libs on my
desktop. So now I'm all for having static-libs USE flag. But please,
don't do that on per-package base. Make an eclass for that. Think about
not-autotools packages, and don't put it in the tree until we start
using USE deps.


Thanks for reiterating this discussion. I wanted to return to it soon as
seems that USE deps are really about to enter our life.

And BTW, seems that all gnome packages obey EXTRA_ECONF ;)

-- 
Peter.

[-- Attachment #2: disable-static-broken-packages.list --]
[-- Type: text/plain, Size: 216 bytes --]

dev-libs/popt
dev-libs/lzo
dev-libs/libpcre
dev-libs/xmlrpc-c
dev-libs/libol
media-libs/pdflib
media-sound/audacity
sys-devel/gdb
sys-devel/libtool
sys-apps/ed
sys-apps/ed
sys-fs/fuse
dev-ruby/rcairo
dev-ruby/rcairo

Re: [gentoo-dev] RFC: Installation of static libraries, USE=static-libs proposal

[Mart Raudsepp]

[-- Attachment #1: Type: text/plain, Size: 3229 bytes --]

On Sun, 2008-07-20 at 15:40 +0400, Peter Volkov wrote:
> В Втр, 01/07/2008 в 05:05 +0300, Mart Raudsepp пишет:
> > Over a year or two ago, it was communicated that it supposedly a policy
> > that USE=static
> 
> Well, I don't have web-reference at hand now, but there was a thread in
> gentoo-dev with the subject: "Say no to static libraries!". Summarizing
> some ideas from there:
> 
> 1. Some packages will break if you build their deps with USE=static.
> This can be fixed when we start to use USE-deps in the tree.
> 
> 2. We already have mechanism to make what you want. Just drop
> EXTRA_ECONF="--disable-static" into your make.conf and to workaround
> problem stated in point 1 use
> 
> EXTRA_ECONF="${EXTRA_ECONF/--disable-static}" 
> 
> in /etc/portage/env/cat/pkg. (For those who interested  list of packages
> for which I have to filter --disable-static is in attachment).

I'm going to simply drop the remaining USE=static's where appropriate,
if upstreams choice is to disable static libraries, and we'll see even
later what to do about those that don't default to not building static
libraries. For now I stumbled on just gtk-engines.

That is, some GNOME packages make it their own business to override the
automake default of building both - they call AM_DISABLE_STATIC.
So for those I'll just drop any --{enable,disable}-static (as I notice)
and honor upstreams choice in not building static libraries. This also
means nothing uses or needs the static libraries because all GNOME
tinderboxes and jhbuild based development machines would scream in agony
if it broke anything.

> Well, I'm using EXTRA_ECONF for more then year now and I'd like to say
> that it's not perfect solution. Not all packages are autotools based and
> ignore --disable-static and now I have 103M of static libs on my
> desktop. So now I'm all for having static-libs USE flag. But please,
> don't do that on per-package base. Make an eclass for that. Think about
> not-autotools packages, and don't put it in the tree until we start
> using USE deps.

I know of absolutely no GNOME package that needs its static libraries
installed. Only exception is glib, but that is a lower level library,
not a GNOME one - there we explicitly enable it for syslog-ng possible
use primarily. And for glib I'm quite cool in enabling it always, as my
take on glib is that it's a standard C library that makes C actually
usable and powerful, just as libstdc++ makes C++ more powerful, and
should be universally available to all. Though an alternative would be
to install it in /lib, so that boot tools can use it and not need to
link statically - having syslog-ng in mind primarily.

> Thanks for reiterating this discussion. I wanted to return to it soon as
> seems that USE deps are really about to enter our life.

And now I'm reiterating it again the first time since 3 months.

> And BTW, seems that all gnome packages obey EXTRA_ECONF ;)

And probably G2CONF too, but we like to use that for ebuild use only -
we aren't sure we have G2CONF="${G2CONF} foo" everywhere, etc.


-- 
Mart Raudsepp
Gentoo Developer
Mail: leio@gentoo.org
Weblog: http://planet.gentoo.org/developers/leio

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 197 bytes --]