[gentoo-dev] The future of elibtoolize

[gentoo-dev] The future of elibtoolize

[Michał Górny]

[-- Attachment #1: Type: text/plain, Size: 1506 bytes --]

Hi, everyone.

Quoting the top of libtool.eclass:

# @DESCRIPTION:
# This eclass patches ltmain.sh distributed with libtoolized packages with the
# relink and portage patch among others
#
# Note, this eclass does not require libtool as it only applies patches to
# generated libtool files.  We do not run the libtoolize program because that
# requires a regeneration of the main autotool files in order to work properly.

However, it seems that over time the eclass has grown huge to apply
a lot of random patches not only to libtool but also to generated
configure scripts etc.

It looks like the original intent is that the eclass would be used for
all autotools-based builds. It's called directly by some packages, more
frequently indirectly called by various eclasses (gnome2, xfconf...)
and also at the end of eautoreconf.

That said, I don't find the current solution really optimal. A lot of
ebuilds (mine, for example) are not using elibtoolize, and I expect
that they may randomly fail for some people in corner cases. But I
don't feel like adding another eclass to all ebuilds in the tree is
a good idea.

Portage already does some configure updates in econf. How about we move
the whole thing straight into Portage, implicitly activated by econf?
That would certainly increase coverage, remove some QA violations from
ECLASSDIR and possibly solve the problem long-term.

What do you think?

-- 
Best regards,
Michał Górny
<http://dev.gentoo.org/~mgorny/>

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 931 bytes --]

Re: [gentoo-dev] The future of elibtoolize

[James Le Cuirot]

On Tue, 20 Sep 2016 09:15:50 +0200
Michał Górny <mgorny@gentoo.org> wrote:

> That said, I don't find the current solution really optimal. A lot of
> ebuilds (mine, for example) are not using elibtoolize, and I expect
> that they may randomly fail for some people in corner cases. But I
> don't feel like adding another eclass to all ebuilds in the tree is
> a good idea.
> 
> Portage already does some configure updates in econf. How about we
> move the whole thing straight into Portage, implicitly activated by
> econf? That would certainly increase coverage, remove some QA
> violations from ECLASSDIR and possibly solve the problem long-term.
> 
> What do you think?

I support this. I don't know if it's as big a problem as it was when I
last looked at it but cross-compiling often failed without the sysroot
patch. Much like you, before becoming a dev, I did not want to file a
whole string of bug reports requesting that elibtoolize be added to
loads of ebuilds.

-- 
James Le Cuirot (chewi)
Gentoo Linux Developer

Re: [gentoo-dev] The future of elibtoolize

[Alexis Ballier]

On Tue, 20 Sep 2016 13:58:32 +0100
James Le Cuirot <chewi@gentoo.org> wrote:

> On Tue, 20 Sep 2016 09:15:50 +0200
> Michał Górny <mgorny@gentoo.org> wrote:
> 
> > That said, I don't find the current solution really optimal. A lot
> > of ebuilds (mine, for example) are not using elibtoolize, and I
> > expect that they may randomly fail for some people in corner cases.
> > But I don't feel like adding another eclass to all ebuilds in the
> > tree is a good idea.
> > 
> > Portage already does some configure updates in econf. How about we
> > move the whole thing straight into Portage, implicitly activated by
> > econf? That would certainly increase coverage, remove some QA
> > violations from ECLASSDIR and possibly solve the problem long-term.
> > 
> > What do you think?  
> 
> I support this. I don't know if it's as big a problem as it was when I
> last looked at it but cross-compiling often failed without the sysroot
> patch. Much like you, before becoming a dev, I did not want to file a
> whole string of bug reports requesting that elibtoolize be added to
> loads of ebuilds.
> 


there is a simple solution to this: profile.bashrc :)



i think there was a project/idea to do this in a cleaner way, supported
by diego, like 10 years ago, but i cant remember more

Re: [gentoo-dev] The future of elibtoolize

[James Le Cuirot]

On Tue, 20 Sep 2016 17:13:50 +0200
Alexis Ballier <aballier@gentoo.org> wrote:

> On Tue, 20 Sep 2016 13:58:32 +0100
> James Le Cuirot <chewi@gentoo.org> wrote:
> 
> > On Tue, 20 Sep 2016 09:15:50 +0200
> > Michał Górny <mgorny@gentoo.org> wrote:
> >   
> > > That said, I don't find the current solution really optimal. A lot
> > > of ebuilds (mine, for example) are not using elibtoolize, and I
> > > expect that they may randomly fail for some people in corner
> > > cases. But I don't feel like adding another eclass to all ebuilds
> > > in the tree is a good idea.
> > > 
> > > Portage already does some configure updates in econf. How about we
> > > move the whole thing straight into Portage, implicitly activated
> > > by econf? That would certainly increase coverage, remove some QA
> > > violations from ECLASSDIR and possibly solve the problem
> > > long-term.
> > > 
> > > What do you think?    
> > 
> > I support this. I don't know if it's as big a problem as it was
> > when I last looked at it but cross-compiling often failed without
> > the sysroot patch. Much like you, before becoming a dev, I did not
> > want to file a whole string of bug reports requesting that
> > elibtoolize be added to loads of ebuilds.
> >   
> 
> 
> there is a simple solution to this: profile.bashrc :)

Indeed, I did some godawful things with bashrc that make my own eyes
bleed but I stopped short of adding elibtoolize. It might work but if
it would work that reliably, why not make it standard?

-- 
James Le Cuirot (chewi)
Gentoo Linux Developer

Re: [gentoo-dev] The future of elibtoolize

[Alexis Ballier]

On Tue, 20 Sep 2016 16:21:36 +0100
James Le Cuirot <chewi@gentoo.org> wrote:

> On Tue, 20 Sep 2016 17:13:50 +0200
> Alexis Ballier <aballier@gentoo.org> wrote:
> 
> > On Tue, 20 Sep 2016 13:58:32 +0100
> > James Le Cuirot <chewi@gentoo.org> wrote:
> >   
> > > On Tue, 20 Sep 2016 09:15:50 +0200
> > > Michał Górny <mgorny@gentoo.org> wrote:
> > >     
> > > > That said, I don't find the current solution really optimal. A
> > > > lot of ebuilds (mine, for example) are not using elibtoolize,
> > > > and I expect that they may randomly fail for some people in
> > > > corner cases. But I don't feel like adding another eclass to
> > > > all ebuilds in the tree is a good idea.
> > > > 
> > > > Portage already does some configure updates in econf. How about
> > > > we move the whole thing straight into Portage, implicitly
> > > > activated by econf? That would certainly increase coverage,
> > > > remove some QA violations from ECLASSDIR and possibly solve the
> > > > problem long-term.
> > > > 
> > > > What do you think?      
> > > 
> > > I support this. I don't know if it's as big a problem as it was
> > > when I last looked at it but cross-compiling often failed without
> > > the sysroot patch. Much like you, before becoming a dev, I did not
> > > want to file a whole string of bug reports requesting that
> > > elibtoolize be added to loads of ebuilds.
> > >     
> > 
> > 
> > there is a simple solution to this: profile.bashrc :)  
> 
> Indeed, I did some godawful things with bashrc that make my own eyes
> bleed but I stopped short of adding elibtoolize. It might work but if
> it would work that reliably, why not make it standard?
> 

yes it should; not sure why previous attempts aborted

Re: [gentoo-dev] The future of elibtoolize

[James Le Cuirot]

[-- Attachment #1: Type: text/plain, Size: 2297 bytes --]

> On Tue, 20 Sep 2016 15:32:05
> > Alexis Ballier <aballier@g.o> wrote:
> >
> On Tue, 20 Sep 2016 16:21:36 +0100
> James Le Cuirot <chewi@g.o> wrote:
> 
> > On Tue, 20 Sep 2016 17:13:50 +0200
> > Alexis Ballier <aballier@g.o> wrote:
> >
> > > On Tue, 20 Sep 2016 13:58:32 +0100
> > > James Le Cuirot <chewi@g.o> wrote:
> > >
> > > > On Tue, 20 Sep 2016 09:15:50 +0200
> > > > Michał Górny <mgorny@g.o> wrote:
> > > >
> > > > > That said, I don't find the current solution really optimal. A
> > > > > lot of ebuilds (mine, for example) are not using elibtoolize,
> > > > > and I expect that they may randomly fail for some people in
> > > > > corner cases. But I don't feel like adding another eclass to
> > > > > all ebuilds in the tree is a good idea.
> > > > >
> > > > > Portage already does some configure updates in econf. How
> > > > > about we move the whole thing straight into Portage,
> > > > > implicitly activated by econf? That would certainly increase
> > > > > coverage, remove some QA violations from ECLASSDIR and
> > > > > possibly solve the problem long-term.
> > > > >
> > > > > What do you think?
> > > >
> > > > I support this. I don't know if it's as big a problem as it was
> > > > when I last looked at it but cross-compiling often failed
> > > > without the sysroot patch. Much like you, before becoming a
> > > > dev, I did not want to file a whole string of bug reports
> > > > requesting that elibtoolize be added to loads of ebuilds.
> > > >
> > >
> > >
> > > there is a simple solution to this: profile.bashrc :)
> >
> > Indeed, I did some godawful things with bashrc that make my own eyes
> > bleed but I stopped short of adding elibtoolize. It might work but
> > if it would work that reliably, why not make it standard?
> >
> 
> yes it should; not sure why previous attempts aborted

I've just started cross-compiling again for the first time in about two
years. Now I remember why I couldn't rely on bashrc for this.
elibtoolize comes from the libtool eclass and you can't inherit
additional eclasses from bashrc. I've already been bitten by this issue
on several ebuilds today such as dev-lang/orc-0.4.26-r1. What can I do
to help drive this forwards?

-- 
James Le Cuirot (chewi)
Gentoo Linux Developer

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 949 bytes --]

Re: [gentoo-dev] The future of elibtoolize

[Alexis Ballier]

On Sun, 25 Sep 2016 11:05:27 +0100
James Le Cuirot <chewi@gentoo.org> wrote:

> I've just started cross-compiling again for the first time in about
> two years. Now I remember why I couldn't rely on bashrc for this.
> elibtoolize comes from the libtool eclass and you can't inherit
> additional eclasses from bashrc.

Well, technically you can but it becomes really hackish :)


I doubt including elibtoolize in portage/econf/pms is a viable
solution: there is way too much code, N patches times M libtool
versions to handle, and in rare cases one needs to call elibtoolize
with some optional arguments. Specing that would be a pain for no real
gain.


On the other hand, we could go the complete opposite direction of what
has been done in the past years with PMS: provide a generic way to
extend ebuild env from profiles, with the ability to "include"
some "eclasses", define default phases and pre/post phases hooks. This
would have, e.g., saved the need to completely rewrite and spec epatch,
avoided every PM to copy/paste default phases implementations from PMS,
etc. However, this has somewhat the same disadvantage than eclasses:
one commits crap there and breaks every system in subtle ways...

Re: [gentoo-dev] The future of elibtoolize

[James Le Cuirot]

[-- Attachment #1: Type: text/plain, Size: 1574 bytes --]

On Mon, 26 Sep 2016 17:53:43 +0200
Alexis Ballier <aballier@gentoo.org> wrote:

> I doubt including elibtoolize in portage/econf/pms is a viable
> solution: there is way too much code, N patches times M libtool
> versions to handle, and in rare cases one needs to call elibtoolize
> with some optional arguments. Specing that would be a pain for no real
> gain.

Having just looked at the code a bit closer, I see what you mean but…

> On the other hand, we could go the complete opposite direction of what
> has been done in the past years with PMS: provide a generic way to
> extend ebuild env from profiles, with the ability to "include"
> some "eclasses", define default phases and pre/post phases hooks. This
> would have, e.g., saved the need to completely rewrite and spec
> epatch, avoided every PM to copy/paste default phases implementations
> from PMS, etc. However, this has somewhat the same disadvantage than
> eclasses: one commits crap there and breaks every system in subtle
> ways...

I don't think I want to open this can of worms either. :(

You said that flameeyes raised this about 10 years ago. It has indeed
been 10 years!

https://archives.gentoo.org/gentoo-dev/message/caa153de0d23dc264330f5e702f26e58

The solution he preferred back then was to split elibtoolize into its
own package and have Portage depend on it. I hadn't considered that and
I quite like it too. There was only one brief reply to the thread back
then. Can you think of any downsides now?

-- 
James Le Cuirot (chewi)
Gentoo Linux Developer

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 949 bytes --]

Re: [gentoo-dev] The future of elibtoolize

[Alexis Ballier]

On Mon, 26 Sep 2016 21:35:29 +0100
James Le Cuirot <chewi@gentoo.org> wrote:
[...]
> > On the other hand, we could go the complete opposite direction of
> > what has been done in the past years with PMS: provide a generic
> > way to extend ebuild env from profiles, with the ability to
> > "include" some "eclasses", define default phases and pre/post
> > phases hooks. This would have, e.g., saved the need to completely
> > rewrite and spec epatch, avoided every PM to copy/paste default
> > phases implementations from PMS, etc. However, this has somewhat
> > the same disadvantage than eclasses: one commits crap there and
> > breaks every system in subtle ways...  
> 
> I don't think I want to open this can of worms either. :(
> 
> You said that flameeyes raised this about 10 years ago. It has indeed
> been 10 years!
> 
> https://archives.gentoo.org/gentoo-dev/message/caa153de0d23dc264330f5e702f26e58
> 
> The solution he preferred back then was to split elibtoolize into its
> own package and have Portage depend on it. I hadn't considered that
> and I quite like it too. There was only one brief reply to the thread
> back then. Can you think of any downsides now?


Well, I don't see any fundamental difference in specing 'call this
utility' vs. proper profile.bashrc. If you don't want specing, then
indeed an utility is the way to go, but this could imply some packages
build with portage because it elibtoolizes them and fail with PMs that
don't.

Also, keep in mind that with an external utility you have far less
control on what is executed than with something in $PORTDIR: people may
use an older buggy version of the utility, while when shipping it in
$PORTDIR you are sure that the version is up to date.

Re: [gentoo-dev] The future of elibtoolize

[Kent Fredric]

[-- Attachment #1: Type: text/plain, Size: 539 bytes --]

On Tue, 27 Sep 2016 11:24:19 +0200
Alexis Ballier <aballier@gentoo.org> wrote:

> Also, keep in mind that with an external utility you have far less
> control on what is executed than with something in $PORTDIR: people may
> use an older buggy version of the utility, while when shipping it in
> $PORTDIR you are sure that the version is up to date.

Conversely ... you can't force people to use a specific version of portage.

But you can force a dependency on a newer version of the external utility
if the package needs to.

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 801 bytes --]

Re: [gentoo-dev] The future of elibtoolize

[Alexis Ballier]

On Tue, 27 Sep 2016 22:52:08 +1300
Kent Fredric <kentnl@gentoo.org> wrote:

> On Tue, 27 Sep 2016 11:24:19 +0200
> Alexis Ballier <aballier@gentoo.org> wrote:
> 
> > Also, keep in mind that with an external utility you have far less
> > control on what is executed than with something in $PORTDIR: people
> > may use an older buggy version of the utility, while when shipping
> > it in $PORTDIR you are sure that the version is up to date.  
> 
> Conversely ... you can't force people to use a specific version of
> portage.
> 
> But you can force a dependency on a newer version of the external
> utility if the package needs to.


which kind of defeats the point of having it done automagically without
touching ebuilds :)

Re: [gentoo-dev] The future of elibtoolize

[James Le Cuirot]

[-- Attachment #1: Type: text/plain, Size: 1941 bytes --]

On Tue, 27 Sep 2016 11:24:19 +0200
Alexis Ballier <aballier@gentoo.org> wrote:

> > You said that flameeyes raised this about 10 years ago. It has
> > indeed been 10 years!
> > 
> > https://archives.gentoo.org/gentoo-dev/message/caa153de0d23dc264330f5e702f26e58
> > 
> > The solution he preferred back then was to split elibtoolize into
> > its own package and have Portage depend on it. I hadn't considered
> > that and I quite like it too. There was only one brief reply to the
> > thread back then. Can you think of any downsides now?  
> 
> 
> Well, I don't see any fundamental difference in specing 'call this
> utility' vs. proper profile.bashrc. If you don't want specing, then
> indeed an utility is the way to go, but this could imply some packages
> build with portage because it elibtoolizes them and fail with PMs that
> don't.

So we mandate it in PMS.

> Also, keep in mind that with an external utility you have far less
> control on what is executed than with something in $PORTDIR: people
> may use an older buggy version of the utility, while when shipping it
> in $PORTDIR you are sure that the version is up to date.

I was going to say what kent\n said, that Portage itself can just as
easily be outdated. He also makes a good point about depending on a
minimum version when necessary. This shouldn't be needed often. Ideally
Portage would keep pulling in a recent version anyway.

I went ahead and converted libtool.eclass into an external script with
very few changes to start with, just as a proof of concept. I removed
the few references to other eclass helpers but still retained a little
dependence on variables exported by Portage. I then stuck a call to
this to near the top of econf() and tried out some packages, including
those that had failed on me before. Well whaddya know, it works. I
guess I should continue?

-- 
James Le Cuirot (chewi)
Gentoo Linux Developer

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 949 bytes --]

Re: [gentoo-dev] The future of elibtoolize

[Dan Douglas]

On Sun, Sep 25, 2016 at 5:05 AM, James Le Cuirot <chewi@gentoo.org> wrote:
> Now I remember why I couldn't rely on bashrc for this.
> elibtoolize comes from the libtool eclass and you can't inherit
> additional eclasses from bashrc.


This is what I'm dealing with all the time. It would be easier if
eclasses weren't the solution for all library code. The main feature
of an eclass was supposed to be EXPORT_FUNCTIONS for extending phase
functions, and many eclasses don't even make use of that.