[gentoo-dev] RFC: language bindings as separate packages

[gentoo-dev] RFC: language bindings as separate packages

[Enrico Weigelt]

Hi folks,


while building yum, I again ran into trouble because one 
dependency has to be rebuilt with an specific useflag first 
(in this case it was libxml2 + python useflag). Actually, 
there are *lots* of these cases and (AFAIK) portage has no 
way for properly handling this - it's up to the individual 
ebuilds to check for those situations and artifically breaking
the build. Of coure, breaking builds are ugly.

My suggestion: make those language bindings being separate
packages. So, other packages can depend on them directly, 
instead of the current, build-breaking hack.

I'm not advocating gentoo should do this step alone, but 
instead join in the upstream and solve it there.


cu
-- 
---------------------------------------------------------------------
 Enrico Weigelt    ==   metux IT service - http://www.metux.de/
---------------------------------------------------------------------
 Please visit the OpenSource QM Taskforce:
 	http://wiki.metux.de/public/OpenSource_QM_Taskforce
 Patches / Fixes for a lot dozens of packages in dozens of versions:
	http://patches.metux.de/
---------------------------------------------------------------------
-- 
gentoo-dev@lists.gentoo.org mailing list

Re: [gentoo-dev] RFC: language bindings as separate packages

[Łukasz P. Michalik]

Ever heard of use dependencies?  Way you are proposing this is just
more work and no solutions to real problems.

On Thu, May 1, 2008 at 5:09 PM, Enrico Weigelt <weigelt@metux.de> wrote:
>
> Hi folks,
>
>
> while building yum, I again ran into trouble because one
> dependency has to be rebuilt with an specific useflag first
> (in this case it was libxml2 + python useflag). Actually,
> there are *lots* of these cases and (AFAIK) portage has no
> way for properly handling this - it's up to the individual
> ebuilds to check for those situations and artifically breaking
> the build. Of coure, breaking builds are ugly.
>
> My suggestion: make those language bindings being separate
> packages. So, other packages can depend on them directly,
> instead of the current, build-breaking hack.
>
> I'm not advocating gentoo should do this step alone, but
> instead join in the upstream and solve it there.
>
>
> cu


Thanks
ŁM

Re: [gentoo-dev] RFC: language bindings as separate packages

[Santiago M. Mola]

On Thu, May 1, 2008 at 5:09 PM, Enrico Weigelt <weigelt@metux.de> wrote:
>
>  Hi folks,
>
>  while building yum, I again ran into trouble because one
>  dependency has to be rebuilt with an specific useflag first
>  (in this case it was libxml2 + python useflag). Actually,
>  there are *lots* of these cases and (AFAIK) portage has no
>  way for properly handling this - it's up to the individual
>  ebuilds to check for those situations and artifically breaking
>  the build. Of coure, breaking builds are ugly.

The proper solution for these cases is implementing USE dependencies,
which would obsolete pkg_setup checks, and would provide portage with
info about which USE flags are needed for each dependency. This
feature is already implemented in other package managers (it's in
Paludis, maybe in pkgcore too?) and I think we all look forward its
inclusion in portage.

>  My suggestion: make those language bindings being separate
>  packages. So, other packages can depend on them directly,
>  instead of the current, build-breaking hack.
>
>  I'm not advocating gentoo should do this step alone, but
>  instead join in the upstream and solve it there.
>

Yes, sometimes it makes sense for upstream to split packages, anyone
is free to push them for doing so.

Regards,

-- 
Santiago M. Mola
Jabber ID: cooldwind@gmail.com
-- 
gentoo-dev@lists.gentoo.org mailing list

Re: [gentoo-dev] RFC: language bindings as separate packages

[Mart Raudsepp]

[-- Attachment #1: Type: text/plain, Size: 1940 bytes --]

On N, 2008-05-01 at 17:52 +0200, Santiago M. Mola wrote:
> On Thu, May 1, 2008 at 5:09 PM, Enrico Weigelt <weigelt@metux.de> wrote:
> >
> >  Hi folks,
> >
> >  while building yum, I again ran into trouble because one
> >  dependency has to be rebuilt with an specific useflag first
> >  (in this case it was libxml2 + python useflag). Actually,
> >  there are *lots* of these cases and (AFAIK) portage has no
> >  way for properly handling this - it's up to the individual
> >  ebuilds to check for those situations and artifically breaking
> >  the build. Of coure, breaking builds are ugly.
> 
> The proper solution for these cases is implementing USE dependencies,
> which would obsolete pkg_setup checks, and would provide portage with
> info about which USE flags are needed for each dependency. This
> feature is already implemented in other package managers (it's in
> Paludis, maybe in pkgcore too?) and I think we all look forward its
> inclusion in portage.

I do not see that as a solution, but instead maybe only a fix for the
fact that it's failing at some point in an emerge run instead of knowing
beforehand.
It still means a rebuild of the binding providing library, which
involves unnecessary recompilation of the (typically) C or C++ library,
which in some cases can be a huge time sink - that in the case that the
library in question isn't at that time pulled in (in that case the
package manager can enable it at first merge), but already installed
without the bindings USE flags.

So splitting packages is the perfect solution in my opinion, given
unlimited maintainer time and such. In real world that might be too hard
to maintain if upstream doesn't go along, but not in all cases (depends
on how much time the maintainer has, how complex the build system is,
etc).


-- 
Mart Raudsepp
Gentoo Developer
Mail: leio@gentoo.org
Weblog: http://planet.gentoo.org/developers/leio

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 197 bytes --]

Re: [gentoo-dev] RFC: language bindings as separate packages

[Ciaran McCreesh]

[-- Attachment #1: Type: text/plain, Size: 851 bytes --]

On Thu, 01 May 2008 19:05:35 +0300
Mart Raudsepp <leio@gentoo.org> wrote:
> It still means a rebuild of the binding providing library, which
> involves unnecessary recompilation of the (typically) C or C++
> library, which in some cases can be a huge time sink - that in the
> case that the library in question isn't at that time pulled in (in
> that case the package manager can enable it at first merge), but
> already installed without the bindings USE flags.

Not really. For packages that support it, it just means adding a way
for that package to be told to do a partial rebuild for the addition of
a use flag.

But that discussion can come after Portage gets use dependencies...
Which, as I understand it, won't be for at least another eighteen months
because three more people have just asked for them.

-- 
Ciaran McCreesh

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-dev] RFC: language bindings as separate packages

[Enrico Weigelt]

* Ciaran McCreesh <ciaran.mccreesh@googlemail.com> schrieb:

> Not really. For packages that support it, it just means adding a way
> for that package to be told to do a partial rebuild for the addition of
> a use flag.

Ugh, partial rebuilds are alway tricky and tend to make things
even more complicated and error-prone. See moz or oo. ;-P

> But that discussion can come after Portage gets use dependencies...
> Which, as I understand it, won't be for at least another eighteen months
> because three more people have just asked for them.

I'm a bit confused - does portage already support use-deps or
does not not ?


cu
-- 
---------------------------------------------------------------------
 Enrico Weigelt    ==   metux IT service - http://www.metux.de/
---------------------------------------------------------------------
 Please visit the OpenSource QM Taskforce:
 	http://wiki.metux.de/public/OpenSource_QM_Taskforce
 Patches / Fixes for a lot dozens of packages in dozens of versions:
	http://patches.metux.de/
---------------------------------------------------------------------
-- 
gentoo-dev@lists.gentoo.org mailing list

Re: [gentoo-dev] RFC: language bindings as separate packages

[Ciaran McCreesh]

[-- Attachment #1: Type: text/plain, Size: 471 bytes --]

On Thu, 1 May 2008 18:23:28 +0200
Enrico Weigelt <weigelt@metux.de> wrote:
> > But that discussion can come after Portage gets use dependencies...
> > Which, as I understand it, won't be for at least another eighteen
> > months because three more people have just asked for them.
> 
> I'm a bit confused - does portage already support use-deps or
> does not not ?

https://bugs.gentoo.org/show_bug.cgi?id=2272

Six years and counting.

-- 
Ciaran McCreesh

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-dev] RFC: language bindings as separate packages

[Enrico Weigelt]

* Ciaran McCreesh <ciaran.mccreesh@googlemail.com> schrieb:

> https://bugs.gentoo.org/show_bug.cgi?id=2272

According to this bug, portage still can't handle use-deps,
there's just a standard way for breaking the build on missing
useflags.

Exactly what I already suspected.


cu
-- 
---------------------------------------------------------------------
 Enrico Weigelt    ==   metux IT service - http://www.metux.de/
---------------------------------------------------------------------
 Please visit the OpenSource QM Taskforce:
 	http://wiki.metux.de/public/OpenSource_QM_Taskforce
 Patches / Fixes for a lot dozens of packages in dozens of versions:
	http://patches.metux.de/
---------------------------------------------------------------------
-- 
gentoo-dev@lists.gentoo.org mailing list

Re: [gentoo-dev] RFC: language bindings as separate packages

[Luca Barbato]

Enrico Weigelt wrote:
> My suggestion: make those language bindings being separate
> packages. So, other packages can depend on them directly, 
> instead of the current, build-breaking hack.
> 
> I'm not advocating gentoo should do this step alone, but 
> instead join in the upstream and solve it there.

The issue is upstream related, we can workaround it using a way to 
express that requirement (usedeps, checks in pkg_setup, whatever), 
obviously trying to cooperate with upstream in order to get the optional 
bindings build w/out the main program would make our life simpler and 
probably their as well.

Partial builds are quite a problem since they are anything but reliable.

lu

-- 

Luca Barbato
Gentoo Council Member
Gentoo/linux Gentoo/PPC
http://dev.gentoo.org/~lu_zero

-- 
gentoo-dev@lists.gentoo.org mailing list

Re: [gentoo-dev] RFC: language bindings as separate packages

[Enrico Weigelt]

* Luca Barbato <lu_zero@gentoo.org> schrieb:
> Enrico Weigelt wrote:
> >My suggestion: make those language bindings being separate
> >packages. So, other packages can depend on them directly, 
> >instead of the current, build-breaking hack.
> >
> >I'm not advocating gentoo should do this step alone, but 
> >instead join in the upstream and solve it there.
> 
> The issue is upstream related, we can workaround it using a way to 
> express that requirement (usedeps, checks in pkg_setup, whatever), 
> obviously trying to cooperate with upstream in order to get the optional 
> bindings build w/out the main program would make our life simpler and 
> probably their as well.
> 
> Partial builds are quite a problem since they are anything but reliable.

ACK. These are just hacks to work around upstream's design 
problems. For me, working much embedded environments, those
hacks are not an option, since builds MUST be reliable
(the packages MUST work IMMEDIATELY after deployment, since
there is no chance for doing things like revdep-rebuild).

My vote is: declaring guidelines (or better: constraints) for
clean builds and then working directly within the upstream to
get it on the road. If the upstream really blocks it, do a 
fork / maintain a patchline (like OSS-QM project does).

I'm already doing so with several packages.


cu
-- 
---------------------------------------------------------------------
 Enrico Weigelt    ==   metux IT service - http://www.metux.de/
---------------------------------------------------------------------
 Please visit the OpenSource QM Taskforce:
 	http://wiki.metux.de/public/OpenSource_QM_Taskforce
 Patches / Fixes for a lot dozens of packages in dozens of versions:
	http://patches.metux.de/
---------------------------------------------------------------------
-- 
gentoo-dev@lists.gentoo.org mailing list

Re: [gentoo-dev] RFC: language bindings as separate packages

[Rémi Cardona]

Enrico Weigelt a écrit :
> * Luca Barbato <lu_zero@gentoo.org> schrieb:
>> Enrico Weigelt wrote:
>>> My suggestion: make those language bindings being separate
>>> packages. So, other packages can depend on them directly, 
>>> instead of the current, build-breaking hack.
>>>
>>> I'm not advocating gentoo should do this step alone, but 
>>> instead join in the upstream and solve it there.
>> The issue is upstream related, we can workaround it using a way to 
>> express that requirement (usedeps, checks in pkg_setup, whatever), 
>> obviously trying to cooperate with upstream in order to get the optional 
>> bindings build w/out the main program would make our life simpler and 
>> probably their as well.
>>
>> Partial builds are quite a problem since they are anything but reliable.
> 
> ACK. These are just hacks to work around upstream's design 
> problems. For me, working much embedded environments, those
> hacks are not an option, since builds MUST be reliable
> (the packages MUST work IMMEDIATELY after deployment, since
> there is no chance for doing things like revdep-rebuild).
> 
> My vote is: declaring guidelines (or better: constraints) for
> clean builds and then working directly within the upstream to
> get it on the road.

Best example on how to do that is gstreamer. All the plugins come in 3 
tarballs but each can be built individually. Really clean.

> If the upstream really blocks it, do a 
> fork / maintain a patchline (like OSS-QM project does).
> 
> I'm already doing so with several packages.

I've seen you talk about that project before but I don't feel 
comfortable going down that road. We want to work with upstream and let 
them know what our needs are. Maintaining patches is a lot of work and 
forking is even more work. Even though I'm still a relatively young 
Gentoo dev (only been here for 1.5 years), I have yet to see upstream 
projects reject build patches that make our lives easier.

Cheers,

Rémi
-- 
gentoo-dev@lists.gentoo.org mailing list

Re: [gentoo-dev] RFC: language bindings as separate packages

[Enrico Weigelt]

* Rémi Cardona <remi@gentoo.org> schrieb:

> Best example on how to do that is gstreamer. All the plugins come in 3 
> tarballs but each can be built individually. Really clean.

ACK. That's how it always should be.
All my own packages also work this way - ev'ry thing else
doesnt get released ;-P

> >If the upstream really blocks it, do a 
> >fork / maintain a patchline (like OSS-QM project does).
> >
> >I'm already doing so with several packages.
> 
> I've seen you talk about that project before but I don't feel 
> comfortable going down that road. We want to work with upstream 
> and let them know what our needs are. Maintaining patches is a 
> lot of work and forking is even more work. 

Right. Therefore OSS-QM is meant as just an intermedia step.
To form a bridge between (virtually all) distros and upstream.

Sometimes the upstream has even valid reasons for not having 
distro's or embedded needs as high priority, eg. if their 
primary goal are new features - many upstream folks are coding
ONLY just for fun and simply don't want to care about stability.

In those cases we simply should let them pass and avoid 
unnecessary conflicts. In fact we (as OSS-QM) would then 
maintain the stable branch, while the upstream is just the
devel branch.

Maybe you remember the discussions about stable vs. dev kernel
branches: the kernel folks wanted to give up stable branches,
leaving them to the individual distros and concentrate just on
devel branch. A lot of people were totally unhappy with this 
idea, so they abondened the idea. Otherwise the kernel would 
have been the killer job for an project like OSS-QM.

> Even though I'm still a relatively young Gentoo dev (only been 
> here for 1.5 years), I have yet to see upstream projects reject 
> build patches that make our lives easier.

Yeah, I've experienced this a lot of times and learned that
it doesn't make sense trying to convice people to things 
they simply do not want. 

So I founded OSS-QM as an proxy/overlay project with the primary 
goals: reacting ASAP, providing things the distros need in an 
*generic* way, but not developing new features. The upstream gets
announced about patches, but doesn't get penetrated with things 
they're obviously not interested in.


cu
-- 
---------------------------------------------------------------------
 Enrico Weigelt    ==   metux IT service - http://www.metux.de/
---------------------------------------------------------------------
 Please visit the OpenSource QM Taskforce:
 	http://wiki.metux.de/public/OpenSource_QM_Taskforce
 Patches / Fixes for a lot dozens of packages in dozens of versions:
	http://patches.metux.de/
---------------------------------------------------------------------
--
gentoo-dev@lists.gentoo.org mailing list

Re: [gentoo-dev] RFC: language bindings as separate packages

[Jan Kundrát]

Enrico Weigelt wrote:
> Maybe you remember the discussions about stable vs. dev kernel
> branches: the kernel folks wanted to give up stable branches,
> leaving them to the individual distros and concentrate just on
> devel branch. A lot of people were totally unhappy with this 
> idea, so they abondened the idea. Otherwise the kernel would 
> have been the killer job for an project like OSS-QM.

What you mean by the "stable branch", the 2.4.x, 2.6.16.x or something
completely different?

Cheers,
-jkt


-- 
gentoo-dev@lists.gentoo.org mailing list