[gentoo-dev] some multilib-minimal enhancements [2/6]: add frob for consumers to disable automagic header wrapping

[gentoo-dev] some multilib-minimal enhancements [2/6]: add frob for consumers to disable automagic header wrapping

[Greg Turner]

[-- Attachment #1: Type: text/plain, Size: 1 bytes --]



[-- Attachment #2: 002-MULTILIB_INSECURE_INSTALL.patch --]
[-- Type: text/x-patch, Size: 2306 bytes --]

Add a MULTILIB_INSECURE_INSTALL variable to eclass/multilib-minimal.eclass

Sometimes the "multilib magic header" business is an unwanted
feature.  For example, it is infuriating to be forced
to wrap a header file (or, less offensively, but still quite
offensively, to be forced to implement inter-ABI
header-smuggling code) just to appease multilib_check_headers,
in the case where an ABI-specific header triggers a multilib_check_headers
failure.

This provides a frob to turn the whole thing off, collisions be
damned.  Arguably, it would be better to have a per-header frob,
as well, but this this is better than nothing and trivial to implement.

--- 001-debug-print-function/multilib-minimal.eclass	2013-12-03 02:17:30.144384409 -0800
+++ 002-MULTILIB_INSECURE_INSTALL/multilib-minimal.eclass	2013-12-03 02:23:06.888008751 -0800
@@ -29,6 +29,20 @@ case ${EAPI:-0} in
 	*) die "EAPI=${EAPI} is not supported" ;;
 esac
 
+# @ECLASS-VARIABLE: MULTILIB_INSECURE_INSTALL
+# @DEFAULT-UNSET
+# @DESCRIPTION:
+# If set to a nonempty value, multilib-minimal_src_install will not perform
+# automatic checking of headers for inter-ABI conflicts, nor will it automate
+# wrapping of header files.  Instead, multilib_src_install pseudophases will
+# run without any special protection and the MULTILIB_WRAPPED_HEADERS array
+# will be ignored.
+# See:
+# @CODE@
+# http://devmanual.gentoo.org/eclass-reference/multilib-build.eclass/index.html
+# @CODE@
+# (or the multilib-build.eclass source itself) for further information about this
+# feature.
 
 inherit eutils multilib-build
 
@@ -108,15 +122,19 @@ multilib-minimal_src_install() {
 				emake DESTDIR="${D}" install
 			fi
 		fi
-		# Do multilib magic only when >1 ABI is used.
-		if [[ ${#MULTIBUILD_VARIANTS[@]} -gt 1 ]]; then
+
+		# Do multilib magic only when >1 ABI is used and
+		# MULTILIB_INSECURE_INSTALL is not set
+		if [[ ${#MULTIBUILD_VARIANTS[@]} -gt 1 && \
+		      -z ${MULTILIB_INSECURE_INSTALL} ]]; then
 			multilib_prepare_wrappers
 			multilib_check_headers
 		fi
 		popd >/dev/null || die
 	}
 	multilib_foreach_abi multilib-minimal_abi_src_install
-	multilib_install_wrappers
+
+	[[ ${MULTILIB_INSECURE_INSTALL} ]] || multilib_install_wrappers
 
 	if declare -f multilib_src_install_all >/dev/null ; then
 		multilib_src_install_all

Re: [gentoo-dev] some multilib-minimal enhancements [2/6]: add frob for consumers to disable automagic header wrapping

[hasufell]

On 12/11/2013 10:18 PM, Greg Turner wrote:
> 

this needs more explanation. Why do we want this?

Re: [gentoo-dev] some multilib-minimal enhancements [2/6]: add frob for consumers to disable automagic header wrapping

[Greg Turner]

On Wed, Dec 11, 2013 at 1:37 PM, hasufell <hasufell@gentoo.org> wrote:
> On 12/11/2013 10:18 PM, Greg Turner wrote:
>>
>
> this needs more explanation. Why do we want this?

Sometimes the automagic header stuff is working against the ebuild
author, or at least threatens to, in the future.

The most plausible etiology would be: ABI "X" is going to generate
header_x.h but ABI "Y" is going to generate header_y.h, or no header
at all.  An argument could certainly be made this this calls for
either (a) a way to exempt a particular header from the header
automagic -- not all of them or (b) a general exemption from
ebuild-crashing, for headers that are present for a certain ABI but
not in other ABI's.  The only reason I didn't implement either of
those (both of which are probably preferable to mine) is that it
seemed nontrivial, and I'm lazy.

Regardless, if our standard advice is "try not to use this automagic
header wrapping feature, it can break autoconf assumptions" (IIRC, it
is -- but if it isn't, it probably should be), then we ought to
provide /some/ convenient means to get around it, other than sneaking
those headers in through some kind of inter-abi back-door, in order to
fake out the automagic (which is, effectively, what we require right
now).

FWIW, I'm pretty sure that thus far, every time I've thought,
initially, that I needed this feature, I've ended up deciding I didn't
need it, after all.

-gmt

Re: [gentoo-dev] some multilib-minimal enhancements [2/6]: add frob for consumers to disable automagic header wrapping

[Michał Górny]

[-- Attachment #1: Type: text/plain, Size: 1870 bytes --]

Dnia 2013-12-11, o godz. 17:20:08
Greg Turner <gmt@malth.us> napisał(a):

> On Wed, Dec 11, 2013 at 1:37 PM, hasufell <hasufell@gentoo.org> wrote:
> > On 12/11/2013 10:18 PM, Greg Turner wrote:
> >>
> >
> > this needs more explanation. Why do we want this?
> 
> Sometimes the automagic header stuff is working against the ebuild
> author, or at least threatens to, in the future.

If you can't solve a problem, ask someone else. Working around it is
almost never a good solution.

> The most plausible etiology would be: ABI "X" is going to generate
> header_x.h but ABI "Y" is going to generate header_y.h, or no header
> at all.  An argument could certainly be made this this calls for
> either (a) a way to exempt a particular header from the header
> automagic -- not all of them or (b) a general exemption from
> ebuild-crashing, for headers that are present for a certain ABI but
> not in other ABI's.  The only reason I didn't implement either of
> those (both of which are probably preferable to mine) is that it
> seemed nontrivial, and I'm lazy.

Then you need to wrap all the possible headers. The function works well
if you specify headers that are only available in some of the ABIs.

> Regardless, if our standard advice is "try not to use this automagic
> header wrapping feature, it can break autoconf assumptions" (IIRC, it
> is -- but if it isn't, it probably should be), then we ought to
> provide /some/ convenient means to get around it, other than sneaking
> those headers in through some kind of inter-abi back-door, in order to
> fake out the automagic (which is, effectively, what we require right
> now).

The advice tells to do things properly, not to choose even worse
solution. If wrapping breaks something, random header install is going
to screw up even worse.

-- 
Best regards,
Michał Górny

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 966 bytes --]

Re: [gentoo-dev] some multilib-minimal enhancements [2/6]: add frob for consumers to disable automagic header wrapping

[Greg Turner]

On Wed, Dec 11, 2013 at 10:33 PM, Michał Górny <mgorny@gentoo.org> wrote:
>> Regardless, if our standard advice is "try not to use this automagic
>> header wrapping feature, it can break autoconf assumptions" (IIRC, it
>> is -- but if it isn't, it probably should be), then we ought to
>> provide /some/ convenient means to get around it, other than sneaking
>> those headers in through some kind of inter-abi back-door, in order to
>> fake out the automagic (which is, effectively, what we require right
>> now).
>
> The advice tells to do things properly, not to choose even worse
> solution. If wrapping breaks something, random header install is going
> to screw up even worse.

The matter of whether ebuild authors know how to implement multilib
headers correctly or not is mostly orthogonal to that of whether or
not multilib-minimal.eclass throws up obstructions in the way of a
correct (or incorrect) implementation.  I'm attempting to address the
latter issue, not the former.

Encouraging everyone to wrap headers, even, for example, in
pathological cases where there is not, in fact, any header conflict
between ABI's, to begin with, seems to me like incurring a cost
(likelihood of broken autotools macros) with no benefit, I can
identify, to end-users.

Furthermore, the possibility of a superior, but more involved and
hypothetical solution to a problem is, in my experience, not always a
good reason to avoid a quick and simple solution, available
immediately.  If you prefer that I code up one of the more
sophisticated approaches I mentioned, I'd be happy and perfectly
capable to do so, just let me know which you prefer.

-gmt

Re: [gentoo-dev] some multilib-minimal enhancements [2/6]: add frob for consumers to disable automagic header wrapping

[Michał Górny]

[-- Attachment #1: Type: text/plain, Size: 467 bytes --]

Dnia 2013-12-11, o godz. 23:10:12
Greg Turner <gmt@malth.us> napisał(a):

> Encouraging everyone to wrap headers, even, for example, in
> pathological cases where there is not, in fact, any header conflict
> between ABI's, to begin with, seems to me like incurring a cost
> (likelihood of broken autotools macros) with no benefit, I can
> identify, to end-users.

Real life example first, pathological theories later.

-- 
Best regards,
Michał Górny

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 966 bytes --]

Re: [gentoo-dev] some multilib-minimal enhancements [2/6]: add frob for consumers to disable automagic header wrapping

[Greg Turner]

On Wed, Dec 11, 2013 at 11:20 PM, Michał Górny <mgorny@gentoo.org> wrote:
> Real life example first, pathological theories later.

As I stated before, I don't have a 100% real-life example as I've
found other workarounds in each case, as, clearly, has gx86, thus-far.
 However, the problem pertains to any non-header-wrapping solution to
a header conflict as defined by the header automagic.  Here's an only
partially-contrived example that I came up against, recently, when I
was hacking on mysql:

When compiling multi-abi mysql, I had things set up only to compile
the server portion for the best abi, and the client portion for every
abi.  This resulted in a situation where the header file generated
included a named-pipe file constant for the best ABI, but not for the
others; the header files were otherwise identical.  In fact, I managed
to solve that, by passing the named-pipe constant configuration
parameter to the non-best ABI's as well as the best ABI; however, what
if this simpler solution had not been available, for example, because
the constant was only applicable to the server build?

Then, I would have wanted to achieve the same effect, by using the
header generated for the best ABI for all of the ABI's, as the
presence of the constant would have had no detrimental effect on the
non-best ABI's, while it would have been required for the best-ABI on
the server-side.

Now, admittedly, we are drifting into a hypothetical case, but, I'd
say, a plausible one, likely to be similar to other cases that will
come up in future real-world situations.

How should I achieve the aforementioned result, assuming that my mysql
ebuild was based on multilib-minimal?

There are a number of ways, but perhaps the easiest would be to delete
the header file from each of the non-best ABI's before the header
automagic has the chance to run, while for the best ABI, instead of
deleting the header file, I stash it away somewhere in ${T} and then
restore it from ${T} during multilib_src_install_all.

While this solution works, note that I've had to jump through hoops to
prevent the header automagic feature from biting my ebuild in the
butt.  Mysql generates precisely the header file I want in the correct
moment, the best ABI installation phase, but I can't just leave it
there... instead, I must play a shell game where I hide it from the
header automagic and restore it afterwards, when the header automagic
isn't looking.

It's not a bug -- the header automagic and the mysql build system have
all performed their functions perfectly.  But it strongly suggests
that a misfeature exists, because we've forced the ebuild author to
play a shell game to deceive the header automagic framework that was
supposed to be doing them a favor.

This same misfeature exists in any case where the header automagic
insists that an identical header must be installed by all ABI's when,
in fact, we have found some better solution than header-wrapping.  As
I've already stated, the identical problem even exists in the case
that a header only gets generated for a single ABI, in which case,
there isn't even a conflict between the ABI's at all.  It would also
exist if, for example, the build script generated a high-resolution
build time specifier in a header somewhere... I could invent any
number of scenarios but the long and the short of it is, the header
wrapping feature is supposed to be an option to solve a conflict, not
a requirement -- yet we've provided no elegant, easy-to-use
alternative.

-gmt