[gentoo-user] Multilib ABI specific CPU use flags?

[gentoo-user] Multilib ABI specific CPU use flags?

[Jonathan Yong]

Hello,

I am getting AVX2 related compile errors in 32bit 
media-libs/opencv-4.4.0, there are some internet text mentioning not all 
AVX2 instructions are supported on 32bit x86.

For now, I have set -abi_x86_32 since I don't need it now.

Any plans to support ABI specific USE masking/setting?

[gentoo-user] Multilib ABI specific CPU use flags?

[Jonathan Yong]

[-- Attachment #1.1.1: Type: text/plain, Size: 304 bytes --]

Hello,

I am getting AVX2 related compile errors in 32bit 
media-libs/opencv-4.4.0, there are some internet text mentioning not all 
AVX2 instructions are supported on 32bit x86.

For now, I have set -abi_x86_32 since I don't need it now.

Any plans to support ABI specific USE masking/setting?

[-- Attachment #1.1.2: OpenPGP_0x713B5FE29C145D45_and_old_rev.asc --]
[-- Type: application/pgp-keys, Size: 8035 bytes --]

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 840 bytes --]

[gentoo-user] Re: Multilib ABI specific CPU use flags?

[Ashley Dixon]

[-- Attachment #1: Type: text/plain, Size: 2226 bytes --]

On Thu, Oct 08, 2020 at 11:57:52AM +0000, Jonathan Yong wrote:
> I am getting AVX2 related compile errors in 32bit media-libs/opencv-4.4.0,
> there are some internet text mentioning not all AVX2 instructions are
> supported on 32bit x86.
> 
> For now, I have set -abi_x86_32 since I don't need it now.
> 
> Any plans to support ABI specific USE masking/setting?

Do you mean that you want to install the 32-bit and 64-bit version of a package,
enabling one set of flags on one, and another set on the other?

The problem you will face is that the CPU_FLAGS_X86 option, which allows you  to
disable AVX(2)  instructions,  amongst  many  other  processor  extensions  [1],
applies to both x86 and amd64 architectures, and from reading the according wiki
page [2], I don't think it's possible to discriminate between those two;  it  is
the responsibility of the package to test  whether  a  particular  AVX  function
should be employed or not.

Can you post the output of `cpuid2cpuflags`  (from  app-portage/cpuid2cpuflags)?
If this list includes "avx2", and the compilation is still failing,  this  is  a
BUG and should be reported as such.  I think it's known to the Gentoo  community
already, although it seems to be an issue with upstream [3].   Analogous  issues
exist in other packages [4].

The problem in upstream is probably caused by this call to  _mm256_extract_epi64
[5], introduced quite a while ago (the nature of this function, being  exclusive
to 64-bit processors, was even noted in the commit message [6]).

It will probably be fixed soon. No need to worry since you don't need the 32-bit
binary interface anyway.

[1] https://gitweb.gentoo.org/repo/gentoo.git/plain/profiles/desc/cpu_flags_x86.desc
[2] https://wiki.gentoo.org/wiki/CPU_FLAGS_X86#Configuration
[3] https://bugs.gentoo.org/747163
[4] https://github.com/pytorch/pytorch/issues/17901
[5] https://github.com/opencv/opencv/blame/152e6476d9a270e6ce35d3f1b200a303654e576e/modules/core/include/opencv2/core/hal/intrin_avx.hpp#L130
[6] https://github.com/opencv/opencv/commit/75315fb297a010153d3acab8111ebb87d909cdcf

-- 

Ashley Dixon
suugaku.co.uk

2A9A 4117
DA96 D18A
8A7B B0D2
A30E BF25
F290 A8AA


[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [gentoo-user] Re: Multilib ABI specific CPU use flags?

[Jonathan Yong]

[-- Attachment #1.1.1: Type: text/plain, Size: 3006 bytes --]

On 10/8/20 3:40 PM, Ashley Dixon wrote:
> On Thu, Oct 08, 2020 at 11:57:52AM +0000, Jonathan Yong wrote:
>> I am getting AVX2 related compile errors in 32bit media-libs/opencv-4.4.0,
>> there are some internet text mentioning not all AVX2 instructions are
>> supported on 32bit x86.
>>
>> For now, I have set -abi_x86_32 since I don't need it now.
>>
>> Any plans to support ABI specific USE masking/setting?
> 
> Do you mean that you want to install the 32-bit and 64-bit version of a package,
> enabling one set of flags on one, and another set on the other?
> 

Yes exactly, but more to CPU, I understand perfectly other USE flags 
will make dependency calculation much more complicated.

> The problem you will face is that the CPU_FLAGS_X86 option, which allows you  to
> disable AVX(2)  instructions,  amongst  many  other  processor  extensions  [1],
> applies to both x86 and amd64 architectures, and from reading the according wiki
> page [2], I don't think it's possible to discriminate between those two;  it  is
> the responsibility of the package to test  whether  a  particular  AVX  function
> should be employed or not.
> 
> Can you post the output of `cpuid2cpuflags`  (from  app-portage/cpuid2cpuflags)?
> If this list includes "avx2", and the compilation is still failing,  this  is  a
> BUG and should be reported as such.  I think it's known to the Gentoo  community
> already, although it seems to be an issue with upstream [3].   Analogous  issues
> exist in other packages [4].
> 

I have a skylake system:
CPU_FLAGS_X86: aes avx avx2 f16c fma3 mmx mmxext pclmul popcnt rdrand 
sse sse2 sse3 sse4_1 sse4_2 ssse3

> The problem in upstream is probably caused by this call to  _mm256_extract_epi64
> [5], introduced quite a while ago (the nature of this function, being  exclusive
> to 64-bit processors, was even noted in the commit message [6]).
> 

Yes this was the same error I had, albeit through some C++ error message 
mangling.

> It will probably be fixed soon. No need to worry since you don't need the 32-bit
> binary interface anyway.
> 

This time, I can avoid it, since OpenCV is not a dependency of 32bit 
Wine, which I do use.

> [1] https://gitweb.gentoo.org/repo/gentoo.git/plain/profiles/desc/cpu_flags_x86.desc
> [2] https://wiki.gentoo.org/wiki/CPU_FLAGS_X86#Configuration
> [3] https://bugs.gentoo.org/747163
> [4] https://github.com/pytorch/pytorch/issues/17901
> [5] https://github.com/opencv/opencv/blame/152e6476d9a270e6ce35d3f1b200a303654e576e/modules/core/include/opencv2/core/hal/intrin_avx.hpp#L130
> [6] https://github.com/opencv/opencv/commit/75315fb297a010153d3acab8111ebb87d909cdcf
> 

I'm open to fixing ebuild packages by myself in my own overlay, though I 
prefer not to due to eventual bit rot. Any hints on how to mask CPU USE 
flags based on multilib arch in the .ebuild file itself?

My previous attempt was through portage bashrc, but it was too late in 
the emerge processing.

[-- Attachment #1.1.2: OpenPGP_0x713B5FE29C145D45_and_old_rev.asc --]
[-- Type: application/pgp-keys, Size: 8035 bytes --]

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 840 bytes --]

Re: [gentoo-user] Re: Multilib ABI specific CPU use flags?

[Walter Dnes]

On Fri, Oct 09, 2020 at 12:26:17AM +0000, Jonathan Yong wrote
> 
> I have a skylake system:
> CPU_FLAGS_X86: aes avx avx2 f16c fma3 mmx mmxext pclmul popcnt rdrand 
> sse sse2 sse3 sse4_1 sse4_2 ssse3

  A heavy-handed 2-part solution
1) remove "avx2" from CPU_FLAGS_X86
2) assuming you have "-march=native" or "-march=skylake" in your CFLAGS
and CXXFLAGS, follow that with "-march=no-avx2".  e.g...
"-march=skylake -march=no-avx2"

  This tells gcc "I don't have avx2, so don't compile any avx2 code".
I think the performance loss will be minimal.

-- 
Walter Dnes <waltdnes@waltdnes.org>
I don't run "desktop environments"; I run useful applications

Re: [gentoo-user] Re: Multilib ABI specific CPU use flags?

[Jonathan Yong]

[-- Attachment #1.1.1: Type: text/plain, Size: 889 bytes --]

On 10/9/20 5:44 AM, Walter Dnes wrote:
> On Fri, Oct 09, 2020 at 12:26:17AM +0000, Jonathan Yong wrote
>>
>> I have a skylake system:
>> CPU_FLAGS_X86: aes avx avx2 f16c fma3 mmx mmxext pclmul popcnt rdrand
>> sse sse2 sse3 sse4_1 sse4_2 ssse3
> 
>    A heavy-handed 2-part solution
> 1) remove "avx2" from CPU_FLAGS_X86
This was my initial reaction, then I realized I don't need 32bit OpenCV 
(for now), since the only package depending on it is 64bit.

> 2) assuming you have "-march=native" or "-march=skylake" in your CFLAGS
> and CXXFLAGS, follow that with "-march=no-avx2".  e.g...
> "-march=skylake -march=no-avx2"
> 
>    This tells gcc "I don't have avx2, so don't compile any avx2 code".
> I think the performance loss will be minimal.
> 

The CMake configure arguments still tell OpenCV to build the AVX2 code 
unfortunately. So #1 is required regardless.

[-- Attachment #1.1.2: OpenPGP_0x713B5FE29C145D45_and_old_rev.asc --]
[-- Type: application/pgp-keys, Size: 8035 bytes --]

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 840 bytes --]

Re: [gentoo-user] Re: Multilib ABI specific CPU use flags?

[Ashley Dixon]

[-- Attachment #1: Type: text/plain, Size: 1614 bytes --]

On Fri, Oct 09, 2020 at 12:26:17AM +0000, Jonathan Yong wrote:
> I'm open to fixing ebuild packages by myself in my own overlay, though I
> prefer not to due to eventual bit rot. Any hints on how to mask CPU USE
> flags based on multilib arch in the .ebuild file itself?

I suppose you could detect whether a CPU_FLAG is enabled for a x86_32  multilib,
although be wary that `MULTILIB_ABI_FLAG` is only  exported  for  some  multilib
functions, such as `multilib_src_configure` [1]:

        if [[ ${MULTILIB_ABI_FLAG} == abi_x86_32 ]] && use cpu_flags_x86_avx2;
        then
		ewarn "Disabling AVX2 extensions for 32-bit build."
		# circumvention measures
	fi

I'm not sure of the best way to achieve the "circumvention measures", hence  the
comment.  You could run this conditional after  GLOBALCMAKEARGS  has  been  set,
omitting the CPU_BASELINE definition in the main block [2],  or  you  could  sed
through the USE flags and disable `cpu_flags_x86_avx2` there.   You're  probably
better asking on IRC for the best-practice way to achieve this.

As mentioned in the relevant bug report, this is not  an  issue  caused  by  the
ebuild, so although this might prove to be a temporary fix, it will hopefully be
resolved by upstream very soon. Just be patient. ;-)

[1] https://wiki.gentoo.org/wiki/Project:Multilib/multilib-build#MULTILIB_ABI_FLAG
[2] https://github.com/opencv/opencv/wiki/CPU-optimizations-build-options#customizing-cmake-options
[3] https://bugs.gentoo.org/747163#c15

-- 

Ashley Dixon
suugaku.co.uk

2A9A 4117
DA96 D18A
8A7B B0D2
A30E BF25
F290 A8AA


[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [gentoo-user] Re: Multilib ABI specific CPU use flags?

[Jonathan Yong]

On 10/9/20 9:27 AM, Ashley Dixon wrote:
> On Fri, Oct 09, 2020 at 12:26:17AM +0000, Jonathan Yong wrote:
>> I'm open to fixing ebuild packages by myself in my own overlay, though I
>> prefer not to due to eventual bit rot. Any hints on how to mask CPU USE
>> flags based on multilib arch in the .ebuild file itself?
> 
> I suppose you could detect whether a CPU_FLAG is enabled for a x86_32  multilib,
> although be wary that `MULTILIB_ABI_FLAG` is only  exported  for  some  multilib
> functions, such as `multilib_src_configure` [1]:
> 
>          if [[ ${MULTILIB_ABI_FLAG} == abi_x86_32 ]] && use cpu_flags_x86_avx2;
>          then
> 		ewarn "Disabling AVX2 extensions for 32-bit build."
> 		# circumvention measures
> 	fi
> 
> I'm not sure of the best way to achieve the "circumvention measures", hence  the
> comment.  You could run this conditional after  GLOBALCMAKEARGS  has  been  set,
> omitting the CPU_BASELINE definition in the main block [2],  or  you  could  sed
> through the USE flags and disable `cpu_flags_x86_avx2` there.   You're  probably
> better asking on IRC for the best-practice way to achieve this.
> 

OK, thanks for the hints, I had something similar when I tried 
/etc/portage/bashrc, so MULTILIB_ABI_FLAG is the variable to look out for.

I will try this next time if another package misbehaves with AVX2. I'm 
OK with trial and error learning.

> As mentioned in the relevant bug report, this is not  an  issue  caused  by  the
> ebuild, so although this might prove to be a temporary fix, it will hopefully be
> resolved by upstream very soon. Just be patient. ;-)
> > [1] 
https://wiki.gentoo.org/wiki/Project:Multilib/multilib-build#MULTILIB_ABI_FLAG
> [2] https://github.com/opencv/opencv/wiki/CPU-optimizations-build-options#customizing-cmake-options
> [3] https://bugs.gentoo.org/747163#c15
> 

I'm not so worried about OpenCV itself, since it is very much an active 
project.

The notion that I am unable to fix a problem myself, on a system that I 
own, because of some technicality, even as a temporary measure is 
disconcerting. Controlling every piece of how a software is configured, 
built and installed is why I love Gentoo's portage system so much.

Re: [gentoo-user] Re: Multilib ABI specific CPU use flags?

[Jonathan Yong]

[-- Attachment #1.1.1: Type: text/plain, Size: 2244 bytes --]

On 10/9/20 9:27 AM, Ashley Dixon wrote:
> On Fri, Oct 09, 2020 at 12:26:17AM +0000, Jonathan Yong wrote:
>> I'm open to fixing ebuild packages by myself in my own overlay, though I
>> prefer not to due to eventual bit rot. Any hints on how to mask CPU USE
>> flags based on multilib arch in the .ebuild file itself?
> 
> I suppose you could detect whether a CPU_FLAG is enabled for a x86_32  multilib,
> although be wary that `MULTILIB_ABI_FLAG` is only  exported  for  some  multilib
> functions, such as `multilib_src_configure` [1]:
> 
>          if [[ ${MULTILIB_ABI_FLAG} == abi_x86_32 ]] && use cpu_flags_x86_avx2;
>          then
> 		ewarn "Disabling AVX2 extensions for 32-bit build."
> 		# circumvention measures
> 	fi
> 
> I'm not sure of the best way to achieve the "circumvention measures", hence  the
> comment.  You could run this conditional after  GLOBALCMAKEARGS  has  been  set,
> omitting the CPU_BASELINE definition in the main block [2],  or  you  could  sed
> through the USE flags and disable `cpu_flags_x86_avx2` there.   You're  probably
> better asking on IRC for the best-practice way to achieve this.
> 

OK, thanks for the hints, I had something similar when I tried 
/etc/portage/bashrc, so MULTILIB_ABI_FLAG is the variable to look out for.

I will try this next time if another package misbehaves with AVX2. I'm 
OK with trial and error learning.

> As mentioned in the relevant bug report, this is not  an  issue  caused  by  the
> ebuild, so although this might prove to be a temporary fix, it will hopefully be
> resolved by upstream very soon. Just be patient. ;-)
> > [1] 
https://wiki.gentoo.org/wiki/Project:Multilib/multilib-build#MULTILIB_ABI_FLAG
> [2] https://github.com/opencv/opencv/wiki/CPU-optimizations-build-options#customizing-cmake-options
> [3] https://bugs.gentoo.org/747163#c15
> 

I'm not so worried about OpenCV itself, since it is very much an active 
project.

The notion that I am unable to fix a problem myself, on a system that I 
own, because of some technicality, even as a temporary measure is 
disconcerting. Controlling every piece of how a software is configured, 
built and installed is why I love Gentoo's portage system so much.

[-- Attachment #1.1.2: OpenPGP_0x713B5FE29C145D45_and_old_rev.asc --]
[-- Type: application/pgp-keys, Size: 8035 bytes --]

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 840 bytes --]

Re: [gentoo-user] Re: Multilib ABI specific CPU use flags?

[Ashley Dixon]

[-- Attachment #1: Type: text/plain, Size: 1230 bytes --]

On Sat, Oct 10, 2020 at 03:47:42AM +0000, Jonathan Yong wrote:
> The notion that I am unable to fix a problem myself, on a system that I own,
> because of some technicality, even as a temporary measure is disconcerting.
> Controlling every piece of how a software is configured, built and installed
> is why I love Gentoo's portage system so much.

We cannot possibly be expected to have the knowledge to fix every  single  issue
which might arise on our systems.  Linux, and Gentoo in particular, is great due
to the freedom it offers.  Just because you do not know how  to  fix  a  problem
immediately, does not mean you cannot learn.

I would be more disconcerted if I was using a Windows or iOS  system,  in  which
the vendor(s) intentionally take steps  to  prevent  users  from  fixing  tricky
problems  like  these,  delegating  the  responsibility  to  their   technically
incompetent front-line support teams.  A  lack  of  hand-holding  and  presumed-
ability is not a need for concern, but more of an opportunity to grow.

P.S.  Do you know you're sending most messages twice, one signed, and  one  not?

-- 

Ashley Dixon
suugaku.co.uk

2A9A 4117
DA96 D18A
8A7B B0D2
A30E BF25
F290 A8AA


[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [gentoo-user] Re: Multilib ABI specific CPU use flags?

[Jonathan Yong]

[-- Attachment #1.1.1: Type: text/plain, Size: 1724 bytes --]

On 10/10/20 7:53 AM, Ashley Dixon wrote:
> On Sat, Oct 10, 2020 at 03:47:42AM +0000, Jonathan Yong wrote:
>> The notion that I am unable to fix a problem myself, on a system that I own,
>> because of some technicality, even as a temporary measure is disconcerting.
>> Controlling every piece of how a software is configured, built and installed
>> is why I love Gentoo's portage system so much.
> 
> We cannot possibly be expected to have the knowledge to fix every  single  issue
> which might arise on our systems.  Linux, and Gentoo in particular, is great due
> to the freedom it offers.  Just because you do not know how  to  fix  a  problem
> immediately, does not mean you cannot learn.
> 

This is the reason I sent out an email to ask if such a system is 
already in place, and if there are better ways to do it. Regardless, the 
issue with OpenCV is fixed for now.

> I would be more disconcerted if I was using a Windows or iOS  system,  in  which
> the vendor(s) intentionally take steps  to  prevent  users  from  fixing  tricky
> problems  like  these,  delegating  the  responsibility  to  their   technically
> incompetent front-line support teams.  A  lack  of  hand-holding  and  presumed-
> ability is not a need for concern, but more of an opportunity to grow.
>
The opaque nature of Windows is why I migrated away from it, it is now 
only used for video games.

> P.S.  Do you know you're sending most messages twice, one signed, and  one  not?
> 

Was it just the previous message? I canceled sending the message when I 
realized it wasn't signed. Google SMTP must have accepted it anyway.

...and I just found the option to sign emails by default in Thunderbird 78.


[-- Attachment #1.1.2: OpenPGP_0x713B5FE29C145D45_and_old_rev.asc --]
[-- Type: application/pgp-keys, Size: 8035 bytes --]

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 840 bytes --]

Re: [gentoo-user] Re: Multilib ABI specific CPU use flags?

[Wols Lists]

On 11/10/20 05:42, Jonathan Yong wrote:
> Was it just the previous message? I canceled sending the message when I
> realized it wasn't signed. Google SMTP must have accepted it anyway.

You can't cancel a message. Once it's left your inbox, it's gone.

And Google won't/can't do anything (unless the recipient is gmail)
because just as it will have gone from your outbox, it will have gone
from their system before you've even realised "oh shit I didn't mean to
hit send".

You're effectively relying on the recipient's mail client to honour a
request to throw the previous message away, and many clients either
don't honour, or don't even recognise, such a request.

That's why a previous mail client of mine (Turnpike), by default,
wouldn't empty the outbox until the contents were at least a minute old.

Cheers,
Wol

Re: [gentoo-user] Re: Multilib ABI specific CPU use flags?

[Jonathan Yong]

[-- Attachment #1.1.1: Type: text/plain, Size: 1022 bytes --]

On 10/11/20 7:59 AM, Wols Lists wrote:
> On 11/10/20 05:42, Jonathan Yong wrote:
>> Was it just the previous message? I canceled sending the message when I
>> realized it wasn't signed. Google SMTP must have accepted it anyway.
> 
> You can't cancel a message. Once it's left your inbox, it's gone.
> 
> And Google won't/can't do anything (unless the recipient is gmail)
> because just as it will have gone from your outbox, it will have gone
> from their system before you've even realised "oh shit I didn't mean to
> hit send".
> 
> You're effectively relying on the recipient's mail client to honour a
> request to throw the previous message away, and many clients either
> don't honour, or don't even recognise, such a request.
> 
> That's why a previous mail client of mine (Turnpike), by default,
> wouldn't empty the outbox until the contents were at least a minute old.
> 

No, I mean the cancel button in progress box that Thunderbird shows when 
an email is sent, not Outlook style cancel.

[-- Attachment #1.1.2: OpenPGP_0x713B5FE29C145D45_and_old_rev.asc --]
[-- Type: application/pgp-keys, Size: 8035 bytes --]

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 840 bytes --]