[gentoo-dev] New global use flags: 3dnowext, mmxext, ssse3, sse4_1, avx, avx2

[gentoo-dev] New global use flags: 3dnowext, mmxext, ssse3, sse4_1, avx, avx2

[Matt Turner]

And at the same time, clean up the descriptions of the other flags.
The existing descriptions were clearly copy-and-pasted and contained
things like "faster floating point optimization for SSSE3 capable
chips" when SSSE3 didn't add any floating point instructions.

3dnow: Use the 3DNow! instruction set
3dnowext: Use the Enhanced 3DNow! instruction set
mmx: Use the MMX instruction set
mmxext: Use the Extended MMX instruction set (intersection of Enhanced
3DNow! and SSE instruction sets) (3dnowext or sse in cpuinfo)
sse: Use the SSE instruction set
sse2: Use the SSE2 instruction set
sse3: Use the SSE3 instruction set (pni in cpuinfo)
ssse3: Use the SSSE3 instruction set
sse4_1: Use the SSE 4.1 instruction set
avx: Use the AVX instruction set
avx2: Use the AVX2 instruction set

I'll make these changes in a few days.

We don't seem to have a use of an sse4_2 USE flag anywhere yet, notably.

Unfortunately we do have two uses of "sse4", which should be corrected
to be more specific:

media-libs/freeverb3
net-misc/bfgminer

Re: [gentoo-dev] New global use flags: 3dnowext, mmxext, ssse3, sse4_1, avx, avx2

[Andreas K. Huettel]

Am Montag, 16. Dezember 2013, 00:34:13 schrieb Matt Turner:
> 3dnow: Use the 3DNow! instruction set
> 3dnowext: Use the Enhanced 3DNow! instruction set
> mmx: Use the MMX instruction set
> mmxext: Use the Extended MMX instruction set (intersection of Enhanced
> 3DNow! and SSE instruction sets) (3dnowext or sse in cpuinfo)
> sse: Use the SSE instruction set
> sse2: Use the SSE2 instruction set
> sse3: Use the SSE3 instruction set (pni in cpuinfo)
> ssse3: Use the SSSE3 instruction set
> sse4_1: Use the SSE 4.1 instruction set
> avx: Use the AVX instruction set
> avx2: Use the AVX2 instruction set

What's the point of these flags?
(or to be more precise, are they really justified whenever they are used?)

Usually the set of cpu instructions should be controlled by your CFLAGS, and 
I've been actively patching packages (that do not do manually coded assembly) 
to make such flags unnecessary.

-- 

Andreas K. Huettel
Gentoo Linux developer 
dilfridge@gentoo.org
http://www.akhuettel.de/

Re: [gentoo-dev] New global use flags: 3dnowext, mmxext, ssse3, sse4_1, avx, avx2

[Matt Turner]

On Sun, Dec 15, 2013 at 4:20 PM, Andreas K. Huettel
<dilfridge@gentoo.org> wrote:
> Am Montag, 16. Dezember 2013, 00:34:13 schrieb Matt Turner:
>> 3dnow: Use the 3DNow! instruction set
>> 3dnowext: Use the Enhanced 3DNow! instruction set
>> mmx: Use the MMX instruction set
>> mmxext: Use the Extended MMX instruction set (intersection of Enhanced
>> 3DNow! and SSE instruction sets) (3dnowext or sse in cpuinfo)
>> sse: Use the SSE instruction set
>> sse2: Use the SSE2 instruction set
>> sse3: Use the SSE3 instruction set (pni in cpuinfo)
>> ssse3: Use the SSSE3 instruction set
>> sse4_1: Use the SSE 4.1 instruction set
>> avx: Use the AVX instruction set
>> avx2: Use the AVX2 instruction set
>
> What's the point of these flags?
> (or to be more precise, are they really justified whenever they are used?)
>
> Usually the set of cpu instructions should be controlled by your CFLAGS, and
> I've been actively patching packages (that do not do manually coded assembly)
> to make such flags unnecessary.

Often they're for enabling assembly code that uses these instruction
sets. For pixman, a package that I'm very familiar with, they turn on
code using these instruction sets using intrinsics in C. I believe
they are justified.

If the package simply uses the flag to add an -m<isa> flag to CFLAGS,
then we should definitely remove it. If I recall correctly, I have
only seen one instance of this.

Re: [gentoo-dev] New global use flags: 3dnowext, mmxext, ssse3, sse4_1, avx, avx2

[Francesco R.]

Il 16/12/2013 01:30, Matt Turner ha scritto:
> On Sun, Dec 15, 2013 at 4:20 PM, Andreas K. Huettel
> <dilfridge@gentoo.org> wrote:
>> Am Montag, 16. Dezember 2013, 00:34:13 schrieb Matt Turner:
>>> 3dnow: Use the 3DNow! instruction set
>>> 3dnowext: Use the Enhanced 3DNow! instruction set
>>> mmx: Use the MMX instruction set
>>> mmxext: Use the Extended MMX instruction set (intersection of Enhanced
>>> 3DNow! and SSE instruction sets) (3dnowext or sse in cpuinfo)
>>> sse: Use the SSE instruction set
>>> sse2: Use the SSE2 instruction set
>>> sse3: Use the SSE3 instruction set (pni in cpuinfo)
>>> ssse3: Use the SSSE3 instruction set
>>> sse4_1: Use the SSE 4.1 instruction set
>>> avx: Use the AVX instruction set
>>> avx2: Use the AVX2 instruction set
>> What's the point of these flags?
>> (or to be more precise, are they really justified whenever they are used?)
>>
>> Usually the set of cpu instructions should be controlled by your CFLAGS, and
>> I've been actively patching packages (that do not do manually coded assembly)
>> to make such flags unnecessary.
> Often they're for enabling assembly code that uses these instruction
> sets. For pixman, a package that I'm very familiar with, they turn on
> code using these instruction sets using intrinsics in C. I believe
> they are justified.
>
> If the package simply uses the flag to add an -m<isa> flag to CFLAGS,
> then we should definitely remove it. If I recall correctly, I have
> only seen one instance of this.
>
another possible case are packages that do run-time checking of usable
instruction set.
The use flag could restrict the code to be compiled and installed from
the ebuild.
Probably never used like this tough

Re: [gentoo-dev] New global use flags: 3dnowext, mmxext, ssse3, sse4_1, avx, avx2

[Jeroen Roovers]

On Mon, 16 Dec 2013 02:13:28 +0100
"Francesco R." <vivo75@gmail.com> wrote:

> another possible case are packages that do run-time checking of usable
> instruction set.

www-plugins/adobe-flash[1]

> The use flag could restrict the code to be compiled and installed from
> the ebuild.

Yes, and in that extreme case, it installs nothing at all. :)

      jer


[1] https://bugs.gentoo.org/show_bug.cgi?id=410547

Re: [gentoo-dev] New global use flags: 3dnowext, mmxext, ssse3, sse4_1, avx, avx2

[Rick "Zero_Chaos" Farina]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On 12/15/2013 07:20 PM, Andreas K. Huettel wrote:
> Am Montag, 16. Dezember 2013, 00:34:13 schrieb Matt Turner:
>> 3dnow: Use the 3DNow! instruction set
>> 3dnowext: Use the Enhanced 3DNow! instruction set
>> mmx: Use the MMX instruction set
>> mmxext: Use the Extended MMX instruction set (intersection of Enhanced
>> 3DNow! and SSE instruction sets) (3dnowext or sse in cpuinfo)
>> sse: Use the SSE instruction set
>> sse2: Use the SSE2 instruction set
>> sse3: Use the SSE3 instruction set (pni in cpuinfo)
>> ssse3: Use the SSSE3 instruction set
>> sse4_1: Use the SSE 4.1 instruction set
>> avx: Use the AVX instruction set
>> avx2: Use the AVX2 instruction set
> 
> What's the point of these flags?
> (or to be more precise, are they really justified whenever they are used?)
> 
> Usually the set of cpu instructions should be controlled by your CFLAGS, and 
> I've been actively patching packages (that do not do manually coded assembly) 
> to make such flags unnecessary.
> 
I think this is a great time to point out three things:

TL;DR Some packages are sensitive, easier for binpkgs in certain cases,
broken build systems may require it.

1.) Some packages have really sensitive cflags (mostly these packages
offer USE=custom-cflags).  Packages such as app-crypt/johntheripper
offer a bunch of cflag options for optimization of supported cpu
features while not actually using the user's full cflags which would
yield poor optimization or broken code.  I can't say I'm in love with
this situation, but facts are facts, and turning on USE=custom-cflags
seems to inevitably yield significantly slower benchmarks in the case of
jtr while the use flags work well.

2.) Gentoo supports binary packages.  In the case of apps like
johntheripper (and a few other crackers who wish to remain nameless)
these use flags are really helpful for the users as portage can
automatically know it should rebuild the package for higher optimization
or w/e.

3.) Broken build systems.  Forgive me for the term, but packages like
libpng seem to require arcane configure flags like
"--enable-arm-neon=$(usex neon on off)" to enable my neon fpu despite
passing -mfpu=neon.  Retarded? Probably. Fact? Definitely.

Thanks,
Zero
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.22 (GNU/Linux)
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/

iQIcBAEBAgAGBQJSr4e+AAoJEKXdFCfdEflK9TkQAJ0AnIQMcvDKwo/WWLmsoZHh
B5WD1WrXwTl2xkwBs64lRBw4tFl1zU14zHvFmC3T4rPd6aEA2J7DNiYAnztGGTNu
RWeUWYL3Qs+RJ4jASssJrMQmc3aEHEEzqMBE/yFL+x3ioeTilrROq+P1UlwoyUBh
6wJtkFiuTE7/DeqxhSaE81JvPVxRVt/i0nJoTvN6Kd4WLOzHZejekANErXaDF9FA
MFXLI9OpQ4lY5QrZMJ7msSZJA0DnGt9Hi16QHtaaDQ0MPlJmtoVJlulsc14QgCCk
a73+ndo20apOqAUOTy1cxXGxbrnU4TcDB16e/Sfc4eqAWiDEugXT6NTH+hnTegQ5
qE3N6genNGvRTTchVmNemnSs90UPTFhKhC5fqOo7hyqokpK3mvbU+tKL4NoOKjIQ
LyoaghSRfzFcQhpRaEGhviGnm+Mvs9wSglBonob4PShrlcB2mpoZicA7oPuaE6Wi
UC52KoryvsXtVxe5djziwd/1kgaC60wF1LnJ9s0BzxSoWMyibuD7wv5+hI5G9sQf
0zg/uzGlFVrMKjwCFz0Vr4STDRVAIULeKlbfMACzOE1jUGw5TXLjdWq+yMCTN0eH
jqYo15IkPEC/a9+P+3fMVOL+7ZvQj5neoqSn1d0dEqo3y5d8Z1lZIw/ZYosWWObd
qr08PGwWw+fElOa5BnSj
=hhoR
-----END PGP SIGNATURE-----

Re: [gentoo-dev] New global use flags: 3dnowext, mmxext, ssse3, sse4_1, avx, avx2

[Jeroen Roovers]

On Mon, 16 Dec 2013 18:07:42 -0500
"Rick \"Zero_Chaos\" Farina" <zerochaos@gentoo.org> wrote:

> 3.) Broken build systems.  Forgive me for the term, but packages like
> libpng seem to require arcane configure flags like
> "--enable-arm-neon=$(usex neon on off)" to enable my neon fpu despite
> passing -mfpu=neon.  Retarded? Probably. Fact? Definitely.

Doesn't look broken or retarded to me. Read the arm/filter_neon.S and
arm/filter_neon_intrinsics.c comments for pretty good reasons why it's
done this way. Also, patches are welcome.


     jer

[gentoo-dev] Re: New global use flags: 3dnowext, mmxext, ssse3, sse4_1, avx, avx2

[Duncan]

Matt Turner posted on Sun, 15 Dec 2013 15:34:13 -0800 as excerpted:

> sse3: Use the SSE3 instruction set (pni in cpuinfo)
> ssse3: Use the SSSE3 instruction set

I'd suggest a parenthetical on ssse3 as well, something like:

ssse3: Use the SSSE3 instruction set (NOT sse3, three s)

I know that confused me for awhile, and I tend to be reasonably literate 
(as a user, anyway) on this sort of thing, so I'd not be surprised in the 
least if a lot of sse3 only folks end up enabling it in error, as I did.  
The explicit three-count pointer should help eliminate that sort of 
confusion.

-- 
Duncan - List replies preferred.   No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master."  Richard Stallman

Re: [gentoo-dev] Re: New global use flags: 3dnowext, mmxext, ssse3, sse4_1, avx, avx2

[Michael Orlitzky]

On 12/16/2013 05:44 AM, Duncan wrote:
> Matt Turner posted on Sun, 15 Dec 2013 15:34:13 -0800 as excerpted:
> 
>> sse3: Use the SSE3 instruction set (pni in cpuinfo)
>> ssse3: Use the SSSE3 instruction set
> 
> I'd suggest a parenthetical on ssse3 as well, something like:
> 
> ssse3: Use the SSSE3 instruction set (NOT sse3, three s)
> 
> I know that confused me for awhile, and I tend to be reasonably literate 
> (as a user, anyway) on this sort of thing,

Why not go all the way?

"Enable use of the SSSE3 instruction set (NOT sse3). This is needed by
projects which contain assembly code or which use certain compiler
intrinsics. It is not a replacement for CFLAGS and friends."

You can make a case for brevity, but how many times do you find yourself
(a) reading USE flag descriptions, and (b) thinking they're too long?

Re: [gentoo-dev] Re: New global use flags: 3dnowext, mmxext, ssse3, sse4_1, avx, avx2

[Alan McKinnon]

On 16/12/2013 17:38, Michael Orlitzky wrote:
> On 12/16/2013 05:44 AM, Duncan wrote:
>> Matt Turner posted on Sun, 15 Dec 2013 15:34:13 -0800 as excerpted:
>>
>>> sse3: Use the SSE3 instruction set (pni in cpuinfo)
>>> ssse3: Use the SSSE3 instruction set
>>
>> I'd suggest a parenthetical on ssse3 as well, something like:
>>
>> ssse3: Use the SSSE3 instruction set (NOT sse3, three s)
>>
>> I know that confused me for awhile, and I tend to be reasonably literate 
>> (as a user, anyway) on this sort of thing,
> 
> Why not go all the way?
> 
> "Enable use of the SSSE3 instruction set (NOT sse3). This is needed by
> projects which contain assembly code or which use certain compiler
> intrinsics. It is not a replacement for CFLAGS and friends."

The second and third sentences add nothing to the description. They only
describe how cpu instruction sets in general work and are used, but tell
you nothing about ssse3.

I recommend a short expansion on what ssse3 is, and shortening the rest
while retaining it's meaning, then modifying all cpu instruction set
flags similarly

> You can make a case for brevity, but how many times do you find yourself
> (a) reading USE flag descriptions, and (b) thinking they're too long?

It's possible to have a very verbose, and meaningless, description. be
verbose by all means if the verbosity carries useful information

-- 
Alan McKinnon
alan.mckinnon@gmail.com

Re: [gentoo-dev] Re: New global use flags: 3dnowext, mmxext, ssse3, sse4_1, avx, avx2

[Michael Orlitzky]

On 12/16/2013 01:21 PM, Alan McKinnon wrote:
>>
>> "Enable use of the SSSE3 instruction set (NOT sse3). This is needed by
>> projects which contain assembly code or which use certain compiler
>> intrinsics. It is not a replacement for CFLAGS and friends."
> 
> The second and third sentences add nothing to the description. They only
> describe how cpu instruction sets in general work and are used, but tell
> you nothing about ssse3.

USE flags are a user interface, not documentation. A good description is
one that helps the user decide, "do I want to enable this?" I'm not
particular about the wording -- and my use of "needed" was a mistake --
but with that purpose in mind, a good description should contain,

  1. Why might I want to enable this?

  2. Why might I want to disable this?

So e.g. my description is missing #2, "You should not enable this unless
you have a processor supporting SSSE3."


> then modifying all cpu instruction set flags similarly

Yup.

Re: [gentoo-dev] Re: New global use flags: 3dnowext, mmxext, ssse3, sse4_1, avx, avx2

[Alan McKinnon]

On 16/12/2013 22:17, Michael Orlitzky wrote:
> On 12/16/2013 01:21 PM, Alan McKinnon wrote:
>>>
>>> "Enable use of the SSSE3 instruction set (NOT sse3). This is needed by
>>> projects which contain assembly code or which use certain compiler
>>> intrinsics. It is not a replacement for CFLAGS and friends."
>>
>> The second and third sentences add nothing to the description. They only
>> describe how cpu instruction sets in general work and are used, but tell
>> you nothing about ssse3.
> 
> USE flags are a user interface, not documentation. A good description is
> one that helps the user decide, "do I want to enable this?" I'm not
> particular about the wording -- and my use of "needed" was a mistake --
> but with that purpose in mind, a good description should contain,
> 
>   1. Why might I want to enable this?
> 
>   2. Why might I want to disable this?
> 
> So e.g. my description is missing #2, "You should not enable this unless
> you have a processor supporting SSSE3."
> 
> 
>> then modifying all cpu instruction set flags similarly
> 
> Yup.


Is it worthwhile adding a link in the description to a resource that
describes a USE flag in greater detail?

Most flags are obvious as to what they do, like jpeg/png/gif: the user
simply asks himself if he wants to use those formats. Most flags are
like this, and come down to user preference; portage merges in any
missing parts required

cpu sets are different: they also depend on whether they CAN be used in
a given environment. Portage cannot know if ssse3 will work for the cpu
running the code so cannot fill in the missing bits. The user must first
check if the intended cpu supports ssse3 at all (often preceded by
finding out how to find this out).

That's a lot of vital ,necessary info and it deserves the full
documentation treatment. A USE description is not the place for that,
but a link could work nicely. Especially if the linked page describes
the intent and usage of the USE flag in full.



-- 
Alan McKinnon
alan.mckinnon@gmail.com

[gentoo-dev] Re: New global use flags: 3dnowext, mmxext, ssse3, sse4_1, avx, avx2

[Matt Turner]

On Sun, Dec 15, 2013 at 3:34 PM, Matt Turner <mattst88@gentoo.org> wrote:
> I'll make these changes in a few days.

Committed. Feel free to remove the local descriptions from your packages.

Re: [gentoo-dev] New global use flags: 3dnowext, mmxext, ssse3, sse4_1, avx, avx2

[Jan Matejka]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA512

On Sun, 15 Dec 2013 15:34:13 -0800
Matt Turner <mattst88@gentoo.org> wrote:

> And at the same time, clean up the descriptions of the other flags.
> The existing descriptions were clearly copy-and-pasted and contained
> things like "faster floating point optimization for SSSE3 capable
> chips" when SSSE3 didn't add any floating point instructions.
> 
> 3dnow: Use the 3DNow! instruction set
> 3dnowext: Use the Enhanced 3DNow! instruction set
> mmx: Use the MMX instruction set
> mmxext: Use the Extended MMX instruction set (intersection of Enhanced
> 3DNow! and SSE instruction sets) (3dnowext or sse in cpuinfo)
> sse: Use the SSE instruction set
> sse2: Use the SSE2 instruction set
> sse3: Use the SSE3 instruction set (pni in cpuinfo)
> ssse3: Use the SSSE3 instruction set
> sse4_1: Use the SSE 4.1 instruction set
> avx: Use the AVX instruction set
> avx2: Use the AVX2 instruction set
> 
> I'll make these changes in a few days.
> 
> We don't seem to have a use of an sse4_2 USE flag anywhere yet,
> notably.
> 
> Unfortunately we do have two uses of "sse4", which should be corrected
> to be more specific:
> 
> media-libs/freeverb3
> net-misc/bfgminer
> 


Is it possible to make this in a way so all the instruction set use
flags can be read from the use.desc by some simple epression?

Like 1. "is" prefix/postfix for Instruction Set or 2. include the
"\sinstruction set\s" in the description? or 3. have them listed in
special file for that or 4. whatever we can agree on is the proper way.

I'm asking because https://github.com/yaccz/cufd


- --
Jan Matějka        | Gentoo Developer
https://gentoo.org | Gentoo Linux
GPG: A33E F5BC A9F6 DAFD 2021  6FB6 3EBF D45B EEB6 CA8B
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.22 (GNU/Linux)

iQEcBAEBCgAGBQJSsuKDAAoJEIN+7RD5ejahBlkH+wRiFLAzxazKQ9a/jdpYOeJr
j4uY8Q9iAu/4xK3QMXscosaShrz5bW/XGAxRvfT0pqAe8APUrQTw5V+0cFX/yVJ1
2FQBSPgGXPKyq/AQQ6kPlwsQaCVaYxcWA5bOv+dxfVsEcSMYSQsGeX1BdK2S7wHN
h6upIw3qFWln75TLUcO52PHR9YNgWTYZvqJWmaLJDDXBDzcuJAVmLJLtf+ketiCK
SjNxZlUQpKQzgszb3dTUPeMSbpPuiCNRG9JFG/q8eXlrfLt9qygJvYpFn7OKmEem
8Bmc1LeERhDEvxCb+xAJDFF4UTHNOpj5H57EMmuakiIrEd4f+xMUK4lJtvRTpbU=
=25vk
-----END PGP SIGNATURE-----