[gentoo-user] How to determine 'Least Common Denominator' between Xen(Server) Hosts

[gentoo-user] How to determine 'Least Common Denominator' between Xen(Server) Hosts

[Pandu Poluan]

[-- Attachment #1: Type: text/plain, Size: 775 bytes --]

Hello list!

My company has 2 HP DL585 G5 servers and 5 Dell R... something servers. All
using AMD processors. They currently are acting as XenServer hosts.

How do I determine the 'least common denominator' for Gentoo VMs (running
as XenServer guests), especially for gcc flags?

I know that the (theoretical) best performance is to use -march=native ,
but since the processors of the HP servers are not exactly the same as the
Dell's, I'm concerned that compiling with -march=native will render the VMs
unable to migrate between the different hosts.

Note: Yes I know the HP servers are much older than the Dell ones, but if I
go -march=native then perform an emerge when the guest is on the Dell host,
the guest VM might not be able to migrate to the older HPs.

Rgds,
--

[-- Attachment #2: Type: text/html, Size: 909 bytes --]

Re: [gentoo-user] How to determine 'Least Common Denominator' between Xen(Server) Hosts

[the]

On 08/14/13 09:18, Pandu Poluan wrote:
> Hello list!
>
> My company has 2 HP DL585 G5 servers and 5 Dell R... something servers.
> All using AMD processors. They currently are acting as XenServer hosts.
>
> How do I determine the 'least common denominator' for Gentoo VMs
> (running as XenServer guests), especially for gcc flags?
>
> I know that the (theoretical) best performance is to use -march=native ,
> but since the processors of the HP servers are not exactly the same as
> the Dell's, I'm concerned that compiling with -march=native will render
> the VMs unable to migrate between the different hosts.
>
> Note: Yes I know the HP servers are much older than the Dell ones, but
> if I go -march=native then perform an emerge when the guest is on the
> Dell host, the guest VM might not be able to migrate to the older HPs.
>
> Rgds,
> --
>
-march=i386?

-- 
Stop talking and start compiling.
Linux user #557897

Re: [gentoo-user] How to determine 'Least Common Denominator' between Xen(Server) Hosts

[joost]

[-- Attachment #1: Type: text/plain, Size: 1087 bytes --]

Pandu.

I don't have the link handy. But there is a list of safe CFLAGS. Pick the one for the older CPU.
That should be safe.

--
Joost

Pandu Poluan <pandu@poluan.info> wrote:
>Hello list!
>
>My company has 2 HP DL585 G5 servers and 5 Dell R... something servers.
>All
>using AMD processors. They currently are acting as XenServer hosts.
>
>How do I determine the 'least common denominator' for Gentoo VMs
>(running
>as XenServer guests), especially for gcc flags?
>
>I know that the (theoretical) best performance is to use -march=native
>,
>but since the processors of the HP servers are not exactly the same as
>the
>Dell's, I'm concerned that compiling with -march=native will render the
>VMs
>unable to migrate between the different hosts.
>
>Note: Yes I know the HP servers are much older than the Dell ones, but
>if I
>go -march=native then perform an emerge when the guest is on the Dell
>host,
>the guest VM might not be able to migrate to the older HPs.
>
>Rgds,
>--

-- 
Sent from my Android phone with K-9 Mail. Please excuse my brevity.

[-- Attachment #2: Type: text/html, Size: 1431 bytes --]

Re: [gentoo-user] How to determine 'Least Common Denominator' between Xen(Server) Hosts

[Helmut Jarausch]

On 08/14/2013 07:18:41 AM, Pandu Poluan wrote:
> Hello list!
> 
> My company has 2 HP DL585 G5 servers and 5 Dell R... something  
> servers. All
> using AMD processors. They currently are acting as XenServer hosts.
> 
> How do I determine the 'least common denominator' for Gentoo VMs  
> (running
> as XenServer guests), especially for gcc flags?
> 
> I know that the (theoretical) best performance is to use  
> -march=native ,
> but since the processors of the HP servers are not exactly the same  
> as the
> Dell's, I'm concerned that compiling with -march=native will render  
> the VMs
> unable to migrate between the different hosts.
> 
> Note: Yes I know the HP servers are much older than the Dell ones,  
> but if I
> go -march=native then perform an emerge when the guest is on the Dell  
> host,
> the guest VM might not be able to migrate to the older HPs.
> 
> Rgds,
> --

Why not compute it yourself?

Do   cat /proc/cpuinfo on all machines and compute the intersection of  
all "flags"
Then enter something like the following into /etc/portage/make.conf

CFLAGS="-O3 -pipe -msse -msse2 -msse3 -msse4a -m3dnow"

Helmut

Re: [gentoo-user] How to determine 'Least Common Denominator' between Xen(Server) Hosts

[Wang Xuerui]

2013/8/14 Helmut Jarausch <jarausch@igpm.rwth-aachen.de>:
> Why not compute it yourself?
>
> Do   cat /proc/cpuinfo on all machines and compute the intersection of all
> "flags"
> Then enter something like the following into /etc/portage/make.conf
>
> CFLAGS="-O3 -pipe -msse -msse2 -msse3 -msse4a -m3dnow"


Or even better, directly intersect the GCC flags which can be obtained
in this way:

echo | gcc -O2 -pipe -march=native -E -v - 2>&1 | grep cc1

Also, according to the Gentoo Handbook going -O3 for the entire system
may cause instability and other problems. Has the situation changed
over the years?

Re: [gentoo-user] How to determine 'Least Common Denominator' between Xen(Server) Hosts

[Bruce Hill]

On Wed, Aug 14, 2013 at 12:18:41PM +0700, Pandu Poluan wrote:
> Hello list!
> 
> My company has 2 HP DL585 G5 servers and 5 Dell R... something servers. All
> using AMD processors. They currently are acting as XenServer hosts.
> 
> How do I determine the 'least common denominator' for Gentoo VMs (running
> as XenServer guests), especially for gcc flags?
> 
> I know that the (theoretical) best performance is to use -march=native ,
> but since the processors of the HP servers are not exactly the same as the
> Dell's, I'm concerned that compiling with -march=native will render the VMs
> unable to migrate between the different hosts.
> 
> Note: Yes I know the HP servers are much older than the Dell ones, but if I
> go -march=native then perform an emerge when the guest is on the Dell host,
> the guest VM might not be able to migrate to the older HPs.

To check what options CFLAGS set as -march=native would use:
gcc -march=native -E -v - </dev/null 2>&1 | sed -n 's/.* -v - //p'
(the first thing in the output is what CPU -march=native would enable)

Then you can run:
diff -u <(gcc -Q --help=target) <(gcc -march=native -Q --help=target)
to display target-specific options, versus native ones.

Assuming the 2 HP servers are identical, and the 5 Dell servers are identical,
you then only need to get the commonality of two processors (HP and Dell).
Since they're both AMD, you should have a good set of common features to help
you determine that "least common denominator", or target.
-- 
Happy Penguin Computers               >')
126 Fenco Drive                       ( \
Tupelo, MS 38801                       ^^
support@happypenguincomputers.com
662-269-2706 662-205-6424
http://happypenguincomputers.com/

A: Because it messes up the order in which people normally read text.
Q: Why is top-posting such a bad thing?
A: Top-posting.
Q: What is the most annoying thing in e-mail?

Don't top-post: http://en.wikipedia.org/wiki/Top_post#Top-posting

Re: [gentoo-user] How to determine 'Least Common Denominator' between Xen(Server) Hosts

[Paul Hartman]

[-- Attachment #1: Type: text/plain, Size: 538 bytes --]

On Wed, Aug 14, 2013 at 12:18 AM, Pandu Poluan <pandu@poluan.info> wrote:

> I know that the (theoretical) best performance is to use -march=native ,
> but since the processors of the HP servers are not exactly the same as the
> Dell's, I'm concerned that compiling with -march=native will render the VMs
> unable to migrate between the different hosts.



I use -mtune=native rather than -march=native, that way I can use some
advanced processor features if they are available, but my system will still
run if moved to a different host.

[-- Attachment #2: Type: text/html, Size: 878 bytes --]

Re: [gentoo-user] How to determine 'Least Common Denominator' between Xen(Server) Hosts

[Pandu Poluan]

[-- Attachment #1: Type: text/plain, Size: 1044 bytes --]

Thanks, Wang Xuerui and Bruce!

That's exactly helpful.

I'm going to do some testing Real Soon.


Rgds,
--



On Wed, Aug 14, 2013 at 1:28 PM, Wang Xuerui <idontknw.wang@gmail.com>wrote:

> 2013/8/14 Helmut Jarausch <jarausch@igpm.rwth-aachen.de>:
> > Why not compute it yourself?
> >
> > Do   cat /proc/cpuinfo on all machines and compute the intersection of
> all
> > "flags"
> > Then enter something like the following into /etc/portage/make.conf
> >
> > CFLAGS="-O3 -pipe -msse -msse2 -msse3 -msse4a -m3dnow"
>
>
> Or even better, directly intersect the GCC flags which can be obtained
> in this way:
>
> echo | gcc -O2 -pipe -march=native -E -v - 2>&1 | grep cc1
>
> Also, according to the Gentoo Handbook going -O3 for the entire system
> may cause instability and other problems. Has the situation changed
> over the years?
>
>


-- 
FdS Pandu E Poluan
* ~ IT Optimizer ~**
*
 • LOPSA Member #15248
 • Blog : http://pepoluan.tumblr.com
 • Linked-In : http://id.linkedin.com/in/pepoluan

[-- Attachment #2: Type: text/html, Size: 2436 bytes --]

Re: [gentoo-user] How to determine 'Least Common Denominator' between Xen(Server) Hosts

[Kerin Millar]

On 14/08/2013 13:15, Bruce Hill wrote:
> On Wed, Aug 14, 2013 at 12:18:41PM +0700, Pandu Poluan wrote:
>> Hello list!
>>
>> My company has 2 HP DL585 G5 servers and 5 Dell R... something servers. All
>> using AMD processors. They currently are acting as XenServer hosts.
>>
>> How do I determine the 'least common denominator' for Gentoo VMs (running
>> as XenServer guests), especially for gcc flags?
>>
>> I know that the (theoretical) best performance is to use -march=native ,
>> but since the processors of the HP servers are not exactly the same as the
>> Dell's, I'm concerned that compiling with -march=native will render the VMs
>> unable to migrate between the different hosts.

A couple of points:

* The effect of setting -march=native depends on the characteristics of
   the CPU (be it virtual or otherwise)
* The characteristics of the vCPU are defined by qemu's -cpu parameter
* qemu can emulate features not implemented by the host CPU (at a cost)

One way to go about it is to start qemu with a -cpu model that exposes 
features that all of your host CPUs have in common (or a subset 
thereof). In that case, -march=native is fine because all of the 
features that it detects as being available will be supported in 
hardware on the host side.

Another way is to expose the host CPU fully with "-cpu host" and to 
define your guest CFLAGS according to the most optimal subset. If you 
are looking for a 'perfect' configuration then this this would be the 
most effective method, if applied correctly.

Irrespective of the method, by examining /proc/cpuinfo and using the 
diff technique mentioned by Bruce, you should be able to determine the 
optimal configuration.

Finally, in cases where the host CPUs differ significantly - in that 
native would imply a different -march value - you may choose to augment 
your CFLAGS with -mtune=generic to even out performance across the 
board. I don't think this would apply to you though.

>>
>> Note: Yes I know the HP servers are much older than the Dell ones, but if I
>> go -march=native then perform an emerge when the guest is on the Dell host,
>> the guest VM might not be able to migrate to the older HPs.
>
> To check what options CFLAGS set as -march=native would use:
> gcc -march=native -E -v - </dev/null 2>&1 | sed -n 's/.* -v - //p'
> (the first thing in the output is what CPU -march=native would enable)
>
> Then you can run:
> diff -u <(gcc -Q --help=target) <(gcc -march=native -Q --help=target)
> to display target-specific options, versus native ones.
> Assuming the 2 HP servers are identical, and the 5 Dell servers are identical,
> you then only need to get the commonality of two processors (HP and Dell).
> Since they're both AMD, you should have a good set of common features to help
> you determine that "least common denominator", or target.
>

Re: [gentoo-user] How to determine 'Least Common Denominator' between Xen(Server) Hosts

[Kerin Millar]

On 14/08/2013 16:23, Paul Hartman wrote:
>
> On Wed, Aug 14, 2013 at 12:18 AM, Pandu Poluan <pandu@poluan.info
> <mailto:pandu@poluan.info>> wrote:
>
>     I know that the (theoretical) best performance is to use
>     -march=native , but since the processors of the HP servers are not
>     exactly the same as the Dell's, I'm concerned that compiling with
>     -march=native will render the VMs unable to migrate between the
>     different hosts.
>
>
>
> I use -mtune=native rather than -march=native, that way I can use some
> advanced processor features if they are available, but my system will
> still run if moved to a different host.

That's not how -mtune works. If -march is unspecified, it will default 
to the lowest common denominator for the platform which prevents the use 
of any distinguished processor features. For an amd64 install, that 
would be "-march=x86-64".

Instead, -mtune affects everything that -march doesn't. Though it 
doesn't affect the instructions that *can* be used, it may effect which 
of the allowed instructions are used and how. For instance, gcc includes 
processor pipeline descriptions for different microarchitectures so as 
to emit instructions in a way that tries to avoid pipeline hazards:

http://gcc.gnu.org/onlinedocs/gccint/Processor-pipeline-description.html

If performance matters, a better approach is to look at what 
-march=native enables and manually specify all options that are common 
between the hosts. Further, if the host CPU microarchitecture varies 
then I would suggest adding -mtune=generic so as not to make potentially 
erroneous assumptions in the course of applying that type of 
optimisation. Indeed, -mtune=generic is the default if neither -march 
nor -mtune are specified.

Regarding qemu, the main thing is never to use a feature that would 
incur costly emulation on the host side.

--Kerin

Re: [gentoo-user] How to determine 'Least Common Denominator' between Xen(Server) Hosts

[Paul Hartman]

On Thu, Aug 15, 2013 at 12:41 PM, Kerin Millar <kerframil@fastmail.co.uk> wrote:
>> I use -mtune=native rather than -march=native, that way I can use some
>> advanced processor features if they are available, but my system will
>> still run if moved to a different host.
>
>
> That's not how -mtune works. If -march is unspecified, it will default to
> the lowest common denominator for the platform which prevents the use of any
> distinguished processor features. For an amd64 install, that would be
> "-march=x86-64".
>
> Instead, -mtune affects everything that -march doesn't. Though it doesn't
> affect the instructions that *can* be used, it may effect which of the
> allowed instructions are used and how. For instance, gcc includes processor
> pipeline descriptions for different microarchitectures so as to emit
> instructions in a way that tries to avoid pipeline hazards:

Thanks very much for the clarification, I appreciate it.

Re: [gentoo-user] How to determine 'Least Common Denominator' between Xen(Server) Hosts

[Pandu Poluan]

[-- Attachment #1: Type: text/plain, Size: 2609 bytes --]

On Aug 16, 2013 12:26 AM, "Kerin Millar" <kerframil@fastmail.co.uk> wrote:
>
> On 14/08/2013 13:15, Bruce Hill wrote:
>>
>> On Wed, Aug 14, 2013 at 12:18:41PM +0700, Pandu Poluan wrote:
>>>
>>> Hello list!
>>>
>>> My company has 2 HP DL585 G5 servers and 5 Dell R... something servers.
All
>>> using AMD processors. They currently are acting as XenServer hosts.
>>>
>>> How do I determine the 'least common denominator' for Gentoo VMs
(running
>>> as XenServer guests), especially for gcc flags?
>>>
>>> I know that the (theoretical) best performance is to use -march=native ,
>>> but since the processors of the HP servers are not exactly the same as
the
>>> Dell's, I'm concerned that compiling with -march=native will render the
VMs
>>> unable to migrate between the different hosts.
>
>
> A couple of points:
>
> * The effect of setting -march=native depends on the characteristics of
>   the CPU (be it virtual or otherwise)
> * The characteristics of the vCPU are defined by qemu's -cpu parameter
> * qemu can emulate features not implemented by the host CPU (at a cost)
>
> One way to go about it is to start qemu with a -cpu model that exposes
features that all of your host CPUs have in common (or a subset thereof).
In that case, -march=native is fine because all of the features that it
detects as being available will be supported in hardware on the host side.
>
> Another way is to expose the host CPU fully with "-cpu host" and to
define your guest CFLAGS according to the most optimal subset. If you are
looking for a 'perfect' configuration then this this would be the most
effective method, if applied correctly.
>

AFAIK, that's how XenServer configured its hosts. There's "CPU Masking"
option when a heterogeneous pool of hosts were created, but I have the
impression that "CPU Masking" is only being employed by the 'xe toolstack'
(CloudStack) layer to determine to which host a VM can be migrated.

> Irrespective of the method, by examining /proc/cpuinfo and using the diff
technique mentioned by Bruce, you should be able to determine the optimal
configuration.
>
> Finally, in cases where the host CPUs differ significantly - in that
native would imply a different -march value - you may choose to augment
your CFLAGS with -mtune=generic to even out performance across the board. I
don't think this would apply to you though.
>

Certainly doesn't apply to me. Based on tech spec I have on the servers,
the processors are very similar... I just want to be doubly sure :-)

Thanks for the explanation (including the difference between 'march' and
'mtune' in your other email)!

Rgds,
--

[-- Attachment #2: Type: text/html, Size: 3249 bytes --]