[gentoo-dev] ifc USE flag, fortran support and some general notes about multi-compiler support

[gentoo-dev] ifc USE flag, fortran support and some general notes about multi-compiler support

[Anton Starikov]

Hi!

Recently there was added "ifc" flag to support Intel Fortran Compiler. 
And few ebuilds already take it into account. But...

1) Mainly it corresponds to some scientific libraries and programs. And 
I guess that not 99% of such users use ifc. What about ones who use 
Absoft, Lahey Fujitsu, Portland Group and so on? I guess IFC didn't take 
99% of market yet. Because other companies still alive. :)  Moreover, 
AFAIK a lot of people prefer different compilers because IFC still has 
some compatibility problems with complicated codes. Which can be 
compiled without problems with Absoft or Lahey, for example.

2) Optimization flags. I've take a short look at some ebuilds which use 
IFC and ICC flags. Mostly they have hard-encoded into ebuilds 
optimization flags. Which also doesn't seems to be really brilliant 
idea. Of course, some of them check IFCFLAGS and ICCFLAGS environment 
variables...but it is not common, and it is less portable from pint of 
view of different compilers.

OK, partly you can solve such problems with setting some environment 
variables, like F77 and so on. But...it not looks like a really gentoo 
solution, it is not really reliable solution (for example blas-atlas 
will just ignore it, AFIR) and so on.

So, probably there is some need to make some kind of redesign of the 
idea. OK, for C and C++ it is not so important probably, because there 
is not such big choice and anyway mostly you would prefer GCC, and if 
you want, support for icc and tiny-cc here already.

Situation with Fortran just looks a bit more complicated. And different 
from C world.

So, has it sense to think about implementing some abstract 
multi-compiler support? at least for Fortran guys.
But generally, probably, it could be just let say "minor compiler 
support", which means, for C and F you still have gcc and g77 (gfortran 
in future)  as a main compiler of the system, but you can also choose 
some kind of minor compiler. Now it almost work in such way with ICC.
ICC behaves just as some kind of minor compiler and some software that 
can be compiled with ICC will be compiled with icc.

This idea "of minor compiler support" could be also interesting for 
different applications. For example, hypothetic situation: I have SPARC 
with Solaris. I'm not going to completely change my OS, but, probably, I 
would like to take advantages of gentoo portage  also. And probably I 
would like to choose second compiler (SUN compiler of course), which 
will be used for compilation of some stuff that can be compiled with it.
And, if software use automake and autoconf, it probably will be compiled 
without real troubles.

It basically looks very interesting and not such hypothetic, because 
anyway nowadays you fill your Solaris box (or Irix box) with incredible 
amount of GNU stuff, if you want to have some comfort :) And, I guess, 
portage can be really, easy adopted to that. OK, it couldn't be in 
mainstream. But...if, let say, you have 300 Solaris clients and ppl 
whant to use a really a LOT of different stuff, either you compile 
program by program by yourself and put in on a servers into shared /usr 
folder (or whatevere else), or you just force portage think that you 
have gentoo and portage do you job for you and just go and drink coffe 
:) Thats real example.


Of course, there is two ways to implement this.
One - it's just to add more and more USE flags for different compilers. 
it is some kind of clean way with granted result, if you ebuild support 
this compiler which you have in USE flags...but...
I guess, at least in case of Fortran. ebuild that can be compiled with 
ANY Fortran compiler wail NEVER check for all these flags, because for 
such behavior ebuild should be tested with ALL these compilers, reported 
that it works  and submitted a patch for ebuild for particular compiler. 
That practically probably never will be happen :)

But almost for sure a lot of software will be compiled without any 
problems with most of major Fortran compiler. And even with almost any 
widely spread C compilers. Especially if it case of scientific software.
I really can't imaging situation, when MPICH will not compile itself and 
will not compile F90 interface with any non-exotic compiler. I've just 
never meet such situation. :)

One more thing. Lets imaging that next month on market will be available 
one more C++ or Fortran compiler, which really excellent :) Most of 
software can be build with it...but... it will take long time to add 
more flag to USE, change all ebuilds with it and so on.

So, there is coming in mind some idea of different approach. Transparent 
  level of multicompiler support. OK, it is not granted. 
But...scientific guys use to live in not granted world :)
It probably looks like:
1) You have two more flags: "altcc" and "altfc"
2) somewhere (in make.conf for ex) you have specify :
ALTCC=C-compiler-which-I-really-like
ALTCXX=C++-compiler-which-I-really-like
ALTF77=F77-compiler-which-I-really-like
ALTF90=F90-compiler-which-I-really-like
ALTCFLAGS="optimize-it-to-make-a-rocket"
ALTF77LAGS="optimize-it-to-make-a-rocket"

and thats all.
Ebuild just check for "altcc" and "altfc" flags in USE, if they are, it 
force configure or whatever else to use it. Thats done. Of course you 
can't be sure that it will be compiled on all possible alternative 
compilers, but you can try :)

And it make your life really flexible. You do can what you want. Thats 
Gentoo philosophy, I guess :)

It just a general talk, don't kick me if I'm wrong :)

Anton Starikov

--
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] ifc USE flag, fortran support and some general notes about multi-compiler support

[Donnie Berkholz]

[-- Attachment #1: Type: text/plain, Size: 2068 bytes --]

On Wed, 2004-08-25 at 15:06, Anton Starikov wrote:

<snip>

> So, has it sense to think about implementing some abstract 
> multi-compiler support? at least for Fortran guys.
> But generally, probably, it could be just let say "minor compiler 
> support", which means, for C and F you still have gcc and g77 (gfortran 
> in future)  as a main compiler of the system, but you can also choose 
> some kind of minor compiler. Now it almost work in such way with ICC.
> ICC behaves just as some kind of minor compiler and some software that 
> can be compiled with ICC will be compiled with icc.

This should be added to gcc-config, so it can support independent
language compilers.

<snip>

> So, there is coming in mind some idea of different approach. Transparent 
>   level of multicompiler support. OK, it is not granted. 
> But...scientific guys use to live in not granted world :)
> It probably looks like:
> 1) You have two more flags: "altcc" and "altfc"
> 2) somewhere (in make.conf for ex) you have specify :
> ALTCC=C-compiler-which-I-really-like
> ALTCXX=C++-compiler-which-I-really-like
> ALTF77=F77-compiler-which-I-really-like
> ALTF90=F90-compiler-which-I-really-like
> ALTCFLAGS="optimize-it-to-make-a-rocket"
> ALTF77LAGS="optimize-it-to-make-a-rocket"

Yep. Some alternative compilers have their own configuration files for
compiler-specific flags, however. They should still draw the generic
flags from make.conf, such as CFLAGS, CXXFLAGS or FFLAGS (fortran).

<snip>

> Ebuild just check for "altcc" and "altfc" flags in USE, if they are, it 
> force configure or whatever else to use it. Thats done. Of course you 
> can't be sure that it will be compiled on all possible alternative 
> compilers, but you can try :)

Nah, USE isn't the right place for it. Ebuilds should respect the
environment variables selecting compilers, such as CC, CXX, FC and so
forth. The USE flags should be for other things, e.g. patches that are
required for a specific compiler to work.
-- 
Donnie Berkholz
Gentoo Linux

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-dev] ifc USE flag, fortran support and some general notes about multi-compiler support

[Donnie Berkholz]

[-- Attachment #1: Type: text/plain, Size: 488 bytes --]

On Wed, 2004-08-25 at 15:25, Donnie Berkholz wrote:
> Nah, USE isn't the right place for it. Ebuilds should respect the
> environment variables selecting compilers, such as CC, CXX, FC and so
> forth. The USE flags should be for other things, e.g. patches that are
> required for a specific compiler to work.

This doesn't make any sense. USE flags shouldn't be sticking their heads
in at all. The patches should check what CC/etc. are set to.
-- 
Donnie Berkholz
Gentoo Linux

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-dev] ifc USE flag, fortran support and some general notes about multi-compiler support

[Anton Starikov]

Donnie Berkholz wrote:
> This should be added to gcc-config, so it can support independent
> language compilers.
It is not compleet solution. Change everytime default compiler, if I 
understand correctly...not sounds nice.

> Yep. Some alternative compilers have their own configuration files for
> compiler-specific flags, however. They should still draw the generic
> flags from make.conf, such as CFLAGS, CXXFLAGS or FFLAGS (fortran).
Did you read correctly?  I whant to deal with two compilers at the same 
time. And it is really not excellent idea to froce them to use the SAME 
flags from make.conf. As example, -O3 is not really best for Intel 
compilers sometime. But can be goof enough for GCC. Ok, you can set 
IFCFLAGS on make.conf, but it is sollution for IFC ONLY.
Unfortunatelly, you have ONLY ONE set of flags for Fortran compiler 
(FFLAGS) in make.conf. :)

> Nah, USE isn't the right place for it. Ebuilds should respect the
> environment variables selecting compilers, such as CC, CXX, FC and so
> forth. The USE flags should be for other things, e.g. patches that are
> required for a specific compiler to work.
Again. they will respect SOMETIMES. But what if I want to use TWO 
compilers? Main and alternative at the same time? Should I change 
variables it every time?
I'm just speaking about adding some fucntionality for GENERAL 
alternative compiler. It can be used in some particular applications. 
Not everywhere. And such applications or libraries will try to compile 
with GENERAL alternative compiler. No patches for it. Just you own risk.
No changes in make.conf and environment variables each time when toy 
merge such applications.
Simple example. I really like to have fresh version of LAM-MPI in my 
developmnet machine. But I whant to build it with ICC and IFC. Nowdays 
if I have IFC and ICC and flags - it give me nothing. If I'm going to 
change my environmnet variables - I have to remove it from autoupdate 
and merge it manually, each time. With stetting these variables 
manually. Basically, it give ,me just nothing. it almost the same, like 
check is there newer version,  download tgz with sources, run configure 
and install manually.
Ok, I probably whant to use alternative compiler (GENERAL ONE) not for 
all ebuilds which support it. I can use /etc/portage/packages.use and 
I'm happy. It is pretty general thing if you doing some csience and 
dealing with fortran.

I really whant for me to have all near-scientific stuff to be compiled 
with alternative compilers, but NOT THE REST.

Do you really see currently something implemented to have such things in 
portage? I don't. So, I doing this almost manualy. I really can't leave 
it for emerge -u world. Because or it will try to compile with ICC and 
IFC, or with Lahey or with Absoft something that I don't whant to be 
compiled in such way.
Or it just compile everything with GCC and G77, and at morning I will 
find that my programs really don't want to be linked against MPI anymore :)

There is no currently support for GENERAL ALTERNATIVE COMPILER in 
gentoo. Which can be really helpfull in some situations.

I'm speaking about such mechanism.
I want to emerge -u world and be happy without doing a lot of things 
manually.
Or I want to bootstrap machine from scratch without siting near to it 
and handling all those things manually. I just want to give some 
configuration to portage that I want to have and go to do something else.

I really like gentoo. I use it for a two years. And OK, boostraping was 
nice. Updating was nice. Latter I handled all thing with near-scientific 
stuff manually on my home and office machines with different compilers. 
But latter my collegs start to be interesting from me to migrate from 
other distributions like Debian and mandrake to gentoo. And now they 
also have to deal with all this stuff. You know...basically me now have 
to deal with such stuff on their machines also, which is not greatest 
pleasure and time-spending.

If I understand correctly, gentoo  means freedom to have what you whant 
:) And I guess there is some, minor of course, part of gentoo users, 
which can be really happy with multi (or double) compiler support.
Again. There is no such functionality now in gentoo. It is really 
something new. Even normal support for use of ICC and GCC at the SAME 
TIME does not exist.

I can provide patches for some part of near-scientific ebuilds to 
provide such functionality. Because I anyway doing and mainteining such 
pathes now for me and my collegs. I really don't think that most of 
ebuilds need it. But for some it could be helpfull. Later probably it 
could be found that for a bigger number of ebuilds and users it is not 
so bad also to support it and people will support such support to other 
ebuilds.

At least I think there is something that could be discussed. Probably 
idea will changed after discussion.
But if I maintain already some extra fuctionality for gentoo for some 
people. I guess it is not so bad to maintain it for more, and I know 
that other people, who use gentoo for the same tasks will like it also.

Anton.

--
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] ifc USE flag, fortran support and some general notes about multi-compiler support

[Anton Starikov]

Donnie Berkholz wrote:
> On Wed, 2004-08-25 at 15:25, Donnie Berkholz wrote:
> 
>>Nah, USE isn't the right place for it. Ebuilds should respect the
>>environment variables selecting compilers, such as CC, CXX, FC and so
>>forth. The USE flags should be for other things, e.g. patches that are
>>required for a specific compiler to work.
> 
> 
> This doesn't make any sense. USE flags shouldn't be sticking their heads
> in at all. The patches should check what CC/etc. are set to.

About USE flags, it's completely different idea from IFC or ICC flags.
Just couple flags only (could be more, for example if some devs and some 
users want to support alternative java machines, not system default for 
their ebuilds) which tell that you want to for something to be compiled 
with alternative set of compiler and compiler flags, differ from system 
default. It just flags that provide extra-functionality. OK, probably it 
can be done in different way, here I agree. But currently it looks with 
/etc/portage/package.use very simple and comfortable and coming into 
gentoo really very smoothly.

BTW, I'm not asking to make something for me.
I'm just telling that there is place, where extra functionality can be 
added into gentoo. And I can work for that. Anyway I do it already :)

And I just want to discuss the possible ways of implementing it.
And for sure currently there is no normal and comfortable ways to have 
it. Probably as usually my explanations not so good :)

Anton.

--
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] ifc USE flag, fortran support and some general notes about multi-compiler support

[Donnie Berkholz]

[-- Attachment #1: Type: text/plain, Size: 1603 bytes --]

On Wed, 2004-08-25 at 16:26, Anton Starikov wrote:
> Donnie Berkholz wrote:
> > On Wed, 2004-08-25 at 15:25, Donnie Berkholz wrote:
> > 
> >>Nah, USE isn't the right place for it. Ebuilds should respect the
> >>environment variables selecting compilers, such as CC, CXX, FC and so
> >>forth. The USE flags should be for other things, e.g. patches that are
> >>required for a specific compiler to work.
> > 
> > 
> > This doesn't make any sense. USE flags shouldn't be sticking their heads
> > in at all. The patches should check what CC/etc. are set to.
> 
> About USE flags, it's completely different idea from IFC or ICC flags.
> Just couple flags only (could be more, for example if some devs and some 
> users want to support alternative java machines, not system default for 
> their ebuilds) which tell that you want to for something to be compiled 
> with alternative set of compiler and compiler flags, differ from system 
> default. It just flags that provide extra-functionality. OK, probably it 
> can be done in different way, here I agree. But currently it looks with 
> /etc/portage/package.use very simple and comfortable and coming into 
> gentoo really very smoothly.

USE flags are for features _within_ packages, not for things you do _to_
the package. Choosing a toolchain shouldn't be done with USE flags.
That's what gcc-config is for. You seem to be rejecting it out of hand
rather than considering an expansion of its functionality. For example,
it could read a file to switch compilers in a package-dependent way.
-- 
Donnie Berkholz
Gentoo Linux

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-dev] ifc USE flag, fortran support and some general notes about multi-compiler support

[Olivier Fisette]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Hi,

What about per package environment variable declarations? I have been 
experiencing problems similar to Anton's: I compile my system using GCC, but 
I use ICC for some packages I use in intensive calculations. To do so, I must 
check every "world" update and manually set env variables such as CC. 
Additionnally, I do not use the same "CFLAGS" for these packages as for the 
rest of my system. I am not aware of any mechanism allowing me to automate 
this process with the current Portage system.

What about having a file such as /etc/portage/package.env, that we could use 
to set per package compilers, "CFLAGS" and such? It would be pretty nice to 
be able to configure something along those lines:

=app-sci/emboss-2.9* CC="icc" CFLAGS="IWantFastSequenceAlignments"

... and not have to worry about manually setting environment variables 
whenever these packages are updated.

This would certainly not solve all issues related to alternative compilers, 
but it would make many things easier, and we could get rid of silly "USE" 
flags like "icc", that are not related to the packages' content (the ebuilds 
could simply check their environment and adapt the build process or report a 
compiler incompatibility).

What do you think?

- -- 
Olivier Fisette (ribosome)
Gentoo Linux Developer
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.2.5 (GNU/Linux)

iD8DBQFBLSTgr/GGfYtwOqoRAnpoAJ9cUHhIXXIn9ViLKDrIy1QoADS3/gCeNGV6
I1BLY6gTmq8V/8to8V1eoYY=
=/M8O
-----END PGP SIGNATURE-----

--
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] ifc USE flag, fortran support and some general notes about multi-compiler support

[Anton Starikov]

Donnie Berkholz wrote:

 > USE flags are for features _within_ packages, not for things you do _to_
 > the package. Choosing a toolchain shouldn't be done with USE flags.
 > That's what gcc-config is for. You seem to be rejecting it out of hand
 > rather than considering an expansion of its functionality. For example,
 > it could read a file to switch compilers in a package-dependent way.

Yes, complitely agree. This way reqiure to make some changes to portage 
system. If someone want to implement it. But this way of course much 
better. Here I agree for 100%.
I just use USE flags as some kind of quasi sollution for now. Without 
changing something in portage. Of course if it will be accepted, it is 
much better to implement it into more clean way.

Anton.


--
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] ifc USE flag, fortran support and some general notes about multi-compiler support

[Anton Starikov]

Olivier Fisette wrote:
> What about per package environment variable declarations? I have been 
> experiencing problems similar to Anton's: I compile my system using GCC, but 
> I use ICC for some packages I use in intensive calculations. To do so, I must 
> check every "world" update and manually set env variables such as CC. 
> Additionnally, I do not use the same "CFLAGS" for these packages as for the 
> rest of my system. I am not aware of any mechanism allowing me to automate 
> this process with the current Portage system.
The same tasks, the same problems :)

> What about having a file such as /etc/portage/package.env, that we could use 
> to set per package compilers, "CFLAGS" and such? It would be pretty nice to 
> be able to configure something along those lines:
> 
> =app-sci/emboss-2.9* CC="icc" CFLAGS="IWantFastSequenceAlignments"
> 
> ... and not have to worry about manually setting environment variables 
> whenever these packages are updated.
> This would certainly not solve all issues related to alternative compilers, 
> but it would make many things easier, and we could get rid of silly "USE" 
> flags like "icc", that are not related to the packages' content (the ebuilds 
> could simply check their environment and adapt the build process or report a 
> compiler incompatibility).
> 
> What do you think?
> 
Yes, it can be also a solution. Anyway, it will be nessecary to change 
slightly some ebuilds, but in my case you have to change anyway. This is 
even better sollution, but require changes in portage. I just didn't 
implement it because I don't want to patch each time portage :) But if 
it will be there "out of box" - that will be perfect.

Anton.



--
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] ifc USE flag, fortran support and some general notes about multi-compiler support

[Anton Starikov]

Donnie Berkholz wrote:
> You seem to be rejecting it out of hand
> rather than considering an expansion of its functionality. 
Sorry, that was my fault. Just misunderstanding.

Anton.




--
gentoo-dev@gentoo.org mailing list