[gentoo-dev] [RFC] new eselect module: compiler

[gentoo-dev] [RFC] new eselect module: compiler

[Lei Zhang]

[-- Attachment #1: Type: text/plain, Size: 1166 bytes --]

Hi,

I'm proposing to offer a new eselect module "compiler". Briefly
speaking, it manages /usr/bin/{cc,c++} as symlinks to C/C++ compilers
specified by the user.

This is related to my GSoC project: to use clang as the native
compiler in Gentoo. While putting "CC=clang" in make.defaults could
make portage use clang, symlinking cc/c++ to clang allows build tools
like GNU Make and CMake to also use clang as the default compiler,
making it feel more "native".

This module should be orthogonal to gcc-config. It just switches
between gcc, clang or whatever compiler, and doesn't care what version
of gcc is in use. Currently gcc, clang and icc are supported in my
implementation.

Another two programs that might be of interest are /usr/bin/{c89,c99}.
Currently they are just scripts that invoke gcc with proper "-std=..."
option. I'm not sure if any build tools use them instead of cc. In
that case, they probably should be handled by `eselect compiler` as
well.

A side note: actually there's a small issue with GNU make and I'm
trying to fix it. Please refer to
https://bugs.gentoo.org/show_bug.cgi?id=589894 and
https://github.com/gentoo/gentoo/pull/1982


Lei

[-- Attachment #2: compiler.eselect --]
[-- Type: application/octet-stream, Size: 2299 bytes --]

# -*-eselect-*-  vim: ft=eselect
# Copyright 2005-2016 Gentoo Foundation
# Distributed under the terms of the GNU GPL version 2 or later

DESCRIPTION="Manage system-wide default compiler"
MAINTAINER="zhanglei.april@gmail.com"

# Find a list of compilers installed on this host, in the form of their
# absolute paths.
find_compiler_paths() {
	for f in "${EROOT}"/usr/bin/gcc \
		 "${EROOT}"/usr/bin/clang \
		 "${EROOT}"/opt/intel/bin/icc; do
		[[ -f ${f} ]] && echo ${f}
	done
}

# symlink /usr/bin/cc to a given C compiler
symlink_cc() {
	ln -sf "${EROOT}"/$1 "${EROOT}"/usr/bin/cc
}

# symlink /usr/bin/c++ to a given C++ compiler
symlink_cxx() {
	ln -sf "${EROOT}"/$1 "${EROOT}"/usr/bin/c++
}

### list action ###

describe_list() {
	echo "List available compilers"
}

do_list() {
	local i paths=( $(find_compiler_paths) )

	write_list_start "Available compilers:"
	for (( i = 0; i < ${#paths[@]}; i++ )); do
		local compiler=$(basename ${paths[i]})

		diff "${EROOT}"/usr/bin/cc ${paths[i]} >/dev/null \
		&& compiler=$(highlight_marker ${compiler})

		write_numbered_list_entry $(expr $i + 1) "${compiler}"
	done
}

### set action ###

describe_set() {
	echo "Set the default compiler to use"
}

describe_set_parameters() {
	echo "<target>"
}

describe_set_options() {
	echo "target: Target name or number (from 'list' action)"
}

do_set() {
	[[ -z $1 ]] && die -q "Target not specified"
	[[ $# -gt 1 ]] && die -q "Too many arguments"

	local compiler
	if is_number $1; then
		local paths=( $(find_compiler_paths) )
		if [[ $1 -lt 1 ]] || [[ $1 -gt ${#paths[@]} ]]; then
			die -q "Invalid target"
		fi
		compiler=$(basename ${paths[$1-1]})
	else
		compiler=$1
	fi

	if [[ ${compiler} == gcc ]]; then
		symlink_cc /usr/bin/gcc
		symlink_cxx /usr/bin/g++
	elif [[ ${compiler} == clang ]]; then
		symlink_cc /usr/bin/clang
		symlink_cxx /usr/bin/clang++
	elif [[ ${compiler} == icc ]]; then
		symlink_cc /opt/intel/bin/icc
		symlink_cxx /opt/intel/bin/icpc
	else
		die -q "Invalid target"
	fi
}

### show action ###

describe_show() {
	echo "Show current default compiler in use"
}

do_show() {
	write_list_start "Current default compiler:"
	for path in $(find_compiler_paths); do
		diff "${EROOT}"/usr/bin/cc ${path} >/dev/null \
		&& write_kv_list_entry $(basename ${path}) "" && break
	done
}

Re: [gentoo-dev] [RFC] new eselect module: compiler

[James Le Cuirot]

[-- Attachment #1: Type: text/plain, Size: 725 bytes --]

On Thu, 28 Jul 2016 13:16:45 +0800
Lei Zhang <zhanglei.april@gmail.com> wrote:

> I'm proposing to offer a new eselect module "compiler". Briefly
> speaking, it manages /usr/bin/{cc,c++} as symlinks to C/C++ compilers
> specified by the user.

I'm not on the toolchain team so I cannot comment much but be aware
that there was previously an eselect module called "compiler". It was
removed in 2007 because it did silly things. See this bug.

https://bugs.gentoo.org/show_bug.cgi?id=199914

I'm not suggesting that your version does silly things. :) I don't know 
how practical is it to swap "cc" with something other than gcc as I've
never tried clang.

-- 
James Le Cuirot (chewi)
Gentoo Linux Developer

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 951 bytes --]

Re: [gentoo-dev] [RFC] new eselect module: compiler

[Lei Zhang]

2016-08-06 21:45 GMT+08:00 James Le Cuirot <chewi@gentoo.org>:
> On Thu, 28 Jul 2016 13:16:45 +0800
> Lei Zhang <zhanglei.april@gmail.com> wrote:
>
>> I'm proposing to offer a new eselect module "compiler". Briefly
>> speaking, it manages /usr/bin/{cc,c++} as symlinks to C/C++ compilers
>> specified by the user.
>
> I'm not on the toolchain team so I cannot comment much but be aware
> that there was previously an eselect module called "compiler". It was
> removed in 2007 because it did silly things. See this bug.
>
> https://bugs.gentoo.org/show_bug.cgi?id=199914
>
> I'm not suggesting that your version does silly things. :) I don't know
> how practical is it to swap "cc" with something other than gcc as I've
> never tried clang.

"cc" is the standard C compiler name defined in POSIX, so ideally any
gcc-agnostic programs should use "cc" instead of "gcc". Practically,
build tools like GNU Make and CMake would be affected as they use "cc"
implicitly.

OTOH, POSIX doesn't require the presence of C++ compiler on a
conformant system; "c++' is just a de facto standard.


Lei

Re: [gentoo-dev] [RFC] new eselect module: compiler

[R0b0t1]

On Mon, Aug 8, 2016 at 11:23 AM, Lei Zhang <zhanglei.april@gmail.com> wrote:
> "cc" is the standard C compiler name defined in POSIX, so ideally any
> gcc-agnostic programs should use "cc" instead of "gcc". Practically,
> build tools like GNU Make and CMake would be affected as they use "cc"
> implicitly.

It is not just programs which rely on GNU extensions, but poorly
created scripts that rely on a compiler directly or otherwise break
portability.

Re: [gentoo-dev] [RFC] new eselect module: compiler

[Fabian Groffen]

[-- Attachment #1: Type: text/plain, Size: 1094 bytes --]

On 08-08-2016 13:45:07 -0500, R0b0t1 wrote:
> On Mon, Aug 8, 2016 at 11:23 AM, Lei Zhang <zhanglei.april@gmail.com> wrote:
> > "cc" is the standard C compiler name defined in POSIX, so ideally any
> > gcc-agnostic programs should use "cc" instead of "gcc". Practically,
> > build tools like GNU Make and CMake would be affected as they use "cc"
> > implicitly.
> 
> It is not just programs which rely on GNU extensions, but poorly
> created scripts that rely on a compiler directly or otherwise break
> portability.

I'd agree and say "gcc" is hardcoded in many places, that's why I
believe Apple includes a gcc which is clang on their systems, same for
cc.

As a question to Lei, I'm wondering why you chose eselect compiler, and
not gcc-config to manage the links.  In a way, gcc-config is tailored
towards gcc, but it does a lot of things also for the environment.  With
clang, from my experience, you just want it as drop-in replacement for
gcc as it doesn't give you too much issues (on Darwin at least).

Fabian


-- 
Fabian Groffen
Gentoo on a different level

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 181 bytes --]

Re: [gentoo-dev] [RFC] new eselect module: compiler

[james]

On 08/09/2016 12:58 AM, Fabian Groffen wrote:
> On 08-08-2016 13:45:07 -0500, R0b0t1 wrote:
>> On Mon, Aug 8, 2016 at 11:23 AM, Lei Zhang <zhanglei.april@gmail.com> wrote:
>>> "cc" is the standard C compiler name defined in POSIX, so ideally any
>>> gcc-agnostic programs should use "cc" instead of "gcc". Practically,
>>> build tools like GNU Make and CMake would be affected as they use "cc"
>>> implicitly.
>>
>> It is not just programs which rely on GNU extensions, but poorly
>> created scripts that rely on a compiler directly or otherwise break
>> portability.
>
> I'd agree and say "gcc" is hardcoded in many places, that's why I
> believe Apple includes a gcc which is clang on their systems, same for
> cc.
>
> As a question to Lei, I'm wondering why you chose eselect compiler, and
> not gcc-config to manage the links.  In a way, gcc-config is tailored
> towards gcc, but it does a lot of things also for the environment.  With
> clang, from my experience, you just want it as drop-in replacement for
> gcc as it doesn't give you too much issues (on Darwin at least).
>
> Fabian
>
>

There are many things afoot with compilers, particularly related to 
distributed and tightly coupled parallel systems. Perhaps a name change
from gcc-config to cc-config would be more accurate or better if that
aspect of choice is to accurately reflect the choices going forward?

Also, when you get down to smaller microprocessor levels there are often 
numerous choices for compilers. Granted, today, most of those are still 
commercial, but, pressure over time could very likely see many of those 
compilers going the open source route, confounding the choice issue for 
a more open naming convention for gcc-config?


hth,
James

Re: [gentoo-dev] [RFC] new eselect module: compiler

[Zac Medico]

On 08/08/2016 10:58 PM, Fabian Groffen wrote:
> On 08-08-2016 13:45:07 -0500, R0b0t1 wrote:
>> On Mon, Aug 8, 2016 at 11:23 AM, Lei Zhang <zhanglei.april@gmail.com> wrote:
>>> "cc" is the standard C compiler name defined in POSIX, so ideally any
>>> gcc-agnostic programs should use "cc" instead of "gcc". Practically,
>>> build tools like GNU Make and CMake would be affected as they use "cc"
>>> implicitly.
>>
>> It is not just programs which rely on GNU extensions, but poorly
>> created scripts that rely on a compiler directly or otherwise break
>> portability.
> 
> I'd agree and say "gcc" is hardcoded in many places, that's why I
> believe Apple includes a gcc which is clang on their systems, same for
> cc.
> 
> As a question to Lei, I'm wondering why you chose eselect compiler, and
> not gcc-config to manage the links.  In a way, gcc-config is tailored
> towards gcc, but it does a lot of things also for the environment.  With
> clang, from my experience, you just want it as drop-in replacement for
> gcc as it doesn't give you too much issues (on Darwin at least).
> 
> Fabian
> 
> 

I'm guessing you Darwin folks don't compile many Linux kernels? The last
I heard, clang is not yet a drop-in replacement for gcc when building
the Linux kernel:

    http://llvm.linuxfoundation.org/index.php/Main_Page
-- 
Thanks,
Zac

Re: [gentoo-dev] [RFC] new eselect module: compiler

[Lei Zhang]

2016-08-09 13:58 GMT+08:00 Fabian Groffen <grobian@gentoo.org>:
> As a question to Lei, I'm wondering why you chose eselect compiler, and
> not gcc-config to manage the links.  In a way, gcc-config is tailored
> towards gcc, but it does a lot of things also for the environment.  With
> clang, from my experience, you just want it as drop-in replacement for
> gcc as it doesn't give you too much issues (on Darwin at least).

In its current form, gcc-config specializes in handling different
versions of gcc. If we extend it to cover other compilers (and rename
it to cc-config as James suggested), should it handle different
versions of clang? What about different versions of icc?

I'm just afraid gcc-config would become too complex that way, so I
prefer a simpler approach: let eselect-compiler be version-agnostic.
Then we can have clang-config to handle the versioning of clang,
icc-config to handle icc, etc.


Lei

Re: [gentoo-dev] [RFC] new eselect module: compiler

[M. J. Everitt]

[-- Attachment #1: Type: text/plain, Size: 1234 bytes --]

On 10/08/16 01:39, Lei Zhang wrote:
> 2016-08-09 13:58 GMT+08:00 Fabian Groffen <grobian@gentoo.org>:
>> As a question to Lei, I'm wondering why you chose eselect compiler, and
>> not gcc-config to manage the links.  In a way, gcc-config is tailored
>> towards gcc, but it does a lot of things also for the environment.  With
>> clang, from my experience, you just want it as drop-in replacement for
>> gcc as it doesn't give you too much issues (on Darwin at least).
> In its current form, gcc-config specializes in handling different
> versions of gcc. If we extend it to cover other compilers (and rename
> it to cc-config as James suggested), should it handle different
> versions of clang? What about different versions of icc?
>
> I'm just afraid gcc-config would become too complex that way, so I
> prefer a simpler approach: let eselect-compiler be version-agnostic.
> Then we can have clang-config to handle the versioning of clang,
> icc-config to handle icc, etc.
>
>
> Lei
>
Extending the ideas presented in this thread .. you could introduce
cc-config, and which utility script it runs would then be governed by
eselect compiler .. eg. gcc would have gcc-config, clang would run
clang-config ..


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 901 bytes --]

Re: [gentoo-dev] [RFC] new eselect module: compiler

[Lei Zhang]

2016-08-10 0:16 GMT+08:00 Zac Medico <zmedico@gentoo.org>:
> I'm guessing you Darwin folks don't compile many Linux kernels? The last
> I heard, clang is not yet a drop-in replacement for gcc when building
> the Linux kernel:
>
>     http://llvm.linuxfoundation.org/index.php/Main_Page

Yes, Linux kernel has to be built by gcc at the moment. Last time I
checked it, the kernel source is hardcoded to use gcc, so changing cc
won't affect it.

Other than the kernel, non-gcc compilers should work just fine in
userland. FWIW, I've tried rebuilding @world with clang on my host;
only two packages failed and they are very easy to fix [1].


[1] https://blogs.gentoo.org/gsoc2016-native-clang/2016/07/24/a-new-gentoo-stage4-musl-clang/

Lei

Re: [gentoo-dev] [RFC] new eselect module: compiler

[Benda Xu]

Lei Zhang <zhanglei.april@gmail.com> writes:

> I'm just afraid gcc-config would become too complex that way, so I
> prefer a simpler approach: let eselect-compiler be version-agnostic.
> Then we can have clang-config to handle the versioning of clang,
> icc-config to handle icc, etc.

If we model after eselect python here, `eselect python` governs versions
of python as well as pypy.

Benda

Re: [gentoo-dev] [RFC] new eselect module: compiler

[Michał Górny]

[-- Attachment #1: Type: text/plain, Size: 1508 bytes --]

On Wed, 10 Aug 2016 01:52:29 +0100
"M. J. Everitt" <m.j.everitt@iee.org> wrote:

> On 10/08/16 01:39, Lei Zhang wrote:
> > 2016-08-09 13:58 GMT+08:00 Fabian Groffen <grobian@gentoo.org>:
> >> As a question to Lei, I'm wondering why you chose eselect compiler, and
> >> not gcc-config to manage the links.  In a way, gcc-config is tailored
> >> towards gcc, but it does a lot of things also for the environment.  With
> >> clang, from my experience, you just want it as drop-in replacement for
> >> gcc as it doesn't give you too much issues (on Darwin at least).
> > In its current form, gcc-config specializes in handling different
> > versions of gcc. If we extend it to cover other compilers (and rename
> > it to cc-config as James suggested), should it handle different
> > versions of clang? What about different versions of icc?
> >
> > I'm just afraid gcc-config would become too complex that way, so I
> > prefer a simpler approach: let eselect-compiler be version-agnostic.
> > Then we can have clang-config to handle the versioning of clang,
> > icc-config to handle icc, etc.
> >
> >
> > Lei
> >
> Extending the ideas presented in this thread .. you could introduce
> cc-config, and which utility script it runs would then be governed by
> eselect compiler .. eg. gcc would have gcc-config, clang would run
> clang-config ..

.. to switch between the one version of clang that can be installed?

-- 
Best regards,
Michał Górny
<http://dev.gentoo.org/~mgorny/>

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 949 bytes --]

Re: [gentoo-dev] [RFC] new eselect module: compiler

[M. J. Everitt]

[-- Attachment #1: Type: text/plain, Size: 1597 bytes --]

On 10/08/16 06:08, Michał Górny wrote:
> On Wed, 10 Aug 2016 01:52:29 +0100
> "M. J. Everitt" <m.j.everitt@iee.org> wrote:
>
>> On 10/08/16 01:39, Lei Zhang wrote:
>>> 2016-08-09 13:58 GMT+08:00 Fabian Groffen <grobian@gentoo.org>:
>>>> As a question to Lei, I'm wondering why you chose eselect compiler, and
>>>> not gcc-config to manage the links.  In a way, gcc-config is tailored
>>>> towards gcc, but it does a lot of things also for the environment.  With
>>>> clang, from my experience, you just want it as drop-in replacement for
>>>> gcc as it doesn't give you too much issues (on Darwin at least).
>>> In its current form, gcc-config specializes in handling different
>>> versions of gcc. If we extend it to cover other compilers (and rename
>>> it to cc-config as James suggested), should it handle different
>>> versions of clang? What about different versions of icc?
>>>
>>> I'm just afraid gcc-config would become too complex that way, so I
>>> prefer a simpler approach: let eselect-compiler be version-agnostic.
>>> Then we can have clang-config to handle the versioning of clang,
>>> icc-config to handle icc, etc.
>>>
>>>
>>> Lei
>>>
>> Extending the ideas presented in this thread .. you could introduce
>> cc-config, and which utility script it runs would then be governed by
>> eselect compiler .. eg. gcc would have gcc-config, clang would run
>> clang-config ..
> .. to switch between the one version of clang that can be installed?
>
Tis early days Mr Gorny .. who knows what the future holds .. and Gentoo
is all about choice, right?!


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 901 bytes --]

Re: [gentoo-dev] [RFC] new eselect module: compiler

[Fabian Groffen]

[-- Attachment #1: Type: text/plain, Size: 1622 bytes --]

On 10-08-2016 08:39:26 +0800, Lei Zhang wrote:
> 2016-08-09 13:58 GMT+08:00 Fabian Groffen <grobian@gentoo.org>:
> > As a question to Lei, I'm wondering why you chose eselect compiler, and
> > not gcc-config to manage the links.  In a way, gcc-config is tailored
> > towards gcc, but it does a lot of things also for the environment.  With
> > clang, from my experience, you just want it as drop-in replacement for
> > gcc as it doesn't give you too much issues (on Darwin at least).
> 
> In its current form, gcc-config specializes in handling different
> versions of gcc. If we extend it to cover other compilers (and rename
> it to cc-config as James suggested), should it handle different
> versions of clang? What about different versions of icc?
> 
> I'm just afraid gcc-config would become too complex that way, so I
> prefer a simpler approach: let eselect-compiler be version-agnostic.
> Then we can have clang-config to handle the versioning of clang,
> icc-config to handle icc, etc.

Alright.  As Zac pointed out, my gcc-config idea was a bad one.  And it
wouldn't work very well with icc and many other compilers as well.

Reason I thought about it, is that I'd like to avoid setting
CC/CXX/OBJC/OBJCXX/BUILD_CC/BUILD_CXX and whatnot.  Currently,
toolchain-funcs' tc-getPROG does some magic to produce TRIPLET-PROG kind
of thing, and configure gets the triplet passed by Portage as well.  I
was hoping for it to find TRIPLET-clang at some point with all the
tooling in place without "hard-overriding" CC in make.conf.

Fabian


-- 
Fabian Groffen
Gentoo on a different level

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 181 bytes --]