[gentoo-dev] New USE=-native-symlinks for gcc-config and binutils-config

[gentoo-dev] New USE=-native-symlinks for gcc-config and binutils-config

[Sergei Trofimovich]

'tc-directly' tracker https://bugs.gentoo.org/243502 tracks
packages that don't respect users' CC/AR/LD flags.

I added new USE=-native-symlinks mode for gcc-config and
binutils-config to ease detection of such packages.

Native symlinks are still installed by default. Nothing should
break for users who use default USE flags.

USE=-native-symlinks removes a bunch of links that most packages
use by default until are overridden explicitly. Incomplete list is:
- /lib/cpp
- /usr/bin/{gcc,cc,g++,c++,...}
- /usr/bin/{as,ld,ranlib,dwp,...}

The rule of thumb is: if a tool does not have ${CTARGET}- prefix
it will probably disappear with USE=-native-symlinks.

(At least eventually) 'emerge' should still be able to build most
of packages in such environment. I expect initial breakage will be
huge though.

Using './configure && make && make install' style workflow will be more
tedious. One workaround at least for gcc is to use something like:
    $ PATH="$(gcc-config -B):$PATH"
It's not perfect. We'll see if toolchain can provide nicer environment.

Typical fixes for autoconf style build systems is to use macros like:
- AC_PROG_CC
- AM_PROG_AR
- AC_CHECK_TOOL(DWP, dwp)
and so on to detect tool that corresponds to --host=/--target= flags
and allows user's override via environment variable.

-- 

  Sergei

Re: [gentoo-dev] New USE=-native-symlinks for gcc-config and binutils-config

[Michał Górny]

[-- Attachment #1: Type: text/plain, Size: 987 bytes --]

On Fri, 2020-05-22 at 22:36 +0100, Sergei Trofimovich wrote:
> 'tc-directly' tracker https://bugs.gentoo.org/243502 tracks
> packages that don't respect users' CC/AR/LD flags.
> 
> I added new USE=-native-symlinks mode for gcc-config and
> binutils-config to ease detection of such packages.
> 
> Native symlinks are still installed by default. Nothing should
> break for users who use default USE flags.
> 
> USE=-native-symlinks removes a bunch of links that most packages
> use by default until are overridden explicitly. Incomplete list is:
> - /lib/cpp
> - /usr/bin/{gcc,cc,g++,c++,...}
> - /usr/bin/{as,ld,ranlib,dwp,...}
> 
> The rule of thumb is: if a tool does not have ${CTARGET}- prefix
> it will probably disappear with USE=-native-symlinks.

Does this list include 'ar' or not?  Asking because there's been
a number of false positives reported for 'ar' being used as an archiver
(e.g. to work on .deb packages).

-- 
Best regards,
Michał Górny


[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 618 bytes --]

Re: [gentoo-dev] New USE=-native-symlinks for gcc-config and binutils-config

[Sergei Trofimovich]

On Sat, 23 May 2020 08:05:46 +0200
Michał Górny <mgorny@gentoo.org> wrote:

> On Fri, 2020-05-22 at 22:36 +0100, Sergei Trofimovich wrote:
> > 'tc-directly' tracker https://bugs.gentoo.org/243502 tracks
> > packages that don't respect users' CC/AR/LD flags.
> > 
> > I added new USE=-native-symlinks mode for gcc-config and
> > binutils-config to ease detection of such packages.
> > 
> > Native symlinks are still installed by default. Nothing should
> > break for users who use default USE flags.
> > 
> > USE=-native-symlinks removes a bunch of links that most packages
> > use by default until are overridden explicitly. Incomplete list is:
> > - /lib/cpp
> > - /usr/bin/{gcc,cc,g++,c++,...}
> > - /usr/bin/{as,ld,ranlib,dwp,...}
> > 
> > The rule of thumb is: if a tool does not have ${CTARGET}- prefix
> > it will probably disappear with USE=-native-symlinks.  
> 
> Does this list include 'ar' or not?  Asking because there's been
> a number of false positives reported for 'ar' being used as an archiver
> (e.g. to work on .deb packages).

USE=-native-symlinks does remove /usr/bin/ar. Full(er) list for binutils is:

addr2line
ar
as
c++filt
coffdump
dlltool
dllwrap
dwp
elfedit
gprof
ld
ld.bfd
ld.gold
nm
objcopy
objdump
ranlib
readelf
size
srconv
strings
strip
sysdump
windmc
windres

-- 

  Sergei

RE: [gentoo-dev] New USE=-native-symlinks for gcc-config and binutils-config

[Pengcheng Xu]

[-- Attachment #1: Type: text/plain, Size: 1051 bytes --]

> USE=-native-symlinks removes a bunch of links that most packages use by default
> until are overridden explicitly. Incomplete list is:
> - /lib/cpp
> - /usr/bin/{gcc,cc,g++,c++,...}
> - /usr/bin/{as,ld,ranlib,dwp,...}
> 
> The rule of thumb is: if a tool does not have ${CTARGET}- prefix it will probably
> disappear with USE=-native-symlinks.
> 
> (At least eventually) 'emerge' should still be able to build most of packages
> in such environment. I expect initial breakage will be huge though.
> 
> Using './configure && make && make install' style workflow will be more tedious.
> One workaround at least for gcc is to use something like:
>     $ PATH="$(gcc-config -B):$PATH"
> It's not perfect. We'll see if toolchain can provide nicer environment.
> 

Do we currently have (or is there a plan for) a mechanism to manage the symbolic links and/or create them after merging the package when necessary?  It's quite tiresome to type in $CHOST-gcc for simple everyday tasks.

Regards,
-- 
Pengcheng Xu
https://jsteward.moe

[-- Attachment #2: openpgp-digital-signature.asc --]
[-- Type: application/pgp-signature, Size: 834 bytes --]

Re: [gentoo-dev] New USE=-native-symlinks for gcc-config and binutils-config

[Mike Gilbert]

On Fri, May 22, 2020 at 5:36 PM Sergei Trofimovich <slyfox@gentoo.org> wrote:
>
> 'tc-directly' tracker https://bugs.gentoo.org/243502 tracks
> packages that don't respect users' CC/AR/LD flags.
>
> I added new USE=-native-symlinks mode for gcc-config and
> binutils-config to ease detection of such packages.
>
> Native symlinks are still installed by default. Nothing should
> break for users who use default USE flags.
>
> USE=-native-symlinks removes a bunch of links that most packages
> use by default until are overridden explicitly. Incomplete list is:
> - /lib/cpp
> - /usr/bin/{gcc,cc,g++,c++,...}
> - /usr/bin/{as,ld,ranlib,dwp,...}
>
> The rule of thumb is: if a tool does not have ${CTARGET}- prefix
> it will probably disappear with USE=-native-symlinks.
>
> (At least eventually) 'emerge' should still be able to build most
> of packages in such environment. I expect initial breakage will be
> huge though.

Could you please add this flag to package.use.force? I don't think we
want to give people the impression that this is a well-supported
configuration. I would only expect people to disable this if they want
to break their system intentionally.

Also, people are likely to disable this accidentally via USE="-*".

Re: [gentoo-dev] New USE=-native-symlinks for gcc-config and binutils-config

[Joonas Niilola]

[-- Attachment #1.1: Type: text/plain, Size: 186 bytes --]


On 5/24/20 5:41 AM, Mike Gilbert wrote:
> Also, people are likely to disable this accidentally via USE="-*".

Counts as

> if they want to break their system intentionally.
>


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 642 bytes --]

Re: [gentoo-dev] New USE=-native-symlinks for gcc-config and binutils-config

[Matt Turner]

On Sat, May 23, 2020 at 10:21 PM Joonas Niilola <juippis@gentoo.org> wrote:
>
>
> On 5/24/20 5:41 AM, Mike Gilbert wrote:
> > Also, people are likely to disable this accidentally via USE="-*".
>
> Counts as
>
> > if they want to break their system intentionally.

Yes, but unfortunately catalyst's stage1 build does exactly this.

Suggestions how to avoid doing this would be appreciated, but until
that's resolved we don't have carte blanche to break USE="-*".

Re: [gentoo-dev] New USE=-native-symlinks for gcc-config and binutils-config

[Kent Fredric]

[-- Attachment #1: Type: text/plain, Size: 636 bytes --]

On Sun, 24 May 2020 09:40:50 +0800
"Pengcheng Xu" <i@jsteward.moe> wrote:

> Do we currently have (or is there a plan for) a mechanism to manage the symbolic links and/or create them after merging the package when necessary?  It's quite tiresome to type in $CHOST-gcc for simple everyday tasks.

There are (currently undocumented) methods that work regardless of the
USE flag:

 {binutils,gcc}-config <profile> --enable-native-links
 {binutils,gcc}-config <profile> --disable-native-links

All the use flag does is dictate which of those "--<x>-native-links"
behaviours occur when no "--<x>-native-links" flags are passed.

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [gentoo-dev] New USE=-native-symlinks for gcc-config and binutils-config

[Kent Fredric]

[-- Attachment #1: Type: text/plain, Size: 478 bytes --]

On Sat, 23 May 2020 22:41:02 -0400
Mike Gilbert <floppym@gentoo.org> wrote:

> Could you please add this flag to package.use.force? I don't think we

Sounds more like a case for package.use.stable.force or something.

For non-stable, we don't give guarantees about well-supported, only
that there will be bugs, and they should probably be reported.

( It doesn't tend to 'break' your system, it just makes upgrades fail,
runtime itself seems unaffected in general )

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [gentoo-dev] New USE=-native-symlinks for gcc-config and binutils-config

[Michał Górny]

[-- Attachment #1: Type: text/plain, Size: 727 bytes --]

On Sun, 2020-05-24 at 20:15 +1200, Kent Fredric wrote:
> On Sat, 23 May 2020 22:41:02 -0400
> Mike Gilbert <floppym@gentoo.org> wrote:
> 
> > Could you please add this flag to package.use.force? I don't think we
> 
> Sounds more like a case for package.use.stable.force or something.
> 
> For non-stable, we don't give guarantees about well-supported, only
> that there will be bugs, and they should probably be reported.
> 
> ( It doesn't tend to 'break' your system, it just makes upgrades fail,
> runtime itself seems unaffected in general )

If it creates an invalid environment that is known to break packages
and is not QA-sanctioned, it should be masked.  Period.

-- 
Best regards,
Michał Górny


[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 618 bytes --]

Re: [gentoo-dev] New USE=-native-symlinks for gcc-config and binutils-config

[Kent Fredric]

[-- Attachment #1: Type: text/plain, Size: 502 bytes --]

On Sun, 24 May 2020 10:33:18 +0200
Michał Górny <mgorny@gentoo.org> wrote:

> If it creates an invalid environment that is known to break packages
> and is not QA-sanctioned, it should be masked.  Period.


Seems like yet another argument in favour of my initial position, that
it probably shouldn't be controlled by a USE flag, and should *only* be
controlled by the tool itself via local config persistence. (Where
presently, there's no config persistence, the USE flag is all there is)

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [gentoo-dev] New USE=-native-symlinks for gcc-config and binutils-config

[Sergei Trofimovich]

On Sat, 23 May 2020 23:40:22 -0700
Matt Turner <mattst88@gentoo.org> wrote:

> On Sat, May 23, 2020 at 10:21 PM Joonas Niilola <juippis@gentoo.org> wrote:
> >
> >
> > On 5/24/20 5:41 AM, Mike Gilbert wrote:  
> > > Also, people are likely to disable this accidentally via USE="-*".  
> >
> > Counts as
> >  
> > > if they want to break their system intentionally.  
> 
> Yes, but unfortunately catalyst's stage1 build does exactly this.
> 
> Suggestions how to avoid doing this would be appreciated, but until
> that's resolved we don't have carte blanche to break USE="-*".

Ah, I keep forgetting about catalyst. Use-forced flags as:
    https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=0f8b5b847429642977d4c0497068a93222ff3136

While at it added a page that explains how to turn it back by enthusiasts:
    https://wiki.gentoo.org/wiki/Project:Toolchain/use_native_symlinks
and will collect more details on typical fixes and "this is fine to ignore" pitfalls.

-- 

  Sergei

Re: [gentoo-dev] New USE=-native-symlinks for gcc-config and binutils-config

[Sergei Trofimovich]

On Sat, 23 May 2020 22:41:02 -0400
Mike Gilbert <floppym@gentoo.org> wrote:

> On Fri, May 22, 2020 at 5:36 PM Sergei Trofimovich <slyfox@gentoo.org> wrote:
> >
> > 'tc-directly' tracker https://bugs.gentoo.org/243502 tracks
> > packages that don't respect users' CC/AR/LD flags.
> >
> > I added new USE=-native-symlinks mode for gcc-config and
> > binutils-config to ease detection of such packages.
> >
> > Native symlinks are still installed by default. Nothing should
> > break for users who use default USE flags.
> >
> > USE=-native-symlinks removes a bunch of links that most packages
> > use by default until are overridden explicitly. Incomplete list is:
> > - /lib/cpp
> > - /usr/bin/{gcc,cc,g++,c++,...}
> > - /usr/bin/{as,ld,ranlib,dwp,...}
> >
> > The rule of thumb is: if a tool does not have ${CTARGET}- prefix
> > it will probably disappear with USE=-native-symlinks.
> >
> > (At least eventually) 'emerge' should still be able to build most
> > of packages in such environment. I expect initial breakage will be
> > huge though.  
> 
> Could you please add this flag to package.use.force?

Added as https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=0f8b5b847429642977d4c0497068a93222ff3136

> I don't think we
> want to give people the impression that this is a well-supported
> configuration. I would only expect people to disable this if they want
> to break their system intentionally.

Yeah, today it's certainly a way to get your system in a miserable state.
My hope is that USE=-native-symlinks can get you a working Gentoo in a
near future by only fixing real package problems and limitations of their
build systems.

-- 

  Sergei

Re: [gentoo-dev] New USE=-native-symlinks for gcc-config and binutils-config

[Fabian Groffen]

[-- Attachment #1: Type: text/plain, Size: 1258 bytes --]

On 24-05-2020 10:48:47 +0100, Sergei Trofimovich wrote:
> On Sat, 23 May 2020 22:41:02 -0400
> Mike Gilbert <floppym@gentoo.org> wrote:
> > I don't think we
> > want to give people the impression that this is a well-supported
> > configuration. I would only expect people to disable this if they want
> > to break their system intentionally.
> 
> Yeah, today it's certainly a way to get your system in a miserable state.
> My hope is that USE=-native-symlinks can get you a working Gentoo in a
> near future by only fixing real package problems and limitations of their
> build systems.

Portage adds PREROOTPATH, ROOTPATH and a standard set of
/usr/{local/,}{s,}bin to PATH in _doebuild_path.  That means if you add
something like /usr/bin/native-toolchain to PATH only, users will get
gcc, ld, etc., while root and Portage will lack these.  One can wonder
if root should have direct access to compilers, but it's easy enough to
add the compilers to PATH if really necessary.

I guess what I'm trying to say is: you can hide effect of the setup for
users if you'd like.  That is, after we had buildbots point out the bulk
of packages that are wrong of course.

Thanks,
Fabian
 

-- 
Fabian Groffen
Gentoo on a different level

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

Re: [gentoo-dev] New USE=-native-symlinks for gcc-config and binutils-config

[Sergei Trofimovich]

On Sun, 24 May 2020 09:40:50 +0800
"Pengcheng Xu" <i@jsteward.moe> wrote:

> > USE=-native-symlinks removes a bunch of links that most packages use by default
> > until are overridden explicitly. Incomplete list is:
> > - /lib/cpp
> > - /usr/bin/{gcc,cc,g++,c++,...}
> > - /usr/bin/{as,ld,ranlib,dwp,...}
> > 
> > The rule of thumb is: if a tool does not have ${CTARGET}- prefix it will probably
> > disappear with USE=-native-symlinks.
> > 
> > (At least eventually) 'emerge' should still be able to build most of packages
> > in such environment. I expect initial breakage will be huge though.
> > 
> > Using './configure && make && make install' style workflow will be more tedious.
> > One workaround at least for gcc is to use something like:
> >     $ PATH="$(gcc-config -B):$PATH"
> > It's not perfect. We'll see if toolchain can provide nicer environment.
> >   
> 
> Do we currently have (or is there a plan for) a mechanism to manage the symbolic links and/or create them after merging the package when necessary?  It's quite tiresome to type in $CHOST-gcc for simple everyday tasks.

There currently is no nice way to get stable path with up-to-date
symlinks for current gcc/binutils. I think of adding a gentoo-specific
directory to manage symlinks similar to what 'gcc-config -B' provides
in a stable path that you can write once into user's PATH.

No concrete implementation yet. Filed https://bugs.gentoo.org/724980
to track it.

-- 

  Sergei