[gentoo-dev] glibc-2.26 and changes with SunRPC, libtirpc, ntirpc, libnsl (NIS and friends), ...

[gentoo-dev] glibc-2.26 and changes with SunRPC, libtirpc, ntirpc, libnsl (NIS and friends), ...

[Andreas K. Huettel]

[-- Attachment #1: Type: text/plain, Size: 2958 bytes --]

So glibc-2.26 is already out for some time, but we still haven't keyworded it 
yet. Why?

* I want to use the opportunity to make the long-delayed switchover from 
glibc-internal SunRPC (long deprecated and outdated) to external 
implementations (libtirpc, and possibly ntirpc).

* The (outdated and deprecated) NIS(YP) and NIS+ support (libnsl) has been 
removed from glibc (except for a compatibility library that doesnt install 
headers), and is now provided by net-libs/libnsl (increased soversion).

This mail is mainly about how to best structure the transition.
Comments, suggestions, corrections, feedback welcome.

1) About RPC. 

AFACIS there are three implementations:
a) SunRPC, headers in /usr/include, code provided by glibc
b) net-libs/libtirpc, headers in /usr/include/libtirpc, library -l tirpc
c) (?) net-libs/ntirpc, headers in /usr/include/ntirpc, library -l ntirpc

Option a) is going away with sys-libs/glibc-2.26-r1. 
Options b) and c) may in addition need headers from net-libs/rpcsvc-proto
I haven't done any testing with c) yet, will do so.
a), b), and c) are co-installable.

My suggestion for an ideal implementation would be that any package that uses 
RPC defines useflags:
sunrpc - build against glibc
libtirpc - build against net-libs/libtirpc
ntirpc - build against net-libs/ntirpc
with 
REQUIRED_USE="^^ ( sunrpc libtirpc ntirpc )"
If rpc support is optional with useflag rpc, then this becomes
REQUIRED_USE="rpc? ( ^^ ( sunrpc libtirpc ntirpc ) )"

Since the three options are coinstallable I see no problems with a package 
only supporting a subset, but I have no clue how this interacts at runtime.

Of course this "ideal option" is also the most work-intensive.

Both libtirpc and ntirpc supply a packageconfig file. Porting a package means 
pointing it to the right include directory (at compile time) and library name 
(at link time); if that's not done, a build fails because the rpc headers 
cannot be found.
(In my @system chroot, python fails atm.)

2) About YP / NIS / NIS+.

a) The old libnsl implementation is provided by glibc, soversion 1. 
b) An updated and much improved implementation is provided by net-libs/libnsl, 
soversion 2.

glibc-2.26 installs only the library for a), and no headers.
Since I dont want to mess with currently used glibc ebuilds, net-libs/libnsl 
requires at least our glibc-2.26 (otherwise you get file collisions).

Porting a package means adding a dependency in the style of 
|| ( <sys-libs/glibc-2.26 net-libs/libnsl )

It may not always be obvious where this is needed, since net-libs/libnsl is 
already pulled in deep in the dependency tree (my @system chroot has it).


So... that's it for the moment.
Comments, ideas?

Cheers, 
Andreas 
(off for a swim now... https://www.instagram.com/p/BZLJn1qF09B/ )


-- 
Andreas K. Hüttel
dilfridge@gentoo.org
Gentoo Linux developer (council, perl, libreoffice)

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [gentoo-dev] glibc-2.26 and changes with SunRPC, libtirpc, ntirpc, libnsl (NIS and friends), ...

[M. J. Everitt]

[-- Attachment #1.1: Type: text/plain, Size: 1980 bytes --]

On 18/09/17 10:56, Andreas K. Huettel wrote:
> So glibc-2.26 is already out for some time, but we still haven't keyworded it 
> yet. Why?
>
> * I want to use the opportunity to make the long-delayed switchover from 
> glibc-internal SunRPC (long deprecated and outdated) to external 
> implementations (libtirpc, and possibly ntirpc).
>
> * The (outdated and deprecated) NIS(YP) and NIS+ support (libnsl) has been 
> removed from glibc (except for a compatibility library that doesnt install 
> headers), and is now provided by net-libs/libnsl (increased soversion).
>
> This mail is mainly about how to best structure the transition.
> Comments, suggestions, corrections, feedback welcome.
>
> 1) About RPC. 
>
> AFACIS there are three implementations:
> a) SunRPC, headers in /usr/include, code provided by glibc
> b) net-libs/libtirpc, headers in /usr/include/libtirpc, library -l tirpc
> c) (?) net-libs/ntirpc, headers in /usr/include/ntirpc, library -l ntirpc
>
> Option a) is going away with sys-libs/glibc-2.26-r1. 
> Options b) and c) may in addition need headers from net-libs/rpcsvc-proto
> I haven't done any testing with c) yet, will do so.
> a), b), and c) are co-installable.
>
> My suggestion for an ideal implementation would be that any package that uses 
> RPC defines useflags:
> sunrpc - build against glibc
> libtirpc - build against net-libs/libtirpc
> ntirpc - build against net-libs/ntirpc
> with 
> REQUIRED_USE="^^ ( sunrpc libtirpc ntirpc )"
> If rpc support is optional with useflag rpc, then this becomes
> REQUIRED_USE="rpc? ( ^^ ( sunrpc libtirpc ntirpc ) )"
>
> Since the three options are coinstallable I see no problems with a package 
> only supporting a subset, but I have no clue how this interacts at runtime.
>
> Of course this "ideal option" is also the most work-intensive.
<snip>

Would a virtual help any? Probably overlooking a good number of factors,
but wasn't mentioned yet ...

MJE


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 819 bytes --]

Re: [gentoo-dev] glibc-2.26 and changes with SunRPC, libtirpc, ntirpc, libnsl (NIS and friends), ...

[Andreas K. Huettel]

Am Montag, 18. September 2017, 14:28:37 CEST schrieb M. J. Everitt:
>
> <snip>
> 
> Would a virtual help any? Probably overlooking a good number of factors,
> but wasn't mentioned yet ...
> 

So far I don't see how...  Virtual would mean that the same functionality is 
provided by different packages, and that they can be exchanged at runtime. 
However, for rpc there are different library *names* (sonames) to be linked, 
and for nls the soversion is different... Even if the headers were identical, 
that wouldnt work :/

-- 
Andreas K. Hüttel
dilfridge@gentoo.org
Gentoo Linux developer (council, perl, libreoffice)

Re: [gentoo-dev] glibc-2.26 and changes with SunRPC, libtirpc, ntirpc, libnsl (NIS and friends), ...

[M. J. Everitt]

[-- Attachment #1.1: Type: text/plain, Size: 747 bytes --]

On 18/09/17 16:36, Andreas K. Huettel wrote:
> Am Montag, 18. September 2017, 14:28:37 CEST schrieb M. J. Everitt:
>> <snip>
>>
>> Would a virtual help any? Probably overlooking a good number of factors,
>> but wasn't mentioned yet ...
>>
> So far I don't see how...  Virtual would mean that the same functionality is 
> provided by different packages, and that they can be exchanged at runtime. 
> However, for rpc there are different library *names* (sonames) to be linked, 
> and for nls the soversion is different... Even if the headers were identical, 
> that wouldnt work :/
>
Gotcha, thanks for the explanation. So you'd even need a common API
(header, etc) to be able to do some fudgery linking the lib names even.

MJE


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 819 bytes --]

Re: [gentoo-dev] glibc-2.26 and changes with SunRPC, libtirpc, ntirpc, libnsl (NIS and friends), ...

[Andreas K. Huettel]

> My suggestion for an ideal implementation would be that any package that
> uses RPC defines useflags:
> sunrpc - build against glibc
> libtirpc - build against net-libs/libtirpc
> ntirpc - build against net-libs/ntirpc
> with
> REQUIRED_USE="^^ ( sunrpc libtirpc ntirpc )"
> If rpc support is optional with useflag rpc, then this becomes
> REQUIRED_USE="rpc? ( ^^ ( sunrpc libtirpc ntirpc ) )"

... or the same in blue with a USE_EXPAND named RPC_PROVIDER; maybe that is 
cleaner.


-- 
Andreas K. Hüttel
dilfridge@gentoo.org
Gentoo Linux developer (council, perl, libreoffice)

Re: [gentoo-dev] glibc-2.26 and changes with SunRPC, libtirpc, ntirpc, libnsl (NIS and friends), ...

[Alexis Ballier]

On Mon, 18 Sep 2017 11:56:30 +0200
"Andreas K. Huettel" <dilfridge@gentoo.org> wrote:

> Porting a package means adding a dependency in the style of 
> || ( <sys-libs/glibc-2.26 net-libs/libnsl )

Better make that:
|| ( net-libs/libns <sys-libs/glibc-2.26 )

so that the prefer-leftmost rule never ever tries to downgrade glibc

Re: [gentoo-dev] glibc-2.26 and changes with SunRPC, libtirpc, ntirpc, libnsl (NIS and friends), ...

[Andreas K. Huettel]

Am Montag, 18. September 2017, 20:51:22 CEST schrieb Alexis Ballier:
> On Mon, 18 Sep 2017 11:56:30 +0200
> 
> "Andreas K. Huettel" <dilfridge@gentoo.org> wrote:
> > Porting a package means adding a dependency in the style of
> > 
> > || ( <sys-libs/glibc-2.26 net-libs/libnsl )
> 
> Better make that:
> || ( net-libs/libns <sys-libs/glibc-2.26 )
> 
> so that the prefer-leftmost rule never ever tries to downgrade glibc

Works for me.

-- 
Andreas K. Hüttel
dilfridge@gentoo.org
Gentoo Linux developer (council, perl, libreoffice)

[gentoo-dev] Re: glibc-2.26 and changes with SunRPC, libtirpc, ntirpc, libnsl (NIS and friends), ...

[Duncan]

Andreas K. Huettel posted on Mon, 18 Sep 2017 11:56:30 +0200 as excerpted:

> It may not always be obvious where this is needed, since net-libs/libnsl
> is already pulled in deep in the dependency tree (my @system chroot has
> it).

FWIW, while it may be deep in the @system dependency tree, I don't have 
libnsl installed here, so if it's not in @system directly, it's 
apparently pulled into @system by USE deps of some sort, via USE flags 
that aren't required to be on, in at least some reasonably general 
purpose plasma desktop configurations.

(I run with no @system, having long ago negated the entire thing, at the 
time via individual -pkg entries in /etc/portage/profile/packages, now 
via a single -* entry, from the inherited profile and listed deps I 
actually needed in sets pulled in by world_sets.  And my global USE 
starts with -*, adding only the flags I actually want/need, so whatever 
USE is pulling it in obviously isn't something I wanted or found I needed 
due to required-deps or something, here.)

-- 
Duncan - List replies preferred.   No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master."  Richard Stallman

Re: [gentoo-dev] Re: glibc-2.26 and changes with SunRPC, libtirpc, ntirpc, libnsl (NIS and friends), ...

[Andreas K. Huettel]

Am Dienstag, 19. September 2017, 07:06:24 CEST schrieb Duncan:
> Andreas K. Huettel posted on Mon, 18 Sep 2017 11:56:30 +0200 as excerpted:
> > It may not always be obvious where this is needed, since net-libs/libnsl
> > is already pulled in deep in the dependency tree (my @system chroot has
> > it).
> 
> FWIW, while it may be deep in the @system dependency tree, I don't have
> libnsl installed here, so if it's not in @system directly, it's
> apparently pulled into @system by USE deps of some sort, via USE flags
> that aren't required to be on, in at least some reasonably general
> purpose plasma desktop configurations.
 
[snip]

Do you run glibc-2.26 already? I hope not, because it's >>unkeyworded<< and 
I'm still changing things without warning there... :) [*]

With any other glibc, it's part of sys-libs/glibc.

And you will need it, because part of dev-lang/python links to it (for us) 
unconditionally.


[*] If yes, please join #gentoo-toolchain, because that's the only place where 
I right now mention when I break something. :)

-- 
Andreas K. Hüttel
dilfridge@gentoo.org
Gentoo Linux developer (council, perl, libreoffice)

[gentoo-dev] Re: glibc-2.26 and changes with SunRPC, libtirpc, ntirpc, libnsl (NIS and friends), ...

[Duncan]

Andreas K. Huettel posted on Tue, 19 Sep 2017 14:53:20 +0200 as excerpted:

> Am Dienstag, 19. September 2017, 07:06:24 CEST schrieb Duncan:
>> Andreas K. Huettel posted on Mon, 18 Sep 2017 11:56:30 +0200 as
>> excerpted:
>> > It may not always be obvious where this is needed, since
>> > net-libs/libnsl is already pulled in deep in the dependency tree (my
>> > @system chroot has it).
>> 
>> FWIW, while it may be deep in the @system dependency tree, I don't have
>> libnsl installed here
> 
> Do you run glibc-2.26 already? I hope not, because it's >>unkeyworded<<
> and I'm still changing things without warning there... :) [*]
> 
> With any other glibc, it's part of sys-libs/glibc.
> 
> And you will need it, because part of dev-lang/python links to it (for
> us) unconditionally.

Thanks for the clarification.  I'm still on glibc-2.25-r5 here

(Having read the thread as it developed I seem to have forgotten the bit 
about it being included in 2.25 by the time I replied, so the 
clarification is very helpful. =:^)

-- 
Duncan - List replies preferred.   No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master."  Richard Stallman

Re: [gentoo-dev] glibc-2.26 and changes with SunRPC, libtirpc, ntirpc, libnsl (NIS and friends), ...

[Michał Górny]

W dniu pon, 18.09.2017 o godzinie 11∶56 +0200, użytkownik Andreas K.
Huettel napisał:
> It may not always be obvious where this is needed, since net-libs/libnsl is 
> already pulled in deep in the dependency tree (my @system chroot has it).
> 

'emerge --depclean -v net-libs/libnsl' will tell you the truth.


-- 
Best regards,
Michał Górny

Re: [gentoo-dev] glibc-2.26 and changes with SunRPC, libtirpc, ntirpc, libnsl (NIS and friends), ...

[Alexis Ballier]

On Mon, 18 Sep 2017 11:56:30 +0200
"Andreas K. Huettel" <dilfridge@gentoo.org> wrote:

> sunrpc - build against glibc

Now that I think about it: What about other libcs ? musl, uclibc,
freebsd or even the prefix ones ?

[...]
> Porting a package means adding a dependency in the style of 
> || ( <sys-libs/glibc-2.26 net-libs/libnsl )


Shouldn't we prefix that || by elibc_glibc ? or only the glibc dep ?

[gentoo-dev] Re: glibc-2.26 and changes with SunRPC, libtirpc, ntirpc, libnsl (NIS and friends), ...

[Luca Barbato]

On 18/09/2017 11:56, Andreas K. Huettel wrote:
> So glibc-2.26 is already out for some time, but we still haven't keyworded it
> yet. Why?
> 
> * I want to use the opportunity to make the long-delayed switchover from
> glibc-internal SunRPC (long deprecated and outdated) to external
> implementations (libtirpc, and possibly ntirpc).
> 
> * The (outdated and deprecated) NIS(YP) and NIS+ support (libnsl) has been
> removed from glibc (except for a compatibility library that doesnt install
> headers), and is now provided by net-libs/libnsl (increased soversion).
> 
> This mail is mainly about how to best structure the transition.
> Comments, suggestions, corrections, feedback welcome.
> 
> 1) About RPC.
> 
> AFACIS there are three implementations:
> a) SunRPC, headers in /usr/include, code provided by glibc
> b) net-libs/libtirpc, headers in /usr/include/libtirpc, library -l tirpc
> c) (?) net-libs/ntirpc, headers in /usr/include/ntirpc, library -l ntirpc
> 
> Option a) is going away with sys-libs/glibc-2.26-r1.
> Options b) and c) may in addition need headers from net-libs/rpcsvc-proto
> I haven't done any testing with c) yet, will do so.
> a), b), and c) are co-installable.
> 
> My suggestion for an ideal implementation would be that any package that uses
> RPC defines useflags:
> sunrpc - build against glibc
So this one is transient.

> libtirpc - build against net-libs/libtirpc
> ntirpc - build against net-libs/ntirpc
> with
> REQUIRED_USE="^^ ( sunrpc libtirpc ntirpc )"
> If rpc support is optional with useflag rpc, then this becomes
> REQUIRED_USE="rpc? ( ^^ ( sunrpc libtirpc ntirpc ) )"
> 
> Since the three options are coinstallable I see no problems with a package
> only supporting a subset, but I have no clue how this interacts at runtime.

If they aren't ABI-compatible you would expect explosions once you link 
two libraries linked to the two different implementation (assuming they 
aren't macro-mangling everything).

> Of course this "ideal option" is also the most work-intensive.
> 
> Both libtirpc and ntirpc supply a packageconfig file. Porting a package means
> pointing it to the right include directory (at compile time) and library name
> (at link time); if that's not done, a build fails because the rpc headers
> cannot be found.
> (In my @system chroot, python fails atm.)
> 
> 2) About YP / NIS / NIS+.
> 
> a) The old libnsl implementation is provided by glibc, soversion 1.
> b) An updated and much improved implementation is provided by net-libs/libnsl,
> soversion 2.
> 
> glibc-2.26 installs only the library for a), and no headers.
> Since I dont want to mess with currently used glibc ebuilds, net-libs/libnsl
> requires at least our glibc-2.26 (otherwise you get file collisions).
> 
> Porting a package means adding a dependency in the style of
> || ( <sys-libs/glibc-2.26 net-libs/libnsl )
> 
> It may not always be obvious where this is needed, since net-libs/libnsl is
> already pulled in deep in the dependency tree (my @system chroot has it).
> 
> 
> So... that's it for the moment.
> Comments, ideas?

Beside making sure the providers are mix-and-matchable w/out pain sounds 
good.

We could check if the other libc could be switched to the external 
provider and play the lazy card and just always force an external 
implementation.

lu

Re: [gentoo-dev] Re: glibc-2.26 and changes with SunRPC, libtirpc, ntirpc, libnsl (NIS and friends), ...

[Andreas K. Huettel]

Am Dienstag, 19. September 2017, 22:38:17 CEST schrieb Luca Barbato:
> > REQUIRED_USE="^^ ( sunrpc libtirpc ntirpc )"
> > If rpc support is optional with useflag rpc, then this becomes
> > REQUIRED_USE="rpc? ( ^^ ( sunrpc libtirpc ntirpc ) )"
> > 
> > Since the three options are coinstallable I see no problems with a package
> > only supporting a subset, but I have no clue how this interacts at
> > runtime.
> 
> If they aren't ABI-compatible you would expect explosions once you link
> two libraries linked to the two different implementation (assuming they
> aren't macro-mangling everything).

Yep. So, apart from requiring "use the same implementation everywhere", i.e. 
set the flag globally, and stating "if you micromanage, you have to contain 
the explosions yourself" - is there anything else we can realistically do?

> We could check if the other libc could be switched to the external
> provider and play the lazy card and just always force an external
> implementation.

Two or three implementations doesnt make that much of a difference anymore...

-- 
Andreas K. Hüttel
dilfridge@gentoo.org
Gentoo Linux developer (council, perl, libreoffice)

Re: [gentoo-dev] Re: glibc-2.26 and changes with SunRPC, libtirpc, ntirpc, libnsl (NIS and friends), ...

[Michał Górny]

Dnia 20 września 2017 10:23:42 CEST, "Andreas K. Huettel" <dilfridge@gentoo.org> napisał(a):
>Am Dienstag, 19. September 2017, 22:38:17 CEST schrieb Luca Barbato:
>> > REQUIRED_USE="^^ ( sunrpc libtirpc ntirpc )"
>> > If rpc support is optional with useflag rpc, then this becomes
>> > REQUIRED_USE="rpc? ( ^^ ( sunrpc libtirpc ntirpc ) )"
>> > 
>> > Since the three options are coinstallable I see no problems with a
>package
>> > only supporting a subset, but I have no clue how this interacts at
>> > runtime.
>> 
>> If they aren't ABI-compatible you would expect explosions once you
>link
>> two libraries linked to the two different implementation (assuming
>they
>> aren't macro-mangling everything).
>
>Yep. So, apart from requiring "use the same implementation everywhere",
>i.e. 
>set the flag globally, and stating "if you micromanage, you have to
>contain 
>the explosions yourself" - is there anything else we can realistically
>do?

dev-libs/foo[sunrpc=,tirpc=...]? 

>
>> We could check if the other libc could be switched to the external
>> provider and play the lazy card and just always force an external
>> implementation.
>
>Two or three implementations doesnt make that much of a difference
>anymore...


-- 
Best regards,
Michał Górny (by phone)

Update - Re: [gentoo-dev] glibc-2.26 and changes with SunRPC, libtirpc, ntirpc, libnsl (NIS and friends), ...

[Andreas K. Huettel]

[-- Attachment #1: Type: text/plain, Size: 1167 bytes --]

Update... 

So I asked ntirpc upstream, and they do not intend to remain compatible with 
sunrpc / libtirpc. Meaning we only have two RPC alternatives, one of which 
will go away, and things get a lot easier. 

> 1) About RPC.

https://wiki.gentoo.org/wiki/Project:Toolchain/RPC_implementation

^ please see this Wiki page for a proposed workflow and autoconf examples.

In short, IUSE=libtirpc switches the implementation, while IUSE=rpc switches 
an optional rpc support on/off.
(Similar to ssl, libressl)


> 2) About YP / NIS / NIS+.
> 
> a) The old libnsl implementation is provided by glibc, soversion 1.
> b) An updated and much improved implementation is provided by
> net-libs/libnsl, soversion 2.
> 
> glibc-2.26 installs only the library for a), and no headers.
> Since I dont want to mess with currently used glibc ebuilds, net-libs/libnsl
> requires at least our glibc-2.26 (otherwise you get file collisions).
> 
> Porting a package means adding a dependency in the style of

|| ( net-libs/libnsl <sys-libs/glibc-2.26 )



-- 
Andreas K. Hüttel
dilfridge@gentoo.org
Gentoo Linux developer (council, perl, libreoffice)

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 833 bytes --]