[gentoo-dev] ssl vs openssl vs libressl vs gnutls USE flag foo

[gentoo-dev] ssl vs openssl vs libressl vs gnutls USE flag foo

[hasufell]

I've seen a lot of ebuilds lately that use 'openssl' USE flag for the
purpose of enabling ssl features. I think this should be discouraged
since it introduces inconsistency and is especially confusing for
packages like media-video/ffmpeg, where'd you expect to get ssl support
by having the global ssl USE flag enabled.

Furthermore, some packages have started to do things like
REQUIRED_USE="^^ ( openssl libressl )"
which is even more inconsistent now and will make it very hard for
people to switch to libressl without figuring out a lot of blockers,
since we have conflicting meanings of 'openssl' now. One uses it as a
feature flag, the other as a provider flag.


We have two choices here afais to design the USE flag meanings and how
they are used by ebuild authors:

A) 1 feature flag, 2 lax provider flags, 1 strict provider flag
* ssl: enable any sort of SSL/TLS support
* gnutls: primarily to enable gnutls provided ssl support in case
  there is a choice
* openssl: primarily to enable openssl provided ssl support in case
           there is a choice (might be implemented as !gnutls? instead)
* libressl: switch the openssl provider to libressl _without_
            conflicting with openssl, so any alternative usage of
            'openssl' USE flag will not break the libressl dep-graph

consequences:
* REQUIRED_USE="^^ ( openssl libressl )" is _disallowed_
* packages like media-video/ffmpeg should switch the USE flag
  openssl->ssl to avoid confusing global user configuration, but it's
  not strictly disallowed


B) 1 feature flag, 3 strict provider flags
* ssl: enable any sort of SSL/TLS support
* gnutls: only to enable gnutls provided ssl support in case there
          is a choice
* openssl: only to enable openssl provided ssl support in case
           there is a choice (should not be implemented as !gnutls?)
* libressl: only to enable libressl provided ssl support in case there
            is a choice, must conflict with 'openssl' USE flag

consequences:
* REQUIRED_USE="^^ ( openssl libressl )" is not only allowed, it is
  _mandatory_
* packages like media-video/ffmpeg _must_ switch the USE flag
  openssl->ssl to avoid breaking global USE flags
* !gnutls? ( dev-libs/openssl:0 ) will be bad form or even disallowed


A is not that difficult. Most uses of 'openssl' can just be replaced
with 'ssl', others probably with '!gnutls?' even. A few exotic ones
might stay and we will have to advice users to set USE="openssl
libressl" instead of USE="-openssl libressl".
B will definitely be more work, but ofc is also a lot cleaner and
totally unambigous.

I hope I didn't confuse anything here.

Re: [gentoo-dev] ssl vs openssl vs libressl vs gnutls USE flag foo

[Rich Freeman]

On Tue, Oct 27, 2015 at 10:06 PM, hasufell <hasufell@gentoo.org> wrote:
>
> B) 1 feature flag, 3 strict provider flags
> * ssl: enable any sort of SSL/TLS support
> * gnutls: only to enable gnutls provided ssl support in case there
>           is a choice
> * openssl: only to enable openssl provided ssl support in case
>            there is a choice (should not be implemented as !gnutls?)
> * libressl: only to enable libressl provided ssl support in case there
>             is a choice, must conflict with 'openssl' USE flag
>
> consequences:
> * REQUIRED_USE="^^ ( openssl libressl )" is not only allowed, it is
>   _mandatory_
> * packages like media-video/ffmpeg _must_ switch the USE flag
>   openssl->ssl to avoid breaking global USE flags
> * !gnutls? ( dev-libs/openssl:0 ) will be bad form or even disallowed
>
> B will definitely be more work, but ofc is also a lot cleaner and
> totally unambigous.
>

++

The pain is for a short time.  Then we have to live with this for a
long time.  USE flags should have one meaning.  The fact that this
isn't the case right now is already a bug.  We don't need to
perpetuate it.

Honestly, this just seems like "the right thing" so if there isn't
opposition then I'd suggest to "just do it" and commit fixes to
ebuilds that need the fix (ie if maintainer doesn't respond to bug
quickly just take care of it).  If people object they should speak up
now, and we can take it up at the next council meeting if necessary
(which is right around the corner).

-- 
Rich

Re: [gentoo-dev] ssl vs openssl vs libressl vs gnutls USE flag foo

[Gordon Pettey]

[-- Attachment #1: Type: text/plain, Size: 1762 bytes --]

Is this not precisely what USE_EXPAND is supposed to be for? Take CURL_SSL
and make it generic...

On Tue, Oct 27, 2015 at 9:46 PM, Rich Freeman <rich0@gentoo.org> wrote:

> On Tue, Oct 27, 2015 at 10:06 PM, hasufell <hasufell@gentoo.org> wrote:
> >
> > B) 1 feature flag, 3 strict provider flags
> > * ssl: enable any sort of SSL/TLS support
> > * gnutls: only to enable gnutls provided ssl support in case there
> >           is a choice
> > * openssl: only to enable openssl provided ssl support in case
> >            there is a choice (should not be implemented as !gnutls?)
> > * libressl: only to enable libressl provided ssl support in case there
> >             is a choice, must conflict with 'openssl' USE flag
> >
> > consequences:
> > * REQUIRED_USE="^^ ( openssl libressl )" is not only allowed, it is
> >   _mandatory_
> > * packages like media-video/ffmpeg _must_ switch the USE flag
> >   openssl->ssl to avoid breaking global USE flags
> > * !gnutls? ( dev-libs/openssl:0 ) will be bad form or even disallowed
> >
> > B will definitely be more work, but ofc is also a lot cleaner and
> > totally unambigous.
> >
>
> ++
>
> The pain is for a short time.  Then we have to live with this for a
> long time.  USE flags should have one meaning.  The fact that this
> isn't the case right now is already a bug.  We don't need to
> perpetuate it.
>
> Honestly, this just seems like "the right thing" so if there isn't
> opposition then I'd suggest to "just do it" and commit fixes to
> ebuilds that need the fix (ie if maintainer doesn't respond to bug
> quickly just take care of it).  If people object they should speak up
> now, and we can take it up at the next council meeting if necessary
> (which is right around the corner).
>
> --
> Rich
>
>

[-- Attachment #2: Type: text/html, Size: 2474 bytes --]

[gentoo-dev] Re: ssl vs openssl vs libressl vs gnutls USE flag foo

[Ryan Hill]

[-- Attachment #1: Type: text/plain, Size: 2394 bytes --]

On Tue, 27 Oct 2015 22:46:35 -0400
Rich Freeman <rich0@gentoo.org> wrote:

> On Tue, Oct 27, 2015 at 10:06 PM, hasufell <hasufell@gentoo.org> wrote:
> >
> > B) 1 feature flag, 3 strict provider flags
> > * ssl: enable any sort of SSL/TLS support
> > * gnutls: only to enable gnutls provided ssl support in case there
> >           is a choice
> > * openssl: only to enable openssl provided ssl support in case
> >            there is a choice (should not be implemented as !gnutls?)
> > * libressl: only to enable libressl provided ssl support in case there
> >             is a choice, must conflict with 'openssl' USE flag
> >
> > consequences:
> > * REQUIRED_USE="^^ ( openssl libressl )" is not only allowed, it is
> >   _mandatory_
> > * packages like media-video/ffmpeg _must_ switch the USE flag
> >   openssl->ssl to avoid breaking global USE flags
> > * !gnutls? ( dev-libs/openssl:0 ) will be bad form or even disallowed
> >
> > B will definitely be more work, but ofc is also a lot cleaner and
> > totally unambigous.
> >  
> 
> ++
> 
> The pain is for a short time.  Then we have to live with this for a
> long time.  USE flags should have one meaning.  The fact that this
> isn't the case right now is already a bug.  We don't need to
> perpetuate it.
> 
> Honestly, this just seems like "the right thing" so if there isn't
> opposition then I'd suggest to "just do it" and commit fixes to
> ebuilds that need the fix (ie if maintainer doesn't respond to bug
> quickly just take care of it).  If people object they should speak up
> now, and we can take it up at the next council meeting if necessary
> (which is right around the corner).

Agreed.  If there's one choice then "ssl" should be used.  openssl/libressl/etc
should really be considered sub-flags of ssl.

I really wish we had some way of specifying this to make things clearer to the
user, so they could see exactly how these flags interact with each other.
Something like (emerge -pv):

ssl [ (openssl) libressl gnutls ] 
 - if USE ssl then pick one of, default openssl if none chosen

ssl [[ (openssl) libressl gnutls ]]
 - if USE ssl then one or more of... etc.

But I suppose that's another topic.


-- 
Ryan Hill                        psn: dirtyepic_sk
   gcc-porting/toolchain/wxwidgets @ gentoo.org

47C3 6D62 4864 0E49 8E9E  7F92 ED38 BD49 957A 8463

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 475 bytes --]

Re: [gentoo-dev] Re: ssl vs openssl vs libressl vs gnutls USE flag foo

[hasufell]

On 10/28/2015 07:23 AM, Ryan Hill wrote:
> 
> Agreed.  If there's one choice then "ssl" should be used.  openssl/libressl/etc
> should really be considered sub-flags of ssl.
> 
> I really wish we had some way of specifying this to make things clearer to the
> user, so they could see exactly how these flags interact with each other.
> Something like (emerge -pv):
> 

The problem here is that we introduced REQUIRED_USE foo for these cases
which is again highly ambigous instead of making the PM aware that this
is an actual sub-USE flag.

This is outside of the scope of this thread, but there are already
distros that have fixed this:
1. NixOS [0] with truly declarative configuration format, e.g. something
like:
packages.ssl.provider = openssl;

or somesuch (just an example)

2. Exherbo partly [1] with providers syntax:
*/* providers: -openssl libressl

and the exheres would then do something like:
DEPENDENCIES="
    build+run:
        providers:openssl?  ( dev-libs/openssl:0 )
        providers:libressl? ( dev-libs/libressl )
"

which is a lot cleaner than USE_EXPAND + REQUIRED_USE, which still can
have arbitrary meanings.

But NIH will prevent us from learning here I guess.


[0] https://nixos.org/nixos/manual/
[1] http://exherbo.org/docs/eapi/providers-and-virtuals.html

Re: [gentoo-dev] Re: ssl vs openssl vs libressl vs gnutls USE flag foo

[Anthony G. Basile]

On 10/28/15 7:16 AM, hasufell wrote:
> On 10/28/2015 07:23 AM, Ryan Hill wrote:
>> Agreed.  If there's one choice then "ssl" should be used.  openssl/libressl/etc
>> should really be considered sub-flags of ssl.

This is what I did with curl.  USE=ssl means one and exactly one ssl 
provider must be specified.  I suggested making it a model gentoo wide, 
but there were criticisms, I forget what, but the made sense to me at 
teh time.

>>
>> I really wish we had some way of specifying this to make things clearer to the
>> user, so they could see exactly how these flags interact with each other.
>> Something like (emerge -pv):
>>
> The problem here is that we introduced REQUIRED_USE foo for these cases
> which is again highly ambigous instead of making the PM aware that this
> is an actual sub-USE flag.

A properly designed sub-USE flag would be useful here and clearly better 
than our REQUIRED_USE.  I think REQUIRED_USE is fine for heterogeneous 
cases, but not when you have something like curl where you can either 
turn ssl on or off.  If it is off, nothing more needs to be specified, 
if it is on, then you must further specify one and exactly one ssl provider.

>
> This is outside of the scope of this thread, but there are already
> distros that have fixed this:
> 1. NixOS [0] with truly declarative configuration format, e.g. something
> like:
> packages.ssl.provider = openssl;
>
> or somesuch (just an example)
>
> 2. Exherbo partly [1] with providers syntax:
> */* providers: -openssl libressl
>
> and the exheres would then do something like:
> DEPENDENCIES="
>      build+run:
>          providers:openssl?  ( dev-libs/openssl:0 )
>          providers:libressl? ( dev-libs/libressl )
> "
>
> which is a lot cleaner than USE_EXPAND + REQUIRED_USE, which still can
> have arbitrary meanings.
>
> But NIH will prevent us from learning here I guess.
>
>
> [0] https://nixos.org/nixos/manual/
> [1] http://exherbo.org/docs/eapi/providers-and-virtuals.html
>


-- 
Anthony G. Basile, Ph.D.
Gentoo Linux Developer [Hardened]
E-Mail    : blueness@gentoo.org
GnuPG FP  : 1FED FAD9 D82C 52A5 3BAB  DC79 9384 FA6E F52D 4BBA
GnuPG ID  : F52D4BBA

Re: [gentoo-dev] Re: ssl vs openssl vs libressl vs gnutls USE flag foo

[hasufell]

On 10/28/2015 12:23 PM, Anthony G. Basile wrote:
> 
> A properly designed sub-USE flag would be useful here and clearly better
> than our REQUIRED_USE.  I think REQUIRED_USE is fine for heterogeneous
> cases, but not when you have something like curl where you can either
> turn ssl on or off.  If it is off, nothing more needs to be specified,
> if it is on, then you must further specify one and exactly one ssl
> provider.
> 

Uhm, curl makes use of REQUIRED_USE heavily, otherwise the use_expand
would not work:
> REQUIRED_USE="
> 	curl_ssl_winssl? ( elibc_Winnt )
> 	threads? ( !adns )
> 	ssl? (
> 		^^ (
> 			curl_ssl_axtls
> 			curl_ssl_gnutls
> 			curl_ssl_libressl
> 			curl_ssl_openssl
> 			curl_ssl_nss
> 			curl_ssl_polarssl
> 			curl_ssl_winssl
> 		)
> 	)"

With the providers syntax from exherbo, this monster would basically be
gone.

Re: [gentoo-dev] Re: ssl vs openssl vs libressl vs gnutls USE flag foo

[Anthony G. Basile]

On 10/28/15 7:30 AM, hasufell wrote:
> On 10/28/2015 12:23 PM, Anthony G. Basile wrote:
>> A properly designed sub-USE flag would be useful here and clearly better
>> than our REQUIRED_USE.  I think REQUIRED_USE is fine for heterogeneous
>> cases, but not when you have something like curl where you can either
>> turn ssl on or off.  If it is off, nothing more needs to be specified,
>> if it is on, then you must further specify one and exactly one ssl
>> provider.
>>
> Uhm, curl makes use of REQUIRED_USE heavily, otherwise the use_expand
> would not work:
>> REQUIRED_USE="
>> 	curl_ssl_winssl? ( elibc_Winnt )
>> 	threads? ( !adns )
>> 	ssl? (
>> 		^^ (
>> 			curl_ssl_axtls
>> 			curl_ssl_gnutls
>> 			curl_ssl_libressl
>> 			curl_ssl_openssl
>> 			curl_ssl_nss
>> 			curl_ssl_polarssl
>> 			curl_ssl_winssl
>> 		)
>> 	)"
> With the providers syntax from exherbo, this monster would basically be
> gone.
>
Yes, we're in agreement.  I'm saying I'm *trying* to do what exherbo 
does the best I can but really dont like REQUIRED_USE for this.

-- 
Anthony G. Basile, Ph.D.
Gentoo Linux Developer [Hardened]
E-Mail    : blueness@gentoo.org
GnuPG FP  : 1FED FAD9 D82C 52A5 3BAB  DC79 9384 FA6E F52D 4BBA
GnuPG ID  : F52D4BBA

Re: [gentoo-dev] Re: ssl vs openssl vs libressl vs gnutls USE flag foo

[Kristian Fiskerstrand]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA512

On 10/28/2015 12:23 PM, Anthony G. Basile wrote:
> On 10/28/15 7:16 AM, hasufell wrote:
>> On 10/28/2015 07:23 AM, Ryan Hill wrote:
>>> Agreed.  If there's one choice then "ssl" should be used. 
>>> openssl/libressl/etc should really be considered sub-flags of
>>> ssl.
> 
> This is what I did with curl.  USE=ssl means one and exactly one
> ssl provider must be specified.  I suggested making it a model
> gentoo wide, but there were criticisms, I forget what, but the made
> sense to me at teh time.

Such a setup makes sense to me as well, although likely want it to be
more generic and maybe use a prioritized list rather than a use expand
per package / group of packages. Maybe something similar to
python_compat can be used for the TLS / crypto provider


- -- 
Kristian Fiskerstrand
Public PGP key 0xE3EDFAE3 at hkp://pool.sks-keyservers.net
fpr:94CB AFDD 3034 5109 5618 35AA 0B7F 8B60 E3ED FAE3
-----BEGIN PGP SIGNATURE-----

iQEcBAEBCgAGBQJWMLIvAAoJECULev7WN52Fdb8IALLlM/763gnsx3iEc7NIh7IL
HWDfBwfY2vuDKWGGVlKIXn2CZfIrDhcqOZxtprFcrK073q+Gu6ZDqVd+og+EWTJw
Erv7aq5Mi4poBPtcQT6P3CJG77RXCQPdzgPG6K8FaWyVPh+RkKFr7VsJ615visk6
UmRbwpehU51Vb/qTWl94J1z0SvIo3gusSzeK9FpkSFimrZQtsByWFtIijWYeToLH
Mv4hl7BVwpHWctTwLDObuqkl+zWEITapJ3CIYz93jrfnyWZXi/DFvOyWcK3YOES2
vDcIDWGKt0SunZgka0VTJ+CTbfCcuvxb0sXUJfJNdy6P/p7aJyzZllPEZ2ciVVk=
=iN64
-----END PGP SIGNATURE-----

Re: [gentoo-dev] Re: ssl vs openssl vs libressl vs gnutls USE flag foo

[Rich Freeman]

On Wed, Oct 28, 2015 at 7:16 AM, hasufell <hasufell@gentoo.org> wrote:
>
> This is outside of the scope of this thread, but there are already
> distros that have fixed this:
> 1. NixOS [0] with truly declarative configuration format, e.g. something
> like:
> packages.ssl.provider = openssl;

Well, we can accomplish this in our syntax.  Just RDEPEND on openssl,
and set USE requirements for openssl on any dependencies that offer
both.

NixOS is still bound by the constraint that the two libraries have
colliding namespace, so a package needs to have a dependency chain
that exclusively uses one or the other.

However, assuming all your packages are able to work with either
library the thing NixOS does have going for it is that it would let
you have apache using openssl and postfix using libressl on the same
system, with side-by-side versions of any shared dependencies built
against each.

Their approach (as I understand it) is basically that every process is
almost containerized on the same filesystem.

>
> which is a lot cleaner than USE_EXPAND + REQUIRED_USE, which still can
> have arbitrary meanings.
>

Well, I think we can accomplish all of the above using the tools we
already have, but I agree that we tend to do it in one namespace while
other distros are using more than one.  That is probably a good idea
just to improve consistency.

We should probably pursue both.  For ssl we need the best solution we
can implement today.  However, for a future EAPI we should pursue a
better way to handle this.

-- 
Rich

Re: [gentoo-dev] Re: ssl vs openssl vs libressl vs gnutls USE flag foo

[Kristian Fiskerstrand]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA512

On 10/28/2015 07:23 AM, Ryan Hill wrote:

> 
> Agreed.  If there's one choice then "ssl" should be used.
> openssl/libressl/etc should really be considered sub-flags of ssl.
> 

If we are introducing a new and proper way to define this it might
make sense to not refer to ssl (is anything actually using SSL these
days? I certainly hope not), tls is the natural replacement unless we
want to go for something more generic.

- -- 
Kristian Fiskerstrand
Public PGP key 0xE3EDFAE3 at hkp://pool.sks-keyservers.net
fpr:94CB AFDD 3034 5109 5618 35AA 0B7F 8B60 E3ED FAE3
-----BEGIN PGP SIGNATURE-----

iQEcBAEBCgAGBQJWMK+QAAoJECULev7WN52FFesH/2uEi1vBt5QpR/OEusr2EFVi
IlkM9+hNNxkFO2pFriB63+iQX90k1+p6jB5X/0ARzXaaL4jfnQlq7XtvBmN6HFvI
ROBuWIpqtotM6Bm2EKDyDABXxExGcWSNPHPQS4hkyNVSfaredCoQMaweMExDPsip
Ief8T1R6orqPTv2iDN4Q7uypHUbOi0ogF7r1SAD/CQGL0a9WwFtYVzXBhj90O2F/
4OYTnzMirhDtyypPZb++H3J4U2CMm2kcoRtPomXybghTGjuWwz7We2lF61QrNeVA
IPODRSUDcDPgU8wp2knMGOmKxUQ6Ny8DH9IdQtY4BYdztDYVEV6jCRHGur1gVag=
=4KHj
-----END PGP SIGNATURE-----

Re: [gentoo-dev] Re: ssl vs openssl vs libressl vs gnutls USE flag foo

[hasufell]

On 10/28/2015 12:20 PM, Kristian Fiskerstrand wrote:
> On 10/28/2015 07:23 AM, Ryan Hill wrote:
> 
> 
>> Agreed.  If there's one choice then "ssl" should be used.
>> openssl/libressl/etc should really be considered sub-flags of ssl.
> 
> 
> If we are introducing a new and proper way to define this it might
> make sense to not refer to ssl (is anything actually using SSL these
> days? I certainly hope not), tls is the natural replacement unless we
> want to go for something more generic.
> 

That's nitpick for no gain and will break more configurations than any
of the previously suggested formats.

Re: [gentoo-dev] ssl vs openssl vs libressl vs gnutls USE flag foo

[Michał Górny]

[-- Attachment #1: Type: text/plain, Size: 1820 bytes --]

On Tue, 27 Oct 2015 22:46:35 -0400
Rich Freeman <rich0@gentoo.org> wrote:

> On Tue, Oct 27, 2015 at 10:06 PM, hasufell <hasufell@gentoo.org> wrote:
> >
> > B) 1 feature flag, 3 strict provider flags
> > * ssl: enable any sort of SSL/TLS support
> > * gnutls: only to enable gnutls provided ssl support in case there
> >           is a choice
> > * openssl: only to enable openssl provided ssl support in case
> >            there is a choice (should not be implemented as !gnutls?)
> > * libressl: only to enable libressl provided ssl support in case there
> >             is a choice, must conflict with 'openssl' USE flag
> >
> > consequences:
> > * REQUIRED_USE="^^ ( openssl libressl )" is not only allowed, it is
> >   _mandatory_
> > * packages like media-video/ffmpeg _must_ switch the USE flag
> >   openssl->ssl to avoid breaking global USE flags
> > * !gnutls? ( dev-libs/openssl:0 ) will be bad form or even disallowed
> >
> > B will definitely be more work, but ofc is also a lot cleaner and
> > totally unambigous.
> >  
> 
> ++
> 
> The pain is for a short time.  Then we have to live with this for a
> long time.  USE flags should have one meaning.  The fact that this
> isn't the case right now is already a bug.  We don't need to
> perpetuate it.

No, the pain is neverending. You define a number of flags which are
scattered all over the place and there's practically no good value but
the 'default'.

We have no way of saying 'I prefer polarssl, then gnutls, then
libressl, and never openssl'. Whatever I put in USE, I'm going to hit
one kind of REQUIRED_USE issues, or other. And in the end, I end up
having huge package.use just to make things work.

How is that a 'short time' pain?

-- 
Best regards,
Michał Górny
<http://dev.gentoo.org/~mgorny/>

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 949 bytes --]

Re: [gentoo-dev] ssl vs openssl vs libressl vs gnutls USE flag foo

[hasufell]

On 10/30/2015 06:55 PM, Michał Górny wrote:
> 
> We have no way of saying 'I prefer polarssl, then gnutls, then
> libressl, and never openssl'.

I don't think this is something that can be reasonably supported and it
sounds awfully automagic. And I don't see how this is possible right
now, so I'm not really sure what you expect to get worse.

E.g. -gnutls pulling in dev-libs/openssl is not really something you'd
expect. If we go for provider USE flags, then things become consistent,
explicit and unambiguous. The only problem is our crappy implementation
of providers USE flags via REQUIRED_USE.

Re: [gentoo-dev] ssl vs openssl vs libressl vs gnutls USE flag foo

[Anthony G. Basile]

On 10/30/15 3:35 PM, hasufell wrote:
> On 10/30/2015 06:55 PM, Michał Górny wrote:
>> We have no way of saying 'I prefer polarssl, then gnutls, then
>> libressl, and never openssl'.
> I don't think this is something that can be reasonably supported and it
> sounds awfully automagic. And I don't see how this is possible right
> now, so I'm not really sure what you expect to get worse.
>
> E.g. -gnutls pulling in dev-libs/openssl is not really something you'd
> expect. If we go for provider USE flags, then things become consistent,
> explicit and unambiguous. The only problem is our crappy implementation
> of providers USE flags via REQUIRED_USE.
>
I'm not sure what mgorny has in mind, but the problem I see with saying 
I want just X to be my provider system wide is that some pkgs build with 
X others don't, other pkgs might need a different provider.  So it might 
make sense to order them in terms of preference: X1 > X2 > X3 ... and 
then when emerging a package, the first provider in the preference list 
that works is pulled in for that package.

-- 
Anthony G. Basile, Ph.D.
Gentoo Linux Developer [Hardened]
E-Mail    : blueness@gentoo.org
GnuPG FP  : 1FED FAD9 D82C 52A5 3BAB  DC79 9384 FA6E F52D 4BBA
GnuPG ID  : F52D4BBA

Re: [gentoo-dev] ssl vs openssl vs libressl vs gnutls USE flag foo

[Rich Freeman]

On Fri, Oct 30, 2015 at 5:16 PM, Anthony G. Basile <blueness@gentoo.org> wrote:
> On 10/30/15 3:35 PM, hasufell wrote:
>>
>> On 10/30/2015 06:55 PM, Michał Górny wrote:
>>>
>>> We have no way of saying 'I prefer polarssl, then gnutls, then
>>> libressl, and never openssl'.
>>
>> I don't think this is something that can be reasonably supported and it
>> sounds awfully automagic. And I don't see how this is possible right
>> now, so I'm not really sure what you expect to get worse.
>>
>> E.g. -gnutls pulling in dev-libs/openssl is not really something you'd
>> expect. If we go for provider USE flags, then things become consistent,
>> explicit and unambiguous. The only problem is our crappy implementation
>> of providers USE flags via REQUIRED_USE.
>>
> I'm not sure what mgorny has in mind, but the problem I see with saying I
> want just X to be my provider system wide is that some pkgs build with X
> others don't, other pkgs might need a different provider.  So it might make
> sense to order them in terms of preference: X1 > X2 > X3 ... and then when
> emerging a package, the first provider in the preference list that works is
> pulled in for that package.

I think that would be useful in general.  It would probably not be
useful in this case, since it was somebody's bright idea to make it
essentially impossible to install two of the options on the same
system (and that wasn't directed at hasufell).  Users could of course
still express the preference, but the PM would need to be smart enough
to ignore that preference on 95% of packages that support both options
so that it can take the lower preference on the 5% of packages that
only support the option the user didn't really want.

-- 
Rich

Re: [gentoo-dev] ssl vs openssl vs libressl vs gnutls USE flag foo

[Michał Górny]

[-- Attachment #1: Type: text/plain, Size: 2445 bytes --]

On Fri, 30 Oct 2015 18:25:14 -0400
Rich Freeman <rich0@gentoo.org> wrote:

> On Fri, Oct 30, 2015 at 5:16 PM, Anthony G. Basile <blueness@gentoo.org> wrote:
> > On 10/30/15 3:35 PM, hasufell wrote:  
> >>
> >> On 10/30/2015 06:55 PM, Michał Górny wrote:  
> >>>
> >>> We have no way of saying 'I prefer polarssl, then gnutls, then
> >>> libressl, and never openssl'.  
> >>
> >> I don't think this is something that can be reasonably supported and it
> >> sounds awfully automagic. And I don't see how this is possible right
> >> now, so I'm not really sure what you expect to get worse.
> >>
> >> E.g. -gnutls pulling in dev-libs/openssl is not really something you'd
> >> expect. If we go for provider USE flags, then things become consistent,
> >> explicit and unambiguous. The only problem is our crappy implementation
> >> of providers USE flags via REQUIRED_USE.
> >>  
> > I'm not sure what mgorny has in mind, but the problem I see with saying I
> > want just X to be my provider system wide is that some pkgs build with X
> > others don't, other pkgs might need a different provider.  So it might make
> > sense to order them in terms of preference: X1 > X2 > X3 ... and then when
> > emerging a package, the first provider in the preference list that works is
> > pulled in for that package.  
> 
> I think that would be useful in general.  It would probably not be
> useful in this case, since it was somebody's bright idea to make it
> essentially impossible to install two of the options on the same
> system (and that wasn't directed at hasufell).  Users could of course
> still express the preference, but the PM would need to be smart enough
> to ignore that preference on 95% of packages that support both options
> so that it can take the lower preference on the 5% of packages that
> only support the option the user didn't really want.

No, that's not *the* problem. LibreSSL vs OpenSSL is actually
the *least* problematic one since we intend to support them as
'drop-in-plus-rebuild' replacements.

The real problem is those fancy upstreams who believe they're doing
everyone a favor by providing the choice between multiple SSL
providers. This is what brings the real conflicts here, and this what
often loves to break stuff even further by introducing cross-package
implementation match requirements...

-- 
Best regards,
Michał Górny
<http://dev.gentoo.org/~mgorny/>

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 949 bytes --]

Re: [gentoo-dev] ssl vs openssl vs libressl vs gnutls USE flag foo

[hasufell]

On 10/30/2015 10:16 PM, Anthony G. Basile wrote:
> On 10/30/15 3:35 PM, hasufell wrote:
>> On 10/30/2015 06:55 PM, Michał Górny wrote:
>>> We have no way of saying 'I prefer polarssl, then gnutls, then
>>> libressl, and never openssl'.
>> I don't think this is something that can be reasonably supported and it
>> sounds awfully automagic. And I don't see how this is possible right
>> now, so I'm not really sure what you expect to get worse.
>>
>> E.g. -gnutls pulling in dev-libs/openssl is not really something you'd
>> expect. If we go for provider USE flags, then things become consistent,
>> explicit and unambiguous. The only problem is our crappy implementation
>> of providers USE flags via REQUIRED_USE.
>>
> I'm not sure what mgorny has in mind, but the problem I see with saying
> I want just X to be my provider system wide is that some pkgs build with
> X others don't, other pkgs might need a different provider.  So it might
> make sense to order them in terms of preference: X1 > X2 > X3 ... and
> then when emerging a package, the first provider in the preference list
> that works is pulled in for that package.
> 

Isn't that basically what the proposal B already was, except that we
don't use REQUIRED_USE for it but some sort of pkg_setup/pkg_pretend
function? I don't see how those ideas even conflict.

Re: [gentoo-dev] ssl vs openssl vs libressl vs gnutls USE flag foo

[Michał Górny]

[-- Attachment #1: Type: text/plain, Size: 1746 bytes --]

On Fri, 30 Oct 2015 23:40:28 +0100
hasufell <hasufell@gentoo.org> wrote:

> On 10/30/2015 10:16 PM, Anthony G. Basile wrote:
> > On 10/30/15 3:35 PM, hasufell wrote:  
> >> On 10/30/2015 06:55 PM, Michał Górny wrote:  
> >>> We have no way of saying 'I prefer polarssl, then gnutls, then
> >>> libressl, and never openssl'.  
> >> I don't think this is something that can be reasonably supported and it
> >> sounds awfully automagic. And I don't see how this is possible right
> >> now, so I'm not really sure what you expect to get worse.
> >>
> >> E.g. -gnutls pulling in dev-libs/openssl is not really something you'd
> >> expect. If we go for provider USE flags, then things become consistent,
> >> explicit and unambiguous. The only problem is our crappy implementation
> >> of providers USE flags via REQUIRED_USE.
> >>  
> > I'm not sure what mgorny has in mind, but the problem I see with saying
> > I want just X to be my provider system wide is that some pkgs build with
> > X others don't, other pkgs might need a different provider.  So it might
> > make sense to order them in terms of preference: X1 > X2 > X3 ... and
> > then when emerging a package, the first provider in the preference list
> > that works is pulled in for that package.
> >   
> 
> Isn't that basically what the proposal B already was, except that we
> don't use REQUIRED_USE for it but some sort of pkg_setup/pkg_pretend
> function? I don't see how those ideas even conflict.

And some sort of magical USE flag meanings? Please stop this right
here. We don't need 16 USE flag package variants which mean 4 things in
different, random and unexpected ways.

-- 
Best regards,
Michał Górny
<http://dev.gentoo.org/~mgorny/>

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 949 bytes --]

Re: [gentoo-dev] ssl vs openssl vs libressl vs gnutls USE flag foo

[hasufell]

On 10/30/2015 11:56 PM, Michał Górny wrote:
> On Fri, 30 Oct 2015 23:40:28 +0100
> hasufell <hasufell@gentoo.org> wrote:
> 
>> On 10/30/2015 10:16 PM, Anthony G. Basile wrote:
>>> On 10/30/15 3:35 PM, hasufell wrote:  
>>>> On 10/30/2015 06:55 PM, Michał Górny wrote:  
>>>>> We have no way of saying 'I prefer polarssl, then gnutls, then
>>>>> libressl, and never openssl'.  
>>>> I don't think this is something that can be reasonably supported and it
>>>> sounds awfully automagic. And I don't see how this is possible right
>>>> now, so I'm not really sure what you expect to get worse.
>>>>
>>>> E.g. -gnutls pulling in dev-libs/openssl is not really something you'd
>>>> expect. If we go for provider USE flags, then things become consistent,
>>>> explicit and unambiguous. The only problem is our crappy implementation
>>>> of providers USE flags via REQUIRED_USE.
>>>>  
>>> I'm not sure what mgorny has in mind, but the problem I see with saying
>>> I want just X to be my provider system wide is that some pkgs build with
>>> X others don't, other pkgs might need a different provider.  So it might
>>> make sense to order them in terms of preference: X1 > X2 > X3 ... and
>>> then when emerging a package, the first provider in the preference list
>>> that works is pulled in for that package.
>>>   
>>
>> Isn't that basically what the proposal B already was, except that we
>> don't use REQUIRED_USE for it but some sort of pkg_setup/pkg_pretend
>> function? I don't see how those ideas even conflict.
> 
> And some sort of magical USE flag meanings? Please stop this right
> here. We don't need 16 USE flag package variants which mean 4 things in
> different, random and unexpected ways.
> 

I really have no idea what you mean. This is about NOT doing things
magically and not having magical USE flag meanings.

Then you complained that you cannot set gnutls and openssl at the same
time... and the only way around that is not having REQUIRED_USE.

It seems to me you don't really know what you want. Either give an
actual proposal or let us move on.

Re: [gentoo-dev] ssl vs openssl vs libressl vs gnutls USE flag foo

[Luis Ressel]

On Fri, 30 Oct 2015 23:40:28 +0100
hasufell <hasufell@gentoo.org> wrote:

> On 10/30/2015 10:16 PM, Anthony G. Basile wrote:
> > On 10/30/15 3:35 PM, hasufell wrote:  
> >> On 10/30/2015 06:55 PM, Michał Górny wrote:  
> >>> We have no way of saying 'I prefer polarssl, then gnutls, then
> >>> libressl, and never openssl'.  
> >> I don't think this is something that can be reasonably supported
> >> and it sounds awfully automagic. And I don't see how this is
> >> possible right now, so I'm not really sure what you expect to get
> >> worse.
> >>
> >> E.g. -gnutls pulling in dev-libs/openssl is not really something
> >> you'd expect. If we go for provider USE flags, then things become
> >> consistent, explicit and unambiguous. The only problem is our
> >> crappy implementation of providers USE flags via REQUIRED_USE.
> >>  
> > I'm not sure what mgorny has in mind, but the problem I see with
> > saying I want just X to be my provider system wide is that some
> > pkgs build with X others don't, other pkgs might need a different
> > provider.  So it might make sense to order them in terms of
> > preference: X1 > X2 > X3 ... and then when emerging a package, the
> > first provider in the preference list that works is pulled in for
> > that package. 
> 
> Isn't that basically what the proposal B already was, except that we
> don't use REQUIRED_USE for it but some sort of pkg_setup/pkg_pretend
> function? I don't see how those ideas even conflict.
> 

Well, not exactly. If I understood them right, mgorny and blueness are
asking for a user-supplied preference list (e.g. "I want packages to
link with libressl if possible, gnutls otherwise"), not an
ebuild-supplied preference list ("This package prefers gnutls, but
openssl is also supported").

Side note: These ebuild-side preferences are used by some ebuilds (e.g.
cyrus-sasl, it uses gdbm if both gdbm and berkdb use flags are
enabled), but for ssl, we might want to specify "REQUIRED_USE = ^^
(..)" so it's possible to use USE dependencies in order to avoid
namespace conflicts. If there's no REQUIRED_USE,
"somelibrary[libressl]" might be satisfied even though somelibrary is
actually linked to openssl.


-- 
Regards,
Luis Ressel

Re: [gentoo-dev] ssl vs openssl vs libressl vs gnutls USE flag foo

[Rich Freeman]

On Fri, Oct 30, 2015 at 1:55 PM, Michał Górny <mgorny@gentoo.org> wrote:
>>
>> The pain is for a short time.  Then we have to live with this for a
>> long time.  USE flags should have one meaning.  The fact that this
>> isn't the case right now is already a bug.  We don't need to
>> perpetuate it.
>
> No, the pain is neverending. You define a number of flags which are
> scattered all over the place and there's practically no good value but
> the 'default'.
>

My response was intended as a comparison of the two options presented,
which so far are the only options that have been suggested by anybody
that don't require EAPI changes.

I wasn't suggesting that there wasn't room for improvement in general.
However, short of banning libressl until EAPI7 and actually doing
something in EAPI7 this is our current best option.

-- 
Rich

Re: [gentoo-dev] ssl vs openssl vs libressl vs gnutls USE flag foo

[Alexis Ballier]

On Wed, 28 Oct 2015 03:06:59 +0100
hasufell <hasufell@gentoo.org> wrote:
> A is not that difficult. Most uses of 'openssl' can just be replaced
> with 'ssl', others probably with '!gnutls?' even. A few exotic ones
> might stay and we will have to advice users to set USE="openssl
> libressl" instead of USE="-openssl libressl".
> B will definitely be more work, but ofc is also a lot cleaner and
> totally unambigous.


You haven't taken into consideration the licence incompatibilities:
    http://www.gnu.org/licenses/license-list.en.html#OpenSSL
it gets really messy for libraries: a gpl binary linking against a
   library linking against openssl means the binary can be
   redistributed, but not with such a library linked against openssl...

the point of the 'openssl' useflag is to have something that is not
enabled by default and that can be used in RESTRICT="openssl?
( bindist )" expressions...

Re: [gentoo-dev] ssl vs openssl vs libressl vs gnutls USE flag foo

[hasufell]

On 10/28/2015 09:36 AM, Alexis Ballier wrote:
> On Wed, 28 Oct 2015 03:06:59 +0100
> hasufell <hasufell@gentoo.org> wrote:
>> A is not that difficult. Most uses of 'openssl' can just be replaced
>> with 'ssl', others probably with '!gnutls?' even. A few exotic ones
>> might stay and we will have to advice users to set USE="openssl
>> libressl" instead of USE="-openssl libressl".
>> B will definitely be more work, but ofc is also a lot cleaner and
>> totally unambigous.
> 
> 
> You haven't taken into consideration the licence incompatibilities:
>     http://www.gnu.org/licenses/license-list.en.html#OpenSSL
> it gets really messy for libraries: a gpl binary linking against a
>    library linking against openssl means the binary can be
>    redistributed, but not with such a library linked against openssl...
> 
> the point of the 'openssl' useflag is to have something that is not
> enabled by default and that can be used in RESTRICT="openssl?
> ( bindist )" expressions...
> 


You can just do RESTRICT="ssl? ( bindist )" and disable bindist by
default or vote for solution B.

Re: [gentoo-dev] ssl vs openssl vs libressl vs gnutls USE flag foo

[Chí-Thanh Christopher Nguyễn]

hasufell schrieb:
> I've seen a lot of ebuilds lately that use 'openssl' USE flag for the
> purpose of enabling ssl features. I think this should be discouraged
> since it introduces inconsistency and is especially confusing for
> packages like media-video/ffmpeg, where'd you expect to get ssl support
> by having the global ssl USE flag enabled.
>
> Furthermore, some packages have started to do things like
> REQUIRED_USE="^^ ( openssl libressl )"
> which is even more inconsistent now and will make it very hard for
> people to switch to libressl without figuring out a lot of blockers,
> since we have conflicting meanings of 'openssl' now. One uses it as a
> feature flag, the other as a provider flag.

It has been discussed before how to map this to USE flags[1], but that 
turned out to be quite difficult. Especially if you want to express 
something like "this package must use the same crypto library as its 
dependency".

REQUIRED_USE="^^ ( openssl libressl )" is one way to make things easy 
for the ebuild developer, but nasty for the user.

For the users, the easiest way would be to set USE="openssl libressl" 
(or some USE_EXPAND) if they are fine with any of these, but this makes 
depending on a package which must be built e.g. against libressl and not 
openssl hard.

Best regards,
Chí-Thanh Christopher Nguyễn


[1] 
https://archives.gentoo.org/gentoo-dev/message/3fd9df7fdd7ac976b87e4e15587bfa63