[gentoo-dev] [RFC] Replacing binary-only SLOTs with separate packages

[gentoo-dev] [RFC] Replacing binary-only SLOTs with separate packages

[Michał Górny]

[-- Attachment #1: Type: text/plain, Size: 3814 bytes --]

Hello,

Since I've been working on the early gx86-multilib, we've been using
'binary-only SLOTs' to support providing old versions of libraries for
prebuilt software.  I think this was a bad idea, and I'd like to suggest
replacing it with something cleaner, for the reasons outlined below.


Current state
=============

Let's take dev-libs/openssl as an example.  This package has three SLOTs
right now:

  0.9.8: 0.9.8z_p8-r1
  1.0.0: 1.0.2q-r200
  0    : 1.0.2q 1.1.0j 1.1.1a 1.1.1a-r1

The real package is provided as slot :0, that includes all libraries,
headers and executables.  Slots 0.9.8 and 1.0.0 only install .so.N*
libraries that can be used to satisfy dependencies of prebuilt packages.


Problems with the current state
===============================

Firstly, it is confusing to developers.  Let's analyze the dependencies
on dev-libs/openssl.  A quick grep reveals seven patterns.  They are
listed below, along with occurrence counts and percentages:

  dev-libs/openssl          278    7.8%  }
  dev-libs/openssl:*         49    1.4%  } 14.2%
  dev-libs/openssl:=        178    5.0%  }
  dev-libs/openssl:0        660   18.6%
  dev-libs/openssl:0=      2381   67.0%
  dev-libs/openssl:0/0        4    0.1%
  dev-libs/openssl:0/1.1      2    0.1%

(note that those are rough measures, not guaranteed to be precise)

So apparently 14.2% of dependencies allow any slot of OpenSSL which is
most likely wrong, and 1.4% explicitly claim that's what the package
wants.  This could be valid only if e.g. the package supported multiple
ABIs of OpenSSL libraries and used dlopen() with a few possible SONAMEs
which I honestly doubt any of those packages is doing.

In other words, 14.2% of dependencies on OpenSSL are plain wrong,
and 6.4% are wrong in a way that isn't going to be reported by repoman. 
1.4% of cases are using ':*' which probably indicates the developer
decided to silence repoman without understanding how slot operators work
which is a horrible thing from QA perspective.

We also have a few cases that require specific OpenSSL subslot (e.g.
forcing old version into :0 slot) but *none* actually using the binary
compatibility slots.


Secondly, it is confusing to users.  If we remove old versions and only
keep binary compatibility slots, users can be easily tricked into
installing them and being surprised it's not a complete package.  If we
keep old versions, we end up having different revisions of the same
version in different slots which is also easily confused.


Thirdly, it is cumbersome to introduce.  If we are to introduce a binary
compatibility slot for a package that didn't have it, we need to update
all reverse dependencies.  This usually means researching whether we
should use ':0' or ':0=', and if we get this wrong, we just silence
repoman warning about missing slot-op.


All of this considered, I can't think of a single real benefit of using
slots for this purpose.  They work but there's nothing really special
about them.


Suggested replacement
=====================

My suggestion is to move binary compatibility slots into separate
packages.  For example, dev-libs/openssl would be split into:

  dev-libs/openssl  -- containing the actual package

  dev-libs/openssl-bin-compat  -- containing binary compatibility slots

In this case, all dependencies on dev-libs/openssl would become correct
(or semi-correct, wrt missing := dep) again.  Since packages are co-
installable the same way slots are, there is no loss there.  Since
nothing depends on binary compatibility slots, we do not even need to
update anything (but if we had, the update cost would be minimal both to
developers and to users).


What do you think?

-- 
Best regards,
Michał Górny

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 963 bytes --]

Re: [gentoo-dev] [RFC] Replacing binary-only SLOTs with separate packages

[James Le Cuirot]

[-- Attachment #1: Type: text/plain, Size: 5059 bytes --]

On Fri, 18 Jan 2019 21:13:34 +0100
Michał Górny <mgorny@gentoo.org> wrote:

> Hello,
> 
> Since I've been working on the early gx86-multilib, we've been using
> 'binary-only SLOTs' to support providing old versions of libraries for
> prebuilt software.  I think this was a bad idea, and I'd like to suggest
> replacing it with something cleaner, for the reasons outlined below.
> 
> 
> Current state
> =============
> 
> Let's take dev-libs/openssl as an example.  This package has three SLOTs
> right now:
> 
>   0.9.8: 0.9.8z_p8-r1
>   1.0.0: 1.0.2q-r200
>   0    : 1.0.2q 1.1.0j 1.1.1a 1.1.1a-r1
> 
> The real package is provided as slot :0, that includes all libraries,
> headers and executables.  Slots 0.9.8 and 1.0.0 only install .so.N*
> libraries that can be used to satisfy dependencies of prebuilt packages.
> 
> 
> Problems with the current state
> ===============================
> 
> Firstly, it is confusing to developers.  Let's analyze the dependencies
> on dev-libs/openssl.  A quick grep reveals seven patterns.  They are
> listed below, along with occurrence counts and percentages:
> 
>   dev-libs/openssl          278    7.8%  }
>   dev-libs/openssl:*         49    1.4%  } 14.2%
>   dev-libs/openssl:=        178    5.0%  }
>   dev-libs/openssl:0        660   18.6%
>   dev-libs/openssl:0=      2381   67.0%
>   dev-libs/openssl:0/0        4    0.1%
>   dev-libs/openssl:0/1.1      2    0.1%
> 
> (note that those are rough measures, not guaranteed to be precise)
> 
> So apparently 14.2% of dependencies allow any slot of OpenSSL which is
> most likely wrong, and 1.4% explicitly claim that's what the package
> wants.  This could be valid only if e.g. the package supported multiple
> ABIs of OpenSSL libraries and used dlopen() with a few possible SONAMEs
> which I honestly doubt any of those packages is doing.
> 
> In other words, 14.2% of dependencies on OpenSSL are plain wrong,
> and 6.4% are wrong in a way that isn't going to be reported by repoman. 
> 1.4% of cases are using ':*' which probably indicates the developer
> decided to silence repoman without understanding how slot operators work
> which is a horrible thing from QA perspective.
> 
> We also have a few cases that require specific OpenSSL subslot (e.g.
> forcing old version into :0 slot) but *none* actually using the binary
> compatibility slots.

I have noticed this and more generally that slot operators are poorly
understood, which is frustrating. I was initially inclined to say that
I think the model still fits and we should educate devs better but...

> Secondly, it is confusing to users.  If we remove old versions and only
> keep binary compatibility slots, users can be easily tricked into
> installing them and being surprised it's not a complete package.  If we
> keep old versions, we end up having different revisions of the same
> version in different slots which is also easily confused.

I can't say I've ever seen this happen but I don't speak to many users.
I'll buy it.

> Thirdly, it is cumbersome to introduce.  If we are to introduce a binary
> compatibility slot for a package that didn't have it, we need to update
> all reverse dependencies.  This usually means researching whether we
> should use ':0' or ':0=', and if we get this wrong, we just silence
> repoman warning about missing slot-op.

This I certainly agree with. It's so bad that it puts you off adding it
at all.

> All of this considered, I can't think of a single real benefit of using
> slots for this purpose.  They work but there's nothing really special
> about them.
> 
> 
> Suggested replacement
> =====================
> 
> My suggestion is to move binary compatibility slots into separate
> packages.  For example, dev-libs/openssl would be split into:
> 
>   dev-libs/openssl  -- containing the actual package
> 
>   dev-libs/openssl-bin-compat  -- containing binary compatibility slots
> 
> In this case, all dependencies on dev-libs/openssl would become correct
> (or semi-correct, wrt missing := dep) again.  Since packages are co-
> installable the same way slots are, there is no loss there.  Since
> nothing depends on binary compatibility slots, we do not even need to
> update anything (but if we had, the update cost would be minimal both to
> developers and to users).
> 
> 
> What do you think?

I'm on board as I have to deal with this a lot in games and I think
there were one or two more on my list to add.

The only downside is that packages requiring what is currently the
latest version would need to be updated later, though I guess you could
use || instead. Take libpng, for example:

|| ( =media-libs/libpng-1.6* media-libs/libpng-bin-compat:1.6 )

Or perhaps?

|| ( media-libs/libpng:0/16 media-libs/libpng-bin-compat:1.6 )

The security team may want to comment on how this will affect GLSAs,
especially as these old versions are frequently vulnerable.

-- 
James Le Cuirot (chewi)
Gentoo Linux Developer

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 981 bytes --]

Re: [gentoo-dev] [RFC] Replacing binary-only SLOTs with separate packages

[Michał Górny]

[-- Attachment #1: Type: text/plain, Size: 3059 bytes --]

On Fri, 2019-01-18 at 22:22 +0000, James Le Cuirot wrote:
> On Fri, 18 Jan 2019 21:13:34 +0100
> Michał Górny <mgorny@gentoo.org> wrote:
> 
> > So apparently 14.2% of dependencies allow any slot of OpenSSL which is
> > most likely wrong, and 1.4% explicitly claim that's what the package
> > wants.  This could be valid only if e.g. the package supported multiple
> > ABIs of OpenSSL libraries and used dlopen() with a few possible SONAMEs
> > which I honestly doubt any of those packages is doing.
> > 
> > In other words, 14.2% of dependencies on OpenSSL are plain wrong,
> > and 6.4% are wrong in a way that isn't going to be reported by repoman. 
> > 1.4% of cases are using ':*' which probably indicates the developer
> > decided to silence repoman without understanding how slot operators work
> > which is a horrible thing from QA perspective.
> > 
> > We also have a few cases that require specific OpenSSL subslot (e.g.
> > forcing old version into :0 slot) but *none* actually using the binary
> > compatibility slots.
> 
> I have noticed this and more generally that slot operators are poorly
> understood, which is frustrating. I was initially inclined to say that
> I think the model still fits and we should educate devs better but...

At some point you realize that the sheer amount of knowledge needed to
contribute to Gentoo is just too large.  You can't expect people to
spend days (finding and) reading documentation for all the corner cases.
I believe that if we can make something more obvious, we should go for
it.

> > Secondly, it is confusing to users.  If we remove old versions and only
> > keep binary compatibility slots, users can be easily tricked into
> > installing them and being surprised it's not a complete package.  If we
> > keep old versions, we end up having different revisions of the same
> > version in different slots which is also easily confused.
> 
> I can't say I've ever seen this happen but I don't speak to many users.
> I'll buy it.

I don't know if it happens, to be honest.  However, I'm going for
the assumptions that there is no reason why a regular user would need to
know the purpose or meaning of different version schemes and slotting
on various packages.

> What do you think?
> 
> I'm on board as I have to deal with this a lot in games and I think
> there were one or two more on my list to add.
> 
> The only downside is that packages requiring what is currently the
> latest version would need to be updated later, though I guess you could
> use || instead. Take libpng, for example:
> 
> > > ( =media-libs/libpng-1.6* media-libs/libpng-bin-compat:1.6 )
> 
> Or perhaps?
> 
> > > ( media-libs/libpng:0/16 media-libs/libpng-bin-compat:1.6 )
> 

This happens with the current model as well, and I don't think '1.6*'
solution is commonly applicable.  For the less friendly cases, you need
stuff like:

  || ( dev-libs/openssl:0/0 dev-libs/openssl:1.0.0 )

which isn't very obvious either.

-- 
Best regards,
Michał Górny

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 963 bytes --]

Re: [gentoo-dev] [RFC] Replacing binary-only SLOTs with separate packages

[Robin H. Johnson]

[-- Attachment #1: Type: text/plain, Size: 1582 bytes --]

On Fri, Jan 18, 2019 at 09:13:34PM +0100, Michał Górny wrote:
> Firstly, it is confusing to developers.  Let's analyze the dependencies
> on dev-libs/openssl.  A quick grep reveals seven patterns.  They are
> listed below, along with occurrence counts and percentages:
> 
>   dev-libs/openssl          278    7.8%  }
>   dev-libs/openssl:*         49    1.4%  } 14.2%
>   dev-libs/openssl:=        178    5.0%  }
>   dev-libs/openssl:0        660   18.6%
>   dev-libs/openssl:0=      2381   67.0%
>   dev-libs/openssl:0/0        4    0.1%
>   dev-libs/openssl:0/1.1      2    0.1%
This was based just on ebuilds right?

> So apparently 14.2% of dependencies allow any slot of OpenSSL which is
> most likely wrong, and 1.4% explicitly claim that's what the package
> wants.  This could be valid only if e.g. the package supported multiple
> ABIs of OpenSSL libraries and used dlopen() with a few possible SONAMEs
> which I honestly doubt any of those packages is doing.
There's a valid case for accepting ANY openssl: tooling that explicitly
calls the binary tools provided by OpenSSL, and does link or dlopen any
of the openssl libraries.

Such usage has to be careful, because it could depend on OpenSSL
compile-time options, like 'srp', which used to depend on USE=bindist.

Your solution however will also improve this case.

-- 
Robin Hugh Johnson
Gentoo Linux: Dev, Infra Lead, Foundation Treasurer
E-Mail   : robbat2@gentoo.org
GnuPG FP : 11ACBA4F 4778E3F6 E4EDF38E B27B944E 34884E85
GnuPG FP : 7D0B3CEB E9B85B1F 825BCECF EE05E6F6 A48F6136

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 1113 bytes --]

Re: [gentoo-dev] [RFC] Replacing binary-only SLOTs with separate packages

[James Le Cuirot]

[-- Attachment #1: Type: text/plain, Size: 809 bytes --]

On Fri, 18 Jan 2019 23:31:39 +0000
"Robin H. Johnson" <robbat2@gentoo.org> wrote:

> > So apparently 14.2% of dependencies allow any slot of OpenSSL which is
> > most likely wrong, and 1.4% explicitly claim that's what the package
> > wants.  This could be valid only if e.g. the package supported multiple
> > ABIs of OpenSSL libraries and used dlopen() with a few possible SONAMEs
> > which I honestly doubt any of those packages is doing.  
>
> There's a valid case for accepting ANY openssl: tooling that explicitly
> calls the binary tools provided by OpenSSL, and does link or dlopen any
> of the openssl libraries.

The binary-only SLOTs don't include those tools because they would
conflict. Library-only is a more accurate term.

-- 
James Le Cuirot (chewi)
Gentoo Linux Developer

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 981 bytes --]

Re: [gentoo-dev] [RFC] Replacing binary-only SLOTs with separate packages

[Michał Górny]

[-- Attachment #1: Type: text/plain, Size: 929 bytes --]

On Fri, 2019-01-18 at 23:31 +0000, Robin H. Johnson wrote:
> On Fri, Jan 18, 2019 at 09:13:34PM +0100, Michał Górny wrote:
> > Firstly, it is confusing to developers.  Let's analyze the dependencies
> > on dev-libs/openssl.  A quick grep reveals seven patterns.  They are
> > listed below, along with occurrence counts and percentages:
> > 
> >   dev-libs/openssl          278    7.8%  }
> >   dev-libs/openssl:*         49    1.4%  } 14.2%
> >   dev-libs/openssl:=        178    5.0%  }
> >   dev-libs/openssl:0        660   18.6%
> >   dev-libs/openssl:0=      2381   67.0%
> >   dev-libs/openssl:0/0        4    0.1%
> >   dev-libs/openssl:0/1.1      2    0.1%
> 
> This was based just on ebuilds right?

On md5-cache, to be more precise.  The numbers and percentages are
on total dependency occurrences (i.e. if an ebuild lists openssl three
times, it counts as 3).

-- 
Best regards,
Michał Górny

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 963 bytes --]

[gentoo-dev] Re: [RFC] Replacing binary-only SLOTs with separate packages

[Ulrich Mueller]

[-- Attachment #1: Type: text/plain, Size: 846 bytes --]

>>>>> On Fri, 18 Jan 2019, Michał Górny wrote:

> My suggestion is to move binary compatibility slots into separate
> packages.  For example, dev-libs/openssl would be split into:

>   dev-libs/openssl  -- containing the actual package

>   dev-libs/openssl-bin-compat  -- containing binary compatibility slots

A long time ago, that scheme was used, but around 2010 is was changed
to the current slot-based solution. For example, there used to be
media-libs/jpeg-compat and x11-libs/openmotif-compat, and these were
moved to compatibility slots of the main package.

So before reverting that change, I would suggest to look up the old
discussion, in order to find out if there was a valid reason for it.

Also the "bin" is confusing, if the package is compiled from source.
Why not simply dev-libs/openssl-compat?

Ulrich

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 487 bytes --]

Re: [gentoo-dev] [RFC] Replacing binary-only SLOTs with separate packages

[Mart Raudsepp]

[-- Attachment #1: Type: text/plain, Size: 4426 bytes --]

Ühel kenal päeval, R, 18.01.2019 kell 21:13, kirjutas Michał Górny:
> Hello,
> 
> Since I've been working on the early gx86-multilib, we've been using
> 'binary-only SLOTs' to support providing old versions of libraries
> for
> prebuilt software.  I think this was a bad idea, and I'd like to
> suggest
> replacing it with something cleaner, for the reasons outlined below.
> 
> 
> Current state
> =============
> 
> Let's take dev-libs/openssl as an example.  This package has three
> SLOTs
> right now:
> 
>   0.9.8: 0.9.8z_p8-r1
>   1.0.0: 1.0.2q-r200
>   0    : 1.0.2q 1.1.0j 1.1.1a 1.1.1a-r1
> 
> The real package is provided as slot :0, that includes all libraries,
> headers and executables.  Slots 0.9.8 and 1.0.0 only install .so.N*
> libraries that can be used to satisfy dependencies of prebuilt
> packages.
> 
> 
> Problems with the current state
> ===============================
> 
> Firstly, it is confusing to developers.  Let's analyze the
> dependencies
> on dev-libs/openssl.  A quick grep reveals seven patterns.  They are
> listed below, along with occurrence counts and percentages:
> 
>   dev-libs/openssl          278    7.8%  }
>   dev-libs/openssl:*         49    1.4%  } 14.2%
>   dev-libs/openssl:=        178    5.0%  }
>   dev-libs/openssl:0        660   18.6%
>   dev-libs/openssl:0=      2381   67.0%
>   dev-libs/openssl:0/0        4    0.1%
>   dev-libs/openssl:0/1.1      2    0.1%
> 
> (note that those are rough measures, not guaranteed to be precise)
> 
> So apparently 14.2% of dependencies allow any slot of OpenSSL which
> is
> most likely wrong, and 1.4% explicitly claim that's what the package
> wants.  This could be valid only if e.g. the package supported
> multiple
> ABIs of OpenSSL libraries and used dlopen() with a few possible
> SONAMEs
> which I honestly doubt any of those packages is doing.
> 
> In other words, 14.2% of dependencies on OpenSSL are plain wrong,
> and 6.4% are wrong in a way that isn't going to be reported by
> repoman. 
> 1.4% of cases are using ':*' which probably indicates the developer
> decided to silence repoman without understanding how slot operators
> work
> which is a horrible thing from QA perspective.
> 
> We also have a few cases that require specific OpenSSL subslot (e.g.
> forcing old version into :0 slot) but *none* actually using the
> binary
> compatibility slots.
> 
> 
> Secondly, it is confusing to users.  If we remove old versions and
> only
> keep binary compatibility slots, users can be easily tricked into
> installing them and being surprised it's not a complete package.  If
> we
> keep old versions, we end up having different revisions of the same
> version in different slots which is also easily confused.
> 
> 
> Thirdly, it is cumbersome to introduce.  If we are to introduce a
> binary
> compatibility slot for a package that didn't have it, we need to
> update
> all reverse dependencies.  This usually means researching whether we
> should use ':0' or ':0=', and if we get this wrong, we just silence
> repoman warning about missing slot-op.
> 
> 
> All of this considered, I can't think of a single real benefit of
> using
> slots for this purpose.  They work but there's nothing really special
> about them.
> 
> 
> Suggested replacement
> =====================
> 
> My suggestion is to move binary compatibility slots into separate
> packages.  For example, dev-libs/openssl would be split into:
> 
>   dev-libs/openssl  -- containing the actual package
> 
>   dev-libs/openssl-bin-compat  -- containing binary compatibility
> slots
> 
> In this case, all dependencies on dev-libs/openssl would become
> correct
> (or semi-correct, wrt missing := dep) again.  Since packages are co-
> installable the same way slots are, there is no loss there.  Since
> nothing depends on binary compatibility slots, we do not even need to
> update anything (but if we had, the update cost would be minimal both
> to
> developers and to users).
> 
> 
> What do you think?

I can only find bikeshed painting for the naming of the legacy
packages, so I guess sounds good. Though that historical digging about
why we moved away from it may be useful, if just to find that it was
just a "SLOTs are cool, lets use them" case, to strengthen the argument
to move back to separate package again.


Mart

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 981 bytes --]

Re: [gentoo-dev] [RFC] Replacing binary-only SLOTs with separate packages

[Andrew Savchenko]

[-- Attachment #1: Type: text/plain, Size: 4982 bytes --]

On Fri, 18 Jan 2019 21:13:34 +0100 Michał Górny wrote:
> Hello,
> 
> Since I've been working on the early gx86-multilib, we've been using
> 'binary-only SLOTs' to support providing old versions of libraries for
> prebuilt software.  I think this was a bad idea, and I'd like to suggest
> replacing it with something cleaner, for the reasons outlined below.
> 
> 
> Current state
> =============
> 
> Let's take dev-libs/openssl as an example.  This package has three SLOTs
> right now:
> 
>   0.9.8: 0.9.8z_p8-r1
>   1.0.0: 1.0.2q-r200
>   0    : 1.0.2q 1.1.0j 1.1.1a 1.1.1a-r1
> 
> The real package is provided as slot :0, that includes all libraries,
> headers and executables.  Slots 0.9.8 and 1.0.0 only install .so.N*
> libraries that can be used to satisfy dependencies of prebuilt packages.
> 
> 
> Problems with the current state
> ===============================
> 
> Firstly, it is confusing to developers.  Let's analyze the dependencies
> on dev-libs/openssl.  A quick grep reveals seven patterns.  They are
> listed below, along with occurrence counts and percentages:
> 
>   dev-libs/openssl          278    7.8%  }
>   dev-libs/openssl:*         49    1.4%  } 14.2%
>   dev-libs/openssl:=        178    5.0%  }
>   dev-libs/openssl:0        660   18.6%
>   dev-libs/openssl:0=      2381   67.0%
>   dev-libs/openssl:0/0        4    0.1%
>   dev-libs/openssl:0/1.1      2    0.1%
> 
> (note that those are rough measures, not guaranteed to be precise)
> 
> So apparently 14.2% of dependencies allow any slot of OpenSSL which is
> most likely wrong, and 1.4% explicitly claim that's what the package
> wants.  This could be valid only if e.g. the package supported multiple
> ABIs of OpenSSL libraries and used dlopen() with a few possible SONAMEs
> which I honestly doubt any of those packages is doing.
> 
> In other words, 14.2% of dependencies on OpenSSL are plain wrong,
> and 6.4% are wrong in a way that isn't going to be reported by repoman. 
> 1.4% of cases are using ':*' which probably indicates the developer
> decided to silence repoman without understanding how slot operators work
> which is a horrible thing from QA perspective.
> 
> We also have a few cases that require specific OpenSSL subslot (e.g.
> forcing old version into :0 slot) but *none* actually using the binary
> compatibility slots.
> 
> 
> Secondly, it is confusing to users.  If we remove old versions and only
> keep binary compatibility slots, users can be easily tricked into
> installing them and being surprised it's not a complete package.  If we
> keep old versions, we end up having different revisions of the same
> version in different slots which is also easily confused.
> 
> 
> Thirdly, it is cumbersome to introduce.  If we are to introduce a binary
> compatibility slot for a package that didn't have it, we need to update
> all reverse dependencies.  This usually means researching whether we
> should use ':0' or ':0=', and if we get this wrong, we just silence
> repoman warning about missing slot-op.
> 
> 
> All of this considered, I can't think of a single real benefit of using
> slots for this purpose.  They work but there's nothing really special
> about them.
> 
> 
> Suggested replacement
> =====================
> 
> My suggestion is to move binary compatibility slots into separate
> packages.  For example, dev-libs/openssl would be split into:
> 
>   dev-libs/openssl  -- containing the actual package
> 
>   dev-libs/openssl-bin-compat  -- containing binary compatibility slots
> 
> In this case, all dependencies on dev-libs/openssl would become correct
> (or semi-correct, wrt missing := dep) again.  Since packages are co-
> installable the same way slots are, there is no loss there.  Since
> nothing depends on binary compatibility slots, we do not even need to
> update anything (but if we had, the update cost would be minimal both to
> developers and to users).
> 
> 
> What do you think?

I do not like the idea. Slots are very elegant and effective
mechanism and is one of the points where Gentoo outruns other
distributions. Proposed approach with compat packages will
effectively disable slots for most cases.

Also proposed change will create a lot of unneeded technical
difficulties in maintaining packages: there will be double ebuilds
to maintain instead of a single one, ${PN} references will be
broken without additional changes and so far and so on. A lot of
unnecessary rebuilds will happen due slot to package moves.

Aside from development questions for me in a role of sysadmin slots
are much easier and understandable to manage than zoo of *compat*
packages in other distributions.

Assumption that :* is always wrong is invalid, since there are
valid cases: there are apps supporting various API versions or
using tools/data files without any care from where they are coming.

Best regards,
Andrew Savchenko

[-- Attachment #2: Type: application/pgp-signature, Size: 833 bytes --]

Re: [gentoo-dev] [RFC] Replacing binary-only SLOTs with separate packages

[Michał Górny]

[-- Attachment #1: Type: text/plain, Size: 6265 bytes --]

On Sat, 2019-01-19 at 17:37 +0300, Andrew Savchenko wrote:
> On Fri, 18 Jan 2019 21:13:34 +0100 Michał Górny wrote:
> > Hello,
> > 
> > Since I've been working on the early gx86-multilib, we've been using
> > 'binary-only SLOTs' to support providing old versions of libraries for
> > prebuilt software.  I think this was a bad idea, and I'd like to suggest
> > replacing it with something cleaner, for the reasons outlined below.
> > 
> > 
> > Current state
> > =============
> > 
> > Let's take dev-libs/openssl as an example.  This package has three SLOTs
> > right now:
> > 
> >   0.9.8: 0.9.8z_p8-r1
> >   1.0.0: 1.0.2q-r200
> >   0    : 1.0.2q 1.1.0j 1.1.1a 1.1.1a-r1
> > 
> > The real package is provided as slot :0, that includes all libraries,
> > headers and executables.  Slots 0.9.8 and 1.0.0 only install .so.N*
> > libraries that can be used to satisfy dependencies of prebuilt packages.
> > 
> > 
> > Problems with the current state
> > ===============================
> > 
> > Firstly, it is confusing to developers.  Let's analyze the dependencies
> > on dev-libs/openssl.  A quick grep reveals seven patterns.  They are
> > listed below, along with occurrence counts and percentages:
> > 
> >   dev-libs/openssl          278    7.8%  }
> >   dev-libs/openssl:*         49    1.4%  } 14.2%
> >   dev-libs/openssl:=        178    5.0%  }
> >   dev-libs/openssl:0        660   18.6%
> >   dev-libs/openssl:0=      2381   67.0%
> >   dev-libs/openssl:0/0        4    0.1%
> >   dev-libs/openssl:0/1.1      2    0.1%
> > 
> > (note that those are rough measures, not guaranteed to be precise)
> > 
> > So apparently 14.2% of dependencies allow any slot of OpenSSL which is
> > most likely wrong, and 1.4% explicitly claim that's what the package
> > wants.  This could be valid only if e.g. the package supported multiple
> > ABIs of OpenSSL libraries and used dlopen() with a few possible SONAMEs
> > which I honestly doubt any of those packages is doing.
> > 
> > In other words, 14.2% of dependencies on OpenSSL are plain wrong,
> > and 6.4% are wrong in a way that isn't going to be reported by repoman. 
> > 1.4% of cases are using ':*' which probably indicates the developer
> > decided to silence repoman without understanding how slot operators work
> > which is a horrible thing from QA perspective.
> > 
> > We also have a few cases that require specific OpenSSL subslot (e.g.
> > forcing old version into :0 slot) but *none* actually using the binary
> > compatibility slots.
> > 
> > 
> > Secondly, it is confusing to users.  If we remove old versions and only
> > keep binary compatibility slots, users can be easily tricked into
> > installing them and being surprised it's not a complete package.  If we
> > keep old versions, we end up having different revisions of the same
> > version in different slots which is also easily confused.
> > 
> > 
> > Thirdly, it is cumbersome to introduce.  If we are to introduce a binary
> > compatibility slot for a package that didn't have it, we need to update
> > all reverse dependencies.  This usually means researching whether we
> > should use ':0' or ':0=', and if we get this wrong, we just silence
> > repoman warning about missing slot-op.
> > 
> > 
> > All of this considered, I can't think of a single real benefit of using
> > slots for this purpose.  They work but there's nothing really special
> > about them.
> > 
> > 
> > Suggested replacement
> > =====================
> > 
> > My suggestion is to move binary compatibility slots into separate
> > packages.  For example, dev-libs/openssl would be split into:
> > 
> >   dev-libs/openssl  -- containing the actual package
> > 
> >   dev-libs/openssl-bin-compat  -- containing binary compatibility slots
> > 
> > In this case, all dependencies on dev-libs/openssl would become correct
> > (or semi-correct, wrt missing := dep) again.  Since packages are co-
> > installable the same way slots are, there is no loss there.  Since
> > nothing depends on binary compatibility slots, we do not even need to
> > update anything (but if we had, the update cost would be minimal both to
> > developers and to users).
> > 
> > 
> > What do you think?
> 
> I do not like the idea. Slots are very elegant and effective
> mechanism and is one of the points where Gentoo outruns other
> distributions. Proposed approach with compat packages will
> effectively disable slots for most cases.

You haven't brought a single argument as to how slots are better than
compat packages.  In fact, it's not even clear if you're talking about
the same use of slots that I am.

> Also proposed change will create a lot of unneeded technical
> difficulties in maintaining packages: there will be double ebuilds
> to maintain instead of a single one,

I don't see how this would change the effective number of ebuilds.  You
can't produce two slots out of a single ebuild, so you always have to
duplicate them.

>  ${PN} references will be
> broken without additional changes and so far and so on. A lot of
> unnecessary rebuilds will happen due slot to package moves.

Are you saying that there are 'lots' of binary compatibility slots being
currently used right now?  Could you support this claim with actual
numbers or examples?

> Aside from development questions for me in a role of sysadmin slots
> are much easier and understandable to manage than zoo of *compat*
> packages in other distributions.

I don't see how binary compatibility slots are even relevant to
sysadmins.  Their only purpose is to satisfy dependencies of other
packages, so sysadmin should never have to care about them.

> Assumption that :* is always wrong is invalid, since there are
> valid cases: there are apps supporting various API versions

Binary compatibility slots are used for ABI incompatibility, not any API
incompatibility.  Also, I have the feeling those 'various API versions'
require headers to build.

>  or
> using tools/data files without any care from where they are coming.

Tools aren't included in binary compatibility slots.  As James already
pointed out in this thread.

-- 
Best regards,
Michał Górny

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 963 bytes --]

Re: [gentoo-dev] [RFC] Replacing binary-only SLOTs with separate packages

[Marc Schiffbauer]

[-- Attachment #1: Type: text/plain, Size: 1552 bytes --]

* Michał Górny schrieb am 19.01.19 um 17:25 Uhr:
> On Sat, 2019-01-19 at 17:37 +0300, Andrew Savchenko wrote:
> > I do not like the idea. Slots are very elegant and effective
> > mechanism and is one of the points where Gentoo outruns other
> > distributions. Proposed approach with compat packages will
> > effectively disable slots for most cases.
> 
> You haven't brought a single argument as to how slots are better than
> compat packages.  In fact, it's not even clear if you're talking about
> the same use of slots that I am.

One was, that you have one package (ebuild directory) for each piece of 
software. Unlike other distributions which have lots of different 
packages all for the same source an many cases.

So I'd agree that its the more gentooish way to have it all in one 
ebuild directory.

That having said I agree that current use of compat slots might be 
somewhat confusing even to developers.

So why not improve usability here?

I think it would increase the understanding of compat slots if we
changed the naming of thoss slots.

How about this:

dev-libs/openssl
     (0.9.8-compat) 0.9.8u 0.9.8v 0.9.8w 0.9.8x ~0.9.8y 0.9.8z_p8-r1
     (0)            1.0.0h 1.0.0i […] [M]~1.1.1a [M]~1.1.1a-r1
     (1.0.0-compat) ~1.0.2q-r200


I can imagine that this might improve the situation a lot.

Of cource thet woult still be the job to fix ebuilds where dependencies
are wrong.

-Marc

-- 
0xCA3E7BF67F979BE5 - F7FB 78F7 7CC3 79F6 DF07
                     6E9E CA3E 7BF6 7F97 9BE5

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]