* Re: [gentoo-dev] [RFC] Replacing binary-only SLOTs with separate packages
2019-01-18 20:13 [gentoo-dev] [RFC] Replacing binary-only SLOTs with separate packages Michał Górny
@ 2019-01-18 22:22 ` James Le Cuirot
2019-01-18 22:35 ` Michał Górny
2019-01-18 23:31 ` Robin H. Johnson
` (3 subsequent siblings)
4 siblings, 1 reply; 11+ messages in thread
From: James Le Cuirot @ 2019-01-18 22:22 UTC (permalink / raw
To: gentoo-dev
[-- Attachment #1: Type: text/plain, Size: 5059 bytes --]
On Fri, 18 Jan 2019 21:13:34 +0100
Michał Górny <mgorny@gentoo.org> wrote:
> Hello,
>
> Since I've been working on the early gx86-multilib, we've been using
> 'binary-only SLOTs' to support providing old versions of libraries for
> prebuilt software. I think this was a bad idea, and I'd like to suggest
> replacing it with something cleaner, for the reasons outlined below.
>
>
> Current state
> =============
>
> Let's take dev-libs/openssl as an example. This package has three SLOTs
> right now:
>
> 0.9.8: 0.9.8z_p8-r1
> 1.0.0: 1.0.2q-r200
> 0 : 1.0.2q 1.1.0j 1.1.1a 1.1.1a-r1
>
> The real package is provided as slot :0, that includes all libraries,
> headers and executables. Slots 0.9.8 and 1.0.0 only install .so.N*
> libraries that can be used to satisfy dependencies of prebuilt packages.
>
>
> Problems with the current state
> ===============================
>
> Firstly, it is confusing to developers. Let's analyze the dependencies
> on dev-libs/openssl. A quick grep reveals seven patterns. They are
> listed below, along with occurrence counts and percentages:
>
> dev-libs/openssl 278 7.8% }
> dev-libs/openssl:* 49 1.4% } 14.2%
> dev-libs/openssl:= 178 5.0% }
> dev-libs/openssl:0 660 18.6%
> dev-libs/openssl:0= 2381 67.0%
> dev-libs/openssl:0/0 4 0.1%
> dev-libs/openssl:0/1.1 2 0.1%
>
> (note that those are rough measures, not guaranteed to be precise)
>
> So apparently 14.2% of dependencies allow any slot of OpenSSL which is
> most likely wrong, and 1.4% explicitly claim that's what the package
> wants. This could be valid only if e.g. the package supported multiple
> ABIs of OpenSSL libraries and used dlopen() with a few possible SONAMEs
> which I honestly doubt any of those packages is doing.
>
> In other words, 14.2% of dependencies on OpenSSL are plain wrong,
> and 6.4% are wrong in a way that isn't going to be reported by repoman.
> 1.4% of cases are using ':*' which probably indicates the developer
> decided to silence repoman without understanding how slot operators work
> which is a horrible thing from QA perspective.
>
> We also have a few cases that require specific OpenSSL subslot (e.g.
> forcing old version into :0 slot) but *none* actually using the binary
> compatibility slots.
I have noticed this and more generally that slot operators are poorly
understood, which is frustrating. I was initially inclined to say that
I think the model still fits and we should educate devs better but...
> Secondly, it is confusing to users. If we remove old versions and only
> keep binary compatibility slots, users can be easily tricked into
> installing them and being surprised it's not a complete package. If we
> keep old versions, we end up having different revisions of the same
> version in different slots which is also easily confused.
I can't say I've ever seen this happen but I don't speak to many users.
I'll buy it.
> Thirdly, it is cumbersome to introduce. If we are to introduce a binary
> compatibility slot for a package that didn't have it, we need to update
> all reverse dependencies. This usually means researching whether we
> should use ':0' or ':0=', and if we get this wrong, we just silence
> repoman warning about missing slot-op.
This I certainly agree with. It's so bad that it puts you off adding it
at all.
> All of this considered, I can't think of a single real benefit of using
> slots for this purpose. They work but there's nothing really special
> about them.
>
>
> Suggested replacement
> =====================
>
> My suggestion is to move binary compatibility slots into separate
> packages. For example, dev-libs/openssl would be split into:
>
> dev-libs/openssl -- containing the actual package
>
> dev-libs/openssl-bin-compat -- containing binary compatibility slots
>
> In this case, all dependencies on dev-libs/openssl would become correct
> (or semi-correct, wrt missing := dep) again. Since packages are co-
> installable the same way slots are, there is no loss there. Since
> nothing depends on binary compatibility slots, we do not even need to
> update anything (but if we had, the update cost would be minimal both to
> developers and to users).
>
>
> What do you think?
I'm on board as I have to deal with this a lot in games and I think
there were one or two more on my list to add.
The only downside is that packages requiring what is currently the
latest version would need to be updated later, though I guess you could
use || instead. Take libpng, for example:
|| ( =media-libs/libpng-1.6* media-libs/libpng-bin-compat:1.6 )
Or perhaps?
|| ( media-libs/libpng:0/16 media-libs/libpng-bin-compat:1.6 )
The security team may want to comment on how this will affect GLSAs,
especially as these old versions are frequently vulnerable.
--
James Le Cuirot (chewi)
Gentoo Linux Developer
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 981 bytes --]
^ permalink raw reply [flat|nested] 11+ messages in thread* Re: [gentoo-dev] [RFC] Replacing binary-only SLOTs with separate packages
2019-01-18 22:22 ` James Le Cuirot
@ 2019-01-18 22:35 ` Michał Górny
0 siblings, 0 replies; 11+ messages in thread
From: Michał Górny @ 2019-01-18 22:35 UTC (permalink / raw
To: gentoo-dev
[-- Attachment #1: Type: text/plain, Size: 3059 bytes --]
On Fri, 2019-01-18 at 22:22 +0000, James Le Cuirot wrote:
> On Fri, 18 Jan 2019 21:13:34 +0100
> Michał Górny <mgorny@gentoo.org> wrote:
>
> > So apparently 14.2% of dependencies allow any slot of OpenSSL which is
> > most likely wrong, and 1.4% explicitly claim that's what the package
> > wants. This could be valid only if e.g. the package supported multiple
> > ABIs of OpenSSL libraries and used dlopen() with a few possible SONAMEs
> > which I honestly doubt any of those packages is doing.
> >
> > In other words, 14.2% of dependencies on OpenSSL are plain wrong,
> > and 6.4% are wrong in a way that isn't going to be reported by repoman.
> > 1.4% of cases are using ':*' which probably indicates the developer
> > decided to silence repoman without understanding how slot operators work
> > which is a horrible thing from QA perspective.
> >
> > We also have a few cases that require specific OpenSSL subslot (e.g.
> > forcing old version into :0 slot) but *none* actually using the binary
> > compatibility slots.
>
> I have noticed this and more generally that slot operators are poorly
> understood, which is frustrating. I was initially inclined to say that
> I think the model still fits and we should educate devs better but...
At some point you realize that the sheer amount of knowledge needed to
contribute to Gentoo is just too large. You can't expect people to
spend days (finding and) reading documentation for all the corner cases.
I believe that if we can make something more obvious, we should go for
it.
> > Secondly, it is confusing to users. If we remove old versions and only
> > keep binary compatibility slots, users can be easily tricked into
> > installing them and being surprised it's not a complete package. If we
> > keep old versions, we end up having different revisions of the same
> > version in different slots which is also easily confused.
>
> I can't say I've ever seen this happen but I don't speak to many users.
> I'll buy it.
I don't know if it happens, to be honest. However, I'm going for
the assumptions that there is no reason why a regular user would need to
know the purpose or meaning of different version schemes and slotting
on various packages.
> What do you think?
>
> I'm on board as I have to deal with this a lot in games and I think
> there were one or two more on my list to add.
>
> The only downside is that packages requiring what is currently the
> latest version would need to be updated later, though I guess you could
> use || instead. Take libpng, for example:
>
> > > ( =media-libs/libpng-1.6* media-libs/libpng-bin-compat:1.6 )
>
> Or perhaps?
>
> > > ( media-libs/libpng:0/16 media-libs/libpng-bin-compat:1.6 )
>
This happens with the current model as well, and I don't think '1.6*'
solution is commonly applicable. For the less friendly cases, you need
stuff like:
|| ( dev-libs/openssl:0/0 dev-libs/openssl:1.0.0 )
which isn't very obvious either.
--
Best regards,
Michał Górny
[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 963 bytes --]
^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: [gentoo-dev] [RFC] Replacing binary-only SLOTs with separate packages
2019-01-18 20:13 [gentoo-dev] [RFC] Replacing binary-only SLOTs with separate packages Michał Górny
2019-01-18 22:22 ` James Le Cuirot
@ 2019-01-18 23:31 ` Robin H. Johnson
2019-01-18 23:36 ` James Le Cuirot
2019-01-19 7:55 ` Michał Górny
2019-01-19 9:58 ` [gentoo-dev] " Ulrich Mueller
` (2 subsequent siblings)
4 siblings, 2 replies; 11+ messages in thread
From: Robin H. Johnson @ 2019-01-18 23:31 UTC (permalink / raw
To: gentoo-dev; +Cc: multilib, qa
[-- Attachment #1: Type: text/plain, Size: 1582 bytes --]
On Fri, Jan 18, 2019 at 09:13:34PM +0100, Michał Górny wrote:
> Firstly, it is confusing to developers. Let's analyze the dependencies
> on dev-libs/openssl. A quick grep reveals seven patterns. They are
> listed below, along with occurrence counts and percentages:
>
> dev-libs/openssl 278 7.8% }
> dev-libs/openssl:* 49 1.4% } 14.2%
> dev-libs/openssl:= 178 5.0% }
> dev-libs/openssl:0 660 18.6%
> dev-libs/openssl:0= 2381 67.0%
> dev-libs/openssl:0/0 4 0.1%
> dev-libs/openssl:0/1.1 2 0.1%
This was based just on ebuilds right?
> So apparently 14.2% of dependencies allow any slot of OpenSSL which is
> most likely wrong, and 1.4% explicitly claim that's what the package
> wants. This could be valid only if e.g. the package supported multiple
> ABIs of OpenSSL libraries and used dlopen() with a few possible SONAMEs
> which I honestly doubt any of those packages is doing.
There's a valid case for accepting ANY openssl: tooling that explicitly
calls the binary tools provided by OpenSSL, and does link or dlopen any
of the openssl libraries.
Such usage has to be careful, because it could depend on OpenSSL
compile-time options, like 'srp', which used to depend on USE=bindist.
Your solution however will also improve this case.
--
Robin Hugh Johnson
Gentoo Linux: Dev, Infra Lead, Foundation Treasurer
E-Mail : robbat2@gentoo.org
GnuPG FP : 11ACBA4F 4778E3F6 E4EDF38E B27B944E 34884E85
GnuPG FP : 7D0B3CEB E9B85B1F 825BCECF EE05E6F6 A48F6136
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 1113 bytes --]
^ permalink raw reply [flat|nested] 11+ messages in thread* Re: [gentoo-dev] [RFC] Replacing binary-only SLOTs with separate packages
2019-01-18 23:31 ` Robin H. Johnson
@ 2019-01-18 23:36 ` James Le Cuirot
2019-01-19 7:55 ` Michał Górny
1 sibling, 0 replies; 11+ messages in thread
From: James Le Cuirot @ 2019-01-18 23:36 UTC (permalink / raw
To: gentoo-dev
[-- Attachment #1: Type: text/plain, Size: 809 bytes --]
On Fri, 18 Jan 2019 23:31:39 +0000
"Robin H. Johnson" <robbat2@gentoo.org> wrote:
> > So apparently 14.2% of dependencies allow any slot of OpenSSL which is
> > most likely wrong, and 1.4% explicitly claim that's what the package
> > wants. This could be valid only if e.g. the package supported multiple
> > ABIs of OpenSSL libraries and used dlopen() with a few possible SONAMEs
> > which I honestly doubt any of those packages is doing.
>
> There's a valid case for accepting ANY openssl: tooling that explicitly
> calls the binary tools provided by OpenSSL, and does link or dlopen any
> of the openssl libraries.
The binary-only SLOTs don't include those tools because they would
conflict. Library-only is a more accurate term.
--
James Le Cuirot (chewi)
Gentoo Linux Developer
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 981 bytes --]
^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: [gentoo-dev] [RFC] Replacing binary-only SLOTs with separate packages
2019-01-18 23:31 ` Robin H. Johnson
2019-01-18 23:36 ` James Le Cuirot
@ 2019-01-19 7:55 ` Michał Górny
1 sibling, 0 replies; 11+ messages in thread
From: Michał Górny @ 2019-01-19 7:55 UTC (permalink / raw
To: gentoo-dev; +Cc: multilib, qa
[-- Attachment #1: Type: text/plain, Size: 929 bytes --]
On Fri, 2019-01-18 at 23:31 +0000, Robin H. Johnson wrote:
> On Fri, Jan 18, 2019 at 09:13:34PM +0100, Michał Górny wrote:
> > Firstly, it is confusing to developers. Let's analyze the dependencies
> > on dev-libs/openssl. A quick grep reveals seven patterns. They are
> > listed below, along with occurrence counts and percentages:
> >
> > dev-libs/openssl 278 7.8% }
> > dev-libs/openssl:* 49 1.4% } 14.2%
> > dev-libs/openssl:= 178 5.0% }
> > dev-libs/openssl:0 660 18.6%
> > dev-libs/openssl:0= 2381 67.0%
> > dev-libs/openssl:0/0 4 0.1%
> > dev-libs/openssl:0/1.1 2 0.1%
>
> This was based just on ebuilds right?
On md5-cache, to be more precise. The numbers and percentages are
on total dependency occurrences (i.e. if an ebuild lists openssl three
times, it counts as 3).
--
Best regards,
Michał Górny
[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 963 bytes --]
^ permalink raw reply [flat|nested] 11+ messages in thread
* [gentoo-dev] Re: [RFC] Replacing binary-only SLOTs with separate packages
2019-01-18 20:13 [gentoo-dev] [RFC] Replacing binary-only SLOTs with separate packages Michał Górny
2019-01-18 22:22 ` James Le Cuirot
2019-01-18 23:31 ` Robin H. Johnson
@ 2019-01-19 9:58 ` Ulrich Mueller
2019-01-19 11:14 ` [gentoo-dev] " Mart Raudsepp
2019-01-19 14:37 ` Andrew Savchenko
4 siblings, 0 replies; 11+ messages in thread
From: Ulrich Mueller @ 2019-01-19 9:58 UTC (permalink / raw
To: Michał Górny; +Cc: gentoo-dev, multilib, qa
[-- Attachment #1: Type: text/plain, Size: 846 bytes --]
>>>>> On Fri, 18 Jan 2019, Michał Górny wrote:
> My suggestion is to move binary compatibility slots into separate
> packages. For example, dev-libs/openssl would be split into:
> dev-libs/openssl -- containing the actual package
> dev-libs/openssl-bin-compat -- containing binary compatibility slots
A long time ago, that scheme was used, but around 2010 is was changed
to the current slot-based solution. For example, there used to be
media-libs/jpeg-compat and x11-libs/openmotif-compat, and these were
moved to compatibility slots of the main package.
So before reverting that change, I would suggest to look up the old
discussion, in order to find out if there was a valid reason for it.
Also the "bin" is confusing, if the package is compiled from source.
Why not simply dev-libs/openssl-compat?
Ulrich
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 487 bytes --]
^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: [gentoo-dev] [RFC] Replacing binary-only SLOTs with separate packages
2019-01-18 20:13 [gentoo-dev] [RFC] Replacing binary-only SLOTs with separate packages Michał Górny
` (2 preceding siblings ...)
2019-01-19 9:58 ` [gentoo-dev] " Ulrich Mueller
@ 2019-01-19 11:14 ` Mart Raudsepp
2019-01-19 14:37 ` Andrew Savchenko
4 siblings, 0 replies; 11+ messages in thread
From: Mart Raudsepp @ 2019-01-19 11:14 UTC (permalink / raw
To: gentoo-dev; +Cc: multilib, qa
[-- Attachment #1: Type: text/plain, Size: 4426 bytes --]
Ühel kenal päeval, R, 18.01.2019 kell 21:13, kirjutas Michał Górny:
> Hello,
>
> Since I've been working on the early gx86-multilib, we've been using
> 'binary-only SLOTs' to support providing old versions of libraries
> for
> prebuilt software. I think this was a bad idea, and I'd like to
> suggest
> replacing it with something cleaner, for the reasons outlined below.
>
>
> Current state
> =============
>
> Let's take dev-libs/openssl as an example. This package has three
> SLOTs
> right now:
>
> 0.9.8: 0.9.8z_p8-r1
> 1.0.0: 1.0.2q-r200
> 0 : 1.0.2q 1.1.0j 1.1.1a 1.1.1a-r1
>
> The real package is provided as slot :0, that includes all libraries,
> headers and executables. Slots 0.9.8 and 1.0.0 only install .so.N*
> libraries that can be used to satisfy dependencies of prebuilt
> packages.
>
>
> Problems with the current state
> ===============================
>
> Firstly, it is confusing to developers. Let's analyze the
> dependencies
> on dev-libs/openssl. A quick grep reveals seven patterns. They are
> listed below, along with occurrence counts and percentages:
>
> dev-libs/openssl 278 7.8% }
> dev-libs/openssl:* 49 1.4% } 14.2%
> dev-libs/openssl:= 178 5.0% }
> dev-libs/openssl:0 660 18.6%
> dev-libs/openssl:0= 2381 67.0%
> dev-libs/openssl:0/0 4 0.1%
> dev-libs/openssl:0/1.1 2 0.1%
>
> (note that those are rough measures, not guaranteed to be precise)
>
> So apparently 14.2% of dependencies allow any slot of OpenSSL which
> is
> most likely wrong, and 1.4% explicitly claim that's what the package
> wants. This could be valid only if e.g. the package supported
> multiple
> ABIs of OpenSSL libraries and used dlopen() with a few possible
> SONAMEs
> which I honestly doubt any of those packages is doing.
>
> In other words, 14.2% of dependencies on OpenSSL are plain wrong,
> and 6.4% are wrong in a way that isn't going to be reported by
> repoman.
> 1.4% of cases are using ':*' which probably indicates the developer
> decided to silence repoman without understanding how slot operators
> work
> which is a horrible thing from QA perspective.
>
> We also have a few cases that require specific OpenSSL subslot (e.g.
> forcing old version into :0 slot) but *none* actually using the
> binary
> compatibility slots.
>
>
> Secondly, it is confusing to users. If we remove old versions and
> only
> keep binary compatibility slots, users can be easily tricked into
> installing them and being surprised it's not a complete package. If
> we
> keep old versions, we end up having different revisions of the same
> version in different slots which is also easily confused.
>
>
> Thirdly, it is cumbersome to introduce. If we are to introduce a
> binary
> compatibility slot for a package that didn't have it, we need to
> update
> all reverse dependencies. This usually means researching whether we
> should use ':0' or ':0=', and if we get this wrong, we just silence
> repoman warning about missing slot-op.
>
>
> All of this considered, I can't think of a single real benefit of
> using
> slots for this purpose. They work but there's nothing really special
> about them.
>
>
> Suggested replacement
> =====================
>
> My suggestion is to move binary compatibility slots into separate
> packages. For example, dev-libs/openssl would be split into:
>
> dev-libs/openssl -- containing the actual package
>
> dev-libs/openssl-bin-compat -- containing binary compatibility
> slots
>
> In this case, all dependencies on dev-libs/openssl would become
> correct
> (or semi-correct, wrt missing := dep) again. Since packages are co-
> installable the same way slots are, there is no loss there. Since
> nothing depends on binary compatibility slots, we do not even need to
> update anything (but if we had, the update cost would be minimal both
> to
> developers and to users).
>
>
> What do you think?
I can only find bikeshed painting for the naming of the legacy
packages, so I guess sounds good. Though that historical digging about
why we moved away from it may be useful, if just to find that it was
just a "SLOTs are cool, lets use them" case, to strengthen the argument
to move back to separate package again.
Mart
[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 981 bytes --]
^ permalink raw reply [flat|nested] 11+ messages in thread* Re: [gentoo-dev] [RFC] Replacing binary-only SLOTs with separate packages
2019-01-18 20:13 [gentoo-dev] [RFC] Replacing binary-only SLOTs with separate packages Michał Górny
` (3 preceding siblings ...)
2019-01-19 11:14 ` [gentoo-dev] " Mart Raudsepp
@ 2019-01-19 14:37 ` Andrew Savchenko
2019-01-19 16:25 ` Michał Górny
4 siblings, 1 reply; 11+ messages in thread
From: Andrew Savchenko @ 2019-01-19 14:37 UTC (permalink / raw
To: gentoo-dev
[-- Attachment #1: Type: text/plain, Size: 4982 bytes --]
On Fri, 18 Jan 2019 21:13:34 +0100 Michał Górny wrote:
> Hello,
>
> Since I've been working on the early gx86-multilib, we've been using
> 'binary-only SLOTs' to support providing old versions of libraries for
> prebuilt software. I think this was a bad idea, and I'd like to suggest
> replacing it with something cleaner, for the reasons outlined below.
>
>
> Current state
> =============
>
> Let's take dev-libs/openssl as an example. This package has three SLOTs
> right now:
>
> 0.9.8: 0.9.8z_p8-r1
> 1.0.0: 1.0.2q-r200
> 0 : 1.0.2q 1.1.0j 1.1.1a 1.1.1a-r1
>
> The real package is provided as slot :0, that includes all libraries,
> headers and executables. Slots 0.9.8 and 1.0.0 only install .so.N*
> libraries that can be used to satisfy dependencies of prebuilt packages.
>
>
> Problems with the current state
> ===============================
>
> Firstly, it is confusing to developers. Let's analyze the dependencies
> on dev-libs/openssl. A quick grep reveals seven patterns. They are
> listed below, along with occurrence counts and percentages:
>
> dev-libs/openssl 278 7.8% }
> dev-libs/openssl:* 49 1.4% } 14.2%
> dev-libs/openssl:= 178 5.0% }
> dev-libs/openssl:0 660 18.6%
> dev-libs/openssl:0= 2381 67.0%
> dev-libs/openssl:0/0 4 0.1%
> dev-libs/openssl:0/1.1 2 0.1%
>
> (note that those are rough measures, not guaranteed to be precise)
>
> So apparently 14.2% of dependencies allow any slot of OpenSSL which is
> most likely wrong, and 1.4% explicitly claim that's what the package
> wants. This could be valid only if e.g. the package supported multiple
> ABIs of OpenSSL libraries and used dlopen() with a few possible SONAMEs
> which I honestly doubt any of those packages is doing.
>
> In other words, 14.2% of dependencies on OpenSSL are plain wrong,
> and 6.4% are wrong in a way that isn't going to be reported by repoman.
> 1.4% of cases are using ':*' which probably indicates the developer
> decided to silence repoman without understanding how slot operators work
> which is a horrible thing from QA perspective.
>
> We also have a few cases that require specific OpenSSL subslot (e.g.
> forcing old version into :0 slot) but *none* actually using the binary
> compatibility slots.
>
>
> Secondly, it is confusing to users. If we remove old versions and only
> keep binary compatibility slots, users can be easily tricked into
> installing them and being surprised it's not a complete package. If we
> keep old versions, we end up having different revisions of the same
> version in different slots which is also easily confused.
>
>
> Thirdly, it is cumbersome to introduce. If we are to introduce a binary
> compatibility slot for a package that didn't have it, we need to update
> all reverse dependencies. This usually means researching whether we
> should use ':0' or ':0=', and if we get this wrong, we just silence
> repoman warning about missing slot-op.
>
>
> All of this considered, I can't think of a single real benefit of using
> slots for this purpose. They work but there's nothing really special
> about them.
>
>
> Suggested replacement
> =====================
>
> My suggestion is to move binary compatibility slots into separate
> packages. For example, dev-libs/openssl would be split into:
>
> dev-libs/openssl -- containing the actual package
>
> dev-libs/openssl-bin-compat -- containing binary compatibility slots
>
> In this case, all dependencies on dev-libs/openssl would become correct
> (or semi-correct, wrt missing := dep) again. Since packages are co-
> installable the same way slots are, there is no loss there. Since
> nothing depends on binary compatibility slots, we do not even need to
> update anything (but if we had, the update cost would be minimal both to
> developers and to users).
>
>
> What do you think?
I do not like the idea. Slots are very elegant and effective
mechanism and is one of the points where Gentoo outruns other
distributions. Proposed approach with compat packages will
effectively disable slots for most cases.
Also proposed change will create a lot of unneeded technical
difficulties in maintaining packages: there will be double ebuilds
to maintain instead of a single one, ${PN} references will be
broken without additional changes and so far and so on. A lot of
unnecessary rebuilds will happen due slot to package moves.
Aside from development questions for me in a role of sysadmin slots
are much easier and understandable to manage than zoo of *compat*
packages in other distributions.
Assumption that :* is always wrong is invalid, since there are
valid cases: there are apps supporting various API versions or
using tools/data files without any care from where they are coming.
Best regards,
Andrew Savchenko
[-- Attachment #2: Type: application/pgp-signature, Size: 833 bytes --]
^ permalink raw reply [flat|nested] 11+ messages in thread* Re: [gentoo-dev] [RFC] Replacing binary-only SLOTs with separate packages
2019-01-19 14:37 ` Andrew Savchenko
@ 2019-01-19 16:25 ` Michał Górny
2019-01-21 14:14 ` Marc Schiffbauer
0 siblings, 1 reply; 11+ messages in thread
From: Michał Górny @ 2019-01-19 16:25 UTC (permalink / raw
To: gentoo-dev
[-- Attachment #1: Type: text/plain, Size: 6265 bytes --]
On Sat, 2019-01-19 at 17:37 +0300, Andrew Savchenko wrote:
> On Fri, 18 Jan 2019 21:13:34 +0100 Michał Górny wrote:
> > Hello,
> >
> > Since I've been working on the early gx86-multilib, we've been using
> > 'binary-only SLOTs' to support providing old versions of libraries for
> > prebuilt software. I think this was a bad idea, and I'd like to suggest
> > replacing it with something cleaner, for the reasons outlined below.
> >
> >
> > Current state
> > =============
> >
> > Let's take dev-libs/openssl as an example. This package has three SLOTs
> > right now:
> >
> > 0.9.8: 0.9.8z_p8-r1
> > 1.0.0: 1.0.2q-r200
> > 0 : 1.0.2q 1.1.0j 1.1.1a 1.1.1a-r1
> >
> > The real package is provided as slot :0, that includes all libraries,
> > headers and executables. Slots 0.9.8 and 1.0.0 only install .so.N*
> > libraries that can be used to satisfy dependencies of prebuilt packages.
> >
> >
> > Problems with the current state
> > ===============================
> >
> > Firstly, it is confusing to developers. Let's analyze the dependencies
> > on dev-libs/openssl. A quick grep reveals seven patterns. They are
> > listed below, along with occurrence counts and percentages:
> >
> > dev-libs/openssl 278 7.8% }
> > dev-libs/openssl:* 49 1.4% } 14.2%
> > dev-libs/openssl:= 178 5.0% }
> > dev-libs/openssl:0 660 18.6%
> > dev-libs/openssl:0= 2381 67.0%
> > dev-libs/openssl:0/0 4 0.1%
> > dev-libs/openssl:0/1.1 2 0.1%
> >
> > (note that those are rough measures, not guaranteed to be precise)
> >
> > So apparently 14.2% of dependencies allow any slot of OpenSSL which is
> > most likely wrong, and 1.4% explicitly claim that's what the package
> > wants. This could be valid only if e.g. the package supported multiple
> > ABIs of OpenSSL libraries and used dlopen() with a few possible SONAMEs
> > which I honestly doubt any of those packages is doing.
> >
> > In other words, 14.2% of dependencies on OpenSSL are plain wrong,
> > and 6.4% are wrong in a way that isn't going to be reported by repoman.
> > 1.4% of cases are using ':*' which probably indicates the developer
> > decided to silence repoman without understanding how slot operators work
> > which is a horrible thing from QA perspective.
> >
> > We also have a few cases that require specific OpenSSL subslot (e.g.
> > forcing old version into :0 slot) but *none* actually using the binary
> > compatibility slots.
> >
> >
> > Secondly, it is confusing to users. If we remove old versions and only
> > keep binary compatibility slots, users can be easily tricked into
> > installing them and being surprised it's not a complete package. If we
> > keep old versions, we end up having different revisions of the same
> > version in different slots which is also easily confused.
> >
> >
> > Thirdly, it is cumbersome to introduce. If we are to introduce a binary
> > compatibility slot for a package that didn't have it, we need to update
> > all reverse dependencies. This usually means researching whether we
> > should use ':0' or ':0=', and if we get this wrong, we just silence
> > repoman warning about missing slot-op.
> >
> >
> > All of this considered, I can't think of a single real benefit of using
> > slots for this purpose. They work but there's nothing really special
> > about them.
> >
> >
> > Suggested replacement
> > =====================
> >
> > My suggestion is to move binary compatibility slots into separate
> > packages. For example, dev-libs/openssl would be split into:
> >
> > dev-libs/openssl -- containing the actual package
> >
> > dev-libs/openssl-bin-compat -- containing binary compatibility slots
> >
> > In this case, all dependencies on dev-libs/openssl would become correct
> > (or semi-correct, wrt missing := dep) again. Since packages are co-
> > installable the same way slots are, there is no loss there. Since
> > nothing depends on binary compatibility slots, we do not even need to
> > update anything (but if we had, the update cost would be minimal both to
> > developers and to users).
> >
> >
> > What do you think?
>
> I do not like the idea. Slots are very elegant and effective
> mechanism and is one of the points where Gentoo outruns other
> distributions. Proposed approach with compat packages will
> effectively disable slots for most cases.
You haven't brought a single argument as to how slots are better than
compat packages. In fact, it's not even clear if you're talking about
the same use of slots that I am.
> Also proposed change will create a lot of unneeded technical
> difficulties in maintaining packages: there will be double ebuilds
> to maintain instead of a single one,
I don't see how this would change the effective number of ebuilds. You
can't produce two slots out of a single ebuild, so you always have to
duplicate them.
> ${PN} references will be
> broken without additional changes and so far and so on. A lot of
> unnecessary rebuilds will happen due slot to package moves.
Are you saying that there are 'lots' of binary compatibility slots being
currently used right now? Could you support this claim with actual
numbers or examples?
> Aside from development questions for me in a role of sysadmin slots
> are much easier and understandable to manage than zoo of *compat*
> packages in other distributions.
I don't see how binary compatibility slots are even relevant to
sysadmins. Their only purpose is to satisfy dependencies of other
packages, so sysadmin should never have to care about them.
> Assumption that :* is always wrong is invalid, since there are
> valid cases: there are apps supporting various API versions
Binary compatibility slots are used for ABI incompatibility, not any API
incompatibility. Also, I have the feeling those 'various API versions'
require headers to build.
> or
> using tools/data files without any care from where they are coming.
Tools aren't included in binary compatibility slots. As James already
pointed out in this thread.
--
Best regards,
Michał Górny
[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 963 bytes --]
^ permalink raw reply [flat|nested] 11+ messages in thread* Re: [gentoo-dev] [RFC] Replacing binary-only SLOTs with separate packages
2019-01-19 16:25 ` Michał Górny
@ 2019-01-21 14:14 ` Marc Schiffbauer
0 siblings, 0 replies; 11+ messages in thread
From: Marc Schiffbauer @ 2019-01-21 14:14 UTC (permalink / raw
To: gentoo-dev
[-- Attachment #1: Type: text/plain, Size: 1552 bytes --]
* Michał Górny schrieb am 19.01.19 um 17:25 Uhr:
> On Sat, 2019-01-19 at 17:37 +0300, Andrew Savchenko wrote:
> > I do not like the idea. Slots are very elegant and effective
> > mechanism and is one of the points where Gentoo outruns other
> > distributions. Proposed approach with compat packages will
> > effectively disable slots for most cases.
>
> You haven't brought a single argument as to how slots are better than
> compat packages. In fact, it's not even clear if you're talking about
> the same use of slots that I am.
One was, that you have one package (ebuild directory) for each piece of
software. Unlike other distributions which have lots of different
packages all for the same source an many cases.
So I'd agree that its the more gentooish way to have it all in one
ebuild directory.
That having said I agree that current use of compat slots might be
somewhat confusing even to developers.
So why not improve usability here?
I think it would increase the understanding of compat slots if we
changed the naming of thoss slots.
How about this:
dev-libs/openssl
(0.9.8-compat) 0.9.8u 0.9.8v 0.9.8w 0.9.8x ~0.9.8y 0.9.8z_p8-r1
(0) 1.0.0h 1.0.0i […] [M]~1.1.1a [M]~1.1.1a-r1
(1.0.0-compat) ~1.0.2q-r200
I can imagine that this might improve the situation a lot.
Of cource thet woult still be the job to fix ebuilds where dependencies
are wrong.
-Marc
--
0xCA3E7BF67F979BE5 - F7FB 78F7 7CC3 79F6 DF07
6E9E CA3E 7BF6 7F97 9BE5
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]
^ permalink raw reply [flat|nested] 11+ messages in thread