[gentoo-dev] [RFC] multiversion ebuilds

[gentoo-dev] [RFC] multiversion ebuilds

[Gerion Entrup]

[-- Attachment #1: Type: text/plain, Size: 1434 bytes --]

Hi,

just an idea for now. But what you think about multiversion ebuilds?
Technically this could be realized with the following line in the ebuild itself:
```
VERSIONS=( 3.0.11 3.0.12 3.1 )
```

and the filename without version:
<dev-category>/<package-name>/<package-name>.ebuild

together with this set of rules:
1. If there is an ebuild with had a version in its name, this ebuild is preferred.
 e.g. is a tree consists of "foobar/foobar-1.1.ebuild" and "foobar/foobar.ebuild" for version 1.1 the specific ebuild is taken.
2. If the ebuild has the variable VERSIONS specified but also a version in its name, the version in its name is taken.
3. There can be only one multiversioned ebuild per package.

Different version keywording can be done as before:
```
if [[ ${PV} == "3.1" ]] ; then
	KEYWORDS="~amd64 ~x86"
else
	KEYWORDS="amd64 x86"
fi
```

The resolution of versions can be done as before, with the difference that one ebuild can represent multiple versions.

The "ebuild" tool needs some adjustments. Maybe it tries to download and build all version per default and has an additional flag to specify a single version.

The advantages of this idea I see are:
- Ebuilds are written in a multiversion manner anyway, and then get copied (or linked?), so it can be made explicit.
- The diffs between different versions of ebuilds and the commit history are way more readable.
- The size of the tree reduces.

Regards,
Gerion


[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 659 bytes --]

Re: [gentoo-dev] [RFC] multiversion ebuilds

[Rich Freeman]

On Sat, May 12, 2018 at 8:20 AM Gerion Entrup <gerion.entrup@flump.de>
wrote:


> Different version keywording can be done as before:
> ```
> if [[ ${PV} == "3.1" ]] ; then
>          KEYWORDS="~amd64 ~x86"
> else
>          KEYWORDS="amd64 x86"
> fi
> ```

 From a readability standpoint I could see this getting out of hand if we
aren't careful.  Also, we'd want to keep this simple enough that the simple
act of stabilizing a package doesn't introduce regressions (perhaps even in
versions that we don't intend to touch).  We'd also need rules about
conditionals so that the metadata cache works.

You also would need to be careful about ebuild revisions in general, since
you're potentially affecting multiple versions at the same time.

How would revbumps fit into this?  All kinds of bad things can happen by
editing ebuilds in place, and this would compound the issue.

(Don't get me wrong - the concept is interesting...)

-- 
Rich

Re: [gentoo-dev] [RFC] multiversion ebuilds

[Gerion Entrup]

[-- Attachment #1: Type: text/plain, Size: 2149 bytes --]

Am Samstag, 12. Mai 2018, 15:47:57 CEST schrieb Rich Freeman:
> On Sat, May 12, 2018 at 8:20 AM Gerion Entrup <gerion.entrup@flump.de>
> wrote:
> 
> 
> > Different version keywording can be done as before:
> > ```
> > if [[ ${PV} == "3.1" ]] ; then
> >          KEYWORDS="~amd64 ~x86"
> > else
> >          KEYWORDS="amd64 x86"
> > fi
> > ```
> 
>  From a readability standpoint I could see this getting out of hand if we
> aren't careful. 
I have kind of copied this code from a package, where a 9999 ebuild exists. So it is already there.

> Also, we'd want to keep this simple enough that the simple
> act of stabilizing a package doesn't introduce regressions (perhaps even in
> versions that we don't intend to touch).  We'd also need rules about
> conditionals so that the metadata cache works.
Maybe this rule can be added:
4. Stable ebuilds are never multiversioned.

So once an ebuild get stable it is split out of the multiversion ebuild (aka copied) and not touched anymore.
Maybe this rule can be smoothed by moving the split to the point where new features are introduced in the ebuild:
4. If a multiversion ebuild contains a stable version and is changed featurewise (e.g. a new useflag), then the stable version is split out of the multiversion ebuild.

I don't know the internals of the metadata cache, so I can't say something to that.


> You also would need to be careful about ebuild revisions in general, since
> you're potentially affecting multiple versions at the same time.
You are right. But maybe it's the same fix, that is applied to all versions at once.

> How would revbumps fit into this?  All kinds of bad things can happen by
> editing ebuilds in place, and this would compound the issue.
Maybe it is possible to solve with the following method:
1. If the fix affects all ebuild versions, revbump all versions in the ebuild.
2. If the fix affects only one version, split out this version of the ebuild and make it an explicit one (together with the fix).

The thing is that it is an additional concept. It does not replace the current mechanisms and at best reduces overhead for ebuilds that are identical.

Gerion

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 659 bytes --]

Re: [gentoo-dev] [RFC] multiversion ebuilds

[Ulrich Mueller]

[-- Attachment #1: Type: text/plain, Size: 965 bytes --]

>>>>> On Sat, 12 May 2018, Gerion Entrup wrote:

> just an idea for now. But what you think about multiversion ebuilds?
> Technically this could be realized with the following line in the
> ebuild itself:
> ```
> VERSIONS=( 3.0.11 3.0.12 3.1 )
> ```

> [...]

> The advantages of this idea I see are:
> - Ebuilds are written in a multiversion manner anyway, and then get
> copied (or linked?), so it can be made explicit.
> - The diffs between different versions of ebuilds and the commit
> history are way more readable.

That depends on the options you specify for git diff (e.g.,
--find-renames, --find-copies, or --find-copies-harder).

> - The size of the tree reduces.

I very much doubt that (or at least it remains to be proven).

Currently, when an ebuild is copied for a version bump, it will reuse
the same blob in the Git repository. With the scheme above, you would
have to modify the ebuild, which would add a new blob for every
version bump.

Ulrich

[-- Attachment #2: Type: application/pgp-signature, Size: 490 bytes --]

Re: [gentoo-dev] [RFC] multiversion ebuilds

[Gerion Entrup]

[-- Attachment #1: Type: text/plain, Size: 1466 bytes --]

Am Samstag, 12. Mai 2018, 16:21:26 CEST schrieb Ulrich Mueller:
> >>>>> On Sat, 12 May 2018, Gerion Entrup wrote:
> 
> > just an idea for now. But what you think about multiversion ebuilds?
> > Technically this could be realized with the following line in the
> > ebuild itself:
> > ```
> > VERSIONS=( 3.0.11 3.0.12 3.1 )
> > ```
> 
> > [...]
> 
> > The advantages of this idea I see are:
> > - Ebuilds are written in a multiversion manner anyway, and then get
> > copied (or linked?), so it can be made explicit.
> > - The diffs between different versions of ebuilds and the commit
> > history are way more readable.
> 
> That depends on the options you specify for git diff (e.g.,
> --find-renames, --find-copies, or --find-copies-harder).
This does not apply to the online diffs (see e.g. [1]). I assume that most users don't have
the git repository checked out.

> 
> > - The size of the tree reduces.
> 
> I very much doubt that (or at least it remains to be proven).
> 
> Currently, when an ebuild is copied for a version bump, it will reuse
> the same blob in the Git repository. With the scheme above, you would
> have to modify the ebuild, which would add a new blob for every
> version bump.
You are right, I've not thought about that. However, this is only true for
the repository not for the rsync copy most(?) users have and not for a checkout.

Gerion

[1] https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=d9397ab8d48feb4b1360be614da35fa2faae44d9


[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 659 bytes --]

Re: [gentoo-dev] [RFC] multiversion ebuilds

[Dennis Schridde]

[-- Attachment #1: Type: text/plain, Size: 914 bytes --]

On Saturday, 12 May 2018 16:36:13 CEST Gerion Entrup wrote:
> Am Samstag, 12. Mai 2018, 16:21:26 CEST schrieb Ulrich Mueller:
> > >>>>> On Sat, 12 May 2018, Gerion Entrup wrote:
> > > - The size of the tree reduces.
> > 
> > I very much doubt that (or at least it remains to be proven).
> > 
> > Currently, when an ebuild is copied for a version bump, it will reuse
> > the same blob in the Git repository. With the scheme above, you would
> > have to modify the ebuild, which would add a new blob for every
> > version bump.
> 
> You are right, I've not thought about that. However, this is only true for
> the repository not for the rsync copy most(?) users have and not for a
> checkout.

As I understand it, rsync will indeed re-use blocks from existing files on the 
receiver [1], and hence copy files with the same content only once.

--Dennis

[1]: http://www.anchor.com.au/blog/2013/08/out-tridging-tridge/

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 659 bytes --]

Re: [gentoo-dev] [RFC] multiversion ebuilds

[R0b0t1]

On Sat, May 12, 2018 at 7:20 AM, Gerion Entrup <gerion.entrup@flump.de> wrote:
> - The size of the tree reduces.
>

If this is a big concern you may be able to mount the portage tree
under a compressed loopback filesystem. It may even be worth
considering that as a recommended-by-handbook default as the benefit
for compressing the ebuilds is likely huge anyway.

Cheers,
     R0b0t1

Re: [gentoo-dev] [RFC] multiversion ebuilds

[Gerion Entrup]

[-- Attachment #1: Type: text/plain, Size: 557 bytes --]

Am Samstag, 12. Mai 2018, 16:24:00 CEST schrieb R0b0t1:
> On Sat, May 12, 2018 at 7:20 AM, Gerion Entrup <gerion.entrup@flump.de> wrote:
> > - The size of the tree reduces.
> >
> 
> If this is a big concern you may be able to mount the portage tree
> under a compressed loopback filesystem. It may even be worth
> considering that as a recommended-by-handbook default as the benefit
> for compressing the ebuilds is likely huge anyway.
This seems reasonable. However, first duplicating code and then deduplicate it at file system level seems weird.

Gerion

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 659 bytes --]

Re: [gentoo-dev] [RFC] multiversion ebuilds

[Georgy Yakovlev]

[-- Attachment #1: Type: text/plain, Size: 706 bytes --]

On Saturday, May 12, 2018 5:20:30 AM PDT Gerion Entrup wrote:
> Hi,
> 
> just an idea for now. But what you think about multiversion ebuilds?
> Technically this could be realized with the following line in the ebuild
> itself: ```
> VERSIONS=( 3.0.11 3.0.12 3.1 )
> ```
You may also be interested checking out exherbo's approach:

exheres (ebuild) holds some metadata and version and sometimes some code 
overrides.
exlib (similar to eclass, but not quite so) holds the most of the code an 
logic. can be per-package and per-category.


examples:
https://git.exherbo.org/arbor.git/tree/packages/net-wireless/wpa_supplicant
https://git.exherbo.org/arbor.git/tree/packages/sys-libs/glibc

--
Regards,
Georgy.

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [gentoo-dev] [RFC] multiversion ebuilds

[Mathy Vanvoorden]

[-- Attachment #1: Type: text/plain, Size: 1163 bytes --]

2018-05-12 14:20 GMT+02:00 Gerion Entrup <gerion.entrup@flump.de>:

just an idea for now. But what you think about multiversion ebuilds?
> Technically this could be realized with the following line in the ebuild
> itself:
> ```
> VERSIONS=( 3.0.11 3.0.12 3.1 )
> ```
>

I like the idea of multiversion ebuilds but why would you complicate the
process by putting it in a variable? Why not just use symlinks and have the
following:

foobar/foobar-1.x
foobar/foobar-1.1.ebuild -> foobar-1.x
foobar/foobar-1.2.ebuild -> foobar-1.x
foobar/foobar-2.x
foobar/foobar-2.1.ebuild -> foobar-2.x

It would result in the same outcome but it seems to me that a lot less work
(almost none?) is needed to implement it.

Benefits compared to your suggestion:
* you don't need to add the extra VERSIONS variable and related logic
* you don't need the set of rules
* you can have multiple multiversioned ebuilds per package

I'm not sure if adding the foobar-1.x file is allowed by portage.

You would still need logic like this for the keywording:

```
> if [[ ${PV} == "3.1" ]] ; then
>         KEYWORDS="~amd64 ~x86"
> else
>         KEYWORDS="amd64 x86"
> fi
> ```
>

br,
Mathy

[-- Attachment #2: Type: text/html, Size: 1901 bytes --]

[gentoo-dev] Re: [RFC] multiversion ebuilds

[Duncan]

Mathy Vanvoorden posted on Tue, 15 May 2018 11:32:30 +0200 as excerpted:

> 2018-05-12 14:20 GMT+02:00 Gerion Entrup <gerion.entrup@flump.de>:
> 
> just an idea for now. But what you think about multiversion ebuilds?
>> Technically this could be realized with the following line in the ebuild
>> itself:
>> ```
>> VERSIONS=( 3.0.11 3.0.12 3.1 )
>> ```
>>
> 
> I like the idea of multiversion ebuilds but why would you complicate the
> process by putting it in a variable? Why not just use symlinks and have the
> following:
> 
> foobar/foobar-1.x
> foobar/foobar-1.1.ebuild -> foobar-1.x
> foobar/foobar-1.2.ebuild -> foobar-1.x
> foobar/foobar-2.x
> foobar/foobar-2.1.ebuild -> foobar-2.x

AFAIK symlinks aren't allowed in the gentoo tree, with the given reason
being that some users, particularly those with limited net access and
thus "sneakernetting" from where they /do/ have net access, may place
the tree on or transfer it via no-symlink-support FAT32 or similar,
perhaps downloading it from an MS machine or the like.

Of course users may use symlinks on their own copies, but they're not
allowed in the official tree.

Tho perhaps that can be reevaluated.  But while there's more connectivity
now than over a decade ago when that policy was created, I expect there's
still those paying by the meg or gig for net access locally, that won't
enjoy having their sneakernet sync routine disrupted.

-- 
Duncan - List replies preferred.   No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master."  Richard Stallman

Re: [gentoo-dev] Re: [RFC] multiversion ebuilds

[R0b0t1]

On Tue, May 15, 2018 at 11:15 PM, Duncan <1i5t5.duncan@cox.net> wrote:
> Mathy Vanvoorden posted on Tue, 15 May 2018 11:32:30 +0200 as excerpted:
>
>> 2018-05-12 14:20 GMT+02:00 Gerion Entrup <gerion.entrup@flump.de>:
>>
>> just an idea for now. But what you think about multiversion ebuilds?
>>> Technically this could be realized with the following line in the ebuild
>>> itself:
>>> ```
>>> VERSIONS=( 3.0.11 3.0.12 3.1 )
>>> ```
>>>
>>
>> I like the idea of multiversion ebuilds but why would you complicate the
>> process by putting it in a variable? Why not just use symlinks and have the
>> following:
>>
>> foobar/foobar-1.x
>> foobar/foobar-1.1.ebuild -> foobar-1.x
>> foobar/foobar-1.2.ebuild -> foobar-1.x
>> foobar/foobar-2.x
>> foobar/foobar-2.1.ebuild -> foobar-2.x
>
> AFAIK symlinks aren't allowed in the gentoo tree, with the given reason
> being that some users, particularly those with limited net access and
> thus "sneakernetting" from where they /do/ have net access, may place
> the tree on or transfer it via no-symlink-support FAT32 or similar,
> perhaps downloading it from an MS machine or the like.
>
> Of course users may use symlinks on their own copies, but they're not
> allowed in the official tree.
>
> Tho perhaps that can be reevaluated.  But while there's more connectivity
> now than over a decade ago when that policy was created, I expect there's
> still those paying by the meg or gig for net access locally, that won't
> enjoy having their sneakernet sync routine disrupted.
>

Cygwin and MSYS(2) are currently mostly supported by Prefix, so using
symlinks might kill them as well. There is some kind of symlinking
support for NTFS now but it is very primitive.

Cheers,
     R0b0t1

[gentoo-dev] Re: [RFC] multiversion ebuilds

[Martin Vaeth]

R0b0t1 <r030t1@gmail.com> wrote:
> On Tue, May 15, 2018 at 11:15 PM, Duncan <1i5t5.duncan@cox.net> wrote:
>>
>> AFAIK symlinks aren't allowed in the gentoo tree [...]
>>
>> Tho perhaps that can be reevaluated. [...]
>
> Cygwin and MSYS(2) are currently mostly supported by Prefix [...]

For rsync users, the non-symlink policy is not necessary:
They could simply pass the option --copy-links to get a working
tree (of course, correspondingly longer).

For git users (essential for overlays!), I do not know a simple way.
It might be possible to set up appropriate git hooks, but that
might be tricky: The hooks would need to apply before the files
are actually checked out, and it might become even more tricky if
one does not want that subsequent git commands do consider the
transformation as a local modification.

For git perhaps an opposite approach might be easier: One could
provide hooks which every git user is supposed to use which manage
symlinks and/or copies.
More precisely, these hooks should have a symlink and a non-symlink
mode (depending on whether the underlying (file)system can generate
symlinks which can easily and quickly be detected), and the tree
would need to contain a symlinks.dat file with a list of symlinks,
and a .gitignore containing the symlinks.
During checkout, the hooks should simply generate the corresponding
symlinks/files (depending on symlinks.dat) according to the stored
list of symlinks.
During checkin in symlink mode, these hooks automatically update
symlinks.dat and .gitignore so that most developers simply can
use/modify symlinks transparently.
During checkin in non-symlink mode the hook does nothing:
Developers who checkin on filesystems without symlink have to update
these 2 files manually, of course, when they want to
create/remove/rename a symlink. Maybe one could provide
corresponding helper scripts for those developers with
non-symlink-aware filesystems if necessary.

Re: [gentoo-dev] [RFC] multiversion ebuilds

[Paweł Hajdan, Jr.]

[-- Attachment #1.1: Type: text/plain, Size: 572 bytes --]

On 12/05/2018 14:20, Gerion Entrup wrote:
> just an idea for now. But what you think about multiversion ebuilds?
> Technically this could be realized with the following line in the ebuild itself:
> ```
> VERSIONS=( 3.0.11 3.0.12 3.1 )
> ```
> 
> and the filename without version:
> <dev-category>/<package-name>/<package-name>.ebuild

I'm wondering: if the main goal would be more code sharing between
ebuilds, would something like
<https://exherbo.org/docs/eapi/exheres-for-smarties.html#exlibs>
(essentially per-package eclasses) be an option?

Paweł


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 827 bytes --]

Re: [gentoo-dev] [RFC] multiversion ebuilds

[Ulrich Mueller]

[-- Attachment #1: Type: text/plain, Size: 296 bytes --]

>>>>> On Wed, 16 May 2018, Paweł Hajdan, wrote:

> I'm wondering: if the main goal would be more code sharing between
> ebuilds, would something like
> <https://exherbo.org/docs/eapi/exheres-for-smarties.html#exlibs>
> (essentially per-package eclasses) be an option?

eblits!
/me hides

[-- Attachment #2: Type: application/pgp-signature, Size: 490 bytes --]

Re: [gentoo-dev] [RFC] multiversion ebuilds

[Ulrich Mueller]

[-- Attachment #1: Type: text/plain, Size: 990 bytes --]

>>>>> On Wed, 16 May 2018, Ulrich Mueller wrote:

>>>>> On Wed, 16 May 2018, Paweł Hajdan, wrote:
>> I'm wondering: if the main goal would be more code sharing between
>> ebuilds, would something like
>> <https://exherbo.org/docs/eapi/exheres-for-smarties.html#exlibs>
>> (essentially per-package eclasses) be an option?

> eblits!
> /me hides

But seriously: Per-package eclasses were proposed in 2010 already and
made it into a pre-draft GLEP [1]. See also the HTML comments in that
text pointing out some problems with that approach.

The subject came up again about two years ago, but I fail to find the
discussion (maybe it was on IRC). AFAIR there were doubts that such a
concept would be widely adopted, because of its increased complexity,
and the few packages in need of it could use regular eclasses (living
in the eclass/ dir) instead.

Ulrich

[1] https://sources.gentoo.org/cgi-bin/viewvc.cgi/gentoo/users/antarus/projects/gleps/glep-XX.txt?view=markup

[-- Attachment #2: Type: application/pgp-signature, Size: 490 bytes --]

Re: [gentoo-dev] [RFC] multiversion ebuilds

[Michał Górny]

W dniu sob, 12.05.2018 o godzinie 14∶20 +0200, użytkownik Gerion Entrup
napisał:
> Hi,
> 
> just an idea for now. But what you think about multiversion ebuilds?
> Technically this could be realized with the following line in the ebuild itself:
> ```
> VERSIONS=( 3.0.11 3.0.12 3.1 )
> ```
> 
> and the filename without version:
> <dev-category>/<package-name>/<package-name>.ebuild
> 
> together with this set of rules:
> 1. If there is an ebuild with had a version in its name, this ebuild is preferred.
>  e.g. is a tree consists of "foobar/foobar-1.1.ebuild" and "foobar/foobar.ebuild" for version 1.1 the specific ebuild is taken.
> 2. If the ebuild has the variable VERSIONS specified but also a version in its name, the version in its name is taken.
> 3. There can be only one multiversioned ebuild per package.
> 
> Different version keywording can be done as before:
> ```
> if [[ ${PV} == "3.1" ]] ; then
> 	KEYWORDS="~amd64 ~x86"
> else
> 	KEYWORDS="amd64 x86"
> fi
> ```
> 
> The resolution of versions can be done as before, with the difference that one ebuild can represent multiple versions.
> 
> The "ebuild" tool needs some adjustments. Maybe it tries to download and build all version per default and has an additional flag to specify a single version.
> 
> The advantages of this idea I see are:
> - Ebuilds are written in a multiversion manner anyway, and then get copied (or linked?), so it can be made explicit.
> - The diffs between different versions of ebuilds and the commit history are way more readable.
> - The size of the tree reduces.
> 

In my opinion, this puts more effort into inventing a problem than
solving one.  In order to consider a particular idea thouroughly, you
should also consider potential disadvantages, rather than focusing
purely on advantages as you see them.

While one might think that this will help developers, it is going to be
a maintenance nightmare.  Just compare the workflow.

Currently, adding a patch affecting runtime involves copying the ebuild,
and applying the patch.  Later on, the old revision is just removed. 
With your solution, you need to adjust VERSIONS, add conditional; later
on, you need to adjust VERSIONS, find all conditionals, remove them. 
Not only it involves more work but also increases the risk of accidental
breakage.

The arch team work is going to become a nightmare.  Currently, they just
use 'ekeyword' tool to mass-edit ebuilds.  With your solution, they're
now have to find the correct conditional (or add one), and update
keywords there.  I can already imagine monsters like:

  if pv1; then
    KEYWORDS="~amd64"
  elif pv2; then
    KEYWORDS="amd64 ~arm64 x86"
  elif pv3; then
    KEYWORDS="~amd64 ~arm64 ~x86"
  elif pv4; then
    KEYWORDS="amd64 ~arm64 ~x86"
  fi

Basically, any action requiring >1 arch team would involve creating
a new version at least temporarily.  And I seriously doubt arch teams
would really want to spend time collapsing those afterwards.

The algorithm you presented might look nice at first.  But remember that
the developers will now have to do the same thing -- i.e. discover which
ebuild should they look at.  You're moving for a clear 1 ebuild : 1 file
mapping to N ebuilds : 1 file.

-- 
Best regards,
Michał Górny

Re: [gentoo-dev] [RFC] multiversion ebuilds

[R0b0t1]

On Wed, May 16, 2018 at 2:38 AM, Michał Górny <mgorny@gentoo.org> wrote:
> W dniu sob, 12.05.2018 o godzinie 14∶20 +0200, użytkownik Gerion Entrup
> napisał:
>> Hi,
>>
>> just an idea for now. But what you think about multiversion ebuilds?
>> Technically this could be realized with the following line in the ebuild itself:
>> ```
>> VERSIONS=( 3.0.11 3.0.12 3.1 )
>> ```
>>
>> and the filename without version:
>> <dev-category>/<package-name>/<package-name>.ebuild
>>
>> together with this set of rules:
>> 1. If there is an ebuild with had a version in its name, this ebuild is preferred.
>>  e.g. is a tree consists of "foobar/foobar-1.1.ebuild" and "foobar/foobar.ebuild" for version 1.1 the specific ebuild is taken.
>> 2. If the ebuild has the variable VERSIONS specified but also a version in its name, the version in its name is taken.
>> 3. There can be only one multiversioned ebuild per package.
>>
>> Different version keywording can be done as before:
>> ```
>> if [[ ${PV} == "3.1" ]] ; then
>>       KEYWORDS="~amd64 ~x86"
>> else
>>       KEYWORDS="amd64 x86"
>> fi
>> ```
>>
>> The resolution of versions can be done as before, with the difference that one ebuild can represent multiple versions.
>>
>> The "ebuild" tool needs some adjustments. Maybe it tries to download and build all version per default and has an additional flag to specify a single version.
>>
>> The advantages of this idea I see are:
>> - Ebuilds are written in a multiversion manner anyway, and then get copied (or linked?), so it can be made explicit.
>> - The diffs between different versions of ebuilds and the commit history are way more readable.
>> - The size of the tree reduces.
>>
>
> In my opinion, this puts more effort into inventing a problem than
> solving one.  In order to consider a particular idea thouroughly, you
> should also consider potential disadvantages, rather than focusing
> purely on advantages as you see them.
>
> While one might think that this will help developers, it is going to be
> a maintenance nightmare.  Just compare the workflow.
>
> Currently, adding a patch affecting runtime involves copying the ebuild,
> and applying the patch.  Later on, the old revision is just removed.
> With your solution, you need to adjust VERSIONS, add conditional; later
> on, you need to adjust VERSIONS, find all conditionals, remove them.
> Not only it involves more work but also increases the risk of accidental
> breakage.
>
> The arch team work is going to become a nightmare.  Currently, they just
> use 'ekeyword' tool to mass-edit ebuilds.  With your solution, they're
> now have to find the correct conditional (or add one), and update
> keywords there.  I can already imagine monsters like:
>
>   if pv1; then
>     KEYWORDS="~amd64"
>   elif pv2; then
>     KEYWORDS="amd64 ~arm64 x86"
>   elif pv3; then
>     KEYWORDS="~amd64 ~arm64 ~x86"
>   elif pv4; then
>     KEYWORDS="amd64 ~arm64 ~x86"
>   fi
>

Instead of VERSIONS=( 3.0.11 3.0.12 3.1 ) use KEYWORD_AMD64=( 3.0.11
3.0.12 ~3.1 ).

This would require a variable per arch. It would be possible to create
another structure to contain *these* in some way if having multiple
variables is something that should be avoided at all costs.

Cheers,
     R0b0t1

Re: [gentoo-dev] [RFC] multiversion ebuilds

[Gerion Entrup]

Am Mittwoch, 16. Mai 2018, 09:38:30 CEST schrieb Michał Górny:
> W dniu sob, 12.05.2018 o godzinie 14∶20 +0200, użytkownik Gerion Entrup
> napisał:
> > Hi,
> > 
> > just an idea for now. But what you think about multiversion ebuilds?
> > Technically this could be realized with the following line in the ebuild itself:
> > ```
> > VERSIONS=( 3.0.11 3.0.12 3.1 )
> > ```
> > 
> > and the filename without version:
> > <dev-category>/<package-name>/<package-name>.ebuild
> > 
> > together with this set of rules:
> > 1. If there is an ebuild with had a version in its name, this ebuild is preferred.
> >  e.g. is a tree consists of "foobar/foobar-1.1.ebuild" and "foobar/foobar.ebuild" for version 1.1 the specific ebuild is taken.
> > 2. If the ebuild has the variable VERSIONS specified but also a version in its name, the version in its name is taken.
> > 3. There can be only one multiversioned ebuild per package.
> > 
> > Different version keywording can be done as before:
> > ```
> > if [[ ${PV} == "3.1" ]] ; then
> > 	KEYWORDS="~amd64 ~x86"
> > else
> > 	KEYWORDS="amd64 x86"
> > fi
> > ```
> > 
> > The resolution of versions can be done as before, with the difference that one ebuild can represent multiple versions.
> > 
> > The "ebuild" tool needs some adjustments. Maybe it tries to download and build all version per default and has an additional flag to specify a single version.
> > 
> > The advantages of this idea I see are:
> > - Ebuilds are written in a multiversion manner anyway, and then get copied (or linked?), so it can be made explicit.
> > - The diffs between different versions of ebuilds and the commit history are way more readable.
> > - The size of the tree reduces.
> > 
> 
> In my opinion, this puts more effort into inventing a problem than
> solving one.  In order to consider a particular idea thouroughly, you
> should also consider potential disadvantages, rather than focusing
> purely on advantages as you see them.
> 
> While one might think that this will help developers, it is going to be
> a maintenance nightmare.  Just compare the workflow.
> 
> Currently, adding a patch affecting runtime involves copying the ebuild,
> and applying the patch.  Later on, the old revision is just removed. 
> With your solution, you need to adjust VERSIONS, add conditional; later
> on, you need to adjust VERSIONS, find all conditionals, remove them. 
> Not only it involves more work but also increases the risk of accidental
> breakage.
I'm not sure, if I understand you correctly, so I try to express myself:
If an ebuild (say foobar-1.0.ebuild) needs a patch affecting runtime, the
current workflow is to copy the ebuild, add the code that applies the patch
and do a revbump (to foobar-1.0-r1.ebuild). The old ebuild remains
untouched.
With the VERSIONS approach, the VERSIONS need to be adjusted to contain
the revbump version as well, and add a conditional that applies only in the
revbump version the patch and otherwise not.

The VERSIONS approach does not break the old mechanism. So if a patch
needs to be applied, the multiversion ebuild (that contains other versions
as well, say foobar.ebuild with VERSIONS="1.0 1.1 1.2") can just be copied,
renamed with the revbump number (to foobar-1.0-r1.ebuild) and completed
with the apply line. Once the old versions should be deleted the VERSIONS
variable can be adjusted (to VERSIONS="1.1 1.2").


> The arch team work is going to become a nightmare.  Currently, they just
> use 'ekeyword' tool to mass-edit ebuilds.  With your solution, they're
> now have to find the correct conditional (or add one), and update
> keywords there.  I can already imagine monsters like:
> 
>   if pv1; then
>     KEYWORDS="~amd64"
>   elif pv2; then
>     KEYWORDS="amd64 ~arm64 x86"
>   elif pv3; then
>     KEYWORDS="~amd64 ~arm64 ~x86"
>   elif pv4; then
>     KEYWORDS="amd64 ~arm64 ~x86"
>   fi
> 
> Basically, any action requiring >1 arch team would involve creating
> a new version at least temporarily.  And I seriously doubt arch teams
> would really want to spend time collapsing those afterwards.
You are right, I have not thought about that. Maybe the approach of R0b0t1
would work.


> The algorithm you presented might look nice at first.  But remember that
> the developers will now have to do the same thing -- i.e. discover which
> ebuild should they look at.  You're moving for a clear 1 ebuild : 1 file
> mapping to N ebuilds : 1 file.

Gerion

Re: [gentoo-dev] [RFC] multiversion ebuilds

[Rich Freeman]

On Thu, May 17, 2018 at 11:44 AM Gerion Entrup <gerion.entrup@flump.de>
wrote:

> The VERSIONS approach does not break the old mechanism. So if a patch
> needs to be applied, the multiversion ebuild (that contains other versions
> as well, say foobar.ebuild with VERSIONS="1.0 1.1 1.2") can just be
copied,
> renamed with the revbump number (to foobar-1.0-r1.ebuild) and completed
> with the apply line. Once the old versions should be deleted the VERSIONS
> variable can be adjusted (to VERSIONS="1.1 1.2").


Sure, but now you're back to single-version ebuilds except that you have to
stick the version inside the ebuild instead of just in the filename.

When would you actually get any of the code-sharing benefits of
multi-version ebuilds?  If you fork them anytime there is a revbump then to
actually share code you'd need the ebuilds for both versions to be issued
at the same time and have near-identical code.  How often does that happen?

I think in practice this just makes things more complicated.

And this problem isn't just limited to keywording.  If you don't want to
make retroactive changes to released packages then you need to make all
code changes conditional on version, which makes the ebuild a rat's nest of
conditionals.

I could really only see something like this working if the entire Gentoo
repo were release-based, because then you'd tag a release, and then you can
modify all the code you want in-place because it wouldn't affect anything
in production.  Then you do a round of QA and issue a new release.  Since
Gentoo's QA operates at the package-version level you really need to make
your changes in new package-versions (including ebuild revision numbers).

-- 
Rich