[gentoo-python] Handling packages not supporting multiple Python implementations

[gentoo-python] Handling packages not supporting multiple Python implementations

[Michał Górny]

[-- Attachment #1: Type: text/plain, Size: 2457 bytes --]

Hello,

After starting to deploy python-r1 on packages supporting multiple
Python implementations, I believe it is time to start thinking about
those packages which don't support that. Firstly, I would like to gain
a general feedback/ideas on the possible solutions, without getting too
deep into the technical details of it.

As far as I can think, we have the following possibilities:


1) Assume that installing stuff for a single Python implementation is
deprecated and let the packages rot with the old eclass.

It is probably the simplest solution (i.e. not doing anything to
address the issue) but truth be told, I doubt this will actually work.
People will just keep using the old eclass which doesn't really do much
good for those packages...

And even if some people will actually start supporting multiple
implementations... that may be even worse. Just look at dev-libs/boost
to see what I mean.


2) Use a xor-type REQUIRED_USE for those packages.

Put the whole set of PYTHON_TARGETS but add a REQUIRED_USE='^^ ( ... )'
for them, effectively requesting only a single implementation being
enabled.

I believe that this is quite a good solution, at least from
the dependency point of view. We clearly express which Python
implementations are supported by a particular package and which one was
enabled. We can express cross-package dependencies cleanly.

The problem lies in user-friendliness. Although with the current
default (python2_7 only) it wouldn't cause any trouble, whenever user
enables more than a single implementation, every single-implementation
package will require package.use adjustment. This will become an even
more widespread issue when we decide to re-enable Python 3.

To be honest, I don't see any good way around that.


3) Use implicit implementation selection (like python.eclass).

Well, as some say, this is a very good solution since it's well tested.
Its limitations and brokenness are obvious. Just I doubt it is really
worth the effort to write something that bad.

The main problem here is that the chosen Python implementation is
implicit. Binary packages don't express it. Cross-package dependencies
don't express it. User changes the implementation, stuff breaks
silently and you end up with some kind of python-updater (why a tool
to fix breakage is called 'updater'?!).


Do you have any more ideas? Opinions?

-- 
Best regards,
Michał Górny

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 316 bytes --]

[gentoo-python] Re: Handling packages not supporting multiple Python implementations

[Mike Gilbert]

On Tue, Oct 23, 2012 at 5:58 PM, Michał Górny <mgorny@gentoo.org> wrote:
> Hello,
>
> After starting to deploy python-r1 on packages supporting multiple
> Python implementations, I believe it is time to start thinking about
> those packages which don't support that. Firstly, I would like to gain
> a general feedback/ideas on the possible solutions, without getting too
> deep into the technical details of it.
>
> As far as I can think, we have the following possibilities:
>
>
> 1) Assume that installing stuff for a single Python implementation is
> deprecated and let the packages rot with the old eclass.
>
> It is probably the simplest solution (i.e. not doing anything to
> address the issue) but truth be told, I doubt this will actually work.
> People will just keep using the old eclass which doesn't really do much
> good for those packages...
>
> And even if some people will actually start supporting multiple
> implementations... that may be even worse. Just look at dev-libs/boost
> to see what I mean.
>
>
> 2) Use a xor-type REQUIRED_USE for those packages.
>
> Put the whole set of PYTHON_TARGETS but add a REQUIRED_USE='^^ ( ... )'
> for them, effectively requesting only a single implementation being
> enabled.
>
> I believe that this is quite a good solution, at least from
> the dependency point of view. We clearly express which Python
> implementations are supported by a particular package and which one was
> enabled. We can express cross-package dependencies cleanly.
>
> The problem lies in user-friendliness. Although with the current
> default (python2_7 only) it wouldn't cause any trouble, whenever user
> enables more than a single implementation, every single-implementation
> package will require package.use adjustment. This will become an even
> more widespread issue when we decide to re-enable Python 3.
>
> To be honest, I don't see any good way around that.
>
>
> 3) Use implicit implementation selection (like python.eclass).
>
> Well, as some say, this is a very good solution since it's well tested.
> Its limitations and brokenness are obvious. Just I doubt it is really
> worth the effort to write something that bad.
>
> The main problem here is that the chosen Python implementation is
> implicit. Binary packages don't express it. Cross-package dependencies
> don't express it. User changes the implementation, stuff breaks
> silently and you end up with some kind of python-updater (why a tool
> to fix breakage is called 'updater'?!).
>
>
> Do you have any more ideas? Opinions?
>

Like you, I really can't come up with an ideal solution here.

We really have 2 classes of packages here:

1. Packages that don't care what version of python you use, but
install files outside of site-packages.

2. Packages that build code (like libraries) against a specific
version of python/libpython.

The implicit implementation selection works fine for #1, but not so well for #2.

[gentoo-python] Re: Handling packages not supporting multiple Python implementations

[Jesus Rivero (Neurogeek)]

On Thu, Oct 25, 2012 at 5:00 PM, Mike Gilbert <floppym@gentoo.org> wrote:
> On Tue, Oct 23, 2012 at 5:58 PM, Michał Górny <mgorny@gentoo.org> wrote:
>> Hello,
>>
>> After starting to deploy python-r1 on packages supporting multiple
>> Python implementations, I believe it is time to start thinking about
>> those packages which don't support that. Firstly, I would like to gain
>> a general feedback/ideas on the possible solutions, without getting too
>> deep into the technical details of it.
>>
>> As far as I can think, we have the following possibilities:
>>
>>
>> 1) Assume that installing stuff for a single Python implementation is
>> deprecated and let the packages rot with the old eclass.
>>
>> It is probably the simplest solution (i.e. not doing anything to
>> address the issue) but truth be told, I doubt this will actually work.
>> People will just keep using the old eclass which doesn't really do much
>> good for those packages...
>>
>> And even if some people will actually start supporting multiple
>> implementations... that may be even worse. Just look at dev-libs/boost
>> to see what I mean.
>>
>>
>> 2) Use a xor-type REQUIRED_USE for those packages.
>>
>> Put the whole set of PYTHON_TARGETS but add a REQUIRED_USE='^^ ( ... )'
>> for them, effectively requesting only a single implementation being
>> enabled.
>>
>> I believe that this is quite a good solution, at least from
>> the dependency point of view. We clearly express which Python
>> implementations are supported by a particular package and which one was
>> enabled. We can express cross-package dependencies cleanly.
>>
>> The problem lies in user-friendliness. Although with the current
>> default (python2_7 only) it wouldn't cause any trouble, whenever user
>> enables more than a single implementation, every single-implementation
>> package will require package.use adjustment. This will become an even
>> more widespread issue when we decide to re-enable Python 3.
>>
>> To be honest, I don't see any good way around that.
>>
>>
>> 3) Use implicit implementation selection (like python.eclass).
>>
>> Well, as some say, this is a very good solution since it's well tested.
>> Its limitations and brokenness are obvious. Just I doubt it is really
>> worth the effort to write something that bad.
>>
>> The main problem here is that the chosen Python implementation is
>> implicit. Binary packages don't express it. Cross-package dependencies
>> don't express it. User changes the implementation, stuff breaks
>> silently and you end up with some kind of python-updater (why a tool
>> to fix breakage is called 'updater'?!).
>>
>>
>> Do you have any more ideas? Opinions?
>>
>
> Like you, I really can't come up with an ideal solution here.
>
> We really have 2 classes of packages here:
>
> 1. Packages that don't care what version of python you use, but
> install files outside of site-packages.
>
> 2. Packages that build code (like libraries) against a specific
> version of python/libpython.
>
> The implicit implementation selection works fine for #1, but not so well for #2.

I wonder if we could have a "special target" that could expand to the
current eselected python version. That way, the package not supporting
a multiple targets, could be treated the same as those who do.

Something like:

PYTHON_COMPAT=( python_current ), or something like that.

-- 
Jesus Rivero (Neurogeek)
Gentoo Developer

Re: [gentoo-python] Re: Handling packages not supporting multiple Python implementations

[Michał Górny]

[-- Attachment #1: Type: text/plain, Size: 4057 bytes --]

On Thu, 25 Oct 2012 17:25:47 -0400
"Jesus Rivero (Neurogeek)" <neurogeek@gentoo.org> wrote:

> On Thu, Oct 25, 2012 at 5:00 PM, Mike Gilbert <floppym@gentoo.org> wrote:
> > On Tue, Oct 23, 2012 at 5:58 PM, Michał Górny <mgorny@gentoo.org> wrote:
> >> Hello,
> >>
> >> After starting to deploy python-r1 on packages supporting multiple
> >> Python implementations, I believe it is time to start thinking about
> >> those packages which don't support that. Firstly, I would like to gain
> >> a general feedback/ideas on the possible solutions, without getting too
> >> deep into the technical details of it.
> >>
> >> As far as I can think, we have the following possibilities:
> >>
> >>
> >> 1) Assume that installing stuff for a single Python implementation is
> >> deprecated and let the packages rot with the old eclass.
> >>
> >> It is probably the simplest solution (i.e. not doing anything to
> >> address the issue) but truth be told, I doubt this will actually work.
> >> People will just keep using the old eclass which doesn't really do much
> >> good for those packages...
> >>
> >> And even if some people will actually start supporting multiple
> >> implementations... that may be even worse. Just look at dev-libs/boost
> >> to see what I mean.
> >>
> >>
> >> 2) Use a xor-type REQUIRED_USE for those packages.
> >>
> >> Put the whole set of PYTHON_TARGETS but add a REQUIRED_USE='^^ ( ... )'
> >> for them, effectively requesting only a single implementation being
> >> enabled.
> >>
> >> I believe that this is quite a good solution, at least from
> >> the dependency point of view. We clearly express which Python
> >> implementations are supported by a particular package and which one was
> >> enabled. We can express cross-package dependencies cleanly.
> >>
> >> The problem lies in user-friendliness. Although with the current
> >> default (python2_7 only) it wouldn't cause any trouble, whenever user
> >> enables more than a single implementation, every single-implementation
> >> package will require package.use adjustment. This will become an even
> >> more widespread issue when we decide to re-enable Python 3.
> >>
> >> To be honest, I don't see any good way around that.
> >>
> >>
> >> 3) Use implicit implementation selection (like python.eclass).
> >>
> >> Well, as some say, this is a very good solution since it's well tested.
> >> Its limitations and brokenness are obvious. Just I doubt it is really
> >> worth the effort to write something that bad.
> >>
> >> The main problem here is that the chosen Python implementation is
> >> implicit. Binary packages don't express it. Cross-package dependencies
> >> don't express it. User changes the implementation, stuff breaks
> >> silently and you end up with some kind of python-updater (why a tool
> >> to fix breakage is called 'updater'?!).
> >>
> >>
> >> Do you have any more ideas? Opinions?
> >>
> >
> > Like you, I really can't come up with an ideal solution here.
> >
> > We really have 2 classes of packages here:
> >
> > 1. Packages that don't care what version of python you use, but
> > install files outside of site-packages.
> >
> > 2. Packages that build code (like libraries) against a specific
> > version of python/libpython.
> >
> > The implicit implementation selection works fine for #1, but not so well for #2.
> 
> I wonder if we could have a "special target" that could expand to the
> current eselected python version. That way, the package not supporting
> a multiple targets, could be treated the same as those who do.
> 
> Something like:
> 
> PYTHON_COMPAT=( python_current ), or something like that.

Well, unless I'm missing something, that's just a tiny implementation
detail and I don't like to get that deep already.

Unless you are actually suggesting using that special target in deps
and so on -- but that effectively means 3) and makes it impossible to
cleanly support changing the current Python version.

-- 
Best regards,
Michał Górny

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 316 bytes --]

Re: [gentoo-python] Re: Handling packages not supporting multiple Python implementations

[Michał Górny]

[-- Attachment #1: Type: text/plain, Size: 3582 bytes --]

On Thu, 25 Oct 2012 17:00:22 -0400
Mike Gilbert <floppym@gentoo.org> wrote:

> On Tue, Oct 23, 2012 at 5:58 PM, Michał Górny <mgorny@gentoo.org> wrote:
> > Hello,
> >
> > After starting to deploy python-r1 on packages supporting multiple
> > Python implementations, I believe it is time to start thinking about
> > those packages which don't support that. Firstly, I would like to gain
> > a general feedback/ideas on the possible solutions, without getting too
> > deep into the technical details of it.
> >
> > As far as I can think, we have the following possibilities:
> >
> >
> > 1) Assume that installing stuff for a single Python implementation is
> > deprecated and let the packages rot with the old eclass.
> >
> > It is probably the simplest solution (i.e. not doing anything to
> > address the issue) but truth be told, I doubt this will actually work.
> > People will just keep using the old eclass which doesn't really do much
> > good for those packages...
> >
> > And even if some people will actually start supporting multiple
> > implementations... that may be even worse. Just look at dev-libs/boost
> > to see what I mean.
> >
> >
> > 2) Use a xor-type REQUIRED_USE for those packages.
> >
> > Put the whole set of PYTHON_TARGETS but add a REQUIRED_USE='^^ ( ... )'
> > for them, effectively requesting only a single implementation being
> > enabled.
> >
> > I believe that this is quite a good solution, at least from
> > the dependency point of view. We clearly express which Python
> > implementations are supported by a particular package and which one was
> > enabled. We can express cross-package dependencies cleanly.
> >
> > The problem lies in user-friendliness. Although with the current
> > default (python2_7 only) it wouldn't cause any trouble, whenever user
> > enables more than a single implementation, every single-implementation
> > package will require package.use adjustment. This will become an even
> > more widespread issue when we decide to re-enable Python 3.
> >
> > To be honest, I don't see any good way around that.
> >
> >
> > 3) Use implicit implementation selection (like python.eclass).
> >
> > Well, as some say, this is a very good solution since it's well tested.
> > Its limitations and brokenness are obvious. Just I doubt it is really
> > worth the effort to write something that bad.
> >
> > The main problem here is that the chosen Python implementation is
> > implicit. Binary packages don't express it. Cross-package dependencies
> > don't express it. User changes the implementation, stuff breaks
> > silently and you end up with some kind of python-updater (why a tool
> > to fix breakage is called 'updater'?!).
> >
> >
> > Do you have any more ideas? Opinions?
> >
> 
> Like you, I really can't come up with an ideal solution here.
> 
> We really have 2 classes of packages here:

Thanks for pointing that out.

> 1. Packages that don't care what version of python you use, but
> install files outside of site-packages.

That sounds a bit like a custom case to me. Not sure if python-r1
should support those out-of-the-box or just provide a few utility
functions (python-utils-r1?) to help installing them.

> 2. Packages that build code (like libraries) against a specific
> version of python/libpython.
> 
> The implicit implementation selection works fine for #1, but not so well for #2.

Indeed. The #2 will be probably handled through REQUIRED_USE, if noone
comes up with a better idea.

-- 
Best regards,
Michał Górny

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 316 bytes --]

Re: [gentoo-python] Re: Handling packages not supporting multiple Python implementations

[Mike Gilbert]

On Fri, Oct 26, 2012 at 5:41 PM, Michał Górny <mgorny@gentoo.org> wrote:
> On Thu, 25 Oct 2012 17:00:22 -0400
> Mike Gilbert <floppym@gentoo.org> wrote:
>
>> On Tue, Oct 23, 2012 at 5:58 PM, Michał Górny <mgorny@gentoo.org> wrote:
>> > Hello,
>> >
>> > After starting to deploy python-r1 on packages supporting multiple
>> > Python implementations, I believe it is time to start thinking about
>> > those packages which don't support that. Firstly, I would like to gain
>> > a general feedback/ideas on the possible solutions, without getting too
>> > deep into the technical details of it.
>> >
>> > As far as I can think, we have the following possibilities:
>> >
>> >
>> > 1) Assume that installing stuff for a single Python implementation is
>> > deprecated and let the packages rot with the old eclass.
>> >
>> > It is probably the simplest solution (i.e. not doing anything to
>> > address the issue) but truth be told, I doubt this will actually work.
>> > People will just keep using the old eclass which doesn't really do much
>> > good for those packages...
>> >
>> > And even if some people will actually start supporting multiple
>> > implementations... that may be even worse. Just look at dev-libs/boost
>> > to see what I mean.
>> >
>> >
>> > 2) Use a xor-type REQUIRED_USE for those packages.
>> >
>> > Put the whole set of PYTHON_TARGETS but add a REQUIRED_USE='^^ ( ... )'
>> > for them, effectively requesting only a single implementation being
>> > enabled.
>> >
>> > I believe that this is quite a good solution, at least from
>> > the dependency point of view. We clearly express which Python
>> > implementations are supported by a particular package and which one was
>> > enabled. We can express cross-package dependencies cleanly.
>> >
>> > The problem lies in user-friendliness. Although with the current
>> > default (python2_7 only) it wouldn't cause any trouble, whenever user
>> > enables more than a single implementation, every single-implementation
>> > package will require package.use adjustment. This will become an even
>> > more widespread issue when we decide to re-enable Python 3.
>> >
>> > To be honest, I don't see any good way around that.
>> >
>> >
>> > 3) Use implicit implementation selection (like python.eclass).
>> >
>> > Well, as some say, this is a very good solution since it's well tested.
>> > Its limitations and brokenness are obvious. Just I doubt it is really
>> > worth the effort to write something that bad.
>> >
>> > The main problem here is that the chosen Python implementation is
>> > implicit. Binary packages don't express it. Cross-package dependencies
>> > don't express it. User changes the implementation, stuff breaks
>> > silently and you end up with some kind of python-updater (why a tool
>> > to fix breakage is called 'updater'?!).
>> >
>> >
>> > Do you have any more ideas? Opinions?
>> >
>>
>> Like you, I really can't come up with an ideal solution here.
>>
>> We really have 2 classes of packages here:
>
> Thanks for pointing that out.
>
>> 1. Packages that don't care what version of python you use, but
>> install files outside of site-packages.
>
> That sounds a bit like a custom case to me. Not sure if python-r1
> should support those out-of-the-box or just provide a few utility
> functions (python-utils-r1?) to help installing them.
>
>> 2. Packages that build code (like libraries) against a specific
>> version of python/libpython.
>>
>> The implicit implementation selection works fine for #1, but not so well for #2.
>
> Indeed. The #2 will be probably handled through REQUIRED_USE, if noone
> comes up with a better idea.
>

Yeah, I probably need to remove python3_2 from arch/*/make.defaults
before we move forward with that plan. I'm sure that will make a few
people feel better anyway.

Re: [gentoo-python] Re: Handling packages not supporting multiple Python implementations

[Michał Górny]

[-- Attachment #1: Type: text/plain, Size: 4333 bytes --]

On Fri, 26 Oct 2012 19:55:30 -0400
Mike Gilbert <floppym@gentoo.org> wrote:

> On Fri, Oct 26, 2012 at 5:41 PM, Michał Górny <mgorny@gentoo.org> wrote:
> > On Thu, 25 Oct 2012 17:00:22 -0400
> > Mike Gilbert <floppym@gentoo.org> wrote:
> >
> >> On Tue, Oct 23, 2012 at 5:58 PM, Michał Górny <mgorny@gentoo.org> wrote:
> >> > Hello,
> >> >
> >> > After starting to deploy python-r1 on packages supporting multiple
> >> > Python implementations, I believe it is time to start thinking about
> >> > those packages which don't support that. Firstly, I would like to gain
> >> > a general feedback/ideas on the possible solutions, without getting too
> >> > deep into the technical details of it.
> >> >
> >> > As far as I can think, we have the following possibilities:
> >> >
> >> >
> >> > 1) Assume that installing stuff for a single Python implementation is
> >> > deprecated and let the packages rot with the old eclass.
> >> >
> >> > It is probably the simplest solution (i.e. not doing anything to
> >> > address the issue) but truth be told, I doubt this will actually work.
> >> > People will just keep using the old eclass which doesn't really do much
> >> > good for those packages...
> >> >
> >> > And even if some people will actually start supporting multiple
> >> > implementations... that may be even worse. Just look at dev-libs/boost
> >> > to see what I mean.
> >> >
> >> >
> >> > 2) Use a xor-type REQUIRED_USE for those packages.
> >> >
> >> > Put the whole set of PYTHON_TARGETS but add a REQUIRED_USE='^^ ( ... )'
> >> > for them, effectively requesting only a single implementation being
> >> > enabled.
> >> >
> >> > I believe that this is quite a good solution, at least from
> >> > the dependency point of view. We clearly express which Python
> >> > implementations are supported by a particular package and which one was
> >> > enabled. We can express cross-package dependencies cleanly.
> >> >
> >> > The problem lies in user-friendliness. Although with the current
> >> > default (python2_7 only) it wouldn't cause any trouble, whenever user
> >> > enables more than a single implementation, every single-implementation
> >> > package will require package.use adjustment. This will become an even
> >> > more widespread issue when we decide to re-enable Python 3.
> >> >
> >> > To be honest, I don't see any good way around that.
> >> >
> >> >
> >> > 3) Use implicit implementation selection (like python.eclass).
> >> >
> >> > Well, as some say, this is a very good solution since it's well tested.
> >> > Its limitations and brokenness are obvious. Just I doubt it is really
> >> > worth the effort to write something that bad.
> >> >
> >> > The main problem here is that the chosen Python implementation is
> >> > implicit. Binary packages don't express it. Cross-package dependencies
> >> > don't express it. User changes the implementation, stuff breaks
> >> > silently and you end up with some kind of python-updater (why a tool
> >> > to fix breakage is called 'updater'?!).
> >> >
> >> >
> >> > Do you have any more ideas? Opinions?
> >> >
> >>
> >> Like you, I really can't come up with an ideal solution here.
> >>
> >> We really have 2 classes of packages here:
> >
> > Thanks for pointing that out.
> >
> >> 1. Packages that don't care what version of python you use, but
> >> install files outside of site-packages.
> >
> > That sounds a bit like a custom case to me. Not sure if python-r1
> > should support those out-of-the-box or just provide a few utility
> > functions (python-utils-r1?) to help installing them.
> >
> >> 2. Packages that build code (like libraries) against a specific
> >> version of python/libpython.
> >>
> >> The implicit implementation selection works fine for #1, but not so well for #2.
> >
> > Indeed. The #2 will be probably handled through REQUIRED_USE, if noone
> > comes up with a better idea.
> >
> 
> Yeah, I probably need to remove python3_2 from arch/*/make.defaults
> before we move forward with that plan. I'm sure that will make a few
> people feel better anyway.

Hmm, so we still have that somewhere? I thought folks forced us to
remove it completely. Probably p-d-ng wasn't spread enough for them to
notice ;).

-- 
Best regards,
Michał Górny

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 316 bytes --]