[gentoo-user] python mess - random winge!

[gentoo-user] python mess - random winge!

[William Kenworthy]

I synced portage a couple of days now and now my systems are rebuilding
python modules for 3.10 without any input from me (prior to this 3.10
was on the system but wasn't picked up by applications.)  This is
breaking non portage apps like homeassistant which are still not fully
3.10 safe - ok that's sort of expected and in this case will be fixed,
but I cant find anything definitive on the task of "I want to control
which python is used" and when to update.

I eventually found that changing the order in python-exec.conf helped on
the homeassistant system.  There is a LOT of out of date documentation
out there, particularly with eselect being used but is actually not used
with python anymore (why? - from a user point of view having consistent
access to configuration is a no brainer!) - so how can one get python to
behave reliably and override its automatic get things wrong installation
system?  Is manually editing python-exec.conf the way (which seems to
get overwritten - shouldn't that be a protected config file then?)

BillK

[gentoo-user] Re: python mess - random winge!

[Nikos Chantziaras]

On 05/07/2022 08:04, William Kenworthy wrote:
> I synced portage a couple of days now and now my systems are rebuilding
> python modules for 3.10 without any input from me [...]
Yes, that's normal and there was a news item about it. Do:

   eselect news list

to get the and then use the NUMBER of the "Python 3.10 to become the 
default on 2022-07-01" news item:

   eselect read NUMBER

This will give you details on how to do the upgrade properly.

Re: [gentoo-user] python mess - random winge!

[cal]

On 7/4/22 22:04, William Kenworthy wrote:
> I synced portage a couple of days now and now my systems are rebuilding
> python modules for 3.10 without any input from me (prior to this 3.10
> was on the system but wasn't picked up by applications.)  This is
> breaking non portage apps like homeassistant which are still not fully
> 3.10 safe - ok that's sort of expected and in this case will be fixed,
> but I cant find anything definitive on the task of "I want to control
> which python is used" and when to update.

As Nikos mentioned in another reply, you should pay attention to the
eselect news items as python upgrades are usually announced weeks in
advance.

> 
> I eventually found that changing the order in python-exec.conf helped on
> the homeassistant system.

> There is a LOT of out of date documentation
> out there, particularly with eselect being used but is actually not used
> with python anymore (why? - from a user point of view having consistent
> access to configuration is a no brainer!)

I don't see any deprecation notice on
https://packages.gentoo.org/packages/app-eselect/eselect-python, but
someone could correct me.

> - so how can one get python to
> behave reliably and override its automatic get things wrong installation
> system?  Is manually editing python-exec.conf the way (which seems to
> get overwritten - shouldn't that be a protected config file then?)

The python package supports installing multiple versions to different
slots.  The easiest way I've found to run python software that depends
on a specific version, without mucking around with global python
settings, is to install the relevant version (e.g. emerge python:3.8)
and then use venv to run that software in a virtual environment
(python3.8 -m venv .env; .env/bin/python something.py).

> 
> BillK

Re: [gentoo-user] python mess - random winge!

[wkuz]

[-- Attachment #1: Type: text/plain, Size: 1755 bytes --]

Dnia 2022-07-05, o godz. 13:04:07
William Kenworthy <billk@iinet.net.au> napisał(a):

> I synced portage a couple of days now and now my systems are
> rebuilding python modules for 3.10 without any input from me (prior
> to this 3.10 was on the system but wasn't picked up by applications.)
>  This is breaking non portage apps like homeassistant which are still
> not fully 3.10 safe - ok that's sort of expected and in this case
> will be fixed, but I cant find anything definitive on the task of "I
> want to control which python is used" and when to update.
> 
> I eventually found that changing the order in python-exec.conf helped
> on the homeassistant system.  There is a LOT of out of date
> documentation out there, particularly with eselect being used but is
> actually not used with python anymore (why? - from a user point of
> view having consistent access to configuration is a no brainer!) - so
> how can one get python to behave reliably and override its automatic
> get things wrong installation system?  Is manually editing
> python-exec.conf the way (which seems to get overwritten - shouldn't
> that be a protected config file then?)
> 
> BillK
> 
> 
> 
> 

Hello!

In "eselect news" info about python update there is a paragraph about
blocking the upgrade. It just means adding:

	*/* PYTHON_TARGETS: -* python3_9
	*/* PYTHON_SINGLE_TARGET: -* python3_9

to /etc/portage/make.conf or /etc/portage/package.use or
/etc/portage/package.use/zz-somename - whichever suites you best.

You can also change these settings just for some packages, by adding:

cat/pkg PYTHON_TARGETS: -* python3_9 PYTHON_SINGLE_TARGET: -* python3_9

to one of aforementioned files.

Hope that helps!

-- 
xWK

[-- Attachment #2: Podpis cyfrowy OpenPGP --]
[-- Type: application/pgp-signature, Size: 228 bytes --]

Re: [gentoo-user] python mess - random winge!

[William Kenworthy]

[-- Attachment #1: Type: text/plain, Size: 2454 bytes --]


On 5/7/22 14:24, wkuz@op.pl wrote:
> Dnia 2022-07-05, o godz. 13:04:07
> William Kenworthy <billk@iinet.net.au> napisał(a):
>
>> I synced portage a couple of days now and now my systems are
>> rebuilding python modules for 3.10 without any input from me (prior
>> to this 3.10 was on the system but wasn't picked up by applications.)
>>  This is breaking non portage apps like homeassistant which are still
>> not fully 3.10 safe - ok that's sort of expected and in this case
>> will be fixed, but I cant find anything definitive on the task of "I
>> want to control which python is used" and when to update.
>>
>> I eventually found that changing the order in python-exec.conf helped
>> on the homeassistant system.  There is a LOT of out of date
>> documentation out there, particularly with eselect being used but is
>> actually not used with python anymore (why? - from a user point of
>> view having consistent access to configuration is a no brainer!) - so
>> how can one get python to behave reliably and override its automatic
>> get things wrong installation system?  Is manually editing
>> python-exec.conf the way (which seems to get overwritten - shouldn't
>> that be a protected config file then?)
>>
>> BillK
>>
>>
>>
>>
> Hello!
>
> In "eselect news" info about python update there is a paragraph about
> blocking the upgrade. It just means adding:
>
> 	*/* PYTHON_TARGETS: -* python3_9
> 	*/* PYTHON_SINGLE_TARGET: -* python3_9
>
> to /etc/portage/make.conf or /etc/portage/package.use or
> /etc/portage/package.use/zz-somename - whichever suites you best.
>
> You can also change these settings just for some packages, by adding:
>
> cat/pkg PYTHON_TARGETS: -* python3_9 PYTHON_SINGLE_TARGET: -* python3_9
>
> to one of aforementioned files.
>
> Hope that helps!


I did read the news item and set the systems as above with multiple
python targets - there is no mention of python-exec and its role in
which python version is in use for packages that just call "python". 
Perhaps I should have been clearer - what I see is with multiple python
targets present the python ebuild automatically selects the latest
version that is stable via python-exec - ok, some would want that.  But
what it should do is respect the users choice of running version and not
automaticly overide it without asking.  It looks like python-exec is the
controlling factor so I'll try CONFIG_PROTECTon that file and manually
manage it via ansible.

BillK


[-- Attachment #2: Type: text/html, Size: 3207 bytes --]

Re: [gentoo-user] python mess - random winge!

[Neil Bothwick]

[-- Attachment #1: Type: text/plain, Size: 1092 bytes --]

On Tue, 5 Jul 2022 15:31:38 +0800, William Kenworthy wrote:

> I did read the news item and set the systems as above with multiple
> python targets - there is no mention of python-exec and its role in
> which python version is in use for packages that just call "python". 
> Perhaps I should have been clearer - what I see is with multiple python
> targets present the python ebuild automatically selects the latest
> version that is stable via python-exec - ok, some would want that.  But
> what it should do is respect the users choice of running version and not
> automaticly overide it without asking.  It looks like python-exec is the
> controlling factor so I'll try CONFIG_PROTECTon that file and manually
> manage it via ansible.

It is already CONFIG_PROTECTed, I had to approve the changes in the usual
way before they went ahead. What I find slightly odd is that this file is
also managed by eselect-python, but that is not installed by default. I
would have expected it to be part of @system.


-- 
Neil Bothwick

Quick!! Act as if nothing has happened!

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [gentoo-user] python mess - random winge!

[Petr Vaněk]

On Tue, Jul 05, 2022 at 08:56:11AM +0100, Neil Bothwick wrote:
> On Tue, 5 Jul 2022 15:31:38 +0800, William Kenworthy wrote:
> 
> > I did read the news item and set the systems as above with multiple
> > python targets - there is no mention of python-exec and its role in
> > which python version is in use for packages that just call "python". 
> > Perhaps I should have been clearer - what I see is with multiple python
> > targets present the python ebuild automatically selects the latest
> > version that is stable via python-exec - ok, some would want that.  But
> > what it should do is respect the users choice of running version and not
> > automaticly overide it without asking.  It looks like python-exec is the
> > controlling factor so I'll try CONFIG_PROTECTon that file and manually
> > manage it via ansible.
> 
> It is already CONFIG_PROTECTed, I had to approve the changes in the usual
> way before they went ahead. What I find slightly odd is that this file is
> also managed by eselect-python, but that is not installed by default. I
> would have expected it to be part of @system.

This change is described in
https://www.gentoo.org/support/news-items/2021-01-30-python-preference-to-follow-python-targets.html

Petr

Re: [gentoo-user] python mess - random winge!

[Neil Bothwick]

[-- Attachment #1: Type: text/plain, Size: 626 bytes --]

On Tue, 5 Jul 2022 11:17:27 +0200, Petr Vaněk wrote:

> > It is already CONFIG_PROTECTed, I had to approve the changes in the
> > usual way before they went ahead. What I find slightly odd is that
> > this file is also managed by eselect-python, but that is not
> > installed by default. I would have expected it to be part of @system.
> >  
> 
> This change is described in
> https://www.gentoo.org/support/news-items/2021-01-30-python-preference-to-follow-python-targets.html

Worth reading, I missed that one, thanks.


-- 
Neil Bothwick

"Do you reply to our surveys.?"
[X]Never [ ]Always [ ]Sometimes

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

[gentoo-user] Re: python mess - random winge!

[Grant Edwards]

On 2022-07-05, William Kenworthy <billk@iinet.net.au> wrote:

> I synced portage a couple of days now and now my systems are rebuilding
> python modules for 3.10 without any input from me [...]

Every time there's a Python upgrade like this, it turns into a bit of
an ordeal because I always have a small handful of packages that don't
support the newer version.

The news item offers no advice on what to do in this situation other
than completely postponing the upgrade of everything (which doesn't
seem like the best choice.)

It would be nice if the news item explained how to let the upgrade
procede while holding back a few packages.

Can you set 3_9 and 3_10 globally, and then disable 3_10 for a few
individual packages that can't be built with 3_10?

--
Grant

Re: [gentoo-user] Re: python mess - random winge!

[Jack]

On 2022.07.05 12:24, Grant Edwards wrote:
> On 2022-07-05, William Kenworthy <billk@iinet.net.au> wrote:
> 
> > I synced portage a couple of days now and now my systems are  
> rebuilding
> > python modules for 3.10 without any input from me [...]
> 
> Every time there's a Python upgrade like this, it turns into a bit of
> an ordeal because I always have a small handful of packages that don't
> support the newer version.
> 
> The news item offers no advice on what to do in this situation other
> than completely postponing the upgrade of everything (which doesn't
> seem like the best choice.)
> 
> It would be nice if the news item explained how to let the upgrade
> procede while holding back a few packages.
> 
> Can you set 3_9 and 3_10 globally, and then disable 3_10 for a few
> individual packages that can't be built with 3_10?
As far as I can tell, you just need to add python_targets_python3_9 for  
the package in the appropriate package.use file.

[gentoo-user] Re: python mess - random winge!

[Grant Edwards]

On 2022-07-05, Jack <ostroffjh@users.sourceforge.net> wrote:
> On 2022.07.05 12:24, Grant Edwards wrote:
>> On 2022-07-05, William Kenworthy <billk@iinet.net.au> wrote:

>> It would be nice if the news item explained how to let the upgrade
>> procede while holding back a few packages.
>> 
>> Can you set 3_9 and 3_10 globally, and then disable 3_10 for a few
>> individual packages that can't be built with 3_10?

> As far as I can tell, you just need to add python_targets_python3_9 for  
> the package in the appropriate package.use file.

I've tried that, and it takes forever. Everything time you do it, the
next emerge attempt will fail because one of that package's
dependencies doesn't have 3_9 set. So you set that one, do an emerge,
and it fails because there's another depenency that doesn't have 3_9
set. Repeat this for an hour or two...

If it would tell you about all of them at once, it wouldn't be so bad.
But, if you're trying to hold back a large application with dozens and
dozens of dependancies it makes you want to scream.

Or doesn't it torture you like that any longer?

--
Grant

Re: [gentoo-user] Re: python mess - random winge!

[Jack]

On 2022.07.05 12:43, Grant Edwards wrote:
> On 2022-07-05, Jack <ostroffjh@users.sourceforge.net> wrote:
> > On 2022.07.05 12:24, Grant Edwards wrote:
> >> On 2022-07-05, William Kenworthy <billk@iinet.net.au> wrote:
> 
> >> It would be nice if the news item explained how to let the upgrade
> >> procede while holding back a few packages.
> >>
> >> Can you set 3_9 and 3_10 globally, and then disable 3_10 for a few
> >> individual packages that can't be built with 3_10?
> 
> > As far as I can tell, you just need to add python_targets_python3_9  
> for
> > the package in the appropriate package.use file.
> 
> I've tried that, and it takes forever. Everything time you do it, the
> next emerge attempt will fail because one of that package's
> dependencies doesn't have 3_9 set. So you set that one, do an emerge,
> and it fails because there's another depenency that doesn't have 3_9
> set. Repeat this for an hour or two...
> 
> If it would tell you about all of them at once, it wouldn't be so bad.
> But, if you're trying to hold back a large application with dozens and
> dozens of dependancies it makes you want to scream.
> 
> Or doesn't it torture you like that any longer?
Oh, it still does.  I just think I anticipate it more often, so I am  
frequently able to deal with more than one "layer" in each round, so  
the total process doesn't take so long.  I've run into problems both  
for dependencies of the package I changed first, and packages for which  
It is a dependency.

On major updates, such as KDE, emerge @world often shows me a long list  
of packages it will upgrade, followed by a long list of slot  
conflicts.  Over the years I've not necessarily gotten better at  
directly interpreting portage's output, but at least figuring out I've  
got one package somehow related to the ones to be upgraded  (usually  
somewhere in the dependency tree) which itself doesn't have an upgrade,  
but does need to be rebuilt  at the same time.  I'll often end up doing  
"eix-installed -a | grep kde" (or python) and look for a package that  
isn't being upgraded, but is in the dependency tree of one or more  
which is.  It is also common that the problem is a package I've added  
under /etc/portage, and I just need to figure out if it still needs to  
be there, or else how that non-standard entry affects other packages.

Re: [gentoo-user] Re: python mess - random winge!

[Rich Freeman]

On Tue, Jul 5, 2022 at 12:36 PM Jack <ostroffjh@users.sourceforge.net> wrote:
>
> On 2022.07.05 12:24, Grant Edwards wrote:
> > On 2022-07-05, William Kenworthy <billk@iinet.net.au> wrote:
> >
> > > I synced portage a couple of days now and now my systems are
> > rebuilding
> > > python modules for 3.10 without any input from me [...]
> >
> > Every time there's a Python upgrade like this, it turns into a bit of
> > an ordeal because I always have a small handful of packages that don't
> > support the newer version.
> >
> > The news item offers no advice on what to do in this situation other
> > than completely postponing the upgrade of everything (which doesn't
> > seem like the best choice.)
> >
> > It would be nice if the news item explained how to let the upgrade
> > procede while holding back a few packages.
> >
> > Can you set 3_9 and 3_10 globally, and then disable 3_10 for a few
> > individual packages that can't be built with 3_10?
> As far as I can tell, you just need to add python_targets_python3_9 for
> the package in the appropriate package.use file.
>

That and its dependencies.  Obviously you need to be caught up before
things get removed from the repo, but the offending package itself
will get removed when that happens anyway.

You can always just globally keep the older version around longer if
you don't want to deal with a bunch of cruft in
/etc/portage/package.use.  The news item explains how to do this.

-- 
Rich

Re: [gentoo-user] Re: python mess - random winge!

[Wols Lists]

On 05/07/2022 17:44, Rich Freeman wrote:
> hat and its dependencies.  Obviously you need to be caught up before
> things get removed from the repo, but the offending package itself
> will get removed when that happens anyway.
> 
> You can always just globally keep the older version around longer if
> you don't want to deal with a bunch of cruft in
> /etc/portage/package.use.  The news item explains how to do this.

I'd be inclined to lock the older version of the package you want so it 
won't upgrade automatically (don't know how to do this, probably put 
that particular version in @world), and also give that a python 9 
dependency in package.use.

That way, when they update it to python 10, you can delete the version 
lock on your package, and it will upgrade and get rid of all the python 
9 stuff without you having to worry about it.

I always try and make sure anything I don't care about but that needs 
specific stuff is always specified by version. That way, as the system 
moves on, all that cruft just falls off the wagon ...

Cheers,
Wol

Re: [gentoo-user] Re: python mess - random winge!

[Rich Freeman]

On Tue, Jul 5, 2022 at 1:10 PM Wols Lists <antlists@youngman.org.uk> wrote:
>
> On 05/07/2022 17:44, Rich Freeman wrote:
> > hat and its dependencies.  Obviously you need to be caught up before
> > things get removed from the repo, but the offending package itself
> > will get removed when that happens anyway.
> >
> > You can always just globally keep the older version around longer if
> > you don't want to deal with a bunch of cruft in
> > /etc/portage/package.use.  The news item explains how to do this.
>
> I'd be inclined to lock the older version of the package you want so it
> won't upgrade automatically (don't know how to do this, probably put
> that particular version in @world), and also give that a python 9
> dependency in package.use.

The news item explains how to get the upgrade earlier, delay the
upgrade entirely, or have support for both versions.  I'd suggest
doing it the way the news item suggests, because when you start
fighting the devs who maintain the python update logic, you're
probably going to find yourself swearing a lot.

-- 
Rich