[gentoo-python] Supporting non-UTF-8 locales

[gentoo-python] Supporting non-UTF-8 locales

[Mike Gilbert]

This issue seems to keep coming up in bug reports, especially those
filed by Diego from his tinderbox. Basically, the way I/O is handled
in python3 often causes build failures or test failures if you do not
have a UTF-8 locale set. Ideally, upstream developers would write
their code to deal with this, but that's a bit of a pipe dream.

Gentoo as a whole can't seem to make its mind up, but can the python
team come to a rough consensus as to whether we want to support
non-UTF-8 locales for python packages?

I'm sitting on the fence at the moment: I tend to fix issues if they
cause build failures, but I ignore any test failures.

If we decide to support non-UTF-8 locales, we need to stop leaving
bugzilla comments like "please use a sane locale".

If we decide against supporting non-UTF-8 locales, I think we should
begin to close such bugs as WONTFIX on a consistent basis.

My vote would be to not support non-UTF-8 locales -- it just isn't
worth the effort.

Thoughts?

Re: [gentoo-python] Supporting non-UTF-8 locales

[Brian Dolbec]

On Sun, 2013-02-17 at 13:48 -0500, Mike Gilbert wrote:

> My vote would be to not support non-UTF-8 locales -- it just isn't
> worth the effort.
> 
> Thoughts?
> 

+1, but I'm not also using/needing a non-utf8 locale.  Those that do
need a non-utf8 locale need to speak up as for the reasons.

Perhaps if more distros force utf-8 locales, then those apps/sites that
are using non-utf-8 will migrate their code to a utf-8 equivalent
(hopefully)

Re: [gentoo-python] Supporting non-UTF-8 locales

[IAN DELANEY]

On Sun, 17 Feb 2013 13:48:28 -0500
Mike Gilbert <floppym@gentoo.org> wrote:

> This issue seems to keep coming up in bug reports, especially those
> filed by Diego from his tinderbox. Basically, the way I/O is handled
> in python3 often causes build failures or test failures if you do not
> have a UTF-8 locale set. Ideally, upstream developers would write
> their code to deal with this, but that's a bit of a pipe dream.
> 
> Gentoo as a whole can't seem to make its mind up, but can the python
> team come to a rough consensus as to whether we want to support
> non-UTF-8 locales for python packages?
> 
> I'm sitting on the fence at the moment: I tend to fix issues if they
> cause build failures, but I ignore any test failures.
> 
Well, I tend to fix issues, be they builds of tests.

> If we decide to support non-UTF-8 locales, we need to stop leaving
> bugzilla comments like "please use a sane locale".
> 
> If we decide against supporting non-UTF-8 locales, I think we should
> begin to close such bugs as WONTFIX on a consistent basis.
> 
> My vote would be to not support non-UTF-8 locales -- it just isn't
> worth the effort.
> 
> Thoughts?
> 

Firstly, thumbs up for floating the issue. "please use a sane locale"
fundamentally undermines the merit of the tinderbox meister making such
bugs.  He deliberately makes such locales to flush out such packages
supporting only the utf8.  He's clearly convinced of the righteousness
of the bug submissions, but then again, so to for everything else he
does under the banner of working gentoo. 

The ignoring of such tests to me implies it's not really a bug, and
frankly this notion has obvious merit.  If there are utf and non utf
locales and the utf makes it work, then arguably the package is
buildable and installable.  The selection of a normative locale across
the board has obvious merit. If not utf8 simply einfo the user to set
one.

One statement arguing against I could cite from am upstream dev who,
lumbered with this exact task, agreed in principal they should not
assume a utf locale, and a fix for it has been made already by a few
maintainers of other packages, e.g. ipython.  Trouble is, he's one of a
very long queue of upstream devs.

to not support non-UTF-8 locales -- it just isn't
> worth the effort.

This is more or less the assumed and invisible policy, and
frankly on that basis I go along with Mike's selection.  Making this a
formal issue and declaring a python team policy is afaict the only
legitimate method of 'informing' the tinderbox meister these are not
bugs, these just further spam bugzilla. Then we can close such bugs
as WONTFIX  on a consistent basis without anticipation of the next dose
o' the long ago banned whip. After all, these bug submissions come
almost exclusively from him.

-- 
kind regards

Ian Delaney

Re: [gentoo-python] Supporting non-UTF-8 locales

[Michał Górny]

[-- Attachment #1: Type: text/plain, Size: 358 bytes --]

On Sun, 17 Feb 2013 13:48:28 -0500
Mike Gilbert <floppym@gentoo.org> wrote:

> My vote would be to not support non-UTF-8 locales -- it just isn't
> worth the effort.
> 
> Thoughts?

How about following upstream here? Reporting the bugs upstream, letting
them fix or WONTFIX them, and following their judgment.

-- 
Best regards,
Michał Górny

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 316 bytes --]

Re: [gentoo-python] Supporting non-UTF-8 locales

[Mike Gilbert]

On Mon, Feb 18, 2013 at 3:59 AM, Michał Górny <mgorny@gentoo.org> wrote:
> On Sun, 17 Feb 2013 13:48:28 -0500
> Mike Gilbert <floppym@gentoo.org> wrote:
>
>> My vote would be to not support non-UTF-8 locales -- it just isn't
>> worth the effort.
>>
>> Thoughts?
>
> How about following upstream here? Reporting the bugs upstream, letting
> them fix or WONTFIX them, and following their judgment.
>

I suspect a common outcome will be the the third option: upstream
ignores the bug. But it still seems like a sensible thing to do.

For any other user, I would tell them to report the bug upstream
themselves. I'll have to see how Diego responds to that. ;-)

[gentoo-python] Re: Supporting non-UTF-8 locales

[Mike Gilbert]

On Mon, Feb 18, 2013 at 3:23 AM, Dirkjan Ochtman <dirkjan@ochtman.nl> wrote:
> On Sun, Feb 17, 2013 at 7:48 PM, Mike Gilbert <floppym@gentoo.org> wrote:
>> My vote would be to not support non-UTF-8 locales -- it just isn't
>> worth the effort.
>>
>> Thoughts?
>
> I agree, though I think there are some people in the wider Gentoo
> project who will thoroughly dislike that. And perhaps we should bring
> up the issue of changing the default locale on gentoo-dev, again?
>

I think the only way that is going to happen is if a "senior" dev just
goes ahead and does it. Or if the council actually makes a hard
decision for once.

I would prefer not to start another gentoo-dev thread, but I will
leave a +1 if anyone else wants to do so.