[gentoo-dev] [RFC] How to deal with LINGUAS mess?

[gentoo-dev] [RFC] How to deal with LINGUAS mess?

[Michał Górny]

[-- Attachment #1: Type: text/plain, Size: 2250 bytes --]

Hello,


Those of you who read my blog post on LINGUAS [1] may already know
what's going on. For those who didn't, short summary:

In EAPI 5 and newer, all variables listed in USE_EXPAND are supposed to
be unconditionally exported with their values reduced to enabled USE
flags listed in IUSE. In particular, this means that if ebuild does not
list any linguas_* flags in IUSE, PM exports *empty* LINGUAS (i.e.
disables all localizations with the implicit gettext behavior).

Portage had so far some ugly hack-logic that tried to keep LINGUAS
working somehow in place. However, the patches to enable PMS-compliant
behavior are on their way, so it's about time to decide what to do
about LINGUAS.


I see the following possibilities:

1. We start explicitly listing linguas_* in all ebuilds, no matter how
tiny they are. Maintainers are required to keep IUSE up-to-date
and users are forced to rebuild a lot. This is also a QA violation
in terms of invalid use of USE flags.

2. We hack-unset LINGUAS in ebuilds. This is a lot of effort, easy to
miss and probably would need to repeated for every single phase anyway
due to how global variables are handled in PMS. Additionally, it may
break at some point since those variables are likely expected to be
read-only anyway.

3. We remove LINGUAS from USE_EXPAND and stop using it. If ebuilds have
a good reason to use flags for localization, we introduce a new,
non-colliding USE_EXPAND for that. We also ask users to replace LINGUAS
with the new flag in their make.conf files. LINGUAS gets the original
upstream behavior back, and we eventually discourage it in favor of new
INSTALL_MASK features (WiP) [2].

4. We fix build systems not to do magic depending on whether LINGUAS
is unset or set-to-empty. Instead, we could some special special value
like '-' to signify not installing localizations at all. But that's
upstream thing to do, and breaks backwards compatibility with existing
systems disabling localizations.


Your thoughts?


[1]:https://blogs.gentoo.org/mgorny/2016/05/16/how-linguas-are-thrice-wrong/
[2]:https://wiki.gentoo.org/wiki/User:MGorny/GLEP:INSTALL_MASK

-- 
Best regards,
Michał Górny
<http://dev.gentoo.org/~mgorny/>

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 949 bytes --]

Re: [gentoo-dev] [RFC] How to deal with LINGUAS mess?

[Kent Fredric]

On 21 May 2016 at 19:41, Michał Górny <mgorny@gentoo.org> wrote:
> Hello,
>
>
> Those of you who read my blog post on LINGUAS [1] may already know
> what's going on. For those who didn't, short summary:
>
> In EAPI 5 and newer, all variables listed in USE_EXPAND are supposed to
> be unconditionally exported with their values reduced to enabled USE
> flags listed in IUSE. In particular, this means that if ebuild does not
> list any linguas_* flags in IUSE, PM exports *empty* LINGUAS (i.e.
> disables all localizations with the implicit gettext behavior).
>
> Portage had so far some ugly hack-logic that tried to keep LINGUAS
> working somehow in place. However, the patches to enable PMS-compliant
> behavior are on their way, so it's about time to decide what to do
> about LINGUAS.
>
>
> I see the following possibilities:
>
> 1. We start explicitly listing linguas_* in all ebuilds, no matter how
> tiny they are. Maintainers are required to keep IUSE up-to-date
> and users are forced to rebuild a lot. This is also a QA violation
> in terms of invalid use of USE flags.
>
> 2. We hack-unset LINGUAS in ebuilds. This is a lot of effort, easy to
> miss and probably would need to repeated for every single phase anyway
> due to how global variables are handled in PMS. Additionally, it may
> break at some point since those variables are likely expected to be
> read-only anyway.
>
> 3. We remove LINGUAS from USE_EXPAND and stop using it. If ebuilds have
> a good reason to use flags for localization, we introduce a new,
> non-colliding USE_EXPAND for that. We also ask users to replace LINGUAS
> with the new flag in their make.conf files. LINGUAS gets the original
> upstream behavior back, and we eventually discourage it in favor of new
> INSTALL_MASK features (WiP) [2].
>
> 4. We fix build systems not to do magic depending on whether LINGUAS
> is unset or set-to-empty. Instead, we could some special special value
> like '-' to signify not installing localizations at all. But that's
> upstream thing to do, and breaks backwards compatibility with existing
> systems disabling localizations.
>
>
> Your thoughts?

I think its a regretful situation we find ourselves in where
portage/PMS control values that have to go via ENV, leak beyond ENV
into packaging.

This has already been a bit of a worry for us, because we may have
wanted to use PERL_ as a prefix for env vars in an eclass, but there's
a possibility of accidentally picking the same ENV var that is
inherently used in some perl part.
( And at very least, `perl -V` shows all ENV vars that are prefixed
with PERL_ .... eg: PERL_MOM_YOURS=1 perl -V )

Though I just don't see how we can generically avoid that scope
leakage, because this is bash, and we're short on private ways to pass
control values around, other than going crazy on functions.

The only thing I thought of when reading the writeup was tagged
install mask verbs.

That is:

- Always install all language support possible by default.
- Have an ebuild mapping of some description that annotates "files X
add support for languages ( y, z, x )"

And then maybe you'd have a parameterized INSTALL_MASK system where you could do

INSTALL_MASK_RULES="linguas(-* en)"

Then it would be "Packager decides" if they want to trip a rebuild at
the end user when they "discovered" certain files pertain to specific
language support.

Then the only reason you'd need USE flags is if you wanted to toggle
compile time support that pulled in dependencies to support the
language in question.


-- 
Kent

KENTNL - https://metacpan.org/author/KENTNL

[gentoo-dev] Re: [RFC] How to deal with LINGUAS mess?

[Ulrich Mueller]

[-- Attachment #1: Type: text/plain, Size: 564 bytes --]

>>>>> On Sat, 21 May 2016, Michał Górny wrote:

> I see the following possibilities:

> 1. We start explicitly listing linguas_* in all ebuilds, no matter
> how tiny they are. Maintainers are required to keep IUSE up-to-date
> and users are forced to rebuild a lot.

Why would users have to rebuild more often? Language support in a
package will change with a version bump, when they must rebuild in any
case.

> This is also a QA violation in terms of invalid use of USE flags.

I fail to see why this would be a QA violation.

> [...]

Ulrich

[-- Attachment #2: Type: application/pgp-signature, Size: 490 bytes --]

Re: [gentoo-dev] Re: [RFC] How to deal with LINGUAS mess?

[Kent Fredric]

On 21 May 2016 at 21:00, Ulrich Mueller <ulm@gentoo.org> wrote:
> Why would users have to rebuild more often? Language support in a
> package will change with a version bump, when they must rebuild in any
> case.


But changing IUSE would cause users to cause rebuilds under --new-use,
even though no actual changes happened ( this happens a bit as it is
).

-- 
Kent

KENTNL - https://metacpan.org/author/KENTNL

Re: [gentoo-dev] Re: [RFC] How to deal with LINGUAS mess?

[Ulrich Mueller]

[-- Attachment #1: Type: text/plain, Size: 606 bytes --]

>>>>> On Sat, 21 May 2016, Kent Fredric wrote:

> On 21 May 2016 at 21:00, Ulrich Mueller <ulm@gentoo.org> wrote:
>> Why would users have to rebuild more often? Language support in a
>> package will change with a version bump, when they must rebuild in
>> any case.

> But changing IUSE would cause users to cause rebuilds under
> --new-use, even though no actual changes happened ( this happens a
> bit as it is ).

They would have to rebuild the package once, when linguas_* flags
are first added to IUSE. After that, I don't see what would trigger
additional rebuilds outside of a version bump.

Ulrich

[-- Attachment #2: Type: application/pgp-signature, Size: 490 bytes --]

Re: [gentoo-dev] [RFC] How to deal with LINGUAS mess?

[Michael Orlitzky]

On 05/21/2016 03:41 AM, Michał Górny wrote:
> 
> I see the following possibilities:
> 

#2 is ugly and requires a special case due to a bad choice of variable
name; #4 will never work.


> 3. We remove LINGUAS from USE_EXPAND and stop using it. If ebuilds have
> a good reason to use flags for localization, we introduce a new,
> non-colliding USE_EXPAND for that. We also ask users to replace LINGUAS
> with the new flag in their make.conf files. LINGUAS gets the original
> upstream behavior back, and we eventually discourage it in favor of new
> INSTALL_MASK features (WiP) [2].
> 

This is probably the best option because it fixes the real problem: we
tried to repurpose somebody else's environment variable for our package
manager. If we try to keep the name "LINGUAS", we may run into some
other problem down the line.


> 1. We start explicitly listing linguas_* in all ebuilds, no matter how
> tiny they are. Maintainers are required to keep IUSE up-to-date
> and users are forced to rebuild a lot. This is also a QA violation
> in terms of invalid use of USE flags.

This isn't as bad as you make it sound... the rebuilds would happen once
on a revision bump. The QA violation (I'm guessing) is that USE flags
shouldn't be used to control the installation of small text files.

I prefer a looser interpretation of that rule: maintainers don't have to
waste their time and complicate their ebuilds with USE flags to control
the installation of small text files if they don't want to. Basically an
"it's OK to install systemd unit files unconditionally" rule. Or in
other words, the rule is "it's OK not to do it" rather than "it's not OK
to do it."

I think if someone /wants/ to have a bunch of logic controlling the
installation of localization files, that's fine. But, this option would
force everyone to do it in order to work around an unfortunate choice of
variable name. It also adds some eternal mental overhead in that we have
to collectively remember that LINGUAS is special somehow and teach that
to everyone.

For those reasons I think #3 is a better long-term solution.

Re: [gentoo-dev] [RFC] How to deal with LINGUAS mess?

[waltdnes]

On Sat, May 21, 2016 at 09:41:28AM +0200, Micha?? Górny wrote

> I see the following possibilities:
> 
> 1. We start explicitly listing linguas_* in all ebuilds, no matter how
> tiny they are. Maintainers are required to keep IUSE up-to-date
> and users are forced to rebuild a lot. This is also a QA violation
> in terms of invalid use of USE flags.
> 
> 2. We hack-unset LINGUAS in ebuilds. This is a lot of effort, easy to
> miss and probably would need to repeated for every single phase anyway
> due to how global variables are handled in PMS. Additionally, it may
> break at some point since those variables are likely expected to be
> read-only anyway.
> 
> 3. We remove LINGUAS from USE_EXPAND and stop using it. If ebuilds have
> a good reason to use flags for localization, we introduce a new,
> non-colliding USE_EXPAND for that. We also ask users to replace LINGUAS
> with the new flag in their make.conf files. LINGUAS gets the original
> upstream behavior back, and we eventually discourage it in favor of new
> INSTALL_MASK features (WiP) [2].
> 
> 4. We fix build systems not to do magic depending on whether LINGUAS
> is unset or set-to-empty. Instead, we could some special special value
> like '-' to signify not installing localizations at all. But that's
> upstream thing to do, and breaks backwards compatibility with existing
> systems disabling localizations.
> 
> 
> Your thoughts?

5. An reversed variant of INSTALL_MASK in make.conf, e.g.
LOCALE_ALLOW="foo bar fubar"

which would block installing files in /usr/share/locale/* and
/usr/share/man/* EXCEPT for...

/usr/share/locale/foo
/usr/share/locale/bar
/usr/share/locale/fubar
/usr/share/man/foo
/usr/share/man/bar
/usr/share/man/fubar

6. Integrate localepurge into Portage, and run it post install

  There are some lazy programmers out there who *DO NOT* respect the
LINGUAS setting, and splatter files throughout /usr/share/locale/* and
/usr/share/man/* regardless.  That's the reason "localepurge" was
written in the first place.  Any proposed solution should take that
problem into consideration, and handle it too.

-- 
Walter Dnes <waltdnes@waltdnes.org>
I don't run "desktop environments"; I run useful applications

Re: [gentoo-dev] Re: [RFC] How to deal with LINGUAS mess?

[Michał Górny]

[-- Attachment #1: Type: text/plain, Size: 1171 bytes --]

On Sat, 21 May 2016 11:00:08 +0200
Ulrich Mueller <ulm@gentoo.org> wrote:

> >>>>> On Sat, 21 May 2016, Michał Górny wrote:  
> 
> > I see the following possibilities:  
> 
> > 1. We start explicitly listing linguas_* in all ebuilds, no matter
> > how tiny they are. Maintainers are required to keep IUSE up-to-date
> > and users are forced to rebuild a lot.  
> 
> Why would users have to rebuild more often? Language support in a
> package will change with a version bump, when they must rebuild in any
> case.

Except whenever:

1. developer fails to update lingua list, and needs to do so afterwards,

2. user changes his preference (i.e. I just built my system and figured
out I should really strip those damn locales),

3. user attempts to use binary packages (unless someone goes for
providing 2^n binary package variants).

> 
> > This is also a QA violation in terms of invalid use of USE flags.  
> 
> I fail to see why this would be a QA violation.

It falls into the rule for controlling installation of small files.
Most of localizations are < 20 KiB.

-- 
Best regards,
Michał Górny
<http://dev.gentoo.org/~mgorny/>

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 949 bytes --]

Re: [gentoo-dev] [RFC] How to deal with LINGUAS mess?

[Mart Raudsepp]

Ühel kenal päeval, L, 21.05.2016 kell 11:19, kirjutas
waltdnes@waltdnes.org:
> On Sat, May 21, 2016 at 09:41:28AM +0200, Micha?? Górny wrote
> > 3. We remove LINGUAS from USE_EXPAND and stop using it. If ebuilds
> > have
> > a good reason to use flags for localization, we introduce a new,
> > non-colliding USE_EXPAND for that. We also ask users to replace
> > LINGUAS
> > with the new flag in their make.conf files. LINGUAS gets the
> > original
> > upstream behavior back, and we eventually discourage it in favor of
> > new
> > INSTALL_MASK features (WiP) [2].

Short of making an exception to LINGUAS in the USE_EXPAND rules, I
think this is the only way.

> 5. An reversed variant of INSTALL_MASK in make.conf, e.g.
> LOCALE_ALLOW="foo bar fubar"
> 
> 6. Integrate localepurge into Portage, and run it post install
> 
>   There are some lazy programmers out there who *DO NOT* respect the
> LINGUAS setting, and splatter files throughout /usr/share/locale/*
> and
> /usr/share/man/* regardless.  That's the reason "localepurge" was
> written in the first place.  Any proposed solution should take that
> problem into consideration, and handle it too.

For both of these cases, I have to point out that e.g
LINGUAS="en et pl"
does not mean "keep only /usr/share/locale/{en,et,pl}/*", it means have
support for only these languages. This means for example that *.desktop
files will have translations in them only for those language. Same for
dconf schema files. Same for many other things. MO Translation files
and manuals aren't the only thing here, especially with intltool (and
many of these intltool features are now part of gettext proper).
In short, LINGUAS affects the content of files too, not only the
existence of files. See any file in /usr/share/applications/ for
example.
I always put "en" as my first in LINGUAS, due to historical abuse of
the first value there, I think e.g mplayer or vlc used to do this.
LINGUAS is an unordered list, but some packages used to took the first
value as meaning the default and switch the UI to that by default,
instead of honoring LC_* variables. Another reason I put "en" there, is
because of IUSE_EXPAND and packages that might not be natively english,
but do have english translations (I think I've seen some french ones
like that :D)


And no, localepurge is not capable of stripping these translations out
of existing files. To my knowledge at least. Though it could be
improved to do so in some cases.


Mart

Re: [gentoo-dev] [RFC] How to deal with LINGUAS mess?

[Michał Górny]

[-- Attachment #1: Type: text/plain, Size: 4491 bytes --]

On Sat, 21 May 2016 20:14:46 +1200
Kent Fredric <kentfredric@gmail.com> wrote:

> On 21 May 2016 at 19:41, Michał Górny <mgorny@gentoo.org> wrote:
> > Hello,
> >
> >
> > Those of you who read my blog post on LINGUAS [1] may already know
> > what's going on. For those who didn't, short summary:
> >
> > In EAPI 5 and newer, all variables listed in USE_EXPAND are supposed to
> > be unconditionally exported with their values reduced to enabled USE
> > flags listed in IUSE. In particular, this means that if ebuild does not
> > list any linguas_* flags in IUSE, PM exports *empty* LINGUAS (i.e.
> > disables all localizations with the implicit gettext behavior).
> >
> > Portage had so far some ugly hack-logic that tried to keep LINGUAS
> > working somehow in place. However, the patches to enable PMS-compliant
> > behavior are on their way, so it's about time to decide what to do
> > about LINGUAS.
> >
> >
> > I see the following possibilities:
> >
> > 1. We start explicitly listing linguas_* in all ebuilds, no matter how
> > tiny they are. Maintainers are required to keep IUSE up-to-date
> > and users are forced to rebuild a lot. This is also a QA violation
> > in terms of invalid use of USE flags.
> >
> > 2. We hack-unset LINGUAS in ebuilds. This is a lot of effort, easy to
> > miss and probably would need to repeated for every single phase anyway
> > due to how global variables are handled in PMS. Additionally, it may
> > break at some point since those variables are likely expected to be
> > read-only anyway.
> >
> > 3. We remove LINGUAS from USE_EXPAND and stop using it. If ebuilds have
> > a good reason to use flags for localization, we introduce a new,
> > non-colliding USE_EXPAND for that. We also ask users to replace LINGUAS
> > with the new flag in their make.conf files. LINGUAS gets the original
> > upstream behavior back, and we eventually discourage it in favor of new
> > INSTALL_MASK features (WiP) [2].
> >
> > 4. We fix build systems not to do magic depending on whether LINGUAS
> > is unset or set-to-empty. Instead, we could some special special value
> > like '-' to signify not installing localizations at all. But that's
> > upstream thing to do, and breaks backwards compatibility with existing
> > systems disabling localizations.
> >
> >
> > Your thoughts?  
> 
> I think its a regretful situation we find ourselves in where
> portage/PMS control values that have to go via ENV, leak beyond ENV
> into packaging.
> 
> This has already been a bit of a worry for us, because we may have
> wanted to use PERL_ as a prefix for env vars in an eclass, but there's
> a possibility of accidentally picking the same ENV var that is
> inherently used in some perl part.
> ( And at very least, `perl -V` shows all ENV vars that are prefixed
> with PERL_ .... eg: PERL_MOM_YOURS=1 perl -V )
> 
> Though I just don't see how we can generically avoid that scope
> leakage, because this is bash, and we're short on private ways to pass
> control values around, other than going crazy on functions.

Yes, that's one side of the problem. The other is that this was done
like this intentionally (i.e. to have USE flags implicitly converted
into LINGUAS). Just someone didn't have enough foresight where this
would lead.

> The only thing I thought of when reading the writeup was tagged
> install mask verbs.
> 
> That is:
> 
> - Always install all language support possible by default.

This still leaves the problem: how to enforce this? As long as LINGUAS
leaks into ebuild env (either from USE_EXPAND magic, from make.conf or
from parent env), we're in trouble.

> - Have an ebuild mapping of some description that annotates "files X
> add support for languages ( y, z, x )"
> 
> And then maybe you'd have a parameterized INSTALL_MASK system where you could do
> 
> INSTALL_MASK_RULES="linguas(-* en)"
> 
> Then it would be "Packager decides" if they want to trip a rebuild at
> the end user when they "discovered" certain files pertain to specific
> language support.
> 
> Then the only reason you'd need USE flags is if you wanted to toggle
> compile time support that pulled in dependencies to support the
> language in question.

Yes, exactly. Though my plan was for it to be more like:

  INSTALL_MASK="-@linguas @linguas_pl"

or alike. Though it's all implementation detail.

-- 
Best regards,
Michał Górny
<http://dev.gentoo.org/~mgorny/>

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 949 bytes --]

Re: [gentoo-dev] [RFC] How to deal with LINGUAS mess?

[Michał Górny]

[-- Attachment #1: Type: text/plain, Size: 2167 bytes --]

On Sat, 21 May 2016 11:19:07 -0400
waltdnes@waltdnes.org wrote:

> On Sat, May 21, 2016 at 09:41:28AM +0200, Micha?? Górny wrote
> 
> > I see the following possibilities:
> > 
> > 1. We start explicitly listing linguas_* in all ebuilds, no matter how
> > tiny they are. Maintainers are required to keep IUSE up-to-date
> > and users are forced to rebuild a lot. This is also a QA violation
> > in terms of invalid use of USE flags.
> > 
> > 2. We hack-unset LINGUAS in ebuilds. This is a lot of effort, easy to
> > miss and probably would need to repeated for every single phase anyway
> > due to how global variables are handled in PMS. Additionally, it may
> > break at some point since those variables are likely expected to be
> > read-only anyway.
> > 
> > 3. We remove LINGUAS from USE_EXPAND and stop using it. If ebuilds have
> > a good reason to use flags for localization, we introduce a new,
> > non-colliding USE_EXPAND for that. We also ask users to replace LINGUAS
> > with the new flag in their make.conf files. LINGUAS gets the original
> > upstream behavior back, and we eventually discourage it in favor of new
> > INSTALL_MASK features (WiP) [2].
> > 
> > 4. We fix build systems not to do magic depending on whether LINGUAS
> > is unset or set-to-empty. Instead, we could some special special value
> > like '-' to signify not installing localizations at all. But that's
> > upstream thing to do, and breaks backwards compatibility with existing
> > systems disabling localizations.
> > 
> > 
> > Your thoughts?  
> 
> 5. An reversed variant of INSTALL_MASK in make.conf, e.g.
> LOCALE_ALLOW="foo bar fubar"
> 
> which would block installing files in /usr/share/locale/* and
> /usr/share/man/* EXCEPT for...
> 
> /usr/share/locale/foo
> /usr/share/locale/bar
> /usr/share/locale/fubar
> /usr/share/man/foo
> /usr/share/man/bar
> /usr/share/man/fubar

This can be accomplished using inclusion/exclusion logic included in
the patches I've recently sent for Portage.

  INSTALL_MASK="/usr/share/locale -/usr/share/locale/foo"

-- 
Best regards,
Michał Górny
<http://dev.gentoo.org/~mgorny/>

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 949 bytes --]

Re: [gentoo-dev] [RFC] How to deal with LINGUAS mess?

[Michał Górny]

[-- Attachment #1: Type: text/plain, Size: 3329 bytes --]

On Fri, 27 May 2016 10:17:20 +0300
Mart Raudsepp <leio@gentoo.org> wrote:

> Ühel kenal päeval, L, 21.05.2016 kell 11:19, kirjutas
> waltdnes@waltdnes.org:
> > On Sat, May 21, 2016 at 09:41:28AM +0200, Micha?? Górny wrote  
> > > 3. We remove LINGUAS from USE_EXPAND and stop using it. If ebuilds
> > > have
> > > a good reason to use flags for localization, we introduce a new,
> > > non-colliding USE_EXPAND for that. We also ask users to replace
> > > LINGUAS
> > > with the new flag in their make.conf files. LINGUAS gets the
> > > original
> > > upstream behavior back, and we eventually discourage it in favor of
> > > new
> > > INSTALL_MASK features (WiP) [2].  
> 
> Short of making an exception to LINGUAS in the USE_EXPAND rules, I
> think this is the only way.
> 
> > 5. An reversed variant of INSTALL_MASK in make.conf, e.g.
> > LOCALE_ALLOW="foo bar fubar"
> > 
> > 6. Integrate localepurge into Portage, and run it post install
> > 
> >   There are some lazy programmers out there who *DO NOT* respect the
> > LINGUAS setting, and splatter files throughout /usr/share/locale/*
> > and
> > /usr/share/man/* regardless.  That's the reason "localepurge" was
> > written in the first place.  Any proposed solution should take that
> > problem into consideration, and handle it too.  
> 
> For both of these cases, I have to point out that e.g
> LINGUAS="en et pl"
> does not mean "keep only /usr/share/locale/{en,et,pl}/*", it means have
> support for only these languages. This means for example that *.desktop
> files will have translations in them only for those language. Same for
> dconf schema files. Same for many other things. MO Translation files
> and manuals aren't the only thing here, especially with intltool (and
> many of these intltool features are now part of gettext proper).
> In short, LINGUAS affects the content of files too, not only the
> existence of files. See any file in /usr/share/applications/ for
> example.

Just to be clear, I don't think we care that much about filtering
those. I can understand people not wanting to install a dozen
translation files because of their potential size, especially when
filesystem can't handle small files efficiently. But stripping extra
descriptions from .desktop files doesn't seem like a major use case.

> I always put "en" as my first in LINGUAS, due to historical abuse of
> the first value there, I think e.g mplayer or vlc used to do this.
> LINGUAS is an unordered list, but some packages used to took the first
> value as meaning the default and switch the UI to that by default,
> instead of honoring LC_* variables.

That's yet another reason not to extend the abuse of LINGUAS. I don't
really see adding another rule to PMS that attempts to enforce this.

> Another reason I put "en" there, is
> because of IUSE_EXPAND and packages that might not be natively english,
> but do have english translations (I think I've seen some french ones
> like that :D)
> 
> 
> And no, localepurge is not capable of stripping these translations out
> of existing files. To my knowledge at least. Though it could be
> improved to do so in some cases.

Well, this is certainly something that can be done.

-- 
Best regards,
Michał Górny
<http://dev.gentoo.org/~mgorny/>

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 949 bytes --]

Re: [gentoo-dev] [RFC] How to deal with LINGUAS mess?

[waltdnes]

On Sun, May 29, 2016 at 02:58:03PM +0200, Micha?? Górny wrote
> On Sat, 21 May 2016 11:19:07 -0400
> waltdnes@waltdnes.org wrote:
> 
> > 5. An reversed variant of INSTALL_MASK in make.conf, e.g.
> > LOCALE_ALLOW="foo bar fubar"
> > 
> > which would block installing files in /usr/share/locale/* and
> > /usr/share/man/* EXCEPT for...
> > 
> > /usr/share/locale/foo
> > /usr/share/locale/bar
> > /usr/share/locale/fubar
> > /usr/share/man/foo
> > /usr/share/man/bar
> > /usr/share/man/fubar
> 
> This can be accomplished using inclusion/exclusion logic included in
> the patches I've recently sent for Portage.
> 
>   INSTALL_MASK="/usr/share/locale -/usr/share/locale/foo"

  Thanks.  localepurge cleans up after the fact.  Your patches would
prevent the problem in the first place, which is a better approach.  It
also handles systemd files and can be used when some new stuff comes up
in future that we haven't thought of yet.

-- 
Walter Dnes <waltdnes@waltdnes.org>
I don't run "desktop environments"; I run useful applications

Re: [gentoo-dev] [RFC] How to deal with LINGUAS mess?

[Andrew Savchenko]

[-- Attachment #1: Type: text/plain, Size: 3475 bytes --]

On Sun, 29 May 2016 14:58:03 +0200 Michał Górny wrote:
> On Sat, 21 May 2016 11:19:07 -0400
> waltdnes@waltdnes.org wrote:
> 
> > On Sat, May 21, 2016 at 09:41:28AM +0200, Micha?? Górny wrote
> > 
> > > I see the following possibilities:
> > > 
> > > 1. We start explicitly listing linguas_* in all ebuilds, no matter how
> > > tiny they are. Maintainers are required to keep IUSE up-to-date
> > > and users are forced to rebuild a lot. This is also a QA violation
> > > in terms of invalid use of USE flags.
> > > 
> > > 2. We hack-unset LINGUAS in ebuilds. This is a lot of effort, easy to
> > > miss and probably would need to repeated for every single phase anyway
> > > due to how global variables are handled in PMS. Additionally, it may
> > > break at some point since those variables are likely expected to be
> > > read-only anyway.
> > > 
> > > 3. We remove LINGUAS from USE_EXPAND and stop using it. If ebuilds have
> > > a good reason to use flags for localization, we introduce a new,
> > > non-colliding USE_EXPAND for that. We also ask users to replace LINGUAS
> > > with the new flag in their make.conf files. LINGUAS gets the original
> > > upstream behavior back, and we eventually discourage it in favor of new
> > > INSTALL_MASK features (WiP) [2].
> > > 
> > > 4. We fix build systems not to do magic depending on whether LINGUAS
> > > is unset or set-to-empty. Instead, we could some special special value
> > > like '-' to signify not installing localizations at all. But that's
> > > upstream thing to do, and breaks backwards compatibility with existing
> > > systems disabling localizations.
> > > 
> > > 
> > > Your thoughts?  
> > 
> > 5. An reversed variant of INSTALL_MASK in make.conf, e.g.
> > LOCALE_ALLOW="foo bar fubar"
> > 
> > which would block installing files in /usr/share/locale/* and
> > /usr/share/man/* EXCEPT for...
> > 
> > /usr/share/locale/foo
> > /usr/share/locale/bar
> > /usr/share/locale/fubar
> > /usr/share/man/foo
> > /usr/share/man/bar
> > /usr/share/man/fubar
> 
> This can be accomplished using inclusion/exclusion logic included in
> the patches I've recently sent for Portage.
> 
>   INSTALL_MASK="/usr/share/locale -/usr/share/locale/foo"
 
A proper way will be to fix ebuilds to respect LINGUAS properly.
l10n.eclass makes this quite easy. Build and install files just to
remove them later before/after merge to a live system is ridiculous,
especially considering that LINGUAS covers not only manuals, but
also html docs and other files.

Why users should care about additional INSTALL_MASK ordeal if they
already set up LINGUAS properly? This is PM and ebuild maintainers
job.

The same states for cron, systemd, logrotate and other potentially
unneeded files. Install mask is dangerous, very dangerous, because
locations may move, mandatory files may have accidentally removed
by an oversight. INSTALL_MASK is indeed useful, essential tool and
it is nice that it is being extended, but it should be a last
resort way to fix stuff, not a recommended technique.

The real problem is that "small files" control creates much burden
for package maintainers. It should be fixed by versatile and easy
to use eclasses, ideally the most popular helpers should go to PMS,
e.g. to default src_install().

Moving all burden of "small files" control and optional cleanup
from developers to users is not nice.

Best regards,
Andrew Savchenko

[-- Attachment #2: Type: application/pgp-signature, Size: 819 bytes --]