[gentoo-dev] Understanding the LINGUAS variable and use-expand

[gentoo-dev] Understanding the LINGUAS variable and use-expand

[Mike Gilbert]

I just want to sanity-check my brain here.

LINGUAS seems to be mainly a variable to control the behavior of
gettext's autoconf code, installed as /usr/share/aclocal/po.m4.

If LINGUAS is set to a list of language codes, the build system will
only build/install MO files for those languages. If LINGUAS is unset,
the build system will build/install MO files for all available
languages.

Portage will also use-expand LINGUAS, so an ebuild *can* make use of
it where USE flags are needed. For example, in SRC_URI, where the USE
flags may be used to control downloading of extra language packs.

Given this information, I come to a few conclusions:

1. There is technically no need to define IUSE="linguas_$x" if an
ebuild is not using the USE flags in other ebuild metadata (like
SRC_URI).

2. In cases where the USE flags are needed, they should be enabled by
default to emulate the "default-all" behavior of the autoconf macros.
For example: IUSE="+linguas_fr_FR +linguas_de_DE".

3. An ebuild could use LINGUAS to control installation of translation
information which does not come from gettext PO files. It does not
necessarily need to make use of the linguas_$x USE flags to do so.

Does all of that make sense?

I am considering simplifying www-client/chromium from the current mess
based on the linguas USE flags to basically just this:

if [[ ${LINGUAS} ]]; then
  for x in *.pak; do
    mylang=${x%.pak}
    mylang=%{x/-/_}
    has $mylang $LINGUAS || rm $x
  done
fi

As well, there are probably a few other places in the tree where
conclusion #1 and #2 should be applied.

Re: [gentoo-dev] Understanding the LINGUAS variable and use-expand

[Mart Raudsepp]

On K, 2012-02-08 at 11:32 -0500, Mike Gilbert wrote:
> I just want to sanity-check my brain here.
> 
> LINGUAS seems to be mainly a variable to control the behavior of
> gettext's autoconf code, installed as /usr/share/aclocal/po.m4.
> 
> If LINGUAS is set to a list of language codes, the build system will
> only build/install MO files for those languages. If LINGUAS is unset,
> the build system will build/install MO files for all available
> languages.
> 
> Portage will also use-expand LINGUAS, so an ebuild *can* make use of
> it where USE flags are needed. For example, in SRC_URI, where the USE
> flags may be used to control downloading of extra language packs.
> 
> Given this information, I come to a few conclusions:
> 
> 1. There is technically no need to define IUSE="linguas_$x" if an
> ebuild is not using the USE flags in other ebuild metadata (like
> SRC_URI).

I'm not sure, but it is not feasible to list them for exposing languages
a package has been translated to via gettext/intltool. That's completely
unmaintainable and unnecessary. Each version bump could change it, hard
and time consuming to validate, etc.

> 2. In cases where the USE flags are needed, they should be enabled by
> default to emulate the "default-all" behavior of the autoconf macros.
> For example: IUSE="+linguas_fr_FR +linguas_de_DE".

I would like that, but I don't think any or many packages do so when
they use it in SRC_URI

> 3. An ebuild could use LINGUAS to control installation of translation
> information which does not come from gettext PO files. It does not
> necessarily need to make use of the linguas_$x USE flags to do so.

One such widely used way is to use intltool, which amongst other things
handles PO files as well (of course via gettext tools in the end), but
also other things.

> Does all of that make sense?
> 
> I am considering simplifying www-client/chromium from the current mess
> based on the linguas USE flags to basically just this:
> 
> if [[ ${LINGUAS} ]]; then
>   for x in *.pak; do
>     mylang=${x%.pak}
>     mylang=%{x/-/_}
>     has $mylang $LINGUAS || rm $x
>   done
> fi

It would perhaps be nicer if upstream honored LINGUAS itself too or
so...

I think users could be surprised a bit about cases where you have
variants or dialects, e.g currently as IUSE=linguas_fr_FR, when users
only have LINGUAS=fr - in gettext they often don't have a fr_FR.po, just
fr.po; if locale has LC_MESSAGE and LANG at fr_FR, it will look at "fr"
if more specific fr_FR not found.
I define things like LINGUAS="en en_US et et_EE" to be really sure, but
I doubt many users do that (but that's just a guess).
If it's exposed via IUSE, then they can at least have some visual cue of
that.
I guess it wouldn't be a concern if we had a tool to set the LINGUAS
that handled this variant logic nicely, or just educating users in
documentation, make.conf.example comments and so on.

> As well, there are probably a few other places in the tree where
> conclusion #1 and #2 should be applied.


Best,
Mart Raudsepp

Re: [gentoo-dev] Understanding the LINGUAS variable and use-expand

[Mike Gilbert]

On Wed, Feb 8, 2012 at 12:04 PM, Mart Raudsepp <leio@gentoo.org> wrote:
> On K, 2012-02-08 at 11:32 -0500, Mike Gilbert wrote:
>> I am considering simplifying www-client/chromium from the current mess
>> based on the linguas USE flags to basically just this:
>>
>> if [[ ${LINGUAS} ]]; then
>>   for x in *.pak; do
>>     mylang=${x%.pak}
>>     mylang=%{x/-/_}
>>     has $mylang $LINGUAS || rm $x
>>   done
>> fi

> I think users could be surprised a bit about cases where you have
> variants or dialects, e.g currently as IUSE=linguas_fr_FR, when users
> only have LINGUAS=fr - in gettext they often don't have a fr_FR.po, just
> fr.po; if locale has LC_MESSAGE and LANG at fr_FR, it will look at "fr"
> if more specific fr_FR not found.
> I define things like LINGUAS="en en_US et et_EE" to be really sure, but
> I doubt many users do that (but that's just a guess).
> If it's exposed via IUSE, then they can at least have some visual cue of
> that.
> I guess it wouldn't be a concern if we had a tool to set the LINGUAS
> that handled this variant logic nicely, or just educating users in
> documentation, make.conf.example comments and so on.

Thanks for catching that Mart. I think I have addressed the dialect
issue by more directly emulating the autoconf macro. See below.

I would greatly appreciate any additional feedback anyone has on this subject.


New code, currently used (experimentally) in
google-chrome-19.0.1049.3_alpha123257.ebuild:

 if [[ "%UNSET%" != "${LINGUAS-%UNSET}" ]]; then
   local found desiredlang presentlang pak pakname

   pushd "${CHROME_HOME}locales" > /dev/null || die

   for pak in *.pak; do
     pakname="${pak%.pak}"
     pakname="${pakname/-/_}"
     presentlang="$(chromium_lang "${pakname}")"

     # Do not issue warning for en_US locale. This is the fallback
     # locale so it should always be installed.
     if [[ "${presentlang}" == "en_US" ]]; then
       continue
     fi

     found=
     for desiredlang in ${LINGUAS}; do
       if [[ "${desiredlang}" == "${presentlang}"* ]]; then
         found=1
         break
       fi
     done

     if [[ -z ${found} ]]; then
       rm "${pak}" || die
     fi
   done

   popd > /dev/null
 fi