[gentoo-devhelp] LINGUAS vs LANGUAGES

[gentoo-devhelp] LINGUAS vs LANGUAGES

[Nikos Chantziaras]

Reading the Gentoo Development Guide doesn't offer much help on LINGUAS:

   http://devmanual.gentoo.org/general-concepts/linguas/index.html

It says to look at games-rpg/nwn, but that ones seems to use LANGUAGES 
instead of LINGUAS.  My ebuild is EAPI=2 and the application in question 
supports English and German.  What is correct, LINGUAS="en de" or 
LANGUAGES="en de"?

Re: [gentoo-devhelp] LINGUAS vs LANGUAGES

[Nikos Chantziaras]

(It would be nice to reply on-list so the list's archives will be 
helpful to others searching through it.)

Daniel Pielmeier wrote:
> 2009/5/15 Nikos Chantziaras <realnc@arcor.de>:
>> Reading the Gentoo Development Guide doesn't offer much help on LINGUAS:
>>
>>  http://devmanual.gentoo.org/general-concepts/linguas/index.html
>>
>> It says to look at games-rpg/nwn, but that ones seems to use LANGUAGES
>> instead of LINGUAS.  My ebuild is EAPI=2 and the application in question
>> supports English and German.  What is correct, LINGUAS="en de" or
>> LANGUAGES="en de"?
> 
> LINGUAS holds the languages you want to install from make.conf.
> LANGUAES or LANGS or any other variable name you choose in your ebuild
> should refer to the languages available in the application to be
> installed.

Then I guess LINGUAS is what I need, since a user not having "de" in 
make.conf wouldn't want a German translation of the application's GUI 
installed.


> How you treat localisation in your ebuild depends on how the
> application itself treats the installation of locales. Take a look at
> net-irc/quassel for an example of an application that also uses cmake
> as build system. Maybe there is a better way to achieve locales
> installation but this one works. For cmake you need to know if the
> application supports customized installation of locales by using
> LINGUAS. Take a look in CMakelists.txt in the source for hints.

The application uses qmake and it simply installs all languages (*.qm 
files) unconditionally during "make install" and loads the appropriate 
one according to the LANG env variable at runtime, falling back to 
English if one isn't available.  At this time, German ("de") is the only 
one supported though (the app's built-in being English).

The solution I arrived at in my ebuild is:

   IUSE="linguas_de"

and then in src_install() I don't use the "install" make target which 
would install the *.qm files unconditionally, but rather use the other 
targets "install" depends upon, omitting "install_i18n":

   # Install everything except documentation and i18n.
   emake INSTALL_ROOT="${D}" install_target install_charmaps \
   || die "make install failed"
   # Install i18n files.
   if use linguas_de; then
     insinto "${GAMES_DATADIR}/${PN}/i18n"
     doins "${PN}_de.qm" || die "doins ${PN}_de.qm failed"
   fi

It works.  I assume putting linguas_de directly in IUSE is correct?

Re: [gentoo-devhelp] LINGUAS vs LANGUAGES

[Daniel Pielmeier]

2009/5/15 Nikos Chantziaras <realnc@arcor.de>:
> (It would be nice to reply on-list so the list's archives will be helpful to
> others searching through it.)

Aaargh. I vote for setting the reply-to-ist header for all gentoo lists.

> The application uses qmake and it simply installs all languages (*.qm files)
> unconditionally during "make install" and loads the appropriate one
> according to the LANG env variable at runtime, falling back to English if
> one isn't available.  At this time, German ("de") is the only one supported
> though (the app's built-in being English).

Ah okay qmake. I somehow mixed this up with cmake.

> The solution I arrived at in my ebuild is:
>
>  IUSE="linguas_de"
>
> and then in src_install() I don't use the "install" make target which would
> install the *.qm files unconditionally, but rather use the other targets
> "install" depends upon, omitting "install_i18n":
>
>  # Install everything except documentation and i18n.
>  emake INSTALL_ROOT="${D}" install_target install_charmaps \
>  || die "make install failed"
>  # Install i18n files.
>  if use linguas_de; then
>    insinto "${GAMES_DATADIR}/${PN}/i18n"
>    doins "${PN}_de.qm" || die "doins ${PN}_de.qm failed"
>  fi

This should work but more convenient would be if there is some
configuration switch you can use to define the installed languages.

> It works.  I assume putting linguas_de directly in IUSE is correct?
>

I use this for appending LINGUAS to IUSE but only because I have seen
this many times in other ebuilds

IUSE="dbus debug kde monolithic +oxygen phonon postgres +server +ssl webkit +X"

LANGS="cs da de fr hu it nb_NO ru sl tr"
for l in ${LANGS}; do
	IUSE="${IUSE} linguas_${l}"
done

Putting it directly in IUSE will also work.

-- 
Daniel Pielmeier

Re: [gentoo-devhelp] LINGUAS vs LANGUAGES

[Daniel Pielmeier]

[-- Attachment #1: Type: text/plain, Size: 1078 bytes --]

Daniel Pielmeier schrieb am 15.05.2009 17:01:
> 2009/5/15 Nikos Chantziaras <realnc@arcor.de>:
>>
>>  # Install everything except documentation and i18n.
>>  emake INSTALL_ROOT="${D}" install_target install_charmaps \
>>  || die "make install failed"
>>  # Install i18n files.
>>  if use linguas_de; then
>>    insinto "${GAMES_DATADIR}/${PN}/i18n"
>>    doins "${PN}_de.qm" || die "doins ${PN}_de.qm failed"
>>  fi

You also might consider using a loop for installing the linguas else you
have to add the same almost identical block for every new language
introduced. This way the intermediate variable LANGS used for IUSE
injection comes handy.

# IUSE definition:

IUSE="flags"

LANGS="de en"
for i in ${LANGS}; do
	IUSE="${IUSE} linguas_${i}"
done

# in src_install:

local my_langs

for j in ${LINGUAS}; do
	if has ${j} ${LANGS}; then
		my_langs="${j} ${my_langs}"
	fi
done

insinto "${GAMES_DATADIR}/${PN}/i18n"
for k in ${my_langs}; do
	doins "${PN}_${k}.qm" || die "doins ${PN}_${k}.qm failed"
done

-- 
Daniel Pielmeier




[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 261 bytes --]

[gentoo-devhelp] Re: LINGUAS vs LANGUAGES

[Nikos Chantziaras]

Daniel Pielmeier wrote:
> [...]
> You also might consider using a loop for installing the linguas else you
> have to add the same almost identical block for every new language
> introduced. This way the intermediate variable LANGS used for IUSE
> injection comes handy.
> 
> # IUSE definition:
> 
> IUSE="flags"
> 
> LANGS="de en"
> for i in ${LANGS}; do
> 	IUSE="${IUSE} linguas_${i}"
> done
> 
> # in src_install:
> 
> local my_langs
> 
> for j in ${LINGUAS}; do
> 	if has ${j} ${LANGS}; then
> 		my_langs="${j} ${my_langs}"
> 	fi
> done
> 
> insinto "${GAMES_DATADIR}/${PN}/i18n"
> for k in ${my_langs}; do
> 	doins "${PN}_${k}.qm" || die "doins ${PN}_${k}.qm failed"
> done

Thanks.  I ended up doing it this way, though with only one loop in 
src_install(), which seems to be a bit more efficient and shorter:


LANGUAGES="de"
for i in ${LANGUAGES}; do
	IUSE="${IUSE} linguas_${i}"
done

src_install()
#...
insinto "${GAMES_DATADIR}/${PN}/i18n"
for i in ${LANGUAGES}; do
	if has ${i} ${LINGUAS}; then
		doins "${PN}_${i}.qm" || die #...
	fi
done


In this case, LANGUAGES must not include "en" though, since there's no 
${PN}_en.qm file (English is built-in).  I'm not sure if the following 
would be better though, in case a user puts linguas_<language> in USE 
instead of <language> in LINGUAS:

   if has linguas_${i} ${USE}; then

But I suppose it would be the user's fault if he/she does that? :P

[gentoo-devhelp] Re: LINGUAS vs LANGUAGES

[Steven J Long]

Nikos Chantziaras wrote:

> Thanks.  I ended up doing it this way, though with only one loop in
> src_install(), which seems to be a bit more efficient and shorter:
> 
> 
> LANGUAGES="de"
> for i in ${LANGUAGES}; do
> IUSE="${IUSE} linguas_${i}"
> done
>
Just on a side-note (not saying it's how you want to do this one), this is
something that BASH arrays are nice for (saving another loop):
$ foo=(bar baz quux)
$ echo "prefixed: '${foo[*]/#/pfx_}'"
prefixed: 'pfx_bar pfx_baz pfx_quux'

Parameter Expansion (incl on arrays) is discussed more at:
http://mywiki.wooledge.org/BashFAQ/073

> src_install()
> #...
> insinto "${GAMES_DATADIR}/${PN}/i18n"
> for i in ${LANGUAGES}; do
> if has ${i} ${LINGUAS}; then
> doins "${PN}_${i}.qm" || die #...
> fi
> done
> 
> 
> In this case, LANGUAGES must not include "en" though, since there's no
> ${PN}_en.qm file (English is built-in).  I'm not sure if the following
> would be better though, in case a user puts linguas_<language> in USE
> instead of <language> in LINGUAS:
> 
>    if has linguas_${i} ${USE}; then
> 
> But I suppose it would be the user's fault if he/she does that? :P

Well yeah it's not the right way to set USE_EXPAND vars; LINGUAS is, and
will be handled specially since it's one of that set.
-- 
#friendly-coders -- We're friendly but we're not /that/ friendly ;-)

Re: [gentoo-devhelp] Re: LINGUAS vs LANGUAGES

[Mike Frysinger]

[-- Attachment #1: Type: text/plain, Size: 845 bytes --]

On Tuesday 21 July 2009 06:13:25 Steven J Long wrote:
> Nikos Chantziaras wrote:
> > Thanks.  I ended up doing it this way, though with only one loop in
> > src_install(), which seems to be a bit more efficient and shorter:
> >
> >
> > LANGUAGES="de"
> > for i in ${LANGUAGES}; do
> > IUSE="${IUSE} linguas_${i}"
> > done
>
> Just on a side-note (not saying it's how you want to do this one), this is
> something that BASH arrays are nice for (saving another loop):
> $ foo=(bar baz quux)
> $ echo "prefixed: '${foo[*]/#/pfx_}'"
> prefixed: 'pfx_bar pfx_baz pfx_quux'

printf would probably be better as it is typically a shell builtin and it 
doesnt require use of arrays/uncommon syntax.
media-gfx/exiv2/exiv2-0.18.ebuild:
IUSE_LINGUAS="de es fi fr pl ru sk"
IUSE="${IUSE} $(printf 'linguas_%s ' ${IUSE_LINGUAS})"
-mike

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 836 bytes --]

[gentoo-devhelp] Re: Re: LINGUAS vs LANGUAGES

[Steven J Long]

Mike Frysinger wrote:

> On Tuesday 21 July 2009 06:13:25 Steven J Long wrote:
>> Nikos Chantziaras wrote:
>> > Thanks.  I ended up doing it this way, though with only one loop in
>> > src_install(), which seems to be a bit more efficient and shorter:
>> >
>> >
>> > LANGUAGES="de"
>> > for i in ${LANGUAGES}; do
>> > IUSE="${IUSE} linguas_${i}"
>> > done
>>
>> Just on a side-note (not saying it's how you want to do this one), this
>> is something that BASH arrays are nice for (saving another loop):
>> $ foo=(bar baz quux)
>> $ echo "prefixed: '${foo[*]/#/pfx_}'"
>> prefixed: 'pfx_bar pfx_baz pfx_quux'
> 
> printf would probably be better as it is typically a shell builtin and it
> doesnt require use of arrays/uncommon syntax.
> media-gfx/exiv2/exiv2-0.18.ebuild:
> IUSE_LINGUAS="de es fi fr pl ru sk"
> IUSE="${IUSE} $(printf 'linguas_%s ' ${IUSE_LINGUAS})"

Doh, forgot about printf. (We have alias print="printf '%s\n'" in our lib
code which comes in handy too.) Nice one.

The only issue with the above is that it requires a subshell; forking isn't
cheap (especially on Interix/cygwin/doze) and in general it's considered a
bit lame (at least amongst the ##bash old-timers that I bump heads with) to
need forking in BASH, though ofc not in SH, which is why it might not be
the best here, since the metadata generation phase is a restricted subset
of SH, leave alone BASH, at least aiui.

An example of BASH saving forking, would be where printf -v comes into play.
A more portable, though less flexible command in terms of allowed
characters is read from <<< [1]. Though that can be worked round, it's nice
not to have to; a shell that is progressing while still retaining excellent
portability is useful imo (so long as we don't get our knickers in a twist
about using new features, since we can set a required EAPI per-profile in
any case.)

[1] @freenode: /msg greybot <<<

Yes, I'm aware there are two versions of portability being discussed there.
I figured our readership can differentiate, and if not you can always
lead 'em gently by the hand to Massachusetts.. ;P
-- 
#friendly-coders -- We're friendly but we're not /that/ friendly ;-)

[gentoo-devhelp] Re: Re: LINGUAS vs LANGUAGES

[Steven J Long]

Steven J Long wrote:

> since the metadata generation phase is a
> restricted subset of SH, leave alone BASH, at least aiui.
> 
Er that's wrong; that applies to make.conf not ebuilds; I recall
hearing "full BASH is required for metadata generation" several times
on -dev.

Given that, saving a fork during cache building (or indeed at any time) is A
Good Idea™, even when we're not on crippleDOS.

-- 
#friendly-coders -- We're friendly but we're not /that/ friendly ;-)

Re: [gentoo-devhelp] Re: Re: LINGUAS vs LANGUAGES

[Mike Frysinger]

[-- Attachment #1: Type: text/plain, Size: 1793 bytes --]

On Saturday 01 August 2009 11:22:41 Steven J Long wrote:
> Mike Frysinger wrote:
> > On Tuesday 21 July 2009 06:13:25 Steven J Long wrote:
> >> Nikos Chantziaras wrote:
> >> > Thanks.  I ended up doing it this way, though with only one loop in
> >> > src_install(), which seems to be a bit more efficient and shorter:
> >> >
> >> >
> >> > LANGUAGES="de"
> >> > for i in ${LANGUAGES}; do
> >> > IUSE="${IUSE} linguas_${i}"
> >> > done
> >>
> >> Just on a side-note (not saying it's how you want to do this one), this
> >> is something that BASH arrays are nice for (saving another loop):
> >> $ foo=(bar baz quux)
> >> $ echo "prefixed: '${foo[*]/#/pfx_}'"
> >> prefixed: 'pfx_bar pfx_baz pfx_quux'
> >
> > printf would probably be better as it is typically a shell builtin and it
> > doesnt require use of arrays/uncommon syntax.
> > media-gfx/exiv2/exiv2-0.18.ebuild:
> > IUSE_LINGUAS="de es fi fr pl ru sk"
> > IUSE="${IUSE} $(printf 'linguas_%s ' ${IUSE_LINGUAS})"
>
> Doh, forgot about printf. (We have alias print="printf '%s\n'" in our lib
> code which comes in handy too.) Nice one.
>
> The only issue with the above is that it requires a subshell; forking isn't
> cheap (especially on Interix/cygwin/doze) and in general it's considered a
> bit lame (at least amongst the ##bash old-timers that I bump heads with) to
> need forking in BASH, though ofc not in SH, which is why it might not be
> the best here, since the metadata generation phase is a restricted subset
> of SH, leave alone BASH, at least aiui.

forks are cheap on any sane *nix box, and subshells/builtins are better than 
external programs.  i'm pretty sure the overhead here tends to be less than 
the overhead of running for loops, especially as the LINGUAS gets bigger.
-mike

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 836 bytes --]

[gentoo-devhelp] Re: Re: Re: LINGUAS vs LANGUAGES

[Steven J Long]

Mike Frysinger wrote:

> On Saturday 01 August 2009 11:22:41 Steven J Long wrote:
>> Mike Frysinger wrote:
>> > On Tuesday 21 July 2009 06:13:25 Steven J Long wrote:
>> >> Nikos Chantziaras wrote:
>> >> > Thanks.  I ended up doing it this way, though with only one loop in
>> >> > src_install(), which seems to be a bit more efficient and shorter:
>> >> >
>> >> >
>> >> > LANGUAGES="de"
>> >> > for i in ${LANGUAGES}; do
>> >> > IUSE="${IUSE} linguas_${i}"
>> >> > done
>> >>
Oh, don't forget to: unset LANGUAGES # unless you need to reexamine it in
later phases.

>> >> Just on a side-note (not saying it's how you want to do this one),
>> >> this is something that BASH arrays are nice for (saving another loop):
>> >> $ foo=(bar baz quux)
>> >> $ echo "prefixed: '${foo[*]/#/pfx_}'"
>> >> prefixed: 'pfx_bar pfx_baz pfx_quux'
>> >
>> > printf would probably be better as it is typically a shell builtin and
>> > it doesnt require use of arrays/uncommon syntax.
>> > media-gfx/exiv2/exiv2-0.18.ebuild:
>> > IUSE_LINGUAS="de es fi fr pl ru sk"
>> > IUSE="${IUSE} $(printf 'linguas_%s ' ${IUSE_LINGUAS})"
>>
>> Doh, forgot about printf. (We have alias print="printf '%s\n'" in our lib
>> code which comes in handy too.) Nice one.
>>
>> The only issue with the above is that it requires a subshell; forking
>> isn't cheap (especially on Interix/cygwin/doze) and in general it's
>> considered a bit lame (at least amongst the ##bash old-timers that I bump
>> heads with) to need forking in BASH, though ofc not in SH, which is why
>> it might not be the best here, since the metadata generation phase is a
>> restricted subset of SH, leave alone BASH, at least aiui.
> 
> forks are cheap on any sane *nix box
They certainly are in comparison to doze; however they are still a load of
work, even with copy-on-write. I've done loads of BASH, and quite large
scripts; believe me that eliminating subshells makes things quicker. 
(Don't forget BASH is more of a pig than most sh.) In the context of cache
generation, we've heard lots about it being slow on -dev; eliminating
subshells, where appropriate, is one of those things that can make it
quicker without needing to bastardise the format.

> , and subshells/builtins are better than external programs.
That's true enough, since the main shell has to fork for any external, and a
fork plus builtin is better than fork + load an external. We're not using
an external at all here though, so I don't see how it applies.

> i'm pretty sure the overhead here tends to be less 
> than the overhead of running for loops, especially as the LINGUAS gets
> bigger.
Timings would settle it; a for does append to a string on each iteration.
Which is why parameter expansion on arrays is so nice: we don't even need a
loop and BASH can do it as part of the expansion it always has to do
anyhow. It does speed things up.

Only other change is: LANGUAGES=(en de fr) # or the like vs: LANGUAGES='en
fr de'. LANGUAGES="de" will work, but only for single values. Recently
there's been more usage of arrays in eclasses; I don't think expecting
people to learn the syntax for the language, over time, is that big an ask.
After all, it's certainly more transferable than learning the ins and outs
of an EAPI.

Having said that we allow the user to use either arrays or strings split on
spaces for configuration of update[1]. I didn't want to get into the
isArr() function here since it uses a subshell ;) I was against the use of
both, as I saw no reason a user couldn't use an array, and thus learn a bit
more BASH, but a forum-user talked me into it. For an end-user wrapper for
emerge, it seems fair enough. But for writing and maintaining ebuilds (the
definition of what a Gentoo developer is)? Expecting people to deal with an
array-configured value is useful for the eclasses they might come across
later, and means eclass authors can use them without worrying.

Regards,
Steve.

[1] http://forums.gentoo.org/viewtopic-t-546828.html (get git version if you
want to try it though.)
-- 
#friendly-coders -- We're friendly but we're not /that/ friendly ;-)

[gentoo-devhelp] Re: LINGUAS vs LANGUAGES

[Nikos Chantziaras]

On 07/21/2009 01:13 PM, Steven J Long wrote:
> Nikos Chantziaras wrote:
>
>> Thanks.  I ended up doing it this way, though with only one loop in
>> src_install(), which seems to be a bit more efficient and shorter:
>>
>>
>> LANGUAGES="de"
>> for i in ${LANGUAGES}; do
>> IUSE="${IUSE} linguas_${i}"
>> done
>>
> Just on a side-note (not saying it's how you want to do this one), this is
> something that BASH arrays are nice for (saving another loop):
> $ foo=(bar baz quux)
> $ echo "prefixed: '${foo[*]/#/pfx_}'"
> prefixed: 'pfx_bar pfx_baz pfx_quux'
>
> Parameter Expansion (incl on arrays) is discussed more at:
> http://mywiki.wooledge.org/BashFAQ/073

(Late reply because I was away on vacation.)

I ended up removing LINGUAS support altogether and installing the extra 
language file regardless of the user's LINGUAS.  The Gentoo developer 
maintaining the ebuild said that's how it should be done when the 
package only supports two languages.