* [gentoo-dev] [RFC] How to deal with LINGUAS mess? @ 2016-05-21 7:41 Michał Górny 2016-05-21 8:14 ` Kent Fredric ` (3 more replies) 0 siblings, 4 replies; 14+ messages in thread From: Michał Górny @ 2016-05-21 7:41 UTC (permalink / raw To: gentoo-dev; +Cc: qa [-- Attachment #1: Type: text/plain, Size: 2250 bytes --] Hello, Those of you who read my blog post on LINGUAS [1] may already know what's going on. For those who didn't, short summary: In EAPI 5 and newer, all variables listed in USE_EXPAND are supposed to be unconditionally exported with their values reduced to enabled USE flags listed in IUSE. In particular, this means that if ebuild does not list any linguas_* flags in IUSE, PM exports *empty* LINGUAS (i.e. disables all localizations with the implicit gettext behavior). Portage had so far some ugly hack-logic that tried to keep LINGUAS working somehow in place. However, the patches to enable PMS-compliant behavior are on their way, so it's about time to decide what to do about LINGUAS. I see the following possibilities: 1. We start explicitly listing linguas_* in all ebuilds, no matter how tiny they are. Maintainers are required to keep IUSE up-to-date and users are forced to rebuild a lot. This is also a QA violation in terms of invalid use of USE flags. 2. We hack-unset LINGUAS in ebuilds. This is a lot of effort, easy to miss and probably would need to repeated for every single phase anyway due to how global variables are handled in PMS. Additionally, it may break at some point since those variables are likely expected to be read-only anyway. 3. We remove LINGUAS from USE_EXPAND and stop using it. If ebuilds have a good reason to use flags for localization, we introduce a new, non-colliding USE_EXPAND for that. We also ask users to replace LINGUAS with the new flag in their make.conf files. LINGUAS gets the original upstream behavior back, and we eventually discourage it in favor of new INSTALL_MASK features (WiP) [2]. 4. We fix build systems not to do magic depending on whether LINGUAS is unset or set-to-empty. Instead, we could some special special value like '-' to signify not installing localizations at all. But that's upstream thing to do, and breaks backwards compatibility with existing systems disabling localizations. Your thoughts? [1]:https://blogs.gentoo.org/mgorny/2016/05/16/how-linguas-are-thrice-wrong/ [2]:https://wiki.gentoo.org/wiki/User:MGorny/GLEP:INSTALL_MASK -- Best regards, Michał Górny <http://dev.gentoo.org/~mgorny/> [-- Attachment #2: OpenPGP digital signature --] [-- Type: application/pgp-signature, Size: 949 bytes --] ^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: [gentoo-dev] [RFC] How to deal with LINGUAS mess? 2016-05-21 7:41 [gentoo-dev] [RFC] How to deal with LINGUAS mess? Michał Górny @ 2016-05-21 8:14 ` Kent Fredric 2016-05-29 12:30 ` Michał Górny 2016-05-21 9:00 ` [gentoo-dev] " Ulrich Mueller ` (2 subsequent siblings) 3 siblings, 1 reply; 14+ messages in thread From: Kent Fredric @ 2016-05-21 8:14 UTC (permalink / raw To: gentoo-dev; +Cc: qa On 21 May 2016 at 19:41, Michał Górny <mgorny@gentoo.org> wrote: > Hello, > > > Those of you who read my blog post on LINGUAS [1] may already know > what's going on. For those who didn't, short summary: > > In EAPI 5 and newer, all variables listed in USE_EXPAND are supposed to > be unconditionally exported with their values reduced to enabled USE > flags listed in IUSE. In particular, this means that if ebuild does not > list any linguas_* flags in IUSE, PM exports *empty* LINGUAS (i.e. > disables all localizations with the implicit gettext behavior). > > Portage had so far some ugly hack-logic that tried to keep LINGUAS > working somehow in place. However, the patches to enable PMS-compliant > behavior are on their way, so it's about time to decide what to do > about LINGUAS. > > > I see the following possibilities: > > 1. We start explicitly listing linguas_* in all ebuilds, no matter how > tiny they are. Maintainers are required to keep IUSE up-to-date > and users are forced to rebuild a lot. This is also a QA violation > in terms of invalid use of USE flags. > > 2. We hack-unset LINGUAS in ebuilds. This is a lot of effort, easy to > miss and probably would need to repeated for every single phase anyway > due to how global variables are handled in PMS. Additionally, it may > break at some point since those variables are likely expected to be > read-only anyway. > > 3. We remove LINGUAS from USE_EXPAND and stop using it. If ebuilds have > a good reason to use flags for localization, we introduce a new, > non-colliding USE_EXPAND for that. We also ask users to replace LINGUAS > with the new flag in their make.conf files. LINGUAS gets the original > upstream behavior back, and we eventually discourage it in favor of new > INSTALL_MASK features (WiP) [2]. > > 4. We fix build systems not to do magic depending on whether LINGUAS > is unset or set-to-empty. Instead, we could some special special value > like '-' to signify not installing localizations at all. But that's > upstream thing to do, and breaks backwards compatibility with existing > systems disabling localizations. > > > Your thoughts? I think its a regretful situation we find ourselves in where portage/PMS control values that have to go via ENV, leak beyond ENV into packaging. This has already been a bit of a worry for us, because we may have wanted to use PERL_ as a prefix for env vars in an eclass, but there's a possibility of accidentally picking the same ENV var that is inherently used in some perl part. ( And at very least, `perl -V` shows all ENV vars that are prefixed with PERL_ .... eg: PERL_MOM_YOURS=1 perl -V ) Though I just don't see how we can generically avoid that scope leakage, because this is bash, and we're short on private ways to pass control values around, other than going crazy on functions. The only thing I thought of when reading the writeup was tagged install mask verbs. That is: - Always install all language support possible by default. - Have an ebuild mapping of some description that annotates "files X add support for languages ( y, z, x )" And then maybe you'd have a parameterized INSTALL_MASK system where you could do INSTALL_MASK_RULES="linguas(-* en)" Then it would be "Packager decides" if they want to trip a rebuild at the end user when they "discovered" certain files pertain to specific language support. Then the only reason you'd need USE flags is if you wanted to toggle compile time support that pulled in dependencies to support the language in question. -- Kent KENTNL - https://metacpan.org/author/KENTNL ^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: [gentoo-dev] [RFC] How to deal with LINGUAS mess? 2016-05-21 8:14 ` Kent Fredric @ 2016-05-29 12:30 ` Michał Górny 0 siblings, 0 replies; 14+ messages in thread From: Michał Górny @ 2016-05-29 12:30 UTC (permalink / raw To: Kent Fredric; +Cc: gentoo-dev, qa [-- Attachment #1: Type: text/plain, Size: 4491 bytes --] On Sat, 21 May 2016 20:14:46 +1200 Kent Fredric <kentfredric@gmail.com> wrote: > On 21 May 2016 at 19:41, Michał Górny <mgorny@gentoo.org> wrote: > > Hello, > > > > > > Those of you who read my blog post on LINGUAS [1] may already know > > what's going on. For those who didn't, short summary: > > > > In EAPI 5 and newer, all variables listed in USE_EXPAND are supposed to > > be unconditionally exported with their values reduced to enabled USE > > flags listed in IUSE. In particular, this means that if ebuild does not > > list any linguas_* flags in IUSE, PM exports *empty* LINGUAS (i.e. > > disables all localizations with the implicit gettext behavior). > > > > Portage had so far some ugly hack-logic that tried to keep LINGUAS > > working somehow in place. However, the patches to enable PMS-compliant > > behavior are on their way, so it's about time to decide what to do > > about LINGUAS. > > > > > > I see the following possibilities: > > > > 1. We start explicitly listing linguas_* in all ebuilds, no matter how > > tiny they are. Maintainers are required to keep IUSE up-to-date > > and users are forced to rebuild a lot. This is also a QA violation > > in terms of invalid use of USE flags. > > > > 2. We hack-unset LINGUAS in ebuilds. This is a lot of effort, easy to > > miss and probably would need to repeated for every single phase anyway > > due to how global variables are handled in PMS. Additionally, it may > > break at some point since those variables are likely expected to be > > read-only anyway. > > > > 3. We remove LINGUAS from USE_EXPAND and stop using it. If ebuilds have > > a good reason to use flags for localization, we introduce a new, > > non-colliding USE_EXPAND for that. We also ask users to replace LINGUAS > > with the new flag in their make.conf files. LINGUAS gets the original > > upstream behavior back, and we eventually discourage it in favor of new > > INSTALL_MASK features (WiP) [2]. > > > > 4. We fix build systems not to do magic depending on whether LINGUAS > > is unset or set-to-empty. Instead, we could some special special value > > like '-' to signify not installing localizations at all. But that's > > upstream thing to do, and breaks backwards compatibility with existing > > systems disabling localizations. > > > > > > Your thoughts? > > I think its a regretful situation we find ourselves in where > portage/PMS control values that have to go via ENV, leak beyond ENV > into packaging. > > This has already been a bit of a worry for us, because we may have > wanted to use PERL_ as a prefix for env vars in an eclass, but there's > a possibility of accidentally picking the same ENV var that is > inherently used in some perl part. > ( And at very least, `perl -V` shows all ENV vars that are prefixed > with PERL_ .... eg: PERL_MOM_YOURS=1 perl -V ) > > Though I just don't see how we can generically avoid that scope > leakage, because this is bash, and we're short on private ways to pass > control values around, other than going crazy on functions. Yes, that's one side of the problem. The other is that this was done like this intentionally (i.e. to have USE flags implicitly converted into LINGUAS). Just someone didn't have enough foresight where this would lead. > The only thing I thought of when reading the writeup was tagged > install mask verbs. > > That is: > > - Always install all language support possible by default. This still leaves the problem: how to enforce this? As long as LINGUAS leaks into ebuild env (either from USE_EXPAND magic, from make.conf or from parent env), we're in trouble. > - Have an ebuild mapping of some description that annotates "files X > add support for languages ( y, z, x )" > > And then maybe you'd have a parameterized INSTALL_MASK system where you could do > > INSTALL_MASK_RULES="linguas(-* en)" > > Then it would be "Packager decides" if they want to trip a rebuild at > the end user when they "discovered" certain files pertain to specific > language support. > > Then the only reason you'd need USE flags is if you wanted to toggle > compile time support that pulled in dependencies to support the > language in question. Yes, exactly. Though my plan was for it to be more like: INSTALL_MASK="-@linguas @linguas_pl" or alike. Though it's all implementation detail. -- Best regards, Michał Górny <http://dev.gentoo.org/~mgorny/> [-- Attachment #2: OpenPGP digital signature --] [-- Type: application/pgp-signature, Size: 949 bytes --] ^ permalink raw reply [flat|nested] 14+ messages in thread
* [gentoo-dev] Re: [RFC] How to deal with LINGUAS mess? 2016-05-21 7:41 [gentoo-dev] [RFC] How to deal with LINGUAS mess? Michał Górny 2016-05-21 8:14 ` Kent Fredric @ 2016-05-21 9:00 ` Ulrich Mueller 2016-05-21 9:04 ` Kent Fredric 2016-05-21 19:35 ` Michał Górny 2016-05-21 12:20 ` [gentoo-dev] " Michael Orlitzky 2016-05-21 15:19 ` waltdnes 3 siblings, 2 replies; 14+ messages in thread From: Ulrich Mueller @ 2016-05-21 9:00 UTC (permalink / raw To: Michał Górny; +Cc: gentoo-dev, qa [-- Attachment #1: Type: text/plain, Size: 564 bytes --] >>>>> On Sat, 21 May 2016, Michał Górny wrote: > I see the following possibilities: > 1. We start explicitly listing linguas_* in all ebuilds, no matter > how tiny they are. Maintainers are required to keep IUSE up-to-date > and users are forced to rebuild a lot. Why would users have to rebuild more often? Language support in a package will change with a version bump, when they must rebuild in any case. > This is also a QA violation in terms of invalid use of USE flags. I fail to see why this would be a QA violation. > [...] Ulrich [-- Attachment #2: Type: application/pgp-signature, Size: 490 bytes --] ^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: [gentoo-dev] Re: [RFC] How to deal with LINGUAS mess? 2016-05-21 9:00 ` [gentoo-dev] " Ulrich Mueller @ 2016-05-21 9:04 ` Kent Fredric 2016-05-21 9:32 ` Ulrich Mueller 2016-05-21 19:35 ` Michał Górny 1 sibling, 1 reply; 14+ messages in thread From: Kent Fredric @ 2016-05-21 9:04 UTC (permalink / raw To: gentoo-dev; +Cc: Michał Górny, qa On 21 May 2016 at 21:00, Ulrich Mueller <ulm@gentoo.org> wrote: > Why would users have to rebuild more often? Language support in a > package will change with a version bump, when they must rebuild in any > case. But changing IUSE would cause users to cause rebuilds under --new-use, even though no actual changes happened ( this happens a bit as it is ). -- Kent KENTNL - https://metacpan.org/author/KENTNL ^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: [gentoo-dev] Re: [RFC] How to deal with LINGUAS mess? 2016-05-21 9:04 ` Kent Fredric @ 2016-05-21 9:32 ` Ulrich Mueller 0 siblings, 0 replies; 14+ messages in thread From: Ulrich Mueller @ 2016-05-21 9:32 UTC (permalink / raw To: Kent Fredric; +Cc: gentoo-dev, Michał Górny, qa [-- Attachment #1: Type: text/plain, Size: 606 bytes --] >>>>> On Sat, 21 May 2016, Kent Fredric wrote: > On 21 May 2016 at 21:00, Ulrich Mueller <ulm@gentoo.org> wrote: >> Why would users have to rebuild more often? Language support in a >> package will change with a version bump, when they must rebuild in >> any case. > But changing IUSE would cause users to cause rebuilds under > --new-use, even though no actual changes happened ( this happens a > bit as it is ). They would have to rebuild the package once, when linguas_* flags are first added to IUSE. After that, I don't see what would trigger additional rebuilds outside of a version bump. Ulrich [-- Attachment #2: Type: application/pgp-signature, Size: 490 bytes --] ^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: [gentoo-dev] Re: [RFC] How to deal with LINGUAS mess? 2016-05-21 9:00 ` [gentoo-dev] " Ulrich Mueller 2016-05-21 9:04 ` Kent Fredric @ 2016-05-21 19:35 ` Michał Górny 1 sibling, 0 replies; 14+ messages in thread From: Michał Górny @ 2016-05-21 19:35 UTC (permalink / raw To: Ulrich Mueller; +Cc: gentoo-dev, qa [-- Attachment #1: Type: text/plain, Size: 1171 bytes --] On Sat, 21 May 2016 11:00:08 +0200 Ulrich Mueller <ulm@gentoo.org> wrote: > >>>>> On Sat, 21 May 2016, Michał Górny wrote: > > > I see the following possibilities: > > > 1. We start explicitly listing linguas_* in all ebuilds, no matter > > how tiny they are. Maintainers are required to keep IUSE up-to-date > > and users are forced to rebuild a lot. > > Why would users have to rebuild more often? Language support in a > package will change with a version bump, when they must rebuild in any > case. Except whenever: 1. developer fails to update lingua list, and needs to do so afterwards, 2. user changes his preference (i.e. I just built my system and figured out I should really strip those damn locales), 3. user attempts to use binary packages (unless someone goes for providing 2^n binary package variants). > > > This is also a QA violation in terms of invalid use of USE flags. > > I fail to see why this would be a QA violation. It falls into the rule for controlling installation of small files. Most of localizations are < 20 KiB. -- Best regards, Michał Górny <http://dev.gentoo.org/~mgorny/> [-- Attachment #2: OpenPGP digital signature --] [-- Type: application/pgp-signature, Size: 949 bytes --] ^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: [gentoo-dev] [RFC] How to deal with LINGUAS mess? 2016-05-21 7:41 [gentoo-dev] [RFC] How to deal with LINGUAS mess? Michał Górny 2016-05-21 8:14 ` Kent Fredric 2016-05-21 9:00 ` [gentoo-dev] " Ulrich Mueller @ 2016-05-21 12:20 ` Michael Orlitzky 2016-05-21 15:19 ` waltdnes 3 siblings, 0 replies; 14+ messages in thread From: Michael Orlitzky @ 2016-05-21 12:20 UTC (permalink / raw To: gentoo-dev On 05/21/2016 03:41 AM, Michał Górny wrote: > > I see the following possibilities: > #2 is ugly and requires a special case due to a bad choice of variable name; #4 will never work. > 3. We remove LINGUAS from USE_EXPAND and stop using it. If ebuilds have > a good reason to use flags for localization, we introduce a new, > non-colliding USE_EXPAND for that. We also ask users to replace LINGUAS > with the new flag in their make.conf files. LINGUAS gets the original > upstream behavior back, and we eventually discourage it in favor of new > INSTALL_MASK features (WiP) [2]. > This is probably the best option because it fixes the real problem: we tried to repurpose somebody else's environment variable for our package manager. If we try to keep the name "LINGUAS", we may run into some other problem down the line. > 1. We start explicitly listing linguas_* in all ebuilds, no matter how > tiny they are. Maintainers are required to keep IUSE up-to-date > and users are forced to rebuild a lot. This is also a QA violation > in terms of invalid use of USE flags. This isn't as bad as you make it sound... the rebuilds would happen once on a revision bump. The QA violation (I'm guessing) is that USE flags shouldn't be used to control the installation of small text files. I prefer a looser interpretation of that rule: maintainers don't have to waste their time and complicate their ebuilds with USE flags to control the installation of small text files if they don't want to. Basically an "it's OK to install systemd unit files unconditionally" rule. Or in other words, the rule is "it's OK not to do it" rather than "it's not OK to do it." I think if someone /wants/ to have a bunch of logic controlling the installation of localization files, that's fine. But, this option would force everyone to do it in order to work around an unfortunate choice of variable name. It also adds some eternal mental overhead in that we have to collectively remember that LINGUAS is special somehow and teach that to everyone. For those reasons I think #3 is a better long-term solution. ^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: [gentoo-dev] [RFC] How to deal with LINGUAS mess? 2016-05-21 7:41 [gentoo-dev] [RFC] How to deal with LINGUAS mess? Michał Górny ` (2 preceding siblings ...) 2016-05-21 12:20 ` [gentoo-dev] " Michael Orlitzky @ 2016-05-21 15:19 ` waltdnes 2016-05-27 7:17 ` Mart Raudsepp 2016-05-29 12:58 ` Michał Górny 3 siblings, 2 replies; 14+ messages in thread From: waltdnes @ 2016-05-21 15:19 UTC (permalink / raw To: gentoo-dev On Sat, May 21, 2016 at 09:41:28AM +0200, Micha?? Górny wrote > I see the following possibilities: > > 1. We start explicitly listing linguas_* in all ebuilds, no matter how > tiny they are. Maintainers are required to keep IUSE up-to-date > and users are forced to rebuild a lot. This is also a QA violation > in terms of invalid use of USE flags. > > 2. We hack-unset LINGUAS in ebuilds. This is a lot of effort, easy to > miss and probably would need to repeated for every single phase anyway > due to how global variables are handled in PMS. Additionally, it may > break at some point since those variables are likely expected to be > read-only anyway. > > 3. We remove LINGUAS from USE_EXPAND and stop using it. If ebuilds have > a good reason to use flags for localization, we introduce a new, > non-colliding USE_EXPAND for that. We also ask users to replace LINGUAS > with the new flag in their make.conf files. LINGUAS gets the original > upstream behavior back, and we eventually discourage it in favor of new > INSTALL_MASK features (WiP) [2]. > > 4. We fix build systems not to do magic depending on whether LINGUAS > is unset or set-to-empty. Instead, we could some special special value > like '-' to signify not installing localizations at all. But that's > upstream thing to do, and breaks backwards compatibility with existing > systems disabling localizations. > > > Your thoughts? 5. An reversed variant of INSTALL_MASK in make.conf, e.g. LOCALE_ALLOW="foo bar fubar" which would block installing files in /usr/share/locale/* and /usr/share/man/* EXCEPT for... /usr/share/locale/foo /usr/share/locale/bar /usr/share/locale/fubar /usr/share/man/foo /usr/share/man/bar /usr/share/man/fubar 6. Integrate localepurge into Portage, and run it post install There are some lazy programmers out there who *DO NOT* respect the LINGUAS setting, and splatter files throughout /usr/share/locale/* and /usr/share/man/* regardless. That's the reason "localepurge" was written in the first place. Any proposed solution should take that problem into consideration, and handle it too. -- Walter Dnes <waltdnes@waltdnes.org> I don't run "desktop environments"; I run useful applications ^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: [gentoo-dev] [RFC] How to deal with LINGUAS mess? 2016-05-21 15:19 ` waltdnes @ 2016-05-27 7:17 ` Mart Raudsepp 2016-05-29 13:03 ` Michał Górny 2016-05-29 12:58 ` Michał Górny 1 sibling, 1 reply; 14+ messages in thread From: Mart Raudsepp @ 2016-05-27 7:17 UTC (permalink / raw To: gentoo-dev Ühel kenal päeval, L, 21.05.2016 kell 11:19, kirjutas waltdnes@waltdnes.org: > On Sat, May 21, 2016 at 09:41:28AM +0200, Micha?? Górny wrote > > 3. We remove LINGUAS from USE_EXPAND and stop using it. If ebuilds > > have > > a good reason to use flags for localization, we introduce a new, > > non-colliding USE_EXPAND for that. We also ask users to replace > > LINGUAS > > with the new flag in their make.conf files. LINGUAS gets the > > original > > upstream behavior back, and we eventually discourage it in favor of > > new > > INSTALL_MASK features (WiP) [2]. Short of making an exception to LINGUAS in the USE_EXPAND rules, I think this is the only way. > 5. An reversed variant of INSTALL_MASK in make.conf, e.g. > LOCALE_ALLOW="foo bar fubar" > > 6. Integrate localepurge into Portage, and run it post install > > There are some lazy programmers out there who *DO NOT* respect the > LINGUAS setting, and splatter files throughout /usr/share/locale/* > and > /usr/share/man/* regardless. That's the reason "localepurge" was > written in the first place. Any proposed solution should take that > problem into consideration, and handle it too. For both of these cases, I have to point out that e.g LINGUAS="en et pl" does not mean "keep only /usr/share/locale/{en,et,pl}/*", it means have support for only these languages. This means for example that *.desktop files will have translations in them only for those language. Same for dconf schema files. Same for many other things. MO Translation files and manuals aren't the only thing here, especially with intltool (and many of these intltool features are now part of gettext proper). In short, LINGUAS affects the content of files too, not only the existence of files. See any file in /usr/share/applications/ for example. I always put "en" as my first in LINGUAS, due to historical abuse of the first value there, I think e.g mplayer or vlc used to do this. LINGUAS is an unordered list, but some packages used to took the first value as meaning the default and switch the UI to that by default, instead of honoring LC_* variables. Another reason I put "en" there, is because of IUSE_EXPAND and packages that might not be natively english, but do have english translations (I think I've seen some french ones like that :D) And no, localepurge is not capable of stripping these translations out of existing files. To my knowledge at least. Though it could be improved to do so in some cases. Mart ^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: [gentoo-dev] [RFC] How to deal with LINGUAS mess? 2016-05-27 7:17 ` Mart Raudsepp @ 2016-05-29 13:03 ` Michał Górny 0 siblings, 0 replies; 14+ messages in thread From: Michał Górny @ 2016-05-29 13:03 UTC (permalink / raw To: Mart Raudsepp; +Cc: gentoo-dev [-- Attachment #1: Type: text/plain, Size: 3329 bytes --] On Fri, 27 May 2016 10:17:20 +0300 Mart Raudsepp <leio@gentoo.org> wrote: > Ühel kenal päeval, L, 21.05.2016 kell 11:19, kirjutas > waltdnes@waltdnes.org: > > On Sat, May 21, 2016 at 09:41:28AM +0200, Micha?? Górny wrote > > > 3. We remove LINGUAS from USE_EXPAND and stop using it. If ebuilds > > > have > > > a good reason to use flags for localization, we introduce a new, > > > non-colliding USE_EXPAND for that. We also ask users to replace > > > LINGUAS > > > with the new flag in their make.conf files. LINGUAS gets the > > > original > > > upstream behavior back, and we eventually discourage it in favor of > > > new > > > INSTALL_MASK features (WiP) [2]. > > Short of making an exception to LINGUAS in the USE_EXPAND rules, I > think this is the only way. > > > 5. An reversed variant of INSTALL_MASK in make.conf, e.g. > > LOCALE_ALLOW="foo bar fubar" > > > > 6. Integrate localepurge into Portage, and run it post install > > > > There are some lazy programmers out there who *DO NOT* respect the > > LINGUAS setting, and splatter files throughout /usr/share/locale/* > > and > > /usr/share/man/* regardless. That's the reason "localepurge" was > > written in the first place. Any proposed solution should take that > > problem into consideration, and handle it too. > > For both of these cases, I have to point out that e.g > LINGUAS="en et pl" > does not mean "keep only /usr/share/locale/{en,et,pl}/*", it means have > support for only these languages. This means for example that *.desktop > files will have translations in them only for those language. Same for > dconf schema files. Same for many other things. MO Translation files > and manuals aren't the only thing here, especially with intltool (and > many of these intltool features are now part of gettext proper). > In short, LINGUAS affects the content of files too, not only the > existence of files. See any file in /usr/share/applications/ for > example. Just to be clear, I don't think we care that much about filtering those. I can understand people not wanting to install a dozen translation files because of their potential size, especially when filesystem can't handle small files efficiently. But stripping extra descriptions from .desktop files doesn't seem like a major use case. > I always put "en" as my first in LINGUAS, due to historical abuse of > the first value there, I think e.g mplayer or vlc used to do this. > LINGUAS is an unordered list, but some packages used to took the first > value as meaning the default and switch the UI to that by default, > instead of honoring LC_* variables. That's yet another reason not to extend the abuse of LINGUAS. I don't really see adding another rule to PMS that attempts to enforce this. > Another reason I put "en" there, is > because of IUSE_EXPAND and packages that might not be natively english, > but do have english translations (I think I've seen some french ones > like that :D) > > > And no, localepurge is not capable of stripping these translations out > of existing files. To my knowledge at least. Though it could be > improved to do so in some cases. Well, this is certainly something that can be done. -- Best regards, Michał Górny <http://dev.gentoo.org/~mgorny/> [-- Attachment #2: OpenPGP digital signature --] [-- Type: application/pgp-signature, Size: 949 bytes --] ^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: [gentoo-dev] [RFC] How to deal with LINGUAS mess? 2016-05-21 15:19 ` waltdnes 2016-05-27 7:17 ` Mart Raudsepp @ 2016-05-29 12:58 ` Michał Górny 2016-05-30 2:21 ` waltdnes 2016-05-30 10:47 ` Andrew Savchenko 1 sibling, 2 replies; 14+ messages in thread From: Michał Górny @ 2016-05-29 12:58 UTC (permalink / raw To: waltdnes; +Cc: gentoo-dev [-- Attachment #1: Type: text/plain, Size: 2167 bytes --] On Sat, 21 May 2016 11:19:07 -0400 waltdnes@waltdnes.org wrote: > On Sat, May 21, 2016 at 09:41:28AM +0200, Micha?? Górny wrote > > > I see the following possibilities: > > > > 1. We start explicitly listing linguas_* in all ebuilds, no matter how > > tiny they are. Maintainers are required to keep IUSE up-to-date > > and users are forced to rebuild a lot. This is also a QA violation > > in terms of invalid use of USE flags. > > > > 2. We hack-unset LINGUAS in ebuilds. This is a lot of effort, easy to > > miss and probably would need to repeated for every single phase anyway > > due to how global variables are handled in PMS. Additionally, it may > > break at some point since those variables are likely expected to be > > read-only anyway. > > > > 3. We remove LINGUAS from USE_EXPAND and stop using it. If ebuilds have > > a good reason to use flags for localization, we introduce a new, > > non-colliding USE_EXPAND for that. We also ask users to replace LINGUAS > > with the new flag in their make.conf files. LINGUAS gets the original > > upstream behavior back, and we eventually discourage it in favor of new > > INSTALL_MASK features (WiP) [2]. > > > > 4. We fix build systems not to do magic depending on whether LINGUAS > > is unset or set-to-empty. Instead, we could some special special value > > like '-' to signify not installing localizations at all. But that's > > upstream thing to do, and breaks backwards compatibility with existing > > systems disabling localizations. > > > > > > Your thoughts? > > 5. An reversed variant of INSTALL_MASK in make.conf, e.g. > LOCALE_ALLOW="foo bar fubar" > > which would block installing files in /usr/share/locale/* and > /usr/share/man/* EXCEPT for... > > /usr/share/locale/foo > /usr/share/locale/bar > /usr/share/locale/fubar > /usr/share/man/foo > /usr/share/man/bar > /usr/share/man/fubar This can be accomplished using inclusion/exclusion logic included in the patches I've recently sent for Portage. INSTALL_MASK="/usr/share/locale -/usr/share/locale/foo" -- Best regards, Michał Górny <http://dev.gentoo.org/~mgorny/> [-- Attachment #2: OpenPGP digital signature --] [-- Type: application/pgp-signature, Size: 949 bytes --] ^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: [gentoo-dev] [RFC] How to deal with LINGUAS mess? 2016-05-29 12:58 ` Michał Górny @ 2016-05-30 2:21 ` waltdnes 2016-05-30 10:47 ` Andrew Savchenko 1 sibling, 0 replies; 14+ messages in thread From: waltdnes @ 2016-05-30 2:21 UTC (permalink / raw To: gentoo-dev On Sun, May 29, 2016 at 02:58:03PM +0200, Micha?? Górny wrote > On Sat, 21 May 2016 11:19:07 -0400 > waltdnes@waltdnes.org wrote: > > > 5. An reversed variant of INSTALL_MASK in make.conf, e.g. > > LOCALE_ALLOW="foo bar fubar" > > > > which would block installing files in /usr/share/locale/* and > > /usr/share/man/* EXCEPT for... > > > > /usr/share/locale/foo > > /usr/share/locale/bar > > /usr/share/locale/fubar > > /usr/share/man/foo > > /usr/share/man/bar > > /usr/share/man/fubar > > This can be accomplished using inclusion/exclusion logic included in > the patches I've recently sent for Portage. > > INSTALL_MASK="/usr/share/locale -/usr/share/locale/foo" Thanks. localepurge cleans up after the fact. Your patches would prevent the problem in the first place, which is a better approach. It also handles systemd files and can be used when some new stuff comes up in future that we haven't thought of yet. -- Walter Dnes <waltdnes@waltdnes.org> I don't run "desktop environments"; I run useful applications ^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: [gentoo-dev] [RFC] How to deal with LINGUAS mess? 2016-05-29 12:58 ` Michał Górny 2016-05-30 2:21 ` waltdnes @ 2016-05-30 10:47 ` Andrew Savchenko 1 sibling, 0 replies; 14+ messages in thread From: Andrew Savchenko @ 2016-05-30 10:47 UTC (permalink / raw To: gentoo-dev [-- Attachment #1: Type: text/plain, Size: 3475 bytes --] On Sun, 29 May 2016 14:58:03 +0200 Michał Górny wrote: > On Sat, 21 May 2016 11:19:07 -0400 > waltdnes@waltdnes.org wrote: > > > On Sat, May 21, 2016 at 09:41:28AM +0200, Micha?? Górny wrote > > > > > I see the following possibilities: > > > > > > 1. We start explicitly listing linguas_* in all ebuilds, no matter how > > > tiny they are. Maintainers are required to keep IUSE up-to-date > > > and users are forced to rebuild a lot. This is also a QA violation > > > in terms of invalid use of USE flags. > > > > > > 2. We hack-unset LINGUAS in ebuilds. This is a lot of effort, easy to > > > miss and probably would need to repeated for every single phase anyway > > > due to how global variables are handled in PMS. Additionally, it may > > > break at some point since those variables are likely expected to be > > > read-only anyway. > > > > > > 3. We remove LINGUAS from USE_EXPAND and stop using it. If ebuilds have > > > a good reason to use flags for localization, we introduce a new, > > > non-colliding USE_EXPAND for that. We also ask users to replace LINGUAS > > > with the new flag in their make.conf files. LINGUAS gets the original > > > upstream behavior back, and we eventually discourage it in favor of new > > > INSTALL_MASK features (WiP) [2]. > > > > > > 4. We fix build systems not to do magic depending on whether LINGUAS > > > is unset or set-to-empty. Instead, we could some special special value > > > like '-' to signify not installing localizations at all. But that's > > > upstream thing to do, and breaks backwards compatibility with existing > > > systems disabling localizations. > > > > > > > > > Your thoughts? > > > > 5. An reversed variant of INSTALL_MASK in make.conf, e.g. > > LOCALE_ALLOW="foo bar fubar" > > > > which would block installing files in /usr/share/locale/* and > > /usr/share/man/* EXCEPT for... > > > > /usr/share/locale/foo > > /usr/share/locale/bar > > /usr/share/locale/fubar > > /usr/share/man/foo > > /usr/share/man/bar > > /usr/share/man/fubar > > This can be accomplished using inclusion/exclusion logic included in > the patches I've recently sent for Portage. > > INSTALL_MASK="/usr/share/locale -/usr/share/locale/foo" A proper way will be to fix ebuilds to respect LINGUAS properly. l10n.eclass makes this quite easy. Build and install files just to remove them later before/after merge to a live system is ridiculous, especially considering that LINGUAS covers not only manuals, but also html docs and other files. Why users should care about additional INSTALL_MASK ordeal if they already set up LINGUAS properly? This is PM and ebuild maintainers job. The same states for cron, systemd, logrotate and other potentially unneeded files. Install mask is dangerous, very dangerous, because locations may move, mandatory files may have accidentally removed by an oversight. INSTALL_MASK is indeed useful, essential tool and it is nice that it is being extended, but it should be a last resort way to fix stuff, not a recommended technique. The real problem is that "small files" control creates much burden for package maintainers. It should be fixed by versatile and easy to use eclasses, ideally the most popular helpers should go to PMS, e.g. to default src_install(). Moving all burden of "small files" control and optional cleanup from developers to users is not nice. Best regards, Andrew Savchenko [-- Attachment #2: Type: application/pgp-signature, Size: 819 bytes --] ^ permalink raw reply [flat|nested] 14+ messages in thread
end of thread, other threads:[~2016-05-30 10:48 UTC | newest] Thread overview: 14+ messages (download: mbox.gz follow: Atom feed -- links below jump to the message on this page -- 2016-05-21 7:41 [gentoo-dev] [RFC] How to deal with LINGUAS mess? Michał Górny 2016-05-21 8:14 ` Kent Fredric 2016-05-29 12:30 ` Michał Górny 2016-05-21 9:00 ` [gentoo-dev] " Ulrich Mueller 2016-05-21 9:04 ` Kent Fredric 2016-05-21 9:32 ` Ulrich Mueller 2016-05-21 19:35 ` Michał Górny 2016-05-21 12:20 ` [gentoo-dev] " Michael Orlitzky 2016-05-21 15:19 ` waltdnes 2016-05-27 7:17 ` Mart Raudsepp 2016-05-29 13:03 ` Michał Górny 2016-05-29 12:58 ` Michał Górny 2016-05-30 2:21 ` waltdnes 2016-05-30 10:47 ` Andrew Savchenko
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox