[gentoo-user] flag icu

[gentoo-user] flag icu

[james]

Hello,

Q1} In my attempts to minimize flag settings, why do I need the flag "icu" ?

+ - icu   Enable ICU (Internationalization Components for Unicode)
		support, using dev-libs/icu




Q2} Ultimately, I'm trying to discover that minimum of flags for amd64
servers and workstations, that allow them to function nominally
normal, with most flags set per package.use. The bare_bones flags really
work great for gentoo clusters. I'm specifically trying to avoid the
advanced profiles, as I intend to manage flags dynamically as frameworks
(collections of codes) are added or removed from the cluster.


Any hints on a systematic by system parsing this sort of minimized-flag
data :

 [12]  default/linux/amd64/17.0 (stable) *


would be keenly appreciated. "eselect profile list" is great. but
I need it per many different architectures and do not have one
of each of the systems I need to experiment on. How are those flag_sets
discovered in some sort of systematic approach?



James

Re: [gentoo-user] flag icu

[Corentin “Nado” Pazdera]

August 13, 2018 4:31 PM, "james" <garftd@verizon.net> wrote:

> Any hints on a systematic by system parsing this sort of minimized-flag
> data :
> 
> [12] default/linux/amd64/17.0 (stable) *
> 
> would be keenly appreciated. "eselect profile list" is great. but
> I need it per many different architectures and do not have one
> of each of the systems I need to experiment on. How are those flag_sets
> discovered in some sort of systematic approach?

Hi,

I don't know if this will be of any help but I made a script [1] recently to analyze the
inheritance of system set.
It may have a few bugs I did that for learning the inner workings of profiles mainly.
It should'nt need much work to make it print USE flags details

[1] https://gist.github.com/nado/44b392b50c0b71a7e22b98d6909bfa72

Best regards,
--
Corentin “Nado” Pazdera

Re: [gentoo-user] flag icu

[james]

On 08/13/18 11:36, Corentin �Nado� Pazdera wrote:
> August 13, 2018 4:31 PM, "james" <garftd@verizon.net> wrote:
> 
>> Any hints on a systematic by system parsing this sort of minimized-flag
>> data :
>>
>> [12] default/linux/amd64/17.0 (stable) *
>>
>> would be keenly appreciated. "eselect profile list" is great. but
>> I need it per many different architectures and do not have one
>> of each of the systems I need to experiment on. How are those flag_sets
>> discovered in some sort of systematic approach?
> 
> Hi,
> 
> I don't know if this will be of any help but I made a script [1] recently to analyze the
> inheritance of system set.
> It may have a few bugs I did that for learning the inner workings of profiles mainly.
> It should'nt need much work to make it print USE flags details
> 
> [1] https://gist.github.com/nado/44b392b50c0b71a7e22b98d6909bfa72
> 
> Best regards,
> --
> Corentin �Nado� Pazdera
> 
> 


Hey thanks,

I'm testing now an it has given me a few ideas to extend the
capabilities....


James

Re: [gentoo-user] flag icu

[james]

On 08/13/18 11:47, james wrote:
> On 08/13/18 11:36, Corentin �Nado� Pazdera wrote:
>> August 13, 2018 4:31 PM, "james" <garftd@verizon.net> wrote:
>>
>>> Any hints on a systematic by system parsing this sort of minimized-flag
>>> data :
>>>
>>> [12] default/linux/amd64/17.0 (stable) *
>>>
>>> would be keenly appreciated. "eselect profile list" is great. but
>>> I need it per many different architectures and do not have one
>>> of each of the systems I need to experiment on. How are those flag_sets
>>> discovered in some sort of systematic approach?
>>
>> Hi,
>>
>> I don't know if this will be of any help but I made a script [1] recently to analyze the
>> inheritance of system set.
>> It may have a few bugs I did that for learning the inner workings of profiles mainly.
>> It should'nt need much work to make it print USE flags details
>>
>> [1] https://gist.github.com/nado/44b392b50c0b71a7e22b98d6909bfa72
>>
>> Best regards,
>> --
>> Corentin �Nado� Pazdera
>>
>>
> 
> 
> Hey thanks,
> 
> I'm testing now an it has given me a few ideas to extend the
> capabilities....
> 
> 
> James
> 

Here's what I got running your script::


/etc # /root/profile-explorer.sh
--- Invalid atom in /etc/portage/package.use/explored-packages:
/usr/portage/profiles/base/packages
--- Invalid atom in /etc/portage/package.use/explored-packages:
*>=sys-apps/baselayout-2
--- Invalid atom in /etc/portage/package.use/explored-packages:
*>=sys-apps/findutils-4.4
--- Invalid atom in /etc/portage/package.use/explored-packages:
*>=sys-devel/patch-2.7
--- Invalid atom in /etc/portage/package.use/explored-packages:
/usr/portage/profiles/default/linux/packages
--- Invalid atom in /etc/portage/package.use/explored-packages:
/usr/portage/profiles/base/packages
--- Invalid atom in /etc/portage/package.use/explored-packages:
*>=sys-apps/baselayout-2
--- Invalid atom in /etc/portage/package.use/explored-packages:
*>=sys-apps/findutils-4.4
--- Invalid atom in /etc/portage/package.use/explored-packages:
*>=sys-devel/patch-2.7
--- Invalid atom in /etc/portage/package.use/explored-packages:
/usr/portage/profiles/default/linux/packages


Manually looking a the

less /etc/portage/package.use/explored-packages:
/usr/portage/profiles/base/packages


I see:
# Old ICU is unsupported. ICU 58 only remains for 13.0 based profiles.
<dev-libs/icu-59
<dev-libs/icu-layoutex-59

But the system has::

[I] dev-libs/icu ....   60.2

equery uses icu

gives me similar info:


--- Invalid atom in /etc/portage/package.use/explored-packages:
/usr/portage/profiles/base/packages
--- Invalid atom in /etc/portage/package.use/explored-packages:
*>=sys-apps/baselayout-2
--- Invalid atom in /etc/portage/package.use/explored-packages:
*>=sys-apps/findutils-4.4
--- Invalid atom in /etc/portage/package.use/explored-packages:
*>=sys-devel/patch-2.7
--- Invalid atom in /etc/portage/package.use/explored-packages:
/usr/portage/profiles/default/linux/packages
--- Invalid atom in /etc/portage/package.use/explored-packages:
/usr/portage/profiles/base/packages
--- Invalid atom in /etc/portage/package.use/explored-packages:
*>=sys-apps/baselayout-2
--- Invalid atom in /etc/portage/package.use/explored-packages:
*>=sys-apps/findutils-4.4
--- Invalid atom in /etc/portage/package.use/explored-packages:
*>=sys-devel/patch-2.7
--- Invalid atom in /etc/portage/package.use/explored-packages:
/usr/portage/profiles/default/linux/packages
[ Legend : U - final flag setting for installation]
[        : I - package is installed with flag     ]
[ Colors : set, unset                             ]
 * Found these USE flags for dev-libs/icu-60.2:
 U I
 + + abi_x86_32  : 32-bit (x86) libraries
 - - debug       : Enable extra debug codepaths, like asserts and extra
output. If
                   you want to get meaningful backtraces see

https://wiki.gentoo.org/wiki/Project:Quality_Assurance/Backtraces
 - - doc         : Add extra documentation (API, Javadoc, etc). It is
recommended to
                   enable per package instead of globally
 + + examples    : Install examples, usually source code
 - - static-libs : Build static versions of dynamic libraries as well




Which begs the Q1}    can I get rid of the flag icu? What are
consequences, as a baseline system flag, of it's removal ?


less /usr/portage/profiles/base/packages
show me more of what the @system set contains. Very interesting and
useful. I'm thinking of aggregation of those listed packages
and some basic (ascii) table form (equery,emerge, eix) parsed listing
of the default and current flag settings. A "verification" tool
if you like. Surely it would help if this info was (is?) more readily
available and organized for folks that need a systematic approach, like
heterogeneous HPC clusters. The tools exist for 'ad-hoc' and one off,
but more of an organized representation at least at the set level.

I feel like there is an existing tool that can yield all of this
information, as it is on a current system. I've read where there are
efforts to clean up the packages and default flags used in @system,
so the bare minimum list per arch/profiles  would ultimately be
a useful listing, particular for my HPC. In HPC less is always faster
and better, as it is in security and so many more aspects of CS.


Obviosly, I have a few things to fix on this (fragile) system, but
that'll happen as I'm at the beginning stages of auto_installs of
minimized systems. What are your plans for you little script?

Just to match equery uses <flag> and such?


Here's a cutie:
 /usr/portage/profiles/default/linux/amd64/package.use.mask

# Mike Frysinger <vapier@gentoo.org> (08 May 2016)
# This target supports VTV #547040.
>=sys-devel/gcc-4.9 -vtv

# Mike Frysinger <vapier@gentoo.org> (21 Oct 2014)
# This target supports ASAN/etc... #504200.
sys-devel/gcc -sanitize


And where was it that the processor/arch flags are now listed?


tia,
James
cat

Re: [gentoo-user] flag icu

[Corentin “Nado” Pazdera]

August 13, 2018 6:58 PM, "james" <garftd@verizon.net> wrote:

> Here's what I got running your script::
> 
> /etc # /root/profile-explorer.sh
> --- Invalid atom in /etc/portage/package.use/explored-packages:
> /usr/portage/profiles/base/packages
> --- Invalid atom in /etc/portage/package.use/explored-packages:
> *>=sys-apps/baselayout-2
> --- Invalid atom in /etc/portage/package.use/explored-packages:
> *>=sys-apps/findutils-4.4
> --- Invalid atom in /etc/portage/package.use/explored-packages:
> *>=sys-devel/patch-2.7
> --- Invalid atom in /etc/portage/package.use/explored-packages:
> /usr/portage/profiles/default/linux/packages
> --- Invalid atom in /etc/portage/package.use/explored-packages:
> /usr/portage/profiles/base/packages
> --- Invalid atom in /etc/portage/package.use/explored-packages:
> *>=sys-apps/baselayout-2
> --- Invalid atom in /etc/portage/package.use/explored-packages:
> *>=sys-apps/findutils-4.4
> --- Invalid atom in /etc/portage/package.use/explored-packages:
> *>=sys-devel/patch-2.7
> --- Invalid atom in /etc/portage/package.use/explored-packages:
> /usr/portage/profiles/default/linux/packages
> 
> Manually looking a the

Seems weird, also no need to run it as root...
Here's my output for comparison :

```
% ./profile-explorer.sh
[+] EROOT : /
[+] PORTDIR : /var/db/repos/gentoo
[+] CURPROFILE: default/linux/amd64/17.0
[+] EAPI : 5

[+] packages (@system)
/var/db/repos/gentoo/profiles/base/packages
/var/db/repos/gentoo/profiles/default/linux/packages
```

And the `explored-packages` file should symply contain a copy of the different inherited packages
files.

> less /etc/portage/package.use/explored-packages:
> /usr/portage/profiles/base/packages
> 
> I see:
> # Old ICU is unsupported. ICU 58 only remains for 13.0 based profiles.
> <dev-libs/icu-59
> <dev-libs/icu-layoutex-59
> 
> But the system has::
> 
> [I] dev-libs/icu .... 60.2
> 
> equery uses icu
> 
> gives me similar info:
> 
> --- Invalid atom in /etc/portage/package.use/explored-packages:
> /usr/portage/profiles/base/packages
> --- Invalid atom in /etc/portage/package.use/explored-packages:
> *>=sys-apps/baselayout-2
> --- Invalid atom in /etc/portage/package.use/explored-packages:
> *>=sys-apps/findutils-4.4
> --- Invalid atom in /etc/portage/package.use/explored-packages:
> *>=sys-devel/patch-2.7
> --- Invalid atom in /etc/portage/package.use/explored-packages:
> /usr/portage/profiles/default/linux/packages
> --- Invalid atom in /etc/portage/package.use/explored-packages:
> /usr/portage/profiles/base/packages
> --- Invalid atom in /etc/portage/package.use/explored-packages:
> *>=sys-apps/baselayout-2
> --- Invalid atom in /etc/portage/package.use/explored-packages:
> *>=sys-apps/findutils-4.4
> --- Invalid atom in /etc/portage/package.use/explored-packages:
> *>=sys-devel/patch-2.7
> --- Invalid atom in /etc/portage/package.use/explored-packages:
> /usr/portage/profiles/default/linux/packages
> [ Legend : U - final flag setting for installation]
> [ : I - package is installed with flag ]
> [ Colors : set, unset ]
> * Found these USE flags for dev-libs/icu-60.2:
> U I
> + + abi_x86_32 : 32-bit (x86) libraries
> - - debug : Enable extra debug codepaths, like asserts and extra
> output. If
> you want to get meaningful backtraces see
> 
> https://wiki.gentoo.org/wiki/Project:Quality_Assurance/Backtraces
> - - doc : Add extra documentation (API, Javadoc, etc). It is
> recommended to
> enable per package instead of globally
> + + examples : Install examples, usually source code
> - - static-libs : Build static versions of dynamic libraries as well
> 
> Which begs the Q1} can I get rid of the flag icu? What are
> consequences, as a baseline system flag, of it's removal ?
> 
> less /usr/portage/profiles/base/packages
> show me more of what the @system set contains. Very interesting and
> useful. I'm thinking of aggregation of those listed packages
> and some basic (ascii) table form (equery,emerge, eix) parsed listing
> of the default and current flag settings. A "verification" tool
> if you like. Surely it would help if this info was (is?) more readily
> available and organized for folks that need a systematic approach, like
> heterogeneous HPC clusters. The tools exist for 'ad-hoc' and one off,
> but more of an organized representation at least at the set level.
> 
> I feel like there is an existing tool that can yield all of this
> information, as it is on a current system. I've read where there are
> efforts to clean up the packages and default flags used in @system,
> so the bare minimum list per arch/profiles would ultimately be
> a useful listing, particular for my HPC. In HPC less is always faster
> and better, as it is in security and so many more aspects of CS.
> 
> Obviosly, I have a few things to fix on this (fragile) system, but
> that'll happen as I'm at the beginning stages of auto_installs of
> minimized systems. What are your plans for you little script?
> 
> Just to match equery uses <flag> and such?
> 
> Here's a cutie:
> /usr/portage/profiles/default/linux/amd64/package.use.mask
> 
> # Mike Frysinger <vapier@gentoo.org> (08 May 2016)
> # This target supports VTV #547040.
>> =sys-devel/gcc-4.9 -vtv
> 
> # Mike Frysinger <vapier@gentoo.org> (21 Oct 2014)
> # This target supports ASAN/etc... #504200.
> sys-devel/gcc -sanitize
> 
> And where was it that the processor/arch flags are now listed?
> 
> tia,
> James
> cat

To check impact on negating icu on your system : `USE="-icu" emerge -puDU --with-bdeps=y world`

And concerning processor/arch flags I’d suggest keep exploring profiles, take a look at
make.defaults files.
Here the different files you can find according to PMS
https://dev.gentoo.org/~ulm/pms/head/pms.html#x1-430005.2

--
Corentin “Nado” Pazdera

[gentoo-user] Re: flag icu

[Nikos Chantziaras]

On 13/08/18 17:31, james wrote:
> Hello,
> 
> Q1} In my attempts to minimize flag settings, why do I need the flag "icu" ?
> 
> + - icu   Enable ICU (Internationalization Components for Unicode)
> 		support, using dev-libs/icu

Depends on the software. If my software uses ICU for Unicode-related 
stuff, I might provide a way to disable Unicode support, or have a 
half-assed custom implementation that doesn't need ICU but is slower 
and/or doesn't support all Unicode features.

The default USE flags are supposed to reflect the recommended way to 
build the various software packages, either as recommended by upstream 
or determined by the package maintainers.

Unfortunately, most packages don't document their USE flags in their 
metadata, so the user has to go hunting for an explanation for each 
package. But even if each package explained what a USE flag actually 
means for that package, it would be borderline impossible to go through 
each every package of a system to verify whether you lose something 
important or not when you change a USE flag.

Re: [gentoo-user] flag icu

[james]

On 08/13/18 13:14, Corentin �Nado� Pazdera wrote:
> August 13, 2018 6:58 PM, "james" <garftd@verizon.net> wrote:
> 
>> Here's what I got running your script::
>>
>> /etc # /root/profile-explorer.sh
>> --- Invalid atom in /etc/portage/package.use/explored-packages:
>> /usr/portage/profiles/base/packages
>> --- Invalid atom in /etc/portage/package.use/explored-packages:
>> *>=sys-apps/baselayout-2
>> --- Invalid atom in /etc/portage/package.use/explored-packages:
>> *>=sys-apps/findutils-4.4
>> --- Invalid atom in /etc/portage/package.use/explored-packages:
>> *>=sys-devel/patch-2.7
>> --- Invalid atom in /etc/portage/package.use/explored-packages:
>> /usr/portage/profiles/default/linux/packages
>> --- Invalid atom in /etc/portage/package.use/explored-packages:
>> /usr/portage/profiles/base/packages
>> --- Invalid atom in /etc/portage/package.use/explored-packages:
>> *>=sys-apps/baselayout-2
>> --- Invalid atom in /etc/portage/package.use/explored-packages:
>> *>=sys-apps/findutils-4.4
>> --- Invalid atom in /etc/portage/package.use/explored-packages:
>> *>=sys-devel/patch-2.7
>> --- Invalid atom in /etc/portage/package.use/explored-packages:
>> /usr/portage/profiles/default/linux/packages
>>
>> Manually looking a the
> 
> Seems weird, also no need to run it as root...

Exact same output run as user... I suspect it's garbage that's
left from  the 13.0 profiles days. It's was installed
about 5 years ago, and is pretty hacked up (base stabe plus).

I guess I can manually edit those files indicated and then your
sort of out put will most likely occur.... That's my next test.


> Here's my output for comparison :
> 
> ```
> % ./profile-explorer.sh
> [+] EROOT : /
> [+] PORTDIR : /var/db/repos/gentoo
> [+] CURPROFILE: default/linux/amd64/17.0
> [+] EAPI : 5
> 
> [+] packages (@system)
> /var/db/repos/gentoo/profiles/base/packages
> /var/db/repos/gentoo/profiles/default/linux/packages
> ```
> 
> And the `explored-packages` file should symply contain a copy of the different inherited packages
> files.
> 
>> less /etc/portage/package.use/explored-packages:
>> /usr/portage/profiles/base/packages
>>
>> I see:
>> # Old ICU is unsupported. ICU 58 only remains for 13.0 based profiles.
>> <dev-libs/icu-59
>> <dev-libs/icu-layoutex-59
>>
>> But the system has::
>>
>> [I] dev-libs/icu .... 60.2
>>
>> equery uses icu
>>
>> gives me similar info:
>>
>> --- Invalid atom in /etc/portage/package.use/explored-packages:
>> /usr/portage/profiles/base/packages
>> --- Invalid atom in /etc/portage/package.use/explored-packages:
>> *>=sys-apps/baselayout-2
>> --- Invalid atom in /etc/portage/package.use/explored-packages:
>> *>=sys-apps/findutils-4.4
>> --- Invalid atom in /etc/portage/package.use/explored-packages:
>> *>=sys-devel/patch-2.7
>> --- Invalid atom in /etc/portage/package.use/explored-packages:
>> /usr/portage/profiles/default/linux/packages
>> --- Invalid atom in /etc/portage/package.use/explored-packages:
>> /usr/portage/profiles/base/packages
>> --- Invalid atom in /etc/portage/package.use/explored-packages:
>> *>=sys-apps/baselayout-2
>> --- Invalid atom in /etc/portage/package.use/explored-packages:
>> *>=sys-apps/findutils-4.4
>> --- Invalid atom in /etc/portage/package.use/explored-packages:
>> *>=sys-devel/patch-2.7
>> --- Invalid atom in /etc/portage/package.use/explored-packages:
>> /usr/portage/profiles/default/linux/packages
>> [ Legend : U - final flag setting for installation]
>> [ : I - package is installed with flag ]
>> [ Colors : set, unset ]
>> * Found these USE flags for dev-libs/icu-60.2:
>> U I
>> + + abi_x86_32 : 32-bit (x86) libraries
>> - - debug : Enable extra debug codepaths, like asserts and extra
>> output. If
>> you want to get meaningful backtraces see
>>
>> https://wiki.gentoo.org/wiki/Project:Quality_Assurance/Backtraces
>> - - doc : Add extra documentation (API, Javadoc, etc). It is
>> recommended to
>> enable per package instead of globally
>> + + examples : Install examples, usually source code
>> - - static-libs : Build static versions of dynamic librarieshttps://inductiveautomation.com/ as well
>>
>> Which begs the Q1} can I get rid of the flag icu? What are
>> consequences, as a baseline system flag, of it's removal ?
>>
>> less /usr/portage/profiles/base/packages
>> show me more of what the @system set contains. Very interesting and
>> useful. I'm thinking of aggregation of those listed packages
>> and some basic (ascii) table form (equery,emerge, eix) parsed listing
>> of the default and current flag settings. A "verification" tool
>> if you like. Surely it would help if this info was (is?) more readily
>> available and organized for folks that need a systematic approach, like
>> heterogeneous HPC clusters. The tools exist for 'ad-hoc' and one off,
>> but more of an organized representation at least at the set level.
>>
>> I feel like there is an existing tool that can yield all of this
>> information, as it is on a current system. I've read where there are
>> efforts to clean up the packages and default flags used in @system,
>> so the bare minimum list per arch/profiles would ultimately be
>> a useful listing, particular for my HPC. In HPC less is always faster
>> and better, as it is in security and so many more aspects of CS.
>>
>> Obviosly, I have a few things to fix on this (fragile) system, but
>> that'll happen as I'm at the beginning stages of auto_installs of
>> minimized systems. What are your plans for you little script?
>>
>> Just to match equery uses <flag> and such?
>>
>> Here's a cutie:
>> /usr/portage/profiles/default/linux/amd64/package.use.mask
>>
>> # Mike Frysinger <vapier@gentoo.org> (08 May 2016)
>> # This target supports VTV #547040.
>>> =sys-devel/gcc-4.9 -vtv
>>
>> # Mike Frysinger <vapier@gentoo.org> (21 Oct 2014)
>> # This target supports ASAN/etc... #504200.
>> sys-devel/gcc -sanitize
>>
>> And where was it that the processor/arch flags are now listed?
>>
>> tia,
>> James
>> cat
> 
> To check impact on negating icu on your system : `USE="-icu" emerge -puDU --with-bdeps=y world`


I'd have to marinate on the -U option. I see what it s proposing.

> 
> And concerning processor/arch flags I�d suggest keep exploring profiles, take a look at
> make.defaults files.

Exactly. If there ware a list of 5 or 10 of the common arches, those
minimal @system and default flag settings would be keenly  beneficial.

Now that I'm researching to add one (arm64) with at lest 4G of DDR 4, to
the HPC, just for testing; I long for a systematic publish reference of
of these packages and subsequent (default) flags per arch or per processor.


> Here the different files you can find according to PMS
> https://dev.gentoo.org/~ulm/pms/head/pms.html#x1-430005.2

That is an excellent read!

I'll post back, after some experimentation.

> 
> --
> Corentin �Nado� Pazdera
> 
> 

James

Re: [gentoo-user] Re: flag icu

[james]

On 08/13/18 13:17, Nikos Chantziaras wrote:
> On 13/08/18 17:31, james wrote:
>> Hello,
>>
>> Q1} In my attempts to minimize flag settings, why do I need the flag
>> "icu" ?
>>
>> + - icu�� Enable ICU (Internationalization Components for Unicode)
>> ������� support, using dev-libs/icu
> 
> Depends on the software. If my software uses ICU for Unicode-related
> stuff, I might provide a way to disable Unicode support, or have a
> half-assed custom implementation that doesn't need ICU but is slower
> and/or doesn't support all Unicode features.

yep, no such replacement (yet) but up for suggestions, if anything
already exist. What you state is exactly the point, I'm a bit shy
on system wide removal of this flag, without a much deeper understanding
of it's potential effects, which may be catastrophic. I'm not sure how
to dissect ICU (dev-libs/icu) and the 'icu' flag. What the heck oare
'international components' ?

I get the universal ascii character set, that fine. but is that all it
is or is there more baggage strictly not necessary ?

What is Unicode?
Unicode provides a unique number for every character, no matter what the
platform, no matter what the program, no matter what the language. I
have no need to support chineese or any other language other that
english (AM or EN). Does UTF-8 get rid of significant bloat?

Currently I have::

# locale -a
C
POSIX
en_US
en_US.iso88591
en_US.utf8


Perhaps I've read 'too much' and should just run with icu as a global flag?


> 
> The default USE flags are supposed to reflect the recommended way to
> build the various software packages, either as recommended by upstream
> or determined by the package maintainers.
> 
> Unfortunately, most packages don't document their USE flags in their
> metadata, so the user has to go hunting for an explanation for each
> package. But even if each package explained what a USE flag actually
> means for that package, it would be borderline impossible to go through
> each every package of a system to verify whether you lose something
> important or not when you change a USE flag.


Agreed with all you state. I do not intend to do this, except for those
flags set in the relevant @system (package) listing.

Everything else can be part of a 'framework' that is additive/removable
on the HPC cluster, dynamically. Since these extra  frameworks are
already often 'huge' it matters only that all of those packages in a
framework, work as needed.


But optimizing the engine underneath, across the cluster, it's very
important to have things minimized.  And since the clusters are
'loosely coupled' many different processors and architectures can
participate. The smaller the critical package list is and the subsequent
flags, the easier the cluster will be to  boot/run/optimize/maintain.
Thus the requirement for the @system package listings with flags per
many arches are tremendously beneficial to my research and testing.
Yes, I do need to robustly examine not only the @system package list,
but each and every flag.


I pretty much agree with your sentiments, even get a bit of a chuckle as
one of the few that has been 86'd off both gentoo-user and gentoo-dev,
but wanting to stay focused on the needs of my HPC semantics.


James

Re: [gentoo-user] flag icu

[james]

On 08/13/18 13:14, Corentin �Nado� Pazdera wrote:
> August 13, 2018 6:58 PM, "james" <garftd@verizon.net> wrote:
> 
>> Here's what I got running your script::
>>
>> /etc # /root/profile-explorer.sh
>> --- Invalid atom in /etc/portage/package.use/explored-packages:
>> /usr/portage/profiles/base/packages
>> --- Invalid atom in /etc/portage/package.use/explored-packages:
>> *>=sys-apps/baselayout-2
>> --- Invalid atom in /etc/portage/package.use/explored-packages:
>> *>=sys-apps/findutils-4.4
>> --- Invalid atom in /etc/portage/package.use/explored-packages:
>> *>=sys-devel/patch-2.7
>> --- Invalid atom in /etc/portage/package.use/explored-packages:
>> /usr/portage/profiles/default/linux/packages
>> --- Invalid atom in /etc/portage/package.use/explored-packages:
>> /usr/portage/profiles/base/packages
>> --- Invalid atom in /etc/portage/package.use/explored-packages:
>> *>=sys-apps/baselayout-2
>> --- Invalid atom in /etc/portage/package.use/explored-packages:
>> *>=sys-apps/findutils-4.4
>> --- Invalid atom in /etc/portage/package.use/explored-packages:
>> *>=sys-devel/patch-2.7
>> --- Invalid atom in /etc/portage/package.use/explored-packages:
>> /usr/portage/profiles/default/linux/packages
>>
>> Manually looking a the
> 
> Seems weird, also no need to run it as root...
> Here's my output for comparison :
> 
> ```
> % ./profile-explorer.sh
> [+] EROOT : /
> [+] PORTDIR : /var/db/repos/gentoo
> [+] CURPROFILE: default/linux/amd64/17.0
> [+] EAPI : 5
> 
> [+] packages (@system)
> /var/db/repos/gentoo/profiles/base/packages
> /var/db/repos/gentoo/profiles/default/linux/packages
> ```
> 
> And the `explored-packages` file should symply contain a copy of the different inherited packages
> files.
> 
>> less /etc/portage/package.use/explored-packages:
>> /usr/portage/profiles/base/packages
>>
>> I see:
>> # Old ICU is unsupported. ICU 58 only remains for 13.0 based profiles.
>> <dev-libs/icu-59
>> <dev-libs/icu-layoutex-59
>>
>> But the system has::
>>
>> [I] dev-libs/icu .... 60.2
>>
>> equery uses icu
>>
>> gives me similar info:
>>
>> --- Invalid atom in /etc/portage/package.use/explored-packages:
>> /usr/portage/profiles/base/packages
>> --- Invalid atom in /etc/portage/package.use/explored-packages:
>> *>=sys-apps/baselayout-2
>> --- Invalid atom in /etc/portage/package.use/explored-packages:
>> *>=sys-apps/findutils-4.4
>> --- Invalid atom in /etc/portage/package.use/explored-packages:
>> *>=sys-devel/patch-2.7
>> --- Invalid atom in /etc/portage/package.use/explored-packages:
>> /usr/portage/profiles/default/linux/packages
>> --- Invalid atom in /etc/portage/package.use/explored-packages:
>> /usr/portage/profiles/base/packages
>> --- Invalid atom in /etc/portage/package.use/explored-packages:
>> *>=sys-apps/baselayout-2
>> --- Invalid atom in /etc/portage/package.use/explored-packages:
>> *>=sys-apps/findutils-4.4
>> --- Invalid atom in /etc/portage/package.use/explored-packages:
>> *>=sys-devel/patch-2.7
>> --- Invalid atom in /etc/portage/package.use/explored-packages:
>> /usr/portage/profiles/default/linux/packages
>> [ Legend : U - final flag setting for installation]
>> [ : I - package is installed with flag ]
>> [ Colors : set, unset ]
>> * Found these USE flags for dev-libs/icu-60.2:
>> U I
>> + + abi_x86_32 : 32-bit (x86) libraries
>> - - debug : Enable extra debug codepaths, like asserts and extra
>> output. If
>> you want to get meaningful backtraces see
>>
>> https://wiki.gentoo.org/wiki/Project:Quality_Assurance/Backtraces
>> - - doc : Add extra documentation (API, Javadoc, etc). It is
>> recommended to
>> enable per package instead of globally
>> + + examples : Install examples, usually source code
>> - - static-libs : Build static versions of dynamic libraries as well
>>
>> Which begs the Q1} can I get rid of the flag icu? What are
>> consequences, as a baseline system flag, of it's removal ?
>>
>> less /usr/portage/profiles/base/packages
>> show me more of what the @system set contains. Very interesting and
>> useful. I'm thinking of aggregation of those listed packages
>> and some basic (ascii) table form (equery,emerge, eix) parsed listing
>> of the default and current flag settings. A "verification" tool
>> if you like. Surely it would help if this info was (is?) more readily
>> available and organized for folks that need a systematic approach, like
>> heterogeneous HPC clusters. The tools exist for 'ad-hoc' and one off,
>> but more of an organized representation at least at the set level.
>>
>> I feel like there is an existing tool that can yield all of this
>> information, as it is on a current system. I've read where there are
>> efforts to clean up the packages and default flags used in @system,
>> so the bare minimum list per arch/profiles would ultimately be
>> a useful listing, particular for my HPC. In HPC less is always faster
>> and better, as it is in security and so many more aspects of CS.
>>
>> Obviosly, I have a few things to fix on this (fragile) system, but
>> that'll happen as I'm at the beginning stages of auto_installs of
>> minimized systems. What are your plans for you little script?
>>
>> Just to match equery uses <flag> and such?
>>
>> Here's a cutie:
>> /usr/portage/profiles/default/linux/amd64/package.use.mask
>>
>> # Mike Frysinger <vapier@gentoo.org> (08 May 2016)
>> # This target supports VTV #547040.
>>> =sys-devel/gcc-4.9 -vtv
>>
>> # Mike Frysinger <vapier@gentoo.org> (21 Oct 2014)
>> # This target supports ASAN/etc... #504200.
>> sys-devel/gcc -sanitize
>>
>> And where was it that the processor/arch flags are now listed?
>>
>> tia,
>> James
>> cat
> 
> To check impact on negating icu on your system : `USE="-icu" emerge -puDU --with-bdeps=y world`
> 
> And concerning processor/arch flags I�d suggest keep exploring profiles, take a look at
> make.defaults files.
> Here the different files you can find according to PMS
> https://dev.gentoo.org/~ulm/pms/head/pms.html#x1-430005.2
> 
> --
> Corentin �Nado� Pazdera
> 
> 

Yea, it's now working::

$ ./profile-explorer.sh
[+] EROOT     : /
[+] PORTDIR   : /usr/portage
[+] CURPROFILE: default/linux/amd64/17.0
[+] EAPI      : 5

[+] packages (@system)
/usr/portage/profiles/base/packages
/usr/portage/profiles/default/linux/packages



So the explored-packages file looks complete (45)
for my current minimize workstation (lxde) amd64 .


So now parsing out these package listings with the current or default
flag settings; I'm going to have to marinate on that. I actually need
the reference (current default) flags once. But a method for dynamic
verification of gentoo systems that ask to join a cluster,  will have to
have these packages/flags verified to join (down the road issue).
amd64 is ok for now). Ideas on finding the default @system flag settings?

Maybe find a amd64-stage-3 to unpack and parse out that data?
Looking at the list of (45), the virtuals can be ignored, as there are
no flags to set? The exact version installed is not present in that
reported list. Some packages like gcc could be difficult with my
pedestrian sed|awk|sort|uniq skills with regular expressions.  Oh well
time for a review...

Would the  parsing 'metadata.xml'  per package be the easiest way to get
the total list of flags for a given package (ebuild) ?
Suggestions are welcome.

thx Corentin,



James