[gentoo-dev] [RFC] check-reqs.eclass: fail check-reqs_memory() for virtual rather than physical RAM

[gentoo-dev] [RFC] check-reqs.eclass: fail check-reqs_memory() for virtual rather than physical RAM

[Christian Bricart]

Dear devs,

i've just hit the constraint CHECKREQS_MEMORY=3G in
>=www-client/chromium-44.* on a 2 GiB RAM machine.. (having additionally
5G Swap on SSD..)

I personally doubt that check-reqs_memory() in the eclass should only
check for *physical* rather than *virtual* RAM to fulfill this
constraint and fail if unsatisfied.

So my proposal would rather be:

* $CHECKREQS_MEMORY<=pysical RAM -> ok
* $CHECKREQS_MEMORY>pysical RAM && <= virtual RAM -> ewarn
* $CHECKREQS_MEMORY>virtual RAM -> eerror (may still be overridden by
${I_KNOW_WHAT_I_AM_DOING} in check-reqs_output() )

what do you think?

cheers
  Christian

Re: [gentoo-dev] [RFC] check-reqs.eclass: fail check-reqs_memory() for virtual rather than physical RAM

[Ciaran McCreesh]

[-- Attachment #1: Type: text/plain, Size: 479 bytes --]

On Wed, 03 Jun 2015 22:22:39 +0200
> i've just hit the constraint CHECKREQS_MEMORY=3G in
> >=www-client/chromium-44.* on a 2 GiB RAM machine.. (having
> >additionally
> 5G Swap on SSD..)
> 
> I personally doubt that check-reqs_memory() in the eclass should only
> check for *physical* rather than *virtual* RAM to fulfill this
> constraint and fail if unsatisfied.

Swap is horrifically slow. It's better to fail than to use swap for
stuff...

-- 
Ciaran McCreesh

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 181 bytes --]

Re: [gentoo-dev] [RFC] check-reqs.eclass: fail check-reqs_memory() for virtual rather than physical RAM

[Christian Bricart]

[-- Attachment #1: Type: text/plain, Size: 777 bytes --]

Am 03.06.2015 um 22:32 schrieb Ciaran McCreesh:
> On Wed, 03 Jun 2015 22:22:39 +0200
>> i've just hit the constraint CHECKREQS_MEMORY=3G in
>>> =www-client/chromium-44.* on a 2 GiB RAM machine.. (having
>>> additionally
>> 5G Swap on SSD..)
>>
>> I personally doubt that check-reqs_memory() in the eclass should only
>> check for *physical* rather than *virtual* RAM to fulfill this
>> constraint and fail if unsatisfied.
> 
> Swap is horrifically slow. It's better to fail than to use swap for
> stuff...
> 
(basically) agreed - but in distcc/icecream/whatever builds, linking
*may* also be done on remote machines, which *may* fulfill the memory
constraint..
we're talking about *build* constraints rather than *run* constraints,
right..?

Christian



[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 181 bytes --]

Re: [gentoo-dev] [RFC] check-reqs.eclass: fail check-reqs_memory() for virtual rather than physical RAM

[Michał Górny]

[-- Attachment #1: Type: text/plain, Size: 1022 bytes --]

Dnia 2015-06-03, o godz. 22:39:10
Christian Bricart <christian@bricart.de> napisał(a):

> Am 03.06.2015 um 22:32 schrieb Ciaran McCreesh:
> > On Wed, 03 Jun 2015 22:22:39 +0200
> >> i've just hit the constraint CHECKREQS_MEMORY=3G in
> >>> =www-client/chromium-44.* on a 2 GiB RAM machine.. (having
> >>> additionally
> >> 5G Swap on SSD..)
> >>
> >> I personally doubt that check-reqs_memory() in the eclass should only
> >> check for *physical* rather than *virtual* RAM to fulfill this
> >> constraint and fail if unsatisfied.
> > 
> > Swap is horrifically slow. It's better to fail than to use swap for
> > stuff...
> > 
> (basically) agreed - but in distcc/icecream/whatever builds, linking
> *may* also be done on remote machines, which *may* fulfill the memory
> constraint..

Nay, linking is always done locally.

And even if you were right, this isn't a lottery. It ain't 'let's let
user dumbly waste 4 hours, maybe it will link for him this time'.

-- 
Best regards,
Michał Górny

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 949 bytes --]

Re: [gentoo-dev] [RFC] check-reqs.eclass: fail check-reqs_memory() for virtual rather than physical RAM

[Mike Gilbert]

On Wed, Jun 3, 2015 at 4:22 PM, Christian Bricart <christian@bricart.de> wrote:
> Dear devs,
>
> i've just hit the constraint CHECKREQS_MEMORY=3G in
>>=www-client/chromium-44.* on a 2 GiB RAM machine.. (having additionally
> 5G Swap on SSD..)
>
> I personally doubt that check-reqs_memory() in the eclass should only
> check for *physical* rather than *virtual* RAM to fulfill this
> constraint and fail if unsatisfied.
>
> So my proposal would rather be:
>
> * $CHECKREQS_MEMORY<=pysical RAM -> ok
> * $CHECKREQS_MEMORY>pysical RAM && <= virtual RAM -> ewarn
> * $CHECKREQS_MEMORY>virtual RAM -> eerror (may still be overridden by
> ${I_KNOW_WHAT_I_AM_DOING} in check-reqs_output() )
>
> what do you think?
>

The chromium build issue is a point of some contention; see the bug below.

https://bugs.gentoo.org/show_bug.cgi?id=471810

I agree that it makes sense to check virtual memory. I guess that
would be MemTotal + SwapTotal in /proc/meminfo.

It would also probably make more sense to check available memory
(MemAvailable + SwapFree). Maybe that should be implemented in a new
function, like check-reqs_memory_available.

I don't like your weird ewarn/eerror proposal. We should either fail,
or not fail in check-reqs_pkg_pretend and check-reqs_pkg_setup. And
the helper functions should continue to return a non-zero status on
failure.

Re: [gentoo-dev] [RFC] check-reqs.eclass: fail check-reqs_memory() for virtual rather than physical RAM

[Diamond]

On Wed, 3 Jun 2015 21:32:34 +0100
Ciaran McCreesh <ciaran.mccreesh@googlemail.com> wrote:

> 
> Swap is horrifically slow. It's better to fail than to use swap for
> stuff...
> 

Is it better to fail when you have already lost several hours in
compilations? Or is it better to use some additional parallel swap on a
bunch of HDDs or SDDs to FINISH emerge during 30 minutes more? I think
the latter is better than fail for no reason. BTW, RAM + swap is better
than swap only.

Re: [gentoo-dev] [RFC] check-reqs.eclass: fail check-reqs_memory() for virtual rather than physical RAM

["Paweł Hajdan, Jr."]

[-- Attachment #1: Type: text/plain, Size: 786 bytes --]

On 6/3/15 10:56 PM, Mike Gilbert wrote:
> The chromium build issue is a point of some contention; see the bug below.
> 
> https://bugs.gentoo.org/show_bug.cgi?id=471810
> 
> I agree that it makes sense to check virtual memory. I guess that
> would be MemTotal + SwapTotal in /proc/meminfo.
> 
> It would also probably make more sense to check available memory
> (MemAvailable + SwapFree). Maybe that should be implemented in a new
> function, like check-reqs_memory_available.

Agreed.

I think one of the issues is that the eclass aims to support multiple
kernels, not just Linux.

It's not obvious how to implement such check portably for non-Linux
systems. Maybe we could keep it just Linux-only.

I'm totally open to suggestions from the community.

Paweł


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 841 bytes --]