[gentoo-user] Change MAKEOPTS on the fly?

[gentoo-user] Change MAKEOPTS on the fly?

[William Kenworthy]

Hi,

    is there a way to change the MAKEOPTS setting on a running emerge? 
I am using "-j 5 -l 4" whilst emerging gcc-9.3 but its creating too much
pressure  on memory.  I expect the emerge to take many more hours but
complete eventually - but reducing it to "-j2" will help other
operations whilst not losing whats already been completed (this is an
old atom N330 with 4G ram and is my gateway/router/firewall/snort/...
and the overload is starting to affect the network throughput
significantly).


BillK

[gentoo-user] Re: Change MAKEOPTS on the fly?

[Nikos Chantziaras]

On 12/07/2020 09:04, William Kenworthy wrote:
> Hi,
> 
>      is there a way to change the MAKEOPTS setting on a running emerge?
> I am using "-j 5 -l 4" whilst emerging gcc-9.3 but its creating too much
> pressure  on memory.  I expect the emerge to take many more hours but
> complete eventually - but reducing it to "-j2" will help other
> operations whilst not losing whats already been completed (this is an
> old atom N330 with 4G ram and is my gateway/router/firewall/snort/...
> and the overload is starting to affect the network throughput
> significantly).

No. But what you can do is lower its nice level to 19, and CPU and IO 
priority to "idle". First, find the process IDs of emerge and make:

   ps aux | grep emerge
   ps aux | grep make

The first number after the user name (which is "root") is the pid. Then 
do the following for both pids:

   schedtool -D -n 19 pid
   ionice -c 3 -p pid

ionice is in sys-apps/util-linux, so it's probably already installed. 
schedtool though is in sys-process/schedtool and it might not be 
installed. Nothing you can do about that right now. You have to wait it 
out. ionice should help a bit though.

In the future, I *highly* recommend installing schedtool, and then put 
this in your make.conf:

   PORTAGE_NICENESS=19
   PORTAGE_IONICE_COMMAND="sh -c \"schedtool -D \${PID}; ionice -c 3 -p 
\${PID}\""

I have used this for many years now. It makes emerge have a virtually 
imperceptible impact on my system. I can emerge for example gcc or 
libreoffice with -j4 on my 4 cores/4 threads CPU, and I feel no slowdown 
at all. This won't help with running out of RAM obviously, but it helps 
immensely with keeping the system highly responsive.

Another thing I recommend is getting rid of "-j5". Use -j4. The "+1" 
recommendation from decades ago does not apply anymore with modern Linux 
kernels. You can test this yourself by emerging a smaller package that 
takes like 2 minutes or so to emerge, and compare times with j4 and j5. 
Most likely you will see no difference, other than j5 using more RAM.

Re: [gentoo-user] Re: Change MAKEOPTS on the fly?

[tastytea]

On 2020-07-12 11:29+0300 Nikos Chantziaras <realnc@gmail.com> wrote:

> On 12/07/2020 09:04, William Kenworthy wrote:
> > Hi,
> > 
> >      is there a way to change the MAKEOPTS setting on a running
> > emerge? I am using "-j 5 -l 4" whilst emerging gcc-9.3 but its
> > creating too much pressure  on memory.  I expect the emerge to take
> > many more hours but complete eventually - but reducing it to "-j2"
> > will help other operations whilst not losing whats already been
> > completed (this is an old atom N330 with 4G ram and is my
> > gateway/router/firewall/snort/... and the overload is starting to
> > affect the network throughput significantly).  
> 
> […]
> ionice is in sys-apps/util-linux, so it's probably already installed. 
> schedtool though is in sys-process/schedtool and it might not be 
> installed. 
> […]

The nice level can also be adjusted with `renice` from
sys-apps/util-linux:

    renice -n 19 -p pid

> Another thing I recommend is getting rid of "-j5". Use -j4. The "+1" 
> recommendation from decades ago does not apply anymore with modern
> Linux kernels. You can test this yourself by emerging a smaller
> package that takes like 2 minutes or so to emerge, and compare times
> with j4 and j5. Most likely you will see no difference, other than j5
> using more RAM.

You can also set MAKEOPTS and other Portage variables per package, for
example:

    echo 'MAKEOPTS="-j1"' >> /etc/portage/env/onejob.conf
    echo 'sys-devel/gcc onejob.conf' >> /etc/portage/package.env/onejob

See <https://wiki.gentoo.org/wiki//etc/portage/package.env> for more
information.

[gentoo-user] Re: Change MAKEOPTS on the fly?

[Nikos Chantziaras]

On 12/07/2020 11:54, tastytea wrote:
> On 2020-07-12 11:29+0300 Nikos Chantziaras <realnc@gmail.com> wrote:
> 
>> On 12/07/2020 09:04, William Kenworthy wrote:
>>> Hi,
>>>
>>>       is there a way to change the MAKEOPTS setting on a running
>>> emerge? I am using "-j 5 -l 4" whilst emerging gcc-9.3 but its
>>> creating too much pressure  on memory.  I expect the emerge to take
>>> many more hours but complete eventually - but reducing it to "-j2"
>>> will help other operations whilst not losing whats already been
>>> completed (this is an old atom N330 with 4G ram and is my
>>> gateway/router/firewall/snort/... and the overload is starting to
>>> affect the network throughput significantly).
>>
>> […]
>> ionice is in sys-apps/util-linux, so it's probably already installed.
>> schedtool though is in sys-process/schedtool and it might not be
>> installed.
>> […]
> 
> The nice level can also be adjusted with `renice` from
> sys-apps/util-linux:
> 
>      renice -n 19 -p pid

Yep, forgot about that. However, I have observed that the nice level 
doesn't have much impact. But setting a process to SCHED_IDLEPRIO 
(schedtool -D) and nice level 19 (schedtool -n 19) has a huge impact.

Re: [gentoo-user] Re: Change MAKEOPTS on the fly?

[Michael]

[-- Attachment #1: Type: text/plain, Size: 2758 bytes --]

On Sunday, 12 July 2020 09:29:08 BST Nikos Chantziaras wrote:
> On 12/07/2020 09:04, William Kenworthy wrote:
> > Hi,
> > 
> >      is there a way to change the MAKEOPTS setting on a running emerge?
> > 
> > I am using "-j 5 -l 4" whilst emerging gcc-9.3 but its creating too much
> > pressure  on memory.  I expect the emerge to take many more hours but
> > complete eventually - but reducing it to "-j2" will help other
> > operations whilst not losing whats already been completed (this is an
> > old atom N330 with 4G ram and is my gateway/router/firewall/snort/...
> > and the overload is starting to affect the network throughput
> > significantly).
> 
> No. But what you can do is lower its nice level to 19, and CPU and IO
> priority to "idle". First, find the process IDs of emerge and make:
> 
>    ps aux | grep emerge
>    ps aux | grep make
> 
> The first number after the user name (which is "root") is the pid. Then
> do the following for both pids:
> 
>    schedtool -D -n 19 pid
>    ionice -c 3 -p pid
> 
> ionice is in sys-apps/util-linux, so it's probably already installed.
> schedtool though is in sys-process/schedtool and it might not be
> installed. Nothing you can do about that right now. You have to wait it
> out. ionice should help a bit though.
> 
> In the future, I *highly* recommend installing schedtool, and then put
> this in your make.conf:
> 
>    PORTAGE_NICENESS=19
>    PORTAGE_IONICE_COMMAND="sh -c \"schedtool -D \${PID}; ionice -c 3 -p
> \${PID}\""
> 
> I have used this for many years now. It makes emerge have a virtually
> imperceptible impact on my system. I can emerge for example gcc or
> libreoffice with -j4 on my 4 cores/4 threads CPU, and I feel no slowdown
> at all. This won't help with running out of RAM obviously, but it helps
> immensely with keeping the system highly responsive.

Another trick to use if the atom is becoming I/O disk bound is:

echo bfq > /sys/block/sda/queue/scheduler

This will have more of an impact if the PC is swapping heavily and the I/O on 
/dev/sda is choking other processes accessing the disk.


> Another thing I recommend is getting rid of "-j5". Use -j4. The "+1"
> recommendation from decades ago does not apply anymore with modern Linux
> kernels. You can test this yourself by emerging a smaller package that
> takes like 2 minutes or so to emerge, and compare times with j4 and j5.
> Most likely you will see no difference, other than j5 using more RAM.

On an older PC with 16G RAM I have noticed the +1 giving marginally faster 
results each time.  I tried repeated compiles of ffmpeg and n+1 or 2(n+1) was 
faster, as long as RAM was not exhausted.  On larger packages/less RAM, 
limiting MAKEOPTS individually would be advisable to avoid William's problem.

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

[gentoo-user] Re: Change MAKEOPTS on the fly?

[Nikos Chantziaras]

On 12/07/2020 11:59, Michael wrote:
> On Sunday, 12 July 2020 09:29:08 BST Nikos Chantziaras wrote:
>> No. But what you can do is lower its nice level to 19, and CPU and IO
>> priority to "idle".
>>
>>     schedtool -D -n 19 pid
>>     ionice -c 3 -p pid
> 
> Another trick to use if the atom is becoming I/O disk bound is:
> 
> echo bfq > /sys/block/sda/queue/scheduler
> 
> This will have more of an impact if the PC is swapping heavily and the I/O on
> /dev/sda is choking other processes accessing the disk.

bfq seems to help a bit (although not as much as some years ago, when 
bfq was an actual disk scheduler rather than just a scheduling policy 
tweak.)

I have bfq enabled by default for everything by putting the following in 
/etc/udev/rules.d/20-block.rules:

   ACTION=="add|change", SUBSYSTEM=="block", ATTR{queue/scheduler}="bfq"

This will use bfq for all storage (including storage devices plugged in 
at runtime, like USB disks.)

Re: [gentoo-user] Re: Change MAKEOPTS on the fly?

[William Kenworthy]

On 12/7/20 6:03 pm, Nikos Chantziaras wrote:
> echo bfq > /sys/block/sda/queue/schedule


Thanks for the hints,

    ive gone with schedtool and ionice for now (seems to be working) and
will configure that as the defaults when this run finishes.  I have not
built the bfq scheduler in this kernel so will give that a try later -
its currently using mq-deadline with a WD blue SSD (will bfq be better
than a deadline on an ssd? - will try it and see). It has 4g ram and 4g
swap with about half in use - in normal running swap isn't used much, if
at all.

As well as emerge, the io load is also coming from the network as I have
3x1g bonded interfaces routing busy vlans including a moosefs SAN and a
100mhz uplink which was suffering delays and timeouts.  The root cause
is trying to do too much with too little - :)

BillK

Re: [gentoo-user] Re: Change MAKEOPTS on the fly?

[Michael]

[-- Attachment #1: Type: text/plain, Size: 1278 bytes --]

On Sunday, 12 July 2020 12:38:22 BST William Kenworthy wrote:
> On 12/7/20 6:03 pm, Nikos Chantziaras wrote:
> > echo bfq > /sys/block/sda/queue/schedule
> 
> Thanks for the hints,
> 
>     ive gone with schedtool and ionice for now (seems to be working) and
> will configure that as the defaults when this run finishes.  I have not
> built the bfq scheduler in this kernel so will give that a try later -
> its currently using mq-deadline with a WD blue SSD (will bfq be better
> than a deadline on an ssd? - will try it and see). It has 4g ram and 4g
> swap with about half in use - in normal running swap isn't used much, if
> at all.

BFQ manages the I/O pipe more effectively/efficiently, so if the pipe gets 
full due to e.g. heavy swapping, you'll see an improvement.  However, with an 
SSD the improvement will be less noticeable than with a spinning disk and with 
an NVMe even less.


> As well as emerge, the io load is also coming from the network as I have
> 3x1g bonded interfaces routing busy vlans including a moosefs SAN and a
> 100mhz uplink which was suffering delays and timeouts.  The root cause
> is trying to do too much with too little - :)
> 
> BillK

Network storage will be managed by the remote kernel, so your atom's scheduler 
won't help with that.

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [gentoo-user] Re: Change MAKEOPTS on the fly?

[Matt Connell (Gmail)]

On Sun, 2020-07-12 at 11:29 +0300, Nikos Chantziaras wrote:
> Another thing I recommend is getting rid of "-j5". Use -j4. The "+1" 
> recommendation from decades ago does not apply anymore with modern Linux 
> kernels. You can test this yourself by emerging a smaller package that 
> takes like 2 minutes or so to emerge, and compare times with j4 and j5. 
> Most likely you will see no difference, other than j5 using more RAM.

I can confirm this as well.  On RAM-limited systems (like this laptop
with 8 logical cores and only 8GB of memory...) I have to use "-j2" in
order to keep the machine usable while compiling.

Re: [gentoo-user] Re: Change MAKEOPTS on the fly?

[Neil Bothwick]

[-- Attachment #1: Type: text/plain, Size: 1022 bytes --]

On Mon, 13 Jul 2020 11:10:01 -0500, Matt Connell (Gmail) wrote:

> > Another thing I recommend is getting rid of "-j5". Use -j4. The "+1" 
> > recommendation from decades ago does not apply anymore with modern
> > Linux kernels. You can test this yourself by emerging a smaller
> > package that takes like 2 minutes or so to emerge, and compare times
> > with j4 and j5. Most likely you will see no difference, other than j5
> > using more RAM.  
> 
> I can confirm this as well.  On RAM-limited systems (like this laptop
> with 8 logical cores and only 8GB of memory...) I have to use "-j2" in
> order to keep the machine usable while compiling.


You have to consider the --jobs option passed to emerge at the same time.
It's no use limiting each emerge to 2 processes if you then run multiple
emerges in parallel. Setting --load in MAKEOPTS and --load-average in
EMERGE_DEFAULT_OPTS is worthwhile on a constrained system.


-- 
Neil Bothwick

Knock firmly but softly. I like soft firm knockers.

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [gentoo-user] Re: Change MAKEOPTS on the fly?

[Matt Connell (Gmail)]

On Mon, 2020-07-13 at 17:33 +0100, Neil Bothwick wrote:
> You have to consider the --jobs option passed to emerge at the same time.
> It's no use limiting each emerge to 2 processes if you then run multiple
> emerges in parallel.

According to the man page, this defaults to 1 if emerging
interactively, which I always do.

I'll try setting the other options sometime as well, thank you.

Re: [gentoo-user] Re: Change MAKEOPTS on the fly?

[Neil Bothwick]

[-- Attachment #1: Type: text/plain, Size: 582 bytes --]

On Mon, 13 Jul 2020 11:50:08 -0500, Matt Connell (Gmail) wrote:

> > You have to consider the --jobs option passed to emerge at the same
> > time. It's no use limiting each emerge to 2 processes if you then run
> > multiple emerges in parallel.  
> 
> According to the man page, this defaults to 1 if emerging
> interactively, which I always do.

It does, but that's a very inefficient use of resources as a single emerge
often spends a lot of its time not hammering the CPU.


-- 
Neil Bothwick

"Bad dog! Leave that wire alone.....click.....###@*##....NO TERRIER

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [gentoo-user] Re: Change MAKEOPTS on the fly?

[Sam Jorna (wraeth)]

[-- Attachment #1.1: Type: text/plain, Size: 786 bytes --]

On 14/7/20 2:33 am, Neil Bothwick wrote:
> You have to consider the --jobs option passed to emerge at the same time.
> It's no use limiting each emerge to 2 processes if you then run multiple
> emerges in parallel. Setting --load in MAKEOPTS and --load-average in
> EMERGE_DEFAULT_OPTS is worthwhile on a constrained system.

It's also worth pointing out that emerge's --load-average is only
evaluated when it's looking to start building a new package/job, as
opposed to MAKEOPTS' (ie. make's) --load which is evaluated every time
make wants to spawn a new thread. If your load average drops below the
threshold, emerge will start a new job with potentially -jN make threads
(though this is mitigated somewhat if also using --load in MAKEOPTS).

-- 
Sam Jorna (wraeth)


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

Re: [gentoo-user] Re: Change MAKEOPTS on the fly?

[Sam Jorna (wraeth)]

[-- Attachment #1.1: Type: text/plain, Size: 980 bytes --]

On 14/7/20 9:34 am, Sam Jorna (wraeth) wrote:
> On 14/7/20 2:33 am, Neil Bothwick wrote:
>> You have to consider the --jobs option passed to emerge at the same time.
>> It's no use limiting each emerge to 2 processes if you then run multiple
>> emerges in parallel. Setting --load in MAKEOPTS and --load-average in
>> EMERGE_DEFAULT_OPTS is worthwhile on a constrained system.
> 
> It's also worth pointing out that emerge's --load-average is only
> evaluated when it's looking to start building a new package/job, as
> opposed to MAKEOPTS' (ie. make's) --load which is evaluated every time
> make wants to spawn a new thread. If your load average drops below the
> threshold, emerge will start a new job with potentially -jN make threads
> (though this is mitigated somewhat if also using --load in MAKEOPTS).

Re-sending with (what should now be the) correct GPG signature.

Sorry, new mail client, and it's trying to be "helpful".

-- 
Sam Jorna (wraeth)


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 833 bytes --]