[gentoo-user] PORTAGE_NICENESS for real

[gentoo-user] PORTAGE_NICENESS for real

[Grant]

Is there a way to make the emerge command only use system resources
that are not requested by anything else?  Whenever I 'emerge sync' or
emerge a package my system slows way down and the sound sometimes
stutters badly.  I'd rather have the emerge process just move really
slowly if necessary.  I'm using PORTAGE_NICENESS="19" in
/etc/make.conf and it helps but not enough.

- Grant

-- 
gentoo-user@gentoo.org mailing list

Re: [gentoo-user] PORTAGE_NICENESS for real

[Iain Buchanan]

On Tue, 2006-04-04 at 17:13 -0700, Grant wrote:
> I'm using PORTAGE_NICENESS="19" in
> /etc/make.conf and it helps but not enough.

meetoo!  I've noticed it helps with cpu allocation, but not with the
disk.  If you're emerging something like sources, or openoffice, portage
still gets too much disk "time" and other processes suffer.  nice
doesn't seem to affect disk access.
-- 
Iain Buchanan <iaindb at netspace dot net dot au>

The naked truth of it is, I have no shirt.
		-- William Shakespeare, "Love's Labour's Lost"

-- 
gentoo-user@gentoo.org mailing list

Re: [gentoo-user] PORTAGE_NICENESS for real

[Roy Wright]

Iain Buchanan wrote:
> On Tue, 2006-04-04 at 17:13 -0700, Grant wrote:
>   
>> I'm using PORTAGE_NICENESS="19" in
>> /etc/make.conf and it helps but not enough.
>>     
>
> meetoo!  I've noticed it helps with cpu allocation, but not with the
> disk.  If you're emerging something like sources, or openoffice, portage
> still gets too much disk "time" and other processes suffer.  nice
> doesn't seem to affect disk access.
>   
What I do is run emerge ---sync every night as a cron job.  Then during 
the day I run
"nice emerge -uDNav world" usually when reading email.  CPU load maxes 
out at around
50%.

HTH,
Roy

-- 
gentoo-user@gentoo.org mailing list

Re: [gentoo-user] PORTAGE_NICENESS for real

[Iain Buchanan]

On Tue, 2006-04-04 at 20:18 -0500, Roy Wright wrote:

>  during 
> the day I run
> "nice emerge -uDNav world" usually when reading email.  CPU load maxes 
> out at around
> 50%.

this is strange - CPU should still go to 100% (if it would without the
nice command that is).  Nice only affects which processes get the cpu
time first - it doesn't apply a "maximum cpu limit" to a process, so if
you have only one process running, even niced to 19, it can still use
100% cpu...

It would be nice if there was a nice command for making disk access nice
so I could nicely say `nice emerge -uD world` and it wouldn't take a
whole lot of disk time away from my nice browsing, music playing, etc...

eh... 
-- 
Iain Buchanan <iaindb at netspace dot net dot au>

Unix will self-destruct in five seconds... 4... 3... 2... 1...

-- 
gentoo-user@gentoo.org mailing list

Re: [gentoo-user] PORTAGE_NICENESS for real

[Frédéric Grosshans]

Le mercredi 05 avril 2006 à 11:21 +0930, Iain Buchanan a écrit :
> On Tue, 2006-04-04 at 20:18 -0500, Roy Wright wrote:

> It would be nice if there was a nice command for making disk access nice
> so I could nicely say `nice emerge -uD world` and it wouldn't take a
> whole lot of disk time away from my nice browsing, music playing, etc...

I think /usr/src/linux/Documentation/block/ioprio.txt is your friend.
(I'll read it when (if?) I'll have the time, so I can't sum it up for
you !)

 	Fred



-- 
gentoo-user@gentoo.org mailing list

Re: [gentoo-user] PORTAGE_NICENESS for real

[Frédéric Grosshans]

Le mercredi 05 avril 2006 à 12:07 +0200, Frédéric Grosshans a écrit :
> Le mercredi 05 avril 2006 à 11:21 +0930, Iain Buchanan a écrit :
> > On Tue, 2006-04-04 at 20:18 -0500, Roy Wright wrote:
> 
> > It would be nice if there was a nice command for making disk access nice
> > so I could nicely say `nice emerge -uD world` and it wouldn't take a
> > whole lot of disk time away from my nice browsing, music playing, etc...
> 
> I think /usr/src/linux/Documentation/block/ioprio.txt is your friend.
> (I'll read it when (if?) I'll have the time, so I can't sum it up for
> you !)

OK, I've read it.
If you configure the CFQ-IOscheduler in your kernel, you can write an
ionice tool (source code given) which is like nice, but for the disk.
Furthermore, there exist a IOPRIO_CLASS_IDLE priority class which is
exactly what we are looking for (no disk access if any other process
wants a disk access).

Is there a plan to use those IOPRIO classes in Portage ? It could be
very useful for laptops (with very slow disks).

BTW, is there a plan to use the new SCHED_BATCH scheduling policy
introduced in the last (2.6.16) kernel ? (see
http://www.kernel.org/git/?p=linux/kernel/git/torvalds/linux-2.6.git;a=commit;h=b0a9499c3dd50d333e2aedb7e894873c58da3785 )


	Fred



-- 
gentoo-user@gentoo.org mailing list

Re: [gentoo-user] PORTAGE_NICENESS for real

[Grant]

> > > It would be nice if there was a nice command for making disk access nice
> > > so I could nicely say `nice emerge -uD world` and it wouldn't take a
> > > whole lot of disk time away from my nice browsing, music playing, etc...
> >
> > I think /usr/src/linux/Documentation/block/ioprio.txt is your friend.
> > (I'll read it when (if?) I'll have the time, so I can't sum it up for
> > you !)
>
> OK, I've read it.
> If you configure the CFQ-IOscheduler in your kernel, you can write an
> ionice tool (source code given) which is like nice, but for the disk.
> Furthermore, there exist a IOPRIO_CLASS_IDLE priority class which is
> exactly what we are looking for (no disk access if any other process
> wants a disk access).
>
> Is there a plan to use those IOPRIO classes in Portage ? It could be
> very useful for laptops (with very slow disks).
>
> BTW, is there a plan to use the new SCHED_BATCH scheduling policy
> introduced in the last (2.6.16) kernel ? (see
> http://www.kernel.org/git/?p=linux/kernel/git/torvalds/linux-2.6.git;a=commit;h=b0a9499c3dd50d333e2aedb7e894873c58da3785 )
>
>
>         Fred

So those tools aren't useful for this problem until they are
integrated into portage?

- Grant

-- 
gentoo-user@gentoo.org mailing list

Re: [gentoo-user] PORTAGE_NICENESS for real

[Vladimir G. Ivanovic]

Con Kolivas's kernel

        http://members.optusnet.com.au/ckolivas/kernel/
        
or

        emerge ck-sources
        
use Kolivas' toolsched (http://ck.kolivas.org/apps/toolsched/) so that
background jobs only run when no other task wants to run. toolsched uses
sys-process/schedtool to control the scheduling of processes.

--- Vladimir

On Wed, 2006-04-05 at 09:49 -0700, Grant wrote:

> > BTW, is there a plan to use the new SCHED_BATCH scheduling policy
> > introduced in the last (2.6.16) kernel ? (see
> > http://www.kernel.org/git/?p=linux/kernel/git/torvalds/linux-2.6.git;a=commit;h=b0a9499c3dd50d333e2aedb7e894873c58da3785 )


Vladimir G. Ivanovic
Palo Alto, CA 94306
+1 650 678 8014

-- 
gentoo-user@gentoo.org mailing list

Re: [gentoo-user] PORTAGE_NICENESS for real

[Glenn Enright]

On Wednesday 05 April 2006 12:56, Iain Buchanan wrote:
> meetoo!  I've noticed it helps with cpu allocation, but not with the
> disk.  If you're emerging something like sources, or openoffice, portage
> still gets too much disk "time" and other processes suffer.  nice
> doesn't seem to affect disk access.

This is surely very dependant on you specific hardware and kernel 
configuration. There are some systems that just dont 'multitask' very 
effectively. Also, sounds like you have either an older HDD or your 
motherboard is struggling to keep up with the I/O requirements of compiling. 
Remeber that compiling is non-trivial work for *any* system.

Also check that your kernel is making full use of any features available on 
you system such as SMP or Hyperthreading etc. As a point of comparison, I'm 
running a P4 2.6 on an IS7 motherboard with SATA disks, and only under the 
heaviest compilations, for short periods of time do I notice any slowdowns.

-- 
About the only thing on a farm that has an easy time is the dog.
-- 
gentoo-user@gentoo.org mailing list

Re: [gentoo-user] PORTAGE_NICENESS for real

[Iain Buchanan]

On Wed, 2006-04-05 at 14:00 +1200, Glenn Enright wrote:
> On Wednesday 05 April 2006 12:56, Iain Buchanan wrote:
> > meetoo!  I've noticed it helps with cpu allocation, but not with the
> > disk.  If you're emerging something like sources, or openoffice, portage
> > still gets too much disk "time" and other processes suffer.  nice
> > doesn't seem to affect disk access.
> 
> This is surely very dependant on you specific hardware and kernel 
> configuration. There are some systems that just dont 'multitask' very 
> effectively. Also, sounds like you have either an older HDD or your 
> motherboard is struggling to keep up with the I/O requirements of compiling. 
> Remeber that compiling is non-trivial work for *any* system.

weeeelll, it's not exactly old - it's a 3.0GHz P4 laptop (Inspiron
9100).  It could be a RAM issue - only 512 Mb.

But I definately notice it when emerging sources, and then trying to
switch virtual workspaces when I have, say, firefox & evolution & vmware
running...  even if emerge is niced.

> Also check that your kernel is making full use of any features available on 
> you system such as SMP or Hyperthreading etc. As a point of comparison, I'm 
> running a P4 2.6 on an IS7 motherboard with SATA disks, and only under the 
> heaviest compilations, for short periods of time do I notice any slowdowns.

but do you actually do anything to notice this slowdown?  ie. are you
changing desktops, starting other applications, etc?

thanks for the tips.  I'll check out the ioprio someone else
mentioned...
-- 
Iain Buchanan <iaindb at netspace dot net dot au>

It takes a smart husband to have the last word and not use it.

-- 
gentoo-user@gentoo.org mailing list

Re: [gentoo-user] PORTAGE_NICENESS for real

[Richard Fish]

On 4/5/06, Iain Buchanan <iaindb@netspace.net.au> wrote:
> weeeelll, it's not exactly old - it's a 3.0GHz P4 laptop (Inspiron
> 9100).  It could be a RAM issue - only 512 Mb.

More memory would definitely help.  I have 2G in my laptop, and it is
much more responsive.

Extracted kernel sources consume about 300M of disk space, so the
system will try to use around 600M of cache (300M for reading, 300M
for writing) during the install phase of the kernel....and throw all
other cached pages away, including memory mapped files.  This is
almost certainly what causes the delay in switching desktops

-Richard

-- 
gentoo-user@gentoo.org mailing list

Re: [gentoo-user] PORTAGE_NICENESS for real

[Glenn Enright]

On Thursday 06 April 2006 11:08, Iain Buchanan wrote:
> weeeelll, it's not exactly old - it's a 3.0GHz P4 laptop (Inspiron
> 9100).  It could be a RAM issue - only 512 Mb.
>
> But I definately notice it when emerging sources, and then trying to
> switch virtual workspaces when I have, say, firefox & evolution & vmware
> running...  even if emerge is niced.
>
what are the specs on your HDD? surely seems like the key here. Like Richard 
pointed out, as soon as IO systems become loaded they struggle to keep up 
with demand. Laptops usually have 5400rpm HDD dont they?

> but do you actually do anything to notice this slowdown?  ie. are you
> changing desktops, starting other applications, etc?
>
absolutely, running openoffice, juk, kate, kmail, firefox and gkrellm atm on 
four desktops amongst others. Emergeing at the same time, Firefox can grind 
on the rendering but otherwise its pretty sweet. BTW I only have 512 MB RAM 
also. 

OT my desktop stystem generates significant heat, often often over 35c in the 
case alone, but laptops can't get away with disipating as much heat so there 
must be tradeoffs somewhere. Actually Im impressed that its still going after 
3 years and the amount of work it does. :)

> thanks for the tips.  I'll check out the ioprio someone else
> mentioned...

Yeah looks keen as :)

-- 
panic("esp_handle: current_SC == penguin within interrupt!");
        linux-2.2.16/drivers/scsi/esp.c
-- 
gentoo-user@gentoo.org mailing list

Re: [gentoo-user] PORTAGE_NICENESS for real

[Iain Buchanan]

On Thu, 2006-04-06 at 14:57 +1200, Glenn Enright wrote:
> On Thursday 06 April 2006 11:08, Iain Buchanan wrote:
> > weeeelll, it's not exactly old - it's a 3.0GHz P4 laptop (Inspiron
> > 9100).  It could be a RAM issue - only 512 Mb.
> >
> > But I definately notice it when emerging sources, and then trying to
> > switch virtual workspaces when I have, say, firefox & evolution & vmware
> > running...  even if emerge is niced.
> >
> what are the specs on your HDD? surely seems like the key here. Like Richard 
> pointed out, as soon as IO systems become loaded they struggle to keep up 
> with demand. Laptops usually have 5400rpm HDD dont they?

Just put in a 7200rpm 8Mb cache 100Gb Seagate :)  It's faster in hdparm
tests than my 5400, but I still get a bit of 'slowdown' when emerging...

-- 
Iain Buchanan <iaindb at netspace dot net dot au>

What's all this brouhaha?

-- 
gentoo-user@gentoo.org mailing list

Re: [gentoo-user] PORTAGE_NICENESS for real

[Hemmann, Volker Armin]

On Wednesday 05 April 2006 02:13, Grant wrote:
> Is there a way to make the emerge command only use system resources
> that are not requested by anything else?  Whenever I 'emerge sync' or
> emerge a package my system slows way down and the sound sometimes
> stutters badly.  I'd rather have the emerge process just move really
> slowly if necessary.  I'm using PORTAGE_NICENESS="19" in
> /etc/make.conf and it helps but not enough.
>
> - Grant

I don't know about PORTAGE_NICENESS, but MAKEOPTS="-j1" helped me a lot.
-- 
gentoo-user@gentoo.org mailing list