[gentoo-user] Looong delays

[gentoo-user] Looong delays

[Dirk Uys]

[-- Attachment #1: Type: text/plain, Size: 958 bytes --]

Hi

This has been bothering me for some time now. I have a Dell PC at work,
Intel Core2 Duo with 4gb ram etc. Whenever something does a lot of disk
access, the PC slows down to a halt? I remember some issue between Firefox
and the kernel causing long pauses, but I've had several different kernel
versions running.

I also use the ntfs-3g driver for write access to a doze partition, but
although the degradation in performance more severe with the ntfs-3g driver,
access to the native (ext3) partition also drags the system down for a
while.

I checked obvious things like whether or not I enable SMP in the kernel. I
tried changing the kernel pre-emption from low latency desktop to desktop,
but the problem persist. The application that is mostly involved when I get
these long delays is FireFox, VMWare and emerge (emerge --sync).  Everything
is compiled 64bit but I have the 32bit emulation libs.

Can anyone point me into some direction?

Regards
Dirk

[-- Attachment #2: Type: text/html, Size: 1014 bytes --]

Re: [gentoo-user] Looong delays

[Jesús Guerrero]

On Wed, 2 Dec 2009 13:22:38 +0200, Dirk Uys <dirkcuys@gmail.com> wrote:
> Hi
> 
> This has been bothering me for some time now. I have a Dell PC at work,
> Intel Core2 Duo with 4gb ram etc. Whenever something does a lot of disk
> access, the PC slows down to a halt? I remember some issue between
Firefox
> and the kernel causing long pauses, but I've had several different
kernel
> versions running.
> 
> I also use the ntfs-3g driver for write access to a doze partition, but
> although the degradation in performance more severe with the ntfs-3g
> driver,
> access to the native (ext3) partition also drags the system down for a
> while.
> 
> I checked obvious things like whether or not I enable SMP in the kernel.
I
> tried changing the kernel pre-emption from low latency desktop to
desktop,
> but the problem persist. The application that is mostly involved when I
get
> these long delays is FireFox, VMWare and emerge (emerge --sync). 
> Everything
> is compiled 64bit but I have the 32bit emulation libs.
> 
> Can anyone point me into some direction?
> 
> Regards
> Dirk

I know I am hitting at the obvious, but I can't be sure you already
checked that.

Since the applications you are using can be quite intensive in memory
usage, did you check whether you are hitting swap or not?

-- 
Jesús Guerrero

[gentoo-user] Re: Looong delays

[Nikos Chantziaras]

On 12/02/2009 01:22 PM, Dirk Uys wrote:
> Hi
>
> This has been bothering me for some time now. I have a Dell PC at work,
> Intel Core2 Duo with 4gb ram etc. Whenever something does a lot of disk
> access, the PC slows down to a halt? I remember some issue between
> Firefox and the kernel causing long pauses, but I've had several
> different kernel versions running.
>[...]

It's a known problem.  I have the same issue.  But there is a solution: 
start disk I/O heavy tasks with "ionice -c3".  For emerge, this can be 
done automatically by putting this in your make.conf:

   PORTAGE_IONICE_COMMAND="ionice -c 3 -p \${PID}"

ionice is in sys-apps/util-linux so it should be installed already.

Re: [gentoo-user] Looong delays

[Willie Wong]

On Wed, Dec 02, 2009 at 12:32:20PM +0100, Penguin Lover Jes??s Guerrero squawked:
> On Wed, 2 Dec 2009 13:22:38 +0200, Dirk Uys <dirkcuys@gmail.com> wrote:
> > This has been bothering me for some time now. I have a Dell PC at work,
> > Intel Core2 Duo with 4gb ram etc. Whenever something does a lot of disk
> > access, the PC slows down to a halt? 

> > The application that is mostly involved when I get
> > these long delays is FireFox, VMWare and emerge (emerge --sync). 
> 
> I know I am hitting at the obvious, but I can't be sure you already
> checked that.
> 
> Since the applications you are using can be quite intensive in memory
> usage, did you check whether you are hitting swap or not?

I realize that Firefox is a memory hog, but how many tabs must be open
for Swap to hit severely on a machine with 4gb ram? :)

Question in general: emerge --sync and VMWare I can see, but why does
FireFox require heavy disk access? (Actually, this is an honest
question: my work machine had a problem yesterday where everytime I
click a link in FireFox the computer freezes for about 30 seconds.
Turns out the problem was that someone else's rogue process was
hitting the NFS server like crazy so whatever disk-related activity
FireFox does after every link click cannot get through. It is somehow
worrisome that background IO like writing to the History file can lock
up the UI...)

W
-- 
When two egotists meet, it's an I for an I.
Sortir en Pantoufles: up 1090 days, 13:55

Re: [gentoo-user] Looong delays

[Willie Wong]

On Wed, Dec 02, 2009 at 01:22:38PM +0200, Penguin Lover Dirk Uys squawked:
> This has been bothering me for some time now. I have a Dell PC at work,
> Intel Core2 Duo with 4gb ram etc. Whenever something does a lot of disk
> access, the PC slows down to a halt? I remember some issue between Firefox
> and the kernel causing long pauses, but I've had several different kernel
> versions running.
> 
> I also use the ntfs-3g driver for write access to a doze partition, but
> although the degradation in performance more severe with the ntfs-3g driver,
> access to the native (ext3) partition also drags the system down for a
> while.

Since you already have decided that the problem is disk activity
related, perhaps check hdparm (if applicable)? 

Do you happen to know what kind of harddisk you are using?
From what you described it seems not to be a driver issue. 

Cheers, 

W
-- 
Research has led to the discovery of the heaviest element yet known to
science. The new element, Governmentium (Gv), has one neutron, 25
assistant neutrons, 88 deputy neutrons, and 198 assistant deputy
neutrons, giving it an atomic mass of 312.

These 312 particles are held together by forces called morons, which
are surrounded by vast quantities of lepton-like particles called
peons. Since Governmentium has no electrons, it is inert; however, it
can be detected, because it impedes every reaction with which it comes
into contact. A tiny amount of Governmentium can cause a reaction
normally taking less than a second, to take from four days to four
years to complete.

Governmentium has a normal half-life of 2-6 years. It does not decay,
but undergoes a reorganization in which a portion of the assistant
neutrons and deputy neutrons exchange places. In fact, Governmentium's
mass will actually increase over time, since each reorganization will
cause more morons to become neutrons, forming isodopes, not to mention
multiple oxymorons.

This characteristic of moron promotion leads some scientists to
believe that Governmentium is formed whenever morons reach a critical
concentration. That hypothetical quantity might normally be called
'critical mass' but, in this unique case it is known as 'critical
mess'.

When catalyzed with money, Governmentium becomes Administratium (Am),
another just-discovered element that radiates just as much energy as
Governmentium since it has half as many peons but twice as many
morons.
    ~sm62704 (957197) on /. cid 23224540
Sortir en Pantoufles: up 1090 days, 14:03

Re: [gentoo-user] Re: Looong delays

[Willie Wong]

On Wed, Dec 02, 2009 at 03:50:30PM +0200, Penguin Lover Nikos Chantziaras squawked:
> On 12/02/2009 01:22 PM, Dirk Uys wrote:
>> This has been bothering me for some time now. I have a Dell PC at work,
>> Intel Core2 Duo with 4gb ram etc. Whenever something does a lot of disk
>> access, the PC slows down to a halt? I remember some issue between
>> Firefox and the kernel causing long pauses, but I've had several
>> different kernel versions running.
>> [...]
>
> It's a known problem.  I have the same issue.  But there is a solution: 
> start disk I/O heavy tasks with "ionice -c3".  For emerge, this can be done 
> automatically by putting this in your make.conf:
>
>   PORTAGE_IONICE_COMMAND="ionice -c 3 -p \${PID}"
>
> ionice is in sys-apps/util-linux so it should be installed already.

Hum, I had forgotten about this command. It would have come in handy a
few days ago. But in the case of FireFox, wouldn't that make it worse? 

Also, what do you mean by known problem? What sort of set-up causes
the problem? Is this related at all to the hardware used? Or is this
purely in software?

Cheers, 

W


-- 
"Reality is a crutch for people who can't handle drugs."
        -- Lily Tomlin
Sortir en Pantoufles: up 1090 days, 14:08

Re: [gentoo-user] Looong delays

[Jesús Guerrero]

On Wed, 2 Dec 2009 10:10:30 -0500, Willie Wong <wwong@math.princeton.edu>
wrote:
> On Wed, Dec 02, 2009 at 12:32:20PM +0100, Penguin Lover Jes??s Guerrero
> squawked:
>> On Wed, 2 Dec 2009 13:22:38 +0200, Dirk Uys <dirkcuys@gmail.com> wrote:
>> > This has been bothering me for some time now. I have a Dell PC at
work,
>> > Intel Core2 Duo with 4gb ram etc. Whenever something does a lot of
disk
>> > access, the PC slows down to a halt? 
> 
>> > The application that is mostly involved when I get
>> > these long delays is FireFox, VMWare and emerge (emerge --sync). 
>> 
>> I know I am hitting at the obvious, but I can't be sure you already
>> checked that.
>> 
>> Since the applications you are using can be quite intensive in memory
>> usage, did you check whether you are hitting swap or not?
> 
> I realize that Firefox is a memory hog, but how many tabs must be open
> for Swap to hit severely on a machine with 4gb ram? :)

A lot. But even though it's possible to hog that system with firefox
alone, I wasn't thinking in that extreme case. I was more thinking along
the lines of wmware running a huge vm inside of it and I only meant firefox
and emerge as little Satan's helpers :D

> Question in general: emerge --sync and VMWare I can see, but why does
> FireFox require heavy disk access?

Well, that's why I ask if he's hitting swap. ANY app will require disk
access, even if not directly, if the ram is full. I have no idea if that's
the case though, I was just pointing at a possibility :)
-- 
Jesús Guerrero

[gentoo-user] Re: Looong delays

[Nikos Chantziaras]

On 12/02/2009 05:10 PM, Willie Wong wrote:
> On Wed, Dec 02, 2009 at 12:32:20PM +0100, Penguin Lover Jes??s Guerrero squawked:
>> On Wed, 2 Dec 2009 13:22:38 +0200, Dirk Uys<dirkcuys@gmail.com>  wrote:
>>> This has been bothering me for some time now. I have a Dell PC at work,
>>> Intel Core2 Duo with 4gb ram etc. Whenever something does a lot of disk
>>> access, the PC slows down to a halt?
>
>>> The application that is mostly involved when I get
>>> these long delays is FireFox, VMWare and emerge (emerge --sync).
>>
>> I know I am hitting at the obvious, but I can't be sure you already
>> checked that.
>>
>> Since the applications you are using can be quite intensive in memory
>> usage, did you check whether you are hitting swap or not?
>
> I realize that Firefox is a memory hog, but how many tabs must be open
> for Swap to hit severely on a machine with 4gb ram? :)
>
> Question in general: emerge --sync and VMWare I can see, but why does
> FireFox require heavy disk access?

It does not require heavy disk access.  It does however do frequent 
accesses.  Due to the kernel problems with simultaneous disk I/O on 
desktop systems, apps freeze regardless of whether their disk access is 
"heavy" or not.  The disk I/O code in the Linux kernel is probably 
written with servers in mind, not desktops, and you need to work around 
this with ionice, which I wrote about in another post.

[gentoo-user] Re: Looong delays

[Nikos Chantziaras]

On 12/02/2009 05:19 PM, Willie Wong wrote:
> On Wed, Dec 02, 2009 at 03:50:30PM +0200, Penguin Lover Nikos Chantziaras squawked:
>> On 12/02/2009 01:22 PM, Dirk Uys wrote:
>>> This has been bothering me for some time now. I have a Dell PC at work,
>>> Intel Core2 Duo with 4gb ram etc. Whenever something does a lot of disk
>>> access, the PC slows down to a halt? I remember some issue between
>>> Firefox and the kernel causing long pauses, but I've had several
>>> different kernel versions running.
>>> [...]
>>
>> It's a known problem.  I have the same issue.  But there is a solution:
>> start disk I/O heavy tasks with "ionice -c3".  For emerge, this can be done
>> automatically by putting this in your make.conf:
>>
>>    PORTAGE_IONICE_COMMAND="ionice -c 3 -p \${PID}"
>>
>> ionice is in sys-apps/util-linux so it should be installed already.
>
> Hum, I had forgotten about this command. It would have come in handy a
> few days ago. But in the case of FireFox, wouldn't that make it worse?

No.  If you run other tasks as "ionice -c3" they will stop blocking Firefox.


> Also, what do you mean by known problem? What sort of set-up causes
> the problem? Is this related at all to the hardware used? Or is this
> purely in software?

No, it's related to the Linux kernel blocking other applications for too 
long when one of them does heavy I/O.  This is done so that each task 
has a chance to get more work done for the amount of time it gets to do 
I/O.  As you can imagine, this hurts non-server systems quite badly.

Another solution is to buy more hard disks and set up a RAID.

Re: [gentoo-user] Re: Looong delays

[Willie Wong]

On Wed, Dec 02, 2009 at 06:24:49PM +0200, Penguin Lover Nikos Chantziaras squawked:
>> Hum, I had forgotten about this command. It would have come in handy a
>> few days ago. But in the case of FireFox, wouldn't that make it worse?
>
> No.  If you run other tasks as "ionice -c3" they will stop blocking 
> Firefox.

I see. I guess it would not have helped in my case as it was, since the 
problem was due to another user's process blocking the IO, and I am not 
root on my work machine. 
>
>> Also, what do you mean by known problem? What sort of set-up causes
>> the problem? Is this related at all to the hardware used? Or is this
>> purely in software?
>
> No, it's related to the Linux kernel blocking other applications for too 
> long when one of them does heavy I/O.  This is done so that each task has a 
> chance to get more work done for the amount of time it gets to do I/O.  As 
> you can imagine, this hurts non-server systems quite badly.

Ah, so ionice -c 3 is to be used for the other (dare I say
non-interactive?) processes. Thanks for the explanation. 

Cheers, 

W

-- 
A young woman was jogging when she saw a wizened old man, smiling at her
from his porch. 
"You look so happy!" she said to him. "What's your secret for a long
satisfying life?"
"Well, I smoke three packs of cigarettes every day and I drink a case of
wisky every week. on top of that I never excersice, and I eat lots of 
fatty foods."
"That's amazing," the woman said. "And how old are you?"
He answered, "thirty two."
Sortir en Pantoufles: up 1090 days, 18:04

Re: [gentoo-user] Looong delays

[James Ausmus]

[-- Attachment #1: Type: text/plain, Size: 1153 bytes --]

On Wed, Dec 2, 2009 at 3:22 AM, Dirk Uys <dirkcuys@gmail.com> wrote:

> Hi
>
> This has been bothering me for some time now. I have a Dell PC at work,
> Intel Core2 Duo with 4gb ram etc. Whenever something does a lot of disk
> access, the PC slows down to a halt? I remember some issue between Firefox
> and the kernel causing long pauses, but I've had several different kernel
> versions running.
>
> I also use the ntfs-3g driver for write access to a doze partition, but
> although the degradation in performance more severe with the ntfs-3g driver,
> access to the native (ext3) partition also drags the system down for a
> while.
>
> I checked obvious things like whether or not I enable SMP in the kernel. I
> tried changing the kernel pre-emption from low latency desktop to desktop,
> but the problem persist. The application that is mostly involved when I get
> these long delays is FireFox, VMWare and emerge (emerge --sync).  Everything
> is compiled 64bit but I have the 32bit emulation libs.
>
> Can anyone point me into some direction?
>


A tool I've always found useful for determining IO hog processes is
sys-process/iotop

HTH-

James

[-- Attachment #2: Type: text/html, Size: 1454 bytes --]

Re: [gentoo-user] Re: Looong delays

[Dirk Uys]

[-- Attachment #1: Type: text/plain, Size: 897 bytes --]

On Wed, Dec 2, 2009 at 3:50 PM, Nikos Chantziaras <realnc@arcor.de> wrote:

> On 12/02/2009 01:22 PM, Dirk Uys wrote:
>
>> Hi
>>
>> This has been bothering me for some time now. I have a Dell PC at work,
>> Intel Core2 Duo with 4gb ram etc. Whenever something does a lot of disk
>> access, the PC slows down to a halt? I remember some issue between
>> Firefox and the kernel causing long pauses, but I've had several
>> different kernel versions running.
>> [...]
>>
>
> It's a known problem.  I have the same issue.  But there is a solution:
> start disk I/O heavy tasks with "ionice -c3".  For emerge, this can be done
> automatically by putting this in your make.conf:
>
>  PORTAGE_IONICE_COMMAND="ionice -c 3 -p \${PID}"
>
> ionice is in sys-apps/util-linux so it should be installed already.
>
>
>

Thanx, I don't have time at the moment to test it, but I will surely try
this.

Regards
Dirk

[-- Attachment #2: Type: text/html, Size: 1444 bytes --]

Re: [gentoo-user] Looong delays

[Dirk Uys]

[-- Attachment #1: Type: text/plain, Size: 1916 bytes --]

2009/12/2 Jesús Guerrero <i92guboj@terra.es>

> On Wed, 2 Dec 2009 13:22:38 +0200, Dirk Uys <dirkcuys@gmail.com> wrote:
> > Hi
> >
> > This has been bothering me for some time now. I have a Dell PC at work,
> > Intel Core2 Duo with 4gb ram etc. Whenever something does a lot of disk
> > access, the PC slows down to a halt? I remember some issue between
> Firefox
> > and the kernel causing long pauses, but I've had several different
> kernel
> > versions running.
> >
> > I also use the ntfs-3g driver for write access to a doze partition, but
> > although the degradation in performance more severe with the ntfs-3g
> > driver,
> > access to the native (ext3) partition also drags the system down for a
> > while.
> >
> > I checked obvious things like whether or not I enable SMP in the kernel.
> I
> > tried changing the kernel pre-emption from low latency desktop to
> desktop,
> > but the problem persist. The application that is mostly involved when I
> get
> > these long delays is FireFox, VMWare and emerge (emerge --sync).
> > Everything
> > is compiled 64bit but I have the 32bit emulation libs.
> >
> > Can anyone point me into some direction?
> >
> > Regards
> > Dirk
>
> I know I am hitting at the obvious, but I can't be sure you already
> checked that.
>
> Since the applications you are using can be quite intensive in memory
> usage, did you check whether you are hitting swap or not?
>
> --
> Jesús Guerrero
>
>

It's possible, but I don't remember running into swap that much. If I ran
VMWare, I normally use a machine with 2GB memory. But still, my system at
home which use to have 512mb didn't run into swap with firefox running a
similar amount of tabs (I normally have about 12 tabs open of which most is
static HTML - reference docs)

I'll try to see if there is something else eating RAM and causing me to go
into swap.

Thanks
Dirk

[-- Attachment #2: Type: text/html, Size: 2477 bytes --]

Re: [gentoo-user] Looong delays

[Peter Humphrey]

On Wednesday 02 December 2009 15:10:30 Willie Wong wrote:

> It is somehow worrisome that background IO like writing to the History
> file can lock up the UI...)

Indeed. That does smell unwholesome.

-- 
Rgds
Peter