[gentoo-amd64] System becomes almost unusable when compling c++ code

[gentoo-amd64] System becomes almost unusable when compling c++ code

[Shaochun Wang]

Every time I compile C++ code, e.g. app-i18n/scim-qtimm, 
my desktop system becomes almost not
interactive. I have already set PORTAGE_NICENESS="15" in 
/etc/make.conf.

Any suggestion?

-- 
Shaochun Wang <scwang@ios.ac.cn>

Jabber:	fungusw@jabber.org
-- 
gentoo-amd64@gentoo.org mailing list

Re: [gentoo-amd64] System becomes almost unusable when compling c++ code

[Volker Armin Hemmann]

On Freitag, 3. August 2007, Shaochun Wang wrote:
> Every time I compile C++ code, e.g. app-i18n/scim-qtimm,
> my desktop system becomes almost not
> interactive. I have already set PORTAGE_NICENESS="15" in
> /etc/make.conf.
>
> Any suggestion?
>

1) PORTAGE_NICENESS=19" is A LOT better.
read /usr/src/linux/Documentation/sched-design.txt

here is the important part:
 batch scheduling. A significant proportion of computing-intensive tasks
   benefit from batch-scheduling, where timeslices are long and processes
   are roundrobin scheduled. The new scheduler does such batch-scheduling
   of the lowest priority tasks - so nice +19 jobs will get
   'batch-scheduled' automatically. With this scheduler, nice +19 jobs are
   in essence SCHED_IDLE, from an interactiveness point of view.


2) Maybe you need more ram?

free when you are compiling c++ stuff?


-- 
gentoo-amd64@gentoo.org mailing list

Re: [gentoo-amd64] System becomes almost unusable when compling c++ code

[Nicolas Merline]

[-- Attachment #1: Type: text/plain, Size: 464 bytes --]

When you start to compile, use "top", and see what the nice level of
emerge-portage is or w/e

On 8/3/07, Shaochun Wang <scwang@ios.ac.cn> wrote:
>
> Every time I compile C++ code, e.g. app-i18n/scim-qtimm,
> my desktop system becomes almost not
> interactive. I have already set PORTAGE_NICENESS="15" in
> /etc/make.conf.
>
> Any suggestion?
>
> --
> Shaochun Wang <scwang@ios.ac.cn>
>
> Jabber: fungusw@jabber.org
> --
> gentoo-amd64@gentoo.org mailing list
>
>

[-- Attachment #2: Type: text/html, Size: 902 bytes --]

[gentoo-amd64] Re: System becomes almost unusable when compling c++ code

[Duncan]

Shaochun Wang <scwang@ios.ac.cn> posted 20070803065913.GA23254@localhost,
excerpted below, on  Fri, 03 Aug 2007 14:59:13 +0800:

> Every time I compile C++ code, e.g. app-i18n/scim-qtimm, my desktop
> system becomes almost not
> interactive. I have already set PORTAGE_NICENESS="15" in /etc/make.conf.
> 
> Any suggestion?

Do you have a single-core single-CPU system, or multi-one-or-the-other?

In any case, unless you are running folding@home or the like, something 
truly idle-only that you want the emerge to get higher priority than, you 
should consider PORTAGE_NICENESS=19.   The reason being, a +19 nice is 
treated as idle priority by the scheduler, giving the rest of the system 
slightly higher responsiveness (lower latency), while giving the idle 
task somewhat longer time slices.  The effect can actually be to /speed/ 
/up/ compiles over a positive niceness <19 or even over normal 
scheduling, due to the longer timeslices.

What clock tick setting are you using, and what is your preemption 
setting?  Particularly if you are single-core and CPU, a higher clock 
tick setting (Timer frequency), not 100 certainly (that's for servers), 
probably 300 or 1000, will increase responsiveness at the cost of tasks 
taking longer (shorter timeslices, more overhead processing timeslices 
each second).  Similarly with preemption.  You'll want that set to 
Preemptible Kernel (Low-Latence Desktop) or at least Voluntary Kernel 
Premption (Desktop).  Also be sure the Preempt the Big Kernel Lock option 
is toggled ON.

Conversely, lower settings, No Forced Preemption (Server) or Voluntary 
Kernel Preemption (Desktop), and 100, 250 or 300 tick rate should work 
better for multi-core or multi-CPU SMP systems, because they can spread 
the load a bit more.  FWIW, dual Opteron 242 (so dual single cores) here, 
I'm running voluntary preemption, BKL preemption, and 300 Hz tick 
frequency.  That's the highest I'd recommend for a general purpose multi-
core or multi-CPU system, but tho I'd recommend as above, 1000 Hz tick 
and full preemption for single CPU/core systems.

You may also wish to play with the MAKEOPTS setting, typically -jX, where 
X is the number of CPUs/cores plus one to 150% of the CPUs/cores.  Thus, 
a single core/cpu system's recommended setting is -j2.  However, you may 
find -j1 increases your responsiveness.  Or you can try -j2, but add -l1 
or the like.  With GNU make (but not all others, you may have to remove 
the -lX portion for some merges), that'll tell it to allow up to two jobs 
(the -j2) but ONLY start a second one if the load average is below 1 (-
l1).  That's generally fairly effective.

There are other things to consider as well.  How do you usually compile, 
in a terminal window (xterm, konsole, gterm, etc) or at the text 
console?  I've noted that at least with konsole and with composite 
rendering (real transparency) turned on, non-niced CPU usage goes thru 
the roof trying to keep the konsole updated at times.  Causing the 
konsole window not to display, either minimizing it, shading it, or 
switching to a different desktop workspace so the konsole isn't shown, 
eliminates the issue.  If I'm /really/ planning on going to town (say a 
new KDE release came out and I have 100 plus packages of mostly C++ to 
compile), I'll turn off composite rendering as well.  When there's rapid 
display updates such as when compile output is scrolling by, if X and the 
X clients don't have to do all that extra work drawing and compositing 
areas normally hidden by other windows, it makes a big difference.  (It 
should be noted that typical composite overhead is <5% of a single CPU, 
here, more like 2% unless I've a huge bunch of windows open, and that's 
on a dual 1600x1200 display.  Radeon 9250, the last Radeons for which 
there's decent freedomware drivers, FWIW, tho the reverse engineering 
effort on the r300 and r400 series is progressing nicely.)

Then there's the standard stuff, but it'd tend to affect more than simply 
C++ compiling.  Make sure your SATA/PATA/SCSI chipsets are running their 
correct drivers with DMA enabled, not just generic, no DMA compatibility 
mode.  That's a big one but if it affected you you'd probably notice it 
elsewhere as well.  

If you have lots of memory (2 gigs or better, 4 gigs is nice, I have 8 
but that's overkill), strongly consider setting up your PORTAGE_TMPDIR 
(/var/tmp by default) on tmpfs.  Having all those temporary files 
typically used during a compile and merge written to memory only, instead 
of having to wait for several orders of magnitude slower hard disk 
access, DRAMATICALLY speeds up compiles, while at the SAME time speeding 
up general system responsiveness during the merge, because disk access 
slows down the /entire/ system, especially when whatever else you are 
working on is trying to access the disk at the same time.

To give you an example of what things /can/ be like, with my now aging 
dual Opteron 242 setup here (soon to be upgraded to dual-core Opteron 
290s), with 8 gigs memory (as I said, overkill, 4 gig would be fine), /
tmp on tmpfs (with $PORTAGE_TMPDIR=/tmp), 4-disk RAID-6 system (two 
parity stripes, so effectively 2-way striped), RAID-0 $PORTDIR and 
ccache, I routinely run MAKEOPTS="-j1000" (not that it ever gets there, 
but some builds don't like the unlimited -j, no number), and run five or 
more parallel emerges at the same time (using emerge -pt to get a tree 
output and -a for verification, so I can setup non-conflicting parallel 
emerges).  That's usually for a KDE update where I have 100 or so C++ KDE 
packages to merge, so it's mostly C++.  Because the config and certain 
other sections aren't parallelized and thus do only a single job, even 
with that, my load average seldom rises above 20 or 25.  Or, on the 
kernel, which is C not C++ but parallelizes VERY well, I'll get a load 
average of several /hundred/.  It's fun to see it go that high! =8^)  
Still, even with a 300-500 load average or 20ish load average on C++ (the 
most I seem to hit is 30), while I do get a bit of lag on the mouse, and 
the panel clock and ksysguard displays sometimes freeze for 10 seconds at 
a time, it's still surprising to me it's not /entirely/ unusable.  As 
well, I can be playing an Internet radio stream the entire time (and no, 
I don't have anything set real-time, either), with few if any dropouts at 
all.  That's REALLY astounding to me!  Up to a 500 load average, yet the 
scheduler continues to work well enough to give the network, player and 
audio system all the time it needs to prevent both dropped network 
packets and dropped audio data!  (It's obvious the scheduler prioritizes 
both the IP stack and the audio system without intervention, and equally 
obvious the KDE panel doesn't get the same auto prioritization, as 
clearly, 10 second updates as on the panel simply wouldn't cut it on the 
network or audio stream.  That be as it may, it's still fascinating to 
watch, seeing the load average climb to several hundred when I 
deliberately compile a kernel with -j1000, without a single hitch or skip 
in the audio playback at all!)

-- 
Duncan - List replies preferred.   No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master."  Richard Stallman

-- 
gentoo-amd64@gentoo.org mailing list

Re: [gentoo-amd64] Re: System becomes almost unusable when compling c++ code

[Richard Freeman]

[-- Attachment #1: Type: text/plain, Size: 955 bytes --]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Duncan wrote:
> If you have lots of memory (2 gigs or better, 4 gigs is nice, I have 8 
> but that's overkill), strongly consider setting up your PORTAGE_TMPDIR 
> (/var/tmp by default) on tmpfs.  

Actually, as long as you have enough swap your RAM doesn't really
matter.  In the worst case it just swaps to disk, which is what it would
be doing anyway if you didn't have a tmpfs.  Even if only 10% of the
short-lived files didn't make it to disk it would be a huge savings in
IO waits.

Also, consider using ionice (from schedutils I think) - it lets you
schedule IO the way nice schedules CPU.  It can have a big impact if you
have a lot of disk churn.
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.7 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFGszhsG4/rWKZmVWkRAhwIAKCMAIhQqVaaPrHi3IIbigc3tpVHSQCgtsEg
HDrQL97HsW8c5NmVKy+24jc=
=tmSA
-----END PGP SIGNATURE-----

[-- Attachment #2: S/MIME Cryptographic Signature --]
[-- Type: application/x-pkcs7-signature, Size: 4101 bytes --]

[gentoo-amd64] Re: System becomes almost unusable when compling c++ code

[Duncan]

Richard Freeman <rich@thefreemanclan.net> posted
46B33881.2060801@thefreemanclan.net, excerpted below, on  Fri, 03 Aug 2007
10:15:29 -0400:

> Actually, as long as you have enough swap your RAM doesn't really
> matter.  In the worst case it just swaps to disk, which is what it would
> be doing anyway if you didn't have a tmpfs.  Even if only 10% of the
> short-lived files didn't make it to disk it would be a huge savings in
> IO waits.

Thanks for pointing that out, yes.

> Also, consider using ionice (from schedutils I think) - it lets you
> schedule IO the way nice schedules CPU.  It can have a big impact if you
> have a lot of disk churn.

It can.  However, with the proper i/o scheduler (CFQ), plain nice is 
often good enough, since the i/o scheduler implements logical defaults 
based on nice value -- higher nice means higher default ionice.

That did get me to switch i/o schedulers here, BTW.  I had been using 
anticipatory until I setup the RAID, at which point I switched to the 
simpler deadline, since anticipatory does its anticipation stuff assuming 
a single spindle.  After reading a bit about the workings of ionice 
however, and how CFQ defaults i/o priority based on CPU timeslice 
priority (and therefore nice), I switched to it.  With the memory I have, 
most disk access after the initial is cached, so it didn't affect me a 
lot, and RAID helps in that regard as well, but it did seem to make some 
difference.

So anyway, I'm in pretty strong agreement with you. =8^)

-- 
Duncan - List replies preferred.   No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master."  Richard Stallman

-- 
gentoo-amd64@gentoo.org mailing list

Re: [gentoo-amd64] System becomes almost unusable when compling c++ code

[Luis Medinas]

On Fri, 2007-08-03 at 14:59 +0800, Shaochun Wang wrote:
> Every time I compile C++ code, e.g. app-i18n/scim-qtimm, 
> my desktop system becomes almost not
> interactive. I have already set PORTAGE_NICENESS="15" in 
> /etc/make.conf.
> 
> Any suggestion?
> 
I suggest to reduce the MAKEOPTS. I had this problem with my macbook
with -j4 now i'm using -j3 and it doesn't turn the system unusuable. In
certain apps i suggest to use -j1 like wxgtk.

-- 
gentoo-amd64@gentoo.org mailing list

Re: [gentoo-amd64] Re: System becomes almost unusable when compling c++ code

[Volker Armin Hemmann]

On Freitag, 3. August 2007, Richard Freeman wrote:
> Duncan wrote:
> > If you have lots of memory (2 gigs or better, 4 gigs is nice, I have 8
> > but that's overkill), strongly consider setting up your PORTAGE_TMPDIR
> > (/var/tmp by default) on tmpfs.
>
> Actually, as long as you have enough swap your RAM doesn't really
> matter.  In the worst case it just swaps to disk, which is what it would
> be doing anyway if you didn't have a tmpfs.  Even if only 10% of the
> short-lived files didn't make it to disk it would be a huge savings in
> IO waits.

emm, from my very personal point of view: no

swap is horrible slow. Its use must be avoided at any cost. Swap sucks. 
Everything is faster than accessing swap. So hitting the disk to read or 
write some files is IMHO better than hitting the disk to shove X into swap.

X in swap is another problem. You can be sure, if X is forced into swap, 
because gcc uses up all ram for itself, everything sucks. Mouse is jerky, 
windows need ages to get displayed.

Everything that makes swapping more likely is bad advise and shall be avoided.
-- 
gentoo-amd64@gentoo.org mailing list

Re: [gentoo-amd64] Re: System becomes almost unusable when compling c++ code

[Bernhard Auzinger]

> emm, from my very personal point of view: no
>
> swap is horrible slow. Its use must be avoided at any cost. Swap sucks.
> Everything is faster than accessing swap. So hitting the disk to read or
> write some files is IMHO better than hitting the disk to shove X into swap.
>
> X in swap is another problem. You can be sure, if X is forced into swap,
> because gcc uses up all ram for itself, everything sucks. Mouse is jerky,
> windows need ages to get displayed.
>
> Everything that makes swapping more likely is bad advise and shall be
> avoided.

I'm that opinion too. Particularly when system and swap are on the same disk. 
And even if swap and system are located on different disks, it's one of the 
worst things that can happen to the performance of a system.

Anyway, I've never experienced such behaviour since I've always had enough 
ram. And I never set portage niceness. Even on desktops with single cores, 
100Hz timer frequency and without kernel preemption the system was 
responsive. I think if a system gets "freezes" it must have something to do 
with heavy IO-load and to less main memory.

Rgds
Bernhard
-- 
gentoo-amd64@gentoo.org mailing list

Re: [gentoo-amd64] Re: System becomes almost unusable when compling c++ code

[Richard Freeman]

[-- Attachment #1: Type: text/plain, Size: 2124 bytes --]

Volker Armin Hemmann wrote:
> emm, from my very personal point of view: no
> 
> swap is horrible slow. Its use must be avoided at any cost. Swap sucks. 
> Everything is faster than accessing swap. So hitting the disk to read or 
> write some files is IMHO better than hitting the disk to shove X into swap.
> 
> X in swap is another problem. You can be sure, if X is forced into swap, 
> because gcc uses up all ram for itself, everything sucks. Mouse is jerky, 
> windows need ages to get displayed.
> 

Depends - if you're not using X then you won't mind X getting swapped 
(running emerge from ssh/etc).  If you are using X then chance are it 
won't get swapped in the first place.  If the issue is gcc using all ram 
for itself, then the presence/absence of tmpfs probably won't make much 
difference - gcc still has to run.

I don't see why swap should be any better/worse than any other form of 
disk access.  If anything it is superior as it allows the kernel to 
manage it like any other form of RAM, and the kernel isn't forced to 
flush it out to disk within some time (writes to a filesystem are forced 
to sync within some time to prevent data loss - this hurts performance 
and is totally unnecessary for temporary build files that will get 
deleted when you re-run emerge anyway).

Even if you don't have a tmpfs writing to disk will tend to drive unused 
ram into swap - the system will swap idle memory to make room for 
cache/buffers - where the recently-written files will reside in ram.

Now, in an extreme case where you have less RAM than the resident size 
of the apps you have running then swap will be horrible - but that is 
because you're continuously swapping in and out.  And I doubt a tmpfs 
will make much difference either way.

Now, if somebody has empirical data I'll certainly pay attention, or at 
least a lot of expertise.  However, I wouldn't jump to the conclusion 
that swap is worse than ordinary disk writes - linux manages swap fairly 
well all things considered.  If you don't like how it is being managed 
there are kernel settings that can be used to tweak it (swappiness, etc).

[-- Attachment #2: S/MIME Cryptographic Signature --]
[-- Type: application/x-pkcs7-signature, Size: 4101 bytes --]

Re: [gentoo-amd64] Re: System becomes almost unusable when compling c++ code

[Volker Armin Hemmann]

On Freitag, 3. August 2007, Richard Freeman wrote:
> Volker Armin Hemmann wrote:
> > emm, from my very personal point of view: no
> >
> > swap is horrible slow. Its use must be avoided at any cost. Swap sucks.
> > Everything is faster than accessing swap. So hitting the disk to read or
> > write some files is IMHO better than hitting the disk to shove X into
> > swap.
> >
> > X in swap is another problem. You can be sure, if X is forced into swap,
> > because gcc uses up all ram for itself, everything sucks. Mouse is jerky,
> > windows need ages to get displayed.
>
> Depends - if you're not using X then you won't mind X getting swapped
> (running emerge from ssh/etc).  If you are using X then chance are it
> won't get swapped in the first place.  If the issue is gcc using all ram
> for itself, then the presence/absence of tmpfs probably won't make much
> difference - gcc still has to run.
>
> I don't see why swap should be any better/worse than any other form of
> disk access.  If anything it is superior as it allows the kernel to
> manage it like any other form of RAM, and the kernel isn't forced to
> flush it out to disk within some time (writes to a filesystem are forced
> to sync within some time to prevent data loss - this hurts performance
> and is totally unnecessary for temporary build files that will get
> deleted when you re-run emerge anyway).

where is the difference between 'app syncs its files to disk' and 'kernel 
swaps app to disk, than decides to swap it in, so app can decides which files 
it wants to sync to disk, then swap it out again'?

oh, more disk access...

>
> Now, if somebody has empirical data I'll certainly pay attention, or at
> least a lot of expertise.  However, I wouldn't jump to the conclusion
> that swap is worse than ordinary disk writes - linux manages swap fairly
> well all things considered.  If you don't like how it is being managed
> there are kernel settings that can be used to tweak it (swappiness, etc).

swappiness just prevents the kernel from swapping, when their is enough ram 
left. It does not improve the speed of swap.

And honestly? From my 'feelings' swap is way slower than normal disk accesses. 

-- 
gentoo-amd64@gentoo.org mailing list

Re: [gentoo-amd64] Re: System becomes almost unusable when compling c++ code

[Bernhard Auzinger]

> Depends - if you're not using X then you won't mind X getting swapped
> (running emerge from ssh/etc).  If you are using X then chance are it
> won't get swapped in the first place.  If the issue is gcc using all ram
> for itself, then the presence/absence of tmpfs probably won't make much
> difference - gcc still has to run.
>
> I don't see why swap should be any better/worse than any other form of
> disk access.  If anything it is superior as it allows the kernel to
> manage it like any other form of RAM, and the kernel isn't forced to
> flush it out to disk within some time (writes to a filesystem are forced
> to sync within some time to prevent data loss - this hurts performance
> and is totally unnecessary for temporary build files that will get
> deleted when you re-run emerge anyway).
>
> Even if you don't have a tmpfs writing to disk will tend to drive unused
> ram into swap - the system will swap idle memory to make room for
> cache/buffers - where the recently-written files will reside in ram.
>
> Now, in an extreme case where you have less RAM than the resident size
> of the apps you have running then swap will be horrible - but that is
> because you're continuously swapping in and out.  And I doubt a tmpfs
> will make much difference either way.
>
> Now, if somebody has empirical data I'll certainly pay attention, or at
> least a lot of expertise.  However, I wouldn't jump to the conclusion
> that swap is worse than ordinary disk writes - linux manages swap fairly
> well all things considered.  If you don't like how it is being managed
> there are kernel settings that can be used to tweak it (swappiness, etc).

Yes you are completely right, but that is not the point I wanted to make. I 
just think avoiding any unnecessary disk access is the best solution as long 
there will be the bottleneck.

Rgds
Bernhard
-- 
gentoo-amd64@gentoo.org mailing list

Re: [gentoo-amd64] Re: System becomes almost unusable when compling c++ code

[Richard Freeman]

[-- Attachment #1: Type: text/plain, Size: 779 bytes --]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Bernhard Auzinger wrote:
> 
> Yes you are completely right, but that is not the point I wanted to make. I 
> just think avoiding any unnecessary disk access is the best solution as long 
> there will be the bottleneck.
> 

Agreed - spending more money on RAM/CPU/disk/etc will often improve
performance more than better-using the resources already available, but
at some point you decide to stop spending money and at that point you
have to work with what you already have...  :)
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.7 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFGs5SlG4/rWKZmVWkRAqcvAKCMXqZYdPCjuQuMoVWausYS0AV5/QCfWpGY
UnLBMLnSAlsz/4Id92TLZek=
=hG8y
-----END PGP SIGNATURE-----

[-- Attachment #2: S/MIME Cryptographic Signature --]
[-- Type: application/x-pkcs7-signature, Size: 4101 bytes --]

Re: [gentoo-amd64] Re: System becomes almost unusable when compling c++ code

[Richard Freeman]

[-- Attachment #1: Type: text/plain, Size: 3678 bytes --]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Volker Armin Hemmann wrote:
> 
> where is the difference between 'app syncs its files to disk' and 'kernel 
> swaps app to disk, than decides to swap it in, so app can decides which files 
> it wants to sync to disk, then swap it out again'?
> 
> oh, more disk access...
> 

Actually, you're not quite looking at it the right way.  Suppose during
the course of an emerge I need 150MB of temp files.  I have two choices
- - write to disk, or write to tmpfs.

In the former case every write goes to disk buffer (ram), and then is
synced to disk within about 5 seconds.  In the latter case every write
goes to tmpfs (ram), and if necessary gets written to swap at an
opportune time (when the system needs more ram, the page is idle for a
long time, the system is otherwise idle, etc) - and if this need doesn't
come up before the file is deleted it never gets written to disk at all.

In the worst case both get synced to disk, but in the former case the
kernel does not get to choose when outside the forced setting of about 5
seconds.  In the latter case the kernel can schedule disk access more
opportunistically - because the kernel knows the data isn't that
important and doesn't need to be written to disk quickly.

Generally well-written apps don't decide what files it wants syned to
disk - the kernel does that.  If a file is written to a normal
filesystem it gets synced according to the rules of the filesystem, and
if it is written to tmpfs it gets written to disk if the system needs
more ram and the file is more idle than most of the rest of RAM.

I think you'd be surprised at how similarly files on disk and memory in
ram get handled.  Both get "swapped" in and out of memory (if the file
is mmapped - which is often the case), and writes to files get stored in
buffers/cache just like files in tmpfs.  The big difference is that
tmpfs files are not guaranteed to ever get written to disk, which means
the kernel can handle them more flexibly.  Normal files are ALWAYS
written to disk (unless they're deleted in the few seconds before they
are synced), and the kernel isn't allowed to just keep them in buffers
for 30 minutes if the disk is busy.

In the extreme of excessive RAM the tmpfs is going to be many orders of
magnitude faster than writing to disk.  In the extreme of a very
ram-starved system it will probably be comparable.

> 
> And honestly? From my 'feelings' swap is way slower than normal disk accesses. 
> 

What kinds of access are you comparing?  If you're talking about
swapping in and out a 50MB binary vs a 50kb text file then you're
comparing apples to oranges.  Ditto if you're comparing time to access
just part of a file.  And any time you run a program you're using disk
swapping - even if you have no swap partition.  Linux just mmaps the
executable into memory and starts running it (with little to nothing in
RAM) - any time the program accesses its own code it triggers a swap-in.
 This means that code that is unused might never get read off the disk,
and the kernel will opportunistically read-ahead when it thinks it makes
sense to do so - just as it does with any other kind of swapped memory,
or a regular file.

Swap and normal file access aren't handled completely identically, so
there can potentially be differences.  However, there is so much in
common that I'd be surprised if there is as big a difference as you suggest.
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.7 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFGs5eOG4/rWKZmVWkRAmRpAJ96U/jIjZQmtItisYKi88F+OoKpfQCgnpzB
nQv1vfz65sXNroRWmKLsZTs=
=8Rna
-----END PGP SIGNATURE-----

[-- Attachment #2: S/MIME Cryptographic Signature --]
[-- Type: application/x-pkcs7-signature, Size: 4101 bytes --]

Re: [gentoo-amd64] Re: System becomes almost unusable when compling c++ code

[Volker Armin Hemmann]

On Freitag, 3. August 2007, Richard Freeman wrote:
> Volker Armin Hemmann wrote:
> > where is the difference between 'app syncs its files to disk' and 'kernel
> > swaps app to disk, than decides to swap it in, so app can decides which
> > files it wants to sync to disk, then swap it out again'?
> >
> > oh, more disk access...
>
> Actually, you're not quite looking at it the right way.  Suppose during
> the course of an emerge I need 150MB of temp files.  I have two choices
> - write to disk, or write to tmpfs.
>
> In the former case every write goes to disk buffer (ram), and then is
> synced to disk within about 5 seconds.  In the latter case every write
> goes to tmpfs (ram), and if necessary gets written to swap at an
> opportune time (when the system needs more ram, the page is idle for a
> long time, the system is otherwise idle, etc) - and if this need doesn't
> come up before the file is deleted it never gets written to disk at all.
>
> In the worst case both get synced to disk, but in the former case the
> kernel does not get to choose when outside the forced setting of about 5
> seconds.  In the latter case the kernel can schedule disk access more
> opportunistically - because the kernel knows the data isn't that
> important and doesn't need to be written to disk quickly.
>
> Generally well-written apps don't decide what files it wants syned to
> disk - the kernel does that.  If a file is written to a normal
> filesystem it gets synced according to the rules of the filesystem, and
> if it is written to tmpfs it gets written to disk if the system needs
> more ram and the file is more idle than most of the rest of RAM.
>
> I think you'd be surprised at how similarly files on disk and memory in
> ram get handled.  Both get "swapped" in and out of memory (if the file
> is mmapped - which is often the case), and writes to files get stored in
> buffers/cache just like files in tmpfs.  The big difference is that
> tmpfs files are not guaranteed to ever get written to disk, which means
> the kernel can handle them more flexibly.  Normal files are ALWAYS
> written to disk (unless they're deleted in the few seconds before they
> are synced), and the kernel isn't allowed to just keep them in buffers
> for 30 minutes if the disk is busy.
>
> In the extreme of excessive RAM the tmpfs is going to be many orders of
> magnitude faster than writing to disk.  In the extreme of a very
> ram-starved system it will probably be comparable.
>
> > And honestly? From my 'feelings' swap is way slower than normal disk
> > accesses.
>
> What kinds of access are you comparing?  If you're talking about
> swapping in and out a 50MB binary vs a 50kb text file then you're
> comparing apples to oranges. 

I am comparing an open konqueror window that was minimized, had a webpage open 
and was swapped out with the rest of X and the KDE session compared with log 
in, start konqueror and loading that page.

You see, both give me a usable window in the same timeframe (+/- 1 or 2 
seconds) but the second choice does not make the mouse jerking around.

The worst: as soon as say 100mb are in swap, everything is slow for the rest 
of the evening. Best solution?
swapoff -a && swapon -a. And have some popcorn when the disk is accessed like 
mad.

>even if you have no swap partition.  Linux just mmaps the
> executable into memory and starts running it (with little to nothing in
> RAM) - any time the program accesses its own code it triggers a swap-in.
>
I know that - and it is faster to read that executable from fs than to swap it 
in....
-- 
gentoo-amd64@gentoo.org mailing list