[gentoo-amd64] tmpfs help

[gentoo-amd64] tmpfs help

[Beso]

[-- Attachment #1: Type: text/plain, Size: 199 bytes --]

i recently put portage on tmpfs and i'm happy about it since i got a nice
speedup in compilation, but i get an issue: tmpfs is only 450mb long.
is there a way to increase it?!?!

-- 
dott. ing. beso

[-- Attachment #2: Type: text/html, Size: 228 bytes --]

Re: [gentoo-amd64] tmpfs help

[Mateusz Mierzwinski]

Beso pisze:
> i recently put portage on tmpfs and i'm happy about it since i got a 
> nice speedup in compilation, but i get an issue: tmpfs is only 450mb 
> long.
> is there a way to increase it?!?!
>
> -- 
> dott. ing. beso 
Well, You should remove distfiles in portage tree after compilation. All 
downloaded files are placed in distfiles directory, and this can take 
Your free space. Also if You need to compile only some part of packages, 
You can make bindings for some packeges and only a part of portage tree 
store in tmpfs, and part bind by "mount -o bind" or "ln -sf" into tmpfs 
tree from normal fs. Also I must warn You, that tmpfs is not good idea 
to place portage tree because of system dependency like /proc or /sys. 
Tmpfs is SHM.

Greetings
Mateusz M.
-- 
gentoo-amd64@lists.gentoo.org mailing list

Re: [gentoo-amd64] tmpfs help

[Pascal BERTIN]

Beso a écrit :
> i recently put portage on tmpfs and i'm happy about it since i got a
> nice speedup in compilation, but i get an issue: tmpfs is only 450mb long.
> is there a way to increase it?!?!
> 
> -- 
> dott. ing. beso

You can give a size for tmpfs in the option
here is an extract from my /etc/fstab :


tmp   /tmp            tmpfs           size=1000000000 0 0



-- 
gentoo-amd64@lists.gentoo.org mailing list

Re: [gentoo-amd64] tmpfs help

[Beso]

[-- Attachment #1: Type: text/plain, Size: 1202 bytes --]

2008/2/12, Mateusz Mierzwinski <mateuszmierzwinski@o2.pl>:
>
> Beso pisze:
> > i recently put portage on tmpfs and i'm happy about it since i got a
> > nice speedup in compilation, but i get an issue: tmpfs is only 450mb
> > long.
> > is there a way to increase it?!?!
> >
> > --
> > dott. ing. beso
> Well, You should remove distfiles in portage tree after compilation. All
> downloaded files are placed in distfiles directory, and this can take
> Your free space. Also if You need to compile only some part of packages,
> You can make bindings for some packeges and only a part of portage tree
> store in tmpfs, and part bind by "mount -o bind" or "ln -sf" into tmpfs
> tree from normal fs. Also I must warn You, that tmpfs is not good idea
> to place portage tree because of system dependency like /proc or /sys.
> Tmpfs is SHM.
>
> Greetings
> Mateusz M.
> --
> gentoo-amd64@lists.gentoo.org mailing list
>
>
i didn't put portage tree on tmpfs, but just portage compilation directory.
even distfiles aren't on tmpfs since i don't want to download distfiles
everytime i compile something. the problem is that it's too small for
certain packages like kdelibs and other big ones.

-- 
dott. ing. beso

[-- Attachment #2: Type: text/html, Size: 1610 bytes --]

Re: [gentoo-amd64] tmpfs help

[Beso]

[-- Attachment #1: Type: text/plain, Size: 682 bytes --]

2008/2/12, Pascal BERTIN <pascal.bertin@corobor.com>:
>
> Beso a écrit :
> > i recently put portage on tmpfs and i'm happy about it since i got a
> > nice speedup in compilation, but i get an issue: tmpfs is only 450mb
> long.
> > is there a way to increase it?!?!
> >
> > --
> > dott. ing. beso
>
> You can give a size for tmpfs in the option
> here is an extract from my /etc/fstab :
>
>
> tmp   /tmp            tmpfs           size=1000000000 0 0


i'll try that. setting it to about 3/4 of swap is good?! i have 8gb swap and
1 gb ram but ram is always full. after setting tmpfs the ram is full but
also the swap fills-up quite well.

-- 
dott. ing. beso

[-- Attachment #2: Type: text/html, Size: 1087 bytes --]

Re: [gentoo-amd64] tmpfs help

[Pascal BERTIN]

Beso a écrit :
>     You can give a size for tmpfs in the option
>     here is an extract from my /etc/fstab :
> 
> 
>     tmp   /tmp            tmpfs           size=1000000000 0 0
> 
> 
> i'll try that. setting it to about 3/4 of swap is good?! i have 8gb swap
> and 1 gb ram but ram is always full. after setting tmpfs the ram is full
> but also the swap fills-up quite well.
> 
> -- 
> dott. ing. beso

On one of my system, with 1G of RAM and 6 GB of swap,
I set it to 6GB (so that I can compile openoffice).
Although It's slow (anyway I start openoffice compilation at the end of a day and check on
the next morning), it works well, and openoofice compiles.

-- 
gentoo-amd64@lists.gentoo.org mailing list

Re: [gentoo-amd64] tmpfs help

[Beso]

[-- Attachment #1: Type: text/plain, Size: 1382 bytes --]

2008/2/12, Pascal BERTIN <pascal.bertin@corobor.com>:
>
> Beso a écrit :
> >     You can give a size for tmpfs in the option
> >     here is an extract from my /etc/fstab :
> >
> >
> >     tmp   /tmp            tmpfs           size=1000000000 0 0
> >
> >
> > i'll try that. setting it to about 3/4 of swap is good?! i have 8gb swap
> > and 1 gb ram but ram is always full. after setting tmpfs the ram is full
> > but also the swap fills-up quite well.
> >
> > --
> > dott. ing. beso
>
> On one of my system, with 1G of RAM and 6 GB of swap,
> I set it to 6GB (so that I can compile openoffice).
> Although It's slow (anyway I start openoffice compilation at the end of a
> day and check on
> the next morning), it works well, and openoofice compiles.


for openoffice you might want to try the experimental ebuilds from the
personal overlay of diego peteno from flameeyes-overlay. it seems that he's
trying to remove internal support for apps that have been already installed
from openoffice. this is still in a hardmasked and so very experimental
stage but whoever compiles openoffice might want to give it a shot. compile
the binpkg with portage before trying it so that, if something goes wrong
you wouldn't have to pass another night for it to compile again and you'd
just need to reinstall the stuff.
thanks for the help.
-- 
dott. ing. beso

[-- Attachment #2: Type: text/html, Size: 1884 bytes --]

Re: [gentoo-amd64] tmpfs help

[Mateusz Mierzwinski]

Pascal BERTIN pisze:
> Beso a écrit :
>   
>>     You can give a size for tmpfs in the option
>>     here is an extract from my /etc/fstab :
>>
>>
>>     tmp   /tmp            tmpfs           size=1000000000 0 0
>>
>>
>> i'll try that. setting it to about 3/4 of swap is good?! i have 8gb swap
>> and 1 gb ram but ram is always full. after setting tmpfs the ram is full
>> but also the swap fills-up quite well.
>>
>> -- 
>> dott. ing. beso
>>     
>
> On one of my system, with 1G of RAM and 6 GB of swap,
> I set it to 6GB (so that I can compile openoffice).
> Although It's slow (anyway I start openoffice compilation at the end of a day and check on
> the next morning), it works well, and openoofice compiles.
>
>   
If You see a FSTAB:

tmpfs        /dev/shm  tmpfs   defaults          0 0

there is probably some thing like dev link to shared memory - I think, 
that SHM is needed by some proceses, if You put something in SHM, then 
after process starting, that opens SHM from IPC, it's allocated in space 
of tmpfs, and tmpfs is dropping into swapfile. If I don't get wrong, 
then You should stop using tmpfs...

|# mount -o remount,size=8G /dev/shm
||# mount -t tmpfs -o size=5G,nr_inodes=5k,mode=700 tmpfs /disk2/tmpfs

"|shm / shmfs is also known as tmpfs, which is a common name for a 
temporary file storage facility on many Unix-like operating systems. It 
is intended to appear as a mounted file system, but one which uses 
virtual memory instead of a persistent storage device.

If you type mount command you will see /dev/shm as a tempfs file system. 
Therefore, it is a file system, which keeps all files in virtual memory. 
Everything in tmpfs is temporary in the sense that no files will be 
created on your hard drive. If you unmount a tmpfs instance, everything 
stored therein is lost. By default almost all Linux distros configured 
to use /dev/shm."

|
This is what i found about resizing SHM (tmpfs)
.
Greetings
|
-- 
gentoo-amd64@lists.gentoo.org mailing list

Re: [gentoo-amd64] tmpfs help

[Volker Armin Hemmann]

Hi,

with your small amount of memory tmpfs hurts you more than helps you. The 
small compilations might to be sped up. But what about the big ones? 

Slow downs, because swap is used. Big slow downs, because swap is horribly 
slow. Slow downs in daily usage because space that could be used for 
cache&buffers is wasted for tmpfs.

In your case: get 2gb (minimum!) first before playing around with tmpfs.
-- 
gentoo-amd64@lists.gentoo.org mailing list

Re: [gentoo-amd64] tmpfs help

[Richard Freeman]

Volker Armin Hemmann wrote:
> 
> with your small amount of memory tmpfs hurts you more than helps you. The 
> small compilations might to be sped up. But what about the big ones? 
> 
> Slow downs, because swap is used. Big slow downs, because swap is horribly 
> slow. 

Do you have benchmarks to support this?

Why would swap be any slower than compiling to disk?  If the compilation 
generates 6GB of files and you have 512MB of spare RAM, then most likely 
5.5GB of that stuff will have gotten written to disk during the course 
of compilation - with very opportunistic writing.  The last 512MB of 
data will probably be deleted from RAM before it ever gets written.

On the other hand, without a tmpfs then all 6GB is guaranteed to be 
written to disk.  Any files that are created and deleted 10 seconds 
later will be written to disk with a guaranteed sync.  Every write will 
get synced withing a few seconds.

I would think that swap would be more efficient than a million files on 
a filesystem.  The kernel can flush tmpfs pages at any time - and it 
doesn't have a compulsion to flush out writes immediately since there 
isn't any sense of permenancy to the data.

 > Slow downs in daily usage because space that could be used for
 > cache&buffers is wasted for tmpfs.

As opposed to wasting all your RAM on cache&buffers to hold all those 
files being accessed?  During compilation that RAM is going to be 
heavily used - no getting around that.  Once you're done any files left 
sitting on a tmpfs will just get paged out until accessed.  They 
shouldn't really use any RAM at all.  Even if those files were on disk 
they would consume RAM in the form of caching until they're considered 
unneeded.

Basically tmpfs lets the kernel have more flexibility in memory 
management, while with disk-based temporary filesystems you're forcing 
the kernel to treat temporary data the same way you'd treat more 
critical files.

I know that this is a bit of a religious debate, however I believe it is 
full of misconceptions.  I'd be very interested in actual benchmarks - 
although I'm sure this stuff isn't easy to test.  I certainly agree that 
swap is slow compared to RAM, but it isn't slow compared to a disk-based 
filesystem.  Of course adding more RAM will never hurt, but that incurs 
significant cost and you can at least maximize your current hardware 
before investing in more of it.
-- 
gentoo-amd64@lists.gentoo.org mailing list

Re: [gentoo-amd64] tmpfs help

[Volker Armin Hemmann]

On Mittwoch, 13. Februar 2008, Richard Freeman wrote:
> Volker Armin Hemmann wrote:
> > with your small amount of memory tmpfs hurts you more than helps you. The
> > small compilations might to be sped up. But what about the big ones?
> >
> > Slow downs, because swap is used. Big slow downs, because swap is
> > horribly slow.
>
> Do you have benchmarks to support this?

which numbers? that swap is horrible slow compared to ram?

>
> Why would swap be any slower than compiling to disk?  If the compilation
> generates 6GB of files and you have 512MB of spare RAM, then most likely
> 5.5GB of that stuff will have gotten written to disk during the course
> of compilation - with very opportunistic writing.  The last 512MB of
> data will probably be deleted from RAM before it ever gets written.

think about this: instead of having the compiler in ram and files on disk, you 
have to swap the compiler out, read the files from swap into ram, swap the 
compiler in, do some stuff, repeat. Slowdown.

>
> On the other hand, without a tmpfs then all 6GB is guaranteed to be
> written to disk.  Any files that are created and deleted 10 seconds
> later will be written to disk with a guaranteed sync.  Every write will
> get synced withing a few seconds.

nope. Stuff that is written and deleted some seconds later never hit the disk. 
That is why cache&buffer exist.

>>
>  > Slow downs in daily usage because space that could be used for
>  > cache&buffers is wasted for tmpfs.
>
> As opposed to wasting all your RAM on cache&buffers to hold all those
> files being accessed?  During compilation that RAM is going to be
> heavily used - no getting around that.  Once you're done any files left
> sitting on a tmpfs will just get paged out until accessed.  They
> shouldn't really use any RAM at all.  Even if those files were on disk
> they would consume RAM in the form of caching until they're considered
> unneeded.

cached&buffered information can be discarded anytime if the ram is needed 
elsewhere. tmpfs has to be shoved into swap.


>
> Basically tmpfs lets the kernel have more flexibility in memory
> management, while with disk-based temporary filesystems you're forcing
> the kernel to treat temporary data the same way you'd treat more
> critical files.

with the files on disk, the kernel can hold as much data in ram as he likes 
to. with a big tmpfs (and 512mb out of 1gb) he is always shoving someting 
into swap or get something from there. 

Extreme example?

kdepim with the enablefinal flag. On amd64 every single gcc instance needs 
~900mb at two points of compilation. With 1gb ram and no tmpfs it sucks.

With 1gb ram and 512mb of that reserved for tmpfs, you'll get a swapstorm.

>
> I know that this is a bit of a religious debate, however I believe it is
> full of misconceptions.  I'd be very interested in actual benchmarks -
> although I'm sure this stuff isn't easy to test.  I certainly agree that
> swap is slow compared to RAM, but it isn't slow compared to a disk-based
> filesystem.

really? Every really swapped? IMHO it feels like every single byte is fetched 
by a mule.

 Of course adding more RAM will never hurt, but that incurs 
> significant cost and you can at least maximize your current hardware
> before investing in more of it.

significant costs like 30euros for 2gb?
-- 
gentoo-amd64@lists.gentoo.org mailing list

Re: [gentoo-amd64] tmpfs help

[Steve Buzonas]

[-- Attachment #1: Type: text/plain, Size: 1072 bytes --]

>
> really? Every really swapped? IMHO it feels like every single byte is
> fetched
> by a mule.
>

 I have to agree.  With no real management over what goes in and out of
swap.  With your ram pushed to full capacity, compiling on tmpfs is by far
slower than compiling on disk with a filesystem like xfs or zfs.  I would
suggest to find a decent filesystem specific to your needs and optimize it.
With little ram you lose time pushing it in and out of swap.  I used to
compile in tmpfs until I compared my benchmarks to a fine tuned ext3
filesystem.  The biggest speed increase i've ever achieved was upgrading to
a faster hard drive, if your disk can't keep up with your ram and processor
you'll never get the results you're looking for anyhow.  I just did a
bootstrap in 20 mins, emerge -e system in 38, and had a gentoo lamp server
up and running in under an hour.  (That's no longer the purpose of the
server after I saw the performance advantage it had over the other servers).

--
> gentoo-amd64@lists.gentoo.org mailing list
>
>


-- 
Thank you,

Steve Buzonas Jr.

[-- Attachment #2: Type: text/html, Size: 1606 bytes --]

[gentoo-amd64] Re: tmpfs help

[Duncan]

Pascal BERTIN <pascal.bertin@corobor.com> posted
47B17CA6.9000506@corobor.com, excerpted below, on  Tue, 12 Feb 2008
12:01:58 +0100:

> Beso a écrit :
[Pascal wrote...]
>>     You can give a size for tmpfs in the option here is an extract from
>>     my /etc/fstab :
>> 
>>     tmp   /tmp            tmpfs           size=1000000000 0 0
>> 
>> i'll try that. setting it to about 3/4 of swap is good?! i have 8gb
>> swap and 1 gb ram but ram is always full. after setting tmpfs the ram
>> is full but also the swap fills-up quite well.
> 
> On one of my system, with 1G of RAM and 6 GB of swap, I set it to 6GB
> (so that I can compile openoffice). Although It's slow (anyway I start
> openoffice compilation at the end of a day and check on the next
> morning), it works well, and openoofice compiles.

Agreed.

On the worth it or not thing, certainly 1 gig real memory is a bit low 
for compiling into using tmpfs, but for the reasons Richard F gave, it 
should in theory at least, remain faster than compiling into a temp 
location on disk.

As he points out, if it's compiled to disk, it (1) sits in cache and is 
thus in memory anyway, and (2) if it's in cache more than a few seconds, 
it's flushed to disk thus incurring the slowdown of writing to disk.

If it's compiled to tmpfs, then as the kernel needs memory, it writes out 
the least recently accessed stuff to swap.  That applies regardless of 
whether it's apps or tmpfs data that's last used (as for cache, that 
depends on the swappiness setting, /proc/sys/vm/swappiness, with a 
default of 60, 0 means dump all cache before swapping apps, 100 or higher 
means swap apps before dumping cache, so 60 is leaning just slightly 
toward keeping cache and swapping apps).  Thus, the active and most 
recently active parts of the compiler won't be swapped out if there's 
less recently active tmpfs files to swap first.  Similarly with the 
scratch data on tmpfs.  In theory, that should be the most efficient use 
of memory possible, certainly more so than using conventional disk for 
temp data, since that forces it to disk while likely keeping it in cache 
anyway.

However, the theory doesn't always match reality as I'm sure most of us 
realize by now.  Whether it does here or not, I haven't tested, and as 
I've 8 gigs memory, don't care to test for the 1 gig situation.

Regardless, I think we can all agree that it'll be far more practical 
with 2-4 gigs of memory, and for most, the upgrade from 1 to 2 gigs 
minimum for memory should be well worth it in terms of value for cost.  

Meanwhile, I say let the folks dealing with the problem decide their 
policy, testing it and reporting the results if they feel the desire too, 
or just going with what they think works best for them if not.

Given that some are choosing to try it, regardless of why, the original 
question deserves an answer, as Pascal posted above.  It may however be 
worth noting that mount also accepts more human readable numbers as well, 
6G or whatever instead of the long string of numbers.  Thus, here's my 
entry:

/tmp   /tmp      tmpfs  size=6g,nodev,nosuid                      0 0

Again, that's with 8 gigs RAM, so I don't have to worry about swapping so 
much, tho even when I do it's to 4-way striped swap, so 4 times as fast 
as a single disk would be, and I have swappiness set to 100, so the 
kernel swaps apps (at 4-way-stripped swap, twice as fast as reading them 
in off 4-way RAID-6, thus two-way-striped), keeping cache intact.  That 
works great for me! =8^)  Whether it works better than a disk based /tmp 
and PORTAGE_TMPDIR for those with only a gig of memory is something they 
ultimately must decide, regardless of the arguments pro and con here.

FWIW and for clarity, since there seems to be a bit of confusion between 
tmpfs as used here and the FHS/LSB mandated /dev/shm, I have an entry for 
that as well:

shm    /dev/shm  tmpfs  size=20m,noexec,nodev,nosuid              0 0

Note that it's a separate entry.  More than one tmpfs mount is allowed 
and they are all kept separate.  Also note that I have the max size set 
far smaller for it than for /tmp, since I don't have much that uses it 
and only keep it around in case something wants to.  (I do have 
PORTAGE_TMPFS, which is used for very small files, lock files and the 
like, set to /dev/shm, so portage uses it for that even tho 
PORTAGE_TMPDIR is set to /tmp, for the much bigger stuff.)

I actually have another tmpfs mount as well, /dev, for udev:

dev    /dev      tmpfs  mode=0755,size=2m,noexec,noauto          0 0

baselayout-1 users won't have that entry in fstab, as it sets it 
differently, but may have the following instead (tho I think this was 
left over from the never stabilized AFAIK baselayout-1.13-alphas):

init.d /lib64/rcscripts/init.d tmpfs  mode=0755,size=512k,noauto 0 0

So I actually have four tmpfs entries in fstab, altho only three are 
active as that last one is left from an earlier time because the scripts 
no longer call for it to be mounted and the general system mount ignores 
it due to the noauto.  All four are entirely separate mounts, separate 
options, seperate mountpoints, and separately treated by the system.

Also, a warning I've started noting every time this comes up, those who 
point PORTAGE_TMPDIR at a tmpfs AND use ccache will want to make sure 
CCACHE_DIR is pointed somewhere other than a subdir of PORTAGE_TMPDIR, 
since keeping ccache files around is rather the point of running it, but 
the default as found in make.conf.example is to point it at a subdir of 
PORTAGE_TMPDIR.

-- 
Duncan - List replies preferred.   No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master."  Richard Stallman

-- 
gentoo-amd64@lists.gentoo.org mailing list

Re: [gentoo-amd64] Re: tmpfs help

[Volker Armin Hemmann]

On Mittwoch, 13. Februar 2008, Duncan wrote:
>>removed lots of irrelevant 'my hardware is so cool' stuff'.

You forget some (little) things. Not everything can be swapped out. Swap is 
extremly slow AND it is much worse to swapout/swapin programm code that 
should be run, instead of fetching some files from disk while the programm 
runs.
-- 
gentoo-amd64@lists.gentoo.org mailing list

Re: [gentoo-amd64] tmpfs help

[Richard Freeman]

Volker Armin Hemmann wrote:
> On Mittwoch, 13. Februar 2008, Richard Freeman wrote:
>> Do you have benchmarks to support this?
> 
> which numbers? that swap is horrible slow compared to ram?
> 

No - that compiling in tmpfs is horrible compared to compiling to disk. 
  Neither of us is proposing getting rid of disk access entirely - we're 
just debating two different ways of doing it.


> think about this: instead of having the compiler in ram and files on disk, you 
> have to swap the compiler out, read the files from swap into ram, swap the 
> compiler in, do some stuff, repeat. Slowdown.
> 

Uh, every file that you compile has to be read into RAM and then 
binaries have to be written back to disk with your proposal.  With swap 
stuff only gets written to disk if RAM gets low - which in the worst 
case is equivalent to writing everything to disk - which is your typical 
case.  The kernel would not be swapping the compiler in and out in these 
kinds of circumstances - it would probably be swapping in and out the 
tmpfs.  However, if some pages in the compiler are completely idle they 
might get swapped out, making more RAM available for tmpfs use.

> 
> nope. Stuff that is written and deleted some seconds later never hit the disk. 
> That is why cache&buffer exist.

If "some seconds" is greater than about 10 it will be flushed to disk 
100% of the time with most filesystems.  The exact figures probably vary 
by filesystem.  Most filesystems sync their buffers within some time 
after they are written.  tmpfs does not have this feature - which is 
good if you care about your files and bad if you don't.

>>
>> As opposed to wasting all your RAM on cache&buffers to hold all those
>> files being accessed?  During compilation that RAM is going to be
>> heavily used - no getting around that.  Once you're done any files left
>> sitting on a tmpfs will just get paged out until accessed.  They
>> shouldn't really use any RAM at all.  Even if those files were on disk
>> they would consume RAM in the form of caching until they're considered
>> unneeded.
> 
> cached&buffered information can be discarded anytime if the ram is needed 
> elsewhere. tmpfs has to be shoved into swap.
> 

Uh, data written to a file has to be written to disk before the 
cache/buffer is flushed.  The difference is that the data is written to 
disk within about 10 seconds of being written to the buffer, and then 
retained in cache maybe for a few hours longer (optimistically).  With 
tmpfs the write doesn't happen until the space is needed.  Either way 
every byte gets written - with tmpfs it is deferred as late as possible 
but with disk-based filesystems the write is done as early as possible.

> 
> with the files on disk, the kernel can hold as much data in ram as he likes 
> to. with a big tmpfs (and 512mb out of 1gb) he is always shoving someting 
> into swap or get something from there. 
> 

You can put 6GB of data in a 7GB tmpfs filesystem on a system with 512MB 
of RAM and still have 400MB of free memory.  tmpfs does not reserve 
memory - if it did it would certainly use tons of memory.

> Extreme example?
> 
> kdepim with the enablefinal flag. On amd64 every single gcc instance needs 
> ~900mb at two points of compilation. With 1gb ram and no tmpfs it sucks.
> 
> With 1gb ram and 512mb of that reserved for tmpfs, you'll get a swapstorm.

Oh, it will certainly swap a great deal.  However, I don't see why it 
would be any slower - either way you're doing a ton of disk access.  Do 
you have actual benchmarks?

Again, tmpfs doesn't "reserve" memory - it uses memory - just like 
cache/buffers.

>> I certainly agree that
>> swap is slow compared to RAM, but it isn't slow compared to a disk-based
>> filesystem.
> 
> really? Every really swapped? IMHO it feels like every single byte is fetched 
> by a mule.
> 

Right now I've got 589MB of free RAM (-/+ buffers/cache) with 2GB of RAM 
total.  I've got 705MB of swap used.  Works just fine IMHO.  Sure, if I 
make it really busy it can get slow, although with nice/ionice there 
isn't much visible impact.  And it doesn't really seem much slower using 
swap vs on-disk.  I'd expect some swap slowdown in my case as I'm using 
encrypted swap, but I don't think most users will be doing that.

>  Of course adding more RAM will never hurt, but that incurs 
>> significant cost and you can at least maximize your current hardware
>> before investing in more of it.
> 
> significant costs like 30euros for 2gb?

Sure.  Compared to about $1 for 2GB of hard drive space 30 euros is 
significant.  And even if he had 2GB of RAM I submit that it would 
probably work better if he compiled in tmpfs than on disk.  There is no 
question that a $550 computer will outperform a $500 computer, and a 
$600 computer will outperform a $550 computer, and so on.  You can 
always spend 30 more euros and improve your system.  What I'm interested 
in is maximizing the performance of the system I have now.  I can always 
spend $50 and make it even faster.  However, there are lots of things I 
can spend $50 on - I'd rather spend it on something else all things 
being equal.
-- 
gentoo-amd64@lists.gentoo.org mailing list

Re: [gentoo-amd64] Re: tmpfs help

[Richard Freeman]

Volker Armin Hemmann wrote:
> On Mittwoch, 13. Februar 2008, Duncan wrote:
>>> removed lots of irrelevant 'my hardware is so cool' stuff'.
> 
> You forget some (little) things. Not everything can be swapped out. 

How is this relevant?  Some memory is locked or effectively locked. 
That limits buffers/cache in the disk-based fs case, and swap in the 
tmpfs case.  I don't see that there is much of a difference.

 > Swap is
> extremly slow AND it is much worse to swapout/swapin programm code that 
> should be run, instead of fetching some files from disk while the programm 
> runs.

You keep asserting this.  Why should fetching a page from swap be slower 
than fetching the equivalent space from a file?  In fact, if mmap is 
used to read a file the same code probably is used for both.

And the kernel is not likely to swap out recently-run program pages.  It 
isn't going to keep a bunch of compiled binaries in memory for 10 
minutes while it swaps in and out the compiler - it would swap in and 
out source files and binary files - just as with a disk-based 
filesystem. If for some reason part of the compiler is swapped out, it 
would be rewritten to disk every time it got swapped out - only the 
first time.  I'm sure the kernel would notice that the pages were never 
written since the last swap and would just flush them from memory 
without another write.  However, this should still only happen on idle 
pages, and swapping those out is probably better than any other use of 
memory (and those pages would get swapped out even with a disk-based 
filesystem in use).

I'm of course interested in benchmarks.  In theory swap should be faster 
than files.  However, implementation does matter and maybe there is some 
flaw that makes things slower.  My assertion is that a swap-based 
solution gives the kernel more freedom to optimize memory use than a 
disk-based solution.  So, this should be a fundamentally-superior solution.
-- 
gentoo-amd64@lists.gentoo.org mailing list

Re: [gentoo-amd64] tmpfs help

[Volker Armin Hemmann]

On Mittwoch, 13. Februar 2008, Richard Freeman wrote:
> Volker Armin Hemmann wrote:
> > On Mittwoch, 13. Februar 2008, Richard Freeman wrote:
> >> Do you have benchmarks to support this?
> >
> > which numbers? that swap is horrible slow compared to ram?
>
> No - that compiling in tmpfs is horrible compared to compiling to disk.
>   Neither of us is proposing getting rid of disk access entirely - we're
> just debating two different ways of doing it.

tmpfs is fine - if you have enough ram. 1gb is not enough.



> >> As opposed to wasting all your RAM on cache&buffers to hold all those
> >> files being accessed?  During compilation that RAM is going to be
> >> heavily used - no getting around that.  Once you're done any files left
> >> sitting on a tmpfs will just get paged out until accessed.  They
> >> shouldn't really use any RAM at all.  Even if those files were on disk
> >> they would consume RAM in the form of caching until they're considered
> >> unneeded.
> >
> > cached&buffered information can be discarded anytime if the ram is needed
> > elsewhere. tmpfs has to be shoved into swap.
>
> Uh, data written to a file has to be written to disk before the
> cache/buffer is flushed.  The difference is that the data is written to
> disk within about 10 seconds of being written to the buffer, and then
> retained in cache maybe for a few hours longer (optimistically).  With
> tmpfs the write doesn't happen until the space is needed.  Either way
> every byte gets written - with tmpfs it is deferred as late as possible
> but with disk-based filesystems the write is done as early as possible.
>

emm, no.


> > Extreme example?
> >
> > kdepim with the enablefinal flag. On amd64 every single gcc instance
> > needs ~900mb at two points of compilation. With 1gb ram and no tmpfs it
> > sucks.
> >
> > With 1gb ram and 512mb of that reserved for tmpfs, you'll get a
> > swapstorm.
>
> Oh, it will certainly swap a great deal.  However, I don't see why it
> would be any slower - either way you're doing a ton of disk access.  Do
> you have actual benchmarks?

Not real benchmarks. But kdepim with enablefinal, 1gb of ram and -j2 several 
hours. With j1 2h.
.
kdepim with 2gb of ram and j2 30m
Just because first case swap storn, last case no swap at all.

>
> Again, tmpfs doesn't "reserve" memory - it uses memory - just like
> cache/buffers.

but while cache/buffers can be discarded when the ram is needed, tmpfs has to 
be shoved into butt-slow swap.

>
> >> I certainly agree that
> >> swap is slow compared to RAM, but it isn't slow compared to a disk-based
> >> filesystem.

yes, yes it is. It is faster to start an app from disk, then to fetch it out 
of swap. My very personal experience since many many years.


> >
> > really? Every really swapped? IMHO it feels like every single byte is
> > fetched by a mule.
>
> Right now I've got 589MB of free RAM (-/+ buffers/cache) with 2GB of RAM
> total.  I've got 705MB of swap used.  Works just fine IMHO.  Sure, if I
> make it really busy it can get slow, although with nice/ionice there
> isn't much visible 

your system would feel and act a lot faster if you don't have anything in 
swap. 'Fine' is good - as long as you don't know the alternative.

> >  Of course adding more RAM will never hurt, but that incurs
> >
> >> significant cost and you can at least maximize your current hardware
> >> before investing in more of it.
> >
> > significant costs like 30euros for 2gb?
>
> Sure.  Compared to about $1 for 2GB of hard drive space 30 euros is
> significant.

don't forget that ram is also roughly 100 times faster than harddisk.

> And even if he had 2GB of RAM I submit that it would 
> probably work better if he compiled in tmpfs than on disk. 

with 2gb he could do it. But with 1gb he just hurts himself.


> There is no 
> question that a $550 computer will outperform a $500 computer, and a
> $600 computer will outperform a $550 computer, and so on.  You can
> always spend 30 more euros and improve your system.  What I'm interested
> in is maximizing the performance of the system I have now.  I can always
> spend $50 and make it even faster.  However, there are lots of things I
> can spend $50 on - I'd rather spend it on something else all things
> being equal.

you could stop shoving everything in swap - no costs involved and system is a 
lot faster.


-- 
gentoo-amd64@lists.gentoo.org mailing list

[gentoo-amd64] Re: tmpfs help

[Duncan]

Volker Armin Hemmann <volker.armin.hemmann@tu-clausthal.de> posted
200802131717.57766.volker.armin.hemmann@tu-clausthal.de, excerpted below,
on  Wed, 13 Feb 2008 17:17:57 +0100:

> Not real benchmarks. But kdepim with enablefinal, 1gb of ram and -j2
> several hours. With j1 2h.
> .
> kdepim with 2gb of ram and j2 30m
> Just because first case swap storn, last case no swap at all.

But that's measuring the effect of the additional memory as much as it is 
the effect of swap.  What you need to do is kdepim with enablefinal, a 
gig of RAM and -j1 with PORTAGE_TMPDIR set to someplace on disk, against 
the same thing but with PORTAGE_TMPDIR set to a tmpfs of sufficient 
size.  Then and only then are you measuring the effect of tmpfs vs no 
tmpfs.  He (and I) are arguing that the tmpfs case will take less time 
because some files will get deleted before they end up being written to 
disk at all, as opposed to /all/ of them being written to disk (well, if 
they remain more than a few seconds anyway).

If you like you can of course do the test with -j2, but then you must use 
-j2 for both cases, so again PORTAGE_TMPDIR on tmpfs vs on disk is the 
only variable.

This of course assumes similar system load otherwise, presumably either 
an idle system (in X/GNOME/KDE/whatever both times or at the CLI both 
times) or only something steady, say streaming Internet radio, the same 
station at the same bits per second without visualizations, going on, 
with the same apps loaded in the background.  No answering mails or 
browsing or anything else interactive or variable since it's possible you 
could then be doing something different at a critical time in one case 
vs. the other.

-- 
Duncan - List replies preferred.   No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master."  Richard Stallman

-- 
gentoo-amd64@lists.gentoo.org mailing list

[gentoo-amd64] Re: tmpfs help

[Duncan]

Volker Armin Hemmann <volker.armin.hemmann@tu-clausthal.de> posted
200802131346.26316.volker.armin.hemmann@tu-clausthal.de, excerpted below,
on  Wed, 13 Feb 2008 13:46:26 +0100:

> On Mittwoch, 13. Februar 2008, Duncan wrote:
>>>removed lots of irrelevant 'my hardware is so cool' stuff'.
> 
> You forget some (little) things. Not everything can be swapped out. Swap
> is extremly slow AND it is much worse to swapout/swapin programm code
> that should be run, instead of fetching some files from disk while the
> programm runs.

It's not always much worse, because as I explained, in my case, swap is 4-
way striped while most of the main system is only two-way striped.  Thus, 
that "irrelevant" stuff is relevant after all, because it alters the 
conditions of the case in debate, because swap reads in at ~2x the speed 
of most data read off disk including apps (which is itself ~2x what a 
single-disk system might reasonably expect).

I've a feeling not appreciating this, not appreciating that your "test" 
case example of compiling with 2 gigs RAM vs only 1 has little to do with 
what might occur with PORTAGE_TMPDIR on tmpfs vs on disk, and not 
appreciating the point RF and I are both trying to make, is due to the 
same logic flaw.

-- 
Duncan - List replies preferred.   No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master."  Richard Stallman

-- 
gentoo-amd64@lists.gentoo.org mailing list

Re: [gentoo-amd64] Re: tmpfs help

[Beso]

[-- Attachment #1: Type: text/plain, Size: 2569 bytes --]

2008/2/13, Duncan <1i5t5.duncan@cox.net>:
>
> Volker Armin Hemmann <volker.armin.hemmann@tu-clausthal.de> posted
> 200802131346.26316.volker.armin.hemmann@tu-clausthal.de, excerpted below,
> on  Wed, 13 Feb 2008 13:46:26 +0100:
>
> > On Mittwoch, 13. Februar 2008, Duncan wrote:
> >>>removed lots of irrelevant 'my hardware is so cool' stuff'.
> >
> > You forget some (little) things. Not everything can be swapped out. Swap
> > is extremly slow AND it is much worse to swapout/swapin programm code
> > that should be run, instead of fetching some files from disk while the
> > programm runs.
>
> It's not always much worse, because as I explained, in my case, swap is 4-
> way striped while most of the main system is only two-way striped.  Thus,
> that "irrelevant" stuff is relevant after all, because it alters the
> conditions of the case in debate, because swap reads in at ~2x the speed
> of most data read off disk including apps (which is itself ~2x what a
> single-disk system might reasonably expect).
>
> I've a feeling not appreciating this, not appreciating that your "test"
> case example of compiling with 2 gigs RAM vs only 1 has little to do with
> what might occur with PORTAGE_TMPDIR on tmpfs vs on disk, and not
> appreciating the point RF and I are both trying to make, is due to the
> same logic flaw.


well, for now, the fact for me are:
1. no ram upgrade is good -> notebook ram costs much more than desktop one
and the notebook itself has a limit
2. small packages, that have much update during the single days of the week
(i do sync once a day) get compiled at least 2-3 times faster than normal
compilation into disk space
3. big packages that usually compile in 30-40 mins now compile in about
10minutes or so faster. to see how it feels to use tmpfs for compilation i
have to upgrade kde. usually kde3 would build into 2 days of about 13 hours
compilation time each. i'll have to see how fast would kde4 build.
4. the sync now takes less than 2mins while normally it would take about
10mins.
i'll try out duncan's speedups for shm and but for the dev one i don't use
baselayout 2 and i'm still with the 1st version, since i don't feel like
upgrading to it yet. but i'd like to know some more about what are the
improvements of the second version.
5. as for the raid stuff i cannot do it since i've only got one disk. i'll
try to see what happens with swap set to 100.
6. if i use some other programs while compiling into tmpfs bigones i need to
nice the process or i'll get some misbehaviour from the other programs.

-- 
dott. ing. beso

[-- Attachment #2: Type: text/html, Size: 3156 bytes --]

Re: [gentoo-amd64] tmpfs help

[Richard Freeman]

[-- Attachment #1: Type: text/plain, Size: 3641 bytes --]

Volker Armin Hemmann wrote:
> emm, no.

Look, if you're not going to actually respond to something, then either 
don't reply or at least don't quote it.  One-liners really aren't 
helpful - this isn't mean to be a flame-war over disk-fs vs disk-swap. 
I'm as interested as anybody to understand the pros/cons of both, but 
let's keep it civil and about finding answers, not beating our chests...

> 
> Not real benchmarks. But kdepim with enablefinal, 1gb of ram and -j2 several 
> hours. With j1 2h.
> .
> kdepim with 2gb of ram and j2 30m
> Just because first case swap storn, last case no swap at all.

Well, sure.  But that isn't apples to apples.  And even with more RAM 
you might have a performance difference due to disk-thrashing.

I'll see if I can do some benchmarking between disk and swap.

> 
>> Again, tmpfs doesn't "reserve" memory - it uses memory - just like
>> cache/buffers.
> 
> but while cache/buffers can be discarded when the ram is needed, tmpfs has to 
> be shoved into butt-slow swap.

It can only be discarded after it is flushed.  Just like swap.  The only 
difference is when the flushing occurs.  With files the flushing happens 
right after a write, with swapping it happens when the memory is needed 
for something else.


> 
>>>> I certainly agree that
>>>> swap is slow compared to RAM, but it isn't slow compared to a disk-based
>>>> filesystem.
> 
> yes, yes it is. It is faster to start an app from disk, then to fetch it out 
> of swap. My very personal experience since many many years.
> 

Uh, you do know that when you start an app from disk that the kernel 
just mmaps the file, right?  Then any access to process memory triggers 
a page fault and the page is swapped in.  When you start a program from 
disk it IS swapped to start with - the image on disk is just treated 
like a swap file and the same code is used to read it into memory as any 
other missing page.  The same applies to any other use of mmap.  If you 
go back "many many" years the behavior might have been different, but it 
has been this way for a while.

> 
> your system would feel and act a lot faster if you don't have anything in 
> swap. 'Fine' is good - as long as you don't know the alternative.

How do you know?  What is in the swap?  Maybe it is just pages with 
memory leaks.  If it is never read why would the system be slower?  I 
wouldn't say that performance is any better after a reboot when swap is 
empty.

>> Sure.  Compared to about $1 for 2GB of hard drive space 30 euros is
>> significant.
> 
> don't forget that ram is also roughly 100 times faster than harddisk.

Absolutely.  If I wanted the fastest computer possible I'd have way more 
RAM (and CPU) than I have now.  However, my finances aren't unlimited, 
and I'd rather make the most of what I have than just throw cash at 
problems.  And if I did have more RAM I'd still want to make the most of it.

> 
> you could stop shoving everything in swap - no costs involved and system is a 
> lot faster.
> 

Ok, it is obvious that absent benchmarks that nobody is convincing 
anybody here.  I think that most with a good knowledge of the linux 
kernel would disagree with you.  I don't profess to have that level of 
expertise, but I don't see anything being posted by you which indicates 
why swap should be slower than filesystem access, other than just a 
general statement that swap is "butt slow".

If I can come up with some measurements I'll post them.  My intent isn't 
really to get into a "he said she said" over this.  I'd like to inform 
and be informed, and I'm interested in all experiences although anecdote 
is obviously very limited.

[-- Attachment #2: S/MIME Cryptographic Signature --]
[-- Type: application/x-pkcs7-signature, Size: 4101 bytes --]

[gentoo-amd64] Re: tmpfs help

[Duncan]

Beso <givemesugarr@gmail.com> posted
d257c3560802131127g51bccaf6s920e73d1f2416766@mail.gmail.com, excerpted
below, on  Wed, 13 Feb 2008 19:27:44 +0000:

> i'll try out duncan's speedups for shm and but for the dev one i don't
> use baselayout 2 and i'm still with the 1st version, since i don't feel
> like upgrading to it yet. but i'd like to know some more about what are
> the improvements of the second version.

Well, among other things it's possible to /really/ do parallel startup 
now.  The option was there before, but it didn't parallelize that much.  
Of course, if you have any startup scripts running that the dependencies 
aren't straight on, that might be hidden now, but will very likely show 
itself once the system actually does try things in parallel.  (For those 
who prefer traditional serial startup, that remains the safer default.)

Startup's also much faster as certain parts are written in C now, instead 
of as scripts.  Of course, the various service scripts remain just that, 
scripts.

With baselayout-1, the very early core scripts, clock, modules, lvm, 
raid, etc, were actually ordered based on a list rather than on their 
normal Gentoo dependencies (before, after, uses, needs, etc).  That was 
because the dependency resolver didn't work quite right that early on.  
That has now been fixed, and all scripts including the early stuff like 
clock, start in the order the dependencies would indicate, not based on 
an arbitrary list.

Various settings are in more logical/traditional locations with 
baselayout 2.  An example is the /dev filesystem mounted if you have udev 
active.  Previously, its configuration was in one or more of the 
baselayout config files (probably /etc/conf.d/rc but that was quite 
awhile ago here, and I've forgotten the details so can't be sure).  Now, 
the setting in /etc/fstab for that filesystem is honored, as one might 
ordinarily expect for any filesystem.

Former addons like lvm and raid now have their own initscripts, just as 
any other boot service.

> 5. as for the raid stuff i cannot do it since i've only got one disk.
> i'll try to see what happens with swap set to 100.

For a single disk, it's possible you'll actually want it set the other 
way, toward 0 from 60, especially if you are annoyed at the time it takes 
to swap programs back in after big compiles or a night's system scan for 
slocate (I forgot what the scanner is called, as I don't have slocate 
installed here so don't run the database updater/scanner).  On a system 
such as mine with swap roughly twice as fast as the main filesystems, 
however, keeping cache and swapping out apps makes much more sense, since 
it's faster to read back in the apps from swap than it is to reload the 
data I'd dump from cache to keep the apps in memory.  I actually don't 
even notice swap usage unless I happen to look at ksysguard, unless it's 
swapping over a gig, which doesn't happen too often with 8 gigs RAM.  
Still, it's quite common to have a quarter to three quarter gig of 
swapped out apps since I've set swappiness to 100, thereby telling the 
kernel to keep cache if at all possible, and I routinely do -j12 
compiles, often several at a time even, so several gigs of tmpfs plus 
several gigs of gcc instances in memory, thereby forcing minor swapping 
even with 8 gig RAM, isn't unusual.  (Of course, I use emerge --pretend 
to ensure the packages I'm emerging in parallel don't conflict with or 
depend on each other, so the merges remain parallel.)

> 6. if i use some
> other programs while compiling into tmpfs bigones i need to nice the
> process or i'll get some misbehaviour from the other programs.

Here's another hint.  Consider setting PORTAGE_NICENESS=19 in make.conf.  
Not only does it stop the hogging, the system then considers it batch 
priority and gives it longer timeslices.  Thus, counter-intuitively for 
lowering the priority, a merge can actually take less real clock time, 
because the CPU is spending less time shuffling processes around and more 
time actually doing work.  Of course, if you run folding@home or the 
like, you'll either want to turn that off when doing this, or set portage 
to 18 or better instead of 19, so it's not competing directly with your 
idle time client.

With kernel 2.6.24, there's also a new scheduling option that groups by 
user.  By default, root gets 2048 while other users get 1024, half the 
share of root.  If you have portage set to compile with userprivs, it'll 
be using the portage user, with its own 1024 default setting, but if it's 
using root privs, it'll be getting root's default 2048 share.  
Particularly if you use a -j setting >2 jobs per cpu/core, you may wish 
to tweak this some.  I use FEATURES=userpriv and userfetch, so most of 
the work gets done as the portage user altho the actual qmerge step is 
obviously done as root, and MAKEOPTS="-j20 -l12" (twenty jobs, but don't 
start new ones if the load is above 12) and my load average runs pretty 
close, actually about 12.5-13 when emerging stuff.  With four cores (two 
CPUs times two cores each), that's a load average of just over three jobs 
per core.  As mentioned, I have PORTAGE_NICENESS=19.  Given all that, I 
find increasing my regular user to 2048 works about the best, keeping X 
and my various scrollers updating smoothly, amarok playing smoothly (it 
normally does) and updating its voiceprint almost smoothly (it doesn't, 
if I don't adjust the user share to 2048), etc.

This setting can be found under /sys/kernel/uids/<uid>/cpushare.  Read it 
to see what a user's share is, write it to write a new share for that 
user.  (/etc/passwd lists the user/uid correspondence.)

In ordered to have these files and this feature, you'll need to have Fair 
group CPU scheduling (FAIR_GROUP_SCHED), under General setup, and then 
its suboption, Basis for grouping tasks, set to user id 
(FAIR_USER_SCHED).  Unfortunately, the files appear as a user logs in (or 
is invoked by the system, for daemon users) and disappear as they logout, 
so it's not as simple as setting an initscript to set these up at boot 
and forgetting about it.  There's a sample script available that's 
supposed to automate setting and resetting these shares based on kernel 
events, but I couldn't get it to work last time I tried it, which was 
during the rcs I should say, so it's possible it wasn't all working just 
right, yet.  However, altering the share files manually works, and isn't 
too bad for simple changes, as long as you don't go logging in and out a 
lot, thus losing the file and the changes made to it every time there's 
nothing running as a particular user any longer.

Another setting that may be useful is the kernel's I/O scheduler.  That's 
configured under Block layer, I/O Schedulers.  CFQ (IOSCHED_CFQ) is what 
I choose.  Among other things, it automatically prioritizes I/O requests 
based on CPU job priority (altho less granularly, as there's only a 
handful of I/O priority levels compared to the 40 CPU priority levels).  
You can tweak priorities further if desired, but this seems a great 
default and is something deadline definitely doesn't have and I don't 
believe anticipatory has either.  With memory as tight as a gig, having 
the ability to batch-priority schedule i/o along with CPU is a very good 
thing, particularly when PORTAGE_NICENESS is set.  Due to the nature of 
disk I/O, it's not going to stop the worst thrashing entirely, but it 
should significantly lessen its impact, and at less than worst case, it 
will likely make the system significantly more workable than it might be 
otherwise under equivalent load.

Of course, YMMV, but I'd look at those, anyway.  It could further 
increase efficiency.

-- 
Duncan - List replies preferred.   No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master."  Richard Stallman

-- 
gentoo-amd64@lists.gentoo.org mailing list

Re: [gentoo-amd64] Re: tmpfs help

[Mateusz Mierzwinski]

Beso pisze:
>
>
> 2008/2/13, Duncan <1i5t5.duncan@cox.net <mailto:1i5t5.duncan@cox.net>>:
>
>     Volker Armin Hemmann <volker.armin.hemmann@tu-clausthal.de
>     <mailto:volker.armin.hemmann@tu-clausthal.de>> posted
>     200802131346.26316.volker.armin.hemmann@tu-clausthal.de
>     <mailto:200802131346.26316.volker.armin.hemmann@tu-clausthal.de>,
>     excerpted below,
>     on  Wed, 13 Feb 2008 13:46:26 +0100:
>
>     > On Mittwoch, 13. Februar 2008, Duncan wrote:
>     >>>removed lots of irrelevant 'my hardware is so cool' stuff'.
>     >
>     > You forget some (little) things. Not everything can be swapped
>     out. Swap
>     > is extremly slow AND it is much worse to swapout/swapin programm
>     code
>     > that should be run, instead of fetching some files from disk
>     while the
>     > programm runs.
>
>     It's not always much worse, because as I explained, in my case,
>     swap is 4-
>     way striped while most of the main system is only two-way
>     striped.  Thus,
>     that "irrelevant" stuff is relevant after all, because it alters the
>     conditions of the case in debate, because swap reads in at ~2x the
>     speed
>     of most data read off disk including apps (which is itself ~2x what a
>     single-disk system might reasonably expect).
>
>     I've a feeling not appreciating this, not appreciating that your
>     "test"
>     case example of compiling with 2 gigs RAM vs only 1 has little to
>     do with
>     what might occur with PORTAGE_TMPDIR on tmpfs vs on disk, and not
>     appreciating the point RF and I are both trying to make, is due to the
>     same logic flaw.
>
>
> well, for now, the fact for me are:
> 1. no ram upgrade is good -> notebook ram costs much more than desktop 
> one and the notebook itself has a limit
> 2. small packages, that have much update during the single days of the 
> week (i do sync once a day) get compiled at least 2-3 times faster 
> than normal compilation into disk space
> 3. big packages that usually compile in 30-40 mins now compile in 
> about 10minutes or so faster. to see how it feels to use tmpfs for 
> compilation i have to upgrade kde. usually kde3 would build into 2 
> days of about 13 hours compilation time each. i'll have to see how 
> fast would kde4 build.
> 4. the sync now takes less than 2mins while normally it would take 
> about 10mins.
> i'll try out duncan's speedups for shm and but for the dev one i don't 
> use baselayout 2 and i'm still with the 1st version, since i don't 
> feel like upgrading to it yet. but i'd like to know some more about 
> what are the improvements of the second version.
> 5. as for the raid stuff i cannot do it since i've only got one disk. 
> i'll try to see what happens with swap set to 100.
> 6. if i use some other programs while compiling into tmpfs bigones i 
> need to nice the process or i'll get some misbehaviour from the other 
> programs.
>
> -- 
> dott. ing. beso 
yeah! Notebooks limit's... My notebook have EM64T extension so it's 36 
bit physical and 48 bit virtual address space for memory mappings - that 
means I could insert more than 2 GB of memory ;). My notebook is 
designed for 4GB.. is this not much? :> Half of my current linux 
filesystem in memory (including /etc and /usr recursively) :D:D:D:D. 
Sorry, but those limits are sick for normal usage of normal citizen 
:D:D:D. I think about, what You said about memory for laptops... I have 
DDR2 memory and it cost's same as normal DDR2 memory for standard PC - 
thats how it works in poland - just You need to find right shop ;).

Greetings
Mateusz M.
-- 
gentoo-amd64@lists.gentoo.org mailing list