From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from pigeon.gentoo.org ([69.77.167.62] helo=lists.gentoo.org) by finch.gentoo.org with esmtp (Exim 4.60) (envelope-from ) id 1JPJLc-0002An-Nu for garchives@archives.gentoo.org; Wed, 13 Feb 2008 15:16:33 +0000 Received: from pigeon.gentoo.org (localhost [127.0.0.1]) by pigeon.gentoo.org (Postfix) with SMTP id 5F63CE07C3; Wed, 13 Feb 2008 15:16:30 +0000 (UTC) Received: from vms040pub.verizon.net (vms040pub.verizon.net [206.46.252.40]) by pigeon.gentoo.org (Postfix) with ESMTP id 31204E07C3 for ; Wed, 13 Feb 2008 15:16:30 +0000 (UTC) Received: from gw.thefreemanclan.net ([72.81.8.99]) by vms040.mailsrvcs.net (Sun Java System Messaging Server 6.2-6.01 (built Apr 3 2006)) with ESMTPA id <0JW60046DNVARAF0@vms040.mailsrvcs.net> for gentoo-amd64@lists.gentoo.org; Wed, 13 Feb 2008 09:18:47 -0600 (CST) Received: from [127.0.0.1] (localhost [127.0.0.1]) by gw.thefreemanclan.net (Postfix) with ESMTP id B71F112418F for ; Wed, 13 Feb 2008 10:16:27 -0500 (EST) Date: Wed, 13 Feb 2008 10:16:27 -0500 From: Richard Freeman Subject: Re: [gentoo-amd64] tmpfs help In-reply-to: <200802130404.32122.volker.armin.hemmann@tu-clausthal.de> To: gentoo-amd64@lists.gentoo.org Message-id: <47B309CB.9060101@gentoo.org> Precedence: bulk List-Post: List-Help: List-Unsubscribe: List-Subscribe: List-Id: Gentoo Linux mail X-BeenThere: gentoo-amd64@lists.gentoo.org Reply-to: gentoo-amd64@lists.gentoo.org MIME-version: 1.0 Content-type: text/plain; charset=ISO-8859-15; format=flowed Content-transfer-encoding: 7bit References: <200802130149.14808.volker.armin.hemmann@tu-clausthal.de> <47B2451A.5020505@gentoo.org> <200802130404.32122.volker.armin.hemmann@tu-clausthal.de> User-Agent: Thunderbird 2.0.0.9 (X11/20071116) X-Archives-Salt: f1a177e1-6400-4109-a5de-8fabb9b433ef X-Archives-Hash: 8ef4cb52f232df8896dc3adf36c597aa Volker Armin Hemmann wrote: > On Mittwoch, 13. Februar 2008, Richard Freeman wrote: >> Do you have benchmarks to support this? > > which numbers? that swap is horrible slow compared to ram? > No - that compiling in tmpfs is horrible compared to compiling to disk. Neither of us is proposing getting rid of disk access entirely - we're just debating two different ways of doing it. > think about this: instead of having the compiler in ram and files on disk, you > have to swap the compiler out, read the files from swap into ram, swap the > compiler in, do some stuff, repeat. Slowdown. > Uh, every file that you compile has to be read into RAM and then binaries have to be written back to disk with your proposal. With swap stuff only gets written to disk if RAM gets low - which in the worst case is equivalent to writing everything to disk - which is your typical case. The kernel would not be swapping the compiler in and out in these kinds of circumstances - it would probably be swapping in and out the tmpfs. However, if some pages in the compiler are completely idle they might get swapped out, making more RAM available for tmpfs use. > > nope. Stuff that is written and deleted some seconds later never hit the disk. > That is why cache&buffer exist. If "some seconds" is greater than about 10 it will be flushed to disk 100% of the time with most filesystems. The exact figures probably vary by filesystem. Most filesystems sync their buffers within some time after they are written. tmpfs does not have this feature - which is good if you care about your files and bad if you don't. >> >> As opposed to wasting all your RAM on cache&buffers to hold all those >> files being accessed? During compilation that RAM is going to be >> heavily used - no getting around that. Once you're done any files left >> sitting on a tmpfs will just get paged out until accessed. They >> shouldn't really use any RAM at all. Even if those files were on disk >> they would consume RAM in the form of caching until they're considered >> unneeded. > > cached&buffered information can be discarded anytime if the ram is needed > elsewhere. tmpfs has to be shoved into swap. > Uh, data written to a file has to be written to disk before the cache/buffer is flushed. The difference is that the data is written to disk within about 10 seconds of being written to the buffer, and then retained in cache maybe for a few hours longer (optimistically). With tmpfs the write doesn't happen until the space is needed. Either way every byte gets written - with tmpfs it is deferred as late as possible but with disk-based filesystems the write is done as early as possible. > > with the files on disk, the kernel can hold as much data in ram as he likes > to. with a big tmpfs (and 512mb out of 1gb) he is always shoving someting > into swap or get something from there. > You can put 6GB of data in a 7GB tmpfs filesystem on a system with 512MB of RAM and still have 400MB of free memory. tmpfs does not reserve memory - if it did it would certainly use tons of memory. > Extreme example? > > kdepim with the enablefinal flag. On amd64 every single gcc instance needs > ~900mb at two points of compilation. With 1gb ram and no tmpfs it sucks. > > With 1gb ram and 512mb of that reserved for tmpfs, you'll get a swapstorm. Oh, it will certainly swap a great deal. However, I don't see why it would be any slower - either way you're doing a ton of disk access. Do you have actual benchmarks? Again, tmpfs doesn't "reserve" memory - it uses memory - just like cache/buffers. >> I certainly agree that >> swap is slow compared to RAM, but it isn't slow compared to a disk-based >> filesystem. > > really? Every really swapped? IMHO it feels like every single byte is fetched > by a mule. > Right now I've got 589MB of free RAM (-/+ buffers/cache) with 2GB of RAM total. I've got 705MB of swap used. Works just fine IMHO. Sure, if I make it really busy it can get slow, although with nice/ionice there isn't much visible impact. And it doesn't really seem much slower using swap vs on-disk. I'd expect some swap slowdown in my case as I'm using encrypted swap, but I don't think most users will be doing that. > Of course adding more RAM will never hurt, but that incurs >> significant cost and you can at least maximize your current hardware >> before investing in more of it. > > significant costs like 30euros for 2gb? Sure. Compared to about $1 for 2GB of hard drive space 30 euros is significant. And even if he had 2GB of RAM I submit that it would probably work better if he compiled in tmpfs than on disk. There is no question that a $550 computer will outperform a $500 computer, and a $600 computer will outperform a $550 computer, and so on. You can always spend 30 more euros and improve your system. What I'm interested in is maximizing the performance of the system I have now. I can always spend $50 and make it even faster. However, there are lots of things I can spend $50 on - I'd rather spend it on something else all things being equal. -- gentoo-amd64@lists.gentoo.org mailing list