From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from lists.gentoo.org (pigeon.gentoo.org [208.92.234.80]) by finch.gentoo.org (Postfix) with ESMTP id AC2561381F3 for ; Sat, 31 Aug 2013 10:53:46 +0000 (UTC) Received: from pigeon.gentoo.org (localhost [127.0.0.1]) by pigeon.gentoo.org (Postfix) with SMTP id 1EAE7E0CEB; Sat, 31 Aug 2013 10:53:43 +0000 (UTC) Received: from plane.gmane.org (plane.gmane.org [80.91.229.3]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by pigeon.gentoo.org (Postfix) with ESMTPS id 0DEDFE0CE1 for ; Sat, 31 Aug 2013 10:53:41 +0000 (UTC) Received: from list by plane.gmane.org with local (Exim 4.69) (envelope-from ) id 1VFioG-0008LE-9q for gentoo-amd64@lists.gentoo.org; Sat, 31 Aug 2013 12:53:40 +0200 Received: from ip68-231-22-224.ph.ph.cox.net ([68.231.22.224]) by main.gmane.org with esmtp (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Sat, 31 Aug 2013 12:53:40 +0200 Received: from 1i5t5.duncan by ip68-231-22-224.ph.ph.cox.net with local (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Sat, 31 Aug 2013 12:53:40 +0200 X-Injected-Via-Gmane: http://gmane.org/ To: gentoo-amd64@lists.gentoo.org From: Duncan <1i5t5.duncan@cox.net> Subject: [gentoo-amd64] Re: Hard drive (installation) Date: Sat, 31 Aug 2013 10:53:22 +0000 (UTC) Message-ID: References: <521E4E74.9020005@jamadots.com> < pan$6ef98$70058058$78799247$722cd31d@cox.net> < CAGfcS_k46L32WaSodeE=hBCepw7dDLTUo3dPeo6ZGfV+ouVu+Q@mail.gmail.com> Precedence: bulk List-Post: List-Help: List-Unsubscribe: List-Subscribe: List-Id: Gentoo Linux mail X-BeenThere: gentoo-amd64@lists.gentoo.org Reply-to: gentoo-amd64@lists.gentoo.org Mime-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-Complaints-To: usenet@ger.gmane.org X-Gmane-NNTP-Posting-Host: ip68-231-22-224.ph.ph.cox.net User-Agent: Pan/0.140 (Chocolate Salty Balls; GIT 3b8c3f7 /usr/src/portage/src/egit-src/pan2) X-Archives-Salt: 30f934e4-5a5f-45ed-bb3c-7c03bcf878e5 X-Archives-Hash: 49cf223ce2947f3f7b89b9f907b83a71 Rich Freeman posted on Fri, 30 Aug 2013 21:50:47 -0400 as excerpted: > Anybody with a motherboard supporting USB2 almost certainly had a > motherboard supporting PATA at a faster transfer rate. [Long winded and thread tangential. TL;DR readers just skip it, but I know some people find these things interesting/useful. There's some SSD deployment discussion further down that I certainly could have used a few months ago!] Well, yes, but... With my 8-year-old native USB1 (with a USB2 addon card, it wouldn't do USB3 as it didn't have PCIE, only PCI-X, dual socket original 3-digit Opteron maxed out at dual Opteron-290, which are dual-core @ 2.8 GHz, so it was effectively quad-core at 2.8, not /too/ shabby, just getting seriously dated in terms of buses, etc) system that died last year, I setup an unbootable USB external spinning rust drive as a backup, with a USB thumbdrive on the bootable native USB1 for /boot to load the kernel, which could then handle the PCI-card USB2 that the BIOS couldn't, and thus get to the backup root on the external USB2 spinning rust. That's how I know that not all external USB devices are bootable, as that external drive wasn't, even when I had it on the bios-supported native USB1, thus the earlier warning about that, and how to work around it. But the point is, that setup wasn't /significantly/ slower in normal operation than my fancy multi-disk md-raid, and in fact, was NOTICEABLY faster than my original mdraid-6 setup (thus the discussion on it a few months ago), tho mdraid-1 was a bit faster after I switched to it, particularly for multi-thread read as typically happens during boot. Now for transferring hundreds of megabytes (as when actually making the backups), yeah, doing that over the USB2 was a bit slow, roughly comparable to backing up to a different partition on the same spindle internally (with the disk seeks that means, except that backing up to the external was to a separate external physical device so without the USB2 bottleneck it should have been faster). However, for ordinary use, the 6 gigs of RAM (at one point 8, two 2-gig sticks per socket, but one stick died and I never replaced it) was plenty to cover both my normal memory usage and working set disk cache with reasonable margin to spare as I normally run only 2-3 gigs apps+buffers+cache (top line of free), except when I'm doing updates or large media files and thus have all that cached too. But even cold-booting off the USB2 external was enough faster than the mdraid-6 to demonstrate how bad mdraid6 can be, and convince me to switch to mdraid-1, which was indeed faster. > I do agree that random access speed does lower the effective rate. My > hard drives are running at 3GB/s transfer rates each on a dedicated > channel, and yet they're probably not any faster than they would have > been under PATA (assuming one drive per cable). > > Hopefully one of these days there will be a decent SSD cache option for > Linux. Bcache is still fairly experimental, and I'm not sure how well > it performs in practice with btrfs - plus it is a device layer and not > filesystem layer implementation (ie if you have mirrored drives you end > up with mirrored cache which seems a bit dumb, especially if the mirrors > end up being on separate partitions on the same device). Here, I ended up putting root and home (and log and the gentoo tree and overlays and...) on btrfs on SSD, with, effectively, only my media (and secondary backups) remaining on spinning rust, as I couldn't justify the cost to put it on SSD. A 64-gig SSD is now well below the best-price-point knee at least around here, and I calculated that would be minimal for working copy and primary backup of the OS and /home. However, once I looked at prices (as of a few months ago), I found 128 gig SSDs were actually the low end of the price-point-knee, with 256 gig the high end, and decided the over- provisioning would be a good idea due to the limited write-cycle issue on multi-level SSDs, so actually planned on that. But when I went in to Fry's Electronics to buy, turned out all their good deals on 128 gig SSDs were selling out faster than they could get them in stock, and they happened to be out of the good deals there. I use public transit and didn't feel like going home empty handed only to come back the day the truck came in, so I ended up with 256 gig SSDs, still at the price-point-knee tho at the upper end of it, and was able to put even MORE on the SSDs -- while still keeping a very healthy nearly 100% over- provisioning, tho I expect that might dwindle a bit over time. (With 16 GiB RAM I decided I didn't need swap.) I did spend a bit more than planned, buying three of them, two for the main machine, now mostly configured in btrfs raid1 mode in ordered to actually make use of btrfs' data integrity checksumming and scrub abilities, with the third to eventually be used in my netbook (which being SATA2 will bottleneck on that, but the slower ones weren't significantly cheaper, and I expect it'll outlast my netbook, to be used either in my main machine or a netbook replacement, later, at which point I'll probably actually use the full speed, as I do with the other two now). But I've been working extra hours and will have it paid off ~three months from purchase so not so much interest, and I'm happy with it. The point being... if you do your partitioning right, you can get 90% of the benefits of full SSD at *FAR* more reasonable costs than full SSD. Basically SSD caching, only without the cache, simply putting the stuff that will truly benefit from SSD on SSD, while leaving the stuff that won't on cheaper spinning rust. I calculated that I only NEEDED about 64 GB for the most important stuff, with backups on spinning rust, and that would have left a healthy over-provisioning too. (I figured I could fit what I really wanted on SSD in 32 gig if I had to, but it would have been a pretty tight fit!) Here's my gdisk -l output for one of the SSD pair (I've been running GUID Partition Tables, GPT, for some time now, some boilerplate omitted for posting): Disk /dev/sda: 500118192 sectors, 238.5 GiB Partition table holds up to 128 entries Partitions will be aligned on 2048-sector boundaries Total free space is 246364781 sectors (117.5 GiB) No. Start(sector) End(sector) Size Code Name 1 2048 8191 3.0 MiB EF02 bi0238gcn1+35l0 2 8192 262143 124.0 MiB EF00 ef0238gcn1+35l0 3 262144 786431 256.0 MiB 8300 bt0238gcn1+35l0 4 786432 2097151 640.0 MiB 8300 lg0238gcn1+35l0 5 2097152 18874367 8.0 GiB 8300 rt0238gcn1+35l0 6 18874368 60817407 20.0 GiB 8300 hm0238gcn1+35l0 7 60817408 111149055 24.0 GiB 8300 pk0238gcn1+35l0 8 111149056 127926271 8.0 GiB 8300 nr0238gcn1+35l0 9 127926272 144703487 8.0 GiB 8300 rt0238gcn1+35l1 10 144703488 186646527 20.0 GiB 8300 hm0238gcn1+35l1 11 186646528 236978175 24.0 GiB 8300 pk0238gcn1+35l1 12 236978176 253755391 8.0 GiB 8300 nr0238gcn1+35l1 I wanted my partitions 4 MiB aligned for efficient erase-block handling, tho the first one's only 1 MiB aligned. One of the features of GPT partitioning is that it allows partition names/labels much like filesystems normally do. That's what's shown in the last column. I have a standard naming scheme, developed back when I was running multiple mdraid devices, some of which were themselves partitioned, on a 4-spindle set, that I use for both my partition labels and my filesystem labels, for both my main machine and my netbook: * 15 characters long 123456789012345 ff bbB ymd ssssS t n Using a different example from my mdraid days: rt0005gmd3+9bc0 ff: 2-char function abbreviation bi=bios (gpt dedicated legacy BIOS boot partition, used by grub2). ef=efi (gpt dedicated efi partition, unused here ATM but reserved for forward compatibility) bt=boot, lg=log, rt=root, hm=home, lg=log, pk=package (gentoo tree, layman overlays, sources, binpkgs, kernel tree), nr=netbook-root (separate 32-bit chroot build-image filesystem/partition). Example rt=root Device-ID consisting of ssssSbbB: ssssS: 4-digit size, 1-char multiplier. This is the size of the underlying/containing media, NOT the partition/filesystem (I'm IDing the containing device). bbB: 2-char media brand ID, 1-digit sequence number. The md example is a 5 GiB mdraid volume (/dev/md3), which might itself be partitioned (IIRC I was keeping /usr/local/ on a separate filesystem/ partition on the same mdraid as root, back then). In the above output, 0238 GiB, corsair neutron. The paired SSDs are 0 and 1 (0 is installed as sdb, 1 as sda, as I reversed them in installation). So the device-ids are 0238gcn1 and 0238gcn0, with the common btrfs on top of both device-IDed as 0238gcnx. t: single-char target/separator. This serves as both a target ID and a visual separator. I use . for my netbook (dot 'cause it's small), + for the main machine, and % for portable disk partitions intended to be used on both. Both the output and the md example are for my main machine, so +. ymd: 1-char-each year/month/day. y=last digit of year (I might use storage devices for years but it's unlikely I'll need to track decade wrap). m=month (1-9abc) d=day (1-9a-v) This is generally the day I setup the partition. Back when I was still on MBR I used to relabel the filesystem on my backup partitions with the new date when I did a mkfs and a clean backup, but when I switched to GPT and could label the partitions too I decided to keep them date-matched as my fstab uses LABEL= mounting, and it was a hassle updating the fstab when I blew away an old backup and redid it. The md example is 2009-11-12. The table is using 2013-05-21. n: 1-digit copy number. The working copy is 0, primary backup 1... The md example is the working copy. The table has the working copy and for some partitions the primary backup. mdraid example all together: rt0005gmd3+9bc0 rt root 0005gmd3 5-gig /dev/md3 + for use on the main machine 9bc 2009-11-12 (Nov 12) 0 working copy So from the above you can see I have a 3 MiB BIOS partition (grub2 uses, per-drive, size chosen to put all later partitions on 4 MiB boundaries), a 124 MiB EFI partition (reserved for future use, later partitions are now on 128 MiB boundaries). A 256 MiB /boot comes next. (This is a separate filesystem for each drive, the second one for backup, working copy updated every time I install a new kernel, which I do a lot as I run git kernels, backup updated once per cycle with the 3.x.0 release, I can direct the BIOS at either one or at the spinning rust /boot, my secondary /boot backup). That's followed by /var/log at 640 MiB, leaving me at 1 GiB boundaries. This partition and later are btrfs raid1 mode, but I only keep the working copy of log, not the working and backup copies I keep for everything else. Then come the root (8 GiB), home (20 GiB), package (24 GiB), and netbook- root-build-image (8 GiB) partitions, working copies followed by primary backups. Secondary backups and media remain as I said on spinning rust. That's the bulk of my data but it's also the least speed-critical. As you can see with a bit of math (I can cheat here and just look it up in gdisk), that's 121 GiB used, 117.5 GiB free, just worse than 50/50, so near 100% over-provisioning. I shouldn't have to worry too much about write-cycle wear-out with that, even if I add a few gigs of partitions later. (Recommended SSD overprovisioning is 25-33%, thus actually using 3/4-4/5. I'm barely over half! =:^) With a bit more squeezing I could have fit home and both main root and netbook root on a 32-gig with no overprovisioning, and a 64 gig would have fit 1 copy of everything listed, with the primary backups on spinning rust, and that or an 80 gig to allow a bit of overprovisioning is what I was originally targeting. Then I looked at the prices and saw 128 gigs at not /that/ much over 80 gigs and at a cheaper per-gig, so that's what I actually thought I'd buy. But I'm not complaining about the 256 gig (238 GiB, unfortunately they're carrying over the marketing practices from spinning rust, I AM complaining about that!), and I'm *DEFINITELY* not complaining about boot or emerge --sync speed on the SSDs! =:^) -- Duncan - List replies preferred. No HTML msgs. "Every nonfree program has a lord, a master -- and if you use the program, he is your master." Richard Stallman