From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from lists.gentoo.org (pigeon.gentoo.org [208.92.234.80]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by finch.gentoo.org (Postfix) with ESMTPS id 4D8F81382C5 for ; Tue, 23 Jun 2020 18:44:42 +0000 (UTC) Received: from pigeon.gentoo.org (localhost [127.0.0.1]) by pigeon.gentoo.org (Postfix) with SMTP id 0F661E0900; Tue, 23 Jun 2020 18:44:37 +0000 (UTC) Received: from wout5-smtp.messagingengine.com (wout5-smtp.messagingengine.com [64.147.123.21]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by pigeon.gentoo.org (Postfix) with ESMTPS id A08ACE088B for ; Tue, 23 Jun 2020 18:44:36 +0000 (UTC) Received: from compute7.internal (compute7.nyi.internal [10.202.2.47]) by mailout.west.internal (Postfix) with ESMTP id AB9EDABD for ; Tue, 23 Jun 2020 14:44:35 -0400 (EDT) Received: from imap21 ([10.202.2.71]) by compute7.internal (MEProxy); Tue, 23 Jun 2020 14:44:35 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=aeam.us; h= mime-version:message-id:in-reply-to:references:date:from:to :subject:content-type; s=fm2; bh=+mmZriTvnvgqi6iLjcXn1vjYjui4NsK fYvzwOudGJD0=; b=K5q34LqMfNMD/rTrD+pJTjKo02Ps1Gnd3Or79aCXLEmHnjS XfdAMPQI1/eHVvmrimARVoF2V8IJ3e7mUQXb6t2Xmt96khZCY5uTe5NkA2sK2MPb qPpH4FhdPIbH4xQmaD524JK/vpv/dNdr7qzS/KwdrFiZbsJV9ibzFC3CPVbHXCCF IbqYp62V2mzKvSHCXo/O1TKS6Yx0LIUjnojQgrgfgFp+qHAuRrchU7mbxiiDBIve r/iEgcdO2aOCd75aLWt13eewRQatMJAWbI0AbHW2G3QxsasaSla+OvDe65gW4zfn CXbM/LiPlp5t2vz2FZLPoVH8soGVv+E6fSG/UYQ== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=content-type:date:from:in-reply-to :message-id:mime-version:references:subject:to:x-me-proxy :x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s=fm3; bh=+mmZri Tvnvgqi6iLjcXn1vjYjui4NsKfYvzwOudGJD0=; b=n2epuBvHCl86D8r1J6JWRb NbzFnbCqE9boIJ0+kwrk7uIw8oOsD1c+tO9aImeFBh+yBwe8Zu3qlAv2/sLNRhS+ zA86T8glhK6BGXoW7UlB3keQCgzypfHspkc38hsdKeNWwc2QNuENrjJp+MLt9Msu EiI9XPgix1ZuoTO/FOJiceYBmPSv+KJhiND00I+ntny15GQoOr4CJSfPduFXJPip WqlJLHHvv/vhUVPM2GaYQrZawV0acN/dvOAHxHxo7qYtmRIsgNm8K4BYiswDGTZy MLi7+Ic+QHdjfY+MG9Ft3CfsbDRl9VliN9oYF76cjkKYb6JL/SnVpprH873jQdag == X-ME-Sender: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgeduhedrudekhedguddtudcutefuodetggdotefrod ftvfcurfhrohhfihhlvgemucfhrghsthforghilhdpqfgfvfdpuffrtefokffrpgfnqfgh necuuegrihhlohhuthemuceftddtnecunecujfgurhepofgfggfkjghffffhvffutgesth dtredtreertdenucfhrhhomhepfdfuihguucfuphhrhidfuceoshhiugesrggvrghmrdhu sheqnecuggftrfgrthhtvghrnhepueekfeekgfegjeejveekgeffvdfggfeuteeufeeghe ejhffhkeefjeeivefhveelnecuffhomhgrihhnpeifihhkihhpvgguihgrrdhorhhgnecu vehluhhsthgvrhfuihiivgeptdenucfrrghrrghmpehmrghilhhfrhhomhepshhiugesrg gvrghmrdhush X-ME-Proxy: Received: by mailuser.nyi.internal (Postfix, from userid 501) id 7F44066007F; Tue, 23 Jun 2020 14:44:34 -0400 (EDT) X-Mailer: MessagingEngine.com Webmail Interface User-Agent: Cyrus-JMAP/3.3.0-dev0-543-gda70334-fm-20200618.004-gda703345 Precedence: bulk List-Post: List-Help: List-Unsubscribe: List-Subscribe: List-Id: Gentoo Linux mail X-BeenThere: gentoo-user@lists.gentoo.org Reply-to: gentoo-user@lists.gentoo.org X-Auto-Response-Suppress: DR, RN, NRN, OOF, AutoReply Mime-Version: 1.0 Message-Id: <8be8c914-dccd-4ccb-b6f2-31f1585765a9@www.fastmail.com> In-Reply-To: References: <6d77acb3-5754-06cb-b8ef-2f1a5d7d8084@gmail.com> <5EE8A6C9.9020900@youngman.org.uk> <5611481.lOV4Wx5bFT@lenovo.localdomain> <4f35306e-e509-4cae-907d-7eb026c008e9@www.fastmail.com> Date: Tue, 23 Jun 2020 13:44:14 -0500 From: "Sid Spry" To: gentoo-user@lists.gentoo.org Subject: Re: [gentoo-user] Testing a used hard drive to make SURE it is good. Content-Type: text/plain X-Archives-Salt: 7b6998cf-e770-4e17-aa62-04ce9d1c492d X-Archives-Hash: a6345aaecfdf70d1e0bc4cdfa636cab2 On Tue, Jun 23, 2020, at 12:20 PM, Rich Freeman wrote: > On Tue, Jun 23, 2020 at 12:14 PM Sid Spry wrote: > > > > So if I'm understanding properly most drive firmware won't let you > > operate the device in an append-only mode? > > So, there are several types of SMR drives. > > There are host-managed, drive-managed, and then hybrid devices that > default to drive-managed for compatibility reasons but the host can > send them a command to take full control so that it is the same as > host-managed. > > A host-managed drive just does what the host tells it to. If the host > tells it to do a write that obliterates some other data on the disk, > the drive just does it, and it is the job of the host > OS/filesystem/application to make sure that they protect any data they > care about. At the drive level these perform identically to CMR > because they just seek and write like any other drive. At the > application level these could perform differently since the > application might end up having to work around the drive. However, > these drives are generally chosen for applications where this is not a > big problem or where the problems can be efficiently mitigated. > > A drive-managed drive just looks like a regular drive to the host, and > it ends up having to do a lot of read-before-rewrite operations > because the host is treating it like it is CMR but the drive has to > guarantee that nothing gets lost. A drive-managed disk has no way to > operate in "append-only" mode. I'm not an expert in ATA but I believe > disks are just given an LBA and a set of data to write. Without > support for TRIM the drive has no way to know if it is safe to > overwrite nearby cylinders, which means it has to preserve them. > Yeah, this is what I was wondering. It looks like there are devices that do management that keeps you from using them at their full performance. > The biggest problem is that the vendors were trying to conceal the > nature of the drives. If they advertised TRIM support it would be > pretty obvious they were SMR. > It looks like I was right then. Maybe the market will settle soon and I will be able to buy properly marked parts. It's a good thing I stumbled into this, I was going to be buying more storage shortly. > > If any do I suspect > > NILFS (https://en.wikipedia.org/wiki/NILFS) may be a good choice: > > > > "NILFS is a log-structured filesystem, in that the storage medium is treated > > like a circular buffer and new blocks are always written to the end. [...]" > > > > On a host-managed disk this would perform the same as on a CMR disk, > with the possible exception of any fixed metadata (I haven't read the > gory details on the filesystem). If it has no fixed metadata (without > surrounding unused tracks) then it would have no issues at all on SMR. > F2FS takes a similar approach for SSDs, though it didn't really take > off because ext4's support is good enough and I suspect that modern > SSDs are fast enough at erasing. > There is not really a lot to NILFS' structure save the fact it doesn't delete. It ends up being fairly similar to f2fs. On a SMR w/ TRIM drive this would imply no (or very little) penalties for write operations as all writes are actually just appends. I'm not sure the impact it would have on seek/read time with a normal workload but some people report slight improvements on SMR drives, especially if they are helium filled, as the denser packing and lighter gas lead to increased read speeds. Actually, f2fs may be the better choice, I had almost forgotten about it. Bigger rollout and more testing. I would need to check feature set in more detail to make a choice. --- Weirdly benchmarking tends to show f2fs as deficient to ext4 in most cases. For a while I've been interested in f2fs (or now nilfs) as a backing block layer for a real filesystem. It seems to have a better data model than some tree based filesystems, but I think we are seeing filesystems suck up features like snapshotting and logging as a must-have instead of the older LVM/raid based "storage pipelines." But then, this is just reimplementing a smart storage controller on your CPU, though that may be the best place for it.