Re: [gentoo-user] Testing a used hard drive to make SURE it is good.

["Sid Spry" <sid@aeam.us>]

On Tue, Jun 23, 2020, at 12:20 PM, Rich Freeman wrote:
> On Tue, Jun 23, 2020 at 12:14 PM Sid Spry <sid@aeam.us> wrote:
> >
> > So if I'm understanding properly most drive firmware won't let you
> > operate the device in an append-only mode?
> 
> So, there are several types of SMR drives.
> 
> There are host-managed, drive-managed, and then hybrid devices that
> default to drive-managed for compatibility reasons but the host can
> send them a command to take full control so that it is the same as
> host-managed.
> 
> A host-managed drive just does what the host tells it to.  If the host
> tells it to do a write that obliterates some other data on the disk,
> the drive just does it, and it is the job of the host
> OS/filesystem/application to make sure that they protect any data they
> care about.  At the drive level these perform identically to CMR
> because they just seek and write like any other drive.  At the
> application level these could perform differently since the
> application might end up having to work around the drive.  However,
> these drives are generally chosen for applications where this is not a
> big problem or where the problems can be efficiently mitigated.
> 
> A drive-managed drive just looks like a regular drive to the host, and
> it ends up having to do a lot of read-before-rewrite operations
> because the host is treating it like it is CMR but the drive has to
> guarantee that nothing gets lost.  A drive-managed disk has no way to
> operate in "append-only" mode.  I'm not an expert in ATA but I believe
> disks are just given an LBA and a set of data to write.  Without
> support for TRIM the drive has no way to know if it is safe to
> overwrite nearby cylinders, which means it has to preserve them.
> 

Yeah, this is what I was wondering. It looks like there are devices that
do management that keeps you from using them at their full
performance.

> The biggest problem is that the vendors were trying to conceal the
> nature of the drives.  If they advertised TRIM support it would be
> pretty obvious they were SMR.
> 

It looks like I was right then. Maybe the market will settle soon and I
will be able to buy properly marked parts. It's a good thing I stumbled
into this, I was going to be buying more storage shortly.

> > If any do I suspect
> > NILFS (https://en.wikipedia.org/wiki/NILFS) may be a good choice:
> >
> > "NILFS is a log-structured filesystem, in that the storage medium is treated
> > like a circular buffer and new blocks are always written to the end. [...]"
> >
> 
> On a host-managed disk this would perform the same as on a CMR disk,
> with the possible exception of any fixed metadata (I haven't read the
> gory details on the filesystem).  If it has no fixed metadata (without
> surrounding unused tracks) then it would have no issues at all on SMR.
> F2FS takes a similar approach for SSDs, though it didn't really take
> off because ext4's support is good enough and I suspect that modern
> SSDs are fast enough at erasing.
> 

There is not really a lot to NILFS' structure save the fact it doesn't delete.
It ends up being fairly similar to f2fs. On a SMR w/ TRIM drive this would
imply no (or very little) penalties for write operations as all writes are
actually just appends. I'm not sure the impact it would have on seek/read
time with a normal workload but some people report slight improvements
on SMR drives, especially if they are helium filled, as the denser packing
and lighter gas lead to increased read speeds.

Actually, f2fs may be the better choice, I had almost forgotten about it.
Bigger rollout and more testing. I would need to check feature set in more
detail to make a choice.

---

Weirdly benchmarking tends to show f2fs as deficient to ext4 in most
cases.

For a while I've been interested in f2fs (or now nilfs) as a backing block
layer for a real filesystem. It seems to have a better data model than some
tree based filesystems, but I think we are seeing filesystems suck up
features like snapshotting and logging as a must-have instead of the
older LVM/raid based "storage pipelines."

But then, this is just reimplementing a smart storage controller on your
CPU, though that may be the best place for it.