[gentoo-user] defect management block device

[gentoo-user] defect management block device

[Enrico Weigelt]

Hi folks,


does anyone know an (virtual) block device which can do automatic
defect management (if the underlying disks have badblocks) ?

My idea goes like this: 
* one or more devices are assigned to one block device
* a bunch of spare blocks are reserved for defect management 
  (so the device looks smaller than the sum of assigned disks)
* if an badblock is detected, it's automatically remapped 
  to an spare block
  
In fact, just what drive-internal defect manangement does, but
at OS / driver level.


cu
-- 
---------------------------------------------------------------------
 Enrico Weigelt    ==   metux IT service - http://www.metux.de/
---------------------------------------------------------------------
 Please visit the OpenSource QM Taskforce:
 	http://wiki.metux.de/public/OpenSource_QM_Taskforce
 Patches / Fixes for a lot dozens of packages in dozens of versions:
	http://patches.metux.de/
---------------------------------------------------------------------
-- 
gentoo-user@lists.gentoo.org mailing list

Re: [gentoo-user] defect management block device

[Alan McKinnon]

On Monday 24 March 2008, Enrico Weigelt wrote:
> Hi folks,
>
>
> does anyone know an (virtual) block device which can do automatic
> defect management (if the underlying disks have badblocks) ?
>
> My idea goes like this:
> * one or more devices are assigned to one block device
> * a bunch of spare blocks are reserved for defect management
>   (so the device looks smaller than the sum of assigned disks)
> * if an badblock is detected, it's automatically remapped
>   to an spare block
>
> In fact, just what drive-internal defect manangement does, but
> at OS / driver level.

I don't see the point, unless you are dealing with drives that do not 
have defect management.

What makes you think you can accomplish this result better than the 
firmware on the drive? It seems to me that if the drive firmware missed 
the opportunity to relocate the bad block, then your window of 
opportunity to do it in your code has long since passed. IOW, the OS 
code cannot possibly ever achieve it's design result.

Just a thought, maybe you know some aspect of disks that I don't and can 
see where this would be useful. From where I sit, I can;t see any such 
use-case.


-- 
Alan McKinnon
alan dot mckinnon at gmail dot com

-- 
gentoo-user@lists.gentoo.org mailing list

Re: [gentoo-user] defect management block device

[Eric Martin]

Alan McKinnon wrote:
> On Monday 24 March 2008, Enrico Weigelt wrote:
>   
>> does anyone know an (virtual) block device which can do automatic
>> defect management (if the underlying disks have badblocks) ?
>>
>> My idea goes like this:
>> * one or more devices are assigned to one block device
>> * a bunch of spare blocks are reserved for defect management
>>   (so the device looks smaller than the sum of assigned disks)
>> * if an badblock is detected, it's automatically remapped
>>   to an spare block
>>
>> In fact, just what drive-internal defect manangement does, but
>> at OS / driver level.
>>     
>
> I don't see the point, unless you are dealing with drives that do not 
> have defect management.
>
> What makes you think you can accomplish this result better than the 
> firmware on the drive? It seems to me that if the drive firmware missed 
> the opportunity to relocate the bad block, then your window of 
> opportunity to do it in your code has long since passed. IOW, the OS 
> code cannot possibly ever achieve it's design result.
>
> Just a thought, maybe you know some aspect of disks that I don't and can 
> see where this would be useful. From where I sit, I can;t see any such 
> use-case.
>
>   

While I see what Alan is saying, I'm pretty sure LVM does it.  Device 
Drivers -> Multiple Devices Driver Support -> Bad Block Relocation 
Device Target.  I've never played with it but I assume there's a lot of 
good reading on it.

-- 
HTH, Eric
-- 
gentoo-user@lists.gentoo.org mailing list

Re: [gentoo-user] defect management block device

[Alan McKinnon]

On Monday 24 March 2008, Eric Martin wrote:
> > Just a thought, maybe you know some aspect of disks that I don't
> > and can see where this would be useful. From where I sit, I can;t
> > see any such use-case.
> >
> >  
>
> While I see what Alan is saying, I'm pretty sure LVM does it.  Device
> Drivers -> Multiple Devices Driver Support -> Bad Block Relocation
> Device Target.  I've never played with it but I assume there's a lot
> of good reading on it.

Which gives me an idea on where such a thing might be useful - RAID

If SMART (or something conceptually similar) detects that a drive might 
be failing and be beyond the range of the drive's ability to cope, it 
could raise an event and move the blocks used to another disk. This 
could give another level of data protection rather than just simply 
relying on multiple copies as current RAID schemes mostly do

-- 
Alan McKinnon
alan dot mckinnon at gmail dot com

--
gentoo-user@lists.gentoo.org mailing list

Re: [gentoo-user] defect management block device

[Enrico Weigelt]

* Alan McKinnon <alan.mckinnon@gmail.com> wrote:

> If SMART (or something conceptually similar) detects that a drive might 
> be failing and be beyond the range of the drive's ability to cope, it 
> could raise an event and move the blocks used to another disk.

And it even would get funnier if the drive's relocation table 
could be accessed (no idea if this is possible): 
The LVM would notice if the drive has relocated an (LBA) block,
move it out of the way (somewhere else in the LV) and then 
remove the relocation (never access that LBA block anymore). 
This way an slowly dying disk can be used for quite a long time.
Think of boxes with very limited physical access (eg. outoor field 
systems) or huge archives w/ non-critical/regeneratable data
(eg. media collections w/ originals available, mirrors, etc).

The idea of using even old and damaged disks at really low costs
(not counting the power consumption ;-P) is seems quite fascinating
to me :)


cu
-- 
---------------------------------------------------------------------
 Enrico Weigelt    ==   metux IT service - http://www.metux.de/
---------------------------------------------------------------------
 Please visit the OpenSource QM Taskforce:
 	http://wiki.metux.de/public/OpenSource_QM_Taskforce
 Patches / Fixes for a lot dozens of packages in dozens of versions:
	http://patches.metux.de/
---------------------------------------------------------------------
-- 
gentoo-user@lists.gentoo.org mailing list

Re: [gentoo-user] defect management block device

[Alan McKinnon]

On Tuesday 25 March 2008, Enrico Weigelt wrote:
> * Alan McKinnon <alan.mckinnon@gmail.com> wrote:
> > If SMART (or something conceptually similar) detects that a drive
> > might be failing and be beyond the range of the drive's ability to
> > cope, it could raise an event and move the blocks used to another
> > disk.
>
> And it even would get funnier if the drive's relocation table
> could be accessed (no idea if this is possible):
> The LVM would notice if the drive has relocated an (LBA) block,
> move it out of the way (somewhere else in the LV) and then
> remove the relocation (never access that LBA block anymore).
> This way an slowly dying disk can be used for quite a long time.
> Think of boxes with very limited physical access (eg. outoor field
> systems) or huge archives w/ non-critical/regeneratable data
> (eg. media collections w/ originals available, mirrors, etc).
>
> The idea of using even old and damaged disks at really low costs
> (not counting the power consumption ;-P) is seems quite fascinating
> to me :)

The tricky part is to figure out where a concept like this fits in the 
various abstraction layers. You couldn't build it into LVM, as LVM 
doesn't really know about individual blocks on the disk, it only works 
with it's own segments. The disk driver itself only really understands 
it's own IDE/SCSI commands and manipulates the data at the sector 
level. The file system can work at the block level.

So there are at least 3 different allocation unit sizes and no obvious 
way to write one driver that spans all three. The one idea that keeps 
coming back to me is a new function in the kernel where a driver for a 
physical storage device can fire an event if it detects a failing unit, 
and other drivers can register to receive those events. When it gets 
one, the high level driver can decide if it should move it's own blocks 
of data around. In a way, conceptually similar to a GUI event 
framework.

Could be very useful to a niche market - Linux already has 1000s of 
those :-) - but would require very careful design to not break 
everything else in the system
-- 
Alan McKinnon
alan dot mckinnon at gmail dot com

-- 
gentoo-user@lists.gentoo.org mailing list