[gentoo-user] Big USB disks

[gentoo-user] Big USB disks

[Peter Humphrey]

Greetings,

Just a quickie - is there a way to enable a machine built on an MSDOS BIOS to 
recognise a partition on a 4TB external disk? I think I know the answer 
already, but just in case...

-- 
Regards,
Peter.

Re: [gentoo-user] Big USB disks

[Rich Freeman]

On Tue, Dec 22, 2020 at 7:58 AM Peter Humphrey <peter@prh.myzen.co.uk> wrote:
>
> Just a quickie - is there a way to enable a machine built on an MSDOS BIOS to
> recognise a partition on a 4TB external disk? I think I know the answer
> already, but just in case...

I suspect most newer firmwares are fine with it (even in legacy mode),
and ancient ones would need a firmware update, so good luck with that.

You might be able to mitigate the issue with a boot partition close to
the start of the disk.  I'm honestly not sure if that works with
something like a USB hard drive.  I'm not sure how old this system is
either.

Back in the day MBR patches for large disks were a pretty common
thing.  I'm not sure how much any modern OS depends on the firmware to
do disk IO once it is booted, and I'm not sure that most bootloaders
even need help from the firmware beyond maybe whatever goes in the
boot sector.  This is why some GRUB modes require a secondary
partition near the start of the disk to handle certain
filesystems/etc.  (Or at least used to - GRUB and EFI have gotten so
good that I don't pay much attention to that stuff on newer hardware
and with common filesystems, even including ZFS.)

Rich

-- 
Rich

Re: [gentoo-user] Big USB disks

[Peter Humphrey]

On Tuesday, 22 December 2020 15:32:27 GMT Rich Freeman wrote:
> On Tue, Dec 22, 2020 at 7:58 AM Peter Humphrey <peter@prh.myzen.co.uk> 
wrote:
> > Just a quickie - is there a way to enable a machine built on an MSDOS BIOS
> > to recognise a partition on a 4TB external disk? I think I know the
> > answer already, but just in case...
> 
> I suspect most newer firmwares are fine with it (even in legacy mode),
> and ancient ones would need a firmware update, so good luck with that.

As I expected...

> You might be able to mitigate the issue with a boot partition close to
> the start of the disk.  I'm honestly not sure if that works with
> something like a USB hard drive.  I'm not sure how old this system is
> either.

I wondered about that. I'm nervous, though, because this is my ultimate backup 
disk, and of course I don't want to endanger it. This disk is an external USB 
unit, not for booting from.

Mr Google tells me the model was releases in 2007. I will have bought it 
within a year or so of that.

> Back in the day MBR patches for large disks were a pretty common
> thing.  I'm not sure how much any modern OS depends on the firmware to
> do disk IO once it is booted, and I'm not sure that most bootloaders
> even need help from the firmware beyond maybe whatever goes in the
> boot sector.  This is why some GRUB modes require a secondary
> partition near the start of the disk to handle certain
> filesystems/etc.  (Or at least used to - GRUB and EFI have gotten so
> good that I don't pay much attention to that stuff on newer hardware
> and with common filesystems, even including ZFS.)

Thanks for the extra detail, Rich.

-- 
Regards,
Peter.

Re: [gentoo-user] Big USB disks

[Rich Freeman]

On Tue, Dec 22, 2020 at 11:36 AM Peter Humphrey <peter@prh.myzen.co.uk> wrote:
>
> I wondered about that. I'm nervous, though, because this is my ultimate backup
> disk, and of course I don't want to endanger it. This disk is an external USB
> unit, not for booting from.
>

Oh, if you aren't booting from it then I don't think the firmware
would be involved at all.  Only the OS should matter, so as long as
that is something modern you should be fine (not sure how linux from
1998 handles a 14TB USB drive).

Your biggest issue is probably going to be that if you have a lot of
data to back up then it will take forever if the system only has USB2.

I'm actually storing a lot of my data on USB3 external drives now with
lizardfs.  With Pi4s having 2x USB3 hosts you can keep up to four
spinning disks near-100% occupied, and this is mostly for static data
so I could handle more disks than that (rebuilds are going to be
limited by the gigabit LAN port).  I was using LSI HBAs but have been
having SATA errors with those - I suspect that the ones you can get
cheap on eBay have a LOT of hours on them and are getting flaky - plus
one of those HBAs probably pulls more power than a dozen Pis anyway.
I wouldn't do this with older Pi models due to the USB being far more
limited (USB2 only, and the LAN was shared with that too) - there are
other SBCs that are options as well.

Obviously none of this is going to be competing with block storage
solutions on SSD/NVMe.  This is just for media/etc where capacity and
redundancy and cheap matters most.

-- 
Rich

Re: [gentoo-user] Big USB disks

[Peter Humphrey]

On Tuesday, 22 December 2020 16:48:26 GMT Rich Freeman wrote:
> On Tue, Dec 22, 2020 at 11:36 AM Peter Humphrey <peter@prh.myzen.co.uk> 
wrote:
> > I wondered about that. I'm nervous, though, because this is my ultimate
> > backup disk, and of course I don't want to endanger it. This disk is an
> > external USB unit, not for booting from.
> 
> Oh, if you aren't booting from it then I don't think the firmware
> would be involved at all.  Only the OS should matter, so as long as
> that is something modern you should be fine (not sure how linux from
> 1998 handles a 14TB USB drive).

Er... it's 4TB, and although I didn't say so, this is the Thinkpad T61 I 
mentioned in another thread, which is having Gentoo installed. I ditched the 
Windows partitions, since there was no point in keeping a possibility of 
recovering the OS - it was XP!

So far in the new installation, Gentoo can't see any partitions. I'd better 
check through all the FS settings I have in the kernel.

> Your biggest issue is probably going to be that if you have a lot of
> data to back up then it will take forever if the system only has USB2.

Yes, that's true, but the laptop's disk is only 120 GB.

> I'm actually storing a lot of my data on USB3 external drives now with
> lizardfs.  With Pi4s having 2x USB3 hosts you can keep up to four
> spinning disks near-100% occupied, and this is mostly for static data
> so I could handle more disks than that (rebuilds are going to be
> limited by the gigabit LAN port).  I was using LSI HBAs but have been
> having SATA errors with those - I suspect that the ones you can get
> cheap on eBay have a LOT of hours on them and are getting flaky - plus
> one of those HBAs probably pulls more power than a dozen Pis anyway.
> I wouldn't do this with older Pi models due to the USB being far more
> limited (USB2 only, and the LAN was shared with that too) - there are
> other SBCs that are options as well.

Lots more interesting ideas there - thanks again Rich.

> Obviously none of this is going to be competing with block storage
> solutions on SSD/NVMe.  This is just for media/etc where capacity and
> redundancy and cheap matters most.

I only have a couple of home machines to think about, so only fairly modest 
backup is in order.

-- 
Regards,
Peter.

[gentoo-user] Re: Big USB disks

[Nikos Chantziaras]

On 22/12/2020 14:58, Peter Humphrey wrote:
> Greetings,
> 
> Just a quickie - is there a way to enable a machine built on an MSDOS BIOS

A *what* BIOS? Do mean you run MS-DOS as on OS?

Re: [gentoo-user] Re: Big USB disks

[Peter Humphrey]

On Wednesday, 23 December 2020 09:54:36 GMT Nikos Chantziaras wrote:
> On 22/12/2020 14:58, Peter Humphrey wrote:
> > Greetings,
> > 
> > Just a quickie - is there a way to enable a machine built on an MSDOS BIOS
> 
> A *what* BIOS? Do mean you run MS-DOS as on OS?

What would you call it?

$ file /boot/boot.0800: DOS/MBR boot sector.

-- 
Regards,
Peter.

Re: [gentoo-user] Re: Big USB disks

[Mark Knecht]

[-- Attachment #1: Type: text/plain, Size: 655 bytes --]

On Wed, Dec 23, 2020 at 11:25 AM Peter Humphrey <peter@prh.myzen.co.uk>
wrote:
>
> On Wednesday, 23 December 2020 09:54:36 GMT Nikos Chantziaras wrote:
> > On 22/12/2020 14:58, Peter Humphrey wrote:
> > > Greetings,
> > >
> > > Just a quickie - is there a way to enable a machine built on an MSDOS
BIOS
> >
> > A *what* BIOS? Do mean you run MS-DOS as on OS?
>
> What would you call it?
>
> $ file /boot/boot.0800: DOS/MBR boot sector.
>
> --
> Regards,
> Peter.

Are you possibly conflating BIOS which is computer code in ROM on the
motherboard and what the CPU runs on power up with the MBR which resides on
the hard drive you are booting from?

- Mark

[-- Attachment #2: Type: text/html, Size: 899 bytes --]

Re: [gentoo-user] Re: Big USB disks

[Michael]

[-- Attachment #1: Type: text/plain, Size: 1218 bytes --]

On Wednesday, 23 December 2020 18:56:37 GMT Mark Knecht wrote:
> On Wed, Dec 23, 2020 at 11:25 AM Peter Humphrey <peter@prh.myzen.co.uk>
> 
> wrote:
> > On Wednesday, 23 December 2020 09:54:36 GMT Nikos Chantziaras wrote:
> > > On 22/12/2020 14:58, Peter Humphrey wrote:
> > > > Greetings,
> > > > 
> > > > Just a quickie - is there a way to enable a machine built on an MSDOS
> 
> BIOS
> 
> > > A *what* BIOS? Do mean you run MS-DOS as on OS?
> > 
> > What would you call it?
> > 
> > $ file /boot/boot.0800: DOS/MBR boot sector.
> > 
> > --
> > Regards,
> > Peter.
> 
> Are you possibly conflating BIOS which is computer code in ROM on the
> motherboard and what the CPU runs on power up with the MBR which resides on
> the hard drive you are booting from?
> 
> - Mark

The file boot.0800 appears to be a backup of the MBR bootloader code and I bet 
it is 512B in size.  It reflects dev with major number 08 and minor number 00, 
e.g. /dev/hda1, or /dev/sda1.   I would also bet this was created by LILO, 
which is polite enough to back up the existing MBR before dumping its own code 
in the first sector of a hard disk.  Characteristically, MSWindows is not that 
accommodating for other persons' bootloader codes.

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [gentoo-user] Re: Big USB disks

[Peter Humphrey]

On Wednesday, 23 December 2020 18:56:37 GMT Mark Knecht wrote:

> Are you possibly conflating BIOS which is computer code in ROM on the
> motherboard and what the CPU runs on power up with the MBR which resides on
> the hard drive you are booting from?

No, I just wanted to distinguish the old-style boot method from UEFI.

-- 
Regards,
Peter.