[gentoo-user] Trouble with backup harddisks

[gentoo-user] Trouble with backup harddisks

[tuxic]

Hi,

recently I switched from the old MBR-scheme to GPT on
my new PC.

I have two external USB-harddisk, which were partioned/formatted with
a MBR-scheme/MSDOS partition (but were never used to boot from. They are pure
data containers).

When I connect these to my new PC, only the device is shown:
/dev/sdb, /dev/sdc.

The kernel is configured (beside other things) as follows:

#
# Partition Types
#
CONFIG_PARTITION_ADVANCED=y
# CONFIG_ACORN_PARTITION is not set
# CONFIG_AIX_PARTITION is not set
# CONFIG_OSF_PARTITION is not set
# CONFIG_AMIGA_PARTITION is not set
# CONFIG_ATARI_PARTITION is not set
# CONFIG_MAC_PARTITION is not set
CONFIG_MSDOS_PARTITION=y
# CONFIG_BSD_DISKLABEL is not set
# CONFIG_MINIX_SUBPARTITION is not set
# CONFIG_SOLARIS_X86_PARTITION is not set
# CONFIG_UNIXWARE_DISKLABEL is not set
# CONFIG_LDM_PARTITION is not set
# CONFIG_SGI_PARTITION is not set
# CONFIG_ULTRIX_PARTITION is not set
# CONFIG_SUN_PARTITION is not set
# CONFIG_KARMA_PARTITION is not set
CONFIG_EFI_PARTITION=y
# CONFIG_SYSV68_PARTITION is not set
# CONFIG_CMDLINE_PARTITION is not set
# end of Partition Types

CONFIG_BLOCK_COMPAT=y
CONFIG_BLK_MQ_PCI=y
CONFIG_BLK_MQ_VIRTIO=y
CONFIG_BLK_PM=y

dmesg shows this:
[14617.672363] usb 2-2: New USB device found, idVendor=1058, idProduct=25a2, bcdDevice=10.21
[14617.672364] usb 2-2: New USB device strings: Mfr=1, Product=2, SerialNumber=3
[14617.672364] usb 2-2: Product: Elements 25A2
[14617.672365] usb 2-2: Manufacturer: Western Digital
[14617.672366] usb 2-2: SerialNumber: 575844314132383959393934
[14617.681660] usb-storage 2-2:1.0: USB Mass Storage device detected
[14617.681737] scsi host10: usb-storage 2-2:1.0
[14618.725450] scsi 10:0:0:0: Direct-Access     WD       Elements 25A2    1021 PQ: 0 ANSI: 6
[14618.725594] sd 10:0:0:0: Attached scsi generic sg1 type 0
[14618.728090] sd 10:0:0:0: [sdb] Spinning up disk...
[14619.748918] ...ready

I tried different USB ports in a desperate hope of success...
...no, same problem.

Interestingly fdisk shows the following:

host> sudo fdisk -l /dev/sdb
Disk /dev/sdb: 931.49 GiB, 1000170586112 bytes, 1953458176 sectors
Disk model: Elements 25A2   
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: dos
Disk identifier: 0x16f2a91f

Device     Boot Start        End    Sectors   Size Id Type
/dev/sdb1           1 1953458175 1953458175 931.5G ee GPT

The type is shown as GPT...but the drive has a MSDOS partition table.

Reading my (old) internal harddrive with an external USB docking
station is possible without any problems, though.

Unfortunatelu I have no space for 1T image of that drive -- otherwise
I would have made an image copy and experiment with that.

So better ask, than sorry.... ;)

Is this fixable or did I lost my backups?

Cheers!
Meino

Re: [gentoo-user] Trouble with backup harddisks

[Wols Lists]

On 30/04/20 10:32, tuxic@posteo.de wrote:
> Hi,
> 
> recently I switched from the old MBR-scheme to GPT on
> my new PC.
> 
> I have two external USB-harddisk, which were partioned/formatted with
> a MBR-scheme/MSDOS partition (but were never used to boot from. They are pure
> data containers).
> 
> When I connect these to my new PC, only the device is shown:
> /dev/sdb, /dev/sdc.
> 
> The kernel is configured (beside other things) as follows:
> 
> #
> # Partition Types
> #
> CONFIG_PARTITION_ADVANCED=y
> # CONFIG_ACORN_PARTITION is not set
> # CONFIG_AIX_PARTITION is not set
> # CONFIG_OSF_PARTITION is not set
> # CONFIG_AMIGA_PARTITION is not set
> # CONFIG_ATARI_PARTITION is not set
> # CONFIG_MAC_PARTITION is not set
> CONFIG_MSDOS_PARTITION=y
> # CONFIG_BSD_DISKLABEL is not set
> # CONFIG_MINIX_SUBPARTITION is not set
> # CONFIG_SOLARIS_X86_PARTITION is not set
> # CONFIG_UNIXWARE_DISKLABEL is not set
> # CONFIG_LDM_PARTITION is not set
> # CONFIG_SGI_PARTITION is not set
> # CONFIG_ULTRIX_PARTITION is not set
> # CONFIG_SUN_PARTITION is not set
> # CONFIG_KARMA_PARTITION is not set
> CONFIG_EFI_PARTITION=y
> # CONFIG_SYSV68_PARTITION is not set
> # CONFIG_CMDLINE_PARTITION is not set
> # end of Partition Types
> 
> CONFIG_BLOCK_COMPAT=y
> CONFIG_BLK_MQ_PCI=y
> CONFIG_BLK_MQ_VIRTIO=y
> CONFIG_BLK_PM=y
> 
> dmesg shows this:
> [14617.672363] usb 2-2: New USB device found, idVendor=1058, idProduct=25a2, bcdDevice=10.21
> [14617.672364] usb 2-2: New USB device strings: Mfr=1, Product=2, SerialNumber=3
> [14617.672364] usb 2-2: Product: Elements 25A2
> [14617.672365] usb 2-2: Manufacturer: Western Digital
> [14617.672366] usb 2-2: SerialNumber: 575844314132383959393934
> [14617.681660] usb-storage 2-2:1.0: USB Mass Storage device detected
> [14617.681737] scsi host10: usb-storage 2-2:1.0
> [14618.725450] scsi 10:0:0:0: Direct-Access     WD       Elements 25A2    1021 PQ: 0 ANSI: 6
> [14618.725594] sd 10:0:0:0: Attached scsi generic sg1 type 0
> [14618.728090] sd 10:0:0:0: [sdb] Spinning up disk...
> [14619.748918] ...ready
> 
> I tried different USB ports in a desperate hope of success...
> ...no, same problem.
> 
> Interestingly fdisk shows the following:
> 
> host> sudo fdisk -l /dev/sdb
> Disk /dev/sdb: 931.49 GiB, 1000170586112 bytes, 1953458176 sectors
> Disk model: Elements 25A2   
> Units: sectors of 1 * 512 = 512 bytes
> Sector size (logical/physical): 512 bytes / 512 bytes
> I/O size (minimum/optimal): 512 bytes / 512 bytes

This could be the key. Sector sizes have been changing from 512 to 4096
over many years. If your kernel has been updated to expect/use 4096 byte
sectors, it might not be able to read the disk properly.

> Disklabel type: dos
> Disk identifier: 0x16f2a91f
> 
> Device     Boot Start        End    Sectors   Size Id Type
> /dev/sdb1           1 1953458175 1953458175 931.5G ee GPT
> 
> The type is shown as GPT...but the drive has a MSDOS partition table.
> 
> Reading my (old) internal harddrive with an external USB docking
> station is possible without any problems, though.
> 
> Unfortunatelu I have no space for 1T image of that drive -- otherwise
> I would have made an image copy and experiment with that.
> 
> So better ask, than sorry.... ;)
> 
> Is this fixable or did I lost my backups?
> 
Do you have access to an old kernel?

The other thing is try using gdisk (or that could be fdisk under another
name :-( But some partitioning schemes can write a GPT with protective
MBR - if you can find something that will take the MBR and write it as a
GPT that might help, too.

Cheers,
Wol

Re: [gentoo-user] Trouble with backup harddisks

[tuxic]

On 04/30 10:55, Wols Lists wrote:
> On 30/04/20 10:32, tuxic@posteo.de wrote:
> > Hi,
> > 
> > recently I switched from the old MBR-scheme to GPT on
> > my new PC.
> > 
> > I have two external USB-harddisk, which were partioned/formatted with
> > a MBR-scheme/MSDOS partition (but were never used to boot from. They are pure
> > data containers).
> > 
> > When I connect these to my new PC, only the device is shown:
> > /dev/sdb, /dev/sdc.
> > 
> > The kernel is configured (beside other things) as follows:
> > 
> > #
> > # Partition Types
> > #
> > CONFIG_PARTITION_ADVANCED=y
> > # CONFIG_ACORN_PARTITION is not set
> > # CONFIG_AIX_PARTITION is not set
> > # CONFIG_OSF_PARTITION is not set
> > # CONFIG_AMIGA_PARTITION is not set
> > # CONFIG_ATARI_PARTITION is not set
> > # CONFIG_MAC_PARTITION is not set
> > CONFIG_MSDOS_PARTITION=y
> > # CONFIG_BSD_DISKLABEL is not set
> > # CONFIG_MINIX_SUBPARTITION is not set
> > # CONFIG_SOLARIS_X86_PARTITION is not set
> > # CONFIG_UNIXWARE_DISKLABEL is not set
> > # CONFIG_LDM_PARTITION is not set
> > # CONFIG_SGI_PARTITION is not set
> > # CONFIG_ULTRIX_PARTITION is not set
> > # CONFIG_SUN_PARTITION is not set
> > # CONFIG_KARMA_PARTITION is not set
> > CONFIG_EFI_PARTITION=y
> > # CONFIG_SYSV68_PARTITION is not set
> > # CONFIG_CMDLINE_PARTITION is not set
> > # end of Partition Types
> > 
> > CONFIG_BLOCK_COMPAT=y
> > CONFIG_BLK_MQ_PCI=y
> > CONFIG_BLK_MQ_VIRTIO=y
> > CONFIG_BLK_PM=y
> > 
> > dmesg shows this:
> > [14617.672363] usb 2-2: New USB device found, idVendor=1058, idProduct=25a2, bcdDevice=10.21
> > [14617.672364] usb 2-2: New USB device strings: Mfr=1, Product=2, SerialNumber=3
> > [14617.672364] usb 2-2: Product: Elements 25A2
> > [14617.672365] usb 2-2: Manufacturer: Western Digital
> > [14617.672366] usb 2-2: SerialNumber: 575844314132383959393934
> > [14617.681660] usb-storage 2-2:1.0: USB Mass Storage device detected
> > [14617.681737] scsi host10: usb-storage 2-2:1.0
> > [14618.725450] scsi 10:0:0:0: Direct-Access     WD       Elements 25A2    1021 PQ: 0 ANSI: 6
> > [14618.725594] sd 10:0:0:0: Attached scsi generic sg1 type 0
> > [14618.728090] sd 10:0:0:0: [sdb] Spinning up disk...
> > [14619.748918] ...ready
> > 
> > I tried different USB ports in a desperate hope of success...
> > ...no, same problem.
> > 
> > Interestingly fdisk shows the following:
> > 
> > host> sudo fdisk -l /dev/sdb
> > Disk /dev/sdb: 931.49 GiB, 1000170586112 bytes, 1953458176 sectors
> > Disk model: Elements 25A2   
> > Units: sectors of 1 * 512 = 512 bytes
> > Sector size (logical/physical): 512 bytes / 512 bytes
> > I/O size (minimum/optimal): 512 bytes / 512 bytes
> 
> This could be the key. Sector sizes have been changing from 512 to 4096
> over many years. If your kernel has been updated to expect/use 4096 byte
> sectors, it might not be able to read the disk properly.
> 
> > Disklabel type: dos
> > Disk identifier: 0x16f2a91f
> > 
> > Device     Boot Start        End    Sectors   Size Id Type
> > /dev/sdb1           1 1953458175 1953458175 931.5G ee GPT
> > 
> > The type is shown as GPT...but the drive has a MSDOS partition table.
> > 
> > Reading my (old) internal harddrive with an external USB docking
> > station is possible without any problems, though.
> > 
> > Unfortunatelu I have no space for 1T image of that drive -- otherwise
> > I would have made an image copy and experiment with that.
> > 
> > So better ask, than sorry.... ;)
> > 
> > Is this fixable or did I lost my backups?
> > 
> Do you have access to an old kernel?
> 
> The other thing is try using gdisk (or that could be fdisk under another
> name :-( But some partitioning schemes can write a GPT with protective
> MBR - if you can find something that will take the MBR and write it as a
> GPT that might help, too.
> 
> Cheers,
> Wol
> 

Hi Wol,

thank you for your posting! :)

I switched to my new PC around 31.3.2020...
so there is no change "over the years" involved here...I think.
Where in the kernel this update is made?
I cannot remember to have configured such a thing manually...

Changing the partitiontable in any way is risky to the data 
I think...I am very unsure to do this.

I have access to an old kernel / system and can boot it.

But than I have the same problem another way around: I can
no longer access my new system ... due to the different 
sector size....

Are there any other ways to fix this problem?

Cheers!
Meino

Re: [gentoo-user] Trouble with backup harddisks

[Wols Lists]

On 30/04/20 11:36, tuxic@posteo.de wrote:
> But than I have the same problem another way around: I can
> no longer access my new system ... due to the different 
> sector size....
> 
> Are there any other ways to fix this problem?

All I can suggest is to check the kernel and see if it's an option that
has been disabled (512-byte sectors, that is).

Cheers,
Wol

Re: [gentoo-user] Trouble with backup harddisks

[Wynn Wolf Arbor]

Hi,

On 2020-04-30 13:17, Wols Lists wrote:
>All I can suggest is to check the kernel and see if it's an option that
>has been disabled (512-byte sectors, that is).

As far as I know the kernel still uses 512 bytes internally [1], and I 
do not recall having seen an option that enables or disables support for 
512/4K sectors.

That said, the problem may well be stemming from a sector size 
discrepancy, but as I understand it, it would have to do with how and 
when the partition table was created. That is, like described in [2], 
some USB enclosures seem to be a bit overzealous with obscure features, 
and might take eight disk sectors and bundle them together into one 4K 
sector.

If the disk was partitioned in the exact same enclosure, and is read 
from the exact same enclosure right now, there shouldn't be any 
problems. Is this the case, Meino?

Also, when did you last access the drives successfully, and with which 
system?

On 2020-04-30 11:32, Meino wrote:
> Disk /dev/sdb: 931.49 GiB, 1000170586112 bytes, 1953458176 sectors
> Disk model: Elements 25A2   
> Units: sectors of 1 * 512 = 512 bytes
> Sector size (logical/physical): 512 bytes / 512 bytes
> I/O size (minimum/optimal): 512 bytes / 512 bytes
> 
> Disklabel type: dos
> Disk identifier: 0x16f2a91f
> 
> Device     Boot Start        End    Sectors   Size Id Type
> /dev/sdb1           1 1953458175 1953458175 931.5G ee GPT

Interestingly, this reads *exactly* like the Protective MBR [3] that GPT 
has. That is, the disklabel type is DOS whilst the partition ID is EE.  
There's a single partition that spans the entire drive (and it's also 
seemingly not aligned properly - usually you see Start at 2048).

As a comparison, here's the output from fdisk for the Protective MBR 
from one of my GPT drives:

> Disklabel type: dos
> Disk identifier: 0x00000000

> Device     Boot Start        End    Sectors Size Id Type
> /dev/sdc1           1 4294967295 4294967295   2T ee GPT

I'd assume that the missing disk identifier here is coming from 
different tools writing the protective MBR differently.

With that said, are you absolutely certain that you did not partition 
this drive with GPT instead of MBR? Did you do the partitioning in 
something like fdisk (which asks you specifically what you want), or 
some other application? Did you maybe format this drive on a Windows 
system?

I'm not one to discount entirely strange things happening, but I have 
never before seen a proper MBR laid out like a protective MBR. Indeed it 
would be quite impossible to have systems access data through such a 
table, since the partitions are hidden within that one huge contiguous 
block.

Ordinarily I'd point to fdisk not reading the partition table properly, 
but it seems that although your kernel has support for GPT, it doesn't 
seem to see the partitions either (assuming a proper GPT exists at all).
Do you have some other GPT drives you can access successfully?

I'd say that this requires some more forensic work. Perhaps extracting 
the first few megabytes of the disk and seeing whether there's a proper 
GPT or not. This would of course require manual work.

A few more things to try:

To see what the kernel uses for a particular disk, you can run the 
following: cat /sys/block/sdb/queue/{physical,logical}_block_size

fdisk takes a sector size with -b, --sector-size (should be 
non-destructive as long as you don't write anything, but I am not sure). 
Also, fdisk has a compatibility mode for dos with -c=dos. Might be worth 
a short.

fdisk should support GPT starting with util-linux 2.23, but you can also 
try gptfdisk (it's in the tree).

Hope this helps.

[1] https://github.com/torvalds/linux/blob/master/include/linux/types.h#L120
[2] https://superuser.com/questions/679725/how-to-correct-512-byte-sector-mbr-on-a-4096-byte-sector-disk/679800#679800
[3] https://en.wikipedia.org/wiki/GUID_Partition_Table#PROTECTIVE-MBR

-- 
Wolf

Re: [gentoo-user] Trouble with backup harddisks

[tuxic]

Hi Wolf

thank you very much for your analysis ! :)

On 04/30 03:10, Wynn Wolf Arbor wrote:
> Hi,
> 
> On 2020-04-30 13:17, Wols Lists wrote:
> > All I can suggest is to check the kernel and see if it's an option that
> > has been disabled (512-byte sectors, that is).
> 
> As far as I know the kernel still uses 512 bytes internally [1], and I do
> not recall having seen an option that enables or disables support for 512/4K
> sectors.
> 
> That said, the problem may well be stemming from a sector size discrepancy,
> but as I understand it, it would have to do with how and when the partition
> table was created. That is, like described in [2], some USB enclosures seem
> to be a bit overzealous with obscure features, and might take eight disk
> sectors and bundle them together into one 4K sector.
> 
> If the disk was partitioned in the exact same enclosure, and is read from
> the exact same enclosure right now, there shouldn't be any problems. Is this
> the case, Meino?

I have two external disks of that kind, same product, nearly the same
contents, both have a "fixed case" (sorry...no native speaker...the
case and the disk mechanics/electronics is "one thing").
> 
> Also, when did you last access the drives successfully, and with which
> system?

Three weeks ago...about...guessed...with my old system.
Because these were backup disks I *NEVER* touch them with
any partitioning tool whatsoever...

It is complete unknown to me, what could have destroyed the
partitioning...

> On 2020-04-30 11:32, Meino wrote:
> > Disk /dev/sdb: 931.49 GiB, 1000170586112 bytes, 1953458176 sectors
> > Disk model: Elements 25A2   Units: sectors of 1 * 512 = 512 bytes
> > Sector size (logical/physical): 512 bytes / 512 bytes
> > I/O size (minimum/optimal): 512 bytes / 512 bytes
> > 
> > Disklabel type: dos
> > Disk identifier: 0x16f2a91f
> > 
> > Device     Boot Start        End    Sectors   Size Id Type
> > /dev/sdb1           1 1953458175 1953458175 931.5G ee GPT
> 
> Interestingly, this reads *exactly* like the Protective MBR [3] that GPT
> has. That is, the disklabel type is DOS whilst the partition ID is EE.
> There's a single partition that spans the entire drive (and it's also
> seemingly not aligned properly - usually you see Start at 2048).

> As a comparison, here's the output from fdisk for the Protective MBR from
> one of my GPT drives:
> 
> > Disklabel type: dos
> > Disk identifier: 0x00000000
> 
> > Device     Boot Start        End    Sectors Size Id Type
> > /dev/sdc1           1 4294967295 4294967295   2T ee GPT
> 
> I'd assume that the missing disk identifier here is coming from different
> tools writing the protective MBR differently.
> 
> With that said, are you absolutely certain that you did not partition this
> drive with GPT instead of MBR? 

YES! These drives were used as backup and as backup of the backup. I
NEVER touched them with any partitioning tool after I had put a
filesystem and data on it.

> Did you do the partitioning in something like
> fdisk (which asks you specifically what you want), or some other
> application? Did you maybe format this drive on a Windows system?

Windows? ....no thank you...all my windows here are transparent and
open,.. ;)
Joke aside..
No Windows (tm) here. The disks are used only with my Linux box using
ext4 as filesystem.

> 
> I'm not one to discount entirely strange things happening, but I have never
> before seen a proper MBR laid out like a protective MBR. Indeed it would be
> quite impossible to have systems access data through such a table, since the
> partitions are hidden within that one huge contiguous block.
> 
> Ordinarily I'd point to fdisk not reading the partition table properly, but
> it seems that although your kernel has support for GPT, it doesn't seem to
> see the partitions either (assuming a proper GPT exists at all).

> Do you have some other GPT drives you can access successfully?
With my new PC (my old MBR-based system was 12 years old and needed to
be replaced. My kernel became "GPT aware onlu with my new PC.
Everything before was based on MBR.

My new PC can read GPT disks...my system is based on it.
> 
> I'd say that this requires some more forensic work. Perhaps extracting the
> first few megabytes of the disk and seeing whether there's a proper GPT or
> not. This would of course require manual work.

I copied the first 230GB of that disk to an empty partition of my new
system and run "testdisk" on it....after the analysis it came back
with "this partition cannot be recovered" but did not sau. whether the
reason is a partition table, which is broken beyond repair, or simply
due to the incomplete image file...

> 
> A few more things to try:
> 
> To see what the kernel uses for a particular disk, you can run the
> following: cat /sys/block/sdb/queue/{physical,logical}_block_size
 

host:/home/user>cat /sys/block/sdb/queue/physical_block_size 
512
solfire:/home/user>cat /sys/block/sdb/queue/logical_block_size 
512
host:/home/user>

> fdisk takes a sector size with -b, --sector-size (should be non-destructive
> as long as you don't write anything, but I am not sure). Also, fdisk has a
> compatibility mode for dos with -c=dos. Might be worth a short.
 
host:data/pool07>fdisk -c=dos testf.bin 

Welcome to fdisk (util-linux 2.35.1).
Changes will remain in memory only, until you decide to write them.
Be careful before using the write command.

GPT PMBR size mismatch (1953458175 != 455078119) will be corrected by write.
DOS-compatible mode is deprecated.

Command (m for help): p

Disk testf.bin: 216.102 GiB, 232999997440 bytes, 455078120 sectors
Geometry: 256 heads, 63 sectors/track, 28327 cylinders
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: dos
Disk identifier: 0x16f2a91f

Device     Boot Start       End   Sectors  Size Id Type
testf.bin1          1 455078119 455078119  217G ee GPT

Command (m for help): 



> fdisk should support GPT starting with util-linux 2.23, but you can also try
> gptfdisk (it's in the tree).
> 
> Hope this helps.
> 
> [1] https://github.com/torvalds/linux/blob/master/include/linux/types.h#L120
> [2] https://superuser.com/questions/679725/how-to-correct-512-byte-sector-mbr-on-a-4096-byte-sector-disk/679800#679800
> [3] https://en.wikipedia.org/wiki/GUID_Partition_Table#PROTECTIVE-MBR
> 
> -- 
> Wolf

Thanks a lot Wolf!!!

Cheers!
Meino

Re: [gentoo-user] Trouble with backup harddisks

[Wynn Wolf Arbor]

Hi Meino,

Thanks very much for the info. At this point I'm convinced you're 
running into the problem Andrea described in another reply in this 
thread - best to follow up there :)

-- 
Wolf

Re: [gentoo-user] Trouble with backup harddisks

[antlists]

On 30/04/2020 18:04, tuxic@posteo.de wrote:
> I copied the first 230GB of that disk to an empty partition of my new
> system and run "testdisk" on it....after the analysis it came back
> with "this partition cannot be recovered" but did not sau. whether the
> reason is a partition table, which is broken beyond repair, or simply
> due to the incomplete image file...

Just come up with another idea that will hopefully give us some clues...

https://raid.wiki.kernel.org/index.php/Asking_for_help#lsdrv

Ignore the fact that it's the raid wiki - this tool basically delves 
into the drive and tries to find mbr, gpt, superblock, lvm and raid 
signatures, everything ...

So run that and see what it tells us ...

Cheers,
Wol

Re: [gentoo-user] Trouble with backup harddisks

[tuxic]

On 04/30 08:32, antlists wrote:
> On 30/04/2020 18:04, tuxic@posteo.de wrote:
> > I copied the first 230GB of that disk to an empty partition of my new
> > system and run "testdisk" on it....after the analysis it came back
> > with "this partition cannot be recovered" but did not sau. whether the
> > reason is a partition table, which is broken beyond repair, or simply
> > due to the incomplete image file...
> 
> Just come up with another idea that will hopefully give us some clues...
> 
> https://raid.wiki.kernel.org/index.php/Asking_for_help#lsdrv
> 
> Ignore the fact that it's the raid wiki - this tool basically delves into
> the drive and tries to find mbr, gpt, superblock, lvm and raid signatures,
> everything ...
> 
> So run that and see what it tells us ...
> 
> Cheers,
> Wol
> 

Hi Wol,

thank you for the link! I have installed mdadm (its on portage) but I
have not used it, since there is another soultion for this, which I
tried first. 

Andrea posted this wonderful line:

mount -o ro,offset=512 /dev/sdb /mnt/xxx

which mounts the filesystem by skipping the partitiontable completly.

Currently I am rescueing the data. If all is safed, I will tru
mdadm...

Thank you very much for your help!

As soon as there are more information, I will report back.
Maybe this can also help others with similiar problems.

Cheers!
Meino

Re: [gentoo-user] Trouble with backup harddisks

[tuxic]

On 05/01 03:59, tuxic@posteo.de wrote:
> On 04/30 08:32, antlists wrote:
> > On 30/04/2020 18:04, tuxic@posteo.de wrote:
> > > I copied the first 230GB of that disk to an empty partition of my new
> > > system and run "testdisk" on it....after the analysis it came back
> > > with "this partition cannot be recovered" but did not sau. whether the
> > > reason is a partition table, which is broken beyond repair, or simply
> > > due to the incomplete image file...
> > 
> > Just come up with another idea that will hopefully give us some clues...
> > 
> > https://raid.wiki.kernel.org/index.php/Asking_for_help#lsdrv
> > 
> > Ignore the fact that it's the raid wiki - this tool basically delves into
> > the drive and tries to find mbr, gpt, superblock, lvm and raid signatures,
> > everything ...
> > 
> > So run that and see what it tells us ...
> > 
> > Cheers,
> > Wol
> > 
> 
> Hi Wol,
> 
> thank you for the link! I have installed mdadm (its on portage) but I
> have not used it, since there is another soultion for this, which I
> tried first. 
> 
> Andrea posted this wonderful line:
> 
> mount -o ro,offset=512 /dev/sdb /mnt/xxx
> 
> which mounts the filesystem by skipping the partitiontable completly.
> 
> Currently I am rescueing the data. If all is safed, I will tru
> mdadm...
> 
> Thank you very much for your help!
> 
> As soon as there are more information, I will report back.
> Maybe this can also help others with similiar problems.
> 
> Cheers!
> Meino
> 
> 
> 

Hi Wol,

data copied ! :)

I did a 

    mdadm --examine /dev/sdb

on one of the damaged partitions and this was print
on the console

host:/root>mdadm --examine /dev/sdb
/dev/sdb:
   MBR Magic : aa55
Partition[0] :   1953458175 sectors at            1 (type ee)


Cheers!
Meino

Re: [gentoo-user] Trouble with backup harddisks

[antlists]

On 01/05/2020 09:03, tuxic@posteo.de wrote:
> Hi Wol,
> 
> data copied !:)
> 
> I did a
> 
>      mdadm --examine /dev/sdb

Except I pointed you at a utility called lsdrv, not mdadm ... :-)

Cheers,
Wol

Re: [gentoo-user] Trouble with backup harddisks

[tuxic]

On 05/01 09:27, antlists wrote:
> On 01/05/2020 09:03, tuxic@posteo.de wrote:
> > Hi Wol,
> > 
> > data copied !:)
> > 
> > I did a
> > 
> >      mdadm --examine /dev/sdb
> 
> Except I pointed you at a utility called lsdrv, not mdadm ... :-)
> 
> Cheers,
> Wol
> 

Hi Wol,

Ouuouud...oh damn! Sorry, Wol...

I donwloaded the lsdrv from github and this is, what it prints

  File "./lsdrv", line 323
    os.mkdir('/dev/block', 0755)
                              ^
SyntaxError: leading zeros in decimal integer literals are not permitted; use an 0o prefix for octal integers
[1]    4367 exit 1     ./lsdrv


Seems a little buggy...

Cheers!
Meino

Re: [gentoo-user] Trouble with backup harddisks

[Wols Lists]

On 02/05/20 02:42, tuxic@posteo.de wrote:
> On 05/01 09:27, antlists wrote:
>> On 01/05/2020 09:03, tuxic@posteo.de wrote:
>>> Hi Wol,
>>>
>>> data copied !:)
>>>
>>> I did a
>>>
>>>      mdadm --examine /dev/sdb
>>
>> Except I pointed you at a utility called lsdrv, not mdadm ... :-)
>>
>> Cheers,
>> Wol
>>
> 
> Hi Wol,
> 
> Ouuouud...oh damn! Sorry, Wol...
> 
> I donwloaded the lsdrv from github and this is, what it prints
> 
>   File "./lsdrv", line 323
>     os.mkdir('/dev/block', 0755)
>                               ^
> SyntaxError: leading zeros in decimal integer literals are not permitted; use an 0o prefix for octal integers
> [1]    4367 exit 1     ./lsdrv
> 
> 
> Seems a little buggy...
> 
did you read the note that says it is a Python 2 script? Unless it's
been updated (I haven't checked) you need to change the shebang line to
force python2.

Cheers,
Wol

Re: [gentoo-user] Trouble with backup harddisks

[tuxic]

On 05/02 06:31, Wols Lists wrote:
> On 02/05/20 02:42, tuxic@posteo.de wrote:
> > On 05/01 09:27, antlists wrote:
> >> On 01/05/2020 09:03, tuxic@posteo.de wrote:
> >>> Hi Wol,
> >>>
> >>> data copied !:)
> >>>
> >>> I did a
> >>>
> >>>      mdadm --examine /dev/sdb
> >>
> >> Except I pointed you at a utility called lsdrv, not mdadm ... :-)
> >>
> >> Cheers,
> >> Wol
> >>
> > 
> > Hi Wol,
> > 
> > Ouuouud...oh damn! Sorry, Wol...
> > 
> > I donwloaded the lsdrv from github and this is, what it prints
> > 
> >   File "./lsdrv", line 323
> >     os.mkdir('/dev/block', 0755)
> >                               ^
> > SyntaxError: leading zeros in decimal integer literals are not permitted; use an 0o prefix for octal integers
> > [1]    4367 exit 1     ./lsdrv
> > 
> > 
> > Seems a little buggy...
> > 
> did you read the note that says it is a Python 2 script? Unless it's
> been updated (I haven't checked) you need to change the shebang line to
> force python2.
> 
> Cheers,
> Wol
> 
> 

Hi,

ok...will try that - thanks for the info !!! :)

Cheers!
Meino

Re: [gentoo-user] Trouble with backup harddisks

[tuxic]

On 05/02 06:31, Wols Lists wrote:
> On 02/05/20 02:42, tuxic@posteo.de wrote:
> > On 05/01 09:27, antlists wrote:
> >> On 01/05/2020 09:03, tuxic@posteo.de wrote:
> >>> Hi Wol,
> >>>
> >>> data copied !:)
> >>>
> >>> I did a
> >>>
> >>>      mdadm --examine /dev/sdb
> >>
> >> Except I pointed you at a utility called lsdrv, not mdadm ... :-)
> >>
> >> Cheers,
> >> Wol
> >>
> > 
> > Hi Wol,
> > 
> > Ouuouud...oh damn! Sorry, Wol...
> > 
> > I donwloaded the lsdrv from github and this is, what it prints
> > 
> >   File "./lsdrv", line 323
> >     os.mkdir('/dev/block', 0755)
> >                               ^
> > SyntaxError: leading zeros in decimal integer literals are not permitted; use an 0o prefix for octal integers
> > [1]    4367 exit 1     ./lsdrv
> > 
> > 
> > Seems a little buggy...
> > 
> did you read the note that says it is a Python 2 script? Unless it's
> been updated (I haven't checked) you need to change the shebang line to
> force python2.
> 
> Cheers,
> Wol
> 
> 

Hi Wol,

the shebang was already changed. It onlu works, when called like

python2.7 lsdrv 

for me.

The output is similiar to what lsblk creates...

Cheers!
Meino

Re: [gentoo-user] Trouble with backup harddisks

[Andrea Conti]

Hi,

>> CONFIG_PARTITION_ADVANCED=y
>> CONFIG_MSDOS_PARTITION=y
>> CONFIG_EFI_PARTITION=y

That's all you need.

> This could be the key. Sector sizes have been changing from 512 to 4096
> over many years. If your kernel has been updated to expect/use 4096 byte
> sectors, it might not be able to read the disk properly.

Sector size is only a (fixed) property of a specific block device, not something expected or required by the kernel.
Sector sizes other than 512B have been around for ages without any problems, even in consumer hardware (e.g. CDs and DVDs have 2KB sectors).

> Disklabel type: dos

> Device     Boot Start        End    Sectors   Size Id Type
> /dev/sdb1           1 1953458175 1953458175 931.5G ee GPT

That looks... broken.
fdisk is recognizing the disk as MBR (a GPT disk would have "Disklabel type: gpt"), but the partition table is a protective MBR, which makes no sense in a non-GPT disk.

My guess is that this disk was at some time partitioned with GPT (possibly it came that way from WD?), but then it was only used in machines with no kernel support for GPT.
Such a machine will happily treat that as a normal MBR and allow you to access the protective entry as a normal partition, which means you can create a filesystem on it and fill it with data, destroying the GPT structures.

A GPT-aware kernel on the other hand will recognize that as a protective MBR and it will ignore it --but since the disk does not contain any valid GPT structures, it will not show any partitions.

Try running "gdisk -l /dev/sdb"; for a valid GPT disk it will say:

Partition table scan:
   MBR: protective
   BSD: not present
   APM: not present
   GPT: present

If that's not the case and you have no GPT, you will have to fix things manually.
Since the disk is only 1TB, there is no reason to use GPT at all, so your best bet is to use fdisk to make that a standard MBR by changing the partition type from 'ee' to '83'.

andrea

Re: [gentoo-user] Trouble with backup harddisks

[tuxic]

On 04/30 03:44, Andrea Conti wrote:
> Hi,
> 
> > > CONFIG_PARTITION_ADVANCED=y
> > > CONFIG_MSDOS_PARTITION=y
> > > CONFIG_EFI_PARTITION=y
> 
> That's all you need.
> 
> > This could be the key. Sector sizes have been changing from 512 to 4096
> > over many years. If your kernel has been updated to expect/use 4096 byte
> > sectors, it might not be able to read the disk properly.
> 
> Sector size is only a (fixed) property of a specific block device, not something expected or required by the kernel.
> Sector sizes other than 512B have been around for ages without any problems, even in consumer hardware (e.g. CDs and DVDs have 2KB sectors).
> 
> > Disklabel type: dos
> 
> > Device     Boot Start        End    Sectors   Size Id Type
> > /dev/sdb1           1 1953458175 1953458175 931.5G ee GPT
> 
> That looks... broken.
> fdisk is recognizing the disk as MBR (a GPT disk would have "Disklabel type: gpt"), but the partition table is a protective MBR, which makes no sense in a non-GPT disk.
> 
> My guess is that this disk was at some time partitioned with GPT (possibly it came that way from WD?), but then it was only used in machines with no kernel support for GPT.
> Such a machine will happily treat that as a normal MBR and allow you to access the protective entry as a normal partition, which means you can create a filesystem on it and fill it with data, destroying the GPT structures.
> 
> A GPT-aware kernel on the other hand will recognize that as a protective MBR and it will ignore it --but since the disk does not contain any valid GPT structures, it will not show any partitions.
> 
> Try running "gdisk -l /dev/sdb"; for a valid GPT disk it will say:
> 
> Partition table scan:
>   MBR: protective
>   BSD: not present
>   APM: not present
>   GPT: present
> 
> If that's not the case and you have no GPT, you will have to fix things manually.
> Since the disk is only 1TB, there is no reason to use GPT at all, so your best bet is to use fdisk to make that a standard MBR by changing the partition type from 'ee' to '83'.
> 
> andrea
> 

Hi Andreas,

thank you very much for your analysis! :)

when doing a gdisk -l /dev/sdb I get this Rocky Horror Picture Show:


PT fdisk (gdisk) version 1.0.5

Caution: invalid main GPT header, but valid backup; regenerating main header
from backup!

Caution! After loading partitions, the CRC doesn't check out!
Warning: Invalid CRC on main header data; loaded backup partition table.
Warning! Main and backup partition tables differ! Use the 'c' and 'e' options
on the recovery & transformation menu to examine the two tables.

Warning! One or more CRCs don't match. You should repair the disk!
Main header: ERROR
Backup header: OK
Main partition table: ERROR
Backup partition table: ERROR

Partition table scan:
  MBR: protective
  BSD: not present
  APM: not present
  GPT: damaged

****************************************************************************
Caution: Found protective or hybrid MBR and corrupt GPT. Using GPT, but disk
verification and recovery are STRONGLY recommended.
****************************************************************************
Disk /dev/sdb: 1953458176 sectors, 931.5 GiB
Model: Elements 25A2   
Sector size (logical/physical): 512/512 bytes
Disk identifier (GUID): BCA95A74-1E0A-4648-9971-20ED4049CAE5
Partition table holds up to 128 entries
Main partition table begins at sector 2 and ends at sector 33
First usable sector is 34, last usable sector is 1953458142
Partitions will be aligned on 2048-sector boundaries
Total free space is 1953458109 sectors (931.5 GiB)

Number  Start (sector)    End (sector)  Size       Code  Name



"fix manually" scares me...especially because I have no place for
1TB of an image file to with which I can experiment ...

Any ideas which could ease my burden and to un-scare my 
"need to fix it manually" ??? ;) ;)

Cheers!
Meino

Re: [gentoo-user] Trouble with backup harddisks

[Wynn Wolf Arbor]

All the following assuming that the disk was originally partitioned as 
GPT, but after that exclusively accessed as an MBR disk.

>PT fdisk (gdisk) version 1.0.5
>
>Caution: invalid main GPT header, but valid backup; regenerating main header
>from backup!

This makes sense since the GPT backup at the very end of the disk would 
most likely still be intact. gdisk identifies it correctly, but assumes 
(wrongly) that the data on the disk is governed by the GPT layout.

Since the disk was only ever accessed through an operating system that 
knew solely about MBR, the GPT data meant nothing to it. It happily 
wrote data past the MBR headers. Because the protective MBR is 
positioned before GPT information, the primary GPT header was destroyed 
and most likely overwritten with the file system. See also [1], the 
actual file system data probably begins somewhere past LBA 0.

>Caution! After loading partitions, the CRC doesn't check out!
>Warning: Invalid CRC on main header data; loaded backup partition table.
>Warning! Main and backup partition tables differ! Use the 'c' and 'e' options
>on the recovery & transformation menu to examine the two tables.

This is because the backup GPT written when first partitioned does no 
longer match the data present at the very beginning of the disk.

If the initial assumption is correct, GPT *must not* be restored. Your 
modern PC sees the GPT partition type and assumes the existence of a 
GPT. It should, however, access the MBR layout and interpret the 
partition marked with the GPT ID as a regular partition.

Now, how to fix this?

Like Andrea already said earlier: 

> Since the disk is only 1TB, there is no reason to use GPT at all, so 
> your best bet is to use fdisk to make that a standard MBR by changing 
> the partition type from 'ee' to '83'. 

This would *not* repartition or reformat any data, it would simply tell 
your modern operating system to access the protective partition as a 
regular one.

It would, however, require writing the new type to disk. What you could 
do to be more safe here is to take a backup of the first 512 bytes with 
`dd', then change the partition ID with `fdisk', and try mounting it.

If it works, great. If not, you can restore the first 512 bytes of the 
disk with the backup.

>"fix manually" scares me...especially because I have no place for
>1TB of an image file to with which I can experiment ...

>Any ideas which could ease my burden and to un-scare my
>"need to fix it manually" ??? ;) ;)

There's a few alternatives:

1) Boot an older system that only understands MBR, and mount the disk 
there. This was suggested earlier but was dismissed because we assumed 
the sector size had something to do with it. I do not think this is the 
case anymore - the old system should be able to read it.

2) Boot a VM with a kernel that only understands MBR, pass USB through 
to the virtual machine, mount the disk there.

3) Try confirming that there exists file system data past the MBR header.

Maybe something like this:

# dd if=/dev/sdb of=sdb-data bs=512 skip=1 count=16384
$ file sdb-data

As established, the block size is 512 bytes. We skip the first 512 bytes 
since that is the protective MBR. sdb-data should then contain the first 
8MiB worth of actual file system data. The `file' utility can tell you 
what kind of data it is.

[1] https://en.wikipedia.org/wiki/GUID_Partition_Table#/media/File:GUID_Partition_Table_Scheme.svg

-- 
Wolf

Re: [gentoo-user] Trouble with backup harddisks

[tuxic]

Hi Wolf,

thanks for your great input again!

(see below)

On 04/30 09:27, Wynn Wolf Arbor wrote:
> All the following assuming that the disk was originally partitioned as GPT,
> but after that exclusively accessed as an MBR disk.
> 
> > PT fdisk (gdisk) version 1.0.5
> > 
> > Caution: invalid main GPT header, but valid backup; regenerating main header
> > from backup!
> 
> This makes sense since the GPT backup at the very end of the disk would most
> likely still be intact. gdisk identifies it correctly, but assumes (wrongly)
> that the data on the disk is governed by the GPT layout.
> 
> Since the disk was only ever accessed through an operating system that knew
> solely about MBR, the GPT data meant nothing to it. It happily wrote data
> past the MBR headers. Because the protective MBR is positioned before GPT
> information, the primary GPT header was destroyed and most likely
> overwritten with the file system. See also [1], the actual file system data
> probably begins somewhere past LBA 0.
> 
> > Caution! After loading partitions, the CRC doesn't check out!
> > Warning: Invalid CRC on main header data; loaded backup partition table.
> > Warning! Main and backup partition tables differ! Use the 'c' and 'e' options
> > on the recovery & transformation menu to examine the two tables.
> 
> This is because the backup GPT written when first partitioned does no longer
> match the data present at the very beginning of the disk.
> 
> If the initial assumption is correct, GPT *must not* be restored. Your
> modern PC sees the GPT partition type and assumes the existence of a GPT. It
> should, however, access the MBR layout and interpret the partition marked
> with the GPT ID as a regular partition.
> 
> Now, how to fix this?
> 
> Like Andrea already said earlier:
> 
> > Since the disk is only 1TB, there is no reason to use GPT at all, so
> > your best bet is to use fdisk to make that a standard MBR by changing
> > the partition type from 'ee' to '83'.
> 
> This would *not* repartition or reformat any data, it would simply tell your
> modern operating system to access the protective partition as a regular one.
> 
> It would, however, require writing the new type to disk. What you could do
> to be more safe here is to take a backup of the first 512 bytes with `dd',
> then change the partition ID with `fdisk', and try mounting it.
> 
> If it works, great. If not, you can restore the first 512 bytes of the disk
> with the backup.
> 
> > "fix manually" scares me...especially because I have no place for
> > 1TB of an image file to with which I can experiment ...
> 
> > Any ideas which could ease my burden and to un-scare my
> > "need to fix it manually" ??? ;) ;)
> 
> There's a few alternatives:
> 
> 1) Boot an older system that only understands MBR, and mount the disk there.
> This was suggested earlier but was dismissed because we assumed the sector
> size had something to do with it. I do not think this is the case anymore -
> the old system should be able to read it.
> 
> 2) Boot a VM with a kernel that only understands MBR, pass USB through to
> the virtual machine, mount the disk there.
> 
> 3) Try confirming that there exists file system data past the MBR header.
> 
> Maybe something like this:
> 
> # dd if=/dev/sdb of=sdb-data bs=512 skip=1 count=16384
> $ file sdb-data
> 
> As established, the block size is 512 bytes. We skip the first 512 bytes
> since that is the protective MBR. sdb-data should then contain the first
> 8MiB worth of actual file system data. The `file' utility can tell you what
> kind of data it is.
> 
> [1] https://en.wikipedia.org/wiki/GUID_Partition_Table#/media/File:GUID_Partition_Table_Scheme.svg
> 
> -- 
> Wolf
> 

I had booted into my old system, attached the disks and both show the
same behaviour: Only the device itself (/dev/sdb) was recognized.

'file' shows the following output:

file sdb-data
sdb-data: Linux rev 1.0 ext4 filesystem data, UUID=2f063705-0d3a-4790-9203-1b4edab7788c (extents) (64bit) (large files) (huge files)

Looks better than I have thought...or?

I will take a deeper look tommorrow...I am too tired to
"fix partition tables manually" this evening!

Read you tommorrow! :)

Cheers!
Meino

Re: [gentoo-user] Trouble with backup harddisks

[Wynn Wolf Arbor]

Hi Meino,

On 2020-04-30 21:46, tuxic@posteo.de wrote:
>I had booted into my old system, attached the disks and both show the
>same behaviour: Only the device itself (/dev/sdb) was recognized.

Now that is very curious. Just to make sure, the old system definitely 
does not understand GPT? CONFIG_EFI_PARTITION should be unset.

>'file' shows the following output:
>
>file sdb-data
>sdb-data: Linux rev 1.0 ext4 filesystem data, UUID=2f063705-0d3a-4790-9203-1b4edab7788c (extents) (64bit) (large files) (huge files)
>
>Looks better than I have thought...or?

This does indeed look promising (and is what I expected in the best 
case). Now, of course, the problem is figuring out how to get to that 
data.

>I will take a deeper look tommorrow...I am too tired to
>"fix partition tables manually" this evening!
>
>Read you tommorrow! :)

Hope the rest of your evening will be more relaxing!

-- 
Wolf

Re: [gentoo-user] Trouble with backup harddisks

[Andrea Conti]

> Since the disk was only ever accessed through an operating system that knew solely about MBR, the GPT data meant nothing to it. It happily wrote data past the MBR headers. Because the protective MBR is positioned before GPT information, the primary GPT header was destroyed and most likely overwritten with the file system. See also [1], the actual file system data probably begins somewhere past LBA 0.

If it was created according to the MBR that was shown previously, it will start at the beginning of the protective entry, i.e. at sector 1.

> If the initial assumption is correct, GPT *must not* be restored. Your modern PC sees the GPT partition type and assumes the existence of a GPT. It should, however, access the MBR layout and interpret the partition marked with the GPT ID as a regular partition.

It won't, as long as it recognizes it as a protective MBR. Which is the right thing to do, as a disk with a protective MBR and no valid GPT is inherently broken.

> 1) Boot an older system that only understands MBR, and mount the disk there. This was suggested earlier but was dismissed because we assumed the sector size had something to do with it. I do not think this is the case anymore - the old system should be able to read it.
> 
> 2) Boot a VM with a kernel that only understands MBR, pass USB through to the virtual machine, mount the disk there.
> 
> 3) Try confirming that there exists file system data past the MBR header.
>
> Maybe something like this:
> 
> # dd if=/dev/sdb of=sdb-data bs=512 skip=1 count=16384
> $ file sdb-data 


... or just bypass the partition table altogether. The filesystem starts at sector 1, i.e. 1*512B, so:

mount -o ro,offset=512 /dev/sdb /mnt/xxx

But while this will allow you to access your data, you will still have a broken disk until you fix the MBR.


andrea

Re: [gentoo-user] Trouble with backup harddisks

[Wynn Wolf Arbor]

On 2020-04-30 22:21, Andrea Conti wrote:
>It won't, as long as it recognizes it as a protective MBR. Which is the 
>right thing to do, as a disk with a protective MBR and no valid GPT is 
>inherently broken.

True. It was more my intention to depict what the system "should" do in 
order to access the file system.

>... or just bypass the partition table altogether. The filesystem 
>starts at sector 1, i.e. 1*512B, so:
>
>mount -o ro,offset=512 /dev/sdb /mnt/xxx

Interesting, thanks. I was initially considering something like this 
myself, but after a cursory check of the manual, I was under the 
assumption that 'offset' was only valid with loop devices and dismissed 
that solution.

Turns out if you mount a drive like this, the kernel uses a loop device 
in the background and you can use the 'offset' option with block devices 
as well. I feel the documentation could be improved here.

-- 
Wolf

Re: [gentoo-user] Trouble with backup harddisks

[tuxic]

On 04/30 10:47, Wynn Wolf Arbor wrote:
> On 2020-04-30 22:21, Andrea Conti wrote:
> > It won't, as long as it recognizes it as a protective MBR. Which is the
> > right thing to do, as a disk with a protective MBR and no valid GPT is
> > inherently broken.
> 
> True. It was more my intention to depict what the system "should" do in
> order to access the file system.
> 
> > ... or just bypass the partition table altogether. The filesystem starts
> > at sector 1, i.e. 1*512B, so:
> > 
> > mount -o ro,offset=512 /dev/sdb /mnt/xxx
> 
> Interesting, thanks. I was initially considering something like this myself,
> but after a cursory check of the manual, I was under the assumption that
> 'offset' was only valid with loop devices and dismissed that solution.
> 
> Turns out if you mount a drive like this, the kernel uses a loop device in
> the background and you can use the 'offset' option with block devices as
> well. I feel the documentation could be improved here.
> 
> -- 
> Wolf
> 

Hi Andrea, hi Wolf

does my posting from this morning reached you ?
...I did not received anything back from the mailinglist...

Cheers!
Meino

Re: [gentoo-user] Trouble with backup harddisks

[Andrea Conti]

> does my posting from this morning reached you ?
> ...I did not received anything back from the mailinglist...

Nope. Just this night's response to Wol.

Re: [gentoo-user] Trouble with backup harddisks

[tuxic]

On 05/01 08:52, Andrea Conti wrote:
> 
> > does my posting from this morning reached you ?
> > ...I did not received anything back from the mailinglist...
> 
> Nope. Just this night's response to Wol.
> 
> 

(hmmm...ok, two send good news two times is not
that bad in this times, I think... ;)

Hi Andrea, hi Wolf,

this magic one liner does the trick indeed!  I have my data back and
copied it to my new system spread over four separate partitions, which
are currently still empty. No error while copuing and both disk
matched.

A very *#BIG THANK YOU#* for all the great help, the research and
the solution. I myself am back in "normal mode" :)

Wol has suggested to run mdadm on the bad harddisk to see, whether
this tool is able to fix it -- which I will next. Since I have
my data back on the PC internal disk, this is no problem at all.

One thing remains...
I want to prevent this kind of hassle in the future... ;)

Perhaps it is a good idea to re-partitions the disk to get rid of
any bogus bit, format the partition and copy back the data then.

What is the most reasonable setup here:
GPT without any hybrid magic and ext4 because it is so common?
Any other, possible more robust configuration, which is also
common with rescue tools and -distributions?

I have to say it once again: Thank you very much for solving
the hassle puzzle! :)

Cheers!
Meino

Re: [gentoo-user] Trouble with backup harddisks

[Wynn Wolf Arbor]

On 2020-05-01 09:18, tuxic@posteo.de wrote:
> A very *#BIG THANK YOU#* for all the great help, the research and
> the solution. I myself am back in "normal mode" :)

Glad it helped!

> One thing remains...
> I want to prevent this kind of hassle in the future... ;)

The most important thing to keep in mind here is to _always_ partition 
and format a drive yourself. That way, you know what is on there and how 
it was partitioned.

> Perhaps it is a good idea to re-partitions the disk to get rid of
> any bogus bit, format the partition and copy back the data then.

Whilst you can definitely just change the partition type from EE to 83, 
now that you can move the data around safely, I'd probably repartition 
the drives fully, yeah.

> What is the most reasonable setup here:
> GPT without any hybrid magic and ext4 because it is so common?
> Any other, possible more robust configuration, which is also
> common with rescue tools and -distributions?

For external backup disks like these I'd go with GPT and ext4. Seems to 
be the most robust configuration for that use case.

If by "hybrid magic" you mean the Protective MBR, that is not something 
you can simply disable. The standard includes the Protective MBR, and 
tools adhere to that.

Bottom line: Repartition the disks with GPT, then create the necessary 
file systems. Don't turn on any special knobs, the tool's defaults are 
all decent.

-- 
Wolf

Re: [gentoo-user] Trouble with backup harddisks

[Andrea Conti]

> A very *#BIG THANK YOU#* for all the great help, the research and
> the solution. I myself am back in "normal mode" :)

You're welcome!

> What is the most reasonable setup here:
> GPT without any hybrid magic and ext4 because it is so common?

I would go with MBR and a single ext4 partition. GPT is fine too, but for a 1TB disk with a single partition it has absolutely zero advantage over MBR.

andrea

Re: [gentoo-user] Trouble with backup harddisks

[Peter Humphrey]

On Friday, 1 May 2020 17:00:56 BST Andrea Conti wrote:

> GPT is fine too, but for a 1TB disk with a single partition it has absolutely
> zero advantage over MBR.

I can think of one or two people who might demur there.

-- 
Regards,
Peter.

Re: [gentoo-user] Trouble with backup harddisks

[tuxic]

On 05/01 05:32, Peter Humphrey wrote:
> On Friday, 1 May 2020 17:00:56 BST Andrea Conti wrote:
> 
> > GPT is fine too, but for a 1TB disk with a single partition it has absolutely
> > zero advantage over MBR.
> 
> I can think of one or two people who might demur there.
> 
> -- 
> Regards,
> Peter.
> 
> 
> 
> 

Hi Wolf, hi Andrea, hi Peter,

*Partition type '83':
I changed the type to 83 using fdisk and this worked fine. Now
/dev/sdb1 is recognized again.
Just to confirm this...
I think, I feel better if I repartitioning/reformat both drives,
though.

*GPT/MBR
From a discussion based on a "GPT or MBR for my system drive" in
conjunction with UEFI it was said, that GPT is more modern and
save.
My question was meant not so much as "MBR or GPT?" 
but more whether there are some variants of GPT (with 
protected MBR for example -- which was completly new to me),
which I should use or avoid.

But: Are rescue systems for USB-stick more UEFI/GPT aware nowadays
or "traditionally" based on MBR/BIOS-boot?

One thing I found is really handy: An USB-stick with an rEfind
installation. As long as your PC supports UEFI (or can switched to it)
rEfind is able to boot "everything" without prior configuration.

Some rescue-system which really shines and with which you have made good
experiences?

Cheers!
Meino

Re: [gentoo-user] Trouble with backup harddisks

[Andrea Conti]

> I think, I feel better if I repartitioning/reformat both drives,
> though.

It's not necessary, but if it makes you feel better by all means do so.

> *GPT/MBR
> From a discussion based on a "GPT or MBR for my system drive" in
> conjunction with UEFI it was said, that GPT is more modern and
> save.


More modern I concur. For the rest it's mainly about features: >2TB partitions and way more metadata, plus not having to bother with CHS values which make no sense in today's drives.
And being able to define >4 partitions without littering the disk with extended boot records, which is probably the only thing I'd call "safer".

My point was that none of this is relevant in an external drive which is under 1TB and will only hold a single partition starting at sector 1 and spanning the rest of the disk.
A system drive, especially if booting from UEFI is a different case for which GPT absolutely makes sense.

> My question was meant not so much as "MBR or GPT?" 
> but more whether there are some variants of GPT (with 
> protected MBR for example -- which was completly new to me),
> which I should use or avoid.

There are really no "variants" of GPT. The protective MBR is only there to make all space in the disk look allocated to MBR partitioning tools that are not GPT-aware, and is automatically written for you by all GPT partitioning tools.

In addition to the opaque entry of type 0xee, this MBR can also contain entries pointing to at least some of the actual partitions; this is called a 'hybrid' MBR and allows MBR-only access to partitions that are within the limits of MBR addressing (start and end sector <2TB). These are only useful in very specific cases an I would consider them a hack more than a solution; while gpt-fdisk has some support for creating hybrid MBRs (don't know about fdisk), you won't get one unless you specifically ask for it.

> But: Are rescue systems for USB-stick more UEFI/GPT aware nowadays
> or "traditionally" based on MBR/BIOS-boot?

I think that anything that's not ancient will have tools and kernel support for both MBR and GPT, and will boot fine in both BIOS and UEFI modes.

> One thing I found is really handy: An USB-stick with an rEfind
> installation. As long as your PC supports UEFI (or can switched to it)
> rEfind is able to boot "everything" without prior configuration.

You can probably do the same with GRUB2, albeit in a way less user-friendly fashion :)
But why do you consider the ability to boot anything but the rescue system itself important in a rescue system?

> 
> Some rescue-system which really shines and with which you have made good
> experiences?

My usual go-to is SystemRescueCD (the old 5.x gentoo-based one). 

andrea

Re: [gentoo-user] Trouble with backup harddisks

[tuxic]

On 05/02 09:49, Andrea Conti wrote:
> > I think, I feel better if I repartitioning/reformat both drives,
> > though.
> 
> It's not necessary, but if it makes you feel better by all means do so.
> 
> > *GPT/MBR
> > From a discussion based on a "GPT or MBR for my system drive" in
> > conjunction with UEFI it was said, that GPT is more modern and
> > save.
> 
> 
> More modern I concur. For the rest it's mainly about features: >2TB partitions and way more metadata, plus not having to bother with CHS values which make no sense in today's drives.
> And being able to define >4 partitions without littering the disk with extended boot records, which is probably the only thing I'd call "safer".
> 
> My point was that none of this is relevant in an external drive which is under 1TB and will only hold a single partition starting at sector 1 and spanning the rest of the disk.
> A system drive, especially if booting from UEFI is a different case for which GPT absolutely makes sense.
> 

Ok, the other way around: Does GPT hurt more than MBT on a external HD
used for backup puporses (no boot), has 1T and 1 partion of that size?


> > My question was meant not so much as "MBR or GPT?" 
> > but more whether there are some variants of GPT (with 
> > protected MBR for example -- which was completly new to me),
> > which I should use or avoid.
> 
> There are really no "variants" of GPT. The protective MBR is only there to make all space in the disk look allocated to MBR partitioning tools that are not GPT-aware, and is automatically written for you by all GPT partitioning tools.
> 
> In addition to the opaque entry of type 0xee, this MBR can also contain entries pointing to at least some of the actual partitions; this is called a 'hybrid' MBR and allows MBR-only access to partitions that are within the limits of MBR addressing (start and end sector <2TB). These are only useful in very specific cases an I would consider them a hack more than a solution; while gpt-fdisk has some support for creating hybrid MBRs (don't know about fdisk), you won't get one unless you specifically ask for it.
>: 

Thanks of the information! :)

> > But: Are rescue systems for USB-stick more UEFI/GPT aware nowadays
> > or "traditionally" based on MBR/BIOS-boot?
> 
> I think that anything that's not ancient will have tools and kernel support for both MBR and GPT, and will boot fine in both BIOS and UEFI modes.
> 
> > One thing I found is really handy: An USB-stick with an rEfind
> > installation. As long as your PC supports UEFI (or can switched to it)
> > rEfind is able to boot "everything" without prior configuration.
> 
> You can probably do the same with GRUB2, albeit in a way less user-friendly fashion :)
> But why do you consider the ability to boot anything but the rescue system itself important in a rescue system?

Recently a BIOS update deleted all UEFI entries and the system no
longer boots. With rEfind from a USBstick I was able to boot
the sustem nonetheless and the reinstallation of grub solves
the problem.
Task accomplished! :)

> > 
> > Some rescue-system which really shines and with which you have made good
> > experiences?
> 
> My usual go-to is SystemRescueCD (the old 5.x gentoo-based one). 
> 
> andrea

Thanks for the info, Andrea!

Cheers!
Meino


 

Re: [gentoo-user] Trouble with backup harddisks

[Michael]

[-- Attachment #1: Type: text/plain, Size: 3984 bytes --]

On Saturday, 2 May 2020 09:39:12 BST tuxic@posteo.de wrote:
> On 05/02 09:49, Andrea Conti wrote:
> > > I think, I feel better if I repartitioning/reformat both drives,
> > > though.
> > 
> > It's not necessary, but if it makes you feel better by all means do so.
> > 
> > > *GPT/MBR
> > > From a discussion based on a "GPT or MBR for my system drive" in
> > > conjunction with UEFI it was said, that GPT is more modern and
> > > save.
> > 
> > More modern I concur. For the rest it's mainly about features: >2TB
> > partitions and way more metadata, plus not having to bother with CHS
> > values which make no sense in today's drives. And being able to define >4
> > partitions without littering the disk with extended boot records, which
> > is probably the only thing I'd call "safer".
> > 
> > My point was that none of this is relevant in an external drive which is
> > under 1TB and will only hold a single partition starting at sector 1 and
> > spanning the rest of the disk. A system drive, especially if booting from
> > UEFI is a different case for which GPT absolutely makes sense.
> Ok, the other way around: Does GPT hurt more than MBT on a external HD
> used for backup puporses (no boot), has 1T and 1 partion of that size?

Unless you're planning to boot from Windows XP or some antiquated old LiveCD, 
a GPT partitioning scheme is better in *all* respects and it is more robust 
than MBR because:

- The partitioning tables created by GPT are backed up at the end of the disk.
- GPT uses CRC make sure its data is intact, or will warn of corruption and 
attempt to restore from the back up.


> > > My question was meant not so much as "MBR or GPT?"
> > > but more whether there are some variants of GPT (with
> > > protected MBR for example -- which was completly new to me),
> > > which I should use or avoid.
> > 
> > There are really no "variants" of GPT. The protective MBR is only there to
> > make all space in the disk look allocated to MBR partitioning tools that
> > are not GPT-aware, and is automatically written for you by all GPT
> > partitioning tools.
> > 
> > In addition to the opaque entry of type 0xee, this MBR can also contain
> > entries pointing to at least some of the actual partitions; this is
> > called a 'hybrid' MBR and allows MBR-only access to partitions that are
> > within the limits of MBR addressing (start and end sector <2TB). These
> > are only useful in very specific cases an I would consider them a hack
> > more than a solution; while gpt-fdisk has some support for creating
> > hybrid MBRs (don't know about fdisk), you won't get one unless you
> > specifically ask for it.
> Thanks of the information! :)
> 
> > > But: Are rescue systems for USB-stick more UEFI/GPT aware nowadays
> > > or "traditionally" based on MBR/BIOS-boot?
> > 
> > I think that anything that's not ancient will have tools and kernel
> > support for both MBR and GPT, and will boot fine in both BIOS and UEFI
> > modes.> 
> > > One thing I found is really handy: An USB-stick with an rEfind
> > > installation. As long as your PC supports UEFI (or can switched to it)
> > > rEfind is able to boot "everything" without prior configuration.
> > 
> > You can probably do the same with GRUB2, albeit in a way less
> > user-friendly fashion :) But why do you consider the ability to boot
> > anything but the rescue system itself important in a rescue system?
> Recently a BIOS update deleted all UEFI entries and the system no
> longer boots. With rEfind from a USBstick I was able to boot
> the sustem nonetheless and the reinstallation of grub solves
> the problem.
> Task accomplished! :)
> 
> > > Some rescue-system which really shines and with which you have made good
> > > experiences?
> > 
> > My usual go-to is SystemRescueCD (the old 5.x gentoo-based one).
> > 
> > andrea
> 
> Thanks for the info, Andrea!
> 
> Cheers!
> Meino

Any up to date Linux LiveCD/USB should be able to boot your PC and 
automatically recognise its GPT partitioning.

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 833 bytes --]