[gentoo-user] md-device too small: where's my mistake?

[gentoo-user] md-device too small: where's my mistake?

[Stefan G. Weichinger]

Greets, I currently try to setup a md-device containing of 4 partitions
on 4 SATA-drives:

# fdisk -l | grep "sd"
Platte /dev/sda: 1000.2 GByte, 1000204886016 Byte
/dev/sda1               1           2       16033+  fd  Linux raid
autodetect
/dev/sda2               3          21      152617+  82  Linux Swap / Solaris
/dev/sda3              22        1116     8795587+  fd  Linux raid
autodetect
/dev/sda4            1117      121601   967795762+  fd  Linux raid
autodetect
Platte /dev/sdb: 1000.2 GByte, 1000204886016 Byte
/dev/sdb1               1           2       16033+  fd  Linux raid
autodetect
/dev/sdb2               3          21      152617+  82  Linux Swap / Solaris
/dev/sdb3              22        1116     8795587+  fd  Linux raid
autodetect
/dev/sdb4            1117      121601   967795762+  fd  Linux raid
autodetect
Platte /dev/sdc: 1000.2 GByte, 1000204886016 Byte
/dev/sdc1               1           2       16033+  fd  Linux raid
autodetect
/dev/sdc2               3          21      152617+  82  Linux Swap / Solaris
/dev/sdc3              22        1116     8795587+  fd  Linux raid
autodetect
/dev/sdc4            1117      121601   967795762+  fd  Linux raid
autodetect
Platte /dev/sdd: 1000.2 GByte, 1000204886016 Byte
/dev/sdd1               1           2       16033+  fd  Linux raid
autodetect
/dev/sdd2               3          21      152617+  82  Linux Swap / Solaris
/dev/sdd3              22        1116     8795587+  fd  Linux raid
autodetect
/dev/sdd4            1117      121601   967795762+  fd  Linux raid
autodetect


I want a RAID5-device md2 (without hotspare, while md1 *has* a
hotspare-partition) containing the partitions "sd?4".

I have now:

cat /proc/mdstat
Personalities : [raid0] [raid1] [raid10] [raid6] [raid5] [raid4]
md1 : active raid5 sdd3[3](S) sdc3[2] sdb3[1] sda3[0]
      17591040 blocks level 5, 64k chunk, algorithm 2 [3/3] [UUU]

md2 : active raid5 sdd4[3] sdc4[2] sdb4[1] sda4[0]
      2903386944 blocks level 5, 64k chunk, algorithm 2 [4/4] [UUUU]

md0 : active raid1 sdb1[1] sda1[0]
      15936 blocks [2/2] [UU]


md2 is the one that gives me headaches. AFAI understand it should be
about 3TB in size, but it is only 774 GB ....

# fdisk -l /dev/md2

Platte /dev/md2: 774.0 GByte, 774044975104 Byte
2 Köpfe, 4 Sektoren/Spuren, 188975824 Zylinder
Einheiten = Zylinder von 8 × 512 = 4096 Bytes
Disk identifier: 0x00000000


Why?

# cat /proc/version
Linux version 2.6.25-gentoo-r8 (root@horde) (gcc version 4.1.2 (Gentoo
4.1.2 p1.1)) #2 SMP Wed Oct 22 11:07:19 CEST 2008

Maybe anything wrong in my kernel?

I also noticed that only 1G RAM of 2 had been detected as I had no
SMP-support in the previous kernel ... now I have ...

Thanks for any pointer ....
Stefan

Re: [gentoo-user] md-device too small: where's my mistake?

[Stefan G. Weichinger]

Stefan G. Weichinger schrieb:

> md2 is the one that gives me headaches. AFAI understand it should be
> about 3TB in size, but it is only 774 GB ....
> 
> # fdisk -l /dev/md2
> 
> Platte /dev/md2: 774.0 GByte, 774044975104 Byte

I am surprised that noone seems to have advice for me.
Is my mistake so obvious?

Stefan

Re: [gentoo-user] md-device too small: where's my mistake?

[Heiko Wundram]

Am Wednesday 22 October 2008 15:58:44 schrieb Stefan G. Weichinger:
> md2 is the one that gives me headaches. AFAI understand it should be
> about 3TB in size, but it is only 774 GB ....
>
> # fdisk -l /dev/md2
>
> Platte /dev/md2: 774.0 GByte, 774044975104 Byte
> 2 Köpfe, 4 Sektoren/Spuren, 188975824 Zylinder
> Einheiten = Zylinder von 8 × 512 = 4096 Bytes
> Disk identifier: 0x00000000
>
>
> Why?

You cannot manage disks >= ~2TB with fdisk (i.e., DOS partition tables), as 
they (or rather the on-disk-structure of DOS partition tables) have an 
inherent limitation in the maximum number of LBA48-blocks they can address.

I'd presume that because of this inherent limitation, fdisk is reporting the 
wrong total size (2TB+774G+epsilon ~ 3TB; sounds like somewhere someone is 
doing a modulo operation, possibly), and completely "off" values for 
heads/sectors.

Anyway, md-devices cannot be partitioned anyway (of course you can write a 
partition table on them, but the kernel won't use that to create md2-1,-2, 
etc.), so using fdisk is wrong.

If you want to check the "real" size of the device, don't use fdisk, but 
rather use

blockdev --getsize64 /dev/md2

which shows you the byte-count of the corresponding volume, and which I think 
will be 3TB, as you want it to be.

If you want to subpartition large devices, use lvm(2), which does not have the 
2TB limitation on size.

Hope this helps!

-- 
Heiko Wundram
hackerkey://v4sw7CHJLSUY$hw5ln5pr7FOP$ck2ma9u7FL$w3DVWXm0l7GL$i65e6t3EMRSXb7ADORen5a26s5MSr2p-6.62/-6.56g5AORZ

Re: [gentoo-user] md-device too small: where's my mistake?

[Stefan G. Weichinger]

Heiko Wundram schrieb:

> You cannot manage disks >= ~2TB with fdisk (i.e., DOS partition tables), as 
> they (or rather the on-disk-structure of DOS partition tables) have an 
> inherent limitation in the maximum number of LBA48-blocks they can address.
> 
> I'd presume that because of this inherent limitation, fdisk is reporting the 
> wrong total size (2TB+774G+epsilon ~ 3TB; sounds like somewhere someone is 
> doing a modulo operation, possibly), and completely "off" values for 
> heads/sectors.
> 
> Anyway, md-devices cannot be partitioned anyway (of course you can write a 
> partition table on them, but the kernel won't use that to create md2-1,-2, 
> etc.), so using fdisk is wrong.
> 
> If you want to check the "real" size of the device, don't use fdisk, but 
> rather use
> 
> blockdev --getsize64 /dev/md2
> 
> which shows you the byte-count of the corresponding volume, and which I think 
> will be 3TB, as you want it to be.

Nope, it did show the same 774 GB.

> If you want to subpartition large devices, use lvm(2), which does not have the 
> 2TB limitation on size.
> 
> Hope this helps!

Thanks for your explanations and suggestions ... but ...

I just now received a reply to my posting on the german list, the
mistake was that CONFIG_LBD was not set in my kernel.

Now I get:

# blockdev --getsize64 /dev/md2
2973068230656

# fdisk -l
Platte /dev/md2: 2973.0 GByte, 2973068230656 Byte

Thanks anyway!
Stefan