[gentoo-user] dd says no space left on device

[gentoo-user] dd says no space left on device

[Adam Carter]

[-- Attachment #1: Type: text/plain, Size: 965 bytes --]

I'm cloning a windows disk using gentoo;

On the old 66GB disk;
# dd if=/dev/sdb of=/root/winmbr.bin bs=512 count=1
# dd if=/dev/sdb1 bs=10M | gzip -v > winpartition.gz

Then after swapping in the new 500GB disk;
dd if=/root/winmbr.bin of=/dev/sdb bs=512 count=1
# gunzip -c winpartition.gz | dd of=/dev/sdb1 bs=10M
dd: writing `/dev/sdb1': No space left on device
0+306 records in
0+305 records out
10137600 bytes (10 MB) copied, 0.109885 s, 92.3 MB/s
# fdisk -l /dev/sdb

Disk /dev/sdb: 500.1 GB, 500107862016 bytes
255 heads, 63 sectors/track, 60801 cylinders, total 976773168 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0xe3f7e3f7

   Device Boot      Start         End      Blocks   Id  System
/dev/sdb1   *      206848   117207039    58500096    7  HPFS/NTFS/exFAT

Why is dd saying no space left after copying 10MB when sdb1 is 65GB?

[-- Attachment #2: Type: text/html, Size: 1073 bytes --]

Re: [gentoo-user] dd says no space left on device

[Joost Roeleveld]

On Sunday 15 May 2011 17:45:05 Adam Carter wrote:
> I'm cloning a windows disk using gentoo;
> 
> On the old 66GB disk;
> # dd if=/dev/sdb of=/root/winmbr.bin bs=512 count=1
> # dd if=/dev/sdb1 bs=10M | gzip -v > winpartition.gz
> 
> Then after swapping in the new 500GB disk;
> dd if=/root/winmbr.bin of=/dev/sdb bs=512 count=1
> # gunzip -c winpartition.gz | dd of=/dev/sdb1 bs=10M
> dd: writing `/dev/sdb1': No space left on device
> 0+306 records in
> 0+305 records out
> 10137600 bytes (10 MB) copied, 0.109885 s, 92.3 MB/s
> # fdisk -l /dev/sdb
> 
> Disk /dev/sdb: 500.1 GB, 500107862016 bytes
> 255 heads, 63 sectors/track, 60801 cylinders, total 976773168 sectors
> Units = sectors of 1 * 512 = 512 bytes
> Sector size (logical/physical): 512 bytes / 512 bytes
> I/O size (minimum/optimal): 512 bytes / 512 bytes
> Disk identifier: 0xe3f7e3f7
> 
>    Device Boot      Start         End      Blocks   Id  System
> /dev/sdb1   *      206848   117207039    58500096    7  HPFS/NTFS/exFAT
> 
> Why is dd saying no space left after copying 10MB when sdb1 is 65GB?

Did you reboot after the first "dd"?
Or at least, force a re-read of the partition tables?

Linux caches the partition tables and when overwriting the partition table, 
strange things will happen.

--
Joost

Re: [gentoo-user] dd says no space left on device

[Alex Schuster]

Joost Roeleveld writes:

> On Sunday 15 May 2011 17:45:05 Adam Carter wrote:

>> Why is dd saying no space left after copying 10MB when sdb1 is 65GB?
> 
> Did you reboot after the first "dd"?

Probably, undless he is using som external drive.

> Or at least, force a re-read of the partition tables?

partprobe /dev/sdb will do this. partprobe is in sys-block/parted.

> Linux caches the partition tables and when overwriting the partition table, 
> strange things will happen.

Sounds like the drive is new, and probably not partitioned at all. When
/dev/sdb1 shows up, I's say the partitioning is okay. Is fdisk (I prefer
cfdisk) showing the correct layout?

	Wonko

Re: [gentoo-user] dd says no space left on device

[Mick]

[-- Attachment #1: Type: Text/Plain, Size: 1902 bytes --]

On Sunday 15 May 2011 08:45:05 Adam Carter wrote:
> I'm cloning a windows disk using gentoo;
> 
> On the old 66GB disk;
> # dd if=/dev/sdb of=/root/winmbr.bin bs=512 count=1
> # dd if=/dev/sdb1 bs=10M | gzip -v > winpartition.gz
> 
> Then after swapping in the new 500GB disk;
> dd if=/root/winmbr.bin of=/dev/sdb bs=512 count=1
> # gunzip -c winpartition.gz | dd of=/dev/sdb1 bs=10M
> dd: writing `/dev/sdb1': No space left on device
> 0+306 records in
> 0+305 records out
> 10137600 bytes (10 MB) copied, 0.109885 s, 92.3 MB/s
> # fdisk -l /dev/sdb
> 
> Disk /dev/sdb: 500.1 GB, 500107862016 bytes
> 255 heads, 63 sectors/track, 60801 cylinders, total 976773168 sectors
> Units = sectors of 1 * 512 = 512 bytes
> Sector size (logical/physical): 512 bytes / 512 bytes
> I/O size (minimum/optimal): 512 bytes / 512 bytes
> Disk identifier: 0xe3f7e3f7
> 
>    Device Boot      Start         End      Blocks   Id  System
> /dev/sdb1   *      206848   117207039    58500096    7  HPFS/NTFS/exFAT
> 
> Why is dd saying no space left after copying 10MB when sdb1 is 65GB?

Not sure if the bs=10M is too large?

You can try finding the optimum size of the bs= value by creating a partition 
on the new disk, formating it and then run something like:

dd if=/dev/zero bs=1024 count=1000000 of=/1G_test.file
dd if=/dev/zero bs=2048 count=500000 of=/1G_test.file
dd if=/dev/zero bs=4096 count=250000 of=/1G_test.file
dd if=/dev/zero bs=8192 count=125000 of=/1G_test.file

and compare the results that dd reports.  bs=4096 often gives best performance 
(on my drives at least) but with the new 1T+ drives you may find that another 
block size does the job better.

Then zero the drive first using dd:

dd if=/dev/zero of=/dev/sdb bs=4096 oflag=direct conv=notrunc

and try repeating your restoring from back up with a more suitable block size.
-- 
Regards,
Mick

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: [gentoo-user] dd says no space left on device

[Volker Armin Hemmann]

On Sunday 15 May 2011 18:52:21 Mick wrote:
> On Sunday 15 May 2011 08:45:05 Adam Carter wrote:
> > I'm cloning a windows disk using gentoo;
> > 
> > On the old 66GB disk;
> > # dd if=/dev/sdb of=/root/winmbr.bin bs=512 count=1
> > # dd if=/dev/sdb1 bs=10M | gzip -v > winpartition.gz
> > 
> > Then after swapping in the new 500GB disk;
> > dd if=/root/winmbr.bin of=/dev/sdb bs=512 count=1
> > # gunzip -c winpartition.gz | dd of=/dev/sdb1 bs=10M
> > dd: writing `/dev/sdb1': No space left on device
> > 0+306 records in
> > 0+305 records out
> > 10137600 bytes (10 MB) copied, 0.109885 s, 92.3 MB/s
> > # fdisk -l /dev/sdb
> > 
> > Disk /dev/sdb: 500.1 GB, 500107862016 bytes
> > 255 heads, 63 sectors/track, 60801 cylinders, total 976773168 sectors
> > Units = sectors of 1 * 512 = 512 bytes
> > Sector size (logical/physical): 512 bytes / 512 bytes
> > I/O size (minimum/optimal): 512 bytes / 512 bytes
> > Disk identifier: 0xe3f7e3f7
> > 
> >    Device Boot      Start         End      Blocks   Id  System
> > 
> > /dev/sdb1   *      206848   117207039    58500096    7  HPFS/NTFS/exFAT
> > 
> > Why is dd saying no space left after copying 10MB when sdb1 is 65GB?
> 
> Not sure if the bs=10M is too large?
> 
> You can try finding the optimum size of the bs= value by creating a
> partition on the new disk, formating it and then run something like:
> 
> dd if=/dev/zero bs=1024 count=1000000 of=/1G_test.file
> dd if=/dev/zero bs=2048 count=500000 of=/1G_test.file
> dd if=/dev/zero bs=4096 count=250000 of=/1G_test.file
> dd if=/dev/zero bs=8192 count=125000 of=/1G_test.file
> 
> and compare the results that dd reports.  bs=4096 often gives best
> performance (on my drives at least) but with the new 1T+ drives you may
> find that another block size does the job better.
> 
> Then zero the drive first using dd:
> 
> dd if=/dev/zero of=/dev/sdb bs=4096 oflag=direct conv=notrunc
> 
> and try repeating your restoring from back up with a more suitable block
> size.

a) sector sizes are mentioned in the docu
b) compeletly unrelated.

Re: [gentoo-user] dd says no space left on device

[Mick]

[-- Attachment #1: Type: Text/Plain, Size: 2364 bytes --]

On Sunday 15 May 2011 19:15:16 Volker Armin Hemmann wrote:
> On Sunday 15 May 2011 18:52:21 Mick wrote:
> > On Sunday 15 May 2011 08:45:05 Adam Carter wrote:
> > > I'm cloning a windows disk using gentoo;
> > > 
> > > On the old 66GB disk;
> > > # dd if=/dev/sdb of=/root/winmbr.bin bs=512 count=1
> > > # dd if=/dev/sdb1 bs=10M | gzip -v > winpartition.gz
> > > 
> > > Then after swapping in the new 500GB disk;
> > > dd if=/root/winmbr.bin of=/dev/sdb bs=512 count=1
> > > # gunzip -c winpartition.gz | dd of=/dev/sdb1 bs=10M
> > > dd: writing `/dev/sdb1': No space left on device
> > > 0+306 records in
> > > 0+305 records out
> > > 10137600 bytes (10 MB) copied, 0.109885 s, 92.3 MB/s
> > > # fdisk -l /dev/sdb
> > > 
> > > Disk /dev/sdb: 500.1 GB, 500107862016 bytes
> > > 255 heads, 63 sectors/track, 60801 cylinders, total 976773168 sectors
> > > Units = sectors of 1 * 512 = 512 bytes
> > > Sector size (logical/physical): 512 bytes / 512 bytes
> > > I/O size (minimum/optimal): 512 bytes / 512 bytes
> > > Disk identifier: 0xe3f7e3f7
> > > 
> > >    Device Boot      Start         End      Blocks   Id  System
> > > 
> > > /dev/sdb1   *      206848   117207039    58500096    7  HPFS/NTFS/exFAT
> > > 
> > > Why is dd saying no space left after copying 10MB when sdb1 is 65GB?
> > 
> > Not sure if the bs=10M is too large?
> > 
> > You can try finding the optimum size of the bs= value by creating a
> > partition on the new disk, formating it and then run something like:
> > 
> > dd if=/dev/zero bs=1024 count=1000000 of=/1G_test.file
> > dd if=/dev/zero bs=2048 count=500000 of=/1G_test.file
> > dd if=/dev/zero bs=4096 count=250000 of=/1G_test.file
> > dd if=/dev/zero bs=8192 count=125000 of=/1G_test.file
> > 
> > and compare the results that dd reports.  bs=4096 often gives best
> > performance (on my drives at least) but with the new 1T+ drives you may
> > find that another block size does the job better.
> > 
> > Then zero the drive first using dd:
> > 
> > dd if=/dev/zero of=/dev/sdb bs=4096 oflag=direct conv=notrunc
> > 
> > and try repeating your restoring from back up with a more suitable block
> > size.
> 
> a) sector sizes are mentioned in the docu
> b) compeletly unrelated.

You're right, but why is it stopping after the first 10M is transferred then?
-- 
Regards,
Mick

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: [gentoo-user] dd says no space left on device

[Volker Armin Hemmann]

On Sunday 15 May 2011 19:40:30 Mick wrote:
> On Sunday 15 May 2011 19:15:16 Volker Armin Hemmann wrote:
> > On Sunday 15 May 2011 18:52:21 Mick wrote:
> > > On Sunday 15 May 2011 08:45:05 Adam Carter wrote:
> > > > I'm cloning a windows disk using gentoo;
> > > > 
> > > > On the old 66GB disk;
> > > > # dd if=/dev/sdb of=/root/winmbr.bin bs=512 count=1
> > > > # dd if=/dev/sdb1 bs=10M | gzip -v > winpartition.gz
> > > > 
> > > > Then after swapping in the new 500GB disk;
> > > > dd if=/root/winmbr.bin of=/dev/sdb bs=512 count=1
> > > > # gunzip -c winpartition.gz | dd of=/dev/sdb1 bs=10M
> > > > dd: writing `/dev/sdb1': No space left on device
> > > > 0+306 records in
> > > > 0+305 records out
> > > > 10137600 bytes (10 MB) copied, 0.109885 s, 92.3 MB/s
> > > > # fdisk -l /dev/sdb
> > > > 
> > > > Disk /dev/sdb: 500.1 GB, 500107862016 bytes
> > > > 255 heads, 63 sectors/track, 60801 cylinders, total 976773168
> > > > sectors
> > > > Units = sectors of 1 * 512 = 512 bytes
> > > > Sector size (logical/physical): 512 bytes / 512 bytes
> > > > I/O size (minimum/optimal): 512 bytes / 512 bytes
> > > > Disk identifier: 0xe3f7e3f7
> > > > 
> > > >    Device Boot      Start         End      Blocks   Id 
> > > >    System
> > > > 
> > > > /dev/sdb1   *      206848   117207039    58500096    7 
> > > > HPFS/NTFS/exFAT
> > > > 
> > > > Why is dd saying no space left after copying 10MB when sdb1 is
> > > > 65GB?
> > > 
> > > Not sure if the bs=10M is too large?
> > > 
> > > You can try finding the optimum size of the bs= value by creating a
> > > partition on the new disk, formating it and then run something like:
> > > 
> > > dd if=/dev/zero bs=1024 count=1000000 of=/1G_test.file
> > > dd if=/dev/zero bs=2048 count=500000 of=/1G_test.file
> > > dd if=/dev/zero bs=4096 count=250000 of=/1G_test.file
> > > dd if=/dev/zero bs=8192 count=125000 of=/1G_test.file
> > > 
> > > and compare the results that dd reports.  bs=4096 often gives best
> > > performance (on my drives at least) but with the new 1T+ drives you
> > > may
> > > find that another block size does the job better.
> > > 
> > > Then zero the drive first using dd:
> > > 
> > > dd if=/dev/zero of=/dev/sdb bs=4096 oflag=direct conv=notrunc
> > > 
> > > and try repeating your restoring from back up with a more suitable
> > > block size.
> > 
> > a) sector sizes are mentioned in the docu
> > b) compeletly unrelated.
> 
> You're right, but why is it stopping after the first 10M is transferred
> then?

not sure - but I would unpack the file first. Just in case.

Re: [gentoo-user] dd says no space left on device

[Adam Carter]

[-- Attachment #1: Type: text/plain, Size: 861 bytes --]

WRT why it stopped after 10MB, if i specified a smaller size it would just
stop after whatever was specified, so its just doing a single chunk equal to
whatever bs has been specified as.

I think the re-read of the partition table is probably the problem - so
thanks for that suggestion.

To check my understanding - would it be correct to say that;
1. Using dd to copy the first 512 bytes (MBR) is ALL that is needed to setup
the partitions - that is i wont need to run fdisk etc afterward.
2. Using dd in this way of course will not update the kernel's knowledge of
the partition table so a partprobe is necessary
3. When using fdisk to write a partition table and exit, it calls a re-read
of the partition table by the kernel so any changes should be ready straight
away. (there's a message about calling ioctl when it exits - so i guess that
is the update)

[-- Attachment #2: Type: text/html, Size: 901 bytes --]

Re: [gentoo-user] dd says no space left on device

[Mick]

[-- Attachment #1: Type: Text/Plain, Size: 2235 bytes --]

On Monday 16 May 2011 02:01:13 Adam Carter wrote:
> WRT why it stopped after 10MB, if i specified a smaller size it would just
> stop after whatever was specified, so its just doing a single chunk equal
> to whatever bs has been specified as.

I recall zeroing drives/partitions and getting this message on the *second* 
run, when the partition table had already been deleted.  Recreating a 
partition table with fdisk allowed another run by dd.  Floppies did not have 
this problem (no partition tables on them).

What I suggested was to experiment with another bs just in case that was 
causing the problem of writing only one block.


> I think the re-read of the partition table is probably the problem - so
> thanks for that suggestion.

This is the most likely cause, but I cannot understand why it will write only 
one block and not the lot.


> To check my understanding - would it be correct to say that;
> 1. Using dd to copy the first 512 bytes (MBR) is ALL that is needed to
> setup the partitions - that is i wont need to run fdisk etc afterward.

This is correct if you only have primary partitions.  It will not copy the 
extended partition and any logical partitions in it.  They reside in the first 
sector of the extended partition, which is not a boot sector, but contains the 
logical partition table. (I found this out the hard way!)

Have a look at this to see how you can back up the extended partition tables 
with sfdisk (there's more than one of these, if you have more than one logical 
partition) :

http://www.partimage.org/Partimage-manual_Backup-partition-table


> 2. Using dd in this way of course will not update the kernel's knowledge of
> the partition table so a partprobe is necessary

Yes, or a reboot.


> 3. When using fdisk to write a partition table and exit, it calls a re-read
> of the partition table by the kernel so any changes should be ready
> straight away. (there's a message about calling ioctl when it exits - so i
> guess that is the update)

They are ready (i.e. written) but not yet read by the OS.  Tools like gparted 
(part)probe the device to re-read the partition table after saving changes to 
disk. 
-- 
Regards,
Mick

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: [gentoo-user] dd says no space left on device

[Adam Carter]

[-- Attachment #1: Type: text/plain, Size: 1496 bytes --]

> > To check my understanding - would it be correct to say that;
> > 1. Using dd to copy the first 512 bytes (MBR) is ALL that is needed to
> > setup the partitions - that is i wont need to run fdisk etc afterward.
>
> This is correct if you only have primary partitions.  It will not copy the
> extended partition and any logical partitions in it.  They reside in the
> first
> sector of the extended partition, which is not a boot sector, but contains
> the
> logical partition table. (I found this out the hard way!)
>
> Have a look at this to see how you can back up the extended partition
> tables
> with sfdisk (there's more than one of these, if you have more than one
> logical
> partition) :
>
> http://www.partimage.org/Partimage-manual_Backup-partition-table
>
>
> > 2. Using dd in this way of course will not update the kernel's knowledge
> of
> > the partition table so a partprobe is necessary
>
> Yes, or a reboot.
>
>
> > 3. When using fdisk to write a partition table and exit, it calls a
> re-read
> > of the partition table by the kernel so any changes should be ready
> > straight away. (there's a message about calling ioctl when it exits - so
> i
> > guess that is the update)
>
> They are ready (i.e. written) but not yet read by the OS.  Tools like
> gparted
> (part)probe the device to re-read the partition table after saving changes
> to
> disk.
>

Thanks Mick. Great info, esp about the extended partitions. Fortunately, I
dont have any on this disk but good to know.

[-- Attachment #2: Type: text/html, Size: 1951 bytes --]

Re: [gentoo-user] dd says no space left on device

[Mick]

On 16 May 2011 07:31, Adam Carter <adamcarter3@gmail.com> wrote:
>
>> > To check my understanding - would it be correct to say that;
>> > 1. Using dd to copy the first 512 bytes (MBR) is ALL that is needed to
>> > setup the partitions - that is i wont need to run fdisk etc afterward.
>>
>> This is correct if you only have primary partitions.  It will not copy the
>> extended partition and any logical partitions in it.  They reside in the
>> first
>> sector of the extended partition, which is not a boot sector, but contains
>> the
>> logical partition table. (I found this out the hard way!)
>>
>> Have a look at this to see how you can back up the extended partition
>> tables
>> with sfdisk (there's more than one of these, if you have more than one
>> logical
>> partition) :
>>
>> http://www.partimage.org/Partimage-manual_Backup-partition-table
>>
>>
>> > 2. Using dd in this way of course will not update the kernel's knowledge
>> > of
>> > the partition table so a partprobe is necessary
>>
>> Yes, or a reboot.
>>
>>
>> > 3. When using fdisk to write a partition table and exit, it calls a
>> > re-read
>> > of the partition table by the kernel so any changes should be ready
>> > straight away. (there's a message about calling ioctl when it exits - so
>> > i
>> > guess that is the update)
>>
>> They are ready (i.e. written) but not yet read by the OS.  Tools like
>> gparted
>> (part)probe the device to re-read the partition table after saving changes
>> to
>> disk.
>
> Thanks Mick. Great info, esp about the extended partitions. Fortunately, I
> dont have any on this disk but good to know.

OK, this is what I would do:

dd over the MBR (bs=512 count=1).  This will bring over the bootloader
code and the primary partition table.  Any primary partitions you had
will be copied over, same number and same size.

Then reboot.  This will read the new primary partition table.

Then run your dd command on the respective partition.  It should not
error out on the first bs, but I suggest that you also add
conv=notrunc.  and perhaps conv=notrunc,noerror.  The notrunc is
necessary to copy all sectors, otherwise dd will stop as soon as it
reaches unused sectors and truncate the test of the copy.  The noerror
will make it carry on even if there are read errors.  In this way what
you get on the new disk should be identical bit by bit with what's on
the old disk including empty space.

Then you can use gparted and resize partitions, add new ones, etc.
BTW do not resize ntfs partitions unless you have booted into them
defragged them first.

Let us know how it goes.
-- 
Regards,
Mick

Re: [gentoo-user] dd says no space left on device

[Neil Bothwick]

[-- Attachment #1: Type: text/plain, Size: 426 bytes --]

On Mon, 16 May 2011 11:37:10 +0100, Mick wrote:

> Then you can use gparted and resize partitions, add new ones, etc.
> BTW do not resize ntfs partitions unless you have booted into them
> defragged them first.

If you're going to resize/move partitions afterwards, you may as well
just dd the whole drive in one go.


-- 
Neil Bothwick

Bus: (n.) a connector you plug money into, something like a slot machine.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: [gentoo-user] dd says no space left on device

[Mick]

On 16 May 2011 11:45, Neil Bothwick <neil@digimed.co.uk> wrote:
> On Mon, 16 May 2011 11:37:10 +0100, Mick wrote:
>
>> Then you can use gparted and resize partitions, add new ones, etc.
>> BTW do not resize ntfs partitions unless you have booted into them
>> defragged them first.
>
> If you're going to resize/move partitions afterwards, you may as well
> just dd the whole drive in one go.

But the OP's new drive is larger, so I assume that he will be
rearranging partitions afterwards - could be wrong.
-- 
Regards,
Mick

Re: [gentoo-user] dd says no space left on device

[Neil Bothwick]

[-- Attachment #1: Type: text/plain, Size: 679 bytes --]

On Mon, 16 May 2011 12:16:51 +0100, Mick wrote:

> > If you're going to resize/move partitions afterwards, you may as well
> > just dd the whole drive in one go.  
> 
> But the OP's new drive is larger, so I assume that he will be
> rearranging partitions afterwards - could be wrong.

So if you've got to rearrange the partitions anyway, it is easier to just
dd the whole thing in one go and then do the rearranging.

Alternatively, set up the new partition table manually and them copy each
filesystem. dding the partition table separately from all the partitions
makes no sense.


-- 
Neil Bothwick

Of all the people I've met you're certainly one of them

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: [gentoo-user] dd says no space left on device

[Adam Carter]

[-- Attachment #1: Type: text/plain, Size: 1043 bytes --]

> > But the OP's new drive is larger, so I assume that he will be
> > rearranging partitions afterwards - could be wrong.
>
> So if you've got to rearrange the partitions anyway, it is easier to just
> dd the whole thing in one go and then do the rearranging.
>
> Alternatively, set up the new partition table manually and them copy each
> filesystem. dding the partition table separately from all the partitions
> makes no sense.
>
>
Yes the new drive is bigger, going from 66G to 500G. Single partition only,
but IIRC there is a 100MB unlabelled space (doesnt come up with fdisk -l).
Is this is an OEM recovery hidden partition?

So how do i proceed? Is it;
1. dd the mbr without partition table, to get the boot code (so bs=446
count=1)
2. use fdisk to set one big primary partition, mark it bootable and NTFS
(type 7)
3. dd into what will be /dev/sdb1

Then what? Will I be able to expand NTFS or it is better to make sure
parition size = NTFS filesystem size so it doesnt get confused, the boot
into windows and expand it?

Thanks again.

[-- Attachment #2: Type: text/html, Size: 1317 bytes --]

Re: [gentoo-user] dd says no space left on device

[Stroller]

On 16/5/2011, at 12:56pm, Adam Carter wrote:
> ...
> Yes the new drive is bigger, going from 66G to 500G. Single partition only, ...
> 
> So how do i proceed? Is it;
> 1. dd the mbr without partition table, to get the boot code (so bs=446 count=1)
> 2. use fdisk to set one big primary partition, mark it bootable and NTFS (type 7)
> 3. dd into what will be /dev/sdb1

Just `dd if=/dev/sda of=/dev/sdb` if you can.

As Neil posted: there is no need to copy MBR & partitions separately in such a simple situation. I don't know that I have ever needed to clone a hard-drive in the way that you're attempting - I have *always* cloned the whole drive (and I've cloned quite a few Windows drives this way). `dd if=/dev/sda of=/dev/sdb` with no numbers on the end of sda or sdb.

> but IIRC there is a 100MB unlabelled space (doesnt come up with fdisk -l). Is this is an OEM recovery hidden partition?

Possibly. But this question should be preceded by: "do you have an OEM recovery hidden partition?"

I mean, if the computer starts up and says "press R to boot into the Compaq recovery partition" and there's no other place for it to be, then yes. But frequently OEM recovery hidden partitions are visible in Windows' disk management console (Start > Run > "diskmgmt.msc") as partitions of DOS or unknown type, or simply showing as "hidden" or with no drive letter assigned. Therefore I'd expect them to be visible in fdisk as well. 

Whatever, if you clone the whole disk you will copy the recovery partition, if its present.

> Will I be able to expand NTFS or it is better to make sure parition size = NTFS filesystem size so it doesnt get confused, the boot into windows and expand it?

It probably depends on the version of NTFS whether this can be done natively or not. I believe that in XP you can expand D: and E: partitions from within Windows, but not C:. So to expand an XP system drive you would use Partition Magic or the GParted LiveCD [1]. I guess that Windows 7 allows you to expand C: from the management console, if not GParted claims to support it (and Vista).

I would be trying to do this *after* you have cloned the whole drive and booted with the new copy. Don't try to be clever about whether the filesystem should be the same size as the partition or not - just copy the whole lot verbatim, so they'll remain the same sizes they are now. Then use the GUI tools to expand the partition+filesystem afterwards and let those GUI tools worry about it - preferably use Windows' own tools, otherwise use Partition Magic or GParted.  

Partition Magic is my resizing tool of choice for XP, but it's neither free nor Free, nor is it supported on Vista or Windows 7.  GParted is the next choice, then - I understand it to be more than "just a graphical front-end", and I don't think you'll have such good results trying to use command-line tools to expand NTFS partitions.

Stroller.


[1] http://gparted.sourceforge.net/livecd.php

Re: [gentoo-user] dd says no space left on device

[Mick]

On 16 May 2011 15:21, Stroller <stroller@stellar.eclipse.co.uk> wrote:
>
> On 16/5/2011, at 12:56pm, Adam Carter wrote:
>> ...
>> Yes the new drive is bigger, going from 66G to 500G. Single partition only, ...
>>
>> So how do i proceed? Is it;
>> 1. dd the mbr without partition table, to get the boot code (so bs=446 count=1)
>> 2. use fdisk to set one big primary partition, mark it bootable and NTFS (type 7)
>> 3. dd into what will be /dev/sdb1
>
> Just `dd if=/dev/sda of=/dev/sdb` if you can.
>
> As Neil posted: there is no need to copy MBR & partitions separately in such a simple situation. I don't know that I have ever needed to clone a hard-drive in the way that you're attempting - I have *always* cloned the whole drive (and I've cloned quite a few Windows drives this way). `dd if=/dev/sda of=/dev/sdb` with no numbers on the end of sda or sdb.
>
>> but IIRC there is a 100MB unlabelled space (doesnt come up with fdisk -l). Is this is an OEM recovery hidden partition?
>
> Possibly. But this question should be preceded by: "do you have an OEM recovery hidden partition?"
>
> I mean, if the computer starts up and says "press R to boot into the Compaq recovery partition" and there's no other place for it to be, then yes. But frequently OEM recovery hidden partitions are visible in Windows' disk management console (Start > Run > "diskmgmt.msc") as partitions of DOS or unknown type, or simply showing as "hidden" or with no drive letter assigned. Therefore I'd expect them to be visible in fdisk as well.
>
> Whatever, if you clone the whole disk you will copy the recovery partition, if its present.
>
>> Will I be able to expand NTFS or it is better to make sure parition size = NTFS filesystem size so it doesnt get confused, the boot into windows and expand it?
>
> It probably depends on the version of NTFS whether this can be done natively or not. I believe that in XP you can expand D: and E: partitions from within Windows, but not C:. So to expand an XP system drive you would use Partition Magic or the GParted LiveCD [1]. I guess that Windows 7 allows you to expand C: from the management console, if not GParted claims to support it (and Vista).
>
> I would be trying to do this *after* you have cloned the whole drive and booted with the new copy. Don't try to be clever about whether the filesystem should be the same size as the partition or not - just copy the whole lot verbatim, so they'll remain the same sizes they are now. Then use the GUI tools to expand the partition+filesystem afterwards and let those GUI tools worry about it - preferably use Windows' own tools, otherwise use Partition Magic or GParted.
>
> Partition Magic is my resizing tool of choice for XP, but it's neither free nor Free, nor is it supported on Vista or Windows 7.  GParted is the next choice, then - I understand it to be more than "just a graphical front-end", and I don't think you'll have such good results trying to use command-line tools to expand NTFS partitions.
>
> Stroller.
>
>
> [1] http://gparted.sourceforge.net/livecd.php

Only to add that the new larger drive will appear as small as the
original because the fs size is after all that of the smaller drive.
After your dd the data over to the new disk you will need to run
gparted as suggested by Stroller, or use ntfsresize which is what
gparted uses anyway.
-- 
Regards,
Mick

Re: [gentoo-user] dd says no space left on device

[Stroller]

On 16/5/2011, at 4:35pm, Mick wrote:
>> GParted is the next choice, then - I understand it to be more than "just a graphical front-end", and I don't think you'll have such good results trying to use command-line tools to expand NTFS partitions.
> ...
> After your dd the data over to the new disk you will need to run
> gparted as suggested by Stroller, or use ntfsresize which is what
> gparted uses anyway.

I believe that GParted uses the ntfsresize *libraries* directly, rather than the ntfsresize command-line program.

I believe that's why GParted behaves *better* than ntfsresize - I'm sure there has been at least one occasion on which I found it better to use GParted than ntfsresize (which wouldn't do what I wanted). 

I made the quoted statement for a reason.

Stroller.

Re: [gentoo-user] dd says no space left on device

[Mick]

[-- Attachment #1: Type: Text/Plain, Size: 1509 bytes --]

On Monday 16 May 2011 17:03:36 Stroller wrote:
> On 16/5/2011, at 4:35pm, Mick wrote:
> >> GParted is the next choice, then - I understand it to be more than "just
> >> a graphical front-end", and I don't think you'll have such good results
> >> trying to use command-line tools to expand NTFS partitions.
> > 
> > ...
> > After your dd the data over to the new disk you will need to run
> > gparted as suggested by Stroller, or use ntfsresize which is what
> > gparted uses anyway.
> 
> I believe that GParted uses the ntfsresize *libraries* directly, rather
> than the ntfsresize command-line program.
> 
> I believe that's why GParted behaves *better* than ntfsresize - I'm sure
> there has been at least one occasion on which I found it better to use
> GParted than ntfsresize (which wouldn't do what I wanted).
> 
> I made the quoted statement for a reason.

You could be right, I don't know what gparted runs exactly, but recall from 
the gparted logs that it runs some sort of script where it sequentially runs 
ntfsresize (the command, I suppose) --check, then performs a dry run with the 
--no-action option, then --force to resize the fs and mark it for a 
consistency check with chkdsk when it finally boots into MSWindows, then I 
think it checks it again.  This is all from memory, so it may do other stuff 
too, like check for bad sectors, etc.

Did you check that the problems you experienced were not due to different 
ntfsresize versions?

-- 
Regards,
Mick

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: [gentoo-user] dd says no space left on device

[Neil Bothwick]

[-- Attachment #1: Type: text/plain, Size: 585 bytes --]

On Mon, 16 May 2011 16:35:07 +0100, Mick wrote:

> Only to add that the new larger drive will appear as small as the
> original because the fs size is after all that of the smaller drive.
> After your dd the data over to the new disk you will need to run
> gparted as suggested by Stroller, or use ntfsresize which is what
> gparted uses anyway.

The drive will not appear small, only the partition. The same is true
using the originally suggested method of copying the MBR and partition
separately. 


-- 
Neil Bothwick

WinErr 001: Windows loaded - System in danger

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: [gentoo-user] dd says no space left on device

[Mick]

[-- Attachment #1: Type: Text/Plain, Size: 741 bytes --]

On Monday 16 May 2011 20:54:58 Neil Bothwick wrote:
> On Mon, 16 May 2011 16:35:07 +0100, Mick wrote:
> > Only to add that the new larger drive will appear as small as the
> > original because the fs size is after all that of the smaller drive.
> > After your dd the data over to the new disk you will need to run
> > gparted as suggested by Stroller, or use ntfsresize which is what
> > gparted uses anyway.
> 
> The drive will not appear small, only the partition. The same is true
> using the originally suggested method of copying the MBR and partition
> separately.

Oops!  Sorry, I meant to say partition - just thinking about ntfs and "mapping 
drives" in MSWindows was enough to confuse me ...  :@
-- 
Regards,
Mick

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: [gentoo-user] dd says no space left on device

[Adam Carter]

[-- Attachment #1: Type: text/plain, Size: 1627 bytes --]

On Tue, May 17, 2011 at 12:21 AM, Stroller
<stroller@stellar.eclipse.co.uk>wrote:

>
> On 16/5/2011, at 12:56pm, Adam Carter wrote:
> > ...
> > Yes the new drive is bigger, going from 66G to 500G. Single partition
> only, ...
> >
> > So how do i proceed? Is it;
> > 1. dd the mbr without partition table, to get the boot code (so bs=446
> count=1)
> > 2. use fdisk to set one big primary partition, mark it bootable and NTFS
> (type 7)
> > 3. dd into what will be /dev/sdb1
>
> Just `dd if=/dev/sda of=/dev/sdb` if you can.
>

Done - and its worked.

Here's what i did;
1. Take existing drive out of laptop and connect to Gentoo box using an
esata box, then
sphinx ~ # dd if=/dev/sdb bs=10M conv=notrunc,noerror | gzip > windisk.gz
5723+1 records in
5723+1 records out
60011642880 bytes (60 GB) copied, 5667.78 s, 10.6 MB/s

For interests sake, windows reports that 51gig is in use, which along with
the free space has compressed down to
sphinx ~ # ls -lh win*
-rw-r--r-- 1 root root 37G May 17 12:29 windisk.gz

2. Swap existing disk to new drive, then
sphinx ~ # gunzip -c windisk.gz | dd of=/dev/sdb bs=10M conv=notrunc
0+1819751 records in
0+1819751 records out
60011642880 bytes (60 GB) copied, 940.652 s, 63.8 MB/s

3. Boot into windows. After login it says "You must restart your computer to
apply these changes", so i restart. Then go into  Disk managment and select
"Extend Volume", which immediately makes all the space was immediately
available. Paranoia says run a disk check, which windows offers to schedule
at next reboot. I accept and reboot, check runs and no errors are reported,
so im :)

Thanks again list!

[-- Attachment #2: Type: text/html, Size: 2046 bytes --]