[gentoo-amd64] Cloning a system drive

[gentoo-amd64] Cloning a system drive

[Mark Knecht]

Hi,
   My system drive is making some naughty sounding noises to I'm
thinking I'd better do something fairly quickly. I'm wondering what
the best solution for this problem is?

   I'm really looking for an almost 1 step fix if possible. If I could
get a new drive, put it in the box, and then clone to that drive
directly that would be great.

   The system has both Win XP and Gentoo AMD64. The disk layout is
shown below. I beleive the way I shoehorned XP into this machine was
to steal the original boot partition as a small C: drive and then the
buld of Windows is on a larger partition at the end of the drive.

   The drive is (I think) only about 1/2 partitioned so there is the
possibility of creating a new partition to tar something to and then
transferring that over the network to some other box for safe keeping.

   I've never done this before but would like to do something before I
find myself having to start over from scratch.

   Thanks in advance for your inputs.

Cheers,
Mark

lightning ~ # fdisk -l

Disk /dev/sda: 250.0 GB, 250059350016 bytes
255 heads, 63 sectors/track, 30401 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes

   Device Boot      Start         End      Blocks   Id  System
/dev/sda1   *           1           6       48163+   7  HPFS/NTFS
/dev/sda2             137        1352     9767520   83  Linux
/dev/sda3            1353       30401   233336092+   5  Extended
/dev/sda5            1353        6216    39070048+  83  Linux
/dev/sda6            6217        6703     3911796   83  Linux
/dev/sda7            6704        8163    11727418+  83  Linux
/dev/sda8            8164        9988    14659281   83  Linux
/dev/sda9   *        9989       10001      104391   83  Linux
/dev/sda10          10002       10251     2008093+  82  Linux swap / Solaris
/dev/sda11          10252       14075    30716248+   7  HPFS/NTFS
/dev/sda12          14076       15292     9775521    b  W95 FAT32
lightning ~ #
-- 
gentoo-amd64@gentoo.org mailing list

Re: [gentoo-amd64] Cloning a system drive

[Dieter Ries]

[-- Attachment #1: Type: text/plain, Size: 2922 bytes --]

Put a new harddisk into your system, partition it as you like, and use

tar -cSp --numeric-owner --atime-preserve -f - /mnt/OLD/* | ( cd
mnt/NEW/ && tar -xSpv --atime-preserve -f - )

to copy your linux system/data/whatever. I have done so many times, this
works perfectly, e.g. also to change filesystem.

For the windows thing, dd works quite well, you should create the new
windows partitions about the same size as they are now, or you will
loose space. I am but not sure how your windows reacts, if e.g. the
bigger windows partition is not #11 anymore, afterwards.

Is your C: alone bootable? If yes, I think it should work, but no
guarantees.

hope that helps

cu
Dieter


Mark Knecht schrieb:
> Hi,
>    My system drive is making some naughty sounding noises to I'm
> thinking I'd better do something fairly quickly. I'm wondering what
> the best solution for this problem is?
> 
>    I'm really looking for an almost 1 step fix if possible. If I could
> get a new drive, put it in the box, and then clone to that drive
> directly that would be great.
> 
>    The system has both Win XP and Gentoo AMD64. The disk layout is
> shown below. I beleive the way I shoehorned XP into this machine was
> to steal the original boot partition as a small C: drive and then the
> buld of Windows is on a larger partition at the end of the drive.
> 
>    The drive is (I think) only about 1/2 partitioned so there is the
> possibility of creating a new partition to tar something to and then
> transferring that over the network to some other box for safe keeping.
> 
>    I've never done this before but would like to do something before I
> find myself having to start over from scratch.
> 
>    Thanks in advance for your inputs.
> 
> Cheers,
> Mark
> 
> lightning ~ # fdisk -l
> 
> Disk /dev/sda: 250.0 GB, 250059350016 bytes
> 255 heads, 63 sectors/track, 30401 cylinders
> Units = cylinders of 16065 * 512 = 8225280 bytes
> 
>    Device Boot      Start         End      Blocks   Id  System
> /dev/sda1   *           1           6       48163+   7  HPFS/NTFS
> /dev/sda2             137        1352     9767520   83  Linux
> /dev/sda3            1353       30401   233336092+   5  Extended
> /dev/sda5            1353        6216    39070048+  83  Linux
> /dev/sda6            6217        6703     3911796   83  Linux
> /dev/sda7            6704        8163    11727418+  83  Linux
> /dev/sda8            8164        9988    14659281   83  Linux
> /dev/sda9   *        9989       10001      104391   83  Linux
> /dev/sda10          10002       10251     2008093+  82  Linux swap / Solaris
> /dev/sda11          10252       14075    30716248+   7  HPFS/NTFS
> /dev/sda12          14076       15292     9775521    b  W95 FAT32
> lightning ~ #

-- 
3rd Law of Computing:
        Anything that can go wr
fortune: Segmentation violation -- Core dumped


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 252 bytes --]

Re: [gentoo-amd64] Cloning a system drive

[Drake Donahue]

----- Original Message ----- 
From: "Mark Knecht" <markknecht@gmail.com>
To: <gentoo-amd64@lists.gentoo.org>
Sent: Friday, October 05, 2007 4:58 PM
Subject: [gentoo-amd64] Cloning a system drive


> Hi,
>   My system drive is making some naughty sounding noises to I'm
> thinking I'd better do something fairly quickly. I'm wondering what
> the best solution for this problem is?
>
>   I'm really looking for an almost 1 step fix if possible. If I could
> get a new drive, put it in the box, and then clone to that drive
> directly that would be great.
>
>   The system has both Win XP and Gentoo AMD64. The disk layout is
> shown below. I beleive the way I shoehorned XP into this machine was
> to steal the original boot partition as a small C: drive and then the
> buld of Windows is on a larger partition at the end of the drive.
>
>   The drive is (I think) only about 1/2 partitioned so there is the
> possibility of creating a new partition to tar something to and then
> transferring that over the network to some other box for safe keeping.
>
>   I've never done this before but would like to do something before I
> find myself having to start over from scratch.
>
>   Thanks in advance for your inputs.
>
> Cheers,
> Mark
>

The Gparted live cd now has clonezilla incorporated, the claims for 
clonezilla suggest it would be perfect for your purpose.
http://sourceforge.net/project/showfiles.php?group_id=115843&package_id=173828

link may be fractured in transmission

> lightning ~ # fdisk -l
>
> Disk /dev/sda: 250.0 GB, 250059350016 bytes
> 255 heads, 63 sectors/track, 30401 cylinders
> Units = cylinders of 16065 * 512 = 8225280 bytes
>
>   Device Boot      Start         End      Blocks   Id  System
> /dev/sda1   *           1           6       48163+   7  HPFS/NTFS
> /dev/sda2             137        1352     9767520   83  Linux
> /dev/sda3            1353       30401   233336092+   5  Extended
> /dev/sda5            1353        6216    39070048+  83  Linux
> /dev/sda6            6217        6703     3911796   83  Linux
> /dev/sda7            6704        8163    11727418+  83  Linux
> /dev/sda8            8164        9988    14659281   83  Linux
> /dev/sda9   *        9989       10001      104391   83  Linux
> /dev/sda10          10002       10251     2008093+  82  Linux swap / 
> Solaris
> /dev/sda11          10252       14075    30716248+   7  HPFS/NTFS
> /dev/sda12          14076       15292     9775521    b  W95 FAT32
> lightning ~ #
> -- 
> gentoo-amd64@gentoo.org mailing list
> 

-- 
gentoo-amd64@gentoo.org mailing list

Re: [gentoo-amd64] Cloning a system drive

[Mark Knecht]

On 10/5/07, Drake Donahue <donahue95@comcast.net> wrote:
>
> ----- Original Message -----
> From: "Mark Knecht" <markknecht@gmail.com>
> To: <gentoo-amd64@lists.gentoo.org>
> Sent: Friday, October 05, 2007 4:58 PM
> Subject: [gentoo-amd64] Cloning a system drive
>
>
> > Hi,
> >   My system drive is making some naughty sounding noises to I'm
> > thinking I'd better do something fairly quickly. I'm wondering what
> > the best solution for this problem is?
> >
> >   I'm really looking for an almost 1 step fix if possible. If I could
> > get a new drive, put it in the box, and then clone to that drive
> > directly that would be great.
> >
> >   The system has both Win XP and Gentoo AMD64. The disk layout is
> > shown below. I beleive the way I shoehorned XP into this machine was
> > to steal the original boot partition as a small C: drive and then the
> > buld of Windows is on a larger partition at the end of the drive.
> >
> >   The drive is (I think) only about 1/2 partitioned so there is the
> > possibility of creating a new partition to tar something to and then
> > transferring that over the network to some other box for safe keeping.
> >
> >   I've never done this before but would like to do something before I
> > find myself having to start over from scratch.
> >
> >   Thanks in advance for your inputs.
> >
> > Cheers,
> > Mark
> >
>
> The Gparted live cd now has clonezilla incorporated, the claims for
> clonezilla suggest it would be perfect for your purpose.
> http://sourceforge.net/project/showfiles.php?group_id=115843&package_id=173828
>
> link may be fractured in transmission

Thanks to you and Dieter for responding.

It certainly looks like it's worth burning a CD and seeing what it looks like.

With respect to both of your answers I presume that with any solution,
since this is my main Gentoo system drive, I need to not be running
Gentoo from that drive when I do the clone, correct? Any solution
would require me to boot from some other media?

Also, I really don't need to change any partitions sizes but I'm
unclear whether I am responsible for creating the partitions myself?
It seems with Dieter's solution I have to do that. I'm unsure if they
have to be exactly the same size, in the same location, etc. With the
gparted solution maybe it does some (or all) of this for me?

Anyway, thanks for the link.

Cheers,
Mark
-- 
gentoo-amd64@gentoo.org mailing list

Re: [gentoo-amd64] Cloning a system drive

[Drake Donahue]

----- Original Message ----- 
From: "Mark Knecht" <markknecht@gmail.com>
To: <gentoo-amd64@lists.gentoo.org>
Sent: Friday, October 05, 2007 7:07 PM
Subject: Re: [gentoo-amd64] Cloning a system drive


> On 10/5/07, Drake Donahue <donahue95@comcast.net> wrote:
>>
>> ----- Original Message -----
>> From: "Mark Knecht" <markknecht@gmail.com>
>> To: <gentoo-amd64@lists.gentoo.org>
>> Sent: Friday, October 05, 2007 4:58 PM
>> Subject: [gentoo-amd64] Cloning a system drive
>>
>>
>> > Hi,
>> >   My system drive is making some naughty sounding noises to I'm
>> > thinking I'd better do something fairly quickly. I'm wondering what
>> > the best solution for this problem is?
>> >
>> >   I'm really looking for an almost 1 step fix if possible. If I could
>> > get a new drive, put it in the box, and then clone to that drive
>> > directly that would be great.
>> >
>> >   The system has both Win XP and Gentoo AMD64. The disk layout is
>> > shown below. I beleive the way I shoehorned XP into this machine was
>> > to steal the original boot partition as a small C: drive and then the
>> > buld of Windows is on a larger partition at the end of the drive.
>> >
>> >   The drive is (I think) only about 1/2 partitioned so there is the
>> > possibility of creating a new partition to tar something to and then
>> > transferring that over the network to some other box for safe keeping.
>> >
>> >   I've never done this before but would like to do something before I
>> > find myself having to start over from scratch.
>> >
>> >   Thanks in advance for your inputs.
>> >
>> > Cheers,
>> > Mark
>> >
>>
>> The Gparted live cd now has clonezilla incorporated, the claims for
>> clonezilla suggest it would be perfect for your purpose.
>> http://sourceforge.net/project/showfiles.php?group_id=115843&package_id=173828
>>
>> link may be fractured in transmission
>
> Thanks to you and Dieter for responding.
>
> It certainly looks like it's worth burning a CD and seeing what it looks 
> like.
>
> With respect to both of your answers I presume that with any solution,
> since this is my main Gentoo system drive, I need to not be running
> Gentoo from that drive when I do the clone, correct? Any solution
> would require me to boot from some other media?
>
> Also, I really don't need to change any partitions sizes but I'm
> unclear whether I am responsible for creating the partitions myself?
> It seems with Dieter's solution I have to do that. I'm unsure if they
> have to be exactly the same size, in the same location, etc. With the
> gparted solution maybe it does some (or all) of this for me?
>
> Anyway, thanks for the link.
>

Clonezilla should (if it meets its claims) simply clone one drive from 
another, duplicating partitions and file systems as maxblast and wdtools 
used to do. It even claims the ability to clone from larger to smaller drive 
if there is empty space  in the original.
Clonezilla and gparted do run from the livecd and ramdisk.

> Cheers,
> Mark
> -- 
> gentoo-amd64@gentoo.org mailing list
> 

-- 
gentoo-amd64@gentoo.org mailing list

Re: [gentoo-amd64] Cloning a system drive

[Dieter Ries]

[-- Attachment #1: Type: text/plain, Size: 1810 bytes --]

Mark Knecht schrieb:
> On 10/5/07, Drake Donahue <donahue95@comcast.net> wrote:
>> The Gparted live cd now has clonezilla incorporated, the claims for
>> clonezilla suggest it would be perfect for your purpose.
>> http://sourceforge.net/project/showfiles.php?group_id=115843&package_id=173828
>>
>> link may be fractured in transmission
> 
> Thanks to you and Dieter for responding.
> 
> It certainly looks like it's worth burning a CD and seeing what it looks like.
> 
> With respect to both of your answers I presume that with any solution,
> since this is my main Gentoo system drive, I need to not be running
> Gentoo from that drive when I do the clone, correct? Any solution
> would require me to boot from some other media?

Definitely. After reading what Drake wrote, I guess the GParted CD
should be your first choice, I remember I saw an earlier version of that
CD doing a great job in guessing a lost partition table.

The command I wrote initially is more or less a, slightly more CPU
intensive, copy command, which takes care of all the permissions etc
stuff, without making you any trouble. It has to be issued from a live
CD, in the example with your old HD mounted to /mnt/OLD and your new one
to /mnt/NEW.


> Also, I really don't need to change any partitions sizes but I'm
> unclear whether I am responsible for creating the partitions myself?
> It seems with Dieter's solution I have to do that. I'm unsure if they
> have to be exactly the same size, in the same location, etc. With the
> gparted solution maybe it does some (or all) of this for me?

Yes, it should.

> 
> Anyway, thanks for the link.
> 
> Cheers,
> Mark

cu
Dieter

-- 
3rd Law of Computing:
        Anything that can go wr
fortune: Segmentation violation -- Core dumped


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 252 bytes --]

[gentoo-amd64] Re: Cloning a system drive

[Duncan]

"Mark Knecht" <markknecht@gmail.com> posted
5bdc1c8b0710051358g74f95702rb82a29219d890919@mail.gmail.com, excerpted
below, on  Fri, 05 Oct 2007 13:58:18 -0700:

>    My system drive is making some naughty sounding noises to I'm
> thinking I'd better do something fairly quickly. I'm wondering what the
> best solution for this problem is?
> 
>    I'm really looking for an almost 1 step fix if possible. If I could
> get a new drive, put it in the box, and then clone to that drive
> directly that would be great.
> 
>    The system has both Win XP and Gentoo AMD64. The disk layout is
> shown below. I beleive the way I shoehorned XP into this machine was to
> steal the original boot partition as a small C: drive and then the buld
> of Windows is on a larger partition at the end of the drive.

I've migrated hard drives under old-one-dieing conditions a couple times 
in the last few years.  Fortunately, the old one has always been still 
usable, but that's why I went with RAID this time.

What I do is take the opportunity to redesign my partition layouts and 
the like (altho this time I have most stuff on LVM, which should help 
next time).  The disk is always larger, and sometimes I've switched 
distributions or something, so need to optimize my layout.  Thus, the 
first step is simply deciding how I want my partitions split up and what 
size each will be, then choosing the order I want to lay them out.  (This 
time, the first with RAID and LVM, I had that to think about too, 
partitions on the physical disks for the various RAID levels I wanted to 
run, then the RAID, choosing partitioned or not as appropriate, then the 
filesystem directly on RAID for the boot, system, and sysbackup volumes, 
LVM on RAID, and filesystems on LVM, for user and operational data.)

After setting up the partitions, the hard part, it's simply creating the 
filesystems and copying stuff over after that.  As for many of my 
sysadmin type tasks, I use mc, aka midnight commander, for the copying.  
Its defaults preserve ownership/permissions/etc and "just work" when 
exotic stuff like symlinks, sockets and device nodes are encountered, so 
no worrying about getting everything right.  The exception is stopping at 
the filesystem boundary if appropriate, but a well chosen mount layout 
minimizing recursive mounts on mounts minimizes that issue, and for 
filesystems such as root, there's mount-bind.  (I actually have a 
rootbind mountpoint and an entry in fstab for it, so all I have to do is 
mount that entry, and copy from it instead of from / itself, to eliminate 
worries of copying more than the root filesystem.)

IOW, the hard part is simply deciding on the new layout I want and doing 
the partitioning (and raid creation) as appropriate.  After that, as with 
the backup discussion recently, I simply copy the data from one place to 
the other, as I would do any other routine copying, so after the 
partitioning and etc. setup, it's pretty much just a routine backup copy 
operation from my perspective.  If there are locking issues, I've never 
seen them, so it seems they only apply to databases and the like.  Of 
course, I'm not trying to copy partitions whilst simultaneously copying 
CD/DVD sized images around on the partition I'm copying, either, but that 
would just be stupid.

I've actually done this for years, so it works.  As I mentioned in 
passing above and have noted before, I had two drives go out at almost 
exactly the 1-year mark (the last one due to overheating due to a failed 
AC), the reason I went with RAID this time.  The RAID has been up and 
running almost two years now, however, and I recently decided to update 
my backups then boot to them and delete and recreate my normally working 
copies, thus effectively "defragging", as I imagine the constant 
upgrading especially the system/root filesystem had well fragmented 
things and I had been routinely replacing the backup images but working 
from the same working copy main image, without defragging or anything.  I 
think I noticed a good increase in system responsiveness, but I didn't do 
any benchmarks or whatever to test, so it could be placebo effect.  Of 
course, the transfer to backup and then rewriting my main/working copies 
went without a hitch as it has always gone, but the point is that both 
the backup and the working copies have been redone "fresh" fairly 
recently and I'm up and running on them without issue, so the procedure 
simply works, and works well.

BTW, for trying to recover data on partially bad drives as I did twice in 
the last five years, dd-rescue works well.  It does direct block by block 
copies just as regular dd does, but it's optimized for bad disk recovery, 
such that when it hits several unreadable blocks in a row, it starts from 
the other end of the image and works backward.  When it hits several in a 
row from that end, it tries to find good spots in the middle between the 
two bad spots, and expands the recovered blocks from there if it finds 
any it can read.  Thus, since the bad-block retries are what takes 
forever in recovery situations, dd-rescue recovers much more data far 
faster than normal dd would, since dd would hit the bad spot and continue 
to try reading block after block in order, with the user likely giving up 
after a few hours or a day or two, possibly before dd reads thru the bad 
spot to the still good data on the other side.  Unlike dd, once dd-rescue 
has been running on an image for a number of hours with only incremental 
progress, the user can be reasonably sure it has recovered pretty much 
all the data that's going to be recovered, so can cancel the operation 
without much fear of leaving still recoverable data on the disk.

-- 
Duncan - List replies preferred.   No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master."  Richard Stallman

-- 
gentoo-amd64@gentoo.org mailing list

Re: [gentoo-amd64] Re: Cloning a system drive

[Richard Freeman]

[-- Attachment #1: Type: text/plain, Size: 1157 bytes --]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Duncan wrote:
> 
> What I do is take the opportunity to redesign my partition layouts and 
> the like (altho this time I have most stuff on LVM, which should help 
> next time).  

This is good advice in general.  If you have never used LVM look into
it.  Once your partitions are on LVM it is VERY easy to migrate them to
a new drive.  You can do it while the system is running as a matter of fact.

Ditto for raid.  I don't know that it is really essential in all cases,
but I've got a mythtv setup with the better part of 1TB of storage, so I
migrated it to software raid.  It isn't worth the money to come up with
an offsite backup solution for TV shows, but I'd rather not lose
everything if one of my multitude of hard drives fails either.  So I use
RAID5 to cover drive failures, and I do offsite backups of selected
files of higher value (a few GB).
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.7 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFHBvRhG4/rWKZmVWkRAlJlAKCHxu8LtaZW4GsjL2O0uJFCh6DwhwCgp4Ad
icifZzcEUWso/tL+QpyjUUE=
=1sBj
-----END PGP SIGNATURE-----

[-- Attachment #2: S/MIME Cryptographic Signature --]
[-- Type: application/x-pkcs7-signature, Size: 4101 bytes --]

Re: [gentoo-amd64] Re: Cloning a system drive

[Mark Knecht]

On 10/5/07, Richard Freeman <rich@thefreemanclan.net> wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> Duncan wrote:
> >
> > What I do is take the opportunity to redesign my partition layouts and
> > the like (altho this time I have most stuff on LVM, which should help
> > next time).
>
> This is good advice in general.  If you have never used LVM look into
> it.  Once your partitions are on LVM it is VERY easy to migrate them to
> a new drive.  You can do it while the system is running as a matter of fact.
>
> Ditto for raid.  I don't know that it is really essential in all cases,
> but I've got a mythtv setup with the better part of 1TB of storage, so I
> migrated it to software raid.  It isn't worth the money to come up with
> an offsite backup solution for TV shows, but I'd rather not lose
> everything if one of my multitude of hard drives fails either.  So I use
> RAID5 to cover drive failures, and I do offsite backups of selected
> files of higher value (a few GB).

In my case I'm very noise sensitive. I do a lot of audio work and
don't want additional hard drive noise in here. In my mind that rules
out multi-drive RAID and I guess I don't see how any form of
single-drive RAID helps if the issue is the drive doing bad.

Probably I'm not aware of all the value of doing all of that work.
Maybe there would be significant reliability gains but it's a subject
that is pretty far beyond me today.

I'm also has concerns that 1)  the drive could go sooner than later
causing me to have more work getting the system set up again and 2) if
I do add some form of RAID that it will cause problems for the cloned
Win XP installation being it isn't there now.

And how does LVM work for Windows anyway? I thought that was a Linux thing?

Maybe everything Duncan said is right, and I'll give it some thought,
but my #1 worry is trying to make sure the system doesn't go down hard
and leave me with a week's worth of work getting Humpty Dumpty back
together again that I don't need right now.

Thanks all,
Mark
-- 
gentoo-amd64@gentoo.org mailing list

[gentoo-amd64] Re: Cloning a system drive

[Duncan]

"Mark Knecht" <markknecht@gmail.com> posted
5bdc1c8b0710051943w4cd14241v795c599285891959@mail.gmail.com, excerpted
below, on  Fri, 05 Oct 2007 19:43:45 -0700:

> In my case I'm very noise sensitive. I do a lot of audio work and don't
> want additional hard drive noise in here. In my mind that rules out
> multi-drive RAID and I guess I don't see how any form of single-drive
> RAID helps if the issue is the drive doing bad.

Noise sensitive... go with lower RPM drives.  Some Seagate 4200 or 5400 
RPM drives may be good, and Seagate still has a 5 year warrantee while 
many of the others are now one year.

Many drives are also adjustable for quietness vs. performance, if you 
have the right hdparm or smartctl type utils to set them.  I've not 
bothered and it has been awhile since I read up on the details, so I'm a 
bit fuzzy on them now, but many, particularly here in the US, would come 
pre-set to the performance side.  (In parts of the EU at least, I 
understand there's some pretty strict regulations on workplace 
environment including noise, and it's likely that some drives set for 
performance in the US market come preset for toward quietness in that 
market.)

Note that it's also possible to setup the drives in a separate external 
enclosure and/or to house either that or the entire computer in another 
room and run video and control cables.  SATA/eSATA Five-drive external 
enclosures, complete with their own power supply and fans, with hotplug 
backplane (change out a bad drive while the system is in use), run $300 
or so.  Add another $100 and throw in a 5:1 SATA port multiplier so you 
only run a single SATA/eSATA cable back to the computer.  The computer 
itself can then be much smaller/lighter/quieter, since it doesn't contain 
the drives nor does it need to power them from its own power supply.

Another option: For RAID-1/mirroring, the ultimate in reliability since 
the data is copied N-way, Linux makes it possible to declare some of the 
devices write-mostly, and make them remote.  The Linux RAID driver will 
write to write-mostly devices as normal, mirroring the data to all 
devices as usual, but will only read from the write-mostly devices if the 
normal (non-write-mostly) devices in the array die or return errors.  
This DRAMATICALLY increases mirroring flexibility, as provided you can 
keep open a connection with enough bandwidth to maintain expected write-
rates, you can locate the write-mostly devices basically anywhere in the 
world -- connected over the Internet or whatever if necessary.  Want your 
data automatically mirrored to your co-hosted server three states over, 
to another in Australia, and another in Europe, thus ensuring the 
ultimate in disaster recoverability?  Not a problem!  Connect them up 
over the Internet and create write-mostly devices for them, and your data 
will be auto-mirrored N-ways including to all those off-site backup 
locations!

Of course, the same technology can be used to host a much-more-local 
mirror as well, at the other end of the house or at your next-door-
neighbor's over Ethernet, or to your server located at work/home from the 
other, over Internet.  You get full data redundancy without requiring but 
your single normal drive in your on-location computer.

If you combine write-mostly with a local but external SATA/eSATA solution 
hosted in another room, even with heavy data needs, you can be as fully 
redundant as you wish, with no drives at all in the computer itself, thus 
lowering the weight/noise/power-requirements of the computer rather 
drastically.

> Probably I'm not aware of all the value of doing all of that work. Maybe
> there would be significant reliability gains but it's a subject that is
> pretty far beyond me today.

That was my reaction too, plus the expense, until I had two drives in a 
row go out in a year each, while my normal replace cycle would be more 
like three years.  That plus the now relatively low entry cost and 
complexity, simple SATA drives and the kernel's own software RAID driver, 
made actually possible what I hadn't even considered within my reach, 
until that second drive overheated and I began doing research on a 
suitable replacement with a bit more reliability. (The overheat killed 
parts of the drive, basically the parts that it tried to read while 
overheated, but I could still use the rest of it, including the second 
copy of my data on the same drive, and was still operational on that, so 
I had some time to research, but didn't want to press my luck...)

Basically, minimum cost now is the cost of the additional drive(s), as 
long as your computer has spare SATA ports, drive bay space, and a power 
supply sufficient to power the additional drives.  Of course, for your 
low noise needs, you may pay a bit more, but the cost is still only a few 
hundred dollars total, reasonably comparable to that of getting a second 
computer, not the thousands of dollars for Enterprise equipment one used 
to think of in the context of RAID.

> I'm also has concerns that 1)  the drive could go sooner than later
> causing me to have more work getting the system set up again

If your entire system is RAID-1/mirrored or RAID-10, and to a somewhat 
lessor extent if it's RAID-1/5 or 1/6 mixed (as mine is, basically, 
RAID-0 for the temp-only stuff, but that's not redundant and I know I'd 
lose it if I lost a drive), particularly if you are running one of the 
newer hot-plug SATA things, either internal or external, not only need 
your system never even go down for a drive outage as you can replace it 
with the system running, but getting out of degraded mode into full 
redundancy protection once again can literally be as simple as turning 
the lock and pushing the button to release the old drive in its tray, 
taking it out of the tray and putting the new drive in, reloading and 
relocking the tray complete with new drive in place, and running a couple 
mdadm commands to tell the RAID driver to add it as a spare and 
reinitialize.  Then it's simply a matter of waiting the few hours of 
still operational but degraded performance until the new drive is brought 
online fully up to speed.

Restating in briefer form, simply push a button, slide out the old drive 
and replace it with the new, tell mdadm it's now part of the RAID, and 
wait thru a few hours of lower than normal performance until it's brought 
online and the system is fully recovered.

No need to even reboot; it's all accomplished "live", with a couple of 
button pushes and issuing a couple of commands, plus a few hours of 
somewhat lower than normal performance as the new drive is brought up to 
speed.  That's it!  More work?  No, LESS work, by FAR!  

While that is the simplest case, full RAID-1 mirroring, if you skimp and 
go RAID-5 or RAID-6 due to cost, mixed with a RAID-1 for /boot and 
perhaps a RAID-0/striped for speed for your temp stuff, you do add a few 
more commands, mainly fdisk to set up the partitions for the mixed RAID 
before you add the new disk to the RAID, you lose the temp stuff on the 
RAID-0 and have to rebuild that, and you need to reboot after the fdisk, 
but the copying over of your data remains fully automatic, and it's still 
far less work than recovery from a single main disk going bad could every 
be.

> and 2) if I do add some form of RAID that it will cause problems for
> the cloned Win XP installation being it isn't there now.

Well, if you use Linux-kernel RAID, of course you do lose the ability to 
read it from MS, and even using hardware RAID, there's the eXPrivacy anti-
privacy activation hoops you have to go thru... You certainly do have a 
point there!  

Of course, that's one of the big reasons I left MS and switched to 
Linux... that was a line I simply could not and would not cross. I simply 
refused, and when MS insisted, as with any relationship where one partner 
makes demands for something the other simply cannot and will not do, it 
ends the relationship.  Needless to say given no other choice, we parted 
ways.  So I have MS to thank for finally pushing me to Linux.  Oh well, 
their loss as I spent a decent amount of money on them, my ever so great 
gain! =8^)

> And how does LVM work for Windows anyway? I thought that was a Linux
> thing?

It is... unless you are running one in a VM on the other, of course.  
Then the host system supplies the devices and drivers. =8^)

> Maybe everything Duncan said is right, and I'll give it some thought,
> but my #1 worry is trying to make sure the system doesn't go down hard
> and leave me with a week's worth of work getting Humpty Dumpty back
> together again that I don't need right now.

Of course, you know I'd be dumping MS, but it's your system and your 
choices to make.  In any case, I'd certainly consider buying a separate 
system to run MS on (or run it in a VM... you may actually be surprised 
to find it runs faster in a VM than it does on the bare metal -- provided 
you have sufficient memory, of course).  Either your MS or Linux side is 
likely sufficiently low demanding to run on what is now days a fairly 
cheap system, even new, so the option shouldn't be more than a few 
hundred dollars.

Actually, I intended for that to mainly convey the message that my 
relatively basic new-drive transfer method, three steps partition/mkfs/
conventional-copy, has served me very well for many years and over a wide 
variety of implementation details, including my new mixed RAID/LVM 
setup.  It was therefore intended to be less about the RAID/LVM, which 
was supposed to be simply an incidental mention, and more about the 
simple partition/mkfs/copy transfer method, but that's not how it turned 
out, obviously. =8^?

In any case, if you are keeping eXPrivacy on there and want to continue 
to run it on the bare metal and not in a VM, and if you need most of your 
Linux side to be eXPrivacy accessible, that's going to limit your options 
somewhat and the LVM/RAID stuff definitely won't be as practical as it 
could arguably be if you were Linux-only.  I still think the basic 
partition/mkfs/conventional-copy method is simpler than most of the 
others, tho, particularly since you don't have to worry about additional 
software, and the repartitioning will allow you to better adapt to the 
larger drive than straight imaging, or even using dd and then growing the 
partitions.

-- 
Duncan - List replies preferred.   No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master."  Richard Stallman

-- 
gentoo-amd64@gentoo.org mailing list

Re: [gentoo-amd64] Re: Cloning a system drive

[Mark Knecht]

On 10/6/07, Duncan <1i5t5.duncan@cox.net> wrote:
> "Mark Knecht" <markknecht@gmail.com> posted
<SNIP>

> > and 2) if I do add some form of RAID that it will cause problems for
> > the cloned Win XP installation being it isn't there now.
>
> Well, if you use Linux-kernel RAID, of course you do lose the ability to
> read it from MS, and even using hardware RAID, there's the eXPrivacy anti-
> privacy activation hoops you have to go thru... You certainly do have a
> point there!
>
> Of course, that's one of the big reasons I left MS and switched to
> Linux... that was a line I simply could not and would not cross. I simply
> refused, and when MS insisted, as with any relationship where one partner
> makes demands for something the other simply cannot and will not do, it
> ends the relationship.  Needless to say given no other choice, we parted
> ways.  So I have MS to thank for finally pushing me to Linux.  Oh well,
> their loss as I spent a decent amount of money on them, my ever so great
> gain! =8^)

I did too Duncan. I ran only Linux for about 5 years. About 3 years
ago I left my Silicon Valley career behind and about a year ago I
started trading stocks full time. Unfortunately (until very recently)
there hasn't a viable stock trading platform for Linux. Now, since
thinkorswim's platform is written in Java, there is at least the
possibility of trading from a Gentoo machine. I cannot get it to run
on Gentoo 64-bit but when I do Windows will be gone again.

- Mark
-- 
gentoo-amd64@gentoo.org mailing list

Re: [gentoo-amd64] Re: Cloning a system drive

[Drake Donahue]

----- Original Message ----- 
From: "Mark Knecht" <markknecht@gmail.com>
To: <gentoo-amd64@lists.gentoo.org>
Sent: Saturday, October 06, 2007 9:16 AM
Subject: Re: [gentoo-amd64] Re: Cloning a system drive

weekend update:

Gparted-clonezilla link that works (hard to find now):
http://linux.softpedia.com/progDownload/GParted-Clonezilla-Download-25168.html

Apparently the gparted and clonezilla folk decided to split and the combined 
livecd is being maintained by one gparted guy with a french homesite and a 
chinese mirror. Neither providing downloads just now.

Gparted iso:

http://sourceforge.net/project/showfiles.php?group_id=115843&package_id=173828

Clonezilla iso:

http://clonezilla.sourceforge.net/download/sourceforge/

Sorry about the bad info. Too bad the two groups did not continue to link 
their efforts. 

-- 
gentoo-amd64@gentoo.org mailing list

Re: [gentoo-amd64] Re: Cloning a system drive

[Peter Davoust]

This may be a little noobish, and it may have been said, but can't you
just install the new drive, partition it identically to the original
drive and then...

dd if=/dev/hda1 of=/dev/hdb1
dd if=/dev/hda2 of=/dev/hdb2

so on and so forth until you've got everything copied? Or event just

dd if=/dev/hda of=/dev/hdb

Wouldn't that work?

-Peter

On 10/6/07, Drake Donahue <donahue95@comcast.net> wrote:
> ----- Original Message -----
> From: "Mark Knecht" <markknecht@gmail.com>
> To: <gentoo-amd64@lists.gentoo.org>
> Sent: Saturday, October 06, 2007 9:16 AM
> Subject: Re: [gentoo-amd64] Re: Cloning a system drive
>
> weekend update:
>
> Gparted-clonezilla link that works (hard to find now):
> http://linux.softpedia.com/progDownload/GParted-Clonezilla-Download-25168.html
>
> Apparently the gparted and clonezilla folk decided to split and the combined
> livecd is being maintained by one gparted guy with a french homesite and a
> chinese mirror. Neither providing downloads just now.
>
> Gparted iso:
>
> http://sourceforge.net/project/showfiles.php?group_id=115843&package_id=173828
>
> Clonezilla iso:
>
> http://clonezilla.sourceforge.net/download/sourceforge/
>
> Sorry about the bad info. Too bad the two groups did not continue to link
> their efforts.
>
> --
> gentoo-amd64@gentoo.org mailing list
>
>
-- 
gentoo-amd64@gentoo.org mailing list

[gentoo-amd64] Re: Cloning a system drive

[Duncan]

"Peter Davoust" <worldgnat@gmail.com> posted
7c08b4dd0710062149p7c7548aai7ba062ea4e17d2a3@mail.gmail.com, excerpted
below, on  Sun, 07 Oct 2007 00:49:11 -0400:

> This may be a little noobish, and it may have been said, but can't you
> just install the new drive, partition it identically to the original
> drive and then...
> 
> dd if=/dev/hda1 of=/dev/hdb1
> dd if=/dev/hda2 of=/dev/hdb2
> 
> so on and so forth until you've got everything copied? Or event just
> 
> dd if=/dev/hda of=/dev/hdb
> 
> Wouldn't that work?

It should, you're right.

However, that's a direct image straight across, fragmented files, even 
filesystem corruption if it exists on the source, and it's nice to take 
the opportunity to copy file-by file so everything gets defragmented and 
all the file nodes get organized, if possible, particularly since few 
Linux filesystems /have/ a defrag.  Few need it very badly as long as a 
decent amount of free-space is kept on each filesystem (50% is great, 25% 
minimum for best operation, 10% and performance /does/ start to suffer), 
but it's still nice to organize it, getting files all in one piece and 
directories all located together, while one can.

Also, new drives are generally larger and it's nice to be able to take 
the opportunity to reorganize the partitions.

All those are reasons I like my simple partition/mkfs/copy-the-files-over 
method, but some folks don't seem to like that idea for whatever reason.  
<shrug>

-- 
Duncan - List replies preferred.   No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master."  Richard Stallman

-- 
gentoo-amd64@gentoo.org mailing list

Re: [gentoo-amd64] Re: Cloning a system drive

[Brian Litzinger]

On Sun, Oct 07, 2007 at 12:49:11AM -0400, Peter Davoust wrote:
> This may be a little noobish, and it may have been said, but can't you
> just install the new drive, partition it identically to the original
> drive and then...
> 
> dd if=/dev/hda1 of=/dev/hdb1
> dd if=/dev/hda2 of=/dev/hdb2
> 
> so on and so forth until you've got everything copied? Or event just
> 
> dd if=/dev/hda of=/dev/hdb
> 
> Wouldn't that work?

The latter works fine in my experience.  I do it regularly.

The downside, is that cloning a 750GB drive takes a while
as it duplicates everything including unused sectors.

Things like clonezilla just copy the "used"/active sectors.

A popular way is to use sfdisk.  I do not remember the exact
syntax, but a pair of sfdisk commands can transfer the partition
information directly between two drives.

Then use rsync to move the data across.

You may have to run grub setup on the new disk too.

-- 
Brian Litzinger
-- 
gentoo-amd64@gentoo.org mailing list

Re: [gentoo-amd64] Re: Cloning a system drive

[Beso]

[-- Attachment #1: Type: text/plain, Size: 1162 bytes --]

does this work from hd to external usb disk?

2007/10/7, Brian Litzinger <brian@worldcontrol.com>:
>
> On Sun, Oct 07, 2007 at 12:49:11AM -0400, Peter Davoust wrote:
> > This may be a little noobish, and it may have been said, but can't you
> > just install the new drive, partition it identically to the original
> > drive and then...
> >
> > dd if=/dev/hda1 of=/dev/hdb1
> > dd if=/dev/hda2 of=/dev/hdb2
> >
> > so on and so forth until you've got everything copied? Or event just
> >
> > dd if=/dev/hda of=/dev/hdb
> >
> > Wouldn't that work?
>
> The latter works fine in my experience.  I do it regularly.
>
> The downside, is that cloning a 750GB drive takes a while
> as it duplicates everything including unused sectors.
>
> Things like clonezilla just copy the "used"/active sectors.
>
> A popular way is to use sfdisk.  I do not remember the exact
> syntax, but a pair of sfdisk commands can transfer the partition
> information directly between two drives.
>
> Then use rsync to move the data across.
>
> You may have to run grub setup on the new disk too.
>
> --
> Brian Litzinger
> --
> gentoo-amd64@gentoo.org mailing list
>
>


-- 
dott. ing. beso

[-- Attachment #2: Type: text/html, Size: 1595 bytes --]

Re: [gentoo-amd64] Re: Cloning a system drive

[Brian Litzinger]

On Mon, Oct 08, 2007 at 11:17:40AM +0200, Beso wrote:
> does this work from hd to external usb disk?

dd will not work between disparate media.  It is even
risky between different (capacity, manufacturer) drives.

If by "this" you mean the latter stategy involving
sfdisk/rsync/grub the sfdisk step will mostly not work
between disparate media.

> 2007/10/7, Brian Litzinger <brian@worldcontrol.com>:
> >
> > On Sun, Oct 07, 2007 at 12:49:11AM -0400, Peter Davoust wrote:
> > > This may be a little noobish, and it may have been said, but can't you
> > > just install the new drive, partition it identically to the original
> > > drive and then...
> > >
> > > dd if=/dev/hda1 of=/dev/hdb1
> > > dd if=/dev/hda2 of=/dev/hdb2
> > >
> > > so on and so forth until you've got everything copied? Or event just
> > >
> > > dd if=/dev/hda of=/dev/hdb
> > >
> > > Wouldn't that work?
> >
> > The latter works fine in my experience.  I do it regularly.
> >
> > The downside, is that cloning a 750GB drive takes a while
> > as it duplicates everything including unused sectors.
> >
> > Things like clonezilla just copy the "used"/active sectors.
> >
> > A popular way is to use sfdisk.  I do not remember the exact
> > syntax, but a pair of sfdisk commands can transfer the partition
> > information directly between two drives.
> >
> > Then use rsync to move the data across.
> >
> > You may have to run grub setup on the new disk too.
> >
> > --
> > Brian Litzinger
-- 
gentoo-amd64@gentoo.org mailing list

Re: [gentoo-amd64] Re: Cloning a system drive

[Mark Knecht]

On 10/8/07, Brian Litzinger <brian@worldcontrol.com> wrote:
> On Mon, Oct 08, 2007 at 11:17:40AM +0200, Beso wrote:
> > does this work from hd to external usb disk?
>
> dd will not work between disparate media.  It is even
> risky between different (capacity, manufacturer) drives.

For my purpose, and I think most anyone in my situation, this is a key
issue. I built this AMD64 machine 2-3 years ago. Any drive I put in
today is going to have completely different drive geometries.

I am buying a drive today and will hopefully get started on this
project this evening to tomorrow. I'm leaning the gparted-clonezilla
direction but not overly confident at this point. Still have much to
learn.

Thanks,
Mark

>
> If by "this" you mean the latter stategy involving
> sfdisk/rsync/grub the sfdisk step will mostly not work
> between disparate media.
>
> > 2007/10/7, Brian Litzinger <brian@worldcontrol.com>:
> > >
> > > On Sun, Oct 07, 2007 at 12:49:11AM -0400, Peter Davoust wrote:
> > > > This may be a little noobish, and it may have been said, but can't you
> > > > just install the new drive, partition it identically to the original
> > > > drive and then...
> > > >
> > > > dd if=/dev/hda1 of=/dev/hdb1
> > > > dd if=/dev/hda2 of=/dev/hdb2
> > > >
> > > > so on and so forth until you've got everything copied? Or event just
> > > >
> > > > dd if=/dev/hda of=/dev/hdb
> > > >
> > > > Wouldn't that work?
> > >
> > > The latter works fine in my experience.  I do it regularly.
> > >
> > > The downside, is that cloning a 750GB drive takes a while
> > > as it duplicates everything including unused sectors.
> > >
> > > Things like clonezilla just copy the "used"/active sectors.
> > >
> > > A popular way is to use sfdisk.  I do not remember the exact
> > > syntax, but a pair of sfdisk commands can transfer the partition
> > > information directly between two drives.
> > >
> > > Then use rsync to move the data across.
> > >
> > > You may have to run grub setup on the new disk too.
> > >
> > > --
> > > Brian Litzinger
> --
> gentoo-amd64@gentoo.org mailing list
>
>
-- 
gentoo-amd64@gentoo.org mailing list

Re: [gentoo-amd64] Re: Cloning a system drive

[Beso]

[-- Attachment #1: Type: text/plain, Size: 1922 bytes --]

so for backuping a gentoo installation on usb disk is still better to build
a stage4 with the script. i need to make a backup working copy of my gentoo
notebook box and i have only one disk drive. that is the real problem with
using dd or clonezilla...

2007/10/8, Brian Litzinger <brian@worldcontrol.com>:
>
> On Mon, Oct 08, 2007 at 11:17:40AM +0200, Beso wrote:
> > does this work from hd to external usb disk?
>
> dd will not work between disparate media.  It is even
> risky between different (capacity, manufacturer) drives.
>
> If by "this" you mean the latter stategy involving
> sfdisk/rsync/grub the sfdisk step will mostly not work
> between disparate media.
>
> > 2007/10/7, Brian Litzinger <brian@worldcontrol.com>:
> > >
> > > On Sun, Oct 07, 2007 at 12:49:11AM -0400, Peter Davoust wrote:
> > > > This may be a little noobish, and it may have been said, but can't
> you
> > > > just install the new drive, partition it identically to the original
> > > > drive and then...
> > > >
> > > > dd if=/dev/hda1 of=/dev/hdb1
> > > > dd if=/dev/hda2 of=/dev/hdb2
> > > >
> > > > so on and so forth until you've got everything copied? Or event just
> > > >
> > > > dd if=/dev/hda of=/dev/hdb
> > > >
> > > > Wouldn't that work?
> > >
> > > The latter works fine in my experience.  I do it regularly.
> > >
> > > The downside, is that cloning a 750GB drive takes a while
> > > as it duplicates everything including unused sectors.
> > >
> > > Things like clonezilla just copy the "used"/active sectors.
> > >
> > > A popular way is to use sfdisk.  I do not remember the exact
> > > syntax, but a pair of sfdisk commands can transfer the partition
> > > information directly between two drives.
> > >
> > > Then use rsync to move the data across.
> > >
> > > You may have to run grub setup on the new disk too.
> > >
> > > --
> > > Brian Litzinger
> --
> gentoo-amd64@gentoo.org mailing list
>
>


-- 
dott. ing. beso

[-- Attachment #2: Type: text/html, Size: 2633 bytes --]

Re: [gentoo-amd64] Re: Cloning a system drive

[Peter Davoust]

It may have already been mentioned, but I read about a program called
PartImage (www.partimage.org). It looks like more of an automatic
backup program, but it does all of the backing up and restoring for
you, I think, and I'm pretty sure you can do manual backups. I've
never personally used it though.

Duncan, I agree, copying the files makes sense, because if you make an
image or tar the files, you're copying the files anyway, but it takes
time to tar the files or make an image. I'm not sure what the time
gained for tar-ing the files is vs. just straight copying them, but
I'd imagine it's a pretty small if there is one. Then again, I'm not
familiar with how tar works, so I could easily be wrong.

-Peter

On 10/8/07, Beso <givemesugarr@gmail.com> wrote:
> so for backuping a gentoo installation on usb disk is still better to build
> a stage4 with the script. i need to make a backup working copy of my gentoo
> notebook box and i have only one disk drive. that is the real problem with
> using dd or clonezilla...
>
> 2007/10/8, Brian Litzinger <brian@worldcontrol.com>:
> > On Mon, Oct 08, 2007 at 11:17:40AM +0200, Beso wrote:
> > > does this work from hd to external usb disk?
> >
> > dd will not work between disparate media.  It is even
> > risky between different (capacity, manufacturer) drives.
> >
> > If by "this" you mean the latter stategy involving
> > sfdisk/rsync/grub the sfdisk step will mostly not work
> > between disparate media.
> >
> > > 2007/10/7, Brian Litzinger < brian@worldcontrol.com>:
> > > >
> > > > On Sun, Oct 07, 2007 at 12:49:11AM -0400, Peter Davoust wrote:
> > > > > This may be a little noobish, and it may have been said, but can't
> you
> > > > > just install the new drive, partition it identically to the original
> > > > > drive and then...
> > > > >
> > > > > dd if=/dev/hda1 of=/dev/hdb1
> > > > > dd if=/dev/hda2 of=/dev/hdb2
> > > > >
> > > > > so on and so forth until you've got everything copied? Or event just
> > > > >
> > > > > dd if=/dev/hda of=/dev/hdb
> > > > >
> > > > > Wouldn't that work?
> > > >
> > > > The latter works fine in my experience.  I do it regularly.
> > > >
> > > > The downside, is that cloning a 750GB drive takes a while
> > > > as it duplicates everything including unused sectors.
> > > >
> > > > Things like clonezilla just copy the "used"/active sectors.
> > > >
> > > > A popular way is to use sfdisk.  I do not remember the exact
> > > > syntax, but a pair of sfdisk commands can transfer the partition
> > > > information directly between two drives.
> > > >
> > > > Then use rsync to move the data across.
> > > >
> > > > You may have to run grub setup on the new disk too.
> > > >
> > > > --
> > > > Brian Litzinger
> > --
> > gentoo-amd64@gentoo.org mailing list
> >
> >
>
>
>
> --
> dott. ing. beso
-- 
gentoo-amd64@gentoo.org mailing list