[gentoo-user] Ghosting a Ext3 partition

[gentoo-user] Ghosting a Ext3 partition

[Jonathan Haws]

I am having a major problem right now with my laptop.

I regularly make backups of my system using Norton Ghost 2003 to DVD.  
However, my laptop crashed and I tried to restore my backup that I had made 
and it restores just find but when I try and boot it tells me that my Ext3 
filesystem is corrupt and had errors and I would have to run fsck manually.

When I ran fsck it told be that about every inode was invalid and that Group X 
had all sorts of other problems (I can't remember every little detail).

Doesn't Ghost work with Ext3?  What can I do to recover my system without 
reinstalling from scratch?

Thanks!

Jon

-- 
gentoo-user@lists.gentoo.org mailing list

Re: [gentoo-user] Ghosting a Ext3 partition

[maxim wexler]

> Doesn't Ghost work with Ext3?  What can I do to
> recover my system without 
> reinstalling from scratch?
> 

I've had success with #dd if=<partition-to-be-copied>
of=<partition-to-be-copied-to> bs=<varies>


mw


      ____________________________________________________________________________________
Be a better friend, newshound, and 
know-it-all with Yahoo! Mobile.  Try it now.  http://mobile.yahoo.com/;_ylt=Ahu06i62sR8HDtDypao8Wcj9tAcJ 

-- 
gentoo-user@lists.gentoo.org mailing list

Re: [gentoo-user] Ghosting a Ext3 partition

[Ritesh Kumar]

[-- Attachment #1: Type: text/plain, Size: 1177 bytes --]

On Fri, Feb 29, 2008 at 8:23 PM, maxim wexler <blissfix@yahoo.com> wrote:

> > Doesn't Ghost work with Ext3?  What can I do to
> > recover my system without
> > reinstalling from scratch?
> >
>
> I've had success with #dd if=<partition-to-be-copied>
> of=<partition-to-be-copied-to> bs=<varies>
>
>
Is there a reason why you backup the filesystem along with the data on it? I
do only minor backups... but even for anything major I would use a tool like
tar or rsync and drop the filesystem metadata entirely.

Also directly reading from the block device is hazardous unless you umount
(or mount as readonly) the filesystem in question. This is because, the
filesystem may not keep all the data synced to the disk at all points in
time.

Actually, even the tar/rsync method may lead to inconsistent files if the
files are backed up while they are still in active use. However, this is
potentially less damaging than having invalid filesystem metadata backed up
because the filesystem was not in sync. If you *have to* backup an always
active system, I would first create a snapshot and then back up the
snapshot. That ofcourse requires a more involved disk setup using LVM.

_r

[-- Attachment #2: Type: text/html, Size: 1559 bytes --]

Re: [gentoo-user] Ghosting a Ext3 partition

[Dan Farrell]

On Sat, 1 Mar 2008 02:04:31 -0500
"Ritesh Kumar" <ritesh@cs.unc.edu> wrote:

> On Fri, Feb 29, 2008 at 8:23 PM, maxim wexler <blissfix@yahoo.com>
> wrote:
> 
> > > Doesn't Ghost work with Ext3?  What can I do to
> > > recover my system without
> > > reinstalling from scratch?
> > >
> >
> > I've had success with #dd if=<partition-to-be-copied>
> > of=<partition-to-be-copied-to> bs=<varies>
> >
> >
> Is there a reason why you backup the filesystem along with the data
> on it? I do only minor backups... but even for anything major I would
> use a tool like tar or rsync and drop the filesystem metadata
> entirely.
> 
> Also directly reading from the block device is hazardous unless you
> umount (or mount as readonly) the filesystem in question. This is
> because, the filesystem may not keep all the data synced to the disk
> at all points in time.

not that i'd recommend it for production systems, but you could mount
with the 'sync' option to help with this.  

-- 
gentoo-user@lists.gentoo.org mailing list

Re: [gentoo-user] Ghosting a Ext3 partition

[Mark Kirkwood]

Dan Farrell wrote:
> On Sat, 1 Mar 2008 02:04:31 -0500
> "Ritesh Kumar" <ritesh@cs.unc.edu> wrote:
>
>   
>> On Fri, Feb 29, 2008 at 8:23 PM, maxim wexler <blissfix@yahoo.com>
>> wrote:
>>
>>     
>>>> Doesn't Ghost work with Ext3?  What can I do to
>>>> recover my system without
>>>> reinstalling from scratch?
>>>>
>>>>         
>>> I've had success with #dd if=<partition-to-be-copied>
>>> of=<partition-to-be-copied-to> bs=<varies>
>>>
>>>
>>>       
>> Is there a reason why you backup the filesystem along with the data
>> on it? I do only minor backups... but even for anything major I would
>> use a tool like tar or rsync and drop the filesystem metadata
>> entirely.
>>
>> Also directly reading from the block device is hazardous unless you
>> umount (or mount as readonly) the filesystem in question. This is
>> because, the filesystem may not keep all the data synced to the disk
>> at all points in time.
>>     
>
> not that i'd recommend it for production systems, but you could mount
> with the 'sync' option to help with this.  
>
>   
Even mounting sync is not safe, if you want to use dd for a backup then 
boot from the live cd to backup everything. Otherwise using these 
methods is risking a backup that once restored, does not work - not good 
for the blood pressure...

If you want to back the system up while it is running (in particular /), 
then you need to use a tool that understands how to create a backup 
image that is valid (i.e will boot) - something like xfsdump, dumpe2fs 
etc or smart tar/dump based tools like Amanda.

I would recommend using one of the dump tools for /boot, /, /usr, /var 
*at least*. I've had the misfortune of helping many people restore their 
broken Linux and Freebsd systems... and the only backups I've never had 
issues with have been the *dump variety. They are a little unfriendly at 
first, but they work.

regards

Mark




-- 
gentoo-user@lists.gentoo.org mailing list

Re: [gentoo-user] Ghosting a Ext3 partition

[Mark Kirkwood]

I wrote:
>
>
> If you want to back the system up while it is running (in particular 
> /), then you need to use a tool that understands how to create a 
> backup image that is valid (i.e will boot) - something like xfsdump, 
> *dumpe2fs* etc or smart tar/dump based tools like Amanda.
Hmm - dunno what I was thinking there -  'dumpe2fs' is completely wrong, 
should have written 'dump', sorry!

regards

Mark
-- 
gentoo-user@lists.gentoo.org mailing list

Re: [gentoo-user] Ghosting a Ext3 partition

[Rasmus Andersen]

On Sun, Mar 02, 2008 at 05:51:06PM +1300, Mark Kirkwood wrote:
> I wrote:
>>
>>
>> If you want to back the system up while it is running (in particular /), 
>> then you need to use a tool that understands how to create a backup image 
>> that is valid (i.e will boot) - something like xfsdump, *dumpe2fs* etc or 
>> smart tar/dump based tools like Amanda.
> Hmm - dunno what I was thinking there -  'dumpe2fs' is completely wrong, 
> should have written 'dump', sorry!

If you do backup live filesystems/data then dump is on par with dd; both
read from the underlying device and might bypass the kernel's page cache.
Ie., there might be unwritten data cached thats not on disk yet.
Tar/rdiff-backup/etc reads through the pagecache and avoids this problem.

The dump people talk a bit about this themselves on

   http://dump.sourceforge.net/isdumpdeprecated.html

Note I dont want to dis dump, backing up live filesystems is just tricky
(depending on your consistency requirements :) and dump adds another
level to that.

Cheers,
  Rasmus
-- 
gentoo-user@lists.gentoo.org mailing list

Re: [gentoo-user] Ghosting a Ext3 partition

[Mark Kirkwood]

Rasmus Andersen wrote:
>
> If you do backup live filesystems/data then dump is on par with dd; both
> read from the underlying device and might bypass the kernel's page cache.
> Ie., there might be unwritten data cached thats not on disk yet.
> Tar/rdiff-backup/etc reads through the pagecache and avoids this problem.
>
> The dump people talk a bit about this themselves on
>
>    http://dump.sourceforge.net/isdumpdeprecated.html
>
> Note I dont want to dis dump, backing up live filesystems is just tricky
> (depending on your consistency requirements :) and dump adds another
> level to that.
>
>
>   

Understood - I have seen that article too. I must say, I've mainly had 
experience with 'dump' on Freebsd and 'xfsdump' on Linux, and never had 
restore issues with *either* of these. Now I'm not sure whether these 
are supposed to be better than 'dump' on Linux aimed at ext2|3 
filesystems - certainly Freebsd's 'dump' has an option to tell it that 
it is dumping a 'live' filesystem, and the man pages for xfsrestore have 
notes concerning what happens when restoring an (xfs)dump from a 'live' 
filesystem - so they may well be!

On the other hand I've certainly routinely seen cases of people using dd 
(rsync, cpio, tar etc) and coming to grief at restore time. I am 
reluctant to suggest that folks use xfs and hence get access to xfsdump, 
as one of the nice things  about Linux is the  choice of a variety of 
filesystems - but it is pretty important to get able to backup of (for 
instance ) / ... and you usually don't have much option other than doing 
it live!

regards

Mark
 
-- 
gentoo-user@lists.gentoo.org mailing list

Re: [gentoo-user] Ghosting a Ext3 partition

[Rasmus Andersen]

On Sun, Mar 02, 2008 at 09:51:47PM +1300, Mark Kirkwood wrote:
> Understood - I have seen that article too. I must say, I've mainly had 
> experience with 'dump' on Freebsd and 'xfsdump' on Linux, and never had 
> restore issues with *either* of these. Now I'm not sure whether these are 
> supposed to be better than 'dump' on Linux aimed at ext2|3 filesystems - 
> certainly Freebsd's 'dump' has an option to tell it that it is dumping a 
> 'live' filesystem, and the man pages for xfsrestore have notes concerning 
> what happens when restoring an (xfs)dump from a 'live' filesystem - so they 
> may well be!

FreeBSD's softupdates should make filesystem state always consistent,
metadatawise. Or so I think I remember, its been a while. That might
aleviate some of the problems noted on the dump page I referenced.

> On the other hand I've certainly routinely seen cases of people using dd 
> (rsync, cpio, tar etc) and coming to grief at restore time. I am reluctant 
> to suggest that folks use xfs and hence get access to xfsdump, as one of 
> the nice things  about Linux is the  choice of a variety of filesystems - 
> but it is pretty important to get able to backup of (for instance ) / ... 
> and you usually don't have much option other than doing it live!

I use rdiff-backup for my backups but then again I have low requirements
wrt. consistency outside file-level. I have considered LVM snapshots
since I use LVM already but havent bothered so far.

Cheers,
  Rasmus
-- 
gentoo-user@lists.gentoo.org mailing list

Re: [gentoo-user] Ghosting a Ext3 partition

[Mark Kirkwood]

Rasmus Andersen wrote:
>
>
> FreeBSD's softupdates should make filesystem state always consistent,
> metadatawise. Or so I think I remember, its been a while. That might
> aleviate some of the problems noted on the dump page I referenced.
>
>   
>
Freebsd's dump -L (live option) uses ufs2 snapshot capability to ensure 
consistency, so its dump is reasonably smart. (FWIW / on Freebsd usually 
does not have softupdates specified, so usually won't help here). So 
yeah, it looks like some care is needed with plain old 'dump' on Linux - 
which is a bit of a pest!
> I use rdiff-backup for my backups but then again I have low requirements
> wrt. consistency outside file-level. I have considered LVM snapshots
> since I use LVM already but havent bothered so far.
>
>
>   
Right - what you intend the backup to protect against drives all this 
sort of stuff.



-- 
gentoo-user@lists.gentoo.org mailing list

Re: [gentoo-user] Ghosting a Ext3 partition

[Jonathan Haws]

On Sunday March 2 2008 16:43, Mark Kirkwood wrote:
> Right - what you intend the backup to protect against drives all this
> sort of stuff.

The thing that is driving my backups is a hard disk failure.  Hence I was 
using Ghost instead of something else so I can backup the entire drive and 
not just a single partition.  That enables the quickest recovery of the 
entire system in the event of a failure.

I have looked everywhere I can think of to find a tool that is similar to 
Ghost that will backup the entire hard drive to an image that I can put to 
DVD, without including free blocks on the disk (I don't want an 80GB image of 
an 80GB drive when only 5GB are in use at the time).  Does anyone know of a 
tool capable of this that runs on Linux and has FULL Linux fs support?

-- 
Jonathan

-- 
gentoo-user@lists.gentoo.org mailing list

Re: [gentoo-user] Ghosting a Ext3 partition

[Mark Kirkwood]

Jonathan Haws wrote:
> On Sunday March 2 2008 16:43, Mark Kirkwood wrote:
>   
>> Right - what you intend the backup to protect against drives all this
>> sort of stuff.
>>     
>
> The thing that is driving my backups is a hard disk failure.  Hence I was 
> using Ghost instead of something else so I can backup the entire drive and 
> not just a single partition.  That enables the quickest recovery of the 
> entire system in the event of a failure.
>
> I have looked everywhere I can think of to find a tool that is similar to 
> Ghost that will backup the entire hard drive to an image that I can put to 
> DVD, without including free blocks on the disk (I don't want an 80GB image of 
> an 80GB drive when only 5GB are in use at the time).  Does anyone know of a 
> tool capable of this that runs on Linux and has FULL Linux fs support?
>
>   
I actually think that 'dump' will do what you want... provided you can 
choose a time when the machine is not busy (should be easy if it's your 
desktop!). You have to do 1 dump per filesystem, but many desktop 
installations only consist of / (+ maybe /boot) anyway. Also dump of a 
80Gb system that only uses 5Gb will produce a 5Gb image.... Also it can 
do incremental an cumulative backups.

Some friends of mine use Amanda to backup their (Redhat/Centos) servers, 
that may worth looking at too.

Cheers

Mark
-- 
gentoo-user@lists.gentoo.org mailing list

Re: [gentoo-user] Ghosting a Ext3 partition

[Matthias Bethke]

[-- Attachment #1: Type: text/plain, Size: 1599 bytes --]

Hi Mark,
on Tue, Mar 04, 2008 at 05:39:12PM +1300, you wrote:
>> {"Ghost" functionality]
>>   
> I actually think that 'dump' will do what you want... provided you can 
> choose a time when the machine is not busy (should be easy if it's your 
> desktop!). You have to do 1 dump per filesystem, but many desktop 
> installations only consist of / (+ maybe /boot) anyway. Also dump of a 80Gb 
> system that only uses 5Gb will produce a 5Gb image.... Also it can do 
> incremental an cumulative backups.
>
> Some friends of mine use Amanda to backup their (Redhat/Centos) servers, 
> that may worth looking at too.

Amanda is very versatile but it can be a bitch to configure and is IMHO
only really worth it for larger installations with at least more than
one machine and preferably a backup server. But it can also use dump(1)
internally. I didn't really follow the thread but it seems dump has a
problem with busy file systems? I used amanda with dump on several
machines for a few years and never had any so it should be fine for a
desktop.
One method I used for getting the image size down (but which is no good
on a live system) was to use "dd if=/dev/zero of=dummy bs=1M" to quicky
write a file of zeroes that would fill up all free blocks, then dd the
whole partition through gzip that would just compress away the free
blocks. Works fine for install images but only when the disk is not
mounted r/w during imaging.

cheers,
	Matthias
-- 
I prefer encrypted and signed messages. KeyID: FAC37665
Fingerprint: 8C16 3F0A A6FC DF0D 19B0  8DEF 48D9 1700 FAC3 7665

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

[gentoo-user] Re: Ghosting a Ext3 partition

[Michael Schmarck]

Jonathan Haws <Jonathan.Haws@sdl.usu.edu> wrote:

> On Sunday March 2 2008 16:43, Mark Kirkwood wrote:
>> Right - what you intend the backup to protect against drives all this
>> sort of stuff.
> 
> The thing that is driving my backups is a hard disk failure.

Of course :)

> Hence I was 
> using Ghost instead of something else so I can backup the entire drive and
> not just a single partition.  That enables the quickest recovery of the
> entire system in the event of a failure.

Depends. If you'd just store tar files, you'd just have to create the
filesystem beforehand and then restore the tar files. Won't take much
longer than restoring a Ghost image, I'd suppose.

> I have looked everywhere I can think of to find a tool that is similar to
> Ghost that will backup the entire hard drive to an image that I can put to
> DVD, without including free blocks on the disk (I don't want an 80GB image
> of
> an 80GB drive when only 5GB are in use at the time).

That's why I'd rather use tar... An additional benefit is, that tar
is *MUCH* simpler than Ghost (or partimage). Because of that, you could
use your tar file in many more applications (for example if you want
to have a look at what the file was at your last backup).

> Does anyone know of 
> a tool capable of this that runs on Linux and has FULL Linux fs support?

partimage.

Michael

-- 
gentoo-user@lists.gentoo.org mailing list

Re: [gentoo-user] Ghosting a Ext3 partition

[Mark Kirkwood]

Jonathan Haws wrote:
> I regularly make backups of my system using Norton Ghost 2003 to DVD.  
> However, my laptop crashed and I tried to restore my backup that I had made 
> and it restores just find but when I try and boot it tells me that my Ext3 
> filesystem is corrupt and had errors and I would have to run fsck manually.
>
> When I ran fsck it told be that about every inode was invalid and that Group X 
> had all sorts of other problems (I can't remember every little detail).
>
> Doesn't Ghost work with Ext3?  What can I do to recover my system without 
> reinstalling from scratch?
>
>   

Hmm - I would have thought it would... but for future reference I would 
recommend using a more Linux friendly (and reliable) tool like dumpe2fs ...

Mark

-- 
gentoo-user@lists.gentoo.org mailing list

Re: [gentoo-user] Ghosting a Ext3 partition

[Etaoin Shrdlu]

On Friday 29 February 2008, Jonathan Haws wrote:

> Doesn't Ghost work with Ext3?  What can I do to recover my system
> without reinstalling from scratch?

Don't know about ghost, but take a look at partimage 
(http://partimage.org).
-- 
gentoo-user@lists.gentoo.org mailing list

Re: [gentoo-user] Ghosting a Ext3 partition

[Florian Philipp]

[-- Attachment #1: Type: text/plain, Size: 804 bytes --]


On Fri, 2008-02-29 at 12:27 -0700, Jonathan Haws wrote:
> I am having a major problem right now with my laptop.
> 
> I regularly make backups of my system using Norton Ghost 2003 to DVD.  
> However, my laptop crashed and I tried to restore my backup that I had made 
> and it restores just find but when I try and boot it tells me that my Ext3 
> filesystem is corrupt and had errors and I would have to run fsck manually.
> 
> When I ran fsck it told be that about every inode was invalid and that Group X 
> had all sorts of other problems (I can't remember every little detail).
> 
> Doesn't Ghost work with Ext3?  What can I do to recover my system without 
> reinstalling from scratch?
> 
> Thanks!
> 
> Jon
> 

You could try testdisk. Maybe it can recover. Or try mke2fs -S 

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 189 bytes --]