[gentoo-user] filesystems

[gentoo-user] filesystems

[William Kenworthy]

Currently I have around 3 terrabytes of storage across a number of
gentoo machines (4 at the moment) - at any one time 1/2 to 1 terrabyte
is unused, but mostly in scattered chunks.  Some space is exported via
NFS and samba for backups and shared files.

What I would really like is a file system that would unify these spaces
and present them to the network as storage space - ideally with
redundant data storage so one or more machines can dissappear and the
data is still available.

AFS is not quite what I want (or maybe it is, but it doesn't seem to
handle transient storage duplication)

I have a memory of seeing a beast that does this in the past but cant
remember what it is - any suggestions?

BillK



-- 
William Kenworthy <billk@iinet.net.au>
Home in Perth!

Re: [gentoo-user] [OT] filesystems

[Kobboi]

On Mon, 2008-11-24 at 07:31 +0900, William Kenworthy wrote:
> Currently I have around 3 terrabytes of storage across a number of
> gentoo machines (4 at the moment) - at any one time 1/2 to 1 terrabyte
> is unused, but mostly in scattered chunks.  Some space is exported via
> NFS and samba for backups and shared files.

OT: the prefix is "tera" not "terra".

Re: [gentoo-user] [OT] filesystems

[Dale]

Kobboi wrote:
> On Mon, 2008-11-24 at 07:31 +0900, William Kenworthy wrote:
>   
>> Currently I have around 3 terrabytes of storage across a number of
>> gentoo machines (4 at the moment) - at any one time 1/2 to 1 terrabyte
>> is unused, but mostly in scattered chunks.  Some space is exported via
>> NFS and samba for backups and shared files.
>>     
>
> OT: the prefix is "tera" not "terra".
>
>
>
>   

I think it is LVMS or something.  Linux volume management system??  I
think Redhat calls it EVMS or something.  If that doesn't help, let me
know and I'll google it some.

Dale

:-)  :-) 

Re: [gentoo-user] [OT] filesystems

[W.Kenworthy]

On Sun, 2008-11-23 at 19:06 -0600, Dale wrote:
> Kobboi wrote:
> > On Mon, 2008-11-24 at 07:31 +0900, William Kenworthy wrote:
> >   
> >> Currently I have around 3 terrabytes of storage across a number of
> >> gentoo machines (4 at the moment) - at any one time 1/2 to 1 terrabyte
> >> is unused, but mostly in scattered chunks.  Some space is exported via
> >> NFS and samba for backups and shared files.
> >>     
> >
> > OT: the prefix is "tera" not "terra".
> >
> >
> >
> >   
> 
> I think it is LVMS or something.  Linux volume management system??  I
> think Redhat calls it EVMS or something.  If that doesn't help, let me
> know and I'll google it some.
> 
> Dale
> 
> :-)  :-) 
> 
I think LVM is only useful on the same system - it doesnt deal with
network resources.  Most of my systems are using LVM2 at the moment.

Billk

Re: [gentoo-user] [OT] filesystems

[Roy Wright]

W.Kenworthy wrote:
> On Sun, 2008-11-23 at 19:06 -0600, Dale wrote:
>> Kobboi wrote:
>>> On Mon, 2008-11-24 at 07:31 +0900, William Kenworthy wrote:
>>>   
>>>> Currently I have around 3 terrabytes of storage across a number of
>>>> gentoo machines (4 at the moment) - at any one time 1/2 to 1 terrabyte
>>>> is unused, but mostly in scattered chunks.  Some space is exported via
>>>> NFS and samba for backups and shared files.
>>>>     

maybe ZFS?

http://en.wikipedia.org/wiki/ZFS

Re: [gentoo-user] [OT] filesystems

[Dirk Heinrichs]

Am Montag, 24. November 2008 02:06:04 schrieb Dale:

> I think it is LVMS or something.  Linux volume management system??  I
> think Redhat calls it EVMS or something.

Two things, (more ore less) one purpose:

1) LVM: Logical Volume Management
2) EVMS: Enterprise Volume Management System

1) is used for management of Logical Volumes, organised in Volume Groups, 
which could be spread accross one or more Physical Volumes.

@William: If one or more of the PVs is a Network Block Device, you're not 
bound to the local machine.

2) From IBM, not RH. It's an umbrella for the whole storage management chain 
from fdisk over (SW-) RAID and Logical Volumes to filesystem creation and 
maintenance.

HTH...

	Dirk

Re: [gentoo-user] filesystems

[Dirk Heinrichs]

Am Sonntag, 23. November 2008 23:31:30 schrieb William Kenworthy:

> What I would really like is a file system that would unify these spaces
> and present them to the network as storage space - ideally with
> redundant data storage so one or more machines can dissappear and the
> data is still available.
>
> AFS is not quite what I want (or maybe it is, but it doesn't seem to
> handle transient storage duplication)

For a non-native speaker, could you explain "transient storage duplication" a 
bit more? Because I think AFS may well be what you're looking for, or maybe 
its cousin Coda.

Bye...

	Dirk

Re: [gentoo-user] [OT] filesystems

[Dale]

Dirk Heinrichs wrote:
> Am Montag, 24. November 2008 02:06:04 schrieb Dale:
>
>   
>> I think it is LVMS or something.  Linux volume management system??  I
>> think Redhat calls it EVMS or something.
>>     
>
> Two things, (more ore less) one purpose:
>
> 1) LVM: Logical Volume Management
> 2) EVMS: Enterprise Volume Management System
>
> 1) is used for management of Logical Volumes, organised in Volume Groups, 
> which could be spread accross one or more Physical Volumes.
>
> @William: If one or more of the PVs is a Network Block Device, you're not 
> bound to the local machine.
>
> 2) From IBM, not RH. It's an umbrella for the whole storage management chain 
> from fdisk over (SW-) RAID and Logical Volumes to filesystem creation and 
> maintenance.
>
> HTH...
>
> 	Dirk
>
>
>   

I knew it was something like that.  I thought it was networkable but was
not sure.  You guys sure know more about that than I do. 

Dale

:-)  :-) 

Re: [gentoo-user] filesystems

[William Kenworthy]

On Mon, 2008-11-24 at 07:30 +0100, Dirk Heinrichs wrote:
> Am Sonntag, 23. November 2008 23:31:30 schrieb William Kenworthy:
> 
> > What I would really like is a file system that would unify these spaces
> > and present them to the network as storage space - ideally with
> > redundant data storage so one or more machines can dissappear and the
> > data is still available.
> >
> > AFS is not quite what I want (or maybe it is, but it doesn't seem to
> > handle transient storage duplication)
> 
> For a non-native speaker, could you explain "transient storage duplication" a 
> bit more? Because I think AFS may well be what you're looking for, or maybe 
> its cousin Coda.
> 
> Bye...
> 
> 	Dirk
> 

By transient storage I mean that the data is duplicated across across
physical storage spaces so that if a machine goes down, the data is
still available.  I thought Andrews FS did that, but didnt see when
looking at it yesterday.

BillK

Re: [gentoo-user] [OT] filesystems

[Volker Armin Hemmann]

On Montag 24 November 2008, Dale wrote:
> Dirk Heinrichs wrote:
> > Am Montag, 24. November 2008 02:06:04 schrieb Dale:
> >> I think it is LVMS or something.  Linux volume management system??  I
> >> think Redhat calls it EVMS or something.
> >
> > Two things, (more ore less) one purpose:
> >
> > 1) LVM: Logical Volume Management
> > 2) EVMS: Enterprise Volume Management System
> >
> > 1) is used for management of Logical Volumes, organised in Volume Groups,
> > which could be spread accross one or more Physical Volumes.
> >
> > @William: If one or more of the PVs is a Network Block Device, you're not
> > bound to the local machine.
> >
> > 2) From IBM, not RH. It's an umbrella for the whole storage management
> > chain from fdisk over (SW-) RAID and Logical Volumes to filesystem
> > creation and maintenance.
> >
> > HTH...
> >
> > 	Dirk
>
> I knew it was something like that.  I thought it was networkable but was
> not sure.  You guys sure know more about that than I do.

- evms was used for a while by Suse - I don't know if they still do.
- there is a long lvm-is-broken-threadon f.g.o.

Re: [gentoo-user] [OT] filesystems

[Dale]

Volker Armin Hemmann wrote:
> On Montag 24 November 2008, Dale wrote:
>   
>>
>> I knew it was something like that.  I thought it was networkable but was
>> not sure.  You guys sure know more about that than I do.
>>     
>
> - evms was used for a while by Suse - I don't know if they still do.
> - there is a long lvm-is-broken-threadon f.g.o.
>
>
>
>   

I used to be subscribed to the mailing list, thought about using one or
the other.  Just before I unsubscribed, there were some people trying to
get it back up and going.  I'm not sure how that went or if it is still
being worked on or not.  It seemed pretty neat but I just couldn't never
get up the nerve to switch over.

Maybe it will survive.  I'm waiting on reiserfs4 to go stable.  ;-) 

Dale

:-)  :-) 

Re: [gentoo-user] filesystems

[Dirk Heinrichs]

Am Montag, 24. November 2008 11:30:25 schrieb William Kenworthy:

> By transient storage I mean that the data is duplicated across across
> physical storage spaces so that if a machine goes down, the data is
> still available.

OK, thanks.

> I thought Andrews FS did that, but didnt see when
> looking at it yesterday.

Yes, (Open-)AFS indeed does this. However, this replication is read-only. This 
means you can read the data as long as at least one replica is available and 
write the data as long as the original (the read-write) volume is available. 
There are also some other things to keep in mind:

* AFS' primary tool for access control are its access control lists (ACL), but 
those are not posix, but AFS ACLs and they apply at the directory (not file) 
level. However, that's usually sufficient, because one can work with subdirs 
and symbolic links to implement more restrictive access for some files in the 
same directory.

* ACLs can also contain host names.

* If a volume is replicated, the client always prefers the read-only path 
(read-write volumes are usually accessed via /afs/.mycell.mydomain, while 
read-only volumes (if they exist) are accessed via /afs/mycell.mydomain). So 
if you want to modify a file you must explicitely open it via the rw-path.

* Replication doesn't happen automatically, needs an explicit command.

* Support for backup volumes is also there (comes with its own backup system).

* Can move volumes to different servers while online.

* Data is cached on the client.

* You'll need Kerberos 5.

If you have further questions, feel free to ask.

Bye...

	Dirk

Re: [gentoo-user] filesystems

[William Kenworthy]

On Mon, 2008-11-24 at 12:07 +0100, Dirk Heinrichs wrote:
> Am Montag, 24. November 2008 11:30:25 schrieb William Kenworthy:
> 
> > By transient storage I mean that the data is duplicated across across
> > physical storage spaces so that if a machine goes down, the data is
> > still available.
> 
> OK, thanks.
> 
> > I thought Andrews FS did that, but didnt see when
> > looking at it yesterday.
> 
> Yes, (Open-)AFS indeed does this. However, this replication is read-only. This 
> means you can read the data as long as at least one replica is available and 
> write the data as long as the original (the read-write) volume is available. 
> There are also some other things to keep in mind:
> 
> * AFS' primary tool for access control are its access control lists (ACL), but 
> those are not posix, but AFS ACLs and they apply at the directory (not file) 
> level. However, that's usually sufficient, because one can work with subdirs 
> and symbolic links to implement more restrictive access for some files in the 
> same directory.
> 
> * ACLs can also contain host names.
> 
> * If a volume is replicated, the client always prefers the read-only path 
> (read-write volumes are usually accessed via /afs/.mycell.mydomain, while 
> read-only volumes (if they exist) are accessed via /afs/mycell.mydomain). So 
> if you want to modify a file you must explicitely open it via the rw-path.
> 
> * Replication doesn't happen automatically, needs an explicit command.
> 
> * Support for backup volumes is also there (comes with its own backup system).
> 
> * Can move volumes to different servers while online.
> 
> * Data is cached on the client.
> 
> * You'll need Kerberos 5.
> 
> If you have further questions, feel free to ask.
> 
> Bye...
> 
> 	Dirk
> 

Discovered this:

http://en.wikipedia.org/wiki/List_of_file_systems#Distributed_file_systems

Thats going to keep me busy for awhile!

BillK


-- 
William Kenworthy <billk@iinet.net.au>
Home in Perth!

Re: [gentoo-user] filesystems

[Dirk Heinrichs]

Am Montag, 24. November 2008 13:03:13 schrieb William Kenworthy:

> Discovered this:
>
> http://en.wikipedia.org/wiki/List_of_file_systems#Distributed_file_systems
>
> Thats going to keep me busy for awhile!

Interesting link. However, NFS, SMB, AFP and NCP are NOT distributed 
filesystems. They're just network filesystems.

Bye...

	Dirk

Re: [gentoo-user] [OT] filesystems

[Dirk Heinrichs]

Am Montag, 24. November 2008 12:07:34 schrieb Dale:

> Maybe it will survive.  I'm waiting on reiserfs4 to go stable.  ;-)

Well, with its inventor being imprisoned for the next 15 years or so, you'll 
have to be patient. I for one wait for btrfs.

Bye...

	Dirk

Re: [gentoo-user] filesystems

[Stroller]

On 24 Nov 2008, at 11:07, Dirk Heinrichs wrote:
> ...
> If you have further questions, feel free to ask.

I would love a file system that transparently replicates over several  
systems - say 2 - 5.

It doesn't need to amalgamate spare in any way (as BillK requests),  
let's just say I just have a couple of gig on each machine that I want  
replicated.

I should be able to read & operate on the files on the "partition"  
just as normal, but when a file is saved to or deleted from any one  
machine the change should be replicated on all the others across the  
(slow) network.

Basically, the idea is that I should be able to set machines A, B & C  
as MX for my domain and be able to read a new message whichever  
machine receives it. I should be able to run all 3 machines as IMAP  
servers and connect to any one of them to see the same view of my  
messages. When the IMAP server deletes or moves a message (on, say, A)  
that transaction should be replicated across B & C. (But likewise if  
the message is moved or deleted on B then the transaction should be  
replicated across A & C).

I suspect I would be optimistic if I hoped for something so  
sophisticated to be readily available, as I am aware that this would  
be problematic to implement. But do you have any suggestions?

Stroller.

Re: [gentoo-user] filesystems

[William Kenworthy]

On Mon, 2008-11-24 at 12:35 +0000, Stroller wrote:
> On 24 Nov 2008, at 11:07, Dirk Heinrichs wrote:
> > ...
> > If you have further questions, feel free to ask.
> 
> I would love a file system that transparently replicates over several  
> systems - say 2 - 5.
> 
> It doesn't need to amalgamate spare in any way (as BillK requests),  
> let's just say I just have a couple of gig on each machine that I want  
> replicated.
> 
> I should be able to read & operate on the files on the "partition"  
> just as normal, but when a file is saved to or deleted from any one  
> machine the change should be replicated on all the others across the  
> (slow) network.
> 
> Basically, the idea is that I should be able to set machines A, B & C  
> as MX for my domain and be able to read a new message whichever  
> machine receives it. I should be able to run all 3 machines as IMAP  
> servers and connect to any one of them to see the same view of my  
> messages. When the IMAP server deletes or moves a message (on, say, A)  
> that transaction should be replicated across B & C. (But likewise if  
> the message is moved or deleted on B then the transaction should be  
> replicated across A & C).
> 
> I suspect I would be optimistic if I hoped for something so  
> sophisticated to be readily available, as I am aware that this would  
> be problematic to implement. But do you have any suggestions?
> 
> Stroller.
> 
> 

I set up an openmosix cluster once using dfs I think.  It replicated
data just like you want so each exported thread was seeing consistent
file space.  It did work, but had a few issues ... I think it was
designed by MS being one :)

http://en.wikipedia.org/wiki/Distributed_File_System_(Microsoft)

BillK

Re: [gentoo-user] filesystems

[Dirk Heinrichs]

Am Montag, 24. November 2008 13:35:25 schrieb Stroller:

> I suspect I would be optimistic if I hoped for something so  
> sophisticated to be readily available, as I am aware that this would  
> be problematic to implement. But do you have any suggestions?

Maybe Coda.

Bye...

	Dirk

Re: [gentoo-user] [OT] filesystems

[Volker Armin Hemmann]

On Montag 24 November 2008, Dirk Heinrichs wrote:
> Am Montag, 24. November 2008 12:07:34 schrieb Dale:
> > Maybe it will survive.  I'm waiting on reiserfs4 to go stable.  ;-)
>
> Well, with its inventor being imprisoned for the next 15 years or so,
> you'll have to be patient. I for one wait for btrfs.
>
> Bye...
>
> 	Dirk

Edward is not imprisioned and doing a fine job. Even in face of such current 
sabotage attempts as by Morton/Piggin.

Re: [gentoo-user] filesystems

[Dirk Heinrichs]

Am Montag, 24. November 2008 13:44:06 schrieb William Kenworthy:

> I set up an openmosix cluster once using dfs I think.  It replicated
> data just like you want so each exported thread was seeing consistent
> file space.  It did work, but had a few issues ... I think it was
> designed by MS being one :)
>
> http://en.wikipedia.org/wiki/Distributed_File_System_(Microsoft)

I strongly doubt that you used MS DFS in a Linux based cluster. If it was 
really DFS, then this one: 
http://en.wikipedia.org/wiki/DCE_Distributed_File_System.

Both are totally different beasts.

Bye...

	Dirk

Re: [gentoo-user] [OT] filesystems

[Dirk Heinrichs]

Am Montag, 24. November 2008 13:49:38 schrieb Volker Armin Hemmann:
> On Montag 24 November 2008, Dirk Heinrichs wrote:
> > Am Montag, 24. November 2008 12:07:34 schrieb Dale:
> > > Maybe it will survive.  I'm waiting on reiserfs4 to go stable.  ;-)
> >
> > Well, with its inventor being imprisoned for the next 15 years or so,
> > you'll have to be patient. I for one wait for btrfs.
>
> Edward is not imprisioned and doing a fine job. Even in face of such
> current sabotage attempts as by Morton/Piggin.

Is he the inventor? AFAIK he's (one of) the last remaining developer(s). 
However, btrfs also seems to be the favourite of many kernel hackers as they 
want to have a ZFS competitor.

Bye...

	Dirk

Re: [gentoo-user] filesystems

[William Kenworthy]

On Mon, 2008-11-24 at 13:50 +0100, Dirk Heinrichs wrote:
> Am Montag, 24. November 2008 13:44:06 schrieb William Kenworthy:
> 
> > I set up an openmosix cluster once using dfs I think.  It replicated
> > data just like you want so each exported thread was seeing consistent
> > file space.  It did work, but had a few issues ... I think it was
> > designed by MS being one :)
> >
> > http://en.wikipedia.org/wiki/Distributed_File_System_(Microsoft)
> 
> I strongly doubt that you used MS DFS in a Linux based cluster. If it was 
> really DFS, then this one: 
> http://en.wikipedia.org/wiki/DCE_Distributed_File_System.
> 
> Both are totally different beasts.
> 
> Bye...
> 
> 	Dirk
> 

Your right, it was mfs - I still had the relevant line in an fstab -
must have been 3-4 years ago at least.

#mfs			/mfs		mfs		dfsa=1,noauto		0 0

Its the dfsa argument that confised my memory.

BillK

-- 
William Kenworthy <billk@iinet.net.au>
Home in Perth!

Re: [gentoo-user] [OT] filesystems

[Volker Armin Hemmann]

On Montag 24 November 2008, Dirk Heinrichs wrote:
> Am Montag, 24. November 2008 13:49:38 schrieb Volker Armin Hemmann:
> > On Montag 24 November 2008, Dirk Heinrichs wrote:
> > > Am Montag, 24. November 2008 12:07:34 schrieb Dale:
> > > > Maybe it will survive.  I'm waiting on reiserfs4 to go stable.  ;-)
> > >
> > > Well, with its inventor being imprisoned for the next 15 years or so,
> > > you'll have to be patient. I for one wait for btrfs.
> >
> > Edward is not imprisioned and doing a fine job. Even in face of such
> > current sabotage attempts as by Morton/Piggin.
>
> Is he the inventor? AFAIK he's (one of) the last remaining developer(s).
> However, btrfs also seems to be the favourite of many kernel hackers as
> they want to have a ZFS competitor.
>

he is not - but after the invention is implemented, the inventor is not needed 
anymore ;)

btrfs looks very promising. I hope it will become a good fs. Fast for 
everybody, stable, efficient. We will see. Until then I will stay with 
r4+compression.

Re: [gentoo-user] [OT] filesystems

[GMail]

On Monday 24 November 2008 14:49:38 Volker Armin Hemmann wrote:
> On Montag 24 November 2008, Dirk Heinrichs wrote:
> > Am Montag, 24. November 2008 12:07:34 schrieb Dale:
> > > Maybe it will survive.  I'm waiting on reiserfs4 to go stable.  ;-)
> >
> > Well, with its inventor being imprisoned for the next 15 years or so,
> > you'll have to be patient. I for one wait for btrfs.
> >
> > Bye...
> >
> > 	Dirk
>
> Edward is not imprisioned and doing a fine job. Even in face of such
> current sabotage attempts as by Morton/Piggin.

Never mind Hans' troubles, whoever maintains Reiser4 still has to get it past 
Linux, Alan Cox, Greg KH and co. That is not likely to be easy.



-- 
alan dot mckinnon at gmail dot com

Re: [gentoo-user] [OT] filesystems

[GMail]

On Monday 24 November 2008 13:07:34 Dale wrote:
> I used to be subscribed to the mailing list, thought about using one or
> the other.  Just before I unsubscribed, there were some people trying to
> get it back up and going.  I'm not sure how that went or if it is still
> being worked on or not.  It seemed pretty neat but I just couldn't never
> get up the nerve to switch over.
>
> Maybe it will survive.  I'm waiting on reiserfs4 to go stable.  ;-)

dream on brother, dream on. Ain't gonna happen anytime soon. You'll have 
better luck getting Sun to dual-license ZFS under GPL :-)

OTOH, ext4 and btrfs seem to have some interesting feature sets in the roadmap

-- 
alan dot mckinnon at gmail dot com

Re: [gentoo-user] [OT] filesystems

[GMail]

On Monday 24 November 2008 08:28:33 Dirk Heinrichs wrote:
> @William: If one or more of the PVs is a Network Block Device, you're not
> bound to the local machine.

I'd never thought of that, but it makes sense. PV wants a raw block device and 
couldn't care less if it leads to local disk or something else.

How does it cope with network outages though? In my experience, LVM is not 
exactly graceful when one of it's PVs goes away

-- 
alan dot mckinnon at gmail dot com

Re: [gentoo-user] [OT] filesystems

[GMail]

On Monday 24 November 2008 07:58:55 Roy Wright wrote:
> W.Kenworthy wrote:
> > On Sun, 2008-11-23 at 19:06 -0600, Dale wrote:
> >> Kobboi wrote:
> >>> On Mon, 2008-11-24 at 07:31 +0900, William Kenworthy wrote:
> >>>> Currently I have around 3 terrabytes of storage across a number of
> >>>> gentoo machines (4 at the moment) - at any one time 1/2 to 1 terrabyte
> >>>> is unused, but mostly in scattered chunks.  Some space is exported via
> >>>> NFS and samba for backups and shared files.
>
> maybe ZFS?
>
> http://en.wikipedia.org/wiki/ZFS

But not on Linux as a kernel module sadly

There's a FUSE implementation which is considerably slower (being FUSE)

-- 
alan dot mckinnon at gmail dot com

Re: [gentoo-user] [OT] filesystems

[Joerg Schilling]

GMail <alan.mckinnon@gmail.com> wrote:

> On Monday 24 November 2008 13:07:34 Dale wrote:
> > I used to be subscribed to the mailing list, thought about using one or
> > the other.  Just before I unsubscribed, there were some people trying to
> > get it back up and going.  I'm not sure how that went or if it is still
> > being worked on or not.  It seemed pretty neat but I just couldn't never
> > get up the nerve to switch over.
> >
> > Maybe it will survive.  I'm waiting on reiserfs4 to go stable.  ;-)
>
> dream on brother, dream on. Ain't gonna happen anytime soon. You'll have 
> better luck getting Sun to dual-license ZFS under GPL :-)

There is no need to dual license ZFS. There is absolutely no problem
with using ZFS from Linux. What's missing is the will from the kernel developers
to work on the incompatible VFS interface in the linux kernel.



Jörg

-- 
 EMail:joerg@schily.isdn.cs.tu-berlin.de (home) Jörg Schilling D-13353 Berlin
       js@cs.tu-berlin.de                (uni)  
       schilling@fokus.fraunhofer.de     (work) Blog: http://schily.blogspot.com/
 URL:  http://cdrecord.berlios.de/private/ ftp://ftp.berlios.de/pub/schily

Re: [gentoo-user] [OT] filesystems

[Stroller]

On 24 Nov 2008, at 14:12, GMail wrote:

> On Monday 24 November 2008 07:58:55 Roy Wright wrote:
>> W.Kenworthy wrote:
>>> On Sun, 2008-11-23 at 19:06 -0600, Dale wrote:
>>>> Kobboi wrote:
>>>>> On Mon, 2008-11-24 at 07:31 +0900, William Kenworthy wrote:
>>>>>> Currently I have around 3 terrabytes of storage across a number  
>>>>>> of
>>>>>> gentoo machines (4 at the moment) - at any one time 1/2 to 1  
>>>>>> terrabyte
>>>>>> is unused, but mostly in scattered chunks.  Some space is  
>>>>>> exported via
>>>>>> NFS and samba for backups and shared files.
>>
>> maybe ZFS?
>>
>> http://en.wikipedia.org/wiki/ZFS
>
> But not on Linux as a kernel module sadly
>
> There's a FUSE implementation which is considerably slower (being  
> FUSE)

IIRC the author of Linux-ZFS cites the NTFS implementation as  
demonstrating that FUSE can produce quite acceptable performance.

Of course, maybe performance of NTFS would be better were it a kernel  
module, but I get the strong impression Linux-ZFS is poor because it  
doesn't have the developer resources needed to improve it.

Stroller.

Re: [gentoo-user] [OT] filesystems

[Dirk Heinrichs]

Am Montag, 24. November 2008 15:12:00 schrieb GMail:

> How does it cope with network outages though? In my experience, LVM is not
> exactly graceful when one of it's PVs goes away

Don't know. I just know it's possible but never did it myself.

Bye...

	Dirk

Re: [gentoo-user] [OT] filesystems

[Dirk Heinrichs]

Am Montag, 24. November 2008 14:50:30 schrieb Volker Armin Hemmann:

> he is not - but after the invention is implemented, the inventor is not
> needed anymore ;)

Yes, that's right.

Bye...

	Dirk

Re: [gentoo-user] filesystems

[Dirk Heinrichs]

Am Montag, 24. November 2008 12:07:55 schrieb Dirk Heinrichs:

> If you have further questions, feel free to ask.

One smalll thing to add: If you decide to use it, there's a Howto under 
http://www.gentoo-wiki.info/OpenAFS. Do NOT use the one from gentoo.org, it's 
old, outdated and partly incorrect.

Bye...

	Dirk

Re: [gentoo-user] [OT] filesystems

[Joerg Schilling]

Roy Wright <roy@wright.org> wrote:

> W.Kenworthy wrote:
> > On Sun, 2008-11-23 at 19:06 -0600, Dale wrote:
> >> Kobboi wrote:
> >>> On Mon, 2008-11-24 at 07:31 +0900, William Kenworthy wrote:
> >>>   
> >>>> Currently I have around 3 terrabytes of storage across a number of
> >>>> gentoo machines (4 at the moment) - at any one time 1/2 to 1 terrabyte
> >>>> is unused, but mostly in scattered chunks.  Some space is exported via
> >>>> NFS and samba for backups and shared files.
> >>>>     
>
> maybe ZFS?
>
> http://en.wikipedia.org/wiki/ZFS

ZFS seems to be the best match as ZFS is implemented on top of zpools
that allows you to share the underlying data store.

Jörg

-- 
 EMail:joerg@schily.isdn.cs.tu-berlin.de (home) Jörg Schilling D-13353 Berlin
       js@cs.tu-berlin.de                (uni)  
       schilling@fokus.fraunhofer.de     (work) Blog: http://schily.blogspot.com/
 URL:  http://cdrecord.berlios.de/private/ ftp://ftp.berlios.de/pub/schily

Re: [gentoo-user] [OT] filesystems

[Joerg Schilling]

Volker Armin Hemmann <volker.armin.hemmann@tu-clausthal.de> wrote:

> he is not - but after the invention is implemented, the inventor is not needed 
> anymore ;)

I hope this is not the reason for putting him into prison ;-)

Note the sign at the Springfield prison:

"If you commited murder, you'd be home by now."


> btrfs looks very promising. I hope it will become a good fs. Fast for 
> everybody, stable, efficient. We will see. Until then I will stay with 
> r4+compression.

Well, it is under a restrictive license, so there is no chance that this 
filestem will become popular on many OS platforms.

Jörg

-- 
 EMail:joerg@schily.isdn.cs.tu-berlin.de (home) Jörg Schilling D-13353 Berlin
       js@cs.tu-berlin.de                (uni)  
       schilling@fokus.fraunhofer.de     (work) Blog: http://schily.blogspot.com/
 URL:  http://cdrecord.berlios.de/private/ ftp://ftp.berlios.de/pub/schily

Re: [gentoo-user] [OT] filesystems

[Dale]

Volker Armin Hemmann wrote:
> On Montag 24 November 2008, Dirk Heinrichs wrote:
>   
>> Am Montag, 24. November 2008 13:49:38 schrieb Volker Armin Hemmann:
>>     
>>> On Montag 24 November 2008, Dirk Heinrichs wrote:
>>>       
>>>> Am Montag, 24. November 2008 12:07:34 schrieb Dale:
>>>>         
>>>>> Maybe it will survive.  I'm waiting on reiserfs4 to go stable.  ;-)
>>>>>           
>>>> Well, with its inventor being imprisoned for the next 15 years or so,
>>>> you'll have to be patient. I for one wait for btrfs.
>>>>         
>>> Edward is not imprisioned and doing a fine job. Even in face of such
>>> current sabotage attempts as by Morton/Piggin.
>>>       
>> Is he the inventor? AFAIK he's (one of) the last remaining developer(s).
>> However, btrfs also seems to be the favourite of many kernel hackers as
>> they want to have a ZFS competitor.
>>
>>     
>
> he is not - but after the invention is implemented, the inventor is not needed 
> anymore ;)
>
> btrfs looks very promising. I hope it will become a good fs. Fast for 
> everybody, stable, efficient. We will see. Until then I will stay with 
> r4+compression.
>
>
>   
It has been a while but I heard some people was working on it.  I know
about the inventors legal issues but that doesn't mean someone else
can't pick up where he left off.  I'm currently using reiserfs and love
the heck out of it.  I'm not real big on ext.  I wouldn't use XFS unless
it was all that was left.  I tried it once a while back and found out it
does not like power failures at all.  Each time I had a power failure, I
had to reinstall from scratch. 

Here's to hoping.

Dale

:-)  :-) 

Re: [gentoo-user] [OT] filesystems

[Nicolas Sebrecht]

On Mon, Nov 24, 2008 at 04:41:14PM +0100, Joerg Schilling wrote:

> > btrfs looks very promising. I hope it will become a good fs. Fast for 
> > everybody, stable, efficient. We will see. Until then I will stay with 
> > r4+compression.
> 
> Well, it is under a restrictive license, so there is no chance that this 
> filestem will become popular on many OS platforms.

btrfs is under GPL...
http://git.kernel.org/?p=linux/kernel/git/dwmw2/btrfs-kernel-unstable.git;a=blob;f=COPYING;h=ca442d313d86dc67e0a2e5d584b465bd382cbf5c;hb=e0dfd0d76e9205a542222f04c07072814c0ab282

-- 
Nicolas Sebrecht

Re: [gentoo-user] [OT] filesystems

[Alan McKinnon]

On Monday 24 November 2008 23:47:14 Nicolas Sebrecht wrote:
> On Mon, Nov 24, 2008 at 04:41:14PM +0100, Joerg Schilling wrote:
> > > btrfs looks very promising. I hope it will become a good fs. Fast for
> > > everybody, stable, efficient. We will see. Until then I will stay with
> > > r4+compression.
> >
> > Well, it is under a restrictive license, so there is no chance that this
> > filestem will become popular on many OS platforms.
>
> btrfs is under GPL...
> http://git.kernel.org/?p=linux/kernel/git/dwmw2/btrfs-kernel-unstable.git;a
>=blob;f=COPYING;h=ca442d313d86dc67e0a2e5d584b465bd382cbf5c;hb=e0dfd0d76e9205
>a542222f04c07072814c0ab282

That's Joerg's point. GPL is restrictive when compared to other OSS licenses. 
As a filesystem it pretty much goes into a kernel. It's an original work, so 
can only go into other kernels under the GPL. Effectively the only one that 
can work for is Linux.

Joerg isn't a Linux man, he codes for other platforms too. His viewpoint from 
what he's posted in the post is usually something like "can this be used on 
other systems too?"

For btrfs the answer is unfortunately "not really"

-- 
alan dot mckinnon at gmail dot com

Re: [gentoo-user] [OT] filesystems

[Volker Armin Hemmann]

On Montag 24 November 2008, Nicolas Sebrecht wrote:
> On Mon, Nov 24, 2008 at 04:41:14PM +0100, Joerg Schilling wrote:
> > > btrfs looks very promising. I hope it will become a good fs. Fast for
> > > everybody, stable, efficient. We will see. Until then I will stay with
> > > r4+compression.
> >
> > Well, it is under a restrictive license, so there is no chance that this
> > filestem will become popular on many OS platforms.
>
> btrfs is under GPL...

you can stop right here. Jörg thinks that the GPL is restrictive and the CPPL 
much more 'freedomy'. Don't try to argue. It will result in some flamefest.

Re: [gentoo-user] [OT] filesystems

[Alan McKinnon]

On Tuesday 25 November 2008 00:15:55 Volker Armin Hemmann wrote:
> On Montag 24 November 2008, Nicolas Sebrecht wrote:
> > On Mon, Nov 24, 2008 at 04:41:14PM +0100, Joerg Schilling wrote:
> > > > btrfs looks very promising. I hope it will become a good fs. Fast for
> > > > everybody, stable, efficient. We will see. Until then I will stay
> > > > with r4+compression.
> > >
> > > Well, it is under a restrictive license, so there is no chance that
> > > this filestem will become popular on many OS platforms.
> >
> > btrfs is under GPL...
>
> you can stop right here. Jörg thinks that the GPL is restrictive and the
> CPPL much more 'freedomy'. Don't try to argue. It will result in some
> flamefest.

I dunno about that. About the flamefest I mean. For the past 6 months Joerg 
has been a decent helpful member around here. He answers up every time his 
code is involved, doesn't rise to the bait with the occasional dumb user 
question and is mostly your typical geek with straight answers - with a bit 
of slack cut because he's not native English speaking.

It wasn't always like that, but I think we should acknowledge things that 
change for the better.

-- 
alan dot mckinnon at gmail dot com

Re: [gentoo-user] [OT] filesystems

[Volker Armin Hemmann]

On Montag 24 November 2008, Alan McKinnon wrote:
> On Tuesday 25 November 2008 00:15:55 Volker Armin Hemmann wrote:
> > On Montag 24 November 2008, Nicolas Sebrecht wrote:
> > > On Mon, Nov 24, 2008 at 04:41:14PM +0100, Joerg Schilling wrote:
> > > > > btrfs looks very promising. I hope it will become a good fs. Fast
> > > > > for everybody, stable, efficient. We will see. Until then I will
> > > > > stay with r4+compression.
> > > >
> > > > Well, it is under a restrictive license, so there is no chance that
> > > > this filestem will become popular on many OS platforms.
> > >
> > > btrfs is under GPL...
> >
> > you can stop right here. Jörg thinks that the GPL is restrictive and the
> > CPPL much more 'freedomy'. Don't try to argue. It will result in some
> > flamefest.
>
> I dunno about that. About the flamefest I mean. For the past 6 months Joerg
> has been a decent helpful member around here. He answers up every time his
> code is involved, doesn't rise to the bait with the occasional dumb user
> question and is mostly your typical geek with straight answers - with a bit
> of slack cut because he's not native English speaking.
>
> It wasn't always like that, but I think we should acknowledge things that
> change for the better.

I am not saying that it is Jörg's fault. Just saying that arguing will end in 
a flame fest. I have seen him writing about the GPL and his more favorite 
licences before - nothing Nicolas or anybody else says will change his mind. 
And nothing he will say will change the mind of the GPL fans. 
So there will be some clash of egos and a big, fat flame war, each side 
convinced to speak the ultimate truth. No thanks.

[gentoo-user] Re: [OT] filesystems

[»Q«]

On Tue, 25 Nov 2008 00:41:51 +0100
Volker Armin Hemmann <volker.armin.hemmann@tu-clausthal.de> wrote:

> On Montag 24 November 2008, Alan McKinnon wrote:
> > On Tuesday 25 November 2008 00:15:55 Volker Armin Hemmann wrote:
> > > On Montag 24 November 2008, Nicolas Sebrecht wrote:
> > > > On Mon, Nov 24, 2008 at 04:41:14PM +0100, Joerg Schilling wrote:
> > > > > > btrfs looks very promising. I hope it will become a good
> > > > > > fs. Fast for everybody, stable, efficient. We will see.
> > > > > > Until then I will stay with r4+compression.
> > > > >
> > > > > Well, it is under a restrictive license, so there is no
> > > > > chance that this filestem will become popular on many OS
> > > > > platforms.
> > > >
> > > > btrfs is under GPL...
> > >
> > > you can stop right here. Jörg thinks that the GPL is restrictive
> > > and the CPPL much more 'freedomy'. Don't try to argue. It will
> > > result in some flamefest.
> >
> > I dunno about that. About the flamefest I mean. For the past 6
> > months Joerg has been a decent helpful member around here. He
> > answers up every time his code is involved, doesn't rise to the
> > bait with the occasional dumb user question and is mostly your
> > typical geek with straight answers - with a bit of slack cut
> > because he's not native English speaking.
> >
> > It wasn't always like that, but I think we should acknowledge
> > things that change for the better.
> 
> I am not saying that it is Jörg's fault. Just saying that arguing
> will end in a flame fest. I have seen him writing about the GPL and
> his more favorite licences before - nothing Nicolas or anybody else
> says will change his mind. And nothing he will say will change the
> mind of the GPL fans. So there will be some clash of egos and a big,
> fat flame war, each side convinced to speak the ultimate truth. No
> thanks.

Also, no one reading it would learn anything more than they could have
by googling a little while.  All the arguments about whether copyleft is
more or less freedomish than non-copyleft are already out there for
anyone who wants to read them. 

-- 
»Q«
     Kleeneness is next to Gödelness.

Re: [gentoo-user] [OT] filesystems

[Dirk Heinrichs]

Am Montag, 24. November 2008 22:09:52 schrieb Dale:

> I wouldn't use XFS unless
> it was all that was left.  I tried it once a while back and found out it
> does not like power failures at all.  Each time I had a power failure, I
> had to reinstall from scratch.

Hmm, I use it because of its resistance to power failures. When was it that 
you had such problems?

Bye...

	Dirk

Re: [gentoo-user] [OT] filesystems

[Dale]

Dirk Heinrichs wrote:
> Am Montag, 24. November 2008 22:09:52 schrieb Dale:
>
>   
>> I wouldn't use XFS unless
>> it was all that was left.  I tried it once a while back and found out it
>> does not like power failures at all.  Each time I had a power failure, I
>> had to reinstall from scratch.
>>     
>
> Hmm, I use it because of its resistance to power failures. When was it that 
> you had such problems?
>
> Bye...
>
> 	Dirk
>
>
>   

Its been a while but it happened several times.  I just got tired of
having to reinstall every time the power blinked.  Turned out the wire
was loose on the transformer so they blinked a lot, every couple days or
so.  I think it was Mandrake 9.2.

I have had a power failure or two with reiserfs and it recovered.  It
did the check thing but ran fine.

Just my experience.  Your mileage may vary.

Dale

:-)  :- 

Re: [gentoo-user] [OT] filesystems

[Joerg Schilling]

Nicolas Sebrecht <nicolas.s-dev@laposte.net> wrote:

>
> On Mon, Nov 24, 2008 at 04:41:14PM +0100, Joerg Schilling wrote:
>
> > > btrfs looks very promising. I hope it will become a good fs. Fast for 
> > > everybody, stable, efficient. We will see. Until then I will stay with 
> > > r4+compression.
> > 
> > Well, it is under a restrictive license, so there is no chance that this 
> > filestem will become popular on many OS platforms.
>
> btrfs is under GPL...

Correct, and this is the reason why it cannot appear on other platforms.

The problem with the GPL is that it tries to prevent collaboration between 
different license camps. The GPL is an asymmetric license that allows "other
code" to be used by GPLd code (this is why ZFS being under CDDL is no problem
for a linux integration), but it does not allow other code (even code unter an 
approved OpenSource license) to use GPLd code.

It ZFS was under GPL, it did not appear on FreeBSD and Mac OS X.

What I expect from a promising new filesystem is that is may be integrated in a 
large variety of Platforms.

Note that I am a supporter of collaboration in OSS and that it is important for 
me to write software in a highly portable way so anybody may use it....

I do not like the camp mentality.

Jörg

-- 
 EMail:joerg@schily.isdn.cs.tu-berlin.de (home) Jörg Schilling D-13353 Berlin
       js@cs.tu-berlin.de                (uni)  
       schilling@fokus.fraunhofer.de     (work) Blog: http://schily.blogspot.com/
 URL:  http://cdrecord.berlios.de/private/ ftp://ftp.berlios.de/pub/schily

Re: [gentoo-user] [OT] filesystems

[Joerg Schilling]

Volker Armin Hemmann <volker.armin.hemmann@tu-clausthal.de> wrote:

> > > Well, it is under a restrictive license, so there is no chance that this
> > > filestem will become popular on many OS platforms.
> >
> > btrfs is under GPL...
>
> you can stop right here. Jörg thinks that the GPL is restrictive and the CPPL 
> much more 'freedomy'. Don't try to argue. It will result in some flamefest.

If everybody uses arguments, there will be no flamefest.....

Jörg

-- 
 EMail:joerg@schily.isdn.cs.tu-berlin.de (home) Jörg Schilling D-13353 Berlin
       js@cs.tu-berlin.de                (uni)  
       schilling@fokus.fraunhofer.de     (work) Blog: http://schily.blogspot.com/
 URL:  http://cdrecord.berlios.de/private/ ftp://ftp.berlios.de/pub/schily

Re: [gentoo-user] [OT] filesystems

[Daniel Troeder]

[-- Attachment #1: Type: text/plain, Size: 1059 bytes --]

Am Montag, den 24.11.2008, 16:12 +0200 schrieb GMail:
> On Monday 24 November 2008 08:28:33 Dirk Heinrichs wrote:
> > @William: If one or more of the PVs is a Network Block Device, you're not
> > bound to the local machine.
You could also use iSCSI. On your client you'll get SCSI-device-nodes
(/dev/sdx) which you can use as PVs. It's very fast and reliable.

> How does it cope with network outages though? In my experience, LVM is not 
> exactly graceful when one of it's PVs goes away
My experience too :(
After reconnecting to the iSCSI-target I have to

# vgchange -a n vg-iscsi
# vgchange -a y vg-iscsi

to get my devices in /dev/vg-iscsi/* to work again (during disconnection
they keep existing, but are not usable with misleading error messages).

When using iSCSI-devices you should use /dev/disk/by-path/<taget-id> and
not /dev/sdx, as these device names can change.

Bye,
Daniel

-- 
PGP key: http://pgpkeys.pca.dfn.de/pks/lookup?search=0xBB9D4887&op=get
# gpg --recv-keys --keyserver hkp://subkeys.pgp.net 0xBB9D4887


[-- Attachment #2: Dies ist ein digital signierter Nachrichtenteil --]
[-- Type: application/pgp-signature, Size: 197 bytes --]

Re: [gentoo-user] [OT] filesystems

[Alan McKinnon]

On Tuesday 25 November 2008 11:07:26 Joerg Schilling wrote:
> It ZFS was under GPL, it did not appear on FreeBSD and Mac OS X.
>
> What I expect from a promising new filesystem is that is may be integrated
> in a large variety of Platforms.
>
> Note that I am a supporter of collaboration in OSS and that it is important
> for me to write software in a highly portable way so anybody may use it....
>
> I do not like the camp mentality.

I used to be a totally-GPL fan but I changed my stance a few years back. The 
thing that did it for me was the TCP/IP stack. If this were not BSD licensed, 
it would not have been adopted as widely as it was, and we would not have an 
internet today. So keeping in mind that the GPL was designed to be used to 
create a free-standing body of free code that comprised an entire Unix-like 
system, I now advocate the following:

Low level code that is intended to be used everywhere - on the order of 
filesystems, networking standards, block devices and such - ideally should be 
BSD licensed. Then anyone anywhere can use it. 

GPL in userland is fine, as apps tend to be free-standing and do not conflict 
with other code, hence the "mere-aggregation" clause.

Kernel modules are different and cannot work this way. Expect in very unusual 
circumstances (eg XFS in the linux kernel) they are derivative works and the 
GPL kicks in. Which is fine, most people will contribute their changes back 
upstream anyway just like GPL demands. But GPL is incompatible with other 
licenses which prohibits equal two-way sharing. The easiest possible solution 
as I see it is to just license this low-level utility code as BSD.

-- 
alan dot mckinnon at gmail dot com

Re: [gentoo-user] Re: [OT] filesystems

[Joerg Schilling]

»Q« <boxcars@gmx.net> wrote:

> Also, no one reading it would learn anything more than they could have
> by googling a little while.  All the arguments about whether copyleft is
> more or less freedomish than non-copyleft are already out there for
> anyone who wants to read them. 

Speaking about the filesystems, the GPL limits the freedom to use the code and 
for this reason, brtfs cannot be used by Solaris, FreeBSD, Mac OS X, .....

As a rule of thumb: of you like to widely spread an implementation don't 
license it under GPL.

Jörg

-- 
 EMail:joerg@schily.isdn.cs.tu-berlin.de (home) Jörg Schilling D-13353 Berlin
       js@cs.tu-berlin.de                (uni)  
       schilling@fokus.fraunhofer.de     (work) Blog: http://schily.blogspot.com/
 URL:  http://cdrecord.berlios.de/private/ ftp://ftp.berlios.de/pub/schily

Re: [gentoo-user] Re: [OT] filesystems

[Volker Armin Hemmann]

On Tuesday 25 November 2008, Joerg Schilling wrote:
> »Q« <boxcars@gmx.net> wrote:
> > Also, no one reading it would learn anything more than they could have
> > by googling a little while.  All the arguments about whether copyleft is
> > more or less freedomish than non-copyleft are already out there for
> > anyone who wants to read them.
>
> Speaking about the filesystems, the GPL limits the freedom to use the code
> and for this reason, brtfs cannot be used by Solaris, FreeBSD, Mac OS X,
> .....
>
> As a rule of thumb: of you like to widely spread an implementation don't
> license it under GPL.
>
> Jörg

or dual licence it. But Sun didn't dual licence their ZFS code either so ...

Re: [gentoo-user] Re: [OT] filesystems

[Joerg Schilling]

Volker Armin Hemmann <volker.armin.hemmann@tu-clausthal.de> wrote:

> > As a rule of thumb: of you like to widely spread an implementation don't
> > license it under GPL.
> >
> > Jörg
>
> or dual licence it. But Sun didn't dual licence their ZFS code either so ...

The problem with dual licensing is that you cannot ensure that everybody 
publishes enhancements that are also dual licensed. Bad people will publish
extensions under one license only and thus make the enhancement unusable for 
others.

For this reason it seems that dual licensing is not the solution.

Jörg

-- 
 EMail:joerg@schily.isdn.cs.tu-berlin.de (home) Jörg Schilling D-13353 Berlin
       js@cs.tu-berlin.de                (uni)  
       schilling@fokus.fraunhofer.de     (work) Blog: http://schily.blogspot.com/
 URL:  http://cdrecord.berlios.de/private/ ftp://ftp.berlios.de/pub/schily

Re: [gentoo-user] [OT] filesystems

[Paul Hartman]

On Tue, Nov 25, 2008 at 2:27 AM, Dale <rdalek1967@gmail.com> wrote:
> Dirk Heinrichs wrote:
>> Am Montag, 24. November 2008 22:09:52 schrieb Dale:
>>
>>
>>> I wouldn't use XFS unless
>>> it was all that was left.  I tried it once a while back and found out it
>>> does not like power failures at all.  Each time I had a power failure, I
>>> had to reinstall from scratch.
>>>
>>
>> Hmm, I use it because of its resistance to power failures. When was it that
>> you had such problems?
>>
>> Bye...
>>
>>       Dirk
>>
>>
>>
>
> Its been a while but it happened several times.  I just got tired of
> having to reinstall every time the power blinked.  Turned out the wire
> was loose on the transformer so they blinked a lot, every couple days or
> so.  I think it was Mandrake 9.2.
>
> I have had a power failure or two with reiserfs and it recovered.  It
> did the check thing but ran fine.
>
> Just my experience.  Your mileage may vary.
>
> Dale
>
> :-)  :-

I have a similar story, but for me it was JFS instead of XFS. I will
never, ever, ever use JFS for anything again. I had XFS on a file
server RAID box with a failing power supply and it died over and over
and the FS stayed functional, so YMMV indeed. (I haven't tried reiser,
I'm still scared about the corruption stories from years ago.)

I suppose if you ask enough people, there will be horror stories about
every filesystem.

No matter which FS you choose, I wish you good luck and hope you have
no new horror stories. :)

Paul

Re: [gentoo-user] [OT] filesystems

[Jorge Peixoto de Morais Neto]

>>>> I wouldn't use XFS unless
>>>> it was all that was left.  I tried it once a while back and found out it
>>>> does not like power failures at all.  Each time I had a power failure, I
>>>> had to reinstall from scratch.
>>> Hmm, I use it because of its resistance to power failures. When was it that
>>> you had such problems?
>> Its been a while but it happened several times.  I just got tired of
>> having to reinstall every time the power blinked.  Turned out the wire
>> was loose on the transformer so they blinked a lot, every couple days or
>> so.  I think it was Mandrake 9.2.
>>
>> I have had a power failure or two with reiserfs and it recovered.  It
>> did the check thing but ran fine.
>>
>> Just my experience.  Your mileage may vary.
> I have a similar story, but for me it was JFS instead of XFS. I will
> never, ever, ever use JFS for anything again. I had XFS on a file
> server RAID box with a failing power supply and it died over and over
> and the FS stayed functional, so YMMV indeed. (I haven't tried reiser,
> I'm still scared about the corruption stories from years ago.)
>
> I suppose if you ask enough people, there will be horror stories about
> every filesystem.
I use reiserfs and I twice got serious filesystem corruptions after
crashes, and one was very serious. It is unclear whether this was
reiserfs's fault or the hardware. You see, I was using athcool to save
electricity, and it seems that when the bit "Disconnect enable when
STPGNT detected" is set on the Northbridge (this is what athcool does)
and you are using a PixelView PV-M4900 FM.RC (specially if you are
recording tv - with mencoder - as opposed to just viewing it - with
mplayer), your computer malfunctions.
I was able to recover much of the data with reiserfsck --rebuild-tree,
but some of the files had part of their content replaced with a string
of null bytes. I heard somewhere that reiserfs is infamous for
replacing file content with a string of null bytes, so maybe this is
indeed reiserfs fault, and not just bad hardware.

By the way, I chose reiserfs (some 3 years ago I believe) because of
its speed fame, but now, thinking of it, there are only four computer
activities that make my system slow:
1) launch heavy programs such as firefox (when not in cache)
2) compile software
3) view certain web pages in firefox
4) encode video

Now, since I usually compile software in a tmpfs, I guess the
filesystem makes nearly zero difference. Video encoding is obviously
bound by CPU, cache and RAM speed, not filesystem. Web rendering is
also hardly affected by filesystem . And launching programs means
mostly reading files, and would reiserfs be significantly faster than
ext3 for this, specially considering that my system is minimalist and
the root partition is only 7% used?

So it seems I should not have chosen reiserfs, which has a fame of
being less safe than ext3, and certainly has less software support
than ext3. The next time I format my root partition, I will choose
ext3 (then move to ext4 when it is stable).

-- 
Software is like sex: it is better when it is free - Linus Torvalds

Re: [gentoo-user] [OT] filesystems

[Jorge Peixoto de Morais Neto]

> Now, since I usually compile software in a tmpfs, I guess the
> filesystem makes nearly zero difference. Video encoding is obviously
> bound by CPU, cache and RAM speed, not filesystem. Web rendering is
> also hardly affected by filesystem . And launching programs means
> mostly reading files, and would reiserfs be significantly faster than
> ext3 for this, specially considering that my system is minimalist and
> the root partition is only 7% used?
>
> So it seems I should not have chosen reiserfs, which has a fame of
> being less safe than ext3, and certainly has less software support
> than ext3. The next time I format my root partition, I will choose
> ext3 (then move to ext4 when it is stable).

Oh, and according to this benchmark
http://linuxgazette.net/122/piszcz.html
reiserfs does not deserve its speed fame.

Re: [gentoo-user] [OT] filesystems

[Volker Armin Hemmann]

On Tuesday 25 November 2008, Paul Hartman wrote:
> On Tue, Nov 25, 2008 at 2:27 AM, Dale <rdalek1967@gmail.com> wrote:
> > Dirk Heinrichs wrote:
> >> Am Montag, 24. November 2008 22:09:52 schrieb Dale:
> >>> I wouldn't use XFS unless
> >>> it was all that was left.  I tried it once a while back and found out
> >>> it does not like power failures at all.  Each time I had a power
> >>> failure, I had to reinstall from scratch.
> >>
> >> Hmm, I use it because of its resistance to power failures. When was it
> >> that you had such problems?
> >>
> >> Bye...
> >>
> >>       Dirk
> >
> > Its been a while but it happened several times.  I just got tired of
> > having to reinstall every time the power blinked.  Turned out the wire
> > was loose on the transformer so they blinked a lot, every couple days or
> > so.  I think it was Mandrake 9.2.
> >
> > I have had a power failure or two with reiserfs and it recovered.  It
> > did the check thing but ran fine.
> >
> > Just my experience.  Your mileage may vary.
> >
> > Dale
> >
> > :-)  :-
>
> I have a similar story, but for me it was JFS instead of XFS. I will
> never, ever, ever use JFS for anything again. I had XFS on a file
> server RAID box with a failing power supply and it died over and over
> and the FS stayed functional, so YMMV indeed. (I haven't tried reiser,
> I'm still scared about the corruption stories from years ago.)

the corruption stories were caused by vm changes that were not tested against 
reiserfs. Thank R.v.Riel, Andrea Arcangeli and of course Linus Torvalds for 
that mess.

Re: [gentoo-user] [OT] filesystems

[Volker Armin Hemmann]

On Tuesday 25 November 2008, Jorge Peixoto de Morais Neto wrote:

> I was able to recover much of the data with reiserfsck --rebuild-tree,
> but some of the files had part of their content replaced with a string
> of null bytes. I heard somewhere that reiserfs is infamous for
> replacing file content with a string of null bytes, so maybe this is
> indeed reiserfs fault, and not just bad hardware.

no, that is xfs.

> So it seems I should not have chosen reiserfs, which has a fame of
> being less safe than ext3, and certainly has less software support
> than ext3. The next time I format my root partition, I will choose
> ext3 (then move to ext4 when it is stable).

reiserfs has barriers turned on by default - which makes it a bit slower but a 
lot safer for data. ext3 has them turned off by default - ext3 devs don't care 
about data - only speed. You turn on barriers, performance goes down by 30%.

Re: [gentoo-user] [OT] filesystems

[Jorge Peixoto de Morais Neto]

> reiserfs has barriers turned on by default - which makes it a bit slower but a
> lot safer for data. ext3 has them turned off by default - ext3 devs don't care
> about data - only speed. You turn on barriers, performance goes down by 30%.
I read an article about that, and if I recall correctly the assumption
was that the likelihood of data loss occurring due to the barriers
issue was negligible. I have no expertise to decide on that matter,
but the fact that pretty much every linux distribution chooses ext3 by
default suggests it is the safest (at least for simple desktop/laptop
usage), no?

Somewhat offtopic:
What do you suggest for me? I care about data safety, but am too lazy
to make frequent backups, so filesystem robustness and availability of
data recovery tools is pretty important; and as I said before, the
only performance problem with my computer that I think may be related
to filesystem is boot time and launching heavy programs not in cache;
keep in mind my root partition is only 3,8 GB used and 93% free -
maybe in this condition the filesystem is not stressed and only the
actual HD speed matters? Valerie Henson from VAH Consulting says that
every file system goes fast with:

* O(1000) files per directory
* File size a few KB to a few GB
* Read-mostly access
* Infrequent file creation/deletion
* Sequential file read/write patterns
* Shallow directory depth (< 10 levels)
* Total file system size O(100 GB)

Re: [gentoo-user] [OT] filesystems

[Volker Armin Hemmann]

On Tuesday 25 November 2008, Jorge Peixoto de Morais Neto wrote:
> > Now, since I usually compile software in a tmpfs, I guess the
> > filesystem makes nearly zero difference. Video encoding is obviously
> > bound by CPU, cache and RAM speed, not filesystem. Web rendering is
> > also hardly affected by filesystem . And launching programs means
> > mostly reading files, and would reiserfs be significantly faster than
> > ext3 for this, specially considering that my system is minimalist and
> > the root partition is only 7% used?
> >
> > So it seems I should not have chosen reiserfs, which has a fame of
> > being less safe than ext3, and certainly has less software support
> > than ext3. The next time I format my root partition, I will choose
> > ext3 (then move to ext4 when it is stable).
>
> Oh, and according to this benchmark
> http://linuxgazette.net/122/piszcz.html
> reiserfs does not deserve its speed fame.

they tested crap.

As I wrote in the other mail. XFS and reiserfs turn on barriers by default, 
ext3 turns them off.
With barriers on for ext3 it looses 30%(!). reiserfs and xfs don't suffer as 
much, but suffer they do. So if the test did not turn on or off barriers for 
all fs who support them, ext3 had an unfair advantage.

And you want barriers. 

Re: [gentoo-user] [OT] filesystems

[KH]

Jorge Peixoto de Morais Neto schrieb:
> I have no expertise to decide on that matter,
> but the fact that pretty much every linux distribution chooses ext3 by
> default suggests it is the safest (at least for simple desktop/laptop
> usage), no?
>   

Most people and companies / organisations use M$ Windows. Would you say
that this is saver than your Linux? You are outnumbered for sure ;-)

Re: [gentoo-user] [OT] filesystems

[Volker Armin Hemmann]

On Tuesday 25 November 2008, Jorge Peixoto de Morais Neto wrote:
> > reiserfs has barriers turned on by default - which makes it a bit slower
> > but a lot safer for data. ext3 has them turned off by default - ext3 devs
> > don't care about data - only speed. You turn on barriers, performance
> > goes down by 30%.
>
> I read an article about that, and if I recall correctly the assumption
> was that the likelihood of data loss occurring due to the barriers
> issue was negligible. I have no expertise to decide on that matter,
> but the fact that pretty much every linux distribution chooses ext3 by
> default suggests it is the safest (at least for simple desktop/laptop
> usage), no?

fedora turns on 4k stack - well knowing that it kills xfs. Do you want to 
rephrase your question?

>
> Somewhat offtopic:
> What do you suggest for me? I care about data safety, but am too lazy
> to make frequent backups, so filesystem robustness and availability of
> data recovery tools is pretty important;

so use whatever you want, get a nice cheap dlt from ebay and let a cronjob 
write to it. No 'lazy' problem. Very secure.

Re: [gentoo-user] [OT] filesystems

[Jorge Peixoto de Morais Neto]

>> [...] I have no expertise to decide on that matter,
>> but the fact that pretty much every linux distribution chooses ext3 by
>> default suggests it is the safest (at least for simple desktop/laptop
>> usage), no?
>
> fedora turns on 4k stack - well knowing that it kills xfs. Do you want to
> rephrase your question?
Well, I said I have little expertise. Won't argue.
>>
>> Somewhat offtopic:
>> What do you suggest for me? I care about data safety, but am too lazy
>> to make frequent backups, so filesystem robustness and availability of
>> data recovery tools is pretty important;
>
> so use whatever you want, get a nice cheap dlt from ebay and let a cronjob
> write to it. No 'lazy' problem. Very secure.

I live in Brasil, and due to huge taxes, poor infrastructure and the
currency exchange ratio, computer stuff is far more expensive than in
the US. And then you have to factor that the average Brazilian is much
poorer than the average US citizen.

But anyway, I know I must make backups, but I still want a robust
filesystem with good software support (such as data recovery
utilities). Could you give me your suggestion for the safest
filesystem for a desktop user that only uses 3,8G of his 54G root
partition? I care about speed, but I think that my usage pattern does
not stress the filesystem (if what Valerie Henson says is true).

Re: [gentoo-user] [OT] filesystems

[Alan McKinnon]

On Tuesday 25 November 2008 19:57:19 Paul Hartman wrote:
> I have a similar story, but for me it was JFS instead of XFS. I will
> never, ever, ever use JFS for anything again. I had XFS on a file
> server RAID box with a failing power supply and it died over and over
> and the FS stayed functional, so YMMV indeed. (I haven't tried reiser,
> I'm still scared about the corruption stories from years ago.)

Sounds like you used JFS in a case it was not designed for. XFS for instance 
can be best described as "a filesystem that does aggressive caching, so if 
you install it you need to guarantee that it will never lose power, i.e. use 
a UPS". It's OK for SGI to have done this, considering the kind of rendering 
clusters they were running it on. Use it outside that viewpoint and hey, 
JMMV. JFS will have it's own specific "best use" scenario

The reiser stories are just that, horror stories from years ago. Then it was 
beta software, it is not beta any more. I've used it for over 4 years now on 
every machine I have and suffered no data loss that was not directly because 
of me being stupid. I don't think I can blame Hans if I run fsck with the 
wrong options at the wrong time :-)

> I suppose if you ask enough people, there will be horror stories about
> every filesystem.

yes, very much so. Much more so than for any other kind of driver by my 
experience.

-- 
alan dot mckinnon at gmail dot com

Re: [gentoo-user] [OT] filesystems

[Alan McKinnon]

On Tuesday 25 November 2008 20:37:13 Jorge Peixoto de Morais Neto wrote:
> Now, since I usually compile software in a tmpfs, I guess the
> filesystem makes nearly zero difference. Video encoding is obviously
> bound by CPU, cache and RAM speed, not filesystem. Web rendering is
> also hardly affected by filesystem . And launching programs means
> mostly reading files, and would reiserfs be significantly faster than
> ext3 for this, specially considering that my system is minimalist and
> the root partition is only 7% used?

I find that in normal use, most filesystems have a large range of number of 
files per directory and the spread of how big those files are. In other 
words, a huge mixture of everything.

reiser and ext both have areas they are very good at but in normal use the 
good and bad performance evens out so you get roughly the same with both 
filesystems. The deciding factor then becomes "which filesystem tools are you 
most comfortable with?" because that's the one you should be using.

There are special cases - if the portage tree is on it's own filesystem, ext3 
does give better performance. 

> So it seems I should not have chosen reiserfs, which has a fame of
> being less safe than ext3, and certainly has less software support
> than ext3. The next time I format my root partition, I will choose
> ext3 (then move to ext4 when it is stable).

As I said in another post, I don't believe that either reiser or ext3 is 
inherently more or less safe than the other. Your upgrade path to ext4 does 
change things, so yeah, you have a perfectly valid reason to switch to ext3 
right away

-- 
alan dot mckinnon at gmail dot com

Re: [gentoo-user] [OT] filesystems

[Alan McKinnon]

On Tuesday 25 November 2008 21:24:48 Jorge Peixoto de Morais Neto wrote:
> I have no expertise to decide on that matter,
> but the fact that pretty much every linux distribution chooses ext3 by
> default suggests it is the safest (at least for simple desktop/laptop
> usage), no?

I don't think that has anything to do with performance or safety. Instead:

1. Red Hat suffers from a serious case of Not Invented Here Syndrome. They do 
good work, but they have that little eccentricity too. ReiserFS was funded in 
large part by SuSE, therefore RH are ill-inclined to use it. Many distros 
follow Red Hat's lead, very few go with SuSE to wherever SuSE is going. Who 
knows why Debian made their choice - it' s probably as simple as ext3 traces 
it roots back much further than Reiser can

2. NameSys was largely driven by the fame (infamy?) of it's owner - a typical 
mad scientist geek who writes excellent code. But he got himself in jail and 
the risk associated with using his filesystem sans reliable know maintainer 
is too great a risk for most distros

-- 
alan dot mckinnon at gmail dot com

Re: [gentoo-user] [OT] filesystems

[Volker Armin Hemmann]

On Dienstag 25 November 2008, Jorge Peixoto de Morais Neto wrote:

> > so use whatever you want, get a nice cheap dlt from ebay and let a
> > cronjob write to it. No 'lazy' problem. Very secure.
>
> I live in Brasil, and due to huge taxes, poor infrastructure and the
> currency exchange ratio, computer stuff is far more expensive than in
> the US. 

it is more expensive in europe too ;)

> And then you have to factor that the average Brazilian is much
> poorer than the average US citizen.

and because of that I talked about dlt. A nice, used dlt 35/70 will work for 
another couple of years, is not very expensive (anymore), and very robust.

>
> But anyway, I know I must make backups, but I still want a robust
> filesystem with good software support (such as data recovery
> utilities). Could you give me your suggestion for the safest
> filesystem for a desktop user that only uses 3,8G of his 54G root
> partition? I care about speed, but I think that my usage pattern does
> not stress the filesystem (if what Valerie Henson says is true).

xfs, reiserfs, ext3 all work fine. I would stay away from xfs with unstable 
electricity. I would also stay away from jfs, because almost nobody uses it.

I have used reiserfs in the past, I am using reiser4 now. But I don't 
recommend r4. It is working great for ME. But that doesn't mean that it is the 
right choice for anybody else.

[gentoo-user] Re: [OT] filesystems

[James]

Jorge Peixoto de Morais Neto <please.no.spam.here <at> gmail.com> writes:


> But anyway, I know I must make backups, but I still want a robust
> filesystem with good software support (such as data recovery
> utilities). Could you give me your suggestion for the safest
> filesystem for a desktop user that only uses 3,8G of his 54G root
> partition? I care about speed, but I think that my usage pattern does
> not stress the filesystem (if what Valerie Henson says is true).


reiserfs, Never lost anything.....I could not recover across dozens of
systems.....

ymmv


Backups.

get a usb stick and manually copy your stuff to it, periodically.

safe and cheap. Don't bother backing up the operating system as it
can be re-created. /etc is a good idea and a few other key files
 and your personal files....

ymmv.


James

Re: [gentoo-user] Re: [OT] filesystems

[Joerg Schilling]

James <wireless@tampabay.rr.com> wrote:

> Backups.
>
> get a usb stick and manually copy your stuff to it, periodically.


Where do you get these 1 TB USB sticks?

Jörg

-- 
 EMail:joerg@schily.isdn.cs.tu-berlin.de (home) Jörg Schilling D-13353 Berlin
       js@cs.tu-berlin.de                (uni)  
       schilling@fokus.fraunhofer.de     (work) Blog: http://schily.blogspot.com/
 URL:  http://cdrecord.berlios.de/private/ ftp://ftp.berlios.de/pub/schily

Re: [gentoo-user] Re: [OT] filesystems

[Alan McKinnon]

On Tuesday 25 November 2008 23:15:27 Joerg Schilling wrote:
> James <wireless@tampabay.rr.com> wrote:
> > Backups.
> >
> > get a usb stick and manually copy your stuff to it, periodically.
>
> Where do you get these 1 TB USB sticks?

To be fair, he was responding to a parent that asked about backing up 3.8G of 
data, so USB sticks are not unfeasible. Until they get lost that is.

Personally, I'm very interested in seeing where Intel go with their new SSDs. 
The reason is that I'm getting sick and tired of having to explain and 
justify why the laws of physics prevent my girlfriend from being able to 
backup her 5T banking warehouse on the schedule the law of the country would 
like her to. There's a solution, but it's hackish and ugly and involves 
extremely careful management of LVM snapshots. This is simply way too much 
admin effort for what should really be a simple incremental backup process. 
And let's not even get into what restores involve

Let's face it, disks only spin so fast and Moore's law does not apply to disk 
and tape speed. Vastly improving the first half - drive speed - is a huge 
first step in the right direction.

-- 
alan dot mckinnon at gmail dot com

Re: [gentoo-user] Re: [OT] filesystems

[Joerg Schilling]

Alan McKinnon <alan.mckinnon@gmail.com> wrote:

> On Tuesday 25 November 2008 23:15:27 Joerg Schilling wrote:
> > James <wireless@tampabay.rr.com> wrote:
> > > Backups.
> > >
> > > get a usb stick and manually copy your stuff to it, periodically.
> >
> > Where do you get these 1 TB USB sticks?
>
> To be fair, he was responding to a parent that asked about backing up 3.8G of 
> data, so USB sticks are not unfeasible. Until they get lost that is.
>
> Personally, I'm very interested in seeing where Intel go with their new SSDs. 

Two weeks ago, I did some tests with a SSD (ZFS and UFS) and it is really 
promising.

They still have "seek times" but much smaller than rotating media.
ZFS behaves (as expected) much better on SSD than UFS.
ZFS even supports (or will soon support) to add separate SSDs as write cache and 
as meta data cache.

Jörg

-- 
 EMail:joerg@schily.isdn.cs.tu-berlin.de (home) Jörg Schilling D-13353 Berlin
       js@cs.tu-berlin.de                (uni)  
       schilling@fokus.fraunhofer.de     (work) Blog: http://schily.blogspot.com/
 URL:  http://cdrecord.berlios.de/private/ ftp://ftp.berlios.de/pub/schily

Re: [gentoo-user] Re: [OT] filesystems

[Alan McKinnon]

On Tuesday 25 November 2008 23:43:34 Joerg Schilling wrote:
> Alan McKinnon <alan.mckinnon@gmail.com> wrote:

> > Personally, I'm very interested in seeing where Intel go with their new
> > SSDs.
>
> Two weeks ago, I did some tests with a SSD (ZFS and UFS) and it is really
> promising.

That's what Linux said on his blog too. They just worked and worked well and 
impressed him technically with no silly rotating media characteristics 
carried over to SSD. Which is a refreshing change indeed

> They still have "seek times" but much smaller than rotating media.
> ZFS behaves (as expected) much better on SSD than UFS.
> ZFS even supports (or will soon support) to add separate SSDs as write
> cache and as meta data cache.

I think I need to investigate this deeper and do a write-up for work. I have a 
FreeBSD-7 deployment system (nothing for OpenSolaris) so I'll use that. Any 
comment on ZFS performace/stability on FreeBSD?

-- 
alan dot mckinnon at gmail dot com

Re: [gentoo-user] [OT] filesystems

[William Kenworthy]

On Tue, 2008-11-25 at 17:24 -0200, Jorge Peixoto de Morais Neto wrote:
...
> but the fact that pretty much every linux distribution chooses ext3 by
> default suggests it is the safest (at least for simple desktop/laptop
> usage), no?
> 
...

No, for me ext2 = continual lost data issues from even the smallest
glitch.  I had (up to a couple of weeks ago ext2 on a freerunner phone -
almost daily data problems (a freerunner should be packed in foam - it
crashes 2-3 times a day if you use it!),  Since using ext3, the problems
are drasticly reduced but still occur ever few days.  Even VFAT has less
problems that ext2, but ext3 is a little better.  Note this is using the
defaults - this conversation reminds me that I should look at this
again.

The only FS I have lost complete systems (2 laptops, flat batteries when
not present) from were ext3, as well as continuous more minor corruption
issues (love backups)

reiserfs has had corruption issues in the past, but is currently very
stable.  Any issues that have developed have always been fixable with no
lost data.  I did run into a few repeatable issues with NFS - about 5
years ago.  None since from this.  A couple of minor issues with
crashes, easily fixed and some hardware failures.

I ran ext3 on a dirvish backup server - lasted two days, resierfs is
still going after a couple of years.  dirvish REALLY hammers a file
system.

Participating in a few of these discussions over the years has brought
home to me that YMMV really does apply to filesystems. Your usage, data
profile, power/hardware stability are all variables and any two peoples
experience almost assuredly wont be the same.

BillK

Re: [gentoo-user] Re: [OT] filesystems

[Joerg Schilling]

Alan McKinnon <alan.mckinnon@gmail.com> wrote:

> I think I need to investigate this deeper and do a write-up for work. I have a 
> FreeBSD-7 deployment system (nothing for OpenSolaris) so I'll use that. Any 
> comment on ZFS performace/stability on FreeBSD?

Up to a year ago, they found a star bug every two months that later turned out 
to be a FreeBSD/ZFS bug. I haven't heard similar reports since a year.

FreeBSD has a "more similar" VFS so a port is easier to implement.

Jörg

-- 
 EMail:joerg@schily.isdn.cs.tu-berlin.de (home) Jörg Schilling D-13353 Berlin
       js@cs.tu-berlin.de                (uni)  
       schilling@fokus.fraunhofer.de     (work) Blog: http://schily.blogspot.com/
 URL:  http://cdrecord.berlios.de/private/ ftp://ftp.berlios.de/pub/schily

Re: [gentoo-user] Re: [OT] filesystems

[Jorge Peixoto de Morais Neto]

On Tue, Nov 25, 2008 at 7:15 PM, Joerg Schilling
<Joerg.Schilling@fokus.fraunhofer.de> wrote:
> James <wireless@tampabay.rr.com> wrote:
>
>> Backups.
>>
>> get a usb stick and manually copy your stuff to it, periodically.
>
>
> Where do you get these 1 TB USB sticks?
>
You are seeing from the perspective of a sysadmin.
He was replying to me, a desktop user. My most important data occupy
little space and can surely be backed up in a 1 GB pen drive. The
music ripped from CDs, the linux ISOs and other kind of recoverable
data do not need backup.
As for my photos, I can back all the collection to a single DVD (and
to a second one, since I keep hearing that DVD-Rs are unreliable), and
since I don't take new photos every week, this solution is fine.

Re: [gentoo-user] [OT] filesystems

[Jorge Peixoto de Morais Neto]

> I ran ext3 on a dirvish backup server - lasted two days, resierfs is
> still going after a couple of years.  dirvish REALLY hammers a file
> system.
>
> Participating in a few of these discussions over the years has brought
> home to me that YMMV really does apply to filesystems. Your usage, data
> profile, power/hardware stability are all variables and any two peoples
> experience almost assuredly wont be the same.
In this discussion multiple people have defended reiserfs as a safe
filesystem. This is novel to me. Reiserfs is always bashed as being an
unsafe filesystem, developed with only speed in mind; a filesystem to
be used only by childish ricers or in specific situations where
filesystem performance is critical. For example, once I tried
genkernel (but did not like it and decide to go on with manual kernel
maintainance) and this message was in an ewarn
        ewarn "This package is known to not work with reiser4.  If you
are running"
	ewarn "reiser4 and have a problem, do not file a bug.  We know it does not"
	ewarn "work and we don't plan on fixing it since reiser4 is the one that is"
	ewarn "broken in this regard.  Try using a sane filesystem like ext3 or"
	ewarn "even reiser3."
They explicitly claim reiser4 is broken and insane, and their wording
implicitly suggests that ext3 is better than reiser3.

But in this discussion people are saying reiserfs is in fact safer than ext3.

I have not dived in the Linux developers x Hans Reiser battle, so I
don't know which side is right and which side is guilty, but think
that either
A) reiserfs is a good filesystem, but the battle between Hans Reiser
and Linux developers caused people to dislike reiserfs for
non-technical reasons.
or
B) reiserfs is a bad filesystem but for some reason a lot of reiserfs
fans appeared in this thread


Note: don't talk about the unfortunate horrible story of Hans' family,
the details of which we don't know. People were bashing reiserfs (both
versions 3 and 4) well before that.

Re: [gentoo-user] [OT] filesystems

[W.Kenworthy]

...
> I have not dived in the Linux developers x Hans Reiser battle, so I
> don't know which side is right and which side is guilty, but think
> that either
> A) reiserfs is a good filesystem, but the battle between Hans Reiser
> and Linux developers caused people to dislike reiserfs for
> non-technical reasons.
> or
> B) reiserfs is a bad filesystem but for some reason a lot of reiserfs
> fans appeared in this thread
> 
A is the answer.  Hans Reiser is by all accounts a brilliant, eccentric
but deeply flawed individual.  He did not get on at a personal or
professional level with the world in general.  It almost seems like
ext3/4 were developed to spite him and give alternatives so they would
not have to deal with him.  Unprofessional words and actions were taken
on both sides, but the animosity caused by Hans (and others in response)
means that this will take forever to blow over, even with Hans out of
the picture.

There is a huge amount out there on this.  There are also may other
highly valued developers out there who may also be a little eccentric
(to be kind!).

In the meantime, my opinion is that reiserfs3 is great, ext3 not quite
so good, and ext2/4 and reiserfs4 are for those who live on the edge :)

BillK

Re: [gentoo-user] [OT] filesystems

[Volker Armin Hemmann]

On Mittwoch 26 November 2008, Jorge Peixoto de Morais Neto wrote:
> > I ran ext3 on a dirvish backup server - lasted two days, resierfs is
> > still going after a couple of years.  dirvish REALLY hammers a file
> > system.
> >
> > Participating in a few of these discussions over the years has brought
> > home to me that YMMV really does apply to filesystems. Your usage, data
> > profile, power/hardware stability are all variables and any two peoples
> > experience almost assuredly wont be the same.
>
> In this discussion multiple people have defended reiserfs as a safe
> filesystem. This is novel to me. Reiserfs is always bashed as being an
> unsafe filesystem, developed with only speed in mind; a filesystem to
> be used only by childish ricers or in specific situations where
> filesystem performance is critical. For example, once I tried
> genkernel (but did not like it and decide to go on with manual kernel
> maintainance) and this message was in an ewarn
>         ewarn "This package is known to not work with reiser4.  If you
> are running"
> 	ewarn "reiser4 and have a problem, do not file a bug.  We know it does
> not" ewarn "work and we don't plan on fixing it since reiser4 is the one
> that is" ewarn "broken in this regard.  Try using a sane filesystem like
> ext3 or" ewarn "even reiser3."

reiser4 and reiserfs are two completly unrelated file systems.

reiserfs is the oldest journaling fs for linux. It had been broken in early 
2.4 development by careless vm patches which weren't tested prior to 
inclusion. This early breakage still haunts reiserfs.

If you look at lkml, there are regularly reports about problems with ext3 and 
xfs. But very few with reiserfs - and none with jfs because nobody is using 
it.


> They explicitly claim reiser4 is broken and insane, and their wording
> implicitly suggests that ext3 is better than reiser3.

And I claim that genkernel is a broken piece of shit, so what?
ext3 has enough problems - look at lkml. After that you might rethink claims 
that ext3 is 'stable'. 

>
> But in this discussion people are saying reiserfs is in fact safer than
> ext3.

experience. Obervation. I haven't seen reiserfs problems that were not the 
hardware's fault.

>
> I have not dived in the Linux developers x Hans Reiser battle, so I
> don't know which side is right and which side is guilty, but think
> that either

Hans Reiser has zero people skills and clashed with people who also have zero 
people skills. Add some misunderstandings (like plugins - they aren't 
plugins), a fat 'it is not developed here' syndrom and some bias and you get a 
nice explosive mess.
HR is completly out of the picture. Edward is doing reiser4 development today 
and he is doing a good job.


> A) reiserfs is a good filesystem, but the battle between Hans Reiser
> and Linux developers caused people to dislike reiserfs for
> non-technical reasons.

reiserfs is a good filesystem that was broken by third parties. Btw, some days 
ago Nick Piggin broke reiser4 in -mm. And instead of fixing it, they disabled 
reiser4. Which tells you a lot about the 'if you have something in kernel, it 
will be fixed when changes break it' lie.

> or
> B) reiserfs is a bad filesystem but for some reason a lot of reiserfs
> fans appeared in this thread

reiserfs is a stable filesystem. For ages no new features have been added. 
Unlike ext3 only bug fixes have been went in. The problem is, that redhat was 
behind ext3 - and redhat pushs all their stuff, while agressively attacking 
everything not made by them.


> Note: don't talk about the unfortunate horrible story of Hans' family,
> the details of which we don't know. People were bashing reiserfs (both
> versions 3 and 4) well before that.

because they don't understand either. reiser4 has tons of nice and good ideas 
- but some people saw Reiser's name and went beserk.

Re: [gentoo-user] [OT] filesystems

[Philip Webb]

081126 W.Kenworthy wrote:
>> A) reiserfs is a good filesystem,
>> but the battle between Hans Reiser and Linux developers
>> caused people to dislike reiserfs for non-technical reasons.
> A is the answer.  Hans Reiser is by all accounts a brilliant,
> eccentric but deeply flawed individual.  He did not get on
> at a personal or professional level with the world in general.
> It almost seems like ext3/4 were developed to spite him
> and give alternatives so they would not have to deal with him.
> Unprofessional words and actions were taken on both sides,
> but the animosity caused by Hans and others in response means
> this will take forever to blow over, even with Hans out of the picture.

Yes, very much my own take on the story.
I used Reiserfs in the computers I built in 2003 & 2007
& have never had any problems with either installation.

My  CAD 0,02 .

-- 
========================,,============================================
SUPPORT     ___________//___,   Philip Webb
ELECTRIC   /] [] [] [] [] []|   Cities Centre, University of Toronto
TRANSIT    `-O----------O---'   purslowatchassdotutorontodotca

Re: [gentoo-user] [OT] filesystems

[Dale]

Paul Hartman wrote:
> On Tue, Nov 25, 2008 at 2:27 AM, Dale <rdalek1967@gmail.com> wrote:
>   
>> Dirk Heinrichs wrote:
>>     
>>> Am Montag, 24. November 2008 22:09:52 schrieb Dale:
>>>
>>>
>>>       
>>>> I wouldn't use XFS unless
>>>> it was all that was left.  I tried it once a while back and found out it
>>>> does not like power failures at all.  Each time I had a power failure, I
>>>> had to reinstall from scratch.
>>>>
>>>>         
>>> Hmm, I use it because of its resistance to power failures. When was it that
>>> you had such problems?
>>>
>>> Bye...
>>>
>>>       Dirk
>>>
>>>
>>>
>>>       
>> Its been a while but it happened several times.  I just got tired of
>> having to reinstall every time the power blinked.  Turned out the wire
>> was loose on the transformer so they blinked a lot, every couple days or
>> so.  I think it was Mandrake 9.2.
>>
>> I have had a power failure or two with reiserfs and it recovered.  It
>> did the check thing but ran fine.
>>
>> Just my experience.  Your mileage may vary.
>>
>> Dale
>>
>> :-)  :-
>>     
>
> I have a similar story, but for me it was JFS instead of XFS. I will
> never, ever, ever use JFS for anything again. I had XFS on a file
> server RAID box with a failing power supply and it died over and over
> and the FS stayed functional, so YMMV indeed. (I haven't tried reiser,
> I'm still scared about the corruption stories from years ago.)
>
> I suppose if you ask enough people, there will be horror stories about
> every filesystem.
>
> No matter which FS you choose, I wish you good luck and hope you have
> no new horror stories. :)
>
> Paul
>
>
>   

LOL.  I have two hard drives and copy my main drive over to the second
drive pretty regular.  One is reiserfs and the other ext3.  What are the
chances both would screw up at the same time?  ;-) 

It could have been a bad version of XFS or something but after about
three or four times, it just got old.  I put ext3 on it after that and
it would recover fine, except for the griping about not being shutdown
properly and such.  It was a in-law so no clue exactly what it said but
it booted and worked.

You are right, no matter what FS you use, there is somebody that hates
it.  I guess you just have to install, make a back-up, then pull the
plug and see if it survives or not.  If it does, you got a keeper, if
not, restore to another FS and try again.  o_O

Dale

:-)  :-)

Re: [gentoo-user] [OT] filesystems

[Dale]

W.Kenworthy wrote:
> ...
>   
>> I have not dived in the Linux developers x Hans Reiser battle, so I
>> don't know which side is right and which side is guilty, but think
>> that either
>> A) reiserfs is a good filesystem, but the battle between Hans Reiser
>> and Linux developers caused people to dislike reiserfs for
>> non-technical reasons.
>> or
>> B) reiserfs is a bad filesystem but for some reason a lot of reiserfs
>> fans appeared in this thread
>>
>>     
> A is the answer.  Hans Reiser is by all accounts a brilliant, eccentric
> but deeply flawed individual.  He did not get on at a personal or
> professional level with the world in general.  It almost seems like
> ext3/4 were developed to spite him and give alternatives so they would
> not have to deal with him.  Unprofessional words and actions were taken
> on both sides, but the animosity caused by Hans (and others in response)
> means that this will take forever to blow over, even with Hans out of
> the picture.
>
> There is a huge amount out there on this.  There are also may other
> highly valued developers out there who may also be a little eccentric
> (to be kind!).
>
> In the meantime, my opinion is that reiserfs3 is great, ext3 not quite
> so good, and ext2/4 and reiserfs4 are for those who live on the edge :)
>
> BillK
>
>
>   

I'm not expecting a answer but along the lines of a viewpoint in a
question form.  Why is it that smart, I mean seriously smart, people
have the worst social skills?  They can invent a super fast CPU, memory
chip, hard drive some new chemical, or some other ingenious thing but
can't say a kind word if you give them a double dose of Prozac.

Sort of strange huh?

Dale

:-)  :-) 

Re: [gentoo-user] Re: [OT] filesystems

[Stroller]

On 25 Nov 2008, at 21:35, Alan McKinnon wrote:
> ...
> Personally, I'm very interested in seeing where Intel go with their  
> new SSDs.
> The reason is that I'm getting sick and tired of having to explain and
> justify why the laws of physics prevent my girlfriend from being  
> able to
> backup her 5T banking warehouse on the schedule the law of the  
> country would
> like her to. There's a solution, but it's hackish and ugly and  
> involves
> extremely careful management of LVM snapshots. This is simply way  
> too much
> admin effort for what should really be a simple incremental backup  
> process.

Sounds like you want ZFS.
On Solaris.
On a little dedicated box which exports space via NFS.

Stroller.

Re: [gentoo-user] Re: [OT] filesystems

[Stroller]

On 25 Nov 2008, at 21:15, Joerg Schilling wrote:
> James <wireless@tampabay.rr.com> wrote:
>
>> Backups.
>>
>> get a usb stick and manually copy your stuff to it, periodically.
>
>
> Where do you get these 1 TB USB sticks?

http://www.amazon.co.uk/Western-Digital-Essential-External-Drive/dp/B000W9RNOA

Stroller.

Re: [gentoo-user] [OT] filesystems

[Joerg Schilling]

"Jorge Peixoto de Morais Neto" <please.no.spam.here@gmail.com> wrote:

> Oh, and according to this benchmark
> http://linuxgazette.net/122/piszcz.html
> reiserfs does not deserve its speed fame.

The ext filesystem is slow if you meter the right times.

If you e.g. untar a linux kernel tarball and just take the time
GNU tar runs, you have a time but you don't know the related action!

If you do this, disk I/O (with a few exceptions) will typically start
after GNU tar exited.

If you like to compare, you could either use star (that by default calls
fsync(2) on avery single file after extracting) or pull the power cord
after GNU tar finished and then check after a reboot ;-) On a ext filesystem,
star extracts 4x slower in default mode compared to star -no-fsync

I have no times for reiserfs, but it may be that the numbers look completely
different if you get a time for a known action. 

Jörg

-- 
 EMail:joerg@schily.isdn.cs.tu-berlin.de (home) Jörg Schilling D-13353 Berlin
       js@cs.tu-berlin.de                (uni)  
       schilling@fokus.fraunhofer.de     (work) Blog: http://schily.blogspot.com/
 URL:  http://cdrecord.berlios.de/private/ ftp://ftp.berlios.de/pub/schily

Re: [gentoo-user] [OT] filesystems

[Joerg Schilling]

Volker Armin Hemmann <volker.armin.hemmann@tu-clausthal.de> wrote:

> reiserfs has barriers turned on by default - which makes it a bit slower but a 
> lot safer for data. ext3 has them turned off by default - ext3 devs don't care 
> about data - only speed. You turn on barriers, performance goes down by 30%.

There is even a difference between real speed and apparently observed speed.
The latter is optimized :-(

Jörg

-- 
 EMail:joerg@schily.isdn.cs.tu-berlin.de (home) Jörg Schilling D-13353 Berlin
       js@cs.tu-berlin.de                (uni)  
       schilling@fokus.fraunhofer.de     (work) Blog: http://schily.blogspot.com/
 URL:  http://cdrecord.berlios.de/private/ ftp://ftp.berlios.de/pub/schily

Re: [gentoo-user] [OT] filesystems

[Joerg Schilling]

Volker Armin Hemmann <volker.armin.hemmann@tu-clausthal.de> wrote:

> > Oh, and according to this benchmark
> > http://linuxgazette.net/122/piszcz.html
> > reiserfs does not deserve its speed fame.
>
> they tested crap.
>
> As I wrote in the other mail. XFS and reiserfs turn on barriers by default, 
> ext3 turns them off.
> With barriers on for ext3 it looses 30%(!). reiserfs and xfs don't suffer as 
> much, but suffer they do. So if the test did not turn on or off barriers for 
> all fs who support them, ext3 had an unfair advantage.
>
> And you want barriers. 

I am not sure what you call "barriers"....

ext3 slows down by 400% if you call fsync(2) after copying single files.

UFS on Solaris slows down by 10% only because UFS has been optimized for best 
speed _and_ best data integrity.

Jörg

-- 
 EMail:joerg@schily.isdn.cs.tu-berlin.de (home) Jörg Schilling D-13353 Berlin
       js@cs.tu-berlin.de                (uni)  
       schilling@fokus.fraunhofer.de     (work) Blog: http://schily.blogspot.com/
 URL:  http://cdrecord.berlios.de/private/ ftp://ftp.berlios.de/pub/schily

Re: [gentoo-user] [OT] filesystems

[Joerg Schilling]

"Jorge Peixoto de Morais Neto" <please.no.spam.here@gmail.com> wrote:

> I have not dived in the Linux developers x Hans Reiser battle, so I
> don't know which side is right and which side is guilty, but think
> that either
> A) reiserfs is a good filesystem, but the battle between Hans Reiser
> and Linux developers caused people to dislike reiserfs for
> non-technical reasons.
> or

The Linux VFS is far from being optimal. I would guess that the real 
reason for not starting a ZFS port for Linux is the Linux VFS.

The problem with Linux is that important external interfaces are broken 
with every new release but that internal kernel interfaces are not evolved.
Sun claims e.g. that the changes in the Solaris kernel to allow to support
a full blown CIFS in the kernel have been bigger than than the ZFS code
size before the change.

If Linux does not evolve the Linux VFS layer, these battles will never end.

Jörg

-- 
 EMail:joerg@schily.isdn.cs.tu-berlin.de (home) Jörg Schilling D-13353 Berlin
       js@cs.tu-berlin.de                (uni)  
       schilling@fokus.fraunhofer.de     (work) Blog: http://schily.blogspot.com/
 URL:  http://cdrecord.berlios.de/private/ ftp://ftp.berlios.de/pub/schily

Re: [gentoo-user] [OT] filesystems

[Mike Edenfield]

Dale wrote:

> I'm not expecting a answer but along the lines of a viewpoint in a
> question form.  Why is it that smart, I mean seriously smart, people
> have the worst social skills?  They can invent a super fast CPU, memory

http://en.wikipedia.org/wiki/Asperger_syndrome

Re: [gentoo-user] Re: [OT] filesystems

[Florian Philipp]

Jorge Peixoto de Morais Neto schrieb:
> On Tue, Nov 25, 2008 at 7:15 PM, Joerg Schilling
> <Joerg.Schilling@fokus.fraunhofer.de> wrote:
>> James <wireless@tampabay.rr.com> wrote:
>>
[...]
> As for my photos, I can back all the collection to a single DVD (and
> to a second one, since I keep hearing that DVD-Rs are unreliable), and
> since I don't take new photos every week, this solution is fine.
> 

A second DVD-R won't solve the problem because optical disks degrade 
over time and the second one will degrade just as fast as the first. 
What you need to do is to check the disks periodically (once a year is a 
good time frame).
I myself would add a textfile with md5sums for all files to the DVD so 
you don't have to check them visually.

Re: [gentoo-user] [OT] filesystems

[Alan McKinnon]

On Wednesday 26 November 2008 07:05:39 Dale wrote:
> I'm not expecting a answer but along the lines of a viewpoint in a
> question form.  Why is it that smart, I mean seriously smart, people
> have the worst social skills?  They can invent a super fast CPU, memory
> chip, hard drive some new chemical, or some other ingenious thing but
> can't say a kind word if you give them a double dose of Prozac.
>
> Sort of strange huh?

Easy.

It's not that smart people have zero social skills. Smart people have the same 
spread of social skills as average and dumb people.

Some smart people do not suffer fools gladly and they rise to prominence 
whereas others just act like everyone else and you do not especially note 
this fact. Smart people who work with machines get to be very good at it, but 
machines don't talk back. Some smart folk take to talking to people the way 
they talk to machines and this is noteworthy. Again, you do not take note of 
the majority that do not do this.


-- 
alan dot mckinnon at gmail dot com

Re: [gentoo-user] Re: [OT] filesystems

[Alan McKinnon]

On Wednesday 26 November 2008 11:16:51 Stroller wrote:
> On 25 Nov 2008, at 21:35, Alan McKinnon wrote:
> > ...
> > Personally, I'm very interested in seeing where Intel go with their
> > new SSDs.
> > The reason is that I'm getting sick and tired of having to explain and
> > justify why the laws of physics prevent my girlfriend from being
> > able to
> > backup her 5T banking warehouse on the schedule the law of the
> > country would
> > like her to. There's a solution, but it's hackish and ugly and
> > involves
> > extremely careful management of LVM snapshots. This is simply way
> > too much
> > admin effort for what should really be a simple incremental backup
> > process.
>
> Sounds like you want ZFS.
> On Solaris.
> On a little dedicated box which exports space via NFS.

No can do :-)

But I didn't give you the full story. Those 5TB are stored in a column-based 
database on a raw device. The current setup is in production, in a commercial 
bank and is already delivering aggregated analytical data to the users. When 
I say "users" you should read "actuaries that determine the bank's business 
direction for the next several years"

Microsoft will ditch Windows and fully embrace OSS long before that lot will 
entertain the mere idea of a platform change :-)



-- 
alan dot mckinnon at gmail dot com

Re: [gentoo-user] [OT] filesystems

[Dale]

Alan McKinnon wrote:
> On Wednesday 26 November 2008 07:05:39 Dale wrote:
>   
>> I'm not expecting a answer but along the lines of a viewpoint in a
>> question form.  Why is it that smart, I mean seriously smart, people
>> have the worst social skills?  They can invent a super fast CPU, memory
>> chip, hard drive some new chemical, or some other ingenious thing but
>> can't say a kind word if you give them a double dose of Prozac.
>>
>> Sort of strange huh?
>>     
>
> Easy.
>
> It's not that smart people have zero social skills. Smart people have the same 
> spread of social skills as average and dumb people.
>
> Some smart people do not suffer fools gladly and they rise to prominence 
> whereas others just act like everyone else and you do not especially note 
> this fact. Smart people who work with machines get to be very good at it, but 
> machines don't talk back. Some smart folk take to talking to people the way 
> they talk to machines and this is noteworthy. Again, you do not take note of 
> the majority that do not do this.
>
>
>   


I guess we just notice the "bad ones," if you want to call them that. 

Dale

:-)  :-) 

Re: [gentoo-user] Re: [OT] filesystems

[Anthony Metcalf]

[-- Attachment #1: Type: text/plain, Size: 636 bytes --]

Alan McKinnon wrote:
> On Wednesday 26 November 2008 11:16:51 Stroller wrote:
>   
>> Sounds like you want ZFS.
>> On Solaris.
>> On a little dedicated box which exports space via NFS.
>>     
>
> No can do :-)
>
> But I didn't give you the full story. Those 5TB are stored in a column-based 
> database on a raw device. 

Replace the last line with "On a little dedicated box which exports
space over iSCSI"?

Windows will see that as a raw device, using the very nice iscsi
initiator they provide.....

From tutorials I've seen, the creation of the ZFS pool, and iSCSI export
is like 4 commands in total....

A

[-- Attachment #2: Type: text/html, Size: 1101 bytes --]

Re: [gentoo-user] Re: [OT] filesystems

[Daniel Troeder]

[-- Attachment #1: Type: text/plain, Size: 1463 bytes --]

Am Mittwoch, den 26.11.2008, 15:26 +0100 schrieb Florian Philipp:
> Jorge Peixoto de Morais Neto schrieb:
> > On Tue, Nov 25, 2008 at 7:15 PM, Joerg Schilling
> > <Joerg.Schilling@fokus.fraunhofer.de> wrote:
> >> James <wireless@tampabay.rr.com> wrote:
> >>
> [...]
> > As for my photos, I can back all the collection to a single DVD (and
> > to a second one, since I keep hearing that DVD-Rs are unreliable), and
> > since I don't take new photos every week, this solution is fine. 
> 
> A second DVD-R won't solve the problem because optical disks degrade 
> over time and the second one will degrade just as fast as the first. 
> What you need to do is to check the disks periodically (once a year is a 
> good time frame).
> I myself would add a textfile with md5sums for all files to the DVD so 
> you don't have to check them visually.
You can buy so called "archival grade" DVD-Rs that should work for 10-20
years in a good environment. There are hugh differences between
products. In germany you can buy very good ones from Verbatim for around
2€/disk.

Also keep your photos on 2 HDDs (in different places), and copy them to
new HDDs when you buy new ones. Nowadays you buy new HDDs every 2-3
years or so, and normally the old ones should not have failed until then
(at least not 2 disks on the same sectors).
The MD5/SHA-checks should be done before copying to the new disks, so
you can be sure to copy only healthy files.

[-- Attachment #2: Dies ist ein digital signierter Nachrichtenteil --]
[-- Type: application/pgp-signature, Size: 197 bytes --]

Re: [gentoo-user] Re: [OT] filesystems

[KH]

Daniel Troeder schrieb:
> Am Mittwoch, den 26.11.2008, 15:26 +0100 schrieb Florian Philipp:
>   
>> Jorge Peixoto de Morais Neto schrieb:
>>     
>>> On Tue, Nov 25, 2008 at 7:15 PM, Joerg Schilling
>>> <Joerg.Schilling@fokus.fraunhofer.de> wrote:
>>>       
>>>> James <wireless@tampabay.rr.com> wrote:
>>>>
>>>>         
>> [...]
>>     
>>> As for my photos, I can back all the collection to a single DVD (and
>>> to a second one, since I keep hearing that DVD-Rs are unreliable), and
>>> since I don't take new photos every week, this solution is fine. 
>>>       
>> A second DVD-R won't solve the problem because optical disks degrade 
>> over time and the second one will degrade just as fast as the first. 
>> What you need to do is to check the disks periodically (once a year is a 
>> good time frame).
>> I myself would add a textfile with md5sums for all files to the DVD so 
>> you don't have to check them visually.
>>     
> You can buy so called "archival grade" DVD-Rs that should work for 10-20
> years in a good environment. There are hugh differences between
> products. In germany you can buy very good ones from Verbatim for around
> 2€/disk.
>
> Also keep your photos on 2 HDDs (in different places), and copy them to
> new HDDs when you buy new ones. Nowadays you buy new HDDs every 2-3
> years or so, and normally the old ones should not have failed until then
> (at least not 2 disks on the same sectors).
> The MD5/SHA-checks should be done before copying to the new disks, so
> you can be sure to copy only healthy files.
>   
If you decide to store dvds for 20 years it would be wise to store a pc
with it. Who knows if there a dvd readers or ide / sata connections
available then ;-)
Everybody who has data to protect should really store them in two
different places which do not burn down at the same time (there can be a
flood / earthquake ... [what I want to say: The places really should be
different!])
Yes check the backup before storing it. I once hat a useless backup. All
the files showed up but they only used some k. Thank god I had a second
backup which was fine.
If you only take photos randomly, you also can use cds, if the size is
big enough. And espacialy for photos hard copys are very nice. One even
can show them to grandparents ... ;-)

kh

Re: [gentoo-user] Re: [OT] filesystems

[Jorge Peixoto de Morais Neto]

On Thu, Nov 27, 2008 at 10:12 AM, Daniel Troeder <daniel@admin-box.com> wrote:
> Am Mittwoch, den 26.11.2008, 15:26 +0100 schrieb Florian Philipp:
>> > As for my photos, I can back all the collection to a single DVD (and
>> > to a second one, since I keep hearing that DVD-Rs are unreliable), and
>> > since I don't take new photos every week, this solution is fine.
>>
>> A second DVD-R won't solve the problem because optical disks degrade
>> over time and the second one will degrade just as fast as the first.
>> What you need to do is to check the disks periodically (once a year is a
>> good time frame).
>> I myself would add a textfile with md5sums for all files to the DVD so
>> you don't have to check them visually.
I have recently taken this decision too. Unfortunately I haven't done
so for some old backups (fortunately they still seem healthy)
> You can buy so called "archival grade" DVD-Rs that should work for 10-20
> years in a good environment. There are hugh differences between
> products. In germany you can buy very good ones from Verbatim for around
> 2€/disk.
This can be hard to find in my mid-sized Brazilian city. If I lived in
the mega-metropolis of São Paulo, this would be far easier. And thanks
very much for recommending Verbatim. I have heard of Taiyo Yuden, but
that would likely be far harder to find.

Speaking of md5sum/shasum, do you know some tool that adds data
redundancy? I heard dvddistaster does this, but I guess it is limited
to DVDs. It would be great fo find a general data redundancy tool. In
the moment, with the tools I know, the best I can do is store the
files twice, with md5sums/shasums to decide which version is correct.
By the way, it seems from my (limited) experience that even sha256sums
are IO-bound (even on my not-so-powerful Athlon XP 2600+), so it makes
sense to calculate sha256sums (as instead of md5sums) even it is
overkill. To be doubly sure, one can calculate sha256sums *and*
md5sums.

Re: [gentoo-user] Re: [OT] filesystems

[Jorge Peixoto de Morais Neto]

On Wed, Nov 26, 2008 at 12:26 PM, Florian Philipp
<lists@f_philipp.fastmail.net> wrote:
>> As for my photos, I can back all the collection to a single DVD (and
>> to a second one, since I keep hearing that DVD-Rs are unreliable), and
>> since I don't take new photos every week, this solution is fine.
>>
>
> A second DVD-R won't solve the problem because optical disks degrade over
> time and the second one will degrade just as fast as the first. What you
> need to do is to check the disks periodically (once a year is a good time
> frame).
I know DVD-Rs degrade, but it is unlikely they would fail at the same
time, so copying twice does significantly alleviate the problem
(AFAIK)
Once a year isn't overkill? Isn't once every two years fine?

> I myself would add a textfile with md5sums for all files to the DVD so you
> don't have to check them visually.
Sure. I am doing that since some time now. Unfortunately I didn't do
so for some old backups. But data DVD-Rs have a considerable amount of
correction code, and if the copy from DVD to hard disk proceeds
without a single error message, there is a quite good chance that the
files are good, right?
(If they were burnt correctly in the fist place, that is. The manual
of my DVD-RW drive warns that the burned disk should be checked before
being trusted as a backup, and even then it - as usual - disclaims all
warranties).

Re: [gentoo-user] Re: [OT] filesystems

[Florian Philipp]

Jorge Peixoto de Morais Neto schrieb:
> On Wed, Nov 26, 2008 at 12:26 PM, Florian Philipp
> <lists@f_philipp.fastmail.net> wrote:
>>> As for my photos, I can back all the collection to a single DVD (and
>>> to a second one, since I keep hearing that DVD-Rs are unreliable), and
>>> since I don't take new photos every week, this solution is fine.
>>>
>> A second DVD-R won't solve the problem because optical disks degrade over
>> time and the second one will degrade just as fast as the first. What you
>> need to do is to check the disks periodically (once a year is a good time
>> frame).
> I know DVD-Rs degrade, but it is unlikely they would fail at the same
> time, so copying twice does significantly alleviate the problem
> (AFAIK)

I'm not so sure in this regard. If we were talking about HDDs you were 
right: it is very unlikely for two of them to fail at the same time due 
to mechanical defects. But we are talking about optical media. They fail 
because of chemical reactions. That's why two disks, stored equally, 
bought at the same time from the same trader, produced by the same 
company, should degrade equally fast and therefore fail at about the 
same time. And since you want to check them less than once a year, "at 
about the same time" means within the same year.


> Once a year isn't overkill? Isn't once every two years fine?
> 

I'm not sure. I myself wouldn't trust normal CD/DVD-Rs for more than 
three years and CD/DVD-RWs for more than one year (cheap RWs degrade 
much faster than Rs).

Additionally, having such long intervals between checks makes it easier 
to forget them completely. Can you remember whether you checked your 
disks last year or the year before? I know I couldn't.

> Sure. I am doing that since some time now. Unfortunately I didn't do
> so for some old backups. But data DVD-Rs have a considerable amount of
> correction code, and if the copy from DVD to hard disk proceeds
> without a single error message, there is a quite good chance that the
> files are good, right?

I would think so.

Re: [gentoo-user] Re: [OT] filesystems

[Shawn Haggett]

On Sat, 29 Nov 2008 04:21:44 pm Jorge Peixoto de Morais Neto wrote:
> On Thu, Nov 27, 2008 at 10:12 AM, Daniel Troeder <daniel@admin-box.com> 
wrote:
> > Am Mittwoch, den 26.11.2008, 15:26 +0100 schrieb Florian Philipp:
> > You can buy so called "archival grade" DVD-Rs that should work for 10-20
> > years in a good environment. There are hugh differences between
> > products. In germany you can buy very good ones from Verbatim for around
> > 2€/disk.
>
> This can be hard to find in my mid-sized Brazilian city. If I lived in
> the mega-metropolis of São Paulo, this would be far easier. And thanks
> very much for recommending Verbatim. I have heard of Taiyo Yuden, but
> that would likely be far harder to find.
>
> Speaking of md5sum/shasum, do you know some tool that adds data
> redundancy? I heard dvddistaster does this, but I guess it is limited
> to DVDs. It would be great fo find a general data redundancy tool. In
> the moment, with the tools I know, the best I can do is store the
> files twice, with md5sums/shasums to decide which version is correct.

Have a look at app-arch/par2cmdline ( http://parchive.sourceforge.net/ ). It 
will create parity files for an arbitrary set of data files and you can 
choose your level of redundency (from 0 = now redundency, just integrity 
checking, up to 100%). Although expect your parity files to be on the order 
of the percentage for size, i.e. 50% redundancy for some given files to take 
about 50% of their size for the parity files).

The down side I find with the tool is that it doesn't currently support 
directories. This isn't so bad for creating parity files, but during 
checking/restore, the program expects all files to exist in the current 
directory, despite which sub-dirs they were originally in. You can get around 
this with a tar/rar/zip first, then calculate parities on the archive though.

> By the way, it seems from my (limited) experience that even sha256sums
> are IO-bound (even on my not-so-powerful Athlon XP 2600+), so it makes
> sense to calculate sha256sums (as instead of md5sums) even it is
> overkill. To be doubly sure, one can calculate sha256sums *and*
> md5sums.

Re: [gentoo-user] Re: [OT] filesystems

[Jorge Peixoto de Morais Neto]

On Mon, Dec 1, 2008 at 1:38 AM, Shawn Haggett <podge@podgeweb.com> wrote:
> On Sat, 29 Nov 2008 04:21:44 pm Jorge Peixoto de Morais Neto wrote:
>> Speaking of md5sum/shasum, do you know some tool that adds data
>> redundancy? I heard dvddistaster does this, but I guess it is limited
>> to DVDs. It would be great fo find a general data redundancy tool. In
>> the moment, with the tools I know, the best I can do is store the
>> files twice, with md5sums/shasums to decide which version is correct.
>
> Have a look at app-arch/par2cmdline ( http://parchive.sourceforge.net/ ). It
> will create parity files for an arbitrary set of data files and you can
> choose your level of redundency (from 0 = now redundency, just integrity
> checking, up to 100%). Although expect your parity files to be on the order
> of the percentage for size, i.e. 50% redundancy for some given files to take
> about 50% of their size for the parity files).
>
> The down side I find with the tool is that it doesn't currently support
> directories. This isn't so bad for creating parity files, but during
> checking/restore, the program expects all files to exist in the current
> directory, despite which sub-dirs they were originally in. You can get around
> this with a tar/rar/zip first, then calculate parities on the archive though.
>
Thank you very much. I have taken a quick look at this, and seems to
be what I look for. In a few days, when I have time, I will try it on
some files and see the results.

[gentoo-user] Re: [OT] filesystems

[Alex Schuster]

Jorge Peixoto de Morais Neto wrote:

> Speaking of md5sum/shasum, do you know some tool that adds data
> redundancy? I heard dvddistaster does this, but I guess it is limited
> to DVDs.

No, it is not.

> It would be great fo find a general data redundancy tool.

emerge dvdisaster and see if it suits you.

	Wonko