[gentoo-dev] gentoo-sources-r9 kernel sources vs. XFS patch

[gentoo-dev] gentoo-sources-r9 kernel sources vs. XFS patch

[John Newman]

Why is it that the XFS patches from SGI (available at
ftp://oss.sgi.com/projects/xfs/download/patches/2.4.19) will not patch
properly against the gentoo-sources kernel 2.4.19-gentoo-r2?  90% of the
patch goes through but several hunks do fail.  Which patch already in 
2.4.19-gentoo-r2 is incompatible with XFS?

A related question - why does the gentoo XFS-sources kernel source package
not come with the preempt-ac patch?  Is preempt incompatible with XFS?

thanks for any help,

-- 
john

Re: [gentoo-dev] gentoo-sources-r9 kernel sources vs. XFS patch

[Gustavo Felisberto]

> Why is it that the XFS patches from SGI (available at
> ftp://oss.sgi.com/projects/xfs/download/patches/2.4.19) will not patch
> properly against the gentoo-sources kernel 2.4.19-gentoo-r2?  90% of the
> patch goes through but several hunks do fail.  Which patch already in
> 2.4.19-gentoo-r2 is incompatible with XFS?
>
> A related question - why does the gentoo XFS-sources kernel source
> package not come with the preempt-ac patch?  Is preempt incompatible
> with XFS?
>

Alot of people have been having serious problems with XFS (I am not off
them and have been using xfs in debian and gentoo for about one year now
in some 20 machines).
I have found somewhere a patches-2.4.19-gentoo-r10.tar.bz2 sized 2665464.
This patch applied to a clean 2.4.19 clean tree and had XFS and the other
gentoo patches that we are used.I have found this in a gentoo page
somewhere but cant remenber where (the EXTRAVERSION is rmap14b).

I know ALOT of people had problems after power failures with XFS, but i
have ups in my systems and had no problem. And i was a bit sad when the
gentoo people decided to remove XFS from gentoo-sources package.

Gustavo Felisberto

Re: [gentoo-dev] gentoo-sources-r9 kernel sources vs. XFS patch

[Seth Mos]

At 04:12 24-10-2002 -0500, John Newman wrote:
>Why is it that the XFS patches from SGI (available at
>ftp://oss.sgi.com/projects/xfs/download/patches/2.4.19) will not patch
>properly against the gentoo-sources kernel 2.4.19-gentoo-r2?  90% of the
>patch goes through but several hunks do fail.  Which patch already in
>2.4.19-gentoo-r2 is incompatible with XFS?

The most recent XFS CVS should work with preempt but it is not thoroughly 
tested. A number  of users have corrupted filesystems when they were 
running with preempt. Both with clean shutdowns and reboots as well as 
"power failures".

I suggest not using preempt with XFS for production systems.

>A related question - why does the gentoo XFS-sources kernel source package
>not come with the preempt-ac patch?  Is preempt incompatible with XFS?

For some reason or the other they had problems with XFS and power failures 
which they were not able to resolve. Most likely this had to do with the 
bad preempt patches interaction.

Note that one of the biggest problems people have had to date is with IDE 
disks that have had the write cache turned on. No filesystem out there can 
recover from a power failure on a disk with write caching turned on. In 
special journaling filesystems have problems with respect to this problem 
since there is _NO_ guarantee that the LOG was actually _written_ to disk 
when the call returns from the block layer. And disastrous things happen 
when the journal/log is corrupt.

The other problem is that some IDE disks are lying about whether the write 
cache is actually turned off.

Otherwise it's a fine filesystem that works just like the others. I have 
had little problems with it to date. I have used it in production for over 
a year.

Cheers

--
Seth
It might just be your lucky day, if you only knew.

Re: [gentoo-dev] gentoo-sources-r9 kernel sources vs. XFS patch

[John Newman]

On Thu, Oct 24, 2002 at 12:02:10PM +0200, Seth Mos wrote:
> I suggest not using preempt with XFS for production systems.

Ok.   Does my desktop count as production :)

> Otherwise it's a fine filesystem that works just like the others. I have 
> had little problems with it to date. I have used it in production for over 
> a year.

I've been using it on my Gentoo system since I first installed.  I really
wanted to upgrade to get some of the grsecurity stuff implemented but I also
wanted to try the preempt patch.  I went ahead and hacked the xfs patches
in.  I had to fix about a dozen rejected patches but it was all pretty
straight forward stuff. If anybody wants a copy from my tree of slightly
hacked xfs patches that willy apply to their gentoo-sources (which apparently 
may be a bad idea) send me an email.


Thanks for the quick responses!

--
john

Re: [gentoo-dev] gentoo-sources-r9 kernel sources vs. XFS patch

[John Newman]

My kernel didn't quite compile.  Anyone know where this symbol got off to:

ld -m elf_i386 -T /usr/src/linux-2.4.19-gentoo-r9/arch/i386/vmlinux.lds -e

[ SNIP .............  ]

/usr/src/linux-2.4.19-gentoo-r9/arch/i386/lib/lib.a --end-group  -o vmlinux
fs/fs.o: In function `fsync_dev_lockfs':
fs/fs.o(.text+0x3bb8): undefined reference to `DQUOT_SYNC'
make[1]: *** [kallsyms] Error 1

Grepping the source shows no typedef or function called DQUOT_SYNC
(although there are typedefs DQUOT_SYNC_DEV and DQUOT_SYNC_SB in
include/linux/quotaops.h).

Here's my .config if anyone is interested:

http://www.cachehit.net/gentoo-r9+xfs.config.txt


thanks,

--
-john

Re: [gentoo-dev] gentoo-sources-r9 kernel sources vs. XFS patch

[Seth Mos]

At 05:47 24-10-2002 -0500, John Newman wrote:
>My kernel didn't quite compile.  Anyone know where this symbol got off to:
>
>ld -m elf_i386 -T /usr/src/linux-2.4.19-gentoo-r9/arch/i386/vmlinux.lds -e
>
>Grepping the source shows no typedef or function called DQUOT_SYNC
>(although there are typedefs DQUOT_SYNC_DEV and DQUOT_SYNC_SB in
>include/linux/quotaops.h).

It's the disk quota's. Did you enable any of them?

There have been some changes in the quota format in the past few months 
which might explain this.

Cheers

--
Seth
It might just be your lucky day, if you only knew.

Re: [gentoo-dev] gentoo-sources-r9 kernel sources vs. XFS patch

[Brandon Low]

The XFS patch is writtten very badly and 'touches' many files outside of
the filesystem specific code, it is therefore very difficult to patch it
against modified kernels such as the gentoo kernel.  I and the other
kernel developers at gentoo are working to get XFS into the gentoo-sources
again, and currently have a testing patch that is part of
gentoo-sources-2.4.19-r10 (enabled by emerging it with the non official
useflag of "xfs" set) Again, this is an experimental patch, and any bugs
reported based on using the gentoo-sources-2.4.19-r10 with this enabled
will not be treated as critical until they are reproduced without the xfs
patch in the kernel.  Hopefully as SGI improves their patch and brings it
closer to in sync with the main kernel trees, we will have less trouble
supporting it in the gentoo-sources.  For the time being however, I
recommend the use of xfs-sources for XFS as this is a safe patchset for
xfs, and is based on the SGI xfs tree rather than the vanilla or -ac 
kernels as the gentoo-sources are.

As for preempt with XFS, most of the major performance enhancing patches
are dropped from xfs-sources when it is created from the gentoo-sources
because they tend to cause conflicts of various sorts.  

Hope this clears some things up,

--Brandon

On Thu, 10/24/02 at 04:12:51 -0500, John Newman wrote:
> Why is it that the XFS patches from SGI (available at
> ftp://oss.sgi.com/projects/xfs/download/patches/2.4.19) will not patch
> properly against the gentoo-sources kernel 2.4.19-gentoo-r2?  90% of the
> patch goes through but several hunks do fail.  Which patch already in 
> 2.4.19-gentoo-r2 is incompatible with XFS?
> 
> A related question - why does the gentoo XFS-sources kernel source package
> not come with the preempt-ac patch?  Is preempt incompatible with XFS?
> 
> thanks for any help,
> 
> -- 
> john
> _______________________________________________
> gentoo-dev mailing list
> gentoo-dev@gentoo.org
> http://lists.gentoo.org/mailman/listinfo/gentoo-dev

Re: [gentoo-dev] gentoo-sources-r9 kernel sources vs. XFS patch

[Seth Mos]

At 11:13 24-10-2002 -0500, Brandon Low wrote:
>The XFS patch is writtten very badly and 'touches' many files outside of
>the filesystem specific code, it is therefore very difficult to patch it
>against modified kernels such as the gentoo kernel.  I and the other
>kernel developers at gentoo are working to get XFS into the gentoo-sources
>again, and currently have a testing patch that is part of

Great!

>Hopefully as SGI improves their patch and brings it
>closer to in sync with the main kernel trees, we will have less trouble
>supporting it in the gentoo-sources.

XFS is already merged in since 2.5.36. The integration with the 2.4 tree is 
a ongoing project. It might be integrated into 2.4 in the future when 
marcelo thinks the time is right.

Note that work is currently underway for a 1.2 release of XFS which would 
solve a number of these problems. It fixes a lot of bugs that were 
originally in the 1.1 release and also adds a number of new features that 
benefit people with md raid 5, LVM, LVM2 en EVMS.

>As for preempt with XFS, most of the major performance enhancing patches
>are dropped from xfs-sources when it is created from the gentoo-sources
>because they tend to cause conflicts of various sorts.

That sums it up nicely, both XFS and preempt "hook" into the VM and a 
slight patch mistake, typo or other mishap can have rather annoying results ;-/

>Hope this clears some things up,

Yes it does.

Cheers

--
Seth
It might just be your lucky day, if you only knew.

Re: [gentoo-dev] gentoo-sources-r9 kernel sources vs. XFS patch

[Brandon Low]

> XFS is already merged in since 2.5.36. The integration with the 2.4 tree is 
> a ongoing project. It might be integrated into 2.4 in the future when 
> marcelo thinks the time is right.
> 
It isn't a matter of Marcelo feeling that the time is right, it is a 
matter of someone taking the time (as was done for 2.5) to simplify the 
patch and get rid of the extraneous s**t that it messes with outside of 
what a filesystem should.  I think I posted elsewhere about this, but 
merging JFS into 2.4 involved about a 2000 line patch to kernel core 
code, XFS's patch is closer to 40000 lines.  Normally, adding a filesystem 
to a kernel tree is not a big deal, but as you might guess getting a 
massive 40,000 line hunk of code past Marcelo is NOT going to happen.  The 
XFS people (or some other kind soul, as I mentioned) need to 1) start 
feeding good bits of the XFS patch to Marcelo and 2) start synching it up 
with his tree so that JUST the filesystem can be merged w/o all the random 
other stuff they touch.

Blah, that is badly written, but whatever, I just woke up.

--Brandon

Re: [gentoo-dev] gentoo-sources-r9 kernel sources vs. XFS patch

[Seth Mos]

At 13:07 25-10-2002 -0500, Brandon Low wrote:
> > XFS is already merged in since 2.5.36. The integration with the 2.4 
> tree is
> > a ongoing project. It might be integrated into 2.4 in the future when
> > marcelo thinks the time is right.
> >
>It isn't a matter of Marcelo feeling that the time is right, it is a
>matter of someone taking the time (as was done for 2.5) to simplify the
>patch and get rid of the extraneous s**t that it messes with outside of
>what a filesystem should.

That's what the split patches are for. They are respun every once in a while.
Merging a complete XFS patch from CVS into a vanilla tree is not a smart 
idea since it also contains KDB for example.
The split patches are divided into the fs, dmapi, acl, kdb and extended 
attributes.

You don't need to merge them all. The current XFS in 2.5 has the only the 
fs layer. For all the extras it's a smarter idea to check out the 2.5 cvs 
tree. Dmapi needs more work and be more general so it can/might be used on 
ext2/3 before they integrate it.
Acls are still in progress bit will end up there eventually.

>   I think I posted elsewhere about this, but
>merging JFS into 2.4 involved about a 2000 line patch to kernel core
>code, XFS's patch is closer to 40000 lines.  Normally, adding a filesystem
>to a kernel tree is not a big deal, but as you might guess getting a
>massive 40,000 line hunk of code past Marcelo is NOT going to happen.  The
>XFS people (or some other kind soul, as I mentioned) need to 1) start
>feeding good bits of the XFS patch to Marcelo and 2) start synching it up
>with his tree so that JUST the filesystem can be merged w/o all the random
>other stuff they touch.

Christoph Hellwig was the man who is submitting lot's of cleaned up stuff 
to Linus which made it into 2.5.36.
The main focus at the moment is bug fixing and cleaning up in general. Both 
in 2.5 as well as in the 2.4 tree.

Don't worry well get there eventually.
The reason for the slow integration with 2.4 is because a lot of the 
interfaces that XFS used were only available in 2.5 or were made as a stand 
alone function inside XFS. The 2.5 code doesn't touch nearly as much as the 
2.4 code does.

>Blah, that is badly written, but whatever, I just woke up.

:-)

Cheers
--
Seth
It might just be your lucky day, if you only knew.

Re: [gentoo-dev] gentoo-sources-r9 kernel sources vs. XFS patch

[John Newman]

On Fri, Oct 25, 2002 at 09:08:19AM +0200, Seth Mos wrote:
> At 11:13 24-10-2002 -0500, Brandon Low wrote:
> XFS is already merged in since 2.5.36. The integration with the 2.4 tree is 
> a ongoing project. It might be integrated into 2.4 in the future when 
> marcelo thinks the time is right.

Speaking of 2.5.X.... does anyone know of a current 2.5.X kernel that is
know to be more-or-less stable and well-performing?  And XFS on-disk
format is the same for 2.5 and 2.4, right?

thanks,

--
john

Re: [gentoo-dev] gentoo-sources-r9 kernel sources vs. XFS patch

[Seth Mos]

On Sat, 26 Oct 2002, John Newman wrote:

> On Fri, Oct 25, 2002 at 09:08:19AM +0200, Seth Mos wrote:
> > At 11:13 24-10-2002 -0500, Brandon Low wrote:
> > XFS is already merged in since 2.5.36. The integration with the 2.4 tree is
> > a ongoing project. It might be integrated into 2.4 in the future when
> > marcelo thinks the time is right.
>
> Speaking of 2.5.X.... does anyone know of a current 2.5.X kernel that is
> know to be more-or-less stable and well-performing?  And XFS on-disk
> format is the same for 2.5 and 2.4, right?

Yes.

I have reports that 2.5.43 seemed to be almost usable for a desktop.
At least it wouldn't eat your filesystems on IDE disks.

Cheers

Re: [gentoo-dev] gentoo-sources-r9 kernel sources vs. XFS patch

[Brandon Low]

>     Hmm.. I haven't heard any success stories; yet. Anyone care to post 
> experiences? Anyway, 2.5.x has a lot of nice additions that we enjoy as 
> 2.4 backport patches in gentoo-sources, but this XFS buisness is 
> dissapointing. I would jump to 2.5.x right now and start testing/helping 

As has been mentioned about 1 dozen times before, patching XFS against 2.4 
is not a job for the weak of heart.  When I first got put in charge of 
managing the releases of gentoo-sources, drobbins and I agreed to drop XFS 
and to maintain it separately in xfs-sources, because that way we could 
offer a more complete and feature rich kernel for gentoo-sources.  I have 
been working hard and so has MJC in order to get XFS back into 
gentoo-sources since there is obvious demand for it, that is why it is 
there as an experimental option for gentoo-sources-2.4.19-r10.  The kernel 
that we are currently working on will hopefully have a better tested and 
better integrated XFS patch, borrowed from andrea archangeli's kernel.  
Speaking of which, we have available for your testing and XFS pleasure in 
the portage tree "aa-sources" which may work for you.

> for a speedy 2.6 release, but nVidia's binary drivers are for 2.4 only 
> and haven't been usable patched (what could be patched) since 
> 2.5.24-dj2. This means that nVidia/XFS users are stuck playing patch 
> limbo. Does anyone have any helpful sollutions? What changes are going 
> to be made in regard to XFS for the next gentoo-sources kernel 
> (2.4.19-r10 w/ use="xfs" will *not* boot for me at all, and 
> 2.4.19-xfs-r2 is not stable enough for my tastes).
>     I would start over with ReiserFS, but I *prefer *XFS and have not 
> had problems with a custom 2.4.18 kernel with *lots* of patches.

In my experience, xfs-sources-2.4.19-r2 is one of the most stable kernels 
I've ever used and I do mean ever.  Of course you seem to have a different 
experience there.  Now this is where we hit a highly opinionated brick 
wall.  carpaski@gentoo.org and I have been using reiserfs on production 
grade machines all over our school for close to a year now (this includes 
the time when reiser was considered 'unstable') and I won't use anything 
else in a production environment any more, because reiser has proven 
itself through kernel crash after kernel crash and school power outage 
after power outage.  The big difference in the kernel world between these 
two filesystems is that reiser works and plays well with the data 
structures present in 2.4 and XFS doesn't, plain and simple.  This makes 
the XFS patch, as I've mentioned before, require modifying tens of 
thousands of lines of kernel core code where other filesystems such as JFS 
or Reiserfs only needed to modify a few hundred (even to include quota and 
other features I believe reiser still modifys less than 2000 lines of core 
code).  

The translation of all this is as follows:  Until you are going to do the 
work to patch XFS into a kernel for us, don't whine too much, we're doing 
our best to get it in, but it tends to destablalize things WAY too much 
for us to make it standard.

--Brandon (This is the last time I will comment about XFS on the lists 
until someone brings up something new, because I've said everything I just 
said above several times before)