[gentoo-server] Issues with GFS.

[gentoo-server] Issues with GFS.

["Todd M. Hébert"]

Hello all, 
I'm hoping someone has dealt with this by now.

The kernel seems only to support GFS2, but the user-space tools seem to only support GFS (GFS1?) 

So.. I am able to format GFS filesystems in userspace, but I cannot mount them because the kernel doesn't understand them. (and the separate GFS kernel module will not compile,.. maybe because the one in the kernel source that handles GFS2 uses some of the same necessary bits?)

I need to find a way to get either GFS or GFS2 working properly. (I'd think GFS2 would be preferable.)

I'm using the 2007.0 profile, and I already have the cluster elements configured, and only need to format & mount the filesystems at this point.

Can anyone point me to a resource that will help me get this last bit sorted?

Thanks,

--Todd

(P.S. Pardon anything below this line.. it is required under Irish law. --TMH)

**********************************************************************
                "Private Confidential & Privileged"
This Email and any files and attachments transmitted with it are
confidential and/or privileged.  They are intended solely for
the use of the intended recipient.  Any views and opinions expressed
are those of the individual author/sender and are not necessarily
shared or endorsed by Independent Newspapers (Ireland) Limited or any
associated or related company.  The content of this Email and any file
or attachment transmitted with it may have been changed or altered
without the consent of the author.  If you are not the intended
recipient, please note that any review, dissemination, disclosure,
alteration, printing, circulation or transmission of this Email and/or
any file or attachment transmitted with it, is prohibited and may be
unlawful.  If you have received this Email or any file attachment
transmitted with it in error, please notify support@iil.ie or contact
Systems Administrator Tel:  +353 (1) 411 2244
Internet Interaction Limited
3050 Lake Drive, Citywest digital Park, Co Dublin
CRO # 247783


-- 
gentoo-server@lists.gentoo.org mailing list

Re: [gentoo-server] Issues with GFS.

[Brian Kroth]

[-- Attachment #1: Type: text/plain, Size: 3524 bytes --]

I have also been dealing with this.  There aren't any gfs2 tools in
portage yet, and as I'm doing a new install I didn't want to go with an
"old" gfs, especially since the kernel module doesn't work with newer
kernels anymore.  I ended up writing my own ebuilds.  Have a look at these
links below for more info.  The offsite link in the forum post is more up
to date (better init scripts).  If you're upgrading from gfs please let me
know how that goes with the new ebuilds.  I think they should work just
fine, but I haven't tested it yet.

Thanks,
Brian

http://bugs.gentoo.org/show_bug.cgi?id=184850
http://forums.gentoo.org/viewtopic-p-4732772.html#4732772
https://mywebspace.wisc.edu/bpkroth/web/gentoo/fs-cluster-ebuilds/fs-cluster_ebuilds.tar.bz2

BTW, I long time ago I posted a question regarding OCFS2 vs. GFS2.  In my
research I decided OCFS2 didn't have all the features I wanted (quotas,
acls, cluster flock, etc.) but I did run some simple benchmarks of how
they respond with multiple clients.  Basically just "iozone -a" on 5 machines at
once.  Here's another link in case anyone's interested.  Sorry for the
format, but I had a student make them "pretty" and he used MS Excel 2007.

https://mywebspace.wisc.edu/bpkroth/web/fs-cluster/fs-cluster-iozone-results.xls


"Todd M. Hébert" <todd@iil.ie>:
> Hello all, 
> I'm hoping someone has dealt with this by now.
> 
> The kernel seems only to support GFS2, but the user-space tools seem to only support GFS (GFS1?) 
> 
> So.. I am able to format GFS filesystems in userspace, but I cannot mount them because the kernel doesn't understand them. (and the separate GFS kernel module will not compile,.. maybe because the one in the kernel source that handles GFS2 uses some of the same necessary bits?)
> 
> I need to find a way to get either GFS or GFS2 working properly. (I'd think GFS2 would be preferable.)
> 
> I'm using the 2007.0 profile, and I already have the cluster elements configured, and only need to format & mount the filesystems at this point.
> 
> Can anyone point me to a resource that will help me get this last bit sorted?
> 
> Thanks,
> 
> --Todd
> 
> (P.S. Pardon anything below this line.. it is required under Irish law. --TMH)
> 
> **********************************************************************
>                 "Private Confidential & Privileged"
> This Email and any files and attachments transmitted with it are
> confidential and/or privileged.  They are intended solely for
> the use of the intended recipient.  Any views and opinions expressed
> are those of the individual author/sender and are not necessarily
> shared or endorsed by Independent Newspapers (Ireland) Limited or any
> associated or related company.  The content of this Email and any file
> or attachment transmitted with it may have been changed or altered
> without the consent of the author.  If you are not the intended
> recipient, please note that any review, dissemination, disclosure,
> alteration, printing, circulation or transmission of this Email and/or
> any file or attachment transmitted with it, is prohibited and may be
> unlawful.  If you have received this Email or any file attachment
> transmitted with it in error, please notify support@iil.ie or contact
> Systems Administrator Tel:  +353 (1) 411 2244
> Internet Interaction Limited
> 3050 Lake Drive, Citywest digital Park, Co Dublin
> CRO # 247783
> 
> 
> -- 
> gentoo-server@lists.gentoo.org mailing list
> 

[-- Attachment #2: smime.p7s --]
[-- Type: application/x-pkcs7-signature, Size: 2192 bytes --]

Re: [gentoo-server] Issues with GFS2.

["Todd M. Hébert"]

Thanks Brian, there was some great info.. and even greater ebuilds.

I've gotten things working.. but with some caveats.  Locking isn't working properly.

I can get filesystems mounted on both servers just fine.. I can copy a file into the shared space on one server, copy it out on the other, even delete them.  (copy from a, delete from b, copy from b, delete from a.)

BUT... as soon as I copy in several files and issue a rm /<mount-point>/*

everything stops working for that mount. (Other mounts will continue to work until a similar command is issued)

It appears that anything trying to access a LIST of files causes a serious problem, and everything related to the affected mount just freezes.

The /var/log/messages file shows that it freezes because it's waiting for locks on the files to clear.

I'm getting no compiling errors.

Has anyone seen similar problems or even found a way to fix this?

Thanks in advance for any further help!

--Todd

(P.S. please excuse the company information at the bottom.. it's required under Irish law.)

Brian Kroth wrote:
> I have also been dealing with this.  There aren't any gfs2 tools in
> portage yet, and as I'm doing a new install I didn't want to go with an
> "old" gfs, especially since the kernel module doesn't work with newer
> kernels anymore.  I ended up writing my own ebuilds.  Have a look at these
> links below for more info.  The offsite link in the forum post is more up
> to date (better init scripts).  If you're upgrading from gfs please let me
> know how that goes with the new ebuilds.  I think they should work just
> fine, but I haven't tested it yet.
> 
> Thanks,
> Brian
> 
> http://bugs.gentoo.org/show_bug.cgi?id=184850
> http://forums.gentoo.org/viewtopic-p-4732772.html#4732772
> https://mywebspace.wisc.edu/bpkroth/web/gentoo/fs-cluster-ebuilds/fs-cluster_ebuilds.tar.bz2
> 
> BTW, I long time ago I posted a question regarding OCFS2 vs. GFS2.  In my
> research I decided OCFS2 didn't have all the features I wanted (quotas,
> acls, cluster flock, etc.) but I did run some simple benchmarks of how
> they respond with multiple clients.  Basically just "iozone -a" on 5 machines at
> once.  Here's another link in case anyone's interested.  Sorry for the
> format, but I had a student make them "pretty" and he used MS Excel 2007.
> 
> https://mywebspace.wisc.edu/bpkroth/web/fs-cluster/fs-cluster-iozone-results.xls
> 
> 
> "Todd M. Hébert" <todd@iil.ie>:
>> Hello all, 
>> I'm hoping someone has dealt with this by now.
>>
>> The kernel seems only to support GFS2, but the user-space tools seem to only support GFS (GFS1?) 
>>
>> So.. I am able to format GFS filesystems in userspace, but I cannot mount them because the kernel doesn't understand them. (and the separate GFS kernel module will not compile,.. maybe because the one in the kernel source that handles GFS2 uses some of the same necessary bits?)
>>
>> I need to find a way to get either GFS or GFS2 working properly. (I'd think GFS2 would be preferable.)
>>
>> I'm using the 2007.0 profile, and I already have the cluster elements configured, and only need to format & mount the filesystems at this point.
>>
>> Can anyone point me to a resource that will help me get this last bit sorted?
>>
>> Thanks,
>>
>> --Todd
>>
>> (P.S. Pardon anything below this line.. it is required under Irish law. --TMH)
>>
>> **********************************************************************
>>                 "Private Confidential & Privileged"
>> This Email and any files and attachments transmitted with it are
>> confidential and/or privileged.  They are intended solely for
>> the use of the intended recipient.  Any views and opinions expressed
>> are those of the individual author/sender and are not necessarily
>> shared or endorsed by Independent Newspapers (Ireland) Limited or any
>> associated or related company.  The content of this Email and any file
>> or attachment transmitted with it may have been changed or altered
>> without the consent of the author.  If you are not the intended
>> recipient, please note that any review, dissemination, disclosure,
>> alteration, printing, circulation or transmission of this Email and/or
>> any file or attachment transmitted with it, is prohibited and may be
>> unlawful.  If you have received this Email or any file attachment
>> transmitted with it in error, please notify support@iil.ie or contact
>> Systems Administrator Tel:  +353 (1) 411 2244
>> Internet Interaction Limited
>> 3050 Lake Drive, Citywest digital Park, Co Dublin
>> CRO # 247783
>>
>>
>> -- 
>> gentoo-server@lists.gentoo.org mailing list
>>



**********************************************************************
                "Private Confidential & Privileged"
This Email and any files and attachments transmitted with it are
confidential and/or privileged.  They are intended solely for
the use of the intended recipient.  Any views and opinions expressed
are those of the individual author/sender and are not necessarily
shared or endorsed by Independent Newspapers (Ireland) Limited or any
associated or related company.  The content of this Email and any file
or attachment transmitted with it may have been changed or altered
without the consent of the author.  If you are not the intended
recipient, please note that any review, dissemination, disclosure,
alteration, printing, circulation or transmission of this Email and/or
any file or attachment transmitted with it, is prohibited and may be
unlawful.  If you have received this Email or any file attachment
transmitted with it in error, please notify support@iil.ie or contact
Systems Administrator Tel:  +353 (1) 411 2244
Internet Interaction Limited
3050 Lake Drive, Citywest digital Park, Co Dublin
CRO # 247783


-- 
gentoo-server@lists.gentoo.org mailing list

Re: [gentoo-server] Issues with GFS2.

[Brian Kroth]

[-- Attachment #1: Type: text/plain, Size: 6582 bytes --]

I noticed that problem once as well, but it was while issuing the "rm *" command
on the shared space from cssh to 5 machines at once, which seems more
understandable that you'd see a problem doing it that way.  However, it should
be able to handle that and I haven't been able to reproduce it yet, so I
couldn't track down what actually caused it.  I can send you my config details
if you'd like to compare.

Brian

"Todd M. Hébert" <todd@iil.ie>:
> Thanks Brian, there was some great info.. and even greater ebuilds.
>
> I've gotten things working.. but with some caveats.  Locking isn't working 
> properly.
>
> I can get filesystems mounted on both servers just fine.. I can copy a file 
> into the shared space on one server, copy it out on the other, even delete 
> them.  (copy from a, delete from b, copy from b, delete from a.)
>
> BUT... as soon as I copy in several files and issue a rm /<mount-point>/*
>
> everything stops working for that mount. (Other mounts will continue to 
> work until a similar command is issued)
>
> It appears that anything trying to access a LIST of files causes a serious 
> problem, and everything related to the affected mount just freezes.
>
> The /var/log/messages file shows that it freezes because it's waiting for 
> locks on the files to clear.
>
> I'm getting no compiling errors.
>
> Has anyone seen similar problems or even found a way to fix this?
>
> Thanks in advance for any further help!
>
> --Todd
>
> (P.S. please excuse the company information at the bottom.. it's required 
> under Irish law.)
>
> Brian Kroth wrote:
>> I have also been dealing with this.  There aren't any gfs2 tools in
>> portage yet, and as I'm doing a new install I didn't want to go with an
>> "old" gfs, especially since the kernel module doesn't work with newer
>> kernels anymore.  I ended up writing my own ebuilds.  Have a look at these
>> links below for more info.  The offsite link in the forum post is more up
>> to date (better init scripts).  If you're upgrading from gfs please let me
>> know how that goes with the new ebuilds.  I think they should work just
>> fine, but I haven't tested it yet.
>> Thanks,
>> Brian
>> http://bugs.gentoo.org/show_bug.cgi?id=184850
>> http://forums.gentoo.org/viewtopic-p-4732772.html#4732772
>> https://mywebspace.wisc.edu/bpkroth/web/gentoo/fs-cluster-ebuilds/fs-cluster_ebuilds.tar.bz2
>> BTW, I long time ago I posted a question regarding OCFS2 vs. GFS2.  In my
>> research I decided OCFS2 didn't have all the features I wanted (quotas,
>> acls, cluster flock, etc.) but I did run some simple benchmarks of how
>> they respond with multiple clients.  Basically just "iozone -a" on 5 
>> machines at
>> once.  Here's another link in case anyone's interested.  Sorry for the
>> format, but I had a student make them "pretty" and he used MS Excel 2007.
>> https://mywebspace.wisc.edu/bpkroth/web/fs-cluster/fs-cluster-iozone-results.xls
>> "Todd M. Hébert" <todd@iil.ie>:
>>> Hello all, I'm hoping someone has dealt with this by now.
>>>
>>> The kernel seems only to support GFS2, but the user-space tools seem to 
>>> only support GFS (GFS1?) 
>>> So.. I am able to format GFS filesystems in userspace, but I cannot mount 
>>> them because the kernel doesn't understand them. (and the separate GFS 
>>> kernel module will not compile,.. maybe because the one in the kernel 
>>> source that handles GFS2 uses some of the same necessary bits?)
>>>
>>> I need to find a way to get either GFS or GFS2 working properly. (I'd 
>>> think GFS2 would be preferable.)
>>>
>>> I'm using the 2007.0 profile, and I already have the cluster elements 
>>> configured, and only need to format & mount the filesystems at this 
>>> point.
>>>
>>> Can anyone point me to a resource that will help me get this last bit 
>>> sorted?
>>>
>>> Thanks,
>>>
>>> --Todd
>>>
>>> (P.S. Pardon anything below this line.. it is required under Irish law. 
>>> --TMH)
>>>
>>> **********************************************************************
>>>                 "Private Confidential & Privileged"
>>> This Email and any files and attachments transmitted with it are
>>> confidential and/or privileged.  They are intended solely for
>>> the use of the intended recipient.  Any views and opinions expressed
>>> are those of the individual author/sender and are not necessarily
>>> shared or endorsed by Independent Newspapers (Ireland) Limited or any
>>> associated or related company.  The content of this Email and any file
>>> or attachment transmitted with it may have been changed or altered
>>> without the consent of the author.  If you are not the intended
>>> recipient, please note that any review, dissemination, disclosure,
>>> alteration, printing, circulation or transmission of this Email and/or
>>> any file or attachment transmitted with it, is prohibited and may be
>>> unlawful.  If you have received this Email or any file attachment
>>> transmitted with it in error, please notify support@iil.ie or contact
>>> Systems Administrator Tel:  +353 (1) 411 2244
>>> Internet Interaction Limited
>>> 3050 Lake Drive, Citywest digital Park, Co Dublin
>>> CRO # 247783
>>>
>>>
>>> -- 
>>> gentoo-server@lists.gentoo.org mailing list
>>>
>
>
>
> **********************************************************************
>                "Private Confidential & Privileged"
> This Email and any files and attachments transmitted with it are
> confidential and/or privileged.  They are intended solely for
> the use of the intended recipient.  Any views and opinions expressed
> are those of the individual author/sender and are not necessarily
> shared or endorsed by Independent Newspapers (Ireland) Limited or any
> associated or related company.  The content of this Email and any file
> or attachment transmitted with it may have been changed or altered
> without the consent of the author.  If you are not the intended
> recipient, please note that any review, dissemination, disclosure,
> alteration, printing, circulation or transmission of this Email and/or
> any file or attachment transmitted with it, is prohibited and may be
> unlawful.  If you have received this Email or any file attachment
> transmitted with it in error, please notify support@iil.ie or contact
> Systems Administrator Tel:  +353 (1) 411 2244
> Internet Interaction Limited
> 3050 Lake Drive, Citywest digital Park, Co Dublin
> CRO # 247783
>
>
> -- 
> gentoo-server@lists.gentoo.org mailing list
>

[-- Attachment #2: smime.p7s --]
[-- Type: application/x-pkcs7-signature, Size: 2192 bytes --]

Re: [gentoo-server] Issues with GFS2.

["Todd M. Hébert"]

Pretty please with sugar on top? =D

--Todd

Brian Kroth wrote:
> I noticed that problem once as well, but it was while issuing the "rm *" command
> on the shared space from cssh to 5 machines at once, which seems more
> understandable that you'd see a problem doing it that way.  However, it should
> be able to handle that and I haven't been able to reproduce it yet, so I
> couldn't track down what actually caused it.  I can send you my config details
> if you'd like to compare.
> 
> Brian
> 
> "Todd M. Hébert" <todd@iil.ie>:
>> Thanks Brian, there was some great info.. and even greater ebuilds.
>>
>> I've gotten things working.. but with some caveats.  Locking isn't working 
>> properly.
>>
>> I can get filesystems mounted on both servers just fine.. I can copy a file 
>> into the shared space on one server, copy it out on the other, even delete 
>> them.  (copy from a, delete from b, copy from b, delete from a.)
>>
>> BUT... as soon as I copy in several files and issue a rm /<mount-point>/*
>>
>> everything stops working for that mount. (Other mounts will continue to 
>> work until a similar command is issued)
>>
>> It appears that anything trying to access a LIST of files causes a serious 
>> problem, and everything related to the affected mount just freezes.
>>
>> The /var/log/messages file shows that it freezes because it's waiting for 
>> locks on the files to clear.
>>
>> I'm getting no compiling errors.
>>
>> Has anyone seen similar problems or even found a way to fix this?
>>
>> Thanks in advance for any further help!
>>
>> --Todd
>>
>> (P.S. please excuse the company information at the bottom.. it's required 
>> under Irish law.)
>>
>> Brian Kroth wrote:
>>> I have also been dealing with this.  There aren't any gfs2 tools in
>>> portage yet, and as I'm doing a new install I didn't want to go with an
>>> "old" gfs, especially since the kernel module doesn't work with newer
>>> kernels anymore.  I ended up writing my own ebuilds.  Have a look at these
>>> links below for more info.  The offsite link in the forum post is more up
>>> to date (better init scripts).  If you're upgrading from gfs please let me
>>> know how that goes with the new ebuilds.  I think they should work just
>>> fine, but I haven't tested it yet.
>>> Thanks,
>>> Brian
>>> http://bugs.gentoo.org/show_bug.cgi?id=184850
>>> http://forums.gentoo.org/viewtopic-p-4732772.html#4732772
>>> https://mywebspace.wisc.edu/bpkroth/web/gentoo/fs-cluster-ebuilds/fs-cluster_ebuilds.tar.bz2
>>> BTW, I long time ago I posted a question regarding OCFS2 vs. GFS2.  In my
>>> research I decided OCFS2 didn't have all the features I wanted (quotas,
>>> acls, cluster flock, etc.) but I did run some simple benchmarks of how
>>> they respond with multiple clients.  Basically just "iozone -a" on 5 
>>> machines at
>>> once.  Here's another link in case anyone's interested.  Sorry for the
>>> format, but I had a student make them "pretty" and he used MS Excel 2007.
>>> https://mywebspace.wisc.edu/bpkroth/web/fs-cluster/fs-cluster-iozone-results.xls
>>> "Todd M. Hébert" <todd@iil.ie>:
>>>> Hello all, I'm hoping someone has dealt with this by now.
>>>>
>>>> The kernel seems only to support GFS2, but the user-space tools seem to 
>>>> only support GFS (GFS1?) 
>>>> So.. I am able to format GFS filesystems in userspace, but I cannot mount 
>>>> them because the kernel doesn't understand them. (and the separate GFS 
>>>> kernel module will not compile,.. maybe because the one in the kernel 
>>>> source that handles GFS2 uses some of the same necessary bits?)
>>>>
>>>> I need to find a way to get either GFS or GFS2 working properly. (I'd 
>>>> think GFS2 would be preferable.)
>>>>
>>>> I'm using the 2007.0 profile, and I already have the cluster elements 
>>>> configured, and only need to format & mount the filesystems at this 
>>>> point.
>>>>
>>>> Can anyone point me to a resource that will help me get this last bit 
>>>> sorted?
>>>>
>>>> Thanks,
>>>>
>>>> --Todd
>>>>
>>>> (P.S. Pardon anything below this line.. it is required under Irish law. 
>>>> --TMH)

**********************************************************************
                "Private Confidential & Privileged"
This Email and any files and attachments transmitted with it are
confidential and/or privileged.  They are intended solely for
the use of the intended recipient.  Any views and opinions expressed
are those of the individual author/sender and are not necessarily
shared or endorsed by Independent Newspapers (Ireland) Limited or any
associated or related company.  The content of this Email and any file
or attachment transmitted with it may have been changed or altered
without the consent of the author.  If you are not the intended
recipient, please note that any review, dissemination, disclosure,
alteration, printing, circulation or transmission of this Email and/or
any file or attachment transmitted with it, is prohibited and may be
unlawful.  If you have received this Email or any file attachment
transmitted with it in error, please notify support@iil.ie or contact
Systems Administrator Tel:  +353 (1) 411 2244
Internet Interaction Limited
3050 Lake Drive, Citywest digital Park, Co Dublin
CRO # 247783


-- 
gentoo-server@lists.gentoo.org mailing list

Re: [gentoo-server] Issues with GFS2.

["Todd M. Hébert"]

Brian,
I would appreciate greatly if you could send those config details.

It's really the elements around locking & fencing that seems to be causing issues. (although I don't think I really need fencing.)

Thanks in advance!

--Todd (todd at iil.ie)

Brian Kroth wrote:
> I noticed that problem once as well, but it was while issuing the "rm *" command
> on the shared space from cssh to 5 machines at once, which seems more
> understandable that you'd see a problem doing it that way.  However, it should
> be able to handle that and I haven't been able to reproduce it yet, so I
> couldn't track down what actually caused it.  I can send you my config details
> if you'd like to compare.
> 
> Brian
> 
> "Todd M. Hébert" <todd@iil.ie>:
>> Thanks Brian, there was some great info.. and even greater ebuilds.
>>
>> I've gotten things working.. but with some caveats.  Locking isn't working 
>> properly.
>>
>> I can get filesystems mounted on both servers just fine.. I can copy a file 
>> into the shared space on one server, copy it out on the other, even delete 
>> them.  (copy from a, delete from b, copy from b, delete from a.)
>>
>> BUT... as soon as I copy in several files and issue a rm /<mount-point>/*
>>
>> everything stops working for that mount. (Other mounts will continue to 
>> work until a similar command is issued)
>>
>> It appears that anything trying to access a LIST of files causes a serious 
>> problem, and everything related to the affected mount just freezes.
>>
>> The /var/log/messages file shows that it freezes because it's waiting for 
>> locks on the files to clear.
>>
>> I'm getting no compiling errors.
>>
>> Has anyone seen similar problems or even found a way to fix this?
>>
>> Thanks in advance for any further help!
>>
>> --Todd
>>
>> (P.S. please excuse the company information at the bottom.. it's required 
>> under Irish law.)
>>
>> Brian Kroth wrote:
>>> I have also been dealing with this.  There aren't any gfs2 tools in
>>> portage yet, and as I'm doing a new install I didn't want to go with an
>>> "old" gfs, especially since the kernel module doesn't work with newer
>>> kernels anymore.  I ended up writing my own ebuilds.  Have a look at these
>>> links below for more info.  The offsite link in the forum post is more up
>>> to date (better init scripts).  If you're upgrading from gfs please let me
>>> know how that goes with the new ebuilds.  I think they should work just
>>> fine, but I haven't tested it yet.
>>> Thanks,
>>> Brian
>>> http://bugs.gentoo.org/show_bug.cgi?id=184850
>>> http://forums.gentoo.org/viewtopic-p-4732772.html#4732772
>>> https://mywebspace.wisc.edu/bpkroth/web/gentoo/fs-cluster-ebuilds/fs-cluster_ebuilds.tar.bz2
>>> BTW, I long time ago I posted a question regarding OCFS2 vs. GFS2.  In my
>>> research I decided OCFS2 didn't have all the features I wanted (quotas,
>>> acls, cluster flock, etc.) but I did run some simple benchmarks of how
>>> they respond with multiple clients.  Basically just "iozone -a" on 5 
>>> machines at
>>> once.  Here's another link in case anyone's interested.  Sorry for the
>>> format, but I had a student make them "pretty" and he used MS Excel 2007.
>>> https://mywebspace.wisc.edu/bpkroth/web/fs-cluster/fs-cluster-iozone-results.xls
>>> "Todd M. Hébert" <todd@iil.ie>:
>>>> Hello all, I'm hoping someone has dealt with this by now.
>>>>
>>>> The kernel seems only to support GFS2, but the user-space tools seem to 
>>>> only support GFS (GFS1?) 
>>>> So.. I am able to format GFS filesystems in userspace, but I cannot mount 
>>>> them because the kernel doesn't understand them. (and the separate GFS 
>>>> kernel module will not compile,.. maybe because the one in the kernel 
>>>> source that handles GFS2 uses some of the same necessary bits?)
>>>>
>>>> I need to find a way to get either GFS or GFS2 working properly. (I'd 
>>>> think GFS2 would be preferable.)
>>>>
>>>> I'm using the 2007.0 profile, and I already have the cluster elements 
>>>> configured, and only need to format & mount the filesystems at this 
>>>> point.
>>>>
>>>> Can anyone point me to a resource that will help me get this last bit 
>>>> sorted?
>>>>
>>>> Thanks,
>>>>
>>>> --Todd
>>>>
>>>> (P.S. Pardon anything below this line.. it is required under Irish law. 
>>>> --TMH)
>>>>


**********************************************************************
                "Private Confidential & Privileged"
This Email and any files and attachments transmitted with it are
confidential and/or privileged.  They are intended solely for
the use of the intended recipient.  Any views and opinions expressed
are those of the individual author/sender and are not necessarily
shared or endorsed by Independent Newspapers (Ireland) Limited or any
associated or related company.  The content of this Email and any file
or attachment transmitted with it may have been changed or altered
without the consent of the author.  If you are not the intended
recipient, please note that any review, dissemination, disclosure,
alteration, printing, circulation or transmission of this Email and/or
any file or attachment transmitted with it, is prohibited and may be
unlawful.  If you have received this Email or any file attachment
transmitted with it in error, please notify support@iil.ie or contact
Systems Administrator Tel:  +353 (1) 411 2244
Internet Interaction Limited
3050 Lake Drive, Citywest digital Park, Co Dublin
CRO # 247783


-- 
gentoo-server@lists.gentoo.org mailing list

Re: [gentoo-server] Issues with GFS2.

[Brian Kroth]

[-- Attachment #1: Type: text/plain, Size: 7155 bytes --]

Sorry for the late reply.  I was digging into your problem a little
further and was also able to reproduce the problem.  Basically something
like this from cssh so that it happens on all machines at once:

# Make some files - this works.
for i in `seq 1 1000`; do echo $HOSTNAME: $RANDOM >> $i.txt; done
# cat *.txt to verify that it actually works

# Delete them.  This should work, but generally deadlocks.
rm *.txt

I haven't been able to get any debug info from it yet.

I did some searching on RedHat's bugzilla and it looks like others have
experienced something like this as well with RHEL.  It was enough to
convince me that the devs were probably right and to stick with GFS1,
which with this 2.01.00 set of ebuilds builds on newer kernels and hasn't
shown problems like this yet (see /usr/share/doc/gfs/usage.txt for
details).  With the updated framework conversion from GFS1 to GFS2 should
be easier as well.

Anyways, I'm fairly certain this is a problem with GFS2 and not those
ebuilds so I doubt this is really the place to discuss it.  I'd be willing
to continue it offlist if you want, but I'm obviously no expert either.

Brian

https://mywebspace.wisc.edu/bpkroth/web/gentoo/fs-cluster-ebuilds/gfs-configs.tar.bz2

"Todd M. Hébert" <todd@iil.ie>:
> Brian,
> I would appreciate greatly if you could send those config details.
> 
> It's really the elements around locking & fencing that seems to be causing issues. (although I don't think I really need fencing.)
> 
> Thanks in advance!
> 
> --Todd (todd at iil.ie)
> 
> Brian Kroth wrote:
> > I noticed that problem once as well, but it was while issuing the "rm *" command
> > on the shared space from cssh to 5 machines at once, which seems more
> > understandable that you'd see a problem doing it that way.  However, it should
> > be able to handle that and I haven't been able to reproduce it yet, so I
> > couldn't track down what actually caused it.  I can send you my config details
> > if you'd like to compare.
> > 
> > Brian
> > 
> > "Todd M. Hébert" <todd@iil.ie>:
> >> Thanks Brian, there was some great info.. and even greater ebuilds.
> >>
> >> I've gotten things working.. but with some caveats.  Locking isn't working 
> >> properly.
> >>
> >> I can get filesystems mounted on both servers just fine.. I can copy a file 
> >> into the shared space on one server, copy it out on the other, even delete 
> >> them.  (copy from a, delete from b, copy from b, delete from a.)
> >>
> >> BUT... as soon as I copy in several files and issue a rm /<mount-point>/*
> >>
> >> everything stops working for that mount. (Other mounts will continue to 
> >> work until a similar command is issued)
> >>
> >> It appears that anything trying to access a LIST of files causes a serious 
> >> problem, and everything related to the affected mount just freezes.
> >>
> >> The /var/log/messages file shows that it freezes because it's waiting for 
> >> locks on the files to clear.
> >>
> >> I'm getting no compiling errors.
> >>
> >> Has anyone seen similar problems or even found a way to fix this?
> >>
> >> Thanks in advance for any further help!
> >>
> >> --Todd
> >>
> >> (P.S. please excuse the company information at the bottom.. it's required 
> >> under Irish law.)
> >>
> >> Brian Kroth wrote:
> >>> I have also been dealing with this.  There aren't any gfs2 tools in
> >>> portage yet, and as I'm doing a new install I didn't want to go with an
> >>> "old" gfs, especially since the kernel module doesn't work with newer
> >>> kernels anymore.  I ended up writing my own ebuilds.  Have a look at these
> >>> links below for more info.  The offsite link in the forum post is more up
> >>> to date (better init scripts).  If you're upgrading from gfs please let me
> >>> know how that goes with the new ebuilds.  I think they should work just
> >>> fine, but I haven't tested it yet.
> >>> Thanks,
> >>> Brian
> >>> http://bugs.gentoo.org/show_bug.cgi?id=184850
> >>> http://forums.gentoo.org/viewtopic-p-4732772.html#4732772
> >>> https://mywebspace.wisc.edu/bpkroth/web/gentoo/fs-cluster-ebuilds/fs-cluster_ebuilds.tar.bz2
> >>> BTW, I long time ago I posted a question regarding OCFS2 vs. GFS2.  In my
> >>> research I decided OCFS2 didn't have all the features I wanted (quotas,
> >>> acls, cluster flock, etc.) but I did run some simple benchmarks of how
> >>> they respond with multiple clients.  Basically just "iozone -a" on 5 
> >>> machines at
> >>> once.  Here's another link in case anyone's interested.  Sorry for the
> >>> format, but I had a student make them "pretty" and he used MS Excel 2007.
> >>> https://mywebspace.wisc.edu/bpkroth/web/fs-cluster/fs-cluster-iozone-results.xls
> >>> "Todd M. Hébert" <todd@iil.ie>:
> >>>> Hello all, I'm hoping someone has dealt with this by now.
> >>>>
> >>>> The kernel seems only to support GFS2, but the user-space tools seem to 
> >>>> only support GFS (GFS1?) 
> >>>> So.. I am able to format GFS filesystems in userspace, but I cannot mount 
> >>>> them because the kernel doesn't understand them. (and the separate GFS 
> >>>> kernel module will not compile,.. maybe because the one in the kernel 
> >>>> source that handles GFS2 uses some of the same necessary bits?)
> >>>>
> >>>> I need to find a way to get either GFS or GFS2 working properly. (I'd 
> >>>> think GFS2 would be preferable.)
> >>>>
> >>>> I'm using the 2007.0 profile, and I already have the cluster elements 
> >>>> configured, and only need to format & mount the filesystems at this 
> >>>> point.
> >>>>
> >>>> Can anyone point me to a resource that will help me get this last bit 
> >>>> sorted?
> >>>>
> >>>> Thanks,
> >>>>
> >>>> --Todd
> >>>>
> >>>> (P.S. Pardon anything below this line.. it is required under Irish law. 
> >>>> --TMH)
> >>>>
> 
> 
> **********************************************************************
>                 "Private Confidential & Privileged"
> This Email and any files and attachments transmitted with it are
> confidential and/or privileged.  They are intended solely for
> the use of the intended recipient.  Any views and opinions expressed
> are those of the individual author/sender and are not necessarily
> shared or endorsed by Independent Newspapers (Ireland) Limited or any
> associated or related company.  The content of this Email and any file
> or attachment transmitted with it may have been changed or altered
> without the consent of the author.  If you are not the intended
> recipient, please note that any review, dissemination, disclosure,
> alteration, printing, circulation or transmission of this Email and/or
> any file or attachment transmitted with it, is prohibited and may be
> unlawful.  If you have received this Email or any file attachment
> transmitted with it in error, please notify support@iil.ie or contact
> Systems Administrator Tel:  +353 (1) 411 2244
> Internet Interaction Limited
> 3050 Lake Drive, Citywest digital Park, Co Dublin
> CRO # 247783
> 
> 
> -- 
> gentoo-server@lists.gentoo.org mailing list
> 

[-- Attachment #2: smime.p7s --]
[-- Type: application/x-pkcs7-signature, Size: 2192 bytes --]