[gentoo-user] [OT] NFSv4: 32-bit server versus 64-bit client?

[gentoo-user] [OT] NFSv4: 32-bit server versus 64-bit client?

[walt]

I'm trying to be a good gentoo netizen by nfs-sharing /usr/portage between
my three local gentoo machines, and failing :(

After weeks of fiddling, I discovered today that my problems come from
using a 32-bit machine to serve my two 64-bit NFS clients(!)

(I'll mention up front that NFSv3 works perfectly -- only NFSv4 is bad.)

For reasons I don't know, the 64-bit client machines mount the 32-bit
NFSv4 share with UID/GID 0xffffffe, which won't let even root write to
the rw share.

I googled an old thread mentioning that 0xffff is decimal 65534, a UID
traditionally assigned to the user 'nobody'.

Can anyone else reproduce my problem, or give a hint how to work around
it?

(This list is so quiet today I'm wondering if gmane.org is down.)

Re: [gentoo-user] [OT] NFSv4: 32-bit server versus 64-bit client?

[Todd Goodman]

* walt <w41ter@gmail.com> [110804 17:26]:
> I'm trying to be a good gentoo netizen by nfs-sharing /usr/portage between
> my three local gentoo machines, and failing :(
> 
> After weeks of fiddling, I discovered today that my problems come from
> using a 32-bit machine to serve my two 64-bit NFS clients(!)
> 
> (I'll mention up front that NFSv3 works perfectly -- only NFSv4 is bad.)
> 
> For reasons I don't know, the 64-bit client machines mount the 32-bit
> NFSv4 share with UID/GID 0xffffffe, which won't let even root write to
> the rw share.
> 
> I googled an old thread mentioning that 0xffff is decimal 65534, a UID
> traditionally assigned to the user 'nobody'.
> 
> Can anyone else reproduce my problem, or give a hint how to work around
> it?
> 
> (This list is so quiet today I'm wondering if gmane.org is down.)
> 

Does using the no_root_squash export option help on the machine
exporting the filesystem?  (See man exports)

Todd

Re: [gentoo-user] [OT] NFSv4: 32-bit server versus 64-bit client?

[Joost Roeleveld]

On Thursday, August 04, 2011 02:53:28 PM walt wrote:
> I'm trying to be a good gentoo netizen by nfs-sharing /usr/portage between
> my three local gentoo machines, and failing :(
> 
> After weeks of fiddling, I discovered today that my problems come from
> using a 32-bit machine to serve my two 64-bit NFS clients(!)
> 
> (I'll mention up front that NFSv3 works perfectly -- only NFSv4 is bad.)
> 
> For reasons I don't know, the 64-bit client machines mount the 32-bit
> NFSv4 share with UID/GID 0xffffffe, which won't let even root write to
> the rw share.
> 
> I googled an old thread mentioning that 0xffff is decimal 65534, a UID
> traditionally assigned to the user 'nobody'.
> 
> Can anyone else reproduce my problem, or give a hint how to work around
> it?

This is how I do this:
** (On server)
~ $ cat /etc/exports 
/usr/portage            
*(rw,sync,all_squash,anonuid=250,anongid=250,no_subtree_check)

~ $ id portage
uid=250(portage) gid=250(portage) groups=250(portage)
**

** (On client)
~ $ cat /etc/fstab
server:/usr/portage                 /usr/portage            nfs     
tcp,nodev,nosuid        0 0
**

I stripped non-related parts from the above and the "exports" and "fstab" 
lines should be on a single line.

The bit in the exports-file will force all access to "/usr/portage" to be 
linked to the uid and gid of portage. (Change the values if your portage-user 
has a different uid and/or gid)

I have also opened it to all servers (the * at the beginning of the line) with 
a firewall limiting access.

> (This list is so quiet today I'm wondering if gmane.org is down.)

Or simply busy? :)

Hope this helps,

Joost

[gentoo-user] Re: [OT] NFSv4: 32-bit server versus 64-bit client?

[walt]

On 08/05/2011 12:10 AM, Joost Roeleveld wrote:
> On Thursday, August 04, 2011 02:53:28 PM walt wrote:

>> For reasons I don't know, the 64-bit client machines mount the 32-bit
>> NFSv4 share with UID/GID 0xffffffe, which won't let even root write to
>> the rw share.
>>
>> I googled an old thread mentioning that 0xffff is decimal 65534, a UID
>> traditionally assigned to the user 'nobody'.
> 
> This is how I do this:
> ** (On server)
> ~ $ cat /etc/exports 
> /usr/portage            
> *(rw,sync,all_squash,anonuid=250,anongid=250,no_subtree_check)

(Thanks also to Todd and Victor for their replies.  I hope this will
address your points too.)

Joost, here's what I'm using at the moment:

#grep portage /etc/exports

/usr/portage    192.168.0.100/29(rw,sync,all_squash,anonuid=250,anongid=250,no_subtree_check)

This is what my 32-bit server thinks:

#exportfs -v | grep portage

/usr/portage    192.168.0.100/29(rw,wdelay,root_squash,all_squash,no_subtree_check,anonuid=250,anongid=250)

Unfortunately, this is what the 64-bit client thinks:

#mount | grep portage

k2:/usr/portage on /mnt/nfs type nfs (rw,vers=4,addr=192.168.0.100,clientaddr=192.168.0.102)

#ls -la /mnt/nfs | head

total 2120
drwxr-xr-x  164 4294967294 4294967294    4096 Aug  5 04:32 .
drwxr-xr-x    5 root       root          4096 Jun 17  2008 ..
-rw-r--r--    1 4294967294 4294967294 1258556 Aug  5 04:33 .ebuild.x
drwxr-xr-x   44 4294967294 4294967294    4096 Aug  5 03:31 app-accessibility
drwxr-xr-x  198 4294967294 4294967294    4096 Aug  5 03:31 app-admin
drwxr-xr-x    3 4294967294 4294967294    4096 Aug  5 03:31 app-antivirus
drwxr-xr-x   92 4294967294 4294967294    4096 Aug  5 03:31 app-arch
drwxr-xr-x   36 4294967294 4294967294    4096 Aug  5 03:31 app-backup
drwxr-xr-x   30 4294967294 4294967294    4096 Aug  5 03:31 app-benchmarks

When I look at the uid/gid of that mount, my gut tells me there's a bug in there
somewhere.  Do you disagree?

I have only one 32-bit machine, so I'm thinking I can install a 32-bit gentoo
virtualbox guest to test my 32-bit versus 64-bit theory.  That's going to wait
for tomorrow, though.

Meanwhile, anyone know of a live CD that would let me mount an NFS4 share?  My
ubuntu vbox guest supports NFS3 but not NFS4 :(

Re: [gentoo-user] [OT] NFSv4: 32-bit server versus 64-bit client?

[victor romanchuk]

> I'm trying to be a good gentoo netizen by nfs-sharing /usr/portage between
> my three local gentoo machines, and failing :(
>
> After weeks of fiddling, I discovered today that my problems come from
> using a 32-bit machine to serve my two 64-bit NFS clients(!)
>
> (I'll mention up front that NFSv3 works perfectly -- only NFSv4 is bad.)
>

this is due to different authentication methods used in nfs3 and nfs4 and does
not rely on installation arch (32/64bit). you have to tune up nfs4
infrastructure. on both client and server make sure you have

- nfs4 and inotify support in kernel
- net-fs/nfs-utils installed with nfs4 support
- grep NFS_NEEDED_SERVICES /etc/conf.d/nfs shows 'NFS_NEEDED_SERVICES="rpc.idmapd"'
- grep Domain /etc/idmapd.conf shows 'Domain = <your local domain>'
- rpc.idmapd daemon is running (if it does not, restart nfs stack)
- surely portage uid/gid are the same on all nfs-ed machines

server side:
/etc/exports: /usr/portage   
192.168.1.0/24(async,no_root_squash,rw,no_subtree_check)

client side:
grep nfs /etc/fstab: server:/usr/portage /usr/portage nfs4 defaults,rw 0 1

consult rpc.idmapd(8) for details

that way i'm sharing portage at home. works pretty good for months i've migrated
to nfs4

hth

[gentoo-user] Re: [almost SOLVED] NFSv4: 32-bit server versus 64-bit client?

[walt]

On 08/05/2011 03:51 AM, victor romanchuk wrote:
> 
>> I'm trying to be a good gentoo netizen by nfs-sharing /usr/portage between
>> my three local gentoo machines, and failing :(
>>
>> After weeks of fiddling, I discovered today that my problems come from
>> using a 32-bit machine to serve my two 64-bit NFS clients(!)
>>
>> (I'll mention up front that NFSv3 works perfectly -- only NFSv4 is bad.)
>>
> 
> this is due to different authentication methods used in nfs3 and nfs4 and does
> not rely on installation arch (32/64bit). you have to tune up nfs4
> infrastructure. on both client and server make sure you have
> 
> - nfs4 and inotify support in kernel
> - net-fs/nfs-utils installed with nfs4 support
> - grep NFS_NEEDED_SERVICES /etc/conf.d/nfs shows 'NFS_NEEDED_SERVICES="rpc.idmapd"'
> - grep Domain /etc/idmapd.conf shows 'Domain = <your local domain>'

That was a good hint, thanks.  I finally figured out by trial and error that the
correct gentoo way to start idmapd is by starting /etc/init.d/nfsmount on the
client.  That fixed the bad uid/gid numbers I was seeing.

But, I still have a permissions problem I can't figure out.

The exported /usr/portage mounts okay on the client with rw privileges, but I still
get a "read-only filesystem" error when I try to write to it.

Again by trial and error I discovered that restarting the nfs server fixes the write
problem, but with a gotcha:  the first write to the mounted NFS filesystem hangs for
about a minute before it finally succeeds.  Everything works normally after that.

That first write process hangs in a D+ state, apparently waiting for something to
time out after a minute or so.

Any idea what could cause that?

Many thanks!